#metabrainz

/

      • D4RK-PH0ENiX has quit
      • D4RK-PH0ENiX joined the channel
      • Nyanko-sensei joined the channel
      • D4RK-PH0ENiX has quit
      • Nyanko-sensei has quit
      • D4RK-PH0ENiX joined the channel
      • yokel has quit
      • yokel joined the channel
      • rdswift_lt has quit
      • c1e0 joined the channel
      • c1e0 has quit
      • Lotheric has quit
      • Lotheric joined the channel
      • Lotheric has quit
      • Leo_Verto_ joined the channel
      • Lotheric joined the channel
      • Leo_Verto has quit
      • Leo_Verto_ is now known as Leo_Verto
      • G0re joined the channel
      • Gore|woerk has quit
      • michelv joined the channel
      • c1e0 joined the channel
      • outsidecontext joined the channel
      • iliekcomputers
        ruaok: any idea how I look at datanode logs on rec-master.listenbrainz.org
      • CatQuest
        \o iliekcomputers
      • iliekcomputers
        Hi CatQuest =)
      • github joined the channel
      • github
        [picard-plugins] mineo opened pull request #192: cuesheet: Bump the version number (2.0...cuesheet-version) https://git.io/fpAnu
      • github has left the channel
      • Freso
        thefar8[m]: Great! Thanks! I'll give it a look. :)
      • ruaok
        mooooin!
      • iliekcomputers: let me look in a second.
      • Mineo, reosarevok: the wikidata bot seems much much better now.
      • reosarevok
        It does! Artists, woo!
      • ruaok
        fixing redirects, adding claims.
      • and it is moving at a good clip too.
      • do you have access to the servers, yet, reosarevok ?
      • reosarevok
        I do
      • ruaok
        on zappa: docker logs -f wikidata-bot
      • if you want to follow the logs yourself.
      • reosarevok
        Ok :)
      • Wow, ok, that does look quite fast :D
      • Sweet!
      • github joined the channel
      • github
        [listenbrainz-server] vansika opened pull request #470: Introduced 'Remember me' functionality (master...master) https://git.io/fpAB7
      • github has left the channel
      • iliekcomputers
        hadoop configuration is definitely not fun.
      • zas
        samj1912: i deployed new solr version, the cluster looks ok, i'll wait few hours be concluding anything but it seems it doesn't disk read as much as previous version and response times are a tad better
      • ruaok
        iliekcomputers: in theory we should be able to see the logs from the workers using something like this:
      • docker service logs yarn-workers
      • but right now that says that the worker nodes are not available.
      • ruaok shrugs
      • iliekcomputers
        ruaok: the rpc write here (https://github.com/metabrainz/listenbrainz-reco...) doesn't work, for some reason. same error we were getting.
      • i've been mucking about with config values trying to fix, but no cigar.
      • maybe it's because the worker nodes aren't available, but HTTP write works...
      • ruaok
        iliekcomputers: is this a docker or a hadoop issue?
      • ruaok goes back to unicorn hunting
      • iliekcomputers
        I was thinking hadoop
      • But after seeing the logs problem, I'm not so sure
      • ruaok
        lets examine the docker level first.
      • iliekcomputers
        Man, Unicorn hunter is a nice job
      • ruaok
        have you stopped/restarted the workers?
      • iliekcomputers
        Yes, many times
      • I have some college stuff, be back in some time.
      • ruaok
        k, I'll check the docker level.
      • alastairp
        iliekcomputers: we got this new PR from vansika on LB about Flask Remember Me - https://github.com/metabrainz/listenbrainz-serv..., LB-398
      • BrainzBot
        LB-398: Use flask-login "remember me" functionality https://tickets.metabrainz.org/browse/LB-398
      • alastairp
        do you want to work through this, or do you want me to give a hand with it (I note that it doesn't cover any of the concerns that I wrote in the ticket, so I think it'll need quite a bit of discussion/development still)
      • reosarevok
        Oh god. Trying to properly document an event, it sucks so much. Why did you let me write this.
      • bitmap: while converting the release editor, can you make sure as much as possible of the stuff is reusable? (if it isn't already). We really need per-performed-work proper data at some point :/
      • iliekcomputers
        alastairp: be really helpful if you review =)
      • alastairp
        ok, I've assigned myself, I'll try and write some notes on it this week :)
      • do we know this contributor? I see it's their first PR. GCI student? potential SoC? Someone you know?
      • iliekcomputers
        Yeah, good friend of mine, potential summer of Code
      • :)
      • alastairp
        ah! great :)
      • ruaok
        good to know. I had the same sort of reaction. :)
      • iliekcomputers
        Well, soc is a long way away still, let's see.
      • Slurpee has quit
      • Slurpee joined the channel
      • c1e0 has quit
      • D4RK-PH0ENiX has quit
      • george
        thefar8[m]: That report is sweet
      • D4RK-PH0ENiX joined the channel
      • slurpee- joined the channel
      • c1e0 joined the channel
      • Slurpee has quit
      • thefar8[m]
        Okay Thank you Freso (sorry for the late response, got a lot things to do today)
      • george: Thanks !
      • bitmap
        reosarevok: which parts of the release editor would apply to works?
      • reosarevok
        I was thinking the UI bits would be fairly relevant (since "artist, work" seems to be reasonably close to "artist, track name") - only that I guess with an option to find the works in MB rather than just a string name
      • Freso
        thefar8[m]: Well, my response was even later. :)
      • reosarevok
        But we would need to think more specifically about how to store the info in a way that can be reused and queried, because that's the worst part about the current setlists
      • You can't really get "this song was played on these concerts" at all
      • (we could just use normal relationships for that, maybe ordered, but then that would just duplicate half the event setlist info)
      • bitmap
        right, missed the previous line where you mentioned events :)
      • TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | GSoC https://goo.gl/7jsjG2 | Meeting agenda (17th is last meeting of 2018!): Reviews, Google Code-in (Freso), mini summit plans (iliekcomputers)
      • TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | GSoC https://goo.gl/7jsjG2 | Meeting agenda (17th is last meeting of 2018!): Reviews, Google Code-in (Freso), mini summit plans (iliekcomputers), Holidays/next meeting (Freso)
      • reosarevok
        bitmap: do you have any good ideas about storing setlists in a way that allows for them to be integrated properly, like the rest of MB is?
      • Without adding extra entities, that is :D adding them it's easier
      • bitmap
        add a setlist table? or is that cheating
      • all it needs is (event, work, artist_credit, name) right?
      • , position
      • if we had n-ary relationships we might be able to abuse those instead, but
      • c1e0 has quit
      • c1e0 joined the channel
      • yvanzo
        reosarevok: an idea mentionned earlier is to handle setlist similarly to tracklist
      • slurpee- has quit
      • c1e0 has quit
      • reosarevok
        Ideally we'd be able to also add relationships (orchestra, conductor, etc) to each entry, but I guess that'd be a good start that could be made into something deeper more easily later too if desired
      • Also would need to allow for string-only comments and whatnot for the equivalent of the current # but that's probably not too hard either
      • iliekcomputers
        ruaok: `docker service logs yarn-workers -f` works 😅
      • probably a bug in docker...
      • ruaok
        wait, what?
      • -f in the wrong place?
      • sorry, that was going to be the next thing on my list. still chasing people to pay us.
      • iliekcomputers
        yeah, no worries =)
      • yvanzo
        reosarevok: by "string-only comments", you mean annotation :)
      • ruaok
        <close tab> <close tab> <close accounting software> <smile> <smile>
      • we crossed the $500,000 income line sometime mid december.
      • iliekcomputers
        i don't think the placement of the `-f` matters, `docker service logs yarn-workers` doesn't work, adding a -f does.
      • >we crossed the $500,000 income line sometime mid december.
      • ruaok
        now back to real business!
      • iliekcomputers
        noiiice
      • ruaok
        that does sound like a docker bug
      • CatQuest
        [14:48] <reosarevok> I was thinking the UI bits would be fairly relevant (since "artist, work" seems to be reasonably close to "artist, track name") - only that I guess with an option to find the works in MB rather than just a string name
      • I'd except that the *relationship* editor would make more sence for work?
      • i mena i'd love to see an extention of it basiclaly with a separate column with all the stuff for works as well (but for every work thing o an artist)
      • well i guess for a release
      • thefar8[m]
        Freso: :) I've sent my response on that
      • ruaok
        iliekcomputers: so, need anything from me now?
      • iliekcomputers
        no, i'll go back to seeing why the dataframes aren't being written.
      • CatQuest
        oh this is for setlists
      • ruaok
        let me know if you want a second set of eyes, iliekcomputers
      • iliekcomputers
        ok.
      • CatQuest
        yea probably liek a tracklsit would work then
      • iliekcomputers
        so remember we talked about how there's a RPC interface and an HTTP interface for HDFS
      • well, pyspark uses the RPC interface for stuff like this (https://github.com/metabrainz/listenbrainz-reco...)
      • CatQuest
        but a setlist doesnt have to be a list of recordings (thy might not be recorded or even if not in the database (but be added) or it *can* be recordings)
      • iliekcomputers
        and the RPC interface works from inside the hadoop-master container.
      • ruaok
        but not outside.
      • iliekcomputers
        but not from inside other containers for some reason.
      • yeah.
      • CatQuest
        or sohuld they always be as recordings becasue they *existed* see like radio-brodcatsts
      • ruaok
        do you know what port numbers need to be open for this?
      • iliekcomputers
        let me check.
      • ruaok: 9864, 9866, 9867
      • ruaok
        ok.
      • iliekcomputers
        dfs.datanode.address, dfs.datanode.http.address, dfs.datanode.ipc.address