#metabrainz

/

      • Maxr1998
        I do wonder though, my listen exports after this incident produce much smaller files. The number of listens seems fine, but the individual listens lack the mbid mappings or have only the recording mbid mapped, compared to a dump from August where a lot more data was available. I don't really mind, but wanted to bring it up in case it's unintentional.
      • v6lur has quit
      • BrainzGit
        [musicbrainz-server] 14mwiencek opened pull request #3074 (03master…mbs-13347): MBS-13347: Relationship Type edit search times out https://github.com/metabrainz/musicbrainz-serve...
      • pbryan has quit
      • pbryan joined the channel
      • pbryan has quit
      • [musicbrainz-server] 14mwiencek opened pull request #3075 (03master…mbs-13309): MBS-13309: Restrict cross-origin requests to /ws/js/edit https://github.com/metabrainz/musicbrainz-serve...
      • opal has quit
      • opal joined the channel
      • relaxoMob has quit
      • lusciouslover has quit
      • lucifer
        Maxr1998: can you send me a copy of both the dumps (one recent and one old) if you still have it?
      • relaxoMob joined the channel
      • relaxoMob has quit
      • relaxoMob joined the channel
      • lusciouslover joined the channel
      • outsidecontext
        lucifer, mayhem: will the data under "Taste" come back (love, hated and pins) or is this gone?
      • lucifer
        outsidecontext: it should be there, let me check
      • outsidecontext
        love/hated is missing, pins are there but without track data
      • lucifer
        outsidecontext: the data is there for love/hate, the autogenerated mapped data is missing.
      • i'll run the command to regenerate the mapped data and hopefully it should all come back
      • i guess i was so occupied in recovering the listen data that i didn't bother to see if something else got deleted as well.
      • outsidecontext
        Great, thanks. Let's see how it looks afterwards
      • Another question is the many now unmapped listens. I assume these will recover with the automapper?
      • lucifer
        i think we have a dump for that too but not sure
      • relaxoMob
        Thank you for bringig back listens. And the timestamps. Fortunatly I have only some listens in that period which i can add manually.
      • outsidecontext: Art Generator does not work now. Is that related to the lost listens?
      • lucifer
        relaxoMob: looking into it
      • yup, relaxoMob.
      • should be fixed today i hope.
      • will take 6-8 hours for all the mapped data to regenerate which should fix most of the issues.
      • outsidecontext: we have a dump for that too, data will be back soon.
      • outsidecontext
        Great
      • lusciouslover has quit
      • relaxoMob
        lucifer: Thank you. No hurry. Just noticed it last night while I want do show it to a friend of mine.
      • mayhem
        moin!
      • lucifer: check the popular schema too -- this endpoint returns bad data: https://api.listenbrainz.org/1/popularity/top-r...
      • and thus LB radio is not working.
      • lucifer
        mayhem: afk rn but yeah i suppose all tables will have to be checked. Will do it when back
      • mayhem
        makes sense
      • LumePart joined the channel
      • LumePart
      • mayhem
        hi LumePart!
      • yes, there are some data sets that we still need to recreate to get everything working again.
      • hopefully we can get those done today.
      • LumePart
        Okay, thanks for the quick answer!
      • mayhem
        LumePart: actually, I just checked, that part is already running. should be fixed in an hour or two. 🤞
      • LumePart
        Oh, awesome!
      • petitminion joined the channel
      • kepstin has quit
      • kepstin joined the channel
      • kepstin has quit
      • Maxr1998
        lucifer: sure, still have those files. Any preference on how I should send them to you?
      • kepstin joined the channel
      • Maxr1998 has quit
      • Maxr1998 joined the channel
      • antlarr has quit
      • antlarr joined the channel
      • petitminion has quit
      • ApeKattQuest
        ack, i'm sick of being sick. wanna be healty and energetic so i cna work now pls
      • outsidecontext
        ApeKattQuest: I wish you that, get well
      • ApeKattQuest
        🙏 thanks
      • relaxoMob has quit
      • relaxoMob joined the channel
      • LumePart has quit
      • Maxr1998 has quit
      • Maxr1998 joined the channel
      • Maxr1998 has quit
      • Maxr1998 joined the channel
      • relaxoMob has quit
      • Pratha-Fish
        Get well soon ApeKattQuest :)
      • ApeKattQuest
        thanks fish
      • relaxoMob joined the channel
      • relaxoMob
        Is it technically possible to become an auto editor for only one entity? For me it would be nice to be auto editor for Events and Event Series.
      • Maxr1998 has quit
      • Maxr1998 joined the channel
      • rdswift
        I don't think so. I believe it's (almost) all or nothing.
      • The reason I say almost is because autoeditors still can't edit instruments or areas.
      • kellnerd
        Yeah, but you don't have to auto-edit as an auto-editor. There's always the checkbox to make the edits votable.
      • lusciouslover joined the channel
      • outsidecontext
        Are the imports for the LB mapping dumps still running?
      • kirtan joined the channel
      • kirtan has quit
      • mayhem
        outsidecontext: it failed and was re-run. 6 hours hopefully. 🤞
      • outsidecontext
        fingers crossed
      • relaxoMob
        Okay, then I maybe try to get nominated? If i am luckey, some auto-editors are here and someone wants to nominate me?
      • relaxoMob has quit
      • relaxoMob joined the channel
      • petitminion joined the channel
      • relaxoMob has quit
      • relaxoMob joined the channel
      • Pratha-Fish
        bitmap: do we have any option to update python on wolf to 3.11 or something?
      • Or maybe use something like miniconda instead of touching the default ubuntu python installation
      • bitmap
        you can run your code in a python 3.11 docker image
      • Pratha-Fish
        I'll also be connecting to databases and stuff from inside the docker image. Would that work?
      • Sorry I still ain't very good with docker yet
      • bitmap
        yeah let me see
      • lucifer
        mayhem, outsidecontext: still fails, will have to take a look again. its nothing to do with the data but seems like the server is killing the query for taking too long.
      • bitmap
        the database is also on wolf? do you know which port?
      • mayhem
        lucifer: anything I can help with?
      • Pratha-Fish
        bitmap: I am currently just using the existing MusicBrainz-server installation on wolf since I am only doing reading and no writing on wolf
      • bitmap
        ok, I'll find it
      • Pratha-Fish
        I'd love to setup another testing environment on wolf too, but let's keep that one for later to avoid complications
      • lucifer
        mayhem: i checked statement timeouts, docker logs of postgres containers and pgbouncer. nothing obvious as to what's killing the query.
      • just this error on the client side
      • Pratha-Fish
        Currently, I am just trying to run a few scripts to fetch data from wikidata and the musicbrainz_db setup on wolf. ig I'll just tweak the requirements.txt file a bit to add backwards compatibility with Python 3.10 for now
      • mayhem
        odd.
      • lucifer: that is on the MB prod server?
      • lucifer
        yes
      • mayhem
        what if we used wolf instead?
      • the data isn't 100% up to date, but quite close.
      • and its likely not busy.
      • mayhem looks sideways at Pratha-Fish
      • lucifer
        sure we can try that
      • mayhem
        want me to try?
      • lucifer
        yes sure. i'll see if i can debug the prod thing meanwhile
      • Pratha-Fish stares at mayhem and lucifer https://usercontent.irccloud-cdn.com/file/VPc4LacE/image.png
      • mayhem
        Pratha-Fish: are you planning to run something heavy on wolf?
      • lucifer
        those wikidata scripts should be pretty light so shouldn't be an issue.
      • mayhem
        ok, cool.
      • Pratha-Fish
        Nop. Just a fuck ton of network requests to fetch data from wikidata
      • lucifer: exactly 😁
      • mayhem
        ok, I'll run this on gaga.
      • lucifer
        gaga?
      • mayhem
        tunnel to wolf for MB data, but write data to gaga.
      • I could run on wolf and tunnel to gaga.
      • lucifer
        oh clever, makes sense.
      • mayhem
        k
      • bitmap
        Pratha-Fish: try docker exec -it snaek-python /bin/bash
      • lucifer
        run on gaga and tunnel to wolf, more secure that way.
      • mayhem
        yep, agreed.
      • I'll have you review my config.py before I kick it off.
      • lucifer
        sure
      • bitmap: if you are around for a bit more, can you point me to where to look for pgbouncer logs?
      • bitmap
        lucifer: for the main cluster?
      • lucifer
        yes
      • docker logs pgbouncer-master didn't turn up anything useful.
      • Pratha-Fish
        bitmap: That worked! But unfortunately the whole thing also requires access to MusicBrainz docker, etc. I am not sure how commands running inside the docker container would directly be able to interact with the rest of the environment outside the 3.11 container
      • bitmap
        lucifer: that's all pgbouncer logs AFAIK. there is probably a verbosity setting that isn't enabled
      • what sort of info are you looking for?
      • lucifer
        when are connections getting disconnected exactly
      • and if its pgbouncer that disconnecting connections or the postgres server
      • mayhem
        bitmap: if I want to connect to the LB database on jimmy/hendrix, which port do I connect to?
      • lucifer
        mayhem: you don't need to connect to that
      • mayhem
        I'll be coming from gaga -- is the PG port exposed locally?
      • bitmap
        the master pgbouncer port is 65436
      • lucifer
        you need to connect to LB database on gaga.
      • mayhem
        ok.
      • lucifer: let me blank out that connect string then.