#metabrainz

I do wonder though, my listen exports after this incident produce much smaller files. The number of listens seems fine, but the individual listens lack the mbid mappings or have only the recording mbid mapped, compared to a dump from August where a lot more data was available. I don't really mind, but wanted to bring it up in case it's unintentional.

[musicbrainz-server] 14mwiencek opened pull request #3074 (03master…mbs-13347): MBS-13347: Relationship Type edit search times out https://github.com/metabrainz/musicbrainz-serve...

[musicbrainz-server] 14mwiencek opened pull request #3075 (03master…mbs-13309): MBS-13309: Restrict cross-origin requests to /ws/js/edit https://github.com/metabrainz/musicbrainz-serve...

Maxr1998: can you send me a copy of both the dumps (one recent and one old) if you still have it?

lucifer, mayhem: will the data under "Taste" come back (love, hated and pins) or is this gone?

outsidecontext: it should be there, let me check

love/hated is missing, pins are there but without track data

outsidecontext: the data is there for love/hate, the autogenerated mapped data is missing.

i guess i was so occupied in recovering the listen data that i didn't bother to see if something else got deleted as well.

Great, thanks. Let's see how it looks afterwards

Another question is the many now unmapped listens. I assume these will recover with the automapper?

i think we have a dump for that too but not sure

Thank you for bringig back listens. And the timestamps. Fortunatly I have only some listens in that period which i can add manually.

outsidecontext: Art Generator does not work now. Is that related to the lost listens?

relaxoMob: looking into it

yup, relaxoMob.

should be fixed today i hope.

will take 6-8 hours for all the mapped data to regenerate which should fix most of the issues.

outsidecontext: we have a dump for that too, data will be back soon.

Great

lucifer: Thank you. No hurry. Just noticed it last night while I want do show it to a friend of mine.

moin!

lucifer: check the popular schema too -- this endpoint returns bad data: https://api.listenbrainz.org/1/popularity/top-r...

and thus LB radio is not working.

mayhem: afk rn but yeah i suppose all tables will have to be checked. Will do it when back

makes sense

hi LumePart!

yes, there are some data sets that we still need to recreate to get everything working again.

hopefully we can get those done today.

Okay, thanks for the quick answer!

LumePart: actually, I just checked, that part is already running. should be fixed in an hour or two. 🤞

Oh, awesome!

lucifer: sure, still have those files. Any preference on how I should send them to you?

ack, i'm sick of being sick. wanna be healty and energetic so i cna work now pls

ApeKattQuest: I wish you that, get well

🙏 thanks

Get well soon ApeKattQuest :)

thanks fish

Is it technically possible to become an auto editor for only one entity? For me it would be nice to be auto editor for Events and Event Series.

I don't think so. I believe it's (almost) all or nothing.

The reason I say almost is because autoeditors still can't edit instruments or areas.

Yeah, but you don't have to auto-edit as an auto-editor. There's always the checkbox to make the edits votable.

Are the imports for the LB mapping dumps still running?

outsidecontext: it failed and was re-run. 6 hours hopefully. 🤞

fingers crossed

Okay, then I maybe try to get nominated? If i am luckey, some auto-editors are here and someone wants to nominate me?

bitmap: do we have any option to update python on wolf to 3.11 or something?

Or maybe use something like miniconda instead of touching the default ubuntu python installation

you can run your code in a python 3.11 docker image

I'll also be connecting to databases and stuff from inside the docker image. Would that work?

Sorry I still ain't very good with docker yet

yeah let me see

mayhem, outsidecontext: still fails, will have to take a look again. its nothing to do with the data but seems like the server is killing the query for taking too long.

the database is also on wolf? do you know which port?

lucifer: anything I can help with?

bitmap: I am currently just using the existing MusicBrainz-server installation on wolf since I am only doing reading and no writing on wolf

ok, I'll find it

I'd love to setup another testing environment on wolf too, but let's keep that one for later to avoid complications

mayhem: i checked statement timeouts, docker logs of postgres containers and pgbouncer. nothing obvious as to what's killing the query.

just this error on the client side

Currently, I am just trying to run a few scripts to fetch data from wikidata and the musicbrainz_db setup on wolf. ig I'll just tweak the requirements.txt file a bit to add backwards compatibility with Python 3.10 for now

odd.

lucifer: that is on the MB prod server?

yes

what if we used wolf instead?

the data isn't 100% up to date, but quite close.

and its likely not busy.

sure we can try that

want me to try?

yes sure. i'll see if i can debug the prod thing meanwhile

Pratha-Fish: are you planning to run something heavy on wolf?

those wikidata scripts should be pretty light so shouldn't be an issue.

ok, cool.

Nop. Just a fuck ton of network requests to fetch data from wikidata

lucifer: exactly 😁

ok, I'll run this on gaga.

gaga?

tunnel to wolf for MB data, but write data to gaga.

I could run on wolf and tunnel to gaga.

oh clever, makes sense.

k

Pratha-Fish: try docker exec -it snaek-python /bin/bash

run on gaga and tunnel to wolf, more secure that way.

yep, agreed.

I'll have you review my config.py before I kick it off.

sure

bitmap: if you are around for a bit more, can you point me to where to look for pgbouncer logs?

lucifer: for the main cluster?

yes

docker logs pgbouncer-master didn't turn up anything useful.

bitmap: That worked! But unfortunately the whole thing also requires access to MusicBrainz docker, etc. I am not sure how commands running inside the docker container would directly be able to interact with the rest of the environment outside the 3.11 container

lucifer: that's all pgbouncer logs AFAIK. there is probably a verbosity setting that isn't enabled

what sort of info are you looking for?

when are connections getting disconnected exactly

and if its pgbouncer that disconnecting connections or the postgres server

bitmap: if I want to connect to the LB database on jimmy/hendrix, which port do I connect to?

mayhem: you don't need to connect to that

I'll be coming from gaga -- is the PG port exposed locally?

the master pgbouncer port is 65436

you need to connect to LB database on gaga.

ok.

lucifer: let me blank out that connect string then.