#metabrainz

zas: there's not much info in pg logs other than that checkpoints were occurring too frequently (usually indicating heavy writes)

however the load averages on wolf and the spark cluster nodes were higher than usual during the same time frame, so perhaps something heavy was being processed there

[docker-python] 14mwiencek opened pull request #19 (03master…update-3.11): [WIP] Update 3.11 image https://github.com/metabrainz/docker-python/pul...

bitmap, reosarevok: Updating all MB background task containers for reports and admin/run.

ok

Actually subscriptions are still being processed, will do this afternoon.

bitmap: there was something running on gaga matching time floyd was under load

ah

mbid mapper I guess?

this RO DB server could be a really good idea

around 16UTC when floyd WALs started to accumulate

ironically that is the MB metadata cache, designed to keep the load on MB low. 🤭

mayhem: we write data to a few tables too so we need write access as well.

btw, the same thing happened on 7 sep, at same time

yes, we can write back to the RW server.

zas: its a cron job, so yes.

look at WAL graphs at the bottom

what peaks? I dont see any giant GB large peaks. do you?

😒

lucifer: this is something for us to discuss in person. but I think we should make more use of couchdb.

actually I think it starts a bit before 16UTC, perhaps that's the previous cron job starting the issue

Oct 5 15:00:01 gaga CRON[6412]: (listenbrainz) CMD ([ $(date +%d) -le 7 ] && /usr/local/bin/python /code/mapper/manage.py cron-build-mb-metadata-cache >> /code/mapper/cron-mb-metadata-cache.log 2>&1)

mayhem: we can do that yes but we won't be able to join that with any of our other data then.

i guess it would be fine to store the mb_metadata_cache but it wouldn't lower the load on MB db because all the data is still read from there.

imo we need to full proof the incremental updates so that we can totally get rid of the bulk generate from scratch.

can you use unlogged tables? that would solve the wal accumulation issues. (you won't be able to query the tables from a standby though, you'll have to use the master)

also Jackson5 were clearly active during the time floyd was under load, so this has to be checked too

bitmap: that should be possible i guess. will need to check why so much wal accumulated though. we don't write back much, just read a lot.

lucifer: yes, to all that. the inability to do joins limits what we can do, but I think there are many datasets that we envision hosting with DSH-on-spark could actually be handled by couchdb. mb metadata may not be the best example. but something like last_listened_at for all users could.

mayhem: yes that makes sense

and on making the cached data incremental updates work -- we'll have a chance to look at that real soon. joy.

bitmap: https://stats.metabrainz.org/d/000000067/postgr... look at db_inserted

yes I was just checking that

oh, yes! Everyone, please bring your summit shirt today! we need to make a group photo

lucifer: the graph zas linked shows 9k insert ops/s, does that seem wrong?

(that is based on tup_inserted so should mean 9k rows)

bitmap: did some rough calculations, 9k ops/s for ~8 mins ~ 4.5M inserts is possible (actual number is probably closer to 3M) but yes.

I'm curious about the drop of db_inserted during half an hour

lucifer: are the tables being inserted into all in the mapping schema?

bitmap: yes

bitmap: are you aware of Aperture ? https://github.com/fluxninja/aperture https://blog.fluxninja.com/blog/protecting-post...

I am not

will check it out

mayhem: I cannot find mine D: did you find a random L shirt laying about in the office any of these days? 😅

(I will be at the office in 30)

I mean, if that gets me a t-shirt again, I'll take the L

outsidecontext: error handling has been fixed now on the DSH endpoint.

mayhem: will check in a minute

now to actually work on why some bits dont return data.

yvanzo: ping

curl -X POST -k -H 'Content-Type: application/json' -i 'https://labs.api.listenbrainz.org/mbid-mapping-release/json' --data '[{"[artist_credit_name]": "Paradise Lost", "[recording_name]": "Paradise Lost", "[release_name]": "Drown in Darkness \\u2013 The Early Demos"}, {"[artist_credit_name]": "Paradise Lost", "[recording_name]": "Paradise Lost (live)", "[release_name]": "Drown in Darkness \\u2013 The Early Demos"},

{"[artist_credit_name]": "Paradise Lost", "[recording_name]": "Paradise Lost", "[release_name]": "Lost Paradise"}, {"[artist_credit_name]": "Opened Paradise", "[recording_name]": "Opened Paradise", "[release_name]": "Occult"}]'

amazing https://www.404media.co/bing-is-generating-imag...

mayhem: things start work well now, but the above curl returns empty. in general this seems to happen whenever track title matches artist name

outsidecontext: can you paste that as a pastebin?

or something that doesn't break the url...

yes, sorry. actually that was the intention :) one sec

monkey: ^^ I get this error when I try to start lb-server on the moar-cached-data branch

mayhem: https://gist.github.com/phw/92fef67e4ef2958f301...

thx

OK, let me push

lucifer: you about?

yes

see the query that outsidecontext posted above. that is against the mapping with releases endpoint.

can you plz take a look at that if you have a moment?

mayhem: pushed, should be fixed now

lucifer: all the requested tracks are supposed to yield a result as there is a recording for it. but recording name equals band name. I can reliably reproduce the issue with this endpoint

mayhem, outsidecontext: sure looking into it

thx monkey

thx lucifer

what branch is it running on?

thats on labs, no? should be current prod more or less.

cool got it

monkey: different error now

did I forget to run something? I did a develop.sh build

mayhem: looking at the explain-mbid-mapping output, this seems to be like a typesense thing

Reminder to everyone to please put any important discussions/to-dos/decisions into the summit meeting notes: https://docs.google.com/document/d/1bz32duq95jS...

Both to keep us on track/as a handy reminder, and to keep the community and those who couldn’t make it in the loop ➰

mayhem: Looks like the error is related to the manifest file we write to disk (so that flask knows which javascript file to load). I don't think we changed anything, so that's surprising

[musicbrainz-server] 14akshaaatt closed pull request #2507 (03akshat/intro-new…akshat/login): Login Section https://github.com/metabrainz/musicbrainz-serve...

ok, how do we get oast that monkey?

past, even

I'm looking into it, but I currently have no idea. Maybe comment out this line?

yeah, that built for me

go two warnings, but looks a lot better.

css esta borked, however.

It's Taco'clock ! https://tacoalto.es/carta

monkey: which was the taco that did no longer eixst?

monkey: Quesadilla de Pibil , Taco Baja, Taco Tinga Tinga, Taco Gringa, Taco Sinaloa

exist too

mayhem: pull and try again, I had an error in my less file

reosarevok: Quesadilla de Nopales

No, they had that one

*big

They didn't have the Chihuahua taco, and the Quesadilla de Jamón con Champiñones was actually Tinga con Champiñones instead

[listenbrainz-android] 14akshaaatt opened pull request #265 (03main…dev): Dev to Main https://github.com/metabrainz/listenbrainz-andr...

[listenbrainz-android] 14akshaaatt merged pull request #265 (03main…dev): Dev to Main https://github.com/metabrainz/listenbrainz-andr...

Here's the up-to-date menu: https://usercontent.irccloud-cdn.com/file/nbDt6...

mayhem: no gringa taco, qyuesadilla gringa instead?

taco pirata instread, plz

bitmap: MBS-12969 has the reproduceable bug case

MBS-12969: Recording relationships are not shown as deleted in the "Related Works" column https://tickets.metabrainz.org/browse/MBS-12969

thx

monkey: pushed the change to fetch artist data and emit it into props.

Weeeeeee

scrollback is long.

Hop ^

thx

I guess ignore parts of the data structure at the bottom, as we discussed

lucifer: no pings from you, so I assume you have enough to work with? Keep me updated!

lucifer: the autotag that outsidecontext is working on, is working quite well already. quite exciting, really.

!m outsidecontext

You're doing good work, outsidecontext!

!m lucifer