i think both old and new popularity datasets have standalone recs but in general they are few in MB and plus randomness in selection makes the problem not as frequent.
the number of backends connected shot up a while ago.
dynu joined the channel
yeah, looks like a DB issue.
bitmap
yeah
mayhem
anything I can do to help?
dynu
Why i get a 502 error, when i try to login with my ListenBrainz ????
mayhem
dynu: MusicBrainz is down right now, and that is an integral part of logging in.
sit tight, we're looking at the problem
dynu
arhhh thanks alot :-)
bitmap
mayhem: I'm seeing a lot of errors in mb website logs from YandexBot crawling instrument recording listings, like /instrument/b3eac5f9-7859-4416-ac39-7154e2e8d348/recordings?page=3341
mayhem
zas: you still up?
bitmap
those large OFFSET queries are ddosing the website with so many being requested at once
mayhem
oh fun.
dynu has quit
ok, assuming zas and atj are not about, (just turned midnight here) how do we mitigate this?
503 that endpoint and deploy new front ends?
oy. load 69 on floyd.
bitmap
yeah, I could have MBS return 503 for pages past 100 or so
i'll work on it
mayhem
yeah, seems like a decent approach for the moment.
derwin joined the channel
derwin
hiya, I presume someone is aware mb site is down?
or at least malfunctioning significantly?
mayhem
hey, yes.
we think a yandex bot is doing something dumb. bitmap is looking at a stop-gap measure
bitmap
deployed the mitigation but we might have to 503 the whole endpoint
mayhem
still see a lot of log entries?
bitmap
yeah
mayhem
yeah, do it to rule it out.
that did the trick!
bitmap
that's helping more
mayhem
thanks bitmap
!m bitmap
BrainzBot
You're doing good work, bitmap!
mayhem
ok, finally off to bed then. nn!
bitmap
still getting flooded with requests from them but these were the heaviest ones I think
aerozol
Mōrena!
bitmap
good night!
aerozol
:P
yvanzo: I’m still keen! I let setting up the local stuff slide because it was agreed a while ago that I would do more mockups than actual pulls etc
But I’m keen to learn
Time zone wise it looks like my evening/your morning might be best? I can do your 9am today, tomorrow, and a few days next week