bitmap: We'll have to replace a SSD on yamaoka, can you check if we can move services elsewhere (or afford to stop them a short time for the replacement)? I'll handle that tomorrow (that's late here, going to bed)
2025-08-06 21818, 2025
bitmap[m]
zas: I've temporarily started artwork-indexer prod and CB prod containers on selda. other non-redundant containers on yamaoka include CB beta, mbspotify, and jenkins, but downtime shouldn't be problematic for those (not 100% sure about mbspotify though)
2025-08-06 21822, 2025
bitmap[m]
apparently CB was already running on selda but wasn't added to scripts/nodes/selda.sh
2025-08-06 21810, 2025
DjSlash has quit
2025-08-06 21833, 2025
DjSlash joined the channel
2025-08-06 21856, 2025
d4rk-ph0enix has quit
2025-08-06 21818, 2025
d4rk-ph0enix joined the channel
2025-08-06 21842, 2025
d4rk-ph0enix has quit
2025-08-06 21828, 2025
d4rk-ph0enix joined the channel
2025-08-06 21856, 2025
d4rk-ph0enix has quit
2025-08-06 21823, 2025
d4rk-ph0enix joined the channel
2025-08-06 21842, 2025
d4rk-ph0enix has quit
2025-08-06 21800, 2025
d4rk-ph0enix joined the channel
2025-08-06 21804, 2025
lucifer[m]
bitmap: i was looking at all yesterday to figure out which ones were running CB and i didn't have selda in my local script so skipped it. thanks!
2025-08-06 21835, 2025
lucifer[m]
reosarevok: CB issues should now be fixed, yamaoka and selda were both running CB but on different versions and that caused all the reported issues.
2025-08-06 21812, 2025
lucifer[m]
i have stopped all CB containers on yamaoka.
2025-08-06 21807, 2025
yvanzo[m]
<aerozol[m]> "yvanzo: I think you might be..." <- aerozol: I clearly wrote that “it is open to suggestions and the script’s code is organized in a way that makes it easy to play with colors”. There is no need to split people into made-up opposite camps, thank you. Please make actual suggestions for the color scheme instead: Pick one image such the cover art archive table structure, change the colors, and we will see.
2025-08-06 21833, 2025
aerozol[m]
I wasn't splitting anyone up... I just read a different part of a very long comment that you linked to.
2025-08-06 21821, 2025
aerozol[m]
My advice is to remove the background color and leave it plain. I will take the time to change it if it isn't going to be changed back
For everyone else, I continued this conversation in private with aerozol to go beyond unjustified assumptions before having to go. To be continued soon…
yamaoka disk replacement should occur within 2-4 hours
2025-08-06 21849, 2025
petitminion joined the channel
2025-08-06 21827, 2025
zas[m]
yamaoka disk was replaced, RAID array is rebuilding (ETA ~146 minutes), it is back to work
2025-08-06 21807, 2025
lucifer[m]
mayhem: troi ratelimiting is broken because multiple concurrent requests to explore/lb-radio invoke troi in separate threads and the global dict to manage ratelimits is concurrently modified.
the del operation fails because key is already removed from the dict.
2025-08-06 21845, 2025
lucifer[m]
by another thread/request.
2025-08-06 21833, 2025
lucifer[m]
there are two ways to fix this imo, either add a lock and only make one global request for troi at a time. but that will hurt lb-radio performance significantly.
2025-08-06 21800, 2025
lucifer[m]
make the dict storing the ratelimits a thead local. each thread/request gets its own dict.
2025-08-06 21847, 2025
lucifer[m]
there is a third option to make the dict accesses safer, use get/pop with defaults but might defeat the purpose of ratelimit logic.
2025-08-06 21819, 2025
lucifer[m]
hmm, no actually that might work fine too. we just need to decide whether we want global ratelimits on a best effort basis or per-thread ratelimits.
