#metabrainz

/

      • d4rkie has quit
      • 2025-08-06 21822, 2025

      • d4rkie joined the channel
      • 2025-08-06 21859, 2025

      • wargreen has quit
      • 2025-08-06 21851, 2025

      • wargreen joined the channel
      • 2025-08-06 21823, 2025

      • zas[m]
        bitmap: We'll have to replace a SSD on yamaoka, can you check if we can move services elsewhere (or afford to stop them a short time for the replacement)? I'll handle that tomorrow (that's late here, going to bed)
      • 2025-08-06 21818, 2025

      • bitmap[m]
        zas: I've temporarily started artwork-indexer prod and CB prod containers on selda. other non-redundant containers on yamaoka include CB beta, mbspotify, and jenkins, but downtime shouldn't be problematic for those (not 100% sure about mbspotify though)
      • 2025-08-06 21822, 2025

      • bitmap[m]
        apparently CB was already running on selda but wasn't added to scripts/nodes/selda.sh
      • 2025-08-06 21810, 2025

      • DjSlash has quit
      • 2025-08-06 21833, 2025

      • DjSlash joined the channel
      • 2025-08-06 21856, 2025

      • d4rk-ph0enix has quit
      • 2025-08-06 21818, 2025

      • d4rk-ph0enix joined the channel
      • 2025-08-06 21842, 2025

      • d4rk-ph0enix has quit
      • 2025-08-06 21828, 2025

      • d4rk-ph0enix joined the channel
      • 2025-08-06 21856, 2025

      • d4rk-ph0enix has quit
      • 2025-08-06 21823, 2025

      • d4rk-ph0enix joined the channel
      • 2025-08-06 21842, 2025

      • d4rk-ph0enix has quit
      • 2025-08-06 21800, 2025

      • d4rk-ph0enix joined the channel
      • 2025-08-06 21804, 2025

      • lucifer[m]
        bitmap: i was looking at all yesterday to figure out which ones were running CB and i didn't have selda in my local script so skipped it. thanks!
      • 2025-08-06 21835, 2025

      • lucifer[m]
        reosarevok: CB issues should now be fixed, yamaoka and selda were both running CB but on different versions and that caused all the reported issues.
      • 2025-08-06 21812, 2025

      • lucifer[m]
        i have stopped all CB containers on yamaoka.
      • 2025-08-06 21807, 2025

      • yvanzo[m]
        <aerozol[m]> "yvanzo: I think you might be..." <- aerozol: I clearly wrote that “it is open to suggestions and the script’s code is organized in a way that makes it easy to play with colors”. There is no need to split people into made-up opposite camps, thank you. Please make actual suggestions for the color scheme instead: Pick one image such the cover art archive table structure, change the colors, and we will see.
      • 2025-08-06 21833, 2025

      • aerozol[m]
        I wasn't splitting anyone up... I just read a different part of a very long comment that you linked to.
      • 2025-08-06 21821, 2025

      • aerozol[m]
        My advice is to remove the background color and leave it plain. I will take the time to change it if it isn't going to be changed back
      • 2025-08-06 21835, 2025

      • __BrainzGit
        [troi-recommendation-playground] 14amCap1712 merged pull request #174 (03main…fix-import): misc small fixes https://github.com/metabrainz/troi-recommendation…
      • 2025-08-06 21808, 2025

      • __BrainzGit
        [troi-recommendation-playground] release 03v2025.08.06.1 has been published by 14amCap1712: https://github.com/metabrainz/troi-recommendation…
      • 2025-08-06 21823, 2025

      • yvanzo[m]
        For everyone else, I continued this conversation in private with aerozol to go beyond unjustified assumptions before having to go. To be continued soon…
      • 2025-08-06 21825, 2025

      • __BrainzGit
        [listenbrainz-server] 14amCap1712 opened pull request #3335 (03master…update-troi): Update troi to 2025.8.6.1 https://github.com/metabrainz/listenbrainz-server…
      • 2025-08-06 21806, 2025

      • d4rkie has quit
      • 2025-08-06 21833, 2025

      • d4rkie joined the channel
      • 2025-08-06 21859, 2025

      • SigHunter has quit
      • 2025-08-06 21855, 2025

      • SigHunter joined the channel
      • 2025-08-06 21838, 2025

      • SigHunter has quit
      • 2025-08-06 21825, 2025

      • SigHunter joined the channel
      • 2025-08-06 21823, 2025

      • __BrainzGit
        [listenbrainz-server] 14amCap1712 merged pull request #3335 (03master…update-troi): Update troi to 2025.8.6.1 https://github.com/metabrainz/listenbrainz-server…
      • 2025-08-06 21811, 2025

      • __BrainzGit
        [listenbrainz-server] release 03v-2025-08-06.0 has been published by 14github-actions[bot]: https://github.com/metabrainz/listenbrainz-server…
      • 2025-08-06 21838, 2025

      • fettuccinae[m] has quit
      • 2025-08-06 21804, 2025

      • zas[m]
        yamaoka disk replacement should occur within 2-4 hours
      • 2025-08-06 21849, 2025

      • petitminion joined the channel
      • 2025-08-06 21827, 2025

      • zas[m]
        yamaoka disk was replaced, RAID array is rebuilding (ETA ~146 minutes), it is back to work
      • 2025-08-06 21807, 2025

      • lucifer[m]
        mayhem: troi ratelimiting is broken because multiple concurrent requests to explore/lb-radio invoke troi in separate threads and the global dict to manage ratelimits is concurrently modified.
      • 2025-08-06 21804, 2025

      • lucifer[m]
        the ratelimiting broken is not the issue this time but this: https://github.com/metabrainz/troi-recommendation…
      • 2025-08-06 21837, 2025

      • lucifer[m]
        the del operation fails because key is already removed from the dict.
      • 2025-08-06 21845, 2025

      • lucifer[m]
        by another thread/request.
      • 2025-08-06 21833, 2025

      • lucifer[m]
        there are two ways to fix this imo, either add a lock and only make one global request for troi at a time. but that will hurt lb-radio performance significantly.
      • 2025-08-06 21800, 2025

      • lucifer[m]
        make the dict storing the ratelimits a thead local. each thread/request gets its own dict.
      • 2025-08-06 21847, 2025

      • lucifer[m]
        there is a third option to make the dict accesses safer, use get/pop with defaults but might defeat the purpose of ratelimit logic.
      • 2025-08-06 21819, 2025

      • lucifer[m]
        hmm, no actually that might work fine too. we just need to decide whether we want global ratelimits on a best effort basis or per-thread ratelimits.
      • 2025-08-06 21812, 2025

      • mayhem[m] reads
      • 2025-08-06 21815, 2025

      • mayhem[m]
        the del that fails, can we just ignore the failure. (likely not that simple)
      • 2025-08-06 21803, 2025

      • mayhem[m]
        <lucifer[m]> "mayhem: troi ratelimiting is..." <- let me clarify -- this is separate *processes* not requests, right?
      • 2025-08-06 21818, 2025

      • mayhem[m]
        threads
      • 2025-08-06 21826, 2025

      • lucifer[m]
        I'll have to check uwsgi configuration to see whether it's doing threads or processes, likely a combination of both
      • 2025-08-06 21830, 2025

      • mayhem[m]
        not processes. still waking up.
      • 2025-08-06 21849, 2025

      • mayhem[m]
        troi doesn't use threads, let me check.
      • 2025-08-06 21858, 2025

      • lucifer[m]
        No troi doesn't but uwsgi might be
      • 2025-08-06 21807, 2025

      • mayhem[m]
        it doesn't.
      • 2025-08-06 21812, 2025

      • lucifer[m]
        The issues was seen in lb prod api logs.
      • 2025-08-06 21821, 2025

      • mayhem[m]
        its specifically multi-process, but the processes themselves may use threads.
      • 2025-08-06 21833, 2025

      • mayhem[m]
        (on uwsgi)
      • 2025-08-06 21813, 2025

      • lucifer[m]
      • 2025-08-06 21816, 2025

      • mayhem[m]
        so, this double manipulation of the dict is rather curious.
      • 2025-08-06 21830, 2025

      • lucifer[m]
        We have enabled thread use from what I understand
      • 2025-08-06 21839, 2025

      • lucifer[m]
        So multiple threads per uwsgi process
      • 2025-08-06 21849, 2025

      • mayhem[m]
        lucifer[m]: yes, this means threads for child processes are enabled. I believe we can take that option away, since we're not usingit.
      • 2025-08-06 21815, 2025

      • lucifer[m]
        I think we are.
      • 2025-08-06 21825, 2025

      • lucifer[m]
        I'll check uwsgi docs again later to confirm
      • 2025-08-06 21803, 2025

      • mayhem[m]
        from gemini: "Enabling threads in uWSGI doesn't change it into a "threading server" in the same way a single-process, multi-threaded application works. Instead, it enables threading within each of its worker processes. The primary concurrency model of uWSGI remains multiprocess."
      • 2025-08-06 21816, 2025

      • lucifer[m]
      • 2025-08-06 21818, 2025

      • mayhem[m]
        as I expected.
      • 2025-08-06 21837, 2025

      • mayhem[m]
        the stack overflow anwswer is shit.
      • 2025-08-06 21823, 2025

      • lucifer[m]
      • 2025-08-06 21832, 2025

      • lucifer[m]
        There are multiple ones saying the same though
      • 2025-08-06 21840, 2025

      • lucifer[m]
        lucifer[m]: My understanding has always been this
      • 2025-08-06 21856, 2025

      • lucifer[m]
        But i'll check docs when at PC later to confirm
      • 2025-08-06 21803, 2025

      • mayhem[m]
        see the second answer.
      • 2025-08-06 21842, 2025

      • mayhem[m]
        I understood it just like you do, but that is not how it works. faster-fuzzy exists because of this. I know the pain.
      • 2025-08-06 21813, 2025

      • lucifer[m]
        I understand that but with sleep the control can be returned to the process to start running another thread.
      • 2025-08-06 21844, 2025

      • mayhem[m]
        lucifer[m]: no. uwsgi itself uses no threads.
      • 2025-08-06 21850, 2025

      • lucifer[m]
        asyncio does that and if there is monkey patching involved using greenlet or gevent etc it can happen here too.
      • 2025-08-06 21802, 2025

      • mayhem[m]
        let me try turning off threads on lb-test to see if anything breaks, ok?
      • 2025-08-06 21807, 2025

      • lucifer[m]
        Okay sure.
      • 2025-08-06 21859, 2025

      • lucifer[m]
        lucifer[m]: Basically the second answer's first comment.
      • 2025-08-06 21824, 2025

      • mayhem[m]
        "/usr/local/lib/python3.13/site-packages/sentry_sdk/_compat.py:86: Warning: IMPORTANT: We detected the use of uWSGI in preforking mode without thread support. This might lead to crashing workers. Please run uWSGI with both "--enable-threads" and "--py-call-uwsgi-fork-hooks" for full support." when I turn threads to zero.
      • 2025-08-06 21831, 2025

      • mayhem[m]
        time to read about pre-forking mode.
      • 2025-08-06 21811, 2025

      • mayhem[m]
        oh, I see. its not a pure threading model, but a hybrid threads inside process model.
      • 2025-08-06 21840, 2025

      • mayhem[m]
        in that case, putting a simple Lock around the rate limit should fix the problem nicely.
      • 2025-08-06 21808, 2025

      • mayhem[m]
        well, it has the potential to fix it. but I am not conviced. lets pick this up when you're near a computer.
      • 2025-08-06 21855, 2025

      • __BrainzGit
        [troi-recommendation-playground] 14mayhem opened pull request #175 (03main…lock-ratelimit-dict): Add a lock to the rate-limit dict https://github.com/metabrainz/troi-recommendation…
      • 2025-08-06 21819, 2025

      • mayhem[m]
        if the dict munging theory is correct, this should fix that.
      • 2025-08-06 21816, 2025

      • aerozol[m]
        julian45: Another 165 spam comments on the blog, since last time we talked D: Definitely need to get Akismet set up again
      • 2025-08-06 21832, 2025

      • mayhem[m]
        yea, on my task list for very soon.
      • 2025-08-06 21804, 2025

      • aerozol[m]
        Ah thanks mayhem
      • 2025-08-06 21818, 2025

      • lucifer[m]
        mayhem: yes that, ` but a hybrid threads inside process model.`
      • 2025-08-06 21800, 2025

      • lucifer[m]
        a lock will fix it yes by blocking other requests from progressing.
      • 2025-08-06 21858, 2025

      • mayhem[m]
        lucifer[m]: the locks are not held for long. should be fine.
      • 2025-08-06 21855, 2025

      • mayhem[m]
        remember, it works in most cases, so its a race condition. and an occaisonal block for a microsecond or two won't do any harm we can notice.
      • 2025-08-06 21855, 2025

      • lucifer[m]
        <mayhem[m]> "the del that fails, can we..." <- yes but i think we can do without locks, by this.
      • 2025-08-06 21825, 2025

      • lucifer[m]
        instead of del use pop with a None default.
      • 2025-08-06 21829, 2025

      • mayhem[m]
        sure, lets try that.
      • 2025-08-06 21800, 2025

      • lucifer[m]
        one more change by combining the key in dict and dict.get check.
      • 2025-08-06 21837, 2025

      • lucifer[m]
        will open a new pr to try that and otherwise we can use a lock yeah.
      • 2025-08-06 21820, 2025

      • lucifer[m]
        just for completeness sake, one more way to do this is thread local. which is what flask uses for its g and app context.
      • 2025-08-06 21844, 2025

      • d4rkie has quit
      • 2025-08-06 21815, 2025

      • d4rkie joined the channel
      • 2025-08-06 21857, 2025

      • petitminion has quit
      • 2025-08-06 21830, 2025

      • mayhem[m]
        we just received a 1000Eur donation via a bank transfer and I have no way to contact them to thank them. how odd.
      • 2025-08-06 21847, 2025

      • dvirtz[m] joined the channel
      • 2025-08-06 21848, 2025

      • dvirtz[m] posted a file: (19KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/chatbrainz.org/uRMLJkBpNtHhTbjtMMdnYHYA/DBDefs.pm >
      • 2025-08-06 21848, 2025

      • dvirtz[m]
        I moved to a new machine and now my MB dev setup stopped working. I'm using musicbrainz-docker and can see in the db logs... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/download/chatbrainz.org/OfGEUuWvvywrcGrevlBKPukf>)
      • 2025-08-06 21851, 2025

      • SothoTalKer__ has quit
      • 2025-08-06 21803, 2025

      • petitminion joined the channel
      • 2025-08-06 21831, 2025

      • petitminion has quit
      • 2025-08-06 21817, 2025

      • petitminion joined the channel
      • 2025-08-06 21845, 2025

      • __BrainzGit
        [troi-recommendation-playground] 14amCap1712 opened pull request #176 (03main…ratelimits): Use more defensive checks for ratelimits https://github.com/metabrainz/troi-recommendation…
      • 2025-08-06 21851, 2025

      • bitmap[m]
        dvirtz: that's odd, sounds like it's connecting to the wrong DB or as a different user (affecting the `search_path`). the connection info in DBDefs.pm comes from `%ENV` so you'd have to check the `env` variables inside the musicbrainz container
      • 2025-08-06 21829, 2025

      • dvirtz[m] sent a code block: https://matrix.chatbrainz.org/_matrix/media/v3/download/chatbrainz.org/KplttsORCcrsAZYnvCHvIDNX
      • 2025-08-06 21840, 2025

      • dvirtz[m]
        * ```... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/download/chatbrainz.org/KTfjmpcNwhIPumhdoJPhtjnb>)
      • 2025-08-06 21845, 2025

      • bitmap[m]
        dvirtz: looks fine to me, just to clarify, are you are trying to access the website and it's giving an Internal Server Error?
      • 2025-08-06 21811, 2025

      • bitmap[m]
        does docker compose exec musicbrainz bash -c 'carton exec -- admin/psql READWRITE' work for you? if yes, then it's not a configuration issue
      • 2025-08-06 21854, 2025

      • dvirtz[m]
        bitmap
      • 2025-08-06 21854, 2025

      • dvirtz[m]
        That command results in `bash: line 1: carton: command not found`.
      • 2025-08-06 21854, 2025

      • dvirtz[m]
        Most of the pages work but entity edit/create give `Couldn't render template "undef error` and a long html following
      • 2025-08-06 21802, 2025

      • monkey[m] has quit
      • 2025-08-06 21807, 2025

      • bwebbo[m] joined the channel
      • 2025-08-06 21807, 2025

      • bwebbo[m]
        Hi all - I have downloaded a PostgreSQL Data Dump and using pg admin to query. When trying to identify genre or tag info, I'm finding a lot of empty tables / tables with very few rows. Please would someone be able to advise the best way to retrieve these attributes or if it sounds like there may have been an import error etc?
      • 2025-08-06 21807, 2025

      • bwebbo[m]
        (empty tables : release_group_tag, release_tag, recording_tag, artist_tag, recording_first_release_date, l_artist_genre, l_genre_release, l_genre_recording, l_genre_release_group, artist_release_group)
      • 2025-08-06 21857, 2025

      • reosarevok[m]
        bwebbo: did you download the derived dump too? https://musicbrainz.org/doc/MusicBrainz_Database/… that's the one with tags
      • 2025-08-06 21836, 2025

      • bwebbo[m]
        reosarevok[m]: Thanks! I look into this
      • 2025-08-06 21811, 2025

      • reosarevok[m]
        l_x_genre tables shouldn't be particularly relevant to your interests if you're looking for genre tags for stuff
      • 2025-08-06 21833, 2025

      • reosarevok[m]
        (l_area_genre will have stuff like "this genre is originally from Brazil", but that's probably not what you're looking for)
      • 2025-08-06 21818, 2025

      • bwebbo[m]
        * Thanks! I will look into
      • 2025-08-06 21808, 2025

      • bwebbo[m]
        Ah ok, thank you again!
      • 2025-08-06 21800, 2025

      • bitmap[m]
        dvirtz: just tested this from different musicbrainz-docker setups on two different hosts: https://gist.github.com/mwiencek/b119ceae0ed328ed…
      • 2025-08-06 21814, 2025

      • bitmap[m]
        maybe you should remove and rebuild your musicbrainz container, because I'm not sure what's wrong with it
      • 2025-08-06 21823, 2025

      • dvirtz[m]
        bitmap
      • 2025-08-06 21823, 2025

      • dvirtz[m]
        I'm using the musicbrainz-dev image and I don't see Carton being installed there
      • 2025-08-06 21834, 2025

      • bitmap[m]
        ah, maybe the dev image doesn't use carton
      • 2025-08-06 21800, 2025

      • bitmap[m]
        can you paste your .env file so I can try to replicate your setup?
      • 2025-08-06 21810, 2025

      • dvirtz[m]
        There's quite a few differences with the prod image. Is this being maintained?
      • 2025-08-06 21810, 2025

      • dvirtz[m]
        Its database schema is still on 29 for example
      • 2025-08-06 21833, 2025

      • dvirtz[m]
        I updated it manually to 30
      • 2025-08-06 21854, 2025

      • bitmap[m]
        it should be maintained, I haven't used it for development but I'll try to set it up and fix it if needed
      • 2025-08-06 21826, 2025

      • dvirtz[m] posted a file: env (0KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/chatbrainz.org/sUhAscRMdfTdPNsWmDoFXYLj >
      • 2025-08-06 21826, 2025

      • dvirtz[m]
        Thanks