#metabrainz

/

      • petitminion has quit
      • petitminion joined the channel
      • petitminion has quit
      • suvid[m]
        lucifer i think the user_data_export table exists
      • shouldn't i create user_data_import table similarly?
      • lucifer[m]
        [@suvid:matrix.org](https://matrix.to/#/@suvid:matrix.org) what data would you store in that new table?
      • suvid[m] uploaded an image: (43KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/UGPOFzmxxRtYcGfbchAYNfZM/image.png >
      • suvid[m]
        yea it kinda seems redundant
      • but that's how export task is created as well
      • to track its progress maybe
      • as there is no progress indicator in background_task
      • pite_ has quit
      • lucifer[m]
        suvid: the export task is created to store the progress and the file metadata as the exports should be available for download later for the user.
      • for imports, you can store the status in https://github.com/metabrainz/listenbrainz-serv...
      • that's what the last.fm importer is doing
      • in your case, just make sure of one thing. create an entry for the user in the listens_importer table with the spotify service to ensure the update works otherwise there won't be anything to update
      • zas[m] joined the channel
      • zas[m]
        I will have to reboot yamaoka, it is spamming logs with kern messages related to pci aer, it was almost disk full. I took measures to limit log sizes, but the core issue will likely need at least one reboot. I just reclaimed 800G
      • MeB team: There are various non-redundant services running there
      • I mitigated the disk space issue, but the reboot is rather urgent, so please have a look asap and move stuff if needed
      • davic has quit
      • lucifer[m]
        zas: suggestion for which server to move CB to from yamaoka?
      • zas[m]
        Perhaps selda
      • davic joined the channel
      • lucifer[m]
        moved CB.
      • zas: can you check if the CB gateways config is fine? i migrated to selda and the containers are running fine but the website returns a 502.
      • or if there is nothing else important on yamaoka might as well reboot it and i'll move CB back.
      • zas[m]
        Fixed, there was an issue with serviceregistrator on selda, restarted, and the service appeared
      • lucifer[m]
        thanks!
      • Maxr1998_ joined the channel
      • Maxr1998 has quit
      • _BrainzGit
        [musicbrainz-server] 14reosarevok merged pull request #3554 (03master…MBS-14041): MBS-14041: Don't display ended primary alias by entities https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-14041: Don't display ended primary alias next to name https://tickets.metabrainz.org/browse/MBS-14041
      • _BrainzGit
        [musicbrainz-server] 14reosarevok merged pull request #3539 (03master…remove-model_to_search_hint_type_id): Remove unused model_to_search_hint_type_id https://github.com/metabrainz/musicbrainz-serve...
      • [musicbrainz-server] 14reosarevok merged pull request #3544 (03master…MBS-14024): MBS-14024: Show genre rels on more places https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-14024: Artist-Genre relationships not visible on artist page https://tickets.metabrainz.org/browse/MBS-14024
      • _BrainzGit
        [metabrainz.org] 14fettuccinae opened pull request #508 (03metabrainz-notifications…metabrainz-notifications): Add notification/send endpoint. https://github.com/metabrainz/metabrainz.org/pu...
      • [musicbrainz-server] 14reosarevok merged pull request #3543 (03master…MBS-14021): MBS-14021: Report for pseudo-releases with cover art https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-14021: Report: pseudo-releases with cover art https://tickets.metabrainz.org/browse/MBS-14021
      • mamanullah7[m] joined the channel
      • mamanullah7[m]
        hey lucifer i'm having trouble in verifying `OAuth2` implementation as well as i cant validate api becoz funkwhale setup when i tried to login with superuser funkwhale i'm getting `access denied`
      • how can fix this?
      • should i push this commit ?
      • reosarevok[m]
        yvanzo: I suspect SEARCH-745 is an oversight, but can you confirm it's not intentional?
      • BrainzBot
        SEARCH-745: Aliases in search responses lack "ended" data https://tickets.metabrainz.org/browse/SEARCH-745
      • lucifer[m]
        <mamanullah7[m]> "hey lucifer i'm having trouble..." <- > <@m.amanullah7:matrix.org> hey lucifer i'm having trouble in verifying `OAuth2` implementation as well as i cant validate api becoz funkwhale setup when i tried to login with superuser funkwhale i'm getting `access denied`... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)
      • mamanullah7[m]
        <lucifer[m]> "> <@m.amanullah7:matrix.org> hey..." <- Sure thanks!
      • _BrainzGit
        [musicbrainz-server] 14reosarevok opened pull request #3556 (03master…MBS-14050): MBS-14050: Add tests for find_best_primary_alias https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-14050: Add tests for find_best_primary_alias https://tickets.metabrainz.org/browse/MBS-14050
      • reosarevok[m]
        derat: if you want to take a quick look at some point in case you can think of more test cases, it'd be appreciated :) ^
      • _BrainzGit
        [listenbrainz-server] 14RayyanSeliya opened pull request #3295 (03master…add-ia-indexer): Add Internet Archive indexer for metadata cache https://github.com/metabrainz/listenbrainz-serv...
      • rayyan_seliya123
        _BrainzGit: Hey lucifer: just added my indexer script for your review let me know if any changes needed I tried to run the script but it was giving me redis connection errors even my docker was running used `./develop.sh build ` and` ./develop.sh up ` and running `python -m listenbrainz.metadata_cache.internetarchive.ia_indexer` 🙂
      • petitminion joined the channel
      • _BrainzGit
        [metabrainz.org] 14mayhem merged pull request #507 (03metabrainz-notifications…notification-table): Add fetch, mark and delete endpoints for notifications https://github.com/metabrainz/metabrainz.org/pu...
      • mamanullah7[m]
        <lucifer[m]> "> <@m.amanullah7:matrix.org> hey..." <- lucifer: Done u can check now! and do let me know if any changes! its untested from my end i hope it dont have much errors or issues!!
      • petitminion has quit
      • mayhem[m]
        (.ve) blackbox:~/metabrainz/faster-fuzzy/build->./mapping_server ../index/ "Kikagaku Moyo" nana "masana temples"... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)
      • monkey: ^^
      • fettuccinae[m] has quit
      • zas[m]
        yamaoka was rebooted and the spammy message in logs tamed
      • mayhem[m]
        2025-06-05 15:24:13: artist index size: 344452391
      • where did my code invent all these artists from???
      • julian45[m] has quit
      • oh. bytes. not rows. 😆
      • pite joined the channel
      • julian45[m] joined the channel
      • julian45[m]
        <mayhem[m]> "(.ve) blackbox:~/metabrainz/..." <- > <@mayhem:chatbrainz.org> (.ve) blackbox:~/metabrainz/faster-fuzzy/build->./mapping_server ../index/ "Kikagaku Moyo" nana "masana temples"... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)
      • mayhem[m]
        well spotted, but not a cause for alarm. The package in question is unidecode that attempts to come up with a stable ASCII serialization for all "non ascii words". The point is never to put these in front of humans, since they are a ton of garbage.
      • they are, however , quite useful for making search indexes, where the search results will need to be checked against the original strings and not the index garbage.
      • julian45[m]
        i see
      • makes sense
      • Sophist_UK joined the channel
      • Sophist-UK has quit
      • _BrainzGit
        [musicbrainz-server] 14reosarevok opened pull request #3557 (03master…MBS-14046): MBS-14046: Use user time zone for anniversary message https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-14046: Anniversary message displayed too soon https://tickets.metabrainz.org/browse/MBS-14046
      • Sophist_UK is now known as Sophist-UK
      • Kladky joined the channel
      • _BrainzGit
        [musicbrainz-server] 14reosarevok opened pull request #3558 (03master…MBS-13983): MBS-13983: Load primary aliases in more places for autocomplete / ws::js https://github.com/metabrainz/musicbrainz-serve...
      • BrainzBot
        MBS-13983: Primary aliases not always shown on the release relationship editor https://tickets.metabrainz.org/browse/MBS-13983
      • lucifer[m]
        holycow23: Listens have msid so I will need a mapping of msid to the mbid, yes but they can also have mbids. there are multiple cases possible. listens with only msid - no mapped mbid, listens with msid - mapped to a mbid. listens with a msid and user specified mbid. rest is correct.
      • petitminion joined the channel
      • holycow23[m]
        I am talking about the moment when a user plays a song, it comes in as a listen just with the msid right, then it is mapped to the correspoding mbid
      • petitminion has quit
      • lucifer[m]
        [@holycow23:matrix.org](https://matrix.to/#/@holycow23:matrix.org) it's assigned a msid and we try to map it to a mbid but a match may not be found always.
      • holycow23[m]
        lucifer[m]: Okay
      • _BrainzGit
        [metabrainz.org] 14amCap1712 opened pull request #509 (03master…client-credentials-grant): oauth: add client credentials grant https://github.com/metabrainz/metabrainz.org/pu...
      • lucifer[m]
        mayhem: @fettuccinae:matrix.org will need this for the auth parts of his project at some point ^
      • holycow23[m]
        lucifer: I have a code snippet below, so these are the queries being used right now in the `listening_activity` stats... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)
      • * lucifer: I have a code snippet below, so these are the queries being used right now in the `listening_activity` stats... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)
      • * lucifer: I have a code snippet below, so these are the queries being used right now in the `listening_activity` stats... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)
      • * lucifer: I have a code snippet below, so these are the queries being used right now in the `listening_activity` stats... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)
      • lucifer[m]
        holycow23: if you are running it on spark then it doesn't involve timescaledb at all. but yes this particular query could be run on timescaledb by accessing listens from listens table if you wanted to.
      • we don't run stats queries on timescaledb at all to avoid slowing down the rest of the website.
      • holycow23[m]
        lucifer[m]: So is data being sent over from timescaleDB to Spark in those specific table format?
      • lucifer[m]
        the data is sent in a parquet files, for instance you can download this dump https://data.metabrainz.org/pub/musicbrainz/lis..., extract it and get parquet files.
      • you can load those parquet files in pandas or duckdb etc and explore their content.
      • holycow23[m]
        Okay will look into it
      • lucifer[m]
        the schema is spark terms is defined at: https://github.com/metabrainz/listenbrainz-serv... but browsing the file with pandas/tool of your choice might help make it clearer
      • _BrainzGit
        [listenbrainz-server] 14MonkeyDo opened pull request #3296 (03master…bootstrap-fixes): Fix Bootstrap5 migration issues https://github.com/metabrainz/listenbrainz-serv...
      • lucifer[m]
        rayyan_seliya123: did you run the `python -m listenbrainz.metadata_cache.internetarchive.ia_indexer` command inside the docker container or outside?
      • try running ./develop.sh exec web python -m listenbrainz.metadata_cache.internetarchive.ia_indexer
      • JadedBlueEyes has quit
      • JadedBlueEyes joined the channel
      • rayyan_seliya123
        <lucifer[m]> "rayyan_seliya123: did you run..." <- I think outside !
      • <lucifer[m]> "try running ./develop.sh exec..." <- Okk let me try this or should i also need to run `./develop.sh `build and up ?
      • lucifer[m]
        rayyan_seliya123: your docker containers need to up when you run this so if they are not already running you need to run `./develop.sh up` yes.
      • if your docker containers are running already then just run the exec command.
      • rayyan_seliya123
        lucifer[m]: Okk I am building and running it
      • lucifer[m]
        the code inside listenbrainz directory is volume mapped to the container so it will update automatically. you need to run build only if you add a python or javascript dependency.
      • rayyan_seliya123
        lucifer[m]: Sure !
      • rostiku[m] joined the channel
      • rostiku[m]
        Hi guys, can I ask a question here about track filtering?
      • rayyan_seliya123 uploaded an image: (45KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/ObjbaRuIhPfkZKTAtDocvGWz/image.png >
      • rayyan_seliya123
        rayyan_seliya123: i am getting this after build and up and that command
      • lucifer[m]
        <rayyan_seliya123> "image.png" <- you need to import it as listenbrainz.metadata_cache.internetarchive
      • <rostiku[m]> "Hi guys, can I ask a question..." <- sure
      • rostiku[m]
        hey lucifer
      • this question isnt really about metabrainz, but musicbrainz database structure
      • lucifer[m]
        musicbrainz dev team is in the channel so they can answer your questions.
      • rostiku[m]
        I see musicbrainz categorizes albums into release_group entries, and then a release_group can have many releases. I'm looking for a way to filter for the original release in a release_group. As I can see that some releases belonging to a release_group can be versions from Japan / Europe, which I am not interested in, but they are still marked as "official" in release_status
      • lucifer[m]
        bitmap, reosarevok ^
      • rostiku[m]
        I would come up with a sketchy way to do this myself, but looking to see an expert opinion :P
      • lucifer[m]
        i don't think there is a concept of the original release but it should be possible to find the earliest release.
      • rostiku[m]
        thats kind of what I'm thinking, it looks like the "original" release would be the one that came first, and since we have release dates, we can filter out that way
      • reosarevok[m]
        That sounds about right
      • Feel free to just filter by earliest if you prefer. Selecting by min date would work probably?
      • rostiku[m]
        well they can also sometimes share dates
      • bitmap[m]
        you can join with release_first_release_date to do that
      • rostiku[m]
        release_first_release_date is a table?
      • bitmap[m]
        if you want to prioritize certain countries, too, you can join with the release_event view
      • (or just the release_country table if you want to exclude releases without any country set)
      • rostiku[m]: yes, and it has to be built first
      • ./admin/BuildMaterializedTables release_first_release_date
      • rostiku[m]
        cool, I will look into it, thanks guys
      • spynxic has quit
      • spynxic joined the channel
      • Jade[m] joined the channel
      • Jade[m]
        re that DMARC report ticket by the way, there's a fair chance that it isn't our issue at all and is in fact someone else trying to send email from the metabrainz domain
      • I've got DMARC reports coming in on a domain which I know isn't sending any emails now, for example
      • spynxic has quit
      • spynxic joined the channel
      • Also julian45 you mentioned when we were talking that it was HAProxy? Got sent this today, perhaps it'll be useful https://progress.opensuse.org/news/125
      • derwin joined the channel
      • julian45[m]
        thx, will have a look at these!
      • derwin
        hi. I am finally getting around to writing a bot for Remix relationships. and by "I" of course I mean vibe coding by Claude.ai. any advice on whether I should do one bot like murdos_bot or one bot per function or etc?
      • Kladky has quit
      • aerozol[m] has quit