#metabrainz

/

      • lucifer
      • riksucks has quit
      • yyoung[m] has quit
      • the4oo4 has quit
      • rcombs has quit
      • the4oo4 joined the channel
      • rcombs joined the channel
      • riksucks joined the channel
      • d4rkie has quit
      • everdred has quit
      • rdrg109_ has quit
      • milkii_ has quit
      • BrainzGit has quit
      • Freso has quit
      • zas has quit
      • everdred joined the channel
      • zas joined the channel
      • d4rkie joined the channel
      • milkii joined the channel
      • zas has quit
      • zas joined the channel
      • Freso joined the channel
      • rdrg109_ joined the channel
      • BrainzGit joined the channel
      • elomatreb[m] has quit
      • reosarevok has quit
      • akshaaatt has quit
      • void09 has quit
      • reosarevok joined the channel
      • akshaaatt joined the channel
      • void09 joined the channel
      • yyoung[m] joined the channel
      • elomatreb[m] joined the channel
      • param_ joined the channel
      • rektide_ joined the channel
      • riksucks has quit
      • kgz has quit
      • Guest185 has quit
      • param has quit
      • rektide has quit
      • param_ is now known as param
      • kgz joined the channel
      • Guest185 joined the channel
      • riksucks joined the channel
      • Pratha-Fish
        Moin'
      • alastairp: I just realized you've worked w/ freesound.org as well???
      • That's insane. I've used that site for so many of my personal music projects lool
      • Also, my most popular meme was also a fart sample that I took from freesound.org and visualized with a spectrum analyzer and posted on r/FL_studio because it looked beautiful XD
      • Good times
      • odnes joined the channel
      • odnes has quit
      • mayhem
        moin moin!
      • Pratha-Fish
        mayhem: hey tell me the story of how you got amazon to pay a 3 year due invoice by sending them a cake sometime lol
      • mayhem
      • Pratha-Fish
        That text on the cake tho 💀👌
      • leonardo
        just a little passive aggressive
      • mayhem
        it got the job done.
      • leonardo
        that's what is called a sweet revenge
      • alastairp
        morning
      • Pratha-Fish: yes, freesound too
      • yvanzo
        O’Moin
      • reosarevok
        yvanzo: moin! Anything special for this docker release?
      • yvanzo
        hi reosarevok: yes, I drafted release notes but have to make some improvements.
      • reosarevok
        Ok :) I'll start the prod release (won't be around in the evening probably)
      • but we can look at that bit later
      • CatQuest
        oh hey reo ˆ__ˆ
      • reosarevok
        Hi!
      • CatQuest
        :D
      • mayhem
        lucifer: on TS the listened_at_track_name_user_id_ndx_listen index was created live and we didn't decide at the time if we wanted to keep it, yes?
      • because if that is so then PR 2042 makes sense. :)
      • lucifer
        mayhem: yes it was created live. we needed it to keep the on conflict clauses working.
      • still need to figure out how many dupes are there in the db and how to delete those.
      • mayhem
        there is dup detection and removal code in the MBID mapping stuff, you can take a look at it.
      • to use it for TS, I think we would have to do it on a set of chunks at the same time
      • well, one at a time, once the new index is in place.
      • lucifer
        we cant create the index without deleting dupes.
      • mayhem
        why not delete the dups?
      • lucifer
        ah no, i mean we should delete the dupes. i misunderstood your message as to create index first and delete afterwards
      • mayhem
        that would be ideal, but not possible.
      • we will have the problem that new dups can be created while we are deleting the old ones.
      • but I wonder if we can make the script that deletes dups work on ranges or the whole listen table.
      • then we do a month or so at a time and then once that is done, we try to create the index.
      • if that fails, we delete dups across the whole table.
      • but I doubt that would work, so we might end up chasing our tail on this one.
      • lucifer
        i think dup deletion should be fast enough that we can stop ts writer while the script runs.
      • mayhem
        I really doubt that.
      • lucifer
        i see, lets try how fast it goes on one chunk and then decide what to do accordingly.
      • mayhem
        well, if we do it in python then maybe. but pure SQL, I think that is going to OOM
      • lucifer
        hmm, dont think it should oom but yeah really cant say without trying
      • mayhem
        if we just fetch all the tracks ordered by listened_at and the other dedup fields and then just slowly delete all the dups, that could work. it might be fast enough for the second pass to run with TS writer stopped.
      • lucifer
        makes sense
      • yvanzo
        reosarevok: the release draft is up for review
      • reosarevok
        yvanzo: I forget, is this expected?
      • yvanzo
        it depends, for which container?
      • reosarevok
        aphex
      • prod
      • ws
      • yvanzo
        I guess it is expected as the webservice container is running neither renderer nor website.
      • reosarevok
        Ok :)
      • yellowhatpro
        akshaaatt: can you create a new branch for bp integration. I was thinking to create a pr that covers services and some repo stuff
      • reosarevok
        yvanzo: blog post ready for review
      • akshaaatt
        Done, yellowhatpro . Please use the ‘brainzplayer’ branch for the same
      • yellowhatpro
        yussir thankss
      • reosarevok
        yvanzo: all ready, when you feel it looks good, please put out the blog and the docker release (linked the tag already)
      • BrainzGit
        [musicbrainz-android] 14yellowHatpro opened pull request #126 (03brainzplayer…bp-implementation): initial BrainzPlayer integration commit https://github.com/metabrainz/musicbrainz-andro...
      • yvanzo
        reosarevok: I just unlinked bfan.link, looks good otherwise. Can you please review the docker release if you have time?
      • reosarevok
        I did, nothing seemed *weird* but of course I also didn't try the steps :)
      • yvanzo
        I did, so we are good for release :)
      • BrainzGit
        [musicbrainz-docker] release 03v-2022-06-20 has been published by 14yvanzo: https://github.com/metabrainz/musicbrainz-docke...
      • reosarevok
        Yay
      • !m yvanzo
      • BrainzBot
        You're doing good work, yvanzo!
      • reosarevok goes back to the sauna
      • yellowhatpro
        akshaaatt: the pr is very basic for now, the classes were interdependent so I had to create all of the classes related to the service
      • Currently it doesnt do much work, but I am soon gonna update the pr with more stuf
      • akshaaatt
        Sure yellowhatpro. Would you like me to review it now or after a while once you’ve added more stuff? I’m fine with anything.
      • yellowhatpro
        Umm, anything would be fine. Currently the classes aren't complete but yeah reviews will be great..
      • Etua joined the channel
      • Etua has quit
      • KevlarNoir has quit
      • KevlarNoir joined the channel
      • BrainzGit
        [listenbrainz-server] 14amCap1712 merged pull request #2040 (03master…user-mbids-spark): Include user submitted mbids in spark dumps https://github.com/metabrainz/listenbrainz-serv...
      • yvanzo
        Just published the blog post, apparently my first attempt failed.
      • mayhem
        lucifer: which is the last_played API endpoint? I can't find it in the docs...
      • lucifer
        mayhem: you mean when the recommendation was last played? if so, there is not separate endpoint. the recs json includes the timestamp with the mbid.
      • mayhem
        ahhh, ok, no wonder I couldn't find it.
      • lucifer
        those times are available for all recordings but only stored in spark currently. before sending recs to LB, that data is merged with the recs to add a timestamp filed.
      • mayhem
        easy then. :)
      • Pratha-Fish
        hey alastairp sorry for the delay. Had couldn't do a lot today, but I am getting started with the updated to-do list right now.
      • The to-do list is hosted in the journal BTW.
      • Updating it with specifics of the artist conflation issue too
      • BrainzGit
        [acousticbrainz-server] 14alastair merged pull request #403 (03master…python3): Start Python 3 migration https://github.com/metabrainz/acousticbrainz-se...
      • [acousticbrainz-server] 14alastair closed pull request #392 (03master…dump): Use a loop to iterate dataset tables during dumps https://github.com/metabrainz/acousticbrainz-se...
      • [acousticbrainz-server] 14alastair closed pull request #396 (03master…AB-407): AB-407: Redirect legacy API endpoints to new endpoint with http redirect https://github.com/metabrainz/acousticbrainz-se...
      • [acousticbrainz-server] 14alastair opened pull request #424 (03master…disable-submissions): Update data/download descriptions to shut down AB https://github.com/metabrainz/acousticbrainz-se...
      • Pratha-Fish
        alastairp: also, you mentioned the part about making a csv with the following columns: mlhd_recording_mbid, mlhd_artist_mbid, mlhd_recording_name, mlhd_artist_name, mb_recording_artist_credit, mb_artist_mbids, mb_canonical_recording_mbid
      • TBH I am still a bit confused about this one. Maybe breaking it down into some macro steps could help :)
      • BrainzGit
        [critiquebrainz] 14alastair opened pull request #438 (03master…sampledb-missing-entities): Always return dummy data in debug mode if it's not in MusicBrainz https://github.com/metabrainz/critiquebrainz/pu...
      • alastairp
        Pratha-Fish: sure. maybe let's deal with the first 4 columns then
      • you already look up these fields from the mlhd dataset in the `recording` table and the `artist` table
      • this will just involve selecting the `name` field from these tables too, and writing them to a new csv file
      • lucifer
      • to delete listens submitted with same listened_at, user_id but different case for track_name.
      • alastairp
        lucifer: that code skeleton looks familiar ;)
      • lucifer
        alastairp: hehe yes, i copied it from your listen_fill_userid script :D
      • alastairp
        so to confirm, we already reject exact duplicates of (userid, submitted, track_name), but we found these cases where we had case-insensitive dups on track name?
      • lucifer
        currently we don't reject those. the PR adds an index to fix that.
      • before we create the index, we need to cleanup the existing dupes.
      • the intent is we do 1 pass, then turn off ts writer, do another pass. try to create index. restart ts writer.