#metabrainz

/

      • aerozol
        Related to submitting listens - Flaky asks "There is one thing I need to sort out though: recordings with no timestamps. Should those be added on MBz where possible or not submitted _at all_?"
      • "if someone's submitting a single recording that's fine, but it's more about when there's releases that have recordings with no timestamps (messes with calculations)"
      • lucifer: thoughts? ^
      • smach has quit
      • wargreen has quit
      • wargreen joined the channel
      • saumon has quit
      • lucifer
        aerozol: do you/Flaky mean with no duration in MB?
      • saumon joined the channel
      • aerozol
        lucifer: I've asked, but presumably in mb since you can just load any mb release to submit (e.g. a vinyl release, no need for the music files)
      • lucifer
        right. so the user has to choose the time which should be submitted with the listen. and when you are submitting a release you probably dont want to specify timings of each recording manually.
      • rather specify a start time of the listen of the release and fill in timestamps for tracks from MB.
      • *timestamps for tracks using duration data from MB
      • d4rk joined the channel
      • d4rk-ph0enix has quit
      • jivte joined the channel
      • jivte
        aerosol: The image I sent was of a chat application which I worked on so I was telling if we can implement his UI like replacing chat name with playlist name, add users with add tracks and to show errors if a particular track is not found and that croos button to remove a track from playlist
      • monkey: Hey is there an api to confirm a track is present or not and an api to search tracks using a single character or word like in spotify :)
      • reosarevok
        aerozol: he did send a fair amount, but that's still less than we get in one day :/
      • (so it's not really sustainable to do it all by hand)
      • jivte has quit
      • jivte joined the channel
      • jivte has quit
      • aerozol
        jivte: The look and feel needs to match what’s currently in LB - what monkey shared is probably a good place to start from. You can combine it with your chat design if you want, but we need a mockup with this functionality to sign off
      • So your next step is to submit a mockup, can be pencil and paper or digital, with what you wanna do :)
      • Here’s a screenshot of the functions Flaky’s plugin has, we might not need all of them but it all looks good to me
      • jivte joined the channel
      • jivte
        aerozol: mockup of playlist thing which monkey told or the submit a listen feature can u please clarify?
      • aerozol
        I believe monkey was just saying to use that existing playlist UI as a starting point for your submit listens task, but maybe I read it wrong
      • jivte has quit
      • jivte joined the channel
      • jivte
      • hey I mad this prototype UI on a paper can u please review it as it will work or not
      • aerozol
        Thanks jivte, what ticket is that for?
      • jivte
        LB-1157 ticket and monkey told me to work on playlist part to improve it as a starting task
      • BrainzBot
        LB-1157: Provide a way to manually add a listen on the website https://tickets.metabrainz.org/browse/LB-1157
      • jivte
        so I made the above UI
      • d4rk-ph0enix joined the channel
      • d4rkie has quit
      • vibhoo_24 joined the channel
      • mayhem
        mooooin!
      • vibhoo_24 has quit
      • monkey
        Hi jivte, indeed I was just suggesting the playlist page's add-a-track component as a potential starting point for this new submit-a-listen component, not suggesting to improve the playlist page. The mockup we need is for the submit a listen component that is going to go on the user's listens page
      • jivte
        okk okk I understood I am currently working on the mockup design of submit-a-listen component after completing I will share with you
      • thanks for clarifying :)
      • yvanzo
        O’Moin
      • lucifer
      • yvanzo: what happened to the recording index?
      • Sophist-UK has quit
      • Sophist-UK joined the channel
      • yvanzo
        lucifer: It completed but in 20h.
      • lucifer
        yvanzo: i see. how long does it approximately take on v3.0.1?
      • yvanzo
        IIRC it used to take less time than that to rebuild all search indexes.
      • lucifer
        i see
      • yvanzo
        I’ll run it again to have the exact time with 8 import threads, then with 4 import threads.
      • lucifer
        yvanzo: can you instead try 8 import threads with query_batch_size = 25000.
      • to confirm, try on the same commit which took 20h.
      • yvanzo
        lucifer: Actually v3.0.1 fails too with 8 import threads, but it fails faster than master :D
      • lucifer
        hehe lol
      • yvanzo
        I guess that the load considerably increased since the last schema change.
      • lucifer
        i guess 16gb ram isn't enough to run indexer + db + solr.
      • i wonder if we should reduce the amount of data in search response to alleviate load.
      • yvanzo
        It used to run with 4.
      • (not with the current schema though)
      • lucifer
        makes sense
      • yvanzo
        I’ll try your settings
      • monkey
        Hah lucifer love it. I'm a born-again Flexboxian but still get PTSD…
      • lucifer
        🤣
      • jivte
        aerozol: monkey: lucifer: I completed the design of the LB-1157 TICKET
      • BrainzBot
        LB-1157: Provide a way to manually add a listen on the website https://tickets.metabrainz.org/browse/LB-1157
      • jivte
      • can u please review it
      • lucifer
        jivte: looks good but this will need a way to enter the listen timestamp as well at least.
      • BrainzGit
        [listenbrainz-server] 14amCap1712 merged pull request #2276 (03master…move-mbid-mapping): Move listenbrainz/mbid_mapping to repo root https://github.com/metabrainz/listenbrainz-serv...
      • [listenbrainz-server] 14amCap1712 opened pull request #2281 (03master…savepoint): Try using a SAVEPOINT to avoid transaction failures https://github.com/metabrainz/listenbrainz-serv...
      • lucifer
        alastairp: is it fine i rebase LB#2157 ? the files have moved now.
      • BrainzBot
        Add dumps for musicbrainz metadata tables: https://github.com/metabrainz/listenbrainz-serv...
      • yvanzo
        lucifer: It crashed even faster using 25000
      • At least we optimized something :)
      • lucifer
        lolol
      • yvanzo: can you run docker stats in a terminal side by side and try to monitor (not sure writing it to a file would work) which container is using how much memory?
      • just a hunch that the culprit might not be the db but rather the indexer.
      • alastairp
        lucifer: go for it
      • lucifer
        👍 thanks
      • jivte
        lucifer: like could u clarify on timestamps as I am not aware of it :(
      • aerozol
        jivte: have a look at the screenshot I sent of flaky's plugin - need to be able to say when it was submitted, and an option to paste in a whole album would be ideal. A common use case will be someone listening to a whole album and wanting to submit it. On their ipod or on vinyl or whatever
      • jasje joined the channel
      • lucifer
        jivte: see left of the heart icons, there is a time displayed showing when the track was listened. https://usercontent.irccloud-cdn.com/file/qvCn0...
      • aerozol
        You'll see that screenshot has two options - 'now', which sets the time stamp to the users current time, and then adds all the listens in order of time after that
      • lucifer
        this listened_at time, track name, artist name are the three bare minimum things needed to submit a listen to LB.
      • aerozol
        And the second option, 'past listen', which let's you set a time and date for the tracks your submitting
      • jasje has quit
      • yvanzo
        lucifer: right before it crashes, db uses up to 4.x GB and indexer up to 8.x GB
      • jasje joined the channel
      • lucifer
        yvanzo: makes sense. let me push a quick patch to test something.
      • yvanzo
        (using 25000 for query_batch_size)
      • lucifer
        which branch are you testing this on currently?
      • yvanzo
        raiseload but rebased on master
      • it should probably be rebased anyway
      • Can test master instead
      • Have not seen any difference with the WIP raiseload patch anyway
      • note that search (Solr) stays stable with ~1.4 GB
      • (its logs show some activity though)
      • jivte
        lucifer: aerozol : like should we take these timestamps from user or like find a inner way to show it
      • lucifer
        jivte: as aerozol said, you can use this as a reference. https://usercontent.irccloud-cdn.com/file/9rqiB...
      • if its set to now, then use that time, if set to past listen, then probably show a datetime selector to let the user select a time.
      • aerozol
        jivte: if you want to test out the Picard plugin here's the link: https://gitlab.com/Flaky/picard-submit-to-liste...
      • If you haven't used Picard before it will be a good exercise in trying that out as well!
      • Part of the metabrainz ecosystem :)
      • lucifer
        yvanzo: makes sense, can you try again with this: https://github.com/metabrainz/sir/tree/limit-lr...
      • yvanzo
        using 25000 for query_batch_size?
      • lucifer
        yes
      • since that crashes faster we'll know sooner if this change has any affect.
      • yvanzo
        not really
      • (any effect)
      • In dmesg logs, the OOM was about postgres, not python
      • (but we don’t limit indexer memory indeed)
      • Maxr1998 has quit
      • lucifer
        yes right. my hunch is that python consumes all the memory leaving nothing for PG.
      • jivte
        lucifer: aerosol: like when user selects a listen we will show another modal to select timestamps or embed in this same modal itself
      • lucifer
        mayhem: hi! if you are around, can you also review this sql file? https://github.com/metabrainz/listenbrainz-serv...
      • jivte
        and we will have two options for timestamps like now and past listen?
      • lucifer
        i will then do a dry run of 2022 stats to find what else needs to be fixed in the pipeline.
      • Maxr1998 joined the channel
      • jivte: yes to both options. about modal same or different not sure, maybe others have suggestions.
      • BrainzGit
        [listenbrainz-server] 14amCap1712 merged pull request #2281 (03master…savepoint): Try using a SAVEPOINT to avoid transaction failures https://github.com/metabrainz/listenbrainz-serv...
      • jivte
        okk aerozol is not around i think and one thing i have exams next week so the work will be bit slow hope there is no problem :)
      • lucifer
        no worries, all the best.
      • jivte
        thanks
      • monkey: can u tell me about that modal thing
      • monkey
        So for date and time selection you can use the same one we use on the user listen page navigation, that one in the middle: https://usercontent.irccloud-cdn.com/file/nHAEF...
      • As they said above, I think the Picard plugin does it well: two radio options: "now" and "past listen". Next to past listen you can put that date-time select component.
      • (It'll need to be disabled when the "now" radio option is selected)
      • aerozol
        jivte: no rush! For the time thing is would be good to look at flakys plugin and submit a listen so you know what we mean
      • The are also last.fm submitters that you can look up and try out
      • *time being
      • jivte
        aeroazol
      • aerozol: like I don't know how to use picard plugin
      • can u help like how to get started
      • aerozol
      • monkey
        quick ugly mockup of what I meant: https://usercontent.irccloud-cdn.com/file/8ODIS...
      • aerozol
        And then that link to flakys plugin has install instructions. But just ask if you get stuck
      • jivte
        monkey: like can we set default option as now or do we need to show date pickup only when option is selected
      • and when no option selected no date pickup component to show
      • monkey
        Default selection should be "now", and I think we can show the date selector even when "now" is selected, but disabled.
      • jivte
        okk now got clarity
      • Thank you all
      • mayhem
        lucifer: lgtm
      • lucifer
        thanks 👍