#metabrainz

Related to submitting listens - Flaky asks "There is one thing I need to sort out though: recordings with no timestamps. Should those be added on MBz where possible or not submitted _at all_?"

"if someone's submitting a single recording that's fine, but it's more about when there's releases that have recordings with no timestamps (messes with calculations)"

lucifer: thoughts? ^

aerozol: do you/Flaky mean with no duration in MB?

lucifer: I've asked, but presumably in mb since you can just load any mb release to submit (e.g. a vinyl release, no need for the music files)

right. so the user has to choose the time which should be submitted with the listen. and when you are submitting a release you probably dont want to specify timings of each recording manually.

rather specify a start time of the listen of the release and fill in timestamps for tracks from MB.

*timestamps for tracks using duration data from MB

aerosol: The image I sent was of a chat application which I worked on so I was telling if we can implement his UI like replacing chat name with playlist name, add users with add tracks and to show errors if a particular track is not found and that croos button to remove a track from playlist

monkey: Hey is there an api to confirm a track is present or not and an api to search tracks using a single character or word like in spotify :)

aerozol: he did send a fair amount, but that's still less than we get in one day :/

(so it's not really sustainable to do it all by hand)

jivte: The look and feel needs to match what’s currently in LB - what monkey shared is probably a good place to start from. You can combine it with your chat design if you want, but we need a mockup with this functionality to sign off

So your next step is to submit a mockup, can be pencil and paper or digital, with what you wanna do :)

Here’s a screenshot of the functions Flaky’s plugin has, we might not need all of them but it all looks good to me

aerozol: mockup of playlist thing which monkey told or the submit a listen feature can u please clarify?

I believe monkey was just saying to use that existing playlist UI as a starting point for your submit listens task, but maybe I read it wrong

aerozol: https://drive.google.com/file/d/1E-0HhWw3lwJyDj...

hey I mad this prototype UI on a paper can u please review it as it will work or not

Thanks jivte, what ticket is that for?

LB-1157 ticket and monkey told me to work on playlist part to improve it as a starting task

LB-1157: Provide a way to manually add a listen on the website https://tickets.metabrainz.org/browse/LB-1157

so I made the above UI

mooooin!

Hi jivte, indeed I was just suggesting the playlist page's add-a-track component as a potential starting point for this new submit-a-listen component, not suggesting to improve the playlist page. The mockup we need is for the submit a listen component that is going to go on the user's listens page

okk okk I understood I am currently working on the mockup design of submit-a-listen component after completing I will share with you

thanks for clarifying :)

O’Moin

monkey: https://twitter.com/ggallynn/status/16005219474...

yvanzo: what happened to the recording index?

lucifer: It completed but in 20h.

yvanzo: i see. how long does it approximately take on v3.0.1?

IIRC it used to take less time than that to rebuild all search indexes.

i see

I’ll run it again to have the exact time with 8 import threads, then with 4 import threads.

yvanzo: can you instead try 8 import threads with query_batch_size = 25000.

to confirm, try on the same commit which took 20h.

lucifer: Actually v3.0.1 fails too with 8 import threads, but it fails faster than master :D

hehe lol

I guess that the load considerably increased since the last schema change.

i guess 16gb ram isn't enough to run indexer + db + solr.

i wonder if we should reduce the amount of data in search response to alleviate load.

It used to run with 4.

(not with the current schema though)

makes sense

I’ll try your settings

Hah lucifer love it. I'm a born-again Flexboxian but still get PTSD…

🤣

aerozol: monkey: lucifer: I completed the design of the LB-1157 TICKET

LB-1157: Provide a way to manually add a listen on the website https://tickets.metabrainz.org/browse/LB-1157

can u please review it

jivte: looks good but this will need a way to enter the listen timestamp as well at least.

[listenbrainz-server] 14amCap1712 merged pull request #2276 (03master…move-mbid-mapping): Move listenbrainz/mbid_mapping to repo root https://github.com/metabrainz/listenbrainz-serv...

[listenbrainz-server] 14amCap1712 opened pull request #2281 (03master…savepoint): Try using a SAVEPOINT to avoid transaction failures https://github.com/metabrainz/listenbrainz-serv...

alastairp: is it fine i rebase LB#2157 ? the files have moved now.

Add dumps for musicbrainz metadata tables: https://github.com/metabrainz/listenbrainz-serv...

lucifer: It crashed even faster using 25000

At least we optimized something :)

lolol

yvanzo: can you run docker stats in a terminal side by side and try to monitor (not sure writing it to a file would work) which container is using how much memory?

just a hunch that the culprit might not be the db but rather the indexer.

lucifer: go for it

👍 thanks

lucifer: like could u clarify on timestamps as I am not aware of it :(

jivte: have a look at the screenshot I sent of flaky's plugin - need to be able to say when it was submitted, and an option to paste in a whole album would be ideal. A common use case will be someone listening to a whole album and wanting to submit it. On their ipod or on vinyl or whatever

jivte: see left of the heart icons, there is a time displayed showing when the track was listened. https://usercontent.irccloud-cdn.com/file/qvCn0...

You'll see that screenshot has two options - 'now', which sets the time stamp to the users current time, and then adds all the listens in order of time after that

this listened_at time, track name, artist name are the three bare minimum things needed to submit a listen to LB.

And the second option, 'past listen', which let's you set a time and date for the tracks your submitting

lucifer: right before it crashes, db uses up to 4.x GB and indexer up to 8.x GB

yvanzo: makes sense. let me push a quick patch to test something.

(using 25000 for query_batch_size)

which branch are you testing this on currently?

raiseload but rebased on master

it should probably be rebased anyway

Can test master instead

Have not seen any difference with the WIP raiseload patch anyway

note that search (Solr) stays stable with ~1.4 GB

(its logs show some activity though)

lucifer: aerozol : like should we take these timestamps from user or like find a inner way to show it

jivte: as aerozol said, you can use this as a reference. https://usercontent.irccloud-cdn.com/file/9rqiB...

if its set to now, then use that time, if set to past listen, then probably show a datetime selector to let the user select a time.

jivte: if you want to test out the Picard plugin here's the link: https://gitlab.com/Flaky/picard-submit-to-liste...

If you haven't used Picard before it will be a good exercise in trying that out as well!

Part of the metabrainz ecosystem :)

yvanzo: makes sense, can you try again with this: https://github.com/metabrainz/sir/tree/limit-lr...

using 25000 for query_batch_size?

yes

since that crashes faster we'll know sooner if this change has any affect.

not really

(any effect)

In dmesg logs, the OOM was about postgres, not python

(but we don’t limit indexer memory indeed)

yes right. my hunch is that python consumes all the memory leaving nothing for PG.

lucifer: aerosol: like when user selects a listen we will show another modal to select timestamps or embed in this same modal itself

mayhem: hi! if you are around, can you also review this sql file? https://github.com/metabrainz/listenbrainz-serv...

and we will have two options for timestamps like now and past listen?

i will then do a dry run of 2022 stats to find what else needs to be fixed in the pipeline.

jivte: yes to both options. about modal same or different not sure, maybe others have suggestions.

[listenbrainz-server] 14amCap1712 merged pull request #2281 (03master…savepoint): Try using a SAVEPOINT to avoid transaction failures https://github.com/metabrainz/listenbrainz-serv...

okk aerozol is not around i think and one thing i have exams next week so the work will be bit slow hope there is no problem :)

no worries, all the best.

thanks

monkey: can u tell me about that modal thing

So for date and time selection you can use the same one we use on the user listen page navigation, that one in the middle: https://usercontent.irccloud-cdn.com/file/nHAEF...

As they said above, I think the Picard plugin does it well: two radio options: "now" and "past listen". Next to past listen you can put that date-time select component.

(It'll need to be disabled when the "now" radio option is selected)

jivte: no rush! For the time thing is would be good to look at flakys plugin and submit a listen so you know what we mean

The are also last.fm submitters that you can look up and try out

*time being

aeroazol

aerozol: like I don't know how to use picard plugin

can u help like how to get started

Install https://picard.musicbrainz.org

quick ugly mockup of what I meant: https://usercontent.irccloud-cdn.com/file/8ODIS...

And then that link to flakys plugin has install instructions. But just ask if you get stuck

monkey: like can we set default option as now or do we need to show date pickup only when option is selected

and when no option selected no date pickup component to show

Default selection should be "now", and I think we can show the date selector even when "now" is selected, but disabled.

okk now got clarity

Thank you all

lucifer: lgtm

thanks 👍