#metabrainz

[musicbrainz-server] 14y-young opened pull request #2267 (03master…external-links-editor-ui-improvement): MBS-11919: Optimize UI of external links editor https://github.com/metabrainz/musicbrainz-serve...

bitmap: Is it intentional that pages like https://musicbrainz.org/artist/b1f99e23-6537-44... do not show external links on the sidebar?

Or is it just because the URLs are accidentally not loaded because of the restriction to the specific reltype?

[critiquebrainz] 14jdaok opened pull request #383 (03master…add-bulk-get-review-endpoint): Add bulk get review by id endpoint https://github.com/metabrainz/critiquebrainz/pu...

monkey: "Error while submitting feedback JSON document must contain recording_msid and score top level keys" when I click heart on listening now track on lb prod.

also mooooin!

"[21:17] <ruaok> I am now, lucifer "

#messagestakenoutofcontext

(sorry :D)

ruaok: I suppose that's expected, the playing_now listens don't have the necessary recording_msid

Buttons should be disabled in that case though. Can you mae a ticekt please?

Actually, never mind, I got it

Time to refactor the listen component !

may the force be with you, monkey

I'll have to think of how to make that work on the backend.

now playing listens are not lookup up for msid. since that part was moved to ts writer.

yes, is the problem.

we could add msid lookup for now playing listens directly in api or websockets though.

only api actually (putting in ws will cause inconsistencies)

that leaves the possibility of having feedback for msids that may not turn into listens.

if the user clicks hate and skips the track before 30 seconds, we've got shit data.

right, but we can already love/hate another person's listens?

we can on test

how complex would it be to have a "queue" of likes (username, artist-recording credit)?

which we might not have listened anyways

and then turn them into real likes as a listen comes in

I guess that doesn't help it if the track never turns into a listen (user skips before 30s)

more complex than I care for, really. but that seems to be the right answer.

perhaps the queue is not the right method here

redis with a timeout of 1hr?

or, for a like without an msid, directly do an acr lookup and store it as an mbid like

i think its fine as is. if you have a listen for msid, the feedback is considered otherwise spark or whatever else ignores it

lucifer: how is it fine? it doesn't provide a feature I feel is missing.

acr lookup is an interesting idea, alastairp

I think my redis idea is doable:

1. when we get love/hate on now listening, we add an item to redis: (user, artist_name, track_name) with expiry of 1hr

ruaok: i meant fine as in let people submit feedback as they want regarless of listens. if you give feedback for a track that you haven't listened to it will just get ignored.

2. when the TS writer gets a listen, it checks to see if that exists in redis. if so, add feedback, remove redis key

lucifer: ah, I see. but then we have cruft in the DB.

yes but two things to consider, we have an api which i am almost sure does not have any of these checks https://listenbrainz.readthedocs.io/en/producti...

2) we will probably want people to give feedback on their recommendations, not necessary that people listen to those before providing feedback

yes, good points.

esp the hate part. we don't want to force people to listen to a track they hate at second 3.

so, if we add a add-now-listening-feedback API endpoint which provides user, artist, track, we generate MSIDs, and add a feeback event.

like that?

But maybe it gets *AMAZING* at second 5! :P

your mom is amazing at 5 seconds.

an endpoint for now playing listens sounds good

k, let me add a ticket for that.

and as for the feedback itself, my suggestion is keep that feedback. ignore feedback without listens for the time being. once we incorporate feedback with listens in recs, we can try to figure out to use the one without listens as well.

agreed. and we could always cleanup feedback later if we find it needs to be done.

+1

LB-950: Add support for love/hate on a now listening listen

lucifer: another thing we discussed in person yesterday...

the listen_mbid_mapping table currently uses artist_credit_id, when it would clearly be better to use artist_mbids[].

and the fact that the MBID mapping tables previously didn't contain all of the releases (it skipped VA releases) and that the old mapping had some minor data instability.

it would be good to re-generate the table, but named tmp_listen_mbid_mapping.

once that is caught up, we stop both mbid-mapping writers and swap one table for the other. then restart the new mapping writer as the default one. then drop the old table.

make sense?

does pg do table aliases?

I guess that's basically a view

and I guess table renames are instant too

yeah, you can do it all in one transaction.

ruaok:

> The DROP COLUMN form does not physically remove the column, but simply makes it invisible to SQL operations. Subsequent insert and update operations in the table will store a null value for the column. Thus, dropping a column is quick but it will not immediately reduce the on-disk size of your table, as the space occupied by the dropped column is not reclaimed. The space will be reclaimed over time as existing rows are updated.

although since we're append-only we're not going to free any space unless we vacuum full

(or migrate to a larger server at some time in the future)

I think that is all fine.

agreed

[listenbrainz-server] 14mayhem opened pull request #1618 (03master…include-all-releases-in-mapping): Add all releases (not use non VA) into the mapping https://github.com/metabrainz/listenbrainz-serv...

ruaok: yup, makes sense. +1

Something else we talked about yesterday, not sure if that's still not working as intended: LB-493

LB-493: Now Playing not resetting after elapsed track time https://tickets.metabrainz.org/browse/LB-493

we don't have a standard track duration field iirc?

so its possible that the track length is being sent in some field that we don't try to detect. https://github.com/metabrainz/listenbrainz-serv...

while we are at source url stuff, maybe we should document duration also?

huh, I thought that it was already documented

i thought the same but can't find it in docs currently so probably not.

LB-119

LB-119: Add track duration as optional field to listens https://tickets.metabrainz.org/browse/LB-119

lucifer: do the spark dumps have all of the metadata field?

and, does the spark infrastructure have an interactive shell/notebook functionality?

that is - can I log in and play around with the data on a fast cluster somehow?

no, just the artist/recording/release name and the associated mbids.

ok, no prob. let me put it on one of my fast clusters

we have a jupyter server - it does not 24/7 as its a bit hacky. i usually bring one up when i need it.

*run 24/7

i can bring it up now if you want?

I'm just interested in a histogram of the additional data fields

nah, there's not enough data there, I'll do it with a dump myself

ah ok 👍

when you a moment lucifer , please take a look at: https://github.com/metabrainz/listenbrainz-serv...

it would help me along with other things for the day

ruaok: looking at it now, is `1` the artist credit id for VA?

yes.

this is the same type of hack that we discussed yesterday. (committed over 20 years ago)

:), seems to be working well enough.

yeah

iiuc, so this code first fetches without VA and insert that in the DB and later fetches VA. if a VA is found, it will overwrite earlier results?

[musicbrainz-server] 14reosarevok opened pull request #2268 (03master…MBS-11968): MBS-11968: Allow VIAF, DNB, LoC links for series https://github.com/metabrainz/musicbrainz-serve...

its the exact opposite.

once one pair (<artist>, <recording>) gets accepted, that pair will never be accepted again.

so the VA albums recordings will only ever be accepted for pairs we don't already have.

I'm building this index on bono to run the tests again, to ensure that they still pass.

ah ok, makes sense. can you please comment this in the method and also that `1` represents VA. otherwise, LGTM

ok, thanks!

reosarevok, yvanzo : Should we highlight the URL when it's an old link but has a new relationship?

To me it seems useful, but the problem is how to handle it, should it be highlighted as newly added or modified?

[musicbrainz-server] 14reosarevok opened pull request #2269 (03master…MBS-11960): MBS-11960: Also assign seeded URL to rawURL https://github.com/metabrainz/musicbrainz-serve...

yyoung[m]: ^ do make sure this seems fine to you when you have some time, please :)

yyoung[m]: hmm, can't we just highlight the relationship(s)?

Sure

[listenbrainz-server] 14mayhem merged pull request #1618 (03master…include-all-releases-in-mapping): Add all releases (not use non VA) into the mapping https://github.com/metabrainz/listenbrainz-serv...

So, if the user changes the existing one, mark yellow, if the user removes, then red, if adds, then green? Or something :)

Ideally that would be better, but select boxes are not suitable for highlights

But we also have "type: ", right?

Why not add a background to the whole div?

Or what about changing the background of select, like the red one before?

Highlighting the whole div probably works though.

Either works IMO, but the div might be more visible

OK then, I'll have a try

And what about highlights of URL?

ruaok: looking at sitewide stats in spark, it seems the sitewide one is calculating the top artists for each day in past 2 weeks, each day in past 2 months, each month in the past year 2 years and each year since the start. this looks like an overkill to me. we do this sort of stuff during user listening activity calculation which makes sense but for user top artists we do not.

Well, I'd expect to highlight what has changed

So if the URL has changed, highlight that, if the relationship, highlight that

i think we should not do it for sitewide top artists either. thoughts?

If all uses of a URL are being removed, maybe highlight the whole block in red?

But I'd have to see it to be sure it'd work :)

Yes

!m reosarevok

You're doing good work, reosarevok!

thanks for fixing those bugs I submitted <3

Np :)

Not sure when it'll get merged, but I'm hoping release after next (so, beta in two weeks, out in three)

yvanzo, reosarevok : BTW, do you think displaying type combinations in a dialog is enough for MBS-11899?

MBS-11899: Document possible combinations of relationships to the same URL https://tickets.metabrainz.org/browse/MBS-11899

Do you mean having a bubble with documentation, or having a select with all the allowed combinations, or a third, different thing? :)

lucifer: not sure I follow the question. are you suggesting that we only calculate sitewide stats for all time and not the other ranges?

reosarevok: I mean providing a link in error message for users to view a list of allowed type combinations. My first thought is using a dialog since we don't favor bubbles now.

[musicbrainz-server] 14reosarevok opened pull request #2270 (03master…MBS-10056): MBS-10056: Don't add redundant credit to relationship on entity merge https://github.com/metabrainz/musicbrainz-serve...