why do we have SCHEMA_VERSION_CORE and SCHEMA_VERSION_TIMESCALE? My knee jerk reaction says that we need only one.
2021-11-15 31927, 2021
alastairp
there were previously 2 schema numbers, both named the same thing. this change only makes them explicitly different. however you're right - we discussed this mid-last week
2021-11-15 31936, 2021
alastairp
1 schema version number might be easier to manage. it means that a change in a public table in the pg database will cause the public timescale dumps to also increase
2021-11-15 31956, 2021
ruaok
I dont see an issue with that.
2021-11-15 31910, 2021
alastairp
in terms of real impact on other people... I'm not sure how many people are importing our public (non-listen) dumps anyway
2021-11-15 31922, 2021
ruaok
wild guess? zero.
2021-11-15 31944, 2021
ruaok
I'd love to be wrong. perhaps we'll meet someone who is doing that. that'd be nice too. lol
2021-11-15 31925, 2021
alastairp
while we're talking about this - I think we put these schema numbers in the private dump too, even though we say "This value must be incremented after schema changes on tables that are included in the public dump"
2021-11-15 31934, 2021
alastairp
I guess we've never tried to import a private dump either
2021-11-15 31948, 2021
ruaok
correct
2021-11-15 31935, 2021
alastairp
that is, if we make a private dump, then change the structure of a table, then don't update the schema number (because it's not public), and then dump again, we have 2 differently structured dumps with the same schema
2021-11-15 31955, 2021
alastairp
should we have PRIVATE_SCHEMA, PUBLIC_SCHEMA, LISTENS_SCHEMA then?
2021-11-15 31930, 2021
ruaok
that case feels contrived to me. we don't change schema numbers willy nilly. let's have one and define a policy for when to increment it.
2021-11-15 31908, 2021
alastairp
fine by me. Do you think it needs to be in this PR or should I open a ticket?
2021-11-15 31920, 2021
ruaok
the latter is fine.
2021-11-15 31925, 2021
ruaok
lets get this PR done
2021-11-15 31903, 2021
alastairp
lucifer: did you create a ticket for the discussion we had about disaster recovery of MeB services?
2021-11-15 31952, 2021
lucifer
alastairp: LB-899 is the one I had created before it. but didn't update with the discussion
ruaok: i was thinking to work on the spotify mapping stuff soon. i am thinking to create a table for recording_msid, spotify_album_artist_ids, spotify_album_id, spotify_artist_ids, spotify_id, isrc. thoughts?
2021-11-15 31956, 2021
ruaok
the schema makes sense, though I am not sure that ISRC fits here or should have its own table.
2021-11-15 31908, 2021
ruaok
where and how would you update the mapping?
2021-11-15 31947, 2021
lucifer
for the listens coming in now, i am thinking to run this query just after a match is found by the writer.
2021-11-15 31911, 2021
lucifer
if the mapper finds a match and the listen has this field, update the table.
2021-11-15 31936, 2021
ruaok
ok, makes sense.
2021-11-15 31942, 2021
lucifer
for the "legacy" listens, i was thinking add another column "processed_by_spotify"
2021-11-15 31955, 2021
lucifer
and then do what the writer does currently for match_type
2021-11-15 31932, 2021
lucifer
*processed_for_spotify
2021-11-15 31900, 2021
ruaok
I wonder if it needs legacy listen support stuff.
2021-11-15 31905, 2021
yvanzo
reosarevok: nothing to add, thanks
2021-11-15 31945, 2021
lucifer
do you mean we don't need to scan existing listens or we should use an alternative way to scan existing listens?
you're mapping against msids -- if that is correct, then you're just recording a match between spotify id and recording_msid.
2021-11-15 31904, 2021
alastairp
docs versioned by release tag
2021-11-15 31919, 2021
ruaok
which is actually already all present in the listens table. that data we can simply query for.
2021-11-15 31923, 2021
alastairp
at least we have up-to-date docs now, but I'm not sure I'm a fan of the url changing on every release we make
2021-11-15 31930, 2021
reosarevok
yvanzo: the blog looks ok too? :)
2021-11-15 31933, 2021
alastairp
lucifer: perhaps going back to merging to `production` is the best idea here
2021-11-15 31947, 2021
ruaok
if, however, you're more into recording a mapping between mbid and spotify, then it needs more thought.
2021-11-15 31953, 2021
lucifer
alastairp: sure, maybe we could the action do it for us :)
2021-11-15 31911, 2021
alastairp
lucifer: yeah, if the action can do that then I'm happy to do it that way
2021-11-15 31919, 2021
monkey
There's also the "latest" tag for the doc versions
2021-11-15 31930, 2021
yvanzo
zas, outsidecontext: I inserted a "read more" block in your last blog post (as discussed during the summit), please let me know if that causes any issue.
2021-11-15 31931, 2021
lucifer
ruaok: uh right, the intent is recording_msid gets us to the mbid through listen_mbid_mapping table.
2021-11-15 31948, 2021
lucifer
spotify -> recording_msid -> recording_mbid
2021-11-15 31954, 2021
ruaok
lucifer: then I think there is no need for this project at all, honestly.
2021-11-15 31904, 2021
alastairp
monkey: I'm not sure if latest is just a pointer to the most recent version, or if it's based on a git tag?
2021-11-15 31912, 2021
ruaok
perhaps a materialized view?
2021-11-15 31917, 2021
monkey
Most recent version I believe
2021-11-15 31900, 2021
lucifer
interesting... makes sense.
2021-11-15 31921, 2021
alastairp
monkey: right, it seems that you're correct. though we still have to see if we can make it work without filling the version popup with a million tags https://usercontent.irccloud-cdn.com/file/pJivcEC…
2021-11-15 31906, 2021
monkey
Ah, yes
2021-11-15 31913, 2021
alastairp
I added an action/trigger on readthedocs to activate this version based on the presence of a new tag matching a regex. it's possible that we can disable old ones with a similar trigger, or make it latest without "activating" it
2021-11-15 31931, 2021
lucifer
ruaok: thinking about it. i am liking the materialized view route but how do we keep timescale play well with it. the listen counts faced issues due to materialzed views right?
2021-11-15 31916, 2021
ruaok
well... cont aggs which are implemented with mat views.
2021-11-15 31943, 2021
ruaok
I guess the question is this: what is our use case for the mapping? To do content resolution for brainzplayer?
2021-11-15 31902, 2021
lucifer
yes that and feed data to MB
2021-11-15 31950, 2021
ruaok
we should try a materialized view and see how that behaves. then we can plan further.
2021-11-15 31958, 2021
lucifer
sounds good
2021-11-15 31908, 2021
yvanzo
reosarevok: Was MBS-12067 affecting beta only? If so, it doesn’t have to be in release notes.
outsidecontext: It's been “decided” to give it a try at least :) I suggested using it to ease browsing through latest blog posts.
2021-11-15 31920, 2021
outsidecontext
it makes sense, some posts are long and if you are not interested you need to scroll a long way for the next post
2021-11-15 31955, 2021
reosarevok
Yeah, and it's easy to click and see more :)
2021-11-15 31940, 2021
yvanzo
outsidecontext: For MBS, we use a draft as a template to create new posts (by clicking on “…” > "Copy post”), so we don’t have to remember how to do it. I just created a general-purpose template/draft that can be used or modified for Picard.
2021-11-15 31906, 2021
BrainzGit
[listenbrainz-server] 14MonkeyDo merged pull request #1731 (03master…monkey-MBIDs-in-UserChartEntity): Add missing MBIDs to UserChartEntity items for display https://github.com/metabrainz/listenbrainz-server…
2021-11-15 31904, 2021
outsidecontext
yvanzo: we also usually use an existing post to start with, depending on the release
2021-11-15 31927, 2021
tandy
@musicbrainz team, is there any plan to update the current genres?
2021-11-15 31900, 2021
tandy
its a shame that "City Pop", and a number of other genres aren't recognised ..
2021-11-15 31948, 2021
alastairp
tandy: you can propose for tags to get turned into genres, reosarevok takes care of that
2021-11-15 31903, 2021
tandy
oh awesome
2021-11-15 31910, 2021
alastairp
of course, you might find people arguing with you that City Pop isn't a real genre!
