Alright theflash__ (IRC) let me know if you need help
2024-06-17 16938, 2024
Sophist-UK has quit
2024-06-17 16944, 2024
Sophist_UK joined the channel
2024-06-17 16922, 2024
pranav[m] joined the channel
2024-06-17 16922, 2024
pranav[m]
Hey aerozol , I was offline yesterday, Ill go through the replies today... thanks
2024-06-17 16903, 2024
Sophist_UK has quit
2024-06-17 16944, 2024
Sophist-UK joined the channel
2024-06-17 16936, 2024
aerozol[m]
np!
2024-06-17 16954, 2024
saumon has quit
2024-06-17 16910, 2024
saumon joined the channel
2024-06-17 16905, 2024
zerodogg has quit
2024-06-17 16904, 2024
pite has quit
2024-06-17 16922, 2024
zerodogg joined the channel
2024-06-17 16912, 2024
Kladky joined the channel
2024-06-17 16952, 2024
yellowhatpro[m] joined the channel
2024-06-17 16952, 2024
yellowhatpro[m]
<yvanzo[m]> "It would be tricky to implement..." <- yvanzo: bitmap https://github.com/yellowHatpro/mb-exurl-ia-servi…... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/download/chatbrainz.org/pitzXodtsZAXYMAbUsMhreMe>)
yvanzo: can you check sir queues? We still have many entries in search failed, and it keeps growing. Not sure how to interpret this.
2024-06-17 16958, 2024
pranav[m]
aerozol: re playlists, I would first like to complete the core features of the user pages first and then do the playlists tab
2024-06-17 16949, 2024
zerodogg has quit
2024-06-17 16948, 2024
pranav[m] uploaded an image: (1141KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/bzzKuSEfpBLKXrytiJYvdvJe/Screenshot%202024-06-17%20at%203.01.29%E2%80%AFPM.png >
2024-06-17 16906, 2024
pranav[m]
aerozol: Could u lemme know what is to be done when user is already followed? Just need color and text of that button
2024-06-17 16945, 2024
rimskii[m]
reosarevok: hi! I wanted to ask if you can clarify some things regarding spotify lookups ( I remember you mentioned that we can ask you for help regarding MusicBrainz if needed.)
2024-06-17 16945, 2024
rimskii[m]
I’ve checked the spotify_lookup_from_mbid api and it seems it works by looking for an exact track match using the MBID to fetch the Spotify ID. Is that correct? Does that mean that each recording in MB contains spotify id?
2024-06-17 16901, 2024
rimskii[m]
I was thinking how can I implement the same thing for apple music for example
2024-06-17 16942, 2024
zerodogg joined the channel
2024-06-17 16922, 2024
d4rk joined the channel
2024-06-17 16922, 2024
d4rkie has quit
2024-06-17 16914, 2024
reosarevok[m] joined the channel
2024-06-17 16914, 2024
reosarevok[m]
Hi! That wasn't written by the MusicBrainz team I think so I'd have to check, can you point me to the code?
2024-06-17 16919, 2024
mayhem[m] joined the channel
2024-06-17 16919, 2024
mayhem[m]
lucifer: please let me know when you're about, I'd love a quick catchup regarding spotify playlists and stats issues.
If you're not sure, no worries at all! I guess it's outside of MB
2024-06-17 16955, 2024
yvanzo[m]
Hi zas, on it.
2024-06-17 16939, 2024
mayhem[m]
yvanzo: zas can one of you catch me up to the current status of the solr cluster? is there anything I can do to help it become more stable?
2024-06-17 16917, 2024
yvanzo[m]
mayhem, zas : SIR is currently indexing documents at full speed again.
2024-06-17 16907, 2024
mayhem[m]
have we switched to the new cluster yet?
2024-06-17 16913, 2024
reosarevok[m]
rimskii: I think mayhem is the person for this :)
2024-06-17 16928, 2024
mayhem[m]
sorry for not keeping up, but last week sleep was utter hell. I feel somewhat normal today.
2024-06-17 16944, 2024
rimskii[m]
reosarevok[m]: okay thanks ! :)
2024-06-17 16944, 2024
rimskii[m]
Sorry for bothering you
2024-06-17 16954, 2024
yvanzo[m]
mayhem: Not yet, we want to do it these days if not today. But we might have to keep the old cluster in parallel for a month or so.
2024-06-17 16921, 2024
mayhem[m]
ok, that's good. so all the problems, in theory, will go away soon? 🤞
2024-06-17 16914, 2024
mayhem[m]
rimskii: let me see if I can help you. we do not have 1:1 mapping of MBIDs to Spotify IDs.
2024-06-17 16921, 2024
mayhem[m]
(we wish, but alas no)
2024-06-17 16945, 2024
rimskii[m]
<rimskii[m]> "reosarevok: hi! I wanted to..." <- > <@kubrimskii:matrix.org> reosarevok: hi! I wanted to ask if you can clarify some things regarding spotify lookups ( I remember you mentioned that we can ask you for help regarding MusicBrainz if needed.)... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/download/chatbrainz.org/lhiRwTzJpGENUzhYrHkgAwiJ>)
2024-06-17 16921, 2024
mayhem[m]
yes, sure.
2024-06-17 16934, 2024
rimskii[m]
i say you already answered haha
2024-06-17 16939, 2024
rimskii[m]
s/say/saw/
2024-06-17 16949, 2024
mayhem[m]
the goal is to match a MBID to an apple music id, yes?
lets step through this and see if we can work it out.
2024-06-17 16944, 2024
mayhem[m]
the first two "with" statements do:
2024-06-17 16934, 2024
mayhem[m]
1. Resolve redirects: MBIDs can at times be deprecated and point to newer MBIDs. This step ensures that the given MBIDs are currently valid MBIDs (e.g. they are valid or were redirected)
2024-06-17 16901, 2024
zerodogg has quit
2024-06-17 16954, 2024
mayhem[m]
2. Resolve recording MBIDs to "canonical recording" MBIDs. A recording of a song may have more than one actual recording in MB, so this step ensures that we find the recording we consider to be "canonical". Canonical is important because we can find the metadata for each canonical recording easily in the metadata cache.
2024-06-17 16933, 2024
mayhem[m]
so in the for loop, we check the redirects, then the canonicals, then fetch the metadata.
2024-06-17 16952, 2024
mayhem[m]
all this was basically to take an MBID and resolve it to the correct metadata for that MBID.
2024-06-17 16904, 2024
mayhem[m]
because that is how the spotify data index is created.
2024-06-17 16921, 2024
mayhem[m]
then, finally the look up is done via the lookup_using_metadata(ordered_metadata) function.
<mayhem[m]> "the first two "with" statements..." <- well If im going to do such lookup for apple music should I do the same here too?
2024-06-17 16900, 2024
mayhem[m]
so in the utils module I just linked, it attempts to find the right spotify matches. and note that it does not do that with spotify ids, but with metadata lookups.
2024-06-17 16928, 2024
mayhem[m]
I think in the end you'll pretty much copy everything that was done for spotify to the apple setup.
2024-06-17 16941, 2024
rimskii[m]
haha ok
2024-06-17 16941, 2024
rimskii[m]
ok
2024-06-17 16900, 2024
rimskii[m]
if I remember correcly lucifer said we do have for apple music
so far it seems that all the code between apple and spotify is wholly separate, so copying the spotify code and modifying it for apple seems to be the right approach.
2024-06-17 16936, 2024
rimskii[m]
i see
2024-06-17 16926, 2024
rimskii[m]
So as far as I got, the spotify_lookup first finds the right mbid then searches for spotify metadata, right?
2024-06-17 16939, 2024
mayhem[m]
yes
2024-06-17 16927, 2024
rimskii[m]
mayhem[m]: spotify metadata is spotify_cache here, right?
2024-06-17 16937, 2024
mayhem[m]
yes
2024-06-17 16944, 2024
rimskii[m]
ahh i see
2024-06-17 16947, 2024
rimskii[m]
I got it
2024-06-17 16900, 2024
rimskii[m]
thank you soo much, appreciate it much !! :)
2024-06-17 16907, 2024
mayhem[m]
no problem.
2024-06-17 16939, 2024
lucifer
mayhem[m]: hi!
2024-06-17 16900, 2024
lucifer
rimskii[m]: will check and let you know in a bit.
2024-06-17 16919, 2024
lucifer
mayhem[m], outsidecontext: spark cluster was unavailable last week for a few days because of memory and storage issues apparently. i reset-ed the cluster on the weekend and did a fresh install, its processing requests as we speak. stats are done, recommendations and playlists queued.
2024-06-17 16908, 2024
lucifer
mayhem[m]: the deleteme hetzner can be removed now.
2024-06-17 16949, 2024
lucifer
i have setup the ansible playbooks for the spark cluster and used it to do the fresh installs.
2024-06-17 16959, 2024
mayhem[m]
Lucifer: one request -- when you expect downtime for the cluster, ping me please. So I can tweet/post about doing maintenance.
2024-06-17 16916, 2024
lucifer
mayhem[m]: it was unexpected.
2024-06-17 16928, 2024
lucifer
i discovered it when diagnosing why stats were not updating.
2024-06-17 16921, 2024
lucifer
after discovering it, i got to fixing it but that ended up taking over a day.
2024-06-17 16951, 2024
mayhem[m]
Ah, another case for more monitoring. But next time you do notice, please let me know
2024-06-17 16900, 2024
lucifer
yes sure
2024-06-17 16907, 2024
mayhem[m]
Deleteme server, deleted. Thanks.
2024-06-17 16954, 2024
lucifer
(to be clear, i discovered the memory and storage issues first and then reset it, not the other way around)
2024-06-17 16932, 2024
lucifer
btw we should be able to add new servers to the cluster with ease now.
2024-06-17 16901, 2024
lucifer
should be handy for trying out MLHD data processing stuff.
2024-06-17 16922, 2024
mayhem[m]
Oh yisss! Great news!
2024-06-17 16937, 2024
mayhem[m]
shall we do that after south or even before?
2024-06-17 16958, 2024
mayhem[m]
s/south/oauth/
2024-06-17 16919, 2024
lucifer
i guess after might be better
2024-06-17 16946, 2024
lucifer
have to also deal with LB gaga disks and typesense
2024-06-17 16909, 2024
zerodogg joined the channel
2024-06-17 16921, 2024
lucifer
Jigen, bitmap[m]: restored the data. thanks!
2024-06-17 16920, 2024
lucifer
volumetrique: yes, we currently don't consider track_mbids during mapping.
2024-06-17 16929, 2024
lucifer
but iirc mayhem[m] and i discussed previously to change that (not to consider track mbids in mapping but prefer track mbids if submitted over mapping assigned mbids), so feel free to open a ticket and i look into adding that later.
