last week I helped zas deploy rakim. This didn't really require me to do very much at all, which is a good sign as it indicates the Ansible playbooks and roles are working consistently.
2022-03-14 07341, 2022
zas
they do ;)
2022-03-14 07303, 2022
monkey
Nice!
2022-03-14 07305, 2022
atj
I also added a new role to manage the Shorewall6 IPv6 firewall and deployed it on aphex, floyd, gaga and rakim
2022-03-14 07304, 2022
atj
then I began the process of creating new metabrainz git repos for each role, complete with CI for lint and functional testing.
2022-03-14 07350, 2022
atj
that process is still ongoing but I'm getting there.
2022-03-14 07359, 2022
atj
CatQuest: go!
2022-03-14 07306, 2022
CatQuest
Meow! `😺`
2022-03-14 07312, 2022
CatQuest
Have been creating tickets for bookbrainz, and having many fruitful discussions with monkey and shubh about bookbrainz and tabviews and so on.
as well as continuing work on recorders, pictures and adding aliases ♒
2022-03-14 07343, 2022
CatQuest
fin, go reosarevok
2022-03-14 07349, 2022
reosarevok
Hi!
2022-03-14 07302, 2022
Freso
(People still up: lucifer, akshaaatt, alastairp, bitmap, mayhem, Freso – anyone else who wish to give review, let me know ASAP!)
2022-03-14 07329, 2022
reosarevok
I did a lot of test improving and documenting, finally merged the PR for porting edit lists to React, and worked on a few smaller things
2022-03-14 07341, 2022
reosarevok
Also started implementing small schema change tickets
2022-03-14 07353, 2022
reosarevok
(which I'll keep doing today / tomorrow)
2022-03-14 07359, 2022
reosarevok
That's about it. Freso?
2022-03-14 07309, 2022
atj
Freso: aerozol is here!
2022-03-14 07310, 2022
Freso
🙋
2022-03-14 07328, 2022
Freso
atj: If they tell me they’re going to give review, I’ll add them. :)
2022-03-14 07338, 2022
CatQuest
morena!
2022-03-14 07344, 2022
atj
ok, noted :)
2022-03-14 07349, 2022
aerozol
Oh nope no review, just listening in :D
2022-03-14 07311, 2022
aerozol
CatQuest: morena <3
2022-03-14 07352, 2022
v6lur joined the channel
2022-03-14 07320, 2022
Freso
Spent a bunch of time dealing with reported editors and also spent some time with forum admin’ing related to GSoC, such as hiding old applications and also updating the application template on the forum to reflect the newest one in the wiki/docs.
2022-03-14 07346, 2022
Freso
Other than that, being around and about etc.
2022-03-14 07347, 2022
Freso
fin.
2022-03-14 07354, 2022
Freso
lucifer: Go!
2022-03-14 07305, 2022
lucifer
hi all!
2022-03-14 07303, 2022
PrathameshG joined the channel
2022-03-14 07317, 2022
lucifer
In the first half of previous week, I worked with alastairp on upgrading our Postgres/Timescale installation on gaga. It was mostly smooth. After that, worked with mayhem on some tuning Collaborative filtering parameters for LB recommendations. other than that misc bugfixes and a couple of small UI PRs.
2022-03-14 07341, 2022
lucifer
also, misc discussions with LB team on various upcoming features. fin. alastairp next?
2022-03-14 07346, 2022
alastairp
hullo
2022-03-14 07304, 2022
alastairp
I merged a few of Ansh's PRs to CB and deployed them, thanks Ansh!
2022-03-14 07300, 2022
alastairp
I also worked on some upgrades and bug fixes in CB, during the process I found a few bits of templates that had stopped working due to backend changes so I fixed them again and also made some changes to BU MB database access to add some more data which is needed by CB and is currently missing
2022-03-14 07341, 2022
alastairp
then as lucifer said, we upgraded postgres on gaga. it mostly involved waiting for things to finish and then either breathing a sigh of relief or panicking ;)
2022-03-14 07304, 2022
PrathameshG
Hey mayhem, can I get some info on the upcoming ListenBrainz MLHD cleanup & Spotify Cache projects?
2022-03-14 07305, 2022
PrathameshG
I've been working on some pretty similar stuff recently, so I'd love to dive into such projects at a larger scale :)
2022-03-14 07308, 2022
alastairp
I learned a few things about the upgrade process that we can keep in mind in case we have to do this in the future
2022-03-14 07328, 2022
alastairp
thanks also to atj for helping me find a fast compression tool!
2022-03-14 07337, 2022
alastairp
akshaaatt: you're up
2022-03-14 07339, 2022
mayhem
PrathameshG: sure, but lets us let the community meeting finish for now.
2022-03-14 07343, 2022
akshaaatt
Hi everyone!
2022-03-14 07348, 2022
CatQuest
\o
2022-03-14 07358, 2022
akshaaatt
Last week I continued work on the MusicBrainz revamp work and made quite some progress by fixing PRs and adding new sections.
2022-03-14 07308, 2022
Freso
(Only bitmap and mayhem left on my list. Last call for anyone else who wish to give review!)
2022-03-14 07311, 2022
akshaaatt
On LB side of things, I continued work on the bootstrap v5 PR and made updates to get the changes made.
2022-03-14 07312, 2022
PrathameshG
mayhem: sure, please ping me if possible once you're free!
2022-03-14 07325, 2022
akshaaatt
I worked on implementing a new look to the android app to welcome Jetpack compose and introduce bottom navigation.
2022-03-14 07347, 2022
akshaaatt
I discussed some new ideas for LB and MB Android App, most of which have now been added to the ideas page.
2022-03-14 07316, 2022
akshaaatt
Also, I connected with prospective students over the week and guided them on getting ready for GSoC.
2022-03-14 07323, 2022
akshaaatt
That's about it for me. fin!
2022-03-14 07327, 2022
akshaaatt
Go mayhem!
2022-03-14 07334, 2022
mayhem
hey!
2022-03-14 07346, 2022
mayhem
last week was super hectic in real life and that too huge amounts of my time.
2022-03-14 07355, 2022
mayhem
*took
2022-03-14 07312, 2022
CatQuest
that too took huge amounts of time
2022-03-14 07316, 2022
mayhem
so, I did PRs and the usual background MeB work and other things...
2022-03-14 07354, 2022
mayhem
I did create a postgres based fuzzy artist_credit recording lookup, which might be a good way to simplify our mapping. I'll continue on that.
2022-03-14 07304, 2022
alastairp
🎉 real life
2022-03-14 07324, 2022
mayhem
then I also worked with lucifer on tweaking our collaborative filtering and the results are much better already. there is much more to be done, but the results are no longer :facepalm:
2022-03-14 07328, 2022
mayhem
fin.
2022-03-14 07330, 2022
mayhem
bitmap: go
2022-03-14 07335, 2022
bitmap
hey
2022-03-14 07351, 2022
bitmap
last week I worked on migrating the redis store on yehudi to rakim, & implemented some changes to MBS to allow us to make the move without any downtime https://github.com/metabrainz/musicbrainz-server/…
2022-03-14 07307, 2022
bitmap
this will also be useful if we want to move to a new redis/keydb cluster without downtime
2022-03-14 07316, 2022
mayhem
bye aerozol
2022-03-14 07327, 2022
bitmap
I also did a fair amount of code review and worked a bit more on the new artist credit editor code I mentioned last(?) week - still trying to integrate it properly in the current release editor so it keeps focus and doesn't flicker when you move between tracks, but javascript is being annoying
2022-03-14 07333, 2022
bitmap
fin!
2022-03-14 07354, 2022
monkey
>javascript is being annoying
2022-03-14 07357, 2022
monkey feels you
2022-03-14 07304, 2022
CatQuest
I was gonna say..
2022-03-14 07305, 2022
CatQuest
:D
2022-03-14 07329, 2022
bitmap
:)
2022-03-14 07332, 2022
bitmap
back to Freso I think
2022-03-14 07353, 2022
Freso
Yeah, no one else left on my list. :)
2022-03-14 07304, 2022
Freso
Thank you everyone who gave review!
2022-03-14 07317, 2022
Freso
And no more items on today’s agenda, so this also wraps up today’s meeting!
2022-03-14 07326, 2022
CatQuest
notbad.jpg
2022-03-14 07334, 2022
CatQuest
28 minutes, i think that's a new record
2022-03-14 07337, 2022
Freso
Thank you all for your time, and please remember to stay safe! 😷
2022-03-14 07341, 2022
Freso
</BANG>
2022-03-14 07344, 2022
yvanzo
Thanks Freso!
2022-03-14 07347, 2022
CatQuest
<3 everyone
2022-03-14 07351, 2022
monkey
Thanks !
2022-03-14 07355, 2022
akshaaatt
Thank you!
2022-03-14 07357, 2022
TOPIC: MetaBrainz Community and Development channel | MusicBrainz non-development: #musicbrainz | BookBrainz: #bookbrainz | Channel is logged; see https://musicbrainz.org/doc/IRC for details | Agenda: Reviews, Next meeting (Freso)
2022-03-14 07358, 2022
milosh joined the channel
2022-03-14 07309, 2022
atj
zas: if you need any new roles for rudi, e.g. for managing haproxy then let me know
2022-03-14 07309, 2022
Freso
PrathameshG: Meeting’s over now. :)
2022-03-14 07318, 2022
alastairp
thank you everyone
2022-03-14 07322, 2022
CatQuest
oh man we forgot a trick. tody is cetus
2022-03-14 07333, 2022
atj
I'm guessing we'll need to create a least one new group
2022-03-14 07339, 2022
CatQuest
the one day that the starsign cetus is in the zodiac :D
2022-03-14 07344, 2022
PrathameshG
Freso Appreciate the ping 👍
2022-03-14 07344, 2022
atj
I'll be around tomorrow to discuss
2022-03-14 07350, 2022
alastairp
oh, and I didn't mention that I recognise pi day under duress
2022-03-14 07301, 2022
CatQuest
wait pi day?
2022-03-14 07312, 2022
CatQuest
uh 14. 3 ?
2022-03-14 07313, 2022
alastairp
at least pi approximation day writes the numbers correctly
2022-03-14 07318, 2022
CatQuest
that seems ackwards :D
2022-03-14 07328, 2022
alastairp
and is accurate to the same number of decimal places
2022-03-14 07340, 2022
CatQuest
wich is pi approximation day
2022-03-14 07341, 2022
alastairp
CatQuest: yeah, exactly. 3.14 for some people like bitmap
2022-03-14 07349, 2022
alastairp
CatQuest: 22/7
2022-03-14 07355, 2022
CatQuest
:O cool
2022-03-14 07321, 2022
CatQuest
hmm.. 3,14159265358979 vs 3,14285714285714
2022-03-14 07338, 2022
alastairp
didn't say it was a good approximation
2022-03-14 07343, 2022
CatQuest
heh :D
2022-03-14 07331, 2022
PrathameshG
So, can someone kindly give me any details on the Listenbrainz projects that I mentioned above? (MLHD cleanup & Spotify Cache)
2022-03-14 07333, 2022
CatQuest
still find it weird that the 14th of the 3rds is somehow 3 14
hi PrathameshG, what kind of things are you currently working on?
2022-03-14 07304, 2022
PrathameshG34 joined the channel
2022-03-14 07351, 2022
PrathameshG34
Hey alastairp, I am currently working on a project that aims to scrape last.fm data through APIs, clean it up, and then do some further interesting stuff with it by combining it with more metadata from the spotify API
2022-03-14 07341, 2022
mayhem
well, that sounds very much like the MHLD projects.
2022-03-14 07343, 2022
alastairp
oh neat. MHLD fits right in there, then - have you had a look at the dataset yet?
2022-03-14 07347, 2022
mayhem
you know about that data, yes?
2022-03-14 07346, 2022
PrathameshG34
Sadly Lastfm only stores the first artist in the song, so a LOT of metadata is being missed out on at the moment. It doesn't even remotely relate to other rich data like production, composition, etc credits that could enrich the MusicBrainz DB as well.
2022-03-14 07317, 2022
PrathameshG has quit
2022-03-14 07339, 2022
PrathameshG34
TLDR, my plan mostly revolves around creating the best possible quality data with lastfm and spotify APIs and using it for my own analysis stuff, as well as using it to aid the musicbrainz DB
2022-03-14 07310, 2022
PrathameshG34
alastairp I tried taking a look at the data, but for some reasons I wasn't able to download it at all (some site issues ig)
2022-03-14 07357, 2022
PrathameshG34
mayhem yes, I have a bit of idea about the data as per the description on the MLHD site as well as some data dumps that you shared the last time I was active on this IRC (a few months ago)
2022-03-14 07341, 2022
mayhem
PrathameshG34: ok, cool.
2022-03-14 07351, 2022
mayhem
so, you know that the MHLD has some problems, yes?
2022-03-14 07357, 2022
PrathameshG34
Yep
2022-03-14 07303, 2022
mayhem
namely last.fm's MBID mapping system was.... not very good.
2022-03-14 07322, 2022
mayhem
we have two datasets that can help us:
2022-03-14 07326, 2022
PrathameshG34
Oh yes, I've noticed that a lot in my current project as well
2022-03-14 07308, 2022
mayhem
1. canonical recordings: this identifies all of the recordings MB that are canonical. (the first release of a track, ignoring re-releases, compilations, etc).
2022-03-14 07331, 2022
mayhem
2. recording redirects -- when a recording MBID is retired, we add it to a redirect table in MB.
2022-03-14 07344, 2022
mayhem
so what we should do in this project is this:
2022-03-14 07350, 2022
mayhem
1. Download a MLHD chunk.
2022-03-14 07354, 2022
mayhem
2. Load it into spark.
2022-03-14 07315, 2022
mayhem
3. Remove rows that are incomplete (e.g. no recording_mbid, toss it)
2022-03-14 07340, 2022
mayhem
4. Check for redirects (was a recording mbid retired, if so, us the modern one)
2022-03-14 07311, 2022
mayhem
5. Canonicalize the track. If it is not a canonical recording, replace the recording_mbid with the canonical one.
2022-03-14 07341, 2022
mayhem
6. Compute artist and recording popularity (as a nifty by product of this page -- its a simple histogram operation)
2022-03-14 07357, 2022
mayhem
7. Re-write new data chunks with the updated data.
2022-03-14 07320, 2022
mayhem
Then we repeat this process for all the chunks.
2022-03-14 07336, 2022
PrathameshG34
Epic
2022-03-14 07336, 2022
PrathameshG34
Sounds extremely interesting, and very doable as well
2022-03-14 07340, 2022
mayhem
and in the end we combine the artist/recording popularity data for each of the chunks.
2022-03-14 07347, 2022
mayhem
ok, kewl.
2022-03-14 07301, 2022
mayhem
now, we're not certain that this will work well.
2022-03-14 07315, 2022
mayhem
it might work well to clean up the data. it might not.
