#metabrainz

ruaok: let's see

Looks like it could be interesting? Some stuff I've listened to before, but I guess not this year, yeah

Most I haven't heard

[listenbrainz-server] 14MonkeyDo merged pull request #1761 (03master…stat-bug-fix): Remove unused props from user page https://github.com/metabrainz/listenbrainz-serv...

ruaok: monkey: akshaaatt: weekly year in music status check-in time?

Hi!

I'm in

Hello there :))

I'm here

Mind if I ask what you guys are working on 👀

something like spotify wrapped but for ListenBrainz.

Not a lot to report on my side. Started work on the year in music page, but mostly putting placeholder components and creating a header image

Didn't get much more done that that

I've added the component to image part in lb and we can now generate images of our cards for social sharing across various platforms

Sounds cool!

Yeah, that a great addition akshaaatt

I was also adding in some changes to the UI but got a little stuck with the responses, working on that currently

Is there anything that i can help you guys with? I have been playing a lot with my scrapped last.fm scrobbles on jupyter notebooks

I was hoping to implement it somewhere as well

To begin with PrathmeshG, I suggest you pick some relevant tickets from https://tickets.metabrainz.org/ and discuss any potential work you might have to suggest :)

oh that sounds nice. although it might be a bit late to get it implemented in prod for this year's year in music. but we are always looking for cool additions to LB in general. maybe there's something we could add to that.

lucifer: I guess we'll need to coordinate to make a transition from back-end to front-end, mainly putting the relevant data in the YIM page props

Sure, I'll try to make a ticket once I am a little confident haha

Welcome PrathameshG ! Interested to see what you've been working on

Coool, I'll try sharing some of it over ehre

monkey: sure, lmk what you need. the api was on beta till some days ago before i replaced it with another PR. we can make year in music PR permanently appear on test or beta for the next few days.

Uploaded file: https://uploads.kiwiirc.com/files/b3659fab2a874...

Scrapped some data off of last.fm, and cleaned it up into a very simple dataframe

Uploaded file: https://uploads.kiwiirc.com/files/4bf63535a59c7...

I guess the part I'm not going to want to touch are fetching the necessary data for the initial props load. I can figure out the API calls for updates on the front-end

yvanzo: how's that fix looking? I'll start working on a blog template and whatnot at least :)

we were planning to send all data in one api call, no?

Yuss

AFAIR yes

Sorry for spamming, last image incoming:

Uploaded file: https://uploads.kiwiirc.com/files/5f4243bfbdfaa...

I meant more initial load of page props from flask endpoint rather than from API call

i see, what should those props include? (other than the global context stuff)

At first glance, everything we would return from the YIM API call

Same format makes sense

PrathameshG: i suggest you to take a look at our incremental dumps, we have them in jsonl and parquet format. the jsonl ones are actual data submitted by users. and the parquet ones are subset of those data we currently use to generate stats.

monkey, sure can do that.

Hi everyone! This is Anshika from DTU currently in my 3rd year of Computer Engineering. I am very fond of contributing in open source projects. Looking forward for my participation in 2022.

Welcome Anshika581 !

lucifer: thats exactly what I was looking for! Mind linking it?

Welcome Anshika! You finally made it here :)

Anshika581 Nice to meet you!

:)

PrathameshG: https://data.musicbrainz.org/pub/musicbrainz/li...

each dir contains dump of listens for one day.

the spark dump ones are parquet and the other ones are jsonl.

Hmmm I'll try doing something with the JSON ones

The project that I shared above was all in JSON as well

is there anything specific that you might want me to do? e.g. I can write scripts to clean up and separate data

Cool, thanks lucifer I think with that I can start plugging in the data and we can see what we're missing

Sounds good to me as well monkey

PrathameshG: feel free to share your findings and ask for help in case needed here. the documentation can be a bit thin at places.

Surely, I'll try something out and let you know :))

ruaok: discovering some good tunes in that playlist! Mission accomplished!

I was also planning on using the Musicbrainz API to infer more from the existing data as well. Worried about the rate limiting tho.

PrathameshG: i don't think we need anything particular at the moment, we have a pipeline of stats ready and i would consider the parquet dumps to rather cleaned well already.

btw ruaok is not around currently, maybe ask him/talk to him as well but other than that a good starting point would be looking at existing tickets as akshaaatt suggested earlier.

Hmmm alright. ig I'll try browsing the tickets as well and come up with something

I am a bit new to all this so might take a little while to get used to I guess

sure no worries. sounds good.

Thanks for being this accomodating :))

We're here to help PrathameshG !

monkey: i was thinking that sending the YIM data in props could make the page load a bit slow because YIM has a lot of data (although we can remove the unneeded data, once we are clear on what frontend needs) but then this page is going to be heavy to laod anyways (i think?) so perhaps fine.

I think in general we're going to slowly have to move towards passing initial props for all pages, to ease moving towards server-side rendering when the time comes

ah ok, sounds good then.

At which level do you anticipate it'll slow down? Back-end? Front-end?

frontend.

I don't think parsing more props from the page will have significant impact

And I'm guessing it's faster than having to call the AI and do the same parsing anyway.

API*

makes sense. yeah i have been playing with some large json files today so probably overthinking this. in general efferct should be not noticeable.

!m lucifer with all the json work!

You're doing good work, lucifer with all the json work!!

hi PrathameshG! Welcome to the community :D. Excited to see your work!

thanks a lot! riksucks excited to be a part of the community as well :))

yvanzo: post preview at https://wordpress.com/post/blog.metabrainz.org/...

btw i forgot to mention my update for YIM, i dumped the data for AB and transferred it to michael. however am not sure on what stat to build out of it. we discussed weekday vs bpm but there are a few issues with it. i was hoping to enlist alastairp's help on that but since he is unavailable currently will either ask ruaok's help or drop AB stat plans.

sorry, I keep forgetting our extra meeting.

but I've been sharing my results frequently, so I am guessing you all know what I've been working on.

with the last playlist being a hit, so it seems, I'm nearly done with my work.

yup i think all of your playlists are done now?

nice :D

I only need write code to list the playlists for each user in the YIM stats table.

yes, all one. I made the decision to pivot on the last one, I was expecting bad results. now we have one with good results.

which means that I can help you tomorrow on the AB front, lucifer

i see, but we currently don't have any way for troi to connect stats table.

awesome! :D

yeah, I haven't worked out those details yet.

maybe dump to a json/csv from troi and have a script run on lb backend insert in yim table.

I might write a csv file and then a short script to import the data to stats

lol. great minds!

:D

reosarevok: thank you, looks good to me but the TBD :)

heh. by listening to the top missed tracks, I'm influencing my listening future.

I didn't think I had to consider time travel downsides when making playlists

wait what?

explain

I'm listening to stuff that I haven't listened to yet. so when this playlist runs for real, it will have been affected by the tracks I listened to just now.

reosarevok: let's release musicbrainz-docker without the patch (still WIP), the issue was just servers being temporarily unavailable.

yvanzo: oh, perfect

ruaok: oooh, heh

<BANG>

It’s International Monday of Solidarity with the Palestinian People!

People up for reviews: bitmap, akshat, ruaok, lucifer, reosarevok, alastairp, zas, yvanzo, monkey, CatQuest, Freso – anyone else who want to give review, let me know ASAP!

bitmap: Go!

hi

I was sick last week so was mostly resting and didn't get much work done. feeling a lot better today

yay

not much to report otherwise. fin, go yvanzo!

Hi!

Reviewed some MBS pull requests

Fixed a small bug in new MBS deployment script

Investigated load issue with dump builders on aretha

Fin. Go reosarevok!

Hi!

I did quite a bit of work on updating old PRs and improving them, and finally improving our autoselect support for Apple Music links now that we can use multi-types (thanks again yyoung[m])

And I also spent quite a bit of time looking at a ton of genre lists and whatnot for alastairp :)

Currently preparing for a prod server release, and that's about it from me

CatQuest: what about you?

Did several more summit tickets!

!m CatQuest

You're doing good work, CatQuest!

but ya'll cna do the rest now, I'm done :P

:D

The next four weeks I'll be busy doing various preparations, baking cookies, etcetra, so you'll see less of me :3

♏

fin, go zas

Hey

(People still up: akshat, ruaok, lucifer, alastairp, monkey, Freso – anyone else who want to give review, let me know ASAP!)

I continued my research on new gen gateways, started to write this down

I also worked on Picard, mainly code cleanup, we should release another beta tomorrow

Saturday there was an issue with github, and I noticed gitzconsul was creating zombie processes

(btw, this issue was also existing with git2consul)

so I investigated and patched it. The issue was very tricky, some subprocesses of git weren't properly killed on timeout

I also investigated aretha load issue with the help of yvanzo, solutions are on the way

!m zas