#metabrainz

ansh: there is a separate docker-compose file for spark, and developer docs are at: https://listenbrainz.readthedocs.io/en/latest/d... . you can remove the two ports here https://github.com/metabrainz/listenbrainz-serv...

i haven't used spark on wolf in a long time so if you face any issues lmk

lucifer: I removed the ports, and updated the network's name here https://github.com/metabrainz/listenbrainz-serv... to listenbrainz_ansh_default.

I'm getting the error https://www.irccloud.com/pastebin/MwVAZttL/

ansh: did you make any changes to project name when running the containers?

main LB web containers i mean

yes, I updated the name to listenbrainz_ansh here: https://github.com/metabrainz/listenbrainz-serv...

change listenbrainzspark to listenbrainz_ansh as well everywhere in that file

It did not work

it should be listenbrainz_ansh not listenbrainzspark_ansh

ah actually nvm, you need to change the network to musicbrainz-docker_default

because your other containers are running ther

worked

I have now imported the data into spark. I don't have an idea on how spark works exactly, but how will the recommendations come up? Like will it process previous listen history? Or will it work on new listens

imported the data - means you imported listen dumps right?

Yes, I followed this step: https://listenbrainz.readthedocs.io/en/latest/d...

cool you can now run `./develop.sh manage spark cron_request_recommendations`

and spark cluster should start working on recommendations.

but you want personalised user release recommendations right?

Yes, i want personalised recommendations using this API https://api.listenbrainz.org/1/user/anshgoyal31...

for that you should call, `./develop.sh manage spark request_fresh_releases --days NUMBER_OF_DAYS`

okay, so I've got a couple of errors now.

After running `./develop.sh spark up` https://www.irccloud.com/pastebin/p0Xw05v6/

can you try running the incremental import again

seems like that didn't work/run

Yes, now it is processing the releases

And the endpoint is now working. Thanks lucifer

no worries, nice

bitmap: I started looking at https://tickets.metabrainz.org/browse/MBS-6741 (IIRC one of the plans we had by adding the area_containment table was to be able to implement something like this) but I'm not sure what'd be a good way of doing it :) Maybe you have time at some point to brainstorm it a bit?

MBS-6741: Include subdivision and country in the webservice when returning an area

I was thinking of an inc= for this, but we're not even passing includes when serializing the artist areas for example :)

Would it make sense to just do it always?

Hey reosarevok I was wondering if I'd really need to setup the full version of musicbrainz-db for this particular project?

The only issue is that I don't have a ton of storage on my laptop. Plus, I wont have access to wifi for a week, starting from today 6PM to download the DB🥲

If it's necessary, I can try to get it downloaded somewhere on an external HDD maybe

Hmm. Wonder if it would be possible to make a sample + areas dump

Since you clearly don't need all the recordings for example, just areas and urls and l_area_area plus l_area_url I guess

_exactly_

If there's a way, I would be happy to do it.

OR if possible, we could maybe setup a mirror on one of MeB servers. Especially given the fact that the actual bot would be running the same way

Yes, that's also an option

Ideally we'd eventually do that, anyway, but a dump wouldn't hurt...

There's the JSON dumps where area data is all separate, but given we want to run this from the DB, it might not make sense to develop using the JSON dump, even as a test :)

That said, it's only 32 MB for the areas so you should have a copy just in case

reosarevok: it should be simple to dump the tables with COPY command or pg_dump fwiw. if you want i can do that

That'd be nice - I *think* the following list would be enough:

(ideally we'd filter link to only l_area_area and l_area_url linked entries to make it smaller, not sure how big that is... but maybe it's not too bad)

lucifer: yes, it would be a lot of help if you could help me make the dumps.

Here's the steps that are apparent to me:

Log onto wolf, fire up psql, and find a command to dump the tables reosarevok mentioned. Then I can just copy the dumps locally with sshcopy

Do I have to do anything else apart from this?

Pratha-Fish: `pg_dump postgresql://musicbrainz:musicbrainz@127.0.0.1:5432/musicbrainz_db -a -t area -t area_type > dump.sql` so on, before each table name you have to specify -t

then yes copy it to using sshcopy and import in your database

lucifer: thankyou, just did that!

^made a little bashscript to run it in fact

awesome

looks like it created a 1GB dump. we're good to go 😁

[listenbrainz-server] 14amCap1712 merged pull request #2581 (03master…apple-cache): Fixes for running apple metadata cache https://github.com/metabrainz/listenbrainz-serv...

Neat!

Hopefully it imports well :)

[listenbrainz-server] 14amCap1712 merged pull request #2584 (03master…mapping-config-fix): Remove the config file from mbid-mapping Dockerfile https://github.com/metabrainz/listenbrainz-serv...

reosarevok: yess, I'll get it verified as soon as I return from lunch

Yum, yum... a difficult choice https://usercontent.irccloud-cdn.com/file/ImXD2...

pretty much none of those!

we can buy most that here.

Ritter Sport would be the local manufacturer

YES!

Aye!

Loads please, esp the less-common ones.

and I would like to bring chocolate to the mentor summit -- but I won't have a chance to go to germany before thne.

I haven't seen the Lindt Creation ones before, wouldn't mind trying some, but if they're available in BCN I can just buy them there :D

could I pester you to bring another 1-2Kg for that purpose??

Wait, I just realized a thing

you're going through germany, reosarevok ?

FRA? or MUC?

Looking at the summit page, *I* am supposed to share a room with kellnerd before the main summit starts

All the chocolate is going to be there

That seems unsafe

👀

meh, I was trying to get you to buy more chocolate in germany.

I know, I can get to that

I *am* flying through Germany

I could buy some chocolate there if the airport shops have it. Some might even arrive.

the options at FRA are not nearly as good as at MUC.

FRA, sadly

yeah, FRA doesn't have a convenient supermarket to go to. inside security is all the same crap -- hard to find anything good.

but still, load up. :)

kellnerd: auch milka und ausgefallene milka sorten, bitte!

Sure, I will go to a bigger supermarket and have a look there.

❤️

when in doubt, get more geeks from all the world are counting on you!

Milka is not available in BCN or only the standard flavours?

standard flavors.

some fancy ones, but often its too sweet what they offer. simpler bars with just nuts and so, are just bog standard varieties.

kellnerd later this week is like

I usually eat only stuff with at least 50% cocoa but I will also bring some milk chocolate for those who prefer it sweet.

milka zartherb!

and yes, for the GSoc Summit bound batch, make it 50/50 milk/dark.

No white? dats racist

aerozol: Do you eat regular chocolate or should I have a look for vegan chocolate as well?

(sure, I know that's *barely* chocolate, but)

only Green & Blacks white with vanilla bits.

I don't think I've tried that, but sounds promising

we should see if atj can find it and bring some. its just about the only white chocolate worth eating.

Ok, so I can bring a 23 kg case and an 8 kg hand luggage. Not sure how many kg of chocolate I will fit in there, it's been a while since my last flight and I don't remember the weight of my case for a week.

mayhem: https://www.sainsburys.co.uk/gol-ui/product/gre... ?

I will definitely fill it up to the limit and in doubt there will be less chocolate left for the GSoC mentor summit after the MeB summit :P

atj: THAT. Grab 10 bars, please. I'll happily reimburse.

kellnerd: great! when in doubt. grab more.

i feel like i should bring some UK sweets that you'll all love, like Fisherman's Friend's

mayhem: Do you know an approximate number of how much chocolate "disappeared" last summit?

our summit or the gsoc one?

Our summit

for us, 15-30 bars maybe?

Sorry I am uninitated, but what's up with the GSoC summit?

Is it something by MeB or by Google?

GSoC holds a summit in California after each round for select mentors to attend.

and lucifer and I are going. lucfier is there to represent MeB and I am there to make the chocolate table happen.

this is what a good chocolate table looks like: https://photos.app.goo.gl/LxKw4RNipvyNw74Y9

ours are obviously more modest.

Damn! That looks delicious 😋

it is amazing.

I wish they did more of these for students as well... But nah, they're in talks of ending GSoC 💀

Seeing all the hype for GSoC in India lately, I really wish they just end up expanding it instead, or at least hold local meetups 🫠

its not the GSoC team's choice. they would. but the beancounters at google dont understand GSoC because it has no measurable ROI, so it must go away.

mayhem: When would you be visiting India again? :D

Ok, so the lower chocolate bound is 15 * 0.1 kg + 1 kg = 2.5 kg and the upper bound is 30 * 0.1 kg + 2 kg = 5.0 kg 🧑‍🔬

I'd argue GSoC has negative ROI for google at this point 💀

Pratha-Fish: there will be a MeB mini-summit in Delhi Nov 17/18.

mayhem: noiceeee. I'll try my very best to attend :))

kellnerd: make the upper bound 6Kg. :)

I need to start planning the Delhi summit too. too many summits, really. :)

mayhem: what do you like to eat? I'll see if I can bring something from Pune. It's a different state all together so it has some variety in food as well hehe

Pratha-Fish: yes. the more off the beaten path and spicy! the better. :)

I can already visualize misal, but I doubt it would last 🫠

This is what misal looks like. Infused with spices to death. Super delicious tho https://usercontent.irccloud-cdn.com/file/PGPgV...

THAT has my name written all over it!