Something in Lidarr broke because of the schema change, and their wording is kind of vague on whose “fault” it was. Sounds like people who rely on MB so heavily need to subscribe to the blog tbh. https://github.com/Lidarr/Lidarr/issues/5498
Not a big deal but thought I’d mention it.
reosarevok[m] joined the channel
reosarevok[m]
Back from hols. julian45 now available to test things too
reosarevok: can you please add all the GSoC contributors to weekly meeting reviews list (if not already there): m.amanullah7 , suvid , holycow23 , @fettuccinae:matrix.org and Hemang Mishra
(jade is already on the list)
reosarevok[m]
Sure!
lucifer[m]
thanks!
reosarevok[m]
Done :)
holycow23[m] joined the channel
holycow23[m]
<_BrainzGit> "[listenbrainz-server] amCap1712..." <- lucifer: thanks for this, could you share the dump and if possible guide me on how to go ahead with using the dump
lucifer[m]
holycow23: first of all setup the spark cluster locally by running `./develop.sh spark up` and then run `./develop.sh spark format` in another terminal.
then run ./develop.sh manage spark request_import_sample to import the sample dump.
after this you should import a few incremental dumps by running ./develop.sh manage spark request_import_incremental --dump-id DUMP_ID HERE, the dump ids should be available here: https://data.metabrainz.org/pub/musicbrainz/lis... (2123 - 2153).
this command may fail, if it does let me know and i'll try to fix that.
in future, i am thinking to include a bunch of listens in the spark sample dumps but there's a few things to discuss about it first.
holycow23[m]
Okay running the first 2 queries right now
snobdiggy292838 has quit
<lucifer[m]> "this command may fail, if it..." <- It ran without an error
lucifer[m]
okay, try to run some stats: ./develop.sh manage spark request_user_stats --entity artists --type entity --range all_time
holycow23[m]
Done, now do I start the server?
lucifer[m]
ensure that api_url in your config is set to localhost and start the server and you should see some stats.
you can take a look at logs of the request_consumer container to ensure it ran successfully
holycow23[m]
yes its set to localhost, testing the stats right now
I don't see the stats, and for some reason the container seems to have closed
lucifer[m]
try starting the container again and share its logs.
holycow23[m]
Do I try rebuilding the entire thing?
Also just noticed that ./develop.sh spark format was trying to ping Redis which didn't work
And can
* And can't see the spark container
snobdiggy292838 joined the channel
lucifer[m]
holycow23: run `./develop.sh up -d web`
then run the two spark commands again
holycow23[m]
When I run the second command it terminates the first one that's fine right?
And the spark containers have again vanished
snobdiggy292838 has quit
<lucifer[m]> "after this you should import a..." <- also I think this is `--id` and not `--dump-id`
lucifer[m]
holycow23: yes that is fine, you can start the spark containers again after format.
--id is fine
holycow23[m]
I did the same process but the spark container's aren't there
lucifer[m]
even after running ./develop.sh spark up again?
holycow23[m]
Yeah unless I am going wrong somewhere
Do I need to run the spark up command again
lucifer[m]
yeah try once again, the containers should start or you should see an error
do you see any stopped spark containers in docker ps -a
holycow23[m]
yes
holycow23[m] uploaded an image: (23KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/nwQqsMjrjKzpHGbpxygsUhew/image.png >
The first one
lucifer[m]
share its logs.
holycow23[m] uploaded an image: (36KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/WjcVYNSQrYCxZRQpERVcXuON/image.png >
rayyan_seliya123: hi! let me know when you are around to discuss your project.
<lucifer[m]> "holycow23: you need to create..." <- lucifer: I rechecked the process, re-did the entire process again the containers are running but I can't see the stats
lucifer[m]
holycow23: share the logs of the spark container
holycow23[m] uploaded an image: (58KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/aFtZWQgJYieKBlSnVHndqgFU/image.png >
holycow23[m]
Its been returning empty internal queue
For every call
lucifer[m]
the request consumer container
holycow23[m] uploaded an image: (295KiB) < https://matrix.chatbrainz.org/_matrix/media/v3/download/matrix.org/sVyyMotzFkfmcmnWjAfYEIWW/image.png >
holycow23: try restarting the request consumer container and then running the command again.
if it still doesn't work then share the new logs again.
holycow23[m]
<holycow23[m]> "https://gist.github.com/granth23..." <- I have updated the logs
lucifer[m]: I stopped both the dockers ran the command you gave and re ran the two commands
lucifer[m]
which two commands?
holycow23[m]
./develop.sh up and ./develop.sh spark up
reosarevok[m]
<BANG>
lucifer[m]
holycow23: i see, let's continue after the meeting.
reosarevok[m]
Welcome to another MetaBrainz Monday Meeting!
I seem to have two mailed reviews today
suvid says:
"implemented import-files endpoint and logic for uploading files and checks for file size and extension. will implement background task very soon"
And mayhem says:
'''
Today I'll be out playing with blinking lights, trying to trap musicians. :)
Last week was pretty short for me, since I was out Monday/Tuesday. The
rest of the week was spent on mentoring, PR review, MetaBrainz tasks and
just a little bit of programming on the new mapper.
Just adding a library to query postgres took me more than a day to get working since package management on C++ basically doesn't exist. Super fun. As of
right now half of all the tough test cases that were submitted pass and I have a plan on how to fix the rest.
The person wearing a green shirt (or closest to it) goes next!
