10:34 AM
iliekcomputers
pristine__: we really need to look into the dev environment of labs before gsoc
2020-01-04 00431, 2020
10:34 AM
pristine__
I am on it
2020-01-04 00434, 2020
10:34 AM
pristine__
:)
2020-01-04 00445, 2020
10:34 AM
pristine__
I am just trying to upload mapping and listens
2020-01-04 00403, 2020
10:35 AM
sarthak_jain
iliekcomputers, in docs it is not mentioned, the file in which I need to enter Music Brainz credentials ?
2020-01-04 00411, 2020
10:35 AM
pristine__
So that new contributors have some data to play with and yes, as you say the docs:)
2020-01-04 00412, 2020
10:35 AM
iliekcomputers
there's no docs, it has dependencies, hell, i just test shit on leader. not sure that'll be helpful to people coming in.
2020-01-04 00442, 2020
10:35 AM
iliekcomputers
sarthak_jain: you should update the newly created config.py file.
2020-01-04 00456, 2020
10:35 AM
iliekcomputers
sarthak_jain: nice feedback. that sentence could definitely be clearer.
2020-01-04 00418, 2020
10:36 AM
pristine__
With docs it will be. Maybe we can ask sarthak_jain to help us write docs for labs
2020-01-04 00448, 2020
10:37 AM
sarthak_jain
Yes pristine__ '=D
2020-01-04 00456, 2020
10:37 AM
pristine__
Thanks :)
2020-01-04 00415, 2020
10:38 AM
sarthak_jain
iliekcomputers done till the point of entering MUSICBRAINZ_ID and secret in config.py file.
2020-01-04 00425, 2020
10:38 AM
iliekcomputers
great.
2020-01-04 00442, 2020
10:38 AM
iliekcomputers
let's move on to setting up last.fm
2020-01-04 00446, 2020
10:38 AM
sarthak_jain
Yes
2020-01-04 00410, 2020
10:39 AM
sarthak_jain
2020-01-04 00422, 2020
10:39 AM
sarthak_jain
Where do I find the API Key?
2020-01-04 00452, 2020
10:39 AM
iliekcomputers
you'll need to create a new key
2020-01-04 00402, 2020
10:40 AM
sarthak_jain
Yes
2020-01-04 00413, 2020
10:40 AM
iliekcomputers
there's a getting started section on the page which has a few instructions
2020-01-04 00432, 2020
10:40 AM
sarthak_jain
So, first we need to create an account?
2020-01-04 00436, 2020
10:40 AM
iliekcomputers
yes
2020-01-04 00448, 2020
10:40 AM
iliekcomputers
this section needs to be rewritten basically.
2020-01-04 00407, 2020
10:41 AM
sarthak_jain
Yes, exactly!
2020-01-04 00441, 2020
10:41 AM
sarthak_jain
What do we write in the application name, description?
2020-01-04 00410, 2020
10:42 AM
iliekcomputers
alright, let's skip this for now.
2020-01-04 00435, 2020
10:42 AM
sarthak_jain
Okay
2020-01-04 00435, 2020
10:42 AM
iliekcomputers
i'm gonna come up with some better instructions for last.fm and spotify setup (both of those are optional)
2020-01-04 00443, 2020
10:42 AM
sarthak_jain
Okay sure.
2020-01-04 00459, 2020
10:42 AM
iliekcomputers
2020-01-04 00409, 2020
10:43 AM
iliekcomputers
this is compulsory tho
2020-01-04 00445, 2020
10:43 AM
sarthak_jain
You mean LASTFM_API_URL in config.py?
2020-01-04 00402, 2020
10:44 AM
iliekcomputers
no, there's a different variable in config.py called `API_URL`
2020-01-04 00405, 2020
10:44 AM
iliekcomputers
(lol)
2020-01-04 00427, 2020
10:44 AM
sarthak_jain
Hehe oka
2020-01-04 00448, 2020
10:44 AM
sarthak_jain
SO, API_URL = 'http://localhost.'
2020-01-04 00450, 2020
10:44 AM
sarthak_jain
this?
2020-01-04 00457, 2020
10:44 AM
iliekcomputers
yes
2020-01-04 00400, 2020
10:45 AM
iliekcomputers
no full stop
2020-01-04 00407, 2020
10:45 AM
iliekcomputers
just `http://localhost`
2020-01-04 00408, 2020
10:45 AM
sarthak_jain
Yes
2020-01-04 00430, 2020
10:45 AM
sarthak_jain
Yup done with that
2020-01-04 00433, 2020
10:45 AM
iliekcomputers
alright.
2020-01-04 00440, 2020
10:45 AM
iliekcomputers
let's move on to develop.sh
2020-01-04 00406, 2020
10:46 AM
sarthak_jain
Yes, so I run sudo ./develop.sh ?
2020-01-04 00413, 2020
10:46 AM
sarthak_jain
in the terminal?
2020-01-04 00431, 2020
10:46 AM
iliekcomputers
no sudo
2020-01-04 00436, 2020
10:46 AM
iliekcomputers
just ./develop.sh
2020-01-04 00459, 2020
10:47 AM
sarthak_jain
2020-01-04 00402, 2020
10:48 AM
sarthak_jain
I get this
2020-01-04 00452, 2020
10:48 AM
iliekcomputers
ah
2020-01-04 00459, 2020
10:48 AM
iliekcomputers
right
2020-01-04 00410, 2020
10:49 AM
iliekcomputers
the script changed but the docs didn't update.
2020-01-04 00457, 2020
10:49 AM
sarthak_jain
what do I do now?
2020-01-04 00417, 2020
10:51 AM
iliekcomputers
gimme a second
2020-01-04 00454, 2020
10:51 AM
iliekcomputers
2020-01-04 00420, 2020
10:52 AM
sarthak_jain
Oh okayy
2020-01-04 00429, 2020
10:52 AM
sarthak_jain
./develop.sh build?
2020-01-04 00442, 2020
10:52 AM
iliekcomputers
yes
2020-01-04 00432, 2020
10:53 AM
sarthak_jain
2020-01-04 00437, 2020
10:53 AM
sarthak_jain
This is what I get now.
2020-01-04 00441, 2020
10:54 AM
iliekcomputers
right
2020-01-04 00446, 2020
10:54 AM
iliekcomputers
i've seen this before
2020-01-04 00403, 2020
10:55 AM
iliekcomputers
you need to add your linux user to the `docker` group
2020-01-04 00413, 2020
10:55 AM
iliekcomputers
`sudo usermod -aG docker $USER`
2020-01-04 00422, 2020
10:55 AM
iliekcomputers
run this, log out and log in and try again
2020-01-04 00441, 2020
10:55 AM
sarthak_jain
what location do I run this command at ?
2020-01-04 00447, 2020
10:55 AM
sarthak_jain
Inside LB-server ?
2020-01-04 00452, 2020
10:55 AM
iliekcomputers
doesn't matter
2020-01-04 00436, 2020
10:56 AM
sarthak_jain has quit
2020-01-04 00425, 2020
10:57 AM
sarthak_jain joined the channel
2020-01-04 00416, 2020
10:58 AM
sarthak_jain
iliekcomputers I tried again
2020-01-04 00421, 2020
10:58 AM
sarthak_jain
shows the same error
2020-01-04 00439, 2020
10:58 AM
iliekcomputers
did you log out of your OS and log back in?
2020-01-04 00451, 2020
10:58 AM
sarthak_jain
Yup
2020-01-04 00455, 2020
10:58 AM
sarthak_jain
Should I restart?
2020-01-04 00423, 2020
10:59 AM
iliekcomputers
can't hurt to try
2020-01-04 00444, 2020
10:59 AM
sarthak_jain
Okay. I'll brb.
2020-01-04 00419, 2020
11:00 AM
sarthak_jain has quit
2020-01-04 00418, 2020
11:02 AM
sarthak_jain joined the channel
2020-01-04 00425, 2020
11:02 AM
sarthak_jain
iliekcomputers restarting did help.
2020-01-04 00430, 2020
11:02 AM
sarthak_jain
Now itworking
2020-01-04 00406, 2020
11:03 AM
rektide has quit
2020-01-04 00414, 2020
11:04 AM
pristine__
iliekcomputers: is the upload complete?
2020-01-04 00417, 2020
11:04 AM
pristine__
Of listens
2020-01-04 00443, 2020
11:04 AM
iliekcomputers
it was stuck at a df.count in a logging statement lol
2020-01-04 00444, 2020
11:04 AM
sarthak_jain
iliekcomputers successfully built.
2020-01-04 00453, 2020
11:04 AM
iliekcomputers
i removed the log and started over.
2020-01-04 00419, 2020
11:05 AM
sarthak_jain
Now, we move to initialising LB containers?
2020-01-04 00421, 2020
11:05 AM
iliekcomputers
sarthak_jain: what do you mean? elaborate pls?
2020-01-04 00445, 2020
11:05 AM
sarthak_jain
I mean I ran the command.
2020-01-04 00451, 2020
11:05 AM
sarthak_jain
It did run this time
2020-01-04 00456, 2020
11:05 AM
iliekcomputers
and it worked successfully
2020-01-04 00457, 2020
11:05 AM
sarthak_jain
and in the end it says:
2020-01-04 00400, 2020
11:06 AM
iliekcomputers
?
2020-01-04 00406, 2020
11:06 AM
sarthak_jain
Successfully built 6834b8ce53a3Successfully tagged listenbrainz_static_builder:latest
2020-01-04 00420, 2020
11:06 AM
sarthak_jain
Yup successfully.
2020-01-04 00456, 2020
11:06 AM
sarthak_jain
Now, initializing DB ?
2020-01-04 00402, 2020
11:07 AM
iliekcomputers
yep
2020-01-04 00407, 2020
11:07 AM
iliekcomputers
2020-01-04 00419, 2020
11:07 AM
iliekcomputers
not the ones in readthedocs
2020-01-04 00407, 2020
11:08 AM
sarthak_jain
Yup
2020-01-04 00432, 2020
11:08 AM
pristine__
iliekcomputers: can you tell me why it was stuck? I was facing the same problem when I tried to upload listens.
2020-01-04 00459, 2020
11:08 AM
iliekcomputers
2020-01-04 00415, 2020
11:09 AM
iliekcomputers
this count was taking a long time
2020-01-04 00417, 2020
11:09 AM
iliekcomputers
but
2020-01-04 00425, 2020
11:09 AM
iliekcomputers
it still hasn't uploaded a single file yet
2020-01-04 00432, 2020
11:09 AM
pristine__
But why? It's weird
2020-01-04 00432, 2020
11:09 AM
iliekcomputers
so i'm not sure if it's completely fixed?
2020-01-04 00446, 2020
11:09 AM
iliekcomputers
question: how did the listens get deleted?
2020-01-04 00402, 2020
11:10 AM
pristine__
> it still hasn't uploaded a single file yet
2020-01-04 00417, 2020
11:10 AM
pristine__
Because it hasn't moved pased the count statement
2020-01-04 00418, 2020
11:10 AM
iliekcomputers
the write is very slow
2020-01-04 00425, 2020
11:10 AM
iliekcomputers
i removed the count
2020-01-04 00428, 2020
11:10 AM
pristine__
> it still hasn't uploaded a single file yet
2020-01-04 00429, 2020
11:10 AM
pristine__
Ifk
2020-01-04 00432, 2020
11:10 AM
pristine__
Idk
2020-01-04 00458, 2020
11:10 AM
iliekcomputers
i'm gonna try again, but this is much slower than I remember
2020-01-04 00406, 2020
11:11 AM
iliekcomputers
the entire import used to take around 40 min
2020-01-04 00407, 2020
11:11 AM
pristine__
> so i'm not sure if it's completely fixed?
2020-01-04 00417, 2020
11:11 AM
iliekcomputers
i'd been waiting for 10 min for the first file to be written
2020-01-04 00455, 2020
11:11 AM
pristine__
The error you pasted was because of upload, which works now. If you check /tmp there will be files so it is fixed
2020-01-04 00415, 2020
11:12 AM
pristine__
> i'm gonna try again, but this is much slower than I remember
2020-01-04 00432, 2020
11:12 AM
pristine__
Exactly. This is what I am worried about :(
2020-01-04 00442, 2020
11:12 AM
sarthak_jain
iliekcomputers done with the DB's.
2020-01-04 00451, 2020
11:12 AM
pristine__
> question: how did the listens get deleted?
2020-01-04 00454, 2020
11:12 AM
iliekcomputers
do you think it might be because of the config change re trash
2020-01-04 00403, 2020
11:13 AM
pristine__
I have not tested anything on leader from quiet some time :(
2020-01-04 00403, 2020
11:13 AM
iliekcomputers
sarthak_jain: awesome, keep moving forward
2020-01-04 00408, 2020
11:13 AM
sarthak_jain
Now, I need to install node dependencies?
2020-01-04 00413, 2020
11:13 AM
iliekcomputers
yes
2020-01-04 00428, 2020
11:13 AM
pristine__
> do you think it might be because of the config change re trash
2020-01-04 00432, 2020
11:13 AM
sarthak_jain
Cool
2020-01-04 00420, 2020
11:14 AM
sarthak_jain
2020-01-04 00429, 2020
11:14 AM
pristine__
I don't have an opinion on that. The trash just keeps the fine for 3 min and deletes them. How can it make the cluster slow? But nice thought. We can check all the possibilities since spark is yet hard to understand :(
2020-01-04 00435, 2020
11:14 AM
sarthak_jain
iliekcomputers: I get this when I run ./develop.sh npm
2020-01-04 00441, 2020
11:14 AM
pristine__
Files*
2020-01-04 00454, 2020
11:14 AM
iliekcomputers
pristine__: have there been any other config changes?
2020-01-04 00404, 2020
11:15 AM
pristine__
No
2020-01-04 00421, 2020
11:15 AM
iliekcomputers
i'd like to revert the change and try again.
2020-01-04 00424, 2020
11:15 AM
pristine__
When you are done with uploading lemme know, I can test by rmoving trash config
2020-01-04 00432, 2020
11:15 AM
pristine__
:p
2020-01-04 00433, 2020
11:15 AM
pristine__
Same
2020-01-04 00435, 2020
11:15 AM
iliekcomputers
lol, the upload isn't working
2020-01-04 00444, 2020
11:15 AM
pristine__
What does it say?
2020-01-04 00450, 2020
11:15 AM
iliekcomputers
it's been writing one file for 10 min now
2020-01-04 00456, 2020
11:15 AM
iliekcomputers
it has to write 100s of these files
2020-01-04 00448, 2020
11:16 AM
iliekcomputers
sarthak_jain: hmm, first time i'm seeing that error.
2020-01-04 00418, 2020
11:17 AM
sarthak_jain
=(
