0:23 AM
D4RK-PH0ENiX joined the channel
2019-03-01 06007, 2019
0:30 AM
CallerNo6 joined the channel
2019-03-01 06048, 2019
0:53 AM
CallerNo6 has quit
2019-03-01 06043, 2019
1:20 AM
PelvicSorcery has quit
2019-03-01 06015, 2019
2:33 AM
Rotab joined the channel
2019-03-01 06047, 2019
2:44 AM
yokel has quit
2019-03-01 06012, 2019
2:55 AM
Slurpee joined the channel
2019-03-01 06012, 2019
2:55 AM
Slurpee has quit
2019-03-01 06012, 2019
2:55 AM
Slurpee joined the channel
2019-03-01 06025, 2019
3:13 AM
travis-ci joined the channel
2019-03-01 06026, 2019
3:13 AM
travis-ci
2019-03-01 06026, 2019
3:13 AM
travis-ci has left the channel
2019-03-01 06004, 2019
3:50 AM
Leo_Verto_ joined the channel
2019-03-01 06016, 2019
3:52 AM
Leo_Verto has quit
2019-03-01 06016, 2019
3:52 AM
Leo_Verto_ is now known as Leo_Verto
2019-03-01 06053, 2019
4:37 AM
Rotab has quit
2019-03-01 06040, 2019
5:48 AM
pulkit6559 joined the channel
2019-03-01 06030, 2019
5:51 AM
Slurpee has quit
2019-03-01 06032, 2019
6:18 AM
Rotab joined the channel
2019-03-01 06007, 2019
7:03 AM
Nyanko-sensei joined the channel
2019-03-01 06059, 2019
7:04 AM
D4RK-PH0ENiX has quit
2019-03-01 06017, 2019
7:43 AM
Freso
Well, there's no link tax yet, so it's still free to link to things. :)
2019-03-01 06022, 2019
7:43 AM
silverstar joined the channel
2019-03-01 06037, 2019
7:58 AM
Nyanko-sensei has quit
2019-03-01 06041, 2019
8:02 AM
D4RK-PH0ENiX joined the channel
2019-03-01 06029, 2019
8:09 AM
michelv joined the channel
2019-03-01 06012, 2019
8:11 AM
reosarevok
For at least one more month
2019-03-01 06051, 2019
8:18 AM
pristine_
Moin moin!!
2019-03-01 06029, 2019
8:33 AM
pulkit6559 has quit
2019-03-01 06034, 2019
8:59 AM
ruaok
moooin!
2019-03-01 06033, 2019
9:00 AM
ruaok
2019-03-01 06004, 2019
9:01 AM
ruaok
(best played at full volume to get the proper effect, even 6 stories up)
2019-03-01 06032, 2019
9:09 AM
pulkit6559 joined the channel
2019-03-01 06008, 2019
9:12 AM
pristine_
ruaok: ping
2019-03-01 06015, 2019
9:12 AM
ruaok
boing!
2019-03-01 06051, 2019
9:20 AM
pristine_
Lol
2019-03-01 06015, 2019
9:21 AM
pristine_
Should we set any targets for Monday?
2019-03-01 06025, 2019
9:21 AM
pristine_
I will work this Sat and sunday
2019-03-01 06038, 2019
9:21 AM
pristine_
Since was afk on Thursday and Friday
2019-03-01 06051, 2019
9:21 AM
pristine_
iliekcomputers: ^^
2019-03-01 06021, 2019
9:22 AM
ruaok
just a reminder, we're not slave drivers. if you want to take the weekend off, you owe us nothing.
2019-03-01 06027, 2019
9:22 AM
ruaok
make sure to enjoy your life, please.
2019-03-01 06035, 2019
9:22 AM
michelv has quit
2019-03-01 06026, 2019
9:26 AM
iliekcomputers
2019-03-01 06026, 2019
9:26 AM
pristine_
ruaok: I want to, please :)
2019-03-01 06026, 2019
9:26 AM
pristine_
Lol
2019-03-01 06037, 2019
9:26 AM
iliekcomputers
Yes boss!
2019-03-01 06056, 2019
9:26 AM
ruaok
such a great picture.
2019-03-01 06003, 2019
9:27 AM
pristine_
iliekcomputers: woh!
2019-03-01 06006, 2019
9:27 AM
pristine_
Picture is amazing!
2019-03-01 06043, 2019
9:30 AM
ruaok
ok, let me get another coffee and start thinking about LB and whatnot.
2019-03-01 06054, 2019
9:32 AM
pristine_
ruaok: sure:)
2019-03-01 06003, 2019
9:33 AM
pristine_
Is coffee effective?
2019-03-01 06036, 2019
9:36 AM
ruaok
normally is. especially after being woken from a deep sleep by a samba band.
2019-03-01 06051, 2019
9:36 AM
ruaok
but those are the dangers of living in this city.
2019-03-01 06056, 2019
9:39 AM
ruaok
ok, I see that we have a lot of piece in place now, ready for review.
2019-03-01 06025, 2019
9:43 AM
ruaok
I'll work on getting people paid, doing reviews and then some follow page work with Mr_Monkey today.
2019-03-01 06046, 2019
9:43 AM
ruaok
thinking ahead, we need to improve how we show graphs on
lb.org
2019-03-01 06020, 2019
9:44 AM
ruaok
right now we have a demo bar graph up -- in Delhi we talked about getting prettier graphs and hoping to enlist the help of chhavi_.
2019-03-01 06032, 2019
9:44 AM
Nyanko-sensei joined the channel
2019-03-01 06054, 2019
9:44 AM
ruaok
on the more hard-core geeky side of things, continuing exploring yarn would be good.
2019-03-01 06034, 2019
9:45 AM
Nyanko-sensei has quit
2019-03-01 06034, 2019
9:45 AM
ruaok
would yarn make it easier to schedule all of these resources, so we don't have to do this super difficult arithmetic? 😆
2019-03-01 06028, 2019
9:47 AM
samj1912
Btw we use Oozie with yarn for workflow management and scheduling
2019-03-01 06040, 2019
9:47 AM
D4RK-PH0ENiX has quit
2019-03-01 06052, 2019
9:47 AM
pristine_
ruaok: even if we use yarn....we need this arithmetic
2019-03-01 06004, 2019
9:48 AM
samj1912
2019-03-01 06012, 2019
9:48 AM
ruaok
hi samj1912
2019-03-01 06017, 2019
9:48 AM
samj1912
Hello
2019-03-01 06019, 2019
9:48 AM
ruaok
that sounds dirty.
2019-03-01 06021, 2019
9:48 AM
pristine_
Yarn is an application/cluster manager
2019-03-01 06040, 2019
9:48 AM
ruaok
do we need it yet? or will soon?
2019-03-01 06043, 2019
9:48 AM
pristine_
But we need to specify the eexecutor count and other things.
2019-03-01 06003, 2019
9:49 AM
samj1912
ruaok: makes life much easier with Hadoop
2019-03-01 06004, 2019
9:49 AM
pristine_
And now we have everything sought out.
2019-03-01 06034, 2019
9:49 AM
pristine_
ruaok: not exactly sure. Yarn is mostly pre installed on hadoop
2019-03-01 06045, 2019
9:49 AM
ruaok
we don't use hadoop much, except for storage.
2019-03-01 06054, 2019
9:49 AM
samj1912
You use spark right?
2019-03-01 06056, 2019
9:49 AM
pristine_
It is better application manager than spark standalone
2019-03-01 06057, 2019
9:49 AM
ruaok
yes
2019-03-01 06007, 2019
9:50 AM
samj1912
Hmm, well your call
2019-03-01 06013, 2019
9:50 AM
pristine_
ruaok: we can integrate spark with yarn
2019-03-01 06019, 2019
9:50 AM
samj1912
Gotta go
2019-03-01 06026, 2019
9:50 AM
samj1912
o/
2019-03-01 06030, 2019
9:50 AM
ruaok
bai
2019-03-01 06044, 2019
9:50 AM
ruaok
well, if it is a good near-term goal, then work on that over the weekend?
2019-03-01 06024, 2019
9:51 AM
ruaok
the next task after that would be to start working on the queries that massages the data so it can be consumed in the collaborative filtering algorithm.
2019-03-01 06030, 2019
9:51 AM
pristine_
It provides flexibility. But whatever work we have at this moment we can do it with standalone too. We may need yarn in future ( I am not sure)
2019-03-01 06054, 2019
9:51 AM
pristine_
Okay. Yarn then
2019-03-01 06026, 2019
9:53 AM
pristine_
ruaok: I don't know about the algorithm
2019-03-01 06035, 2019
9:53 AM
pristine_
Would you like to brief :)
2019-03-01 06015, 2019
9:54 AM
ruaok
2019-03-01 06000, 2019
9:55 AM
ruaok
this script loads the data, massages it and then persists some data frames.
2019-03-01 06017, 2019
9:55 AM
ruaok
it uses the old method, before we started using HDFS and started really getting a clue.
2019-03-01 06029, 2019
9:55 AM
ruaok
all of this needs to be ported to our new setup.
2019-03-01 06042, 2019
9:55 AM
pristine_
Oh okay.
2019-03-01 06054, 2019
9:55 AM
ruaok
to create new intermediate tables that we can re-generate at any time.
2019-03-01 06057, 2019
9:55 AM
pristine_
Should we use Rabbitmq for that as well?
2019-03-01 06018, 2019
9:56 AM
ruaok
for now, let's keep it simple and learn.
2019-03-01 06034, 2019
9:56 AM
ruaok
just a batch system to take the data as it is right now and generate intermediate tables.
2019-03-01 06037, 2019
9:56 AM
ruaok
then next is this:
2019-03-01 06037, 2019
9:56 AM
pristine_
If I get it right....we need to import data from server in PG?
2019-03-01 06037, 2019
9:56 AM
ruaok
2019-03-01 06001, 2019
9:57 AM
ruaok
no, we've already done that. we have listens in spark.
2019-03-01 06025, 2019
9:57 AM
ruaok
next are the steps to process that data to be ready to train models.
2019-03-01 06043, 2019
9:57 AM
ruaok
the training of models is where we see train the machine learning algorithm.
2019-03-01 06057, 2019
9:57 AM
pristine_
Umm...okay
2019-03-01 06058, 2019
9:57 AM
ruaok
the final step is to use the model we've created and to make new playlists.
2019-03-01 06013, 2019
9:58 AM
ruaok
e.g. make actual recommendations.
2019-03-01 06047, 2019
9:58 AM
pristine_
I am trying to understand :)
2019-03-01 06024, 2019
9:59 AM
ruaok
keep asking questions. :)
2019-03-01 06049, 2019
9:59 AM
pristine_
Could you write in points and share in chronological order :)
2019-03-01 06001, 2019
10:00 AM
pristine_
Like set up yarn....fetch queries.. ..etc etc
2019-03-01 06037, 2019
10:00 AM
ruaok puts on morpheus sunglasses and opens his hand
2019-03-01 06046, 2019
10:00 AM
samj1912
Oh PS - ruaok in case you want to play with pyspark - use the jupyter pyspark driver for prototyping models/data analysis
2019-03-01 06059, 2019
10:00 AM
CatQuest is so divided about being woken up by samba :/
2019-03-01 06005, 2019
10:01 AM
ruaok
you have two choices, pristine_.
2019-03-01 06022, 2019
10:01 AM
ruaok
if you take the blue pill you wake up a free person and will remember nothing.
2019-03-01 06033, 2019
10:01 AM
CatQuest
on one hand: awesome
2019-03-01 06033, 2019
10:01 AM
CatQuest
on the other: fecking NOISE ლ(ಠ益ಠლ)
2019-03-01 06040, 2019
10:01 AM
ruaok
if you take the red pill you see all the truth and will get added to our trello board.
2019-03-01 06042, 2019
10:01 AM
pristine_
What? Lol
2019-03-01 06052, 2019
10:01 AM
ruaok
you've seen the matrix, yes?
2019-03-01 06053, 2019
10:01 AM
CatQuest
pristine_: matrix meme
2019-03-01 06001, 2019
10:02 AM
pristine_
No.
2019-03-01 06007, 2019
10:02 AM
pristine_
Shit
2019-03-01 06008, 2019
10:02 AM
CatQuest
ruaok: matrix is already old af
2019-03-01 06018, 2019
10:02 AM
iliekcomputers
You're old af
2019-03-01 06019, 2019
10:02 AM
CatQuest
need soemthinmore current yada
2019-03-01 06027, 2019
10:02 AM
ruaok
2019-03-01 06036, 2019
10:02 AM
CatQuest is 37 :V
2019-03-01 06002, 2019
10:03 AM
CatQuest
that's old i gues. but not af o___O
2019-03-01 06011, 2019
10:03 AM
iliekcomputers
Kidding :)
2019-03-01 06012, 2019
10:03 AM
pristine_
Can someone tell me what does it mean?
2019-03-01 06024, 2019
10:03 AM
ruaok
watch the video.
2019-03-01 06025, 2019
10:03 AM
CatQuest
it means like one ting yo ucna go away free
2019-03-01 06027, 2019
10:03 AM
silverstar has quit
2019-03-01 06032, 2019
10:03 AM
CatQuest
theo ther you will sink in deep
2019-03-01 06034, 2019
10:03 AM
pristine_
ruaok: I will return in a while.
2019-03-01 06054, 2019
10:03 AM
pristine_
It will be great if you make the points.
2019-03-01 06006, 2019
10:04 AM
ruaok
tell me which pill you choose.
2019-03-01 06014, 2019
10:04 AM
pristine_
Okay. Red
2019-03-01 06016, 2019
10:04 AM
iliekcomputers
Green
2019-03-01 06019, 2019
10:04 AM
CatQuest
hah
2019-03-01 06024, 2019
10:04 AM
CatQuest
orange!
2019-03-01 06035, 2019
10:04 AM
ruaok
2019-03-01 06036, 2019
10:04 AM
iliekcomputers
Ooh
2019-03-01 06059, 2019
10:04 AM
pristine_
Blue
2019-03-01 06001, 2019
10:05 AM
Gazooo has quit
2019-03-01 06014, 2019
10:05 AM
ruaok
once you chose red, you cannot go back.
