me and ferbncode are sitting in the same room, so it's okay if we want to move to better things to do
2018-03-31 09034, 2018
ruaok
let's do that.
2018-03-31 09040, 2018
ruaok
try running the script.
2018-03-31 09053, 2018
ruaok
meh, I didn't hit the save button.
2018-03-31 09008, 2018
ruaok
try the new account again, just for science.
2018-03-31 09034, 2018
iliekcomputers
got in
2018-03-31 09036, 2018
iliekcomputers
🎉
2018-03-31 09005, 2018
iliekcomputers
i ran it and it's logging a lot of stuff :-)
2018-03-31 09021, 2018
ruaok
loads and loads/
2018-03-31 09031, 2018
ruaok
it will fail making recommendations -- that is the first step to fix.
2018-03-31 09049, 2018
ruaok
I need to go food shopping -- I have nothing in the house.
2018-03-31 09001, 2018
ruaok
then I can get started on the cluster.
2018-03-31 09015, 2018
iliekcomputers
alright, we'll take a look at recommendations
2018-03-31 09015, 2018
ruaok
you two ok working on the part that does recommendations while I am out?
2018-03-31 09019, 2018
ruaok
kewl.
2018-03-31 09020, 2018
iliekcomputers
in the meanwhile
2018-03-31 09021, 2018
ruaok
bbiab
2018-03-31 09033, 2018
iliekcomputers
✋🏽
2018-03-31 09041, 2018
bukwurm
LordSputnik: Any special reason to use camelcase in bb-data? Postgres uses case-insensitive identifiers with underscores, it simply converts camelcase to lower case.
2018-03-31 09027, 2018
bukwurm
That's an issue, because it reduces readability when I get raw query results.
2018-03-31 09007, 2018
LordSputnik
bukwurm: because when they end up as JS objects, we want them to be camelCase since that's the convention
2018-03-31 09035, 2018
bukwurm
LordSputnik: Ok
2018-03-31 09010, 2018
bukwurm
LordSputnik: Also,
2018-03-31 09038, 2018
bukwurm
> I was looking into adding raw queries into bb-data.
ruaok: it says you'll like 'all i want for christmas is you' :D
2018-03-31 09022, 2018
ruaok
oh, lol.
2018-03-31 09031, 2018
ruaok
not sure if that is better than baseline. :)
2018-03-31 09059, 2018
ruaok
ok, great. that's progress already. :)
2018-03-31 09016, 2018
ruaok
so, in order to improve that we need to run all of the data.
2018-03-31 09031, 2018
ruaok
which is why we need to figure out how to load larger data sets and massage them.
2018-03-31 09015, 2018
ruaok
commit that code and let's move on to phase 2?
2018-03-31 09024, 2018
iliekcomputers
cool
2018-03-31 09026, 2018
iliekcomputers
one pr coming up
2018-03-31 09045, 2018
ruaok
don't worry about PRs for this codebase this weekend.
2018-03-31 09048, 2018
ruaok
just commit it.
2018-03-31 09005, 2018
ruaok
let me know if you don't have commit access, I haven't looked at perms closely.
2018-03-31 09033, 2018
iliekcomputers
okay
2018-03-31 09010, 2018
bukwurm
LordSputnik: Also, please could you add Immutable as dependency in the bb-data project and push to master? If I'll do it, it'll mess up the lock file with system specific dependencies. :)
2018-03-31 09013, 2018
iliekcomputers
ruaok: pushed to master
2018-03-31 09015, 2018
bukwurm
Updating knex in the lock file would also be a good idea, they've changed how raw queries return response (changed from simple tuple to js object)
2018-03-31 09008, 2018
ruaok
great. I'll start working on a getting a cluster going. can you two start working on load-data.py? that one should be pretty straightforward.
2018-03-31 09058, 2018
iliekcomputers
ruaok: load_data.py would just create a (user_id, recording_id, ts) RDD and persist it, yes?
2018-03-31 09009, 2018
iliekcomputers
from the LB dumps
2018-03-31 09013, 2018
bukwurm
That done, I'll be ready to send a PR which can extract the recent revisions and return Immutable object as `List [ Map { "editorName", "aliasName", "revisionId", "createdAt", "parentId", "action" }` as soon as I figure out how to test it. :D
2018-03-31 09024, 2018
ruaok
load-data just loads the data without transformations.
2018-03-31 09041, 2018
ruaok
prepare-data would do the user_id and recording_id transformations.
2018-03-31 09008, 2018
ruaok
not 100% certain they make sense in two steps, but I am guessing at a lot of things right now. :)
zas: FYI: I'm going to rent 4 cloud instances on hetzner cloud for our recommendation project.
2018-03-31 09058, 2018
Gazooo has quit
2018-03-31 09037, 2018
Gazooo joined the channel
2018-03-31 09014, 2018
CatQuest
i thought we had a manko on more 'brainz :P
2018-03-31 09015, 2018
iliekcomputers
ruaok: we've lost electricity so taking a break for a bit 😅😅
2018-03-31 09036, 2018
CatQuest
#codinginindia
2018-03-31 09036, 2018
CatQuest
:P
2018-03-31 09005, 2018
CatQuest
hmm maybe also #coodingonamountaininnorwayduingstorms :D
2018-03-31 09014, 2018
CatQuest
but that's a bit long
2018-03-31 09016, 2018
CatQuest
:D
2018-03-31 09016, 2018
iliekcomputers
CatQuest: yep 😓
2018-03-31 09022, 2018
CatQuest
:D
2018-03-31 09031, 2018
iliekcomputers
No storms but codinginthemountains is accurate
2018-03-31 09041, 2018
ruaok
lol
2018-03-31 09057, 2018
ruaok
I've commissioned machines, but still trying to get logged in. :(
2018-03-31 09039, 2018
CatQuest
it's finally starting to spring here so I'm a bit delerious (also I am wondering how much flood is gonna happen soon (after all the mountains are so full of snow the gov are worried 😵))
2018-03-31 09013, 2018
ruaok
iliekcomputers: are you have to test a ssh login right now?
2018-03-31 09018, 2018
iliekcomputers
Yeah
2018-03-31 09007, 2018
LordSputnik
bukwurm: bit busy today and tomorrow so might have to wait until Monday, I'm afraid
