me and ferbncode are sitting in the same room, so it's okay if we want to move to better things to do
ruaok
let's do that.
try running the script.
meh, I didn't hit the save button.
try the new account again, just for science.
iliekcomputers
got in
🎉
i ran it and it's logging a lot of stuff :-)
ruaok
loads and loads/
it will fail making recommendations -- that is the first step to fix.
I need to go food shopping -- I have nothing in the house.
then I can get started on the cluster.
iliekcomputers
alright, we'll take a look at recommendations
ruaok
you two ok working on the part that does recommendations while I am out?
kewl.
iliekcomputers
in the meanwhile
ruaok
bbiab
iliekcomputers
✋🏽
bukwurm
LordSputnik: Any special reason to use camelcase in bb-data? Postgres uses case-insensitive identifiers with underscores, it simply converts camelcase to lower case.
That's an issue, because it reduces readability when I get raw query results.
LordSputnik
bukwurm: because when they end up as JS objects, we want them to be camelCase since that's the convention
bukwurm
LordSputnik: Ok
LordSputnik: Also,
> I was looking into adding raw queries into bb-data.
ruaok: it says you'll like 'all i want for christmas is you' :D
ruaok
oh, lol.
not sure if that is better than baseline. :)
ok, great. that's progress already. :)
so, in order to improve that we need to run all of the data.
which is why we need to figure out how to load larger data sets and massage them.
commit that code and let's move on to phase 2?
iliekcomputers
cool
one pr coming up
ruaok
don't worry about PRs for this codebase this weekend.
just commit it.
let me know if you don't have commit access, I haven't looked at perms closely.
iliekcomputers
okay
bukwurm
LordSputnik: Also, please could you add Immutable as dependency in the bb-data project and push to master? If I'll do it, it'll mess up the lock file with system specific dependencies. :)
iliekcomputers
ruaok: pushed to master
bukwurm
Updating knex in the lock file would also be a good idea, they've changed how raw queries return response (changed from simple tuple to js object)
ruaok
great. I'll start working on a getting a cluster going. can you two start working on load-data.py? that one should be pretty straightforward.
iliekcomputers
ruaok: load_data.py would just create a (user_id, recording_id, ts) RDD and persist it, yes?
from the LB dumps
bukwurm
That done, I'll be ready to send a PR which can extract the recent revisions and return Immutable object as `List [ Map { "editorName", "aliasName", "revisionId", "createdAt", "parentId", "action" }` as soon as I figure out how to test it. :D
ruaok
load-data just loads the data without transformations.
prepare-data would do the user_id and recording_id transformations.
not 100% certain they make sense in two steps, but I am guessing at a lot of things right now. :)
zas: FYI: I'm going to rent 4 cloud instances on hetzner cloud for our recommendation project.
Gazooo has quit
Gazooo joined the channel
CatQuest
i thought we had a manko on more 'brainz :P
iliekcomputers
ruaok: we've lost electricity so taking a break for a bit 😅😅
CatQuest
#codinginindia
:P
hmm maybe also #coodingonamountaininnorwayduingstorms :D
but that's a bit long
:D
iliekcomputers
CatQuest: yep 😓
CatQuest
:D
iliekcomputers
No storms but codinginthemountains is accurate
ruaok
lol
I've commissioned machines, but still trying to get logged in. :(
CatQuest
it's finally starting to spring here so I'm a bit delerious (also I am wondering how much flood is gonna happen soon (after all the mountains are so full of snow the gov are worried 😵))
ruaok
iliekcomputers: are you have to test a ssh login right now?
iliekcomputers
Yeah
LordSputnik
bukwurm: bit busy today and tomorrow so might have to wait until Monday, I'm afraid
