is there anything we can learn from the post (haven't read it yet)
2020-02-19 05037, 2020
ruaok
success is scary. :)
2020-02-19 05046, 2020
Gazooo joined the channel
2020-02-19 05006, 2020
ruaok
good tips for operating at scale, which isn't our problem... yet/
2020-02-19 05013, 2020
ruaok
I've skimmed it, but will read fully later.
2020-02-19 05021, 2020
iliekcomputers
>We processed ~5X the amount of data compared to the Wrapped 2018 campaign while spending 25% less overall for processing This was achieved by designing our system in a way that reduced group-by key operations, and reused the output of yearly jobs to produce top of the decade metrics.
iliekcomputers: so, good news. the upgrade to influxdb 1.7.9 seemed to work well. I need to make a few tests, to ensure that it did, but it seems so.
2020-02-19 05038, 2020
ruaok
I also make a backup of the data and loaded the data back into a 1.2.4 instance, without problems.
2020-02-19 05020, 2020
ruaok
the new version has more features for analyzing our data structure and ensure if we're doing things in a scalable way.
2020-02-19 05048, 2020
sumedh has quit
2020-02-19 05048, 2020
ruaok
it will be good to use those tools to see what we need for greater scalability. and it appears that 1.7 is already more scalable than previous versions.
2020-02-19 05000, 2020
ruaok
I'll prepare a migration doc today.
2020-02-19 05010, 2020
iliekcomputers
ruaok: how did you test it?
2020-02-19 05019, 2020
ruaok
just a cursory query so far. I hope to actually load a full version of LB (including PG) on this test server so I can run tests and actually use the data.
2020-02-19 05003, 2020
iliekcomputers
ah
2020-02-19 05008, 2020
iliekcomputers
one thing to consider
2020-02-19 05021, 2020
iliekcomputers
the production version of LB is probably running an older version of the client
2020-02-19 05034, 2020
iliekcomputers
(when compared to master)
2020-02-19 05042, 2020
ruaok
yes, indeed.
2020-02-19 05037, 2020
shivam-kapila joined the channel
2020-02-19 05049, 2020
ruaok
rough outline for starters.
2020-02-19 05048, 2020
iliekcomputers
looking sane to me
2020-02-19 05001, 2020
iliekcomputers
to upgrade the client, you'll make a pr against the production branch ?
if we need to hotfix something, we have a mess on our hands.
2020-02-19 05015, 2020
iliekcomputers
seems not a good*
2020-02-19 05027, 2020
iliekcomputers
ruaok: yeah, agreed.
2020-02-19 05035, 2020
iliekcomputers
however, we should prioritize it.
2020-02-19 05038, 2020
ruaok
but I think this should be a top priority goal for us.
2020-02-19 05042, 2020
ruaok
lol.
2020-02-19 05050, 2020
iliekcomputers
yep.
2020-02-19 05010, 2020
promehul joined the channel
2020-02-19 05002, 2020
mzfr[m] has quit
2020-02-19 05002, 2020
divyanshumahajan has quit
2020-02-19 05018, 2020
divyanshumahajan joined the channel
2020-02-19 05025, 2020
mzfr[m] joined the channel
2020-02-19 05035, 2020
promehul has quit
2020-02-19 05023, 2020
iliekcomputers
while we're at it, it might be a good idea to discuss ways to remove this toil when i'm in bcn.
2020-02-19 05027, 2020
promehul joined the channel
2020-02-19 05001, 2020
ruaok
in general I need to bug others about joining us that weekend...
2020-02-19 05038, 2020
ruaok
iliekcomputers: also, do we want to focus on bog standard LB stuff and push to make progress there, or should we also see about having Mr_Monkey join us and focus on some of the recommendation tasks that could benefit from his help?
2020-02-19 05027, 2020
D4RK-PH0ENiX has quit
2020-02-19 05059, 2020
D4RK-PH0ENiX joined the channel
2020-02-19 05047, 2020
promehul has quit
2020-02-19 05037, 2020
iliekcomputers
ruaok: haven't thought about it much yet.
2020-02-19 05046, 2020
iliekcomputers
what is the recommendation stuff?
2020-02-19 05053, 2020
madmouser1 joined the channel
2020-02-19 05009, 2020
yvanzo
iliekcomputers: I found libxslt1-dev is installed in many of our Python projects, do you have any idea why?
2020-02-19 05028, 2020
yvanzo
git log didn’t really help as it is present in initial commit of Dockerfiles.
2020-02-19 05009, 2020
yvanzo
I found it is a requirement of lxml which is not used anymore but with mb-rngpy.
