-
Gentlecat
oh, that's neat
2015-09-21 26414, 2015
-
zas
it looks like it was a fun night ruaok ;)
2015-09-21 26424, 2015
-
ruaok
zas: massively fun. :)
2015-09-21 26434, 2015
-
ruaok
not the best for a sunday. :)
2015-09-21 26439, 2015
-
zas
ahah
2015-09-21 26400, 2015
-
zas
what is the history behind this event ?
2015-09-21 26402, 2015
-
Gentlecat
possibly the most fun I ever had :P
2015-09-21 26424, 2015
-
alastairp
2015-09-21 26424, 2015
-
ruaok
Gentlecat: wow. :)
2015-09-21 26446, 2015
-
alastairp
though I don't really trust english wikipedia to explain everything
2015-09-21 26410, 2015
-
alastairp
2015-09-21 26404, 2015
-
zas
the french page tells it exists back to middle age, around 11th century, and it also happens in the very south of France
2015-09-21 26410, 2015
-
reosarevok has quit
2015-09-21 26425, 2015
-
alastairp
yeah, which is also historically catalunya
2015-09-21 26450, 2015
-
zas
yup
2015-09-21 26419, 2015
-
samphippen has quit
2015-09-21 26408, 2015
-
samphippen joined the channel
2015-09-21 26437, 2015
-
zas
Hell seems to become more organized : "Federació de Diables i Dimonis de Catalunya" ;)
2015-09-21 26414, 2015
-
ruaok
this is very serious business here.
2015-09-21 26423, 2015
-
ruaok
every neighborhood has a troupe
2015-09-21 26421, 2015
-
zas
similar to carnavals in Brazil
2015-09-21 26401, 2015
-
ruaok nods
2015-09-21 26424, 2015
-
Mineo joined the channel
2015-09-21 26433, 2015
-
UmkaDK_ joined the channel
2015-09-21 26403, 2015
-
UmkaDK has quit
2015-09-21 26411, 2015
-
CatCat has quit
2015-09-21 26417, 2015
-
alastairp
ocharles__: strange. your other listens still haven't come up yet. it looks like it's doing a lot of catching up...
2015-09-21 26425, 2015
-
ocharles__
indeed
2015-09-21 26442, 2015
-
alastairp
(unless it crashed again... *checks*)
2015-09-21 26450, 2015
-
Freso
alastairp: Did you deploy the new code?
2015-09-21 26410, 2015
-
alastairp
no
2015-09-21 26415, 2015
-
Gentlecat
2015-09-21 26433, 2015
-
CatCat joined the channel
2015-09-21 26458, 2015
-
alastairp
ocharles__: still running, I guess. It's caught up 2 million (!) since I started it
2015-09-21 26409, 2015
-
ruaok
alastairp: did it crash?
2015-09-21 26436, 2015
-
alastairp
ruaok: yeah, LB-47
2015-09-21 26422, 2015
-
ruaok
ah, yes. I've seen this one before.
2015-09-21 26414, 2015
-
alastairp
but it's different to no response from MessyBrainz, because in that case it loads but validate returns false
2015-09-21 26412, 2015
-
ruaok
2015-09-21 26415, 2015
-
reosarevok joined the channel
2015-09-21 26417, 2015
-
reosarevok has quit
2015-09-21 26429, 2015
-
ruaok
alastairp: yes, periodically we get a response from messybrainz that lacks all the msids.
2015-09-21 26439, 2015
-
ruaok
those get tossed by the listenstore importer.
2015-09-21 26458, 2015
-
alastairp
yep, that problem is LB-42
2015-09-21 26459, 2015
-
ruaok
but the root of the problem lies at msb, since the response from msb is valid, but empty
2015-09-21 26404, 2015
-
alastairp
I'm looking into it now
2015-09-21 26418, 2015
-
ruaok
k
2015-09-21 26441, 2015
-
samphippen has quit
2015-09-21 26418, 2015
-
samphippen joined the channel
2015-09-21 26436, 2015
-
xps2 joined the channel
2015-09-21 26446, 2015
-
alastairp
ruaok: do we want to start looking at how we are going to counting? Open a ticket, get some people giving feedback?
2015-09-21 26416, 2015
-
ruaok
to document our redis idea?
2015-09-21 26416, 2015
-
samphippen has quit
2015-09-21 26428, 2015
-
alastairp
yeah
2015-09-21 26432, 2015
-
ruaok
sure
2015-09-21 26416, 2015
-
samphippen joined the channel
2015-09-21 26442, 2015
-
ocharles__
considered any large data set approximations like hyperloglog?
2015-09-21 26402, 2015
-
ruaok
what does that do?
2015-09-21 26440, 2015
-
alastairp
perhaps not until real counting becomes too slow for us
2015-09-21 26451, 2015
-
alastairp
we only have 200 users (at the moment...)
2015-09-21 26437, 2015
-
alastairp
our plan is to have a redis counter per user, which shouldn't cause many problems for now
2015-09-21 26450, 2015
-
ocharles__
It's a probabilistic data structure that gives you an approximate count of elements in a multiset
2015-09-21 26450, 2015
-
alastairp
I think hyperloglog is too complex too quickly
2015-09-21 26401, 2015
-
ocharles__
fair enough
2015-09-21 26412, 2015
-
alastairp
but it's cool
2015-09-21 26430, 2015
-
alastairp
and if we want to count per-track and per-artist too, it might be something worth looking at
2015-09-21 26455, 2015
-
ocharles__
are there plans to have LB have those kind of metrics like last.fm?
2015-09-21 26459, 2015
-
ocharles__
or is that mostly to be decided?
2015-09-21 26413, 2015
-
alastairp
mostly to be decided :)
2015-09-21 26420, 2015
-
alastairp
what do you want? we'll add it!
2015-09-21 26441, 2015
-
ocharles__
you mean "what do you want? you add it!", right? :)
2015-09-21 26448, 2015
-
alastairp
:(
2015-09-21 26453, 2015
-
ocharles__
though maybe I shouldn't argue with your willingness to do work for me
2015-09-21 26459, 2015
-
ocharles__
so... a pony?
2015-09-21 26404, 2015
-
alastairp
oh, right. /you'll/ do it
2015-09-21 26406, 2015
-
alastairp
go for it
2015-09-21 26409, 2015
-
github joined the channel
2015-09-21 26409, 2015
-
github
2015-09-21 26409, 2015
-
github
messybrainz-server/ujson f3d0775 Alastair Porter: Add some missing indexes for lookup
2015-09-21 26409, 2015
-
github
messybrainz-server/ujson 394719f Alastair Porter: Use ujson to read requests and make responses
2015-09-21 26409, 2015
-
github has left the channel
2015-09-21 26432, 2015
-
ruaok
2015-09-21 26454, 2015
-
github joined the channel
2015-09-21 26454, 2015
-
github
2015-09-21 26454, 2015
-
github has left the channel
2015-09-21 26444, 2015
-
alastairp
ruaok: today I want to do LB-21, LB-40, LB-42, LB-43. Let me know if you want to pick up one of them
2015-09-21 26418, 2015
-
alastairp
--> lunch
2015-09-21 26418, 2015
-
ruaok
alastairp: the big question for me is if we still want to move the LB sanity checking to MSB in order to avoid duplicate parsing.
2015-09-21 26408, 2015
-
xps2 has quit
2015-09-21 26444, 2015
-
xps2 joined the channel
2015-09-21 26454, 2015
-
samphippen has quit
2015-09-21 26457, 2015
-
alastairp
ruaok: I agree. I like that it could make things faster, with less parsing, but it still feels like the wrong place to do it, to me
2015-09-21 26449, 2015
-
ruaok
no doubt. logically it is a mess. :(
2015-09-21 26446, 2015
-
Mogmi joined the channel
2015-09-21 26458, 2015
-
CatQuest has quit
2015-09-21 26401, 2015
-
alastairp
so many jira tabs opened
2015-09-21 26415, 2015
-
Freso has quit
2015-09-21 26422, 2015
-
alastairp
2015-09-21 26403, 2015
-
Freso joined the channel
2015-09-21 26441, 2015
-
Gentlecat
alastairp: yeah, let's close it
2015-09-21 26446, 2015
-
alastairp
cool
2015-09-21 26456, 2015
-
alastairp
I'm doing AB dev full time now!
2015-09-21 26411, 2015
-
alastairp
well, maybe not full time, 60% or so
2015-09-21 26416, 2015
-
alastairp
not that you are, traitor
2015-09-21 26409, 2015
-
Gentlecat
I'm there if you need any help
2015-09-21 26418, 2015
-
alastairp
awww
2015-09-21 26425, 2015
-
alastairp
ruaok: you pay the right people
2015-09-21 26436, 2015
-
ruaok
wait, I did something right?
2015-09-21 26437, 2015
-
Gentlecat
actually hope that I get some project which would basically allow me to continue working on it
2015-09-21 26445, 2015
-
Gentlecat
on "university time"
2015-09-21 26435, 2015
-
ruaok
alastairp: I've got a pile of non-coding stuff to do and a skype call this afternoon.
2015-09-21 26445, 2015
-
alastairp
ruaok: no problem
2015-09-21 26400, 2015
-
ruaok
around 17:30h I can be free to help code the optimizations, unless you're done with them by then.
2015-09-21 26401, 2015
-
Freso
alastairp: \o/
2015-09-21 26404, 2015
-
alastairp
I'll see if I can get some more msb stuff done, since it kind of fits in with real work
2015-09-21 26404, 2015
-
ruaok
chances are...
2015-09-21 26429, 2015
-
alastairp
Gentlecat: right, let's talk
2015-09-21 26453, 2015
-
alastairp
it'd be a good project if we can find a end-to-end project with a research output
2015-09-21 26405, 2015
-
ruaok
mozartkugeln in the demo room.
2015-09-21 26416, 2015
-
alastairp
yes
2015-09-21 26417, 2015
-
ruaok
alastairp: you should go steal some for the team. :)
2015-09-21 26419, 2015
-
alastairp
already eaten
2015-09-21 26423, 2015
-
ruaok
damn.
2015-09-21 26424, 2015
-
alastairp
all gone, I think
2015-09-21 26429, 2015
-
alastairp
at least I had one
2015-09-21 26445, 2015
-
alastairp
2015-09-21 26414, 2015
-
alastairp
1) if you're able to give an opinion now, please do; 2) if you think it requires a discussion in person, can we do that this week?
2015-09-21 26452, 2015
-
ruaok
mirror what MB does, I would say.
2015-09-21 26456, 2015
-
ruaok
that is to say that we should:
2015-09-21 26412, 2015
-
ruaok
dump a last incremental (and stop accepting new submissions during this)
2015-09-21 26422, 2015
-
ruaok
do the schema change
2015-09-21 26428, 2015
-
ruaok
do a full dump
2015-09-21 26435, 2015
-
ruaok
continue to do incrementals
2015-09-21 26421, 2015
-
alastairp
thanks. Added as a comment. I'll work towards that
2015-09-21 26425, 2015
-
ruaok
k
2015-09-21 26444, 2015
-
ruaok
the stopping submissions is kinda hard in this case.
2015-09-21 26451, 2015
-
ruaok
not hard, sucky
2015-09-21 26457, 2015
-
ruaok
because even an incremental will take a while.
2015-09-21 26404, 2015
-
ruaok
maybe consider doing two incrementals.
2015-09-21 26413, 2015
-
ruaok
one to get the bulk out of the way.
2015-09-21 26418, 2015
-
alastairp
right
2015-09-21 26421, 2015
-
ruaok
and another one while the site is in read only mode
2015-09-21 26431, 2015
-
alastairp
man, we should put kafka in acousticbrainz!
2015-09-21 26439, 2015
-
ruaok
and MB!
2015-09-21 26445, 2015
-
alastairp
allthebrainz
2015-09-21 26447, 2015
-
ruaok
it is clearly the right tool for that job.
2015-09-21 26458, 2015
-
alastairp
actually, this is interesting: the schema change will only be on the high level tables
2015-09-21 26403, 2015
-
alastairp
can we use that to our advantage?
2015-09-21 26419, 2015
-
alastairp
I'll think about it
2015-09-21 26434, 2015
-
ruaok
it allows zero downtime.
