a_k: you're still not telling me what you want to do. what do you want to do?
2016-03-01 06105, 2016
a_k
ruaok: i want to contribute to these projects
2016-03-01 06122, 2016
ruaok
how do you want to contribute to these projects. and please don't tell me that you want to write code to make us happy.
2016-03-01 06119, 2016
a_k
help in solving bugs i guess, i am not sure
2016-03-01 06131, 2016
a_k
ruaok:
2016-03-01 06157, 2016
ruaok
ok, please come back when you are sure of what you want to do.
2016-03-01 06131, 2016
a_k
can you tell me the ways to contribute
2016-03-01 06150, 2016
Gentlecat
there's information about that in the wiki, you should read it first
2016-03-01 06102, 2016
Gentlecat
find a project you are interested in, try to install it
2016-03-01 06138, 2016
Gentlecat
if you can't do that, you'll have a hard time later
2016-03-01 06148, 2016
rahulr has quit
2016-03-01 06130, 2016
alastairp deletes migrated AB database
2016-03-01 06131, 2016
alastairp
sigh
2016-03-01 06150, 2016
Gentlecat
?
2016-03-01 06106, 2016
alastairp
I didn't fill in the 'created' time correctly. it was set to now()
2016-03-01 06127, 2016
alastairp
so all submissions in AB are now created between 26th and 27th of feb
2016-03-01 06129, 2016
alastairp
this year
2016-03-01 06131, 2016
alastairp
*all* of them
2016-03-01 06123, 2016
alastairp
how did I miss that?
2016-03-01 06140, 2016
alastairp
ah, actually. ids are the same. I should be able to copy them over
2016-03-01 06120, 2016
jesus2099 joined the channel
2016-03-01 06140, 2016
ruaok
alastairp: :( bummer.
2016-03-01 06128, 2016
alastairp
ruaok: I think it's OK. I *did* copy ids manually instead of using the sequence, so they should be the same in both databases. I'll just copy the created time over
2016-03-01 06125, 2016
Nyanko-sensei has quit
2016-03-01 06152, 2016
D4RK-PH0ENiX joined the channel
2016-03-01 06122, 2016
hitesh96db joined the channel
2016-03-01 06153, 2016
ruaok
alastairp: I wonder if that will be faster than re-doing the migration.
2016-03-01 06102, 2016
alastairp
I think so
2016-03-01 06112, 2016
alastairp
because it's just ("just") 3 million inserts
2016-03-01 06116, 2016
ruaok
ok, I think the first version of my presentation to the EC is ready-enough.
2016-03-01 06120, 2016
ruaok
"just"
2016-03-01 06123, 2016
jesus2099 has left the channel
2016-03-01 06137, 2016
D4RK-PH0ENiX has quit
2016-03-01 06138, 2016
alastairp
whereas the migration is select, split, alter, re-encode, insert 20 rows, repeat 3 million times
2016-03-01 06114, 2016
ruaok
the only thing slow about that is the insert.
2016-03-01 06120, 2016
ruaok
and I think insert is still faster than update.
2016-03-01 06126, 2016
alastairp
right
2016-03-01 06131, 2016
ruaok
I'm really curious about this. keep track of the time, please?
2016-03-01 06133, 2016
alastairp
let me try it
2016-03-01 06144, 2016
ruaok
of course. this is great learning.
2016-03-01 06141, 2016
ruaok
Gentlecat, zas: invoices please.
2016-03-01 06123, 2016
alastairp
ruaok: huh
2016-03-01 06139, 2016
alastairp
so, I think it was about 30 seconds per 1000 rows to insert
2016-03-01 06145, 2016
alastairp
update is about 10s
2016-03-01 06105, 2016
hitesh96db has quit
2016-03-01 06120, 2016
D4RK-PH0ENiX joined the channel
2016-03-01 06122, 2016
alastairp
though I'm doing 1000 updates in a transaction then moving on. I wonder if it would be faster if I did one query per 1000 items
2016-03-01 06124, 2016
hitesh96db joined the channel
2016-03-01 06114, 2016
ruaok
Freso: where is that GSoC flow chart from last year?
2016-03-01 06115, 2016
ruaok
we need it.
2016-03-01 06146, 2016
alastairp
ruaok: also, I had no indexes, pk, or fk during the insert, but I do now for the update
2016-03-01 06151, 2016
alastairp
perhaps that's related
2016-03-01 06157, 2016
alastairp
I wonder if there's an index on created
2016-03-01 06136, 2016
ruaok
that has a drastic impact on the speed of things.
2016-03-01 06150, 2016
ruaok
which is exactly why a re-migration might just be simpler.
2016-03-01 06152, 2016
alastairp
huh. there's not. that's actually a mistake, but I guess it'll be faster
2016-03-01 06159, 2016
ruaok
either that or drop the index and then re-created.
2016-03-01 06123, 2016
alastairp
anyway, update is 3x faster than migration, so I'll leave it running
2016-03-01 06137, 2016
alastairp
although I kinda hoped it would have been 10x faster :)
2016-03-01 06135, 2016
reosarevok
ruaok: thanks for the money! :)
2016-03-01 06143, 2016
reosarevok buys tickets for La traviata tomorrow
2016-03-01 06147, 2016
ruaok
:)
2016-03-01 06130, 2016
ruaok
me neither. forward to reosarevok? he's seems to deal well with these kinds of emails. :)
2016-03-01 06144, 2016
reosarevok shudders
2016-03-01 06119, 2016
alastairp
I don't even recognise the result format. is it search server json?
2016-03-01 06157, 2016
alastairp
reosarevok: email?
2016-03-01 06104, 2016
reosarevok
My nick at meb
2016-03-01 06115, 2016
rahulr joined the channel
2016-03-01 06119, 2016
ruaok
ok, time to run off to the EC. I even polished my shoes!!! :)
2016-03-01 06106, 2016
Gentlecat
good luck
2016-03-01 06113, 2016
ruaok
thanks.
2016-03-01 06120, 2016
alastairp
hmm. it's slowing down :-P
2016-03-01 06130, 2016
ruaok
I'mn tempted to plaster the EC meeting rooms with MB stickers. :)
2016-03-01 06137, 2016
alastairp
I'll leave it running for a few hours, and see if it's worth re-importing instead
2016-03-01 06102, 2016
ruaok
alastairp: keep me posted, please
2016-03-01 06106, 2016
ruaok is off
2016-03-01 06137, 2016
alastairp
oooh
2016-03-01 06122, 2016
alastairp drops the date column, creates a new table with just the date, then creates a new table as a join query on both tables
2016-03-01 06118, 2016
Gentlecat
💥
2016-03-01 06136, 2016
alastairp
it's still kinda slow
2016-03-01 06102, 2016
alastairp
acousticbrainz=> \copy ll_submitted from 'submitted_times.csv' with csv;
2016-03-01 06102, 2016
alastairp
COPY 3508050
2016-03-01 06106, 2016
alastairp
that wasn't slow!
2016-03-01 06111, 2016
alastairp
acousticbrainz=> create unlogged table lowlevel_correct_submitted as (select ll.id, ll.mbid, ll.build_sha1, ll.lossless, lls.submitted from lowlevel ll join ll_submitted lls on ll.id=lls.id);
2016-03-01 06111, 2016
alastairp
SELECT 3499808
2016-03-01 06113, 2016
alastairp
neither was that!
2016-03-01 06131, 2016
ruaok
Wow. Nice.
2016-03-01 06141, 2016
alastairp
I know, right. work smart, not hard
2016-03-01 06100, 2016
alastairp
recreated the table, dropped fks, renamed, recreated fks. just finished
2016-03-01 06109, 2016
alastairp
well, just adding one more fk
2016-03-01 06114, 2016
ruaok
Very good.
2016-03-01 06139, 2016
Freso
ruaok: I'll try and dig it up once Firefox has started up.
that thing where there isn't an index on the submitted date
2016-03-01 06159, 2016
Freso
I just wanted to get a quick update out to redirect people to the relevant wiki pages.
2016-03-01 06103, 2016
alastairp
and that's why counts ordered by date are slow
2016-03-01 06112, 2016
ruaok
Freso: I did, thank you for posting that.
2016-03-01 06133, 2016
Freso
ruaok: Yeah. I didn't think about it until too late, but figured it'd be better to push it out than wait for your feedback. Seems like a handful of people already made their way to IRC, so maybe a handful of people also already saw the blog post.
2016-03-01 06146, 2016
Freso
(Even if not the same ones.)
2016-03-01 06113, 2016
Gentlecat
alastairp: also all points are exactly an hour apart
