(as of the dump on the 14th we had 623969 editors)
2012-07-18 20010, 2012
ianmcorvidae
hm, maybe that's not the peak then? unsure
2012-07-18 20016, 2012
ianmcorvidae
er, maybe that's not post-trim then
2012-07-18 20019, 2012
ianmcorvidae
eh, I'll check tomorrow
2012-07-18 20051, 2012
nikkimini joined the channel
2012-07-18 20019, 2012
warp
hrm. not working yet.
2012-07-18 20002, 2012
Freso joined the channel
2012-07-18 20008, 2012
ijabz joined the channel
2012-07-18 20013, 2012
underscor has left the channel
2012-07-18 20050, 2012
ijabz joined the channel
2012-07-18 20056, 2012
ijabz
So starts another day
2012-07-18 20046, 2012
djce joined the channel
2012-07-18 20042, 2012
stermi joined the channel
2012-07-18 20050, 2012
stermi
<stermi> it stops here
2012-07-18 20041, 2012
ianmcorvidae joined the channel
2012-07-18 20042, 2012
ianmcorvidae joined the channel
2012-07-18 20028, 2012
djce joined the channel
2012-07-18 20035, 2012
ocharles
warp: reviewing recaptcha now
2012-07-18 20009, 2012
ijabz
cherry1
2012-07-18 20014, 2012
ijabz
ffs
2012-07-18 20037, 2012
ocharles hopes that is not what he thinks it is :)
2012-07-18 20045, 2012
ocharles
wtf. our daily cron job takes over 4 hours to run
2012-07-18 20051, 2012
ocharles
am I really reading that correctly?
2012-07-18 20011, 2012
ocharles
starts at "Wed Jul 18 00:10:02 UTC 2012", finishes at "Wed Jul 18 04:22:16 UTC 2012"
2012-07-18 20025, 2012
nikkimini
isn't that including the full dump?
2012-07-18 20050, 2012
ocharles
yes, but that is pretty quick
2012-07-18 20041, 2012
nikkimini
it is? I've noticed my subscriptions email comes late on full dump days
2012-07-18 20012, 2012
ocharles
hmm, not quite as slow as i thought
2012-07-18 20017, 2012
ocharles
but it still takes an hour before the dump even starts
2012-07-18 20040, 2012
ocharles
as fast as i thought*
2012-07-18 20050, 2012
nikkimini
like today was 05:50, yesterday was 02:52, day before was 02:54, day before that was 02:53...
2012-07-18 20008, 2012
nikkimini
day before that was 06:43
2012-07-18 20054, 2012
nikkimini
subscriptions seem to take a while. it's ~25 minutes between the two I get
2012-07-18 20032, 2012
djce joined the channel
2012-07-18 20012, 2012
ianmcorvidae joined the channel
2012-07-18 20035, 2012
ijabz
btw ocharles let me clarify something, I don't disagree with a new NES or the goals of it, i just don't think its the top priority task
2012-07-18 20053, 2012
ijabz
and I don't see starting with a database layer that helpful to creating NES either, the input/ouput of NES is going to be the webservice (i.e submit edit, view edit) so it makes more sense to me to
2012-07-18 20044, 2012
ijabz
consider how to implement these tasks and break them down, rather than created some database calls that amy not ultimately be what is required
2012-07-18 20010, 2012
ijabz
Which is one of may reasons why I think the most useful thing we could do is separate the website from the web service AND get website to use the web service for all database access
2012-07-18 20035, 2012
ocharles
yes, but that's a very large amount of work to take in one go
2012-07-18 20042, 2012
ocharles
i don't think we can coordinate that much work at once
2012-07-18 20055, 2012
ocharles
so what I want to do is keep the website the same, but change how the Data::* section works
2012-07-18 20015, 2012
ocharles
then once that is done, and the website is talking to a NES backed database, we can change the website to start talking to our public web service
2012-07-18 20033, 2012
ocharles
I don't plan to start writing database methods that won't be called, i agree - that's fairly pointless
2012-07-18 20055, 2012
ijabz
The 'very large amount of work' argument is a different one, its not your main drive though is it. i.e if we had masses of developers which way would you approach it ?
2012-07-18 20040, 2012
ocharles
probably still the same way
2012-07-18 20003, 2012
ocharles
but that seems irrelevant, we don't have that, and we're not going to, so why choose a strategy that doesn't apply to us?
2012-07-18 20027, 2012
ocharles
I'm interested in talking small progressive steps to nirvana, not rewriting everything and experiencing NGS again
2012-07-18 20052, 2012
ijabz
Well, because there are easy ways round that. You could create a new web service layer from scratch, and just code the NES stuff from web service layer down to db as stage 1
2012-07-18 20007, 2012
ijabz
stage2:website talks to new webservice
2012-07-18 20023, 2012
ijabz
stage 3:merge new web service with old
2012-07-18 20029, 2012
djce joined the channel
2012-07-18 20012, 2012
ocharles
that seems to require that we have a design for /ws/3 as well, no?
2012-07-18 20044, 2012
ijabz
Of course you don't plan to start writing database methods that won't be called, but it seems very difficult to avoid working bottom up
2012-07-18 20036, 2012
ijabz
No, i dont see that has to be done upfront
2012-07-18 20053, 2012
ocharles
how can we write a web service if we don't know what the web service is?
2012-07-18 20022, 2012
ijabz
How is this different to the website talking to a NES backed database
2012-07-18 20018, 2012
ijabz
Im saying you don't need to define a whole ws/3 upfront, but clearly you need to think about how you want the website to interact with the database/webservice either way
2012-07-18 20042, 2012
ocharles
the website is already defined in terms of some general operations, so we're just reimplementing them with a NES database, rather than the NGS database
2012-07-18 20013, 2012
ocharles
I don't like the idea of writing an ultimately public facing web service without a clear design first. building it adhoc in terms of what we need seems like a recipe for yet another poorly scaling WS
2012-07-18 20053, 2012
ijabz
The idea of writing a new data layer that the website talks to without considering the web service that the website is meant to talk to doesn't sound like an efficient use of resources to me
2012-07-18 20007, 2012
ijabz
re ws/2 scalability it would be good to have this documented somewhere, other than the problems with a few artists called with a particular set of inc arguments I'm not aware of all what all the other problems are
2012-07-18 20041, 2012
ocharles
ijabz: the fact that it's near impossible to set Last-Modified
2012-07-18 20050, 2012
ocharles
that's more my bigger worry
2012-07-18 20030, 2012
ijabz
right, can see that issue, would be interested in what the alternative solution is you have in mind, i know you want to get rid of incs, but not sure what the replacement is you have in mind
2012-07-18 20058, 2012
ocharles
nor am I, because I figured I'd focus on building NES first :)
2012-07-18 20030, 2012
ocharles
i personally think even trying to do the editable web service at the time is a bad idea and we're biting off more than we can chew with even just NES
2012-07-18 20039, 2012
ocharles
at the same time*
2012-07-18 20024, 2012
ijabz
I thought editable web service was on hold until NES had progressed
2012-07-18 20011, 2012
ocharles
warp's work on JSON for /ws/2 is part of editable ws
2012-07-18 20044, 2012
ocharles
and we seem to have lost warp so I better adopt this recaptcha work
2012-07-18 20017, 2012
ijabz
Well so far there is nothing editable about it, it long been requested to provide ws output as json so I see no conflict
2012-07-18 20031, 2012
ocharles
there is value in providing json
2012-07-18 20043, 2012
ocharles
but the motivation is because we will only support writing via json, thus we want read/write symmetry
2012-07-18 20053, 2012
ijabz
But that is irrelevant, the read still need to be done, and is not causing aproblem
2012-07-18 20047, 2012
ijabz
in fact the NES roadmap you've laid out seems to create an awful lot of bottlenecks , so no other major work can be done until you've finished
2012-07-18 20005, 2012
ijabz
i.e editable web service, separate website from web service, move way from perl
2012-07-18 20039, 2012
ocharles
yes, i see personally see NES as more important than any of those
2012-07-18 20043, 2012
ocharles
editable WS is very close though
2012-07-18 20009, 2012
ocharles
but i don't see value in an editable WS that isn't on NES, when we plan to have NES so soon. introducing an api then almost immediately deprecating it is not very social
2012-07-18 20002, 2012
djce joined the channel
2012-07-18 20017, 2012
ijabz
taking your small progressive steps analogy, working now on editable ws based on current system, and then updating it to support the new features of new seems reasonable to me
2012-07-18 20040, 2012
ijabz
the editable web service doesn't have to actually be released and made public until works with NES
2012-07-18 20003, 2012
ocharles
what's the point in writing it until NES then?
2012-07-18 20042, 2012
ocharles
warp: ok, my suggestion for HTML::FHX::Field::reCAPTCHA is not working out. throwing that work away and shipping yours
2012-07-18 20029, 2012
ijabz
small steps, parallel working, the lapsed time to delivery is going to be much longer if work isn't started until you've finished NES
2012-07-18 20030, 2012
ocharles
my small steps is about small *deliverable* steps
2012-07-18 20000, 2012
lfranchi joined the channel
2012-07-18 20002, 2012
ocharles
and if you can't deliver something until something else is done, you might as well wait for it to be done. sure you could do 'the rest' while waiting, but there isn't really anything that depends on NES
2012-07-18 20013, 2012
ocharles
plus, it's entirely possible i haven't got NES right and it has to be refined
2012-07-18 20022, 2012
ijabz
'there isn't really anything that depends on NES' now Im confused, whats the problem then ?
2012-07-18 20022, 2012
ocharles
warp, you haven't commited register.js :(
2012-07-18 20000, 2012
ocharles
anything else*
2012-07-18 20013, 2012
ocharles
i meant there is hardly anything to write about the editable web service without NES
2012-07-18 20021, 2012
ocharles
the entire service builds on it
2012-07-18 20019, 2012
ijabz
I don't see that, we have all the web service side for a start
2012-07-18 20044, 2012
ijabz
Anyway, can't spend all day chatting of for some food
2012-07-18 20052, 2012
warp
hello!
2012-07-18 20004, 2012
djce joined the channel
2012-07-18 20035, 2012
djce1 joined the channel
2012-07-18 20048, 2012
warp
ocharles: the recaptcha branch is supposed to remove register.js
2012-07-18 20052, 2012
ocharles
warp: it didn't, but it does now
2012-07-18 20057, 2012
ocharles
i'm just about to deploy it
2012-07-18 20023, 2012
the_metalgamer joined the channel
2012-07-18 20035, 2012
warp
cool
2012-07-18 20051, 2012
warp
did you replace the REMOTE_ADDR stuff with $c->req->address ?
2012-07-18 20011, 2012
ocharles
yea
2012-07-18 20016, 2012
ocharles
it seems to work on test
2012-07-18 20028, 2012
warp
good :)
2012-07-18 20024, 2012
warp
cool, I got a google streetview building/house number in the captcha
2012-07-18 20055, 2012
ocharles
lol
2012-07-18 20007, 2012
warp
ocharles: tested entering an incorrect and correct captcha, both seem to work as expected on test.musicbrainz.org.
