16:36 PM
samphippen
you'd call reportscrobbles
2015-09-18 26148, 2015
16:36 PM
samphippen
with the task again right?
2015-09-18 26151, 2015
16:36 PM
Craig has quit
2015-09-18 26153, 2015
16:36 PM
alastairp
yeah
2015-09-18 26158, 2015
16:36 PM
samphippen
as opposed to dispatch
2015-09-18 26159, 2015
16:36 PM
samphippen
I mean
2015-09-18 26100, 2015
16:37 PM
samphippen
at the end of the day
2015-09-18 26106, 2015
16:37 PM
samphippen
they get deduplicated at the server side
2015-09-18 26111, 2015
16:37 PM
samphippen
so it's probably not an issue right?
2015-09-18 26113, 2015
16:37 PM
Craig joined the channel
2015-09-18 26114, 2015
16:37 PM
alastairp
well
2015-09-18 26124, 2015
16:37 PM
alastairp
we have 2 clients sending us data as fast as they can
2015-09-18 26129, 2015
16:37 PM
alastairp
and we’re responding with http400
2015-09-18 26135, 2015
16:37 PM
alastairp
and it seems that they’re trying again
2015-09-18 26145, 2015
16:37 PM
weeksio_ has quit
2015-09-18 26118, 2015
16:38 PM
samphippen
ok
2015-09-18 26127, 2015
16:38 PM
samphippen
I'd put a settimeout
2015-09-18 26128, 2015
16:38 PM
alastairp
I’m just concerned that the for loop in dispatch can be called from many callbacks, causing multiple submissions of data in toReport
2015-09-18 26129, 2015
16:38 PM
samphippen
and retry in there
2015-09-18 26141, 2015
16:38 PM
samphippen
really
2015-09-18 26143, 2015
16:38 PM
samphippen
instead of looping it
2015-09-18 26146, 2015
16:38 PM
samphippen
you should pop each item
2015-09-18 26148, 2015
16:38 PM
samphippen
until it's empty
2015-09-18 26148, 2015
16:38 PM
samphippen
then
2015-09-18 26149, 2015
16:38 PM
alastairp
yeah
2015-09-18 26154, 2015
16:38 PM
alastairp
sounds good
2015-09-18 26155, 2015
16:38 PM
ruaok
one easy step is to not ever retry a submission that gets a 400 error.
2015-09-18 26156, 2015
16:38 PM
samphippen
that case can't happen
2015-09-18 26114, 2015
16:39 PM
alastairp
ok. what error handler will we see 400 in? onerror or onabort?
2015-09-18 26121, 2015
16:39 PM
samphippen
literally no idea
2015-09-18 26125, 2015
16:39 PM
alastairp
ok, thanks
2015-09-18 26127, 2015
16:39 PM
samphippen
:P
2015-09-18 26146, 2015
16:39 PM
alastairp
nice video, btw
2015-09-18 26106, 2015
16:42 PM
samphippen
thanks
2015-09-18 26107, 2015
16:42 PM
samphippen
:D
2015-09-18 26124, 2015
16:42 PM
samphippen
it's a hard life, being in chicago, appearing on the BBC, speaking at conferences etc
2015-09-18 26151, 2015
16:42 PM
Bookzombie has quit
2015-09-18 26104, 2015
16:43 PM
ruaok
say hi to your abused circadian rhythm for me. :)
2015-09-18 26108, 2015
16:43 PM
Bookzombie joined the channel
2015-09-18 26159, 2015
16:43 PM
Craig has quit
2015-09-18 26107, 2015
16:45 PM
alastairp
I have no idea what I’m doing
2015-09-18 26114, 2015
16:45 PM
Craig joined the channel
2015-09-18 26129, 2015
16:45 PM
alastairp
I would have thought that an onerror event would have a status field, or something
2015-09-18 26129, 2015
16:46 PM
Craig1 joined the channel
2015-09-18 26157, 2015
16:49 PM
Craig has quit
2015-09-18 26124, 2015
16:50 PM
github joined the channel
2015-09-18 26124, 2015
16:50 PM
github
2015-09-18 26124, 2015
16:50 PM
github
listenbrainz-server/master 6d8aa0c Robert Kaye: Do not throw 400 on empty payload
2015-09-18 26124, 2015
16:50 PM
github
listenbrainz-server/master 1629807 Robert Kaye: Merge branch 'master' of github.com:metabrainz/listenbrainz-server
2015-09-18 26124, 2015
16:50 PM
github has left the channel
2015-09-18 26148, 2015
16:50 PM
chirlu`
alastairp: jQuery?
2015-09-18 26104, 2015
16:51 PM
alastairp
no, xmlhttprequest
2015-09-18 26124, 2015
16:51 PM
Craig1 has quit
2015-09-18 26151, 2015
16:52 PM
chirlu`
It seems it has?
2015-09-18 26110, 2015
16:53 PM
chirlu`
XMLHttpRequest.status Read only
2015-09-18 26110, 2015
16:53 PM
chirlu`
Returns an unsigned short with the status of the response of the request. This is the HTTP result code (for example, status is 200 for a successful request).
2015-09-18 26103, 2015
16:57 PM
alastairp
yeah, I found that finally
2015-09-18 26110, 2015
16:57 PM
alastairp
so is that the argument of the callback?
2015-09-18 26128, 2015
17:00 PM
samphippen has quit
2015-09-18 26116, 2015
17:01 PM
chirlu`
Not sure what the callback receives, but you can access the original object in any case.
2015-09-18 26158, 2015
17:02 PM
alastairp
ah, ok
2015-09-18 26121, 2015
17:04 PM
chirlu`
2015-09-18 26121, 2015
17:04 PM
chirlu`
Invoke callback with five arguments, the first one having the value of E's message attribute, the second having the value of E's filename attribute, the third having the value of E's lineno attribute, the fourth having the value of E's colno attribute, the fifth having the value of E's error attribute, and with the callback this value set to E's currentTarget. Let return value be the callback's return value. [WEBIDL]
2015-09-18 26156, 2015
17:04 PM
chirlu`
2015-09-18 26104, 2015
17:08 PM
UmkaDK_ has quit
2015-09-18 26146, 2015
17:11 PM
JesseW has quit
2015-09-18 26127, 2015
17:14 PM
fractalizator has quit
2015-09-18 26132, 2015
17:18 PM
github joined the channel
2015-09-18 26132, 2015
17:18 PM
github
2015-09-18 26132, 2015
17:18 PM
github
listenbrainz-server/master a031040 Robert Kaye: Use ujson for the kafka consumer
2015-09-18 26132, 2015
17:18 PM
github has left the channel
2015-09-18 26158, 2015
17:20 PM
alastairp
russss: how do I update chef? (and therefore pull in pgbouncer)
2015-09-18 26147, 2015
17:38 PM
CatQuest
uh hi, should i stop importing my stuff? it's me
2015-09-18 26125, 2015
17:39 PM
CatQuest
in anycase, my browser is eating like 11 gigs already
2015-09-18 26128, 2015
17:40 PM
ruaok
heh.
2015-09-18 26140, 2015
17:40 PM
ruaok
we know of a bug, but have not pushed a fix yet
2015-09-18 26102, 2015
17:41 PM
alastairp
well done to the developers of your browser
2015-09-18 26116, 2015
17:43 PM
russss
alastairp: um, I'm not sure, RJ2 set that bit up and I don't know how he did it
2015-09-18 26136, 2015
17:43 PM
alastairp
ok. we might have just apt-get install’d pgbouncer and copied the configuration in
2015-09-18 26115, 2015
17:44 PM
russss
that's fine for the moment
2015-09-18 26117, 2015
17:47 PM
ruaok
russss: we're still having some performance issues.
2015-09-18 26122, 2015
17:47 PM
ruaok
got a moment to log in and look around?
2015-09-18 26132, 2015
17:47 PM
ruaok
some 504s, but overall the site is very slow.
2015-09-18 26140, 2015
17:47 PM
ruaok
and we just deployed pgbouncer.
2015-09-18 26144, 2015
17:47 PM
ruaok
no improvement
2015-09-18 26127, 2015
17:48 PM
russss
sure, you'll need to create me an account though :)
2015-09-18 26136, 2015
17:48 PM
ruaok
ok.
2015-09-18 26147, 2015
17:48 PM
Craig joined the channel
2015-09-18 26122, 2015
17:49 PM
ruaok
pmesg me an ssh key?
2015-09-18 26128, 2015
17:49 PM
russss
2015-09-18 26126, 2015
17:50 PM
Craig has quit
2015-09-18 26132, 2015
17:51 PM
ruaok
russss: give it a shot
2015-09-18 26140, 2015
17:51 PM
ruaok
user russ
2015-09-18 26147, 2015
17:51 PM
ruaok
2015-09-18 26159, 2015
17:51 PM
ruaok
yay
2015-09-18 26101, 2015
17:52 PM
russss
cool, I'm in
2015-09-18 26125, 2015
17:52 PM
russss
holy uwsgi batman
2015-09-18 26134, 2015
17:52 PM
ruaok
lol
2015-09-18 26140, 2015
17:52 PM
ruaok
you now have sudo privs
2015-09-18 26145, 2015
17:52 PM
russss
cool
2015-09-18 26147, 2015
17:52 PM
ruaok
ok, I need to run off for food.
2015-09-18 26152, 2015
17:52 PM
ruaok
thanks for helping russ. :)
2015-09-18 26100, 2015
17:53 PM
alastairp
and I’m heading home, I’ll be back in 10
2015-09-18 26112, 2015
17:53 PM
ruaok has quit
2015-09-18 26129, 2015
17:53 PM
willejs joined the channel
2015-09-18 26127, 2015
17:57 PM
CatQuest
alastairp: well it's literally cutting edge firefox so..
2015-09-18 26138, 2015
17:57 PM
CatQuest
cutting as in my space is minche mear :P
2015-09-18 26146, 2015
17:58 PM
CatQuest
meat*
2015-09-18 26140, 2015
18:00 PM
CatQuest
had 500 mg left, turned it off and now i'e got 3 gigs back
2015-09-18 26155, 2015
18:00 PM
CatQuest
3.8
2015-09-18 26121, 2015
18:01 PM
CatQuest
so yea. complain to firefox (not that that will fix anything :()
2015-09-18 26135, 2015
18:01 PM
CatQuest
now 4,88
2015-09-18 26136, 2015
18:01 PM
CatQuest
yea
2015-09-18 26150, 2015
18:01 PM
CatQuest
steadily releasing gigs at a time
2015-09-18 26130, 2015
18:06 PM
willejs has quit
2015-09-18 26108, 2015
18:12 PM
russss
can't get anything obvious, I'm tempted to hook it up to newrelic temporarily as that'll sort it out
2015-09-18 26127, 2015
18:12 PM
russss
although if I sign up for an account I'll have their sales people all over me for ever
2015-09-18 26142, 2015
18:16 PM
alastairp
I’m not seeing any HTTP504 in the nginx logs
2015-09-18 26102, 2015
18:17 PM
alastairp
so maybe it’s just settling down, or people aren’t submitting
2015-09-18 26140, 2015
18:18 PM
russss
stuff's still slow
2015-09-18 26157, 2015
18:18 PM
russss
check out "uwsgitop localhost:5050"
2015-09-18 26127, 2015
18:19 PM
russss
(am just about to restart it)
2015-09-18 26106, 2015
18:21 PM
alastairp
uwsgitop! I didn’t know that existed
2015-09-18 26135, 2015
18:21 PM
CatQuest
I stopped submitting if that helps
2015-09-18 26136, 2015
18:21 PM
russss
you have to install it from pip
2015-09-18 26154, 2015
18:21 PM
alastairp
CatQuest: many people submitting
2015-09-18 26109, 2015
18:22 PM
CatQuest
aha
2015-09-18 26124, 2015
18:22 PM
CatQuest
I just... because I have like 18k pages as you know :P
2015-09-18 26138, 2015
18:22 PM
alastairp
russss: we seem to have a problem when restarting wsgi doesn’t kill old child processes, I don’t think think that would be related though
2015-09-18 26114, 2015
18:23 PM
russss
I enabled the Werkzeug profiler, which is saying something like this
2015-09-18 26117, 2015
18:23 PM
russss
2015-09-18 26135, 2015
18:23 PM
russss
which is not *terribly* helpful as it doesn't tell you which socket
2015-09-18 26142, 2015
18:24 PM
russss
oh wait hang on no
2015-09-18 26154, 2015
18:24 PM
russss
urllib2 is your answer
2015-09-18 26104, 2015
18:25 PM
zxdert joined the channel
2015-09-18 26151, 2015
18:25 PM
alastairp
we use urllib to connect to messybrainz
2015-09-18 26114, 2015
18:26 PM
russss
if each call to messybrainz is taking 10ms and you've got 5000 calls to make, that would presumably do it
2015-09-18 26122, 2015
18:26 PM
russss
I think messybrainz needs a bulk lookup API
2015-09-18 26100, 2015
18:27 PM
alastairp
right
2015-09-18 26133, 2015
18:27 PM
russss
that would probably be a sensible thing, I have a suspicion that messybrainz may be stalling on something as well so I'll look at that
2015-09-18 26154, 2015
18:27 PM
alastairp
I checked indexes this morning, and it’s been a lot happier since
2015-09-18 26113, 2015
18:28 PM
alastairp
I wonder if for the scale of listenbrainz we should bypass http when connecting to messybrainz
2015-09-18 26125, 2015
18:28 PM
russss
might be sensible
2015-09-18 26128, 2015
18:28 PM
alastairp
abstract it into a library that connects to the database directly
2015-09-18 26132, 2015
18:28 PM
alastairp
since they
2015-09-18 26140, 2015
18:28 PM
alastairp
‘re both on the same machine for now
2015-09-18 26131, 2015
18:29 PM
russss
uwsgitop localhost:5051 is the messybrainz uwsgi
2015-09-18 26136, 2015
18:29 PM
russss
and it does seem pretty busy as well
2015-09-18 26138, 2015
18:30 PM
alastairp
it would be more than LB actually, since it responds to 50 queries per single LB query
2015-09-18 26157, 2015
18:35 PM
zxdert has left the channel
