10:17 AM
ocharles
I'm going to go down the no email means it's all ok route
2011-08-04 21643, 2011
10:17 AM
djce
k
2011-08-04 21655, 2011
10:18 AM
reosarevok joined the channel
2011-08-04 21615, 2011
10:22 AM
djce
oh, ocharles can I borrow you for 3 min on a different topic? ping me when ready....
2011-08-04 21621, 2011
10:22 AM
ocharles
djce: fire away
2011-08-04 21631, 2011
10:22 AM
djce
2011-08-04 21637, 2011
10:22 AM
djce
I updated the spec on that ticket
2011-08-04 21650, 2011
10:22 AM
voiceinsideyou joined the channel
2011-08-04 21652, 2011
10:22 AM
djce
and I think it'll mainly (probably only) affect the MBS codebase in fact.
2011-08-04 21616, 2011
10:23 AM
ocharles
yay, md5sums
2011-08-04 21625, 2011
10:23 AM
djce
So some combination of a FYI, and a "please shout if you see a problem",
2011-08-04 21635, 2011
10:23 AM
djce
and my next task is to estimate how long that'll take
2011-08-04 21637, 2011
10:23 AM
ocharles
Yea, it looks like it's just MBS
2011-08-04 21652, 2011
10:23 AM
djce
so I can keep my boss updated as to how long I'm bunking off BBC work to do MBH instead :-)
2011-08-04 21608, 2011
10:24 AM
ocharles
:)
2011-08-04 21620, 2011
10:24 AM
djce
I'm still unclear as to where I'll be testing this, alas.
2011-08-04 21631, 2011
10:24 AM
djce
Might have to resurrect replication packet emitting on hobbes.
2011-08-04 21640, 2011
10:24 AM
djce
but I have no access to any MB slaves :-(
2011-08-04 21600, 2011
10:25 AM
ocharles
hobbes is pretty much ready for that I'd think
2011-08-04 21627, 2011
10:25 AM
djce
(well.... hmm... I could use pretty much any machine in fact if I just mock the bit where it loads the packet into the database I suppose).
2011-08-04 21646, 2011
10:25 AM
djce
ok. Thanks. that's all for now, just wanted to keep you informed mainly.
2011-08-04 21610, 2011
10:26 AM
djce
(damn, 10 sec over my 3 minute chat estimate ;-)
2011-08-04 21617, 2011
10:26 AM
ocharles
lol
2011-08-04 21635, 2011
10:53 AM
jdamcd joined the channel
2011-08-04 21646, 2011
10:56 AM
voiceinsideyou joined the channel
2011-08-04 21612, 2011
11:02 AM
cjk32 joined the channel
2011-08-04 21659, 2011
11:02 AM
cjk32
ISE on the release editor, MBS-3165 if someone wants to capture stack traces etc from the error log.
2011-08-04 21616, 2011
11:18 AM
ocharles
Sigh, we're getting 502s again it seems
2011-08-04 21648, 2011
11:28 AM
jdamcd joined the channel
2011-08-04 21600, 2011
11:34 AM
voiceinsideyou joined the channel
2011-08-04 21609, 2011
12:00 PM
warp
goodmorning!
2011-08-04 21618, 2011
12:02 PM
warp
cjk32: regarding the session timeout, it's probably better to submit what you have, and add an editnote saying you will add the remaining discs later.
2011-08-04 21635, 2011
12:02 PM
warp
still, it shoudln't "internal server error" ofcourse
2011-08-04 21656, 2011
12:22 PM
voiceinsideyou joined the channel
2011-08-04 21628, 2011
13:34 PM
voiceinsideyou joined the channel
2011-08-04 21620, 2011
13:53 PM
ocharles
PostgreSQL indexing and plperl just blew me away
2011-08-04 21631, 2011
13:53 PM
ocharles
I can find all edit relationship edits that use a certain relationship type in....
2011-08-04 21633, 2011
13:53 PM
ocharles
0.36ms
2011-08-04 21652, 2011
13:58 PM
luks
planning to use plperl in MB?
2011-08-04 21628, 2011
14:03 PM
ocharles
only for this json searching
2011-08-04 21647, 2011
14:03 PM
ocharles
i can't see any other way to query it fast otherwise
2011-08-04 21650, 2011
14:03 PM
ocharles
2011-08-04 21608, 2011
14:06 PM
bitmap_ joined the channel
2011-08-04 21651, 2011
14:14 PM
adhawkins
Does the android app use /ws/2 or /ws/1?
2011-08-04 21638, 2011
14:17 PM
adhawkins
If I search for release 'Aerial' and select the Kate Bush one, it only lists 5 matches, all for 'A Sky of Honey'. None for the other disc in the release
2011-08-04 21643, 2011
14:20 PM
adhawkins
The matches all have 16 tracks, whereas in reality it's two discs, one with 7, the other with 9.
2011-08-04 21658, 2011
15:30 PM
ocharles
2011-08-04 21613, 2011
15:32 PM
ocharles
2011-08-04 21625, 2011
15:32 PM
ocharles
echo "SELECT plainto_tsquery('mb_simple', '☺')" | ./admin/psql READWRITE just gives "NOTICE: text-search query contains only stop words or doesn't contain lexemes, ignored"
2011-08-04 21608, 2011
15:35 PM
warp
ocharles: when in the release editor you add a track to a VA release, the artist credit for that track will be empty.
2011-08-04 21637, 2011
15:35 PM
warp
ocharles: if you submit that you will get a warning that the track needs an artist
2011-08-04 21626, 2011
15:36 PM
warp
ocharles: but without these changes you can never submit a new artist, because while iterating over each credit 'current' will be falsy (because it is empty) for each credit.
2011-08-04 21654, 2011
15:36 PM
ocharles
hrm
2011-08-04 21616, 2011
15:37 PM
luks
ocharles: not very easy to fix, we always had that problem (mainly with !!! :))
2011-08-04 21619, 2011
15:37 PM
ocharles
well, I don't really get it but if it works the rest looks ok :)
2011-08-04 21623, 2011
15:37 PM
ocharles
luks: yea
2011-08-04 21641, 2011
15:37 PM
ocharles
I wondered if there was a way to just do a direct text comparison as well
2011-08-04 21601, 2011
15:38 PM
warp
ocharles: hrm, perhaps it would be cleaner and clearer to just compare to.names.length with from.names.length
2011-08-04 21628, 2011
15:38 PM
luks
well, you can do "... OR artist_name.name = ?" :)
2011-08-04 21630, 2011
15:38 PM
warp
ocharles: before the $.each() over to.names.
2011-08-04 21647, 2011
15:38 PM
ocharles
luks: yea, that's probably what I will be doing
2011-08-04 21653, 2011
15:38 PM
luks
but it definitely doesn't look like it needs to get fixed in 2011-08-08
2011-08-04 21655, 2011
15:38 PM
ocharles
but that should presumably rank at the very top too
2011-08-04 21600, 2011
15:39 PM
ocharles
well, it's in there :)
2011-08-04 21607, 2011
15:39 PM
luks
warp put it there
2011-08-04 21607, 2011
15:39 PM
ocharles
I asked for other work yesterday, and didn't really get any
2011-08-04 21611, 2011
15:39 PM
ocharles
yea
2011-08-04 21612, 2011
15:39 PM
luks
it's a very minor issue
2011-08-04 21629, 2011
15:39 PM
warp wouldn't consider it minor.
2011-08-04 21643, 2011
15:39 PM
luks
MB had it since the beginning
2011-08-04 21603, 2011
15:40 PM
ocharles
2011-08-04 21603, 2011
15:40 PM
warp
sure, that doesn't make it any less serious :)
2011-08-04 21606, 2011
15:40 PM
luks
if we lived with it for 10 years, it can wait if there is more important work (which I'm sure there is)
2011-08-04 21626, 2011
15:40 PM
luks
ocharles: all ModBot errors fixed? :)
2011-08-04 21632, 2011
15:40 PM
ocharles
all known ones
2011-08-04 21605, 2011
15:41 PM
ocharles
not that it's too easy to filter a list of 700 tickets I have assigned to me :P
2011-08-04 21608, 2011
15:41 PM
warp
luks: ofcourse to some extent minor/critical is subjective. I have music my this particular artist in /mnt/music/tagged, so I'd more likely to be hit by it than most people :)
2011-08-04 21615, 2011
15:44 PM
luks
ocharles: go by the number of votes
2011-08-04 21633, 2011
15:44 PM
ocharles
2011-08-04 21656, 2011
15:44 PM
ocharles
but I get told off for not working off my CI list :)
2011-08-04 21603, 2011
15:45 PM
luks
working on that search thing really seems like a waste of time to me
2011-08-04 21613, 2011
15:45 PM
ocharles
yep
2011-08-04 21641, 2011
15:45 PM
luks
or just adding to the list of exceptions, which currently contains only !!!
2011-08-04 21647, 2011
15:45 PM
luks
add
2011-08-04 21609, 2011
15:46 PM
ocharles
we only do that for indexed search atm, not direct searches
2011-08-04 21631, 2011
15:46 PM
luks
oh, I thought the direct search also used it
2011-08-04 21633, 2011
15:47 PM
warp
but it really should be much work to allow the direct search to do exact matches as well.
2011-08-04 21637, 2011
15:47 PM
warp
+not
2011-08-04 21609, 2011
15:48 PM
warp
ocharles: if you don't feel like working on it, assign it to me and I'll look into it.
2011-08-04 21622, 2011
15:48 PM
ocharles
i don't mind working on it, I just kinda agree that we have more pressing things
2011-08-04 21640, 2011
15:48 PM
ocharles
2011-08-04 21618, 2011
15:49 PM
warp
you cannot assign this artist to tracks and recordings, etc.. at all now. that seems pretty important.
2011-08-04 21605, 2011
15:50 PM
kepstin-laptop joined the channel
2011-08-04 21622, 2011
15:50 PM
bitmap joined the channel
2011-08-04 21639, 2011
16:05 PM
ocharles
right, time to wander home
2011-08-04 21608, 2011
16:54 PM
ruaok joined the channel
2011-08-04 21602, 2011
17:03 PM
voiceinsideyou1 joined the channel
2011-08-04 21654, 2011
17:07 PM
ocharles
ruaok: we're getting a lot of 502 errors the last few days
2011-08-04 21610, 2011
17:08 PM
ruaok
yes, I've been hearing that.
2011-08-04 21613, 2011
17:08 PM
ocharles
ah, ok
2011-08-04 21614, 2011
17:08 PM
ruaok
not sure what to do about it.
2011-08-04 21631, 2011
17:08 PM
ruaok
heard it on twitter, actually. ;)
2011-08-04 21639, 2011
17:08 PM
ocharles
yea, I saw that
2011-08-04 21644, 2011
17:08 PM
ocharles
but also saw it mentioned in IRC the day before
2011-08-04 21654, 2011
17:08 PM
ocharles
I thought it was due to having an open psql shell to totoro, but that's apparently not it
2011-08-04 21606, 2011
17:09 PM
ruaok
why would that do it?
2011-08-04 21617, 2011
17:09 PM
ocharles
connection count
2011-08-04 21619, 2011
17:09 PM
ruaok
the backend servers are all happy. nothing out of sorts when what I saw.
2011-08-04 21627, 2011
17:09 PM
voiceinsideyou joined the channel
2011-08-04 21627, 2011
17:09 PM
ruaok
our connection count is nuts.
2011-08-04 21629, 2011
17:09 PM
ocharles
before I've accidently left a transaction open too, and that wrecked a bit of havoc
2011-08-04 21636, 2011
17:09 PM
ocharles
but that wasn't the problem this time (there was no transaction open)
2011-08-04 21641, 2011
17:09 PM
ruaok
*that* will do it.
2011-08-04 21604, 2011
17:10 PM
ruaok
but once you start hitting your connection limit, closing one connection won't help.
2011-08-04 21613, 2011
17:10 PM
ocharles
right
2011-08-04 21629, 2011
17:10 PM
ruaok
closing a 100 connections might have a positive effect.
2011-08-04 21641, 2011
17:10 PM
ocharles
it was only a wild guess, I doubt that's what it was
2011-08-04 21644, 2011
17:10 PM
ruaok
I think I will restart the front end proxy and see what that will do.
2011-08-04 21650, 2011
17:10 PM
ocharles
I think it was just coincidence 502s stopped briefly when I quit the shell
2011-08-04 21658, 2011
17:10 PM
ruaok nods
2011-08-04 21602, 2011
17:11 PM
reosarevokphone joined the channel
2011-08-04 21615, 2011
17:11 PM
ruaok
to diagnose 502s, here is your process:
2011-08-04 21635, 2011
17:11 PM
ruaok
1. Are the back ends happy? if not, the front end requests are piling up for some reason. thus 502s.
2011-08-04 21638, 2011
17:11 PM
warp pays attention.
2011-08-04 21650, 2011
17:11 PM
ocharles
how do I measure happiness?
2011-08-04 21656, 2011
17:11 PM
ocharles
tail the logs?
2011-08-04 21659, 2011
17:11 PM
ruaok
you look at server loads at the first clue.
2011-08-04 21606, 2011
17:12 PM
ocharles
ok
2011-08-04 21609, 2011
17:12 PM
ruaok
then you start looking at error/access logs
2011-08-04 21620, 2011
17:12 PM
ruaok
but the mrtg graphs are your first line of insight.
2011-08-04 21643, 2011
17:12 PM
ruaok
if the back ends are happy its likely that the proxy itself is having issues.
2011-08-04 21656, 2011
17:12 PM
ruaok
which is kinda hard to diagose at times.
2011-08-04 21607, 2011
17:13 PM
ruaok
but looking at the logs is the first thing to do.
2011-08-04 21610, 2011
17:13 PM
ocharles
other than just kick it and see what happens :)
2011-08-04 21619, 2011
17:13 PM
warp
ruaok: server load average should be < # of cores in the system?
2011-08-04 21629, 2011
17:13 PM
ruaok
ocharles: that used to be my preferred way of doing things, but rather ineffective. ;)
2011-08-04 21634, 2011
17:13 PM
ocharles
heh
2011-08-04 21638, 2011
17:13 PM
ruaok
warp: yeah.
2011-08-04 21651, 2011
17:13 PM
ruaok
given that, astro and pingu sitting at load 6 is cake.
2011-08-04 21607, 2011
17:14 PM
ruaok
totoro at 2... he's nearly napping he's so bored.
2011-08-04 21644, 2011
17:14 PM
ocharles
oh, talking of totoro, poke me when you're free later - need to discuss something with you
2011-08-04 21611, 2011
17:15 PM
nikki
totoro napping seems so fitting :P
2011-08-04 21634, 2011
17:15 PM
ocharles
haha
2011-08-04 21642, 2011
17:15 PM
ruaok
ocharles: after coffee. ;)
2011-08-04 21645, 2011
17:15 PM
ruaok
2011/08/04 17:15:24 [warn] 26957#0: *1590920301 an upstream response is buffered to a temporary file
2011-08-04 21648, 2011
17:15 PM
ocharles
sure
2011-08-04 21650, 2011
17:15 PM
ruaok
we're seeing too many of those.
2011-08-04 21625, 2011
17:16 PM
ruaok
did we just push out something that made the size of the /ws/1/release responses bigger?
2011-08-04 21657, 2011
17:20 PM
ruaok
ok, I'll keep tailing the logs for 502s.
2011-08-04 21617, 2011
17:23 PM
nikki prods ocharles
