0:13 AM
yeeeargh has quit
2015-10-25 29845, 2015
1:13 AM
Nyanko-sensei has quit
2015-10-25 29858, 2015
1:33 AM
Nyanko-sensei joined the channel
2015-10-25 29803, 2015
1:34 AM
Nyanko-sensei has quit
2015-10-25 29830, 2015
1:34 AM
Nyanko-sensei joined the channel
2015-10-25 29812, 2015
2:34 AM
JonnyJD joined the channel
2015-10-25 29850, 2015
3:17 AM
JonnyJD has quit
2015-10-25 29800, 2015
3:39 AM
ariZon_a has quit
2015-10-25 29850, 2015
4:21 AM
Gentlecat has quit
2015-10-25 29850, 2015
5:02 AM
ariZon_a joined the channel
2015-10-25 29806, 2015
6:10 AM
emitattuo joined the channel
2015-10-25 29821, 2015
6:49 AM
yeeeargh joined the channel
2015-10-25 29812, 2015
7:23 AM
JesseW joined the channel
2015-10-25 29852, 2015
7:30 AM
JesseW has quit
2015-10-25 29835, 2015
7:42 AM
emitattuo has quit
2015-10-25 29809, 2015
8:07 AM
JonnyJD joined the channel
2015-10-25 29859, 2015
8:23 AM
ariscop has quit
2015-10-25 29816, 2015
8:43 AM
yeeeargh has quit
2015-10-25 29808, 2015
8:46 AM
yeeeargh joined the channel
2015-10-25 29805, 2015
8:57 AM
ruaok joined the channel
2015-10-25 29844, 2015
9:01 AM
samphippen joined the channel
2015-10-25 29809, 2015
9:04 AM
CatCat
444444444444444444tyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy.0
2015-10-25 29833, 2015
9:05 AM
CatQuest
sorry guys, that was my cat
2015-10-25 29844, 2015
9:05 AM
CatQuest
I've disconected that keyboard now so
2015-10-25 29810, 2015
9:06 AM
CatQuest
currently sitting on that keyboard
2015-10-25 29813, 2015
9:06 AM
CatQuest
jeeesh
2015-10-25 29857, 2015
9:06 AM
CatQuest
christmas is going to be a hell with this one >__<
2015-10-25 29816, 2015
9:17 AM
zas
Good moning
2015-10-25 29821, 2015
9:17 AM
zas
morning*
2015-10-25 29831, 2015
9:17 AM
zas
damn i need another coffee already
2015-10-25 29843, 2015
9:17 AM
zas
ruaok: check my mail about search servers
2015-10-25 29844, 2015
9:18 AM
CatQuest
zas: good moaning!
2015-10-25 29816, 2015
9:19 AM
CatQuest
as zas is french he perhaps doesn't know the "Allo, allo"
2015-10-25 29828, 2015
9:19 AM
ruaok
first note, global limit is at 1500, which is why things might be stable.
2015-10-25 29833, 2015
9:19 AM
ruaok
should I move to 2000?
2015-10-25 29856, 2015
9:25 AM
zas
ruaok: i would first make roobarb behave more like dora, the server is restart far too often imho
2015-10-25 29810, 2015
9:26 AM
zas
restarted*
2015-10-25 29829, 2015
9:26 AM
ruaok
ok
2015-10-25 29853, 2015
9:26 AM
ruaok
I think you have some good points in the email. the server didn't use to start up so slowly.
2015-10-25 29858, 2015
9:26 AM
zas
and i think the issue starts when the two servers are restarted within the same time window
2015-10-25 29815, 2015
9:27 AM
ruaok
I've been thinking the same.
2015-10-25 29832, 2015
9:27 AM
ruaok
another thing that I think we ought to do is to warm up the servers a little.
2015-10-25 29837, 2015
9:27 AM
Leftmost
ruaok, can you provide any idea of the cost of a week in BCN?
2015-10-25 29842, 2015
9:27 AM
ruaok
do at least one search on each of the indexes.
2015-10-25 29812, 2015
9:28 AM
ruaok
Leftmost: in the context of the summit, metabrainz will pay for nearly everything.
2015-10-25 29834, 2015
9:28 AM
ruaok
outside of that you're on your own, save for lodging which is still covered.
2015-10-25 29842, 2015
9:28 AM
zas_ joined the channel
2015-10-25 29846, 2015
9:28 AM
Leftmost
Well that's handy. Thanks.
2015-10-25 29859, 2015
9:28 AM
ruaok
food is likely to be your greatest cost outside of the summit.
2015-10-25 29828, 2015
9:29 AM
zas
ruaok: what do you think about the rate limiter thing ? wouldn't be safer to enable it by default instead of relying on this wget remote command ?
2015-10-25 29836, 2015
9:29 AM
ruaok
transport is cheap, if you decide to take bus/metro. If you buy a T-10 card, you get 10 trips for 10€.
2015-10-25 29849, 2015
9:29 AM
ruaok
zas: yes, let's do that
2015-10-25 29806, 2015
9:31 AM
ruaok
Leftmost: a cheap dinner will cost 10€. A nice dinner 20€. Really nice with wine, 30€.
2015-10-25 29848, 2015
9:31 AM
ruaok
Leftmost: got a pic of the stickers yet?
2015-10-25 29856, 2015
9:31 AM
Gentlecat joined the channel
2015-10-25 29850, 2015
9:32 AM
Leftmost
Oh, yeah. Let me do that shortly here.
2015-10-25 29809, 2015
9:33 AM
Leftmost
There was also a pin and two stickers with the logo of the company.
2015-10-25 29848, 2015
9:39 AM
zas
ruaok: can we just copy tomcat-8.0.28 dir from dora to roobarb and symlink it ? or there is more to do ?
2015-10-25 29826, 2015
9:40 AM
ruaok
or grab the dist of tomcat and unzip it into place
2015-10-25 29842, 2015
9:40 AM
ruaok
then jar xf <war file> into webapps/root
2015-10-25 29813, 2015
9:41 AM
zas
where is the war file ?
2015-10-25 29832, 2015
9:41 AM
ruaok
2015-10-25 29852, 2015
9:41 AM
Leftmost
Thanks, ruaok.
2015-10-25 29801, 2015
9:42 AM
ruaok
this is going to be the tricky thing for us -- anyone else who uses that isn't going to want to use the rate limiter.
2015-10-25 29827, 2015
9:42 AM
regagain joined the channel
2015-10-25 29830, 2015
9:43 AM
zas
then let's enabled it for us only, and ignore the PR
2015-10-25 29854, 2015
9:43 AM
ruaok
a chef script for installing would be lovely. :)
2015-10-25 29825, 2015
9:46 AM
zas
yes ... but first i need to understand how it is installed ;)
2015-10-25 29842, 2015
9:46 AM
ruaok
those two steps are literally it.
2015-10-25 29856, 2015
9:46 AM
ruaok
unwar, chown to tomcat.
2015-10-25 29830, 2015
9:50 AM
regagain has quit
2015-10-25 29827, 2015
10:21 AM
ariscop joined the channel
2015-10-25 29833, 2015
10:40 AM
CatQuest has quit
2015-10-25 29843, 2015
10:43 AM
CatQuest joined the channel
2015-10-25 29840, 2015
11:10 AM
zas_ has quit
2015-10-25 29844, 2015
11:17 AM
regagain joined the channel
2015-10-25 29835, 2015
11:39 AM
Freso
2015-10-25 29802, 2015
11:40 AM
Freso
Gentlecat: How hard do you think it would be to fix ^ ?
2015-10-25 29828, 2015
11:42 AM
Gentlecat
I thought we fixed it a long time ago
2015-10-25 29843, 2015
11:42 AM
Gentlecat
apparently not
2015-10-25 29856, 2015
11:43 AM
Freso
Gentlecat: It is fixed once the db has been synced.
2015-10-25 29807, 2015
11:44 AM
Freso
In ~20 minutes the AB page will work fine.
2015-10-25 29821, 2015
11:44 AM
Gentlecat
still shouldn't fail
2015-10-25 29832, 2015
11:44 AM
Freso
Yeah.
2015-10-25 29852, 2015
11:44 AM
Freso
It should just use the submitted file metadata until the data is available via MB's db.
2015-10-25 29815, 2015
11:45 AM
alastairp
Gentlecat: db.cache is whack
2015-10-25 29818, 2015
11:45 AM
Freso
(The SAR was added to MB ~10 min ago.)
2015-10-25 29850, 2015
11:45 AM
Gentlecat
alastairp: what does that mean?
2015-10-25 29802, 2015
11:46 AM
alastairp
set returns True, but if I call get on the exact same input, I only get back about 10% of the stuff I put in
2015-10-25 29816, 2015
11:49 AM
Gentlecat
you mean set_multi and get_multi?
2015-10-25 29856, 2015
11:49 AM
Gentlecat
because set_multi shouldn't return a boolean
2015-10-25 29807, 2015
11:50 AM
alastairp
no, I’m using set individually
2015-10-25 29810, 2015
11:50 AM
Gentlecat
it's supposed to be a list of items which failed to be stored
2015-10-25 29812, 2015
11:50 AM
Gentlecat
ok
2015-10-25 29828, 2015
11:51 AM
alastairp
ok. set_multi returned []
2015-10-25 29816, 2015
11:52 AM
Gentlecat
you can try using memcached module directly
2015-10-25 29836, 2015
11:52 AM
Gentlecat
our wrapper just adds namespaces
2015-10-25 29853, 2015
11:52 AM
alastairp
oh
2015-10-25 29855, 2015
11:52 AM
alastairp
one sec
2015-10-25 29843, 2015
11:53 AM
alastairp
weirf
2015-10-25 29851, 2015
11:54 AM
alastairp
the keys which I’m setting are not uuids
2015-10-25 29858, 2015
11:54 AM
alastairp
but they were a few commands ago
2015-10-25 29840, 2015
11:55 AM
alastairp
yeah. what
2015-10-25 29802, 2015
11:56 AM
Nyanko-sensei has quit
2015-10-25 29820, 2015
11:56 AM
alastairp
oh
2015-10-25 29824, 2015
11:56 AM
alastairp
does your _prep_dict(mapping, namespace)
2015-10-25 29834, 2015
11:56 AM
alastairp
alter the dictionary in place?
2015-10-25 29855, 2015
11:56 AM
alastairp
2015-10-25 29825, 2015
11:57 AM
Gentlecat
oh
2015-10-25 29855, 2015
11:58 AM
Gentlecat
that's a bug, yeah
2015-10-25 29827, 2015
11:59 AM
alastairp
:(
2015-10-25 29833, 2015
11:59 AM
alastairp
all last night + this morning
2015-10-25 29838, 2015
11:59 AM
alastairp
no unit tests?
2015-10-25 29811, 2015
12:00 PM
zas
2015-10-25 29813, 2015
12:00 PM
Gentlecat
not for this one, no
2015-10-25 29854, 2015
12:00 PM
ruaok
zas: 😄
2015-10-25 29807, 2015
12:02 PM
zas
i increased roobarb monit memory to 16Gb, we'll see
2015-10-25 29829, 2015
12:02 PM
zas
the rate limit is still on 1500 N
2015-10-25 29831, 2015
12:02 PM
zas
?
2015-10-25 29838, 2015
12:02 PM
ruaok
yes
2015-10-25 29843, 2015
12:02 PM
ruaok
back to 2000?
2015-10-25 29856, 2015
12:02 PM
zas
yes, else we'll see nothing ;)
2015-10-25 29813, 2015
12:03 PM
ruaok
k
2015-10-25 29829, 2015
12:03 PM
ruaok
done
2015-10-25 29835, 2015
12:05 PM
zas
ok % of 200 responses is increasing
2015-10-25 29828, 2015
12:09 PM
Nyanko-sensei joined the channel
2015-10-25 29829, 2015
12:10 PM
zas
ruaok: FYI, for tomcat mem/cpu i'm using test. prefix for carbon, and it is configured to keep only 3 days of stats (hence the "test."), so don't be astonished if stats older than 3 days aren't shown
2015-10-25 29835, 2015
12:13 PM
ruaok
ok
2015-10-25 29844, 2015
12:13 PM
ruaok
can we graph number of sockets in CLOSE_WAIT as well?
2015-10-25 29855, 2015
12:13 PM
zas
sure
2015-10-25 29849, 2015
12:14 PM
zas
2015-10-25 29829, 2015
12:15 PM
zas
but grep 'restart_search_server' /var/log/auth.log on dora/roobarb shows there was only 2 minutes between
2015-10-25 29855, 2015
12:15 PM
ruaok
2 minutes is clearly not enough anymore
2015-10-25 29831, 2015
12:16 PM
zas
in which project the push script is ?
2015-10-25 29846, 2015
12:17 PM
zas
2015-10-25 29847, 2015
12:17 PM
reosarevok joined the channel
2015-10-25 29830, 2015
12:18 PM
ruaok
2015-10-25 29831, 2015
12:18 PM
zas
2015-10-25 29835, 2015
12:18 PM
zas
:))
2015-10-25 29806, 2015
12:19 PM
zas
bump to 5 mins ?
2015-10-25 29814, 2015
12:19 PM
ruaok
at least.
2015-10-25 29834, 2015
12:19 PM
ruaok
I wonder if we should have different monit limits on the two servers.
2015-10-25 29855, 2015
12:19 PM
ruaok
that might reduce the chance the they both get restarted by monit near the same time
2015-10-25 29823, 2015
12:21 PM
zas
imho we should just use a file to prevent a restart if the other just restarted or smt like that
2015-10-25 29836, 2015
12:22 PM
diana_olhovik_ joined the channel
2015-10-25 29823, 2015
12:23 PM
ruaok
sure
2015-10-25 29825, 2015
12:26 PM
zas
ruaok: can you check search-indexer repo on cartman, there are changes which weren't committed
2015-10-25 29850, 2015
12:26 PM
Gentlecat
2015-10-25 29859, 2015
12:26 PM
zas
i'll report tcp close-wait on dora/roobarb
2015-10-25 29800, 2015
12:27 PM
Gentlecat
didn't test it yet. have to run
