16:58 PM
kepstin-laptop notes that the X240 has a squished keyboard with a firmware issue that turns the end key into insert when you enable function-lock (i.e. so your F* keys are F* keys instead of brightness, etc.)
16:59 PM
kepstin-laptop
since insert is Fn+End, and the Fn-lock swaps all of the top-row keys with special functions :/
17:03 PM
whoa, that's weird. on the new X1 carbon they moved the `~ key to between the right alt and ctrl at the bottom
17:03 PM
and escape went where it used to be :/
17:04 PM
(this video appears to be using the US layout)
17:05 PM
adhawkins
ocharles: Figured you'd have seen it already. Made me chuckle :D
17:09 PM
Server still having problems? Any ETA? (Not hassling, just wondering when I should next check it)
17:10 PM
chirlu`
Reminds me, there is a small Haskell program that I need to finish.
17:10 PM
CallerNo6 , when he finally gets the thinkpad he's lusting after, will probably glue a cover over the touchpad/trackpad/thingy. hate those things.
17:11 PM
kepstin-laptop
CallerNo6: easier to disable it in software, looks better too.
17:12 PM
CallerNo6
But what if I glue somethng /useful/ over it? Picture frame? Big Gulp holder?
17:12 PM
kepstin-laptop
and if you get a late-2013/2014 model, you need the touchpad available because its touch-sensitivity is used to tell which trackpoint button you're pressing.
17:13 PM
(this means anything with the last two digits of the model number >30)
17:14 PM
CallerNo6
fortunately, a new one isn't in the budget anyway
17:15 PM
voiceinsideyou joined the channel
17:18 PM
voiceinsideyou2 joined the channel
17:19 PM
rvedotrc joined the channel
17:43 PM
misterswag joined the channel
19:19 PM
djinni` joined the channel
19:33 PM
zas__
19:42 PM
ijabz joined the channel
19:42 PM
bitmap
zas__: can you check if the left pane is clipped (has scrollable contents) and just resize the window/pane accordingly?
19:43 PM
I'm not too fond of creating our own checkbox widget
19:44 PM
zas__
bitmap: the left pane (tree) isn't scrollable atm
19:45 PM
bitmap: i'm not fond of new checkbox widget too, and perhaps some other widget has the same behavior...
19:46 PM
bitmap
hm, it's scrollable on osx
19:47 PM
in any case there might be a way to compare the width of the pane to the width of the inner tree, but I'm not too sure
19:59 PM
chirlu`
20:00 PM
ruaok
bang!
20:00 PM
meeting time.
20:00 PM
ianmcorvidae
hello
20:00 PM
ruaok
is there a postgres doctor in the house? ;-)
20:00 PM
morning ianmcorvidae
20:00 PM
fractalizator2 joined the channel
20:00 PM
ocharles and I made nearly no progress on totoro.
20:00 PM
I've got a script that kills stucks procs, which is keeping things sane.
20:01 PM
we've tried to up shared_buffers. no effect.
20:01 PM
ianmcorvidae
yeah. I hear there was some vacuuming that found weirdnesses in the edit table too, or?
20:01 PM
ruaok
I just upped effective_cache_size to 20GB and that relieved some pressure, but that artist edits still don't load.
20:01 PM
yes, we got two error messages, but nothing else
20:01 PM
I'm flat out of ideas, sadly.
20:02 PM
ianmcorvidae
I wonder if we've done the same for edit_artist
20:02 PM
but yeah, I'm not sure either
20:02 PM
ruaok
have not.
20:02 PM
probably a good idea.
20:02 PM
LordSputnik joined the channel
20:02 PM
shall I start that?
20:02 PM
ianmcorvidae
(it's mostly been artists that have been the issue, so)
20:02 PM
well, if it's fine for now we may as well do our meeting at least :P
20:02 PM
ruaok
ok.
20:03 PM
besides travel and new years and pg issues, I've got nothing really new.
20:03 PM
anyone have anything to discuss?
20:03 PM
Freso likes that ruaok and ianmcorvidae's nicks both show up as cyan - also, is here
20:03 PM
ruaok kicks off the vacuum
20:03 PM
Freso
nikki: Ping?
20:04 PM
ianmcorvidae
not much here either; I took wednesday off and worked on some various things around it
20:04 PM
did at least get one long-lived ticket up for review
20:04 PM
Freso
I guess nikki's the only one with something on the agenda. She was around a few minutes ago.
20:04 PM
ruaok
ianmcorvidae: did you test increasing the rate limit and seeing if it had an effect on rika?
20:04 PM
ianmcorvidae
I've increased it, but haven't seen what it's done for rika
20:05 PM
nikki, reoafk, bitmap, chirlu might be able to weigh in if it's been better the last couple days or not
20:05 PM
ruaok
another thought would be to pay attention to the peaks and troughs in our 95th graph.
20:05 PM
rika should be nicer to use in a trough, than a peak
20:05 PM
bitmap
rika was fast for me when I was using it yesterday
20:05 PM
ruaok
if there is anything to the bandwidth limiting idea.
20:05 PM
djinni` joined the channel
20:05 PM
derwin
stop typoing "riak"
20:06 PM
bitmap
was even able to submit a 40 disc release on it :)
20:06 PM
chirlu`
There is a delay when typing on rika, but I would consider it normal.
20:06 PM
Can't compare to previous weeks, though.
20:06 PM
reodroid joined the channel
20:06 PM
ruaok
hmm.
20:06 PM
ianmcorvidae
okay. well, we'll see if the small improvements stick
20:06 PM
Freso
Seems snappier to me too right now.
20:06 PM
bitmap
the ssh delay is better than it usually is right now
20:07 PM
ruaok
well, if anyone encounters trouble, note where in the 95th graph we are
20:07 PM
ianmcorvidae
it's always been slightly slow, intentionally, but usably so is a change
20:07 PM
ruaok
20:07 PM
interestingly enough we're headed to a peak now and its doing well.
20:07 PM
what else can we discuss on rika?
20:07 PM
watch and observe, what else?
20:08 PM
ianmcorvidae
I'm not sure there's much else that'd be part of this topic
20:08 PM
I need to dist-upgrade it at some point but that's probably unrelated
20:09 PM
ruaok
k.
20:09 PM
any other topics?
20:09 PM
Freso
What's for dinner?
20:09 PM
Freso hides
20:09 PM
ruaok
burger wraps here. :)
20:10 PM
ianmcorvidae
a few hours away, here :P
20:10 PM
but nothing more from me, I don't think
20:10 PM
ruaok
well, if no one has anything else to discuss, then I think I'd love to chat with ianmcorvidae and ocharles about totoro.
20:10 PM
see if the three of us (and anyone else who'd care to participate) can think up new things to check out.
20:11 PM
ianmcorvidae
how's that edit_artist vacuum doing?
20:11 PM
ruaok is looking forward to a more action packed meeting next week.
20:11 PM
and the main thing that comes to mind is why the MBS timeout isn't working
20:11 PM
ruaok
still going. load 13, 6-10MB/s io
20:11 PM
let's close the general meeting, then.
20:11 PM
ianmcorvidae
we fix that and it still 504s, but it doesn't start slowly breaking totoro in the process
20:12 PM
ruaok
sorry for not delving into DRs this week, but the sick DB server is important for us to pay attention to.
20:12 PM
thanks everyone!
20:12 PM
/bang!
20:12 PM
reodroid
heh that was fast :)
20:12 PM
nikki
oh
20:12 PM
ruaok
nikki: ?
20:12 PM
ianmcorvidae: yes, the timeout killer is odd -- ocharles tried it a couple of different ways and it worked.
20:12 PM
except in production
20:13 PM
ruaok shrugs
20:13 PM
nikki
I managed to miss the entire meeting :P
20:13 PM
ianmcorvidae
one question I had was what our old version of pgbouncer was
20:14 PM
since with the dist-upgrade we switched to a package manager-installed one from a hand-compiled one
20:14 PM
ocharles: you'd seem to be the one who might know on that count
20:15 PM
Freso
nikki: I did try to ping you. :)
20:15 PM
ruaok
ianmcorvidae: ah, that is an interesting idea.
20:15 PM
which fits with your observation that something changed at dist-upgrade time
20:15 PM
also, jozo asked if the collate problem that prevented the replication packets from applying could hamper our server.
20:15 PM
ianmcorvidae
yeah -- though I will stress again that it wasn't anything near this bad until last night, of course
20:15 PM
ruaok
do we need to rebuild those indexes?
20:16 PM
ianmcorvidae: yes, but I've seen this pattern before.
20:16 PM
things a slightly elevated and then you hit some inflection point and BAM, things go south.
20:16 PM
ianmcorvidae
we might need to because postgresql-musicbrainz-collate's behavior has changed, but we should actually update it first
20:16 PM
ruaok
I've kept forgetting to mention this for the past few meetings. this had been making me uneasy, TBH
20:16 PM
ianmcorvidae
I think ollie got the package updated though, so if you've done some provisions we may be fine
20:16 PM
(on the collate issue)
20:17 PM
ruaok
we did a provision today to up the shared_buffers.
20:17 PM
ianmcorvidae
any chance you remember if it updated a postgresql-musicbrainz-collate package in the mix there?
20:17 PM
ruaok
that reminds me, I need to back-propagate the effective_cache_size change
20:17 PM
not i.
20:17 PM
ocharles was driving.
20:18 PM
should we pre-emptively hand-compile a pg-bouncer?
20:18 PM
ianmcorvidae
without knowing what version the old one was I don't know that it'd help
20:18 PM
I don't know if we upgraded or downgraded or neither :)
20:18 PM
ruaok
where would that have gotten installed?
20:18 PM
into the postgres install dir?
20:18 PM
or elsewhere?
20:18 PM
maybe its still around.
20:19 PM
ianmcorvidae
it installed to /usr/local/pgbouncer or so, but you can't run the binaries
20:19 PM
because of missing libraries from the pre-dist-upgrade server
20:19 PM
ruaok
strings?
20:19 PM
ianmcorvidae
hm, maybe
20:19 PM
ruaok
vacuum of edit_artist is still going
20:20 PM
ianmcorvidae
1.5.2, current version installed is 1.5.4-3.pgdg12.4+1
20:20 PM
ruaok
yay.
20:20 PM
ianmcorvidae wonders if I can find a pgbouncer changelog
20:20 PM
I'd say lets compile 1.5.2 and try to use it and see what happens.
20:22 PM
20:23 PM
ianmcorvidae also looks into pgbouncer configuration options that might help us
20:27 PM
ianmcorvidae
hm
20:27 PM
only the 'dangerous timeouts' section looks like it could help, so that's less than ideal
20:28 PM
ruaok
huh.
20:28 PM
the vacuum is still going and yet things are really calm now.