#musicbrainz-devel

/

      • ruaok
        the index use rates and cache hit rates look *great*.

        • 2014-01-06 00641, 2014

      • ocharles
        ok, well the vacuum is in progress now

        • 2014-01-06 00603, 2014

      • ocharles
        I wish we knew the expected IO for the drives in totoro :(

        • 2014-01-06 00619, 2014

      • ruaok
        real life IO or paper IO?

        • 2014-01-06 00620, 2014

      • ocharles
        iotop reports 10M/s reads at the moment, and I'm not sure what to really be expecting

        • 2014-01-06 00627, 2014

      • ocharles
        real life, ideally

        • 2014-01-06 00642, 2014

      • ruaok
        I've seen bursts of 40MB/s

        • 2014-01-06 00654, 2014

      • ruaok
        which is damn good, compared to some other drives in our servers. :)

        • 2014-01-06 00603, 2014

      • ruaok
        but, the idea is to not need the disk at all.

        • 2014-01-06 00603, 2014

      • ocharles
        Yea, graphite does have a peak at 40MB/s

        • 2014-01-06 00618, 2014

      • ocharles
        true

        • 2014-01-06 00619, 2014

      • ruaok
        our DB ain't that big

        • 2014-01-06 00629, 2014

      • ocharles
        WARNING: relation "edit" page 1431011 is uninitialized --- fixing

        • 2014-01-06 00631, 2014

      • ocharles

        • 2014-01-06 00641, 2014

      • ruaok
        huh

        • 2014-01-06 00644, 2014

      • ocharles
        that's a bit weird, I've never seen that before

        • 2014-01-06 00613, 2014

      • ruaok
        load is only 14. I was expecting much worse.

        • 2014-01-06 00632, 2014

      • ocharles
        ah, there we go, 40MB/s

        • 2014-01-06 00658, 2014

      • ruaok
        which is kinda impressive.

        • 2014-01-06 00604, 2014

      • ruaok
        my first hard drive was 10MB

        • 2014-01-06 00610, 2014

      • ruaok
        and it took an hour or so to format.

        • 2014-01-06 00629, 2014

      • ocharles
        I found out about the secure erase feature of SSDs yesterday. Pretty cool that you can completely wipe them in about 30 seconds

        • 2014-01-06 00632, 2014

      • ruaok
        reading 4 times that in one second is pretty cool.

        • 2014-01-06 00658, 2014

      • ocharles
        (yes Freso, that's for your laptop ;))

        • 2014-01-06 00602, 2014

      • ruaok
        wait til a virus gets a hold of that.

        • 2014-01-06 00618, 2014

      • ocharles
        indeed

        • 2014-01-06 00628, 2014

      • ocharles
        though it does need a password to be done, so in theory if you set a password in the first place you'd be ok

        • 2014-01-06 00639, 2014

      • ocharles
        but my drive had no password, so all I had to do was set one and then immediately use it

        • 2014-01-06 00653, 2014

      • kuno
        ocharles: hm, buh how secure is that? I would expect it does something similar as trim, in order to not completely destroy the longevity of the drive itself.

        • 2014-01-06 00657, 2014

      • kuno
        s/buh/but/

        • 2014-01-06 00609, 2014

      • ocharles
        kuno: yea, I'm not sure how secure the secure is

        • 2014-01-06 00614, 2014

      • ocharles
        there is also "enhanced secure" :)

        • 2014-01-06 00603, 2014

      • reosarevok
        Oh, ocharles

        • 2014-01-06 00609, 2014

      • reosarevok
        You have a t440, right?

        • 2014-01-06 00618, 2014

      • reosarevok
        How weird does the mouse feel?

        • 2014-01-06 00621, 2014

      • reosarevok
        (trackpad)

        • 2014-01-06 00653, 2014

      • ocharles
        I don't like it

        • 2014-01-06 00609, 2014

      • ocharles
        but i also don't hugely care, because I don't like pointing in general :)

        • 2014-01-06 00640, 2014

      • reosarevok
        Hmm

        • 2014-01-06 00608, 2014

      • reosarevok
        I like pretty much everything else about the laptop, but what I've seen in the videos makes that feel super-weird :/

        • 2014-01-06 00622, 2014

      • ocharles
        It doesn't feel weird anymore

        • 2014-01-06 00628, 2014

      • ocharles
        It's sort of nice to just press down anywhere to click

        • 2014-01-06 00637, 2014

      • ocharles
        What I don't like is the Linux drivers currently mean that doing that moves the mouse

        • 2014-01-06 00642, 2014

      • ocharles
        And that is really. fucking. annoying.

        • 2014-01-06 00657, 2014

      • ocharles
        but I imagine that will get solved in due time

        • 2014-01-06 00600, 2014

      • reosarevok
        Can't you just disable the trackpad if you don't use it as such?

        • 2014-01-06 00610, 2014

      • ocharles
        I think you could

        • 2014-01-06 00616, 2014

      • ocharles
        Most of the time I have it docked and use a USB mouse

        • 2014-01-06 00625, 2014

      • ocharles
        ruaok: still vacuuming

        • 2014-01-06 00631, 2014

      • ruaok nods

        • 2014-01-06 00640, 2014

      • ruaok
        I'm watching top and iotop and my script

        • 2014-01-06 00643, 2014

      • ocharles
        ah, k

        • 2014-01-06 00615, 2014

      • ruaok
        ian and I haven't been doing much besides babysitting totoro for the last 12 or so hour

        • 2014-01-06 00638, 2014

      • ocharles
        I wish Pg had better status reporting on these operations :(

        • 2014-01-06 00658, 2014

      • Freso
        ocharles: \o/

        • 2014-01-06 00604, 2014

      • ocharles
        ruaok: do you think we should just go to full downtime to try and focus on the vacuum?

        • 2014-01-06 00619, 2014

      • ruaok
        naw.

        • 2014-01-06 00628, 2014

      • ruaok
        short of wasting our time, things are doing ok, I think.

        • 2014-01-06 00630, 2014

      • ocharles
        ok

        • 2014-01-06 00610, 2014

      • uk_ joined the channel

        • 2014-01-06 00604, 2014

      • uk_ has left the channel

        • 2014-01-06 00605, 2014

      • Jozo
        Is musicbrainz main database server upgraded lately? And not MBS-7024 is related to this?

        • 2014-01-06 00605, 2014

      • mb-chat-logger
        http://tickets.musicbrainz.org/browse/MBS-7024

        • 2014-01-06 00615, 2014

      • ruaok
        ocharles: that ought to be closed now, right? we've gone past this issue.

        • 2014-01-06 00626, 2014

      • ruaok
        nm.

        • 2014-01-06 00628, 2014

      • ruaok reads

        • 2014-01-06 00631, 2014

      • ruaok
        it is closed.

        • 2014-01-06 00647, 2014

      • Jozo
        ruaok: But if database is upgraded then index sizes is doubled or something... Or I'm wrong?

        • 2014-01-06 00631, 2014

      • ocharles
        Jozo: you know, I'm not sure

        • 2014-01-06 00654, 2014

      • ocharles
        that might have something to do with it

        • 2014-01-06 00659, 2014

      • Jozo
        ocharles: It's just guess...

        • 2014-01-06 00603, 2014

      • ocharles
        We now have a different result from musicbrainz_collate

        • 2014-01-06 00609, 2014

      • ocharles
        so all the indexes might be wrong

        • 2014-01-06 00625, 2014

      • ocharles
        possibly...

        • 2014-01-06 00635, 2014

      • ocharles
        but I think we only use them for sorting anyway

        • 2014-01-06 00657, 2014

      • ruaok
        I would think that would manifest itself in a different way, but I can't say for sure.

        • 2014-01-06 00609, 2014

      • ocharles
        I'm quite sure what it would do

        • 2014-01-06 00616, 2014

      • ruaok
        what?

        • 2014-01-06 00629, 2014

      • Jozo
        ocharles: But they are indexes (what should be in memory, for performance)

        • 2014-01-06 00631, 2014

      • ocharles
        You have a functional index on foo(artist.name) for example

        • 2014-01-06 00645, 2014

      • ocharles
        If you change what foo() returns then your indexes don't match what you're querying for

        • 2014-01-06 00614, 2014

      • ruaok
        i get that. then what.

        • 2014-01-06 00621, 2014

      • ocharles
        yea, that's the bit I can't answer :)

        • 2014-01-06 00622, 2014

      • ruaok
        failed query when it should've succeeded?

        • 2014-01-06 00655, 2014

      • ruaok
        well, our immediate problems are wrt to the indexes that use collation.

        • 2014-01-06 00601, 2014

      • ruaok
        does edit use such an index?

        • 2014-01-06 00608, 2014

      • ocharles
        no

        • 2014-01-06 00612, 2014

      • kepstin-work joined the channel

        • 2014-01-06 00614, 2014

      • ruaok
        er. our problems are related to the edit table.

        • 2014-01-06 00624, 2014

      • ocharles
        I don't think collations plays here

        • 2014-01-06 00634, 2014

      • ocharles
        explain analyze select id FROM edit WHERE (id IN (SELECT edit FROM edit_artist WHERE artist = '119241')) LIMIT 1000;

        • 2014-01-06 00636, 2014

      • ruaok
        but just an increased size of index.

        • 2014-01-06 00641, 2014

      • ocharles
        that doesn't return fast, and I would have thought it should

        • 2014-01-06 00653, 2014

      • Jozo
        Ok, I just wanted remind that possibilty.

        • 2014-01-06 00601, 2014

      • ruaok
        well, there is an vacuum going.

        • 2014-01-06 00604, 2014

      • ruaok
        nothing is going to be fast.

        • 2014-01-06 00606, 2014

      • ocharles
        Jozo: that's a good reminder and I think it's worth investigating

        • 2014-01-06 00609, 2014

      • ocharles
        ruaok: that was before the vacuum

        • 2014-01-06 00617, 2014

      • ruaok
        but, concurrently reindexing would fix this problem, no?

        • 2014-01-06 00625, 2014

      • ruaok
        which tables have these indexes?

        • 2014-01-06 00632, 2014

      • ocharles
        that's why I suggested the vacuum. The plan looks sane, so the only thing I could think of was maybe stuff is badly partitioned

        • 2014-01-06 00636, 2014

      • ocharles
        and there's too much bloat

        • 2014-01-06 00655, 2014

      • ruaok
        wow, the explain is cranking the io. :(

        • 2014-01-06 00659, 2014

      • ocharles
        ruaok: CreateIndexes.sql can tell you exactly which indexes need to be re-calculated

        • 2014-01-06 00607, 2014

      • ruaok
        the last few indexes in the file then.

        • 2014-01-06 00629, 2014

      • ruaok cancels his explain

        • 2014-01-06 00649, 2014

      • ruaok
        well, lets pop the index re-creation onto our stack of things to examine.

        • 2014-01-06 00605, 2014

      • ruaok
        we'll have a look after the vacuum.

        • 2014-01-06 00628, 2014

      • ocharles nods

        • 2014-01-06 00624, 2014

      • reosarevok
        At least voting still works :)

        • 2014-01-06 00649, 2014

      • Jozo
        I'm repeating myself. How about adding banner?

        • 2014-01-06 00605, 2014

      • kepstin-laptop_ joined the channel

        • 2014-01-06 00614, 2014

      • reosarevok
        hi kepstin-laptop_!

        • 2014-01-06 00617, 2014

      • kepstin-work has left the channel

        • 2014-01-06 00624, 2014

      • ocharles
        ruaok: all done

        • 2014-01-06 00635, 2014

      • ruaok
        ok, lets see.

        • 2014-01-06 00654, 2014

      • ocharles
        still looks in a bad state :(

        • 2014-01-06 00659, 2014

      • ruaok
        hmm. not very interesting.

        • 2014-01-06 00614, 2014

      • reosarevok
        Yeah, not loading yet

        • 2014-01-06 00632, 2014

      • ruaok
        lets give it a couple of minutes to see if things level out.

        • 2014-01-06 00644, 2014

      • Jozo
        12h not working and still not "We have some issues" banner... :/

        • 2014-01-06 00654, 2014

      • ruaok
        Jozo: we get it. thanks.

        • 2014-01-06 00607, 2014

      • ocharles
        Jozo: I'll see if I can set that now, thanks for the reminder

        • 2014-01-06 00659, 2014

      • ruaok
        ocharles: one thing that has changed: now we have fat writes too!

        • 2014-01-06 00611, 2014

      • ruaok
        also, do you still have an explain running?

        • 2014-01-06 00615, 2014

      • ruaok
        I see one in iotop

        • 2014-01-06 00631, 2014

      • ocharles
        I doe

        • 2014-01-06 00632, 2014

      • ocharles
        do*

        • 2014-01-06 00638, 2014

      • ruaok
        go ahead and kill it.

        • 2014-01-06 00642, 2014

      • ocharles
        It'd be good to see the final cost

        • 2014-01-06 00646, 2014

      • ruaok
        lets see what the disk io does.

        • 2014-01-06 00651, 2014

      • ocharles
        alright

        • 2014-01-06 00600, 2014

      • ruaok
        ok, writes are gone.

        • 2014-01-06 00605, 2014

      • ruaok
        reads still high.

        • 2014-01-06 00622, 2014

      • ruaok
        reindex index label_idx_musicbrainz_collate;

        • 2014-01-06 00632, 2014

      • ruaok
        label is a small table.

        • 2014-01-06 00636, 2014

      • ruaok
        see what that does?

        • 2014-01-06 00642, 2014

      • ocharles
        sure, though it will lock

        • 2014-01-06 00602, 2014

      • ruaok
        I thought there was a "concurrent" flag, but I can't see that now.

        • 2014-01-06 00609, 2014

      • ruaok
        different command?

        • 2014-01-06 00611, 2014

      • ocharles
        not to reindex in 9.1

        • 2014-01-06 00618, 2014

      • ocharles
        i think 9.2 or 9.3 has it

        • 2014-01-06 00621, 2014

      • ruaok
        ah

        • 2014-01-06 00628, 2014

      • ocharles
        you can create a new index and drop the old one

        • 2014-01-06 00633, 2014

      • ocharles
        or just live with a brief lock

        • 2014-01-06 00639, 2014

      • ocharles
        like you say, label is small

        • 2014-01-06 00644, 2014

      • ruaok
        what if we take a little downtime?

        • 2014-01-06 00650, 2014

      • ruaok
        take the site down.

        • 2014-01-06 00653, 2014

      • ruaok
        reindex those tables.

        • 2014-01-06 00659, 2014

      • ruaok
        bump up shared_buffers