#musicbrainz-devel

FWIW, I turned off the recording index generation.

I may need to go install mbslave on cartman. :(

I wonder if the flaky carl kept our traffic artificially low.

and now it works, everything else is under more stress.

possible, I guess. not sure how we'd test the hypothesis

first step: get log analysis running again.

well, proof positive its not the search indexeing that is causing problem.

totoro is pissed off again.

ianmcorvidae: still up?

right then. we may have a bit of a problem.

Our DB server has 48GB of ram. Our postgres config:

shared_buffers = 8192MB

anyone who is not alastairp see a problem?

ooh, me, me

Just swap it!

i know sir

Oh, 48.

ok, configuration updated.

does pg 9.1 need a full restart for this to take?

our kernel.shmmax allows 27474836480 bytes.

the docs don't state that we need a restart.

I'll try hupping.

ok, time for some lunch.

oh looky, load 5 now. :)

(I’m not sure how large the whole DB currently is.)

I went with the docs that suggested 1/4 - 1/3 ram.

so, I went with 14GB.

of 48GB

It also seems effective_cache_size should be set to something like 40 GB or so.

currently 20GB

does that setting need a full restart or reload ok?

HUP, according to https://wiki.postgresql.org/wiki/Tuning_Your_Po...

ok, coming right up.

server signaled

“… if you have more than 1 GB …” :)

heh

index builder restarted.

that seems to have dropped the io load.

and now the load is spiking.

surely because of the search indexes.

fuck. back into swap.

Hm.

search indexer killed

Has anything changed between yesterday and today?

and the load is drastically dropping.

not that I am aware of.

According to the graph, there were some swap-outs, slowly, over long periods, but never before a quick increase in swap usage.

ok, wasn't seach indexes.

load spiked again

Swapping again.

yep.

The only DB-intensive thing that has changed (that I see) is sitemaps, but that should run only during UTC night.

and it would presumably only be one process. not the whole DB server.

the last server update was 3 days ago.

that would not explain why it started happeing some 12 hours ago.

we've ruled out the search indexer.

I believe that we have enough indexes.

just to rule things out, I've build indexes for the redirect tables and edit_url (edit).

sitemaps stuff got included in a hotfix

no significant changes.

A new beta was rolled out around that time.

But I see no heavy changes, as I said.

should I turn off beta for a bit, just to check?

and now the load magically dropped again.

back to 8

looking at https://bitbucket.org/metabrainz/musicbrainz-se... the hotfix was probably some time after 14 hours ago

Regarding the indexes, these should already have had some.

they didn't.

But for edit_url, there is CREATE INDEX edit_url_idx ON edit_url (url) in CreateIndexes.sql

and (edit, url) is the PK, so that has an index automatically.#

-#

The redirect tables, it’s possible that there is/was none for the other direction.

ah yes.

the edit_url had two indexes.

I added one soley on edit, but that also made no diff.

(edit, url) can be used for lookups on (edit), too.

Though there seems to be/have been no index for the *_redirects on new_id, indeed.

Was forgotten when load_gid_redirects was added, apparently.

But this went live in v-2015-01-12 already.

ok, here is the next load spike.

Hi !

I am a computer science undergrad from India looking to participate in GSoC 2015. I wish to know more about AcousticBrainz Project. Any info would be of great help

what do you want to know about?

I wish to know about the AcousticBrainz project and what are the organizations expectations with regards to the project ideas about it ..

you can read up some more background at http://chatlogs.musicbrainz.org/musicbrainz-dev...

it would be great if you could let us know some things that you think might be useful, based on your understanding of the project

how do people find this channel but not the "about" page of AB?

to be fair, the proposal stub on the ideas page was very brief, and didn’t explain our specific problem very well

I’ve edited it a bit to add some more information

ruaok: there are several pull requests for CB that need to be reviewed

I know. more server drama keeping me busy. sorry. :(

Project critiquebrainz build #94: SUCCESS in 2 min 0 sec: http://ci.musicbrainz.org/job/critiquebrainz/94/

* priyankbhatt1234: Added title attriubute in anchor tag stating View on AcousticBrainz

the letter of the law has been satisfied, yes. the spirit remains 404

pop quiz: explain this curious graph http://stats.musicbrainz.org/mrtg/drraw/drraw.c...

looks more like what I normally do to LEDs

anyone else worried/amazed/puzzled by this graph?

only totoro has spikes. everything else is purring and happy. (for once)

ruaok: Actually, search requests don’t look OK either: http://stats.musicbrainz.org/webstats/nginx-rrd...

Now, this might be caused by people having trouble reaching mb.org, hence searching less.

But search-public also has some lows.

yeah, this looks like a knock on effect to me.

the overall shape of the curve remains the same.

at least the graph was pretty for 4 days, no?

:)

Sounds like a tomb inscription … “it worked well for four days”.

sounds better than what I expect to have on my tombstone: “Dumbass”

I like Pratchett's post-mortem tweets.

huh, look the search server is wedged.

interesting that.