#musicbrainz-devel

Freso: ah, I've never used the output for anything

alastairp: You should start hanging out in #musicbottle ;)

what cool things go on there?

alastairp: Well, I'll be asking questions about testing Python soon. :p

hello!

Hey guys, I'm installing my own mirror locally to use with headphones..I'm at the step where "carton install --deployment" is running...how long should I wait for it to finish before it's safe to say it's hung?

scratch that, it's moving

So, about that long eh? ;-)

lol...yeah :) I've had 4 cups of coffee since I woke up at 3 am to do this, so I'm a little antse

DrDyna: on most machines it should be installing packages after a few seconds. but for some reason on some setups carton decides to just stall for 10 or 20 minutes or so before doing anything. very annoying, we don't know what causes it.

yeah, it was my top cpu and memory user for about 10 mins or so, with no console feedback, which is why I was wondering

I'm about halfway through the rembo10 walkthrough, gotta download the databases now

i really hope I can figure this out lol

this seems like a ton of stuff to have installed, I probably should have dug out an old box to run it dedicated

Freso: I'm going to Malmo tomorrow

do you have time to come and hang out?

Holidays? work?

I spent christmas in Copenhagen with a friend

I was in Montreal most of December - passed through here on my way back to Barcelona

alastairp: Time now? Tomorrow? When? :p

"tomorrow when we wake up"

That's... a vague time. :D

Considering that I'm recently found going to bed when "normal" people wake up. :/

ah, right

probably around midday

Okay.

Wait, weren't you going to Malmö tomorrow?

oh, right.

sorry - that's what I meant

want to come to Malmo?

otherwise we could find a pub somewhere tomorrow evening

Heh. I've no moneys for the train or the bridge, also I just recalled my cousin's daughter is turning 1 year tomorrow, so I'll probably be off celebrating her anyway. So tomorrow eve somewhere sounds like a better plan.

ok. i'll txt you in the afternoon once we've made plans then :)

whoa, there's a virtual machine version of the server....

INSTALL.md says: git clone git://git.musicbrainz.org/musicbrainz-server.git

yeah, I think I'll be downloading that.

but git.musicbrainz.org is a bit slow... github mirror is much faster :)

git.mb.o 17 KiB/s, github 2.28 MiB/s

are there any other mirrors for the NGS ova besides the one UK mirror?

167k/sec is torture

yay

reosarevok: I love when I google for postgres error messages that the ONLY result from duckduckgo is a musicbrainz irc chatlog.

lmao

heeh

One with help? :p

no, not really.

ok, I'm going to try and contact oliver on a mobile phone.

oh great, I have four telephone numbers for oliver charles.

ruaok: yay!

hi!

you're here :)

what's the problem?

I've not seen any nagios emails.

500, I see.

the problem is with people trying to enter edits

Dec 27 18:32:15 totoro postgres[17976]: [8-1] LOG: process 17976 still waiting for RowExclusiveLock on relation 1601221 of database 1416385 after 1000.103 ms at character 13

I see a lot of locks in the totoro postgres serverlog

which I think are the cause of this.

we've had similar trouble in july 2012, but I didn't see a clear cause or solution in the irc log I found.

is ocharles around?

that was more his cuppa

ruaok: no, I have tried to call his cell, but no answer.

and coffee.

what have you done so far?

just looked at the load on frontend servers and totoro, but that's fine.

and looked at the postgres log.

well, if every page gives a 500, then things are seriously backed up. :(

people in #musicbrainz describe issues with editing or adding releases, which fits with the theory of these locks being the problem.

no, musicbrainz seems to otherwise work fine for me.

the home page gives a 500

It just loaded for me

But it gave a 500 before

So...

http://musicbrainz.org/release/aff4a693-5970-4e... loads fine. the homepage is indeed now having issues.

nope 500 for me

Loads for me

it loaded for me, without css

Seems to be pretty random

ok, first step, lets see if we can get things moving again.

warp: lets log into the front ends and take both nginx and the backend down for each machine.

then we stop postgres.

then bring everything back up and observe.

because everything is utterly clogged right now.

Oliver just called, he's on his way home. he will join us on irc when he gets home in about half an hour.

I'll do astro and I am logged into totoro.

take down pingu and asterix.

ruaok: ok

astro is quiet

ruaok: just "sudo svc -d ." in /etc/service/mb_server-fastcgi, right?

ah there we go. svstat now reports asterix down.

I did it in /etc/init.d/musicbrainz-fastcgi

and pingu as well.

ok, /etc/init.d/mb_server-fastcgi does the same thing.

ok

ruaok: oh, I'll take down beta as well.

totoro going down.

stopped

er.

how does one start totoro?

ha. as root. :)

replace stop with start?

I use pg_ctl directly to stop it.

I'll do astro

postgresql-9.1-musicbrainz_0-0~0jozo0_amd64.deb ... first step :)

ok

totoro serverlog quite unhappy.

LOG: could not send data to client: Broken pipe

??

odd

the site seems really snappy now.

serverlog is quiet now too.

lets observe for the time being.

ok, no 502 when editing for me.