#musicbrainz-devel

dmesg talks about possible SYN flooding

it always does that

we've just discovered

well, sorta. it got a lot more chatty about that as of the start of this problem.

dmesg on lenny is way more noisy than usual.

correlation,causation,pinch of salt

we need to catch s shuttle now

back in a bit...

Walking and Ircing. Not smart.

Oh right. This bus has WiFi.

heh

okay, blocking that UA seems to be helping

force-503ing it

suddenly ssh is usable

I got nothing to offer right now other than some not-yet-related observations from graphs.

what are those observations, rvedotrc?

"Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3" was causing a lot of traffic so I've force-503'd it at nginx

we'll see what that improves

(for context, since I don't think you were around when I was mentioning it before, rvedotrc)

high load on the apps servers, slow responses on the search servers, high traffic on the ports already mentioned,

and a few IPs hitting fairly hard

but, same old same old.

how did you determine the IPs that are hitting hard?

well, for some definition of hard. just the 'top' script on the ratelimit server.

rvedotrc: k

I think we're going to leave that UA block in place and check in in a little while.

but it might just be beer:30 now.

Very interesting grepping for that UA in yesterday's log.

ya?

one query at 18/Oct/2013:08:15:16

then at 18/Oct/2013:09:15:15, boom, the flood starts.

nothing in between.

no ramp up.

gonna bail for today. Maybe I'll check in tomorrow and see how things are.

nn

okay

oh, they're gone

but anyway, hopefully that's a fix for now

yeah. lets keep our eye on things for now.

I'm going to head to the bar and find a beer.

Project full-import build #153: STILL FAILING in 6 min 51 sec: http://ci.musicbrainz.org/job/full-import/153/

http://userscripts.org/scripts/show/180309 <- quite happy I've finally got round to making that

heh, almost three million requests from that bad UA

(in the log for today, noting that that includes things after we blocked it since those still hit lenny)

from a bit more than 300,000 different IP addresses

more than half of those IPs only made one request, but

and apparently the UA was at least semi-accurate one way, inasmuch as most of the IPs geoiplookup could actually resolve were in russia :P

(~124k it couldn't find, ~122k russia, ~22k ukraine, ~13k USA, ~10k kazahkstan, ~7k belarus, ~2k germany, ~1.5k moldova, rest below 1000)

Does kind of sound like a DDoS.

yeah, a bit :P

luks: feedback on https://github.com/musicbrainz/picard/pull/125 ?

hm, long discussion

hm, long patch too :)

hmm. I really should finish my patch for picard-259 and submit it

just one minor cleanup I wanted to do to it.

hmm, actually, no. it's good as-is.

luks: that was fast. :) ... 4 months in the waiting when all I had to do was nudge you

well, ever expect I know about something :)

*never

I might have been away for a few days when you pinged me via the PR and I missed the mail

heh, it wasn't me who did the pinging. I wondered what happened that there was no feedback, but I was afk-ish for 3 months so didn't bother too much about it. until today. :)

fyi, this is the project I mentioned yesterday https://bitbucket.org/lalinsky/mbdata

if I feel like it can be finished to some usable state, either mbslave will be merged to it, or this to mbslave

that was always kind of my ultimate goal with mbslave, but I was never motivated to work on it beoynd the replication stuff

woh, I will use those SQLalchemy models... :) (I have started write SQLaclhemy reflected models... never finished)

the models are the main reason why I started with this :)

then I started on the api and the website to have an excuse to have a look at angularjs :)

luks: from the readme: "you can use the mbdata.models module to get " ... get what?:)

Mineo: left to reader's imagination :)

Hi everyone, I'm working on spinning up a musicbrainz server and I'm almost there, but when I bring up plack, it throws this warning: [warning] Unicode::Encoding plugin is auto-applied, please remove this from your appclass and make sure to define "encoding" config

Then reports that it's accepting connections

But on each subsequent request to plack, this error is thrown: [error] Couldn't render template "undef error - /home/jhill/musicbrainz/root/static/common.css: Does not exist at lib/MusicBrainz/Server/Data/FileCache.pm line 33.

Any idea what might be going on, or if these two issues are related?

No clue what's going on, but they don't seem related.

Can you connect to it, or does it bail out because of that missing .css file?

It bails out, and I get the same error reported in the browser.

ocharles: Should the CSS files get compiled "manually", or should that happen by itself?

I've brought up an instance before without having to compile manually, but I'll go ahead and compile the less and see what happens

sear: We recently switched from using CSS to using... some pre-CSS system, which I can't remember the name of. I'm thinking your latter error might be related to that.

Freso: It's less. Makes sense that I didn't get that error before, then

less, yeah.

I'll compile it myself

:)

Thanks <3

sear: Heh. You're welcome. :)

sear: you shouldn't need to compile it if DEVELOPMENT_SERVER is set to 1; the unicode plugin error you can ignore

(if that DBDefs option is set, it'll use less.js to do the compilation in the browser instead)

ianmcorvidae: Ahh, that'll do it. I have it set to 0. I compiled it and it worked fine though

yeah, once compiled it'd also work fin

but you'll want to ensure you do it each time the .less files change, assuming you care about broken CSS:)

ianmcorvidae: Any ETA on when you'll get around to recovering the ModBot deleted places?

Freso: not really, tonight/tomorrow likely

your time?

CatQuest: ^

yes, my time

once the GSoC mentor summit is over :P

it's kind of nice to see sqlalchemy handling 5+ level deep relationships efficiently

I've been wrong about ORMs for so long :/

oh?

I always advocated using hand written SQL for working with MB

MB server could have been much simpler with a good ORM

CallerNo6: *cough* ^

luks: the "good" part of "good orm" seems significant though.