#musicbrainz-devel

i.e. interface:vlan

em2:0 -> eth1.0

no. not correct.

em2:0 -> eth1.1

em2:2 -> eth1.3

that all plays with how iptable rules are written.

I've having to tweak a lot of things.

and yes, I can ping 3.25 from carl

the . notation is 1 based. the : notation is 0 based.

dib config is still wrong.

did you change it?

nope.

ah, that's why it's not working then. nor did I :-)

oh.

shall I?

please. :)

damn, my pseudo-ios-fu has left me. forgot how to save config :-(

Hmm. github not allowing me access to syswiki (sure that's the right url?), and I've forgotten how to commit switch config changes.

and I have to head into town.

ok, I'll try and do it via the web interface.

if you have some time later, I have a few more questions for you.

Mayeb it'll all come back later, but right now, ... nada.

understood.

ok, sure I'll be back later.

ttfn!

bai

reosarevok: sorry for the emails ;)

haha

It's not a problem in that way, it's good stuff is happening :)

I just don't have time to get involved in BB dev so I'd rather not get those - I could filter them out on my side but I suspect I'm not the only one

chirlu`: may I borrow your iptables knowledge?

If you return it …

crap. deal is off. :)

I'm trying to move our mail related ips to the new gateway.

the mail server sits on an internal ip, so we use SNAT routing to forward packets

-A POSTROUTING -o em1 -s 10.1.1.242 -j SNAT --to-source 72.29.167.155

-A PREROUTING ! -i em2:0 -p tcp -m tcp -d 72.29.167.155 --dport 25 -j DNAT --to-destination 10.1.1.242

first issue: any rule that I've tested that used the em2:0 iface did not work. if I use em2 it works.

so, I've been changing the interfaces, but in this case, it didn't work.

so using em2:0 doesn't work either.

I've never used {S|D}NAT rules before, so I have no idea what the issue might be.

The rules are on ernie/bert?

yes

there are the live rules on carl:

-A PREROUTING -d 72.29.167.155/32 ! -i em2:0 -p tcp -m tcp --dport 25 -j DNAT --to-destination 10.1.1.242

-A POSTROUTING -s 10.1.1.242/32 -o em1 -j SNAT --to-source 72.29.167.155

sorry, wrong window. ignore those.

-A POSTROUTING -s 10.1.1.242/32 -o eth0 -j SNAT --to-source 72.29.167.155

What are the various interfaces? em2 internal, em1 external?

that is for ernie

carl: https://gist.github.com/mayhem/2da1e1267a31d97d...

I remember there was some confusion about the different tables yesterday, are those rules in the right section (i.e. “nat”)?

yes

would you like me to mail you the whole file?

Hm. Problem is I need to leave in five minutes or so, so not really enough time to look into a large file now. :)

ok, np. poke me should you have time later.

I'm hoping rvedotrc will be back before too long. :)

I won’t be back before midnight, so she has a good chance of being here earlier. :-)

likely, yes.

I'll try moving the lb1 point over. it shouldn't require iptables changes.

Anyway, I have to go now. Viel Erfolg!

danke

Mineo: still around?

is logging

Users on #musicbrainz-devel: dom96 surtin Leo_Verto night199uk D4RK-PH0ENiX Leftmost legoktm michiwend adhawkins johtso Sebastinas Guest74450 pprkut kloeri _5moufl Muz mat_ kurros d356_ ocharles_

anyone know how to deal with a "TCP: too many orphaned sockets" message in syslog?

is logging

Users on #musicbrainz-devel: D4RK-PH0ENiX Leftmost michiwend johtso ocharles_ d356_ kurros Muz Guest74450 Sebastinas

Gah, 502s like crazy

(assuming it's known, but just in case - beta)

I tried to flip back to the old gateway, but that didn

't work.

ruaok: http://kaivanov.blogspot.fr/2013/01/troubleshoo... gives few pointer

now trying to figure out what I am getting so much packet loss

91041121388182082

those values look quite low imho

cat /proc/net/sockstat ?

sockets: used 17254

UDP: inuse 19 mem 9

RAW: inuse 0

agreed.

So I suspect that the .1 interfaces on carl were VLANs, but the :1 interfaces on ernie are just aliases.

And by dropping the suffix from the iptables rule, it becomes too general.

mem 4084 << 182082 , so it isnt the cause (tcp mem is enough), what does cat /proc/sys/net/ipv4/tcp_max_orphans give ?

65535

chirlu-mobile: sorry, but now we're trying to tweak the TCP stack on ernie to give better performance than the shit we have now. :(

Yeah, I'm just writing for later reference.

ruaok: looks enough according to "orphan 1293", the document recommends x4 to be safe

yeah, I increased it and the warnings went away.

now I'm not getting much in syslog

The train will soon pass through an area without network anyway. :-)

hey alastairp. how is your TCP stack tuning foo?

the network to ernie used to be really bad and laggy. its fine now.

so, something has improved.

But if I'm right, NAT might be working for traffic coming from the Internet side, just not from internal.

huh. I can't ssh from carl to ernie.

that's weird.

bitmap: you around?

ruaok: not done it

sorry

no worries.

ruaok: yep

kewl. got a minute to help?

the main traffic is now running via the new gateway.

but we're getting MASSIVE numbers of 502s

hm...

I'm wondering if restarting the web front ends would help.

not sure though.

yeah, not sure but we could try

:/ws/2/recording/?query=artist:Miley+Cyrus+recording:See+You+Again", host: "www.musicbrainz.org"

astro has loads of these.

let me know when you want me to take a server out.

you can take astro out

should be out.

it appears search or search load balance related.

how is it going?

astro should be good now, it looked like there were a lot of old starman processes lying around

astro in, pingu out

hm, the provisions all hangs on 'git pull' now

I can just restart the service but that seems bad

please do a restart.

ok, done