19:51 PM
rvedotrc
OK. So, what problem would you like me to look into?
2015-04-24 11426, 2015
19:51 PM
ruaok
rvedotrc: why does dnscache/tinydns not work.
2015-04-24 11432, 2015
19:51 PM
ruaok
well, dnscache might be working.
2015-04-24 11434, 2015
19:51 PM
rvedotrc
Define "not work".
2015-04-24 11444, 2015
19:51 PM
ruaok
but tinydns hangs when making a request.
2015-04-24 11444, 2015
19:51 PM
rvedotrc
Symptom?
2015-04-24 11451, 2015
19:51 PM
ruaok
the request fails with no servers could be reached.
2015-04-24 11453, 2015
19:51 PM
kepstin-laptop
if there is no listener to *, nginx *only* binds to the separate ips.
2015-04-24 11404, 2015
19:52 PM
rvedotrc
ok
2015-04-24 11419, 2015
19:52 PM
kepstin-laptop
or rather, it gives gives errors.
2015-04-24 11424, 2015
19:52 PM
kepstin-laptop
or something
2015-04-24 11437, 2015
19:52 PM
kepstin-laptop
or maybe only errors if you mix * and not *? I'm confused now.
2015-04-24 11448, 2015
19:52 PM
ruaok
kepstin-laptop: why would that fail only on startup, but on redeploy it works?
2015-04-24 11422, 2015
19:53 PM
kepstin-laptop
if your nginx config only has "listen" with an ip address, and the ip address is not present when nginx starts up, that server block won't start.
2015-04-24 11401, 2015
19:54 PM
ruaok
OH
2015-04-24 11408, 2015
19:54 PM
kepstin-laptop
there will be an 'emerg' level log in nginx's error log in that case
2015-04-24 11414, 2015
19:54 PM
ruaok
no wait.
2015-04-24 11430, 2015
19:54 PM
kepstin-laptop
saying "Cannot assign requested address"
2015-04-24 11437, 2015
19:54 PM
ruaok
let me look for that
2015-04-24 11404, 2015
19:55 PM
rvedotrc
ruaok: tinydns is working. It only responds for data it hosts.
2015-04-24 11411, 2015
19:55 PM
rvedotrc
e.g. dig @127.0.0.1 ntp.localdomain
2015-04-24 11414, 2015
19:55 PM
rvedotrc
gets a response
2015-04-24 11414, 2015
19:55 PM
ruaok
DOH.
2015-04-24 11418, 2015
19:55 PM
rvedotrc
2015-04-24 11419, 2015
19:55 PM
rvedotrc
doesn't.
2015-04-24 11430, 2015
19:55 PM
ruaok
that is *exaclty* the query I was using.
2015-04-24 11439, 2015
19:55 PM
ruaok
heh, nice and simple fit.
2015-04-24 11440, 2015
19:55 PM
rvedotrc read the log :-)
2015-04-24 11459, 2015
19:55 PM
ruaok
which log?
2015-04-24 11411, 2015
19:56 PM
rvedotrc
sudo tail -F /etc/tinydns/log/main/current | tai64nlocal
2015-04-24 11418, 2015
19:56 PM
ruaok
kepstin-laptop: grepping for that error helped.
2015-04-24 11434, 2015
19:56 PM
ruaok
hm. I was looking there.
2015-04-24 11444, 2015
19:56 PM
ruaok
so, dnscache is for handling other queries?
2015-04-24 11454, 2015
19:56 PM
rvedotrc
dnscache is the recursing resolver, yes.
2015-04-24 11409, 2015
19:57 PM
rvedotrc
auth dns and resolving dns are two completely separate services.
2015-04-24 11420, 2015
19:57 PM
ruaok
hmm
2015-04-24 11422, 2015
19:57 PM
ruaok
2015-04-24 11425, 2015
19:57 PM
ruaok
does not work.
2015-04-24 11430, 2015
19:57 PM
ruaok
though I think it should.
2015-04-24 11432, 2015
19:57 PM
rvedotrc
ok /me digs
2015-04-24 11444, 2015
19:57 PM
ruaok
thanks!
2015-04-24 11450, 2015
19:57 PM
ruaok is being summoned for dinner.
2015-04-24 11453, 2015
19:57 PM
ruaok
back in a bit
2015-04-24 11427, 2015
19:58 PM
rvedotrc
cat /etc/dnscache/env/IPSEND
2015-04-24 11453, 2015
19:58 PM
rvedotrc
it's sending its outgoing traffic using the other gateway's IP address.
2015-04-24 11429, 2015
20:00 PM
kepstin-laptop
hmm. one thing you might want to look into is making sure that corosync/pacemaker configures all the ip addesses on the same machine, if that's what you expect
2015-04-24 11439, 2015
20:00 PM
kepstin-laptop
by default i think it'll try to balance them over both
2015-04-24 11410, 2015
20:01 PM
rvedotrc
certainly back in the heartbeat days (before corosync) you configured "resource groups"
2015-04-24 11420, 2015
20:01 PM
rvedotrc
each group would be on one host or the other
2015-04-24 11431, 2015
20:01 PM
kepstin-laptop
yeah, i think there's still something similar.
2015-04-24 11434, 2015
20:01 PM
rvedotrc
and a group could consist of many resources (IPs etc)
2015-04-24 11427, 2015
20:04 PM
kepstin-laptop
hmm. looks the way you do it now is by putting a colocation constraint between resources.
2015-04-24 11440, 2015
20:04 PM
kepstin-laptop
or are there still groups?
2015-04-24 11443, 2015
20:04 PM
kepstin-laptop keeps looking
2015-04-24 11452, 2015
20:05 PM
rvedotrc doesn't even know where to find the syswiki repo these days :-/
2015-04-24 11432, 2015
20:08 PM
kepstin-laptop
the only thing I can see in the docs is the colocation constraints
2015-04-24 11409, 2015
20:09 PM
kepstin-laptop
to set those up, it would mean picking one ip to be the 'master' one, then adding constraints between that and each other ip to keep them together
2015-04-24 11436, 2015
20:09 PM
kepstin-laptop
unless I'm horribly mistaken
2015-04-24 11441, 2015
20:10 PM
ijabz2 joined the channel
2015-04-24 11402, 2015
20:17 PM
ruaok returns
2015-04-24 11409, 2015
20:17 PM
ruaok
lets me fix the dns stuff first.
2015-04-24 11449, 2015
20:23 PM
kepstin-laptop
rvedotrc, ah, yeah, there are still groups. looks like grouping the ips is the way to go.
2015-04-24 11411, 2015
20:24 PM
ruaok
I've love a config snippet. or just commit something. :)
2015-04-24 11438, 2015
20:33 PM
ruaok
rvedotrc: I've got IPSEND set correctly now, but dnscache still doesn't work. :(
2015-04-24 11403, 2015
20:34 PM
rvedotrc looks
2015-04-24 11441, 2015
20:34 PM
kepstin-laptop
ruaok, i've got a pull request open for the stuff for grouping the ips. I guess you edited it concurrently, so I couldn't just commit stuff.
2015-04-24 11402, 2015
20:35 PM
kepstin-laptop
note that i've adapted the configuration commands to run in a single transaction rather than be committed separately.
2015-04-24 11416, 2015
20:35 PM
rvedotrc
Did you restart dnscahce?
2015-04-24 11422, 2015
20:35 PM
ruaok
yep.
2015-04-24 11432, 2015
20:35 PM
ruaok
that was to rvedotrc
2015-04-24 11424, 2015
20:37 PM
ruaok
merged. thanks!
2015-04-24 11436, 2015
20:37 PM
rvedotrc
ruaok: iptables.
2015-04-24 11443, 2015
20:37 PM
rvedotrc
s/250/150/ or thereabouts.
2015-04-24 11405, 2015
20:38 PM
CatCat joined the channel
2015-04-24 11406, 2015
20:38 PM
ruaok
ahhhh.
2015-04-24 11414, 2015
20:38 PM
rvedotrc
-A INPUT -d 10.1.1.250/32 -i lo -p tcp -m tcp --dport 53 -j ACCEPT
2015-04-24 11414, 2015
20:47 PM
ruaok
rvedotrc:
2015-04-24 11418, 2015
20:47 PM
ruaok
-A INPUT -d 10.1.1.150/32 -i em2.1 -p udp -m udp --dport 53 -j ACCEPT
2015-04-24 11420, 2015
20:47 PM
ruaok
-A INPUT -d 10.1.1.150/32 -i lo -p udp -m udp --dport 53 -j ACCEPT
2015-04-24 11422, 2015
20:47 PM
ruaok
-A INPUT -d 10.1.1.150/32 -i lo -p tcp -m tcp --dport 53 -j ACCEPT
2015-04-24 11445, 2015
20:47 PM
ruaok
but dig still hangs. :(
2015-04-24 11425, 2015
20:48 PM
rvedotrc
The packet isn't reaching the dnscache socket, for some reaso.
2015-04-24 11438, 2015
20:48 PM
rvedotrc
i.e. dnscache never receives your query.
2015-04-24 11411, 2015
20:51 PM
ruaok
sure looks like the firewall. :(
2015-04-24 11429, 2015
20:51 PM
ruaok
telnet 10.1.1.150 53
2015-04-24 11432, 2015
20:51 PM
ruaok
just hangs.
2015-04-24 11425, 2015
20:52 PM
ruaok
ok, there are multiple problems. :(
2015-04-24 11451, 2015
20:52 PM
ruaok
when I flush the rules I can get to dnscache, though it just hangs up on me.
2015-04-24 11407, 2015
20:53 PM
ruaok
but it *still* doesn't resolve it
2015-04-24 11418, 2015
20:53 PM
ruaok
> connection timed out; no servers could be reached
2015-04-24 11408, 2015
20:55 PM
rvedotrc
2:freenode/
2015-04-24 11420, 2015
20:55 PM
rvedotrc
$ cat /etc/dnscache/root/ip/10.1.
2015-04-24 11421, 2015
20:55 PM
rvedotrc
10.1.1.1 10.1.3
2015-04-24 11430, 2015
20:55 PM
rvedotrc
only 10.1.1.1 and 10.1.3.x are allowed as clients.
2015-04-24 11423, 2015
20:56 PM
rvedotrc
compare that dir to carl.
2015-04-24 11452, 2015
21:00 PM
ruaok
ok, when I touch the local ips in it works.
2015-04-24 11418, 2015
21:01 PM
ruaok
why 10.1.1.1? that machine is decommissioned.
2015-04-24 11425, 2015
21:01 PM
ruaok
and what is the 10.1.3.x subnet for?
2015-04-24 11425, 2015
21:01 PM
rvedotrc
presumably you want it to contain 10.1.1 , like carl?
2015-04-24 11444, 2015
21:01 PM
rvedotrc
i.e. allow any IP 10.1.1.x
2015-04-24 11448, 2015
21:01 PM
ruaok
OH.
2015-04-24 11454, 2015
21:01 PM
ruaok
I clearly need to stop soon.
2015-04-24 11400, 2015
21:02 PM
rvedotrc sends coffee
2015-04-24 11400, 2015
21:02 PM
ruaok
not parsing these things anymore.
2015-04-24 11404, 2015
21:02 PM
ruaok
:-D
2015-04-24 11425, 2015
21:05 PM
LordSputnik joined the channel
2015-04-24 11423, 2015
21:13 PM
rvedotrc
10.1.3.x was for rika, if memory serves.
2015-04-24 11455, 2015
21:13 PM
ruaok returns
2015-04-24 11418, 2015
21:20 PM
ruaok
rvedotrc: still here?
2015-04-24 11428, 2015
21:20 PM
ruaok
the firewall rules are still interfering. :(
2015-04-24 11456, 2015
21:20 PM
rvedotrc
hi
2015-04-24 11457, 2015
21:20 PM
ruaok
I have these rules in place now:
2015-04-24 11458, 2015
21:20 PM
ruaok
-A INPUT -i em2 -d 10.1.1.150 -p tcp -m tcp --dport 53 -j ACCEPT
2015-04-24 11400, 2015
21:21 PM
ruaok
-A INPUT -i lo -d 10.1.1.150 -p tcp -m tcp --dport 53 -j ACCEPT
2015-04-24 11410, 2015
21:21 PM
ruaok
flush firewall rules and everything works.
2015-04-24 11439, 2015
21:21 PM
ruaok
does the rule need to be on em1 ?
2015-04-24 11444, 2015
21:21 PM
ruaok
the first two rules?
2015-04-24 11432, 2015
21:22 PM
rvedotrc
no, your internal clients are on em2
2015-04-24 11444, 2015
21:22 PM
rvedotrc
but ... also em2:0 and em2:2. hmm.
2015-04-24 11404, 2015
21:23 PM
rvedotrc
but still, it should work on lo.
2015-04-24 11402, 2015
21:26 PM
rvedotrc
ruaok: your .150 rules are in the wrong table.
2015-04-24 11409, 2015
21:26 PM
rvedotrc
is *nat. should be in *filter.
2015-04-24 11409, 2015
21:27 PM
ruaok
sorry, not grokking that.
2015-04-24 11430, 2015
21:27 PM
ruaok
the thing is I started with the iptables file from carl
2015-04-24 11423, 2015
21:28 PM
rvedotrc
iptables has (at least) three sets of rules. filter, nat, mangle.
2015-04-24 11441, 2015
21:28 PM
rvedotrc
That's introduced by the lines "*filter", "*nat", "*mangle" in the file.
2015-04-24 11448, 2015
21:28 PM
ruaok
ah!
2015-04-24 11451, 2015
21:28 PM
rvedotrc
sorry. *raw not *mangle.
2015-04-24 11407, 2015
21:29 PM
rvedotrc
both :-) anyway, it's in the wrong section.
2015-04-24 11422, 2015
21:29 PM
rvedotrc
needs to be in *filter, like the .250 rules are.
2015-04-24 11403, 2015
21:30 PM
ruaok
\ΓΈ/
2015-04-24 11406, 2015
21:30 PM
ruaok
thanks! ;)
2015-04-24 11410, 2015
21:30 PM
rvedotrc
working?
2015-04-24 11425, 2015
21:30 PM
rvedotrc
w00t
2015-04-24 11446, 2015
21:30 PM
CatQuest joined the channel
2015-04-24 11446, 2015
21:30 PM
rvedotrc checks tcp. works too.
2015-04-24 11453, 2015
21:31 PM
rvedotrc
and, works from another client (carl).
2015-04-24 11411, 2015
21:32 PM
rvedotrc
2015-04-24 11425, 2015
21:32 PM
ruaok
lol
2015-04-24 11430, 2015
21:32 PM
rvedotrc
Turns out, goole.com's hosting looks inferior to google's. Who knew?
2015-04-24 11441, 2015
21:32 PM
ruaok
heh
2015-04-24 11412, 2015
21:34 PM
ruaok
ok, dns works now. phew.
2015-04-24 11418, 2015
21:34 PM
ruaok
thanks for your help!
2015-04-24 11442, 2015
21:34 PM
ruaok
ok, now back to the nginx issue
2015-04-24 11459, 2015
21:34 PM
ruaok
kepstin-laptop: the grouping of the ip addresses -- was that the fix for the nginx issue?
2015-04-24 11407, 2015
21:35 PM
ruaok
if so, its not clear to me why.
2015-04-24 11422, 2015
21:35 PM
ruaok
because ernie has the public ip address setup.
2015-04-24 11443, 2015
21:35 PM
ruaok
at least one block should succeed -- the one for gtest. all the other ones will fail, which seems fine for this test case.
2015-04-24 11432, 2015
21:38 PM
kepstin-laptop
ruaok, I was thinking it might make sense to have the cluster stuff manage the nginx server; starting nginx tied to the external ips.
2015-04-24 11448, 2015
21:39 PM
ruaok
instead of using daemontools?
