I have consul as nameserver that resolves address from two instances of a service. DNS info via dig interface.http.service.consul:
...
interface.http.service.consul. 0 IN A 10.0.0.85
interface.http.service.consul. 0 IN A 10.0.1.22
...
ping will alternate between both addresses:
while true; do ping -c 1 interface.http.service.consul | grep PING; sleep 1; done
PING interface.http.service.consul (10.0.0.85) 56(84) bytes of data.
PING interface.http.service.consul (10.0.0.85) 56(84) bytes of data.
PING interface.http.service.consul (10.0.0.85) 56(84) bytes of data.
PING interface.http.service.consul (10.0.1.22) 56(84) bytes of data.
PING interface.http.service.consul (10.0.0.85) 56(84) bytes of data.
PING interface.http.service.consul (10.0.1.22) 56(84) bytes of data.
PING interface.http.service.consul (10.0.0.85) 56(84) bytes of data.
However curl or wget won't alternate between these two IPs. I checked the DNS requests:
They always prefer the "more local" target during a multiple curl calls:
19:43:17.979701 IP nginx.stage.35800 > dns01.node.staging.consul.domain: 49288+ A? interface.http.service.consul. (54)
19:43:17.980586 IP dns01.node.staging.consul.domain > nginx.stage.35800: 49288* 2/0/0 A 10.0.1.22, A 10.0.0.85 (86)
19:43:19.056563 IP nginx.stage.56584 > dns01.node.staging.consul.domain: 44478+ A? interface.http.service.consul. (54)
19:43:19.057605 IP dns01.node.staging.consul.domain > nginx.stage.56584: 44478* 2/0/0 A 10.0.0.85, A 10.0.1.22 (86)
19:43:34.873807 IP nginx.facemantest.36293 > dns01.node.staging.consul.domain: 43958+ A? interface.http.service.consul. (54)
19:43:34.875065 IP dns01.node.staging.consul.domain > nginx.stage.36293: 43958* 2/0/0 A 10.0.0.85, A 10.0.1.22 (86)
So the DNS response have either 10.0.0.85 or 10.0.1.22 as first A record. But curl always uses 10.0.1.22, never 10.0.0.85.
The machine where I run curl has 10.0.1.10 as IP address. It also looks like this affects the nginx proxy pass mechanism.
Here my question: Do http clients check the IP, and choose an IP that looks closer? Can I disable such a behavior?
edits
Because you asked, this is how I test curl and wget:
while true; do wget -O - -4 -q interface.http.service.consul > /dev/null; sleep 1 ; done
while true; do curl -4 -s interface.http.service.consul > /dev/null; sleep 1 ; done
The output of wget from a host with the IP 10.0.1.10 is always
...
Resolving interface.http.service.consul (interface.http.service.consul)... 10.0.1.22, 10.0.0.85
...
The output of wget from a host with the IP 10.0.0.10 is always
...
Resolving interface.http.service.consul (interface.http.service.consul)... 10.0.0.85, 10.0.1.22
...
So wget shows indeed a sorting of A records based on "distance". Who is responsible for that, and how can I disable this?
I also watch the access logs on both HTTP servers behind the IPs 10.0.0.85 and 10.0.1.22
The DNS server is dnsmasq, which has a fallthrough to the DNS-server of consul. This is my local nsswitch:
cat /etc/nsswitch.conf
passwd: compat
group: compat
shadow: compat
gshadow: files
hosts: files dns
networks: files
protocols: db files
services: db files
ethers: db files
rpc: db files
netgroup: nis
This is the local resolv.conf
cat /etc/resolv.conf
# --- BEGIN PVE ---
search test
nameserver 10.0.1.2
nameserver 192.168.155.1
nameserver 8.8.8.8
# --- END PVE ---
There is no relevant entry in /etc/hosts. The whole DNS server config should be irrelevant, because I can see that identical DNS requests are performed, either by ping, wget or curl. But why do curl and wget always prefer to browse 10.0.1.22 when run from a host with 10.0.1.10,
ping
but not how you testcurl
/wget
. All applications should be governed by options in/etc/resolv.conf
that can have consequences on which IP are used (like thesortlist
option). But that would impactping
as well, so I do not know (but some applications do DNS themselves, not using the OS). Depending on how they query the OS, the applications might get back only one IP (as decided by the OS), so nothing to base a selection from. You could also try testing with temporary records in/etc/hosts
. And watch/etc/nsswitch.conf
too. Do you usenscd
?wget
seems to use the OS butcurl
seems to do DNS by itself.