2

I have consul as nameserver that resolves address from two instances of a service. DNS info via dig interface.http.service.consul:

...
interface.http.service.consul. 0    IN A    10.0.0.85
interface.http.service.consul. 0    IN A    10.0.1.22
...

ping will alternate between both addresses:

while true; do ping -c 1 interface.http.service.consul | grep PING; sleep 1; done
PING interface.http.service.consul (10.0.0.85) 56(84) bytes of data.
PING interface.http.service.consul (10.0.0.85) 56(84) bytes of data.
PING interface.http.service.consul (10.0.0.85) 56(84) bytes of data.
PING interface.http.service.consul (10.0.1.22) 56(84) bytes of data.
PING interface.http.service.consul (10.0.0.85) 56(84) bytes of data.
PING interface.http.service.consul (10.0.1.22) 56(84) bytes of data.
PING interface.http.service.consul (10.0.0.85) 56(84) bytes of data.

However curl or wget won't alternate between these two IPs. I checked the DNS requests:

They always prefer the "more local" target during a multiple curl calls:

19:43:17.979701 IP nginx.stage.35800 > dns01.node.staging.consul.domain: 49288+ A? interface.http.service.consul. (54)
19:43:17.980586 IP dns01.node.staging.consul.domain > nginx.stage.35800: 49288* 2/0/0 A 10.0.1.22, A 10.0.0.85 (86)
19:43:19.056563 IP nginx.stage.56584 > dns01.node.staging.consul.domain: 44478+ A? interface.http.service.consul. (54)
19:43:19.057605 IP dns01.node.staging.consul.domain > nginx.stage.56584: 44478* 2/0/0 A 10.0.0.85, A 10.0.1.22 (86)
19:43:34.873807 IP nginx.facemantest.36293 > dns01.node.staging.consul.domain: 43958+ A? interface.http.service.consul. (54)
19:43:34.875065 IP dns01.node.staging.consul.domain > nginx.stage.36293: 43958* 2/0/0 A 10.0.0.85, A 10.0.1.22 (86)

So the DNS response have either 10.0.0.85 or 10.0.1.22 as first A record. But curl always uses 10.0.1.22, never 10.0.0.85.

The machine where I run curl has 10.0.1.10 as IP address. It also looks like this affects the nginx proxy pass mechanism.

Here my question: Do http clients check the IP, and choose an IP that looks closer? Can I disable such a behavior?

edits

Because you asked, this is how I test curl and wget:

while true; do wget -O - -4 -q interface.http.service.consul > /dev/null; sleep 1 ; done
while true; do curl -4 -s interface.http.service.consul > /dev/null; sleep 1 ; done

The output of wget from a host with the IP 10.0.1.10 is always

...
Resolving interface.http.service.consul (interface.http.service.consul)... 10.0.1.22, 10.0.0.85
...

The output of wget from a host with the IP 10.0.0.10 is always

...
Resolving interface.http.service.consul (interface.http.service.consul)... 10.0.0.85, 10.0.1.22
...

So wget shows indeed a sorting of A records based on "distance". Who is responsible for that, and how can I disable this?

I also watch the access logs on both HTTP servers behind the IPs 10.0.0.85 and 10.0.1.22

The DNS server is dnsmasq, which has a fallthrough to the DNS-server of consul. This is my local nsswitch:

cat /etc/nsswitch.conf 
passwd:         compat
group:          compat
shadow:         compat
gshadow:        files

hosts:          files dns
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis

This is the local resolv.conf

cat /etc/resolv.conf 
# --- BEGIN PVE ---
search test
nameserver 10.0.1.2
nameserver 192.168.155.1
nameserver 8.8.8.8
# --- END PVE ---

There is no relevant entry in /etc/hosts. The whole DNS server config should be irrelevant, because I can see that identical DNS requests are performed, either by ping, wget or curl. But why do curl and wget always prefer to browse 10.0.1.22 when run from a host with 10.0.1.10,

  • You are showing how you test ping but not how you test curl/wget. All applications should be governed by options in /etc/resolv.conf that can have consequences on which IP are used (like the sortlist option). But that would impact ping as well, so I do not know (but some applications do DNS themselves, not using the OS). Depending on how they query the OS, the applications might get back only one IP (as decided by the OS), so nothing to base a selection from. You could also try testing with temporary records in /etc/hosts. And watch /etc/nsswitch.conf too. Do you use nscd? – Patrick Mevzek Jan 12 '18 at 23:01
  • From their documentation: wget seems to use the OS but curl seems to do DNS by itself. – Patrick Mevzek Jan 12 '18 at 23:04
1

Since wget and curl should be using getaddrinfo (I don't know about ping), checked the man page on my archlinux (BSD man page did not show this):

There are several reasons why the linked list may have more than one addrinfo structure, including: the network host is multihomed, accessible over multiple protocols (e.g., both AF_INET and AF_INET6); or the same service is available from multiple socket types (one SOCK_STREAM address and another SOCK_DGRAM address, for example). Normally, the application should try using the addresses in the order in which they are returned. The sorting function used within getaddrinfo() is defined in RFC 3484; the order can be tweaked for a particular system by editing /etc/gai.conf (available since glibc 2.5).

I need to go deeper... read RFC 3484 and modify /etc/gai.conf

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.