DNS resolution for pools on XROOT doors

[cfgamboa]

Dear all,

Currently we are testing dCache doors with standard pool redirection for transfers.
I noticed that unlike with DAVs I must make sure the pool’s IPs must be on /etc/hosts at the door to be able to make the transfer work.

Why I do not have to do this for DAVs to work?

See below:
For xrootd doors tests external and internal transfers need the door to be able to resolve to external IP address pair of the pool or the internal depending on the clients network locality the:
Example external client access via xroot

[cgamboa@lxplus914 ~]$ gfal-copy -f /etc/services root://dcint-door002.sdcc.bnl.gov:1096/pnfs/usatlas.bnl.gov/cgamboa/test.root.write.cern.secure.1
Copying file:///etc/services   [FAILED]  after 0s                                                                                                                            
gfal-copy error: 29 (Illegal seek) - Error on XrdCl::CopyProcess::Run(): [ERROR] Invalid redirect URL:  (destination)

Door logs

At the door:

=============
1 Nov 2024 11:03:42 (Xrootd-dcint-door002-externalipv6) [door:Xrootd-dcint-door002-externalipv6@xrootd-dcint-door002Domain:AAYl2z2eJcA] Unable to resolve IP address 2620:0:210:1:0:0:0:80 to a canonical name
=============

The pool's DNS resolution is at the door is:

[root@dcint-door002 ~]# ping dcint-pool04
PING dcint-pool04(dcint-pool04.sdcc.bnl.gov (2620:0:210:8803::112)) 56 data bytes
64 bytes from dcint-pool04.sdcc.bnl.gov (2620:0:210:8803::112): icmp_seq=1 ttl=61 time=0.262 ms
64 bytes from dcint-pool04.sdcc.bnl.gov (2620:0:210:8803::112): icmp_seq=2 ttl=61 time=0.247 ms
 
[root@dcint-door002 ~]# nslookup dcint-pool04
Server:                 10.42.34.6
Address:             10.42.34.6#53
 
Name:  dcint-pool04.sdcc.bnl.gov
Address: 10.42.38.112
Name:  dcint-pool04.sdcc.bnl.gov
Address: 2620:0:210:8803::112
Transfer works, if explicitly the pool external pair is enabled to resolve at the door
 
[root@dcint-door002 ~]# cat /etc/hosts|grep pool04
2620:0:210:1::80            dcint-pool04.sdcc.bnl.gov         dcint-pool04
192.12.15.128                  dcint-pool04.sdcc.bnl.gov         dcint-pool04
 
 
Reload nscd
systemctl reload nscd.service
 
Check the external pair is resolved
ping6 dcint-pool04
ping dcint-pool04

The xroot door needed to be reloaded

[root@dcint-door002 ~]# systemctl list-dependencies dcache.target
dcache.target
● ├─dcache@webdav-dcint-door002_httpsDomain.service
● └─[email protected]
[root@dcint-door002 ~]# systemctl restart dcache.target


[cgamboa@lxplus914 ~]$ gfal-copy -f /etc/services root://dcint-door002.sdcc.bnl.gov:1096/pnfs/usatlas.bnl.gov/cgamboa/test.root.write.cern.secure.1
Copying file:///etc/services   [DONE]  after 2s  
For secure xroot the IPV4/IPV6 counter parts needs to exits in the /etc/hosts resolution otherwise
[cgamboa@spar0101 ~]$ gfal-copy -f /etc/services roots://dcint-door002.sdcc.bnl.gov:1096/pnfs/usatlas.bnl.gov/cgamboa/test.root.write.cern.secure.1
Copying file:///etc/services   [FAILED]  after 1s                                                                                                                            
gfal-copy error: 40 (Too many levels of symbolic links) - Error on XrdCl::CopyProcess::Run(): [FATAL] Redirect limit has been reached:  (destination)

At the door,

01 Nov 2024 11:35:22 (Xrootd-dcint-door002-internal) [door:Xrootd-dcint-door002-internal@xrootd-dcint-door002Domain:AAYl267UoWA] Unable to resolve IP address 10.42.38.112 to a canonical name

Enabling the pools external and internal IPV4/V6 counterparts in the /etc/hosts file allows root and roots transfers to work.

gfal-copy -f /etc/services roots://dcint-door002.sdcc.bnl.gov:1096/pnfs/usatlas.bnl.gov/cgamboa/test.root.write.cern.secure.1
Copying file:///etc/services   [DONE]  after 3s

For DAVs this is not necessary. Transfers worked without issue.

[DmitryLitvintsev]

Hi Carlos,

I looked at your issue a little more carefully and realized this is not what I initially thought. In our case on dual stack pool nodes that resolve both IPv4 and IPv6 we had to add explicit name resolution to /etc/hosts on these pools but not on the door.

In your case you are adding pool name resolution to the door /etc/host. That's not right.

I have question

You pool node resolves:

[root@dcint-door002 ~]# nslookup dcint-pool04
Server:                 10.42.34.6
Address:             10.42.34.6#53
 
Name:  dcint-pool04.sdcc.bnl.gov
Address: 10.42.38.112
Name:  dcint-pool04.sdcc.bnl.gov
Address: 2620:0:210:8803::112

But you put :

[root@dcint-door002 ~]# cat /etc/hosts|grep pool04
2620:0:210:1::80            dcint-pool04.sdcc.bnl.gov         dcint-pool04
192.12.15.128                  dcint-pool04.sdcc.bnl.gov         dcint-pool04

in /etc/hosts on the door. Why these IPs don't match nslookup output?

[cfgamboa]

Hi Dmitry,

Thank you for clarifying we did not have to do anything on the pools. Only in the doors for this DUAL home
All 4 IPs needs to be able to be on

cat /etc/hosts|grep pool04
2620:0:210:1::80                dcint-pool04.sdcc.bnl.gov       dcint-pool04
192.12.15.128                   dcint-pool04.sdcc.bnl.gov       dcint-pool04
10.42.38.112                    dcint-pool04.sdcc.bnl.gov       dcint-pool04
2620:0:210:8803::112            dcint-pool04.sdcc.bnl.gov       dcint-pool04

The resolution via DNS is enabled on doors and pools resolve to internal (IPV4/IPV6) components. Currently the production system will prefer using the internal LAN to communicate to all elements on the system.

This is why the internal components are resolve via DNS

[root@dcint-door002 ~]# nslookup dcint-pool04
Server:                 10.42.34.6
Address:             10.42.34.6#53
 
Name:  dcint-pool04.sdcc.bnl.gov
Address: 10.42.38.112
Name:  dcint-pool04.sdcc.bnl.gov
Address: 2620:0:210:8803::112

Carlos