I am developing and testing (via requests) the code of a remote server, from some day I suddenly began to receive responses with a delay of 70-80 seconds to the simplest requests. At the same time, the same requests from my computer via curl, wget or the browser is executed instantly, like the same Python script run from third-party servers.

I tried to connect both over httpS and HTTP (apparently it's not about SSL), locally using Python 3.7 and 2.7, the result is the same everywhere. I have an OS MacOSX 10.15.1, Ubuntu 18.04.5, NginX and Aiohttp are on the server, but I raised hello-world on Flask'e, the result is the same, apparently, the server code does not affect either.

Question: why does the request take so long, from my computer to this particular server and only through Python?! And how can this be better handled to find the cause of the problem?

I tried this, but it didn't do much good:

import requests
import http.client
import logging

logging.basicConfig(level=logging.DEBUG)
http.client.HTTPConnection.debuglevel = 1
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

# Домен конечно изменён
response = requests.get('http://sub.test.com:5990/')
print('** Code:', response.status_code)
print('** Response:', response.content.decode('utf-8'))

Conclusion:

DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): sub.test.com:5990
# Спустя 75 секунд:
send: b'GET / HTTP/1.1\r\nHost: sub.test.com:5990\r\nUser-Agent: python-requests/2.22.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: text/html; charset=utf-8
header: Content-Length: 13
header: Date: Thu, 03 Sep 2020 19:26:25 GMT
header: Server: Python/3.6 aiohttp/3.6.2
DEBUG:urllib3.connectionpool:http://sub.test.com:5990" GET / HTTP/1.1" 200 13
** Code: 200
** Response: Hello, World!

If you kill the script while waiting, the traceback it turns out this way:

Traceback (most recent call last):
  File "test.py", line 19, in <module>
    response = requests.get(URL)
  File "/Library/Python/3.7/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/Library/Python/3.7/site-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Library/Python/3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Python/3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Python/3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/Library/Python/3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/Library/Python/3.7/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Applications/.../Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Applications/.../Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Applications/.../Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Applications/.../Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/Applications/.../Python3.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/Library/Python/3.7/site-packages/urllib3/connection.py", line 184, in connect
    conn = self._new_conn()
  File "/Library/Python/3.7/site-packages/urllib3/connection.py", line 157, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/Library/Python/3.7/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)

Update

It was noticed that specifying the IP address the connection is successful, apparently the problem is in DNS resolve. The code analog from the falling urllib3/util/connection.py is executed without problems:

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(('sub.test.com', 5990))
sock.close()

However, it was found that in the real connection.py, socket.AF_INET6 is used, and with this argument, the connection just hangs.

Solution

It turned out that the domain in the DNS has an entry AAAA with an invalid IPv6 address, in addition to A an entry with a valid IPv4, and deleting this entry solved the problem.