HTTP Error 403: Forbidden, python3 urlllib, http://cbr.ru/scripts/XML daily.asp
The script runs normally from the local computer. the xml file is accepted successfully. But when I run the following script from the hosting pythonanywhere.com the HTTP Error 403: Forbidden error appears. Can anyone tell me what the problem is and how to get around it?
Part of the code:
import urllib, os
from xml.etree import ElementTree as ET
def update_currencies():
url = 'http://cbr.ru/scripts/XML_daily.asp'
hdr = {}
hdr['User-Agent'] = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.111 Safari/537.36'
hdr['Accept'] = 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
req = urllib.request.Request(url, headers=hdr)
content = urllib.request.urlopen(req)
tree = ET.parse(content)
root = tree.getroot()
...
Thanks for attention.
2 answers
Apparently, the website of the Central Bank of the Russian Federation does not like when it is accessed too often. Add a cache of requests for the exchange rate of the currency you are interested in (for example, by saving a locally downloaded data file), wait a day and try downloading the data again.
You can also refer to one of the well-known mirrors of the Central Bank of the Russian Federation service. You can also find course data in a more user-friendly format like JSON.
Example of issuing such a mirror:
Compare with the original:
Description of other output formats on the service's website.
It is not a fact that any such services will react well to the fact that you will pump out exchange rates every time a user visits the site. You should always cache the received exchange rates, if only because they are updated once a day and only by default. weekdays.
Use a proxy server and check if the error persists. You can test it by Smoking. If everything works when using a proxy, then this may mean:
- You were banned for too frequent requests. As mentioned in the next answer-cache the result.
- The entire hosting is banned for various reasons (the entire subnet was banned due to frequent requests or something like that). In this case, you either need to use a proxy, or change the hosting. Alternatively, write a tearful request to remove ban.