You can either use the response.raw file object, or iterate over the response. To use the response.raw file-like object will not, by default, decode compressed responses (with GZIP or deflate). You can force it to decompress for you anyway by setting the decode_content attribute to True (requests sets it to False to control decoding itself). … Read more
With the following streaming code, the Python memory usage is restricted regardless of the size of the downloaded file: def download_file(url): local_filename = url.split(“https://stackoverflow.com/”)[-1] # NOTE the stream=True parameter below with requests.get(url, stream=True) as r: r.raise_for_status() with open(local_filename, ‘wb’) as f: for chunk in r.iter_content(chunk_size=8192): # If you have chunk encoded response uncomment if # … Read more
Have a look at the Requests exception docs. In short: In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception. In the event of the rare invalid HTTP response, Requests will raise an HTTPError exception. If a request times out, a Timeout exception is raised. If … Read more
Requests is not a built in module (does not come with the default python installation), so you will have to install it: OSX/Linux Python 2: sudo pip install requests Python 3: sudo pip3 install requests if you have pip installed (pip is the package installer for python and should come by default with your python … Read more
I know it’s been said already, but I’d highly recommend the requests Python package. If you’ve used languages other than python, you’re probably thinking urllib and urllib2 are easy to use, not much code, and highly capable, that’s how I used to think. But the requests package is so unbelievably useful and short that everyone … Read more
Starting with Requests version 2.4.2, you can use the json= parameter (which takes a dictionary) instead of data= (which takes a string) in the call: >>> import requests >>> r = requests.post(‘http://httpbin.org/post’, json={“key”: “value”}) >>> r.status_code 200 >>> r.json() {‘args’: {}, ‘data’: ‘{“key”: “value”}’, ‘files’: {}, ‘form’: {}, ‘headers’: {‘Accept’: ‘*/*’, ‘Accept-Encoding’: ‘gzip, deflate’, ‘Connection’: … Read more