requests does not handle parsing XML responses, no. XML responses are much more complex in nature than JSON responses, how you’d serialize XML data into Python structures is not nearly as straightforward. Python comes with built-in XML parsers. I recommend you use the ElementTree API: import requests from xml.etree import ElementTree response = requests.get(url) tree … Read more
Requests doesn’t have an equivalent to Urlib2’s read(). >>> import requests >>> response = requests.get(“http://www.google.com”) >>> print response.content ‘<!doctype html><html itemscope=”” itemtype=”http://schema.org/WebPage”><head>….’ >>> print response.content == response.text True It looks like the POST request you are making is returning no content. Which is often the case with a POST request. Perhaps it set a cookie? … Read more
The latest release of Requests will build CookieJars for you from simple dictionaries. import requests cookies = {‘enwiki_session’: ’17ab96bd8ffbe8ca58a78657a918558′} r = requests.post(‘http://wikipedia.org’, cookies=cookies) Enjoy 🙂
Just use requests features: import requests from requests.adapters import HTTPAdapter from urllib3.util.retry import Retry session = requests.Session() retry = Retry(connect=3, backoff_factor=0.5) adapter = HTTPAdapter(max_retries=retry) session.mount(‘http://’, adapter) session.mount(‘https://’, adapter) session.get(url) This will GET the URL and retry 3 times in case of requests.exceptions.ConnectionError. backoff_factor will help to apply delays between attempts to avoid to fail again … Read more
You need to enable debugging at httplib level (requests → urllib3 → httplib). Here’s some functions to both toggle (…_on() and …_off()) or temporarily have it on: import logging import contextlib try: from http.client import HTTPConnection # py3 except ImportError: from httplib import HTTPConnection # py2 def debug_requests_on(): ”’Switches on logging of the requests module.”’ … Read more
try: r = requests.post(url, data=data, verify=’/path/to/public_key.pem’)
You can easily create a persistent session using: s = requests.Session() After that, continue with your requests as you would: s.post(‘https://localhost/login.py’, login_data) # logged in! cookies saved for future requests. r2 = s.get(‘https://localhost/profile_data.json’, …) # cookies sent automatically! # do whatever, s will keep your cookies intact 🙂 For more about Sessions: https://requests.readthedocs.io/en/latest/user/advanced/#session-objects
From http://docs.python-requests.org/en/latest/user/quickstart/ url=”https://api.github.com/some/endpoint” payload = {‘some’: ‘data’} headers = {‘content-type’: ‘application/json’} r = requests.post(url, data=json.dumps(payload), headers=headers) You just need to create a dict with your headers (key: value pairs where the key is the name of the header and the value is, well, the value of the pair) and pass that dict to the headers … Read more
The requests.Response class documentation has more details: r.text is the content of the response in Unicode, and r.content is the content of the response in bytes.
Provide a User-Agent header: import requests url=”http://www.ichangtou.com/#company:data_000008.html” headers = {‘User-Agent’: ‘Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36’} response = requests.get(url, headers=headers) print(response.content) FYI, here is a list of User-Agent strings for different browsers: List of all Browsers As a side note, there is a pretty useful third-party package called … Read more