You should NEVER set that header yourself. We set the header properly with the boundary. If you set that header, we won’t and your server won’t know what boundary to expect (since it is added to the header). Remove your custom Content-Type header and you’ll be fine.
Add a ‘Cache-Control: no-cache’ header: self.request = requests.get(‘http://google.com’, headers={‘Cache-Control’: ‘no-cache’}) See https://stackoverflow.com/a/55613686/469045 for complete answer.
It doesn’t look like a problem of waiting, it looks like the element is being created by JavaScript, requests can’t handle dynamically generated elements by JavaScript. A suggestion is to use selenium together with PhantomJS to get the page source, then you can use BeautifulSoup for your parsing, the code shown below will do exactly … Read more
To upload a list of files with the same key value in a single request, you can create a list of tuples with the first item in each tuple as the key value and the file object as the second: files = [(‘file’, open(‘report.xls’, ‘rb’)), (‘file’, open(‘report2.xls’, ‘rb’))]
It looks like the server is terminating the connection early without sending a full response. I’ve skimmed the docs and I think this is the problem (emphasis added): send_response(code, message=None) Adds a response header to the headers buffer and logs the accepted request. The HTTP response line is written to the internal buffer, followed by … Read more
The server in question is giving you a gzipped response. The server is also very broken; it sends the following headers: $ curl -D – -o /dev/null -s -H ‘Accept-Encoding: gzip, deflate’ http://www.wrcc.dri.edu/WRCCWrappers.py?sodxtrmts+028815+por+por+pcpn+none+mave+5+01+F HTTP/1.1 200 OK Date: Tue, 06 Jan 2015 17:46:49 GMT Server: Apache <!DOCTYPE HTML PUBLIC “-//W3C//DTD XHTML 1.0 Transitional//EN” “DTD/xhtml1-transitional.dtd”><html xmlns=”http: //www.w3.org/1999/xhtml” … Read more
Improving on @alecxe’s answer: if you use a Python Generator and a requests HTTP session you can improve the performance and resource usage if you are querying lots of pages or very large pages. import requests session = requests.Session() def get_jobs(): url = “https://api.angel.co/1/tags/1664/jobs” first_page = session.get(url).json() yield first_page num_pages = first_page[‘last_page’] for page in … Read more
To follow up on @WeaselFox’s answer, they introduced a patch that accepts a quote_via keyword argument to urllib.parse.urlencode. Now you could do this: import requests import urllib payload = {‘key1’: ‘value 1’, ‘key2’: ‘value 2’} headers = {‘Content-Type’: ‘application/json;charset=UTF-8’} params = urllib.parse.urlencode(payload, quote_via=urllib.parse.quote) r = requests.get(“http://example.com/service”, params=params, headers=headers, auth=(“admin”, “password”))
this is not an extra answer just try to combine the solution code from question with extra information So others can copy it directly without extra try It is not only a DH Key issues in server side, but also lots of different libraries are mismatched in python modules. Code segment below is used to … Read more
If you want the response to raise an exception for a non-200 status code use response.raise_for_status(). Your code would then look like: testURL = ‘http://httpbin.org/status/404’ def return_json(URL): response = requests.get(testURL) try: response.raise_for_status() except requests.exceptions.HTTPError as e: # Whoops it wasn’t a 200 return “Error: ” + str(e) # Must have been a 200 status code … Read more