httplib – Tarik Billa

How do I get the IP address from a http request using the requests library?

November 28, 2023 by Tarik

It turns out that it’s rather involved. Here’s a monkey-patch while using requests version 1.2.3: Wrapping the _make_request method on HTTPConnectionPool to store the response from socket.getpeername() on the HTTPResponse instance. For me on python 2.7.3, this instance was available on response.raw._original_response. from requests.packages.urllib3.connectionpool import HTTPConnectionPool def _make_request(self,conn,method,url,**kwargs): response = self._old_make_request(conn,method,url,**kwargs) sock = getattr(conn,’sock’,False) if … Read more

error: can’t start new thread

July 30, 2023 by Tarik

The “can’t start new thread” error almost certainly due to the fact that you have already have too many threads running within your python process, and due to a resource limit of some kind the request to create a new thread is refused. You should probably look at the number of threads you’re creating; the … Read more

Python httplib ResponseNotReady

May 30, 2023 by Tarik

Previous answers are correct, but there’s another case where you could get that exception: Making multiple requests without reading any intermediate responses completely. For instance: conn.request(‘PUT’,…) conn.request(‘GET’,…) # will not work: raises ResponseNotReady conn.request(‘PUT’,…) r = conn.getresponse() r.read() # <– that’s the important call! conn.request(‘GET’,…) r = conn.getresponse() r.read() # <– same thing and so … Read more

HTTPS connection Python

May 10, 2023 by Tarik

Python 2.x: docs.python.org/2/library/httplib.html: Note: HTTPS support is only available if the socket module was compiled with SSL support. Python 3.x: docs.python.org/3/library/http.client.html: Note HTTPS support is only available if Python was compiled with SSL support (through the ssl module). #!/usr/bin/env python import httplib c = httplib.HTTPSConnection(“ccc.de”) c.request(“GET”, “https://stackoverflow.com/”) response = c.getresponse() print response.status, response.reason data = … Read more

Python urllib vs httplib?

April 8, 2023 by Tarik

urllib (particularly urllib2) handles many things by default or has appropriate libs to do so. For example, urllib2 will follow redirects automatically and you can use cookiejar to handle login scripts. These are all things you’d have to code yourself if you were using httplib.

Python-Requests close http connection

February 27, 2023 by Tarik

I think a more reliable way of closing a connection is to tell the sever explicitly to close it in a way compliant with HTTP specification: HTTP/1.1 defines the “close” connection option for the sender to signal that the connection will be closed after completion of the response. For example, Connection: close in either the … Read more

Proxying to another web service with Flask

February 13, 2023 by Tarik

I spent a good deal of time working on this same thing and eventually found a solution using the requests library that seems to work well. It even handles setting multiple cookies in one response, which took a bit of investigation to figure out. Here’s the flask view function: from dotenv import load_dotenv # pip … Read more

How to send POST request?

September 28, 2022 by Tarik

If you really want to handle with HTTP using Python, I highly recommend Requests: HTTP for Humans. The POST quickstart adapted to your question is: >>> import requests >>> r = requests.post(“http://bugs.python.org”, data={‘number’: ‘12524’, ‘type’: ‘issue’, ‘action’: ‘show’}) >>> print(r.status_code, r.reason) 200 OK >>> print(r.text[:300] + ‘…’) <!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Transitional//EN” “http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd”> … Read more