python-requests – Page 2

sending raw data in python requests

January 3, 2024 by Tarik

There’s no issue sending raw post data: raw_data=”__VIEWSTATE=%2FwEPDwUJODgwODc4MzI2D2QWBAIEDxYCHgdWaXNpYmxlaGQCBg8WAh8AZxYCZg9kFhBmDw8WAh4EVGV4dAUl16jXmdep15XXnSDXntep16rXntepINeX15PXqSDXnNeQ16rXqGRkAgEPFgIeBWNsYXNzBSNmb3JtLWdyb3VwIGhhcy1mZWVkYmFjayBoYXMtc3VjY2VzcxYIAgEPDxYCHwEFLSog16nXnSDXntep16rXntepICjXkdeZ158gNCDXnCAxMiDXqteV15nXnSkgOmRkAgUPDxYCHwBoZGQCBw8PFgQeCENzc0NsYXNzZR4EXyFTQgICFgIfAgUwZ2x5cGhpY29uIGZvcm0tY29udHJvbC1mZWVkYmFjayBnbHlwaGljb24tcmVtb3ZlZAIJDxYGHwIFE2FsZXJ0IGFsZXJ0LXN1Y2Nlc3MeBXN0eWxlBQ5kaXNwbGF5OmJsb2NrOx4JaW5uZXJodG1sBRjXqdedINee16nXqtee16kg16TXoNeV15lkAgIPFgIfAGgWAmYPFgIfAgUXZm9ybS1ncm91cCBoYXMtZmVlZGJhY2sWBAIDDw8WBB8DZR8EAgJkZAIFDxYCHwUFDWRpc3BsYXk6bm9uZTtkAgMPFgIfAgUXZm9ybS1ncm91cCBoYXMtZmVlZGJhY2sWBgIBDw8WAh8AaGRkAgUPDxYEHwNlHwQCAmRkAgcPFgIfBQUNZGlzcGxheTpub25lO2QCBA8WAh8CBRdmb3JtLWdyb3VwIGhhcy1mZWVkYmFjaxYEAgMPDxYEHwNlHwQCAmRkAgUPFgIfBQUNZGlzcGxheTpub25lO2QCBQ8WAh8CBRdmb3JtLWdyb3VwIGhhcy1mZWVkYmFjaxYEAgMPDxYEHwNlHwQCAmRkAgUPFgIfBQUNZGlzcGxheTpub25lO2QCEw8WAh8CBRdmb3JtLWdyb3VwIGhhcy1mZWVkYmFjaxYEAgUPD2QWAh8FBQ1kaXNwbGF5Om5vbmU7ZAIHDw8WBB8DZR8EAgJkZAIXDxYGHwIFEmFsZXJ0IGFsZXJ0LWRhbmdlch8FBQ5kaXNwbGF5OmJsb2NrOx8GBSjXl9eV15HXlCDXnNeQ16nXqCDXkNeqINeq16DXkNeZINeU15DXqteoZBgBBR5fX0NvbnRyb2xzUmVxdWlyZVBvc3RCYWNrS2V5X18WAwUJYWxsb3dtYWlsBQhTZW5kTmV3cwUIY2hrdGVybXNHWETH5Z00Bw%2FRQ%2BPP7XKuKE1Yc7MfMt6j3NmJGEldOg%3D%3D&__VIEWSTATEGENERATOR=98F5786E&__EVENTVALIDATION=%2FwEWPwLc4fuhDgLPv6LBCALyveCRDwKt9JiEDQKyzcaDDQLQzfKVCgLi0PKVCgKwgbuWDQK4qIuIDAKDhrjXCAKT%2B%2B00AqGSlqELAoPh28wDAvSit9QCAsvep4YKAtj71VwC9pD07goC8rfjvgoCoryYqAECv6uC5A4C2dmTnQUCrdnKtgICyJiTrwsCjLGlBQKNsaUFAo6xpQUCj7GlBQKIsaUFAomxpQUCirGlBQKbsaUFApSxpQUCjLHlBgKMsekGAoyx7QYCjLHRBgKMsdUGAoyx2QYCjLHdBgKMscEGAoyxhQUCjLGJBQKNseUGAo2x6QYCjbHtBgKNsdEGAo2x1QYCjbHZBgKNsd0GAo2xwQYCjbGFBQKNsYkFAo6x5QYCjrHpBgLopM%2F9CwLsyZauAQL4oO3lBAK8yuqBCwL1hriZBwLUpvv7CALsqKmSDgLqoY%2BHAwLCi9reA5HII3R9bARNVKmrB9WBnfeJepHFJrdPAtcLnXlE%2BdKP&username2=gfk7ljlyks&password=boolbool1&password2=boolbool1&email=myeail%40gf.com&fname=&lname=&phone=&street=&city=&BYear=&Bmonth=%D7%99%D7%A0%D7%95%D7%90%D7%A8&Bday=1&career=&signature=&homepage=&icq=&Morehobbies=&allowmail=on&SendNews=on&chkterms=on&btnSubmit=%D7%9C%D7%97%D7%A5+%D7%9C%D7%A1%D7%99%D7%95%D7%9D+%D7%94%D7%94%D7%A8%D7%A9%D7%9E%D7%94+%D7%9C%D7%90%D7%AA%D7%A8″ requests.post(url, data=raw_data) A minor bonus is that your data is already percent encoded. From the doc string: post(url, data=None, json=None, **kwargs) Sends a POST request. :param url: URL for the new :class:`Request` object. :param data: (optional) Dictionary, bytes, or file-like object to send in the body of … Read more

How can I set a single proxy for a requests session object?

December 29, 2023 by Tarik

In addition to @neowu’ answer, if you would like to set a proxy for the lifetime of a session object, you can also do the following – import requests proxies = {‘http’: ‘http://10.11.4.254:3128’} s = requests.session() s.proxies.update(proxies) s.get(“http://www.example.com”) # Here the proxies will also be automatically used because we have attached those to the session … Read more

Requests — always call raise_for_status

December 24, 2023 by Tarik

Create a session with a hook: session = requests.Session() session.hooks = { ‘response’: lambda r, *args, **kwargs: r.raise_for_status() } x = session.get(url1) y = session.delete(url2) z = session.post(url3, data={‘foo’: ‘bar’})

Set port in requests

December 24, 2023 by Tarik

Request is an HTTP library. You can specify the port in the URL http://example.com:4028/…. But, from what I can read in a hurry here cgminer provides a RPC API (or JSON RPC?) not an HTTP interface.

Create url without request execution

December 22, 2023 by Tarik

You can use PreparedRequests. To build the URL, you can build your own Request object and prepare it: from requests import Session, Request s = Session() p = Request(‘GET’, ‘http://someurl.com’, params=request_parameters).prepare() log.append(p.url) Later, when you’re ready to send, you can just do this: r = s.send(p) The relevant section of the documentation is here.

Python Requests POST doing a GET?

December 20, 2023 by Tarik

To be clear, whenever requests receives a redirect (with a certain status code) we have to perform certain transformations on the request. In cases like this, when you see something very unexpected the best debugging tips are to retry your request but with allow_redirects=False. This will immediately return the 30x response. Alternatively, you can also … Read more

Convert text data from requests object to dataframe with pandas

December 19, 2023 by Tarik

Try this import requests import pandas as pd import io urlData = requests.get(url).content rawData = pd.read_csv(io.StringIO(urlData.decode(‘utf-8’)))

Python Requests getting (‘Connection aborted.’, BadStatusLine(“””,)) error

December 13, 2023 by Tarik

The error you get indicates the host isn’t responding in the expected manner. In this case, it’s because it detects that you’re trying to scrape it and deliberately disconnecting you. If you try your requests code with this URL from a test website: http://mirror.internode.on.net/pub/test/5meg.test1, you’ll see that it downloads normally. To get around this, fake … Read more

Python Requests and Unicode

December 13, 2023 by Tarik

You can use req.text instead of req.content to ensure that you get Unicode. This is described in: https://requests.readthedocs.io/en/latest/api/#requests.Response.text

unable to decode Python web request

December 11, 2023 by Tarik

According to the response.headers (that you did not provide, but that are easily recoverable by running your code), the response is encoded using Brotli compression (Content-Encoding’: ‘br’). You can decompress it with brotlipy: import brotli brotli.decompress(response.content) #b'{“status”:”success”,”data”:[{“placed_by”:”XE4670″,”order_id”:”180331000000385″, #”exchange_order_id”:null,”parent_order_id”:null,”status”:”REJECTED”, #”status_message”:”ADAPTER is down”,”order_timestamp”:”2018-03-31 07:59:42″, #”exchange_update_timestamp”:null,…} Now, it’s JSON, as promised (‘Content-Type’: ‘application/json’).