noindex and nofollow mean that you do not want any search engines like Google to crawl your website. So, simply put the following code into your robots.txt file: User-agent: * Disallow: / It means noindex and nofollow.
You may create and use an http.Request forged by you, which deliberately returns an error when reading its body. You don’t necessarily need a whole new request, a faulty body is enough (which is an io.ReadCloser). Simplest achieved by using the httptest.NewRequest() function where you can pass an io.Reader value which will be used (wrapped … Read more
There’s no issue sending raw post data: raw_data=”__VIEWSTATE=%2FwEPDwUJODgwODc4MzI2D2QWBAIEDxYCHgdWaXNpYmxlaGQCBg8WAh8AZxYCZg9kFhBmDw8WAh4EVGV4dAUl16jXmdep15XXnSDXntep16rXntepINeX15PXqSDXnNeQ16rXqGRkAgEPFgIeBWNsYXNzBSNmb3JtLWdyb3VwIGhhcy1mZWVkYmFjayBoYXMtc3VjY2VzcxYIAgEPDxYCHwEFLSog16nXnSDXntep16rXntepICjXkdeZ158gNCDXnCAxMiDXqteV15nXnSkgOmRkAgUPDxYCHwBoZGQCBw8PFgQeCENzc0NsYXNzZR4EXyFTQgICFgIfAgUwZ2x5cGhpY29uIGZvcm0tY29udHJvbC1mZWVkYmFjayBnbHlwaGljb24tcmVtb3ZlZAIJDxYGHwIFE2FsZXJ0IGFsZXJ0LXN1Y2Nlc3MeBXN0eWxlBQ5kaXNwbGF5OmJsb2NrOx4JaW5uZXJodG1sBRjXqdedINee16nXqtee16kg16TXoNeV15lkAgIPFgIfAGgWAmYPFgIfAgUXZm9ybS1ncm91cCBoYXMtZmVlZGJhY2sWBAIDDw8WBB8DZR8EAgJkZAIFDxYCHwUFDWRpc3BsYXk6bm9uZTtkAgMPFgIfAgUXZm9ybS1ncm91cCBoYXMtZmVlZGJhY2sWBgIBDw8WAh8AaGRkAgUPDxYEHwNlHwQCAmRkAgcPFgIfBQUNZGlzcGxheTpub25lO2QCBA8WAh8CBRdmb3JtLWdyb3VwIGhhcy1mZWVkYmFjaxYEAgMPDxYEHwNlHwQCAmRkAgUPFgIfBQUNZGlzcGxheTpub25lO2QCBQ8WAh8CBRdmb3JtLWdyb3VwIGhhcy1mZWVkYmFjaxYEAgMPDxYEHwNlHwQCAmRkAgUPFgIfBQUNZGlzcGxheTpub25lO2QCEw8WAh8CBRdmb3JtLWdyb3VwIGhhcy1mZWVkYmFjaxYEAgUPD2QWAh8FBQ1kaXNwbGF5Om5vbmU7ZAIHDw8WBB8DZR8EAgJkZAIXDxYGHwIFEmFsZXJ0IGFsZXJ0LWRhbmdlch8FBQ5kaXNwbGF5OmJsb2NrOx8GBSjXl9eV15HXlCDXnNeQ16nXqCDXkNeqINeq16DXkNeZINeU15DXqteoZBgBBR5fX0NvbnRyb2xzUmVxdWlyZVBvc3RCYWNrS2V5X18WAwUJYWxsb3dtYWlsBQhTZW5kTmV3cwUIY2hrdGVybXNHWETH5Z00Bw%2FRQ%2BPP7XKuKE1Yc7MfMt6j3NmJGEldOg%3D%3D&__VIEWSTATEGENERATOR=98F5786E&__EVENTVALIDATION=%2FwEWPwLc4fuhDgLPv6LBCALyveCRDwKt9JiEDQKyzcaDDQLQzfKVCgLi0PKVCgKwgbuWDQK4qIuIDAKDhrjXCAKT%2B%2B00AqGSlqELAoPh28wDAvSit9QCAsvep4YKAtj71VwC9pD07goC8rfjvgoCoryYqAECv6uC5A4C2dmTnQUCrdnKtgICyJiTrwsCjLGlBQKNsaUFAo6xpQUCj7GlBQKIsaUFAomxpQUCirGlBQKbsaUFApSxpQUCjLHlBgKMsekGAoyx7QYCjLHRBgKMsdUGAoyx2QYCjLHdBgKMscEGAoyxhQUCjLGJBQKNseUGAo2x6QYCjbHtBgKNsdEGAo2x1QYCjbHZBgKNsd0GAo2xwQYCjbGFBQKNsYkFAo6x5QYCjrHpBgLopM%2F9CwLsyZauAQL4oO3lBAK8yuqBCwL1hriZBwLUpvv7CALsqKmSDgLqoY%2BHAwLCi9reA5HII3R9bARNVKmrB9WBnfeJepHFJrdPAtcLnXlE%2BdKP&username2=gfk7ljlyks&password=boolbool1&password2=boolbool1&email=myeail%40gf.com&fname=&lname=&phone=&street=&city=&BYear=&Bmonth=%D7%99%D7%A0%D7%95%D7%90%D7%A8&Bday=1&career=&signature=&homepage=&icq=&Morehobbies=&allowmail=on&SendNews=on&chkterms=on&btnSubmit=%D7%9C%D7%97%D7%A5+%D7%9C%D7%A1%D7%99%D7%95%D7%9D+%D7%94%D7%94%D7%A8%D7%A9%D7%9E%D7%94+%D7%9C%D7%90%D7%AA%D7%A8″ requests.post(url, data=raw_data) A minor bonus is that your data is already percent encoded. From the doc string: post(url, data=None, json=None, **kwargs) Sends a POST request. :param url: URL for the new :class:`Request` object. :param data: (optional) Dictionary, bytes, or file-like object to send in the body of … Read more
private handleError(error: any) { // previously // return Observable.throw(‘Some error information’); // now return throwError(‘Some error information’); } See also How to catch exception correctly from http.request()?
Try ETag: “4ae413bd”. The value of an ETag must follow the ABNF form: entity-tag = [ weak ] opaque-tag weak = “W/” opaque-tag = quoted-string quoted-string = ( <“> *(qdtext | quoted-pair ) <“> ) qdtext = <any TEXT except <“>> quoted-pair = “\” CHAR CHAR = <any US-ASCII character (octets 0 – 127)> TEXT … Read more
“a very simple proxy example that can be connected to and will then itself try to connect to the address passed to it.” That is practically the definition of an HTTP proxy. There’s a really simple proxy example here: http://effbot.org/librarybook/simplehttpserver.htm The core of it is just 3 lines: class Proxy(SimpleHTTPServer.SimpleHTTPRequestHandler): def do_GET(self): self.copyfile(urllib.urlopen(self.path), self.wfile) So … Read more
Ok, this seems to be a by-design behavior and a perfect example of a vexing exception. This can be solved with this: public static HttpWebResponse GetHttpResponse(this HttpWebRequest request) { try { return (HttpWebResponse) request.GetResponse(); } catch (WebException ex) { if(ex.Response == null || ex.Status != WebExceptionStatus.ProtocolError) throw; return (HttpWebResponse)ex.Response; } }
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Net; using System.IO; namespace WebserverInteractionClassLibrary { public class RequestManager { public string LastResponse { protected set; get; } CookieContainer cookies = new CookieContainer(); internal string GetCookieValue(Uri SiteUri,string name) { Cookie cookie = cookies.GetCookies(SiteUri)[name]; return (cookie == null) ? null : cookie.Value; } public string GetResponseContent(HttpWebResponse response) … Read more
I want to allow any valid UTF-8 characters in usernames and passwords. Abandon all hope. Basic Authentication and Unicode don’t mix. There is no standard(*) for how to encode non-ASCII characters into a Basic Authentication username:password token before base64ing it. Consequently every browser does something different: Opera uses UTF-8; IE uses the system’s default codepage … Read more
Finally figured out the cause of this (both for JBoss and Apache). Both applications intentionally reject URIs with an encoded slash (%2F for / and %5C for \) to prevent possible security vulnerabilities. Links: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-0450 http://securitytracker.com/id/1018110 (Look at section 4. Solution) And here are the instructions they provide for enabling this behavior in JBoss: Note: … Read more