How to increase the timeout in CasperJS
Use that to increase the timeout of every wait() functions : casper.options.waitTimeout = 20000; (20sec)
casperjs
console.log doesn’t work in CasperJS’ evaluate with setTimeout
Because you’re mixing up casperjs and remote page environments. The evaluate function will execute code within the remote page env, so the console.log call won’t output anything. If you want to catch remote console.log calls, listen to the remote.message event: casper.on(‘remote.message’, function(msg) { this.echo(‘remote message caught: ‘ + msg); }) Btw, documentation for events is … Read more
How do I access an iframe from CasperJS?
Spent forever looking for this, and of course I found the answer minutes after posting the question. I can use the new frame switching commands added to phantomjs in this commit. Specifically, the this.page.switchToChildFrame(0) and this.page.switchToParentFrame() functions. It appears undocumented, and it also seems that the methods have been changed for upcoming releases, but it … Read more
How to login into a website with CasperJS?
You will need to use Casper fill() function. Below is an example which login to Facebook and print out your name after login. Note that you need to put in your username and password: var casper = require(‘casper’).create({ verbose: true, logLevel: ‘debug’, pageSettings: { loadImages: false, // The WebPage instance used by Casper will loadPlugins: … Read more
Installing CasperJS on Windows: How to do it correctly?
Poor documentation for windows. http://casperjs.org/installation.html#windows It starts off assuming you have already installed without telling you how to install. So here it is if anyone else is confused about this. There is no actual install. It’s just extracting zip contents to the right place. download phantomjs for windows from the phantomjs site (it’s a zip … Read more
Write results into a file using CasperJS
var fs = require(‘fs’); fs.write(myfile, myData, ‘w’); for saving daily scrapes I do: var currentTime = new Date(); var month = currentTime.getMonth() + 1; var day = currentTime.getDate(); var year = currentTime.getFullYear(); var myfile = “data-“+year + “-” + month + “-” + day+”.html”;
Installing PhantomJS on Mac
If you are using Homebrew, you can type: brew tap homebrew/cask brew cask install phantomjs
Frontend testing: what and how to test, and what tool to use?
I had the same questions a few months ago and, after talking to many developers and doing a lot of research, this is what I found out. You should unit test your JavaScript, write a small set of UI integration tests and avoid record and playback testing tools. Let me explain that in more detail. … Read more
What Does ‘Then’ Really Mean in CasperJS
then() basically adds a new navigation step in a stack. A step is a javascript function which can do two different things: waiting for the previous step – if any – being executed waiting for a requested url and related page to load Let’s take a simple navigation scenario: var casper = require(‘casper’).create(); casper.start(); casper.then(function … Read more
Headless Browser and scraping – solutions [closed]
If Ruby is your thing, you may also try: https://github.com/chriskite/anemone (dev stopped) https://github.com/sparklemotion/mechanize https://github.com/postmodern/spidr https://github.com/stewartmckee/cobweb http://watirwebdriver.com/ (Selenium) also, Nokogiri gem can be used for scraping: http://nokogiri.org/ there is a dedicated book about how to utilise nokogiri for scraping by packt publishing