Another approach is to just ask PhantomJS to wait for a bit after the page has loaded before doing the render, as per the regular rasterize.js example, but with a longer timeout to allow the JavaScript to finish loading additional resources: page.open(address, function (status) { if (status !== ‘success’) { console.log(‘Unable to load the address!’); … Read more
Before starting, please make sure you understand what google requires, particularly the use of pretty and ugly URLs. Now lets see the implementation: Client Side On the client side you only have a single html page which interacts with the server dynamically via AJAX calls. that’s what SPA is about. All the a tags in … Read more
They are attacking different problems. Since PhantomJS runs perfectly on the command-line, it is suitable as the first layer of smoke testing, whether as part of development workflow and/or in a continuous integration server. Selenium targets multiple browsers and hence it is very useful to ensure cross-browser consistency and carry out extensive testings across different … Read more
I figured it out. Basically it’s an async issue. You can’t just submit and expect to render the subsequent page immediately. You have to wait until the onLoad event for the next page is triggered. My code is below: var page = new WebPage(), testindex = 0, loadInProgress = false; page.onConsoleMessage = function(msg) { console.log(msg); … Read more
Guidouil’s answer put me on the right track. I had to add one additional symlink to /usr/bin/, and I did direct symlinks for all 3 – see below. I’m installing on Ubuntu server Natty Narwhal. This is exactly what I did. cd /usr/local/share sudo wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-1.9.7-linux-x86_64.tar.bz2 sudo tar xjf phantomjs-1.9.7-linux-x86_64.tar.bz2 sudo ln -s /usr/local/share/phantomjs-1.9.7-linux-x86_64/bin/phantomjs /usr/local/share/phantomjs … Read more
The easiest way to use PhantomJS in python is via Selenium. The simplest installation method is Install NodeJS Using Node’s package manager install phantomjs: npm -g install phantomjs-prebuilt install selenium (in your virtualenv, if you are using that) After installation, you may use phantom as simple as: from selenium import webdriver driver = webdriver.PhantomJS() # … Read more
If Ruby is your thing, you may also try: https://github.com/chriskite/anemone (dev stopped) https://github.com/sparklemotion/mechanize https://github.com/postmodern/spidr https://github.com/stewartmckee/cobweb http://watirwebdriver.com/ (Selenium) also, Nokogiri gem can be used for scraping: http://nokogiri.org/ there is a dedicated book about how to utilise nokogiri for scraping by packt publishing