I just saw an articled on Hacker News announcing that Chrome 59 is going to have cross-platform headless support.
A lot of people are excited by this announcement, mainly because it will allow them to move away from PhantomJS – a buggy headless browser currently used as a de-facto solution to run unit tests and other automated tasks.
I too have had my struggles with PhantomJS. One of my first tasks at my current job was resurrecting our long abandoned test framework. I spent some time trying to get everything working with PhanotmJS, but ran into a lot of issues and was forced to look for an alternative solution.
Reading through comments at Hacker News I didn’t see anyone mentioned the approach that we ended up taking (update, some popped up after I finished writing this article). In addition, some people mentioned that the current headless Chrome is not quite where it needs to be. I decided to share our trick since it has served us well for the past 9 months AND it can work with other browsers.
TLDR; We are using xvfb as a virtual display server to run a real Chrome browser in effectively HEADLESS WAY.
Simple test on my Mac
At work we use Protractor as our test framework with Chrome and this approach will work with anything that runs in any browser supported by Unix/Linux system.
For the purpose of this exercise I wanted to write something simple, so I decided to use Python binding for Selenium. I will also use Firefox to demonstrate that this approach works with other browsers.
First I had to run the test on my Mac. I already had Firefox and Python installed, but I had to install the selenium binding
pip install selenium and
brew install geckodriver to allow selenium binding to interact with Firefox.
Then I ran my simple test script.
from selenium import webdriver import time driver = webdriver.Firefox() # Connect to Firefox driver.get("https://github.com") # Navigate to Github print driver.title # Print page title time.sleep(3) # Wait 3 seconds driver.quit() # Close the browser
As expected, the script opened Firefox, printed out the GitHub’s page title and closed the browser. That is how I got the fancy screenshot at the top of this page 🙂
Simulating the build server
I’ve used vagrant to start an Ubuntu 14.04 in a Virtual machine from within same directory, so I can easily have access to my test script. Among other things, Vagrant synchronized current folder with the Virtual machine under
/vagrant path, as we’ll see shortly.
vagrant init ubuntu/trusty64 vagrant up vagrant ssh
After ssh’ing into the virtual machine, I did the following:
sudo apt-get update # Update dependencies sudo apt-get install python-pip # Install pip (Ubuntu already had python 2) sudo pip install selenium
geckodriver was a bit trickier:
wget https://github.com/mozilla/geckodriver/releases/download/v0.15.0/geckodriver-v0.15.0-linux64.tar.gz tar -xvzf geckodriver-v0.15.0-linux64.tar.gz chmod +x geckodriver sudo cp geckodriver /usr/local/bin/
At this point I’ve had all dependencies installed like I did on my Mac, except that my test was on a server, without X Window System running. As expected, the test hang for a while and failed with the following error:
$ cd /vagrant $ python test.py Traceback (most recent call last): File "test.py", line 4, in <module> driver = webdriver.Firefox() File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 154, in __init__ keep_alive=True) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 98, in __init__ self.start_session(desired_capabilities, browser_profile) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 185, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 249, in execute self.error_handler.check_response(response) File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 193, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: connection refused
Running it Headless
Fixing the error was as easy as:
sudo apt-get install xvfb xvfb-run --server-args='-screen 0 1280x1024x24' python test.py # -screen 0 1280x1024x24 means -- configured with a root window # of 1024 by 768 pixels and a color depth of 24 bits # and taken right out of the man file :)
Which printed out a nice message:
The world's leading software development platform · GitHub
So there you have it a (somewhat) headless Firefox running on a server.
Like everyone else, I am very excited about the Headless Chrome announcement. At the same time I wanted to share the
xvfb-run technique because:
- It provides a good alternative to PhantomJS
- It works with other browsers, not just Chrome
- It allows to run identical browsers used by real users, instead of modified (headless) versions.
A day after headless Chrome announcement, PhantomJS maintainer announced that he will be stepping down from the project. I felt bad for bashing PhantomJS. It is a great project that I’ve use for majority of my development career.
— Alex Kras (@akras14) April 13, 2017