Alex Kras

Archives for April 2017

OMG – NPM clone that finally makes sense

by Leave a Comment

Note: Everything I’ve assumed was wrong, I am keeping track of things as I discover them via the medium post: https://medium.com/@akras14/omg-npm-clone-that-finally-makes-sense-3478588879

I’ll update this post when the dust settles 🙂

Last week I came across a yet another NPM clone, called pnpm, but unlike other clones (cough Yarn) this one is actually interesting!

What so interesting about it? One word – SYMLINKS.

In few more words, instead of having to re-download libraries over and over and over (and over and over, I wonder if I made my point…) again every time you run npm install in a new folder, pnpm will download that version of the library once in some global context and will create a symlink to this library when you run the pnpm install command.

So your node_modules folder will look something like this:

This way you get the benefits of only having to download the library once AND being able to freely pick what libraries/versions to use on per project basis.

Now that is really cool!

If this sounds interesting to you, please check out this post by the author explaining his take on how pnpm compare to yarn and npm.

Running Chrome and Other Browsers in (almost) Headless Mode

by Leave a Comment

I just saw an articled on Hacker News announcing that Chrome 59 is going to have cross-platform headless support.

A lot of people are excited by this announcement, mainly because it will allow them to move away from PhantomJS – a buggy headless browser currently used as a de-facto solution to run unit tests and other automated tasks.

I too have had my struggles with PhantomJS. One of my first tasks at my current job was resurrecting our long abandoned test framework. I spent some time trying to get everything working with PhanotmJS, but ran into a lot of issues and was forced to look for an alternative solution.

Reading through comments at Hacker News I didn’t see anyone mentioned the approach that we ended up taking (update, some popped up after I finished writing this article). In addition, some people mentioned that the current headless Chrome is not quite where it needs to be. I decided to share our trick since it has served us well for the past 9 months AND it can work with other browsers.

TLDR; We are using xvfb as a virtual display server to run a real Chrome browser in effectively HEADLESS WAY.

Simple test on my Mac

At work we use Protractor as our test framework with Chrome and this approach will work with anything that runs in any browser supported by Unix/Linux system.

For the purpose of this exercise I wanted to write something simple, so I decided to use Python binding for Selenium. I will also use Firefox to demonstrate that this approach works with other browsers.

First I had to run the test on my Mac. I already had Firefox and Python installed, but I had to install the selenium binding pip install selenium and brew install geckodriver to allow selenium binding to interact with Firefox.

Then I ran my simple test script.

1
2
3
4
5
6
7
8
9
from selenium import webdriver
import time
 
driver = webdriver.Firefox()     # Connect to Firefox
driver.get("https://github.com") # Navigate to Github
print driver.title               # Print page title
time.sleep(3)                    # Wait 3 seconds
driver.quit()                    # Close the browser
 

As expected, the script opened Firefox, printed out the GitHub’s page title and closed the browser. That is how I got the fancy screenshot at the top of this page 🙂

Simulating the build server

I’ve used vagrant to start an Ubuntu 14.04 in a Virtual machine from within same directory, so I can easily have access to my test script. Among other things, Vagrant synchronized current folder with the Virtual machine under /vagrant path, as we’ll see shortly.

1
2
3
4
vagrant init ubuntu/trusty64
vagrant up
vagrant ssh
 

After ssh’ing into the virtual machine, I did the following:

1
2
3
4
sudo apt-get update                # Update dependencies
sudo apt-get install python-pip    # Install pip (Ubuntu already had python 2)
sudo pip install selenium
 

Installing geckodriver was a bit trickier:

1
2
3
4
5
wget https://github.com/mozilla/geckodriver/releases/download/v0.15.0/geckodriver-v0.15.0-linux64.tar.gz
tar -xvzf geckodriver-v0.15.0-linux64.tar.gz
chmod +x geckodriver
sudo cp geckodriver /usr/local/bin/
 

At this point I’ve had all dependencies installed like I did on my Mac, except that my test was on a server, without X Window System running. As expected, the test hang for a while and failed with the following error:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ cd /vagrant
$ python test.py
Traceback (most recent call last):
  File "test.py", line 4, in <module>
    driver = webdriver.Firefox()
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 154, in __init__
    keep_alive=True)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 98, in __init__
    self.start_session(desired_capabilities, browser_profile)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 185, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 249, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 193, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: connection refused
 

Running it Headless

Fixing the error was as easy as:

1
2
3
4
5
6
7
sudo apt-get install xvfb
xvfb-run --server-args='-screen 0 1280x1024x24' python test.py
 
# -screen 0 1280x1024x24 means -- configured with a root window
# of 1024 by 768 pixels and a color depth of 24 bits
# and taken right out of the man file :)
 

Which printed out a nice message:

1
The world's leading software development platform · GitHub

So there you have it a (somewhat) headless Firefox running on a server.

Conclusion

Like everyone else, I am very excited about the Headless Chrome announcement. At the same time I wanted to share the xvfb-run technique because:

  1. It provides a good alternative to PhantomJS
  2. It works with other browsers, not just Chrome
  3. It allows to run identical browsers used by real users, instead of modified (headless) versions.
I hope this post will save somebody from the horrors of PhantomJS.

A day after headless Chrome announcement, PhantomJS maintainer announced that he will be stepping down from the project. I felt bad for bashing PhantomJS. It is a great project that I’ve use for majority of my development career.

Convert video to images via command line

by Leave a Comment

I often find myself needing to convert video to a series of images. It’s a great way to look through the content to find some details or identify screenshots that are worth keeping. For example I used this method to look through all talks given at Chrome Dev Summit to supplement my notes. Another use case was converting an HD footage of my son’s birthday to a series of images to print.

The command line tool I use is called ffmpeg. You can download it form the official site, but it’s also available via Homeberew on Mac with brew install ffmpeg command (that is how I have it installed).

Once you have ffmpeg on your system, you can convert your video to images via the following:

1
2
3
4
5
6
# cd to folder with video file
 
mkdir photos # Photos will be stored here
 
ffmpeg -i your-video-file.mp4 -r 1 photos/output_%05d.png
 

-i your-video-file.mp4 points to the file that you are trying to convert.

-r 1 parameter means frame rate, i.e. how many frames per second. In case of r 1 it will produce 1 photo per second. If more picture is needed, -r 10 which will produce 10 photos per second, if less r 0.1 will produce 1 photo for every 10 seconds etc.

photos/output_%05d.png specified that files should be saved into photos directory, and named in the format output_00001.png, output_00002.png etc.

After the conversion is done, I usually open photo files in Preview App (on Mac), where I go through the images one by one, deleting what I don’t need. This leaves me with the needed collection of screenshots.

Enjoy 🙂

Save Ammonite (Scala REPL) History to File

by Leave a Comment

I use Ammonite and ipython like REPL for Scala.

I find it handy to save history to file, so here is how to do it in Ammonite.

1
2
3
4
5
6
7
8
import java.io.BufferedWriter
import java.io.FileWriter
val file = new File("some-file.scala")
val bw = new BufferedWriter(new FileWriter(file))
val history = repl.history.mkString("\n")
bw.write(history)
bw.close()
 

Ammonite exposes a global object called repl, which has a method called history. The code above converts history to a new line separated string and writes it to file.

Enjoy 🙂

Write or Paste Multi-Line Code in Scala REPL

by Leave a Comment

I love Python and ipython, so I often crave similar experience when working with Scala.

Scala already has a built in REPL which is a huge plus, however, I wasn’t sure how to copy/paste multi-line text.

For example, I had a snippet of code saved somewhere that looked as such:

1
2
3
someList
   .map(someFunction)
 

This would work fine if entered like this:

1
2
someList.map(someFunction)
 

But Scala would evaluate the someList first, if input was entered across multiple lines.

Turns out, Scala REPL has a :paste command to support a paste mode. So you can just do:

1
2
3
4
5
6
7
:paste // Hit enter
 
someList
   .map(someFunction)
 
//Hit ctr+D to execute block of code
 

I often use Ammonite, an ipython like took for Scala REPL. It suffers from the same problem, but does not support the :paste command.

Similar behavior, however, can be achieved by simple wrapping your code in brackets, as such:

1
2
3
4
5
{
someList
   .map()
}
 

Wrapping the code in {}, unfortunately will make tab completion useless. So while this works great for copy/pasting code, it does not work when I am trying to prototype something, and would like to spread it out across multiple lines.

If you know how to write . chained functions across multiple line AND keep the tab completion working, or any other cool Scala REPL tricks, please let me know in the comment area bellow.

64% of Developers are still using Angular 1 vs Angular 2+

by Leave a Comment

Two years ago I wrote a blog post on Angular Performance, that remains the most popular post on this site. Even two years ago it seemed like Angular 1 was on it’s way out, so I was surprise to see my Angular 1 post still getting so much traffic.

I conducted a poll on Twitter, to see if people still were using Angular 1.

Looks like 2/3 of developers (that are using Angular), are still using Angular 1. My co-worker pointed out that this result makes sense, considering how many legacy apps were built with Angular 1.

This poll did not reflect a total number of developers who are using Angular vs people who moved on to other frameworks instead of upgrading to Angular 2+, but it would be interesting to know.

What about you? Are you using (or have you used) Angular? Are you still using Angular 1, upgraded to Angular 2 or moved to a different framework? Please share in the comment area bellow.

Update: Some people expressed concerns over the small sample size.

Using http://www.surveysystem.com/sscalc.htm with population of let say 100,000K Front End developers(or more) and sample size of 44, 64% gives us confidence interval of 14.18.

So:

  • 64 – 14.18 = 49.82%
  • 36 + 14.18 = 50.18%

It’s also good too remember, that it could go the other way 🙂 , like such:

  • 64 + 14.18 = 78.18%
  • 36 – 14.18 = 21.82%

In other words, even assuming the worst error possible, I am 95% confident that at least half of Front End developers are still using Angular 1 vs Angular 2+, based purely on my poll data. Even though those results are not surprising, I prefer to look at real data (when available) instead of trusting my hunches.