alexkras.com

  • Home
  • Top Posts
  • Resume
  • Projects
  • YouTube
  • Boots to Bytes
  • About
  • Contact

How to use Virtualenv in Python to Install Packages Locally

August 20, 2018 by Alex Kras 3 Comments

There are two philosophies when it comes to package installation, global first and local first. Global meaning all applications that rely on a certain package have access to the same copy of the library that was installed once. Local means that each project has its own folder of dependencies installed specifically for this project and each library is installed many times. NPM, for example, uses local first strategy. Pip in Python uses global first strategy by default.

Both approaches have their benefits and drawbacks. In this post I wanted to show how to use a Python library called virtualenv to created isolated (local) python environments.

Installing virtualenv

As usual, it’s as simple as:

pip install virtualenv

Create a new virtual environment

Navigate to the folder where you want the project set up and run:

virtualenv any_name_you_want

Normally, I just name my python folder as venv so in my case I would run:

virtualenv venv

This will set up Python as well as pip that you can use locally.

Note that this command will install either Python 2 or Python 3 depending on your system set up. In my case it installs Python 3.

If I wanted to use Python 2, I can specify the path to Python version to use.

$ which python2
/usr/local/bin/python2 # <------- Use this

$ virtualenv -p /usr/local/bin/python2 venv

For the purpose of this tutorial, let’s stick with default Python 3 installation.

Activating Virtual Environment

At this point we are still running Python in usual mode and running pip install will install a package globally.

We need to “switch” into the virtual environment by running the activation command:

source venv/bin/activate

At this point your session will be updated to point to your local python and pip from your venv folder.

You can verify this by running:

(venv) $ which python
/someproject/venv/bin/python

(venv) $ which pip
/someproject/venv/bin/pip

which will show that both Python and Pip are running locally.

Using the Virtual Environment

This is really it. You can now do your work in the virtual environment and anything installed with pip install will show up in your local venv folder.

For example let’s install Pandas.

(venv) $ pip install pandas
# ... pip magic

(venv) $ ls /someproject/venv/lib/python3.6/site-packages/ | grep pandas
pandas
pandas-0.23.4.dist-info

Deactivating the Virtual Environment

When you are done with local environment, simply run deactivate in your shell to go back to the regular Python flow.

(venv) $ which python
/someproject/venv/bin/python

(venv) $ deactivate

$ which python
/usr/local/bin/python

Putting it all together

I have a repo where I show how to use Google Cloud API to convert speech to text. Let’s go ahead and clone it.

$ git clone git@github.com:akras14/speech-to-text.git
$ cd speech-to-text/
$ ls
.                .gitignore       requirements.txt transcripts
..               README.md        slow.py          venv
.git             fast.py          source

$ cat .gitignore
api-key.json
parts/
venv

As you can see above, in my .gitignore file I have an entry for my venv folder, because I don’t want to check-in my local dependencies. Since there is no venv folder in this cloned repo, so let’s create it and source it.

$ virtualenv venv
$ source venv/bin/activate

At this point I am running local python and pip. Let me go ahead and install all dependencies from requirements.txt file.

$ pip install -r requirements.txt

After install is done I will see all needed packages installed in my /speech-to-text/venv/lib/python3.6/site-packages/ folder.


(venv) $ cat requirements.txt google-api-python-client==1.6.4 httplib2==0.10.3 oauth2client==4.1.2 pyasn1==0.4.2 pyasn1-modules==0.2.1 rsa==3.4.2 six==1.11.0 SpeechRecognition==3.8.1 tqdm==4.19.5 uritemplate==3.0.0 (venv) $ ls venv/lib/python3.6/site-packages/ SpeechRecognition-3.8.1.dist-info oauth2client pyasn1_modules-0.2.1.dist-info tqdm __pycache__ oauth2client-4.1.2.dist-info rsa tqdm-4.19.5.dist-info apiclient pip rsa-3.4.2.dist-info uritemplate easy_install.py pip-18.0.dist-info setuptools uritemplate-3.0.0.dist-info google_api_python_client-1.6.4.dist-info pkg_resources setuptools-40.1.0.dist-info wheel googleapiclient pyasn1 six-1.11.0.dist-info wheel-0.31.1.dist-info httplib2 pyasn1-0.4.2.dist-info six.py httplib2-0.10.3.dist-info pyasn1_modules speech_recognition

As a side note, requirements.txt is a pip convention for storing dependencies. Kind of like package.json file in NPM.

pip has a command called freeze which will list all dependencies for the project.

$ pip freeze
google-api-python-client==1.6.4
httplib2==0.10.3
oauth2client==4.1.2
pyasn1==0.4.2
pyasn1-modules==0.2.1
rsa==3.4.2
six==1.11.0
SpeechRecognition==3.8.1
tqdm==4.19.5
uritemplate==3.0.0

To record project dependancies we just have to redirect result of pip freeze into a txt file.

pip freeze > requirements.txt

Checking this file in, will make sure that the next person using this project will have the same dependencies as we did.

In any case, now that our dependancies are installed, we are ready to run the repo. Once we are done, we can de-activate it or simply exit this shell session.

deactivate

That’s really it. In true spirit of Python – using virtualenv is very simple and effective.

Filed Under: Python, Tools

I work for Evernote and we are hiring!

Subscribe to this Blog via Email

New posts only. No other messages will be sent.

You can find me on LinkedIn, GitHub, Twitter or Facebook.

This blog is moving to techtldr.com

Comments

    Leave a Reply Cancel reply

  1. Mohammed Saleh says

    December 15, 2019 at 3:04 am

    Thanks a lot i watched video about how to do this, but it was years ago and i forgot how to do it, and since i was looking for any Tutorial about how to do it. You did help me.

    Reply
  2. Aldo says

    August 22, 2018 at 11:44 pm

    Thanks for this article. Small typo fix: “folder of depEndancies” instead of “depndancies”

    Reply
    • Alex Kras says

      August 23, 2018 at 10:16 am

      Thank you! It’s fixed.

      Reply

Copyright © 2021 · eleven40 Pro Theme on Genesis Framework · WordPress · Log in