Create a Virtual Environment in Python with Conda

The present gist is a hybrid between a 'go-to' cheat sheet and a tutorial when starting a new Data Science Project.

Its purpose is to create a virtual environment for Python with the package manager Conda.

Table of contents

• Create a Virtual Environment in Python with Conda

  • System Settings

    • Microsoft Windows Operating System
    • Microsoft Visual Studio Code
    • Miniconda 3

  • Steps to creating a Conda virtual environment

    • Create a folder for the project

    • Create a virtual environment for the project

    • Create 'default' folders and files

    • Manage Conda

    • Manage Environments

      • Manage environments from the default environment folder
      • Manage environments from a specific location

    • Manage Python

    • Manage Packages

    • Use a package list to create a NEW (identical) environment

      • Use a package list from a requirements.txt file
      • Use a package list from an 'environment.yaml' file
      • Other options for managing Conda environments and packages

System Settings

Settings at the time of writing this gist (20^th of January 2021).

Microsoft Windows Operating System

xxxxxxxxxx
Edition: Windows 10 Home
Version: 1909
OS build: 18363.1256
System type: 64-bits operating, x64-based processor

Microsoft Visual Studio Code

xxxxxxxxxx
Version: 1.52.1 (user setup)
Date: 2020-12-16T16:34:46.910Z
Electron: 9.3.5
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: 8.3.110.13-electron.0
OS: Windows_NT x64 10.0.18363

Miniconda 3

xxxxxxxxxx
Conda version: 4.9.2
Python version: 3.8.5

NOTE: Python is automatically installed when installing Miniconda. This gist does not explain how to install Miniconda.

Steps to creating a Conda virtual environment

Create a folder for the project

There are 2 options:

1. Create a repository on GitHub.com BEFORE creating the new project folder on the local machine.

   Once the repository is created, clone it onto the local machine.

   NOTE: This procedure will not be covered here but it will be in a future Gist.

   OR

2. Create the new project folder locally (see below gist)

   • First, open a Terminal Prompt within VS Code.

   • Then, go to the folder where the new project is to be created, i.e. go to the 'working folder'.

     For example, if the main folder is called python_projects, go to

     xxxxxxxxxx
     C:\python_projects
     

     Hence, for the relative path, type:

     xxxxxxxxxx
     cd /python_projects
     

   • Create a folder for the project called project_name and check if it was created in the working folder

     xxxxxxxxxx
     mkdir project_name && dir
     

   • cd into this folder

     xxxxxxxxxx
     cd project_name
     

   • Open the project folder project_name from the menu in VS Code

Create a virtual environment for the project

• Create a .py python file into project_name and cd into it

  xxxxxxxxxx
  echo >> python_script.py
  

  Alternatively, create the .py file from VS Code menu.

• Open python_script.py file

  NOTE: it is important to create AND open a .py file BEFORE creating and activating the venv, as it helps with venv activation

• Choose the Python instance from the list of environments (bottom left). The Python version should be listed with conda in parentheses, e.g.

  xxxxxxxxxx
  Python 3.8.5 64-bit (conda)
  

  NOTE: the directory path to the project folder may be preceded by PS, which means the default shell is 'PowerShell'. However, Conda does not work with PowerShell in VS Code. Hence, the default shell MUST be changed to 'Command Prompt' (cmd).

• Then, open a new terminal It should now show (base) BEFORE the directory path, e.g.

  xxxxxxxxxx
  (base) /Users/user_name
  

  NOTE: (base) represents an 'alias' and is inside parentheses This is important later on.

Create 'default' folders and files

These will be needed for the project, e.g.:

xxxxxxxxxx
mkdir data, docs, tests, sources, scripts, figures

echo >> __init__.py
echo >> main.py
echo >> config.py
echo >> setup.py
echo >> requirements.txt

OPTION: if the project is to be hosted on GitHub, the following files can be created, or done automatically when creating a repository.

xxxxxxxxxx
echo >> README.md
echo >> LICENSE
echo >> .gitignore

NOTE: the 'LICENSE' and '.gitignore' files do NOT take a file extension. Only 'README.md' does.

Manage Conda

• Check Conda version and update

  xxxxxxxxxx
  conda info
  conda --version
  

• Update Conda from an environment named environment_name

  xxxxxxxxxx
  conda update --name environment_name conda
  

  IMPORTANT: like with Python base configuration, do NOT update the base environment !! This is what virtual environments are for.

• Update Conda from inside the 'active' environment

  xxxxxxxxxx
  conda update conda
  

Manage Environments

There are 2 ways:

1. Manage environments from the default environment folder
2. Manage environments from a specific location

Manage environments from the default environment folder

i.e. from /miniconda3/envs

• List all environments

  xxxxxxxxxx
  conda env list
  

  OR

  xxxxxxxxxx
  conda info --envs
  

  The output will look like this:

  xxxxxxxxxx
  base             *  C:\miniconda3
  

  NOTE: when running 'conda env list', the (current) active environment in the environment list will have an asterisk * in front of the directory path (see example below).

• Create a new Conda virtual environment named environment_name

  The command below will create a new folder called environment_name.

  xxxxxxxxxx
  conda create --name environment_name
  

  The output will look like this:

  xxxxxxxxxx
  base             *  /miniconda3
  environment_name    /miniconda3/envs/environment_name
  

  NOTE 1: by default, new Conda environments are created in the envs folder where Miniconda is installed, e.g.

  xxxxxxxxxx
  /miniconda3/envs/environment_name
  

  NOTE 2: there should now be (environment_name) BEFORE the directory path. This is an 'alias' as mentioned earlier, and it is inside parentheses e.g.

  xxxxxxxxxx
  (environment_name) /Users/user_name
  

  IMPORTANT: in fact, the newly created environment will not be very useful unless a Python version is installed (see Manage Python section below)

• Clone the environment environment_name and create a new one called clone_environment

  xxxxxxxxxx
  conda create --name clone_environment --clone environment_name
  

  The output will look like this:

  xxxxxxxxxx
  base             * /miniconda3
  environment_name   /miniconda3/envs/environment_name
  clone_environment  /miniconda3/envs/clone_environment
  

• Activate the environment named environment_name

  xxxxxxxxxx
  conda activate environment_name
  

  The output will look like this:

  xxxxxxxxxx
  base               /miniconda3
  environment_name * /miniconda3/envs/environment_name
  clone_environment  /miniconda3/envs/clone_environment
  

• Deactivate current environment

  xxxxxxxxxx
  conda deactivate
  

  This means that the deactivated environment will lose its * when running conda env list. Thus, the active environment will revert to base environment.

  The output will look like this:

  xxxxxxxxxx
  base             * /miniconda3
  environment_name   /miniconda3/envs/environment_name
  clone_environment  /miniconda3/envs/clone_environment
  

• Delete an environment (and ALL its packages) This can be done only if the environment is NOT active.

  xxxxxxxxxx
  conda deactivate
  conda remove --name environment_name --all
  

  The output will look like this:

  xxxxxxxxxx
  base               *  /miniconda3
  clone_environment     /miniconda3/envs/clone_environment
  

• Delete unused packages from writable package caches

  xxxxxxxxxx
  conda clean --packages
  

• After deleting an environment, clear up Conda's cache to delete left-overs

  xxxxxxxxxx
  conda clean --all
  

Manage environments from a specific location

i.e. not in Miniconda's envs folder (see previous section)

• First, cd into the project working directory, e.g.

  xxxxxxxxxx
  (base) /Users/user_name
  

  OR

  xxxxxxxxxx
  cd /python_projects
  

  Then, create a project folder named project_name and cd into it, e.g.

  xxxxxxxxxx
  mkdir project_name && cd project_name
  

• Create a virtual environment called environment_name. However, the virtual environment name can be anything (like banana). For the sake of simplicity and reproducibility, let's just call it conda_env. This is because it is common practice to use just venv when a virtual environment is created with Python's 'venv' module. This is handy when copying/pasting code snippets.

  Then, use the RELATIVE path with the --prefix flag to create conda_env. It will be INSIDE the newly created project folder named project_name.

  Thus, from the Terminal Prompt, run the following code (no need to cd into a specific folder):

  xxxxxxxxxx
  conda create --prefix ./conda_env
  conda activate ./conda_env
  

• Check the environment list.

  xxxxxxxxxx
  conda env list
  

  The output will look like this:

  xxxxxxxxxx
  base         C:\miniconda3  
          *    C:\python_projects\project_name_conda\conda_env
  

  IMPORTANT: before, the new environment_name was stored within the base configuration and it possessed an 'alias' in front of the directory path. This is not the case when using this way to creating Conda virtual environments. Thus, the ABSOLUTE path name is used as 'alias' and is inside parentheses BEFORE the ABSOLUTE path, e.g.

  xxxxxxxxxx
  (C:\python_projects\project_name\conda_env) C:\python_projects\project_name
  

  NOTE: it is possible to give an alias to the absolute directory path by modifying the .condarc file (check futur Gist on 'environment variable' for more).

• Delete this environment as shown earlier, i.e. run the following code:

  xxxxxxxxxx
  conda deactivate
  conda remove --prefix ./conda_env --all
  conda clean --packages
  conda clean --all
  

Manage Python

• List versions of Python available to install

  xxxxxxxxxx
  conda search python
  

• Check the Python version installed

  xxxxxxxxxx
  python --version
  

• Create new Conda environment named conda_env with Python installed

  xxxxxxxxxx
  conda create --prefix ./conda_env python
  

  NOTE: if not specified, the 'default' Python version installed. Here, it is version 3.8.5.

• Create new Conda environment named conda_env with a specific version of Python (e.g. version 3.9) and a package named package_name

  xxxxxxxxxx
  conda create --prefix ./conda_env python=3.9 package_name
  

  NOTE: see Manage Packages section below for more.

• Update Python from inside the active environment to the latest version of a 3.x branch (e.g. version 3.4)

  xxxxxxxxxx
  conda update python
  

• Upgrade Python to another 3.x (e.g. version 3.6) by installing that version of Python

  xxxxxxxxxx
  conda install python=3.6
  

Manage Packages

• Search for package in Anaconda's library/repository

  xxxxxxxxxx
  conda search package_name
  

• Install a package named package_name in the current environment

  xxxxxxxxxx
  conda install package_name
  

• Install several packages

  xxxxxxxxxx
  conda install package_name_1 package_name_2
  

• Install package from a specific source like a URL address

  xxxxxxxxxx
  conda install --channel url_address package_name
  

  IMPORTANT: it is advised to install all the packages at the same time to avoid dependency conflicts.

• Search and install a package not available in Anaconda's library. Look into Conda-Forge repository

  xxxxxxxxxx
  conda search --channel conda-forge package_name
  conda install --channel conda-forge package_name
  

• If the package is not in Conda-Forge either, try pip install (LAST RESORT!!)

  xxxxxxxxxx
  pip install package_name
  

• Remove a package named package_name from the current environment

  xxxxxxxxxx
  conda remove package_name
  

• List of all packages in the current environment

  xxxxxxxxxx
  conda list
  

• Save package list inside a requirements.txt file

  xxxxxxxxxx
  conda list > requirements.txt
  

  NOTE: this is equivalent to the pip list command when using Python 'venv' module. Thus, it not appropriate when creating a new environment using the requirements.txt file.

  The conda list > requirements.txt command lists the libraries installed manually (i.e. the packages installed via conda install) but also ALL the libraries installed with Conda. However, if the command pip list > requirements.txt is used then the default Conda libraries are not included.

  The following is equivalent to pip freeze:

  xxxxxxxxxx
  conda list --export > requirements.txt
  

  Save package URLs from Anaconda repository with the following command:

  xxxxxxxxxx
  conda list --explicit > requirements.txt
  

• Alternatively, save package list inside an environment.yaml file

  xxxxxxxxxx
  conda env export > environment.yaml
  

  NOTE: this is similar to a requirements.txt file with conda/pip list/freeze.

Use a package list to create a NEW (identical) environment

There are 2 ways:

1. Use a package list from a requirements.txt file
2. Use a package list from an environment.yaml file

Use a package list from a requirements.txt file

• Create a new environment called conda_env from the new project folder

  xxxxxxxxxx
  conda create --prefix ./conda_env --file requirements.txt
  conda activate ./conda_env
  

  IMPORTANT: the requirements.txt file must be INSIDE the project folder path (i.e. /project_name)

• Alternatively, if conda_env already exists, run the following code:

  xxxxxxxxxx
  conda install --file requirements.txt
  conda activate ./conda_env
  

  NOTE: also works with a requirement file created with the list --explicit flag.

Use a package list from an 'environment.yaml' file

• Create a new environment called conda_env from the new project folder

  xxxxxxxxxx
  conda env create --prefix ./conda_env --file environment.yaml
  conda activate ./conda_env
  

  IMPORTANT: the 'environment.yaml' file must be INSIDE the project folder path (i.e. /project_name)

  NOTE: if the purpose were to create a new environment inside Miniconda's default folder, one would run the code below. No need to be inside Miniconda's envs folder.

  xxxxxxxxxx
  conda env create --file environment.yaml --name conda_env
  

Other options for managing Conda environments and packages

• Track (package) changes in current environment over time

  xxxxxxxxxx
  conda list --revisions
  

• Roll back the current environment to a previous (package) version

  xxxxxxxxxx
  conda install --revision revision_number
  

  NOTE: e.g. conda install --revision 2. IMPORTANT: using revision '0' is not recommended.