Skip to content

Python environment replication

Python environment refers to conda environment and Python virtual environment. As far as we know, we can see 3 different cases:

  • Cloning a conda environment into the same miniconda distribution.
  • Replication of a conda environment.
  • Replication of a Python virtual environment.

As the replication of pure conda environment is easy and fast, we recommand to create conda environment and install your packages exclusively with the command conda install. If some of your packages have to be installed by pip, we recommend to install packages from pyinfo and not from repositories as the automatic replication procedure, that is described below, fails to reproduce the installation of such packages.

Tips

Run conda env export within an activated conda environment. If your conda environement has any pip installed packages, they will appear in a specific entry named pip in the dependencies list (near the end of the output of the command).

Tips

Conda keeps a log for every instruction that modifies a conda environement. The log can be found at this path: /<conda_home>/envs/<environement_name>/conda-meta/history

Cloning conda environments

The following instructions assume that you want to replicate a conda environment just in the same Anaconda or Miniconda distribution. Let the environment to be replicated be called source and the created environment be called destination.

conda create -n destination --clone source

Replication of conda environements

The following instructions assume that you have a miniconda distribution already installed on the computer where you want to replicate the conda environment (e.g. command conda is accessible). Let the environment to be replicated be called source and the created environment be called destination.

Pure conda environments

This method describes how to replicate conda environment where packages were installed exclusively by conda (not by pip). It is quite fast but it is based on the URL address of the packages. If one of the URL is broken, this method won't succeed to reproduce the environment perfectly.

  • Get the specifications of the source environment:
conda activate source
conda list --explicit > source_conda_explicit_list.cfg
  • Replicate the environment:
conda create -n destination --file source_conda_explicit_list.cfg

Conda environment eventually with pip installed packages

This method replicates conda environment made of mix of conda and pip installed packages. It is rather slow as it is based on package specifications (version and hash) which dependencies are resolved by conda.

  • Get the specifications of the source environment:
conda activate source
conda env export > source_conda_env_export.yml
  • Replicate the environment:
conda env create -n destination --file source_conda_env_export.yml

Warning

Some package are named differently in conda and pip so the replication can fail for these packages. The workaround is to get the name of the package for pip, searching in this page, then open source_conda_env_export.yml and change it (pip package specifications are located in the section dependencies, item pip, near the end of the file). Package python-graphiz is known to fail the replication. Rename it into graphiz, e.g. 'python-graphviz==0.19.1' into 'graphviz==0.19.1'.

Warning

The following procedure fails when pip packages were installed from repositories (Github, etc.).

Info

The replication based on the output of the command pip freeze cannot help you as pip almost ignores the packages installed by conda.

Python virtual environments

Let the environment to be replicated be called source and the created environment be called destination.

Getting the specifications

source "/path/to/the/env/directory/bin/activate"
pip freeze > source_pip_freeze.cfg

Replication

The following instructions assume that you have Python distribution already installed on the computer where you want to replicate the environment (you may load a Python module).

mkdir -p "${HOME}/virtual_envs" # Create the environment parent dir, if it is not already done.
python -m destination "${HOME}/virtual_envs/destination" # Create the virtual environment named destination.
source "${HOME}/virtual_envs/destination/bin/activate" # Activate the virtual environment.
pip install -U pip # Update pip.
pip install -r source_pip_freeze.cfg