Skip to content

JupyterHub

JupyterLab is a web-based interactive development environment for jupyter notebooks, code and data.

Jupyter Notebook is a web-based interactive computational environment. It allows to create notebooks containing code, text, visualizations. It is language agnostic and is compatible with more than 40 programming languages depending on kernels installed.

JupyterHub is a cloud plateform that can run instances of Jupyterlab or Jupyter notebooks.

Access

Warning

For the moment the JupyterHub is only available inside IPSL network. If you want to access to it from ouside see how to do it on the connection page.

The JupyterHub will be avaible from outside before the end of 2022.

How to get access to JupyterHub

See How to get access to JupyterHub.

How to log in

Go to the JupyterHub

If you are outside the network of an IPSL laboratories you need to use a VPN or proxy (see FoxyProxy instructions). Once you are on JupyterLab authentification form, enter your mesocenter credentials.

Resources and cluster

Which server to choose ?

There are three servers available :

  • meso-X 1 core, 4 GB, 4 hours walltime, small pangeo
  • meso-X 1 core, 8 GB, 4 hours walltime, medium pangeo
  • meso-X 1 core, 16 GB, 4 hours walltime, big pangeo

By default, the small pangeo is selected. If you don't need a lot of resources please choose the small one.

If you need more ressources to run your Jupyter Notebook or JupterLab, 2 solutions are available: - use Dask Jobqueue (see Software::Python menu in the cluster entries). - submit your Jupyter Notebook or JupterLab as a cluster job using the method describe here.

How much memory does my server use ?

The current memory usage is displyed at bottom left of your dashboard.

current_memory_usage.png

Access to data

At the moment ESPRI JupyterHub is only backed on meso-X cluster. If you are a user of meso-SU cluster go to this page to know how to access your data.

Kernels

Python

The pangeo environment runs python 3.8.* and contains the following modules :

modules
pangeo-notebook intake-esm numcodecs sat-search xmitgcm
pip=20 intake-geopandas numpy sat-stac xpublish
adlfs intake-stac openpyxl satpy xrft
awscli intake-xarray pandas scikit-image zarr
boto3 ipympl panel scikit-learn batchspawner
bottleneck ipyleaflet parcels scipy
cartopy ipywidgets prefect seaborn
cfgrib jupyterlab-git pyarrow sparse
cmip6_preprocessing jupyterlab_execute_time pycamhd tiledb-py
ciso jupyter_bokeh pydap voila
gh jupyter-panel-proxy pyhdf xarray
geocube jupyter-resource-usage pystac xarrayutils
geopandas lz4 python-blosc xarray_leaflet
geoviews-core matplotlib-base python-gist xarray-spatial
gsw metpy python-graphviz xcape
h5netcdf nb_conda_kernels rasterio xcube
h5py nbstripout rechunker xesmf
holoviews nc-time-axis rio-cogeo xgcm
hvplot netcdf4 rioxarray xhistogram
intake nomkl s3fs>0.5 xlayers

How to install mesocenter modules and Python environments accessible from JupyterLab

Get the mesocenter modules and your own Python virtual environments or conda environements (Miniconda, Anaconda, etc.) in JupyterLab or Jupyter Notebook. Follow the steps below before launching your JupyterLab session:

  1. Activate the module or environment.
  2. Install the module or the environement into JupyterLab.
  3. A new kernel will be available at the next start of JupyterLab.

Example for the module pangeo-meso/2024.01.22:

module load pangeo-meso/2024.01.22
# check your are using the right python
which python
# install the kernel
python -m ipykernel install --user --name "pangeo-meso/2024.01.22" --display-name "pangeo-meso/2024.01.22"

Example for a conda environement (named myenv), provided the package ipykernel is already installed:

conda activate myenv
python -m ipykernel install --user --name=myenv

Example for a Python virtual environement (named myenv), provided the package ipykernel is already installed:

source "/path_to_my_env/myenv/bin/activate"
python -m ipykernel install --user --name=myenv

Example for an extended mesocenter Python module:

The module, named module_name, is extended by a Python virtual environement, named myenv. For more informations, read the module extension procedure.

module load module_name
source "/path_to_my_env/myenv/bin/activate"
python -m ipykernel install --user --name=myenv

How to make JupyterLab automatically discover your environments

The nb_conda_kernels module gives the JupyterLab instance the ability to automatically discover the conda environments available in user home. However, for an environment to be discovered, it needs to have an ipython kernel installed.

You need to run these commands inside the environment so that JupyterLab can discover it:

source activate <env>
conda install ipykernel "pyzmq<=20.0.0"

Known issue with mesocenter

There is a known incompatibility between new versions of the pyzmq module and the mesocenter linux kernel. This is why you need to install a specific version of pyzmq. Otherwise, you'll get a GLIBC error.

If your conda environnment is not stored in your HOME - JupyterLab won't be able to detect it. In this case use the method describe here

R environment

R/4.0.5 is available on JupyterLab. To launch R environment - select the following icon on your JupyterLab server.

r_icon.png

You can modify R environment by following instructions here.

JupyterLab extensions

Several JupyterLab extensions are installed. We do not allow user to install other JupyterLab.

List of extensions

jupyterlab/git jupyterlab-leaflet nbdime-jupyterlab dask-labextension jupyterlab-execute-time
jupyter-bokeh voila-dashboard jupyterlab-pyviz

jupyterlab-git

This extension allows you to use git with a gui but be aware that it is not possible to push or pull if your remote use ssh. Only https remote repositories are working but you will have to enter your password each time you do a fetch, pull or push.

jupyterlab-git.png

If you want to keep a ssh remote. You can do commit, branch with the jupyterlab-git extension and use a terminal for the fetch, pull and push commands.

How to logout properly

On the navigation top bar, go to "File" > "Hub control panel".

hub_control_panel.png

Then click on "Stop My Server" red button.


stop_server.png

You can then log out from the "Home" page. Click on "Logout" button on the top right corner of the screen.