JupyterHub
JupyterLab is a web-based interactive development environment for jupyter notebooks, code and data.
Jupyter Notebook is a web-based interactive computational environment. It allows to create notebooks containing code, text, visualizations. It is language agnostic and is compatible with more than 40 programming languages depending on kernels installed.
JupyterHub is a cloud plateform that can run instances of Jupyterlab or Jupyter notebooks.
Access
Warning
For the moment the JupyterHub is only available inside IPSL network. If you want to access to it from ouside see how to do it on the connection page.
The JupyterHub will be avaible from outside before the end of 2022.
How to get access to JupyterHub
See How to get access to JupyterHub.
How to log in
Go to the JupyterHub
If you are outside the network of an IPSL laboratories you need to use a VPN or proxy (see FoxyProxy instructions). Once you are on JupyterLab authentification form, enter your mesocenter credentials.
Resources and cluster
Which server to choose ?
There are three servers available :
- meso-X 1 core, 4 GB, 4 hours walltime,
small pangeo
- meso-X 1 core, 8 GB, 4 hours walltime,
medium pangeo
- meso-X 1 core, 16 GB, 4 hours walltime,
big pangeo
By default, the small pangeo
is selected. If you don't need a lot of resources please choose the small one.
If you need more ressources to run your Jupyter Notebook or JupterLab, 2 solutions are available: - use Dask Jobqueue (see Software::Python menu in the cluster entries). - submit your Jupyter Notebook or JupterLab as a cluster job using the method describe here.
How much memory does my server use ?
The current memory usage is displyed at bottom left of your dashboard.
Access to data
At the moment ESPRI JupyterHub is only backed on meso-X cluster. If you are a user of meso-SU cluster go to this page to know how to access your data.
Kernels
Python
The pangeo environment runs python 3.8.*
and contains the following modules :
modules | ||||
---|---|---|---|---|
pangeo-notebook | intake-esm | numcodecs | sat-search | xmitgcm |
pip=20 | intake-geopandas | numpy | sat-stac | xpublish |
adlfs | intake-stac | openpyxl | satpy | xrft |
awscli | intake-xarray | pandas | scikit-image | zarr |
boto3 | ipympl | panel | scikit-learn | batchspawner |
bottleneck | ipyleaflet | parcels | scipy | |
cartopy | ipywidgets | prefect | seaborn | |
cfgrib | jupyterlab-git | pyarrow | sparse | |
cmip6_preprocessing | jupyterlab_execute_time | pycamhd | tiledb-py | |
ciso | jupyter_bokeh | pydap | voila | |
gh | jupyter-panel-proxy | pyhdf | xarray | |
geocube | jupyter-resource-usage | pystac | xarrayutils | |
geopandas | lz4 | python-blosc | xarray_leaflet | |
geoviews-core | matplotlib-base | python-gist | xarray-spatial | |
gsw | metpy | python-graphviz | xcape | |
h5netcdf | nb_conda_kernels | rasterio | xcube | |
h5py | nbstripout | rechunker | xesmf | |
holoviews | nc-time-axis | rio-cogeo | xgcm | |
hvplot | netcdf4 | rioxarray | xhistogram | |
intake | nomkl | s3fs>0.5 | xlayers |
How to install mesocenter modules and Python environments accessible from JupyterLab
Get the mesocenter modules and your own Python virtual environments or conda environements (Miniconda, Anaconda, etc.) in JupyterLab or Jupyter Notebook. Follow the steps below before launching your JupyterLab session:
- Activate the module or environment.
- Install the module or the environement into JupyterLab.
- A new kernel will be available at the next start of JupyterLab.
Example for the module pangeo-meso/2024.01.22
:
module load pangeo-meso/2024.01.22
# check your are using the right python
which python
# install the kernel
python -m ipykernel install --user --name "pangeo-meso/2024.01.22" --display-name "pangeo-meso/2024.01.22"
Example for a conda environement (named myenv), provided the package ipykernel is already installed:
Example for a Python virtual environement (named myenv), provided the package ipykernel is already installed:
Example for an extended mesocenter Python module:
The module, named module_name, is extended by a Python virtual environement, named myenv. For more informations, read the module extension procedure.
module load module_name
source "/path_to_my_env/myenv/bin/activate"
python -m ipykernel install --user --name=myenv
How to make JupyterLab automatically discover your environments
The nb_conda_kernels
module gives the JupyterLab instance the ability to automatically discover the conda environments available in user home.
However, for an environment to be discovered, it needs to have an ipython
kernel installed.
You need to run these commands inside the environment so that JupyterLab can discover it:
Known issue with mesocenter
There is a known incompatibility between new versions of the pyzmq
module and the mesocenter linux kernel. This is why you need to install a specific version of pyzmq
. Otherwise, you'll get a GLIBC
error.
If your conda environnment is not stored in your HOME
- JupyterLab won't be able to detect it. In this case use the method describe here
R environment
R/4.0.5
is available on JupyterLab. To launch R environment - select the following icon on your JupyterLab server.
You can modify R environment by following instructions here.
JupyterLab extensions
Several JupyterLab extensions are installed. We do not allow user to install other JupyterLab.
List of extensions
jupyterlab/git | jupyterlab-leaflet | nbdime-jupyterlab | dask-labextension | jupyterlab-execute-time |
jupyter-bokeh | voila-dashboard | jupyterlab-pyviz |
jupyterlab-git
This extension allows you to use git with a gui but be aware that it is not possible to push or pull if your remote use ssh. Only https remote repositories are working but you will have to enter your password each time you do a fetch, pull or push.
If you want to keep a ssh remote. You can do commit, branch with the jupyterlab-git extension and use a terminal for the fetch, pull and push commands.
How to logout properly
On the navigation top bar, go to "File" > "Hub control panel".
Then click on "Stop My Server" red button.
You can then log out from the "Home" page. Click on "Logout" button on the top right corner of the screen.
Useful links
- Project jupyter
- Jupyter Documentation
- JupyterLab Documentation
- Jupyter Notebook Documentation
- Detailled Tutorial about jupyterlab functionalities