Note on Python Versions

Working versions of python on the clusters

The current (2021/09) linux version of the ESPRI mesocenter (climserv and ciclad) servers is not compatible anymore with every version of python available with anaconda python distribution.

Recent version of anaconda and miniconda have the same problem and updating python to a more recent version of python can break your environment

By default when you create an environment, anaconda will download the latest version of python available and you will encounter a message of the type:

python: /lib64/libc.so.6: version `GLIBC_2.15' not found (required by python)
python: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by python)
python: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by python)

known to work on IPSL mesocenter :

There is no fix for the installer himself

cannot be installed :

https://repo.anaconda.com/archive/Anaconda3-2021.11-Linux-x86_64.sh https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

Fixing GLIBC error

But It's now possible to fix a broken environment This fix is working for python himself and also for GLIBC error on import for module in python

module load fix-my-python
fix-my-python.sh

Usage sample

ciclad-ng ~]$ conda activate broken
(broken)[ciclad-ng ~]$ python
Python 3.8.8 (default, Apr 13 2021, 19:58:26) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cartopy
Traceback (most recent call last):
  File "/home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/__init__.py", line 22, in <module>
    from . import multiarray
  File "/home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so)
>>>

Using tools to fix

(broken)[user@ciclad-ng ~]$ module load fix-my-python
Loading fix-my-python/0.2
(broken)[user@ciclad-ng ~]$ fix-my-python.sh 
INFO: Searching for broken GLIBC binaries in your environment, please wait ...
INFO: BROKEN GLIBC binaries found (9) in /home/user/.conda/envs/broken, need to patch
INFO: Searching for broken GLIBC libs in your environment, please wait ...
INFO: BROKEN GLIBC python libs found (28) in /home/user/.conda/envs/broken, need to patch
------------------------------------------------------------------------------
trying to fix your python environment broken in 
Ready to start(y/n)? y
------------------------------------------------------------------------------
INFO: doing tar Backup before modification, please wait ... 
INFO: tar backup in /scratchu/user/broken.tgz
Trying to export your env in yaml file
INFO: yaml backup in /scratchu/user/broken.yaml
SUCCESS: patchelf for /home/user/.conda/envs/broken/bin/nghttpx complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/bin/pngfix complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/bin/png-fix-itxt complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/bin/projsync complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/bin/python complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/sbin/kadmind complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/sbin/kprop complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/sbin/kpropd complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/sbin/krb5kdc complete
Trying now to fix dynamic library
SUCCESS: /home/user/.conda/envs/broken/lib/krb5/plugins/kdb/db2.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/krb5/plugins/preauth/otp.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libcrypto.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libcurl.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libgssapi_krb5.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkadm5clnt_mit.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkadm5clnt.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkadm5srv_mit.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkadm5srv.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkdb5.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkrb5.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libpng16.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libpng.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libproj.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libssh2.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libtcl8.6.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libtinfo.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libtinfow.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/_rational_tests.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/_simd.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/_umath_tests.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/fft/_pocketfft_internal.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/random/bit_generator.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/random/_bounded_integers.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/random/_common.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/random/_generator.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/random/mtrand.cpython-38-x86_64-linux-gnu.so fixed
-----------------------------------------
Your Python environment is fixed 
if you modify your environment :
adding package or changing version of package 
It's could be necessary to rerun this script
to fix the new libraries  
you have a backup of libraries before patch
in /scratchu/user/broken.tgz
and a /scratchu/user/broken.yaml
this file could be use to reproduce your environment 
------------------------------------------

Testing patched python

(broken)[user@ciclad-ng ~]$ python
Python 3.8.8 (default, Apr 13 2021, 19:58:26) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cartopy
>>>