Note on Python Versions
Working versions of python on the clusters
The current (2021/09) linux version of the ESPRI mesocenter (climserv and ciclad) servers is not compatible anymore with every version of python available with anaconda python distribution.
Recent version of anaconda and miniconda have the same problem and updating python to a more recent version of python can break your environment
By default when you create an environment, anaconda will download the latest version of python available and you will encounter a message of the type:
python: /lib64/libc.so.6: version `GLIBC_2.15' not found (required by python)
python: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by python)
python: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by python)
- https://repo.anaconda.com/archive/Anaconda3-2021.05-Linux-x86_64.sh
- https://repo.anaconda.com/miniconda/Miniconda3-py39_4.9.2-Linux-x86_64.sh
There is no fix for the installer himself
cannot be installed :
https://repo.anaconda.com/archive/Anaconda3-2021.11-Linux-x86_64.sh https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
Fixing GLIBC error
But It's now possible to fix a broken environment This fix is working for python himself and also for GLIBC error on import for module in python
Usage sample
ciclad-ng ~]$ conda activate broken
(broken)[ciclad-ng ~]$ python
Python 3.8.8 (default, Apr 13 2021, 19:58:26)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cartopy
Traceback (most recent call last):
File "/home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/__init__.py", line 22, in <module>
from . import multiarray
File "/home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/multiarray.py", line 12, in <module>
from . import overrides
File "/home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so)
>>>
(broken)[user@ciclad-ng ~]$ module load fix-my-python
Loading fix-my-python/0.2
(broken)[user@ciclad-ng ~]$ fix-my-python.sh
INFO: Searching for broken GLIBC binaries in your environment, please wait ...
INFO: BROKEN GLIBC binaries found (9) in /home/user/.conda/envs/broken, need to patch
INFO: Searching for broken GLIBC libs in your environment, please wait ...
INFO: BROKEN GLIBC python libs found (28) in /home/user/.conda/envs/broken, need to patch
------------------------------------------------------------------------------
trying to fix your python environment broken in
Ready to start(y/n)? y
------------------------------------------------------------------------------
INFO: doing tar Backup before modification, please wait ...
INFO: tar backup in /scratchu/user/broken.tgz
Trying to export your env in yaml file
INFO: yaml backup in /scratchu/user/broken.yaml
SUCCESS: patchelf for /home/user/.conda/envs/broken/bin/nghttpx complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/bin/pngfix complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/bin/png-fix-itxt complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/bin/projsync complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/bin/python complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/sbin/kadmind complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/sbin/kprop complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/sbin/kpropd complete
SUCCESS: patchelf for /home/user/.conda/envs/broken/sbin/krb5kdc complete
Trying now to fix dynamic library
SUCCESS: /home/user/.conda/envs/broken/lib/krb5/plugins/kdb/db2.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/krb5/plugins/preauth/otp.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libcrypto.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libcurl.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libgssapi_krb5.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkadm5clnt_mit.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkadm5clnt.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkadm5srv_mit.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkadm5srv.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkdb5.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libkrb5.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libpng16.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libpng.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libproj.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libssh2.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libtcl8.6.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libtinfo.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/libtinfow.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/_rational_tests.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/_simd.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/core/_umath_tests.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/fft/_pocketfft_internal.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/random/bit_generator.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/random/_bounded_integers.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/random/_common.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/random/_generator.cpython-38-x86_64-linux-gnu.so fixed
SUCCESS: /home/user/.conda/envs/broken/lib/python3.8/site-packages/numpy/random/mtrand.cpython-38-x86_64-linux-gnu.so fixed
-----------------------------------------
Your Python environment is fixed
if you modify your environment :
adding package or changing version of package
It's could be necessary to rerun this script
to fix the new libraries
you have a backup of libraries before patch
in /scratchu/user/broken.tgz
and a /scratchu/user/broken.yaml
this file could be use to reproduce your environment
------------------------------------------