Python3, CentOS and pip

Following on from last weeks post, this post is going to discuss using Python 3 virtual environments under CentOS 7. More specifically, how to work around bug #1263057 until it's fixed.

How it should work

Python 3.4 is packaged for CentOS 7 as part of the EPEL project. Getting it installed is just a case of running a couple of yum commands:

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum install python34

Once you've got the python34 package installed you should be able to run Python 3:

[bob@localhost ~]$ python3
Python 3.4.5 (default, Nov  9 2016, 16:24:59)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

At this point we should be able to use the venv module to create a new environment, with a command similar to the following:

python3 -m venv new-env

Unfortunately venv doesn't quite work as expected:

[bob@localhost ~]$ python3 -m venv new-env
Error: Command '['/home/bob/new-env/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

Note: it is possible to work around this problem by disabling pip in the environment:

python3 -m venv --without-pip new-env

This will produce a working environment, all be it without pip:

(no-pip) [bob@localhost ~]$ pip
-bash: pip: command not found

Missing modules

When venv runs it uses the ensurepip module to bootstrap pip into the new virtual environment. Unfortunately the python34-libs package includes ensurepip, but doesn't include the wheel files for pip and setuptools.

Calling ensurepip directly makes it fairly obvious which version of the wheel file is missing:

[bob@localhost ~]$ python3 -Im ensurepip --upgrade --default-pip
Traceback (most recent call last):
  File "/usr/lib64/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib64/python3.4/ensurepip/__main__.py", line 4, in <module>
    ensurepip._main()
  File "/usr/lib64/python3.4/ensurepip/__init__.py", line 209, in _main
    default_pip=args.default_pip,
  File "/usr/lib64/python3.4/ensurepip/__init__.py", line 98, in bootstrap
    "_bundled/{}".format(wheel_name),
  File "/usr/lib64/python3.4/pkgutil.py", line 629, in get_data
    return loader.get_data(resource_name)
  File "<frozen importlib._bootstrap>", line 1623, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/usr/lib64/python3.4/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl'

Restoring pip and setuptools

The easiest way to restore the missing wheel files, is to download them from PyPI:

mkdir -p /usr/lib64/python3.4/ensurepip/_bundled
curl -o /usr/lib64/python3.4/ensurepip/_bundled/setuptools-20.10.1-py2.py3-none-any.whl \
  https://pypi.python.org/packages/c5/e2/72d706eeda837564b9fecdc8b2bf48b33467ae928ed05d4ac157463c90fb/setuptools-20.10.1-py2.py3-none-any.whl
curl -o /usr/lib64/python3.4/ensurepip/_bundled/pip-8.1.1-py2.py3-none-any.whl \
  https://pypi.python.org/packages/31/6a/0f19a7edef6c8e5065f4346137cc2a08e22e141942d66af2e1e72d851462/pip-8.1.1-py2.py3-none-any.whl

Note: it's important the version numbers match the expected versions in /usr/lib64/python3.4/ensurepip/__init__.py.

Back to normal

Once you've restored the missing wheel files everything should work as expected:

[bob@localhost ~]$ python3 -m venv new-env
[bob@localhost ~]$ source new-env/bin/activate
(new-env) [bob@localhost ~]$ python --version
Python 3.4.5
(new-env) [bob@localhost ~]$ pip --version
pip 8.1.1 from /home/bob/new-env/lib64/python3.4/site-packages (python 3.4)