python-r1 User's Guide

1.  Multi-implementation support

Short introduction

Currently, Gentoo supports three different Python interpreters — CPython, PyPy and Jython. CPython has two distinct major branches — called Python 2 & Python 3. These are the four interpreter groups.

Each group is further sub-divided into branches, which are identified by minor version numbers: Python 2.7, 2.6, 2.5, 3.3, 3.2, 3.1; PyPy 1.9 and 1.8; and lastly, Jython 2.5. All these versions can be installed in Gentoo concurrently, and are collectively referred to as Python implementations.

There may be various reasons to install multiple Python implementations. The most important one is that many Python packages in Gentoo still don't support Python 3 while a few require it. Therefore, in order to maintain compatibility with the both kinds of packages, Gentoo enables both versions by default.

Having two or more implementations installed allows our users to run all kinds of scripts. It gives them an ability to test their own scripts against multiple Python implementations.

PYTHON_TARGETS setting

Having a number of implementations to choose from, the modern Python packages provide users with an ability to explicitly select one or more Python implementations. This selection is performed through the use of PYTHON_TARGETS expanded USE flags.

$ emerge -pv app-portage/flaggie

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R   *] app-portage/flaggie-9999::mgorny  PYTHON_TARGETS="python2_6 python2_7 python3_2 -python3_1" 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB

As you can see in the above sample, the package in question supports four Python implementations of which three are enabled. This means that all the Python modules and scripts will be installed for those three Python versions, and thus they will be available to the scripts run using those versions.

CPython versions 2.7 and 3.2 are enabled by default. If you wish to use a different set of enabled implementations, you have to set the PYTHON_TARGETS variable in make.conf. Please note that it is not incremental — that is, you need to list all the enabled implementations.

PYTHON_TARGETS="python2_6 python2_7 python3_2"

Selecting active Python implementation

The app-admin/eselect-python package provides a tool to select the currently active Python implementations. It provides the choice of ‘active’ Python 2 and Python 3 interpreters, and which one is preferred.

The selections done through eselect-python influence:

1. the interpreter started by python, python2 and python3 commands,
2. the interpreter used to start non-versioned Python scripts,
3. the Python implementations for which packages not respecting PYTHON_TARGETS are built (unless overriden).

In order to obtain a list of installed Python interpreters supported by eselect-python, use the eselect python list command:

$ eselect python list
Available Python interpreters:
  [1]   python2.5
  [2]   python2.6
  [3]   python2.7 *
  [4]   python3.1
  [5]   python3.2

The asterisk highlights the currently active ‘main’ Python interpreter. The choice of main Python interpreter both influences the Python 2/3 preference and the active Python version in the appropriate group. In order to set another implementation, use eselect python set:

$ eselect python set python3.2
$ eselect python show
python3.2

Aside to setting the main Python interpreter, eselect-python is capable of setting the active version of Python 2 and Python 3. In order to enable that mode, the --python2 or --python3 parameter should be added after the command:

$ eselect python list --python3
Available Python 3 interpreters:
  [1]   python3.1
  [2]   python3.2 *
$ eselect python set --python3 python3.2
$ eselect python show --python3
python3.2
$ eselect python show
python2.7

The previous example emphasizes that setting the active Python 3 interpreter does not influence the main version of Python interpreter if it is Python 2. The same rule applies to setting active Python 2 when the main interpreter is Python 3.

Compatibility with python.eclass

Currently, most of the Python packages in Gentoo do not support explicit Python implementation selection via PYTHON_TARGETS. For that reason, it is common that packages supporting use of PYTHON_TARGETS variable can depend on those which don't, and vice versa. Therefore, it is necessary to ensure that those inter-dependent packages work correctly.

If your PYTHON_TARGETS setting consists of exactly one CPython 2 version and one CPython 3 version, it is enough to ensure that these versions are selected using eselect python. This is often the case with default PYTHON_TARGETS value.

$ emerge --info | grep -o 'PYTHON_TARGETS="[^"]*"'
PYTHON_TARGETS="python3_2 python2_7"
$ eselect python set --python3 python3.2
$ eselect python set --python2 python2.7

If your PYTHON_TARGETS setting contains only a single version of Python which is either CPython 2 or CPython 3, then it's necessary to ensure that no CPython version from the other group is installed. If that is undesired, USE_PYTHON must be used.

$ emerge --info | grep -o 'PYTHON_TARGETS="[^"]*"'
PYTHON_TARGETS="python2_7"
$ eselect python set python2.7
$ emerge --unmerge python:{3.1,3.2,3.3}
[…]

If your PYTHON_TARGETS has ony other value or you don't want to change the active Python versions, you can list the same implementations in the USE_PYTHON variable. It uses different name semantics than PYTHON_TARGETS, therefore the values need to be translated.

PYTHON_TARGETS="python2_6 python2_7 python3_2 python3_3 pypy1_8 pypy1_9"
USE_PYTHON="2.6 2.7 3.2 3.3 2.7-pypy-1.8 2.7-pypy-1.9"

Python script renaming

You may have noticed already that the default installation of Python scripts in Gentoo differs to that of other distributions (and Python packages themselves). This is done in order to completely support multiple Python implementations being used on the same system.

$ ls -1 /usr/bin/flaggie*
/usr/bin/flaggie
/usr/bin/flaggie-python2.6
/usr/bin/flaggie-python2.7
/usr/bin/flaggie-python3.2

Firstly, all Python scripts installed by packages are installed in per-implementation variants. This ensures that any changes necessary for a given Python implementation are preserved in the scripts. It makes it possible to easily run the script with the desired interpreter.

In addition, a wrapper is installed in place of the standard script. The wrapper is a very simple tool which checks which of the Python implementations that the script supports would be ‘preferred’, and runs that script variant.

2.  Common problems

REQUIRED_USE flag constraints unsatisfied

In some cases, an attempt to install or upgrade a package may result in portage throwing an error described as ‘unsatisfied REQUIRED_USE flag constraint’.

$ emerge -1 argparse

These are the packages that would be merged, in order:

Calculating dependencies \

!!! Problem resolving dependencies for dev-python/argparse
... done!

!!! The ebuild selected to satisfy "argparse" has unmet requirements.
- dev-python/argparse-1.2.1-r2::gentoo-cvs USE="(multilib)" PYTHON_TARGETS="-jython2_5 -python2_5 -python2_6 -python3_1"

  The following REQUIRED_USE flag constraints are unsatisfied:
    any-of ( python_targets_python2_5 python_targets_python2_6 python_targets_python3_1 python_targets_jython2_5 )

This error message means that you are attempting to install a package which does not support any of the Python implementations enabled on your system.

In the most common case, the package in question does not support the particular implementation of your choice. The issue could be solved through enabling one of the listed implementations by adding it to PYTHON_TARGETS.

In a few very specific cases, this might also mean that a particular package is unsuitable or unnecessary on your system. For example, the argparse Python module is included in CPython versions 2.7, 3.2 and newer, and thus installing it is not meaningful on systems using only those versions of Python. Then the issue should be solved through removing the offending package.

The contents of this document, unless otherwise expressly stated, are licensed under the CC-BY-SA-3.0 license. The Gentoo Name and Logo Usage Guidelines apply.