Gentoo Python Developers Guide

1.  Bumps, Drops, Stabilization

Version Bumps and Fixing Bugs

Never bump one of the following packages yourself if permission to do so is not explicitly granted by a (Co-)Lead:

• dev-lang/python
• dev-python/pycrypto

While doing version bumps is surely appreciated, don't do it blindly. There were many bugs in the past which had been carried from version to version without being noticed. Make also sure that you check bugzilla before the bump to see whether there are open bugs for the package. Reading the ChangeLog of a package is also a good idea to catch new, changed or optional dependencies.

Not all existing ebuilds in the tree use the eclasses properly (see below), so please fix bugs on sight. Build the packages you're bumping or fixing even on small changes. Not all ebuilds have been written carefully while others might have been perfect when they have been committed. But over time, practice and rules change.

The same goes for fixing bugs in the ebuilds. Please check whether there is a new version of the package out and do a version bump accordingly. Closing bugs is good, but not enough. Your primary objective should not be to close bugs but to maintain the packages in the herd.

Ask for and do peer reviews. With such a practice, ebuild quality increases and it is a good way to transfer knowledge.

Dropping old versions and Stabilization

Every team member should try to keep the package folders clean and uncluttered. Besides the obvious checks (last stable for an arch, last not p.masked, other packages depend on an exact version of this package), there are some other things which you should consider before dropping an old version:

• When dropping an unstable version in the presence of a stable one: Does the version you are going to drop have serious bugs which avoid stabilization? Otherwise you might keep it and open a stabilization bug.
• The same consideration also applies if there is no stable version yet: Are there users who might want a stable version? Is this package mature enough to go stable? If you decide to stabilize it, also think about how arch team members could test it.
• Do not stabilize alpha and beta versions nor release candidates wherever possible. There are exceptions to this (if upstream just produces beta-ware or the package is desperately needed for another app). If unsure, talk to the Lead first.

2.  Correct eclass usage

There are currently 3 python related eclasses: python, distutils and twisted.

python.eclass

Here are some important things to remember when using this eclass:

• Call python_version before using PYVER, PYVER_MINOR or PYVER_MAJOR.
• Insert a line "NEED_PYTHON=MY_VERSION" before the inherit to depend on python. If you don't need a specific version, put virtual/python in DEPEND/RDEPEND.
• If you don't use distutils (and/or your package installs .py files somewhere else than site-packages), you have to call python_mod_optimize and python_mod_cleanup yourself. If your package doesn't install itself into a subdir, omit "/YOURPACKAGE". You can pass any path you want to python_mod_optimize/cleanup. python_mod_cleanup will also remove empty directories after the cleanup. If you call python as root, .py files will get byte-compiled to .pyc but not recorded in the package's CONTENTS. They therefore won't get removed on update/removal of the package.

inherit python multilib

[...]

pkg_postinst() {
	python_version
	python_mod_optimize ${ROOT}usr/$(get_libdir)/python${PYVER}/site-packages/YOURPACKAGE
}

pkg_postinst() {
	python_mod_cleanup
}

distutils.eclass

• To depend on a specific version of python, put NEED_PYTHON=MY_VERSION before the inherit (rather than DEPEND/RDEPEND on dev-lang/python-MY_VERSION) or leave it away to implicitly depend on virtual/python through the distutils eclass.
• If the ebuild name (in ${PN}) differs from the folder created by the package in site-packages/, you have to define a variable PYTHON_MODNAME to tell distutils where to look for the module.
• Set the DOCS variable to install additional (pure-text) docs. If you write the src_install-function, you can put the definition of the DOCS var in the function but before calling distutils_src_install.

# Copyright 1999-2007 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: /var/cvsroot/gentoo/xml/htdocs/proj/en/Python/developersguide.xml,v 1.4 2007/07/11 01:54:28 hawking Exp $

NEED_PYTHON=2.3

inherit distutils

DESCRIPTION="An advanced interactive shell for Python."
HOMEPAGE="http://ipython.scipy.org/"
SRC_URI="http://ipython.scipy.org/dist/${P}.tar.gz"

LICENSE="BSD"
SLOT="0"
KEYWORDS="~amd64 ~ia64 ~ppc ~s390 ~x86"
IUSE="doc examples emacs gnuplot test"

DEPEND="test? ( dev-python/pexpect )"
RDEPEND="gnuplot? ( dev-python/gnuplot-py )"

PYTHON_MODNAME="IPython"

[...]

src_install() {
	DOCS="foo.txt bar.txt"
	distutils_src_install
	dohtml foo.html bar.png
}

DDOCS="CHANGELOG KNOWN_BUGS MAINTAINERS PKG-INFO CONTRIBUTORS TODO NEWS"
DDOCS="${DDOCS} Change* MANIFEST* README*"

twisted.eclass

TBD

3.  Ebuild writing

Here are some important things to keep in mind when writing ebuilds for python packages.

Calling python in ebuilds

To call python in one of your ebuilds, use "${python}" to do it (defined in distutils.eclass).

Checking for tkinter support

To check whether python has tkinter support you can use one of python_tkinter_exists (from python.eclass) or distutils_tkinter_exists (from distutils.eclass) in pkg_setup. These two functions are identical.

4.  Common Problems and Mistakes

Below are common problems you may face and common mistakes made when writing ebuilds for python packages.

setuptools: *_require and use_setuptools()

Packages that use setuptools to install use _require options like tests_require,install_require,setup_requires in setup.py. These are nice to learn about dependencies but you don't want them in setup.py when you're installing the package. The following is from the setuptools homepage section on setup_requires:

A string or list of strings specifying what other distributions need to be present in order for the setup script to run. setuptools will attempt to obtain these (even going so far as to download them using EasyInstall) before processing the rest of the setup script or commands.

—setuptools developer's guide

We have lovely package managers which can download stuff for us and verify their digests thus we don't want to download any packages using EasyInstall. There are other options like tests_require, install_requires that behave the same way.

Some packages have a ez_setup.py along with the usual setup.py. This is a python script to download and install appropriate setuptools. To do this use_setuptools() is called from ez_setup.py before importing setuptools.

def use_setuptools(
    version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir,
    download_delay=15
):
    """Automatically find/download setuptools and make it available on sys.path
    [...]

Just like the _require options, if a setup.py script calls use_setuptools() from ez_setup.py you should remove it. Below is an example which illustrates how to do it.

from ez_setup import use_setuptools
use_setuptools()
from setuptools import setup, find_packages

[...]

install_requires=["Routes >= 1.0", "Paste", "PasteDeploy", "PasteScript"],

[...]

src_unpack() {
	unpack ${A}
	cd "${S}"
	sed -i \
		-e '/use_setuptools/d' \
		-e '/install_requires=\[.*\],/d' \
		setup.py || die "sed failed"
}

src_test and PYTHONPATH

When testing python packages it's important to make sure we're actually testing the package that is going to be merged not the already installed package. We can solve the problem by setting the PYTHONPATH environment variable which augments the default search path for module files. Here are two examples:

src_test() {
	PYTHONPATH=build/lib/ "${python}" test.py || die "tests failed"
}

src_test() {
	PYTHONPATH="$(ls -d build/lib.*)" "${python}" examples/genkey.py || die "genkey test failed"
}

As you may have noticed if the module is written in languages like C, C++, etc. the name of the directory in build varies between architectures but it always starts with lib.

The contents of this document are licensed under the Creative Commons - Attribution / Share Alike license.