Table of Contents
List of Tables
Table of Contents
Table of Contents
If a bashrc file is located at /etc/portage/bashrc
then it will be sourced before an ebuild is executed.
A phase hook function name begins with a pre_ or post_ prefix to indicate that it will be called before or after one of the ebuild phases. The prefix is followed by the name of the ebuild function that the hook will be associated with. For example, a hook named pre_src_compile will be called before src_compile, and a hook named post_src_compile will be called after src_compile.
Table of Contents
Table of Contents
Dependency resolution involves satisfaction of many constraints:
Persistent configuration parameters, like those that come from make.profile, make.conf, and the /etc/portage directory.
Current command parameters, which may include options, atoms, or sets.
If one package blocks another package, the two packages conflict such that they cannot be installed simultaneously. These conflicts are often due to file collisions. In some cases, packages that block each other can be temporarily installed simultaneously. In order to resolve file collisions that occur between two blocking packages that are installed simultaneously, the overlapping files must be removed from the contents list of the package which was installed first.
Some cases may exist such that temporary simultaneous installation of blocking packages will cause some sort of problem. However, this type of solution will only be chosen for blockers that can not be satisfied in any other way, such as by simple adjustment of merge order. In addition, this type of solution will not be chosen if a blocking package will overwrite files belonging to packages from the system set, or packages that are runtime dependencies of Portage itself. These constraints serve to limit the probability that a chosen solution will cause an unforeseen problem.
If two different packages that occupy the same slot are chosen to satisfy dependencies, a slot conflict occurs. The two packages cannot be installed simultaneously and therefore the respective dependencies will not be satisfied simultaneously.
In order to significantly reduce the resources consumed by the modeling process, the dependencies of installed packages may be neglected.
If a more complete dependency calculation is desired, there is a --complete-graph option which will ensure that the dependencies of installed packages are properly considered.
Table of Contents
In terms of boolean logic, a dependency expression can be expressed in disjunctive normal form (DNF), which is a disjunction of conjunctive clauses. Each conjunctive clause represents one possible alternative combination of dependency atoms capable of satisfying the dependency expression.
When there are multiple combinations to choose from, a look-ahead mechanism will choose an optimal combination to satisfy constraints and minimize cost. The following package states influence the cost calculation for a given combination:
installed
selected (for installation)
not selected (for installation)
In cost calculations, virtual packages by themselves are considered to cost nothing since they do not directly install anything. It is the dependencies of a virtual package that contribute to it's cost.
Combinations that include packages from the "installed" or "selected" categories are less costly than those that include packages from the "not selected" category. When a package is chosen for installation, it transitions to the "selected" state. This state change propagates to the cost calculations of later decisions, influencing later decisions to be consistent with earlier decisions. This feedback mechanism serves to propagate constraints and can influence the modeling process to converge on a more optimal final state.
When evaluating virtual atoms, an expanded search space is considered which recursively traverses the dependencies of virtual packages from all slots matching a given virtual atom. All combinations in this expanded search space are considered when choosing an optimal combination to satisfy constraints with minimal cost.
All tasks are executed in an order such that a task's dependencies are satisfied when it is executed. Dependency relationships between tasks form a directed graph.
Sometimes a package installation order exists such that it is possible to avoid having two conflicting packages installed simultaneously. If a currently installed package conflicts with a new package that is planned to be installed, it may be possible to solve the conflict by replacing the installed package with a different package that occupies the same slot.
In order to avoid a conflict, a package may need to be uninstalled rather than replaced. The following constraints protect inappropriate packages from being chosen for automatic uninstallation:
Installed packages that have been pulled into the current dependency graph will not be uninstalled. Due to dependency neglection and special properties of packages in the "system" set, other checks may be necessary in order to protect inappropriate packages from being uninstalled.
An installed package that is matched by a dependency atom from the "system" set will not be uninstalled in advance since it might not be safe. Such a package will only be uninstalled through replacement.
An installed package that is matched by a dependency atom from the "world" set will not be uninstalled if the dependency graph does not contain a replacement package that is matched by the same dependency atom.
In order to ensure that package files remain installed in a usable state whenever possible, uninstallation operations are not executed until after all associated conflicting packages have been installed. When file collisions occur between conflicting packages, the contents entries for those files are removed from the packages that are scheduled for uninstallation. This prevents uninstallation operations from removing overlapping files that have been claimed by conflicting packages.
TODO: Automatically solve circular dependencies by temporarily disabling conditional dependencies and then rebuilding packages with the conditional dependencies enabled.
The algorithm used to choose packages that will execute concurrently with other packages is as conservative as possible in the sense that a given package will not be executed if the subgraph composed of its direct and indirect dependencies contains any scheduled merges. By ensuring that the subgraph of deep dependencies is fully up to date in this way, potential problems are avoided which could be triggered by other build orders that are less optimal.
Table of Contents
Table of Contents
Ebuild execution is divided into a series of phases. In order to implement a phase, an ebuild defines a function to serve as an entry point for execution of that phase. This design is similar to the template method pattern that is commonly used in object oriented programming languages. An ebuild can inherit or override a template method from an eclass.
The function names for the ebuild phases, listed in order of execution:
pkg_setup
src_unpack
src_compile
src_test
src_install
pkg_preinst
pkg_postinst
pkg_prerm
pkg_postrm
The order for upgrade and downgrade operations changed in version 2.1.5, but the order for reinstall operations remained unchanged.
pkg_preinst
pkg_postinst
pkg_prerm
pkg_postrm
The new order for upgrades and downgrades is identical to the order used for reinstall operations:
pkg_preinst
pkg_prerm
pkg_postrm
pkg_postinst
Now that pkg_postinst is called after all other phases, it's not possible to call has_version in pkg_postinst to detect whether the current install operation is an upgrade or downgrade. If this information is needed during the pkg_postinst phase, do the has_version call in an earlier phase (such as pkg_preinst) and store the result in a global variable to be accessed by pkg_postinst when it is called.
Like einfo, we output a helpful message and then hint that the following operation may take some time to complete. Once the task is finished, you need to call eend.
Followup the ebegin message with an appropriate "OK" or "!!" (for errors) marker. If status is non-zero, then the additional error message is displayed.
Same as elog, but should be used when the message isn't important to the user (like progress or status messages during the build process).
If you need to display a message that you wish the user to read and take notice of, then use elog. It works just like echo(1), but adds a little more to the output so as to catch the user's eye. The message will also be logged by portage for later review.
A + or - prefix added to the beginning of a flag in IUSE creates a default USE setting that respectively enables or disables the corresponding USE flag.
Support for the ECONF_SOURCE variable, which is also supported by econf, has been added to the default src_compile implementation.
src_compile() {
if [[ -x ${ECONF_SOURCE:-.}/configure ]] ; then
econf
fi
if [ -f Makefile ] || [ -f GNUmakefile ] || [ -f makefile ] ; then
emake || die "emake failed"
fi
}
Blocker atoms which use the previously existing !atom syntax now have a slightly different meaning. These blocker atoms indicate that conflicting packages may be temporarily installed simultaneously. When temporary simultaneous installation of conflicting packages occurs, the installation of a newer package may overwrite any colliding files that belong to an older package which is explicitly blocked. When such file collisions occur, the colliding files cease to belong to the older package, and they remain installed after the older package is eventually uninstalled. The older package is uninstalled only after any newer blocking packages have been merged on top of it.
A new !!atom syntax is now supported, for use in special cases for which temporary simultaneous installation of conflicting packages should not be allowed. If a given package happens to be blocked my a mixture of atoms consisting of both the !atom and !!atom syntaxes, the !!atom syntax takes precedence over the !atom syntax.
A new syntax is supported which allows customization of the output file name for a given URI. In order to customize the output file name, a given URI should be followed by a "->" operator which, in turn, should be followed by the desired output file name. As usual, all tokens, including the operator and output file name, should be separated by whitespace.
A new src_prepare function is called after the src_unpack function, with cwd initially set to $S.
The configure portion of the src_compile function has been split into a separate function which is named src_configure. The src_configure function is called in-between the src_prepare and src_compile functions.
src_configure() {
if [[ -x ${ECONF_SOURCE:-.}/configure ]] ; then
econf
fi
}
src_compile() {
if [ -f Makefile ] || [ -f GNUmakefile ] || [ -f makefile ] ; then
emake || die "emake failed"
fi
}
Table 5.5. Execution Order of Phase Functions
| Phase Function Name |
|---|
| pkg_setup |
| src_unpack |
| src_prepare |
| src_configure |
| src_compile |
| src_test |
| src_install |
| pkg_preinst |
| pkg_postinst |
| pkg_prerm |
| pkg_postrm |
The default pkg_nofetch and src_* phase functions are now accessible via a function having a name that begins with default_ and ends with the respective phase function name. For example, a call to a function with the name default_src_compile is equivalent to a call to the default src_compile implementation.
Table 5.6. Default Phase Functions
| Function Name |
|---|
| default_pkg_nofetch |
| default_src_unpack |
| default_src_prepare |
| default_src_configure |
| default_src_compile |
| default_src_test |
A function named "default" is redefined for each phase so that it will call the default_* function corresponding to the current phase. For example, a call to the function named "default" during the src_compile phase is equivalent to a call to the function named default_src_compile.
Beginning with EAPI 3, all helpers use ${ED} instead of ${D} when appropriate. For example, see econf and einstall below.
${ECONF_SOURCE:-.}/configure \
${CBUILD:+--build=${CBUILD}} \
--datadir="${EPREFIX}"/usr/share \
--host=${CHOST} \
--infodir="${EPREFIX}"/usr/share/info \
--localstatedir="${EPREFIX}"/var/lib \
--prefix="${EPREFIX}"/usr \
--mandir="${EPREFIX}"/usr/share/man \
--sysconfdir="${EPREFIX}"/etc \
${CTARGET:+--target=${CTARGET}} \
${EXTRA_ECONF} \
configure options || die "econf failed"
Note that, for make-based packages, 'emake install DESTDIR=${D}' (with DESTDIR=${D} rather than ${ED}) is still preferred over einstall.
make \
prefix=${ED}/usr \
datadir=${ED}/usr/share \
infodir=${ED}/usr/share/info \
localstatedir=${ED}/var/lib \
mandir=${ED}/usr/share/man \
sysconfdir=${ED}/etc \
${EXTRA_EINSTALL} \
make options \
install
Table 5.7. Installation Prefix Variables
| Variable Name | Description |
|---|---|
| ED | Contains the path "${D%/}${EPREFIX}/" for convenience purposes. For EAPI values prior to EAPI 3 which do not support ${ED}, helpers use ${D} where they would otherwise use ${ED}. Do not modify this variable. |
| EPREFIX | Contains the offset that this Portage was configured for during installation. The offset is sometimes necessary in an ebuild or eclass, and is available in such cases as ${EPREFIX}. EPREFIX does not contain a trailing slash, therefore an absent offset is represented by the empty string. Do not modify this variable. |
| EROOT | Contains "${ROOT%/}${EPREFIX}/" for convenience purposes. Do not modify this variable. |
Table of Contents
Table of Contents
Here we'll go over each QA notice and what you (as a developer) can do to fix the issue. If you're a user, you should of course go file a bug. We'll only cover the non-obvious notices here.
In pretty much all cases, you should try and get these issues resolved upstream rather than simply fixing them in our ebuilds.
QA Notice: The following files contain insecure RUNPATHs
Some of the ELFs that would be installed on the system have insecure dynamic RUNPATH tags. RUNPATH tags are a hardcoded list of filesystem paths that will be searched at runtime when the ELF is executed. If the ELF has a world accessible directory hardcoded in it, then a malicious person can inject code at runtime by adding their own libraries to the directory.
Here are some of the common problems and their solutions.
Libtool - old versions of libtool would use too many -rpath flags
Solution: Regenerate the autotool code
Perl - some versions of perl would use incorrect -rpath flags
Solution: Upgrade system perl build modules
Crappy build system - the custom build system uses -rpath incorrectly
Solution: Review the LDFLAGS in the build system and make them not suck
Crappy ebuild - the ebuild installs ELFs instead of using the package's build system
Solution: Fix the crappy ebuild to use the package's build system
QA Notice: The following files contain runtime text relocations
Please see the Gentoo Hardened PIC Fix Guide.
QA Notice: The following files contain executable stacks
Please see the Gentoo Hardened GNU Stack Guide.
QA Notice: The following shared libraries lack a SONAME
A shared library that you would link against lacks an ELF SONAME tag. With simpler libraries, this can be acceptable, but with any sort of ABI sane setup, you need the SONAME tag. This tag is how the system linker tells the loader what libraries a program needs at runtime. With a missing SONAME, the linker needs to guess and with many cases, this guess will not work for long.
To fix this issue, make sure the shared library is linked with the proper
-Wl,-soname,... flag. You will need to replace the
... part with the actual ABI name. For example,
if the library is named libfoo.so.1.2.3, you will
probably want to specify -Wl,-soname,libfoo.so.1.
Note that this warning only applies to shared libraries that you would link
against. It certainly does not apply to plugins that you would dynamically
load. However, plugins should not exist in the main library directory, but
rather an application specific subdirectory in the library directory. In
other words, it should be /usr/lib/app/plugin.so rather
than /usr/lib/plugin.so.
QA Notice: The following shared libraries lack NEEDED entries
This warning comes up when a library does not actually seem to need any other libraries in order to run. Rarely is this true as almost every library will need at least the system C library.
Once you've determined that the library is indeed being generated incorrectly, you will need to dig into the build system to make sure that it pulls in the libraries it needs. Often times, this is because the build system invokes the system linker (ld) directly instead of the system compiler driver (gcc).
QA Notice: Found an absolute symlink in a library directory
If you want to use symlinks in library directories, please use either a relative symlink or a linker script. This can cause problems when working with cross-compiler systems or when accessing systems in a different ROOT directory.
If you have a library installed into /lib/ and you want
to have it accessible in /usr/lib/, then you should
generate a linker script so that the system toolchain can handle it properly.
Please see the linker script section
for more information.
QA Notice: Missing gen_usr_ldscript
If you have a shared library in /lib/ and a static
library in /usr/lib/, but no linker script in
/usr/lib/, then the toolchain will choose the incorrect
version when linking. The system linker will find the static library first
and not bother searching for a dynamic version. To overcome this, you need
to use the gen_usr_ldscript function found in the
toolchain-funcs.eclass. Refer to the
man page for information on how to use it. See this
bug report for some history
on this issue.
QA Notice: Excessive files found in the / partition
You should not store files that are not critical to boot and recovery in
the root filesystem. This means that static libraries and libtool scripts do
not belong in the /lib/ directory. Fix your ebuild so
it does not install there.
QA Notice: ... appears to contain PORTAGE_TMPDIR paths
Older versions of libtool would incorrectly record the build and/or install directory in the libtool script (*.la). This would lead to problems when building other things against your package as libtool would be confused by the old paths.
You may be able to cheat and use the elibtoolize function in the libtool.eclass. However, if that does not help, you will probably need to regenerate all of the autotool files.
QA Notice: Package has poor programming practices which may compile
fine but exhibit random runtime failures.
...: warning: dereferencing type-punned pointer will break strict-aliasing rules
This warning crops up when code starts casting distinct pointer types and then dereferencing them. Generally, this is a violation of aliasing rules which are part of the C standard. Historically, these warnings did not show up as the optimization was not turned on by default. With gcc-4.1.x and newer though, the -O2 optimization level enables strict aliasing support. For information, please review these links: NetBSD Explanation, Gentoo Dev Thread, GCC Docs
To fix this issue, use the methods proposed in the links mentioned earlier. If you're unable to do so, then a work around would be to append the gcc -fno-strict-aliasing flag to CFLAGS in the ebuild.
QA Notice: Package has poor programming practices which may compile
fine but exhibit random runtime failures.
...: warning: implicit declaration of function ...
...: warning: incompatible implicit declaration of built-in function ...
Your code is calling functions which lack prototypes. In C++, this would have been a build failure, but C is lazy so you just get a warning. This can be a problem as gcc has to guess at what sort of arguments a function takes based upon how it was called and often times, this is not the same as what the function actually takes. The function return type is also unknown so it's just assumed to be an integer (which is often times wrong). This can get to be a problem when the size of the types guessed do not actually match the size of the types the function expects. Generally, this corresponds directly to proper coding practices (and the lack thereof). Also, by including proper prototypes, the compiler often helps by checking types used, proper number of arguments passed, etc...
To fix this, just include the proper header files for the functions in question. If the function is a package-specific one, then you may have to create a header/function prototype for it.
QA Notice: Package has poor programming practices which may compile
fine but exhibit random runtime failures.
...: warning: is used uninitialized in this function
This means code uses a variable without actually setting it first. In other words, the code is basically using random garbage.
The fix here is simple: make sure variables are initialized properly before using them.
QA Notice: Package has poor programming practices which may compile
fine but exhibit random runtime failures.
...: warning: comparisons like X<=Y<=Z do not have their mathematical meaning
This warning crops up either when the programmer expected the expression to work or they just forgot to use sufficient parentheses. For example, the following code snippets are wrong (we won't get into the technical argument of this being valid C code; just change the code to not be ambiguous).
if (x <= y <= z)
...;
if (a < b <= c)
...;
To fix this, read the code to figure out what exactly the programmer meant.
QA Notice: Package has poor programming practices which may compile
fine but exhibit random runtime failures.
...: warning: null argument where non-null required
Many functions take pointers as arguments and require that the pointer never be NULL. To this end, you can declare function prototypes that instruct the compiler to do simple checks to make sure people do not incorrectly call the function with NULL values. This warning pops up when someone calls a function and they use NULL when they should not. Depending on the library, the function may actually crash (they told you not to use NULL after-all, so it's your fault :P).
You will need to read the code and fix it so that it does not incorrectly call the relevant functions with NULL values.
QA Notice: Package has poor programming practices which may compile
but will almost certainly crash on 64bit architectures.
A large portion of code in the open source world is developed on the 32bit x86 architecture. Unfortunately, this has led to many pieces of code not handling pointer types properly. When compiled and run on a 64bit architecture, the code in question will probably crash horribly. Some common examples are assuming that an integer type is large enough to hold pointers. This is true on 32bit architectures (an integer can hold 32bits and a pointer is 32bits big), but not true on 64bit architectures (an integer still holds just 32bits, but a pointer is 64bits big).
Since this issue can manifest itself in many ways (as there are many ways to improperly truncate a pointer), you will need to read the source code starting with the displayed warning. Make sure types are declared, used, and passed properly. Make sure that all function prototypes are found (see the Implicit Declarations section for more information). So on and so forth.