portage.dbapi.vartree module

Bases: object

This class provides an interface to the installed package database At present this is implemented as a text backend in /var/db/pkg.

_acquire_slot_locks(db)

Acquire slot locks for the current package and blocked packages.

_add_preserve_libs_to_contents(preserve_paths)

Preserve libs returned from _find_libs_to_preserve().

_clear_contents_cache()
_collision_protect(srcroot, destroot, mypkglist, file_list, symlink_list)
_contents_re = re.compile('^((?P<dir>(dev|dir|fif) (.+))|(?P<obj>(obj) (.+) (\\S+) (\\d+))|(?P<sym>(sym) (.+) -> (.+) ((\\d+)|(?P<oldsym>(\\(\\d+, \\d+L, \\d+L, \\d+, \\d+, \\d+, \\d+L, \\d+, (\\d+), \\d+\\))))))$')
_display_merge(msg, level=0, noiselevel=0)
_eerror(phase, lines)
_elog(funcname, phase, lines)
_elog_process(phasefilter=None)
_emerge_log(msg)
_eqawarn(phase, lines)
_find_libs_to_preserve(unmerge=False)

Get set of relative paths for libraries to be preserved. When unmerge is False, file paths to preserve are selected from self._installed_instance. Otherwise, paths are selected from self.

_find_unused_preserved_libs(unmerge_no_replacement)

Find preserved libraries that don’t have any consumers left.

_format_contents_line(node_type, abs_path, md5_digest=None, symlink_target=None, mtime_ns=None)
_get_protect_obj()
_ignored_rmdir_errnos = (17, 39, 16, 2, 20, 21, 1)
_infodir_cleanup = frozenset({'dir', 'dir.old'})
_linkmap_rebuild(**kwargs)

Rebuild the self._linkmap if it’s not broken due to missing scanelf binary. Also, return early if preserve-libs is disabled and the preserve-libs registry is empty.

_lstat_inode_map(path_iter)
Use lstat to create a map of the form:

{(st_dev, st_ino) : set([path1, path2, …])}

Multiple paths may reference the same inode due to hardlinks. All lstat() calls are relative to self.myroot.

_match_contents(filename, destroot=None)

The matching contents entry is returned, which is useful since the path may differ from the one given by the caller, due to symlinks.

Return type

String

Returns

the contents entry corresponding to the given path, or False if the file is not owned by this package.

_merge_contents(srcroot, destroot, cfgfiledict)
_merged_path(path, lstatobj, exists=True)
_new_backup_path(p)

The works for any type path, such as a regular file, symlink, or directory. The parent directory is assumed to exist. The returned filename is of the form p + ‘.backup.’ + x, where x guarantees that the returned path does not exist yet.

_normalize_needed = re.compile('//|^[^/]|./$|(^|/)\\.\\.?(/|$)')
_post_merge_sync()

Call this after merge or unmerge, in order to sync relevant files to disk and avoid data-loss in the event of a power failure. This method does nothing if FEATURES=merge-sync is disabled.

_pre_merge_backup(backup_dblink, downgrade)
_pre_unmerge_backup(background)
_protect(cfgfiledict, protect_if_modified, src_md5, src_link, dest, dest_real, dest_mode, dest_md5, dest_link)
_prune_plib_registry(unmerge=False, needed=None, preserve_paths=None)
_release_slot_locks()

Release all slot locks.

_remove_preserved_libs(cpv_lib_map)

Remove files returned from _find_unused_preserved_libs().

_security_check(installed_instances)
_send_mtimes(mtimes)
_show_unmerge(zing, desc, file_type, file_name)
_slot_locked()

A decorator function which, when parallel-install is enabled, acquires and releases slot locks for the current package and blocked packages. This is required in order to account for interactions with blocked packages (involving resolution of file collisions).

_unmerge_dirs(dirs, infodirs_inodes, protected_symlinks, unmerge_desc, unlink, os)
_unmerge_pkgfiles(pkgfiles, others_in_slot)

Unmerges the contents of a package from the liveFS Removes the VDB entry for self

Parameters

  • pkgfiles (Dictionary { filename: [ 'type', '?', 'md5sum' ] }) – typically self.getcontents()

  • others_in_slot (list) – all dblink instances in this slot, excluding self

Return type

None

clearcontents()

For a given db entry (self), erase the CONTENTS values.

copyfile(fname)
delete()

Remove this entry from the database

exists()

does the db entry exist? boolean.

getcontents()

Get the installed files of a given package (aka what that package installed)

getelements(ename)
getfile(fname)
getpath()

return path to location of db information (for >>> informational display)

getstring(name)

returns contents of a file with whitespace converted to spaces

isowner(filename, destroot=None)

Check if a file belongs to this package. This may result in a stat call for the parent directory of every installed file, since the inode numbers are used to work around the problem of ambiguous paths caused by symlinked directories. The results of stat calls are cached to optimize multiple calls to this method.

Parameters

  • filename –

  • destroot –

Return type

Boolean

Returns

  1. True if this package owns the file.

  2. False if this package does not own the file.

isprotected(obj)
isregular()

Is this a regular package (does it have a CATEGORY file? A dblink can be virtual and regular)

lockdb()
merge(*args, **kwargs)
mergeme(srcroot, destroot, outfile, secondhand, stufftomerge, cfgfiledict, thismtime)

This function handles actual merging of the package contents to the livefs. It also handles config protection.

Parameters

  • srcroot (String (Path)) – Where are we copying files from (usually ${D})

  • destroot (String (Path)) – Typically ${ROOT}

  • outfile (File Object) – File to log operations to

  • secondhand – A set of items to merge in pass two (usually

or symlinks that point to non-existing files that may get merged later) :type secondhand: List :param stufftomerge: Either a diretory to merge, or a list of items. :type stufftomerge: String or List :param cfgfiledict: { File:mtime } mapping for config_protected files :type cfgfiledict: Dictionary :param thismtime: None or new mtime for merged files (expressed in seconds in Python <3.3 and nanoseconds in Python >=3.3) :type thismtime: None or Int :rtype: None or Boolean :return: 1. True on failure 2. None otherwise

quickpkg(output_file, metadata=None, include_config=False, include_unmodified_config=False)

Create a tar file appropriate for use by quickpkg.

Parameters

  • output_file (file) – Write binary tar stream to file.

  • include_config (bool) – Include all files protected by CONFIG_PROTECT (as a security precaution, default is False).

  • include_unmodified_config (bool) – Include files protected by CONFIG_PROTECT that have not been modified since installation (as a security precaution, default is False).

Return type

list

Returns

Paths of protected configuration files which have been omitted.

setelements(mylist, ename)
setfile(fname, data)
treewalk(srcroot, destroot, inforoot, myebuild, cleanup=0, mydbapi=None, prev_mtimes=None, counter=None)

This function does the following:

calls doebuild(mydo=instprep) calls get_ro_checker to retrieve a function for checking whether Portage will write to a read-only filesystem, then runs it against the directory list calls self._preserve_libs if FEATURES=preserve-libs calls self._collision_protect if FEATURES=collision-protect calls doebuild(mydo=pkg_preinst) Merges the package to the livefs unmerges old version (if required) calls doebuild(mydo=pkg_postinst) calls env_update

Parameters

  • srcroot (String (Path)) – Typically this is ${D}

  • destroot (String (Path)) – ignored, self.settings[‘ROOT’] is used instead

  • inforoot (String (Path)) – root of the vardb entry ?

  • myebuild (String (Path)) – path to the ebuild that we are processing

  • mydbapi (portdbapi instance) – dbapi which is handed to doebuild.

  • prev_mtimes (Dictionary) – { Filename:mtime } mapping for env_update

Return type

Boolean

Returns

  1. 0 on success

  2. 1 on failure

secondhand is a list of symlinks that have been skipped due to their target not existing; we will merge these symlinks at a later time.

unlockdb()
unmerge(*args, **kwargs)
updateprotect()
portage.dbapi.vartree.merge(mycat, mypkg, pkgloc, infloc, myroot=None, settings=None, myebuild=None, mytree=None, mydbapi=None, vartree=None, prev_mtimes=None, blockers=None, scheduler=None, fd_pipes=None)
Parameters

myroot – ignored, settings[‘EROOT’] is used instead

portage.dbapi.vartree.tar_contents(contents, root, tar, protect=None, onProgress=None, xattrs=False)
portage.dbapi.vartree.unmerge(cat, pkg, myroot=None, settings=None, mytrimworld=None, vartree=None, ldpath_mtimes=None, scheduler=None)
Parameters

  • myroot – ignored, settings[‘EROOT’] is used instead

  • mytrimworld – ignored

class portage.dbapi.vartree.vardbapi(_unused_param=<class 'DeprecationWarning'>, categories=None, settings=None, vartree=None)

Bases: portage.dbapi.dbapi

_add(pkg_dblink)
property _aux_cache
_aux_cache_init()
_aux_cache_keys_re = re.compile('^NEEDED\\..*$')
_aux_cache_threshold = 5
_aux_cache_version = '1'

Search environment.bz2 for the specified variables. Returns a dict mapping variables to values, and any variables not found in the environment will not be included in the dict. This is useful for querying variables like ${SRC_URI} and ${A}, which are not saved in separate files but are available in environment.bz2 (see bug #395463).

_aux_get(mycpv, wants, st=None)
_aux_multi_line_re = re.compile('^(CONTENTS|NEEDED\\..*)$')
_bump_mtime(cpv)

This is called before an after any modifications, so that consumers can use directory mtimes to validate caches. See bug #290428.

_categories = None
_category_re = re.compile('^\\w[-.+\\w]*$')
_clear_cache()
_clear_pkg_cache(pkg_dblink)
static _cmp_cpv(cpv1, cpv2)
static _cpv_sort_ascending(cpv_list)

Use this to sort self.cp_list() results in ascending order. It sorts in place and returns None.

_excluded_dirs = re.compile('^(\\..*|-MERGING-.*|CVS|lost\\+found)$')
_fs_lock()

Acquire a reentrant lock, blocking, for cooperation with concurrent processes.

_fs_unlock()

Release a lock, decrementing the recursion level.

_iter_cpv_all(use_cache=True, sort=False)
_iter_match(atom, cpv_iter)
_iter_match_repo(atom, cpv_iter)
_iter_match_slot(atom, cpv_iter)
_iter_match_use(atom, cpv_iter)

  1. Check for required IUSE intersection (need implicit IUSE here).

  2. Check enabled/disabled flag states.

_iuse_implicit_cnstr(pkg, metadata)

Construct a callable that checks if a given USE flag should be considered to be a member of the implicit IUSE for the given package.

Parameters
Returns

a callable that accepts a single USE flag argument, and returns True only if the USE flag should be considered to be a member of the implicit IUSE for the given package.

Return type

callable

_known_keys = frozenset({'BDEPEND', 'DEFINED_PHASES', 'DEPEND', 'DESCRIPTION', 'EAPI', 'HOMEPAGE', 'IDEPEND', 'INHERIT', 'INHERITED', 'IUSE', 'KEYWORDS', 'LICENSE', 'PDEPEND', 'PROPERTIES', 'RDEPEND', 'REQUIRED_USE', 'RESTRICT', 'SLOT', 'SRC_URI'})
_match_use(atom, pkg, metadata, ignore_profile=False)
class _owners_cache(vardb)

Bases: object

This class maintains an hash table that serves to index package contents by mapping the basename of file to a list of possible packages that own it. This is used to optimize owner lookups by narrowing the search down to a smaller number of packages.

_add_path(path, pkg_hash)

Empty path is a code that represents empty contents.

_hash_bits = 16
_hash_pkg(cpv)
_hash_str(s)
_hex_chars = 4
_new_hash(*, usedforsecurity=True)

Returns a md5 hash object; optionally initialized with a string

add(cpv)
_owners_cache_version = '1'
class _owners_db(vardb)

Bases: object

_iter_owners_low_mem(path_list)

This implemention will make a short-lived dblink instance (and parse CONTENTS) for every single installed package. This is slower and but uses less memory than the method which uses the basename cache.

_populate()
getFileOwnerMap(path_iter)
get_owners(path_iter)

:return the owners as a dblink -> set(files) mapping.

iter_owners(path_iter)

Iterate over tuples of (dblink, path). In order to avoid consuming too many resources for too much time, resources are only allocated for the duration of a given iter_owners() call. Therefore, to maximize reuse of resources when searching for multiple files, it’s best to search for them all in a single call.

populate()
_pkg_str(cpv, repo)

This is used to contruct _pkg_str instances on-demand during matching. If cpv is a _pkg_str instance with slot attribute, then simply return it. Otherwise, fetch metadata and construct a _pkg_str instance. This may raise KeyError or InvalidData.

_pkg_str_aux_keys = ('EAPI', 'KEYWORDS', 'SLOT', 'repository', 'BUILD_ID', 'BUILD_TIME', '_mtime_')
_remove(pkg_dblink)
_slot_lock(slot_atom)

Acquire a slot lock (reentrant).

WARNING: The varbapi._slot_lock method is not safe to call in the main process when that process is scheduling install/uninstall tasks in parallel, since the locks would be inherited by child processes. In order to avoid this sort of problem, this method should be called in a subprocess (typically spawned by the MergeProcess class).

_slot_unlock(slot_atom)

Release a slot lock (or decrementing recursion level).

_use_mutable = False
aux_get(mycpv, wants, myrepo=None)

This automatically caches selected keys that are frequently needed by emerge for dependency calculations. The cached metadata is considered valid if the mtime of the package directory has not changed since the data was cached. The cache is stored in a pickled dict object with the following format:

{version:”1”, “packages”:{cpv1:(mtime,{k1,v1, k2,v2, …}), cpv2…}}

If an error occurs while loading the cache pickle or the version is unrecognized, the cache will simple be recreated from scratch (it is completely disposable).

aux_update(cpv, values)
Args:
cpv - “sys-apps/foo-1.0”

metadata_updates = { key : newvalue }

Returns:

None

property categories

Use self.cp_all() to generate a category list. Mutable instances can delete the self._categories attribute in cases when the cached categories become invalid and need to be regenerated.

checkblockers(origdep)
close_caches()
counter_tick(myroot=None, mycpv=None)
Parameters

myroot – ignored, self._eroot is used instead

counter_tick_core(myroot=None, incrementing=1, mycpv=None)

This method will grab the next COUNTER value and record it back to the global file. Note that every package install must have a unique counter, since a slotmove update can move two packages into the same SLOT and in that case it’s important that both packages have different COUNTER metadata.

Parameters

  • myroot – ignored, self._eroot is used instead

  • mycpv – ignored

Return type

int

Returns

new counter value

cp_all(use_cache=1, sort=False)

Implement this in a child class Args

sort - return sorted results

Returns:

A list of strings 1 per CP in the datastore

cp_list(mycp, use_cache=1)
cpv_all(use_cache=1)

Set use_cache=0 to bypass the portage.cachedir() cache in cases when the accuracy of mtime staleness checks should not be trusted (generally this is only necessary in critical sections that involve merge or unmerge of packages).

cpv_counter(mycpv)

This method will grab the COUNTER. Returns a counter value.

cpv_exists(mykey, myrepo=None)

Tells us whether an actual ebuild exists on disk (no masking)

cpv_inject(mycpv)

injects a real package into our on-disk database; assumes mycpv is valid and doesn’t already exist

findname(mycpv, myrepo=None)
flush_cache()

If the current user has permission and the internal aux_get cache has been updated, save it to disk and mark it unmodified. This is called by emerge after it has loaded the full vdb for use in dependency calculations. Currently, the cache is only written if the user has superuser privileges (since that’s required to obtain a lock), but all users have read access and benefit from faster metadata lookups (as long as at least part of the cache is still valid).

get_counter_tick_core(myroot=None, mycpv=None)

Use this method to retrieve the counter instead of having to trust the value of a global counter file that can lead to invalid COUNTER generation. When cache is valid, the package COUNTER files are not read and we rely on the timestamp of the package directory to validate cache. The stat calls should only take a short time, so performance is sufficient without having to rely on a potentially corrupt global counter file.

The global counter file located at $CACHE_PATH/counter serves to record the counter of the last installed package and it also corresponds to the total number of installation actions that have occurred in the history of this package database.

Parameters

myroot – ignored, self._eroot is used instead

getpath(mykey, filename=None)
invalidentry(mypath)
isInjected(mycpv)
lock()

Acquire a reentrant lock, blocking, for cooperation with concurrent processes. State is inherited by subprocesses, allowing subprocesses to reenter a lock that was acquired by a parent process. However, a lock can be released only by the same process that acquired it.

match(origdep, use_cache=1)

caching match function

move_ent(mylist, repo_match=None)
move_slot_ent(mylist, repo_match=None)

This function takes a sequence: Args:

mylist: a sequence of (atom, originalslot, newslot) repo_match: callable that takes single repo_name argument

and returns True if the update should be applied

Returns:

The number of slotmoves this function did

removeFromContents(pkg, paths, relative_paths=True)
Parameters

  • pkg (string) – cpv for an installed package

  • paths (iterable) – paths of files to remove from contents

property root
unlock()

Release a lock, decrementing the recursion level. Each unlock() call must be matched with a prior lock() call, or else an AssertionError will be raised if unlock() is called while not locked.

async unpack_contents(pkg, dest_dir, include_config=None, include_unmodified_config=None, loop=None)

Unpack package contents to a directory. This method is a coroutine.

This copies files from the installed system, in the same way as the quickpkg(1) command. Default behavior for handling of protected configuration files is controlled by the QUICKPKG_DEFAULT_OPTS variable. The relevant quickpkg options are –include-config and –include-unmodified-config. When a configuration file is not included because it is protected, an ewarn message is logged.

Parameters

  • pkg (_pkg_str or portage.config) – package to unpack

  • dest_dir (str) – destination directory

  • include_config (bool) – Include all files protected by CONFIG_PROTECT (as a security precaution, default is False unless modified by QUICKPKG_DEFAULT_OPTS).

  • include_unmodified_config (bool) – Include files protected by CONFIG_PROTECT that have not been modified since installation (as a security precaution, default is False unless modified by QUICKPKG_DEFAULT_OPTS).

async unpack_metadata(pkg, dest_dir, loop=None)

Unpack package metadata to a directory. This method is a coroutine.

Parameters

  • pkg (_pkg_str or portage.config) – package to unpack

  • dest_dir (str) – destination directory

update_ents(updates, onProgress=None, onUpdate=None)

Update metadata of all packages for package moves. :param updates: A list of move commands, or dict of {repo_name: list} :type updates: list or dict :param onProgress: A progress callback function :type onProgress: a callable that takes 2 integer arguments: maxval and curval :param onUpdate: A progress callback function called only

for packages that are modified by updates.

property writable

Check if var/db/pkg is writable, or permissions are sufficient to create it if it does not exist yet. :rtype: bool :return: True if var/db/pkg is writable or can be created,

False otherwise

writeContentsToContentsFile(pkg, new_contents, new_needed=None)
Parameters

  • pkg (dblink) – package to write contents file for

  • new_contents (contents dictionary of the form {u'/path/to/file' : (contents_attribute 1, ...), ...}) – contents to write to CONTENTS file

  • new_needed (list of NeededEntry) – new NEEDED entries

class portage.dbapi.vartree.vartree(root=None, virtual=<class 'DeprecationWarning'>, categories=None, settings=None)

Bases: object

this tree will scan a var/db/pkg database located at root (passed to init)

dep_bestmatch(mydep, use_cache=1)

compatibility method – all matches, not just visible ones

dep_match(mydep, use_cache=1)

compatibility method – we want to see all matches, not just visible ones

exists_specific(cpv)
get_all_provides()
get_provide(mycpv)
getallcpv()

temporary function, probably to be renamed — Gets a list of all category/package-versions installed on the system.

getallnodes()

new behavior: these are all unmasked nodes. There may or may not be available masked package for nodes in this nodes list.

getebuildpath(fullpackage)
getpath(mykey, filename=None)
getslot(mycatpkg)

Get a slot for a catpkg; assume it exists.

inject(mycpv)
populate()
property root
zap(mycpv)
portage.dbapi.vartree.write_contents(contents, root, f)

Write contents to any file like object. The file will be left open.