portage.dbapi.porttree module

class portage.dbapi.porttree.FetchlistDict(pkgdir, settings, mydbapi)

Bases: portage.cache.mappings.Mapping

This provide a mapping interface to retrieve fetch lists. It’s used to allow portage.manifest.Manifest to access fetch lists via a standard mapping interface rather than use the dbapi directly.

get(key, default=None)

has_key(pkg_key)

Returns true if the given package exists within pkgdir.

items()

iteritems()

iterkeys()

itervalues()

keys()

values()

portage.dbapi.porttree._async_manifest_fetchlist(portdb, repo_config, cp, cpv_list=None, max_jobs=None, max_load=None, loop=None)

Asynchronous form of FetchlistDict, with max_jobs and max_load parameters in order to control async_aux_get concurrency.

Parameters

• portdb (portdbapi) – portdbapi instance

• repo_config (RepoConfig) – repository configuration for a Manifest

• cp (str) – cp for a Manifest

• cpv_list (list) – list of ebuild cpv values for a Manifest

• max_jobs (int) – max number of futures to process concurrently (default is portage.util.cpuinfo.get_cpu_count())

• max_load (int or float) – max load allowed when scheduling a new future, otherwise schedule no more than 1 future at a time (default is portage.util.cpuinfo.get_cpu_count())

• loop (EventLoop) – event loop

Returns

a Future resulting in a Mapping compatible with FetchlistDict

Return type

asyncio.Future (or compatible)

class portage.dbapi.porttree._better_cache(repositories)

Bases: object

The purpose of better_cache is to locate catpkgs in repositories using os.listdir() as much as possible, which is less expensive IO-wise than exhaustively doing a stat on each repo for a particular catpkg. better_cache stores a list of repos in which particular catpkgs appear. Various dbapi methods use better_cache to locate repositories of interest related to particular catpkg rather than performing an exhaustive scan of all repos/overlays.

Better_cache.items data may look like this:

{ "sys-apps/portage" : [ repo1, repo2 ] }

Without better_cache, Portage will get slower and slower (due to excessive IO) as more overlays are added.

Also note that it is OK if this cache has some ‘false positive’ catpkgs in it. We use it to search for specific catpkgs listed in ebuilds. The likelihood of a false positive catpkg in our cache causing a problem is extremely low, because the user of our cache is passing us a catpkg that came from somewhere and has already undergone some validation, and even then will further interrogate the short-list of repos we return to gather more information on the catpkg.

Thus, the code below is optimized for speed rather than painstaking correctness. I have added a note to dbapi.getRepositories() to ensure that developers are aware of this just in case.

The better_cache has been redesigned to perform on-demand scans – it will only scan a category at a time, as needed. This should further optimize IO performance by not scanning category directories that are not needed by Portage.

_scan_cat(cat)

class portage.dbapi.porttree._dummy_list(iterable=(), /)

Bases: list

append(object, /)

Append object to the end of the list.

clear()

Remove all items from list.

copy()

Return a shallow copy of the list.

count(value, /)

Return number of occurrences of value.

extend(iterable, /)

Extend list by appending elements from the iterable.

index(value, start=0, stop=9223372036854775807, /)

Return first index of value.

Raises ValueError if the value is not present.

insert(index, object, /)

Insert object before index.

pop(index=- 1, /)

Remove and return item at index (default last).

Raises IndexError if list is empty or index is out of range.

remove(item)

Remove first occurrence of value.

Raises ValueError if the value is not present.

reverse()

Reverse IN PLACE.

sort(*, key=None, reverse=False)

Sort the list in ascending order and return None.

The sort is in-place (i.e. the list itself is modified) and stable (i.e. the order of two equal elements is maintained).

If a key function is given, apply it once to each list item and sort them, ascending or descending, according to their function values.

The reverse flag can be set to sort in descending order.

portage.dbapi.porttree._parse_uri_map(cpv, metadata, use=None)

portage.dbapi.porttree.close_portdbapi_caches()

class portage.dbapi.porttree.portagetree(root=<class 'DeprecationWarning'>, virtual=<class 'DeprecationWarning'>, settings=None)

Bases: object

dep_bestmatch(mydep)

compatibility method

dep_match(mydep)

compatibility method

exists_specific(cpv)

getallnodes()

new behavior: these are all unmasked nodes. There may or may not be available masked package for nodes in this nodes list.

getname(pkgname)

Deprecated. Use the portdbapi findname method instead.

getslot(mycatpkg)

Get a slot for a catpkg; assume it exists.

property portroot

Deprecated. Use the portdbapi getRepositoryPath method instead.

property root

property virtual

class portage.dbapi.porttree.portdbapi(_unused_param=<class 'DeprecationWarning'>, mysettings=None)

Bases: portage.dbapi.dbapi

this tree will scan a portage directory located at root (passed to init)

static _aux_get_cancel(proc, future)

_aux_get_return(future, mycpv, mylist, myebuild, ebuild_hash, mydata, mylocation, cache_me, proc)

property _categories

_category_re = re.compile('^\\w[-.+\\w]*$')

static _cmp_cpv(cpv1, cpv2)

static _cpv_sort_ascending(cpv_list)

Use this to sort self.cp_list() results in ascending order. It sorts in place and returns None.

_create_pregen_cache(tree)

property _event_loop

_get_porttrees()

_init_cache_dirs()

Create /var/cache/edb/dep and adjust permissions for the portage group.

_iter_match(atom, cpv_iter)

_iter_match_repo(atom, cpv_iter)

_iter_match_slot(atom, cpv_iter)

_iter_match_use(atom, cpv_iter)

1. Check for required IUSE intersection (need implicit IUSE here).

2. Check enabled/disabled flag states.

_iter_visible(cpv_iter, myrepo=None)

Return a new list containing only visible packages.

_iuse_implicit_cnstr(pkg, metadata)

Construct a callable that checks if a given USE flag should be considered to be a member of the implicit IUSE for the given package.

Parameters

• pkg (_pkg_str) – package

• metadata (Mapping) – package metadata

Returns

a callable that accepts a single USE flag argument, and returns True only if the USE flag should be considered to be a member of the implicit IUSE for the given package.

Return type

callable

_known_keys = frozenset({'BDEPEND', 'DEFINED_PHASES', 'DEPEND', 'DESCRIPTION', 'EAPI', 'HOMEPAGE', 'IDEPEND', 'INHERIT', 'INHERITED', 'IUSE', 'KEYWORDS', 'LICENSE', 'PDEPEND', 'PROPERTIES', 'RDEPEND', 'REQUIRED_USE', 'RESTRICT', 'SLOT', 'SRC_URI'})

_match_use(atom, pkg, metadata, ignore_profile=False)

_pkg_str(cpv, repo)

This is used to contruct _pkg_str instances on-demand during matching. If cpv is a _pkg_str instance with slot attribute, then simply return it. Otherwise, fetch metadata and construct a _pkg_str instance. This may raise KeyError or InvalidData.

_pkg_str_aux_keys = ('EAPI', 'KEYWORDS', 'SLOT', 'repository')

_pull_valid_cache(cpv, ebuild_path, repo_path)

_set_porttrees(porttrees)

Consumers, such as emirrordist, may modify the porttrees attribute in order to modify the effective set of repositories for all portdbapi operations.

Parameters

porttrees (list) – list of repo locations, in ascending order by repo priority

_use_mutable = True

_visible(cpv, metadata)

_write_cache(cpv, repo_path, metadata, ebuild_hash)

async_aux_get(mycpv, mylist, mytree=None, myrepo=None, loop=None)

Asynchronous form form of aux_get.

Parameters

• mycpv (str) – cpv for an ebuild

• mylist (list) – list of metadata keys

• mytree (str) – The canonical path of the tree in which the ebuild is located, or None for automatic lookup

• myrepo (str) – name of the repo in which the ebuild is located, or None for automatic lookup

• loop (EventLoop) – event loop (defaults to global event loop)

Returns

list of metadata values

Return type

asyncio.Future (or compatible)

async_fetch_map(mypkg, useflags=None, mytree=None, loop=None)

Asynchronous form of getFetchMap.

Parameters

• mypkg (String) – cpv for an ebuild

• useflags (set, or None to enable all conditionals) – a collection of enabled USE flags, for evaluation of conditionals

• mytree – The canonical path of the tree in which the ebuild is located, or None for automatic lookup

• loop (EventLoop) – event loop (defaults to global event loop)

Returns

A future that results in a dict which maps each file name to a set of alternative URIs.

Return type

asyncio.Future (or compatible)

async async_xmatch(level, origdep, loop=None)

Asynchronous form of xmatch.

Parameters

• level (str) – xmatch level (bestmatch-visible, match-all-cpv-only match-allmatch-visible, minimum-all, minimum-all-ignore-profile, or minimum-visible)

• origdep (portage.dep.Atom or str) – dependency to match (may omit category)

• loop (EventLoop) – event loop (defaults to global event loop)

Returns

match result(s)

Return type

asyncio.Future (or compatible), which results in a _pkg_str or list of _pkg_str (depends on level)

aux_get(mycpv, mylist, mytree=None, myrepo=None)

stub code for returning auxilliary db information, such as SLOT, DEPEND, etc.

aux_update(cpv, metadata_updates)

Args:

cpv - “sys-apps/foo-1.0”

metadata_updates = { key : newvalue }

Returns:

None

property categories

Use self.cp_all() to generate a category list. Mutable instances can delete the self._categories attribute in cases when the cached categories become invalid and need to be regenerated.

close_caches()

cp_all(categories=None, trees=None, reverse=False, sort=True)

This returns a list of all keys in our tree or trees :param categories: optional list of categories to search or

defaults to self.settings.categories

Parameters

• trees – optional list of trees to search the categories in or defaults to self.porttrees

• reverse – reverse sort order (default is False)

• sort – return sorted results (default is True)

:rtype list of [cat/pkg,…]

cp_list(mycp, use_cache=1, mytree=None)

cpv_all()

Return all CPVs in the db Args:

None

Returns:

A list of Strings, 1 per CPV

This function relies on a subclass implementing cp_all, this is why the hasattr is there

cpv_exists(mykey, myrepo=None)

Tells us whether an actual ebuild exists on disk (no masking)

property eclassdb

fetch_check(mypkg, useflags=None, mysettings=None, all=False, myrepo=None)

TODO: account for PORTAGE_RO_DISTDIRS

findLicensePath(license_name)

findname(mycpv, mytree=None, myrepo=None)

findname2(mycpv, mytree=None, myrepo=None)

Returns the location of the CPV, and what overlay it was in. Searches overlays first, then PORTDIR; this allows us to return the first matching file. As opposed to starting in portdir and then doing overlays second, we would have to exhaustively search the overlays until we found the file we wanted. If myrepo is not None it will find packages from this repository(overlay)

flush_cache()

freeze()

getFetchMap(mypkg, useflags=None, mytree=None)

Get the SRC_URI metadata as a dict which maps each file name to a set of alternative URIs.

Parameters

• mypkg (String) – cpv for an ebuild

• useflags (set, or None to enable all conditionals) – a collection of enabled USE flags, for evaluation of conditionals

• mytree – The canonical path of the tree in which the ebuild is located, or None for automatic lookup

Returns

A dict which maps each file name to a set of alternative URIs.

Return type

dict

getIgnoredRepos()

Returns a list of repository paths that have been ignored, because another repo with the same name exists.

getMissingRepoNames()

Returns a list of repository paths that lack profiles/repo_name.

getRepositories(catpkg=None)

With catpkg=None, this will return a complete list of repositories in this dbapi. With catpkg set to a value, this method will return a short-list of repositories that contain this catpkg. Use this second approach if possible, to avoid exhaustively searching all repos for a particular catpkg. It’s faster for this method to find the catpkg than for you do it yourself. When specifying catpkg, you should have reasonable assurance that the category is valid and PMS-compliant as the caching mechanism we use does not perform validation checks for categories.

This function is required for GLEP 42 compliance.

Parameters

catpkg – catpkg for which we want a list of repositories; we’ll get a list of all repos containing this catpkg; if None, return a list of all Repositories that contain a particular catpkg.

Returns

a list of repositories.

getRepositoryName(canonical_repo_path)

This is the inverse of getRepositoryPath(). :param canonical_repo_path: the canonical path of a repository, as

resolved by os.path.realpath()

Returns

The repo_name for the corresponding repository, or None if the path does not correspond a known repository

Return type

String or None

getRepositoryPath(repository_id)

This function is required for GLEP 42 compliance; given a valid repository ID it must return a path to the repository TreeMap = { id:path }

getfetchsizes(mypkg, useflags=None, debug=0, myrepo=None)

gvisible(mylist)

invalidentry(mypath)

match(mydep, use_cache=1)

Given a dependency, try to find packages that match Args:

origdep - Depend atom use_cache - Boolean indicating if we should use the cache or not NOTE: Do we ever not want the cache?

Returns:

a list of packages that match origdep

melt()

move_slot_ent(mylist, repo_match=None)

This function takes a sequence: Args:

mylist: a sequence of (atom, originalslot, newslot) repo_match: callable that takes single repo_name argument

and returns True if the update should be applied

Returns:

The number of slotmoves this function did

portdbapi_instances = []

property porttree_root

property porttrees

update_ents(updates, onProgress=None, onUpdate=None)

Update metadata of all packages for package moves. :param updates: A list of move commands, or dict of {repo_name: list} :type updates: list or dict :param onProgress: A progress callback function :type onProgress: a callable that takes 2 integer arguments: maxval and curval :param onUpdate: A progress callback function called only

for packages that are modified by updates.

visible(cpv_iter)

xmatch(level, origdep, mydep=<class 'DeprecationWarning'>, mykey=<class 'DeprecationWarning'>, mylist=<class 'DeprecationWarning'>)

Caching match function.

Parameters

• level (str) – xmatch level (bestmatch-visible, match-all-cpv-only match-allmatch-visible, minimum-all, minimum-all-ignore-profile, or minimum-visible)

• origdep (portage.dep.Atom or str) – dependency to match (may omit category)

Returns

match result(s)

Return type

_pkg_str or list of _pkg_str (depends on level)