Package portage :: Package util
[hide private]

Package util

source code

Submodules [hide private]

Classes [hide private]
  _getconfig_shlex
  cmp_sort_key
In python-3.0 the list.sort() method no longer has a "cmp" keyword argument.
  atomic_ofstream
Write a file atomically via os.rename().
  LazyItemsDict
A mapping object that behaves like a standard dict except that it allows for lazy initialization of values via callable objects.
  ConfigProtect
Functions [hide private]
 
initialize_logger(level=30)
Sets up basic logging of portage activities...
source code
 
writemsg(mystr, noiselevel=0, fd=None)
Prints out warning and debug messages based on the noiselimit setting
source code
 
writemsg_stdout(mystr, noiselevel=0)
Prints messages stdout based on the noiselimit setting
source code
 
writemsg_level(msg, level=0, noiselevel=0)
Show a message for the given level as defined by the logging module (default is 0).
source code
 
normalize_path(mypath)
os.path.normpath("//foo") returns "//foo" instead of "/foo" We dislike this behavior so we create our own normpath func to fix it.
source code
 
grabfile(myfilename, compat_level=0, recursive=0, remember_source_file=False)
This function grabs the lines in a file, normalizes whitespace and returns lines in a list; if a line begins with a #, it is ignored, as are empty lines
source code
 
map_dictlist_vals(func, myDict)
Performs a function on each value of each key in a dictlist.
source code
 
stack_dictlist(original_dicts, incremental=0, incrementals=[], ignore_none=0)
Stacks an array of dict-types into one array.
source code
 
stack_dicts(dicts, incremental=0, incrementals=[], ignore_none=0)
Stacks an array of dict-types into one array.
source code
 
append_repo(atom_list, repo_name, remember_source_file=False)
Takes a list of valid atoms without repo spec and appends ::repo_name.
source code
 
stack_lists(lists, incremental=1, remember_source_file=False, warn_for_unmatched_removal=False, strict_warn_for_unmatched_removal=False, ignore_repo=False)
Stacks an array of list-types into one array.
source code
 
grabdict(myfilename, juststrings=0, empty=0, recursive=0, incremental=1)
This function grabs the lines in a file, normalizes whitespace and returns lines in a dictionary
source code
 
read_corresponding_eapi_file(filename, default=u'0')
Read the 'eapi' file from the directory 'filename' is in.
source code
 
grabdict_package(myfilename, juststrings=0, recursive=0, allow_wildcard=False, allow_repo=False, verify_eapi=False, eapi=None)
Does the same thing as grabdict except it validates keys with isvalidatom()
source code
 
grabfile_package(myfilename, compatlevel=0, recursive=0, allow_wildcard=False, allow_repo=False, remember_source_file=False, verify_eapi=False, eapi=None) source code
 
_recursive_basename_filter(f) source code
 
_recursive_file_list(path) source code
 
grablines(myfilename, recursive=0, remember_source_file=False) source code
 
writedict(mydict, myfilename, writekey=True)
Writes out a dict to a file; writekey=0 mode doesn't write out the key and assumes all values are strings, not lists.
source code
 
shlex_split(s)
This is equivalent to shlex.split, but if the current interpreter is python2, it temporarily encodes unicode strings to bytes since python2's shlex.split() doesn't handle unicode strings.
source code
 
getconfig(mycfg, tolerant=False, allow_sourcing=False, expand=True, recursive=False) source code
 
varexpand(mystring, mydict=None, error_leader=None) source code
 
pickle_read(filename, default=None, debug=0) source code
 
dump_traceback(msg, noiselevel=1) source code
 
unique_array(s)
lifted from python cookbook, credit: Tim Peters Return a list of the elements in s in arbitrary order, sans duplicates
source code
 
unique_everseen(iterable, key=None)
List unique elements, preserving order.
source code
 
apply_permissions(filename, uid=-1, gid=-1, mode=-1, mask=-1, stat_cached=None, follow_links=True)
Apply user, group, and mode bits to a file if the existing bits do not already match.
source code
 
apply_stat_permissions(filename, newstat, **kwargs)
A wrapper around apply_secpass_permissions that gets uid, gid, and mode from a stat object
source code
 
apply_recursive_permissions(top, uid=-1, gid=-1, dirmode=-1, dirmask=-1, filemode=-1, filemask=-1, onerror=None)
A wrapper around apply_secpass_permissions that applies permissions recursively.
source code
 
apply_secpass_permissions(filename, uid=-1, gid=-1, mode=-1, mask=-1, stat_cached=None, follow_links=True)
A wrapper around apply_permissions that uses secpass and simple logic to apply as much of the permissions as possible without generating an obviously avoidable permission exception.
source code
 
write_atomic(file_path, content, **kwargs) source code
 
ensure_dirs(dir_path, **kwargs)
Create a directory and call apply_permissions.
source code
 
new_protect_filename(mydest, newmd5=None, force=False)
Resolves a config-protect filename for merging, optionally using the last filename if the md5 matches.
source code
 
find_updated_config_files(target_root, config_protect)
Return a tuple of configuration files that needs to be updated.
source code
 
getlibpaths(root, env=None) source code
Variables [hide private]
  noiselimit = 0
  _eapi_cache = {u'/mnt/storage/primary/var/portage/repos/gentoo...
  _invalid_var_name_re = re.compile(r'^\d|\W')
  _varexpand_word_chars = frozenset([u'0', u'1', u'2', u'3', u'4...
  _varexpand_unexpected_eof_msg = u'unexpected EOF while looking...
  pickle_write = None
hash(x)
  _ld_so_include_re = re.compile(r'^include\s+(\S.*)')
  __package__ = 'portage.util'

Imports: deepcopy, errno, io, filterfalse, logging, re, shlex, stat, string, sys, traceback, glob, portage, os, _encodings, _os_merge, _unicode_encode, _unicode_decode, VCS_DIRS, InvalidAtom, PortageException, FileNotFound, IsADirectory, OperationNotPermitted, ParseError, PermissionDenied, ReadOnlyFileSystem, _, ObjectProxy, UserDict, _unicode, Atom, ExtractKernelVersion, SlotObject, _ShelveUnicodeWrapper, _argparse, _async, _ctypes, _desktop_entry, _dyn_libs, _eventloop, _get_vm_info, _info_files, _path, _pty, _urlopen, digraph, env_update, iterators, lafilefixer, listdir, movefile, mtimedb, path, pickle, subprocess, writeable_check


Function Details [hide private]

initialize_logger(level=30)

source code 
Sets up basic logging of portage activities
Args:
        level: the level to emit messages at ('info', 'debug', 'warning' ...)
Returns:
        None

writemsg_level(msg, level=0, noiselevel=0)

source code 

Show a message for the given level as defined by the logging module (default is 0). When level >= logging.WARNING then the message is sent to stderr, otherwise it is sent to stdout. The noiselevel is passed directly to writemsg().

Parameters:
  • msg (str) - a message string, including newline if appropriate
  • level (int) - a numeric logging level (see the logging module)
  • noiselevel (int) - passed directly to writemsg

map_dictlist_vals(func, myDict)

source code 

Performs a function on each value of each key in a dictlist. Returns a new dictlist.

stack_dictlist(original_dicts, incremental=0, incrementals=[], ignore_none=0)

source code 

Stacks an array of dict-types into one array. Optionally merging or
overwriting matching key/value pairs for the dict[key]->list.
Returns a single dict. Higher index in lists is preferenced.

Example usage:
   >>> from portage.util import stack_dictlist
        >>> print stack_dictlist( [{'a':'b'},{'x':'y'}])
        >>> {'a':'b','x':'y'}
        >>> print stack_dictlist( [{'a':'b'},{'a':'c'}], incremental = True )
        >>> {'a':['b','c'] }
        >>> a = {'KEYWORDS':['x86','alpha']}
        >>> b = {'KEYWORDS':['-x86']}
        >>> print stack_dictlist( [a,b] )
        >>> { 'KEYWORDS':['x86','alpha','-x86']}
        >>> print stack_dictlist( [a,b], incremental=True)
        >>> { 'KEYWORDS':['alpha'] }
        >>> print stack_dictlist( [a,b], incrementals=['KEYWORDS'])
        >>> { 'KEYWORDS':['alpha'] }

@param original_dicts a list of (dictionary objects or None)
@type list
@param incremental True or false depending on whether new keys should overwrite
   keys which already exist.
@type boolean
@param incrementals A list of items that should be incremental (-foo removes foo from
   the returned dict).
@type list
@param ignore_none Appears to be ignored, but probably was used long long ago.
@type boolean

stack_dicts(dicts, incremental=0, incrementals=[], ignore_none=0)

source code 

Stacks an array of dict-types into one array. Optionally merging or overwriting matching key/value pairs for the dict[key]->string. Returns a single dict.

append_repo(atom_list, repo_name, remember_source_file=False)

source code 

Takes a list of valid atoms without repo spec and appends ::repo_name. If an atom already has a repo part, then it is preserved (see bug #461948).

stack_lists(lists, incremental=1, remember_source_file=False, warn_for_unmatched_removal=False, strict_warn_for_unmatched_removal=False, ignore_repo=False)

source code 

Stacks an array of list-types into one array. Optionally removing distinct values using '-value' notation. Higher index is preferenced.

all elements must be hashable.

grabdict(myfilename, juststrings=0, empty=0, recursive=0, incremental=1)

source code 

This function grabs the lines in a file, normalizes whitespace and returns lines in a dictionary

@param myfilename: file to process
@type myfilename: string (path)
@param juststrings: only return strings
@type juststrings: Boolean (integer)
@param empty: Ignore certain lines
@type empty: Boolean (integer)
@param recursive: Recursively grab ( support for /etc/portage/package.keywords/* and friends )
@type recursive: Boolean (integer)
@param incremental: Append to the return list, don't overwrite
@type incremental: Boolean (integer)
@rtype: Dictionary
@return:
1.  Returns the lines in a file in a dictionary, for example:
        'sys-apps/portage x86 amd64 ppc'
        would return
        {"sys-apps/portage" : ['x86', 'amd64', 'ppc']}

read_corresponding_eapi_file(filename, default=u'0')

source code 

Read the 'eapi' file from the directory 'filename' is in. Returns "0" if the file is not present or invalid.

unique_everseen(iterable, key=None)

source code 

List unique elements, preserving order. Remember all elements ever seen. Taken from itertools documentation.

apply_permissions(filename, uid=-1, gid=-1, mode=-1, mask=-1, stat_cached=None, follow_links=True)

source code 

Apply user, group, and mode bits to a file if the existing bits do not already match. The default behavior is to force an exact match of mode bits. When mask=0 is specified, mode bits on the target file are allowed to be a superset of the mode argument (via logical OR). When mask>0, the mode bits that the target file is allowed to have are restricted via logical XOR. Returns True if the permissions were modified and False otherwise.

apply_recursive_permissions(top, uid=-1, gid=-1, dirmode=-1, dirmask=-1, filemode=-1, filemask=-1, onerror=None)

source code 

A wrapper around apply_secpass_permissions that applies permissions recursively. If optional argument onerror is specified, it should be a function; it will be called with one argument, a PortageException instance. Returns True if all permissions are applied and False if some are left unapplied.

apply_secpass_permissions(filename, uid=-1, gid=-1, mode=-1, mask=-1, stat_cached=None, follow_links=True)

source code 

A wrapper around apply_permissions that uses secpass and simple logic to apply as much of the permissions as possible without generating an obviously avoidable permission exception. Despite attempts to avoid an exception, it's possible that one will be raised anyway, so be prepared. Returns True if all permissions are applied and False if some are left unapplied.

ensure_dirs(dir_path, **kwargs)

source code 

Create a directory and call apply_permissions. Returns True if a directory is created or the permissions needed to be modified, and False otherwise.

This function's handling of EEXIST errors makes it useful for atomic directory creation, in which multiple processes may be competing to create the same directory.

new_protect_filename(mydest, newmd5=None, force=False)

source code 

Resolves a config-protect filename for merging, optionally using the last filename if the md5 matches. If force is True, then a new filename will be generated even if mydest does not exist yet. (dest,md5) ==> 'string' --- path_to_target_filename (dest) ==> ('next', 'highest') --- next_target and most-recent_target

find_updated_config_files(target_root, config_protect)

source code 

Return a tuple of configuration files that needs to be updated.
The tuple contains lists organized like this:
        [protected_dir, file_list]
If the protected config isn't a protected_dir but a procted_file, list is:
        [protected_file, None]
If no configuration files needs to be updated, None is returned


Variables Details [hide private]

_eapi_cache

Value:
{u'/mnt/storage/primary/var/portage/repos/gentoo/profiles/base/eapi': \
u'5',
 u'/mnt/storage/primary/var/portage/repos/gentoo/profiles/releases/13.\
0/eapi': u'5',
 u'/mnt/storage/primary/var/portage/repos/zmedico-prefix-hybrid/profil\
es/eapi': u'5-progress',
 u'/usr/portage/profiles/eapi': u'5'}

_varexpand_word_chars

Value:
frozenset([u'0',
           u'1',
           u'2',
           u'3',
           u'4',
           u'5',
           u'6',
           u'7',
...

_varexpand_unexpected_eof_msg

Value:
u'unexpected EOF while looking for matching `}''