_emerge.Scheduler module

class _emerge.Scheduler.Scheduler(settings, trees, mtimedb, myopts, spinner, mergelist=None, favorites=None, graph_config=None)

Bases: _emerge.PollScheduler.PollScheduler

class _ConfigPool(root, allocate, deallocate)

Bases: object

Interface for a task to temporarily allocate a config instance from a pool. This allows a task to be constructed long before the config instance actually becomes needed, like when prefetchers are constructed for the whole merge list.

_allocate

_deallocate

_root

allocate()

deallocate(settings)

_add_packages()

_add_prefetchers()

_allocate_config(root)

Allocate a unique config instance for a task in order to prevent interference between parallel tasks.

_background_mode()

Check if background mode is enabled and adjust states as necessary.

Return type

bool

Returns

True if background mode is enabled, False otherwise.

class _binpkg_opts_class(**kwargs)

Bases: portage.util.SlotObject.SlotObject

copy()

Create a new instance and copy all attributes defined from __slots__ (including those from inherited classes).

fetchonly

getbinpkg

pretend

_build_exit(build)

class _build_opts_class(**kwargs)

Bases: portage.util.SlotObject.SlotObject

buildpkg

buildpkg_exclude

buildpkgonly

copy()

Create a new instance and copy all attributes defined from __slots__ (including those from inherited classes).

fetch_all_uri

fetchonly

pretend

_calc_resume_list()

Use the current resume list to calculate a new one, dropping any packages with unsatisfied deps. :rtype: bool :return: True if successful, False otherwise.

_can_add_job()

_check_manifests()

_choose_pkg()

Choose a task that has all its dependencies satisfied. This is used for parallel build scheduling, and ensures that we don’t build anything with deep dependencies that have yet to be merged.

_cleanup()

Cleanup any callbacks that have been registered with the global event loop.

_create_prefetcher(pkg)

Returns

a prefetcher, or None if not applicable

_deallocate_config(settings)

_dependent_on_scheduled_merges(pkg, later)

Traverse the subgraph of the given packages deep dependencies to see if it contains any scheduled merges. :param pkg: a package to check dependencies for :type pkg: Package :param later: packages for which dependence should be ignored

since they will be merged later than pkg anyway and therefore delaying the merge of pkg will not result in a more optimal merge order

Return type

bool

Returns

True if the package is dependent, False otherwise.

_destroy_graph()

Use this to free memory at the beginning of _calc_resume_list(). After _calc_resume_list(), the _init_graph() method must to be called in order to re-generate the structures that this method destroys.

_do_merge_exit(merge)

_elog_listener(mysettings, key, logentries, fulltext)

class _emerge_log_class(**kwargs)

Bases: portage.util.SlotObject.SlotObject

copy()

Create a new instance and copy all attributes defined from __slots__ (including those from inherited classes).

log(*pargs, **kwargs)

xterm_titles

_extract_exit(build)

class _failed_pkg(**kwargs)

Bases: portage.util.SlotObject.SlotObject

build_dir

build_log

copy()

Create a new instance and copy all attributes defined from __slots__ (including those from inherited classes).

pkg

postinst_failure

returncode

_failed_pkg_msg(failed_pkg, action, preposition)

class _fetch_iface_class(**kwargs)

Bases: portage.util.SlotObject.SlotObject

copy()

Create a new instance and copy all attributes defined from __slots__ (including those from inherited classes).

log_file

schedule

_find_blockers(new_pkg)

Returns a callable.

_find_blockers_impl(new_pkg)

_find_system_deps()

Find system packages and their deep runtime dependencies. Before being merged, these packages go to merge_wait_queue, to be merged when no other packages are building. NOTE: This can only find deep system deps if the system set has been added to the graph and traversed deeply (the depgraph “complete” parameter will do this, triggered by emerge –complete-graph option).

_generate_digests()

Generate digests if necessary for –digests or FEATURES=digest. In order to avoid interference, this must done before parallel tasks are started.

_get_interactive_tasks()

_get_prefetcher(pkg)

_handle_self_update()

class _iface_class(event_loop, is_background=None, **kwargs)

Bases: portage.util._async.SchedulerInterface.SchedulerInterface

_asyncio_child_watcher

_asyncio_wrapper

_event_loop

_event_loop_attrs = ('add_reader', 'add_writer', 'call_at', 'call_exception_handler', 'call_later', 'call_soon', 'call_soon_threadsafe', 'close', 'create_future', 'default_exception_handler', 'get_debug', 'is_closed', 'is_running', 'remove_reader', 'remove_writer', 'run_in_executor', 'run_until_complete', 'set_debug', 'time', '_asyncio_child_watcher', '_asyncio_wrapper')

_is_background

static _return_false()

add_reader

add_writer

async async_output(msg, log_file=None, background=None, level=0, noiselevel=- 1)

Output a msg to stdio (if not in background) and to a log file if provided.

Parameters

• msg (str) – a message string, including newline if appropriate

• log_file (file) – log file in binary mode

• background (bool) – send messages only to log (not to stdio)

• level (int) – a numeric logging level (see the logging module)

• noiselevel (int) – passed directly to writemsg

call_at

call_exception_handler

call_later

call_soon

call_soon_threadsafe

close

copy()

Create a new instance and copy all attributes defined from __slots__ (including those from inherited classes).

create_future

default_exception_handler

fetch

get_debug

is_closed

is_running

output(msg, log_path=None, background=None, level=0, noiselevel=- 1)

Output msg to stdout if not self._is_background(). If log_path is not None then append msg to the log (appends with compression if the filename extension of log_path corresponds to a supported compression type).

remove_reader

remove_writer

run_in_executor

run_until_complete

scheduleSetup

scheduleUnpack

set_debug

time

_init_graph(graph_config)

Initialization structures used for dependency calculations involving currently installed packages.

_is_background()

_is_work_scheduled()

_job_delay()

Return type

bool

Returns

True if job scheduling should be delayed, False otherwise.

_keep_scheduling()

Return type

bool

Returns

True if there may be remaining tasks to schedule, False otherwise.

_loadavg_latency = 30

_locate_failure_log(failed_pkg)

_main_loop()

_main_loop_cleanup()

_max_display_latency = 3

_merge()

_merge_exit(merge)

_merge_wait_exit_handler(task)

_opts_ignore_blockers = frozenset({'--buildpkgonly', '--fetch-all-uri', '--fetchonly', '--nodeps', '--pretend'})

_opts_no_background = frozenset({'--fetch-all-uri', '--fetchonly', '--pretend'})

_opts_no_self_update = frozenset({'--buildpkgonly', '--fetch-all-uri', '--fetchonly', '--pretend'})

_pkg(cpv, type_name, root_config, installed=False, operation=None, myrepo=None)

Get a package instance from the cache, or create a new one if necessary. Raises KeyError from aux_get if it failures for some reason (package does not exist or is corrupt).

class _pkg_count_class(**kwargs)

Bases: portage.util.SlotObject.SlotObject

copy()

Create a new instance and copy all attributes defined from __slots__ (including those from inherited classes).

curval

maxval

exception _pkg_failure(*pargs)

Bases: portage.exception.PortageException

An instance of this class is raised by unmerge() when an uninstallation fails.

args

status = 1

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

_prevent_builddir_collisions()

When building stages, sometimes the same exact cpv needs to be merged to both $ROOTs. Add edges to the digraph in order to avoid collisions in the builddir. Currently, normal file locks would be inappropriate for this purpose since emerge holds all of it’s build dir locks from the main process.

_prune_digraph()

Prune any root nodes that are irrelevant.

_record_pkg_failure(pkg, settings, ret)

Record a package failure. This eliminates the package from the –keep-going merge list, and immediately calls _failed_pkg_msg if we have not been terminated.

async _run_pkg_pretend(loop=None)

Since pkg_pretend output may be important, this method sends all output directly to stdout (regardless of options like –quiet or –jobs).

_running_job_count()

_save_resume_list()

Do this before verifying the ebuild Manifests since it might be possible for the user to use –resume –skipfirst get past a non-essential package with a broken digest.

_schedule()

Calls _schedule_tasks() and automatically returns early from any recursive calls to this method that the _schedule_tasks() call might trigger. This makes _schedule() safe to call from inside exit listeners. This method always returns True, so that it may be scheduled continuously via EventLoop.timeout_add().

_schedule_fetch(fetcher, force_queue=False)

Schedule a fetcher, in order to control the number of concurrent fetchers. If self._max_jobs is greater than 1 then the fetch queue is bypassed and the fetcher is started immediately, otherwise it is added to the front of the parallel-fetch queue. NOTE: The parallel-fetch queue is currently used to serialize access to the parallel-fetch log, so changes in the log handling would be required before it would be possible to enable concurrent fetching within the parallel-fetch queue.

_schedule_merge_wakeup(future)

_schedule_setup(setup_phase)

Schedule a setup phase on the merge queue, in order to serialize unsandboxed access to the live filesystem.

_schedule_tasks()

This is called from inside the _schedule() method, which guarantees the following:

1. It will not be called recursively.

2. _terminate_tasks() will not be called while it is running.

3. The state of the boolean _terminated_tasks variable will not change while it is running.

Unless this method is used to perform user interface updates, or something like that, the first thing it should do is check the state of _terminated_tasks and if that is True then it should return immediately (since there’s no need to schedule anything after _terminate_tasks() has been called).

_schedule_tasks_imp()

Return type

bool

Returns

True if state changed, False otherwise.

_schedule_unpack(unpack_phase)

Schedule an unpack phase on the unpack queue, in order to serialize $DISTDIR access for live ebuilds.

_set_graph_config(graph_config)

_set_max_jobs(max_jobs)

_show_list()

_sigcont_handler(signum, frame)

_status_msg(msg)

Display a brief status message (no newlines) in the status display. This is called by tasks to provide feedback to the user. This delegates the resposibility of generating

and control characters,

to guarantee that lines are created or erased when necessary and appropriate.

type msg

str

param msg

a brief status message (no newlines allowed)

_system_merge_started(merge)

Add any unsatisfied runtime deps to self._unsatisfied_system_deps. In general, this keeps track of installed system packages with unsatisfied RDEPEND or PDEPEND (circular dependencies). It can be a fragile situation, so we don’t execute any unrelated builds until the circular dependencies are built and installed.

_task(pkg)

_task_complete(pkg)

_task_queues_class

alias of portage.cache.mappings.slot_dict_class.<locals>.SlotDict

_terminate_tasks()

Send signals to terminate all tasks. This is called once from _keep_scheduling() or _is_work_scheduled() in the event dispatching thread. It will not be called while the _schedule_tasks() implementation is running, in order to avoid potential interference. All tasks should be cleaned up at the earliest opportunity, but not necessarily before this method returns. Typically, this method will send kill signals and return without waiting for exit status. This allows basic cleanup to occur, such as flushing of buffered output to logs.

_termination_check(retry=False)

Calls _terminate_tasks() if appropriate. It’s guaranteed not to call it while _schedule_tasks() is being called. This method must only be called via the event loop thread.

Parameters

retry (bool) – If True then reschedule if scheduling state prevents immediate termination.

exception _unknown_internal_error(value='')

Bases: portage.exception.PortageException

Used internally to terminate scheduling. The specific reason for the failure should have been dumped to stderr.

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

_world_atom(pkg)

Add or remove the package to the world file, but only if it’s supposed to be added or removed. Otherwise, do nothing.

merge()

terminate()

Schedules asynchronous, graceful termination of the scheduler at the earliest opportunity.

This method is thread-safe (and safe for signal handlers).