_emerge.EbuildFetcher module¶
- class _emerge.EbuildFetcher.EbuildFetcher(**kwargs)¶
Bases:
_emerge.CompositeTask.CompositeTask
- _TASK_QUEUED = -1¶
- _assert_current(task)¶
Raises an AssertionError if the given task is not the same one as self._current_task. This can be useful for detecting bugs.
- _async_wait()¶
Subclasses call this method in order to invoke exit listeners when self.returncode is set. Subclasses may override this method in order to perform cleanup. The default implementation for this method simply calls self.wait(), which will immediately raise an InvalidStateError if the event loop is running and self.returncode is None.
- _cancel()¶
Subclasses should implement this, as a template method to be called by AsynchronousTask.cancel().
- _cancelled_returncode = -2¶
- _current_task¶
- _default_exit(task)¶
Calls _assert_current() on the given task and then sets the composite returncode attribute if task.returncode != os.EX_OK. If the task failed then self._current_task will be set to None. Subclasses can use this as a generic task exit callback.
- Return type
int
- Returns
The task.returncode attribute.
- _default_final_exit(task)¶
This calls _final_exit() and then wait().
Subclasses can use this as a generic final task exit callback.
- _exit_listener_cb(listener)¶
- _exit_listener_handles¶
- _exit_listeners¶
- _fetcher_proc¶
- _final_exit(task)¶
Assumes that task is the final task of this composite task. Calls _default_exit() and sets self.returncode to the task’s returncode and sets self._current_task to None.
- _poll()¶
This does a loop calling self._current_task.poll() repeatedly as long as the value of self._current_task keeps changing. It calls poll() a maximum of one time for a given self._current_task instance. This is useful since calling poll() on a task can trigger advance to the next task could eventually lead to the returncode being set in cases when polling only a single task would not have the same effect.
- _start()¶
- _start_fetch(uri_map_task)¶
- _start_hook()¶
- _start_listeners¶
- _start_task(task, exit_handler)¶
Register exit handler for the given task, set it as self._current_task, and call task.start().
Subclasses can use this as a generic way to start a task.
- _start_with_metadata(aux_get_task)¶
- _task_queued(task)¶
- _task_queued_start_handler(task)¶
- _task_queued_wait()¶
- _wait_hook()¶
Call this method after the task completes, just before returning the returncode from wait() or poll(). This hook is used to trigger exit listeners when the returncode first becomes available.
- _was_cancelled()¶
If cancelled, set returncode if necessary and return True. Otherwise, return False.
- addExitListener(f)¶
The function will be called with one argument, a reference to self.
- addStartListener(f)¶
The function will be called with one argument, a reference to self.
- async_already_fetched(settings)¶
Returns True if all files already exist locally and have correct digests, otherwise return False. When returning True, appropriate digest checking messages are produced for display and/or logging. When returning False, no messages are produced, since we assume that a fetcher process will later be executed in order to produce such messages. This will raise InvalidDependString if SRC_URI is invalid.
- async_wait()¶
Wait for returncode asynchronously. Notification is available via the add_done_callback method of the returned Future instance.
- Returns
Future, result is self.returncode
- background¶
- cancel()¶
Cancel the task, but do not wait for exit status. If asynchronous exit notification is desired, then use addExitListener to add a listener before calling this method. NOTE: Synchronous waiting for status is not supported, since it would be vulnerable to hitting the recursion limit when a large number of tasks need to be terminated simultaneously, like in bug #402335.
- cancelled¶
- config_pool¶
- copy()¶
Create a new instance and copy all attributes defined from __slots__ (including those from inherited classes).
- ebuild_path¶
- fetchall¶
- fetchonly¶
- isAlive()¶
- logfile¶
- pkg¶
- poll()¶
- prefetch¶
- removeExitListener(f)¶
- removeStartListener(f)¶
- returncode¶
- scheduler¶
- start()¶
Start an asynchronous task and then return as soon as possible.
- wait()¶
Wait for the returncode attribute to become ready, and return it. If the returncode is not ready and the event loop is already running, then the async_wait() method should be used instead of wait(), because wait() will raise asyncio.InvalidStateError in this case.
- Return type
int
- Returns
the value of self.returncode
- class _emerge.EbuildFetcher._EbuildFetcherProcess(**kwargs)¶
Bases:
portage.util._async.ForkProcess.ForkProcess
- _CGROUP_CLEANUP_RETRY_MAX = 8¶
- _async_uri_map()¶
This calls the portdbapi.async_fetch_map method and returns the resulting Future (may contain InvalidDependString exception).
- _async_wait()¶
Subclasses call this method in order to invoke exit listeners when self.returncode is set. Subclasses may override this method in order to perform cleanup. The default implementation for this method simply calls self.wait(), which will immediately raise an InvalidStateError if the event loop is running and self.returncode is None.
- _async_waitpid()¶
Wait for exit status of self.pid asynchronously, and then set the returncode, and finally notify exit listeners via the _async_wait method. Subclasses may override this method in order to implement an alternative means to retrieve pid exit status, or as a means to delay action until some pending task(s) have completed (such as reading data that the subprocess is supposed to have written to a pipe).
- _async_waitpid_cb(pid, returncode)¶
- _bootstrap(fd_pipes)¶
- _bufsize = 4096¶
- _can_log(slave_fd)¶
- _cancel()¶
Subclasses should implement this, as a template method to be called by AsynchronousTask.cancel().
- _cancel_timeout = 1¶
- _cancelled_returncode = -2¶
- _cgroup_cleanup()¶
- _check_already_fetched(settings, uri_map)¶
- _digests¶
- _dummy_pipe_fd¶
- _eerror(lines)¶
- _elog(elog_funcname, lines)¶
- _exit_listener_cb(listener)¶
- _exit_listener_handles¶
- _exit_listeners¶
- _files¶
- _get_digests()¶
- _get_ebuild_path()¶
- _get_manifest()¶
- async _main(build_logger, pipe_logger, loop=None)¶
- _main_cancel(build_logger, pipe_logger)¶
- _main_exit(main_task)¶
- _main_task¶
- _main_task_cancel¶
- _manifest¶
- _orphan_process_warn()¶
- _pipe(fd_pipes)¶
When appropriate, use a pty so that fetcher progress bars, like wget has, will work properly.
- _poll()¶
- _prefetch_size_ok(uri_map, settings, ebuild_path)¶
- _proc¶
- async _proc_join(proc, loop=None)¶
- _proc_join_done(proc, future)¶
Extend _proc_join_done to emit an eerror message for fetch failure.
- _proc_join_interval = 0.1¶
- _proc_join_task¶
- _read_array(f)¶
NOTE: array.fromfile() is used here only for testing purposes, because it has bugs in all known versions of Python (including Python 2.7 and Python 3.2). See PipeReaderArrayTestCase.
A benchmark that copies bytes from /dev/zero to /dev/null shows that arrays give a 15% performance improvement for Python 2.7.14. However, arrays significantly decrease performance for Python 3.
- _read_buf(fd)¶
Read self._bufsize into a string of bytes, handling EAGAIN and EIO. This will only call os.read() once, so the caller should call this method in a loop until either None or an empty string of bytes is returned. An empty string of bytes indicates EOF. None indicates EAGAIN.
- NOTE: os.read() will be called regardless of the event flags,
since otherwise data may be lost (see bug #531724).
- Parameters
fd (int) – file descriptor (non-blocking mode required)
- Return type
bytes or None
- Returns
A string of bytes, or None
- _registered¶
- _run()¶
- _selinux_type¶
- _settings¶
- _spawn(args, fd_pipes=None, **kwargs)¶
Override SpawnProcess._spawn to fork a subprocess that calls self._run(). This uses multiprocessing.Process in order to leverage any pre-fork and post-fork interpreter housekeeping that it provides, promoting a healthy state for the forked interpreter.
- _spawn_kwarg_names = ('env', 'opt_name', 'fd_pipes', 'uid', 'gid', 'groups', 'umask', 'logfile', 'path_lookup', 'pre_exec', 'close_fds', 'cgroup', 'unshare_ipc', 'unshare_mount', 'unshare_pid', 'unshare_net')¶
- _start()¶
- _start_hook()¶
- _start_listeners¶
- _unregister()¶
Unregister from the scheduler and close open files.
- _uri_map¶
- _wait_hook()¶
Call this method after the task completes, just before returning the returncode from wait() or poll(). This hook is used to trigger exit listeners when the returncode first becomes available.
- _wait_loop(timeout=None)¶
- _waitpid_id¶
- _was_cancelled()¶
If cancelled, set returncode if necessary and return True. Otherwise, return False.
- addExitListener(f)¶
The function will be called with one argument, a reference to self.
- addStartListener(f)¶
The function will be called with one argument, a reference to self.
- args¶
- async_already_fetched(settings)¶
- async_wait()¶
Wait for returncode asynchronously. Notification is available via the add_done_callback method of the returned Future instance.
- Returns
Future, result is self.returncode
- background¶
- cancel()¶
Cancel the task, but do not wait for exit status. If asynchronous exit notification is desired, then use addExitListener to add a listener before calling this method. NOTE: Synchronous waiting for status is not supported, since it would be vulnerable to hitting the recursion limit when a large number of tasks need to be terminated simultaneously, like in bug #402335.
- cancelled¶
- cgroup¶
- close_fds¶
- config_pool¶
- copy()¶
Create a new instance and copy all attributes defined from __slots__ (including those from inherited classes).
- ebuild_path¶
- env¶
- fd_pipes¶
- fetchall¶
- fetchonly¶
- gid¶
- groups¶
- isAlive()¶
- log_filter_file¶
- logfile¶
- opt_name¶
- path_lookup¶
- pid¶
- pkg¶
- poll()¶
- pre_exec¶
- prefetch¶
- removeExitListener(f)¶
- removeStartListener(f)¶
- returncode¶
- scheduler¶
- src_uri¶
- start()¶
Start an asynchronous task and then return as soon as possible.
- uid¶
- umask¶
- wait()¶
Wait for the returncode attribute to become ready, and return it. If the returncode is not ready and the event loop is already running, then the async_wait() method should be used instead of wait(), because wait() will raise asyncio.InvalidStateError in this case.
- Return type
int
- Returns
the value of self.returncode