Runner

Runner.py: Task scheduling and execution

waflib.Runner.GAP = 5

Wait for at least GAP * njobs before trying to enqueue more tasks to run

class waflib.Runner.Consumer(spawner, task)[source]

Bases: threading.Thread

Daemon thread object that executes a task. It shares a semaphore with the coordinator waflib.Runner.Spawner. There is one instance per task to consume.

task = None

Task to execute

spawner = None

Coordinator object

run()[source]

Processes a single task

class waflib.Runner.Spawner(master)[source]

Bases: threading.Thread

Daemon thread that consumes tasks from waflib.Runner.Parallel producer and spawns a consuming thread waflib.Runner.Consumer for each waflib.Task.Task instance.

master = None

waflib.Runner.Parallel producer instance

sem = None

Bounded semaphore that prevents spawning more than n concurrent consumers

run()[source]

Spawns new consumers to execute tasks by delegating to waflib.Runner.Spawner.loop()

loop()[source]

Consumes task objects from the producer; ends when the producer has no more task to provide.

class waflib.Runner.Parallel(bld, j=2)[source]

Bases: object

Schedule the tasks obtained from the build context for execution.

__init__(bld, j=2)[source]

The initialization requires a build context reference for computing the total number of jobs.

numjobs = None

Amount of parallel consumers to use

bld = None

Instance of waflib.Build.BuildContext

outstanding = None

Heap of waflib.Task.Task that may be ready to be executed

postponed = None

Heap of waflib.Task.Task which are not ready to run for non-DAG reasons

incomplete = None

List of waflib.Task.Task waiting for dependent tasks to complete (DAG)

ready = None

List of waflib.Task.Task ready to be executed by consumers

out = None

List of waflib.Task.Task returned by the task consumers

count = None

Amount of tasks that may be processed by waflib.Runner.TaskConsumer

processed = None

Amount of tasks processed

stop = None

Error flag to stop the build

error = None

Tasks that could not be executed

biter = None

Task iterator which must give groups of parallelizable tasks when calling next()

dirty = None

Flag that indicates that the build cache must be saved when a task was executed (calls waflib.Build.BuildContext.store())

revdeps = None

The reverse dependency graph of dependencies obtained from Task.run_after

spawner = None

Coordinating daemon thread that spawns thread consumers

get_next_task()[source]

Obtains the next Task instance to run

Return type:waflib.Task.Task
postpone(tsk)[source]

Adds the task to the list waflib.Runner.Parallel.postponed. The order is scrambled so as to consume as many tasks in parallel as possible.

Parameters:tsk (waflib.Task.Task) – task instance
refill_task_list()[source]

Pulls a next group of tasks to execute in waflib.Runner.Parallel.outstanding. Ensures that all tasks in the current build group are complete before processing the next one.

add_more_tasks(tsk)[source]

If a task provides waflib.Task.Task.more_tasks, then the tasks contained in that list are added to the current build and will be processed before the next build group.

The priorities for dependent tasks are not re-calculated globally

Parameters:tsk (waflib.Task.Task) – task instance
get_out()[source]

Waits for a Task that task consumers add to waflib.Runner.Parallel.out after execution. Adds more Tasks if necessary through waflib.Runner.Parallel.add_more_tasks.

Return type:waflib.Task.Task
add_task(tsk)[source]

Enqueue a Task to waflib.Runner.Parallel.ready so that consumers can run them.

Parameters:tsk (waflib.Task.Task) – task instance
process_task(tsk)[source]

Processes a task and attempts to stop the build in case of errors

skip(tsk)[source]

Mark a task as skipped/up-to-date

cancel(tsk)[source]

Mark a task as failed because of unsatisfiable dependencies

error_handler(tsk)[source]

Called when a task cannot be executed. The flag waflib.Runner.Parallel.stop is set, unless the build is executed with:

$ waf build -k
Parameters:tsk (waflib.Task.Task) – task instance
task_status(tsk)[source]

Obtains the task status to decide whether to run it immediately or not.

Returns:the exit status, for example waflib.Task.ASK_LATER
Return type:integer
start()[source]

Obtains Task instances from the BuildContext instance and adds the ones that need to be executed to waflib.Runner.Parallel.ready so that the waflib.Runner.Spawner consumer thread has them executed. Obtains the executed Tasks back from waflib.Runner.Parallel.out and marks the build as failed by setting the stop flag. If only one job is used, then executes the tasks one by one, without consumers.

prio_and_split(tasks)[source]

Label input tasks with priority values, and return a pair containing the tasks that are ready to run and the tasks that are necessarily waiting for other tasks to complete.

The priority system is really meant as an optional layer for optimization: dependency cycles are found quickly, and builds should be more efficient. A high priority number means that a task is processed first.

This method can be overridden to disable the priority system:

def prio_and_split(self, tasks):
        return tasks, []
Returns:A pair of task lists
Return type:tuple