TaskGen

Task generators

The class waflib.TaskGen.task_gen encapsulates the creation of task objects (low-level code) The instances can have various parameters, but the creation of task nodes (Task.py) is deferred. To achieve this, various methods are called from the method “apply”

waflib.TaskGen.feats = {'*': set(['process_rule', 'process_source']), 'asm': set(['apply_incpaths', 'propagate_uselib_vars', 'apply_nasm_vars', 'apply_link']), 'c': set(['set_macosx_deployment_target', 'process_use', 'apply_incpaths', 'apply_link', 'apply_flags_msvc', 'propagate_uselib_vars']), 'cprogram': set(['set_full_paths_hpux', 'create_task_macapp', 'create_task_macplist', 'apply_manifest']), 'cs': set(['use_cs', 'propagate_uselib_vars', 'debug_cs', 'apply_cs', 'doc_cs']), 'cshlib': set(['apply_implib', 'apply_vnum', 'apply_manifest', 'apply_bundle']), 'cxx': set(['set_macosx_deployment_target', 'process_use', 'apply_incpaths', 'apply_link', 'apply_flags_msvc', 'propagate_uselib_vars']), 'cxxprogram': set(['set_full_paths_hpux', 'create_task_macapp', 'create_task_macplist', 'apply_manifest']), 'cxxshlib': set(['apply_implib', 'apply_vnum', 'apply_manifest', 'apply_bundle']), 'd': set(['process_use', 'apply_incpaths', 'propagate_uselib_vars', 'process_header', 'apply_link']), 'dshlib': set(['apply_vnum']), 'fake_lib': set(['process_lib']), 'fake_obj': set(['process_objs']), 'fc': set(['process_use', 'apply_incpaths', 'propagate_uselib_vars', 'apply_link', 'apply_flags_ifort']), 'fcprogram': set(['apply_manifest_ifort']), 'fcprogram_test': set(['apply_manifest_ifort']), 'fcshlib': set(['apply_implib', 'apply_vnum', 'apply_manifest_ifort']), 'glib2': set(['process_settings']), 'grep_for_endianness': set(['grep_for_endianness_fun']), 'gresource': set(['process_gresource_bundle']), 'includes': set(['apply_incpaths']), 'install_task': set(['process_install_task']), 'intltool_in': set(['apply_intltool_in_f']), 'intltool_po': set(['apply_intltool_po']), 'jar': set(['jar_files', 'use_jar_files']), 'javac': set(['apply_java', 'propagate_uselib_vars', 'use_javac_files', 'set_classpath']), 'javadoc': set(['create_javadoc']), 'link_lib_test': set(['link_lib_test_fun']), 'link_main_routines_func': set(['link_main_routines_tg_method']), 'perlext': set(['init_perlext']), 'py': set(['feature_py']), 'pyembed': set(['init_pyembed']), 'pyext': set(['set_lib_pat', 'set_bundle', 'init_pyext']), 'qt5': set(['apply_qt5', 'process_mocs']), 'rubyext': set(['init_rubyext', 'apply_ruby_so_name']), 'seq': set(['sequence_order']), 'skip_stlib_link_deps': set(['apply_skip_stlib_link_deps']), 'subst': set(['process_subst']), 'test': set(['make_test']), 'test_exec': set(['test_exec_fun']), 'test_scripts': set(['make_interpreted_test']), 'tex': set(['apply_tex']), 'use': set(['process_use']), 'uselib': set(['propagate_uselib_vars']), 'vnum': set(['apply_vnum']), 'winapp': set(['make_windows_app']), 'winphoneapp': set(['make_winphone_app'])}

remember the methods declaring features

class waflib.TaskGen.task_gen(*k, **kw)[source]

Bases: object

Instances of this class create waflib.Task.Task when calling the method waflib.TaskGen.task_gen.post() from the main thread. A few notes:

  • The methods to call (self.meths) can be specified dynamically (removing, adding, ..)
  • The ‘features’ are used to add methods to self.meths and then execute them
  • The attribute ‘path’ is a node representing the location of the task generator
  • The tasks created are added to the attribute tasks
  • The attribute ‘idx’ is a counter of task generators in the same path
mappings = {'.ASM': <function asm_hook at 0x7fe5a6607230>, '.C': <function cxx_hook at 0x7fe5a663f050>, '.D': <function d_hook at 0x7fe5a6685668>, '.F': <function fc_hook at 0x7fe5a66b3410>, '.F03': <function fc_hook at 0x7fe5a66b3410>, '.F08': <function fc_hook at 0x7fe5a66b3410>, '.F90': <function fc_hook at 0x7fe5a66b3410>, '.F95': <function fc_hook at 0x7fe5a66b3410>, '.FOR': <function fc_hook at 0x7fe5a66b3410>, '.S': <function asm_hook at 0x7fe5a6607230>, '.SPP': <function asm_hook at 0x7fe5a6607230>, '.asm': <function asm_hook at 0x7fe5a6607230>, '.c': <function c_hook at 0x7fe5a67001b8>, '.c++': <function cxx_hook at 0x7fe5a6700398>, '.cc': <function cxx_hook at 0x7fe5a663f050>, '.cpp': <function cxx_hook at 0x7fe5a663f050>, '.cxx': <function cxx_hook at 0x7fe5a663f050>, '.d': <function d_hook at 0x7fe5a6685668>, '.di': <function d_hook at 0x7fe5a6685668>, '.f': <function fc_hook at 0x7fe5a66b3410>, '.f03': <function fc_hook at 0x7fe5a66b3410>, '.f08': <function fc_hook at 0x7fe5a66b3410>, '.f90': <function fc_hook at 0x7fe5a66b3410>, '.f95': <function fc_hook at 0x7fe5a66b3410>, '.for': <function fc_hook at 0x7fe5a66b3410>, '.gresource.xml': <function process_gresource_source at 0x7fe5a66467d0>, '.gs': <function vala_file at 0x7fe5a67838c0>, '.l': <function x_file at 0x7fe5a6699488>, '.lua': <function add_lua at 0x7fe5a66070c8>, '.o': <function add_those_o_files at 0x7fe5a66e5ed8>, '.obj': <function add_those_o_files at 0x7fe5a66e5ed8>, '.pc.in': <function add_pcfile at 0x7fe5a67d9140>, '.py': <function process_py at 0x7fe5a65f0050>, '.qrc': <function create_rcc_task at 0x7fe5a663ae60>, '.rb': <function process at 0x7fe5a6650668>, '.rc': <function rc_file at 0x7fe5a65d18c0>, '.s': <function asm_hook at 0x7fe5a6607230>, '.spp': <function asm_hook at 0x7fe5a6607230>, '.ts': <function add_lang at 0x7fe5a663af50>, '.ui': <function create_uic_task at 0x7fe5a663aed8>, '.vala': <function vala_file at 0x7fe5a67838c0>, '.vapi': <function vapi_file at 0x7fe5a6783938>, '.xs': <function xsubpp_file at 0x7fe5a6607848>, '.y': <function big_bison at 0x7fe5a65dd6e0>, '.yc': <function big_bison at 0x7fe5a65dd6e0>, '.yy': <function big_bison at 0x7fe5a65dd6e0>}

Mappings are global file extension mappings that are retrieved in the order of definition

prec = {'apply_bundle': set(['propagate_uselib_vars', 'apply_link', 'init_pyext']), 'apply_cs': set(['use_cs', 'debug_cs', 'process_source']), 'apply_intltool_in_f': set(['process_source']), 'apply_java': set(['jar_files', 'use_javac_files', 'set_classpath', 'process_source']), 'apply_link': set(['process_use', 'apply_qt5', 'apply_manifest_ifort', 'apply_manifest', 'test_exec_fun', 'set_full_paths_hpux', 'make_test', 'create_task_macapp', 'apply_implib', 'apply_flags_msvc', 'apply_vnum', 'create_task_macplist', 'apply_flags_ifort']), 'apply_ruby_so_name': set(['propagate_uselib_vars', 'apply_link']), 'apply_skip_stlib_link_deps': set(['process_use']), 'apply_tex': set(['process_source']), 'debug_cs': set(['doc_cs']), 'feature_py': set(['process_source']), 'init_perlext': set(['apply_incpaths', 'propagate_uselib_vars', 'apply_link']), 'init_pyembed': set(['propagate_uselib_vars']), 'init_pyext': set(['propagate_uselib_vars', 'apply_link']), 'init_rubyext': set(['apply_incpaths', 'apply_bundle', 'apply_link', 'process_source']), 'jar_files': set(['use_jar_files', 'process_source']), 'link_lib_test_fun': set(['process_source']), 'link_main_routines_tg_method': set(['process_source']), 'process_dbus': set(['process_source']), 'process_enums': set(['process_source']), 'process_install_task': set(['process_rule', 'process_source']), 'process_marshal': set(['process_source']), 'process_mocs': set(['process_source']), 'process_objs': set(['process_source']), 'process_rule': set(['create_javadoc', 'process_source']), 'process_source': set(['process_use', 'apply_incpaths', 'grep_for_endianness_fun', 'apply_link']), 'process_subst': set(['process_rule', 'process_source']), 'process_use': set(['make_windows_app', 'make_test', 'set_full_paths_hpux', 'apply_incpaths', 'make_winphone_app', 'propagate_uselib_vars']), 'propagate_uselib_vars': set(['make_windows_app', 'apply_incpaths', 'apply_vnum', 'set_classpath', 'make_winphone_app']), 'set_bundle': set(['apply_link', 'apply_bundle']), 'set_lib_pat': set(['propagate_uselib_vars', 'apply_link']), 'use_cs': set(['debug_cs']), 'use_javac_files': set(['propagate_uselib_vars', 'jar_files', 'set_classpath'])}

Dict that holds the precedence execution rules for task generator methods

__init__(*k, **kw)[source]

Task generator objects predefine various attributes (source, target) for possible processing by process_rule (make-like rules) or process_source (extensions, misc methods)

Tasks are stored on the attribute ‘tasks’. They are created by calling methods listed in self.meths or referenced in the attribute features A topological sort is performed to execute the methods in correct order.

The extra key/value elements passed in kw are set as attributes

meths = None

List of method names to execute (internal)

features = None

List of feature names for bringing new methods in

tasks = None

Tasks created are added to this list

__str__()[source]

Debugging helper

__repr__()[source]

Debugging helper

get_cwd()[source]

Current working directory for the task generator, defaults to the build directory. This is still used in a few places but it should disappear at some point as the classes define their own working directory.

Return type:waflib.Node.Node
get_name()[source]

If the attribute name is not set on the instance, the name is computed from the target name:

def build(bld):
        x = bld(name='foo')
        x.get_name() # foo
        y = bld(target='bar')
        y.get_name() # bar
Return type:string
Returns:name of this task generator
name

If the attribute name is not set on the instance, the name is computed from the target name:

def build(bld):
        x = bld(name='foo')
        x.get_name() # foo
        y = bld(target='bar')
        y.get_name() # bar
Return type:string
Returns:name of this task generator
to_list(val)[source]

Ensures that a parameter is a list, see waflib.Utils.to_list()

Parameters:val (string or list of string) – input to return as a list
Return type:list
post()[source]

Creates tasks for this task generators. The following operations are performed:

  1. The body of this method is called only once and sets the attribute posted
  2. The attribute features is used to add more methods in self.meths
  3. The methods are sorted by the precedence table self.prec or :waflib:attr:waflib.TaskGen.task_gen.prec
  4. The methods are then executed in order
  5. The tasks created are added to waflib.TaskGen.task_gen.tasks
get_hook(node)[source]

Returns the @extension method to call for a Node of a particular extension.

Parameters:node (waflib.Tools.Node.Node) – Input file to process
Returns:A method able to process the input node by looking at the extension
Return type:function
create_task(name, src=None, tgt=None, **kw)[source]

Creates task instances.

Parameters:
  • name (string) – task class name
  • src (list of waflib.Tools.Node.Node) – input nodes
  • tgt (list of waflib.Tools.Node.Node) – output nodes
Returns:

A task object

Return type:

waflib.Task.Task

clone(env)[source]

Makes a copy of a task generator. Once the copy is made, it is necessary to ensure that the it does not create the same output files as the original, or the same files may be compiled several times.

Parameters:env (waflib.ConfigSet.ConfigSet) – A configuration set
Returns:A copy
Return type:waflib.TaskGen.task_gen
process_dbus()

Processes the dbus files stored in the attribute dbus_lst to create waflib.Tools.dbus.dbus_binding_tool instances.

process_enums()

Processes the enum files stored in the attribute enum_list to create waflib.Tools.glib2.glib_mkenums instances.

process_marshal()

Processes the marshal files stored in the attribute marshal_list to create waflib.Tools.glib2.glib_genmarshal instances. Adds the c file created to the list of source to process.

waflib.TaskGen.declare_chain(name='', rule=None, reentrant=None, color='BLUE', ext_in=[], ext_out=[], before=[], after=[], decider=None, scan=None, install_path=None, shell=False)[source]

Creates a new mapping and a task class for processing files by extension. See Tools/flex.py for an example.

Parameters:
  • name (string) – name for the task class
  • rule (string or function) – function to execute or string to be compiled in a function
  • reentrant (int) – re-inject the output file in the process (done automatically, set to 0 to disable)
  • color (string) – color for the task output
  • ext_in (list of string) – execute the task only after the files of such extensions are created
  • ext_out (list of string) – execute the task only before files of such extensions are processed
  • before (list of string) – execute instances of this task before classes of the given names
  • after (list of string) – execute instances of this task after classes of the given names
  • decider (function) – if present, function that returns a list of output file extensions (overrides ext_out for output files, but not for the build order)
  • scan (function) – scanner function for the task
  • install_path (string) – installation path for the output nodes
waflib.TaskGen.taskgen_method(func)

Decorator that registers method as a task generator method. The function must accept a task generator as first parameter:

from waflib.TaskGen import taskgen_method
@taskgen_method
def mymethod(self):
        pass
Parameters:func (function) – task generator method to add
Return type:function
waflib.TaskGen.feature(*k)

Decorator that registers a task generator method that will be executed when the object attribute feature contains the corresponding key(s):

from waflib.Task import feature
@feature('myfeature')
def myfunction(self):
        print('that is my feature!')
def build(bld):
        bld(features='myfeature')
Parameters:k (list of string) – feature names
waflib.TaskGen.before_method(*k)[source]

Decorator that registera task generator method which will be executed before the functions of given name(s):

from waflib.TaskGen import feature, before
@feature('myfeature')
@before_method('fun2')
def fun1(self):
        print('feature 1!')
@feature('myfeature')
def fun2(self):
        print('feature 2!')
def build(bld):
        bld(features='myfeature')
Parameters:k (list of string) – method names
waflib.TaskGen.before(*k)

Decorator that registera task generator method which will be executed before the functions of given name(s):

from waflib.TaskGen import feature, before
@feature('myfeature')
@before_method('fun2')
def fun1(self):
        print('feature 1!')
@feature('myfeature')
def fun2(self):
        print('feature 2!')
def build(bld):
        bld(features='myfeature')
Parameters:k (list of string) – method names
waflib.TaskGen.after_method(*k)[source]

Decorator that registers a task generator method which will be executed after the functions of given name(s):

from waflib.TaskGen import feature, after
@feature('myfeature')
@after_method('fun2')
def fun1(self):
        print('feature 1!')
@feature('myfeature')
def fun2(self):
        print('feature 2!')
def build(bld):
        bld(features='myfeature')
Parameters:k (list of string) – method names
waflib.TaskGen.after(*k)

Decorator that registers a task generator method which will be executed after the functions of given name(s):

from waflib.TaskGen import feature, after
@feature('myfeature')
@after_method('fun2')
def fun1(self):
        print('feature 1!')
@feature('myfeature')
def fun2(self):
        print('feature 2!')
def build(bld):
        bld(features='myfeature')
Parameters:k (list of string) – method names
waflib.TaskGen.extension(*k)

Decorator that registers a task generator method which will be invoked during the processing of source files for the extension given:

from waflib import Task
class mytask(Task):
        run_str = 'cp ${SRC} ${TGT}'
@extension('.moo')
def create_maa_file(self, node):
        self.create_task('mytask', node, node.change_ext('.maa'))
def build(bld):
        bld(source='foo.moo')
waflib.TaskGen.to_nodes(self, lst, path=None)[source]

Task generator method

Flatten the input list of string/nodes/lists into a list of nodes.

It is used by waflib.TaskGen.process_source() and waflib.TaskGen.process_rule(). It is designed for source files, for folders, see waflib.Tools.ccroot.to_incnodes():

Parameters:
  • lst (list of string and nodes) – input list
  • path (waflib.Tools.Node.Node) – path from which to search the nodes (by default, waflib.TaskGen.task_gen.path)
Return type:

list of waflib.Tools.Node.Node

waflib.TaskGen.process_source(self)[source]

Task generator method

Processes each element in the attribute source by extension.

  1. The source list is converted through waflib.TaskGen.to_nodes() to a list of waflib.Node.Node first.
  2. File extensions are mapped to methods having the signature: def meth(self, node) by waflib.TaskGen.extension()
  3. The method is retrieved through waflib.TaskGen.task_gen.get_hook()
  4. When called, the methods may modify self.source to append more source to process
  5. The mappings can map an extension or a filename (see the code below)
Feature:all
waflib.TaskGen.process_rule(self)[source]

Task generator method

Processes the attribute rule. When present, waflib.TaskGen.process_source() is disabled:

def build(bld):
        bld(rule='cp ${SRC} ${TGT}', source='wscript', target='bar.txt')

Main attributes processed:

  • rule: command to execute, it can be a tuple of strings for multiple commands
  • chmod: permissions for the resulting files (integer value such as Utils.O755)
  • shell: set to False to execute the command directly (default is True to use a shell)
  • scan: scanner function
  • vars: list of variables to trigger rebuilds, such as CFLAGS
  • cls_str: string to display when executing the task
  • cls_keyword: label to display when executing the task
  • cache_rule: by default, try to re-use similar classes, set to False to disable
  • source: list of Node or string objects representing the source files required by this task
  • target: list of Node or string objects representing the files that this task creates
  • cwd: current working directory (Node or string)
  • stdout: standard output, set to None to prevent waf from capturing the text
  • stderr: standard error, set to None to prevent waf from capturing the text
  • timeout: timeout for command execution (Python 3)
  • always: whether to always run the command (False by default)
  • deep_inputs: whether the task must depend on the input file tasks too (False by default)
Feature:all
Before:waflib.TaskGen.process_source()
waflib.TaskGen.sequence_order(self)[source]

Task generator method

Adds a strict sequential constraint between the tasks generated by task generators. It works because task generators are posted in order. It will not post objects which belong to other folders.

Example:

bld(features='javac seq')
bld(features='jar seq')

To start a new sequence, set the attribute seq_start, for example:

obj = bld(features='seq')
obj.seq_start = True

Note that the method is executed in last position. This is more an example than a widely-used solution.

Feature:seq
class waflib.TaskGen.subst_pc(*k, **kw)[source]

Bases: waflib.Task.Task

Creates .pc files from .pc.in. The task is executed whenever an input variable used in the substitution changes.

hcode = '\tdef run(self):\n\t\t"Substitutes variables in a .in file"\n\n\t\tif getattr(self.generator, \'is_copy\', None):\n\t\t\tfor i, x in enumerate(self.outputs):\n\t\t\t\tx.write(self.inputs[i].read(\'rb\'), \'wb\')\n\t\t\t\tstat = os.stat(self.inputs[i].abspath()) # Preserve mtime of the copy\n\t\t\t\tos.utime(self.outputs[i].abspath(), (stat.st_atime, stat.st_mtime))\n\t\t\tself.force_permissions()\n\t\t\treturn None\n\n\t\tif getattr(self.generator, \'fun\', None):\n\t\t\tret = self.generator.fun(self)\n\t\t\tif not ret:\n\t\t\t\tself.force_permissions()\n\t\t\treturn ret\n\n\t\tcode = self.inputs[0].read(encoding=getattr(self.generator, \'encoding\', \'latin-1\'))\n\t\tif getattr(self.generator, \'subst_fun\', None):\n\t\t\tcode = self.generator.subst_fun(self, code)\n\t\t\tif code is not None:\n\t\t\t\tself.outputs[0].write(code, encoding=getattr(self.generator, \'encoding\', \'latin-1\'))\n\t\t\tself.force_permissions()\n\t\t\treturn None\n\n\t\t# replace all % by %% to prevent errors by % signs\n\t\tcode = code.replace(\'%\', \'%%\')\n\n\t\t# extract the vars foo into lst and replace @foo@ by %(foo)s\n\t\tlst = []\n\t\tdef repl(match):\n\t\t\tg = match.group\n\t\t\tif g(1):\n\t\t\t\tlst.append(g(1))\n\t\t\t\treturn "%%(%s)s" % g(1)\n\t\t\treturn \'\'\n\t\tcode = getattr(self.generator, \'re_m4\', re_m4).sub(repl, code)\n\n\t\ttry:\n\t\t\td = self.generator.dct\n\t\texcept AttributeError:\n\t\t\td = {}\n\t\t\tfor x in lst:\n\t\t\t\ttmp = getattr(self.generator, x, \'\') or self.env[x] or self.env[x.upper()]\n\t\t\t\ttry:\n\t\t\t\t\ttmp = \'\'.join(tmp)\n\t\t\t\texcept TypeError:\n\t\t\t\t\ttmp = str(tmp)\n\t\t\t\td[x] = tmp\n\n\t\tcode = code % d\n\t\tself.outputs[0].write(code, encoding=getattr(self.generator, \'encoding\', \'latin-1\'))\n\t\tself.generator.bld.raw_deps[self.uid()] = lst\n\n\t\t# make sure the signature is updated\n\t\ttry:\n\t\t\tdelattr(self, \'cache_sig\')\n\t\texcept AttributeError:\n\t\t\tpass\n\n\t\tself.force_permissions()\n'
force_permissions()[source]

Private for the time being, we will probably refactor this into run_str=[run1,chmod]

run()[source]

Substitutes variables in a .in file

sig_vars()[source]

Compute a hash (signature) of the variables used in the substitution

waflib.TaskGen.add_pcfile(self, node)[source]

Processes .pc.in files to .pc. Installs the results to ${PREFIX}/lib/pkgconfig/ by default

def build(bld):
bld(source=’foo.pc.in’, install_path=’${LIBDIR}/pkgconfig/’)
waflib.TaskGen.process_subst(self)[source]

Task generator method

Defines a transformation that substitutes the contents of source files to target files:

def build(bld):
        bld(
                features='subst',
                source='foo.c.in',
                target='foo.c',
                install_path='${LIBDIR}/pkgconfig',
                VAR = 'val'
        )

The input files are supposed to contain macros of the form @VAR@, where VAR is an argument of the task generator object.

This method overrides the processing by waflib.TaskGen.process_source().

Feature:subst
Before:waflib.TaskGen.process_source(), waflib.TaskGen.process_rule()