c_preproc¶

C/C++ preprocessor for finding dependencies

Reasons for using the Waf preprocessor by default

Some c/c++ extensions (Qt) require a custom preprocessor for obtaining the dependencies (.moc files)
Not all compilers provide .d files for obtaining the dependencies (portability)
A naive file scanner will not catch the constructs such as “#include foo()”
A naive file scanner will catch unnecessary dependencies (change an unused header -> recompile everything)

Regarding the speed concerns:

the preprocessing is performed only when files must be compiled
the macros are evaluated only for #if/#elif/#include
system headers are not scanned by default

Now if you do not want the Waf preprocessor, the tool +gccdeps* uses the .d files produced during the compilation to track the dependencies (useful when used with the boost libraries). It only works with gcc >= 4.4 though.

A dumb preprocessor is also available in the tool c_dumbpreproc

exception waflib.Tools.c_preproc.PreprocError(msg='', ex=None)[source]¶

__annotations__ = {}¶

waflib.Tools.c_preproc.POPFILE = '-'¶: Constant representing a special token used in waflib.Tools.c_preproc.c_parser.start() iteration to switch to a header read previously

waflib.Tools.c_preproc.recursion_limit = 150¶: Limit on the amount of files to read in the dependency scanner

waflib.Tools.c_preproc.go_absolute = False¶: Set to True to track headers on files in /usr/include, else absolute paths are ignored (but it becomes very slow)

waflib.Tools.c_preproc.use_trigraphs = 0¶: Apply trigraph rules (False by default)

waflib.Tools.c_preproc.g_optrans = {'and': '&&', 'and_eq': '&=', 'bitand': '&', 'bitor': '|', 'compl': '~', 'not': '!', 'not_eq': '!', 'or': '||', 'or_eq': '|=', 'xor': '^', 'xor_eq': '^='}¶: Operators such as and/or/xor for c++. Set an empty dict to disable.

waflib.Tools.c_preproc.re_lines = re.compile('^[ \t]*(?:#|%:)[ \t]*(ifdef|ifndef|if|else|elif|endif|include|import|define|undef|pragma)[ \t]*(.*)\r*$', re.IGNORECASE|re.MULTILINE)¶: Match #include lines

waflib.Tools.c_preproc.re_mac = re.compile('^[a-zA-Z_]\\w*')¶: Match macro definitions

waflib.Tools.c_preproc.re_fun = re.compile('^[a-zA-Z_][a-zA-Z0-9_]*[(]')¶: Match macro functions

waflib.Tools.c_preproc.re_pragma_once = re.compile('^\\s*once\\s*', re.IGNORECASE)¶: Match #pragma once statements

waflib.Tools.c_preproc.re_nl = re.compile('\\\\\r*\n', re.MULTILINE)¶: Match newlines

waflib.Tools.c_preproc.re_cpp = re.compile('//.*?$|/\\*.*?\\*/|\\\'(?:\\\\.|[^\\\\\\\'])*\\\'|"(?:\\\\.|[^\\\\"])*"', re.MULTILINE|re.DOTALL)¶: Filter C/C++ comments

waflib.Tools.c_preproc.trig_def = [('??=', '#'), ('??-', '~'), ('??/', '\\'), ('??!', '|'), ("??'", '^'), ('??(', '['), ('??)', ']'), ('??<', '{'), ('??>', '}')]¶: Trigraph definitions

waflib.Tools.c_preproc.chr_esc = {"'": 39, '0': 0, '\\': 92, 'a': 7, 'b': 8, 'f': 11, 'n': 10, 'r': 13, 't': 9, 'v': 12}¶: Escape characters

waflib.Tools.c_preproc.NUM = 'i'¶: Number token

waflib.Tools.c_preproc.OP = 'O'¶: Operator token

waflib.Tools.c_preproc.IDENT = 'T'¶: Identifier token

waflib.Tools.c_preproc.STR = 's'¶: String token

waflib.Tools.c_preproc.CHAR = 'c'¶: Character token

waflib.Tools.c_preproc.tok_types = ['i', 's', 'T', 'O']¶: Token types

waflib.Tools.c_preproc.exp_types = ["0[xX](?P<hex>[a-fA-F0-9]+)(?P<qual1>[uUlL]*)|L*?'(?P<char>(\\\\.|[^\\\\'])+)'|(?P<n1>\\d+)[Ee](?P<exp0>[+-]*?\\d+)(?P<float0>[fFlL]*)|(?P<n2>\\d*\\.\\d+)([Ee](?P<exp1>[+-]*?\\d+))?(?P<float1>[fFlL]*)|(?P<n4>\\d+\\.\\d*)([Ee](?P<exp2>[+-]*?\\d+))?(?P<float2>[fFlL]*)|(?P<oct>0*)(?P<n0>\\d+)(?P<qual2>[uUlL]*)", 'L?"([^"\\\\]|\\\\.)*"', '[a-zA-Z_]\\w*', '%:%:|<<=|>>=|\\.\\.\\.|<<|<%|<:|<=|>>|>=|\\+\\+|\\+=|--|->|-=|\\*=|/=|%:|%=|%>|==|&&|&=|\\|\\||\\|=|\\^=|:>|!=|##|[\$\$\\{\\}\\[\\]<>\\?\\|\\^\\*\\+&=:!#;,%/\\-\\?\\~\\.]']¶: Expression types

waflib.Tools.c_preproc.re_clexer = re.compile('(?P<i>0[xX](?P<hex>[a-fA-F0-9]+)(?P<qual1>[uUlL]*)|L*?\'(?P<char>(\\\\.|[^\\\\\'])+)\'|(?P<n1>\\d+)[Ee](?P<exp0>[+-]*?\\d+)(?P<float0>[fFlL]*)|(?P<n2>\\d*\\.\\d+)([Ee](?P<exp1>[+-]*?\\d+))?(?P<float1, re.MULTILINE)¶: Match expressions into tokens

waflib.Tools.c_preproc.accepted = 'a'¶: Parser state is accepted

waflib.Tools.c_preproc.ignored = 'i'¶: Parser state is ignored, for example preprocessor lines in an #if 0 block

waflib.Tools.c_preproc.undefined = 'u'¶: Parser state is undefined at the moment

waflib.Tools.c_preproc.skipped = 's'¶: Parser state is skipped, for example preprocessor lines in a #elif 0 block

waflib.Tools.c_preproc.repl(m)[source]¶: Replace function used with waflib.Tools.c_preproc.re_cpp

waflib.Tools.c_preproc.prec = {'!=': 4, '%': 0, '&': 5, '&&': 6, '*': 0, '+': 1, ',': 7, '-': 1, '/': 0, '<': 3, '<<': 2, '<=': 3, '==': 4, '>': 3, '>=': 3, '>>': 2, '^': 5, '|': 5, '||': 6}¶

Operator precedence rules required for parsing expressions of the form:

#if 1 && 2 != 0

waflib.Tools.c_preproc.reduce_nums(val_1, val_2, val_op)[source]¶

Apply arithmetic rules to compute a result

Parameters

val1 (int or string) – input parameter
val2 (int or string) – input parameter
val_op (string) – C operator in +, /, -, etc

Return type

int

waflib.Tools.c_preproc.get_num(lst)[source]¶

Try to obtain a number from a list of tokens. The token types are defined in waflib.Tools.ccroot.tok_types.

Parameters: lst (list of tuple (tokentype, value)) – list of preprocessor tokens
Returns: a pair containing the number and the rest of the list
Return type: tuple(value, list)

waflib.Tools.c_preproc.get_term(lst)[source]¶

Evaluate an expression recursively, for example:

1+1+1 -> 2+1 -> 3

Parameters: lst (list of tuple(token, value)) – list of tokens
Returns: the value and the remaining tokens
Return type: value, list

waflib.Tools.c_preproc.reduce_eval(lst)[source]¶

Take a list of tokens and output true or false for #if/#elif conditions.

Parameters: lst (list of tuple(token, value)) – a list of tokens
Returns: a token
Return type: tuple(NUM, int)

waflib.Tools.c_preproc.stringize(lst)[source]¶

Merge a list of tokens into a string

Parameters: lst (list of tuple(token, value)) – a list of tokens
Return type: string

waflib.Tools.c_preproc.paste_tokens(t1, t2)[source]¶

Token pasting works between identifiers, particular operators, and identifiers and numbers:

a ## b  ->  ab
> ## =  ->  >=
a ## 2  ->  a2

Parameters

t1 (tuple(type, value)) – token
t2 (tuple(type, value)) – token

waflib.Tools.c_preproc.reduce_tokens(lst, defs, ban=[])[source]¶

Replace the tokens in lst, using the macros provided in defs, and a list of macros that cannot be re-applied

Parameters

lst (list of tuple(token, value)) – list of tokens
defs (dict) – macro definitions
ban (list of string) – macros that cannot be substituted (recursion is not allowed)

Returns

the new list of tokens

Return type

value, list

waflib.Tools.c_preproc.eval_macro(lst, defs)[source]¶

Reduce the tokens by waflib.Tools.c_preproc.reduce_tokens() and try to return a 0/1 result by waflib.Tools.c_preproc.reduce_eval().

Parameters

lst (list of tuple(token, value)) – list of tokens
defs (dict) – macro definitions

Return type

int

waflib.Tools.c_preproc.extract_macro(txt)[source]¶

Process a macro definition of the form::: #define f(x, y) x * y

into a function or a simple macro without arguments

Parameters: txt (string) – expression to exact a macro definition from
Returns: a tuple containing the name, the list of arguments and the replacement
Return type: tuple(string, [list, list])

waflib.Tools.c_preproc.extract_include(txt, defs)[source]¶

Process a line in the form:

#include foo

Parameters

txt (string) – include line to process
defs (dict) – macro definitions

Returns

the file name

Return type

string

waflib.Tools.c_preproc.parse_char(txt)[source]¶

Parse a c character

Parameters: txt (string) – character to parse
Returns: a character literal
Return type: string

waflib.Tools.c_preproc.tokenize(s)[source]¶

Convert a string into a list of tokens (shlex.split does not apply to c/c++/d)

Parameters: s (string) – input to tokenize
Returns: a list of tokens
Return type: list of tuple(token, value)

class waflib.Tools.c_preproc.c_parser(nodepaths=None, defines=None)[source]¶

Used by waflib.Tools.c_preproc.scan() to parse c/h files. Note that by default, only project headers are parsed.

__annotations__ = {}¶

lines¶: list of lines read

nodepaths¶: Include paths

nodes¶: List of waflib.Node.Node found so far

names¶: List of file names that could not be matched by any file

curfile¶: Current file

ban_includes¶: Includes that must not be read (#pragma once)

listed¶: Include nodes/names already listed to avoid duplicates in self.nodes/self.names

cached_find_resource(node, filename)[source]¶

Find a file from the input directory

Parameters

node (waflib.Node.Node) – directory
filename (string) – header to find

Returns

the node if found, or None

Return type

waflib.Node.Node

tryfind(filename, kind='"', env=None)[source]¶

Try to obtain a node from the filename based from the include paths. Will add the node found to waflib.Tools.c_preproc.c_parser.nodes or the file name to waflib.Tools.c_preproc.c_parser.names if no corresponding file is found. Called by waflib.Tools.c_preproc.c_parser.start.

Parameters: filename (string) – header to find
Returns: the node if found
Return type: waflib.Node.Node

filter_comments(node)[source]¶

Filter the comments from a c/h file, and return the preprocessor lines. The regexps waflib.Tools.c_preproc.re_cpp, waflib.Tools.c_preproc.re_nl and waflib.Tools.c_preproc.re_lines are used internally.

Returns: the preprocessor directives as a list of (keyword, line)
Return type: a list of string pairs

addlines(node)[source]¶

Add the lines from a header in the list of preprocessor lines to parse

Parameters: node (waflib.Node.Node) – header

start(node, env)[source]¶

Preprocess a source file to obtain the dependencies, which are accumulated to waflib.Tools.c_preproc.c_parser.nodes and waflib.Tools.c_preproc.c_parser.names.

Parameters

node (waflib.Node.Node) – source file
env (waflib.ConfigSet.ConfigSet) – config set containing additional defines to take into account

define_name(line)[source]¶

Parameters: line (string) – define line
Return type: string
Returns: the define name

waflib.Tools.c_preproc.scan(task)[source]¶

Get the dependencies using a c/c++ preprocessor, this is required for finding dependencies of the kind:

#include some_macro()

This function is bound as a task method on waflib.Tools.c.c and waflib.Tools.cxx.cxx for example

c_preproc¶

Previous topic

Next topic

This Page