A Tour of Waf

A Waf build is the execution of a set of interdependent tasks. A task consumes a number of sources and produces a number of targets. Sources and targets may be files, which Waf abstracts into objects called nodes. Sources may also be values of variables in a Waf environment. Tasks may be defined directly in a wscript file that provides Waf directives expressed in the Python language. Tasks may also be created during the build with task generators. Code to produce tasks may be bundled into reusable Waf tools. The detailed nature of the build may be controlled through a Waf configuration and that configuration is communicated and persisted with one or more Waf environments. The graph of tasks, sources and targets may be factored into one or more subgraphs or Waf groups. Each group will be executed in sequence. Within a group, tasks will execute in parallel to the extent possible, limited by dependencies and a maximum allowed number of jobs. Finally, configuration and build and other commands are managed through Waf contexts.

This tour will have a number of examples. They are run as part of preparing this document into HTML or PDF. Generally they include three major waf commands in a step like:

waf distclean configure build

The distclean is not typically required nor desired. It is included here to assure each example runs from a known initial state.

In general, the value of a variable in the calling shell environment has no impact on values in a Waf environment. However, some Waf code will allow the shell environment to influence values in the Waf environment.

from waflib.Logs import info
def configure(cfg):
    cfg.env.FOO = "bar"
    cfg.find_program("cc")
    info(str(cfg.env))
def build(bld):
    info(str(bld.env))

In configure() we set the Waf environment variable FOO. We also instruct Waf to find the program cc. By default, the resulting location will be placed in an environment variable of the same name but capitalized, namely here CC.

waf distclean configure

'distclean' finished successfully (0.002s)
Setting top to                           : /home/bv/org/topics/waf/tour/calling-environment 
Setting out to                           : /home/bv/org/topics/waf/tour/calling-environment/build 
Checking for program 'cc'                : /usr/bin/cc 
'BINDIR' '/usr/local/bin'
'CC' ['/usr/bin/cc']
'FOO' 'bar'
'LIBDIR' '/usr/local/lib'
'PREFIX' '/usr/local'
'configure' finished successfully (0.003s)

In addition to the variables we set directly or through find_program(), Waf has set PREFIX, BINDIR and LIBDIR. We may return configure with some shell environment variables set.

PREFIX=/tmp CC=gcc FOO=baz waf configure

Setting top to                           : /home/bv/org/topics/waf/tour/calling-environment 
Setting out to                           : /home/bv/org/topics/waf/tour/calling-environment/build 
Checking for program 'cc'                : gcc 
from configure:
	FOO = bar
	CC = ['gcc']
'configure' finished successfully (0.003s)

Note FOO remains unchanged while CC takes the value we gave in the shell environment. Not only is PREFIX changed but so are the two directories. Waf will persist an environments persist between calls to configure and build.

PREFIX=/nope waf build

Waf: Entering directory `/home/bv/org/topics/waf/tour/calling-environment/build'
from build:
	FOO = bar
	CC = ['gcc']
Waf: Leaving directory `/home/bv/org/topics/waf/tour/calling-environment/build'
'build' finished successfully (0.011s)

Rerunning configure will reset all variables.

waf configure

Setting top to                           : /home/bv/org/topics/waf/tour/calling-environment 
Setting out to                           : /home/bv/org/topics/waf/tour/calling-environment/build 
Checking for program 'cc'                : /usr/bin/cc 
'BINDIR' '/usr/local/bin'
'CC' ['/usr/bin/cc']
'FOO' 'bar'
'LIBDIR' '/usr/local/lib'
'PREFIX' '/usr/local'
'configure' finished successfully (0.004s)

from waflib.Logs import info
def configure(cfg):
    info(f'cfg.path = {cfg.path}')
    cfg.find_program('cat')
def build(bld):
    info(f'bld.path = {bld.path}')
    info(f'bld.bldnode = {bld.bldnode}')
    wscript = bld.path.find_resource("wscript")
    info(f'wscript = {wscript}')
    info(f'parent = {wscript.parent}')
    this = bld.path.make_node("this.txt")
    this.write("# end of file.\n")
    that = bld.path.find_or_declare("that.txt")
    bld(rule="${CAT} ${SRC} > ${TGT}", shell=True,
        source=[wscript, this], target=that)

Here we print a few path values, make use of .parent. We also call node methods find_resource() (a node that must exist in the source) and find_or_declare() (a node that may exist already in the source or the build area or which may be created during the build) and make_node() to create a node from thin air. We also use a node as an file-like object to write. Finally, we tie it all into a rule that combines two source files, one that we created, to a target.

waf configure build

Setting top to                           : /home/bv/org/topics/waf/tour/node-features 
Setting out to                           : /home/bv/org/topics/waf/tour/node-features/build 
cfg.path = /home/bv/org/topics/waf/tour/node-features
Checking for program 'cat'               : /usr/bin/cat 
'configure' finished successfully (0.004s)
Waf: Entering directory `/home/bv/org/topics/waf/tour/node-features/build'
bld.path = /home/bv/org/topics/waf/tour/node-features
bld.bldnode = /home/bv/org/topics/waf/tour/node-features/build
wscript = /home/bv/org/topics/waf/tour/node-features/wscript
parent = /home/bv/org/topics/waf/tour/node-features
Waf: Leaving directory `/home/bv/org/topics/waf/tour/node-features/build'
'build' finished successfully (0.011s)

tail -1 build/that.txt

# end of file.

Produce one file from another file with a command and everything is hard-wired.

def configure(cfg):
    pass
def build(bld):
    bld(rule="cp ../this.txt that.txt")

Waf runs tasks from the build/ directory and normally all targets produced by a task should place files into this same directory. Waf will assure proper file placement but here we are cutting Waf out of that duty. Next section will repeat this example in a more proper manner.

date > this.txt
waf distclean configure build

'distclean' finished successfully (0.002s)
Setting top to                           : /home/bv/org/topics/waf/tour/this-to-that 
Setting out to                           : /home/bv/org/topics/waf/tour/this-to-that/build 
'configure' finished successfully (0.002s)
Waf: Entering directory `/home/bv/org/topics/waf/tour/this-to-that/build'
[1/1] Running cp ../this.txt that.txt
Waf: Leaving directory `/home/bv/org/topics/waf/tour/this-to-that/build'
'build' finished successfully (0.019s)
Sat Apr  8 05:19:18 PM EDT 2023
Sat Apr  8 05:19:18 PM EDT 2023

cat this.txt build/that.txt

Even though we have denied Waf any knowledge of what files the rule produced, it knows that it successfully ran the task once and will not rerun it:

waf build

Waf: Entering directory `/home/bv/org/topics/waf/tour/this-to-that/build'
Waf: Leaving directory `/home/bv/org/topics/waf/tour/this-to-that/build'
'build' finished successfully (0.018s)

Here we do not deny Waf the knowledge of the input and output to a rule to repeat the above example in a more proper way.

def configure(cfg):
    pass
def build(bld):
    bld(rule="cp ${SRC} ${TGT}", source="this.txt", target="that.txt")

The SRC and TGT variables referenced in the rule string map to an list of input and output nodes, respectively. A node is a Waf file or directory object. By default, ${SRC} expands to a space separated list of the paths of the nodes in the input array. Individual nodes in the array may be addressed through indexing, as in ${SRC[0]}. Node methods may be called for example, ${SRC[0].abspath()}.

date > this.txt
waf distclean configure build

We suppress the output this time as it is same as above.

So far, a rule has been given as a parameterized or templated command line string. A rule can also be a Python function.

from waflib.Utils import subst_vars
def copy(task):
    src = task.inputs[0]
    tgt = task.outputs[0]
    tgt.write(src.read())
def configure(cfg):
    pass
def build(bld):
    bld(rule=copy, source="this.txt", target="that.txt")

The copy() rule implements a primitive version of the cp command. We will again omit the output of running this example.