Using Waf to build an iOS Universal Framework

I've used a lot of build systems, and so far my favorite (when I'm "forced" to use something outside of Visual Studio) is probably waf.  Besides being based on a proper scripting language, waf doesn't feel like it's doing a whole lot of "wicked voodoo magic" behind the scenes (like scons sometimes does).  It's also fairly easy to extend, and we used it to make and deploy all of our Nacl executables.

Today, I figured out a way to make waf do some interesting things concerning iOS and creating libraries / frameworks.

iOS Universal Frameworks require that you create libraries for 2 versions of the simulator (i386 and x86_64) as well as 2 versions of ARM (armv6 and armv7) if you want to run on all of those platforms.  Each of these libraries is then linked together using a tool called lipo.  Waf can easily build one library, but building multiple, especially as part of one build step, can be tricky.  What you do is create a separate environment for each target, then link them together in a single build step. Here's how it goes.

First, waf doesn't understand how to handle .m / .mm files and pass them on to clang properly, so we want to teach it how to do that. The simplest way to do that is like so:

@TaskGen.extension('.mm')
def mm_hook(self, node):
    "Bind the c++ file extensions to the creation of a :py:class:`waflib.Tools.cxx.cxx` instance"
    return self.create_compiled_task('mm', node)

class mm(Task.Task):
    "Compile MM files into object files"
    run_str = '${CXX} ${ARCH_ST:ARCH} ${MMFLAGS} ${FRAMEWORKPATH_ST:FRAMEWORKPATH} ${CPPPATH_ST:INCPATHS} ${DEFINES_ST:DEFINES} ${CXX_SRC_F}${SRC} ${CXX_TGT_F}${TGT}'
    vars    = ['CXXDEPS'] # unused variable to depend on, just in case
    ext_in  = ['.h'] # set the build order easily by using ext_out=['.h']

This will use whatever the CXX compiler is set to (which we'll set to clang) using MMFLAGS over CXXFLAGS or CCFLAGS. Everything else will stay the same. This can be added anywhere in the file, but I tend to add it under configure, before build.

Next, we set up configure to create an ios base environment, off of which we're going to base other targets.  Because I use this script to build for multiple environments on multiple operating systems, I check to make sure that this is being performed on Mac OS before creating the environment.

IOS_MIN_VER = '6.0'
IOS_SDK_LOC = '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneSimulator6.0.sdk'
IOS_SDK_SIM_LOC = '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator6.0.sdk'

def configure(conf):
    # ... Any other configure steps
    
    # Make sure we're running on a Mac
    if sys.platform.startswith('darwin'):
        conf.setenv('ios')
        # Set the C++ plugin here to use clang over gcc  g++
        conf.env.CXX = 'clang'
        conf.env.CC = 'clang'
        conf.env.LIPO = 'lipo'
        conf.load('gxx')
        conf.load('gcc')
        # set any CFLAGS, CXXFLAGS, MMFLAGS and DEFINES for ios in general

Next, we create an environment for each target (debug or release) and each platform (i386, x86_64, armv6 and armv7). If you're not targeting an x86_64 simulator or an armv6 device, you don't need to build these, but since post first build this is fairly quick, I just build all of them.

        for arch, tgt in itertools.product(['i386', 'x86_64', 'armv6', 'armv7'], ['dbg','rel']):
            conf.setenv('ios')
            newEnv = conf.env.derive()
            newEnv.detach()
            conf.setenv('ios_' + arch + '_' + tgt, newEnv)
            #... set any target specific flags 
            # These flags set up the sysroot for ios based on simulator or device
            if arch.startswith('arm'):
                addflags = [ '-mios-version-min=' + IOS_MIN_VER, '-isysroot' + IOS_SDK_LOC ]
            else:
                addflags = [ '-mios-version-min=' + IOS_MIN_VER, '-isysroot' + IOS_SDK_LOC ]
            conf.env.CCFLAGS += addflags
            conf.env.CXXFLAGS += addflags
            conf.env.MMFLAGS += addflags    
            conf.env.ARCH = arch

Now we have to create commands that use each of these environments. This is exactly the same as using multiple variants / platforms, if you've seen that example. The other thing we're going to do is make a LIPO command for each target, so we don't have to re-specify how to LIPO our libs together (this is useful if you've got multiple like I do).

def init(ctx):
    # any other architectures / platforms

    for arch in ['i386', 'x86_64', 'armv6', 'armv7']:
        for tgt in ['dbg', 'rel', 'retail']:
            for inheritingClass in (BuildContext, CleanContext, InstallContext, UninstallContext):
                name = inheritingClass.__name__.replace('Context','').lower()
                class tempClass(inheritingClass):
                    # this is used in the next step to actually call the step
                    cmd = name + '_ios_' + arch + '_' + tgt
                    # This must match your environment name, and sets the output directory
                    variant = 'ios_' + arch + '_' + tgt  
                    # These are for testing in your build script, if you need them
                    architecture = arch
                    target = tgt
                    platform = 'ios'

      for tgt in ['dbg', 'rel', 'retail']:
          class tempClass(BuildContext):
              cmd = 'lipo_ios_' + tgt
              fun = 'lipo_ios'
              variant = 'ios'
              target = tgt
              platform = 'ios'

Last, we create a build_ios command to build all the variants for a target (I default to release), and a command called lipo_ios to LIPO each one together. Here's how it goes.

def _lipo_lib(ctx, lib_name):
    ctx(rule='lipo -create ${SRC} -output ${TGT}',
        shell = True,
        target = 'lib%s-%s.a' % (lib_name, ctx.target),
        source = ['build/ios_%s_%s/lib%s.a' % (x, ctx.target, lib_name) for x in IOS_ARCHS]
    )

def build_ios(ctx):
    for x in ['i386', 'x86_64', 'armv6', 'armv7']:
        waflib.Options.commands.append('build_ios_' + x + '_rel')

    waflib.Options.commands.append('lipo_ios_rel')

def lipo_ios(ctx):
    """ This creates universal libraries in ios_target made from the architecture 
        builds of each lib. """
    _lipo_lib(ctx, 'A')
    _lipo_lib(ctx, 'B')
    _lipo_lib(ctx, 'C')

So this is not at all straight forward, and took a lot of trial and error, so hopefully this helps some people.

The upshot here is that waf's dependency detection in terms of libraries is way better than XCode's, so you can use multiple dependent libraries together and it will only ever rebuild exactly what's necessary. However, the preprocessor that determines include dependency only supports C / C++, not Objective-C's #import directive. So I would avoid doing this if you're working a lot with Objective-C dependencies, at least until waf gets real objective-C support.

Also, you could use these same principles to create build_ios_simulator and build_ios_device targets instead, without too much effort, but I like having all targets build into the universal libs, and going from there.

Bookmark the permalink.

One Response to Using Waf to build an iOS Universal Framework

  1. Randi Hooker says:

    I am trying to see if you are in the market for a new and challenging position. My company is looking for a Game Developer. Let me know if you are interested and I will send you the description.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>