There comes a point in any project that when it reaches a certain size, a flat directory hierarchy no longer works. There are too many files, some perhaps similarly named, and additional organization becomes needed. One approach is to use recursive Makefiles, with a top-level set of Makefiles controlling the build which call into multiple subdirectories that contain the actual source code.
This article describes one such approach, with the additional requirements that it support cross-compiling for another CPU target and tracks file dependencies. Additional goodies include packaging the generated executables into a filesystem image and digitally signing it with GnuPG.
Much of this work is inspired from various manuals, blog posts, and Stackoverflow questions found on the Internet. Unfortunately this framework is several years old and I did not record information sources at the time it was initially constructed. Many thanks are owed to anonymous coders on the Internet.
This framework uses GNU Make and has not been tested with other implementations.
A complete copy of this example can be found on my GitHub page here.
Table of Contents
Structure Layout
At the root level, three Makefiles are utilized.
common.mk – common targets and definitions that are global to the projectversion.mk – specifies version control informationMakefile – top-level Makefile which ties it all together
Adjacent to these Makefiles, one or more subdirectories are created to hold appropriately grouped code. These are nominally components that will each be compiled into an executable binary or a static .a library. Shared .so libraries was outside the scope of my needs, but likely simple to implement.
The example framework consists of three components.
exeA – executable process “A”, which also depends on libAexeB – executable process “B”libA – static .a library pulled in by exeA
The full directory tree thus looks like:
$ ls -l
total 24
-rw-rw-r-- 1 eric eric 1450 Jan 20 22:22 common.mk
drwxrwxr-x 2 eric eric 4096 Jan 20 22:59 exeA
drwxrwxr-x 2 eric eric 4096 Jan 20 22:59 exeB
drwxrwxr-x 2 eric eric 4096 Jan 20 22:59 libA
-rw-rw-r-- 1 eric eric 1067 Jan 20 21:41 Makefile
-rw-rw-r-- 1 eric eric 892 Jan 20 22:05 version.mk
Makefile
The top-level Makefile is used to define components and major build targets.
include version.mk
libA := libA
libraries := $(libA)
exeA := exeA
exeB := exeB
imagedir := image
imagenamebase := $(partNumber)_$(appName)_v$(versionMajor).$(versionMinor).$(versionRevision)
imagefile := $(imagenamebase).squashfs
.PHONY: all $(exeA) $(exeB) $(libraries)
all: $(exeA) $(exeB) $(libraries)
install: $(exeA) $(exeB)
$(RM) -rf $(imagedir) $(imagefile)
install -D -m 755 -t $(imagedir)/ $(exeA)/exeA
install -D -m 755 -t $(imagedir)/ $(exeB)/exeB
mksquashfs $(imagedir) $(imagefile)
sign: install
gpg --detach-sign $(imagefile)
mkdir $(imagenamebase)
mv $(imagefile) $(imagefile).sig $(imagenamebase)
zip -r $(imagenamebase).zip $(imagenamebase)
rm -rf $(imagenamebase)
$(eval versionAutoNum=$(shell echo $$(($(versionAutoNum)+1))))
@echo $(versionAutoNum) > $(autoNumFile)
clean:
$(MAKE) -C . TARGET=clean
rm -rf image *.squashfs *.zip
$(exeA) $(exeB) $(libraries):
$(MAKE) --directory=$@ $(TARGET)
# Configure the various module dependencies
$(exeA): $(libA)
Next we will break down the major pieces of this Makefile.
include version.mk
Include version.mk, as we will need to know versioning information for constructing the final firmware filename, etc.
libA := libA
libraries := $(libA)
exeA := exeA
exeB := exeB
Define various variables that can be reused later, rather than hard-coding specific component names. The example looks rather redundant, but becomes helpful once scaled out a bit. Additionally, all libraries are grouped into the libraries variable.
imagedir := image
imagenamebase := $(partNumber)_$(appName)_v$(versionMajor).$(versionMinor).$(versionRevision)
imagefile := $(imagenamebase).squashfs
Various file and directory names that will be used later for firmware image construction. imagedir is the temporary directory to use to stage the firmware filesystem, imagenamebase is the base filename for the image which takes into account some metadata including software part number and version, and imagefile is the final target filesystem image. As the name implies, this image will be a squashfs image.
.PHONY: all $(exeA) $(exeB) $(libraries)
all: $(exeA) $(exeB) $(libraries)
Set up the all build target. This will compile each executable and library.
install: $(exeA) $(exeB)
$(RM) -rf $(imagedir) $(imagefile)
install -D -m 755 -t $(imagedir)/ $(exeA)/exeA
install -D -m 755 -t $(imagedir)/ $(exeB)/exeB
mksquashfs $(imagedir) $(imagefile)
Set up the install build target. This will copy various ad-hoc files into $(imagedir), which is them constructed into the filesystem image.
sign: install
gpg --detach-sign $(imagefile)
mkdir $(imagenamebase)
mv $(imagefile) $(imagefile).sig $(imagenamebase)
zip -r $(imagenamebase).zip $(imagenamebase)
rm -rf $(imagenamebase)
$(eval versionAutoNum=$(shell echo $$(($(versionAutoNum)+1))))
@echo $(versionAutoNum) > $(autoNumFile)
Sign and archive the filesystem image. There is a lot going on here, so we will break down each step.
- Create a detached signature for the image. This will use whatever default private key GnuPG selects.
- Create a staging directory to archive both the filesystem image and the signature.
- Move the filesystem image and the signature into the staging directory.
- Archive the staging directory into the firmware image package.
- Remove staging directory.
- The final two lines track the build “auto-number”. This build number is appended to the firmware image in order to differentiate subsequent builds during the development process, and incremented here.
clean:
$(MAKE) -C . TARGET=clean
rm -rf image *.squashfs *.zip
Clean the project, with a slightly convoluted approach. Call this Makefile again, but with the TARGET=clean variable set. This will be picked up by the following build target.
$(exeA) $(exeB) $(libraries):
$(MAKE) --directory=$@ $(TARGET)
Target for the various executables and libraries. By default this will recurse into those component subdirectories and call make, which will then build the all target (the code). Should make clean have been called above, the recursed target will instead be clean, as defined in common.mk. This will clean up the various executables, libraries, object files, etc.
# Configure the various module dependencies
$(exeA): $(libA)
Defines component dependencies. In this case, exeA depends on libA, such that libA will be checked for being up-to-date and rebuilt automatically if not.
common.mk
This file defines global data and recipes, including how to compile C files, dependency information, libraries, and link executables.
# Set to 1 to enable debugging
# Be sure to clean and rebuild after modifying
DEBUG=0
MV := mv -f
RM := rm -f
SED := sed
# If building for docker/host system, add extra options
# Otherwise, specify the cross compiler
ifeq ($(DOCKER),1)
# Add any extra options here...
else
CROSS := arm-linux-
endif
CC := $(CROSS)gcc
LD := $(CROSS)ld
AR := $(CROSS)ar
objects := $(subst .c,.o,$(sources))
dependencies := $(subst .c,.d,$(sources))
# Add any custom global cpp flags you might need here...
CPPFLAGS += -DSOMETHING=1
# General flags and includes
include_dirs := ../libA
CFLAGS += -Wall -Werror
ifeq ($(DEBUG),1)
CFLAGS += -ggdb
else
CFLAGS += -O3
endif
# Combine include dirs for the pre-processor
CPPFLAGS += $(addprefix -I,$(include_dirs))
vpath %.h $(include_dirs)
.PHONY: library
library: $(library)
$(library): $(objects)
$(AR) $(ARFLAGS) $@ $^
.PHONY: program
program: $(program)
$(program): $(objects) $(libraries)
$(CC) -o $(program) $(LDFLAGS) $(objects) $(libraries) $(LDLIBS)
.PHONY: clean
clean:
$(RM) $(objects) $(program) $(library) $(dependencies)
ifneq "$(MAKECMDGOALS)" "clean"
-include $(dependencies)
endif
%.c %.h: %.y
$(YACC.y) --defines $<
$(MV) y.tab.c $*.c
$(MV) y.tab.h $*.h
%.d: %.c
$(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -M $< | \
$(SED) 's,\($*\.o\) *:,\1 $@: ,' > $@.tmp
$(MV) $@.tmp $@
Next we will break down the major pieces of this Makefile.
# Set to 1 to enable debugging
# Be sure to clean and rebuild after modifying
DEBUG=0
Flag enabling debug builds. It can be set to 1 here, or invoked via make DEBUG=1.
MV := mv -f
RM := rm -f
SED := sed
Various standard utilities.
# If building for docker/host system, add extra options
# Otherwise, specify the cross compiler
ifeq ($(DOCKER),1)
# Add any extra options here...
else
CROSS := arm-linux-
endif
If DOCKER=1 is defined, run a Docker build. This is also synonymous with a host build. The intent of it is for compiling the application which can be run within a Docker container for localize execution and debugging.
If not defined, specify the prefix of the cross-compiler to use.
CC := $(CROSS)gcc
LD := $(CROSS)ld
AR := $(CROSS)ar
Define our compiler, linker, and archiver. If $(CROSS) is defined, it will expand to our cross-compiler. Otherwise, just use the host versions.
objects := $(subst .c,.o,$(sources))
dependencies := $(subst .c,.d,$(sources))
Macro expansion of the sources variable, defined in the component subdirectory Makefiles, generating lists of corresponding .o object files and .d dependency files.
# Add any custom global cpp flags you might need here...
CPPFLAGS += -DSOMETHING=1
Somewhere to set any global defines.
# General flags and includes
include_dirs := ../libA
CFLAGS += -Wall -Werror
Configure global project include directories and base C flags. In this case, we warn on everything and consider warnings to be errors.
ifeq ($(DEBUG),1)
CFLAGS += -ggdb
else
CFLAGS += -O3
endif
If this was called as make DEBUG=1, then enable GDB debugging data with no optimizations. Otherwise for a standard build, turn on optimizations.
# Combine include dirs for the pre-processor
CPPFLAGS += $(addprefix -I,$(include_dirs))
Macro expansion to turn all include_dirs into -I arguments for the compiler.
vpath %.h $(include_dirs)
Tell the dependency checker where to find project-global include directories for header files.
.PHONY: library
library: $(library)
$(library): $(objects)
$(AR) $(ARFLAGS) $@ $^
How we build a library. This depends on the specific objects, so all this needs to do is archive them into a .a file.
.PHONY: program
program: $(program)
$(program): $(objects) $(libraries)
$(CC) -o $(program) $(LDFLAGS) $(objects) $(libraries) $(LDLIBS)
How we build a program executable. This depends on the specific objects, so all this needs to do is link them into the executable.
.PHONY: clean
clean:
$(RM) $(objects) $(program) $(library) $(dependencies)
How to clean a component directory. This will remove any objects, the executable, library, and dependency (.d) files.
ifneq "$(MAKECMDGOALS)" "clean"
-include $(dependencies)
endif
If cleaning, do not generate dependencies before cleaning as it would be pointless and wasteful.
%.c %.h: %.y
$(YACC.y) --defines $<
$(MV) y.tab.c $*.c
$(MV) y.tab.h $*.h
Recipe for compiling yacc files. Not sure exactly why this is here, or if it is really necessary.
%.d: %.c
$(CC) $(CFLAGS) $(CPPFLAGS) -M $< | \
$(SED) 's,\($*\.o\) *:,\1 $@: ,' > $@.tmp
$(MV) $@.tmp $@
How to generate dependency files. This is done by invoking the compiler and converting the output into a .d file corresponding to the .c file. This generates a fair bit of noise at compile-time, which can be suppressed by prefixing $(CC) and $(MV) with @ if desired.
version.mk
This file defines various version control information, exposing it for the build process and also the compiled software. An “auto number file” is utilized to append -devX to the build, for developer builds, or in the case of GitLab CI/CD pipelines, the commit hash.
# Basic application data
partNumber := 12345
appName := APPNAME
# Version number
versionMajor := 1
versionMinor := 0
# versionMinor with leading zeroes stripped
versionMinorEx := $(shell echo $(versionMinor) | sed -E 's/^0+([0-9]+)/\1/')
versionRevision := dev
versionAutoNum := 0
autoNumFile := .autonum
ifeq ($(origin CI_COMMIT_SHORT_SHA), undefined)
# If not building under GitLab, auto-increment the build number
num := $(shell cat ${autoNumFile})
ifeq ($(num),)
else
versionAutoNum := $(num)
endif
versionRevision := $(versionRevision)$(versionAutoNum)
else
# Otherwise, override version revision
versionRevision := $(CI_COMMIT_SHORT_SHA)
endif
# If debug build, append "-debug" to the string
ifeq ($(DEBUG), 1)
versionRevision := $(versionRevision)-debug
endif
CPPFLAGS += -DVERSIONMAJOR=$(versionMajor) -DVERSIONMINOR=$(versionMinorEx) -D$(appName)
export CPPFLAGS
Next we will break down the major pieces of this Makefile.
# Basic application data
partNumber := 12345
appName := APPNAME
As the comment says, basic application data. The part number and application name are used for firmware image filename construction, and the application name is passed to the compiler as -D$(appName).
# Version number
versionMajor := 1
versionMinor := 0
# versionMinor with leading zeroes stripped
versionMinorEx := $(shell echo $(versionMinor) | sed -E 's/^0+([0-9]+)/\1/')
versionRevision := dev
versionAutoNum := 0
autoNumFile := .autonum
Version number information. The compiler is given -DVERSIONMAJOR=$(versionMajor) and -DVERSIONMINOR=$(versionMinorEx). versionMinorEx is a copy of versionMinor with leading zeroes removed, as once you get to 08 this becomes an invalid octal number.
versionRevision defaults to “dev”, which will have versionAutoNum appended. This will be overridden with the git commit hash in the case of GitLab pipelines.
ifeq ($(origin CI_COMMIT_SHORT_SHA), undefined)
# If not building under GitLab, auto-increment the build number
num := $(shell cat ${autoNumFile})
ifeq ($(num),)
else
versionAutoNum := $(num)
endif
versionRevision := $(versionRevision)$(versionAutoNum)
else
# Otherwise, override version revision
versionRevision := $(CI_COMMIT_SHORT_SHA)
endif
Configure versionRevision based on developer build or GitLab pipeline.
# If debug build, append "-debug" to the string
ifeq ($(DEBUG), 1)
versionRevision := $(versionRevision)-debug
endif
If this is a debug build, include that in the firmware filename as well.
CPPFLAGS += -DVERSIONMAJOR=$(versionMajor) -DVERSIONMINOR=$(versionMinorEx) -D$(appName)
export CPPFLAGS
Expose version information via C pre-processor flags as described above.
Component Makefiles
Finally, the hard part is over. Next we configure the component-specific Makefiles. For the most part, these simply define the source files, target output name, and perhaps some additional compiler options. All of the heavy lifting is done by including common.mk.
exeA/Makefile
Executable A has a single .c file, and also depends on libA.
sources := \
exeA.c
libraries := ../libA/liblibA.a
program := exeA
# Add any other customizations here, such as CFLAGS
all: $(program)
include ../common.mk
First, sources is simply a list of all constituent .c files.
libraries is an optional variable that contains a list of libraries, either as relative .a files or -l options for ld. In this case we need libA.
program is the target executable name that the .c files will link to.
all: $(program) is the recipe to compile the target executable. Probably this could be generalized in common.mk somehow.
Finally, include common.mk for the rest of the build process.
exeB/Makefile
Executable B has a single .c file, with no library dependencies.
sources := \
exeB.c
libraries :=
program := exeB
all: $(program)
include ../common.mk
As described for exeA/Makefile, except no library dependencies.
libA/Makefile
This library consists of a single .c file.
library := liblibA.a
sources := \
libA.c
# Add any other customizations here, such as CFLAGS
include ../common.mk
As described for exeA/Makefile, except here we define library as the target .a file instead of program. No all recipe is required for libraries.
Adding New Components
Adding new components is relatively straightforward using this framework.
- Create a new component directory.
- Add source files as needed.
- Create a
Makefile in the component directory by copying an existing executable or library Makefile as appropriate.
- Update the new Makefile to define the
program, library, and sources variables as needed.
- Update the root/top-level
Makefile to make it aware of the new component.
This final step is the most complicated, as there is a fair bit of copy/pasting going on as can likely be improved in some way. Start by adding the component to the variable lists at the top of the file. Next, update the .PHONY: all and all targets in the same way the other components are listed. At the bottom of the file, list the components to enable Make recursion and dependency tracking as appropriate. Finally, update the install target to copy any output files into the firmware image.