The purpose of this One True Makefile project is to provide a template for projects that use Autoconf, Libtool and Make, but not Automake. (I prefer not to use Automake, although obviously I like some of its brothers.) It is placed in the public domain, since I don't see the point of making a copyleft build process template.
If you're going to use this template project, you will probably want
to already have a working knowledge of how Autoconf, Libtool and Make
work, although if you're still learning, you might like to use the One
True Makefile as a working example. (This README won't be much of a
tutorial, though.) In order to understand the logic of e.g. what goes
in share
directories and what goes in etc
directories, you should
probably also have a basic familiarity with the Linux Filesystem
Hierarchy Standard.
Rules are set up with C++ in mind, although it should be easy to add rules for other languages.
The One True Makefile's build philosophy is based on the one from Peter Miller's "Recursive Make Considered Harmful" paper, although it also has some other nice features. Specifically, the One True Makefile provides:
-
Modularity.
-
Incremental builds. Make knows about all the dependencies, so everything Just Works with a minimum of compiling. There is no need to run
make clean
every single compilation cycle. -
Automatic
#include
directive dependency generation. (You need to have a compiler that accepts the-MM
flag, like gcc.) However, please note that there's no way for the computer to know that e.g. your foo program depends on the baz module, and also on libz.so, so you still have to manually specify library dependencies and link targets in the module.mk files. -
Avoidence of recursive Make. (See Miller's paper; it's easy to find with google.)
-
Parallel building using Make's
-j
flag. This is robust and works well with incremental builds thanks to point 4). -
Unit tests, build artifacts, and final output all go in their own dynamically-generated subdirectories. Because of Libtool, you can run executables in the final output (staging) directory in-place, and shared libraries will Just Work, even if you haven't run make install.
-
A
make uninstall
target. (I wish more autotools-based projects provided that.) -
A unit testing framework that leverages Python's unittest library. (Yes, unittest is designed for Python projects, but it's pretty trivial to use the subprocess module to wrap C and/or C++ programs, and Python is a nice "glue" language to use if your testing process gets sophisticated. For example, you may want to start doing things like starting multiple services before running for a single test, and stopping them afterward, and Python's
unittest
framework provides hooks for that.) -
Flexibility, in that I've tried to write everything in such a way that, if you don't like what the One True Makefile is doing (blasphemer!), it should be obvious how it works and how you can change it.
-
I've included a copy of Lee Thomason's excellent TinyXML2 library, since I like using xml for config files. If you don't like and/or need it, removing it is as easy as removing
tinyxml2
from the modules section ofMakefile.in
, and runningrm -rf tinyxml2
. (This is an example of the modularity of which I spoke, up in point 1).) Note that the actual library will be calledlibtinyxml2-otm.so
, with a-otm
at the end, in order to avoid conflicts with any otherlibtinyxml2.so
shared library files that may be installed. (Again, if you don't like the-otm
suffix, it's easy to change; in fact, you should probably change that suffix to something that calls to mind the name of your project.)
Note that I have tried to provide the above benefits using minimalist approaches; for example, I use implicit rules whenever possible. This makes the build machinery more readable, easier to understand and easier to maintain. If you feel overwhelmed reading the Makefile.in, I highly recommend reading the Recursive Make Considered Harmful paper by Miller. It's easy to find via Google.
Please note that, if your project provides e.g. a
MyProject::Foo::Foodle
class, then you should specify its interface
with an #include
directive that mirrors the namespace hierarchy,
like so:
#include "MyProject/Foo/Foodle.h"
Thus, if the interface file is going to end up installed in a normal
way, at /usr/local/include/MyProject/Foo/Foodle.h
, then the people
who use your library only have to give the compiler a
-I/usr/local/include
flag, and in fact compilers usually already
have that path specified by default, so your users actually don't have
to do anything. The #include
directives mirror the namespace, and
compiling Just Works.
Please note that some people use a dirty scheme whereby their include
directive for a file installed somewhere like
/usr/local/include/MyProject/Foo/Foodle.h
looks like this:
#include "Foodle.h" // Bad!
This means that their users have to pass the compiler a
-I/usr/local/include/MyProject/Foo
flag, since the compiler doesn't
know about the MyProject/Foo subdirectory by default. In fact, the
user has to include another compiler flag for every dependency that
does this.
But you don't do that, because you're better than that.
(Please note that my rant is somewhat spoiled by the fact that the
included copy of TinyXML2 (by Lee Thomason) does not use this
project's namespace (i.e. something like
OneTrueMakefile::TinyXML2::TiXmlDocument
), since I didn't write the
(excellent) TinyXML2 library, and instead I just dropped it into this
One True Makefile project. I didn't change how namespaces work in
TinyXML2 because I didn't want to muck around in the code too much,
for fear of introducing a subtle bug somewhere.)
This template setup should build a minimal example project right out of the box, using the standard Autotools commands. That is, to build the minimal example, run:
autoconf
./configure
make
The included minimal example doesn't require any extra libraries, but
as an example, if your project were to require some libraries that
happen to be installed in your $HOME/.local
directory, you would
run:
autoconf
LDFLAGS=-L$HOME/.local/lib CPPFLAGS=-I$HOME/.local/include ./configure
make
After the the build process completes, the final build targets should
be sitting in Linux-stage
. (I'm assuming you're running Linux, but
if not, the slightly different name of the actual staging directory
should be obvious. Also, I've named the build, staging and test
directories based on the output of uname
so that you can have
multiple architectures building in the same network-mounted directory,
but if you don't like that naming convention, it's easy to change.)
You can verify everything is working by running:
./Linux-stage/bin/hello
Note that the hello
program successfully finds the libfoo.so
shared library, thanks to Libtool.
You can build the unit tests by running make tests
, and you can run
them by running make runtests
. The installation process will not
install unit tests into the install prefix with all the other build
targets; unit tests are in their "own little world."
You could also do a make install
and a make uninstall
, if you're
feeling saucy. If experimenting with /usr/local gives you the creeps,
you could run configure
with a --prefix=$HOME/tmp/local
flag to
install everything into a temp directory.
Now: the point of this template is to start you off with a lot of
boilerplate written. That doesn't mean that you won't have to
maintain your build process; this will just start you off with some
nice features like make uninstall
, so that you don't have to write
another uninstall
rule every time you start a new project. In order
to adapt this template for your own project, you will probably want to
do the following:
-
Replace this
README.md
file with one that is appropriate for your project. You should probably also replace (or delete) theCOPYING
file, and maybe theINSTALL
file, if you've seen fit to modify the build/install process. -
Add in your own modules. (You can replace the
foo
module and/or thetinyxml2
module, and use them as templates. It's unlikely the "Hello, world!" functionality provided by the foo module will be super-useful for your project, anyway.) -
Edit
Makefile.in
:-
Change
pkg_name := one_true_makefile
to refer to the name of your project. -
Change the
modules :=
area to reflect the names of your modules. For example, if you killed thefoo
module, kept thetinyxml2
module, and added two more modules called bar and baz, you would change this section to:modules := \ tinyxml2 bar baz
-
Change the
testmodules :=
area to reflect the names of your unit test modules. For example, if you used thetestfoo
module as a template to make atestbar
module, you would change this section to:modules := \ testbar
-
-
Edit your
module.mk
files to list all of the files you want to be listed as dependencies, and all the files you want to be affected by themake install
target. Everything uses the bottom source directory as the base directory, and the One True Makefile is set up to mirror the organization of the staging directory. Things inetc
orshare
will just get copied around by Make, since they don't need to be compiled.For example, if you made a
baz
module with a config file atbaz/etc/my_project/baz/bazqux.conf
, then you want to make surebaz/module.mk
has this line:baz_etcs := $(stage_dir)/etc/$(PKG_NAME)/baz/bazqux.conf
On the other hand, maybe you want your
bazqux.conf
file to go in the bottom ofetc
, instead of in a series of subdirectories. In that case, then in yourbaz
module the config file would be located atbaz/etc/bazqux.conf
, and the line inbaz/module.mk
would look like this:baz_etcs := $(stage_dir)/etc/bazqux.conf
In the first case, running
make install
with the default/usr/local
value as your install prefix would install that config file to/usr/local/etc/my_project/baz/bazqux.conf
. In the second case, it would get installed to/usr/local/etc/bazqux.conf
. That is, in yourmodule.mk
file, you specify build targets in the staging directory (even if the "building" is just trivial copying). The Makefile manages everything with implicit rules, and works such that the organization of your module, the organization of the staging directory, and the organization of the final install directory (e.g./usr/local
) will all mirror each other.Note that your
module.mk
files will probably also need to include logic regarding linking, near the bottom. Be sure to include order-only rules (with a vertical bar) to tell Make to create relevant directories. Thetest_foo
module has an example of some slightly nontrivial linking logic, near the bottom of the file.Also note that
*.h
and*.hpp
files that aren't going to be installed should not be mentioned in themodule.mk
files. The dependency autogeneration feature from gcc will deal with them, and since they aren't going to be installed, there's no reason to list them manually anywhere. (Also note that.h
and.hpp
files that are not going to be installed should go in src/ directories, and not in include/ directories.) -
If you're keeping tinyxml2, and you've renamed your package from something other than
one_true_makefile
(which is understandable), then you'll need to edit the tops of the tinyxml2 source file to reflect that. For example, if your new project is called MyProject, then in tinyxml2.cpp and xmltest.cpp, you'll have to change the lines that say#include "one_true_makefile/tinyxml2/tinyxml2.h"
to
#include "MyProject/tinyxml2/tinyxml2.h"
-
Rename all the directories named
one_true_makefile
to your project's name. -
Once your modules are set up, run
autoscan
, and replaceconfigure.ac
. It is convenient to have your own Autoconf tests in a separate file (or files), so it is probably a good idea to put your own Autoconf tests in theproject.m4
file, and add a line like this to your newconfigure.ac
:m4_include([project.m4])
The provided
project.m4
file has a handful of tests you might find useful.One slightly non-standard aspect of the One True Makefile setup is that I've put
config.h
in subdirectory calledinclude
, instead of with all the build stuff in the base directory, so you will also want to modify theAC_CONFIG_HEADERS
andAC_CONFIG_SRCDIR
lines in a newly generatedconfigure.ac
to this:AC_CONFIG_SRCDIR([include/config.h.in]) AC_CONFIG_HEADERS([include/config.h])
Our default version of
configure.ac
already has these changes made, but if and when you runautoscan
and replaceconfigure.ac
, you'll have to redo them. (Minimizing the amount of stuff you have to redo after runningautoscan
is the purpose of theproject.m4
scheme.)If you don't like the
include/config.h
scheme, it's easy to change; you just have to change some lines inMakefile.in
, near the configure rules, along with a line near the top ofproject.m4
. (In both cases, it should be obvious what to change.) Also,foo/module.mk
has a-Iinclude
compiler flag, so if you don't like theinclude/config.h
scheme, your modules should specify-I.
instead. (The dot after the -I is not a typo. Since all paths are relative to the base directory, whereMakefile.in
is, the dot refers to that base directory.) -
Run
autoheader
.
Note that, while make clean
works automagically, there is also a
make clobber
target. This removes final build targets and
dependency (*.d
) files. It also removes unit test build targets;
however, that part isn't handled automatically for you, in the sense
that the logic for removing unit test build targets has to be manually
added to the bottom of the unit test modules' module.mk
files.
(Check out the bottom of test_foo/module.mk
to see what I mean.)
If something is screwed up, sometimes Make will silently die, rather
than giving a helpful error message. You can tell whether Make is
silently dying by typing make
; if it just returns, without saying
make: Nothing to do for 'all'.
or anything else, then it is silently
dying. Another obvious way is to see if there is a problem is to
check the return code by running:
make || echo err
If it says "err", the return code of make
incicated that something
went wrong, even if you don't have an error message.
Trying passing various debugging flags to make
, such as make -d
,
make --debug=a
, make --debug=b
, make --debug=v
, make --debug=i
, make --debug=j
and/or make --debug=m
may be helpful.
Sometimes renaming files can cause problems, in that this causes the
dependency files (i.e. the *.d
files) to be inaccurate. A way to
fix this is to delete all the dependency files so that they will be
regenerated correctly. One way to do this is to run:
find . -type f -name "*.d" -exec rm -vf {} \;
You may also want to start fresh by deleting all the output of the entire build process by running:
rm Linux-* -rf
(If you're not on Linux, it should be obvious how to change this line.)
Often, the problem will be in a specific spot in Makefile.in
or one
of the module.mk
files. A way to identify the problem area is to
edit the modules :=
area of Makefile.in
and remove one module,
leaving all the other modules in the list. If you can get it to say
make: Nothing to do for 'all'.
, then the module you've removed is
the problem child. Once you've narrowed the problem down to one file,
it should be easier to figure out what's wrong. (Of course, if one of
the remaining modules depends on the removed module, Make may die with
an error message without getting to the end. In that case, you may
have to play games with removing multiple modules at once.)
Another helpful source of information is to look in the Linux-build
directory (or whatever you've named it). For example, if
foo/module.mk
contains the problem area, and build/foo/foo.lo
exists, but Linux-stage/bin
does not contain the foo
executable,
then Make apparently died before creating the foo
executable but
after successfully compiling foo.lo
. Maybe that would suggest
something to you. (This isn't definitive-- maybe the foo
executable
also has other dependencies, and one of those is causing the problem.)
Problems with Make can be opaque sometimes. At one point I had a
problem with the include/stamp-h.in
logic in Makefile.in
, and it
seemed like Make was dying in one of the modules, in a completely
unrelated spot, since that module happened to look at
include/config.h
. Again: using the One True Makefile as a template
does not save you from ever having to debug Makefiles again. It just
saves you from having to write a lot of boilerplate every time you
start a new project.