-
Notifications
You must be signed in to change notification settings - Fork 55
/
README
337 lines (257 loc) · 13.7 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
Darwinbuild
Updated 2009-02-17
Kevin Van Vechten <[email protected]>
William Siegrist <[email protected]>
1. Overview
1.1 Availability, Bug Reports, Contributions, and Discussion
2. Installation
2.1 Creating the Build Directory
3. Building Darwin Projects (darwinbuild)
3.1 Install Headers
3.2 Chroot Builds
4. Build Environment Variables
4.1 Build Aliases and Alternate Targets
5. Tips and Techniques
5.1 Private Headers
5.2 Build Tools
5.3 Static Libraries
5.4 Alternate BuildRoot Storage
Appendix
A. darwinxref
===========
1. Overview
===========
Darwinbuild is a collection of tools that assist compilation of
the many projects contained in Darwin[1], the open source base of Apple's
Mac OS X operating system[2]. Apple publishes the sources of these projects
in an archive format (.tar.gz). An archive is published for each project
version on Apple's site[3].
These tools will provide the proper build environment as well as help to
resolve any necessary dependencies prior to building.
[1] <http://developer.apple.com/darwin/>
[2] <http://www.apple.com/macosx/>
[3] <http://www.opensource.apple.com/>
1.1 Availability, Bug Reports, Contributions, and Discussion
Due to the constant development of darwinbuild, the only version you should
be working with is trunk of the subversion repository[1]. Trunk should
always be stable enough for building projects. If you find a problem or have
a enhancement in mind, file a ticket at the website[2].
Discussion about darwinbuild, or building Darwin projects in
general, should take place on the darwinbuild-dev[3] mailing list.
[1] <http://svn.macosforge.org/repository/darwinbuild/trunk>
[2] <http://darwinbuild.macosforge.org/report/1>
[3] <http://lists.macosforge.org/mailman/listinfo/darwinbuild-dev>
===============
2. Installation
===============
You can install the latest version of darwinbuild using MacPorts.
% port install darwinbuild
Or, you can also install by checking out the source and building
darwinbuild yourself. The included Xcode project will compile the tools and install
them into the location specified by the PREFIX environment parameter. If the
PREFIX parameter is not specified, the files will be installed into /usr/local.
By default, darwinbuild will build for x86_64 and i386. You can override this with
the ARCHS environment variable. However, building trunk for ppc is not supported
since trunk is targeted for Darwin 10. The release branches for Darwin 8 and 9 do
support building for ppc.
To build and install darwinbuild to the boot partition:
% cd <path to top of source>
% sudo xcodebuild install DSTROOT=/
2.1 Creating the Build Directory
After installation, you must initialize the build directory using darwinbuild.
Assuming you wanted to build projects from 9G55 in your home directory:
# mkdir ~/9G55
# cd ~/9G55
# sudo -s
# darwinbuild -init 9G55
Creating build root disk image ...
Attempting to download http://svn.macosforge.org/repository/darwinbuild/trunk/plists//9G55.plist ...
Download complete
Attempting to download http://svn.macosforge.org/repository/darwinbuild/trunk/plists//9F33.plist ...
Download complete
...
Attempting to download http://svn.macosforge.org/repository/darwinbuild/trunk/plists//9A581.plist ...
Download complete
Initialization Complete
# ls
.build Headers Logs Roots Sources Symbols
After initialization, the build directory will contain the following:
.build contains private data for the darwinbuild system
Headers contains the resulting header files from previous builds
Logs contains logs of previous build attempts
Roots contains the finished products of previous successful builds
Sources contains sources downloaded from the web
Symbols contains debug symbol versions of previous build products
When using the darwinbuild command, it is necessary that the current
working directory be this build directory, or alternatively,
that the DARWIN_BUILDROOT environment variable be set to the absolute
path of the destination directory. The example usages in this document
assume the current working directory is this build directory.
=========================================
3. Building Darwin Projects (darwinbuild)
=========================================
To build a Darwin project, for example xnu, the darwinbuild script can be
used in the following manner:
# darwinbuild xnu
The darwinxref tool is consulted to find the version that corresponds to the
build specified when the build directory was initialized. It is necessary
to run the darwinbuild tool as root so that projects can set the proper
ownership and permissions on the build results.
darwinbuild first looks in the Sources directory for a directory containing
the sources to be built (Sources/xnu-517.11.1), or a .tar.gz archive
containing the sources (Sources/xnu-517.11.1.tar.gz). If neither is found,
darwinbuild will attempt to download the sources from the web.
If it does not already exist, a BuildRoot directory will be created. This
is where the build will actually take place. During the build, darwinbuild
will change the root directory to BuildRoot (see the chroot(8) man page for
details). Darwinbuild is capable of copying the required tools, libraries
and headers from the Roots directory into the BuildRoot prior to building.
If a necessary dependency is not found in the Roots directory, it will be
downloaded from the web.
The build output will be written to the console, and additionally logged
into a file in the Logs directory. The above example produces the
following file:
Logs/xnu/xnu-517.11.1.log~1
The ~1 indicates that this log file corresponds to the first attempt to
build xnu version 517.11.1. Each subsequent attempt will add one to this
build number.
If the build succeeds, the finished product will be copied out of the
BuildRoot directory and into the Roots directory:
Roots/xnu/xnu-517.11.1.root~1
After the copy, darwinbuild traverses the directory and records all files
found in the darwinxref database. This makes it possible to query which
project a file is produced by. When a Mach-O executable, library, or bundle
is found during the traversal, the dynamic library load commands are recorded
in the darwinxref database. This makes it possible to query which additional
projects are required to run an executable produced by the project.
Additionally, any products containing debug symbols will be placed into
the Symbols directory.
3.1 Install Headers
Passing the -headers flag to darwinbuild will start an alternative build
style where only the project's headers are produced. This is useful for
working around various circular dependencies while compiling low level
projects like xnu, cctools, Libc, etc.
The results of a -headers build are placed in the Headers directory, in
the same style as the Roots directory for a regular build. For example:
% sudo darwinbuild -headers IOKitUser
% ls Headers/IOKitUser/IOKitUser-184.hdrs~1
3.3 Logging Dependencies
Passing the -logdeps flag to darwinbuild will build the specified project
while recording the paths of all files openened and all executables invoked.
These paths are written into the Logs directory, and if the build is
successful, will be imported into the darwinxref database.
==============================
4. Build Environment Variables
==============================
At the minimum, the build environment consists of the creation of SRCROOT,
OBJROOT, SYMROOT, and DSTROOT environment variables. These variables contain
an absolute path to a directory which must exist prior to invoking the Makefile.
User configurable environment variables, such as the target architecture,
can be set in the build plist file that is loaded via loadIndex. The
darwinbuild script creates the aforementioned directories, sets the environment
variables, and issues the appropriate make command.
Within the plist file, the RC_ARCHS variable indicates which target
architectures should be included in the build. Currently only ppc and i386
are available. For each architecture in RC_ARCHS, and equivalent RC_{arch}
should be set to YES (i.e. RC_ARCHS="ppc" RC_ppc=YES).
The RC_NONARCH_CFLAGS specify additional command line flags to be passed to the C
compiler during the build. On Darwin, the -no-cpp-precomp flag should be
passed since cpp-precomp is not available. During the build, the RC_ARCHS and
RC_NONARCH_CFLAGS variables will be combined to create the RC_ARCHS environment
variable; a -arch flag will be appended for each architecture listed in RC_ARCHS.
The MACOSX_DEPLOYMENT_TARGET variable should match the major Mac OS X release
that is being targeted, such as 10.2 or 10.3.
4.1 Build Aliases and Alternate Targets
Some projects may produce different results based on the contents of
the RC_ProjectName variable. When the same source archive is used
to create more than one component of Darwin, it's referred to as a build
alias. Because of this, it is important to always provide accurrate
project name and version information in the environment. By default,
darwinbuild sets these variables appropriately.
By default, "install" (for non-headers builds) is the first argument passed
to the make tool (gnumake or xcodebuild). However, some projects produce
different results based on alternate targets. If the project in the property
list contains a "target" attribute, that string will be passed instead.
======================
5. Tips and Techniques
======================
5.1 Private Headers
Many open source header files are not present in a standard Mac OS X install.
These "private headers" are not needed by application developers, and are not
part of any SDK officially supported by Apple. However, Darwin projects are
part of the operating system itself and these headers are required. The
darwinbuildheaders command uses the darwinxref tool and darwinbuild -headers
commands to produce all header files for a given darwin release.
% bin/darwinbuildheaders
The resulting header files from each project are placed into the Headers
directory. The headers can be subsequently installed into the BuildRoot
directory using the installheaders command.
% bin/installheaders
By modifying the build plist file, it is possible to include these headers
instead of the default system headers, giving more accurate build results and
avoiding failures. Edit the RC_NONARCH_CFLAGS variable to provide the following
include paths to gcc (substituting the actual installation directory).
-I/usr/local/darwinbuild/BuildRoot/usr/include
-I/usr/local/darwinbuild/BuildRoot/usr/local/include
5.2 Build Tools
In addition to private headers, there are numerous command line tools which
are used exclusively to build Darwin projects. These tools are placed in
/usr/local/bin. For example, building xnu requires kextsymboltool,
relpath, and decomment.
5.3 Static Libraries
Several projects require static libraries from other projects. These libraries
are usually placed in /usr/local/lib/system. The most notable examples are
xnu's usage of libkld.a which is produced by cctools_ofiles, and Libsystem's
use of many static libraries from Libc, Libinfo, and more.
5.4 Alternate BuildRoot Storage
By default, "darwinbuild -init" will setup your environment to use a HFSX
sparsebundle to host your BuildRoot. A sparsebundle is used to work around
problems with using xcodebuild inside of a chroot. There are two other
storage methods to choose from: NFS Loopback and Legacy Directory.
1. NFS Loopback (ex: darwinbuild -init 9G55 -nfs)
The NFS loopback method places a directory under .build and adds that
directory to /etc/exports. When you try to build a project, darwinbuild
will mount the NFS export on localhost. The NFS mount also works around
the xcodebuild problems.
2. Legacy Directory (ex: darwinbuild -init 9G55 -nodmg)
This is just the way darwinbuild worked before the sparsebundle and NFS
support was added. It just makes a directory for BuildRoot and builds
on whatever filesystem you are on at the time. If you are on HFS or ZFS,
this storage will cause problems for Xcode-based projects.
=============
A. darwinxref
=============
The darwinxref tool allows you to query which source version of a Darwin
project is present in a particular Mac OS X build. It also stores information
about what dependencies a particular project has, and what files the project
produces. As each Mac OS X release is made available, Apple publishes a
property list file containing the project names and versions in that release.
These property lists are read by the darwinxref tool to seed its internal
database. Property lists are available via Subversion[1].
The darwinxref tool uses SQLite[2] to maintain its database of projects,
versions, files and dependencies. This distribution includes a pre-built
version of the sqlite library in the file libsqlite3.a, and its associated
header file, sqlite3.h.
[1] <http://svn.macosforge.org/repository/darwinbuild/trunk/plists>
[2] <http://www.sqlite.org/>
An example of downloading, installing, and querying a property list:
% curl http://svn.macosforge.org/repository/darwinbuild/trunk/plists/7U16.plist > \
plists/7U16.plist
% bin/darwinxref loadIndex plists/7U16.plist
268 of 268 projects loaded.
% bin/darwinxref -b 7U16 version xnu
xnu-517.11.1
To list all projects in a build, use the special project name '*':
% bin/darwinxref -b 7U16 version '*'
To register the results of a previous build with the database use the
register command. Note this is done automatically by darwinbuild:
% bin/darwinxref register adv_cmds 63 Roots/adv_cmds/adv_cmds-63.root~1
/bin
/bin/ps
/bin/stty
...
To find which project produces the 'whois' command by searching the
list of previously registered files:
% bin/darwinxref findFile whois
adv_cmds-63:
/usr/bin/whois