|
|
Implementation Options and Specification for Shadow Compilation
This document describes three implementation options for providing and maintaining gcc support in ON. We've selected shadow compilation as the most cost-effective option. This option is described in detail, along with its costs and benefits.
Copyright 2006 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
ident "@(#)d-compilers-impl.txt 1.1 06/03/06 SMI"
OpenSolaris
SUPPORTED COMPILER POLICY, V1 [DRAFT 2]: IMPLEMENTATION ADDENDUM
1. Summary
This addendum to the OpenSolaris Supported Compiler Policy describes
three possible tools implementations which would satisfy the
requirements established by that policy[0]. The three approaches are a
modification to the compiler wrapper, a complete second build within the
same workspace, and the use of entirely distinct workspaces for each
compiler. Estimated costs are provided for each.
Regardless of which solution is ultimately selected, the Tonic team
will be responsible for implementation, initial testing, and integration
of that solution into ON.
2. Shadow Compilation
Shadow compilation is implemented as a modification to the compiler
wrapper (cw). The concept of primary and shadow compilers is
introduced; the primary compiler is specified by one of the -_cc, -_gcc,
-_CC, or -_g++ options and produces the binary objects, preprocessor
files, and other build products as in the existing cw implementation.
The compiler complementary to the primary compiler which accepts the
same input language(s) is defined to be the shadow compiler. The shadow
compiler will be invoked in parallel with the primary compiler, with the
following modifications and exceptions:
(a). If CW_NO_SHADOW is present in the environment, cw will
invoke only the specified primary compiler.
(b). If the command line given to cw includes '-o ' or
'-o', is replaced by a temporary file on the command line
given to the shadow compiler. Otherwise, a -o option is added, along
with a temporary filename, to the shadow compiler's command line. In
either case, this temporary file is unlinked by cw after the shadow
compiler terminates.
(c). If the command line given to cw does not include one of
'-c', '-S', '-P', or '-E', the shadow compiler is not invoked. That is,
no attempt is made to link previously compiled objects into a library or
executable.
(d). All instances of environment variables whose names begin
with 'SUNPRO_DEPEND' are removed prior to execution of the shadow
compiler. This prevents corruption of make's dependency tracking files
by the shadow compiler.
(e). The command line for the shadow compiler is otherwise
translated as it would be if the shadow compiler were the primary
compiler.
Standard output and standard error for both the primary and shadow
compiler, if invoked, are left unchanged by cw, so that warnings and
errors for both are displayed to the user or logged, as appropriate. cw
terminates only after both the primary and shadow compiler, if invoked,
have terminated. cw's exit status will be 0 if both primary and shadow
compilers, if invoked, exit with status 0, or nonzero otherwise.
2.1. Build Time Cost
2.1.1. Testing Methodology
The same 2 machines were used for all tests, and the same system
software was used for all tests.
"Standard cc" and "Standard gcc" results are taken from the 3
February automatic nightly builds of /net/tonic-gate/tonic/gcc-gate
using the default Studio compiler and gcc 3.4.3 from /usr/sfw,
respectively. Nightly environment files used for these four builds can
be found at /net/tonic-gate/tonic/gcc-gate/public/env/.
"Combined-cw" results are taken from nightly builds run on 2
February ("cc") and 3 February ("no shadow") using the default Studio
compiler (and gcc 3.4.3 from /usr/sfw, if shadow compilation is
enabled). Nightly environment files used for these four builds can be
found at /home/wesolows/env/gcc.{x86,sparc}. The only difference
between the "cc" and "no shadow" builds is the existence of the
CW_NO_SHADOW environment variable for the "no shadow" build, which
causes cw to fall back to its traditional behaviour by invoking only the
primary compiler. One would expect these times to be nearly identical
to "Standard cc" times.
Differences between the built workspaces were no greater than one
day's putbacks; the "Standard" workspaces are sync'd with gcc-gate
daily, while the "Combined-cw" workspaces were sync'd on 1 February
only.
Both machines were otherwise idle during all builds.
"user+sys" times include both DEBUG and non-DEBUG time. The reason
for including this will be made clear in the discussion.
2.1.2. Results
Table 1: Raw times (wall)
x86 (tonic-gate, 4-way v40z, snv_30)
DEBUG non-DEBUG lint
Standard cc 25:36 52:28 51:22
Standard gcc 16:48 37:42 50:00
Combined-cw (cc) 33:22 1:07:18 50:17
Combined-cw (no shadow) 28:16 52:25 59:17
SPARC (stomper, 8-way x2 V890, fma_gcc[cc] (onnv_33))
DEBUG non-DEBUG lint
Standard cc 41:28 1:04:42 42:22
Standard gcc 34:03 56:21 43:18
Combined-cw (cc) 40:42 1:08:07 43:15
Combined-cw (no shadow) 42:22 1:06:52 43:45
Table 2: user+sys times
x86
DEBUG+non lint
Standard cc 3:08:06 1:46:46
Standard gcc 2:11:03 1:44:25
Combined-cw (cc) 4:14:12 1:44:24
Combined-cw (no shadow) 3:06:38 1:59:00
SPARC
DEBUG+non lint
Standard cc 6:46:28 1:32:24
Standard gcc 6:22:50 1:33:23
Combined-cw (cc) 10:31:50 1:33:00
Combined-cw (no shadow) 6:43:06 1:33:00
Table 3: Estimated tax rate as a fraction of "Standard cc"
x86
Shadow gcc lint
Wall (4-way) 0.29 0.66
user+sys 0.35 0.57
SPARC
Shadow gcc lint
Wall (16-way) 0.02 0.40
user+sys 0.55 0.23
2.1.3. Discussion
This comparison offers strong evidence that the tax associated with
shadow compilation is, surprisingly, quite small indeed and in some
build environments may even be in the noise. A reasonable
interpretation holds that since the shadow compilation occurs in
parallel with primary compilation, total build times will be less
affected by it on machines with many otherwise idle slower CPUs (where
build time is governed mainly by serialized java compilation, not C
compilation) than on machines which are fully utilized or have fewer,
faster CPUs. The shadow compilation tax would therefore be at a maximum
on a uniprocessor system; we can arrive at a reasonable worst-case
estimate of this tax on a uniprocessor build system by using the
user+sys times.
SPARC systems with 2 or fewer CPUs are rarely used to build ON; all
SPARC public build machines for ON Nevada have at least 6 cores; most
have 12 or more. Therefore, real-world tax rates for shadow gcc
compilation are likely to range between 0 and 0.35 against 0.40 to 0.66
for lint. Like lint, shadow gcc can be enabled and disabled easily in a
nightly(1) configuration, can be run selectively within each subtree
from the command line, and does not interfere with incremental builds.
Unlike lint, shadow gcc requires zero additional disk space for each
build and does not require project teams to introduce or use additional
makefile macros or targets.
2.2. Disk Space Costs
Because the shadow compiler's additional output is immediately
discarded, the marginal disk space requirement for this implementation
is negligible.
2.3. Active Engineer Time Costs
Shadow compilation is fully automatic and requires no configuration
or additional manual activity. The only active engineering cost
associated with this solution is applicable to all three solutions: time
spent diagnosing and correcting errors or warnings triggered by the
secondary compiler. This time is expected to vary greatly from project
to project.
2.4. Risks and Drawbacks
The most significant drawback to this approach is that linking is
never done by the shadow compiler or on its products. This affects a
number of commands built from a single source file; there would be no
automated way to ensure that these remain gcc-clean. It may in some
cases be possible to avoid this problem by executing the shadow compiler
for linking when it is known that the compiler writes only to the
designated output file or safely-named temporary files.
3. Second Internal Build
This approach is similar to the Export Source Product build in that
it creates a second source tree within the same build workspace. That
second tree is then built with the secondary compiler. This approach
produces a set of build products for each compiler that can be installed
and tested. This approach requires no changes to any tools except
nightly. When given the 'g' option, nightly creates and builds the
secondary workspace. This workspace is built in both DEBUG and
non-DEBUG variants if nightly is so configured.
3.1. Build Time Costs
On a 4-way Opteron, a full DEBUG+non-DEBUG build was measured at
2:51:36 wall time without the secondary compiler and 4:35:07 with it.
The time cost is therefore slightly higher than that of shadow
compilation. We expect that the cost in other build environments would
be similar as the time required to perform builds with gcc is
well-known.
3.2. Disk Space Costs
This method requires a full copy of the source and binary products
and therefore has a hefty cost in space. Measurements on x86 indicate
that secondary compilation adds about 3.0 GB to each full build on x86,
or almost 100% of the primary-compiler-only space requirement. We would
expect SPARC costs to be slightly higher in absolute terms and similar
relative to the size of a primary-compiler-only build.
3.3. Active Engineer Time Costs
This solution is automatic and requires only that engineers add a
single flag to their nightly environment files. Engineers performing
manual builds with dmake will have to find an alternate means of
verifying that their code is acceptable to the secondary compiler, such
as performing a second manual build or using nightly before putback.
Otherwise, this solution requires only that engineers identify,
diagnose, and correct any warnings or errors generated during the
secondary build.
3.4. Risks and Drawbacks
This solution is considerably safer than shadow compilation; it does
not alter the semantics of the compiler wrapper and make it much less
likely that objects may be accidentally overwritten. It also builds all
binaries and thus offers more complete coverage. The main drawback to
this approach is massive disk space consumption; a lesser unfortunate
side effect is that it provides no help to engineers wishing to make a
secondary compiler pass over only part of a workspace.
4. Second Workspace
The simplest and most obvious solution, this approach requires that
each engineer build his or her workspace a second time on each supported
architecture. Much as each architecture's build must be done in a
separate workspace, so too would each compiler's. This solution
requires no modification to existing tools, and all normal build
products would be generated. A single environment variable (__GNUC)
generally controls which compiler is used for a particular build.
4.1. Build Time Costs
The cost of this option is approximately equivalent to that of a
second internal build as described in section 3.
4.2. Disk Space Costs
This option approximately doubles the disk space requirements for a
complete build. Of course, builds could be done with the primary and
secondary compiler on separate hosts and/or at separate times so that at
any given time the disk space consumed might be less than that amount.
4.3. Active Engineer Time Costs
Engineers will be required to perform steps similar to those already
required for building on multiple architectures, such as creation of
additional environment files. No new tasks are associated with this
solution, however; the amount of active time spent on doing builds would
simply be doubled. We estimate that this amounts to no more than a few
minutes a week for each engineer.
4.4. Risks and Drawbacks
Because this solution requires no tools changes and relies only on
well-understood existing behaviours, it presents the fewest risks.
However, it requires the greatest amount of build time and disk space,
and requires an (admittedly trivial) set of additional steps of each
engineer. Without automated tool assistance, it's likely that secondary
compilation would be overlooked by some fraction of implementors.
|