Flag Day: ON tree split into open and closed source
Date: Wed, 21 Dec 2005 17:53:48 -0800
From: Mike Kupfer <mike dot kupfer at sun dot com>
To: on-all at eng dot sun dot com, onnv-gate at onnv dot eng dot sun dot com
Subject: Flag Day: ON tree split into open and closed source
My recent putback for
6316770 split the tree into open and closed trees
6303550 xencrypt, xdecrypt no longer mangled
6321333 Install.sh is over-zealous looking for kmods
6289535 build_cscope complains a bit on x86
causes a flag day for tools and incremental builds. Furthermore, it
introduces a big change in the organization of the ON source; you will
need to understand and deal with this change.
The rest of this email is organized as follows:
* what this putback did
* what you need to know going forward
* gotchas
-- what this putback did --
This putback split the source tree into two trees: usr/src and
usr/closed. usr/src contains the code that we are making available in
OpenSolaris. usr/closed contains everything else.
Much of the code in usr/closed is there for legal reasons. Note that
the contracts that require the code to be closed often contain a
confidentiality clause. This means that when discussing code in
usr/closed, you should avoid mentioning the names of third parties who
appear tied to the code.
usr/closed also contains some open code that can't build in
OpenSolaris because of dependencies on code in usr/closed. And it
contains some drivers that were determined to be open after the cutoff
for this putback. Those drivers will get moved to usr/src over the
next few builds.
This putback also updated the ON tools to understand the new
organization. See "gotchas", below. Once you have the new tools,
nightly(1) will build the entire tree, and the resulting proto area
should look just like the proto area did before the source was split.
The makefiles have been organized according to the following strategy:
if an entire tree is in usr/closed, its makefiles are there, too. If
only part of the tree is in usr/closed, the makefiles live in usr/src.
The build is driven from usr/src, and it jumps into usr/closed as
needed. The upshot is that you can still build the entire tree with
"cd $SRC; dmake install".
This putback is not the final OpenSolaris change that we'll be making.
Future putbacks will include
- support for OpenSolaris non-debug builds
- support for automated generation of OpenSolaris deliveries from
the ON gate
- moving code from usr/closed to usr/src, if we have determined that
it can be opened
- package reorganization
Those putbacks should be a lot less disruptive than this one.
-- what you need to know going forward --
Ever since the OpenSolaris launch in June, we have been publishing the
open ON code externally. This means that projects need to file any
patent paperwork prior to integration into ON. Hopefully, everyone is
already aware of this, but this putback seemed like a good opportunity
to remind you all. :-)
As mentioned above, usr/src now contains the files that we are
delivering externally. As a general rule, all new code should live in
usr/src. If you think you have an exception, please contact the
Solaris PAC. Note that new platform code goes into usr/src, even for
unannounced Sun products.
The usr/src tree currently has two build-time dependencies on the
usr/closed tree[1]. Adding further dependencies is not allowed. We
don't yet have a good way for you to build just usr/src (i.e., just
the open tree); it's on our list of things to do. If you find
yourself referencing the closed tree from an open makefile, you're
probably doing something wrong. Contact tonic-iteam at sun dot com for help.
If you are working on a new kernel "implementation", note that we have
changed the way Install(1) finds kernel modules. Instead of parsing
kernel makefiles, it invokes "make modlist" in the kernel and uses the
results to find kernel modules. If you are working on a new kernel
implementation, please look at the changes to
uts/sun4u/darwin/Makefile to see how to update your implementation's
makefile.
-- gotchas --
There are some gotchas that affect only people with existing
workspaces, and there are gotchas that affect everyone.
* existing workspaces
Please do a clobber build after your next bringover from the gate.
Incremental builds are not guaranteed to work.
If your workspace was created by bringing over usr/src (e.g., using an
older version of nightly(1)), please be sure to change your
Codemgr_wsdata/args from "usr/src" to "usr". Failure to do this could
mean that subsequent bringovers or putbacks will not include all the
files that you want.
Similarly, if you use wx, please check wx/srcroot_dir to make sure
that it contains "usr", not "usr/src". Failure to do this could cause
wx not to see files that you want it to see.
Several kernel makefiles (particularly Makefile.files and
Makefile.rules) were split into open and closed makefiles. When
resolving conflicts, Teamware will update the makefile in usr/src; you
may need to move your changes to the makefile in usr/closed.
Several other kernel makefiles (Makefile.targ and various
Makefile.<subdir> makefiles) were changed to a "shared" makefile that
is used by both the open and closed trees. Because of a limitation
with Teamware, these makefiles were not literally renamed. Instead, a
new makefile was created with the old contents, and the old makefile
was changed so that it includes the shared makefile. This means that
Teamware will not automatically merge your changes into those
makefiles. For example, if you edited
usr/src/uts/sparc/Makefile.sparc (one of the Makefile.<subdir>
makefiles), you must manually apply your changes to
usr/src/uts/sparc/Makefile.sparc.shared.
* everyone
As mentioned above, the ON build tools have been updated to track the
new source organization. You definitely do not want versions of
nightly(1) or wx(1) that are older than November 14th. The easiest
way to deal with the tools update is to wait until tomorrow and
install the updated SUNWonbld from /ws/onnv-gate/packages/`uname -p`.
If you can't wait until tomorrow for the build tools, make sure you
have a nightly(1) that is more recent than November 14th, and add -t
to your NIGHTLY_OPTIONS.
If you have "bringover usr/src" hardwired into your fingers, you'll
need to change "usr/src" to either "usr" or "usr/src usr/closed".
The new version of Install(1) wants dmake 7.6 (Studio 10) or later.
If you have access to /ws/onnv-tools, you should not need to do
anything. Otherwise, please make sure that dmake 7.6 will be found
first in your PATH when you run Install(1). If Install(1) is unable
to find a good version of dmake, it will use /usr/ccs/bin/make, which
will cause Install(1) to run a lot slower.
If you have any questions, please email tonic-iteam at sun dot com.
----
Notes:
[1] usr/src/lib/libc depends on usr/closed/lib/libc_i18n, and
usr/src/cmd/prtdiag depends on usr/closed/lib/libprtdiag.
|