OpenSolaris

  subsites:   Code Reviews   Gates   Issues   Defects   Polls   Test   PKG   Planet   Mail
You are not signed in. Sign in or register.
Last updated: 5/19/06

ON Consolidation: SCM-related Tools

There are two tables of data


COLUMN DESCRIPTIONS

  • Name/Description/Owner

    • Name of the tool

  • Short description of the tool

  • Owner, if not noted by the table title

  • ++ in this column with text denotes standards enforced by the tool

  • ** in this column denotes functionality the owner(s) or user(s) would like to have

  • User(s)

    • Known user(s) of the tool

  • Use Frequency

    • How often the tool is used/run

  • Source Location

    • Where the source can be found

  • Implementation Language

    • Language in which tool is written

  • UI Employed

    • User interface(s) employed by the tool, i.e., the UI the tool presents to the user

  • SCM Hooks Used by Tool

    • SCM hooks used to invoke the tool

  • Run Type

    • How the tool is run: on demand or automatically

  • Dependencies

    • Dependencies on SCCS and/or TeamWare

  • Usually denotes 

      • the use of SCCS and/or TeamWare in the tool's normal operations

  • intimate knowledge about the internals of SCCS and/or TeamWare

  • Access Needs

    • Does the tool need to access history information, i.e., non-current data in the tree

  • Input Data

    • Does the tool require input data from sources other than what is (currently) found in the SCM repository

  • Produced Data

    • Does the tool produce data that is in turn consumed by other systems such as e-mail

  • Output Archived

    • Does the output of the tool require archiving


TABLE NOTES

  • Tables are numbered with prefixes to make entries unique

    • OG = Tool owned by the ON gatekeepers

  • OC = Tool owned by the ON community

  • C = Tool owned by another community

  • Tools are listed alphabetically within a table


CONTENT NOTES

  • For ON, there are tools whose only connection to SCCS or TeamWare are that they explicitly avoid the SCCS directory or make some other incidental use of the SCM.  These types of tools are believed to number between one and 24.  They are not listed on these tables.



Tools Owned by the ON Gatekeepers

Tools Owned by the ON Gatekeepers

ID

Name/Description

User(s)

Use Frequency

Source Location

Implementation Language

UI Employed

SCM Hooks Used by Tool

Run Type

Dependencies

Access Needs

Input Data

Produced Data

Output Archived

OG1

add-gatelingadds a person to the gatelings and notify lists when they do a bringover from the gate or the clone

** It would be great to have a real email address for the person bringing over, rather than a username which might be unique to someone's laptop.  We have to prune the lists wenever someone with a non-global userid brings over.  There's a bunch of hackery in there right now that almost completely reduces that to zero, but that won't scale outside Sun.

Everyone who brings over from ON (~700); 
ON gatekeeping staff is primary user

every few minutes

/ws/onnv-gate/public/gate/bin/add-gateling
also in the IHV gate

ksh

CLI

bringover

On demand

TeamWare

None

None

Creates a mailing list

Yes

OG2

backoutaids in creating an anti-delta in order to back out a bug

ON gatekeeping staff

1x-2x per build

/ws/onnv-gate/public/gk/bin/backout

ksh

CLI

Uses putback mail logs to find files associated with a bugid;
parses SCCS history to discover what deltas are associated with a bugid.

On demand

SCCS

History

Putback Logs

Anti-deltas.  Applies them to files in a workspace which can then be checked in and putback.

No

OG3

buglist: extracts the list of bugs that went into a build

ON gatekeeping staff

1x-2x per build

/w/onnv-gate/public/gk/bin/buglist

ksh

CLI

None

Automatic/
sometimes on demand

Putback logs

History

Yes

List of bugs.  List used by the RTC and to mark fix delivered.

Yes

OG4

clone_update: does the nightly clone "snapshot"

** It would be nice for the clone to be a ZFS snapshot of the gate which would require (other than actually having the gate on ZFS) a productized version of zbringover.

ON gatekeeping staff

nightly

/ws/onnv-gate/public/gate/bin/clone_update

ksh

CLI

None

On demand

TeamWare

None

Works with TeamWare's lock file, breaking all locks before doing the bringover (and notifying people that it's done so)

None (except bringover itself)

No

OG5

daily_snapshot: creates a daily snapshot of the gate sources

** ZFS integration would be fantastic.  Danek is currently running a parallel clone that does take a snapshot every night, but those snapshots need to be available as separate workspaces.

ON gatekeeping staff

nightly

/ws/onnv-gate/public/gate/bin/daily_snapshot

ksh

CLI

None

Automatic

Mostly doesn't depend on TeamWare: uses cpio to transfer the files, though it isn't clear why this is the case.

None

None

Email (informative)

No (other than the snapshots themselves)

OG6

daily_update: maintains a shadow gate to allow for diffs generation

** This tool shouldn't exist.  If the SCM we choose requires something like this, we've picked the wrong SCM.

ON gatekeeping staff

nightly, after the clone, to make sure that the "daily" workspace isn't out of date

/ws/onnv-gate/public/gate/bin/daily_update

ksh

CLI

None

Automatic

TeamWare, for bringover

None

None, other than the locking mechanism

Email
(almost always uninteresting)


OG7

lock-check: warns if the gate is locked for too long

ON gatekeeping staff

every 5 minutes

/ws/onnv-gate/public/gk/bin/lock-check

ksh

CLI

TeamWare's lock file

Automatic (via cron)

TeamWare

None

None

Email

No

OG8

lock-gate/unlock-gate: lock the gate from all putbacks/bringovers in case of emergency

ON gatekeeping staff

1x per build

/ws/onnv-gate/public/gk/bin/lock-gate

ksh

CLI

None/All:
Modifies the SCM's access control file

On demand

TeamWare

None

Manipulates repository-wide metadata

None

No

OG9

pbcheck: not to be confused with "wx pbcheck": does a subset of "wx pbchk" checking on the gate after each putback, including cstyle, jstyle, hdrchk, as well as some permissions checking, sending an email if something needs to be fixed

++ Enforces development standards.

** It should perform its check before the putback happens.  It should also do the full "wx pbchk", plus a few other things, but wx doesn't have the ability to check an arbitrary workspace based on a putback that's already happened (it's possible, but hacky, and hasn't been high enough priority).

ON gatekeeping staff/
ON community

on every putback

/ws/onnv-gate/public/gate/bin/pbcheck

ksh

CLI

putback-to

Automatic

SCCS/TeamWare

History (immediately preceding delta)

None

Email (when finds problems)

No

OG10

pbconfirm: sends a message to the engineer preforming a putback that the putback happened, what build it went into, and what the next steps are

ON gatekeeping staff/
ON community

on every putback

/ws/onnv-gate/public/gate/bin/pbconfirm
also in the IHV gate

ksh

CLI

putback-to

Automatic

TeamWare

None

Build Schedule

Email

No

OG11

pre-mkdelivery: takes the snapshot, does all the biweekly-build bookkeeping, kicks off the builds

ON gatekeeping staff

bi-weekly

/ws/onnv-gate/public/gk/bin/pre-mkdelivery

ksh

CLI

None

Automatic (can be on demand)

SCCS: minor: updates the onnv web page which is under SCCS control.

TeamWare: uses bringover to create the snapshots and pre-populate the build workspaces, as well as doing the equivalent of lock-gate (#4) on the snapshot.  Intimately familiar with the structure of the ON workspace under TeamWare to shware the snapshot properly.

None

Not directly (some of the tools it runs look in the logs)



OG12

putback_diffs: backend workhorse - creates diffs/webrev, pings doc writes, checks to make sure RTIs are approved

** RTI should be checked before putback.  There should be no need for the daily workspace - indeed, diffs generation should be a trivial operation with any modern SCM.

ON gatekeeping staff/
few other who have expressed interest in these diffs

on every putback

/ws/onnv-gate/public/gate/bin/putback_diffs

ksh

CLI

putback-to

Automatic

SCCS: to produce the delta tree diffs

TeamWare

History

RTI database, daily workspace

Email/
creates web pages/
maintains another workspace

Yes

OG13

update_SUNWonbld: maintains the copies of SUNWonbld on the gate machines

ON gatekeeping staff

run nightly (doesn't necessarily do something every night)

/ws/onnv-gate/public/gate/bin/update_SUNWonbld

ksh

CLI

None

Automatic

TeamWare: to maintain the tools build workspace

None

None

Email (for human to verify that everything worked)

No

OG14

update_flagdays: maintains the ON flagday web page

** NLP to merge updates to flag-day/heads-up messages.

ON gatekeeping staff

after many putbacks

/ws/onnv-gate/public/gate/bin/update_flagdays

ksh

CLI

putback-to

Automatic

TeamWare: parses TeamWare putback messages

None

None

Maintains a web page

Yes



Tools Owned by the ON Community

Tools Owned by the ON Community

ID

Name/Description

User(s)

Use Frequency

Source Location

Implementation Language

UI Employed

SCM Hooks Used by Tool

Run Type

Dependencies

Access Needs

Input Data

Produced Data

Output Archived

OC1

keywords: checks a soruce file to ensure that SCCS keywords haven't improperly been checked in in an expanded format

++ Enforces development standards

** Following bugs are filed against keywords:
4701837 keywords should also work on files not in SCCS
4747405 RFE: keywords should have an option to generate canonical ident strings
4793568 keywords -p should complain about #pragma in non-C files

At least on keywords wad is in flight, which will fix 4793568.  Unclear whether others have plans for keywords that aren't logged in bugster.

Owner: ON community.  Usually maintained by people on the gatekeeping alias.

ON community

on every putback/plus uncountable times on individual workspaces

/ws/onnv-gate/usr/src/tools/scripts/keywords.sh

sh

CLI

None

On demand/
Automatic after every putback

SCCS

Checked-out copy of a file

None

Output is consumed only by humans

No

OC2

nightly: builds ON and some other consolidations

++ Enforces development standards

** There are 40+ bugs in bugster against nightly, and probably everyone has his/her own set of pet peeves about it since it is one of the most highly visible faces of ON.

Owner: ON community.  Usually maintained by people on the gatekeeper alias.

ON community

nightly/and many other times

/ws/onnv-gate/usr/src/tools/scripts/nightly.sh

sh

CLI

None

On demand/gatekeepers run automatically

SCCS/TeamWare

Fully historied tree.  None of its semantic operations require this (though some operations are made easier by the presences of SCCS directories)

Nees a file generated by the user to operate fully correctly

Email

No

OC3

sccscp/sccsmv/sccsrm/sccshist: tools to manipulate both the SCCS s-file and g-file in a workspace

Owner: ON community.  Usually maintained by people on the gatekeeper alias.

ON community

occasionally

/ws/onnv-gate/usr/src/tools/scripts/sccscp.sh

sh

CLI

None

On demand

SCCS

SCCS files must be present

None

None

No

OC4

squelch_client: client that is triggered on every putback to maintain the read-only ON Subversion repository on serum.sfbay

Owner: Steve Lau

Steve Lau

on every putback

/ws/onnv-gate/public/gate/bin/squelch_client

perl

CLI

putback-to

Automatic

TeamWare: mail output format

None

None

Yes - passed to a server

Yes

OC5

webrev: creates a set of web pages containing the differences between two workspaces

++ Used to aid the eyeball enforcement of development standards

** As with nightly, lots of people have their own ideas about what other cools things it could do.

Owner: ON community.  Usually maintained by people on the gatekeeper alias.

ON community

very frequently

/ws/onnv-gate/usr/src/tools/scripts/webrev.sh

ksh

CLI

None

On demand/some gatekeepers run automatically

SCCS/TeamWare

File history

Access to the parent repository

Creates a web page

No (but it often is)

OC6

wx: wrapper for SCCS/TeamWare

++ Enforces development standards: cstyle, &c, copyrights, CDDL comment block

** cf nightly, webrev

** Suggestion to replace this with a tool that performs the same functions but that operates faster (performance is very bad, at least for remote users)

Owner: ON community.  Usually maintained by people on the gatekeeper alias.  Will Fiveash currently has the dirtiest fingers.

ON community

all the time

/ws/onnv-gate/usr/src/tools/scripts/wx.sh

ksh

CLI

None

Mostly On demand

SCCS/TeamWare

History access

Access to the parent repository, as well as a few files it maintains itself (as directed by the user)

None

No

OC7

zbringover: quickly do an initial bringover of a pre-built ON workspace using ZFS snapshot/clone to create a new workspace

Owner: ZFS Team

ZFS Team

Few times per week

/ws/zfs-gate/usr/src/cmd/zbringover/zbringover.sh

ksh

CLI

Manually groks TeamWare data and "fixes up" workspace to look like a real child of the original parent

On demand

TeamWare

None

None

None

No