You are not signed in. or register.

Community: Tools

OpenSolaris

SCM Project Console Specification

(DRAFT)

ident "@(#)d-scm-operational.txt 1.2 06/02/17 SMI"

1. Summary

This document, in concert with the approach outlined in "OpenSolaris source code management implementation" [1], specifies the technical requirements to be added to the Projects feature to support managed source code repositories.

2. Discussion

Items suffixed with a (II) are for implementation during the distributed SCM phase. Items suffixed with a (*) are future implementation possibilities.

2.1. Components

The implementation requires support from multiple components within the larger infrastructure. Components affected include:

the web application,
a PAM session module,
an identity update facility,
an automount map generator, and
one or more SCM-specific local execution programs.

(The latter four are assumed to be running on a separate host from the web application, and will be implemented by Sun employees.) This document outlines the changes to the web application that integrate the remaining components. As such, informational text describing the other four components follows.

2.1.1. PAM session module.

A PAM session module, pam_host_scm.so, will be placed in the local PAM configuration of the SCM host machine. This module will chroot(2) the user to a safe environment and set the umask(2) to 0000.

2.1.2. Identity update facility.

A cron(1M)-driven job, the update facility will examine the user table on a periodic basis.

Passwords are encoded in the webapp database with an MD5 hash.

2.1.3. Automount map generator

The safe environment is constructed from a set of key-tied automount entries. These entries are based on each user's status as a committer in one or more projects.

2.1.4. SCM-specific local execution programs.

The local execution programs may be separated into three classes: reporting only, modifying, and notification hooks.

Reporting only programs will be implemented as CGI scripts or equivalents that will report as text/plain in response to an HTTP GET query at one or more defined URLs. For example, the svnlook(1) command of Subversion can have its reporting capabilities wrapped in a simple CGI script.

Modifying programs, which require a more complex error channel, will be implemented to invoke functionality from the SCM host daemon. See 2.1.5 for more details.

Minimal use of hook functionality will be offered in the initial release. The only hook, which may need to be installed for multiple operations, will notify the mail alias associated with the repository on SCM operations that modify the files within the repository. (Notification is not provided for administrative operations, such as description changes or committer modifications.)

2.1.5 SCM host daemon

This daemon is responsible for allowing the web application to interact with the SCM host. Specific responsibilities include:

Authenticating and Performing remote requests
Securing Remote Requests

Since this is written in Java to facilitate interoperation with the web application, the "SCM-specific local execution programs" must be able to easily interact with Java. The lowest common denominator approach would be to provide Java clients which can be invoked as sub-processes if the "SCM-specific local execution programs" aren't written in Java.

Interface Definition

The SCM host daemon offers the following remote interfaces:

Admin interface

The interface that an Admin server must implement so that an Admin client can determine which operations may be performed by the server and operations that facilitate synchronous or asynchronous execution of a task.

String[] getOps() throws RemoteException;
TaskResult executeTask(Task t) throws RemoteException;
URL executeAsyncTask(Task t) throws RemoteException;

Task interface

The interface that an Admin client must implement to allow execution by the Admin server.

TaskResult execute();
String getName();

2.2. Database modifications.

A new object, called a repository, is now tracked by the application. A repository is associated with a project and has a "short name" and a description. The repository is associated with an SCM system of a given type; the first supported type is "svn". A table to track these objects is expected to exist. The repository may or may not be able to be accessed anonymously; the initial setting for the field tracking this access is "true", implying that all repositories allow anonymous access. Each repository has an associated email alias for change notifications. Each repository has a timestamp field containing the date of its creation.

Determining the list of repositories associated with a specific project is accomplished by a query on the repository table that specifies the project ID of the requested project.

Each project now has an additional field, an integer specifying the maximum number of repositories permitted for this project.

Each user now has an additional relationship with each repository on the system. The existence of an entry in the committer table, between a user ID and a repository ID, indicates the ability to perform SCM client operations on that repository. In the initial implementation, although the committer-repository relations are tracked separately, the application will expose only one set of committers per project, and assign each committer to each repository associated with the project.

If a user is made a "project editor", his or her account should also be given committer privileges for the repositories associated with that project. Upon loss of editorship, the user retains committer rights (which may be removed manually).

Upon user deletion, committer rights entries should be removed.

The database is implemented in MySQL. See Appendix A for MySQL specific tables outlining the required database changes.

2.3. Project page changes and related modifications

The project page, http://opensolaris.org/os/project/[project], shall have a new left hand menu item entitled "Repositories". Clicking on that link will load a new page, as described in the following section.

The project profile page, entitled "Edit Project" and with URL http://www.opensolaris.org/os/projects/editProject.jspa, needs a new property entry, "Maximum number of repositories".

2.4. Repositories page

The new "Repositories" page is the primary reporting page for a project's source code repositories. The page will consist of a table that lists the code repositories, displaying their names and descriptions, the mailing list they send notifications to, and the URLs used to access them from their appropriate SCM and from HTTP access. The table will further list the committers to the repository.

Furthermore, each repository will offer a list of status operation links to the SCM host. The URL for these links for repositories of type "svn" is


http://svn.opensolaris.org/svnlook.cgi?project=[project]& \
repository=[repository]&operation=[operation]&result=xhtml

Valid operations for this listing are "author", "changed", "date", "dirs-changed", "info", "log", "uuid", "youngest". (The svnlook.cgi script, noted in 2.1.4, may support additional invocation options that can be supported at a later date.)

2.5. Project leader menu

The project leader menu will require the addition of items allowing access to repository operations. To be consistent with the other menus of this kind, items entitled "Find Repositories" and "Add Repository" will be added.

Additionally, to manage committers, a "Manage Committers" item will be needed.

Each of these items brings up a distinct page, as explained in the following subsections.

2.5.1. Find Repositories page

The Find Repository page allows the querying of the set of repositories owned by this project. Upon page load, the Find Repositories page will display the results of the default query, which is unpopulated and meant to match all repositories. The project leader may then perform an additional filtering query to reduce the result set.

The query form should present three fields for the query: short name, description, and repository type. The short name and description fields are text entry fields. The repository type is a radio selection from "Subversion" and "Don't Care".

Selecting a specific repository from the result set table leads to the "Manage Repository" page being loaded.

2.5.2. Manage Repository page

The Manage Repository page allows the modification of mutable repository properties and the invocation of repository administrative operations. It also permits the deletion of repositories.

The mutable fields for the initial release are the Description text field and the Notification Alias text field. The top portion of the page should present these fields and a "Save" button that submits the operation. Completing the submission should result in a reload of the Manage Repository page, with a short banner encoded as a DIV entity with class "appResult" (henceforth an "appResult" banner) that outlines the success or failure of the modification.

The middle section of the page is concerned with repository operations. These are specific to the repository type, which should be identified in the header at the beginning of the page section.

At this point, the only supported operation is "verify". XXX How is this operation requested?

Completion of the submission of a repository operation should again result in a reload of the Manage Repository page, with a short "appResult" banner at the top of the middle section that describes the outcome of the operation.

The bottom section of the page consists of a single question, "Delete this repository?", and a submit button. A submission of this kind should result in a second verification question ("Are you certain?"). Following this confirmation, the application should request of the SCM host that the repository be deleted. Upon a successful deletion, the application may delete the repository and associated entries from the database. The application should then load the Find Repositories page, and display an "appResult" banner outlining the deletion's success. In the cases that SCM deletion was unsuccessful, the Manage Repositories page should be displayed with an appropriate "appResult" banner. (The SCM execution agent will send a notification message to the site administrator that contains the reason for the failure.)

2.5.3. Add Repository page

The Add Repository page allows the creation of new repositories on the system.

The page asks for the following:

a short name, as a text field,
a description, as a text field,
a notification alias, as a text field,
a repository type, as a radio button field with one choice, "Subversion", and
an allow anonymous access field, as a radio button field with one choice, "Yes".

This should be followed by a submit button with label "Save". The short name must be unique to the repositories managed by this project. The default short name is "main". The submission will result in a request of the SCM execution agent.

On success, the application will record these values, along with the timestamp of the operation, and also create appropriate committer entries. (The initial set of committers is comprised of the project leads at the time of repository creation.) The application should the Find Repositories page, with an appropriate appResult banner at the top of the page.

On failure, the application will redisplay the Add Repository page, with an appResult banner explaining the failure.

2.5.4. Manage Committers page

This feature is the first where a role other than the Site Administrator can grant capabilites to an individual user. Committers only have capabilities on the SCM host(s), and have no new role in the sense of the web application.

The Manage Committers page presents a selection list of the committers associated with the project. Then, a text field, labelled "New Committers", with an associated submit button, "Add", constitutes the add committer operation. The text field contents may be a space or comma separated list. Each user ID must be a valid OpenSolaris ID.

A further submit button, labelled "Delete Selected", submits a request to removed the selected committer entries from the displayed list.

Both of these operations should reload the Manage Committers page, with an appResult banner outlining the operational outcome.

3. Future work

If the site develops support for in-line content macro expansion, macros that generate a list of the project's repositories should be provided.

Anonymous access may be disallowed for repositories in future ("committers only").

At least one additional repository SCM type is expected.

Additional access to repository information via the Repositories page, such as to OpenGrok-based code browsing is expected.

4. References

[1] S. Hahn, "OpenSolaris source code management implementation", 2006.

[2] B. Collins-Sussman, B. W. Fitzpatrick, and C. M. Pilato, "Version Control with Subversion", http://svnbook.red-bean.com/, 2006.

Appendix A

Database Modifications

New Table Definitions

Repository

+-------------+------------+-----+---+-----------+-------------------------+
|Column Name  |Data Type   |Null |Key|Default    |Notes                    |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|ShortName    |TINYTEXT    |  N  | Y |           |Only one row initially   |
|             |            |     |   |           |with the value svn.      |
|             |            |     |   |           |                         |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|Description  |TEXT        |  Y  | N |           |                         |
|             |            |     |   |           |                         |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|ProjectID    |INT UNSIGNED|  N  | N |           |Foreign Key from Project |
|             |            |     |   |           |Table. Confirm data type |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|SCMType      |TINYTEXT    |  N  | N |           |Foreign Key to SCMType   |
|             |            |     |   |           |table, ShortName Column. |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|AnonymousAcc\|TINYINT     |  N  | N |1          |MySQL doesn't have a     |
|ess          |            |     |   |           |BOOLEAN type, so use     |
|             |            |     |   |           |TINYINT(1).              |
+-------------+------------+-----+---+-----------+-------------------------+
|Email        |TINYTEXT    |  N  | N |           |                         |
|             |            |     |   |           |                         |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|Created      |DATETIME    | N   | N |           |Requirements say         |
|             |            |     |   |           |TIMESTAMP, but I think   |
|             |            |     |   |           |this is better.          |
+-------------+------------+-----+---+-----------+-------------------------+

SCMType

+-------------+------------+-----+---+-----------+-------------------------+
|Column Name  |Data Type   |Null |Key|Default    |Notes                    |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|ShortName    |TINYTEXT    |  N  | Y |           |Only one row initially   |
|             |            |     |   |           |with the value svn.      |
|             |            |     |   |           |                         |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|Description  |TEXT        |  Y  | N |           |                         |
|             |            |     |   |           |                         |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+

UserRepositories (AKA Committer Table ?)

+-------------+------------+-----+---+-----------+-------------------------+
|Column Name  |Data Type   |Null |Key|Default    |Notes                    |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|UserName     |TEXT        | N   | Y |           |Foreign Key to UserName  |
|             |            |     |   |           |on User Table. Confirm   |
|             |            |     |   |           |Data Type.               |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|RepositoryNa\|TINYTEXT    |  N  | Y |           |Foreign Key to ShortName |
|me           |            |     |   |           |on Repository Table.     |
|             |            |     |   |           |                         |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+

Modified Table Definitions

Project

+-------------+------------+-----+---+-----------+-------------------------+
|Column Name  |Data Type   |Null |Key|Default    |Notes                    |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|MaxRepositor\|INT         |  N  | N |           |                         |
|ies          |            |     |   |           |                         |
|             |            |     |   |           |                         |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+

User

+-------------+------------+-----+---+-----------+-------------------------+
|Column Name  |Data Type   |Null |Key|Default    |Notes                    |
|             |            |     |   |           |                         |
+-------------+------------+-----+---+-----------+-------------------------+
|ProjectEditor|TINYINT(1)  |  N  | N |           |If set to TRUE, associate|
|             |            |     |   |           |the user with all project|
|             |            |     |   |           |repositories.            |
+-------------+------------+-----+---+-----------+-------------------------+

Stephen Hahn, PhD Solaris Kernel Development, Sun Microsystems
stephen dot hahn at sun dot com http://blogs.sun.com/sch/