|
Replies:
25
-
Last Post:
Feb 17, 2006 12:01 PM
by: James Blackwell
|
|
|
Stephen Hahn
sch@eng.sun.com
|
|
|
|
Distributed SCM requirements draft
Posted:
Dec 1, 2005 2:01 PM
|
|
As I mentioned a few weeks ago, we want to evaluate the various
distributed source code management systems available to see which are
a best fit for OpenSolaris. The following document outlines a roughly
ranked set of requirements to use in the evaluation. Thoughts,
corrections, and comments are welcomed.
- Stephen
----
ident "@(#)d-scm-requirements.txt 1.3 05/12/01 SMI"
OpenSolaris
DISTRIBUTED SOURCE CODE MANAGEMENT REQUIREMENTS [DRAFT]
1. Summary
This document identifies and explains the requirements for a
distributed source code management (SCM) solution to be used with
OpenSolaris. The requirements are grouped into three sets of
decreasing importance. It outlines a number of specific evaluations
that will be used to determine whether a candidate SCM meets the
various requirements.
2. Discussion
The requirements described below arise from a number of distinct
classes: some are social, in that the requirement is believe
necessary for successful use in the community; some are technical,
in that the requirement is believed necessary to successfully
produce software in a multi-project, multi-committer, multi-site
development organization; and some are economic, in that the
requirement is attempting to describe attributes that would limit
the costs of the ongoing use of the tool.
In an attempt to use neutral terms, we use the phrase "candidate
SCM" to describe the SCM solution we are evaluating and "current
SCM" to refer to the distributed SCM solution in use (inside Sun) at
present. (Not all consolidations participating in OpenSolaris use a
distributed SCM at present; their SCM requirements are not discussed
in this document.)
The requirements are ranked by necessity, using the terminology
proposed in IEEE Std 830-1998 [1].
2.1. "Essential" requirements
E0. Open source
To be considered for use by the OpenSolaris community, the
candidate SCM is expected to be available under an OSI-approved
license.
E1. Unbiased and disconnected distribution
Although a distributed SCM may choose to implement some form of
dependency relationship between source trees (such as a
"parent-child" convention) that relationship must not need to be
continuously available for sensible SCM operation. Moreover,
the candidate SCM must support source code updates between two
distinct repositories with a common ancestor that have had no
other contact. Sensible operation for disconnected use
encompasses all SCM operations that act only on the local
repository: creation, modification, or deletion of files or
directories or the metadata describing these objects or the
changes to them.
E2. Networked operation
The candidate SCM must be able to operate in a sensible and well
performing manner between two hosts in separate administrative
domains. Beyond the data contained within the candidate SCM's
representation, the only common administrative requirement
should be a credential identifying the remote operator
initiating the transaction to the other host.
One mechanism that meets this requirement is to tunnel the
candidate SCM operation through ssh(1). Candidate SCMs that use
an implementation that requires domains to change security
policies to open unusual or believed risky network ports will be
considered to be minimally compliant with this requirement.
Performance measurements will be used to compare candidates, as
outlined below. A candidate SCM with performance results in the
bottom third of all candidates will be deemed to have failed to
meet this requirement.
E3. Interface stability and completeness
The storage representation, command line interfaces, network
protocols, and hooks interfaces should be documented and have
some level of declared commitment. The state of the storage
representation and the operations that modify it should be well
defined, so that use with advanced file system capabilities can
be assessed for hazards. (For example, consistent use with file
system snapshot capabilities.)
E4. Standard operations and transactions
The candidate SCM is expected to support rename and deletion
transactions at the file and directory levels. A
history-preserving copy operation, followed by a delete
operation, may be considered equivalent to a rename;
equivalency, and the reasons for rename omission, are to be
assessed and documented by the evaluating engineer.
E5. Per changeset metadata.
The candidate SCM must be able to associate, at a minimum, an
unstructured text fragment with each changeset.
2.2. "Conditional" requirements
C6. Ease of use
The candidate SCM should be easy to install in a reasonably
self-contained fashion. In principle, shipment in an
OpenSolaris consolidation should be possible with a finite
investment of resources.
The primary interfaces should be understandable to a user
familiar with distributed SCM concepts.
The candidate SCM should offer some assistance with conflict
resolution during an update, the issuance of source code
patches, and the ability to browse the source tree via a web
server.
The candidate SCM should be able to undo the application of a
specific changeset ("backout") atomically and easily.
C7. "No dedicated server" operational mode
In the interests of machine resource conservation, the candidate
SCM should have a mode in which it can operate without a
continuously running server process. This mode may have
concurrency restrictions or performance limitations compared to
its primary server mode.
For instance, within a large administrative domain, it may be
more convenient to utilize NFS and a shared identity
infrastructure than to rely on the networked operating mode
required by E2. A candidate SCM which can sensibly operate in a
pure OpenSolaris NFS environment without the establishment of a
dedicated server process would meet this requirement.
C8. Tool community health
The community or author of the candidate SCM needs to be active
and engaged with their user population. The ability of the
candidate SCM's community to absorb, directly or through a
liaison, the defects and feature requests of the OpenSolaris
community should be estimated, preferably by a direct inquiry to
the candidate SCM community.
C9. OpenSolaris community implementation expertise
One or more contributors within the OpenSolaris community need
to be able to assess potential defects in the implementation of
the candidate SCM and potentially participate in the development
of new features or supporting tools for the candidate SCM.
C10. Interface extensibility
Beyond the requirements of E3, an extensible interface, so that
OpenSolaris-specific tools might be integrated with SCM
operations is desired. Such an interface might be composed of a
documented "hooks" interface, a documented library interface, or
some other modular approach. An extensive hooks interface, with
hook evaluations able to terminate operations, is a strongly
desired attribute in a candidate SCM; a candidate SCM with such
an interface will be considered to meet fully this requirement.
C11. Transactional operations and corruption recovery
The operations on the candidate repository should have defined
semantics, in particular identifying non-atomic transactions and
mechanisms for recovery from a corrupted repository.
C12. Content generality
The candidate SCM should be able to represent safely and track
files with binary content, in addition to text files.
2.3 "Optional" requirements
O13. Partial trees
The structure of the ON consolidation and the current SCM
solution allow a contributor to work on specific subsets of the
source tree in a supported fashion. This requirement states
that, while such a mode with support for expressing dependencies
between files and directories is valuable, support for partial
tree repositories is not necessary.
O14. Per-file histories
The current SCM uses SCCS as a per-file revision storage format.
As such, each file has an individual history. This feature
allows the combination of disjoint issues to be addressed in a
single commit without connecting the per-file history. It is
believed that the ability to meet the other requirements stated
in this document is sufficiently more valuable than the support
of per-file revision histories. Moreover, the construction of
per-file histories in reporting and browsing tools can be
accomplished by convention in many cases.
That is to say, a candidate SCM that meets E5 is sufficient.
3. Evaluations
We anticipate a number of qualitative and quantitative tests to
evaluate the satisfaction of the various requirements, where a
"meets" or "does not meet" result is not applicable.
3.1. Representational and performance criteria
These criteria focus on the ability of the candidate SCM to
represent a large, long-running, and active source tree. The ON
consolidation represents more than 25 000 changesets by over 1300
committers against approximately 40 000 files.
The expected set of meaningful operations for performance evaluation
are:
- first pull/clone operation,
- subsequent pull/update operation, and
- push/commit operation.
Performance results for the set of operations will be captured for
three distinct scenarios: within a campus, across SWAN between
sites, and between two Internet sites. SWAN measurements will be
captured between each of Menlo Park, CA and Burlington, MA, Manchester,
UK, and Beijing, PRC. (Equivalent sites may be added or
substituted.) For comparison, results will be phrased both
as as a percentage of sustained bandwidth, and as absolute time
elapsed (for an identical pair of endpoints). Baseline absolute
time comparisons will be made against standard and "turbo" TeamWare
for within-SWAN scenarios, and against an rsync copy of the same
data for all scenarios.
The candidate SCM will be evaluated for data integrity by
interruption of the set of operations by signal and by machine
failure.
The safety of the candidate SCM with respect to file system
capabilities will be evaluated using ZFS snapshot/clone technology
for safe repository copies.
3.2. Implementation criteria
The candidate SCM implementation will be assessed by a design and
code review by an OpenSolaris contributor with expertise in the
implementation language of the candidate SCM.
3.3. Tools criteria
If available, the candidate SCM is expected to provide or identify a
graphical merge program that can be used to resolve conflicts
resulting from an update operation. In the case that no known
program can be used, the evaluating contributor will assess the work
necessary to use one of the standard graphical merge programs.
4. References
[1] IEEE Std 830-1998, "IEEE Recommended Practice for Software
Requirements Specifications", 1998.
--
Stephen Hahn, PhD Solaris Kernel Development, Sun Microsystems
stephen dot hahn at sun dot com http://blogs.sun.com/sch/
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
Posts:
1,901
From:
NZ
Registered:
6/16/05
|
|
|
|
Re: Distributed SCM requirements draft
Posted:
Dec 11, 2005 7:33 PM
in response to: Stephen Hahn
|
|
Hi,
Though perhaps more a 'Social' question, I'm rather interested to see
how things stand from a process point of view with the distributed
requirement on a source code management system.
Will there be tools to easily publish child workspaces? Will those child
workspaces be publically available? Will changes to those child
workspaces be trackable through some sort of change set hook [ie.
loginfo for CVS?] - ie. how is all this going to work when there's the
potential for contributions outside and inside Sun's firewall, and what
steps/discussions are happening to make this the transparent process
that it needs to be?
Glynn
On Thu, 2005-12-01 at 14:01 -0800, Stephen Hahn wrote:
> As I mentioned a few weeks ago, we want to evaluate the various
> distributed source code management systems available to see which are
> a best fit for OpenSolaris. The following document outlines a roughly
> ranked set of requirements to use in the evaluation. Thoughts,
> corrections, and comments are welcomed.
>
> - Stephen
>
> ----
>
> ident "@(#)d-scm-requirements.txt 1.3 05/12/01 SMI"
>
> OpenSolaris
> DISTRIBUTED SOURCE CODE MANAGEMENT REQUIREMENTS [DRAFT]
>
> 1. Summary
>
> This document identifies and explains the requirements for a
> distributed source code management (SCM) solution to be used with
> OpenSolaris. The requirements are grouped into three sets of
> decreasing importance. It outlines a number of specific evaluations
> that will be used to determine whether a candidate SCM meets the
> various requirements.
>
> 2. Discussion
>
> The requirements described below arise from a number of distinct
> classes: some are social, in that the requirement is believe
> necessary for successful use in the community; some are technical,
> in that the requirement is believed necessary to successfully
> produce software in a multi-project, multi-committer, multi-site
> development organization; and some are economic, in that the
> requirement is attempting to describe attributes that would limit
> the costs of the ongoing use of the tool.
>
> In an attempt to use neutral terms, we use the phrase "candidate
> SCM" to describe the SCM solution we are evaluating and "current
> SCM" to refer to the distributed SCM solution in use (inside Sun) at
> present. (Not all consolidations participating in OpenSolaris use a
> distributed SCM at present; their SCM requirements are not discussed
> in this document.)
>
> The requirements are ranked by necessity, using the terminology
> proposed in IEEE Std 830-1998 [1].
>
> 2.1. "Essential" requirements
>
> E0. Open source
>
> To be considered for use by the OpenSolaris community, the
> candidate SCM is expected to be available under an OSI-approved
> license.
>
> E1. Unbiased and disconnected distribution
>
> Although a distributed SCM may choose to implement some form of
> dependency relationship between source trees (such as a
> "parent-child" convention) that relationship must not need to be
> continuously available for sensible SCM operation. Moreover,
> the candidate SCM must support source code updates between two
> distinct repositories with a common ancestor that have had no
> other contact. Sensible operation for disconnected use
> encompasses all SCM operations that act only on the local
> repository: creation, modification, or deletion of files or
> directories or the metadata describing these objects or the
> changes to them.
>
> E2. Networked operation
>
> The candidate SCM must be able to operate in a sensible and well
> performing manner between two hosts in separate administrative
> domains. Beyond the data contained within the candidate SCM's
> representation, the only common administrative requirement
> should be a credential identifying the remote operator
> initiating the transaction to the other host.
>
> One mechanism that meets this requirement is to tunnel the
> candidate SCM operation through ssh(1). Candidate SCMs that use
> an implementation that requires domains to change security
> policies to open unusual or believed risky network ports will be
> considered to be minimally compliant with this requirement.
>
> Performance measurements will be used to compare candidates, as
> outlined below. A candidate SCM with performance results in the
> bottom third of all candidates will be deemed to have failed to
> meet this requirement.
>
> E3. Interface stability and completeness
>
> The storage representation, command line interfaces, network
> protocols, and hooks interfaces should be documented and have
> some level of declared commitment. The state of the storage
> representation and the operations that modify it should be well
> defined, so that use with advanced file system capabilities can
> be assessed for hazards. (For example, consistent use with file
> system snapshot capabilities.)
>
> E4. Standard operations and transactions
>
> The candidate SCM is expected to support rename and deletion
> transactions at the file and directory levels. A
> history-preserving copy operation, followed by a delete
> operation, may be considered equivalent to a rename;
> equivalency, and the reasons for rename omission, are to be
> assessed and documented by the evaluating engineer.
>
> E5. Per changeset metadata.
>
> The candidate SCM must be able to associate, at a minimum, an
> unstructured text fragment with each changeset.
>
> 2.2. "Conditional" requirements
>
> C6. Ease of use
>
> The candidate SCM should be easy to install in a reasonably
> self-contained fashion. In principle, shipment in an
> OpenSolaris consolidation should be possible with a finite
> investment of resources.
>
> The primary interfaces should be understandable to a user
> familiar with distributed SCM concepts.
>
> The candidate SCM should offer some assistance with conflict
> resolution during an update, the issuance of source code
> patches, and the ability to browse the source tree via a web
> server.
>
> The candidate SCM should be able to undo the application of a
> specific changeset ("backout") atomically and easily.
>
> C7. "No dedicated server" operational mode
>
> In the interests of machine resource conservation, the candidate
> SCM should have a mode in which it can operate without a
> continuously running server process. This mode may have
> concurrency restrictions or performance limitations compared to
> its primary server mode.
>
> For instance, within a large administrative domain, it may be
> more convenient to utilize NFS and a shared identity
> infrastructure than to rely on the networked operating mode
> required by E2. A candidate SCM which can sensibly operate in a
> pure OpenSolaris NFS environment without the establishment of a
> dedicated server process would meet this requirement.
>
> C8. Tool community health
>
> The community or author of the candidate SCM needs to be active
> and engaged with their user population. The ability of the
> candidate SCM's community to absorb, directly or through a
> liaison, the defects and feature requests of the OpenSolaris
> community should be estimated, preferably by a direct inquiry to
> the candidate SCM community.
>
> C9. OpenSolaris community implementation expertise
>
> One or more contributors within the OpenSolaris community need
> to be able to assess potential defects in the implementation of
> the candidate SCM and potentially participate in the development
> of new features or supporting tools for the candidate SCM.
>
> C10. Interface extensibility
>
> Beyond the requirements of E3, an extensible interface, so that
> OpenSolaris-specific tools might be integrated with SCM
> operations is desired. Such an interface might be composed of a
> documented "hooks" interface, a documented library interface, or
> some other modular approach. An extensive hooks interface, with
> hook evaluations able to terminate operations, is a strongly
> desired attribute in a candidate SCM; a candidate SCM with such
> an interface will be considered to meet fully this requirement.
>
> C11. Transactional operations and corruption recovery
>
> The operations on the candidate repository should have defined
> semantics, in particular identifying non-atomic transactions and
> mechanisms for recovery from a corrupted repository.
>
> C12. Content generality
>
> The candidate SCM should be able to represent safely and track
> files with binary content, in addition to text files.
>
> 2.3 "Optional" requirements
>
> O13. Partial trees
>
> The structure of the ON consolidation and the current SCM
> solution allow a contributor to work on specific subsets of the
> source tree in a supported fashion. This requirement states
> that, while such a mode with support for expressing dependencies
> between files and directories is valuable, support for partial
> tree repositories is not necessary.
>
> O14. Per-file histories
>
> The current SCM uses SCCS as a per-file revision storage format.
> As such, each file has an individual history. This feature
> allows the combination of disjoint issues to be addressed in a
> single commit without connecting the per-file history. It is
> believed that the ability to meet the other requirements stated
> in this document is sufficiently more valuable than the support
> of per-file revision histories. Moreover, the construction of
> per-file histories in reporting and browsing tools can be
> accomplished by convention in many cases.
>
> That is to say, a candidate SCM that meets E5 is sufficient.
>
> 3. Evaluations
>
> We anticipate a number of qualitative and quantitative tests to
> evaluate the satisfaction of the various requirements, where a
> "meets" or "does not meet" result is not applicable.
>
> 3.1. Representational and performance criteria
>
> These criteria focus on the ability of the candidate SCM to
> represent a large, long-running, and active source tree. The ON
> consolidation represents more than 25 000 changesets by over 1300
> committers against approximately 40 000 files.
>
> The expected set of meaningful operations for performance evaluation
> are:
>
> - first pull/clone operation,
> - subsequent pull/update operation, and
> - push/commit operation.
>
> Performance results for the set of operations will be captured for
> three distinct scenarios: within a campus, across SWAN between
> sites, and between two Internet sites. SWAN measurements will be
> captured between each of Menlo Park, CA and Burlington, MA, Manchester,
> UK, and Beijing, PRC. (Equivalent sites may be added or
> substituted.) For comparison, results will be phrased both
> as as a percentage of sustained bandwidth, and as absolute time
> elapsed (for an identical pair of endpoints). Baseline absolute
> time comparisons will be made against standard and "turbo" TeamWare
> for within-SWAN scenarios, and against an rsync copy of the same
> data for all scenarios.
>
> The candidate SCM will be evaluated for data integrity by
> interruption of the set of operations by signal and by machine
> failure.
>
> The safety of the candidate SCM with respect to file system
> capabilities will be evaluated using ZFS snapshot/clone technology
> for safe repository copies.
>
> 3.2. Implementation criteria
>
> The candidate SCM implementation will be assessed by a design and
> code review by an OpenSolaris contributor with expertise in the
> implementation language of the candidate SCM.
>
> 3.3. Tools criteria
>
> If available, the candidate SCM is expected to provide or identify a
> graphical merge program that can be used to resolve conflicts
> resulting from an update operation. In the case that no known
> program can be used, the evaluating contributor will assess the work
> necessary to use one of the standard graphical merge programs.
>
> 4. References
>
> [1] IEEE Std 830-1998, "IEEE Recommended Practice for Software
> Requirements Specifications", 1998.
>
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Posts:
917
From:
Registered:
4/28/05
|
|
|
|
Re: Distributed SCM requirements draft
Posted:
Dec 20, 2005 5:29 PM
in response to: Stephen Hahn
|
|
I'm curious where this process is leading. I don't know of any tool that meets all these requirements except perhaps BitKeeper. Subversion certainly doesn't meet the requirement outlined in C7, even if it is distributed.
This all sounds more like a plan to create a new SCM rather than adopt an existing one, unless there is a tool in the back of peoples minds that I'm unaware of.
Beyond just this, however, I think the scope of this project needs to be clearly defined from the outset. Are we looking for a tool that existing TeamWare gates could pump incrementals to or a single shared internal/external source code management system that would phase out TeamWare? The vibe I'm getting is for the latter.
benr.
|
|
|
|
|
Posts:
1,901
From:
NZ
Registered:
6/16/05
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Dec 20, 2005 8:57 PM
in response to: benr
|
|
Hey,
On Tue, 2005-12-20 at 17:29 -0800, Ben Rockwood wrote:
> Beyond just this, however, I think the scope of this project needs to
> be clearly defined from the outset. Are we looking for a tool that
> existing TeamWare gates could pump incrementals to or a single shared
> internal/external source code management system that would phase out
> TeamWare? The vibe I'm getting is for the latter.
On top of this, I'd like a simple question answered - Are we trying to
fit a source code management solution to the process, or a process to
the source code management system? It feels like the first and I have
mixed concerns as to whether that's a good idea or not, in terms of
community development [1]
Glynn
[1] My opinion probably doesn't count in terms of ON development, but
just trying to see the side from a community point of view.
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Posts:
818
From:
San Francisco CA US
Registered:
3/9/05
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Dec 21, 2005 10:47 AM
in response to: gman
|
|
On Wed, Dec 21, 2005 at 03:57:26PM +1100, Glynn Foster wrote:
> On top of this, I'd like a simple question answered - Are we trying to
> fit a source code management solution to the process, or a process to
> the source code management system? It feels like the first and I have
> mixed concerns as to whether that's a good idea or not, in terms of
> community development [1]
It's never correct to choose tools and then try to fit a process to
them. Ignoring the fundamental problems with that approach, the
immediate practical question is "If I'm not choosing tools to support
a process, on what criteria will I base that selection?" In practice
the answers tend to fall into two categories: inertia and fad worship.
We're explicitly not allowing inertia to drive the choice: TeamWare in
its current form fails to meet the essential requirements; it's clear
that these were not written with the advance intent to select
TeamWare. Fad worship is at best shortsighted and intellectually
lazy, entirely inappropriate for a project team desirous of long-term
success.
The requirements reflect the need for development widely separated by
administrative domains, poor connectivity, user namespace, and so on.
I don't understand how the requirements in question pose a barrier to
community development; in particular E0, E1, and E2 seem intended
explicitly to remove potential barriers to non-Sun participation.
What changes or additions do you recommend?
--
Keith M Wesolowski "Sir, we're surrounded!"
Solaris Kernel Team "Excellent; we can attack in any direction!"
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Posts:
1,901
From:
NZ
Registered:
6/16/05
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Dec 21, 2005 10:12 PM
in response to: wesolows
|
|
Hey,
On Wed, 2005-12-21 at 10:47 -0800, Keith M Wesolowski wrote:
> It's never correct to choose tools and then try to fit a process to
> them. Ignoring the fundamental problems with that approach, the
> immediate practical question is "If I'm not choosing tools to support
> a process, on what criteria will I base that selection?" In practice
> the answers tend to fall into two categories: inertia and fad worship.
> We're explicitly not allowing inertia to drive the choice: TeamWare in
> its current form fails to meet the essential requirements; it's clear
> that these were not written with the advance intent to select
> TeamWare. Fad worship is at best shortsighted and intellectually
> lazy, entirely inappropriate for a project team desirous of long-term
> success.
You're absolutely right. You can't choose the tools without a process,
but neither can you choose a process without the tools. Maybe I'm taking
an overly simplistic view of how to approach this, but there's been very
little discussion on how the *process* is actually supposed to work.
> The requirements reflect the need for development widely separated by
> administrative domains, poor connectivity, user namespace, and so on.
> I don't understand how the requirements in question pose a barrier to
> community development; in particular E0, E1, and E2 seem intended
> explicitly to remove potential barriers to non-Sun participation.
> What changes or additions do you recommend?
I just like to see some discussion on how people think this might work.
The only mail that came close to it was Roy's mail on the CAB discuss
list -
http://www.opensolaris.org/jive/thread.jspa?threadID=2806&tstart=15
I don't want to be a negative blocker on progress [I appreciate the work
being done], but I don't even understand the process coming from within
Sun - how do we expect a community to understand and adopt it? At the
moment, they're living in a relatively sheltered 'throw their patches
over the wall' environment.
Maybe I'm totally off base with this....shrug.
Glynn
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Posts:
1,710
From:
NZ
Registered:
4/27/05
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Dec 21, 2005 10:28 PM
in response to: gman
|
|
Glynn Foster wrote:
>Hey,
>
>On Wed, 2005-12-21 at 10:47 -0800, Keith M Wesolowski wrote:
>
>
>>It's never correct to choose tools and then try to fit a process to
>>them. Ignoring the fundamental problems with that approach, the
>>immediate practical question is "If I'm not choosing tools to support
>>a process, on what criteria will I base that selection?" In practice
>>the answers tend to fall into two categories: inertia and fad worship.
>>We're explicitly not allowing inertia to drive the choice: TeamWare in
>>its current form fails to meet the essential requirements; it's clear
>>that these were not written with the advance intent to select
>>TeamWare. Fad worship is at best shortsighted and intellectually
>>lazy, entirely inappropriate for a project team desirous of long-term
>>success.
>>
>>
>
>You're absolutely right. You can't choose the tools without a process,
>but neither can you choose a process without the tools. Maybe I'm taking
>an overly simplistic view of how to approach this, but there's been very
>little discussion on how the *process* is actually supposed to work.
>
>
Is there _A_ process? Or does each consolidation follow its own?
Ian
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Posts:
818
From:
San Francisco CA US
Registered:
3/9/05
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Dec 22, 2005 9:16 AM
in response to: ian
|
|
On Thu, Dec 22, 2005 at 07:28:33PM +1300, Ian Collins wrote:
> Is there _A_ process? Or does each consolidation follow its own?
There is a single process with numerous steps, most of which can be
omitted under some circumstances. Consolidation managers may in some
cases decide not to require steps they believe are inappropriate or
inapplicable. It would be excessively and needlessly confusing,
however, if every consolidation had a completely different process.
--
Keith M Wesolowski "Sir, we're surrounded!"
Solaris Kernel Team "Excellent; we can attack in any direction!"
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Mike Kupfer
kupfer@athyra.sfbay....
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Dec 21, 2005 10:51 AM
in response to: gman
|
|
>>>>> "Glynn" == Glynn Foster <Glynn dot Foster at Sun dot COM> writes:
Glynn> Are we trying to fit a source code management solution to the
Glynn> process, or a process to the source code management system?
I think it's more the first, but some sort of feedback cycle and
iteration might be needed.
Glynn> It feels like the first and I have mixed concerns as to whether
Glynn> that's a good idea or not, in terms of community development
Obviously there's a downside to picking a tool or development model that
the external community is not familiar with. On the other hand, if the
commonly used tools and development practices don't scale to something
the size of ON, that's a problem, too.
I'd encourage anyone who thinks there are missing requirements, or who
thinks that some of the stated requirements are unnecessary, to speak
up. Be specific (what requirement should be added or changed, and why).
The current process and SCM requirements are the evolutionary result of
13+ years using Teamware. They can evolve some more to meet the needs
of OpenSolaris.
cheers,
mike
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Mike Kupfer
kupfer@athyra.sfbay....
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Dec 21, 2005 10:29 AM
in response to: benr
|
|
>>>>> "Ben" == Ben Rockwood <benr at cuddletech dot com> writes:
Ben> I'm curious where this process is leading. I don't know of any
Ben> tool that meets all these requirements except perhaps BitKeeper.
Are you referring to all the listed requirements, or just the "must
have" list?
Ben> This all sounds more like a plan to create a new SCM rather than
Ben> adopt an existing one,
Ick, no. I can't speak for Stephen (who is away this week), but my
understanding of the requirements is they are based on the ON group's
13+ years experience of using Teamware: what is needed to support a very
large, globally dispersed, development community.
I don't know of anyone who wants to create a new SCM. We may find that
none of the existing tools meets the stated requirements. In that case,
I see the options as relaxing the stated requirements, improving
existing tools, or both.
Ben> Are we looking for a tool that existing TeamWare gates could pump
Ben> incrementals to or a single shared internal/external source code
Ben> management system that would phase out TeamWare?
We're looking for a single system for a given consolidation. The
long-term goal is to have the master workspace be external.
Not all the consolidations need use the same SCM. Consolidations with
smaller code bases or smaller numbers of contributors might not care
about some of the requirements in the requirements list.
mike
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Stephen Hahn
sch@eng.sun.com
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Jan 6, 2006 4:02 PM
in response to: Mike Kupfer
|
|
I think there are two key concerns raised in this thread. I'll deal
with them by complexity.
A. Are we proposing writing a new SCM, as the complete set of
requirements seem essentially unsatisfiable?
No. There is no intent to start a "next next generation SCM"
development project as a result of this requirements draft. There
is a large pool of OSS (or potentially OSS for the TeamWare case)
distributed SCM solutions; we are seeking to find the best fit to
current distributed development practices as embodied by the
consolidations presently using TeamWare. Once that best fit
system has been selected, the opensolaris.org community might
choose to become more involved with the development of that
system, if critical missing capabilities become evident. However,
we cannot just sit and wait for the perfect SCM solution to come
along--we may have to choose one that lacks some of the features
TeamWare has (in the happy little world of the SWAN) and switch
and improve things over time.
B. Are the tools driving the process, or the process the tools?
The process is driving the tools choice. There have been a number
of assertions about how the tools choice mandates the relationship
between developers, the nature of shared state, or even the warp
and weft of the social fabric of the entire developer community.
(Such ideas are interesting, and I would love to read any extended
essays (with citations) that make the connections explicit and
complete the obvious analogy to the Sapir-Whorf hypothesis in
linguistics.) However, I think it would be more helpful to talk
about how the typical developer interaction goes on today, at
least for ON. (I know some of the other consolidations use this
exact model, but with different tools, and some eliminate certain
of the checks or restrictions in the ON process.)
1. The developer takes a child of the repository. If this is
his/her first child for the release, they are automatically
added to the alias that gets notices of commits and undos.
2. The developer fixes one or more bugs.
3. The developer solicits code reviews from nearby peers, from
remote peers, from area experts--whatever is appropriate for
the complexity/risk associated with changes in that area.
This step may be repeated until the reviewers are satisfied
with the proposed fix.
4. The developer submits a "request to integrate" to a specific
member of the "change review team" (CRT), identifying the bugs
addressed, the workspace with the changes, and the code
reviewers (and a little more about risk, impact, and other
details about the change). This CRT member may not be
one of the listed code reviewers. The CRT member may accept
the request, or seek more detail, or take an action to
reassign to a more expert CRT member. He/she may also request
additional relevant reviewers examine the change prior to
approval. (There are more outcomes: an interface change may
require ARC approval, etc.) If the request is accepts, then
we say the "RTI is approved".
5. With an approved RTI, the developer can connect to the gate
repository and integrate. Directly--meaning that there are
over 800 active committers to just ON.
While aspects of this process may need to be modified or need
further web support (like finding a code reviewer), it's our
intent for ON to preserve the "many committers" model. In fact, I
assess that a number of the folks submitting code changes would
be committers if we had already resolved the distributed SCM
choice.
Now, you could pretty easily argue that any SCM can handle the
above series of steps--and you would be mostly correct. There are
three reasons that distributed SCM tools tend to be very helpful:
1. Clone repositories. The notion of a read-only clone
repository that people can follow and pull from insulates the
bulk of developers from broken integrations (because the
gatekeepers undo them in the main repository before they hits
the clones.) Clones are also nice to have in the different
geographies, as it means that the slow link is only used for
commits (which tend to be small), while the clone's pulls can
be automated.
With a distributed SCM, being a child of the clone isn't a
penalty; one simply commits to the main repository.
2. Trial commits. Even though the release team catches build
errors quickly, it's best--least other people's time
wasted--if such errors are never introduced. One such
technique is to do a "trial commit" to a new copy of the main
repository, and then do a build of the copy. If you forgot a
file creation, rename, or deletion in your commit, this
technique will catch it.
3. Projects. Large changesets that touch many files across the
system, and involve multiple developers to construct, need to
manage the incoming rate of change from the main release to
make progress. Such efforts typically construct a "project
gate", a child repository that the project team in turn takes
children from and commit to as the project proceeds.
There were numerous project gates running through 5.10 for ON;
I can think of a few for kernel and networking for Nevada
already.
The main point for the distributed SCM is that (a) these are all
just clone/child repository operations and (b) the release team
takes no special action to enable any of them, without reference
to the number required.
Obviously, I am happy to hear further questions or comments.
- Stephen
--
Stephen Hahn, PhD Solaris Kernel Development, Sun Microsystems
stephen dot hahn at sun dot com http://blogs.sun.com/sch/
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Posts:
64
From:
Registered:
6/14/05
|
|
|
|
Re: Distributed SCM requirements draft
Posted:
Jan 6, 2006 8:45 AM
in response to: benr
|
|
> I'm curious where this process is leading. I don't
> know of any tool that meets all these requirements
> except perhaps BitKeeper.
I'm taking a look at these with monotone (http://www.venge.net/monotone) as candidate here:
E0: yes. monotone is GPL
E1: yes. it works peer to peer between nodes, no parent-child relation exists apart from default server settings, and potentially a central server by decree of the users
E2: yes (in a branch). monotone has its own, efficient protocol. there's a branch that provides support for tunneling through ssh (or through pipes in general). (iirc) not merged as of yet mostly because the win32 side of that feature is lacking.
E3: limited. protocol, storage layout and hook scripts are still subject to change, but developers care for stability (except when it really gets in the way) and always provide migration guides (for hooks) and tools (for storage)
E4: yes. supports transactional and versioned rename and delete of files. support for directories is in progress (soon finished)
E5: yes. monotone allows many "certs" per revision which are cryptographically signed by the issuer. they are unchangeable (due to synchronization issues in distributed designs). certs are key/value pairs (textfields)
C6: yes. single binary, some info file and locales. extensive documentation. UI as simple as devs and users could do it. 3way merge integrated, hooks for graphical conflict resolution tools. webtools exist but may need refinement. reverting a changeset works by applying the inverse (monotone takes care of that)
C7: yes. monotone works on sqlite databases. concurrency is limited, but it can be done over NFS. may need improved error resolution
C8: yes. the developers are very active and helpful on the mailing list and on IRC. many changes in the tool happened because of user needs.
C9: TBD
C10: yes. monotone provides an stdio interface by which one monotone process can do many separate jobs. pretty standardized interface with computer parsable output. afaik termination of operations is not possible at this time
C11: yes. the database has ACID properties. every full checkout contains exactly the same dataset, so that fallback/syncback provides the same view again
C12: yes. internally monotone keeps everything as binary and stores revisions per xdelta (binary diff). content encoding conversion (CR/LF, utf8<->other encodings) can be done by monotone on a per file basis, defaulting to off.
O13: yes. called "restrictions". partial commits etc. work, partial checkout is work in progress.
O14: limited. not directly, but per-file caching of revision numbers could be implemented
that makes one unanswered (C9 - the one about people in the opensolaris community taking care of things - can't happen yet), and two limited (O14 - per file history, E3 - protocol stability. the latter being a big issue).
size might be an issue, but there is a means to cache "inode signatures" in a checkout to shortcut most operations (mostly diff and all that depend on it) as long as the inode signature (inode number, date, size, ...) stays the same. might not work with checkouts on an NFS share (but with checkouts from a repository on an NFS share, naturally)
there is one big change in repository format soon (see E3, for the curious, see http://lists.gnu.org/archive/html/monotone-devel/2005-11/msg00102.html or search for "rosters" on that mailinglist), which will fix many small issues (certain merge scenarios, ...), but I don't think that the decision and some kind of migration would happen over the next two weeks anyway.
user experience shouldn't change too much, so it should be possible to take a look at things right now already and get a feel about the general feasibility and necessary changes.
patrick mauritz
|
|
|
|
|
Stephen Hahn
sch@eng.sun.com
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Jan 6, 2006 3:21 PM
in response to: oxygene
|
|
* Patrick Mauritz <oxygene at studentenbude dot ath dot cx> [2006-01-06 08:51]:
> I'm taking a look at these with monotone (http://www.venge.net/monotone) as candidate here:
Patrick,
Thanks; this summary is very helpful.
1. I suppose one thing that would be interesting would be timing for
trivial operations, like taking a recent drop of the ON sources,
importing them into a repository, and then doing some timing of
pulling a child/clone on local filesystems or over a LAN.
2. Do you know of a project or group of folks using Monotone--or what
you've been using it for?
Cheers
Stephen
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Posts:
64
From:
Registered:
6/14/05
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Jan 7, 2006 12:39 AM
in response to: Stephen Hahn
|
|
> 1. I suppose one thing that would be interesting would be timing for
> trivial operations, like taking a recent drop of the ON sources,
> importing them into a repository, and then doing some timing of
> pulling a child/clone on local filesystems or over a LAN.
for monotone, the topology is a bit different: every user has a database (literally, it's an sqlite file), which is the repository. you can checkout from databases (which is a local operation, always), and you can push/pull/sync between databases (which is a remote operation).
so a checkout from remote is always a two-step operation, first pull, then checkout. this is deliberate, as sort of a "firewall" between the checkout and remote data.
I'd like to collect a bunch of operations that are of interest before benchmarking. things that come to mind:
- sync two repositories (might be the same as checkout for some tools - not a good benchmark to compare different tools because of the huge differences in what's transferred, but might still be interesting)
- checkout a full tree (with unpacking a tarfile of the same dataset as comparison)
- changing a couple of files, diff (with both "all optimizations" the tool provides, and none), multiple times for cold and hot cache behaviour
- changing a couple of files, adding some, renaming some, "status" (or whatever it is to get an overview of changed files, without the diffs)
- commit
- sync to server
- (repeat change + commit step multiple times)
- sync to server (multiple commits)
- checkout old tree, update with 1 commit
- checkout old tree, update to newest (all commits)
- annotate a file (if available)
anything else?
> 2. Do you know of a project or group of folks using Monotone--or what
> you've been using it for?
http://www.venge.net/monotone/wiki/ProjectsUsingMonotone
as you see, all of them are relatively small, both in number of revisions and number of files, at least compared to opensolaris.
I use monotone for a couple of small trees (in the dozens of files), and I also used it to track the opensolaris source drops, though I'm currently waiting for the rosters branch (that storage redesign) before I start on that work again, as it should fix a couple of things.
it took 5-10minutes for the commit of each opensolaris revision, but it's a special case because I had new trees, so the inode fingerprint optimization can't kick in. with fingerprints, it only has to stat each file that is unchanged, without them it has to run them through sha1.
patrick mauritz
|
|
|
|
|
Posts:
1,218
From:
Registered:
3/9/05
|
|
|
|
Re: Re: Re: Distributed SCM requirements draft
Posted:
Jan 7, 2006 2:03 AM
in response to: oxygene
|
|
Patrick Mauritz wrote:
> for monotone, the topology is a bit different: every user has a database (literally, it's an sqlite file), which is the repository. you can checkout from databases (which is a local operation, always), and you can push/pull/sync between databases (which is a remote operation).
> so a checkout from remote is always a two-step operation, first pull, then checkout. this is deliberate, as sort of a "firewall" between the checkout and remote data.
My experiences of sqlite have been less than good - I used it for some
of the source analysis work but it was so flaky I had to ditch it and
switch to MySQL. For that reason alone I'd be inclined to say 'no' to
monotone.
--
Alan Burlison
--
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Posts:
64
From:
Registered:
6/14/05
|
|
|
|
Re: Re: Re: Distributed SCM requirements draft
Posted:
Jan 7, 2006 2:18 AM
in response to: alanbur
|
|
hmm.. never had issues with it, what exactly were your problems?
also, svc seems to be using it: /lib/svc/bin/sqlite
patrick mauritz
|
|
|
|
|
Posts:
1,218
From:
Registered:
3/9/05
|
|
|
|
Re: Re: Re: Re: Distributed SCM requirements draft
Posted:
Jan 7, 2006 11:00 AM
in response to: oxygene
|
|
Patrick Mauritz wrote:
> hmm.. never had issues with it, what exactly were your problems?
> also, svc seems to be using it: /lib/svc/bin/sqlite
Basically its grasp of SQL is tenuous at best, it's as buggy as hell,
the bugs take forever to get fixed, and if you want to do a query that's
remotely complex the chances are you will get the wrong results back.
If all you want is simple lookups on single tables, it will probably be
fine - but in that case you'd probably be better off using Berkley DB
anyway.
--
Alan Burlison
--
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Posts:
64
From:
Registered:
6/14/05
|
|
|
|
Re: Re: Re: Re: Distributed SCM requirements draft
Posted:
Jan 7, 2006 2:32 PM
in response to: alanbur
|
|
monotone uses sqlite as its storage layer - the sql-subset statements are written for and tested on it, and bugs can be fixed in monotone's internal copy of sqlite.
I might agree to your doubts if sqlite were an (or the default) option besides other SQLdbs, but it isn't. while I don't think we can claim extensive testing of any subset of sqlite, issues with sqlite in monotone should appear pretty quickly because it's the sole option.
sure, sqlite has its share of issues every now and then, but so does berkeley db (it was famous for killing rpm's databases for a while) and any other db (or software package).
maybe you expected more from it (eg. more complete SQL adherence), maybe people expected too much from berkeley db where it failed - for the use cases where they're successfully employed, they tend to work fine though (otherwise they'd be dropped, or fixed - and monotone dropped lots of libraries because they didn't work). I think, monotone is such a use case for sqlite.
patrick mauritz
|
|
|
|
|
Posts:
276
From:
US
Registered:
3/9/05
|
|
|
|
Re: Re: Re: Re: Distributed SCM requirements draft
Posted:
Jan 7, 2006 11:22 AM
in response to: oxygene
|
|
Patrick Mauritz writes:
> hmm.. never had issues with it, what exactly were your problems?
> also, svc seems to be using it: /lib/svc/bin/sqlite
I won't try to speak to Alan's problems with sqlite, but we're using a
restricted and well-tested (by us) subset of functionality in sqlite
for SMF. Our use of sqlite doesn't constitute endorsement for general
purpose use, only endorsement of its suitability for the subset of
functionality that we require. For that subset, it works well for us.
liane
--
Liane Praza, Solaris Kernel Development
liane dot praza at sun dot com - http://blogs.sun.com/lianep
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Posts:
17
From:
GB
Registered:
3/9/05
|
|
|
|
Re: Distributed SCM requirements draft
Posted:
Jan 9, 2006 3:17 AM
in response to: Stephen Hahn
|
|
Wouldn't it be a better idea to focus on the content of the document before we start trying to apply the document to any evaluations? I think that was the original purpose of Stephen's posting.
Anyway, with that in mind, here's my set of comments.
Line 18 "believe" should be "believed".
C6 should, perhaps, be broken down into separate requirements. I think, for instance, that the last paragraph of this requirement is "Essential" and not "Conditional".
C11, should be "Essential" and not "Conditional", since I think that an SCM lacking these characteristics would be unacceptable.
In fact, wouldn't it be better to combine the last paragraph of C6 with C11 and merge them with E4?:
+++START+++
E4. Transactional Operation
The candidate SCM must provide transactional operations on its repository, with clearly defined semantics. Mechanisms for recovering from system failure must be provided, with preference given to systems which automate such recovery. Undoing the application of a specific changeset ("backout") must be easy. Renaming and deletion of repository sub-trees must be supported.
NB: A history-preserving copy operation, followed by a delete operation, may be considered
equivalent to a rename; equivalency, and the reasons for rename omission, are to be assessed
and documented by the evaluating engineer.
+++END+++
I really racked my brains trying to think of additional requirements that were absent and I'm pleased to say that I can't think of any. However, I'd like everyone reading this thread to work as hard as they can on this aspect of the document because it's really important that we fully understand all the SCM requirements before we proceed to evaluation. In particular, input from non Sun employees is greatly valued as their perspectives may help them identify absent requirements.
I'd like to see a couple of phrases clearly defined somewhere in the document. My candidate definitions are included for comment:
changeset - A set of identifiable changes applied between repositories with associated metadata.
SWAN - Sun Wide Area Network, the Sun internal network.
|
|
|
|
|
Posts:
64
From:
Registered:
6/14/05
|
|
|
|
Re: Distributed SCM requirements draft
Posted:
Jan 9, 2006 4:00 AM
in response to: garypen
|
|
> Wouldn't it be a better idea to focus on the content
> of the document before we start trying to apply the
> document to any evaluations? I think that was the
> original purpose of Stephen's posting.
it seemed like a good way to get people talking about tools, and their requirement again.
> C6 should, perhaps, be broken down into separate
> requirements.
totally agree - I felt that way while applying it to monotone, but didn't want to clutter the eval. I think it's telling that I had to answer multiple questions for that point..
> In fact, wouldn't it be better to combine the last
> paragraph of C6 with C11 and merge them with E4?:
that would combine things even more.. it might make sense to move them together, but these are different issues (transactional operation, error recovery, undo operators), so maybe make them E4, E4a, E4b (or better, E4, E5, ..)
having transactional operation doesn't garantee the presence of undo (see gnu arch, undoing a patch generates some problems, no matter how you do it - but it's transactional), or error recovery (subversion with berkeley db backend was nasty in that regard for a while)
> it's really important that we fully understand all
> the SCM requirements before we proceed to evaluation.
I doubt we'll understand them "all" before setting up test scenarios, preferably with different tools.. every weakness of a tool that gets in the way generates a new requirement... some might not apply to the other tools at all because of their different design, but that's an answer to a requirement, too.
> In particular, input from non Sun employees is
> greatly valued as their perspectives may help them
> identify absent requirements.
in general, I found everything I cared about, otherwise I had added more points to my evaluation.
> changeset - A set of identifiable changes applied
> between repositories with associated metadata.
that's where trouble starts with different SCM tools and terminology: they can be different things for different tools
for some, changesets are really snapshots with delta storage. for others, they are "patches" that can be applied in a certain order and only taken away in the reverse order, for others, each can be applied and removed at will.
that can also affect how well a tree state, a checkout that is defined by a given set of changesets, is defined.
similar for repository: some call them database, some archive, some repository. for some, a repository (ie. one of the aliases above) is the same as a checkout (as these carry everything), or a local entity separate to a checkout, or a remote entity. (or any combination of those)
because of issues like these introducing another set of definitions should be done with great care - so that a definition helps to figure out what's being talked about, but that it's still loose enough to not exclude a tool just because the requirements become overly rigid by accident.
|
|
|
|
|
Posts:
53
From:
Registered:
3/9/05
|
|
|
|
Re: Distributed SCM requirements draft
Posted:
Feb 16, 2006 7:50 PM
in response to: garypen
|
|
As a user I would like to add the following features to the list of requirements:
1. Preview.
Each transaction should have a preview mode, which allows to see what will happen
during transaction, but without any changes in developer's and integration areas.
2. Atomic transaction.
Each transaction should be atomic, which means it should lock the necessary part of
the integration area. Ideally the lock should "disappear" if the transaction was interrupted
in any way (system reboot, kill -9, lost connection - whatever).
3. Undo
There should be an easy way to undo a "wrong" transaction. Ideally there should be a
way to undo all transactions, one by one, but at least the last one.
4. Filemerge
There should be a comfortable tool to do a 3-way merge (parent, child, ancestor).
Something like filemerge in TeamWare.
|
|
|
|
|
Bryan O'Sullivan
bos@serpentine.com
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Feb 16, 2006 9:52 PM
in response to: nikm
|
|
On Thu, 2006-02-16 at 19:50 -0800, Nikolay Molchanov wrote:
> 1. Preview.
> Each transaction should have a preview mode, which allows to see what will happen
> during transaction, but without any changes in developer's and integration areas.
What do you mean by a transaction here? Most tools have the equivalent
of a "status" command already, which tells you everything you need to
know about changes that you've made within your local working copy.
This, together with a "diff" command, essentially acts as a preview of
what a commit will contain.
For the other major transactional operations, push and pull, Mercurial
provides "incoming" (what would a "pull" bring in?) and "outgoing" (what
would a "push" send out?) commands.
> 2. Atomic transaction.
> Each transaction should be atomic, which means it should lock the necessary part of
> the integration area.
All DSCMs that I know of lock a repository during a commit.
> Ideally the lock should "disappear" if the transaction was interrupted
> in any way (system reboot, kill -9, lost connection - whatever).
This is next to impossible to do in the presence of NFS, which is an
exceptionally common use case (nobody trusts POSIX locks to be turned on
or work over NFS, so people use lock directories or symlinks instead,
where the semantics of creation are well defined and usually work). The
best you can really hope for there is "pid X on host Y has the repo
locked".
> 3. Undo
> There should be an easy way to undo a "wrong" transaction. Ideally there should be a
> way to undo all transactions, one by one, but at least the last one.
Mercurial has an "undo" command that rolls back the last transaction.
What this means is that if a "pull" brings in 4 changesets in a single
transaction, an undo that immediately follows will make them all go
away.
You can make history prior to the last transaction disappear using the
"clone -r" (built in) or "strip" (available as part of the mq extension)
commands. The strip command actually saves a "bundle" of the changes
that it removes from the repository, so if you decide you stripped the
wrong changes, you can reapply them.
> 4. Filemerge
> There should be a comfortable tool to do a 3-way merge (parent, child, ancestor).
> Something like filemerge in TeamWare.
Most DSCMs punt to an external tool for this, and let you plug in
whatever tool you personally prefer. It's been years since I've used
Teamware, but I think that almost any of the modern free merge tools is
likely to be quite a bit better than my admittedly fuzzy recollection of
Teamware's filemerge tool.
Some DSCMs do a better job at automated merges than others. Bazaar-NG
uses an SCCS-like weave to provide history-sensitive automated merges.
This merge machinery has been ported to run under Mercurial, too, but
it's not yet available by default. I'd expect it to be a bit more work
to port it to Monotone, but it shouldn't be hard in that case, either.
There's also been quite a bit of quasi-theoretical work in the free DSCM
community on automated history-sensitive merging. None of it has really
seen practical implementation yet, but some of the approaches appear to
show promise.
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Stephen Hahn
sch@eng.sun.com
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Feb 17, 2006 12:43 AM
in response to: nikm
|
|
* Nikolay Molchanov <Nikolay dot Molchanov at Sun dot COM> [2006-02-16 19:51]:
The requirements document is not, in general, going to change at this
point in the evaluation. Specific comments follow:
> As a user I would like to add the following features to the list of requirements:
> 1. Preview.
> Each transaction should have a preview mode, which allows to see what will happen
> during transaction, but without any changes in developer's and integration areas.
A preview mode, or dry run mode, or do nothing mode is not strictly
necessary on transactions. An implementation might instead choose to
have an accurate child status transaction. Because of this
equivalence, neither E3 nor C11 require a preview mode.
> 2. Atomic transaction.
> Each transaction should be atomic, which means it should lock the necessary part of
> the integration area. Ideally the lock should "disappear" if the transaction was interrupted
> in any way (system reboot, kill -9, lost connection - whatever).
> 3. Undo
> There should be an easy way to undo a "wrong" transaction. Ideally there should be a
> way to undo all transactions, one by one, but at least the last one.
These are specific implementation choices that satisfy C11.
> 4. Filemerge
> There should be a comfortable tool to do a 3-way merge (parent, child, ancestor).
> Something like filemerge in TeamWare.
All of the candidates can use meld, kdiff, and the many other
pick-your-toolkit 3-way diff merge tools. Some candidates go further
in automating merge resolution, but the requirements don't demand
automated merging.
If people would like clarifications of this kind added to the
requirements, then that is fine: I can make such edits.
- Stephen
--
Stephen Hahn, PhD Solaris Kernel Development, Sun Microsystems
stephen dot hahn at sun dot com http://blogs.sun.com/sch/
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
Posts:
976
From:
US
Registered:
3/9/05
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Feb 17, 2006 5:35 AM
in response to: Stephen Hahn
|
|
On Fri, 2006-02-17 at 00:43 -0800, Stephen Hahn wrote:
> > As a user I would like to add the following features to the list of requirements:
> > 1. Preview.
> > Each transaction should have a preview mode, which allows to see what will happen
> > during transaction, but without any changes in developer's and integration areas.
>
> A preview mode, or dry run mode, or do nothing mode is not strictly
> necessary on transactions. An implementation might instead choose to
> have an accurate child status transaction. Because of this
> equivalence, neither E3 nor C11 require a preview mode.
yep, though preview/dry run has an important usability advantage; when
you get to the point where you're ready to putback/commit/integrate, you
can do a dry run and check the output, then (using a shell with history
editing) delete the "-n" or "--dry-run" or whatever the option is out of
the previous command and run it for real.
- Bill
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
James Blackwell
jblack@merconline.com
|
|
|
|
Re: Re: Distributed SCM requirements draft
Posted:
Feb 17, 2006 12:01 PM
in response to: nikm
|
|
|
|
On Thu, Feb 16, 2006 at 07:50:38PM -0800, Nikolay Molchanov wrote:
> As a user I would like to add the following features to the list of requirements:
> 1. Preview.
> Each transaction should have a preview mode, which allows to see what will happen
> during transaction, but without any changes in developer's and integration areas.
Yes. You need this sort of thing in centralized revision contol systems.
In those systems, you can end up with a tree that has changes that don't
live in any RCS. One bad pull with lots of conflicts, and you've lost your
code. This problem doesn't really exist in the DRCS world. Users are much
more inclined to keep their local changes in a local RCS branch. If an
operation goes bad, then one just reverts the tree and still has their
local changes:
Centralized RCS | Decentralized RCS
1. make changes | 1. branch
2. pull | 2. change, commit
3. change, change, change | 3. merge
4. pull | 4. change&commit, change&commit, change&commit
5. BOOM* | 5. Merge & BOOM**
* "Why couldn't I have done a preview before I had this mess?"
** In the DRCS world we would run some equiv to bzr revert and consider
possible plans of attack.
That said, it doesn't mean that one can't preview. The DRCS world
typically supports people that like to do dry runs by giving the user the
ability to easily copy the working tree and perform a dry run there.
> 2. Atomic transaction.
> Each transaction should be atomic, which means it should lock the necessary part of
> the integration area. Ideally the lock should "disappear" if the transaction was
> interrupted in any way (system reboot, kill -9, lost connection - whatever).
Any DRCS on my radar supports atomic transactions. It kind of comes with
the business. The locking business is a different story. Presume that I am
A, you are B and we are both pushing to a filesystem on a different
machine, C. Without a server involved, its difficult for B to tell whether
A gave up the ghost while talking to C, or is just slow.
There's no algorithmic away around it because the remote filesystems
supported these days do not provide when it comes to locking. That said, a
DRCS can dothings like guess that if a lock is old or is "yours"
(with care!), that it it can be disregarded.
> There should be an easy way to undo a "wrong" transaction. Ideally there should be a
> way to undo all transactions, one by one, but at least the last one.
Some decentralized revision control systems, including Bazaar-NG, support
uncommit. If a DRCS allows uncommit, then it will allow as any uncommits
as you want.
Most decentralized revision control systems (again, including Bazaar-NG)
will allow one to reverse merge old revisions as well.
> 4. Filemerge
> There should be a comfortable tool to do a 3-way merge (parent, child, ancestor).
> Something like filemerge in TeamWare.
Revision control systems these days (including bazaar-ng) tend to avoid
advanced UIs and instead rely upon third party tools. This isn't because
revision control systems think that real men use vi and real women use
emacs (switch according to gender and editor preference). Its because
graphical UIs cut back on portability.
My personal favorite merge utility is meld. It has support for a variety
of revision control systems and has a version floating around with
bazaar-ng support.
Regards,
James
--
My home page: James Blackwell
Gnupg 06357400 F-print AAE4 8C76 58DA 5902 761D 247A 8A55 DA73 0635 7400
_______________________________________________
tools-discuss mailing list
tools-discuss at opensolaris dot org
|
|
|
|
|
