|
Replies:
9
-
Last Post:
Jun 12, 2008 8:36 AM
by: bustos
|
|
|
Posts:
303
From:
Dublin, Ireland
Registered:
10/17/05
|
|
|
|
some thoughts on SMF backend support for handling
ordering of property values
Posted:
Jun 10, 2008 8:18 AM
|
|
hi folks
I've been looking into a candidate solution for supporting the ordering of property values in the SMF repository. The subject of the ordering of property values has come up here a few times before, for example see:
http://www.opensolaris.org/jive/thread.jspa?messageID=53690톺
A key advantage of preserving ordering would be that it would no longer be necessary to store order-sensitive multiply-valued values in a single astring property using a separator to delimit the individual values.
The issue is that at present, SMF properties with multiple values are not stored in the repository in a manner which allows the values to be retrieved in the same order in which they were added. To support such ordering, a few things need to be done. Firstly, libscf, more specifically, scf_entry_add_value() - the function which collates values for commitment to the repository - needs to collect values in the order in which they were added. At present it adds values at the head of the list for commitment, so that we commit values in reverse order. This part is easily fixed by adding new values at the tail of the list. From there, the repository protocol needs to do the right thing with respect to ordering, which I believe it does. Finally, at the backend, in the sqlite .db files which comprise the repository, we need to store values in such a way that they can be retrieved in order. At present, property values are stored as multiple rows in a table (value_tbl), and are associated with a particular property via an identifier they share (value_id). When these are retrieved, we carry out a SELECT operation which selects all values with the required value_id, specifying no particular order (though it seems like we get the right order due to sqlite implementation details which we shouldn't rely on).
To support ordering at the repository level, I suggest adding an additional column to the value_tbl, named value_order. When we add property values, we would need to also specify ordering (0, 1, 2, ..) for this column. On retrieval then, we can modify the backend SELECT statement to ORDER BY the value_order column and thus ordering is preserved.
A major concern with such a change of course is handling upgrade to a new repository format, while also supporting downgrade from the new format to the previous format. To ensure svc.configd can deal with the repository format, the repository carries a schema version number which must match the version encoded in svc.configd in order for the repository to be usable. My suggestion is that we do not bump this version number for this change. A downgraded svc.configd can actually handle a repository with the extra value_order column perfectly (it merely ignores it). It might make sense introduce a minor version number into the schema_version table, to reflect that the schema has changed, but not in an incompatible manner.
This leaves the issue of upgrading to the new repository format. Ideally, we would like to add the new column as soon as the repository is writable on upgrade, but we have to deal with a few complications:
- during boot, we need to read property values prior to the point when we can do the repository upgrade (i.e. prior to the repository being writable) - the version of sqlite used does not support the ADD COLUMN command, so adding a new column to the value_tbl is not straightforward.
To address the first issue, we need to keep track of whether upgrade has been carried out for a particular backend. This is important, because during first boot post-upgrade, we will need to read values before we can write to (and upgrade) the repository. This means we need to modify the SELECT statement that retrieves values based on whether the upgrade has been done (since prior to upgrade it cannot use the ORDER BY value_order clause in the SELECT statement that retrieves values).
To address the second issue, we just need to do a more complex dance of creating a temporary table which includes the new column, populate it with the value_tbl values, delete the original value_tbl, create the new value_tbl (with the additional column), populate it with the values from the temporary value table, and finally delete the temporary value table. Testing indicates this just takes a few seconds.
I've posted a webrev which contains what I have so far:
http://cr.opensolaris.org/~amaguire/smf_values/
I've confirmed this approach works, and supports upgrade/downgrade, but I think it needs some additional investigation to be determined sufficiently robust (I still see one complaint about the lack of a value_order column during first boot post-bfu). I'm assuming we'd also need to assess the impact on performance (if any) that adding an additional column to the value table would have, but I thought it'd be useful to have a rough candidate solution to start discussing how/if the ordering issue can be resolved.
Comments are appreciated!
Alan _______________________________________________ smf-discuss mailing list smf-discuss at opensolaris dot org
|
|
|
Posts:
941
From:
Redwood City, CA
Registered:
3/9/05
|
|
|
|
Re: some thoughts on SMF backend support for handling
ordering of property values
Posted:
Jun 10, 2008 3:54 PM
in response to: amaguire
|
|
Quoth Alan Maguire on Tue, Jun 10, 2008 at 04:19:31PM +0100: > I've been looking into a candidate solution for supporting > the ordering of property values in the SMF repository.
Wow, Alan, this is great work! I hope hacking on svc.configd didn't make you pull too much hair out.
> The subject of the ordering of property values has come > up here a few times before, for example see: > > http://www.opensolaris.org/jive/thread.jspa?messageID=53690톺
So what is your take on Stephen's network repository issue? I wonder if we could add a piece of metadata to properties which specify whether they're ordered or unordered. By default they'd be ordered, but if an application doesn't care then it can make them unordered, which would free svc.configd to avoid emulating order in an unordered backend. Or maybe the ordering should be a flag on the retrieval interface. Or maybe the current retrieval interfaces should be designated "ordered", and we can add separate unordered retrieval interfaces. If we wanted to do this, we could probably put it off actual unordered properties until we have a network backend, and make any interface changes as opportunities arise.
> To support ordering at the repository level, I suggest > adding an additional column to the value_tbl, named > value_order. When we add property values, we would > need to also specify ordering (0, 1, 2, ..) for this column. > On retrieval then, we can modify the backend SELECT > statement to ORDER BY the value_order column and > thus ordering is preserved.
Sounds good.
> A major concern with such a change of course is > handling upgrade to a new repository format, while > also supporting downgrade from the new format to the > previous format. To ensure svc.configd can deal > with the repository format, the repository carries a > schema version number which must match the version > encoded in svc.configd in order for the repository > to be usable. My suggestion is that we do not bump > this version number for this change. A downgraded > svc.configd can actually handle a repository with the > extra value_order column perfectly (it merely ignores it). > It might make sense introduce a minor version number into > the schema_version table, to reflect that the schema > has changed, but not in an incompatible manner.
Is supporting downgrade a hard requirement?
> This leaves the issue of upgrading to the new > repository format. Ideally, we would like to > add the new column as soon as the repository > is writable on upgrade, but we have to deal > with a few complications: > > - during boot, we need to read property values prior to the > point when we can do the repository upgrade (i.e. prior to > the repository being writable) > - the version of sqlite used does not support the ADD COLUMN > command, so adding a new column to the value_tbl > is not straightforward. > > To address the first issue, we need to keep track of > whether upgrade has been carried out for a particular > backend. This is important, because during first boot > post-upgrade, we will need to read values before > we can write to (and upgrade) the repository. This > means we need to modify the SELECT statement > that retrieves values based on whether the upgrade > has been done (since prior to upgrade it cannot use > the ORDER BY value_order clause in the SELECT > statement that retrieves values).
True. We should be able to assume that the nonpersistent repository is in the new format, though, shouldn't we? Unless I suppose we want to support changing the svc.configd binary without rebooting.
Did you happen to determine when ADD COLUMN was added? If was before 3.0, upgrading might be an option.
David
> I've posted a webrev which contains what I have so > far: > > http://cr.opensolaris.org/~amaguire/smf_values/ > > I've confirmed this approach works, and supports > upgrade/downgrade, but I think it needs some additional > investigation to be determined sufficiently robust (I still see > one complaint about the lack of a value_order column during > first boot post-bfu). I'm assuming we'd also need to assess the > impact on performance (if any) that adding an additional column > to the value table would have, but I thought it'd be useful to have > a rough candidate solution to start discussing how/if the ordering > issue can be resolved. _______________________________________________ smf-discuss mailing list smf-discuss at opensolaris dot org
|
|
|
|
Posts:
303
From:
Dublin, Ireland
Registered:
10/17/05
|
|
|
|
Re: some thoughts on SMF backend support for handling
ordering of property values
Posted:
Jun 11, 2008 2:55 AM
in response to: bustos
|
|
David Bustos wrote: > >> The subject of the ordering of property values has come >> up here a few times before, for example see: >> >> http://www.opensolaris.org/jive/thread.jspa?messageID=53690톺 >> > > So what is your take on Stephen's network repository issue? I wonder if > we could add a piece of metadata to properties which specify whether > they're ordered or unordered. By default they'd be ordered, but if an > application doesn't care then it can make them unordered, which would > free svc.configd to avoid emulating order in an unordered backend. Or > maybe the ordering should be a flag on the retrieval interface. Or > maybe the current retrieval interfaces should be designated "ordered", > and we can add separate unordered retrieval interfaces. If we wanted to > do this, we could probably put it off actual unordered properties until > we have a network backend, and make any interface changes as > opportunities arise. > > Nice summary of the options. Defaulting to ordered storage/retrieval would be my preference here, with possible future changes based on explicit requirements for unordered value support. I'm not quite sure what constraints the network repository requirements would impose - what do you think Stephen?
>> To support ordering at the repository level, I suggest >> adding an additional column to the value_tbl, named >> value_order. When we add property values, we would >> need to also specify ordering (0, 1, 2, ..) for this column. >> On retrieval then, we can modify the backend SELECT >> statement to ORDER BY the value_order column and >> thus ordering is preserved. >> > > Sounds good. > > >> A major concern with such a change of course is >> handling upgrade to a new repository format, while >> also supporting downgrade from the new format to the >> previous format. To ensure svc.configd can deal >> with the repository format, the repository carries a >> schema version number which must match the version >> encoded in svc.configd in order for the repository >> to be usable. My suggestion is that we do not bump >> this version number for this change. A downgraded >> svc.configd can actually handle a repository with the >> extra value_order column perfectly (it merely ignores it). >> It might make sense introduce a minor version number into >> the schema_version table, to reflect that the schema >> has changed, but not in an incompatible manner. >> > > Is supporting downgrade a hard requirement? > > If we explicitly bumped the version number - and thus needed to do an explicit downgrade - we could add logic to bfu etc to remove the value_order column (this could be done using /lib/svc/bin/sqlite). In testing, it does seem like earlier configd versions can handle a repository with the additional value_order column, but it seems a bit hacky not to register the fact that the repository is upgraded just to make downgrade easier (adding a minor version to the schema_version table might be enough though). >> This leaves the issue of upgrading to the new >> repository format. Ideally, we would like to >> add the new column as soon as the repository >> is writable on upgrade, but we have to deal >> with a few complications: >> >> - during boot, we need to read property values prior to the >> point when we can do the repository upgrade (i.e. prior to >> the repository being writable) >> - the version of sqlite used does not support the ADD COLUMN >> command, so adding a new column to the value_tbl >> is not straightforward. >> >> To address the first issue, we need to keep track of >> whether upgrade has been carried out for a particular >> backend. This is important, because during first boot >> post-upgrade, we will need to read values before >> we can write to (and upgrade) the repository. This >> means we need to modify the SELECT statement >> that retrieves values based on whether the upgrade >> has been done (since prior to upgrade it cannot use >> the ORDER BY value_order clause in the SELECT >> statement that retrieves values). >> > > True. We should be able to assume that the nonpersistent repository is > in the new format, though, shouldn't we? Unless I suppose we want to > support changing the svc.configd binary without rebooting. > > Hmm, perhaps the latter scenario may be required for "pkg upgrade" (though I believe by default it upgrades the inactive BE). > Did you happen to determine when ADD COLUMN was added? If was before > 3.0, upgrading might be an option. > > Looks like it was version 3.2.0 unfortunately (see http://sqlite.org/changes.html). Thanks!
Alan _______________________________________________ smf-discuss mailing list smf-discuss at opensolaris dot org
|
|
|
|
Posts:
941
From:
Redwood City, CA
Registered:
3/9/05
|
|
|
|
Re: some thoughts on SMF backend support for handling
ordering of property values
Posted:
Jun 11, 2008 11:43 AM
in response to: amaguire
|
|
Quoth Alan Maguire on Wed, Jun 11, 2008 at 10:55:53AM +0100: > Nice summary of the options. Defaulting to ordered > storage/retrieval would be my preference here, with > possible future changes based on explicit requirements > for unordered value support. I'm not quite sure what > constraints the network repository requirements would > impose - what do you think Stephen?
Let's just resolve to add an unordered-property creation interface in the future.
> > Is supporting downgrade a hard requirement? > > If we explicitly bumped the version number - and > thus needed to do an explicit downgrade - we could > add logic to bfu etc to remove the value_order column > (this could be done using /lib/svc/bin/sqlite). In testing, > it does seem like earlier configd versions can handle a repository > with the additional value_order column, but it seems > a bit hacky not to register the fact that the repository is > upgraded just to make downgrade easier (adding a minor > version to the schema_version table might be enough though).
A minor version seems hygenic, but I don't see a lot of tangible benefit. I think I agree that allowing downgrade is easy enough to just do.
> > True. We should be able to assume that the nonpersistent repository is > > in the new format, though, shouldn't we? Unless I suppose we want to > > support changing the svc.configd binary without rebooting. > > Hmm, perhaps the latter scenario may be required for "pkg upgrade" > (though I believe by default it upgrades the inactive BE).
Let's just assume that svc.configd is sufficiently critical that it won't be changed without rebooting. If that changes in the future, it should be trivial to fix.
David _______________________________________________ smf-discuss mailing list smf-discuss at opensolaris dot org
|
|
|
|
Posts:
303
From:
Dublin, Ireland
Registered:
10/17/05
|
|
|
|
Re: some thoughts on SMF backend support for handling
ordering of property values
Posted:
Jun 12, 2008 1:25 AM
in response to: bustos
|
|
David Bustos wrote: > Quoth Alan Maguire on Wed, Jun 11, 2008 at 10:55:53AM +0100: > >> Nice summary of the options. Defaulting to ordered >> storage/retrieval would be my preference here, with >> possible future changes based on explicit requirements >> for unordered value support. I'm not quite sure what >> constraints the network repository requirements would >> impose - what do you think Stephen? >> > > Let's just resolve to add an unordered-property creation interface in > the future. > > sounds good.
>>> Is supporting downgrade a hard requirement? >>> >> If we explicitly bumped the version number - and >> thus needed to do an explicit downgrade - we could >> add logic to bfu etc to remove the value_order column >> (this could be done using /lib/svc/bin/sqlite). In testing, >> it does seem like earlier configd versions can handle a repository >> with the additional value_order column, but it seems >> a bit hacky not to register the fact that the repository is >> upgraded just to make downgrade easier (adding a minor >> version to the schema_version table might be enough though). >> > > A minor version seems hygenic, but I don't see a lot of tangible > benefit. I suppose the only benefit would be that we'd have a slightly cleaner test (check minor version number - if any - versus check if value_order column exists) to determine if upgrade is needed. Probably not worth it really. > I think I agree that allowing downgrade is easy enough to just > do. > > By allowing downgrade, do you mean not bumping the version number and letting older svc.configd's use the upgraded repository? Or that we do bump the version number when we upgrade, and then have to support downgrade for backwards BFU/package removal?
I was thinking about this a bit, and my assumption had been that to support downgrade in the case where we bump the version number, we'd need to modify bfu to catch backwards bfu cases and do the right thing. That's doable I think, but I'm less sure about the package-based downgrade scenario. My initial thought had been that we'd simply downgrade the repository in SUNWcsr's preremove script, but thinking about it that then breaks upgrade to a Nevada build that also supports value ordering, as the downgrade action would remove the value_order column which maintains all our ordering information. Looking forward, I'm also unsure how to handle a change like this in the IPS (no scripting) environment.
For these reasons, I'm leaning towards the approach of avoiding bumping the schema version (and thus letting older svc.configds use the repository with the extra column. Since the value_order column makes values default to 0, this works fine). Perhaps there are there ways round these problems though - I haven't had to deal with these sorts of upgrade/downgrade issues much, so I'm not sure. >>> True. We should be able to assume that the nonpersistent repository is >>> in the new format, though, shouldn't we? Unless I suppose we want to >>> support changing the svc.configd binary without rebooting. >>> >> Hmm, perhaps the latter scenario may be required for "pkg upgrade" >> (though I believe by default it upgrades the inactive BE). >> > > Let's just assume that svc.configd is sufficiently critical that it > won't be changed without rebooting. If that changes in the future, it > should be trivial to fix. > > Sounds good - I'll modify to only upgrade the persistent repository.
Nicolas Williams wrote: > On Wed, Jun 11, 2008 at 04:37:20PM -0500, Nicolas Williams wrote: > >> Comments: >> >> - Why not update BACKEND_SCHEMA_VERSION? >> >> One possible answer: because minor schema changes don't warrant the >> complexity of updating schema_version. It's easy enough to detect >> the actual schema version by testing whether some statement compiles >> or by looking at sqlite_master (idmapd does the latter, your changes >> do the former). >> > > And I see the real answer now too: to support downgrades. idmapd > doesn't, but that's OK there, whereas for SMF not supporting downgrades > would be... bad (no more backwards BFUs!). > > So yeah, just add a comment. > Thanks for taking a look at the code Nico! I'll certainly add some more comments to make things clearer, particularly upgrade/downgrade handling. I'll respin the webrev once we've converged on the right upgrade/downgrade approach.
Alan _______________________________________________ smf-discuss mailing list smf-discuss at opensolaris dot org
|
|
|
|
Posts:
941
From:
Redwood City, CA
Registered:
3/9/05
|
|
|
|
Re: some thoughts on SMF backend support for handling
ordering of property values
Posted:
Jun 12, 2008 8:36 AM
in response to: amaguire
|
|
Quoth Alan Maguire on Thu, Jun 12, 2008 at 09:25:46AM +0100: > > A minor version seems hygenic, but I don't see a lot of tangible > > benefit. > > I suppose the only benefit would be that we'd have a > slightly cleaner test (check minor version number - > if any - versus check if value_order column exists) to > determine if upgrade is needed. Probably not worth > it really.
Right.
> For these reasons, I'm leaning towards the > approach of avoiding bumping the schema version > (and thus letting older svc.configds use the repository > with the extra column. Since the value_order column > makes values default to 0, this works fine). Perhaps > there are there ways round these problems though - > I haven't had to deal with these sorts of upgrade/downgrade > issues much, so I'm not sure.
Yes, let's do that. The version number can represent incompatible change.
David _______________________________________________ smf-discuss mailing list smf-discuss at opensolaris dot org
|
|
|
|
Posts:
3,422
From:
Austin, TX, USA
Registered:
6/15/05
|
|
|
|
Re: some thoughts on SMF backend support for handling
ordering of property values
Posted:
Jun 11, 2008 12:10 PM
in response to: amaguire
|
|
On Wed, Jun 11, 2008 at 10:55:53AM +0100, Alan Maguire wrote: > David Bustos wrote: > > So what is your take on Stephen's network repository issue? I wonder if > > we could add a piece of metadata to properties which specify whether > > they're ordered or unordered. By default they'd be ordered, but if an > > application doesn't care then it can make them unordered, which would > > free svc.configd to avoid emulating order in an unordered backend. Or > > maybe the ordering should be a flag on the retrieval interface. Or > > maybe the current retrieval interfaces should be designated "ordered", > > and we can add separate unordered retrieval interfaces. If we wanted to > > do this, we could probably put it off actual unordered properties until > > we have a network backend, and make any interface changes as > > opportunities arise. > > > > > Nice summary of the options. Defaulting to ordered > storage/retrieval would be my preference here, with > possible future changes based on explicit requirements > for unordered value support. I'm not quite sure what > constraints the network repository requirements would > impose - what do you think Stephen?
What would happen on upgrade? Which order would be assumed for existing multi-valued props? Or would those be flagged as unordered? I recommend the latter.
Also, svccfg/svcprop/libscf would have to change too so that the set/sequence nature of a property could specified/observed.
Nico -- _______________________________________________ smf-discuss mailing list smf-discuss at opensolaris dot org
|
|
|
|
Posts:
303
From:
Dublin, Ireland
Registered:
10/17/05
|
|
|
|
Re: some thoughts on SMF backend support for handling
ordering of property values
Posted:
Jun 11, 2008 12:38 PM
in response to: nico
|
|
Nicolas Williams wrote: > On Wed, Jun 11, 2008 at 10:55:53AM +0100, Alan Maguire wrote: > >> David Bustos wrote: >> >>> So what is your take on Stephen's network repository issue? I wonder if >>> we could add a piece of metadata to properties which specify whether >>> they're ordered or unordered. By default they'd be ordered, but if an >>> application doesn't care then it can make them unordered, which would >>> free svc.configd to avoid emulating order in an unordered backend. Or >>> maybe the ordering should be a flag on the retrieval interface. Or >>> maybe the current retrieval interfaces should be designated "ordered", >>> and we can add separate unordered retrieval interfaces. If we wanted to >>> do this, we could probably put it off actual unordered properties until >>> we have a network backend, and make any interface changes as >>> opportunities arise. >>> >>> >>> >> Nice summary of the options. Defaulting to ordered >> storage/retrieval would be my preference here, with >> possible future changes based on explicit requirements >> for unordered value support. I'm not quite sure what >> constraints the network repository requirements would >> impose - what do you think Stephen? >> > > What would happen on upgrade? Which order would be assumed for existing > multi-valued props? Or would those be flagged as unordered? I > recommend the latter. > > The thing is, we cannot determine anything about user intent regarding ordering of properties that exist prior to upgrade to the order-preserving repository format (well we can I suppose - consumers should not expect any particular ordering). In practice, what will happen is that existing property values will share a value_order of 0, so when these are retrieved, the value_order will not order things in any definitive way. However, if/when such properties are later set, the order of setting will respect the order of properties added via scf_entry_add_value(). The benefit of this change is more for services/properties added after it is made - these will be able to take advantage of definitive ordering of values.
> Also, svccfg/svcprop/libscf would have to change too so that the > set/sequence nature of a property could specified/observed. > > No need as far as I can see - testing indicates these all just work, since scf_entry_add_value() and the backend do the right thing. I think what we´d need to do is ensure that manpages document the ordering behaviour.
Alan _______________________________________________ smf-discuss mailing list smf-discuss at opensolaris dot org
|
|
|
|
Posts:
248
From:
US
Registered:
3/9/05
|
|
|
|
Re: some thoughts on SMF backend support for handling
ordering of property values
Posted:
Jun 11, 2008 12:40 PM
in response to: nico
|
|
Nicolas Williams wrote: > What would happen on upgrade? Which order would be assumed for existing > multi-valued props? Or would those be flagged as unordered? I > recommend the latter.
I'm having a hard time seeing a requirement for unordered lists. If you're expecting an unordered list and you happen to get an ordered one, how could you care?
As a related matter, I don't think it matters what order existing multi-valued properties get, since nothing could previously have depended on it. Whatever that order is should probably be stable - an existing previously unordered list should become an ordered list with some unspecified order - but it doesn't matter what the order is.
> Also, svccfg/svcprop/libscf would have to change too so that the > set/sequence nature of a property could specified/observed.
Do they? I could perhaps see a desire to add ways to insert, delete, and replace values by index, but I don't immediately see why existing interfaces would need to change.
_______________________________________________ smf-discuss mailing list smf-discuss at opensolaris dot org
|
|
|
|
Posts:
3,422
From:
Austin, TX, USA
Registered:
6/15/05
|
|
|
|
Re: some thoughts on SMF backend support for handling
ordering of property values
Posted:
Jun 11, 2008 2:16 PM
in response to: jordan
|
|
On Wed, Jun 11, 2008 at 12:40:05PM -0700, Jordan Brown wrote: > Nicolas Williams wrote: > >What would happen on upgrade? Which order would be assumed for existing > >multi-valued props? Or would those be flagged as unordered? I > >recommend the latter. > > I'm having a hard time seeing a requirement for unordered lists. If > you're expecting an unordered list and you happen to get an ordered one, > how could you care?
I think there are places where order is certainly irrelevant. Having a distinction might help set user expectations, but this is of rather low value.
In any case, I thought the proposal was for adding a user/app-visible notion of whether any given protperty's value set is ordered. Whereas it seems instead to say that property value sets will always be ordered (except for pre-existing ones).
> As a related matter, I don't think it matters what order existing > multi-valued properties get, since nothing could previously have > depended on it. Whatever that order is should probably be stable - an > existing previously unordered list should become an ordered list with > some unspecified order - but it doesn't matter what the order is.
I'm not so sure. Though today property value sets are documented as unordered, the order produced is still deterministic, IIRC, with the current database used under the hood.
So while nothing ought to depend on order, there may be things that do.
> >Also, svccfg/svcprop/libscf would have to change too so that the > >set/sequence nature of a property could specified/observed. > > Do they? I could perhaps see a desire to add ways to insert, delete, > and replace values by index, but I don't immediately see why existing > interfaces would need to change.
See my other reply just now. _______________________________________________ smf-discuss mailing list smf-discuss at opensolaris dot org
|
|
|
|
|