|
Replies:
81
-
Last Post:
Apr 30, 2007 4:39 PM
by: rheilke
|
|
|
Posts:
80
From:
Registered:
4/27/05
|
|
|
|
How much do we really want zpool remove?
Posted:
Jan 18, 2007 2:55 AM
|
|
On the issue of the ability to remove a device from a zpool, how
useful/pressing is this feature? Or is this more along the line of
"nice to have"?
--
Regards,
Jeremy
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 3:31 AM
in response to: nerant
|
|
On Thu, Jan 18, 2007 at 06:55:39PM +0800, Jeremy Teo wrote:
> On the issue of the ability to remove a device from a zpool, how
> useful/pressing is this feature? Or is this more along the line of
> "nice to have"?
If you think "remove a device from a zpool" = "to shrink a pool" then
it is really usefull. Definitely really usefull.
Do you need any example ?
przemol
----------------------------------------------------------------------
Lufa dla generala. Zobacz >> http://link.interia.pl/f19e1
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
262
From:
Cardiff, Wales, United Kingdom
Registered:
5/2/06
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 3:34 AM
in response to: nerant
|
|
On 18/01/07, Jeremy Teo <white dot wristband at gmail dot com> wrote:
> On the issue of the ability to remove a device from a zpool, how
> useful/pressing is this feature? Or is this more along the line of
> "nice to have"?
It's very useful if you accidentally create a concat rather than mirror
of an existing zpool. Otherwise you have to buy another drive :)
--
Rasputin :: Jack of All Trades - Master of Nuns
http://number9.hellooperator.net/
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
201
From:
AU
Registered:
6/14/05
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 5:24 AM
in response to: nerant
|
|
On 18/01/2007, at 9:55 PM, Jeremy Teo wrote:
> On the issue of the ability to remove a device from a zpool, how
> useful/pressing is this feature? Or is this more along the line of
> "nice to have"?
Assuming we're talking about removing a top-level vdev..
I introduce new sysadmins to ZFS on a weekly basis. After 2 hours of
introduction this is the single feature that they most often realise
is "missing".
The most common reason is migration of data to new storage
infrastructure. The experience is often that the growth in disk size
allows the new storage to consist of fewer disks/LUNs than the old.
I can see that is will come increasingly needed as more and more
storage goes under ZFS. Sure, we can put 256 quadrillion zettabytes
in the pool, but if you accidentally add a disk to the wrong pool or
with the wrong redundancy you have a long long wait for your tape
drive :)
Boyd
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
59
From:
Registered:
4/10/06
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 5:32 AM
in response to: boyd
To: Communities » zfs » discuss
|
|
I can vouch for this situation. I had to go through a long maintenance to accomplish the following:
- 50 x 64GB drives in a zpool; needed to seperate out 15 of them out due to performance issues. There was no need to increase storage capacity.
Because I couldn't yank 15 drives from the existing pool to create a UFS filesystem I had to go evacuate the entire 50 disk pool, recreate a new pool and the UFS filesystem, and then repopulate the filesystems.
I think this feature will add to the adoption rate of ZFS. However, I feel that this shouldn't be at the top of the 'to-do' list. I'll trade this feature for some of the performance enhancements that've been discussed on this group.
|
|
|
|
|
Posts:
413
From:
US
Registered:
3/9/05
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 10:51 AM
in response to: nerant
|
|
Jeremy Teo wrote:
> On the issue of the ability to remove a device from a zpool, how
> useful/pressing is this feature? Or is this more along the line of
> "nice to have"?
This is a pretty high priority. We are working on it.
--matt
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
196
From:
Santa Clara, CA
Registered:
6/23/06
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 11:25 AM
in response to: ahrens
|
|
On Thu, 2007-01-18 at 10:51 -0800, Matthew Ahrens wrote:
> Jeremy Teo wrote:
> > On the issue of the ability to remove a device from a zpool, how
> > useful/pressing is this feature? Or is this more along the line of
> > "nice to have"?
>
> This is a pretty high priority. We are working on it.
>
> --matt
I'd consider it a lower priority than say, adding a drive to a RAIDZ
vdev, but yes, being able to reduce a zpool's size by removing devices
is quite useful, as it adds a considerable degree of flexibility that
(we) admins crave.
--
Erik Trimble
Java System Support
Mailstop: usca14-102
Phone: x17195
Santa Clara, CA
Timezone: US/Pacific (GMT-0800)
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
125
From:
MPLS
Registered:
1/5/07
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 11:29 AM
in response to: et151817
|
|
zfs-discuss-bounces at opensolaris dot org wrote on 01/18/2007 01:29:23 PM:
> On Thu, 2007-01-18 at 10:51 -0800, Matthew Ahrens wrote:
> > Jeremy Teo wrote:
> > > On the issue of the ability to remove a device from a zpool, how
> > > useful/pressing is this feature? Or is this more along the line of
> > > "nice to have"?
> >
> > This is a pretty high priority. We are working on it.
> >
> > --matt
>
> I'd consider it a lower priority than say, adding a drive to a RAIDZ
> vdev, but yes, being able to reduce a zpool's size by removing devices
> is quite useful, as it adds a considerable degree of flexibility that
> (we) admins crave.
>
I would be surprised if much of the code to allow removal does not bring
device adds closer to reality -- assuming device removal migrates data and
resilvers to optimal stripe online..
-Wade
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 12:31 PM
in response to: wstuart
|
|
Would this be the same as failing a drive on purpose to remove it?
I was under the impression that was supported, but I wasn't sure if
shrinking a ZFS pool would work though.
On 1/18/07, Wade dot Stuart at fallon dot com <Wade dot Stuart at fallon dot com> wrote:
> > > This is a pretty high priority. We are working on it.
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
202
From:
Singapore
Registered:
6/17/05
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 5:33 PM
in response to: mike
|
|
On 1/19/07, mike <mike503 at gmail dot com> wrote:
> Would this be the same as failing a drive on purpose to remove it?
>
> I was under the impression that was supported, but I wasn't sure if
> shrinking a ZFS pool would work though.
Not quite. I suspect you are thinking about drive replacement rather
than removal.
Drive replacement is already supported in ZFS and the task involves
rebuilding data on the disk from data available elsewhere. Drive
removal involves rebalancing data from the target drive to other
disks. The latter is non-trivial.
--
Just me,
Wire ...
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 6:01 PM
in response to: weeyeh
|
|
what is the technical difference between forcing a removal and an
actual failure?
isn't it the same process? except one is manually triggered? i would
assume the same resilvering process happens when a usable drive is put
back in...
On 1/18/07, Wee Yeh Tan <weeyeh at gmail dot com> wrote:
> Not quite. I suspect you are thinking about drive replacement rather
> than removal.
>
> Drive replacement is already supported in ZFS and the task involves
> rebuilding data on the disk from data available elsewhere. Drive
> removal involves rebalancing data from the target drive to other
> disks. The latter is non-trivial.
>
>
> --
> Just me,
> Wire ...
>
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
196
From:
Santa Clara, CA
Registered:
6/23/06
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 6:25 PM
in response to: mike
|
|
Mike,
I think you are missing the point. What we are talking about is
removing a drive from a zpool, that is, reducing the zpool's total
capacity by a drive. Say you have 4 drives of 100GB in size,
configured in a striped mirror, capacity of 200GB usable. We're
discussing the case where if the zpool's total used space is under
100GB, we could remove the second vdev (consisting of a mirror) from the
zpool, and have ZFS evacuate all the data from the to-be-removed vdev
before we actually remove the disks (or, maybe we simply want to
reconfigure them into another zpool). In this case, after the drive
remoovals, the zpool would be left with a 100GB capacity, and be a
simple 2-drive mirror.
What you are talking about is replacement of a drive, whether or not it
is actually bad. In your instance, the zpool capacity size remains the
same, and it will return to optimal performance when a new drive is
inserted (and, no, there is no difference between a manual and automatic
"removal" in the case of marking a drive bad for replacement).
-Erik
On Thu, 2007-01-18 at 18:01 -0800, mike wrote:
> what is the technical difference between forcing a removal and an
> actual failure?
>
> isn't it the same process? except one is manually triggered? i would
> assume the same resilvering process happens when a usable drive is put
> back in...
>
> On 1/18/07, Wee Yeh Tan <weeyeh at gmail dot com> wrote:
> > Not quite. I suspect you are thinking about drive replacement rather
> > than removal.
> >
> > Drive replacement is already supported in ZFS and the task involves
> > rebuilding data on the disk from data available elsewhere. Drive
> > removal involves rebalancing data from the target drive to other
> > disks. The latter is non-trivial.
> >
> >
> > --
> > Just me,
> > Wire ...
> >
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris dot org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
--
Erik Trimble
Java System Support
Mailstop: usca14-102
Phone: x17195
Santa Clara, CA
Timezone: US/Pacific (GMT-0800)
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 7:07 PM
in response to: et151817
|
|
I get that part. I think I asked that question before (although not as
direct) - basically you're talking about the ability to shrink volumes
and/or disable/change the mirroring/redundancy options if there is
space available to account for it.
If this was allowed, this would also allow for a conversion from RAIDZ
to RAIDZ2, or vice-versa then, correct?
On 1/18/07, Erik Trimble <Erik dot Trimble at sun dot com> wrote:
> Mike,
>
> I think you are missing the point. What we are talking about is
> removing a drive from a zpool, that is, reducing the zpool's total
> capacity by a drive. Say you have 4 drives of 100GB in size,
> configured in a striped mirror, capacity of 200GB usable. We're
> discussing the case where if the zpool's total used space is under
> 100GB, we could remove the second vdev (consisting of a mirror) from the
> zpool, and have ZFS evacuate all the data from the to-be-removed vdev
> before we actually remove the disks (or, maybe we simply want to
> reconfigure them into another zpool). In this case, after the drive
> remoovals, the zpool would be left with a 100GB capacity, and be a
> simple 2-drive mirror.
>
>
> What you are talking about is replacement of a drive, whether or not it
> is actually bad. In your instance, the zpool capacity size remains the
> same, and it will return to optimal performance when a new drive is
> inserted (and, no, there is no difference between a manual and automatic
> "removal" in the case of marking a drive bad for replacement).
>
> -Erik
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Robert Milkowski
rmilkowski@task.gda.pl
|
|
|
|
Re[2]: How much do we really want zpool remove?
Posted:
Jan 19, 2007 12:28 AM
in response to: mike
|
|
Hello mike,
Friday, January 19, 2007, 4:07:31 AM, you wrote:
m> I get that part. I think I asked that question before (although not as
m> direct) - basically you're talking about the ability to shrink volumes
m> and/or disable/change the mirroring/redundancy options if there is
m> space available to account for it.
You can already convert RAID-10 to RAID-0 and vice versa with ZFS.
Just attach/detach disks.
m> If this was allowed, this would also allow for a conversion from RAIDZ
m> to RAIDZ2, or vice-versa then, correct?
Not really - at least not directly.
--
Best regards,
Robert mailto:rmilkowski at task dot gda dot pl
http://milek.blogspot.com
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
9
From:
Registered:
9/12/06
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 12:33 PM
in response to: wstuart
To: Communities » zfs » discuss
|
|
Both removing disks from a zpool and modifying raidz arrays would be very useful.
I would also still love to have ditto data blocks. Is there any progress on this?
Celso.
|
|
|
|
|
Posts:
12
From:
Registered:
12/5/05
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 1:22 PM
in response to: celso
|
|
Celso wrote:
> Both removing disks from a zpool and modifying raidz arrays would be very useful.
Add my vote for this.
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
23
From:
US
Registered:
11/5/06
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 18, 2007 7:06 PM
in response to: ahrens
To: Communities » zfs » discuss
|
|
> Jeremy Teo wrote:
> > On the issue of the ability to remove a device from
> a zpool, how
> > useful/pressing is this feature? Or is this more
> along the line of
> > "nice to have"?
>
> This is a pretty high priority. We are working on
> it.
Good news! Where is the discussion on the best approach to take?
> On 18/01/2007, at 9:55 PM, Jeremy Teo wrote:
> The most common reason is migration of data to new
> storage
> infrastructure. The experience is often that the
> growth in disk size
> allows the new storage to consist of fewer disks/LUNs
> than the old.
I agree completely. No matter how wonderful your current FC/SAS/whatever cabinet is, at some point in the future you will want to migrate to another newer/faster array with a better/faster interface, probably on fewer disks. The "just add another top level vdev" approach to growing a RAIDZ pool seems a bit myopic.
> On Thu, 2007-01-18 at 10:51 -0800, Matthew Ahrens
> wrote:
> I'd consider it a lower priority than say, adding a
> drive to a RAIDZ
> vdev, but yes, being able to reduce a zpool's size by
> removing devices
> is quite useful, as it adds a considerable degree of
> flexibility that
> (we) admins crave.
These two items (removing a vdev and restriping an array) are probably closely related. At the core of either operation likely will center around some metaslab_evacuate() routine which empties a metaslab and puts the data onto another metaslab.
Evacuating a vdev could be no more than evacuating all of the metaslabs in the vdev.
Restriping (adding/removing a data/parity disk) could be no more than progressively evacuating metaslabs with the old stripe geometry and writing the data to metaslabs with the new stripe geometry. The biggest challenge while restriping might be getting the read routine to figure out on-the-fly which geometry is in use for any particular stripe. Even so, this shouldn't be too big of a challenge: one geometry will checksum correctly and the other will not.
Marty
|
|
|
|
|
Darren Dunham
ddunham@taos.com
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 19, 2007 1:57 PM
in response to: martysch
|
|
> These two items (removing a vdev and restriping an array) are probably
> closely related. At the core of either operation likely will center
> around some metaslab_evacuate() routine which empties a metaslab and
> puts the data onto another metaslab.
> Evacuating a vdev could be no more than evacuating all of the
> metaslabs in the vdev.
How would snapshots behave here? Would they prevent evacuation, or
would they be able to refer to a migrated block?
--
Darren Dunham ddunham at taos dot com
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Robert Milkowski
rmilkowski@task.gda.pl
|
|
|
|
Re[2]: How much do we really want zpool remove?
Posted:
Jan 19, 2007 12:31 AM
in response to: ahrens
|
|
Hello Matthew,
Thursday, January 18, 2007, 7:51:18 PM, you wrote:
MA> Jeremy Teo wrote:
>> On the issue of the ability to remove a device from a zpool, how
>> useful/pressing is this feature? Or is this more along the line of
>> "nice to have"?
MA> This is a pretty high priority. We are working on it.
Quick, precise and "informative".
Ok, can you give us any details? Like only removal or also adding a
disk and re-writing all data to much new stripe width? Also conversion
Z1<->Z2? Other also (10->Z1, ...)? Any estimate when and what would
hit ON?
--
Best regards,
Robert mailto:rmilkowski at task dot gda dot pl
http://milek.blogspot.com
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
383
From:
GB
Registered:
10/27/05
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 19, 2007 2:42 AM
in response to: nerant
|
|
On Thu, Jan 18, 2007 at 06:55:39PM +0800, Jeremy Teo wrote:
> On the issue of the ability to remove a device from a zpool, how
> useful/pressing is this feature? Or is this more along the line of
> "nice to have"?
We definitely need it. As a usage case, on occasion we have had to move
SAN sites, and the easiest way to that by far is to snap on the new site
and remove the old one once it's synced.
Ceri
--
That must be wonderful! I don't understand it at all.
-- Moliere
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (FreeBSD)
iD8DBQFFsKB8ocfcwTS3JF8RAq0xAJ9Hxbmy1N+BRP95hSqZ6moMmzMeFgCgpt/G
Io9Bb8dpPZjeEGwYtELMkIE=
=GwFD
-----END PGP SIGNATURE-----
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
651
From:
CA
Registered:
6/14/05
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 19, 2007 2:27 PM
in response to: nerant
To: Communities » zfs » discuss
|
|
If you are referring to shrinking a pool/file system, where I work this is considered very high on the list. It isn't a truly dynamic file system if we can't shrink it.
As a practical example, you have a test server with several projects being worked on. When a project finishes 9for whatever reason), you delete the files, shrink the pool, and give the LUN back to the storage folks to assign to another server that may be running out of space.
Having a SAN, we see it as more important than, say, the RAIDZ work. But, I can see why other people with different needs will argue otherwise. There are other things I would like to see as well (ability to find what files got corrupted due to a HW failure, and so on--see my other threads in this forum), but from the enterprise perspective of the company I work for, this is right up there. Just throwing our $.02 in. :-)
Rainer
|
|
|
|
|
Posts:
95
From:
Registered:
2/27/06
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 20, 2007 1:49 AM
in response to: rheilke
To: Communities » zfs » discuss
|
|
we migrate in our solaris8+vxvm+SAN environment 500tb to new storage arrays.
we have seen a lot of migration ways, falconstore etc., but the only acceptable way is the host based mirror with vxvm. so we can migrate manuelly in a few weeks but without downtime.
tell me how we can do this with zpools without the ability to remove luns from the pool. This is our view and when you only have thumbers this is not important.
|
|
|
|
|
Posts:
651
From:
CA
Registered:
6/14/05
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 22, 2007 7:49 AM
in response to: mario2
To: Communities » zfs » discuss
|
|
> but the only acceptable way is the host based
> mirror with vxvm. so we can migrate manuelly in a few
> weeks but without downtime.
Detaching mirrors is actually easy with ZFS. I've done in several times. Look at:
zpool detach pool device
The problem here is that the detached side loses all information about the ZPool, the ZFS filoe system(s), etc. This really bit me in the butt when I was trying to figure out which of two HDD really failed (the failed disk worked "sort of").
The part that isn't there yet is shrinking, say, a 50% used 500GB pool down to a 300GB pool.
Rainer
|
|
|
|
|
Posts:
95
From:
Registered:
2/27/06
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 22, 2007 12:28 PM
in response to: rheilke
To: Communities » zfs » discuss
|
|
this is a good point, the mirror loses all information about the zpool.
this is very important for the ZFS Root pool, i don't know how often i have broken the svm-mirror of the root disks, to clone a system and bring the disk to a other system or use "live upgrade" and so on.
|
|
|
|
|
Posts:
3,793
From:
GB
Registered:
3/9/05
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 23, 2007 3:32 AM
in response to: mario2
|
|
mario heimel wrote:
> this is a good point, the mirror loses all information about the zpool.
> this is very important for the ZFS Root pool, i don't know how often i have broken the svm-mirror of the root disks, to clone a system and bring the disk to a other system or use "live upgrade" and so on.
For the live upgrade case you don't need to break the mirror for cloning
with ZFS root. Live upgrade will (probably) just run zfs clone and you
would boot from that (regardless of the presence or not of a mirror).
For the "clone another system" zfs send/recv might be useful
--
Darren J Moffat
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
1,264
From:
US
Registered:
8/5/05
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 23, 2007 3:37 AM
in response to: darrenm
|
|
On 1/23/07, Darren J Moffat <Darren dot Moffat at sun dot com> wrote:
> For the "clone another system" zfs send/recv might be useful
Having support for this directly in flarcreate would be nice. It
would make differential flars very quick and efficient.
Mike
--
Mike Gerdts
http://mgerdts.blogspot.com/
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
651
From:
CA
Registered:
6/14/05
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 23, 2007 9:42 AM
in response to: darrenm
To: Communities » zfs » discuss
|
|
> For the "clone another system" zfs send/recv might be
> useful
Keeping in mind that you only want to send/recv one half of the ZFS mirror...
Rainer
|
|
|
|
|
Posts:
3,793
From:
GB
Registered:
3/9/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 24, 2007 1:59 AM
in response to: rheilke
|
|
Rainer Heilke wrote:
>> For the "clone another system" zfs send/recv might be
>> useful
>
> Keeping in mind that you only want to send/recv one half of the ZFS mirror...
Huh ?
That doesn't make any sense. You can't send half a mirror. When you
are running zfs send it is a "read" and ZFS will read the data from all
available mirrors to help performance. When it is zfs recv it will
write to all sides of the mirror on the destination.
What are you actually trying to say here ?
--
Darren J Moffat
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
651
From:
CA
Registered:
6/14/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 24, 2007 9:06 AM
in response to: darrenm
To: Communities » zfs » discuss
|
|
The issue is that we want to split off one of the mirrors of a ZFS mirror set in a way that the "split off" mirror still contains all of the zpool information, allowing it to be used (send/recv, or import, or whatever) on another system, or renamed on the same system and used as a clean-room copy of the (now N-1) mirror without affecting the original mirror zpool.
Start with mirror N.
Split off mirror to get N-1 and 1.
Continue using N-1 as the regular mirror, and allow use of 1 elsewhere, data intact.
I'm sorry, I thought this was clear in the context of the complete discussion.
Rainer
|
|
|
|
|
Posts:
12
From:
Germany
Registered:
8/12/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Apr 26, 2007 8:36 AM
in response to: rheilke
To: Communities » zfs » discuss
|
|
You can - easily:
# zpool export [i]mypool[/i]
Then you take out one of the disks and put it into another system or a safe place.
Afterwards you simply import the pool again:
# zpool import [i]mypool[/i]
Note - you can NOT import both disks separately, as they are both taged to belong to the same zpool.
I just tried this, using files as pool-devices. But I didn't test it with real disks/slices - but it shouldn't make any difference.
HTH,
Nenad.
PS: I know, the reply is pretty late... I just read this thread.
|
|
|
|
|
Posts:
404
From:
US
Registered:
11/2/05
|
|
|
|
Re: Re: Re: Re: How much do we really want zpool remove?
Posted:
Apr 26, 2007 11:00 AM
in response to: nino
|
|
Nenad,
I've seen this solution offered before, but I would not recommend this
except as a last resort, unless you didn't care about the health of
the original pool.
Removing a device from an exported pool, could be very bad, depending
on the pool's redundancy. You might not get your all data back unless
you put the disk back.
See the output below.
Definitely not for a pool and data on a production system.
Cindy
# zpool status epool
pool: epool
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
epool ONLINE 0 0 0
mirror ONLINE 0 0 0
c7t6d0 ONLINE 0 0 0
c7t5d0 ONLINE 0 0 0
c5t5d0 ONLINE 0 0 0
c6t6d0 ONLINE 0 0 0
c6t5d0 ONLINE 0 0 0
c6t7d0 ONLINE 0 0 0
errors: No known data errors
# cfgadm | grep c6t7d0
sata4/7::dsk/c6t7d0 disk connected configured ok
# zpool export epool
# cfgadm -c unconfigure sata4/7
Unconfigure the device at: /devices/pci@2,0/pci1022,7458@7/pci11ab,11ab@1:7
This operation will suspend activity on the SATA device
Continue (yes/no)? y
# zpool import epool
# zpool status epool
pool: epool
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas
exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
see: http://www.sun.com/msg/ZFS-8000-D3
scrub: resilver completed with 0 errors on Thu Apr 26 11:38:21 2007
config:
NAME STATE READ WRITE CKSUM
epool DEGRADED 0 0 0
mirror DEGRADED 0 0 0
c7t6d0 ONLINE 0 0 0
c7t5d0 ONLINE 0 0 0
c5t5d0 ONLINE 0 0 0
c6t6d0 ONLINE 0 0 0
c6t5d0 ONLINE 0 0 0
c6t7d0 UNAVAIL 0 0 0 cannot open
errors: No known data errors
#
Nenad Cimerman wrote:
> You can - easily:
> # zpool export [i]mypool[/i]
> Then you take out one of the disks and put it into another system or a safe place.
> Afterwards you simply import the pool again:
> # zpool import [i]mypool[/i]
>
> Note - you can NOT import both disks separately, as they are both taged to belong to the same zpool.
>
> I just tried this, using files as pool-devices. But I didn't test it with real disks/slices - but it shouldn't make any difference.
>
> HTH,
> Nenad.
>
> PS: I know, the reply is pretty late... I just read this thread.
>
>
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris dot org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Robert Milkowski
rmilkowski@task.gda.pl
|
|
|
|
Re[2]: Re: Re: Re: How much do we really want zpool
remove?
Posted:
Apr 26, 2007 3:19 PM
in response to: cindys
|
|
Hello Cindy,
Thursday, April 26, 2007, 8:57:54 PM, you wrote:
CSSC> Nenad,
CSSC> I've seen this solution offered before, but I would not recommend this
CSSC> except as a last resort, unless you didn't care about the health of
CSSC> the original pool.
CSSC> Removing a device from an exported pool, could be very bad, depending
CSSC> on the pool's redundancy. You might not get your all data back unless
CSSC> you put the disk back.
CSSC> See the output below.
CSSC> Definitely not for a pool and data on a production system.
CSSC> Cindy
CSSC> # zpool status epool
CSSC> pool: epool
CSSC> state: ONLINE
CSSC> scrub: none requested
CSSC> config:
CSSC> NAME STATE READ WRITE CKSUM
CSSC> epool ONLINE 0 0 0
CSSC> mirror ONLINE 0 0 0
CSSC> c7t6d0 ONLINE 0 0 0
CSSC> c7t5d0 ONLINE 0 0 0
CSSC> c5t5d0 ONLINE 0 0 0
CSSC> c6t6d0 ONLINE 0 0 0
CSSC> c6t5d0 ONLINE 0 0 0
CSSC> c6t7d0 ONLINE 0 0 0
CSSC> errors: No known data errors
CSSC> # cfgadm | grep c6t7d0
CSSC> sata4/7::dsk/c6t7d0 disk connected configured ok
CSSC> # zpool export epool
CSSC> # cfgadm -c unconfigure sata4/7
CSSC> Unconfigure the device at:
CSSC> /devices/pci@2,0/pci1022,7458@7/pci11ab,11ab@1:7
CSSC> This operation will suspend activity on the SATA device
CSSC> Continue (yes/no)? y
CSSC> # zpool import epool
CSSC> # zpool status epool
CSSC> pool: epool
CSSC> state: DEGRADED
CSSC> status: One or more devices could not be opened. Sufficient replicas
CSSC> exist for
CSSC> the pool to continue functioning in a degraded state.
CSSC> action: Attach the missing device and online it using 'zpool online'.
CSSC> see: http://www.sun.com/msg/ZFS-8000-D3
CSSC> scrub: resilver completed with 0 errors on Thu Apr 26 11:38:21 2007
CSSC> config:
CSSC> NAME STATE READ WRITE CKSUM
CSSC> epool DEGRADED 0 0 0
CSSC> mirror DEGRADED 0 0 0
CSSC> c7t6d0 ONLINE 0 0 0
CSSC> c7t5d0 ONLINE 0 0 0
CSSC> c5t5d0 ONLINE 0 0 0
CSSC> c6t6d0 ONLINE 0 0 0
CSSC> c6t5d0 ONLINE 0 0 0
CSSC> c6t7d0 UNAVAIL 0 0 0 cannot open
CSSC> errors: No known data errors
CSSC> #
What's wrong with above? It's perfectly normal and in such a config
it's definitely safe (there're still 5 copies of valid data).
--
Best regards,
Robert mailto:rmilkowski at task dot gda dot pl
http://milek.blogspot.com
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
404
From:
US
Registered:
11/2/05
|
|
|
|
Re: Re: Re: Re: How much do we really want zpool remove?
Posted:
Apr 26, 2007 3:30 PM
in response to: Robert Milkowski
|
|
Hi Robert,
I just want to be clear that you can't just remove a disk from an
exported pool without penalty upon import:
- If the underlying redundancy of the original pool doesn't support
it and you lose data
- Some penalty exists even for redundant pools, which is running
in DEGRADED mode until you put the disk back
Cindy
Robert Milkowski wrote:
> Hello Cindy,
>
> Thursday, April 26, 2007, 8:57:54 PM, you wrote:
>
> CSSC> Nenad,
>
> CSSC> I've seen this solution offered before, but I would not recommend this
> CSSC> except as a last resort, unless you didn't care about the health of
> CSSC> the original pool.
>
> CSSC> Removing a device from an exported pool, could be very bad, depending
> CSSC> on the pool's redundancy. You might not get your all data back unless
> CSSC> you put the disk back.
>
> CSSC> See the output below.
>
> CSSC> Definitely not for a pool and data on a production system.
>
> CSSC> Cindy
>
> CSSC> # zpool status epool
> CSSC> pool: epool
> CSSC> state: ONLINE
> CSSC> scrub: none requested
> CSSC> config:
>
> CSSC> NAME STATE READ WRITE CKSUM
> CSSC> epool ONLINE 0 0 0
> CSSC> mirror ONLINE 0 0 0
> CSSC> c7t6d0 ONLINE 0 0 0
> CSSC> c7t5d0 ONLINE 0 0 0
> CSSC> c5t5d0 ONLINE 0 0 0
> CSSC> c6t6d0 ONLINE 0 0 0
> CSSC> c6t5d0 ONLINE 0 0 0
> CSSC> c6t7d0 ONLINE 0 0 0
>
> CSSC> errors: No known data errors
> CSSC> # cfgadm | grep c6t7d0
> CSSC> sata4/7::dsk/c6t7d0 disk connected configured ok
> CSSC> # zpool export epool
> CSSC> # cfgadm -c unconfigure sata4/7
> CSSC> Unconfigure the device at:
> CSSC> /devices/pci@2,0/pci1022,7458@7/pci11ab,11ab@1:7
> CSSC> This operation will suspend activity on the SATA device
> CSSC> Continue (yes/no)? y
> CSSC> # zpool import epool
> CSSC> # zpool status epool
> CSSC> pool: epool
> CSSC> state: DEGRADED
> CSSC> status: One or more devices could not be opened. Sufficient replicas
> CSSC> exist for
> CSSC> the pool to continue functioning in a degraded state.
> CSSC> action: Attach the missing device and online it using 'zpool online'.
> CSSC> see: http://www.sun.com/msg/ZFS-8000-D3
> CSSC> scrub: resilver completed with 0 errors on Thu Apr 26 11:38:21 2007
> CSSC> config:
>
> CSSC> NAME STATE READ WRITE CKSUM
> CSSC> epool DEGRADED 0 0 0
> CSSC> mirror DEGRADED 0 0 0
> CSSC> c7t6d0 ONLINE 0 0 0
> CSSC> c7t5d0 ONLINE 0 0 0
> CSSC> c5t5d0 ONLINE 0 0 0
> CSSC> c6t6d0 ONLINE 0 0 0
> CSSC> c6t5d0 ONLINE 0 0 0
> CSSC> c6t7d0 UNAVAIL 0 0 0 cannot open
>
> CSSC> errors: No known data errors
> CSSC> #
>
> What's wrong with above? It's perfectly normal and in such a config
> it's definitely safe (there're still 5 copies of valid data).
>
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Robert Milkowski
rmilkowski@task.gda.pl
|
|
|
|
Re[2]: Re: Re: Re: How much do we really want zpool
remove?
Posted:
Apr 26, 2007 3:39 PM
in response to: cindys
|
|
Hello Cindy,
Friday, April 27, 2007, 1:28:05 AM, you wrote:
CSSC> Hi Robert,
CSSC> I just want to be clear that you can't just remove a disk from an
CSSC> exported pool without penalty upon import:
CSSC> - If the underlying redundancy of the original pool doesn't support
CSSC> it and you lose data
Yep, as with any other SW or HW RAID if you do it uncleanly.
As I understand it's being worked to allow to cleanly remove a device
from a pool (redundant or not).
CSSC> - Some penalty exists even for redundant pools, which is running
CSSC> in DEGRADED mode until you put the disk back
What penalty? It's just a warning then one device is missing. Nothing
else really happens and you should be able to just remove unavail
device and get rid of that message if you want.
--
Best regards,
Robert mailto:rmilkowski at task dot gda dot pl
http://milek.blogspot.com
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
651
From:
CA
Registered:
6/14/05
|
|
|
|
Re: Re: Re: Re: How much do we really want zpool remove?
Posted:
Apr 27, 2007 4:10 PM
in response to: cindys
To: Communities » zfs » discuss
|
|
> Nenad,
>
> I've seen this solution offered before, but I would
> not recommend this
> except as a last resort, unless you didn't care about
> the health of
> the original pool.
This is emphatically not what was being requested by me, in fact. I agree, I would be highly suspicious of the data's integrity depending upon the zpool structure.
There are a couple other things to consider. First, this requires exporting the whole pool, which automatically assumes an outage is acceptable. Second, it assumes the other half of the mirror will be used elsewhere as in "on another system". This is also not a fair automatic assumption to make. The comments I made in this thread (I can't speak for others) were that the broken-off mirror may be used safely elsewhere, with the understanding this could even be on the same system, perhaps for archival reference or some-such. "Elsewhere" does not automatically imply "on another server". My apologies if I did not make this clear.
Rainer
|
|
|
|
|
Posts:
404
From:
US
Registered:
11/2/05
|
|
|
|
Re: Re: Re: Re: Re: How much do we really want zpool
remove?
Posted:
Apr 30, 2007 8:40 AM
in response to: rheilke
|
|
Hi Rainer,
This is a long thread and I wasn't commenting on your previous
replies regarding mirror manipulation. If I was, I would have done
so directly. :-)
I saw the export-a-pool-to-remove-a-disk-solution described in
a Sun doc.
My point and (I agree with your points below) is that making a pool
unavailable to remove a disk is not a good administrative practice if
you are unclear of the impact on the overall health of the original
pool.
Currently, if what you really want is to remove a disk from a
redundant pool, then better options are zpool detach or zpool replace.
Cindy
Rainer Heilke wrote:
>>Nenad,
>>
>>I've seen this solution offered before, but I would
>>not recommend this
>>except as a last resort, unless you didn't care about
>>the health of
>>the original pool.
>
>
> This is emphatically not what was being requested by me, in fact. I agree, I would be highly suspicious of the data's integrity depending upon the zpool structure.
>
> There are a couple other things to consider. First, this requires exporting the whole pool, which automatically assumes an outage is acceptable. Second, it assumes the other half of the mirror will be used elsewhere as in "on another system". This is also not a fair automatic assumption to make. The comments I made in this thread (I can't speak for others) were that the broken-off mirror may be used safely elsewhere, with the understanding this could even be on the same system, perhaps for archival reference or some-such. "Elsewhere" does not automatically imply "on another server". My apologies if I did not make this clear.
>
> Rainer
>
>
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris dot org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
651
From:
CA
Registered:
6/14/05
|
|
|
|
Re: Re: Re: Re: Re: How much do we really want zpool
remove?
Posted:
Apr 30, 2007 4:39 PM
in response to: cindys
To: Communities » zfs » discuss
|
|
> Hi Rainer,
>
> This is a long thread and I wasn't commenting on your
> previous
> replies regarding mirror manipulation. If I was, I
> would have done
> so directly. :-)
Yes, I realize. I did the response on your post because I was agreeing with you. :-) I was just extending your comment by indicating the idea proposed before your post was not acceptable, and didn't address the core functionality missing. Sorry if my placing of my post confused things.
Rainer
|
|
|
|
|
Posts:
2
From:
Registered:
1/25/07
|
|
|
|
Re: How much do we really want zpool remove?
Posted:
Jan 25, 2007 7:54 AM
in response to: nerant
To: Communities » zfs » discuss
|
|
The ability to shrink a pool by removing devices is the only reason my enterprise is not yet using ZFS, simply because it prevents us from easily migrating storage.
|
|
|
|
|
Posts:
803
From:
Plano, TX
Registered:
4/27/05
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 25, 2007 9:35 AM
in response to: stevew
|
|
On Thu, 25 Jan 2007, SteveW wrote:
... reformatted ...
> The ability to shrink a pool by removing devices is the only reason my
> enterprise is not yet using ZFS, simply because it prevents us from
> easily migrating storage.
That logic is totally bogus AFAIC. There are so many advantages to
running ZFS that denying yourself that opportunity is very short sighted -
especially when there are lots of ways of working around this minor
feature deficiency.
Perhaps if you post some specifics of your environment and usage scenario,
we can suggest workarounds.
Regards,
Al Hopper Logical Approach Inc, Plano, TX. al@logical-approach.com
Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT
OpenSolaris.Org Community Advisory Board (CAB) Member - Apr 2005
OpenSolaris Governing Board (OGB) Member - Feb 2006
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
221
From:
Philadelphia, PA
Registered:
1/12/06
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 25, 2007 9:51 AM
in response to: alhopper
|
|
On Thu, Jan 25, 2007 at 11:35:54AM -0600, Al Hopper wrote:
> On Thu, 25 Jan 2007, SteveW wrote:
>
> ... reformatted ...
> > The ability to shrink a pool by removing devices is the only reason my
> > enterprise is not yet using ZFS, simply because it prevents us from
> > easily migrating storage.
>
> That logic is totally bogus AFAIC. There are so many advantages to
> running ZFS that denying yourself that opportunity is very short sighted -
> especially when there are lots of ways of working around this minor
> feature deficiency.
The other point is, how many other volume management systems allow you to remove
disks? I bet if the answer is not zero, it's not large. ;)
-brian
--
"The reason I don't use Gnome: every single other window manager I know of is
very powerfully extensible, where you can switch actions to different mouse
buttons. Guess which one is not, because it might confuse the poor users?
Here's a hint: it's not the small and fast one." --Linus
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Darren Dunham
ddunham@taos.com
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 25, 2007 10:51 AM
in response to: wonko
|
|
> The other point is, how many other volume management systems allow you
> to remove disks? I bet if the answer is not zero, it's not large. ;)
As far as Solaris is concerned, I'm only aware of two significant such
systems. SVM and VxVM.
SVM doesn't really manage disks per-se. So there's really nothing in it
that disallows removing them or reusing them. Of course it offers no
help in migrating data off of any such disks.
VxVM does have tools to migrate data from a disk, and to either remove a
disk from a pool, or even migrate data on a disk into another pool. In
many cases, enterprise customers have this existing functionality in
mind when considering ZFS.
As a third point, follow the Network Appliance list a bit and you'll see
that the question comes up with their systems fairly often. It's not
uncommon for someone to accidentally add a spare disk to a volume or
aggregate. Said disk cannot be retreived without destroying and
recreating (a blank) volume or aggregate, followed by a restore.
See also ZFS automatically increasing the size of a pool to take up all
space, even if I don't want it to...
--
Darren Dunham ddunham at taos dot com
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
2
From:
Registered:
1/25/07
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 25, 2007 12:15 PM
in response to: Darren Dunham
To: Communities » zfs » discuss
|
|
> > The other point is, how many other volume management systems allow you
> > to remove disks? I bet if the answer is not zero, it's not large. ;)
>
> As far as Solaris is concerned, I'm only aware of two significant such
> systems. SVM and VxVM.
Correct, in our case we've been using VxVM for years.
We're comfortable with VxVM, it does what we want, vxfs performance is outstanding, and we move storage around [u]A LOT[/u] as our SAN storage changes. So for the time being, the benefits of ZFS won't allow us to give up the ability to shrink/remove on the fly.
I should add that I am very excited about ZFS, and hope to be able to use it [i]everywhere[/i] as soon as possible.
|
|
|
|
|
Posts:
262
From:
Cardiff, Wales, United Kingdom
Registered:
5/2/06
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 4:24 AM
in response to: wonko
|
|
On 25/01/07, Brian Hechinger <wonko at 4amlunch dot net> wrote:
> The other point is, how many other volume management systems allow you to remove
> disks? I bet if the answer is not zero, it's not large. ;)
Even Linux LVM can do this (with pvmove) - slow, but you can do it online.
--
Rasputin :: Jack of All Trades - Master of Nuns
http://number9.hellooperator.net/
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
1
From:
Registered:
2/21/07
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Feb 21, 2007 7:34 AM
in response to: alhopper
To: Communities » zfs » discuss
|
|
> > The ability to shrink a pool by removing devices is the only reason my
> > enterprise is not yet using ZFS, simply because it prevents us from
> > easily migrating storage.
>
> That logic is totally bogus AFAIC. There are so many advantages to
> running ZFS that denying yourself that opportunity is very short sighted -
> especially when there are lots of ways of working around this minor
> feature deficiency.
I cannot let you say that.
Here in my company we are very interested in ZFS, but we do not care about the RAID/mirror features, because we already have a SAN with RAID-5 disks, and dual fabric connection to the hosts.
We would have migrated already if we could simply migrate data from a storage array to another (which we do more often than you might think).
Currently we use (and pay for) VXVM, here is how we do a migration:
1/ Allocate disks from the new array, visible by the host.
2/ Add the disks in the diskgroup.
3/ Run vxevac to evacuate data from "old" disks.
4/ Remove old disks from the DG.
If you explain how to do that with ZFS, no downtime, and new disks with different capacities, you're my hero ;-)
|
|
|
|
|
Posts:
1,091
From:
CA
Registered:
4/27/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 21, 2007 7:37 AM
in response to: valoche
|
|
On Wed, 21 Feb 2007, Valery Fouques wrote:
> Here in my company we are very interested in ZFS, but we do not care
> about the RAID/mirror features, because we already have a SAN with
> RAID-5 disks, and dual fabric connection to the hosts.
... And presumably you've read the threads where ZFS has helped find
(and repair) corruption in such setups?
(But yeah, I agree the ability to shrink a pool is important.)
--
Rich Teer, SCSA, SCNA, SCSECA, OpenSolaris CAB member
President,
Rite Online Inc.
Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
3,398
From:
Registered:
3/9/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 21, 2007 7:43 AM
in response to: valoche
|
|
>I cannot let you say that.
>Here in my company we are very interested in ZFS, but we do not care
>about the RAID/mirror features, because we already have a SAN with
>RAID-5 disks, and dual fabric connection to the hosts.
But you understand that these underlying RAID mechanism give absolutely
no guarantee about data integrity but only that some data was found were
some (possibly other) data was written? (RAID5 never verifies the
checkum is correct on reads; it only uses it to reconstruct data when
reads fail)
Casper
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
242
From:
Registered:
7/7/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 21, 2007 10:31 AM
in response to: casper
|
|
On February 21, 2007 4:43:34 PM +0100 Casper.***@Sun.COM wrote:
>
>> I cannot let you say that.
>> Here in my company we are very interested in ZFS, but we do not care
>> about the RAID/mirror features, because we already have a SAN with
>> RAID-5 disks, and dual fabric connection to the hosts.
>
> But you understand that these underlying RAID mechanism give absolutely
> no guarantee about data integrity but only that some data was found were
> some (possibly other) data was written? (RAID5 never verifies the
> checkum is correct on reads; it only uses it to reconstruct data when
> reads fail)
um, I thought smarter arrays did that these days. Of course it's not
end-to-end so the parity verification isn't as useful as it should be;
gigo.
-frank
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
3,398
From:
Registered:
3/9/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 21, 2007 10:59 AM
in response to: frank
|
|
>On February 21, 2007 4:43:34 PM +0100 Casper.***@Sun.COM wrote:
>>
>>> I cannot let you say that.
>>> Here in my company we are very interested in ZFS, but we do not care
>>> about the RAID/mirror features, because we already have a SAN with
>>> RAID-5 disks, and dual fabric connection to the hosts.
>>
>> But you understand that these underlying RAID mechanism give absolutely
>> no guarantee about data integrity but only that some data was found were
>> some (possibly other) data was written? (RAID5 never verifies the
>> checkum is correct on reads; it only uses it to reconstruct data when
>> reads fail)
>
>um, I thought smarter arrays did that these days. Of course it's not
>end-to-end so the parity verification isn't as useful as it should be;
>gigo.
Generate extra I/O and verify parity, is that not something that may
be a problem in performance benchmarking?
For mirroring, a similar problem exists, of course. ZFS reads from the
right side of the mirror and corrects the wrong side if it finds an
error. RAIDs do not.
Casper
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 22, 2007 12:37 AM
in response to: casper
|
|
On Wed, Feb 21, 2007 at 04:43:34PM +0100, Casper.***@Sun.COM wrote:
>
> >I cannot let you say that.
> >Here in my company we are very interested in ZFS, but we do not care
> >about the RAID/mirror features, because we already have a SAN with
> >RAID-5 disks, and dual fabric connection to the hosts.
>
> But you understand that these underlying RAID mechanism give absolutely
> no guarantee about data integrity but only that some data was found were
> some (possibly other) data was written? (RAID5 never verifies the
> checkum is correct on reads; it only uses it to reconstruct data when
> reads fail)
But you understand that he perhaps knows that but so far nothing wrong
happened [*] and migration is still very important feature for him ?
[*] almost every big company has its data center with SAN and FC
connections with RAID-5 or RAID-10 in their storage arrays
and they are treated as reliable
przemol
----------------------------------------------------------------------
Wpadka w kosciele - zobacz >> http://link.interia.pl/f19ea
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Jason J. W. Wil...
jasonjwwilliams@gmai...
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 22, 2007 11:21 AM
in response to: przemolicc@pocz...
|
|
Hi Przemol,
I think Casper had a good point bringing up the data integrity
features when using ZFS for RAID. Big companies do a lot of things
"just because that's the certified way" that end up biting them in the
rear. Trusting your SAN arrays is one of them. That all being said,
the need to do migrations is a very valid concern.
Best Regards,
Jason
On 2/22/07, przemolicc at poczta dot fm <przemolicc at poczta dot fm> wrote:
> On Wed, Feb 21, 2007 at 04:43:34PM +0100, Casper.***@Sun.COM wrote:
> >
> > >I cannot let you say that.
> > >Here in my company we are very interested in ZFS, but we do not care
> > >about the RAID/mirror features, because we already have a SAN with
> > >RAID-5 disks, and dual fabric connection to the hosts.
> >
> > But you understand that these underlying RAID mechanism give absolutely
> > no guarantee about data integrity but only that some data was found were
> > some (possibly other) data was written? (RAID5 never verifies the
> > checkum is correct on reads; it only uses it to reconstruct data when
> > reads fail)
>
> But you understand that he perhaps knows that but so far nothing wrong
> happened [*] and migration is still very important feature for him ?
>
> [*] almost every big company has its data center with SAN and FC
> connections with RAID-5 or RAID-10 in their storage arrays
> and they are treated as reliable
>
> przemol
>
> ----------------------------------------------------------------------
> Wpadka w kosciele - zobacz >> http://link.interia.pl/f19ea
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris dot org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 27, 2007 1:21 AM
in response to: Jason J. W. Wil...
|
|
On Thu, Feb 22, 2007 at 12:21:50PM -0700, Jason J. W. Williams wrote:
> Hi Przemol,
>
> I think Casper had a good point bringing up the data integrity
> features when using ZFS for RAID. Big companies do a lot of things
> "just because that's the certified way" that end up biting them in the
> rear. Trusting your SAN arrays is one of them. That all being said,
> the need to do migrations is a very valid concern.
Jason,
I don't claim that SAN/RAID solutions are the best and don't have any
mistakes/failures/problems. But if SAN/RAID is so bad why companies
using them survive ?
Imagine also that some company is using SAN/RAID for a few years
and doesn't have any problems (or once a few months). Also from time to
time they need to migrate between arrays (for whatever reason). Now you come and say
that they have unreliable SAN/RAID and you offer something new (ZFS)
which is going to make it much more reliable but migration to another array
will be painfull. What do you think what they choose ?
BTW: I am a fan of ZFS. :-)
przemol
----------------------------------------------------------------------
Ustawiaj rekordy DNS dla swojej domeny >>
http://link.interia.pl/f1a1a
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
1,154
From:
US
Registered:
6/14/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 27, 2007 1:29 AM
in response to: przemolicc@pocz...
|
|
On 27/02/07, przemolicc at poczta dot fm <przemolicc at poczta dot fm> wrote:
> On Thu, Feb 22, 2007 at 12:21:50PM -0700, Jason J. W. Williams wrote:
> > Hi Przemol,
> >
> > I think Casper had a good point bringing up the data integrity
> > features when using ZFS for RAID. Big companies do a lot of things
> > "just because that's the certified way" that end up biting them in the
> > rear. Trusting your SAN arrays is one of them. That all being said,
> > the need to do migrations is a very valid concern.
>
> Jason,
>
> I don't claim that SAN/RAID solutions are the best and don't have any
> mistakes/failures/problems. But if SAN/RAID is so bad why companies
> using them survive ?
I think he was trying to say that people that believe that those
solutions are reliable just because they are based on SAN/RAID
technology and are not aware of the true situation surrounding them.
--
"Less is only more where more is no good." --Frank Lloyd Wright
Shawn Walker, Software and Systems Analyst
binarycrusader at gmail dot com - http://binarycrusader.blogspot.com/
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 27, 2007 2:28 AM
in response to: swalker
|
|
On Tue, Feb 27, 2007 at 08:29:04PM +1100, Shawn Walker wrote:
> On 27/02/07, przemolicc at poczta dot fm <przemolicc at poczta dot fm> wrote:
> >On Thu, Feb 22, 2007 at 12:21:50PM -0700, Jason J. W. Williams wrote:
> >> Hi Przemol,
> >>
> >> I think Casper had a good point bringing up the data integrity
> >> features when using ZFS for RAID. Big companies do a lot of things
> >> "just because that's the certified way" that end up biting them in the
> >> rear. Trusting your SAN arrays is one of them. That all being said,
> >> the need to do migrations is a very valid concern.
> >
> >Jason,
> >
> >I don't claim that SAN/RAID solutions are the best and don't have any
> >mistakes/failures/problems. But if SAN/RAID is so bad why companies
> >using them survive ?
>
> I think he was trying to say that people that believe that those
> solutions are reliable just because they are based on SAN/RAID
> technology and are not aware of the true situation surrounding them.
Is the "true situation" really so bad ?
My feeling was that he was trying to say that there is no SAN/RAID
solution without data integrity problem. Is it really true ?
Does anybody have any paper (*) about percentage of problems in SAN/RAID
because of data integrity ? Is it 5 % ? Or 30 % ? Or maybe 60 % ?
(*) Maybe such paper/report should be a start point for our discussion.
przemol
----------------------------------------------------------------------
Gdy nie ma dzieci... - zobacz >> http://link.interia.pl/f19eb
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Robert Milkowski
rmilkowski@task.gda.pl
|
|
|
|
Re[2]: Re: Re: How much do we really want zpool remove?
Posted:
Feb 27, 2007 4:01 AM
in response to: przemolicc@pocz...
|
|
Hello przemolicc,
Tuesday, February 27, 2007, 11:28:59 AM, you wrote:
ppf> On Tue, Feb 27, 2007 at 08:29:04PM +1100, Shawn Walker wrote:
>> On 27/02/07, przemolicc at poczta dot fm <przemolicc at poczta dot fm> wrote:
>> >On Thu, Feb 22, 2007 at 12:21:50PM -0700, Jason J. W. Williams wrote:
>> >> Hi Przemol,
>> >>
>> >> I think Casper had a good point bringing up the data integrity
>> >> features when using ZFS for RAID. Big companies do a lot of things
>> >> "just because that's the certified way" that end up biting them in the
>> >> rear. Trusting your SAN arrays is one of them. That all being said,
>> >> the need to do migrations is a very valid concern.
>> >
>> >Jason,
>> >
>> >I don't claim that SAN/RAID solutions are the best and don't have any
>> >mistakes/failures/problems. But if SAN/RAID is so bad why companies
>> >using them survive ?
>>
>> I think he was trying to say that people that believe that those
>> solutions are reliable just because they are based on SAN/RAID
>> technology and are not aware of the true situation surrounding them.
ppf> Is the "true situation" really so bad ?
ppf> My feeling was that he was trying to say that there is no SAN/RAID
ppf> solution without data integrity problem. Is it really true ?
ppf> Does anybody have any paper (*) about percentage of problems in SAN/RAID
ppf> because of data integrity ? Is it 5 % ? Or 30 % ? Or maybe 60 % ?
ppf> (*) Maybe such paper/report should be a start point for our discussion.
See http://sunsolve.sun.com/search/document.do?assetkey=1-26-102815-1
as one example. This is entry level array but still such things
happens. I do also observer similar problems with IBM's array
(larger).
It's just that people are used to fsck from time to time not really
knowing why and in many cases they do not realize that their data is
not exactly what they expect it to be.
However from my experience I must admit the problem is almost only
seen with SATA drives.
I had a problem with SCSI adapter which was
sending some warnings (driver) but still was passing IOs - it turned
out data were corrupted. Changing SCSI adapter solved the problem.
The point is that thanks to ZFS we caught the problem, replaced
bad card, did zpool scrub and everything was in perfect shape. No need
to resynchronize data, etc.
Another time I had a problem with FC array and lost some data but
there's no ZFS on it :(((
On all other arrays, jbods, etc. with SCSI and/or FC disks I haven't
seen (yet) checksum errors reported by ZFS.
--
Best regards,
Robert mailto:rmilkowski at task dot gda dot pl
http://milek.blogspot.com
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
196
From:
Santa Clara, CA
Registered:
6/23/06
|
|
|
|
Re: Re[2]: Re: Re: How much do we really want zpool
remove?
Posted:
Feb 27, 2007 8:47 AM
in response to: Robert Milkowski
|
|
<huge forwards on how bad SANs really are for data integrity removed>
The answer is: insufficient data.
With modern journalling filesystems, I've never had to fsck anything or
run a filesystem repair. Ever. On any of my SAN stuff.
The sole place I've run into filesystem corruption in the traditional
sense is with faulty hardware controllers; and, I'm not even sure ZFS
could recover from those situations, though less dire ones where the
controllers are merely emitting slightly wonky problems certainly would
be within ZFS's ability to fix, vice the inability of a SAN to determine
that the data was bad.
That said, the primary issue here is that nobody really has any idea
about silent corruption - that is, blocks which change value, but are
data, not filesystem-relevant. Bit flips and all. Realistically, the
only way previous to ZFS to detect this was to do bit-wise comparisons
against backups, which becomes practically impossible on an active data
set.
SAN/RAID equipment still has a very considerable place over JBODs in
most large-scale places, particularly in areas of configuration
flexibility, security, and management. That said, I think we're arguing
at cross-purposes: the real solution for most enterprise customers is
SAN + ZFS, not either just by itself.
--
Erik Trimble
Java System Support
Mailstop: usca14-102
Phone: x17195
Santa Clara, CA
Timezone: US/Pacific (GMT-0800)
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Robert Milkowski
rmilkowski@task.gda.pl
|
|
|
|
Re[4]: Re: Re: How much do we really want zpool remove?
Posted:
Feb 27, 2007 3:40 PM
in response to: et151817
|
|
Hello Erik,
Tuesday, February 27, 2007, 5:47:42 PM, you wrote:
ET> <huge forwards on how bad SANs really are for data integrity removed>
ET> The answer is: insufficient data.
ET> With modern journalling filesystems, I've never had to fsck anything or
ET> run a filesystem repair. Ever. On any of my SAN stuff.
I'm not sure if you consider UFS in S10 as a modern journalling
filesystem but in case you do:
Feb 13 12:03:16 XXXX ufs: [ID 879645 kern.notice] NOTICE: /opt/d1635: unexpected free inode 54305084, run fsck(1M) -o f
This file system is on a medium large array (IBM) in a SAN
environment.
--
Best regards,
Robert mailto:rmilkowski at task dot gda dot pl
http://milek.blogspot.com
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
196
From:
Santa Clara, CA
Registered:
6/23/06
|
|
|
|
Re: Re[4]: Re: Re: How much do we really want zpool
remove?
Posted:
Feb 27, 2007 3:55 PM
in response to: Robert Milkowski
|
|
Honestly, no, I don't consider UFS a modern file system. :-)
It's just not in the same class as JFS for AIX, xfs for IRIX, or even
VxFS.
-Erik
On Wed, 2007-02-28 at 00:40 +0100, Robert Milkowski wrote:
> Hello Erik,
>
> Tuesday, February 27, 2007, 5:47:42 PM, you wrote:
>
> ET> <huge forwards on how bad SANs really are for data integrity removed>
>
>
> ET> The answer is: insufficient data.
>
>
> ET> With modern journalling filesystems, I've never had to fsck anything or
> ET> run a filesystem repair. Ever. On any of my SAN stuff.
>
> I'm not sure if you consider UFS in S10 as a modern journalling
> filesystem but in case you do:
>
> Feb 13 12:03:16 XXXX ufs: [ID 879645 kern.notice] NOTICE: /opt/d1635: unexpected free inode 54305084, run fsck(1M) -o f
>
> This file system is on a medium large array (IBM) in a SAN
> environment.
>
>
>
--
Erik Trimble
Java System Support
Mailstop: usca14-102
Phone: x17195
Santa Clara, CA
Timezone: US/Pacific (GMT-0800)
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Robert Milkowski
rmilkowski@task.gda.pl
|
|
|
|
Re[6]: Re: Re: How much do we really want zpool remove?
Posted:
Feb 28, 2007 1:17 AM
in response to: et151817
|
|
Hello Erik,
Wednesday, February 28, 2007, 12:55:24 AM, you wrote:
ET> Honestly, no, I don't consider UFS a modern file system. :-)
ET> It's just not in the same class as JFS for AIX, xfs for IRIX, or even
ET> VxFS.
The point is that fsck was due to an array corrupting data.
IMHO it would hit JFS, XFS or VxFS as bad as UFS if not worse.
--
Best regards,
Robert mailto:rmilkowski at task dot gda dot pl
http://milek.blogspot.com
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
125
From:
US
Registered:
12/1/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 27, 2007 5:35 PM
in response to: Robert Milkowski
|
|
> With modern journalling filesystems, I've never had to fsck anything or
> run a filesystem repair. Ever. On any of my SAN stuff.
you will.. even if the SAN is perfect, you will hit
bugs in the filesystem code.. from lots of rsync hard
links or like this one from raidtools last week:
Feb 9 05:38:39 orbit kernel: mptbase: ioc2: IOCStatus(0x0043): SCSI Device Not There
Feb 9 05:38:39 orbit kernel: md: write_disk_sb failed for device sdp1
Feb 9 05:38:39 orbit kernel: md: errors occurred during superblock update, repeating
Feb 9 05:39:01 orbit kernel: raid6: Disk failure on sdp1, disabling device. Operation continuing on 13 devices
Feb 9 05:39:09 orbit kernel: mptscsi: ioc2: attempting task abort! (sc=cb17c800)
Feb 9 05:39:10 orbit kernel: RAID6 conf printout:
Feb 9 05:39:10 orbit kernel: --- rd:14 wd:13 fd:1
Feb 9 05:44:37 orbit kernel: EXT3-fs error (device dm-0): ext3_readdir: bad entry in directory #10484: rec_len %$
Feb 9 05:44:37 orbit kernel: Aborting journal on device dm-0.
Feb 9 05:44:37 orbit kernel: ext3_abort called.
Feb 9 05:44:37 orbit kernel: EXT3-fs error (device dm-0): ext3_journal_start_sb: Detected aborted journal
Feb 9 05:44:37 orbit kernel: Remounting filesystem read-only
Feb 9 05:44:37 orbit kernel: attempt to access beyond end of device
Feb 9 05:44:44 orbit kernel: oom-killer: gfp_mask=0xd0
<death and crupt fs>
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
1,858
From:
US
Registered:
6/17/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 27, 2007 10:11 AM
in response to: przemolicc@pocz...
|
|
przemolicc at poczta dot fm wrote:
> Is the "true situation" really so bad ?
The failure mode is silent error. By definition, it is hard to
count silent errors. What ZFS does is improve the detection of
silent errors by a rather considerable margin. So, what we are
seeing is that suddenly people are seeing errors that they didn't
see before (or do you "hear" silent errors? ;-). That has been
surprising and leads some of us to recommend ZFS no matter what
your storage looks like, even if silent error detection is the
only benefit.
-- richard
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Jason J. W. Wil...
jasonjwwilliams@gmai...
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 27, 2007 3:51 PM
in response to: przemolicc@pocz...
|
|
Hi Przemol,
I think migration is a really important feature...think I said that...
;-) SAN/RAID is not awful...frankly there's not been better solution
(outside of NetApp's WAFL) till ZFS. SAN/RAID just has its own
reliability issues you accept unless you don't have to....ZFS :-)
-J
On 2/27/07, przemolicc at poczta dot fm <przemolicc at poczta dot fm> wrote:
> On Thu, Feb 22, 2007 at 12:21:50PM -0700, Jason J. W. Williams wrote:
> > Hi Przemol,
> >
> > I think Casper had a good point bringing up the data integrity
> > features when using ZFS for RAID. Big companies do a lot of things
> > "just because that's the certified way" that end up biting them in the
> > rear. Trusting your SAN arrays is one of them. That all being said,
> > the need to do migrations is a very valid concern.
>
> Jason,
>
> I don't claim that SAN/RAID solutions are the best and don't have any
> mistakes/failures/problems. But if SAN/RAID is so bad why companies
> using them survive ?
>
> Imagine also that some company is using SAN/RAID for a few years
> and doesn't have any problems (or once a few months). Also from time to
> time they need to migrate between arrays (for whatever reason). Now you come and say
> that they have unreliable SAN/RAID and you offer something new (ZFS)
> which is going to make it much more reliable but migration to another array
> will be painfull. What do you think what they choose ?
>
> BTW: I am a fan of ZFS. :-)
>
> przemol
>
> ----------------------------------------------------------------------
> Ustawiaj rekordy DNS dla swojej domeny >>
> http://link.interia.pl/f1a1a
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris dot org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
1,858
From:
US
Registered:
6/17/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 21, 2007 10:55 AM
in response to: valoche
|
|
Valery Fouques wrote:
>>> The ability to shrink a pool by removing devices is the only reason my
>>> enterprise is not yet using ZFS, simply because it prevents us from
>>> easily migrating storage.
>> That logic is totally bogus AFAIC. There are so many advantages to
>> running ZFS that denying yourself that opportunity is very short sighted -
>> especially when there are lots of ways of working around this minor
>> feature deficiency.
>
> I cannot let you say that.
> Here in my company we are very interested in ZFS, but we do not care about the RAID/mirror features, because we already have a SAN with RAID-5 disks, and dual fabric connection to the hosts.
>
> We would have migrated already if we could simply migrate data from a storage array to another (which we do more often than you might think).
>
> Currently we use (and pay for) VXVM, here is how we do a migration:
But you describe VxVM feature, not a file system feature.
> 1/ Allocate disks from the new array, visible by the host.
> 2/ Add the disks in the diskgroup.
> 3/ Run vxevac to evacuate data from "old" disks.
> 4/ Remove old disks from the DG.
>
> If you explain how to do that with ZFS, no downtime, and new disks with different capacities, you're my hero ;-)
zpool replace old-disk new-disk
The caveat is that new-disk must be as big or bigger than old-disk.
This caveat is the core of the shrink "problem"
-- richard
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
242
From:
Registered:
7/7/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 21, 2007 11:50 AM
in response to: relling
|
|
On February 21, 2007 10:55:43 AM -0800 Richard Elling
<Richard dot Elling at Sun dot COM> wrote:
> Valery Fouques wrote:
>>>> The ability to shrink a pool by removing devices is the only reason my
>>>> enterprise is not yet using ZFS, simply because it prevents us from
>>>> easily migrating storage.
>>> That logic is totally bogus AFAIC. There are so many advantages to
>>> running ZFS that denying yourself that opportunity is very short
>>> sighted - especially when there are lots of ways of working around this
>>> minor feature deficiency.
>>
>> I cannot let you say that.
>> Here in my company we are very interested in ZFS, but we do not care
>> about the RAID/mirror features, because we already have a SAN with
>> RAID-5 disks, and dual fabric connection to the hosts.
>>
>> We would have migrated already if we could simply migrate data from a
>> storage array to another (which we do more often than you might think).
>>
>> Currently we use (and pay for) VXVM, here is how we do a migration:
>
> But you describe VxVM feature, not a file system feature.
But in the context of zfs, this is appropriate.
-frank
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Peter Schuller
peter.schuller@infid...
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 25, 2007 10:22 AM
in response to: stevew
|
|
> The ability to shrink a pool by removing devices is the only reason my
> enterprise is not yet using ZFS, simply because it prevents us from easily
> migrating storage.
Being able to do this would be very high on my wishlist from the perspective
of a home user.
But also from the perspective of more serious user (though I am not involved
in using ZFS in such a case - not yet anyway...) it is most definitely a very
nice thing to be able to do, in various stituations.
Example of things I would love to be able to do, which I would with such a
feature:
* Easily convert between mirror/striping/raidz/raidz2 (no need to purchase
twice the capacity for temporary storage during a conversion).
* Easily move storage between physical machines as needs change (assuming a
situation where you want drives locally attached to the machines in question,
and iSCSI and similar is not an option).
* Revert stupid misstake: accidentally adding something to a pool that should
not be there :)
* Easily - even live under the right circumstances - temporarily evacuate a
disk in order to e.g. perform drive testing if suspicious behavior is present
without a known cause.
* If a drive starts going bad and I do not have a spare readily available
(typical home use situation), I may want to evacuate the semi-broken drive so
that I do not loose redundancy until I can get another disk. May or may not
be practical depending on current disk space usage of course.
* Some machine A needs a spare drive but there is none, and I have free disk
space ond disk B and B has matching drives. Evacuate a disk on B and use as
replacement in A (again, typical home use situation). Once I obtain a new
drive revert B's disk into B again, or alternatively keep it in A and use the
new drive in B.
--
/ Peter Schuller
PGP userID: 0xE9758B7D or 'Peter Schuller <peter dot schuller at infidyne dot com>'
Key retrieval: Send an E-Mail to getpgpkey at scode dot org
E-Mail: peter dot schuller at infidyne dot com Web: http://www.scode.org
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
112
From:
DE
Registered:
3/7/06
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 1:00 AM
in response to: Peter Schuller
|
|
Hi,
I do agree that zpool remove is a _very_ desirable feature, not doubt about
that.
Here are a couple of thoughts and workarounds, in random order, that might
give us some more perspective:
- My home machine has 4 disks and a big zpool across them. Fine. But what
if a controller fails or worse, a CPU? Right, I need a second machine, if
I'm really honest with myself and serious with my data. Don't laugh, ZFS
on a Solaris server is becoming my mission-critical home storage solution
that is supposed to last beyond CDs and DVDs and other vulnerable media.
So, if I was an enterprise, I'd be willing to keep enough empty LUNs
available to facilitate at least the migration of one or more filesystems
if not complete pools. With a little bit of scripting, this can be done
quite easily and efficiently through zfs send/receive and some LUN
juggling.
If I was an enterprise's server admin and the storage guys wouldn't have
enough space for migrations, I'd be worried.
- We need to avoid customers thinking "Veritas can shrink, ZFS can't". That
is wrong. ZFS _filesystems_ grow and shrink all the time, it's just the
pools below them that can just grow. And Veritas does not even have pools.
People have started to follow a One-pool-to-store-them-all which I think
is not always appropriate. Some alternatives:
- One pool per zone might be a good idea if you want to migrate zones
across systems which then becomes easy through zpool export/import in
a SAN.
- One pool per service level (mirror, RAID-Z2, fast, slow, cheap, expensive)
might be another idea. Keep some cheap mirrored storage handy for your pool
migration needs and you could wiggle your life around zpool remove.
Switching between Mirror, RAID-Z, RAID-Z2 then becomes just a zfs
send/receive pair.
Shrinking a pool requires some more zfs send/receiving and maybe some
scripting, but these are IMHO less painful than living without ZFS'
data integrity and the other joys of ZFS.
Sorry if I'm stating the obvious or stuff that has been discussed before,
but the more I think about zpool remove, the more I think it's a question
of willingness to plan/work/script/provision vs. a real show stopper.
Best regards,
Constantin
P.S.: Now with my big mouth I hope I'll survive a customer confcall next
week with a customer asking for exactly zpool remove :).
--
Constantin Gonzalez Sun Microsystems GmbH, Germany
Platform Technology Group, Client Solutions http://www.sun.de/
Tel.: +49 89/4 60 08-25 91 http://blogs.sun.com/constantin/
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Darren Dunham
ddunham@taos.com
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 8:16 AM
in response to: constant
|
|
> - We need to avoid customers thinking "Veritas can shrink, ZFS can't". That
> is wrong. ZFS _filesystems_ grow and shrink all the time, it's just the
> pools below them that can just grow. And Veritas does not even have pools.
I'm sure that this issue is different for different environments, but I
assure you it wasn't raised because we're looking at a spec chart and
someone saw a missing check in the ZFS column. The ability to
deallocate in-use storage without having to migrate the existing data is
used today by many administrators. We'll live with this not being
possible in ZFS at the moment, but the limitation is real and the
flexibility of filesystems within the pool doesn't alleviate it.
> Sorry if I'm stating the obvious or stuff that has been discussed before,
> but the more I think about zpool remove, the more I think it's a question
> of willingness to plan/work/script/provision vs. a real show stopper.
Show stopper would depend on the environment. It's certainly not that
in many places. I agree that if I could exactly plan all my storage
perfectly in advance, then several ways that it would be really useful
would be reduced. However one of the reasons to have it is precisely
because it is so difficult to get good predictions for storage use.
I know just a touch of the internals of ZFS to understand why
remove/split/evacuate is much more difficult than it might be in simpler
volume managers. I'm happy we've got what we have today and that people
have already thought up ways of attacking this problem to make ZFS even
better.
--
Darren Dunham ddunham at taos dot com
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
125
From:
MPLS
Registered:
1/5/07
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 8:55 AM
in response to: constant
|
|
zfs-discuss-bounces at opensolaris dot org wrote on 01/26/2007 03:00:13 AM:
> Hi,
>
> I do agree that zpool remove is a _very_ desirable feature, not doubt
about
> that.
>
> Here are a couple of thoughts and workarounds, in random order, that
might
> give us some more perspective:
>
> - My home machine has 4 disks and a big zpool across them. Fine. But what
> if a controller fails or worse, a CPU? Right, I need a second machine,
if
> I'm really honest with myself and serious with my data. Don't laugh,
ZFS
> on a Solaris server is becoming my mission-critical home storage
solution
> that is supposed to last beyond CDs and DVDs and other vulnerable
media.
>
> So, if I was an enterprise, I'd be willing to keep enough empty LUNs
> available to facilitate at least the migration of one or more
filesystems
> if not complete pools. With a little bit of scripting, this can be done
> quite easily and efficiently through zfs send/receive and some LUN
> juggling.
>
> If I was an enterprise's server admin and the storage guys wouldn't
have
> enough space for migrations, I'd be worried.
>
I think you may find in practice that many medium to large enterprise IT
departments are in this exact situation -- we do not have luns sitting
stagnant just waiting for data migrations of our largest data sets. We
have been sold (and rightly so, because it works and is cost effective and
has no downtime) that we should be able to move luns around at will without
duplicating (to tape or disk) and dumping. You are really expecting to
have the storage guys to have 40tb of disk just sitting collecting dust
when you want to pull out 10 disks from a 44tb system? This type of
thinking may very well be why Sun has hard time in the last few years
(although zfs, and recent products show that the tide is turning).
> - We need to avoid customers thinking "Veritas can shrink, ZFS can't".
That
> is wrong. ZFS _filesystems_ grow and shrink all the time, it's just the
> pools below them that can just grow. And Veritas does not even have
pools.
>
Sorry, that is silly. Can we compare if we call them both "volumes or
filesystems (or any virtualization of each) which are reserved for data in
which we want to remove and add disks online"? vxfs can grow and shrink
and the volumes can grow and shrink. Pools may blur the line of volume/fs
but they are still delivering the same constraints to administrators trying
to admin these boxes and the disks attached to them.
> People have started to follow a One-pool-to-store-them-all which I
think
> is not always appropriate. Some alternatives:
>
> - One pool per zone might be a good idea if you want to migrate zones
> across systems which then becomes easy through zpool export/import in
> a SAN.
>
> - One pool per service level (mirror, RAID-Z2, fast, slow, cheap,
expensive)
> might be another idea. Keep some cheap mirrored storage handy
> for your pool
> migration needs and you could wiggle your life around zpool remove.
You went from one pool to share data (the major advantage of the pool
concept) to a bunch of constrained pools. Also how does this resolve the
issue of lun migration online?
>
> Switching between Mirror, RAID-Z, RAID-Z2 then becomes just a zfs
> send/receive pair.
>
> Shrinking a pool requires some more zfs send/receiving and maybe some
> scripting, but these are IMHO less painful than living without ZFS'
> data integrity and the other joys of ZFS.
Ohh, never mind, dump to tape and restore (err disk) -- you do realize
that the industry has been selling products that have made this behavior
obsolete for close to 10 years now?
>
> Sorry if I'm stating the obvious or stuff that has been discussed before,
> but the more I think about zpool remove, the more I think it's a question
> of willingness to plan/work/script/provision vs. a real show stopper.
>
No, it is a specific workflow that requires disk to stay online, while
allowing for economically sound use of resources -- this is not about
laziness (that is how I am reading your view) or not wanting to script up
solutions.
> Best regards,
> Constantin
>
> P.S.: Now with my big mouth I hope I'll survive a customer confcall next
> week with a customer asking for exactly zpool remove :).
I hope so, you may want to rethink the "script and go back in sysadmin
time 10 years" approach. ZFS buys alot and is a great filesystem but there
are places such as this that are still weak and need fixing for many
environments to be able to replace vxvm/vxfs or other solutions. Sure, you
will find people that are viewing this new pooled filesystem with old eyes,
but there are admins on this list that actually understand what they are
missing and the other options for working around these issues. We don't
look at this like a feature tickmark, but as a feature that we know is
missing that we really need to consider moving some of our systems from
vxvm/fs to zfs.
-Wade Stuart
>
> --
> Constantin Gonzalez Sun Microsystems GmbH,
Germany
> Platform Technology Group, Client Solutions
http://www.sun.de/
> Tel.: +49 89/4 60 08-25 91
http://blogs.sun.com/constantin/
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris dot org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
651
From:
CA
Registered:
6/14/05
|
|
|
|
Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 1:12 PM
in response to: constant
To: Communities » zfs » discuss
|
|
> So, if I was an enterprise, I'd be willing to keep
> enough empty LUNs
> available to facilitate at least the migration of
> one or more filesystems
> if not complete pools.
You might be, but don't be surprised when the Financials folks laugh you out of their office. Large corporations do not make money by leaving wads of cash lying around, and that's exactly what a few terabytes of unused storage in a high-end SAN is. This is in addition to the laughter generated by the comment that, "not a big deal if the Financials and HR databases are offline for three days while we do the migration." Good luck writing up a business case that justifies this sort of fiscal generosity.
Sorry, this argument smacks a little too much of being out of touch with the fiscal (and time) restrictions of working in a typical corporation, as opposed to a well-funded research group.
I hope I'm not sounding rude, but those of us working in medium to large corporations simply do not have the money for such luxuries. Period.
Rainer
|
|
|
|
|
Posts:
1,858
From:
US
Registered:
6/17/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 1:53 PM
in response to: rheilke
|
|
Rainer Heilke wrote:
>> So, if I was an enterprise, I'd be willing to keep
>> enough empty LUNs
>> available to facilitate at least the migration of
>> one or more filesystems
>> if not complete pools.
>
> You might be, but don't be surprised when the Financials folks laugh you out of their office. Large corporations do not make money by leaving wads of cash lying around, and that's exactly what a few terabytes of unused storage in a high-end SAN is. This is in addition to the laughter generated by the comment that, "not a big deal if the Financials and HR databases are offline for three days while we do the migration." Good luck writing up a business case that justifies this sort of fiscal generosity.
To be fair, you can replace vdevs with same-sized or larger vdevs online.
The issue is that you cannot replace with smaller vdevs nor can you
eliminate vdevs. In other words, I can migrate data around without
downtime, I just can't shrink or eliminate vdevs without send/recv.
This is where the philosophical disconnect lies. Everytime we descend
into this rathole, we stir up more confusion :-(
If you consider your pool of storage as a zpool, then the management of
subparts of the pool is done at the file system level. This concept is
different than other combinations of devices and file systems such as
SVM+UFS. When answering the ZFS shrink question, you need to make sure
you're not applying the old concepts to the new model.
Personally, I've never been in the situation where users ask for less storage,
but maybe I'm just the odd guy out? ;-) Others have offered cases where
a shrink or vdev restructuring could be useful. But I still see some
confusion with file system management (including zvols) and device management.
The shrink feature is primarily at the device management level.
-- richard
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Jason J. W. Wil...
jasonjwwilliams@gmai...
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 4:07 PM
in response to: relling
|
|
> To be fair, you can replace vdevs with same-sized or larger vdevs online.
> The issue is that you cannot replace with smaller vdevs nor can you
> eliminate vdevs. In other words, I can migrate data around without
> downtime, I just can't shrink or eliminate vdevs without send/recv.
> This is where the philosophical disconnect lies. Everytime we descend
> into this rathole, we stir up more confusion :-(
We did just this to move off RAID-5 LUNs that were the vdevs for a
pool, to RAID-10 LUNs. Worked very well, and as Richard said was done
all on-line. Doesn't really address the shrinking issue though. :-)
Best Regards,
Jason
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
377
From:
McLean, VA USA
Registered:
1/24/06
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 5:34 PM
in response to: relling
|
|
Richard Elling wrote:
>
> Personally, I've never been in the situation where users ask for less
> storage,
> but maybe I'm just the odd guy out? ;-)
You just realized that JoeSysadmin allocated ten luns to the zpool when
he realy only should have allocated one.
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
651
From:
CA
Registered:
6/14/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 8:23 PM
in response to: relling
To: Communities » zfs » discuss
|
|
Richard Elling wrote:
> Rainer Heilke wrote:
>
>>> So, if I was an enterprise, I'd be willing to keep
>>> enough empty LUNs
>>> available to facilitate at least the migration of
>>> one or more filesystems
>>> if not complete pools.
>>
>>
>>
>> You might be, but don't be surprised when the Financials folks laugh you out of their office. Large corporations do not make money by leaving wads of cash lying around, and that's exactly what a few terabytes of unused storage in a high-end SAN is. This is in addition to the laughter generated by the comment that, "not a big deal if the Financials and HR databases are offline for three days while we do the migration." Good luck writing up a business case that justifies this sort of fiscal generosity.
>
>
>
> To be fair, you can replace vdevs with same-sized or larger vdevs online.
> The issue is that you cannot replace with smaller vdevs nor can you
> eliminate vdevs. In other words, I can migrate data around without
> downtime, I just can't shrink or eliminate vdevs without send/recv.
> This is where the philosophical disconnect lies. Everytime we descend
> into this rathole, we stir up more confusion :-(
>
> If you consider your pool of storage as a zpool, then the management of
> subparts of the pool is done at the file system level. This concept is
> different than other combinations of devices and file systems such as
> SVM+UFS. When answering the ZFS shrink question, you need to make sure
> you're not applying the old concepts to the new model.
>
> Personally, I've never been in the situation where users ask for less storage,
> but maybe I'm just the odd guy out? ;-) Others have offered cases where
> a shrink or vdev restructuring could be useful. But I still see some
> confusion with file system management (including zvols) and device management.
> The shrink feature is primarily at the device management level.
> -- richard
I understand these arguments, and the differences (and that most users will never ask for less storage), but there are many instances where storage needs to move around, even between systems, and in that case, unless a whole zpool of storage is going, how do you do it? You need to give back two LUN's in a 6-LUN zpool. Oh, wait. You can't shrink a zpool.
Many people here are giving examples of where this capability is needed. We need to agree that different users' needs vary, and that there are real reasons for this.
Rainer
|
|
|
|
|
Posts:
803
From:
Plano, TX
Registered:
4/27/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 3:58 PM
in response to: rheilke
|
|
On Fri, 26 Jan 2007, Rainer Heilke wrote:
> > So, if I was an enterprise, I'd be willing to keep
> > enough empty LUNs
> > available to facilitate at least the migration of
> > one or more filesystems
> > if not complete pools.
.... reformatted ...
> You might be, but don't be surprised when the Financials folks laugh you
> out of their office. Large corporations do not make money by leaving
> wads of cash lying around, and that's exactly what a few terabytes of
> unused storage in a high-end SAN is. This is in addition to the laughter
But this is exactly where ZFS distrupts "Large corporations" thinking.
You're talking about (for example) 2 terabytes on a high-end SAN which
costs (what ?) per GB (including the capital cost of the hi-end SAN)
versus a dual Opteron box with 12 * 500Gb SATA disk drives that gives you
5TB of storage for (in round numbers) a total of ~ $6k. And how much are
your ongoing monthlies on that hi-end SAN box? (Don't answer) So - aside
from the occasional use of the box for data migration, this ZFS "storage
box" has 1,001 other uses. Pick any two (uses), based on your knowledge
of big corporation thinking and its an easy sell to management.
Now your accounting folks are going to be asking you to justify the
purchase of that hi-end SAN box.... and why you're not using ZFS
everywhere. :)
Oh - and the accounting folks love it when you tell them there's no
ongoing cost of ownership - because Joe Screwdriver can swap out a failed
Seagate 500Gb SATA drive after he picks up a replacement from Frys on his
lunch break!
> generated by the comment that, "not a big deal if the Financials and HR
> databases are offline for three days while we do the migration." Good
Again - sounds like more "legacy" thinking. With multiple gigabit
ethernet connections you can move terrabytes of information in a hour,
instead of in 24-hours - using legacy tape systems etc. This can be
easily handled during scheduled downtime.
> luck writing up a business case that justifies this sort of fiscal
> generosity.
> Sorry, this argument smacks a little too much of being out of touch with
> the fiscal (and time) restrictions of working in a typical corporation,
> as opposed to a well-funded research group.
>
> I hope I'm not sounding rude, but those of us working in medium to large
> corporations simply do not have the money for such luxuries. Period.
On the contrary - if you're not thinking ZFS, you're wasting a ton of IT
$s and hurting the competitiveness of your business.
Regards,
Al Hopper Logical Approach Inc, Plano, TX. al@logical-approach.com
Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT
OpenSolaris.Org Community Advisory Board (CAB) Member - Apr 2005
OpenSolaris Governing Board (OGB) Member - Feb 2006
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Toby Thain
toby@smartgames.ca
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 4:50 PM
in response to: alhopper
|
|
>
> Oh - and the accounting folks love it when you tell them there's no
> ongoing cost of ownership - because Joe Screwdriver can swap out a
> failed
> Seagate 500Gb SATA drive after he picks up a replacement from Frys
> on his
> lunch break!
Why do people think this will work? I never could figure it out.
There's many a slip 'twixt cup and lip. You need the spare already
sitting there.
--T
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
803
From:
Plano, TX
Registered:
4/27/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 5:24 PM
in response to: Toby Thain
|
|
On Fri, 26 Jan 2007, Toby Thain wrote:
> >
> > Oh - and the accounting folks love it when you tell them there's no
> > ongoing cost of ownership - because Joe Screwdriver can swap out a
> > failed
> > Seagate 500Gb SATA drive after he picks up a replacement from Frys
> > on his
> > lunch break!
>
> Why do people think this will work? I never could figure it out.
>
> There's many a slip 'twixt cup and lip. You need the spare already
> sitting there.
Agreed. I remember years ago, when a Sun service tech came onsite at a
fortune 100 company I was working in at the time and we stopped him,
handed him a disk drive in an anti-static bag and said - "don't unpack
your tools - it was a bad disk, we replaced it from our spares, here's the
bad one - please replace it under the service agreement". He thought
about this for about 5 Seconds and said; "I wish all my customers were
like you guys". Then he was gone! :)
Regards,
Al Hopper Logical Approach Inc, Plano, TX. al@logical-approach.com
Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT
OpenSolaris.Org Community Advisory Board (CAB) Member - Apr 2005
OpenSolaris Governing Board (OGB) Member - Feb 2006
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
377
From:
McLean, VA USA
Registered:
1/24/06
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 4:55 PM
in response to: alhopper
|
|
Al Hopper wrote:
>
> Now your accounting folks are going to be asking you to justify the
> purchase of that hi-end SAN box.... and why you're not using ZFS
> everywhere. :)
>
> Oh - and the accounting folks love it when you tell them there's no
> ongoing cost of ownership - because Joe Screwdriver can swap out a failed
> Seagate 500Gb SATA drive after he picks up a replacement from Frys on his
> lunch break!
Because ZFS doesn't run everywhere.
Because most low end JBODs are "low end" for a reason. They aren't as
reliable, have crappy monitoring, etc.
Fix those two things when you get a chance. ;)
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
803
From:
Plano, TX
Registered:
4/27/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 5:26 PM
in response to: tmcmahon
|
|
On Fri, 26 Jan 2007, Torrey McMahon wrote:
> Al Hopper wrote:
> >
> > Now your accounting folks are going to be asking you to justify the
> > purchase of that hi-end SAN box.... and why you're not using ZFS
> > everywhere. :)
> >
> > Oh - and the accounting folks love it when you tell them there's no
> > ongoing cost of ownership - because Joe Screwdriver can swap out a failed
> > Seagate 500Gb SATA drive after he picks up a replacement from Frys on his
> > lunch break!
>
> Because ZFS doesn't run everywhere.
> Because most low end JBODs are "low end" for a reason. They aren't as
> reliable, have crappy monitoring, etc.
Agreed. There will never be one screwdriver that fits everything. I was
simply trying to re-inforce my point.
> Fix those two things when you get a chance. ;)
Have a good weekend Torrey (and zfs-discuss).
Regards,
Al Hopper Logical Approach Inc, Plano, TX. al@logical-approach.com
Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT
OpenSolaris.Org Community Advisory Board (CAB) Member - Apr 2005
OpenSolaris Governing Board (OGB) Member - Feb 2006
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
377
From:
McLean, VA USA
Registered:
1/24/06
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 5:35 PM
in response to: alhopper
|
|
Al Hopper wrote:
> On Fri, 26 Jan 2007, Torrey McMahon wrote:
>
>
>> Al Hopper wrote:
>>
>>> Now your accounting folks are going to be asking you to justify the
>>> purchase of that hi-end SAN box.... and why you're not using ZFS
>>> everywhere. :)
>>>
>>> Oh - and the accounting folks love it when you tell them there's no
>>> ongoing cost of ownership - because Joe Screwdriver can swap out a failed
>>> Seagate 500Gb SATA drive after he picks up a replacement from Frys on his
>>> lunch break!
>>>
>> Because ZFS doesn't run everywhere.
>> Because most low end JBODs are "low end" for a reason. They aren't as
>> reliable, have crappy monitoring, etc.
>>
>
> Agreed. There will never be one screwdriver that fits everything. I was
> simply trying to re-inforce my point.
>
It's a good point. We just need to make sure we don't forget that part.
People love to pull email threads out of context....or google for that
matter. ;)
>
>> Fix those two things when you get a chance. ;)
>>
>
> Have a good weekend Torrey (and zfs-discuss).
Same to you Al. (and zfs-discuss).
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
651
From:
CA
Registered:
6/14/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 26, 2007 8:27 PM
in response to: alhopper
To: Communities » zfs » discuss
|
|
Al Hopper wrote:
> On Fri, 26 Jan 2007, Rainer Heilke wrote:
>
>>> So, if I was an enterprise, I'd be willing to keep
>>> enough empty LUNs
>>> available to facilitate at least the migration of
>>> one or more filesystems
>>> if not complete pools.
>
> .... reformatted ...
>
>> You might be, but don't be surprised when the Financials folks laugh you
>> out of their office. Large corporations do not make money by leaving
>> wads of cash lying around, and that's exactly what a few terabytes of
>> unused storage in a high-end SAN is. This is in addition to the laughter
>>
>
>
> But this is exactly where ZFS distrupts "Large corporations" thinking.
Yes and no. A corporation has a SAN for reasons that have been valid for years; you won't turn that ship around on a skating rink.
> You're talking about (for example) 2 terabytes on a high-end SAN which
> costs (what ?) per GB (including the capital cost of the hi-end SAN)
> versus a dual Opteron box with 12 * 500Gb SATA disk drives that gives you
> 5TB of storage for (in round numbers) a total of ~ $6k. And how much are
> your ongoing monthlies on that hi-end SAN box? (Don't answer) So - aside
> from the occasional use of the box for data migration, this ZFS "storage
> box" has 1,001 other uses. Pick any two (uses), based on your knowledge
> of big corporation thinking and its an easy sell to management.
>
> Now your accounting folks are going to be asking you to justify the
> purchase of that hi-end SAN box.... and why you're not using ZFS
> everywhere. :)
No, they're going to be asking me why I want to run a $400K server holding all of our inventory and financials data on a cheap piece of storage I picked up at Pa's Pizza Parlor and Computer Parts. There are values (real and imagined, perhaps) that a SAN offers. And, when the rest of the company is running on the SAN, why aren't you?
As a side-note, if your company has a mainframe (yes, they still exist!), when will ZFS run on it? We'll need the SAN for a while, yet.
>> generated by the comment that, "not a big deal if the Financials and HR
>> databases are offline for three days while we do the migration." Good
> Again - sounds like more "legacy" thinking. With multiple gigabit
> ethernet connections you can move terrabytes of information in a hour,
> instead of in 24-hours - using legacy tape systems etc. This can be
> easily handled during scheduled downtime.
If your company is graced with being able to cost-justify the rip-and-replace of the entire 100Mb network, more power to you. Someone has to pay for all of this, and good luck fobbing it all of on some client.
>> Sorry, this argument smacks a little too much of being out of touch with
>> the fiscal (and time) restrictions of working in a typical corporation,
>> as opposed to a well-funded research group.
>>
>> I hope I'm not sounding rude, but those of us working in medium to large
>> corporations simply do not have the money for such luxuries. Period.
> On the contrary - if you're not thinking ZFS, you're wasting a ton of IT
> $s and hurting the competitiveness of your business.
But you can't write off the investment of the old gear in six months and move on. I wish life worked like that, but it doesn't. At least, not where I work. :-(
> Regards,
>
> Al Hopper
Rainer
|
|
|
|
|
Posts:
112
From:
DE
Registered:
3/7/06
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 31, 2007 4:37 AM
in response to: rheilke
|
|
Hi,
I need to be a little bit more precise in how I formulate comments:
1. Yes, zpool remove is a desirable feature, no doubt about that.
2. Most of the cases where customers ask for "zpool remove" can be solved
with zfs send/receive or with zpool replace. Think Pareto's 80-20 rule.
2a. The cost of doing 2., including extra scratch storage space or scheduling
related work into planned downtimes is smaller than the cost of not using
ZFS at all.
2b. Even in the remaining 20% of cases (figuratively speaking, YMMV) where
zpool remove would be the only solution, I feel that the cost of
sacrificing the extra storage space that would have become available
through "zpool remove" is smaller than the cost of the project not
benefitting from the rest of ZFS' features.
3. Bottom line: Everybody wants zpool remove as early as possible, but IMHO
this is not an objective barrier to entry for ZFS.
Note my use of the word "objective". I do feel that we have to implement
zpool remove for subjective reasons, but that is a non technical matter.
Is this an agreeable summary of the situation?
Best regards,
Constantin
--
Constantin Gonzalez Sun Microsystems GmbH, Germany
Platform Technology Group, Client Solutions http://www.sun.de/
Tel.: +49 89/4 60 08-25 91 http://blogs.sun.com/constantin/
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
|
|
|
|
|
Posts:
651
From:
CA
Registered:
6/14/05
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Jan 31, 2007 1:34 PM
in response to: constant
To: Communities » zfs » discuss
|
|
Hello.
> 2. Most of the cases where customers ask for "zpool
> remove" can be solved
> with zfs send/receive or with zpool replace. Think
> Pareto's 80-20 rule.
This depends on where you define "most". In the cases I am looking at, I would have to disagree.
> 2a. The cost of doing 2., including extra scratch
> storage space or scheduling
> related work into planned downtimes is smaller
> than the cost of not using
> ZFS at all.
Not really. In a SAN environment, many of the ZFS features you list are either already in place, or irrelevant. As I commented to Al Hopper, if your company has a SAN, they're going to expect you to use it effectively. If your data is critical (and the last five years have seen a successful argument that SAN storage should be used for this data), then you aren't going to be able to convince management to use cheaper storage with ZFS any time soon. One other thing we've found is that "cheap storage", and by that I'm including an older SAN frame we have, doesn't have the cache and speed to keep up with the database usage on the ZPool. Performance sucks, and the developers and testers are getting uptight. Luckily our data shuffle should be done by Friday, and they'll all be back on the high-end SAN. So, combine these with the cost of high-end SAN storage, and you have a strong case for giving back unused space (and largely negating your 2b argument).
Without the ability to give back one LUN of a 5-LUN zpool, ZFS buys us two things:
1) it's faster (that includes admin, as well as performance)
2) I can create a storage pool with 2 million+ files/inodes.
The other features, while I see them as vital to other people's environments, aren't that big to ours, largely due to our SAN. The export/import will be good during lifecycle upgrades, though (ok, make that three things ;-).
I hope that helps clarify my arguments, as your statements clarified yours.
> Best regards,
> Constantin
Cheers,
Rainer
|
|
|
|
|
Posts:
26
From:
Registered:
7/24/06
|
|
|
|
Re: Re: Re: How much do we really want zpool remove?
Posted:
Feb 23, 2007 3:43 PM
in response to: rheilke
To: Communities » zfs » discuss
|
|
Actually, I'm using ZFS in a SAN environment often importing LUNS to save management overhead and make snapshots easily available, among other things. I would love zfs remove because it allows me, in conjunction with containers, to build up a single managable pool for a number of local host systems while prototyping real storage requirements. I'd love the ability to migrate a test container to new hardware and replicate the data with zfs send/receive, then be able to resize my local pool. For production migration I already use 1 pool per container then export/import to pool into other systems on the SAN for redundancy/upgrades. Another thing that would be hot would be the ability to swap in and out a slower/differently sized set of drives into a pool for a faster set of drives, without incurring downtime. I ran into a situation where I wanted to migrate data out to more spindles on smaller drives, which I had available. Had remove been available I could have simply added the new smaller drive mirrors into the pool then removed the larger drives and freed up their use with no downtime. As your dataset gets larger in production, this becomes more of a desirable feature.
|
|
|
|
|
