|
Replies:
5
-
Last Post:
Jun 17, 2005 8:29 PM
by: esaxe
|
|
|
|
|
|
|
Wait I/O Calculations
Posted:
Jun 16, 2005 2:59 PM
|
|
Howdy,
This question has been asked in numerous forums, but I can't seem to find
a definitive answer (please forward me to documentation/BLOG postings if
a definitive answer exists). I am trying to find out which actions are
directly responsible for attributing to the "%wio" column in vmstat. In
Jim Mauro's article on the WIO topic:
http://sunsite.uakom.sk/sunworldonline/swol-08-1997/swol-08-insidesolaris.html
Jim Mentions that the following structure is maintained for each CPU:
typedef struct cpu_syswait {
int iowait; /* procs waiting for block I/O */
int swap; /* XXX: UNUSED */
int physio; /* XXX: UNUSED */
} cpu_syswait_t;
And is incremented/decremented by the biowait() routine. The article also
mentions that skew occurs when the iowait variable is incremented just
prior to the clock interrupt firing, since the kenel could have been
in another state for the majority of the clock cycle. Based on this
information, how reliable is the value reported by the %wio column on a
Solaris 9 host? Are there any other items besides disk and tape
reads/writes that account for wio? Does micro state accounting make this
value more reliable ( I assume so, but assumptions aren't definitive )?
Thanks for any insight,
- Ryan
_______________________________________________
perf-discuss mailing list
perf-discuss at opensolaris dot org
|
|
|
Posts:
205
From:
Registered:
3/9/05
|
|
|
|
Re: Wait I/O Calculations
Posted:
Jun 16, 2005 4:18 PM
in response to: matty91@gmail.com
|
|
%wio is a meaningless statistic. On Solaris 9, each thread that was blocked in biowait(), waiting for I/O, would lead to a counter being incremented. When the I/O completed, the counter was decremented. This counter was what was used to generate %wio. The problem is that we attempted to normalize the I/O-wait as portion of CPU time, when it is really the count of threads sleeping for block-I/O. Attempting to decompose the number of threads blocked for I/O into a percentage of CPU-time-used turns out to be both confusing and not particularly useful.
Becuase %wio has lead to so much confusion, and didn't represent any real quantity of CPU time, we've removed the statistic in Solaris 10. %wio will now always return zero. If you're interested more useful I/O statistics, check out iostat(1M), which will correctly show you the percentage of time each I/O transaction spends waiting to be serviced.
|
|
|
|
|
Posts:
198
From:
Shanghai
Registered:
6/14/05
|
|
|
|
Re: Re: Wait I/O Calculations
Posted:
Jun 16, 2005 5:25 PM
in response to: johansen
|
|
On 6/16/05, johansen <johansen at sun dot com> wrote:
%wio
is a meaningless statistic. On Solaris 9, each thread that
was blocked in biowait(), waiting for I/O, would lead to a counter
being incremented. When the I/O completed, the counter was
decremented. This counter was what was used to generate
%wio.
This is how AIX keeps tracking I/O wait count too.
The
problem is that we attempted to normalize the I/O-wait as portion of
CPU time, when it is really the count of threads sleeping for
block-I/O. Attempting to decompose the number of threads
blocked for I/O into a percentage of CPU-time-used turns out to be both
confusing and not particularly useful.
True. AIX charges CPU time as WIO time if
a) CPU is idle (no runnable thread)
_and_
b) wait count (number of threads waiting in bio) is greater than 0 ( doesn't matter how many are waiting )
Now, the result is confusing too.
A user would see 100% I/O wait in vmstat report when the CPU is idle and there's only one thread waiting for I/O,
he would also see 0% wait when there're 100 threads waiting for I/O but just one thread keeps the CPU busy at the same time,
but which situation is more I/O-bound?
A _lot_ of AIX users are confused with this vmstat wio value.
Becuase
%wio has lead to so much confusion, and didn't represent any real
quantity of CPU time, we've removed the statistic in Solaris
10. %wio will now always return zero.
That's a surprise.
If
you're interested more useful I/O statistics, check out iostat(1M),
which will correctly show you the percentage of time each I/O
transaction spends waiting to be serviced.
This message posted from opensolaris.org
Ultimately, when there is a real I/O problem, you would
want to know when each I/O (size, location) starts and ends (response
time) at _each_ layer.
For example:
read/write quest -> pagefault_start -> LVM -> disk/path -> FC adapter
pagefault_end <-
The problem can exist in any of this layers.
This can be done by analyzing AIX kernel trace ( I have a pretty big
shell/awk script to generate detailed report from the trace ).
Obviously you can use DTrace in Solaris, you just have to know all the
corresponding probes and record entry and return timestamp.
I've been thinking about using DTrace to create "raw" report files
similar to the AIX trace report so that I can use my script on Solaris
also.
Hope I will have time to do it soon.
When I have question about the probes, I will ask here.
Tao
_______________________________________________
perf-discuss mailing list
perf-discuss at opensolaris dot org
|
|
|
|
|
Alexander Kolba...
akolb@eng.sun.com
|
|
|
|
Re: Re: Wait I/O Calculations
Posted:
Jun 16, 2005 5:45 PM
in response to: taochen
|
|
> Ultimately, when there is a real I/O problem, you would want to know when
> each I/O (size, location) starts and ends (response time) at _each_ layer.
> For example:
> read/write quest -> pagefault_start -> LVM -> disk/path -> FC adapter
> pagefault_end <-
>
> The problem can exist in any of this layers.
>
> This can be done by analyzing AIX kernel trace ( I have a pretty big
> shell/awk script to generate detailed report from the trace ).
> Obviously you can use DTrace in Solaris, you just have to know all the
> corresponding probes and record entry and return timestamp.
>
> I've been thinking about using DTrace to create "raw" report files similar
> to the AIX trace report so that I can use my script on Solaris also.
> Hope I will have time to do it soon.
> When I have question about the probes, I will ask here.
DTrace provides IO provider:
http://docs.sun.com/app/docs/doc/817-6223/6mlkidllc?q=dtrace&a=view>
- Alexander Kolbasov
http://blogs.sun.com/akolb
_______________________________________________
perf-discuss mailing list
perf-discuss at opensolaris dot org
|
|
|
|
|
|
|
|
|
Re: Re: Wait I/O Calculations
Posted:
Jun 16, 2005 7:55 PM
in response to: johansen
|
|
On Thu, 16 Jun 2005, johansen wrote:
> %wio is a meaningless statistic. On Solaris 9, each thread that was blocked in biowait(), waiting for I/O, would lead to a counter being incremented. When the I/O completed, the counter was decremented. This counter was what was used to generate %wio. The problem is that we attempted to normalize the I/O-wait as portion of CPU time, when it is really the count of threads sleeping for block-I/O. Attempting to decompose the number of threads blocked for I/O into a percentage of CPU-time-used turns out to be both confusing and not particularly useful.
>
> Becuase %wio has lead to so much confusion, and didn't represent any real quantity of CPU time, we've removed the statistic in Solaris 10. %wio will now always return zero. If you're interested more useful I/O statistics, check out iostat(1M), which will correctly show you the percentage of time each I/O transaction spends waiting to be serviced.
> This message posted from opensolaris.org
Thanks for the reply. I accidentally mentioned "%wio" and vmstat, when I
meant to say "wi" and mpstat.
Thanks again for the info!!
- Ryan
> _______________________________________________
> perf-discuss mailing list
> perf-discuss at opensolaris dot org
>
_______________________________________________
perf-discuss mailing list
perf-discuss at opensolaris dot org
|
|
|
|
|
Posts:
196
From:
US
Registered:
3/9/05
|
|
|
|
Re: Wait I/O Calculations
Posted:
Jun 17, 2005 8:29 PM
in response to: matty91@gmail.com
|
|
mpstat(1M) really is a strange place to report I/O wait time. Those last 4 columns are supposed to give a breakdown as to where a CPU is spending it's time. Trouble is, waiting for I/O isn't something the CPU spends it's time doing. Notice in
biowait() that the current thread gives up the CPU (blocks) while waiting for I/O completion via the call to sema_p().
The CPU then goes on to do something else. Looking at I/O wait time from a thread's context (or even from a device's context), rather than the CPU's context makes more sense.
This DTrace script does exactly that.
Hope that helps...
-Eric
|
|
|
|
|
