OpenSolaris

Discussions Communities Projects Download Source Browser

Home » OpenSolaris Forums » performance » discuss

Thread: FileBench Discuss

Welcome, Guest Help
Login Login
Guest Settings Guest Settings
Reply to this Thread Reply to this Thread Search Forum Search Forum Back to Thread List Back to Thread List

Permlink Replies: 20 - Last Post: Nov 2, 2006 7:48 PM by: genikims
rmc

Posts: 42
From:

Registered: 3/9/05
FileBench Discuss
Posted: Aug 9, 2005 3:57 PM

  Click to reply to this thread Reply

It's obviously the week of Open Source benchmarks :-) I've just added the FileBench page to the performance community, and links to the source and binary downloads.

http://opensolaris.org/os/community/performance/filebench

It's still early days for filebench; we are honing the workload list as we expand the number of comparisons we're doing.

We've been using this internally for NFS comparisons (nfs3 vs 4 and nfs vs SAN), and also for file system comparisons (ZFS, UFS, QFS et al).

My intention is to open up a results forum, so that we can share new findings are we gather data.

All comments are of course welcome!

Thanks,

Richard.

mpogue

Posts: 47
From: US

Registered: 5/12/05
Re: FileBench Discuss
Posted: Aug 9, 2005 4:13 PM   in response to: rmc

  Click to reply to this thread Reply

Nice.

I had some thoughts on this. I noticed that Filebench and libMicro were similar, but different. I think it would be a Good Idea to converge on a common output format.

For example, I like the 'tattle' header from libMicro, which tells me absolutely everything I need to know (hopefully) about the configuration that we ran on (we might have to add some stuff that filebench users need to know?). And, I like the % calculation, so I can see at a glance how much different the numbers really are. libMicro also uses shades of green and red to indicate how far off the numbers are.

Another possibility would be to recast the filebench tests *as* libMicro tests, to take advantage of the statistical convergence stuff that is in there (libMicro is pretty smart about minimizing the time it takes to come up with a statistically valid result).

Anyway, might be some opportunity for common code there.

Mike

klong

Posts: 2
From:

Registered: 8/10/05
Re: FileBench Discuss
Posted: Aug 11, 2005 10:02 AM   in response to: rmc

  Click to reply to this thread Reply

Its good to see a public discussion open up on FileBench. I've been
playing with it for a while, but would like to create a custom
workload. Since workload definition syntax is not (well) documented,
I've not been able to confidently build a custom workload.

Basically, what I'm trying to mimic is this environment:
- 60% reads, 40% writes, all random
- Most file sizes are either 5k or 15k in size
- Directory structure is similar to /somehashN/bucketN/files
- Buckets can hold up to 10,000 files

Can anyone provide any documentation on how that would be done?

rmc

Posts: 42
From:

Registered: 3/9/05
Re: FileBench Discuss
Posted: Aug 11, 2005 5:04 PM   in response to: klong

  Click to reply to this thread Reply

Hi Kevin,

I'd be happy to help. I had to make a few assumptions, but here's an example. I assumed that you want to create and write files that are on average 10k and read files which are the same size.

Here's a simple script which creates a set of files and then does 60% reads of those files, and 40% writes to another set of filenames:

<pre>
set $dir=/tmp
set $nfiles=1000
set $meandirwidth=10
set $filesize=10k
set $nthreads=2
set $meaniosize=128k

define fileset name=readfileset,path=$dir,size=$filesize,entries=$nfiles,dirwidth=$meandirwidt h,prealloc
define fileset name=writefileset,path=$dir,size=$filesize,entries=$nfiles,dirwidth=$meandirwid th

define process name=filereader,instances=1
{
thread name=filereaderthread,memsize=10m,instances=$nthreads
{
flowop createfile name=createfile1,filesetname=writefileset,fd=1
flowop writewholefile name=writewholefile1,iosize=$meaniosize,fd=1
flowop closefile name=closefile1,fd=1
flowop createfile name=createfile2,filesetname=writefileset,fd=1
flowop writewholefile name=writewholefile2,iosize=$meaniosize,fd=1
flowop closefile name=closefile2,fd=1
flowop openfile name=openfile3,filesetname=readfileset,fd=1
flowop readwholefile name=readfile3,fd=1
flowop closefile name=closefile3,fd=1
flowop openfile name=openfile4,filesetname=readfileset,fd=1
flowop readwholefile name=readfile4,fd=1
flowop closefile name=closefile4,fd=1
flowop openfile name=openfile5,filesetname=readfileset,fd=1
flowop readwholefile name=readfile5,fd=1
flowop closefile name=closefile5,fd=1
}
}
</pre>

The script defaults to 2 threads doing 2 create/writes then 3 open/read/close. If we add more threads, it will increase parallelism without changing the ratios.

When I run this, I see the correct 60r/40w mix.

<pre>
/opt/filebench/bin/filebench
filebench> load kevin
7353: 0.076: Kevin Version 1.0 personality successfully loaded
filebench> run 10
7353: 0.138: Fileset writefileset: 1000 files, avg dir = 10.0, avg depth = 3.0, mbytes=9
7353: 5.664: Removed any existing fileset writefileset in 6 seconds
7353: 5.665: Creating fileset writefileset...
7353: 5.698: Preallocated 0 of 1000 of fileset writefileset in 1 seconds
7353: 5.769: Fileset readfileset: 1000 files, avg dir = 10.0, avg depth = 3.0, mbytes=9
7353: 8.000: Removed any existing fileset readfileset in 3 seconds
7353: 8.001: Creating fileset readfileset...
7353: 17.070: Preallocated 1000 of 1000 of fileset readfileset in 10 seconds
7353: 17.072: Creating/pre-allocating files
7353: 17.072: Starting 1 filereader instances
7360: 18.081: Starting 2 filereaderthread threads
7353: 21.091: Running...
7353: 31.161: Run took 10 seconds...
7353: 31.918: Per-Operation Breakdown
closefile5 41ops/s 0.0mb/s 0.0ms/op 27us/op-cpu
readfile5 41ops/s 0.4mb/s 12.5ms/op 346us/op-cpu
openfile5 41ops/s 0.0mb/s 0.2ms/op 89us/op-cpu
closefile4 41ops/s 0.0mb/s 0.0ms/op 27us/op-cpu
readfile4 41ops/s 0.3mb/s 13.0ms/op 338us/op-cpu
openfile4 41ops/s 0.0mb/s 0.1ms/op 90us/op-cpu
closefile3 41ops/s 0.0mb/s 0.0ms/op 27us/op-cpu
readfile3 41ops/s 0.4mb/s 17.8ms/op 357us/op-cpu
openfile3 41ops/s 0.0mb/s 0.2ms/op 89us/op-cpu
closefile2 41ops/s 0.0mb/s 0.0ms/op 52us/op-cpu
writewholefile2 41ops/s 0.4mb/s 0.4ms/op 380us/op-cpu
createfile2 41ops/s 0.0mb/s 0.9ms/op 495us/op-cpu
closefile1 41ops/s 0.0mb/s 0.0ms/op 58us/op-cpu
writewholefile1 41ops/s 0.3mb/s 0.5ms/op 407us/op-cpu
createfile1 41ops/s 0.0mb/s 1.2ms/op 611us/op-cpu

7353: 31.993:
IO Summary: 6255 ops 612.8 ops/s, (123/82 r/w) 1.9mb/s, 838us cpu/op, 9.4ms latency
</pre>

Please let me know if this is what you were looking for?

Regards,

Richard.

rmc

Posts: 42
From:

Registered: 3/9/05
Re: FileBench Discuss
Posted: Aug 11, 2005 5:23 PM   in response to: rmc

  Click to reply to this thread Reply

Keep in mind that I set the default mean directory width to 10, rather than the 1000 you were looking for. The algorithm uses a gamma distribution by default, so setting it to 10 means there will be a range of directory widths, with a mean of 10. For example:

/filebench/readfileset
/filebench/readfileset/00000016
/filebench/readfileset/00000016/00000003
/filebench/readfileset/00000016/00000003/00000001
/filebench/readfileset/00000016/00000003/00000001/00000005
/filebench/readfileset/00000016/00000003/00000001/00000005/00000011
/filebench/readfileset/00000016/00000003/00000001/00000005/00000011/00000004< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000011/00000003< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000011/00000002< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000011/00000001< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010
/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000017< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000016< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000015< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000014< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000013< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000012< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000011< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000010< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000009< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000008< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000007< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000006< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000005< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000004< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000003< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000002< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000010/00000001< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000009
/filebench/readfileset/00000016/00000003/00000001/00000005/00000009/00000007< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000009/00000006< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000009/00000005< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000009/00000004< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000009/00000003< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000009/00000002< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000009/00000001< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000008
/filebench/readfileset/00000016/00000003/00000001/00000005/00000008/00000006< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000008/00000005< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000008/00000004< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000008/00000003< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000008/00000002< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000008/00000001< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007
/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000015< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000014< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000013< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000012< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000011< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000010< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000009< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000008< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000007< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000006< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000005< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000004< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000003< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000002< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000007/00000001< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000006
/filebench/readfileset/00000016/00000003/00000001/00000005/00000006/00000001< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000005
/filebench/readfileset/00000016/00000003/00000001/00000005/00000005/00000005< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000005/00000004< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000005/00000003< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000005/00000002< br />/filebench/readfileset/00000016/00000003/00000001/00000005/00000005/00000001< br />
The file sizes are also a gamma distribution by default, so setting the mean to 10k gives a range of sizes:

rmc@devnull src/filebench/kevin > ls -l /filebench/readfileset/00000016/00000003/00000001/00000005/00000010/
total 376
-rw------- 1 rmc staff 1106 Aug 11 16:59 00000001
-rw------- 1 rmc staff 11036 Aug 11 16:59 00000002
-rw------- 1 rmc staff 14718 Aug 11 16:59 00000003
-rw------- 1 rmc staff 6999 Aug 11 16:59 00000004
-rw------- 1 rmc staff 6252 Aug 11 16:59 00000005
-rw------- 1 rmc staff 11035 Aug 11 16:59 00000006
-rw------- 1 rmc staff 7714 Aug 11 16:59 00000007
-rw------- 1 rmc staff 12667 Aug 11 16:59 00000008
-rw------- 1 rmc staff 9941 Aug 11 16:59 00000009
-rw------- 1 rmc staff 7604 Aug 11 16:59 00000010
-rw------- 1 rmc staff 23295 Aug 11 16:59 00000011
-rw------- 1 rmc staff 12408 Aug 11 16:59 00000012
-rw------- 1 rmc staff 6522 Aug 11 16:59 00000013
-rw------- 1 rmc staff 1730 Aug 11 16:59 00000014
-rw------- 1 rmc staff 4157 Aug 11 16:59 00000015
-rw------- 1 rmc staff 15516 Aug 11 16:59 00000016
-rw------- 1 rmc staff 29808 Aug 11 16:59 00000017

Thanks,

Richard.

klong

Posts: 2
From:

Registered: 8/10/05
Re: FileBench Discuss
Posted: Aug 17, 2005 7:05 AM   in response to: rmc

  Click to reply to this thread Reply

Richard:

Thanks for creating this workload. I am having some problems with it, though.

When comparing it to a workload distributed with the package, some "exiting" lines show up following the "Running..." notification, and the workload run seems to abort prematurely. A run of "varmail", by comparison, does not show the "flowop exiting" lines and runs the duration specified on the in the "run" command.

I am using the current filebench package from Sourceforge on Sparc (v440):

klong{gmfstest}~/gmfs$ pkginfo -l filebench
PKGINST: filebench
NAME: FileBench
CATEGORY: application
ARCH: sparc,i386
VERSION: 12 Jul 05
BASEDIR: /opt
VENDOR: Richard Mc Dougall
DESC: FileBench
PSTAMP: 1.64.3_s10_x86_sparc_PRERELEASE


filebench> load kevin
4347: 2.279: syntax error, token expected on line 32
filebench> run 60
4347: 5.137: Fileset writefileset: 1000 files, avg dir = 0.0, avg depth = -0.0, mbytes=9
4347: 5.539: Removed any existing fileset writefileset in 1 seconds
4347: 5.540: Creating fileset writefileset...
4347: 5.550: Preallocated 0 of 1000 of fileset writefileset in 1 seconds
4347: 5.592: Fileset readfileset: 1000 files, avg dir = 0.0, avg depth = -0.0, mbytes=9
4347: 6.282: Removed any existing fileset readfileset in 1 seconds
4347: 6.282: Creating fileset readfileset...
4347: 14.580: Preallocated 1000 of 1000 of fileset readfileset in 9 seconds
4347: 14.580: Creating/pre-allocating files
4347: 14.580: Starting 1 filereader instances
4352: 15.589: Starting 2 filereaderthread threads
4347: 18.598: Running...
4352: 23.689: filereaderthread: exiting flowop createfile1-1
4352: 23.694: filereaderthread: exiting flowop createfile1-1
4347: 24.658: Run took 6 seconds...
4347: 24.669: Per-Operation Breakdown
closefile5 83ops/s 0.0mb/s 0.0ms/op 8us/op-cpu
readfile5 83ops/s 0.7mb/s 3.4ms/op 79us/op-cpu
openfile5 83ops/s 0.0mb/s 0.0ms/op 25us/op-cpu
closefile4 83ops/s 0.0mb/s 0.0ms/op 8us/op-cpu
readfile4 83ops/s 0.7mb/s 4.4ms/op 80us/op-cpu
openfile4 83ops/s 0.0mb/s 0.0ms/op 26us/op-cpu
closefile3 83ops/s 0.0mb/s 0.0ms/op 8us/op-cpu
readfile3 83ops/s 0.7mb/s 7.5ms/op 83us/op-cpu
openfile3 83ops/s 0.0mb/s 0.0ms/op 28us/op-cpu
closefile2 83ops/s 0.0mb/s 0.0ms/op 21us/op-cpu
writewholefile2 83ops/s 0.7mb/s 0.1ms/op 117us/op-cpu
createfile2 83ops/s 0.0mb/s 2.1ms/op 396us/op-cpu
closefile1 83ops/s 0.0mb/s 0.0ms/op 22us/op-cpu
writewholefile1 82ops/s 0.7mb/s 0.1ms/op 119us/op-cpu
createfile1 82ops/s 0.0mb/s 2.1ms/op 423us/op-cpu

4347: 24.670:
IO Summary: 7496 ops 1237.2 ops/s, (248/165 r/w) 3.8mb/s, 348us cpu/op, 4.0ms latency
4347: 24.670: Shutting down processes
filebench>

Message was edited by:
klong

Message was edited by:
klong

Edited heavily after finding one question answered later in the thread.

rmc

Posts: 42
From:

Registered: 3/9/05
Re: FileBench Discuss
Posted: Aug 17, 2005 10:44 AM   in response to: klong

  Click to reply to this thread Reply

Hi Kevin,

This workload is running out of files to create after 25 seconds, with the default set of 1000 files.

The creation phase is pre-defining a fileset of 1000 filenames (without actually creating them), then creating them during the run. So, for a longer run more filenames will be required.

Simply increasing the nfiles to a larger value (50000?) should provide enough files to complete the run...

Thanks,

Richard.

mauroj

Posts: 201
From:

Registered: 3/9/05
Re: FileBench Discuss
Posted: Aug 11, 2005 7:23 PM   in response to: klong

  Click to reply to this thread Reply

Hey Kevin - In addition to all the good stuff Richard provided, be aware that you need to set $USE_XANADU = 1; in bin/runbench to enable the Xanadu post-processing of the data.

In your profile (.prof file), set the function to generic_xanadu to enable additional statistics gathering;

# cat oltp-zfs1.prof

DEFAULTS {
runtime = 60;
dir = /zfs/zp2/zspace2;
stats = /filebench/tstats;
filesystem = zfs;
description = "ZFS ZPOOL2 8k";
}

CONFIG large_db_oltp_8k_uncached {
personality = oltp;
function = generic_xanadu;
cached = 0;
directio = 1;
iosize = 8k;
nshadows = 200;
ndbwriters = 10;
usermode = 20000;
filesize = 1g;
memperthread = 1m;
workingset = 0;
}

Note the function entry in the CONFIG section.
Then edit bin/runbench to set $USE_XANADU, and you'll get extra system statistics, nicely formatted for viewing in a WEB browser.

What happens is, when the run completes, in the stats directory (defined in the DEFAULTS section of the .prof file), you'll have a subdirectory with a long name (nodename-file system-prof file name-date and time). In that directory will be another directory with the name of the CONFIG (large_db_oltp_8k_uncached in the example above). In that directory is an html directory, with an index.html file, and a bunch of PNG bitmap files of graphs of all the collected statistics. Point a browser to the index.html file, and just click on the links to get the graphs.

It's very slick, and hugely simplifies the process of gathering and comparing results.

Please let us know if you have further questions.

/jim

jamesd

Posts: 316
From: Milwaukee WI

Registered: 4/27/05
Re: Re: FileBench Discuss
Posted: Aug 11, 2005 7:38 PM   in response to: mauroj

  Click to reply to this thread Reply



On 8/11/05, James Mauro <James dot Mauro at sun dot com> wrote:
Hey Kevin - In addition to all the good stuff Richard provided, be aware that you need to set $USE_XANADU = 1; in bin/runbench to enable the Xanadu post-processing of the data.

In your profile (.prof file), set the function to generic_xanadu to enable additional statistics gathering;

# cat oltp-zfs1.prof

DEFAULTS {
        runtime = 60;
        dir = /zfs/zp2/zspace2;
        stats = /filebench/tstats;
        filesyste m = zfs;
        description = "ZFS ZPOOL2 8k";
}

CONFIG large_db_oltp_8k_uncached {
        personality = oltp;
        function = generic_xanadu;
        cached = 0;
        directio = 1;
        iosize = 8k;
        nshadows = 200;
        ndbwriters = 10;
        usermode = 20000;
        filesize = 1g;
        memperthread = 1m;
        workingset = 0;
}

Note the function entry in the CONFIG section.
Then edit bin/runbench to set $USE_XANADU, and you'll get extra system statistics, nicely formatted for viewing in a WEB browser.

What happens is, when the run completes, in the stats directory (defined in the DEFAULTS section of the .prof file), you'll have a subdirectory with a long name (nodename-file system-prof file name-date and time). In that directory will be another directory with the name of the CONFIG (large_db_oltp_8k_uncached in the example above). In that directory is an html directory, with an index.html file, and a bunch of PNG bitmap files of graphs of all the collected statistics. Point a browser to the index.html file, and just click on the links to get the graphs.

It's very slick, and hugely simplifies the process of gathering and comparing results.

Please let us know if you have further questions.

TEASE you sun guys are only teasing us, you show us config information for a zfs filesystem but don't go ahead and show us the output of the commands. 

James Dickens
uadmin.blogspot.com


 

/jim
This message posted from opensolaris.org
_______________________________________________
perf-discuss mailing list
perf-discuss at opensolaris dot org

_______________________________________________ perf-discuss mailing list perf-discuss at opensolaris dot org

mauroj

Posts: 201
From:

Registered: 3/9/05
Re: FileBench Discuss
Posted: Aug 11, 2005 1:06 PM   in response to: rmc

  Click to reply to this thread Reply

I'd like to take this opportunity to post some general information on what I picked up recently while using filebench to generate some workloads for file system performance. This will be spread across several replies to this topic.

The filebench binaries (for SPARC and x86), as well as filebench source, are available on sourceforge (www.sourceforge.net/projects/filebench). This discussion, at least for now, will assume Solaris (either SPARC or x86) for discussions on directory and file names - all the other stuff will be OS independent. That is, full pathnames referencing the location of filebench files and directories assume a Solaris installation. All the other bits, such as running filebench, creating profiles, etc, is OS independent.

The filebench package I'm currently using is:

solaris10> pkginfo -l filebench
PKGINST: filebench
NAME: FileBench
CATEGORY: application
ARCH: sparc,i386
VERSION: 20 Jul 05
BASEDIR: /opt
VENDOR: Richard Mc Dougall
DESC: FileBench
PSTAMP: 1.64.5_s10_x86_sparc_PRERELEASE
INSTDATE: Aug 01 2005 14:32
EMAIL: Richard dot McDougall at Sun dot COM
STATUS: completely installed
FILES: 266 installed pathnames
6 linked files
28 directories
67 executables
2 setuid/setgid executables
33368 blocks used (approx)


First - a couple things to be aware of regarding the state of filebench and documentation. Aside from the output of "/opt/filebench/bin/filebench -h" (which is actually very usefull - try it!), there are two text files in filebench/docs,, README and README.benchpoint. That's pretty much it for filebench documentation for right now. As we work on fixing that, we will use this forum to share information and help close the documentation gap. We enccourage others to share what they have learned, and of course post questions and queries here.

The README.benchpoint file may cause some confusion. Benchpoint, so far as I can tell, is a generic name used to describe the filebench user environment and/or wrapper script (I think...I was not privy to this work, so I may be wrong). The important thing to note is that the README.benchpoint file refers to directories in the filebench hierarchy that do not exist - specifically, benchpoint/config. The current filebench distribution includes a config directory under /opt/filebench (/opt/filebench/config), which is where the profiles (.prof files) and function files (.func) reside. Also, there is a benchpoint script in /opt/filebench/bin, which simply informs the user that runbench should be used instead of benchpoint. So I think it's fair to assume that we can remove references to benchpoint in filebench, and focus on what the current distribution provides, and how to use it.

The top level filebench directory (/opt/filebench on Solaris) will contain 6 subdirectories.
bin - filebench command binaries.
config - preconfigued profiles (.prof) and function (.func) files, to get things started.
docs - README and README.benchpoint. More to follow.
scripts - various system stat gathering scripts, and other scripted components of the filebench framework.
workloads - configured flowop (.f) files, designed to simulate specific workloads.
xanadu - A framework for post-processing results data, including system statistics.

A few additional points need to be made on the workloads and xanadu directories...
The .f files in the workloads directory are files written in the 'f' language, which is a workload modeling language developed as part of the filebench framework. It is designed to allow for taking known attributes of workload, and representing them in a descriptive language that is than used to generate a load. The syntax, semantics and grammer rules for writing custom .f scripts is not yet documented. /opt/filebench/bin/filebench -h will output a summary of the 'f' language definition. That, coupled with reading existing .f scripts, is sometimes enough to begin creating your own custom workflows. But, we still need to get this documented.

The workloads in the workloads directory fall into one of two categories - micro and macro. The micro workloads are those prefixed with filemicro_XXX.f. As they name implies, these workloads represent lower-level file IO operations that are typically a subset of what a macro workload does. The macro workloads, such as oltp.f and varmail.f, use descriptive names to represent they type of workload they are designed to generate.

Xanadu is a framework developed in the Performance & Availability Engineering (PAE) group at Sun (the group Richard and I work for). It's a very slick set of tools designed to grab loads of benchmark-generated output (including the output of system stat tools like mpstat(1)), and provide summary tables and graphs of the data. Additionally, Xanadu is designed to make it easy to compare data from different benchmark runs. We will further explore using Xanadu in the next post.

With that, the information below was created by Richard McDougall. It is a cut-and-paste from Richard's internal (Sun internal) WEB site, and provides a great jumpstart to using filebench.
Installing

The workspace is at /home/rmc/ws/filebench-gate, the gate log is here, and a download of a filebench package (Filebench) here.

# zcat 1.28_s10_x86_sparc.tar.Z |tar xvf -
...
# pkgadd -d 1.28_s10_x86_sparc

The following packages are available:
1 filebench FileBench
(sparc,i386) 15 Oct 04

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]: 1

Processing package instance from

## Processing package information.
## Processing system information.
10 package pathnames are already properly installed.
## Verifying disk space requirements.
## Checking for conflicts with packages already installed.

Installing FileBench as

## Installing part 1 of 1.
/opt/filebench/README
/opt/filebench/README.benchpoint
/opt/filebench/bin/Comm.pl
/opt/filebench/bin/benchpoint
/opt/filebench/bin/i386/fastsu
/opt/filebench/bin/i386/filebench
/opt/filebench/bin/i386/gnuplot
/opt/filebench/bin/sparcv9/fastsu
/opt/filebench/bin/sparcv9/filebench
/opt/filebench/bin/sparcv9/gnuplot
/opt/filebench/runscripts/runem.sh
/opt/filebench/tools/collect_iostat
/opt/filebench/tools/collect_lockstat
/opt/filebench/tools/collect_vmstat
/opt/filebench/tools/filebench_plot
/opt/filebench/tools/filebench_summary
/opt/filebench/workloads/bringover.f
/opt/filebench/workloads/createfiles.f
/opt/filebench/workloads/deletefiles.f
/opt/filebench/workloads/fileserver.f
/opt/filebench/workloads/multistreamread.f
/opt/filebench/workloads/oltp.f
/opt/filebench/workloads/postmark.f
/opt/filebench/workloads/randomread.f
/opt/filebench/workloads/singlestreamread.f
/opt/filebench/workloads/varmail.f
/opt/filebench/workloads/webproxy.f
/opt/filebench/workloads/webserver.f
[ verifying class ]

Installation of was successful.

Getting Started Running Filebench

Example varmail run:

The workloads have been encapsulated in workload definition files
and the environment is now quite simple to drive.

There are simple workload personalities, which configure the type
of workload to simulate. An example is a /var/mail directory
simulation (like postmark):

$ /opt/filebench/bin/filebench
filebench> load varmail
8395: 3.898: Varmail personality successfully loaded
8395: 3.899: Usage: set $dir=<dir>
8395: 3.900: set $filesize=<size> defaults to 16384
8395: 3.900: set $nfiles=<value> defaults to 1000
8395: 3.901: set $dirwidth=<value> defaults to 20
8395: 3.901: set $nthreads=<value> defaults to 1
8395: 3.902: set $meaniosize=<value> defaults to 16384
8395: 3.902: run <runtime>
filebench> set $dir=/tmp
filebench> run 10
8395: 14.886: Fileset mailset: 1000 files, avg dir = 20, avg depth = 2.305865, mbytes=15
8395: 15.301: Preallocated fileset mailset in 1 seconds
8395: 15.301: Starting 1 filereader instances
8396: 16.313: Starting 1 filereaderthread threads
8395: 19.323: Running for 10 seconds...
8395: 29.333: Stats period = 10s
8395: 29.347: IO Summary: 21272 iops 2126.0 iops/s, (1063/1063 r/w) 32.1mb/s, 338us cpu/op, 0.3ms latency
8395: 29.348: Shutting down processes
filebench> stats dump stats.varmail
filebench> quit

This run did 21272 logical operations per second - e.g. 21272 open,
close, read/write etc...

$ more stats.varmail

Flowop totals:
closefile4 492ops/s 0.0mb/s 0.0ms/op 31us/op-cpu
readfile4 492ops/s 7.1mb/s 0.6ms/op 171us/op-cpu
openfile4 492ops/s 0.0mb/s 5.4ms/op 653us/op-cpu
closefile3 492ops/s 0.0mb/s 1.1ms/op 150us/op-cpu
appendfilerand3 492ops/s 7.7mb/s 0.0ms/op 55us/op-cpu
readfile3 492ops/s 7.2mb/s 0.6ms/op 163us/op-cpu
openfile3 492ops/s 0.0mb/s 5.1ms/op 623us/op-cpu
closefile2 492ops/s 0.0mb/s 0.9ms/op 136us/op-cpu
appendfilerand2 492ops/s 7.7mb/s 0.1ms/op 67us/op-cpu
createfile2 492ops/s 0.0mb/s 8.9ms/op 1151us/op-cpu
deletefile1 492ops/s 0.0mb/s 8.4ms/op 1007us/op-cpu

IO Summary: 649116 iops 5409.0 iops/s, 983/983 r/w 29.7mb/s, 1461uscpu/op


Running multiple workloads automatically

$ mkdir mybench
$ cd mybench
$ cp /opt/filebench/config/filemicro.prof mybench.prof
or
$ cp /opt/filebench/config/filemacro.prof mybench.prof

$ vi mybench.prof

set your stats dir to where you want the logs to go...

$ /opt/filebench/bin/benchpoint mybench

$ browse the index.html in the stats dir



Application-Emulation Workloads Currently Available in Filebench package

varmail
A /var/mail NFS mail server emaulation, following the workload of postmark, but multi-threaded. The workload consists of a multi-threaded set of open/read/close, open/append/close and deletes in a single directory.
fileserver
A file system workload, similar to SPECsfs. This workload performs a sequence of creates, deletes, appends, reads, writes and attribute operations on the file system. A configurable hierarchical directory structure is used for the file set.
oltp
A database emulator. This workload performance transactions into a filesystem using an I/O model from Oracle 9i. This workload tests for the performance of small random reads & writes, and is sensitive to the latency of moderate (128k+) synchronous writes as in the log file. It by default lanches 200 reader processes, 10 processes for asyncrhonous writing, and a log writer. The emulation includes use of ISM shared memory as per Oracle, Sybase etc which is critical to I/O efficiency (as_lock optimizations).
dss
DSS Database (not yet in pkg)
webserver
A mix of open/read/close of multiple files in a directory tree, plus a file append (to simulate the web log). 100 threads are used by default. 16k is appended to the weblog for every 10 reads.
webproxy
A mix of create/write/close, open/read/close, delete of multiple files in a directory tree, plus a file append (to simulate the proxy log). 100 threads are used by default. 16k is appended to the log for every 10 read/writes.

Micro-benchmark Workloads Currently Available in Filebench package

copyfiles
A copy of a large directory tree. This workload creates a hierarchical directory tree, then measures the rate at which files can be copied from the source tree to a new tree. A single thread is used by default, although this is configurable.
createfiles
Create a directory tree and fill files. A populate of files of specified sizes a directory tree. File sizes are chosen according to a gamma distribution of 1.5, with a mean size of 16k.
randomread
A multi-threaded read of a single large file, defaulting to 8k reads. A single thread is used by default, although configurable by $nthreads.
randomwrite
A multi-threaded write of a single large file, defaulting to 8k writes. A single thread is used by default, although configurable by $nthreads.
singlestreamread
A sequential read of a large file. 1MB reads are used by default.
singlestreamwrite
A sequential write of a large file. 1MB writes are used by default.
A sequential read of 4 large files, each with their own reader thread. 1MB reads are used by default.
multistreamwrite
A sequential write of 4 large files, each with their own writer thread. 1MB writes are used by default.

Recommended parameters

Recommended defaults for small configurations (1 disk):
For fileserver bringover createfiles deletefiles varmail webproxy webserver:
set $nfiles=50000 For randomread singlestreamread multistreamread singlestreamwrite multistreamwrite:
set $filesize=1g For oltp:
set $filesize=1g
Recommended defaults for large configurations (20+ disks):
For fileserver bringover createfiles deletefiles varmail webproxy webserver:
set $nfiles=100000 For randomread randomwrite
set $filesize=5g set $nthreads=256 For singlestreamread multistreamread singlestreamwrite multistreamwrite:
set $filesize=5g For oltp:
set $filesize=5g

Workload Model Language Example (Subject to lots of change at the moment)

#!/home/rmc/ws/filebench-rmc/src/filebench/filebench -f

debug 1

define file name=bigfile1,path=$datadir1/myrandfile,size=50g,prealloc,reuse,paralloc

define process name=rand-read,instances=1
{
thread name=rand-thread,memsize=10m,instances=100
{
flowop read name=rand-read1,filename=bigfile1,iosize=$iosize,random,directio
flowop eventlimit name=rand-rate
}
}


Workload Model Language Spec (Subject to lots of change at the moment)

Usage:
filebench: interpret f script and generate file workload
Options:
[-h] Display verbose help
[-p] Disable opening /proc to set uacct to enable truss

'f' language definition:

Variables:
set $var = value
$var - regular variables
${var} - internal special variables
$(var) - environment variables


define file name=<name>,path=<pathname>,size=<size>
[,paralloc]
[,prealloc]
[,reuse]

define fileset name=<name>,path=<pathname>,entries=<number>
[,dirwidth=[width]
[,dirgamma=[100-10000] (Gamma * 1000)
[,sizegamma=[100-10000] (Gamma * 1000)
[,prealloc=[percent]]

define process name=<name>[,instances=<count>]
{
thread ...
thread ...
thread ...
}


thread name=<name>[,instances=<count>]

{
flowop ...
flowop ...
flowop ...
}

flowop [aiowrite|write|read] name=<name>,
filename=<fileobjname>,
iosize=<size>
[,directio]
[,dsync]
[,iters=<count>]
[,random]
[,workingset=<size>]

flowop aiowait name=<name>,target=<aiowrite-flowop>

flowop sempost name=<name>,target=<semblock-flowop>,
value=<increment-to-post>

flowop semblock name=<name>,value=<decrement-to-receive>,
highwater=<inbound-queue-max>

flowop block name=<name>

flowop hog name=<name>,value=<number-of-mem-ops>

flowop wakeup name=<name>,target=<block-flowop>,

flowop eventlimit name=<name>
flowop bwlimit name=<name>

flowop [readfile|writefile] name=<name>,
filename=<fileobjname|filesetname>

Commands:

eventgen rate=<rate>
create [files|processes]
stats [clear|snap]
stats command "shell command $var1,$var2..."
stats directory <directory>
sleep <sleep-value>
quit

mauroj

Posts: 201
From:

Registered: 3/9/05
Re: FileBench Discuss
Posted: Aug 11, 2005 1:10 PM   in response to: mauroj

  Click to reply to this thread Reply

Please disregard the "workspace" reference in the previous post (right under 'Install').

I mistakenly included that in the cut from Richard's WEB site on getting started with filebench.

The sourceforge link is the correct place to go for the filebench package.

Sorry

mpogue

Posts: 47
From: US

Registered: 5/12/05
Re: FileBench Discuss
Posted: Aug 12, 2005 12:02 PM   in response to: rmc

  Click to reply to this thread Reply

Richard,

I'm trying to compile and run just the X86 version, without the Sparc version, since many in the community do not have Sparc boxen, and (like me) they'll want to run this just on X86. I got filebench to compile (and startup), but I can't seem to get it to build a package without the Sparc binaries present. I thought I might be able to get around this by doing a 'make install', a la the Linux instructions, but I'm having massive difficulties with 'install' (neither /usr/ucb/install, nor /usr/sbin/install seem work correctly, files in workloads/build seem to need write permission for root to overwrite them, etc.).

Do you perhaps have a version that will compile/run, without requiring all the Sparc stuff, or some suggestions on how to make this work?

Thanks,
Mike

palowoda

Posts: 352
From:

Registered: 5/1/05
Re: FileBench Discuss
Posted: Aug 12, 2005 2:37 PM   in response to: mpogue

  Click to reply to this thread Reply

> Richard,
>
> I'm trying to compile and run just the X86 version,
> n, without the Sparc version, since many in the
> community do not have Sparc boxen, and (like me)
> they'll want to run this just on X86. I got
> filebench to compile (and startup), but I can't seem
> to get it to build a package without the Sparc
> binaries present. I thought I might be able to get
> around this by doing a 'make install', a la the Linux
> instructions, but I'm having massive difficulties
> with 'install' (neither /usr/ucb/install, nor
> /usr/sbin/install seem work correctly, files in
> workloads/build seem to need write permission for
> root to overwrite them, etc.).

It's using the gnu version of install. Install the Companion CD,
it's in a package called SFWgfile.

Another way is to install the blastwave version of CSWgfile which
would be:
pkg-get -i common

Using the blastwave tools on x86 I did ./configure gmake install no errors.

---Bob




>
> Do you perhaps have a version that will compile/run,
> without requiring all the Sparc stuff, or some
> suggestions on how to make this work?
>
> Thanks,
> Mike

rmc

Posts: 42
From:

Registered: 3/9/05
Re: FileBench Discuss
Posted: Aug 12, 2005 3:04 PM   in response to: palowoda

  Click to reply to this thread Reply

Hi Mike, The binary package under downloads has both the SPARC and x86/64 versions compiled and built. You should be just able to download, pkgadd and run. Please let me know if this is still an issue? Thanks, Richard.

mpogue

Posts: 47
From: US

Registered: 5/12/05
Re: FileBench Discuss
Posted: Aug 12, 2005 3:57 PM   in response to: rmc

  Click to reply to this thread Reply

Richard,
Yep, I know there's a pre-built binary (and thank you much for that!)... but I think that to do open development on filebench (or anything else) in the community, it needs to be compilable using the resources that most people have.

So, I think that probably means: being able to compile with gcc (the compiler included in S10), as well as the SOS10 compiler (which is now freely available in binary form), and being able to compile without having a Sparc machine handy. That's why I attempted this, rather than just downloading the pre-built binaries -- because it's what most people will have to work with, when they decide to jump in and become contributors.

In my previous post, I detailed some of the bugs I encountered trying to do this. I think they're probably minor, and easily fixed (at least that's my guess, since I hacked around them, rather than attempt a real fix!).

rmc

Posts: 42
From:

Registered: 3/9/05
Re: FileBench Discuss
Posted: Aug 12, 2005 5:20 PM   in response to: mpogue

  Click to reply to this thread Reply

>but I think that to do open development on filebench (or anything else) in the community, it needs to be compilable using the resources that most people have.

I agree. Our initial porting priorities were Solaris in binary/package and Linux in buildable source. Getting the source to build on Solaris with the standard built-ins should be quite straight forward; happy to see the wrinkles with gcc etc get ironed out so we can put back the changes...

Thanks,

Richard.

mpogue

Posts: 47
From: US

Registered: 5/12/05
Re: FileBench Discuss
Posted: Aug 12, 2005 3:44 PM   in response to: palowoda

  Click to reply to this thread Reply

Using gmake seems to help it a lot! It's better now, but not 100% yet.

Here's what I've figured out so far:
1) I overrode MAKE to get gmake, on the configure line. This worked.
2) I can't seem to override CC and CFLAGS at all (overriding them on the ./configure line has no effect), so I had to manually edit configure (ooh...bad).
3) make install tries to execute this in workloads/build:
/usr/ucb/install -m 644 ./BUILD /opt/filebench/workloads/BUILD
Error message is:
install: ./BUILD: No such file or directory
This doesn't look right to me. BUILD is a make target, not something to be installed. Looks to me like BUILD is in workloads_DATA, so that it's run as a build target, but then when the install step runs, we're using workloads_DATA as a list of files, and BUILD doesn't exist as a file.
temporary workaround: touch BUILD after it's run as a target, but before the final install step.

So, here's the exact sequence of what I did to get it installed on X86, using the pre-installed compiler (gcc), and with no need for Sparc binaries):

[pre]
su
chmod 777 filebench/
cd filebench <-- I have the tar file in a directory called filebench, which must be writeable by us
tar xvf filebench-1.64-alpha-src.tar
cd filebench-1.64
vi configure <-- workaround for the inability to override CC and CFLAGS
[do a find on the string 'solaris\*i386',
change CC=cc to CC=gcc,
change CFLAGS="-g -xO3 -DYYDEBUG=1" to CFLAGS="-g -O2 -DYYDEBUG=1"]]] load varmail
filebench> run 10
[/pre]

Note: Even though it seems to run OK and it gives me some results, I am still getting an error message after load varmail, that looks like this:
...
[pre]
10091: 2.306: run runtime (e.g. run 60)
10091: 2.306: syntax error, token expected on line 51 <-- ERROR MESSAGE
[/p]

rmc

Posts: 42
From:

Registered: 3/9/05
Re: FileBench Discuss
Posted: Aug 12, 2005 4:22 PM   in response to: mpogue

  Click to reply to this thread Reply

>10091: 2.306: run runtime (e.g. run 60)
>10091: 2.306: syntax error, token expected on line 51 <-- ERROR MESSAGE
>Anybody have a clue on this one (or, is this normal?)?

This is something that crept in very recently when the Linux port was integrated. It's not supposed to be there, but has no side affects. I'll take a look as soon as I get a chance...

Thanks,

Richard.

genikims

Posts: 5
From: PA, USA

Registered: 11/1/06
Re: FileBench Discuss
Posted: Nov 1, 2006 8:38 PM   in response to: mpogue
To: Communities » performance » discuss
Cc: OpenSolaris » discuss
  Click to reply to this thread Reply

Richard,

I was also trying to comiple and run filebench on linux environment. But it still has massive problems such as gsl link problem, autoconf/makefiles and so on. Is there any simple way to install and run it on linux? Or is there any binary file to run it within linux? Please get us to the starting point on linux.

Thank you.
Young

Message was edited by:
genikims

mgerdts

Posts: 1,262
From: US

Registered: 8/5/05
Re: Re: FileBench Discuss
Posted: Nov 2, 2006 5:56 AM   in response to: genikims

  Click to reply to this thread Reply

On 11/1/06, Youngjae <youkim at cse dot psu dot edu> wrote:
> Richard,
>
> I was also trying to comiple and run filebench on linux environment. But it still has massive problems such as gsl link problem, autoconf/makefiles and so on. Is there any simple way to install and run it on linux? Or is there any binary file to run it within linux? Please get us to the starting point on linux.
>
> Thank you.

Some (all?) of your problems are likely addressed by the various
patches I have posted to
http://sourceforge.net/tracker/?group_id=133644&atid=727883

Mike

--
Mike Gerdts
http://mgerdts.blogspot.com/
_______________________________________________
perf-discuss mailing list
perf-discuss at opensolaris dot org



genikims

Posts: 5
From: PA, USA

Registered: 11/1/06
Re: Re: FileBench Discuss
Posted: Nov 2, 2006 7:48 PM   in response to: mgerdts
To: Communities » performance » discuss
  Click to reply to this thread Reply

Thanks for your feedback. But it's still not working. It's giving these errors;

./configure: line 1738: syntax error near unexpected token `filebench,'
./configure: line 1738: `AM_INIT_AUTOMAKE(filebench, 1.0)'
make: *** [config.status] Error 2

The environment running this is... linux kernel 2.6.18, gcc 4.1.1, and automake 1.9.6. I just guess something problem is in automake. Would you please give me some help?

Thank you.
Young




Terms of Use | Privacy | Trademarks | Copyright Policy | Site Guidelines
Your use of this web site or any of its content or software indicates your agreement to be bound by these Terms of Use.
Copyright © 1995-2005 Sun Microsystems, Inc.