|
Replies:
5
-
Last Post:
Nov 15, 2007 8:21 AM
by: taylor
|
|
|
Posts:
6
From:
Registered:
10/22/07
|
|
|
|
Should the iSCSI Initiator use the Nagle Algorithm by default
Posted:
Oct 24, 2007 11:07 AM
To: Communities » storage » discuss
|
|
I recently posted about a serious performance issue when running a single thread against my Solaris iSCSI Software Initiator connected volume. The cause of this performance issue is due to the Nagle algorithm being used for the network traffic sent to and from the iSCSI target. Simply disabling the Nagle algorithm solved the problem.
Easy enough fix, but it raised a question. If having the Nagle algorithm enabled could cause this serious a performance degradation (300k/sec I get with it enabled, 40MB/sec with it disabled), should it be disabled by default?
I guess I would be curious as to the reason it is enabled by default. Does the performance advantage on some targets outweigh the performance degradation seen when it is enabled for other targets? 300k/sec is pretty dreadful performance, and for most applications it could be considered unusable. Granted my testing was to the raw device, but any application that accesses the raw device directly is also affected by this (think Solaris Volume Manager, and I wonder how ZFS would fair, I can test if necessary).
How would I go about filing a request to change the default to not use the Nagle algorithm?
Patrick
|
|
|
Posts:
191
From:
US
Registered:
6/25/05
|
|
|
|
Re: Should the iSCSI Initiator use the
Nagle Algorithm by default
Posted:
Oct 24, 2007 2:02 PM
in response to: psully
|
|
On Oct 24, 2007, at 14:07, Patrick Sullivan wrote:
> I recently posted about a serious performance issue when running a
> single thread against my Solaris iSCSI Software Initiator connected
> volume. The cause of this performance issue is due to the Nagle
> algorithm being used for the network traffic sent to and from the
> iSCSI target. Simply disabling the Nagle algorithm solved the
> problem.
>
> Easy enough fix, but it raised a question. If having the Nagle
> algorithm enabled could cause this serious a performance
> degradation (300k/sec I get with it enabled, 40MB/sec with it
> disabled), should it be disabled by default?
>
> I guess I would be curious as to the reason it is enabled by
> default. Does the performance advantage on some targets outweigh
> the performance degradation seen when it is enabled for other
> targets? 300k/sec is pretty dreadful performance, and for most
> applications it could be considered unusable. Granted my testing
> was to the raw device, but any application that accesses the raw
> device directly is also affected by this (think Solaris Volume
> Manager, and I wonder how ZFS would fair, I can test if necessary).
>
> How would I go about filing a request to change the default to not
> use the Nagle algorithm?
File an RFE here:
http://www.opensolaris.org/bug/report.jspa
pick the latest nevada release
---
.je
_______________________________________________
storage-discuss mailing list
storage-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
|
|
|
|
|
Posts:
416
From:
US
Registered:
5/12/06
|
|
|
|
Re: Should the iSCSI Initiator use the Nagle
Algorithm by default
Posted:
Oct 24, 2007 2:48 PM
in response to: jone
|
|
I did some reading on this. Nagle was originally trying to stem the flood of 'tinygrams' sent during lightweight network sessions such as telnet (where single bytes (keystrokes) were being given their own packet, and generating lots of overhead). It's my impression that since many of us are using iSCSI to move large files, we aren't likely to see the algorithm affect us in the way previously described. Is this perhaps why the algorithm is enabled by default?
Blake On 10/24/07, Jonathan Edwards <Jonathan dot Edwards at sun dot com> wrote:
On Oct 24, 2007, at 14:07, Patrick Sullivan wrote:
> I recently posted about a serious performance issue when running a
> single thread against my Solaris iSCSI Software Initiator connected
> volume. The cause of this performance issue is due to the Nagle
> algorithm being used for the network traffic sent to and from the
> iSCSI target. Simply disabling the Nagle algorithm solved the
> problem.
>
> Easy enough fix, but it raised a question. If having the Nagle
> algorithm enabled could cause this serious a performance
> degradation (300k/sec I get with it enabled, 40MB/sec with it
> disabled), should it be disabled by default?
>
> I guess I would be curious as to the reason it is enabled by
> default. Does the performance advantage on some targets outweigh
> the performance degradation seen when it is enabled for other
> targets? 300k/sec is pretty dreadful performance, and for most
> applications it could be considered unusable. Granted my testing
> was to the raw device, but any application that accesses the raw
> device directly is also affected by this (think Solaris Volume
> Manager, and I wonder how ZFS would fair, I can test if necessary).
>
> How would I go about filing a request to change the default to not
> use the Nagle algorithm?
File an RFE here:
http://www.opensolaris.org/bug/report.jspa
pick the latest nevada release
---
.je
______________________________________________ _
storage-discuss mailing list
storage-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
|
|
|
|
|
Posts:
307
From:
US
Registered:
3/9/05
|
|
|
|
Re: Should the iSCSI Initiator use the Nagle
Algorithm by default
Posted:
Oct 25, 2007 5:08 AM
in response to: bcirvin
|
|
Blake wrote: I did some reading on this. Nagle was originally trying to stem the flood of 'tinygrams' sent during lightweight network sessions such as telnet (where single bytes (keystrokes) were being given their own packet, and generating lots of overhead). It's my impression that since many of us are using iSCSI to move large files, we aren't likely to see the algorithm affect us in the way previously described. Is this perhaps why the algorithm is enabled by default?
Please take a look at the node where the iSCSI Initiator is running and determine if the following parameter is being set.
[ /kernel/drv/iscsi.conf ] add the following: # # Disable Nagle within the iSCSI Initiator # tcp-nodelay=1;
The tcp-nodelay setting, alters the iSCSI Initiators socket options by setting the well-known option TCP_NODELAY, or more specifically turning off the "Nagel Algorithm". For performance reasons, this setting is hard-coded in the iSCSI Target and also Availability Suite's SNDR, both being TCP/IP based data movers of disk blocks.
For both iSCSI and SNDR, the Nagel algorithm interacts badly when using TCP delayed acknowledgments, of which both iSCSI and SNDR relies upon. These performance issues have been noted by many others, including the author John Nagel. See link at bottom of this page: http://en.wikipedia.org/wiki/Nagle's_algorithm
Blake
On 10/24/07, Jonathan Edwards <Jonathan dot Edwards at sun dot com> wrote:
On Oct 24, 2007, at 14:07, Patrick Sullivan wrote:
> I recently posted about a serious performance issue when running a
> single thread against my Solaris iSCSI Software Initiator connected
> volume. The cause of this performance issue is due to the Nagle
> algorithm being used for the network traffic sent to and from the
> iSCSI target. Simply disabling the Nagle algorithm solved the
> problem.
>
> Easy enough fix, but it raised a question. If having the Nagle
> algorithm enabled could cause this serious a performance
> degradation (300k/sec I get with it enabled, 40MB/sec with it
> disabled), should it be disabled by default?
>
> I guess I would be curious as to the reason it is enabled by
> default. Does the performance advantage on some targets outweigh
> the performance degradation seen when it is enabled for other
> targets? 300k/sec is pretty dreadful performance, and for most
> applications it could be considered unusable. Granted my testing
> was to the raw device, but any application that accesses the raw
> device directly is also affected by this (think Solaris Volume
> Manager, and I wonder how ZFS would fair, I can test if necessary).
>
> How would I go about filing a request to change the default to not
> use the Nagle algorithm?
File an RFE here:
http://www.opensolaris.org/bug/report.jspa
pick the latest nevada release
---
.je
______________________________________________ _
storage-discuss mailing list
storage-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
_______________________________________________ storage-discuss mailing list
< BR>
_______________________________________________
storage-discuss mailing list
storage-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
|
|
|
|
|
Posts:
191
From:
US
Registered:
6/25/05
|
|
|
|
Re: Should the iSCSI Initiator use the Nagle
Algorithm by default
Posted:
Oct 26, 2007 12:24 PM
in response to: jdunham
|
|
Hi Jim
On Oct 25, 2007, at 08:08, you wrote:
> Please take a look at the node where the iSCSI Initiator is running
> and determine if the following parameter is being set.
>
> [ /kernel/drv/iscsi.conf ] add the following:
> #
> # Disable Nagle within the iSCSI Initiator
> #
> tcp-nodelay=1;
Is there a quick way to probe w/ mdb to see if this is set?
tia
---
.je
_______________________________________________
storage-discuss mailing list
storage-discuss at opensolaris dot org
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
|
|
|
|
|
Posts:
32
From:
US
Registered:
3/9/05
|
|
|
|
Re: Should the iSCSI Initiator use the Nagle
Algorithm by default
Posted:
Nov 15, 2007 8:21 AM
in response to: jone
To: Communities » storage » discuss
|
|
The easiest way is to run "prtconf -v".
- Eric
|
|
|
|
|
