|
Replies:
13
-
Last Post:
Mar 10, 2006 1:28 PM
by: darrenr
|
|
|
Posts:
42
From:
UK
Registered:
6/29/05
|
|
|
|
Clearview IP-Level Observability Devices:
updated design document
Posted:
Feb 8, 2006 3:29 AM
|
|
Hi,
I've incorporated all the design review comments I received and also
made some changes to the document. Of note is the addition of a /dev/lo0
device to provide consistency with other Unix variants. The revised
document can be found here:
http://opensolaris.org/os/community/networking/ipobs-design.pdf
If there are any comments on the revised document please let me know,
likewise if I have failed to incorporate some of your comments let me
know and I'll update the document accordingly.
Thank you you for all your input so far, it has been very valuable.
Phil
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
Posts:
2,060
From:
Registered:
6/8/05
|
|
|
|
Re: Clearview IP-Level Observability Devices:
updated design document
Posted:
Mar 6, 2006 2:33 PM
in response to: philk
|
|
Hi,
Firstly, in the requirements, numbers (2) and (3) read like this:
"2. From the global zone it must be possible to view all IP traffic on the machine. THis means all loopback IP traffic, traffic from remote machines, traffic sent from this machine, forwarded traffic and inter/intra-zone traffic."
"3. From a local zone, it must be possible to view all IP traffic sent to or from this zone, including traffic local to the zone. It must not be possible to see any other traffic."
These two requirements conflict with statements later in the design doc.
On page 4, the document reads:
"These devices will provide access to all packets with IP addresses local to the system which includes inter-zone and intra-zone traffic."
On page 5:
"Opening these devices will provide access to all IP packets with addresses associated with the interface."
The statements on 4 and 5 also conflict with the 2nd option for passing a packet to a consumer (but this does satisfy the requirements (2) & (3)):
"2. DL_PROMISC_PHYS is enabled and the interface was used for input or output of the packet to or from the link layer AND the consumer is in the global zone or in a non-global zone to which the packet is destined."
(that should probably be more than one sentence.)
In the table on page 6, for "(2) & !(1)", the comment for "Received" is misleading. For "(2)" to be satisfied, promiscuous mode must be set, so "(2) & !(1)" should simply be "Yes".
On page 7, this sentence is included:
"However, during the discussions it became clear that there were potential problems making the Hooks Framework generic at this time so we will implement our own specific hooks in ip."
Having been to PSARC a few times and got to know what they think about "hooks in IP", if I were PSARC reading this, it would be like waving a red flag in front of a bull. i.e. we need to change this story. It may be that the IP observability project would perhaps benefit more from PEF (packet event framework) to get packets than pfhooks. At some point in time, it is possible that pfhooks will use PEF as the means to provide events for some packets. But right now I can see one or both projects being told to go away and come back when you've fixed the story unless there is an impeccible reason for engineering it like this.
Darren
|
|
|
|
|
Posts:
6,813
From:
US
Registered:
3/9/05
|
|
|
|
Re: Re: Clearview IP-Level Observability Devices:
updated design document
Posted:
Mar 6, 2006 2:56 PM
in response to: darrenr
|
|
Darren Reed writes:
> "However, during the discussions it became clear that there were
> potential problems making the Hooks Framework generic at this time
> so we will implement our own specific hooks in ip."
Yeah, that's not pleasant. From an ARC point of view, there are
several issues to contend with:
- You're right that we don't want to see multiple implementations of
the same thing. However, it's also true that we don't want to
hold a project hostage to unsolved areas elsewhere in the system.
It's a bit of a tightrope walk to make that sort of judgement
call.
- If this project is right, and the Hooks Framework is not general
enough to solve this problem, then it's certainly fair to ask why
that should be so. It seems, at least at first blush, that
anything that can filter packets ought to be also functional
enough to simply _observe_ them. If that's not true, it'd be nice
to know why. This would likely end up as advice to management.
- It's also fair to ask that the project team go off and discuss the
issues with the other teams involved. Reasonable answers this
team could come back with would include, "that project's schedule
doesn't fit with our schedule, but after they integrate, we commit
to reworking ours to fit." But it'd be good to know the answers.
As things stand today, there is no one, generally-available, generic
framework for packet inspection. Every project to date places its own
hooks into IP, sometimes labels them as yet another "generic
solution," and then drives on. That alone is worth some advice.
> Having been to PSARC a few times and got to know what they think
> about "hooks in IP", if I were PSARC reading this, it would be like
> waving a red flag in front of a bull. i.e. we need to change this
> story. It may be that the IP observability project would perhaps
> benefit more from PEF (packet event framework) to get packets than
> pfhooks. At some point in time, it is possible that pfhooks will
> use PEF as the means to provide events for some packets. But right
> now I can see one or both projects being told to go away and come
> back when you've fixed the story unless there is an impeccible
> reason for engineering it like this.
Again from the ARC perspective, I don't think I'd ask such a thing.
PEF doesn't exist. There's no ARC documentation that describes it.
We could certainly ask the project team to go off and discuss the
issue and come back, but I can't see how we could ever demand that
this project depend on something that doesn't exist.
It's the PEF project's problem to deal with the system as it is when
(and if) that project comes forward for review and then integration.
Outside of the ARC, it's management's job to make sure the projects
are aligned as best they can. If we have multiple projects all
attempting to deliver the same thing into Solaris, then that's a
management problem.
The one thing we could reasonably ask about is dependencies between
this project and the IP Filter Hooks project. But if that latter one
doesn't have the features required for this one, and can't be reworked
to have them, then we've got a bit of a misfit. The choice then comes
down to applying force (force that'll likely cause this project to
fail to deliver) or letting it go with noses held and strong advice to
management. We often do the latter (sigh), but there's no way to
predict.
--
James Carlson, KISS Network <james dot d dot carlson at sun dot com>
Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
Posts:
619
From:
US
Registered:
3/9/05
|
|
|
|
Re: Re: Clearview IP-Level Observability Devices:
updated design document
Posted:
Mar 6, 2006 4:25 PM
in response to: carlsonj
|
|
James Carlson wrote:
> - If this project is right, and the Hooks Framework is not general
> enough to solve this problem, then it's certainly fair to ask why
> that should be so. It seems, at least at first blush, that
> anything that can filter packets ought to be also functional
> enough to simply _observe_ them. If that's not true, it'd be nice
> to know why. This would likely end up as advice to management.
My take is that there are two things going on here for the various
callouts from the IP code paths.
One is whether there is a small set of places where the callouts happen,
or whether each "function" ends up wanting access to packets at slightly
different places in the IP stack.
The second is whether we want all those callout locations to support the
richest semantics.
For the first item, for the set of callouts we have (IPsec policy
lookup/enforcement, IPsec application of encrypt/decrypt, IPQoS
classification, IPQoS queueing/dropping, IP Filter drop/allow, IP Filter
NAT, IP observability), is that there isn't a lot of commonality for
their placement; maybe a few of them need to be at exactly the same
place in the stack.)
Looking at the second item, there can be different environmental impact
statements for a hook in terms of the impact on the code in IP which
calls the hook. This ranges from the simplest to the hardest as:
- just do observability - some code which looks at the packet (and
perhaps copies it). IP just calls this hook and moves on.
- above + allow/drop. IP calls the hook and checks some return value
whether the packet was dropped or not.
- above + optional queuing. IP calls the hook, but also need to
provide some indication of where to resume processing when the packet is
dequeued.
- above + optional rewrite. On return (or resume after dequeue) the
packet headers might be different. This requires some re-evalution of
other parts (IRE lookup, re-classification).
I don't think it makes sense to have 7 generic transmit and 7 generic
receive hooks that can all do the most complex operation.
For instance, I think it makes more sense to have explicitly designed
hooks for a specific purpose. If we can place the rewrite capability
closest to the wire, then the impact of re-classification is minimized.
> As things stand today, there is no one, generally-available, generic
> framework for packet inspection. Every project to date places its own
> hooks into IP, sometimes labels them as yet another "generic
> solution," and then drives on. That alone is worth some advice.
The solution to that might be to not claim that any of them are generic.
If it turns out that two hooks end up in the same place, then we can
collapse them into one hook.
Erik
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
Posts:
6,813
From:
US
Registered:
3/9/05
|
|
|
|
Re: Re: Clearview IP-Level Observability Devices:
updated design document
Posted:
Mar 7, 2006 6:15 AM
in response to: nordmark
|
|
Erik Nordmark writes:
> For the first item, for the set of callouts we have (IPsec policy
> lookup/enforcement, IPsec application of encrypt/decrypt, IPQoS
> classification, IPQoS queueing/dropping, IP Filter drop/allow, IP Filter
> NAT, IP observability), is that there isn't a lot of commonality for
> their placement; maybe a few of them need to be at exactly the same
> place in the stack.)
That's certainly true.
> I don't think it makes sense to have 7 generic transmit and 7 generic
> receive hooks that can all do the most complex operation.
Nor do I, though, somewhat confusingly, it seems to be close to the
direction that other platforms are taking.
> > As things stand today, there is no one, generally-available, generic
> > framework for packet inspection. Every project to date places its own
> > hooks into IP, sometimes labels them as yet another "generic
> > solution," and then drives on. That alone is worth some advice.
>
> The solution to that might be to not claim that any of them are generic.
> If it turns out that two hooks end up in the same place, then we can
> collapse them into one hook.
That's exactly what I'd suggest. There's likely also a lot of
simplification available once the "one hook for all (or most or many)
users" design goal is dropped. It's no longer necessary, for
instance, to have elaborate registration mechanisms.
The things that need to be explained to the ARC members are why all of
the previous claims to generic hook features are unusable and why
generic hooks aren't even considered feasible. That's something I'd
hoped to do back when we were writing the filtering document, but that
essentially go no consensus and went nowhere.
Darren Reed writes:
> The biggest problem I can see with combining the two is that as a
> result of discussion over the design, I abandoned trying to resolve
> the problem of multiple consumers for the same hook. The result
> here would be not being able to use /dev/ipnet/* at the same time
> as IPFilter if both used the same hook rather than their own.
Is it a problem, though, that actually needs to be solved? I think
that's the unclear part for both projects.
> The root of that decision is trying to answer the question of
> "if both are enabled, who should see a packet first and how does
> one specify that preference or its inverse?"
>
> ...for which no good answer could be found :(
Yep.
--
James Carlson, KISS Network <james dot d dot carlson at sun dot com>
Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
Posts:
619
From:
US
Registered:
3/9/05
|
|
|
|
Re: Re: Clearview IP-Level Observability Devices:
updated design document
Posted:
Mar 7, 2006 1:59 PM
in response to: carlsonj
|
|
James Carlson wrote:
> The things that need to be explained to the ARC members are why all of
> the previous claims to generic hook features are unusable and why
> generic hooks aren't even considered feasible. That's something I'd
> hoped to do back when we were writing the filtering document, but that
> essentially go no consensus and went nowhere.
Jim and Darren,
If there is a place where I can help to clarify this (in a meeting, some
slides, some text in a document), please let me know and I'll contribute.
> Darren Reed writes:
>> The biggest problem I can see with combining the two is that as a
>> result of discussion over the design, I abandoned trying to resolve
>> the problem of multiple consumers for the same hook. The result
>> here would be not being able to use /dev/ipnet/* at the same time
>> as IPFilter if both used the same hook rather than their own.
>
> Is it a problem, though, that actually needs to be solved? I think
> that's the unclear part for both projects.
If we have some targeted hook somewhere which can do queuing and/or
rewriting of the headers, then I think that the order matters.
But I think this can be addressed by explicitly making such hooks single
consumer - not allowing multiple parties to register.
Should there be hooks that are for observability only, or even those
that can do pass/drop, then there isn't much of an issue with the order.
Thus such targeted hooks can potentially allow multiple registrants.
Erik
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
Posts:
3,422
From:
Austin, TX, USA
Registered:
6/15/05
|
|
|
|
Re: Re: Clearview IP-Level Observability
Devices: updated design document
Posted:
Mar 7, 2006 2:06 PM
in response to: nordmark
|
|
On Tue, Mar 07, 2006 at 01:59:08PM -0800, Erik Nordmark wrote:
> If we have some targeted hook somewhere which can do queuing and/or
> rewriting of the headers, then I think that the order matters.
> But I think this can be addressed by explicitly making such hooks single
> consumer - not allowing multiple parties to register.
>
> Should there be hooks that are for observability only, or even those
> that can do pass/drop, then there isn't much of an issue with the order.
> Thus such targeted hooks can potentially allow multiple registrants.
There should be a single re-writer, and at worst "around" hooks to
observe packets before and after re-writes (someone's bound to want
that). No?
Nico
--
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
Posts:
6,813
From:
US
Registered:
3/9/05
|
|
|
|
Re: Re: Clearview IP-Level Observability Devices:
updated design document
Posted:
Mar 7, 2006 2:13 PM
in response to: nico
|
|
Nicolas Williams writes:
> There should be a single re-writer, and at worst "around" hooks to
> observe packets before and after re-writes (someone's bound to want
> that). No?
Should that single re-writer be NAT or IPsec ESP?
--
James Carlson, KISS Network <james dot d dot carlson at sun dot com>
Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
Posts:
3,422
From:
Austin, TX, USA
Registered:
6/15/05
|
|
|
|
Re: Re: Clearview IP-Level
Observability Devices: updated design document
Posted:
Mar 7, 2006 2:26 PM
in response to: carlsonj
|
|
On Tue, Mar 07, 2006 at 05:13:24PM -0500, James Carlson wrote:
> Nicolas Williams writes:
> > There should be a single re-writer, and at worst "around" hooks to
> > observe packets before and after re-writes (someone's bound to want
> > that). No?
>
> Should that single re-writer be NAT or IPsec ESP?
I can see NAT as something that plugs into hooks; hooking ESP/AH in
seems less appropriate, but maybe that's just my sense of aesthetics
(NAT sucks; IPsec is almost a fundamental component of the IP
architecture).
In any case, order should only matter amongst re-writers/queuers. To
really push observability we should have hooks around, not just before
or after, re-write points.
Am I interested in snooping packets as they come off the wire? Or posts
IPsec processing? Why not both? That way I get to confirm that ESP/AH
are being used.
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
Posts:
1
From:
Registered:
3/9/05
|
|
|
|
Re: Re: Clearview IP-Level Observability Devices:
updated design document
Posted:
Mar 7, 2006 6:13 PM
in response to: nordmark
|
|
Erik Nordmark wrote On 2006年03月08日 05:59,:
<pre wrap="">James Carlson wrote:
</pre>
<pre wrap="">The things that need to be explained to the ARC members are why all of
the previous claims to generic hook features are unusable and why
generic hooks aren't even considered feasible. That's something I'd
hoped to do back when we were writing the filtering document, but that
essentially go no consensus and went nowhere.
</pre>
<pre wrap=""><!---->
Jim and Darren,
If there is a place where I can help to clarify this (in a meeting, some
slides, some text in a document), please let me know and I'll contribute.</pre>
</blockquote>
<br>
I recall Darren actually has worked on a doc in analyzing why the
previous generic hook features are unusable. <br>
For one, IPPF design doc specifically mentioned that the design has
NOT been consulted the firewall requirement<br>
due to resource/time constraint. Maybe that doc should be appended to
this design doc under reviewing. <br>
<br>
Darren?<br>
<br>
-Kevin<br>
<br>
<blockquote type="cite" cite="mid440E022C dot 5080307 at sun dot com">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">Darren Reed writes:
</pre>
<blockquote type="cite">
<pre wrap="">The biggest problem I can see with combining the two is that as a
result of discussion over the design, I abandoned trying to resolve
the problem of multiple consumers for the same hook. The result
here would be not being able to use /dev/ipnet/* at the same time
as IPFilter if both used the same hook rather than their own.
</pre>
</blockquote>
<pre wrap="">Is it a problem, though, that actually needs to be solved? I think
that's the unclear part for both projects.
</pre>
</blockquote>
<pre wrap=""><!---->
If we have some targeted hook somewhere which can do queuing and/or
rewriting of the headers, then I think that the order matters.
But I think this can be addressed by explicitly making such hooks single
consumer - not allowing multiple parties to register.
Should there be hooks that are for observability only, or even those
that can do pass/drop, then there isn't much of an issue with the order.
Thus such targeted hooks can potentially allow multiple registrants.
Erik
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
</pre>
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
Posts:
2,060
From:
Registered:
6/8/05
|
|
|
|
Re: Re: Clearview IP-Level Observability Devices:
updated design document
Posted:
Mar 7, 2006 3:00 PM
in response to: carlsonj
|
|
James Carlson wrote:
...
>The things that need to be explained to the ARC members are why all of
>the previous claims to generic hook features are unusable and why
>generic hooks aren't even considered feasible. That's something I'd
>hoped to do back when we were writing the filtering document, but that
>essentially go no consensus and went nowhere.
>
I'm not sure how many others than the hooks for ip_policy() there
are, as I didn't see any obvious ones in reading through the code,
but I can say that this set of "hooks" were (a) badly placed and
(b) too complex/overweight. I'm not even sure how they believed
they were in any position to come even close to accurately metering
traffic as it definately does not see packets as they go to or appear
on the wire (and not even all.) I could go on a lot more but I think
it's just a waste of time, unless someone really wants me to disect
it in detail. I think more than an appropriate amount of time has
already been spent on it.
>Darren Reed writes:
>
>>The biggest problem I can see with combining the two is that as a
>>result of discussion over the design, I abandoned trying to resolve
>>the problem of multiple consumers for the same hook. The result
>>here would be not being able to use /dev/ipnet/* at the same time
>>as IPFilter if both used the same hook rather than their own.
>>
>
>Is it a problem, though, that actually needs to be solved? I think
>that's the unclear part for both projects.
>
If you're just a consumer of "observability" events (all the data
passed through is 'read-only') then the order doesn't really matter,
except for local timestamping.
When the data can change, it becomes a problem...and so just because
two hooks are in the same place does not imply they can be collapsed
into one if there isn't any understanding of who sees what first if one
of the hooks allows for data to be modified.
Maybe allowing multiple callbacks for those that will change the packet
does make it overly complex and needlessly so. Linux requires this
because they have NAT and filtering split into seperate modules, rather
than all part of the one.
As the Solaris TCP/IP gradually moves from STREAMS to something
else, whatever is surplanting the module stacking needs to allow for
ordering to be defined and observed. That said, there are few cases
when this is done and the only one I'm aware of is when people want
to use IPFilter with Sun Cluster (unsupported config) and have the
autopush file with both pfil and the Sun cluster module listed.
Darren
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
Posts:
2,060
From:
Registered:
6/8/05
|
|
|
|
Re: Re: Clearview IP-Level Observability Devices:
updated design document
Posted:
Mar 6, 2006 4:28 PM
in response to: carlsonj
|
|
James Carlson wrote:
>Darren Reed writes:
>
>>"However, during the discussions it became clear that there were
>>potential problems making the Hooks Framework generic at this time
>>so we will implement our own specific hooks in ip."
>>
>
>Yeah, that's not pleasant. From an ARC point of view, there are
>several issues to contend with:
>
> - You're right that we don't want to see multiple implementations of
> the same thing. However, it's also true that we don't want to
> hold a project hostage to unsolved areas elsewhere in the system.
> It's a bit of a tightrope walk to make that sort of judgement
> call.
>
> - If this project is right, and the Hooks Framework is not general
> enough to solve this problem, then it's certainly fair to ask why
> that should be so. It seems, at least at first blush, that
> anything that can filter packets ought to be also functional
> enough to simply _observe_ them. If that's not true, it'd be nice
> to know why. This would likely end up as advice to management.
>
Reading the documentation, the hooks framework is intended to be
used but they feel there is a need to add another, different hook
than those being proposed for the purpose of filtering packets.
So my quandry is should something be done to address this?
The biggest problem I can see with combining the two is that as a
result of discussion over the design, I abandoned trying to resolve
the problem of multiple consumers for the same hook. The result
here would be not being able to use /dev/ipnet/* at the same time
as IPFilter if both used the same hook rather than their own.
The root of that decision is trying to answer the question of
"if both are enabled, who should see a packet first and how does
one specify that preference or its inverse?"
...for which no good answer could be found :(
...
>Again from the ARC perspective, I don't think I'd ask such a thing.
>PEF doesn't exist.
>
ick :(
Darren
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
Posts:
42
From:
UK
Registered:
6/29/05
|
|
|
|
Re: Re: Clearview IP-Level Observability Devices:
updated design document
Posted:
Mar 10, 2006 4:32 AM
in response to: darrenr
|
|
Hi Darren,
Thanks for the comments.
>These two requirements conflict with statements later in the design doc.
>On page 4, the document reads:
If I understand this comment are you saying some there needs to be some
additional text to explain although the devices will provide access to
all IP
traffic there are restrictions depending on whether you are in the global or
a non-global zone?
>(that should probably be more than one sentence.)
I'll have a look to see if there's a clean way of splitting it.
>In the table on page 6, for "(2) & !(1)", the comment for "Received"
is misleading. For "(2)" to be satisfied, promiscuous mode must >be
set, so "(2) & !(1)" should simply be "Yes".
Ok.
>Reading the documentation, the hooks framework is intended to be
>used but they feel there is a need to add another, different hook
>than those being proposed for the purpose of filtering packets.
As you know we were going to use the hooks framework and we discussed
this but
then the problem of multiple consumers came up and as you say no good answer
could be found. Given this we (the Clearview team) agreed that it would
best to
implement our own hooks. I agree it would be great if PEF existed or
pfhooks was
a generic framework this isn't the case. I think the thing that really
needs some
detail is why pfhooks couldn't be made generic at this time and I can
certainly
add something to the document, and the PSARC case, to cover this.
Thanks
Phil
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
Posts:
2,060
From:
Registered:
6/8/05
|
|
|
|
Re: Re: Clearview IP-Level Observability Devices:
updated design document
Posted:
Mar 10, 2006 1:28 PM
in response to: philk
|
|
Philip Kirk - Solaris Sustaining wrote:
> Hi Darren,
>
> Thanks for the comments.
>
> >These two requirements conflict with statements later in the design
> doc.
> >On page 4, the document reads:
>
> If I understand this comment are you saying some there needs to be some
> additional text to explain although the devices will provide access to
> all IP
> traffic there are restrictions depending on whether you are in the
> global or
> a non-global zone?
No. In three places you talk about what packets are received as a result
of snoop'ing on /dev/ipnet/bge0. The description of what packets are
available as a result of this is not consistent across all three.
Darren
_______________________________________________
networking-discuss mailing list
networking-discuss at opensolaris dot org
|
|
|
|
|
