[Python-Dev] Issues with Py3.1's new ipaddr

[Clay McClure]

On Tue, Jun 2, 2009 at 4:53 PM, R. David Murray <rdmurray at bitdance.com> wrote:

> Having thought more about this, I will agree with you that it would
> be useful to have an address-without-netmask class.

Not only useful, but necessary. It seems there are few people on this
list who understand IP well enough to realize how distorted ipaddr
actually is.

> (Note, however, that even a DNS entry can conceptually be considered to
> be a host route.)

Not at all. A host route and a host address (as represented in DNS)
are fundamentally different concepts. Please see my recent post to
ipaddr-py-dev for a refresher on these concepts:

http://groups.google.com/group/ipaddr-py-dev/t/94d54fe581d24e72

> So I'm not in favor of pulling ipaddr from 3.1, and it's too late in
> the release cycle to change anything.

I'm not sure why you say that when others have said that another
release candidate is planned, and that removing ipaddr is essentially
trivial to do.

> I wish you had brought this energy to bear earlier, when changes could
> have been made.  Reality is what it is, though, and now we should work
> on making improvements for the next release.  I see in the ticket that
> the netaddr folks were going to propose improvements so that they could
> build netaddr on top of ipaddr, but I guess that didn't happen (yet?).

I don't think that can happen, actually. If I was a netaddr committer
(which I'm not), I would find it hard to salvage anything reusable
from ipaddr. It is certainly simpler and clearer to start over with an
object model that actually makes sense.

> I have no association with Google, by the way, and I do intend to use
> ipaddr in upcoming code, and have hacked my own address manipulation
> stuff in previous code.

Sorry, I wasn't aware of that. My mistake. Regardless, I find that
your understanding of IP is similar to that of ipaddr's authors, which
is to say imprecise.

> I don't understand why you are trying to use ifconfig as an example.

Because it's an obvious real world example that explains why these two
strings are not equivalent:

192.168.1.1

and

192.168.1.1/32

You and others continue to suggest that those strings are equivalent,
yet ifconfig is a tool that has been around for thirty years that
clearly demonstrates that those strings are not equivalent. If what
you say is true, I should be able to pass either string to ifconfig
and get the same result. That is not the case, because the strings are
not equivalent, because a host route is not the same thing as a host
address.

> So this does the Right Thing:
>
>    myip = ipaddr.IP('192.168.1.1/26')
>    system('ifconfig eth0 {}'.format(myip))

Sure, but shouldn't this also do the right thing?

address = ipaddr.IP('192.168.1.1')
netmask = ipaddr.IP('255.255.255.192')
system("ifconfig eth0 %s/%s" % (address, netmask))

It doesn't.

> Hmm.  I think there is a conceptual divide here.  You have said you
> think about IP addresses and networks as separate objects, so I wonder
> if you would be pulling the netmask for ifconfig out of a separate
> network object?

Of course, because addresses don't have masks; networks do. This command:

ifconfig en0 192.168.1.1/24

is shorthand for operator convenience. What's going on behind the
scenes is quite a lot different than it looks. First, ifconfig
computes a network address by masking the supplied interface address
with the supplied network mask. It is then able to configure a route
for the proper network: 192.168.1.0/24.

The fact that "192.168.1.1/24" appears in the command does *not* mean
that the address 192.168.1.1 has a mask of /24. That is absurd.
Addresses don't have masks; networks do. That's why they're called
netmasks.

> On the other hand I, a network professional, think about an IP address
> paired with a netmask as a fundamental object.

The IT industry is unique among engineering disciplines in that formal
training and licensing are typically not required for IT
professionals. Whereas concepts like resistance, current, and voltage
have very specific meanings to electrical engineers, the IT vernacular
is not so precise. Since formal training is rare, and what little is
available is often high-priced and vendor-specific, IT professionals
tend to learn their trade from trade books, word of mouth, and
hands-on experience. As a result, IT professionals tend to have a good
working knowledge of how technology applies to their particular job,
but may not have an appreciation of the more theoretical aspects of
the discipline.

What this means in practice is that your experience as a network
professional may not resemble the experiences of other network
professionals. That you tend to think of addresses as having masks is
probably not universal, or even particularly common, among network
professionals. Some electrical engineers probably think of voltage as
pressure, and that may be a useful abstraction, but I would be
surprised to see a voltmeter calibrated in pascals.

What are we to do? How do we arrive at a common understanding of our
domain? We should consult the canonical sources of truth: RFC-791, and
the BSD IP implementation. In neither of those will you see that an IP
address has a mask.

If this were any other problem domain, I think it should be obvious
that the design of the library is flawed. But given that this is IP, a
subject that many people think they understand but actually don't, the
design flaws are obscured by ambiguity.

Guido's earlier comment about "political correctness" underscores this
point. This is not simply a case of me preferring my way of thinking
about IP addresses to the way the ipaddr authors think about IP
addresses. I'm simply stating a fact that, were it not true, the
Internet would not function: addresses and networks are not the same
thing. To represent them in the same class is a mistake.

> But I think both of
> those can be implemented fairly trivially as subclasses of the existing
> ipaddr objects.

Yes, I could certainly see BaseIPAddress and BaseIPNetwork classes
inheriting from BaseIP, with the ipaddr.IP function selectively return
an object of either type depending on what the user passes in the
constructor. If they include a mask, it's a network. If they don't,
it's an address. I think that particular change might be backwards
compatible, but I'm not sure. None the less, other changes that I
think are important (returning IP objects instead of strings and ints
for some properties; renaming poorly named methods) do change the API
and thus are not backwards compatible without adding lots of cruft.
Better to fix those now, in my opinion.

> In short, netaddr's object model does not match my desired model, while
> ipaddr's is a lot closer to my desired model.

I'm not advocating netaddr. The decision of which library to use has
already been made; I'm not debating that point. I'm merely suggesting
that we pull ipaddr from the release until such time that it can be
evolved to have a more agreeable API. ipaddr might be usable for you
(probably because your limited understanding of IP matches the ipaddr
developers'), but we've already heard from a handful of others on this
list who would rather roll their own library than suffer through the
quirks in ipaddr's current implementation.

Clay