[Python-ideas] IntFlags

Fri Mar 6 10:42:51 CET 2015

On Fri, Mar 6, 2015 at 4:28 AM, Andrew Barnert <abarnert at yahoo.com> wrote:

> On Mar 5, 2015, at 20:26, Neil Girdhar <mistersheik at gmail.com> wrote:
>
> Even if you constrain yourself to the BitFlags rather than the more
> general BitFields, I strongly disagree with the interface that people are
> proposing involving & and ~ operators.  In general, good interface design
> reflects the way we think about objects — not their underlying
> representation.
>
>
> But sometimes the object really is "an integer used as a set of bits in
> some C structure/protocol field/well-known API".
>

You can always get that integer by casting to integer.

>
> For example, if we were designing os.open or mmap or whatever as a
> Pythonic interface, it wouldn't have a "flags" value that or's together
> multiple integers. We'd probably have separate keyword-only arguments for
> the less common flags, etc. But they weren't designed from scratch; they
> were designed to closely mirror the POSIX APIs. And that doesn't just mean
> simpler implementation, it means people who are familiar with those APIs
> know how to use them. It means the vast volumes of tutorials and sample
> code for opening file handles or mapping memory written for C applies to
> Python. And so on. So, the interface makes sense.
>

I disagree that there is any need to follow the style of the "vast volumes
of tutorials and sample code in C" when designing Python libraries.  The
goal is for the Python code to be as natural as possible.  Member access,
and building constants using | are natural.  Using &~ to clear a bit is not
natural;  It is a coincidence of implementation that distracts from what is
happening.

>
> So, one very good use for something like IntFlags is to allow people to
> keep using that C sample code (with trivial, easy-to-understand changes),
> but get better debugging, etc. when they do so--e.g., when you introspect
> an mmap object, it would be great if it could tell you that it was opened
> with PROT_READ | PROT_EXEC, instead of telling you "3", which you have to
> manually convert to bits and reverse-lookup in the docs or the module dict.
>

Yes, totally agree.

>
> Not allowing people to use C-style operations if they use named bits means
> that someone who wants the advantages of named bits has to rewrite their
> familiar C-style code. Sure, maybe the result will be more readable
> (although that's arguable; the suggested alternatives are pretty
> verbose--especially since people keep suggesting mutating-only APIs...),
> but it means many people will stick with plain ints rather than rewrite,
> and those who do rewrite will end up with code that doesn't look like the
> familiar code that everyone knows how to read from C.
>

I totally agree with you that there should not only be mutating-only
functions.  I agree that | should be used for comining bit fields or
flags.  However, the people who are "familiar with C" (including me) are
frankly dying :)  Pandering to the past really gets you nowhere.  Try to be
a bit idealistic so that new Python code is natural, succinct, and
human-readable — rather than the C values of reflecting the underlying
representation in spite of the human being.

>   The fact is that a BitSet's main operations are set and clear individual
> bits.  It is as if the BitFlags are a namedtuple with Boolean elements
> whose underlying storage happens to be an integer.
>
>
> In the case where you don't really care that the underlying storage is an
> integer, why use an integer in the first place? Why not use a namedtuple,
> or a set, or whatever else is appropriate? In the very rare case where you
> need to store a million of these things (and can't store them even more
> compactly with array or NumPy or similar), you can go get a third-party
> lib; the vast majority of the time, there's no advantage to using an
> integer.
>

The main reason is so that you can cast it to "int" and produce something
that some API requires.

>
> Except, of course, when the underlying representation is the whole point,
> because you're dealing with an API that's written in terms of integers.
>
> right.

>   Therefore, the interface that makes the most sense is member access:
>
> my_bit_flags.some_bit = True
> my_bit_flags.some_bit = False
>
> I don't see the justification for writing these as
>
> my_bit_flags |= TheBitFlagsClass.some_bit
> my_bit_flags &= ~TheBitFlagsClass.some_bit
>
> The second line is particularly terrible because it exposes you to making
> mistakes like:
>
> my_bit_flags &= TheBitFlagsClass.some_bit
> my_bit_flags |= ~TheBitFlagsClass.some_bit
>
> — both of which are meaningless.
>
>
> No they're not. Put some real names instead of toy names there:
>
>     readable = m.prot
>     readable &= ProtFlags.Readable
>
> Now it's true iff m.prot includes the Readable flag.
>
> Of course usually you'd write this in a single line without mutation:
>
>     readable = m.prot & ProtFlags.Readable
>

We both know that the most readable version is just member access, like you
would on any object:

readable = m.prot.readable

This usage of & to filter is unnecessarily complicated.  The fact that the
machine does so is no reason for the programmer to write it so.

>
> But that just goes to show that the primary interface of bit flags is an
> immutable one; trying to force people to use mutating methods like set_bit
> and clear_bit is just getting in people's way. (And try to come up with a
> good name for the non-mutating operation that's obvious and reads like
> English and isn't approaching the ridiculous Apple level of verbosity you
> get in Cocoa methods like "bitSetWithBitClear:".)
>

I agree with you here.  I think you should also have | so that you can
build constants the way you're used to, although I'm not sure about & since
I don't see when you would use it in preference to member access.

> It also makes it hard to convert code between the alternate implementation
> of using a namedtuple.  It should be easy to do that in my opinion.
>
>

>
> Best,
>
> Neil
>
> On Thu, Mar 5, 2015 at 12:57 PM, Serhiy Storchaka <storchaka at gmail.com>
> wrote:
>
>> On 05.03.15 19:29, Neil Girdhar wrote:
>>
>>> Have you looked at my IntFields generalization of IntFlags?  It seems
>>> that many of your examples (permissions, e.g.) are better expressed with
>>> fields than with flags.
>>>
>>
>> It looks too complicated for such simple case. And it has an interface
>> incompatible with plain int.
>>
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>> --
>>
>> --- You received this message because you are subscribed to a topic in
>> the Google Groups "python-ideas" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/
>> topic/python-ideas/L5KfCEXFaII/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> python-ideas+unsubscribe at googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150306/5860385f/attachment-0001.html>