[C++-sig] Re: Adding __len__ to range objects

[Joel de Guzman]

Raoul Gough <RaoulGough at yahoo.co.uk> wrote:
> "Joel de Guzman" <djowel at gmx.co.uk> writes:
> 
> I see - so the slicing had to be factored out, because you can't slice
> an associative container. In theory you could use lower_bound and
> upper_bound to do some kind of slicing using the key type rather than
> positional indexing. 

Yes, that occured to me.

> The Python mapping types don't seem to support
> this, though (I guess the Python dictionary type is hash-based, so it
> couldn't easily support this idea). 

Exactly. 

> Actually, that's a whole other
> kettle of fish, looking at the Python mapping functions!
> 
> Well, I wonder about taking the same approach further, e.g. with any
> iterator range extensions - it seems like a lot of complexity involved
> in *removing* expected functionality. Maybe it would be possible to
> split the base class into separate functional units which can be
> recombined as appropriate, rather than having an all-in-one base
> class?

That's a possibility, yes. We can contain the functionality into a couple
of protocol groups. Making the suite finer-grained, but not too much.
Can you think of a nice way to group the existing and future functionality?
As it is, the protocols I outlined (proxyability, slicability, resizability and
mutability) are somewhat implementation-centric.

> I used something like that approach in my iterator_range extensions
> (at http://home.clara.net/raoulgough/boost/). Basically, I use the MPL
> to choose what def() functions to call, and rely on the compiler not
> instantiating unused templated functions. e.g.

[snip]

> This only does the class_.def() for __len__ when the iterator is (at
> least) a forward_iterator. The unspecialized maybe_add_len has an
> empty apply function, so if the <true> specialization is not used:
> 
> 1. In Python, len() automatically produces "TypeError: len() of
>    unsized object"
> 
> 2. distance<Range> is never used (i.e. its address is never taken) and
>    won't even be instantiated.
> 
> This could be extended using a traits class that defined things like
> has_len, has_slicing, has_append, etc...

In some cases, hard-coding the enabling/disabling of features to types 
is not desireable. Sometimes, for example, you don't want proxying for
vectors, even if it can. I opt for more user control over which is and which
is not enabled.

>> You can see the factored out code in <indexing_suite_detail.hpp>
>> (see no_proxy_helper, proxy_helper, slice_helper and
>> no_slice_helper).  The indexing suite chooses the appropriate
>> handler based on some flags and the types involved. The same system
>> will need to be set in place to support resizability and mutability.
> 
> Could you explain the role of the element proxy vectors (proxy_group)?
> I wrote a test case which echoes construction and destruction of
> contained elements to stdout, and it looks to me like the elements are
> being copied even when the proxy is in use (i.e. a copy happens if I
> do "print vector[0]" from Python). Initially, I thought the proxy was
> some way to avoid extra copying of UDTs, but I guess that isn't it.

Following Pythion semantics, basically, we want this:

    val = c[i]
    c[i].m()
    val == c[i]

where m is a non-const (mutating) member function (method). Yet, we
also want this:

    val = c[i]
    del c[i]
    # do something with val

No dangling references == no return_internal_reference


A proxy is basically a smart-pointer that has a reference to a container
and an index. When *attached*, dereferencing the proxy will index the 
container. A proxy can be detached from the container. In the *detached* 
state, it holds the actual object which it returns when dereferenced. A proxy
will be detached whenever you remove or replace that element from the 
container (e.g. del C[I], C[0:I] = [a,b,c], C[I] = x).

The proxy_group manages the proxies and is responsible for detaching
a proxy appropriately when necessary.

>> Is it feasible? Of course ;-)
> 
> I thought the same thing about extending iterator_range to support len
> and getitem, and look where that got me! (I tried three separate
> implementations before arriving at the current one). Maybe it was
> *too* feasible :-)

Haha! ;-) 

-- 
Joel de Guzman
http://www.boost-consulting.com
http://spirit.sf.net