ANN: a mini-language for encapsulating deep-copy operations on Python data structures

[Steve Howell]

On Nov 18, 4:34 am, "M.-A. Lemburg" <m... at egenix.com> wrote:
> Steve Howell wrote:
> > [...]
> > Here is an example of the DDL (and I hate the terminology "DDL," just
> > cannot think of anything better):
>
> >             {
> >                 'show_table_of_contents',
> >                 'author' {
> >                     .person 'name',
> >                     .person 'location' as city,
> >                     .favorite_books()[
> >                         .title,
> >                         .cost() as expense
> >                         ] as books}
> >             }
>
> > There are more details here:
>
> >http://showellonprogramming.blogspot.com/2009/11/more-on-python-deep-...

Thanks for your feedback, Marc-Andre!  Comments inline...

>
> Personally, I find the explicit approach more intuitive, since
> you immediately see what you are going to get:
>
> show_table_of_contents = context['show_table_of_contents']
> author = context['author']
> d = {
>     'show_table_of_contents': show_table_of_contents,
>     'author': {
>         'name': author.person['name'],
>         'city': author.person['location'],
>         'books': [{
>             'title': item.title,
>             'expense': item.cost(),
>             }
>             for item in author.favorite_books()],
>         },
>     }
>

I understand what you mean about the explicit approach.  When you go
for brevity, you sacrifice some explicitness. One of the things I'm
actually looking to do next is to be able to generate code like what
you've written above.  You could make the generated code even more
procedural in terms of how it walks the structure--for example,
replace the list comprehension with a for loop.  This could be useful
for inserting tracing code, for example.

> I particularly find this part non-intuitive:
>
> >                     .favorite_books()[
> >                         .title,
> >                         .cost() as expense
> >                         ] as books}
>

Yep, I think what I really want is this:

                    .favorite_books()[{
                        .title,
                        .cost() as expense
                        }] as books}

I am not sure the curlies make the transformation completely
intuitive, but they are more suggestive that you are generating a list
of dictionaries.

> and would leave in the square brackets for __getitem__
> lookups on these:
>
> >                     .person 'name',
> >                     .person 'location' as city,
>

Yep, I agree.  The square brackets were not a deliberate omission.
They were just slightly more tricky to implement because of '[' having
double syntactical meaning.

> For additional inspiration, you might want to look at XSLT
> which provides similar transformations on XML data structures.
>
> There are also a number of other transformation languages:
>
> http://en.wikipedia.org/wiki/Model_Transformation_Languagehttp://en.wikipedia.org/wiki/Data_transformation
>

Thanks for the links!

> Regarding the term "DDL": that's normally used for "data definition
> language" and doesn't really have all that much to do with
> transforming data. You normally define data structures using
> DDL - without actually putting data into those structures.
>
> Why not "PyDTL".... Python data transformation language ?!
>

Yep, I like that suggestion.  The language does indeed describe a
transformation.  In some ways I was wanting to think of it more
broadly, in terms that the transformation also acts as a schema for
any input objects.  For example, you could use '{.title, .publisher,
{.name} }' as a schema to validate that an object behaves like a
book.  I suppose that's still a transformation in an abstract sense,
where the output is just True, False, or maybe even the slice of the
transformation that can/cannot be executed.