An idiom for code generation with exec

[eliben]

On Jun 20, 9:17 am, Bruno Desthuilliers <bruno.
42.desthuilli... at websiteburo.invalid> wrote:
> eliben a écrit :> Hello,
>
> > In a Python program I'm writing I need to dynamically generate
> > functions[*]
>
> (snip)
>
> > [*] I know that each time a code generation question comes up people
> > suggest that there's a better way to achieve this, without using exec,
> > eval, etc.
>
> Just to make things clear: you do know that you can dynamically build
> functions without exec, do you ?
>

Yes, but the other options for doing so are significantly less
flexible than exec.

> > But in my case, for reasons too long to fully lay out, I
> > really need to generate non-trivial functions with a lot of hard-coded
> > actions for performance.
>
> Just out of curiousity : could you tell a bit more about your use case
> and what makes a simple closure not an option ?

Okay.

I work in the field of embedded programming, and one of the main uses
I have for Python (and previously Perl) is writing GUIs for
controlling embedded systems. The communication protocols are usually
ad-hoc messages (headear, footer, data, crc) built on top of serial
communication (RS232).

The packets that arrive have a known format. For example (YAMLish
syntax):

packet_length: 10
fields:
  - name: header
    offset: 0
    length: 1
  - name: time_tag
    offset: 1
    length: 1
    transform: val * 2048
    units: ms
  - name: counter
    offset: 2
    length: 4
    bytes-msb-first: true
  - name: bitmask
    offset: 6
    length: 1
    bit_from: 0
    bit_to: 5
...

This is a partial capability display. Fields have defined offsets and
lengths, can be only several bits long, can have defined
transformations and units for convenient display.

I have a program that should receive such packets from the serial port
and display their contents in tabular form. I want the user to be able
to specify the format of his packets in a file similar to above.

Now, in previous versions of this code, written in Perl, I found out
that the procedure of extracting field values from packets is very
inefficient. I've rewritten it using a dynamically generated procedure
for each field, that does hard coded access to its data. For example:

def get_counter(packet):
  data = packet[2:6]
  data.reverse()
  return data

This gave me a huge speedup, because each field now had its specific
function sitting in a dict that quickly extracted the field's data
from a given packet.

Now I'm rewriting this program in Python and am wondering about the
idiomatic way to use exec (in Perl, eval() replaces both eval and exec
of Python).

Eli