An idiom for code generation with exec
Bruno Desthuilliers
bruno.42.desthuilliers at websiteburo.invalid
Fri Jun 20 12:04:22 EDT 2008
eliben a écrit :
> On Jun 20, 9:17 am, Bruno Desthuilliers <bruno.
> 42.desthuilli... at websiteburo.invalid> wrote:
>> eliben a écrit :> Hello,
>>
>>> In a Python program I'm writing I need to dynamically generate
>>> functions[*]
>> (snip)
>>
>>> [*] I know that each time a code generation question comes up people
>>> suggest that there's a better way to achieve this, without using exec,
>>> eval, etc.
>> Just to make things clear: you do know that you can dynamically build
>> functions without exec, do you ?
>>
>
> Yes, but the other options for doing so are significantly less
> flexible than exec.
Let's see...
>>> But in my case, for reasons too long to fully lay out, I
>>> really need to generate non-trivial functions with a lot of hard-coded
>>> actions for performance.
>> Just out of curiousity : could you tell a bit more about your use case
>> and what makes a simple closure not an option ?
>
> Okay.
>
> I work in the field of embedded programming, and one of the main uses
> I have for Python (and previously Perl) is writing GUIs for
> controlling embedded systems. The communication protocols are usually
> ad-hoc messages (headear, footer, data, crc) built on top of serial
> communication (RS232).
ok
> The packets that arrive have a known format. For example (YAMLish
> syntax):
>
> packet_length: 10
> fields:
> - name: header
> offset: 0
> length: 1
> - name: time_tag
> offset: 1
> length: 1
> transform: val * 2048
> units: ms
> - name: counter
> offset: 2
> length: 4
> bytes-msb-first: true
> - name: bitmask
> offset: 6
> length: 1
> bit_from: 0
> bit_to: 5
> ...
>
> This is a partial capability display. Fields have defined offsets and
> lengths, can be only several bits long, can have defined
> transformations and units for convenient display.
ok
> I have a program that should receive such packets from the serial port
> and display their contents in tabular form. I want the user to be able
> to specify the format of his packets in a file similar to above.
ok
> Now, in previous versions of this code, written in Perl, I found out
> that the procedure of extracting field values from packets is very
> inefficient. I've rewritten it using a dynamically generated procedure
> for each field, that does hard coded access to its data. For example:
>
> def get_counter(packet):
> data = packet[2:6]
> data.reverse()
> return data
>
> This gave me a huge speedup, because each field now had its specific
> function sitting in a dict that quickly extracted the field's data
> from a given packet.
ok. So if I get it right, you build the function's code as a string
based on the YAML specification.
If so, well, I can't think of anything really better[1] - at least *if*
dynamically generated procedures are really better performance wise,
which may *or not* be the case in Python.
[1] except using compile to build a code object with the function's
body, then instanciate a function object using this code, but I'm not
sure whether it will buy you much more performance-wise. I'd personnaly
prefer this because I find it more explicit and readable, but YMMV.
> Now I'm rewriting this program in Python and am wondering about the
> idiomatic way to use exec (in Perl, eval() replaces both eval and exec
> of Python).
Well... So far, the most pythonic way to use exec is to avoid using it -
unless it's the right tool for the job !-)
More information about the Python-list
mailing list