From rdmurray at bitdance.com Sat Mar 6 05:27:41 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 05 Mar 2010 23:27:41 -0500 Subject: [Email-SIG] More API thoughts (policy objects) Message-ID: <20100306042741.8C74E1A766D@kimball.webabinitio.net> I've been mulling on policy objects since talking with Barry about them at the sprint. Today I sat down and started on the "real" API documentation (ie: in Sphinx, fully written out not just sketched). So far I have the stub of the main email doc copied from the current package, with the intro turned into an XXX, and the new changes sections marked with an XXX, and a cut down table of contents listing only the new docs I've written. There are two doc sections currently, Policy Objects, and Headers. Headers is somewhat revised from both my original proposal and from the code actually implemented in the bzr branch. This is because I realized as I was writing the docs that I'd failed to think through how header subclasses were going to get created. Creating the correct subclass requires knowing the field name of the header for which an object is being instantiated, which means that the parser can't just gather header lines and pass them to the header class to parse the way I'd originally visualized. Instead we need a factory function that will look up the field name in the registry and instantiate the correct class, passing it the name and value. Since the registry is going to be in the policy object, that led me to write the policy object docs. After several iterations over both documents, I have something that I think at least hangs together, and which I think is fairly decent from a usability standpoint (I hope...that opinion is based on thought experiments, not real code, after all!). I'm not sure yet about the higher levels (the parser...or is the Message object the parser?), but I've got a vision of how the Message objects, the Header objects, and Message sub-objects plug together. The plugging-together isn't documented yet, although the existing docs give some good glimpses. One potentially controversial bit of this design is that I'm planning to have the header factory pull together three independent classes and dynamically build a composite class out of them to instantiate an object from. This is not something I've ever done before in code, so I'm not sure it will even work. But it seems to make sense as a design, so I've speced it that way. I'll be curious if anyone has any insights, opinions, or better ways of accomplishing the same goal. This design arose from the fact that in my overall design the Bytes and String classes share a common base class and that most of the code is common between the two. When thinking about the registry of structured headers, it felt rather yucky to have either two registries (bytes and string) or a registry with (bytes class, string class) tuples. Add to this the fact that it would be really nice for an application to have a way to substitute in its own base class without having to build a whole new subclass tree, and I decided to give dynamic class building a try. So the design I've speced has a header factory method looking up the base class, the string or byte class, and the specific header class in the registry, and then composing the three of them in order to return a header instance. Am I crazy? Anyway, if you want to look over the docs (most of which are probably valid even if I end up having to give up on my composition-based registry), they are up on my web site here: http://www.bitdance.com/projects/email6/doc/email/ --David PS: it's not clear to me that policy objects make thread safety natural, although they do make it *possible*. From barry at python.org Tue Mar 9 04:07:48 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 8 Mar 2010 22:07:48 -0500 Subject: [Email-SIG] More API thoughts (policy objects) In-Reply-To: <20100306042741.8C74E1A766D@kimball.webabinitio.net> References: <20100306042741.8C74E1A766D@kimball.webabinitio.net> Message-ID: <20100308220748.1b7966e6@heresy.wooz.org> This looks pretty good David. Nice to see you run with it. (Small nit: there are a few typos here and there.) On Mar 05, 2010, at 11:27 PM, R. David Murray wrote: >Since the registry is going to be in the policy object, that led me >to write the policy object docs. After several iterations over both >documents, I have something that I think at least hangs together, and >which I think is fairly decent from a usability standpoint (I hope...that >opinion is based on thought experiments, not real code, after all!). I've read the document through a few times, and while I'm sure my lack of sleep is a contributing factor (the other part being my denseness ;), I'm not quite getting how the Policy and PolicySet classes work together, nor how the policy_overlay argument comes into play. >One potentially controversial bit of this design is that I'm planning >to have the header factory pull together three independent classes >and dynamically build a composite class out of them to instantiate an >object from. This is not something I've ever done before in code, so I'm >not sure it will even work. But it seems to make sense as a design, >so I've speced it that way. I'll be curious if anyone has any insights, >opinions, or better ways of accomplishing the same goal. > >This design arose from the fact that in my overall design the Bytes and >String classes share a common base class and that most of the code is >common between the two. When thinking about the registry of structured >headers, it felt rather yucky to have either two registries (bytes and >string) or a registry with (bytes class, string class) tuples. Add to >this the fact that it would be really nice for an application to have a >way to substitute in its own base class without having to build a whole >new subclass tree, and I decided to give dynamic class building a try. >So the design I've speced has a header factory method looking up the >base class, the string or byte class, and the specific header class in >the registry, and then composing the three of them in order to return >a header instance. > >Am I crazy? Not necessarily! It's an interesting take on the design problem that makes some sense. The other way to handle it is through composition and delegation, but honestly IMO Python does not have a great delegation story (no, __getattr*__ does not cut it ;). It's also pretty easy to do in Python, if I understand what you're trying to get at: Python 3.1.2rc1+ (release31-maint:78807, Mar 8 2010, 22:02:30) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class Foo: pass ... >>> class Bar: pass ... >>> class Baz: pass ... >>> Qux = type('Qux', (Foo, Bar, Baz), {}) >>> type(Qux) >>> Qux() <__main__.Qux object at 0x7f0f7e62a750> >>> Qux.__bases__ (, , ) A few other thoughts. * Do we need raise_on_defect, handle_defect() and register_defect(). Seems a bit of overkill; what's the use case? * Why does header_indent default to 8 spaces? Shouldn't it be a tab or single space? * Do you think newline should be renamed end_of_line, linesep, or end? The latter two are evocative of os.linesep and print(..., end='\n', ...). "newline" specifically is actually misleading. That's it for now. Cool stuff! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From rdmurray at bitdance.com Tue Mar 9 19:16:54 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 09 Mar 2010 13:16:54 -0500 Subject: [Email-SIG] More API thoughts (policy objects) In-Reply-To: <20100308220748.1b7966e6@heresy.wooz.org> References: <20100306042741.8C74E1A766D@kimball.webabinitio.net> <20100308220748.1b7966e6@heresy.wooz.org> Message-ID: <20100309181654.5BAAD1AB5AB@kimball.webabinitio.net> On Mon, 08 Mar 2010 22:07:48 -0500, Barry Warsaw wrote: > This looks pretty good David. Nice to see you run with it. > > (Small nit: there are a few typos here and there.) Yes, I've caught a few, feel free to point out any others :) > On Mar 05, 2010, at 11:27 PM, R. David Murray wrote: > >Since the registry is going to be in the policy object, that led me > >to write the policy object docs. After several iterations over both > >documents, I have something that I think at least hangs together, and > >which I think is fairly decent from a usability standpoint (I hope...that > >opinion is based on thought experiments, not real code, after all!). > > I've read the document through a few times, and while I'm sure my lack of > sleep is a contributing factor (the other part being my denseness ;), I'm not > quite getting how the Policy and PolicySet classes work together, nor how the > policy_overlay argument comes into play. No, you are right. What I speced works, I think, but in the light of morning it is way too confusing, and users would get it "wrong", which means the API design is flawed. I haven't quite wrapped my head around how to do it right, but I do plan to fix it. Here are what I think the design requirements are: 1) a complete set of defaults has to come from somewhere and be accessible to every object that uses policy. 2) an object needs to be able to provide local overrides to these defaults that are normally obeyed. 3) an application should to be able to specify changes to the global defaults without having to specify all the defaults, and those changes should not affect the object-local overrides. 4) an application should be able to locally override the application global defaults on specific method calls that support doing so. I think these overrides *should* override the object-local overrides. 5) ideally there should be one set of pre-defined 'policy objects' that can be used as application level defaults and as local-command overrides (this is where I got into trouble, I think). Whatever else is true, the docs need examples (ie: use cases). If I write those next, I'll probably figure out a better API. > >One potentially controversial bit of this design is that I'm planning > >to have the header factory pull together three independent classes > >and dynamically build a composite class out of them to instantiate an > >object from. This is not something I've ever done before in code, so I'm > >not sure it will even work. But it seems to make sense as a design, > >so I've speced it that way. I'll be curious if anyone has any insights, > >opinions, or better ways of accomplishing the same goal. > > > >This design arose from the fact that in my overall design the Bytes and > >String classes share a common base class and that most of the code is > >common between the two. When thinking about the registry of structured > >headers, it felt rather yucky to have either two registries (bytes and > >string) or a registry with (bytes class, string class) tuples. Add to > >this the fact that it would be really nice for an application to have a > >way to substitute in its own base class without having to build a whole > >new subclass tree, and I decided to give dynamic class building a try. > >So the design I've speced has a header factory method looking up the > >base class, the string or byte class, and the specific header class in > >the registry, and then composing the three of them in order to return > >a header instance. > > > >Am I crazy? > > Not necessarily! It's an interesting take on the design problem that > makes some sense. The other way to handle it is through composition > and delegati on, but honestly IMO Python does not have a great > delegation story (no, __getattr*__ does not cut it ;). It's also Right, this is a Python weak spot. > pretty easy to do in Python, if I understand what you're trying to > get at: > > Python 3.1.2rc1+ (release31-maint:78807, Mar 8 2010, 22:02:30)=20 > [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" > for more information. > >>> class Foo: pass > ...=20 > >>> class Bar: pass > ...=20 > >>> class Baz: pass > ...=20 > >>> Qux = type('Qux', (Foo, Bar, Baz), {}) type(Qux) > > >>> Qux() > <__main__.Qux object at 0x7f0f7e62a750> > >>> Qux.__bases__ > (, , '__main__.Baz'>) Ah, good, that is what I thought it would look like, but I hadn't run the experiment yet. > A few other thoughts. > > * Do we need raise_on_defect, handle_defect() and register_defect(). > Seems a bit of overkill; what's the use case? Perhaps I'm over-designing :). My thought was that this gave the policy object centralized control over what happens to defects. The flag controls the raise-or-not. handle_defect is on the policy object so that every method that needs to handle a defect doesn't have to implement its own raise-or-not logic. register_defect is the hook to do something non-standard with defects that aren't raised. The most obvious use case for this would be to log the defects instead of/in addition to putting them on the object. Perhaps handle_defect should be private method. Since the application can control raise-or-not via the flag, there's no real need to have that method be part of the public API. > * Why does header_indent default to 8 spaces? Shouldn't it be a tab > or single space? You are right, it should be tab. I will fix it. I was carrying Python's anti-tab sentiment a little too far ;) > * Do you think newline should be renamed end_of_line, linesep, or end? > The latter two are evocative of os.linesep and print(..., end='\n', > ...). "newline" specifically is actually misleading. Yes, I would like to rename newline to *something* else. I was following the lead of the io module docs in this, but I don't like it. Maybe we could change the io docs, too? :) I think I like using 'linesep' better than 'end'. In the print function, 'end' comes literally at the end of the string being output, whereas in the email package case we are talking about what goes in between lines, as well as at the end. > That's it for now. Cool stuff! -Barry Thanks for the feedback! --David From barry at python.org Fri Mar 12 23:21:48 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 12 Mar 2010 17:21:48 -0500 Subject: [Email-SIG] Fw: MIME objects Message-ID: <20100312172148.31d7e74d@heresy.wooz.org> My colleague Robert Collins sent me this and asked that I forward it to the email-sig for discussion. -Barry Begin forwarded message: Date: Sun, 28 Feb 2010 17:20:00 +1100 From: Robert Collins To: Barry Warsaw Subject: MIME objects I wanted to use MIME in selftest machinery, but the stdlib stuff was uhm, missing. So I wrote one; its working well - perhaps you could include it in the stdlib? http://bazaar.launchpad.net/~testtools-dev/testtools/trunk/annotate/head %3A/testtools/content_type.py http://bazaar.launchpad.net/~testtools-dev/testtools/trunk/annotate/head %3A/testtools/content.py (branch testtools, read the README + pydoc, look at tests. Features: - 2 and 3 clean - understands byte serialisation vs string representation - streaming interface: does not require or imply buffering - common case use is trivial (define one lambda: [value]). Cheers, Rob -- -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: