From vinstspel at wbl.nu Fri Jan 18 10:03:44 2002 From: vinstspel at wbl.nu (Svenskt Spelsystem) Date: Fri, 18 Jan 2002 10:03:44 +0100 Subject: [C++-sig] Ny otrolig Tipssensation reserverad bara dig! Message-ID: <200201180903.KAA10731@d1o906.telia.com> PROFLEX-134 Plus en otrolig bonus. HELT GRATIS (se l?ngre ner) Passa p? nu n?r det ?r jackpott i helgen, Med v?r hj?lp kan du plocka hem de stora vinsterna! Detta ?r ett helt nytt s?tt att spela p? Tipset, Du kan vara bland de f?rsta att plocka hem dom stora vinsterna! Detta helt nya revolutionerande tipssystem ?r utan tvekan det absolut b?sta som har tagits fram f?r detta ?ndam?l. Det ?r helt individuellt anpassat bara f?r dig. Det ?r mycket enkelt att anv?nda. Med PROFLEX-134 medf?ljer naturligvis en mycket l?ttf?rst?lig svensk bruksanvisning och garantitabell. PROFLEX-134 levereras komplett med garantitabell och mycket l?ttf?rst?eliga anvisningar. Dessutom lovar vi pengarna tillbaka som du har betalt f?r PROFLEX-134 om du inte f?r vinst redan den f?rsta veckan OBS!GL?M INTE DIN OTROLIGA GRATIS BONUS L?NGRE NER Best?ll redan i dag och du har chans att vinna upp till 75 miljoner, eller ?nnu mer vid jackpott, redan n?sta vecka. Du har inget att f?rlora utan allt att vinna! Best?ller du dessutom inom 10 dagar f?r du utan extra kostnad; Det nya stryktips programmmet "Olympia tips 2002". i en helt ny version bara det v?rt 999:- och som ?r ett enormt hj?lpmedel f?r dig som f?redrar att spela p? stryktipset HELT GRATIS! Och det ?r inte nog med det: Vi skickar ?ven med det fantasiska "V75 Max" f?r datorn. En spel sensation som p? ett enkelt s?tt hj?lper dig att plocka ut de r?tta h?starna. HELT GRATIS Kom ih?g att PROFLEX-134 sk?ter sig helt sj?lv, Du beh?ver bara fylla i matcherna efter anvisningarna Ja Tack! Jag best?ller PROFLEX-134 f?r introduktionspriset 298:- (Ord pris 999:-) Vid f?rskottsbetalning =, (skickas via din emejl) 298 : -tillkommer inga avgifter! Vid f?rskottsbetalning= Postgironummer 920 11 81-6 tillkommer inga avgifter (Gl?m inte namn, adress och telenummer) Mot postf?rskott =, Pappers version = 298:- (plus posten & fraktavg =+81:-) Du kan mejla in din order: D? ?r adressen : vinstspel at wbl.nu Du kan ?ven ringa: Order telefon 0300-71871 DYGNET RUNT (Ordermottagare) Naturligvis kan du ?ven skriva in din best?llning Skriv d? p? kuvertet : Svenskt Spelsystem , FRISVAR Box 10082, 434 21 Kungsbacka, s? slipper du portot. Gl?m inte skriva vilken produkt det g?ller. (PROFLEX-134) OBS gl?m inte skriva in namn och adress vid best?llningen! PS! GL?M INTE DIN OTROLIGA GRATIS BONUS Bifogar du dessutom 5 v?nners emejladresser f?r du utan n?gon som helst kostnad 5 helt nya spelsystem , f?r V75, Stryktipset, Lotto, V64 och m?ltipset. HELT GRATIS! Dem b?rjar inte marknadf?ras f?ren tidigast 2002-09-15 s? passa medans du ?r f?rst. Med v?nlig h?lsning, och hopp om en trevlig v?r ?nskar vi p? Svenskt Spelsystem vinstspel at wbl.nu Svenskt spelsystem ?r nu st?rst p? spelsystem och spelhj?lpmedel i Norden! (etablerat 1987) Vilket borgar f?r n?jda kunder och h?g kvalitet! Leverans tiden kan variera Dock max 2 veckor From arnaldur at decode.is Thu Jan 3 20:52:43 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Thu, 3 Jan 2002 19:52:43 +0000 Subject: boost::python, returning new PyObject references Message-ID: --0__=00256B36006B2DF58f9e8a93df938690918c00256B36006B2DF5 Content-type: text/plain; charset=iso-8859-1 Dave, > > It is true that it is just used > > for mapping. > > (Should be used for dictionary also). The reason for it in those cases > > is > > detecting illegal key access > > but allowing setting new key-value pair. > Normally a proxy would only do the access on demand. I can't think of > any reason to hold a reference to an object other than the underlying > sequence or mapping in the proxy. Oh, well I guess in the mapping case > the proxy should hold a reference to the key object and the underlying > sequence. > > You're right that it is probably better to override the get() and > > reference > > () methods for mapping > > (dictionary later) instead of putting it in object_base. > Did I suggest that? I'm not sure what you're referring to. The proxy supports the sequence, mapping and number interfaces (plus object interface through virtual inheritance) by inheriting from them. It sent the ref to the sequence/mapping item to the corresponding constructors. What happens when we have a proxy for a key in a map when the key does not exist. a) If the proxy is used in an assignment that is OK and a new key is added b) Otherwise an Illegal Key error should be thrown. How can we distinguish between a) and b)? I decided to do that by grabbing the request for the underlying item in the get() and reference() methods. Thus by storing a NULL ref the request can be grabbed and a callback used to call the key get method. If you have a better design by all means tell me. I?m not overly pleased but I thought it was ok. I have rewritten it though in accordance to some of the things you said and this logic is just in the proxy. The get() and reference() have to be virtual though. I've combined the proxies into 1 , mutable_proxy, now used by sequence, mapping and list. The callbacks should be clearer (called get_ and set_ as you suggested). I believe this is cleaner than before and I hope you can make more out of it. Here is objects.hpp: (See attached file: objects.hpp) > What's the point of having different C++ types, then? You could just use > the generic object for everything. I suppose there would be a very minor > speed improvement if you know that an object is in fact a list rather > than some generic sequence, but I don't think too many people are > concerned about the speed of code operating at the C++/Python boundary. The C++ types export the corresponding C API parts: object - PyObject_* sequence - PySequence etc. I guess maybe sequence, mapping, number should then inherit from object_base not object. The proxy should then also inherit from object. Although a list does not support the __call__ interface we could have a user-defined class that supports both the sequence and __call__ interfaces. I am inclined to letting sequence, mapping etc. inherit from object_base but supporting more combinations of interfaces: object and sequence , object and mapping etc. What do you think? Cheers Arnaldur --0__=00256B36006B2DF58f9e8a93df938690918c00256B36006B2DF5 Content-type: application/octet-stream; name="objects.hpp" Content-Disposition: attachment; filename="objects.hpp" Content-transfer-encoding: base64 Ly8gIChDKSBDb3B5cmlnaHQgRGF2aWQgQWJyYWhhbXMgMjAwMC4gUGVybWlzc2lvbiB0byBjb3B5 LCB1c2UsIG1vZGlmeSwgc2VsbCBhbmQKLy8gIGRpc3RyaWJ1dGUgdGhpcyBzb2Z0d2FyZSBpcyBn cmFudGVkIHByb3ZpZGVkIHRoaXMgY29weXJpZ2h0IG5vdGljZSBhcHBlYXJzCi8vICBpbiBhbGwg Y29waWVzLiBUaGlzIHNvZnR3YXJlIGlzIHByb3ZpZGVkICJhcyBpcyIgd2l0aG91dCBleHByZXNz IG9yIGltcGxpZWQKLy8gIHdhcnJhbnR5LCBhbmQgd2l0aCBubyBjbGFpbSBhcyB0byBpdHMgc3Vp dGFiaWxpdHkgZm9yIGFueSBwdXJwb3NlLgovLwovLyAgVGhlIGF1dGhvciBncmF0ZWZ1bGx5IGFj a25vd2xlZ2VzIHRoZSBzdXBwb3J0IG9mIERyYWdvbiBTeXN0ZW1zLCBJbmMuLCBpbgovLyAgcHJv ZHVjaW5nIHRoaXMgd29yay4KCiNpZm5kZWYgT0JKRUNUU19EV0EwNTExMDBfSF8KIyBkZWZpbmUg T0JKRUNUU19EV0EwNTExMDBfSF8KCiMgaW5jbHVkZSA8Ym9vc3QvcHl0aG9uL2RldGFpbC93cmFw X3B5dGhvbi5ocHA+CiMgaW5jbHVkZSA8Ym9vc3QvcHl0aG9uL2RldGFpbC9jb25maWcuaHBwPgoj IGluY2x1ZGUgPGJvb3N0L3B5dGhvbi9yZWZlcmVuY2UuaHBwPgojIGluY2x1ZGUgImJvb3N0L29w ZXJhdG9ycy5ocHAiCiMgaW5jbHVkZSAiYm9vc3QvZnVuY3Rpb24uaHBwIgojIGluY2x1ZGUgImJv b3N0L2JpbmQuaHBwIgojIGluY2x1ZGUgImJvb3N0L2l0ZXJhdG9yX2FkYXB0b3JzLmhwcCIKIyBp bmNsdWRlICJQeXRob24uaCIKIyBpbmNsdWRlIDx1dGlsaXR5PgojaW5jbHVkZSA8aW9zdHJlYW0+ CgpuYW1lc3BhY2UgYm9vc3QgeyBuYW1lc3BhY2UgcHl0aG9uIHsKCgovKioKICogQSB3cmFwcGVy IGZvciBhIHJlZiAoc21hcnQgcG9pbnRlciB0byBhIFB5T2JqZWN0LCBzZWUgcmVmZXJlbmNlLmhw cCkuCiAqIFRoZSBiYXNlIGZvciBhbGwgcHl0aG9uIGludGVyZmFjZXMuCiAqIFxwYXIgRXh0ZW5z aW9uOgogKiBBbGwgcHl0aG9uIGludGVyZmFjZXMgc2hvdWxkIGluaGVyaXQgKGRpcmVjdGx5IG9y IGluZGlyZWN0bHkpIGZyb20gdGhpcyBjbGFzcy4KICoKICovCmNsYXNzIG9iamVjdF9iYXNlCnsK IHB1YmxpYzoKICAgIGV4cGxpY2l0IG9iamVjdF9iYXNlKHJlZiBwKSA6IG1fcChwKSB7IH0KICAg IAogICAgLy8gTk9URSEgVGhpcyBvbmx5IGFwcGllcyBpZiByaHMgaXMgYSBtdXRhYmxlX3Byb3h5 OiBJZiByaHMubV9wLmdldCgpIGlzIDAsCiAgICAvLyBhIHNlZ21lbnRhdGlvbiBmYXVsdCBvY2N1 cnMgaWYgdGhlIGRlZmF1bHQgY29weSBjb25zdHJ1Y3RvciBpcyB1c2VkIAogICAgLy8gKHBlY3Vs aWFyIHNpbmNlIFB5X1hJTkNSRUYgaXMgdXNlZCBpbiByZWYgY29weSBjb25zdHJ1Y3RvcikKICAg IC8vIFdlIG11c3QgY2FsbCByaHMucmVmZXJlbmNlKCkgdG8gY2FsbCB0aGUgY2FsbGJhY2sgZnVu Y3Rpb24gaW4gdGhhdCBjYXNlLgogICAgb2JqZWN0X2Jhc2UoY29uc3Qgb2JqZWN0X2Jhc2UmIHJo cykgOiBtX3AocmhzLnJlZmVyZW5jZSgpKSB7IH0KICAgIAogICAgLy8gU2ltaWxhciByZW1hcmtz IHRvIHRoZSBjb3B5IGNvbnN0cnVjdG9yCiAgICBvYmplY3RfYmFzZSYgb3BlcmF0b3I9KGNvbnN0 IG9iamVjdF9iYXNlJiByaHMpCiAgICB7CiAgICAgICBpZiAodGhpcyA9PSAmcmhzKSByZXR1cm4g KnRoaXM7CiAgICAgICBtX3AgPSByaHMucmVmZXJlbmNlKCk7CiAgICAgICByZXR1cm4gKnRoaXM7 CiAgICB9CiAgICAKICAgIC8vIFJldHVybiBhIHJlZmVyZW5jZSB0byB0aGUgaGVsZCBvYmplY3QK ICAgIHZpcnR1YWwgcmVmIHJlZmVyZW5jZSgpIGNvbnN0CiAgICB7CiAgICAgICByZXR1cm4gbV9w OwogICAgfQoKICAgIC8vIFJldHVybiBhIHJhdyBwb2ludGVyIHRvIHRoZSBoZWxkIG9iamVjdAog ICAgdmlydHVhbCBQeU9iamVjdCogZ2V0KCkgY29uc3QKICAgIHsKICAgICAgIHJldHVybiBtX3Au Z2V0KCk7CiAgICB9CiAgICAKICAgIHZvaWQgcmVzZXQocmVmIG5ld19yZWYpIGNvbnN0CiAgICB7 CiAgICAgICBtX3AgPSBuZXdfcmVmOwogICAgfQogCgogICAgb3BlcmF0b3IgcmVmKCkgY29uc3QK ICAgIHsKICAgICAgIHJldHVybiByZWZlcmVuY2UoKTsgCiAgICB9CiAgICAKIHByaXZhdGU6CiAg ICBtdXRhYmxlIHJlZiBtX3A7Cn07CgoKCgoKY2xhc3Mgb2JqZWN0IDogcHVibGljIG9iamVjdF9i YXNlCnsKICAgICAgCnB1YmxpYzoKICAgc3RhdGljIGJvb2wgYWNjZXB0cyhyZWYgcCkKICAgewog ICAgICByZXR1cm4gcC5nZXQoKTsKICAgfQogICAKICAgZXhwbGljaXQgb2JqZWN0KHJlZiBwb2Jq KTogb2JqZWN0X2Jhc2UocG9iaikgeyB9CgogICB0ZW1wbGF0ZTx0eXBlbmFtZSBUPgogICB2b2lk IHNldGF0dHIoY2hhciogYXR0cl9uYW1lLFQgdmFsKQogICB7CiAgICAgIFB5T2JqZWN0X1NldEF0 dHJTdHJpbmcoZ2V0KCksYXR0cl9uYW1lLG1ha2VfcmVmKHZhbCkuZ2V0KCkpOwogICB9CgogICB0 ZW1wbGF0ZTx0eXBlbmFtZSBUPgogICB2b2lkIGdldGF0dHIoY2hhciogYXR0cl9uYW1lLFQgJiB2 YWwpIGNvbnN0CiAgIHsKICAgICAgdmFsID0gZnJvbV9weXRob24ocmVmKFB5T2JqZWN0X0dldEF0 dHJTdHJpbmcoZ2V0KCksYXR0cl9uYW1lKSkuZ2V0KCksCiAgICAgICAgICAgICAgICAgICAgICAg IHB5dGhvbjo6dHlwZTxUPigpKTsKICAgfQoKICAgb2JqZWN0IGdldGF0dHIoY2hhciogYXR0cl9u YW1lKSBjb25zdAogICB7CiAgICAgIHJldHVybiBvYmplY3QocmVmKFB5T2JqZWN0X0dldEF0dHJT dHJpbmcoZ2V0KCksYXR0cl9uYW1lKSkpOwogICB9CiAgIAoKICAgYm9vbCBoYXNhdHRyKGNoYXIq IGF0dHJfbmFtZSkgY29uc3QKICAgewogICAgICByZXR1cm4gUHlPYmplY3RfSGFzQXR0clN0cmlu ZyhnZXQoKSxhdHRyX25hbWUpICE9IDA7CiAgIH0KICAgCiAgIGludCBsZW4oKSBjb25zdAogICB7 CiAgICAgIHJldHVybiBQeU9iamVjdF9MZW5ndGgoZ2V0KCkpOwogICB9CiAgIAogICBzdGQ6OnN0 cmluZyBzdHIoKSBjb25zdAogICB7CiAgICAgIHJlZiBzdHIoUHlPYmplY3RfU3RyKGdldCgpKSk7 CiAgICAgIHJldHVybiBmcm9tX3B5dGhvbihzdHIuZ2V0KCkscHl0aG9uOjp0eXBlPHN0ZDo6c3Ry aW5nPigpKTsKICAgfQoKICAgb2JqZWN0IG9wZXJhdG9yKCkoKQogICB7CiAgICAgIHJldHVybiBv YmplY3QocmVmKFB5T2JqZWN0X0NhbGxGdW5jdGlvbihnZXQoKSwiKCkiKSkpOwogICB9CgogICBv YmplY3QgY2FsbF9tZXRob2QoY2hhciogbWV0aG9kX25hbWUpCiAgIHsKICAgICAgcmV0dXJuIG9i amVjdChyZWYoUHlPYmplY3RfQ2FsbE1ldGhvZChnZXQoKSxtZXRob2RfbmFtZSwiKCkiKSkpOwog ICB9CgogICB0ZW1wbGF0ZSA8Y2xhc3MgQTE+CiAgIG9iamVjdCBvcGVyYXRvcigpKGNvbnN0IEEx JiBhMSkKICAgewogICAgICByZXR1cm4gb2JqZWN0KHJlZihQeU9iamVjdF9DYWxsRnVuY3Rpb24o Z2V0KCksIihPKSIsbWFrZV9yZWYoYTEpLmdldCgpKSkpOwogICB9CgogICB0ZW1wbGF0ZSA8Y2xh c3MgQTE+CiAgIG9iamVjdCBjYWxsX21ldGhvZChjaGFyKiBtZXRob2RfbmFtZSxjb25zdCBBMSYg YTEpCiAgIHsKICAgICAgcmV0dXJuIG9iamVjdChyZWYoUHlPYmplY3RfQ2FsbE1ldGhvZChnZXQo KSxtZXRob2RfbmFtZSwiKE8pIiwKICAgICAgICAgICAgICAgICAgICBtYWtlX3JlZihhMSkuZ2V0 KCkpKSk7CiAgIH0KCiAgIHRlbXBsYXRlIDxjbGFzcyBBMSwgY2xhc3MgQTI+CiAgIG9iamVjdCBv cGVyYXRvcigpKGNvbnN0IEExJiBhMSwgY29uc3QgQTImIGEyKQogICB7CiAgICAgIHJldHVybiBv YmplY3QocmVmKFB5T2JqZWN0X0NhbGxGdW5jdGlvbihnZXQoKSwiKE9PKSIsbWFrZV9yZWYoYTEp LmdldCgpLAogICAgICAgICAgICAgICAgICAgIG1ha2VfcmVmKGEyKS5nZXQoKSkpKTsKICAgfQoK ICAgdGVtcGxhdGUgPGNsYXNzIEExLCBjbGFzcyBBMj4KICAgb2JqZWN0IGNhbGxfbWV0aG9kKGNo YXIqIG1ldGhvZF9uYW1lLGNvbnN0IEExJiBhMSwgY29uc3QgQTImIGEyKQogICB7CiAgICAgIHJl dHVybiBvYmplY3QocmVmKFB5T2JqZWN0X0NhbGxNZXRob2QoZ2V0KCksbWV0aG9kX25hbWUsIihP TykiLAogICAgICAgICAgICAgICAgICAgIG1ha2VfcmVmKGExKS5nZXQoKSxtYWtlX3JlZihhMiku Z2V0KCkpKSk7CiAgIH0KICAgCiAgIHRlbXBsYXRlPGNsYXNzIFQ+CiAgIG9wZXJhdG9yIFQoKSBj b25zdAogICB7CiAgICAgIGNvdXQgPDwgIm9wZXJhdG9yIFQoKSIgPDwgZW5kbDsKICAgICAgcmV0 dXJuIGZyb21fcHl0aG9uKGdldCgpLHR5cGU8VD4oKSk7CiAgIH0KICAgCn07CgoKY2xhc3MgbnVt YmVyIDogcHVibGljIHZpcnR1YWwgb2JqZWN0CnsKcHVibGljOgogICBzdGF0aWMgYm9vbCBhY2Nl cHRzKHJlZiByKSB7IHJldHVybiBQeU51bWJlcl9DaGVjayhyLmdldCgpKTsgfQogICAKICAgZXhw bGljaXQgbnVtYmVyKGNvbnN0IG9iamVjdCAmIG9iaik6IG9iamVjdChvYmopIHt9CiAgIAogICBl eHBsaWNpdCBudW1iZXIoY29uc3QgcmVmJiByaHMpOiBvYmplY3QocmhzKSB7fQoKCiAgIC8vIG9w ZXJhdG9yIHN1cHBvcnQgZm9yIEMrKyB0eXBlcwoKICAgdGVtcGxhdGU8dHlwZW5hbWUgVD4KICAg bnVtYmVyJiBvcGVyYXRvcis9KFQgeCkKICAgewogICAgICByZWYgdG1wKFB5TnVtYmVyX0luUGxh Y2VBZGQoZ2V0KCksbWFrZV9yZWYoeCkuZ2V0KCkpKTsKICAgICAgcmVzZXQodG1wKTsKICAgICAg cmV0dXJuICp0aGlzOwogICB9CiAgIAogICB0ZW1wbGF0ZTx0eXBlbmFtZSBUPgogICBudW1iZXIm IG9wZXJhdG9yLT0oVCB4KQogICB7CiAgICAgIHJlZiB0bXAoUHlOdW1iZXJfSW5QbGFjZVN1YnRy YWN0KGdldCgpLG1ha2VfcmVmKHgpLmdldCgpKSk7CiAgICAgIHJlc2V0KHRtcCk7CiAgICAgIHJl dHVybiAqdGhpczsKICAgfQoKICAgdGVtcGxhdGU8dHlwZW5hbWUgVD4KICAgbnVtYmVyJiBvcGVy YXRvcio9KFQgeCkKICAgewogICAgICByZWYgdG1wKFB5TnVtYmVyX0luUGxhY2VNdWx0aXBseShn ZXQoKSxtYWtlX3JlZih4KS5nZXQoKSkpOwogICAgICByZXNldCh0bXApOwogICAgICByZXR1cm4g KnRoaXM7CiAgIH0KCiAgIHRlbXBsYXRlPHR5cGVuYW1lIFQ+CiAgIG51bWJlciYgb3BlcmF0b3Iv PShUIHgpCiAgIHsKICAgICAgcmVmIHRtcChQeU51bWJlcl9JblBsYWNlRGl2aWRlKGdldCgpLG1h a2VfcmVmKHgpLmdldCgpKSk7CiAgICAgIHJlc2V0KHRtcCk7CiAgICAgIHJldHVybiAqdGhpczsK ICAgfQoKICAgdGVtcGxhdGU8dHlwZW5hbWUgVD4KICAgbnVtYmVyJiBvcGVyYXRvciU9KFQgeCkK ICAgewogICAgICByZWYgdG1wKFB5TnVtYmVyX0luUGxhY2VSZW1haW5kZXIoZ2V0KCksbWFrZV9y ZWYoeCkuZ2V0KCkpKTsKICAgICAgcmVzZXQodG1wKTsKICAgICAgcmV0dXJuICp0aGlzOwogICB9 CgogICB0ZW1wbGF0ZTx0eXBlbmFtZSBUPgogICBudW1iZXImIG9wZXJhdG9yfD0oVCB4KQogICB7 CiAgICAgIHJlZiB0bXAoUHlOdW1iZXJfSW5QbGFjZU9yKGdldCgpLG1ha2VfcmVmKHgpLmdldCgp KSk7CiAgICAgIHJlc2V0KHRtcCk7CiAgICAgIHJldHVybiAqdGhpczsKICAgfQoKICAgdGVtcGxh dGU8dHlwZW5hbWUgVD4KICAgbnVtYmVyJiBvcGVyYXRvciY9KFQgeCkKICAgewogICAgICByZWYg dG1wKFB5TnVtYmVyX0luUGxhY2VBbmQoZ2V0KCksbWFrZV9yZWYoeCkuZ2V0KCkpKTsKICAgICAg cmVzZXQodG1wKTsKICAgICAgcmV0dXJuICp0aGlzOwogICB9CgogICB0ZW1wbGF0ZTx0eXBlbmFt ZSBUPgogICBudW1iZXImIG9wZXJhdG9yXj0oVCB4KQogICB7CiAgICAgIHJlZiB0bXAoUHlOdW1i ZXJfSW5QbGFjZVhvcihnZXQoKSxtYWtlX3JlZih4KS5nZXQoKSkpOwogICAgICByZXNldCh0bXAp OwogICAgICByZXR1cm4gKnRoaXM7CiAgIH0KICAgCiAgIAp9OwoKCnRlbXBsYXRlPHR5cGVuYW1l IFQ+Cm51bWJlciBvcGVyYXRvcisoVCB2YWwsY29uc3QgbnVtYmVyJiBuKQp7CiAgIHJldHVybiBu dW1iZXIobikgKz0gdmFsOwp9Cgp0ZW1wbGF0ZTx0eXBlbmFtZSBUPgpudW1iZXIgb3BlcmF0b3Ir KGNvbnN0IG51bWJlciYgbiwgVCB2YWwpCnsKICAgcmV0dXJuIG51bWJlcihuKSArPSB2YWw7Cn0K Cm51bWJlciBvcGVyYXRvcisoY29uc3QgbnVtYmVyJiBuLCBjb25zdCBudW1iZXImIHZhbCk7CgoK CmNsYXNzIG11dGFibGVfcHJveHk7CgoKY2xhc3Mgc2VxdWVuY2UgOiBwdWJsaWMgdmlydHVhbCBv YmplY3QKewpwdWJsaWM6CiAgIHN0YXRpYyBib29sIGFjY2VwdHMocmVmIHApCiAgIHsKICAgICAg cmV0dXJuIFB5U2VxdWVuY2VfQ2hlY2socC5nZXQoKSk7CiAgIH0KCiAgIGV4cGxpY2l0IHNlcXVl bmNlKGNvbnN0IG9iamVjdCAmIG9iaik6IG9iamVjdChvYmopIHt9CiAgIAogICBleHBsaWNpdCBz ZXF1ZW5jZShjb25zdCByZWYmIHJocyk6IG9iamVjdChyaHMpIHt9CiAgIAogICBjb25zdCBtdXRh YmxlX3Byb3h5IG9wZXJhdG9yW10oaW50IGkpIGNvbnN0OwoKICAgbXV0YWJsZV9wcm94eSBvcGVy YXRvcltdKGludCBpKTsKICAgCiAgIHRlbXBsYXRlPGNsYXNzIFQ+CiAgIHZvaWQgYXBwZW5kKGNv bnN0IFQmIHZhbCk7IC8vIGJ1aWxkcyBhIGxpc3QgaG9sZGluZyBqdXN0IHZhbCwgYW5kIGNhbGxz ICs9LgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAvLyBJbXBsZW0uIGFmdGVyIGxpc3QK ICAgCiAgIGludCBzaXplKCkgY29uc3QKICAgewogICAgICByZXR1cm4gUHlTZXF1ZW5jZV9TaXpl KGdldCgpKTsKICAgfQogICAKICAgcmVmIGdldF9pdGVtKGludCBpKSBjb25zdAogICB7CiAgICAg IHJldHVybiByZWYoUHlTZXF1ZW5jZV9HZXRJdGVtKGdldCgpLGkpKTsKICAgfQoKICAgdGVtcGxh dGU8Y2xhc3MgVD4KICAgdm9pZCBzZXRfaXRlbShpbnQgaSxjb25zdCBUJiB2YWwpCiAgIHsKICAg ICAgc2V0X2l0ZW0oaSxtYWtlX3JlZih2YWwpKTsKICAgfQoKICAgdm9pZCBzZXRfaXRlbShpbnQg aSxjb25zdCByZWYmIHJ2YWwpCiAgIHsKICAgICAgUHlTZXF1ZW5jZV9TZXRJdGVtKGdldCgpLGks cnZhbC5nZXQoKSk7CiAgIH0gIAoKICAgdm9pZCByc2V0X2l0ZW0oaW50IGksY29uc3QgcmVmJiBy dmFsKQogICB7CiAgICAgIFB5U2VxdWVuY2VfU2V0SXRlbShnZXQoKSxpLHJ2YWwuZ2V0KCkpOwog ICB9ICAKCiAgIAogICB0ZW1wbGF0ZTxjbGFzcyBUPgogICBzZXF1ZW5jZSYgb3BlcmF0b3IrPShj b25zdCBUJiB2YWwpCiAgIHsKICAgICAgcmVmIHRtcChQeVNlcXVlbmNlX0luUGxhY2VDb25jYXQo Z2V0KCksbWFrZV9yZWYodmFsKS5nZXQoKSkpOwogICAgICByZXNldCh0bXApOwogICAgICByZXR1 cm4gKnRoaXM7CiAgIH0KCn07CgoKY2xhc3MgbWFwcGluZyA6IHB1YmxpYyB2aXJ0dWFsIG9iamVj dAp7CnB1YmxpYzoKCiAgIHN0YXRpYyBib29sIGFjY2VwdHMocmVmIHApCiAgIHsKICAgICAgcmV0 dXJuIFB5TWFwcGluZ19DaGVjayhwLmdldCgpKTsKICAgfQogICAKICAgZXhwbGljaXQgbWFwcGlu Zyhjb25zdCBvYmplY3QgJiBvYmopOiBvYmplY3Qob2JqKSB7fQogICAKICAgZXhwbGljaXQgbWFw cGluZyhjb25zdCByZWYmIHJocyk6IG9iamVjdChyaHMpIHt9CgogICB0ZW1wbGF0ZTxjbGFzcyBU PgogICBtdXRhYmxlX3Byb3h5IG9wZXJhdG9yW10oY29uc3QgVCAmIGtleSk7CgogICB0ZW1wbGF0 ZTxjbGFzcyBUPgogICBjb25zdCBtdXRhYmxlX3Byb3h5IG9wZXJhdG9yW10oY29uc3QgVCAmIGtl eSkgY29uc3Q7CgogICB0ZW1wbGF0ZTxjbGFzcyBUPgogICBib29sIGhhc19rZXkoY29uc3QgVCAm IGtleSkgY29uc3QKICAgewogICAgICByZXR1cm4gaGFzX2tleShtYWtlX3JlZihrZXkpKTsKICAg fQoKICAgYm9vbCBoYXNfa2V5KGNvbnN0IHJlZiAmIGtleSkgY29uc3QKICAgewogICAgICByZXR1 cm4gUHlNYXBwaW5nX0hhc0tleShnZXQoKSxrZXkuZ2V0KCkpOwogICB9CiAgIAogICBzZXF1ZW5j ZSBrZXlzKCkgY29uc3QKICAgewogICAgICByZXR1cm4gc2VxdWVuY2UocmVmKFB5TWFwcGluZ19L ZXlzKGdldCgpKSkpOwogICB9CgogICBzZXF1ZW5jZSB2YWx1ZXMoKSBjb25zdAogICB7CiAgICAg IHJldHVybiBzZXF1ZW5jZShyZWYoUHlNYXBwaW5nX1ZhbHVlcyhnZXQoKSkpKTsKICAgfQoKICAg c2VxdWVuY2UgaXRlbXMoKSBjb25zdAogICB7CiAgICAgIHJldHVybiBzZXF1ZW5jZShyZWYoUHlN YXBwaW5nX0l0ZW1zKGdldCgpKSkpOwogICB9CgogICB0ZW1wbGF0ZSA8Y2xhc3MgS2V5PgogICBy ZWYgZ2V0X2l0ZW0oY29uc3QgS2V5JiBrZXkpIGNvbnN0CiAgIHsgCiAgICAgIHJldHVybiB0aGlz LT5nZXRfaXRlbShtYWtlX3JlZihrZXkpKTsgCiAgIH0KCiAgIHJlZiBnZXRfaXRlbShjb25zdCBy ZWYmIGtleSkgY29uc3QKICAgewogICAgICByZXR1cm4gcmVmKFB5T2JqZWN0X0dldEl0ZW0oZ2V0 KCksa2V5LmdldCgpKSk7CiAgIH0KCiAgIHJlZiByZ2V0X2l0ZW0oY29uc3QgcmVmJiBrZXkpIGNv bnN0CiAgIHsKICAgICAgcmV0dXJuIHJlZihQeU9iamVjdF9HZXRJdGVtKGdldCgpLGtleS5nZXQo KSkpOwogICB9CgogICB0ZW1wbGF0ZSA8Y2xhc3MgS2V5LCBjbGFzcyBWYWx1ZT4KICAgdm9pZCBz ZXRfaXRlbShjb25zdCBLZXkmIGtleSwgY29uc3QgVmFsdWUmIHZhbHVlKQogICB7CiAgICAgIHRo aXMtPnNldF9pdGVtKG1ha2VfcmVmKGtleSksIG1ha2VfcmVmKHZhbHVlKSk7IAogICB9CgogICB2 b2lkIHNldF9pdGVtKGNvbnN0IHJlZiYga2V5LCBjb25zdCByZWYmIHZhbHVlKQogICB7CiAgICAg IFB5T2JqZWN0X1NldEl0ZW0oZ2V0KCksa2V5LmdldCgpLHZhbHVlLmdldCgpKTsKICAgfQoKCiAg IHZvaWQgcnNldF9pdGVtKGNvbnN0IHJlZiYga2V5LCBjb25zdCByZWYmIHZhbHVlKQogICB7CiAg ICAgIFB5T2JqZWN0X1NldEl0ZW0oZ2V0KCksa2V5LmdldCgpLHZhbHVlLmdldCgpKTsKICAgfQoK Cn07CgoKCmNsYXNzIHNlcW1hcCA6IHB1YmxpYyBzZXF1ZW5jZSAsIHB1YmxpYyBtYXBwaW5nCnsK cHVibGljOgoKICAgc3RhdGljIGJvb2wgYWNjZXB0cyhyZWYgcCkKICAgewogICAgICByZXR1cm4g c2VxdWVuY2U6OmFjY2VwdHMocCkgJiYgbWFwcGluZzo6YWNjZXB0cyhwKTsKICAgfQoKICAgZXhw bGljaXQgc2VxbWFwKGNvbnN0IG9iamVjdCAmIG9iaik6IG9iamVjdChvYmopLCBzZXF1ZW5jZShv YmplY3Q6OnJlZmVyZW5jZSgpKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIG1hcHBpbmcob2JqZWN0OjpyZWZlcmVuY2UoKSkge30KICAgCiAgIGV4cGxpY2l0IHNlcW1h cChjb25zdCByZWYmIHJocyk6IG9iamVjdChyaHMpLCBzZXF1ZW5jZShvYmplY3Q6OnJlZmVyZW5j ZSgpKSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1hcHBpbmcob2JqZWN0 OjpyZWZlcmVuY2UoKSkge30KCiAgIGNvbnN0IG11dGFibGVfcHJveHkgb3BlcmF0b3JbXShpbnQg aSkgY29uc3Q7CiAgIG11dGFibGVfcHJveHkgb3BlcmF0b3JbXShpbnQgaSk7CgogICB0ZW1wbGF0 ZTxjbGFzcyBUPgogICBtdXRhYmxlX3Byb3h5IG9wZXJhdG9yW10oY29uc3QgVCAmIGtleSk7CiAg IAogICB0ZW1wbGF0ZTxjbGFzcyBUPgogICBjb25zdCBtdXRhYmxlX3Byb3h5IG9wZXJhdG9yW10o Y29uc3QgVCAmIGtleSkgY29uc3Q7CiAgIAogICAvLyBEZWZpbmVkIGJlY2F1c2UgdGhlIGRlZmF1 bHQgaW1wbGVtZW50YXRpb24gaXMgdW5zcGVjaWZpZWQgcmVnYXJkaW5nIAogICAvLyBob3cgb2Z0 ZW4gYSB2aXJ0dWFsIGJhc2UgY2xhc3MgaXMgYXNzaWduZWQKICAgc2VxbWFwJiBvcGVyYXRvcj0o Y29uc3Qgc2VxbWFwJiByaHMpCiAgIHsKICAgICAgdGhpcy0+b2JqZWN0X2Jhc2U6Om9wZXJhdG9y PShyaHMpOwogICAgICByZXR1cm4gKnRoaXM7CiAgIH0KICAgCn07CgoKY2xhc3Mgc2VxbWFwbnVt IDogcHVibGljIHNlcW1hcCwgcHVibGljIG51bWJlcgp7CnB1YmxpYzoKCiAgIHN0YXRpYyBib29s IGFjY2VwdHMocmVmIHApCiAgIHsKICAgICAgcmV0dXJuIHNlcW1hcDo6YWNjZXB0cyhwKSAmJiBu dW1iZXI6OmFjY2VwdHMocCk7CiAgIH0KCiAgIGV4cGxpY2l0IHNlcW1hcG51bShjb25zdCBvYmpl Y3QgJiBvYmopOiBvYmplY3Qob2JqKSwgc2VxbWFwKG9iamVjdDo6cmVmZXJlbmNlKCkpLAogICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbnVtYmVyKG9iamVjdDo6cmVm ZXJlbmNlKCkpIHt9CiAgIAogICBleHBsaWNpdCBzZXFtYXBudW0oY29uc3QgcmVmJiByaHMpOiBv YmplY3QocmhzKSwgc2VxbWFwKG9iamVjdDo6cmVmZXJlbmNlKCkpLAogICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBudW1iZXIob2JqZWN0OjpyZWZlcmVuY2UoKSkge30KCiAg IC8vIERlZmluZWQgYmVjYXVzZSB0aGUgZGVmYXVsdCBpbXBsZW1lbnRhdGlvbiBpcyB1bnNwZWNp ZmllZCByZWdhcmRpbmcgCiAgIC8vIGhvdyBvZnRlbiBhIHZpcnR1YWwgYmFzZSBjbGFzcyBpcyBh c3NpZ25lZAogICBzZXFtYXBudW0mIG9wZXJhdG9yPShjb25zdCBzZXFtYXBudW0mIHJocykKICAg ewogICAgICB0aGlzLT5vYmplY3RfYmFzZTo6b3BlcmF0b3I9KHJocyk7CiAgICAgIHJldHVybiAq dGhpczsKICAgfQoKICAgLy8gRXhwb3J0IHRoZSBudW1iZXI6Ois9IGluc3RlYWQgb2Ygc2VxdWVu Y2U6Ois9CiAgIHRlbXBsYXRlPHR5cGVuYW1lIFQ+CiAgIHNlcW1hcG51bSYgb3BlcmF0b3IrPShU IHgpCiAgIHsKICAgICAgdGhpcy0+bnVtYmVyOjpvcGVyYXRvcis9KHgpOwogICAgICByZXR1cm4g KnRoaXM7CiAgIH0KICAgCn07CgoKY2xhc3MgbXV0YWJsZV9wcm94eSA6IHB1YmxpYyBzZXFtYXBu dW0KewogcHVibGljOgogICAgZXhwbGljaXQgbXV0YWJsZV9wcm94eShib29zdDo6ZnVuY3Rpb248 cmVmPiBnZXRfY2FsbGJhY2spOiAgb2JqZWN0KHJlZigpKSwKICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgc2VxbWFwbnVtKG9iamVjdDo6cmVmZXJlbmNlKCkpLCBnZXRfKGdldF9jYWxsYmFj aykKICAgIHsgfQogICAgCiAgICBleHBsaWNpdCBtdXRhYmxlX3Byb3h5KGJvb3N0OjpmdW5jdGlv bjxyZWY+IGdldF9jYWxsYmFjaywKICAgICAgICAgICAgICAgICAgICAgICAgICAgYm9vc3Q6OmZ1 bmN0aW9uPHZvaWQscmVmPiBzZXRfY2FsbGJhY2spOiAgb2JqZWN0KHJlZigpKSwKICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgc2VxbWFwbnVtKG9iamVjdDo6cmVmZXJlbmNlKCkpLCBnZXRf KGdldF9jYWxsYmFjayksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHNldF8oc2V0X2Nh bGxiYWNrKSB7fQoKICAgCiAgIC8vIFJldHVybiBhIHJlZmVyZW5jZSB0byB0aGUgaGVsZCBvYmpl Y3QKICAgdmlydHVhbCByZWYgcmVmZXJlbmNlKCkgY29uc3QKICAgewogICAgICBpZiAoIW9iamVj dF9iYXNlOjpnZXQoKSkKICAgICAgICAgcmVzZXQoZ2V0XygpKTsKCiAgICAgIHJldHVybiBvYmpl Y3RfYmFzZTo6cmVmZXJlbmNlKCk7CiAgIH0KCiAgICAvLyBSZXR1cm4gYSByYXcgcG9pbnRlciB0 byB0aGUgaGVsZCBvYmplY3QKICAgIHZpcnR1YWwgUHlPYmplY3QqIGdldCgpIGNvbnN0CiAgICB7 CiAgICAgIGlmICghb2JqZWN0X2Jhc2U6OmdldCgpKQogICAgICAgICByZXNldChnZXRfKCkpOwog ICAgICByZXR1cm4gb2JqZWN0X2Jhc2U6OmdldCgpOyAKICAgIH0KCiAgIHRlbXBsYXRlPGNsYXNz IFQ+CiAgIG11dGFibGVfcHJveHkmIG9wZXJhdG9yPShjb25zdCBUJiByaHMpCiAgIHsKICAgICAg cmVmIHZhbChtYWtlX3JlZihyaHMpKTsKICAgICAgaWYgKHNldF8uZW1wdHkoKSkKICAgICAgICAg dGhyb3cgcnVudGltZV9lcnJvcigiQXNzaWdubWVudCBub3QgcG9zc2libGUhIik7CiAgICAgIHNl dF8odmFsKTsKICAgICAgcmVzZXQodmFsKTsKICAgICAgcmV0dXJuICp0aGlzOwogICB9CgogICBt dXRhYmxlX3Byb3h5JiBvcGVyYXRvcj0oY29uc3QgbXV0YWJsZV9wcm94eSYgcmhzKQogICB7CiAg ICAgIGlmICh0aGlzID09ICZyaHMpIHJldHVybiAqdGhpczsKICAgICAgaWYgKHNldF8uZW1wdHko KSkKICAgICAgICAgdGhyb3cgcnVudGltZV9lcnJvcigiQXNzaWdubWVudCBub3QgcG9zc2libGUh Iik7CiAgICAgIHNldF8ocmhzLnJlZmVyZW5jZSgpKTsKICAgICAgcmVzZXQocmhzLnJlZmVyZW5j ZSgpKTsKICAgICAgcmV0dXJuICp0aGlzOwogICB9CgogICB0ZW1wbGF0ZTx0eXBlbmFtZSBUPgog ICBtdXRhYmxlX3Byb3h5JiBvcGVyYXRvcis9KFQgeCkKICAgewogICAgICBzZXFtYXBudW06Om9w ZXJhdG9yKz0oeCk7CiAgICAgIHNldF8ocmVmZXJlbmNlKCkpOwogICAgICByZXR1cm4gKnRoaXM7 CiAgIH0KCnByaXZhdGU6CiAgIGJvb3N0OjpmdW5jdGlvbjxyZWY+IGdldF87CiAgIGJvb3N0Ojpm dW5jdGlvbjx2b2lkLHJlZj4gc2V0XzsKCn07IAoKCgoKaW5saW5lIG11dGFibGVfcHJveHkgc2Vx dWVuY2U6Om9wZXJhdG9yW10oaW50IGkpCnsKICAgcmV0dXJuIG11dGFibGVfcHJveHkoYm9vc3Q6 OmJpbmQoJnNlcXVlbmNlOjpnZXRfaXRlbSx0aGlzLGkpLCAKICAgICAgICAgICAgICAgICAgICAg ICAgYm9vc3Q6OmJpbmQoJnNlcXVlbmNlOjpyc2V0X2l0ZW0sdGhpcyxpLF8xKSk7Cn0KCmlubGlu ZSBjb25zdCBtdXRhYmxlX3Byb3h5IHNlcXVlbmNlOjpvcGVyYXRvcltdKGludCBpKSBjb25zdAp7 CiAgIHJldHVybiBtdXRhYmxlX3Byb3h5KGJvb3N0OjpiaW5kKCZzZXF1ZW5jZTo6Z2V0X2l0ZW0s dGhpcyxpKSk7Cn0KCgoKCnRlbXBsYXRlPGNsYXNzIFQ+CmlubGluZSBtdXRhYmxlX3Byb3h5IG1h cHBpbmc6Om9wZXJhdG9yW10oY29uc3QgVCAmIGtleSkKewogICByZWYga2V5cmVmKHRvX3B5dGhv bihrZXkpKTsKICAgcmV0dXJuIG11dGFibGVfcHJveHkoYm9vc3Q6OmJpbmQoJm1hcHBpbmc6OnJn ZXRfaXRlbSx0aGlzLGtleXJlZiksIAogICAgICAgICAgICAgICAgICAgICAgICBib29zdDo6Ymlu ZCgmbWFwcGluZzo6cnNldF9pdGVtLHRoaXMsa2V5cmVmLF8xKSk7Cn0KCgp0ZW1wbGF0ZTxjbGFz cyBUPgppbmxpbmUgY29uc3QgbXV0YWJsZV9wcm94eSBtYXBwaW5nOjpvcGVyYXRvcltdKGNvbnN0 IFQgJiBrZXkpIGNvbnN0CnsKICAgcmVmIGtleXJlZih0b19weXRob24oa2V5KSk7CiAgIHJldHVy biBtdXRhYmxlX3Byb3h5KGJvb3N0OjpiaW5kKCZtYXBwaW5nOjpyZ2V0X2l0ZW0sdGhpcyxrZXly ZWYpKTsgCn0KCgppbmxpbmUgbXV0YWJsZV9wcm94eSBzZXFtYXA6Om9wZXJhdG9yW10oaW50IGkp CnsKICAgcmV0dXJuIHNlcXVlbmNlOjpvcGVyYXRvcltdKGkpOwp9CgppbmxpbmUgY29uc3QgbXV0 YWJsZV9wcm94eSBzZXFtYXA6Om9wZXJhdG9yW10oaW50IGkpIGNvbnN0CnsKICAgcmV0dXJuIHNl cXVlbmNlOjpvcGVyYXRvcltdKGkpOwp9CgoKdGVtcGxhdGU8Y2xhc3MgVD4KaW5saW5lIG11dGFi bGVfcHJveHkgc2VxbWFwOjpvcGVyYXRvcltdKGNvbnN0IFQgJiBrZXkpCnsKICAgcmV0dXJuIG1h cHBpbmc6Om9wZXJhdG9yW10oa2V5KTsKfQoKdGVtcGxhdGU8Y2xhc3MgVD4KaW5saW5lIGNvbnN0 IG11dGFibGVfcHJveHkgc2VxbWFwOjpvcGVyYXRvcltdKGNvbnN0IFQgJiBrZXkpIGNvbnN0CnsK ICAgcmV0dXJuIG1hcHBpbmc6Om9wZXJhdG9yW10oa2V5KTsKfQogICAKICAgCgpjbGFzcyB0dXBs ZSA6IHB1YmxpYyBvYmplY3RfYmFzZQp7CiBwdWJsaWM6CiAgICBleHBsaWNpdCB0dXBsZShzdGQ6 OnNpemVfdCBuID0gMCk7CiAgICBleHBsaWNpdCB0dXBsZShyZWYgcCk7CgogICAgdGVtcGxhdGUg PGNsYXNzIEZpcnN0LCBjbGFzcyBTZWNvbmQ+CiAgICB0dXBsZShjb25zdCBzdGQ6OnBhaXI8Rmly c3QsU2Vjb25kPiYgeCkKICAgICAgICA6IG9iamVjdF9iYXNlKHJlZihQeVR1cGxlX05ldygyKSkp CiAgICB7CiAgICAgICAgc2V0X2l0ZW0oMCwgeC5maXJzdCk7CiAgICAgICAgc2V0X2l0ZW0oMSwg eC5zZWNvbmQpOwogICAgfQogICAgCiAgICB0ZW1wbGF0ZSA8Y2xhc3MgRmlyc3QsIGNsYXNzIFNl Y29uZD4KICAgIHR1cGxlKGNvbnN0IEZpcnN0JiBmaXJzdCwgY29uc3QgU2Vjb25kJiBzZWNvbmQp CiAgICAgICAgOiBvYmplY3RfYmFzZShyZWYoUHlUdXBsZV9OZXcoMikpKQogICAgewogICAgICAg IHNldF9pdGVtKDAsIGZpcnN0KTsKICAgICAgICBzZXRfaXRlbSgxLCBzZWNvbmQpOwogICAgfQog ICAgCiAgICB0ZW1wbGF0ZSA8Y2xhc3MgRmlyc3QsIGNsYXNzIFNlY29uZCwgY2xhc3MgVGhpcmQ+ CiAgICB0dXBsZShjb25zdCBGaXJzdCYgZmlyc3QsIGNvbnN0IFNlY29uZCYgc2Vjb25kLCBjb25z dCBUaGlyZCYgdGhpcmQpCiAgICAgICAgOiBvYmplY3RfYmFzZShyZWYoUHlUdXBsZV9OZXcoMykp KQogICAgewogICAgICAgIHNldF9pdGVtKDAsIGZpcnN0KTsKICAgICAgICBzZXRfaXRlbSgxLCBz ZWNvbmQpOwogICAgICAgIHNldF9pdGVtKDIsIHRoaXJkKTsKICAgIH0KICAgIAogICAgdGVtcGxh dGUgPGNsYXNzIEZpcnN0LCBjbGFzcyBTZWNvbmQsIGNsYXNzIFRoaXJkLCBjbGFzcyBGb3VydGg+ CiAgICB0dXBsZShjb25zdCBGaXJzdCYgZmlyc3QsIGNvbnN0IFNlY29uZCYgc2Vjb25kLCBjb25z dCBUaGlyZCYgdGhpcmQsIGNvbnN0IEZvdXJ0aCYgZm91cnRoKQogICAgICAgIDogb2JqZWN0X2Jh c2UocmVmKFB5VHVwbGVfTmV3KDQpKSkKICAgIHsKICAgICAgICBzZXRfaXRlbSgwLCBmaXJzdCk7 CiAgICAgICAgc2V0X2l0ZW0oMSwgc2Vjb25kKTsKICAgICAgICBzZXRfaXRlbSgyLCB0aGlyZCk7 CiAgICAgICAgc2V0X2l0ZW0oMywgZm91cnRoKTsKICAgIH0KICAgIAogICAgc3RhdGljIFB5VHlw ZU9iamVjdCogdHlwZV9vYmooKTsKICAgIHN0YXRpYyBib29sIGFjY2VwdHMocmVmIHApOwogICAg c3RkOjpzaXplX3Qgc2l6ZSgpIGNvbnN0OwogICAgcmVmIG9wZXJhdG9yW10oc3RkOjpzaXplX3Qg cG9zKSBjb25zdDsKCiAgICB0ZW1wbGF0ZSA8Y2xhc3MgVD4KICAgIHZvaWQgc2V0X2l0ZW0oc3Rk OjpzaXplX3QgcG9zLCBjb25zdCBUJiByaHMpCiAgICB7CiAgICAgICAgdGhpcy0+c2V0X2l0ZW0o cG9zLCBtYWtlX3JlZihyaHMpKTsKICAgIH0KICAgIAogICAgdm9pZCBzZXRfaXRlbShzdGQ6OnNp emVfdCBwb3MsIGNvbnN0IHJlZiYgcmhzKTsKICAgIAogICAgdHVwbGUgc2xpY2UoaW50IGxvdywg aW50IGhpZ2gpIGNvbnN0OwoKICAgIGZyaWVuZCB0dXBsZSBvcGVyYXRvcisoY29uc3QgdHVwbGUm LCBjb25zdCB0dXBsZSYpOwogICAgdHVwbGUmIG9wZXJhdG9yKz0oY29uc3QgdHVwbGUmIHJocyk7 Cn07CgoKc3RydWN0IGxpc3RfaXRlcmF0b3JfcG9saWNpZXMgOiBwdWJsaWMgYm9vc3Q6OmRlZmF1 bHRfaXRlcmF0b3JfcG9saWNpZXMKewogICB0ZW1wbGF0ZTwgY2xhc3MgSXRlcmF0b3JBZGFwdG9y PgogICB0eXBlbmFtZSBJdGVyYXRvckFkYXB0b3I6OnJlZmVyZW5jZSBkZXJlZmVyZW5jZShjb25z dCBJdGVyYXRvckFkYXB0b3ImIGl0ZXIpIGNvbnN0CiAgIHsKICAgICAgdHlwZWRlZiB0eXBlbmFt ZSBJdGVyYXRvckFkYXB0b3I6OnJlZmVyZW5jZSByZXN1bHRfdDsKICAgICAgcmV0dXJuIHJlc3Vs dF90KGl0ZXIuYmFzZSgpKTsKICAgfQogIAp9OwoKCmNsYXNzIGxpc3QgOiBwdWJsaWMgb2JqZWN0 X2Jhc2UKewogICAgc3RydWN0IGl0ZXJhdG9yX3Byb3h5IDogcHVibGljIHNlcW1hcAogICAgewog ICAgICAgZXhwbGljaXQgaXRlcmF0b3JfcHJveHkoUHlPYmplY3QqKiBfYmFzZSkgOiBvYmplY3Qo cmVmKCpfYmFzZSkpLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHNlcW1hcChv YmplY3Q6OnJlZmVyZW5jZSgpKSAsIGJhc2UoX2Jhc2UpIHsgfQogICAgICAgCiAgICAgIHRlbXBs YXRlPGNsYXNzIFQ+CiAgICAgIGl0ZXJhdG9yX3Byb3h5JiBvcGVyYXRvcj0oY29uc3QgVCYgcmhz KQogICAgICB7CiAgICAgICAgIHJlc2V0KG1ha2VfcmVmKHJocykpOwogICAgICAgICAqYmFzZSA9 IGdldCgpOwogICAgICAgICByZXR1cm4gKnRoaXM7CiAgICAgIH0KCiAgICAgIGl0ZXJhdG9yX3By b3h5JiBvcGVyYXRvcj0oY29uc3QgaXRlcmF0b3JfcHJveHkmIHJocykKICAgICAgewogICAgICAg ICByZXNldChyaHMucmVmZXJlbmNlKCkpOwogICAgICAgICAqYmFzZSA9IGdldCgpOwogICAgICAg ICByZXR1cm4gKnRoaXM7CiAgICAgIH0KICAgICAgCiAgICAgIHByaXZhdGU6CiAgICAgICAgIFB5 T2JqZWN0KiogYmFzZTsKICAgIH07CiAgICAKICAgIAogICAgCiAgICBzdHJ1Y3QgcHJveHk7CiAg ICBzdHJ1Y3Qgc2xpY2VfcHJveHk7CiBwdWJsaWM6CiAKICAgIHR5cGVkZWYgYm9vc3Q6Oml0ZXJh dG9yX2FkYXB0b3I8UHlPYmplY3QqKixsaXN0X2l0ZXJhdG9yX3BvbGljaWVzLAogICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBib29zdDo6aXRlcmF0b3JfY2F0ZWdvcnlfaXM8c3Rk OjppbnB1dF9pdGVyYXRvcl90YWc+LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBib29zdDo6dmFsdWVfdHlwZV9pczxpdGVyYXRvcl9wcm94eT4sCiAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIGJvb3N0OjpyZWZlcmVuY2VfaXM8aXRlcmF0b3JfcHJveHk+ID4g aXRlcmF0b3I7CgogICAgdHlwZWRlZiBib29zdDo6aXRlcmF0b3JfYWRhcHRvcjxQeU9iamVjdCoq LGxpc3RfaXRlcmF0b3JfcG9saWNpZXMsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIGJvb3N0OjppdGVyYXRvcl9jYXRlZ29yeV9pczxzdGQ6OmlucHV0X2l0ZXJhdG9yX3RhZz4s CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJvb3N0Ojp2YWx1ZV90eXBlX2lz PGNvbnN0IGl0ZXJhdG9yX3Byb3h5PiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgYm9vc3Q6OnJlZmVyZW5jZV9pczxjb25zdCBpdGVyYXRvcl9wcm94eT4gPiBjb25zdF9pdGVy YXRvcjsKIAogICAgZXhwbGljaXQgbGlzdChjb25zdCBvYmplY3QgJiBvKTogb2JqZWN0X2Jhc2Uo by5yZWZlcmVuY2UoKSkge30KICAgIGV4cGxpY2l0IGxpc3QocmVmIHApOwogICAgZXhwbGljaXQg bGlzdChzdGQ6OnNpemVfdCBzeiA9IDApOwogICAgc3RhdGljIFB5VHlwZU9iamVjdCogdHlwZV9v YmooKTsKICAgIHN0YXRpYyBib29sIGFjY2VwdHMocmVmIHApOwogICAgc3RkOjpzaXplX3Qgc2l6 ZSgpIGNvbnN0OwogICAgY29uc3QgbXV0YWJsZV9wcm94eSBvcGVyYXRvcltdKHN0ZDo6c2l6ZV90 IHBvcykgY29uc3QKICAgIHsKICAgICAgIHJldHVybiBtdXRhYmxlX3Byb3h5KGJvb3N0OjpiaW5k KCZsaXN0OjpnZXRfaXRlbSx0aGlzLHBvcykpOwogICAgfQoKICAgIG11dGFibGVfcHJveHkgb3Bl cmF0b3JbXShzdGQ6OnNpemVfdCBwb3MpCiAgICB7CiAgICAgICByZXR1cm4gbXV0YWJsZV9wcm94 eShib29zdDo6YmluZCgmbGlzdDo6Z2V0X2l0ZW0sdGhpcyxwb3MpLCAKICAgICAgICAgICAgICAg ICAgICAgICAgYm9vc3Q6OmJpbmQoJmxpc3Q6OnNldF9pdGVtPHJlZj4sdGhpcyxwb3MsXzEpKTsK ICAgIH0KCiAgICByZWYgZ2V0X2l0ZW0oc3RkOjpzaXplX3QgcG9zKSBjb25zdDsKCiAgICB0ZW1w bGF0ZSA8Y2xhc3MgVD4KICAgIHZvaWQgc2V0X2l0ZW0oc3RkOjpzaXplX3QgcG9zLCBjb25zdCBU JiB4KQogICAgICAgIHsgdGhpcy0+c2V0X2l0ZW0ocG9zLCBtYWtlX3JlZih4KSk7IH0KICAgIHZv aWQgc2V0X2l0ZW0oc3RkOjpzaXplX3QgcG9zLCBjb25zdCByZWYmICk7CiAgICAKCiAgICB0ZW1w bGF0ZSA8Y2xhc3MgVD4KICAgIHZvaWQgaW5zZXJ0KHN0ZDo6c2l6ZV90IGluZGV4LCBjb25zdCBU JiB4KQogICAgICAgIHsgdGhpcy0+aW5zZXJ0KGluZGV4LCBtYWtlX3JlZih4KSk7IH0KICAgIHZv aWQgaW5zZXJ0KHN0ZDo6c2l6ZV90IGluZGV4LCBjb25zdCByZWYmIGl0ZW0pOwoKICAgIHRlbXBs YXRlIDxjbGFzcyBUPgogICAgdm9pZCBwdXNoX2JhY2soY29uc3QgVCYgaXRlbSkKICAgICAgICB7 IHRoaXMtPnB1c2hfYmFjayhtYWtlX3JlZihpdGVtKSk7IH0KICAgIHZvaWQgcHVzaF9iYWNrKGNv bnN0IHJlZiYgaXRlbSk7CiAgICAKICAgIHRlbXBsYXRlIDxjbGFzcyBUPgogICAgdm9pZCBhcHBl bmQoY29uc3QgVCYgaXRlbSkKICAgICAgICB7IHRoaXMtPmFwcGVuZChtYWtlX3JlZihpdGVtKSk7 IH0KICAgIHZvaWQgYXBwZW5kKGNvbnN0IHJlZiYgaXRlbSk7CiAgICAKICAgIGxpc3Qgc2xpY2Uo aW50IGxvdywgaW50IGhpZ2gpIGNvbnN0OwogICAgc2xpY2VfcHJveHkgc2xpY2UoaW50IGxvdywg aW50IGhpZ2gpOwogICAgdm9pZCBzb3J0KCk7CiAgICB2b2lkIHJldmVyc2UoKTsKICAgIHR1cGxl IGFzX3R1cGxlKCkgY29uc3Q7CiAgICAKICAgIGl0ZXJhdG9yIGJlZ2luKCkKICAgIHsKICAgICAg IHJldHVybiBpdGVyYXRvcihyZWludGVycHJldF9jYXN0PFB5TGlzdE9iamVjdCo+KGdldCgpKS0+ b2JfaXRlbSk7CiAgICB9CgogICAgY29uc3RfaXRlcmF0b3IgYmVnaW4oKSBjb25zdAogICAgewog ICAgICAgcmV0dXJuIGNvbnN0X2l0ZXJhdG9yKHJlaW50ZXJwcmV0X2Nhc3Q8UHlMaXN0T2JqZWN0 Kj4oZ2V0KCkpLT5vYl9pdGVtKTsKICAgIH0KCiAgICBpdGVyYXRvciBlbmQoKQogICAgewogICAg ICAgUHlMaXN0T2JqZWN0KiBwdHJfbGlzdCA9IHJlaW50ZXJwcmV0X2Nhc3Q8UHlMaXN0T2JqZWN0 Kj4oZ2V0KCkpOwogICAgICAgcmV0dXJuIGl0ZXJhdG9yKHB0cl9saXN0LT5vYl9pdGVtICsgcHRy X2xpc3QtPm9iX3NpemUpOwogICAgfQoKICAgIGNvbnN0X2l0ZXJhdG9yIGVuZCgpIGNvbnN0CiAg ICB7CiAgICAgICBQeUxpc3RPYmplY3QqIHB0cl9saXN0ID0gcmVpbnRlcnByZXRfY2FzdDxQeUxp c3RPYmplY3QqPihnZXQoKSk7CiAgICAgICByZXR1cm4gY29uc3RfaXRlcmF0b3IocHRyX2xpc3Qt Pm9iX2l0ZW0gKyBwdHJfbGlzdC0+b2Jfc2l6ZSk7CiAgICB9Cgp9OwoKCnRlbXBsYXRlPGNsYXNz IFQ+CnZvaWQgc2VxdWVuY2U6OmFwcGVuZChjb25zdCBUJiB2YWwpCnsKICAgbGlzdCB0bXA7CiAg IHRtcC5hcHBlbmQodmFsKTsKICAgKnRoaXMgKz0gdG1wOwp9CgoKCmNsYXNzIHN0cmluZwogICAg OiBwdWJsaWMgb2JqZWN0X2Jhc2UsIHB1YmxpYyBib29zdDo6bXVsdGlwbGlhYmxlMjxzdHJpbmcs IHVuc2lnbmVkIGludD4KewogcHVibGljOgogICAgLy8gQ29uc3RydWN0IGZyb20gYW4gb3duZWQg UHlPYmplY3QqLgogICAgLy8gUHJlY29uZGl0aW9uOiBwIG11c3QgcG9pbnQgdG8gYSBweXRob24g c3RyaW5nLgogICAgZXhwbGljaXQgc3RyaW5nKHJlZiBwKTsKICAgIGV4cGxpY2l0IHN0cmluZyhj b25zdCBjaGFyKiBzKTsKICAgIHN0cmluZyhjb25zdCBjaGFyKiBzLCBzdGQ6OnNpemVfdCBsZW5n dGgpOwogICAgc3RyaW5nKGNvbnN0IHN0cmluZyYgcmhzKTsKCiAgICBlbnVtIGludGVybmVkX3Qg eyBpbnRlcm5lZCB9OwogICAgc3RyaW5nKGNvbnN0IGNoYXIqIHMsIGludGVybmVkX3QpOwogICAg CiAgICAvLyBHZXQgdGhlIHR5cGUgb2JqZWN0IGZvciBTdHJpbmdzCiAgICBzdGF0aWMgUHlUeXBl T2JqZWN0KiB0eXBlX29iaigpOwoKICAgIC8vIFJldHVybiB0cnVlIGlmIHRoZSBnaXZlbiBvYmpl Y3QgaXMgYSBweXRob24gc3RyaW5nCiAgICBzdGF0aWMgYm9vbCBhY2NlcHRzKHJlZiBvKTsKCiAg ICAvLyBSZXR1cm4gdGhlIGxlbmd0aCBvZiB0aGUgc3RyaW5nLgogICAgc3RkOjpzaXplX3Qgc2l6 ZSgpIGNvbnN0OwoKICAgIC8vIFJldHVybnMgYSBudWxsLXRlcm1pbmF0ZWQgcmVwcmVzZW50YXRp b24gb2YgdGhlIGNvbnRlbnRzIG9mIHN0cmluZy4KICAgIC8vIFRoZSBwb2ludGVyIHJlZmVycyB0 byB0aGUgaW50ZXJuYWwgYnVmZmVyIG9mIHN0cmluZywgbm90IGEgY29weS4KICAgIC8vIFRoZSBk YXRhIG11c3Qgbm90IGJlIG1vZGlmaWVkIGluIGFueSB3YXkuIEl0IG11c3Qgbm90IGJlIGRlLWFs bG9jYXRlZC4gCiAgICBjb25zdCBjaGFyKiBjX3N0cigpIGNvbnN0OwoKICAgIHN0cmluZyYgb3Bl cmF0b3IqPSh1bnNpZ25lZCBpbnQgcmVwZWF0X2NvdW50KTsKICAgIHN0cmluZyYgb3BlcmF0b3Ir PShjb25zdCBzdHJpbmcmIHJocyk7CiAgICBmcmllbmQgc3RyaW5nIG9wZXJhdG9yKyhzdHJpbmcg eCwgc3RyaW5nIHkpOwogICAgc3RyaW5nJiBvcGVyYXRvcis9KGNvbnN0IGNoYXIqIHJocyk7CiAg ICBmcmllbmQgc3RyaW5nIG9wZXJhdG9yKyhzdHJpbmcgeCwgY29uc3QgY2hhciogeSk7CiAgICBm cmllbmQgc3RyaW5nIG9wZXJhdG9yKyhjb25zdCBjaGFyKiB4LCBzdHJpbmcgeSk7CgogICAgdm9p ZCBpbnRlcm4oKTsKCiAgICBmcmllbmQgc3RyaW5nIG9wZXJhdG9yJShjb25zdCBzdHJpbmcmIGZv cm1hdCwgY29uc3QgdHVwbGUmIGFyZ3MpOwp9OwoKY2xhc3MgZGljdGlvbmFyeSA6IHB1YmxpYyBv YmplY3RfYmFzZQp7CiBwcml2YXRlOgogICAgc3RydWN0IHByb3h5OwogICAgCiBwdWJsaWM6CiAg ICBleHBsaWNpdCBkaWN0aW9uYXJ5KGNvbnN0IG9iamVjdCAmIG8pOiBvYmplY3RfYmFzZShvLnJl ZmVyZW5jZSgpKSB7fQogICAgZXhwbGljaXQgZGljdGlvbmFyeShyZWYgcCk7CiAgICBkaWN0aW9u YXJ5KCk7CiAgICB2b2lkIGNsZWFyKCk7CgogICAgc3RhdGljIFB5VHlwZU9iamVjdCogdHlwZV9v YmooKTsKICAgIHN0YXRpYyBib29sIGFjY2VwdHMocmVmIHApOwogICAgCiBwdWJsaWM6CiAgICB0 ZW1wbGF0ZSA8Y2xhc3MgS2V5PgogICAgcHJveHkgb3BlcmF0b3JbXShjb25zdCBLZXkmIGtleSkK ICAgICAgICB7IHJldHVybiB0aGlzLT5vcGVyYXRvcltdKG1ha2VfcmVmKGtleSkpOyB9CiAgICBw cm94eSBvcGVyYXRvcltdKHJlZiBrZXkpOwogICAgCiAgICB0ZW1wbGF0ZSA8Y2xhc3MgS2V5Pgog ICAgcmVmIG9wZXJhdG9yW10oY29uc3QgS2V5JiBrZXkpIGNvbnN0CiAgICAgICAgeyByZXR1cm4g dGhpcy0+b3BlcmF0b3JbXShtYWtlX3JlZihrZXkpKTsgfQogICAgcmVmIG9wZXJhdG9yW10ocmVm IGtleSkgY29uc3Q7CgogICAgdGVtcGxhdGUgPGNsYXNzIEtleT4KICAgIHJlZiBnZXRfaXRlbShj b25zdCBLZXkmIGtleSkgY29uc3QKICAgICAgICB7IHJldHVybiB0aGlzLT5nZXRfaXRlbShtYWtl X3JlZihrZXkpKTsgfQogICAgcmVmIGdldF9pdGVtKGNvbnN0IHJlZiYga2V5KSBjb25zdDsKICAg IAogICAgdGVtcGxhdGUgPGNsYXNzIEtleSwgY2xhc3MgRGVmYXVsdD4KICAgIHJlZiBnZXRfaXRl bShjb25zdCBLZXkmIGtleSwgY29uc3QgRGVmYXVsdCYgZGVmYXVsdF8pIGNvbnN0CiAgICAgICAg eyByZXR1cm4gdGhpcy0+Z2V0X2l0ZW0obWFrZV9yZWYoa2V5KSwgbWFrZV9yZWYoZGVmYXVsdF8p KTsgfQogICAgcmVmIGdldF9pdGVtKGNvbnN0IHJlZiYga2V5LCBjb25zdCByZWYmIGRlZmF1bHRf KSBjb25zdDsKICAgIAogICAgdGVtcGxhdGUgPGNsYXNzIEtleSwgY2xhc3MgVmFsdWU+CiAgICB2 b2lkIHNldF9pdGVtKGNvbnN0IEtleSYga2V5LCBjb25zdCBWYWx1ZSYgdmFsdWUpCiAgICAgICAg eyB0aGlzLT5zZXRfaXRlbShtYWtlX3JlZihrZXkpLCBtYWtlX3JlZih2YWx1ZSkpOyB9CiAgICB2 b2lkIHNldF9pdGVtKGNvbnN0IHJlZiYga2V5LCBjb25zdCByZWYmIHZhbHVlKTsKCiAgICB0ZW1w bGF0ZSA8Y2xhc3MgS2V5PgogICAgdm9pZCBlcmFzZShjb25zdCBLZXkmIGtleSkKICAgICAgICB7 IHRoaXMtPmVyYXNlKG1ha2VfcmVmKGtleSkpOyB9CiAgICB2b2lkIGVyYXNlKHJlZiBrZXkpOwoK Ly8gICAgcHJveHkgb3BlcmF0b3JbXShjb25zdCBvYmplY3QmIGtleSk7Ci8vICAgIHJlZiBvcGVy YXRvcltdKGNvbnN0IG9iamVjdCYga2V5KSBjb25zdDsKCi8vICAgIHJlZiBnZXRfaXRlbShjb25z dCBvYmplY3QmIGtleSwgcmVmIGRlZmF1bHRfID0gcmVmKCkpIGNvbnN0OwovLyAgICB2b2lkIHNl dF9pdGVtKGNvbnN0IG9iamVjdCYga2V5LCBjb25zdCByZWYmIHZhbHVlKTsKICAgICAgICAKLy8g ICAgdm9pZCBlcmFzZShjb25zdCBvYmplY3QmIGtleSk7CgogICAgbGlzdCBpdGVtcygpIGNvbnN0 OwogICAgbGlzdCBrZXlzKCkgY29uc3Q7CiAgICBsaXN0IHZhbHVlcygpIGNvbnN0OwoKICAgIHN0 ZDo6c2l6ZV90IHNpemUoKSBjb25zdDsKICAgIC8vIFRPRE86IGl0ZXJhdG9yIHN1cHBvcnQKfTsK CnN0cnVjdCBkaWN0aW9uYXJ5Ojpwcm94eQp7CiAgICB0ZW1wbGF0ZSA8Y2xhc3MgVD4KICAgIGNv bnN0IHJlZiYgb3BlcmF0b3I9KGNvbnN0IFQmIHJocykKICAgICAgICB7IHJldHVybiAoKnRoaXMp ID0gbWFrZV9yZWYocmhzKTsgfQogICAgY29uc3QgcmVmJiBvcGVyYXRvcj0oY29uc3QgcmVmJiBy aHMpOwoKICAgIG9wZXJhdG9yIHJlZigpIGNvbnN0OwogcHJpdmF0ZToKICAgIGZyaWVuZCBjbGFz cyBkaWN0aW9uYXJ5OwogICAgcHJveHkoY29uc3QgcmVmJiBkaWN0LCBjb25zdCByZWYmIGtleSk7 CgogICAgLy8gVGhpcyBpcyBuZWVkZWQgdG8gd29yayBhcm91bmQgdGhlIHZlcnkgc3RyYW5nZSBN U1ZDIGVycm9yIHJlcG9ydCB0aGF0IHRoZQogICAgLy8gcmV0dXJuIHR5cGUgb2YgdGhlIGJ1aWx0 LWluIG9wZXJhdG9yPSBkaWZmZXJzIGZyb20gdGhhdCBvZiB0aGUgb25lcwogICAgLy8gZGVmaW5l ZCBhYm92ZS4gQ291bGRuJ3QgaHVydCB0byBtYWtlIHRoZXNlIHVuLWFzc2lnbmFibGUgYW55d2F5 LCB0aG91Z2guCiAgICBjb25zdCByZWYmIG9wZXJhdG9yPShjb25zdCBwcm94eSYpOyAvLyBOb3Qg YWN0dWFsbHkgaW1wbGVtZW50ZWQKIHByaXZhdGU6CiAgICByZWYgbV9kaWN0OwogICAgcmVmIG1f a2V5Owp9OwoKc3RydWN0IGxpc3Q6OnByb3h5IDogcHVibGljIHNlcW1hcG51bQp7CiAgICB0ZW1w bGF0ZSA8Y2xhc3MgVD4KICAgIGNvbnN0IHJlZiYgb3BlcmF0b3I9KGNvbnN0IFQmIHJocykKICAg ICAgICB7IHJldHVybiAoKnRoaXMpID0gbWFrZV9yZWYocmhzKTsgfQogICAgY29uc3QgcmVmJiBv cGVyYXRvcj0oY29uc3QgcmVmJiByaHMpOwogICAgCiAgICAvL29wZXJhdG9yIHJlZigpIGNvbnN0 OwogICAgCiBwcml2YXRlOgogICAgcHJveHkoY29uc3QgcHJveHkmIHJocykgOiBvYmplY3Qocmhz KSwgc2VxbWFwbnVtKG9iamVjdDo6cmVmZXJlbmNlKCkpICwgCiAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIG1fbGlzdChyaHMubV9saXN0KSAsIG1faW5kZXgocmhzLm1faW5kZXgpIHt9CiAg ICBmcmllbmQgY2xhc3MgbGlzdDsKICAgIHByb3h5KGNvbnN0IHJlZiYgbGlzdCwgc3RkOjpzaXpl X3QgaW5kZXgpOwogICAgCiAgICAvLyBUaGlzIGlzIG5lZWRlZCB0byB3b3JrIGFyb3VuZCB0aGUg dmVyeSBzdHJhbmdlIE1TVkMgZXJyb3IgcmVwb3J0IHRoYXQgdGhlCiAgICAvLyByZXR1cm4gdHlw ZSBvZiB0aGUgYnVpbHQtaW4gb3BlcmF0b3I9IGRpZmZlcnMgZnJvbSB0aGF0IG9mIHRoZSBvbmVz CiAgICAvLyBkZWZpbmVkIGFib3ZlLiBDb3VsZG4ndCBodXJ0IHRvIG1ha2UgdGhlc2UgdW4tYXNz aWduYWJsZSBhbnl3YXksIHRob3VnaC4KICAgIGNvbnN0IHJlZiYgb3BlcmF0b3I9KGNvbnN0IHBy b3h5Jik7IC8vIE5vdCBhY3R1YWxseSBpbXBsZW1lbnRlZAogcHJpdmF0ZToKICAgIGxpc3QgbV9s aXN0OwogICAgc3RkOjpzaXplX3QgbV9pbmRleDsKfTsKCnN0cnVjdCBsaXN0OjpzbGljZV9wcm94 eQp7CiAgICBjb25zdCBsaXN0JiBvcGVyYXRvcj0oY29uc3QgbGlzdCYgcmhzKTsKICAgIG9wZXJh dG9yIHJlZigpIGNvbnN0OwogICAgb3BlcmF0b3IgbGlzdCgpIGNvbnN0OwogICAgc3RkOjpzaXpl X3Qgc2l6ZSgpIGNvbnN0OwogICAgcmVmIG9wZXJhdG9yW10oc3RkOjpzaXplX3QgcG9zKSBjb25z dDsKIHByaXZhdGU6CiAgICBmcmllbmQgY2xhc3MgbGlzdDsKICAgIHNsaWNlX3Byb3h5KGNvbnN0 IHJlZiYgbGlzdCwgaW50IGxvdywgaW50IGhpZ2gpOwogcHJpdmF0ZToKICAgIHJlZiBtX2xpc3Q7 CiAgICBpbnQgbV9sb3csIG1faGlnaDsKfTsKCn19IC8vIG5hbWVzcGFjZSBib29zdDo6cHl0aG9u CgpCT09TVF9QWVRIT05fQkVHSU5fQ09OVkVSU0lPTl9OQU1FU1BBQ0UKCi8vIEdlbmVyaWMgb2Jq ZWN0cwoKUHlPYmplY3QqIHRvX3B5dGhvbihjb25zdCBib29zdDo6cHl0aG9uOjpvYmplY3QmKTsK Ym9vc3Q6OnB5dGhvbjo6b2JqZWN0IGZyb21fcHl0aG9uKFB5T2JqZWN0KiBwLCBib29zdDo6cHl0 aG9uOjp0eXBlPGJvb3N0OjpweXRob246Om9iamVjdD4pOwoKaW5saW5lIGJvb3N0OjpweXRob246 Om9iamVjdCBmcm9tX3B5dGhvbihQeU9iamVjdCogcCwgYm9vc3Q6OnB5dGhvbjo6dHlwZTxjb25z dCBib29zdDo6cHl0aG9uOjpvYmplY3QmPikKewogICAgcmV0dXJuIGZyb21fcHl0aG9uKHAsIGJv b3N0OjpweXRob246OnR5cGU8Ym9vc3Q6OnB5dGhvbjo6b2JqZWN0PigpKTsKfQoKUHlPYmplY3Qq IHRvX3B5dGhvbihjb25zdCBib29zdDo6cHl0aG9uOjpzZXF1ZW5jZSYpOwpib29zdDo6cHl0aG9u OjpzZXF1ZW5jZSBmcm9tX3B5dGhvbihQeU9iamVjdCogcCwgYm9vc3Q6OnB5dGhvbjo6dHlwZTxi b29zdDo6cHl0aG9uOjpzZXF1ZW5jZT4pOwoKaW5saW5lIGJvb3N0OjpweXRob246OnNlcXVlbmNl IGZyb21fcHl0aG9uKFB5T2JqZWN0KiBwLCBib29zdDo6cHl0aG9uOjp0eXBlPGNvbnN0IGJvb3N0 OjpweXRob246OnNlcXVlbmNlJj4pCnsKICAgIHJldHVybiBmcm9tX3B5dGhvbihwLCBib29zdDo6 cHl0aG9uOjp0eXBlPGJvb3N0OjpweXRob246OnNlcXVlbmNlPigpKTsKfQoKClB5T2JqZWN0KiB0 b19weXRob24oY29uc3QgYm9vc3Q6OnB5dGhvbjo6bnVtYmVyJik7CmJvb3N0OjpweXRob246Om51 bWJlciBmcm9tX3B5dGhvbihQeU9iamVjdCogcCwgYm9vc3Q6OnB5dGhvbjo6dHlwZTxib29zdDo6 cHl0aG9uOjpudW1iZXI+KTsKCmlubGluZSBib29zdDo6cHl0aG9uOjpudW1iZXIgZnJvbV9weXRo b24oUHlPYmplY3QqIHAsIGJvb3N0OjpweXRob246OnR5cGU8Y29uc3QgYm9vc3Q6OnB5dGhvbjo6 bnVtYmVyJj4pCnsKICAgIHJldHVybiBmcm9tX3B5dGhvbihwLCBib29zdDo6cHl0aG9uOjp0eXBl PGJvb3N0OjpweXRob246Om51bWJlcj4oKSk7Cn0KCgpQeU9iamVjdCogdG9fcHl0aG9uKGNvbnN0 IGJvb3N0OjpweXRob246Om1hcHBpbmcmKTsKYm9vc3Q6OnB5dGhvbjo6bWFwcGluZyBmcm9tX3B5 dGhvbihQeU9iamVjdCogcCwgYm9vc3Q6OnB5dGhvbjo6dHlwZTxib29zdDo6cHl0aG9uOjptYXBw aW5nPik7CgppbmxpbmUgYm9vc3Q6OnB5dGhvbjo6bWFwcGluZyBmcm9tX3B5dGhvbihQeU9iamVj dCogcCwgYm9vc3Q6OnB5dGhvbjo6dHlwZTxjb25zdCBib29zdDo6cHl0aG9uOjptYXBwaW5nJj4p CnsKICAgIHJldHVybiBmcm9tX3B5dGhvbihwLCBib29zdDo6cHl0aG9uOjp0eXBlPGJvb3N0Ojpw eXRob246Om1hcHBpbmc+KCkpOwp9CgpQeU9iamVjdCogdG9fcHl0aG9uKGNvbnN0IGJvb3N0Ojpw eXRob246OnNlcW1hcCYpOwpib29zdDo6cHl0aG9uOjpzZXFtYXAgZnJvbV9weXRob24oUHlPYmpl Y3QqIHAsIGJvb3N0OjpweXRob246OnR5cGU8Ym9vc3Q6OnB5dGhvbjo6c2VxbWFwPik7Cgppbmxp bmUgYm9vc3Q6OnB5dGhvbjo6c2VxbWFwIGZyb21fcHl0aG9uKFB5T2JqZWN0KiBwLCBib29zdDo6 cHl0aG9uOjp0eXBlPGNvbnN0IGJvb3N0OjpweXRob246OnNlcW1hcCY+KQp7CiAgICByZXR1cm4g ZnJvbV9weXRob24ocCwgYm9vc3Q6OnB5dGhvbjo6dHlwZTxib29zdDo6cHl0aG9uOjpzZXFtYXA+ KCkpOwp9CgpQeU9iamVjdCogdG9fcHl0aG9uKGNvbnN0IGJvb3N0OjpweXRob246OnNlcW1hcG51 bSYpOwpib29zdDo6cHl0aG9uOjpzZXFtYXBudW0gZnJvbV9weXRob24oUHlPYmplY3QqIHAsIGJv b3N0OjpweXRob246OnR5cGU8Ym9vc3Q6OnB5dGhvbjo6c2VxbWFwbnVtPik7CgppbmxpbmUgYm9v c3Q6OnB5dGhvbjo6c2VxbWFwbnVtIGZyb21fcHl0aG9uKFB5T2JqZWN0KiBwLCBib29zdDo6cHl0 aG9uOjp0eXBlPGNvbnN0IGJvb3N0OjpweXRob246OnNlcW1hcG51bSY+KQp7CiAgICByZXR1cm4g ZnJvbV9weXRob24ocCwgYm9vc3Q6OnB5dGhvbjo6dHlwZTxib29zdDo6cHl0aG9uOjpzZXFtYXBu dW0+KCkpOwp9CgpQeU9iamVjdCogdG9fcHl0aG9uKGNvbnN0IGJvb3N0OjpweXRob246Om11dGFi bGVfcHJveHkmKTsKCgoKCi8vIENvbmNyZXRlIG9iamVjdHMKCgpQeU9iamVjdCogdG9fcHl0aG9u KGNvbnN0IGJvb3N0OjpweXRob246OnR1cGxlJik7CmJvb3N0OjpweXRob246OnR1cGxlIGZyb21f cHl0aG9uKFB5T2JqZWN0KiBwLCBib29zdDo6cHl0aG9uOjp0eXBlPGJvb3N0OjpweXRob246OnR1 cGxlPik7CgppbmxpbmUgYm9vc3Q6OnB5dGhvbjo6dHVwbGUgZnJvbV9weXRob24oUHlPYmplY3Qq IHAsIGJvb3N0OjpweXRob246OnR5cGU8Y29uc3QgYm9vc3Q6OnB5dGhvbjo6dHVwbGUmPikKewog ICAgcmV0dXJuIGZyb21fcHl0aG9uKHAsIGJvb3N0OjpweXRob246OnR5cGU8Ym9vc3Q6OnB5dGhv bjo6dHVwbGU+KCkpOwp9CgpQeU9iamVjdCogdG9fcHl0aG9uKGNvbnN0IGJvb3N0OjpweXRob246 Omxpc3QmKTsKYm9vc3Q6OnB5dGhvbjo6bGlzdCBmcm9tX3B5dGhvbihQeU9iamVjdCogcCwgYm9v c3Q6OnB5dGhvbjo6dHlwZTxib29zdDo6cHl0aG9uOjpsaXN0Pik7CgppbmxpbmUgYm9vc3Q6OnB5 dGhvbjo6bGlzdCBmcm9tX3B5dGhvbihQeU9iamVjdCogcCwgYm9vc3Q6OnB5dGhvbjo6dHlwZTxj b25zdCBib29zdDo6cHl0aG9uOjpsaXN0Jj4pCnsKICAgIHJldHVybiBmcm9tX3B5dGhvbihwLCBi b29zdDo6cHl0aG9uOjp0eXBlPGJvb3N0OjpweXRob246Omxpc3Q+KCkpOwp9CgpQeU9iamVjdCog dG9fcHl0aG9uKGNvbnN0IGJvb3N0OjpweXRob246OnN0cmluZyYpOwpib29zdDo6cHl0aG9uOjpz dHJpbmcgZnJvbV9weXRob24oUHlPYmplY3QqIHAsIGJvb3N0OjpweXRob246OnR5cGU8Ym9vc3Q6 OnB5dGhvbjo6c3RyaW5nPik7CgppbmxpbmUgYm9vc3Q6OnB5dGhvbjo6c3RyaW5nIGZyb21fcHl0 aG9uKFB5T2JqZWN0KiBwLCBib29zdDo6cHl0aG9uOjp0eXBlPGNvbnN0IGJvb3N0OjpweXRob246 OnN0cmluZyY+KQp7CiAgICByZXR1cm4gZnJvbV9weXRob24ocCwgYm9vc3Q6OnB5dGhvbjo6dHlw ZTxib29zdDo6cHl0aG9uOjpzdHJpbmc+KCkpOwp9CgpQeU9iamVjdCogdG9fcHl0aG9uKGNvbnN0 IGJvb3N0OjpweXRob246OmRpY3Rpb25hcnkmKTsKYm9vc3Q6OnB5dGhvbjo6ZGljdGlvbmFyeSBm cm9tX3B5dGhvbihQeU9iamVjdCogcCwgYm9vc3Q6OnB5dGhvbjo6dHlwZTxib29zdDo6cHl0aG9u OjpkaWN0aW9uYXJ5Pik7CgppbmxpbmUgYm9vc3Q6OnB5dGhvbjo6ZGljdGlvbmFyeSBmcm9tX3B5 dGhvbihQeU9iamVjdCogcCwgYm9vc3Q6OnB5dGhvbjo6dHlwZTxjb25zdCBib29zdDo6cHl0aG9u OjpkaWN0aW9uYXJ5Jj4pCnsKICAgIHJldHVybiBmcm9tX3B5dGhvbihwLCBib29zdDo6cHl0aG9u Ojp0eXBlPGJvb3N0OjpweXRob246OmRpY3Rpb25hcnk+KCkpOwp9CgpCT09TVF9QWVRIT05fRU5E X0NPTlZFUlNJT05fTkFNRVNQQUNFCgojZW5kaWYgLy8gT0JKRUNUU19EV0EwNTExMDBfSF8K --0__=00256B36006B2DF58f9e8a93df938690918c00256B36006B2DF5-- From david.abrahams at rcn.com Fri Jan 4 23:22:31 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Fri, 4 Jan 2002 17:22:31 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" > > What's the point of having different C++ types, then? You could just use > > the generic object for everything. I suppose there would be a very minor > > speed improvement if you know that an object is in fact a list rather > > than some generic sequence, but I don't think too many people are > > concerned about the speed of code operating at the C++/Python boundary. > > The C++ types export the corresponding C API parts: > object - PyObject_* > sequence - PySequence > etc. > > I guess maybe sequence, mapping, number should then inherit from > object_base not object. > The proxy should then also inherit from object. > > Although a list does not support the __call__ interface we could have a > user-defined class that supports both the sequence and __call__ interfaces. > > I am inclined to letting sequence, mapping etc. inherit from object_base > but supporting more combinations of interfaces: > object and sequence , object and mapping etc. > > What do you think? I think your design is more complicated and less clean than it needs to be, because either I have failed to communicate to you how I think it should go, or you are ignoring my suggestions. I keep trying to push you in the direction of writing a single generic object class with /all/ of the capabilities that are accessible through python's generic interface (number, sequence, mapping, attribute access, call, buffer,...), and then using private non-virtual inheritance to provide more-specialized sequence, number, etc. interfaces. Is this suggestion unclear to you? Do you think there's something wrong with it? I'd really like to know. Thanks, Dave From david.abrahams at rcn.com Sat Jan 5 03:06:08 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Fri, 4 Jan 2002 21:06:08 -0500 Subject: The future of the name "ref" Message-ID: I really want to get rid of this name by its current meaning. One reason is that "ref" already means something very different in namespace boost: it generates a reference wrapper for its argument for use with the bind and function libraries. I expect that Boost.Python users will want to use boost::ref() for this purpose, since in the new system we traffic much more heavily in function objects. Another reason is that the generalized "reference<>" smart pointer template is almost never instantiated with anything other than PyObject as its template parameter (ref is just a typedef for reference), and "reference" would be a far superior name for what we're currently calling "ref". I realize that this would be a change that breaks backward-compatibility, but now is the time to do that, since the new standard-conformant system will neccessarily break backward-compatibility for some users. I'd like to hear if there are any serious objections or modifications to the idea. Thanks, Dave =================================================== David Abrahams, C++ library designer for hire resume: http://users.rcn.com/abrahams/resume.html C++ Booster (http://www.boost.org) email: david.abrahams at rcn.com =================================================== From aleax at aleax.it Sat Jan 5 10:07:42 2002 From: aleax at aleax.it (Alex Martelli) Date: Sat, 5 Jan 2002 10:07:42 +0100 Subject: The future of the name "ref" Message-ID: On Saturday 05 January 2002 03:06, David Abrahams wrote: > I really want to get rid of this name by its current meaning. One reason ... > as its template parameter (ref is just a typedef for > reference), and "reference" would be a far superior name for > what we're currently calling "ref". ... > I realize that this would be a change that breaks backward-compatibility, > but now is the time to do that, since the new standard-conformant system > will neccessarily break backward-compatibility for some users. I'd like > to hear if there are any serious objections or modifications to the idea. Speaking as a _user_ of Boost, and particularly Boost Python (haven't contributed to it, but I've used it quite a bit), I'm +1 on this change. Yes, it's backwards-incompatible, but in a way that lets me big-bang- migrate my old BPL-using sources to the new regime by one pass of global search and replace -- I can deal with that. Alex From arnaldur at decode.is Sun Jan 6 16:08:01 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Sun, 6 Jan 2002 15:08:01 +0000 Subject: boost::python, returning new PyObject references Message-ID: > I think your design is more complicated and less clean than it needs to be, > because either I have failed to communicate to you how I think it should go, > or you are ignoring my suggestions. > > I keep trying to push you in the direction of writing a single generic > object class with /all/ of the capabilities that are accessible through > python's generic interface (number, sequence, mapping, attribute access, > call, buffer,...), and then using private non-virtual inheritance to provide > more-specialized sequence, number, etc. interfaces. Is this suggestion > unclear to you? Do you think there's something wrong with it? I'd really > like to know. Evidently I misunderstood you. I always thought about object as export ing the PyObject_* interface. When you were talking about inheriting privately from it I was not thinking about an object with all interfaces (PyObject_?, number, sequence, mapping and buffer). It's not that I think there is something wrong with that. The PyObject struct in python is like that so surely it is quite acceptable to reflect that. My idea was to have the interfaces separate and then combine them through multiple inheritance. If we go about having 1 generic object and then specialized interfaces through private inheritance we get rid of the multiple inheritance so you're probably right that it is cleaner. The reason I ignored your suggestions was I didn't realise you were talking about an object that had all interfaces. Sorry. This still leaves the issue of the callbacks in mutable_proxy. We could change get() and reference() to return just the result of the callback, but in the case of get(), the Python reference would be borrowed, since the corresponding ref would be a temporary object. The underlying sequence/mapping still would hold it and therefore cause the refcount to be >=1. You ok with returning a borrowed reference in get() for mutable_proxy? (that would mean m_p in object_base does not have to be mutable). Cheers Arnaldur From arnaldur at decode.is Sun Jan 6 16:12:43 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Sun, 6 Jan 2002 15:12:43 +0000 Subject: The future of the name "ref" Message-ID: I am in favor of changing the name. I only use reference<> as ref but from reading the code it was my understanding you also use it for PyTypeObject: reference. Anyway, I agree ref should be changed. Arnaldur From david.abrahams at rcn.com Sun Jan 6 17:51:19 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sun, 6 Jan 2002 11:51:19 -0500 Subject: No subject Message-ID: This is a multi-part message in MIME format. ------=_NextPart_000_0A51_01C196A8.744D2710 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit ------=_NextPart_000_0A51_01C196A8.744D2710 Content-Type: message/rfc822; name="Re_ boost__python, returning new PyObject references.eml" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="Re_ boost__python, returning new PyObject references.eml" Reply-To: "David Abrahams" From: "David Abrahams" To: "Arnaldur Gylfason" References: Subject: Re: boost::python, returning new PyObject references Date: Sun, 6 Jan 2002 11:11:27 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 ----- Original Message ----- From: "Arnaldur Gylfason" > > > I think your design is more complicated and less clean than it needs to > be, > > because either I have failed to communicate to you how I think it should > go, > > or you are ignoring my suggestions. > > > > I keep trying to push you in the direction of writing a single generic > > object class with /all/ of the capabilities that are accessible through > > python's generic interface (number, sequence, mapping, attribute access, > > call, buffer,...), and then using private non-virtual inheritance to > provide > > more-specialized sequence, number, etc. interfaces. Is this suggestion > > unclear to you? Do you think there's something wrong with it? I'd really > > like to know. > > > Evidently I misunderstood you. I always thought about object as export ing > the PyObject_* interface. What is the PyObject_* interface? Is int PyMapping_Size(PyObject *o) included in it, for example? What are the criteria which decide what's included? > When you were talking about inheriting privately from it I was not thinking > about an object with all interfaces (PyObject_?, number, sequence, mapping > and buffer). > It's not that I think there is something wrong with that. > The PyObject struct in python is like that so surely it is quite acceptable > to reflect that. > My idea was to have the interfaces separate and then combine them through > multiple inheritance. > If we go about having 1 generic object and then specialized interfaces > through private inheritance we get rid of the multiple inheritance so > you're probably right that > it is cleaner. > The reason I ignored your suggestions was I didn't realise you were talking > about an object that had all interfaces. > Sorry. No problem. I just wanted to get the communication channels clear. There is a possible downside to my idea: conversions from, e.g. object->mapping won't be provided automatically as upcasts by the inheritance relationship, so you'd need to write implicit conversions. A possible hybrid approach might be: struct base { void f() const; void g() const; }; struct F : private virtual base { public: using base::f; }; struct G : private virtual base { public: using base::g; }; struct H : F, G { }; // test code void f(F const&); void g(G const&); void h(H const& x) { x.f(); x.g(); f(x); g(x); } You can think of H, F, and G as object, number, and mapping respectively. I'm not sure whether this is an improvement, so you should give it a little thought. I guess one problem with my first suggestion is that there would be no way to provide the object& -> sequence& or object const& -> sequence const& conversions. > This still leaves the issue of the callbacks in mutable_proxy. > We could change get() and reference() to return just the result of the > callback, but in the case of get(), the Python reference would be borrowed, > since the corresponding > ref would be a temporary object. That doesn't borrow a reference. The whole point of ref is that references are never borrowed, so it never has to worry about how to properly manage reference counts. A borrowed reference is one whose count is not managed because you are counting on the lifetime of some other owner to keep it alive. > The underlying sequence/mapping still > would hold it and therefore cause the refcount to be >=1. > You ok with returning a borrowed reference in get() for mutable_proxy? > (that would mean m_p in object_base does not have to be mutable). I guess I don't understand what you're doing again. BTW, it looks like mutable_proxy has a function for += and =, but not, e.g., for -=. Mistake? -Dave ------=_NextPart_000_0A51_01C196A8.744D2710-- From david.abrahams at rcn.com Sun Jan 6 17:59:12 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sun, 6 Jan 2002 11:59:12 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" To: "David Abrahams" Sent: Sunday, January 06, 2002 11:16 AM Subject: Re: boost::python, returning new PyObject references > > > __ You wrote __ > > > This still leaves the issue of the callbacks in mutable_proxy. > > We could change get() and reference() to return just the result of the > > callback, but in the case of get(), the Python reference would be > borrowed, > > since the corresponding > > ref would be a temporary object. > > That doesn't borrow a reference. The whole point of ref is that references > are never borrowed, so it never has to worry about how to properly manage > reference counts. A borrowed reference is one whose count is not managed > because you are counting on the lifetime of some other owner to keep it > alive. > > > The underlying sequence/mapping still > > would hold it and therefore cause the refcount to be >=1. > > You ok with returning a borrowed reference in get() for mutable_proxy? > > (that would mean m_p in object_base does not have to be mutable). > > I guess I don't understand what you're doing again. > > BTW, it looks like mutable_proxy has a function for += and =, but not, > e.g., > for -=. Mistake? > _______________ > > the get would be: > > PyObject* get() > { > return get_().get(); > } > > get_() is the callback that calls get_item on sequence/mapping and returns > a ref. > The return value is not stored. Thus it is a temporary. > By directly calling get() on this temporary ref we get the PyObject* but > the mutable_proxy does not own it. > Only the underlying sequence/mapping owns it. That is what I was referring > to by a borrowed reference. Oh, I understand now! Do we really want to expose a function which directly returns a PyObject* at this level? My feeling is that we don't... but I see that the existing design does it :-(. Anyway, it makes sense that if we /do/ have such a function, it should return a borrowed reference. I'm not sure that there's any relationship to the fact that the ref is a temporary, though. Does that change anything? > Regarding -=, it is true that it is missing (along with *=, /= etc.). > += is there to confirm the design. The interface is not complete. > I was thinking about finishing it later. > Similar remarks go for the binary operators, only operator+ has been > implemented. OK. It seems to me that it should be possible to handle all of these issues with a single implementation of the inplace operators used for both numbers and proxies. You just always go through a virtual function (or callback) to get the PyObject* to operate on. BTW, do you understand how the following works under-the-covers in Python? I don't: >>> x = [ 1 , 2 ] >>> x[1] += 3 >>> x [1, 5] Since integers are immutable, it seems to me that x[1] has to return some sort of proxy. I guess that could work as long as >>> x[1] = 2 caused a call to PyObject_SetItem instead of producing the proxy... I ask about this because we clearly want the same usage to work in C++ through our wrappers. -Dave From david.abrahams at rcn.com Sun Jan 6 18:11:29 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sun, 6 Jan 2002 12:11:29 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" > > What is the PyObject_* interface? Is > > int PyMapping_Size(PyObject *o) > > included in it, for example? What are the criteria which decide what's > > included? > > I am referring to the Object Protocol in the Abstract Objects Layer in the > Python/C API, chapter 6.1. > PyMapping_Size would not be included (Mapping Protocol chapter 6.4). > I put PyObject_GetItem and SetItem into the mapping interface/protocol. > The Mapping protocol adds PyMapping_GetItemString for key strings. Hmm, I see. I had a conversation with Guido some time ago about the Python concept model. He said that they had been having trouble formalizing it, and this may be evidence of it. I guess the question is, which of the concepts Number,Mapping,Sequence,Object,Buffer... logically subsume the others? Are they actually orthogonal (in which case, a Number is not neccessarily an Object)? A list is surely a sequence, but it also supports some attributes, so that makes it an Object as well? Maybe it makes sense not to second-guess the way that Python has delineated these things, despite the fact that my instinct tells me that everything is an Object. [Guido: the context for this discussion is that Arnaldur is reworking the object wrapping interface in Boost.Python which supplies a syntax similar to that used by Python for operating on generic Python objects, with classes to represent Number,Mapping,List,String,etc.... We're trying to understand what the inheritance relationships should be, under the assumption that an object wrapper can only be constructed around a PyObject* if the PyObject* meets the appropriate concept requirements, and if class A is derived from class B, one can assume that an A supplies all the requirements of B]. -Dave From david.abrahams at rcn.com Sun Jan 6 18:20:00 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sun, 6 Jan 2002 12:20:00 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" > Let's imagine 3 different designs, A, B and C: > > A] > > struct object_base; > > > struct object: object_base > > struct number : private object > { > public: > using ... > > }; > > mapping, sequence > > > > B] > > struct object_base; > > > struct object: object_base > > struct number : private virtual object > { > public: > using ... > > }; > > mapping, sequence > > struct object : number, sequence, mapping, ... > { }; > > > C] > > struct object_base; > > struct abstract_object: public virtual object_base > { > // the Abstract Object Protocol > ... > }; > > struct number: public virtual object_base > { > // the Abstract Number Protocol > ... > }; > > sequence, mapping > > > struct object: abstract_object, number, sequence, mapping > { > // The generic object interface > > ... > > }; > > > --- > > A] is clean but there is this matter of conversions you mentioned. > B] is the H,F,G, base example you came up with. > C] is what I originally was aiming for. > > If A] would do it is cleanest. > > If we have to go for B] I can't see why it is necessarily better than C] I guess I agree with you, now that we've considered the issues. I like where you've gone with C], with object derived from abstract_object. I was never very fond of the intermediate seqmap class, and it seems to be unneeded in C]; am I right? Before moving along, you should probably take a moment to look at http://cxx.sourceforge.net/. The object interface is one of CXX's central features, so it may be more fully-developed than what we have in Boost.Python. It would be a shame to overlook something which CXX has already handled in a superior way. Thanks for your patient work on this project, Arnaldur! -Dave From david.abrahams at rcn.com Sun Jan 6 18:37:07 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sun, 6 Jan 2002 12:37:07 -0500 Subject: The future of the name "ref" Message-ID: I suppose I can always leave the old name "ref" there for backward-compatibility purposes. I don't think too many people are using "reference" for anything. -Dave ----- Original Message ----- From: "Alex Martelli" To: "David Abrahams" ; "Alex Martelli" ; "Anton Gluck" ; "Arnaldur Gylfason" ; "Barry Scott" ; "Mark Evans" ; "Martin Casado" ; "Niklas Blomberg" ; "Ralf W. Grosse-Kunstleve" ; "Ullrich Koethe" ; "Xavier Defrang" ; "scott snyder" < snyder at fnal.gov>; "Martin Casado" Sent: Saturday, January 05, 2002 4:07 AM Subject: Re: The future of the name "ref" > On Saturday 05 January 2002 03:06, David Abrahams wrote: > > I really want to get rid of this name by its current meaning. One reason > ... > > as its template parameter (ref is just a typedef for > > reference), and "reference" would be a far superior name for > > what we're currently calling "ref". > ... > > I realize that this would be a change that breaks backward-compatibility, > > but now is the time to do that, since the new standard-conformant system > > will neccessarily break backward-compatibility for some users. I'd like > > to hear if there are any serious objections or modifications to the idea. > > Speaking as a _user_ of Boost, and particularly Boost Python (haven't > contributed to it, but I've used it quite a bit), I'm +1 on this change. > > Yes, it's backwards-incompatible, but in a way that lets me big-bang- > migrate my old BPL-using sources to the new regime by one pass of global > search and replace -- I can deal with that. > > > Alex From arnaldur at decode.is Sun Jan 6 19:00:11 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Sun, 6 Jan 2002 18:00:11 +0000 Subject: boost::python, returning new PyObject references Message-ID: > I guess I agree with you, now that we've considered the issues. > > I like where you've gone with C], with object derived from abstract_object. > I was never very fond of the intermediate seqmap class, and it seems to be > unneeded in C]; am I right? Thank you. You're right seqmap is unneeded. This latest design is where I was heading but it was just materializing now, so I don't blame you for being confused. It is not perfect though. A part of the abstract_object interface applies to most/all objects, e.g. str(). In this design number, list etc. would not have these methods. Only abstract_object and object would have them (plus the proxies). Using the private inheritance design could fix that. I am not sure if this is a problem though. What do you think? Regarding attributes for list, I think the list interface has everything one needs for working with a list, so it doesn't have to inherit from any of the generic interface classes. > Before moving along, you should probably take a moment to look at > http://cxx.sourceforge.net/. The object interface is one of CXX's central > features, so it may be more fully-developed than what we have in > Boost.Python. It would be a shame to overlook something which CXX has > already handled in a superior way. True. I have given it a look and I think we should go carefully over it before completing the interfaces. I personally think though, that the design we're aiming at is superior. > Thanks for your patient work on this project, Arnaldur! You're welcome. I am happy to have this opportunity to work with you. Cheers Arnaldur From david.abrahams at rcn.com Sun Jan 6 19:21:40 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sun, 6 Jan 2002 13:21:40 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" > This latest design is where I was heading but it was just materializing > now, so I don't blame you for being confused. > It is not perfect though. > A part of the abstract_object interface applies to most/all objects, e.g. > str(). Yes, I was trying to make that point in a previous message, e.g. that List is not just a Sequence but also an Object. > In this design number, list etc. would not have these methods. Well, you could derive list from sequence and object, right? > Only abstract_object and object would have them (plus the proxies). > Using the private inheritance design could fix that. > I am not sure if this is a problem though. > What do you think? I think that if we are going to respect the Python concept model, we must derive all types known to have str() from abstract_object as well as the other concepts they might model. > Regarding attributes for list, I think the list interface has everything > one needs for working with a list, > so it doesn't have to inherit from any of the generic interface classes. It seems to me that list should just inherit sequence and abstract_object, and supply the appropriate constructors and accepts() function. All the rest of the interface would be taken care of by the base classes. > > Before moving along, you should probably take a moment to look at > > http://cxx.sourceforge.net/. The object interface is one of CXX's central > > features, so it may be more fully-developed than what we have in > > Boost.Python. It would be a shame to overlook something which CXX has > > already handled in a superior way. > > True. I have given it a look and I think we should go carefully over it > before completing the interfaces. > I personally think though, that the design we're aiming at is superior. :^) :^) From guido at python.org Sun Jan 6 19:58:02 2002 From: guido at python.org (Guido van Rossum) Date: Sun, 06 Jan 2002 13:58:02 -0500 Subject: boost::python, returning new PyObject references Message-ID: > From: "Arnaldur Gylfason" > > > > What is the PyObject_* interface? Is > > > int PyMapping_Size(PyObject *o) > > > included in it, for example? What are the criteria which decide what's > > > included? > > > > I am referring to the Object Protocol in the Abstract Objects Layer in the > > Python/C API, chapter 6.1. > > PyMapping_Size would not be included (Mapping Protocol chapter 6.4). > > I put PyObject_GetItem and SetItem into the mapping interface/protocol. > > The Mapping protocol adds PyMapping_GetItemString for key strings. > > Hmm, I see. > > I had a conversation with Guido some time ago about the Python concept > model. He said that they had been having trouble formalizing it, and this > may be evidence of it. I guess the question is, which of the concepts > Number,Mapping,Sequence,Object,Buffer... logically subsume the others? Are > they actually orthogonal (in which case, a Number is not neccessarily an > Object)? A list is surely a sequence, but it also supports some attributes, > so that makes it an Object as well? Maybe it makes sense not to second-guess > the way that Python has delineated these things, despite the fact that my > instinct tells me that everything is an Object. I don't think I follow this -- maybe because you're imposing your own teminology? *Everything* in Python is an object, period. Number, mapping, sequence are mostly orthogonal, except that they overload some of the same operators (e.g. + and * apply to numbers and sequences, getitem/setitem apply to sequences and mappings). > [Guido: the context for this discussion is that Arnaldur is reworking the > object wrapping interface in Boost.Python which supplies a syntax similar to > that used by Python for operating on generic Python objects, with classes to > represent Number,Mapping,List,String,etc.... We're trying to understand what > the inheritance relationships should be, under the assumption that an object > wrapper can only be constructed around a PyObject* if the PyObject* meets > the appropriate concept requirements, and if class A is derived from class > B, one can assume that an A supplies all the requirements of B]. > > -Dave Since Python doesn't base its categorization on inheritance, I'm not sure how I can help. --Guido van Rossum (home page: http://www.python.org/~guido/) From david.abrahams at rcn.com Sun Jan 6 20:56:59 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sun, 6 Jan 2002 14:56:59 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Guido van Rossum" > > From: "Arnaldur Gylfason" > > > > > > What is the PyObject_* interface? Is > > > > int PyMapping_Size(PyObject *o) > > > > included in it, for example? What are the criteria which decide what's > > > > included? > > > > > > I am referring to the Object Protocol in the Abstract Objects Layer in the > > > Python/C API, chapter 6.1. > > > PyMapping_Size would not be included (Mapping Protocol chapter 6.4). > > > I put PyObject_GetItem and SetItem into the mapping interface/protocol. > > > The Mapping protocol adds PyMapping_GetItemString for key strings. > > > > Hmm, I see. > > > > I had a conversation with Guido some time ago about the Python concept > > model. He said that they had been having trouble formalizing it, and this > > may be evidence of it. I guess the question is, which of the concepts > > Number,Mapping,Sequence,Object,Buffer... logically subsume the others? Are > > they actually orthogonal (in which case, a Number is not neccessarily an > > Object)? A list is surely a sequence, but it also supports some attributes, > > so that makes it an Object as well? Maybe it makes sense not to second-guess > > the way that Python has delineated these things, despite the fact that my > > instinct tells me that everything is an Object. > > I don't think I follow this -- maybe because you're imposing your own > teminology? I don't think so. We're at least trying to follow the Python terminology as given in chapter 6 of the Python/C API reference. > *Everything* in Python is an object, period. What does "is-a" mean in this case? It certainly doesn't mean that everything supports all operations of the Object Protocol. I expect this calls PyObject_GetItem: >>> 1[2] Traceback (most recent call last): File "", line 1, in ? TypeError: unsubscriptable object I would expect a KeyError or IndexError if the protocol were "supported", but maybe there's a more appropriate defintion for "supported" in this case. If everything "is-a" Object, then we can ask, is a List a Number? It supports some of the operations in the number protocol, after all (e.g. +=). How does one decide the answer to that question in general? > Number, > mapping, sequence are mostly orthogonal, except that they overload > some of the same operators (e.g. + and * apply to numbers and > sequences, getitem/setitem apply to sequences and mappings). Yes, that's understood. > > [Guido: the context for this discussion is that Arnaldur is reworking the > > object wrapping interface in Boost.Python which supplies a syntax similar to > > that used by Python for operating on generic Python objects, with classes to > > represent Number,Mapping,List,String,etc.... We're trying to understand what > > the inheritance relationships should be, under the assumption that an object > > wrapper can only be constructed around a PyObject* if the PyObject* meets > > the appropriate concept requirements, and if class A is derived from class > > B, one can assume that an A supplies all the requirements of B]. > > > > -Dave > > Since Python doesn't base its categorization on inheritance, I'm not > sure how I can help. Inheritance is only relevant in the sense that C++ supplies implicit derived->base conversions. We want to structure things so that more-general protocols are represented by bases of the classes which represent the specific protocols including the more-general ones. It sounds like on the basis of what you've said, that all the other classes should be derived from the one for Object, but it isn't clear exactly how the rest of this universe should be structured. The fact that it seems there's no requirement in Python that all elements of a Protocol are implemented, and that the response to unimplemented operations is very close to that for operations which are not called correctly (an exception) makes it hard to decide what's what. At this point, I'm resorting to the vision of the designer for some insight, since I can't find any empirical way to make the determination. -Dave From guido at python.org Mon Jan 7 01:29:36 2002 From: guido at python.org (Guido van Rossum) Date: Sun, 06 Jan 2002 19:29:36 -0500 Subject: boost::python, returning new PyObject references Message-ID: > > I don't think I follow this -- maybe because you're imposing your own > > teminology? > > I don't think so. We're at least trying to follow the Python terminology as > given in chapter 6 of the Python/C API reference. > > > *Everything* in Python is an object, period. > > What does "is-a" mean in this case? It certainly doesn't mean that > everything supports all operations of the Object Protocol. What's the "Object Protocol"? Are we still talking about Python here? > I expect this > calls PyObject_GetItem: > > >>> 1[2] > Traceback (most recent call last): > File "", line 1, in ? > TypeError: unsubscriptable object > > I would expect a KeyError or IndexError if the protocol were > "supported", but maybe there's a more appropriate defintion for > "supported" in this case. You're missing something here. The TypeError is part of the defined behavior: some objects "support" the abstract "getitem" operation and others don't. Not supporting "getitem" doesn't make them not an object. > If everything "is-a" Object, then we can ask, is a List a Number? It > supports some of the operations in the number protocol, after all (e.g. +=). > How does one decide the answer to that question in general? By asking you to unask the question. Python doesn't have a classification like this, so there's no point in trying. There's a set of possible operations and each object type can decide which subset of those it supports. Certain subsets make the type smell more numeric, others make it smell like a sequence, but there's no absolute answer. It's *possible* though unlikely to have a sequence that doesn't support len(), for example. > > Number, mapping, sequence are mostly orthogonal, except that they > > overload some of the same operators (e.g. + and * apply to numbers > > and sequences, getitem/setitem apply to sequences and mappings). > > Yes, that's understood. > > > > [Guido: the context for this discussion is that Arnaldur is > > > reworking the object wrapping interface in Boost.Python which > > > supplies a syntax similar to that used by Python for operating > > > on generic Python objects, with classes to represent > > > Number,Mapping,List,String,etc.... We're trying to understand > > > what the inheritance relationships should be, under the > > > assumption that an object wrapper can only be constructed around > > > a PyObject* if the PyObject* meets the appropriate concept > > > requirements, and if class A is derived from class B, one can > > > assume that an A supplies all the requirements of B]. > > > > > > -Dave > > > > Since Python doesn't base its categorization on inheritance, I'm > > not sure how I can help. > > Inheritance is only relevant in the sense that C++ supplies implicit > derived->base conversions. We want to structure things so that > more-general protocols are represented by bases of the classes which > represent the specific protocols including the more-general ones. It > sounds like on the basis of what you've said, that all the other > classes should be derived from the one for Object, but it isn't > clear exactly how the rest of this universe should be > structured. The fact that it seems there's no requirement in Python > that all elements of a Protocol are implemented, and that the > response to unimplemented operations is very close to that for > operations which are not called correctly (an exception) makes it > hard to decide what's what. At this point, I'm resorting to the > vision of the designer for some insight, since I can't find any > empirical way to make the determination. I hope the above helps. --Guido van Rossum (home page: http://www.python.org/~guido/) From david.abrahams at rcn.com Mon Jan 7 02:03:54 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sun, 6 Jan 2002 20:03:54 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Guido van Rossum" > What's the "Object Protocol"? Are we still talking about Python here? http://www.python.org/doc/current/api/object.html > > I expect this > > calls PyObject_GetItem: > > > > >>> 1[2] > > Traceback (most recent call last): > > File "", line 1, in ? > > TypeError: unsubscriptable object > > > > I would expect a KeyError or IndexError if the protocol were > > "supported", but maybe there's a more appropriate defintion for > > "supported" in this case. > > You're missing something here. The TypeError is part of the defined > behavior: some objects "support" the abstract "getitem" operation and > others don't. Not supporting "getitem" doesn't make them not an > object. > > > If everything "is-a" Object, then we can ask, is a List a Number? It > > supports some of the operations in the number protocol, after all (e.g. +=). > > How does one decide the answer to that question in general? > > By asking you to unask the question. Python doesn't have a > classification like this, http://www.python.org/doc/current/api/abstract.html seems to indicate otherwise. Also PyMapping_Check et. al, and such functions in python itself as callable(), isinstance(), and issubclass(), though admittedly the lack of functions like issequence() or ismapping() seems to remove the emphasis on this idea. I guess there's no point in arguing with GVR about the nature of Python. FWIW, though, at least a few of us independently drew the same conclusions about the existence and possible importance of categories based on the documentation and the facilities provided by the language. > so there's no point in trying. There's a > set of possible operations and each object type can decide which > subset of those it supports. Certain subsets make the type smell more > numeric, others make it smell like a sequence, but there's no absolute > answer. It's *possible* though unlikely to have a sequence that > doesn't support len(), for example. I knew that; since the boundaries are so malleable I was trying to capture your mental model. Maybe your mental model is actually "one big soup of types with no structure, even informally". In that case, we may have to rethink the idea of classification altogether. -Dave From guido at python.org Mon Jan 7 02:22:17 2002 From: guido at python.org (Guido van Rossum) Date: Sun, 06 Jan 2002 20:22:17 -0500 Subject: boost::python, returning new PyObject references Message-ID: > > What's the "Object Protocol"? Are we still talking about Python here? > > http://www.python.org/doc/current/api/object.html Fair enough. I don't write the docs. :-) > > By asking you to unask the question. Python doesn't have a > > classification like this, > > http://www.python.org/doc/current/api/abstract.html seems to indicate > otherwise. These protocols are just convenient groupings for sets of operations. > Also PyMapping_Check et. al, and such functions in python itself > as callable(), isinstance(), and issubclass(), though admittedly the lack of > functions like issequence() or ismapping() seems to remove the emphasis on > this idea. isinstance() and issubclass() don't prove anything -- they make tests on the class hierarchy available for those that find such tests convenient. (Read their docs -- they take a second argument that's a class.) callable() is a test for a *single* operation, "function call". The others are left out on purpose, except there are some remnants (e.g. in the operator module) where I couldn't curtail the enthusiasm of others. :-) > I guess there's no point in arguing with GVR about the nature of > Python. FWIW, though, at least a few of us independently drew the > same conclusions about the existence and possible importance of > categories based on the documentation and the facilities provided by > the language. Maybe the docs need to be clarified. You cannot write a function that decides Sequence-ness of an arbitrary object, you can only write something that recognizes clear hits and clear misses correctly -- but there's a gray area where the proper approach is to test for the existence of a specific operation. Fortunately, starting in 2.2, you can check whether e.g. __getitem__ exists and that will give you the right answer for classes as well as for built-in types -- although 3rd party extensions may still be incompletely introspectable. > > so there's no point in trying. There's a > > set of possible operations and each object type can decide which > > subset of those it supports. Certain subsets make the type smell more > > numeric, others make it smell like a sequence, but there's no absolute > > answer. It's *possible* though unlikely to have a sequence that > > doesn't support len(), for example. > > I knew that; since the boundaries are so malleable I was trying to > capture your mental model. Maybe your mental model is actually "one > big soup of types with no structure, even informally". In that case, > we may have to rethink the idea of classification altogether. My mental model has some types that are clearly sequences, mappings, etc., and some that are clearly not, but no exact rule to always decide what something is. This has changed; ten years ago, I *did* think these were recognizable categories, but the language and my thinking have evolved away from this. --Guido van Rossum (home page: http://www.python.org/~guido/) From david.abrahams at rcn.com Mon Jan 7 04:38:59 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sun, 6 Jan 2002 22:38:59 -0500 Subject: boost::python, returning new PyObject references Message-ID: > > http://www.python.org/doc/current/api/abstract.html seems to indicate > > otherwise. > > These protocols are just convenient groupings for sets of operations. > > > Also PyMapping_Check et. al, and such functions in python itself > > as callable(), isinstance(), and issubclass(), though admittedly the lack of > > functions like issequence() or ismapping() seems to remove the emphasis on > > this idea. > > isinstance() and issubclass() don't prove anything -- they make tests > on the class hierarchy available for those that find such tests > convenient. (Read their docs -- they take a second argument that's a > class.) > > callable() is a test for a *single* operation, "function call". I understand. Remember when we discussed generic programming and Concept defintions a while back? A single, sufficiently important operation may be all that's needed to capture a Concept. "Callable" has granularity comparable to that of the Assignable and CopyConstructible concepts we use in C++, so it is natural to think of this function as an introspection that defines a Concept. Among the other concept models used in the C++ standard library we also have 5 flavors of Iterators (input, output, forward, bidirectional, random-access), Sequence, and Associative Container (c.f. Mapping). Maybe now you can see why C++ programmers are likely to find a strong analogy with what's going on in this are of the Python docs. > The others are left out on purpose, except there are some remnants > (e.g. in the operator module) where I couldn't curtail the enthusiasm > of others. :-) > > > I guess there's no point in arguing with GVR about the nature of > > Python. FWIW, though, at least a few of us independently drew the > > same conclusions about the existence and possible importance of > > categories based on the documentation and the facilities provided by > > the language. > > Maybe the docs need to be clarified. Probably. Note also that Paul Dubois seems to have made similar inferences from looking at the design of CXX. About 1/4 of the way down the page at http://cxx.sourceforge.net/ there is a refinement hierarchy for the CXX object wrappers. > You cannot write a function that > decides Sequence-ness of an arbitrary object, you can only write > something that recognizes clear hits and clear misses correctly -- but > there's a gray area where the proper approach is to test for the > existence of a specific operation. Fortunately, starting in 2.2, you > can check whether e.g. __getitem__ exists and that will give you the > right answer for classes as well as for built-in types -- although 3rd > party extensions may still be incompletely introspectable. Only in pure python, right? Extension writers can still look at the slots to see what operations may be supported. A malicious author who fills in slots with functions that always raise exceptions is doing something that doesn't even play well with the native Python interpreter code anyway, IIRC (I know, descriptors are an exception to this rule, but they're new). > > > so there's no point in trying. There's a > > > set of possible operations and each object type can decide which > > > subset of those it supports. Certain subsets make the type smell more > > > numeric, others make it smell like a sequence, but there's no absolute > > > answer. It's *possible* though unlikely to have a sequence that > > > doesn't support len(), for example. > > > > I knew that; since the boundaries are so malleable I was trying to > > capture your mental model. Maybe your mental model is actually "one > > big soup of types with no structure, even informally". In that case, > > we may have to rethink the idea of classification altogether. > > My mental model has some types that are clearly sequences, mappings, > etc., and some that are clearly not, but no exact rule to always > decide what something is. This has changed; ten years ago, I *did* > think these were recognizable categories, but the language and my > thinking have evolved away from this. In that case, I'll suggest again that you consider heading back the way you came ;-). Some clear and officially sanctioned Concept definitions would be very useful to writers of generic Python code. Note that while the ability to introspect about Concepts is nice, it's not the most important reason to define them**. The reason you want concepts is so that writers of generic code can specify the contract with their clients. It isn't a disaster if you pick the "wrong" definition, e.g. for Sequence. If you decide that a Sequence implements __len__, and someone wants to write a generic function which depends on the rest of the Sequence requirements but not __len__, she has two choices: 1. Ignore this difference and document that their function requires a Sequence parameter 2. Make up her own Concept definition which precisely describes the requirements imposed by her function Of course, it's much better if you can try to delineate all of the useful gradations in a particular domain at once, so that the community doesn't end up with lots of different names for the same concept. You might provide Sequence and MeasurableSequence, for example. In any case, it seems as though Arnaldur and I have a choice. We can: 1. Ignore the possibility of Concepts altogether and implement a single generalized object interface with all capabilities. 2. Acknowledge that Concepts would be useful, but the ones that the Python docs seem to imply are not really intentional. They seem to have the right names, but we'd need to change them and probably carve them up a bit, and that would only cause confusion, so again we throw up our hands. 3. Work out a set of concept definitions that make sense, and carefully document them so that people are aware that "Sequence" might mean something different when we say it than when the Python docs say it. Then implement C++ classes to represent those concepts. I'm in favor of #3, but open to others. Opinions, anyone? -Dave **When Concepts were first applied in C++ nobody knew how to introspect about them other than to let the compiler bark when generic code uses an unimplemented operation. We now know how to check a very few requirements without causing errors when they're not satisfied, and we can often generate more coherent error messages than what you'd get otherwise in the other cases, but the use of Concepts remains focused on specification, not introspection. From guido at python.org Mon Jan 7 05:51:25 2002 From: guido at python.org (Guido van Rossum) Date: Sun, 06 Jan 2002 23:51:25 -0500 Subject: boost::python, returning new PyObject references Message-ID: > > Maybe the docs need to be clarified. > > Probably. Note also that Paul Dubois seems to have made similar inferences > from looking at the design of CXX. About 1/4 of the way down the page at > http://cxx.sourceforge.net/ there is a refinement hierarchy for the CXX > object wrappers. The concepts are fine to explain things. The break down when you try to pin down exact definitions. > > You cannot write a function that decides Sequence-ness of an > > arbitrary object, you can only write something that recognizes > > clear hits and clear misses correctly -- but there's a gray area > > where the proper approach is to test for the existence of a > > specific operation. Fortunately, starting in 2.2, you can check > > whether e.g. __getitem__ exists and that will give you the right > > answer for classes as well as for built-in types -- although 3rd > > party extensions may still be incompletely introspectable. > > Only in pure python, right? Extension writers can still look at the > slots to see what operations may be supported. A malicious author > who fills in slots with functions that always raise exceptions is > doing something that doesn't even play well with the native Python > interpreter code anyway, IIRC (I know, descriptors are an exception > to this rule, but they're new). Correct. > > My mental model has some types that are clearly sequences, mappings, > > etc., and some that are clearly not, but no exact rule to always > > decide what something is. This has changed; ten years ago, I *did* > > think these were recognizable categories, but the language and my > > thinking have evolved away from this. > > In that case, I'll suggest again that you consider heading back the > way you came ;-). Some clear and officially sanctioned Concept > definitions would be very useful to writers of generic Python > code. Note that while the ability to introspect about Concepts is > nice, it's not the most important reason to define them**. The > reason you want concepts is so that writers of generic code can > specify the contract with their clients. IMO the concepts are sufficiently well-defined. You only get in trouble when you mistakenly try to map them to a class hierarchy. > It isn't a disaster if you pick the "wrong" definition, e.g. for > Sequence. If you decide that a Sequence implements __len__, and > someone wants to write a generic function which depends on the rest > of the Sequence requirements but not __len__, she has two choices: > > 1. Ignore this difference and document that their function requires a > Sequence parameter > 2. Make up her own Concept definition which precisely describes the > requirements imposed by her function > > Of course, it's much better if you can try to delineate all of the > useful gradations in a particular domain at once, so that the > community doesn't end up with lots of different names for the same > concept. You might provide Sequence and MeasurableSequence, for > example. That's exactly the problem: there are too many sub-concepts. E.g. sliceable sequence, mutable sequence, sliceable mutable sequence, and so on. > In any case, it seems as though Arnaldur and I have a choice. We can: > > 1. Ignore the possibility of Concepts altogether and implement a single > generalized object interface with all capabilities. > > 2. Acknowledge that Concepts would be useful, but the ones that the Python > docs seem to imply are not really intentional. They seem to have the right > names, but we'd need to change them and probably carve them up a bit, and > that would only cause confusion, so again we throw up our hands. > > 3. Work out a set of concept definitions that make sense, and carefully > document them so that people are aware that "Sequence" might mean something > different when we say it than when the Python docs say it. Then implement > C++ classes to represent those concepts. > > I'm in favor of #3, but open to others. > Opinions, anyone? I don't know enough of the other constraints on your system to be of much help here. I only note that in 2.2 I am moving away from making the concepts concrete in the implementation. A type has exactly those slots filled that it implements (and there's a complex mechanism that sets or clears the tp_foo slot when a __foo__ method is added to the class or removed from it. --Guido van Rossum (home page: http://www.python.org/~guido/) From david.abrahams at rcn.com Mon Jan 7 07:04:15 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 7 Jan 2002 01:04:15 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Guido van Rossum" > The concepts are fine to explain things. The break down when you try > to pin down exact definitions. What are they good for explaining? > > > My mental model has some types that are clearly sequences, mappings, > > > etc., and some that are clearly not, but no exact rule to always > > > decide what something is. This has changed; ten years ago, I *did* > > > think these were recognizable categories, but the language and my > > > thinking have evolved away from this. > > > > In that case, I'll suggest again that you consider heading back the > > way you came ;-). Some clear and officially sanctioned Concept > > definitions would be very useful to writers of generic Python > > code. Note that while the ability to introspect about Concepts is > > nice, it's not the most important reason to define them**. The > > reason you want concepts is so that writers of generic code can > > specify the contract with their clients. > > IMO the concepts are sufficiently well-defined. Sufficient for what purpose, though? From my POV, they're well-defined enough to lead people like Paul Dubois and me astray, without being sufficiently well-defined to be useful for anything. AFAICT, they're not defined at all, but rather hinted at in various ways. > You only get in > trouble when you mistakenly try to map them to a class hierarchy. I'm not sure that's the problem. Surely: * List is a refinement of Sequence * Sequence is a refinement of Object ?? > > Of course, it's much better if you can try to delineate all of the > > useful gradations in a particular domain at once, so that the > > community doesn't end up with lots of different names for the same > > concept. You might provide Sequence and MeasurableSequence, for > > example. > > That's exactly the problem: there are too many sub-concepts. > E.g. sliceable sequence, mutable sequence, sliceable mutable sequence, > and so on. Dividing it that way is clearly going to cause an explosion. However, granular concepts can be combined. It would make sense to say: "x fulfills the requirements of Sequence, Sliceable, and Callable" (for example). When defining Sequence, you can even add requirements that depend on other concepts. For example, if there's a separate way to determine the mutability of any type, you can require that if a Sequence is also Mutable, it must support setattr, for example. Also, not every allowable granule makes a useful concept, so for example there is no point in defining a concept around support for inplace ^= (numeric concepts would probably group related operations into concepts like AbelianGroup and Floating). I don't think it's an intractable problem, and I don't think a useful start needs to solve it completely (if that's even possible). > I don't know enough of the other constraints on your system to be of > much help here. I only note that in 2.2 I am moving away from making > the concepts concrete in the implementation. That's as it should be, in some sense. Concepts are supposed to be abstract. It's just a quirk of the relationship between Python and C++ that allows us to think about representing them with classes. > A type has exactly those > slots filled that it implements (and there's a complex mechanism that > sets or clears the tp_foo slot when a __foo__ method is added to the > class or removed from it. Acknowledged. Thanks for letting me harangue you about this, Dave From david.abrahams at rcn.com Mon Jan 7 07:56:12 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 7 Jan 2002 01:56:12 -0500 Subject: Conversions between classes related by inheritance Message-ID: Hi Ulli (and anyone else who cares!), I'm getting ready to implement implicit upcasting/downcasting and (yes!) cross-casting of class instances that are related by inheritance for the Boost.Python rewrite. Ullrich has suggested that it would be good to optimize these conversions, especially in order to speed overload resolution in the cases where some attempts will fail. When attempting to unwrap a class X, the current approach performs a depth-first traversal of the inheritance DAG in which X participates, starting at X and performing conversions at each step, and revisiting nodes in cases of diamond inheritance. Especially in the case of down-cast attempts, this can be slow due to the need for many dynamic_cast<>s. The most obvious optimization to do at first would be to quickly check whether the target class is part of a common inheritance hierarchy with the class of the object held in the Python object being unwrapped before doing any dynamic_cast<>s. We can make that an O(N) operation where N is the number of classes in the hierarchy, for example, by doing a breadth-first search. We could make it an O(1) operation by maintaining a unique identifier for each set of related classes. The above optimization helps us quickly rule out some conversions that won't work, but if we get past that test, we still need to find a path through the inheritance hierarchy that connects the source and target classes... and in the case of downcasts or cross-casts, the conversion /still/ might not succeed. The only obvious way to optimize further, AFAICT, would be to cache a record of the path from source to target. This might be extremely useful for efficiently implementing CLOS-style (multimethod) overload resolution, but it could also be expensive in large class hierarchies (O(N^2) in space). I'd like to hear from people about how important they consider this to be, and what trade-offs their applications demand. Also, if there are other ideas about ways to improve this mechanism, I'd love to hear those as well. Thanks, Dave =================================================== David Abrahams, C++ library designer for hire resume: http://users.rcn.com/abrahams/resume.html C++ Booster (http://www.boost.org) email: david.abrahams at rcn.com =================================================== From arnaldur at decode.is Mon Jan 7 16:50:34 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Mon, 7 Jan 2002 15:50:34 +0000 Subject: boost::python, returning new PyObject references Message-ID: > In any case, it seems as though Arnaldur and I have a choice. We can: > > 1. Ignore the possibility of Concepts altogether and implement a single > generalized object interface with all capabilities. > > 2. Acknowledge that Concepts would be useful, but the ones that the Python > docs seem to imply are not really intentional. They seem to have the right > names, but we'd need to change them and probably carve them up a bit, and > that would only cause confusion, so again we throw up our hands. > > 3. Work out a set of concept definitions that make sense, and carefully > document them so that people are aware that "Sequence" might mean something > different when we say it than when the Python docs say it. Then implement > C++ classes to represent those concepts. > > I'm in favor of #3, but open to others. > Opinions, anyone? I am in favor of #3 as well. It has been interesting to follow the discussion between you and Guido. I think we should work carefully on defining a set of concepts for Python objects. We?ll just have to accept that there can be objects that don't fit (e.g. not defining >= 1 of the sequence methods). Users of such objects would have to be aware of that. (They could use them as a sequence but be aware that if a function/method needs a sequence method the object does not define an error/exception would occur). In view of the discussion I've been wondering about inheritance (e.g. list inheriting from sequence). How about defining a python::SequenceConcept and use template parameters: Instead of namespace python = boost::python; void f( python::sequence & s) { ... } ... python::sequence s(obj); python::list l; ... f(s); f(l); we could have namespace python = boost::python; template void f(SequenceType & s) { boost::function_requires< python::SequenceConcept > (); ... } ... python::sequence s(obj); python::list l; ... f(s); f(l); Any opinions? Cheers Arnaldur From david.abrahams at rcn.com Mon Jan 7 17:19:57 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 7 Jan 2002 11:19:57 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" > I am in favor of #3 as well. > It has been interesting to follow the discussion between you and Guido. > I think we should work carefully on defining a set of concepts for Python > objects. > We?ll just have to accept that there can be objects that don't fit (e.g. > not defining >= 1 of the sequence methods). > Users of such objects would have to be aware of that. (They could use them > as a sequence but be aware that if a > function/method needs a sequence method the object does not define an > error/exception would occur). I disagree. Based on my conversations with Guido, it looks like the Sequence concept is quite minimal - it probably only implies that __getitem__ is implemented. Why not make Sequence include everything in the sequence protocol? Here's why: Once users have constructed an object wrapping a concept in C++, they should be able to rely on the operations of that concept being supported by the object. I want the wrapper to do the concept introspection when it is constructed/assigned, and throw an exception if there is a violation. At that point, I want the entire public interface to be available; it should only throw an exception if they python implementation of the underlying functionality throws. I think I am beginning to see why Guido says that an inheritance hierarchy gets us in trouble: there are too many combinations to create classes for all of them. Instead, we want some way to combine concepts "on-the-fly". In other words, you want to be able to write a function that takes a callable sliceable sequence without first having to create a class which inherits from all three of these. The only way I can think of to represent that is: void f(python::object x) { ... } > In view of the discussion I've been wondering about inheritance (e.g. list > inheriting from sequence). I /do/ think inheritance is appropriate for dealing with the concrete Python types like List. So for example we might have class list : public object { ... }; We'd have to decide what the default properties for object<> are. Probably everything. This is a bigger programming challenge than what we've been considering, but worth it I think. > How about defining a python::SequenceConcept and use template parameters: > Instead of > namespace python = boost::python; > > void f( python::sequence & s) > { > ... > } > > ... > > python::sequence s(obj); > python::list l; > > ... > > f(s); > f(l); > > we could have > > namespace python = boost::python; > > template > void f(SequenceType & s) > { > boost::function_requires< python::SequenceConcept > > (); > ... > } > > ... > > python::sequence s(obj); > python::list l; > > ... > > f(s); > f(l); > I don't think there's any reason that C++ functions which are generic over python types need to be templated. I'd like to help people avoid templates whenever its appropriate. Regards, Dave From arnaldur at decode.is Mon Jan 7 17:35:24 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Mon, 7 Jan 2002 16:35:24 +0000 Subject: boost::python, returning new PyObject references Message-ID: __ You wrote __ I think I am beginning to see why Guido says that an inheritance hierarchy gets us in trouble: there are too many combinations to create classes for all of them. Instead, we want some way to combine concepts "on-the-fly". In other words, you want to be able to write a function that takes a callable sliceable sequence without first having to create a class which inherits from all three of these. The only way I can think of to represent that is: void f(python::object x) { ... } > In view of the discussion I've been wondering about inheritance (e.g. list > inheriting from sequence). I /do/ think inheritance is appropriate for dealing with the concrete Python types like List. So for example we might have class list : public object { ... }; We'd have to decide what the default properties for object<> are. Probably everything. This is a bigger programming challenge than what we've been considering, but worth it I think. _______ I came up with a similar idea some weeks ago (i.e. object ). You mentioned the Andrei's GenScatterHierarchy . I've looked into this and I don't think GenScatterHierarchy is the way to go. Maybe something similar. We can use the same design as in boost::function to have variable number of parameters. Problem is some interfaces intersect (e.g. += for number and sequence etc.). I was wondering if it was possible to define a INTERSECT macro that can handle this but am way out of my league here. It is possible to pull this off using template specialization but the number of specializations would be high (exponential explosion). Cheers Arnaldur From arnaldur at decode.is Mon Jan 7 17:45:54 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Mon, 7 Jan 2002 16:45:54 +0000 Subject: boost::python, returning new PyObject references Message-ID: > I don't think there's any reason that C++ functions which are generic over > python types need to be templated. I'd like to help people avoid templates > whenever its appropriate. Yeah maybe. I think is is easy to get used to template programming but of course if there is no benefit ... Using template functions can of course cause code bloat. If we use inheritance I think we should consider virtual functions. Overriding non-virtual functions is normally considered bad practice. It can be argued there is no reason to override sequence methods in list. The PySequence_* calls should work the same as PyList_* calls. The difference is probably that PyList_* knows exactly what function to call (makes inlining possible ?) but PySequence_* has to use function pointers (o->ob_type->tp_as_sequence->sq_concat). From david.abrahams at rcn.com Mon Jan 7 19:14:48 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 7 Jan 2002 13:14:48 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" > > > I don't think there's any reason that C++ functions which are generic > over > > python types need to be templated. I'd like to help people avoid > templates > > whenever its appropriate. > > Yeah maybe. I think is is easy to get used to template programming but of > course if there is no benefit ... There /is/ a benefit. For example, only a concrete function can be exported to Python. > Using template functions can of course cause code bloat. > If we use inheritance I think we should consider virtual functions. > Overriding non-virtual functions is normally > considered bad practice. In this case I think it would be fine, since the semantics would be identical. The only advantage to using virtual functions is that invoking operator[] on a python::sequence& bound to a python::list would be ever so slightly faster... but the virtual function overhead might just as well cancel that speed improvement out anyway. > It can be argued there is no reason to override sequence methods in list. > The PySequence_* calls should work the same as PyList_* calls. > The difference is probably that PyList_* knows exactly what function to > call (makes inlining possible ?) > but PySequence_* has to use function pointers > (o->ob_type->tp_as_sequence->sq_concat). I don't think the extra level of indirection is really significant, but if you really want to optimize I think replacing the non-virtual function is the way to go. -Dave From arnaldur at decode.is Mon Jan 7 19:39:53 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Mon, 7 Jan 2002 18:39:53 +0000 Subject: boost::python, returning new PyObject references Message-ID: Dave, It would be great if you could give an example (maybe something that can solve object) so I can see the way. I have only taken a brief look at 2 files in boost::mpl but am interested in exploring this kind of programming. Regarding virtual/non-virtual, I guess I agree that ovirriding non-virtual functions is ok as long as the semantics are the same, i.e. no side-effects. (When using the object through a base class &/* the effects are the same even though a different method is called since there is no polymorphism). I am beginning to form the design in my head but using mpl wil mean a steep learning curve in the beginning. Cheers Arnaldur ----- Original Message ----- From: "Arnaldur Gylfason" > I came up with a similar idea some weeks ago (i.e. > object ). Silly me for not seeing the wisdom of your idea :( > You mentioned the Andrei's GenScatterHierarchy . > > I've looked into this and I don't think GenScatterHierarchy is the way to > go. Maybe something similar. Hmm. It's hard to say. Something like GenScatterHierarchy could be useful for generating all of the appropriate interface elements (e.g. operator[], operator+=, ...) > We can use the same design as in boost::function to have variable number of > parameters. Yes, or in mpl::type_list. > Problem is some interfaces intersect (e.g. += for number and sequence > etc.). That's why the concept tags should only be high-level objects. At a lower level > I was wondering if it was possible to define a INTERSECT macro that can > handle this but am way out of my league here. > It is possible to pull this off using template specialization but the > number of specializations would be high > (exponential explosion). That's easy with mpl. You can use copy_if to build a type_list that consists of all elements of A that are also in B. I can help with some of the metaprogramming infrastructure. -Dave From david.abrahams at rcn.com Mon Jan 7 21:15:18 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 7 Jan 2002 15:15:18 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" > Dave, > > It would be great if you could give an example (maybe something that can > solve object) so I can see the way. I don't think we're ready to solve that problem yet, but let me try to sketch an approach (just a sketch, keep that in mind): 1. Low-level interface components are supplied by type generators corresponding to fine-grained interface elements. Their constructor contains the validation code: struct getattr_generator { template struct type : Base { void validate() { T* self = static_cast(this); if (!self->ob_type->tp_getattr && !self->ob_type->tp_getattro) throw something; } // maybe this should return object<>, I don't know. ref getattr(char const* name) { T* self = static_cast(this); return ref(PyObject_GetAttrString(self->get_callback()(), name)); } }; }; struct indexable_generator { template struct type : Base { // select the right proxy type typedef typename boost::mpl::select_type< is_mutable::value , mutable_proxy , const_proxy>::type proxy; // implement operator[] for T template proxy operator[](U const& index_) { ref index(to_python(index_)); T* self = static_cast(this); return proxy(some_expression(self->get_callback(), self->set_callback(), ref)); } void validate() { T* self = static_cast(this); if (!self->get_callback()()->ob_type->tp_as_sequence->tp_getitem) throw something; } }; }; Concepts such as sequence may correspond to a type_list of these generators. In the case of sequence, I don't think there's anything more than indexable: struct sequence : mpl::type_list {}; struct integral : mpl::type_list {}; then we can define object as: template struct object : gen_linear_hierarchy< typename get_unique_generators >::type , use_generator< object > > { }; where use_generator is something like: template struct use_generator { template struct apply { typedef Generator::template type type; }; }; and where get_unique_generators is something like: struct all_generators : mpl::type_list< indexable_generator,getattr_generator, setattr_generator...> {}; template struct get_unique_generators { typedef typename mpl::copy_if >::type type; }; and in_a_sublist is: template struct in_a_sublist { template struct apply { BOOST_STATIC_CONSTANT(bool, value = // an expression which is true iff T is in any // of the Lists contained in ListOfLists }; }; Whew, I'm tired now! This is pretty complicated; Perhaps it can be simplified. > I have only taken a brief look at 2 files in boost::mpl but am interested > in exploring this kind of programming. It's a lot of fun. Take a look at the example and test directories also for some illuminating samples. From barry at scottb.demon.co.uk Tue Jan 8 00:53:20 2002 From: barry at scottb.demon.co.uk (Barry Scott) Date: Mon, 7 Jan 2002 23:53:20 -0000 Subject: boost::python, returning new PyObject references Message-ID: PyCXX does 1 for all the reasons that Guido outlined. In PyCXX you call supportXXX functions to wire up Python object slots up to virtual functions of the Extension class. For example this is a fragment that creates an python extension object that acts as supports sequence, getattr, setattr and repr. class bemacs_buffer_data : public Py::PythonExtension { // static methods public: static void init_type() { behaviors().name("bemacs_buffer"); behaviors().doc("bemacs buffer"); behaviors().supportGetattr(); behaviors().supportSetattr(); behaviors().supportRepr(); behaviors().supportSequenceType(); } I took the view that to be a sequence type requires you to wire up all the slots that sequences typically use. I could have allowed each slot to be wire independently, but it seems more useful to wire up all the sequence slots, or all the mapping slots etc. Of course you can make objects are sequences and mappings if that makes sense in your world. BArry -----Original Message----- From: Arnaldur Gylfason [mailto:arnaldur at decode.is] Sent: 07 January 2002 15:51 To: David Abrahams Cc: Alex Martelli; Barry Scott; Martin Casado; Anton Gluck; Guido van Rossum; Ullrich Koethe; Martin Casado; Niklas Blomberg; Ralf W. Grosse-Kunstleve; scott snyder; Xavier Defrang Subject: Re: boost::python, returning new PyObject references > In any case, it seems as though Arnaldur and I have a choice. We can: > > 1. Ignore the possibility of Concepts altogether and implement a single > generalized object interface with all capabilities. > > 2. Acknowledge that Concepts would be useful, but the ones that the Python > docs seem to imply are not really intentional. They seem to have the right > names, but we'd need to change them and probably carve them up a bit, and > that would only cause confusion, so again we throw up our hands. > > 3. Work out a set of concept definitions that make sense, and carefully > document them so that people are aware that "Sequence" might mean something > different when we say it than when the Python docs say it. Then implement > C++ classes to represent those concepts. > > I'm in favor of #3, but open to others. > Opinions, anyone? I am in favor of #3 as well. It has been interesting to follow the discussion between you and Guido. I think we should work carefully on defining a set of concepts for Python objects. We?ll just have to accept that there can be objects that don't fit (e.g. not defining >= 1 of the sequence methods). Users of such objects would have to be aware of that. (They could use them as a sequence but be aware that if a function/method needs a sequence method the object does not define an error/exception would occur). In view of the discussion I've been wondering about inheritance (e.g. list inheriting from sequence). How about defining a python::SequenceConcept and use template parameters: Instead of namespace python = boost::python; void f( python::sequence & s) { ... } ... python::sequence s(obj); python::list l; ... f(s); f(l); we could have namespace python = boost::python; template void f(SequenceType & s) { boost::function_requires< python::SequenceConcept > (); ... } ... python::sequence s(obj); python::list l; ... f(s); f(l); Any opinions? Cheers Arnaldur From david.abrahams at rcn.com Tue Jan 8 01:21:09 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 7 Jan 2002 19:21:09 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Barry Scott" > PyCXX does 1 for all the reasons that Guido outlined. Is the documentation out-of-date? It (cxx.sourceforge.net) says: ... Class Object represents the most general kind of Python object. The rest of the classes that represent Python objects inherit from it. Object Type Int Float Long Sequence String Tuple List Mapping Dict Callable ... > In PyCXX you call supportXXX functions to wire up Python object slots > up to virtual functions of the Extension class. This isn't about extension classes in Boost.Python, FWIW. We've got extension classes completely handled. This is about how to write C++ functions that manipulate other (possibly arbitrary) Python objects. So, if I write a C++ function which takes an arbitrary mapping as an argument, I could use PyObject* and go through the regular Python 'C' API, but that would be relatively unsafe and clumsy. I'd rather write something like: void f(python::mapping x); I'd like to arrange it so that when mappings are constructed from PyObject*, we check that the mapping concept is satisfied (throwing an exception otherwise), and so that the public interface to mapping doesn't contain any operations which are unsupported by the object. > For example this is a fragment that creates an python extension object > that acts as supports sequence, getattr, setattr and repr. > > class bemacs_buffer_data : public Py::PythonExtension > { > // static methods > public: > static void init_type() > { > behaviors().name("bemacs_buffer"); > behaviors().doc("bemacs buffer"); > behaviors().supportGetattr(); > behaviors().supportSetattr(); > behaviors().supportRepr(); > behaviors().supportSequenceType(); > } In Boost.Python you just expose the functions you want with the appropriate names, so to expose a std::vector as a sequence of strings you might have: python::class_builder > vec_string(my_module, "vec_string"); vec_string.def(&std::vector >::operator[], "__getitem__"); You don't need to otherwise declare that it's a sequence. Your module's "vecstring" class now has a __getitem__ which accepts python Integers (and other types convertible to int) and hands back Python Strings. Actually this example needs an additional type cast to select the const version of operator[] since vector's operator[] is overloaded, but I omitted it for clarity. > I took the view that to be a sequence type requires you to wire up all the > slots that sequences typically use. I could have allowed each slot to be wire > independently, but it seems more useful to wire up all the sequence slots, or > all the mapping slots etc. Unfortunately, there are places where the Python interpreter looks at the slots and makes decisions based on what it sees are wired up, so I try hard not to fill any slots that I'm not really implementing. Guido's doing the same thing in the new type system. -Dave From guido at python.org Tue Jan 8 03:50:18 2002 From: guido at python.org (Guido van Rossum) Date: Mon, 07 Jan 2002 21:50:18 -0500 Subject: boost::python, returning new PyObject references Message-ID: I have to end my contributions to this discussion -- I simply can't find the time to keep it up. Here's one last installment. > > The concepts are fine to explain things. The break down when you try > > to pin down exact definitions. > > What are they good for explaining? In the tutorial, we use the concept of sequence to explain the similarity between strings, tuples and lists, for example, and the concept of numbers to explain mixed-mode numerical operations. > > IMO the concepts are sufficiently well-defined. > > Sufficient for what purpose, though? From my POV, they're well-defined > enough to lead people like Paul Dubois and me astray, without being > sufficiently well-defined to be useful for anything. AFAICT, they're not > defined at all, but rather hinted at in various ways. The source of the problem seems to be with attempts to impose the C++ view of the world onto Python (or the other way around). At the boundaries of languages that are so different, you are going to find things that don't map cleanly, no matter how you try it. > > You only get in > > trouble when you mistakenly try to map them to a class hierarchy. > > I'm not sure that's the problem. Surely: > * List is a refinement of Sequence > * Sequence is a refinement of Object > ?? But there is no base class Sequence. > > > Of course, it's much better if you can try to delineate all of the > > > useful gradations in a particular domain at once, so that the > > > community doesn't end up with lots of different names for the same > > > concept. You might provide Sequence and MeasurableSequence, for > > > example. > > > > That's exactly the problem: there are too many sub-concepts. > > E.g. sliceable sequence, mutable sequence, sliceable mutable sequence, > > and so on. > > Dividing it that way is clearly going to cause an explosion. However, > granular concepts can be combined. It would make sense to say: > > "x fulfills the requirements of Sequence, Sliceable, and Callable" > > (for example). When defining Sequence, you can even add requirements that > depend on other concepts. For example, if there's a separate way to > determine the mutability of any type, you can require that if a Sequence is > also Mutable, it must support setattr, for example. Also, not every > allowable granule makes a useful concept, so for example there is no point > in defining a concept around support for inplace ^= (numeric concepts would > probably group related operations into concepts like AbelianGroup and > Floating). I don't think it's an intractable problem, and I don't think a > useful start needs to solve it completely (if that's even possible). > > > I don't know enough of the other constraints on your system to be of > > much help here. I only note that in 2.2 I am moving away from making > > the concepts concrete in the implementation. > > That's as it should be, in some sense. Concepts are supposed to be abstract. > It's just a quirk of the relationship between Python and C++ that allows us > to think about representing them with classes. > > > A type has exactly those > > slots filled that it implements (and there's a complex mechanism that > > sets or clears the tp_foo slot when a __foo__ method is added to the > > class or removed from it. > > Acknowledged. > > Thanks for letting me harangue you about this, > Dave --Guido van Rossum (home page: http://www.python.org/~guido/) From david.abrahams at rcn.com Tue Jan 8 04:37:54 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 7 Jan 2002 22:37:54 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Guido van Rossum" > I have to end my contributions to this discussion -- I simply can't > find the time to keep it up. Here's one last installment. You've already devoted far more energy here than I expected. Thanks for everything. > > > The concepts are fine to explain things. The break down when you try > > > to pin down exact definitions. > > > > What are they good for explaining? > > In the tutorial, we use the concept of sequence to explain the > similarity between strings, tuples and lists, for example, and the > concept of numbers to explain mixed-mode numerical operations. These are not concepts in the formal sense, though. A formal concept defines a set of required operations and their semantics. > > > IMO the concepts are sufficiently well-defined. > > > > Sufficient for what purpose, though? From my POV, they're well-defined > > enough to lead people like Paul Dubois and me astray, without being > > sufficiently well-defined to be useful for anything. AFAICT, they're not > > defined at all, but rather hinted at in various ways. > > The source of the problem seems to be with attempts to impose the C++ > view of the world onto Python (or the other way around). At the > boundaries of languages that are so different, you are going to find > things that don't map cleanly, no matter how you try it. That may be, but this isn't one of them, really! Or, if there is a strong difference here, it's more cultural than technical (e.g. you could explain it by saying that Python users prefer the informal to the formal). When you write a function in Python, you can either: a. tell people to look at the source to figure out how to use it b. tell people that it requires specific types as arguments (i.e. a dict, not just any mapping) c. tell people what the requirements on the operational semantics of the arguments are b. is exactly what is enforced by C++ with ordinary function arguments c. is exactly the practice we have developed for describing template arguments A 'concept' is just a way of naming a useful set of requirements. 'Concepts' apply to any language that allows generic programming, at least if you believe that the "read the source" approach is inadequate. > > > You only get in > > > trouble when you mistakenly try to map them to a class hierarchy. > > > > I'm not sure that's the problem. Surely: > > * List is a refinement of Sequence > > * Sequence is a refinement of Object > > ?? > > But there is no base class Sequence. Just as in C++ there's no base class InputIterator from which we derive ForwardIterator from which we derive BidirectionalIterator from which we derive RandomAccessIterator. On the other hand, we have validator (template) classes for each of these concepts which do have the corresponding inheritance relationship. Best Regards, Dave From david.abrahams at rcn.com Wed Jan 9 00:41:14 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Tue, 8 Jan 2002 18:41:14 -0500 Subject: Rethinking of inheritance and conversions Message-ID: My earlier conclusions that we should only search for paths beginning with upcasts were simply wrong. The counterexample is: X Y \ / Z If I have a polymorphic Z held via X*, it should be convertible to Y. Furthermore, there is not neccessarily a single best path for getting from X to Y: X Y |\ /| | / | |/ \| Z1 Z2 Also, the user may opt not to tell us about parts of the inheritance hierarchy: U ..:.. : : V W / \ / \ X Y Z It might be that a path from S to T that begins with an upcast and contains the fewest downcasts can always be used to get from S to T, but I'm not even confident of that. Okay, the strategy: * The convertability graph is represented in a straightforward manner: if there is a conversion from A -> B, the graph contains an edge from A -> B * Each edge is labelled with "upcast" or "downcast", and a void*(*)(void*) function pointer. The function is an instantiation of one of the following: template void* upcast(void* x) { return static_cast((T*)x); } template void* downcast(void* x) { return dynamic_cast((T*)x); } * When an inheritance relationship is declared, 1 or 2 edges are added to the graph, depending whether the user tells us that the base class is polymorphic (an upcast, or an upcast and a downcast). * Each extension class instance contains one or more "holders". A holder can tell us the type of object it holds, and whether or not the object is held by-value or by some kind of pointer or reference. Multiple holders will occur in case of multiple inheritance from wrapped classes. * To convert an extension class instance to a T*: for each holder of object u: if u is held by-value: breadth-first search the upcast edges of the convertability graph beginning at node U. At each expansion step record a back-link so that the path back to the source node can be retrieved if the node for T is reached, follow the back links to recover the conversion path, execute the conversion corresponding to each edge, and return the result. else: dijkstra search the convertability graph beginning at node U. * upcast edges cost 0; downcast edges cost 1. * each node in the queue is labelled with an address: &u converted to the node's target type. These labels are produced by calling the conversion function associated by each edge as it is explored. * for the purposes of the search, edges are considered to be "in the graph" if the edge is an upcast OR the edge's conversion function applied to the source node's address label is nonzero. if at any point we reach the node for T, we return the node's address label. Optimizing this process looks hard. What we'd really like is some way to find out if a particular node is reachable through a given edge in the graph without passing back thorough the edge's source node. If we knew that, we could explore only the paths that connect the source and target types. One possibility is to pick a method that's right most of the time, but allows some false positives. The better job you do of eliminating false positives, the more you prune the search. If you do a perfect job, the search heads directly for the target node (this begins to sound a lot like A* search, doesn't it?) We could use a function that produces a hash value for any (edge, target) pair, and a bitset S with the rule: if node N is reachable through edge E, S[hash(E,N)] == 1. Users could tune the optimization by changing the size of the bitset. We'd need to turn on new bits whenever new inheritance relationships were registered. Updating would require that the graph be bidirectionally traversable. This approach requires examination of all outgoing edges of the source node, even when there is no path to the target. We could apply the same technique to encode reachability relationships between nodes, further speeding the determination that no conversion is possible. There may be better approaches to generating this sort of hint information. If anybody has suggestions, I'd love to hear them. ---- A less important idea with some promise --- We can always find out the most-derived type to which any held object can be cast. That's just typeid(x), since downcasts are impossible anyway if the type isn't polymorphic. If a type /is/ polymorphic, we can identify the object's address also, via dynamic_cast(x). If that type is registered with the system, we can then convert it to any type from which it is unambiguously derived via a sequence of efficient static_casts. The problem is that the type might not have been registered with the system. Suppose T is derived from U which is derived from V. If the user has wrapped U and V, and returns a smart_ptr pointing to a T, it won't be convertible to U& via this method. In other words, this approach requires the registration of every concrete class that the user will return to us. Note that this is not neccessarily mean the user must have created a class_builder for these types; a different mechanism for registering inheritance is possible. Still, each concrete type in a polymorphic inheritance hierarchy will need to have its inheritance relationship with the most-derived class which is used as an argument to a wrapped function declared (whew!). As you can see, this requirement is difficult to explain, and thus is too much of a burden to place on users. A hybrid approach is possible; if the class' concrete type is registered, we can use the above method. Otherwise, we need to resort to something else. However, that effectively relegates the above to the realm of optimization, so it should only be implemented later. From barry at scottb.demon.co.uk Wed Jan 9 11:38:22 2002 From: barry at scottb.demon.co.uk (Barry Scott) Date: Wed, 9 Jan 2002 10:38:22 -0000 Subject: boost::python, returning new PyObject references Message-ID: > > PyCXX does 1 for all the reasons that Guido outlined. > > Is the documentation out-of-date? It (cxx.sourceforge.net) says: Incomplete, but up to date. I assumed that all this slots talk was for extensions. Now I see what you are solving: Pre 2.2 using the Python predicate tests is workable to detect a dict or list. But with 2.2 allowing objects to be partially like a list its very hard to do a simplistic wrap of a PyObject* into a C++ object that reliably works. In fact this can be a problem before 2.2, but not with the builtin types. I'll re-read the thread and see if I have anything new to say. BArry From david.abrahams at rcn.com Wed Jan 9 13:30:03 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 9 Jan 2002 07:30:03 -0500 Subject: Boost python library Message-ID: Hi Hans, ----- Original Message ----- From: "Rosbach, Hans A." > I'm making a test using the boost python library in order to get > an understanding of how to use it ahead of actual use in real > code. The test is done to enable us to compare different methods to > make our applications scriptable. > > We have some big applications, and a COM based scripting using > Jscript/Javascript/Ecmascript as language. This code we want to replace > with something better, and boost python is one of the alternatives. > > We will both use embedding of python and extending with our own > user visible classes. > > Our C++ classes will be thin, just a wrapper for the real data within > the application. In the test I found that using a static inline member > function, > I could place the boost python code in the same header file as the rest of > the > class definition. I.e. one place for both the C++ and python definition of > the class. > Ideal for making the code easy to maintain. It sounds like a great application for Boost.Python, and the plans we have for the near future will make it even better. > However, when I wanted to have my classes in different modules, I was unable > to do so with the inline init functions. I got an error message from the > MSVC6 compiler - ambiguous call to overloaded function. I think you'll need to show me a lot more detail before I have even a clue of what's going wrong. > As the definition of class attributes for python is something I would find > usefull > to have close to the definition of the attributes for C++, I would like to > have > some advice regarding this. Is this possible, but have I overlooked > something? > Is this something You have considered and rejected for some reason, or is it > something that would be possible to implement? I'm not sure what you're asking, exactly. Did you read http://www.boost.org/libs/python/doc/special.html#getter_setter? If this doesn't address your question, could you try to phrase it more precisely? > I have used boost 1.25.1. The latest CVS state moves all of Boost.Python into a shared library, which should produces /significantly/ smaller executable sizes for any application which uses multiple extension modules. I suggest you try that out. Of course, I realize that has nothing to do with your questions. Regards, Dave From Hans.A.Rosbach at dnv.com Wed Jan 9 14:46:53 2002 From: Hans.A.Rosbach at dnv.com (Rosbach, Hans A.) Date: Wed, 9 Jan 2002 14:46:53 +0100 Subject: Boost python library Message-ID: Thanks for your reply. > Hi Hans, > > ----- Original Message ----- > From: "Rosbach, Hans A." > > > > I'm making a test using the boost python library in order to get > > an understanding of how to use it ahead of actual use in real > > code. The test is done to enable us to compare different methods to > > make our applications scriptable. > > > > We have some big applications, and a COM based scripting using > > Jscript/Javascript/Ecmascript as language. This code we want to replace > > with something better, and boost python is one of the alternatives. > > > > We will both use embedding of python and extending with our own > > user visible classes. > > > > Our C++ classes will be thin, just a wrapper for the real data within > > the application. In the test I found that using a static inline member > > function, > > I could place the boost python code in the same header file as the rest of > > the > > class definition. I.e. one place for both the C++ and python definition > of > > the class. > > Ideal for making the code easy to maintain. > > It sounds like a great application for Boost.Python, and the plans we have > for the near future will make it even better. > > > However, when I wanted to have my classes in different modules, I was > unable > > to do so with the inline init functions. I got an error message from the > > MSVC6 compiler - ambiguous call to overloaded function. > > I think you'll need to show me a lot more detail before I have even a clue > of what's going wrong. > Ok, let me show you my test example. The relevant part consists of two Python modules, with one wrapped class in each. The classes: point and line - #ifndef POINT_H #define POINT_H #include <../application/application_config.h> #include #include #include #include namespace python = boost::python; class APPLICATION_IMPORT_EXPORT point { public: point(double x,double y,double z); point(const point& p); ~point(); virtual double x() const; virtual double y() const; virtual double z() const; virtual void X(double x); /* static void init(python::class_builder& point_class, python::module_builder& module); */ private: double m_x; double m_y; double m_z; }; /* inline void point::init(python::class_builder& point_class, python::module_builder& module) { point_class.def(python::constructor()); point_class.def(python::constructor()); point_class.def(&point::x,"x"); point_class.def(&point::y,"y"); point_class.def(&point::z,"z"); point_class.def(&point::x,"__getattr__X__"); point_class.def(&point::X,"__setattr__X__"); } */ #endif // POINT_H #ifndef LINE_H #define LINE_H #include <../application/application_config.h> #include <../application/point.h> #include #include namespace python = boost::python; class APPLICATION_IMPORT_EXPORT line { public: line(const point& p1,const point& p2); virtual ~line(); virtual point p1() const; virtual point p2() const; static void init(python::class_builder& line_class, python::module_builder& module); protected: private: point m_p1; point m_p2; }; inline void line::init(python::class_builder& line_class, python::module_builder& module) { line_class.def(python::constructor()); line_class.def(&line::p1,"p1"); line_class.def(&line::p2,"p2"); } #endif // LINE_H You can see the inline functions. When these are called making a single module, everything works fine. Using #include #include #include namespace python = boost::python; #include <../application/point.h> #include <../application/line.h> #include <../application/pypseudofile.h> BOOST_PYTHON_MODULE_INIT(geotest) { try { // Create an object representing this extension module. python::module_builder this_module("geotest"); python::class_builder point_class(this_module,"point"); python::export_converters(point_class); /* point_class.def(python::constructor()); point_class.def(python::constructor()); point_class.def(&point::x,"x"); point_class.def(&point::y,"y"); point_class.def(&point::z,"z"); point_class.def(&point::x,"__getattr__X__"); point_class.def(&point::X,"__setattr__X__"); */ point::init(point_class,this_module); } catch(...) { python::handle_exception(); // Deal with the exception for Python } } and #include #include #include namespace python = boost::python; #include <../application/point.h> #include <../application/line.h> #include <../application/pypseudofile.h> BOOST_PYTHON_MODULE_INIT(geotest2) { try { python::module_builder this_module("geotest2"); python::import_converters point_class_converters("geotest","point"); python::class_builder line_class(this_module,"line"); line_class.def(python::constructor()); line_class.def(&line::p1,"p1"); line_class.def(&line::p2,"p2"); // line::init(line_class,this_module); } catch(...) { python::handle_exception(); // Deal with the exception for Python } } gives me these messages from MSVC6 geotest2.cpp e:\boost_1_25_1\boost/python/detail/init_function.hpp(265) : error C2668: 'from_python' : ambiguous call to overloaded function e:\boost_1_25_1\boost/python/detail/init_function.hpp(260) : while compiling class-template member function 'class boost::python::detail::instance_holder_base *__thiscall boost::python::detail::init1 >,class point const &>::create_holder(class boost::python::detail::extension_instance *,struct _object *,struct _object *) const' e:\boost_1_25_1\boost/python/detail/init_function.hpp(266) : error C2661: 'instance_value_holder >::instance_value_holder >' : no overloaded function takes 2 parameters e:\boost_1_25_1\boost/python/detail/init_function.hpp(260) : while compiling class-template member function 'class boost::python::detail::instance_holder_base *__thiscall boost::python::detail::init1 >,class point const &>::create_holder(class boost::python::detail::extension_instance *,struct _object *,struct _object *) const' I hope this is both shows my problem and my wish. I can make it compile fine by using the explicite code instead of the calls to the inline init functions. It is thus possible to create a working example, but not as clean as I would like it. I would prefer the definitions for python and for C++ to reside in the same file. [cuts] > > > I have used boost 1.25.1. > > The latest CVS state moves all of Boost.Python into a shared library, which > should produces /significantly/ smaller executable sizes for any application > which uses multiple extension modules. I suggest you try that out. Of > course, I realize that has nothing to do with your questions. > > Regards, > Dave We don't use CVS, thus I have no experience with it, although I would imagine that my favorite editor probably would make it easy if I got around to it. Any pointers to an easy introduction to setting up CVS for cygwin and/or emacs on ms nt4? Regards, Hans ********************************************************************** Neither the confidentiality nor the integrity of this message can be guaranteed following transmission on the Internet. This message has been swept by MAILsweeper at DNV for the presence of computer viruses. ********************************************************************** From arnaldur at decode.is Wed Jan 9 15:09:59 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Wed, 9 Jan 2002 14:09:59 +0000 Subject: boost::python, returning new PyObject references Message-ID: David, I've taken a brief look at mpl ( understand parts. Almost for_loop but not quite. Hierarchy generators left). I think I would like some time to experiment and learn before contributing in this area (the generator_ - object<...> code). That means I won't be comfortable developingcode like this until maybe in a few months. I have a few comments regarding your generator code though. a) It relies on type casting to get to the PyObject. That way all generators are just interfaces with no data but type_cast the this pointer to access the PyObject they need. Why not type cast to a object_base* and call get() / reference() ? That way we can get to the PyObject through get() and override it when needed (proxy). You use the template parameter T and lets the use_generator pass the parameter to it (why don't we just explicitly type cast to object_base* and get rid of T ?). The const proxy issues could be handled through operator[]() and operator[] () const as before: struct getattr_generator { template struct type : Base { void validate() { PyObject* self = static_cast(this).get(); if (!self->ob_type->tp_getattr && !self->ob_type->tp_getattro) throw something; } // maybe this should return object<>, I don't know. ref getattr(char const* name) { object_base* self = static_cast(this); return ref(PyObject_GetAttrString(self->get(), name)); } }; }; object<...> would inherit from object_base + generated hierarchy for the generator list. b) sequence is a type_list of generators. We would therefore use object instead of sequence ?? c) getting a unique generator list would have to be more advanced. Let's say we combine sequence and number. Both define addable but not in the same way. We have to choose 1 or at least somehow remove any ambiguity. Similar remarks go for using ints as keys in mapping and then combining with sequence. d) Defining generators at this granularity makes it possible to have object interfaces that are complete and guarantee that all operations/methods are supported, which is what you are aiming for. On the other hand this is a bit cumbersome. How about developing the hierarchy we were approaching: object_base abstract_object : object_base callable : virtual abstract_object (slicable : virtual abstract_object) sequence : slicable mapping : virtual abstract_object number : virtual abstract_object object : callable , sequence , mapping , number list : sequence then later offer try the meta-template approach with generators and offer something like object<...> that has more fine-grained granularity and more control/type-checking? (discern between object and object<...> though). For immutable types like tuple we have to have non-const objects that only allow const methods from interfaces like sequence. Have to figure a clean way for this, so tuple can inherit from immutable sequence. What do you think? Cheers Arnaldur From arnaldur at decode.is Wed Jan 9 15:25:51 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Wed, 9 Jan 2002 14:25:51 +0000 Subject: object as a meta-template Message-ID: On second thought maybe it is a good idea to have the template parameter T in the generators. Sorry, I was a bit to rash in answering. I was just considering this and it can be pretty neat and more powerful to have this parameter. From david.abrahams at rcn.com Wed Jan 9 15:36:54 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 9 Jan 2002 09:36:54 -0500 Subject: Boost python library Message-ID: Hans, Firstly, unless I'm missing something, the code you sent me couldn't possibly compile. The only declaration of point::init is commented out, yet you reference it in the geotest module's initialization function. Secondly, you are treading in the hazy area of shared libraries (not covered by the C++ standard) and the even hazier area of VC6, so all bets are off ;-) Third, your point's wrapper for the copy constructor reads: point_class.def(python::constructor()); I think you want this to be a reference------^^^^^^^^^^^ Fourth, however, I'm pretty sure none of those are your problem. Any given module may only be exposed to one of the following: class_builder import_converters for any given T. Each one generates a set of conversion functions _when_ _instantiated_ that is appropriate to whether T is exported from the current module or not. If you declare both in the same translation unit, duplicate to_python/from_python functions with the same signature will be generated. HTH, Dave P.S. The rewrite will not have this limitation, as conversions will be done in a much more cross-module-friendly manner. In fact, there will be no need to declare import converters. ----- Original Message ----- From: "Rosbach, Hans A." To: "'David Abrahams'" ; "Rosbach, Hans A." Cc: "Xavier Defrang" ; "Ullrich Koethe" ; "Ralf W. Grosse-Kunstleve" ; "Niklas Blomberg" ; "Martin Casado" ; "Barry Scott" ; "Arnaldur Gylfason" ; "Anton Gluck" ; "Alex Martelli" ; "Martin Casado" ; "scott snyder" < snyder at fnal.gov> Sent: Wednesday, January 09, 2002 8:46 AM Subject: RE: Boost python library > Thanks for your reply. > > > Hi Hans, > > > > ----- Original Message ----- > > From: "Rosbach, Hans A." > > > > > > > I'm making a test using the boost python library in order to get > > > an understanding of how to use it ahead of actual use in real > > > code. The test is done to enable us to compare different methods to > > > make our applications scriptable. > > > > > > We have some big applications, and a COM based scripting using > > > Jscript/Javascript/Ecmascript as language. This code we want to replace > > > with something better, and boost python is one of the alternatives. > > > > > > We will both use embedding of python and extending with our own > > > user visible classes. > > > > > > Our C++ classes will be thin, just a wrapper for the real data within > > > the application. In the test I found that using a static inline member > > > function, > > > I could place the boost python code in the same header file as the rest > of > > > the > > > class definition. I.e. one place for both the C++ and python definition > > of > > > the class. > > > Ideal for making the code easy to maintain. > > > > It sounds like a great application for Boost.Python, and the plans we have > > for the near future will make it even better. > > > > > However, when I wanted to have my classes in different modules, I was > > unable > > > to do so with the inline init functions. I got an error message from > the > > > MSVC6 compiler - ambiguous call to overloaded function. > > > > I think you'll need to show me a lot more detail before I have even a clue > > of what's going wrong. > > > > Ok, let me show you my test example. The relevant part consists of > two Python modules, with one wrapped class in each. > > The classes: point and line - > > #ifndef POINT_H > #define POINT_H > > #include <../application/application_config.h> > > #include > #include > > #include > #include > > > > namespace python = boost::python; > > class APPLICATION_IMPORT_EXPORT point > { > public: > point(double x,double y,double z); > point(const point& p); > ~point(); > virtual double x() const; > virtual double y() const; > virtual double z() const; > > virtual void X(double x); > > /* static void init(python::class_builder& point_class, > python::module_builder& module); > */ > private: > double m_x; > double m_y; > double m_z; > > }; > /* > inline void point::init(python::class_builder& point_class, > python::module_builder& module) > { > point_class.def(python::constructor()); > point_class.def(python::constructor()); > point_class.def(&point::x,"x"); > point_class.def(&point::y,"y"); > point_class.def(&point::z,"z"); > point_class.def(&point::x,"__getattr__X__"); > point_class.def(&point::X,"__setattr__X__"); > } > > */ > #endif // POINT_H > > > #ifndef LINE_H > #define LINE_H > > #include <../application/application_config.h> > #include <../application/point.h> > > #include > #include > > namespace python = boost::python; > > class APPLICATION_IMPORT_EXPORT line > { > public: > line(const point& p1,const point& p2); > virtual ~line(); > virtual point p1() const; > virtual point p2() const; > > static void init(python::class_builder& line_class, > python::module_builder& module); > > protected: > > private: > point m_p1; > point m_p2; > > }; > > inline void line::init(python::class_builder& line_class, > python::module_builder& module) > { > line_class.def(python::constructor()); > line_class.def(&line::p1,"p1"); > line_class.def(&line::p2,"p2"); > } > > > #endif // LINE_H > > > You can see the inline functions. When these are called making a > single module, everything works fine. Using > > #include > #include > > #include > > namespace python = boost::python; > > #include <../application/point.h> > #include <../application/line.h> > #include <../application/pypseudofile.h> > > > > BOOST_PYTHON_MODULE_INIT(geotest) > { > try > { > // Create an object representing this extension module. > python::module_builder this_module("geotest"); > > python::class_builder point_class(this_module,"point"); > python::export_converters(point_class); > /* > point_class.def(python::constructor()); > point_class.def(python::constructor()); > point_class.def(&point::x,"x"); > point_class.def(&point::y,"y"); > point_class.def(&point::z,"z"); > point_class.def(&point::x,"__getattr__X__"); > point_class.def(&point::X,"__setattr__X__"); > */ > point::init(point_class,this_module); > } > catch(...) > { > python::handle_exception(); // Deal with the exception for Python > } > } > > and > > #include > #include > > #include > > namespace python = boost::python; > > #include <../application/point.h> > #include <../application/line.h> > #include <../application/pypseudofile.h> > > > > BOOST_PYTHON_MODULE_INIT(geotest2) > { > try > { > python::module_builder this_module("geotest2"); > python::import_converters > point_class_converters("geotest","point"); > python::class_builder line_class(this_module,"line"); > > line_class.def(python::constructor()); > line_class.def(&line::p1,"p1"); > line_class.def(&line::p2,"p2"); > > // line::init(line_class,this_module); > > } > catch(...) > { > python::handle_exception(); // Deal with the exception for Python > } > } > > gives me these messages from MSVC6 > > > geotest2.cpp > e:\boost_1_25_1\boost/python/detail/init_function.hpp(265) : error > C2668: 'from_python' : ambiguous call to overloaded function > e:\boost_1_25_1\boost/python/detail/init_function.hpp(260) : > while compiling class-template member function 'class > boost::python::detail::instance_holder_base *__thiscall > boost::python::detail::init1 ue_holder point> >,class point const &>::create_holder(class > boost::python::detail::extension_instance *,struct _object *,struct _object > *) const' > e:\boost_1_25_1\boost/python/detail/init_function.hpp(266) : error > C2661: 'instance_value_holder boost::python::detail::held_instance > >::instance_value_holder boost::python::detail::held_instance s point> >' : no overloaded function takes 2 parameters > e:\boost_1_25_1\boost/python/detail/init_function.hpp(260) : > while compiling class-template member function 'class > boost::python::detail::instance_holder_base *__thiscall > boost::python::detail::init1 ue_holder point> >,class point const &>::create_holder(class > boost::python::detail::extension_instance *,struct _object *,struct _object > *) const' > > > > > I hope this is both shows my problem and my wish. I can make it > compile fine by using the explicite code instead of the calls to the > inline init functions. It is thus possible to create a working > example, but not as clean as I would like it. I would prefer the > definitions for python and for C++ to reside in the same file. > > [cuts] > > > > > I have used boost 1.25.1. > > > > The latest CVS state moves all of Boost.Python into a shared library, > which > > should produces /significantly/ smaller executable sizes for any > application > > which uses multiple extension modules. I suggest you try that out. Of > > course, I realize that has nothing to do with your questions. > > > > Regards, > > Dave > > We don't use CVS, thus I have no experience with it, although I would > imagine that my favorite editor probably would make it easy if I got > around to it. Any pointers to an easy introduction to setting up CVS > for cygwin and/or emacs on ms nt4? > > Regards, > Hans > > > > ********************************************************************** > Neither the confidentiality nor the integrity of this message > can be guaranteed following transmission on the Internet. > This message has been swept by MAILsweeper at DNV for > the presence of computer viruses. > ********************************************************************** From david.abrahams at rcn.com Wed Jan 9 15:57:30 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 9 Jan 2002 09:57:30 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" To: "David Abrahams" > I've taken a brief look at mpl ( understand parts. Almost for_loop but not > quite. Hierarchy generators left). > I think I would like some time to experiment and learn before contributing > in this area (the generator_ - object<...> code). > That means I won't be comfortable developingcode like this until maybe in a > few months. No big hurry ;-) > I have a few comments regarding your generator code though. > > a) > It relies on type casting to get to the PyObject. That way all generators > are just interfaces with no data but type_cast the this pointer to access > the PyObject they need. > Why not type cast to a object_base* and call get() / reference() ? > That way we can get to the PyObject through get() and override it when > needed (proxy). Maybe that's best. In that case, probably no type cast is needed because object_base is the ultimate base class of the whole hierarchy. Probably the default parameter should not be "empty", but "object_base". > You use the template parameter T and lets the use_generator pass the > parameter to it (why don't we just explicitly type cast to object_base* and > get rid of T ?). The difference is basically this: if you do what I did originally, you don't have to go through a callback to get the PyObject* in the case where it's not a proxy. The get() function can just be a regular inlined member function which accesses a PyObject* data member. You can also derive a proxy object from this stack of templates which uses a callback for get(). Or, you can build separate proxies customized for each of the container types being accessed. So, in short, the difference is one of efficiency. Oh, also, you can easily create a const proxy by simply refusing to implement the set() function. :-) In case this doesn't quite jibe with my code, keep in mind that I didn't really see this clearly until I was writing it down just now ;-) Hmm, list can also be a template so that its proxies' capabilities are given by the list's template parameters, e.g: python::list :^) > The const proxy issues could be handled through operator[]() and operator[] > () const as before: I don't see how this sentence is reflected in the code below. Isn't that just what I wrote originally? > struct getattr_generator > { > template > struct type : Base > { > void validate() > { > PyObject* self = static_cast(this).get(); > if (!self->ob_type->tp_getattr && !self->ob_type->tp_getattro) > throw something; > } > > // maybe this should return object<>, I don't know. > ref getattr(char const* name) > { > object_base* self = static_cast(this); > return ref(PyObject_GetAttrString(self->get(), > name)); > } > }; > }; > > object<...> would inherit from object_base + generated hierarchy for the > generator list. > > > b) > sequence is a type_list of generators. > We would therefore use object instead of sequence ?? Precisely. Otherwise, you end up having to create clumsy classes like seqmap to combine fine-grained capabilities. > c) > getting a unique generator list would have to be more advanced. > Let's say we combine sequence and number. Both define addable but not in > the same way. > We have to choose 1 or at least somehow remove any ambiguity. Python has just one way to do it. Okay, that one way looks like: 1. Decide which of two ways to dispatch to 2. Do it that way. We should do the exact same thing. Check the interpreter source code. > Similar remarks go for using ints as keys in mapping and then combining > with sequence. The answer is the same. There may well be predefined functions for this in the API. Only a fraction of what's there is documented :-( > d) > Defining generators at this granularity makes it possible to have object > interfaces that are complete > and guarantee that all operations/methods are supported, which is what you > are aiming for. > On the other hand this is a bit cumbersome. > How about developing the hierarchy we were approaching: > > object_base > abstract_object : object_base > > callable : virtual abstract_object > (slicable : virtual abstract_object) > sequence : slicable > > mapping : virtual abstract_object > number : virtual abstract_object > > object : callable , sequence , mapping , number > > list : sequence > > > then later offer try the meta-template approach with generators and offer > something like object<...> > that has more fine-grained granularity and more control/type-checking? > (discern between object and object<...> though). I think it will be problematic, unless we give up on validity checking. How will users manipulate a sequence that turns out NOT to be sliceable? > For immutable types like tuple we have to have non-const objects that only > allow const methods > from interfaces like sequence. Have to figure a clean way for this, so > tuple can inherit from > immutable sequence. > > What do you think? I would use a nested typedef which indicates immutability, and select the right interface components via a metaprogram. I realize this is not easy to implement, but Guido has convinced me that python objects don't map well to the sort of class hierarchy we were talking about. -Dave From Hans.A.Rosbach at dnv.com Wed Jan 9 17:16:54 2002 From: Hans.A.Rosbach at dnv.com (Rosbach, Hans A.) Date: Wed, 9 Jan 2002 17:16:54 +0100 Subject: Boost python library Message-ID: Dave, > Hans, > Firstly, unless I'm missing something, the code you sent me couldn't > possibly compile. The only declaration of point::init is commented out, yet > you reference it in the geotest module's initialization function. That was me missing something. I have been modifying the code back and forth between different cases, and pasted the wrong version, sorry. > Secondly, you are treading in the hazy area of shared libraries (not covered > by the C++ standard) and the even hazier area of VC6, so all bets are off > ;-) > > Third, your point's wrapper for the copy constructor reads: > > point_class.def(python::constructor()); > I think you want this to be a reference------^^^^^^^^^^^ > I had missed that one, but those types of typos are one reson I would want to have it as close as possible. Easier to spot. The fact that it is not necessary with boost.pyton to specify the arguments other places, and make these mistakes is one of the features that makes me like it. > Fourth, however, I'm pretty sure none of those are your problem. Any given > module may only be exposed to one of the following: > > class_builder > import_converters > > for any given T. Each one generates a set of conversion functions _when_ > _instantiated_ that is appropriate to whether T is exported from the current > module or not. If you declare both in the same translation unit, duplicate > to_python/from_python functions with the same signature will be generated. > For a moment I thought I saw a solution, conditional compile, but (yet at least) no such luck. But then > HTH, > Dave > > P.S. The rewrite will not have this limitation, as conversions will be done > in a much more cross-module-friendly manner. In fact, there will be no need > to declare import converters. this looks promising. Look forward to testing the rewrite. Hans ********************************************************************** Neither the confidentiality nor the integrity of this message can be guaranteed following transmission on the Internet. This message has been swept by MAILsweeper at DNV for the presence of computer viruses. ********************************************************************** From david.abrahams at rcn.com Wed Jan 9 18:00:39 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 9 Jan 2002 12:00:39 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" > As I mentioned in a follow-up letter I've got now why using the template > parameter can be better. > We probably should stick with it. I know; sometimes it's best to spell things out, though. > Precisely. Otherwise, you end up having to create clumsy classes like > seqmap > to combine fine-grained capabilities. > > seqmap is object yes. > I was wondering if we could have object > and sequence as a typedef to it. Yes, that sounds reasonable... as long as people don't try to slice a sequence. Or, maybe we want the ability to impose some kind of checking mask, so that when people say "sequence" it has all of the obvious sequence interface, but only enforces "indexable", and might throw if something is unsupported. I want users to be comfortable as well as safe ;-) > Yes. I'll take a look at the python source. > In some cases we know how to handle things like o[i]: sequence access if i > is integral, mapping access otherwise. Not neccessarily: x = { 1 : 'one', 2 : 'two' } x[1] > Combining sequence an mapping in object would have to > figure that out. > In a generic solution like this how would we do that? I would always use the Python dispatching method unless we know the concrete type of the object. For example, we could have specializations of indexable_generator for when T is list. However, that's an optimization I would leave for later (much). > > then later offer try the meta-template approach with generators and > offer > > something like object<...> > > that has more fine-grained granularity and more control/type-checking? > > (discern between object and object<...> though). > > I think it will be problematic, unless we give up on validity checking. > How will users manipulate a sequence that turns out NOT to be sliceable? > > We would give up on validity checking (offer the accepts but not calling it > in the constructor). > I know it is not the way you want it, but acceptable for now ?? If it brings real benefits, then a small step is better than a big one which never gets finished. Sometimes the small step is just better overall ;-) > We could solve some issues like e.g. sliceable, but a number that does not > define 1 operation like multiplication: > The user would have to be aware of that. OK. I think it would be good to do it in terms of the concepts we've been working with, though. The generators, for example, will make it much easier to transition to a type_list-configurable system. We could imagine a protocol like this one: template struct indexable_generator : Base { ... }; where "checked" determines whether the capability is enforced at construction time. Then you can define your sequence, mapping, number,... and concrete list, dict, integer... types in terms of them. > > Another thing. > let's say we write a function that needs a number and a mapping: > void foo(const object &); > > Then I have an object. > I would like to be able to call foo on it without any problems/casts. I.e. > object is an object. > More generally an object can pass as an object if list2 is a > subset of list1. Yes, I was thinking about that but didn't address it. It is handled fairly easily with a templated constructor in object<>. You do need to STATIC_ASSERT that the capabilities are a subset. Easy enough using mpl. However, since the capabilities will remain relatively static, you might think about a simpler approach for now which uses a collection of BOOST_STATIC_CONSTANTs in a bitmask arrangement. Each generator might define its bitmask values, and they would appear combined in the most-derived class. > Using gen_linear_hierarchy, the inheritance tree (and types in it) depend > upon the order of types in the list. That's why my sample code tried to normalize the order, constructing the generators by iterating through all_capabilities. > It seems to me then that the objects are not connected then (apart > inheriting from individual generators, so they both pass as > indexable_generator etc.) I don't get what you're saying here. From arnaldur at decode.is Wed Jan 9 18:18:38 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Wed, 9 Jan 2002 17:18:38 +0000 Subject: boost::python , dispatching in python Message-ID: I've been looking at the python interpreter (parsetok,pythonrun,tokenizer,ceval,compile,...). It's a lot of code! In ceval the operation is selected on the opcode that the tokenizer or something similar sets. I didn't see the place where the opcode is selected on the symbol. However the following in ceval handle, a + b and a[i]: case BINARY_ADD: w = POP(); v = POP(); if (PyInt_CheckExact(v) && PyInt_CheckExact(w)) { /* INLINE: int + int */ register long a, b, i; a = PyInt_AS_LONG(v); b = PyInt_AS_LONG(w); i = a + b; if ((i^a) < 0 && (i^b) < 0) goto slow_add; x = PyInt_FromLong(i); } else { slow_add: x = PyNumber_Add(v, w); } Py_DECREF(v); Py_DECREF(w); PUSH(x); if (x != NULL) continue; break; case BINARY_SUBSCR: w = POP(); v = POP(); if (PyList_CheckExact(v) && PyInt_CheckExact(w)) { /* INLINE: list[int] */ long i = PyInt_AsLong(w); if (i < 0) i += PyList_GET_SIZE(v); if (i < 0 || i >= PyList_GET_SIZE(v)) { PyErr_SetString(PyExc_IndexError, "list index out of range"); x = NULL; } else { x = PyList_GET_ITEM(v, i); Py_INCREF(x); } } else x = PyObject_GetItem(v, w); Py_DECREF(v); Py_DECREF(w); PUSH(x); if (x != NULL) continue; break; We can see that BINARY_SUBSCR tries list access first but key access otherwise (PyObject_GetItem). sequence access thus should probably have precedence over key access if key is an integer (when an object has both sequence and mapping interfaces) BINARY_ADD calls PyNumber_Add. PyNumber_Add is : PyObject * PyNumber_Add(PyObject *v, PyObject *w) { PyObject *result = binary_op1(v, w, NB_SLOT(nb_add)); if (result == Py_NotImplemented) { PySequenceMethods *m = v->ob_type->tp_as_sequence; Py_DECREF(Py_NotImplemented); if (m && m->sq_concat) { result = (*m->sq_concat)(v, w); } else { PyErr_Format( PyExc_TypeError, "unsupported operand types for +: '%s' and '%s'", v->ob_type->tp_name, w->ob_type->tp_name); result = NULL; } } return result; } It tries nb_add but sequence concat if it fails. > > Yes. I'll take a look at the python source. > > In some cases we know how to handle things like o[i]: sequence access if i > > is integral, mapping access otherwise. > > Not neccessarily: > > x = { 1 : 'one', 2 : 'two' } > x[1] x is a dictionary so it only supports the mapping interface. No conflict with the sequence interface in this case. From arnaldur at decode.is Wed Jan 9 18:36:36 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Wed, 9 Jan 2002 17:36:36 +0000 Subject: boost::python, returning new PyObject references Message-ID: > Yes, that sounds reasonable... as long as people don't try to slice a > sequence. > > Or, maybe we want the ability to impose some kind of checking mask, so that > when people say "sequence" it has all of the obvious sequence interface, but > only enforces "indexable", and might throw if something is unsupported. Sounds good. The general case has to be easy. OK to have the option of total fine-grained control when needed. > I would always use the Python dispatching method unless we know the concrete > type of the object. For example, we could have specializations of > indexable_generator for when T is list. Sounds interesting. Maybe the way to go. > > We could solve some issues like e.g. sliceable, but a number that does not > > define 1 operation like multiplication: > > The user would have to be aware of that. > > OK. I think it would be good to do it in terms of the concepts we've been > working with, though. The generators, for example, will make it much easier > to transition to a type_list-configurable system. We could imagine a > protocol like this one: > > template > struct indexable_generator : Base > { > ... > }; > > where "checked" determines whether the capability is enforced at > construction time. Then you can define your sequence, mapping, number,... > and concrete list, dict, integer... types in terms of them Sounds reasonable. Would we add the type member template then when starting with mpl or is it not necessary for meta-programming? > Yes, I was thinking about that but didn't address it. It is handled fairly > easily with a templated constructor in object<>. You do need to > STATIC_ASSERT that the capabilities are a subset. Easy enough using mpl. > > However, since the capabilities will remain relatively static, you might > think about a simpler approach for now which uses a collection of > BOOST_STATIC_CONSTANTs in a bitmask arrangement. Each generator might define > its bitmask values, and they would appear combined in the most-derived > class. Don't quite get it. Sorry. > > It seems to me then that the objects are not connected then (apart > > inheriting from individual generators, so they both pass as > > indexable_generator etc.) > > I don't get what you're saying here. Has to do with the inheritance above. object: o == object empty - A - B - C - o object: o2 == object empty - A - C - o2 What is the relationship between o and o2? From david.abrahams at rcn.com Wed Jan 9 19:10:29 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 9 Jan 2002 13:10:29 -0500 Subject: boost::python , dispatching in python Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" > I've been looking at the python interpreter > (parsetok,pythonrun,tokenizer,ceval,compile,...). > It's a lot of code! With few comments! > In ceval the operation is selected on the opcode that the tokenizer or > something similar sets. > I didn't see the place where the opcode is selected on the symbol. > However the following in ceval handle, a + b and a[i]: > > case BINARY_ADD: > w = POP(); > v = POP(); > if (PyInt_CheckExact(v) && PyInt_CheckExact(w)) { > /* INLINE: int + int */ > register long a, b, i; > a = PyInt_AS_LONG(v); > b = PyInt_AS_LONG(w); > i = a + b; > if ((i^a) < 0 && (i^b) < 0) > goto slow_add; > x = PyInt_FromLong(i); > } > else { > slow_add: > x = PyNumber_Add(v, w); > } > Py_DECREF(v); > Py_DECREF(w); > PUSH(x); > if (x != NULL) continue; > break; So it looks like PyNumber_Add is the way to go for a + b > case BINARY_SUBSCR: > w = POP(); > v = POP(); > if (PyList_CheckExact(v) && PyInt_CheckExact(w)) { > /* INLINE: list[int] */ > long i = PyInt_AsLong(w); > if (i < 0) > i += PyList_GET_SIZE(v); > if (i < 0 || > i >= PyList_GET_SIZE(v)) { > PyErr_SetString(PyExc_IndexError, > "list index out of range"); > x = NULL; > } > else { > x = PyList_GET_ITEM(v, i); > Py_INCREF(x); > } > } > else > x = PyObject_GetItem(v, w); > Py_DECREF(v); > Py_DECREF(w); > PUSH(x); > if (x != NULL) continue; > break; > > We can see that BINARY_SUBSCR tries list access first but key access > otherwise (PyObject_GetItem). > sequence access thus should probably have precedence over key access if key > is an integer (when an object has both sequence and mapping interfaces) That's the wrong conclusion to draw. The check for list[int] is purely an optimization, just like the check for int+int in the first case. PyObject_GetItem() is the simple generalized routine that works for both sequence and mapping protocols: PyObject * PyObject_GetItem(PyObject *o, PyObject *key) { PyMappingMethods *m; if (o == NULL || key == NULL) return null_error(); m = o->ob_type->tp_as_mapping; if (m && m->mp_subscript) return m->mp_subscript(o, key); if (o->ob_type->tp_as_sequence) { if (PyInt_Check(key)) return PySequence_GetItem(o, PyInt_AsLong(key)); else if (PyLong_Check(key)) { long key_value = PyLong_AsLong(key); if (key_value == -1 && PyErr_Occurred()) return NULL; return PySequence_GetItem(o, key_value); } return type_error("sequence index must be integer"); } return type_error("unsubscriptable object"); } > BINARY_ADD calls PyNumber_Add. > PyNumber_Add is : > > It tries nb_add but sequence concat if it fails. Okay. > > > Yes. I'll take a look at the python source. > > > In some cases we know how to handle things like o[i]: sequence access > if i > > > is integral, mapping access otherwise. > > > > Not neccessarily: > > > > x = { 1 : 'one', 2 : 'two' } > > x[1] > > x is a dictionary so it only supports the mapping interface. No conflict > with the sequence interface in this case. There's no conflict, but the procedure you specified wouldn't work. Just use PyObject_GetItem(). Almost every operation with a possible slot conflict has a similar dispatcher, I'll wager. -Dave From david.abrahams at rcn.com Wed Jan 9 19:26:22 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 9 Jan 2002 13:26:22 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" > Sounds good. > The general case has to be easy. OK to have the option of total > fine-grained control when needed. Yep. > > OK. I think it would be good to do it in terms of the concepts we've been > > working with, though. The generators, for example, will make it much > easier > > to transition to a type_list-configurable system. We could imagine a > > protocol like this one: > > > > template > > struct indexable_generator : Base > > { > > ... > > }; > > > > where "checked" determines whether the capability is enforced at > > construction time. Then you can define your sequence, mapping, number,... > > and concrete list, dict, integer... types in terms of them > > Sounds reasonable. Would we add the type member template then when starting > with mpl or is it not necessary for meta-programming? I'm not sure what you mean by the "type member template". Maybe you mean the nested type "type". Whoops, I realize that I erred. We'd need a formulation that can be used with metafunctions: template index_capability : Base { proxy operator[](... ... }; // this part can be left out until you use mpl struct indexable { // apply is the analogue to operator()() template struct apply { // this is its return type typedef index_capability::type type; }; }; Does that answer your question? > > > > Yes, I was thinking about that but didn't address it. It is handled > fairly > > easily with a templated constructor in object<>. You do need to > > STATIC_ASSERT that the capabilities are a subset. Easy enough using mpl. Do you understand this part, at least abstractly? > > However, since the capabilities will remain relatively static, you might > > think about a simpler approach for now which uses a collection of > > BOOST_STATIC_CONSTANTs in a bitmask arrangement. Each generator might > define > > its bitmask values, and they would appear combined in the most-derived > > class. > > Don't quite get it. Sorry. How many slots are there? N. Now suppose we embed (N+15)/16 = M compile-time constants (e.g. enums) named slots1...slotsM in each capability template. Each one adds a single bit to the constants defined by its Base parameter. Then the assert becomes easy: the templated constructor just checks for I = 1 to M to be sure that (self::slotsI & ~Other::slotsI) == 0. > > > It seems to me then that the objects are not connected then (apart > > > inheriting from individual generators, so they both pass as > > > indexable_generator etc.) > > > > I don't get what you're saying here. > > Has to do with the inheritance above. > object: > o == object > > empty - A - B - C - o > > object: > o2 == object > > empty - A - C - o2 > > What is the relationship between o and o2? None. You can't rely on inheritance to do this conversion. That's why you use templated construction and assignment. -Dave From arnaldur at ru.is Wed Jan 9 20:20:34 2002 From: arnaldur at ru.is (Arnaldur Gylfason) Date: Wed, 9 Jan 2002 19:20:34 -0000 Subject: boost::python , dispatching in python Message-ID: This is a multi-part message in MIME format. ------_=_NextPart_001_01C19942.B6438D7A Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable > So it looks like PyNumber_Add is the way to go for a + b >=20 >=20 > That's the wrong conclusion to draw. The check for list[int] is purely an > optimization, just like the check for int+int in the first case. > PyObject_GetItem() is the simple generalized routine that works for both > sequence and mapping protocols: >=20 > There's no conflict, but the procedure you specified wouldn't work. Just use > PyObject_GetItem(). Almost every operation with a possible slot conflict has > a similar dispatcher, I'll wager. You're right! Forgot to take a look at PyObject_GetItem. I understand where you're going now. By calling PyNumber_Add and PyObject_GetItem, the python implementation already takes care of sequence/number, sequence/mapping conflicts for us. By picking the right python dispatcher we're off the hook.(I wish). Arnaldur From arnaldur at ru.is Wed Jan 9 20:28:34 2002 From: arnaldur at ru.is (Arnaldur Gylfason) Date: Wed, 9 Jan 2002 19:28:34 -0000 Subject: boost::python, returning new PyObject references Message-ID: This is a multi-part message in MIME format. ------_=_NextPart_001_01C19943.D451D1B8 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable > I'm not sure what you mean by the "type member template". Maybe you mean the > nested type "type". Whoops, I realize that I erred. We'd need a formulation > that can be used with metafunctions: >=20 > template > index_capability : Base > { > proxy operator[](... > ... > }; >=20 > // this part can be left out until you use mpl > struct indexable > { > // apply is the analogue to operator()() > template > struct apply > { > // this is its return type > typedef index_capability::type type; > }; > }; >=20 > Does that answer your question? Yes. This looks good! > > > > Yes, I was thinking about that but didn't address it. It is handled > fairly > > easily with a templated constructor in object<>. You do need to > > STATIC_ASSERT that the capabilities are a subset. Easy enough using mpl. > Do you understand this part, at least abstractly? Well. Taking a better look I think so. You're talking about providing an implicit conversion through a non-explicit templated constructor, right? I was so focused on that there should be is-A relationship between them that I thought that 1 should be a base class of the other. You're not talking about that. Am I getting this? > How many slots are there? N. Now suppose we embed (N+15)/16 =3D M compile-time > constants (e.g. enums) named slots1...slotsM in each capability template. > Each one adds a single bit to the constants defined by its Base parameter. > Then the assert becomes easy: the templated constructor just checks for I =3D > 1 to M to be sure that (self::slotsI & ~Other::slotsI) =3D=3D 0. OK. Makes the assert easier in the templated constructor? Cheers Arnaldur From david.abrahams at rcn.com Wed Jan 9 20:49:19 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 9 Jan 2002 14:49:19 -0500 Subject: boost::python, returning new PyObject references Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" <> Yes, just like std::pair, boost::shared_ptr, etc.... IOW, it's an established idiom. <> Perfectly. > How many slots are there? N. Now suppose we embed (N+15)/16 = M compile-time > constants (e.g. enums) named slots1...slotsM in each capability template. > Each one adds a single bit to the constants defined by its Base parameter. > Then the assert becomes easy: the templated constructor just checks for I = > 1 to M to be sure that (self::slotsI & ~Other::slotsI) == 0. <> Yes, at the expense of extensibility. If new slots are added, a user can't neccessarily extend the capabilities without modifying the library. A typelist-based approach could allow that. -Dave From david.abrahams at rcn.com Thu Jan 10 01:17:03 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 9 Jan 2002 19:17:03 -0500 Subject: Priorities Message-ID: Having struggled with the problem of conversions through an inheritance hierarchy for days, and having not heard from Ulli (who originally wrote this feature) about it, I feel the need for some feedback so I can decide what to do. I need to get a sense of the relative importance of this feature. Does it need to be fast? small? implemented quickly? I know that type conversions are an especially strong concern for Martin. Does this include implicit conversions through inheritance? =================================================== David Abrahams, C++ library designer for hire resume: http://users.rcn.com/abrahams/resume.html C++ Booster (http://www.boost.org) email: david.abrahams at rcn.com =================================================== From david.abrahams at rcn.com Thu Jan 10 20:44:20 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Thu, 10 Jan 2002 14:44:20 -0500 Subject: More object model questions Message-ID: ----- Original Message ----- From: "John H. Spicer" > > I'm afraid that after all that I don't have anything very interesting to > say except that it sounds like a good reason for extended RTTI. It also > sounds like we need a variant of dynamic_cast that can take a typeid > as the destination type. Now /that/ sounds promising! It knows something about the static type of the source so it can find vtbls, and the typeid can provide the rest of what's needed. Will you propose an extension? > I also think I agree with your original sentiment that you can't do what > you wanted to do with the dcast entries, although I'm still not sure I > understanding things well enough to be sure. No, I think you are :( > I hope it wasn't too much of a waste of your time explaining all this to me! Not too much; I've been learning a lot as I think about the problem. Man, it's complicated. I'm not sure how heroic to be. This is example is from 5.2.7 in the standard: [Example: class A { virtual void f(); }; class B { virtual void g(); }; class D : public virtual A, private B {}; void g() { D d; B* bp = (B*)&d; // cast needed to break protection A* ap = &d; // public derivation, no cast needed D& dr = dynamic_cast(*bp); // fails ap = dynamic_cast(bp); // fails bp = dynamic_cast(ap); // fails ap = dynamic_cast(&d); // succeeds bp = dynamic_cast(&d); // fails } class E : public D, public B {}; class F : public E, public D {}; void h() { F f; A* ap2 = &f; // succeeds: finds unique A D* dp = dynamic_cast(ap2); // fails: yields 0 // f has two D sub-objects E* ep = (E*)ap2; // ill-formed: // cast from virtual base E* ep1 = dynamic_cast(ap2); // succeeds } The graph looks like this, where A is a virtual base: A B ^ :\ +---+...: \ | \ bp --> D | ap --> |\ | | +-+---+ | | | E | | +-+-+ | ap2 --> F If the user has represented the entire graph to us, we /can/ find a D* given ap2: We detect that the most-derived type is F, so we dynamic_cast(ap2). Then we can cast up to E and from there, unambiguously to D. The other D is completely unreachable. From jhs at edg.com Thu Jan 10 20:56:37 2002 From: jhs at edg.com (John H. Spicer) Date: Thu, 10 Jan 2002 14:56:37 -0500 Subject: More object model questions Message-ID: David Abrahams wrote: > > ----- Original Message ----- > From: "John H. Spicer" > > > > > I'm afraid that after all that I don't have anything very interesting to > > say except that it sounds like a good reason for extended RTTI. It also > > sounds like we need a variant of dynamic_cast that can take a typeid > > as the destination type. > > Now /that/ sounds promising! > It knows something about the static type of the source so it can find vtbls, > and the typeid can provide the rest of what's needed. Will you propose an > extension? > I don't think I want to make a proposal because I don't think I'm in a good position to write a rationale. I'd be happy to contribute to a proposal though. John. From jhs at edg.com Thu Jan 10 20:59:37 2002 From: jhs at edg.com (John H. Spicer) Date: Thu, 10 Jan 2002 14:59:37 -0500 Subject: More object model questions Message-ID: David Abrahams wrote: > > ----- Original Message ----- > From: "John H. Spicer" > > > > > I'm afraid that after all that I don't have anything very interesting to > > say except that it sounds like a good reason for extended RTTI. It also > > sounds like we need a variant of dynamic_cast that can take a typeid > > as the destination type. > > Now /that/ sounds promising! > It knows something about the static type of the source so it can find vtbls, > and the typeid can provide the rest of what's needed. Will you propose an > extension? > I guess a fully general dynamic_cast would take a typeid for both the destination type and the type of the source operand. John. From arnaldur at decode.is Fri Jan 11 15:28:08 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Fri, 11 Jan 2002 14:28:08 +0000 Subject: python::object && mpl Message-ID: Dave, I've glanced through mpl and have a rough idea about it's function. >From what I gather, the use_generator is the Glue function class and the generators have to support templated inheritance but don't need to be function classes. The following design should be possible: template indexable : Base { proxy operator[](... ... }; ... template struct object : gen_linear_hierarchy< typename get_unique_generators >::type , use_generator< object, checked> > > { }; ... template struct use_generator { template struct apply { typedef Generator type; }; }; Is my understanding correct? What design would you opt for? Cheers Arnaldur From david.abrahams at rcn.com Fri Jan 11 16:50:52 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Fri, 11 Jan 2002 10:50:52 -0500 Subject: python::object && mpl Message-ID: ----- Original Message ----- From: "Arnaldur Gylfason" > > Dave, > > I've glanced through mpl and have a rough idea about it's function. > From what I gather, the use_generator Sorry, I've lost track of what use_generator is. Is that something I wrote in my little prototype? Oh, I see below. > is the Glue function class and the > generators have to support templated inheritance but don't need to be > function classes. By templated inheritance I assume you mean inheritance from a template argument? > The following design should be possible: > > template > indexable : Base > { > proxy operator[](... > ... > }; > > ... > > template > struct object > : gen_linear_hierarchy< > typename get_unique_generators >::type > , use_generator< > object, checked> > > > > > { > }; > ... > > template > struct use_generator > { > template > struct apply > { > typedef Generator type; ------------------^^^^^^^^^ this requires Generator to be a template template argument, not supported by all compilers. That's one reason to use the MPL metafunction approach: struct indexable { tempalate class struct apply { typedef indexable_t type; // where indexable_t is your "indexable", renamed }; }; Now you can pass indexable_generator as a regular template type argument. > }; > }; It looks like this design doesn't allow selecting individual capabilities for checking. I think it should be possible to specify that sequence minimally checks for indexable, but may have other capabilities, like sliceable, in its interface. > Is my understanding correct? > What design would you opt for? One possibility might be: template struct object : ... { }; You'd want to STATIC_ASSERT that all elements of CheckedList were in CapabilitiesList. Capabilities in CheckedList would also be checked. Another possibility would be to make all capabilities into templates, so template ()) {...} } This could be a reason to make the metafunctions available even if you don't make the generalized (metaprogram) object template now. -Dave From arnaldur at decode.is Fri Jan 11 17:23:51 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Fri, 11 Jan 2002 16:23:51 +0000 Subject: python::object && mpl Message-ID: <<< > struct use_generator > { > template > struct apply > { > typedef Generator type; ------------------^^^^^^^^^ this requires Generator to be a template template argument, not supported by all compilers. That's one reason to use the MPL metafunction approach: struct indexable { tempalate class struct apply { typedef indexable_t type; // where indexable_t is your "indexable", renamed }; }; Now you can pass indexable_generator as a regular template type argument. >>> I see. If our generators have 2 parameters we could use binary_function: template struct use_generator { template struct apply { typedef mpl::binary_function::type type; }; }; It would also apply to other usage of our generators as metaclass functions. There is this checked thing though. <<< One possibility might be: template struct object : ... { }; You'd want to STATIC_ASSERT that all elements of CheckedList were in CapabilitiesList. Capabilities in CheckedList would also be checked. >>> That's one way to do it. Have to think about this more thoroughly before I form an opinion about the best approach. <<< Another possibility would be to make all capabilities into templates, so template >> Something missing? <<< BTW, if we are going to allow some features to be present in the interface but unchecked there should be an interface for checking for support of those features without throwing an exception: void f(sequence x) { if (x.has_capability()) {...} } This could be a reason to make the metafunctions available even if you don't make the generalized (metaprogram) object template now. >>> Sounds good! How about has_capabilities on a type_list of capabilities and using mpl::for_each to validate all? x.has_capabilities >() / x.has_capabilities() / ... (integral_capabilities = mpl::type_list) Cheers Arnaldur From david.abrahams at rcn.com Fri Jan 11 17:44:09 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Fri, 11 Jan 2002 11:44:09 -0500 Subject: python::object && mpl Message-ID: > I see. > If our generators have 2 parameters we could use binary_function: > > template > struct use_generator > { > template > struct apply > { > typedef mpl::binary_function::type type; > }; > }; Sorry, I'm not familiar with binary_function and I don't know what it does. > <<< > Another possibility would be to make all capabilities into templates, so > > template > >>> > > Something missing? Uh, yeah, sorry. The capability generator would be enum checking_t { unchecked, checked }; template struct indexable { template <...> struct apply { ... }; }; Now you can write indexable<> most of the time, and if you want checking, indexable > Sounds good! > How about has_capabilities on a type_list of capabilities and using > mpl::for_each to validate all? Fine, but probably fancier than the average user needs. It would be perfect for your meta-object's constructor/assignment from PyObject*/ref, though. > x.has_capabilities >() / > x.has_capabilities() / ... > (integral_capabilities = mpl::type_list) Very nice. Looks like you're getting the hang of this metaprogramming stuff! -Dave From rwgk at yahoo.com Fri Jan 18 18:31:55 2002 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Fri, 18 Jan 2002 09:31:55 -0800 (PST) Subject: [C++-sig] Ann: New SIG for the development of Python/C++ integration Message-ID: <20020118173155.72971.qmail@web20203.mail.yahoo.com> The new Python C++ SIG is a forum for discussions about the integration of Python and C++, with an emphasis on the development of the Boost.Python library (http://www.boost.org/libs/python/doc/index.html). Currently, Boost.Python is undergoing a major rewrite to add powerful new features and to take full advantage of the new type system introduced in Python 2.2. A detailed list of the current development goals is posted at: http://www.python.org/sigs/c++-sig/ Suggestions regarding the direction of the development work and contributions in the form of source code should be posted to this SIG. About Boost.Python Boost.Python is a popular open-source tool for interfacing Python and C++. Boost.Python is built on accessible, well-maintained, well documented components (the Boost C++ Library), so it does not tie its users to the services of any one programmer or organization. Developers worldwide have contributed significant unsolicited improvements to the project. The main developer and coordinator of the library is David Abrahams. Relation to Boost forums Boost.Python related issues have also been discussed in the two Yahoo groups boost at yahoogroups.com and boost-users at yahoogroups.com (http://groups.yahoo.com/). Most discussions in these groups are concerned with advanced C++ library development. The Python SIG is focused on Python/C++ interfacing. __________________________________________________ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/ From rwgk at cci.lbl.gov Sat Jan 19 11:13:56 2002 From: rwgk at cci.lbl.gov (Ralf W. Grosse-Kunstleve) Date: Sat, 19 Jan 2002 02:13:56 -0800 (PST) Subject: [C++-sig] class empty: pass Message-ID: <200201191013.g0JADuT110762@boa.lbl.gov> It frequently happens that I want to expose a simple C++ struct to Python, e.g.: struct peak { boost::array index; double value; }; I often hesitate to use class_builder<> for small types like this because I assume that there is a large overhead associated with the proliferation of fully-fledged Python types. Therefore I just turn these structs into tuples or dictionaries. This is not very user-friendly, because it is not intuitive what peak[0] and peak[1] are, and the notation peak["index"] or peak["value"] is quite clunky. In pure Python I often use a construct like this: class empty: pass def foo(): result = empty() result.index = (1,2,3) result.value = 345 return result Could a similar mechanism be provided at the C++ level? E.g.: py_peak = empty_builder; py_peak.def(&peak::index, "index"); py_peak.def(&peak::value, "value"); empty_builder would not build a new extension class for each template argument, but reuse the same Python type. Does this make sense? Would there be a significant saving in object code size (particularly in the new system that is based on the native Python type system)? Thanks, Ralf From rwgk at cci.lbl.gov Sat Jan 19 12:35:16 2002 From: rwgk at cci.lbl.gov (Ralf W. Grosse-Kunstleve) Date: Sat, 19 Jan 2002 03:35:16 -0800 (PST) Subject: [C++-sig] template constructor problem Message-ID: <200201191135.g0JBZGo111373@boa.lbl.gov> The following code compiles fine (e.g. with gcc 3.0.3 or Compaq cxx 6.2) as shown. However, if the constructor for struct stats is turned into a template constructor (move the //), gcc reports: template_constructor.cpp:20: no matching function for call to `vector_min(const boost::python::detail::reference_parameter >&>&)' I assume because reference_parameter<> does not have a typename value_type. Is there an easy way to get around this problem? Thanks, Ralf P.S.: My best workaround involves inheriting from std::vector, class_build both std::vector and the inheriting type, declare_base, define the constructor only for the inheriting type... #include #include template typename VectorType::value_type vector_min(const VectorType& v) { typename VectorType::value_type result = v[0]; for(std::size_t i=1;i v[i]) result = v[i]; } return result; } template struct stats { //template stats(const VectorType& vec) { stats(const std::vector& vec) { m_min = vector_min(vec); } ValueType m_min; }; BOOST_PYTHON_MODULE_INIT(template_constructor) { boost::python::module_builder this_module("template_constructor"); boost::python::class_builder > py_svf(this_module, "svf"); py_svf.def(boost::python::constructor<>()); boost::python::class_builder > py_stats(this_module, "stats"); py_stats.def(boost::python::constructor&>()); } From david.abrahams at rcn.com Sat Jan 19 17:03:11 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sat, 19 Jan 2002 11:03:11 -0500 Subject: [C++-sig] class empty: pass Message-ID: <01a101c1a103$02bcb560$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Ralf W. Grosse-Kunstleve" > It frequently happens that I want to expose a simple C++ struct > to Python, e.g.: > > struct peak { > boost::array index; > double value; > }; > > I often hesitate to use class_builder<> for small types > like this because I assume that there is a large overhead > associated with the proliferation of fully-fledged Python > types. Therefore I just turn these structs into tuples > or dictionaries. This is not very user-friendly, because > it is not intuitive what peak[0] and peak[1] are, and > the notation peak["index"] or peak["value"] is quite > clunky. > > In pure Python I often use a construct like this: > > class empty: pass > > def foo(): > result = empty() > result.index = (1,2,3) > result.value = 345 > return result > > Could a similar mechanism be provided at the C++ level? > E.g.: > > py_peak = empty_builder; > py_peak.def(&peak::index, "index"); > py_peak.def(&peak::value, "value"); > > empty_builder would not build a new extension class for each template > argument, but reuse the same Python type. Yes! But how would you make one? Remember, the type of the object is what you call to create it: x = MyClass() If you have a C++ function which can return them, though, you don't need a separate type. > Does this make sense? Would there be a significant saving > in object code size (particularly in the new system that is > based on the native Python type system)? I think it would be small at best. Aren't you just doing this for convenience? Why don't we discuss the ideal syntax, instead of worrying about implementation details at this stage? -Dave From david.abrahams at rcn.com Sat Jan 19 17:03:38 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sat, 19 Jan 2002 11:03:38 -0500 Subject: [C++-sig] template constructor problem Message-ID: <01a201c1a103$02dce790$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Ralf W. Grosse-Kunstleve" > template > struct stats > { > file://template stats(const VectorType& vec) { > stats(const std::vector& vec) { ----------^^^^^^^^^^^^^^^^^^^^^^^^^ > m_min = vector_min(vec); > } > ValueType m_min; > }; > > BOOST_PYTHON_MODULE_INIT(template_constructor) > { > boost::python::module_builder this_module("template_constructor"); > > boost::python::class_builder > py_svf(this_module, "svf"); > py_svf.def(boost::python::constructor<>()); > > boost::python::class_builder > py_stats(this_module, "stats"); > py_stats.def(boost::python::constructor&>()); --------------------------------------------^^^^^^^^^^^^^^^^^^^ > } > I've begun writing "Foo const&" instead of "const Foo&" because the const seems to get lost in the second case. Also, it's more consistent with pointer notation, e.g. Foo* const. -Dave From david.abrahams at rcn.com Sun Jan 20 01:12:34 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sat, 19 Jan 2002 19:12:34 -0500 Subject: [C++-sig] test Message-ID: <029801c1a147$2a1713f0$0500a8c0@boostconsulting.com> Sorry to waste bandwidth, just testing. =================================================== David Abrahams, C++ library designer for hire resume: http://users.rcn.com/abrahams/resume.html C++ Booster (http://www.boost.org) email: david.abrahams at rcn.com =================================================== From rwgk at cci.lbl.gov Sun Jan 20 01:23:52 2002 From: rwgk at cci.lbl.gov (Ralf W. Grosse-Kunstleve) Date: Sat, 19 Jan 2002 16:23:52 -0800 (PST) Subject: [C++-sig] class empty: pass Message-ID: <200201200023.g0K0NqU116774@boa.lbl.gov> > > Could a similar mechanism be provided at the C++ level? > > E.g.: > > > > py_peak = empty_builder; > > py_peak.def(&peak::index, "index"); > > py_peak.def(&peak::value, "value"); > > > > empty_builder would not build a new extension class for each template > > argument, but reuse the same Python type. > > Yes! But how would you make one? Remember, the type of the object is what > you call to create it: > x = MyClass() Couldn't there be a Boost.Python empty type which is always there? Then the converters for peak could instantiate the type and copy (that would be sufficient) the data members to the __dict__. If the user wants anything more complicated (functions, references) class_builder must be used. > If you have a C++ function which can return them, though, you don't need a > separate type. > > > Does this make sense? Would there be a significant saving > > in object code size (particularly in the new system that is > > based on the native Python type system)? > > I think it would be small at best. Aren't you just doing this for > convenience? Why don't we discuss the ideal syntax, instead of worrying > about implementation details at this stage? No, my main concern is code size. class_builder is quite convenient already. Unless Boost.Python could automatically figure out the list of data members that need to be copied to the __dict__, I have difficulties seeing how the syntax could be improved. Anyway, if there is no significant saving in code size I don't believe a second mechanism besides class_builder is beneficial. Thanks, Ralf From rwgk at cci.lbl.gov Sun Jan 20 01:31:35 2002 From: rwgk at cci.lbl.gov (Ralf W. Grosse-Kunstleve) Date: Sat, 19 Jan 2002 16:31:35 -0800 (PST) Subject: [C++-sig] template constructor problem Message-ID: <200201200031.g0K0VZt116866@boa.lbl.gov> > template > struct stats > { > //template stats(const VectorType& vec) { > stats(const std::vector& vec) { ----------^^^^^^^^^^^^^^^^^^^^^^^^^ > py_stats.def(boost::python::constructor&>()); --------------------------------------------^^^^^^^^^^^^^^^^^^^ > I've begun writing "Foo const&" instead of "const Foo&" because the const > seems to get lost in the second case. Also, it's more consistent with > pointer notation, e.g. Foo* const. Ah, sorry for the little glitch. However, the extra const does not make a difference, no matter where I put the const (const Foo& or Foo const&). The cxx 6.2 error message is: cxx: Error: template_constructor.cpp, line 20: #304 no instance of function template "vector_min" matches the argument list argument types are: (const boost::python::detail::reference_parameter> &>::const_reference>) The (almost identical) code is again attached. Isn't the problem that the compiler does not deduce T const& but boost::python::detail::reference_parameter<...<...>::const_reference>, and that reference_parameter does not have a ::value_type? Thanks, Ralf From rwgk at cci.lbl.gov Sun Jan 20 01:34:48 2002 From: rwgk at cci.lbl.gov (Ralf W. Grosse-Kunstleve) Date: Sat, 19 Jan 2002 16:34:48 -0800 (PST) Subject: [C++-sig] template constructor problem Message-ID: <200201200034.g0K0Ym4116872@boa.lbl.gov> > The (almost identical) code is again attached. Sorry for the lie. Here is the code: #include #include template typename VectorType::value_type vector_min(VectorType const& v) { typename VectorType::value_type result = v[0]; for(std::size_t i=1;i v[i]) result = v[i]; } return result; } template struct stats { template stats(VectorType const& vec) { //stats(std::vector const& vec) { m_min = vector_min(vec); } ValueType m_min; }; BOOST_PYTHON_MODULE_INIT(template_constructor) { boost::python::module_builder this_module("template_constructor"); boost::python::class_builder > py_svf(this_module, "svf"); py_svf.def(boost::python::constructor<>()); boost::python::class_builder > py_stats(this_module, "stats"); py_stats.def(boost::python::constructor const&>()); } From david.abrahams at rcn.com Sun Jan 20 01:58:21 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sat, 19 Jan 2002 19:58:21 -0500 Subject: [C++-sig] class empty: pass References: <200201200023.g0K0NqU116774@boa.lbl.gov> Message-ID: <02ae01c1a14e$3150e270$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Ralf W. Grosse-Kunstleve" > No, my main concern is code size. class_builder is quite convenient > already. Unless Boost.Python could automatically figure out the > list of data members that need to be copied to the __dict__, I have > difficulties seeing how the syntax could be improved. > Anyway, if there is no significant saving in code size I don't > believe a second mechanism besides class_builder is beneficial. In general, every new mechanism adds at least a little code side. This isn't strictly true when you're working with templates, but I don't think a new class will generate very much code in the new system. -Dave From david.abrahams at rcn.com Sun Jan 20 02:36:23 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sat, 19 Jan 2002 20:36:23 -0500 Subject: [C++-sig] template constructor problem References: <200201200034.g0K0Ym4116872@boa.lbl.gov> Message-ID: <02d401c1a153$1b41d020$0500a8c0@boostconsulting.com> I understand the problem. I think I can come up with something for you... -Dave ----- Original Message ----- From: "Ralf W. Grosse-Kunstleve" To: Sent: Saturday, January 19, 2002 7:34 PM Subject: Re: [C++-sig] template constructor problem > > The (almost identical) code is again attached. > > Sorry for the lie. Here is the code: > > #include > #include > > template > typename VectorType::value_type > vector_min(VectorType const& v) > { > typename VectorType::value_type result = v[0]; > for(std::size_t i=1;i if (result > v[i]) result = v[i]; > } > return result; > } > > template > struct stats > { > template stats(VectorType const& vec) { > file://stats(std::vector const& vec) { > m_min = vector_min(vec); > } > ValueType m_min; > }; > > BOOST_PYTHON_MODULE_INIT(template_constructor) > { > boost::python::module_builder this_module("template_constructor"); > > boost::python::class_builder > py_svf(this_module, "svf"); > py_svf.def(boost::python::constructor<>()); > > boost::python::class_builder > py_stats(this_module, "stats"); > py_stats.def(boost::python::constructor const&>()); > } > > _______________________________________________ > C++-sig mailing list > C++-sig at python.org > http://mail.python.org/mailman/listinfo/c++-sig From david.abrahams at rcn.com Sun Jan 20 04:09:54 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Sat, 19 Jan 2002 22:09:54 -0500 Subject: [C++-sig] template constructor problem References: <200201200034.g0K0Ym4116872@boa.lbl.gov> Message-ID: <033f01c1a160$4a467b70$0500a8c0@boostconsulting.com> Okay, try the current CVS state -Dave ----- Original Message ----- From: "Ralf W. Grosse-Kunstleve" To: Sent: Saturday, January 19, 2002 7:34 PM Subject: Re: [C++-sig] template constructor problem > > The (almost identical) code is again attached. > > Sorry for the lie. Here is the code: > > #include > #include > > template > typename VectorType::value_type > vector_min(VectorType const& v) > { > typename VectorType::value_type result = v[0]; > for(std::size_t i=1;i if (result > v[i]) result = v[i]; > } > return result; > } > > template > struct stats > { > template stats(VectorType const& vec) { > file://stats(std::vector const& vec) { > m_min = vector_min(vec); > } > ValueType m_min; > }; > > BOOST_PYTHON_MODULE_INIT(template_constructor) > { > boost::python::module_builder this_module("template_constructor"); > > boost::python::class_builder > py_svf(this_module, "svf"); > py_svf.def(boost::python::constructor<>()); > > boost::python::class_builder > py_stats(this_module, "stats"); > py_stats.def(boost::python::constructor const&>()); > } > > _______________________________________________ > C++-sig mailing list > C++-sig at python.org > http://mail.python.org/mailman/listinfo/c++-sig From eric at enthought.com Sun Jan 20 03:28:13 2002 From: eric at enthought.com (eric) Date: Sat, 19 Jan 2002 21:28:13 -0500 Subject: [C++-sig] weave plus boost? Message-ID: <017401c1a15a$1d717d30$777ba8c0@ericlaptop> Hey group, I've just signed up to listen in on the group. I haven't followed boost to closely, but am interested in catching up with what this community is doing. My main interest here is to see if there is anyway that weave can benefit from the boost effort -- I'm almost sure there is. weave (www.scipy.org/site_content/weave) is a tool for, among other things, inlining C/C++ within Python. It also has a module called ext_tools that allows you to build C/C++ extension modules using Python. Currently, ext_tools only supports building modules that contain functions. I'd like to eventually extend it so that it handles extension classes as well. It looks like it won't be to bad, but boost might make some aspects of this more elegant and easier for the end user. Also, currently, weave converts Python containers to CXX List, Tuple, and Dict class objects. I'm strongly considering switching to SCXX because they are quite a bit simpler and more portable. CXX is a little heavier duty than weave really needs. I expect boost provides another alternative here. I'd like to hear opinions on the subject. The type conversions in weave are actually "pluggable", so I expect in the end there will be conversion classes for using CXX, SCXX, raw PyObject* pointers, and boost allowing the user to choose what he/she likes best. To close, I'll provide a couple of trivial examples of how weave works now. Hopefully that'll give some people ideas on how boost might improve the tool: >>> import weave >>> a = 1 >>> weave.inline('std::cout << a << std::endl;',['a']) 1 >>> a = 'string' >>> weave.inline('std::cout << a << std::endl;',['a']) string -------------------------------------------------------- >>> from weave import ext_tools >>> m = ext_tools.ext_module('test_mod') >>> a = 1 >>> code = 'return_val = Py::new_reference_to(Py::Int(a+1));' >>> f = ext_tools.ext_function('func', code, ['a']) >>> m.add_function(f) >>> m.compile() >>> import test_mod >>> test_mod.func(2) 3 I look forward to a conversation. thanks, eric -- Eric Jones Enthought, Inc. [www.enthought.com and www.scipy.org] (512) 536-1057 From rwgk at yahoo.com Sun Jan 20 15:00:48 2002 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Sun, 20 Jan 2002 06:00:48 -0800 (PST) Subject: [C++-sig] weave plus boost? In-Reply-To: <017401c1a15a$1d717d30$777ba8c0@ericlaptop> Message-ID: <20020120140048.56553.qmail@web20208.mail.yahoo.com> > My main interest here is to see if there is anyway that > weave can benefit from the boost effort -- I'm almost sure > there is. weave (www.scipy.org/site_content/weave) is a > tool for, among other things, inlining C/C++ within > Python. What is the main purpose of weave? Where is it applied? Is my impression correct that weave uses python to call the C/C++ compiler? How do you address the issue of portability (custom compiler flags, choice of compiler, etc.)? My view of the Python/C++ world is roughly this: I use Python where I can (i.e. performance is not critical) and C++ where I must. Universally, the C++ code operates on larger blocks of data (in the simplest case a vector). Standard vector operations would be best covered by a vector library (ecological niche of NumPy, but ideally with true C++ vector types), more complex operations can easily be interfaced via Boost.Python (i.e. the interface code tends to be small compared to the code for the complex operation). How does weave fit into this picture? Thanks, Ralf __________________________________________________ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/ From rwgk at yahoo.com Sun Jan 20 16:37:21 2002 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Sun, 20 Jan 2002 07:37:21 -0800 (PST) Subject: [C++-sig] template constructor problem In-Reply-To: <033f01c1a160$4a467b70$0500a8c0@boostconsulting.com> Message-ID: <20020120153721.44117.qmail@web20209.mail.yahoo.com> --- David Abrahams wrote: > Okay, try the current CVS state Yes! Fantastic! Not only the posted example works, but my application, too. The wrapper code looks a lot nicer now. Thanks a lot! Ralf __________________________________________________ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/ From eric at enthought.com Sun Jan 20 21:25:15 2002 From: eric at enthought.com (eric) Date: Sun, 20 Jan 2002 15:25:15 -0500 Subject: [C++-sig] weave plus boost? References: <20020120140048.56553.qmail@web20208.mail.yahoo.com> Message-ID: <015501c1a1f0$926a5670$777ba8c0@ericlaptop> Hey Ralf, > > My main interest here is to see if there is anyway that > > weave can benefit from the boost effort -- I'm almost sure > > there is. weave (www.scipy.org/site_content/weave) is a > > tool for, among other things, inlining C/C++ within > > Python. > > What is the main purpose of weave? Where is it applied? Right now, I think it is best applied for the following: 1. Fast algorithms. This is what I use it for most. 2. Extending the capabilities of an extension library without building an entire new extension. 3. Handling problems that are more easily solved in C/C++ > Is my impression correct that weave uses python to call the C/C++ > compiler? How do you address the issue of portability (custom > compiler flags, choice of compiler, etc.)? weave uses distutils to build extensions on the fly. It makes a simple hacks to force the linker to be g++ instead of gcc when gnu is used so that C++ files work appropriately. It also works using Microsoft C++ compiler. distutils has a method to specify compiler flags on a global bases. If you need per-file specific flags, this isn't handled. Switching compilers between MSVC and gnu is possible on windows, but not on other platforms. The same compiler that was used to compile Python will be used to compile your extension -- distutils reads this information from a Makefile distributed with with Python. There are ways of customizing this, but it requires effort. > > My view of the Python/C++ world is roughly this: > > I use Python where I can (i.e. performance is not critical) > and C++ where I must. Universally, the C++ code operates on > larger blocks of data (in the simplest case a vector). > Standard vector operations would be best covered by a > vector library (ecological niche of NumPy, but ideally with > true C++ vector types), more complex operations can easily be > interfaced via Boost.Python (i.e. the interface code tends > to be small compared to the code for the complex operation). We have similar Python/C++ world views. I use Python for pretty much everything, but found that, even with Numeric, there were many situations where I needed more speed for an algorithm. This is primarily for scientific applications, but also comes up in other situations. So, weave is designed to solve these problems as easily as possible. > How does weave fit into this picture? I'll give a couple more examples that'll hopefully give you a better picture. For more examples, go here: http://www.scipy.org/site_content/weave/tutorial.html http://www.scipy.org/site_content/weave/python_performance.html Laplace equation: Here is an example of the inline code for solving the Laplace equation. It converts Numeric arrays to Blitz++ arrays (a C++ array library). It is pretty simple to specify how Python types are converted to C++ types, so if you have a vector library you prefer (or STL or whatever), this is not hard. The following is a Python function with the expensive part of the algorithm coded in C++ using inline(): def inlineTimeStep(self, dt=0.0): """ Takes a time step using inlined C code -- this version uses blitz arrays. """ g = self.grid nx, ny = g.u.shape dx2, dy2 = g.dx**2, g.dy**2 dnr_inv = 0.5/(dx2 + dy2) u = g.u code = """ #line 120 "laplace.py" (This is only useful for debugging) double tmp, err, diff; err = 0.0; for (int i=1; i From casado2 at llnl.gov Sun Jan 20 23:53:42 2002 From: casado2 at llnl.gov (Martin Casado) Date: Sun, 20 Jan 2002 14:53:42 -0800 Subject: [C++-sig] weave plus boost? In-Reply-To: <017401c1a15a$1d717d30$777ba8c0@ericlaptop> References: <017401c1a15a$1d717d30$777ba8c0@ericlaptop> Message-ID: <0201201453422D.22770@avalanche.llnl.gov> On Saturday 19 January 2002 18:28, you wrote: > Hey group, > > I've just signed up to listen in on the group. I haven't followed boost to > closely, but am interested in catching up with what this community is > doing. > > My main interest here is to see if there is anyway that weave can benefit > from the boost effort -- I'm almost sure there is. weave > (www.scipy.org/site_content/weave) is a tool for, among other things, > inlining C/C++ within Python. It also has a module called ext_tools that > allows you to build C/C++ extension modules using Python. Currently, > ext_tools only supports building modules that contain functions. I'd like > to eventually extend it so that it handles extension classes as well. It > looks like it won't be to bad, but boost might make some aspects of this > more elegant and easier for the end user. Hi Eric, Here is an idea I toyed around with last summer. One of the challenges with making extension types that represent C++ classes is trying to come up with adequate ways to support templated classes. Pretty much all packages I've looked at to automate the wrapping process require you to create a seperate class for each template instantiation... which makes sense since template instantion is done at compile time. However, if I have a templated class, vector, I would really like to construct an instance of vector in python, like. myVector = vector("std::string") It is impossible to know which instances of vector are going to be needed before runtime with this method, so one option is to have vector(..) generate an instance of vector on the fly, compile it and load it in. Each instance extension would proabaly be cached, keyed by the type so subsequent requests for vector("std::string") wouldn't have to redo the process. I'm not sure how practical this would be, but at one time I had a sloppy but working prototype and it was.. if nothing else.. neat. Cheers, ~~m From eric at enthought.com Mon Jan 21 02:13:53 2002 From: eric at enthought.com (eric) Date: Sun, 20 Jan 2002 20:13:53 -0500 Subject: [C++-sig] weave plus boost? References: <017401c1a15a$1d717d30$777ba8c0@ericlaptop> <0201201453422D.22770@avalanche.llnl.gov> Message-ID: <016901c1a218$e4352ed0$777ba8c0@ericlaptop> Hey Martin, Good to hear from you! Hope all is well. Any chance I'll see you at the Python conference? > It is impossible to know which instances of vector are going to be needed > before runtime with this method, so one option is to have vector(..) generate > an > instance of vector on the fly, compile it and load it in. Each > instance extension > would proabaly be cached, keyed by the type so subsequent requests for > vector("std::string") wouldn't have to redo the process. > > I'm not sure how practical this would be, but at one time I had a sloppy but > working prototype and it was.. if nothing else.. neat. You've pretty much described how weave.inline works. Because python is dynamic and C is statically typed, weave handles all argument types this way. The case of templates fits seamlessly and nicely into this scheme. To describe how it works, lets first look a simple code fragment and see how weave handles it. >>> import weave >>> code = "std::cout << a << std::endl;" >>> a = 1 >>> weave.inline(code, ['a']) 1 >>> a = 'string' >>> weave.inline(code, ['a']) < another delay for the compile> string >>> weave.inline(code, ['a']) 1 In the example, the first time weave is called, it has never seen the C++ code fragment contained in 'code'. As a result, it generates a wrapper function, fires up the compiler, and builds the extension function, . weave enters this function in a "catalog" of all the functions its compiled. The catalog is a dictionary that relates a given C++ code fragment with a *list* of extension functions. catalog[code] = [] The second time weave.inline is called, it is called with the exact same code fragment. weave looks in its catalog and sees that it has compiled functions for this code fragment before. weave then starts at the front of the list of functions and starts calling them one by one. The compiled functions have type checks in them that throw a ConversionError exception (well techincally a TypeError) when the incoming argument types don't match the argument types the function was compiled for. If you get to the end of the list of functions and they have all thrown ConversionError, weave will compile a new function for the new argument types. In the 2nd case above where 'code' is called with a string, , 'code' would get recompiled for strings to a second function and added to the front of the catalog. catalog[code] = [,] The same thing happens for template arguments. one of the "type conversion factories" used to generate the code for converting Python Numeric arrays to C++ data types uses blitz++ arrays on the C++ side. blitz++ arrays have two template parameters, their data type and their size. template class array ... There is an example in the weave/examples directory called cast_copy_transpose.py that is useful in handling some of the issues that come up in using Python with Fortran code. We'll show a modified version of this example to illustrate how this can work: def copy_transpose(a_2d): assert(len(shape(a_2d)) == 2) new_array = zeros(shape(a_2d),a.typecode()) code = """ for(int i = 0; i < _Na_2d[0]; i++) for(int j = 0; j < _Na_2d[1]; j++) new_array(i,j) = a_2d(j,i); """ weave.inline(code,['new_array','a_2d'], type_factories = blitz_type_factories, compiler='gcc') return new_array Numeric arrays come in about 10 different types (byte, int, float , double, complex float, ...) and this function works for all of them -- weave generates them on the fly. The catalog is one of the more involved pieces of code in weave. It handles persistence of functions so that you don't have to re-compile every time you restart the interpreter, and it also does some caching to keep overhead of calling the same function multiple times in an inner loop to a minimum. So, within the context of inline, the catalog handles what you are describing pretty well. If you see anything missing that your implementation did well, please let me know, and we'll try to work it in somehow. thanks, eric ----- Original Message ----- From: "Martin Casado" To: Sent: Sunday, January 20, 2002 5:53 PM Subject: Re: [C++-sig] weave plus boost? > On Saturday 19 January 2002 18:28, you wrote: > > Hey group, > > > > I've just signed up to listen in on the group. I haven't followed boost to > > closely, but am interested in catching up with what this community is > > doing. > > > > My main interest here is to see if there is anyway that weave can benefit > > from the boost effort -- I'm almost sure there is. weave > > (www.scipy.org/site_content/weave) is a tool for, among other things, > > inlining C/C++ within Python. It also has a module called ext_tools that > > allows you to build C/C++ extension modules using Python. Currently, > > ext_tools only supports building modules that contain functions. I'd like > > to eventually extend it so that it handles extension classes as well. It > > looks like it won't be to bad, but boost might make some aspects of this > > more elegant and easier for the end user. > > Hi Eric, > > Here is an idea I toyed around with last summer. One of the > challenges with making extension types that represent C++ classes is > trying to come up with adequate ways to support templated classes. Pretty > much all packages I've looked at to automate the wrapping process require you > to create a seperate class for each template instantiation... which makes > sense > since template instantion is done at compile time. However, if I have a > templated class, vector, I would really like to construct an > instance > of vector in python, like. > > myVector = vector("std::string") > > It is impossible to know which instances of vector are going to be needed > before runtime with this method, so one option is to have vector(..) generate > an > instance of vector on the fly, compile it and load it in. Each > instance extension > would proabaly be cached, keyed by the type so subsequent requests for > vector("std::string") wouldn't have to redo the process. > > I'm not sure how practical this would be, but at one time I had a sloppy but > working prototype and it was.. if nothing else.. neat. > > > Cheers, > ~~m > > _______________________________________________ > C++-sig mailing list > C++-sig at python.org > http://mail.python.org/mailman/listinfo/c++-sig > From david.abrahams at rcn.com Mon Jan 21 08:20:07 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 21 Jan 2002 02:20:07 -0500 Subject: [C++-sig] weave plus boost? References: <017401c1a15a$1d717d30$777ba8c0@ericlaptop> Message-ID: <07ac01c1a24c$11f68f70$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "eric" > Hey group, > > I've just signed up to listen in on the group. I haven't followed boost to > closely, but am interested in catching up with what this community is doing. > > My main interest here is to see if there is anyway that weave can benefit > from the boost effort -- I'm almost sure there is. weave > (www.scipy.org/site_content/weave) is a tool for, among other things, > inlining C/C++ within Python. It also has a module called ext_tools that > allows you to build C/C++ extension modules using Python. Currently, > ext_tools only supports building modules that contain functions. I'd like > to eventually extend it so that it handles extension classes as well. It > looks like it won't be to bad, but boost might make some aspects of this > more elegant and easier for the end user. Possibly so. > Also, currently, weave converts Python containers to CXX List, Tuple, and > Dict class objects. I'm strongly considering switching to SCXX because they > are quite a bit simpler and more portable. CXX is a little heavier duty > than weave really needs. I expect boost provides another alternative here. Yes, it does, although there's no documentation for that part of Boost.Python yet. The framework for manipulating the built-in Python types is currently similar to the facility you get from CXX, but Arlnaldur Gylfason is working on a new design. A great strength of Boost.Python is its ability to provide extensible automatic type conversions between Python and C++ without a separate compilation step. When applied to objects like lists and dicts is that, if you've got to_python converters for, say my_class and std::vector, you can write: int f(myclass const& x, std::vector const& y) { tuple three(1, x, y); std::string msg("this is a test"); dict d; d[msg] = 5; } ...etc. Boost.Python has a different paradigm than weave does. A principal idea of Boost.Python is that you should be able to easily expose a standalone library of C++ types and functions to Python without writing lots of "glue" (e.g. argument checking and translation) and without intrusive changes to the library. Weave's orientation seems to be towards writing C++ bodies for Python functions. It should be very quick for those who don't want to leave their Python editor just to optimize a function. However, it seems like that approach wouldn't be very compatible with the idea of a standalone C++ library unless you just plan to write little wads of glue code in weave and dispatch to your library. It also seems as though it might not scale well, since (and I'm guessing here) you have to cross a Python function boundary every time you want to break off a new piece of functionality. I'm not saying one or the other paradigm is best - they seem to serve different needs. Whether weave can benefit somehow from Boost.Python isn't clear to me, but I hope what I've said helps to give you an idea. -Dave From eric at enthought.com Mon Jan 21 08:37:31 2002 From: eric at enthought.com (eric) Date: Mon, 21 Jan 2002 02:37:31 -0500 Subject: [C++-sig] weave plus boost? References: <017401c1a15a$1d717d30$777ba8c0@ericlaptop> <07ac01c1a24c$11f68f70$0500a8c0@boostconsulting.com> Message-ID: <009901c1a24e$7f6133b0$777ba8c0@ericlaptop> > ----- Original Message ----- > From: "eric" > > > Hey group, > > > > I've just signed up to listen in on the group. I haven't followed boost > to > > closely, but am interested in catching up with what this community is > doing. > > > > My main interest here is to see if there is anyway that weave can benefit > > from the boost effort -- I'm almost sure there is. weave > > (www.scipy.org/site_content/weave) is a tool for, among other things, > > inlining C/C++ within Python. It also has a module called ext_tools that > > allows you to build C/C++ extension modules using Python. Currently, > > ext_tools only supports building modules that contain functions. I'd like > > to eventually extend it so that it handles extension classes as well. It > > looks like it won't be to bad, but boost might make some aspects of this > > more elegant and easier for the end user. > > Possibly so. > > > Also, currently, weave converts Python containers to CXX List, Tuple, and > > Dict class objects. I'm strongly considering switching to SCXX because > they > > are quite a bit simpler and more portable. CXX is a little heavier duty > > than weave really needs. I expect boost provides another alternative > here. > > Yes, it does, although there's no documentation for that part of > Boost.Python yet. The framework for manipulating the built-in Python types > is currently similar to the facility you get from CXX, but Arlnaldur > Gylfason is working on a new design. > > A great strength of Boost.Python is its ability to provide extensible > automatic type conversions between Python and C++ without a separate > compilation step. When applied to objects like lists and dicts is that, if > you've got to_python converters for, say my_class and std::vector, > you can write: > > int f(myclass const& x, std::vector const& y) > { > tuple three(1, x, y); > std::string msg("this is a test"); > dict d; > d[msg] = 5; > } That looks very nice -- especially the d[msg] = 5 like syntax. Can you initialize a tuple, dict, list, etc. from a PyObject*? If so, they are probably close to interchangeable with the current CXX implementation I use, and it would be easy to create a boost backend to weave. How far along are these basic objects in development? Can they handle assignments (like the d[msg]=5 statement) from int,float,strings, etc. now? Also, how is the speed compared to using Python API calls? I found that using the nice features of CXX (indexing, assignment, etc) in innerloops was hard on performance. > > ...etc. > > Boost.Python has a different paradigm than weave does. Right. My understanding is that boost is CXXish (my main point of reference) in flavor -- i.e. made for writing stand alone C++ libs that are usable from Python. weave, on the other hand, takes a "build it entirely in Python then optimize the hot spots approach". weave has to use something underneath, though, to convert from Python to C/C++. This could be Python API calls (and many would rather have this), but that results in not so pretty code that can intimidate people unfamiliar with the API, ref counting, and error handling. CXX handles most of this under the covers. I'm guessing boost does also. I think the benefit is great. The code you have above, in fact, would be familiar to any Python user, so it would be a small learning curve to write a simple inline snippet where they needed it using boost within weave. > A principal idea of > Boost.Python is that you should be able to easily expose a standalone > library of C++ types and functions to Python without writing lots of "glue" > (e.g. argument checking and translation) and without intrusive changes to > the library. Weave's orientation seems to be towards writing C++ bodies for > Python functions. It should be very quick for those who don't want to leave > their Python editor just to optimize a function. However, it seems like that > approach wouldn't be very compatible with the idea of a standalone C++ > library unless you just plan to write little wads of glue code in weave and > dispatch to your library. Right. This would be doable -- though with tools around that already handle this problem (to various levels of satisfaction), I haven't seen a compelling reason yet to head off in that direction (although this hasn't always stopped me before...). > It also seems as though it might not scale well, > since (and I'm guessing here) you have to cross a Python function boundary > every time you want to break off a new piece of functionality. This is true. There are ways around it to some extent. You can define pure C/C++ functions/classes that a weave.inline function, or group of ext_tools functions all use. These guys can call each other at the speed of C/C++ without the Python overhead. > I'm not > saying one or the other paradigm is best - they seem to serve different > needs. yes. > Whether weave can benefit somehow from Boost.Python isn't clear to > me, but I hope what I've said helps to give you an idea. I'll watch with interest. Judging from your little example, I think it might. Also, as weave ventures out into trying to build extension classes in the future using 2.2's new additions, a nice framework that already handles the hard stuff in C++ would definitely help. see ya, eric From david.abrahams at rcn.com Mon Jan 21 14:32:38 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 21 Jan 2002 08:32:38 -0500 Subject: [C++-sig] weave plus boost? References: <017401c1a15a$1d717d30$777ba8c0@ericlaptop> <07ac01c1a24c$11f68f70$0500a8c0@boostconsulting.com> <009901c1a24e$7f6133b0$777ba8c0@ericlaptop> Message-ID: <07e301c1a280$5c913ba0$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "eric" > That looks very nice -- especially the d[msg] = 5 like syntax. Can you > initialize a tuple, dict, list, etc. from a PyObject*? Now that I look, I think as a matter of fact that you can. I might remove that ability, though: the library tries not to encourage direct manipulation of PyObject*, since it's so easy to get the refcounting wrong ("Do I expect the function to steal a reference?"). Instead, we supply a smart pointer type called "ref" which you construct from the PyObject* and use that as an intermediary: dict d(ref(p)) The semantics of ref() construction is reference-stealing unless you explicitly say otherwise: dict d(ref(p, ref::increment_count)); Unfortunately, both ways are needed because Python is so inconsistent about when and where it handles reference-counting for you (e.g. PyObject_SetItem makes its own reference, but PyTyple_SetItem steals one). This approach encourages one to consider reference-counting at the boundary between raw 'C' PyObject* and the much safer world of C++. > If so, they are > probably close to interchangeable with the current CXX implementation I use, > and it would be easy to create a boost backend to weave. How far along are > these basic objects > in development? Can they handle assignments (like the d[msg]=5 statement) > from int,float,strings, etc. now? Yep. > Also, how is the speed compared to using > Python API calls? I never measured it but have no reason to think it wouldn't be nearly equivalent. > I found that using the nice features of CXX (indexing, > assignment, etc) in innerloops was hard on performance. What compiler/version are you using? What are the optimization settings? Python imposes a limitation that's hard to circumvent: it requires the use of proxy objects for C++ wrappers of Python containers. These can be hard on an optimizer. > > Boost.Python has a different paradigm than weave does. > > Right. My understanding is that boost is CXXish (my main point of > reference) in flavor -- i.e. made for writing stand alone C++ libs that are > usable from Python. Close. Boost.Python isn't really for writing C++ libraries, but for wrapping them. You should in theory be able to grab a library for which you have no source code and produce a Python extension module in just a few lines of code. -Dave From david.abrahams at rcn.com Mon Jan 21 15:53:03 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 21 Jan 2002 09:53:03 -0500 Subject: [C++-sig] weave plus boost? References: <017401c1a15a$1d717d30$777ba8c0@ericlaptop> <0201201453422D.22770@avalanche.llnl.gov> Message-ID: <08c301c1a28b$b9059830$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Martin Casado" > Hi Eric, > > Here is an idea I toyed around with last summer. One of the > challenges with making extension types that represent C++ classes is > trying to come up with adequate ways to support templated classes. Pretty > much all packages I've looked at to automate the wrapping process require you > to create a seperate class for each template instantiation... which makes > sense > since template instantion is done at compile time. However, if I have a > templated class, vector, I would really like to construct an > instance > of vector in python, like. > > myVector = vector("std::string") > > It is impossible to know which instances of vector are going to be needed > before runtime with this method, so one option is to have vector(..) generate > an > instance of vector on the fly, compile it and load it in. Each > instance extension > would proabaly be cached, keyed by the type so subsequent requests for > vector("std::string") wouldn't have to redo the process. > > I'm not sure how practical this would be, but at one time I had a sloppy but > working prototype and it was.. if nothing else.. neat. Sounds cool, but if you're working from the Python side anyway, wouldn't you want std::vector so you didn't have to pay to convert std::string to/from Python strings at ever access? -Dave From casado2 at llnl.gov Mon Jan 21 20:40:29 2002 From: casado2 at llnl.gov (Martin Casado) Date: Mon, 21 Jan 2002 11:40:29 -0800 Subject: [C++-sig] weave plus boost? In-Reply-To: <08c301c1a28b$b9059830$0500a8c0@boostconsulting.com> References: <017401c1a15a$1d717d30$777ba8c0@ericlaptop> <0201201453422D.22770@avalanche.llnl.gov> <08c301c1a28b$b9059830$0500a8c0@boostconsulting.com> Message-ID: <0201211140292E.22770@avalanche.llnl.gov> > > > working prototype and it was.. if nothing else.. neat. > > Sounds cool, but if you're working from the Python side anyway, wouldn't > you want std::vector so you didn't have to pay > to convert std::string to/from Python strings at ever access? > > -Dave > The point, originally, was not to create a templated class from the python side but to be able to easily wrap existing templated c++ classes without having to create seperate "thin" classes for each instantiation. If I wanted to wrap a pre-existing class template class Foo{..}; I though it would be interesting to be able to create instances of foo at runtime by passing in the typename of the instance f1 = Foo("Goo") f2 = Foo("std::string") f3 = Foo("GeneralizedPythonObject") etc... The classes I have in mind to be wrapped were not created to be templated over a generalized python object. Of course to do this you would have to compile the instantiation on the fly, and I think it would be highly error prone (read dangerous to use in a production environment), but with caching out the instances the performance may be manageable and it could make for a neat project. The process for doing this could be pretty complicated, for example on AIX you need full closure to create a shared library, therefore to compile foo_type_inst.so you would need to link it with the export files of all libraries whose symbols it uses (blech!). The function of the wrapper code is pretty straightforward, instead of getting a type Foo() you get a factory creation function named Foo() that pops out instances of class Foo (much like most pre-python 2.2 wrappers). The instantiations, once created, are cached (keyed by type) in the creation function. Really, you only need to instantiate a type once, and it will persist as a shared object on disk, so that subsequent runs can check to see if an instance exists, by first checking cache, then disk... and finally creating the instance. ~~m From david.abrahams at rcn.com Mon Jan 21 21:01:59 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 21 Jan 2002 15:01:59 -0500 Subject: [C++-sig] weave plus boost? References: <017401c1a15a$1d717d30$777ba8c0@ericlaptop> <0201201453422D.22770@avalanche.llnl.gov> <08c301c1a28b$b9059830$0500a8c0@boostconsulting.com> <0201211140292E.22770@avalanche.llnl.gov> Message-ID: <001c01c1a2b7$16d88cd0$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Martin Casado" To: Sent: Monday, January 21, 2002 2:40 PM Subject: Re: [C++-sig] weave plus boost? > > > > > working prototype and it was.. if nothing else.. neat. > > > > Sounds cool, but if you're working from the Python side anyway, wouldn't > > you want std::vector so you didn't have to pay > > to convert std::string to/from Python strings at ever access? > > > > -Dave > > > > The point, originally, was not to create a templated class from the python > side but to be able to easily wrap existing templated c++ classes without > having to > create seperate "thin" classes for each instantiation. Oh. You don't need the "thin" classes with Boost.Python. I guess that's what confused me. > If I wanted to wrap a pre-existing class > > template > class Foo{..}; > > > I though it would be interesting to be able to create instances of foo > at runtime by passing in the typename of the instance > > f1 = Foo("Goo") > f2 = Foo("std::string") > f3 = Foo("GeneralizedPythonObject") > etc... > > The classes I have in mind to be wrapped were not created to be templated > over a generalized python object. > > Of course to do this you would have to compile the instantiation on the fly, > and I think it would be highly error prone (read dangerous to use in a > production environment), but with caching out the instances the performance > may be manageable and it could make for a neat project. > > The process for doing this could be pretty complicated, for example on > AIX you need full closure to create a shared library, therefore to > compile foo_type_inst.so you would need to link it with the export files > of all libraries whose symbols it uses (blech!). That sounds just like Windows. Not so terrible; Boost.Build handles it. > The function of the wrapper code is pretty straightforward, > instead of getting a type Foo() you get a factory creation function named > Foo() > that pops out instances of class Foo (much like most pre-python 2.2 wrappers). > The instantiations, once created, are cached (keyed by type) in the creation > function. > > Really, you only need to instantiate a type once, and it will persist as a > shared object on disk, so that subsequent runs can check to see if an > instance exists, by first checking cache, then disk... and finally creating > the instance. Okay, I get the picture now. Thanks for explaining, Dave From s13361562 at bach.sun.ac.za Mon Jan 21 22:24:43 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Mon, 21 Jan 2002 23:24:43 +0200 Subject: [C++-sig] new to python wrapping... Message-ID: <20020121212443.GC24137@vervet> Hello, I've lately started playing with Python wrapping. Tried both SWIG and Boost.Python, I'm thinking so far that Boost.Python's C++ and Python specific nature will make it more ideally suited to what I want to do (I don't care about bidnings other scripting languages). I've got two questions so far. One is what happens with garbage collection: if I create an instance of an object from a Python script, then add this to e.g. a list in some other C++ class, Python won't know it is still being used, and will try to garbage collect it? The other is a problem I'm having with a constructor. It worked previously, but after the latest changes in the source, I'm getting the following (it's a pretty ugly looking constructor ;) : C++ header: class TERRAIN_API Terrain { public: Terrain(const char* szElevationsFilename,const char* szTextureFilename,const char* szDetailTextureFilename,float vertexSpacing,float elevationScale,int maxNumTriangles,bool bUseBorders = false,float offsetX = 0.0f,float offsetY = 0.0f,int numTexturesX = 0,int numTexturesY = 0); Boost.Python using wrapper: python::class_builder Terrain_class(this_module, "Terrain"); Terrain_class.def(python::constructor()); The error I get when compiling it: demeterwrap.cpp: In function `void initdemeterwrap(...)': demeterwrap.cpp:87: wrong number of template arguments (11, should be 10) /usr/include/boost/python/detail/signatures.hpp:134: provided for `template boost::python::constructor' demeterwrap.cpp:87: confused by earlier errors, bailing out I'm using a really old Boost, as it seems to be the latest one that is packaged for Debian, and I'm lazy atm. I'll compile Boost from source if necessary, but it seems from the Changelogs that Boost.Python hasn't had much updates since Y2K? Maybe Boost libs used by Boost.Python have updated, and that makes a difference? The Debian package version is 1.21.1-1, i.e. some incarnation of 1.21, and I think Boost is currently at 1.26 or something? I hope this isn't something silly I've done that causes this, but I definately count 11 parameters in the C++ constructor, as well as my attempt at a python wrapping of it... Any ideas? Thanks, Hugo van der Merwe From s13361562 at bach.sun.ac.za Mon Jan 21 22:32:08 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Mon, 21 Jan 2002 23:32:08 +0200 Subject: [C++-sig] template constructor problem In-Reply-To: <20020120153721.44117.qmail@web20209.mail.yahoo.com> References: <033f01c1a160$4a467b70$0500a8c0@boostconsulting.com> <20020120153721.44117.qmail@web20209.mail.yahoo.com> Message-ID: <20020121213208.GD24137@vervet> On Sun, Jan 20, 2002 at 07:37:21AM -0800, Ralf W. Grosse-Kunstleve wrote: > --- David Abrahams wrote: > > Okay, try the current CVS state CVS version? Is this the new rewrite that I heard about? I think I should probably take a look at that? Where do I find more info? Thanks, Hugo van der Merwe From david.abrahams at rcn.com Mon Jan 21 22:35:56 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 21 Jan 2002 16:35:56 -0500 Subject: [C++-sig] new to python wrapping... References: <20020121212443.GC24137@vervet> Message-ID: <011a01c1a2c3$be4722e0$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Hugo van der Merwe" To: Sent: Monday, January 21, 2002 4:24 PM Subject: [C++-sig] new to python wrapping... > Hello, > > I've lately started playing with Python wrapping. Tried both SWIG and > Boost.Python, I'm thinking so far that Boost.Python's C++ and Python > specific nature will make it more ideally suited to what I want to do (I > don't care about bidnings other scripting languages). > > I've got two questions so far. One is what happens with garbage > collection: if I create an instance of an object from a Python script, > then add this to e.g. a list in some other C++ class, Python won't know > it is still being used, and will try to garbage collect it? No; it took me a while to understand this myself, but Python uses a different sort of GC which is more conservative. Instead of collecting everything it can't prove is referenced, it looks for reference cycles that it /can/ find, and which it can prove are not referenced from outside the cycle (it can do that by looking at the reference counts). > The other is a problem I'm having with a constructor. It worked > previously, but after the latest changes in the source, I'm getting the > following (it's a pretty ugly looking constructor ;) : > > C++ header: > > > I hope this isn't something silly I've done that causes this, but I > definately count 11 parameters in the C++ constructor, as well as my > attempt at a python wrapping of it... You just exceeded Boost.Python's builtin limits, but you can extend them by running python libs/python/src/gen_all.py NN where NN is the maximum number of arguments you need to support. That will generate some C++ header files which you use in place of the corresponding ones in your Boost installation. -Dave From david.abrahams at rcn.com Mon Jan 21 22:44:58 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 21 Jan 2002 16:44:58 -0500 Subject: [C++-sig] template constructor problem References: <033f01c1a160$4a467b70$0500a8c0@boostconsulting.com> <20020120153721.44117.qmail@web20209.mail.yahoo.com> <20020121213208.GD24137@vervet> Message-ID: <014801c1a2c5$263c2f20$0500a8c0@boostconsulting.com> The current SourceForge CVS main trunk contains both the new system (very incomplete) and the existing one. The Jamfile in libs/Python is provisional, builds and tests the new system. The established (release) code is still in the usual places. ----- Original Message ----- From: "Hugo van der Merwe" To: Sent: Monday, January 21, 2002 4:32 PM Subject: Re: [C++-sig] template constructor problem > On Sun, Jan 20, 2002 at 07:37:21AM -0800, Ralf W. Grosse-Kunstleve wrote: > > --- David Abrahams wrote: > > > Okay, try the current CVS state > > CVS version? Is this the new rewrite that I heard about? I think I > should probably take a look at that? Where do I find more info? > > Thanks, > Hugo van der Merwe > > _______________________________________________ > C++-sig mailing list > C++-sig at python.org > http://mail.python.org/mailman/listinfo/c++-sig From arnaldur at ru.is Mon Jan 21 23:55:23 2002 From: arnaldur at ru.is (Arnaldur Gylfason) Date: Mon, 21 Jan 2002 22:55:23 -0000 Subject: [C++-sig] generic object wrappers Message-ID: <07D05A69A3D0C14FAEA60C3ACE8E556401BD6FA9@nike.hir.is> Dave, Just letting you know I'm still alive. I started teaching 2 weeks ago (on top of everything else (stupid me!)) and am a little stressed out these days. Still I?ll find time for implementing the generic object wrappers, hopefully soon. Have faith. I've looked at mpl and am comfortable using it. I feel we should use it while in development and not worry about rewriting stuff unless mpl will not be officially in boost when the boost.python rewrite is ready. I've been thinking about the interface for users. Let's say someone need addable, callable and indexable, the interface would be object If we would use 1 typelist as a template parameter to object<>, the interface would be: typedef mpl::typelist my_capabilities; object Somewhat more cumbersome. I feel the former (as we discussed) is better. That means we offer a variable number of template parameters. I've been looking at the boost.preprocessor library and the preprocessor stuff in mpl, and I now understand how mpl uses macros to generate struct declarations that allow a variable number of parameters. I think we should opt for that approach. I have to say though, that understanding this macro logic is much harder than understanding template programming. Still it is a tool worth using to avoid repetition. We have discussed sets of capabilities like integral, etc. I was wondering if we should pick a few concepts from boost.operators as sets of capabilities. Enough said. I aim to begin by writing a core set of capabilities and then writing the object<> wrapper. As you mentioned, it is probably better to write the capabilities that form the generic object interface and provide it before we turn to the mete-programming approach. Let?s see. If I get enthusiastic I?ll probably start experimenting with the meta-programming stuff. Cheers Arnaldur From david.abrahams at rcn.com Tue Jan 22 00:25:02 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 21 Jan 2002 18:25:02 -0500 Subject: [C++-sig] generic object wrappers References: <07D05A69A3D0C14FAEA60C3ACE8E556401BD6FA9@nike.hir.is> Message-ID: <025801c1a2d2$ebb2a5b0$0500a8c0@boostconsulting.com> > Dave, > > Just letting you know I'm still alive. > I started teaching 2 weeks ago (on top of everything else (stupid me!)) > and am a little stressed out these days. > Still I?ll find time for implementing the generic object wrappers, > hopefully soon. Have faith. OK, I do! > I've looked at mpl and am comfortable using it. I feel we should use it > while in development and not worry about rewriting stuff unless mpl will > not be officially in boost when the boost.python rewrite is ready. Agreed. It might be choking one of my most important targets, but I'm not sure yet. I hope not. > I've been thinking about the interface for users. > Let's say someone need addable, callable and indexable, > the interface would be > object > > If we would use 1 typelist as a template parameter to object<>, the > interface would be: > > typedef mpl::typelist my_capabilities; > object > > Somewhat more cumbersome. Yes, though you don't need a separate declaration. Some of the latest stuff I've checked in puts "aliases" for mpl::typelist in the unnamed namespace for convenience, so we could make it possible to write: python::object >. Still, more cumbersome. > I feel the former (as we discussed) is better. Me too. > That means we offer a variable number of template parameters. You can't really do that, as I'm sure you know. You can only emulate it. > I've been looking at the boost.preprocessor library and the > preprocessor stuff in mpl, and I now understand how mpl uses macros > to generate struct declarations that allow a variable number of > parameters. I think we should opt for that approach. > > I have to say though, that understanding this macro logic is much > harder than understanding template programming. Still it is a tool > worth using to avoid repetition. Take the easy way out and re-use the MPL primitives like I did. from boost/python/class.hpp in CVS: // A type list for specifying bases template < BOOST_MPL_LIST_DEFAULT_PARAMETERS(typename B, ::boost::mpl::null_argument) > struct bases : ::boost::mpl::type_list< BOOST_MPL_LIST_PARAMETERS(B) >::type {}; // A type list for specifying arguments template < BOOST_MPL_LIST_DEFAULT_PARAMETERS(typename A, ::boost::mpl::null_argument) > struct args : ::boost::mpl::type_list< BOOST_MPL_LIST_PARAMETERS(A) >::type {}; It saved me from having to learn all about the preprocessor library right now ;-) > We have discussed sets of capabilities like integral, etc. I was > wondering if we should pick a few concepts from boost.operators as > sets of capabilities. Well, I'm beginning to wonder if this is all too complicated to be worthwhile. If not, that would be my fault :(. I guess the implicit conversions of, e.g. dict->mapping->object could be useful. As I was checking my latest work I ran into lots of reference-counting problems, which just reminded me of the importance of religiously using the stuff in reference.hpp. I began to think about a redesign which would also help hide the fact that, e.g. PyTypeObject isn't really derived from PyObject. The design looks like this: 1. A traits class is defined as follows: template struct base_type_traits { typedef PyObject type; }; This could be specialized for types which are derived from other Python types. For example, it is now possible to make a new extension type derived from dict, list, mapping, or int: template <> struct base_type_traits { typedef PyListObject type; }; 2. A small metaprogram constructs a proxy object which is implicitly convertible to T* for T in the chain of base types: template struct reference_proxy : reference_proxy::type> { typedef typename base_type_traits::type base_t; reference_proxy(T* p) : reference_proxy((base_t*)p) {} operator T*() const { return (T*)m_p; } } template <> struct reference_proxy { public: reference_proxy(PyObject* p) : m_p(p) {} operator PyObject*() const { return (PyObject*)m_p; } PyObject* m_p; } 3. This proxy is the result of reference::get(). The default argument to reference<> is PyObject so we can write python::reference<> most of the time. 4. We can use the traits to manage the implicit convertibility between different reference<> specializations. Well, anyway, it's a thought. Less ambitious than what you're doing and immediately useful. On the other hand, I can do it if you'd prefer ;-) > Enough said. I aim to begin by writing a core set of capabilities > and then writing the object<> wrapper. > As you mentioned, it is probably better to write the capabilities > that form the generic object interface and provide it before we turn > to the mete-programming approach. Let?s see. If I get enthusiastic > I?ll probably start experimenting with the meta-programming stuff. OK. -Dave From arnaldur at decode.is Tue Jan 22 11:08:25 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Tue, 22 Jan 2002 10:08:25 +0000 Subject: [C++-sig] generic object wrappers Message-ID: <<< > I've looked at mpl and am comfortable using it. I feel we should use it > while in development and not worry about rewriting stuff unless mpl will > not be officially in boost when the boost.python rewrite is ready. Agreed. It might be choking one of my most important targets, but I'm not sure yet. I hope not. >>> It's an experiment (an interesting one). It either works or not. If it works the interface to users will be relatively straight forward. The complexity will be hidden from them in the implementation. Even if we have to rewrite things, it won't take too long. (Maybe you were referring to something else?) <<< > We have discussed sets of capabilities like integral, etc. I was > wondering if we should pick a few concepts from boost.operators as > sets of capabilities. Well, I'm beginning to wonder if this is all too complicated to be worthwhile. If not, that would be my fault :(. >>> I don?t think so. Again, complexity should be hidden from the user. Also my comment above shouldn't make things complicated. It?s for convenience. In operators you define addable, subtractable etc. same way we aim for the capabilities. So maybe it makes sense to define sets, like additive, arithmetic, ring etc. just as in operators. <<< Take the easy way out and re-use the MPL primitives like I did. from boost/python/class.hpp in CVS: // A type list for specifying bases template < BOOST_MPL_LIST_DEFAULT_PARAMETERS(typename B, ::boost::mpl::null_argument) > struct bases : ::boost::mpl::type_list< BOOST_MPL_LIST_PARAMETERS(B) >::type {}; // A type list for specifying arguments template < BOOST_MPL_LIST_DEFAULT_PARAMETERS(typename A, ::boost::mpl::null_argument) > struct args : ::boost::mpl::type_list< BOOST_MPL_LIST_PARAMETERS(A) >::type {}; It saved me from having to learn all about the preprocessor library right now ;-) >>> Sounds good! I'll do that. <<< As I was checking my latest work I ran into lots of reference-counting problems, which just reminded me of the importance of religiously using the stuff in reference.hpp. I began to think about a redesign which would also help hide the fact that, e.g. PyTypeObject isn't really derived from PyObject. The design looks like this: 1. A traits class is defined as follows: template struct base_type_traits { typedef PyObject type; }; This could be specialized for types which are derived from other Python types. For example, it is now possible to make a new extension type derived from dict, list, mapping, or int: template <> struct base_type_traits { typedef PyListObject type; }; 2. A small metaprogram constructs a proxy object which is implicitly convertible to T* for T in the chain of base types: template struct reference_proxy : reference_proxy::type> { typedef typename base_type_traits::type base_t; reference_proxy(T* p) : reference_proxy((base_t*)p) {} operator T*() const { return (T*)m_p; } } template <> struct reference_proxy { public: reference_proxy(PyObject* p) : m_p(p) {} operator PyObject*() const { return (PyObject*)m_p; } PyObject* m_p; } 3. This proxy is the result of reference::get(). The default argument to reference<> is PyObject so we can write python::reference<> most of the time. 4. We can use the traits to manage the implicit convertibility between different reference<> specializations. Well, anyway, it's a thought. Less ambitious than what you're doing and immediately useful. On the other hand, I can do it if you'd prefer ;-) >>> This looks neat! Just to be sure, reference::get() would return a reference_proxy which inherits from reference_proxy which inherits from reference_proxy. It would thus provide automatic conversions to PyObject*, PyListObject* and MyListSubtype*, just as if they were related through C++ inheritance. I have been thinking about it's use though. In the generic wrappers I only work with PyObject* and don't need anything else. In concrete objects like list, we know the type and can safely typecast to PyListObject* if/when needed (I did that in the iterator support). The Python C API always work with PyObject*, even in functions working with concrete types like list. The PyObject*is converted to PyListObject* within the function. If we would have API functions for PyObject*, PyListObject* (and MyListSubtype*) etc. I could see direct benefit from the design above. reference::get() would pass in all cases. If you want to provide interfaces for different specializations of reference and provide implicit convertibility on that level, the design would help. function f(reference) could then be called with a reference and reference. I did not foresee many interfaces that work on the reference level though. E.g. rather than f(reference) I would have f(list). a MyListSubtype would be used as a generic object (object) but the automatic conversion to list would call PyList_Check and pass it. Thus the function f can be called on a MyListSubtype. I am probably out in the wild. I'm just trying to imagine usage of the above design. Please tell me what usage you had in mind. Cheers Arnaldur From s13361562 at bach.sun.ac.za Tue Jan 22 11:26:42 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Tue, 22 Jan 2002 12:26:42 +0200 Subject: [C++-sig] new to python wrapping... In-Reply-To: <011a01c1a2c3$be4722e0$0500a8c0@boostconsulting.com> References: <20020121212443.GC24137@vervet> <011a01c1a2c3$be4722e0$0500a8c0@boostconsulting.com> Message-ID: <20020122102642.GG24137@vervet> > No; it took me a while to understand this myself, but Python uses a > different sort of GC which is more conservative. Instead of collecting > everything it can't prove is referenced, it looks for reference cycles that > it /can/ find, and which it can prove are not referenced from outside the > cycle (it can do that by looking at the reference counts). Not sure I followed that 100%. Here is how I understand it at the moment, quite possibly incorrectly, correct me where I go wrong: It uses refcounts, right, so if you do: a = NewClass() the instance of NewClass's refcount just went up by one. Then I do someCPPClass.Add(a) This puts the instance mentioned above in some list. If I now reassign the python "a" to something else, e.g.: a = 0 The refcount for the instance of NewClass will go down by one again, won't it? In which case it will be disposed? Unless the refcount is increased by the someCPPClass.Add(a) call, something that Boost.Python will have to take care of, if I'm correct so far. Where am I going wrong? Or does Boost.Python take care of this? Or will there be a GC problem? I have not yet tested this. Maybe I should some time soon. > You just exceeded Boost.Python's builtin limits, but you can extend them by > running > > python libs/python/src/gen_all.py NN > > where NN is the maximum number of arguments you need to support. That will > generate some C++ header files which you use in place of the corresponding > ones in your Boost installation. Ah, thanks a lot. I suspected as much. I think in my case NN will be something like 17. ;) (I've seen at least one constructor with 17 parameters.) Thanks, Hugo van der Merwe From david.abrahams at rcn.com Tue Jan 22 13:37:00 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Tue, 22 Jan 2002 07:37:00 -0500 Subject: [C++-sig] generic object wrappers References: Message-ID: <04de01c1a341$bd7f41c0$0500a8c0@boostconsulting.com> > <<< > > > I've looked at mpl and am comfortable using it. I feel we should > > use it while in development and not worry about rewriting stuff > > unless mpl will not be officially in boost when the boost.python > > rewrite is ready. > > Agreed. It might be choking one of my most important targets, but > I'm not sure yet. I hope not. > > >>> > > It's an experiment (an interesting one). It either works or not. It works! believe it or not, EDG was choking on the PREPROCESSOR library construct, so Aleksey just checked in an alternative. > If it works the interface to users will be relatively straight > forward. The complexity will be hidden from them in the > implementation. Even if we have to rewrite things, it won't take too > long. (Maybe you were referring to something else?) No, that's OK: I was just "in a mood", I guess. > <<< > > > We have discussed sets of capabilities like integral, etc. I was > > wondering if we should pick a few concepts from boost.operators > > as sets of capabilities. > > Well, I'm beginning to wonder if this is all too complicated to be > worthwhile. If not, that would be my fault :(. > >>> > > I don?t think so. Again, complexity should be hidden from the user. > Also my comment above shouldn't make things complicated. It?s for > convenience. In operators you define addable, subtractable etc. same > way we aim for the capabilities. So maybe it makes sense to define > sets, like additive, arithmetic, ring etc. just as in operators. Maybe. I find all of those combinations in operators.hpp a bit daunting, but I'm sure that some people find them indispensable. > This looks neat! Just to be sure, reference::get() > would return a reference_proxy which inherits from > reference_proxy which inherits from > reference_proxy. It would thus provide automatic > conversions to PyObject*, PyListObject* and MyListSubtype*, just as if > they were related through C++ inheritance. Precisely! > I have been thinking about its use though. In the generic wrappers > I only work with PyObject* and don't need anything else. I would find it useful; there's quite a bit of casting involved in some of the newest code. I always intended that Boost.Python would be good for users who want to create a new extensioni type directly (rather than wrapping some C++ type). In other words, Boost.Python should be a framework, with the high-level parts easiest to use but with lower level facilities which still help you write effective Python components in C++. Your work falls in the middle area. > In concrete objects like list, we know the type and can safely > typecast to PyListObject* if/when needed (I did that in the iterator > support). Typecasts are dangerous and ugly, though. > The Python C API always work with PyObject*, even in > functions working with concrete types like list. The PyObject*is > converted to PyListObject* within the function. If we would have > API functions for PyObject*, PyListObject* (and MyListSubtype*) > etc. I could see direct benefit from the design > above. reference::get() would pass in all cases. Don't forget, once you downcast to PyListObject*, you can build a reference, use that, and all casting disappears. > If you want to provide interfaces for different specializations of > reference and provide implicit convertibility on that level, the > design would help. function f(reference) could then be > called with a reference and reference. I > did not foresee many interfaces that work on the reference level > though. E.g. rather than f(reference) I would have > f(list). a MyListSubtype would be used as a generic object > (object) but the automatic conversion to list would > call PyList_Check and pass it. Thus the function f can be called on a > MyListSubtype. > > I am probably out in the wild. I'm just trying to imagine usage of the > above design. Please tell me what usage you had in mind. Your reasoning is sound, but you may have overlooked cases like boost/python/object/class_wrapper.hpp and libs/python/src/object/class.cpp. Maybe those cases aren't common enough to warrant such a feature. -Dave From david.abrahams at rcn.com Tue Jan 22 13:39:41 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Tue, 22 Jan 2002 07:39:41 -0500 Subject: [C++-sig] new to python wrapping... References: <20020121212443.GC24137@vervet> <011a01c1a2c3$be4722e0$0500a8c0@boostconsulting.com> <20020122102642.GG24137@vervet> Message-ID: <04f801c1a342$713803f0$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Hugo van der Merwe" > > No; it took me a while to understand this myself, but Python uses a > > different sort of GC which is more conservative. Instead of collecting > > everything it can't prove is referenced, it looks for reference cycles that > > it /can/ find, and which it can prove are not referenced from outside the > > cycle (it can do that by looking at the reference counts). > > Not sure I followed that 100%. Here is how I understand it at the > moment, quite possibly incorrectly, correct me where I go wrong: > > It uses refcounts, right, so if you do: > > a = NewClass() > > the instance of NewClass's refcount just went up by one. Then I do > > someCPPClass.Add(a) > > This puts the instance mentioned above in some list. If I now reassign > the python "a" to something else, e.g.: > > a = 0 > > The refcount for the instance of NewClass will go down by one again, > won't it? In which case it will be disposed? Unless the refcount is > increased by the someCPPClass.Add(a) call, something that Boost.Python > will have to take care of, if I'm correct so far. > > Where am I going wrong? Or does Boost.Python take care of this? Or will > there be a GC problem? I have not yet tested this. Maybe I should some > time soon. If you don't traffic in raw PyObject*s, but instead, you use boost::python::references, Boost.Python will take care of it. From arnaldur at decode.is Tue Jan 22 19:20:57 2002 From: arnaldur at decode.is (Arnaldur Gylfason) Date: Tue, 22 Jan 2002 18:20:57 +0000 Subject: [C++-sig] generic object wrappers Message-ID: <<< > convenience. In operators you define addable, subtractable etc. same > way we aim for the capabilities. So maybe it makes sense to define > sets, like additive, arithmetic, ring etc. just as in operators. Maybe. I find all of those combinations in operators.hpp a bit daunting, but I'm sure that some people find them indispensable. >>> I believe many of the abstractions in operators are clear to those used to mathematical language, esp. abstract algebra. For those object would be clear and might be considered superior to object. Those who find it unclear could happily ignore these combinations. This is not important though. <<< I would find it useful; there's quite a bit of casting involved in some of the newest code. I always intended that Boost.Python would be good for users who want to create a new extensioni type directly (rather than wrapping some C++ type). In other words, Boost.Python should be a framework, with the high-level parts easiest to use but with lower level facilities which still help you write effective Python components in C++. Your work falls in the middle area. > The Python C API always work with PyObject*, even in > functions working with concrete types like list. The PyObject*is > converted to PyListObject* within the function. If we would have > API functions for PyObject*, PyListObject* (and MyListSubtype*) > etc. I could see direct benefit from the design > above. reference::get() would pass in all cases. Don't forget, once you downcast to PyListObject*, you can build a reference, use that, and all casting disappears. ... Your reasoning is sound, but you may have overlooked cases like boost/python/object/class_wrapper.hpp and libs/python/src/object/class.cpp. Maybe those cases aren't common enough to warrant such a feature. >>> Yeah I meant to add that maybe you needed it in other parts of boost.python. Sadly, I must admit I have not taken a look at other parts and can't comment on them at this stage. All in all I believe this design is good and 1 case where it will benefit you (or me) is enough to warrant it. Regarding typecasts, yes they're ugly. Inside a concrete object wrapper like list it should be safe though. Using a reinterpret_cast makes the typecasts obvious in code. After downcasting to a PyListObject* I could build a reference and benefit from automatic conversions after that yes, but the usage I had in mind was just touch and go, i.e. there wasn't really any need of keeping a PyListObject*, I just needed the typecast to access an attribute for the PyListObject instance (obitem). I think most uses of these typecasts/conversions in the object interface part are similar to that. Don't get me wrong though. It is quite possible I will find out that this design comes in handy later. Cheers Arnaldur From s13361562 at bach.sun.ac.za Tue Jan 22 20:27:43 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Tue, 22 Jan 2002 21:27:43 +0200 Subject: [C++-sig] thin wrapper example in docs... Message-ID: <20020122192743.GA6807@vervet> The following example appears in the Pointers section of the Boost.Python docs: const Foo* f(); // original function const Foo& f_wrapper() { return *f(); } ... my_module.def(f_wrapper, "f"); Am I correct in thinking: - Here f() returns a new pointer to an instance of Foo. - The Python f() call calls f_wrapper, which returns a reference to this intance of Foo, which is then copied to a new instance created and "tracked" by python, using Foo's copy constructor. - If f() creates a new instance of Foo, nothing disposes this instance, is there then not a memory leak? Technically, the pointer returned by f() may point to an instance that some other part of the code has rw access to, so while the function calling f() can not change the object due to it being const, something else may still change it. Thus behaviour of the python object may be not as expected, since it has been copied, and a change in the original will not be reflected in the "python copy". This brings forwards another thought: if the pointer points to an instance that is used elsewhere, it cannot be disposed after it's been copied? So when, how and where must the Foo* returned by f() be disposed? I do not yet have a firm grasp of all things C++ related. I may be mistaken in my understanding? Thanks, Hugo van der Merwe From david.abrahams at rcn.com Tue Jan 22 20:56:28 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Tue, 22 Jan 2002 14:56:28 -0500 Subject: [C++-sig] generic object wrappers References: Message-ID: <06cb01c1a37f$86e1bec0$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Arnaldur Gylfason" > > I believe many of the abstractions in operators are clear to those used to > mathematical > language, esp. abstract algebra. For those object would be clear and > might be considered superior to object. > Those who find it unclear could happily ignore these combinations. ...or learn them ;-) > This is > not important though. Agreed. > Yeah I meant to add that maybe you needed it in other parts of > boost.python. > Sadly, I must admit I have not taken a look at other parts and can't > comment on > them at this stage. > All in all I believe this design is good and 1 case where it will benefit > you (or me) > is enough to warrant it. > > Regarding typecasts, yes they're ugly. Inside a concrete object wrapper > like list it should > be safe though. Using a reinterpret_cast makes the typecasts obvious in > code. Technically speaking, you shouldn't use reinterpret_cast<> in portable code. It's behavior is implementation-defined (5.2.10). A C-style cast is actually portable and safer. > After downcasting to a PyListObject* I could build a > reference and benefit > from automatic conversions after that yes, but the usage I had in mind was > just touch and go, > i.e. there wasn't really any need of keeping a PyListObject*, I just needed > the typecast > to access an attribute for the PyListObject instance (obitem). > I think most uses of these typecasts/conversions in the object interface > part are similar to that. > > Don't get me wrong though. It is quite possible I will find out that this > design comes in handy > later. Okay, well, I might do it if you don't. Regards, Dave From s13361562 at bach.sun.ac.za Wed Jan 23 15:35:06 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Wed, 23 Jan 2002 16:35:06 +0200 Subject: [C++-sig] Boost.Python and lifetime of instances of C++ classes Message-ID: <20020123143506.GB21624@vervet> I've attached the source for a python extension (test.cpp), and a test.py script, that uses it. Here is the output of the test.py script: ========== k = test.testclass(): test Constructor called. k.printit(): HELLO! l = test.foo() l.add(k) l.printit(): HELLO! k = 0 test Destructor called. l.printit(): HELLO! ========== How does this work? Since "test Destructor called." appears, I'd think k has been destroyed/deallocated. Then l.printit() calls k.printit(), from C++. Something's odd, possibly even dangerous. Can someone explain this? Sorry if this does not compile under Windows - what's required, just the "boilerplate" thingy added at the end of test.cpp? Thanks, Hugo -------------- next part -------------- #!/usr/bin/env python1.5 import test print "k = test.testclass():" k = test.testclass() print "k.printit():" k.printit() print "l = test.foo()" l = test.foo() print "l.add(k)" l.add(k) print "l.printit():" l.printit() print "k = 0" k = 0 print "l.printit():" l.printit() -------------- next part -------------- A non-text attachment was scrubbed... Name: test.cpp Type: text/x-c++src Size: 840 bytes Desc: not available URL: From david.abrahams at rcn.com Wed Jan 23 21:29:02 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 23 Jan 2002 15:29:02 -0500 Subject: [C++-sig] Boost.Python and lifetime of instances of C++ classes References: <20020123143506.GB21624@vervet> Message-ID: <0ce501c1a44d$208d6920$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Hugo van der Merwe" To: Sent: Wednesday, January 23, 2002 9:35 AM Subject: [C++-sig] Boost.Python and lifetime of instances of C++ classes > I've attached the source for a python extension (test.cpp), and a > test.py script, that uses it. > > Here is the output of the test.py script: > > ========== > k = test.testclass(): > test Constructor called. > k.printit(): > HELLO! > l = test.foo() > l.add(k) > l.printit(): > HELLO! > k = 0 > test Destructor called. > l.printit(): > HELLO! > ========== > > How does this work? Luck. > Since "test Destructor called." appears, I'd think > k has been destroyed/deallocated. It has. > Then l.printit() calls k.printit(), > from C++. Something's odd, possibly even dangerous. Can someone explain > this? The memory at the other end of the pointer has been returned to the system, but it might not have been overwritten. Regardless of what happened, it doesn't matter because the pointer doesn't even have to be dereferenced in order to call that member function. You could probably take any random glob of bits, cast them to (test*), and call that member function on it. -Dave From s13361562 at bach.sun.ac.za Wed Jan 23 22:22:07 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Wed, 23 Jan 2002 23:22:07 +0200 Subject: [C++-sig] Boost.Python and lifetime of instances of C++ classes In-Reply-To: <0ce501c1a44d$208d6920$0500a8c0@boostconsulting.com> References: <20020123143506.GB21624@vervet> <0ce501c1a44d$208d6920$0500a8c0@boostconsulting.com> Message-ID: <20020123212207.GD25101@vervet> > > Since "test Destructor called." appears, I'd think > > k has been destroyed/deallocated. > > It has. > > > Then l.printit() calls k.printit(), > > from C++. Something's odd, possibly even dangerous. Can someone explain > > this? > > The memory at the other end of the pointer has been returned to the system, > but it might not have been overwritten. Regardless of what happened, it > doesn't matter because the pointer doesn't even have to be dereferenced in > order to call that member function. You could probably take any random glob > of bits, cast them to (test*), and call that member function on it. True, I didn't bother to have data members, etc. But anyway, that is irrelevant. The question is how can I go about "solving" this? (I mentioned this in a previous thread, but without the code it was probably not clear what I meant.) What I'm thinking of as a possible method to tackle this problem, is to write a wrapper over the foo class, that increases the python object's refcount when add() is called. And can decrease the python object's refcount at a later point, when appropriate. The question is whether this can be done easily with Boost. (I suppose I should go read the Python docs again, about Extending...) I'd inc the refcount in Add, and dec the refcount whenever, well, foo no longer references the instance. This includes e.g. in Foo's destructor. Is this the right solution? Is there another nice way it can be done? Thanks, Hugo From david.abrahams at rcn.com Wed Jan 23 23:04:23 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 23 Jan 2002 17:04:23 -0500 Subject: [C++-sig] Boost.Python and lifetime of instances of C++ classes References: <20020123143506.GB21624@vervet> <0ce501c1a44d$208d6920$0500a8c0@boostconsulting.com> <20020123212207.GD25101@vervet> Message-ID: <0d7801c1a459$ece41620$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Hugo van der Merwe" > True, I didn't bother to have data members, etc. But anyway, that is > irrelevant. The question is how can I go about "solving" this? (I > mentioned this in a previous thread, but without the code it was > probably not clear what I meant.) include include class test { public: test() { std::cout << "test Constructor called." << std::endl; } ~test() { std::cout << "test Destructor called." << std::endl; } void print() { std::cout << "HELLO!" << std::endl; } static boost::shared_ptr create() { return boost::shared_ptr(new test); } }; class foo { public: void add(boost::shared_ptr t) { sav = t; } void print() { sav->print(); } private: boost::shared_ptr sav; }; #include BOOST_PYTHON_MODULE_INIT(test) { boost::python::module_builder this_module("test"); boost::python::class_builder test_class(this_module, "test"); this_module.def(&test::create, "testclass"); test_class.def(&test::print, "printit"); boost::python::class_builder foo_class(this_module, "foo"); foo_class.def(boost::python::constructor<>()); foo_class.def(&foo::add, "add"); foo_class.def(&foo::print, "printit"); } ...or you can wait for Boost.Python V2, coming soon, which will handle this problem much better. From david.abrahams at rcn.com Thu Jan 24 02:43:13 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 23 Jan 2002 20:43:13 -0500 Subject: [C++-sig] thin wrapper example in docs... References: <20020122192743.GA6807@vervet> Message-ID: <0e8301c1a479$231373c0$0500a8c0@boostconsulting.com> Sorry I never replied to this... ----- Original Message ----- From: "Hugo van der Merwe" > The following example appears in the Pointers section of the > Boost.Python docs: > > const Foo* f(); // original function > const Foo& f_wrapper() { return *f(); } > ... > my_module.def(f_wrapper, "f"); > > Am I correct in thinking: > > - Here f() returns a new pointer to an instance of Foo. > - The Python f() call calls f_wrapper, which returns a reference to > this intance of Foo, which is then copied to a new instance created and > "tracked" by python, using Foo's copy constructor. > - If f() creates a new instance of Foo, nothing disposes this instance, > is there then not a memory leak? Yes. The appropriate wrapper for that case is one of these: std::auto_ptr f_wrapper() { return std::auto_ptr(f()); } boost::shared_ptr f_wrapper() { return boost::shared_ptr(f()); } > Technically, the pointer returned by f() may point to an instance that > some other part of the code has rw access to, so while the function > calling f() can not change the object due to it being const, something > else may still change it. Thus behaviour of the python object may be not > as expected, since it has been copied, and a change in the original will > not be reflected in the "python copy". Right. > This brings forwards another > thought: if the pointer points to an instance that is used elsewhere, it > cannot be disposed after it's been copied? So when, how and where must > the Foo* returned by f() be disposed? You must decide whether f() is passing off ownership of the Foo or not. This problem is explained by the introductory text at http://www.boost.org/libs/python/doc/pointers.html > I do not yet have a firm grasp of all things C++ related. I may be > mistaken in my understanding? No, you have grasped the conundrum. From s13361562 at bach.sun.ac.za Thu Jan 24 10:28:34 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Thu, 24 Jan 2002 11:28:34 +0200 Subject: [C++-sig] thin wrapper example in docs... In-Reply-To: <0e8301c1a479$231373c0$0500a8c0@boostconsulting.com> References: <20020122192743.GA6807@vervet> <0e8301c1a479$231373c0$0500a8c0@boostconsulting.com> Message-ID: <20020124092834.GA31760@vervet> > > The following example appears in the Pointers section of the > > Boost.Python docs: > > > > const Foo* f(); // original function > > const Foo& f_wrapper() { return *f(); } > > ... > > my_module.def(f_wrapper, "f"); > > > > Am I correct in thinking: > > > > - Here f() returns a new pointer to an instance of Foo. > > - The Python f() call calls f_wrapper, which returns a reference to > > this intance of Foo, which is then copied to a new instance created and > > "tracked" by python, using Foo's copy constructor. > > - If f() creates a new instance of Foo, nothing disposes this instance, > > is there then not a memory leak? > > Yes. The appropriate wrapper for that case is one of these: > > std::auto_ptr f_wrapper() { return std::auto_ptr(f()); } > boost::shared_ptr f_wrapper() { return boost::shared_ptr(f()); } So the docs require some updating, the example above (and the exception handling). Though I guess with the rewrite currently ongoing, it's a waste of time, as it will all have to be updated for the new Boost.Python anyway? Where can I read up more about how this boost::shared_ptr or std::auto_ptr works? (Do these two do the same thing? Does this work because of auto_ptr and shared_ptr's magic, or because Boost.Python does certain things when it finds an auto_ptr or shared_ptr?) > > This brings forwards another > > thought: if the pointer points to an instance that is used elsewhere, it > > cannot be disposed after it's been copied? So when, how and where must > > the Foo* returned by f() be disposed? > > You must decide whether f() is passing off ownership of the Foo or not. This > problem is explained by the introductory text at > http://www.boost.org/libs/python/doc/pointers.html Is this passing off or not passing off the difference between auto_ptr and shared_ptr? With one of the two, the wrapper function above should automatically dispose of the instance, since it's been copied and is no longer necessary, with the other it should not touch it, because it's still in use somewhere in the C++ code. Right? Thanks a lot, Hugo From s13361562 at bach.sun.ac.za Thu Jan 24 10:38:02 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Thu, 24 Jan 2002 11:38:02 +0200 Subject: [C++-sig] Boost.Python and lifetime of instances of C++ classes In-Reply-To: <0d7801c1a459$ece41620$0500a8c0@boostconsulting.com> References: <20020123143506.GB21624@vervet> <0ce501c1a44d$208d6920$0500a8c0@boostconsulting.com> <20020123212207.GD25101@vervet> <0d7801c1a459$ece41620$0500a8c0@boostconsulting.com> Message-ID: <20020124093802.GB31760@vervet> > include > include > > class test { > public: > test() { std::cout << "test Constructor called." << std::endl; } > ~test() { std::cout << "test Destructor called." << std::endl; } > void print() { std::cout << "HELLO!" << std::endl; } > static boost::shared_ptr create() { > return boost::shared_ptr(new test); > } > }; > > class foo { > public: > void add(boost::shared_ptr t) { sav = t; } > void print() { sav->print(); } > private: > boost::shared_ptr sav; > }; Thanks! So making it boost::shared_ptr causes Boost.Python to automatically call Py_INCREF and Py_DECREF? (I have gone and read about refcounts, and was wondering how I could write a thin wrapper that gives me access to the PyObject* as well, so that I can INCREF/DECREF where appropriate.) Can all of this really be done automatically? If there's some C++ list class, INCREF should be called in the Add() method, DECREF should be called in the Remove() method as well as the destructor, for each of the items in the list... can all of this really be done automatically? > ...or you can wait for Boost.Python V2, coming soon, which will handle this > problem much better. Soom, soon being couple of months, couple of weeks? Eagerly looking forward to it, but in the mean time, I like learning about the current version. (Knowledge gained here is not lost when current version is superseded by a new version. The knowledge applies in other areas as well. Also, I'm anyway releasing a wrapper with the current version, will update it to V2 when it's out.) Another question: is it possible for a second wrapper to use things from a first? (I want to make another wrapper, which must be usable at the same time as the first, but there are e.g. parameters in the second that are of types wrapped in the first.) I think this is probably not possible, and the two will have to be combined into one? Last question: where can I learn more about exceptions? (E.g. how to get a C++ exception passed on into Python.) Thanks, Hugo van der Merwe From s13361562 at bach.sun.ac.za Thu Jan 24 11:07:56 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Thu, 24 Jan 2002 12:07:56 +0200 Subject: [C++-sig] building boost ... Message-ID: <20020124100756.GJ31760@vervet> Can anyone point me to somewhere I can find out how to easily build boost against Python 2.1 instead of 1.5, and to make dynamic libraries for dynamic linking? Thanks, Hugo From david.abrahams at rcn.com Thu Jan 24 14:04:19 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Thu, 24 Jan 2002 08:04:19 -0500 Subject: [C++-sig] building boost ... References: <20020124100756.GJ31760@vervet> Message-ID: <0f6501c1a4d7$aeecc140$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Hugo van der Merwe" To: Sent: Thursday, January 24, 2002 5:07 AM Subject: [C++-sig] building boost ... > Can anyone point me to somewhere I can find out how to easily build > boost against Python 2.1 instead of 1.5, and to make dynamic libraries > for dynamic linking? 1. Get the latest release of boost from www.boost.org 2. Have Python 2.1 installed on your system in the usual place (/usr/local/...) 3. Make sure you have a working Boost.Jam executable (www.boost.org/tools/build) in your path 4. go to the libs/python/build subdirectory of your boost installation, and invoke: jam -sBUILD=release -sTOOLS= -sPYTHON_VERSION=2.1 Where toolset is the name of one of the supported toolsets (e.g. gcc). This will build static and dynamic versions of the Boost.Python library. -Dave From s13361562 at bach.sun.ac.za Thu Jan 24 15:12:46 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Thu, 24 Jan 2002 16:12:46 +0200 Subject: [C++-sig] generic object wrappers In-Reply-To: <06cb01c1a37f$86e1bec0$0500a8c0@boostconsulting.com> References: <06cb01c1a37f$86e1bec0$0500a8c0@boostconsulting.com> Message-ID: <20020124141246.GB32346@vervet> Hehe, dropping in from nowhere, I'm eager to learn: > Technically speaking, you shouldn't use reinterpret_cast<> in portable code. > It's behavior is implementation-defined (5.2.10). A C-style cast is actually > portable and safer. 5.2.10? 5.2.10 of what? (I want to go read that, whatever it is. ;) Thanks, Hugo From s13361562 at bach.sun.ac.za Thu Jan 24 16:29:28 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Thu, 24 Jan 2002 17:29:28 +0200 Subject: [C++-sig] building boost ... In-Reply-To: <0f6501c1a4d7$aeecc140$0500a8c0@boostconsulting.com> References: <20020124100756.GJ31760@vervet> <0f6501c1a4d7$aeecc140$0500a8c0@boostconsulting.com> Message-ID: <20020124152928.GB2376@vervet> > 4. go to the libs/python/build subdirectory of your boost installation, and > invoke: > > jam -sBUILD=release -sTOOLS= -sPYTHON_VERSION=2.1 -sBOOST_ROOT=../../.. > This will build static and dynamic versions of the Boost.Python library. Thanks. If I am in the Boost root dir, it works exactly the same, except it builds more, right? I was missing the -sPYTHON_VERSION and the -sBUILD=release, not sure what the latter does. But I still don't get dynamically linkable .so's. (No output from "find ../../.. | grep so$") This isn't too urgent though. The 2.1 was more important to me. > Where toolset is the name of one of the supported toolsets (e.g. gcc). What is the difference between gcc and gcc-stlport? Thanks, Hugo From david.abrahams at rcn.com Thu Jan 24 16:28:35 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Thu, 24 Jan 2002 10:28:35 -0500 Subject: [C++-sig] generic object wrappers References: <06cb01c1a37f$86e1bec0$0500a8c0@boostconsulting.com> <20020124141246.GB32346@vervet> Message-ID: <103e01c1a4ec$54f3de70$0500a8c0@boostconsulting.com> The C++ standard. Set aside a few weeks ;-) ----- Original Message ----- From: "Hugo van der Merwe" To: Sent: Thursday, January 24, 2002 9:12 AM Subject: Re: [C++-sig] generic object wrappers > Hehe, dropping in from nowhere, I'm eager to learn: > > > Technically speaking, you shouldn't use reinterpret_cast<> in portable code. > > It's behavior is implementation-defined (5.2.10). A C-style cast is actually > > portable and safer. > > 5.2.10? 5.2.10 of what? (I want to go read that, whatever it is. ;) > > Thanks, > Hugo > > _______________________________________________ > C++-sig mailing list > C++-sig at python.org > http://mail.python.org/mailman/listinfo/c++-sig From david.abrahams at rcn.com Thu Jan 24 16:41:23 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Thu, 24 Jan 2002 10:41:23 -0500 Subject: [C++-sig] building boost ... References: <20020124100756.GJ31760@vervet> <0f6501c1a4d7$aeecc140$0500a8c0@boostconsulting.com> <20020124152928.GB2376@vervet> Message-ID: <105401c1a4ed$bea28190$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Hugo van der Merwe" To: Sent: Thursday, January 24, 2002 10:29 AM Subject: Re: [C++-sig] building boost ... > > 4. go to the libs/python/build subdirectory of your boost installation, and > > invoke: > > > > jam -sBUILD=release -sTOOLS= -sPYTHON_VERSION=2.1 > > -sBOOST_ROOT=../../.. Right. Setting it to an absolute path in your environment will save you from having to type it and worry that you got it right each time. > > This will build static and dynamic versions of the Boost.Python library. > > Thanks. If I am in the Boost root dir, it works exactly the same, except > it builds more, right? yep. > I was missing the -sPYTHON_VERSION and the > -sBUILD=release, not sure what the latter does. chooses a release build as opposed to a debug build. > But I still don't get > dynamically linkable .so's. (No output from "find ../../.. | grep so$") I'm sorry; you need to get the latest boost CVS state for this feature. > This isn't too urgent though. The 2.1 was more important to me. > > > Where toolset is the name of one of the supported toolsets (e.g. gcc). > > What is the difference between gcc and gcc-stlport? the latter is for use with STLport (www.stlport.org). Sorry I can't answer more of your basic questions. Most of them can be answered by anyone with experience in C++. Only a few require familiarity with Boost.Python. Hopefully someone else will step up and help you out; I'm out of time. Regards, Dave From rwgk at yahoo.com Thu Jan 24 20:18:16 2002 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Thu, 24 Jan 2002 11:18:16 -0800 (PST) Subject: [C++-sig] Boost.Python and lifetime of instances of C++ classes In-Reply-To: <20020124093802.GB31760@vervet> Message-ID: <20020124191816.75967.qmail@web20201.mail.yahoo.com> --- Hugo van der Merwe wrote: > Thanks! So making it boost::shared_ptr causes Boost.Python to > automatically call Py_INCREF and Py_DECREF? When you pass a shared_ptr to Python, you have a ref-counted object (shared_ptr, please read the docs!) inside another ref-counted object (PyObject). Note: In classic Boost.Python, passing a reference to a C++ object to Python is equivalent to passing the object by value. This is known as the "returned reference problem" (and is the subject of the top development goal shown on the main C++ SIG web page.) > Another question: is it possible for a second wrapper to use things from > a first? (I want to make another wrapper, which must be usable at the > same time as the first, but there are e.g. parameters in the second that > are of types wrapped in the first.) I think this is probably not > possible, and the two will have to be combined into one? Have you looked at the section about cross-module support? Please note that there lures another problem: classic Boost.Python does not support cross-module declare_base(). Resolving this is the second most important development goal (SIG page) (at least as I see it :-). > Last question: where can I learn more about exceptions? (E.g. how to get > a C++ exception passed on into Python.) grep for "error_already_set" in the example directory. Ralf __________________________________________________ Do You Yahoo!? Great stuff seeking new owners in Yahoo! Auctions! http://auctions.yahoo.com From s13361562 at bach.sun.ac.za Thu Jan 24 23:35:21 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Fri, 25 Jan 2002 00:35:21 +0200 Subject: [C++-sig] Boost.Python and lifetime of instances of C++ classes In-Reply-To: <20020124191816.75967.qmail@web20201.mail.yahoo.com> References: <20020124093802.GB31760@vervet> <20020124191816.75967.qmail@web20201.mail.yahoo.com> Message-ID: <20020124223520.GC10410@vervet> > When you pass a shared_ptr to Python, you have a > ref-counted object (shared_ptr, please read the docs!) > inside another ref-counted object (PyObject). Thanks for the answers you've given to all my queries so far. I do'nt mean to take your time away from developing Boost.Python V2. Don't let me do it for no good reason. This is the last email in this thread. (And the other.) > Have you looked at the section about cross-module support? Sorry about not reading the docs. Didn't know I had so much of it already, really bad of me for not first searching for it. I'll make myself scarce for the next few days, or more. (Got lotsa reading to do ;) This is the second time this week I missed a relevant section that was right under my nose. > grep for "error_already_set" in the example directory. Thanks. That's basically all my questions answered. (Though some answers were rtfm - though in a much more polite way. I should've gotten an rtfm right at the start though ;) Bye for now, Hugo From achim.domma at syynx.de Mon Jan 28 13:46:24 2002 From: achim.domma at syynx.de (Achim Domma) Date: Mon, 28 Jan 2002 13:46:24 +0100 Subject: [C++-sig] Current Version Message-ID: Hi, how can I get the current beta version of boost.python ? I hope to write an article about it soon, so I would like to write my code based on the newest version. greetings Achim From david.abrahams at rcn.com Mon Jan 28 15:28:43 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Mon, 28 Jan 2002 09:28:43 -0500 Subject: [C++-sig] Current Version References: Message-ID: <015a01c1a808$9a52d8d0$0500a8c0@boostconsulting.com> Hi Achim, There isn't really a beta version yet; it's more like an alpha. The new code is probably not ready for public consumption yet, but if you tell me what your plans are I may be able to help. -Dave ----- Original Message ----- From: "Achim Domma" To: Sent: Monday, January 28, 2002 7:46 AM Subject: [C++-sig] Current Version > Hi, > > how can I get the current beta version of boost.python ? I hope to write an > article about it soon, so I would like to write my code based on the newest > version. > > greetings > Achim > > > > _______________________________________________ > C++-sig mailing list > C++-sig at python.org > http://mail.python.org/mailman/listinfo/c++-sig From achim.domma at syynx.de Mon Jan 28 16:05:56 2002 From: achim.domma at syynx.de (Achim Domma) Date: Mon, 28 Jan 2002 16:05:56 +0100 Subject: [C++-sig] Current Version In-Reply-To: <015a01c1a808$9a52d8d0$0500a8c0@boostconsulting.com> Message-ID: Hi Dave, > There isn't really a beta version yet; it's more like an alpha. > The new code > is probably not ready for public consumption yet, but if you tell me what > your plans are I may be able to help. the article will give a short introduction in extending and embedding python with boost.python. After showing the basics we will develop a simple ISAPI dll for IIS which uses Python as scripting language. My intension was to use the latest version of boost.python, but if you say it's still alpha it's probably better to use the released version. But anyway, I would be interested in testing the latest alpha. I really like boost.python and I think it's a nice place to improve my template-metaprogramming skills. I don't know if I'm good enought to be of any help but at least I could try it on some different compilers. greetings Achim From s13361562 at bach.sun.ac.za Tue Jan 29 11:20:31 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Tue, 29 Jan 2002 12:20:31 +0200 Subject: [C++-sig] Boost.Python queries Message-ID: <20020129102031.GA32550@vervet> I have the following error: demeter.cpp:134: no matching function for call to `boost::python::tuple::tuple (float &, float &, float &, int &, int &, float &, float &, bool &)' I typecast the last bool to an int when passing to tuple, that solved it. Am I pushing internal but arbitrary limits of Boost again? Exceptions: I checked the examples - they show nicely how to raise an exception in your wrapper code. Is there a way to have exceptions that are raised in the wrapped code automatically converted though? (Otherwise one would have to write an exception-converting thin wrapper for each method.) Is there not some way one could e.g. "install" a "default exception converting wrapper" automatically? (Some way it can be included in V2?) Thanks, Hugo van der Merwe From david.abrahams at rcn.com Tue Jan 29 13:58:22 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Tue, 29 Jan 2002 07:58:22 -0500 Subject: [C++-sig] Boost.Python queries References: <20020129102031.GA32550@vervet> Message-ID: <06c201c1a8c4$b1c68b80$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Hugo van der Merwe" > I have the following error: > > demeter.cpp:134: no matching function for call to `boost::python::tuple::tuple (float &, float &, float &, int &, int &, float &, float &, bool &)' > > I typecast the last bool to an int when passing to tuple, that solved > it. Am I pushing internal but arbitrary limits of Boost again? Well, I guess so, but: there's no 8-argument constructor to tuple as far as I know, so it's hard to imagine why this would work. > Exceptions: I checked the examples - they show nicely how to raise an > exception in your wrapper code. Is there a way to have exceptions that > are raised in the wrapped code automatically converted though? Not yet. You can derive your exceptions from std::exception, as described in http://groups.yahoo.com/group/boost/message/17910, but that doesn't produce very nice results. > (Otherwise one would have to write an exception-converting thin wrapper > for each method.) Is there not some way one could e.g. "install" a > "default exception converting wrapper" automatically? That's in the plan. > (Some way it can > be included in V2?) Yes. See http://www.python.org/sigs/c++-sig From s13361562 at bach.sun.ac.za Tue Jan 29 23:36:53 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Wed, 30 Jan 2002 00:36:53 +0200 Subject: [C++-sig] Boost.Python queries In-Reply-To: <06c201c1a8c4$b1c68b80$0500a8c0@boostconsulting.com> References: <20020129102031.GA32550@vervet> <06c201c1a8c4$b1c68b80$0500a8c0@boostconsulting.com> Message-ID: <20020129223653.GH5033@vervet> > > I have the following error: > > > > demeter.cpp:134: no matching function for call to > `boost::python::tuple::tuple (float &, float &, float &, int &, int &, float > &, float &, bool &)' > > > > I typecast the last bool to an int when passing to tuple, that solved > > it. Am I pushing internal but arbitrary limits of Boost again? > > Well, I guess so, but: there's no 8-argument constructor to tuple as far as > I know, so it's hard to imagine why this would work. Sorry, it didn't ;) Somehow I must have gotten old code or something. (I had two working copies of my work lying around, maybe I took the wrong directory...) So is it easy for me to push this limit up to 8? Or should I try grouping my returned tuples like so: ( (x,y,z), (a,b), (f,g), k) ? > > Exceptions: I checked the examples - they show nicely how to raise an > > exception in your wrapper code. Is there a way to have exceptions that > > are raised in the wrapped code automatically converted though? > > Not yet. You can derive your exceptions from std::exception, as described in > http://groups.yahoo.com/group/boost/message/17910, but that doesn't produce > very nice results. Thanks. I'll take a look. Boost.Python truly is great to work with, thanks! Hugo From david.abrahams at rcn.com Tue Jan 29 23:56:23 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Tue, 29 Jan 2002 17:56:23 -0500 Subject: [C++-sig] Boost.Python queries References: <20020129102031.GA32550@vervet> <06c201c1a8c4$b1c68b80$0500a8c0@boostconsulting.com> <20020129223653.GH5033@vervet> Message-ID: <083001c1a918$5996f860$0500a8c0@boostconsulting.com> ----- Original Message ----- From: "Hugo van der Merwe" > > So is it easy for me to push this limit up to 8? Or should I try > grouping my returned tuples like so: > > ( (x,y,z), (a,b), (f,g), k) ? For now, the easiest thing to do is to initialize your tuple with its length, then call set_item repeatedly to initialize the elements: tuple t(8); t.set_item(0, make_ref(x)); t.set_item(1, make_ref(y)); ... > > > Exceptions: I checked the examples - they show nicely how to raise an > > > exception in your wrapper code. Is there a way to have exceptions that > > > are raised in the wrapped code automatically converted though? > > > > Not yet. You can derive your exceptions from std::exception, as described in > > http://groups.yahoo.com/group/boost/message/17910, but that doesn't produce > > very nice results. > > Thanks. I'll take a look. > > Boost.Python truly is great to work with, thanks! Glad you like it! -Dave From s13361562 at bach.sun.ac.za Wed Jan 30 11:06:30 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Wed, 30 Jan 2002 12:06:30 +0200 Subject: [C++-sig] trouble with virtual methods Message-ID: <20020130100630.GA10208@vervet> I cannot track down what I'm doing wrong. Here follows code extracts from what I'm struggling with. If anyone can point me in the direction I should investigate, that would be great. Terrain.h contains: ------------ class TextureFactory { public: virtual GLuint GetTexture(int index,float originX,float originY,float width,float height) = 0; virtual void UnloadTexture(int index,float originX,float originY) = 0; }; ------------ My wrapper code contains: ------------ class TextureFactory_callback : public TextureFactory { public: TextureFactory_callback(PyObject* self) : /*TextureFactory(),*/ m_self(self) { cout << "here I am" << endl; } ~TextureFactory_callback(){ cout << "ack, I'm dead!" << endl; } GLuint GetTexture(int index, float originX, float originY, float width, float height) { return python::callback ::call_method(m_self, "GetTexture", index, originX, originY, width, height); } void UnloadTexture(int index, float originX, float originY) { return python::callback::call_method(m_self, "UnloadTexture", index, originX, originY); } private: PyObject* m_self; }; void func(TextureFactory* t){ t->GetTexture(0,0,0,0,0); } BOOST_PYTHON_MODULE_INIT(demeter) { python::module_builder this_module("demeter"); python::class_builder TextureFactory_class(this_module, "TextureFactory"); this_module.def(func, "func"); } ------------ Now in Python I do: >>> import demeter >>> class boo(demeter.TextureFactory): ... def GetTexture(a, b, c, d, e): print "gotcha:",a,b,c,d,e ... >>> tf = boo(); here I am >>> demeter.func(tf) Aborted And so it bombs out. I have looked at examples and documentation, but have not found the answer yet. (Note, libs/python/doc/overriding.html has a "struct baz_callback {" which I guess should be "struct baz_callback : baz {" ? I have 1.26.0) Thanks, Hugo From david.abrahams at rcn.com Wed Jan 30 12:37:53 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 30 Jan 2002 06:37:53 -0500 Subject: [C++-sig] trouble with virtual methods References: <20020130100630.GA10208@vervet> Message-ID: <08ee01c1a982$9ef08510$0500a8c0@boostconsulting.com> I don't see anything wrong with it on the surface. If it were me I'd break out the debugger at this point ;-/ HTH, Dave ----- Original Message ----- From: "Hugo van der Merwe" To: Sent: Wednesday, January 30, 2002 5:06 AM Subject: [C++-sig] trouble with virtual methods > I cannot track down what I'm doing wrong. Here follows code extracts > from what I'm struggling with. If anyone can point me in the direction I > should investigate, that would be great. > > Terrain.h contains: > ------------ > class TextureFactory > { > public: > virtual GLuint GetTexture(int index,float originX,float originY,float width,float height) = 0; > virtual void UnloadTexture(int index,float originX,float originY) = 0; > }; > ------------ > > My wrapper code contains: > ------------ > class TextureFactory_callback : public TextureFactory { > public: > TextureFactory_callback(PyObject* self) : /*TextureFactory(),*/ m_self(self) { > cout << "here I am" << endl; > } > ~TextureFactory_callback(){ cout << "ack, I'm dead!" << endl; } > > GLuint GetTexture(int index, float originX, float originY, float width, float height) { > return python::callback > ::call_method(m_self, "GetTexture", index, originX, originY, width, height); > } > > void UnloadTexture(int index, float originX, float originY) { > return python::callback::call_method(m_self, "UnloadTexture", > index, originX, originY); > } > private: > PyObject* m_self; > }; > > void func(TextureFactory* t){ t->GetTexture(0,0,0,0,0); } > > BOOST_PYTHON_MODULE_INIT(demeter) > { > python::module_builder this_module("demeter"); > python::class_builder > TextureFactory_class(this_module, "TextureFactory"); > this_module.def(func, "func"); > > } > ------------ > > Now in Python I do: > > >>> import demeter > >>> class boo(demeter.TextureFactory): > ... def GetTexture(a, b, c, d, e): print "gotcha:",a,b,c,d,e > ... > >>> tf = boo(); > here I am > >>> demeter.func(tf) > Aborted > > And so it bombs out. I have looked at examples and documentation, but > have not found the answer yet. > > (Note, libs/python/doc/overriding.html has a "struct baz_callback {" > which I guess should be "struct baz_callback : baz {" ? I have 1.26.0) > > Thanks, > Hugo > > _______________________________________________ > C++-sig mailing list > C++-sig at python.org > http://mail.python.org/mailman/listinfo/c++-sig From s13361562 at bach.sun.ac.za Wed Jan 30 21:47:28 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Wed, 30 Jan 2002 22:47:28 +0200 Subject: [C++-sig] trouble with virtual methods In-Reply-To: <08ee01c1a982$9ef08510$0500a8c0@boostconsulting.com> References: <20020130100630.GA10208@vervet> <08ee01c1a982$9ef08510$0500a8c0@boostconsulting.com> Message-ID: <20020130204728.GC13110@vervet> On Wed, Jan 30, 2002 at 06:37:53AM -0500, David Abrahams wrote: > I don't see anything wrong with it on the surface. If it were me I'd break > out the debugger at this point ;-/ OK, I'm eager to do that, but I need some tips ... I don't know much about debugging Python Extension modules. I did some searches, and it looks quite complex. Especially a step-by-step guide I saw for Win32. I'm using Linux, anyway. Hoping things have become easier, can you point me in a direction where I can learn how to debug extensions? Thanks, Hugo From david.abrahams at rcn.com Wed Jan 30 22:58:03 2002 From: david.abrahams at rcn.com (David Abrahams) Date: Wed, 30 Jan 2002 16:58:03 -0500 Subject: [C++-sig] trouble with virtual methods References: <20020130100630.GA10208@vervet> <08ee01c1a982$9ef08510$0500a8c0@boostconsulting.com> <20020130204728.GC13110@vervet> Message-ID: <0b9d01c1a9d9$e7807f40$0500a8c0@boostconsulting.com> 1. Find the Python executable 2. Run gdb on it 3. Type "apropos shared" and explore the help to find out how to set breakpoints in your extension modules (which are just shared libraries) 4. You're on your own from here Good Luck, Dave ----- Original Message ----- From: "Hugo van der Merwe" To: Sent: Wednesday, January 30, 2002 3:47 PM Subject: Re: [C++-sig] trouble with virtual methods > On Wed, Jan 30, 2002 at 06:37:53AM -0500, David Abrahams wrote: > > I don't see anything wrong with it on the surface. If it were me I'd break > > out the debugger at this point ;-/ > > OK, I'm eager to do that, but I need some tips ... I don't know much > about debugging Python Extension modules. I did some searches, and it > looks quite complex. Especially a step-by-step guide I saw for Win32. > I'm using Linux, anyway. Hoping things have become easier, can you point > me in a direction where I can learn how to debug extensions? > > Thanks, > Hugo > > _______________________________________________ > C++-sig mailing list > C++-sig at python.org > http://mail.python.org/mailman/listinfo/c++-sig From rwgk at yahoo.com Thu Jan 31 01:13:55 2002 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Wed, 30 Jan 2002 16:13:55 -0800 (PST) Subject: [C++-sig] trouble with virtual methods In-Reply-To: <20020130204728.GC13110@vervet> Message-ID: <20020131001355.21428.qmail@web20210.mail.yahoo.com> --- Hugo van der Merwe wrote: > OK, I'm eager to do that, but I need some tips ... I don't know much > about debugging Python Extension modules. I did some searches, and it > looks quite complex. Especially a step-by-step guide I saw for Win32. > I'm using Linux, anyway. Hoping things have become easier, can you point > me in a direction where I can learn how to debug extensions? I have given up on reading debugger manuals because we have too many platforms. Here is my secret weapon for debugging extensions: #include #define CheckPoint std::cout << __FILE__ << "(" << __LINE__ << ")" << std::endl << std::flush Keep inserting CheckPoint; in your code to narrow down the problem. Ralf __________________________________________________ Do You Yahoo!? Great stuff seeking new owners in Yahoo! Auctions! http://auctions.yahoo.com From s13361562 at bach.sun.ac.za Thu Jan 31 12:12:49 2002 From: s13361562 at bach.sun.ac.za (Hugo van der Merwe) Date: Thu, 31 Jan 2002 13:12:49 +0200 Subject: [C++-sig] trouble with virtual methods In-Reply-To: <20020131001355.21428.qmail@web20210.mail.yahoo.com> References: <20020130204728.GC13110@vervet> <20020131001355.21428.qmail@web20210.mail.yahoo.com> Message-ID: <20020131111249.GA23083@vervet> > I have given up on reading debugger manuals because we have too many > platforms. Here is my secret weapon for debugging extensions: > > #include > #define CheckPoint std::cout << __FILE__ << "(" << __LINE__ << ")" << std::endl > > << std::flush > > Keep inserting > > CheckPoint; > > in your code to narrow down the problem. > > Ralf Thanks, that will work fine for extension debugging itself, I need to debug what Boost is doing though: the python print command gets executed, the C++ function I'm calling is not getting executed: I have thus narrowed it down to Boost (possibly my fault for using it incorrectly, who knows, that's what I want to find out), and I've not the knowledge of Boost to want to fill it with checkpoints. I'll follow Dave's advice, thanks. I assume all I need is to use the debug build of boost binaries - and probably a debug version of Python executable too. (gdb should then give me the right line numbers and filenames.) Will let you know what my results are, thanks. Hugo