From anthony@interlink.com.au Mon Feb 10 09:10:12 2003 From: anthony@interlink.com.au (Anthony Baxter) Date: Mon Feb 10 09:10:12 2003 Subject: [Distutils] first cut at a distutils 'fetch' command Message-ID: <200302101408.h1AE83u14894@bonanza.off.ekorp.com> ------- =_aaaaaaaaaa0 Content-Type: text/plain; charset="us-ascii" Content-ID: <14891.1044886083.1@arbhome.com.au> The following is a first cut at a distutils 'fetch' command, and a small script to make invoking this easier. The script massages the arguments, and calls a distutils 'setup' stub to do the work. It accepts either a filename (which is assumes is a local cached copy of the package, it then unpacks and builds it), a URL (in which case it fetches the URL, unpacks and builds it), or anything else is assumed to be a 'package name' - hopefully PyPI will grow a 'download_url' parameter, and it will "just work". Comments welcome - it's not complete (the zipfile unpacker needs to be done - I plan to "fix" zipfile so it's got a nice tarfile-like interface, probably other stuff), but I'd like to know if this is worth pursuing further... I don't have a name for the script yet - it's called 'distutils-helper' at the moment, but that sucks :) Anthony ------- =_aaaaaaaaaa0 Content-Type: application/octet-stream Content-ID: <14891.1044886083.2@arbhome.com.au> Content-Description: distutils.commands.fetch.py Content-Transfer-Encoding: base64 IiIiZGlzdHV0aWxzLmNvbW1hbmQuZmV0Y2gKCkltcGxlbWVudHMgdGhlIERpc3R1dGlscyAnZmV0 Y2gnIGNvbW1hbmQuCiIiIgoKIyBjcmVhdGVkIDIwMDMvMDIvMTAsIEFudGhvbnkgQmF4dGVyCgpf X3JldmlzaW9uX18gPSAiJElkJCIKCmZyb20gZGlzdHV0aWxzLmNvcmUgaW1wb3J0IENvbW1hbmQK ZnJvbSBkaXN0dXRpbHMuZXJyb3JzIGltcG9ydCAqCgpmcm9tIGRpc3R1dGlscy5jb21tYW5kLnJl Z2lzdGVyIGltcG9ydCByZWdpc3RlcgpQWVBJX1VSTCA9IHJlZ2lzdGVyLkRFRkFVTFRfUkVQT1NJ VE9SWQpkZWwgcmVnaXN0ZXIKCmltcG9ydCB0ZW1wZmlsZSwgb3MsIHN5cwoKY2xhc3MgZmV0Y2gg KENvbW1hbmQpOgoKICAgICMgQnJpZWYgKDQwLTUwIGNoYXJhY3RlcnMpIGRlc2NyaXB0aW9uIG9m IHRoZSBjb21tYW5kCiAgICBkZXNjcmlwdGlvbiA9ICJmZXRjaCBhIHBhY2thZ2UiCgogICAgIyBM aXN0IG9mIG9wdGlvbiB0dXBsZXM6IGxvbmcgbmFtZSwgc2hvcnQgbmFtZSAoTm9uZSBpZiBubyBz aG9ydAogICAgIyBuYW1lKSwgYW5kIGhlbHAgc3RyaW5nLgogICAgdXNlcl9vcHRpb25zID0gWygn cGFja2FnZT0nLCBOb25lLAogICAgICAgICAgICAgICAgICAgICAicGFja2FnZSBuYW1lIHRvIGZl dGNoIiksCiAgICAgICAgICAgICAgICAgICAgKCd1cmw9JywgTm9uZSwKICAgICAgICAgICAgICAg ICAgICAgInVybCBvZiBwYWNrYWdlIHRvIGZldGNoIiksCiAgICAgICAgICAgICAgICAgICAgKCd0 bXBkaXI9JywgTm9uZSwgCiAgICAgICAgICAgICAgICAgICAgICJsb2NhbCB0ZW1wb3JhcnkgZGly ZWN0b3J5IGZvciB1bnBhY2tpbmcgYW5kIGJ1aWxkaW5nIiksCiAgICAgICAgICAgICAgICAgICAg KCdsb2NhbGZpbGU9JywgTm9uZSwKICAgICAgICAgICAgICAgICAgICAgImRvd25sb2FkZWQgbG9j YWwgY29weSBvZiBzb3VyY2UgYXJjaGl2ZSIpLAogICAgICAgICAgICAgICAgICAgICgncHlwaS11 cmw9JywgTm9uZSwKICAgICAgICAgICAgICAgICAgICAgIlVSTCBmb3IgUHlQSSIpLAogICAgICAg ICAgICAgICAgICAgXQoKICAgIGRlZiBpbml0aWFsaXplX29wdGlvbnMgKHNlbGYpOgogICAgICAg IHNlbGYucGFja2FnZSA9IE5vbmUKICAgICAgICBzZWxmLnVybCA9IE5vbmUKICAgICAgICBzZWxm LnRtcGRpciA9IE5vbmUKICAgICAgICBzZWxmLmxvY2FsZmlsZSA9IE5vbmUKICAgICAgICBzZWxm LnB5cGlfdXJsID0gUFlQSV9VUkwKICAgICAgICBzZWxmLmJ1aWxkX2RpciA9IE5vbmUKICAgICMg aW5pdGlhbGl6ZV9vcHRpb25zKCkKCiAgICBkZWYgZmluYWxpemVfb3B0aW9ucyAoc2VsZik6CiAg ICAgICAgaWYgc2VsZi51cmwgaXMgTm9uZSBhbmQgc2VsZi5sb2NhbGZpbGUgaXMgTm9uZToKICAg ICAgICAgICAgc2VsZi51cmwgPSBQeVBJX2xvb2t1cF9kb3dubG9hZF91cmwoc2VsZi5wYWNrYWdl KQogICAgICAgIGlmIHNlbGYubG9jYWxmaWxlOgogICAgICAgICAgICBpZiBub3Qgb3MucGF0aC5l eGlzdHMoc2VsZi5sb2NhbGZpbGUpOgogICAgICAgICAgICAgICAgcmFpc2UgRGlzdHV0aWxzT3B0 aW9uRXJyb3IsIFwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICJmaWxlICVzIGRvZXNuJ3Qg ZXhpc3QiJShzZWxmLmxvY2FsZmlsZSkKICAgICMgZmluYWxpemVfb3B0aW9ucygpCgoKICAgIGRl ZiBydW4gKHNlbGYpOgogICAgICAgIGZyb20gZGlzdHV0aWxzLmNvcmUgaW1wb3J0IHJ1bl9zZXR1 cAogICAgICAgIGJ1aWxkX2RpciA9IHNlbGYudW5wYWNrKCkKICAgICAgICBkaXN0X29iaiA9IHNl bGYuZ2V0ZGlzdCgpCiAgICAgICAgb2xkY3dkID0gb3MuZ2V0Y3dkKCkKICAgICAgICBvcy5jaGRp cihzZWxmLmJ1aWxkX2RpcikKICAgICAgICB0cnk6CiAgICAgICAgICAgIGRpc3Rfb2JqLnJ1bl9j b21tYW5kKCdidWlsZCcpCiAgICAgICAgZXhjZXB0OgogICAgICAgICAgICBwcmludCAiKipFcnJv cjogV2hpbGUgYnVpbGRpbmcgaW4gZGlyZWN0b3J5ICVzOiIlc2VsZi5idWlsZF9kaXIKICAgICAg ICAgICAgcHJpbnQgIkZpeCBwcm9ibGVtLCB0aGVuIHJ1biAnc2V0dXAucHkgaW5zdGFsbCcgaW4g dGhhdCBkaXJlY3RvcnkiCiAgICAgICAgICAgIHJhaXNlCiAgICAgICAgdHJ5OgogICAgICAgICAg ICBkaXN0X29iai5ydW5fY29tbWFuZCgnaW5zdGFsbCcpCiAgICAgICAgZXhjZXB0OgogICAgICAg ICAgICBwcmludCAiKipFcnJvcjogV2hpbGUgaW5zdGFsbGluZyBmcm9tIGRpcmVjdG9yeSAlczoi JXNlbGYuYnVpbGRfZGlyCiAgICAgICAgICAgIHByaW50ICJGaXggcHJvYmxlbSwgdGhlbiBydW4g J3NldHVwLnB5IGluc3RhbGwnIGluIHRoYXQgZGlyZWN0b3J5IgogICAgICAgICAgICByYWlzZQoK ICAgICMgcnVuKCkKCiAgICBkZWYgdW5wYWNrKHNlbGYpOgogICAgICAgIGlmIG5vdCBzZWxmLmxv Y2FsZmlsZToKICAgICAgICAgICAgc2VsZi5sb2NhbGZpbGUgPSBzZWxmLmZldGNoKHNlbGYudXJs KQogICAgICAgIGlmIHNlbGYudG1wZGlyOgogICAgICAgICAgICB1bnBhY2tfZGlyID0gdGVtcGZp bGUubWtkdGVtcChwcmVmaXg9ImRpc3R1dGlscyIsIGRpcj1zZWxmLnRtcGRpcikKICAgICAgICBl bHNlOgogICAgICAgICAgICB1bnBhY2tfZGlyID0gdGVtcGZpbGUubWtkdGVtcChwcmVmaXg9ImRp c3R1dGlscyIpCiAgICAgICAgbmV3ZGlyID0gc2VsZi51bnBhY2tfZGlzdHJpYnV0aW9uKHNlbGYu bG9jYWxmaWxlLCB1bnBhY2tfZGlyKQogICAgICAgIHNlbGYuYnVpbGRfZGlyID0gbmV3ZGlyCgog ICAgZGVmIGdldGRpc3Qoc2VsZik6CiAgICAgICAgZnJvbSBkaXN0dXRpbHMuY29yZSBpbXBvcnQg cnVuX3NldHVwCiAgICAgICAgaWYgbm90IHNlbGYuYnVpbGRfZGlyOgogICAgICAgICAgICByYWlz ZSBEaXN0dXRpbHNJbnRlcm5hbEVycm9yLCAiYnVpbGRfZGlyIG5vdCBzZXQhIgogICAgICAgIHNl dHVwX3B5ID0gb3MucGF0aC5qb2luKHNlbGYuYnVpbGRfZGlyLCAnc2V0dXAucHknKQogICAgICAg IGlmIG5vdCBvcy5wYXRoLmV4aXN0cyhzZXR1cF9weSk6CiAgICAgICAgICAgIHJhaXNlIERpc3RV dGlsc0ZpbGVFcnJvciwgImNhbid0IGZpbmQgc2V0dXAucHkgZmlsZSAlcyIlKHNldHVwX3B5KQog ICAgICAgIGRpc3RvYmogPSBydW5fc2V0dXAoc2V0dXBfcHksIHN0b3BfYWZ0ZXI9ImNvbmZpZyIp CiAgICAgICAgcmV0dXJuIGRpc3RvYmoKCiAgICBkZWYgZmV0Y2ggKHNlbGYsIHVybCk6CiAgICAg ICAgaW1wb3J0IHVybGxpYjIsIHRlbXBmaWxlCiAgICAgICAgb3V0ZmQsIHRtcGZpbGUgPSB0ZW1w ZmlsZS5ta3N0ZW1wKHN1ZmZpeD1vcy5wYXRoLmJhc2VuYW1lKHVybCkpCiAgICAgICAgb3V0ZnAg PSBvcy5mZG9wZW4ob3V0ZmQsICd3JykKICAgICAgICBpbmZwID0gdXJsbGliMi51cmxvcGVuKHVy bCkKICAgICAgICB3aGlsZSAxOgogICAgICAgICAgICBkYXRhID0gaW5mcC5yZWFkKCkKICAgICAg ICAgICAgaWYgbm90IGRhdGE6CiAgICAgICAgICAgICAgICBicmVhawogICAgICAgICAgICBvdXRm cC53cml0ZShkYXRhKQogICAgICAgIG91dGZwLmNsb3NlKCkKICAgICAgICByZXR1cm4gdG1wZmls ZQoKICAgIGRlZiB1bnBhY2tfZGlzdHJpYnV0aW9uKHNlbGYsIGRpc3RmaWxlLCBidWlsZGRpcik6 CiAgICAgICAgIiIidW5wYWNrIGRpc3RyaWJ1dGlvbiBpbiAnZGlzdGZpbGUnIGluICdidWlsZGRp cicuIHJldHVybiB0aGUKICAgICAgICAgZnVsbCBwYXRoIHRvIHRoZSBuZXcgZGlyZWN0b3J5IGNy ZWF0ZWQgaW4gJ2J1aWxkZGlyJyIiIgogICAgICAgIGlmIGRpc3RmaWxlLmxvd2VyKCkuZW5kc3dp dGgoJ3ppcCcpOgogICAgICAgICAgICB1bnBhY2tlciA9IFppcGZpbGVVbnBhY2tlcgogICAgICAg IGVsc2U6CiAgICAgICAgICAgIHVucGFja2VyID0gVGFyZmlsZVVucGFja2VyCiAgICAgICAgb2xk X2N3ZCA9IG9zLmdldGN3ZCgpCiAgICAgICAgb3MuY2hkaXIoYnVpbGRkaXIpCiAgICAgICAgbmV3 ZGlyID0gdW5wYWNrZXIoZGlzdGZpbGUpCiAgICAgICAgb3MuY2hkaXIob2xkX2N3ZCkKICAgICAg ICByZXR1cm4gbmV3ZGlyCgojIGNsYXNzIHgKCmRlZiBQeVBJX2xvb2t1cF9kb3dubG9hZF91cmwo cGFja2FnZSk6CiAgICByYWlzZSBOb3RJbXBsZW1lbnRlZEVycm9yLCAiUHlQSSBkb2Vzbid0IGtu b3cgYWJvdXQgZG93bmxvYWQgVVJMcyB5ZXQhIgoKZGVmIFppcGZpbGVVbnBhY2tlcihmaWxlbmFt ZSk6CiAgICAiIiIgdW5wYWNrIHppcCBmaWxlICdmaWxlbmFtZScgaW4gY3VycmVudCBkaXJlY3Rv cnkKICAgICAgICByZXR1cm5zIGZ1bGwgcGF0aCB0byBuZXcgZGlyZWN0b3J5IiIiCiAgICByYWlz ZSBOb3RJbXBsZW1lbnRlZEVycm9yLCAiemlwIGZpbGVzIG5vdCBzdXBwb3J0ZWQgeWV0IgoKZGVm IFRhcmZpbGVVbnBhY2tlcihmaWxlbmFtZSk6CiAgICAiIiIgdW5wYWNrIHRhcmJhbGwgJ2ZpbGVu YW1lJyBpbiBjdXJyZW50IGRpcmVjdG9yeS4gCiAgICAgICAgcmV0dXJucyBmdWxsIHBhdGggdG8g bmV3IGRpcmVjdG9yeSIiIgogICAgaW1wb3J0IHRhcmZpbGUKICAgICMgdHJhbnNwYXJlbnQgLSBs ZXQgaXQgd29yayBvdXQgY29tcHJlc3Npb24KICAgIHRmID0gdGFyZmlsZS5vcGVuKGZpbGVuYW1l LCAncicpIAogICAgbWVtYmVycyA9IHRmLmdldG1lbWJlcnMoKQogICAgdWRpciA9IG1lbWJlcnNb MF0KICAgIGlmIG5vdCB1ZGlyLmlzZGlyKCk6IAogICAgICAgIHJhaXNlIFVucGFja0Vycm9yLCAi Zmlyc3QgZW50cnkgaW4gdGFyYmFsbCBzaG91bGQgYmUgYSBkaXJlY3RvcnkhIiAKICAgIGZvciBt ZW1iZXIgaW4gdGYuZ2V0bWVtYmVycygpOgogICAgICAgIHRmLmV4dHJhY3QobWVtYmVyKQogICAg cHJpbnQgdWRpci5uYW1lCiAgICByZXR1cm4gb3MucGF0aC5qb2luKG9zLmdldGN3ZCgpLCB1ZGly Lm5hbWUpCgo= ------- =_aaaaaaaaaa0 Content-Type: application/octet-stream Content-ID: <14891.1044886083.3@arbhome.com.au> Content-Description: distutils-helper.py Content-Transfer-Encoding: base64 IyEvdXNyL2Jpbi9lbnYgcHl0aG9uCgpkZWYgcGFyc2VfYXJncygpOgogICAgInJldHVybiBzYW5p dGlzZWQgYXJndW1lbnQgbGlzdCIKICAgIGltcG9ydCBzeXMsIG9zLnBhdGgKCiAgICAKICAgIGlm IGxlbihzeXMuYXJndikgPCAyIG9yIHN5cy5hcmd2WzFdIGluICgiLS1oZWxwIiwiLWgiKToKICAg ICAgICByZXR1cm4gW1snLS1oZWxwJywgJ2ZldGNoJ10sIF0KICAgIGFyZ3MgPSBbXQogICAgZ2xv YmFsYXJncyA9IFtdCiAgICBmb3IgYXJnIGluIHN5cy5hcmd2WzE6XToKICAgICAgICBpZiBhcmcu c3RhcnRzd2l0aCgnLScpOgogICAgICAgICAgICBnbG9iYWxhcmdzLmFwcGVuZChhcmcpCiAgICAg ICAgZWxpZiBvcy5wYXRoLmV4aXN0cyhhcmcpOgogICAgICAgICAgICAjIGl0J3MgYSBsb2NhbCBm aWxlbmFtZSAoYXJjaGl2ZSkKICAgICAgICAgICAgYXJncy5hcHBlbmQoWydmZXRjaCddK2dsb2Jh bGFyZ3MrWyctLWxvY2FsZmlsZT0lcyclYXJnXSkKICAgICAgICBlbGlmICc6Ly8nIGluIGFyZzoK ICAgICAgICAgICAgIyBpdCdzIGEgVVJMCiAgICAgICAgICAgIGFyZ3MuYXBwZW5kKFsnZmV0Y2gn XStnbG9iYWxhcmdzK1snLS11cmw9JXMnJWFyZ10pCiAgICAgICAgZWxzZToKICAgICAgICAgICAg IyBwdW50LCBhc3N1bWUgaXQncyBhIHBhY2thZ2UKICAgICAgICAgICAgYXJncy5hcHBlbmQoWydm ZXRjaCddK2dsb2JhbGFyZ3MrWyctLXBhY2thZ2U9JXMnJWFyZ10pCiAgICByZXR1cm4gYXJncwoK CmRlZiBtYWluKCk6CiAgICBmcm9tIGRpc3R1dGlscy5jb3JlIGltcG9ydCBzZXR1cAoKICAgIGFy Z3MgPSBwYXJzZV9hcmdzKCkKICAgIGZvciBhcmdsaXN0IGluIGFyZ3M6CiAgICAgICAgcHJpbnQg YXJnbGlzdAogICAgICAgIGRpc3QgPSBzZXR1cChuYW1lPSJkdW1teSIsIHNjcmlwdF9hcmdzPWFy Z2xpc3QpCgppZiBfX25hbWVfXyA9PSAiX19tYWluX18iOgogICAgbWFpbigpCgo= ------- =_aaaaaaaaaa0-- From master@89894.com Mon Feb 10 09:57:04 2003 From: master@89894.com (½Å¿ë ö) Date: Mon Feb 10 09:57:04 2003 Subject: [Distutils] (no subject) Message-ID: <67040-22003221125730312@89894.com>

=C1=A4=BA=B8=C5=EB=BD=C5=BA=CE =B1=C7=B0=ED =BB=E7=C7=D7=BF= =A1 =C0=C7=B0=C5 =C1=A6=B8=F1=BF=A1 [=B1=A4=B0=ED]=B6=F3=B0=ED =C7=A5=B1=E2=C7=D1 =B1=A4=B0=ED =B8=DE=C0= =CF=C0=D4=B4=CF=B4=D9=2E
=BC=F6=BD=C5=C0=BB =BF=F8=C4=A1 =BE=CA=C0=B8=BD= =C3=B8=E9 = =BC=F6=BD=C5=B0=C5=BA=CE=B8=A6 =B4=AD=B7=AF=C1=D6=BC=BC=BF=E4

 

[=B0=FA=C7=D0]  =B2=DE=C0=C7 =BF=A1=B3=CA=C1=F6 =B9=AB=C7=D1 =B5=BF= =B7=C2

=B9=AB=C7=D1 =B5=BF=B7=C2=C0=BA =B2=DE=C0=C7 =BF=A1=B3=CA=C1=F6=B0=A1 =BE= =C6=B4=D2 =BC=F6 =BE=F8=BD=C0=B4=CF=B4=D9=2E=2E=2E=2E
=C8=AD=BC=AE =BF=AC= =B7=E1=B8=A6 =BB=E7=BF=EB=C7=CF=C1=F6 =BE=CA=B0=ED,
=BF=AC=B7=E1=B0=A1=BE=F8=C0=CC =B5=B9=BE=C6=B0=A1=B4=C2 = =BF=A3=C1=F8=C0=CC =C0=D6=B4=D9=B8=E9 =BF=A1=B3=CA=C1=F6=BF=A1=20 =B4=EB=C7=D1 =B9=AE=C1=A6=B4=C2 =B8=F0=B5=CE =C7=D8=B0=E1=B5=C9 =BC=F6 =C0= =D6=B0=DA=C1=F6=BF=E4=2E
=C0=DA=B5=BF=C2=F7=B0=A1 =B1=E2=B8=A7=C0=CC =BE= =F8=C0=CC =B4=DE=B8=B1 =BC=F6=C0=D6=B4=D9=B8=E9=2E=2E=2E=2E=2E?
=BF=AC=B7=E1=B8=A6 =BE=B2=C1=F6= =BE=CA=B0=ED =B9=DF=C0=FC=B1=E2=BF=A1=BC=AD =B9=AB=C7=D1=C1=A4 =C0=FC=B1=E2= =B0=A1=20 =B9=DF=BB=FD=B5=C8=B4=D9=B8=E9 =B1=D7 =BE=EE=B5=F0=BF=A1=BC=AD=B5=E7=C1=F6= =C0=CE=B0=A3=C0=C7 =BB=EE=C0=BA =C7=B3=BF=E4=B7=CE=BF=EF=B0=CD=C0=CC=B8=E7= ,
=C8=AD=BC=AE =BF=AC=B7=E1=C0=C7 =B8=C5=BF=AC =B0=F8=C7=D8=B9=AE=C1=A6=B4=C2 =B4=F5 =C0=CC=BB=F3=C0=C7 =BF=AC=B1=B8 =B4=EB= =BB=F3=C0=CC =B5=C9 =BC=F6 =BE=F8=C0=B8=B8=E7=2E=2E=2E=2E=2E

=B1=D7=B7=AF=B3=AA =BE=D6=BC=AE =C7=CF=B0=D4=B5=B5 =BE=C6=C1=F7 =BF=EC=B8= =AE =C0=CE=B7=F9=B4=C2  =C0=CC =B9=AB=C7=D1 =B5=BF=B7=C2=C0=BB =B0=B3= =B9=DF=C0=BB =BC=BA=B0=F8 =C7=CF=C1=F6 =B8=F8=C7=DF=BD=C0=B4=CF=B4=D9=2E=2E=2E=2E=2E

=B8=B9=C0=BA =BB=E7=B6=F7=C0=CC =BF=A9=B1=E2=BF=A1 =B5=B5=C0=FC=C0=BB =C7= =CF=B0=ED =C0=D6=C0=B8=B8=E7 =BC=BA=B0=F8=C0=C7 =B1=D7 =BC=F8=B0=A3  = =C0=CE=B7=F9=C0=C7 =C7=E0=BA=B9=C0=CF=B0=CD=C0=D4=B4=CF=B4=D9=2E=2E=2E=2E
=C0=CE=B0=A3=C0=BA= =C0=FA =B9=AB=C7=D1=C0=C7 =B0=F8=B0=A3=20
=BF=EC=C1=D6=B7=CE  =BF=EC=C1=D6=B7=CE =B0=C5=C4=A7=BE=F8=C0=CC =B9= =DF=C0=FC=C7=D8 =B3=AA=B0=A5 =B0=CD=C0=D4=B4=CF=B4=D9=2E=2E=2E

=C0=CC=B4=C2 =B0=F8=B0=A3 =C1=A6=C7=D1=C0=BB =B9=E7=C1=F6=BE=CA=B0=ED =BF= =A1=B3=CA=C1=F6=B8=A6 =B9=AB=C7=D1=BB=FD=BB=EA=C7=D2 =BC=F6 =C0=D6=B4=C2 =B9= =AB=C7=D1 =B5=BF=B7=C2=C0=CC =C0=D6=C0=BB =B0=E6=BF=EC=BF=A1=B8=B8 =B0=A1=B4=C9=C7=D1 =C0=CF=C0=D4=B4=CF=B4=D9=2E=2E= =2E

=BF=A9=B1=E2=BF=A1 =B0=ED=C1=A4 =BF=A1=B3=CA=C1=F6=B8=A6 =C0=CC=BF=EB=C7= =D1 =BF=A3=C1=F8=C0=BB 16=B3=E2=B0=A3=BF=A1=B0=C9=C3=C4 =B2=D9=C1=D8=C8=F7= =B0=B3=B9=DF=C7=CF=B0=ED=C0=D6=C0=B8=B8=E7 =C3=D6=B1=D9 5=B3=E2=BF=A9=B5=BF=BE=C8 6=C8=B8=BF=A1=B0=C9=C3=C4 =BA=CE=BA= =D0 =BA=CE=BA=D0=C0=BB =C1=A6=C0=DB=C7=CF=BF=A9 =B0=CB=C1=F5=C0=BB =C7=D8=BF= =C2=B9=D9=20 =C1=F6=B1=DD=C0=BA =BC=B3=B0=E8=B0=A1 =B8=B6=B9=AB=B8=AE=B5=C7=BE=EE =C1=BE= =C7=D5 =C0=FB=C0=CE =C1=A6=C0=DB =B0=FA=C1=A4=BF=A1 =BF=CD=C0=D6=C0=B8=B8=E7= ,

=BA=B8=B4=D9 =B8=B9=C0=BA =C0=CC =B5=E9=B7=CE=BA=CE=C5=CD =C7=D4=B2=B2 = =C7=CF=B0=ED=C0=DA =C4=AB=C6=E4 =BB=E7=C0=CC=C6=AE=B8=A6 =B0=B3=BC=B3=C7=CF= =BF=A9 =B5=BF=C8=A3=C0=CE=C0=C7 =B2=DE=C0=BB =B8=B8=B5=E9=BE=EE =B0=A1=B0=ED=C0=DA =C7=D5=B4=CF=B4=D9=2E=2E=2E

=C4=AB=C6=E4 =BB=E7=C0=CC=C6=AE :  cafe=2Edaum=2Enet/106030 =C0=D4= =B4=CF=B4=D9=2E

=B8=B9=C0=CC =B9=E6=B9=AE=C7=CF=BD=C3=BE=EE =C0=DA=B7=E1 =B0=CB=BB=F6=C7= =CF=BD=C3=B0=ED =B6=C7 =B1=DB=C0=BB =B8=B9=C0=CC =BF=C3=B7=C1 =C1=D6=BC=BC= =BF=E4=2E=2E=2E=2E

  ^^ ** ^^   ^^ ** ^^  

tel : 011-281-1434  =BD=C5  =BF=EB  =C3=B6


From Jack.Jansen@oratrix.com Thu Feb 13 17:54:02 2003 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Thu Feb 13 17:54:02 2003 Subject: [Distutils] How can I create prebuilt distributions? Message-ID: I remember that this was discussed some time ago, but at that time I wasn't interested, and now I can't find it back in the archives. Can someone help me out? I want to create binary packages (i.e. packages that are usable if the end-user doesn't have a development environment). For now I'm happy with something that works on Unix (MacOSX to be specific). I had always thought that bdist did this, but it turns out that it does something completely different: it creates a tar file with all pathnames already hard-coded. This isn't good enough, as it doesn't allow the end user to select install location, etc. What I want as output is a tarfile with inside it basically a tree that has seen "setup.py build", but with some extra glue so it doesn't try to build again. If I simply tar the tree after the build this doesn't work: on the destination machine it still tries to build things, probably due to differences in modification times or something. This won't work if the end user doesn't have a c compiler, of course. Also, I wouldn't mind losing all the source code to trim down the archive size. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From theller@python.net Fri Feb 14 03:08:00 2003 From: theller@python.net (Thomas Heller) Date: Fri Feb 14 03:08:00 2003 Subject: [Distutils] How can I create prebuilt distributions? In-Reply-To: References: Message-ID: <7kc3478y.fsf@python.net> Jack Jansen writes: > I remember that this was discussed some time ago, but at that time I > wasn't interested, and now I can't find it back in the archives. Can > someone help me out? > > > I want to create binary packages (i.e. packages that are usable if the > end-user doesn't have a development environment). For now I'm happy > with something that works on Unix (MacOSX to be specific). > > > I had always thought that bdist did this, but it turns out that it > does something completely different: it creates a tar file with all > pathnames already hard-coded. This isn't good enough, as it doesn't > allow the end user to select install location, etc. > > > What I want as output is a tarfile with inside it basically a tree > that has seen "setup.py build", but with some extra glue so it doesn't > try to build again. If I simply tar the tree after the build this > doesn't work: on the destination machine it still tries to build > things, probably due to differences in modification times or > something. This won't work if the end user doesn't have a c compiler, > of course. > IIRC, I've first seen Pete Shinners doing this, so maybe you want to ask him also. I've also thought about this some time, and experimented a little bit. Here is a technique which seems to work (on Windows, with Python2.2.2). Overrider the sdist command in your setup script by a class which doesn't remove the build tree from the file list (I think this may be a buglet: If the build tree is specified in the MANIFEST.in file, it should NOT be removed later): from distutils.command import sdist class my_sdist(sdist.sdist): def prune_file_list (self): """Prune off branches that might slip into the file list as created by 'read_template()', but really don't belong there: * the build tree (typically "build") * the release tree itself (only an issue if we ran "sdist" previously with --keep-temp, or it aborted) * any RCS or CVS directories """ build = self.get_finalized_command('build') base_dir = self.distribution.get_fullname() ## Commented out, because we want the build tree to be included ## self.filelist.exclude_pattern(None, prefix=build.build_base) self.filelist.exclude_pattern(None, prefix=base_dir) self.filelist.exclude_pattern(r'/(RCS|CVS)/.*', is_regex=1) and later: setup(... cmdclass = {'sdist': my_sdist}, ...) Then, in your MANIFEST.in file, insert something like this: recursive-include build *.obj *.pyd Finally, first run 'python setup.py build', and then 'python setup.py sdist'. This creates a zip-file (default on Windows), which can be unpacked somewhere, and, since the timestamps are ok, be 'built' without a C compiler (as long as you don't change the sources). I haven't tried it, but it should also be possible to include prebuild binaries for several Pythin versions. > > Also, I wouldn't mind losing all the source code to trim down the > archive size. Also a task for either the MANIFEST.in file, or your custom sdist class. Thomas From mal@lemburg.com Fri Feb 14 03:41:01 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 14 03:41:01 2003 Subject: [Distutils] How can I create prebuilt distributions? In-Reply-To: <7kc3478y.fsf@python.net> References: <7kc3478y.fsf@python.net> Message-ID: <3E4CAB35.30102@lemburg.com> Thomas Heller wrote: > Jack Jansen writes: >>I want to create binary packages (i.e. packages that are usable if the >>end-user doesn't have a development environment). For now I'm happy >>with something that works on Unix (MacOSX to be specific). This is basically what ActiveState does with their .ppm format: they ship a tarred build directory and then run something like "python setup.py install" on the target machine. At some point way back in time they wanted to make this code available to Python for general usage, but it seems they have lost interest (in so many things :-(). So at least you now know that it does work :-) distutils is your friend and can be tweaked to do many new things. Thomas already hinted at a solution. I'd just create my own bdist_prebuilt and include the needed distutils extensions right along with it. -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 14 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ Python UK 2003, Oxford: 46 days left EuroPython 2003, Charleroi, Belgium: 130 days left From DavidA@ActiveState.com Fri Feb 14 12:04:01 2003 From: DavidA@ActiveState.com (David Ascher) Date: Fri Feb 14 12:04:01 2003 Subject: [Distutils] How can I create prebuilt distributions? References: <7kc3478y.fsf@python.net> <3E4CAB35.30102@lemburg.com> Message-ID: <3E4D209A.7090706@ActiveState.com> M.-A. Lemburg wrote: > This is basically what ActiveState does with their .ppm format: > they ship a tarred build directory and then run something like > "python setup.py install" on the target machine. Right. There were quite a few problems with that -- many of the setup.py's for example could only work with the full source distribution, etc. etc. > At some point way back in time they wanted to make this code > available to Python for general usage, but it seems they have > lost interest (in so many things :-(). Aw, don't be so mean. The PyPPM stuff has two parts -- the server, which is written in Perl (we use the PPM server that was developed for Perl a long time ago), and the client. The client isn't pretty code, it's very brittle, etc. I don't think it's a favor to anyone to use it. I am very interested in doing this still, and I'm hoping that we can find some resources to do it "right" at some point. It's true that we have had to focus on some things beyond others. MAL -- if there are other things that you feel we've dropped the ball on, please let me know (preferably off-list -- unless it has to do with distutils). We're still going to keep ActivePython up to date. We're kept adding Python features to Komodo, we're working on the next generation of Visual Python, we still maintain the ASPN Cookbook, the mailing list archives, etc. PyPPM is what I consider our biggest 'dropped ball', and while I can explain how that ball got dropped, that doesn't really get us here or there. Cheers, --da From mal@lemburg.com Fri Feb 14 14:51:02 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 14 14:51:02 2003 Subject: [Distutils] How can I create prebuilt distributions? In-Reply-To: <3E4D209A.7090706@ActiveState.com> References: <7kc3478y.fsf@python.net> <3E4CAB35.30102@lemburg.com> <3E4D209A.7090706@ActiveState.com> Message-ID: <3E4D4831.2050109@lemburg.com> David Ascher wrote: > M.-A. Lemburg wrote: > >> This is basically what ActiveState does with their .ppm format: >> they ship a tarred build directory and then run something like >> "python setup.py install" on the target machine. > > Right. There were quite a few problems with that -- many of the > setup.py's for example could only work with the full source > distribution, etc. etc. > >> At some point way back in time they wanted to make this code >> available to Python for general usage, but it seems they have >> lost interest (in so many things :-(). > > Aw, don't be so mean. Ah, not being mean... it's just that I have asked so many times to make the code public and nothing ever happened. > The PyPPM stuff has two parts -- the server, which > is written in Perl (we use the PPM server that was developed for Perl a > long time ago), and the client. The client isn't pretty code, it's very > brittle, etc. I don't think it's a favor to anyone to use it. Even if it's ugly code, it could still provide a) information of how this can be done b) opens up the world to ActivePython There has been some work put into a package directory recently (see http://www.python.org/peps/pep-0301.html) and registering .ppm like files in such a directory would sure make the Python experience an even better one :-) Here's a demo: http://www.amk.ca/cgi-bin/pypi.cgi > I am very interested in doing this still, and I'm hoping that we can > find some resources to do it "right" at some point. It's true that we > have had to focus on some things beyond others. That would be great :-) The system is missing the "get the stuff and install it" part (which is good, since its better to do this in small steps rather than coming up with big ideas and then losing interest). -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 14 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ Python UK 2003, Oxford: 46 days left EuroPython 2003, Charleroi, Belgium: 130 days left From theller@python.net Fri Feb 14 15:06:30 2003 From: theller@python.net (Thomas Heller) Date: Fri Feb 14 15:06:30 2003 Subject: [Distutils] How can I create prebuilt distributions? In-Reply-To: <3E4D4831.2050109@lemburg.com> References: <7kc3478y.fsf@python.net> <3E4CAB35.30102@lemburg.com> <3E4D209A.7090706@ActiveState.com> <3E4D4831.2050109@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > There has been some work put into a package directory recently > (see http://www.python.org/peps/pep-0301.html) and registering > .ppm like files in such a directory would sure make the Python > experience an even better one :-) > > Here's a demo: http://www.amk.ca/cgi-bin/pypi.cgi > > > I am very interested in doing this still, and I'm hoping that we can > > find some resources to do it "right" at some point. It's true that > > we have had to focus on some things beyond others. > > > That would be great :-) > > The system is missing the "get the stuff and install it" part > (which is good, since its better to do this in small steps > rather than coming up with big ideas and then losing interest). The system is missing more, IMO, and that's the critical part: There should be a way to programatically find out which file I have to download. I have suggested this when the PEP was posted, but it wasn't included in the PEP afaik. Now they have added a download-url field, but there must be convention how to interpret it's value IMO (depending on platform, version, and so on). Again: determining from a Python script which dist-file I have to download is the most critical part, all the other stuff has been demonstrated in several implementations (ciphon, pypan, maybe more). Thomas From mal@lemburg.com Fri Feb 14 15:25:49 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 14 15:25:49 2003 Subject: [Distutils] How can I create prebuilt distributions? In-Reply-To: References: <7kc3478y.fsf@python.net> <3E4CAB35.30102@lemburg.com> <3E4D209A.7090706@ActiveState.com> <3E4D4831.2050109@lemburg.com> Message-ID: <3E4D4F47.9020701@lemburg.com> Thomas Heller wrote: > "M.-A. Lemburg" writes: > > >>There has been some work put into a package directory recently >>(see http://www.python.org/peps/pep-0301.html) and registering >>.ppm like files in such a directory would sure make the Python >>experience an even better one :-) >> >>Here's a demo: http://www.amk.ca/cgi-bin/pypi.cgi >> >> >>>I am very interested in doing this still, and I'm hoping that we can >>>find some resources to do it "right" at some point. It's true that >>>we have had to focus on some things beyond others. >> >> >>That would be great :-) >> >>The system is missing the "get the stuff and install it" part >>(which is good, since its better to do this in small steps >>rather than coming up with big ideas and then losing interest). > > > The system is missing more, IMO, and that's the critical part: > There should be a way to programatically find out which file > I have to download. I have suggested this when the PEP was posted, > but it wasn't included in the PEP afaik. > > Now they have added a download-url field, but there must be convention > how to interpret it's value IMO (depending on platform, version, and > so on). > > Again: determining from a Python script which dist-file I have to > download is the most critical part, all the other stuff has been > demonstrated in several implementations (ciphon, pypan, maybe more). Agreed. There should be a list of entries (distutils platform string, pyversion, disttype, download URL). There should be enough information to let distutils decide which version to download. I suppose that the register command could be made to generate most of this information from the data used to build the package. -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 14 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ Python UK 2003, Oxford: 46 days left EuroPython 2003, Charleroi, Belgium: 130 days left From DavidA@ActiveState.com Fri Feb 14 15:25:57 2003 From: DavidA@ActiveState.com (David Ascher) Date: Fri Feb 14 15:25:57 2003 Subject: [Distutils] How can I create prebuilt distributions? In-Reply-To: <3E4D4831.2050109@lemburg.com> References: <7kc3478y.fsf@python.net> <3E4CAB35.30102@lemburg.com> <3E4D209A.7090706@ActiveState.com> <3E4D4831.2050109@lemburg.com> Message-ID: <3E4D4F6D.2040003@ActiveState.com> M.-A. Lemburg wrote: > Ah, not being mean... it's just that I have asked so many > times to make the code public and nothing ever happened. The client code ships with ActivePython. Feel free to look at it, and then tell me if you really want it =). On the PPD-generation front, we basically define a new distutils target, and then do "python setup.py bdist_ppm" and then package the setup.py and the distributions subdirectories. The details of bdist_ppm change periodically, and I know I've given that code to some people, I don't remember if I sent it to you. Regardless, it's not a solid foundation for the future. The fundamental problem is that people write setup.py's which aren't tested in the case where the "install" phase is done on a different machine than the build. So the build phase may do lots of machine-specific checking and then the install phase breaks if e.g. Python is installed in a different directory, or it's built on win2k and installed on win9x, etc. The way people manage 'supplementary' files varies also a great deal between packages. It's also very hard to do packages which install different things depending on what is available on the target machine. I think that the "right" answer involves a _lot_ of work specifying and validating the definitions for the targets, _if that approach is to be used_. I'm not 100% sure that that approach is the right one. The bdist_wininst approach seems to work much better in practice, although I'm not sure that it covers all the cases that I wanted to cover (things getting installed in the Tools directory with proper shebang line tweaking, etc.). > There has been some work put into a package directory recently > (see http://www.python.org/peps/pep-0301.html) and registering > .ppm like files in such a directory would sure make the Python > experience an even better one :-) Yes, I know about PyPI, and I'm definitely hoping that we'll be able to help provide PPM-style functionality for everything registered in PyPI. > The system is missing the "get the stuff and install it" part > (which is good, since its better to do this in small steps > rather than coming up with big ideas and then losing interest). One problem with how PyPPM has "happened" in the past has been that the people doing it at ActiveState either weren't knowledgeable enough about Python or distutils, or didn't have the bandwidth needed to effect the required changes in distutils or in people's setup.py's. (I still think that shipping setup.pys as opposed to using a declarative syntax was a mistake, but that's water under the bridge, although maybe we could build a dam). Distutils is a pretty scary beast, and build and installation is a nasty domain in general because there are so many different variations. If I estimated how much time it would take to do PyPPM right from scratch, it would probably amount to six man-months of work of my best build engineer. That's something that I need to justify with a business case, something that's harder than some people expect. =) In the Perl world, PPM is a no-brainer for us to fund, because PPM is only practically available through ActivePerl, and we get a lot of business benefits by providing ActivePerl for free. In the Python world, everytime we talk about doing PPM-style things with ActivePython, key people say "I don't care about ActivePython, make it open source so that we can add it to the core" (note that ActivePython users don't say that =). While I completely understand the reaction, I'm sure you can understand that it makes the business case quite tricky. The good news is that we're reviewing our PPM strategy, and that we may be able to approach the problem in more effective ways than we have in the past. I think that PyPI will help as well because it provides a centralized point by which we can e.g. contact all of the maintainers. We could also presumably add functionality to the PyPI server that validates whether one can make a PPD out of the package, etc. I'm willing to spend some time talking about this, just not able to commit to spending significant development time or promise deliverables. --david From Jack.Jansen@oratrix.com Fri Feb 14 16:16:54 2003 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Fri Feb 14 16:16:54 2003 Subject: [Distutils] How can I create prebuilt distributions? In-Reply-To: <3E4D209A.7090706@ActiveState.com> Message-ID: <7B0C9FF3-4061-11D7-A033-000A27B19B96@oratrix.com> On vrijdag, feb 14, 2003, at 18:00 Europe/Amsterdam, David Ascher wrote: > M.-A. Lemburg wrote: > >> This is basically what ActiveState does with their .ppm format: >> they ship a tarred build directory and then run something like >> "python setup.py install" on the target machine. > > Right. There were quite a few problems with that -- many of the > setup.py's for example could only work with the full source > distribution, etc. etc. I've already given up on the idea, mainly for this reason. Plus, I want a solution that doesn't need modification of the setup.py script. For the immediate future I'm sticking with bdist-dumb distributions, and on the receiving system I can do one of three things: 1. Just unpack it and hope for the best:-) 2. Check the common prefix of the files in the distro (unfortunately filenames seem to have both ./usr/local/... and usr/local form) and check that that matches sys.prefix. 3. In the distribution database record the prefix used to create the archive, and when unpacking the files substitute sys.prefix for recorded prefix. At the moment I do (1), but I plan to switch to 2 or 3 shortly (hopefully before 2.3a2). It would be nice if at some point in the future the two phases of distutils (build and install) could be cleanly separated. The files are mostly separated (although I think that things like scripts and datafiles don't go via the build subtree, but are installed straight from the source, right?), but there should be a way in which distutils could skip the build steps. Although some stuff from the build may need to be executed, I've seen setup scripts where install actually looks inside the build object to obtain data... -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From Jack.Jansen@oratrix.com Fri Feb 14 17:07:02 2003 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Fri Feb 14 17:07:02 2003 Subject: [Distutils] How can I create prebuilt distributions? In-Reply-To: <3E4D4F6D.2040003@ActiveState.com> Message-ID: <725CF7F0-4068-11D7-A033-000A27B19B96@oratrix.com> On vrijdag, feb 14, 2003, at 21:19 Europe/Amsterdam, David Ascher wrote: > The good news is that we're reviewing our PPM strategy, and that we > may be able to approach the problem in more effective ways than we > have in the past. I think that PyPI will help as well because it > provides a centralized point by which we can e.g. contact all of the > maintainers. We could also presumably add functionality to the PyPI > server that validates whether one can make a PPD out of the package, > etc. > > I'm willing to spend some time talking about this, just not able to > commit to spending significant development time or promise > deliverables. I'm interested. I'm working on a thing called the Package Install Manager for Python (pimp) right now. I was planning not to go into design discussions until 2.3 is out and only at that time write up a PEP or something, but as pimp is 90% done I guess I can spare the time. I wanted to wait because pimp is serving a very real need I have with MacPython: on OS9 the MacPython distributions have always contained a lot of extra goodies (Numeric, PIL), and people have come to expect that. This was eating into my time for every distribution, though, and I want to get rid of it. But Mac users often don't have a development environment, and even if they do having them type in distutils commands isn't exactly considered good style. (pet peeve: to me "open source" means that you may download the source, you can build it yourself and you have to option to modify it. Unfortunately the actual state of affair is that in reality it means you must download the source, you will build it yourself and you shall modify it before it works). So the idea behind pimp is that there is an online database of packages that is specific to a (OS-version, python-version) combination and that has a maintainer who is responsible for not adding only packages that have been tested and tried. Preferably the database should have source and binary version of all packages, so the end user can state a preference for either. The database also has dependency information, and bits of code so pimp can actively test whether a specific package is installed into your python, etc etc. The idea of having the database be specific to os/python version happened to make the problem manageable, but the more I think about it the more I like it and I'm now considering it (together with having a scapeg^h^h^h^h^hmaintainer a central part of the design:-) I hope that people will appear who want to maintain the database, and even if I have to do it myself it's still less of a bottleneck if I don't have to do it at the same time of making sure everything in a new Python release works. If people want to have a look at pimp: it's in Lib/plat-mac/pimp.py. It's probably MacOSX-specific right now, but that shouldn't be too difficult to fix. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From DavidA@ActiveState.com Fri Feb 14 17:09:01 2003 From: DavidA@ActiveState.com (David Ascher) Date: Fri Feb 14 17:09:01 2003 Subject: [Distutils] How can I create prebuilt distributions? In-Reply-To: <7B0C9FF3-4061-11D7-A033-000A27B19B96@oratrix.com> References: <7B0C9FF3-4061-11D7-A033-000A27B19B96@oratrix.com> Message-ID: <3E4D68A9.4040907@ActiveState.com> Jack Jansen wrote: > > It would be nice if at some point in the future the two phases of > distutils (build and install) could be cleanly separated. +1e123 But as per my comments, you _can't_ do that without modifying a lot of pre-existing setup.py's. -da From DavidA@ActiveState.com Fri Feb 14 17:31:02 2003 From: DavidA@ActiveState.com (David Ascher) Date: Fri Feb 14 17:31:02 2003 Subject: [Distutils] How can I create prebuilt distributions? In-Reply-To: <725CF7F0-4068-11D7-A033-000A27B19B96@oratrix.com> References: <725CF7F0-4068-11D7-A033-000A27B19B96@oratrix.com> Message-ID: <3E4D6DB1.6060903@ActiveState.com> Jack Jansen wrote: > I'm interested. I'm working on a thing called the Package Install > Manager for Python (pimp) I'm going to say this just once. Please change the name. It's funny, but it's not 'serious'. I told the Piddle folks the same thing, and they regretted not changing it after it was too late. There, it's off my conscience =). > right now. I was planning not to go into > design discussions until 2.3 is out and only at that time write up a PEP > or something, but as pimp is 90% done I guess I can spare the time. Funny -- not going into design discussions until after you're done =). > But Mac users often don't have a development > environment, and even if they do having them type in distutils commands > isn't exactly considered good style. Absolutely. Same problem w/ Windows > So the idea behind pimp is that there is an online database of packages > that is specific to a (OS-version, python-version) combination and that > has a maintainer who is responsible for not adding only packages that > have been tested and tried. Preferably the database should have source > and binary version of all packages, so the end user can state a > preference for either. The database also has dependency information, and > bits of code so pimp can actively test whether a specific package is > installed into your python, etc etc. This is exactly what PPM was designed for, and it works (95% of the time) for Perl. See e.g. http://aspn.activestate.com/ASPN/Downloads/ActivePerl/PPM/Packages The way it works as a user is that if you have ActivePerl installed you can type 'ppm install Yada-Yada-Yada' and it will work (on Solaris only, according to the table above =). We update that from CPAN regularly (now), using an automated build system. We have two people dedicated to that at this time. This is a non-trivial resource commitment, both in hardware and in personel. The idea behind PyPPM is that we would do the same for Python. The problems came about because, as everyone here known, - there is no CPAN (with the actual sources, not just a catalog) - for a long time, there was no distutils - even with distutils, the process is much harder than in Perl. I'm not 100% sure I understand why. One hunch I have is that 95% of CPAN is "strictly" modules. No front-end tools, no Start Menu configurations, very little dependencies on third-party packages, etc. (also, 90% of CPAN is junk IMO, but that's another problem ;-). Regardless of what else you do, I suggest you consider the The Open Software Description Format: http://www.w3.org/TR/NOTE-OSD PPD (the format used in PPM) is a slight derivative of that. > I hope that people will appear who want to maintain the database, and > even if I have to do it myself it's still less of a bottleneck if I > don't have to do it at the same time of making sure everything in a new > Python release works. That job is what ActiveState does for Perl for three platforms right now. Note that there's more to do than maintaining the database - there's building the packages and testing them (and in our case, serving them). > If people want to have a look at pimp: it's in Lib/plat-mac/pimp.py. > It's probably MacOSX-specific right now, but that shouldn't be too > difficult to fix. Famous last words =). Seriously, though -- cross-platform setup.py's are hard to come by as soon as you deviate from pure Python modules. They tend to fail if e.g. the versions of shared libraries installed are of different version. Some setup.py's create scripts -- are those installed in Tools, in the base Python directory, where? Are their shebang lines tweaked on install? Do they even get mentioned in the manifest and put in the distribution directories? Etc. I'm not trying to criticize the authors of setup.py's, btw! I just know from painful experience that building cross-platform build systems takes a _huge_ amount of time. Unfortunately, the highest value Python packages tend to rely on extension modules (one exception to that general statement is platform.py, one of my favorite pure-python modules), and so it's the edge cases that cause the greatest grief on the user side. For example, PIL is notoriously hard to build -- it's not because Fredrik is incompetent or mean -- it's because the number of combintions that have to be dealt with is large, and it's very hard to know when you're "done" except through slow, distributed, painful testing in the field. Don't get me wrong -- I think it's great that you're doing this, but I want to suggest that you're not 90% done =). --david From kevin@cazabon.com Sun Feb 16 14:48:06 2003 From: kevin@cazabon.com (Kevin@Cazabon.com) Date: Sun Feb 16 14:48:06 2003 Subject: [Distutils] full docs? Message-ID: <007601c2d605$25a4c0d0$310aa8c0@duallie> Just built my first dist-utils installer, and it was really simple... thanks! However, the distutils documentation currently on python.org doesn't seem to be complete... I can't find anything on the --post-install option, or other useful features. Where's the best location for the "full" documentation, if any? If that's all there is yet, which source files will supply the info I need? Thanks! Kevin. From rjones@ekit-inc.com Tue Feb 18 19:38:02 2003 From: rjones@ekit-inc.com (Richard Jones) Date: Tue Feb 18 19:38:02 2003 Subject: [Distutils] PyPI now live on python.org Message-ID: <200302191136.55609.rjones@ekit-inc.com> I've just "switched on" the package index on python.org. The register command in python2.3a2 (current CVS) uses this address by default now, and the www.amk.ca script does a "half 301"* to tell users that the server has moved. Thanks Andrew for your support! The new address is http://www.python.org/pypi There's still a couple of outstanding issues: the big one is the download url handling. See http://www.python.org/sf/683939 for info on that. The verbosity issue is known about and awaiting patching (http://www.python.org/sf/684398). Neither are blockers as far as I'm concerned. Richard *"half 301" - for those who are curious, I respond with a Status: 301 and an explanation of what's going on. I don't supply the Location: header because the python urllib libraries will automatically follow it, and not supply the basic auth credentials. The missing Location: results in the register command just displaying the server response. The user is prompted to use the different URL. From pearu@cens.ioc.ee Wed Feb 19 04:32:01 2003 From: pearu@cens.ioc.ee (Pearu Peterson) Date: Wed Feb 19 04:32:01 2003 Subject: [Distutils] Fixing distutils bug #668662 before Py2.3a2 Message-ID: Hi, I have uploaded a patch fixing distutils bug #668662 and I am wondering if it is possible to apply it before Py2.3a2 is released (that should happen today according to GvR). This bug fix is crusial for those extension module builders who specify source codes with absolute paths and don't have write permissions to the location of source codes (the bug is that Py2.3a1 distutils forces the corresponding object files to go to the source directory, not to the build directory where they should go). Thanks, Pearu From fredrik@pythonware.com Wed Feb 19 07:22:21 2003 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed Feb 19 07:22:21 2003 Subject: [Distutils] Re: PyPI now live on python.org References: <200302191136.55609.rjones@ekit-inc.com> Message-ID: Richard Jones wrote: > I've just "switched on" the package index on python.org. excellent! one nit: the RSS feed seems to be broken; it seems to contain HTML stuff mixed in with the RSS data: http://www.python.org/pypi?:action=rss here's what wget gives me: PyPI recent updates http://www.python.org/pypi Updates to the Python Packages Index (PyPI) en Status: 500 Internal Server Error Content-Type: text/html Python Packages Index ... I'm also having serious problems setting trove classifiers for my contributions, but that's probably my fault. From hinsen@cnrs-orleans.fr Wed Feb 19 07:57:01 2003 From: hinsen@cnrs-orleans.fr (Konrad Hinsen) Date: Wed Feb 19 07:57:01 2003 Subject: [Distutils] Re: PyPI now live on python.org In-Reply-To: References: <200302191136.55609.rjones@ekit-inc.com> Message-ID: <200302191353.42359.hinsen@cnrs-orleans.fr> On Wednesday 19 February 2003 12:02, Fredrik Lundh wrote: > Richard Jones wrote: > > I've just "switched on" the package index on python.org. > > excellent! > > one nit: the RSS feed seems to be broken; it seems to contain > HTML stuff mixed in with the RSS data: > > http://www.python.org/pypi?:action=3Drss Here's another problem (I tried submitting a bug report, but I can't figu= re=20 out how to do it!). I tried to register, and after submitting my data I g= ot=20 the following traceback, which seems to indicate a configuration problem: PyPI: Error... Internal Server Error Traceback (most recent call last): File "/usr/local/pypi/lib/pypi/webui.py", line 110, in run self.inner_run() File "/usr/local/pypi/lib/pypi/webui.py", line 368, in inner_run getattr(self, action)() File "/usr/local/pypi/lib/pypi/webui.py", line 1183, in user self.send_email(info['email'], rego_message%info) File "/usr/local/pypi/lib/pypi/webui.py", line 1270, in send_email smtp =3D smtplib.SMTP(self.config.mailhost) File "/usr/lib/python2.2/smtplib.py", line 234, in __init__ (code, msg) =3D self.connect(host, port) File "/usr/lib/python2.2/smtplib.py", line 283, in connect raise socket.error, msg error: (61, 'Connection refused') --=20 -------------------------------------------------------------------------= ------ Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------= ------ From jh@web.de Wed Feb 19 14:30:01 2003 From: jh@web.de (Juergen Hermann) Date: Wed Feb 19 14:30:01 2003 Subject: [Distutils] python-dev not installed on Unix Message-ID: Hi! This=20is=20a=20VERY=20common=20error=20that=20traps=20users=20regularly: distutils.errors.DistutilsPlatformError:=20invalid=20Python=20installation= :=20 unable=20to=20open=20/usr/lib/python2.2/config/Makefile=20(No=20such=20fil= e=20or=20 directory)=20 Could=20we=20extend=20that=20error=20message=20by=20a=20hint=20on=20instal= ling=20the=20python- dev=20.rpm/.deb?=20 =09if=20sys.platform=20=3D=20"unix": =09=09msg=20+=3D=20"\nYou=20probably=20need=20to=20install..." Ciao,=20J=FCrgen From rjones@ekit-inc.com Wed Feb 19 15:50:01 2003 From: rjones@ekit-inc.com (Richard Jones) Date: Wed Feb 19 15:50:01 2003 Subject: [Distutils] Re: PyPI now live on python.org In-Reply-To: <200302191353.42359.hinsen@cnrs-orleans.fr> References: <200302191136.55609.rjones@ekit-inc.com> <200302191353.42359.hinsen@cnrs-orleans.fr> Message-ID: <200302200748.08233.rjones@ekit-inc.com> On Wed, 19 Feb 2003 11:53 pm, Konrad Hinsen wrote: > PyPI: Error... > > Internal Server Error > > Traceback (most recent call last): > File "/usr/local/pypi/lib/pypi/webui.py", line 110, in run > self.inner_run() > File "/usr/local/pypi/lib/pypi/webui.py", line 368, in inner_run > getattr(self, action)() > File "/usr/local/pypi/lib/pypi/webui.py", line 1183, in user > self.send_email(info['email'], rego_message%info) > File "/usr/local/pypi/lib/pypi/webui.py", line 1270, in send_email > smtp = smtplib.SMTP(self.config.mailhost) > File "/usr/lib/python2.2/smtplib.py", line 234, in __init__ > (code, msg) = self.connect(host, port) > File "/usr/lib/python2.2/smtplib.py", line 283, in connect > raise socket.error, msg > error: (61, 'Connection refused') This is fixed. Richard From rjones@ekit-inc.com Wed Feb 19 15:51:05 2003 From: rjones@ekit-inc.com (Richard Jones) Date: Wed Feb 19 15:51:05 2003 Subject: [Distutils] Re: [Catalog-sig] Re: PyPI now live on python.org In-Reply-To: References: <200302191136.55609.rjones@ekit-inc.com> Message-ID: <200302200750.32551.rjones@ekit-inc.com> On Wed, 19 Feb 2003 10:02 pm, Fredrik Lundh wrote: > Richard Jones wrote: > > I've just "switched on" the package index on python.org. > > excellent! > > one nit: the RSS feed seems to be broken; it seems to contain > HTML stuff mixed in with the RSS data: > > http://www.python.org/pypi?:action=rss > > here's what wget gives me: > > > > "http://my.n etscape.com/publish/formats/rss-0.91.dtd"> > > > PyPI recent updates > http://www.python.org/pypi > Updates to the Python Packages Index (PyPI) > en > Status: 500 Internal Server Error > Content-Type: text/html > > > > > Python Packages Index > > > > type="text/css"> href="http://mechanicalcat.net/pypi.css" type="text/css"> > marginwidth="0" marginheight="0" > link="#0000bb" vlink="#551a8b" > alink="#ff0000"> Hurm - I can't reproduce this - if you get it again, could you submit the entire response to the bug tracker on sf.net? The project is "pypi" and is linked to from the pypi pages as "Bug Reports". > I'm also having serious problems setting trove classifiers for my > contributions, but that's probably my fault. Please let me know if there's anything that can be done at my end to make this work better for you. Richard From mal@lemburg.com Wed Feb 19 16:24:10 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed Feb 19 16:24:10 2003 Subject: [Distutils] python-dev not installed on Unix In-Reply-To: References: Message-ID: <3E535B0B.6050101@lemburg.com> Juergen Hermann wrote: > Hi! > > This is a VERY common error that traps users regularly: > > distutils.errors.DistutilsPlatformError: invalid Python installation: > unable to open /usr/lib/python2.2/config/Makefile (No such file or > directory) > > Could we extend that error message by a hint on installing the python- > dev .rpm/.deb? > > if sys.platform = "unix": > msg += "\nYou probably need to install..." Sure. Do you have a list of common package names needed for this ? SuSE uses "python-devel". -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 19 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ Python UK 2003, Oxford: 41 days left EuroPython 2003, Charleroi, Belgium: 125 days left From rjones@ekit-inc.com Wed Feb 19 16:31:30 2003 From: rjones@ekit-inc.com (Richard Jones) Date: Wed Feb 19 16:31:30 2003 Subject: [Distutils] Re: [Catalog-sig] Re: PyPI now live on python.org In-Reply-To: <200302200750.32551.rjones@ekit-inc.com> References: <200302191136.55609.rjones@ekit-inc.com> <200302200750.32551.rjones@ekit-inc.com> Message-ID: <200302200800.15297.rjones@ekit-inc.com> On Thu, 20 Feb 2003 7:50 am, Richard Jones wrote: > Hurm - I can't reproduce this - if you get it again, could you submit the > entire response to the bug tracker on sf.net? The project is "pypi" and is > linked to from the pypi pages as "Bug Reports". Arg, my apologies, I could reproduce this, I was just looking at the wrong ".N" download from wget. It's fixed now. Richard From jh@web.de Wed Feb 19 16:50:00 2003 From: jh@web.de (Juergen Hermann) Date: Wed Feb 19 16:50:00 2003 Subject: [Distutils] python-dev not installed on Unix In-Reply-To: <3E535B0B.6050101@lemburg.com> Message-ID: On=20Wed,=2019=20Feb=202003=2011:23:07=20+0100,=20M.-A.=20Lemburg=20wrote:= >Sure.=20Do=20you=20have=20a=20list=20of=20common=20package=20names=20need= ed=20for >this=20? > >SuSE=20uses=20"python-devel". Debian:=20python2.2-dev Even=20if=20we=20can't=20provide=20a=20complete=20list,=20the=20hint=20wou= ld=20help. Ciao,=20J=FCrgen From theller@python.net Thu Feb 20 16:18:22 2003 From: theller@python.net (Thomas Heller) Date: Thu Feb 20 16:18:22 2003 Subject: [Distutils] Re: [Catalog-sig] Re: PyPI now live on python.org In-Reply-To: References: <200302191136.55609.rjones@ekit-inc.com> Message-ID: "Fredrik Lundh" writes: > Richard Jones wrote: > > > I've just "switched on" the package index on python.org. > > excellent! > > one nit: the RSS feed seems to be broken; it seems to contain > HTML stuff mixed in with the RSS data: > Now that this is fixed, I'm having much fun reading the pypi channel with the effnews reader. Thanks to both of you! Thomas From cbass233@yahoo.com Sun Feb 23 09:32:01 2003 From: cbass233@yahoo.com (Chuck Bass) Date: Sun Feb 23 09:32:01 2003 Subject: [Distutils] Small footprint installs w/distutils Message-ID: <20030223143129.81616.qmail@web13906.mail.yahoo.com> I'm trying to build a small footprint distribution using the setup.py file with distutils. When the code is installed a .py file is copied to the package directory and ultimately it is compiled into .pyo and .pyc files. One of my targets is a slow (133MHz) embedded processor with a flash disk (that doesn't have a great deal of space). I'd like to install just the .pyo files on this target. Is this possible or do I have to resort to some adhackery outside of distutils. Chuck Bass __________________________________________________ Do you Yahoo!? Yahoo! Tax Center - forms, calculators, tips, more http://taxes.yahoo.com/ From mal@lemburg.com Sun Feb 23 12:54:01 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Sun Feb 23 12:54:01 2003 Subject: [Distutils] Small footprint installs w/distutils In-Reply-To: <20030223143129.81616.qmail@web13906.mail.yahoo.com> References: <20030223143129.81616.qmail@web13906.mail.yahoo.com> Message-ID: <3E590A68.50901@lemburg.com> Chuck Bass wrote: > I'm trying to build a small footprint distribution > using the setup.py file with distutils. > > When the code is installed a .py file is copied to the > package directory and ultimately it is compiled into > .pyo and .pyc files. One of my targets is a slow > (133MHz) embedded processor with a flash disk (that > doesn't have a great deal of space). I'd like to > install just the .pyo files on this target. Is this > possible or do I have to resort to some adhackery > outside of distutils. I suppose you are using bdist_wininst. In that case you can compile the files on the build system and prevent compilation on the target system using these options: Options for 'bdist_wininst' command: --no-target-compile (-c) do not compile .py to .pyc on the target system --no-target-optimize (-o) do not compile .py to .pyo (optimized)on the target system -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 23 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ Python UK 2003, Oxford: 37 days left EuroPython 2003, Charleroi, Belgium: 121 days left From Jack.Jansen@oratrix.com Sun Feb 23 17:32:02 2003 From: Jack.Jansen@oratrix.com (Jack Jansen) Date: Sun Feb 23 17:32:02 2003 Subject: [Distutils] An idea on the creation of binary installers Message-ID: <86EED8C4-477E-11D7-AAB4-000A27B19B96@oratrix.com> I have a vague idea a new bdist format that I'd like to bounce off you all before I start thinking about it too hard. In other words, please shoot holes in it. The bdist_dumb formats have exactly that problem: they're too dumb. They basically create a tarfile (or other archive type) with full pathnames in it and that's it. However, there could be a bdist_xxx which would create a tarfile in which everything was relative, but the directory parts are magic. If you extract this tarfile with a normal tar you would get a directory with under it directories INSTALL_BASE, INSTALL_PLATBASE, ROOT, etc etc etc. Or maybe all these would be rooted in a xxx/platform.machine.version directory, probably a better idea. The tar file would also contain the setup.py script. The next part of the idea is that distutils would also learn about an install_xxx command. The default version of this command would simply copy the relevant files from xxx/platform.machine.version to their required locations, but it could of course be customized by the author of the setup script. The final part of the idea is that "install" gets a bit of magic so it notices it is being run from an xxx installation directory. In this case it doesn't go through the build/install sequence, but quickly calls install_xxx. I'm not sure about the magic needed, it could test for the current directory only containing setup.py and xxx (and maybe a couple of files like README and such), it could test for missing source files, I'm not really sure. And maybe this whole bit of magic isn't needed, and the end user will just have to learn to type "install_xxx", if we find a good enough name for xxx. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From s-thapa-11@alumni.uchicago.edu Sun Feb 23 22:54:01 2003 From: s-thapa-11@alumni.uchicago.edu (Suchandra Thapa) Date: Sun Feb 23 22:54:01 2003 Subject: [Distutils] An idea on the creation of binary installers In-Reply-To: <86EED8C4-477E-11D7-AAB4-000A27B19B96@oratrix.com> References: <86EED8C4-477E-11D7-AAB4-000A27B19B96@oratrix.com> Message-ID: <1046058795.1772.71.camel@hepcat> --=-kIhZCjsUQ/MlKsk6kSBa Content-Type: multipart/alternative; boundary="=-xvUTo08VYonJbRjXJcsF" --=-xvUTo08VYonJbRjXJcsF Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On Sun, 2003-02-23 at 16:31, Jack Jansen wrote: The bdist_dumb formats have exactly that problem: they're too dumb.=20 They basically create a tarfile (or other archive type) with full=20 pathnames in it and that's it. However, there could be a bdist_xxx=20 which would create a tarfile in which everything was relative, but the=20 directory parts are magic. If you extract this tarfile with a normal=20 tar you would get a directory with under it directories INSTALL_BASE,=20 INSTALL_PLATBASE, ROOT, etc etc etc. Or maybe all these would be rooted= =20 in a xxx/platform.machine.version directory, probably a better idea.=20 The tar file would also contain the setup.py script. I think the problem is not so much in distributing binaries as generating the binaries and matching them to the user's system. This isn't a problem with pure python modules but it is a significant problem with non pure modules. For example, wxPython requires the wxWindows libraries and due to various issues you can't really create a generic wxPython binary and expect it to work on all versions of a given os. If you try to provide binaries for different versions of a binary, you'll quickly end up with a huge number of environments that you need to support. For linux systems alone, you'll need to support redhat 7.x,redhat 8.x, debian 3.0, and one or two versions of mandrake just to support some of the most popular distributions. Add in the *BSD systems, solaris, and windows and you have a huge task ahead of you. The best option at the moment would probably be to go with the traditional source install and compile on unix based systems along with something like bdist_dumb (but with a little more intelligence) and a binary install for windows. The bdist_dumb option would provide a basic binary install but should be something that requires some conscious decision so that the user realizes that the binary install will probably not work unless the user's system matches the system the module was compiled on. =20 --=20 ------------------------------------------------------------------ Suchandra S. Thapa=20 s-thapa-11@alumni.uchicago.edu ------------------------------------------------------------------ --=-xvUTo08VYonJbRjXJcsF Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Sun, 2003-02-23 at 16:31, Jack Jansen wrote: 
The bdist_dumb formats hav=
e exactly that problem: they're too dumb. 
They basically create a tarfile=
 (or other archive type) with full 
pathnames in it and that's it. =
However, there could be a bdist_xxx 
which would create a tarfile in=
 which everything was relative, but the 
directory parts are magic. If y=
ou extract this tarfile with a normal 
tar you would get a directory w=
ith under it directories INSTALL_BASE, 
INSTALL_PLATBASE, ROOT, etc etc=
 etc. Or maybe all these would be rooted 
in a xxx/platform.machine.versi=
on directory, probably a better idea. 
The tar file would also contain=
 the setup.py script.

I think the problem is not so much in distributing binarie= s as generating the binaries and matching them to the user's system.  = This isn't a problem with pure python modules but it is a significant probl= em with non pure modules.  For example, wxPython requires the wxWindow= s libraries and due to various issues you can't really create a generic wxP= ython binary and expect it to work on all versions of a given os.  If = you try to provide binaries for different versions of a binary, you'll quic= kly end up with a huge number of environments that you need to support.&nbs= p; For linux systems alone, you'll need to support redhat 7.x,redhat 8.x, d= ebian 3.0, and one or two versions of mandrake just to support some of the = most popular distributions.  Add in the *BSD systems, solaris, and win= dows and you have a huge task ahead of you.

The best option at the moment would probably be to go with= the traditional source install and compile on unix based systems along wit= h something like bdist_dumb (but with a little more intelligence) and a bin= ary install for windows. The bdist_dumb option would provide a basic binary= install but should be something that requires some conscious decision so t= hat the user realizes that the binary install will probably not work unless= the user's system matches the system the module was compiled on.  
--=20
------------------------------------------------------------------

Suchandra S. Thapa=20
s-thapa-11@alumni.uchicago.edu

------------------------------------------------------------------
--=-xvUTo08VYonJbRjXJcsF-- --=-kIhZCjsUQ/MlKsk6kSBa Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iEYEABECAAYFAj5ZlysACgkQ6nShCjt5AZLnrwCgkLd7obKFJ/ErT3Pn9zMT7Yfq a3AAni2SUYN5DtPieckHEI6BMYSPMdG3 =jZ8M -----END PGP SIGNATURE----- --=-kIhZCjsUQ/MlKsk6kSBa-- From mal@lemburg.com Mon Feb 24 15:17:08 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon Feb 24 15:17:08 2003 Subject: [Distutils] An idea on the creation of binary installers In-Reply-To: <86EED8C4-477E-11D7-AAB4-000A27B19B96@oratrix.com> References: <86EED8C4-477E-11D7-AAB4-000A27B19B96@oratrix.com> Message-ID: <3E5A7D77.6070402@lemburg.com> Jack Jansen wrote: > I have a vague idea a new bdist format that I'd like to bounce off you > all before I start thinking about it too hard. In other words, please > shoot holes in it. > > The bdist_dumb formats have exactly that problem: they're too dumb. They > basically create a tarfile (or other archive type) with full pathnames > in it and that's it. However, there could be a bdist_xxx which would > create a tarfile in which everything was relative, but the directory > parts are magic. If you extract this tarfile with a normal tar you would > get a directory with under it directories INSTALL_BASE, > INSTALL_PLATBASE, ROOT, etc etc etc. Or maybe all these would be rooted > in a xxx/platform.machine.version directory, probably a better idea. The > tar file would also contain the setup.py script. > > The next part of the idea is that distutils would also learn about an > install_xxx command. The default version of this command would simply > copy the relevant files from xxx/platform.machine.version to their > required locations, but it could of course be customized by the author > of the setup script. > > The final part of the idea is that "install" gets a bit of magic so it > notices it is being run from an xxx installation directory. In this case > it doesn't go through the build/install sequence, but quickly calls > install_xxx. I'm not sure about the magic needed, it could test for the > current directory only containing setup.py and xxx (and maybe a couple > of files like README and such), it could test for missing source files, > I'm not really sure. And maybe this whole bit of magic isn't needed, and > the end user will just have to learn to type "install_xxx", if we find a > good enough name for xxx. xxx = prebuilt ?! After all, that what this seems to be about :-) -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 24 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ Python UK 2003, Oxford: 36 days left EuroPython 2003, Charleroi, Belgium: 120 days left From rjones@ekit-inc.com Wed Feb 26 20:19:02 2003 From: rjones@ekit-inc.com (Richard Jones) Date: Wed Feb 26 20:19:02 2003 Subject: [Distutils] More documentation for setup meta-data Message-ID: <200302271218.46287.rjones@ekit-inc.com> I'm trying to better document the meta-data for the distutils (and hence PyPI). I've added words to the section in the dev doc about meta-data, and would like some feedback before I post the patch to be applied. Sorry, it's in LaTeX form (until someone writes the ReST->python doc converter ;) \subsection{Additional meta-data} \label{meta-data} The setup script may include additional meta-data beyond the name and version. This information includes: \begin{tableiii}{l|l|l|c}{code}% {Meta-Data}{Description}{Value}{Notes} \lineiii{name}{the name of the package}{short string}{(1)} \lineiii{version}{the version of this release}{short string}{(1)(4)} \lineiii{author}{package author's name}{short string}{(2)} \lineiii{author_email}{email address of the package author}{email address}{(2)} \lineiii{maintainer}{package maintainer's name}{short string}{(2)} \lineiii{maintainer_email}{email address of the package maintainer}{email address}{(2)} \lineiii{home_page}{the location of the homepage for the package}{URL}{(1)} \lineiii{description}{a short, summary description of the package}{short string}{} \lineiii{long_description}{a longer description of the package}{long string}{} \lineiii{download_url}{a location where the package may be downloaded}{URL}{(3)} \lineiii{classifiers}{a list of Trove classifiers}{list of strings}{(3)} \end{tableiii} \noindent Notes: \begin{description} \item[(1)] these fields are required \item[(2)] either the author or the maintainer must be nominated \item[(3)] should not be used if your package is to be compatible with Python versions prior to 2.2.3 or 2.3. The list is available from the PyPI website. \item[(4)] it is recommended that versions take the form ..[.] \item["short string"] a single line of text, not more than 200 characters \item["long string"] multiple lines of text \item["list of strings"] see below \end{description} \option{version} encoding is an art in itself. Python packages generally adhere to the version format \em{major.minor.patch}. The major number is 0 for initial, experimental releases of software. It is incremented for releases that represent major milestones in a package. The minor number is incremented when important new features are added to the package. The patch number increments when bug-fix releases are made. Additional trailing version information is sometimes used to indicate sub-releases. These are "a1,a2,...,aN" (for alpha releases, where functionality and API may change), "b1,b2,...,bN" (for beta releases, which only fix bugs) and "pr1,pr2,...,prN" (for final pre-release release testing). Some examples: \begin{description} \item[0.1.0] the first, experimental release of a package \item[1.0.1a2] the second alpha release of the first patch version of 1.0 \end{description} \option{classifiers} are specified in a python list: \begin{verbatim} setup(... classifiers = [ 'Development Status :: 4 - Beta', 'Environment :: Console', 'Environment :: Web Environment', 'Intended Audience :: End Users/Desktop', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'License :: OSI Approved :: Python Software Foundation License', 'Operating System :: MacOS :: MacOS X', 'Operating System :: Microsoft :: Windows', 'Operating System :: POSIX', 'Programming Language :: Python', 'Topic :: Communications :: Email', 'Topic :: Office/Business', 'Topic :: Software Development :: Bug Tracking', ], ... ) \end{verbatim} If you wish to include classifiers in your \file{setup.py} file and also wish to remain backwards-compatible with Python releases prior to 2.2.3, then you can include the following code fragment in your \file{setup.py} before the \code{setup()} call. \begin{verbatim} # patch distutils if it can't cope with the "classifiers" or # "download_url" keywords if sys.version < '2.2.3': from distutils.dist import DistributionMetadata DistributionMetadata.classifiers = None DistributionMetadata.download_url = None \end{verbatim} From mal@lemburg.com Thu Feb 27 03:26:01 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu Feb 27 03:26:01 2003 Subject: [Distutils] More documentation for setup meta-data In-Reply-To: <200302271218.46287.rjones@ekit-inc.com> References: <200302271218.46287.rjones@ekit-inc.com> Message-ID: <3E5DCB6C.9030008@lemburg.com> Richard Jones wrote: > I'm trying to better document the meta-data for the distutils (and hence > PyPI). I've added words to the section in the dev doc about meta-data, and > would like some feedback before I post the patch to be applied. Sorry, it's > in LaTeX form (until someone writes the ReST->python doc converter ;) > > \lineiii{download_url}{a location where the package may be > downloaded}{URL}{(3)} > \lineiii{classifiers}{a list of Trove classifiers}{list of strings}{(3)} > \end{tableiii} download_url is not a valid Distribution option (even though it is listed in the DistributionMetaData). I wonder why you mention it here. The concept of a single URL for downloads seems to simplistic to handle the issues involved with automatic installation. This information should also be provided in a lazy way, so that the package author can easily update the download links, e.g. by placing the information in an XML file on his site and then referencing this file in the distutils meta data. The file should then be parsed by a distutils subcommand to find the right download URL depending on the platform and Python version. -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 27 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ Python UK 2003, Oxford: 33 days left EuroPython 2003, Charleroi, Belgium: 117 days left From lac@strakt.com Thu Feb 27 03:47:02 2003 From: lac@strakt.com (Laura Creighton) Date: Thu Feb 27 03:47:02 2003 Subject: [Catalog-sig] Re: [Distutils] More documentation for setup meta-data In-Reply-To: Message from "M.-A. Lemburg" of "Thu, 27 Feb 2003 09:25:16 +0100." <3E5DCB6C.9030008@lemburg.com> References: <200302271218.46287.rjones@ekit-inc.com> <3E5DCB6C.9030008@lemburg.com> Message-ID: <200302270831.h1R8VTjL028637@ratthing-b246.strakt.com> >The concept of a single URL for downloads seems to simplistic to >handle the issues involved with automatic installation. This >information should also be provided in a lazy way, so that the >package author can easily update the download links, e.g. by >placing the information in an XML file on his site and then >referencing this file in the distutils meta data. > >The file should then be parsed by a distutils subcommand to >find the right download URL depending on the platform and >Python version. > >-- >Marc-Andre Lemburg >eGenix.com > Will this make it possible to list multiple places where one can find software that is hosted more than one place? Seems desirable. Laura Creighton From mal@lemburg.com Thu Feb 27 05:02:02 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu Feb 27 05:02:02 2003 Subject: [Catalog-sig] Re: [Distutils] More documentation for setup meta-data In-Reply-To: <200302270831.h1R8VTjL028637@ratthing-b246.strakt.com> References: <200302271218.46287.rjones@ekit-inc.com> <3E5DCB6C.9030008@lemburg.com> <200302270831.h1R8VTjL028637@ratthing-b246.strakt.com> Message-ID: <3E5DE1D8.5040505@lemburg.com> Laura Creighton wrote: >>The concept of a single URL for downloads seems too simplistic to >>handle the issues involved with automatic installation. This >>information should also be provided in a lazy way, so that the >>package author can easily update the download links, e.g. by >>placing the information in an XML file on his site and then >>referencing this file in the distutils meta data. >> >>The file should then be parsed by a distutils subcommand to >>find the right download URL depending on the platform and >>Python version. > > Will this make it possible to list multiple places where one can find > software that is hosted more than one place? Seems desirable. Right, that's the idea. -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 27 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ Python UK 2003, Oxford: 33 days left EuroPython 2003, Charleroi, Belgium: 117 days left From fdrake@acm.org Thu Feb 27 16:10:03 2003 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu Feb 27 16:10:03 2003 Subject: [Distutils] packages and modules together Message-ID: <15966.32203.998588.883882@grendel.zope.com> Here's a topic that's come up before. Distutils does not currently support having both packages and modules in the same call to distutils.core.setup(). While this is probably fine for many projects, others have been bitten by it; Anthony Baxter asked about this just last month: http://mail.python.org/pipermail/distutils-sig/2003-January/003151.html We're increasingly using distutils with Zope, and would like to make our use of it as "clean" as possible. The current setup script I'm working with has to call distutils.core.setup() three times, which is just too many for me. Some of this can probably be improved by adjusting more of the provided knobs more carefully, but the simple use of both modules and packages seems like aviodable pain. Is there any reason not to support both in the distutils core directly? While the limitation can be worked around, it seems gratuitous, and shouldn't need the extra machinations. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mal@lemburg.com Thu Feb 27 16:25:34 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu Feb 27 16:25:34 2003 Subject: [Distutils] packages and modules together In-Reply-To: <15966.32203.998588.883882@grendel.zope.com> References: <15966.32203.998588.883882@grendel.zope.com> Message-ID: <3E5E81D4.6010703@lemburg.com> Fred L. Drake, Jr. wrote: > Here's a topic that's come up before. > > Distutils does not currently support having both packages and modules > in the same call to distutils.core.setup(). While this is probably > fine for many projects, others have been bitten by it; Anthony Baxter > asked about this just last month: > > http://mail.python.org/pipermail/distutils-sig/2003-January/003151.html > > We're increasingly using distutils with Zope, and would like to make > our use of it as "clean" as possible. The current setup script I'm > working with has to call distutils.core.setup() three times, which is > just too many for me. Some of this can probably be improved by > adjusting more of the provided knobs more carefully, but the simple > use of both modules and packages seems like aviodable pain. > > Is there any reason not to support both in the distutils core > directly? While the limitation can be worked around, it seems > gratuitous, and shouldn't need the extra machinations. This seems to be related to some design decision in build_py.py which is not clear at all: # Two options control which modules will be installed: 'packages' # and 'py_modules'. The former lets us work with whole packages, not # specifying individual modules at all; the latter is for # specifying modules one-at-a-time. Currently they are mutually # exclusive: you can define one or the other (or neither), but not # both. It remains to be seen how limiting this is. # Dispose of the two "unusual" cases first: no pure Python modules # at all (no problem, just return silently), and over-specified # 'packages' and 'py_modules' options. if not self.py_modules and not self.packages: return if self.py_modules and self.packages: raise DistutilsOptionError, \ "build_py: supplying both 'packages' and 'py_modules' " + \ "options is not allowed" # Now we're down to two cases: 'py_modules' only and 'packages' only. if self.py_modules: self.build_modules() else: self.build_packages() I believe that a simple change to the .find_all_modules() method would solve the problem: the method would need to merge the two generated lists (from py_modules and the modules from the packages). -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 27 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ Python UK 2003, Oxford: 33 days left EuroPython 2003, Charleroi, Belgium: 117 days left From rjones@ekit-inc.com Thu Feb 27 17:28:01 2003 From: rjones@ekit-inc.com (Richard Jones) Date: Thu Feb 27 17:28:01 2003 Subject: [Distutils] More documentation for setup meta-data In-Reply-To: <3E5DCB6C.9030008@lemburg.com> References: <200302271218.46287.rjones@ekit-inc.com> <3E5DCB6C.9030008@lemburg.com> Message-ID: <200302280851.52571.rjones@ekit-inc.com> On Thu, 27 Feb 2003 7:25 pm, M.-A. Lemburg wrote: > Richard Jones wrote: > > I'm trying to better document the meta-data for the distutils (and hence > > PyPI). I've added words to the section in the dev doc about meta-data, > > and would like some feedback before I post the patch to be applied. > > Sorry, it's in LaTeX form (until someone writes the ReST->python doc > > converter ;) > > > > \lineiii{download_url}{a location where the package may be > > downloaded}{URL}{(3)} > > \lineiii{classifiers}{a list of Trove classifiers}{list of > > strings}{(3)} \end{tableiii} > > download_url is not a valid Distribution option (even though it > is listed in the DistributionMetaData). I wonder why you mention > it here. It is new in 2.3 > The concept of a single URL for downloads seems to simplistic to > handle the issues involved with automatic installation. This > information should also be provided in a lazy way, so that the > package author can easily update the download links, e.g. by > placing the information in an XML file on his site and then > referencing this file in the distutils meta data. > > The file should then be parsed by a distutils subcommand to > find the right download URL depending on the platform and > Python version. This system sounds quite useful and flexible. It could also get very complicated, very quickly (for a package maintainer). The download_url may still be used for this purpose if it specifies a metadata file as the download. On the other hand, Anthony Baxter has written a distutils command that will download a specified package and install it. This was recently posted to distutils-sig "first cut at a distutils 'fetch' command". Sure, it's not an optimal solution - in the same way that PyPI is going to need tweaking over time once it's actually used. It's a start though. Note that I would like to see an alternative system that is used to disseminate packages which uses a set of mirrors similar to CPAN. There's an accepted naming system so that the package may be found just using the package name and version, and a fatch command that attempts to download the package from one or more of the mirrors (depending on availability). PyPI then supplies a list of the mirror sites. Distutils may also offer an upload command, to make life even easier for the package maintainer. No need to maintain download urls or even download sites. The kicker with this plan is the provision of the bandwidth. The other elements of it are ... well, trivial. They could be implemented before 2.3 is released. Richard From fdrake@acm.org Thu Feb 27 17:36:14 2003 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu Feb 27 17:36:14 2003 Subject: [Distutils] packages and modules together In-Reply-To: <3E5E81D4.6010703@lemburg.com> References: <15966.32203.998588.883882@grendel.zope.com> <3E5E81D4.6010703@lemburg.com> Message-ID: <15966.37225.919975.152160@grendel.zope.com> M.-A. Lemburg writes: > This seems to be related to some design decision in build_py.py > which is not clear at all: No kidding! No clarity there. > I believe that a simple change to the .find_all_modules() > method would solve the problem: the method would need to merge > the two generated lists (from py_modules and the modules from > the packages). I'm sure it won't be too difficult to remove the limitation, but I wanted to determine if there were any objections before I dug into it. I'll try and work up a patch in the next few days and post it on SourceForge. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From hpk@trillke.net Thu Feb 27 17:52:01 2003 From: hpk@trillke.net (holger krekel) Date: Thu Feb 27 17:52:01 2003 Subject: [Catalog-sig] Re: [Distutils] More documentation for setup meta-data In-Reply-To: <200302280851.52571.rjones@ekit-inc.com>; from rjones@ekit-inc.com on Fri, Feb 28, 2003 at 08:51:51AM +1100 References: <200302271218.46287.rjones@ekit-inc.com> <3E5DCB6C.9030008@lemburg.com> <200302280851.52571.rjones@ekit-inc.com> Message-ID: <20030227235040.O11666@prim.han.de> [Richard Jones Fri, Feb 28, 2003 at 08:51:51AM +1100] > Note that I would like to see an alternative system that is used to > disseminate packages which uses a set of mirrors similar to CPAN. There's an > accepted naming system so that the package may be found just using the > package name and version, and a fatch command that attempts to download the > package from one or more of the mirrors (depending on availability). PyPI > then supplies a list of the mirror sites. Distutils may also offer an upload > command, to make life even easier for the package maintainer. No need to > maintain download urls or even download sites. The kicker with this plan is > the provision of the bandwidth. The other elements of it are ... well, > trivial. They could be implemented before 2.3 is released. Maybe asking c.l.py subscribers for servers (with bandwidth) as mirrors for such a project would help. After all, python distutils-packages are usually not that big. And many people are longing for a way to upload/download packages semi-automatically. holger From bh@intevation.de Fri Feb 28 05:47:27 2003 From: bh@intevation.de (Bernhard Herzog) Date: Fri Feb 28 05:47:27 2003 Subject: [Distutils] packages and modules together In-Reply-To: <3E5E81D4.6010703@lemburg.com> References: <15966.32203.998588.883882@grendel.zope.com> <3E5E81D4.6010703@lemburg.com> Message-ID: <6qadggvg3v.fsf@salmakis.intevation.de> "M.-A. Lemburg" writes: > Fred L. Drake, Jr. wrote: > > Here's a topic that's come up before. > > Distutils does not currently support having both packages and modules > > in the same call to distutils.core.setup(). [...] > I believe that a simple change to the .find_all_modules() > method would solve the problem: the method would need to merge > the two generated lists (from py_modules and the modules from > the packages). I ran across this general distutils problem as well when writing the setup.py for Thuban [1] and I derived a new build command just so that I could have both modules and packages. Overriding find_all_modules and run was necessary. find_all_modules as MAL described and run so that it will allow specifying both modules and packages. This has been working very well in Thuban. Bernhard [1] (http://thuban.intevation.org/) -- Intevation GmbH http://intevation.de/ Sketch http://sketch.sourceforge.net/ MapIt! http://www.mapit.de/ From mal@lemburg.com Fri Feb 28 06:45:01 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 28 06:45:01 2003 Subject: [Distutils] packages and modules together In-Reply-To: <6qadggvg3v.fsf@salmakis.intevation.de> References: <15966.32203.998588.883882@grendel.zope.com> <3E5E81D4.6010703@lemburg.com> <6qadggvg3v.fsf@salmakis.intevation.de> Message-ID: <3E5F4B7F.7090502@lemburg.com> Bernhard Herzog wrote: > "M.-A. Lemburg" writes: > > >>Fred L. Drake, Jr. wrote: >> >>>Here's a topic that's come up before. >>>Distutils does not currently support having both packages and modules >>>in the same call to distutils.core.setup(). > > [...] > >>I believe that a simple change to the .find_all_modules() >>method would solve the problem: the method would need to merge >>the two generated lists (from py_modules and the modules from >>the packages). > > > I ran across this general distutils problem as well when writing the > setup.py for Thuban [1] and I derived a new build command just so that I > could have both modules and packages. Overriding find_all_modules and > run was necessary. find_all_modules as MAL described and run so that it > will allow specifying both modules and packages. This has been working > very well in Thuban. Could you submit this as patch for the standard build_py command ? > Bernhard > > [1] (http://thuban.intevation.org/) > -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 28 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ Python UK 2003, Oxford: 32 days left EuroPython 2003, Charleroi, Belgium: 116 days left From bh@intevation.de Fri Feb 28 09:45:02 2003 From: bh@intevation.de (Bernhard Herzog) Date: Fri Feb 28 09:45:02 2003 Subject: [Distutils] packages and modules together References: <15966.32203.998588.883882@grendel.zope.com> <3E5E81D4.6010703@lemburg.com> <6qadggvg3v.fsf@salmakis.intevation.de> <3E5F4B7F.7090502@lemburg.com> Message-ID: <6qsmu8tqi6.fsf@salmakis.intevation.de> "M.-A. Lemburg" writes: > > Overriding find_all_modules and > > run was necessary. find_all_modules as MAL described and run so that it > > will allow specifying both modules and packages. This has been working > > very well in Thuban. > > Could you submit this as patch for the standard build_py > command ? http://www.python.org/sf/695090 Bernhard -- Intevation GmbH http://intevation.de/ Sketch http://sketch.sourceforge.net/ MapIt! http://www.mapit.de/ From jeremy@alum.mit.edu Fri Feb 28 10:26:01 2003 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Fri Feb 28 10:26:01 2003 Subject: [Distutils] installing data files and headers Message-ID: <1046446257.29989.79.camel@localhost.localdomain> Another problem we've been struggling with for Zope projects is that distutils really only installs Python modules and extensions. It's support for data files and C header files is pretty limited. We've got problems with each that could probably be solved in distutils. We often store data files in a package directory. Zope components sometimes have configuration files, presentation files like html and images, and other data. One common case is unit test packages, which often have test data in them. In all these cases, we find it useful to access the data files by loading them from the package directory. You get the package's __path__ attribute and look for data files in that directory. The problem is that distutils won't install these files for us. It ends up being a lot of work to get the files installed. You need to provide a custom distclass to copy the files at build and install time. It would be a lot more convenient if distutils would just install the files by default. I think there are some potential problems with installing non .py files. You need to have some control over what exactly gets built and installed, so that you don't install .py~ files. One possibility is to explicitly list the file extensions that constitute installable data. We did that for Zope3, but the list of extensions ended up being fairly long. The other problem we have is with header files. We'd like to have .h files that are installed inside a directory in /usr/local/include. For example, we'd like source code that uses the persistence API to read like this: #include "persistence/persistence.h" #include "persistence/persistenceAPI.h" I can't figure out any way to instruct distutils to create a persistence directory and put the headers in it. I think we'd need to extend the specification for header files to make this possible. Jeremy From Alexandre.Fayolle@logilab.fr Fri Feb 28 11:17:02 2003 From: Alexandre.Fayolle@logilab.fr (Alexandre) Date: Fri Feb 28 11:17:02 2003 Subject: [Distutils] installing data files and headers In-Reply-To: <1046446257.29989.79.camel@localhost.localdomain> References: <1046446257.29989.79.camel@localhost.localdomain> Message-ID: <20030228162600.GS533@calvin.fayauffre.org> --zCKi3GIZzVBPywwA Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit On Fri, Feb 28, 2003 at 10:31:00AM -0500, Jeremy Hylton wrote: > We often store data files in a package directory. Zope components > sometimes have configuration files, presentation files like html and > images, and other data. One common case is unit test packages, which > often have test data in them. In all these cases, we find it useful to > access the data files by loading them from the package directory. You > get the package's __path__ attribute and look for data files in that > directory. > > The problem is that distutils won't install these files for us. It ends > up being a lot of work to get the files installed. You need to provide > a custom distclass to copy the files at build and install time. It > would be a lot more convenient if distutils would just install the files > by default. Attached to this mail is a Distutils extension I coded to install twisted tml file, but it works fine to install any other file in a python package. Feel free to use, modify and distribute. -- Alexandre Fayolle LOGILAB, Paris (France). http://www.logilab.com http://www.logilab.fr http://www.logilab.org Développement logiciel avancé - Intelligence Artificielle - Formations --zCKi3GIZzVBPywwA Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="twisted_distutils.py" """Distutils extensions for twisted framework. This module enables the installation of plugins.tml files using standard distutils syntax. It adds the following commands to the standard setup.py commands: * build_twisted_plugins: build (i.e. copy) plugins * install_twisted_plugins: install plugins Additionally, the following commands have been modified to deal with plugins files: * sdist * build * install To use these extenstion, you should import the setup fonction from this module, and use it normally. To list the plugins.tml files, use the twisted_plugins keyword argument to the setup function: from twisted_distutils import setup # you can also import Extension if needed if __name__ == '__main__': setup(name='my_twisted_app', version='1.0', author='me', packages=['my_package'], twisted_plugins = ['my_package/plugins.tml']) Note that you can use this to install files that are not twisted plugins in any package directory of your application. """ # # (c) 2002 Alexandre Fayolle # This module is heavily based on code copied from the python distutils # framework, especially distutils.command.build_script, # distutils.command.install_script. Many thanks to the authors of these # modules. # This module is provided as is, I'm not responsible if anything bad # happens to you or your python library while using this module. You may # freely copy it, distribute it and use it in your library or to distribute # your applications. I'd appreciate if you could drop me an email if you plan # to do so . # # Happy twisting! # from distutils.core import Extension,Distribution,Command from distutils.command.install import install from distutils.command.build import build from distutils.command.sdist import sdist from distutils.dep_util import newer from distutils.util import convert_path import os class twisted_sdist(sdist): def add_defaults(self): sdist.add_defaults(self) if self.distribution.has_twisted_plugins(): build_twisted_plugins = self.get_finalized_command('build_twisted_plugins') self.filelist.extend(build_twisted_plugins.get_source_files()) class twisted_install(install): def initialize_options (self): install.initialize_options(self) self.twisted_plugins = None def has_twisted_plugins(self): return self.distribution.has_twisted_plugins() sub_commands = [] sub_commands.extend(install.sub_commands) sub_commands.append(('install_twisted_plugins', has_twisted_plugins)) class twisted_build(build): def initialize_options (self): build.initialize_options(self) self.twisted_plugins = None def has_twisted_plugins(self): return self.distribution.has_twisted_plugins() sub_commands = [] sub_commands.extend(build.sub_commands) sub_commands.append(('build_twisted_plugins', has_twisted_plugins)) class build_twisted_plugins (Command): description = "\"build\" twisted plugins (copy)" user_options = [ ('build-dir=', 'd', "directory to \"build\" (copy) to"), ('force', 'f', "forcibly build everything (ignore file timestamps"), ] boolean_options = ['force'] def initialize_options (self): self.build_dir = None self.twisted_plugins = None self.force = None self.outfiles = None def get_source_files(self): return self.twisted_plugins def finalize_options (self): self.set_undefined_options('build', ('build_lib', 'build_dir'), ('force', 'force')) self.twisted_plugins = self.distribution.twisted_plugins def run (self): if not self.twisted_plugins: return self.copy_twisted_plugins() def copy_twisted_plugins (self): """Copy each plugin listed in 'self.twisted_plugins'. """ self.mkpath(self.build_dir) for plugin in self.twisted_plugins: adjust = 0 plugin = convert_path(plugin) outfile = os.path.join(self.build_dir, plugin) if not self.force and not newer(plugin, outfile): self.announce("not copying %s (up-to-date)" % plugin) continue # Always open the file, but ignore failures in dry-run mode -- # that way, we'll get accurate feedback if we can read the # plugin. try: f = open(plugin, "r") except IOError: if not self.dry_run: raise f = None else: f.close() self.copy_file(plugin, outfile) class install_twisted_plugins(Command): description = "install twisted plugins" user_options = [ ('install-dir=', 'd', "directory to install scripts to"), ('build-dir=','b', "build directory (where to install from)"), ('force', 'f', "force installation (overwrite existing files)"), ('skip-build', None, "skip the build steps"), ] boolean_options = ['force', 'skip-build'] def initialize_options (self): self.install_dir = None self.force = 0 self.build_dir = None self.skip_build = None def finalize_options (self): self.set_undefined_options('build', ('build_lib', 'build_dir')) self.set_undefined_options('install', ('install_lib', 'install_dir'), ('force', 'force'), ('skip_build', 'skip_build'), ) def run (self): if not self.skip_build: self.run_command('build_twisted_plugins') self.outfiles = self.copy_tree(self.build_dir, self.install_dir) def get_inputs (self): return self.distribution.twisted_plugins or [] def get_outputs(self): return self.outfiles or [] class TwistedDistribution(Distribution): def __init__(self,attrs=None): self.twisted_plugins = None Distribution.__init__(self,attrs) self.cmdclass = {'install':twisted_install, 'install_twisted_plugins':install_twisted_plugins, 'build':twisted_build, 'build_twisted_plugins':build_twisted_plugins, 'sdist':twisted_sdist, } def has_twisted_plugins(self): return self.twisted_plugins and len(self.twisted_plugins) > 0 def setup(**attrs): from distutils.core import setup attrs['distclass'] = TwistedDistribution setup(**attrs) --zCKi3GIZzVBPywwA-- From mal@lemburg.com Fri Feb 28 11:23:01 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri Feb 28 11:23:01 2003 Subject: [Distutils] More documentation for setup meta-data In-Reply-To: <200302280851.52571.rjones@ekit-inc.com> References: <200302271218.46287.rjones@ekit-inc.com> <3E5DCB6C.9030008@lemburg.com> <200302280851.52571.rjones@ekit-inc.com> Message-ID: <3E5F8CAC.8010000@lemburg.com> Richard Jones wrote: > On Thu, 27 Feb 2003 7:25 pm, M.-A. Lemburg wrote: > >>Richard Jones wrote: >> >>>I'm trying to better document the meta-data for the distutils (and hence >>>PyPI). I've added words to the section in the dev doc about meta-data, >>>and would like some feedback before I post the patch to be applied. >>>Sorry, it's in LaTeX form (until someone writes the ReST->python doc >>>converter ;) >>> >>> \lineiii{download_url}{a location where the package may be >>>downloaded}{URL}{(3)} >>> \lineiii{classifiers}{a list of Trove classifiers}{list of >>>strings}{(3)} \end{tableiii} >> >>download_url is not a valid Distribution option (even though it >>is listed in the DistributionMetaData). I wonder why you mention >>it here. > > It is new in 2.3 Uhm, it doesn't work in Python 2.3... that's why I was asking. >>The concept of a single URL for downloads seems to simplistic to >>handle the issues involved with automatic installation. This >>information should also be provided in a lazy way, so that the >>package author can easily update the download links, e.g. by >>placing the information in an XML file on his site and then >>referencing this file in the distutils meta data. >> >>The file should then be parsed by a distutils subcommand to >>find the right download URL depending on the platform and >>Python version. > > This system sounds quite useful and flexible. It could also get very > complicated, very quickly (for a package maintainer). The download_url may > still be used for this purpose if it specifies a metadata file as the > download. > > On the other hand, Anthony Baxter has written a distutils command that will > download a specified package and install it. This was recently posted to > distutils-sig "first cut at a distutils 'fetch' command". Sure, it's not an > optimal solution - in the same way that PyPI is going to need tweaking over > time once it's actually used. It's a start though. > > Note that I would like to see an alternative system that is used to > disseminate packages which uses a set of mirrors similar to CPAN. There's an > accepted naming system so that the package may be found just using the > package name and version, and a fatch command that attempts to download the > package from one or more of the mirrors (depending on availability). PyPI > then supplies a list of the mirror sites. Distutils may also offer an upload > command, to make life even easier for the package maintainer. No need to > maintain download urls or even download sites. The kicker with this plan is > the provision of the bandwidth. The other elements of it are ... well, > trivial. They could be implemented before 2.3 is released. If you intend to use the download_url for this purpose, then you ought to reserve it's usage for this now. Otherwise, people will simply put a link to the download web-site into this field. -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Feb 28 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ Python UK 2003, Oxford: 32 days left EuroPython 2003, Charleroi, Belgium: 116 days left From rjones@ekit-inc.com Fri Feb 28 18:38:15 2003 From: rjones@ekit-inc.com (Richard Jones) Date: Fri Feb 28 18:38:15 2003 Subject: [Distutils] More documentation for setup meta-data In-Reply-To: <3E5F8CAC.8010000@lemburg.com> References: <200302271218.46287.rjones@ekit-inc.com> <200302280851.52571.rjones@ekit-inc.com> <3E5F8CAC.8010000@lemburg.com> Message-ID: <200303011034.58921.rjones@ekit-inc.com> [cc'ed only to distutils now] On Sat, 1 Mar 2003 03:22 am, M.-A. Lemburg wrote: > Richard Jones wrote: > > On Thu, 27 Feb 2003 7:25 pm, M.-A. Lemburg wrote: > >>Richard Jones wrote: > >>>I'm trying to better document the meta-data for the distutils (and hen= ce > >>>PyPI). I've added words to the section in the dev doc about meta-data, > >>>and would like some feedback before I post the patch to be applied. > >>>Sorry, it's in LaTeX form (until someone writes the ReST->python doc > >>>converter ;) > >>> > >>> \lineiii{download_url}{a location where the package may be > >>>downloaded}{URL}{(3)} > >>> \lineiii{classifiers}{a list of Trove classifiers}{list of > >>>strings}{(3)} \end{tableiii} > >> > >>download_url is not a valid Distribution option (even though it > >>is listed in the DistributionMetaData). I wonder why you mention > >>it here. > > > > It is new in 2.3 > > Uhm, it doesn't work in Python 2.3... that's why I was asking. Hurm, that's odd. I thought the patch had been applied. I guess it hasn't=20 (it's not mine, so I don't pay close attention to it)... > > Note that I would like to see an alternative system that is used to > > disseminate packages which uses a set of mirrors similar to CPAN. There= 's > > an accepted naming system so that the package may be found just using t= he > > package name and version, and a fatch command that attempts to download > > the package from one or more of the mirrors (depending on availability). > > PyPI then supplies a list of the mirror sites. Distutils may also offer > > an upload command, to make life even easier for the package maintainer. > > No need to maintain download urls or even download sites. The kicker wi= th > > this plan is the provision of the bandwidth. The other elements of it a= re > > ... well, trivial. They could be implemented before 2.3 is released. > > If you intend to use the download_url for this purpose, then > you ought to reserve it's usage for this now. Otherwise, > people will simply put a link to the download web-site > into this field. I think that the download_url and download-mirrors approaches are quite=20 independant. I guess though that download_url would specify a directory, no= t=20 a file. That would then fit in with the above scheme. Then users wouldn't=20 necessarily have to upload their package to the network-of-mirrors. I'm between jobs at the moment, and have unsubscribed from python-list. Wou= ld=20 someone else like to ask there for offers of mirror space? Just so we can s= ee=20 how viable the approach is? I don't believe that creosote will be useful=20 (from what people have said in passing) - but that's ok. I'm not completely= =20 up to speed with mirroring - can it be done in such a way that there's no=20 central "master" server? Or will we have to nominate one? Richard From amk@amk.ca Fri Feb 28 20:40:02 2003 From: amk@amk.ca (A.M. Kuchling) Date: Fri Feb 28 20:40:02 2003 Subject: [Distutils] installing data files and headers In-Reply-To: <1046446257.29989.79.camel@localhost.localdomain>; from jeremy@alum.mit.edu on Fri, Feb 28, 2003 at 10:31:00AM -0500 References: <1046446257.29989.79.camel@localhost.localdomain> Message-ID: <20030228211409.A27766@nyman.amk.ca> On Fri, Feb 28, 2003 at 10:31:00AM -0500, Jeremy Hylton wrote: >Another problem we've been struggling with for Zope projects is that >distutils really only installs Python modules and extensions. It's >support for data files and C header files is pretty limited. We've got Good idea. We have a similar subclass for Quixote that installs *.ptl files, and it's a common need. >installed, so that you don't install .py~ files. One possibility is to >explicitly list the file extensions that constitute installable data. >We did that for Zope3, but the list of extensions ended up being fairly >long. Well, what are the options? 1) List extensions. 2) Explicitly list pathnames for additional files. 3) A MANIFEST.in-like mini-language for specifying which files should be installed. 4) Automatically add things in package directories that aren't obviously irrelevant (*~, *.pyc, CVS, .svn). Any other ideas? 4) probably offers too little control, and 3) might be too much, and adds yet another file to write. What if both 1) and 2) were supported, say, like this: setup(... package_files=['zope/app/config.xml', 'zope/app/dtd.xml'], package_patterns=['*.cfg'], ) So this adds all *.cfg files in any package directory, and the two XML files. We could also allow arbitrary filenames in the 'py_modules' list, but then the very name 'py_modules' is misleading, so IMHO that's a bad idea. One nit is that packages are identified as 'zope.app' in the 'packages' and 'package_dir' keywords. build_py will have to mess around with the package_files strings, but hopefully that won't be too difficult to get right. >The other problem we have is with header files. We'd like to have .h >files that are installed inside a directory in /usr/local/include. For Why isn't installing the headers in /usr/local/python2.2/persistence OK? Are these headers completely independent of Python (e.g. for a standalone C library)? --amk (www.amk.ca) I never think twice about anything; takes too much time. -- Sanders, in "Kinda" From amk@amk.ca Fri Feb 28 20:47:01 2003 From: amk@amk.ca (A.M. Kuchling) Date: Fri Feb 28 20:47:01 2003 Subject: [Distutils] More documentation for setup meta-data In-Reply-To: <200303011034.58921.rjones@ekit-inc.com>; from rjones@ekit-inc.com on Sat, Mar 01, 2003 at 10:34:58AM +1100 References: <200302271218.46287.rjones@ekit-inc.com> <200302280851.52571.rjones@ekit-inc.com> <3E5F8CAC.8010000@lemburg.com> <200303011034.58921.rjones@ekit-inc.com> Message-ID: <20030228212124.B27766@nyman.amk.ca> On Sat, Mar 01, 2003 at 10:34:58AM +1100, Richard Jones wrote: >[cc'ed only to distutils now] > >On Sat, 1 Mar 2003 03:22 am, M.-A. Lemburg wrote: >> Uhm, it doesn't work in Python 2.3... that's why I was asking. Huh? I thought I checked all the relevant bits in before 2.3alpha2, and it seems to work for me. (It wasn't in 2.3alpha1.) MAL, when you say "it doesn't work", what do you mean? --amk (www.amk.ca) Good place to put things, cellars. -- The Doctor, in "Remembrance of the Daleks"