From manlio_perillo at libero.it Mon Aug 6 20:59:19 2007 From: manlio_perillo at libero.it (Manlio Perillo) Date: Mon, 06 Aug 2007 20:59:19 +0200 Subject: [Pydotorg-www] About the new build system Message-ID: <46B76F87.3020400@libero.it> Hi. I'm the maintainer of the python.it web site. We are using a modified version of ht2html, but it has many problems. I'm planning to update the software but it seems that the new build system for python.org has all the features I need. So I would like to contribute to the development, trying (if possibile) to write a reusable core. The current version, in fact, is strictly related to the python.org site. I can adapt the software for python.it, but I think that having the possibility to work on a common piece of software is the best solution. Thanks Manlio Perillo From sdeibel at wingware.com Mon Aug 6 23:15:44 2007 From: sdeibel at wingware.com (Stephan Deibel) Date: Mon, 06 Aug 2007 17:15:44 -0400 Subject: [Pydotorg-www] About the new build system In-Reply-To: <46B76F87.3020400@libero.it> References: <46B76F87.3020400@libero.it> Message-ID: <46B78F80.5080602@wingware.com> Manlio Perillo wrote: > I'm the maintainer of the python.it web site. > > We are using a modified version of ht2html, but it has many problems. > > I'm planning to update the software but it seems that the new build > system for python.org has all the features I need. > > So I would like to contribute to the development, trying (if possibile) > to write a reusable core. > > The current version, in fact, is strictly related to the python.org site. > I can adapt the software for python.it, but I think that having the > possibility to work on a common piece of software is the best solution. You can check it all out from svn+ssh://pydotorg at svn.python.org/trunk/beta.python.org to try it out. However, there are plenty of python.org-specific parts in the build process. The core may be reusable, tho, with some work. You're welcome to hack away at it... - Stephan From manlio_perillo at libero.it Tue Aug 7 06:49:51 2007 From: manlio_perillo at libero.it (Manlio Perillo) Date: Tue, 07 Aug 2007 06:49:51 +0200 Subject: [Pydotorg-www] About the new build system In-Reply-To: <46B78F80.5080602@wingware.com> References: <46B76F87.3020400@libero.it> <46B78F80.5080602@wingware.com> Message-ID: <46B7F9EF.6090801@libero.it> Stephan Deibel ha scritto: > Manlio Perillo wrote: >> I'm the maintainer of the python.it web site. >> >> We are using a modified version of ht2html, but it has many problems. >> >> I'm planning to update the software but it seems that the new build >> system for python.org has all the features I need. >> >> So I would like to contribute to the development, trying (if >> possibile) to write a reusable core. >> >> The current version, in fact, is strictly related to the python.org site. >> I can adapt the software for python.it, but I think that having the >> possibility to work on a common piece of software is the best solution. > > You can check it all out from > svn+ssh://pydotorg at svn.python.org/trunk/beta.python.org > to try it out. Already done, but I have used the https protocol. > However, there are plenty of python.org-specific parts > in the build > process. > The core may be reusable, tho, with some work. You're welcome > to hack > away at it... > The question is: are you interested in a partial rewrite of the software? I have some general questions: 1) What's wrong with pyramid? 2) Is the new build system used to build the current web site? and a few tecnical questions: 1) What is the syntax of the Nav header? 2) In the source code there are references to some "old" files, but in the data directory: there are no more nav.yml and body.html there are only: data/doc/humor/index.yml data/dev/summary/2007-04-01_2007-04-15/index.yml data/dev/summary/2007-04-01_2007-04-15/content.rst > - Stephan > Thanks Manlio Perillo From manlio_perillo at libero.it Tue Aug 7 09:01:12 2007 From: manlio_perillo at libero.it (Manlio Perillo) Date: Tue, 07 Aug 2007 09:01:12 +0200 Subject: [Pydotorg-www] About the new build system In-Reply-To: <46B76F87.3020400@libero.it> References: <46B76F87.3020400@libero.it> Message-ID: <46B818B8.209@libero.it> Manlio Perillo ha scritto: > Hi. > > I'm the maintainer of the python.it web site. > > We are using a modified version of ht2html, but it has many problems. > > I'm planning to update the software but it seems that the new build > system for python.org has all the features I need. > I do not know if someone is interested, but here is the features I'm planning to add: 1) split the software into separate modules: * loader.py: this module is responsible for the loading of the page content. It must return an rfc822 message and a metadata dictionary. The metadata will contain the filename of the content file, and the list of dependencies * parser.py: this module is responsible for the parsing of YML files * generator.py: the main module that generate the html pages with the navigation links. There should be a function createContext (for creating a Mako context for a page) that the user can override. * build.py: the driver 2) generic formatter/fragment renderer Add a formatter register where to register a formatter for a given content type (like rst_html.process_rst for reST) 3) extend the supported headers: * the Content-Type header will supports an additional parameter where one can explicitly specify the formatter to use (Content-Type: text/plain;formatter="myformatter") * add an Include header (I need this because some contents on python.it came from a separate section of the Subversion repository) Include: glossary.xml; Content-Type: application/xml;formatter="glossary_formatter" 4) generic support for yml files like newsindex.yml * using the pyramid extensions to YAML, there should be no need to write a separate function to parse each of such file. As an example a reST text in the newsindex.yml uses the !rest directive, so the code knows how to format it. * a page fragment that needs, as an example, newsindex.yml, can add a Resource header, that declares the resource it needs (that is, the "absolute"[1] path where to find that resource) * the template then, can call load_resource('newsindex.yml', $(resources)) (using Mako namespace directive, instead of passing the function in the context) 5) all the template should go on a template directory (and the user can override this directory via command line). Maybe all the resources (newsindex.py, sigindex.yml, sigindex.yml) should go in the same "resources" directory? [1] All the paths should be relative. The base paths are specified in the build.py module These are just a few ideas (and I'm not sure that I will be able to implement all of them) Thanks and regards Manlio Perillo From amk at amk.ca Tue Aug 7 15:38:06 2007 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 7 Aug 2007 09:38:06 -0400 Subject: [Pydotorg-www] About the new build system In-Reply-To: <46B7F9EF.6090801@libero.it> References: <46B76F87.3020400@libero.it> <46B78F80.5080602@wingware.com> <46B7F9EF.6090801@libero.it> Message-ID: <20070807133757.GA6876@amk-desktop.matrixgroup.net> On Tue, Aug 07, 2007 at 06:49:51AM +0200, Manlio Perillo wrote: > 1) What's wrong with pyramid? It's a very general system; by writing the right set of templates and fragment files, you could assemble all sorts of site designs. But we only really care about python.org, and the generality slowed Pyramid down and complicated the steps needed to add a new page. > 2) Is the new build system used to build the current web site? Yes. > 1) What is the syntax of the Nav header? IIRC, it's an indented list of (url-path {optional link title} link text) lines. > 2) In the source code there are references to some "old" files, > but in the data directory: > there are no more nav.yml and body.html > there are only: Right; body.html was used to specify the template for a page, but that generality is no longer supported. --amk From sdeibel at wingware.com Tue Aug 7 16:02:14 2007 From: sdeibel at wingware.com (Stephan Deibel) Date: Tue, 07 Aug 2007 10:02:14 -0400 Subject: [Pydotorg-www] About the new build system In-Reply-To: <46B818B8.209@libero.it> References: <46B76F87.3020400@libero.it> <46B818B8.209@libero.it> Message-ID: <46B87B66.9070706@wingware.com> Manlio Perillo wrote: > Manlio Perillo ha scritto: > I do not know if someone is interested, but here is the features I'm > planning to add: ... I'm interested in it in a somewhat idle) sense, in that I might like a simple easy-to-understand general purpose version of this. But I don't know whether it'll be worth the effort to move your new general system into use on python.org, where time/resources for maintaining the site are already severely limited. That said, I don't see any reason why you can't create, release, and maintain such a system based on this code. You may want to ask the developers to make the licensing clear -- for the new build that would be mainly Andrew Kuchling, also Martin von Loewis, David Goodger, and I wrote tiny bits myself. There may be others -- svn will know. For Pyramid (if you use any of that) it's Tim Parkin. - Stephan From manlio_perillo at libero.it Tue Aug 7 15:56:02 2007 From: manlio_perillo at libero.it (Manlio Perillo) Date: Tue, 07 Aug 2007 15:56:02 +0200 Subject: [Pydotorg-www] About the new build system In-Reply-To: <20070807133757.GA6876@amk-desktop.matrixgroup.net> References: <46B76F87.3020400@libero.it> <46B78F80.5080602@wingware.com> <46B7F9EF.6090801@libero.it> <20070807133757.GA6876@amk-desktop.matrixgroup.net> Message-ID: <46B879F2.1030806@libero.it> A.M. Kuchling ha scritto: > On Tue, Aug 07, 2007 at 06:49:51AM +0200, Manlio Perillo wrote: >> 1) What's wrong with pyramid? > > It's a very general system; by writing the right set of templates and > fragment files, you could assemble all sorts of site designs. But we > only really care about python.org, and the generality slowed Pyramid > down and complicated the steps needed to add a new page. > Ok, thanks. Just to know: what can be done with pyramid that cannot be done with the new system? I think, however, that the new system can be useful for sites other than python.org. >> 2) Is the new build system used to build the current web site? > > Yes. > >> 1) What is the syntax of the Nav header? > > IIRC, it's an indented list of (url-path {optional link title} link > text) lines. > I have read the pyramid sources. However it seems that now the Nav is indented at only one level, right? I'm asking because I want to try to simplify the code, so it can be more easily understood (if I adopt it for python.it, it should be easy to maintain for the future). >> 2) In the source code there are references to some "old" files, >> but in the data directory: >> there are no more nav.yml and body.html >> there are only: > > Right; body.html was used to specify the template for a page, but that > generality is no longer supported. > > --amk > Thanks and regards Manlio Perillo From manlio_perillo at libero.it Tue Aug 7 16:04:27 2007 From: manlio_perillo at libero.it (Manlio Perillo) Date: Tue, 07 Aug 2007 16:04:27 +0200 Subject: [Pydotorg-www] About the new build system In-Reply-To: <46B87B66.9070706@wingware.com> References: <46B76F87.3020400@libero.it> <46B818B8.209@libero.it> <46B87B66.9070706@wingware.com> Message-ID: <46B87BEB.60305@libero.it> Stephan Deibel ha scritto: > Manlio Perillo wrote: >> Manlio Perillo ha scritto: >> I do not know if someone is interested, but here is the features I'm >> planning to add: > ... > > I'm interested in it in a somewhat idle) sense, in that I might like a > simple easy-to-understand general purpose version of this. > > But I don't know whether it'll be worth the effort to move your new general > system into use on python.org, where time/resources for maintaining the > site are already severely limited. > We have the same problem on python.it... > That said, I don't see any reason why you can't create, release, and > maintain such a system based on this code. Well, because then I have to maintain the system all by myself... > You may want to ask the > developers to make the licensing clear -- for the new build that would > be mainly Andrew Kuchling, also Martin von Loewis, David Goodger, and > I wrote tiny bits myself. There may be others -- svn will know. For > Pyramid (if you use any of that) it's Tim Parkin. > Yes, this is an important question, thanks. Any problems if we are going to use the software, the images and the CSS from python.org? > - Stephan > Regards Manlio Perillo From sdeibel at wingware.com Wed Aug 8 05:03:20 2007 From: sdeibel at wingware.com (Stephan Deibel) Date: Tue, 07 Aug 2007 23:03:20 -0400 Subject: [Pydotorg-www] About the new build system In-Reply-To: <46B87BEB.60305@libero.it> References: <46B76F87.3020400@libero.it> <46B818B8.209@libero.it> <46B87B66.9070706@wingware.com> <46B87BEB.60305@libero.it> Message-ID: <46B93278.3050104@wingware.com> Manlio Perillo wrote: >> That said, I don't see any reason why you can't create, release, and >> maintain such a system based on this code. > > Well, because then I have to maintain the system all by myself... Hmm, yes, so it may not be worth the effort to make it general but instead take it and make it work for python.it? Of course there could be parts that just make sense to generalize and share... I'm a minor contributor at best so don't have immediate ideas on what would and wouldn't make sense. >> You may want to ask the >> developers to make the licensing clear -- for the new build that would >> be mainly Andrew Kuchling, also Martin von Loewis, David Goodger, and >> I wrote tiny bits myself. There may be others -- svn will know. For >> Pyramid (if you use any of that) it's Tim Parkin. >> > Yes, this is an important question, thanks. > Any problems if we are going to use the software, the images and the CSS > from python.org? I don't think there's any problem. We're still trying to do a formal copyright transfer of the logo and python.org design but I really doubt that's going to be an issue. The designer (Tim Parkin) has already agreed wrt the logo, anyway. As for the software, it would be good to have the most active contributor (probably Andrew) slap a license on it and check it in. - Stephan From manlio_perillo at libero.it Wed Aug 8 17:47:39 2007 From: manlio_perillo at libero.it (Manlio Perillo) Date: Wed, 08 Aug 2007 17:47:39 +0200 Subject: [Pydotorg-www] function trim_nav_links in build.py Message-ID: <46B9E59B.5060508@libero.it> Hi. I have another question: what's the use of the function trim_nav_links? It is not clear for me. I have tried to remove it, and the result of the build is the same. Thanks Manlio Perillo From sdeibel at wingware.com Thu Aug 9 01:05:04 2007 From: sdeibel at wingware.com (Stephan Deibel) Date: Wed, 08 Aug 2007 19:05:04 -0400 Subject: [Pydotorg-www] function trim_nav_links in build.py In-Reply-To: <46B9E59B.5060508@libero.it> References: <46B9E59B.5060508@libero.it> Message-ID: <46BA4C20.1000806@wingware.com> Manlio Perillo wrote: > I have another question: what's the use of the function trim_nav_links? > > It is not clear for me. > I have tried to remove it, and the result of the build is the same. Sorry, I don't know. It may be an artifact of earlier code. - Stephan From amk at amk.ca Thu Aug 9 04:01:32 2007 From: amk at amk.ca (A.M. Kuchling) Date: Wed, 8 Aug 2007 22:01:32 -0400 Subject: [Pydotorg-www] function trim_nav_links in build.py In-Reply-To: <46B9E59B.5060508@libero.it> References: <46B9E59B.5060508@libero.it> Message-ID: <20070809020132.GA221@andrew-kuchlings-computer.local> On Wed, Aug 08, 2007 at 05:47:39PM +0200, Manlio Perillo wrote: > I have another question: what's the use of the function trim_nav_links? It takes a header like this: Quick-links: Quick Links (2.5.1) /download/releases/2.5.1 Documentation {Manuals for Latest Stable Release} http://docs.python.org / Windows Installer {Easy Installer of Python under Windows} /ftp/python/2 .5.1/python-2.5.1.msi and strips out the leading whitespace. Perhaps parseIndentedList() handles the leading whitespace now, and trim_nav_links() is no longer needed. (The top content.ht file is the only one with the headers that use this.) --amk From manlio_perillo at libero.it Thu Aug 9 10:01:58 2007 From: manlio_perillo at libero.it (Manlio Perillo) Date: Thu, 09 Aug 2007 10:01:58 +0200 Subject: [Pydotorg-www] function trim_nav_links in build.py In-Reply-To: <20070809020132.GA221@andrew-kuchlings-computer.local> References: <46B9E59B.5060508@libero.it> <20070809020132.GA221@andrew-kuchlings-computer.local> Message-ID: <46BAC9F6.4050406@libero.it> A.M. Kuchling ha scritto: > On Wed, Aug 08, 2007 at 05:47:39PM +0200, Manlio Perillo wrote: >> I have another question: what's the use of the function trim_nav_links? > > It takes a header like this: > > Quick-links: > Quick Links (2.5.1) /download/releases/2.5.1 > Documentation {Manuals for Latest Stable Release} http://docs.python.org > / > Windows Installer {Easy Installer of Python under Windows} /ftp/python/2 > .5.1/python-2.5.1.msi > > and strips out the leading whitespace. Perhaps parseIndentedList() > handles the leading whitespace now, and trim_nav_links() is no longer > needed. (The top content.ht file is the only one with the headers > that use this.) > Right. parseIndentedList handles this, since it computes the indentation of the first non empty line. By the way, I have greatly simplified parseIndentedList to return a result that can be used directly by parse_nav_list (I return a list of lists, not a tuple of tuples, but it seems to work). > --amk P.S. I have sent a message with a bug report about the main navigation list, but it seems that it got lost. Since I have no access to the Trac system, I'll report it here: --------- I have found two strange problems with the main navigation in: http://www.python.org/doc/versions/ http://www.python.org/doc/av/ In the first page, appears a 5-Minutes with Python sub menu, under Audio/Visual Talks It should not stay here, and the url is wrong: http://www.python.org/doc/versions/av/5minutes/ In the second page, does not appear the sub menu with 5-Minutes with Python 5minutes/ Slideshow collections slideshows/ Thanks Manlio Perillo From manlio_perillo at libero.it Sat Aug 11 11:45:41 2007 From: manlio_perillo at libero.it (Manlio Perillo) Date: Sat, 11 Aug 2007 11:45:41 +0200 Subject: [Pydotorg-www] text/plain PEPs conversion in reStructuredText format Message-ID: <46BD8545.2070503@libero.it> Hi. I have written a simple module for converting PEPs from text/plain format to text/x-rst format. The code is very simple, and it converts the footnotes too. At the moment the pep-000 is not yet converted, but i'm working on it. I'm planning to add this pep_parser module to the new build system I'm working on for python.it. If anyone is interested let me know. P.S: i'm having problems with the version of docutils supplied with Debian Etch. Should I just use a recent version, or pep2pyramid uses a modified version? Manlio Perillo From goodger at python.org Mon Aug 13 15:38:47 2007 From: goodger at python.org (David Goodger) Date: Mon, 13 Aug 2007 09:38:47 -0400 Subject: [Pydotorg-www] text/plain PEPs conversion in reStructuredText format In-Reply-To: <46BD8545.2070503@libero.it> References: <46BD8545.2070503@libero.it> Message-ID: <4335d2c40708130638i4f1268aax2afd391fc7f5cff0@mail.gmail.com> On 8/11/07, Manlio Perillo wrote: > I have written a simple module for converting PEPs from text/plain > format to text/x-rst format. Are you aware of pep2html.py, in the peps repository, which converts both plaintext and reST to HTML? > P.S: i'm having problems with the version of docutils supplied with > Debian Etch. > > Should I just use a recent version, or pep2pyramid uses a > modified version? The python.org build system currently uses Docutils revision 4447. -- David Goodger From manlio_perillo at libero.it Mon Aug 13 16:07:21 2007 From: manlio_perillo at libero.it (Manlio Perillo) Date: Mon, 13 Aug 2007 16:07:21 +0200 Subject: [Pydotorg-www] text/plain PEPs conversion in reStructuredText format In-Reply-To: <4335d2c40708130638i4f1268aax2afd391fc7f5cff0@mail.gmail.com> References: <46BD8545.2070503@libero.it> <4335d2c40708130638i4f1268aax2afd391fc7f5cff0@mail.gmail.com> Message-ID: <46C06599.80600@libero.it> David Goodger ha scritto: > On 8/11/07, Manlio Perillo wrote: >> I have written a simple module for converting PEPs from text/plain >> format to text/x-rst format. > > Are you aware of pep2html.py, in the peps repository, which converts > both plaintext and reST to HTML? > Yes. But that script is a mess, IMHO. I think that first converting text/plain PEPs to reST can be a good solution (as an example, footnotes are not processed by pep2html). >> P.S: i'm having problems with the version of docutils supplied with >> Debian Etch. >> >> Should I just use a recent version, or pep2pyramid uses a >> modified version? > > The python.org build system currently uses Docutils revision 4447. > Yes, right. Now I'm using the version from trunk, thanks. P.S. My working on the refactoring of the new build system is almost done. I have named it "Igloo". For people interested: http://manlio-perillo.homedns.org:8080/igloo it runs on my laptop. Note that commit messages are in italian (sorry), but there is a Changelog. Now I'm writing some documentation. Regards Manlio Perillo