[pypy-commit] benchmarks chameleon: update spitfire to master which no longer has psyco nor o4
mattip
pypy.commits at gmail.com
Tue Jan 7 08:30:45 EST 2020
Author: Matti Picus <matti.picus at gmail.com>
Branch: chameleon
Changeset: r431:24f7763db60b
Date: 2020-01-07 14:41 +0200
http://bitbucket.org/pypy/benchmarks/changeset/24f7763db60b/
Log: update spitfire to master which no longer has psyco nor o4
diff too long, truncating to 2000 out of 18070 lines
diff --git a/unladen_swallow/lib/spitfire/AUTHORS b/unladen_swallow/lib/spitfire/AUTHORS
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/AUTHORS
@@ -0,0 +1,16 @@
+# This is the official list of Spitfire authors for copyright purposes.
+# This file is distinct from the CONTRIBUTORS files.
+# See the latter for an explanation.
+#
+# Names should be added to this file as one of
+# Organization's name
+# Individual's name <email address>
+# Individual's name <email address> <additional email address>
+#
+# Please keep the list sorted.
+
+Alex Limi <limi at gmail.com>
+Benjamin Listwon <blistwon at mac.com>
+Google Inc.
+Hanno Schlichting <hanno at hannosch.eu>
+Mike Solomon <msolo at gmail.com>
diff --git a/unladen_swallow/lib/spitfire/CHANGES b/unladen_swallow/lib/spitfire/CHANGES
--- a/unladen_swallow/lib/spitfire/CHANGES
+++ b/unladen_swallow/lib/spitfire/CHANGES
@@ -1,3 +1,10 @@
+0.7.15 - 2012-02-09
+add ability to restrict searchlist access in library templates by adding
+#global declarations.
+add ability to blow up on nested #defs.
+fix escaping in i18n blocks.
+fix some library function calls.
+
0.7.14 - 2011-07-12
added "library" implementation. this allows template code to be reused
without relying on class hierarchy.
diff --git a/unladen_swallow/lib/spitfire/CONTRIBUTORS b/unladen_swallow/lib/spitfire/CONTRIBUTORS
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/CONTRIBUTORS
@@ -0,0 +1,33 @@
+# The AUTHORS file lists the copyright holders; this file
+# lists people. For example, Google employees are listed here
+# but not in AUTHORS, because Google holds the copyright.
+#
+# Names should be added to this file as one of
+# Organization's name
+# Individual's name <email address>
+# Individual's name <email address> <additional email address>
+#
+# Please keep the list sorted.
+
+Aaron Tubbs <tubbs at google.com>
+Alex Limi <limi at gmail.com>
+Alex Nicksay <nicksay at gmail.com> <nicksay at google.com>
+Andrew Braunstein <awbraunstein at google.com>
+Anuraag Agrawal <anuraag at google.com>
+Benjamin Listwon <blistwon at mac.com>
+Billy Biggs <bbiggs at google.com>
+Brian Atkinson <bca at google.com>
+Günther Noack <gnoack at google.com>
+Hanno Schlichting <hanno at hannosch.eu>
+Henrik Jonsson <hkjn at google.com>
+Jason Neufeld <jneufeld at google.com>
+Jiten Vaidya <jitendra.vaidya at gmail.com>
+John Fries <john.a.fries at gmail.com>
+Kaue Soares da Silveira <kaue at google.com>
+Kyle Comer <kylecomer at google.com>
+Mike Solomon <msolo at gmail.com>
+Neal Norwitz <nnorwitz at google.com>
+Peter Bradshaw <bradshaw at google.com>
+Shalabh Chaturvedi <shalabh.chaturvedi at gmail.com>
+Taylor Hughes <teh at google.com>
+Yining Zhao <yining at google.com>
diff --git a/unladen_swallow/lib/spitfire/LICENSE b/unladen_swallow/lib/spitfire/LICENSE
--- a/unladen_swallow/lib/spitfire/LICENSE
+++ b/unladen_swallow/lib/spitfire/LICENSE
@@ -1,26 +1,24 @@
-Copyright (c) 2007 Mike Solomon
-
-All rights reserved.
+Copyright 2007-2016 The Spitfire Authors. All Rights Reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
- * Redistributions of source code must retain the above copyright notice, this
+ * Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above copyright notice,
- this list of conditions and the following disclaimer in the documentation
+ * Redistributions in binary form must reproduce the above copyright notice,
+ this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
- * Neither the name of the project nor the names of its contributors may be
- used to endorse or promote products derived from this software without
+ * Neither the name of the project nor the names of its contributors may be
+ used to endorse or promote products derived from this software without
specific prior written permission.
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
-FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
-SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
-CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
-OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/unladen_swallow/lib/spitfire/Makefile b/unladen_swallow/lib/spitfire/Makefile
--- a/unladen_swallow/lib/spitfire/Makefile
+++ b/unladen_swallow/lib/spitfire/Makefile
@@ -1,86 +1,129 @@
-ifndef PYTHONPATH
- export PYTHONPATH = ../yapps2
+# Copyright 2007 The Spitfire Authors. All Rights Reserved.
+#
+# Use of this source code is governed by a BSD-style
+# license that can be found in the LICENSE file.
+
+# Default
+.PHONY: all
+# Build
+.PHONY: build parser extensions
+# Test
+.PHONY: test tests unit_tests no_whitespace_tests whitespace_tests function_registry_tests optimizer_tests
+# Clean up
+.PHONY: clean clean_build clean_tests
+# Format code
+.PHONY: fix lint
+
+
+PYTHON ?= python
+PIP ?= pip
+YAPF ?= yapf
+
+VIRTUALENV = $(shell $(PYTHON) -c "import sys; print(hasattr(sys, 'real_prefix') or hasattr(sys, 'base_prefix'))")
+ifeq ($(VIRTUALENV),True)
+ PIP_INSTALL_FLAGS=
+else
+ PIP_INSTALL_FLAGS=--user
endif
-ifndef YAPPS
- export YAPPS = ../yapps2/yapps2.py
-endif
+COMPILER = PYTHONPATH=. $(PYTHON) scripts/spitfire-compile
+CRUNNER = PYTHONPATH=. $(PYTHON) scripts/crunner.py
+UNITTEST = $(PYTHON) -m unittest
+YAPPS = $(PYTHON) third_party/yapps2/yapps2.py
-ifndef PYTHON
- PYTHON = $(which python)
-endif
-CRUNNER = $(PYTHON) scripts/crunner.py
-COMPILER = $(PYTHON) scripts/spitfire-compile
+all: build
-spitfire/compiler/parser.py: spitfire/compiler/parser.g
- $(YAPPS) spitfire/compiler/parser.g
+
+build: parser extensions
parser: spitfire/compiler/parser.py
-all: parser
+spitfire/compiler/parser.py: spitfire/compiler/parser.g third_party/yapps2/yapps2.py third_party/yapps2/yappsrt.py
+ $(YAPPS) spitfire/compiler/parser.g
-.PHONY : test_function_registry
-test_function_registry: parser
- $(CRUNNER) -O3 --compile --test-input tests/input/search_list_data.pye -qt tests/test-function-registry.txtx tests/i18n-7.txtx --function-registry-file tests/test-function-registry.cnf
+extensions: spitfire/runtime/_baked.so spitfire/runtime/_template.so spitfire/runtime/_udn.so
-.PHONY : no_whitespace_tests
-no_whitespace_tests: parser
+spitfire/runtime/_baked.so spitfire/runtime/_template.so spitfire/runtime/_udn.so: spitfire/runtime/_baked.c spitfire/runtime/_template.c spitfire/runtime/_udn.c
+ $(PIP) install $(PIP_INSTALL_FLAGS) --editable .
+
+
+fix:
+ @echo; echo 'Auto-formatting code...'
+ -@$(YAPF) --in-place --recursive --verify spitfire scripts
+
+lint:
+ @echo; echo 'Checking code format...'
+ @$(YAPF) --diff --recursive spitfire scripts || (st=$$?; echo 'Please run "make fix" to correct the formatting errors.'; exit $$st)
+
+
+test: tests
+
+tests: unit_tests no_whitespace_tests whitespace_tests function_registry_tests optimizer_tests clean_tests
+
+unit_tests: build
+ $(UNITTEST) discover -s spitfire -p '*_test.py'
+
+test_function_registry: build
$(call _clean_tests)
- $(COMPILER) tests/*.txt tests/*.tmpl
- $(CRUNNER) --test-input tests/input/search_list_data.pye -qt tests/*.txt tests/*.tmpl
+ $(CRUNNER) -O3 --compile --function-registry-file tests/test-function-registry.cnf tests/*.txtx
+
+no_whitespace_tests: build
+ $(call _clean_tests)
+ $(COMPILER) tests/*.txt tests/*.tmpl --compiler-stack-traces
+ $(CRUNNER) tests/*.txt tests/*.tmpl
$(COMPILER) -O1 tests/*.txt tests/*.tmpl
- $(CRUNNER) -O1 --test-input tests/input/search_list_data.pye -qt tests/*.txt tests/*.tmpl
+ $(CRUNNER) -O1 tests/*.txt tests/*.tmpl
$(COMPILER) -O2 tests/*.txt tests/*.tmpl
- $(CRUNNER) -O2 --test-input tests/input/search_list_data.pye -qt tests/*.txt tests/*.tmpl
+ $(CRUNNER) -O2 tests/*.txt tests/*.tmpl
$(COMPILER) -O3 tests/*.txt tests/*.tmpl
- $(CRUNNER) -O3 --test-input tests/input/search_list_data.pye -qt tests/*.txt tests/*.tmpl
+ $(CRUNNER) -O3 tests/*.txt tests/*.tmpl
-.PHONY : whitespace_tests
-whitespace_tests: parser
+whitespace_tests: build
$(call _clean_tests)
$(COMPILER) --preserve-optional-whitespace tests/*.txt tests/*.tmpl
- $(CRUNNER) --preserve-optional-whitespace --test-input tests/input/search_list_data.pye --test-output output-preserve-whitespace -qt tests/*.txt tests/*.tmpl
+ $(CRUNNER) --preserve-optional-whitespace --test-output tests/output-preserve-whitespace tests/*.txt tests/*.tmpl
$(COMPILER) -O1 --preserve-optional-whitespace tests/*.txt tests/*.tmpl
- $(CRUNNER) -O1 --preserve-optional-whitespace --test-input tests/input/search_list_data.pye --test-output output-preserve-whitespace -qt tests/*.txt tests/*.tmpl
+ $(CRUNNER) -O1 --preserve-optional-whitespace --test-output tests/output-preserve-whitespace tests/*.txt tests/*.tmpl
$(COMPILER) -O2 --preserve-optional-whitespace tests/*.txt tests/*.tmpl
- $(CRUNNER) -O2 --preserve-optional-whitespace --test-input tests/input/search_list_data.pye --test-output output-preserve-whitespace -qt tests/*.txt tests/*.tmpl
+ $(CRUNNER) -O2 --preserve-optional-whitespace --test-output tests/output-preserve-whitespace tests/*.txt tests/*.tmpl
$(COMPILER) -O3 --preserve-optional-whitespace tests/*.txt tests/*.tmpl
- $(CRUNNER) -O3 --preserve-optional-whitespace --test-input tests/input/search_list_data.pye --test-output output-preserve-whitespace -qt tests/*.txt tests/*.tmpl
+ $(CRUNNER) -O3 --preserve-optional-whitespace --test-output tests/output-preserve-whitespace tests/*.txt tests/*.tmpl
-.PHONY : test_opt
-test_opt: parser
+optimizer_tests: build
$(call _clean_tests)
- $(COMPILER) -O4 tests/*.txt tests/*.tmpl tests/*.o4txt
- $(CRUNNER) -O4 --test-input tests/input/search_list_data.pye -qt tests/*.txt tests/*.tmpl tests/*.o4txt
- $(COMPILER) -O4 --preserve-optional-whitespace tests/*.txt tests/*.tmpl tests/*.o4txt
- $(CRUNNER) -O4 --preserve-optional-whitespace --test-input tests/input/search_list_data.pye --test-output output-preserve-whitespace -qt tests/*.txt tests/*.tmpl tests/*.o4txt
+ $(COMPILER) -O3 tests/*.o4txt
+ $(CRUNNER) -O3 tests/*.o4txt
+xhtml_tests: build
+ $(call _clean_tests)
+ $(COMPILER) --xspt-mode tests/*.xhtml
+ $(CRUNNER) --xspt-mode --test-output tests/output-xhtml tests/*.xhtml
-.PHONY : xhtml_tests
-xhtml_tests: clean_tests parser
- $(COMPILER) --xspt-mode tests/*.xhtml
- $(CRUNNER) --xspt-mode --test-input tests/input/search_list_data.pye --test-output output-xhtml -qt tests/*.xhtml
-.PHONY : tests
-tests: no_whitespace_tests whitespace_tests test_function_registry test_opt
+clean: clean_build clean_tests
+clean_build:
+ @find spitfire -name '*.pyc' -exec rm {} \;
+ @find third_party -name '*.pyc' -exec rm {} \;
+ @find spitfire -name '*.so' -exec rm {} \;
+ @find third_party -name '*.so' -exec rm {} \;
+ @rm -f spitfire/compiler/parser.py
+ @rm -rf build
+ @if [[ $$($(PIP) show spitfire | fgrep Location: | awk '{print $$2}') == $(PWD) ]]; then \
+ $(PIP) uninstall --yes spitfire; \
+ fi
+ @rm -rf spitfire.egg-info
+
+clean_tests:
+ $(call _clean_tests)
+
+# Note: The define + call for clean_tests is required to force the target to
+# be called before every test execution instead of once before all of them.
+# The other way to do this is with recursive make calls.
define _clean_tests
@rm -f tests/*.py
@rm -f tests/*.pyc
@find tests -name '*.failed' -exec rm {} \;
@touch tests/__init__.py
endef
-
-.PHONY : clean
-clean: clean_tests
- @find . -name '*.pyc' -exec rm {} \;
- @rm -f spitfire/compiler/parser.py
- @rm -rf build
-
-.PHONY : clean_tests
-clean_tests:
- $(call _clean_tests)
-
-.PHONE : clean_all
-clean_all : clean clean_tests
diff --git a/unladen_swallow/lib/spitfire/README.md b/unladen_swallow/lib/spitfire/README.md
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/README.md
@@ -0,0 +1,80 @@
+# ![Spitfire][]
+
+[![Version][]](https://badge.fury.io/py/spitfire)
+[![Status][]](https://travis-ci.org/youtube/spitfire)
+
+
+## Introduction
+
+Spitfire is a high-performance Python template language inspired
+by [Cheetah][]. It originally started out as an experiment to
+see if techniques used in compilers were applicable to
+templates. Spitfire has been the primary template language for
+[youtube.com][] since 2008 and is used to generate
+[billions of views a day][].
+
+
+## Example
+
+```html
+<html>
+<head><title>$title</title></head>
+<body>
+ <ul>
+ #for $user in $users
+ <li><a href="$user.url">$user.name</a></li>
+ #end for
+ </ul>
+</body>
+</html>
+```
+
+
+## Getting Started
+
+Spitfire's syntax is extremely similar to Cheetah, however some
+directives and language features have been omitted. If you're
+already using Cheetah, simple templates will likely compile in
+Spitfire, and there are a couple compatibility modes to ease
+transition.
+
+
+## Performance
+
+Spitfire has a basic optimizer that can make certain operations
+much faster. Using a basic 10x1000 table generation benchmark,
+Spitfire can be faster than other template systems and compares
+very favorably to hand-coded Python (the upper limit of
+performance achievable by compiling to Python bytecode).
+This is by no means exhaustive proof that Spitfire is always
+fast, just that it can provide very high performance.
+
+```
+# Python 2.7.6 [GCC 4.8.2] on linux, 6-core Intel Xeon E5-1650 V3 @ 3.50GHz
+$ python tests/benchmarks/render_benchmark.py --compare --number 1000
+Running benchmarks 1000 times each...
+
+Cheetah template 18.76 ms
+Django template 263.94 ms
+Django template autoescaped 262.89 ms
+Jinja2 template 8.52 ms
+Jinja2 template autoescaped 18.22 ms
+Mako template 3.25 ms
+Mako template autoescaped 11.45 ms
+Python string template 29.78 ms
+Python StringIO buffer 20.92 ms
+Python cStringIO buffer 5.93 ms
+Python list concatenation 2.30 ms
+Spitfire template -O3 6.60 ms
+Spitfire template baked -O3 8.15 ms
+Spitfire template unfiltered -O3 2.17 ms
+```
+
+
+[Cheetah]: http://www.cheetahtemplate.org/
+[youtube.com]: https://www.youtube.com/
+[billions of views a day]: https://www.youtube.com/yt/press/statistics.html
+
+[Spitfire]: https://raw.githubusercontent.com/youtube/spitfire/master/doc/spitfire.png
+[Version]: https://badge.fury.io/py/spitfire.svg
+[Status]: https://secure.travis-ci.org/youtube/spitfire.svg?branch=master
diff --git a/unladen_swallow/lib/spitfire/doc/CompilerDesign.md b/unladen_swallow/lib/spitfire/doc/CompilerDesign.md
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/doc/CompilerDesign.md
@@ -0,0 +1,29 @@
+# Introduction #
+
+Design notes for the major Spitfire revisions.
+
+# 0.6.x #
+
+This is a 3-stage design (mostly).
+ 1. parse the source code (this reports errors, not always in a helpful way, but good enough). lexical and syntactic analysis are combined into one process.
+ 1. create a new intermediate tree after some 'semantic analysis'. mostly this seems to 'fatten' certain node types with some python pseudo-code. all the nodes are cloned and rebuilt. (not sure why any more)
+ 1. run over the tree again (twice in later versions) to perform a series of increasingly complex and potentially fragile optimizations. the are performed inline and modify the tree. this sort of works because the optimizer iterates over copies of nodes.
+
+Problems:
+ * optimizations are becoming increasingly complex - may require full dependency analysis for new expressions.
+ * some optimizations seem to be better suited by a new 'pass' - they don't seem to fit well into the existing model
+ * some blurring of where python is injected. 'semantic analyzer' actually does some pseudo-codegen operations.
+ * AST actually has some pseudo-codegen too - FunctionNode() generates a function header
+ * makes some optimizations more complex - one simple spitfire expression becomes a large chunk of python psuedo-code that is tricky to optimize
+
+# 0.7.x #
+
+Goals:
+ * restructure the compiler phases and simplify things
+ * enable more complex optimizations
+ * filtered placeholder caching
+ * hoisting for all types of spitfire primitives - resolve\_udn, resolve\_placeholder
+
+Specifics:
+ * remove pseudo-codegen and replace it with a richer abstract syntax
+ * use more specific nodes to control code generation
\ No newline at end of file
diff --git a/unladen_swallow/lib/spitfire/doc/SpitfireInstallationInstructions.md b/unladen_swallow/lib/spitfire/doc/SpitfireInstallationInstructions.md
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/doc/SpitfireInstallationInstructions.md
@@ -0,0 +1,32 @@
+# Quick Installation Notes #
+
+Copy this code into an file called example.spt:
+```
+<html>
+ <head>
+ <title>Spitfire example</title>
+ </head>
+ <body><p>Hello, $name!</p></body>
+</html>
+```
+
+Download spitfire into some convenient directory, then build and install it:
+```
+$ git clone https://github.com/youtube/spitfire.git spitfire-read-only
+$ cd spitfire-read-only
+$ python ./setup.py build
+$ python ./setup.py install
+```
+
+Run the compiler on your example file.
+```
+spitfire-compiler example.spt
+```
+This will produce a file called example.py. Now start up your python interpreter and try the following:
+```
+import example
+data = example.example(search_list=[{'name':"Trurl"}]).main()
+print data
+```
+
+You should see the template text, but with "name" substituted for "Trurl".
diff --git a/unladen_swallow/lib/spitfire/doc/SpitfireSearchResultsExample.md b/unladen_swallow/lib/spitfire/doc/SpitfireSearchResultsExample.md
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/doc/SpitfireSearchResultsExample.md
@@ -0,0 +1,38 @@
+This is a basic search template that I use a few hundred times a day in my desktop search tool.
+
+```
+#from spitfire.runtime.filters import escape_html
+#filter escape_html
+<html>
+ <head>
+ <title>search results - "$qstring"</title>
+ </head>
+ <body>
+ <p>
+ <form action="/search" method="get">
+ <input name="q" type="text" value="$qstring" size="50" />
+ <input name="s" type="submit" value="Search" />
+ </form>
+ </p>
+ <p style="font-size:smaller;">
+ Showing 1 - $num_results of $total_hits
+ <span style="font-size:smaller;color:grey;">(q:${query_time|format_string='%.3f'} t:$total_time)</span>
+ </p>
+ #for $result in $results
+ <p>
+ <a href="$result.file_url">$result.title</a>
+ <br/>
+ <span style="font-size:smaller;">${result.snippet|raw}</span>
+ <br/>
+ <span style="font-size:smaller; color:green;">
+ <a style="color:green;" href="$result.file_path">$result.short_path</a>
+ -
+ <a title="text size: $result.text_size">$result.size</a>
+ -
+ $result.time_modified
+ </span>
+ </p>
+ #end for
+ </body>
+</html>
+```
\ No newline at end of file
diff --git a/unladen_swallow/lib/spitfire/doc/SpitfireVsCheetah.md b/unladen_swallow/lib/spitfire/doc/SpitfireVsCheetah.md
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/doc/SpitfireVsCheetah.md
@@ -0,0 +1,102 @@
+# Introduction #
+
+Spitfire is syntactically similar to Cheetah. In fact, most basic Cheetah templates should compile with Spitfire with very minor (potentially zero) modifications.
+
+There are a few notable places where things diverge and some of this is up for debate. Most of the differences stem from the basic philosophy that Spitfire is only for rendering text and logic should be minimal.
+
+# Notable Differences #
+
+### Passing keyword arguments is different. ###
+
+We do not allow whitespace around the '=' and the keyword name itself is not a placeholder - so no '$' is allowed.
+
+You need to do:
+```
+$function(keyword='value')
+```
+
+Instead of:
+```
+$function($keyword = 'value')
+```
+
+You might see this error if you have extra whitespace:
+```
+Trying to find CLOSE_PAREN on line 15:
+> $function(keyword = 'value')
+> ^
+```
+You might see this error if you have an extra dollar sign. Note that the error message is correct, but the error
+caret is slightly off.
+```
+keyword arg cant be complex expression: PlaceholderNode keyword on line 15:
+> $function($keyword='value')
+> ^
+```
+
+### Spitfire does not do autocalling. ###
+
+If you want to call a function, you must explicitly do so. This is the opposite of Cheetah's default behavior. The default filter in Spitfire will replace references to function objects with empty strings.
+
+### Spitfire does not allow slicing of arrays. ###
+
+```
+#for $user in $recent_users[:10]
+...
+#end for
+```
+
+This is legal in Cheetah, but not Spitfire. The rationale is that the servlet or controller code should be doing this. Since the template and servlet are disjoint, slicing in the template is a good way of ensuring that eventually your backend is doing more work than it needs to. Over-fetching data burns resources and should be discouraged.
+
+### Multiple #extends directives are fine. ###
+
+In Cheetah, multiple inheritance is not allowed by default. In Spitfire, it's sort of encouraged. The terminology is not ideal - really you are importing the behavior of another template, which happens to be implemented behind the scenes as multiple inheritance.
+
+```
+#extends base_html ## sets up a standard html page
+#extends layout.three_column ## sets up some more html and gives you three defined areas
+
+#def column1()
+Hello column one!
+#end def
+```
+
+### Missing Directives ###
+
+There are a number of directives that have been omitted, at least for the time being. The main motivation is that in more than three years of working with Cheetah, I've never used most of these myself. When I see them used, it's usually some hack, not something well thought out - with a few notable exceptions.
+ * #assert
+ * #breakpoint
+ * #cache
+ * #compiler-setttings
+ * #include
+ * #del
+ * #echo
+ * #encoding
+ * everything is internally handled as unicode objects and templates are always considered to be in UTF8
+ * #errorCatcher
+ * #pass
+ * #raise
+ * #repeat
+ * #set global
+ * #silent
+ * #stop
+ * #try ... #except ... #finally ... #end try
+ * #unless
+ * #while
+
+### Placeholders and filters should be easier to use in the end. ###
+
+These are handled a little differently internally to avoid issues like double-escaping. The default filter in Spitfire basically allows int/long, float, str and unicode objects through. Anything else is replaced with empty string when the output is written.
+
+ * `$hasVar` and `$getVar` are replaced by `$has_var` and `$get_var` - this is slightly an annoyance and there's not a fantastic reason for it, other than internal naming consistency
+ * trivial placeholders should be treated identically
+ * extended placeholders (those wrapped in braces) have other properties
+ * `${placeholder_expression}`
+ * `${placeholder_expression|filter=html_escape}` - override the filter on this instance
+ * `${placeholder_expression|raw}` - don't filter at all
+ * `${placeholder_expression|format_string='%3.3f'}` - use a format string other than `'%s'`
+
+
+### Filters are implemented differently. ###
+
+The API is very simple, so it's probably just easiest to look at the code.
\ No newline at end of file
diff --git a/unladen_swallow/lib/spitfire/doc/ToFix.md b/unladen_swallow/lib/spitfire/doc/ToFix.md
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/doc/ToFix.md
@@ -0,0 +1,17 @@
+# Usability #
+
+ * Support line continuations - all function arguments shouldn't need to be on the same line.
+
+ * Reassigning a name with `#set` should raise a compiler error if it will get ignored anyway.
+
+ * Support `not in`.
+
+
+# Internationalization #
+
+ * Support placeholder evaluation in $i18n() macros.
+
+ * Check in memtable (previously called xle).
+
+ * Build two implementations of the i18n macros. One using memtable and one using gettext.
+
diff --git a/unladen_swallow/lib/spitfire/doc/spitfire.png b/unladen_swallow/lib/spitfire/doc/spitfire.png
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..ca1fed560d49ad4ae571c6066faa03b6b48613f0
GIT binary patch
[cut]
diff --git a/unladen_swallow/lib/spitfire/scripts/crunner.py b/unladen_swallow/lib/spitfire/scripts/crunner.py
--- a/unladen_swallow/lib/spitfire/scripts/crunner.py
+++ b/unladen_swallow/lib/spitfire/scripts/crunner.py
@@ -1,217 +1,320 @@
#!/usr/bin/env python
+# Copyright 2007 The Spitfire Authors. All Rights Reserved.
+#
+# Use of this source code is governed by a BSD-style
+# license that can be found in the LICENSE file.
+
+from __future__ import print_function
+#from future import standard_library
+#standard_library.install_aliases()
+#from builtins import str
+#from builtins import object
import copy
-import imp
import logging
+import optparse
import os.path
import sys
+import time
import traceback
-from pprint import pprint
+import io as StringIO
-import spitfire.compiler.parser
-import spitfire.compiler.scanner
-import spitfire.compiler.analyzer
-import spitfire.compiler.optimizer
-import spitfire.compiler.util
-import spitfire.runtime.runner
-import spitfire.runtime.udn
-
-from spitfire.compiler import analyzer
-from spitfire.compiler.visitor import print_tree
-from spitfire.compiler.util import Compiler
+from spitfire.compiler import compiler
+from spitfire.compiler import options
+from spitfire.compiler import util
+from spitfire.compiler import visitor
+from spitfire import runtime
+from spitfire.runtime import runner
+from spitfire.runtime import udn
# this class let's me check if placeholder caching is working properly by
# tracking the number of accesses for a single key
class ResolveCounter(object):
- def __init__(self):
- self._dict = {}
- @property
- def resolve_x(self):
- return self._get_item('resolve_x')
+ def __init__(self):
+ self._dict = {}
- @property
- def resolve_y(self):
- return self._get_item('resolve_y')
-
- def _get_item(self, key):
- if key in self._dict:
- self._dict[key] += 1
- else:
- self._dict[key] = 1
- return '%s%s' % (key, self._dict[key])
+ @property
+ def resolve_x(self):
+ return self._get_item('resolve_x')
- def __contains__(self, key):
- return key.startswith('resolve')
-
- def __getitem__(self, key):
- if not key.startswith('resolve'):
- raise KeyError(key)
- return self._get_item(key)
-
- def __getattr__(self, key):
- if not key.startswith('resolve'):
- raise AttributeError(key)
- return self._get_item(key)
+ @property
+ def resolve_y(self):
+ return self._get_item('resolve_y')
+ def _get_item(self, key):
+ if key in self._dict:
+ self._dict[key] += 1
+ else:
+ self._dict[key] = 1
+ return '%s%s' % (key, self._dict[key])
-sys_modules = sys.modules.keys()
+ def __contains__(self, key):
+ return key.startswith('resolve')
+
+ def __getitem__(self, key):
+ if not key.startswith('resolve'):
+ raise KeyError(key)
+ return self._get_item(key)
+
+ def __getattr__(self, key):
+ if not key.startswith('resolve'):
+ raise AttributeError(key)
+ return self._get_item(key)
+
+
+sys_modules = list(sys.modules.keys())
+
+
def reset_sys_modules():
- for key in sys.modules.keys():
- if key not in sys_modules:
- del sys.modules[key]
+ for key in list(sys.modules.keys()):
+ if key not in sys_modules:
+ del sys.modules[key]
+
class TestRunner(object):
- def __init__(self, compiler, options):
- self.compiler = compiler
- self.options = options
- if options.test_input:
- self._search_list = [
- spitfire.runtime.runner.load_search_list(options.test_input),
- {'tier1': {'tier2': ResolveCounter()}},
- {'nest': ResolveCounter()},
- ResolveCounter(),
+
+ def __init__(self, spt_compiler, spt_options, spt_files):
+ self.compiler = spt_compiler
+ self.options = spt_options
+ self.files = spt_files
+ self._search_list = [
+ {'tier1': {'tier2': ResolveCounter()}},
+ {'nest': ResolveCounter()},
+ ResolveCounter(),
]
- else:
- self._search_list = []
+ if self.options.test_input:
+ self._search_list.append(runner.load_search_list(
+ self.options.test_input))
+ self.buffer = StringIO.StringIO()
+ self.start_time = 0
+ self.finish_time = 0
+ self.num_tests_run = 0
+ self.num_tests_failed = 0
- # return a copy of the search_list for each set of tests
- @property
- def search_list(self):
- return copy.deepcopy(self._search_list)
+ # return a copy of the search_list for each set of tests
+ @property
+ def search_list(self):
+ return copy.deepcopy(self._search_list)
- def process_file(self, filename):
- print_lines = []
- def print_output(*args):
- if options.quiet:
- print_lines.append(args)
- else:
- print >> sys.stderr, ' '.join(args)
+ def run(self):
+ self.begin()
+ for filename in self.files:
+ self.process_file(filename)
+ self.end()
- print_output("processing", filename)
- reset_sys_modules()
- classname = spitfire.compiler.util.filename2classname(filename)
- module_name = 'tests.%s' % classname
- if not self.options.quiet or self.options.compile:
- try:
- self.compiler.compile_file(filename)
- except Exception, e:
- print >> sys.stderr, 'compile FAILED:', filename, e
- raise
- if not self.options.quiet:
- if 'parse_tree' in self.options.debug_flags:
- print "parse_tree:"
- print_tree(self.compiler._parse_tree)
- if 'analyzed_tree' in self.options.debug_flags:
- print "analyzed_tree:"
- print_tree(self.compiler._analyzed_tree)
- if 'optimized_tree' in self.options.debug_flags:
- print "optimized_tree:"
- print_tree(self.compiler._optimized_tree)
- if 'hoisted_tree' in self.options.debug_flags:
- print "hoisted_tree:"
- print_tree(self.compiler._hoisted_tree)
- if 'source_code' in self.options.debug_flags:
- print "source_code:"
- for i, line in enumerate(self.compiler._source_code.split('\n')):
- print '% 3s' % (i + 1), line
-
+ def begin(self):
+ self.start_time = time.time()
- if self.options.test:
- print_output("test", classname, '...')
- import tests
+ def end(self):
+ self.finish_time = time.time()
+ print(file=sys.stderr)
+ if self.num_tests_failed > 0:
+ sys.stderr.write(self.buffer.getvalue())
+ print('-' * 70, file=sys.stderr)
+ print('Ran %d tests in %0.3fs' % (
+ self.num_tests_run, self.finish_time - self.start_time), file=sys.stderr)
+ print(file=sys.stderr)
+ if self.num_tests_failed > 0:
+ print('FAILED (failures=%d)' % self.num_tests_failed, file=sys.stderr)
+ sys.exit(1)
+ else:
+ print('OK', file=sys.stderr)
+ sys.exit(0)
- raised_exception = False
+ def process_file(self, filename):
+ buffer = StringIO.StringIO()
+ reset_sys_modules()
- try:
- if not self.options.quiet or self.options.compile:
- template_module = spitfire.compiler.util.load_module_from_src(
- self.compiler._source_code, filename, module_name)
- else:
- template_module = spitfire.runtime.import_module_symbol(module_name)
- except Exception, e:
- print "dynamic import error"
- print filename, module_name
- raise
+ classname = util.filename2classname(filename)
+ modulename = util.filename2modulename(filename)
+ suffix = '.%d' % sys.version_info[0]
+ test_output_path = os.path.join(self.options.test_output,
+ classname + '.txt')
+ if os.path.exists(test_output_path + suffix):
+ test_output_path += suffix
- try:
- template_class = getattr(template_module, classname)
- template = template_class(search_list=self.search_list)
- current_output = template.main().encode('utf8')
- except Exception, e:
- if not self.options.quiet:
- logging.exception("test error:")
- current_output = str(e)
- raised_exception = True
+ if self.options.verbose:
+ sys.stderr.write(modulename + ' ... ')
- test_output_path = os.path.join(os.path.dirname(filename),
- self.options.test_output,
- classname + '.txt')
+ compile_failed = False
+ if self.options.debug or self.options.compile:
+ try:
+ self.compiler.compile_file(filename)
+ except Exception as e:
+ compile_failed = True
+ print('=' * 70, file=buffer)
+ print('FAIL:', modulename, '(' + filename + ')', file=buffer)
+ print('-' * 70, file=buffer)
+ traceback.print_exc(None, buffer)
+ if self.options.debug:
+ if 'parse_tree' in self.options.debug_flags:
+ print("parse_tree:", file=buffer)
+ visitor.print_tree(self.compiler._parse_tree, output=buffer)
+ if 'analyzed_tree' in self.options.debug_flags:
+ print("analyzed_tree:", file=buffer)
+ visitor.print_tree(self.compiler._analyzed_tree,
+ output=buffer)
+ if 'optimized_tree' in self.options.debug_flags:
+ print("optimized_tree:", file=buffer)
+ visitor.print_tree(self.compiler._optimized_tree,
+ output=buffer)
+ if 'hoisted_tree' in self.options.debug_flags:
+ print("hoisted_tree:", file=buffer)
+ visitor.print_tree(self.compiler._hoisted_tree,
+ output=buffer)
+ if 'source_code' in self.options.debug_flags:
+ print("source_code:", file=buffer)
+ for i, line in enumerate(self.compiler._source_code.split(
+ b'\n')):
+ print('% 3s' % (i + 1), line, file=buffer)
- if self.options.accept_test_result:
- test_file = open(test_output_path, 'w')
- test_file.write(current_output)
- test_file.close()
+ test_failed = False
+ if not self.options.skip_test:
+ import tests
- try:
- test_file = open(test_output_path)
- except IOError, e:
- print "current output:"
- print current_output
- raise
+ current_output = None
+ raised_exception = False
+ try:
+ if self.options.debug or self.options.compile:
+ template_module = util.load_module_from_src(
+ self.compiler._source_code, filename, modulename)
+ else:
+ template_module = runtime.import_module_symbol(modulename)
+ except Exception as e:
+ # An exception here means the template is unavailble; the test
+ # fails.
+ test_failed = True
+ raised_exception = True
+ current_output = str(e)
- test_output = test_file.read()
- if current_output != test_output:
- current_output_path = os.path.join(
- os.path.dirname(filename),
- self.options.test_output,
- classname + '.failed')
- f = open(current_output_path, 'w')
- f.write(current_output)
- f.close()
- for line in print_lines:
- print >> sys.stderr, ' '.join(line)
- print >> sys.stderr, "FAILED:", classname
- print >> sys.stderr, ' diff -u', test_output_path, current_output_path
- print >> sys.stderr, ' %s -t' % sys.argv[0], filename
- if raised_exception:
- print >> sys.stderr, current_output
- traceback.print_exc(raised_exception)
- else:
- print_output('OK')
+ if not test_failed:
+ try:
+ template_class = getattr(template_module, classname)
+ template = template_class(search_list=self.search_list)
+ if sys.version_info[0] < 3:
+ current_output = template.main().encode('utf8')
+ else:
+ current_output = template.main()
+ except Exception as e:
+ # An exception here doesn't meant that the test fails
+ # necessarily since libraries don't have a class; as long as
+ # the expected output matches the exception, the test
+ # passes.
+ raised_exception = True
+ current_output = str(e)
+
+ if not test_failed:
+ if self.options.test_accept_result:
+ test_file = open(test_output_path, 'w')
+ test_file.write(current_output)
+ test_file.close()
+ try:
+ test_file = open(test_output_path)
+ except IOError as e:
+ # An excpetion here means that the expected output is
+ # unavailbe; the test fails.
+ test_failed = True
+ raised_exception = True
+ current_output = str(e)
+
+ if test_failed:
+ test_output = None
+ else:
+ test_output = test_file.read()
+ test_file.close()
+ if current_output != test_output:
+ test_failed = True
+ if self.options.debug:
+ print("expected output:", file=buffer)
+ print(test_output, file=buffer)
+ print("actual output:", file=buffer)
+ print(current_output, file=buffer)
+
+ if compile_failed or test_failed:
+ self.num_tests_failed += 1
+ if self.options.verbose:
+ sys.stderr.write('FAIL\n')
+ else:
+ sys.stderr.write('F')
+ current_output_path = os.path.join(self.options.test_output,
+ classname + '.failed')
+ f = open(current_output_path, 'w')
+ f.write(current_output)
+ f.close()
+ print('=' * 70, file=buffer)
+ print('FAIL:', modulename, '(' + filename + ')', file=buffer)
+ print('-' * 70, file=buffer)
+ print('Compare expected and actual output with:', file=buffer)
+ print(' '.join([' diff -u', test_output_path,
+ current_output_path]), file=buffer)
+ print('Show debug information for the test with:', file=buffer)
+ test_cmd = [arg for arg in sys.argv if arg not in self.files]
+ if '--debug' not in test_cmd:
+ test_cmd.append('--debug')
+ test_cmd = ' '.join(test_cmd)
+ print(' ', test_cmd, filename, file=buffer)
+ if raised_exception:
+ print('-' * 70, file=buffer)
+ print(current_output, file=buffer)
+ traceback.print_exc(None, buffer)
+ print(file=buffer)
+ self.buffer.write(buffer.getvalue())
+ else:
+ if self.options.verbose:
+ sys.stderr.write('ok\n')
+ else:
+ sys.stderr.write('.')
+ self.num_tests_run += 1
if __name__ == '__main__':
- reload(sys)
- sys.setdefaultencoding('utf8')
-
- from optparse import OptionParser
- op = OptionParser()
- spitfire.compiler.util.add_common_options(op)
- op.add_option('-c', '--compile', action='store_true', default=False)
- op.add_option('-t', '--test', action='store_true', default=False)
- op.add_option('--test-input')
- op.add_option('--test-output', default='output',
- help="directory for output")
- op.add_option('--accept-test-result', action='store_true', default=False,
- help='accept current code output as correct for future tests')
- op.add_option('-q', '--quiet', action='store_true', default=False)
- op.add_option('-D', dest='debug_flags', action='store',
- default='hoisted_tree,source_code',
- help='parse_tree, analyzed_tree, optimized_tree, hoisted_tree, source_code'
- )
- op.add_option('--enable-c-accelerator', action='store_true', default=False)
- (options, args) = op.parse_args()
- setattr(options, 'debug_flags', getattr(options, 'debug_flags').split(','))
+ if sys.version_info[0] < 3:
+ reload(sys)
+ sys.setdefaultencoding('utf8')
- spitfire.runtime.udn.set_accelerator(
- options.enable_c_accelerator, enable_test_mode=True)
+ option_parser = optparse.OptionParser()
+ options.add_common_options(option_parser)
+ option_parser.add_option('-c',
+ '--compile',
+ action='store_true',
+ default=False)
+ option_parser.add_option('--skip-test', action='store_true', default=False)
+ option_parser.add_option(
+ '--test-input',
+ default='tests/input/search_list_data.pye',
+ help='input data file for templates (.pkl or eval-able file)')
+ option_parser.add_option('--test-output',
+ default='tests/output',
+ help="directory for output")
+ option_parser.add_option(
+ '--test-accept-result',
+ action='store_true',
+ default=False,
+ help='accept current code output as correct for future tests')
+ option_parser.add_option('--debug', action='store_true', default=False)
+ option_parser.add_option(
+ '--debug-flags',
+ action='store',
+ default='hoisted_tree,source_code',
+ help='parse_tree, analyzed_tree, optimized_tree, hoisted_tree, source_code')
+ option_parser.add_option('--enable-c-accelerator',
+ action='store_true',
+ default=False)
- compiler_args = Compiler.args_from_optparse(options)
- compiler = Compiler(**compiler_args)
-
- test_runner = TestRunner(compiler, options)
- for filename in args:
- test_runner.process_file(filename)
+ (spt_options, spt_args) = option_parser.parse_args()
+ if spt_options.debug:
+ spt_options.verbose = True
+ spt_options.debug_flags = getattr(spt_options, 'debug_flags').split(',')
+ else:
+ spt_options.debug_flags = []
+
+ udn.set_accelerator(spt_options.enable_c_accelerator, enable_test_mode=True)
+
+ spt_compiler_args = compiler.Compiler.args_from_optparse(spt_options)
+ spt_compiler = compiler.Compiler(**spt_compiler_args)
+
+ test_runner = TestRunner(spt_compiler, spt_options, spt_args)
+ test_runner.run()
diff --git a/unladen_swallow/lib/spitfire/scripts/spitfire-compile b/unladen_swallow/lib/spitfire/scripts/spitfire-compile
--- a/unladen_swallow/lib/spitfire/scripts/spitfire-compile
+++ b/unladen_swallow/lib/spitfire/scripts/spitfire-compile
@@ -1,87 +1,67 @@
#!/usr/bin/env python
+# Copyright 2007 The Spitfire Authors. All Rights Reserved.
+#
+# Use of this source code is governed by a BSD-style
+# license that can be found in the LICENSE file.
+
+from __future__ import print_function
+
import logging
+import optparse
import os
import os.path
import sys
-import spitfire
-import spitfire.compiler.util
+from spitfire import __version__
+from spitfire.compiler import compiler
+from spitfire.compiler import options
-from spitfire.compiler import analyzer
-from spitfire.compiler.util import Compiler
+def process_file(spt_compiler, filename, options):
-def process_file(compiler, filename, options):
- def print_output(*args):
- if options.verbose:
- print >> sys.stderr, ' '.join(args)
+ def print_output(*args):
+ if options.verbose:
+ print(' '.join(args), file=sys.stderr)
- try:
- if options.output_file:
- compiler.write_file = False
- if options.output_file == '-':
- f = sys.stdout
- else:
- f = open(options.output_file, 'w')
- else:
- compiler.write_file = True
- src_code = compiler.compile_file(filename)
- if options.output_file:
- f.write(src_code)
- f.close()
- except Exception, e:
- error_msg = 'Failed processing file: %s' % filename
- if options.verbose:
- logging.exception(error_msg)
- else:
- print >> sys.stderr, error_msg
- print >> sys.stderr, e
- sys.exit(1)
+ try:
+ if options.output_file:
+ spt_compiler.write_file = False
+ if options.output_file == '-':
+ f = sys.stdout
+ else:
+ f = open(options.output_file, 'w')
+ else:
+ spt_compiler.write_file = True
+ src_code = spt_compiler.compile_file(filename)
+ if options.output_file:
+ f.write(src_code)
+ f.close()
+ except Exception as e:
+ error_msg = 'Failed processing file: %s' % filename
+ if 1 or options.verbose:
+ logging.exception(error_msg)
+ else:
+ print(error_msg, file=sys.stderr)
+ print(e, file=sys.stderr)
+ sys.exit(1)
-# selectively enable psyco on import methods
-def init_psyco(options):
- if options.x_psyco:
- try:
- import psyco
- except ImportError:
- print >> sys.stderr, 'WARNING: unable to import psyco'
- return
-
- import re
- psyco.cannotcompile(re.compile)
-
- if options.x_psyco_profile:
- psyco.log()
- psyco.profile()
- else:
- import spitfire.compiler.scanner
- psyco.bind(spitfire.compiler.scanner.SpitfireScanner.scan)
- import copy
- psyco.bind(copy.deepcopy)
- import yappsrt
- psyco.bind(yappsrt.Scanner.token)
- import spitfire.compiler.ast
- psyco.bind(spitfire.compiler.ast.NodeList.__iter__)
-
if __name__ == '__main__':
- reload(sys)
- sys.setdefaultencoding('utf8')
+ if sys.version_info[0] < 3:
+ reload(sys)
+ sys.setdefaultencoding('utf8')
- from optparse import OptionParser
- op = OptionParser()
- spitfire.compiler.util.add_common_options(op)
- (options, args) = op.parse_args()
+ option_parser = optparse.OptionParser()
+ options.add_common_options(option_parser)
+ (spt_options, spt_args) = option_parser.parse_args()
- if options.version:
- print >> sys.stderr, 'spitfire %s' % spitfire.__version__
- sys.exit(0)
-
- init_psyco(options)
+ if spt_options.version:
+ print('spitfire %s' % __version__, file=std.stderr)
+ sys.exit(0)
- compiler_args = Compiler.args_from_optparse(options)
- compiler = Compiler(**compiler_args)
-
- for filename in args:
- process_file(compiler, filename, options)
+ spt_compiler_args = compiler.Compiler.args_from_optparse(spt_options)
+ spt_compiler = compiler.Compiler(**spt_compiler_args)
+
+ for filename in spt_args:
+ process_file(spt_compiler, filename, spt_options)
diff --git a/unladen_swallow/lib/spitfire/setup.py b/unladen_swallow/lib/spitfire/setup.py
--- a/unladen_swallow/lib/spitfire/setup.py
+++ b/unladen_swallow/lib/spitfire/setup.py
@@ -1,35 +1,65 @@
-from distutils.core import setup, Extension
+# Copyright 2007 The Spitfire Authors. All Rights Reserved.
+#
+# Use of this source code is governed by a BSD-style
+# license that can be found in the LICENSE file.
+
import os.path
+import platform
+from setuptools import Extension
+from setuptools import setup
import spitfire
-setup(
- name="spitfire",
- version=spitfire.__version__,
- description="text-to-python template language",
- author=spitfire.__author__,
- author_email=spitfire.__author_email__,
- license=spitfire.__license__,
- download_url="",
- platforms=["Posix", "MacOS X", "Windows"],
- classifiers=["Development Status :: 3 - Alpha",
- "Intended Audience :: Developers",
- "License :: OSI Approved :: BSD License",
- "Programming Language :: Python",
- "Topic :: Internet :: WWW/HTTP :: Dynamic Content",
- "Topic :: Software Development :: Code Generators",
- "Topic :: Text Processing",
- ],
- packages=["spitfire",
- "spitfire.compiler",
- "spitfire.compiler.macros",
- "spitfire.runtime",
- ],
- py_modules=['yappsrt'],
- scripts=["scripts/crunner.py",
- "scripts/spitfire-compile",
- ],
- ext_modules=[Extension("spitfire.runtime._udn",
- [os.path.join("spitfire", "runtime", "_udn.c")])
- ],
- )
+NAME = 'spitfire'
+
+DESCRIPTION = 'text-to-python template language'
+
+VERSION = spitfire.__version__
+
+AUTHOR = spitfire.__author__
+
+AUTHOR_EMAIL = spitfire.__author_email__
+
+LICENSE = spitfire.__license__
+
+PLATFORMS = ['Posix', 'MacOS X', 'Windows']
+
+CLASSIFIERS = ['Development Status :: 3 - Alpha',
+ 'Intended Audience :: Developers',
+ 'License :: OSI Approved :: BSD License',
+ 'Programming Language :: Python',
+ 'Topic :: Internet :: WWW/HTTP :: Dynamic Content',
+ 'Topic :: Software Development :: Code Generators',
+ 'Topic :: Text Processing']
+
+PACKAGES = ['spitfire',
+ 'spitfire.compiler',
+ 'spitfire.compiler.macros',
+ 'spitfire.runtime']
+
+PY_MODULES = ['third_party.yapps2.yappsrt']
+
+SCRIPTS = ['scripts/crunner.py', 'scripts/spitfire-compile']
+
+EXT_MODULES = [Extension('spitfire.runtime._baked',
+ [os.path.join('spitfire', 'runtime', '_baked.c')]),
+ Extension('spitfire.runtime._template',
+ [os.path.join('spitfire', 'runtime', '_template.c')]),
+ Extension('spitfire.runtime._udn',
+ [os.path.join('spitfire', 'runtime', '_udn.c')])]
+# Disable C extensions for PyPy.
+if platform.python_implementation() == 'PyPy':
+ EXT_MODULES = None
+
+setup(name=NAME,
+ description=DESCRIPTION,
+ version=VERSION,
+ author=AUTHOR,
+ author_email=AUTHOR_EMAIL,
+ license=LICENSE,
+ platforms=PLATFORMS,
+ classifiers=CLASSIFIERS,
+ packages=PACKAGES,
+ py_modules=PY_MODULES,
+ scripts=SCRIPTS,
+ ext_modules=EXT_MODULES)
diff --git a/unladen_swallow/lib/spitfire/spitfire.egg-info/PKG-INFO b/unladen_swallow/lib/spitfire/spitfire.egg-info/PKG-INFO
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/spitfire.egg-info/PKG-INFO
@@ -0,0 +1,19 @@
+Metadata-Version: 1.1
+Name: spitfire
+Version: 0.7.15
+Summary: text-to-python template language
+Home-page: UNKNOWN
+Author: Mike Solomon
+Author-email: <mas63 @t cornell d0t edu>
+License: BSD License
+Description: UNKNOWN
+Platform: Posix
+Platform: MacOS X
+Platform: Windows
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Programming Language :: Python
+Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
+Classifier: Topic :: Software Development :: Code Generators
+Classifier: Topic :: Text Processing
diff --git a/unladen_swallow/lib/spitfire/spitfire.egg-info/SOURCES.txt b/unladen_swallow/lib/spitfire/spitfire.egg-info/SOURCES.txt
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/spitfire.egg-info/SOURCES.txt
@@ -0,0 +1,44 @@
+README.md
+setup.py
+scripts/crunner.py
+scripts/spitfire-compile
+spitfire/__init__.py
+spitfire/test_util.py
+spitfire/text.py
+spitfire.egg-info/PKG-INFO
+spitfire.egg-info/SOURCES.txt
+spitfire.egg-info/dependency_links.txt
+spitfire.egg-info/top_level.txt
+spitfire/compiler/__init__.py
+spitfire/compiler/analyzer.py
+spitfire/compiler/analyzer_test.py
+spitfire/compiler/ast.py
+spitfire/compiler/codegen.py
+spitfire/compiler/compiler.py
+spitfire/compiler/optimizer.py
+spitfire/compiler/optimizer_test.py
+spitfire/compiler/options.py
+spitfire/compiler/parser.py
+spitfire/compiler/parser_test.py
+spitfire/compiler/scanner.py
+spitfire/compiler/util.py
+spitfire/compiler/visitor.py
+spitfire/compiler/walker.py
+spitfire/compiler/xhtml2ast.py
+spitfire/compiler/macros/__init__.py
+spitfire/compiler/macros/i18n.py
+spitfire/runtime/__init__.py
+spitfire/runtime/_baked.c
+spitfire/runtime/_template.c
+spitfire/runtime/_udn.c
+spitfire/runtime/baked.py
+spitfire/runtime/baked_test.py
+spitfire/runtime/filters.py
+spitfire/runtime/repeater.py
+spitfire/runtime/runner.py
+spitfire/runtime/template.py
+spitfire/runtime/template_test.py
+spitfire/runtime/udn.py
+spitfire/runtime/udn_test.py
+third_party/yapps2/__init__.py
+third_party/yapps2/yappsrt.py
\ No newline at end of file
diff --git a/unladen_swallow/lib/spitfire/spitfire.egg-info/dependency_links.txt b/unladen_swallow/lib/spitfire/spitfire.egg-info/dependency_links.txt
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/spitfire.egg-info/dependency_links.txt
@@ -0,0 +1,1 @@
+
diff --git a/unladen_swallow/lib/spitfire/spitfire.egg-info/top_level.txt b/unladen_swallow/lib/spitfire/spitfire.egg-info/top_level.txt
new file mode 100644
--- /dev/null
+++ b/unladen_swallow/lib/spitfire/spitfire.egg-info/top_level.txt
@@ -0,0 +1,2 @@
+spitfire
+third_party
diff --git a/unladen_swallow/lib/spitfire/spitfire/__init__.py b/unladen_swallow/lib/spitfire/spitfire/__init__.py
--- a/unladen_swallow/lib/spitfire/spitfire/__init__.py
+++ b/unladen_swallow/lib/spitfire/spitfire/__init__.py
@@ -1,3 +1,8 @@
+# Copyright 2007 The Spitfire Authors. All Rights Reserved.
+#
+# Use of this source code is governed by a BSD-style
+# license that can be found in the LICENSE file.
+
__author__ = 'Mike Solomon'
__author_email__ = '<mas63 @t cornell d0t edu>'
__version__ = '0.7.15'
diff --git a/unladen_swallow/lib/spitfire/spitfire/compiler/analyzer.py b/unladen_swallow/lib/spitfire/spitfire/compiler/analyzer.py
--- a/unladen_swallow/lib/spitfire/spitfire/compiler/analyzer.py
+++ b/unladen_swallow/lib/spitfire/spitfire/compiler/analyzer.py
@@ -1,141 +1,49 @@
+# Copyright 2007 The Spitfire Authors. All Rights Reserved.
+#
+# Use of this source code is governed by a BSD-style
+# license that can be found in the LICENSE file.
+
import copy
import os.path
-from spitfire.compiler.ast import *
-from spitfire.util import normalize_whitespace
+from spitfire.compiler import ast
+from spitfire.compiler import util
+from spitfire import text
+
def tree_walker(node):
- yield node
- for n in node.child_nodes:
- for ng in tree_walker(n):
- yield ng
-
+ yield node
+ for n in node.child_nodes:
+ for ng in tree_walker(n):
+ yield ng
+
+
class SemanticAnalyzerError(Exception):
- pass
+ pass
+
class MacroError(SemanticAnalyzerError):
- pass
+ pass
+
class MacroParseError(MacroError):
- pass
-
-class AnalyzerOptions(object):
-
- def __init__(self, **kargs):
- self.debug = False
-
- self.ignore_optional_whitespace = False
-
- # adjacent text nodes become one single node
- self.collapse_adjacent_text = False
-
- # generate templates with unicode() instead of str()
- self.generate_unicode = True
-
- # runs of whitespace characters are replace with one space
- self.normalize_whitespace = False
-
- # expensive dotted notations are aliased to a local variable for faster
- # lookups: write = self.buffer.write
- self.alias_invariants = False
-
- # when a variable defined in a block later is accessed, just use the raw
- # identifier, don't incure the cost of a resolve_placeholder call since you
- # know that this local variable will always resolve first
- self.directly_access_defined_variables = False
-
- # examine the 'extends' directive to see what other methods will be
- # defined on this template - that allows use to make fast calls to template
- # methods outside of the immediate file.
- self.use_dependency_analysis = False
-
- # if directly_access_defined_variables is working 100% correctly, you can
- # compleletely ignore the local scope, as those placeholders will have been
- # resolved at compile time. there are some complex cases where there are
- # some problems, so it is disabled for now
- self.omit_local_scope_search = False
-
- # once a placeholder is resolved in a given scope, cache it in a local
- # reference for faster subsequent retrieval
- self.cache_resolved_placeholders = False
- self.cache_resolved_udn_expressions = False
- # when this is enabled, $a.b.c will cache only the result of the entire
- # expression. otherwise, each subexpression will be cached separately
- self.prefer_whole_udn_expressions = False
-
- # Throw an exception when a udn resolution fails rather than providing a
- # default value
- self.raise_udn_exceptions = False
-
- # when adding an alias, detect if the alias is loop invariant and hoist
- # right there on the spot. this has probably been superceded by
- # hoist_loop_invariant_aliases, but does have the advantage of not needing
- # another pass over the tree
- self.inline_hoist_loop_invariant_aliases = False
-
- # if an alias has been generated in a conditional scope and it is also
- # defined in the parent scope, hoist it above the conditional. this
- # requires a two-pass optimization on functions, which adds time and
- # complexity
- self.hoist_conditional_aliases = False
- self.hoist_loop_invariant_aliases = False
-
- # filtering is expensive, especially given the number of function calls
- self.cache_filtered_placeholders = False
-
- # generate functions compatible with Cheetah calling conventions
- self.cheetah_compatibility = False
- # use Cheetah NameMapper to resolve placeholders and UDN
- self.cheetah_cheats = False
-
- # the nested def doesn't make sense - unlike block, so raise an error.
- # default off for now to let people ease into it.
- self.fail_nested_defs = False
-
- self.enable_psyco = False
- self.__dict__.update(kargs)
-
- def update(self, **kargs):
- self.__dict__.update(kargs)
-
- @classmethod
- def get_help(cls):
- return ', '.join(['[no-]' + name.replace('_', '-')
- for name, value in vars(cls()).iteritems()
- if not name.startswith('__') and type(value) == bool])
-
-default_options = AnalyzerOptions()
-o1_options = copy.copy(default_options)
-o1_options.collapse_adjacent_text = True
-
-o2_options = copy.copy(o1_options)
-o2_options.alias_invariants = True
-o2_options.directly_access_defined_variables = True
-o2_options.cache_resolved_placeholders = True
-o2_options.cache_resolved_udn_expressions = True
-o2_options.inline_hoist_loop_invariant_aliases = True
-o2_options.use_dependency_analysis = True
-
-o3_options = copy.copy(o2_options)
-o3_options.inline_hoist_loop_invariant_aliases = False
-o3_options.hoist_conditional_aliases = True
-o3_options.hoist_loop_invariant_aliases = True
-o3_options.cache_filtered_placeholders = True
-
-o4_options = copy.copy(o3_options)
-o4_options.enable_psyco = True
-
-optimizer_map = {
- 0: default_options,
- 1: o1_options,
- 2: o2_options,
- 3: o3_options,
- 4: o4_options,
- }
+ pass
i18n_function_name = 'i18n'
+# This is a whitelist of nodes that are allowed at the top level in
+# template libraries.
+# TODO: Remove ast.TextNode once b/15314057 is fixed.
+_ALLOWED_LIBRARY_NODES = (ast.TextNode, ast.ImplementsNode, ast.ImportNode,
+ ast.WhitespaceNode, ast.LooseResolutionNode,
+ ast.AllowUndeclaredGlobalsNode, ast.CommentNode,
+ ast.DefNode, ast.GlobalNode)
+
+# This is a list of nodes that cannot exclusively make up the body of
+# an if or for block.
+_BLANK_NODES = (ast.WhitespaceNode, ast.CommentNode)
+
# convert the parse tree into something a bit more 'fat' and useful
# is this an AST? i'm not sure. it will be a tree of some sort
# this should simplify the codegen stage into a naive traversal
@@ -146,517 +54,743 @@
# additionally, there are some optimizations that are really more oriented at
# the parse tree, so i do them inline here. it's a bit split-brain, but it's
# seems easier.
+
+
class SemanticAnalyzer(object):
- def __init__(self, classname, parse_root, options, compiler):
- self.classname = classname
- self.parse_root = parse_root
- self.options = options
- self.compiler = compiler
- self.ast_root = None
- self.template = None
- self.strip_lines = False
-
- def get_ast(self):
- ast_node_list = self.build_ast(self.parse_root)
- if len(ast_node_list) != 1:
- raise SemanticAnalyzerError('ast must have 1 root node')
- self.ast_root = ast_node_list[0]
- return self.ast_root
- # build an AST node list from a single parse node
- # need the parent in case we are going to delete a node
- def build_ast(self, node):
- method_name = 'analyze%s' % node.__class__.__name__
- method = getattr(self, method_name, self.default_analyze_node)
- ast_node_list = method(node)
- try:
- if len(ast_node_list) != 1:
+ def __init__(self, classname, parse_root, options, compiler):
+ self.classname = classname
+ self.parse_root = parse_root
+ self.options = options
+ self.compiler = compiler
+ self.ast_root = None
+ self.template = None
+ self.strip_lines = False
+ self.uses_raw = False
+ self.base_extends_identifiers = []
+ if self.compiler.base_extends_package:
+ # this means that extends are supposed to all happen relative to
+ # some other package - this is handy for assuring all templates
+ # reference within a tree, say for localization, where each locale
+ # might have its own package
+ packages = self.compiler.base_extends_package.split('.')
+ self.base_extends_identifiers = [
+ ast.IdentifierNode(module_name) for module_name in packages
+ ]
+
+ def get_ast(self):
+ ast_node_list = self.build_ast(self.parse_root)
+ if len(ast_node_list) != 1:
+ self.compiler.error(SemanticAnalyzerError(
+ 'ast must have 1 root node'))
+ self.ast_root = ast_node_list[0]
+ return self.ast_root
+
+ # build an AST node list from a single parse node
+ # need the parent in case we are going to delete a node
+ def build_ast(self, node):
+ method_name = 'analyze%s' % node.__class__.__name__
+ method = getattr(self, method_name, self.default_analyze_node)
+ # print method_name, node.name, node
+ ast_node_list = method(node)
+ try:
+ if len(ast_node_list) != 1:
+ return ast_node_list
+ except TypeError as e:
+ self.compiler.error(SemanticAnalyzerError('method: %s, result: %s' %
+ (method, ast_node_list)))
+
+ # print '<', ast_node_list[0].name, ast_node_list[0]
return ast_node_list
- except TypeError as e:
- raise SemanticAnalyzerError('method: %s, result: %s' % (
- method, ast_node_list))
- return ast_node_list
+ def default_analyze_node(self, pnode):
+ # print "default_analyze_node", type(pnode)
+ return [pnode]
- def default_analyze_node(self, pnode):
- # print "default_analyze_node", type(pnode)
- return [pnode]
+ # some nodes just don't need analysis
+ def skip_analyze_node(self, pnode):
+ return [pnode]
- # some nodes just don't need analysis
- def skip_analyze_node(self, pnode):
- return [pnode]
- analyzeIdentifierNode = skip_analyze_node
- analyzeLiteralNode = skip_analyze_node
+ analyzeIdentifierNode = skip_analyze_node
+ analyzeLiteralNode = skip_analyze_node
- def analyzeTemplateNode(self, pnode):
- self.template = pnode.copy(copy_children=False)
- self.template.classname = self.classname
+ def analyzeTemplateNode(self, pnode):
+ self.template = pnode.copy(copy_children=False)
+ self.template.classname = self.classname
+ # Baked mode and generate_unicode are incomaptible. This is a
+ # result of SanitizedPlaceholder extending str and not unicode. A
+ # possible solution is to have two classes:
+ # SanitizedPlaceholderUnicode and SanitizedPlaceholderStr, each
+ # that extends unicode or str. Depending on the mode, it would
+ # assign SanitizedPlaceholder to be the correct class.
+ if self.options.generate_unicode and self.options.baked_mode:
+ self.compiler.error(
+ SemanticAnalyzerError(
+ 'Generate unicode is incompatible with baked mode.'),
+ pos=pnode.pos)
+ self.template.baked = self.options.baked_mode
- # Need to build a full list of template_methods before analyzing so we can
- # modify CallFunctionNodes as we walk the tree below.
- for child_node in tree_walker(pnode):
- if isinstance(child_node, DefNode) and not isinstance(child_node, MacroNode):
- if child_node.name in self.template.template_methods:
- raise SemanticAnalyzerError('Redefining #def/#block %s (duplicate def in file?)' % child_node.name)
- self.template.template_methods.add(child_node.name)
+ # Need to build a full list of template_methods before analyzing so we
+ # can modify CallFunctionNodes as we walk the tree below.
+ for child_node in tree_walker(pnode):
+ if (isinstance(child_node, ast.DefNode) and
+ not isinstance(child_node, ast.MacroNode)):
+ if child_node.name in self.template.template_methods:
+ self.compiler.error(
+ SemanticAnalyzerError(
+ 'Redefining #def/#block %s (duplicate def in file?)'
+ % (child_node.name)),
+ pos=pnode.pos)
+ self.template.template_methods.add(child_node.name)
- for pn in self.optimize_parsed_nodes(pnode.child_nodes):
- built_nodes = self.build_ast(pn)
- if built_nodes:
- self.template.main_function.extend(built_nodes)
+ for pn in self.optimize_parsed_nodes(pnode.child_nodes):
+ if self.template.library and not isinstance(pn,
+ _ALLOWED_LIBRARY_NODES):
+ self.compiler.error(
+ SemanticAnalyzerError(
+ 'All library code must be in a function.'),
+ pos=pn.pos)
+ built_nodes = self.build_ast(pn)
+ if built_nodes and not self.template.library:
+ self.template.main_function.extend(built_nodes)
- self.template.main_function.child_nodes = self.optimize_buffer_writes(
- self.template.main_function.child_nodes)
+ if not self.uses_raw and self.template.explicitly_allow_raw:
+ self.compiler.error(SemanticAnalyzerError(
+ '#allow_raw directive is not needed'))
- if self.template.extends_nodes and self.template.library:
- raise SemanticAnalyzerError("library template can't have extends.")
+ self.template.main_function.child_nodes = self.optimize_buffer_writes(
+ self.template.main_function.child_nodes)
- return [self.template]
+ if self.template.extends_nodes and self.template.library:
+ self.compiler.error(SemanticAnalyzerError(
+ "library template can't have extends."))
- def analyzeForNode(self, pnode):
- if not pnode.child_nodes:
- raise SemanticAnalyzerError("can't define an empty #for loop")
+ return [self.template]
- for_node = ForNode()
+ # Recursively grabs identifiers from a ast.TargetListNode, such as in a
+ # ast.ForNode.
+ def _getIdentifiersFromListNode(self, identifier_set, target_list_node):
+ for pn in target_list_node.child_nodes:
+ if isinstance(pn, ast.TargetNode):
+ identifier_set.add(pn.name)
+ elif isinstance(pn, ast.TargetListNode):
+ self._getIdentifiersFromListNode(identifier_set, pn)
- for pn in pnode.target_list.child_nodes:
- for_node.target_list.extend(self.build_ast(pn))
- for pn in pnode.expression_list.child_nodes:
- for_node.expression_list.extend(self.build_ast(pn))
- for pn in self.optimize_parsed_nodes(pnode.child_nodes):
- for_node.extend(self.build_ast(pn))
+ def analyzeForNode(self, pnode):
+ # If all of the children are nodes that get ignored or there are
+ # no nodes, throw an error.
+ if all([isinstance(pn, _BLANK_NODES) for pn in pnode.child_nodes]):
+ self.compiler.error(
+ SemanticAnalyzerError("can't define an empty #for loop"),
+ pos=pnode.pos)
- for_node.child_nodes = self.optimize_buffer_writes(for_node.child_nodes)
+ for_node = ast.ForNode(pos=pnode.pos)
- return [for_node]
+ # Backup original scope identifiers for analysis.
+ template_local_scope_identifiers = set(
+ self.template.local_scope_identifiers)
- def analyzeStripLinesNode(self, pnode):
- if self.strip_lines:
- raise SemanticAnalyzerError("can't nest #strip_lines")
- self.strip_lines = True
- optimized_nodes = self.optimize_parsed_nodes(pnode.child_nodes)
- new_nodes = [self.build_ast(pn) for pn in optimized_nodes]
- self.strip_lines = False
- return self.optimize_buffer_writes(new_nodes)
+ self._getIdentifiersFromListNode(self.template.local_scope_identifiers,
+ pnode.target_list)
- def analyzeGetUDNNode(self, pnode):
- expression = self.build_ast(pnode.expression)[0]
- get_udn_node = GetUDNNode(expression, pnode.name)
- return [get_udn_node]
+ for pn in pnode.target_list.child_nodes:
+ for_node.target_list.extend(self.build_ast(pn))
+ for pn in pnode.expression_list.child_nodes:
+ for_node.expression_list.extend(self.build_ast(pn))
+ for pn in self.optimize_parsed_nodes(pnode.child_nodes):
+ for_node.extend(self.build_ast(pn))
- def analyzeGetAttrNode(self, pnode):
- expression = self.build_ast(pnode.expression)[0]
- get_attr_node = GetAttrNode(expression, pnode.name)
- return [get_attr_node]
+ for_node.child_nodes = self.optimize_buffer_writes(for_node.child_nodes)
- def analyzeIfNode(self, pnode):
- if not pnode.child_nodes:
- raise SemanticAnalyzerError("can't define an empty #if block")
+ # Restore original scope identifiers after children have been analyzed.
+ self.template.local_scope_identifiers = template_local_scope_identifiers
- if_node = IfNode()
- if_node.test_expression = self.build_ast(pnode.test_expression)[0]
- for pn in self.optimize_parsed_nodes(pnode.child_nodes):
- if_node.extend(self.build_ast(pn))
- if_node.child_nodes = self.optimize_buffer_writes(if_node.child_nodes)
- for pn in self.optimize_parsed_nodes(pnode.else_.child_nodes):
- if_node.else_.extend(self.build_ast(pn))
- if_node.else_.child_nodes = self.optimize_buffer_writes(
- if_node.else_.child_nodes)
- return [if_node]
+ return [for_node]
- def analyzeFragmentNode(self, node):
- new_nodes = []
- for n in node.child_nodes:
- new_nodes.extend(self.build_ast(n))
- return new_nodes
+ def analyzeStripLinesNode(self, pnode):
+ if self.strip_lines:
+ self.compiler.error(
+ SemanticAnalyzerError("can't nest #strip_lines"),
+ pos=pnode.pos)
+ self.strip_lines = True
+ optimized_nodes = self.optimize_parsed_nodes(pnode.child_nodes)
+ new_nodes = [self.build_ast(pn) for pn in optimized_nodes]
+ self.strip_lines = False
+ return self.optimize_buffer_writes(new_nodes)
- def analyzeArgListNode(self, pnode):
- list_node = ArgListNode()
- for n in pnode:
- list_node.extend(self.build_ast(n))
- return [list_node]
+ def analyzeGetUDNNode(self, pnode):
+ children = pnode.getChildNodes()
+ if isinstance(children[0], ast.PlaceholderNode):
+ identifier = '.'.join([node.name for node in children])
+ # Some modules are trusted not to need UDN resolution.
+ if self._identifier_can_skip_UDN_resolution(identifier):
+ expr = '%s.%s' % (identifier, pnode.name)
+ return [ast.IdentifierNode(expr, pos=pnode.pos)]
- def analyzeTupleLiteralNode(self, pnode):
- tuple_node = TupleLiteralNode()
- for n in pnode.child_nodes:
- tuple_node.extend(self.build_ast(n))
- return [tuple_node]
+ expression = self.build_ast(pnode.expression)[0]
+ return [ast.GetUDNNode(expression, pnode.name, pos=pnode.pos)]
- def analyzeDictLiteralNode(self, pnode):
- dict_node = DictLiteralNode()
- for key_node, value_node in pnode.child_nodes:
- key_value = [key_node, self.build_ast(value_node)[0]]
- dict_node.child_nodes.extend([key_value])
- return [dict_node]
+ def analyzeGetAttrNode(self, pnode):
+ expression = self.build_ast(pnode.expression)[0]
+ return [ast.GetAttrNode(expression, pnode.name, pos=pnode.pos)]
- def analyzeParameterNode(self, pnode):
- param = pnode
- param.default = self.build_ast(pnode.default)[0]
- return [param]
+ def analyzeIfNode(self, pnode):
+ # If all of the children are nodes that get ignored or there are
+ # no nodes, throw an error.
+ if all([isinstance(pn, _BLANK_NODES) for pn in pnode.child_nodes]):
+ self.compiler.error(
+ SemanticAnalyzerError("can't define an empty #if block"),
+ pos=pnode.pos)
- def analyzeSliceNode(self, pnode):
- snode = pnode
- snode.expression = self.build_ast(pnode.expression)[0]
- snode.slice_expression = self.build_ast(pnode.slice_expression)[0]
- return [snode]
+ if_node = ast.IfNode(pos=pnode.pos)
+ if_node.else_.pos = pnode.else_.pos
+ if_node.test_expression = self.build_ast(pnode.test_expression)[0]
+ for pn in self.optimize_parsed_nodes(pnode.child_nodes):
+ if_node.extend(self.build_ast(pn))
+ if_node.child_nodes = self.optimize_buffer_writes(
+ if_node.child_nodes)
+ for pn in self.optimize_parsed_nodes(pnode.else_.child_nodes):
+ if_node.else_.extend(self.build_ast(pn))
+ if_node.else_.child_nodes = self.optimize_buffer_writes(
+ if_node.else_.child_nodes)
+ return [if_node]
- # FIXME: should I move this to a directive?
- def analyzeImplementsNode(self, pnode):
- if pnode.name == 'library':
- self.template.library = True
- else:
- self.template.main_function.name = pnode.name
- self.template.implements = True
- return []
+ def analyzeFragmentNode(self, node):
More information about the pypy-commit
mailing list