From vernondcole at gmail.com Mon Aug 2 11:57:08 2010 From: vernondcole at gmail.com (Vernon Cole) Date: Mon, 2 Aug 2010 03:57:08 -0600 Subject: [DB-SIG] How can I reliably detect whether an SQL statement is a Query? Message-ID: Dear Gurus: Please give your advice up front to help me avoid making a design error. I am asking for help because: 1) I am not confident in my ability to understand Regular Expression strings. 2) I do not know much about any dialect of SQL other than Microsoft T-SQL (and sometime precious little of that.) I am ready for the next step in development of adodbapi, which is to use real ADO.NET (rather than COM ADO-db) when running on Iron Python. My research indicates that, when using ADO.NET, one must choose to call either an ExecuteReader() method, or an ExecuteNonQuery() method. I am attempting to use a lightweight db-api implementation from FePy for my pattern. It includes the following snippets of code: import re P_IS_QUERY = re.compile('^[ \r\n]*SELECT ',re.IGNORECASE) class Cursor(object): def _is_query(self, operation): '''Identify whether an operation is a query or not''' if P_IS_QUERY.match(operation): return True else: return False if self._is_query(operation): self.reader = command.ExecuteReader() self._set_description() else: command.ExecuteNonQuery() self.description = None It seems to me that this code could be confused by the substring 'SELECT' being included as part of a longer string, or in a string literal. Am reading it wrong? It also seems to me that I should be able to detect a query by the fact that the first token in the command will be either 'SELECT' or 'WITH, but would that still be true for other dialects of SQL?' I am thinking of using something like: def is_query(operation): return operation.split(' ')[0].upcase in ['SELECT','WITH'] Good idea, or Bad idea? Any comments appreciated. -- Vernon -------------- next part -------------- An HTML attachment was scrubbed... URL: From farcepest at gmail.com Mon Aug 2 15:55:24 2010 From: farcepest at gmail.com (Andy Dustman) Date: Mon, 2 Aug 2010 09:55:24 -0400 Subject: [DB-SIG] How can I reliably detect whether an SQL statement is a Query? In-Reply-To: References: Message-ID: On Mon, Aug 2, 2010 at 5:57 AM, Vernon Cole wrote: > Dear Gurus: > > Please give your advice up front to help me avoid making a design error. I > am asking for help because: > 1) I am not confident in my ability to understand Regular Expression > strings. > 2) I do not know much about any dialect of SQL other than Microsoft T-SQL > (and sometime precious little of that.) > > I am ready for the next step in development of adodbapi, which is to use > real ADO.NET (rather than COM ADO-db) when running on Iron Python. > > My research indicates that, when using ADO.NET, one must choose to call > either? an ExecuteReader() method, or an? ExecuteNonQuery() method. > > I am attempting to use a lightweight db-api implementation from FePy for my > pattern. It includes the following snippets of code: > > > import re > P_IS_QUERY = re.compile('^[ \r\n]*SELECT ',re.IGNORECASE) A slightly better expression would be ^\w*SELECT \w matches whitespace. > > class Cursor(object): > > > ??? def _is_query(self, operation): > ??????? '''Identify whether an operation is a query or not''' > ??????? if P_IS_QUERY.match(operation): > ??????????? return True > ??????? else: > ??????????? return False > > > ??????? if self._is_query(operation): > ??????????? self.reader = command.ExecuteReader() > ??????????? self._set_description() > ??????? else: > ??????????? command.ExecuteNonQuery() > ??????????? self.description = None > > > It seems to me that this code could be confused by the substring 'SELECT' > being included as part of a longer string, or in a string literal. Am > reading it wrong? The way your expression is written, it only matches SELECT at the beginning of the line (after any whitespace). > It also seems to me that I should be able to detect a query by the fact that > the first token in the command will be either 'SELECT' or 'WITH, but would > that still be true for other dialects of SQL?' > > I am thinking of using something like: > > def is_query(operation): > ??? return operation.split(' ')[0].upcase in ['SELECT','WITH'] > > > Good idea, or Bad idea? > > Any comments appreciated. What are the consequences of using ExecuteReader() when there is nothing to read? If none, i.e. you get an empty set of results, then I would say to use that all the time, and don't bother to examine your SQL at all. -- Question the answers From julians37 at googlemail.com Tue Aug 3 02:33:25 2010 From: julians37 at googlemail.com (Julian Scheid) Date: Tue, 3 Aug 2010 10:03:25 +0930 Subject: [DB-SIG] How can I reliably detect whether an SQL statement is a Query? In-Reply-To: References: Message-ID: On Mon, Aug 2, 2010 at 11:25 PM, Andy Dustman wrote: > \w matches whitespace. No, it matches alphanumeric characters and the underscore. \s matches whitespace. From Chris.Clark at ingres.com Tue Aug 3 20:32:59 2010 From: Chris.Clark at ingres.com (Chris Clark) Date: Tue, 03 Aug 2010 11:32:59 -0700 Subject: [DB-SIG] How can I reliably detect whether an SQL statement is a Query? In-Reply-To: References: Message-ID: <4C5860DB.8090709@ingres.com> Andy Dustman wrote: > On Mon, Aug 2, 2010 at 5:57 AM, Vernon Cole wrote: > >> ...1) I am not confident in my ability to understand Regular Expression >> strings. >> .... >> It seems to me that this code could be confused by the substring 'SELECT' >> being included as part of a longer string, or in a string literal. Am >> reading it wrong? >> > > The way your expression is written, it only matches SELECT at the > beginning of the line (after any whitespace). > You're both correct :-) Consider the following: --- Cut here --- CREATE VIEW AS SELECT ..... --- Cut here --- This is not a row returning statement :-) Also consider: --- Cut here --- /* This is the best piece of SQL ever! */ SELECT ..... --- Cut here --- This should be a row returning statement :-) It is possible to handle this with a regex but it would need to be slightly more complicated. Regexs are great if you are able to create them quickly as they are easy to use, but they tend to run a lot slower than a hand written piece of code (the hand written code will take longer to write though). >> It also seems to me that I should be able to detect a query by the fact that >> the first token in the command will be either 'SELECT' or 'WITH, but would >> that still be true for other dialects of SQL?' >> >> I am thinking of using something like: >> >> def is_query(operation): >> return operation.split(' ')[0].upcase in ['SELECT','WITH'] >> >> >> Good idea, or Bad idea? >> This won't handle newlines, or tabs. So in its current form I prefer the original regex. >> Any comments appreciated. >> > > What are the consequences of using ExecuteReader() when there is > nothing to read? If none, i.e. you get an empty set of results, then I > would say to use that all the time, and don't bother to examine your > SQL at all. > I think this is worth checking out, if you can avoid parsing (or partial parsing) that is likely to be a better solution (assuming this doesn't cause additional database IO which can be costly across a network). The DBMS already does this sort of work so if the driver can avoid it, that is a good thing. It may not work out but it is worth testing. If you want to play with regex I can recommend http://kodos.sourceforge.net/ it is an excellent GUI to play with regex, it can even generate Python code :-) There is a pyparsing SQL parser that you could use instead, this would handle your requirement of; "reliably detect whether an SQL statement is a Query". The regex and split above are not reliable but would probably handle 80-90% of the SQL you throw at it. Vernon - thanks for looking into this, it will be nice to have a supported/maintained IronPython dbi .NET driver. Chris From phd at phd.pp.ru Wed Aug 11 16:59:50 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Wed, 11 Aug 2010 18:59:50 +0400 Subject: [DB-SIG] SQLObject 0.13.0 Message-ID: <20100811145950.GC20946@phd.pp.ru> Hello! I'm pleased to announce version 0.13.0, the first stable release of branch 0.13 of SQLObject. What is SQLObject ================= SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject ================== Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/0.13.0 News and changes: http://sqlobject.org/News.html What's New ========== News since 0.12 --------------- Features & Interface -------------------- * SQLObject instances that don't have a per-instance connection can be pickled and unpickled. * Validators became stricter: StringCol and UnicodeCol now accept only str, unicode or an instance of a class that implements __unicode__ (but not __str__ because every object has a __str__ method); BoolCol accepts only bool or int or an instance of a class that implements __nonzero__; IntCol accepts int, long or an instance of a class that implements __int__ or __long__; FloatCol accepts float, int, long or an instance of a class that implements __float__, __int__ or __long__. * Added a connection class for rdbhost.com (commercial Postgres-over-Web service). Small Features -------------- * Added TimedeltaCol; currently it's only implemented on PostgreSQL as an INTERVAL type. * Do not pollute the base sqlmeta class to allow Style to set idName. In the case of inherited idName inherited value takes precedence; to allow Style to set idName reset inherited idName to None. * Better handling of circular dependencies in sqlobject-admin - do not include the class in the list of other classes. * Renamed db_encoding to dbEncoding in UnicodeStringValidator. * A new parameter 'sslmode' was added to PostgresConnection. * Removed SQLValidator - its attemptConvert was never called because in FormEncode it's named attempt_convert. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From info at egenix.com Thu Aug 19 18:00:07 2010 From: info at egenix.com (eGenix Team: M.-A. Lemburg) Date: Thu, 19 Aug 2010 18:00:07 +0200 Subject: [DB-SIG] ANN: eGenix mxODBC - Python ODBC Database Interface 3.1.0 Message-ID: <4C6D5507.5040507@egenix.com> ________________________________________________________________________ ANNOUNCING eGenix.com mxODBC - Python ODBC Database Interface Version 3.1.0 mxODBC is our commercially supported Python extension providing ODBC database connectivity to Python applications on Windows, Mac OS X, Unix and BSD platforms This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-mxODBC-3.1.0-GA.html ________________________________________________________________________ INTRODUCTION mxODBC provides an easy-to-use, high-performance, reliable and robust Python interface to ODBC compatible databases such as MS SQL Server, MS Access, Oracle Database, IBM DB2 and Informix , Sybase ASE and Sybase Anywhere, MySQL, PostgreSQL, SAP MaxDB and many more: http://www.egenix.com/products/python/mxODBC/ The "eGenix mxODBC - Python ODBC Database Interface" product is a commercial extension to our open-source eGenix mx Base Distribution: http://www.egenix.com/products/python/mxBase/ ________________________________________________________________________ NEWS The new mxODBC 3.1.0 release comes with many new features and supports more ODBC drivers, manager and platforms than ever before. Release Highlights ------------------ * We've added Python 2.7 support and builds for all platforms. * mxODBC 3.1 adds native support for the Windows 64-bit platforms as well as the Mac OS X 10.6 (Snow Leopard) 64-bit builds of Python. * mxODBC now fully supports the Oracle Instant Client ODBC driver. * We have updated the support for the latest IBM DB2 9.7 ODBC drivers and enhanced compatibility of mxODBC with the MS SQL Server Native Client ODBC driver on Windows and the Sybase ASE 15 ODBC drivers on Unix. * mxODBC 3.1 adds support for large-scale data warehouse databases Netezza and Teradata. * In addition to the Windows, Mac OS X, iODBC and unixODBC ODBC driver managers, we now also include support for the DataDirect ODBC manager. * The 64-bit support on Unix platforms was updated to support the new unixODBC 2.3.0 version. * We've improved the documentation on how to connect to various popular databases and now include many tips & tricks for each database/driver. * The Python 2.7 memoryview object is now supported as binary data container. * We have simplified handling of database warnings using a new customizable .warningformat attribute. * The catalog methods now accept both Unicode and 8-bit strings as parameters. * You can now select whether to use ANSI (8-bit) or Unicode ODBC APIs in the ODBC drivers, removing unnecessary data conversions and enhancing ODBC driver compatibility. For the full set of changes please check the mxODBC change log: http://www.egenix.com/products/python/mxODBC/changelog.html Feature Highlights ------------------ * Python Database API 2.0 Compliance: the mxODBC API is fully Python DB-API 2.0 compatible and implements a large number of powerful extensions. * Support for all popular ODBC Drivers: mxODBC includes adjustments and work-arounds to support MS SQL Server Native Client, MS SQL Server ODBC Driver, FreeTDS ODBC Driver, Oracle Instant Client ODBC Driver, IBM DB2 ODBC Driver, Sybase ASE ODBC Driver, Netezza ODBC Driver, Teradata ODBC Driver, PostgreSQL ODBC Driver, MySQL ODBC Driver, .MaxDB ODBC Driver as well as the ODBC driver sets of EasySoft, DataDirect, OpenLink, Actual Technologies. * Support for all popular ODBC Driver Managers: mxODBC comes with subpackages for the native Windows and Mac OS X ODBC managers, as well as the ODBC managers unixODBC, iODBC and DataDirect, which are commonly used on Unix systems. * Stable, robust and reliable:the mxODBC API has been in active production use for more than 10 years. * Full Python Support: mxODBC works with Python 2.3, 2.4, 2.5, 2.6 and 2.7. * Full 64-bit Support: mxODBC runs on the following 64-bit platforms: Windows, Linux, FreeBSD and Mac OS X. For the full set of features mxODBC has to offer, please see: http://www.egenix.com/products/python/mxODBC/#Features New mxODBC Editions ------------------- Due to popular demand, we have extended the set of available mxODBC editions and included a new low-cost standard edition. mxODBC is now available in thesethree editions: * The low-cost Standard Edition which provides data connectivity to a selected set of database backends. * The Professional Edition, which gives full access to all mxODBC features. * The Product Development Edition, which allows including mxODBC in applications you develop. At the same time we have simplified our license terms to clarify the situation on multi-core and virtual machines. In most cases, you no longer need to purchase more than one license per processor or virtual machine, scaling down the overall license costs significantly compared to earlier mxODBC releases. For a complete overview of the new editions, please see the product page. http://www.egenix.com/products/python/mxODBC/#mxODBCEditions ________________________________________________________________________ DOWNLOADS The download archives and instructions for installing the package can be found at: http://www.egenix.com/products/python/mxODBC/ In order to use the eGenix mxODBC package you will first need to install the eGenix mx Base package: http://www.egenix.com/products/python/mxBase/ ________________________________________________________________________ UPGRADING Users are encouraged to upgrade to this latest mxODBC release to benefit from the new features and updated ODBC driver support. We have taken special care, not to introduce backwards incompatible changes, making the upgrade experience as smooth as possible. Customers who have purchased mxODBC 2.0, 2.1 or 3.0 licenses can upgrade their licenses using the mxODBC Professional Edition Upgrade License. If you want to try the new release before purchace, you can request 30-day evaluation licenses by visiting our web-site http://www.egenix.com/products/python/mxODBC/#Evaluation or by writing to sales at egenix.com, stating your name (or the name of the company) and the number of eval licenses that you need. _______________________________________________________________________ SUPPORT Commercial support for this product is available from eGenix.com. Please see http://www.egenix.com/services/support/ for details about our support offerings. _______________________________________________________________________ INFORMATION About Python (http://www.python.org/): Python is an object-oriented Open Source programming language which runs on all modern platforms. By integrating ease-of-use, clarity in coding, enterprise application connectivity and rapid application design, Python establishes an ideal programming platform for today's IT challenges. About eGenix (http://www.egenix.com/): eGenix is a software project, consulting and product company focusing on expert services and professional quality products for companies, Python users and developers. Enjoy, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 19 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/