Linq to Python

[Duncan Booth]

sturlamolden <sturlamolden at yahoo.no> wrote:

> On 25 Sep, 10:08, Duncan Booth <duncan.bo... at invalid.invalid> wrote:
> 
>> A lot of what LINQ does is already easy to do in Python, and most of
>> the rest can probably be added fairly easily, but it does provide a
>> consistent framework which may make it easier to do complex LINQ
>> statements than complex list comprehensions.
> 
> Yes, that's the word, "consistent framework". I wonder what that
> means? Do you mean Python syntax is inconsitent?

No Python's syntax is fine. The api's aren't as consistent though: 
Microsoft added a common set of extension methods which work on 
databases, xml, builtin sequences and can be easily extended to include 
other custom sequences.

As an example to filter a list in Python you'd use a list comprehension 
with an 'if', but for a database you'd probably prefer a select with a 
'where' clause so as to avoid retrieving and discarding 999999 or your 1 
million rows. The apis defined by LINQ allow that sort of optimisation 
to happen transparently, the simple list would just test each element 
but the database would run an appropriate query.

So what the 'can we have LINQ in Python' people are asking is to be able 
to write things like:

   x = (c for c in customers if c.id=='123')

and know that they aren't doing a linear search unless that is the best 
that can be done. The LINQ equivalent would be something like:

  var x = from c in customers where c.id=='123'
          select new { c.name, c.id };

which is compiled to:

  var x = customers.Where(c => c.id=='123');

and depending on the type of 'customers' the Where method can either get 
a callable function to test the condition or an expression tree which it 
can compile into another language such as SQL (C# lambdas can compile 
either to executable code or to Expression<> objects that you can 
further process).

There's an article at 
http://www.interact-sw.co.uk/iangblog/2005/09/30/expressiontrees which 
has a real example showing how:

DataContext db = new DataContext("server=.;initial catalog=northwind");
Table<Orders> orders = db.GetTable<Orders>();
Table<Customers> customers = db.GetTable<Customers>();

var q = from o in orders, c in customers
        where o.ShipCity == "London" && (o.CustomerID == c.CustomerID)
        select new { o.OrderDate, c.CompanyName, c.ContactTitle,
                     c.ContactName };

actually ends up as a single SQL query:

exec sp_executesql N'SELECT [t1].[CompanyName], [t1].[ContactName], 
[t1].[ContactTitle], [t0].[OrderDate]
FROM [Orders] AS [t0], [Customers] AS [t1]
WHERE ([t0].[ShipCity] = @p0) AND ([t0].[CustomerID] = [t1].
[CustomerID])', N'@p0 nvarchar(6)', @p0 = N'London' 


>> BTW, a minor correction: LINQ statements are closer to generators,
>> not list comprehensions. They don't actually evaluate their results
>> until you iterate over them and you can re-used the same LINQ
>> statement multiple times getting different results if the data has
>> changed. 
> 
> Python has generator expressions with the same syntax as list
> comprehensions, except you use () instead of [].
> 
It might surprise you to know that I have actually come across generator 
expressions. :^)

My wording was deliberate: LINQ queries are re-usable, so are Python's 
generators, but generator expressions are not. The comparison isn't 
exact, you have to call the generator to get an iterator whereas a LINQ 
expression gives you something which is directly iterable.

-- 
Duncan Booth http://kupuguy.blogspot.com