[Tutor] implementing a table data structure in python?

[Kent Johnson]

Thomas Clive Richards wrote:
> Hi,
> 
> I'm trying to implement a reasonably efficient table data structure, but ith a 
> few quirks...
> 
> The application I'm building is a CGI program which allows users to view a 
> large table of data. In addition, users can restrict the data shown. Some 
> sample data might look like this: (note you'll need a fixed width font for 
> this)
> 
> D       x    y    z   val   q
> -------------------------------------
> 1    |  1    2    3   23    0.1
> 13   |  2    3    4   24    0.15
> 24   |  1    1    2   12    0.2
> 45   |  2    3    5   1     0.12
> 116  |  6    2    7   27    0.18
> 211  |  9    4    6   28    0.45
> 
> Actually, the real data I'm using is very different - but this is easy to 
> type :)
> 
> The entire table is stored in a plain text file, and could contain up to 500 
> KB of data.
> 
> Every time my CGI script is run, it must read this file from disk, and return 
> a 2D array of values; However, I need to be able to restrict the columns 
> shown.
> 
> For example, user A might not want to see the D & val columns - I need a way 
> of trimming these easily..
> 
> 
> Normally I'd use a database like mysql, postgreSQL, or even SQLite for this 
> sort of application, but the server I'm working on (which is outside my 
> control) does not have any of these installed.

You could use a pure Python database such as gadfly <http://gadfly.sourceforge.net/> or KirbyBase 
<http://www.netpromi.com/kirbybase.html>. But for your needs it may be overkill.

> The CGI will not see heavy use, and I'm not very worried about race conditions 
> or collisions...
> 
> 
> Does anyone have any advice for me? implementing this as a simple 2D array (or 
> list of lists) seems a little silly - surely there's a better way?

A list of lists is the simple way to implement a 2D array and not silly at all. But really you only 
need one line at a time. I would try something like

columns = [1,3,4]  # Column numbers to show, 0-based

f = open('mydata.txt')
for line in f:
   data = line.split()  # Assuming your data is tab- or space-delimited
   data = [ item for i, item in data if i in columns ]  # Filter out unwanted columns
   print data  # Output one row of text - put your formatting here

There are probably more efficient ways to extract the data but this should get you started.


> Actually, python bindings to the STL (or something similar) would be useful in 
> many situations....

It's been a long time since I've used STL, but I think Python lists and dicts provide similar 
functionality to STL containers with *much* simpler syntax. And many of the STL algorithms can 
probably be found scattered about the Python standard library. Let us know specifically what you are 
missing and we can show you the Python way.

If you truly think a list of lists is 'silly' I suspect you are missing out on a lot of Python's power.

Kent

> 
> 
> 
> Thanks,