[Tutor] Organizing 15500 records, how?

[Alan Gauld]

"Thomas" <tavspam at gmail.com> wrote in message 
news:493b81e30612121739p352b7f7ek6f7069f1d27443b9 at mail.gmail.com...
> I'm writing a program to analyse the profiles of the 15500 users

> though about creating say 10 files to start off with that contained
> dictionaries of userid to field value. That way I'm dealing with 10 
> to
> 50 files instead of 15500.

To be honest, with that numbers you will be better using a database.
Both for storage and  search speed but also for the analysis.
SQL is designed for this kind of job.

You can read a shortish intro to using databases in my tutorial.
It uses SqlLite but is easily adapted to any database engine.
Even if your ISP doesn't support installed databases I would
still consider downloading your data into a local database
for the analysis job and writing HTML import/export functions
to deal with administering the web site. But if you can put a
database on the web site then so much the better. It will
almost certainly simplify your code and improve performance
and/or resource usage.

Alan G