Best approach to create humongous amount of files

Mario R. Osorio nimbiotics at gmail.com
Thu May 21 11:54:28 EDT 2015


On Wednesday, May 20, 2015 at 2:09:59 PM UTC-4, Denis McMahon wrote:
> On Wed, 20 May 2015 17:14:15 +0530, Parul Mogra wrote:
> 
> > Hello everyone,
> > My objective is to create large amount of data files (say a million
> > *.json files), using a pre-existing template file (*.json). Each file
> > would have a unique name, possibly by incorporating time stamp
> > information. The files have to be generated in a folder specified.
> 
> > What is the best strategy to achieve this task, so that the files will
> > be generated in the shortest possible time? Say within an hour.
> 	
> timestamps are normally unixtime in seconds. There are 3600 seconds in an 
> hour. You'll have a hard job creating a million files with timestamp 
> based naming inside of an hour.
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com

I would use a combination of both, timestamp and a serial number, such as:
201505201425440000
201505201425440001
201505201425440002
201505201425440003
201505201425450000
201505201425450001
201505201425460000
.. and so on ..



More information about the Python-list mailing list