Long running process - how to speed up?

[Dan Stromberg]

On Sat, Feb 19, 2022 at 3:29 AM Shaozhong SHI <shishaozhong at gmail.com>
wrote:

> I have a cvs file of 932956 row and have to have time.sleep in a Python
> script.  It takes a long time to process.
>
> How can I speed up the processing?  Can I do multi-processing?
>

How are you doing it right now?

Are you using the csv module?

You might be able to use the GNU "split" command as a prelude to using the
csv module in combination with multiprocessing.  GNU split comes with
Linuxes, but I'm sure you can get it for Windows.  MacOS comes with a
rather less powerful "split" command, but it still might work for you.

You also could try Pypy3.

HTH.