[issue31993] pickle.dump allocates unnecessary temporary bytes / str

[Olivier Grisel]

Olivier Grisel <olivier.grisel at ensta.org> added the comment:

More benchmarks with the unix time command:

```

(py37) ogrisel at ici:~/code/cpython$ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
(py37) ogrisel at ici:~/code/cpython$ time python ~/tmp/large_pickle_dump.py --use-pypickle
Allocating source data...
=> peak memory usage: 2.014 GB
Dumping to disk...
done in 10.677s
=> peak memory usage: 5.936 GB

real	0m11.068s
user	0m0.940s
sys	0m5.204s
(py37) ogrisel at ici:~/code/cpython$ time python ~/tmp/large_pickle_dump.py --use-pypickle
Allocating source data...
=> peak memory usage: 2.014 GB
Dumping to disk...
done in 5.089s
=> peak memory usage: 5.978 GB

real	0m5.367s
user	0m0.840s
sys	0m4.660s
(py37) ogrisel at ici:~/code/cpython$ git checkout issue-31993-pypickle-dump-mem-optim 
Switched to branch 'issue-31993-pypickle-dump-mem-optim'
(py37) ogrisel at ici:~/code/cpython$ time python ~/tmp/large_pickle_dump.py --use-pypickle
Allocating source data...
=> peak memory usage: 2.014 GB
Dumping to disk...
done in 6.974s
=> peak memory usage: 2.014 GB

real	0m7.300s
user	0m0.368s
sys	0m4.640s
(py37) ogrisel at ici:~/code/cpython$ time python ~/tmp/large_pickle_dump.py --use-pypickle
Allocating source data...
=> peak memory usage: 2.014 GB
Dumping to disk...
done in 10.873s
=> peak memory usage: 2.014 GB

real	0m11.178s
user	0m0.324s
sys	0m5.100s
(py37) ogrisel at ici:~/code/cpython$ time python ~/tmp/large_pickle_dump.py --use-pypickle
Allocating source data...
=> peak memory usage: 2.014 GB
Dumping to disk...
done in 4.233s
=> peak memory usage: 2.014 GB

real	0m4.574s
user	0m0.396s
sys	0m4.368s
```

User time is always better in the PR than on master but is also much slower than system time (disk access) in any case. System time is much less deterministic.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue31993>
_______________________________________