[New-bugs-announce] [issue23656] shutil.copyfile (or built-in open) releases sqlite3's transaction lock

[Hiroaki Miyajima]

New submission from Hiroaki Miyajima:

When copying sqlite3's database file by shutil.copyfile while holding its transaction lock, its lock is released unexpectedly.

Attached is a sample program showing this. Inside,
process-1 and process-2 just do following steps respectively:
   (1) open a sqlite dbfile
   (2) execute('begin EXCLUSIVE transaction') to acquire lock
   (3) copy the dbfile
   (4) sleep some seconds for demonstration
   (4) rollback() to release lock

if shutil.copyfile() is used as (3), process-2 gets a lock
while process-1 must hold a lock.
It seems that this happens if database file is opened with built-in open(), which is used in shutil.copyfile.

In addition,
- With os.open(), there seems no problem.
- With linux cp command, it works correct.
- Using multi-thread instead of multi-process, it works expectedly.

I think this is a bug, but does someone explain this behaviour?
Thank you.

Below is the execute example of the attached on python 2.7.9.
----------
# python Test_sqlite3_lock.py
Use <function shutil_copyfile at 0x7fe810b22500>
process-1 : Transaction lock acquired
process-2 : Transaction lock acquired
process-1 : Transaction lock released
process-2 : Transaction lock released

Use <function low_copyfile at 0x7fe810b22578>
process-1 : Transaction lock acquired
process-1 : Transaction lock released
process-2 : Transaction lock acquired
process-2 : Transaction lock released

Use <function command_copyfile at 0x7fe810b225f0>
process-1 : Transaction lock acquired
process-1 : Transaction lock released
process-2 : Transaction lock acquired
process-2 : Transaction lock released

----------

----------
files: Test_sqlite3_lock.py
messages: 238013
nosy: h-miyajima
priority: normal
severity: normal
status: open
title: shutil.copyfile (or built-in open) releases sqlite3's transaction lock
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file38467/Test_sqlite3_lock.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23656>
_______________________________________