RSFile v2.1 released

Pascal Chambon pythoniks at gmail.com
Thu Jun 23 14:28:56 EDT 2016


Dear pythoneers,

I'm pleased to announce a major update of the RSFile I/O Library: 
version 2.1.

RSFile provides drop-in replacements for the classes of the*io *module, 
and for the *open()* builtin.

Its goal is to provide a cross-platform, reliable, and comprehensive 
synchronous file I/O API, with advanced
features like fine-grained opening modes, shared/exclusive file record 
locking, thread-safety, cache synchronization,
file descriptor inheritability, and handy stat getters (size, inode, 
times...).

Possible use cases for this library: concurrently writing to logs 
without ending up with garbled data,
manipulating sensitive data like disk-based databases, synchronizing 
heterogeneous producer/consumer
processes when multiprocessing semaphores aren't an option...

Unix users might particularly be interested by the workaround that this 
library provides, concerning
the weird semantic of fcntl() locks (when any descriptor to a disk file 
is closed, the process loses ALL
locks acquired on this file through any descriptor).

RSFile has been tested with py2.7 and py3.3+, on Windows/Linux systems,
and should theoretically work on other *nix systems and python 
implementations

The technical documentation of RSFile includes a comprehensive description
of concepts and gotchas encountered while developing this library, which 
could
be useful to anyone interested in getting in touch with gory file I/O 
details.

The implementation is currently pure-python, so if you need high 
performances, using standard python streams
in parallel will remain necessary.

/Why v2.1 and not v2.0 you ask ? Just some pypi constraints I wasn't 
aware of, that'll teach me not to erase a just-released version to add 
some cleanup commits to it.../


Downloads:
https://pypi.python.org/pypi/RSFile/2.1

Documentation:
http://rsfile.readthedocs.io/en/latest/

Repository:
https://github.com/pakal/rsfile

_CHANGELOG:_
* Switch from Mercurial to Git
* Remove python2.6 support and its polyfills
* Move backends and test suites inside rsfile package
* Conform rsfile to the behaviour of latest "io" module and "open" builtin
* Make rsfile work against py33, py34 and py35, by leveraging their 
stdlib test suites
* Rename "win32" to "windows" everywhere (even pywin32 extensions 
actually handle x64 system)
* Improve the I/O benchmark runner
* Cache decorated methods to boost performances
* Add support for the new "x" mode flag in rsopen()
* Fix the corner case of uninitialized streams
* Tweak the excessive verbosity of locking tests
* Handle exceptions when closing raw streams (stream is marked as closed 
anyway)
* Normalize the naming of backend modules
* Fix bugs with __getattr__() lookup forwarding
* Use C/N flags for file existence on opening (-/+ supported but deprecated)
* Automatically compare the behaviour of all possible open modes, 
between stdlib/io and rsfile
* Autogenerate equivalence matrix for file opening modes, using 
python-tabulate.
* Switch from distutils to setuptools for setup.py
* Add support for the new "opener" parameter of open() builtin
* Strengthen tests around fileno/handle reuse and duplication
* Fix bug regarding improper value of file "modification_time" on windows
* Add implicit flush() before every sync()
* Remove heavy star imports from pywin32 backend
* Roughly test sync() parameters, via performance measurements
* Rename file "uid()" to "unique_id()", to prevent confusion with "user 
id" (but an alias remains)
* Fix nasty bug where file unique_id could be None in internal registries
* Add lots of defensive assertions
* Make FileTimes repr() more friendly
* Add support for the wrapping of [non-blocking] pipes/fifos
* Reject the opening of directories properly
* Reorganize and cleanup sphinx docs
* Improve docstrings of added/updated methods/attributes
* Explain the file locking semantic better
* Update and correct typos in the "I/O Overview" article
* Document lots of corner cases: thread safety, reentrancy, sync 
parameters, file-share-delete semantic...
* Remove the now obsolete "multi_syscall_lock" (thread-safe interface 
does better)
* Integrate tests and doc building with Tox
* Fix bug with windows/ctypes backend on python3.5 (OVERLAPPED structure 
was broken)
* Add tests for the behaviour of streams after a fork()
* Add optmizations for systems without fork (no need for multiprocessing 
locks then)
* Normalize "__future__" imports and code formatting
* Review and document the exception types used
* Cleanup/DRY tons of obsolete TODOs and comments
* Better document the CAVEATS of rsfile, regarding fcntl and 
interoperability with other I/O libs
* Add standard files to the repository (readme, contributing, changelog 
etc.)
* Integrate with Travis CI
* Add some tweaks to mimick the more tolerant behaviour of python2.7 
open(),
   regarding the mixing of str and unicode
* Add script for aggregate coverage reporting
* Strengthen tests of rsopen() usage errors

regards,
Pascal Chambon


More information about the Python-announce-list mailing list