shutil.copyfile is incomplete (truncated)

[Cameron Simpson]

On 11Apr2013 23:32, Rob Schneider <rmschne at gmail.com> wrote:
| > > Question: is the size of the incomplete file a round number? (Like
| > > a multiple of a decent sized power of 2>)
[...]
| Source (correct one) is 47,970 bytes. Target after copy of 45,056
| bytes.  I've tried changing what gets written to change the file
| size. It is usually this sort of difference.

45046 is exactly 11 * 4096. I'd say your I/O is using 4KB blocks,
and the last partial block (to make it up to 47970) didn't get
written (at the OS level).

Earlier you wrote:
| I have created a file in temp space, then use the function
| "shutil.copyfile(fn,loc+fname)" from "fn" to "loc+fname".
and:
| Yes, there is a close function call  before the copy is launched. No other writes.
| Does Python wait for file close command to complete before proceeding?

Please show us the exact code used to make the temp file.

I would guess the temp file has not been closed (or flushed) before
the call to copyfile.

If you're copying data to a tempfile, it will only have complete
buffers (i.e. multiples of 4096 bytes) in it until the final flush
or close.

So I'm imagining something like:

  tfp = open(tempfilename, "w")
  ... lots of tfp.write() ...
  shutil.copyfile(tempfilename, newfilename)

Note above no flush or close of tfp. So the final incomplete I/O
buffer is still in Python's memory; it hasn't been actually written
to the temp file because the buffer has not been filled, and the file
has not been closed.

Anyway, can you show us the relevant bits of code involved?

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

Processes are like potatoes.    - NCR device driver manual