A limit to writing to a file from a loop?

[Steve]

Very good points.  In short, I wrote two test programs.  
One to write 400 lines of data into a file.  
Another to read it, modify it, and write to a new file.  

With some questionable spacing in the final file, I was able to read/write 400 lines of code disproving my original suspicion. Back to the drawing board but at least I can write all the lines using my work around in the original program.  The question is why is there a difference?.  I will post again if I find something.
The plot thickens.
TTKMAWAN...


>>> Test one: Prove that there is valid input
>>> - is the code successfully reading more than '250' records?

Yes, when I close the file and reopen it, all 500 records are read/written.
I placed an additional line of data into the file being written which should have pushed an offending line of data to the 251st position. 250 lines were read/written and the original 250th line was not written.

>>> Test two: Check the actual data
>>> - is there anything strange about the '250th' record (the one that first 
>>> exhibits the problem)?

I placed an additional line of data at the beginning of file being read, the writing of said data should have pushed an offending line of data being written to the 251st position. 250 lines were read/written and the original 250th line was not written.

>>> Test three: Prove that it is not some Python<->file system issue
>>> - what happens if the first input record is written (to a 'junk file') 
>>> 250+ times?

>>> Test four: (possibly unnecessary after the above) Disprove a size issue
>>> - what happens if you output a simple/hello world string (of the same 
>>> number of bytes) instead of the incoming data, 250+ times?


Footnote:
Ultrasound Technician Asks Pregnant Woman If She’d Like to Know Baby’s Name...

-----Original Message-----
From: DL Neil <PythonList at DancesWithMice.info> 
Sent: Tuesday, February 12, 2019 2:13 PM
To: Steve <Gronicus at SGA.Ninja>; python-list at python.org
Subject: Re: A limit to writing to a file from a loop?

Steve,

On 13/02/19 7:56 AM, Steve wrote:
> My program reads from a text file (A), modifies the data, and writes to another file (B).
> This works until I reach about 300 writes and no more lines are written to file (B).
> I had to create a Counter and increment it to 250 when it gets reset.
> Upon reset, I close the file (B) being written and reopen it for append.
> Then it accepts the entire list of lines of data.
> Bizarre?
> 
>   
>              CycleCounter += 1
>              if CycleCounter > 250:
>                CycleCounter = 1
>                DateReadings.close()
>                DateReadings=open("Date-ReadingsAndDoses.txt", "a")
>                DateReadings.write("{0:15} {1:>8} {2:>8} {3:>8} {4:<2} {5:>8} {6:>8} {7:>10}".format
>                                (ThisTimeDate, ThisReading, ThisDose1, ThisSensor, ThisTrend,
>                                 ThisTS, ThisPercent, SensorNumberDay2) 
> + "\n")


Perhaps part of the transfer to email, but won't this code only output once per 250 records, cf 250 records per cycle?


Some suggestions:-

Test one: Prove that there is valid input
- is the code successfully reading more than '250' records?

Test two: Check the actual data
- is there anything strange about the '250th' record (the one that first 
exhibits the problem)?

Test three: Prove that it is not some Python<->file system issue
- what happens if the first input record is written (to a 'junk file') 
250+ times?

Test four: (possibly unnecessary after the above) Disprove a size issue
- what happens if you output a simple/hello world string (of the same 
number of bytes) instead of the incoming data, 250+ times?


Report back. If there are still problems, please show the original code 
and exception report...

-- 
Regards =dn


Footnote:
Ultrasound Technician Asks Pregnant Woman If She’d Like To Know Baby’s Name

-----Original Message-----
From: DL Neil <PythonList at DancesWithMice.info> 
Sent: Tuesday, February 12, 2019 2:13 PM
To: Steve <Gronicus at SGA.Ninja>; python-list at python.org
Subject: Re: A limit to writing to a file from a loop?

Steve,

On 13/02/19 7:56 AM, Steve wrote:
> My program reads from a text file (A), modifies the data, and writes to another file (B).
> This works until I reach about 300 writes and no more lines are written to file (B).
> I had to create a Counter and increment it to 250 when it gets reset.
> Upon reset, I close the file (B) being written and reopen it for append.
> Then it accepts the entire list of lines of data.
> Bizarre?
> 
>   
>              CycleCounter += 1
>              if CycleCounter > 250:
>                CycleCounter = 1
>                DateReadings.close()
>                DateReadings=open("Date-ReadingsAndDoses.txt", "a")
>                DateReadings.write("{0:15} {1:>8} {2:>8} {3:>8} {4:<2} {5:>8} {6:>8} {7:>10}".format
>                                (ThisTimeDate, ThisReading, ThisDose1, ThisSensor, ThisTrend,
>                                 ThisTS, ThisPercent, SensorNumberDay2) 
> + "\n")


Perhaps part of the transfer to email, but won't this code only output once per 250 records, cf 250 records per cycle?


Some suggestions:-

Test one: Prove that there is valid input
- is the code successfully reading more than '250' records?

Test two: Check the actual data
- is there anything strange about the '250th' record (the one that first 
exhibits the problem)?

Test three: Prove that it is not some Python<->file system issue
- what happens if the first input record is written (to a 'junk file') 
250+ times?

Test four: (possibly unnecessary after the above) Disprove a size issue
- what happens if you output a simple/hello world string (of the same 
number of bytes) instead of the incoming data, 250+ times?


Report back. If there are still problems, please show the original code 
and exception report...

-- 
Regards =dn