[Tutor] Simultaneous read and write on file

[Anshu Kumar]

Hello All,

So much Thanks for your response.

Here is my actual scenario. I have a csv file and it would already be
present. I need to read and remove some rows based on some logic. I have
written earlier two separate file opens which I think was nice and clean.

actual code:

with open(file_path, 'rb') as fr:
    for row in csv.DictReader(fr):
        #Skip for those segments which are part of overridden_ids
        if row['id'] not in overriden_ids:
            segments[row['id']] = {
                'id': row['id'],
                'attrib': json.loads(row['attrib']),
                'stl': json.loads(row['stl']),
                'meta': json.loads(row['meta']),
            }
#rewriting files with deduplicated segments
with open(file_path, 'wb') as fw:
    writer = csv.UnicodeWriter(fw)
    writer.writerow(["id", "attrib", "stl", "meta"])
    for seg in segments.itervalues():
        writer.writerow([seg['id'], json.dumps(seg["attrib"]),
json.dumps(seg["stl"]), json.dumps(seg["meta"])])


I have got review comments to improve this block by having just single
file open and minimum memory usage.


Thanks and Regards,

Anshu



On Tue, Jan 19, 2016 at 11:04 AM, Cameron Simpson <cs at zip.com.au> wrote:

> On 18Jan2016 20:41, Martin A. Brown <martin at linux-ip.net> wrote:
>
>> Yes and so have I. Maybe twice in 30 years of programming. [...]
>>>>
>>>
>>> I may have done it a little more than that; I agree it is very
>>> rare. I may be biased because I was debugging exactly this last
>>> week. (Which itself is an argument against mixed rerad/write with
>>> one file - it was all my own code and I spent over a day chasing
>>> this because I was looking in the wrong spot).
>>>
>>
>> Oh yes.  Ooof.  Today's decisions are tomorrow's albatross.
>>
>
> Actually I have good reason to mix these in this instance, and now that it
> is debugged it is reliable and more efficient to boot.
>
> [...]
>
>> Tip for new players: if you do any .write()s, remember to do a
>>>>> .flush() before doing a seek or a read
>>>>>
>>>>
>>>> That's exactly my point. There are so many things you have to do
>>>> extra when working in mixed mode. Too easy to treat things like
>>>> normal mode files and get it wrong. Experts can do it and make it
>>>> work, but mostly it's just not needed.
>>>>
>>>
>>> Yes. You're write - for simplicity and reliability two distinct
>>> open file instances are much easier.
>>>
>>
>> Yes, he's write [sic].  He writes a bunch!  ;)
>>
>
> Alas, I have a tendency to substitute homophones, or near homophones, when
> typing in a hurry. You'll see this in a bunch of my messages. More
> annoyingly, some are only visible when I reread a posted message instead of
> when I was proofreading prior to send.
>
> [Homonyms mess me up when I'm typing, all sew.]
>>
>
> Homonyms too.
>
> Cheers,
> Cameron Simpson <cs at zip.com.au>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>