[Tutor] Example for read and readlines() (Asad)
Asad
asad.hasan2004 at gmail.com
Sun Nov 11 20:54:00 EST 2018
Hi All ,
Thanks for the reply . I am building a framework for the two error
conditions, therefore I need to read and readlines because in one only
regex is required and in other regex+ n-1 line is required to process :
#Here we are opening the file and substituting space " " for each \n
encountered
f3 = open (r"D:\QI\log.log", 'r')
string = f3.read()
string1 = f3.readlines()
regex = re.compile ( "\n" )
st = regex.sub ( " ", string )
if re.search('ERR1',st):
y=re.findall("[A-Z][a-z][a-z] [ 123][0-9]
[012][0-9]:[0-5][0-9]:[0-5][0-9] [0-9][0-9][0-9][0-9]",st)
print y
patchnumber = re.compile(r'(\d+)\/(\d+)') ======> doesnot
work it only works if I use #string = f3.read()
for j in range(len(string1)):
if re.search ( r'ERR2', string1[j] ):
print "Error line \n", string1[j - 1]
mo = patchnumber.search (string1[j-1])
a = mo.group()
print a
print os.getcwd()
break
Please advice how to proceed.
Thanks,
On Sun, Nov 11, 2018 at 10:30 PM <tutor-request at python.org> wrote:
> Send Tutor mailing list submissions to
> tutor at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://mail.python.org/mailman/listinfo/tutor
> or, via email, send a message with subject or body 'help' to
> tutor-request at python.org
>
> You can reach the person managing the list at
> tutor-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."
> Today's Topics:
>
> 1. Re: Require Python assistance (Alan Gauld)
> 2. Re: Example for read and readlines() (Alan Gauld)
> 3. Re: Example for read and readlines() (Alan Gauld)
> 4. Re: Example for read and readlines() (Asad)
> 5. Re: Example for read and readlines() (Alan Gauld)
>
>
>
> ---------- Forwarded message ----------
> From: Alan Gauld <alan.gauld at yahoo.co.uk>
> To: tutor at python.org
> Cc:
> Bcc:
> Date: Sun, 11 Nov 2018 09:53:23 +0000
> Subject: Re: [Tutor] Require Python assistance
> On 10/11/2018 18:10, Avi Gross wrote:
> > WARNING to any that care:
> >
> > As the following letter is a repeat request without any hint they read
> the earlier comments here, I did a little searching and see very much the
> same request on another forum asking how to do this in MATLAB:
>
> The OP has also repeated posted the same message to this list
> (which I rejected as moderator).
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
>
>
> ---------- Forwarded message ----------
> From: Alan Gauld <alan.gauld at yahoo.co.uk>
> To: tutor at python.org
> Cc:
> Bcc:
> Date: Sun, 11 Nov 2018 10:00:33 +0000
> Subject: Re: [Tutor] Example for read and readlines()
> On 11/11/2018 06:49, Asad wrote:
> > Hi All ,
> >
> > If I am loading a logfile what should I use from the option
> 1,2,3
> >
> > f3 = open ( r"/a/b/c/d/test/test_2814__2018_10_05_12_12_45/logA.log",
> 'r' )
> >
> > 1) should only iterate over f3
>
> This is best for processing line by line which is the most
> common way to handle files. It saves memory and allows you
> to exit early, without reading the entire file if you are
> only looking for say a single entry.
>
> for line in file:
> if terminal_Condition: break
> # process line here
>
> > 2) st = f3.read()
>
> The best solution if you want to process individual characters
> or small character groups. Also best if you want to process
> the entire file at once, for example using a regular expression
> which might span lines.
>
> > 3) st1 = f3.readlines()
>
> Mainly historical and superseded by iterating over the file.
> But sometimes useful if you need to do multiple passes over
> the lines since it only reads the file once. Very heavy
> memory footprint for big files.
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
>
>
> ---------- Forwarded message ----------
> From: Alan Gauld <alan.gauld at yahoo.co.uk>
> To: tutor at python.org
> Cc:
> Bcc:
> Date: Sun, 11 Nov 2018 10:02:40 +0000
> Subject: Re: [Tutor] Example for read and readlines()
> On 11/11/2018 09:40, Steven D'Aprano wrote:
>
> >> f3 = open ( r"/a/b/c/d/test/test_2814__2018_10_05_12_12_45/logA.log",
> 'r' )
> >
> > Don't use raw strings r"..." for pathnames.
>
> Umm, Why not?
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
>
>
> ---------- Forwarded message ----------
> From: Asad <asad.hasan2004 at gmail.com>
> To: tutor at python.org
> Cc:
> Bcc:
> Date: Sun, 11 Nov 2018 15:34:35 +0530
> Subject: Re: [Tutor] Example for read and readlines()
> Hi All,
>
> thanks for the reply so to put into context say I have a file
> logfile formatted in text lines
>
> 1) and I want to extract the start time , error number and end
> time from this logfile so in this case what should I use I guess option 1
> :
>
> with open(filename, 'r') as f:
> for line in f:
> process(line)
>
> should be fine or any other option ?
>
> 2) Another case is a text formatted logfile and I want to print (n-4)
> lines n is the line where error condition was encountered .
>
> 3) Do we need to ensure that each line in the logfile ends with \n .
>
> \n is not visible so can we verify in someway to proof EOL \n is placed
> in the file .
>
>
> Thanks,
>
>
>
>
> >
> >
> >
> >
> >
> > ---------- Forwarded message ----------
> > From: Asad <asad.hasan2004 at gmail.com>
> > To: tutor at python.org
> > Cc:
> > Bcc:
> > Date: Sun, 11 Nov 2018 12:19:36 +0530
> > Subject: [Tutor] Example for read and readlines()
> > Hi All ,
> >
> > If I am loading a logfile what should I use from the option
> 1,2,3
> >
> > f3 = open ( r"/a/b/c/d/test/test_2814__2018_10_05_12_12_45/logA.log",
> 'r' )
> >
> > 1) should only iterate over f3
> >
> > 2) st = f3.read()
> > Should iterate over st
> >
> >
> > 3) st1 = f3.readlines()
> >
> > Should iterate over st1
> >
> > How are the above options different it they are not can there be some
> > examples to describe in which situations should we each method .
> >
> >
> > Thanks,
> >
> > --
> > Asad Hasan
> > +91 9582111698
> >
> >
> >
> >
> > ---------- Forwarded message ----------
> > From: "Steven D'Aprano" <steve at pearwood.info>
> > To: tutor at python.org
> > Cc:
> > Bcc:
> > Date: Sun, 11 Nov 2018 20:40:49 +1100
> > Subject: Re: [Tutor] Example for read and readlines()
> > On Sun, Nov 11, 2018 at 12:19:36PM +0530, Asad wrote:
> > > Hi All ,
> > >
> > > If I am loading a logfile what should I use from the option
> > 1,2,3
> >
> > Depends what you want to do. I assume that the log file is formatted
> > into lines of text, so you probably want to iterate over each line.
> >
> > with open(filename, 'r') as f:
> > for line in f:
> > process(line)
> >
> > is the best idiom to use for line-by-line iteration. It only reads each
> > line as needed, not all at once, so it can handle huge files even if the
> > file is bigger than the memory you have.
> >
> >
> > > f3 = open ( r"/a/b/c/d/test/test_2814__2018_10_05_12_12_45/logA.log",
> > 'r' )
> >
> > Don't use raw strings r"..." for pathnames.
> >
> >
> > > 1) should only iterate over f3
> > >
> > > 2) st = f3.read()
> >
> > Use this if you want to iterate over the file character by character,
> > after reading the entire file into memory at once.
> >
> >
> > > 3) st1 = f3.readlines()
> >
> > Use this if you want to read all the lines into memory at once.
> >
> >
> >
> > --
> > Steve
> >
> > _______________________________________________
> > Tutor maillist - Tutor at python.org
> > https://mail.python.org/mailman/listinfo/tutor
> >
>
>
> --
> Asad Hasan
> +91 9582111698
>
>
>
>
> ---------- Forwarded message ----------
> From: Alan Gauld <alan.gauld at yahoo.co.uk>
> To: tutor at python.org
> Cc:
> Bcc:
> Date: Sun, 11 Nov 2018 14:46:18 +0000
> Subject: Re: [Tutor] Example for read and readlines()
> On 11/11/2018 10:04, Asad wrote:
>
> > 1) and I want to extract the start time , error number and end
> > time from this logfile so in this case what should I use I guess option
> 1 :
> >
> > with open(filename, 'r') as f:
> > for line in f:
> > process(line)
>
> Yes, that woyuld be the best choice in that scenario.
>
> > 2) Another case is a text formatted logfile and I want to print (n-4)
> > lines n is the line where error condition was encountered .
>
> In that case you could use readlines() if it is a small file.
> Or you could save the last 5 lines and print those each time
> you find an error line. You should probably write a function
> to save the line since it needs to move the previous lines
> up one.
>
> buffer = ['','','','','']
>
> def saveLine(line, buff):
> buff[0] = buff[1]
> buff[1] = buff[2]
> buff[2] = buff[3]
> buff[3] = buff[4]
> buff[4] = line
>
> for line in file:
> saveLine(line,buffer)
> if error_condition:
> printBuffer()
>
> readlines is simpler but stores the entire file in memory.
> The buffer saves memory but requires some extra processing
> to save/print. There are some modules for handling cyclic
> stores etc but in a simple case like this they are probably
> overkill.
>
> > 3) Do we need to ensure that each line in the logfile ends with \n .
> >
> > \n is not visible so can we verify in someway to proof EOL \n is
> placed
> > in the file .
>
> textfile lines are defined by the existence of the \n
> so both readlines() and a loop over the file will both
> read multiple lines if a \n is missing.
>
> You could use read() and a regex to check for some
> text marker and insert the newlines. This would be best
> if the whole file had them missing.
>
> If it just an occasional line then you can iterate over
> the file as usual and check each line for a missing \n and
> insert (or split) as needed.
>
> You might want to write the modified lines back to
> a new file.
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
> _______________________________________________
> Tutor maillist - Tutor at python.org
> https://mail.python.org/mailman/listinfo/tutor
>
--
Asad Hasan
+91 9582111698
More information about the Tutor
mailing list