From suryak at live.com  Sun Jul  1 14:34:50 2012
From: suryak at live.com (Surya K)
Date: Sun, 1 Jul 2012 18:04:50 +0530
Subject: [Tutor] How to authorize my application to use Google API
Message-ID: <SNT130-W24E0285770A36846C7353AA4EB0@phx.gbl>


I am building a Facebook application using Django where I am using Blogger API. So, all I want is to just read the data from a public blog (my blog).
I tried to read the documentation and found that we have 3 types of authentication mechanisms (ClientLogin, OAuth, AuthSub Proxy). As I can't directly put login credentials in the application (insecure), I have only one option to use. i.e., OAuth.
The tricky part of my project is, the "users"(facebook users) should be able to access my data without providing any authorization information to Google. However, in the eyes of Google - OAuth, as I am pulling data from blog to display to public, I am considered as "user" who needs to authorize the application when ever someone opens the app on facebook, which is completely absurd...
So, there might be two ways to do this:
I should be able to pull data from blog without any authorization of user..How can I do it? If not
I should be able authorize my application internally for OAuth so that facebook users can access data without providing any information to google.How can I do this?? If possible, is this a secure way?

I am quite new to OAuth, Google API.. so, please suggest the best and secure way.. 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120701/115783ce/attachment.html>

From scarolan at gmail.com  Sun Jul  1 22:49:20 2012
From: scarolan at gmail.com (Sean Carolan)
Date: Sun, 1 Jul 2012 15:49:20 -0500
Subject: [Tutor] Python XML for newbie
Message-ID: <CALrDYWMJ08kCUBv4tP9EyeKEJQ3Kp7ZDtgipSR6UCNe+tJcscg@mail.gmail.com>

I'm trying to parse some XML data (Book titles, ISBN numbers and
descriptions) with Python.  Is there a *simple* way to import an XML
file into a dictionary, list, or other usable data structure?  I've
poked around with minidom, elementtree, and "untangle" but am not
really understanding how they are supposed to work.

Here's some sample data:

<xml>
<fields>
<field>
<name>Title</name>
<id>2</id>
<count>1</count>
<type>11</type>
<search>true</search>
<hasnumber>false</hasnumber>
</field>

...several more fields, then there are the items...

</fields>
<items>
<item>
<id>108</id>
<data>
<datum>
<index>1</index>
<field>2</field>
<value>Essential System Administration</value>
</datum>

For starters, I'd like to be able to just print out the list of titles
in the XML file, using the correct XML parser.  I don't mind doing
some research or reading on my own, but the official documentation
seems terribly confusing to me.

http://docs.python.org/library/xml.dom.minidom.html

Any pointers?

From alan.gauld at btinternet.com  Mon Jul  2 01:10:27 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 02 Jul 2012 00:10:27 +0100
Subject: [Tutor] Python XML for newbie
In-Reply-To: <CALrDYWMJ08kCUBv4tP9EyeKEJQ3Kp7ZDtgipSR6UCNe+tJcscg@mail.gmail.com>
References: <CALrDYWMJ08kCUBv4tP9EyeKEJQ3Kp7ZDtgipSR6UCNe+tJcscg@mail.gmail.com>
Message-ID: <jsqld3$9b6$1@dough.gmane.org>

On 01/07/12 21:49, Sean Carolan wrote:
> ... Is there a *simple* way to import an XML
> file into a dictionary, list, or other usable data structure?

The simplest way using the standard library tools is (IMHO)
elementtree. minidom is a complex beast by comparison,
especially if you are not intimately familiar with
your XML structure.

However hthere are some other add-in packages that are
allegedly much easier still.

But I'd start with the etree tutorial (of which
there are many variations on the web):

The original:
http://effbot.org/zone/element-index.htm

My preference:
http://infohost.nmt.edu/tcc/help/pubs/pylxml/web/index.html

You may not need anything else...

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From scarolan at gmail.com  Mon Jul  2 01:14:50 2012
From: scarolan at gmail.com (Sean Carolan)
Date: Sun, 1 Jul 2012 18:14:50 -0500
Subject: [Tutor] Python XML for newbie
In-Reply-To: <jsqld3$9b6$1@dough.gmane.org>
References: <CALrDYWMJ08kCUBv4tP9EyeKEJQ3Kp7ZDtgipSR6UCNe+tJcscg@mail.gmail.com>
	<jsqld3$9b6$1@dough.gmane.org>
Message-ID: <CALrDYWP06TxZRU2qEfU6_nZMZXJcNAhHznFD5kshFJ_OMOHWSg@mail.gmail.com>

> The simplest way using the standard library tools is (IMHO)
> elementtree. minidom is a complex beast by comparison,
> especially if you are not intimately familiar with
> your XML structure.

Thank you, this is helpful.  Minidom is confusing, even the
documentation confirms this:
"The name of the functions are perhaps misleading...."

> But I'd start with the etree tutorial (of which
> there are many variations on the web):
>
> The original:
> http://effbot.org/zone/element-index.htm
>
> My preference:
> http://infohost.nmt.edu/tcc/help/pubs/pylxml/web/index.html

I'm going to work through those and see what I can come up with.

From scarolan at gmail.com  Mon Jul  2 04:31:06 2012
From: scarolan at gmail.com (Sean Carolan)
Date: Sun, 1 Jul 2012 21:31:06 -0500
Subject: [Tutor] Python XML for newbie
In-Reply-To: <CALrDYWP06TxZRU2qEfU6_nZMZXJcNAhHznFD5kshFJ_OMOHWSg@mail.gmail.com>
References: <CALrDYWMJ08kCUBv4tP9EyeKEJQ3Kp7ZDtgipSR6UCNe+tJcscg@mail.gmail.com>
	<jsqld3$9b6$1@dough.gmane.org>
	<CALrDYWP06TxZRU2qEfU6_nZMZXJcNAhHznFD5kshFJ_OMOHWSg@mail.gmail.com>
Message-ID: <CALrDYWOwOfFA6ep7zCEpd0sSO04kAWUdaMSxu_m_ObH8Onoddg@mail.gmail.com>

> Thank you, this is helpful.  Minidom is confusing, even the
> documentation confirms this:
> "The name of the functions are perhaps misleading...."
>
>> But I'd start with the etree tutorial (of which
>> there are many variations on the web):

Ok, so I read through these tutorials and am at least able to print
the XML output now.  I did this:

doc = etree.parse('computer_books.xml')

and then this:

for elem in doc.iter():
    print elem.tag, elem.text

Here's the data I'm interested in:

index 1
field 11
value 9780596526740
datum

How do you say, "If the field is 11, then print the next value"?  The
raw XML looks like this:

<datum>
<index>1</index>
<field>11</field>
<value>9780470286975</value>
</datum>

Basically I just want to pull all these ISBN numbers from the file.

From davekidd at gmail.com  Mon Jul  2 04:56:14 2012
From: davekidd at gmail.com (David Kidd)
Date: Mon, 2 Jul 2012 12:56:14 +1000
Subject: [Tutor] Python XML for newbie
In-Reply-To: <CALrDYWOwOfFA6ep7zCEpd0sSO04kAWUdaMSxu_m_ObH8Onoddg@mail.gmail.com>
References: <CALrDYWMJ08kCUBv4tP9EyeKEJQ3Kp7ZDtgipSR6UCNe+tJcscg@mail.gmail.com>
	<jsqld3$9b6$1@dough.gmane.org>
	<CALrDYWP06TxZRU2qEfU6_nZMZXJcNAhHznFD5kshFJ_OMOHWSg@mail.gmail.com>
	<CALrDYWOwOfFA6ep7zCEpd0sSO04kAWUdaMSxu_m_ObH8Onoddg@mail.gmail.com>
Message-ID: <CA+Z-2Wh+uY4aX=X84wTkMEP=EB=29-kZphVbgO0dZAVCuEhwmA@mail.gmail.com>

On Mon, Jul 2, 2012 at 12:31 PM, Sean Carolan <scarolan at gmail.com> wrote:

> How do you say, "If the field is 11, then print the next value"?  The
> raw XML looks like this:
>
> <datum>
> <index>1</index>
> <field>11</field>
> <value>9780470286975</value>
> </datum>
>

Instead of iterating over the whole tree, grab all the <datum> elements
then retrieve the <field> child, check the field value, and if '11', then
pull the <value> value.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120702/35083343/attachment-0001.html>

From gollumgreg407366 at gmail.com  Mon Jul  2 06:21:11 2012
From: gollumgreg407366 at gmail.com (Greg Nielsen)
Date: Sun, 1 Jul 2012 23:21:11 -0500
Subject: [Tutor] Math Function and Python
Message-ID: <CAMM_Zy2gfgsWy83es1GayqB_W0WkYkLNDTBYCmKSrOevMoRReg@mail.gmail.com>

Hello Tutor,

     I'm having some trouble with a mathematical function in my code. I am
attempting to create a realistic acceleration in an object, slower to speed
up at its min and max speeds and fastest in the middle. To keep the math
simpl(er), I just wrote this function for a parabola, which models the
acceleration of the object. Y is the acceleration and X is the current
speed of the object.

Y = -.01X^2 * 1.45X

Before I came up with this formula, I was using a very inefficient list
method to do this. The old code is in italics. I still have it in the code
because it works (sort of). The function is commented out because it
doesn't work. It's also in bold, so you can see it easily.

        if keys[pygame.K_UP] and self.dy < (15 + self.speedMod):
            if self.dy <= 0:
               * self.accelPoints += self.accelList[0]*
                *#self.dy += 1*
            else:
                *self.accelPoints += self.accelList[self.dy]*
                *#self.accelPoints += ((-.1 * (self.dy * self.dy)) + (1.45
* self.dy))*
            if self.accelPoints >= self.accelGoal:
                self.dy += 1
                self.nety = self.dy
                self.accelPoints = 0

I did a test, and for some reason, it seems like the output of the function
is never saved into the self.accelPoints variable, because it always prints
out as 0. (For those who don't want to do the math, when dy = 1, the
function should output something close to 1.55) My best guess is that the
accelPoints doesn't like the fact that the function outputs a double, but
even when i make accelPoints a double (made it equal 0.01 instead of 0) it
still didn't work. If you have any thoughts on the matter, please let me
know. I most likely broke it in the most silly way imaginable. Thanks for
reading.

Greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120701/40584a4f/attachment.html>

From steve at pearwood.info  Mon Jul  2 07:14:15 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 2 Jul 2012 15:14:15 +1000
Subject: [Tutor] Math Function and Python
In-Reply-To: <CAMM_Zy2gfgsWy83es1GayqB_W0WkYkLNDTBYCmKSrOevMoRReg@mail.gmail.com>
References: <CAMM_Zy2gfgsWy83es1GayqB_W0WkYkLNDTBYCmKSrOevMoRReg@mail.gmail.com>
Message-ID: <20120702051415.GA27130@ando>

On Sun, Jul 01, 2012 at 11:21:11PM -0500, Greg Nielsen wrote:
> Hello Tutor,
> 
>      I'm having some trouble with a mathematical function in my code. I am
> attempting to create a realistic acceleration in an object, slower to speed
> up at its min and max speeds and fastest in the middle. To keep the math
> simpl(er), I just wrote this function for a parabola, which models the
> acceleration of the object. Y is the acceleration and X is the current
> speed of the object.
> 
> Y = -.01X^2 * 1.45X

Are you sure that's the expression you want to use for acceleration? It 
certainly isn't a parabola. It simplifies to:

Y = -0.0145*X**3

which says that the DECELERATION is proportional to the CUBE of the 
speed.

Also, that's not a function. This would be a function:

def acceleration(v):
    """Return the acceleration of the object given its current
    velocity.
    """
    return -0.0145*v**3



> Before I came up with this formula, I was using a very inefficient list
> method to do this. The old code is in italics. I still have it in the code
> because it works (sort of). The function is commented out because it
> doesn't work. It's also in bold, so you can see it easily.

Eh, no. There are no italics or bold in plain text emails.


>         if keys[pygame.K_UP] and self.dy < (15 + self.speedMod):
>             if self.dy <= 0:
>                * self.accelPoints += self.accelList[0]*
>                 *#self.dy += 1*
>             else:
>                 *self.accelPoints += self.accelList[self.dy]*
>                 *#self.accelPoints += ((-.1 * (self.dy * self.dy)) + (1.45 * self.dy))*
>             if self.accelPoints >= self.accelGoal:
>                 self.dy += 1
>                 self.nety = self.dy
>                 self.accelPoints = 0

Without explaining what "accelPoints", "nety", "dy" etc. mean, there is 
no real way to tell whether this is a realistic model for a moving body. 
Without more context, I can't tell what it actually does, let alone what 
it is supposed to do.

I'm not an expert on PyGame, but I would expect that the way to model 
movement of a sprite is to calculate the delta-X and delta-Y, where X 
and Y are POSITIONS not speed and acceleration. I don't understand 
what "accelList" is for. Then you call the move() method on the sprite's 
rectangle, and redraw it.


> I did a test, and for some reason, it seems like the output of the function
> is never saved into the self.accelPoints variable, because it always prints
> out as 0.

Of course it does. You set it to 0 with this line:

self.accelPoints = 0


> (For those who don't want to do the math, when dy = 1, the
> function should output something close to 1.55) 

It certainly does not.

In your code, you have this expression:

(-.1 * (self.dy * self.dy)) + (1.45 * self.dy)

When self.dy = 1, that returns 1.35, not 1.55.

The other expression you give, early in this post, doesn't have a dy 
term. But if I put Y = dy instead, it gives -0.0145.



> My best guess is that the
> accelPoints doesn't like the fact that the function outputs a double, 

No.

By the way, in Python we don't talk about doubles. We have floats, which 
are implemented as C doubles, but since we don't have C-singles, we just 
call them floats.



-- 
Steven

From alan.gauld at btinternet.com  Mon Jul  2 09:37:18 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 02 Jul 2012 08:37:18 +0100
Subject: [Tutor] Math Function and Python
In-Reply-To: <CAMM_Zy2gfgsWy83es1GayqB_W0WkYkLNDTBYCmKSrOevMoRReg@mail.gmail.com>
References: <CAMM_Zy2gfgsWy83es1GayqB_W0WkYkLNDTBYCmKSrOevMoRReg@mail.gmail.com>
Message-ID: <jsrj3f$nu2$1@dough.gmane.org>

On 02/07/12 05:21, Greg Nielsen wrote:

> models the acceleration of the object. Y is the acceleration and X is
> the current speed of the object.
> Y = -.01X^2 * 1.45X

That's not the same as the formula you have implemented.

> it doesn't work. It's also in bold, so you can see it easily.
>          if keys[pygame.K_UP] and self.dy < (15 + self.speedMod):
>              if self.dy <= 0:
>                  /self.accelPoints += self.accelList[0]/
>                  *#self.dy += 1*
>              else:
>                  /self.accelPoints += self.accelList[self.dy]/
>                  *#self.accelPoints += ((-.1 * (self.dy * self.dy)) + (1.45 * self.dy))*

Are you sure the else is being executed?
Have you tried inserting a print statement to check the value right 
after the block?

> I did a test, and for some reason, it seems like the output of the
> function is never saved into the self.accelPoints variable, because it
> always prints out as 0.

If your dy is very close to zero the value of your formula will be very 
close to zero so accelPoints never increases. But you also commented out 
the dy increment line so dy never increases which migfht suggest the 
else never gets called??

Can you send us the actual code that fails rather than something else 
that nearly works?
Its much easier to debug the thing that's actually broken that something 
that looks vaguely like it.


> outputs a double, but even when i make accelPoints a double (made it
> equal 0.01 instead of 0) it still didn't work.

Provided they are both numbers Python will adapt. That's not likely to 
be the problem.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From __peter__ at web.de  Mon Jul  2 09:57:27 2012
From: __peter__ at web.de (Peter Otten)
Date: Mon, 02 Jul 2012 09:57:27 +0200
Subject: [Tutor] Python XML for newbie
References: <CALrDYWMJ08kCUBv4tP9EyeKEJQ3Kp7ZDtgipSR6UCNe+tJcscg@mail.gmail.com>
	<jsqld3$9b6$1@dough.gmane.org>
	<CALrDYWP06TxZRU2qEfU6_nZMZXJcNAhHznFD5kshFJ_OMOHWSg@mail.gmail.com>
	<CALrDYWOwOfFA6ep7zCEpd0sSO04kAWUdaMSxu_m_ObH8Onoddg@mail.gmail.com>
Message-ID: <jsrk95$11c$1@dough.gmane.org>

Sean Carolan wrote:

>> Thank you, this is helpful.  Minidom is confusing, even the
>> documentation confirms this:
>> "The name of the functions are perhaps misleading...."
>>
>>> But I'd start with the etree tutorial (of which
>>> there are many variations on the web):
> 
> Ok, so I read through these tutorials and am at least able to print
> the XML output now.  I did this:
> 
> doc = etree.parse('computer_books.xml')
> 
> and then this:
> 
> for elem in doc.iter():
>     print elem.tag, elem.text
> 
> Here's the data I'm interested in:
> 
> index 1
> field 11
> value 9780596526740
> datum
> 
> How do you say, "If the field is 11, then print the next value"?  The
> raw XML looks like this:
> 
> <datum>
> <index>1</index>
> <field>11</field>
> <value>9780470286975</value>
> </datum>
> 
> Basically I just want to pull all these ISBN numbers from the file.

With http://lxml.de/ you can use xpath:

$ cat computer_books.xml 
<foo>
    <bar>
        <datum>
            <index>1</index>
            <field>11</field>
            <value>9780470286975</value>
        </datum>
    </bar>
</foo>
$ cat read_isbn.py
from lxml import etree

root = etree.parse("computer_books.xml")
print root.xpath("//datum[field=11]/value/text()")
$ python read_isbn.py 
['9780470286975']
$ 



From Steve.Flynn at capita.co.uk  Mon Jul  2 16:03:12 2012
From: Steve.Flynn at capita.co.uk (Flynn, Stephen (L & P - IT))
Date: Mon, 2 Jul 2012 15:03:12 +0100
Subject: [Tutor] Simple text file processing using fileinput module.
	"Grabbing successive lines" failure
Message-ID: <D35D4ADAE41B404A9EB381E750C1A5A50247FD76@CAPPRWMMBX14.central.ad.capita.co.uk>

Tutors,

Whilst having a play around with reading in textfiles and reformatting them I tried to write a python 3.2 script to read a CSV file, looking for any records which were short (indicating that the data may well contain an embedded CR/LF. I've attached a small sample file with a "split record" at line 3, and my code.

Call the code with

Python pipesmoker.py MyFile.txt ,

(first parameter is the file being read, second parameter is the field separator... a comma in this case)

I can read the file in, I can determine that I'm looking for records which have 13 fields and I can find a record which is too short (line 3).

What I can't do is read the successive line to a short line in order to append it onto the end of short line before writing the entire amended line out. I'm still thinking about how to persuade the fileinput module to leap over the successor line so it doesn't get processed again.

When I run the code as it stands, I get a traceback as I'm obviously not using fileinput.FileInput.readline() correctly.

value of file is C:\myfile.txt
value of the delimiter is ,
I'm looking for  13 , in each currentLine...
???"1","0000000688      ","ABCD","930020854","34","0","1"," ","930020854 ","          ","0","0","0","0"

"2","0000000688      ","ABCD","930020854","99","0","1"," ","930020854 ","          ","0","0","0","0"

short line found at line 3
Traceback (most recent call last):
  File "C:\Documents and Settings\flynns\workspace\PipeSmoker\src\pipesmoker\pipesmoker.py", line 35, in <module>
    nextLine = fileinput.FileInput.readline(args.file)
  File "C:\Python32\lib\fileinput.py", line 301, in readline
    line = self._buffer[self._bufindex]
AttributeError: 'str' object has no attribute '_buffer'


Can someone explain to me how I am supposed to make use of readline() to grab the next line of a text file please? It may be that I should be using some other module, but chose fileinput as I was hoping to make the little routine as generic as possible; able to spot short lines in tab separated, comma separated, pipe separated, ^~~^ separated and anything else which my clients feel like sending me.

-- 
Steve Flynn


This email and any attachment to it are confidential.  Unless you are the intended recipient, you may not use, copy or disclose either the message or any information contained in the message. If you are not the intended recipient, you should delete this email and notify the sender immediately.

Any views or opinions expressed in this email are those of the sender only, unless otherwise stated.  All copyright in any Capita material in this email is reserved.

All emails, incoming and outgoing, may be recorded by Capita and monitored for legitimate business purposes. 

Capita exclude all liability for any loss or damage arising or resulting from the receipt, use or transmission of this email to the fullest extent permitted by law.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: myfile.txt
URL: <http://mail.python.org/pipermail/tutor/attachments/20120702/9edd88b5/attachment.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pipesmoker.py
Type: application/octet-stream
Size: 1466 bytes
Desc: pipesmoker.py
URL: <http://mail.python.org/pipermail/tutor/attachments/20120702/9edd88b5/attachment.obj>

From joel.goldstick at gmail.com  Mon Jul  2 16:20:52 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 2 Jul 2012 10:20:52 -0400
Subject: [Tutor] Simple text file processing using fileinput module.
 "Grabbing successive lines" failure
In-Reply-To: <D35D4ADAE41B404A9EB381E750C1A5A50247FD76@CAPPRWMMBX14.central.ad.capita.co.uk>
References: <D35D4ADAE41B404A9EB381E750C1A5A50247FD76@CAPPRWMMBX14.central.ad.capita.co.uk>
Message-ID: <CAPM-O+w7KSrZ3GSeCbYf1=Tujj57baqWvxCP2y=3eKQzXf7ZRw@mail.gmail.com>

On Mon, Jul 2, 2012 at 10:03 AM, Flynn, Stephen (L & P - IT)
<Steve.Flynn at capita.co.uk> wrote:
> Tutors,
>
> Whilst having a play around with reading in textfiles and reformatting them I tried to write a python 3.2 script to read a CSV file, looking for any records which were short (indicating that the data may well contain an embedded CR/LF. I've attached a small sample file with a "split record" at line 3, and my code.
>
> Call the code with
>
> Python pipesmoker.py MyFile.txt ,
>
> (first parameter is the file being read, second parameter is the field separator... a comma in this case)
>
> I can read the file in, I can determine that I'm looking for records which have 13 fields and I can find a record which is too short (line 3).
>
> What I can't do is read the successive line to a short line in order to append it onto the end of short line before writing the entire amended line out. I'm still thinking about how to persuade the fileinput module to leap over the successor line so it doesn't get processed again.
>
> When I run the code as it stands, I get a traceback as I'm obviously not using fileinput.FileInput.readline() correctly.
>
> value of file is C:\myfile.txt
> value of the delimiter is ,
> I'm looking for ?13 , in each currentLine...
> ???"1","0000000688 ? ? ?","ABCD","930020854","34","0","1"," ","930020854 "," ? ? ? ? ?","0","0","0","0"
>
> "2","0000000688 ? ? ?","ABCD","930020854","99","0","1"," ","930020854 "," ? ? ? ? ?","0","0","0","0"
>
> short line found at line 3
> Traceback (most recent call last):
> ? File "C:\Documents and Settings\flynns\workspace\PipeSmoker\src\pipesmoker\pipesmoker.py", line 35, in <module>
> ? ? nextLine = fileinput.FileInput.readline(args.file)
> ? File "C:\Python32\lib\fileinput.py", line 301, in readline
> ? ? line = self._buffer[self._bufindex]
> AttributeError: 'str' object has no attribute '_buffer'
>
>
> Can someone explain to me how I am supposed to make use of readline() to grab the next line of a text file please? It may be that I should be using some other module, but chose fileinput as I was hoping to make the little routine as generic as possible; able to spot short lines in tab separated, comma separated, pipe separated, ^~~^ separated and anything else which my clients feel like sending me.
>
Take a look at csvreader
http://docs.python.org/library/csv.html#csv.reader.  It comes with
python, and according to the text near this link, it will handle a
situation where EOL characters are contained in quoted fields.  Will
that help you?


-- 
Joel Goldstick

From stefan_ml at behnel.de  Mon Jul  2 18:49:59 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 02 Jul 2012 18:49:59 +0200
Subject: [Tutor] Python XML for newbie
In-Reply-To: <jsrk95$11c$1@dough.gmane.org>
References: <CALrDYWMJ08kCUBv4tP9EyeKEJQ3Kp7ZDtgipSR6UCNe+tJcscg@mail.gmail.com>
	<jsqld3$9b6$1@dough.gmane.org>
	<CALrDYWP06TxZRU2qEfU6_nZMZXJcNAhHznFD5kshFJ_OMOHWSg@mail.gmail.com>
	<CALrDYWOwOfFA6ep7zCEpd0sSO04kAWUdaMSxu_m_ObH8Onoddg@mail.gmail.com>
	<jsrk95$11c$1@dough.gmane.org>
Message-ID: <jssjfn$a4i$1@dough.gmane.org>

Peter Otten, 02.07.2012 09:57:
> Sean Carolan wrote:
>>> Thank you, this is helpful.  Minidom is confusing, even the
>>> documentation confirms this:
>>> "The name of the functions are perhaps misleading...."

Yes, I personally think that (Mini)DOM should be locked away from beginners
as far as possible.


>> Ok, so I read through these tutorials and am at least able to print
>> the XML output now.  I did this:
>>
>> doc = etree.parse('computer_books.xml')
>>
>> and then this:
>>
>> for elem in doc.iter():
>>     print elem.tag, elem.text
>>
>> Here's the data I'm interested in:
>>
>> index 1
>> field 11
>> value 9780596526740
>> datum
>>
>> How do you say, "If the field is 11, then print the next value"?  The
>> raw XML looks like this:
>>
>> <datum>
>> <index>1</index>
>> <field>11</field>
>> <value>9780470286975</value>
>> </datum>
>>
>> Basically I just want to pull all these ISBN numbers from the file.
> 
> With http://lxml.de/ you can use xpath:
> 
> $ cat computer_books.xml 
> <foo>
>     <bar>
>         <datum>
>             <index>1</index>
>             <field>11</field>
>             <value>9780470286975</value>
>         </datum>
>     </bar>
> </foo>
> $ cat read_isbn.py
> from lxml import etree
> 
> root = etree.parse("computer_books.xml")
> print root.xpath("//datum[field=11]/value/text()")
> $ python read_isbn.py 
> ['9780470286975']
> $ 

And lxml.objectify is also a nice tool for this:

  $ cat example.xml
  <items>
   <item>
    <id>108</id>
    <data>
     <datum>
      <index>1</index>
      <field>2</field>
      <value>Essential System Administration</value>
     </datum>
    </data>
   </item>
  </items>

  $ python
  Python 2.7.3
  >>> from lxml import objectify
  >>> t = objectify.parse('example.xml')
  >>> for datum in t.iter('datum'):
  ...     if datum.field == 2:
  ...         print(datum.value)
  ...
  Essential System Administration
  >>>

It's not impossible that this is faster than the XPath version, but that
depends a lot on the data.

Stefan


From david at graniteweb.com  Mon Jul  2 19:39:50 2012
From: david at graniteweb.com (David Rock)
Date: Mon, 2 Jul 2012 12:39:50 -0500
Subject: [Tutor] Simple text file processing using fileinput module.
 "Grabbing successive lines" failure
In-Reply-To: <D35D4ADAE41B404A9EB381E750C1A5A50247FD76@CAPPRWMMBX14.central.ad.capita.co.uk>
References: <D35D4ADAE41B404A9EB381E750C1A5A50247FD76@CAPPRWMMBX14.central.ad.capita.co.uk>
Message-ID: <20120702173950.GB3060@wdfs.gateway.2wire.net>

* Flynn, Stephen (L & P - IT) <Steve.Flynn at capita.co.uk> [2012-07-02 15:03]:
> Tutors,
> 
> Can someone explain to me how I am supposed to make use of readline()
> to grab the next line of a text file please? It may be that I should
> be using some other module, but chose fileinput as I was hoping to
> make the little routine as generic as possible; able to spot short
> lines in tab separated, comma separated, pipe separated, ^~~^
> separated and anything else which my clients feel like sending me.

There are a couple issues that you need to resolve.  For starters, there
is no guarantee that the successive line is actually part of the
preceding line.  It could very well be that the original line is simply
truncated, in which case trying to append the following line would be
incorrect.

What I typically do in a case like this is use a flag variable and pull
the offending line(s).  So, you need to first determine the best course
of action for resolving the inconsistency (eg, how do you verify the
following line belongs with the preceding)?

Try checking the line, if it's less than 13 then flag and store in a
buffer and continue.  The following line _should_ also error, in which
case, you can try to resolve the two lines, or fail out if the criteria
isn't met.  

Essentially, your problem isn't with using fileinput, it's with how you
handle each line that comes in.

-- 
David Rock
david at graniteweb.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/tutor/attachments/20120702/f20d580d/attachment-0001.pgp>

From alan.gauld at btinternet.com  Mon Jul  2 19:50:38 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 02 Jul 2012 18:50:38 +0100
Subject: [Tutor] Simple text file processing using fileinput module.
 "Grabbing successive lines" failure
In-Reply-To: <D35D4ADAE41B404A9EB381E750C1A5A50247FD76@CAPPRWMMBX14.central.ad.capita.co.uk>
References: <D35D4ADAE41B404A9EB381E750C1A5A50247FD76@CAPPRWMMBX14.central.ad.capita.co.uk>
Message-ID: <jssn1f$9eh$1@dough.gmane.org>

On 02/07/12 15:03, Flynn, Stephen (L & P - IT) wrote:
> Whilst having a play around with reading in textfiles and reformatting them I
 > tried to write a python 3.2 script to read a CSV file,


Best tool for csv files is the csv module, it covers most of the gotchas 
associated with such data.

> What I can't do is read the successive line to a short line in order to
 > append it onto the end of short line before writing the
 > entire amended line out.

Maybe so but we can't help with that because you haven't shown us any 
code related to that issue...

> I'm still thinking about how to persuade the fileinput module

fileinput is normally used when processing many similar files. Its not 
usually used when processing a single file. If you wanted to step onto 
the next file in the input list then fileinput would help there.
But processing lines within the file is up to you.

> I get a traceback as I'm obviously not using fileinput.FileInput.readline() correctly.

Nope, it doesn't look like it but you haven't posted enough code to be 
sure what is happening. But I'll take a guess...

> Traceback (most recent call last):
>    File "C:\Documents and Settings\flynns\workspace\PipeSmoker\src\pipesmoker\pipesmoker.py", line 35, in <module>
>      nextLine = fileinput.FileInput.readline(args.file)
>    File "C:\Python32\lib\fileinput.py", line 301, in readline
>      line = self._buffer[self._bufindex]
> AttributeError: 'str' object has no attribute '_buffer'

It looks like you are not creating an instance of the FileInput class.
You are trying to use the methods directly. Thus the class tries to 
execute the call by using args as self. But args is a string not a 
FileInput instance and it therefore finds no _buffer attribute.

Look at the documentation. The very first few lines show what you want:

--------------
This module implements a helper class and functions to quickly write a 
loop over standard input or a list of files. If you just want to read or 
write one file see open().

The typical use is:
import fileinput
for line in fileinput.input():
     process(line)
---------------

Note the reference to processing a single file with open() and note the 
absence of FileInput in the example code.

Further down it says:

--------------
The class which implements the sequence behavior provided by the module 
is available for subclassing as well:

class fileinput.FileInput([files[, inplace[, backup[, mode[, openhook]]]]])

Class FileInput is the implementation; its methods filename(), fileno(), 
lineno(), filelineno(), isfirstline(), isstdin(), nextfile() and close() 
correspond to the functions of the same name in the module. In addition 
it has a readline() method which returns the next input line,
-----------------

So normally you don't need to use FileInput at all, unless you are 
creating some kind of specialized sub class version. But if you do
use it you need to use it like any other class and create an instance.

HTH,

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Mon Jul  2 19:54:14 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 02 Jul 2012 18:54:14 +0100
Subject: [Tutor] Simple text file processing using fileinput module.
 "Grabbing successive lines" failure
In-Reply-To: <20120702173950.GB3060@wdfs.gateway.2wire.net>
References: <D35D4ADAE41B404A9EB381E750C1A5A50247FD76@CAPPRWMMBX14.central.ad.capita.co.uk>
	<20120702173950.GB3060@wdfs.gateway.2wire.net>
Message-ID: <jssn86$9eh$2@dough.gmane.org>

On 02/07/12 18:39, David Rock wrote:

> Essentially, your problem isn't with using fileinput, it's with how you
> handle each line that comes in.

The immediate problem is with mis-using fileinput.
But once you solve that you then have to deal with the
other issues David raises.

Once more I recommend the csv module...

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From torkamani at gmail.com  Mon Jul  2 21:17:49 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Mon, 2 Jul 2012 15:17:49 -0400
Subject: [Tutor] sitecustomize
Message-ID: <CAEmF0k4VpLD39M0GMYXtciy3+b2sRgmv1NQcMKnbYJ54b+mCxw@mail.gmail.com>

Hi,
I'm absolutely newbie, and appreciate your help, I'm using Spyder, some
times it randomly get the following error:

'import sitecustomize' failed; use -v for traceback

while, I'm not importing and have no idea what is importing it, this
happens even if my code is as simple as following:

import sys
import pdb
#import traceback
import numpy as np
#import scipy as sp

#import cmath


class GaussianProcess(object):
    def __init__(self,data=None,data_mask=None):
        self.data,self.data_mask=data,data_mask
        if data==None and data_mask==None:
            self.GeneraterandomData(100,5,3,2)


    def GeneraterandomData(self,N,C,Lag,nUnknown): # N number of Samples,
C: number of features, Lag: Number of autoregressions, nUnknown: number of
unknown features at each time
         pdb.set_trace()
         self.data=np.random.rand(N,C)
         self.data_mask=np.ones((Lag,C))
         self.data_mask[0,0:nUnknown]=0

         return self.data,self.data_mask


def main(argv=None):
    if argv==None:
        argv=sys.argv

   GP=GaussianProcess();
    #print(GP.data,GP.data_mask)



    print("Done!")


if __name__=="__main__":
    sys.exit(main())


Thanks,

Ali
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120702/6b78c2e8/attachment.html>

From scarolan at gmail.com  Mon Jul  2 22:16:08 2012
From: scarolan at gmail.com (Sean Carolan)
Date: Mon, 2 Jul 2012 15:16:08 -0500
Subject: [Tutor] Python XML for newbie
In-Reply-To: <jssjfn$a4i$1@dough.gmane.org>
References: <CALrDYWMJ08kCUBv4tP9EyeKEJQ3Kp7ZDtgipSR6UCNe+tJcscg@mail.gmail.com>
	<jsqld3$9b6$1@dough.gmane.org>
	<CALrDYWP06TxZRU2qEfU6_nZMZXJcNAhHznFD5kshFJ_OMOHWSg@mail.gmail.com>
	<CALrDYWOwOfFA6ep7zCEpd0sSO04kAWUdaMSxu_m_ObH8Onoddg@mail.gmail.com>
	<jsrk95$11c$1@dough.gmane.org> <jssjfn$a4i$1@dough.gmane.org>
Message-ID: <CALrDYWMX8TK=tbVHqdhhLh2S_xMia6k_NyZBsYkJTdCOp8ZVJQ@mail.gmail.com>

> Yes, I personally think that (Mini)DOM should be locked away from beginners
> as far as possible.

Ok, I'm glad to hear that.  I'll continue to work with ElementTree and
lxml and see where it takes me.

From emile at fenx.com  Mon Jul  2 23:01:05 2012
From: emile at fenx.com (Emile van Sebille)
Date: Mon, 02 Jul 2012 14:01:05 -0700
Subject: [Tutor] sitecustomize
In-Reply-To: <CAEmF0k4VpLD39M0GMYXtciy3+b2sRgmv1NQcMKnbYJ54b+mCxw@mail.gmail.com>
References: <CAEmF0k4VpLD39M0GMYXtciy3+b2sRgmv1NQcMKnbYJ54b+mCxw@mail.gmail.com>
Message-ID: <jst22d$21n$1@dough.gmane.org>

On 7/2/2012 12:17 PM Ali Torkamani said...
> Hi,
> I'm absolutely newbie, and appreciate your help, I'm using Spyder, some
> times it randomly get the following error:
>
> 'import sitecustomize' failed; use -v for traceback

This appears to be somewhat common as if you google for 'spyder 
sitecustomize' you'll get lots of hits and references.

I'd start there -- it certainly doesn't seem to be a learning python issue.

Emile




>
> while, I'm not importing and have no idea what is importing it, this
> happens even if my code is as simple as following:
>
> import sys
> import pdb
> #import traceback
> import numpy as np
> #import scipy as sp
>
> #import cmath
>
>
> class GaussianProcess(object):
>      def __init__(self,data=None,data_mask=None):
>          self.data,self.data_mask=data,data_mask
>          if data==None and data_mask==None:
>              self.GeneraterandomData(100,5,3,2)
>
>
>      def GeneraterandomData(self,N,C,Lag,nUnknown): # N number of
> Samples, C: number of features, Lag: Number of autoregressions,
> nUnknown: number of unknown features at each time
>           pdb.set_trace()
>           self.data=np.random.rand(N,C)
>           self.data_mask=np.ones((Lag,C))
>           self.data_mask[0,0:nUnknown]=0
>
>           return self.data,self.data_mask
>
>
> def main(argv=None):
>      if argv==None:
>          argv=sys.argv
>
>     GP=GaussianProcess();
>      #print(GP.data,GP.data_mask)
>
>
>
>      print("Done!")
>
>
> if __name__=="__main__":
>      sys.exit(main())
>
>
> Thanks,
>
> Ali
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor



From d at davea.name  Mon Jul  2 23:17:52 2012
From: d at davea.name (Dave Angel)
Date: Mon, 02 Jul 2012 17:17:52 -0400
Subject: [Tutor] sitecustomize
In-Reply-To: <CAEmF0k4VpLD39M0GMYXtciy3+b2sRgmv1NQcMKnbYJ54b+mCxw@mail.gmail.com>
References: <CAEmF0k4VpLD39M0GMYXtciy3+b2sRgmv1NQcMKnbYJ54b+mCxw@mail.gmail.com>
Message-ID: <4FF21000.7020702@davea.name>

On 07/02/2012 03:17 PM, Ali Torkamani wrote:
> Hi,
> I'm absolutely newbie, and appreciate your help, I'm using Spyder, some
> times it randomly get the following error:

The first place I found Spyder was here: 
https://en.wikipedia.org/wiki/SPYDER which is an anti-aircraft system
based partly on the Python-5 mssile.

But presumably you mean:
http://pypi.python.org/pypi/spyder

> 'import sitecustomize' failed; use -v for traceback
>
> while, I'm not importing and have no idea what is importing it,

site is automatically imported by Python during startup.  It lets you
customize things in a global way before starting your particular
script.  And one of the things it does it to import sitecustomize

See this link:
http://docs.python.org/library/site.html

and also:
http://www.doughellmann.com/PyMOTW/site/


My guess is that something's wrong with your Spyder installation, and
you should check on their mailing list.

But the first thing you might do is to add -v as your error message
suggests.  For some reason Spyder is hiding 3/4 of your error.



-- 

DaveA


From redacted@example.com  Tue Jul  3 00:55:26 2012
From: redacted@example.com (Alexander Q.)
Date: Mon, 2 Jul 2012 15:55:26 -0700
Subject: [Tutor] Returning multiple objects from a function
Message-ID: <CAHgjEe2uNdTjFBoTB2re4sZ0uU5DdsowRrBhwPimKkCVBvT7uQ@mail.gmail.com>

Hello- I'm wondering how to access specific objects returned from a
function when that function returns multiple objects.

For example, if I have "return(list1, list2, list 3)" within a function
"mainFunc()" that takes no arguments, how do I use list1, list2, and list3
outside of the function once they are returned by that function? Thank you.

-Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120702/bf482168/attachment.html>

From wprins at gmail.com  Tue Jul  3 01:11:00 2012
From: wprins at gmail.com (Walter Prins)
Date: Tue, 3 Jul 2012 00:11:00 +0100
Subject: [Tutor] Returning multiple objects from a function
In-Reply-To: <CAHgjEe2uNdTjFBoTB2re4sZ0uU5DdsowRrBhwPimKkCVBvT7uQ@mail.gmail.com>
References: <CAHgjEe2uNdTjFBoTB2re4sZ0uU5DdsowRrBhwPimKkCVBvT7uQ@mail.gmail.com>
Message-ID: <CANLXbfBg3PPVNqRDu0S9LcsMLvnQRj1yu8uHTSBcmRK7-MEvfw@mail.gmail.com>

On 2 July 2012 23:55, Alexander Q. <redacted@example.com> wrote:
> Hello- I'm wondering how to access specific objects returned from a function
> when that function returns multiple objects.
>
> For example, if I have "return(list1, list2, list 3)" within a function

When you have:
return (list1, list2, list3)

... you're actually returning a single tuple object.  Read about
tuples in the documentation.  To access an item from a tuple you index
into it, e.g.

tupleresult = myfunc()
x = tupleresult[0]

... for example

Walter.

From alan.gauld at btinternet.com  Tue Jul  3 01:14:06 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 03 Jul 2012 00:14:06 +0100
Subject: [Tutor] Returning multiple objects from a function
In-Reply-To: <CAHgjEe2uNdTjFBoTB2re4sZ0uU5DdsowRrBhwPimKkCVBvT7uQ@mail.gmail.com>
References: <CAHgjEe2uNdTjFBoTB2re4sZ0uU5DdsowRrBhwPimKkCVBvT7uQ@mail.gmail.com>
Message-ID: <jst9vu$rga$1@dough.gmane.org>

On 02/07/12 23:55, Alexander Q. wrote:

> For example, if I have "return(list1, list2, list 3)" within a function
> "mainFunc()" that takes no arguments, how do I use list1, list2, and
> list3 outside of the function

Just assign the result to 3 variables:

L1,L2,L3 = mainFunc()

This is just a special case of tuple unpacking which means you can do:

a,b,c = 1,2,3

to set a=1, b=2, c=3

The function returns a tuple of values, so you unpack them into variables.

Alternatively you can keep them as a tuple:

values = mainFunc()

and use indexing:

print values[0], values[1], values[2]

or iterate over them:

for value in mainFunc():
     print value

Lots of options depending on what you want to do...

HTH,

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From redacted@example.com  Tue Jul  3 04:23:15 2012
From: redacted@example.com (Alexander Q.)
Date: Mon, 2 Jul 2012 19:23:15 -0700
Subject: [Tutor] Returning multiple objects from a function
In-Reply-To: <CANLXbfBg3PPVNqRDu0S9LcsMLvnQRj1yu8uHTSBcmRK7-MEvfw@mail.gmail.com>
References: <CAHgjEe2uNdTjFBoTB2re4sZ0uU5DdsowRrBhwPimKkCVBvT7uQ@mail.gmail.com>
	<CANLXbfBg3PPVNqRDu0S9LcsMLvnQRj1yu8uHTSBcmRK7-MEvfw@mail.gmail.com>
Message-ID: <CAHgjEe2abgu6xZUWo9PS1EZqGTU27Gg6Pa6v3gqL3L_pOMX6RA@mail.gmail.com>

I understand the basics of tuples, but that formulation returned the
following error:

Traceback (most recent call last):
  File "C:\Users\Owner\Desktop\MIT\Sets\Set3.py", line 34, in <module>
    list4 = tuplesresult[1]
TypeError: 'NoneType' object is not subscriptable

When I tried to assign "tuplesresult[1]" to the variable "list4" (after
assigning tuplesresult = mainFunc(), which is the name of the function that
returns the tuple in my program), the error occurred. That aside, is it all
right if I just code "return list1, list2" without the parens? In that
case, how would I access list1 and list2 when needed?

Thanks for your help.

On Mon, Jul 2, 2012 at 4:11 PM, Walter Prins <wprins at gmail.com> wrote:

> On 2 July 2012 23:55, Alexander Q. <redacted@example.com> wrote:
> > Hello- I'm wondering how to access specific objects returned from a
> function
> > when that function returns multiple objects.
> >
> > For example, if I have "return(list1, list2, list 3)" within a function
>
> When you have:
> return (list1, list2, list3)
>
> ... you're actually returning a single tuple object.  Read about
> tuples in the documentation.  To access an item from a tuple you index
> into it, e.g.
>
> tupleresult = myfunc()
> x = tupleresult[0]
>
> ... for example
>
> Walter.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120702/95fe98ab/attachment.html>

From d at davea.name  Tue Jul  3 05:56:20 2012
From: d at davea.name (Dave Angel)
Date: Mon, 02 Jul 2012 23:56:20 -0400
Subject: [Tutor] Returning multiple objects from a function
In-Reply-To: <CAHgjEe2abgu6xZUWo9PS1EZqGTU27Gg6Pa6v3gqL3L_pOMX6RA@mail.gmail.com>
References: <CAHgjEe2uNdTjFBoTB2re4sZ0uU5DdsowRrBhwPimKkCVBvT7uQ@mail.gmail.com>
	<CANLXbfBg3PPVNqRDu0S9LcsMLvnQRj1yu8uHTSBcmRK7-MEvfw@mail.gmail.com>
	<CAHgjEe2abgu6xZUWo9PS1EZqGTU27Gg6Pa6v3gqL3L_pOMX6RA@mail.gmail.com>
Message-ID: <4FF26D64.1060308@davea.name>

On 07/02/2012 10:23 PM, Alexander Q. wrote:
> I understand the basics of tuples, but that formulation returned the
> following error:
>
> Traceback (most recent call last):
>   File "C:\Users\Owner\Desktop\MIT\Sets\Set3.py", line 34, in <module>
>     list4 = tuplesresult[1]
> TypeError: 'NoneType' object is not subscriptable
>
> When I tried to assign "tuplesresult[1]" to the variable "list4" (after
> assigning tuplesresult = mainFunc(), which is the name of the function that
> returns the tuple in my program), the error occurred. That aside, is it all
> right if I just code "return list1, list2" without the parens? In that
> case, how would I access list1 and list2 when needed?

That's no different:   list1, list2  is a tuple of size 2.
> Thanks for your help.
>
You top-posted, which loses all the context of the earlier messages. 
Please put your new message AFTER the part you're quoting

Anyway, you never showed the function mainFunc(), but from the error
traceback message, it didn't return anything, which means None.

Chances are you have something like:

def mainFunc():
      if  something:
            return 4, 12


And if the if fails, the function will return None.  To fix that, make
sure all paths through the function return a similar object, generally a
tuple of the same size.



-- 

DaveA


From wprins at gmail.com  Tue Jul  3 10:38:18 2012
From: wprins at gmail.com (Walter Prins)
Date: Tue, 3 Jul 2012 09:38:18 +0100
Subject: [Tutor] Returning multiple objects from a function
In-Reply-To: <CAHgjEe2abgu6xZUWo9PS1EZqGTU27Gg6Pa6v3gqL3L_pOMX6RA@mail.gmail.com>
References: <CAHgjEe2uNdTjFBoTB2re4sZ0uU5DdsowRrBhwPimKkCVBvT7uQ@mail.gmail.com>
	<CANLXbfBg3PPVNqRDu0S9LcsMLvnQRj1yu8uHTSBcmRK7-MEvfw@mail.gmail.com>
	<CAHgjEe2abgu6xZUWo9PS1EZqGTU27Gg6Pa6v3gqL3L_pOMX6RA@mail.gmail.com>
Message-ID: <CANLXbfB_mmZh9ffy-M2NB+XMhbCEwVahLtM9ynyZfdS2OMBp6A@mail.gmail.com>

Hi

On 3 July 2012 03:23, Alexander Q. <redacted@example.com> wrote:
> > Traceback (most recent call last):
>   File "C:\Users\Owner\Desktop\MIT\Sets\Set3.py", line 34, in <module>
>     list4 = tuplesresult[1]
> TypeError: 'NoneType' object is not subscriptable

Ok as a bit of clarification: When you have a function that doesn't
actually return anything, for example:

def mainFunc()
    print "I don't return anything..."

And you then do:

x = mainFunc()

... then x will contain the value None with the type NoneType.

If you then try to index into x, e.g...

print x[0]

... you will get the error above, which is saying the the type of the
variable x, (e.g. NoneType) is not subscriptable, that is, you cannot
index into it.


So, what you have to ask yourself is where/how you omitted to have
mainfunc return some values.  (As per Dave's answer, if you have for
example conditional return statements it's probably possibe for
mainFun() to run without returning anything.)

> When I tried to assign "tuplesresult[1]" to the variable "list4" (after
> assigning tuplesresult = mainFunc(), which is the name of the function that
> returns the tuple in my program), the error occurred. That aside, is it all
> right if I just code "return list1, list2" without the parens? In that case,
> how would I access list1 and list2 when needed?

Yes, Python will automatically package the result into a tuple just the same.

Please note, I forgot to mention in my original reply the automatic
tuple unpacking mechanism in Python as mentioned by Alan.  In your
case (as in many cases), it's probably the more Pythonic/neater way to
express yourself.  If your function is returning 4 lists say, then
rather than getting all 4 in a tuple first and then unpacking them
manually yourself, you can just do...

list1, list2, list3, list4 = mainFunc()


HTH,

Walter

From Steve.Flynn at capita.co.uk  Tue Jul  3 10:30:15 2012
From: Steve.Flynn at capita.co.uk (Flynn, Stephen (L & P - IT))
Date: Tue, 3 Jul 2012 09:30:15 +0100
Subject: [Tutor] Simple text file processing using fileinput module.
	"Grabbing successive lines" failure
In-Reply-To: <jssn86$9eh$2@dough.gmane.org>
References: <D35D4ADAE41B404A9EB381E750C1A5A50247FD76@CAPPRWMMBX14.central.ad.capita.co.uk><20120702173950.GB3060@wdfs.gateway.2wire.net>
	<jssn86$9eh$2@dough.gmane.org>
Message-ID: <D35D4ADAE41B404A9EB381E750C1A5A50247FF2B@CAPPRWMMBX14.central.ad.capita.co.uk>

> On 02/07/12 18:39, David Rock wrote:
> 
> > Essentially, your problem isn't with using fileinput, it's with how
you
> > handle each line that comes in.
> 
> The immediate problem is with mis-using fileinput.
> But once you solve that you then have to deal with the
> other issues David raises.
> 
> Once more I recommend the csv module...


Thanks gents - seems the CSV module does everything I require so I'll
get tinkering with it.

Steve.



This email and any attachment to it are confidential.  Unless you are the intended recipient, you may not use, copy or disclose either the message or any information contained in the message. If you are not the intended recipient, you should delete this email and notify the sender immediately.

Any views or opinions expressed in this email are those of the sender only, unless otherwise stated.  All copyright in any Capita material in this email is reserved.

All emails, incoming and outgoing, may be recorded by Capita and monitored for legitimate business purposes. 

Capita exclude all liability for any loss or damage arising or resulting from the receipt, use or transmission of this email to the fullest extent permitted by law.

From __peter__ at web.de  Tue Jul  3 11:35:22 2012
From: __peter__ at web.de (Peter Otten)
Date: Tue, 03 Jul 2012 11:35:22 +0200
Subject: [Tutor] Simple text file processing using fileinput module.
	"Grabbing successive lines" failure
References: <D35D4ADAE41B404A9EB381E750C1A5A50247FD76@CAPPRWMMBX14.central.ad.capita.co.uk>
Message-ID: <jsuecn$op9$1@dough.gmane.org>

Flynn, Stephen (L & P - IT) wrote:

> Tutors,
> 
> Whilst having a play around with reading in textfiles and reformatting
> them I tried to write a python 3.2 script to read a CSV file, looking for
> any records which were short (indicating that the data may well contain an
> embedded CR/LF. I've attached a small sample file with a "split record" at
> line 3, and my code.
> 
> Call the code with
> 
> Python pipesmoker.py MyFile.txt ,
> 
> (first parameter is the file being read, second parameter is the field
> separator... a comma in this case)
> 
> I can read the file in, I can determine that I'm looking for records which
> have 13 fields and I can find a record which is too short (line 3).
> 
> What I can't do is read the successive line to a short line in order to
> append it onto the end of short line before writing the entire amended
> line out. I'm still thinking about how to persuade the fileinput module to
> leap over the successor line so it doesn't get processed again.
> 
> When I run the code as it stands, I get a traceback as I'm obviously not
> using fileinput.FileInput.readline() correctly.
> 
> value of file is C:\myfile.txt
> value of the delimiter is ,
> I'm looking for  13 , in each currentLine...
> ???"1","0000000688      ","ABCD","930020854","34","0","1"," ","930020854
> ","          ","0","0","0","0"
> 
> "2","0000000688      ","ABCD","930020854","99","0","1"," ","930020854 "," 
>         ","0","0","0","0"
> 
> short line found at line 3
> Traceback (most recent call last):
>   File "C:\Documents and
>   Settings\flynns\workspace\PipeSmoker\src\pipesmoker\pipesmoker.py", line
>   35, in <module>
>     nextLine = fileinput.FileInput.readline(args.file)
>   File "C:\Python32\lib\fileinput.py", line 301, in readline
>     line = self._buffer[self._bufindex]
> AttributeError: 'str' object has no attribute '_buffer'
> 
> 
> Can someone explain to me how I am supposed to make use of readline() to
> grab the next line of a text file please? It may be that I should be using
> some other module, but chose fileinput as I was hoping to make the little
> routine as generic as possible; able to spot short lines in tab separated,
> comma separated, pipe separated, ^~~^ separated and anything else which my
> clients feel like sending me.

As you already learned the csv module is the best tool to address your 
problem. 

However, I'd like to show a generic way to get an extra item in a for-loop.

Instead of iterating over the "iterable" (a list or a FileInput object or 
whatever) you first convert it into an iterator explicitly with the iter() 
built-in function and keep the reference around:

iterable = ...
it = iter(iterable)

Then inside the for-loop you get an extra item with the next() function:

for item in it:
    if some_condition():
        extra = next(it)

next() also allows you to provide a default value; without it you may get a 
StopIteration exception when you apply it on an exhausted iterator.

Here's a self-contained example:

>>> items = "alpha- beta gamma- delta- epsilon zeta".split()
>>> it = iter(items)
>>> for item in it:
...     while item.endswith("-"):
...             item += next(it)
...     print item
... 
alpha-beta
gamma-delta-epsilon
zeta



From Steve.Flynn at capita.co.uk  Tue Jul  3 11:54:22 2012
From: Steve.Flynn at capita.co.uk (Flynn, Stephen (L & P - IT))
Date: Tue, 3 Jul 2012 10:54:22 +0100
Subject: [Tutor] Simple text file processing using fileinput
	module."Grabbing successive lines" failure
In-Reply-To: <jsuecn$op9$1@dough.gmane.org>
References: <D35D4ADAE41B404A9EB381E750C1A5A50247FD76@CAPPRWMMBX14.central.ad.capita.co.uk>
	<jsuecn$op9$1@dough.gmane.org>
Message-ID: <D35D4ADAE41B404A9EB381E750C1A5A50247FFA4@CAPPRWMMBX14.central.ad.capita.co.uk>

Hi Peter,

> As you already learned the csv module is the best tool to address your
> problem.

Yup - it's almost going to make my little coding exercise trivial. :)

> However, I'd like to show a generic way to get an extra item in a for-
> loop.

[snip]

> Here's a self-contained example:
> 
> >>> items = "alpha- beta gamma- delta- epsilon zeta".split()
> >>> it = iter(items)
> >>> for item in it:
> ...     while item.endswith("-"):
> ...             item += next(it)
> ...     print item
> ...
> alpha-beta
> gamma-delta-epsilon
> zeta

Perfect - that's essentially what I was trying to do, but I doubt it
would have occurred to me to create my own iterator and make use of it.

Thanks for this example too - nice and easy to understand.

Steve.


This email and any attachment to it are confidential.  Unless you are the intended recipient, you may not use, copy or disclose either the message or any information contained in the message. If you are not the intended recipient, you should delete this email and notify the sender immediately.

Any views or opinions expressed in this email are those of the sender only, unless otherwise stated.  All copyright in any Capita material in this email is reserved.

All emails, incoming and outgoing, may be recorded by Capita and monitored for legitimate business purposes. 

Capita exclude all liability for any loss or damage arising or resulting from the receipt, use or transmission of this email to the fullest extent permitted by law.

From emeraldoffice at hotmail.com  Tue Jul  3 21:43:53 2012
From: emeraldoffice at hotmail.com (Tamar Osher)
Date: Tue, 3 Jul 2012 14:43:53 -0500
Subject: [Tutor] how contact the administrator of PythonTutor?
Message-ID: <BAY167-W12C2F70CDA36DA1E5005D3B9E90@phx.gbl>


How do I contact the administrator of PythonTutor?



My email address is EmeraldOffice at hotmail.com

 








 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120703/b1c3812c/attachment.html>

From emile at fenx.com  Tue Jul  3 23:19:30 2012
From: emile at fenx.com (Emile van Sebille)
Date: Tue, 03 Jul 2012 14:19:30 -0700
Subject: [Tutor] how contact the administrator of PythonTutor?
In-Reply-To: <BAY167-W12C2F70CDA36DA1E5005D3B9E90@phx.gbl>
References: <BAY167-W12C2F70CDA36DA1E5005D3B9E90@phx.gbl>
Message-ID: <jsvngt$h1n$1@dough.gmane.org>

On 7/3/2012 12:43 PM Tamar Osher said...
> How do I contact the administrator of PythonTutor?

One should respond shortly.

Most admin related tasks can be self administered at

http://mail.python.org/mailman/listlinfo/tutor/

Emile


From alan.gauld at btinternet.com  Wed Jul  4 01:28:35 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 04 Jul 2012 00:28:35 +0100
Subject: [Tutor] how contact the administrator of PythonTutor?
In-Reply-To: <BAY167-W12C2F70CDA36DA1E5005D3B9E90@phx.gbl>
References: <BAY167-W12C2F70CDA36DA1E5005D3B9E90@phx.gbl>
Message-ID: <jsvv74$34t$1@dough.gmane.org>

On 03/07/12 20:43, Tamar Osher wrote:
> How do I contact the administrator of PythonTutor?

You just have. But to be honest we don't do much administering.
Its mostly self-service!

> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From redacted@example.com  Wed Jul  4 04:45:53 2012
From: redacted@example.com (Alexander Q.)
Date: Tue, 3 Jul 2012 19:45:53 -0700
Subject: [Tutor] Returning multiple objects from a function
In-Reply-To: <4FF26D64.1060308@davea.name>
References: <CAHgjEe2uNdTjFBoTB2re4sZ0uU5DdsowRrBhwPimKkCVBvT7uQ@mail.gmail.com>
	<CANLXbfBg3PPVNqRDu0S9LcsMLvnQRj1yu8uHTSBcmRK7-MEvfw@mail.gmail.com>
	<CAHgjEe2abgu6xZUWo9PS1EZqGTU27Gg6Pa6v3gqL3L_pOMX6RA@mail.gmail.com>
	<4FF26D64.1060308@davea.name>
Message-ID: <CAHgjEe0Qhozr6P5jt-HXotRDsP+u3jzfcx1nF1jkt1k8uCg=Zg@mail.gmail.com>

On Mon, Jul 2, 2012 at 8:56 PM, Dave Angel <d at davea.name> wrote:

> On 07/02/2012 10:23 PM, Alexander Q. wrote:
> > I understand the basics of tuples, but that formulation returned the
> > following error:
> >
> > Traceback (most recent call last):
> >   File "C:\Users\Owner\Desktop\MIT\Sets\Set3.py", line 34, in <module>
> >     list4 = tuplesresult[1]
> > TypeError: 'NoneType' object is not subscriptable
> >
> > When I tried to assign "tuplesresult[1]" to the variable "list4" (after
> > assigning tuplesresult = mainFunc(), which is the name of the function
> that
> > returns the tuple in my program), the error occurred. That aside, is it
> all
> > right if I just code "return list1, list2" without the parens? In that
> > case, how would I access list1 and list2 when needed?
>
> That's no different:   list1, list2  is a tuple of size 2.
> > Thanks for your help.
> >
> You top-posted, which loses all the context of the earlier messages.
> Please put your new message AFTER the part you're quoting
>
> Anyway, you never showed the function mainFunc(), but from the error
> traceback message, it didn't return anything, which means None.
>
> Chances are you have something like:
>
> def mainFunc():
>       if  something:
>             return 4, 12
>
>
> And if the if fails, the function will return None.  To fix that, make
> sure all paths through the function return a similar object, generally a
> tuple of the same size.
>
>
>
> --
>
> DaveA
>
> Ok thanks- I will try that.

-Alex

P.S. Let me know if this message is sent incorrectly (I scrolled down to
the bottom of the text box to type this, as opposed to writing the message
at the top of the text box. I am guessing this is what is meant by
"top-posting.").
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120703/765a902b/attachment.html>

From steve at pearwood.info  Wed Jul  4 06:51:18 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 4 Jul 2012 14:51:18 +1000
Subject: [Tutor] Returning multiple objects from a function
In-Reply-To: <CAHgjEe0Qhozr6P5jt-HXotRDsP+u3jzfcx1nF1jkt1k8uCg=Zg@mail.gmail.com>
References: <CAHgjEe2uNdTjFBoTB2re4sZ0uU5DdsowRrBhwPimKkCVBvT7uQ@mail.gmail.com>
	<CANLXbfBg3PPVNqRDu0S9LcsMLvnQRj1yu8uHTSBcmRK7-MEvfw@mail.gmail.com>
	<CAHgjEe2abgu6xZUWo9PS1EZqGTU27Gg6Pa6v3gqL3L_pOMX6RA@mail.gmail.com>
	<4FF26D64.1060308@davea.name>
	<CAHgjEe0Qhozr6P5jt-HXotRDsP+u3jzfcx1nF1jkt1k8uCg=Zg@mail.gmail.com>
Message-ID: <20120704045117.GA10454@ando>

On Tue, Jul 03, 2012 at 07:45:53PM -0700, Alexander Q. wrote:

> -Alex
> 
> P.S. Let me know if this message is sent incorrectly (I scrolled down to
> the bottom of the text box to type this, as opposed to writing the message
> at the top of the text box. I am guessing this is what is meant by
> "top-posting.").

I can see your PS, but I can't (easily) see what your comment was that 
you are adding a PS to.

Alex, sorry to nag, but email protocol is that things you type yourself 
should be just ordinary text like this paragraph. Text you are quoting 
from a previous email should start with a less than sign > like the PS 
message I quoted. Multiple levels of quoting should start with multiple 
less than signs.

If you type text starting with a > sign, deliberately or accidentally, 
it looks like you are quoting somebody else. 

This is counter-productive, because many people reading your email will 
just skim quoted text and they won't notice your comment; worse, some 
email programs can be configured to hide large blocks of quoted text, so 
your comment will be hidden and they literally cannot see it.

Strictly speaking, you're also misattributing your own words to somebody 
else. (You are effectively claiming that *they* wrote what *you* 
actually wrote.) Not only does that cause confusion, but it is rude and 
possibly even actionable if serious enough. (Suppose you wrote something 
like "I have a serious drinking problem", but made it look like you were 
quoting Fred and therefore *Fred* had the drinking problem -- Fred might 
decide to sue you for defamation.)

So please take a modicum of care when writing emails that you start a 
new, BLANK line before typing, and not one starting with >

Last but not least, on technical forums like this, the usual protocol is 
neither "top posting" (type your message above the previous email) nor 
"bottom posting" (type your message below the previous email), but 
"inline posting". This takes a little more effort, but in a long thread 
it makes it much easier to follow the discussion.

In inline posting, you add your comments *between* quotes, as if in a 
dialog. Delete parts of the previous conversation that are no longer 
relevant, so as to keep the size of the email manageable. It is polite 
to indicate when you have deleted text, at least the first time, by 
putting in something like 

[...]

or 

[snip]

to indicate that you have edited the previous person's reply. Leave 
enough text to establish context. If you look back over some of the 
archives, you will see plenty of examples from regulars.

http://mail.python.org/pipermail/tutor/


See also:

http://en.wikipedia.org/wiki/Posting_style



-- 
Steven

From electrical at triad.rr.com  Sun Jul  1 23:50:12 2012
From: electrical at triad.rr.com (Jim)
Date: Sun, 01 Jul 2012 17:50:12 -0400
Subject: [Tutor] for loop question
Message-ID: <4FF0C614.60008@triad.rr.com>

Hello Friends,
I apologize for being such a bother. This problem has been evading me 
all day. Can you please give me a hint as to why I cannot put the 
variable UpperCaseSentence outside of the for loop?
I can do it in other instances but not in this one.
Thank you so much,
Jim

#Main function.
def main():

     mySentence = (input("Enter text."))

     mySentenceList = mySentence.split('.')



     #Call fixCase function. Send it mySentenceList and receive result
     #and stores result in variable named output.
     output = fixCase(mySentenceList)
     print(output)



def fixCase(myList):
     #Begin making a loop through the list, using a variable myString
     for myString in range (len(myList)):
         tempString = myList[myString] #Store in temporary variable.
         myList[myString] = tempString[0:1].upper() + 
tempString[1:len(tempString)] #Replace with upper
         UpperCaseSentence = (myList[myString])
         print(UpperCaseSentence)


#Call main function
main()

-------------- next part --------------
A non-text attachment was scrubbed...
Name: electrical.vcf
Type: text/x-vcard
Size: 272 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/tutor/attachments/20120701/164332a3/attachment-0001.vcf>

From mayank25080562 at gmail.com  Mon Jul  2 00:22:14 2012
From: mayank25080562 at gmail.com (mjoll)
Date: Sun, 1 Jul 2012 15:22:14 -0700 (PDT)
Subject: [Tutor] Pygame installation problems
In-Reply-To: <4F9B7625.9080009@pearwood.info>
References: <CAF33E7ZDfn-tcgBg8uzd4qF30EndPPgGM+3VS+jKFGH+nuER8Q@mail.gmail.com>
	<4F9B7625.9080009@pearwood.info>
Message-ID: <1341181334710-4980266.post@n6.nabble.com>

I too am having the same problems.... i have fc17 and i use yum install
pygame to install it!!

--
View this message in context: http://python.6.n6.nabble.com/Tutor-Pygame-installation-problems-tp4935611p4980266.html
Sent from the Python - tutor mailing list archive at Nabble.com.

From hugo.yoshi at gmail.com  Wed Jul  4 11:30:05 2012
From: hugo.yoshi at gmail.com (Hugo Arts)
Date: Wed, 4 Jul 2012 11:30:05 +0200
Subject: [Tutor] for loop question
In-Reply-To: <4FF0C614.60008@triad.rr.com>
References: <4FF0C614.60008@triad.rr.com>
Message-ID: <CAJmBOfnvuSO_dbhrEDO0P-xDzWgDzRhXzGwuz+hr65qJH8292g@mail.gmail.com>

On Sun, Jul 1, 2012 at 11:50 PM, Jim <electrical at triad.rr.com> wrote:

> Hello Friends,
> I apologize for being such a bother. This problem has been evading me all
> day. Can you please give me a hint as to why I cannot put the variable
> UpperCaseSentence outside of the for loop?
> I can do it in other instances but not in this one.
> Thank you so much,
> Jim
>
> #Main function.
> def main():
>
>     mySentence = (input("Enter text."))
>
>     mySentenceList = mySentence.split('.')
>
>
>
>     #Call fixCase function. Send it mySentenceList and receive result
>     #and stores result in variable named output.
>     output = fixCase(mySentenceList)
>     print(output)
>
>
>
> def fixCase(myList):
>     #Begin making a loop through the list, using a variable myString
>     for myString in range (len(myList)):
>         tempString = myList[myString] #Store in temporary variable.
>         myList[myString] = tempString[0:1].upper() +
> tempString[1:len(tempString)] #Replace with upper
>         UpperCaseSentence = (myList[myString])
>         print(UpperCaseSentence)
>
>
> #Call main function
> main()
>
>
Exactly in what way are you trying to hoist it outside of the loop? The way
UpperCaseSentence is defined right now, it depends on myString, which
exists only inside the loop. You can't move that out of the loop since
myString won't exist there.

A few other comments:

- tempString[0:1] may be written as tempString[:1]. Similarly,
tempString[1:len(tempString)] may be written as tempString[1:]. In simple
terms, you can leave out part of the slice and it will default to the end
or beginning.
-  You might want to have a look at str.capitalize(). It does exactly what
you need.

HTH,
Hugo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120704/fc9a39b9/attachment.html>

From emile at fenx.com  Wed Jul  4 16:53:14 2012
From: emile at fenx.com (Emile van Sebille)
Date: Wed, 04 Jul 2012 07:53:14 -0700
Subject: [Tutor] VPYTHON
In-Reply-To: <CAFO9gPpnDcUxnOKCOiaLd8j4w2Qqp8bZ9eis+nPMYwh8FD=y8g@mail.gmail.com>
References: <CAFO9gPpnDcUxnOKCOiaLd8j4w2Qqp8bZ9eis+nPMYwh8FD=y8g@mail.gmail.com>
Message-ID: <jt1lfi$b59$1@dough.gmane.org>

On 6/27/2012 3:59 PM Prajwal Niraula said...
> Hi,
> I am trying to learn VPython for more interesting simulation. I
> downloaded it from the website: www.vpython.org <http://www.vpython.org>
> While I have Python 3.2.3 in my computer, and it seems no equivalent
> version for vpython is available. Besides when installing I had problem
> in which the installer would not recognise the location of python, even
> though it is located at C:\Python32.
> What should I be doing?

Sign up for their mailing list at
https://lists.sourceforge.net/lists/listinfo/visualpython-users
and ask them.

Emile


From emile at fenx.com  Wed Jul  4 17:09:14 2012
From: emile at fenx.com (Emile van Sebille)
Date: Wed, 04 Jul 2012 08:09:14 -0700
Subject: [Tutor] Passing numeral result of a defined function to a named
	value
In-Reply-To: <CA+6+J7bO07NORCSYwK5TDM39MFUET+XKTg=6qvcknRoM-zvHGg@mail.gmail.com>
References: <CA+6+J7bO07NORCSYwK5TDM39MFUET+XKTg=6qvcknRoM-zvHGg@mail.gmail.com>
Message-ID: <jt1mdi$knr$1@dough.gmane.org>

On 6/28/2012 11:51 PM Joseph Hines said...
> Hello. Sorry to bother you, but I'm self-teaching myself python, and
> having difficulty with a script I'm writing to auto-calc various derived
> stats for a PnP game I am hosting soon.
>
> I need to pass a numeral result of a defined function to a "named" value
> (not sure if that's what it's called)
>
> The offending segment of code is this:
>

... and what's the offending error? (provide traceback details please)

Do you mean to be defining a function within a try-except structure?  As 
it stands you'll get a syntax error.

You've likely got an indetation issue on the line before except.

providing int_mod as some integer and reworking it to:

XP_Per_Level_Base = 20-int_mod
XP_Per_Level_Limit = 5
try:
      def XP_Per_Level_Calc(XP_Per_Level_Base, XP_Per_Level_Limit):
          if XP_Per_Level_Base < XP_Per_Level_Limit:
              return XP_Per_Level_Limit
          else:
              return XP_Per_Level_Base
except ValueError:
      print "Script B0RK3D, Link Shall Come to Town."
      print "send him a message at 
Link6746 at gmail.com<mailto:Link6746 at gmail.com>"


XP_Per_Level = XP_Per_Level_Calc(XP_Per_Level_Base, XP_Per_Level_Limit)

gets it going, dut all in all it looks to me like this is simply a max 
function and could be written more simply as:
     max(XP_Per_Level_Base,XP_Per_Level_Limit)

Emile


>
>     XP_Per_Level_Base = 20-int_mod
>     XP_Per_Level_Limit = 5
>     try:
>          def XP_Per_Level_Calc(XP_Per_Level_Base, XP_Per_Level_Limit):
>              if XP_Per_Level_Base < XP_Per_Level_Limit:
>                  return XP_Per_Level_Limit
>              else:
>                  return XP_Per_Level_Base
>     XP_Per_Level = XP_Per_Level_Calc(XP_Per_Level_Base, XP_Per_Level_Limit)
>     except ValueError:
>          print "Script B0RK3D, Link Shall Come to Town."
>          print "send him a message at Link6746 at gmail.com
>     <mailto:Link6746 at gmail.com>"
>
>     #Calcs XP per Level#
>
>
> If you need the whole script,  I'd be glad to send it to you, just
> promise me not to divulge too many details of it
> (It's regarding a custom PnP system that I'm planning on releasing under
> the Open Gaming License, and I'd prefer not to risk it becoming a
> proprietary system for someone I don't know)
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor



From emile at fenx.com  Wed Jul  4 17:30:07 2012
From: emile at fenx.com (Emile van Sebille)
Date: Wed, 04 Jul 2012 08:30:07 -0700
Subject: [Tutor] for loop question
In-Reply-To: <4FF0C614.60008@triad.rr.com>
References: <4FF0C614.60008@triad.rr.com>
Message-ID: <jt1nko$v1v$1@dough.gmane.org>

On 7/1/2012 2:50 PM Jim said...
> Hello Friends,
> I apologize for being such a bother. This problem has been evading me
> all day. Can you please give me a hint as to why I cannot put the
> variable UpperCaseSentence outside of the for loop?
> I can do it in other instances but not in this one.
> Thank you so much,
> Jim



You're not returning anything from fixCase -- change the final print to 
return and try it again.

Emile



>
> #Main function.
> def main():
>     mySentence = (input("Enter text."))
>     mySentenceList = mySentence.split('.')
>
>     #Call fixCase function. Send it mySentenceList and receive result
>     #and stores result in variable named output.
>     output = fixCase(mySentenceList)
>     print(output)
>
> def fixCase(myList):
>     #Begin making a loop through the list, using a variable myString
>     for myString in range (len(myList)):
>         tempString = myList[myString] #Store in temporary variable.
>         myList[myString] = tempString[0:1].upper() + tempString[1:len(tempString)] #Replace with upper
>         UpperCaseSentence = (myList[myString])
>         print(UpperCaseSentence)
>
> #Call main function
> main()
>


From abasiemeka at gmail.com  Wed Jul  4 18:01:34 2012
From: abasiemeka at gmail.com (Osemeka Osuagwu)
Date: Wed, 4 Jul 2012 17:01:34 +0100
Subject: [Tutor] Bothersome NoneType Error for a List object
Message-ID: <CAF33E7ZaobiNfaG4+Lyg218q6o4HwAGorZMzVYBR-i19rWhAqw@mail.gmail.com>

Hi,
I am trying to find the smallest positive number that is divisible by all
of the numbers from 1 to 20 without a remainder. I wrote the following code
(implementation of a solution method found in
http://en.wikipedia.org/wiki/Least_common_multiple#A_method_using_a_table )
but kept getting an error message (also posted) when I ran it. I can't
figure out the problem. I would appreciate help with this. Also, any hint
on how to make the code more elegant is welcome.

Regards,
Abasiemeka

*CODE*
def checkdiv(x, testlist):      #returns True if x can divide ANY member of
testlist,returns False otherwise
    for k in testlist:
        if k%x == 0:
            return True
    return False

def lcm(numlist):               #continuously divides numlist by each in a
list of prime numbers till cannot
    primeslist = [2, 3, 5, 7, 11]
    templist = []

    for prime in primeslist:
        if checkdiv(prime, numlist) == True:
            templist = templist.append(prime)
            for i in range(0,len(numlist)):
                if numlist[i]%prime == 0:
                    numlist[i] = numlist[i]/prime
        else:
            pass

    lcm = reduce(lambda x, y: x*y, templist)     #my first lambda
expression! (multiply all members of templist
    return lcm

print lcm([i for i in range(1, 21)])

*ERROR OUTPUT*
Traceback (most recent call last):
  File "C:\Windows.old\Users\Abasiemeka\Abasiemeka\GOOGLE
University\Python\Python Code\MyCode\Project Euler code\Project Euler
answer 5.py", line 31, in <module>
    print lcm(first20)
  File "C:\Windows.old\Users\Abasiemeka\Abasiemeka\GOOGLE
University\Python\Python Code\MyCode\Project Euler code\Project Euler
answer 5.py", line 20, in lcm
    templist = templist.append(prime)
AttributeError: 'NoneType' object has no attribute 'append'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120704/1cac9413/attachment.html>

From steve at pearwood.info  Wed Jul  4 18:15:08 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 05 Jul 2012 02:15:08 +1000
Subject: [Tutor] Bothersome NoneType Error for a List object
In-Reply-To: <CAF33E7ZaobiNfaG4+Lyg218q6o4HwAGorZMzVYBR-i19rWhAqw@mail.gmail.com>
References: <CAF33E7ZaobiNfaG4+Lyg218q6o4HwAGorZMzVYBR-i19rWhAqw@mail.gmail.com>
Message-ID: <4FF46C0C.4000509@pearwood.info>

Osemeka Osuagwu wrote:

>             templist = templist.append(prime)

The append method operates in place, and returns None. It doesn't return a list:

py> mylist = []
py> x = mylist.append(42)
py> x is None
True
py> mylist
[42]

Replace that line with just

templist.append(prime)


-- 
Steven

From alan.gauld at btinternet.com  Wed Jul  4 18:56:51 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 04 Jul 2012 17:56:51 +0100
Subject: [Tutor] Seeking help with reading and writing files in Python
In-Reply-To: <4FE9FF49.203@triad.rr.com>
References: <4FE9FF49.203@triad.rr.com>
Message-ID: <jt1skj$89p$1@dough.gmane.org>

On 26/06/12 19:28, Aristotle wrote:

> with an assignment that asks me to read from one file and then write to
> another file. The problem is that the file to read from has not been
> created yet.

So create it.
Any old text editor will do.
The whole point is, I believe that you are writing a program to read a 
file created by some other application, perhaps a web application or 
database reporting tool. (or in this case Notepad, say).

How the file is created is irrelevant so long as it has the data that 
you want to extract within it.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Wed Jul  4 19:03:12 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 04 Jul 2012 18:03:12 +0100
Subject: [Tutor] Script
In-Reply-To: <CAEJFaLd8MD-twe6c77OLxmT4AQWGBm7uZ0LLEmtpQSvV0j3XAQ@mail.gmail.com>
References: <CAEJFaLd8MD-twe6c77OLxmT4AQWGBm7uZ0LLEmtpQSvV0j3XAQ@mail.gmail.com>
Message-ID: <jt1t0h$b1d$1@dough.gmane.org>

On 28/06/12 06:21, Amit Prasad wrote:

> I want to write a script to replace hostname from /etc/hosts and
> /etc/sysconfig/network files. And the new hostname will be a new entry
> everytime. I mean everytime when i run the script and give a new
> hostname, it should be replaced with the old one.

You don't say what language/OS you are using, I'll assume Python
on *nix since you are posting here and using /etc/hosts.
Do you know how to program in Python?
Do you know how to program in any other language?

What exactly is the bit you don't understand?

But if you are running out of time and don't already know how to
program I would recommend you study the sed man page....
sed was written for just these kinds of tasks.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Wed Jul  4 19:06:47 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 04 Jul 2012 18:06:47 +0100
Subject: [Tutor] Python and boot sequences
In-Reply-To: <DF8381D0C668344094293D48724112BC0704AD46@SEAEMBX01.olympus.F5Net.com>
References: <DF8381D0C668344094293D48724112BC0704AD46@SEAEMBX01.olympus.F5Net.com>
Message-ID: <jt1t77$cf1$1@dough.gmane.org>

On 28/06/12 13:54, Dave Wilder wrote:

> Can a Python script be written that has the ability to stop a Linux
> device in the middle of a boot when a certain sequence occurs and then
> perform an action?

It depends...

> For example, when I boot my switch (w/ Linux OS 2.7.3), I want to stop
> the boot when I see a series of asterisks.  When I see this, I need to
> hit the <ESC><SHIFT><9>

Could you do it with a bash script? If so you could do it with Python.
But I've no idea whether your switch will be in a state to run user 
processes at that point in its boot sequence. If you don't have a
shell environment its going to be a whole lot harder.

> doing a PXE boot,

???

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Wed Jul  4 19:13:36 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 04 Jul 2012 18:13:36 +0100
Subject: [Tutor] Bothersome NoneType Error for a List object
In-Reply-To: <CAF33E7ZaobiNfaG4+Lyg218q6o4HwAGorZMzVYBR-i19rWhAqw@mail.gmail.com>
References: <CAF33E7ZaobiNfaG4+Lyg218q6o4HwAGorZMzVYBR-i19rWhAqw@mail.gmail.com>
Message-ID: <jt1tk0$fnd$1@dough.gmane.org>

On 04/07/12 17:01, Osemeka Osuagwu wrote:

>      lcm = reduce(lambda x, y: x*y, templist)     #my first lambda
> expression! (multiply all members of templist

Congratulations :-)

But you could have done:

import operator
lcm = reduce(operator.mul, templist)

instead.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Wed Jul  4 19:17:42 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 04 Jul 2012 18:17:42 +0100
Subject: [Tutor] Pygame installation problems
In-Reply-To: <1341181334710-4980266.post@n6.nabble.com>
References: <CAF33E7ZDfn-tcgBg8uzd4qF30EndPPgGM+3VS+jKFGH+nuER8Q@mail.gmail.com>
	<4F9B7625.9080009@pearwood.info>
	<1341181334710-4980266.post@n6.nabble.com>
Message-ID: <jt1trm$fnd$2@dough.gmane.org>

On 01/07/12 23:22, mjoll wrote:
> I too am having the same problems.... i have fc17 and i use yum install
> pygame to install it!!

You might be better asking on the pygame mailing list/forum.

BTW Its a bad idea to reply to a 6 month old post even if it is a 
similar issue, anyone using a threaded reader is unlikely to see
it, you would be better starting a new thread.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From abasiemeka at gmail.com  Wed Jul  4 22:38:39 2012
From: abasiemeka at gmail.com (Osemeka Osuagwu)
Date: Wed, 4 Jul 2012 21:38:39 +0100
Subject: [Tutor] VPYTHON (Emile van Sebille)
Message-ID: <CAF33E7ayQ0gV-3eBTfOMqQwAwFwER7tbScMz6QhzjLdTapU=+A@mail.gmail.com>

Hi Emile,
Re-install your vpython and make sure you select ''C:\Python32'' as install
location and not ''C:\Python32\New Folder'' or any other.

Cheers,
Abasiemeka

On Wed, Jul 4, 2012 at 5:57 PM, <tutor-request at python.org> wrote:

> Send Tutor mailing list submissions to
>         tutor at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mail.python.org/mailman/listinfo/tutor
> or, via email, send a message with subject or body 'help' to
>         tutor-request at python.org
>
> You can reach the person managing the list at
>         tutor-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."
>
>
> Today's Topics:
>
>    1. Re: VPYTHON (Emile van Sebille)
>    2. Re: Passing numeral result of a defined function to a named
>       value (Emile van Sebille)
>    3. Re: for loop question (Emile van Sebille)
>    4. Bothersome NoneType Error for a List object (Osemeka Osuagwu)
>    5. Re: Bothersome NoneType Error for a List object (Steven D'Aprano)
>    6. Re: Seeking help with reading and writing files in Python
>       (Alan Gauld)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 04 Jul 2012 07:53:14 -0700
> From: Emile van Sebille <emile at fenx.com>
> To: tutor at python.org
> Subject: Re: [Tutor] VPYTHON
> Message-ID: <jt1lfi$b59$1 at dough.gmane.org>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 6/27/2012 3:59 PM Prajwal Niraula said...
> > Hi,
> > I am trying to learn VPython for more interesting simulation. I
> > downloaded it from the website: www.vpython.org <http://www.vpython.org>
> > While I have Python 3.2.3 in my computer, and it seems no equivalent
> > version for vpython is available. Besides when installing I had problem
> > in which the installer would not recognise the location of python, even
> > though it is located at C:\Python32.
> > What should I be doing?
>
> Sign up for their mailing list at
> https://lists.sourceforge.net/lists/listinfo/visualpython-users
> and ask them.
>
> Emile
>
>
>
> ------------------------------
>
> Message: 2
> Date: Wed, 04 Jul 2012 08:09:14 -0700
> From: Emile van Sebille <emile at fenx.com>
> To: tutor at python.org
> Subject: Re: [Tutor] Passing numeral result of a defined function to a
>         named   value
> Message-ID: <jt1mdi$knr$1 at dough.gmane.org>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 6/28/2012 11:51 PM Joseph Hines said...
> > Hello. Sorry to bother you, but I'm self-teaching myself python, and
> > having difficulty with a script I'm writing to auto-calc various derived
> > stats for a PnP game I am hosting soon.
> >
> > I need to pass a numeral result of a defined function to a "named" value
> > (not sure if that's what it's called)
> >
> > The offending segment of code is this:
> >
>
> ... and what's the offending error? (provide traceback details please)
>
> Do you mean to be defining a function within a try-except structure?  As
> it stands you'll get a syntax error.
>
> You've likely got an indetation issue on the line before except.
>
> providing int_mod as some integer and reworking it to:
>
> XP_Per_Level_Base = 20-int_mod
> XP_Per_Level_Limit = 5
> try:
>       def XP_Per_Level_Calc(XP_Per_Level_Base, XP_Per_Level_Limit):
>           if XP_Per_Level_Base < XP_Per_Level_Limit:
>               return XP_Per_Level_Limit
>           else:
>               return XP_Per_Level_Base
> except ValueError:
>       print "Script B0RK3D, Link Shall Come to Town."
>       print "send him a message at
> Link6746 at gmail.com<mailto:Link6746 at gmail.com>"
>
>
> XP_Per_Level = XP_Per_Level_Calc(XP_Per_Level_Base, XP_Per_Level_Limit)
>
> gets it going, dut all in all it looks to me like this is simply a max
> function and could be written more simply as:
>      max(XP_Per_Level_Base,XP_Per_Level_Limit)
>
> Emile
>
>
> >
> >     XP_Per_Level_Base = 20-int_mod
> >     XP_Per_Level_Limit = 5
> >     try:
> >          def XP_Per_Level_Calc(XP_Per_Level_Base, XP_Per_Level_Limit):
> >              if XP_Per_Level_Base < XP_Per_Level_Limit:
> >                  return XP_Per_Level_Limit
> >              else:
> >                  return XP_Per_Level_Base
> >     XP_Per_Level = XP_Per_Level_Calc(XP_Per_Level_Base,
> XP_Per_Level_Limit)
> >     except ValueError:
> >          print "Script B0RK3D, Link Shall Come to Town."
> >          print "send him a message at Link6746 at gmail.com
> >     <mailto:Link6746 at gmail.com>"
> >
> >     #Calcs XP per Level#
> >
> >
> > If you need the whole script,  I'd be glad to send it to you, just
> > promise me not to divulge too many details of it
> > (It's regarding a custom PnP system that I'm planning on releasing under
> > the Open Gaming License, and I'd prefer not to risk it becoming a
> > proprietary system for someone I don't know)
> >
> >
> >
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > http://mail.python.org/mailman/listinfo/tutor
>
>
>
>
> ------------------------------
>
> Message: 3
> Date: Wed, 04 Jul 2012 08:30:07 -0700
> From: Emile van Sebille <emile at fenx.com>
> To: tutor at python.org
> Subject: Re: [Tutor] for loop question
> Message-ID: <jt1nko$v1v$1 at dough.gmane.org>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 7/1/2012 2:50 PM Jim said...
> > Hello Friends,
> > I apologize for being such a bother. This problem has been evading me
> > all day. Can you please give me a hint as to why I cannot put the
> > variable UpperCaseSentence outside of the for loop?
> > I can do it in other instances but not in this one.
> > Thank you so much,
> > Jim
>
>
>
> You're not returning anything from fixCase -- change the final print to
> return and try it again.
>
> Emile
>
>
>
> >
> > #Main function.
> > def main():
> >     mySentence = (input("Enter text."))
> >     mySentenceList = mySentence.split('.')
> >
> >     #Call fixCase function. Send it mySentenceList and receive result
> >     #and stores result in variable named output.
> >     output = fixCase(mySentenceList)
> >     print(output)
> >
> > def fixCase(myList):
> >     #Begin making a loop through the list, using a variable myString
> >     for myString in range (len(myList)):
> >         tempString = myList[myString] #Store in temporary variable.
> >         myList[myString] = tempString[0:1].upper() +
> tempString[1:len(tempString)] #Replace with upper
> >         UpperCaseSentence = (myList[myString])
> >         print(UpperCaseSentence)
> >
> > #Call main function
> > main()
> >
>
>
>
> ------------------------------
>
> Message: 4
> Date: Wed, 4 Jul 2012 17:01:34 +0100
> From: Osemeka Osuagwu <abasiemeka at gmail.com>
> To: tutor at python.org
> Subject: [Tutor] Bothersome NoneType Error for a List object
> Message-ID:
>         <
> CAF33E7ZaobiNfaG4+Lyg218q6o4HwAGorZMzVYBR-i19rWhAqw at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi,
> I am trying to find the smallest positive number that is divisible by all
> of the numbers from 1 to 20 without a remainder. I wrote the following code
> (implementation of a solution method found in
> http://en.wikipedia.org/wiki/Least_common_multiple#A_method_using_a_table)
> but kept getting an error message (also posted) when I ran it. I can't
> figure out the problem. I would appreciate help with this. Also, any hint
> on how to make the code more elegant is welcome.
>
> Regards,
> Abasiemeka
>
> *CODE*
> def checkdiv(x, testlist):      #returns True if x can divide ANY member of
> testlist,returns False otherwise
>     for k in testlist:
>         if k%x == 0:
>             return True
>     return False
>
> def lcm(numlist):               #continuously divides numlist by each in a
> list of prime numbers till cannot
>     primeslist = [2, 3, 5, 7, 11]
>     templist = []
>
>     for prime in primeslist:
>         if checkdiv(prime, numlist) == True:
>             templist = templist.append(prime)
>             for i in range(0,len(numlist)):
>                 if numlist[i]%prime == 0:
>                     numlist[i] = numlist[i]/prime
>         else:
>             pass
>
>     lcm = reduce(lambda x, y: x*y, templist)     #my first lambda
> expression! (multiply all members of templist
>     return lcm
>
> print lcm([i for i in range(1, 21)])
>
> *ERROR OUTPUT*
> Traceback (most recent call last):
>   File "C:\Windows.old\Users\Abasiemeka\Abasiemeka\GOOGLE
> University\Python\Python Code\MyCode\Project Euler code\Project Euler
> answer 5.py", line 31, in <module>
>     print lcm(first20)
>   File "C:\Windows.old\Users\Abasiemeka\Abasiemeka\GOOGLE
> University\Python\Python Code\MyCode\Project Euler code\Project Euler
> answer 5.py", line 20, in lcm
>     templist = templist.append(prime)
> AttributeError: 'NoneType' object has no attribute 'append'
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/tutor/attachments/20120704/1cac9413/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 5
> Date: Thu, 05 Jul 2012 02:15:08 +1000
> From: Steven D'Aprano <steve at pearwood.info>
> To: tutor at python.org
> Subject: Re: [Tutor] Bothersome NoneType Error for a List object
> Message-ID: <4FF46C0C.4000509 at pearwood.info>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Osemeka Osuagwu wrote:
>
> >             templist = templist.append(prime)
>
> The append method operates in place, and returns None. It doesn't return a
> list:
>
> py> mylist = []
> py> x = mylist.append(42)
> py> x is None
> True
> py> mylist
> [42]
>
> Replace that line with just
>
> templist.append(prime)
>
>
> --
> Steven
>
>
> ------------------------------
>
> Message: 6
> Date: Wed, 04 Jul 2012 17:56:51 +0100
> From: Alan Gauld <alan.gauld at btinternet.com>
> To: tutor at python.org
> Subject: Re: [Tutor] Seeking help with reading and writing files in
>         Python
> Message-ID: <jt1skj$89p$1 at dough.gmane.org>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 26/06/12 19:28, Aristotle wrote:
>
> > with an assignment that asks me to read from one file and then write to
> > another file. The problem is that the file to read from has not been
> > created yet.
>
> So create it.
> Any old text editor will do.
> The whole point is, I believe that you are writing a program to read a
> file created by some other application, perhaps a web application or
> database reporting tool. (or in this case Notepad, say).
>
> How the file is created is irrelevant so long as it has the data that
> you want to extract within it.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
>
>
>
>
>
> ------------------------------
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
> End of Tutor Digest, Vol 101, Issue 11
> **************************************
>



-- 
*We have enough for our need but not enough for our greed.*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120704/2849ecad/attachment-0001.html>

From abasiemeka at gmail.com  Thu Jul  5 00:49:26 2012
From: abasiemeka at gmail.com (Osemeka Osuagwu)
Date: Wed, 4 Jul 2012 23:49:26 +0100
Subject: [Tutor] Tutor Digest, Vol 101, Issue 12
In-Reply-To: <mailman.4490.1341434321.4696.tutor@python.org>
References: <mailman.4490.1341434321.4696.tutor@python.org>
Message-ID: <CAF33E7YfTA_pwQhnwxCF+PDO_AKnovLXc476H5KVFw-pVJ2JvA@mail.gmail.com>

Hi Alan,
I appreciate the tip. Using the operator module does look better, the speed
is about the same too.

Thanks.
Abasiemeka

------------------------------
>
> Message: 3
> Date: Wed, 04 Jul 2012 18:13:36 +0100
> From: Alan Gauld <alan.gauld at btinternet.com>
> To: tutor at python.org
> Subject: Re: [Tutor] Bothersome NoneType Error for a List object
> Message-ID: <jt1tk0$fnd$1 at dough.gmane.org>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 04/07/12 17:01, Osemeka Osuagwu wrote:
>
> >      lcm = reduce(lambda x, y: x*y, templist)     #my first lambda
> > expression! (multiply all members of templist
>
> Congratulations :-)
>
> But you could have done:
>
> import operator
> lcm = reduce(operator.mul, templist)
>
> instead.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120704/dab0e47f/attachment.html>

From torkamani at gmail.com  Thu Jul  5 18:16:16 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Thu, 5 Jul 2012 12:16:16 -0400
Subject: [Tutor] Factor Analysis
Message-ID: <CAEmF0k7g_h_HdiOEBG17deTT6_zmqMf9CJ-NFmuowVGZ6YxDew@mail.gmail.com>

Dear Tutors,

Two quick questions:

1) Specifically, is there any built-in function for factor analysis is
python?

2) Generally how would you search, to find out if there exist an efficient
python package for some specific task? (Google search usually results in
many hardly-related *false* positives!)

Thanks a bunch!

A
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120705/0853ed1f/attachment.html>

From joel.goldstick at gmail.com  Thu Jul  5 19:06:07 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Thu, 5 Jul 2012 13:06:07 -0400
Subject: [Tutor] Factor Analysis
In-Reply-To: <CAEmF0k7g_h_HdiOEBG17deTT6_zmqMf9CJ-NFmuowVGZ6YxDew@mail.gmail.com>
References: <CAEmF0k7g_h_HdiOEBG17deTT6_zmqMf9CJ-NFmuowVGZ6YxDew@mail.gmail.com>
Message-ID: <CAPM-O+yQBYzQkobq_c+x1oCqW5WOcFTwUEk=Wt_phCH-mi7VMQ@mail.gmail.com>

On Thu, Jul 5, 2012 at 12:16 PM, Ali Torkamani <torkamani at gmail.com> wrote:
> Dear Tutors,
>
> Two quick questions:
>
> 1) Specifically, is there any built-in function for factor analysis is
> python?

It sure looks like there is:
https://www.google.com/search?q=Factor+analysis+python&ie=utf-8&oe=utf-8&client=ubuntu&channel=fs

It looks like MDP might do what you need
>
> 2) Generally how would you search, to find out if there exist an efficient
> python package for some specific task? (Google search usually results in
> many hardly-related false positives!)
>
> Thanks a bunch!
>
> A
>
>
> _______________________________________________
> Tutor maillist ?- ?Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick

From bgailer at gmail.com  Thu Jul  5 22:34:07 2012
From: bgailer at gmail.com (bob gailer)
Date: Thu, 05 Jul 2012 16:34:07 -0400
Subject: [Tutor] Factor Analysis
In-Reply-To: <CAEmF0k7g_h_HdiOEBG17deTT6_zmqMf9CJ-NFmuowVGZ6YxDew@mail.gmail.com>
References: <CAEmF0k7g_h_HdiOEBG17deTT6_zmqMf9CJ-NFmuowVGZ6YxDew@mail.gmail.com>
Message-ID: <4FF5FA3F.6030309@gmail.com>

On 7/5/2012 12:16 PM, Ali Torkamani wrote:
> 1) Specifically, is there any built-in function for factor analysis is 
> python?
No built-in, but take a look at

the SAGE project, which is an open-source alterative to Magma / Maple / Mathematica / Matlab:
	http://www.sagemath.org/
Sage is a funky, through-the-web console that uses Python to drive lots of math software. Note that there's a public server so you can easily take it for a spin:
	http://www.sagenb.org/


-- 
Bob Gailer
919-636-4239
Chapel Hill NC


From steve at pearwood.info  Fri Jul  6 05:30:16 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 06 Jul 2012 13:30:16 +1000
Subject: [Tutor] Factor Analysis
In-Reply-To: <CAEmF0k7g_h_HdiOEBG17deTT6_zmqMf9CJ-NFmuowVGZ6YxDew@mail.gmail.com>
References: <CAEmF0k7g_h_HdiOEBG17deTT6_zmqMf9CJ-NFmuowVGZ6YxDew@mail.gmail.com>
Message-ID: <4FF65BC8.5050407@pearwood.info>

Ali Torkamani wrote:

> 2) Generally how would you search, to find out if there exist an efficient
> python package for some specific task? (Google search usually results in
> many hardly-related *false* positives!)

Start with PyPI, also known as the CheeseShop.

http://pypi.python.org/pypi

It's not quite as powerful as Perl's CPAN, but it's still pretty good.

Then follow up with Google and other search engines. I prefer Duck Duck Go 
because they don't track you and they take privacy seriously:

http://duckduckgo.com/

Also they have dedicated syntax for searching for Python related information.

Finally, ask on Python forums, particularly the comp.lang.python newsgroup 
(also available as python-list at python.org) which is more general than this 
email list. This list is more specialised for learning Python the language.


-- 
Steven

From torkamani at gmail.com  Fri Jul  6 17:47:58 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Fri, 6 Jul 2012 11:47:58 -0400
Subject: [Tutor] Program gets stuck after a creating a list from dictinary
	items!
Message-ID: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>

Dear Tutors,
I'm trying to write a dictionary into a csv file, in the following code
FlatData is the global dictionary whose keys are datetime objects, and the
values are list of dictionaries. sCommonFeatures are key's that exist in
all of such 'sub'-dictionaries, and sOtherFeatures are the key's that are
in some of sub-dictionaries.

The problem is that in line : D=[prog[key1] for key1 in sCommonFeatures]
(line 10 below), the program stucks!

I appreciate any help.

def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
    FD=FlatData[date]

A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures)
    pdb.set_trace()
    with
open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb') as
myfile:
        fout = csv.writer(myfile, delimiter=',', quotechar="'")

        fout.writerow(A)
        for prog in FD:
            D=[prog[key1] for key1 in sCommonFeatures]
            print(prog)
            pdb.set_trace()
            diff=sOtherFeatures.difference(prog.keys())
            prog.update({x:'NONE' for x in diff})

            C=[prog[key2] for key2 in sOtherFeatures]
            A=["%4d%02d%02d"
%(date.year,date.month,date.day),date.year,date.month,date.day]
            fout.writerow(A+D+C)
            pdb.set_trace()
        myfile.flush()
        myfile.close()
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120706/9c54b379/attachment.html>

From ramit.prasad at jpmorgan.com  Fri Jul  6 18:22:31 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Fri, 6 Jul 2012 16:22:31 +0000
Subject: [Tutor] Program gets stuck after a creating a list from
 dictinary	items!
In-Reply-To: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>

From: Ali Torkamani
> I'm trying to write a dictionary into a csv file, in the following code
> FlatData is the global dictionary whose keys are datetime objects, and the
> values are list of dictionaries. sCommonFeatures are key's that exist in all
> of such 'sub'-dictionaries, and sOtherFeatures are the key's that are in some
> of sub-dictionaries.
> 
> The problem is that in line : D=[prog[key1] for key1 in sCommonFeatures]
> (line 10 below), the program stucks!

Gets stuck how? I see nothing in that line that looks invalid. It may just
take awhile depending on the amount of keys it has to go through. Maybe
you just need to wait longer?

> 
> I appreciate any help.
> 
> def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
>     FD=FlatData[date]
> 
> A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures
> )
>     pdb.set_trace()
>     with
> open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb') as
> myfile:
>         fout = csv.writer(myfile, delimiter=',', quotechar="'")
> 
>         fout.writerow(A)
>         for prog in FD:

Try inserting a print here to make sure FD is not an empty list. 

>             D=[prog[key1] for key1 in sCommonFeatures]
>             print(prog)
>             pdb.set_trace()
>             diff=sOtherFeatures.difference(prog.keys())
>             prog.update({x:'NONE' for x in diff})
> 
>             C=[prog[key2] for key2 in sOtherFeatures]
>             A=["%4d%02d%02d"
> %(date.year,date.month,date.day),date.year,date.month,date.day]
>             fout.writerow(A+D+C)
>             pdb.set_trace()
>         myfile.flush()
>         myfile.close()

There is no need to flush/close a file that is opened using "with".

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--


From torkamani at gmail.com  Fri Jul  6 18:27:17 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Fri, 6 Jul 2012 12:27:17 -0400
Subject: [Tutor] Program gets stuck after a creating a list from
	dictinary items!
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
Message-ID: <CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>

Thanks, I checked,  FD is not empty,
Thanks for reminding about the flush and close, but I think some thing is
wrong with:
D=[prog[key1] for key1 in sCommonFeatures]

In the debug mode it works fine from the command line, but in the for loop
it gets stuck.

Ali

On Fri, Jul 6, 2012 at 12:22 PM, Prasad, Ramit <ramit.prasad at jpmorgan.com>wrote:

> From: Ali Torkamani
> > I'm trying to write a dictionary into a csv file, in the following code
> > FlatData is the global dictionary whose keys are datetime objects, and
> the
> > values are list of dictionaries. sCommonFeatures are key's that exist in
> all
> > of such 'sub'-dictionaries, and sOtherFeatures are the key's that are in
> some
> > of sub-dictionaries.
> >
> > The problem is that in line : D=[prog[key1] for key1 in sCommonFeatures]
> > (line 10 below), the program stucks!
>
> Gets stuck how? I see nothing in that line that looks invalid. It may just
> take awhile depending on the amount of keys it has to go through. Maybe
> you just need to wait longer?
>
> >
> > I appreciate any help.
> >
> > def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
> >     FD=FlatData[date]
> >
> >
> A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures
> > )
> >     pdb.set_trace()
> >     with
> > open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb') as
> > myfile:
> >         fout = csv.writer(myfile, delimiter=',', quotechar="'")
> >
> >         fout.writerow(A)
> >         for prog in FD:
>
> Try inserting a print here to make sure FD is not an empty list.
>
> >             D=[prog[key1] for key1 in sCommonFeatures]
> >             print(prog)
> >             pdb.set_trace()
> >             diff=sOtherFeatures.difference(prog.keys())
> >             prog.update({x:'NONE' for x in diff})
> >
> >             C=[prog[key2] for key2 in sOtherFeatures]
> >             A=["%4d%02d%02d"
> > %(date.year,date.month,date.day),date.year,date.month,date.day]
> >             fout.writerow(A+D+C)
> >             pdb.set_trace()
> >         myfile.flush()
> >         myfile.close()
>
> There is no need to flush/close a file that is opened using "with".
>
> Ramit
>
>
> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
> 712 Main Street | Houston, TX 77002
> work phone: 713 - 216 - 5423
>
> --
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120706/181ae697/attachment.html>

From torkamani at gmail.com  Fri Jul  6 18:28:12 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Fri, 6 Jul 2012 12:28:12 -0400
Subject: [Tutor] Program gets stuck after a creating a list from
	dictinary items!
In-Reply-To: <CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>
Message-ID: <CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>

BTW I changed it to:
D=[prog[key1] for key1 in list(sCommonFeatures)]

because sCommonFeatures is actually a set, but it still has the smae problem

On Fri, Jul 6, 2012 at 12:27 PM, Ali Torkamani <torkamani at gmail.com> wrote:

> Thanks, I checked,  FD is not empty,
> Thanks for reminding about the flush and close, but I think some thing is
> wrong with:
>
> D=[prog[key1] for key1 in sCommonFeatures]
>
> In the debug mode it works fine from the command line, but in the for loop
> it gets stuck.
>
> Ali
>
>
> On Fri, Jul 6, 2012 at 12:22 PM, Prasad, Ramit <ramit.prasad at jpmorgan.com>wrote:
>
>> From: Ali Torkamani
>> > I'm trying to write a dictionary into a csv file, in the following code
>> > FlatData is the global dictionary whose keys are datetime objects, and
>> the
>> > values are list of dictionaries. sCommonFeatures are key's that exist
>> in all
>> > of such 'sub'-dictionaries, and sOtherFeatures are the key's that are
>> in some
>> > of sub-dictionaries.
>> >
>> > The problem is that in line : D=[prog[key1] for key1 in sCommonFeatures]
>> > (line 10 below), the program stucks!
>>
>> Gets stuck how? I see nothing in that line that looks invalid. It may just
>> take awhile depending on the amount of keys it has to go through. Maybe
>> you just need to wait longer?
>>
>> >
>> > I appreciate any help.
>> >
>> > def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
>> >     FD=FlatData[date]
>> >
>> >
>> A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures
>> > )
>> >     pdb.set_trace()
>> >     with
>> > open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb')
>> as
>> > myfile:
>> >         fout = csv.writer(myfile, delimiter=',', quotechar="'")
>> >
>> >         fout.writerow(A)
>> >         for prog in FD:
>>
>> Try inserting a print here to make sure FD is not an empty list.
>>
>> >             D=[prog[key1] for key1 in sCommonFeatures]
>> >             print(prog)
>> >             pdb.set_trace()
>> >             diff=sOtherFeatures.difference(prog.keys())
>> >             prog.update({x:'NONE' for x in diff})
>> >
>> >             C=[prog[key2] for key2 in sOtherFeatures]
>> >             A=["%4d%02d%02d"
>> > %(date.year,date.month,date.day),date.year,date.month,date.day]
>> >             fout.writerow(A+D+C)
>> >             pdb.set_trace()
>> >         myfile.flush()
>> >         myfile.close()
>>
>> There is no need to flush/close a file that is opened using "with".
>>
>> Ramit
>>
>>
>> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
>> 712 Main Street | Houston, TX 77002
>> work phone: 713 - 216 - 5423
>>
>> --
>>
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/mailman/listinfo/tutor
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120706/7a9e8efe/attachment-0001.html>

From torkamani at gmail.com  Fri Jul  6 18:32:04 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Fri, 6 Jul 2012 12:32:04 -0400
Subject: [Tutor] Program gets stuck after a creating a list from
	dictinary items!
In-Reply-To: <CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>
	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
Message-ID: <CAEmF0k4sx9qQBZEiM+PZcQ6bzjCYsGLaeabazb_xV82puo+BSQ@mail.gmail.com>

I could resolve it by defining a small function:

def getValue(mydict,keys):
    A=[];
    for i in keys:
        A=A+[mydict[i]]
    return A

and then calling it: D=getValue(prog,sCommonFeatures);
(instead of D=[prog[key1] for key1 in list(sCommonFeatures)];)

but I'm still surprised why the latter one didn't work!


On Fri, Jul 6, 2012 at 12:28 PM, Ali Torkamani <torkamani at gmail.com> wrote:
>
> BTW I changed it to:
> D=[prog[key1] for key1 in list(sCommonFeatures)]
>
> because sCommonFeatures is actually a set, but it still has the smae
problem
>
>
> On Fri, Jul 6, 2012 at 12:27 PM, Ali Torkamani <torkamani at gmail.com>
wrote:
>>
>> Thanks, I checked,  FD is not empty,
>> Thanks for reminding about the flush and close, but I think some thing
is wrong with:
>>
>> D=[prog[key1] for key1 in sCommonFeatures]
>>
>> In the debug mode it works fine from the command line, but in the for
loop it gets stuck.
>>
>> Ali
>>
>>
>> On Fri, Jul 6, 2012 at 12:22 PM, Prasad, Ramit <ramit.prasad at jpmorgan.com>
wrote:
>>>
>>> From: Ali Torkamani
>>> > I'm trying to write a dictionary into a csv file, in the following
code
>>> > FlatData is the global dictionary whose keys are datetime objects,
and the
>>> > values are list of dictionaries. sCommonFeatures are key's that exist
in all
>>> > of such 'sub'-dictionaries, and sOtherFeatures are the key's that are
in some
>>> > of sub-dictionaries.
>>> >
>>> > The problem is that in line : D=[prog[key1] for key1 in
sCommonFeatures]
>>> > (line 10 below), the program stucks!
>>>
>>> Gets stuck how? I see nothing in that line that looks invalid. It may
just
>>> take awhile depending on the amount of keys it has to go through. Maybe
>>> you just need to wait longer?
>>>
>>> >
>>> > I appreciate any help.
>>> >
>>> > def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
>>> >     FD=FlatData[date]
>>> >
>>> >
A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures
>>> > )
>>> >     pdb.set_trace()
>>> >     with
>>> >
open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb') as
>>> > myfile:
>>> >         fout = csv.writer(myfile, delimiter=',', quotechar="'")
>>> >
>>> >         fout.writerow(A)
>>> >         for prog in FD:
>>>
>>> Try inserting a print here to make sure FD is not an empty list.
>>>
>>> >             D=[prog[key1] for key1 in sCommonFeatures]
>>> >             print(prog)
>>> >             pdb.set_trace()
>>> >             diff=sOtherFeatures.difference(prog.keys())
>>> >             prog.update({x:'NONE' for x in diff})
>>> >
>>> >             C=[prog[key2] for key2 in sOtherFeatures]
>>> >             A=["%4d%02d%02d"
>>> > %(date.year,date.month,date.day),date.year,date.month,date.day]
>>> >             fout.writerow(A+D+C)
>>> >             pdb.set_trace()
>>> >         myfile.flush()
>>> >         myfile.close()
>>>
>>> There is no need to flush/close a file that is opened using "with".
>>>
>>> Ramit
>>>
>>>
>>> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
>>> 712 Main Street | Houston, TX 77002
>>> work phone: 713 - 216 - 5423
>>>
>>> --
>>>
>>> _______________________________________________
>>> Tutor maillist  -  Tutor at python.org
>>> To unsubscribe or change subscription options:
>>> http://mail.python.org/mailman/listinfo/tutor
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120706/3867f974/attachment.html>

From ramit.prasad at jpmorgan.com  Fri Jul  6 18:54:55 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Fri, 6 Jul 2012 16:54:55 +0000
Subject: [Tutor] Program gets stuck after a creating a list from
 dictinary items!
In-Reply-To: <CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>
	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>

Please do not top post.

> 
> BTW I changed it to:
> D=[prog[key1] for key1 in list(sCommonFeatures)]
> 
> because sCommonFeatures is actually a set, but it still has the smae problem
> On Fri, Jul 6, 2012 at 12:27 PM, Ali Torkamani <torkamani at gmail.com> wrote:
> Thanks, I checked,  FD is not empty,
> Thanks for reminding about the flush and close, but I think some thing is
> wrong with:
> 
> D=[prog[key1] for key1 in sCommonFeatures]
> In the debug mode it works fine from the command line, but in the for loop it
> gets stuck.

I believe that the usage of 'in <blah>' converts it into a set (or 
set-like) object so probably that is the same as set(list(set())).

Again what do you mean by "stuck"? Does it error or do you just wait? 
Can you provide a small sample script (including sample sCommonFeatures and FD) ?
What version of Python / operating system?

> I could resolve it by defining a small function:
> 
> def getValue(mydict,keys):
>     A=[];
>     for i in keys:
>         A=A+[mydict[i]]
>     return A
> 
> and then calling it: D=getValue(prog,sCommonFeatures);
> (instead of D=[prog[key1] for key1 in list(sCommonFeatures)];)
> 
> but I'm still surprised why the latter one didn't work!

It would be more efficient to do the following

def getValue(mydict, keys):
    A=[]
    for i in keys:
        A.append( mydict[i] )
    return A


Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From ramit.prasad at jpmorgan.com  Fri Jul  6 19:04:40 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Fri, 6 Jul 2012 17:04:40 +0000
Subject: [Tutor] Program gets stuck after a creating a list from
 dictinary items!
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>
	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416544E04@SCACMX008.exchad.jpmchase.net>

> > I could resolve it by defining a small function:
> >
> > def getValue(mydict,keys):
> >     A=[];
> >     for i in keys:
> >         A=A+[mydict[i]]
> >     return A
> >
> > and then calling it: D=getValue(prog,sCommonFeatures);
> > (instead of D=[prog[key1] for key1 in list(sCommonFeatures)];)
> >
> > but I'm still surprised why the latter one didn't work!
> 
> It would be more efficient to do the following
> 
> def getValue(mydict, keys):
>     A=[]
>     for i in keys:
>         A.append( mydict[i] )
>     return A

Less efficiently, but maybe more useful if you need to check
against sOtherFeatures you can do the following.

D = [value for key, value in prog.items() if key in sOtherFeatures 
          or key in sCommonFeatures] # Python 3.x syntax


Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From torkamani at gmail.com  Fri Jul  6 19:21:27 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Fri, 6 Jul 2012 13:21:27 -0400
Subject: [Tutor] Program gets stuck after a creating a list from
	dictinary items!
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF47416544E04@SCACMX008.exchad.jpmchase.net>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>
	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>
	<5B80DD153D7D744689F57F4FB69AF47416544E04@SCACMX008.exchad.jpmchase.net>
Message-ID: <CAEmF0k4XuR7RYQ4fgyhNeosOWSXq+=S56fsz1QR_49Z=rD-Mdw@mail.gmail.com>

I'm using Python 2.7.
By stuck I mean, does not pass that specific line (but no errors).

Like I said FD is a list of dictionaries. and sCommonFeatures are a subset
of key's that have value in all of the dictionaries.



On Fri, Jul 6, 2012 at 1:04 PM, Prasad, Ramit <ramit.prasad at jpmorgan.com>wrote:

> > > I could resolve it by defining a small function:
> > >
> > > def getValue(mydict,keys):
> > >     A=[];
> > >     for i in keys:
> > >         A=A+[mydict[i]]
> > >     return A
> > >
> > > and then calling it: D=getValue(prog,sCommonFeatures);
> > > (instead of D=[prog[key1] for key1 in list(sCommonFeatures)];)
> > >
> > > but I'm still surprised why the latter one didn't work!
> >
> > It would be more efficient to do the following
> >
> > def getValue(mydict, keys):
> >     A=[]
> >     for i in keys:
> >         A.append( mydict[i] )
> >     return A
>
> Less efficiently, but maybe more useful if you need to check
> against sOtherFeatures you can do the following.
>
> D = [value for key, value in prog.items() if key in sOtherFeatures
>           or key in sCommonFeatures] # Python 3.x syntax
>
>
> Ramit
>
>
> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
> 712 Main Street | Houston, TX 77002
> work phone: 713 - 216 - 5423
>
> --
> This email is confidential and subject to important disclaimers and
> conditions including on offers for the purchase or sale of
> securities, accuracy and completeness of information, viruses,
> confidentiality, legal privilege, and legal entity disclaimers,
> available at http://www.jpmorgan.com/pages/disclosures/email.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120706/3e927c8d/attachment-0001.html>

From alan.gauld at btinternet.com  Fri Jul  6 21:57:05 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 06 Jul 2012 20:57:05 +0100
Subject: [Tutor] Program gets stuck after a creating a list from
	dictinary items!
In-Reply-To: <CAEmF0k4XuR7RYQ4fgyhNeosOWSXq+=S56fsz1QR_49Z=rD-Mdw@mail.gmail.com>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>
	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>
	<5B80DD153D7D744689F57F4FB69AF47416544E04@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XuR7RYQ4fgyhNeosOWSXq+=S56fsz1QR_49Z=rD-Mdw@mail.gmail.com>
Message-ID: <jt7fuh$k90$1@dough.gmane.org>

On 06/07/12 18:21, Ali Torkamani wrote:
> I'm using Python 2.7.
> By stuck I mean, does not pass that specific line (but no errors).
>

How do you know that is the line it is stuck on?

def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
     FD=FlatData[date]
 
A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures)
     pdb.set_trace()
     with 
open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb') 
as myfile:
         fout = csv.writer(myfile, delimiter=',', quotechar="'")

         fout.writerow(A)
         for prog in FD:
             D=[prog[key1] for key1 in sCommonFeatures]

How do you know its not one of the lines above?

Also what are you using the pdb traces for?
Do they show any output?

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From torkamani at gmail.com  Fri Jul  6 23:16:46 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Fri, 6 Jul 2012 17:16:46 -0400
Subject: [Tutor] Program gets stuck after a creating a list from
	dictinary items!
In-Reply-To: <jt7fuh$k90$1@dough.gmane.org>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>
	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>
	<5B80DD153D7D744689F57F4FB69AF47416544E04@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XuR7RYQ4fgyhNeosOWSXq+=S56fsz1QR_49Z=rD-Mdw@mail.gmail.com>
	<jt7fuh$k90$1@dough.gmane.org>
Message-ID: <CAEmF0k5pgCZ4GpM1oy-wB4RNxh67bVL6o5D32MX9bfYYb0SkmQ@mail.gmail.com>

Actually printing some thing after that line does not show any thing, I
have the pdb.set_trace()'s for debugging, to see what's going on inside. ho
would you debug that?

On Fri, Jul 6, 2012 at 3:57 PM, Alan Gauld <alan.gauld at btinternet.com>wrote:

> On 06/07/12 18:21, Ali Torkamani wrote:
>
>> I'm using Python 2.7.
>> By stuck I mean, does not pass that specific line (but no errors).
>>
>>
> How do you know that is the line it is stuck on?
>
>
> def WriteOneDayToCSV(**sCommonFeatures,**sOtherFeatures,date):
>     FD=FlatData[date]
>
> A=['UniqeDate','Year','Month',**'Day']+list(sCommonFeatures)+**
> list(sOtherFeatures)
>     pdb.set_trace()
>     with open(str(date.year)+"_"+str(**date.month)+"_"+str(date.day)+**".csv",'wb')
> as myfile:
>         fout = csv.writer(myfile, delimiter=',', quotechar="'")
>
>         fout.writerow(A)
>         for prog in FD:
>             D=[prog[key1] for key1 in sCommonFeatures]
>
> How do you know its not one of the lines above?
>
> Also what are you using the pdb traces for?
> Do they show any output?
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
>
>
>
>
> ______________________________**_________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120706/9f736fa8/attachment.html>

From breamoreboy at yahoo.co.uk  Sat Jul  7 00:09:46 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 06 Jul 2012 23:09:46 +0100
Subject: [Tutor] Program gets stuck after a creating a list from
	dictinary items!
In-Reply-To: <CAEmF0k5pgCZ4GpM1oy-wB4RNxh67bVL6o5D32MX9bfYYb0SkmQ@mail.gmail.com>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>
	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>
	<5B80DD153D7D744689F57F4FB69AF47416544E04@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XuR7RYQ4fgyhNeosOWSXq+=S56fsz1QR_49Z=rD-Mdw@mail.gmail.com>
	<jt7fuh$k90$1@dough.gmane.org>
	<CAEmF0k5pgCZ4GpM1oy-wB4RNxh67bVL6o5D32MX9bfYYb0SkmQ@mail.gmail.com>
Message-ID: <jt7njl$9f8$1@dough.gmane.org>

Please stop top posting, you have been asked at least once before, no 
further comments, TIA.

On 06/07/2012 22:16, Ali Torkamani wrote:
> Actually printing some thing after that line does not show any thing, I
> have the pdb.set_trace()'s for debugging, to see what's going on inside. ho
> would you debug that?
>
> On Fri, Jul 6, 2012 at 3:57 PM, Alan Gauld <alan.gauld at btinternet.com>wrote:
>
>> On 06/07/12 18:21, Ali Torkamani wrote:
>>
>>> I'm using Python 2.7.
>>> By stuck I mean, does not pass that specific line (but no errors).
>>>
>>>
>> How do you know that is the line it is stuck on?
>>
>>
>> def WriteOneDayToCSV(**sCommonFeatures,**sOtherFeatures,date):
>>      FD=FlatData[date]
>>
>> A=['UniqeDate','Year','Month',**'Day']+list(sCommonFeatures)+**
>> list(sOtherFeatures)
>>      pdb.set_trace()
>>      with open(str(date.year)+"_"+str(**date.month)+"_"+str(date.day)+**".csv",'wb')
>> as myfile:
>>          fout = csv.writer(myfile, delimiter=',', quotechar="'")
>>
>>          fout.writerow(A)
>>          for prog in FD:
>>              D=[prog[key1] for key1 in sCommonFeatures]
>>
>> How do you know its not one of the lines above?
>>
>> Also what are you using the pdb traces for?
>> Do they show any output?
>>
>> --
>> Alan G
>> Author of the Learn to Program web site
>> http://www.alan-g.me.uk/
>>
>>
>>
>>
>> ______________________________**_________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>>
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>


-- 
Cheers.

Mark Lawrence.




From ramit.prasad at jpmorgan.com  Sat Jul  7 00:17:32 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Fri, 6 Jul 2012 22:17:32 +0000
Subject: [Tutor] Program gets stuck after a creating a list
	from	dictinary items!
In-Reply-To: <CAEmF0k5pgCZ4GpM1oy-wB4RNxh67bVL6o5D32MX9bfYYb0SkmQ@mail.gmail.com>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>
	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>
	<5B80DD153D7D744689F57F4FB69AF47416544E04@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XuR7RYQ4fgyhNeosOWSXq+=S56fsz1QR_49Z=rD-Mdw@mail.gmail.com>
	<jt7fuh$k90$1@dough.gmane.org>
	<CAEmF0k5pgCZ4GpM1oy-wB4RNxh67bVL6o5D32MX9bfYYb0SkmQ@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416545229@SCACMX008.exchad.jpmchase.net>

> > Also what are you using the pdb traces for?
> > Do they show any output?
> Actually printing some thing after that line does not show any thing, I have the pdb.set_trace()'s for debugging, to see what's going on inside. ho would you debug that?

When debugging, I usually just add more print or logging statements.

             print('Calculating D for prog {0}'.format(prog))
             D=[prog[key1] for key1 in sCommonFeatures]
             print('D calculated to be {0}'.format(D))

This lets me see the inputs and output of a block of code.
And for longer algorithms I would put more print statements
to track what is occurring. For example, if you are in a loop, 
it can be useful to use enumerate() so you have a loop counter
and can see which iteration the problem is occurring on. 


But I have not used pdb very much.

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From alan.gauld at btinternet.com  Sat Jul  7 01:06:28 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 07 Jul 2012 00:06:28 +0100
Subject: [Tutor] Program gets stuck after a creating a list from
	dictinary items!
In-Reply-To: <CAEmF0k5pgCZ4GpM1oy-wB4RNxh67bVL6o5D32MX9bfYYb0SkmQ@mail.gmail.com>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>
	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>
	<5B80DD153D7D744689F57F4FB69AF47416544E04@SCACMX008.exchad.jpmchase.net>
	<CAEmF0k4XuR7RYQ4fgyhNeosOWSXq+=S56fsz1QR_49Z=rD-Mdw@mail.gmail.com>
	<jt7fuh$k90$1@dough.gmane.org>
	<CAEmF0k5pgCZ4GpM1oy-wB4RNxh67bVL6o5D32MX9bfYYb0SkmQ@mail.gmail.com>
Message-ID: <jt7r1k$ils$1@dough.gmane.org>

On 06/07/12 22:16, Ali Torkamani wrote:
> Actually printing some thing after that line does not show any thing,

OK, But it could be any of the lines above the print.
Why are you so sure its that line? You are probably correct but I'd like 
to know why you are so sure? What have you done to isolate the problem?

> have the pdb.set_trace()'s for debugging, to see what's going on inside.

So what does it show?

If you aren't using it get rid of it. The less distractions there are 
the easier it is to debug things.

> ho would you debug that?

I'd either add some more print statements or I'd run it inside a 
debugger. Winpdb is a good example. I'd set a break point at the loop 
then single step through the loop the first couple of times, then set 
one or more breakpoints inside the loop and run to those.
But my first choice would be the print statements.

One just before the loop, one just inside the loop showing the loop 
variable value so I know how often I've gone round.

HTH,

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From steve at pearwood.info  Sat Jul  7 05:31:11 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 07 Jul 2012 13:31:11 +1000
Subject: [Tutor] Program gets stuck after a creating a list from
 dictinary items!
In-Reply-To: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>
Message-ID: <4FF7AD7F.1080904@pearwood.info>

Ali Torkamani wrote:
> Dear Tutors,
> I'm trying to write a dictionary into a csv file, in the following code
> FlatData is the global dictionary whose keys are datetime objects, and the
> values are list of dictionaries. sCommonFeatures are key's that exist in
> all of such 'sub'-dictionaries, and sOtherFeatures are the key's that are
> in some of sub-dictionaries.
> 
> The problem is that in line : D=[prog[key1] for key1 in sCommonFeatures]
> (line 10 below), the program stucks!
> 
> I appreciate any help.


Start by spending some time preparing some sample code we can actually run. 
The code you supply has inconsistent indentation, which means we have to 
*guess* what the indentation is supposed to be. Even if we can guess, we have 
to guess what your data is, which is impossible.

You should try to prepare a minimal working example that anyone can run and 
that demonstrates the problem. Take care to keep your maximum line length 
under 79 characters, so that mail clients won't word-wrap it. See here for 
more information:

http://sscce.org/

>         for prog in FD:
>             D=[prog[key1] for key1 in sCommonFeatures]
>             print(prog)

So FD is a list of dictionaries? What does FD mean? You should have more 
descriptive names.

And prog is not actually a prog(ram), but a dictionary? You *really* should 
have more descriptive names.

Assuming that all the data you are using are standard, built-in lists and 
dicts, I can't see anything that could be causing this problem. Are you using 
any custom subclasses? If so, perhaps there is a bug in one of your 
__getitem__ methods which enters an infinite loop.

How many values are in sCommonFeatures? If there are tens of millions, perhaps 
you are running out of memory and your operating system is trying to page data 
in and out of virtual memory. That can take hours or even days(!), depending 
on how little physical memory you have and how big the list gets.

The symptom of that is that your entire computer basically becomes unresponsive.

What happens if you cut your input data all the way back to a trivially small 
set of values, say, two keys only in sCommonFeatures, prog only having exactly 
two values, exactly one prog in FD? Does the code still get stuck?

If you add another print statement just before the list comprehension, you can 
see whether or not it actually is the list comp that is getting stuck.

What happens if you take out *all* the pdb calls? Not just in your code 
sample, but everywhere in the entire program. Maybe something in the debugger 
is interfering with the list comp.



-- 
Steven

From steve at pearwood.info  Sat Jul  7 05:35:15 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 07 Jul 2012 13:35:15 +1000
Subject: [Tutor] Program gets stuck after a creating a list
 from	dictinary items!
In-Reply-To: <jt7r1k$ils$1@dough.gmane.org>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>	<5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>	<5B80DD153D7D744689F57F4FB69AF47416544E04@SCACMX008.exchad.jpmchase.net>	<CAEmF0k4XuR7RYQ4fgyhNeosOWSXq+=S56fsz1QR_49Z=rD-Mdw@mail.gmail.com>	<jt7fuh$k90$1@dough.gmane.org>	<CAEmF0k5pgCZ4GpM1oy-wB4RNxh67bVL6o5D32MX9bfYYb0SkmQ@mail.gmail.com>
	<jt7r1k$ils$1@dough.gmane.org>
Message-ID: <4FF7AE73.50503@pearwood.info>

Alan Gauld wrote:
> On 06/07/12 22:16, Ali Torkamani wrote:

>> have the pdb.set_trace()'s for debugging, to see what's going on inside.
> 
> So what does it show?
> 
> If you aren't using it get rid of it. The less distractions there are 
> the easier it is to debug things.

I find it ironic that you are suggesting removing the debugger to make it 
easier to debug the code :)

(Not that I entirely disagree -- I find that 9 times out of 10 it's easier to 
debug code just using print statements and the interactive interpreter than 
with the debugger.)


>> ho would you debug that?
> 
> I'd either add some more print statements or I'd run it inside a 
> debugger.

Which is what pdb is :)

Nevertheless, if the Original Poster is not actually using the debugger, he 
should remove the debugger calls.


-- 
Steven

From steve at pearwood.info  Sat Jul  7 05:51:55 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 07 Jul 2012 13:51:55 +1000
Subject: [Tutor] Program gets stuck after a creating a list
 from	dictinary items!
In-Reply-To: <CAEmF0k4sx9qQBZEiM+PZcQ6bzjCYsGLaeabazb_xV82puo+BSQ@mail.gmail.com>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
	<CAEmF0k4sx9qQBZEiM+PZcQ6bzjCYsGLaeabazb_xV82puo+BSQ@mail.gmail.com>
Message-ID: <4FF7B25B.6090307@pearwood.info>

Ali Torkamani wrote:
> I could resolve it by defining a small function:
> 
> def getValue(mydict,keys):
>     A=[];
>     for i in keys:
>         A=A+[mydict[i]]
>     return A


That will be terribly, painfully inefficient for large amounts of data. Do you 
understand Big O notation? The above is O(N**2), compared to O(N) for a list 
comprehension. That means that the amount of work it does increases like the 
square of the number of keys:

- if you double the number of keys, the list comp will do twice as much
   work, but getValue will do four times as much work;

- if you increase the number of keys by 100, the list comp has to do
   100 times more work, but getValue has to do ten thousand times more!


You really want to avoid O(N**2) algorithms.

http://www.joelonsoftware.com/articles/fog0000000319.html

You can fix that and make it O(N) by using list append instead:

def getValue(mydict, keys):
     A=[];
     for key in keys:
         A.append(mydict[key])
     return A

which is a literally translation of the list comprehension
A = [mydict[key] for key in keys]



P.S. please improve your naming conventions. If you have a list of keys, you 
should not use "i" for each key. "i" should only be used for iterating over 
lists of integers.



-- 
Steven

From steve at pearwood.info  Sat Jul  7 06:12:21 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 07 Jul 2012 14:12:21 +1000
Subject: [Tutor] Program gets stuck after a creating a list from
 dictinary items!
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>
Message-ID: <4FF7B725.9020901@pearwood.info>

Prasad, Ramit wrote:

> I believe that the usage of 'in <blah>' converts it into a set (or 
> set-like) object so probably that is the same as set(list(set())).

No, certainly not. That would be terribly inefficient, since it means first 
iterating over blah entirely to convert it into a set, and then iterating over 
the set. That does twice as much work as just iterating over blah once.

Besides, sets cannot contain duplicates. If you convert a list to a set, you 
lose any duplicate values in the set.


What 'for x in <blah>' does is first try to use the iterator protocol, and if 
that doesn't work, it tries the sequence protocol, and if that doesn't work it 
raises TypeError.

The iterator protocol looks something like this, implemented in fast C code:


try:
     it = iter(blah)  # build an iterator object from blah
except TypeError:
     fall back on Sequence Protocol
else:
     try:
         while True:  # loop forever
             x = next(it)  # ask the iterator object for the next value
             process x
     except StopIteration:
         for loop is now done


The advantage here is that building an iterator object need not walk over the 
entire data structure in advance. Iterators should walk over it lazily, only 
as needed, rather than all at once like converting into a set requires.


The Sequence Protocol is the older version, going *way* back to the Python 1.x 
series, again implemented in fast C code. It looks something like this:

i = 0
while True:  # loop forever
     try:
         x = blah[i]  # get the first item, the second, third, fourth, ...
         process x
         i += 1
     except IndexError:
         for loop is now done




-- 
Steven

From alan.gauld at btinternet.com  Sat Jul  7 10:34:10 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 07 Jul 2012 09:34:10 +0100
Subject: [Tutor] Program gets stuck after a creating a list from
	dictinary items!
In-Reply-To: <4FF7AE73.50503@pearwood.info>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>	<5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>	<5B80DD153D7D744689F57F4FB69AF47416544E04@SCACMX008.exchad.jpmchase.net>	<CAEmF0k4XuR7RYQ4fgyhNeosOWSXq+=S56fsz1QR_49Z=rD-Mdw@mail.gmail.com>	<jt7fuh$k90$1@dough.gmane.org>	<CAEmF0k5pgCZ4GpM1oy-wB4RNxh67bVL6o5D32MX9bfYYb0SkmQ@mail.gmail.com>
	<jt7r1k$ils$1@dough.gmane.org> <4FF7AE73.50503@pearwood.info>
Message-ID: <jt8sa2$4nr$1@dough.gmane.org>

On 07/07/12 04:35, Steven D'Aprano wrote:

> I find it ironic that you are suggesting removing the debugger to make
> it easier to debug the code :)

But the pdb.trace() calls are not needed for debugging. The
interactive debugger will work just fine without them.
They are for very specialised use. (ie I've never found
a need for them! ;-)

>> I'd either add some more print statements or I'd run it inside a
>> debugger.
>
> Which is what pdb is :)

Yes but it doesn't need  the trace() calls to use it and
winpdb (or even IDLE) is much easier to use!

> Nevertheless, if the Original Poster is not actually using the debugger,
> he should remove the debugger calls.

And that really was the point I was making.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From breamoreboy at yahoo.co.uk  Sat Jul  7 13:04:42 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 07 Jul 2012 12:04:42 +0100
Subject: [Tutor] Program gets stuck after a creating a list from
	dictinary items!
In-Reply-To: <jt8sa2$4nr$1@dough.gmane.org>
References: <CAEmF0k76XuaWX7amA7GWteBLbkjCJuDR_0zpATOG4cdEm1_J5g@mail.gmail.com>	<5B80DD153D7D744689F57F4FB69AF47416544D42@SCACMX008.exchad.jpmchase.net>	<CAEmF0k4XsnjGTe_sTjJVj0dDA+yets_i37p2qVrN_0YaQXbP5A@mail.gmail.com>	<CAEmF0k7OwfpkYL5rUhCJSpOvTec6O9PLMFASANFVrXhhLUf1oQ@mail.gmail.com>	<5B80DD153D7D744689F57F4FB69AF47416544DE4@SCACMX008.exchad.jpmchase.net>	<5B80DD153D7D744689F57F4FB69AF47416544E04@SCACMX008.exchad.jpmchase.net>	<CAEmF0k4XuR7RYQ4fgyhNeosOWSXq+=S56fsz1QR_49Z=rD-Mdw@mail.gmail.com>	<jt7fuh$k90$1@dough.gmane.org>	<CAEmF0k5pgCZ4GpM1oy-wB4RNxh67bVL6o5D32MX9bfYYb0SkmQ@mail.gmail.com>
	<jt7r1k$ils$1@dough.gmane.org> <4FF7AE73.50503@pearwood.info>
	<jt8sa2$4nr$1@dough.gmane.org>
Message-ID: <jt951l$pfs$1@dough.gmane.org>

On 07/07/2012 09:34, Alan Gauld wrote:
> On 07/07/12 04:35, Steven D'Aprano wrote:
>
>> I find it ironic that you are suggesting removing the debugger to make
>> it easier to debug the code :)
>
> But the pdb.trace() calls are not needed for debugging. The
> interactive debugger will work just fine without them.
> They are for very specialised use. (ie I've never found
> a need for them! ;-)
>
>>> I'd either add some more print statements or I'd run it inside a
>>> debugger.
>>
>> Which is what pdb is :)
>
> Yes but it doesn't need  the trace() calls to use it and
> winpdb (or even IDLE) is much easier to use!
>
>> Nevertheless, if the Original Poster is not actually using the debugger,
>> he should remove the debugger calls.
>
> And that really was the point I was making.
>

I don't use a debbuger much but when I do winpdb is superb.

-- 
Cheers.

Mark Lawrence.




From mylesbroomes at hotmail.co.uk  Sat Jul  7 14:57:27 2012
From: mylesbroomes at hotmail.co.uk (myles broomes)
Date: Sat, 7 Jul 2012 12:57:27 +0000
Subject: [Tutor] While loops
Message-ID: <DUB102-W35A5F4AFCAAEF20F46739897ED0@phx.gbl>


I am currently coding a 'text-based adventure game', and im having a bit of trouble with while loops. Here is my code so far: #Text-based Adventure RPG
#The player travels through different towns and dungeons
#The overall goal of the game is simple; the player must make it to the final town,
#Battling various monsters and overcoming several challenges along the wayimport timeplayer_name = None
player_invent = [30]
emb_town_visit=None
YES='yes'
NO='no'
user_choice=None
plyr_location='Embark'
shopkeep_mny=100
shopkeep_invent=['Bronze Sword (10)','Steel Sword (100)','Rounded Shield (10)','Kite Shield (50)','Medicine (5)','Mana Potion (5)'] class Main_char(object):
        """A class for the main character object."""
        def __init__(self,name,invent,level=1,health=30,mana=15):
                self.name=name
                self.level=level
                self.invent=invent
                self.hp=health
                self.mana=mana
                print("\nYour journey begins here in Embark town,"+self.name)        def __str__(self):
                print("\nName: ",self.name, "\nLevel: ",self.level, "\nYour inventory: ",self.invent)        def town_chcklist(self):
                """A checklist showing a list of the towns the player has already visited."""
                self.checklist=[]
                if town1==YES:
                        self.checklist.append(town1) def use_item(item):
        """Allows the player to use an item in their inventory."""
        if item in ('Medicine','Mana Potion'):
                Hero.invent.delete(item)
                if item=='Medicine':
                        Hero.hp+20
                elif item=='Mana Potion':
                        Hero.mana+10class Shopkeep(object):
        """A blueprint for the shopkeeper objects."""
        def __init__(self,inv,money):
                self.inv=inv
                self.money=money        def __str__(self):
                print("Shopkeeper: Here are my wares: \n\n",self.inv+". \nI currently have",self.money,"gold.") #Create the towns/ dungeons/ areas in different functionsdef Embark():
        """The town in which the player begins there journey."""
        if emb_town_visit==None:
                print("Welcome to Embark Town",player_name+"!")        while True:
                print("\n",hero)
                print("""
                        \n\t\tPress '1' to exit the town...
                        \n\t\tPress '2' to enter the local shop...""")
                user_choice=input('What would you like to do?: ')
                if user_choice=='1':
                        shop()
                        continue
                elif user_choice=='2':
                        if emb_town_visit==None:
                                print("\n\t\t\tAs you leave your home of Embark Town, you look back with a smile, then proceed North.")
                        break
                else:
                        print("\n\t\t\tThat is not a valid choice...")
                        continue
 
        #The player has visited Embark Town
        if emb_town_visit==None:
                emb_town_visit=YES        #Exit the function and change the players location variable
        return 'Left Embark Town',emb_town_visit def shop():
        """A generic shop that is placed in each town."""
        while True:
                print("""
                \nWhat would you like to do?
                \nPress '1' to buy an item...
                \n...Press '2' to exit the shop...
                \n...Or press '3' to ask about the town...""")
                user_choice=input(">")
                if user_choice=='1':
                        print("Shopkeeper: Goodbye.")
                        break
                elif user_choice=='2':
                        print(emb_shopkeep)
                        user_choice=None
                        print("Enter the name of the item you would like to purchase.")
                        user_choice=title(input(">"))
                        for item in emb_shopkeep.inv:
                                if user_choice in emb_shopkeep.inv:
                                        message=handle_pur(user_choice)
                                        print(message)
                                        emb_shopkeep.inv.delete(user_choice)
                                else:
                                        print("\n\t\t\tThat is not a valid choice!")def handle_pur(item):
        """Handles purhchases made by the player in the shop."""
        if item=='Bronze Sword':
                if Hero.invent[0] >= 10:
                        Hero.invent[0]-10
                        Hero.invent.append(item)
                        msg='You now own a',item
                elif Hero.invent[0] < 10:
                        msg='You cannot afford that item.'
        elif item=='Steel Sword':
                if Hero.invent[0] >= 100:
                        Hero.invent[0] - 100
                        Hero.invent.append(item)
                        msg='You now own a',item
                elif Hero.invent[0] < 100:
                        msg='You cannot afford that item.'
        elif item =='Rounded Shield':
                if Hero.invent[0] >= 10:
                        Hero.invent[0] - 10
                        Hero.invent.append(item)
                        msg='You now own a',item
                elif Hero.invent < 10:
                        msg='You cannot afford that item.'
        elif item=='Kite Shield':
                if Hero.invent[0] >= 50:
                        Hero.invent[0] - 50
                        Hero.invent.append(item)
                        msg='You now own a',item
                elif Hero.invent < 50:
                        msg='You cannot afford that item.'
        elif item=='Medicine':
                if Hero.invent[0] >= 5:
                        Hero.invent[0] - 5
                        Hero.invent.append(item)
                        msg='You now own a',item
                elif Hero.invent[0] < 5:
                        msg='You cannot afford that item.'
        elif item=='Mana Potion':
                if Hero.invent[0] >= 5:
                        Hero.invent[0] - 5
                        Hero.invent.append(item)
                        msg='You now own a',item
                elif Hero.invent[0] < 5:
                        msg='You cannot afford that item.'        #Informs the program of which message to tell the player
        return msg emb_shopkeep=Shopkeep(shopkeep_invent,shopkeep_mny) #Player creation loop
while True:
        print("""
        \nValiant hero, your quest begins here in Embark Town.""",
        time.sleep(200),
        """\nWhere you will go is up to you...""", 
        time.sleep(200),
        """\nHow your adventure will pan out is up to you...""",
        time.sleep(200), 
        """\nBut first...""", 
        time.sleep(200),
        """\n...tell me your name...""")
        player_name=title(input(">"))
        print("\nAh! So your name is",player_name,"- a truly heroic name!")
        Hero=Main_char(player_name,player_invent)
        break def main():
        """Main game loop."""
        while True:
                if plyr_location=='Embark':
                        plyr_location,emb_town_visit=Embark()
                elif plyr_location=='Left Embark Town':
                        return #Launch the game
main()
 
 For some reason that I cant figure out, the 'Player creation loop' wont run when I try to run the program. When I try to run it in IDLE, nothing happens, Any help is much appreciated. Myles Broomes 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120707/b430a7c5/attachment-0001.html>

From emile at fenx.com  Sat Jul  7 15:56:30 2012
From: emile at fenx.com (Emile van Sebille)
Date: Sat, 07 Jul 2012 06:56:30 -0700
Subject: [Tutor] While loops
In-Reply-To: <DUB102-W35A5F4AFCAAEF20F46739897ED0@phx.gbl>
References: <DUB102-W35A5F4AFCAAEF20F46739897ED0@phx.gbl>
Message-ID: <jt9f9b$sck$1@dough.gmane.org>

On 7/7/2012 5:57 AM myles broomes said...
>
> I am currently coding a 'text-based adventure game', and im having a bit
> of trouble with while loops. Here is my code so far:

Please paste in the traceback you're getting, and please set your mail 
client program for plain text when posting.  What I see here has the 
indentation stripped out of your script so that before I can even test 
your code I'll have to guess at how you have it indented, and I'm more 
likely than you to get it right and may inadvertently fix your bug and 
never know it even before getting a chance to see the bug.

Emile


>
> #Text-based Adventure RPG
> #The player travels through different towns and dungeons
> #The overall goal of the game is simple; the player must make it to the
> final town,
> #Battling various monsters and overcoming several challenges along the way
> import time
> player_name = None
> player_invent = [30]
> emb_town_visit=None
> YES='yes'
> NO='no'
> user_choice=None
> plyr_location='Embark'
> shopkeep_mny=100
> shopkeep_invent=['Bronze Sword (10)','Steel Sword (100)','Rounded Shield
> (10)','Kite Shield (50)','Medicine (5)','Mana Potion (5)']
>
> class Main_char(object):
> """A class for the main character object."""
> def __init__(self,name,invent,level=1,health=30,mana=15):
> self.name=name
> self.level=level
> self.invent=invent
> self.hp=health
> self.mana=mana
> print("\nYour journey begins here in Embark town,"+self.name)
> def __str__(self):
> print("\nName: ",self.name, "\nLevel: ",self.level, "\nYour inventory:
> ",self.invent)
> def town_chcklist(self):
> """A checklist showing a list of the towns the player has already
> visited."""
> self.checklist=[]
> if town1==YES:
> self.checklist.append(town1)
>
> def use_item(item):
> """Allows the player to use an item in their inventory."""
> if item in ('Medicine','Mana Potion'):
> Hero.invent.delete(item)
> if item=='Medicine':
> Hero.hp+20
> elif item=='Mana Potion':
> Hero.mana+10
> class Shopkeep(object):
> """A blueprint for the shopkeeper objects."""
> def __init__(self,inv,money):
> self.inv=inv
> self.money=money
> def __str__(self):
> print("Shopkeeper: Here are my wares: \n\n",self.inv+". \nI currently
> have",self.money,"gold.")
>
> #Create the towns/ dungeons/ areas in different functions
> def Embark():
> """The town in which the player begins there journey."""
> if emb_town_visit==None:
> print("Welcome to Embark Town",player_name+"!")
> while True:
> print("\n",hero)
> print("""
> \n\t\tPress '1' to exit the town...
> \n\t\tPress '2' to enter the local shop...""")
> user_choice=input('What would you like to do?: ')
> if user_choice=='1':
> shop()
> continue
> elif user_choice=='2':
> if emb_town_visit==None:
> print("\n\t\t\tAs you leave your home of Embark Town, you look back with
> a smile, then proceed North.")
> break
> else:
> print("\n\t\t\tThat is not a valid choice...")
> continue
>
> #The player has visited Embark Town
> if emb_town_visit==None:
> emb_town_visit=YES
> #Exit the function and change the players location variable
> return 'Left Embark Town',emb_town_visit
>
> def shop():
> """A generic shop that is placed in each town."""
> while True:
> print("""
> \nWhat would you like to do?
> \nPress '1' to buy an item...
> \n...Press '2' to exit the shop...
> \n...Or press '3' to ask about the town...""")
> user_choice=input(">")
> if user_choice=='1':
> print("Shopkeeper: Goodbye.")
> break
> elif user_choice=='2':
> print(emb_shopkeep)
> user_choice=None
> print("Enter the name of the item you would like to purchase.")
> user_choice=title(input(">"))
> for item in emb_shopkeep.inv:
> if user_choice in emb_shopkeep.inv:
> message=handle_pur(user_choice)
> print(message)
> emb_shopkeep.inv.delete(user_choice)
> else:
> print("\n\t\t\tThat is not a valid choice!")
> def handle_pur(item):
> """Handles purhchases made by the player in the shop."""
> if item=='Bronze Sword':
> if Hero.invent[0] >= 10:
> Hero.invent[0]-10
> Hero.invent.append(item)
> msg='You now own a',item
> elif Hero.invent[0] < 10:
> msg='You cannot afford that item.'
> elif item=='Steel Sword':
> if Hero.invent[0] >= 100:
> Hero.invent[0] - 100
> Hero.invent.append(item)
> msg='You now own a',item
> elif Hero.invent[0] < 100:
> msg='You cannot afford that item.'
> elif item =='Rounded Shield':
> if Hero.invent[0] >= 10:
> Hero.invent[0] - 10
> Hero.invent.append(item)
> msg='You now own a',item
> elif Hero.invent < 10:
> msg='You cannot afford that item.'
> elif item=='Kite Shield':
> if Hero.invent[0] >= 50:
> Hero.invent[0] - 50
> Hero.invent.append(item)
> msg='You now own a',item
> elif Hero.invent < 50:
> msg='You cannot afford that item.'
> elif item=='Medicine':
> if Hero.invent[0] >= 5:
> Hero.invent[0] - 5
> Hero.invent.append(item)
> msg='You now own a',item
> elif Hero.invent[0] < 5:
> msg='You cannot afford that item.'
> elif item=='Mana Potion':
> if Hero.invent[0] >= 5:
> Hero.invent[0] - 5
> Hero.invent.append(item)
> msg='You now own a',item
> elif Hero.invent[0] < 5:
> msg='You cannot afford that item.'
> #Informs the program of which message to tell the player
> return msg
>
> emb_shopkeep=Shopkeep(shopkeep_invent,shopkeep_mny)
>
> #Player creation loop
> while True:
> print("""
> \nValiant hero, your quest begins here in Embark Town.""",
> time.sleep(200),
> """\nWhere you will go is up to you...""",
> time.sleep(200),
> """\nHow your adventure will pan out is up to you...""",
> time.sleep(200),
> """\nBut first...""",
> time.sleep(200),
> """\n...tell me your name...""")
> player_name=title(input(">"))
> print("\nAh! So your name is",player_name,"- a truly heroic name!")
> Hero=Main_char(player_name,player_invent)
> break
>
> def main():
> """Main game loop."""
> while True:
> if plyr_location=='Embark':
> plyr_location,emb_town_visit=Embark()
> elif plyr_location=='Left Embark Town':
> return
>
> #Launch the game
> main()
>
> For some reason that I cant figure out, the 'Player creation loop' wont
> run when I try to run the program. When I try to run it in IDLE, nothing
> happens, Any help is much appreciated.
>
> Myles Broomes
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor



From alan.gauld at btinternet.com  Sat Jul  7 18:31:20 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 07 Jul 2012 17:31:20 +0100
Subject: [Tutor] While loops
In-Reply-To: <DUB102-W35A5F4AFCAAEF20F46739897ED0@phx.gbl>
References: <DUB102-W35A5F4AFCAAEF20F46739897ED0@phx.gbl>
Message-ID: <jt9o8o$k8$1@dough.gmane.org>

On 07/07/12 13:57, myles broomes wrote:
>
> I am currently coding a 'text-based adventure game', and im having a bit
> of trouble with while loops. Here is my code so far:

What kind of trouble? You have lots of while loops, which loop?
And do you get an error message? If so please post it - all of it!

And looking through the code I see lots of potential problems, I have 
highlighted a few below...


> class Main_char(object):
>          def __init__(self,name,invent,level=1,health=30,mana=15):
>          def __str__(self):
>          def town_chcklist(self):
>                  self.checklist=[]
>                  if town1==YES:
>                          self.checklist.append(town1)

This method always resets the list to empty and then appends town1.
You should move the initialisation of the list into __init__.
But I see no variable defined anywhere called 'town1' so you would
get an error... Fortunately you don't seem to call this method
so no error is ever produced...

> def use_item(item):
>          """Allows the player to use an item in their inventory."""
>          if item in ('Medicine','Mana Potion'):
>                  Hero.invent.delete(item)
>                  if item=='Medicine':
>                          Hero.hp+20

Here you add 20 to the value of Hero.hp but then throw thecresult away.
I suspect you mean:

Hero.hp += 20

>                  elif item=='Mana Potion':
>                          Hero.mana+10

What is Hero.mana? It does not appear in the MainChar class definition?
And as above I suspect you mean += 10?

> class Shopkeep(object):
>          def __init__(self,inv,money):
>                  self.inv=inv
>                  self.money=money
>          def __str__(self):
>                  print("Shopkeeper: Here are my wares: \n\n",self.inv+".
> \nI currently have",self.money,"gold.")
>
> #Create the towns/ dungeons/ areas in different functions
> def Embark():

Its usual practice to keep names startingt with a captial letter for 
Clas names. It gets confusing for the reader(whio might be you!)  otherwise.


>          """The town in which the player begins there journey."""
>          if emb_town_visit==None:
>                  print("Welcome to Embark Town",player_name+"!")
>          while True:
>                  print("\n",hero)
>                  print("""
>                          \n\t\tPress '1' to exit the town...
>                          \n\t\tPress '2' to enter the local shop...""")
>                  user_choice=input('What would you like to do?: ')
>                  if user_choice=='1':
>                          shop()
>                          continue

You don't need 'continue' here because the loop drops out here anyway.
But it won't do any harm.

>                  elif user_choice=='2':
>                          if emb_town_visit==None:
>                                  print("\n\t\t\tAs you leave your home
> of Embark Town, you look back with a smile, then proceed North.")
>                          break
>                  else:
>                          print("\n\t\t\tThat is not a valid choice...")
>                          continue

same here. You only need continue if you are jumping out of the code 
block back to the start of the loop. It should be a relatively rare 
event in well structured code!


>          #The player has visited Embark Town
>          if emb_town_visit==None:
>                  emb_town_visit=YES
>          #Exit the function and change the players location variable
>          return 'Left Embark Town',emb_town_visit

This is broken due to some bad use of variable names.

You define emb_town_visit as a global with value None.
First time into Embark you test the global and if its set to None you 
*create a new local variable* of the same name. You then return it to 
main which sets *yet another local variable* of the same name to YES.
But this does not change the global so next time you visit Embark the 
global is still None and you go through the same routine. If you want to 
use the global value (and especially if you want to change it) you must 
specify it as global in each function . But frankly it should probably 
not be global anyhow since you want to track that per player so it 
should really be part of your character class.

> def shop():
>          """A generic shop that is placed in each town."""

In that case it should probably be a class and you create an instance in 
each town.

>          while True:
>                  print("""
>                  \nWhat would you like to do?
>                  \nPress '1' to buy an item...
>                  \n...Press '2' to exit the shop...
>                  \n...Or press '3' to ask about the town...""")
>                  user_choice=input(">")
>                  if user_choice=='1':
>                          print("Shopkeeper: Goodbye.")
>                          break
>                  elif user_choice=='2':
>                          print(emb_shopkeep)
>                          user_choice=None
>                          print("Enter the name of the item you would
> like to purchase.")
>                          user_choice=title(input(">"))

Its better to use different variable names for the choices.
It makes debugging easier if you can check the values of the original 
choice as well as the item selection.

>                          for item in emb_shopkeep.inv:
>                                  if user_choice in emb_shopkeep.inv:
>                                          message=handle_pur(user_choice)
>                                          print(message)
>
> emb_shopkeep.inv.delete(user_choice)
>                                  else:
>                                          print("\n\t\t\tThat is not a
> valid choice!")

At this point I was losing interest.... so I skipped down a bit...

 > ....
> emb_shopkeep=Shopkeep(shopkeep_invent,shopkeep_mny)
>
> #Player creation loop
> while True:
>          print("""
>          \nValiant hero, your quest begins here in Embark Town.""",
>          time.sleep(200),
>          """\nWhere you will go is up to you...""",
>          time.sleep(200),
>          """\nHow your adventure will pan out is up to you...""",
>          time.sleep(200),
>          """\nBut first...""",
>          time.sleep(200),
>          """\n...tell me your name...""")
>          player_name=title(input(">"))
>          print("\nAh! So your name is",player_name,"- a truly heroic name!")
>          Hero=Main_char(player_name,player_invent)
>          break

Because of the break this will only execute once, so there is no point 
in having a loop.

Also you create your player by passing in player_invent - a list with 
the single value 30.

This will create a reference inside the object to the global value so 
all changes will affect the global. With only one player as here its not 
a problem nbut if you ever try to creaste multiple players they will all 
share and modify the same invent list. Probably not what you want. Get 
used to defining data where it needs to be, either global or in the 
class. Then you could just pass in the integer value 30 and get the 
__init__ to create the list (if indeed it needs to be a list, it seems 
you only ever use the single element!

> def main():
>          """Main game loop."""
>          while True:
>                  if plyr_location=='Embark':
>                          plyr_location,emb_town_visit=Embark()
>                  elif plyr_location=='Left Embark Town':
>                          return

In practice this doesn't need to be a loop since you only have one 
player so it actually resolves to:

def main():
     plyr_location,emb_town_visit=Embark()

which creates two local variables, set them to the return value of 
Embark() and exits. It has no effect on the global values defined at the 
top.

I would strongly recommend you start again and remove all code that you 
are not executing. And build up the code one class/function at a time 
and test it as you go so that you always have working code before adding 
anything new.

You can do this by starting with the high level framework with
a menu and calls to create the character (but the class is still
empty at this point) and then define the class. Then add the
shopkeeper call and empty shopkeeper class, then define the new class, 
and so on.

Alternatively start with the  class definitions - one at a time - and 
write specific test code to check that the class works. Then once the 
classes work add the outer control code.

Either approach works and different programmers prefer different 
approaches. But in either case add code incrementally only after getting 
the previous code working. This keeps the code clearer and makes 
debugging simpler (because the bug is almost certainly in the new
code you added!)

Finally, I'd look to move much more of your data and functions into the 
classes. For example embark could be a method of a player - the first 
thing he does after being created. Similarly shop() could be a player 
method:

Hero = Mainchar(name, value)
Hero.embark()
Hero.shop()

Now, what was the problem with the while loop?

>   For some reason that I cant figure out, the 'Player creation loop'
> wont run when I try to run the program.

As I point out above its not really a loop...
But try adding some print statements to see how far it does get?
Send us any error messages...

HTH,
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From aarontp at rocketmail.com  Sat Jul  7 23:19:03 2012
From: aarontp at rocketmail.com (Aaron Tp)
Date: Sat, 7 Jul 2012 14:19:03 -0700 (PDT)
Subject: [Tutor] The Best Way to go About with Self Modifying Code/Code
	Generation?
Message-ID: <1341695943.35002.YahooMailNeo@web121101.mail.ne1.yahoo.com>



Hey all, I have a question on using self-modifying code/code generation in Python;
namely how to do it. 


I've posted on Stack Overflow and a python forum on this topic, and apparently the current approach I'd devised was....too difficult.
The idea was, for every Python operator and statement and whatnot, having a corresponding class/block of code with which to use it, and 

then using 'exec' to execute the statements...

The first problem here, of course, is that it isn't a self-modifying system at all....second being there'd be no way to edit pre-existing code, and getting it all to be deleted wouldn't...make any sense.

People have recommended the use of Python's AST module, along with the Compile module, to be able to programatically edit/change pre-existing code, but there are two problems here. One being I'm not familiar at all with the AST/Compile modules, and two being that to the best of my understanding the AST module is very difficult for a user, let alone a Python code, to manipulate to yield useful results. (in case your wondering, I was hoping to hook up whatever form of modification/generation to a genetic algorithim or backprop algorithim and hopefully get something that can automatically write simple codes....idk)

My question is....what should I be doing? Would the AST module work? Or am I jst being overly-ambitious?
-Aaron


From alan.gauld at btinternet.com  Sun Jul  8 01:21:56 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 08 Jul 2012 00:21:56 +0100
Subject: [Tutor] The Best Way to go About with Self Modifying Code/Code
	Generation?
In-Reply-To: <1341695943.35002.YahooMailNeo@web121101.mail.ne1.yahoo.com>
References: <1341695943.35002.YahooMailNeo@web121101.mail.ne1.yahoo.com>
Message-ID: <jtagak$pb5$1@dough.gmane.org>

On 07/07/12 22:19, Aaron Tp wrote:

> Hey all, I have a question on using self-modifying code/code generation in Python;
> namely how to do it.

Self generating code is hard to do.
Writing code is hard for intelligent humans, writing code to generate 
code is even harder.

To write code that modifies itself increases the difficulty even further 
(and debugging it by an order of magnitude because you no longer know 
what the code you are debugging is!)

If you are going to do it, its best if you use a language where the code 
and data are interchangeable, and that mainly means some kind of Lisp 
type language. To use a traditional language like Python just makes it 
even more complex.

So in short you have bitten off a huge challenge.

But there are ways and means to do some basic things depending on what 
you need to do. If its just changing the sequencing or control flow of a 
pre-existing set of functions then that's not too bad. Or if its just 
varying some parameters in an existing algorithm again it is doable.
If its dynamically creating new functions or completely changing an 
existing algorithm then you are into really murky waters!

> then using 'exec' to execute the statements...

exec or more likely eval()  is pretty essential if you truly are writing 
new functions/algorithms. But its not needed if just changing control 
sequences.

> The first problem here, of course, is that it isn't a self-modifying
> system at all....second being there'd be no way to edit pre-existing
 > code, and getting it all to be deleted wouldn't...make any sense.

That depends on what you are trying to do.
But one thing that will help a lot is to express your program ass a 
state machine. Then express the state machine as a data table with each 
state represented as an index into the table and with functions to 
transition between states. Then write the functions and put them in a 
dispatch table accessed via indexes.

Now your programs behaviour can be expressed as a series of integers(the 
function indexes) and a state machine expressed as a table. Now we have 
code as data, albeit a limited set of code.
And you can modify the behaviour by modifying the index corresponding to 
a state change and by modifying the next state index.

>
> People have recommended the use of Python's AST module, along with
 > the Compile module,

Thats probably the best bet if you really need to generate/.modify 
existing functions.

> ...Or am I jst being overly-ambitious?

You don't tell us anything about your Python coding experience (or
even you're coding experience in other languages) but it is a big 
challenge. Especially in a language like Python (although much
easier than in a compiled language like C!! And even that is possible: I 
did once implement a state machine solution as above in C  for a remote 
controlled network traffic-routing switch)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From stefan_ml at behnel.de  Sun Jul  8 07:51:12 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 08 Jul 2012 07:51:12 +0200
Subject: [Tutor] The Best Way to go About with Self Modifying Code/Code
	Generation?
In-Reply-To: <1341695943.35002.YahooMailNeo@web121101.mail.ne1.yahoo.com>
References: <1341695943.35002.YahooMailNeo@web121101.mail.ne1.yahoo.com>
Message-ID: <jtb74h$djv$1@dough.gmane.org>

Aaron Tp, 07.07.2012 23:19:
> I have a question on using self-modifying code/code generation
> in Python; namely how to do it.

Don't.

Seriously, the answer you should ask yourself is: why do you think the
genetic algorithm (or whatever you are trying to do exactly) would come up
with any reasonable code constructs other than what you already
anticipated? And if you anticipated them, you can just write them down
statically in the code and only care about recombining them.

Make your code configurable instead, and then change the configuration. I
agree with Alan that a state machine could be a suitable approach (but it's
a bit hidden in his long answer). In any case, think in terms of modifying
data, not code.

Stefan


From steve at pearwood.info  Sun Jul  8 15:48:59 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 08 Jul 2012 23:48:59 +1000
Subject: [Tutor] The Best Way to go About with Self Modifying Code/Code
 Generation?
In-Reply-To: <1341695943.35002.YahooMailNeo@web121101.mail.ne1.yahoo.com>
References: <1341695943.35002.YahooMailNeo@web121101.mail.ne1.yahoo.com>
Message-ID: <4FF98FCB.8080502@pearwood.info>

Aaron Tp wrote:
> 
> Hey all, I have a question on using self-modifying code/code generation in Python;
> namely how to do it. 

I know others have already said not to do this, and to be honest I was going 
to say that same thing, but I have changed my mind. Buggrit, this is Python, 
where we are all consenting adults and you're allowed to shoot yourself in the 
foot if you want.

"Writing self-modifying code" actually covers a large range of techniques, 
ranging from the simple and straight-forward use of conventional factory 
functions, to the completely crazy self-modifying source code. There's a lot 
to learn. Rather than try to cover it all, I'll sketch out the various things 
you should learn about before you can make self-modifying code.

Obviously basic Python skills, how to read and write files, import modules, 
etc. That's the boring part, and I'm going to assume you know all that. If you 
don't, do a few tutorials and then come back.

Still here? Cool! On to the hard part. But first, beware:

     HERE BE DRAGONS


As fans of the British fantasy author Terry Pratchett all know, there are two 
types of dragons, swamp dragons and Draco nobilis, the noble dragon.

Swamp, or common, dragons are small, mostly harmless, with a very finicky 
digestive system and a regrettable tendency to blow up, but with a little bit 
of care and attention that can be occasionally useful.

The noble dragon, on the other hand, are huge, cunning, cruel, malicious, and 
anything but harmless. They don't so much blow up as silently fly down out of 
the sky and vapourize you with white-hot flame. They are best avoided by 
anyone other than experts. Actually, even experts should avoid them.

Writing self-modifying code is like dealing with dragons. And like dealing 
with dragons, you should start off with the fairly harmless variety before 
moving up to the more tricky type.

So start with this simple example of self-modifying code:

http://code.activestate.com/recipes/68429-ring-buffer/

This is probably as harmless and simple as self-modifying code can get: you 
have an instance which dynamically changes it's own class as it goes, so as to 
change its behaviour. Other languages might choke on this, but for Python, 
this counts as trivial.

If this example confuses you, if it's too hard for you to understand how it 
works, then you're not ready to try dragon taming. Learn some more Python 
first before you get into self-modifying code. We can help you with that.


Other simple techniques to start with include:

- dynamic code generation with factory functions and closures

- shadowing globals, built-ins or modules with your own code

- rebinding methods, functions or classes at runtime

- monkey-patching builtin functions

- functions or methods which rebind *themselves* at runtime


Feel free to ask for help on any of these. Or google on key phrases such as 
"monkey-patching", "shadowing builtins", "factory function", "closure".


So far, this all counts as the easy, conservative side of self-modifying code. 
Factory functions are even staid and boring. Shadowing is sometimes useful. 
Even monkey-patching, which is what those craz-ee Ruby hackers do, is almost 
respectable. Respectable like drinking two bottles of gin before going 
fox-hunting -- everybody knows you shouldn't do it, but if you survive the 
first time, you start to think nothing could possibly go wrong.

Be warned that "easy" is relative -- these can actually be fairly advanced 
techniques, and are tricky to get right and difficult to debug when they go 
wrong. And things *will* go wrong. Remember the warnings from the creator of 
the Unix operating system, Brian Kernighan:

     "Everyone knows that debugging is twice as hard as writing a
     program in the first place. So if you?re as clever as you can
     be when you write it, how will you ever debug it?"

     "Debugging is twice as hard as writing the code in the first
     place. Therefore, if you write the code as cleverly as possible,
     you are, by definition, not smart enough to debug it."


Okay. You've got your handgun, you're ready to shoot your own foot.
Once you've learned to tame the swamp dragons of runtime rebinding, 
monkey-patching and dynamic code generation, you're ready to move up to the 
next, slightly more radical, stage: eval and exec.

Here are some links to get you started:

http://lucumr.pocoo.org/2011/2/1/exec-in-python/
http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html
http://code.activestate.com/recipes/500261-named-tuples/


Still got all your toes? Excellent. Now you *might* be ready to go after the 
*really* big dragons:

- AST manipulation

http://www.dalkescientific.com/writings/diary/archive/2010/02/22/instrumenting_the_ast.html
http://docs.python.org/library/ast.html


- byte code hacks

http://nedbatchelder.com/blog/200804/wicked_hack_python_bytecode_tracing.html
http://wiki.python.org/moin/ByteplayDoc
http://code.activestate.com/recipes/498242/
http://www.voidspace.org.uk/python/weblog/arch_d7_2006_11_18.shtml#e553


- source code self-modification

http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=show&ixPost=44953
http://thedailywtf.com/Articles/Self-Modifying-VBA.aspx

And now you are ready for Mel.

http://www.catb.org/jargon/html/story-of-mel.html


Have fun!



-- 
Steven

From aarontp at rocketmail.com  Sun Jul  8 17:19:18 2012
From: aarontp at rocketmail.com (Aaron Tp)
Date: Sun, 8 Jul 2012 08:19:18 -0700 (PDT)
Subject: [Tutor] The Best Way to go About with Self Modifying Code/Code
	Generation?
In-Reply-To: <4FF98FCB.8080502@pearwood.info>
References: <1341695943.35002.YahooMailNeo@web121101.mail.ne1.yahoo.com>
	<4FF98FCB.8080502@pearwood.info>
Message-ID: <1341760758.72799.YahooMailNeo@web121103.mail.ne1.yahoo.com>


Thank you all for your responses. I don't know if I have the capability to even complete such a project, or if I possess the necessary will to start, but at least I understand it a bit more than when I was starting. 

--Aaron


From david at graniteweb.com  Sun Jul  8 17:53:27 2012
From: david at graniteweb.com (David Rock)
Date: Sun, 8 Jul 2012 10:53:27 -0500
Subject: [Tutor] Seeking help with reading and writing files in Python
In-Reply-To: <4FE9FF49.203@triad.rr.com>
References: <4FE9FF49.203@triad.rr.com>
Message-ID: <20120708155327.GF3060@wdfs.gateway.2wire.net>

* Aristotle <Aristotle at triad.rr.com> [2012-06-26 14:28]:
> 
> The sequence of events that I am using now is to open a program that 
> will gather gallons used from user, call a module to make calculations, 
> use the info from the module to write info to a file named 
> FinalProjectBill.txt.
> 
> After that is done, I open another program that will read the info 
> written to FinalProjectBill.txt
> 
> I guess my confusion about the instructions is, if I should read from 
> one file FinalProjectBill.txt first how does data get written to that 
> file to begin with?

Based on this part, you have already answered your own question.
Specifically, you get your input file, FinalProjectBill.txt, by
collecting data from the user, doing some calculations, and outputting
to the file.

I'm not sure I see the problem. :-(

-- 
David Rock
david at graniteweb.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/tutor/attachments/20120708/0e4558b8/attachment.pgp>

From david at graniteweb.com  Sun Jul  8 18:04:54 2012
From: david at graniteweb.com (David Rock)
Date: Sun, 8 Jul 2012 11:04:54 -0500
Subject: [Tutor] Python and boot sequences
In-Reply-To: <DF8381D0C668344094293D48724112BC0704AD46@SEAEMBX01.olympus.F5Net.com>
References: <DF8381D0C668344094293D48724112BC0704AD46@SEAEMBX01.olympus.F5Net.com>
Message-ID: <20120708160454.GG3060@wdfs.gateway.2wire.net>

* Dave Wilder <D.Wilder at F5.com> [2012-06-28 12:54]:
> 
> Can a Python script be written that has the ability to stop a Linux
> device in the middle of a boot when a certain sequence occurs and then
> perform an action?
> 
> For example, when I boot my switch (w/ Linux OS 2.7.3), I want to stop
> the boot when I see a series of asterisks.  When I see this, I need to
> hit the <ESC><SHIFT><9> sequence.  Then, I need to select a menu
> option (e.g. 1 to run a test or Q to quit and continue w/ the boot).
> Another example would be when doing a PXE boot, selecting the image to
> load when prompted during the reboot.

Probably not.  The issue is that you are dealing with the bootstrap
process of a system.  Typically, this is not a place where any semblance
of external user interaction exists.  The examples you show represent
compiled-in options that are more likely part of the initial ramdisk, 
BIOS-level code for a specific device, or something similar.  

The PXE boot menu example is usually something that's being presented by
an external server (via cobbler, or a similar product).  There isn't an
OS at that point that would likely be able to supply a python
environment to use.

I think we would need to better understand exactly what you are trying
to build, but you are probably looking for something that would be a
much lower level than python.

-- 
David Rock
david at graniteweb.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/tutor/attachments/20120708/4a3e58d0/attachment.pgp>

From stefan_ml at behnel.de  Sun Jul  8 18:42:50 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 08 Jul 2012 18:42:50 +0200
Subject: [Tutor] The Best Way to go About with Self Modifying Code/Code
	Generation?
In-Reply-To: <4FF98FCB.8080502@pearwood.info>
References: <1341695943.35002.YahooMailNeo@web121101.mail.ne1.yahoo.com>
	<4FF98FCB.8080502@pearwood.info>
Message-ID: <jtcdab$q6o$1@dough.gmane.org>

Steven D'Aprano, 08.07.2012 15:48:
>> Hey all, I have a question on using self-modifying code/code generation
>> in Python; namely how to do it. 
> 
> I know others have already said not to do this, and to be honest I was
> going to say that same thing, but I have changed my mind. Buggrit, this is
> Python, where we are all consenting adults and you're allowed to shoot
> yourself in the foot if you want.

:)

You're right. Saying "don't do that" is easy, but giving people the freedom
to shoot themselves in the foot sometimes has accidental side effects that
end up pushing our world another bit forward. (Though usually not, but
that's ok)

Stefan


From alan.gauld at btinternet.com  Sun Jul  8 19:08:05 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 08 Jul 2012 18:08:05 +0100
Subject: [Tutor] Seeking help with reading and writing files in Python
In-Reply-To: <4FE9FF49.203@triad.rr.com>
References: <4FE9FF49.203@triad.rr.com>
Message-ID: <jtcepl$6h7$1@dough.gmane.org>

On 26/06/12 19:28, Aristotle wrote:

> The sequence of events that I am using now is to open a program that
> will gather gallons used from user, call a module to make calculations,
> use the info from the module to write info to a file named
> FinalProjectBill.txt.
> ...
>  From Instructor:
> You should read from one file (FinalProjectBill.txt) and then write to
> one (BranleyFinal.txt) with the results as we are learning file i/o
> (input/output) -
> ...
> You should not write to your input file.

> My question is why do I need two text files FinalProjectBill.txt and
> BranleyJamesFinal.txt ?

I didn't read this properly the first time around.

The answer to your question is that you don't. It is a design decision.

But, if you write back to the input file it makes things more complex 
because you may have to read/write from the same file at the same 
time(*). This means moving the file cursor back and forth between the 
point where you are reading and the end of the file. There are functions 
to do this but it adds a lot of extra overhead and is easy to get wrong.
It is much simpler to read the data from one file and write the output 
to another and let Python take care of moving the cursor automatically.

(*)Of course you may choose to read all the input data in, close the 
file then overwrite it with your output. That's easy but sometimes the 
input file is too big or the processing time too limited to do that. Or 
you may need to preserve the data for use in subsequent processing.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From bayespokerguy at gmail.com  Sun Jul  8 23:47:47 2012
From: bayespokerguy at gmail.com (Fred G)
Date: Sun, 8 Jul 2012 14:47:47 -0700
Subject: [Tutor] Mapping ID's for corresponding values in different Columns
Message-ID: <CAMg+7mZ5-mt9-kO4MPzF5G9xw7_sJ_=2TASesMzGUCpEiuMCPQ@mail.gmail.com>

Hi--

My current input looks like the following:

FILE1.csv
PERSON_ID    PERSON_NAME
1                     Jen
2                     Mike
3                     Jim
4
5                     Jane
6                     Joe
7                     Jake

FILE2.csv
PERSON_ID   PERSON_NAME
                     Jim
                     Mike
                     Jane
                     Todd
                     Jen

_________
I want to fill into the PERSON_ID column of FILE2.csv the corresponding
ID's associated with those names as identified in FILE1.csv.

At first I imported the csv module and was using the csv.Reader, but then
it seemed simple enough just to write something like:
for line in file2:
     print(line)

giving me the following output:
PERSON_ID, PERSON_NAME
, Jim
, Mike
, Jane
, Todd
, Jen

I think I understand the issue at a conceptual level, but not quite sure
how to fully implement it:
a) I want to build a dictionary to create keys, such that each number in
file1 corresponds to a unique string in column B of file1.
b) then write a for loop like the following:
for "person_name" in file2:
   if "person_name.file2" == "person_name.file1":
       person_id.file2 == person_id.file1
c) write into file2 the changes to person_id's...

But it's pretty difficult for me to get past this stage. Am I on the right
track? And more importantly, how could I learn how to actually implement
this in smaller stages?

Thanks so much.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120708/7fdf5686/attachment.html>

From bayespokerguy at gmail.com  Mon Jul  9 01:50:45 2012
From: bayespokerguy at gmail.com (Fred G)
Date: Sun, 8 Jul 2012 16:50:45 -0700
Subject: [Tutor] Mapping ID's for corresponding values in different
	Columns (UPDATE)
Message-ID: <CAMg+7maMm9skTej4oqL2wpT6WUo8w3tgJZLc4Ta3HPcEzbQ2RA@mail.gmail.com>

I thought it made sense to read the two columns in File1 in as a dictionary
(where the key is actually the name, so that we can search on it later),
and the column of interest in File2 as a list.  Finding the common values
then is just:

for item in file2_list:
  for line in file1_dict:
    if item == line:
      print item

Creating a new dictionary and filling it in with the common values gives us
something like:
new_dict = {}
for item in file2_list:
    for line in file1_dict:
      if item == line:
        new_dict[item] = line

I'm not quite sure where to go from here, though. I want:
a) to edit the new_dict such that we get the proper key and value
combination, instead of what it is now which is just the proper values.
b) Once we have the correct dictionary with the proper keys and their
values, how do I then output it into the form (???):

File2.csv
PERSON_ID    PERSON_NAME
key1               Jim
key100           Mike
key3              Jane
key989          Todd
etc...             etc...


On Sun, Jul 8, 2012 at 2:47 PM, Fred G <bayespokerguy at gmail.com> wrote:

> Hi--
>
> My current input looks like the following:
>
> FILE1.csv
> PERSON_ID    PERSON_NAME
> 1                     Jen
> 2                     Mike
> 3                     Jim
> 4
> 5                     Jane
> 6                     Joe
> 7                     Jake
>
> FILE2.csv
> PERSON_ID   PERSON_NAME
>                      Jim
>                      Mike
>                      Jane
>                      Todd
>                      Jen
>
> _________
> I want to fill into the PERSON_ID column of FILE2.csv the corresponding
> ID's associated with those names as identified in FILE1.csv.
>
> At first I imported the csv module and was using the csv.Reader, but then
> it seemed simple enough just to write something like:
> for line in file2:
>      print(line)
>
> giving me the following output:
> PERSON_ID, PERSON_NAME
> , Jim
> , Mike
> , Jane
> , Todd
> , Jen
>
> I think I understand the issue at a conceptual level, but not quite sure
> how to fully implement it:
> a) I want to build a dictionary to create keys, such that each number in
> file1 corresponds to a unique string in column B of file1.
> b) then write a for loop like the following:
> for "person_name" in file2:
>    if "person_name.file2" == "person_name.file1":
>        person_id.file2 == person_id.file1
> c) write into file2 the changes to person_id's...
>
> But it's pretty difficult for me to get past this stage. Am I on the right
> track? And more importantly, how could I learn how to actually implement
> this in smaller stages?
>
> Thanks so much.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120708/8ee06a88/attachment.html>

From wprins at gmail.com  Mon Jul  9 02:09:02 2012
From: wprins at gmail.com (Walter Prins)
Date: Mon, 9 Jul 2012 01:09:02 +0100
Subject: [Tutor] Mapping ID's for corresponding values in different
 Columns (UPDATE)
In-Reply-To: <CAMg+7maMm9skTej4oqL2wpT6WUo8w3tgJZLc4Ta3HPcEzbQ2RA@mail.gmail.com>
References: <CAMg+7maMm9skTej4oqL2wpT6WUo8w3tgJZLc4Ta3HPcEzbQ2RA@mail.gmail.com>
Message-ID: <CANLXbfBonSuz2MPCNooJWFJ+JHrTDtpEGMAFTvbzmnGn0bb8ZQ@mail.gmail.com>

On 9 July 2012 00:50, Fred G <bayespokerguy at gmail.com> wrote:
> I thought it made sense to read the two columns in File1 in as a dictionary
> (where the key is actually the name, so that we can search on it later), and

yes...

> the column of interest in File2 as a list.  Finding the common values then
> is just:
>
> for item in file2_list:
>   for line in file1_dict:
>     if item == line:
>       print item

No.  Once you have a dictionary from which you can look up id's from
names, you want to simply read through the lines in file 2, and for
each line, you want to pick up the name from that line and look up the
corresponding id using the dict.  Then, having looked up the number,
you want to then write out a line to a new version of File2 that will
replace File2 once done.  That's more or less it.  Exactly how you
handle the old/new File2 issue is up to you.  Either write a new file
and rename the files once done (safer since you can keep the old file
as a backup), or read the entire file into memory and then process and
write the whole thing back out to the same file (less safe -- what
happens if something goes wrong while you're writing out the data to
the file?)

Walter

From D.Wilder at F5.com  Mon Jul  9 02:15:46 2012
From: D.Wilder at F5.com (Dave Wilder)
Date: Mon, 9 Jul 2012 00:15:46 +0000
Subject: [Tutor] Python and boot sequences
In-Reply-To: <jt1t77$cf1$1@dough.gmane.org>
References: <DF8381D0C668344094293D48724112BC0704AD46@SEAEMBX01.olympus.F5Net.com>
	<jt1t77$cf1$1@dough.gmane.org>
Message-ID: <DF8381D0C668344094293D48724112BC07051AA4@SEAEMBX01.olympus.F5Net.com>

The switch is in a state where user process can be run once the boot sequence has been halted (the aforementioned <ESC><SHIFT><9>).
However the time between when the reboot is kicked off and when the asterisks appear (and the boot must be stopped), there is no shell
Environment.

I may be trying to hammer a nail in with a wrench.  So I'm going to do a little more digging into the switch code and get to a point where 
a shell script can be run as Alan suggested.  Then I'll be back.

Thanks all for the replies.

Dave


-----Original Message-----
From: tutor-bounces+d.wilder=f5.com at python.org [mailto:tutor-bounces+d.wilder=f5.com at python.org] On Behalf Of Alan Gauld
Sent: Wednesday, July 04, 2012 1:07 PM
To: tutor at python.org
Subject: Re: [Tutor] Python and boot sequences

On 28/06/12 13:54, Dave Wilder wrote:

> Can a Python script be written that has the ability to stop a Linux 
> device in the middle of a boot when a certain sequence occurs and then 
> perform an action?

It depends...

> For example, when I boot my switch (w/ Linux OS 2.7.3), I want to stop 
> the boot when I see a series of asterisks.  When I see this, I need to 
> hit the <ESC><SHIFT><9>

Could you do it with a bash script? If so you could do it with Python.
But I've no idea whether your switch will be in a state to run user processes at that point in its boot sequence. If you don't have a shell environment its going to be a whole lot harder.

> doing a PXE boot,

???

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/



_______________________________________________
Tutor maillist  -  Tutor at python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

From gnj091405 at gmail.com  Mon Jul  9 03:10:11 2012
From: gnj091405 at gmail.com (Gregory Lund)
Date: Sun, 8 Jul 2012 18:10:11 -0700
Subject: [Tutor] Reading a csv of coordinates,
	trying to write a csv of bearings.
Message-ID: <CAK=ob9zx1CpTUcKXw1yjO-9LnjjCqGdXmmXye+fr63bWVES6OQ@mail.gmail.com>

I'm Not looking for an absolute solution, but perhaps some insight
into some snippets of code, or
	suggestions of where I should seek out answers to this issue.
Or where I've gone wrong below.
NOTE: currently this 'code' below reads my file and writes a file, but
what it's doing in the middle isn't what it's supposed to do (I know
that!) I just modified what I had to at least read/write, now I need
to add the atan2() stuff.

I'm currently reading 'learning python' (Lutz) for 2.6 (not 3.0!)
I must use 2.6 because that is what esri's ArcGIS 10.0 uses.
I'm Doing (1/2 way through)  the non-programmers tutorial for python
2.6, hoping to 'graduate' to the
http://docs.python.org/release/2.6.8/tutorial/index.html once I get
through the non-programmers tutorial.

Objectives: read a CSV file of coordinates, and generate a
    	new CSV file of bearings.

Provided: A valid CSV file with coordinates (every other line blank,
if that matters)

Intended result: A valid CSV file with bearings

Logic?: one function, accepting a filename for the input, and a filename
	for the output (i.e. Coord_to_bearing(infile, outfile))

Caveats:
	A bearing is declination from north (meaning the angle between the
            direction and north).
	2 points define a line (y = mx + b). (obviously)
	Supposedly atan2() (from import math) will be in the code
	http://en.wikipedia.org/wiki/Atan2

Sample input:
	0,1,1

	1,2,2

	2,3,3

	3,4,3

	4,1268144.125,226540.2969
		
	5,1268463.75,226260.1563


Sample output:
        OID1,OID2,ANGLE
	0,1,45.0
	1,2,45.0
	2,3,90.0
	3,4,?
	4,5,?
	

Since I am a beginner, I re-used some code from a previous task, and
kept lots of it
	commented out to use to help with future coding.
Partial Code written in pursuit of this goal (with limited success)
	it's messy, and incorrect, but it's a start, maybe hurting more than helping.

'''
The code herin was initially written to find a nmea sample file
('nmea_sample.txt) within the same folder as this .py file.
It has been modified to read the giveN CSV file of coordinates, and
generate a new CSV file of bearings.

Python will try to find the first parameter (ie, coordinates.csv) in
the same folder in which the .py file is saved.
The second parameter is the output that will be saved in the same
folder automatically.
Put another way... The new .csv file will be saved to the same folder
in which  .py file is saved.
    I could use user input but trying to keep it simple.


1 Define a function that will be run (he name of the file, in the
working directory so that Python
    knows what file to find, and where.
2. open the file
3. "line' = f.readline()" establishes 'line' and reads/returns one line from f's
    file, up to the end of the line.
4. read the file line by line (while loop)
5. assigning 'data' as splitting the line by commas (',')
6. Extract information by line
7. Calculating the Bearings #this step is missing
8. Creating a new CSV file.

# I left lots of comments and 'old' code in so that I could use this
as a learning experience.


'''
import math     #importing math so that atan2 is available
#1 Establishing the filenames

def coord2bearing(coordFile,csvFile):   # defining a function that
will be used later, a return is required at the end when function is
done.
                                        # these two parameters will be
filled when calling this function in the python Shell.
    import csv #2importing the csv module that has the necessary tools
I need to read/create a csv file.

    csv_writer = csv.writer(open(csvFile, 'wb'), delimiter = ',') #
not sure if I need the ',' in here yet?.
                                                                  #
creating a csv file by using csv module's tool called 'writer' and
'open';
                                                                  #
'csv_writer' is just a variable name that could be anything
                                                                  #
defining 'csv_writer' as an object that has attributes and tools
                                                                  #
Important Note: Using 'wb' instead of 'w' caused the csv file to have
line breaks.
    myfile = open (coordFile) # open the original sample data file by
using the open function (coordinates2.csv) (original coordinates.csv
had blank lines)
    filename = 'coordinates.csv'

    #2 The 'open()' command opens the extablished filename.
    f = open(filename)

    #3 applying 'line' and the f.readline statement

    line = f.readline() #3

    #4 read the file line by line in a 'while' loop.
    OID = 0 # setting the OID varible to zero
    while (line): #...while there is a line to go to...

    #5 assigning 'data' as splitting the line by commas (',')

        data = line.split(',') #not sure yet how the .csv is read, if
it 'sees' a comma or not...?

        if data[0] >= 0: #6just using this to keep indents the same as
previous use of this code as I play with it. should perhaps have it
test 'is string?' then quit.

            #Latitude
            lat_init = data[2] # initial latitude is the third item,
(3-spot in the csv line)
            y = float(lat_init) #Lat Degrees

            #Longitude
            long_init = data[1] # initial longitude is in the 2nd item
(1-spot in the csv line)
            x = float(long_init) #Long Degrees


            #7 Missing: Need to figure out how to convert the
coordinates to a bearing...
            radians = math.atan2(y, x) #aTan2 provides Radians
            degrees = radians*(180/math.pi)
            # Write results to a file
            csv_writer.writerow([OID,float(y),float(x),degrees]) # 8
write OID,y value, x value, degrees (currently) to the csv file.

            OID += 1

    #read the next line so the loop can restart
        line = f.readline()
    f.close()
    myfile.close() #should close 'myfile'.
    return # finish the initial 'def' function line 1

#the following was added so that the user does not have to manualy
modify the code. or use the shell to get it to work, just run it, and
it works.

#You may modify the input .txt file and output location below.
coord2bearing # this command is needed to 'kickstart' the process (so
to speak). (calling it?)
#The full path and file name of the input file (original  sample csv file)
a = r"D:\D_Drive_Documents\Lund_CSV\coordinates.csv"  #the 'r'
preceeds the path so that I don't have to use '\\'
#The full path and file name of the output file (A New CSV file)
b = r"D:\D_Drive_Documents\Lund_CSV\Lund_bearings_CSVfile.csv" #the
'r' is so I don't have to use '\\'
#Run the def with its arguments
coord2bearing(a,b)

From chare at labr.net  Mon Jul  9 03:10:10 2012
From: chare at labr.net (Chris Hare)
Date: Sun, 8 Jul 2012 20:10:10 -0500
Subject: [Tutor] using dynamic import statements
Message-ID: <E21702E0-DE68-4D00-85A2-8B76129A89A8@labr.net>


Here is what I want to do:

I have a bunch of modules to import. instead of duplicating a lot of code for each import, I want to do something like this:

importList = [ "sys", "os", "imp", "stat", "re", "webbrowser", "Image",  "StringIO", "shutil", "datetime" ]

for object in importList:
        try:
                if debug == "ON":
                        print "Importing module %s" % (object)
                exec( "import  " + object)
        except ImportError as error:
                print "%s %s requires the Python %s library.  " % ( appName,
                                                                str(appVersion), object )
                print "An error occurred when attempting to load the library.  The error was '%s'." % ( error )
                exit()

Everything "appears" to run okay, however, when the first piece of code that relies upon one of these imported modules is executed, I get an error:  

Traceback (most recent call last):
  File "a.py", line 122, in <module>
    imp.load_module(object,fp,pathName,description)
  File "./Modules/functions.py", line 133, in <module>
    def special_match(strg, search=re.compile(r'[^a-zA-Z0-9\.\ \-\#\$\*\@\!\%\^\&]').search):
NameError: name 're' is not defined

Is is possible to do what I am trying to do, or am I doing something wrong?

Thanks


From hugo.yoshi at gmail.com  Mon Jul  9 08:57:50 2012
From: hugo.yoshi at gmail.com (Hugo Arts)
Date: Mon, 9 Jul 2012 08:57:50 +0200
Subject: [Tutor] using dynamic import statements
In-Reply-To: <E21702E0-DE68-4D00-85A2-8B76129A89A8@labr.net>
References: <E21702E0-DE68-4D00-85A2-8B76129A89A8@labr.net>
Message-ID: <CAJmBOfngPzDSgV8Zii6HMUVh0P4uxkpAc14sYakHqfbhW+sZBg@mail.gmail.com>

On Mon, Jul 9, 2012 at 3:10 AM, Chris Hare <chare at labr.net> wrote:

>
> Here is what I want to do:
>
> I have a bunch of modules to import. instead of duplicating a lot of code
> for each import, I want to do something like this:
>
> importList = [ "sys", "os", "imp", "stat", "re", "webbrowser", "Image",
>  "StringIO", "shutil", "datetime" ]
>
> for object in importList:
>         try:
>                 if debug == "ON":
>                         print "Importing module %s" % (object)
>                 exec( "import  " + object)
>         except ImportError as error:
>                 print "%s %s requires the Python %s library.  " % (
> appName,
>
> str(appVersion), object )
>                 print "An error occurred when attempting to load the
> library.  The error was '%s'." % ( error )
>                 exit()
>
> Everything "appears" to run okay, however, when the first piece of code
> that relies upon one of these imported modules is executed, I get an error:
>
> Traceback (most recent call last):
>   File "a.py", line 122, in <module>
>     imp.load_module(object,fp,pathName,description)
>   File "./Modules/functions.py", line 133, in <module>
>     def special_match(strg, search=re.compile(r'[^a-zA-Z0-9\.\
> \-\#\$\*\@\!\%\^\&]').search):
> NameError: name 're' is not defined
>
> Is is possible to do what I am trying to do, or am I doing something wrong?
>
>
 Yes, it is possible, but you are doing something wrong. Don't use exec()
for this, it's unnecessary. You can make this work with the exec function,
but the function you really want is called __import__(). It's documented
here: http://docs.python.org/library/functions.html#__import__

HTH,
Hugo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120709/e67d2948/attachment.html>

From hugo.yoshi at gmail.com  Mon Jul  9 09:14:04 2012
From: hugo.yoshi at gmail.com (Hugo Arts)
Date: Mon, 9 Jul 2012 09:14:04 +0200
Subject: [Tutor] Reading a csv of coordinates,
	trying to write a csv of bearings.
In-Reply-To: <CAK=ob9zx1CpTUcKXw1yjO-9LnjjCqGdXmmXye+fr63bWVES6OQ@mail.gmail.com>
References: <CAK=ob9zx1CpTUcKXw1yjO-9LnjjCqGdXmmXye+fr63bWVES6OQ@mail.gmail.com>
Message-ID: <CAJmBOfmgE90QXc-xaTkknOZRiB+2sgFO80zXfc6-8NY9=VU2Bw@mail.gmail.com>

On Mon, Jul 9, 2012 at 3:10 AM, Gregory Lund <gnj091405 at gmail.com> wrote:

> I'm Not looking for an absolute solution, but perhaps some insight
> into some snippets of code, or
>         suggestions of where I should seek out answers to this issue.
> Or where I've gone wrong below.
> NOTE: currently this 'code' below reads my file and writes a file, but
> what it's doing in the middle isn't what it's supposed to do (I know
> that!) I just modified what I had to at least read/write, now I need
> to add the atan2() stuff.
>
>
What is "this issue," exactly? I can not seem to find in your e-mail a
description of what problems you're having exactly. On another note, when
pasting any code beyond a simple single-function example, please consider
using a service such as http://pastebin.com/. Your code becomes very hard
to read once e-mail client formatting is done with it.

Hugo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120709/7186095f/attachment-0001.html>

From hugo.yoshi at gmail.com  Mon Jul  9 09:42:33 2012
From: hugo.yoshi at gmail.com (Hugo Arts)
Date: Mon, 9 Jul 2012 09:42:33 +0200
Subject: [Tutor] Mapping ID's for corresponding values in different
	Columns
In-Reply-To: <CAMg+7mZ5-mt9-kO4MPzF5G9xw7_sJ_=2TASesMzGUCpEiuMCPQ@mail.gmail.com>
References: <CAMg+7mZ5-mt9-kO4MPzF5G9xw7_sJ_=2TASesMzGUCpEiuMCPQ@mail.gmail.com>
Message-ID: <CAJmBOfkYoYSdeO+uq+W8bGcqutwfPOc9QxAdgDGgsx_Bo071Gg@mail.gmail.com>

On Sun, Jul 8, 2012 at 11:47 PM, Fred G <bayespokerguy at gmail.com> wrote:

> Hi--
>
> My current input looks like the following:
>
> FILE1.csv
> PERSON_ID    PERSON_NAME
> 1                     Jen
> 2                     Mike
> 3                     Jim
> 4
> 5                     Jane
> 6                     Joe
> 7                     Jake
>
> FILE2.csv
> PERSON_ID   PERSON_NAME
>                      Jim
>                      Mike
>                      Jane
>                      Todd
>                      Jen
>
> _________
> I want to fill into the PERSON_ID column of FILE2.csv the corresponding
> ID's associated with those names as identified in FILE1.csv.
>
> At first I imported the csv module and was using the csv.Reader, but then
> it seemed simple enough just to write something like:
> for line in file2:
>      print(line)
>
> giving me the following output:
> PERSON_ID, PERSON_NAME
> , Jim
> , Mike
> , Jane
> , Todd
> , Jen
>
> I think I understand the issue at a conceptual level, but not quite sure
> how to fully implement it:
> a) I want to build a dictionary to create keys, such that each number in
> file1 corresponds to a unique string in column B of file1.
> b) then write a for loop like the following:
> for "person_name" in file2:
>    if "person_name.file2" == "person_name.file1":
>        person_id.file2 == person_id.file1
> c) write into file2 the changes to person_id's...
>
> But it's pretty difficult for me to get past this stage. Am I on the right
> track? And more importantly, how could I learn how to actually implement
> this in smaller stages?
>
>
 You're on the right track, and you're almost there! You've already broken
down the problem into steps. You should now try to implement a function for
each step, and finally you should glue these functions together into a
final program.

a) Though you don't *have* to use it, csv.reader is really quite simple,
I'd recommend it. Try and write a function that takes a file name as
argument and returns a dictionary of the form { name: id, name: id } (i.e.
the names are the keys).

b) For this step, you first need a list of all names in file 2. You could
use csv.reader again or you could just parse it. Then, you use the
dictionary to look up the corresponding id. The end goal for this function
is to return a list of lists that looks much like the file you want to end
up with:

[[id, name],
 [id, name],
 [id, name]]

c) this step should now be easy. I'd again, recommend csv.writer, it makes
the process pretty simple. You just pass in the nested list from step (b)
and you're pretty much done.

For tips on the csv module, the list of examples is pretty helpful:
http://docs.python.org/py3k/library/csv.html#examples
If you need help constructing the lists and dictionaries, my tips would be
1) think one row at a time, 2) the for loop is your best friend, and 3)
nested lists usually means nested loops

HTH,
Hugo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120709/26ac5f19/attachment.html>

From alan.gauld at btinternet.com  Mon Jul  9 09:47:04 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 09 Jul 2012 08:47:04 +0100
Subject: [Tutor] using dynamic import statements
In-Reply-To: <E21702E0-DE68-4D00-85A2-8B76129A89A8@labr.net>
References: <E21702E0-DE68-4D00-85A2-8B76129A89A8@labr.net>
Message-ID: <jte29r$cam$1@dough.gmane.org>

On 09/07/12 02:10, Chris Hare wrote:
>
> Here is what I want to do:
>
> I have a bunch of modules to import. instead of duplicating
 > a lot of code for each import, I want to do something like this:

...lots of code stripped....


Why not the more usual:

import sys, os, imp, stat,    \
        re, webbrowser, Image, \
        StringIO, shutil, datetime

And let the Python ImportError alert you to what failed?

Or if you must print your own errors just use

try:
    import ...
except ImportError:
    # your code here

I'm not sure what code you think you need to duplicate or why.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From ganu.ullu at gmail.com  Mon Jul  9 10:14:32 2012
From: ganu.ullu at gmail.com (=?UTF-8?B?YW5rdXIgfiDgpIXgpILgpJXgpYHgpLA=?=)
Date: Mon, 9 Jul 2012 13:44:32 +0530
Subject: [Tutor] using dynamic import statements
In-Reply-To: <E21702E0-DE68-4D00-85A2-8B76129A89A8@labr.net>
References: <E21702E0-DE68-4D00-85A2-8B76129A89A8@labr.net>
Message-ID: <CAOacDnBSQcMAqVDoeMub6HgnqQwH20nwYXuiZZA+rW0aG5Rf=g@mail.gmail.com>

On Mon, Jul 9, 2012 at 6:40 AM, Chris Hare <chare at labr.net> wrote:

>
> Here is what I want to do:
>
> I have a bunch of modules to import. instead of duplicating a lot of code
> for each import, I want to do something like this:
>
> importList = [ "sys", "os", "imp", "stat", "re", "webbrowser", "Image",
>  "StringIO", "shutil", "datetime" ]
>
> for object in importList:
>         try:
>                 if debug == "ON":
>                         print "Importing module %s" % (object)
>                 exec( "import  " + object)
>         except ImportError as error:
>                 print "%s %s requires the Python %s library.  " % (
> appName,
>
> str(appVersion), object )
>                 print "An error occurred when attempting to load the
> library.  The error was '%s'." % ( error )
>                 exit()
>
> Everything "appears" to run okay, however, when the first piece of code
> that relies upon one of these imported modules is executed, I get an error:
>
> Traceback (most recent call last):
>   File "a.py", line 122, in <module>
>     imp.load_module(object,fp,pathName,description)
>   File "./Modules/functions.py", line 133, in <module>
>     def special_match(strg, search=re.compile(r'[^a-zA-Z0-9\.\
> \-\#\$\*\@\!\%\^\&]').search):
> NameError: name 're' is not defined
>
> Is is possible to do what I am trying to do, or am I doing something wrong?
>
> Thanks
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

Hi,

How about using import imp
http://docs.python.org/library/imp.html

calling run time modules as per your need.

thank you
ankur.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120709/8e832faf/attachment.html>

From kwpolska at gmail.com  Mon Jul  9 11:19:09 2012
From: kwpolska at gmail.com (Kwpolska)
Date: Mon, 9 Jul 2012 11:19:09 +0200
Subject: [Tutor] using dynamic import statements
In-Reply-To: <CAMw+j7LT75ZAJGzdSWhXjZckYJyEj6jK-f4nNUbhknSV0gujog@mail.gmail.com>
References: <E21702E0-DE68-4D00-85A2-8B76129A89A8@labr.net>
	<jte29r$cam$1@dough.gmane.org>
	<CAMw+j7LT75ZAJGzdSWhXjZckYJyEj6jK-f4nNUbhknSV0gujog@mail.gmail.com>
Message-ID: <CAMw+j7KsSZkcyh2Z8kHp7zMFgo=ceynLQRVKtdvDttLjKCtaVw@mail.gmail.com>

Why does this bloody ML want me to respond to the last person instead
of tutor at python.org?
---
On Mon, Jul 9, 2012 at 9:47 AM, Alan Gauld <alan.gauld at btinternet.com> wrote:
> On 09/07/12 02:10, Chris Hare wrote:
>>
>>
>> Here is what I want to do:
>>
>> I have a bunch of modules to import. instead of duplicating
>
>> a lot of code for each import, I want to do something like this:
>
> ...lots of code stripped....
>
>
> Why not the more usual:
>
> import sys, os, imp, stat,    \
>        re, webbrowser, Image, \
>        StringIO, shutil, datetime
>
> [snip]

Why not the more standard:
import sys
import os
and so on?  http://www.python.org/dev/peps/pep-0008/#imports
-- 
Kwpolska <http://kwpolska.tk>
stop html mail      | always bottom-post
www.asciiribbon.org | www.netmeister.org/news/learn2quote.html
GPG KEY: 5EAAEA16   | Arch Linux x86_64, zsh, mutt, vim.
# vim:set textwidth=70:

From alan.gauld at btinternet.com  Mon Jul  9 13:12:04 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 09 Jul 2012 12:12:04 +0100
Subject: [Tutor] using dynamic import statements
In-Reply-To: <CAMw+j7KsSZkcyh2Z8kHp7zMFgo=ceynLQRVKtdvDttLjKCtaVw@mail.gmail.com>
References: <E21702E0-DE68-4D00-85A2-8B76129A89A8@labr.net>
	<jte29r$cam$1@dough.gmane.org>
	<CAMw+j7LT75ZAJGzdSWhXjZckYJyEj6jK-f4nNUbhknSV0gujog@mail.gmail.com>
	<CAMw+j7KsSZkcyh2Z8kHp7zMFgo=ceynLQRVKtdvDttLjKCtaVw@mail.gmail.com>
Message-ID: <jteea4$9bv$1@dough.gmane.org>

On 09/07/12 10:19, Kwpolska wrote:
> Why does this bloody ML want me to respond to the last person instead
> of tutor at python.org?

Because that's how it, along with many other mailing lists, works.
If it helps, think of it as you receiving a mail from the sender and CCd 
to the list. Therefore hitting reply sends to the person who sent the 
mail and ReplyAll goes to everyone. Seems logical to me! :-)
It allows me to choose to reply to the OP only or to the group, I use 
both depending on the nature of my reply. (About 80% of my replies go to 
the everyone.) But some prefer it differently... :-)

>> Why not the more usual:
>>
>> import sys, os, imp, stat,    \
>>         re, webbrowser, Image, \
>>         StringIO, shutil, datetime
>
> Why not the more standard:
> import sys
> import os
> and so on?  http://www.python.org/dev/peps/pep-0008/#imports

Indeed but the OP seemed to want to remove duplicate coding so I assumed 
that he included using multiple imports.

Personally I'd probably code the above as:

import sys, os, shutil, stat, datetime
import re, StringIO
import webbrowser,
import Image,
import imp

Which groups things into roughly related categories - system,
strings, other...

OTOH I ignore large chunks of Pep 8 because I find its style harder to 
read than the one I'm used to. But then, I'm not contributing to the 
standard library etc... Style is largely a matter of taste.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From joel.goldstick at gmail.com  Mon Jul  9 13:53:07 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 9 Jul 2012 07:53:07 -0400
Subject: [Tutor] Reading a csv of coordinates,
	trying to write a csv of bearings.
In-Reply-To: <CAJmBOfmgE90QXc-xaTkknOZRiB+2sgFO80zXfc6-8NY9=VU2Bw@mail.gmail.com>
References: <CAK=ob9zx1CpTUcKXw1yjO-9LnjjCqGdXmmXye+fr63bWVES6OQ@mail.gmail.com>
	<CAJmBOfmgE90QXc-xaTkknOZRiB+2sgFO80zXfc6-8NY9=VU2Bw@mail.gmail.com>
Message-ID: <CAPM-O+xVbESPSaaVGH=UirtEQ7zmirgVTiu9nVBzVrNuy8rqhg@mail.gmail.com>

On Mon, Jul 9, 2012 at 3:14 AM, Hugo Arts <hugo.yoshi at gmail.com> wrote:
> On Mon, Jul 9, 2012 at 3:10 AM, Gregory Lund <gnj091405 at gmail.com> wrote:
>>
>> I'm Not looking for an absolute solution, but perhaps some insight
>> into some snippets of code, or
>>         suggestions of where I should seek out answers to this issue.
>> Or where I've gone wrong below.

1. Look at the csv module.  It makes dealing with csv very convenient.
2. The fact that every other line in your input file is blank will
require you to through out those lines.  One way to do this is with
slicing:
    >>> a = [0,1,2,3,4,5,6]
    >>> b = a[::2]
    >>> b
    [0, 2, 4, 6]
    >>>
3. I would read the input into a list of lists (each line item is in a
list, each line is in the containing list
4. get rid of the blanks
5. make a function to take the input lists and do your math,
producting a list of lists to be output
6. use the csv writer to write to a file
7. You have way to many comments in your code.  At the top of your
file, and immediately underneath each function definition use
docstrings (triple quoted multi-line strings).  These are great in
that they provide automatic documentation for your functions using
pydocs, or when you are in the python shell you can do
help(function_name) and find out what it does



-- 
Joel Goldstick

From chare at labr.net  Mon Jul  9 14:10:55 2012
From: chare at labr.net (Chris Hare)
Date: Mon, 9 Jul 2012 07:10:55 -0500
Subject: [Tutor] using dynamic import statements
In-Reply-To: <jteea4$9bv$1@dough.gmane.org>
References: <E21702E0-DE68-4D00-85A2-8B76129A89A8@labr.net>
	<jte29r$cam$1@dough.gmane.org>
	<CAMw+j7LT75ZAJGzdSWhXjZckYJyEj6jK-f4nNUbhknSV0gujog@mail.gmail.com>
	<CAMw+j7KsSZkcyh2Z8kHp7zMFgo=ceynLQRVKtdvDttLjKCtaVw@mail.gmail.com>
	<jteea4$9bv$1@dough.gmane.org>
Message-ID: <DD55226C-F548-4849-A93C-56FA0A1E0B6C@labr.net>


Thanks all for the ideas.  I wanted to have my own error messages printed for the user - something a little more meaningful than the standard error. 

Thanks for the advice - very helpful!

On Jul 9, 2012, at 6:12 AM, Alan Gauld wrote:

> On 09/07/12 10:19, Kwpolska wrote:
>> Why does this bloody ML want me to respond to the last person instead
>> of tutor at python.org?
> 
> Because that's how it, along with many other mailing lists, works.
> If it helps, think of it as you receiving a mail from the sender and CCd to the list. Therefore hitting reply sends to the person who sent the mail and ReplyAll goes to everyone. Seems logical to me! :-)
> It allows me to choose to reply to the OP only or to the group, I use both depending on the nature of my reply. (About 80% of my replies go to the everyone.) But some prefer it differently... :-)
> 
>>> Why not the more usual:
>>> 
>>> import sys, os, imp, stat,    \
>>>        re, webbrowser, Image, \
>>>        StringIO, shutil, datetime
>> 
>> Why not the more standard:
>> import sys
>> import os
>> and so on?  http://www.python.org/dev/peps/pep-0008/#imports
> 
> Indeed but the OP seemed to want to remove duplicate coding so I assumed that he included using multiple imports.
> 
> Personally I'd probably code the above as:
> 
> import sys, os, shutil, stat, datetime
> import re, StringIO
> import webbrowser,
> import Image,
> import imp
> 
> Which groups things into roughly related categories - system,
> strings, other...
> 
> OTOH I ignore large chunks of Pep 8 because I find its style harder to read than the one I'm used to. But then, I'm not contributing to the standard library etc... Style is largely a matter of taste.
> 
> -- 
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> 
> 
> 
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor


From wolfrage8765 at gmail.com  Mon Jul  9 14:47:41 2012
From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com)
Date: Mon, 9 Jul 2012 14:47:41 +0200
Subject: [Tutor] using dynamic import statements
In-Reply-To: <DD55226C-F548-4849-A93C-56FA0A1E0B6C@labr.net>
References: <E21702E0-DE68-4D00-85A2-8B76129A89A8@labr.net>
	<jte29r$cam$1@dough.gmane.org>
	<CAMw+j7LT75ZAJGzdSWhXjZckYJyEj6jK-f4nNUbhknSV0gujog@mail.gmail.com>
	<CAMw+j7KsSZkcyh2Z8kHp7zMFgo=ceynLQRVKtdvDttLjKCtaVw@mail.gmail.com>
	<jteea4$9bv$1@dough.gmane.org>
	<DD55226C-F548-4849-A93C-56FA0A1E0B6C@labr.net>
Message-ID: <CAOhNYv=fgz3t5=G2yLTvtfvZbrUVzUZjGXOsrNQN3APmfh4uHg@mail.gmail.com>

Might I recommend you use the logging module that is part of core
Python rather than rolling your own if Debug. I found the logging
module made growing my logging across multiple applications so easy!
http://docs.python.org/library/logging.html

On Mon, Jul 9, 2012 at 2:10 PM, Chris Hare <chare at labr.net> wrote:
>
> Thanks all for the ideas.  I wanted to have my own error messages printed for the user - something a little more meaningful than the standard error.
>
> Thanks for the advice - very helpful!
>
> On Jul 9, 2012, at 6:12 AM, Alan Gauld wrote:
>
>> On 09/07/12 10:19, Kwpolska wrote:
>>> Why does this bloody ML want me to respond to the last person instead
>>> of tutor at python.org?
>>
>> Because that's how it, along with many other mailing lists, works.
>> If it helps, think of it as you receiving a mail from the sender and CCd to the list. Therefore hitting reply sends to the person who sent the mail and ReplyAll goes to everyone. Seems logical to me! :-)
>> It allows me to choose to reply to the OP only or to the group, I use both depending on the nature of my reply. (About 80% of my replies go to the everyone.) But some prefer it differently... :-)
>>
>>>> Why not the more usual:
>>>>
>>>> import sys, os, imp, stat,    \
>>>>        re, webbrowser, Image, \
>>>>        StringIO, shutil, datetime
>>>
>>> Why not the more standard:
>>> import sys
>>> import os
>>> and so on?  http://www.python.org/dev/peps/pep-0008/#imports
>>
>> Indeed but the OP seemed to want to remove duplicate coding so I assumed that he included using multiple imports.
>>
>> Personally I'd probably code the above as:
>>
>> import sys, os, shutil, stat, datetime
>> import re, StringIO
>> import webbrowser,
>> import Image,
>> import imp
>>
>> Which groups things into roughly related categories - system,
>> strings, other...
>>
>> OTOH I ignore large chunks of Pep 8 because I find its style harder to read than the one I'm used to. But then, I'm not contributing to the standard library etc... Style is largely a matter of taste.
>>
>> --
>> Alan G
>> Author of the Learn to Program web site
>> http://www.alan-g.me.uk/
>>
>>
>>
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/mailman/listinfo/tutor
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From bayespokerguy at gmail.com  Mon Jul  9 17:23:44 2012
From: bayespokerguy at gmail.com (Fred G)
Date: Mon, 9 Jul 2012 08:23:44 -0700
Subject: [Tutor] Mapping ID's for corresponding values in different
	Columns
In-Reply-To: <CAJmBOfkYoYSdeO+uq+W8bGcqutwfPOc9QxAdgDGgsx_Bo071Gg@mail.gmail.com>
References: <CAMg+7mZ5-mt9-kO4MPzF5G9xw7_sJ_=2TASesMzGUCpEiuMCPQ@mail.gmail.com>
	<CAJmBOfkYoYSdeO+uq+W8bGcqutwfPOc9QxAdgDGgsx_Bo071Gg@mail.gmail.com>
Message-ID: <CAMg+7masO3b3vyf80gZXer5HAoT7HTeYYhbzXtQXGP0=1JA10A@mail.gmail.com>

Thank you guys so much.  I'm quite close now, but I'm having a bit of
trouble on the final for-loop to create the new dictionary.  I have the
following 3 functions (note I'm re-typing it from a different computer so
while the identation will be off here, it is correct in the actual code):

#read in file as dictionary.  Name is key, id is value.
def csv_to_dict(filename):
  record = {}
  line = filename.readline()
  for line in filename:
    key = line.split(",", 1)[-1]
    val = line.split(",", 1)[:-1]
  return record

#read in file2 names
def nested_line(filename):
  line = filename.readline()
  new_list = []
  for line in filename:
    name = line.split(",", 1)[-1]
    new_list.append(name)
  return new_list

#this is the function that I'm having trouble with
#create new dict mapping file 1 ids to file2's names
def new_dict (csv_to_dict, nested_line):
  old_dict = cssv_to_dict(file1)
  old_list = nested_line(file2)
  new_dict1 = {}
  for item in old_list:
    new_dict1[item] = item
  return (new_dict1)

I have tried various permutations of the for loop in this final function,
but I haven't quite gotten it.  The current form is the closest I have
gotten to my desired output, since it produces the new dictionary with the
name that I want-- just an invalid id associated with that name.  I tried a
bunch of different nested loops but kept getting it incorrect and then
after so many attempts I got a little confused about what I was trying to
do.  So I backed up a little bit and have this.

Conceptually, I thought that this would give me my desired result:
new_dict
for name in old_list:
  for key, value in old_dict:
    if name == key:
      new_dict1[key] = value
return(new_dict1)

But it wasn't right.  I tried a different approach where I used the
dict.values() function in order to pull out the values from old_dict that
we want to include in the new_dict, but I got a bit lost there, too.

I'm so close right now and I would be so thankful for any bit of
clarification which could get me to the finish line.

On Mon, Jul 9, 2012 at 12:42 AM, Hugo Arts <hugo.yoshi at gmail.com> wrote:

> On Sun, Jul 8, 2012 at 11:47 PM, Fred G <bayespokerguy at gmail.com> wrote:
>
>> Hi--
>>
>> My current input looks like the following:
>>
>> FILE1.csv
>> PERSON_ID    PERSON_NAME
>> 1                     Jen
>> 2                     Mike
>> 3                     Jim
>> 4
>> 5                     Jane
>> 6                     Joe
>> 7                     Jake
>>
>> FILE2.csv
>> PERSON_ID   PERSON_NAME
>>                      Jim
>>                      Mike
>>                      Jane
>>                      Todd
>>                      Jen
>>
>> _________
>> I want to fill into the PERSON_ID column of FILE2.csv the corresponding
>> ID's associated with those names as identified in FILE1.csv.
>>
>> At first I imported the csv module and was using the csv.Reader, but then
>> it seemed simple enough just to write something like:
>> for line in file2:
>>      print(line)
>>
>> giving me the following output:
>> PERSON_ID, PERSON_NAME
>> , Jim
>> , Mike
>> , Jane
>> , Todd
>> , Jen
>>
>> I think I understand the issue at a conceptual level, but not quite sure
>> how to fully implement it:
>> a) I want to build a dictionary to create keys, such that each number in
>> file1 corresponds to a unique string in column B of file1.
>> b) then write a for loop like the following:
>> for "person_name" in file2:
>>    if "person_name.file2" == "person_name.file1":
>>        person_id.file2 == person_id.file1
>> c) write into file2 the changes to person_id's...
>>
>> But it's pretty difficult for me to get past this stage. Am I on the
>> right track? And more importantly, how could I learn how to actually
>> implement this in smaller stages?
>>
>>
>  You're on the right track, and you're almost there! You've already broken
> down the problem into steps. You should now try to implement a function for
> each step, and finally you should glue these functions together into a
> final program.
>
> a) Though you don't *have* to use it, csv.reader is really quite simple,
> I'd recommend it. Try and write a function that takes a file name as
> argument and returns a dictionary of the form { name: id, name: id } (i.e.
> the names are the keys).
>
> b) For this step, you first need a list of all names in file 2. You could
> use csv.reader again or you could just parse it. Then, you use the
> dictionary to look up the corresponding id. The end goal for this function
> is to return a list of lists that looks much like the file you want to end
> up with:
>
> [[id, name],
>  [id, name],
>  [id, name]]
>
> c) this step should now be easy. I'd again, recommend csv.writer, it makes
> the process pretty simple. You just pass in the nested list from step (b)
> and you're pretty much done.
>
> For tips on the csv module, the list of examples is pretty helpful:
> http://docs.python.org/py3k/library/csv.html#examples
> If you need help constructing the lists and dictionaries, my tips would be
> 1) think one row at a time, 2) the for loop is your best friend, and 3)
> nested lists usually means nested loops
>
> HTH,
> Hugo
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120709/9884d4ee/attachment-0001.html>

From tokyo.rook at gmail.com  Mon Jul  9 17:31:29 2012
From: tokyo.rook at gmail.com (Keitaro Kaoru)
Date: Mon, 9 Jul 2012 11:31:29 -0400
Subject: [Tutor] TypeError: not all arguments converted during string
	formatting
Message-ID: <CAM9vinPACw4TyGve5zkX=75O-VLRgX+g6axoRsqO_aBYJVfAGQ@mail.gmail.com>

Traceback (most recent call last):
  File "bot.py", line 351, in <module>
    mgr.main()
  File "/home/bot/bot/ch.py", line 1672, in main
    con._feed(data)
  File "/home/bot/bot/ch.py", line 628, in _feed
    self._process(food.decode("latin-1").rstrip("\r\n")) #numnumz ;3
  File "/home/bot/bot/ch.py", line 643, in _process
    getattr(self, func)(args)
  File "/home/bot/bot/ch.py", line 744, in rcmd_u
    self._callEvent("onMessage", msg.user, msg)
  File "/home/bot/bot/ch.py", line 1091, in _callEvent
    self.mgr.onEventCalled(self, evt, *args, **kw)
  File "bot.py", line 287, in onEventCalled
    self.invokeListeners(room, evt, *args, **kw)
  File "bot.py", line 110, in invokeListeners
    getattr(lis, evt)(self, room, *args, **kw)
  File "modules/lulz2.py", line 47, in onMessage
    self.sendObject(room, ret)
  File "bot.py", line 307, in sendObject
    obj = obj.__botOut__(self, room)
  File "/home/bot/bot/tools.py", line 226, in __botOut__
    return self.html % ca
TypeError: not all arguments converted during string formatting




class Text:
	def __init__(self, text):
		self.text = text
	
	def __botOut__(self, mgr, room):
		return cgi.escape(self.text)

class Html:
	def __init__(self, html, *args):
		self.html = html
		self.args = args
	
	def __botOut__(self, mgr, room):
		if len(self.args) == 0:
			return self.html
		else:
			ca = tuple([cgi.escape(arg) if type(arg) == str else arg for arg in
self.args])
		return self.html % ca

cant seem to find out whats wrong with it
-- 
~~Austin

From joel.goldstick at gmail.com  Mon Jul  9 17:46:35 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 9 Jul 2012 11:46:35 -0400
Subject: [Tutor] TypeError: not all arguments converted during string
	formatting
In-Reply-To: <CAM9vinPACw4TyGve5zkX=75O-VLRgX+g6axoRsqO_aBYJVfAGQ@mail.gmail.com>
References: <CAM9vinPACw4TyGve5zkX=75O-VLRgX+g6axoRsqO_aBYJVfAGQ@mail.gmail.com>
Message-ID: <CAPM-O+yJSVwsXW=d45HmKLwSwwYufHepTmHD2i5t-oTi5=Bdeg@mail.gmail.com>

On Mon, Jul 9, 2012 at 11:31 AM, Keitaro Kaoru <tokyo.rook at gmail.com> wrote:
> Traceback (most recent call last):
>   File "bot.py", line 351, in <module>
>     mgr.main()
>   File "/home/bot/bot/ch.py", line 1672, in main
>     con._feed(data)
>   File "/home/bot/bot/ch.py", line 628, in _feed
>     self._process(food.decode("latin-1").rstrip("\r\n")) #numnumz ;3
>   File "/home/bot/bot/ch.py", line 643, in _process
>     getattr(self, func)(args)
>   File "/home/bot/bot/ch.py", line 744, in rcmd_u
>     self._callEvent("onMessage", msg.user, msg)
>   File "/home/bot/bot/ch.py", line 1091, in _callEvent
>     self.mgr.onEventCalled(self, evt, *args, **kw)
>   File "bot.py", line 287, in onEventCalled
>     self.invokeListeners(room, evt, *args, **kw)
>   File "bot.py", line 110, in invokeListeners
>     getattr(lis, evt)(self, room, *args, **kw)
>   File "modules/lulz2.py", line 47, in onMessage
>     self.sendObject(room, ret)
>   File "bot.py", line 307, in sendObject
>     obj = obj.__botOut__(self, room)
>   File "/home/bot/bot/tools.py", line 226, in __botOut__
>     return self.html % ca
> TypeError: not all arguments converted during string formatting
>
>
>
>
> class Text:
>         def __init__(self, text):
>                 self.text = text
>
>         def __botOut__(self, mgr, room):
>                 return cgi.escape(self.text)
>
> class Html:
>         def __init__(self, html, *args):
>                 self.html = html
>                 self.args = args
>
>         def __botOut__(self, mgr, room):
>                 if len(self.args) == 0:
>                         return self.html
>                 else:
>                         ca = tuple([cgi.escape(arg) if type(arg) == str else arg for arg in
> self.args])
>                 return self.html % ca
>
> cant seem to find out whats wrong with it

You don't show what self.html is.  It must have the same number of %s
values as the length of ca
> --
> ~~Austin
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor



-- 
Joel Goldstick

From steve at pearwood.info  Mon Jul  9 17:47:42 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 10 Jul 2012 01:47:42 +1000
Subject: [Tutor] TypeError: not all arguments converted during string
 formatting
In-Reply-To: <CAM9vinPACw4TyGve5zkX=75O-VLRgX+g6axoRsqO_aBYJVfAGQ@mail.gmail.com>
References: <CAM9vinPACw4TyGve5zkX=75O-VLRgX+g6axoRsqO_aBYJVfAGQ@mail.gmail.com>
Message-ID: <4FFAFD1E.6090701@pearwood.info>

Keitaro Kaoru wrote:
[...]
> TypeError: not all arguments converted during string formatting
[...]
> 		return self.html % ca
> 
> cant seem to find out whats wrong with it


Try experimenting at the interactive interpreter:


[steve at ando ~]$ python
Python 2.6.7 (r267:88850, Mar 10 2012, 12:32:58)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
py> "%s - %s - %s" % (1, 2, 3)
'1 - 2 - 3'

py> "%s - %s - %s" % (1, 2)  # Not enough values
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: not enough arguments for format string

py> "%s - %s - %s" % (1, 2, 3, 4)  # Too many values
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: not all arguments converted during string formatting


-- 
Steven

From chare at labr.net  Mon Jul  9 17:56:48 2012
From: chare at labr.net (Chris Hare)
Date: Mon, 9 Jul 2012 10:56:48 -0500
Subject: [Tutor] confusion about imports
Message-ID: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>


So, I have to admit, imports have me really confused.  I am trying to break apart a 10,000+ line single file into various files, one for each class, and one containing a whole bunch of functions which are used by a lot of classes.  Some of those functions use calls to methods in a Class.  Even though the Class has been imported, I get a nameError where trying to use the class.  I have read about Classes and packages and modules, but import just has me confused.

From joel.goldstick at gmail.com  Mon Jul  9 18:13:41 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 9 Jul 2012 12:13:41 -0400
Subject: [Tutor] confusion about imports
In-Reply-To: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
References: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
Message-ID: <CAPM-O+wd3t7rxQX72WyyjRTYz-rhWinxHAMN-HA4SFxxYZg7gg@mail.gmail.com>

On Mon, Jul 9, 2012 at 11:56 AM, Chris Hare <chare at labr.net> wrote:
>
> So, I have to admit, imports have me really confused.  I am trying to break apart a 10,000+ line single file into various files, one for each class, and one containing a whole bunch of functions which are used by a lot of classes.  Some of those functions use calls to methods in a Class.  Even though the Class has been imported, I get a nameError where trying to use the class.  I have read about Classes and packages and modules, but import just has me confused.


a 10k line file is a scary thought!  Can you make a very small example
that shows your problem.  Something like

import whatever

my_thing = whatever.classname()


and tell us if you get the error?

This is a wild guess, but from reading your question I'm wondering if
you are instantiating the class?
-- 
Joel Goldstick

From wprins at gmail.com  Mon Jul  9 19:42:03 2012
From: wprins at gmail.com (Walter Prins)
Date: Mon, 9 Jul 2012 18:42:03 +0100
Subject: [Tutor] confusion about imports
In-Reply-To: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
References: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
Message-ID: <CANLXbfCkeMvWX5vc+Ux+EJ=acaOQv7hOeMVYja5TRRAm0Wfeyw@mail.gmail.com>

Hi Chris

> So, I have to admit, imports have me really confused.  I am trying to break apart a 10,000+ line single file into various files, one for each class, and one containing a whole bunch of functions which are used by a lot of classes.  Some of those functions use calls to methods in a Class.  Even though the Class has been imported, I get a nameError where trying to use the class.  I have read about Classes and packages and modules, but import just has me confused.

How did you import the class?  Or did you perhaps not import the Class
itself, but rather the module containing the class?

Read this article which explains a bit about Python namespaces:
http://is.gd/e8PAZW  (but I note there's a bit of conflation of
"class" and "class instance" going on at the end.)

Also read this page from the Python documentation, section "Python
Scopes and Namespaces": http://docs.python.org/tutorial/classes.html

If a class is defined in a module (e.g. in the namespace of the
module), and you "import module" the module into the your current
namespace, then from within the current namespace you can access the
class with "module.Class".  If however you import the class itself,
e.g. "from module import Class", into your current namespace, then the
Class itself is directly part of your local namespace and you must
therefore access it unqualified as just "Class".

HTH,

Walter

From breamoreboy at yahoo.co.uk  Mon Jul  9 19:51:48 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 09 Jul 2012 18:51:48 +0100
Subject: [Tutor] confusion about imports
In-Reply-To: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
References: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
Message-ID: <jtf5gv$fke$1@dough.gmane.org>

On 09/07/2012 16:56, Chris Hare wrote:
>
> So, I have to admit, imports have me really confused.  I am trying to break apart a 10,000+ line single file into various files, one for each class, and one containing a whole bunch of functions which are used by a lot of classes.  Some of those functions use calls to methods in a Class.  Even though the Class has been imported, I get a nameError where trying to use the class.  I have read about Classes and packages and modules, but import just has me confused.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

Please don't break the file up for the sake of doing it, you're writing 
python not java :-)

-- 
Cheers.

Mark Lawrence.




From chare at labr.net  Mon Jul  9 20:33:56 2012
From: chare at labr.net (Chris Hare)
Date: Mon, 9 Jul 2012 13:33:56 -0500
Subject: [Tutor] confusion about imports
In-Reply-To: <CANLXbfCkeMvWX5vc+Ux+EJ=acaOQv7hOeMVYja5TRRAm0Wfeyw@mail.gmail.com>
References: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
	<CANLXbfCkeMvWX5vc+Ux+EJ=acaOQv7hOeMVYja5TRRAm0Wfeyw@mail.gmail.com>
Message-ID: <058534DF-854F-418E-AB17-5D2FC7D3F824@labr.net>


On Jul 9, 2012, at 12:42 PM, Walter Prins wrote:

> Hi Chris
> 
>> So, I have to admit, imports have me really confused.  I am trying to break apart a 10,000+ line single file into various files, one for each class, and one containing a whole bunch of functions which are used by a lot of classes.  Some of those functions use calls to methods in a Class.  Even though the Class has been imported, I get a nameError where trying to use the class.  I have read about Classes and packages and modules, but import just has me confused.
> 
> How did you import the class?  Or did you perhaps not import the Class
> itself, but rather the module containing the class?

I am doing import NAME where name is not only the name of the class, but also the name of the file containing the class.  The only reasons I am trying to break up the file is because it is getting to difficult to find stuff.  Some of the content hasn't change while other parts are still in flux a lot.  I figured that splitting it into the various files will make it easier to edit.

so I think I have figured out the problem based upon the resources you specified - the first one helped a bunch.

I was using 

import functions
import os
import db

when everything was all in one file, that worked just fine.  Now, with it all split up, once I changed

r = DbPath()

to

r = functions.DbPath()

things seems to work now.  I hope this is it!!!

Now, I have a bunch of smaller, more manageable files instead of trying to edit one ginormous one :-)

Thanks!

> 
> Read this article which explains a bit about Python namespaces:
> http://is.gd/e8PAZW  (but I note there's a bit of conflation of
> "class" and "class instance" going on at the end.)
> 
> Also read this page from the Python documentation, section "Python
> Scopes and Namespaces": http://docs.python.org/tutorial/classes.html
> 
> If a class is defined in a module (e.g. in the namespace of the
> module), and you "import module" the module into the your current
> namespace, then from within the current namespace you can access the
> class with "module.Class".  If however you import the class itself,
> e.g. "from module import Class", into your current namespace, then the
> Class itself is directly part of your local namespace and you must
> therefore access it unqualified as just "Class".

Thanks for the links -- yep - it helped, although I haven't solved my specific problem yet.
> 
> HTH,
> 
> Walter
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor


From david at graniteweb.com  Mon Jul  9 21:26:32 2012
From: david at graniteweb.com (David Rock)
Date: Mon, 9 Jul 2012 14:26:32 -0500
Subject: [Tutor] confusion about imports
In-Reply-To: <058534DF-854F-418E-AB17-5D2FC7D3F824@labr.net>
References: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
	<CANLXbfCkeMvWX5vc+Ux+EJ=acaOQv7hOeMVYja5TRRAm0Wfeyw@mail.gmail.com>
	<058534DF-854F-418E-AB17-5D2FC7D3F824@labr.net>
Message-ID: <20120709192632.GI3060@wdfs.gateway.2wire.net>

* Chris Hare <chare at labr.net> [2012-07-09 13:33]:
<snip>
> import functions
> import os
> import db
> 
> when everything was all in one file, that worked just fine.  Now, with it all split up, once I changed
> 
> r = DbPath()
> 
> to
> 
> r = functions.DbPath()
> 
> things seems to work now.  I hope this is it!!!
> 

Yes, that's it.  As mentioned elsewhere, you have to reference the
module name in order to let python know _which_ one you want.

For example, you could have the same DbPath() method in two different
modules: module1 and  module2, so you need to be able to differentiate.

module1.DbPath() is not the same thing as module2.DbPath()

Your functions.DbPath() is the way to go.

-- 
David Rock
david at graniteweb.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/tutor/attachments/20120709/89e4b647/attachment.pgp>

From bayespokerguy at gmail.com  Mon Jul  9 22:57:37 2012
From: bayespokerguy at gmail.com (Fred G)
Date: Mon, 9 Jul 2012 13:57:37 -0700
Subject: [Tutor] Mapping ID's for corresponding values in different
	Columns
In-Reply-To: <CAMg+7masO3b3vyf80gZXer5HAoT7HTeYYhbzXtQXGP0=1JA10A@mail.gmail.com>
References: <CAMg+7mZ5-mt9-kO4MPzF5G9xw7_sJ_=2TASesMzGUCpEiuMCPQ@mail.gmail.com>
	<CAJmBOfkYoYSdeO+uq+W8bGcqutwfPOc9QxAdgDGgsx_Bo071Gg@mail.gmail.com>
	<CAMg+7masO3b3vyf80gZXer5HAoT7HTeYYhbzXtQXGP0=1JA10A@mail.gmail.com>
Message-ID: <CAMg+7maYSKodPKahJXaDCmhDTNr-tOiNOZ=O7PzuVRCbKD14LQ@mail.gmail.com>

Got it! Thanks guys for all your help--what a satisfying feeling!  Just out
of curiosity, in the following:

def new_dict (csv_to_dict, nested_line):
    old_dict = csv_to_dict(file1)
    old_list = nested_line(file2)
    new_dict = {}
    #for item in old_list:
    #    new_dict[item] = item
    for k, v in old_dict.iteritems():
        for item in old_list:
            if k == item:
                new_dict[item] = v
    return (new_dict)

when I deleted the third line from the bottom (the "if k == item"), I got
the same value ("3") for each key, instead of the correct output (ie the
different ID's).  Does anyone know why this was the case?

Thanks so much.


On Mon, Jul 9, 2012 at 8:23 AM, Fred G <bayespokerguy at gmail.com> wrote:

> Thank you guys so much.  I'm quite close now, but I'm having a bit of
> trouble on the final for-loop to create the new dictionary.  I have the
> following 3 functions (note I'm re-typing it from a different computer so
> while the identation will be off here, it is correct in the actual code):
>
> #read in file as dictionary.  Name is key, id is value.
> def csv_to_dict(filename):
>   record = {}
>   line = filename.readline()
>   for line in filename:
>     key = line.split(",", 1)[-1]
>     val = line.split(",", 1)[:-1]
>   return record
>
> #read in file2 names
> def nested_line(filename):
>   line = filename.readline()
>   new_list = []
>   for line in filename:
>     name = line.split(",", 1)[-1]
>     new_list.append(name)
>   return new_list
>
> #this is the function that I'm having trouble with
> #create new dict mapping file 1 ids to file2's names
> def new_dict (csv_to_dict, nested_line):
>   old_dict = cssv_to_dict(file1)
>   old_list = nested_line(file2)
>   new_dict1 = {}
>   for item in old_list:
>     new_dict1[item] = item
>   return (new_dict1)
>
> I have tried various permutations of the for loop in this final function,
> but I haven't quite gotten it.  The current form is the closest I have
> gotten to my desired output, since it produces the new dictionary with the
> name that I want-- just an invalid id associated with that name.  I tried a
> bunch of different nested loops but kept getting it incorrect and then
> after so many attempts I got a little confused about what I was trying to
> do.  So I backed up a little bit and have this.
>
> Conceptually, I thought that this would give me my desired result:
> new_dict
> for name in old_list:
>   for key, value in old_dict:
>     if name == key:
>       new_dict1[key] = value
> return(new_dict1)
>
> But it wasn't right.  I tried a different approach where I used the
> dict.values() function in order to pull out the values from old_dict that
> we want to include in the new_dict, but I got a bit lost there, too.
>
> I'm so close right now and I would be so thankful for any bit of
> clarification which could get me to the finish line.
>
> On Mon, Jul 9, 2012 at 12:42 AM, Hugo Arts <hugo.yoshi at gmail.com> wrote:
>
>> On Sun, Jul 8, 2012 at 11:47 PM, Fred G <bayespokerguy at gmail.com> wrote:
>>
>>> Hi--
>>>
>>> My current input looks like the following:
>>>
>>> FILE1.csv
>>> PERSON_ID    PERSON_NAME
>>> 1                     Jen
>>> 2                     Mike
>>> 3                     Jim
>>> 4
>>> 5                     Jane
>>> 6                     Joe
>>> 7                     Jake
>>>
>>> FILE2.csv
>>> PERSON_ID   PERSON_NAME
>>>                      Jim
>>>                      Mike
>>>                      Jane
>>>                      Todd
>>>                      Jen
>>>
>>> _________
>>> I want to fill into the PERSON_ID column of FILE2.csv the corresponding
>>> ID's associated with those names as identified in FILE1.csv.
>>>
>>> At first I imported the csv module and was using the csv.Reader, but
>>> then it seemed simple enough just to write something like:
>>> for line in file2:
>>>      print(line)
>>>
>>> giving me the following output:
>>> PERSON_ID, PERSON_NAME
>>> , Jim
>>> , Mike
>>> , Jane
>>> , Todd
>>> , Jen
>>>
>>> I think I understand the issue at a conceptual level, but not quite sure
>>> how to fully implement it:
>>> a) I want to build a dictionary to create keys, such that each number in
>>> file1 corresponds to a unique string in column B of file1.
>>> b) then write a for loop like the following:
>>> for "person_name" in file2:
>>>    if "person_name.file2" == "person_name.file1":
>>>        person_id.file2 == person_id.file1
>>> c) write into file2 the changes to person_id's...
>>>
>>> But it's pretty difficult for me to get past this stage. Am I on the
>>> right track? And more importantly, how could I learn how to actually
>>> implement this in smaller stages?
>>>
>>>
>>  You're on the right track, and you're almost there! You've already
>> broken down the problem into steps. You should now try to implement a
>> function for each step, and finally you should glue these functions
>> together into a final program.
>>
>> a) Though you don't *have* to use it, csv.reader is really quite simple,
>> I'd recommend it. Try and write a function that takes a file name as
>> argument and returns a dictionary of the form { name: id, name: id } (i.e.
>> the names are the keys).
>>
>> b) For this step, you first need a list of all names in file 2. You could
>> use csv.reader again or you could just parse it. Then, you use the
>> dictionary to look up the corresponding id. The end goal for this function
>> is to return a list of lists that looks much like the file you want to end
>> up with:
>>
>> [[id, name],
>>  [id, name],
>>  [id, name]]
>>
>> c) this step should now be easy. I'd again, recommend csv.writer, it
>> makes the process pretty simple. You just pass in the nested list from step
>> (b) and you're pretty much done.
>>
>> For tips on the csv module, the list of examples is pretty helpful:
>> http://docs.python.org/py3k/library/csv.html#examples
>>  If you need help constructing the lists and dictionaries, my tips would
>> be 1) think one row at a time, 2) the for loop is your best friend, and 3)
>> nested lists usually means nested loops
>>
>> HTH,
>> Hugo
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120709/d27777a3/attachment-0001.html>

From bayespokerguy at gmail.com  Mon Jul  9 23:19:09 2012
From: bayespokerguy at gmail.com (Fred G)
Date: Mon, 9 Jul 2012 14:19:09 -0700
Subject: [Tutor] Mapping ID's for corresponding values in different
	Columns
In-Reply-To: <CAMg+7maYSKodPKahJXaDCmhDTNr-tOiNOZ=O7PzuVRCbKD14LQ@mail.gmail.com>
References: <CAMg+7mZ5-mt9-kO4MPzF5G9xw7_sJ_=2TASesMzGUCpEiuMCPQ@mail.gmail.com>
	<CAJmBOfkYoYSdeO+uq+W8bGcqutwfPOc9QxAdgDGgsx_Bo071Gg@mail.gmail.com>
	<CAMg+7masO3b3vyf80gZXer5HAoT7HTeYYhbzXtQXGP0=1JA10A@mail.gmail.com>
	<CAMg+7maYSKodPKahJXaDCmhDTNr-tOiNOZ=O7PzuVRCbKD14LQ@mail.gmail.com>
Message-ID: <CAMg+7mYBEyeh6gT7UOMVM1_toCCggAnynDx+9azkiemfAhB3sg@mail.gmail.com>

Sorry, just got that as well.  It was the placement of the if-statement in
the nested for-loop.  so v was stuck on 3...thanks again for the help with
this!!!

On Mon, Jul 9, 2012 at 1:57 PM, Fred G <bayespokerguy at gmail.com> wrote:

> Got it! Thanks guys for all your help--what a satisfying feeling!  Just
> out of curiosity, in the following:
>
> def new_dict (csv_to_dict, nested_line):
>     old_dict = csv_to_dict(file1)
>     old_list = nested_line(file2)
>     new_dict = {}
>     #for item in old_list:
>     #    new_dict[item] = item
>     for k, v in old_dict.iteritems():
>         for item in old_list:
>             if k == item:
>                 new_dict[item] = v
>     return (new_dict)
>
> when I deleted the third line from the bottom (the "if k == item"), I got
> the same value ("3") for each key, instead of the correct output (ie the
> different ID's).  Does anyone know why this was the case?
>
> Thanks so much.
>
>
> On Mon, Jul 9, 2012 at 8:23 AM, Fred G <bayespokerguy at gmail.com> wrote:
>
>> Thank you guys so much.  I'm quite close now, but I'm having a bit of
>> trouble on the final for-loop to create the new dictionary.  I have the
>> following 3 functions (note I'm re-typing it from a different computer so
>> while the identation will be off here, it is correct in the actual code):
>>
>> #read in file as dictionary.  Name is key, id is value.
>> def csv_to_dict(filename):
>>   record = {}
>>   line = filename.readline()
>>   for line in filename:
>>     key = line.split(",", 1)[-1]
>>     val = line.split(",", 1)[:-1]
>>   return record
>>
>> #read in file2 names
>> def nested_line(filename):
>>   line = filename.readline()
>>   new_list = []
>>   for line in filename:
>>     name = line.split(",", 1)[-1]
>>     new_list.append(name)
>>   return new_list
>>
>> #this is the function that I'm having trouble with
>> #create new dict mapping file 1 ids to file2's names
>> def new_dict (csv_to_dict, nested_line):
>>   old_dict = cssv_to_dict(file1)
>>   old_list = nested_line(file2)
>>   new_dict1 = {}
>>   for item in old_list:
>>     new_dict1[item] = item
>>   return (new_dict1)
>>
>> I have tried various permutations of the for loop in this final function,
>> but I haven't quite gotten it.  The current form is the closest I have
>> gotten to my desired output, since it produces the new dictionary with the
>> name that I want-- just an invalid id associated with that name.  I tried a
>> bunch of different nested loops but kept getting it incorrect and then
>> after so many attempts I got a little confused about what I was trying to
>> do.  So I backed up a little bit and have this.
>>
>> Conceptually, I thought that this would give me my desired result:
>> new_dict
>> for name in old_list:
>>   for key, value in old_dict:
>>     if name == key:
>>       new_dict1[key] = value
>> return(new_dict1)
>>
>> But it wasn't right.  I tried a different approach where I used the
>> dict.values() function in order to pull out the values from old_dict that
>> we want to include in the new_dict, but I got a bit lost there, too.
>>
>> I'm so close right now and I would be so thankful for any bit of
>> clarification which could get me to the finish line.
>>
>> On Mon, Jul 9, 2012 at 12:42 AM, Hugo Arts <hugo.yoshi at gmail.com> wrote:
>>
>>> On Sun, Jul 8, 2012 at 11:47 PM, Fred G <bayespokerguy at gmail.com> wrote:
>>>
>>>> Hi--
>>>>
>>>> My current input looks like the following:
>>>>
>>>> FILE1.csv
>>>> PERSON_ID    PERSON_NAME
>>>> 1                     Jen
>>>> 2                     Mike
>>>> 3                     Jim
>>>> 4
>>>> 5                     Jane
>>>> 6                     Joe
>>>> 7                     Jake
>>>>
>>>> FILE2.csv
>>>> PERSON_ID   PERSON_NAME
>>>>                      Jim
>>>>                      Mike
>>>>                      Jane
>>>>                      Todd
>>>>                      Jen
>>>>
>>>> _________
>>>> I want to fill into the PERSON_ID column of FILE2.csv the corresponding
>>>> ID's associated with those names as identified in FILE1.csv.
>>>>
>>>> At first I imported the csv module and was using the csv.Reader, but
>>>> then it seemed simple enough just to write something like:
>>>> for line in file2:
>>>>      print(line)
>>>>
>>>> giving me the following output:
>>>> PERSON_ID, PERSON_NAME
>>>> , Jim
>>>> , Mike
>>>> , Jane
>>>> , Todd
>>>> , Jen
>>>>
>>>> I think I understand the issue at a conceptual level, but not quite
>>>> sure how to fully implement it:
>>>> a) I want to build a dictionary to create keys, such that each number
>>>> in file1 corresponds to a unique string in column B of file1.
>>>> b) then write a for loop like the following:
>>>> for "person_name" in file2:
>>>>    if "person_name.file2" == "person_name.file1":
>>>>        person_id.file2 == person_id.file1
>>>> c) write into file2 the changes to person_id's...
>>>>
>>>> But it's pretty difficult for me to get past this stage. Am I on the
>>>> right track? And more importantly, how could I learn how to actually
>>>> implement this in smaller stages?
>>>>
>>>>
>>>  You're on the right track, and you're almost there! You've already
>>> broken down the problem into steps. You should now try to implement a
>>> function for each step, and finally you should glue these functions
>>> together into a final program.
>>>
>>> a) Though you don't *have* to use it, csv.reader is really quite simple,
>>> I'd recommend it. Try and write a function that takes a file name as
>>> argument and returns a dictionary of the form { name: id, name: id } (i.e.
>>> the names are the keys).
>>>
>>> b) For this step, you first need a list of all names in file 2. You
>>> could use csv.reader again or you could just parse it. Then, you use the
>>> dictionary to look up the corresponding id. The end goal for this function
>>> is to return a list of lists that looks much like the file you want to end
>>> up with:
>>>
>>> [[id, name],
>>>  [id, name],
>>>  [id, name]]
>>>
>>> c) this step should now be easy. I'd again, recommend csv.writer, it
>>> makes the process pretty simple. You just pass in the nested list from step
>>> (b) and you're pretty much done.
>>>
>>> For tips on the csv module, the list of examples is pretty helpful:
>>> http://docs.python.org/py3k/library/csv.html#examples
>>>  If you need help constructing the lists and dictionaries, my tips would
>>> be 1) think one row at a time, 2) the for loop is your best friend, and 3)
>>> nested lists usually means nested loops
>>>
>>> HTH,
>>> Hugo
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120709/d4a74057/attachment.html>

From abhishek.vit at gmail.com  Mon Jul  9 23:59:32 2012
From: abhishek.vit at gmail.com (Abhishek Pratap)
Date: Mon, 9 Jul 2012 14:59:32 -0700
Subject: [Tutor] updating step size while in loop
Message-ID: <CAJbA1KCFi-gi3j_30kqd6G3qSRQm2XkFK+pSaPYLU8vvSLLnTA@mail.gmail.com>

hey guys

I want to know whether it is possible for dynamically update the step
size in xrange  or someother slick way.

Here is what I am trying to do, if during a loop I find the x in list
I want to skip next #n iterations.


for x in xrange(start,stop,step):
    if x in list:
         step = 14
    else:
         step = 1



Thanks!
-Abhi

From hugo.yoshi at gmail.com  Tue Jul 10 00:06:49 2012
From: hugo.yoshi at gmail.com (Hugo Arts)
Date: Tue, 10 Jul 2012 00:06:49 +0200
Subject: [Tutor] updating step size while in loop
In-Reply-To: <CAJbA1KCFi-gi3j_30kqd6G3qSRQm2XkFK+pSaPYLU8vvSLLnTA@mail.gmail.com>
References: <CAJbA1KCFi-gi3j_30kqd6G3qSRQm2XkFK+pSaPYLU8vvSLLnTA@mail.gmail.com>
Message-ID: <CAJmBOf=WKXB1XtuYYe=HLNbA5_B9AwiNrAxLwQQa+v7Hsy4h-g@mail.gmail.com>

On Mon, Jul 9, 2012 at 11:59 PM, Abhishek Pratap <abhishek.vit at gmail.com>wrote:

> hey guys
>
> I want to know whether it is possible for dynamically update the step
> size in xrange  or someother slick way.
>
> Here is what I am trying to do, if during a loop I find the x in list
> I want to skip next #n iterations.
>
>
> for x in xrange(start,stop,step):
>     if x in list:
>          step = 14
>     else:
>          step = 1
>
>
>
> Thanks!
> -Abhi
>

It is not possible with a range object. You'll have to make a while loop
and keep track of the step yourself.

Hugo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120710/f4b62924/attachment.html>

From d at davea.name  Tue Jul 10 00:16:32 2012
From: d at davea.name (Dave Angel)
Date: Mon, 09 Jul 2012 18:16:32 -0400
Subject: [Tutor] confusion about imports
In-Reply-To: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
References: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
Message-ID: <4FFB5840.5030506@davea.name>

On 07/09/2012 11:56 AM, Chris Hare wrote:
> So, I have to admit, imports have me really confused.  I am trying to break apart a 10,000+ line single file into various files, one for each class, and one containing a whole bunch of functions which are used by a lot of classes.  Some of those functions use calls to methods in a Class.  Even though the Class has been imported, I get a nameError where trying to use the class.  I have read about Classes and packages and modules, but import just has me confused.
Something I haven't seen explicitly mentioned in this thread is that
when you make those modules to hold classes DO NOT make the module name
the same as the class name.

If you have a class MyClass defined in your 10k file, and you want to
move it to a separate file, and if you really wanted to dedicate the
file to a single class, then the file might be called myclass.py and the
references to it would like something like:

import myclass
...
obj = myclass.MyClass(arg1, arg2)

or alternatively,

from myclass import MyClass
...
obj = MyClass(arg1, arg2)

You wouldn't believe how much confusion people get into when they have
module names that look like class names.


Another thing is that you probably want several related classes and
functions in each module.  This is not Java.  At that point, you might want

from  people  import MyFirstPeopleClass, MySecondPeopleClass, peoplefunction


Finally, upon more careful reading of your original query, you may have
circular dependencies happening here.  A function may use class methods,
and class methods may use the function.  But if both are true, then put
them in the same module.  Having one module import a second one which
imports the first is an invitation to disaster.  And a special place in
debugging hell is reserved for those that try to import the script that
invokes it all.

-- 

DaveA


From abhishek.vit at gmail.com  Tue Jul 10 00:26:39 2012
From: abhishek.vit at gmail.com (Abhishek Pratap)
Date: Mon, 9 Jul 2012 15:26:39 -0700
Subject: [Tutor] updating step size while in loop
In-Reply-To: <CAJmBOf=WKXB1XtuYYe=HLNbA5_B9AwiNrAxLwQQa+v7Hsy4h-g@mail.gmail.com>
References: <CAJbA1KCFi-gi3j_30kqd6G3qSRQm2XkFK+pSaPYLU8vvSLLnTA@mail.gmail.com>
	<CAJmBOf=WKXB1XtuYYe=HLNbA5_B9AwiNrAxLwQQa+v7Hsy4h-g@mail.gmail.com>
Message-ID: <CAJbA1KCUwb8LWqaoa5m5J24oKWtLz-f1TXYdUDm0rBV=DcdObg@mail.gmail.com>

Ok thanks Hugo. I have the while loop working

-A

On Mon, Jul 9, 2012 at 3:06 PM, Hugo Arts <hugo.yoshi at gmail.com> wrote:
> On Mon, Jul 9, 2012 at 11:59 PM, Abhishek Pratap <abhishek.vit at gmail.com>
> wrote:
>>
>> hey guys
>>
>> I want to know whether it is possible for dynamically update the step
>> size in xrange  or someother slick way.
>>
>> Here is what I am trying to do, if during a loop I find the x in list
>> I want to skip next #n iterations.
>>
>>
>> for x in xrange(start,stop,step):
>>     if x in list:
>>          step = 14
>>     else:
>>          step = 1
>>
>>
>>
>> Thanks!
>> -Abhi
>
>
> It is not possible with a range object. You'll have to make a while loop and
> keep track of the step yourself.
>
> Hugo

From chare at labr.net  Tue Jul 10 03:18:31 2012
From: chare at labr.net (Chris Hare)
Date: Mon, 9 Jul 2012 20:18:31 -0500
Subject: [Tutor] confusion about imports
In-Reply-To: <4FFB5840.5030506@davea.name>
References: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
	<4FFB5840.5030506@davea.name>
Message-ID: <35DB6B40-EBC6-46F3-8749-4E16C38DEF1C@labr.net>

Good advice - thanks for that.  And I think you're right - I think what is happening is in fact a bunch of circular references.  As I resolve issues, I will be looking for those!  Appreciate all the advice!

On Jul 9, 2012, at 5:16 PM, Dave Angel wrote:

> On 07/09/2012 11:56 AM, Chris Hare wrote:
>> So, I have to admit, imports have me really confused.  I am trying to break apart a 10,000+ line single file into various files, one for each class, and one containing a whole bunch of functions which are used by a lot of classes.  Some of those functions use calls to methods in a Class.  Even though the Class has been imported, I get a nameError where trying to use the class.  I have read about Classes and packages and modules, but import just has me confused.
> Something I haven't seen explicitly mentioned in this thread is that
> when you make those modules to hold classes DO NOT make the module name
> the same as the class name.
> 
> If you have a class MyClass defined in your 10k file, and you want to
> move it to a separate file, and if you really wanted to dedicate the
> file to a single class, then the file might be called myclass.py and the
> references to it would like something like:
> 
> import myclass
> ...
> obj = myclass.MyClass(arg1, arg2)
> 
> or alternatively,
> 
> from myclass import MyClass
> ...
> obj = MyClass(arg1, arg2)
> 
> You wouldn't believe how much confusion people get into when they have
> module names that look like class names.
> 
> 
> Another thing is that you probably want several related classes and
> functions in each module.  This is not Java.  At that point, you might want
> 
> from  people  import MyFirstPeopleClass, MySecondPeopleClass, peoplefunction
> 
> 
> Finally, upon more careful reading of your original query, you may have
> circular dependencies happening here.  A function may use class methods,
> and class methods may use the function.  But if both are true, then put
> them in the same module.  Having one module import a second one which
> imports the first is an invitation to disaster.  And a special place in
> debugging hell is reserved for those that try to import the script that
> invokes it all.
> 
> -- 
> 
> DaveA
> 


From steve at pearwood.info  Tue Jul 10 04:04:03 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 10 Jul 2012 12:04:03 +1000
Subject: [Tutor] confusion about imports
In-Reply-To: <jtf5gv$fke$1@dough.gmane.org>
References: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
	<jtf5gv$fke$1@dough.gmane.org>
Message-ID: <4FFB8D93.3040902@pearwood.info>

Mark Lawrence wrote:
> On 09/07/2012 16:56, Chris Hare wrote:
>>
>> So, I have to admit, imports have me really confused.  I am trying to 
>> break apart a 10,000+ line single file into various files
> 
> Please don't break the file up for the sake of doing it, you're writing 
> python not java :-)

Agreed, but 10,000 lines is a pretty good reason for breaking up a file.

The largest single module in the Python standard library is decimal.py, and 
that's only 6250 lines. With 18 classes and 20 top-level functions, to my 
mind, that's about the limit of maintainability for a single file.


-- 
Steven


From wayne at waynewerner.com  Tue Jul 10 06:08:59 2012
From: wayne at waynewerner.com (Wayne Werner)
Date: Mon, 9 Jul 2012 23:08:59 -0500 (CDT)
Subject: [Tutor] updating step size while in loop
In-Reply-To: <CAJmBOf=WKXB1XtuYYe=HLNbA5_B9AwiNrAxLwQQa+v7Hsy4h-g@mail.gmail.com>
References: <CAJbA1KCFi-gi3j_30kqd6G3qSRQm2XkFK+pSaPYLU8vvSLLnTA@mail.gmail.com>
	<CAJmBOf=WKXB1XtuYYe=HLNbA5_B9AwiNrAxLwQQa+v7Hsy4h-g@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1207092253120.3314@gilgamesh>

While you can't do it with a straight generator, you can create your own:

class MyIter:
     def __init__(self, start, stop, step=1):
         self.start = start
         self.stop = stop
         self.step = step
     def __iter__(self):
         self.cur = self.start
         while self.cur < self.stop:
             yield self.cur
             self.cur += 1

And then you could do the following:
In [36]: i = MyIter(1, 10)

In [37]: for x in i:
    ....:     if x % 2:
    ....:         i.cur += 1
    ....:     print(x)
    ....:
1
3
5
7
9

-HTH,
Wayne

On Tue, 10 Jul 2012, Hugo Arts wrote:

> On Mon, Jul 9, 2012 at 11:59 PM, Abhishek Pratap <abhishek.vit at gmail.com>
> wrote:
>       hey guys
>
>       I want to know whether it is possible for dynamically update the
>       step
>       size in xrange ?or someother slick way.
>
>       Here is what I am trying to do, if during a loop I find the x in
>       list
>       I want to skip next #n iterations.
> 
>
>       for x in xrange(start,stop,step):
>       ? ? if x in list:
>       ? ? ? ? ?step = 14
>       ? ? else:
>       ? ? ? ? ?step = 1
> 
> 
>
>       Thanks!
>       -Abhi
> 
> 
> It is not possible with a range object. You'll have to make a while loop and
> keep track of the step yourself.
> 
> Hugo
> 
>

From alan.gauld at btinternet.com  Tue Jul 10 09:46:56 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 10 Jul 2012 08:46:56 +0100
Subject: [Tutor] updating step size while in loop
In-Reply-To: <CAJbA1KCFi-gi3j_30kqd6G3qSRQm2XkFK+pSaPYLU8vvSLLnTA@mail.gmail.com>
References: <CAJbA1KCFi-gi3j_30kqd6G3qSRQm2XkFK+pSaPYLU8vvSLLnTA@mail.gmail.com>
Message-ID: <jtgmlh$c5s$1@dough.gmane.org>

On 09/07/12 22:59, Abhishek Pratap wrote:
>
> I want to know whether it is possible for dynamically update the step
> size in xrange  or someother slick way.
>
> Here is what I am trying to do, if during a loop I find the x in list
> I want to skip next #n iterations.

You could use the next() method of the iterator. But you need to make 
the iterator explicit:

it = iter(xrange(start, stop,step))
for n in it:
     if n in mylist:
        for i in range(jump_size):
             it.next()


HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From breamoreboy at yahoo.co.uk  Tue Jul 10 15:35:57 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 10 Jul 2012 14:35:57 +0100
Subject: [Tutor] confusion about imports
In-Reply-To: <4FFB8D93.3040902@pearwood.info>
References: <EF4A04B6-8D32-40FA-8A8F-C47C6CFA48C2@labr.net>
	<jtf5gv$fke$1@dough.gmane.org> <4FFB8D93.3040902@pearwood.info>
Message-ID: <jthb3a$era$1@dough.gmane.org>

On 10/07/2012 03:04, Steven D'Aprano wrote:
> Mark Lawrence wrote:
>> On 09/07/2012 16:56, Chris Hare wrote:
>>>
>>> So, I have to admit, imports have me really confused.  I am trying to
>>> break apart a 10,000+ line single file into various files
>>
>> Please don't break the file up for the sake of doing it, you're
>> writing python not java :-)
>
> Agreed, but 10,000 lines is a pretty good reason for breaking up a file.

Fair comment I should have said this originally.

>
> The largest single module in the Python standard library is decimal.py,
> and that's only 6250 lines. With 18 classes and 20 top-level functions,
> to my mind, that's about the limit of maintainability for a single file.
>
>

Thanks for answering the question that I was about to ask :) Agreed 
about the limit.

-- 
Cheers.

Mark Lawrence.




From bell.james03 at gmail.com  Tue Jul 10 16:58:01 2012
From: bell.james03 at gmail.com (James Bell)
Date: Tue, 10 Jul 2012 10:58:01 -0400
Subject: [Tutor] Examples of "With....As Statement" in Python
Message-ID: <CAAPsuXUcSL+-dpV55Df_ZmbhXnverDjqnTbgbYPbg6jAxrR_dQ@mail.gmail.com>

I'm attempting to learn how to use the "with....as" statement in python.

I've read the documentation and also a few tutorials but I still cannot
understand the concept. or how this is normally used. Can someone please
write an example or 2 of simple ways to use the "with statement".

I understand in java the try...catch...finally block so maybe someone can
point out the relationship between them.

thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120710/9975cf09/attachment.html>

From chare at labr.net  Tue Jul 10 16:56:11 2012
From: chare at labr.net (Chris Hare)
Date: Tue, 10 Jul 2012 09:56:11 -0500
Subject: [Tutor] str object is not callable
Message-ID: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>


This piece of code works:

Big-Mac:Classes chare$ more tmp.py
import re

def special_match(string, search=re.compile(r'[^a-zA-Z0-9\.\ \-\#\$\*\@\!\%\^\&]').search):
        #string = string.rstrip()
        return not bool(search(string))

print special_match("admin")
print special_match("&!*")
print special_match("=")

Big-Mac:Classes chare$ python tmp.py
True
True
False
Big-Mac:Classes chare$ 

However, when I use the EXACT same code in the context of the larger code, I get the error

    return not bool(search(strg))
TypeError: 'str' object is not callable

The input to the function in the larger program is the same as the first test in the small script that works -- "admin".  

As a side note -- the rstrip call is also broken, although the string module is imported.  I just can't figure out why this code works in one context and not in another.



From breamoreboy at yahoo.co.uk  Tue Jul 10 17:25:09 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 10 Jul 2012 16:25:09 +0100
Subject: [Tutor] Examples of "With....As Statement" in Python
In-Reply-To: <CAAPsuXUcSL+-dpV55Df_ZmbhXnverDjqnTbgbYPbg6jAxrR_dQ@mail.gmail.com>
References: <CAAPsuXUcSL+-dpV55Df_ZmbhXnverDjqnTbgbYPbg6jAxrR_dQ@mail.gmail.com>
Message-ID: <jthhg8$7mq$2@dough.gmane.org>

On 10/07/2012 15:58, James Bell wrote:
> I'm attempting to learn how to use the "with....as" statement in python.
>
> I've read the documentation and also a few tutorials but I still cannot
> understand the concept. or how this is normally used. Can someone please
> write an example or 2 of simple ways to use the "with statement".
>
> I understand in java the try...catch...finally block so maybe someone can
> point out the relationship between them.
>
> thanks.
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

You're not the only one, see this 
http://effbot.org/zone/python-with-statement.htm

-- 
Cheers.

Mark Lawrence.




From jeanpierreda at gmail.com  Tue Jul 10 17:33:41 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 10 Jul 2012 11:33:41 -0400
Subject: [Tutor] str object is not callable
In-Reply-To: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
References: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
Message-ID: <CABicbJLp_i3ozwwsD2NjDr25nn=VawLs0fvoFDdjqEqyDoMFOg@mail.gmail.com>

On Tue, Jul 10, 2012 at 10:56 AM, Chris Hare <chare at labr.net> wrote:
> The input to the function in the larger program is the same as the first test in the small script that works -- "admin".
>
> As a side note -- the rstrip call is also broken, although the string module is imported.  I just can't figure out why this code works in one context and not in another.

I suspect you defined "bool" somewhere to be a string. That, or else
you passed in a string as the search argument. Unfortunately Python
doesn't tell you which expression raised the exception, but certainly
it's one of those two.

-- Devin

From mariocatch at gmail.com  Tue Jul 10 17:44:36 2012
From: mariocatch at gmail.com (mariocatch)
Date: Tue, 10 Jul 2012 11:44:36 -0400
Subject: [Tutor] Examples of "With....As Statement" in Python
In-Reply-To: <jthhg8$7mq$2@dough.gmane.org>
References: <CAAPsuXUcSL+-dpV55Df_ZmbhXnverDjqnTbgbYPbg6jAxrR_dQ@mail.gmail.com>
	<jthhg8$7mq$2@dough.gmane.org>
Message-ID: <CAA7L5UDtH5+VGM4LQYPEaHiusxYP7UwXiziMJKNpsBKN+fqJ0g@mail.gmail.com>

On Tue, Jul 10, 2012 at 11:25 AM, Mark Lawrence <breamoreboy at yahoo.co.uk>wrote:

> On 10/07/2012 15:58, James Bell wrote:
>
>> I'm attempting to learn how to use the "with....as" statement in python.
>>
>> I've read the documentation and also a few tutorials but I still cannot
>> understand the concept. or how this is normally used. Can someone please
>> write an example or 2 of simple ways to use the "with statement".
>>
>> I understand in java the try...catch...finally block so maybe someone can
>> point out the relationship between them.
>>
>> thanks.
>> ______________________________**_________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>>
>>
> You're not the only one, see this http://effbot.org/zone/python-**
> with-statement.htm <http://effbot.org/zone/python-with-statement.htm>
>
> --
> Cheers.
>
> Mark Lawrence.
>
>
>
> ______________________________**_________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>

The 'with' statement (
http://docs.python.org/tutorial/inputoutput.html#methods-of-file-objects).
'with' is not like a try/catch/finally. It is more like a using in .NET.
It's simply a way for you to create an instance of a disposable object, and
have it automatically cleaned up for you at the end of the 'with' statement.
Consider the following file usage:
f = open('test.txt', 'r')
f.read()
f.close()
Imagine if I forgot to close the file handler 'f'? Instead of worrying
about that, you can let python handle it for you:
with open('test.txt, 'r') as f:
f.read()
Now, when the 'with' statement ends, it will automatically dispose the file
object 'f' and close the stream for you.
-Mario
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120710/cf7b5c13/attachment-0001.html>

From d at davea.name  Tue Jul 10 17:45:19 2012
From: d at davea.name (Dave Angel)
Date: Tue, 10 Jul 2012 11:45:19 -0400
Subject: [Tutor] str object is not callable
In-Reply-To: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
References: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
Message-ID: <4FFC4E0F.7030805@davea.name>

On 07/10/2012 10:56 AM, Chris Hare wrote:
> This piece of code works:
>
> Big-Mac:Classes chare$ more tmp.py
> import re
>
> def special_match(string, search=re.compile(r'[^a-zA-Z0-9\.\ \-\#\$\*\@\!\%\^\&]').search):
>         #string = string.rstrip()
>         return not bool(search(string))
You just redefined string from the name of a module to a local variable
(function parameter).  Pick a safer name.


> print special_match("admin")
> print special_match("&!*")
> print special_match("=")
>
> Big-Mac:Classes chare$ python tmp.py
> True
> True
> False
> Big-Mac:Classes chare$ 
>
> However, when I use the EXACT same code in the context of the larger code, I get the error
>
>     return not bool(search(strg))
> TypeError: 'str' object is not callable
Since there are two places in that line that look like function calls,
it's hard to guess which one is nor working.  Just put prints
immediately in front of the error line, and see what kind of objects
bool and search are.

Chances are you masked the name bool in your top-level code.

> The input to the function in the larger program is the same as the first test in the small script that works -- "admin".  
>
> As a side note -- the rstrip call is also broken, although the string module is imported.  I just can't figure out why this code works in one context and not in another.
Importing is irrelevant when you hide the name inside your function. 
But since you don't specify what you mean by "broken," who knows if
that's relevant.  The preferred version of rstrip() is  a method of str
object.


>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>



-- 

DaveA


From steve at pearwood.info  Tue Jul 10 18:05:32 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 11 Jul 2012 02:05:32 +1000
Subject: [Tutor] Examples of "With....As Statement" in Python
In-Reply-To: <CAAPsuXUcSL+-dpV55Df_ZmbhXnverDjqnTbgbYPbg6jAxrR_dQ@mail.gmail.com>
References: <CAAPsuXUcSL+-dpV55Df_ZmbhXnverDjqnTbgbYPbg6jAxrR_dQ@mail.gmail.com>
Message-ID: <4FFC52CC.7040808@pearwood.info>

James Bell wrote:
> I'm attempting to learn how to use the "with....as" statement in python.
> 
> I've read the documentation and also a few tutorials but I still cannot
> understand the concept. or how this is normally used. Can someone please
> write an example or 2 of simple ways to use the "with statement".
> 
> I understand in java the try...catch...finally block so maybe someone can
> point out the relationship between them.

Java's try...catch...finally is called try...except...finally in Python.

In simple terms, a with-block uses a "context manager" to automatically wrap 
code in a try...finally block so as to ensure that cleanup code runs 
automatically even in the event of an error.

So that's the basic principle. Instead of having to write this *every time* 
you need to process some stuff:

     initialisation code
     try:
         process stuff
     finally:
         clean up code

you can write the initialisation and cleanup code *once* in a context manager, 
then do:

     with context manager:
         process stuff

and have both the initialisation and cleanup code automatically handled by the 
context manager.

So what's a context manager? I'll tell you what they *aren't* -- they are NOT 
a special type or class. Any class can be a context manager if it obeys the 
right protocol:

* the class must have a special method called __enter__

* the class must have a special method called __exit__

Of course, both __enter__ and __exit__ have to get the details right: they 
have to take the right number of arguments, and do the right thing.

Other than than, any class or type can be context managers, and Python comes 
with a few built in. The most important are file objects: any file object is 
also a context manager, so you can say:

with open("myfile.txt") as f:
     process(f)

and the file object will be automatically closed when the with block is done, 
even if an error occurs.

Some other examples (from memory) include decimal Context objects, unittest 
test features, and probably quite a few others.

How do you use a context manager? Easy -- you just call it using "with".

py> with open("rubbish.txt", "w") as fp:
...     fp.write("starting to write...")
...     fp.write(99)  # oops!
...
20
Traceback (most recent call last):
   File "<stdin>", line 3, in <module>
TypeError: must be str, not int
py>
py> open("rubbish.txt").read()
'starting to write...'


Opening the file a second time (this time *without* the with statement) shows 
that despite the error, the text was saved to disk and the file closed without 
any additional effort.


So how do you write your own context manager?

Naturally you can just define a class and give it the appropriate __enter__ 
and __exit__ methods. But there's an easy way. Python has a library to help 
you write context manager objects. Here's a toy example:

import contextlib
@contextlib.contextmanager
def my_manager(obj):
     print("called manager with argument %s" % obj)
     print("setting up manager...")
     try:
         value = ['anything', 'you', 'like', str(obj)]
         yield value
     finally:
         print("cleanup code running...")


Now I can use my_manager in a with-statement:


py> with my_manager(42) as data:
...     print("processing data")
...     data.sort()
...     print(data)
...
called manager with argument 42
setting up manager...
processing data
['42', 'anything', 'like', 'you']
cleanup code running...



-- 
Steven

From ramit.prasad at jpmorgan.com  Tue Jul 10 17:48:38 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Tue, 10 Jul 2012 15:48:38 +0000
Subject: [Tutor] str object is not callable
In-Reply-To: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
References: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741654C745@SCACMX008.exchad.jpmchase.net>

> This piece of code works:
> 
> Big-Mac:Classes chare$ more tmp.py
> import re
> 
> def special_match(string, search=re.compile(r'[^a-zA-Z0-9\.\ \-
> \#\$\*\@\!\%\^\&]').search):
>         #string = string.rstrip()
>         return not bool(search(string))
> 
> print special_match("admin")
> print special_match("&!*")
> print special_match("=")
> 
> Big-Mac:Classes chare$ python tmp.py
> True
> True
> False
> Big-Mac:Classes chare$
> 
> However, when I use the EXACT same code in the context of the larger code, I
> get the error
> 
>     return not bool(search(strg))
> TypeError: 'str' object is not callable
> 
> The input to the function in the larger program is the same as the first test
> in the small script that works -- "admin".
> 
> As a side note -- the rstrip call is also broken, although the string module
> is imported.  I just can't figure out why this code works in one context and
> not in another.

Usually it is best to provide the full and exact traceback and Python 
version. I suspect somewhere in your module you have a variable bool
that is conflicting with the built-in version of bool. Or you could
have passed in a different search that is a string. You can tell
which is the problem by breaking this up into two lines.

ret = search(string)
return not bool( ret )

Alternatively, you do not really need the bool, although you should 
always avoid shadowing built-ins (using any variable with the same
name as a built-in, unless you are a more advanced Python user).
You could just do the following.

return not search(string)

Oh and shadowing "string" is not a good idea if you are using the 
string module.

Why do you think rstrip is broken?

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From brian.van.den.broek at gmail.com  Tue Jul 10 18:19:10 2012
From: brian.van.den.broek at gmail.com (Brian van den Broek)
Date: Tue, 10 Jul 2012 12:19:10 -0400
Subject: [Tutor] str object is not callable
In-Reply-To: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
References: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
Message-ID: <CAF6Daj+8iiMm7HJOp-=1i4MBrGNOX_23n-EfHJMAg2A7ZfCxAg@mail.gmail.com>

On 10 Jul 2012 11:31, "Chris Hare" <chare at labr.net> wrote:
>
>
> This piece of code works:
>
> Big-Mac:Classes chare$ more tmp.py
> import re
>

<snip>

>         return not bool(search(string))

<snip>

> However, when I use the EXACT same code in the context of the larger
code, I get the error
>
>     return not bool(search(strg))

In addition to the comments about likely shaddowing, contrary to what you
say, those two lines are not the same. Either you retyped instead of copy &
pasting (don't do that), or you are not running the code you think you are.

Best,

Brian vdB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120710/afe8baaa/attachment.html>

From steve at pearwood.info  Tue Jul 10 18:23:24 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 11 Jul 2012 02:23:24 +1000
Subject: [Tutor] str object is not callable
In-Reply-To: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
References: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
Message-ID: <4FFC56FC.6020406@pearwood.info>

Chris Hare wrote:

> def special_match(string, search=re.compile(r'[^a-zA-Z0-9\.\ \-\#\$\*\@\!\%\^\&]').search):
>         #string = string.rstrip()
>         return not bool(search(string))

The call to bool is redundant. Get rid of it. The not operator will
automatically convert its argument into a bool:

py> print('s', bool('s'), not 's')
s True False
py> print(None, bool(None), not None)
None False True


That eliminates one possible source of errors, and will be a tiny bit faster.


Also, a stylistic comment: your function as defined above is painful. Do 
yourself, and your readers, a favour by defining a named constant like this:


SPECIAL_CHARS = re.compile(r'[^a-zA-Z0-9\.\ \-\#\$\*\@\!\%\^\&]')

def special_match(string, search=SPECIAL_CHARS.search):
     return not search(string)



> However, when I use the EXACT same code in the context of the larger code,
> I get the error
> 
> return not bool(search(strg)) TypeError: 'str' object is not callable

As numerous people have already pointed out, you have shadowed either the bool
or the search functions with a variable with the same name containing a
string:


py> len("hello world")
11
py> len = 11  # oops, shadowing the builtin function
py> len("hello world")
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable



> As a side note -- the rstrip call is also broken, although the string
> module is imported.  I just can't figure out why this code works in one
> context and not in another.

For the same reason -- you define a local variable "string", which shadows the 
global "string" module.



-- 
Steven


From chare at labr.net  Tue Jul 10 19:35:18 2012
From: chare at labr.net (Chris Hare)
Date: Tue, 10 Jul 2012 12:35:18 -0500
Subject: [Tutor] str object is not callable
In-Reply-To: <CABicbJLp_i3ozwwsD2NjDr25nn=VawLs0fvoFDdjqEqyDoMFOg@mail.gmail.com>
References: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
	<CABicbJLp_i3ozwwsD2NjDr25nn=VawLs0fvoFDdjqEqyDoMFOg@mail.gmail.com>
Message-ID: <2BD0F69E-CC7B-4A7B-9F59-766F744B398C@labr.net>


Okay - I am officially embarrassed. 

As you might now, I am splitting this 10,000 line file apart, and that is posing certain challenges which I am fixing, and cleaning up stuff that was broken and visible only when doing this split.  

This is one of them.  

What I failed to remember -- and you guys are free to bash me for this  -- was so simple once I put a print in to see what the value was of the string entering the function.

I had (after modifications suggested):

def special_match(s, hunt=SPECIAL_CHARS.search):

BUT

This is defined in a Class - and I missed it.  Should have been

def special_match(self,s, hunt=SPECIAL_CHARS.search):

Sorry guys - but thanks for the excellent suggestions/advice.  I am sure not a python guru, python is my entry into OOP.  So, rest assured I come to the list after trying to figure it out, but lesson learned.  That is why it worked in one piece of code - it was standalone, and didn't work in the class where it has been moved to.  No excuse, but there it is nonetheless.

<sheepishly>
Chris



On Jul 10, 2012, at 10:33 AM, Devin Jeanpierre wrote:

> On Tue, Jul 10, 2012 at 10:56 AM, Chris Hare <chare at labr.net> wrote:
>> The input to the function in the larger program is the same as the first test in the small script that works -- "admin".
>> 
>> As a side note -- the rstrip call is also broken, although the string module is imported.  I just can't figure out why this code works in one context and not in another.
> 
> I suspect you defined "bool" somewhere to be a string. That, or else
> you passed in a string as the search argument. Unfortunately Python
> doesn't tell you which expression raised the exception, but certainly
> it's one of those two.
> 
> -- Devin


From steve at pearwood.info  Tue Jul 10 19:48:07 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 11 Jul 2012 03:48:07 +1000
Subject: [Tutor] str object is not callable
In-Reply-To: <2BD0F69E-CC7B-4A7B-9F59-766F744B398C@labr.net>
References: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>	<CABicbJLp_i3ozwwsD2NjDr25nn=VawLs0fvoFDdjqEqyDoMFOg@mail.gmail.com>
	<2BD0F69E-CC7B-4A7B-9F59-766F744B398C@labr.net>
Message-ID: <4FFC6AD7.8070304@pearwood.info>

Chris Hare wrote:
> Okay - I am officially embarrassed. 
[...]


Meh, don't beat yourself up too badly. We've all been where you are now. 
Sometimes I look back at my early Python code... I tell you, that's always a 
good antidote for a big head.



-- 
Steven


From redacted@example.com  Tue Jul 10 21:26:42 2012
From: redacted@example.com (Alexander Q.)
Date: Tue, 10 Jul 2012 12:26:42 -0700
Subject: [Tutor] Regular expressions: findall vs search
Message-ID: <CAHgjEe2thMsvNYo8qtFWWS5GA+o=xBFGUjOvavTuWMtUw6wirQ@mail.gmail.com>

I'm a bit confused about extracting data using re.search or re.findall.

Say I have the following code: tuples =
re.findall(r'blahblah(\d+)yattayattayatta(\w+)moreblahblahblah(\w+)over',
text)

So I'm looking for that string in 'text', and I intend to extract the parts
which have parentheses around them. And it works: the variable "tuples",
which I assigned to get the return of re.findall, returns a tuple list,
each 'element' therein being a tuple of 3 elements (which is what I wanted
since I had 3 sets of parentheses).

My question is how does Python know to return just the part in the
parentheses and not to return the "blahblah" and the "yattayattayatta",
etc...? The 're.search' function returns the whole thing, and if I want
just the parentheses parts, I do tuples.group(1) or tuples.group(2) or
tuples.group(3), depending on which set of parentheses I want. Does the
re.findall command by default ignore anything outside of the parentheses
and only return the parentheses as a grouping withing one tuple (i.e., the
first element in "tuples" would be, as it is, a list comprised of 3
elements corresponding respectively to the 1st, 2nd, and 3rd parentheses)?
Thank you for reading.

-Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120710/6e0add0a/attachment.html>

From hugo.yoshi at gmail.com  Tue Jul 10 21:45:32 2012
From: hugo.yoshi at gmail.com (Hugo Arts)
Date: Tue, 10 Jul 2012 21:45:32 +0200
Subject: [Tutor] Regular expressions: findall vs search
In-Reply-To: <CAHgjEe2thMsvNYo8qtFWWS5GA+o=xBFGUjOvavTuWMtUw6wirQ@mail.gmail.com>
References: <CAHgjEe2thMsvNYo8qtFWWS5GA+o=xBFGUjOvavTuWMtUw6wirQ@mail.gmail.com>
Message-ID: <CAJmBOfnn3+kxOw7sH5vrfZmcQix8di7k4JrxYs+GoXM3ZS989A@mail.gmail.com>

On Tue, Jul 10, 2012 at 9:26 PM, Alexander Q. <redacted@example.com> wrote:

> I'm a bit confused about extracting data using re.search or re.findall.
>
> Say I have the following code: tuples =
> re.findall(r'blahblah(\d+)yattayattayatta(\w+)moreblahblahblah(\w+)over',
> text)
>
> So I'm looking for that string in 'text', and I intend to extract the
> parts which have parentheses around them. And it works: the variable
> "tuples", which I assigned to get the return of re.findall, returns a tuple
> list, each 'element' therein being a tuple of 3 elements (which is what I
> wanted since I had 3 sets of parentheses).
>
> My question is how does Python know to return just the part in the
> parentheses and not to return the "blahblah" and the "yattayattayatta",
> etc...? The 're.search' function returns the whole thing, and if I want
> just the parentheses parts, I do tuples.group(1) or tuples.group(2) or
> tuples.group(3), depending on which set of parentheses I want. Does the
> re.findall command by default ignore anything outside of the parentheses
> and only return the parentheses as a grouping withing one tuple (i.e., the
> first element in "tuples" would be, as it is, a list comprised of 3
> elements corresponding respectively to the 1st, 2nd, and 3rd parentheses)?
> Thank you for reading.
>
> -Alex
>

from the documentation for findall:

The *string* is scanned left-to-right, and matches are returned in the
order found. If one or more groups are present in the pattern, return a
list of groups; this will be a list of tuples if the pattern has more than
one group.

That should clear everything up. As for *why* it behaves this way, I have
no idea. It may be legacy behavior.

HTH,
Hugo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120710/6efa03e8/attachment.html>

From chare at labr.net  Tue Jul 10 21:11:26 2012
From: chare at labr.net (Chris Hare)
Date: Tue, 10 Jul 2012 14:11:26 -0500
Subject: [Tutor] advice on global variables
Message-ID: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>


I know they are bad.  That is why I would prefer not to use it, but I am not sure how else to handle this problem.

In this app, the user must log in.  Once authenticated, they have a userid stored in the SQLite database.  Before splitting my app into multiple files, I used a global variable.  I know its bad, but it worked.  Now that things are split apart, the classes which used it previously now don't see it anymore, even using the global keyword.  I think this is the expected behavior.  See here

file: a.py

import b
global_var = "global"

def func1():
	global global_var
	print "global var in func1 = %s" % global_var

class intclass:
	def func2(self):
		global global_var
		print "global var in intclass = %s" % global_var

print "global_var = %s" % global_var
func1()
f = intclass()
f.func2()
g = b.extclass()
g.func3()

file: b.py

class extclass:
	def func3(self):
		global global_var
		print "global var in extclass = %s" % global_var

When I run it, I get what I think the expected behavior, that the external class ext class won't be able to see the global_var

Big-Mac:t chare$ python a.py
global_var = global
global var in func1 = global
global var in intclass = global
Traceback (most recent call last):
  File "a.py", line 18, in <module>
    g.func3()
  File "/Users/chare/Development/python/animaltrax/pkg/t/b.py", line 5, in func3
    print "global var in extclass = %s" % global_var
NameError: global name 'global_var' is not defined

So - my question is this:  how do I solve the problem of multiple classes needing to get access to a value which needs to be preserved across the lifetime of the running application?

One thought was a RAM based SQLite database, but that seems like a lot of work.  I dunno, maybe that is the option.

suggestions, ideas, criticisms are all welcome.  Python code aside, I just don't know how to approach this problem in Python.

Thanks, as always for the feedback and guidance.


From alexandre.zani at gmail.com  Tue Jul 10 22:18:51 2012
From: alexandre.zani at gmail.com (Alexandre Zani)
Date: Tue, 10 Jul 2012 13:18:51 -0700
Subject: [Tutor] advice on global variables
In-Reply-To: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
Message-ID: <CAJVMporMG165X=wC+=U0Oj9XpasJuhWBEwr_kSzOpGoiTeV-FQ@mail.gmail.com>

On Tue, Jul 10, 2012 at 12:11 PM, Chris Hare <chare at labr.net> wrote:
>
> I know they are bad.  That is why I would prefer not to use it, but I am not sure how else to handle this problem.
>
> In this app, the user must log in.  Once authenticated, they have a userid stored in the SQLite database.  Before splitting my app into multiple files, I used a global variable.  I know its bad, but it worked.  Now that things are split apart, the classes which used it previously now don't see it anymore, even using the global keyword.  I think this is the expected behavior.  See here
>
> file: a.py
>
> import b
> global_var = "global"
>
> def func1():
>         global global_var
>         print "global var in func1 = %s" % global_var
>
> class intclass:
>         def func2(self):
>                 global global_var
>                 print "global var in intclass = %s" % global_var
>
> print "global_var = %s" % global_var
> func1()
> f = intclass()
> f.func2()
> g = b.extclass()
> g.func3()
>
> file: b.py
>
> class extclass:
>         def func3(self):
>                 global global_var
>                 print "global var in extclass = %s" % global_var
>
> When I run it, I get what I think the expected behavior, that the external class ext class won't be able to see the global_var
>
> Big-Mac:t chare$ python a.py
> global_var = global
> global var in func1 = global
> global var in intclass = global
> Traceback (most recent call last):
>   File "a.py", line 18, in <module>
>     g.func3()
>   File "/Users/chare/Development/python/animaltrax/pkg/t/b.py", line 5, in func3
>     print "global var in extclass = %s" % global_var
> NameError: global name 'global_var' is not defined
>
> So - my question is this:  how do I solve the problem of multiple classes needing to get access to a value which needs to be preserved across the lifetime of the running application?
>
> One thought was a RAM based SQLite database, but that seems like a lot of work.  I dunno, maybe that is the option.
>
> suggestions, ideas, criticisms are all welcome.  Python code aside, I just don't know how to approach this problem in Python.
>
> Thanks, as always for the feedback and guidance.
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

Can you tell us a little more about what this app does? As a general
rule, if you're using a global variable it's a strong sign that you're
doing something wrong. Can we see real code?

To answer your most basic question:

import a
a.global_var

But again, that's probably not the right way to solve your problem.

From eire1130 at gmail.com  Tue Jul 10 22:25:27 2012
From: eire1130 at gmail.com (James Reynolds)
Date: Tue, 10 Jul 2012 16:25:27 -0400
Subject: [Tutor] advice on global variables
In-Reply-To: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
Message-ID: <CAE0jAbqbgOXA20c=J1qNCrm4CiLeuhN5kfp37rvuiFWHykwxYA@mail.gmail.com>

On Tue, Jul 10, 2012 at 3:11 PM, Chris Hare <chare at labr.net> wrote:

>
> I know they are bad.  That is why I would prefer not to use it, but I am
> not sure how else to handle this problem.
>
> In this app, the user must log in.  Once authenticated, they have a userid
> stored in the SQLite database.  Before splitting my app into multiple
> files, I used a global variable.  I know its bad, but it worked.  Now that
> things are split apart, the classes which used it previously now don't see
> it anymore, even using the global keyword.  I think this is the expected
> behavior.  See here
>
> file: a.py
>
> import b
> global_var = "global"
>
> def func1():
>         global global_var
>         print "global var in func1 = %s" % global_var
>
> class intclass:
>         def func2(self):
>                 global global_var
>                 print "global var in intclass = %s" % global_var
>
> print "global_var = %s" % global_var
> func1()
> f = intclass()
> f.func2()
> g = b.extclass()
> g.func3()
>
> file: b.py
>
> class extclass:
>         def func3(self):
>                 global global_var
>                 print "global var in extclass = %s" % global_var
>
> When I run it, I get what I think the expected behavior, that the external
> class ext class won't be able to see the global_var
>
> Big-Mac:t chare$ python a.py
> global_var = global
> global var in func1 = global
> global var in intclass = global
> Traceback (most recent call last):
>   File "a.py", line 18, in <module>
>     g.func3()
>   File "/Users/chare/Development/python/animaltrax/pkg/t/b.py", line 5, in
> func3
>     print "global var in extclass = %s" % global_var
> NameError: global name 'global_var' is not defined
>
> So - my question is this:  how do I solve the problem of multiple classes
> needing to get access to a value which needs to be preserved across the
> lifetime of the running application?
>
> One thought was a RAM based SQLite database, but that seems like a lot of
> work.  I dunno, maybe that is the option.
>
> suggestions, ideas, criticisms are all welcome.  Python code aside, I just
> don't know how to approach this problem in Python.
>
> Thanks, as always for the feedback and guidance.
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>


You should avoid using the global statement.

In your case, I would think you could just add an argument to the method:

class MyObj(object):
    def __init__(self, arg):
        self.arg = arg
    def my_func(self, new_arg):
        self.arg = new_arg

to call it:

arg = 1

m = MyObj(arg)
print m.arg
new_arg = 2
m.my_func(new_arg)
print m.arg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120710/64e30221/attachment.html>

From ramit.prasad at jpmorgan.com  Tue Jul 10 22:26:59 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Tue, 10 Jul 2012 20:26:59 +0000
Subject: [Tutor] advice on global variables
In-Reply-To: <CAJVMporMG165X=wC+=U0Oj9XpasJuhWBEwr_kSzOpGoiTeV-FQ@mail.gmail.com>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
	<CAJVMporMG165X=wC+=U0Oj9XpasJuhWBEwr_kSzOpGoiTeV-FQ@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741654CC20@SCACMX008.exchad.jpmchase.net>

> >
> > file: a.py
> >
> > import b
> > global_var = "global"
> >
> To answer your most basic question:

[file b.py]
> import a
> a.global_var

This would be a cyclical import and is bad even if it does not fail.
Remove the common dependencies (in this case the global variables) 
and place it in a third module (c.py) that both a.py and b.py can 
import.

> But again, that's probably not the right way to solve your problem.


Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From ramit.prasad at jpmorgan.com  Tue Jul 10 22:32:41 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Tue, 10 Jul 2012 20:32:41 +0000
Subject: [Tutor] advice on global variables
In-Reply-To: <CAE0jAbqbgOXA20c=J1qNCrm4CiLeuhN5kfp37rvuiFWHykwxYA@mail.gmail.com>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
	<CAE0jAbqbgOXA20c=J1qNCrm4CiLeuhN5kfp37rvuiFWHykwxYA@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741654CC3E@SCACMX008.exchad.jpmchase.net>

> You should avoid using the global statement.
> 
> In your case, I would think you could just add an argument to the method:
> 
> class MyObj(object):
>     def __init__(self, arg):
>         self.arg = arg
>     def my_func(self, new_arg):
>         self.arg = new_arg
> 
> to call it:
> 
> arg = 1
> 
> m = MyObj(arg)
> print m.arg
> new_arg = 2
> m.my_func(new_arg)
> print m.arg

Just as a note, this would not really work if the variable needs to be 
changed and read from several places when the value is an immutable
type such as numbers / strings. In that case, then you could use 
the same logic but instead place the value in a list and pass that 
and always check/update the first element of the list. 


Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From eire1130 at gmail.com  Tue Jul 10 23:12:00 2012
From: eire1130 at gmail.com (James Reynolds)
Date: Tue, 10 Jul 2012 17:12:00 -0400
Subject: [Tutor] advice on global variables
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF4741654CC3E@SCACMX008.exchad.jpmchase.net>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
	<CAE0jAbqbgOXA20c=J1qNCrm4CiLeuhN5kfp37rvuiFWHykwxYA@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741654CC3E@SCACMX008.exchad.jpmchase.net>
Message-ID: <CAE0jAbqA_5VkacJcevUdXKhGrhv1LXbnnhZojExv3fMdGfZo0A@mail.gmail.com>

On Tue, Jul 10, 2012 at 4:32 PM, Prasad, Ramit <ramit.prasad at jpmorgan.com>wrote:

> > You should avoid using the global statement.
> >
> > In your case, I would think you could just add an argument to the method:
> >
> > class MyObj(object):
> >     def __init__(self, arg):
> >         self.arg = arg
> >     def my_func(self, new_arg):
> >         self.arg = new_arg
> >
> > to call it:
> >
> > arg = 1
> >
> > m = MyObj(arg)
> > print m.arg
> > new_arg = 2
> > m.my_func(new_arg)
> > print m.arg
>
> Just as a note, this would not really work if the variable needs to be
> changed and read from several places when the value is an immutable
> type such as numbers / strings. In that case, then you could use
> the same logic but instead place the value in a list and pass that
> and always check/update the first element of the list.
>
>
> Ramit
>
>
> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
> 712 Main Street | Houston, TX 77002
> work phone: 713 - 216 - 5423
>
> --
>
> This email is confidential and subject to important disclaimers and
> conditions including on offers for the purchase or sale of
> securities, accuracy and completeness of information, viruses,
> confidentiality, legal privilege, and legal entity disclaimers,
> available at http://www.jpmorgan.com/pages/disclosures/email.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>


In this case, you can read the attribute of MyObj and you just pass an
instantiated MyObj around to where it is needed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120710/a505ccd0/attachment-0001.html>

From alexandre.zani at gmail.com  Tue Jul 10 23:32:10 2012
From: alexandre.zani at gmail.com (Alexandre Zani)
Date: Tue, 10 Jul 2012 14:32:10 -0700
Subject: [Tutor] advice on global variables
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF4741654CC3E@SCACMX008.exchad.jpmchase.net>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
	<CAE0jAbqbgOXA20c=J1qNCrm4CiLeuhN5kfp37rvuiFWHykwxYA@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741654CC3E@SCACMX008.exchad.jpmchase.net>
Message-ID: <CAJVMpoqrP0GoquR1nFAjPJmHeqKupdKE3KMG=JnB3nVWmLZLRg@mail.gmail.com>

On Tue, Jul 10, 2012 at 1:32 PM, Prasad, Ramit
<ramit.prasad at jpmorgan.com> wrote:
>> You should avoid using the global statement.
>>
>> In your case, I would think you could just add an argument to the method:
>>
>> class MyObj(object):
>>     def __init__(self, arg):
>>         self.arg = arg
>>     def my_func(self, new_arg):
>>         self.arg = new_arg
>>
>> to call it:
>>
>> arg = 1
>>
>> m = MyObj(arg)
>> print m.arg
>> new_arg = 2
>> m.my_func(new_arg)
>> print m.arg
>
> Just as a note, this would not really work if the variable needs to be
> changed and read from several places when the value is an immutable
> type such as numbers / strings. In that case, then you could use
> the same logic but instead place the value in a list and pass that
> and always check/update the first element of the list.
>

That's a terrible idea. That's basically replicating the behavior of a
global while hiding that fact which makes the code even less readable.

>
> Ramit
>
>
> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
> 712 Main Street | Houston, TX 77002
> work phone: 713 - 216 - 5423
>
> --
>
> This email is confidential and subject to important disclaimers and
> conditions including on offers for the purchase or sale of
> securities, accuracy and completeness of information, viruses,
> confidentiality, legal privilege, and legal entity disclaimers,
> available at http://www.jpmorgan.com/pages/disclosures/email.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From ramit.prasad at jpmorgan.com  Wed Jul 11 00:03:09 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Tue, 10 Jul 2012 22:03:09 +0000
Subject: [Tutor] advice on global variables
In-Reply-To: <CAJVMpoqrP0GoquR1nFAjPJmHeqKupdKE3KMG=JnB3nVWmLZLRg@mail.gmail.com>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
	<CAE0jAbqbgOXA20c=J1qNCrm4CiLeuhN5kfp37rvuiFWHykwxYA@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741654CC3E@SCACMX008.exchad.jpmchase.net>
	<CAJVMpoqrP0GoquR1nFAjPJmHeqKupdKE3KMG=JnB3nVWmLZLRg@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741654CDB1@SCACMX008.exchad.jpmchase.net>

> > Just as a note, this would not really work if the variable needs to be
> > changed and read from several places when the value is an immutable
> > type such as numbers / strings. In that case, then you could use
> > the same logic but instead place the value in a list and pass that
> > and always check/update the first element of the list.
> >
> 
> That's a terrible idea. That's basically replicating the behavior of a
> global while hiding that fact which makes the code even less readable.

I agree, but sometimes getting something working is more important
to the person than doing something correctly. 

Not really sure about the OP's actual use case, but chances are he would 
be served better by creating a properties/static data module and
keeping the flag in there. 

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From eire1130 at gmail.com  Wed Jul 11 01:00:49 2012
From: eire1130 at gmail.com (James Reynolds)
Date: Tue, 10 Jul 2012 19:00:49 -0400
Subject: [Tutor] advice on global variables
In-Reply-To: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
Message-ID: <C2B21B19-889D-40EF-98E4-9907EB8DABC3@gmail.com>



Sent from my iPad

On Jul 10, 2012, at 3:11 PM, Chris Hare <chare at labr.net> wrote:

> 
> I know they are bad.  That is why I would prefer not to use it, but I am not sure how else to handle this problem.
> 
> In this app, the user must log in.  Once authenticated, they have a userid stored in the SQLite database.  Before splitting my app into multiple files, I used a global variable.  I know its bad, but it worked.  Now that things are split apart, the classes which used it previously now don't see it anymore, even using the global keyword.  I think this is the expected behavior.  See here
> 
> file: a.py
> 
> import b
> global_var = "global"
> 
> def func1():
>    global global_var
>    print "global var in func1 = %s" % global_var
> 
> class intclass:
>    def func2(self):
>        global global_var
>        print "global var in intclass = %s" % global_var
> 
> print "global_var = %s" % global_var
> func1()
> f = intclass()
> f.func2()
> g = b.extclass()
> g.func3()
> 
> file: b.py
> 
> class extclass:
>    def func3(self):
>        global global_var
>        print "global var in extclass = %s" % global_var
> 
> When I run it, I get what I think the expected behavior, that the external class ext class won't be able to see the global_var
> 
> Big-Mac:t chare$ python a.py
> global_var = global
> global var in func1 = global
> global var in intclass = global
> Traceback (most recent call last):
>  File "a.py", line 18, in <module>
>    g.func3()
>  File "/Users/chare/Development/python/animaltrax/pkg/t/b.py", line 5, in func3
>    print "global var in extclass = %s" % global_var
> NameError: global name 'global_var' is not defined
> 
> So - my question is this:  how do I solve the problem of multiple classes needing to get access to a value which needs to be preserved across the lifetime of the running application?
> 
> One thought was a RAM based SQLite database, but that seems like a lot of work.  I dunno, maybe that is the option.
> 
> suggestions, ideas, criticisms are all welcome.  Python code aside, I just don't know how to approach this problem in Python.
> 
> Thanks, as always for the feedback and guidance.
> 
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor



I'll admit it's a little tough for me to read your code.

If you need to store some information, store it to your db and access it later.

If would rather have he settings in the source code, study django's conf and how it handles configurations.

Better yet, why not just use django for your app?

From alan.gauld at btinternet.com  Wed Jul 11 01:16:07 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 11 Jul 2012 00:16:07 +0100
Subject: [Tutor] advice on global variables
In-Reply-To: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
Message-ID: <jtid3o$gra$1@dough.gmane.org>

On 10/07/12 20:11, Chris Hare wrote:

> I know they are bad.

They are not totally bad and most real programs wind up
having one or two globals, although usually those globals
are top level container classes, perhaps even an Application
class or similar.

> I am not sure how else to handle this problem.

Others have given lots of ideas but, by way of summary,
there are several common strategies:

1) Use function parameters. Your coding style seems to favour empty 
parameter lists. That's a bad idea. Pass values into functions as 
arguments and return values

def myfunc(n):
     return n+1

foo = 42
foo = myfunc(f00)

rather than

def myfunc():
     global foo
     foo += 1

myfunc()

It avoids hidden, and therefore forgotten, side effects, makes your 
function usable with variables other than foo and is often less code too.

2) Move the  "global" data plus all the functions that access it into a 
class:

class MyFuncs
    def __init__(self, n)
       self.foo = n
    def myfunc(self):
       self.foo += 1

m = MyFuncs(42)
m.myfunc()

Same effect as above but encapsulated in a class so you can create 
multiple instances each with its own value of foo. And you can pass the 
whole class/object around as an argument to other functions (see 
technique 1 above) And you can put it in a module to make the class 
available in other modules.

3) Move all global data into a single module which is then imported by 
any module that needs to access it . This avoids circular dependencies,

By combining these strategies you can minimise the number of globals, 
isolate them in a single module without dependencies  and by making the 
few globals into class instances keep control of the data access.

> One thought was a RAM based SQLite database, but that seems
 > like a lot of work.  I dunno, maybe that is the option.

It is an option, and for things like user ID etc its a valid one. This 
is definitely the best option where the "global" needs to be shared 
across different programs as well as different modules in a single 
program. But it is more work.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Wed Jul 11 01:24:57 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 11 Jul 2012 00:24:57 +0100
Subject: [Tutor] advice on global variables
In-Reply-To: <jtid3o$gra$1@dough.gmane.org>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
	<jtid3o$gra$1@dough.gmane.org>
Message-ID: <jtidk9$i1i$1@dough.gmane.org>

On 11/07/12 00:16, Alan Gauld wrote:

>> One thought was a RAM based SQLite database, but that seems
>  > like a lot of work.  I dunno, maybe that is the option.
>
> is definitely the best option where the "global" needs to be shared
> across different programs as well as different modules in a single

I meant to add its also the right technique where the 'global' value has 
to persist between different execution cycles of the program. (Or indeed 
any kind of value needs to persist across execution cycles!)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From chare at labr.net  Wed Jul 11 02:31:09 2012
From: chare at labr.net (Chris Hare)
Date: Tue, 10 Jul 2012 19:31:09 -0500
Subject: [Tutor] advice on global variables
In-Reply-To: <jtidk9$i1i$1@dough.gmane.org>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
	<jtid3o$gra$1@dough.gmane.org> <jtidk9$i1i$1@dough.gmane.org>
Message-ID: <38EBADCE-C2B1-4F15-B6E1-CB725F80007E@labr.net>


On Jul 10, 2012, at 6:24 PM, Alan Gauld wrote:

> On 11/07/12 00:16, Alan Gauld wrote:
> 
>>> One thought was a RAM based SQLite database, but that seems
>> > like a lot of work.  I dunno, maybe that is the option.
>> 
>> is definitely the best option where the "global" needs to be shared
>> across different programs as well as different modules in a single
> 
> I meant to add its also the right technique where the 'global' value has to persist between different execution cycles of the program. (Or indeed any kind of value needs to persist across execution cycles!)
> 

Thanks Alan -- I am thinking I am just gonna go with the RAM based SQLite database ?. 
> -- 
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> 
> 
> 
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor


From steve at pearwood.info  Wed Jul 11 03:47:11 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 11 Jul 2012 11:47:11 +1000
Subject: [Tutor] Regular expressions: findall vs search
In-Reply-To: <CAHgjEe2thMsvNYo8qtFWWS5GA+o=xBFGUjOvavTuWMtUw6wirQ@mail.gmail.com>
References: <CAHgjEe2thMsvNYo8qtFWWS5GA+o=xBFGUjOvavTuWMtUw6wirQ@mail.gmail.com>
Message-ID: <4FFCDB1F.2050601@pearwood.info>

Alexander Q. wrote:

> My question is how does Python know to return just the part in the
> parentheses and not to return the "blahblah" and the "yattayattayatta",
> etc...? The 're.search' function returns the whole thing, and if I want
> just the parentheses parts, I do tuples.group(1) or tuples.group(2) or
> tuples.group(3), depending on which set of parentheses I want.

The Fine Manual explains this. Is it not clear enough? If not, what parts are 
unclear -- we always appreciate suggested improvements.

http://docs.python.org/library/re.html#re.findall

Or at the interactive prompt, type:

help(re.findall)

which gives you:

Help on function findall in module re:

findall(pattern, string, flags=0)
     Return a list of all non-overlapping matches in the string.

     If one or more groups are present in the pattern, return a
     list of groups; this will be a list of tuples if the pattern
     has more than one group.

     Empty matches are included in the result.




-- 
Steven

From wayne at waynewerner.com  Wed Jul 11 05:27:43 2012
From: wayne at waynewerner.com (Wayne Werner)
Date: Tue, 10 Jul 2012 22:27:43 -0500 (CDT)
Subject: [Tutor] str object is not callable
In-Reply-To: <4FFC6AD7.8070304@pearwood.info>
References: <D03DDEB5-7BDC-4150-A0D6-F11C22C72173@labr.net>
	<CABicbJLp_i3ozwwsD2NjDr25nn=VawLs0fvoFDdjqEqyDoMFOg@mail.gmail.com>
	<2BD0F69E-CC7B-4A7B-9F59-766F744B398C@labr.net>
	<4FFC6AD7.8070304@pearwood.info>
Message-ID: <alpine.DEB.2.02.1207102226270.3761@gilgamesh>

On Wed, 11 Jul 2012, Steven D'Aprano wrote:

> Chris Hare wrote:
>> Okay - I am officially embarrassed. 
> [...]
>
>
> Meh, don't beat yourself up too badly. We've all been where you are now. 
> Sometimes I look back at my early Python code... I tell you, that's always a 
> good antidote for a big head.

I read a marvelous quote today:

"Good code is anything I wrote today. Bad code is anything I wrote more 
than a week ago."

-Wayne

From fulviocasali at gmail.com  Wed Jul 11 07:08:21 2012
From: fulviocasali at gmail.com (Fulvio Casali)
Date: Tue, 10 Jul 2012 22:08:21 -0700
Subject: [Tutor] find and replace multi-line strings in a directory tree
Message-ID: <CACuZjrq1-pX2=ndU37j0rW9nQhfQFWE7SGLb5So3Ef5GAzpK=Q@mail.gmail.com>

Hello Tutor,
First time poster here.

I wrote a little script to solve a find and replace problem, and I was
wondering if it could be improved, made more pythonic, and such.
The problem was that I had a directory tree with several thousand files,
about 2000 of which were static HTML (I blame my predecessor for this),
with the typical Google Analytics tracking code:

<script type="text/javascript">
>
>   var _gaq = _gaq || [];
>   _gaq.push(['_setAccount', 'UA-11111111-1']);
>   _gaq.push(['_trackPageview']);
>
>   (function() {
>     var ga = document.createElement('script'); ga.type =
> 'text/javascript'; ga.async = true;
>     ga.src = ('https:' == document.location.protocol ? 'https://ssl' : '
> http://www') + '.google-analytics.com/ga.js';
>     var s = document.getElementsByTagName('script')[0];
> s.parentNode.insertBefore(ga, s);
>   })();
>
> </script>
>

Now my client started worrying about the privacy of its users, and asked me
to remove all these snippets.  I wrote a regex that would capture this
multi-line snippet
In addition, several HTML files also had event handlers calling the GA
tracking script, e.g:

<a href="crime_and_punishment_jp.pdf" onMouseDown="javascript:
> _gaq.push(['_trackEvent', 'Translations', 'Downloaded PDF', 'Crime and
> Punishment vol 2 (Japanese)']); ">
>

The one improvement I would have hoped to make was to use
fileinput.FileInput, but I couldn't figure out how to make it work beyond
the one-line-at-a-time scenario.  It sure would be nice to hide all the
open(), read(), seek(), write(), truncate(), close() nonsense.

That said, my script works fine, and takes only a few seconds to traverse
the several thousand files.
Here is my script:

from os import walk
> from os.path import join
> from re import compile
>
> def handler(error):
>   print error
>
> script_pattern=compile(r'<script[^<]*_gaq[^<]*</script>')
>
> event_pattern=compile(r'onMouseDown\s*=\s*"\s*javascript:\s*_gaq.push[^"]+"')
>
> modified = []
> event = []
>
> for root, dirs, files in walk('rc', onerror=handler):
>   for name in files:
>     path = join(root, name)
>     file = open(path, 'r+b')
>     all = file.read()
>     file.seek(0)
>     if script_pattern.search(all):
>       modified.append(path)
>       fixed = script_pattern.sub('', all)
>       if event_pattern.search(fixed):
>         event.append(path)
>         fixed = event_pattern.sub('', fixed)
>       file.write(fixed)
>       file.truncate()
>     file.close()
>
> for i in modified: print i
> print len(modified)
>
> for i in event: print i
> print len(event)
>


Thanks for any feedback!

-- 
Fulvio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120710/d96ede11/attachment.html>

From kala586 at gmail.com  Wed Jul 11 10:48:04 2012
From: kala586 at gmail.com (kala Vinay)
Date: Wed, 11 Jul 2012 14:18:04 +0530
Subject: [Tutor] Extracting the list from a string
Message-ID: <CAL+b0G-rBJ2GuKY+63D899d_SQLG4==o5Fha3dQirUM--vizUg@mail.gmail.com>

Hi all,

       say i have a string s="['a','b']" and i want to get a list object
from this string s ie l=['a','b']

Please help.

Thank you in advance

-- 
Regards,
Kala
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120711/2cfef210/attachment-0001.html>

From steve at pearwood.info  Wed Jul 11 11:10:19 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 11 Jul 2012 19:10:19 +1000
Subject: [Tutor] Extracting the list from a string
In-Reply-To: <CAL+b0G-rBJ2GuKY+63D899d_SQLG4==o5Fha3dQirUM--vizUg@mail.gmail.com>
References: <CAL+b0G-rBJ2GuKY+63D899d_SQLG4==o5Fha3dQirUM--vizUg@mail.gmail.com>
Message-ID: <20120711091018.GA21489@ando>

On Wed, Jul 11, 2012 at 02:18:04PM +0530, kala Vinay wrote:
> Hi all,
> 
>        say i have a string s="['a','b']" and i want to get a list object
> from this string s ie l=['a','b']

The AST module contains a safe way to parse and eval literal 
expressions, including strings, nested lists, and builtin constants like 
None, True and False.

py> import ast
py> ast.literal_eval('[23, 42, "hello", "world", -1.5, [], {}]')
[23, 42, 'hello', 'world', None, True, False, -1.5, [], {}]

Unlike the eval command, it is safe and won't execute code:

py> text = '__import__("os").system("echo \\"Your computer is mine now!\\"")')
py> eval(text)
Your computer is mine now!
0
py> ast.literal_eval(text)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/ast.py", line 68, in literal_eval
    return _convert(node_or_string)
  File "/usr/lib/python2.6/ast.py", line 67, in _convert
    raise ValueError('malformed string')
ValueError: malformed string


-- 
Steven


From kala586 at gmail.com  Wed Jul 11 12:09:52 2012
From: kala586 at gmail.com (kala Vinay)
Date: Wed, 11 Jul 2012 15:39:52 +0530
Subject: [Tutor] Tutor Digest, Vol 101, Issue 37
In-Reply-To: <mailman.21.1342000801.7365.tutor@python.org>
References: <mailman.21.1342000801.7365.tutor@python.org>
Message-ID: <CAL+b0G-9KpzxjN95XPCDy-Q-3=Ax-f=WRsbVoxWWG8VXKjOdFA@mail.gmail.com>

On Wed, Jul 11, 2012 at 3:30 PM, <tutor-request at python.org> wrote:

> Send Tutor mailing list submissions to
>         tutor at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mail.python.org/mailman/listinfo/tutor
> or, via email, send a message with subject or body 'help' to
>         tutor-request at python.org
>
> You can reach the person managing the list at
>         tutor-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."
>
>
> Today's Topics:
>
>    1. Re: Extracting the list from a string (Steven D'Aprano)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 11 Jul 2012 19:10:19 +1000
> From: Steven D'Aprano <steve at pearwood.info>
> To: tutor at python.org
> Subject: Re: [Tutor] Extracting the list from a string
> Message-ID: <20120711091018.GA21489 at ando>
> Content-Type: text/plain; charset=us-ascii
>
> On Wed, Jul 11, 2012 at 02:18:04PM +0530, kala Vinay wrote:
> > Hi all,
> >
> >        say i have a string s="['a','b']" and i want to get a list object
> > from this string s ie l=['a','b']
>
> The AST module contains a safe way to parse and eval literal
> expressions, including strings, nested lists, and builtin constants like
> None, True and False.
>
> py> import ast
> py> ast.literal_eval('[23, 42, "hello", "world", -1.5, [], {}]')
> [23, 42, 'hello', 'world', None, True, False, -1.5, [], {}]
>
> Unlike the eval command, it is safe and won't execute code:
>
> py> text = '__import__("os").system("echo \\"Your computer is mine
> now!\\"")')
> py> eval(text)
> Your computer is mine now!
> 0
> py> ast.literal_eval(text)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib/python2.6/ast.py", line 68, in literal_eval
>     return _convert(node_or_string)
>   File "/usr/lib/python2.6/ast.py", line 67, in _convert
>     raise ValueError('malformed string')
> ValueError: malformed string
>
>
> --
> Steven
>




>   Thank You Steven, Its working fine....
>
>
> ------------------------------
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
> End of Tutor Digest, Vol 101, Issue 37
> **************************************
>



-- 
Regards,
Kala
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120711/ed72df06/attachment.html>

From dfjennings at gmail.com  Wed Jul 11 14:34:47 2012
From: dfjennings at gmail.com (Don Jennings)
Date: Wed, 11 Jul 2012 08:34:47 -0400
Subject: [Tutor] advice on global variables
In-Reply-To: <mailman.5627.1341996486.4696.tutor@python.org>
References: <mailman.5627.1341996486.4696.tutor@python.org>
Message-ID: <E998C99B-DF47-4FFB-AA2C-7EEE2A1D4329@gmail.com>


On Jul 11, 2012, at 4:48 AM, tutor-request at python.org wrote:

> Message: 1
> Date: Tue, 10 Jul 2012 19:31:09 -0500
> From: Chris Hare <chare at labr.net>
> To: tutor at python.org
> Subject: Re: [Tutor] advice on global variables
> Message-ID: <38EBADCE-C2B1-4F15-B6E1-CB725F80007E at labr.net>
> Content-Type: text/plain; charset=windows-1252
> 
> 
> On Jul 10, 2012, at 6:24 PM, Alan Gauld wrote:
> 
>> On 11/07/12 00:16, Alan Gauld wrote:
>> 
>>>> One thought was a RAM based SQLite database, but that seems
>>>> like a lot of work.  I dunno, maybe that is the option.
>>> 
>>> is definitely the best option where the "global" needs to be shared
>>> across different programs as well as different modules in a single
>> 
>> I meant to add its also the right technique where the 'global' value has to persist between different execution cycles of the program. (Or indeed any kind of value needs to persist across execution cycles!)
>> 
> 
> Thanks Alan -- I am thinking I am just gonna go with the RAM based SQLite database ?. 

Another option you might want to consider is the y_serial module [1] as it's a lot less work and allows you to **keep** thinking in python :>)

Take care,
Don

[1] http://yserial.sourceforge.net/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120711/9b2d6bc1/attachment.html>

From wprins at gmail.com  Wed Jul 11 15:05:55 2012
From: wprins at gmail.com (Walter Prins)
Date: Wed, 11 Jul 2012 14:05:55 +0100
Subject: [Tutor] advice on global variables
In-Reply-To: <38EBADCE-C2B1-4F15-B6E1-CB725F80007E@labr.net>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
	<jtid3o$gra$1@dough.gmane.org> <jtidk9$i1i$1@dough.gmane.org>
	<38EBADCE-C2B1-4F15-B6E1-CB725F80007E@labr.net>
Message-ID: <CANLXbfC0EGiBXbJb0rH46caRgLu65U+9dCa66CRwo7xwKB6JgA@mail.gmail.com>

Hi,

On 11 July 2012 01:31, Chris Hare <chare at labr.net> wrote:
> Thanks Alan -- I am thinking I am just gonna go with the RAM based SQLite database ?.

That seems an awfully large hammer for a little global variable
problem.  Why can you not (as a start) just move the global(s) into
their own namespace/location that's readily accessible i.e. can be
simply imported where needed/referenced.  In other words as has
essentially been suggested, make a module containing the global
variables directly (or perhaps as part of a class or object that can
act as a container for them) ?    (To be a little more direct: I don't
think a RAM based SQLite database will really do anything for you
beyond what a little shared module/class/object won't already do in
terms of dealing with a global variable problem, and will instead
really just add needless complexity to your program. There is a time
and place for in-memory databases but this is IMHO not really it.)

Your original example modified as demonstration:

a.py:
====
import shared
import b

def func1():
        print "global var in func1 = %s" % shared.global_var

class intclass:
        def func2(self):
                print "global var in intclass = %s" % shared.global_var

print "global_var = %s" % shared.global_var
func1()
f = intclass()
f.func2()
g = b.extclass()
g.func3()

b.py:
====

import shared

class extclass:
        def func3(self):
                print "global var in extclass = %s" % shared.global_var


shared.py:
=======
global_var = "global"

Walter

From susana_87 at hotmail.com  Wed Jul 11 16:20:05 2012
From: susana_87 at hotmail.com (susana moreno colomer)
Date: Wed, 11 Jul 2012 16:20:05 +0200
Subject: [Tutor] get columns from txt file
Message-ID: <BAY164-W2886BE3C3D5A8B0E3D496384D10@phx.gbl>


Hi!
 
I have a group of files in a directory.
I want to extract from  files whose names start with  bb_   column number 5 to an excel file. I have 6  bb_  files, therefore I want to get 6 columns (5th column from each file)
This code is working,with the following errors:

I get the data in only one column, instead of six
I get white cell after every extracted cell
 
I've got  help from http://mail.python.org/pipermail/tutor/2004-November/033474.html, though it is not working for me
 
This is my code:
 
 
import os
import fnmatch
import csv

path = '//......'
files=os.listdir(path)
csv_out=csv.writer(open('out.csv', 'w'), delimiter='  ')

for infile in files:
   
        if fnmatch.fnmatch(infile, 'bb_*'):
                print infile
       
                filename= path+infile
                print filename
        
                f=open(filename)
                for line in f.readlines():
                       b=line.split('\t')
                       csv_out.writerow(b[5])
               f.close
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120711/236bb065/attachment-0001.html>

From dfjennings at gmail.com  Wed Jul 11 16:59:39 2012
From: dfjennings at gmail.com (Don Jennings)
Date: Wed, 11 Jul 2012 10:59:39 -0400
Subject: [Tutor] get columns from txt file
In-Reply-To: <mailman.5674.1342016475.4696.tutor@python.org>
References: <mailman.5674.1342016475.4696.tutor@python.org>
Message-ID: <467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>


On Jul 11, 2012, at 10:21 AM, tutor-request at python.org wrote:
> 
> Message: 4
> Date: Wed, 11 Jul 2012 16:20:05 +0200
> From: susana moreno colomer <susana_87 at hotmail.com>
> To: <tutor at python.org>
> Subject: [Tutor] get columns from txt file
> Message-ID: <BAY164-W2886BE3C3D5A8B0E3D496384D10 at phx.gbl>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> 
> Hi!
> 
> I have a group of files in a directory.
> I want to extract from  files whose names start with  bb_   column number 5 to an excel file. I have 6  bb_  files, therefore I want to get 6 columns (5th column from each file)
> This code is working,with the following errors:
> 
> I get the data in only one column, instead of six
> I get white cell after every extracted cell
> 
> I've got  help from http://mail.python.org/pipermail/tutor/2004-November/033474.html, though it is not working for me


If I understand your requirements correctly, you should read the followup message in that thread:

http://mail.python.org/pipermail/tutor/2004-November/033475.html

Primarily, you'll just have to alter one of the code samples to write it to csv.

Take care,
Don

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120711/49043c14/attachment.html>

From chare at labr.net  Wed Jul 11 16:30:33 2012
From: chare at labr.net (Chris Hare)
Date: Wed, 11 Jul 2012 09:30:33 -0500
Subject: [Tutor] advice on global variables
In-Reply-To: <CANLXbfC0EGiBXbJb0rH46caRgLu65U+9dCa66CRwo7xwKB6JgA@mail.gmail.com>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
	<jtid3o$gra$1@dough.gmane.org> <jtidk9$i1i$1@dough.gmane.org>
	<38EBADCE-C2B1-4F15-B6E1-CB725F80007E@labr.net>
	<CANLXbfC0EGiBXbJb0rH46caRgLu65U+9dCa66CRwo7xwKB6JgA@mail.gmail.com>
Message-ID: <7AF94A0E-09F7-413D-BE70-37ADA082A0B8@labr.net>


On Jul 11, 2012, at 8:05 AM, Walter Prins wrote:

> [snip]

> Your original example modified as demonstration:
> 
> a.py:
> ====
> import shared
> import b
> 
> def func1():
>        print "global var in func1 = %s" % shared.global_var
> 
> class intclass:
>        def func2(self):
>                print "global var in intclass = %s" % shared.global_var
> 
> print "global_var = %s" % shared.global_var
> func1()
> f = intclass()
> f.func2()
> g = b.extclass()
> g.func3()
> 
> b.py:
> ====
> 
> import shared
> 
> class extclass:
>        def func3(self):
>                print "global var in extclass = %s" % shared.global_var
> 
> 
> shared.py:
> =======
> global_var = "global"
> 
> 
I like where this is going Walter.  I guess where I am confused is this:

the globals are not static - they are set once and then won't change during the lifetime of the user's session.   

So, after messing around with the ram DB idea, I realized I was back to the same problem.

Let's say I create a dictionary to store all of the "globals" and other data I want available everywhere and I put that in a class.  Every time I create an instance of the class to be able to access the data, a new instance of the dictionary is created that loads the same data in it.  Seems kinda inefficient for me, especially if the data changes in one instance - how do you keep the others all in sync, or does that just happen automatically?

I think there is something that I fundamentally not understanding and I don't know what it is.   Am I making this too complicated?  




From ramit.prasad at jpmorgan.com  Wed Jul 11 16:42:41 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Wed, 11 Jul 2012 14:42:41 +0000
Subject: [Tutor] get columns from txt file
In-Reply-To: <BAY164-W2886BE3C3D5A8B0E3D496384D10@phx.gbl>
References: <BAY164-W2886BE3C3D5A8B0E3D496384D10@phx.gbl>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741654FAEC@SCACMX008.exchad.jpmchase.net>

> I have a group of files in a directory.
> I want to extract from  files whose names start with  bb_   column number 5 to
> an excel file. I have 6  bb_  files, therefore I want to get 6 columns (5th
> column from each file)
> This code is working,with the following errors:
> * I get the data in only one column, instead of six
> * I get white cell after every extracted cell
> 
> I've got  help from http://mail.python.org/pipermail/tutor/2004-
> November/033474.html, though it is not working for me
> 
> This is my code:
> 
> 
> import os
> import fnmatch
> import csv
> 
> path = '//......'
> files=os.listdir(path)
> csv_out=csv.writer(open('out.csv', 'w'), delimiter='  ')
> 
> for infile in files:
> 
>         if fnmatch.fnmatch(infile, 'bb_*'):
>                 print infile
> 
>                 filename= path+infile
>                 print filename
> 
>                 f=open(filename)
>                 for line in f.readlines():
>                        b=line.split('\t')
>                        csv_out.writerow(b[5])
>                f.close

You get 6 lines because csv_out.writerow writes a row and then
goes to the next line.

What you want to do is read through each file and append each
value to a list. Once you are done reading, write the list using
the csv module.

I am not sure what you mean by "white cell after every extracted
cell" but if you mean spaces you can use (where output is a list).

output.append(b[5].strip()) 

Also, csv files should be opened using 'wb' not 'w' otherwise
you will get extra lines in the file. Maybe this is what you
meant by white cell?


Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From bell.james03 at gmail.com  Wed Jul 11 17:31:33 2012
From: bell.james03 at gmail.com (James Bell)
Date: Wed, 11 Jul 2012 11:31:33 -0400
Subject: [Tutor] Define Build Deployment
Message-ID: <CAAPsuXV0Psk1WVpzO3N6XiEVrpE8-W9N=CXc5uN5YSFSb48p6g@mail.gmail.com>

I'm fairly new to software development in an enterprise environment I'm
constantly hearing the term "build deployment" and do no want to ask what
it means since it seems simple. I cannot find a definition online.

if the word is context specific please describe what it means at your
company.

thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120711/3d09980f/attachment.html>

From eire1130 at gmail.com  Wed Jul 11 17:32:42 2012
From: eire1130 at gmail.com (James Reynolds)
Date: Wed, 11 Jul 2012 11:32:42 -0400
Subject: [Tutor] advice on global variables
In-Reply-To: <7AF94A0E-09F7-413D-BE70-37ADA082A0B8@labr.net>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
	<jtid3o$gra$1@dough.gmane.org> <jtidk9$i1i$1@dough.gmane.org>
	<38EBADCE-C2B1-4F15-B6E1-CB725F80007E@labr.net>
	<CANLXbfC0EGiBXbJb0rH46caRgLu65U+9dCa66CRwo7xwKB6JgA@mail.gmail.com>
	<7AF94A0E-09F7-413D-BE70-37ADA082A0B8@labr.net>
Message-ID: <CAE0jAbq93a_4j+bPh9d9P_=E9YQ1j=OizyOpCm_mZsjfxH-Qxw@mail.gmail.com>

On Wed, Jul 11, 2012 at 10:30 AM, Chris Hare <chare at labr.net> wrote:

>
> On Jul 11, 2012, at 8:05 AM, Walter Prins wrote:
>
> > [snip]
>
> > Your original example modified as demonstration:
> >
> > a.py:
> > ====
> > import shared
> > import b
> >
> > def func1():
> >        print "global var in func1 = %s" % shared.global_var
> >
> > class intclass:
> >        def func2(self):
> >                print "global var in intclass = %s" % shared.global_var
> >
> > print "global_var = %s" % shared.global_var
> > func1()
> > f = intclass()
> > f.func2()
> > g = b.extclass()
> > g.func3()
> >
> > b.py:
> > ====
> >
> > import shared
> >
> > class extclass:
> >        def func3(self):
> >                print "global var in extclass = %s" % shared.global_var
> >
> >
> > shared.py:
> > =======
> > global_var = "global"
> >
> >
> I like where this is going Walter.  I guess where I am confused is this:
>
> the globals are not static - they are set once and then won't change
> during the lifetime of the user's session.
>
> So, after messing around with the ram DB idea, I realized I was back to
> the same problem.
>
> Let's say I create a dictionary to store all of the "globals" and other
> data I want available everywhere and I put that in a class.  Every time I
> create an instance of the class to be able to access the data, a new
> instance of the dictionary is created that loads the same data in it.
>  Seems kinda inefficient for me, especially if the data changes in one
> instance - how do you keep the others all in sync, or does that just happen
> automatically?
>
> I think there is something that I fundamentally not understanding and I
> don't know what it is.   Am I making this too complicated?
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>


You are making this too complicated.

I think you are getting hung up on instantiating objects. You don't need to
instantiate an object everytime you use it. Once you have it, it's a
variable like any other and you can pass it around.

Just have a class body with your variables:

class MyObj(object):
   def __init__(self, *args, **kwargs):
       do stuff with variables here

Then instantiate the object

m = myobject(pass in your parameters)

now pass around the instantiated object on an as needed basis, or, access
the instance directly

example 1:

def func(obj):
   do stuff with obj

to call it: func(m)

example 2:

File 1: #tests

def reset_m(m, settings):
    m.settings = settings

File 2: #test_two

class MyObj(object):
    def __init__(self, settings=True):
        self.settings = settings



if __name__ == '__main__':
    m = MyObj()


File 3: #main

import test_two
import tests

m = test_two.MyObj()

print m.settings

tests.reset_m(m, False)

print m.settings

The above will print True, False

But, you don't need to create a class that has a single attribute, which is
a dictionary. In that case, you can just create a dict and pass that around
(I generally prefer to have a class body and access things using dot
notation rather than dictionary syntax)

Generally though, I think this entire idea of settings dictionary or class
can be simplified. I'll bet most of the settings have some general
similarities or are repeated, or they are the same for each user, each time.

In that case, you should probably move to a more database driven approach,
or hardcode settings in a file and override those hardcoded settings on an
as needed basis.

In django, you have a "settings" file. In there, you hard code some most of
your settings, like apps you are using, where your database resides,
middleware, etc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120711/9d2fe0ae/attachment-0001.html>

From ramit.prasad at jpmorgan.com  Wed Jul 11 18:00:08 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Wed, 11 Jul 2012 16:00:08 +0000
Subject: [Tutor] Define Build Deployment
In-Reply-To: <CAAPsuXV0Psk1WVpzO3N6XiEVrpE8-W9N=CXc5uN5YSFSb48p6g@mail.gmail.com>
References: <CAAPsuXV0Psk1WVpzO3N6XiEVrpE8-W9N=CXc5uN5YSFSb48p6g@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741654FBD2@SCACMX008.exchad.jpmchase.net>

> I'm fairly new to software development in an enterprise environment I'm
> constantly hearing the term "build deployment" and do no want to ask what it
> means since it seems simple. I cannot find a definition online.

Not really a Python question.

A "build" is when you compile (and optionally running any unit testing / 
code analysis) or package a specific version. It is used to describe both 
the action of compiling and also to the resulting compiled code, which
may be a little confusing. Technically, I guess "building" is the compiling
process while a "build" is the result.

Deployment is when you put a "build" (packaged code) into a certain 
environment. This can also be called "build deployment" because you
are "deploying" a "build". 

Ramit

--
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From alexandre.zani at gmail.com  Wed Jul 11 19:14:51 2012
From: alexandre.zani at gmail.com (Alexandre Zani)
Date: Wed, 11 Jul 2012 10:14:51 -0700
Subject: [Tutor] Define Build Deployment
In-Reply-To: <CAAPsuXV0Psk1WVpzO3N6XiEVrpE8-W9N=CXc5uN5YSFSb48p6g@mail.gmail.com>
References: <CAAPsuXV0Psk1WVpzO3N6XiEVrpE8-W9N=CXc5uN5YSFSb48p6g@mail.gmail.com>
Message-ID: <CAJVMpoogaDwJYfM9Uu_uWde-xhzDPx+yypQij2vhoULLVL07oQ@mail.gmail.com>

On Wed, Jul 11, 2012 at 8:31 AM, James Bell <bell.james03 at gmail.com> wrote:
> I'm fairly new to software development in an enterprise environment I'm
> constantly hearing the term "build deployment" and do no want to ask what it
> means since it seems simple. I cannot find a definition online.
>
> if the word is context specific please describe what it means at your
> company.
>
> thanks
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

That's not python-related and the exact meaning will vary a lot
organization to organization. My advice is to learn to ask stupid
questions. That's the way we all learn.

From alan.gauld at btinternet.com  Thu Jul 12 10:54:14 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 12 Jul 2012 09:54:14 +0100
Subject: [Tutor] Extracting the list from a string
In-Reply-To: <CAL+b0G-rBJ2GuKY+63D899d_SQLG4==o5Fha3dQirUM--vizUg@mail.gmail.com>
References: <CAL+b0G-rBJ2GuKY+63D899d_SQLG4==o5Fha3dQirUM--vizUg@mail.gmail.com>
Message-ID: <jtm3bm$rbs$1@dough.gmane.org>

On 11/07/12 09:48, kala Vinay wrote:

>         say i have a string s="['a','b']" and i want to get a list
> object from this string s ie l=['a','b']

It depends on how complex your string is and how much you want to put in 
the list. If you just want to convert the entire expression into a 
Python value then Steven's use of ast is simplest.

If you want to extract elements out of the string, like the letters in 
your example then you could use a list comprehension:

L = [c for c in s if c.isalpha()]

Or for more complex scenarios you can supply a list of valid values:

vowels = 'aeiou'
L = [c for c in s if c in vowels]

Or for even more complex rules you could supply any arbitrary function:

L = [c for c in s if myfunc(c)]

Other options include using a regular expression.
It all depends on exactly what you want to put in your list.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Thu Jul 12 11:11:11 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 12 Jul 2012 10:11:11 +0100
Subject: [Tutor] Define Build Deployment
In-Reply-To: <CAAPsuXV0Psk1WVpzO3N6XiEVrpE8-W9N=CXc5uN5YSFSb48p6g@mail.gmail.com>
References: <CAAPsuXV0Psk1WVpzO3N6XiEVrpE8-W9N=CXc5uN5YSFSb48p6g@mail.gmail.com>
Message-ID: <jtm4bf$37q$1@dough.gmane.org>

On 11/07/12 16:31, James Bell wrote:
> I'm fairly new to software development in an enterprise environment I'm
> constantly hearing the term "build deployment" and do no want to ask
> what it means since it seems simple. I cannot find a definition online.

A build deployment is the deployment of a build.

A build is the assembly of all the parts needed for a project.
For a small project that could be as simple as a single code file or 
executable that gets distributed by email/ftp or a web server.

For bigger projects it will probably involve lots of modules with 
dependencies between them. There will likely be an installation script 
and probably some documentation as well.

On very big projects a build could involve many separate programs (eg 
server processes, a GUI client, an MIS reporting package, a data loader, 
multiple documents (user guide, install guide, maintenance manual, 
online help etc). The project build will ensure that all those files are 
assembled in a consistent way ready to be installed on a target 
environment (a customers site say).

The deployment is the actual installation of that build. For a big 
project the deployment might be a multi day activity in its own right 
with "professional services" people from the supplier going to the 
customer site. It may even involve buying and installing new hardware 
(dedicated servers, routers, or bespoke telemetry for example).
At the other end of the scale its simply a matter of making the build 
available on an ftp server (as a zip file maybe) and the clients do the 
install themselves.

HTH,
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From wprins at gmail.com  Thu Jul 12 11:16:56 2012
From: wprins at gmail.com (Walter Prins)
Date: Thu, 12 Jul 2012 10:16:56 +0100
Subject: [Tutor] advice on global variables
In-Reply-To: <7AF94A0E-09F7-413D-BE70-37ADA082A0B8@labr.net>
References: <9F2449DF-1F09-45AC-B52B-9A7EFD60C7A5@labr.net>
	<jtid3o$gra$1@dough.gmane.org> <jtidk9$i1i$1@dough.gmane.org>
	<38EBADCE-C2B1-4F15-B6E1-CB725F80007E@labr.net>
	<CANLXbfC0EGiBXbJb0rH46caRgLu65U+9dCa66CRwo7xwKB6JgA@mail.gmail.com>
	<7AF94A0E-09F7-413D-BE70-37ADA082A0B8@labr.net>
Message-ID: <CANLXbfDKnYuoGd-9KUUZQz2bqC6o=owj17o9HELVVxVQezxE2g@mail.gmail.com>

Hi Chris,

James has already given a response but I can't resist also chiming in,
see below:

On 11 July 2012 15:30, Chris Hare <chare at labr.net> wrote:
> I like where this is going Walter.  I guess where I am confused is this:
>
> the globals are not static - they are set once and then won't change during the lifetime of the user's session.

Whether they change or not is beside the question.  They can get set
once, they can be set multiple times, it really doesn't matter.


> So, after messing around with the ram DB idea, I realized I was back to the same problem.

Exactly. :)

> Let's say I create a dictionary to store all of the "globals" and other data I want available everywhere and I put that in a class.  > Every time I create an instance of the class to be able to access the data,

Now why do you think that you need to instantiate the class everytime
you want to access the same data?  Just damn well create the object
once and use it when you need it.  Do you not know that your own
objects can be treated as a parameters to a methods/functions just
like any other Python object or type?

> a new instance of the dictionary is created that loads the same data in it.  Seems kinda inefficient for me, especially if the data changes in one instance - how do you keep the others all in sync, or does that just happen automatically?

Well exactly.  Just use the same object everywhere!

> I think there is something that I fundamentally not understanding and I don't know what it is.   Am I making this too complicated?

Yes and yes.  I think you are getting confused with and are maybe
conflating the names an object might be associated to (and the
namespaces those names live in), the lifetime of an object might have
(e.g. when is it created, when does it get destroyed.), and where
objects may be access from. A lot might be said about this but suffice
it to say that you should conceptualize objects as staying in
existence until the last reference to them is removed (reference being
either a name label or entry another container object reference such
as a list.)  Secondly, all you need to access an object is a reference
to it -- either a name label or direct reference (e.g. as passed in a
container such as a list.)  The reference may be received as a
parameter to a function or method either directly or indirectly, or
code may fetch it from a suitable module itself by importing the
module.  (Which incidentally is also an object.)

Anyway I don't have time to elaborate more right now but I suggest you
try to read up on these topics a bit yourself first and then
experiment apart from the program you're working on to help cement the
various basic ideas surrounding objects & their lifetimes, references
to objects, passing parameters and so on.  Then come back with more
questions.

Walter

From dfjennings at gmail.com  Thu Jul 12 14:53:49 2012
From: dfjennings at gmail.com (Don Jennings)
Date: Thu, 12 Jul 2012 08:53:49 -0400
Subject: [Tutor] get columns from txt file
In-Reply-To: <BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>
Message-ID: <15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>

(Please use reply all so the message gets posted for all to see :>)

On Jul 12, 2012, at 7:14 AM, susana moreno colomer wrote:

> Hi!
> Many thanks for your help.
> Now I am trying this, but I get a blank excel document

Not surprising as there are several errors in your code. In fact, I'm surprised it ran without reporting an error. Be sure to copy and paste the code exactly as you are running it.

> (I have tried everything, but I don't get it!)

That's why we're here :>)

As I understand it, you have a file structure like this:

dir/
    bb_csvfile1.txt
    bb_csvfile2.txt
    bb_csvfilen.txt
    someotherfile.txt

You want to read each of the csv files which start with 'bb_', extracting the data in a certain column, and then output that data to a single excel document. Yes?


>  
> import os
> import fnmatch
> import csv
> 
> path = '...'

First of all, this path variable looks odd. Do you mean '.' for the current directory or '..' for the parent directory?

> csv_out=csv.writer(open('out11.csv', 'wb'), delimiter='  ')

Since you said you want an excel document, you should specify dialect='excel'  (or 'excel-tab') instead of a delimiter.

> files=os.listdir(path)
>  
> for infile in files:
>  output=[] 

The variable output should be moved outside the for loop (i.e. put it before the "for infile in files:")

>  if fnmatch.fnmatch(infile, 'bb_*'):
>   global filename

Drop the global statement. Out of curiousity, why did you think you needed that? (There was no use of global in the example code from Kent Johnson.)

>   filename= path+infile

Won't work (even if the path variable was correct)?where's the file path separator? Use the join function of os.path:

filename = os.path.join(path, infile)

>         
>   global f

Again, no need for the global statement.

>   f=open(filename, 'r')
>        
>   for line in f:
>  
>    b=line.split('\t')
>    output.append(b[5].strip())
>   
>    def csvwriter():

No need for a function here. In fact, you don't ever actually call it, so it never gets run! That's a major reason why you aren't getting any output.

>     csv_out.append(output)
>     csv_out.append('\n')

These should be outside of the for loops as well (i.e. remove all indentation).
>    
>   f.close


You need the parentheses to call the function and it should be at the same level as the call where you open it (i.e. the same amount of indentation, before the calls to csv_out.append):

f.close()

Take care,
Don

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/8eddd13f/attachment.html>

From dfjennings at gmail.com  Thu Jul 12 16:26:30 2012
From: dfjennings at gmail.com (Don Jennings)
Date: Thu, 12 Jul 2012 10:26:30 -0400
Subject: [Tutor] get columns from txt file
In-Reply-To: <BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>
Message-ID: <EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>

Oops! Still you forgot to cc: the tutor list. It's really important because if someone (like me, for instance) steers you in the wrong direction, others will jump in with corrections.


On Jul 12, 2012, at 9:48 AM, susana moreno colomer wrote:

> Hi!
> Many thanks!

You're welcome. I see that your code is improving :>)

> Still I get an error: AttributeError: '_cvs.writer' object has no atribute 'append'.

If you would like a response to that error, please send the exact code you tried which didn't work and the error message by copying/pasting. (See, I'm pretty sure that you typed in the error above since you misspelled attribute as 'atribute'.)

> I wanted to solve it with the Def function.

And that's why you should send the original code and error. Now, you have a different problem, but it's a good time to clear up your confusion. Def is not a function. In fact, I don't know what "Def" is. Instead, the def statement?notice that it's all lower case?defines a function which you can call elsewhere in code.

>>> def call_me():
...     print "inside the call_me function"
... 
>>> call_me()
inside the call_me function

Note, that there is no output from our defining the call_me function; it's only when we call it with the parentheses that the code inside of it is executed. Make sense? However, again, it's not necessary for this script. Once you have the csv_out variable, you should be able to 

> Now I am trying this, though I get
>  
>  
> >import os
> >import fnmatch
> >import csv
> 
> >path = '/This is my current directory/'
> >csv_out=csv.writer(open('out13.csv', 'wb'), delimiter=' ')
> >files=os.listdir(path)
> >outfile=open('myfile1', 'w')
> >output=[]
> 
> >for infile in files:
> >      if fnmatch.fnmatch(infile, 'bb_*'):
> >      filename= os.path.join(path,infile)
> >      f=open(filename, 'r')
>   
> >      for line in f:
> >          b=line.split('\t')
> >          output.append(b[5].strip())
> >     f.close()
>  
> >Def csvwriter():       ---->gives me SyntaxError: invalid syntax
> >        excelfile=csv_out
> >        a=excelfile.append(output)
> >        c=excelfile.write(a)
> >        return c
>  
>  
> I am trying with this  because the atribute 'writerows' didn't work.

A different error. There are lots of reasons it might not have worked. Wish you had sent us the code for that one :<( We can't solve it without all the info.

> Is there another way to write in csv files?

Absolutely. At the interactive prompt, dir() shows all the attributes of an object to find out your options:

>>> csv_out=csv.writer(open('out13.csv', 'wb'), dialect='excel')
>>> dir(csv_out)
['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'dialect', 'writerow', 'writerows']

See the writerow and writerows? Those look promising. Alternatively, while it takes a while to learn to read the python documentation, it really is your friend:

http://docs.python.org/library/csv.html

If you'd like further examples, Doug Hellman often provides a great resource with his Python Module of the Week (PYMTOW) series:

http://www.doughellmann.com/PyMOTW/csv/

> I don't know what you mean with specify dialect

Something like this:

csv_out = csv.writer(fileobject, dialect='excel')

Take care,
Don

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/1fa86cc9/attachment-0001.html>

From susana_87 at hotmail.com  Thu Jul 12 16:55:33 2012
From: susana_87 at hotmail.com (susana moreno colomer)
Date: Thu, 12 Jul 2012 16:55:33 +0200
Subject: [Tutor] get columns from txt file
In-Reply-To: <EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
Message-ID: <BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>


Hi!
 
I have a group of files in a directory:
bb_1.txt
bb_2.txt
bb_3.txt
bb_4.txt
bb_5.txt
bb_6.txt
ss_1.txt

I want to extract from  files whose names start with  bb_   column number 5 to an excel file. I have 6  bb_  files, therefore I want to get 6 columns (5th column from each file)
This code is working,with the following errors:


I get the data in only one column, instead of six
 
Many thanks!

 



Subject: Re: [Tutor] get columns from txt file
From: dfjennings at gmail.com
Date: Thu, 12 Jul 2012 10:26:30 -0400
CC: tutor at python.org
To: susana_87 at hotmail.com

Oops! Still you forgot to cc: the tutor list. It's really important because if someone (like me, for instance) steers you in the wrong direction, others will jump in with corrections.





On Jul 12, 2012, at 9:48 AM, susana moreno colomer wrote:


Hi!
Many thanks!


You're welcome. I see that your code is improving :>)




Still I get an error: AttributeError: '_cvs.writer' object has no atribute 'append'. 

If you would like a response to that error, please send the exact code you tried which didn't work and the error message by copying/pasting. (See, I'm pretty sure that you typed in the error above since you misspelled attribute as 'atribute'.)




I wanted to solve it with the Def function.

And that's why you should send the original code and error. Now, you have a different problem, but it's a good time to clear up your confusion. Def is not a function. In fact, I don't know what "Def" is. Instead, the def statement?notice that it's all lower case?defines a function which you can call elsewhere in code.



>>> def call_me():
...     print "inside the call_me function"
... 
>>> call_me()
inside the call_me function


Note, that there is no output from our defining the call_me function; it's only when we call it with the parentheses that the code inside of it is executed. Make sense? However, again, it's not necessary for this script. Once you have the csv_out variable, you should be able to 




Now I am trying this, though I get
 
 
>import os
>import fnmatch
>import csv

>path = '/This is my current directory/'
>csv_out=csv.writer(open('out13.csv', 'wb'), delimiter=' ')
>files=os.listdir(path)
>outfile=open('myfile1', 'w')
>output=[]

>for infile in files:
>      if fnmatch.fnmatch(infile, 'bb_*'):
>      filename= os.path.join(path,infile)
>      f=open(filename, 'r')
  
>      for line in f:
>          b=line.split('\t')
>          output.append(b[5].strip())
>     f.close()
 
>Def csvwriter():       ---->gives me SyntaxError: invalid syntax
>        excelfile=csv_out
>        a=excelfile.append(output)
>        c=excelfile.write(a)
>        return c
 
 
I am trying with this  because the atribute 'writerows' didn't work.


A different error. There are lots of reasons it might not have worked. Wish you had sent us the code for that one :<( We can't solve it without all the info.



Is there another way to write in csv files?


Absolutely. At the interactive prompt, dir() shows all the attributes of an object to find out your options:


>>> csv_out=csv.writer(open('out13.csv', 'wb'), dialect='excel')
>>> dir(csv_out)
['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'dialect', 'writerow', 'writerows']


See the writerow and writerows? Those look promising. Alternatively, while it takes a while to learn to read the python documentation, it really is your friend:


http://docs.python.org/library/csv.html


If you'd like further examples, Doug Hellman often provides a great resource with his Python Module of the Week (PYMTOW) series:


http://www.doughellmann.com/PyMOTW/csv/





I don't know what you mean with specify dialect


Something like this:


csv_out = csv.writer(fileobject, dialect='excel')


Take care,
Don
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/f6f8a978/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: myprogram.txt
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/f6f8a978/attachment.txt>

From dfjennings at gmail.com  Thu Jul 12 17:07:53 2012
From: dfjennings at gmail.com (Don Jennings)
Date: Thu, 12 Jul 2012 11:07:53 -0400
Subject: [Tutor] get columns from txt file
In-Reply-To: <BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>
Message-ID: <A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>


On Jul 12, 2012, at 10:55 AM, susana moreno colomer wrote:

> Hi!
>  
> I have a group of files in a directory:
> bb_1.txt
> bb_2.txt
> bb_3.txt
> bb_4.txt
> bb_5.txt
> bb_6.txt
> ss_1.txt
> 
> I want to extract from  files whose names start with  bb_   column number 5 to an excel file. I have 6  bb_  files, therefore I want to get 6 columns (5th column from each file)
> This code is working,with the following errors:
> 
> I get the data in only one column, instead of six

Great! It sounds like you're almost there. Where's the code which works except that it puts the data all in one column?

Take care,
Don
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/6f65481d/attachment.html>

From susana_87 at hotmail.com  Thu Jul 12 17:10:21 2012
From: susana_87 at hotmail.com (susana moreno colomer)
Date: Thu, 12 Jul 2012 17:10:21 +0200
Subject: [Tutor] get columns from txt file
In-Reply-To: <A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
Message-ID: <BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>


Hi!
It is attached on the email, called myprogram.txt
Thank you!
 



Subject: Re: [Tutor] get columns from txt file
From: dfjennings at gmail.com
Date: Thu, 12 Jul 2012 11:07:53 -0400
CC: tutor at python.org
To: susana_87 at hotmail.com




On Jul 12, 2012, at 10:55 AM, susana moreno colomer wrote:


Hi!
 
I have a group of files in a directory:
bb_1.txt
bb_2.txt
bb_3.txt
bb_4.txt
bb_5.txt
bb_6.txt
ss_1.txt

I want to extract from  files whose names start with  bb_   column number 5 to an excel file. I have 6  bb_  files, therefore I want to get 6 columns (5th column from each file)
This code is working,with the following errors:



I get the data in only one column, instead of six

Great! It sounds like you're almost there. Where's the code which works except that it puts the data all in one column?


Take care,
Don 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/225c9488/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: myprogram.txt
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/225c9488/attachment-0001.txt>

From dfjennings at gmail.com  Thu Jul 12 17:43:02 2012
From: dfjennings at gmail.com (Don Jennings)
Date: Thu, 12 Jul 2012 11:43:02 -0400
Subject: [Tutor] get columns from txt file
In-Reply-To: <BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>
Message-ID: <E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>


On Jul 12, 2012, at 11:10 AM, susana moreno colomer wrote:

> Hi!
> It is attached on the email, called myprogram.txt

and, here are the contents of that file:

> #! /usr/bin/env python
> 
> 
> import os
> import fnmatch
> import csv
> 
> 
> path = '//../my_working_folder/'
> csv_out=csv.writer(open('out14.csv', 'wb'), delimiter=' ')
> files=os.listdir(path)
> 
> outfile=open('myfile1', 'w')

Here you've opened a file called "outfile" which you never use! So, of course it's blank. Just remove this line.

> output=[]
> 
> 
> for infile in files:
          columnData = []
> 	if fnmatch.fnmatch(infile, 'bb_*'):
> 		filename= os.path.join(path,infile)
> 		f=open(filename, 'r')
> 		
> 		for line in f:
> 	
> 			b=line.split('\t')
                          # remove the next line
> 			output.append(b[5].strip())
                          # instead, append to columnData list
                          columnData.append(b[5].strip())
> 		f.close()

                  # now append all of that data as a list to the output
                  output.append(columnData)

  # now, as Kent said, create "a list of row lists from the list of column lists
  # if any of the column lists are too short they will be padded with None"

  rows = map(None, *output)

  # now, write those rows to your file
  csv_out.writerows(rows)

> 
> ########This gives me a single column (I want 6, since I have 6 bb_files:
> 
> csv_out.writerows(output)

One of the things you missed in Kent's code is that the output is a list of lists. So, for each file you need a list which you then append to the output list. I've inserted the appropriate code above and you should have better luck if you copy and paste carefully.

> 
> 
> ########with this I get excel whitesheet
> 
> 
> def csvwriter():	
> 	excelfile=csv_out
> 	a=excelfile.append(output)
> 	c=excelfile.write(a)
> 	return c

Right! As I explained in an earlier email, you never call the function csvwriter; you merely define it. The file is created when you opened it earlier for writing on line 10 of your program, but you never write anything to it.

If you have the time and inclination, I recommend you work through one of the fine tutorials for python. It really is a great language and this is a **great** community of folks for people like me who are learning the language still.

Take care,
Don



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/6c1fb434/attachment.html>

From susana_87 at hotmail.com  Thu Jul 12 17:45:31 2012
From: susana_87 at hotmail.com (susana moreno colomer)
Date: Thu, 12 Jul 2012 17:45:31 +0200
Subject: [Tutor] extracting a column from many files
Message-ID: <BAY164-W220FC709FD7EF5AC71B78F84D00@phx.gbl>


Hi!
 
I have a group of files in a directory:
bb_1.txt
bb_2.txt
bb_3.txt
bb_4.txt
bb_5.txt
bb_6.txt
ss_1.txt

I want to extract from  files whose names start with  bb_   column number 5 and 6. I want to extract all columns number 5 to one file, and all tcolumns number6 to another file. 
I was following this example http://mail.python.org/pipermail/tutor/2009-February/067400.html , and I created my own code wich gives me the following errors:
 

This code gives me 2 files:
                ro with 2 lines
                bf emty
 
I attached my code, with two options (though I was trying wich much more)
Could be awesome If I extrace them in an excel file, Any suggestion????
 
Many thanks!!!
Susana 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/985b56d1/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: multifiles.txt
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/985b56d1/attachment.txt>

From susana_87 at hotmail.com  Thu Jul 12 18:06:23 2012
From: susana_87 at hotmail.com (susana moreno colomer)
Date: Thu, 12 Jul 2012 18:06:23 +0200
Subject: [Tutor] get columns from txt file
In-Reply-To: <E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>,
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>
Message-ID: <BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>



Hi!
This code is working fine!
The only one little thing is that the 6 columns appear together in one column, what means, in eac cell I get 6 numbers. How can I get tit in 6 excel columns?
Many thanks, 
Susana



Subject: Re: [Tutor] get columns from txt file
From: dfjennings at gmail.com
Date: Thu, 12 Jul 2012 11:43:02 -0400
CC: tutor at python.org
To: susana_87 at hotmail.com




On Jul 12, 2012, at 11:10 AM, susana moreno colomer wrote:


Hi!
It is attached on the email, called myprogram.txt

and, here are the contents of that file:





#! /usr/bin/env python




import os
import fnmatch
import csv




path = '//../my_working_folder/'
csv_out=csv.writer(open('out14.csv', 'wb'), delimiter=' ')
files=os.listdir(path)


outfile=open('myfile1', 'w')

Here you've opened a file called "outfile" which you never use! So, of course it's blank. Just remove this line.



output=[]




for infile in files:          columnData = []


if fnmatch.fnmatch(infile, 'bb_*'):
filename= os.path.join(path,infile)
f=open(filename, 'r')

for line in f:

b=line.split('\t')                          # remove the next line


output.append(b[5].strip())                          # instead, append to columnData list
                          columnData.append(b[5].strip())


f.close()

                  # now append all of that data as a list to the output
                  output.append(columnData)


  # now, as Kent said, create "a list of row lists from the list of column lists
  # if any of the column lists are too short they will be padded with None"


  rows = map(None, *output)


  # now, write those rows to your file
  csv_out.writerows(rows)






########This gives me a single column (I want 6, since I have 6 bb_files:


csv_out.writerows(output)

One of the things you missed in Kent's code is that the output is a list of lists. So, for each file you need a list which you then append to the output list. I've inserted the appropriate code above and you should have better luck if you copy and paste carefully.








########with this I get excel whitesheet




def csvwriter(): 
excelfile=csv_out
a=excelfile.append(output)
c=excelfile.write(a)
return c


Right! As I explained in an earlier email, you never call the function csvwriter; you merely define it. The file is created when you opened it earlier for writing on line 10 of your program, but you never write anything to it.


If you have the time and inclination, I recommend you work through one of the fine tutorials for python. It really is a great language and this is a **great** community of folks for people like me who are learning the language still.


Take care,
Don





 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/b82d4edf/attachment-0001.html>

From dfjennings at gmail.com  Thu Jul 12 18:14:38 2012
From: dfjennings at gmail.com (Don Jennings)
Date: Thu, 12 Jul 2012 12:14:38 -0400
Subject: [Tutor] get columns from txt file
In-Reply-To: <BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>,
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>
Message-ID: <0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>


On Jul 12, 2012, at 12:06 PM, susana moreno colomer wrote:

> 
> Hi!
> This code is working fine!
> The only one little thing is that the 6 columns appear together in one column, what means, in eac cell I get 6 numbers. How can I get tit in 6 excel columns?

Programming is hard, so don't feel bad that I'm having to tell you again to specify the dialect as "excel" or "excel-tab":

csv_out=csv.writer(open('out14.csv', 'wb'), dialect='excel')

Take care,
Don
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/59cc62f0/attachment.html>

From ramit.prasad at jpmorgan.com  Thu Jul 12 18:18:33 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 12 Jul 2012 16:18:33 +0000
Subject: [Tutor] get columns from txt file
In-Reply-To: <0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>,
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>
	<0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741655451B@SCACMX008.exchad.jpmchase.net>

>> Hi!
>> This code is working fine!
>> The only one little thing is that the 6 columns appear together?in one column, what means, in eac cell I get 6 numbers. How can I get tit in 6 excel columns?

> Programming is hard, so don't feel bad that I'm having to tell you again to specify the dialect as "excel" or >"excel-tab":

> csv_out=csv.writer(open('out14.csv', 'wb'), dialect='excel')

The default dialect is excel but it is important to also open
the file using Excel as it may default to a text editor.

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From emile at fenx.com  Thu Jul 12 18:26:17 2012
From: emile at fenx.com (Emile van Sebille)
Date: Thu, 12 Jul 2012 09:26:17 -0700
Subject: [Tutor] get columns from txt file
In-Reply-To: <BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>,
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>
Message-ID: <jtmtu7$p61$1@dough.gmane.org>

On 7/12/2012 9:06 AM susana moreno colomer said...
>
> Hi!
> This code is working fine!
> The only one little thing is that the 6 columns appear together in one
> column, what means, in eac cell I get 6 numbers. How can I get tit in 6
> excel columns?


You're creating a list (output) with only one column in it, so you only 
get one column in excel.

Try making these chages (untested):


outfile=open('myfile1', 'w')
cols=[]   #  this will get one list (column) per found file

for infile in files:
     if fnmatch.fnmatch(infile, 'bb_*'):
         thiscol = []  # we're starting a new file so start a new column
         filename= os.path.join(path,infile)
         f=open(filename, 'r')
         for line in f:
             b=line.split('\t')
             thiscol.append(b[5].strip())  # append each col value
         f.close()
     cols.append(thiscol)  # store this cols results

output = zip(cols) # zip transposes the cols to rows


HTH,

Emile



From emknowles at gmail.com  Thu Jul 12 18:29:35 2012
From: emknowles at gmail.com (Emma Knowles)
Date: Thu, 12 Jul 2012 12:29:35 -0400
Subject: [Tutor] Script to collect values from .csv
Message-ID: <CAM=fxFbfX1t3tDN7dceF+ZmNsDjJUBrgZnJJ5cQZtJR=B3J+DA@mail.gmail.com>

Hi all,


I have a very large .csv (correlationfile, which is 16 million lines long)
which I want to split into smaller .csvs. The smaller csvs should be
created be searching for a value and printing any line which contains that
value - all these values are contained in another .csv (vertexfile). I
think that I have an indentation problem or have made a mistake with my
loops because I only get data in one of the output .csvs (outputfile) which
is for the first one of the values. The other .csvs are empty.


Can somebody help me please?


Thanks so much!


Emma


import os

path = os.getcwd()

vertexfile = open(os.path.join(path,'vertices1.csv'),'r')

correlationfile = open(os.path.join(path,'practice.csv'),'r')

x = ''

for v in vertexfile:

    vs = v.replace('\n','')

    outputfile = open(os.path.join(path,vs+'.csv'),'w')

    for c in correlationfile:

        cs = c.replace('\n','').split(',')

        if vs == cs[0]: print vs

     outputfile.write(x)

outputfile.close()
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/e0337d79/attachment.html>

From ramit.prasad at jpmorgan.com  Thu Jul 12 19:07:03 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 12 Jul 2012 17:07:03 +0000
Subject: [Tutor] Script to collect values from .csv
In-Reply-To: <CAM=fxFbfX1t3tDN7dceF+ZmNsDjJUBrgZnJJ5cQZtJR=B3J+DA@mail.gmail.com>
References: <CAM=fxFbfX1t3tDN7dceF+ZmNsDjJUBrgZnJJ5cQZtJR=B3J+DA@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416554633@SCACMX008.exchad.jpmchase.net>

> I have a very large .csv (correlationfile, which is 16 million lines long)
> which I want to split into smaller .csvs. The smaller csvs should be created
> be searching for a value and printing any line which contains that value - all
> these values are contained in another .csv (vertexfile). I think that I have
> an indentation problem or have made a mistake with my loops because I only get
> data in one of the output .csvs (outputfile) which is for the first one of the
> values. The other .csvs are empty.
> 
> Can somebody help me please?
> 
> Thanks so much!
> 
> Emma
> 
> import os
> path = os.getcwd()
> x = ''
> for v in vertexfile:
>     vs = v.replace('\n','')
>     outputfile = open(os.path.join(path,vs+'.csv'),'w')
>     for c in correlationfile:
>         cs = c.replace('\n','').split(',')
>         if vs == cs[0]: print vs
>     outputfile.write(x)
> outputfile.close()

Indent the outputfile.close()  to be inside the for loop.
That should fix your problem.

I would recommend working with csv module instead. No need
to worry about replacing new lines or if a comma is contained 
inside your data. Note, when using the csv module open the 
files as 'rb' and 'wb'.


Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From bayespokerguy at gmail.com  Thu Jul 12 20:03:42 2012
From: bayespokerguy at gmail.com (Fred G)
Date: Thu, 12 Jul 2012 14:03:42 -0400
Subject: [Tutor] Accessing a Website
Message-ID: <CAMg+7maS9_AtfVBb4g3QY+04AUYOcFGVVO3F6fid8w2cqdtmHA@mail.gmail.com>

Hi--

My pseudocode is the following

new_dictionary = []
for name in file:
 #1) log into university account
 #2) go to website with data
 #3) type in search box: name
 #4) click search
 #5) if name is exact match with name of one of the hits:
    line.find("Code Number")
    #6) remove the number directly after "Code Number: " and stop at the
next space
new_dictionary[name] = Code Number

With the exception of step 6, I'm not quite sure how to do this in Python.
 Is it very complicated to write a script that logs onto a website that
requires a user name and password that I have, and then repeatedly enters
names and gets their associated id's that we want?  I used to work at a
cancer lab where we decided we couldn't do this kind of thing to search
PubMed, and that a human would be more accurate even though our criteria
was simply (is there survival data?).  I don't think that this has to be
the case here, but would greatly appreciate any guidance.

Thanks so much.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120712/d69df69c/attachment.html>

From joel.goldstick at gmail.com  Thu Jul 12 20:42:38 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Thu, 12 Jul 2012 14:42:38 -0400
Subject: [Tutor] Accessing a Website
In-Reply-To: <CAMg+7maS9_AtfVBb4g3QY+04AUYOcFGVVO3F6fid8w2cqdtmHA@mail.gmail.com>
References: <CAMg+7maS9_AtfVBb4g3QY+04AUYOcFGVVO3F6fid8w2cqdtmHA@mail.gmail.com>
Message-ID: <CAPM-O+wZdT5hwqqEGEJf81e4iXmms-L15y6Q+QKQH0Z5bUbtSA@mail.gmail.com>

On Thu, Jul 12, 2012 at 2:03 PM, Fred G <bayespokerguy at gmail.com> wrote:
> Hi--
>
> My pseudocode is the following
>
> new_dictionary = []
> for name in file:
>  #1) log into university account
>  #2) go to website with data
>  #3) type in search box: name
>  #4) click search
>  #5) if name is exact match with name of one of the hits:
>     line.find("Code Number")
>     #6) remove the number directly after "Code Number: " and stop at the
> next space
> new_dictionary[name] = Code Number
>
> With the exception of step 6, I'm not quite sure how to do this in Python.
> Is it very complicated to write a script that logs onto a website that
> requires a user name and password that I have, and then repeatedly enters
> names and gets their associated id's that we want?  I used to work at a
> cancer lab where we decided we couldn't do this kind of thing to search
> PubMed, and that a human would be more accurate even though our criteria was
> simply (is there survival data?).  I don't think that this has to be the
> case here, but would greatly appreciate any guidance.
>
> Thanks so much.
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
There are a couple of modules (urllib, urllib2) in python and another
one that I like called requests that let your program access a
website.  (http://docs.python-requests.org/en/latest/index.html)
You can read the examples and see if this will help you on your way.
Also, go to the webpage with the search box, and when you enter a
search term and submit it, see what the url looks like after
submitting.  If it is a 'get' request, your search parameter will be
at the tail of the url.  If it is, you can create those urls in your
code and request the results (with requests module).
There is a great module called Beautiful Soup (use version 4) that can
help you parse through your results


-- 
Joel Goldstick

From ramit.prasad at jpmorgan.com  Thu Jul 12 21:06:09 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 12 Jul 2012 19:06:09 +0000
Subject: [Tutor] Accessing a Website
In-Reply-To: <CAPM-O+wZdT5hwqqEGEJf81e4iXmms-L15y6Q+QKQH0Z5bUbtSA@mail.gmail.com>
References: <CAMg+7maS9_AtfVBb4g3QY+04AUYOcFGVVO3F6fid8w2cqdtmHA@mail.gmail.com>
	<CAPM-O+wZdT5hwqqEGEJf81e4iXmms-L15y6Q+QKQH0Z5bUbtSA@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416554874@SCACMX008.exchad.jpmchase.net>

> > My pseudocode is the following
> >
> > new_dictionary = []
> > for name in file:
> >  #1) log into university account
> >  #2) go to website with data
> >  #3) type in search box: name
> >  #4) click search
> >  #5) if name is exact match with name of one of the hits:
> >     line.find("Code Number")
> >     #6) remove the number directly after "Code Number: " and stop at the
> > next space
> > new_dictionary[name] = Code Number
> >
> > With the exception of step 6, I'm not quite sure how to do this in Python.
> > Is it very complicated to write a script that logs onto a website that
> > requires a user name and password that I have, and then repeatedly enters
> > names and gets their associated id's that we want?  I used to work at a
> > cancer lab where we decided we couldn't do this kind of thing to search
> > PubMed, and that a human would be more accurate even though our criteria was
> > simply (is there survival data?).  I don't think that this has to be the
> > case here, but would greatly appreciate any guidance.
> >

> There are a couple of modules (urllib, urllib2) in python and another
> one that I like called requests that let your program access a
> website.  (http://docs.python-requests.org/en/latest/index.html)
> You can read the examples and see if this will help you on your way.
> Also, go to the webpage with the search box, and when you enter a
> search term and submit it, see what the url looks like after
> submitting.  If it is a 'get' request, your search parameter will be
> at the tail of the url.  If it is, you can create those urls in your
> code and request the results (with requests module).
> There is a great module called Beautiful Soup (use version 4) that can
> help you parse through your results
>

You can do that, but it is going to be difficult and I am not even sure
how you would pass things like POST arguments in this manner.

Hopefully this gives you some ideas: 
http://www.akasig.org/2004/12/29/web-scraping-with-python-part-1-crawling/
(more concisely http://www.akasig.org/2004/09/03/web-scraping-with-python/ )


or http://stackoverflow.com/questions/2081586/web-scraping-with-python


You would be better off asking this question on the main python group 
as this group is a much smaller group with the focus of teaching Python.
That group is much larger and more likely to be able help.




Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--


> -----Original Message-----
> From: tutor-bounces+ramit.prasad=jpmorgan.com at python.org [mailto:tutor-
> bounces+ramit.prasad=jpmorgan.com at python.org] On Behalf Of Joel Goldstick
> Sent: Thursday, July 12, 2012 1:43 PM
> To: Fred G
> Cc: tutor at python.org
> Subject: Re: [Tutor] Accessing a Website
> 
> On Thu, Jul 12, 2012 at 2:03 PM, Fred G <bayespokerguy at gmail.com> wrote:
> > Hi--
> >
> > Thanks so much.
> >
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > http://mail.python.org/mailman/listinfo/tutor
> > 
> 
> --
> Joel Goldstick
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From breamoreboy at yahoo.co.uk  Thu Jul 12 21:49:27 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 12 Jul 2012 20:49:27 +0100
Subject: [Tutor] get columns from txt file
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF4741655451B@SCACMX008.exchad.jpmchase.net>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>,
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>
	<0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741655451B@SCACMX008.exchad.jpmchase.net>
Message-ID: <jtn9l5$r4s$1@dough.gmane.org>

Sorry if this shows up twice

On 12/07/2012 17:18, Prasad, Ramit wrote:
>
>> csv_out=csv.writer(open('out14.csv', 'wb'), dialect='excel')
>
> The default dialect is excel but it is important to also open
> the file using Excel as it may default to a text editor.

In the context we're talking about here this makes no sense to me so can 
you please explain.


>
> Ramit
>


-- 
Cheers.

Mark Lawrence.




From d at davea.name  Thu Jul 12 23:27:05 2012
From: d at davea.name (Dave Angel)
Date: Thu, 12 Jul 2012 17:27:05 -0400
Subject: [Tutor] Script to collect values from .csv
In-Reply-To: <CAM=fxFbfX1t3tDN7dceF+ZmNsDjJUBrgZnJJ5cQZtJR=B3J+DA@mail.gmail.com>
References: <CAM=fxFbfX1t3tDN7dceF+ZmNsDjJUBrgZnJJ5cQZtJR=B3J+DA@mail.gmail.com>
Message-ID: <4FFF4129.60102@davea.name>

On 07/12/2012 12:29 PM, Emma Knowles wrote:
> Hi all,
>
>
> I have a very large .csv (correlationfile, which is 16 million lines
> long) which I want to split into smaller .csvs. The smaller csvs
> should be created be searching for a value and printing any line
> which contains that value - all these values are contained in another
> .csv (vertexfile). I think that I have an indentation problem or have
> made a mistake with my loops because I only get data in one of the
> output .csvs (outputfile) which is for the first one of the values.
> The other .csvs are empty.
>
>
> Can somebody help me please?
>
>
> Thanks so much!
>
>
> Emma
>
>
> import os
>
> path = os.getcwd()
>
> vertexfile = open(os.path.join(path,'vertices1.csv'),'r')
>
> correlationfile = open(os.path.join(path,'practice.csv'),'r')
>
> x = ''
>
> for v in vertexfile:
>
> vs = v.replace('\n','')
>
> outputfile = open(os.path.join(path,vs+'.csv'),'w')
>
> for c in correlationfile:
>
> cs = c.replace('\n','').split(',')
>
> if vs == cs[0]: print vs
>
> outputfile.write(x)
>
> outputfile.close()
>
>

Welcome to the Python list.

There are a number of problems with your script.  I suggest writing
simpler functions, rather than trying to accomplish the whole thing in
one double loop.  I also suggest testing the pieces, instead of trying
to observe behavior of the whole thing.

It's a bit hard to observe indentation since you sent it as an html
message.  Please use plain text messages.  I can find out what you
really intended by looking at the raw message, but that shouldn't be
necessary, and not everyone can do that.

I'm amazed that you see any data in the output files, since you only
write null strings to it.  Variable x is "" and doesn't change, as far
as I can see.

You say vertexfile is a csv, but I don't see any effort to parse it. 
You just assume that each line is the key value.

In the loop  "for c in correlationfile"  you don't write to any file. 
All you do is print the key for each line which "matches" it.  Instead
of printing, you probably meant to do a write(), and instead of writing
vs you probably meant to write c.

As Prasad said, your close is indented wrong.  But that's not causing
your symptoms, as CPython will close your file for you when you open
another one and bind it to outputfile, next time through the loop. 
Still, it's useful to get it right, because the problems can be subtle.

I second the advice to use the csv module.  That's what it's there for. 
But if you know there are no embedded commas in the zeroth field, it'll
probably work okay without.


-- 

DaveA


From ramit.prasad at jpmorgan.com  Thu Jul 12 23:28:40 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 12 Jul 2012 21:28:40 +0000
Subject: [Tutor] get columns from txt file
In-Reply-To: <jtn9l5$r4s$1@dough.gmane.org>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>,
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>
	<0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741655451B@SCACMX008.exchad.jpmchase.net>
	<jtn9l5$r4s$1@dough.gmane.org>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416554B76@SCACMX008.exchad.jpmchase.net>

> >> csv_out=csv.writer(open('out14.csv', 'wb'), dialect='excel')
> >
> > The default dialect is excel but it is important to also open
> > the file using Excel as it may default to a text editor.
> 
> In the context we're talking about here this makes no sense to me so can
> you please explain.
>

Sorry, I think it was something that the OP said in a different email
that led me to believe that maybe she was not opening it in excel.
So I was pointing out that dialect defaults to excel to Don (although, 
explicitly setting it never hurt), and then making sure Susana
opened the resulting file in excel.

Not sure what I was thinking looking back...

Ramit

--
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From steve at pearwood.info  Fri Jul 13 02:27:23 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 13 Jul 2012 10:27:23 +1000
Subject: [Tutor] Accessing a Website
In-Reply-To: <CAMg+7maS9_AtfVBb4g3QY+04AUYOcFGVVO3F6fid8w2cqdtmHA@mail.gmail.com>
References: <CAMg+7maS9_AtfVBb4g3QY+04AUYOcFGVVO3F6fid8w2cqdtmHA@mail.gmail.com>
Message-ID: <4FFF6B6B.4090906@pearwood.info>

Fred G wrote:

> With the exception of step 6, I'm not quite sure how to do this in Python.
>  Is it very complicated to write a script that logs onto a website that
> requires a user name and password that I have, and then repeatedly enters
> names and gets their associated id's that we want?


Python comes with some libraries for downloading web resources, including web 
pages. But if you have to interactive with the web page, such as entering 
names into a search field, your best bet is the third-party library mechanize. 
I have never used it, but I have never heard anything but good things about it.

http://pypi.python.org/pypi/mechanize/

http://www.ibm.com/developerworks/linux/library/l-python-mechanize-beautiful-soup/index.html


> I used to work at a
> cancer lab where we decided we couldn't do this kind of thing to search
> PubMed, and that a human would be more accurate even though our criteria
> was simply (is there survival data?).  I don't think that this has to be
> the case here, but would greatly appreciate any guidance.


In general, web-scraping is fraught with problems. Web sites are written by 
ignorant code-monkeys who can barely spell HTML, or worse, too-clever-by-far 
web designers who write too-clever, subtly broken code that only works with 
Internet Explorer (and if you are lucky, Firefox). Or they stick everything in 
Javascript, or worse, Flash.

And often the web server tries to prevent automated tools from fetching 
information. Or there may be legal barriers, where something which is 
perfectly legal if *you* do it becomes (allegedly) illegal if an automated 
script does it.

So I can perfectly understand why a conservative, risk-adverse university 
might prefer to have a human being mechanically fetch the data.

And yet, with work it is possible to code around nearly all these issues. 
Using tools like mechanize and BeautifulSoup, faking the user-agent string, 
and a few other techniques, most non-Flash non-Javascript sites can be 
successfully web-scraped. Even the legal issue can be coded around by adding 
some human interaction to the script, so that it is not an *automated* script, 
while still keeping most of the benefits of automated scraping.

Don't abuse the privilege:

- obey robots.txt
- obey the site's terms and conditions
- obey copyright law
- make a temporary cache of pages you need to re-visit[1]
- give real human visitors priority
- limit your download rate to something reasonable
- pause between requests so you aren't hitting the server at an
   unreasonable rate
- in general, don't be a dick and disrupt the normal working of the
   server or website with your script.




[1]  I am aware of the irony that this is theoretically forbidden by 
copyright. Nevertheless, it is the right thing to do, both technically and 
ethically.



-- 
Steven


From susana_87 at hotmail.com  Fri Jul 13 08:36:27 2012
From: susana_87 at hotmail.com (susana moreno colomer)
Date: Fri, 13 Jul 2012 08:36:27 +0200
Subject: [Tutor] get columns from txt file
In-Reply-To: <0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>,
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>,
	<0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>
Message-ID: <BAY164-W26FA97246ADB83596DF5D284D70@phx.gbl>


Hi!
I am trying this, but still I get 6 numbers per cell. The only one difference is that I get a comma between numbers instead an space. 
I am opening the document also with excel
Many thanks, 
Susana
 



Subject: Re: [Tutor] get columns from txt file
From: dfjennings at gmail.com
Date: Thu, 12 Jul 2012 12:14:38 -0400
CC: tutor at python.org
To: susana_87 at hotmail.com




On Jul 12, 2012, at 12:06 PM, susana moreno colomer wrote:



Hi!
This code is working fine!
The only one little thing is that the 6 columns appear together in one column, what means, in eac cell I get 6 numbers. How can I get tit in 6 excel columns?


Programming is hard, so don't feel bad that I'm having to tell you again to specify the dialect as "excel" or "excel-tab":


csv_out=csv.writer(open('out14.csv', 'wb'), dialect='excel')


Take care,
Don 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120713/865fb723/attachment.html>

From marc.tompkins at gmail.com  Fri Jul 13 09:28:07 2012
From: marc.tompkins at gmail.com (Marc Tompkins)
Date: Fri, 13 Jul 2012 00:28:07 -0700
Subject: [Tutor] get columns from txt file
In-Reply-To: <BAY164-W26FA97246ADB83596DF5D284D70@phx.gbl>
References: <mailman.5674.1342016475.4696.tutor@python.org>
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>
	<0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>
	<BAY164-W26FA97246ADB83596DF5D284D70@phx.gbl>
Message-ID: <CAKK8jXaVMhnW0eZnjNtS=n+UgFheKYw0CJMNHE53Q38wzwH-MQ@mail.gmail.com>

On Thu, Jul 12, 2012 at 11:36 PM, susana moreno colomer <
susana_87 at hotmail.com> wrote:

>  Hi!
> I am trying this, but still I get 6 numbers per cell. The only one
> difference is that I get a comma between numbers instead an space.
> I am opening the document also with excel
> Many thanks,
> Susana
>

CSV stands for Comma Separated Values.  Those commas are the separators
between cells.  Your current problem is that a CSV file is just a regular
text file (that happens to have a lot of commas in it), and Excel is trying
to read it as a normal text file.

CSV is about the simplest way ever invented to store tabular data in a
file, but there are some complicating factors.  The most obvious is: what
happens if the data you want to store in your file actually contains commas
(e.g. address fields with "city, state zip", or numbers with thousands
separators, etc.)  One way around the problem is to put quotes around
fields, and then separate the fields with commas (but then, what if your
data contains quotes?); another is to separate the fields with tab
characters instead of commas (technically this isn't really a CSV file
anymore, but the acronym TSV never caught on.)

Excel's native flavor* of CSV is the oldest, simplest, and stupidest of all
- just commas between fields, and newlines between records:
1, 2, 3, 4, 5
a, b, c, d, e

Quotes-and-commas style:
"1", "2", "3", "4,000,000", 5
"a", "b", "c", "Dammit, Janet", "e"

Tab-separated (well, you'll just have to imagine; I don't feel like
reconfiguring my text editor):
1       2       3      4      5
a      b      cde     fghi    j

and a bunch of others I can't think of right now.
* Note: Excel will happily import a quotes-and-commas CSV file and display
it normally - but if you export it to CSV, it will revert to the dumb
bare-commas format.

>From the Python csv module docs:

> To make it easier to specify the format of input and output records,
> specific formatting parameters are grouped together into dialects. A
> dialect is a subclass of the Dialect<http://docs.python.org/library/csv.html#csv.Dialect>class having a set of specific methods and a single
> validate() method.
>

So you can specify which dialect you want to read or write, and/or you can
specify which delimiter(s) you want to use.

Hope that helps...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120713/d0613946/attachment-0001.html>

From susana_87 at hotmail.com  Fri Jul 13 11:20:33 2012
From: susana_87 at hotmail.com (susana moreno colomer)
Date: Fri, 13 Jul 2012 11:20:33 +0200
Subject: [Tutor] extracting a column from many files
In-Reply-To: <BAY164-W220FC709FD7EF5AC71B78F84D00@phx.gbl>
References: <BAY164-W220FC709FD7EF5AC71B78F84D00@phx.gbl>
Message-ID: <BAY164-W5443F6B26F2D1FA38DF6C784D70@phx.gbl>



 



From: susana_87 at hotmail.com
To: tutor at python.org
Date: Thu, 12 Jul 2012 17:45:31 +0200
Subject: [Tutor] extracting a column from many files




Hi!
 
I want to extract from certain kind of files  column number 5 and 6. I want to extract acolumns number 5 to one file, and anumber6 to another file. 
I was following this example http://mail.python.org/pipermail/tutor/2009-February/067400.html , and under my understanding I created my own code wich gives me the following errors:
 


This code gives me 2 files:                ro with 2 lines
                bf empty
 
I attached my code, with two options (though I was trying wich much more)
Could be awesome If I extrace them in an excel file, Any suggestion????
 
Many thanks!!!
Susana

_______________________________________________ Tutor maillist - Tutor at python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120713/f29f1769/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: multifiles.txt
URL: <http://mail.python.org/pipermail/tutor/attachments/20120713/f29f1769/attachment.txt>

From marc.tompkins at gmail.com  Fri Jul 13 11:21:40 2012
From: marc.tompkins at gmail.com (Marc Tompkins)
Date: Fri, 13 Jul 2012 02:21:40 -0700
Subject: [Tutor] get columns from txt file
In-Reply-To: <BAY164-W37B868E410D945F2F39A2884D70@phx.gbl>
References: <mailman.5674.1342016475.4696.tutor@python.org>
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>
	<0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>
	<BAY164-W26FA97246ADB83596DF5D284D70@phx.gbl>
	<CAKK8jXaVMhnW0eZnjNtS=n+UgFheKYw0CJMNHE53Q38wzwH-MQ@mail.gmail.com>
	<BAY164-W37B868E410D945F2F39A2884D70@phx.gbl>
Message-ID: <CAKK8jXZdu3Ra2VvqAADiqs72mnmrSMgFVtTnRYpy3ci84vduMA@mail.gmail.com>

Reply to the group, please!

On Fri, Jul 13, 2012 at 2:09 AM, susana moreno colomer <
susana_87 at hotmail.com> wrote:

>
> Hi!
> I am sorry, but still I don't get it!
> I am trying with this (the code is attached)
>
> csv_out=csv.writer(open('out1.csv', 'wb'), delimiter=' ',
> quoting=csv.QUOTE_ALL, dialect='excel')
> and
> csv_out=csv.writer(open('out1.csv', 'wb'), dialect='excel-tab')
>
> When I open out1 with text editor, I get 6 columns, but when I open it
> with excel I get one column (6 numbers on each cell, how can I separate
> it it???)
>
> The files from wich I get the columns are txt files.
>
> Many thanks!!
>

It sounds more like an Excel import problem than a Python output problem at
this point, but it's hard to tell.

Is the data you're dealing with private/confidential?  If not, could you
attach your output CSV file?  A quick look could take the place of hours of
guesswork.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120713/8e77d96b/attachment.html>

From joel.goldstick at gmail.com  Fri Jul 13 14:44:55 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Fri, 13 Jul 2012 08:44:55 -0400
Subject: [Tutor] get columns from txt file
In-Reply-To: <CAKK8jXZdu3Ra2VvqAADiqs72mnmrSMgFVtTnRYpy3ci84vduMA@mail.gmail.com>
References: <mailman.5674.1342016475.4696.tutor@python.org>
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>
	<0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>
	<BAY164-W26FA97246ADB83596DF5D284D70@phx.gbl>
	<CAKK8jXaVMhnW0eZnjNtS=n+UgFheKYw0CJMNHE53Q38wzwH-MQ@mail.gmail.com>
	<BAY164-W37B868E410D945F2F39A2884D70@phx.gbl>
	<CAKK8jXZdu3Ra2VvqAADiqs72mnmrSMgFVtTnRYpy3ci84vduMA@mail.gmail.com>
Message-ID: <CAPM-O+wcHfHPW0oNqzO4Lnt7x2yCV98mRFsFzD8GJQUDuGmR0g@mail.gmail.com>

On Fri, Jul 13, 2012 at 5:21 AM, Marc Tompkins <marc.tompkins at gmail.com> wrote:
> Reply to the group, please!
>
> On Fri, Jul 13, 2012 at 2:09 AM, susana moreno colomer
> <susana_87 at hotmail.com> wrote:
>>
>>
>> Hi!
>> I am sorry, but still I don't get it!
>> I am trying with this (the code is attached)
>>
>> csv_out=csv.writer(open('out1.csv', 'wb'), delimiter=' ',
>> quoting=csv.QUOTE_ALL, dialect='excel')
>> and
>> csv_out=csv.writer(open('out1.csv', 'wb'), dialect='excel-tab')
>>
You need to refer to the spec:  csv.writer(csvfile[,
dialect='excel'][, fmtparam])
(http://docs.python.org/library/csv.html#csv.writer)

so i would use:
    csv_out = csv.writer(open('out1.csv', 'wb'), dialect='excel')

This will give you output separated by commas and (if I am not
mistaken) is will only quote values that are strings.


>> When I open out1 with text editor, I get 6 columns, but when I open it
>> with excel I get one column (6 numbers on each cell, how can I separate it
>> it???)
>>
>> The files from wich I get the columns are txt files.
>>
>> Many thanks!!
>
>
> It sounds more like an Excel import problem than a Python output problem at
> this point, but it's hard to tell.
>
> Is the data you're dealing with private/confidential?  If not, could you
> attach your output CSV file?  A quick look could take the place of hours of
> guesswork.
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
I agree with Marc's comments.  I would use the comma separated value
version of the output file, not the tabbed version.

So if your file looks like this in a text editor:

1,2,3,4,5,6

it will put each value in successive columns.


-- 
Joel Goldstick

From bgailer at gmail.com  Fri Jul 13 17:55:00 2012
From: bgailer at gmail.com (bob gailer)
Date: Fri, 13 Jul 2012 11:55:00 -0400
Subject: [Tutor] extracting a column from many files
In-Reply-To: <BAY164-W5443F6B26F2D1FA38DF6C784D70@phx.gbl>
References: <BAY164-W220FC709FD7EF5AC71B78F84D00@phx.gbl>
	<BAY164-W5443F6B26F2D1FA38DF6C784D70@phx.gbl>
Message-ID: <500044D4.1050701@gmail.com>

On 7/13/2012 5:20 AM, susana moreno colomer wrote:

[snip]

Please post the entire traceback.

You did not tell us what error or where in option 1.


-- 
Bob Gailer
919-636-4239
Chapel Hill NC


From Craig.Prinn at bhemail.com  Thu Jul  5 18:30:27 2012
From: Craig.Prinn at bhemail.com (Prinn, Craig)
Date: Thu, 5 Jul 2012 12:30:27 -0400
Subject: [Tutor] converting EBCIDIC to ASCII
Message-ID: <6B49A56A6E493F4EBA255F6F197F070F081B5058DC@BBH-MAIL1.bbh.priv>

I am trying to convert an EBCIDIC file to ASCII, when the records are fixed length I can convert it fine, I have some files that are coming in as variable length records, is there a way to convert the file in Python? I tried using no length but then it just reads in to a fixed buffer size and I can't seem to break the records properly

Craig Prinn
Manager, Data Management
Phone: 919-767-6640
Cell: 410-320-9962
Address: Bell and Howell
              3600 Clipper Mill Road
              Suite 404
              Baltimore MD 21211

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120705/f83d6c62/attachment-0001.html>

From redstone-cold at 163.com  Sat Jul  7 06:59:26 2012
From: redstone-cold at 163.com (redstone-cold)
Date: Sat, 7 Jul 2012 12:59:26 +0800 (CST)
Subject: [Tutor] =?gbk?q?What=A1=AFs_the_differences_between_these_two__pi?=
	=?gbk?q?eces_of_code_=3F?=
Message-ID: <4a1b894d.4107.1385fce854b.Coremail.redstone-cold@163.com>

What?s the differences between these two  pieces of code ?

(1)

for i in range(1, 7):

print(2 * i, end='   ')

 

 

(2)

for i in range(1, 7):

    print(2 * i, end='   ')

print()

 

 

when executed both  respectively in Python shell ,I  get  the same effect . Who can tell me why  ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120707/855bd6ce/attachment-0001.html>

From blindmaildrop at gmail.com  Thu Jul 12 01:46:22 2012
From: blindmaildrop at gmail.com (blindmaildrop)
Date: Wed, 11 Jul 2012 16:46:22 -0700
Subject: [Tutor] starting to learn
Message-ID: <CAAB-_MhOZkgh6--xTJiYcPgCznNAtoPHpurkh52KBAD36gG=6w@mail.gmail.com>

Hello!

I am just starting to learn python, having signed up for a class on it in
University.  The last time I programmed anything was in the long long long
ago of BASIC and (well since I spent time doing other things) I missed the
development boat.

So starting from scratch, how-to?

thanks

--c.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120711/3ad9fede/attachment-0001.html>

From susana_87 at hotmail.com  Fri Jul 13 11:09:31 2012
From: susana_87 at hotmail.com (susana moreno colomer)
Date: Fri, 13 Jul 2012 11:09:31 +0200
Subject: [Tutor] get columns from txt file
In-Reply-To: <CAKK8jXaVMhnW0eZnjNtS=n+UgFheKYw0CJMNHE53Q38wzwH-MQ@mail.gmail.com>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>,
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>,
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>,
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>,
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>,
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>,
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>,
	<0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>,
	<BAY164-W26FA97246ADB83596DF5D284D70@phx.gbl>,
	<CAKK8jXaVMhnW0eZnjNtS=n+UgFheKYw0CJMNHE53Q38wzwH-MQ@mail.gmail.com>
Message-ID: <BAY164-W37B868E410D945F2F39A2884D70@phx.gbl>



Hi!
I am sorry, but still I don't get it!
I am trying with this (the code is attached)
 
csv_out=csv.writer(open('out1.csv', 'wb'), delimiter=' ', quoting=csv.QUOTE_ALL, dialect='excel')
and 
csv_out=csv.writer(open('out1.csv', 'wb'), dialect='excel-tab')
 
When I open out1 with text editor, I get 6 columns, but when I open it with excel I get one column (6 numbers on each cell, how can I separate it it???)
 
The files from wich I get the columns are txt files.
 
Many thanks!!



Date: Fri, 13 Jul 2012 00:28:07 -0700
Subject: Re: [Tutor] get columns from txt file
From: marc.tompkins at gmail.com
To: susana_87 at hotmail.com
CC: dfjennings at gmail.com; tutor at python.org


On Thu, Jul 12, 2012 at 11:36 PM, susana moreno colomer <susana_87 at hotmail.com> wrote:



Hi!
I am trying this, but still I get 6 numbers per cell. The only one difference is that I get a comma between numbers instead an space. 
I am opening the document also with excel
Many thanks, 
Susana


CSV stands for Comma Separated Values.  Those commas are the separators between cells.  Your current problem is that a CSV file is just a regular text file (that happens to have a lot of commas in it), and Excel is trying to read it as a normal text file. 

CSV is about the simplest way ever invented to store tabular data in a file, but there are some complicating factors.  The most obvious is: what happens if the data you want to store in your file actually contains commas (e.g. address fields with "city, state zip", or numbers with thousands separators, etc.)  One way around the problem is to put quotes around fields, and then separate the fields with commas (but then, what if your data contains quotes?); another is to separate the fields with tab characters instead of commas (technically this isn't really a CSV file anymore, but the acronym TSV never caught on.)  

Excel's native flavor* of CSV is the oldest, simplest, and stupidest of all - just commas between fields, and newlines between records:
1, 2, 3, 4, 5
a, b, c, d, e

Quotes-and-commas style:
"1", "2", "3", "4,000,000", 5
"a", "b", "c", "Dammit, Janet", "e"

Tab-separated (well, you'll just have to imagine; I don't feel like reconfiguring my text editor):
1       2       3      4      5
a      b      cde     fghi    j

and a bunch of others I can't think of right now.
* Note: Excel will happily import a quotes-and-commas CSV file and display it normally - but if you export it to CSV, it will revert to the dumb bare-commas format.

>From the Python csv module docs:

To make it easier to specify the format of input and output records, specific formatting parameters are grouped together into dialects. A dialect is a subclass of the Dialect class having a set of specific methods and a single validate() method. 


So you can specify which dialect you want to read or write, and/or you can specify which delimiter(s) you want to use.

Hope that helps... 


 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120713/17620402/attachment.html>

From amonroe at columbus.rr.com  Fri Jul 13 19:44:39 2012
From: amonroe at columbus.rr.com (R. Alan Monroe)
Date: Fri, 13 Jul 2012 13:44:39 -0400
Subject: [Tutor] starting to learn
In-Reply-To: <CAAB-_MhOZkgh6--xTJiYcPgCznNAtoPHpurkh52KBAD36gG=6w@mail.gmail.com>
References: <CAAB-_MhOZkgh6--xTJiYcPgCznNAtoPHpurkh52KBAD36gG=6w@mail.gmail.com>
Message-ID: <729336624.20120713134439@columbus.rr.com>


> So starting from scratch, how-to?

print "hello world"

Alan


From chare at labr.net  Fri Jul 13 19:54:13 2012
From: chare at labr.net (Chris Hare)
Date: Fri, 13 Jul 2012 12:54:13 -0500
Subject: [Tutor]
 =?windows-1252?q?What=92s_the_differences_between_these_t?=
 =?windows-1252?q?wo__pieces_of_code_=3F?=
In-Reply-To: <4a1b894d.4107.1385fce854b.Coremail.redstone-cold@163.com>
References: <4a1b894d.4107.1385fce854b.Coremail.redstone-cold@163.com>
Message-ID: <5B1154FC-ABC3-42B8-BC3B-E5ADC84BABC9@labr.net>


On Jul 6, 2012, at 11:59 PM, redstone-cold wrote:

> What?s the differences between these two  pieces of code ?
> (1)
> for i in range(1, 7):
> print(2 * i, end='   ')
>  
>  
> (2)
> for i in range(1, 7):
>     print(2 * i, end='   ')
> print()
>  
I think they are exactly the same, except the second example will print an extra "blank" line before exiting


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120713/7c61f931/attachment.html>

From ramit.prasad at jpmorgan.com  Fri Jul 13 19:53:51 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Fri, 13 Jul 2012 17:53:51 +0000
Subject: [Tutor] get columns from txt file
In-Reply-To: <BAY164-W37B868E410D945F2F39A2884D70@phx.gbl>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>,
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>,
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>,
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>,
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>,
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>,
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>,
	<0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>,
	<BAY164-W26FA97246ADB83596DF5D284D70@phx.gbl>,
	<CAKK8jXaVMhnW0eZnjNtS=n+UgFheKYw0CJMNHE53Q38wzwH-MQ@mail.gmail.com>
	<BAY164-W37B868E410D945F2F39A2884D70@phx.gbl>
Message-ID: <5B80DD153D7D744689F57F4FB69AF474165562B1@SCACMX008.exchad.jpmchase.net>

Please always reply to the list!

> 
> Hi!
> I am sorry, but still I don't get it!
> I am trying with this (the code is attached)
> 
> csv_out=csv.writer(open('out1.csv', 'wb'), delimiter=' ',
> quoting=csv.QUOTE_ALL, dialect='excel')
> and
> csv_out=csv.writer(open('out1.csv', 'wb'), dialect='excel-tab')
> 
> When I open out1 with text editor, I get 6 columns, but when I open it with
> excel I get one column (6 numbers on each cell, how can I separate it it???)
> 
> The files from wich I get the columns are txt files.
> 
> Many thanks!!

Use
csv_out=csv.writer(open('out1.csv', 'wb'), dialect='excel')
NOT dialect 'excel-tab' or with the delimiter field.

Alternatively try following these steps,
http://www.ehow.com/how_4449036_import-delimited-file-excel.html



Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From d at davea.name  Fri Jul 13 21:17:56 2012
From: d at davea.name (Dave Angel)
Date: Fri, 13 Jul 2012 15:17:56 -0400
Subject: [Tutor]
 =?windows-1252?q?What=92s_the_differences_between_these_t?=
 =?windows-1252?q?wo__pieces_of_code_=3F?=
In-Reply-To: <4a1b894d.4107.1385fce854b.Coremail.redstone-cold@163.com>
References: <4a1b894d.4107.1385fce854b.Coremail.redstone-cold@163.com>
Message-ID: <50007464.909@davea.name>

On 07/07/2012 12:59 AM, redstone-cold wrote:
> What?s the differences between these two  pieces of code ?
>
> (1)
>
> for i in range(1, 7):
>
> print(2 * i, end='   ')
>
>  
>
>  
>
> (2)
>
> for i in range(1, 7):
>
>     print(2 * i, end='   ')
>
> print()
>
>  
>
>  
>
> when executed both  respectively in Python shell ,I  get  the same effect . Who can tell me why  ?
>

The difference is that the first will give a syntax error, and not run.

Once you fix that, the second will print an extra blank line at the end.

-- 

DaveA


From marc.tompkins at gmail.com  Fri Jul 13 21:30:24 2012
From: marc.tompkins at gmail.com (Marc Tompkins)
Date: Fri, 13 Jul 2012 12:30:24 -0700
Subject: [Tutor] converting EBCIDIC to ASCII
In-Reply-To: <6B49A56A6E493F4EBA255F6F197F070F081B5058DC@BBH-MAIL1.bbh.priv>
References: <6B49A56A6E493F4EBA255F6F197F070F081B5058DC@BBH-MAIL1.bbh.priv>
Message-ID: <CAKK8jXZ+AF1LHSrBwhmWJ4wa+6eujhYPD9pux7aaUaGBT30QPQ@mail.gmail.com>

On Thu, Jul 5, 2012 at 9:30 AM, Prinn, Craig <Craig.Prinn at bhemail.com>wrote:

> ** ** ** ** ** **
>
> I am trying to convert an EBCIDIC file to ASCII, when the records are
> fixed length I can convert it fine, I have some files that are coming in as
> variable length records, is there a way to convert the file in Python? I
> tried using no length but then it just reads in to a fixed buffer size and
> I can?t seem to break the records properly
>

I know of only three varieties of variable-length-record files:
-  Delimited - i.e. there's some special character that ends the record,
and (perhaps) a special character that separates fields.  CSV is the
classic example: newlines to separate records, commas to separate fields.

-  Prefixed - there's a previously-agreed schema of record lengths, where
(for example) a record that starts with "A" is 25 characters long; a "B"
record is 136 characters long, etc.

-  Sequential - record types/lengths appear in a previously-agreed order,
such as 25 characters, 136 characters, 45 characters, etc.

For each of these types, the schema may be externally-published, or it may
be encoded in a special record at the beginning of the file - to use an
example near and dear to my own experience, ANSI X12 EDI files all start
with a fixed-length "ISA" record, which among other things contains the
element separator, repetition separator, sub-element separator, and segment
terminator characters in positions 3, 104, 84, and 105.  To read an X12
file, therefore, you read it in - look at positions 3,84, 104, and 105 -
and then use that information to break up the rest of the file into records
and fields.

How you handle variable-length records depends on what kind they are, and
how much you know about them going in.  Python is just a tool for applying
your specialized domain knowledge - by itself, it doesn't know any more
about your particular solution than you do.

If you have more information about the structure of your files, and need
help implementing an algorithm to deal with 'em, let us know!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120713/92559014/attachment.html>

From david at graniteweb.com  Fri Jul 13 21:47:10 2012
From: david at graniteweb.com (David Rock)
Date: Fri, 13 Jul 2012 14:47:10 -0500
Subject: [Tutor] starting to learn
In-Reply-To: <CAAB-_MhOZkgh6--xTJiYcPgCznNAtoPHpurkh52KBAD36gG=6w@mail.gmail.com>
References: <CAAB-_MhOZkgh6--xTJiYcPgCznNAtoPHpurkh52KBAD36gG=6w@mail.gmail.com>
Message-ID: <20120713194710.GK3060@wdfs.gateway.2wire.net>

* blindmaildrop <blindmaildrop at gmail.com> [2012-07-11 16:46]:
> Hello!
> 
> I am just starting to learn python, having signed up for a class on it in
> University.  The last time I programmed anything was in the long long long
> ago of BASIC and (well since I spent time doing other things) I missed the
> development boat.
> 
> So starting from scratch, how-to?

Coming here is a good start.  Welcome!

Start here for beginning ideas:
http://wiki.python.org/moin/BeginnersGuide

You will find a number of good jumping-off points under the "Learning
Python" section.

Come back as you hit things. :-)

-- 
David Rock
david at graniteweb.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/tutor/attachments/20120713/8fd61b58/attachment.pgp>

From breamoreboy at yahoo.co.uk  Fri Jul 13 23:23:02 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 13 Jul 2012 22:23:02 +0100
Subject: [Tutor]
 =?windows-1252?q?What=92s_the_differences_between_these_t?=
 =?windows-1252?q?wo__pieces_of_code_=3F?=
In-Reply-To: <5B1154FC-ABC3-42B8-BC3B-E5ADC84BABC9@labr.net>
References: <4a1b894d.4107.1385fce854b.Coremail.redstone-cold@163.com>
	<5B1154FC-ABC3-42B8-BC3B-E5ADC84BABC9@labr.net>
Message-ID: <jtq3h3$b1l$1@dough.gmane.org>

On 13/07/2012 18:54, Chris Hare wrote:
>
> On Jul 6, 2012, at 11:59 PM, redstone-cold wrote:
>
>> What?s the differences between these two  pieces of code ?
>> (1)
>> for i in range(1, 7):
>> print(2 * i, end='   ')
>>
>>
>> (2)
>> for i in range(1, 7):
>>      print(2 * i, end='   ')
>> print()
>>
> I think they are exactly the same, except the second example will print an extra "blank" line before exiting

Wrong see Dave Angel's reply, remember that indentation counts in Python :-)

>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
-- 
Cheers.

Mark Lawrence.




From marc.tompkins at gmail.com  Sat Jul 14 01:41:59 2012
From: marc.tompkins at gmail.com (Marc Tompkins)
Date: Fri, 13 Jul 2012 16:41:59 -0700
Subject: [Tutor] converting EBCIDIC to ASCII
In-Reply-To: <6B49A56A6E493F4EBA255F6F197F070F081B7E7763@BBH-MAIL1.bbh.priv>
References: <6B49A56A6E493F4EBA255F6F197F070F081B5058DC@BBH-MAIL1.bbh.priv>
	<CAKK8jXZ+AF1LHSrBwhmWJ4wa+6eujhYPD9pux7aaUaGBT30QPQ@mail.gmail.com>
	<6B49A56A6E493F4EBA255F6F197F070F081B7E7763@BBH-MAIL1.bbh.priv>
Message-ID: <CAKK8jXaO+Os9eeJUEFTVOCPeSaJnfuboujDbrV6MdBhwEo-QHQ@mail.gmail.com>

On Fri, Jul 13, 2012 at 1:28 PM, Prinn, Craig <Craig.Prinn at bhemail.com>wrote:

>  The records are coming off of a mainframe, so there probably was a 2
> byte RDW or length indicator at one point. If there is a x0D x0A at the end
> would that work?****
>
> Thanks****
>
> Craig
>

I presume so, but (despite my bloviating about the generalities of
variable-length records) I don't actually know all that much about how
systems that use EBCDIC tend to structure their files (my "big iron" days
were in an HP 3000 shop, which DID use EBCDIC, but that was  22 years ago -
and at the time I was a database-only programmer and didn't need to worry
my little head about actual file I/O.)  By "at the end" do you mean 'at the
end of each record', or 'at the end of the file'?

If you meant 'at the end of each record', then my approach would be:
-  create an empty list called "lines"
-  read in the file (or buffer-sized chunks of it, anyway) - call it inFile
-  create recordBegin and recordEnd pointers, initialized to 0
-  search for x0D x0A (or whatever) in the stream of bytes
-  each time I find it,
   -  set the recordEnd pointer
   -  make a copy of the bytes between recordBegin and recordEnd and append
it to "lines"
   -  copy recordEnd to recordBegin
   -  lather, rinse, repeat
-  at the end, decode each bytestream in "lines"

If you meant 'at the end of the file', then I'm not sure it helps, and I
don't know what you'd need to move forward.

Good luck!

****
>
> ** **
>  ------------------------------
>
> *From:* Marc Tompkins [mailto:marc.tompkins at gmail.com]
> *Sent:* Friday, July 13, 2012 3:30 PM
> *To:* Prinn, Craig
> *Cc:* tutor at python.org
> *Subject:* Re: [Tutor] converting EBCIDIC to ASCII****
>
> ** **
>
> On Thu, Jul 5, 2012 at 9:30 AM, Prinn, Craig <Craig.Prinn at bhemail.com>
> wrote:****
>
> I am trying to convert an EBCIDIC file to ASCII, when the records are
> fixed length I can convert it fine, I have some files that are coming in as
> variable length records, is there a way to convert the file in Python? I
> tried using no length but then it just reads in to a fixed buffer size and
> I can?t seem to break the records properly****
>
> ** **
>
> I know of only three varieties of variable-length-record files:
> -  Delimited - i.e. there's some special character that ends the record,
> and (perhaps) a special character that separates fields.  CSV is the
> classic example: newlines to separate records, commas to separate fields.
>
> -  Prefixed - there's a previously-agreed schema of record lengths, where
> (for example) a record that starts with "A" is 25 characters long; a "B"
> record is 136 characters long, etc.
>
> -  Sequential - record types/lengths appear in a previously-agreed order,
> such as 25 characters, 136 characters, 45 characters, etc.
>
> For each of these types, the schema may be externally-published, or it may
> be encoded in a special record at the beginning of the file - to use an
> example near and dear to my own experience, ANSI X12 EDI files all start
> with a fixed-length "ISA" record, which among other things contains the
> element separator, repetition separator, sub-element separator, and segment
> terminator characters in positions 3, 104, 84, and 105.  To read an X12
> file, therefore, you read it in - look at positions 3,84, 104, and 105 -
> and then use that information to break up the rest of the file into records
> and fields.
>
> How you handle variable-length records depends on what kind they are, and
> how much you know about them going in.  Python is just a tool for applying
> your specialized domain knowledge - by itself, it doesn't know any more
> about your particular solution than you do.
>
> If you have more information about the structure of your files, and need
> help implementing an algorithm to deal with 'em, let us know!****
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120713/3e3e73df/attachment-0001.html>

From steve at pearwood.info  Sat Jul 14 03:41:53 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 14 Jul 2012 11:41:53 +1000
Subject: [Tutor] converting EBCIDIC to ASCII
In-Reply-To: <6B49A56A6E493F4EBA255F6F197F070F081B5058DC@BBH-MAIL1.bbh.priv>
References: <6B49A56A6E493F4EBA255F6F197F070F081B5058DC@BBH-MAIL1.bbh.priv>
Message-ID: <5000CE61.7040902@pearwood.info>

Prinn, Craig wrote:
> I am trying to convert an EBCIDIC file to ASCII, when the records are fixed
> length I can convert it fine, I have some files that are coming in as
> variable length records, is there a way to convert the file in Python? I
> tried using no length but then it just reads in to a fixed buffer size and
> I can't seem to break the records properly


I'm afraid that I have no idea what you mean here. What are you actually 
doing? What does "tried using no length" mean?

Converting from one encoding to another should have nothing to do with whether 
they are fixed-length records, variable-length records, or free-form text. 
First you read the file as bytes, then use the encoding to convert to text, 
then process the file however you like.

Using Python 3, I prepared an EBCIDIC file. If I open it in binary mode, you 
get the raw bytes, which are a mess:

py> raw = open('/home/steve/ebcidic.text', 'rb').read()
py> print(raw)
b'\xe3\x88\x89\xa2@\x89\xa2@\\\xa2\x96\x94\x85\\@\xe3 ...

For brevity, I truncated the output.

But if you open in text mode, and set the encoding correctly, Python 
automatically converts the bytes into text according to the rules of EBCIDIC:


py> text = open('/home/steve/ebcidic.text', 'r', encoding='cp500').read()
py> print(text)
This is *some* Text containing "punctuation" & other things(!) which
may{?} NOT be the +++same+++ when encoded into ASCII|EBCIDIC.


This is especially useful if you need to process the file line by line. Simple 
open the file with the right encoding, then loop over the file as normal.


f = open('/home/steve/ebcidic.text', 'r', encoding='cp500')
for line in f:
     print(line)


In this case, I used IBM's standard EBCIDIC encoding for Western Europe. 
Python knows about some others, see the documentation for the codecs module 
for the list.

http://docs.python.org/library/codecs.html
http://docs.python.org/py3k/library/codecs.html

Once you have the text, you can then treat it as fixed width, variable width, 
or whatever else you might have.


Python 2 is a little trickier. You can manually decode the bytes:

# not tested
text = open('/home/steve/ebcidic.text', 'rb').read().decode('cp500')

or you can use the codecs manual to get very close to the same functionality 
as Python 3:

# also untested
import codecs
f = codecs.open('/home/steve/ebcidic.text', 'r', encoding='cp500')
for line in f:
     print line



-- 
Steven


From eryksun at gmail.com  Sat Jul 14 04:58:05 2012
From: eryksun at gmail.com (eryksun)
Date: Fri, 13 Jul 2012 22:58:05 -0400
Subject: [Tutor]
	=?utf-8?q?What=E2=80=99s_the_differences_between_these_tw?=
	=?utf-8?q?o_pieces_of_code_=3F?=
In-Reply-To: <4a1b894d.4107.1385fce854b.Coremail.redstone-cold@163.com>
References: <4a1b894d.4107.1385fce854b.Coremail.redstone-cold@163.com>
Message-ID: <CACL+1atju4+KSsrmkDrQYqZektkgzoXx2kqhtaVvBvqm81E2Kw@mail.gmail.com>

On Sat, Jul 7, 2012 at 12:59 AM, redstone-cold <redstone-cold at 163.com> wrote:
>
> for i in range(1, 7):
>
> print(2 * i, end='   ')

The HTML attachment was scrubbed:
http://mail.python.org/pipermail/tutor/attachments/20120707/855bd6ce/attachment-0001.html

Here's the troublesome 'indent':

<P style="TEXT-INDENT: 20.25pt; MARGIN: 0cm 0cm 0pt" class="MsoNormal">

Please post using plain text and indent with spaces.

From sntshkmr60 at gmail.com  Sat Jul 14 05:46:56 2012
From: sntshkmr60 at gmail.com (Santosh Kumar)
Date: Sat, 14 Jul 2012 09:16:56 +0530
Subject: [Tutor] How to get two user inputs in same prompt seperated by a
	special character?
Message-ID: <CAE7MaQaXwLGjVy9r3_ycxXvarMdbfFhqjcPdwS3+RACbcUnmgA@mail.gmail.com>

Like in my script:

#!/usr/bin/env python
#-*- coding:utf-8 -*-

print "You will be prompted to enter your height in feet and inches."
height_in_feet = input("How much feet are you long: ")
remaining_inches = input("How much inches remained? ")

inches_in_feet = height_in_feet * 12

total_height_in_inches = inches_in_feet + remaining_inches

print "You are %d inches long." % total_height_in_inches


Here I want something like:
Enter you height in format like 5' 10"

From mariocatch at gmail.com  Sat Jul 14 06:02:04 2012
From: mariocatch at gmail.com (mariocatch)
Date: Sat, 14 Jul 2012 00:02:04 -0400
Subject: [Tutor] How to get two user inputs in same prompt seperated by
 a special character?
In-Reply-To: <CAE7MaQaXwLGjVy9r3_ycxXvarMdbfFhqjcPdwS3+RACbcUnmgA@mail.gmail.com>
References: <CAE7MaQaXwLGjVy9r3_ycxXvarMdbfFhqjcPdwS3+RACbcUnmgA@mail.gmail.com>
Message-ID: <CAA7L5UC4zMY2D6Nj0gfB0CcF0HjsAR9EWyi6RHZiww8Y+Lve6g@mail.gmail.com>

On Fri, Jul 13, 2012 at 11:46 PM, Santosh Kumar <sntshkmr60 at gmail.com>wrote:

> Like in my script:
>
> #!/usr/bin/env python
> #-*- coding:utf-8 -*-
>
> print "You will be prompted to enter your height in feet and inches."
> height_in_feet = input("How much feet are you long: ")
> remaining_inches = input("How much inches remained? ")
>
> inches_in_feet = height_in_feet * 12
>
> total_height_in_inches = inches_in_feet + remaining_inches
>
> print "You are %d inches long." % total_height_in_inches
>
>
> Here I want something like:
> Enter you height in format like 5' 10"
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

Here's one way:

FEET_SEPERATOR = "'"
height = raw_input("Enter your height (ie: 6'1)")
heightSplit = height.split(FEET_SEPERATOR)
heightFeet = int(heightSplit[0])
heightInches = int(heightSplit[1])
inchesFromFeet = heightFeet*12
totalInches = inchesFromFeet + heightInches
print 'You are {0} inches'.format(totalInches)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120714/ed725ad5/attachment.html>

From d at davea.name  Sat Jul 14 13:21:23 2012
From: d at davea.name (Dave Angel)
Date: Sat, 14 Jul 2012 07:21:23 -0400
Subject: [Tutor] get columns from txt file
In-Reply-To: <BAY164-W37B868E410D945F2F39A2884D70@phx.gbl>
References: <mailman.5674.1342016475.4696.tutor@python.org>,
	<467021CD-C94E-402F-90A3-28B8159E5E0D@gmail.com>,
	<BAY164-W630E56AA0700DEE5896EF784D00@phx.gbl>,
	<15D88AAA-0ED3-43BA-BBD9-6F33ED9B53AF@gmail.com>,
	<BAY164-W516C2FC4320B6AB6EACD0184D00@phx.gbl>,
	<EF8A655A-3252-4E08-B5A0-86B96630C520@gmail.com>,
	<BAY164-W52DB091C9398BF6669696E84D00@phx.gbl>,
	<A8D43833-4ABC-4A3D-A7F1-F0A4F22DB966@gmail.com>,
	<BAY164-W4842A0F5F6CDD13473178984D00@phx.gbl>,
	<E09859A1-9080-4CFD-B258-77B16739E4BD@gmail.com>,
	<BAY164-W28927F7BEC299FEF5C1A8084D00@phx.gbl>,
	<0E59D600-D543-4037-BB5F-3539B84F7980@gmail.com>,
	<BAY164-W26FA97246ADB83596DF5D284D70@phx.gbl>,
	<CAKK8jXaVMhnW0eZnjNtS=n+UgFheKYw0CJMNHE53Q38wzwH-MQ@mail.gmail.com>
	<BAY164-W37B868E410D945F2F39A2884D70@phx.gbl>
Message-ID: <50015633.6020700@davea.name>

On 07/13/2012 05:09 AM, susana moreno colomer wrote:
> 
> 
> Hi!
> I am sorry, but still I don't get it!

1) please don't top-post.  Put your new comments AFTER the part you're
quoting.  Or don't bother quoting, if you think your stuff stands alone.

2) please do a Reply-all, or equivalent.  Anyway, make sure the
python-tutor is included in the list of recipients.  Otherwise, you're
just sending private mail.


> I am trying with this (the code is attached)

3) Don't try to send attachments to the list, They don't work
everywhere.  This is a text list, so your messages should be plain text,
not html, and without colors, bold, or attachments.  For example, I
don't see any code.  But it doesn't matter, as more importantly, I don't
see the output of your code.

>  
> csv_out=csv.writer(open('out1.csv', 'wb'), delimiter=' ', quoting=csv.QUOTE_ALL, dialect='excel')
> and 
> csv_out=csv.writer(open('out1.csv', 'wb'), dialect='excel-tab')
>  
> When I open out1 with text editor, I get 6 columns, but when I open it with excel I get one column (6 numbers on each cell, how can I separate it it???)

As Marc requested in another message, please paste the first few lines
of that out1 text file into your reply, so somebody that knows Excel
(better than I) can try to guess why it's not being imported correctly.

>  
> The files from wich I get the columns are txt files.
>  
> Many thanks!!
> 
<SNIP>
-- 

DaveA



From googcheng at gmail.com  Sun Jul 15 03:47:22 2012
From: googcheng at gmail.com (Goog Cheng)
Date: Sun, 15 Jul 2012 09:47:22 +0800
Subject: [Tutor] google search
Message-ID: <5002212A.5000406@gmail.com>

Hi, all!  I wanna get the number of results for a query term from google 
, I tried some methods , xgoogle is the commonest way but still doesn't 
work well (BWT i'm a beginner) , Could you give me some advice or i have 
to simulate the browser(still unskilled) , I'm pressed for time and to 
calculate the PMI .  Hope for help from you!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120715/41822277/attachment.html>

From steve at pearwood.info  Sun Jul 15 04:28:39 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 15 Jul 2012 12:28:39 +1000
Subject: [Tutor] google search
In-Reply-To: <5002212A.5000406@gmail.com>
References: <5002212A.5000406@gmail.com>
Message-ID: <50022AD7.7010500@pearwood.info>

Goog Cheng wrote:
> Hi, all!  I wanna get the number of results for a query term from google 
> , I tried some methods , xgoogle is the commonest way but still doesn't 
> work well (BWT i'm a beginner) , Could you give me some advice 

Yes.

Don't assume we can read your mind and know what you are talking about, 
because we can't.

What methods did you try?

What did they do?

What is "xgoogle"?

What do you mean "doesn't work well"?


> or i have 
> to simulate the browser(still unskilled) , I'm pressed for time and to 
> calculate the PMI .  Hope for help from you!

What is "PMI"?

I'm sorry that you are pressed for time, but that's your problem, not ours.

In English, we have a proverb: "more haste, less speed". It means, if you are 
in a hurry to do something, you should take the time to do it carefully the 
first time, because it is faster to do it carefully *once* and get it right, 
than to do it fast one, two, three, ... times and making mistakes.

If you want to get good answers to your questions quickly, spend the time to 
ask good questions the first time, and don't waste your time and ours with bad 
questions that cannot be answered.

More information here:

http://catb.org/~esr/faqs/smart-questions.html




-- 
Steven

From sntshkmr60 at gmail.com  Sun Jul 15 05:32:55 2012
From: sntshkmr60 at gmail.com (Santosh Kumar)
Date: Sun, 15 Jul 2012 09:02:55 +0530
Subject: [Tutor] How does this tiny script works?
Message-ID: <CAE7MaQZASSa2BLF8dfqVWYk1XvZ7yJPZp4WOf2nHXCT1G4uM3Q@mail.gmail.com>

I am reading How to Think Like a Computer Scientist with Python. There
is a script in the exercise:

if "Ni!":
    print 'We are the Knights who say, "Ni!"'
else:
    print "Stop it! No more of this!"

if 0:
    print "And now for something completely different.."
else:
    print "Whats all this, then?"


The question asks "Explain what happened and why it happened." I my
self is confused because the output is:

We are the Knights who say, "Ni!"
Whats all this, then?

1. I was assuming that in first "if" statement we will get "Stop it!
No more of this!" because we are no setting any input and matching
string from it (in this case "Ni!").
2. I didn't get 2nd part at all.

From alexandre.zani at gmail.com  Sun Jul 15 07:45:05 2012
From: alexandre.zani at gmail.com (Alexandre Zani)
Date: Sat, 14 Jul 2012 22:45:05 -0700
Subject: [Tutor] How does this tiny script works?
In-Reply-To: <CAE7MaQZASSa2BLF8dfqVWYk1XvZ7yJPZp4WOf2nHXCT1G4uM3Q@mail.gmail.com>
References: <CAE7MaQZASSa2BLF8dfqVWYk1XvZ7yJPZp4WOf2nHXCT1G4uM3Q@mail.gmail.com>
Message-ID: <CAJVMporNGJx8gwwFCM_Qe-uAJ3bpSidvxTshDEA-1A+vMOX2nQ@mail.gmail.com>

On Sat, Jul 14, 2012 at 8:32 PM, Santosh Kumar <sntshkmr60 at gmail.com> wrote:
> I am reading How to Think Like a Computer Scientist with Python. There
> is a script in the exercise:
>
> if "Ni!":
>     print 'We are the Knights who say, "Ni!"'
> else:
>     print "Stop it! No more of this!"
>
> if 0:
>     print "And now for something completely different.."
> else:
>     print "Whats all this, then?"
>
>
> The question asks "Explain what happened and why it happened." I my
> self is confused because the output is:
>
> We are the Knights who say, "Ni!"
> Whats all this, then?
>
> 1. I was assuming that in first "if" statement we will get "Stop it!
> No more of this!" because we are no setting any input and matching
> string from it (in this case "Ni!").
> 2. I didn't get 2nd part at all.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

Long story short, it has to do with how things get translated to
booleans. An empty string, 0, None or an empty collection will be
translated to False. Almost everything else is translated to True.

So in the first case, python "translates" if "Ni!" to if True. In the
second case, python "translates" if 0 to if False.

You can see this if you pull up an interpreter and type

>>> bool("Ni!")
True
>>> bool(0)
False

From alan.gauld at btinternet.com  Sun Jul 15 23:53:36 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 15 Jul 2012 22:53:36 +0100
Subject: [Tutor] starting to learn
In-Reply-To: <CAAB-_MhOZkgh6--xTJiYcPgCznNAtoPHpurkh52KBAD36gG=6w@mail.gmail.com>
References: <CAAB-_MhOZkgh6--xTJiYcPgCznNAtoPHpurkh52KBAD36gG=6w@mail.gmail.com>
Message-ID: <jtve50$70a$1@dough.gmane.org>

On 12/07/12 00:46, blindmaildrop wrote:

> in University.  The last time I programmed anything was in the long long
> long ago of BASIC

If you remember any of the BASIC you might find my tutorial useful since 
it contrasts Python with VBScript which is a modern version of BASIC...

It is aimed at complete beginners and aims to get you up to the level 
where you can read any web pages aimed at programmers and understand them.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Mon Jul 16 01:39:39 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 16 Jul 2012 00:39:39 +0100
Subject: [Tutor]
 =?windows-1252?q?What=92s_the_differences_between_these_t?=
 =?windows-1252?q?wo__pieces_of_code_=3F?=
In-Reply-To: <4a1b894d.4107.1385fce854b.Coremail.redstone-cold@163.com>
References: <4a1b894d.4107.1385fce854b.Coremail.redstone-cold@163.com>
Message-ID: <jtvkbr$e57$1@dough.gmane.org>

On 07/07/12 05:59, redstone-cold wrote:
> What?s the differences between these two pieces of code ?

Apart from the obvious extra print statement its hard to tell
because your message is showing up with different indent levels.
In my mail reader both loops were indented, in my quoted reply
neither is... Others seem to see one indented but not the other.

Please stick to plain text in mails containing Python code.
I know that seems to be hard in some mail clients but it
really helps when Python code is involved.

> (1)
>
> for i in range(1, 7):
> print(2 * i, end='')
>
> (2)
>
> for i in range(1, 7):
> print(2 * i, end='')
> print()
>
> when executed both respectively in Python shell ,I get the same effect .

What do you mean by the Python Shell?
Are you running a vanilla interpreter within a console or some form of 
IDE? Some IDEs swallow characters making it harder to tell exactly what 
is going on.

If it is a native prompt which OS are you using? That too can make a 
difference.

> Who can tell me why?

Once we know what you are typing and which environment you are using we 
probably can explain whatever you are seeing...


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From jugurtha.hadjar at gmail.com  Mon Jul 16 10:18:56 2012
From: jugurtha.hadjar at gmail.com (Jugurtha Hadjar)
Date: Mon, 16 Jul 2012 09:18:56 +0100
Subject: [Tutor] starting to learn
In-Reply-To: <CAAB-_MhOZkgh6--xTJiYcPgCznNAtoPHpurkh52KBAD36gG=6w@mail.gmail.com>
References: <CAAB-_MhOZkgh6--xTJiYcPgCznNAtoPHpurkh52KBAD36gG=6w@mail.gmail.com>
Message-ID: <5003CE70.4010303@gmail.com>

On 07/12/2012 12:46 AM, blindmaildrop wrote:
> Hello!
>
> I am just starting to learn python, having signed up for a class on it in
> University.  The last time I programmed anything was in the long long long
> ago of BASIC and (well since I spent time doing other things) I missed the
> development boat.
>
> So starting from scratch, how-to?
>
> thanks


Hello,

There are different types of people, each with their own learning style.

Personally, I found it hard to read books, because most of them write 40 
pages before getting to the point. Then another 40 pages before getting 
to the other point. If you programmed anything before, then you probably 
need to type code and get familiar with the syntax on a superficial 
level first (like copying examples given in the Index when programming 
BASIC).

There is a book, available for free which might help you do that: 
Directly to the point. Typing code and a lot of it. It's by Zed Shaw, 
and it's called "Learn Python The Hard Way". (If you're anything like 
me, you probably will like the title and like the book).


http://learnpythonthehardway.org


This book aims to "get you going", beating inertia. Then you can really 
start to "learn" Python. Read more code, read other books, etc..


-- 
~Jugurtha Hadjar,



From alan.gauld at btinternet.com  Mon Jul 16 11:51:25 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 16 Jul 2012 10:51:25 +0100
Subject: [Tutor] How to get two user inputs in same prompt seperated by
 a special character?
In-Reply-To: <CAE7MaQaXwLGjVy9r3_ycxXvarMdbfFhqjcPdwS3+RACbcUnmgA@mail.gmail.com>
References: <CAE7MaQaXwLGjVy9r3_ycxXvarMdbfFhqjcPdwS3+RACbcUnmgA@mail.gmail.com>
Message-ID: <ju0o6t$kjs$1@dough.gmane.org>

On 14/07/12 04:46, Santosh Kumar wrote:

> height_in_feet = input("How much feet are you long: ")
> remaining_inches = input("How much inches remained? ")
>
> Here I want something like:
> Enter you height in format like 5' 10"

Mario has given a partial answer for this specific case although he 
missed stripping of the inches symbol " at the end.

However in the general case what you want to do is quite hard (and
the reason most UIs ask for separate values) You need to parse the 
character string input identifying any markers (like ' and ") and 
extract the values. There are several ways to do this:

1) use split() based on the markers, works well for simple cases,
less so for symbols like lbs, ft, etc

2) read the characters in one by one and process them using a state 
machine. Works well for simple cases with multiple char markers (or 
multiple variations of the same (like ' or ft or feet) and for formal 
specification of input format (typically where the input is normally a 
file and exceptionally a human).

3) use regular expressions to extract recognised patterns, works better 
for unstructured input.

3) use a formal parser to extract the values, most powerful but usually 
most work too.

But it is generally easier (for the programmer) and safer (better
data quality) to ask for each value explicitly.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From Steve.Flynn at capita.co.uk  Mon Jul 16 11:41:25 2012
From: Steve.Flynn at capita.co.uk (Flynn, Stephen (L & P - IT))
Date: Mon, 16 Jul 2012 10:41:25 +0100
Subject: [Tutor] converting EBCIDIC to ASCII
In-Reply-To: <6B49A56A6E493F4EBA255F6F197F070F081B5058DC@BBH-MAIL1.bbh.priv>
References: <6B49A56A6E493F4EBA255F6F197F070F081B5058DC@BBH-MAIL1.bbh.priv>
Message-ID: <D35D4ADAE41B404A9EB381E750C1A5A5025FC074@CAPPRWMMBX14.central.ad.capita.co.uk>

I am trying to convert an EBCIDIC file to ASCII, when the records are
fixed length I can convert it fine, I have some files that are coming in
as variable length records, is there a way to convert the file in
Python? I tried using no length but then it just reads in to a fixed
buffer size and I can't seem to break the records properly

Hi Craig,

	You might find it easier to pass the records through iconv if
you're on a Linux/Unix box and convert to ISO8859 from IBM037 (or
whatever codepage your ENCDIC files are in). There are versions of this
gnu software for Windows too, if that's your platform - it's trivial to
use. Shout if you need a hand.

	Saying that, you'll almost certainly find that the 4 byte RDW
has been stripped from the file when it was sent to you, so you're not
being given any information to determine the length of each variable
length record.

	Quick way to check - open the EBCDIC file up in an hex editor (I
use HxD (from http://mh-nexus.de/en/hxd/as it will happily run in EBCDIC
mode). If you can't see 4 bytes at the start of each record, then you're
in trouble as you have no way of determining the record length, without
the copybook for the file on the mainframe.

S.


This email and any attachment to it are confidential.  Unless you are the intended recipient, you may not use, copy or disclose either the message or any information contained in the message. If you are not the intended recipient, you should delete this email and notify the sender immediately.

Any views or opinions expressed in this email are those of the sender only, unless otherwise stated.  All copyright in any Capita material in this email is reserved.

All emails, incoming and outgoing, may be recorded by Capita and monitored for legitimate business purposes. 

Capita exclude all liability for any loss or damage arising or resulting from the receipt, use or transmission of this email to the fullest extent permitted by law.

From Steve.Flynn at capita.co.uk  Mon Jul 16 11:47:56 2012
From: Steve.Flynn at capita.co.uk (Flynn, Stephen (L & P - IT))
Date: Mon, 16 Jul 2012 10:47:56 +0100
Subject: [Tutor] converting EBCIDIC to ASCII
In-Reply-To: <5000CE61.7040902@pearwood.info>
References: <6B49A56A6E493F4EBA255F6F197F070F081B5058DC@BBH-MAIL1.bbh.priv>
	<5000CE61.7040902@pearwood.info>
Message-ID: <D35D4ADAE41B404A9EB381E750C1A5A5025FC081@CAPPRWMMBX14.central.ad.capita.co.uk>

> -----Original Message-----
> From: tutor-bounces+steve.flynn=capita.co.uk at python.org [mailto:tutor-
> bounces+steve.flynn=capita.co.uk at python.org] On Behalf Of Steven
D'Aprano
> Sent: Saturday, July 14, 2012 2:42 AM
> To: tutor at python.org
> Subject: Re: [Tutor] converting EBCIDIC to ASCII
> 
> Prinn, Craig wrote:
> > I am trying to convert an EBCIDIC file to ASCII, when the records
are
> fixed
> > length I can convert it fine, I have some files that are coming in
as
> > variable length records, is there a way to convert the file in
Python? I
> > tried using no length but then it just reads in to a fixed buffer
size
> and
> > I can't seem to break the records properly
> 
> 
> I'm afraid that I have no idea what you mean here. What are you
actually
> doing? What does "tried using no length" mean?

The conversion to ASCII from EBCDIC is only going to get Craig so far -
depending on how the sender transferred the files to him, there's a very
good chance that the 4 byte RDW (Record Descriptor Word) has been
stripped off the start of each record.

This 4 byte RDW should indicate that the next N bytes belong to this
record. Without it, you have no way of determining how long the current
record should be and thus where the next RDW should be. This makes
finding the start and end of records tricky to say the least.

I've written to Craig off list with some info as it's not particularly
relevant to Python, other than letting python do the work of iconv.

S.



This email and any attachment to it are confidential.  Unless you are the intended recipient, you may not use, copy or disclose either the message or any information contained in the message. If you are not the intended recipient, you should delete this email and notify the sender immediately.

Any views or opinions expressed in this email are those of the sender only, unless otherwise stated.  All copyright in any Capita material in this email is reserved.

All emails, incoming and outgoing, may be recorded by Capita and monitored for legitimate business purposes. 

Capita exclude all liability for any loss or damage arising or resulting from the receipt, use or transmission of this email to the fullest extent permitted by law.

From bala.biophysics at gmail.com  Mon Jul 16 13:00:40 2012
From: bala.biophysics at gmail.com (Bala subramanian)
Date: Mon, 16 Jul 2012 13:00:40 +0200
Subject: [Tutor] passing global variable as argument.
Message-ID: <CA+WPOVMmvcq6p=92w1q7ieadiLzJvrxdxxrkDyNAakWfNrrDiA@mail.gmail.com>

Friends,
I want to define a function that can populate an array by taking its name
(which is defined globally). I defined two empty arrays as follows and a
function that can populate the array.

REF_F1=np.array([])
REF_F2=np.array([])

# populating the given array
def ref(ln,REF_F1):
    global REF_F1
    REF_F1=np.zeros((ln,3),dtype='float32')
    for i in range(ln):
        for j in range(3):
           REF_F1x[i,j]=resid[Index[i]].cent()[j]

ref(ln, REF_F2)

In this case, when i pass REF_F2 as argument, the fn. always populates
array REF_F1. I also tried something like the following
*def ref(ln,x=REF_F1)* and then calling as *ref(ln,x=REF_F2)*. The result
is the same. Could someone please give me hint on how pass global variables
as arguments.

Thanks,
Bala








-- 
C. Balasubramanian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120716/89337a9f/attachment.html>

From d at davea.name  Mon Jul 16 13:31:37 2012
From: d at davea.name (Dave Angel)
Date: Mon, 16 Jul 2012 07:31:37 -0400
Subject: [Tutor] passing global variable as argument.
In-Reply-To: <CA+WPOVMmvcq6p=92w1q7ieadiLzJvrxdxxrkDyNAakWfNrrDiA@mail.gmail.com>
References: <CA+WPOVMmvcq6p=92w1q7ieadiLzJvrxdxxrkDyNAakWfNrrDiA@mail.gmail.com>
Message-ID: <5003FB99.7080506@davea.name>

On 07/16/2012 07:00 AM, Bala subramanian wrote:
> Friends,
> I want to define a function that can populate an array by taking its name
> (which is defined globally). I defined two empty arrays as follows and a
> function that can populate the array.
>
> REF_F1=np.array([])
> REF_F2=np.array([])
>
> # populating the given array
> def ref(ln,REF_F1):
>     global REF_F1
>     REF_F1=np.zeros((ln,3),dtype='float32')
>     for i in range(ln):
>         for j in range(3):
>            REF_F1x[i,j]=resid[Index[i]].cent()[j]
>
> ref(ln, REF_F2)
>
> In this case, when i pass REF_F2 as argument, the fn. always populates
> array REF_F1. I also tried something like the following
> *def ref(ln,x=REF_F1)* and then calling as *ref(ln,x=REF_F2)*. The result
> is the same. Could someone please give me hint on how pass global variables
> as arguments.
>
> Thanks,
> Bala
>
>
>

When you use a non-standard library, you really ought to mention what it
is.  I'm assuming np refers to numpy, which i'm not familiar with.  So I
won't address any particular quirks of that library.

When you use the global statement in a function, you are saying you want
a particular global value, so the argument detail is irrelevant.  Seems
to me it should have been a compile error.

Nix the global statement if you really want to pass it as an argument. 
And for readability, call the formal parameter something else, and do
not capitalize it.  it's not read-only; you're intending to change it.

Since I don't know numpy, I'll have to use a built-in type for example. 
I choose list.

(untested)
myglobal1 = [3,4]
myglobal2 = [5,6,7]

def  myfunc(ln, target):
    while target: target.pop()
    for i in xrange ln:
             target.append(3*i)
    for j in xrange(ln):
             target[j] += 5

myfunc(10, myglobal1)
myfunc(5, myglobal2)

A key point is I do not bind my local variable target to a new object. 
If I do, I'll have no effect on the global object.  So I cannot use the
following line:
     target = []

I also notice you use 'ref' several places.  Perhaps you're under the
mistaken belief that Python does references.

-- 

DaveA


From wayne at waynewerner.com  Mon Jul 16 13:37:05 2012
From: wayne at waynewerner.com (Wayne Werner)
Date: Mon, 16 Jul 2012 06:37:05 -0500 (CDT)
Subject: [Tutor] passing global variable as argument.
In-Reply-To: <CA+WPOVMmvcq6p=92w1q7ieadiLzJvrxdxxrkDyNAakWfNrrDiA@mail.gmail.com>
References: <CA+WPOVMmvcq6p=92w1q7ieadiLzJvrxdxxrkDyNAakWfNrrDiA@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1207160624470.3779@gilgamesh>

On Mon, 16 Jul 2012, Bala subramanian wrote:

> Friends,
> I want to define a function that can populate an array by taking its name
> (which is defined globally).

Are you sure this is what you really want to do? I've noticed that many times
that I want to do something, but only because I don't understand how to do it a
better way.

> I defined two empty arrays as follows and a
> function that can populate the array.
>
> REF_F1=np.array([])
> REF_F2=np.array([])
>
> # populating the given array
> def ref(ln,REF_F1):
>    global REF_F1

My command of the global syntax is a little weak, but I'm going to say what is
probably happening with this line is that you are telling python you now want
to refer to the REF_F1 that lives in the global namespace, not the one that was
passed in to your function.

>    REF_F1=np.zeros((ln,3),dtype='float32')

This line here tells me that what I mentioned first is correct. You don't care
*what* the array is before hand, because you're actually zeroing the array.
Read the conclusion below.

>    for i in range(ln):
>        for j in range(3):
>           REF_F1x[i,j]=resid[Index[i]].cent()[j]

I'm not sure what `resid` or `Index` are. Also, REF_F1x wasn't defined anywhere
so you're probably getting a name error here. When sending code, especially
misbehaving code, the best thing to do is provide a SMALL, but complete
program that can be run on its own, as long as one has the dependencies.

>
> ref(ln, REF_F2)
>
> In this case, when i pass REF_F2 as argument, the fn. always populates
> array REF_F1. I also tried something like the following
> *def ref(ln,x=REF_F1)* and then calling as *ref(ln,x=REF_F2)*. The result
> is the same. Could someone please give me hint on how pass global variables
> as arguments.

First off, the whole point of global variables are so that you don't have to
pass them as arguments. This is fine, and there are occasions where you
legitimately want to have a global variable - but this isn't one of them. When
you use globals improperly, they cause maintenance nightmares by making it
really hard to figure out what's going on in your code. At least Python
requires you to explicitly state that you care about some global value with the
global keyword.

But in this case it looks like you don't actually care about the original array
- you're completely replacing the values based on whatever the value of `ln`
   is. So instead you could do something like

     def initialize_array(ln):
         new_array = np.zeros((ln,3),dtype='float32')
         for i in range(ln):
             for j in range(3):
                new_array[i,j]=resid[Index[i]].cent()[j]
         return new_array

And then you would call it like this:

     REF_F2 = ref(ln)

HTH,
Wayne

From chigga101 at gmail.com  Mon Jul 16 14:09:01 2012
From: chigga101 at gmail.com (Matthew Ngaha)
Date: Mon, 16 Jul 2012 13:09:01 +0100
Subject: [Tutor] newbie Questions
Message-ID: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>

Hi all. I'm new to Python and Programming in general. I've started out with
Python for beginners, and so far so good. My friend who i might add, is not
a programmer but has had experience in the world of programming (i dont
know how much but he claims a lot), has told me to forget about Python and
focus on PHP. He knows i studied a HTML and CSS course and told me for the
direction i'm going in, Python is not needed and won't give me all the
options or variety PHP can. Thats he's opinion, i'd rather make my own mind
up, but its lead me to these questions out of curiousity:

a) at this early stage i dont exactly know what web options are:(  but is
Python limited when it comes to matters relating to Web options/developing?
b) Are there better options, or can Python get the job done as good as any?
c) after completing and understanding a beginner's book, would i be at
intermediate level, or still below?
d) Would i need a more advanced tutorial, what do you advise after
finishing a beginners course?

e) And finally, are there other essential things i must learn after Python?
i heard Django is important?
f) is Django the equivelent to PHP's MySql?

You dont have to answer all questions. Just answering one would help me
greatly in my future decisions, as i want to commit fully to the right
programming language.

since this isnt a forum, how can i thank everyone for helping?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120716/3c9ca2d1/attachment.html>

From wprins at gmail.com  Mon Jul 16 14:55:19 2012
From: wprins at gmail.com (Walter Prins)
Date: Mon, 16 Jul 2012 13:55:19 +0100
Subject: [Tutor] newbie Questions
In-Reply-To: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
References: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
Message-ID: <CANLXbfAQTDYLTcDX8ctuB64-euNYzHDfvJ6Z4MjegsxSBSRcyg@mail.gmail.com>

Hi Matthew,

On 16 July 2012 13:09, Matthew Ngaha <chigga101 at gmail.com> wrote:
> Hi all. I'm new to Python and Programming in general. I've started out with
> Python for beginners, and so far so good. My friend who i might add, is not
> a programmer but has had experience in the world of programming (i dont know
> how much but he claims a lot), has told me to forget about Python and focus
> on PHP. He knows i studied a HTML and CSS course and told me for the
> direction i'm going in, Python is not needed and won't give me all the
> options or variety PHP can. Thats he's opinion, i'd rather make my own mind
> up, but its lead me to these questions out of curiousity:

Well firstly you should note you're asking this on a Python mailing
list so you're likely to see some Python bias here. :) That being
said:

> a) at this early stage i dont exactly know what web options are:(  but is
> Python limited when it comes to matters relating to Web options/developing?

It's true that Python is more general than PHP.  So, while PHP
generally is almost exclusively focused on web development, Python is
a far mor generally useful programming language that happens to be
able to used in a web context if so desired.  So it depends on what
you mean by "limited" really.  There are probably more PHP websites
out there than Python based ones.  Does that make Python "more
limited"?  Perhaps not.  You need to probably be more specific.

> b) Are there better options, or can Python get the job done as good as any?

It depends on what the job is.  In general however I'd cautiously say
yes. Python can "get the job done as good as any" in most cases.

> c) after completing and understanding a beginner's book, would i be at
> intermediate level, or still below?

Impossible to say without knowing what book you worked through and testing you.

> d) Would i need a more advanced tutorial, what do you advise after finishing
> a beginners course?

Depends on what you're trying to accomplish.

>  e) And finally, are there other essential things i must learn after Python?

Probably yes.  But this is true of most programming languages...  If
you're going to do web development you will need to learn all there is
to know about how browsers and web servers funtion, how they exchange
requests and responses and everything else related to this (e.g.
cookies etc).  Even if a web framework or language takes care of a lot
of these details for you, you'll still need to get to a point where
you actually understand how everything works together IMHO.

> i heard Django is important?

Django is a web framework for/written in Python.  If you want to do
websites with Python it's a good option, although there are others
such as Turbogears, Web2Py.  See for example: http://is.gd/fo1FZ6

> f) is Django the equivelent to PHP's MySql?

Well, first to note that MySQL doesn't belong to PHP.  MySQL is a
seperate project and is used in many contexts quite apart from PHP.
It can be easily used from Python as well.

Django similarly as metioned is a web framework that can use several
database back-ends, including (but not limited to) MySQL.  For example
it can also use Postgresql, SQLite & Oracle.

>  You dont have to answer all questions. Just answering one would help me
> greatly in my future decisions, as i want to commit fully to the right
> programming language.

"Right" very much depend on your requirements.  That said, I'd prefer
Python over PHP any day.  But then, I would say that. :)

>  since this isnt a forum, how can i thank everyone for helping?

Just say "thank you" when you feel you want to by emailing the list as
you've just done.  :-)

Hope that helps a bit.

Walter

From wprins at gmail.com  Mon Jul 16 15:19:20 2012
From: wprins at gmail.com (Walter Prins)
Date: Mon, 16 Jul 2012 14:19:20 +0100
Subject: [Tutor] newbie Questions
In-Reply-To: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
References: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
Message-ID: <CANLXbfCWbixqb7F=4E8dCtGHeeh19yxZid-EChUY+c7aR6ZGAg@mail.gmail.com>

Hi again Matthew,

I forgot to include the following link which I originally thought to
include, which is one guy's set of (IMHO very cogent) criticisms
against PHP as programming language:  http://is.gd/z1POXC  Hopefully
it gives you something else to think about regarding the PHP vs Python
question apart from just whether doing websites in it is "easy".

Walter

From memilanuk at gmail.com  Mon Jul 16 15:44:40 2012
From: memilanuk at gmail.com (Monte Milanuk)
Date: Mon, 16 Jul 2012 06:44:40 -0700
Subject: [Tutor] newbie Questions
In-Reply-To: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
References: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
Message-ID: <CAC6CachNOAdYNM6sd-ngMnDesBfV3HeVunCbXCGp5kkDtCfpsQ@mail.gmail.com>

Probably the single biggest 'problem' with Python for web development, in
my opinion, is that while a lot of web hosts have all sorts of PHP
templates or frameworks installed and ready for easy deployment... Python
options seem to be a bit sparser.  Individual hosts may vary, but thats the
overall sense of things that I've gotten

Please note I'm not saying that there are fewer Python options overall, or
that its in any way inferior... just a matter of market penetration.  PHP
has been one of the big dogs in open-source web development for a while,
merits or warts aside.  Python might be arguably 'better' in various ways,
but momentum in the market place is hard to ignore.

I'm guessing your friend 'sees' more PHP out there than Python too, hence
his recommendations.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120716/30023633/attachment.html>

From wayne at waynewerner.com  Mon Jul 16 15:54:29 2012
From: wayne at waynewerner.com (Wayne Werner)
Date: Mon, 16 Jul 2012 08:54:29 -0500 (CDT)
Subject: [Tutor] newbie Questions
In-Reply-To: <CANLXbfCWbixqb7F=4E8dCtGHeeh19yxZid-EChUY+c7aR6ZGAg@mail.gmail.com>
References: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
	<CANLXbfCWbixqb7F=4E8dCtGHeeh19yxZid-EChUY+c7aR6ZGAg@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1207160849220.3779@gilgamesh>


On Mon, 16 Jul 2012, Walter Prins wrote:

> Hi again Matthew,
>
> I forgot to include the following link which I originally thought to
> include, which is one guy's set of (IMHO very cogent) criticisms
> against PHP as programming language:  http://is.gd/z1POXC  Hopefully
> it gives you something else to think about regarding the PHP vs Python
> question apart from just whether doing websites in it is "easy".
>

I read that article recently and recommend it as well. I used to
think that PHP was just fine... but since reading that article (and 
several related ones), I've since revised my opinion.

I would now tell someone to learn any language besides php.

-Wayne

From joel.goldstick at gmail.com  Mon Jul 16 16:12:42 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 16 Jul 2012 10:12:42 -0400
Subject: [Tutor] newbie Questions
In-Reply-To: <alpine.DEB.2.02.1207160849220.3779@gilgamesh>
References: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
	<CANLXbfCWbixqb7F=4E8dCtGHeeh19yxZid-EChUY+c7aR6ZGAg@mail.gmail.com>
	<alpine.DEB.2.02.1207160849220.3779@gilgamesh>
Message-ID: <CAPM-O+y8gBvtsdAuQX7p9r9-eamLgYL8Cv1VEsWvi6KAhcVtng@mail.gmail.com>

On Mon, Jul 16, 2012 at 9:54 AM, Wayne Werner <wayne at waynewerner.com> wrote:
>
> On Mon, 16 Jul 2012, Walter Prins wrote:
>
>> Hi again Matthew,
>>
>> I forgot to include the following link which I originally thought to
>> include, which is one guy's set of (IMHO very cogent) criticisms
>> against PHP as programming language:  http://is.gd/z1POXC  Hopefully
>> it gives you something else to think about regarding the PHP vs Python
>> question apart from just whether doing websites in it is "easy".
>>
>
> I read that article recently and recommend it as well. I used to
> think that PHP was just fine... but since reading that article (and several
> related ones), I've since revised my opinion.
>
> I would now tell someone to learn any language besides php.
>
My take on this depends upon the background (and goal) of the new
learner.  In this list, and among people who care about professional
software development I'm sure python would win the question.  But for
someone with a passing curiosity about 'what is programming?', and
'can I make my own website?', he might be better served to learn a
little php coding (along with html and maybe css).  I say this
because, although python is really straightforward to learn as a first
language, or as a language to learn in order to understand computer
science concepts, I think it has a higher barrier to entry than
cobbling together some php and html in a text editor and copying it
into a directory that lets apache serve up php to a web browser.  This
might spur the learner to dig deeper. And that is where python will
prove a better fit.
-- 
Joel Goldstick

From steve at pearwood.info  Mon Jul 16 16:40:51 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 17 Jul 2012 00:40:51 +1000
Subject: [Tutor] newbie Questions
In-Reply-To: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
References: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
Message-ID: <500427F3.7050702@pearwood.info>

Matthew Ngaha wrote:
> Hi all. I'm new to Python and Programming in general. I've started out with
> Python for beginners, and so far so good. My friend who i might add, is not
> a programmer but has had experience in the world of programming (i dont
> know how much but he claims a lot), has told me to forget about Python and
> focus on PHP. He knows i studied a HTML and CSS course and told me for the
> direction i'm going in, Python is not needed and won't give me all the
> options or variety PHP can. Thats he's opinion, i'd rather make my own mind
> up, but its lead me to these questions out of curiousity:

What do you want to do? Do you want to program in a clean, fun language?

Or do you want to find it easy to get a low-paying coding job?

(Admittedly, low-paying is relative -- slinging PHP code may be at the bottom 
of the programming totem pole, but it's better than slinging hamburgers at 
McDonalds.)

There are more PHP jobs available than Python jobs. But there are also more 
PHP developers than Python developers, because every doofus with a copy of 
Idiots Guide To Idiot Programming thinks they can code in PHP :)

Seriously, you will be competing with a million other PHP coders, and ten 
million more in India willing to work for $10 an hour. *And* using a horrible 
language.


> a) at this early stage i dont exactly know what web options are:(  but is
> Python limited when it comes to matters relating to Web options/developing?

Absolutely not. Python has many powerful web development systems:

Django, Zope, Plone, MoinMoin, Trac, Pylons, TurboGears, and my favourite (for 
what little it's worth), CherryPy.

More here: http://wiki.python.org/moin/WebProgramming/

The one limitation is that while nearly all hosting providers supply PHP by 
default, only a few supply Python.


> b) Are there better options, or can Python get the job done as good as any?

The opposite: some systems can get the job done almost as well as Python.


> c) after completing and understanding a beginner's book, would i be at
> intermediate level, or still below?

"If I tie two pieces of short string together, will it be as long as a long 
piece of string?"

:)



> d) Would i need a more advanced tutorial, what do you advise after
> finishing a beginners course?

Tutorials are great, but nothing beats programming. Program program program.

Even if your programs are never finished, just keep coding.


> e) And finally, are there other essential things i must learn after Python?
> i heard Django is important?

Django is one out of many web frameworks. If you don't like Django, there are 
alternatives.


> f) is Django the equivelent to PHP's MySql?

No, and no.

Django is something like an application-builder for the web, using Python.

MySQL is a database which you can use from any programming language, including 
PHP and Python. There are many other databases, ranging from SQLite to 
Oracle's monster database systems. Python can talk to (nearly) them all.


> You dont have to answer all questions. Just answering one would help me
> greatly in my future decisions, as i want to commit fully to the right
> programming language.

You will be a better programmer if you expose yourself to multiple different 
languages and programming paradigms. It's like being a cook: the world-class 
chefs learn many different styles of cooking, and are equally at home cooking 
Italian, Chinese or Cajun, and can invent new fusion recipes that combine the 
best of different cultures. Average chefs can grill a pretty good steak, and 
absolutely nothing else.


> since this isnt a forum, how can i thank everyone for helping?

Dibs on your first born child!!!

But seriously, just remember that this is a community staffed by volunteers. 
Some day, you can give back to the community by helping others just as you 
were helped.



-- 
Steven

From susana_87 at hotmail.com  Mon Jul 16 16:58:38 2012
From: susana_87 at hotmail.com (susana moreno colomer)
Date: Mon, 16 Jul 2012 16:58:38 +0200
Subject: [Tutor] Extracting columns from many files to different files
Message-ID: <BAY164-W34CD4FAF2D0C029CB58C0584D40@phx.gbl>


Hi!
I have a folder, with the following text files with columns:
 
bb_ 1
bb_2
ww_1
ww_2
ff_1
ff_2
 
What I want to do is:

Extract columns 5,6, 8 from files bb_
Extract columns 3,4 from files ww_
Get 5 files, corresponding to different columns:
Files (excel files): 'ro' with colums number 5,  'bf' with colums number 6,  'sm' with column 8,  'se' with columns number 3 and  'dse' with columns number 4
 
I was following the example from: http://mail.python.org/pipermail/tutor/2009-February/067391.html 
 
import os
import fnmatch
import csv
 
path = '//(my path)/'
files=os.listdir(path)
csv_out=csv.writer(open('out1.csv', 'wb'),delimiter=' ')
 
ro=[]; bf=[]; sm=[]; se=[]; dse=[]

listofcolumns=[]
 
def column(tofile):

for infile in files:
filename= os.path.join(path,infile)
f=open(filename, 'r')
  
for line in f.readlines():
b=line.split('\t')

if fnmatch.fnmatch(filename, 'bb_*'):
A=[]; B=[]; C=[]; 
for b in line:
A.append(b[5].strip())
B.append(b[6].strip())
C.append(b[8].strip())
ro.append(A)
bf.append(B)
sm.append(C)
elif fnmatch.fnmatch(filename, 'ww_*'):
D=[]; E=[]
for b in line:
D.append(b[3])
E.append(b[4])
    
se.append(D)
dse.append(E)
f.close() 
#list of pairs of (value list, name)
listofcolumns=[(ro, 'ro'),(bf, 'bf'),(sm, 'sm'),(se, 'se'),(dse,'dse')]  
  
 for values, name in listofcolumns: 
   
output=open(name + '.csv', 'w')
   for value in values:    
     csv_out.writerows(rows)
     csv_out.writerows('\n')
  
 column(listofcolumns)
 
It does't give me any error but gives me the documents in blank
Another question, Is there anyway to write the code properly ? I am sorry about the spaces, I can send a text file 
Many thanks! :D
  		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120716/c2948865/attachment.html>

From steve at pearwood.info  Mon Jul 16 17:06:15 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 17 Jul 2012 01:06:15 +1000
Subject: [Tutor] passing global variable as argument.
In-Reply-To: <CA+WPOVMmvcq6p=92w1q7ieadiLzJvrxdxxrkDyNAakWfNrrDiA@mail.gmail.com>
References: <CA+WPOVMmvcq6p=92w1q7ieadiLzJvrxdxxrkDyNAakWfNrrDiA@mail.gmail.com>
Message-ID: <50042DE7.1070502@pearwood.info>

Bala subramanian wrote:
> Friends,
> I want to define a function that can populate an array by taking its name
> (which is defined globally). I defined two empty arrays as follows and a
> function that can populate the array.

In general it is tricky to resize and populate numpy arrays in place. It is 
usually better to just create a fresh array and reassign it. Something like 
this should probably work:

def ref(length):
     arr = np.zeros((length, 3), dtype='float32')
     for i in range(length):
         for j in range(3):
             arr[i, j] = resid[Index[i]].cent()[j]
     return arr


ref_f1 = ref(3)
ref_f2 = ref(5)


should work for you. (I can't test it because you don't say what resid and 
Index are.)



To explain why your earlier code does not work the way you expect, read on:


> REF_F1=np.array([])
> REF_F2=np.array([])
> 
> # populating the given array
> def ref(ln,REF_F1):

So far this is good -- your function takes an argument called "REF_F1", which 
can be any array you like. It's just a local name.

The function sees REF_F1 is a local variable.


>     global REF_F1

But this is no good, because now you declare the name REF_F1 to be global 
instead of local. So now the function sees REF_F1 as a global variable, and 
everything that you do to it, occurs to the global called REF_F1.

By the way, this bug is no longer permitted in the latest version of Python. 
Using Python 3.2:

py> x = 23
py> def func(x):
...     global x
...     print("x =", x)
...
   File "<stdin>", line 1
SyntaxError: name 'x' is parameter and global


In general, if you feel the need to use "global", 90% of the time you are 
doing something wrong and will have problems. You should avoid using global 
unless absolutely necessary.



-- 
Steven

From alexandre.zani at gmail.com  Mon Jul 16 17:16:16 2012
From: alexandre.zani at gmail.com (Alexandre Zani)
Date: Mon, 16 Jul 2012 08:16:16 -0700
Subject: [Tutor] newbie Questions
In-Reply-To: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
References: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
Message-ID: <CAJVMpoq4s2dE8EiaVafzXDexktrEByRGCL+1JjAg2zj6=wFgwA@mail.gmail.com>

On Mon, Jul 16, 2012 at 5:09 AM, Matthew Ngaha <chigga101 at gmail.com> wrote:
> Hi all. I'm new to Python and Programming in general. I've started out with
> Python for beginners, and so far so good. My friend who i might add, is not
> a programmer but has had experience in the world of programming (i dont know
> how much but he claims a lot), has told me to forget about Python and focus
> on PHP. He knows i studied a HTML and CSS course and told me for the
> direction i'm going in, Python is not needed and won't give me all the
> options or variety PHP can. Thats he's opinion, i'd rather make my own mind
> up, but its lead me to these questions out of curiousity:
>
> a) at this early stage i dont exactly know what web options are:(  but is
> Python limited when it comes to matters relating to Web options/developing?

PHP will get you from 0 to website by the first page of your first
tutorial. That's very attractive if you're interested in web
programming. Most likely, if you use Python, you'll learn how to use
the language more generally, before learning how to apply the language
to a website. However, in my experience (I learned PHP a long time ago
and Python a few years ago) Python gives you the most reward. I've
used Python for website building, but also lots of other useful
applications.

> b) Are there better options, or can Python get the job done as good as any?

It depends upon the job. If you plan on work as a programmer/software
engineer, you will need to learn many languages. Yes, hammers can be
used to put in screws and you could probably figure out how to use a
screwdriver to put in a nail, but really, you're going to be
successful if you have both a hammer and a screwdriver in your
toolbox. Java, C, Python, C++, each have their own uses. However, (and
this may be a biased assessment) PHP is more like a hammer with its
head removed and a screwdriver duct-taped on it. Sure, you can use it,
but it's not going to be a pleasant experience.

The languages which I would say are closest to Python in terms of
where they are put to use are Perl and Ruby. My advice is this: give
them each a shot (the first couple pages of a tutorial shouldn't take
more than a few hours) and see which makes you feel the most
comfortable.

> c) after completing and understanding a beginner's book, would i be at
> intermediate level, or still below?
> d) Would i need a more advanced tutorial, what do you advise after finishing
> a beginners course?

Code, code, code. Programming is a practice. You'll learn the most by
doing and researching solutions for specific problems you are
encountering. Then, read blog posts, watch PyCon videos, look at the
mailing list etc... Just expose yourself to the language and community
and learn through osmosis.

>
> e) And finally, are there other essential things i must learn after Python?
> i heard Django is important?

Django is just one way to do Python web development. It's hugely
useful for some things and terribly useless for others. There are
plenty of other ways to do web development.

> f) is Django the equivelent to PHP's MySql?

As plenty of people have said, Django is a way to make websites while
MySQL is a database system. You can actually use MySQL with Django.

>
> You dont have to answer all questions. Just answering one would help me
> greatly in my future decisions, as i want to commit fully to the right
> programming language.
>
> since this isnt a forum, how can i thank everyone for helping?
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

From bala.biophysics at gmail.com  Mon Jul 16 17:26:31 2012
From: bala.biophysics at gmail.com (Bala subramanian)
Date: Mon, 16 Jul 2012 17:26:31 +0200
Subject: [Tutor] passing global variable as argument.
In-Reply-To: <50042DE7.1070502@pearwood.info>
References: <CA+WPOVMmvcq6p=92w1q7ieadiLzJvrxdxxrkDyNAakWfNrrDiA@mail.gmail.com>
	<50042DE7.1070502@pearwood.info>
Message-ID: <CA+WPOVPPcdU7HxUZXB=oLjviEBou-Gv=Xyz5oS7DVjgL7t0o+w@mail.gmail.com>

Thank you wayne and steven. You suggestion to create a fresh array within
the function and assigning it to variable worked fine and the result was
exactly what i was looking for. In future i remember not to use global
variables as fn. parameters.

thanks once again for detailed explanation,
bala

On Mon, Jul 16, 2012 at 5:06 PM, Steven D'Aprano <steve at pearwood.info>wrote:

> Bala subramanian wrote:
>
>> Friends,
>> I want to define a function that can populate an array by taking its name
>> (which is defined globally). I defined two empty arrays as follows and a
>> function that can populate the array.
>>
>
> In general it is tricky to resize and populate numpy arrays in place. It
> is usually better to just create a fresh array and reassign it. Something
> like this should probably work:
>
> def ref(length):
>     arr = np.zeros((length, 3), dtype='float32')
>     for i in range(length):
>         for j in range(3):
>             arr[i, j] = resid[Index[i]].cent()[j]
>     return arr
>
>
> ref_f1 = ref(3)
> ref_f2 = ref(5)
>
>
> should work for you. (I can't test it because you don't say what resid and
> Index are.)
>
>
>
> To explain why your earlier code does not work the way you expect, read on:
>
>
>
>  REF_F1=np.array([])
>> REF_F2=np.array([])
>>
>> # populating the given array
>> def ref(ln,REF_F1):
>>
>
> So far this is good -- your function takes an argument called "REF_F1",
> which can be any array you like. It's just a local name.
>
> The function sees REF_F1 is a local variable.
>
>
>      global REF_F1
>>
>
> But this is no good, because now you declare the name REF_F1 to be global
> instead of local. So now the function sees REF_F1 as a global variable, and
> everything that you do to it, occurs to the global called REF_F1.
>
> By the way, this bug is no longer permitted in the latest version of
> Python. Using Python 3.2:
>
> py> x = 23
> py> def func(x):
> ...     global x
> ...     print("x =", x)
> ...
>   File "<stdin>", line 1
> SyntaxError: name 'x' is parameter and global
>
>
> In general, if you feel the need to use "global", 90% of the time you are
> doing something wrong and will have problems. You should avoid using global
> unless absolutely necessary.
>
>
>
> --
> Steven
> ______________________________**_________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>



-- 
C. Balasubramanian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120716/11628cf7/attachment-0001.html>

From taserian at gmail.com  Mon Jul 16 18:11:33 2012
From: taserian at gmail.com (taserian)
Date: Mon, 16 Jul 2012 12:11:33 -0400
Subject: [Tutor] Extracting columns from many files to different files
In-Reply-To: <BAY164-W34CD4FAF2D0C029CB58C0584D40@phx.gbl>
References: <BAY164-W34CD4FAF2D0C029CB58C0584D40@phx.gbl>
Message-ID: <CAOgHRJOmTZ2SGY88Dh03ddYGYVeZfp7Qastq0=SAVdxrP2NGVQ@mail.gmail.com>

On Mon, Jul 16, 2012 at 10:58 AM, susana moreno colomer <
susana_87 at hotmail.com> wrote:

>  Hi!
> I have a folder, with the following text files with columns:
>
> bb_ 1
> bb_2
> ww_1
> ww_2
> ff_1
> ff_2
>
> What I want to do is:
>
>    - Extract columns 5,6, 8 from files bb_
>    - Extract columns 3,4 from files ww_
>    - Get 5 files, corresponding to different columns:
>    - Files (excel files): 'ro' with colums number 5,  'bf' with colums
>    number 6,  'sm' with column 8,  'se' with columns number 3 and  'dse' with
>    columns number 4
>
> How are these columns separated? Blank spaces, tabs, commas?

I'm mostly worried about:

for b in line:
    A.append(b[5].strip())
    B.append(b[6].strip())
    C.append(b[8].strip())

For the A List, this will take the 5th character from the line, not the 5th
column. You may need to split the line based on the separators.

AR
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120716/c35c63c7/attachment.html>

From joel.goldstick at gmail.com  Mon Jul 16 18:16:22 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 16 Jul 2012 12:16:22 -0400
Subject: [Tutor] Extracting columns from many files to different files
In-Reply-To: <CAOgHRJOmTZ2SGY88Dh03ddYGYVeZfp7Qastq0=SAVdxrP2NGVQ@mail.gmail.com>
References: <BAY164-W34CD4FAF2D0C029CB58C0584D40@phx.gbl>
	<CAOgHRJOmTZ2SGY88Dh03ddYGYVeZfp7Qastq0=SAVdxrP2NGVQ@mail.gmail.com>
Message-ID: <CAPM-O+yQtv-Q-DSa7nVpDChuw92nby2WwWR-BfF=9KfjeVy-yg@mail.gmail.com>

On Mon, Jul 16, 2012 at 12:11 PM, taserian <taserian at gmail.com> wrote:
> On Mon, Jul 16, 2012 at 10:58 AM, susana moreno colomer
> <susana_87 at hotmail.com> wrote:
>>
>> Hi!
>> I have a folder, with the following text files with columns:
>>
>> bb_ 1
>> bb_2
>> ww_1
>> ww_2
>> ff_1
>> ff_2
>>
>> What I want to do is:
>>
>> Extract columns 5,6, 8 from files bb_
>> Extract columns 3,4 from files ww_
>> Get 5 files, corresponding to different columns:
>> Files (excel files): 'ro' with colums number 5,  'bf' with colums number
>> 6,  'sm' with column 8,  'se' with columns number 3 and  'dse' with columns
>> number 4
>
> How are these columns separated? Blank spaces, tabs, commas?
>
> I'm mostly worried about:
>
>
> for b in line:
>     A.append(b[5].strip())
>     B.append(b[6].strip())
>     C.append(b[8].strip())
>
> For the A List, this will take the 5th character from the line, not the 5th
> column. You may need to split the line based on the separators.
>
> AR
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
To make your code show correctly you must set your email program to
write text and not html.  You should also set your text editor to turn
tabs into 4 spaces.  Tabs work in python, but you can't mix tabs and
spaces, so it is less of a problem if you only use spaces for
indenting


-- 
Joel Goldstick

From susana_87 at hotmail.com  Mon Jul 16 18:24:51 2012
From: susana_87 at hotmail.com (susana moreno colomer)
Date: Mon, 16 Jul 2012 18:24:51 +0200
Subject: [Tutor] Extracting columns from many files to different files
In-Reply-To: <CAPM-O+yQtv-Q-DSa7nVpDChuw92nby2WwWR-BfF=9KfjeVy-yg@mail.gmail.com>
References: <BAY164-W34CD4FAF2D0C029CB58C0584D40@phx.gbl>,
	<CAOgHRJOmTZ2SGY88Dh03ddYGYVeZfp7Qastq0=SAVdxrP2NGVQ@mail.gmail.com>,
	<CAPM-O+yQtv-Q-DSa7nVpDChuw92nby2WwWR-BfF=9KfjeVy-yg@mail.gmail.com>
Message-ID: <BAY164-W6206A167065EF1988A5BF184D40@phx.gbl>


They are separated by tabs

> Date: Mon, 16 Jul 2012 12:16:22 -0400
> Subject: Re: [Tutor] Extracting columns from many files to different files
> From: joel.goldstick at gmail.com
> To: taserian at gmail.com
> CC: susana_87 at hotmail.com; tutor at python.org
> 
> On Mon, Jul 16, 2012 at 12:11 PM, taserian <taserian at gmail.com> wrote:
> > On Mon, Jul 16, 2012 at 10:58 AM, susana moreno colomer
> > <susana_87 at hotmail.com> wrote:
> >>
> >> Hi!
> >> I have a folder, with the following text files with columns:
> >>
> >> bb_ 1
> >> bb_2
> >> ww_1
> >> ww_2
> >> ff_1
> >> ff_2
> >>
> >> What I want to do is:
> >>
> >> Extract columns 5,6, 8 from files bb_
> >> Extract columns 3,4 from files ww_
> >> Get 5 files, corresponding to different columns:
> >> Files (excel files): 'ro' with colums number 5,  'bf' with colums number
> >> 6,  'sm' with column 8,  'se' with columns number 3 and  'dse' with columns
> >> number 4
> >
> > How are these columns separated? Blank spaces, tabs, commas?
> >
> > I'm mostly worried about:
> >
> >
> > for b in line:
> >     A.append(b[5].strip())
> >     B.append(b[6].strip())
> >     C.append(b[8].strip())
> >
> > For the A List, this will take the 5th character from the line, not the 5th
> > column. You may need to split the line based on the separators.
> >
> > AR
> >
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > http://mail.python.org/mailman/listinfo/tutor
> >
> To make your code show correctly you must set your email program to
> write text and not html.  You should also set your text editor to turn
> tabs into 4 spaces.  Tabs work in python, but you can't mix tabs and
> spaces, so it is less of a problem if you only use spaces for
> indenting
> 
> 
> -- 
> Joel Goldstick
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120716/0b1bc4bf/attachment.html>

From joel.goldstick at gmail.com  Mon Jul 16 18:58:39 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 16 Jul 2012 12:58:39 -0400
Subject: [Tutor] Extracting columns from many files to different files
In-Reply-To: <BAY164-W6206A167065EF1988A5BF184D40@phx.gbl>
References: <BAY164-W34CD4FAF2D0C029CB58C0584D40@phx.gbl>
	<CAOgHRJOmTZ2SGY88Dh03ddYGYVeZfp7Qastq0=SAVdxrP2NGVQ@mail.gmail.com>
	<CAPM-O+yQtv-Q-DSa7nVpDChuw92nby2WwWR-BfF=9KfjeVy-yg@mail.gmail.com>
	<BAY164-W6206A167065EF1988A5BF184D40@phx.gbl>
Message-ID: <CAPM-O+wUW_Z5LtYb9aO3D+rgH4hohOci1Qw_oozHXX-UnO2jng@mail.gmail.com>

On Mon, Jul 16, 2012 at 12:24 PM, susana moreno colomer
<susana_87 at hotmail.com> wrote:
> They are separated by tabs
>
>> Date: Mon, 16 Jul 2012 12:16:22 -0400
>> Subject: Re: [Tutor] Extracting columns from many files to different files
>> From: joel.goldstick at gmail.com
>> To: taserian at gmail.com
>> CC: susana_87 at hotmail.com; tutor at python.org
>
>>
>> On Mon, Jul 16, 2012 at 12:11 PM, taserian <taserian at gmail.com> wrote:
>> > On Mon, Jul 16, 2012 at 10:58 AM, susana moreno colomer
>> > <susana_87 at hotmail.com> wrote:
>> >>
>> >> Hi!
>> >> I have a folder, with the following text files with columns:
>> >>
>> >> bb_ 1
>> >> bb_2
>> >> ww_1
>> >> ww_2
>> >> ff_1
>> >> ff_2
>> >>
>> >> What I want to do is:
>> >>
>> >> Extract columns 5,6, 8 from files bb_
>> >> Extract columns 3,4 from files ww_
>> >> Get 5 files, corresponding to different columns:
>> >> Files (excel files): 'ro' with colums number 5, 'bf' with colums number
>> >> 6, 'sm' with column 8, 'se' with columns number 3 and 'dse' with
>> >> columns
>> >> number 4
>> >
>> > How are these columns separated? Blank spaces, tabs, commas?
>> >
>> > I'm mostly worried about:
>> >
>> >
>> > for b in line:
>> > A.append(b[5].strip())
>> > B.append(b[6].strip())
>> > C.append(b[8].strip())
>> >
>> > For the A List, this will take the 5th character from the line, not the
>> > 5th
>> > column. You may need to split the line based on the separators.
>> >
>> > AR
>> >
>> > _______________________________________________
>> > Tutor maillist - Tutor at python.org
>> > To unsubscribe or change subscription options:
>> > http://mail.python.org/mailman/listinfo/tutor
>> >
>> To make your code show correctly you must set your email program to
>> write text and not html. You should also set your text editor to turn
>> tabs into 4 spaces. Tabs work in python, but you can't mix tabs and
>> spaces, so it is less of a problem if you only use spaces for
>> indenting
>>
>>
>> --
>> Joel Goldstick
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
Its great that you are using spaces in your code editor, but the
reason your code isn't formatted in your emails is that you have rich
text editing or html editing on.  If you are using gmail, you can tell
this because it gives a word processor style ribbon of buttons at the
top of your edit window.


-- 
Joel Goldstick

From chigga101 at gmail.com  Mon Jul 16 19:12:31 2012
From: chigga101 at gmail.com (Matthew Ngaha)
Date: Mon, 16 Jul 2012 18:12:31 +0100
Subject: [Tutor] newbie Questions
In-Reply-To: <CAJVMpoq4s2dE8EiaVafzXDexktrEByRGCL+1JjAg2zj6=wFgwA@mail.gmail.com>
References: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
	<CAJVMpoq4s2dE8EiaVafzXDexktrEByRGCL+1JjAg2zj6=wFgwA@mail.gmail.com>
Message-ID: <CACzNyA1-tHYUp_ptnoSNcwfMh97w7cn5MfJhCKE8VW8LPCvTUA@mail.gmail.com>

Thanks guys, i didnt think i would get so many kind and helpful responses.
I am so grateful:x. I have read each and every reply and i am now very
confident in the direction i need to take. Everything is a lot clearer now.
Even though i'm new to programming, i am very intrigued by it and want to
dedicate a good portion of my time to it, and by reading the replies, i
know ive made the right choice in choosing Python over PHP:) To answer a
question on what was more important, a job that pays low, or to learn good
clean coding.. My answer is I want to devote my time in it for a love of
programming and to gain a deeper understanding on it. the money part is not
as important:)The book i am currently reading is "Python Programming for
the absolute beginner". I will save this mail in a special place on my hard
drive:) i really appreciate it guys:)


On Mon, Jul 16, 2012 at 4:16 PM, Alexandre Zani <alexandre.zani at gmail.com>wrote:

> On Mon, Jul 16, 2012 at 5:09 AM, Matthew Ngaha <chigga101 at gmail.com>
> wrote:
> > Hi all. I'm new to Python and Programming in general. I've started out
> with
> > Python for beginners, and so far so good. My friend who i might add, is
> not
> > a programmer but has had experience in the world of programming (i dont
> know
> > how much but he claims a lot), has told me to forget about Python and
> focus
> > on PHP. He knows i studied a HTML and CSS course and told me for the
> > direction i'm going in, Python is not needed and won't give me all the
> > options or variety PHP can. Thats he's opinion, i'd rather make my own
> mind
> > up, but its lead me to these questions out of curiousity:
> >
> > a) at this early stage i dont exactly know what web options are:(  but is
> > Python limited when it comes to matters relating to Web
> options/developing?
>
> PHP will get you from 0 to website by the first page of your first
> tutorial. That's very attractive if you're interested in web
> programming. Most likely, if you use Python, you'll learn how to use
> the language more generally, before learning how to apply the language
> to a website. However, in my experience (I learned PHP a long time ago
> and Python a few years ago) Python gives you the most reward. I've
> used Python for website building, but also lots of other useful
> applications.
>
> > b) Are there better options, or can Python get the job done as good as
> any?
>
> It depends upon the job. If you plan on work as a programmer/software
> engineer, you will need to learn many languages. Yes, hammers can be
> used to put in screws and you could probably figure out how to use a
> screwdriver to put in a nail, but really, you're going to be
> successful if you have both a hammer and a screwdriver in your
> toolbox. Java, C, Python, C++, each have their own uses. However, (and
> this may be a biased assessment) PHP is more like a hammer with its
> head removed and a screwdriver duct-taped on it. Sure, you can use it,
> but it's not going to be a pleasant experience.
>
> The languages which I would say are closest to Python in terms of
> where they are put to use are Perl and Ruby. My advice is this: give
> them each a shot (the first couple pages of a tutorial shouldn't take
> more than a few hours) and see which makes you feel the most
> comfortable.
>
> > c) after completing and understanding a beginner's book, would i be at
> > intermediate level, or still below?
> > d) Would i need a more advanced tutorial, what do you advise after
> finishing
> > a beginners course?
>
> Code, code, code. Programming is a practice. You'll learn the most by
> doing and researching solutions for specific problems you are
> encountering. Then, read blog posts, watch PyCon videos, look at the
> mailing list etc... Just expose yourself to the language and community
> and learn through osmosis.
>
> >
> > e) And finally, are there other essential things i must learn after
> Python?
> > i heard Django is important?
>
> Django is just one way to do Python web development. It's hugely
> useful for some things and terribly useless for others. There are
> plenty of other ways to do web development.
>
> > f) is Django the equivelent to PHP's MySql?
>
> As plenty of people have said, Django is a way to make websites while
> MySQL is a database system. You can actually use MySQL with Django.
>
> >
> > You dont have to answer all questions. Just answering one would help me
> > greatly in my future decisions, as i want to commit fully to the right
> > programming language.
> >
> > since this isnt a forum, how can i thank everyone for helping?
> >
> >
> >
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > http://mail.python.org/mailman/listinfo/tutor
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120716/189b7ba1/attachment-0001.html>

From d at davea.name  Mon Jul 16 20:14:24 2012
From: d at davea.name (Dave Angel)
Date: Mon, 16 Jul 2012 14:14:24 -0400
Subject: [Tutor] Extracting columns from many files to different files
In-Reply-To: <CAPM-O+wUW_Z5LtYb9aO3D+rgH4hohOci1Qw_oozHXX-UnO2jng@mail.gmail.com>
References: <BAY164-W34CD4FAF2D0C029CB58C0584D40@phx.gbl>
	<CAOgHRJOmTZ2SGY88Dh03ddYGYVeZfp7Qastq0=SAVdxrP2NGVQ@mail.gmail.com>
	<CAPM-O+yQtv-Q-DSa7nVpDChuw92nby2WwWR-BfF=9KfjeVy-yg@mail.gmail.com>
	<BAY164-W6206A167065EF1988A5BF184D40@phx.gbl>
	<CAPM-O+wUW_Z5LtYb9aO3D+rgH4hohOci1Qw_oozHXX-UnO2jng@mail.gmail.com>
Message-ID: <50045A00.3050401@davea.name>

On 07/16/2012 12:58 PM, Joel Goldstick wrote:
> On Mon, Jul 16, 2012 at 12:24 PM, susana moreno colomer
> <susana_87 at hotmail.com> wrote:
>> They are separated by tabs
You're top-posting again.  The history of all these other messages is
thoroughly obfuscated if you don't follow the convention of adding your
remarks AFTER the part you're quoting.

>> <BIG SNIP>
>> for b in line:
>> A.append(b[5].strip())
>> B.append(b[6].strip())
>> C.append(b[8].strip())
>>
>> For the A List, this will take the 5th character from the line, not the
>> 5th
>> column. You may need to split the line based on the separators.
>>
>> AR
>>
>> To make your code show correctly you must set your email program to
>> write text and not html. You should also set your text editor to turn
>> tabs into 4 spaces. Tabs work in python, but you can't mix tabs and
>> spaces, so it is less of a problem if you only use spaces for
>> indenting
>>
>>
>> --
>> Joel Goldstick
>>
> Its great that you are using spaces in your code editor, but the
> reason your code isn't formatted in your emails is that you have rich
> text editing or html editing on.  If you are using gmail, you can tell
> this because it gives a word processor style ribbon of buttons at the
> top of your edit window.
>
>

You'll notice that you've successfully confused Joel, by not putting
your remarks AFTER his query.

Since you have tabs in your source file (and some people do prefer tabs
for some obscure reason), the easiest way to get rid of them is the
Linux utility 'expand'.

-- 

DaveA


From wprins at gmail.com  Mon Jul 16 20:29:23 2012
From: wprins at gmail.com (Walter Prins)
Date: Mon, 16 Jul 2012 19:29:23 +0100
Subject: [Tutor] Extracting columns from many files to different files
In-Reply-To: <BAY164-W34CD4FAF2D0C029CB58C0584D40@phx.gbl>
References: <BAY164-W34CD4FAF2D0C029CB58C0584D40@phx.gbl>
Message-ID: <CANLXbfDzPEi-ksjF6D7W2Pgz5yL2XPQBV8+HUQzZkJOkCcVOgw@mail.gmail.com>

Hi

On 16 July 2012 15:58, susana moreno colomer <susana_87 at hotmail.com> wrote:
> Hi!
> I have a folder, with the following text files with columns:

If I may ask, how does this question relate to your previous similar
question(s)?  What do you want to do with these 5 files once you have
them?  (Differently put, does this question imply you've succeeded
with your previous question(s) and now you're onto the next step or a
different problem, or does this question imply you've given up with
the previous line of endeavour and are now just trying a new tack to
the same problem?)

Anyway, it seems to me you're trying to cobble something together for
your purposes from sources on the web without (possibly) really
understanding what the code is doing.  It may be easier and better in
the long run to just write your own solution to your own specific
problem requirements.

Regarding the code/formatting, as others have suggested, you should
ideally try to paste plaintext formatted with nothing more than
spaces, but you may also consider pasting your code using pastebin:
http://pastebin.com/

Cheers,

Walter

From wolfrage8765 at gmail.com  Mon Jul 16 21:25:32 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Mon, 16 Jul 2012 21:25:32 +0200
Subject: [Tutor] newbie Questions
In-Reply-To: <alpine.DEB.2.02.1207160849220.3779@gilgamesh>
References: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
	<CANLXbfCWbixqb7F=4E8dCtGHeeh19yxZid-EChUY+c7aR6ZGAg@mail.gmail.com>
	<alpine.DEB.2.02.1207160849220.3779@gilgamesh>
Message-ID: <50046AAC.6070900@gmail.com>

I would just like to add that I am a web developer and I left PHP for
Python. I left PHP because it was not as powerful server side (Cron Jobs
and Such) and I wanted to possibly create desktop applications, more
recently Android Apps via SL4A and IPhone Apps via pyjamas. PHP is a
limited language although there have been attempts at making it work for
desktop apps,it just was not designed as a generally use language like
Python. But the choice is yours and the path for Python in web
development may be a little more trickier than PHP; but when you want to
do more than web development you will realize Python is the way to go.
Also just one more note Python code is so much easier to read than PHP,
and the language as a whole is much more consistent, thanks PEP 8.

On 07/16/2012 03:54 PM, Wayne Werner wrote:
>
> On Mon, 16 Jul 2012, Walter Prins wrote:
>
>> Hi again Matthew,
>>
>> I forgot to include the following link which I originally thought to
>> include, which is one guy's set of (IMHO very cogent) criticisms
>> against PHP as programming language:  http://is.gd/z1POXC  Hopefully
>> it gives you something else to think about regarding the PHP vs Python
>> question apart from just whether doing websites in it is "easy".
>>
>
> I read that article recently and recommend it as well. I used to
> think that PHP was just fine... but since reading that article (and
> several related ones), I've since revised my opinion.
>
> I would now tell someone to learn any language besides php.
>
> -Wayne
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From learnpythonvs at gmail.com  Mon Jul 16 22:32:04 2012
From: learnpythonvs at gmail.com (Vignesh Sathiamoorthy)
Date: Mon, 16 Jul 2012 13:32:04 -0700
Subject: [Tutor] newbie Questions
In-Reply-To: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
References: <CACzNyA1bfyWVwmQLEg10z2NiJ2QBx9_YuX=yc55ky3h-j3z4vQ@mail.gmail.com>
Message-ID: <CAF7Fu0g3MkSL1Gsu9BTRys=X0VVEfmtDTwmvf=0H4a5t6DuBPw@mail.gmail.com>

Check out http://www.udacity.com/

Enroll in few courses - to begin with, check
http://www.udacity.com/view#Course/cs101/CourseRev/apr2012/Unit/671001/Nugget/675002





On Mon, Jul 16, 2012 at 5:09 AM, Matthew Ngaha <chigga101 at gmail.com> wrote:

> Hi all. I'm new to Python and Programming in general. I've started out
> with Python for beginners, and so far so good. My friend who i might add,
> is not a programmer but has had experience in the world of programming (i
> dont know how much but he claims a lot), has told me to forget about Python
> and focus on PHP. He knows i studied a HTML and CSS course and told me for
> the direction i'm going in, Python is not needed and won't give me all the
> options or variety PHP can. Thats he's opinion, i'd rather make my own mind
> up, but its lead me to these questions out of curiousity:
>
> a) at this early stage i dont exactly know what web options are:(  but is
> Python limited when it comes to matters relating to Web options/developing?
> b) Are there better options, or can Python get the job done as good as any?
> c) after completing and understanding a beginner's book, would i be at
> intermediate level, or still below?
> d) Would i need a more advanced tutorial, what do you advise after
> finishing a beginners course?
>
> e) And finally, are there other essential things i must learn after
> Python? i heard Django is important?
> f) is Django the equivelent to PHP's MySql?
>
> You dont have to answer all questions. Just answering one would help me
> greatly in my future decisions, as i want to commit fully to the right
> programming language.
>
> since this isnt a forum, how can i thank everyone for helping?
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120716/10534e9b/attachment.html>

From learnpythonvs at gmail.com  Mon Jul 16 22:42:21 2012
From: learnpythonvs at gmail.com (Vignesh Sathiamoorthy)
Date: Mon, 16 Jul 2012 13:42:21 -0700
Subject: [Tutor] starting to learn
In-Reply-To: <CAAB-_MhOZkgh6--xTJiYcPgCznNAtoPHpurkh52KBAD36gG=6w@mail.gmail.com>
References: <CAAB-_MhOZkgh6--xTJiYcPgCznNAtoPHpurkh52KBAD36gG=6w@mail.gmail.com>
Message-ID: <CAF7Fu0ihjr7hxU1xLDgGr40dnXt5ku+Q3tBMcFqwYY3LZhW5OA@mail.gmail.com>

Have you explored (
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-189-a-gentle-introduction-to-programming-using-python-january-iap-2011/
)

check out their assignments, sample programs, lecture notes ..

Also start reading/executing examples/codes from
http://docs.python.org/tutorial/index.html



On Wed, Jul 11, 2012 at 4:46 PM, blindmaildrop <blindmaildrop at gmail.com>wrote:

> Hello!
>
> I am just starting to learn python, having signed up for a class on it in
> University.  The last time I programmed anything was in the long long long
> ago of BASIC and (well since I spent time doing other things) I missed the
> development boat.
>
> So starting from scratch, how-to?
>
> thanks
>
> --c.
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120716/3612ce6a/attachment-0001.html>

From sntshkmr60 at gmail.com  Tue Jul 17 17:10:39 2012
From: sntshkmr60 at gmail.com (Santosh Kumar)
Date: Tue, 17 Jul 2012 20:40:39 +0530
Subject: [Tutor] How to print something just after 3 attempts?
Message-ID: <CAE7MaQYvjy6G5cXn5-1F4OZh-c9-+m=RVV09Jy_dx+RpRK307A@mail.gmail.com>

Hello There,

This problem isn't so simple as it looks in the title/subject of this email.

I am doing a tutorial on Python, and there is a script named password.py:

password = "foobar"

while password != "unicorn":
    password = raw_input("Password: ")
print "Welcome in"


The question says "Modify the password guessing program to keep track
of how many times the user has entered the password wrong. If it is
more than 3 times, print ?That must have been complicated.?

So I did, I did it in two ways.
Case I:
password = "foobar"
attempt = 0
while password != "unicorn":
    password = raw_input("Password: ")
    attempt = attempt + 1
    if attempt == 3:
        print "That must have been complicated.."

print "Welcome in.."
# This script does the half of work. This prints that statement after
three attempts but lets you enter password more time.


Case II:
password = "foobar"
attempt = 0
while password != "unicorn":
    password = raw_input("Password: ")
    attempt = attempt + 1
    if attempt == 3:
        print "That must have been complicated.."
        break
print "Welcome in.."
# This script performs better than first one, but the problem is it
welcomes you even if you haven't entered the password correct.

Please tell, what can I do?

From joel.goldstick at gmail.com  Tue Jul 17 17:23:07 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 17 Jul 2012 11:23:07 -0400
Subject: [Tutor] How to print something just after 3 attempts?
In-Reply-To: <CAE7MaQYvjy6G5cXn5-1F4OZh-c9-+m=RVV09Jy_dx+RpRK307A@mail.gmail.com>
References: <CAE7MaQYvjy6G5cXn5-1F4OZh-c9-+m=RVV09Jy_dx+RpRK307A@mail.gmail.com>
Message-ID: <CAPM-O+ystCbpoT85Ce-44iNjciFsfCCu1j_U1aeitWSP8eibrA@mail.gmail.com>

On Tue, Jul 17, 2012 at 11:10 AM, Santosh Kumar <sntshkmr60 at gmail.com> wrote:
> Hello There,
>
> This problem isn't so simple as it looks in the title/subject of this email.
>
> I am doing a tutorial on Python, and there is a script named password.py:
>
> password = "foobar"
>
> while password != "unicorn":
>     password = raw_input("Password: ")
> print "Welcome in"
>
>
> The question says "Modify the password guessing program to keep track
> of how many times the user has entered the password wrong. If it is
> more than 3 times, print ?That must have been complicated.?
>
> So I did, I did it in two ways.
> Case I:
> password = "foobar"
> attempt = 0
> while password != "unicorn":
>     password = raw_input("Password: ")
>     attempt = attempt + 1
>     if attempt == 3:
>         print "That must have been complicated.."
>
> print "Welcome in.."
> # This script does the half of work. This prints that statement after
> three attempts but lets you enter password more time.
>
>
> Case II:
> password = "foobar"
> attempt = 0
> while password != "unicorn":
>     password = raw_input("Password: ")
>     attempt = attempt + 1
>     if attempt == 3:
>         print "That must have been complicated.."
>         break
> print "Welcome in.."
move the print "Welcome under break:
          break
          print "Welcome..."

> # This script performs better than first one, but the problem is it
> welcomes you even if you haven't entered the password correct.
>
> Please tell, what can I do?
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor



-- 
Joel Goldstick

From paulcmcnally at live.com  Tue Jul 17 17:28:22 2012
From: paulcmcnally at live.com (Paul McNally)
Date: Tue, 17 Jul 2012 11:28:22 -0400
Subject: [Tutor] How to print something just after 3 attempts?
In-Reply-To: <CAPM-O+ystCbpoT85Ce-44iNjciFsfCCu1j_U1aeitWSP8eibrA@mail.gmail.com>
References: <CAE7MaQYvjy6G5cXn5-1F4OZh-c9-+m=RVV09Jy_dx+RpRK307A@mail.gmail.com>,
	<CAPM-O+ystCbpoT85Ce-44iNjciFsfCCu1j_U1aeitWSP8eibrA@mail.gmail.com>
Message-ID: <SNT115-W2362769CE138886889A510B0DB0@phx.gbl>




> Date: Tue, 17 Jul 2012 11:23:07 -0400
> From: joel.goldstick at gmail.com
> To: sntshkmr60 at gmail.com
> CC: tutor at python.org
> Subject: Re: [Tutor] How to print something just after 3 attempts?
> 
> On Tue, Jul 17, 2012 at 11:10 AM, Santosh Kumar <sntshkmr60 at gmail.com> wrote:
> > Hello There,
> >
> > This problem isn't so simple as it looks in the title/subject of this email.
> >
> > I am doing a tutorial on Python, and there is a script named password.py:
> >
> > password = "foobar"
> >
> > while password != "unicorn":
> >     password = raw_input("Password: ")
> > print "Welcome in"
> >
> >
> > The question says "Modify the password guessing program to keep track
> > of how many times the user has entered the password wrong. If it is
> > more than 3 times, print ?That must have been complicated.?
> >
> > So I did, I did it in two ways.
> > Case I:
> > password = "foobar"
> > attempt = 0
> > while password != "unicorn":
> >     password = raw_input("Password: ")
> >     attempt = attempt + 1
> >     if attempt == 3:
> >         print "That must have been complicated.."
> >
> > print "Welcome in.."
> > # This script does the half of work. This prints that statement after
> > three attempts but lets you enter password more time.
> >
> >
> > Case II:
> > password = "foobar"
> > attempt = 0
> > while password != "unicorn":
> >     password = raw_input("Password: ")
> >     attempt = attempt + 1
> >     if attempt == 3:
> >         print "That must have been complicated.."
> >         break
> > print "Welcome in.."
> move the print "Welcome under break:
>           break
>           print "Welcome..."
> 
> > # This script performs better than first one, but the problem is it
> > welcomes you even if you haven't entered the password correct.
> >
> > Please tell, what can I do?
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > http://mail.python.org/mailman/listinfo/tutor
> 
> 
> 
> -- 
> Joel Goldstick
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

I was able to get it working like this...

password = "foobar"
attempt = 0
while (password != "unicorn") and (attempt <= 3):
    password = input("Password: ")
    attempt = attempt + 1
    if attempt == 3:
        print ("That must have been complicated..")
        break

if (password == "unicorn"):
    print ("Welcome in..")


Not sure if it helps.

- Paul
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120717/ba871ef3/attachment.html>

From bgailer at gmail.com  Tue Jul 17 18:11:48 2012
From: bgailer at gmail.com (bob gailer)
Date: Tue, 17 Jul 2012 12:11:48 -0400
Subject: [Tutor] How to print something just after 3 attempts?
In-Reply-To: <CAE7MaQYvjy6G5cXn5-1F4OZh-c9-+m=RVV09Jy_dx+RpRK307A@mail.gmail.com>
References: <CAE7MaQYvjy6G5cXn5-1F4OZh-c9-+m=RVV09Jy_dx+RpRK307A@mail.gmail.com>
Message-ID: <50058EC4.8090801@gmail.com>

Simpler solution:

for i in range(3):
   if input("Password: ") == "unicorn":
     print("Welcome in..")
     break
else:
   print("That must have been complicated..")

-- 
Bob Gailer
919-636-4239
Chapel Hill NC


From breamoreboy at yahoo.co.uk  Tue Jul 17 18:31:07 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 17 Jul 2012 17:31:07 +0100
Subject: [Tutor] How to print something just after 3 attempts?
In-Reply-To: <SNT115-W2362769CE138886889A510B0DB0@phx.gbl>
References: <CAE7MaQYvjy6G5cXn5-1F4OZh-c9-+m=RVV09Jy_dx+RpRK307A@mail.gmail.com>,
	<CAPM-O+ystCbpoT85Ce-44iNjciFsfCCu1j_U1aeitWSP8eibrA@mail.gmail.com>
	<SNT115-W2362769CE138886889A510B0DB0@phx.gbl>
Message-ID: <ju43uc$vhl$1@dough.gmane.org>

On 17/07/2012 16:28, Paul McNally wrote:
>
>
>
>> Date: Tue, 17 Jul 2012 11:23:07 -0400
>> From: joel.goldstick at gmail.com
>> To: sntshkmr60 at gmail.com
>> CC: tutor at python.org
>> Subject: Re: [Tutor] How to print something just after 3 attempts?
>>
>> On Tue, Jul 17, 2012 at 11:10 AM, Santosh Kumar <sntshkmr60 at gmail.com> wrote:
>>> Hello There,
>>>
>>> This problem isn't so simple as it looks in the title/subject of this email.
>>>
>>> I am doing a tutorial on Python, and there is a script named password.py:
>>>
>>> password = "foobar"
>>>
>>> while password != "unicorn":
>>>      password = raw_input("Password: ")
>>> print "Welcome in"
>>>
>>>
>>> The question says "Modify the password guessing program to keep track
>>> of how many times the user has entered the password wrong. If it is
>>> more than 3 times, print ?That must have been complicated.?
>>>
>>> So I did, I did it in two ways.
>>> Case I:
>>> password = "foobar"
>>> attempt = 0
>>> while password != "unicorn":
>>>      password = raw_input("Password: ")
>>>      attempt = attempt + 1
>>>      if attempt == 3:
>>>          print "That must have been complicated.."
>>>
>>> print "Welcome in.."
>>> # This script does the half of work. This prints that statement after
>>> three attempts but lets you enter password more time.
>>>
>>>
>>> Case II:
>>> password = "foobar"
>>> attempt = 0
>>> while password != "unicorn":
>>>      password = raw_input("Password: ")
>>>      attempt = attempt + 1
>>>      if attempt == 3:
>>>          print "That must have been complicated.."
>>>          break
>>> print "Welcome in.."
>> move the print "Welcome under break:
>>            break
>>            print "Welcome..."
>>
>>> # This script performs better than first one, but the problem is it
>>> welcomes you even if you haven't entered the password correct.
>>>
>>> Please tell, what can I do?
>>> _______________________________________________
>>> Tutor maillist  -  Tutor at python.org
>>> To unsubscribe or change subscription options:
>>> http://mail.python.org/mailman/listinfo/tutor
>>
>>
>>
>> --
>> Joel Goldstick
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/mailman/listinfo/tutor
>
> I was able to get it working like this...
>
> password = "foobar"
> attempt = 0
> while (password != "unicorn") and (attempt <= 3):

Please we're talking Python here not C so strip out those unneeded 
parenthesis.

>      password = input("Password: ")
>      attempt = attempt + 1
>      if attempt == 3:
>          print ("That must have been complicated..")
>          break
>
> if (password == "unicorn"):

ditto.

>      print ("Welcome in..")
>
>
> Not sure if it helps.
>
> - Paul
>   		 	   		
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
-- 
Cheers.

Mark Lawrence.




From brian.van.den.broek at gmail.com  Tue Jul 17 18:51:00 2012
From: brian.van.den.broek at gmail.com (Brian van den Broek)
Date: Tue, 17 Jul 2012 12:51:00 -0400
Subject: [Tutor] How to print something just after 3 attempts?
In-Reply-To: <ju43uc$vhl$1@dough.gmane.org>
References: <CAE7MaQYvjy6G5cXn5-1F4OZh-c9-+m=RVV09Jy_dx+RpRK307A@mail.gmail.com>
	<CAPM-O+ystCbpoT85Ce-44iNjciFsfCCu1j_U1aeitWSP8eibrA@mail.gmail.com>
	<SNT115-W2362769CE138886889A510B0DB0@phx.gbl>
	<ju43uc$vhl$1@dough.gmane.org>
Message-ID: <CAF6DajLN3yJAUT4_u6-fNOCgtrndB6tTm70JGZVzCCYZxNz9kA@mail.gmail.com>

On 17 Jul 2012 12:39, "Mark Lawrence" <breamoreboy at yahoo.co.uk> wrote:
>
> On 17/07/2012 16:28, Paul McNally wrote:

<snip enough context to perhaps have lost attributions>

>> I was able to get it working like this...
>>
>> password = "foobar"
>> attempt = 0
>> while (password != "unicorn") and (attempt <= 3):
>
>
> Please we're talking Python here not C so strip out those unneeded
parenthesis.

Not so sure I agree. Python doesn't need them, but I often find code easier
to parse when the scope of binary operators is made clear via parens. The
precedence order of python is just one more thing to recall (and is
fighting for space in my head with precedence orders for a bunch of formal
languages). A bit of extra typing can save a second or cognitive lag each
time the line is read.

Best,

Brian vdB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120717/a7d3773d/attachment.html>

From breamoreboy at yahoo.co.uk  Tue Jul 17 19:19:55 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 17 Jul 2012 18:19:55 +0100
Subject: [Tutor] How to print something just after 3 attempts?
In-Reply-To: <CAF6DajLN3yJAUT4_u6-fNOCgtrndB6tTm70JGZVzCCYZxNz9kA@mail.gmail.com>
References: <CAE7MaQYvjy6G5cXn5-1F4OZh-c9-+m=RVV09Jy_dx+RpRK307A@mail.gmail.com>
	<CAPM-O+ystCbpoT85Ce-44iNjciFsfCCu1j_U1aeitWSP8eibrA@mail.gmail.com>
	<SNT115-W2362769CE138886889A510B0DB0@phx.gbl>
	<ju43uc$vhl$1@dough.gmane.org>
	<CAF6DajLN3yJAUT4_u6-fNOCgtrndB6tTm70JGZVzCCYZxNz9kA@mail.gmail.com>
Message-ID: <ju46ri$oof$1@dough.gmane.org>

On 17/07/2012 17:51, Brian van den Broek wrote:
> On 17 Jul 2012 12:39, "Mark Lawrence" <breamoreboy at yahoo.co.uk> wrote:
>>
>> On 17/07/2012 16:28, Paul McNally wrote:
>
> <snip enough context to perhaps have lost attributions>
>
>>> I was able to get it working like this...
>>>
>>> password = "foobar"
>>> attempt = 0
>>> while (password != "unicorn") and (attempt <= 3):
>>
>>
>> Please we're talking Python here not C so strip out those unneeded
> parenthesis.
>
> Not so sure I agree. Python doesn't need them, but I often find code easier
> to parse when the scope of binary operators is made clear via parens. The
> precedence order of python is just one more thing to recall (and is
> fighting for space in my head with precedence orders for a bunch of formal
> languages). A bit of extra typing can save a second or cognitive lag each
> time the line is read.
>
> Best,
>
> Brian vdB
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

I think we'll have to agree to disagree.  I just hate having to use my 
Mk I eyeballs parsing something that I regard as noise.

-- 
Cheers.

Mark Lawrence.




From wprins at gmail.com  Tue Jul 17 21:47:39 2012
From: wprins at gmail.com (Walter Prins)
Date: Tue, 17 Jul 2012 20:47:39 +0100
Subject: [Tutor] How to print something just after 3 attempts?
In-Reply-To: <CAE7MaQYvjy6G5cXn5-1F4OZh-c9-+m=RVV09Jy_dx+RpRK307A@mail.gmail.com>
References: <CAE7MaQYvjy6G5cXn5-1F4OZh-c9-+m=RVV09Jy_dx+RpRK307A@mail.gmail.com>
Message-ID: <CANLXbfA-P=F-EtByqTYs062TywWLiudimT+XCjRG3gNd98EDUw@mail.gmail.com>

Hi Santosh,

On 17 July 2012 16:10, Santosh Kumar <sntshkmr60 at gmail.com> wrote:
> The question says "Modify the password guessing program to keep track
> of how many times the user has entered the password wrong. If it is
> more than 3 times, print ?That must have been complicated.?

FWIW, I interpret the above to mean that the program should continue
to loop until the correct password is entered (or until stopped with
Ctrl-C), e.g.  just like the original, but with this twist, namely
that if the user entered a password more than 3 times (e.g. 4 *or
more* attempts) trying to authenticate, then in addition (prior to)
the "Welcome" message, it prints the "That must've been hard" message.
 This seems a slightly more parsimonious interpretation of the
question IMHO.  Under this interpretation, the following will do:


# Waits until a password has been entered.  Use control-C to break out with out
# the password

password = "foobar"
attempts = 0
while password != "unicorn":
    password = raw_input("Password:")
    attempts += 1

if attempts > 3: print "That must have been complicated..."
print "Welcome in"

For reference, I tracked down the tutorial and original program here:
http://ccis.athabascau.ca/html/courses/comp200n/python/node6.html

Hope that helps,

Walter

From sntshkmr60 at gmail.com  Wed Jul 18 07:09:54 2012
From: sntshkmr60 at gmail.com (Santosh Kumar)
Date: Wed, 18 Jul 2012 10:39:54 +0530
Subject: [Tutor] Why isn't my simple if elif script not working?
Message-ID: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>

Here is my script:

name = raw_input("What's your name? ")

if name == "Santosh":
    print "Hey!! I have the same name."
elif name == "John Cleese" or "Michael Palin":
    print "I have no preference about your name. Really!!"
else:
    print "You have a nice name."


The if part works well. The elif part works well too. The problem is
even if you enter strings other than "Santosh", "John Cleese" and
"Michael Palin" you still get the print from elif part, not from else
part.

From kala586 at gmail.com  Wed Jul 18 07:11:55 2012
From: kala586 at gmail.com (kala Vinay)
Date: Wed, 18 Jul 2012 10:41:55 +0530
Subject: [Tutor] How to print something just after 3 attempts?
Message-ID: <CAL+b0G9mwJoFtM-=s-f2nOStycK4n-vO_e9EKjhhRy5WHVaxdA@mail.gmail.com>

>
> On Tue, Jul 17, 2012 at 11:10 AM, Santosh Kumar <sntshkmr60 at gmail.com>
> wrote:
> > Hello There,
> >
> > This problem isn't so simple as it looks in the title/subject of this
> email.
> >
> > I am doing a tutorial on Python, and there is a script named password.py:
> >
> > password = "foobar"
> >
> > while password != "unicorn":
> >     password = raw_input("Password: ")
> > print "Welcome in"
> >
> >
> > The question says "Modify the password guessing program to keep track
> > of how many times the user has entered the password wrong. If it is
> > more than 3 times, print ?That must have been complicated.?
> >
> > So I did, I did it in two ways.
> > Case I:
> > password = "foobar"
> > attempt = 0
> > while password != "unicorn":
> >     password = raw_input("Password: ")
> >     attempt = attempt + 1
> >     if attempt == 3:
> >         print "That must have been complicated.."
> >
> > print "Welcome in.."
> > # This script does the half of work. This prints that statement after
> > three attempts but lets you enter password more time.
> >
> >
> > Case II:
> > password = "foobar"
> > attempt = 0
> > while password != "unicorn":
> >     password = raw_input("Password: ")
> >     attempt = attempt + 1
> >     if attempt == 3:
> >         print "That must have been complicated.."
> >         break
> > print "Welcome in.."
> move the print "Welcome under break:
>           break
>           print "Welcome..."
>
> > # This script performs better than first one, but the problem is it
> > welcomes you even if you haven't entered the password correct.
> >
> > Please tell, what can I do?
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > http://mail.python.org/mailman/listinfo/tutor
>
>
>
> --
> Joel Goldstick
>
>
> ------------------------------
>

    Hello Joel,

                You can also use getpass() method to read the password, it
will not display the entered characters on screen.

import getpass
attempt=0
for attempt in range(3):
    pw=getpass.getpass()
    if pw=="unicorn":
        print "welcome in.. "
        break
else:
    print "That must have been complicated.. "


-- 
Regards,
Kala
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120718/e14e464e/attachment.html>

From walksloud at gmail.com  Wed Jul 18 07:21:50 2012
From: walksloud at gmail.com (Andre' Walker-Loud)
Date: Tue, 17 Jul 2012 22:21:50 -0700
Subject: [Tutor] Why isn't my simple if elif script not working?
In-Reply-To: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
References: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
Message-ID: <73E9C6F8-A9D3-4FA6-98F6-57BFEA76D993@gmail.com>

Hi Santosh,

On Jul 17, 2012, at 10:09 PM, Santosh Kumar wrote:

> Here is my script:
> 
> name = raw_input("What's your name? ")
> 
> if name == "Santosh":
>    print "Hey!! I have the same name."
> elif name == "John Cleese" or "Michael Palin":
>    print "I have no preference about your name. Really!!"
> else:
>    print "You have a nice name."
> 
> 
> The if part works well. The elif part works well too. The problem is
> even if you enter strings other than "Santosh", "John Cleese" and
> "Michael Palin" you still get the print from elif part, not from else
> part.

you just have to be careful with the multiple boolean line in the elif.  You can use either

elif name == ("John Cleese" or "Michael Palin"):

or

elif name == "John Cleese" or name == "Michael Palin":


With your elif line, it is asking "does name ==  John Cleese" or "Michael Palin", and so if the name is not John Cleese, then I believe it prints "Michael Palin" while reporting True, so satisfying the elif clause.


Cheers,

Andre

From alexandre.zani at gmail.com  Wed Jul 18 07:23:28 2012
From: alexandre.zani at gmail.com (Alexandre Zani)
Date: Tue, 17 Jul 2012 22:23:28 -0700
Subject: [Tutor] Why isn't my simple if elif script not working?
In-Reply-To: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
References: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
Message-ID: <CAJVMpooLgscKopC706kECMJCD5Obs2nNnGP2mYbNyCAGdW_G9w@mail.gmail.com>

On Tue, Jul 17, 2012 at 10:09 PM, Santosh Kumar <sntshkmr60 at gmail.com> wrote:
> Here is my script:
>
> name = raw_input("What's your name? ")
>
> if name == "Santosh":
>     print "Hey!! I have the same name."
> elif name == "John Cleese" or "Michael Palin":
>     print "I have no preference about your name. Really!!"
> else:
>     print "You have a nice name."
>
>
> The if part works well. The elif part works well too. The problem is
> even if you enter strings other than "Santosh", "John Cleese" and
> "Michael Palin" you still get the print from elif part, not from else
> part.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

Think of it this way: The or operator doesn't look at both sides at
the same time. It looks at one side, and then at the other. So on one
side it looks and sees (name == "John Cleese") and decides whether
that is true or not. Then it looks at the other side and sees
("Michael Palin") which is always true because a non-empty string is
always true. It doesn't see "Michael Palin" in the context of name ==
"John Cleese". It sees and treats "Michael Palin" independently. So
because "Michael Palin" is always True, the elif clause is always
true.

What you want to write is this:

elif name == "John Cleese" or name == "Michael Palin":

Alex

From alexandre.zani at gmail.com  Wed Jul 18 07:29:26 2012
From: alexandre.zani at gmail.com (Alexandre Zani)
Date: Tue, 17 Jul 2012 22:29:26 -0700
Subject: [Tutor] Why isn't my simple if elif script not working?
In-Reply-To: <73E9C6F8-A9D3-4FA6-98F6-57BFEA76D993@gmail.com>
References: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
	<73E9C6F8-A9D3-4FA6-98F6-57BFEA76D993@gmail.com>
Message-ID: <CAJVMpooQZ+6KXyaPY7uQwLMn0e+SwFJ3yyQAKO1yJVe3i9Q7cA@mail.gmail.com>

On Tue, Jul 17, 2012 at 10:21 PM, Andre' Walker-Loud
<walksloud at gmail.com> wrote:
> Hi Santosh,
>
> On Jul 17, 2012, at 10:09 PM, Santosh Kumar wrote:
>
>> Here is my script:
>>
>> name = raw_input("What's your name? ")
>>
>> if name == "Santosh":
>>    print "Hey!! I have the same name."
>> elif name == "John Cleese" or "Michael Palin":
>>    print "I have no preference about your name. Really!!"
>> else:
>>    print "You have a nice name."
>>
>>
>> The if part works well. The elif part works well too. The problem is
>> even if you enter strings other than "Santosh", "John Cleese" and
>> "Michael Palin" you still get the print from elif part, not from else
>> part.
>
> you just have to be careful with the multiple boolean line in the elif.  You can use either
>
> elif name == ("John Cleese" or "Michael Palin"):

That won't work.

>>> "John Cleese" == ("John Cleese" or "Michael Palin")
True
>>> "Michael Palin" == ("John Cleese" or "Michael Palin")
False
>>> ("John Cleese" or "Michael Palin")
'John Cleese'

Python will look at the expression ("John Cleese" or "Michael Palin")
since bool("John Cleese") is True, the expression immediately
evaluates to "John Cleese" and the elif clause becomes equivalent to
name == "John Cleese"

>
> or
>
> elif name == "John Cleese" or name == "Michael Palin":
>
>
> With your elif line, it is asking "does name ==  John Cleese" or "Michael Palin", and so if the name is not John Cleese, then I believe it prints "Michael Palin" while reporting True, so satisfying the elif clause.
>
>
> Cheers,
>
> Andre
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From walksloud at gmail.com  Wed Jul 18 07:31:45 2012
From: walksloud at gmail.com (Andre' Walker-Loud)
Date: Tue, 17 Jul 2012 22:31:45 -0700
Subject: [Tutor] Why isn't my simple if elif script not working?
In-Reply-To: <CAJVMpooQZ+6KXyaPY7uQwLMn0e+SwFJ3yyQAKO1yJVe3i9Q7cA@mail.gmail.com>
References: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
	<73E9C6F8-A9D3-4FA6-98F6-57BFEA76D993@gmail.com>
	<CAJVMpooQZ+6KXyaPY7uQwLMn0e+SwFJ3yyQAKO1yJVe3i9Q7cA@mail.gmail.com>
Message-ID: <11A0D108-8FCC-472A-ADEF-00ED8003E1C5@gmail.com>

> 
> On Jul 17, 2012, at 10:29 PM, Alexandre Zani wrote:
> 
> 
> On Tue, Jul 17, 2012 at 10:21 PM, Andre' Walker-Loud
> <walksloud at gmail.com> wrote:
>> Hi Santosh,
>> 
>> On Jul 17, 2012, at 10:09 PM, Santosh Kumar wrote:
>> 
>>> Here is my script:
>>> 
>>> name = raw_input("What's your name? ")
>>> 
>>> if name == "Santosh":
>>>   print "Hey!! I have the same name."
>>> elif name == "John Cleese" or "Michael Palin":
>>>   print "I have no preference about your name. Really!!"
>>> else:
>>>   print "You have a nice name."
>>> 
>>> 
>>> The if part works well. The elif part works well too. The problem is
>>> even if you enter strings other than "Santosh", "John Cleese" and
>>> "Michael Palin" you still get the print from elif part, not from else
>>> part.
>> 
>> you just have to be careful with the multiple boolean line in the elif.  You can use either
>> 
>> elif name == ("John Cleese" or "Michael Palin"):
> 
> That won't work.
> 
>>>> "John Cleese" == ("John Cleese" or "Michael Palin")
> True
>>>> "Michael Palin" == ("John Cleese" or "Michael Palin")
> False
>>>> ("John Cleese" or "Michael Palin")
> 'John Cleese'
> 
> Python will look at the expression ("John Cleese" or "Michael Palin")
> since bool("John Cleese") is True, the expression immediately
> evaluates to "John Cleese" and the elif clause becomes equivalent to
> name == "John Cleese"

thanks - I fell into the same trap!  I tried it in my terminal but it worked for the same reason as the OP.

Andre






From rzzzwilson at gmail.com  Wed Jul 18 07:54:20 2012
From: rzzzwilson at gmail.com (Ross Wilson)
Date: Wed, 18 Jul 2012 15:54:20 +1000
Subject: [Tutor] Why isn't my simple if elif script not working?
In-Reply-To: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
References: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
Message-ID: <50064F8C.7080005@gmail.com>



On 18/07/12 15:09, Santosh Kumar wrote:
> Here is my script:
>
> name = raw_input("What's your name? ")
>
> if name == "Santosh":
>      print "Hey!! I have the same name."
> elif name == "John Cleese" or "Michael Palin":
>      print "I have no preference about your name. Really!!"
> else:
>      print "You have a nice name."
>
>
> The if part works well. The elif part works well too. The problem is
> even if you enter strings other than "Santosh", "John Cleese" and
> "Michael Palin" you still get the print from elif part, not from else
> part.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

Note that you *can* do:

     if name in ("John Cleese", "Michael Palin"):

Ross

From steve at pearwood.info  Wed Jul 18 15:09:40 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 18 Jul 2012 23:09:40 +1000
Subject: [Tutor] Why isn't my simple if elif script not working?
In-Reply-To: <CAJVMpooLgscKopC706kECMJCD5Obs2nNnGP2mYbNyCAGdW_G9w@mail.gmail.com>
References: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
	<CAJVMpooLgscKopC706kECMJCD5Obs2nNnGP2mYbNyCAGdW_G9w@mail.gmail.com>
Message-ID: <5006B594.4050005@pearwood.info>

Alexandre Zani wrote:

> What you want to write is this:
> 
> elif name == "John Cleese" or name == "Michael Palin":

elif name in ("John Cleese", "Michael Palin"):

is better.


-- 
Steven


From alexandre.zani at gmail.com  Wed Jul 18 16:34:59 2012
From: alexandre.zani at gmail.com (Alexandre Zani)
Date: Wed, 18 Jul 2012 07:34:59 -0700
Subject: [Tutor] Why isn't my simple if elif script not working?
In-Reply-To: <5006B594.4050005@pearwood.info>
References: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
	<CAJVMpooLgscKopC706kECMJCD5Obs2nNnGP2mYbNyCAGdW_G9w@mail.gmail.com>
	<5006B594.4050005@pearwood.info>
Message-ID: <CAJVMpopAEPqzL78RpZ2cCOv5P4zuErMctWFA0VspoRc_RgtwjQ@mail.gmail.com>

On Wed, Jul 18, 2012 at 6:09 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Alexandre Zani wrote:
>
>> What you want to write is this:
>>
>> elif name == "John Cleese" or name == "Michael Palin":
>
>
> elif name in ("John Cleese", "Michael Palin"):
>
> is better.
>
>
> --
> Steven
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

Better how?

From steve at pearwood.info  Wed Jul 18 16:44:50 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 19 Jul 2012 00:44:50 +1000
Subject: [Tutor] Why isn't my simple if elif script not working?
In-Reply-To: <CAJVMpopAEPqzL78RpZ2cCOv5P4zuErMctWFA0VspoRc_RgtwjQ@mail.gmail.com>
References: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
	<CAJVMpooLgscKopC706kECMJCD5Obs2nNnGP2mYbNyCAGdW_G9w@mail.gmail.com>
	<5006B594.4050005@pearwood.info>
	<CAJVMpopAEPqzL78RpZ2cCOv5P4zuErMctWFA0VspoRc_RgtwjQ@mail.gmail.com>
Message-ID: <5006CBE2.3060000@pearwood.info>

Alexandre Zani wrote:
> On Wed, Jul 18, 2012 at 6:09 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> Alexandre Zani wrote:
>>
>>> What you want to write is this:
>>>
>>> elif name == "John Cleese" or name == "Michael Palin":
>>
>> elif name in ("John Cleese", "Michael Palin"):
>>
>> is better.

> Better how?


It's shorter, you don't have to repeat the reference to `name` twice, it is 
more easily extensible if you add additional names, and it is likely to be a 
*tiny* bit faster -- it moves the equality test out of pure-Python code into a 
tuple method, which will be written in C.



-- 
Steven

From alexandre.zani at gmail.com  Wed Jul 18 19:01:30 2012
From: alexandre.zani at gmail.com (Alexandre Zani)
Date: Wed, 18 Jul 2012 10:01:30 -0700
Subject: [Tutor] Why isn't my simple if elif script not working?
In-Reply-To: <5006CBE2.3060000@pearwood.info>
References: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
	<CAJVMpooLgscKopC706kECMJCD5Obs2nNnGP2mYbNyCAGdW_G9w@mail.gmail.com>
	<5006B594.4050005@pearwood.info>
	<CAJVMpopAEPqzL78RpZ2cCOv5P4zuErMctWFA0VspoRc_RgtwjQ@mail.gmail.com>
	<5006CBE2.3060000@pearwood.info>
Message-ID: <CAJVMpopd25AewJrMEuvP+R4iRaNZ_TjEbEGkDWHZ1dv-0pfesQ@mail.gmail.com>

On Wed, Jul 18, 2012 at 7:44 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Alexandre Zani wrote:
>>
>> On Wed, Jul 18, 2012 at 6:09 AM, Steven D'Aprano <steve at pearwood.info>
>> wrote:
>>>
>>> Alexandre Zani wrote:
>>>
>>>> What you want to write is this:
>>>>
>>>> elif name == "John Cleese" or name == "Michael Palin":
>>>
>>>
>>> elif name in ("John Cleese", "Michael Palin"):
>>>
>>> is better.
>
>
>> Better how?
>
>
>
> It's shorter, you don't have to repeat the reference to `name` twice, it is
> more easily extensible if you add additional names, and it is likely to be a
> *tiny* bit faster -- it moves the equality test out of pure-Python code into
> a tuple method, which will be written in C.
>

Wadayano... You're right, it is faster!

Thanks.


>
>
>
> --
> Steven
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From aditipai1227 at gmail.com  Wed Jul 18 19:10:51 2012
From: aditipai1227 at gmail.com (Aditi Pai)
Date: Wed, 18 Jul 2012 13:10:51 -0400
Subject: [Tutor] writing function changeColor
Message-ID: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>

Hello,

I am trying to write a function changeColor for an assignment. Two things
that I am unsure about for this assignment are how to assign different
colors to integers so that, red will be 1, blue will be 2, etc. Also, I
learned python so that if I were to put in 0.9, it'd decrease red by 10%.
The way this function needs to be written, -0.1 decreases red by 10%

Where can I find information on these two topics?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120718/b8bfba30/attachment.html>

From emile at fenx.com  Wed Jul 18 20:05:07 2012
From: emile at fenx.com (Emile van Sebille)
Date: Wed, 18 Jul 2012 11:05:07 -0700
Subject: [Tutor] writing function changeColor
In-Reply-To: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
Message-ID: <ju6tsm$ckv$1@dough.gmane.org>

On 7/18/2012 10:10 AM Aditi Pai said...
> Hello,
>
> I am trying to write a function changeColor for an assignment.

I'd look in the documentation of whatever it is you're trying to change 
the color of.  It should be explained there how to set and change colors.

You're going to end up with something that could look like:

def change_color_to(new_color):
     color_of_thing = new_color

def modify_color_of_thing(by_n_pct):
     color_of_thing = old_color_of_thing * by_n_pct

where the docs of the thing you're changing provide the names and 
handling of modifying the color.

HTH,

Emile


> Two
> things that I am unsure about for this assignment are how to assign
> different colors to integers so that, red will be 1, blue will be 2,
> etc. Also, I learned python so that if I were to put in 0.9, it'd
> decrease red by 10%. The way this function needs to be written, -0.1
> decreases red by 10%
>
> Where can I find information on these two topics?
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>




From aditipai1227 at gmail.com  Wed Jul 18 21:07:03 2012
From: aditipai1227 at gmail.com (Aditi Pai)
Date: Wed, 18 Jul 2012 15:07:03 -0400
Subject: [Tutor] writing function changeColor
In-Reply-To: <ju6tsm$ckv$1@dough.gmane.org>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
	<ju6tsm$ckv$1@dough.gmane.org>
Message-ID: <CAF_BoirqULGAWo-UTJud-YVeznD-NV-22jfckpE_ZD6aQY9w6A@mail.gmail.com>

Hey Emile! Thanks for the advice. I think maybe I should have combined the
two parts of my email. The function is just called changeColor. I don't
actually want to change the color as much as alter the color. I'm working
off of this example:

def decreaseRed(picture):
for p in getPixels(picture):
value=getRed(p)
setRed(p,value*0.5)

Except for me, I am asked to also make the argument so that it accepts an
integer and a float. Not sure how to assign different colors and do that.

On Wed, Jul 18, 2012 at 2:05 PM, Emile van Sebille <emile at fenx.com> wrote:

> On 7/18/2012 10:10 AM Aditi Pai said...
>
>  Hello,
>>
>> I am trying to write a function changeColor for an assignment.
>>
>
> I'd look in the documentation of whatever it is you're trying to change
> the color of.  It should be explained there how to set and change colors.
>
> You're going to end up with something that could look like:
>
> def change_color_to(new_color):
>     color_of_thing = new_color
>
> def modify_color_of_thing(by_n_**pct):
>     color_of_thing = old_color_of_thing * by_n_pct
>
> where the docs of the thing you're changing provide the names and handling
> of modifying the color.
>
> HTH,
>
> Emile
>
>
>  Two
>> things that I am unsure about for this assignment are how to assign
>> different colors to integers so that, red will be 1, blue will be 2,
>> etc. Also, I learned python so that if I were to put in 0.9, it'd
>> decrease red by 10%. The way this function needs to be written, -0.1
>> decreases red by 10%
>>
>> Where can I find information on these two topics?
>>
>>
>> ______________________________**_________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>>
>>
>
>
> ______________________________**_________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120718/b935aff2/attachment-0001.html>

From bgailer at gmail.com  Wed Jul 18 21:14:22 2012
From: bgailer at gmail.com (bob gailer)
Date: Wed, 18 Jul 2012 15:14:22 -0400
Subject: [Tutor] writing function changeColor
In-Reply-To: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
Message-ID: <50070B0E.6030607@gmail.com>

On 7/18/2012 1:10 PM, Aditi Pai wrote:
> Hello,
>
> I am trying to write a function changeColor for an assignment.
Is this a class assignment or job-related?

Point us to the assignment, Learn to ask meaningful questions. Otherwise 
you waste your time and ours.
> "assign different colors to integers so that, red will be 1, blue will 
> be 2, etc."
We need a LOT more detail here. color of what? What does etc mean 
(implies we should be able to figure out what comes next) Could be anything!
> Also, I learned python so that if I were to put in 0.9, it'd decrease 
> red by 10%.
Makes no sense.
> The way this function needs to be written, -0.1 decreases red by 10%
>
> Where can I find information on these two topics?
Google?

-- 
Bob Gailer
919-636-4239
Chapel Hill NC


From aditipai1227 at gmail.com  Wed Jul 18 21:20:16 2012
From: aditipai1227 at gmail.com (Aditi Pai)
Date: Wed, 18 Jul 2012 15:20:16 -0400
Subject: [Tutor] writing function changeColor
In-Reply-To: <50070B0E.6030607@gmail.com>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
	<50070B0E.6030607@gmail.com>
Message-ID: <CAF_Boirv8_jXcrbPe7DuCSHRFRy98GVhjnfyskvih3SzuCq1ug@mail.gmail.com>

Dear Bob,

I am trying my best to learn and your attitude was not welcome. If you
don't want to answer my question, I understand. I'm doing research
elsewhere too and because I am new at this, my familiarity with key terms
and search words is also improving, yet at a slower pace. My goal is to
find people who would help me find the resources I need and not cheat,
which is why I didn't want to post my assignment.

Thanks,
Aditi Pai

On Wed, Jul 18, 2012 at 3:14 PM, bob gailer <bgailer at gmail.com> wrote:

> On 7/18/2012 1:10 PM, Aditi Pai wrote:
>
>> Hello,
>>
>> I am trying to write a function changeColor for an assignment.
>>
> Is this a class assignment or job-related?
>
> Point us to the assignment, Learn to ask meaningful questions. Otherwise
> you waste your time and ours.
>
>  "assign different colors to integers so that, red will be 1, blue will be
>> 2, etc."
>>
> We need a LOT more detail here. color of what? What does etc mean (implies
> we should be able to figure out what comes next) Could be anything!
>
>  Also, I learned python so that if I were to put in 0.9, it'd decrease red
>> by 10%.
>>
> Makes no sense.
>
>  The way this function needs to be written, -0.1 decreases red by 10%
>>
>> Where can I find information on these two topics?
>>
> Google?
>
> --
> Bob Gailer
> 919-636-4239
> Chapel Hill NC
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120718/5763cf56/attachment.html>

From emile at fenx.com  Wed Jul 18 21:20:35 2012
From: emile at fenx.com (Emile van Sebille)
Date: Wed, 18 Jul 2012 12:20:35 -0700
Subject: [Tutor] writing function changeColor
In-Reply-To: <CAF_BoirqULGAWo-UTJud-YVeznD-NV-22jfckpE_ZD6aQY9w6A@mail.gmail.com>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
	<ju6tsm$ckv$1@dough.gmane.org>
	<CAF_BoirqULGAWo-UTJud-YVeznD-NV-22jfckpE_ZD6aQY9w6A@mail.gmail.com>
Message-ID: <ju72a6$i77$1@dough.gmane.org>

On 7/18/2012 12:07 PM Aditi Pai said...
> Hey Emile! Thanks for the advice. I think maybe I should have combined
> the two parts of my email. The function is just called changeColor. I
> don't actually want to change the color as much as alter the color.

I'm not sure there's a difference...

> I'm working off of this example:
>
> def decreaseRed(picture):
>     for p in getPixels(picture):
>         value=getRed(p)
>         setRed(p,value*0.5)
>
> Except for me, I am asked to also make the argument so that it accepts
> an integer and a float.

So, if by that you mean that the function should accept two passed 
values, one an integer and the other a float, then your def line should 
look like:

def changeColor(myint, myfloat):

If, on the other hand, you want a single passed value that can be either 
an integer or a float you'd just have:

def changeColor(myiint):


> Not sure how to assign different colors and do that.


If the problem if adapting the example so that the function should 
accept both the picture thingy and a value to replace the hardcoded .5, 
your function def might look like:

def changeColor(picture, myRedIntOrFloatVal):

and then swap out the .5 for myRedIntOrFloatVal

What have you written?

Emile


From aditipai1227 at gmail.com  Wed Jul 18 21:29:29 2012
From: aditipai1227 at gmail.com (Aditi Pai)
Date: Wed, 18 Jul 2012 15:29:29 -0400
Subject: [Tutor] writing function changeColor
In-Reply-To: <ju72a6$i77$1@dough.gmane.org>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
	<ju6tsm$ckv$1@dough.gmane.org>
	<CAF_BoirqULGAWo-UTJud-YVeznD-NV-22jfckpE_ZD6aQY9w6A@mail.gmail.com>
	<ju72a6$i77$1@dough.gmane.org>
Message-ID: <CAF_BoiovE0e_K+vx7bNka-8Sp=6L9CncYO0cQWkqbLOR-qNOcA@mail.gmail.com>

Emile,

So far I have started with

def changeColor(pict,scale,color):

I was told to make different names for the float and integer (float = scale
and color= integer) and then I kept everything else the same, just to test
it out and see if that would work.

So it looks like this:

def changeColor(pict,scale,color):
  for p in getPixels(pict):
    value=getRed(p)
    setRed(p,value*.9)

Unfortunately I keep getting this error message:

The error was:changeColor() takes at least 3 arguments (1 given)
Inappropriate argument type.
An attempt was made to call a function with a parameter of an invalid type.
This means that you did something such as trying to pass a string to a
method that is expecting an integer.

On Wed, Jul 18, 2012 at 3:20 PM, Emile van Sebille <emile at fenx.com> wrote:

> On 7/18/2012 12:07 PM Aditi Pai said...
>
>  Hey Emile! Thanks for the advice. I think maybe I should have combined
>> the two parts of my email. The function is just called changeColor. I
>> don't actually want to change the color as much as alter the color.
>>
>
> I'm not sure there's a difference...
>
>
>  I'm working off of this example:
>>
>> def decreaseRed(picture):
>>     for p in getPixels(picture):
>>         value=getRed(p)
>>         setRed(p,value*0.5)
>>
>> Except for me, I am asked to also make the argument so that it accepts
>> an integer and a float.
>>
>
> So, if by that you mean that the function should accept two passed values,
> one an integer and the other a float, then your def line should look like:
>
> def changeColor(myint, myfloat):
>
> If, on the other hand, you want a single passed value that can be either
> an integer or a float you'd just have:
>
> def changeColor(myiint):
>
>
>
>  Not sure how to assign different colors and do that.
>>
>
>
> If the problem if adapting the example so that the function should accept
> both the picture thingy and a value to replace the hardcoded .5, your
> function def might look like:
>
> def changeColor(picture, myRedIntOrFloatVal):
>
> and then swap out the .5 for myRedIntOrFloatVal
>
> What have you written?
>
> Emile
>
>
> ______________________________**_________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120718/a960c872/attachment.html>

From ramit.prasad at jpmorgan.com  Wed Jul 18 22:04:43 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Wed, 18 Jul 2012 20:04:43 +0000
Subject: [Tutor] writing function changeColor
In-Reply-To: <CAF_Boirv8_jXcrbPe7DuCSHRFRy98GVhjnfyskvih3SzuCq1ug@mail.gmail.com>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
	<50070B0E.6030607@gmail.com>
	<CAF_Boirv8_jXcrbPe7DuCSHRFRy98GVhjnfyskvih3SzuCq1ug@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741657F5EC@SCACMX008.exchad.jpmchase.net>

Please do not top post.

> I am trying my best to learn and your attitude was not welcome. If you don't
> want to answer my question, I understand. I'm doing research elsewhere too and
> because I am new at this, my familiarity with key terms and search words is
> also improving, yet at a slower pace. My goal is to find people who would help
> me find the resources I need and not cheat, which is why I didn't want to post
> my assignment.

> > > I am trying to write a function changeColor for an assignment. Two things that I am unsure about for this assignment are how to assign different colors to integers so that, red will be 1, blue will be 2, etc. Also, I learned python so that if I were to put in 0.9, it'd decrease red by 10%. The way this function needs to be written, -0.1 decreases red by 10%

The problem is that you have given us an incomplete description of what
you need. Without a full description, it is difficult to answer without
making a host of assumptions that are probably all invalid. If you can 
provide a full description then we can help better. Even a code sample 
would give us some context. 

From your original email, I have no idea what UI framework you refer to
(if any), how you get a color or change a color or anything meaningful. 
Although, you might not like Bob's tone he makes a very valid point. 
If you learn to ask better questions, you get better answers.

Thankfully you ended up replying to Emile with some code so I can help
better.

> I was told to make different names for the float and integer (float = scale
> and color= integer) and then I kept everything else the same, just to test it
> out and see if that would work.
> 
> So it looks like this:
> 
> def changeColor(pict,scale,color):
>   for p in getPixels(pict):
>     value=getRed(p)
>     setRed(p,value*.9)
> 
> Unfortunately I keep getting this error message:
> 
> The error was:changeColor() takes at least 3 arguments (1 given)
> Inappropriate argument type.
> An attempt was made to call a function with a parameter of an invalid type.
> This means that you did something such as trying to pass a string to a method
> that is expecting an integer.

Please always post the exact trace back and not just a portion.
It might not seem like all that is needed is the error message
but it can make a big difference in the amount of help.

Luckily, I can tell what the problem is from this error. You are using 
`changeColor(picture)` which no longer works since you changed changeColor 
to take 3 arguments so you need to change each call to 
`changeColor(picture,scale,color)`.

Also, instead of saying getRed(p), I would do getColor(p, color).
Something like below; although, I leave the implementation of
getColor/setColor up to you. Oh, and I fixed your indentation problem.
(Could have been caused by posting in rich/HTML text and not plain text)

def changeColor( pict, scale color ):
''' Change the intensity of color.
scale can be -1.0 <= x <= 1.0
'''
    for p in getPixels(pict):
        value=getColor(p,color)
        change = 1.0 + scale # -.1 now equals .9 
        setColor(p, color, value * change )


Often in Python, you do not worry about whether a number
is a float or an int (unless it happens to be part of the assignment), 
you just do what you need and check that it worked.
You can wrap scale and color in float() or int() respectively
if you want to explicitly convert them.

It might make more sense to take in a % instead, so -10 for scale lessens the 
color by 10%. Lots of UI does things by this using 0.0-1.0 values (as given
in your example), so it makes sense, but it might be easier to think about 
as a percentage. 

Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From steve at pearwood.info  Wed Jul 18 22:20:35 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 19 Jul 2012 06:20:35 +1000
Subject: [Tutor] writing function changeColor
In-Reply-To: <ju72a6$i77$1@dough.gmane.org>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>	<ju6tsm$ckv$1@dough.gmane.org>	<CAF_BoirqULGAWo-UTJud-YVeznD-NV-22jfckpE_ZD6aQY9w6A@mail.gmail.com>
	<ju72a6$i77$1@dough.gmane.org>
Message-ID: <50071A93.7030108@pearwood.info>

Emile van Sebille wrote:
> On 7/18/2012 12:07 PM Aditi Pai said...
>> Hey Emile! Thanks for the advice. I think maybe I should have combined
>> the two parts of my email. The function is just called changeColor. I
>> don't actually want to change the color as much as alter the color.
> 
> I'm not sure there's a difference...


You're being too kind. Of course there isn't a difference. "Change" and 
"alter" are synonyms, so Aditi doesn't want to change the color, but change 
the color instead.

Aditi, that is not very helpful. You're asking us for help, but wasting our 
time with riddles.


-- 
Steven


From steve at pearwood.info  Wed Jul 18 22:42:12 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 19 Jul 2012 06:42:12 +1000
Subject: [Tutor] writing function changeColor
In-Reply-To: <CAF_BoiovE0e_K+vx7bNka-8Sp=6L9CncYO0cQWkqbLOR-qNOcA@mail.gmail.com>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>	<ju6tsm$ckv$1@dough.gmane.org>	<CAF_BoirqULGAWo-UTJud-YVeznD-NV-22jfckpE_ZD6aQY9w6A@mail.gmail.com>	<ju72a6$i77$1@dough.gmane.org>
	<CAF_BoiovE0e_K+vx7bNka-8Sp=6L9CncYO0cQWkqbLOR-qNOcA@mail.gmail.com>
Message-ID: <50071FA4.4060005@pearwood.info>

Aditi Pai wrote:
> Emile,
> 
> So far I have started with
> 
> def changeColor(pict,scale,color):
> 
> I was told to make different names for the float and integer (float = scale
> and color= integer) and then I kept everything else the same, just to test
> it out and see if that would work.
> 
> So it looks like this:
> 
> def changeColor(pict,scale,color):
>   for p in getPixels(pict):
>     value=getRed(p)
>     setRed(p,value*.9)
> 
> Unfortunately I keep getting this error message:
> 
> The error was:changeColor() takes at least 3 arguments (1 given)


Is this error message not clear? You have written a function that requires at 
least 3 values, but you have only given 1 value.

I can't tell you what argument you have given, because you have not told us 
what you have done to get this error message.

Aditi, we want to help you, but you're not giving us enough information to go 
on. You're wasting our time, and your own.

Instead of biting at poor Bob, who so far has given you the best and most 
valuable advise, you should pay attention to what we are trying to teach you. 
Would you rather that we just ignored you in silence?

I'm about to move away from the computer for an hour or so. When I come back, 
I will try to give you some more assistance. Or somebody else may try to help. 
In the meantime, I suggest you try reading this page and see if it enlightens you:

http://sscce.org/



-- 
Steven

From wolfrage8765 at gmail.com  Wed Jul 18 23:07:26 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Wed, 18 Jul 2012 23:07:26 +0200
Subject: [Tutor] string to binary and back... Python 3
Message-ID: <5007258E.6010206@gmail.com>

OK so I have been trying for a couple days now and I am throwing in the
towel, Python 3 wins this one.
I want to convert a string to binary and back again like in this
question: Stack Overflow: Convert Binary to ASCII and vice versa
(Python)
<http://stackoverflow.com/questions/7396849/convert-binary-to-ascii-and-vice-versa-python>
But in Python 3 I consistently get  some sort of error relating to the
fact that nothing but bytes and bytearrays support the buffer interface
or I get an overflow error because something is too large to be
converted to bytes.
Please help me and then explian what I am not getting that is new in
Python 3. I would like to point out I realize that binary, hex, and
encodings are all a very complex subject and so I do not expect to
master it but I do hope that I can gain a deeper insight. Thank you all.

test_script.py:
import binascii

test_int = 109

test_int = int(str(test_int) + '45670')
data = 'Testing XOR Again!'

while sys.getsizeof(data) > test_int.bit_length():

test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1), 'big')))

print('Bit Length: ' + str(test_int.bit_length()))

key = test_int # Yes I know this is an unnecessary step...

data = bin(int(binascii.hexlify(bytes(data, 'UTF-8')), 16))

print(data)

data = int(data, 2)

print(data)

data = binascii.unhexlify('%x' % data)


wolfrage at lm12-laptop02 ~/Projects $ python3 test_script.py
Bit Length: 134
0b10101000110010101110011011101000110100101101110011001110010000001011000010011110101001000100000010000010110011101100001011010010110111000100001
7351954002991226380810260999848996570230305
Traceback (most recent call last):
File "test_script.py", line 24, in <module>
data = binascii.unhexlify('%x' % data)
TypeError: 'str' does not support the buffer interface



test_script2.py:
import binascii
test_int = 109
test_int = int(str(test_int) + '45670')
data = 'Testing XOR Again!'
while sys.getsizeof(data) > test_int.bit_length():
test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1), 'big')))
print('Bit Length: ' + str(test_int.bit_length()))
key = test_int # Yes I know this is an unnecessary step...
data = bin(int(binascii.hexlify(bytes(data, 'UTF-8')), 16))
print(data)
data = int(data, 2)
print(data)
data = binascii.unhexlify(bytes(data, 'utf8'))



wolfrage at lm12-laptop02 ~/Projects $ python3 test_script2.py
Bit Length: 140
0b10101000110010101110011011101000110100101101110011001110010000001011000010011110101001000100000010000010110011101100001011010010110111000100001
7351954002991226380810260999848996570230305
Traceback (most recent call last):
File "test_script.py", line 24, in <module>
data = binascii.unhexlify(bytes(data, 'utf8'))
OverflowError: cannot fit 'int' into an index-sized integer


From jeanpierreda at gmail.com  Wed Jul 18 23:11:43 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Wed, 18 Jul 2012 17:11:43 -0400
Subject: [Tutor] writing function changeColor
In-Reply-To: <CAF_Boirv8_jXcrbPe7DuCSHRFRy98GVhjnfyskvih3SzuCq1ug@mail.gmail.com>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
	<50070B0E.6030607@gmail.com>
	<CAF_Boirv8_jXcrbPe7DuCSHRFRy98GVhjnfyskvih3SzuCq1ug@mail.gmail.com>
Message-ID: <CABicbJJ3POheYcM1EScbhe+MdbCTLbVbmPziHyQ+r-anAv-aqQ@mail.gmail.com>

On Wed, Jul 18, 2012 at 3:20 PM, Aditi Pai <aditipai1227 at gmail.com> wrote:
> I am trying my best to learn and your attitude was not welcome. If you don't
> want to answer my question, I understand. I'm doing research elsewhere too
> and because I am new at this, my familiarity with key terms and search words
> is also improving, yet at a slower pace. My goal is to find people who would
> help me find the resources I need and not cheat, which is why I didn't want
> to post my assignment.

If you don't post the assignment, we don't necessarily know what
advice would constitute cheating.

It's especially important for university students, who can be expelled
for helping other students cheat -- even if they didn't know it was
cheating and the cheater was in another country at another school.

-- Devin

From aditipai1227 at gmail.com  Wed Jul 18 23:31:00 2012
From: aditipai1227 at gmail.com (Aditi Pai)
Date: Wed, 18 Jul 2012 17:31:00 -0400
Subject: [Tutor] writing function changeColor
In-Reply-To: <50071FA4.4060005@pearwood.info>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
	<ju6tsm$ckv$1@dough.gmane.org>
	<CAF_BoirqULGAWo-UTJud-YVeznD-NV-22jfckpE_ZD6aQY9w6A@mail.gmail.com>
	<ju72a6$i77$1@dough.gmane.org>
	<CAF_BoiovE0e_K+vx7bNka-8Sp=6L9CncYO0cQWkqbLOR-qNOcA@mail.gmail.com>
	<50071FA4.4060005@pearwood.info>
Message-ID: <CAF_Boipe6vJ4DW8rerMJEE0mXgbe-axrjKYEHhUiHTs=1bWJ7Q@mail.gmail.com>

Ramit told me not to "top post," and I looked it up and I think I am doing
that same thing again, so if I am let me know how to avoid that. This kind
of discussion board is something I haven't interacted with before.

Steven, thank you for trying to help me. I thought my response to Bob was
completely civil and you seem like a firey character, so I'll try to
respond as evenly as I can to you too. Like I said before, I am on a
learning curve, and the messages Emile and Ramit have sent me both had
constructive criticism as to how to proceed with asking questions as well
as awesome answers to my poorly-worded questions. The little side comments
were not included. I don't mean to "waste your time" or Bob's apparently
but asking that question led me to understand how to ask a better one. I am
trying to improve and I've just started learning. Please don't be mean.
Thank you.

On Wed, Jul 18, 2012 at 4:42 PM, Steven D'Aprano <steve at pearwood.info>wrote:

> Aditi Pai wrote:
>
>> Emile,
>>
>> So far I have started with
>>
>> def changeColor(pict,scale,color):
>>
>> I was told to make different names for the float and integer (float =
>> scale
>> and color= integer) and then I kept everything else the same, just to test
>> it out and see if that would work.
>>
>> So it looks like this:
>>
>> def changeColor(pict,scale,color):
>>   for p in getPixels(pict):
>>     value=getRed(p)
>>     setRed(p,value*.9)
>>
>> Unfortunately I keep getting this error message:
>>
>> The error was:changeColor() takes at least 3 arguments (1 given)
>>
>
>
> Is this error message not clear? You have written a function that requires
> at least 3 values, but you have only given 1 value.
>
> I can't tell you what argument you have given, because you have not told
> us what you have done to get this error message.
>
> Aditi, we want to help you, but you're not giving us enough information to
> go on. You're wasting our time, and your own.
>
> Instead of biting at poor Bob, who so far has given you the best and most
> valuable advise, you should pay attention to what we are trying to teach
> you. Would you rather that we just ignored you in silence?
>
> I'm about to move away from the computer for an hour or so. When I come
> back, I will try to give you some more assistance. Or somebody else may try
> to help. In the meantime, I suggest you try reading this page and see if it
> enlightens you:
>
> http://sscce.org/
>
>
>
> --
> Steven
>
> ______________________________**_________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120718/e28dbf35/attachment-0001.html>

From aditipai1227 at gmail.com  Wed Jul 18 23:37:17 2012
From: aditipai1227 at gmail.com (Aditi Pai)
Date: Wed, 18 Jul 2012 17:37:17 -0400
Subject: [Tutor] writing function changeColor
In-Reply-To: <CAF_Boipe6vJ4DW8rerMJEE0mXgbe-axrjKYEHhUiHTs=1bWJ7Q@mail.gmail.com>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
	<ju6tsm$ckv$1@dough.gmane.org>
	<CAF_BoirqULGAWo-UTJud-YVeznD-NV-22jfckpE_ZD6aQY9w6A@mail.gmail.com>
	<ju72a6$i77$1@dough.gmane.org>
	<CAF_BoiovE0e_K+vx7bNka-8Sp=6L9CncYO0cQWkqbLOR-qNOcA@mail.gmail.com>
	<50071FA4.4060005@pearwood.info>
	<CAF_Boipe6vJ4DW8rerMJEE0mXgbe-axrjKYEHhUiHTs=1bWJ7Q@mail.gmail.com>
Message-ID: <CAF_BoipOs3TF+WotKhHmfyFc8zsF74pSRMw7CYkfs7JaiB16qg@mail.gmail.com>

Hey,

Do you know how to take your post off the discussion board? Devin is right!
I was looking for links or online resources but people keep trying to help
me, which is awesome, but I'd like to do my own research. Thanks!!

On Wed, Jul 18, 2012 at 5:31 PM, Aditi Pai <aditipai1227 at gmail.com> wrote:

> Ramit told me not to "top post," and I looked it up and I think I am doing
> that same thing again, so if I am let me know how to avoid that. This kind
> of discussion board is something I haven't interacted with before.
>
> Steven, thank you for trying to help me. I thought my response to Bob was
> completely civil and you seem like a firey character, so I'll try to
> respond as evenly as I can to you too. Like I said before, I am on a
> learning curve, and the messages Emile and Ramit have sent me both had
> constructive criticism as to how to proceed with asking questions as well
> as awesome answers to my poorly-worded questions. The little side comments
> were not included. I don't mean to "waste your time" or Bob's apparently
> but asking that question led me to understand how to ask a better one. I am
> trying to improve and I've just started learning. Please don't be mean.
> Thank you.
>
>
> On Wed, Jul 18, 2012 at 4:42 PM, Steven D'Aprano <steve at pearwood.info>wrote:
>
>> Aditi Pai wrote:
>>
>>> Emile,
>>>
>>> So far I have started with
>>>
>>> def changeColor(pict,scale,color):
>>>
>>> I was told to make different names for the float and integer (float =
>>> scale
>>> and color= integer) and then I kept everything else the same, just to
>>> test
>>> it out and see if that would work.
>>>
>>> So it looks like this:
>>>
>>> def changeColor(pict,scale,color):
>>>   for p in getPixels(pict):
>>>     value=getRed(p)
>>>     setRed(p,value*.9)
>>>
>>> Unfortunately I keep getting this error message:
>>>
>>> The error was:changeColor() takes at least 3 arguments (1 given)
>>>
>>
>>
>> Is this error message not clear? You have written a function that
>> requires at least 3 values, but you have only given 1 value.
>>
>> I can't tell you what argument you have given, because you have not told
>> us what you have done to get this error message.
>>
>> Aditi, we want to help you, but you're not giving us enough information
>> to go on. You're wasting our time, and your own.
>>
>> Instead of biting at poor Bob, who so far has given you the best and most
>> valuable advise, you should pay attention to what we are trying to teach
>> you. Would you rather that we just ignored you in silence?
>>
>> I'm about to move away from the computer for an hour or so. When I come
>> back, I will try to give you some more assistance. Or somebody else may try
>> to help. In the meantime, I suggest you try reading this page and see if it
>> enlightens you:
>>
>> http://sscce.org/
>>
>>
>>
>> --
>> Steven
>>
>> ______________________________**_________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120718/a5eabab2/attachment.html>

From ramit.prasad at jpmorgan.com  Thu Jul 19 00:15:54 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Wed, 18 Jul 2012 22:15:54 +0000
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5007258E.6010206@gmail.com>
References: <5007258E.6010206@gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>

> OK so I have been trying for a couple days now and I am throwing in the
> towel, Python 3 wins this one.
> I want to convert a string to binary and back again like in this
> question: Stack Overflow: Convert Binary to ASCII and vice versa
> (Python)
> <http://stackoverflow.com/questions/7396849/convert-binary-to-ascii-and-vice-
> versa-python>
> But in Python 3 I consistently get  some sort of error relating to the
> fact that nothing but bytes and bytearrays support the buffer interface
> or I get an overflow error because something is too large to be
> converted to bytes.
> Please help me and then explian what I am not getting that is new in
> Python 3. I would like to point out I realize that binary, hex, and
> encodings are all a very complex subject and so I do not expect to
> master it but I do hope that I can gain a deeper insight. Thank you all.

[ snip program ]

> data = bin(int(binascii.hexlify(bytes(data, 'UTF-8')), 16))
> data = binascii.unhexlify('%x' % data)

[ more snipping ]

> wolfrage at lm12-laptop02 ~/Projects $ python3 test_script.py
> Bit Length: 134
> 0b10101000110010101110011011101000110100101101110011001110010000001011000010011110101001000100000010000010110011101100001011010010110111000100001
> 7351954002991226380810260999848996570230305
> Traceback (most recent call last):
> File "test_script.py", line 24, in <module>
> data = binascii.unhexlify('%x' % data)
> TypeError: 'str' does not support the buffer interface

I think your basic problem is too much conversion because you do not
understand the types. A string is represented by a series of bytes
which are binary numbers. Do you understand the concept behind ASCII?
Each letter has a numeric representation that are sequential.
So the string 'abcd' is equivalent to a series of bytes 65,66,67,68.
It is not equivalent to 65666768 or 65+66+67+68. So your first
task is to convert each character to the numeric equivalent and store
them in a list. Once you have them converted to a list of integers,
you can create another list that is a list of characters.

Look at the functions chr and ord here
( http://docs.python.org/py3k/library/functions.html )

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From d at davea.name  Thu Jul 19 00:16:12 2012
From: d at davea.name (Dave Angel)
Date: Wed, 18 Jul 2012 18:16:12 -0400
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5007258E.6010206@gmail.com>
References: <5007258E.6010206@gmail.com>
Message-ID: <500735AC.3010404@davea.name>

On 07/18/2012 05:07 PM, Jordan wrote:
> OK so I have been trying for a couple days now and I am throwing in the
> towel, Python 3 wins this one.
> I want to convert a string to binary and back again like in this
> question: Stack Overflow: Convert Binary to ASCII and vice versa
> (Python)
> <http://stackoverflow.com/questions/7396849/convert-binary-to-ascii-and-vice-versa-python>
> But in Python 3 I consistently get  some sort of error relating to the
> fact that nothing but bytes and bytearrays support the buffer interface
> or I get an overflow error because something is too large to be
> converted to bytes.
> Please help me and then explian what I am not getting that is new in
> Python 3. I would like to point out I realize that binary, hex, and
> encodings are all a very complex subject and so I do not expect to
> master it but I do hope that I can gain a deeper insight. Thank you all.
>
> test_script.py:
> import binascii
>
> test_int = 109
>
> test_int = int(str(test_int) + '45670')
> data = 'Testing XOR Again!'
>
> while sys.getsizeof(data) > test_int.bit_length():
>
> test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1), 'big')))
>
> print('Bit Length: ' + str(test_int.bit_length()))
>
> key = test_int # Yes I know this is an unnecessary step...
>
> data = bin(int(binascii.hexlify(bytes(data, 'UTF-8')), 16))
>
> print(data)
>
> data = int(data, 2)
>
> print(data)
>
> data = binascii.unhexlify('%x' % data)
>

I don't get the same error you did.  I get:

 File "jordan.py", line 13
    test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1),
'big')))
           ^
IndentationError: expected an indented block


Please post it again, with correct indentation.  if you used tabs, then
expand them to spaces before pasting it into your test-mode mail editor.


I'd also recommend you remove a lot of the irrelevant details there.  if
you have a problem with hexlfy and/or unhexlify, then give a simple byte
string that doesn't work for you, and somebody can probably identify why
not.  And if you want people to run your code, include the imports as well.

As it is, you're apparently looping, comparing the byte memory size of a
string (which is typically 4 bytes per character) with the number of
significant bits in an unrelated number.

I suspect what you want is something resembling (untested):

    mybytes = bytes( "%x" % data, "ascii")
    newdata = binascii.unexlify(mybytes)


-- 
DaveA

From ramit.prasad at jpmorgan.com  Thu Jul 19 00:22:43 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Wed, 18 Jul 2012 22:22:43 +0000
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741657F741@SCACMX008.exchad.jpmchase.net>

> 
> I think your basic problem is too much conversion because you do not
> understand the types. A string is represented by a series of bytes
> which are binary numbers. Do you understand the concept behind ASCII?
> Each letter has a numeric representation that are sequential.
> So the string 'abcd' is equivalent to a series of bytes 65,66,67,68.
> It is not equivalent to 65666768 or 65+66+67+68. So your first
> task is to convert each character to the numeric equivalent and store
> them in a list. Once you have them converted to a list of integers,
> you can create another list that is a list of characters.
> 
> Look at the functions chr and ord here
> ( http://docs.python.org/py3k/library/functions.html )

I forgot to say, that once you have the integer equivalents, 
you can then convert that easily to binary using bin.
I used ast.literal_eval to convert from binary string 
(as returned from bin) to number, but there might be better
ways.

 
Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From steve at pearwood.info  Thu Jul 19 00:34:16 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 19 Jul 2012 08:34:16 +1000
Subject: [Tutor] writing function changeColor
In-Reply-To: <CAF_Boipe6vJ4DW8rerMJEE0mXgbe-axrjKYEHhUiHTs=1bWJ7Q@mail.gmail.com>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
	<ju6tsm$ckv$1@dough.gmane.org>
	<CAF_BoirqULGAWo-UTJud-YVeznD-NV-22jfckpE_ZD6aQY9w6A@mail.gmail.com>
	<ju72a6$i77$1@dough.gmane.org>
	<CAF_BoiovE0e_K+vx7bNka-8Sp=6L9CncYO0cQWkqbLOR-qNOcA@mail.gmail.com>
	<50071FA4.4060005@pearwood.info>
	<CAF_Boipe6vJ4DW8rerMJEE0mXgbe-axrjKYEHhUiHTs=1bWJ7Q@mail.gmail.com>
Message-ID: <20120718223415.GA2214@ando>

On Wed, Jul 18, 2012 at 05:31:00PM -0400, Aditi Pai wrote:
> Ramit told me not to "top post," and I looked it up and I think I am doing
> that same thing again, so if I am let me know how to avoid that. This kind
> of discussion board is something I haven't interacted with before.

In your email program, just move the cursor (the linking line where you 
type) after each comment you want to reply to before answering the 
question.

If your email program won't let you do that, either 

1) use a different, better, email program (I can strongly recommend 
Thunderbird); or

2) apologise, but expect to still be (more, or less, gently) told off 
for top-posting.


> Steven, thank you for trying to help me. I thought my response to Bob was
> completely civil and you seem like a firey character, so I'll try to
> respond as evenly as I can to you too. Like I said before, I am on a
> learning curve, and the messages Emile and Ramit have sent me both had
> constructive criticism as to how to proceed with asking questions as well
> as awesome answers to my poorly-worded questions. The little side comments
> were not included. I don't mean to "waste your time" or Bob's apparently
> but asking that question led me to understand how to ask a better one. I am
> trying to improve and I've just started learning. Please don't be mean.

Nobody here is trying to be mean. If you think this was "firey" or 
mean, you should try asking questions on some of the C programming 
language forums.

But you have to understand, this is the first time you have been in the 
situation of having to ask questions you don't know how to put into 
words. But for us, it is about the millionth time somebody has come here 
asking for help but not giving us enough details that we can help.

Imagine that you are working at a burger restaurant, and somebody comes 
in and asks "I want that burger I had last month, you know, the one with 
the sauce that was so good", and either can't or won't tell you any 
more. You've got twenty different burgers on the menu and thirty 
different sauces, how can you possibly know which one the customer is 
talking about? You want to help, right, but you can't. Pretty 
frustrating, yeah, but you smile and do your best, because that's what 
you're paid for. So you ask them to describe the burger, and they say 
"I had it with fries and a Coke" like that helps.

Now imagine how frustrating it is the tenth time somebody has done it 
this week, and it's only Tuesday. AND YOU'RE A VOLUNTEER. You're not 
being paid, you're doing it for free, to help the community, but 
sometimes it seems like the community is made up of demanding, needy 
dumb-arses who couldn't find their own nose without a team of Sherpas 
and a map...

(No offence :)

Just as we should try to remember what it was like to be in your 
situation, so you (and others like you) should try to imagine what it is 
like to be in our shoes. We're not being paid, we're doing this for free 
out of a desire to help others. A little bit of snarkiness is 
just our way of letting off steam.

Forgive us the occasional grumpiness, and we'll forgive you the 
occasional stupid question.

I'll get back to your actual programming question in a moment.



-- 
Steven

From steve at pearwood.info  Thu Jul 19 00:37:52 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 19 Jul 2012 08:37:52 +1000
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF4741657F741@SCACMX008.exchad.jpmchase.net>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<5B80DD153D7D744689F57F4FB69AF4741657F741@SCACMX008.exchad.jpmchase.net>
Message-ID: <20120718223752.GB2214@ando>

On Wed, Jul 18, 2012 at 10:22:43PM +0000, Prasad, Ramit wrote:

> I forgot to say, that once you have the integer equivalents, 
> you can then convert that easily to binary using bin.
> I used ast.literal_eval to convert from binary string 
> (as returned from bin) to number, but there might be better
> ways.

Do you mean something like this?


py> int('111001010100100', 2)
29348


-- 
Steven

From steve at pearwood.info  Thu Jul 19 01:09:43 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 19 Jul 2012 09:09:43 +1000
Subject: [Tutor] writing function changeColor
In-Reply-To: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
Message-ID: <20120718230943.GC2214@ando>

Coming back to your original question...


On Wed, Jul 18, 2012 at 01:10:51PM -0400, Aditi Pai wrote:
> Hello,
> 
> I am trying to write a function changeColor for an assignment. Two things
> that I am unsure about for this assignment are how to assign different
> colors to integers so that, red will be 1, blue will be 2, etc. Also, I

Is your assignment to invent your own colour scheme? Or to use an 
existing colour scheme?

(I'm Australian, we spell colour with a U.)

If you are supposed to use an existing scheme, then you need to find out 
which one you are supposed to use. I can't help with that.

I can tell you that there are generally two main ways to assign colours:

* palettes (also known as indexed colour)

* colour spaces

Palettes aren't terribly helpful in this question. A palette basically 
is a list of usually 256 integers (but sometimes fewer), each one of 
which arbitrarily represents a colour. So you might have:

0 = light blue
1 = pink
2 = dark green
3 = white
4 = very light grey
...
254 = black
255 = tangerine

or something.

Obviously because the order of colours is arbitrary, it is hard, if not 
impossible, to say "start with tangerine and make it 10% more blue".

Colour spaces are more useful. A colour space defines each colour in 
terms of usually three, but sometimes four, numbers, each one of which 
ranges between 0 and 255. Each number represents a different quality of 
the colour.

There are different colour spaces, because none of them describe colour 
perfectly. Some of them are better for modelling additive colours (like 
from your monitor, which generates coloured light); some are better for 
subtractive colours (like your colour printer). The whole field is 
awfully complex and I'm not an expert, but you can start here:

http://en.wikipedia.org/wiki/Color_space_encoding

I *think* that for your purposes, the best colour space to start with is 
the RGB model. In this, each colour is represented by an amount of red, 
green and blue, added together. For example:

"Black" is the complete absence of any colour, so:

Red = 0
Green = 0
Blue = 0

"White" is a complete mix of all colours, so:

Red = 255
Green = 255
Blue = 255


Everything else is somewhere in between, e.g. "Forest Green":

Red: 33 out of a maximum of 255, or 12.9%
Green: 139, or 54.5%
Blue: 33, or 12.9%

Notice that here the percentage represents the amount out of the maximum 
possible for each value, and then do NOT add to 100% in total.

If you increase the amount of blue by 50%, you get:

Red: 33 or 12.9%
Green: 139 or 54.5%
Blue: 33 + 16.5 = 50 (rounded to the nearest integer) or 19.6%

I have no idea what name this colour might have.

Does this help?


-- 
Steven

From steve at pearwood.info  Thu Jul 19 01:21:13 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 19 Jul 2012 09:21:13 +1000
Subject: [Tutor] writing function changeColor
In-Reply-To: <CAF_BoipOs3TF+WotKhHmfyFc8zsF74pSRMw7CYkfs7JaiB16qg@mail.gmail.com>
References: <CAF_BoirHZb-XmVecUgO7VHCyYGxEriJUA8-rfWiLuwzs4YOXaA@mail.gmail.com>
	<ju6tsm$ckv$1@dough.gmane.org>
	<CAF_BoirqULGAWo-UTJud-YVeznD-NV-22jfckpE_ZD6aQY9w6A@mail.gmail.com>
	<ju72a6$i77$1@dough.gmane.org>
	<CAF_BoiovE0e_K+vx7bNka-8Sp=6L9CncYO0cQWkqbLOR-qNOcA@mail.gmail.com>
	<50071FA4.4060005@pearwood.info>
	<CAF_Boipe6vJ4DW8rerMJEE0mXgbe-axrjKYEHhUiHTs=1bWJ7Q@mail.gmail.com>
	<CAF_BoipOs3TF+WotKhHmfyFc8zsF74pSRMw7CYkfs7JaiB16qg@mail.gmail.com>
Message-ID: <20120718232113.GD2214@ando>

On Wed, Jul 18, 2012 at 05:37:17PM -0400, Aditi Pai wrote:
> Hey,
> 
> Do you know how to take your post off the discussion board? 

Do you mean taking a previous post off the discussion list? You can't. 
Once a post has been made, you can't delete it. It is public knowledge, 
archived on a dozen different websites and hundreds of people's email 
inboxes.

If you mean how to take the discussion off-list and make it private, 
that's easy. Instead of replying to the tutor mailing list, email the 
person you want to talk to directly. Don't forget to take tutor out of 
the CC list.

BUT think VERY hard before doing this. Normally you should keep as much 
of the discussion on-list as possible, for three reasons:

1) We're not doing this just to help *you* personally, but to help the 
many other beginners who are reading but not saying anything. If you 
take the discussion off-list, they will be cut out of the discussion and 
can't learn from your questions or our answers.

2) Often one person will make a comment, and you reply with a question, 
but that person is away from the computer for hours or days or even 
permanently. Or they are too busy to answer, or the question bores them 
and they can't be bothered to answer. If the question is asked on-list, 
somebody else can answer in their stead. If you ask privately, you may 
never get a response.

3) None of us are perfect. I've been programming in Python for well over 
a decade, and I still learn things here occasionally, or make mistakes. 
If the questions remain public, we can *all* learn from each other.


Of course, if you want to say something private or personal, you should 
write privately.



-- 
Steven

From ryan.waples at gmail.com  Thu Jul 19 01:33:20 2012
From: ryan.waples at gmail.com (Ryan Waples)
Date: Wed, 18 Jul 2012 16:33:20 -0700
Subject: [Tutor] Problem When Iterating Over Large Test Files
Message-ID: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>

I'm seeing some unexpected output when I use a script (included at
end) to iterate over large text files.  I am unsure of the source of
the unexpected output and any help would be much appreciated.

Background
Python v 2.7.1
Windows 7 32bit
Reading and writing to an external USB hard drive

Data files are ~4GB text (.fastq) file, it has been uncompressed
(gzip).  This file has no errors or formatting problems, it seems to
have uncompressed just fine.  64M lines, each 'entry' is split across
4 consecutive lines, 16M entries.

My python script iterates over data files 4 lines at a time, selects
and writes groups of four lines to the output file.  I will end up
selecting roughly 85% of the entries.

In my output I am seeing lines that don't occur in the original file,
and that don't match any lines in the original file.  The incidences
of badly formatted lines don't seem to match up with any patterns in
the data file, and occur across multiple different data files.

I've included 20 consecutive lines of input and output.  Each of these
5 'records' should have been selected and printed to the output file.
But there is a problem with the 4th and 5th entries in the output, and
it no longer matches the input as expected.  For example the line:
TTCTGTGAGTGATTTCCTGCAAGACAGGAATGTCAGT
never occurs in the original data.

Sorry for the large block of text below.
Other pertinent info, I've tried a related perl script, and ran into
similar issues, but not in the same places.

Any help or insight would be appreciated.

Thanks


__EXAMPLE RAW DATA FILE REGION__

@HWI-ST0747:167:B02DEACXX:8:1101:3182:167088 1:N:0:
CGCGTGTGCAGGTTTATAGAACAAAACAGCTGCAGATTAGTAGCAGCGCACGGAGAGGTGTGTCTGTTTATTGTCCTCAGCAGGCAGACATGTTTGTGGTC
+
@@@DDADDHHHHHB9+2A<??:?G9+C)???G at DB@@DGFB<0*?FF?0F:@/54'-;;?B;>;6>>>>(5 at CDAC(5(5:5,(8?88?BC@#########
@HWI-ST0747:167:B02DEACXX:8:1101:3134:167090 1:N:0:
TTCTAGTGCAGGGCGACAGCGTTGCGGAGCCGGTCCGAGTCTGCTGGGTCAGTCATGGCTAGTTGGTACTATAACGACACAGGGCGAGACCCAGATGCAAA
+
@CCFFFDFHHHHHIIIIJJIJHHIIIJHGHIJI at GFFDDDFDDCEEEDCCBDCCCDDDDCCB>>@C(4 at ADCA>>?BBBDDABB055<>-?A<B1:@ACC:
@HWI-ST0747:167:B02DEACXX:8:1101:3002:167092 1:N:0:
CTTTGCTGCAGGCTCATCCTGACATGACCCTCCAGCATGACAATGCCACCAGCCATACTGCTCGTTCTGTGTGTGATTTCCAGCACCCCAGTAAATATGTA
+
CCCFFFFFHHHHHIJIEHIH at AHFAGHIGIIGGEIJGIJIIIGIIIGEHGEHIIJIEHH@FHGH@=ACEHHFBFFCE at AACC<ACDB;;B?C3>A>AD>BA
@HWI-ST0747:167:B02DEACXX:8:1101:3022:167094 1:N:0:
ATTCCGTGCAGGCCAACTCCCGACGGACATCCTTGCTCAGACTGCAGCGATAGTGGTCGATCAGGGCCCTGTTGTTCCATCCCACTCCGGCGACCAGGTTC
+
CCCFFFFFHHHHHIDHJIIHIIIJIJIIJJJJGGIIFHJIIGGGGIIEIFHFF>CBAECBDDDC:??B=AAACD?8@:>C@?8CBDDD at D99B@>3884>A
@HWI-ST0747:167:B02DEACXX:8:1101:3095:167100 1:N:0:
CGTGATTGCAGGGACGTTACAGAGACGTTACAGGGATGTTACAGGGACGTTACAGAGACGTTAAAGAGATGTTACAGGGATGTTACAGACAGAGACGTTAC
+


__EXAMPLE PROBLEMATIC OUTPUT FILE REGION__

@HWI-ST0747:167:B02DEACXX:8:1101:3182:167088 1:N:0:
CGCGTGTGCAGGTTTATAGAACAAAACAGCTGCAGATTAGTAGCAGCGCACGGAGAGGTGTGTCTGTTTATTGTCCTCAGCAGGCAGACATGTTTGTGGTC
+
@@@DDADDHHHHHB9+2A<??:?G9+C)???G at DB@@DGFB<0*?FF?0F:@/54'-;;?B;>;6>>>>(5 at CDAC(5(5:5,(8?88?BC@#########
@HWI-ST0747:167:B02DEACXX:8:1101:3134:167090 1:N:0:
TTCTAGTGCAGGGCGACAGCGTTGCGGAGCCGGTCCGAGTCTGCTGGGTCAGTCATGGCTAGTTGGTACTATAACGACACAGGGCGAGACCCAGATGCAAA
+
@CCFFFDFHHHHHIIIIJJIJHHIIIJHGHIJI at GFFDDDFDDCEEEDCCBDCCCDDDDCCB>>@C(4 at ADCA>>?BBBDDABB055<>-?A<B1:@ACC:
@HWI-ST0747:167:B02DEACXX:8:1101:3002:167092 1:N:0:
CTTTGCTGCAGGCTCATCCTGACATGACCCTCCAGCATGACAATGCCACCAGCCATACTGCTCGTTCTGTGTGTGATTTCCAGCACCCCAGTAAATATGTA
+
CCCFFFFFHHHHHIJIEHIH at AHFAGHIGIIGGEIJGIJIIIGIIIGEHGEHIIJIEHH@FHGH@=ACEHHFBFFCE at AACC<ACDB;;B?C3>A>AD>BA
TTCTGTGAGTGATTTCCTGCAAGACAGGAATGTCAGT
+
BCCFFDFFHHHHFIJIJJHIFGIIIIGGGIGGIJIJIGIGIGIGHHIGIIJGJJJIIJIIEHIHHHFFFB@>CCE at BEDCDDAC?CC?ACC??<C>>ADDD
@HWI-ST0747:167:B02DEACXX:8:1304:19473:44548 1:N:0:
CTACAGTGCAGGCACCCGGCCCGCCACAATGAGTCGCTAGAGCGCAATGAGACAAGTAAAGCTCCCCGACCAAACCCTCCCCTAACCCGGACGATGCTGGG
+
BCCFFFFFHHHHHIJEHJJIIGIJIJJJJGIJIDHDGIGJJJJIGGFFFFEEEEED at CCDDC>C>BBD?BDBAABBBBBDDD at BCD@?@BDBDDDBDCCC2




__PYTHON CODE __


import glob

my_in_files = glob.glob ('E:/PINK/Paired_End/raw/gzip/*.fastq')

for each in my_in_files:
	#print(each)
	out = each.replace('/gzip', '/rem_clusters2' )
	#print (out)
	INFILE = open (each, 'r')
	OUTFILE = open (out , 'w')
	
# Tracking Variables
	Reads = 0
	Writes = 0
	Check_For_End_Of_File = 0

#Updates
	print ("Reading File: " + each)
	print ("Writing File: " + out)

# Read FASTQ File by group of four lines
	while Check_For_End_Of_File == 0:

		# Read the next four lines from the FASTQ file
		ID_Line_1		= INFILE.readline()
		Seq_Line 		= INFILE.readline()
		ID_Line_2 		= INFILE.readline()
		Quality_Line 	= INFILE.readline()
		
		# Strip off leading and trailing whitespace characters
		ID_Line_1		= ID_Line_1.strip()	
		Seq_Line		= Seq_Line.strip() 	
		ID_Line_2		= ID_Line_2.strip()	
		Quality_Line 	= Quality_Line.strip()

		Reads = Reads + 1

		#Check that I have not reached the end of file
		if Quality_Line == "":
			#End of file reached, print update
			print ("Saw " + str(Reads) + " reads")
			print ("Wrote " + str(Writes) + " reads")
			Check_For_End_Of_File = 1
			break
		
		#Check that ID_Line_1 starts with @
		if not ID_Line_1.startswith('@'):
			print ("**ERROR**")
			print (each)
			print ("Read Number " + str(Reads))
			print ID_Line_1 + ' does not start with @'
			break #ends the while loop
		
		# Select Reads that I want to keep	
		ID = ID_Line_1.partition(' ')
		if (ID[2] == "1:N:0:" or ID[2] == "2:N:0:"):
			# Write to file, maintaining group of 4
			OUTFILE.write(ID_Line_1 + "\n")
			OUTFILE.write(Seq_Line + "\n")
			OUTFILE.write(ID_Line_2 + "\n")
			OUTFILE.write(Quality_Line + "\n")
			Writes = Writes +1

			
	INFILE.close()
	OUTFILE.close()

From steve at pearwood.info  Thu Jul 19 01:54:26 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 19 Jul 2012 09:54:26 +1000
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
Message-ID: <20120718235426.GE2214@ando>

On Wed, Jul 18, 2012 at 04:33:20PM -0700, Ryan Waples wrote:
> I'm seeing some unexpected output when I use a script (included at
> end) to iterate over large text files.  I am unsure of the source of
> the unexpected output and any help would be much appreciated.

It may help if you can simplify your script to the smallest amount of 
code which demonstrates the problem. See here for more details:

http://sscce.org/

More suggestions follow below.

> In my output I am seeing lines that don't occur in the original file,
> and that don't match any lines in the original file.

How do you know? What are you doing to test that they don't match the 
original?

I'm not suggesting that you are wrong, I'm just trying to see what steps 
you have already taken.


> The incidences
> of badly formatted lines don't seem to match up with any patterns in
> the data file, and occur across multiple different data files.

Do they occur at random, or is this repeatable?

That is, if you get this mysterious output for files A, B, H and Q 
(say), do you *always* get them for A, B, H and Q?


> I've included 20 consecutive lines of input and output.  Each of these
> 5 'records' should have been selected and printed to the output file.

Earlier, you stated that each record should be four lines. But your 
sample data starts with a record of three lines.

More to follow later (time permitting).


-- 
Steven


From abhishek.vit at gmail.com  Thu Jul 19 02:02:33 2012
From: abhishek.vit at gmail.com (Abhishek Pratap)
Date: Wed, 18 Jul 2012 17:02:33 -0700
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
Message-ID: <CAJbA1KAaMs4oZJ7iwp5h+BBAD-zK-v9AOAPqnnBWm3Fh0WHaXw@mail.gmail.com>

Hi Ryan

One quick comment

I dint get through all your code to figure out the  fine details but
my hunch is you might be having issues related to linux to dos  EOF
char.  Could you check the total  number of lines in your fastq# are
same as read by a simple python file iterator. If not then it is
mostly becoz readline is reading more than one line at a time.


-Abhi

On Wed, Jul 18, 2012 at 4:33 PM, Ryan Waples <ryan.waples at gmail.com> wrote:
> I'm seeing some unexpected output when I use a script (included at
> end) to iterate over large text files.  I am unsure of the source of
> the unexpected output and any help would be much appreciated.
>
> Background
> Python v 2.7.1
> Windows 7 32bit
> Reading and writing to an external USB hard drive
>
> Data files are ~4GB text (.fastq) file, it has been uncompressed
> (gzip).  This file has no errors or formatting problems, it seems to
> have uncompressed just fine.  64M lines, each 'entry' is split across
> 4 consecutive lines, 16M entries.
>
> My python script iterates over data files 4 lines at a time, selects
> and writes groups of four lines to the output file.  I will end up
> selecting roughly 85% of the entries.
>
> In my output I am seeing lines that don't occur in the original file,
> and that don't match any lines in the original file.  The incidences
> of badly formatted lines don't seem to match up with any patterns in
> the data file, and occur across multiple different data files.
>
> I've included 20 consecutive lines of input and output.  Each of these
> 5 'records' should have been selected and printed to the output file.
> But there is a problem with the 4th and 5th entries in the output, and
> it no longer matches the input as expected.  For example the line:
> TTCTGTGAGTGATTTCCTGCAAGACAGGAATGTCAGT
> never occurs in the original data.
>
> Sorry for the large block of text below.
> Other pertinent info, I've tried a related perl script, and ran into
> similar issues, but not in the same places.
>
> Any help or insight would be appreciated.
>
> Thanks
>
>
> __EXAMPLE RAW DATA FILE REGION__
>
> @HWI-ST0747:167:B02DEACXX:8:1101:3182:167088 1:N:0:
> CGCGTGTGCAGGTTTATAGAACAAAACAGCTGCAGATTAGTAGCAGCGCACGGAGAGGTGTGTCTGTTTATTGTCCTCAGCAGGCAGACATGTTTGTGGTC
> +
> @@@DDADDHHHHHB9+2A<??:?G9+C)???G at DB@@DGFB<0*?FF?0F:@/54'-;;?B;>;6>>>>(5 at CDAC(5(5:5,(8?88?BC@#########
> @HWI-ST0747:167:B02DEACXX:8:1101:3134:167090 1:N:0:
> TTCTAGTGCAGGGCGACAGCGTTGCGGAGCCGGTCCGAGTCTGCTGGGTCAGTCATGGCTAGTTGGTACTATAACGACACAGGGCGAGACCCAGATGCAAA
> +
> @CCFFFDFHHHHHIIIIJJIJHHIIIJHGHIJI at GFFDDDFDDCEEEDCCBDCCCDDDDCCB>>@C(4 at ADCA>>?BBBDDABB055<>-?A<B1:@ACC:
> @HWI-ST0747:167:B02DEACXX:8:1101:3002:167092 1:N:0:
> CTTTGCTGCAGGCTCATCCTGACATGACCCTCCAGCATGACAATGCCACCAGCCATACTGCTCGTTCTGTGTGTGATTTCCAGCACCCCAGTAAATATGTA
> +
> CCCFFFFFHHHHHIJIEHIH at AHFAGHIGIIGGEIJGIJIIIGIIIGEHGEHIIJIEHH@FHGH@=ACEHHFBFFCE at AACC<ACDB;;B?C3>A>AD>BA
> @HWI-ST0747:167:B02DEACXX:8:1101:3022:167094 1:N:0:
> ATTCCGTGCAGGCCAACTCCCGACGGACATCCTTGCTCAGACTGCAGCGATAGTGGTCGATCAGGGCCCTGTTGTTCCATCCCACTCCGGCGACCAGGTTC
> +
> CCCFFFFFHHHHHIDHJIIHIIIJIJIIJJJJGGIIFHJIIGGGGIIEIFHFF>CBAECBDDDC:??B=AAACD?8@:>C@?8CBDDD at D99B@>3884>A
> @HWI-ST0747:167:B02DEACXX:8:1101:3095:167100 1:N:0:
> CGTGATTGCAGGGACGTTACAGAGACGTTACAGGGATGTTACAGGGACGTTACAGAGACGTTAAAGAGATGTTACAGGGATGTTACAGACAGAGACGTTAC
> +
>
>
> __EXAMPLE PROBLEMATIC OUTPUT FILE REGION__
>
> @HWI-ST0747:167:B02DEACXX:8:1101:3182:167088 1:N:0:
> CGCGTGTGCAGGTTTATAGAACAAAACAGCTGCAGATTAGTAGCAGCGCACGGAGAGGTGTGTCTGTTTATTGTCCTCAGCAGGCAGACATGTTTGTGGTC
> +
> @@@DDADDHHHHHB9+2A<??:?G9+C)???G at DB@@DGFB<0*?FF?0F:@/54'-;;?B;>;6>>>>(5 at CDAC(5(5:5,(8?88?BC@#########
> @HWI-ST0747:167:B02DEACXX:8:1101:3134:167090 1:N:0:
> TTCTAGTGCAGGGCGACAGCGTTGCGGAGCCGGTCCGAGTCTGCTGGGTCAGTCATGGCTAGTTGGTACTATAACGACACAGGGCGAGACCCAGATGCAAA
> +
> @CCFFFDFHHHHHIIIIJJIJHHIIIJHGHIJI at GFFDDDFDDCEEEDCCBDCCCDDDDCCB>>@C(4 at ADCA>>?BBBDDABB055<>-?A<B1:@ACC:
> @HWI-ST0747:167:B02DEACXX:8:1101:3002:167092 1:N:0:
> CTTTGCTGCAGGCTCATCCTGACATGACCCTCCAGCATGACAATGCCACCAGCCATACTGCTCGTTCTGTGTGTGATTTCCAGCACCCCAGTAAATATGTA
> +
> CCCFFFFFHHHHHIJIEHIH at AHFAGHIGIIGGEIJGIJIIIGIIIGEHGEHIIJIEHH@FHGH@=ACEHHFBFFCE at AACC<ACDB;;B?C3>A>AD>BA
> TTCTGTGAGTGATTTCCTGCAAGACAGGAATGTCAGT
> +
> BCCFFDFFHHHHFIJIJJHIFGIIIIGGGIGGIJIJIGIGIGIGHHIGIIJGJJJIIJIIEHIHHHFFFB@>CCE at BEDCDDAC?CC?ACC??<C>>ADDD
> @HWI-ST0747:167:B02DEACXX:8:1304:19473:44548 1:N:0:
> CTACAGTGCAGGCACCCGGCCCGCCACAATGAGTCGCTAGAGCGCAATGAGACAAGTAAAGCTCCCCGACCAAACCCTCCCCTAACCCGGACGATGCTGGG
> +
> BCCFFFFFHHHHHIJEHJJIIGIJIJJJJGIJIDHDGIGJJJJIGGFFFFEEEEED at CCDDC>C>BBD?BDBAABBBBBDDD at BCD@?@BDBDDDBDCCC2
>
>
>
>
> __PYTHON CODE __
>
>
> import glob
>
> my_in_files = glob.glob ('E:/PINK/Paired_End/raw/gzip/*.fastq')
>
> for each in my_in_files:
>         #print(each)
>         out = each.replace('/gzip', '/rem_clusters2' )
>         #print (out)
>         INFILE = open (each, 'r')
>         OUTFILE = open (out , 'w')
>
> # Tracking Variables
>         Reads = 0
>         Writes = 0
>         Check_For_End_Of_File = 0
>
> #Updates
>         print ("Reading File: " + each)
>         print ("Writing File: " + out)
>
> # Read FASTQ File by group of four lines
>         while Check_For_End_Of_File == 0:
>
>                 # Read the next four lines from the FASTQ file
>                 ID_Line_1               = INFILE.readline()
>                 Seq_Line                = INFILE.readline()
>                 ID_Line_2               = INFILE.readline()
>                 Quality_Line    = INFILE.readline()
>
>                 # Strip off leading and trailing whitespace characters
>                 ID_Line_1               = ID_Line_1.strip()
>                 Seq_Line                = Seq_Line.strip()
>                 ID_Line_2               = ID_Line_2.strip()
>                 Quality_Line    = Quality_Line.strip()
>
>                 Reads = Reads + 1
>
>                 #Check that I have not reached the end of file
>                 if Quality_Line == "":
>                         #End of file reached, print update
>                         print ("Saw " + str(Reads) + " reads")
>                         print ("Wrote " + str(Writes) + " reads")
>                         Check_For_End_Of_File = 1
>                         break
>
>                 #Check that ID_Line_1 starts with @
>                 if not ID_Line_1.startswith('@'):
>                         print ("**ERROR**")
>                         print (each)
>                         print ("Read Number " + str(Reads))
>                         print ID_Line_1 + ' does not start with @'
>                         break #ends the while loop
>
>                 # Select Reads that I want to keep
>                 ID = ID_Line_1.partition(' ')
>                 if (ID[2] == "1:N:0:" or ID[2] == "2:N:0:"):
>                         # Write to file, maintaining group of 4
>                         OUTFILE.write(ID_Line_1 + "\n")
>                         OUTFILE.write(Seq_Line + "\n")
>                         OUTFILE.write(ID_Line_2 + "\n")
>                         OUTFILE.write(Quality_Line + "\n")
>                         Writes = Writes +1
>
>
>         INFILE.close()
>         OUTFILE.close()
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From wrw at mac.com  Thu Jul 19 02:53:28 2012
From: wrw at mac.com (William R. Wing (Bill Wing))
Date: Wed, 18 Jul 2012 20:53:28 -0400
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
Message-ID: <F7C6E304-3BF2-407A-806F-85A5820E2225@mac.com>

On Jul 18, 2012, at 7:33 PM, Ryan Waples wrote:

> I'm seeing some unexpected output when I use a script (included at
> end) to iterate over large text files.  I am unsure of the source of
> the unexpected output and any help would be much appreciated.
> 
> Background
> Python v 2.7.1
> Windows 7 32bit
> Reading and writing to an external USB hard drive
> 
> Data files are ~4GB text (.fastq) file, it has been uncompressed
> (gzip).  This file has no errors or formatting problems, it seems to
> have uncompressed just fine.  64M lines, each 'entry' is split across
> 4 consecutive lines, 16M entries.
> 
> My python script iterates over data files 4 lines at a time, selects
> and writes groups of four lines to the output file.  I will end up
> selecting roughly 85% of the entries.
> 
> In my output I am seeing lines that don't occur in the original file,
> and that don't match any lines in the original file.  The incidences
> of badly formatted lines don't seem to match up with any patterns in
> the data file, and occur across multiple different data files.
> 
> I've included 20 consecutive lines of input and output.  Each of these
> 5 'records' should have been selected and printed to the output file.
> But there is a problem with the 4th and 5th entries in the output, and
> it no longer matches the input as expected.  For example the line:
> TTCTGTGAGTGATTTCCTGCAAGACAGGAATGTCAGT
> never occurs in the original data.
> 
> Sorry for the large block of text below.
> Other pertinent info, I've tried a related perl script, and ran into
> similar issues, but not in the same places.
> 
> Any help or insight would be appreciated.
> 
> Thanks

[Data and program snipped]

With apologies - I'm a Mac/UNIX user, not Windows, but those numbers (4GB and 64M lines) look suspiciously close to the file and record pointer limits to a 32-bit file system.  Are you sure you aren't bumping into wrap around issues of some sort?

Just a thought?

-Bill

From missive at hotmail.com  Thu Jul 19 04:05:01 2012
From: missive at hotmail.com (Lee Harr)
Date: Thu, 19 Jul 2012 06:35:01 +0430
Subject: [Tutor] writing function changeColor
Message-ID: <SNT106-W19FFCE2792AEEBEE4ACD97B1D90@phx.gbl>


> I?learned python so that if I were to put in 0.9, it'd decrease red by 10%.> The way this function needs to be written, -0.1 decreases red by 10%
It sounds like you have something like ...
def f1(color, redchange=0, bluechange=0, greenchange=0):? ? # some code here to make the changes? ? return modified_color
calling it like ...
>>> c = Color(red=200, blue=100, green=50)>>> c(200, 100, 50)>>> f1(c, redchange=0.9)(180, 100, 50)
but what you want is ...
>>> f1a(c, redchange=-0.1)(180, 100, 50)

Maybe, since you already have a function that works, youcould write f1a as a function that calls f1 as a helper, usingmodified parameters.
def f1a(color, redchange=0, bluechange=0, greenchange=0):? ? # some code to modify the parameters to fit your function f1? ? return f1(color, modredchange, modbluechange, modgreenchange)

So... we just need some code to convert 0.9 to -0.1 (and weneed to think about other possible values that could be sentto f1 and what f1a would want in the same situation.)

There's a very simple way to convert 0.9 to -0.1 but I'm notsure that's what you want.
Would you also want:
>>> f1(c, redchange=0.5)(100, 100, 50)>>> f1a(c, redchange=-0.5)(100, 100, 50)
How about ...
>>> f1(c, redchange=0)(0, 100, 50)>>> f1a(c, redchange=-1.0)(0, 100, 50)
Or ...
>>> f1(c, redchange=1.2)(240, 100, 50)>>> f1a(c, redchange=0.2)(240, 100, 50)
?
 		 	   		  

From ryan.waples at gmail.com  Thu Jul 19 04:33:27 2012
From: ryan.waples at gmail.com (Ryan Waples)
Date: Wed, 18 Jul 2012 19:33:27 -0700
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <F7C6E304-3BF2-407A-806F-85A5820E2225@mac.com>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
	<F7C6E304-3BF2-407A-806F-85A5820E2225@mac.com>
Message-ID: <CABdrqAFt9krL31RBk69QOOPmX_aPwH3SfWjbma5arX+e5_UsFA@mail.gmail.com>

Thanks for the replies, I'll try to address the questions raised and
spur further conversation.

>"those numbers (4GB and 64M lines) look suspiciously close to the file and record pointer limits to a 32-bit file system.  Are you sure you aren't bumping into wrap around issues of some sort?"

My understanding is that I am taking the files in a stream, one line
at a time and never loading them into memory all at once.  I would
like (and expect) my script to be able to handle files up to at least
50GB.  If this would cause a problem, let me know.

> "my hunch is you might be having issues related to linux to dos  EOF char."

I don't think this is the issue.  99.99% of the lines come out ok,
(see examples).  I do end up with an output file with some 50 some mil
lines.  I can confirm that my python code as written and executed on
Win7 will convert the original file endings from Unix (LF) to windows
(CRLF).  This shouldn't confuse the downstream analysis.

> "What are you doing to test that they don't match the original?"

Those 2 pieces of example data have been grep'd (cygwin) out of the IN
and OUT files, they represent the output of a grep that pull the 20
lines surrounding the line:
@HWI-ST0747:167:B02DEACXX:8:1101:3002:167092 1:N:0:
which is a unique line in each.

I have also grep'd the IN file for a line in the OUT:
  grep ^TTCTGTGAGTGATTTCCTGCAAGACAGGAATGTCAGT$
with no results

The python code posted has a (weak) check, that mostly serves to
confirm that every fourth line of the IN file starts with an "@", this
IS the case for the IN file, but is NOT the case for the OUT file.

I can run my analysis program program on the raw IN file fine, it will
process all entries.  When the OUT file is supplied, it will error at
reads in the pasted text.

> "Earlier, you stated that each record should be four lines. But your sample data starts with a record of three lines."

I've checked again and they look to be four lines, so I'm not sure I understand.
Format:
1) ID line (must start with @) - contains filter criteria
2) Data line 1 - 101 chars
3) just a "+"
4) Data line 2 - 101 chars (may start with @)

> "Do they occur at random, or is this repeatable?"

When I'm back at work I'll confirm again that this is the case, I
should have a better answer here.  I can confirm that it seems to
happen to every (large) file I've tested, no files seem unaffected.

Thanks

__SHORTENED PYTHON CODE__

for each in my_in_files:
        out = each.replace('/gzip', '/rem_clusters2' )
        INFILE = open (each, 'r')
        OUTFILE = open (out , 'w')

# Tracking Variables
        Reads = 0
        Writes = 0
        Check_For_End_Of_File = 0


# Read FASTQ File by group of four lines
        while Check_For_End_Of_File == 0:

                ID_Line_1               = INFILE.readline()
                Seq_Line                = INFILE.readline()
                ID_Line_2               = INFILE.readline()
                Quality_Line    = INFILE.readline()

                ID_Line_1               = ID_Line_1.strip()
                Seq_Line                = Seq_Line.strip()
                ID_Line_2               = ID_Line_2.strip()
                Quality_Line    = Quality_Line.strip()

                Reads = Reads + 1

                #Check that I have not reached the end of file
                if Quality_Line == "":
                        Check_For_End_Of_File = 1
                        break

                #Check that ID_Line_1 starts with @
                if not ID_Line_1.startswith('@'):
                        break

                # Select Reads that I want to keep
                ID = ID_Line_1.partition(' ')
                if (ID[2] == "1:N:0:" or ID[2] == "2:N:0:"):

						# Write to file, maintaining group of 4
                        OUTFILE.write(ID_Line_1 + "\n")
                        OUTFILE.write(Seq_Line + "\n")
                        OUTFILE.write(ID_Line_2 + "\n")
                        OUTFILE.write(Quality_Line + "\n")
                        Writes = Writes +1

        INFILE.close()
        OUTFILE.close()

From wrw at mac.com  Thu Jul 19 05:04:37 2012
From: wrw at mac.com (William R. Wing (Bill Wing))
Date: Wed, 18 Jul 2012 23:04:37 -0400
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <CABdrqAFt9krL31RBk69QOOPmX_aPwH3SfWjbma5arX+e5_UsFA@mail.gmail.com>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
	<F7C6E304-3BF2-407A-806F-85A5820E2225@mac.com>
	<CABdrqAFt9krL31RBk69QOOPmX_aPwH3SfWjbma5arX+e5_UsFA@mail.gmail.com>
Message-ID: <55DA4409-D6E8-4F32-B771-B8478B6D4BE3@mac.com>

On Jul 18, 2012, at 10:33 PM, Ryan Waples wrote:

> Thanks for the replies, I'll try to address the questions raised and
> spur further conversation.
> 
>> "those numbers (4GB and 64M lines) look suspiciously close to the file and record pointer limits to a 32-bit file system.  Are you sure you aren't bumping into wrap around issues of some sort?"
> 
> My understanding is that I am taking the files in a stream, one line
> at a time and never loading them into memory all at once.  I would
> like (and expect) my script to be able to handle files up to at least
> 50GB.  If this would cause a problem, let me know.

[Again, stripping out everything else?]

I don't think you understood my concern.  The issue isn't whether or not the files are being read as a stream, the issue is that at something like those numbers a 32-bit file system can silently fail.  If the pointers that are chaining allocation blocks together (or whatever Windows calls them) aren't capable of indexing to sufficiently large numbers, then you WILL get garbage included in the file stream.

If you copy those files to a different device (one that has just been scrubbed and reformatted), then copy them back and get different results with your application, you've found your problem.

-Bill

From missive at hotmail.com  Thu Jul 19 05:23:22 2012
From: missive at hotmail.com (Lee Harr)
Date: Thu, 19 Jul 2012 07:53:22 +0430
Subject: [Tutor] Problem When Iterating Over Large Test Files
Message-ID: <SNT106-W553953A3CF20793F3DCF70B1D90@phx.gbl>


> ? grep ^TTCTGTGAGTGATTTCCTGCAAGACAGGAATGTCAGT$> with no results

How about:
grep TTCTGTGAGTGATTTCCTGCAAGACAGGAATGTCAGT outfile
Just in case there is some non-printing character in there...

Beyond that ... my guess would be that you are either not readingthe file you think you are, or not writing the file you think you are ?:o)
out = each.replace('/gzip', '/rem_clusters2')
Seems pretty bulletproof, but maybe just print each and out hereto make sure...

Also, I'm curious... Reading your code, I sort of feel like when I amlistening?to a non-native speaker. I always get the urge to throw out thecorrect "Americanisms" for people -- to help them fit in better. So, I hope itdoes?not make me a jerk, but ...
infile = open(each, 'r') # I'd probably drop the 'r' also...
while not check_for_end_of_file:
reads += 1
head, sep, tail = id_line_1.partition(' ') # or, if I'm only using the one thing ..._, _, meaningful_name =?id_line_1.partition(' ') # maybe call it "selector", then ...
if?selector?in ('1:N:0:', '2:N:0:'):

Hope this helps.
 		 	   		  

From ryan.waples at gmail.com  Thu Jul 19 05:27:29 2012
From: ryan.waples at gmail.com (Ryan Waples)
Date: Wed, 18 Jul 2012 20:27:29 -0700
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <55DA4409-D6E8-4F32-B771-B8478B6D4BE3@mac.com>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
	<F7C6E304-3BF2-407A-806F-85A5820E2225@mac.com>
	<CABdrqAFt9krL31RBk69QOOPmX_aPwH3SfWjbma5arX+e5_UsFA@mail.gmail.com>
	<55DA4409-D6E8-4F32-B771-B8478B6D4BE3@mac.com>
Message-ID: <CABdrqAEK_C6c9a7ZXbQPRkBvy7edKOwfP90J78gzbu_ifs7w9Q@mail.gmail.com>

On Wed, Jul 18, 2012 at 8:04 PM, William R. Wing (Bill Wing)
<wrw at mac.com> wrote:
> On Jul 18, 2012, at 10:33 PM, Ryan Waples wrote:
>
>> Thanks for the replies, I'll try to address the questions raised and
>> spur further conversation.
>>
>>> "those numbers (4GB and 64M lines) look suspiciously close to the file and record pointer limits to a 32-bit file system.  Are you sure you aren't bumping into wrap around issues of some sort?"
>>
>> My understanding is that I am taking the files in a stream, one line
>> at a time and never loading them into memory all at once.  I would
>> like (and expect) my script to be able to handle files up to at least
>> 50GB.  If this would cause a problem, let me know.
>
> [Again, stripping out everything else?]
>
> I don't think you understood my concern.  The issue isn't whether or not the files are being read as a stream, the issue is that at something like those numbers a 32-bit file system can silently fail.  If the pointers that are chaining allocation blocks together (or whatever Windows calls them) aren't capable of indexing to sufficiently large numbers, then you WILL get garbage included in the file stream.
>
> If you copy those files to a different device (one that has just been scrubbed and reformatted), then copy them back and get different results with your application, you've found your problem.
>
> -Bill

Thanks for the insistence,  I'll check this out.  If you have any
guidance on how to do so let me know.  I knew my system wasn't
particularly well suited to the task at hand, but I haven't seen how
it would actually cause problems.

-Ryan

From ryan.waples at gmail.com  Thu Jul 19 05:33:21 2012
From: ryan.waples at gmail.com (Ryan Waples)
Date: Wed, 18 Jul 2012 20:33:21 -0700
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <SNT106-W553953A3CF20793F3DCF70B1D90@phx.gbl>
References: <SNT106-W553953A3CF20793F3DCF70B1D90@phx.gbl>
Message-ID: <CABdrqAEOqH2WY6ZhRrxT_04DrmM7WLG2fNgKEiRqeOqxU3yk5w@mail.gmail.com>

On Wed, Jul 18, 2012 at 8:23 PM, Lee Harr <missive at hotmail.com> wrote:
>
>>   grep ^TTCTGTGAGTGATTTCCTGCAAGACAGGAATGTCAGT$> with no results
>
> How about:
> grep TTCTGTGAGTGATTTCCTGCAAGACAGGAATGTCAGT outfile
> Just in case there is some non-printing character in there...

There are many instances of that sequence of characters in the RAW
input file, but that is what I would expect.




>
> Beyond that ... my guess would be that you are either not readingthe file you think you are, or not writing the file you think you are  :o)
> out = each.replace('/gzip', '/rem_clusters2')
> Seems pretty bulletproof, but maybe just print each and out hereto make sure...

Checked this multiple times

>
> Also, I'm curious... Reading your code, I sort of feel like when I amlistening to a non-native speaker. I always get the urge to throw out thecorrect "Americanisms" for people -- to help them fit in better. So, I hope itdoes not make me a jerk, but ...
> infile = open(each, 'r') # I'd probably drop the 'r' also...

working in science, I try to be as explicit as possible, I've come to
dislike Perl for this reason.

> while not check_for_end_of_file:
> reads += 1
> head, sep, tail = id_line_1.partition(' ') # or, if I'm only using the one thing ..._, _, meaningful_name = id_line_1.partition(' ') # maybe call it "selector", then ...
> if selector in ('1:N:0:', '2:N:0:'):
>

Points taken, thanks.


> Hope this helps.

From wolfrage8765 at gmail.com  Thu Jul 19 07:41:42 2012
From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com)
Date: Thu, 19 Jul 2012 07:41:42 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <500735AC.3010404@davea.name>
References: <5007258E.6010206@gmail.com>
	<500735AC.3010404@davea.name>
Message-ID: <CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>

On Thu, Jul 19, 2012 at 12:16 AM, Dave Angel <d at davea.name> wrote:

>  On 07/18/2012 05:07 PM, Jordan wrote:
> > OK so I have been trying for a couple days now and I am throwing in the
> > towel, Python 3 wins this one.
> > I want to convert a string to binary and back again like in this
> > question: Stack Overflow: Convert Binary to ASCII and vice versa
> > (Python)
> > <
> http://stackoverflow.com/questions/7396849/convert-binary-to-ascii-and-vice-versa-python
> >
> > But in Python 3 I consistently get  some sort of error relating to the
> > fact that nothing but bytes and bytearrays support the buffer interface
> > or I get an overflow error because something is too large to be
> > converted to bytes.
> > Please help me and then explian what I am not getting that is new in
> > Python 3. I would like to point out I realize that binary, hex, and
> > encodings are all a very complex subject and so I do not expect to
> > master it but I do hope that I can gain a deeper insight. Thank you all.
> >
> > test_script.py:
> > import binascii
> >
> > test_int = 109
> >
> > test_int = int(str(test_int) + '45670')
> > data = 'Testing XOR Again!'
> >
> > while sys.getsizeof(data) > test_int.bit_length():
> >
> > test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1), 'big')))
> >
> > print('Bit Length: ' + str(test_int.bit_length()))
> >
> > key = test_int # Yes I know this is an unnecessary step...
> >
> > data = bin(int(binascii.hexlify(bytes(data, 'UTF-8')), 16))
> >
> > print(data)
> >
> > data = int(data, 2)
> >
> > print(data)
> >
> > data = binascii.unhexlify('%x' % data)
> >
>
> I don't get the same error you did.  I get:
>
>  File "jordan.py", line 13
>     test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1),
> 'big')))
>            ^
>
test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1), \
    'big')))
# That was probably just do to the copy and paste.

> IndentationError: expected an indented block
>
>
> Please post it again, with correct indentation.  if you used tabs, then
> expand them to spaces before pasting it into your test-mode mail editor.
>
> I only use spaces and this program did not require any indentation until
it was pasted and the one line above became split across two line. Really
though that was a trivial error to correct.

>
> I'd also recommend you remove a lot of the irrelevant details there.  if
> you have a problem with hexlfy and/or unhexlify, then give a simple byte
> string that doesn't work for you, and somebody can probably identify why
> not.  And if you want people to run your code, include the imports as well.
>
> My problem is not specific to hexlify and unhexlify, my problem is trying
to convert from string to binary and back. That is why all of the details,
to show I have tried on my own.
Sorry that I forgot to include sys and os for imports.


> As it is, you're apparently looping, comparing the byte memory size of a
> string (which is typically 4 bytes per character) with the number of
> significant bits in an unrelated number.
>
> I suspect what you want is something resembling (untested):
>
>     mybytes = bytes( "%x" % data, "ascii")
>     newdata = binascii.unexlify(mybytes)
>
> I was comparing them but I think I understand how to compare them well,
now I want to convert them both to binary so that I can XOR them together.
Thank you for your time and help Dave, now I need to reply to Ramit.

>
> --
> DaveA
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120719/ece37ad0/attachment-0001.html>

From steve at pearwood.info  Thu Jul 19 08:00:12 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 19 Jul 2012 16:00:12 +1000
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
Message-ID: <20120719060012.GF2214@ando>

On Wed, Jul 18, 2012 at 04:33:20PM -0700, Ryan Waples wrote:

> I've included 20 consecutive lines of input and output.  Each of these
> 5 'records' should have been selected and printed to the output file.

I count only 19 lines. The first group has only three lines. See below.

There is a blank line, which I take as NOT part of the input but just a 
spacer. Then:

1) Line starting with @
2) Line of bases CGCGT ...
3) Plus sign
4) Line starting with @@@
5) Line starting with @
6) Line of bases TTCTA ...
7) Plus sign

and so on. There are TWO lines before the first +, and three before each 
of the others.



> __EXAMPLE RAW DATA FILE REGION__
> 
> @HWI-ST0747:167:B02DEACXX:8:1101:3182:167088 1:N:0:
> CGCGTGTGCAGGTTTATAGAACAAAACAGCTGCAGATTAGTAGCAGCGCACGGAGAGGTGTGTCTGTTTATTGTCCTCAGCAGGCAGACATGTTTGTGGTC
> +
> @@@DDADDHHHHHB9+2A<??:?G9+C)???G at DB@@DGFB<0*?FF?0F:@/54'-;;?B;>;6>>>>(5 at CDAC(5(5:5,(8?88?BC@#########
> @HWI-ST0747:167:B02DEACXX:8:1101:3134:167090 1:N:0:
> TTCTAGTGCAGGGCGACAGCGTTGCGGAGCCGGTCCGAGTCTGCTGGGTCAGTCATGGCTAGTTGGTACTATAACGACACAGGGCGAGACCCAGATGCAAA
> +
> @CCFFFDFHHHHHIIIIJJIJHHIIIJHGHIJI at GFFDDDFDDCEEEDCCBDCCCDDDDCCB>>@C(4 at ADCA>>?BBBDDABB055<>-?A<B1:@ACC:
> @HWI-ST0747:167:B02DEACXX:8:1101:3002:167092 1:N:0:
> CTTTGCTGCAGGCTCATCCTGACATGACCCTCCAGCATGACAATGCCACCAGCCATACTGCTCGTTCTGTGTGTGATTTCCAGCACCCCAGTAAATATGTA
> +
> CCCFFFFFHHHHHIJIEHIH at AHFAGHIGIIGGEIJGIJIIIGIIIGEHGEHIIJIEHH@FHGH@=ACEHHFBFFCE at AACC<ACDB;;B?C3>A>AD>BA
> @HWI-ST0747:167:B02DEACXX:8:1101:3022:167094 1:N:0:
> ATTCCGTGCAGGCCAACTCCCGACGGACATCCTTGCTCAGACTGCAGCGATAGTGGTCGATCAGGGCCCTGTTGTTCCATCCCACTCCGGCGACCAGGTTC
> +
> CCCFFFFFHHHHHIDHJIIHIIIJIJIIJJJJGGIIFHJIIGGGGIIEIFHFF>CBAECBDDDC:??B=AAACD?8@:>C@?8CBDDD at D99B@>3884>A
> @HWI-ST0747:167:B02DEACXX:8:1101:3095:167100 1:N:0:
> CGTGATTGCAGGGACGTTACAGAGACGTTACAGGGATGTTACAGGGACGTTACAGAGACGTTAAAGAGATGTTACAGGGATGTTACAGACAGAGACGTTAC
> +

Your code says that the first line in each group should start with an @ 
sign. That is clearly not the case for the last two groups.

I suggest that your data files have been corrupted.

> __PYTHON CODE __

I have re-written your code slightly, to be a little closer to "best 
practice", or at least modern practice. If there is anything you don't 
understand, please feel free to ask.

I haven't tested this code, but it should run fine on Python 2.7.

It will be interesting to see if you get different results with this.



import glob

def four_lines(file_object):
        """Yield lines from file_object grouped into batches of four.

        If the file has fewer than four lines remaining, pad the batch 
        with 1-3 empty strings.

        Lines are stripped of leading and trailing whitespace.
        """
        while True:
            # Get the first line. If there is no first line, we are at EOF
            # and we raise StopIteration to indicate we are done.
            line1 = next(file_object).strip()
            # Get the next three lines, padding if needed.
            line2 = next(file_object, '').strip()
            line3 = next(file_object, '').strip()
            line4 = next(file_object, '').strip()
            yield (line1, line2, line3, line4)


my_in_files = glob.glob ('E:/PINK/Paired_End/raw/gzip/*.fastq')
for each in my_in_files:
        out = each.replace('/gzip', '/rem_clusters2' )
        print ("Reading File: " + each)
        print ("Writing File: " + out)
        INFILE = open (each, 'r')
        OUTFILE = open (out , 'w')
        writes = 0

        for reads, lines in four_lines( INFILE ):
                ID_Line_1, Seq_Line, ID_Line_2, Quality_Line = lines
                # Check that ID_Line_1 starts with @
                if not ID_Line_1.startswith('@'):
                        print ("**ERROR**")
                        print ("expected ID_Line to start with @")
                        print (lines)
                        print ("Read Number " + str(Reads))
                        break
                elif Quality_Line != '+':
                        print ("**ERROR**")
                        print ("expected Quality_Line = +")
                        print (lines)
                        print ("Read Number " + str(Reads))
                        break
                # Select Reads that I want to keep      
                ID = ID_Line_1.partition(' ')
                if (ID[2] == "1:N:0:" or ID[2] == "2:N:0:"):
                        # Write to file, maintaining group of 4
                        OUTFILE.write(ID_Line_1 + "\n")
                        OUTFILE.write(Seq_Line + "\n")
                        OUTFILE.write(ID_Line_2 + "\n")
                        OUTFILE.write(Quality_Line + "\n")
                        writes += 1
        # End of file reached, print update
        print ("Saw", reads, "groups of four lines")
        print ("Wrote", writes, "groups of four lines")
        INFILE.close()
        OUTFILE.close()





-- 
Steven

From breamoreboy at yahoo.co.uk  Thu Jul 19 08:14:16 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 19 Jul 2012 07:14:16 +0100
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
Message-ID: <ju88il$o3g$1@dough.gmane.org>

On 19/07/2012 06:41, wolfrage8765 at gmail.com wrote:
> On Thu, Jul 19, 2012 at 12:16 AM, Dave Angel <d at davea.name> wrote:
>
>>   On 07/18/2012 05:07 PM, Jordan wrote:
>>> OK so I have been trying for a couple days now and I am throwing in the
>>> towel, Python 3 wins this one.
>>> I want to convert a string to binary and back again like in this
>>> question: Stack Overflow: Convert Binary to ASCII and vice versa
>>> (Python)
>>> <
>> http://stackoverflow.com/questions/7396849/convert-binary-to-ascii-and-vice-versa-python
>>>
>>> But in Python 3 I consistently get  some sort of error relating to the
>>> fact that nothing but bytes and bytearrays support the buffer interface
>>> or I get an overflow error because something is too large to be
>>> converted to bytes.
>>> Please help me and then explian what I am not getting that is new in
>>> Python 3. I would like to point out I realize that binary, hex, and
>>> encodings are all a very complex subject and so I do not expect to
>>> master it but I do hope that I can gain a deeper insight. Thank you all.
>>>
>>> test_script.py:
>>> import binascii
>>>
>>> test_int = 109
>>>
>>> test_int = int(str(test_int) + '45670')
>>> data = 'Testing XOR Again!'
>>>
>>> while sys.getsizeof(data) > test_int.bit_length():
>>>
>>> test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1), 'big')))
>>>
>>> print('Bit Length: ' + str(test_int.bit_length()))
>>>
>>> key = test_int # Yes I know this is an unnecessary step...
>>>
>>> data = bin(int(binascii.hexlify(bytes(data, 'UTF-8')), 16))
>>>
>>> print(data)
>>>
>>> data = int(data, 2)
>>>
>>> print(data)
>>>
>>> data = binascii.unhexlify('%x' % data)
>>>
>>
>> I don't get the same error you did.  I get:
>>
>>   File "jordan.py", line 13
>>      test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1),
>> 'big')))
>>             ^
>>
> test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1), \
>      'big')))
> # That was probably just do to the copy and paste.
>
>> IndentationError: expected an indented block
>>
>>
>> Please post it again, with correct indentation.  if you used tabs, then
>> expand them to spaces before pasting it into your test-mode mail editor.
>>
>> I only use spaces and this program did not require any indentation until
> it was pasted and the one line above became split across two line. Really
> though that was a trivial error to correct.

Really?  Are you using a forked version of Python that doesn't need 
indentation after a while loop, or are you speaking with a forked 
tongue? :)  Strangely I believe the latter, so please take note of what 
Dave Angel has told you and post with the correct indentation.

>
>>
>> I'd also recommend you remove a lot of the irrelevant details there.  if
>> you have a problem with hexlfy and/or unhexlify, then give a simple byte
>> string that doesn't work for you, and somebody can probably identify why
>> not.  And if you want people to run your code, include the imports as well.
>>
>> My problem is not specific to hexlify and unhexlify, my problem is trying
> to convert from string to binary and back. That is why all of the details,
> to show I have tried on my own.
> Sorry that I forgot to include sys and os for imports.
>
>
>> As it is, you're apparently looping, comparing the byte memory size of a
>> string (which is typically 4 bytes per character) with the number of
>> significant bits in an unrelated number.
>>
>> I suspect what you want is something resembling (untested):
>>
>>      mybytes = bytes( "%x" % data, "ascii")
>>      newdata = binascii.unexlify(mybytes)
>>
>> I was comparing them but I think I understand how to compare them well,
> now I want to convert them both to binary so that I can XOR them together.
> Thank you for your time and help Dave, now I need to reply to Ramit.
>
>>
>> --
>> DaveA
>>
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>


-- 
Cheers.

Mark Lawrence.




From ryan.waples at gmail.com  Thu Jul 19 08:28:20 2012
From: ryan.waples at gmail.com (Ryan Waples)
Date: Wed, 18 Jul 2012 23:28:20 -0700
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <4D30ED4B-CBE8-4112-81D3-85086F23D6E2@mac.com>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
	<F7C6E304-3BF2-407A-806F-85A5820E2225@mac.com>
	<CABdrqAFt9krL31RBk69QOOPmX_aPwH3SfWjbma5arX+e5_UsFA@mail.gmail.com>
	<55DA4409-D6E8-4F32-B771-B8478B6D4BE3@mac.com>
	<CABdrqAEK_C6c9a7ZXbQPRkBvy7edKOwfP90J78gzbu_ifs7w9Q@mail.gmail.com>
	<4D30ED4B-CBE8-4112-81D3-85086F23D6E2@mac.com>
Message-ID: <CABdrqAHf7SHtUkXfYK5u551TecEfLFrYTF8e1JY3hjto2nH14Q@mail.gmail.com>

>>>
>>> If you copy those files to a different device (one that has just been scrubbed and reformatted), then copy them back and get different results with your application, you've found your problem.
>>>
>>> -Bill
>>
>> Thanks for the insistence,  I'll check this out.  If you have any
>> guidance on how to do so let me know.  I knew my system wasn't
>> particularly well suited to the task at hand, but I haven't seen how
>> it would actually cause problems.
>>
>> -Ryan
>> _______________________________________________
>> The last two lines in my MSG pretty much would be the test. Get another flash drive, format it as FAT-32 (I assume that's what you are using), then copy a couple of files to it.  Then copy them back to your current device and run your program again. If you get DIFFERENT, but still wrong results, you've found the problem. The largest positive integer a 32-bit binary number can represent is 2^32, which is 4Gig.  I'm no expert on Window's files, but I'd be very surprised if when the FAT-32 file system was being designed, anyone considered the case where a single file could be that large.
>
> -Bill


The hard-drive is formatted as NTFS, because as you say I'm up against
the file size limit of FAT32 , do think this could still be the issue?

From ryan.waples at gmail.com  Thu Jul 19 09:06:23 2012
From: ryan.waples at gmail.com (Ryan Waples)
Date: Thu, 19 Jul 2012 00:06:23 -0700
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <20120719060012.GF2214@ando>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
	<20120719060012.GF2214@ando>
Message-ID: <CABdrqAFJzCd-fKd5pZRiqy0OGVNdqMycv-drKf2Sw3Fq8j8PkQ@mail.gmail.com>

> I count only 19 lines.

yep, you are right.  My bad, I think I missing copy/pasting line 20.

>The first group has only three lines. See below.

Not so, the first group is actually the first four lines listed below.
 Lines 1-4 serve as one group.  For what it is worth, line four should
have 1 character for each char in line 1, and the first line is much
shorter, contains a space, and for this file always ends in either
"1:N:0:" (keep) "1"Y"0:" (remove).   The EXAMPLE data is correctly
formatted as it should be, but I'm missing line 20.

> There is a blank line, which I take as NOT part of the input but just a
> spacer. Then:
>
> 1) Line starting with @
> 2) Line of bases CGCGT ...
> 3) Plus sign
> 4) Line starting with @@@
> 5) Line starting with @
> 6) Line of bases TTCTA ...
> 7) Plus sign
>
> and so on. There are TWO lines before the first +, and three before each
> of the others.

I think you are just reading one frame shifted, its not a well
designed format because the required start character "@", can appear
other places as well....


>
>
>> __EXAMPLE RAW DATA FILE REGION__
>>
>> @HWI-ST0747:167:B02DEACXX:8:1101:3182:167088 1:N:0:
>> CGCGTGTGCAGGTTTATAGAACAAAACAGCTGCAGATTAGTAGCAGCGCACGGAGAGGTGTGTCTGTTTATTGTCCTCAGCAGGCAGACATGTTTGTGGTC
>> +
>> @@@DDADDHHHHHB9+2A<??:?G9+C)???G at DB@@DGFB<0*?FF?0F:@/54'-;;?B;>;6>>>>(5 at CDAC(5(5:5,(8?88?BC@#########
>> @HWI-ST0747:167:B02DEACXX:8:1101:3134:167090 1:N:0:
>> TTCTAGTGCAGGGCGACAGCGTTGCGGAGCCGGTCCGAGTCTGCTGGGTCAGTCATGGCTAGTTGGTACTATAACGACACAGGGCGAGACCCAGATGCAAA
>> +
>> @CCFFFDFHHHHHIIIIJJIJHHIIIJHGHIJI at GFFDDDFDDCEEEDCCBDCCCDDDDCCB>>@C(4 at ADCA>>?BBBDDABB055<>-?A<B1:@ACC:
>> @HWI-ST0747:167:B02DEACXX:8:1101:3002:167092 1:N:0:
>> CTTTGCTGCAGGCTCATCCTGACATGACCCTCCAGCATGACAATGCCACCAGCCATACTGCTCGTTCTGTGTGTGATTTCCAGCACCCCAGTAAATATGTA
>> +
>> CCCFFFFFHHHHHIJIEHIH at AHFAGHIGIIGGEIJGIJIIIGIIIGEHGEHIIJIEHH@FHGH@=ACEHHFBFFCE at AACC<ACDB;;B?C3>A>AD>BA
>> @HWI-ST0747:167:B02DEACXX:8:1101:3022:167094 1:N:0:
>> ATTCCGTGCAGGCCAACTCCCGACGGACATCCTTGCTCAGACTGCAGCGATAGTGGTCGATCAGGGCCCTGTTGTTCCATCCCACTCCGGCGACCAGGTTC
>> +
>> CCCFFFFFHHHHHIDHJIIHIIIJIJIIJJJJGGIIFHJIIGGGGIIEIFHFF>CBAECBDDDC:??B=AAACD?8@:>C@?8CBDDD at D99B@>3884>A
>> @HWI-ST0747:167:B02DEACXX:8:1101:3095:167100 1:N:0:
>> CGTGATTGCAGGGACGTTACAGAGACGTTACAGGGATGTTACAGGGACGTTACAGAGACGTTAAAGAGATGTTACAGGGATGTTACAGACAGAGACGTTAC
>> +


>
> Your code says that the first line in each group should start with an @
> sign. That is clearly not the case for the last two groups.
>
> I suggest that your data files have been corrupted.

I'm pretty sure that my raw IN files are all good, its hard to be sure
with such a large file, but the very picky downstream analysis program
takes every single raw file just fine (30 of them), and gaks on my
filtered files, at regions that don't conform to the correct
formatting.

>
>> __PYTHON CODE __
>
> I have re-written your code slightly, to be a little closer to "best
> practice", or at least modern practice. If there is anything you don't
> understand, please feel free to ask.
>
> I haven't tested this code, but it should run fine on Python 2.7.
>
> It will be interesting to see if you get different results with this.

--CODE REMOVED--

Thanks, for the suggestions.  I've never really felt super comfortable
using objects at all, but its what I want to learn next.  This will be
helpful, and useful.

 > for reads, lines in four_lines( INFILE ):
                ID_Line_1, Seq_Line, ID_Line_2, Quality_Line = lines

Can you explain what is going on here, or point me In the right
direction?  I see that the parts of 'lines' get assigned, but I'm
missing how the file gets iterated over and how reads gets
incremented.

Do you have a reason why this approach might give a 'better' output?

Thanks again.

From alan.gauld at btinternet.com  Thu Jul 19 09:53:15 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 19 Jul 2012 08:53:15 +0100
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <20120719060012.GF2214@ando>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
	<20120719060012.GF2214@ando>
Message-ID: <ju8edb$276$1@dough.gmane.org>

On 19/07/12 07:00, Steven D'Aprano wrote:

> def four_lines(file_object):
<snipping>....
>              line1 = next(file_object).strip()
>              # Get the next three lines, padding if needed.
>              line2 = next(file_object, '').strip()
>              line3 = next(file_object, '').strip()
>              line4 = next(file_object, '').strip()
>              yield (line1, line2, line3, line4)
<snipping>...

>          for reads, lines in four_lines( INFILE ):
>                  ID_Line_1, Seq_Line, ID_Line_2, Quality_Line = lines

Shouldn't that be

           for reads, lines in enumerate( four_lines(INFILE) ):
                   ID_Line_1, Seq_Line, ID_Line_2, Quality_Line = lines

?

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From bala.biophysics at gmail.com  Thu Jul 19 10:09:19 2012
From: bala.biophysics at gmail.com (Bala subramanian)
Date: Thu, 19 Jul 2012 10:09:19 +0200
Subject: [Tutor] suggestion for an editor
Message-ID: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>

Friends,
At present i write programs using vi editor. I am interested to change to
something else. My specific need is that i want to select a portion/small
segment of my program (for eg. a nested loop) and then monitor processing
time it takes for that portion while i run the program. By this i hope to
find the segment that takes time and modify to achieve better speed. Can
someone please share their experience.

Thanks,
Bala

-- 
C. Balasubramanian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120719/588d93ba/attachment.html>

From ranjithtenz at gmail.com  Thu Jul 19 10:12:38 2012
From: ranjithtenz at gmail.com (Ranjith Kumar)
Date: Thu, 19 Jul 2012 13:42:38 +0530
Subject: [Tutor] suggestion for an editor
In-Reply-To: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>
References: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>
Message-ID: <CAAr0NR-ggiA=prAH_A1aAwLvhBMb4pU=JCv1z+Bek73qvJarmg@mail.gmail.com>

Try Sublime,

On Thu, Jul 19, 2012 at 1:39 PM, Bala subramanian <bala.biophysics at gmail.com
> wrote:

> Friends,
> At present i write programs using vi editor. I am interested to change to
> something else. My specific need is that i want to select a portion/small
> segment of my program (for eg. a nested loop) and then monitor processing
> time it takes for that portion while i run the program. By this i hope to
> find the segment that takes time and modify to achieve better speed. Can
> someone please share their experience.
>
> Thanks,
> Bala
>
> --
> C. Balasubramanian
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>


-- 
Cheers,
Ranjith Kumar K,
Chennai.

http://ranjithtenz.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120719/95799953/attachment.html>

From d at davea.name  Thu Jul 19 12:46:14 2012
From: d at davea.name (Dave Angel)
Date: Thu, 19 Jul 2012 06:46:14 -0400
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
Message-ID: <5007E576.4060108@davea.name>

On 07/19/2012 01:41 AM, wolfrage8765 at gmail.com wrote:
> On Thu, Jul 19, 2012 at 12:16 AM, Dave Angel <d at davea.name> wrote:
>
>> <SNIP>
>> I don't get the same error you did.  I get:
>>
>>  File "jordan.py", line 13
>>     test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1),
>> 'big')))
>>            ^
>>
> test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1), \
>     'big')))
> # That was probably just do to the copy and paste.

That was just the first line that was not indented.  If I thought you
had a one-line while loop, I certainly would have just indented it.  But
I'm sure you have some unknown number of additional lines that were
indented in your original.  Please post in text form.

> <SNIP>
>> I'd also recommend you remove a lot of the irrelevant details there.  if
>> you have a problem with hexlfy and/or unhexlify, then give a simple byte
>> string that doesn't work for you, and somebody can probably identify why
>> not.  And if you want people to run your code, include the imports as well.
>>
>> My problem is not specific to hexlify and unhexlify, my problem is trying
> to convert from string to binary and back. That is why all of the details,
> to show I have tried on my own.
> Sorry that I forgot to include sys and os for imports.

Lots of details that have nothing to do with it.  For example, that
whole thing about adding random digits together.  You could replace the
whole thing with a simple assignment of a value that doesn't work for you.

>
>> As it is, you're apparently looping, comparing the byte memory size of a
>> string (which is typically 4 bytes per character) with the number of
>> significant bits in an unrelated number.
>>
>> I suspect what you want is something resembling (untested):
>>
>>     mybytes = bytes( "%x" % data, "ascii")
>>     newdata = binascii.unexlify(mybytes)
>>
>> I was comparing them but I think I understand how to compare them well,
> now I want to convert them both to binary so that I can XOR them together.
> Thank you for your time and help Dave, now I need to reply to Ramit.

Ah, so you don't actually want binary at all!!!   Why not state the real
problem up front?  You can XOR two integers, without bothering to
convert to a string of ones and zeroes.  Use the carat operator.

print( 40 ^ 12)

I suspect there's an equivalent for strings or byte-strings.  But if
not, it's a simple loop.



-- 

DaveA


From wayne at waynewerner.com  Thu Jul 19 12:52:25 2012
From: wayne at waynewerner.com (Wayne Werner)
Date: Thu, 19 Jul 2012 05:52:25 -0500 (CDT)
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5007258E.6010206@gmail.com>
References: <5007258E.6010206@gmail.com>
Message-ID: <alpine.DEB.2.02.1207190530580.3730@gilgamesh>

I'll preface my response by saying that I know/understand fairly little about
it, but since I've recently been smacked by this same issue when converting
stuff to Python3, I'll see if I can explain it in a way that makes sense.

On Wed, 18 Jul 2012, Jordan wrote:

> OK so I have been trying for a couple days now and I am throwing in the
> towel, Python 3 wins this one.
> I want to convert a string to binary and back again like in this
> question: Stack Overflow: Convert Binary to ASCII and vice versa
> (Python)
> <http://stackoverflow.com/questions/7396849/convert-binary-to-ascii-and-vice-versa-python>
> But in Python 3 I consistently get  some sort of error relating to the
> fact that nothing but bytes and bytearrays support the buffer interface
> or I get an overflow error because something is too large to be
> converted to bytes.
> Please help me and then explian what I am not getting that is new in
> Python 3. I would like to point out I realize that binary, hex, and
> encodings are all a very complex subject and so I do not expect to
> master it but I do hope that I can gain a deeper insight. Thank you all.

The way I've read it - stop thinking about strings as if they are text. The
biggest reason that all this has changed is because Python has grown up and
entered the world where Unicode actually matters. To us poor shmucks in the
English speaking countries of the world it's all very confusing becaust it's
nothing we have to deal with. 26 letters is perfectly fine for us - and if we
want uppercase we'll just throw another 26. Add a few dozen puncuation marks
and 256 is a perfectly fine amount of characters.

To make a slightly relevant side trip, when you were a kid did you ever send
"secret" messages to a friend with a code like this?

A = 1
B = 2
.
.
.
Z = 26

Well, that's basically what is going on when it comes to bytes/text/whatever.
When you input some text, Python3 believes that whatever you wrote was encoded
with Unicode. The nice thing for us 26-letter folks is that the ASCII alphabet
we're so used to just so happens to map quite well to Unicode encodings - so
'A' in ASCII is the same number as 'A' in utf-8.

Now, here's the part that I had to (and still need to) wrap my mind around - if
the string is "just bytes" then it doesn't really matter what the string is
supposed to represent. It could represent the LATIN-1 character set. Or
UTF-8, -16, or some other weird encoding. And all the operations that are
supposed to modify these strings of bytes (e.g. removing spaces, splitting on a
certain "character", etc.) still work. Because if I have this string:

9 45 12 9 13 19 18 9 12 99 102

and I tell you to split on the 9's, it doesn't matter if that's some weird
ASCII character, or some equally weird UTF character, or something else
entirely. And I don't have to worry about things getting munged up when I try
to stick Unicode and ASCII values together - because they're converted to bytes
first.

So the question is, of course, if it's all bytes, then why does it look like
text when I print it out? Well, that's because Python converts that byte stream
to Unicode text when it's printed. Or ASCII, if you tell it to.

But Python3 has converted all(?) of those functions that used to operate on
text and made them operate on byte streams instead. Except for the ones that
operate on text ;)



Well, I hope that's of some use and isn't too much of a lie - like I said, I'm
still trying to wrap my head around things and I've found that explaining (or
trying to explain) to someone else is often the best way to work out the idea
in your own head. If I've gone too far astray I'm sure the other helpful folks
here will correct me :)

HTH,
Wayne

From wayne at waynewerner.com  Thu Jul 19 13:18:00 2012
From: wayne at waynewerner.com (Wayne Werner)
Date: Thu, 19 Jul 2012 06:18:00 -0500 (CDT)
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1207190555140.3730@gilgamesh>

Just a few notes...

On Wed, 18 Jul 2012, Ryan Waples wrote:
<snip>
>
> import glob
>
> my_in_files = glob.glob ('E:/PINK/Paired_End/raw/gzip/*.fastq')
>
> for each in my_in_files:
> 	#print(each)
> 	out = each.replace('/gzip', '/rem_clusters2' )
> 	#print (out)
> 	INFILE = open (each, 'r')
> 	OUTFILE = open (out , 'w')
>

It's slightly confusing to see your comments left-aligned instead of with the
code they refer to. At first glance it looked as though your block ended here,
when it does, in fact, continue.

> # Tracking Variables
> 	Reads = 0
> 	Writes = 0
> 	Check_For_End_Of_File = 0
>
> #Updates
> 	print ("Reading File: " + each)
> 	print ("Writing File: " + out)
>
> # Read FASTQ File by group of four lines
> 	while Check_For_End_Of_File == 0:

This is Python, not C - checking for EOF is probably silly (unless you're
really checking for end of data) - you can just do:

for line in INFILE:
     ID_Line_1 = line
     Seq_line = next(INFILE) # Replace with INFILE.next() for Python2
     ID_Line_2 = next(INFILE)
     Quality_Line = next(INFILE)

>
> 		# Read the next four lines from the FASTQ file
> 		ID_Line_1		= INFILE.readline()
> 		Seq_Line 		= INFILE.readline()
> 		ID_Line_2 		= INFILE.readline()
> 		Quality_Line 	= INFILE.readline()
>
> 		# Strip off leading and trailing whitespace characters
> 		ID_Line_1		= ID_Line_1.strip()
> 		Seq_Line		= Seq_Line.strip()
> 		ID_Line_2		= ID_Line_2.strip()
> 		Quality_Line 	= Quality_Line.strip()
>

Also, it's just extra clutter to call strip like this when you can just tack it
on to your original statement:

for line in INFILE:
     ID_Line_1 = line.strip()
     Seq_line = next(INFILE).strip() # Replace with INFILE.next() for Python2
     ID_Line_2 = next(INFILE).strip()
     Quality_Line = next(INFILE).strip()

> 		Reads = Reads + 1
>
> 		#Check that I have not reached the end of file
> 		if Quality_Line == "":
> 			#End of file reached, print update
> 			print ("Saw " + str(Reads) + " reads")
> 			print ("Wrote " + str(Writes) + " reads")
> 			Check_For_End_Of_File = 1
> 			break

This break is superfluous - it will actually remove you from the while loop -
no further lines of code will be evaluated, including the original `while`
comparison. You can also just test the Quality_Line for truthiness directly,
since empty string evaluate to false. I would actually just say:

if Quality_Line:
     #Do the rest of your stuff here

>
> 		#Check that ID_Line_1 starts with @
> 		if not ID_Line_1.startswith('@'):
> 			print ("**ERROR**")
> 			print (each)
> 			print ("Read Number " + str(Reads))
> 			print ID_Line_1 + ' does not start with @'
> 			break #ends the while loop
>
> 		# Select Reads that I want to keep
> 		ID = ID_Line_1.partition(' ')
> 		if (ID[2] == "1:N:0:" or ID[2] == "2:N:0:"):
> 			# Write to file, maintaining group of 4
> 			OUTFILE.write(ID_Line_1 + "\n")
> 			OUTFILE.write(Seq_Line + "\n")
> 			OUTFILE.write(ID_Line_2 + "\n")
> 			OUTFILE.write(Quality_Line + "\n")
> 			Writes = Writes +1
>
>
> 	INFILE.close()
> 	OUTFILE.close()

You could (as long as you're on 2.6 or greater) just use the `with` block for
reading the files then you don't need to worry about closing - the block takes
care of that, even on errors:

for each in my_in_files:
     out = each.replace('/gzip', '/rem_clusters2' )
     with open (each, 'r') as INFILE, open (out, 'w') as OUTFILE:
         for line in INFILE:
             # Do your work here...


A few stylistic points:
ALL_CAPS are usually reserved for constants - infile and outfile are perfectly
legitimate names.

Caps_In_Variable_Names are usually discouraged. Class names should be CamelCase
(e.g. SimpleHTTPServer), while variable names should be lowercase with
underscores if needed, so id_line_1 instead of ID_Line_1.

If you're using Python3 or from __future__ import print_function, rather than
doing OUTFILE.write(value + '\n') you can do:

     print(value, file=OUTFILE)

Then you get the \n for free. You could also just do:

     print(val1, val2, val3, sep='\n', end='\n', file=OUTFILE)

The end parameter is there for example only, since the default value for end is
'\n'


HTH,
Wayne

From eryksun at gmail.com  Thu Jul 19 13:22:37 2012
From: eryksun at gmail.com (eryksun)
Date: Thu, 19 Jul 2012 07:22:37 -0400
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
Message-ID: <CACL+1aukOw1nnGOK6EX7-WF=Xv=xMVrv84aXSE4QuyQMw6JtrA@mail.gmail.com>

On Thu, Jul 19, 2012 at 1:41 AM, wolfrage8765 at gmail.com
<wolfrage8765 at gmail.com> wrote:
>
> I was comparing them but I think I understand how to compare them well, now
> I want to convert them both to binary so that I can XOR them together. Thank
> you for your time and help Dave, now I need to reply to Ramit.

A bytes object is a container of 8-bit numbers (i.e. range 0 to 255).
If you index it, you'll get an int that supports the XOR operation:

>>> b1 = b'a'
>>> b2 = b'b'
>>> b1[0]
97
>>> b2[0]
98
>>> bin(b1[0])
'0b1100001'
>>> bin(b2[0])
'0b1100010'
>>> bin(b1[0] ^ b2[0])
'0b11'

You can use the int method  "from_bytes" to XOR two bitstrings stored
as Python bytes:

>>> b3 = b'aaaa'
>>> b4 = b'bbbb'
>>> bin(int.from_bytes(b3, 'big') ^ int.from_bytes(b4, 'big'))
'0b11000000110000001100000011'

The computation is done between int objects, not strings. Creating a
string using "bin" is just for presentation.

P.S.:

Instead of "bin" you can use the "format" command to have more
control, such as for zero padding. The integer format code "b" is for
a binary representation. Preceding it by a number starting with zero
will pad with zeros to the given number of characters (e.g. 032 will
prepend zeros to make the result at least 32 characters long):

>>> r = int.from_bytes(b3, 'big') ^ int.from_bytes(b4, 'big')
>>> format(r, "032b")
'00000011000000110000001100000011'

Instead of hard coding the length (e.g. "032"), you can use the length
of the input bitstrings to calculate the size of the result:

>>> size = 8 * max(len(b3), len(b4))
>>> format(r, "0%db" % size)
'00000011000000110000001100000011'

From wayne at waynewerner.com  Thu Jul 19 13:24:34 2012
From: wayne at waynewerner.com (Wayne Werner)
Date: Thu, 19 Jul 2012 06:24:34 -0500 (CDT)
Subject: [Tutor] suggestion for an editor
In-Reply-To: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>
References: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1207190622530.3730@gilgamesh>

On Thu, 19 Jul 2012, Bala subramanian wrote:

> Friends,
> At present i write programs using vi editor. I am interested to change to something else. My specific need is that i want to select a portion/small segment of my
> program (for eg. a nested loop) and then monitor processing time it takes for that portion while i run the program. By this i hope to find the segment that takes
> time and modify to achieve better speed. Can someone please share their experience.

I'm not sure how vi has anything to do with the speed of your program(!)

For performance measurements you should look into the Timeit module. How 
long does it take your program to run currently? After all, premature 
optimisation is the root of all evil...


-Wayne

From sander.sweers at gmail.com  Thu Jul 19 14:04:04 2012
From: sander.sweers at gmail.com (Sander Sweers)
Date: Thu, 19 Jul 2012 14:04:04 +0200
Subject: [Tutor] Pragmatic Unicode, or, How do I stop the pain?
Message-ID: <CACipEsjHjg=91ZdPrnUZqNeRYcvF_08Bit=KzVHEwh81wYEcMg@mail.gmail.com>

https://www.youtube.com/watch?v=sgHbC6udIqc

This is a very good talk on Unicode which was done at PyCon US 2012.
It helped me a lot to understand the pain.

Greets
Sander

From pyprog05 at gmail.com  Thu Jul 19 19:33:02 2012
From: pyprog05 at gmail.com (PyProg PyProg)
Date: Thu, 19 Jul 2012 19:33:02 +0200
Subject: [Tutor] Flatten a list in tuples and remove doubles
Message-ID: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>

Hi all,

I would get a new list as:

[(0, '3eA', 'Dupont', 'Juliette', '11.0/10.0', '4.0/5.0', '17.5/30.0',
'3.0/5.0', '4.5/10.0', '35.5/60.0'), (1, '3eA', 'Pop', 'Iggy',
'12.0/10.0', '3.5/5.0', '11.5/30.0', '4.0/5.0', '5.5/10.0',
'7.5/10.0', '40.5/60.0')]

... from this one:

[(0, '3eA', 'Dupont', 'Juliette', 0, 11.0, 10.0), (0, '3eA', 'Dupont',
'Juliette', 1, 4.0, 5.0), (0, '3eA', 'Dupont', 'Juliette', 2, 17.5,
30.0), (0, '3eA', 'Dupont', 'Juliette', 3, 3.0, 5.0), (0, '3eA',
'Dupont', 'Juliette', 4, 4.5, 10.0), (0, '3eA', 'Dupont', 'Juliette',
5, 35.5, 60.0), (1, '3eA', 'Pop', 'Iggy', 0, 12.0, 10.0), (1, '3eA',
'Pop', 'Iggy', 1, 3.5, 5.0), (1, '3eA', 'Pop', 'Iggy', 2, 11.5, 30.0),
(1, '3eA', 'Pop', 'Iggy', 3, 4.0, 5.0), (1, '3eA', 'Pop', 'Iggy', 4,
5.5, 10.0), (1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0)]

How to make that ? I'm looking for but for now I can't do it.

Thanks in advance.

a+

-- 
http://ekd.tuxfamily.org
http://ekdm.wordpress.com
http://glouk.legtux.org/guiescputil
http://lcs.dunois.clg14.ac-caen.fr/~alama/blog
http://lprod.org/wiki/doku.php/video:encodage:avchd_converter

From pyprog05 at gmail.com  Thu Jul 19 19:35:06 2012
From: pyprog05 at gmail.com (PyProg PyProg)
Date: Thu, 19 Jul 2012 19:35:06 +0200
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
Message-ID: <CAC0U9k7cgrGetYGd8OLrLSS8P5SiJUdEo3ojsZ7ah4LKmqPVjw@mail.gmail.com>

Oh I forgot to mention, with Python 2 (2.7).

-- 
http://ekd.tuxfamily.org
http://ekdm.wordpress.com
http://glouk.legtux.org/guiescputil
http://lcs.dunois.clg14.ac-caen.fr/~alama/blog
http://lprod.org/wiki/doku.php/video:encodage:avchd_converter

From alan.gauld at btinternet.com  Thu Jul 19 19:47:43 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 19 Jul 2012 18:47:43 +0100
Subject: [Tutor] suggestion for an editor
In-Reply-To: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>
References: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>
Message-ID: <ju9h7v$bni$1@dough.gmane.org>

On 19/07/12 09:09, Bala subramanian wrote:
> Friends,
> At present i write programs using vi editor. I am interested to change
> to something else. My specific need is that i want to select a
> portion/small segment of my program (for eg. a nested loop) and then
> monitor processing time it takes for that portion while i run the
> program.

I suspect its not a new editor you need but the profile module...
Take a look at its documentation.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From shantanoo at gmail.com  Thu Jul 19 20:06:13 2012
From: shantanoo at gmail.com (=?utf-8?B?4KS24KSC4KSk4KSo4KWC?=)
Date: Thu, 19 Jul 2012 23:36:13 +0530
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
Message-ID: <168F3B49-2E17-4589-90BD-10E156136E40@gmail.com>

You may use 'set'.

e.g.

===
>>> x
[(1, 2, 3), (1, 1), (2, 2), (1, 1), (2, 2)]
>>> set(x)
set([(2, 2), (1, 1), (1, 2, 3)])
===

On 19-Jul-2012, at 11:03 PM, PyProg PyProg wrote:

> Hi all,
> 
> I would get a new list as:
> 
> [(0, '3eA', 'Dupont', 'Juliette', '11.0/10.0', '4.0/5.0', '17.5/30.0',
> '3.0/5.0', '4.5/10.0', '35.5/60.0'), (1, '3eA', 'Pop', 'Iggy',
> '12.0/10.0', '3.5/5.0', '11.5/30.0', '4.0/5.0', '5.5/10.0',
> '7.5/10.0', '40.5/60.0')]
> 
> ... from this one:
> 
> [(0, '3eA', 'Dupont', 'Juliette', 0, 11.0, 10.0), (0, '3eA', 'Dupont',
> 'Juliette', 1, 4.0, 5.0), (0, '3eA', 'Dupont', 'Juliette', 2, 17.5,
> 30.0), (0, '3eA', 'Dupont', 'Juliette', 3, 3.0, 5.0), (0, '3eA',
> 'Dupont', 'Juliette', 4, 4.5, 10.0), (0, '3eA', 'Dupont', 'Juliette',
> 5, 35.5, 60.0), (1, '3eA', 'Pop', 'Iggy', 0, 12.0, 10.0), (1, '3eA',
> 'Pop', 'Iggy', 1, 3.5, 5.0), (1, '3eA', 'Pop', 'Iggy', 2, 11.5, 30.0),
> (1, '3eA', 'Pop', 'Iggy', 3, 4.0, 5.0), (1, '3eA', 'Pop', 'Iggy', 4,
> 5.5, 10.0), (1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0)]
> 
> How to make that ? I'm looking for but for now I can't do it.
> 
> Thanks in advance.
> 
> a+
> 
> -- 
> http://ekd.tuxfamily.org
> http://ekdm.wordpress.com
> http://glouk.legtux.org/guiescputil
> http://lcs.dunois.clg14.ac-caen.fr/~alama/blog
> http://lprod.org/wiki/doku.php/video:encodage:avchd_converter
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor


From ramit.prasad at jpmorgan.com  Thu Jul 19 20:09:03 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 19 Jul 2012 18:09:03 +0000
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416582859@SCACMX008.exchad.jpmchase.net>

> I would get a new list as:
> 
> [(0, '3eA', 'Dupont', 'Juliette', '11.0/10.0', '4.0/5.0', '17.5/30.0',
> '3.0/5.0', '4.5/10.0', '35.5/60.0'), (1, '3eA', 'Pop', 'Iggy',
> '12.0/10.0', '3.5/5.0', '11.5/30.0', '4.0/5.0', '5.5/10.0',
> '7.5/10.0', '40.5/60.0')]
> 
> ... from this one:
> 
> [(0, '3eA', 'Dupont', 'Juliette', 0, 11.0, 10.0), (0, '3eA', 'Dupont',
> 'Juliette', 1, 4.0, 5.0), (0, '3eA', 'Dupont', 'Juliette', 2, 17.5,
> 30.0), (0, '3eA', 'Dupont', 'Juliette', 3, 3.0, 5.0), (0, '3eA',
> 'Dupont', 'Juliette', 4, 4.5, 10.0), (0, '3eA', 'Dupont', 'Juliette',
> 5, 35.5, 60.0), (1, '3eA', 'Pop', 'Iggy', 0, 12.0, 10.0), (1, '3eA',
> 'Pop', 'Iggy', 1, 3.5, 5.0), (1, '3eA', 'Pop', 'Iggy', 2, 11.5, 30.0),
> (1, '3eA', 'Pop', 'Iggy', 3, 4.0, 5.0), (1, '3eA', 'Pop', 'Iggy', 4,
> 5.5, 10.0), (1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0)]
> 
> How to make that ? I'm looking for but for now I can't do it.


Well first thing to do would be to describe the logic behind what
you are doing. Without knowing that it is difficult to come up
with the correct solution. I am guessing you want a list
where fields 1,2,3 (based on the first element being field 0) 
of field a string of `field 5 + '/' + field 6`. But that does
not tell me what field 0 should be in the new format.

This is a pretty crude sample that should work for you.

lookup = {}

for row in old_list:
    key = (row[1],row[2],row[3])
    field_0, ratios = lookup.setdefault( key, (row[0], []) )
    ratios.append( '{0}/{1}'.format( row[5], row[6] ) )
new_list = []
for key, value in lookup.items():
    row = [ value[0], key[0], key[1], key[2] ]
    row.extend(value[1])
    new_list.append(tuple(row))

# Might need to sort to get the exact same results


Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From wolfrage8765 at gmail.com  Thu Jul 19 20:33:39 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 20:33:39 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
Message-ID: <50085303.5060407@gmail.com>



On 07/19/2012 12:15 AM, Prasad, Ramit wrote:
<SNIP>
> I think your basic problem is too much conversion because you do not
> understand the types. A string is represented by a series of bytes
> which are binary numbers. Do you understand the concept behind ASCII?
> Each letter has a numeric representation that are sequential. So the
> string 'abcd' is equivalent to a series of bytes 65,66,67,68. It is
> not equivalent to 65666768 or 65+66+67+68. So your first task is to
> convert each character to the numeric equivalent and store them in a
> list. Once you have them converted to a list of integers, you can
> create another list that is a list of characters.
Sorry for the long delay in getting back to you, I got called to the field.
Thank you, I agree I do feel like I am doing too much conversion. I do
understand the concept behind ASCII at least enough to know about ord()
although I did for get about chr() which is ord()'s reverse function. I
had tried to break them down to the ordinal value, but I really do want
to get the integer and the data down to binary, as it provides an
advantage for the overall program that I am writing. Thank you for your
time.
>
> Look at the functions chr and ord here
> ( http://docs.python.org/py3k/library/functions.html )
>
> Ramit
>
>
> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
> 712 Main Street | Houston, TX 77002
> work phone: 713 - 216 - 5423
>
> --
>
> This email is confidential and subject to important disclaimers and
> conditions including on offers for the purchase or sale of
> securities, accuracy and completeness of information, viruses,
> confidentiality, legal privilege, and legal entity disclaimers,
> available at http://www.jpmorgan.com/pages/disclosures/email.  
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From wolfrage8765 at gmail.com  Thu Jul 19 20:37:14 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 20:37:14 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <ju88il$o3g$1@dough.gmane.org>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
	<ju88il$o3g$1@dough.gmane.org>
Message-ID: <500853DA.10709@gmail.com>



On 07/19/2012 08:14 AM, Mark Lawrence wrote:
> On 19/07/2012 06:41, wolfrage8765 at gmail.com wrote:
>> On Thu, Jul 19, 2012 at 12:16 AM, Dave Angel <d at davea.name> wrote:
>>
>
<SNIP>
> Really?  Are you using a forked version of Python that doesn't need
> indentation after a while loop, or are you speaking with a forked
> tongue? :)  Strangely I believe the latter, so please take note of
> what Dave Angel has told you and post with the correct indentation.
>
http://www.101emailetiquettetips.com/
Number 101 is for you. Good day.
>>
>>>
>>> I'd also recommend you remove a lot of the irrelevant details
>>> there.  if
>>> you have a problem with hexlfy and/or unhexlify, then give a simple
>>> byte
>>> string that doesn't work for you, and somebody can probably identify
>>> why
>>> not.  And if you want people to run your code, include the imports
>>> as well.
>>>
>>> My problem is not specific to hexlify and unhexlify, my problem is
>>> trying
>> to convert from string to binary and back. That is why all of the
>> details,
>> to show I have tried on my own.
>> Sorry that I forgot to include sys and os for imports.
>>
>>
>>> As it is, you're apparently looping, comparing the byte memory size
>>> of a
>>> string (which is typically 4 bytes per character) with the number of
>>> significant bits in an unrelated number.
>>>
>>> I suspect what you want is something resembling (untested):
>>>
>>>      mybytes = bytes( "%x" % data, "ascii")
>>>      newdata = binascii.unexlify(mybytes)
>>>
>>> I was comparing them but I think I understand how to compare them well,
>> now I want to convert them both to binary so that I can XOR them
>> together.
>> Thank you for your time and help Dave, now I need to reply to Ramit.
>>
>>>
>>> -- 
>>> DaveA
>>>
>>
>>
>>
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/mailman/listinfo/tutor
>>
>
>

From wolfrage8765 at gmail.com  Thu Jul 19 20:40:09 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 20:40:09 +0200
Subject: [Tutor] Fwd:  string to binary and back... Python 3
In-Reply-To: <CANLXbfCbj=KHWhNHOXdDv19_hJsiVxVwP4p-mAJ5=YUL9seFWw@mail.gmail.com>
References: <5007258E.6010206@gmail.com>
	<CANLXbfCbj=KHWhNHOXdDv19_hJsiVxVwP4p-mAJ5=YUL9seFWw@mail.gmail.com>
Message-ID: <50085489.1020103@gmail.com>



On 07/19/2012 10:29 AM, Walter Prins wrote:
> Hi,
>
> Just to show you your original message contained no indentation
> whatsoever.  You might want to check your mail client settings and do
> some experiments to make sure that indentation spaces are let through
> unmolested and not stripped anywhere, otherwise the current little
> brouhaha about formatting will result.  You have to admit, it's not
> easy to read the code below with zero indentation present... :)
Thank you for pointing that out, I did not realize it as I had copied
and pasted it from the python file I was working on. I guess Thunderbird
edited the email on me, even though I had put it into plain text mode.
Next time perhaps I will just attach the file if that is acceptable
rather than getting attacked for what my mail editor did.
>
> Regards
>
> Walter
>
>
> ---------- Forwarded message ----------
> From: Jordan <wolfrage8765 at gmail.com>
> Date: 18 July 2012 22:07
> Subject: [Tutor] string to binary and back... Python 3
> To: tutor at python.org
>
>
> OK so I have been trying for a couple days now and I am throwing in the
> towel, Python 3 wins this one.
> I want to convert a string to binary and back again like in this
> question: Stack Overflow: Convert Binary to ASCII and vice versa
> (Python)
> <http://stackoverflow.com/questions/7396849/convert-binary-to-ascii-and-vice-versa-python>
> But in Python 3 I consistently get  some sort of error relating to the
> fact that nothing but bytes and bytearrays support the buffer interface
> or I get an overflow error because something is too large to be
> converted to bytes.
> Please help me and then explian what I am not getting that is new in
> Python 3. I would like to point out I realize that binary, hex, and
> encodings are all a very complex subject and so I do not expect to
> master it but I do hope that I can gain a deeper insight. Thank you all.
>
> test_script.py:
> import binascii
>
> test_int = 109
>
> test_int = int(str(test_int) + '45670')
> data = 'Testing XOR Again!'
>
> while sys.getsizeof(data) > test_int.bit_length():
>
> test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1), 'big')))
>
> print('Bit Length: ' + str(test_int.bit_length()))
>
> key = test_int # Yes I know this is an unnecessary step...
>
> data = bin(int(binascii.hexlify(bytes(data, 'UTF-8')), 16))
>
> print(data)
>
> data = int(data, 2)
>
> print(data)
>
> data = binascii.unhexlify('%x' % data)
>
>
> wolfrage at lm12-laptop02 ~/Projects $ python3 test_script.py
> Bit Length: 134
> 0b10101000110010101110011011101000110100101101110011001110010000001011000010011110101001000100000010000010110011101100001011010010110111000100001
> 7351954002991226380810260999848996570230305
> Traceback (most recent call last):
> File "test_script.py", line 24, in <module>
> data = binascii.unhexlify('%x' % data)
> TypeError: 'str' does not support the buffer interface
>
>
>
> test_script2.py:
> import binascii
> test_int = 109
> test_int = int(str(test_int) + '45670')
> data = 'Testing XOR Again!'
> while sys.getsizeof(data) > test_int.bit_length():
> test_int = int(str(test_int) + str(int.from_bytes(os.urandom(1), 'big')))
> print('Bit Length: ' + str(test_int.bit_length()))
> key = test_int # Yes I know this is an unnecessary step...
> data = bin(int(binascii.hexlify(bytes(data, 'UTF-8')), 16))
> print(data)
> data = int(data, 2)
> print(data)
> data = binascii.unhexlify(bytes(data, 'utf8'))
>
>
>
> wolfrage at lm12-laptop02 ~/Projects $ python3 test_script2.py
> Bit Length: 140
> 0b10101000110010101110011011101000110100101101110011001110010000001011000010011110101001000100000010000010110011101100001011010010110111000100001
> 7351954002991226380810260999848996570230305
> Traceback (most recent call last):
> File "test_script.py", line 24, in <module>
> data = binascii.unhexlify(bytes(data, 'utf8'))
> OverflowError: cannot fit 'int' into an index-sized integer
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From ramit.prasad at jpmorgan.com  Thu Jul 19 20:45:13 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 19 Jul 2012 18:45:13 +0000
Subject: [Tutor] Fwd:  string to binary and back... Python 3
In-Reply-To: <50085489.1020103@gmail.com>
References: <5007258E.6010206@gmail.com>
	<CANLXbfCbj=KHWhNHOXdDv19_hJsiVxVwP4p-mAJ5=YUL9seFWw@mail.gmail.com>
	<50085489.1020103@gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF474165828FD@SCACMX008.exchad.jpmchase.net>

> > Just to show you your original message contained no indentation
> > whatsoever.  You might want to check your mail client settings and do
> > some experiments to make sure that indentation spaces are let through
> > unmolested and not stripped anywhere, otherwise the current little
> > brouhaha about formatting will result.  You have to admit, it's not
> > easy to read the code below with zero indentation present... :)

> Thank you for pointing that out, I did not realize it as I had copied
> and pasted it from the python file I was working on. I guess Thunderbird
> edited the email on me, even though I had put it into plain text mode.
> Next time perhaps I will just attach the file if that is acceptable
> rather than getting attacked for what my mail editor did.

A fair amount of the list does not get attachments as this is a 
gateway to newsgroups. Copy/paste works for short code fragments 
if you are sure you are posting in plain text. Otherwise, you
can post to services like pastebin and link to that.

Ramit
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From wolfrage8765 at gmail.com  Thu Jul 19 20:48:06 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 20:48:06 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5007E576.4060108@davea.name>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
	<5007E576.4060108@davea.name>
Message-ID: <50085666.9090307@gmail.com>



On 07/19/2012 12:46 PM, Dave Angel wrote:
> On 07/19/2012 01:41 AM, wolfrage8765 at gmail.com wrote:
>> On Thu, Jul 19, 2012 at 12:16 AM, Dave Angel <d at davea.name> wrote:
<SNIP>
> That was just the first line that was not indented.  If I thought you
> had a one-line while loop, I certainly would have just indented it.  But
> I'm sure you have some unknown number of additional lines that were
> indented in your original.  Please post in text form.
I see now, I am sorry I did not know that Thunderbird had eliminated all
of my indentation. I had set it to plain text, but I guess it is not to
be trusted. I simply looked at the reply email and saw the line that you
pointed out and at that time figured Thunderbird had just word wrapped
that line on me. Would it be acceptable to add an attached Python file?
<SNIP>
> Lots of details that have nothing to do with it.  For example, that
> whole thing about adding random digits together.  You could replace the
> whole thing with a simple assignment of a value that doesn't work for you.
OK I will, that was a test script and I was testing multiple things, I
did try to get rid of most of the cruft but I will attempt to do better
in the future. <SNIP>
>> now I want to convert them both to binary so that I can XOR them together.
>> Thank you for your time and help Dave, now I need to reply to Ramit.
> Ah, so you don't actually want binary at all!!!   Why not state the real
> problem up front?  You can XOR two integers, without bothering to
> convert to a string of ones and zeroes.  Use the carat operator.
>
> print( 40 ^ 12)
>
> I suspect there's an equivalent for strings or byte-strings.  But if
> not, it's a simple loop.
Actually I do want binary, as it serves as an advantage for the overall
program that I am building.
>
>

From ramit.prasad at jpmorgan.com  Thu Jul 19 20:53:31 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 19 Jul 2012 18:53:31 +0000
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <50085303.5060407@gmail.com>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>

> > I think your basic problem is too much conversion because you do not
> > understand the types. A string is represented by a series of bytes
> > which are binary numbers. Do you understand the concept behind ASCII?
> > Each letter has a numeric representation that are sequential. So the
> > string 'abcd' is equivalent to a series of bytes 65,66,67,68. It is
> > not equivalent to 65666768 or 65+66+67+68. So your first task is to
> > convert each character to the numeric equivalent and store them in a
> > list. Once you have them converted to a list of integers, you can
> > create another list that is a list of characters.

> Sorry for the long delay in getting back to you, I got called to the field.
> Thank you, I agree I do feel like I am doing too much conversion. I do
> understand the concept behind ASCII at least enough to know about ord()
> although I did for get about chr() which is ord()'s reverse function. I
> had tried to break them down to the ordinal value, but I really do want
> to get the integer and the data down to binary, as it provides an
> advantage for the overall program that I am writing. Thank you for your
> time.

Why not explain your usecase? Technically, everything is binary
on a computer so the question is why do *you* need to see the
binary form? Anyway, you can get the binary string by doing
`bin(ord(character))` and reverse it by doing 
`chr(int(binary_string,2))`. [1]

If you are doing some kind of XOR (I think your first email mentioned 
it) then you can XOR integers. Unless you are doing some kind of 
display of binary output, you usually do not need to actually see 
the binary string as most binary manipulation can be done via the 
integer value.

 [1] - Thanks to Steven.




Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From wolfrage8765 at gmail.com  Thu Jul 19 20:56:11 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 20:56:11 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <alpine.DEB.2.02.1207190530580.3730@gilgamesh>
References: <5007258E.6010206@gmail.com>
	<alpine.DEB.2.02.1207190530580.3730@gilgamesh>
Message-ID: <5008584B.1090504@gmail.com>

My response is down lower, thank you Wayne.

On 07/19/2012 12:52 PM, Wayne Werner wrote:
> I'll preface my response by saying that I know/understand fairly
> little about
> it, but since I've recently been smacked by this same issue when
> converting
> stuff to Python3, I'll see if I can explain it in a way that makes sense.
>
> On Wed, 18 Jul 2012, Jordan wrote:
>
>> OK so I have been trying for a couple days now and I am throwing in the
>> towel, Python 3 wins this one.
>> I want to convert a string to binary and back again like in this
>> question: Stack Overflow: Convert Binary to ASCII and vice versa
>> (Python)
>> <http://stackoverflow.com/questions/7396849/convert-binary-to-ascii-and-vice-versa-python>
>>
>> But in Python 3 I consistently get  some sort of error relating to the
>> fact that nothing but bytes and bytearrays support the buffer interface
>> or I get an overflow error because something is too large to be
>> converted to bytes.
>> Please help me and then explian what I am not getting that is new in
>> Python 3. I would like to point out I realize that binary, hex, and
>> encodings are all a very complex subject and so I do not expect to
>> master it but I do hope that I can gain a deeper insight. Thank you all.
>
> The way I've read it - stop thinking about strings as if they are
> text. The
> biggest reason that all this has changed is because Python has grown
> up and
> entered the world where Unicode actually matters. To us poor shmucks
> in the
> English speaking countries of the world it's all very confusing
> becaust it's
> nothing we have to deal with. 26 letters is perfectly fine for us -
> and if we
> want uppercase we'll just throw another 26. Add a few dozen puncuation
> marks
> and 256 is a perfectly fine amount of characters.
>
> To make a slightly relevant side trip, when you were a kid did you
> ever send
> "secret" messages to a friend with a code like this?
>
> A = 1
> B = 2
> .
> .
> .
> Z = 26
>
> Well, that's basically what is going on when it comes to
> bytes/text/whatever.
> When you input some text, Python3 believes that whatever you wrote was
> encoded
> with Unicode. The nice thing for us 26-letter folks is that the ASCII
> alphabet
> we're so used to just so happens to map quite well to Unicode
> encodings - so
> 'A' in ASCII is the same number as 'A' in utf-8.
>
> Now, here's the part that I had to (and still need to) wrap my mind
> around - if
> the string is "just bytes" then it doesn't really matter what the
> string is
> supposed to represent. It could represent the LATIN-1 character set. Or
> UTF-8, -16, or some other weird encoding. And all the operations that are
> supposed to modify these strings of bytes (e.g. removing spaces,
> splitting on a
> certain "character", etc.) still work. Because if I have this string:
>
> 9 45 12 9 13 19 18 9 12 99 102
>
> and I tell you to split on the 9's, it doesn't matter if that's some
> weird
> ASCII character, or some equally weird UTF character, or something else
> entirely. And I don't have to worry about things getting munged up
> when I try
> to stick Unicode and ASCII values together - because they're converted
> to bytes
> first.
>
> So the question is, of course, if it's all bytes, then why does it
> look like
> text when I print it out? Well, that's because Python converts that
> byte stream
> to Unicode text when it's printed. Or ASCII, if you tell it to.
>
> But Python3 has converted all(?) of those functions that used to
> operate on
> text and made them operate on byte streams instead. Except for the
> ones that
> operate on text ;)
>
>
>
> Well, I hope that's of some use and isn't too much of a lie - like I
> said, I'm
> still trying to wrap my head around things and I've found that
> explaining (or
> trying to explain) to someone else is often the best way to work out
> the idea
> in your own head. If I've gone too far astray I'm sure the other
> helpful folks
> here will correct me :)
>
Thank you for the vary informative post, every bit helps. It has
certainly been a challenge for me with the new everything is bytes
scheme, especially how everything has to be converted to bytes prior to
going on a buffer.
> HTH,
> Wayne

From wolfrage8765 at gmail.com  Thu Jul 19 21:08:46 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 21:08:46 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <CACL+1aukOw1nnGOK6EX7-WF=Xv=xMVrv84aXSE4QuyQMw6JtrA@mail.gmail.com>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
	<CACL+1aukOw1nnGOK6EX7-WF=Xv=xMVrv84aXSE4QuyQMw6JtrA@mail.gmail.com>
Message-ID: <50085B3E.1010802@gmail.com>

A question I have for the group before I respond is a option that I saw
that I had earlier was to ord() each element of a string and then bin()
that number. But since bin() produces a string I could not figure out
the correct way to attach two bin() outputs back together again due to
the leading 'b' and even if I use lstrip('b') I was not sure if that
would be correct?
My next hesitation is can the same or at least similar techniques be
applied to a file? I want to be able to work on both files and strings.

On 07/19/2012 01:22 PM, eryksun wrote:
> On Thu, Jul 19, 2012 at 1:41 AM, wolfrage8765 at gmail.com
> <wolfrage8765 at gmail.com> wrote:
>> I was comparing them but I think I understand how to compare them well, now
>> I want to convert them both to binary so that I can XOR them together. Thank
>> you for your time and help Dave, now I need to reply to Ramit.
> A bytes object is a container of 8-bit numbers (i.e. range 0 to 255).
> If you index it, you'll get an int that supports the XOR operation:
>
>>>> b1 = b'a'
>>>> b2 = b'b'
>>>> b1[0]
> 97
>>>> b2[0]
> 98
>>>> bin(b1[0])
> '0b1100001'
>>>> bin(b2[0])
> '0b1100010'
>>>> bin(b1[0] ^ b2[0])
> '0b11'
>
> You can use the int method  "from_bytes" to XOR two bitstrings stored
> as Python bytes:
>
>>>> b3 = b'aaaa'
>>>> b4 = b'bbbb'
>>>> bin(int.from_bytes(b3, 'big') ^ int.from_bytes(b4, 'big'))
> '0b11000000110000001100000011'
>
> The computation is done between int objects, not strings. Creating a
> string using "bin" is just for presentation.
>
> P.S.:
>
> Instead of "bin" you can use the "format" command to have more
> control, such as for zero padding. The integer format code "b" is for
> a binary representation. Preceding it by a number starting with zero
> will pad with zeros to the given number of characters (e.g. 032 will
> prepend zeros to make the result at least 32 characters long):
The control sounds good and I may need that latter (To adjust things to
a fixed length), but for the purpose of XORing a message padding a key
with zeros would not be desirable if Eve was able to get her hands on
the source code.
>>>> r = int.from_bytes(b3, 'big') ^ int.from_bytes(b4, 'big')
>>>> format(r, "032b")
> '00000011000000110000001100000011'
>
> Instead of hard coding the length (e.g. "032"), you can use the length
> of the input bitstrings to calculate the size of the result:
That sounds good.
>>>> size = 8 * max(len(b3), len(b4))
>>>> format(r, "0%db" % size)
> '00000011000000110000001100000011'
Is this output the output for size rather than the two variables joined
together?

From wolfrage8765 at gmail.com  Thu Jul 19 21:19:56 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 21:19:56 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
Message-ID: <50085DDC.5070405@gmail.com>



On 07/19/2012 08:53 PM, Prasad, Ramit wrote:
>>> I think your basic problem is too much conversion because you do not
>>> understand the types. A string is represented by a series of bytes
>>> which are binary numbers. Do you understand the concept behind ASCII?
>>> Each letter has a numeric representation that are sequential. So the
>>> string 'abcd' is equivalent to a series of bytes 65,66,67,68. It is
>>> not equivalent to 65666768 or 65+66+67+68. So your first task is to
>>> convert each character to the numeric equivalent and store them in a
>>> list. Once you have them converted to a list of integers, you can
>>> create another list that is a list of characters.
>> Sorry for the long delay in getting back to you, I got called to the field.
>> Thank you, I agree I do feel like I am doing too much conversion. I do
>> understand the concept behind ASCII at least enough to know about ord()
>> although I did for get about chr() which is ord()'s reverse function. I
>> had tried to break them down to the ordinal value, but I really do want
>> to get the integer and the data down to binary, as it provides an
>> advantage for the overall program that I am writing. Thank you for your
>> time.
> Why not explain your usecase? Technically, everything is binary
> on a computer so the question is why do *you* need to see the
> binary form? Anyway, you can get the binary string by doing
> `bin(ord(character))` and reverse it by doing 
> `chr(int(binary_string,2))`. [1]
OK. I am using one time pads to XOR data, but the one time pads (keys)
are very large numbers, converting them to binary increases their size
exponentially, which allows me to get more XORing done out of a single
key. I am XORing both files and strings so I need to have code that can
do both even if that means two branches of code via an if/else perhaps
with an isinstance(data, str).
I do not need to actually see the binary form.
> If you are doing some kind of XOR (I think your first email mentioned 
> it) then you can XOR integers. Unless you are doing some kind of 
> display of binary output, you usually do not need to actually see 
> the binary string as most binary manipulation can be done via the 
> integer value.
Agreed. Although the visual does help for validation (seeing is believing).
>  [1] - Thanks to Steven.
Yes thank you Steven. I am working on the code now to see if I can make
the above work for me, if I need further help I will be back.
Thank you all again for your time.
>
>
>
>
> Ramit
>
> This email is confidential and subject to important disclaimers and
> conditions including on offers for the purchase or sale of
> securities, accuracy and completeness of information, viruses,
> confidentiality, legal privilege, and legal entity disclaimers,
> available at http://www.jpmorgan.com/pages/disclosures/email.  
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From ramit.prasad at jpmorgan.com  Thu Jul 19 21:23:05 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 19 Jul 2012 19:23:05 +0000
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <50085B3E.1010802@gmail.com>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
	<CACL+1aukOw1nnGOK6EX7-WF=Xv=xMVrv84aXSE4QuyQMw6JtrA@mail.gmail.com>
	<50085B3E.1010802@gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF474165829B8@SCACMX008.exchad.jpmchase.net>

> A question I have for the group before I respond is a option that I saw
> that I had earlier was to ord() each element of a string and then bin()
> that number. But since bin() produces a string I could not figure out
> the correct way to attach two bin() outputs back together again due to
> the leading 'b' and even if I use lstrip('b') I was not sure if that
> would be correct?

bin(integer).split('b')[1].zfill( multiple_of_eight )

> My next hesitation is can the same or at least similar techniques be
> applied to a file? I want to be able to work on both files and strings.

Probably, but it depends on what you are trying to do and what
data you are dealing with.

Ramit
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From selbyrowleycannon at ymail.com  Thu Jul 19 21:29:23 2012
From: selbyrowleycannon at ymail.com (Selby Rowley-Cannon)
Date: Thu, 19 Jul 2012 20:29:23 +0100
Subject: [Tutor] check against multiple variables
Message-ID: <50086013.1090500@ymail.com>

I am using a hash table in a small randomization program. I know that 
some hash functions can be prone to collisions, so I need a way to 
detect collisions.
The 'hash value' will be stored as a variable. I do not want to check it 
against each singular hash value, as there will be many; I need a way to 
check it against all hash values at once (if possible.) Sorry for those 
who like to reference, but there is no source code as of yet. I will 
need this to be solved before I can start writing, sorry!

If you need any extra info let me know.

         -Selby

From emile at fenx.com  Thu Jul 19 21:49:22 2012
From: emile at fenx.com (Emile van Sebille)
Date: Thu, 19 Jul 2012 12:49:22 -0700
Subject: [Tutor] check against multiple variables
In-Reply-To: <50086013.1090500@ymail.com>
References: <50086013.1090500@ymail.com>
Message-ID: <ju9oc5$abj$1@dough.gmane.org>

On 7/19/2012 12:29 PM Selby Rowley-Cannon said...
> I am using a hash table in a small randomization program. I know that
> some hash functions can be prone to collisions, so I need a way to
> detect collisions.
> The 'hash value' will be stored as a variable. I do not want to check it
> against each singular hash value, as there will be many; I need a way to
> check it against all hash values at once (if possible.)

so keeping the hash values in a dict would allow you to test as follows:

if new_hash_value in dict_of_hash_values:
     # and bob's your uncle.

Emile


> Sorry for those
> who like to reference, but there is no source code as of yet. I will
> need this to be solved before I can start writing, sorry!
>
> If you need any extra info let me know.
>
>          -Selby
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>




From d at davea.name  Thu Jul 19 21:53:24 2012
From: d at davea.name (Dave Angel)
Date: Thu, 19 Jul 2012 15:53:24 -0400
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <50085DDC.5070405@gmail.com>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
	<50085DDC.5070405@gmail.com>
Message-ID: <500865B4.9050105@davea.name>

On 07/19/2012 03:19 PM, Jordan wrote:
> <SNIP>
>
> OK. I am using one time pads to XOR data, but the one time pads (keys)
> are very large numbers, converting them to binary increases their size
> exponentially, which allows me to get more XORing done out of a single

You want to explain this impossibility of increasing size
exponentially?  If you're wanting to waste memory, there are better
ways.  But it's only 8 times as big to save a string of 1's and zeros as
to save the large-int they represent.  And multiplying by 8 isn't an
exponential function.

> key. I am XORing both files and strings so I need to have code that can
> do both even if that means two branches of code via an if/else perhaps
> with an isinstance(data, str).
> I do not need to actually see the binary form.
>

Then don't use the binary form.  It doesn't make the computation any
more "powerful" and it'll certainly slow it down.

Are you trying to match some other program's algorithm, and thus have
strange constraints on your data?  Or are you simply trying to make a
secure way to encrypt binary files, using one-time pads?

A one-time pad is the same size as the message, so you simply need to
convert the message into a large-int, and xor them.


-- 

DaveA


From wolfrage8765 at gmail.com  Thu Jul 19 21:59:39 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 21:59:39 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF474165829B8@SCACMX008.exchad.jpmchase.net>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
	<CACL+1aukOw1nnGOK6EX7-WF=Xv=xMVrv84aXSE4QuyQMw6JtrA@mail.gmail.com>
	<50085B3E.1010802@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF474165829B8@SCACMX008.exchad.jpmchase.net>
Message-ID: <5008672B.6050903@gmail.com>



On 07/19/2012 09:23 PM, Prasad, Ramit wrote:
>> A question I have for the group before I respond is a option that I saw
>> that I had earlier was to ord() each element of a string and then bin()
>> that number. But since bin() produces a string I could not figure out
>> the correct way to attach two bin() outputs back together again due to
>> the leading 'b' and even if I use lstrip('b') I was not sure if that
>> would be correct?
> bin(integer).split('b')[1].zfill( multiple_of_eight )
OK so using this: Hopefully my copy paste works this time.

bin_data = ''

for char in data:

bin_data += bin(ord(char)).split('b')[1].zfill(8)

print(bin_data)

bin_list = [bin_data[x:x + 2] for x in range(0, len(bin_data), 2)]

print(bin_list)



The paste looks good to me at this time.
How do I get back to the string? If I use this:

data2 = []

for item in bin_list:

data2.append(int(item, 2))

print(data2)



The output is all too low of numbers for ord() to convert back to the
correct string.
>
>> My next hesitation is can the same or at least similar techniques be
>> applied to a file? I want to be able to work on both files and strings.
> Probably, but it depends on what you are trying to do and what
> data you are dealing with.
I just want to perform the same conversion on the file data, that is
down to binary and back to it's original state.
I was thinking I would just use the file in binary mode when I open it,
but I am not sure if that is true binary or if it is hex or something
else altogether. I think my confusion came from trying to do both files
and strings at the same time and failing back and forth.
>
> Ramit
> This email is confidential and subject to important disclaimers and
> conditions including on offers for the purchase or sale of
> securities, accuracy and completeness of information, viruses,
> confidentiality, legal privilege, and legal entity disclaimers,
> available at http://www.jpmorgan.com/pages/disclosures/email.  
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From eryksun at gmail.com  Thu Jul 19 22:04:20 2012
From: eryksun at gmail.com (eryksun)
Date: Thu, 19 Jul 2012 16:04:20 -0400
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <50085B3E.1010802@gmail.com>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
	<CACL+1aukOw1nnGOK6EX7-WF=Xv=xMVrv84aXSE4QuyQMw6JtrA@mail.gmail.com>
	<50085B3E.1010802@gmail.com>
Message-ID: <CACL+1as9oHMpGd2sOxmQ+yp-idcoPkJCjztnxXVromF+VXpuSQ@mail.gmail.com>

On Thu, Jul 19, 2012 at 3:08 PM, Jordan <wolfrage8765 at gmail.com> wrote:
>
>>>>> size = 8 * max(len(b3), len(b4))
>>>>> format(r, "0%db" % size)
>> '00000011000000110000001100000011'
> Is this output the output for size rather than the two variables joined
> together?

Using "format" is useful if you need the string to be padded with
zeros for the most significant byte. I wouldn't think it's important
if you're just using the bitstring representation as a sanity check on
your algorithm. In that case you can more easily use "bin".

That said, len(b3) is the number of characters (bytes) in the bytes
object. Since b3 and b4 could be different lengths in general, I took
the max length to use for the zero padding. In this case both b3 and
b4 contain 4 bytes, so "size" is 32.

> OK. I am using one time pads to XOR data, but the one time pads (keys)
> are very large numbers, converting them to binary increases their size
> exponentially, which allows me to get more XORing done out of a single
> key. I am XORing both files and strings so I need to have code that can
> do both even if that means two branches of code via an if/else perhaps
> with an isinstance(data, str).

I'm not an expert with cryptography, but here's a simple XOR example:

>>> from itertools import cycle
>>> text = b'Mary had a little lamb.'
>>> key = b'1234'
>>> cypher = bytes(x^y for x,y in zip(text, cycle(key)))
>>> cypher
b'|SAM\x11ZRP\x11S\x13XXFGXT\x12_U\\P\x1d'
>>> text2 = bytes(x^y for x,y in zip(cypher, cycle(key)))
>>> text2
b'Mary had a little lamb.'

From wolfrage8765 at gmail.com  Thu Jul 19 22:40:06 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 22:40:06 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <500865B4.9050105@davea.name>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
	<50085DDC.5070405@gmail.com> <500865B4.9050105@davea.name>
Message-ID: <500870A6.4090707@gmail.com>



On 07/19/2012 09:53 PM, Dave Angel wrote:
> On 07/19/2012 03:19 PM, Jordan wrote:
>> <SNIP>
>>
>> OK. I am using one time pads to XOR data, but the one time pads (keys)
>> are very large numbers, converting them to binary increases their size
>> exponentially, which allows me to get more XORing done out of a single
> You want to explain this impossibility of increasing size
> exponentially?  If you're wanting to waste memory, there are better
> ways.  But it's only 8 times as big to save a string of 1's and zeros as
> to save the large-int they represent.  And multiplying by 8 isn't an
> exponential function.
>
Yes if you wish to dissect my words the wrong word was chosen...
>> key. I am XORing both files and strings so I need to have code that can
>> do both even if that means two branches of code via an if/else perhaps
>> with an isinstance(data, str).
>> I do not need to actually see the binary form.
>>
> Then don't use the binary form.  It doesn't make the computation any
> more "powerful" and it'll certainly slow it down.
The title of the question is string to binary and back.
>
> Are you trying to match some other program's algorithm, and thus have
> strange constraints on your data?  Or are you simply trying to make a
> secure way to encrypt binary files, using one-time pads?
I already answered this question...
>
> A one-time pad is the same size as the message, so you simply need to
> convert the message into a large-int, and xor them.
>
>

From ramit.prasad at jpmorgan.com  Thu Jul 19 22:41:32 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 19 Jul 2012 20:41:32 +0000
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5008672B.6050903@gmail.com>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
	<CACL+1aukOw1nnGOK6EX7-WF=Xv=xMVrv84aXSE4QuyQMw6JtrA@mail.gmail.com>
	<50085B3E.1010802@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF474165829B8@SCACMX008.exchad.jpmchase.net>
	<5008672B.6050903@gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416582AB2@SCACMX008.exchad.jpmchase.net>

> > bin(integer).split('b')[1].zfill( multiple_of_eight )
> OK so using this: Hopefully my copy paste works this time.
> 
> bin_data = ''
> 
> for char in data:
> 
> bin_data += bin(ord(char)).split('b')[1].zfill(8)
> 
> print(bin_data)
> 
> bin_list = [bin_data[x:x + 2] for x in range(0, len(bin_data), 2)]
> 
> print(bin_list)
> 
> 
> 
> The paste looks good to me at this time.

Not to me, but I can probably figure out enough based on this.

> How do I get back to the string? If I use this:
> 
> data2 = []
> 
> for item in bin_list:
> 
> data2.append(int(item, 2))
> 
> print(data2)
> 
> 
> 
> The output is all too low of numbers for ord() to convert back to the
> correct string.
> >

Sure, this makes perfect sense to me :) (adding indent)

for char in data:
    bin_data += bin(ord(char)).split('b')[1].zfill(8)
bin_list = [bin_data[x:x + 2] for x in range(0, len(bin_data), 2)]

Why are you grabbing 2 binary digits? The only possibilities are 0,1,2,3
and none are ASCII letters. You should be grabbing 8 at a time.

bin_data = [ bin(ord(char)).split('b')[1].zfill(8) for char in data ]
bin_string = ''.join(bin_data)
bin_list = [ chr( int(char, 2) ) for char in bin_data ]

I am not really sure what you are getting at with XOR and one time
padding, but it has been a while since I have done any encryption.

I would think you could do all this by just converting everything
to int and then adding/replacing the pad in the list of ints.


Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From wolfrage8765 at gmail.com  Thu Jul 19 22:43:44 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 22:43:44 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <CACL+1as9oHMpGd2sOxmQ+yp-idcoPkJCjztnxXVromF+VXpuSQ@mail.gmail.com>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
	<CACL+1aukOw1nnGOK6EX7-WF=Xv=xMVrv84aXSE4QuyQMw6JtrA@mail.gmail.com>
	<50085B3E.1010802@gmail.com>
	<CACL+1as9oHMpGd2sOxmQ+yp-idcoPkJCjztnxXVromF+VXpuSQ@mail.gmail.com>
Message-ID: <50087180.5060107@gmail.com>



On 07/19/2012 10:04 PM, eryksun wrote:
> On Thu, Jul 19, 2012 at 3:08 PM, Jordan <wolfrage8765 at gmail.com> wrote:
<SNIP>
> I'm not an expert with cryptography, but here's a simple XOR example:
>>>> from itertools import cycle
>>>> text = b'Mary had a little lamb.'
>>>> key = b'1234'
>>>> cypher = bytes(x^y for x,y in zip(text, cycle(key)))
>>>> cypher
> b'|SAM\x11ZRP\x11S\x13XXFGXT\x12_U\\P\x1d'
>>>> text2 = bytes(x^y for x,y in zip(cypher, cycle(key)))
>>>> text2
> b'Mary had a little lamb.'
Hmm interesting, I am reading up on on itertools.cycle() and zip now.
Thanks always more than one way to solve a problem.

From wolfrage8765 at gmail.com  Thu Jul 19 22:58:16 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 22:58:16 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF47416582AB2@SCACMX008.exchad.jpmchase.net>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
	<CACL+1aukOw1nnGOK6EX7-WF=Xv=xMVrv84aXSE4QuyQMw6JtrA@mail.gmail.com>
	<50085B3E.1010802@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF474165829B8@SCACMX008.exchad.jpmchase.net>
	<5008672B.6050903@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582AB2@SCACMX008.exchad.jpmchase.net>
Message-ID: <500874E8.4010801@gmail.com>

Sorry, I am not sure why Thunderbird is stripping the spaces, may have
something to do with a plug-in that I have installed, I will have to
look into it.

On 07/19/2012 10:41 PM, Prasad, Ramit wrote:
> Sure, this makes perfect sense to me :) (adding indent)
>
> for char in data:
>     bin_data += bin(ord(char)).split('b')[1].zfill(8)
> bin_list = [bin_data[x:x + 2] for x in range(0, len(bin_data), 2)]
>
> Why are you grabbing 2 binary digits? The only possibilities are 0,1,2,3
> and none are ASCII letters. You should be grabbing 8 at a time.
Right, sorry, first time working with binary and I was confused by a
previous attempt.
>
> bin_data = [ bin(ord(char)).split('b')[1].zfill(8) for char in data ]
> bin_string = ''.join(bin_data)
> bin_list = [ chr( int(char, 2) ) for char in bin_data ]
Thank you exactly what I was looking for!
>
> I am not really sure what you are getting at with XOR and one time
> padding, but it has been a while since I have done any encryption.
And I have just started reading "Applied Cryptography", so I am putting
some of what I learn into practice.
>
> I would think you could do all this by just converting everything
> to int and then adding/replacing the pad in the list of ints.
At first I was essentially doing just that, but when I first converted
the large integers that are being used for the one time pad as the key
to binary I saw how much larger it was, and then realized that was the
bit length of the integer (technically Long). By doing that, I can get
more out of the one time pad, but if you XOR binary against Ord, very
few values will be changed because binary is only 1s and 0s as you know.
To optimize the keys use, whether it wastes memory or not, I wanted to
use binary on binary, this really comes into play with files, not so
much the shorter strings.
But since you bring if up, how would you convert a file to a list of ints?
>
>
> Ramit
>
> This email is confidential and subject to important disclaimers and
> conditions including on offers for the purchase or sale of
> securities, accuracy and completeness of information, viruses,
> confidentiality, legal privilege, and legal entity disclaimers,
> available at http://www.jpmorgan.com/pages/disclosures/email.  
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From ramit.prasad at jpmorgan.com  Thu Jul 19 22:48:29 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 19 Jul 2012 20:48:29 +0000
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <500870A6.4090707@gmail.com>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
	<50085DDC.5070405@gmail.com> <500865B4.9050105@davea.name>
	<500870A6.4090707@gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416582AE1@SCACMX008.exchad.jpmchase.net>

> >> <SNIP>
> >>
> >> OK. I am using one time pads to XOR data, but the one time pads (keys)
> >> are very large numbers, converting them to binary increases their size
> >> exponentially, which allows me to get more XORing done out of a single
> > You want to explain this impossibility of increasing size
> > exponentially?  If you're wanting to waste memory, there are better
> > ways.  But it's only 8 times as big to save a string of 1's and zeros as
> > to save the large-int they represent.  And multiplying by 8 isn't an
> > exponential function.
> >
> Yes if you wish to dissect my words the wrong word was chosen...
> >> key. I am XORing both files and strings so I need to have code that can
> >> do both even if that means two branches of code via an if/else perhaps
> >> with an isinstance(data, str).
> >> I do not need to actually see the binary form.
> >>
> > Then don't use the binary form.  It doesn't make the computation any
> > more "powerful" and it'll certainly slow it down.
> The title of the question is string to binary and back.
> >
> > Are you trying to match some other program's algorithm, and thus have
> > strange constraints on your data?  Or are you simply trying to make a
> > secure way to encrypt binary files, using one-time pads?
> I already answered this question...

Yes, you stated that it had to work on string and files, but are the 
files binary? DaveA and I are asking the questions because given
what you are asking it just seems like you are not using the Right
approach. I can touch my nose by touching my nose with my hand, 
or asking the person next to me to pick up my hand and use it to
touch my nose. Both work, one is just faster and easier to
understand.

> >
> > A one-time pad is the same size as the message, so you simply need to
> > convert the message into a large-int, and xor them.

Ramit
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From d at davea.name  Thu Jul 19 23:22:24 2012
From: d at davea.name (Dave Angel)
Date: Thu, 19 Jul 2012 17:22:24 -0400
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5007258E.6010206@gmail.com>
References: <5007258E.6010206@gmail.com>
Message-ID: <50087A90.6050706@davea.name>

On 07/18/2012 05:07 PM, Jordan wrote:
> OK so I have been trying for a couple days now and I am throwing in the
> towel, Python 3 wins this one.

I should have paid more attention to this the first time.  Clearly you
don't want help.


-- 

DaveA


From ramit.prasad at jpmorgan.com  Thu Jul 19 23:28:18 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 19 Jul 2012 21:28:18 +0000
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <500874E8.4010801@gmail.com>
References: <5007258E.6010206@gmail.com> <500735AC.3010404@davea.name>
	<CAOhNYvnx647fDTtvYfbcRqfJnzC9v7nWrN85538vVox9vqq+=A@mail.gmail.com>
	<CACL+1aukOw1nnGOK6EX7-WF=Xv=xMVrv84aXSE4QuyQMw6JtrA@mail.gmail.com>
	<50085B3E.1010802@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF474165829B8@SCACMX008.exchad.jpmchase.net>
	<5008672B.6050903@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582AB2@SCACMX008.exchad.jpmchase.net>
	<500874E8.4010801@gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416582B9A@SCACMX008.exchad.jpmchase.net>

> > bin_data = [ bin(ord(char)).split('b')[1].zfill(8) for char in data ]
> > bin_string = ''.join(bin_data)
> > bin_list = [ chr( int(char, 2) ) for char in bin_data ]
> Thank you exactly what I was looking for!
> >
> > I am not really sure what you are getting at with XOR and one time
> > padding, but it has been a while since I have done any encryption.
> And I have just started reading "Applied Cryptography", so I am putting
> some of what I learn into practice.
> >
> > I would think you could do all this by just converting everything
> > to int and then adding/replacing the pad in the list of ints.
> At first I was essentially doing just that, but when I first converted
> the large integers that are being used for the one time pad as the key
> to binary I saw how much larger it was, and then realized that was the
> bit length of the integer (technically Long). By doing that, I can get
> more out of the one time pad, but if you XOR binary against Ord, very
> few values will be changed because binary is only 1s and 0s as you know.
> To optimize the keys use, whether it wastes memory or not, I wanted to
> use binary on binary, this really comes into play with files, not so
> much the shorter strings.

How are you XOR-ing "binary" against something else? At a low level the data
is pretty similar so that they should be mostly interchangeable. It is
when you start abstracting the data that you have to convert between
abstractions.

Hold on, let me try a different angle. int, binary, and hex version of 
a number (lets say 65) are all just different representations of the same
number. The only thing that changes is the base.

65 in octal (base 10) is 65
65 in hex (base 16) is 41
65 in binary (base 2 ) is 1000001

But they are ALL the same number. 
>>> int( '65', 10 )
65
>>> int( '41', 16 )
65
>>> int( '1000001', 2 )
65


> But since you bring if up, how would you convert a file to a list of ints?

with open(filename, 'r' ) as f:
    ints = [ ord( char ) for line in f for char in line ]

Now all you need to do is modify the list to include your padding.

> but when I first converted
> the large integers that are being used for the one time pad as the key
> to binary I saw how much larger it was, and then realized that was the
> bit length of the integer (technically Long). By doing that, I can get
> more out of the one time pad,

Large integers? Are you adding the integers for some reason? Extended 
ASCII only has ordinal values less than 256.

Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From wolfrage8765 at gmail.com  Thu Jul 19 23:32:37 2012
From: wolfrage8765 at gmail.com (Jordan)
Date: Thu, 19 Jul 2012 23:32:37 +0200
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF47416582AE1@SCACMX008.exchad.jpmchase.net>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
	<50085DDC.5070405@gmail.com> <500865B4.9050105@davea.name>
	<500870A6.4090707@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582AE1@SCACMX008.exchad.jpmchase.net>
Message-ID: <50087CF5.2020005@gmail.com>



On 07/19/2012 10:48 PM, Prasad, Ramit wrote:
>>>> <SNIP>
>>>>
>>>> OK. I am using one time pads to XOR data, but the one time pads (keys)
>>>> are very large numbers, converting them to binary increases their size
>>>> exponentially, which allows me to get more XORing done out of a single
>>> You want to explain this impossibility of increasing size
>>> exponentially?  If you're wanting to waste memory, there are better
>>> ways.  But it's only 8 times as big to save a string of 1's and zeros as
>>> to save the large-int they represent.  And multiplying by 8 isn't an
>>> exponential function.
>>>
>> Yes if you wish to dissect my words the wrong word was chosen...
>>>> key. I am XORing both files and strings so I need to have code that can
>>>> do both even if that means two branches of code via an if/else perhaps
>>>> with an isinstance(data, str).
>>>> I do not need to actually see the binary form.
>>>>
>>> Then don't use the binary form.  It doesn't make the computation any
>>> more "powerful" and it'll certainly slow it down.
>> The title of the question is string to binary and back.
>>> Are you trying to match some other program's algorithm, and thus have
>>> strange constraints on your data?  Or are you simply trying to make a
>>> secure way to encrypt binary files, using one-time pads?
>> I already answered this question...
> Yes, you stated that it had to work on string and files, but are the 
> files binary? DaveA and I are asking the questions because given
> what you are asking it just seems like you are not using the Right
> approach. I can touch my nose by touching my nose with my hand, 
> or asking the person next to me to pick up my hand and use it to
> touch my nose. Both work, one is just faster and easier to
> understand.
I am not sure how to answer that question because all files are binary,
but the files that I will parse have an encoding that allows them to be
read in a non-binary output. But my program will not use the in a
non-binary way, that is why I plan to open them with the 'b' mode to
open them as binary with no encoding assumed by python. I just not have
tested this new technique that you gave me on a binary file yet as I was
still implementing it for strings.
I may not be using the right appraoch that is why I am asking. I also
understand why the questions are needed, so you can understand my
intent, so that you can better help me. But since DaveA and I had a
misunderstanding over the missing indentation, for which I apologized
and explained that my email editor is stripping the spaces, he seems to
be badgering me.

"You want to explain this impossibility of increasing size
exponentially?  If you're wanting to waste memory, there are better
ways."

Now I would like to make it clear I very much so appreciate the help! So
again, Thank you.
>
>>> A one-time pad is the same size as the message, so you simply need to
>>> convert the message into a large-int, and xor them.
> Ramit
> This email is confidential and subject to important disclaimers and
> conditions including on offers for the purchase or sale of
> securities, accuracy and completeness of information, viruses,
> confidentiality, legal privilege, and legal entity disclaimers,
> available at http://www.jpmorgan.com/pages/disclosures/email.  
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From ramit.prasad at jpmorgan.com  Thu Jul 19 23:55:14 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 19 Jul 2012 21:55:14 +0000
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <50087CF5.2020005@gmail.com>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
	<50085DDC.5070405@gmail.com> <500865B4.9050105@davea.name>
	<500870A6.4090707@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582AE1@SCACMX008.exchad.jpmchase.net>
	<50087CF5.2020005@gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416582C17@SCACMX008.exchad.jpmchase.net>

> I am not sure how to answer that question because all files are binary,
> but the files that I will parse have an encoding that allows them to be
> read in a non-binary output. But my program will not use the in a
> non-binary way, that is why I plan to open them with the 'b' mode to
> open them as binary with no encoding assumed by python. I just not have
> tested this new technique that you gave me on a binary file yet as I was
> still implementing it for strings.

As far as I know, even in binary mode, python will convert the 
binary data to read and write strings. So there is no reason 
this technique would not work for binary. Note, I was able to use
the string representation of a PDF file to write another PDF file.
So you do not need to worry about the conversion of binary to strings.
All you need to do is convert the string to int, encrypt, decrypt, 
convert back to string, and write out again.

Note Python3 being Unicode might change things a bit. Not sure if
you will need to convert to bytes or some_string.decode('ascii').

Now if you end up needing to handle non-ASCII data, then this exercise
gets more complicated. Not sure if a simple way to convert all characters
to a numerical point, but it should still be possible. If your data
is binary, then I do not think you will run into any issues.


Ramit


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From steve at pearwood.info  Fri Jul 20 00:20:32 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 20 Jul 2012 08:20:32 +1000
Subject: [Tutor] check against multiple variables
In-Reply-To: <50086013.1090500@ymail.com>
References: <50086013.1090500@ymail.com>
Message-ID: <50088830.6080608@pearwood.info>

Selby Rowley-Cannon wrote:
> I am using a hash table in a small randomization program. I know that 
> some hash functions can be prone to collisions, so I need a way to 
> detect collisions.

I doubt that very much.

This entire question seems like a remarkable case of premature optimization. 
Start with demonstrating that collisions are an actual problem that need fixing.

Unless you have profiled your application and proven that hash collisions is a 
real problem -- and unless you are hashing thousands of float NANs, that is 
almost certainly not the case -- you are just wasting your time and making 
your code slower rather than faster -- a pessimation, not optimization.

And if it *is* a problem, then the solution is to fix your data so that its 
__hash__ method is less likely to collide. If you are rolling your own hash 
method, instead of using one of Python's, that's your first problem.

Python's hash implementation is one of the most finely tuned in the world. 
Many, many years of effort have gone into making it stand up to real-world 
data. You aren't going to beat it with some half-planned pure-Python work-around.



-- 
Steven

From d at davea.name  Fri Jul 20 00:33:10 2012
From: d at davea.name (Dave Angel)
Date: Thu, 19 Jul 2012 18:33:10 -0400
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF47416582C17@SCACMX008.exchad.jpmchase.net>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
	<50085DDC.5070405@gmail.com> <500865B4.9050105@davea.name>
	<500870A6.4090707@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582AE1@SCACMX008.exchad.jpmchase.net>
	<50087CF5.2020005@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582C17@SCACMX008.exchad.jpmchase.net>
Message-ID: <50088B26.6040701@davea.name>

On 07/19/2012 05:55 PM, Prasad, Ramit wrote:
>> I am not sure how to answer that question because all files are binary,
>> but the files that I will parse have an encoding that allows them to be
>> read in a non-binary output. But my program will not use the in a
>> non-binary way, that is why I plan to open them with the 'b' mode to
>> open them as binary with no encoding assumed by python. I just not have
>> tested this new technique that you gave me on a binary file yet as I was
>> still implementing it for strings.
> As far as I know, even in binary mode, python will convert the 
> binary data to read and write strings. So there is no reason 
> this technique would not work for binary. Note, I was able to use
> the string representation of a PDF file to write another PDF file.
> So you do not need to worry about the conversion of binary to strings.
> All you need to do is convert the string to int, encrypt, decrypt, 
> convert back to string, and write out again.
>
> Note Python3 being Unicode might change things a bit. Not sure if
> you will need to convert to bytes or some_string.decode('ascii').

In Python 3, if you open the file  with "b"  (as Jordan has said), it
creates a bytes object.  No use of strings needed or wanted.  And no
assumptions of ascii, except for the output of the % operator on a hex
conversion.


myfile = open(filename, "b")
data = myfile.read(size)

At that point, convert it to hex with:
   hexdata = binascii.hexlify(data)
then convert that to an integer:
   numdata = int(hexdata, 16)

At that point, it's ready to xor with the one-time key, which had better
be the appropriate size to match the data length.

newhexdata = bytes("%x" % numdata, "ascii")
newdata = binascii.unhexlify(newhexdata)

If the file is bigger than the key, you have to get a new key. If the
keys are chosen with a range of 2**200, then you'd read and convert the
file 25 bytes at a time.



-- 

DaveA


From redacted@example.com  Fri Jul 20 00:58:42 2012
From: redacted@example.com (Alexander Q.)
Date: Thu, 19 Jul 2012 15:58:42 -0700
Subject: [Tutor] Calling a function does not return what I want it to return
Message-ID: <CAHgjEe3QBX3rtQq-2zSLtuz6aXKh0xA36_Wyy2QCGsHG3xj04g@mail.gmail.com>

I have this little program that is supposed to calculate how many diagonals
a polygon of x sides has, but it does not return what I have in the
"return" part of the function when I call it. Here is the code:

def num_diag(var):
  ans = 0
  if var <= 3:
    print("No diagonals.")
  else:
    for i in range(num_sides - 3):
      ans = ans + i

  return (((var - 3)*2) + ans)

num_sides = (int(raw_input("Enter sides: ")))
num_diag(num_sides)


Any suggestions as to what is going on? When I run it, it prompts me for
the number of sides, and that's it.
Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120719/4f751682/attachment.html>

From emile at fenx.com  Fri Jul 20 01:10:53 2012
From: emile at fenx.com (Emile van Sebille)
Date: Thu, 19 Jul 2012 16:10:53 -0700
Subject: [Tutor] re 33.116
Message-ID: <jua461$55i$1@dough.gmane.org>

I found ~200k files in /var/log all but 227 look like:

list_boxes.day.1.gz.1.gz.1.gz.3.gz.1.gz.1.gz.2.gz.1.gz.1.gz.2.gz.1.gz
list_boxes.day.1.gz.1.gz.1.gz.3.gz.1.gz.1.gz.2.gz.1.gz.1.gz.2.gz.1.gz.1.gz
list_boxes.day.1.gz.1.gz.1.gz.3.gz.1.gz.1.gz.2.gz.1.gz.1.gz.2.gz.1.gz.1.gz.1.gz
list_boxes.day.1.gz.1.gz.1.gz.3.gz.1.gz.1.gz.2.gz.1.gz.1.gz.2.gz.1.gz.2.gz
list_boxes.day.1.gz.1.gz.1.gz.3.gz.1.gz.1.gz.2.gz.1.gz.1.gz.2.gz.2.gz
list_boxes.day.1.gz.1.gz.1.gz.3.gz.1.gz.1.gz.2.gz.1.gz.1.gz.2.gz.2.gz.1.gz
list_boxes.day.1.gz.1.gz.1.gz.3.gz.1.gz.1.gz.2.gz.1.gz.1.gz.2.gz.3.gz
list_boxes.day.1.gz.1.gz.1.gz.3.gz.1.gz.1.gz.2.gz.1.gz.1.gz.3.gz
list_boxes.day.1.gz.1.gz.1.gz.3.gz.1.gz.1.gz.2.gz.1.gz.1.gz.3.gz.1.gz

in both the day and night variant.  I erased them all as / was at 100% 
used.  It's now at 89% with ~400Mb free.

Emile


From emile at fenx.com  Fri Jul 20 01:17:32 2012
From: emile at fenx.com (Emile van Sebille)
Date: Thu, 19 Jul 2012 16:17:32 -0700
Subject: [Tutor] re 33.116
In-Reply-To: <jua461$55i$1@dough.gmane.org>
References: <jua461$55i$1@dough.gmane.org>
Message-ID: <jua4if$71q$2@dough.gmane.org>

On 7/19/2012 4:10 PM Emile van Sebille said...
> I found ~200k files in /var/log all but 227 look like:

Sorry -- my bad.

Emile




From ramit.prasad at jpmorgan.com  Fri Jul 20 01:17:56 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 19 Jul 2012 23:17:56 +0000
Subject: [Tutor] Calling a function does not return what I want it to
 return
In-Reply-To: <CAHgjEe3QBX3rtQq-2zSLtuz6aXKh0xA36_Wyy2QCGsHG3xj04g@mail.gmail.com>
References: <CAHgjEe3QBX3rtQq-2zSLtuz6aXKh0xA36_Wyy2QCGsHG3xj04g@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416582D68@SCACMX008.exchad.jpmchase.net>

> I have this little program that is supposed to calculate how many diagonals a
> polygon of x sides has, but it does not return what I have in the "return"
> part of the function when I call it. Here is the code:
> 
> def num_diag(var):
>   ans = 0
>   if var <= 3:
>     print("No diagonals.")
>   else:
>     for i in range(num_sides - 3):
>       ans = ans + i
> 
>   return (((var - 3)*2) + ans)
> 
> num_sides = (int(raw_input("Enter sides: ")))
> num_diag(num_sides)
>


>>> num_diag(5)
NameError: global name 'num_sides' is not defined


`for i in range(num_sides - 3):`
Change num_sides to var.

Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From emile at fenx.com  Fri Jul 20 01:15:34 2012
From: emile at fenx.com (Emile van Sebille)
Date: Thu, 19 Jul 2012 16:15:34 -0700
Subject: [Tutor] Calling a function does not return what I want it to
	return
In-Reply-To: <CAHgjEe3QBX3rtQq-2zSLtuz6aXKh0xA36_Wyy2QCGsHG3xj04g@mail.gmail.com>
References: <CAHgjEe3QBX3rtQq-2zSLtuz6aXKh0xA36_Wyy2QCGsHG3xj04g@mail.gmail.com>
Message-ID: <jua4eq$71q$1@dough.gmane.org>

On 7/19/2012 3:58 PM Alexander Q. said...
> I have this little program that is supposed to calculate how many
> diagonals a polygon of x sides has, but it does not return what I have
> in the "return" part of the function when I call it. Here is the code:
>
> def num_diag(var):
>    ans = 0
>    if var <= 3:
>      print("No diagonals.")
>    else:
>      for i in range(num_sides - 3):
>        ans = ans + i
>
>    return (((var - 3)*2) + ans)
>
> num_sides = (int(raw_input("Enter sides: ")))


You're almost there.  Change the following

> num_diag(num_sides)

to

print num_diag(num_sides)
     (for pythons < v3)   or

print (num_diag(num_sides))
     (for python v3 >)

Then see where that takes you.


Emile

>
>
> Any suggestions as to what is going on? When I run it, it prompts me for
> the number of sides, and that's it.
> Thanks.
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>




From d at davea.name  Fri Jul 20 01:21:04 2012
From: d at davea.name (Dave Angel)
Date: Thu, 19 Jul 2012 19:21:04 -0400
Subject: [Tutor] Calling a function does not return what I want it to
 return
In-Reply-To: <CAHgjEe3QBX3rtQq-2zSLtuz6aXKh0xA36_Wyy2QCGsHG3xj04g@mail.gmail.com>
References: <CAHgjEe3QBX3rtQq-2zSLtuz6aXKh0xA36_Wyy2QCGsHG3xj04g@mail.gmail.com>
Message-ID: <50089660.6070501@davea.name>

On 07/19/2012 06:58 PM, Alexander Q. wrote:
> I have this little program that is supposed to calculate how many diagonals
> a polygon of x sides has, but it does not return what I have in the
> "return" part of the function when I call it. Here is the code:
>
> def num_diag(var):
>   ans = 0
>   if var <= 3:
>     print("No diagonals.")
>   else:
>     for i in range(num_sides - 3):
>       ans = ans + i
>
>   return (((var - 3)*2) + ans)
>
> num_sides = (int(raw_input("Enter sides: ")))
> num_diag(num_sides)
>
>
> Any suggestions as to what is going on? When I run it, it prompts me for
> the number of sides, and that's it.
> Thanks.
>
>

You never use the return value.  Try assigning it, and printing it.

result = num_diag(num_sides)
print("final answer=", result)

-- 

DaveA


From mnickey at gmail.com  Fri Jul 20 01:38:53 2012
From: mnickey at gmail.com (Mike Nickey)
Date: Thu, 19 Jul 2012 16:38:53 -0700
Subject: [Tutor] Creating a dictionary on user filter
Message-ID: <CAEywD5DQWsN_vJe2D82PXureGnB4vTNH-5=i5fJHGy5qLpEXqQ@mail.gmail.com>

Hi All,

I have a few lists that I'm trying to put into a dictionary based on
which list the user wants to use as a filter. If the user selects 1
the the dictionary would be created using the first list as the keys
and the secondary items as the values. If the user selects 2, the
dictionary would be created with the second list as the keys, and the
remaining as the values. I think using dict(zip(firstList,
(secondList, thirdList))) is the way to go but I'm having trouble with
the placement of the items.

What I have is this:
firstList = ['a', 'b', 'c']
secondList = [1,2,3]
thirdList = [1.20, 1.23, 2.54]

What I am looking for is something like this for output:
{'a': [1, 1.20], 'b': [2, 1.23], 'c': [3, 2.54]}

What I'm now thinking is that I need to loop over each item in the
list and update the dictionary such as:
for x in range(a):
    compilation = dict(zip(a[x], (b[x], c[x])))

Any help is appreciated.

-- 
~MEN

From alan.gauld at btinternet.com  Fri Jul 20 02:04:23 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 20 Jul 2012 01:04:23 +0100
Subject: [Tutor] Calling a function does not return what I want it to
	return
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF47416582D68@SCACMX008.exchad.jpmchase.net>
References: <CAHgjEe3QBX3rtQq-2zSLtuz6aXKh0xA36_Wyy2QCGsHG3xj04g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582D68@SCACMX008.exchad.jpmchase.net>
Message-ID: <jua7a6$nge$1@dough.gmane.org>

On 20/07/12 00:17, Prasad, Ramit wrote:
>
>> def num_diag(var):
>>    ans = 0
>>    if var <= 3:
>>      print("No diagonals.")
>>    else:
>>      for i in range(num_sides - 3):
>>        ans = ans + i
>>
>>    return (((var - 3)*2) + ans)
>>
>> num_sides = (int(raw_input("Enter sides: ")))
>> num_diag(num_sides)


> NameError: global name 'num_sides' is not defined
>
> `for i in range(num_sides - 3):`
> Change num_sides to var.

It should work without, because it will pick up the
global variable definition.

It's probably not working the way it was intended to,
but it should work... But changing it to use the
argument would definitely be better.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From redacted@example.com  Fri Jul 20 02:15:27 2012
From: redacted@example.com (Alexander Q.)
Date: Thu, 19 Jul 2012 17:15:27 -0700
Subject: [Tutor] Calling a function does not return what I want it to
	return
In-Reply-To: <50089660.6070501@davea.name>
References: <CAHgjEe3QBX3rtQq-2zSLtuz6aXKh0xA36_Wyy2QCGsHG3xj04g@mail.gmail.com>
	<50089660.6070501@davea.name>
Message-ID: <CAHgjEe1EFjA9_XQGUEiiaN+ubp1ZgDLPoOKrFNRC--H=1N4BZw@mail.gmail.com>

On Thu, Jul 19, 2012 at 4:21 PM, Dave Angel <d at davea.name> wrote:

> On 07/19/2012 06:58 PM, Alexander Q. wrote:
> > I have this little program that is supposed to calculate how many
> diagonals
> > a polygon of x sides has, but it does not return what I have in the
> > "return" part of the function when I call it. Here is the code:
> >
> > def num_diag(var):
> >   ans = 0
> >   if var <= 3:
> >     print("No diagonals.")
> >   else:
> >     for i in range(num_sides - 3):
> >       ans = ans + i
> >
> >   return (((var - 3)*2) + ans)
> >
> > num_sides = (int(raw_input("Enter sides: ")))
> > num_diag(num_sides)
> >
> >
> > Any suggestions as to what is going on? When I run it, it prompts me for
> > the number of sides, and that's it.
> > Thanks.
> >
> >
>
> You never use the return value.  Try assigning it, and printing it.
>
> result = num_diag(num_sides)
> print("final answer=", result)
>
> --
>
> DaveA
>
> That did it- thanks Dave!

-Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120719/286894e5/attachment.html>

From abasiemeka at gmail.com  Fri Jul 20 02:36:30 2012
From: abasiemeka at gmail.com (Osemeka Osuagwu)
Date: Fri, 20 Jul 2012 01:36:30 +0100
Subject: [Tutor] Invalid Token Problem
Message-ID: <CAF33E7YJT_QcsCo4-QZK3SGfwFW36=gcsMr07oJwA51K1E9Udw@mail.gmail.com>

Hi folks,
I've been trying to convert numbers from digits to words, I wrote the
following code;


units = ['one', 'two', 'three', 'four', 'five', 'six', 'seven', 'eight',
'nine']
teens = ['eleven', 'twelve', 'thirteen', 'fourteen', 'fifteen', 'sixteen',
'seventeen', 'eighteen', 'nineteen']
tens = ['ten', 'twenty', 'thirty', 'fourty', 'fifty', 'sixty', 'seventy',
'eighty', 'ninety']

def num2word(num):
    wordlist = []
    if len(str(num)) == 4:
        wordlist = [units[1] + 'thousand']

    if len(str(num)) == 3:
        if num%100 == 0:
            wordlist = [units[eval(str(num)[-3])-1] + 'hundred']
        else:
            wordlist = [units[eval(str(num)[-3])-1],'hundred', 'and',
num2word(eval(str(num)[-2:]))]

    if len(str(num)) == 2:
        if num%10 == 0:
            wordlist = [tens[eval(str(num)[-2])-1]]
        elif 10<eval(str(num))<20:
            wordlist = [teens[eval(str(num)[-1])-1]]
        else:
            wordlist = [tens[eval(str(num)[-2])-1],
units[eval(str(num)[-1])-1]]

    if len(str(num)) == 1:
        wordlist = [units[num-1]]
    return ' '.join(wordlist)

for i in range(1, 200):
    print i, num2word(i)


but when I let it run till i = 108, it gives me an invalid token error as
follows;

...
99 ninety nine
100 onehundred
101 one hundred and one
102 one hundred and two
103 one hundred and three
104 one hundred and four
105 one hundred and five
106 one hundred and six
107 one hundred and seven
108

Traceback (most recent call last):
  File "C:/Windows.old/Users/Abasiemeka/Abasiemeka/GOOGLE
University/Python/Python Code/MyCode/Project Euler code/Project Euler
answer 17.py", line 33, in <module>
    print i, num2word(i)
  File "C:/Windows.old/Users/Abasiemeka/Abasiemeka/GOOGLE
University/Python/Python Code/MyCode/Project Euler code/Project Euler
answer 17.py", line 18, in num2word
    wordlist = [units[eval(str(num)[-3])-1],'hundred', 'and',
num2word(eval(str(num)[-2:]))]
  File "<string>", line 1
    08
     ^
SyntaxError: invalid token
>>>


I am at a loss, please help.

gratefully,
Abasiemeka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120720/e614b665/attachment.html>

From d at davea.name  Fri Jul 20 02:45:28 2012
From: d at davea.name (Dave Angel)
Date: Thu, 19 Jul 2012 20:45:28 -0400
Subject: [Tutor] Invalid Token Problem
In-Reply-To: <CAF33E7YJT_QcsCo4-QZK3SGfwFW36=gcsMr07oJwA51K1E9Udw@mail.gmail.com>
References: <CAF33E7YJT_QcsCo4-QZK3SGfwFW36=gcsMr07oJwA51K1E9Udw@mail.gmail.com>
Message-ID: <5008AA28.6020807@davea.name>

On 07/19/2012 08:36 PM, Osemeka Osuagwu wrote:
> <snip>...
> 99 ninety nine
> 100 onehundred
> 101 one hundred and one
> 102 one hundred and two
> 103 one hundred and three
> 104 one hundred and four
> 105 one hundred and five
> 106 one hundred and six
> 107 one hundred and seven
> 108
>
> Traceback (most recent call last):
>   File "C:/Windows.old/Users/Abasiemeka/Abasiemeka/GOOGLE
> University/Python/Python Code/MyCode/Project Euler code/Project Euler
> answer 17.py", line 33, in <module>
>     print i, num2word(i)
>   File "C:/Windows.old/Users/Abasiemeka/Abasiemeka/GOOGLE
> University/Python/Python Code/MyCode/Project Euler code/Project Euler
> answer 17.py", line 18, in num2word
>     wordlist = [units[eval(str(num)[-3])-1],'hundred', 'and',
> num2word(eval(str(num)[-2:]))]
>   File "<string>", line 1
>     08
>      ^
> SyntaxError: invalid token
>

08 isn't a valid literal.  Remove the leading zero.  That says that the
following digits are to be interpreted as octal, and 8 isn't a valid
octal digit.

Much better would be to eliminate the unnecessary use of eval().  It's
dangerous, and sometimes doesn't do what you expect.




-- 

DaveA


From eryksun at gmail.com  Fri Jul 20 02:46:56 2012
From: eryksun at gmail.com (eryksun)
Date: Thu, 19 Jul 2012 20:46:56 -0400
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <50087CF5.2020005@gmail.com>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
	<50085DDC.5070405@gmail.com> <500865B4.9050105@davea.name>
	<500870A6.4090707@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582AE1@SCACMX008.exchad.jpmchase.net>
	<50087CF5.2020005@gmail.com>
Message-ID: <CACL+1as-UkWf6GeGipoi7KN+4JwkbcEXLJ-hG=cW=Av1dcb-CA@mail.gmail.com>

On Thu, Jul 19, 2012 at 5:32 PM, Jordan <wolfrage8765 at gmail.com> wrote:
>
> I am not sure how to answer that question because all files are binary,
> but the files that I will parse have an encoding that allows them to be
> read in a non-binary output. But my program will not use the in a
> non-binary way, that is why I plan to open them with the 'b' mode to
> open them as binary with no encoding assumed by python. I just not have
> tested this new technique that you gave me on a binary file yet as I was
> still implementing it for strings.

Reading from a file in binary mode returns a bytes object in Python 3.
Since iterating over bytes returns ints, you can cycle the key over
the plain text using zip and compute the XOR without having to convert
the entire message into a single big number in memory. Here's my
example from before, adapted for files:

>>> from itertools import cycle
>>> key = b'1234'
>>> kit = cycle(key)
>>> with open('temp.txt', 'rb') as f, open('cipher.txt', 'wb') as fo:
...     fit = iter(lambda: f.read(512), b'')
...     for text in fit:
...         fo.write(bytes(x^y for x,y in zip(text, kit)))

Since the input file could be arbitrarily large and lack newlines, I'm
using "iter" to create a special iterator that reads 512-byte chunks.
The iterator stops when "read" returns an empty bytes object (i.e.
b''). You could use a while loop instead.

I assume here that the key is possibly shorter than the message (e.g.
encrypting 1 megabyte of text with a 128 byte key). If you're making a
one-time pad I think the key is the same length as the message. In
that case you wouldn't have to worry about cycling it. Anyway, I'm not
particularly interested in cryptography. I'm just trying to help with
the operations.

From rzzzwilson at gmail.com  Fri Jul 20 03:01:55 2012
From: rzzzwilson at gmail.com (Ross Wilson)
Date: Fri, 20 Jul 2012 11:01:55 +1000
Subject: [Tutor] Invalid Token Problem
In-Reply-To: <5008AA28.6020807@davea.name>
References: <CAF33E7YJT_QcsCo4-QZK3SGfwFW36=gcsMr07oJwA51K1E9Udw@mail.gmail.com>
	<5008AA28.6020807@davea.name>
Message-ID: <5008AE03.20701@gmail.com>

On 20/07/12 10:45, Dave Angel wrote:
> On 07/19/2012 08:36 PM, Osemeka Osuagwu wrote:
>> <snip>...
>> 99 ninety nine
>> 100 onehundred
>> 101 one hundred and one
>> 102 one hundred and two
>> 103 one hundred and three
>> 104 one hundred and four
>> 105 one hundred and five
>> 106 one hundred and six
>> 107 one hundred and seven
>> 108
>>
>> Traceback (most recent call last):
>>    File "C:/Windows.old/Users/Abasiemeka/Abasiemeka/GOOGLE
>> University/Python/Python Code/MyCode/Project Euler code/Project Euler
>> answer 17.py", line 33, in <module>
>>      print i, num2word(i)
>>    File "C:/Windows.old/Users/Abasiemeka/Abasiemeka/GOOGLE
>> University/Python/Python Code/MyCode/Project Euler code/Project Euler
>> answer 17.py", line 18, in num2word
>>      wordlist = [units[eval(str(num)[-3])-1],'hundred', 'and',
>> num2word(eval(str(num)[-2:]))]
>>    File "<string>", line 1
>>      08
>>       ^
>> SyntaxError: invalid token
>>
>
> 08 isn't a valid literal.  Remove the leading zero.  That says that the
> following digits are to be interpreted as octal, and 8 isn't a valid
> octal digit.

Try to think of another way to convert an integer string into an integer 
value. hINT()

> Much better would be to eliminate the unnecessary use of eval().  It's
> dangerous, and sometimes doesn't do what you expect.

More specifically, eval() is dangerous if you try to evaluate a string 
supplied by someone else.  You really can't predict what will happen.

However, if you use eval() on strings that you create yourself, it can 
be a handy technique.  When you are starting out, it's best to ignore 
eval() until later.

Ross

From lilytran at adobe.com  Thu Jul 19 23:09:39 2012
From: lilytran at adobe.com (Lily Tran)
Date: Thu, 19 Jul 2012 14:09:39 -0700
Subject: [Tutor] Join email list
Message-ID: <CC2DC5A3.3165F%lilytran@adobe.com>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120719/ff914802/attachment.html>

From wolfrage8765 at gmail.com  Fri Jul 20 07:48:49 2012
From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com)
Date: Fri, 20 Jul 2012 07:48:49 +0200
Subject: [Tutor]  string to binary and back... Python 3
In-Reply-To: <CAOhNYv=bHZ6uStt62TOP7c3dnd1b+hc+BY5aSxB_VXKRAf5vgQ@mail.gmail.com>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
	<50085DDC.5070405@gmail.com> <500865B4.9050105@davea.name>
	<500870A6.4090707@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582AE1@SCACMX008.exchad.jpmchase.net>
	<50087CF5.2020005@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582C17@SCACMX008.exchad.jpmchase.net>
	<50088B26.6040701@davea.name>
	<CAOhNYv=bHZ6uStt62TOP7c3dnd1b+hc+BY5aSxB_VXKRAf5vgQ@mail.gmail.com>
Message-ID: <CAOhNYvkUtqCEG2txaD8cMUEL43C7o6RYAx6NguUW7s6yk6cdbA@mail.gmail.com>

  On Fri, Jul 20, 2012 at 12:33 AM, Dave Angel <d at davea.name> wrote:

> On 07/19/2012 05:55 PM, Prasad, Ramit wrote:
> >> I am not sure how to answer that question because all files are binary,
> >> but the files that I will parse have an encoding that allows them to be
> >> read in a non-binary output. But my program will not use the in a
> >> non-binary way, that is why I plan to open them with the 'b' mode to
> >> open them as binary with no encoding assumed by python. I just not have
> >> tested this new technique that you gave me on a binary file yet as I was
> >> still implementing it for strings.
> > As far as I know, even in binary mode, python will convert the
> > binary data to read and write strings. So there is no reason
> > this technique would not work for binary. Note, I was able to use
> > the string representation of a PDF file to write another PDF file.
> > So you do not need to worry about the conversion of binary to strings.
> > All you need to do is convert the string to int, encrypt, decrypt,
> > convert back to string, and write out again.
> >
> > Note Python3 being Unicode might change things a bit. Not sure if
> > you will need to convert to bytes or some_string.decode('ascii').
>
> In Python 3, if you open the file  with "b"  (as Jordan has said), it
> creates a bytes object.  No use of strings needed or wanted.  And no
> assumptions of ascii, except for the output of the % operator on a hex
> conversion.
>
>
> myfile = open(filename, "b")
> data = myfile.read(size)
>
> At that point, convert it to hex with:
>    hexdata = binascii.hexlify(data)
> then convert that to an integer:
>    numdata = int(hexdata, 16)
>
> At that point, it's ready to xor with the one-time key, which had better
> be the appropriate size to match the data length.
>
> newhexdata = bytes("%x" % numdata, "ascii")
> newdata = binascii.unhexlify(newhexdata)
>
> If the file is bigger than the key, you have to get a new key. If the
> keys are chosen with a range of 2**200, then you'd read and convert the
> file 25 bytes at a time.
>
Thanks I will give this a try. Can you explian a little further for me what
exactly this:

newhexdata = bytes("%x" % numdata, "ascii")
line is doing? I don't quite understand the use of the "%x" % on numdata.

>
>
> --
>
> DaveA
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120720/ce21a2f7/attachment.html>

From d at davea.name  Fri Jul 20 08:00:18 2012
From: d at davea.name (Dave Angel)
Date: Fri, 20 Jul 2012 02:00:18 -0400
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <CAOhNYv=bHZ6uStt62TOP7c3dnd1b+hc+BY5aSxB_VXKRAf5vgQ@mail.gmail.com>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
	<50085DDC.5070405@gmail.com> <500865B4.9050105@davea.name>
	<500870A6.4090707@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582AE1@SCACMX008.exchad.jpmchase.net>
	<50087CF5.2020005@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582C17@SCACMX008.exchad.jpmchase.net>
	<50088B26.6040701@davea.name>
	<CAOhNYv=bHZ6uStt62TOP7c3dnd1b+hc+BY5aSxB_VXKRAf5vgQ@mail.gmail.com>
Message-ID: <5008F3F2.7090305@davea.name>

You forgot to include the list.  So I'm forwarding it now, with new remarks.

On 07/20/2012 01:47 AM, wolfrage8765 at gmail.com wrote:
> On Fri, Jul 20, 2012 at 12:33 AM, Dave Angel <d at davea.name> wrote:
> 
>> On 07/19/2012 05:55 PM, Prasad, Ramit wrote:
>>>> I am not sure how to answer that question because all files are binary,
>>>> but the files that I will parse have an encoding that allows them to be
>>>> read in a non-binary output. But my program will not use the in a
>>>> non-binary way, that is why I plan to open them with the 'b' mode to
>>>> open them as binary with no encoding assumed by python. I just not have
>>>> tested this new technique that you gave me on a binary file yet as I was
>>>> still implementing it for strings.
>>> As far as I know, even in binary mode, python will convert the
>>> binary data to read and write strings. So there is no reason
>>> this technique would not work for binary. Note, I was able to use
>>> the string representation of a PDF file to write another PDF file.
>>> So you do not need to worry about the conversion of binary to strings.
>>> All you need to do is convert the string to int, encrypt, decrypt,
>>> convert back to string, and write out again.
>>>
>>> Note Python3 being Unicode might change things a bit. Not sure if
>>> you will need to convert to bytes or some_string.decode('ascii').
>>
>> In Python 3, if you open the file  with "b"  (as Jordan has said), it
>> creates a bytes object.  No use of strings needed or wanted.  And no
>> assumptions of ascii, except for the output of the % operator on a hex
>> conversion.
>>
>>
>> myfile = open(filename, "b")
>> data = myfile.read(size)
>>
>> At that point, convert it to hex with:
>>    hexdata = binascii.hexlify(data)
>> then convert that to an integer:
>>    numdata = int(hexdata, 16)
>>
>> At that point, it's ready to xor with the one-time key, which had better
>> be the appropriate size to match the data length.
>>
>> newhexdata = bytes("%x" % numdata, "ascii")
>> newdata = binascii.unhexlify(newhexdata)
>>
>> If the file is bigger than the key, you have to get a new key. If the
>> keys are chosen with a range of 2**200, then you'd read and convert the
>> file 25 bytes at a time.
>>
> Thanks I will give this a try. Can you explian a little further for me what
> exactly this:
> newhexdata = bytes("%x" % numdata, "ascii")
> line is doing? I don't quite understand the use of the "%x" % on numdata.
> 

When you see an expression you don't understand, then try to decompose it.

"%x" % numdata is just one of many ways of converting an integer into a
hex string.  The %x is a format specifier, and says to use hex.
Actually, with Python 3, the format method is supposed to be preferred,
but I've been doing it the old way to long to learn format quickly.

bytes() is how we encode a string into bytes.  Since I know all the
characters created by my %x expression will be valid ASCII, I can safely
tell it to encode it using ASCII.

This isn't the only way to solve the original problem by a long shot,
but you did say the one-time pad was a numeric int.  If it had been a
byte string, the problem would have been much simpler, though probably
not as quick.

-- 

DaveA



From alan.gauld at btinternet.com  Fri Jul 20 09:16:21 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 20 Jul 2012 08:16:21 +0100
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <CAOhNYvkUtqCEG2txaD8cMUEL43C7o6RYAx6NguUW7s6yk6cdbA@mail.gmail.com>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
	<50085DDC.5070405@gmail.com> <500865B4.9050105@davea.name>
	<500870A6.4090707@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582AE1@SCACMX008.exchad.jpmchase.net>
	<50087CF5.2020005@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582C17@SCACMX008.exchad.jpmchase.net>
	<50088B26.6040701@davea.name>
	<CAOhNYv=bHZ6uStt62TOP7c3dnd1b+hc+BY5aSxB_VXKRAf5vgQ@mail.gmail.com>
	<CAOhNYvkUtqCEG2txaD8cMUEL43C7o6RYAx6NguUW7s6yk6cdbA@mail.gmail.com>
Message-ID: <jub0k5$m1f$1@dough.gmane.org>

On 20/07/12 06:48, wolfrage8765 at gmail.com wrote:

> Thanks I will give this a try. Can you explian a little further for me
> what exactly this:
>
> newhexdata = bytes("%x" % numdata, "ascii")
> line is doing? I don't quite understand the use of the "%x" % on numdata.

It is standard string formatting in pre Python 3 style.
Thus "%x" says I want a string containing a hex  number
and % numdata says use the value of hexdata to populate
the string.

Using the format method it would look like:

newhexdata = bytes("{0:x}".format(numdata), "ascii")

HTH
-
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From __peter__ at web.de  Fri Jul 20 09:19:52 2012
From: __peter__ at web.de (Peter Otten)
Date: Fri, 20 Jul 2012 09:19:52 +0200
Subject: [Tutor] Creating a dictionary on user filter
References: <CAEywD5DQWsN_vJe2D82PXureGnB4vTNH-5=i5fJHGy5qLpEXqQ@mail.gmail.com>
Message-ID: <jub0q7$nch$1@dough.gmane.org>

Mike Nickey wrote:

> What I have is this:
> firstList = ['a', 'b', 'c']
> secondList = [1,2,3]
> thirdList = [1.20, 1.23, 2.54]
> 
> What I am looking for is something like this for output:
> {'a': [1, 1.20], 'b': [2, 1.23], 'c': [3, 2.54]}

To get this combine second and third into the list of values and then build 
the final dict using it:

>>> first = ['a', 'b', 'c']
>>> second = [1, 2, 3]
>>> third = [1.20, 1.23, 2.54]
>>> values = zip(second, third)
>>> values
[(1, 1.2), (2, 1.23), (3, 2.54)]
>>> dict(zip(first, values))
{'a': (1, 1.2), 'c': (3, 2.54), 'b': (2, 1.23)}

If tuples as values are not acceptable: 

values = map(list, values)



From alan.gauld at btinternet.com  Fri Jul 20 09:25:54 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 20 Jul 2012 08:25:54 +0100
Subject: [Tutor] Invalid Token Problem
In-Reply-To: <5008AE03.20701@gmail.com>
References: <CAF33E7YJT_QcsCo4-QZK3SGfwFW36=gcsMr07oJwA51K1E9Udw@mail.gmail.com>
	<5008AA28.6020807@davea.name> <5008AE03.20701@gmail.com>
Message-ID: <jub162$q4k$1@dough.gmane.org>

On 20/07/12 02:01, Ross Wilson wrote:

> More specifically, eval() is dangerous if you try to evaluate a string
> supplied by someone else.  You really can't predict what will happen.

It really doesn't matter who provides the string, Python and eval() 
don't care. They will behave just as dangerously if you provide the 
wrong string.

And that's the problem because even if you think the string you are 
feeding eval() is safe it only needs a small typo to occasionally
turn it into something not safe - and just once is enough to be painful.

So while eval() introduces security issues where other people try to 
maliciously damage your code, eval() is dangerous even in "normal" use
because it has the potential to do damage.

Think of eval() as being like an old fashioned scythe for cutting hay. A 
scythe is dangerous, used wrongly it can cut off your foot. Used 
correctly it won't. But no matter how long you have been using a
scythe it remains dangerous and you forget that at your peril!

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Fri Jul 20 09:27:40 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 20 Jul 2012 08:27:40 +0100
Subject: [Tutor] Join email list
In-Reply-To: <CC2DC5A3.3165F%lilytran@adobe.com>
References: <CC2DC5A3.3165F%lilytran@adobe.com>
Message-ID: <jub19b$q4k$2@dough.gmane.org>

On 19/07/12 22:09, Lily Tran wrote:

> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

The server does not respond to subject-line commands, you have to visit 
the web site

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From steve at pearwood.info  Fri Jul 20 10:43:35 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 20 Jul 2012 18:43:35 +1000
Subject: [Tutor] suggestion for an editor
In-Reply-To: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>
References: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>
Message-ID: <50091A37.2070402@pearwood.info>

Bala subramanian wrote:
> Friends,
> At present i write programs using vi editor. I am interested to change to
> something else. My specific need is that i want to select a portion/small
> segment of my program (for eg. a nested loop) and then monitor processing
> time it takes for that portion while i run the program. By this i hope to
> find the segment that takes time and modify to achieve better speed. Can
> someone please share their experience.

I don't think that what you want exists. As far as I know, even full-featured 
IDEs (Integrated Development Environments) don't include profiling of selected 
sections of code.

The only IDEs I am familiar with were from the 1990s, Lightspeed Pascal (later 
Think Pascal), which included integrated editor, compiler, linker and 
debugger. As far as I know while integrated debuggers are still standard, and 
possibly even whole-program profilers, the ability to select a small portion 
of code and profile just that and nothing else is pure fantasy.

But I could be wrong.

If you learn of such an IDE, please come back and share that knowledge with us.




-- 
Steven

From steve at pearwood.info  Fri Jul 20 11:00:31 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 20 Jul 2012 19:00:31 +1000
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
Message-ID: <50091E2F.9050404@pearwood.info>

PyProg PyProg wrote:
> Hi all,
> 
> I would get a new list as:
> 
> [(0, '3eA', 'Dupont', 'Juliette', '11.0/10.0', '4.0/5.0', '17.5/30.0',
> '3.0/5.0', '4.5/10.0', '35.5/60.0'), (1, '3eA', 'Pop', 'Iggy',
> '12.0/10.0', '3.5/5.0', '11.5/30.0', '4.0/5.0', '5.5/10.0',
> '7.5/10.0', '40.5/60.0')]
> 
> ... from this one:
> 
> [(0, '3eA', 'Dupont', 'Juliette', 0, 11.0, 10.0), (0, '3eA', 'Dupont',
> 'Juliette', 1, 4.0, 5.0), (0, '3eA', 'Dupont', 'Juliette', 2, 17.5,
> 30.0), (0, '3eA', 'Dupont', 'Juliette', 3, 3.0, 5.0), (0, '3eA',
> 'Dupont', 'Juliette', 4, 4.5, 10.0), (0, '3eA', 'Dupont', 'Juliette',
> 5, 35.5, 60.0), (1, '3eA', 'Pop', 'Iggy', 0, 12.0, 10.0), (1, '3eA',
> 'Pop', 'Iggy', 1, 3.5, 5.0), (1, '3eA', 'Pop', 'Iggy', 2, 11.5, 30.0),
> (1, '3eA', 'Pop', 'Iggy', 3, 4.0, 5.0), (1, '3eA', 'Pop', 'Iggy', 4,
> 5.5, 10.0), (1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0)]
> 
> How to make that ? I'm looking for but for now I can't do it.

Good grief! Please take the time to come up with a SIMPLE example! What parts 
of the information there are important and what parts can we ignore?

I can't see anything being flattened there. All I see is duplicates being 
removed. Can you explain what you actually expect?

The best way to remove duplicates will depend on whether or not the items are 
hashable, and whether you care about the order they appear in. Here is a 
simple example of removing duplicates while still keeping the original order:


def remove_dups(sequence):
     new = []
     for item in sequence:
         if item not in new:
             new.append(item)
     return new


And in use:


py> old = [1, 2, 3, 2, 1, 4, 1, 1, 1, 5, 1, 5, 2]
py> remove_dups(old)
[1, 2, 3, 4, 5]

If you don't care about order, a set will be MUCH faster:

py> set(old)
{1, 2, 3, 4, 5}




-- 
Steven

From wprins at gmail.com  Fri Jul 20 11:01:16 2012
From: wprins at gmail.com (Walter Prins)
Date: Fri, 20 Jul 2012 10:01:16 +0100
Subject: [Tutor] check against multiple variables
In-Reply-To: <50088830.6080608@pearwood.info>
References: <50086013.1090500@ymail.com>
	<50088830.6080608@pearwood.info>
Message-ID: <CANLXbfAM-VGdudKaZA7+svY=_BoaiTLWSKB3hoePDWXV+Cm0Cg@mail.gmail.com>

On 19 July 2012 23:20, Steven D'Aprano <steve at pearwood.info> wrote:
> Selby Rowley-Cannon wrote:
>>
>> I am using a hash table in a small randomization program. I know that some
>> hash functions can be prone to collisions, so I need a way to detect
>> collisions.

> This entire question seems like a remarkable case of premature optimization.
> Start with demonstrating that collisions are an actual problem that need
> fixing.

Unless you know all the keys in advance and have consructed a proven
perfect hash, potential collisions are something that *any* hash table
implementation fundamentally *has* to deal with, no matter how small
the probability of a collision.  (If there's a non-zero probability of
collision you *have* to deal with collisions.)

> Unless you have profiled your application and proven that hash collisions is
> a real problem -- and unless you are hashing thousands of float NANs, that
> is almost certainly not the case -- you are just wasting your time and
> making your code slower rather than faster -- a pessimation, not
> optimization.
>
> And if it *is* a problem, then the solution is to fix your data so that its
> __hash__ method is less likely to collide. If you are rolling your own hash
> method, instead of using one of Python's, that's your first problem.

Unfortunately you usually can't prescribe the data to be stored in a
hash table to accomodate a given hashing function.  ;)  You can
however of course improve your hashing function, or (within reason)
increase the hash table size.

But yes, rolling your own is not a good idea given highly optimized
solutions already exist inside the Python language (dict)  Except of
course if you're studying hash tables as part of some course...

Selby, can you clarify please, are you doing this as part of an
assignment or some coursework?  I can understand your question in the
context of learning/computer studies, but otherwise you should just
use Python's dict.

In any case, to somewhat answer you question, dealing with collisions
in hash tables typically involves some variant of one of the following
2 approaches: 1) Each slot in the hash table is a "bucket", usually
implemented as a list (but might be something else), so you do a
search through the hash bucket to see if the key you'r looking for is
in fact in the "bucket" or 2) You have each slot in the hash table be
a single key/value pair and then once you've hashed you look to see if
the key at the entry identified is the key you're looking for.  If not
(collision), you hash ("probe") again by some factor (modulo the size
of you hash table) and look at the next slot identified etc until you
either find the key you're looking for, or an empty slot.  There many
variations one could come up with.  Have a read of this page on
wikipedia: http://en.wikipedia.org/wiki/Hash_table

Walter

From steve at pearwood.info  Fri Jul 20 11:42:06 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 20 Jul 2012 19:42:06 +1000
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <CABdrqAFJzCd-fKd5pZRiqy0OGVNdqMycv-drKf2Sw3Fq8j8PkQ@mail.gmail.com>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>	<20120719060012.GF2214@ando>
	<CABdrqAFJzCd-fKd5pZRiqy0OGVNdqMycv-drKf2Sw3Fq8j8PkQ@mail.gmail.com>
Message-ID: <500927EE.4040104@pearwood.info>

Ryan Waples wrote:
>> I count only 19 lines.
> 
> yep, you are right.  My bad, I think I missing copy/pasting line 20.
> 
>> The first group has only three lines. See below.
> 
> Not so, the first group is actually the first four lines listed below.
>  Lines 1-4 serve as one group.  For what it is worth, line four should
> have 1 character for each char in line 1, and the first line is much
> shorter, contains a space, and for this file always ends in either
> "1:N:0:" (keep) "1"Y"0:" (remove).   The EXAMPLE data is correctly
> formatted as it should be, but I'm missing line 20.


Ah, I had somehow decided that the + was a group delimiter. Which would make 
more sense than having an (apparently) arbitrary plus sign in line 3.

The more information you can supply about the format, the better. Perhaps 
someone will even come up with a standard parser for these files, so that 
every biomed researcher doesn't have to re-invent the wheel every time they 
open one of these files.


[...]
> I think you are just reading one frame shifted, its not a well
> designed format because the required start character "@", can appear
> other places as well....

Yes, likely I am reading it shifted, and no, it is not a well-designed format.


> I'm pretty sure that my raw IN files are all good, its hard to be sure
> with such a large file, but the very picky downstream analysis program
> takes every single raw file just fine (30 of them), and gaks on my
> filtered files, at regions that don't conform to the correct
> formatting.

All I can suggest is that you add more error-checking to your code. For each 
line, or at least for each group of lines, check that the format is as you 
expect, both before and after you write the data.

You might also like to do a disk-check of the disk in question, just in case 
it is faulty and corrupting the data (very unlikely, but it wouldn't hurt to 
check).


>  > for reads, lines in four_lines( INFILE ):
>                 ID_Line_1, Seq_Line, ID_Line_2, Quality_Line = lines

Argggh! I screwed that up. Sorry, I missed an extra call. It should be:


     for reads, lines in enumerate(four_lines( INFILE )):
                 ID_Line_1, Seq_Line, ID_Line_2, Quality_Line = lines


Sorry about that.


> Can you explain what is going on here, or point me In the right
> direction?  I see that the parts of 'lines' get assigned, but I'm
> missing how the file gets iterated over and how reads gets
> incremented.

There are four things you need to know:


(1) File objects in Python are "iterators" -- the idea is that certain objects 
obey a protocol that says, each time you call the next() function on them, 
they hand over one piece of data, and then advance to the next value. File 
objects are one such iterator: each time you call next(file_object), you get 
one line of text, until EOF.

For-loops naturally work with iterators: under the hood, they repeatedly call 
next() on the object, until the iterator signals it is done.

So instead of the old way:

f = open("myfile", "r")
# grab all the lines at once, if you can
lines = f.readlines()  # hope you don't run out of memory...
for line in lines:
     do_something_with(line)


the new way uses less memory and is safer:

f = open("myfile", "r")
for line in f:  # file objects iterate over lines
     do_something_with(line)


(2) But what I do is I create my own iterator, called "four_lines", using what 
Python calls a generator. A generator is a special type of function which 
behaves as an iterator: instead of returning a single value and stopping, a 
generator can return multiple values, one at a time, each time you call next() 
on it. The presence of "yield" instead of "return" makes a generator.

So four_lines() is a generator which takes a file object as argument. It 
repeatedly does the following steps:

   a) grab one line from the file object; if that works, great,
      otherwise signal EOF and we're done;

   b) grab three more lines from the file object, but this time
      don't signal EOF, just pad the lines with empty strings;

   c) package those four lines into a tuple of four values and
      yield them (like a return, only the function doesn't exit yet);

   d) surrender control back to the calling code, in this case
      the for-loop;

   e) wait for the next loop, and go back to step 1.

It does this repeatedly until the file object signals EOF, then this also 
signals EOF.


(3) enumerate() is yet another iterator. (You may notice that Python is really 
into iterators and processing data when needed, instead of up-front in a 
list.) What enumerate does is take another iterator as argument, in this case 
the output of four_lines, and simple package that up with a count of how many 
times you've called it. So instead of keeping your own loop variable, 
enumerate does it for you.

An example:

py> for i, x in enumerate(['fe', 'fi', 'fo', 'fum']):
...     print(i, x)
...
0 fe
1 fi
2 fo
3 fum


Notice that the count starts at 0 instead of 1. If you don't like that, just 
add one to the result.

The end result is that I chain a sequence of iterator calls:

- the INPUT file object iterator provides lines of text, one per call;

- the four_lines iterator accumulates the lines of text from INPUT, strips 
them of whitespace, and provides them in groups of four per call;

- the enumerate iterator grabs each group from four_lines, and provides a loop 
counter and the group it just grabbed;

- finally the for-loop assigns the loop counter to i and the group of lines to 
lines.


(4) Last but not least: the second of the loop variables is called "lines", 
because it holds the group of four lines provided by four_lines. So the first 
thing the for-loop block does is grab those four lines and assign them all to 
names:

ID_Line_1, Seq_Line, ID_Line_2, Quality_Line = lines


Here's a simple example of how this works:

py> group = ('alpha', 'beta', 'gamma')
py> a, b, c = group
py> a
'alpha'
py> b
'beta'
py> c
'gamma'


This is sometimes called "tuple unpacking", or "sequence unpacking".



> Do you have a reason why this approach might give a 'better' output?

If there was a bug in your code, my code would be unlikely to contain the same 
bug and so should give a better -- or at least different -- result.




-- 
Steven


From steve at pearwood.info  Fri Jul 20 11:43:20 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 20 Jul 2012 19:43:20 +1000
Subject: [Tutor] Problem When Iterating Over Large Test Files
In-Reply-To: <ju8edb$276$1@dough.gmane.org>
References: <CABdrqAEhB4B-9UK2992o4OZCszx4wmrcaEAcF6SZwBJD7uJYrw@mail.gmail.com>	<20120719060012.GF2214@ando>
	<ju8edb$276$1@dough.gmane.org>
Message-ID: <50092838.1090001@pearwood.info>

Alan Gauld wrote:
> On 19/07/12 07:00, Steven D'Aprano wrote:

>>          for reads, lines in four_lines( INFILE ):
>>                  ID_Line_1, Seq_Line, ID_Line_2, Quality_Line = lines
> 
> Shouldn't that be
> 
>           for reads, lines in enumerate( four_lines(INFILE) ):
>                   ID_Line_1, Seq_Line, ID_Line_2, Quality_Line = lines


It certainly should!

Mea culpa, sorry for any confusion. This is what happens when I don't test 
code before posting.




-- 
Steven


From steve at pearwood.info  Fri Jul 20 12:06:16 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 20 Jul 2012 20:06:16 +1000
Subject: [Tutor] check against multiple variables
In-Reply-To: <CANLXbfAM-VGdudKaZA7+svY=_BoaiTLWSKB3hoePDWXV+Cm0Cg@mail.gmail.com>
References: <50086013.1090500@ymail.com>	<50088830.6080608@pearwood.info>
	<CANLXbfAM-VGdudKaZA7+svY=_BoaiTLWSKB3hoePDWXV+Cm0Cg@mail.gmail.com>
Message-ID: <50092D98.9000304@pearwood.info>

Walter Prins wrote:
> On 19 July 2012 23:20, Steven D'Aprano <steve at pearwood.info> wrote:
>> Selby Rowley-Cannon wrote:
>>> I am using a hash table in a small randomization program. I know that some
>>> hash functions can be prone to collisions, so I need a way to detect
>>> collisions.
> 
>> This entire question seems like a remarkable case of premature optimization.
>> Start with demonstrating that collisions are an actual problem that need
>> fixing.
> 
> Unless you know all the keys in advance and have consructed a proven
> perfect hash, potential collisions are something that *any* hash table
> implementation fundamentally *has* to deal with, no matter how small
> the probability of a collision.  (If there's a non-zero probability of
> collision you *have* to deal with collisions.)

Absolutely. And did you think that Python, a 20 year old language which uses 
dicts (hash tables) as a fundamental data structure for its own internals, 
somehow neglected to do so? (A rhetorical question, I know you didn't :)

Naturally dictionaries (hash tables) deal with collisions. They couldn't work 
if they didn't.


[...]
> But yes, rolling your own is not a good idea given highly optimized
> solutions already exist inside the Python language (dict)  Except of
> course if you're studying hash tables as part of some course...

Ah, of course that would be an excellent reason for writing your own hash 
table implementation! I didn't think of that at the time.



-- 
Steven


From ljmamoreira at gmail.com  Fri Jul 20 12:07:31 2012
From: ljmamoreira at gmail.com (Jose Amoreira)
Date: Fri, 20 Jul 2012 11:07:31 +0100
Subject: [Tutor] Where to put small auxiliary function
Message-ID: <CAN5tF-HHwOb+2-s6-UhXAM=BYDofs8OcJoOodeg=pxm41HQ9gA@mail.gmail.com>

Hi.
This is a question about style. I have a class definition that calls a
small auxiliary function. Because this function isn't used anywhere
else, I'd like to include it inside the class definition. However, if
I do that, I'll have to use "self" in the call argument, which is (I
think) rather awkward.
Let me give an example:

def is_odd(k):
    if k % 2 == 0:
        return False
    else:
        return True

class MyOddNbr(object):
    def __init__(self, k):
        if is_odd(k):
            self.k = k
        else:
            self.k = k + 1

This works fine, but I'd like to have is_odd defined inside the class
definition, because that's the only context where that function is
used. That would be something like

class MyOddNbr(object):
    def is_odd(self,k):
        if k % 2 == 0:
            return False
        else:
            return True
    def __init__(self,k):
        if self.is_odd(k):
            self.k = k
        else:
            self.k = k + 1

This also works fine, but the function is_odd() is so simple and
generic that I find it strange to define it with is_odd(self,k) or to
call it with is_odd(self,k).
What is the pythonic way of doing this kind of stuff?
Thanks.
Ze

From breamoreboy at yahoo.co.uk  Fri Jul 20 12:32:17 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 20 Jul 2012 11:32:17 +0100
Subject: [Tutor] Where to put small auxiliary function
In-Reply-To: <CAN5tF-HHwOb+2-s6-UhXAM=BYDofs8OcJoOodeg=pxm41HQ9gA@mail.gmail.com>
References: <CAN5tF-HHwOb+2-s6-UhXAM=BYDofs8OcJoOodeg=pxm41HQ9gA@mail.gmail.com>
Message-ID: <jubc09$g6t$1@dough.gmane.org>

On 20/07/2012 11:07, Jose Amoreira wrote:
> Hi.
> This is a question about style. I have a class definition that calls a
> small auxiliary function. Because this function isn't used anywhere
> else, I'd like to include it inside the class definition. However, if
> I do that, I'll have to use "self" in the call argument, which is (I
> think) rather awkward.
> Let me give an example:
>
> def is_odd(k):
>      if k % 2 == 0:
>          return False
>      else:
>          return True

I'll point out before anyone else does that you can write this function 
as a one line return.  I'll leave you to work out how.  Personally I 
prefer the longer version but each to their own.

>
> class MyOddNbr(object):
>      def __init__(self, k):
>          if is_odd(k):
>              self.k = k
>          else:
>              self.k = k + 1
>
> This works fine, but I'd like to have is_odd defined inside the class
> definition, because that's the only context where that function is
> used. That would be something like
>
> class MyOddNbr(object):
>      def is_odd(self,k):
>          if k % 2 == 0:
>              return False
>          else:
>              return True
>      def __init__(self,k):
>          if self.is_odd(k):
>              self.k = k
>          else:
>              self.k = k + 1
>
> This also works fine, but the function is_odd() is so simple and
> generic that I find it strange to define it with is_odd(self,k) or to
> call it with is_odd(self,k).
> What is the pythonic way of doing this kind of stuff?

Don't put it in the class.  It's a general purpose function that can be 
used anywhere so keep it at the module level.

> Thanks.
> Ze
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>


-- 
Cheers.

Mark Lawrence.




From rzzzwilson at gmail.com  Fri Jul 20 12:31:39 2012
From: rzzzwilson at gmail.com (Ross Wilson)
Date: Fri, 20 Jul 2012 20:31:39 +1000
Subject: [Tutor] Invalid Token Problem
In-Reply-To: <jub162$q4k$1@dough.gmane.org>
References: <CAF33E7YJT_QcsCo4-QZK3SGfwFW36=gcsMr07oJwA51K1E9Udw@mail.gmail.com>
	<5008AA28.6020807@davea.name> <5008AE03.20701@gmail.com>
	<jub162$q4k$1@dough.gmane.org>
Message-ID: <5009338B.1020206@gmail.com>

On 20/07/12 17:25, Alan Gauld wrote:
> On 20/07/12 02:01, Ross Wilson wrote:
>
>> More specifically, eval() is dangerous if you try to evaluate a string
>> supplied by someone else.  You really can't predict what will happen.
>
> It really doesn't matter who provides the string, Python and eval()
> don't care. They will behave just as dangerously if you provide the
> wrong string.

But what is the difference if I write incorrect code and *execute* it or 
write an the same code in a string and *eval()* it.  The result is the 
same whether eval() is used or not.  Same result, same risk.

Yes, beginners should be told that eval() is advanced, a little tricky 
and is not usually required.  But "dangerous"?

The risk of eval() (and exec()) is the disconnect when the string is 
supplied by someone else or from another distant part of an application, 
but it's no more 'dangerous' than if I had written the incorrect code 
directly.


From joel.goldstick at gmail.com  Fri Jul 20 13:02:17 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Fri, 20 Jul 2012 07:02:17 -0400
Subject: [Tutor] Where to put small auxiliary function
In-Reply-To: <jubc09$g6t$1@dough.gmane.org>
References: <CAN5tF-HHwOb+2-s6-UhXAM=BYDofs8OcJoOodeg=pxm41HQ9gA@mail.gmail.com>
	<jubc09$g6t$1@dough.gmane.org>
Message-ID: <CAPM-O+wnJaainE1QEg2g-N-rGt7ddVHeBAEQAuUcqAAwTp-X-g@mail.gmail.com>

On Fri, Jul 20, 2012 at 6:32 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> On 20/07/2012 11:07, Jose Amoreira wrote:
>>
>> Hi.
>> This is a question about style. I have a class definition that calls a
>> small auxiliary function. Because this function isn't used anywhere
>> else, I'd like to include it inside the class definition. However, if
>> I do that, I'll have to use "self" in the call argument, which is (I
>> think) rather awkward.
>> Let me give an example:
>>
>> def is_odd(k):
>>      if k % 2 == 0:
>>          return False
>>      else:
>>          return True
>
>
> I'll point out before anyone else does that you can write this function as a
> one line return.  I'll leave you to work out how.  Personally I prefer the
> longer version but each to their own.

I like return k % 2


>
>
>>
>> class MyOddNbr(object):
>>      def __init__(self, k):
>>          if is_odd(k):
>>              self.k = k
>>          else:
>>              self.k = k + 1

   class MyOddNbr(object):
         def __init__(self, k):
             self.k = k + k % 2


>>
>> This works fine, but I'd like to have is_odd defined inside the class
>> definition, because that's the only context where that function is
>> used. That would be something like
>>
>> class MyOddNbr(object):
>>      def is_odd(self,k):
>>          if k % 2 == 0:
>>              return False
>>          else:
>>              return True
>>      def __init__(self,k):
>>          if self.is_odd(k):
>>              self.k = k
>>          else:
>>              self.k = k + 1
>>
>> This also works fine, but the function is_odd() is so simple and
>> generic that I find it strange to define it with is_odd(self,k) or to
>> call it with is_odd(self,k).
>> What is the pythonic way of doing this kind of stuff?
>
>
> Don't put it in the class.  It's a general purpose function that can be used
> anywhere so keep it at the module level.
>
>
>> Thanks.
>> Ze
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/mailman/listinfo/tutor
>>
>
>
> --
> Cheers.
>
> Mark Lawrence.
>
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor



-- 
Joel Goldstick

From ljmamoreira at gmail.com  Fri Jul 20 13:20:07 2012
From: ljmamoreira at gmail.com (Jose Amoreira)
Date: Fri, 20 Jul 2012 12:20:07 +0100
Subject: [Tutor] Where to put small auxiliary function
In-Reply-To: <jubc09$g6t$1@dough.gmane.org>
References: <CAN5tF-HHwOb+2-s6-UhXAM=BYDofs8OcJoOodeg=pxm41HQ9gA@mail.gmail.com>
	<jubc09$g6t$1@dough.gmane.org>
Message-ID: <CAN5tF-H92KGCVY-Y6PiKiurM+O8nv7gaVnD17NWVuBihYSiTnQ@mail.gmail.com>

Hi Mark,
Thanks.

> [SNIP]
>> Let me give an example:
>>
>> def is_odd(k):
>>      if k % 2 == 0:
>>          return False
>>      else:
>>          return True
>
>
> I'll point out before anyone else does that you can write this function as a
> one line return.  I'll leave you to work out how.  Personally I prefer the
> longer version but each to their own.
>

OK, but if I wrote it as an one-liner, then it wouldn't be much use as
an example for my question...

>[SNIP]
> Don't put it in the class.  It's a general purpose function that can be used
> anywhere so keep it at the module level.
>

OK, thanks. That was the kind of advice I was hoping for.
Ze

From breamoreboy at yahoo.co.uk  Fri Jul 20 13:30:13 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 20 Jul 2012 12:30:13 +0100
Subject: [Tutor] Where to put small auxiliary function
In-Reply-To: <CAN5tF-H92KGCVY-Y6PiKiurM+O8nv7gaVnD17NWVuBihYSiTnQ@mail.gmail.com>
References: <CAN5tF-HHwOb+2-s6-UhXAM=BYDofs8OcJoOodeg=pxm41HQ9gA@mail.gmail.com>
	<jubc09$g6t$1@dough.gmane.org>
	<CAN5tF-H92KGCVY-Y6PiKiurM+O8nv7gaVnD17NWVuBihYSiTnQ@mail.gmail.com>
Message-ID: <jubfcn$d0o$1@dough.gmane.org>

On 20/07/2012 12:20, Jose Amoreira wrote:
> Hi Mark,
> Thanks.
>
>> [SNIP]
>>> Let me give an example:
>>>
>>> def is_odd(k):
>>>       if k % 2 == 0:
>>>           return False
>>>       else:
>>>           return True
>>
>>
>> I'll point out before anyone else does that you can write this function as a
>> one line return.  I'll leave you to work out how.  Personally I prefer the
>> longer version but each to their own.
>>
>
> OK, but if I wrote it as an one-liner, then it wouldn't be much use as
> an example for my question...

I didn't say write it as a one-liner, I said write it as a one line 
return.  Joel Goldstick has provided that separately, although I confess 
that I havn't had it independantly tested :)

>
>> [SNIP]
>> Don't put it in the class.  It's a general purpose function that can be used
>> anywhere so keep it at the module level.
>>
>
> OK, thanks. That was the kind of advice I was hoping for.
> Ze
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

-- 
Cheers.

Mark Lawrence.


From steve at pearwood.info  Fri Jul 20 13:30:51 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 20 Jul 2012 21:30:51 +1000
Subject: [Tutor] Where to put small auxiliary function
In-Reply-To: <CAN5tF-HHwOb+2-s6-UhXAM=BYDofs8OcJoOodeg=pxm41HQ9gA@mail.gmail.com>
References: <CAN5tF-HHwOb+2-s6-UhXAM=BYDofs8OcJoOodeg=pxm41HQ9gA@mail.gmail.com>
Message-ID: <5009416B.3030605@pearwood.info>

Jose Amoreira wrote:
> Hi.
> This is a question about style. I have a class definition that calls a
> small auxiliary function. Because this function isn't used anywhere
> else, I'd like to include it inside the class definition. 


*shrug*

Then do so.

class Whatever:
     @staticmethod
     def is_odd(k):
         return k % 2 == 1


but really, this is Python, not Java. Feel free to write top level functions.



-- 
Steven


From sntshkmr60 at gmail.com  Fri Jul 20 13:36:41 2012
From: sntshkmr60 at gmail.com (Santosh Kumar)
Date: Fri, 20 Jul 2012 17:06:41 +0530
Subject: [Tutor] Where do I start developing from?
Message-ID: <CAE7MaQb5e=L6Gk9ayea7X_9rin8WZ8divY4NyeoFKr78cLqsdg@mail.gmail.com>

Hello There,

First time I came in contact with Python programming languages was
nearly 1 year (I am learning this languages from my home). Prior to
this I was good at PHP, HTML and CSS (this doesn't mean that I want to
say that I wanted to become a web developer). I have read many books
and tutorial that which were available for free
<http://en.wikibooks.org/wiki/Non-Programmer%27s_Tutorial_for_Python_2.6>,
<http://learnpythonthehardway.com>,
<http://openbookproject.net/thinkcs/python/english2e/>.

Now I feel bored to learn more and want to implement my knowledge
somewhere. I have no idea what to do, I probably won't program a
software from starch. I want to test how much I am up to. So, where
can I start?

From leamhall at gmail.com  Fri Jul 20 14:03:58 2012
From: leamhall at gmail.com (leam hall)
Date: Fri, 20 Jul 2012 08:03:58 -0400
Subject: [Tutor] Where do I start developing from?
In-Reply-To: <CAE7MaQb5e=L6Gk9ayea7X_9rin8WZ8divY4NyeoFKr78cLqsdg@mail.gmail.com>
References: <CAE7MaQb5e=L6Gk9ayea7X_9rin8WZ8divY4NyeoFKr78cLqsdg@mail.gmail.com>
Message-ID: <CACv9p5rmYuR9aU2sVLcxzEybvhWj_UaAzedzTkRKhNJ9NqXczw@mail.gmail.com>

Santosh,

It is fun to try new activities. Maybe implement some of the PHP
things you did in Python. I find myself making small games or tools
for games when I want to try something out.

You might also want to work with Eclipse and some of the IDE tools, as
well as learn Unittesting and stuff like that.

Leam

On 7/20/12, Santosh Kumar <sntshkmr60 at gmail.com> wrote:
> Hello There,
>
> First time I came in contact with Python programming languages was
> nearly 1 year (I am learning this languages from my home). Prior to
> this I was good at PHP, HTML and CSS (this doesn't mean that I want to
> say that I wanted to become a web developer). I have read many books
> and tutorial that which were available for free
> <http://en.wikibooks.org/wiki/Non-Programmer%27s_Tutorial_for_Python_2.6>,
> <http://learnpythonthehardway.com>,
> <http://openbookproject.net/thinkcs/python/english2e/>.
>
> Now I feel bored to learn more and want to implement my knowledge
> somewhere. I have no idea what to do, I probably won't program a
> software from starch. I want to test how much I am up to. So, where
> can I start?
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>


-- 
Mind on a Mission <http://leamhall.blogspot.com/>

From wayne at waynewerner.com  Fri Jul 20 15:07:28 2012
From: wayne at waynewerner.com (Wayne Werner)
Date: Fri, 20 Jul 2012 08:07:28 -0500 (CDT)
Subject: [Tutor] suggestion for an editor
In-Reply-To: <50091A37.2070402@pearwood.info>
References: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>
	<50091A37.2070402@pearwood.info>
Message-ID: <alpine.DEB.2.02.1207200802140.3909@gilgamesh>



On Fri, 20 Jul 2012, Steven D'Aprano wrote:

> Bala subramanian wrote:
>> Friends,
>> At present i write programs using vi editor. I am interested to change to
>> something else. My specific need is that i want to select a portion/small
>> segment of my program (for eg. a nested loop) and then monitor processing
>> time it takes for that portion while i run the program. By this i hope to
>> find the segment that takes time and modify to achieve better speed. Can
>> someone please share their experience.
>
> I don't think that what you want exists. As far as I know, even full-featured 
> IDEs (Integrated Development Environments) don't include profiling of 
> selected sections of code.

It *would* be possible with a programmers editor (like Vim/Emacs) to write 
a macro or some type of extension to mark a block of code and have it 
profiled for you. But this would most likely consist of taking the exising 
code, finding the place where it's called in your application, and 
replacing that with a call through timeit, executing the app, and then 
removing the code (or do that in a temp file).

But you could also write a fairly simple Python script to do that for you, 
without any type of IDE integration.

-Wayne

From eryksun at gmail.com  Fri Jul 20 15:09:39 2012
From: eryksun at gmail.com (eryksun)
Date: Fri, 20 Jul 2012 09:09:39 -0400
Subject: [Tutor] string to binary and back... Python 3
In-Reply-To: <jub0k5$m1f$1@dough.gmane.org>
References: <5007258E.6010206@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741657F72A@SCACMX008.exchad.jpmchase.net>
	<50085303.5060407@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF4741658291C@SCACMX008.exchad.jpmchase.net>
	<50085DDC.5070405@gmail.com> <500865B4.9050105@davea.name>
	<500870A6.4090707@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582AE1@SCACMX008.exchad.jpmchase.net>
	<50087CF5.2020005@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416582C17@SCACMX008.exchad.jpmchase.net>
	<50088B26.6040701@davea.name>
	<CAOhNYv=bHZ6uStt62TOP7c3dnd1b+hc+BY5aSxB_VXKRAf5vgQ@mail.gmail.com>
	<CAOhNYvkUtqCEG2txaD8cMUEL43C7o6RYAx6NguUW7s6yk6cdbA@mail.gmail.com>
	<jub0k5$m1f$1@dough.gmane.org>
Message-ID: <CACL+1at90xVBCLRmExP2tMV=duRfjXTB_F0tkzVL6qvFb2sNyQ@mail.gmail.com>

On Fri, Jul 20, 2012 at 3:16 AM, Alan Gauld <alan.gauld at btinternet.com> wrote:
> On 20/07/12 06:48, wolfrage8765 at gmail.com wrote:
>
> Using the format method it would look like:
>
> newhexdata = bytes("{0:x}".format(numdata), "ascii")

binascii.unhexlify needs an even number of hexadecimal digits (two per
byte). So you need to either modify the above string to prepend a '0'
if the length of newhexdata is odd, or calculate the size ahead of
time and format with zero padding:

nbytes = (numdata.bit_length() + 7) // 8
size = nbytes * 2
newhexdata = bytes('{0:0{1}x}'.format(numdata, size), 'ascii')

Though creating an intermediate string of hex digits seems the long
way around. In Python 3, I'd skip binascii and instead use
int.from_bytes and int.to_bytes:

>>> numdata = 0x0102
>>> nbytes = (numdata.bit_length() + 7) // 8
>>> numdata.to_bytes(nbytes, 'big')
b'\x01\x02'

That said, it still seems more reasonable and flexible to me to use a
generator expression to do the XOR byte by byte instead of turning the
message into a big integer. You could share a large file of random
data and send a starting position along with the encrypted text. Best
of luck.

From wprins at gmail.com  Fri Jul 20 15:37:09 2012
From: wprins at gmail.com (Walter Prins)
Date: Fri, 20 Jul 2012 14:37:09 +0100
Subject: [Tutor] suggestion for an editor
In-Reply-To: <alpine.DEB.2.02.1207200802140.3909@gilgamesh>
References: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>
	<50091A37.2070402@pearwood.info>
	<alpine.DEB.2.02.1207200802140.3909@gilgamesh>
Message-ID: <CANLXbfAKmccJtg5aHxC3uH8Fb2wugaGVRVONiSBkxEHAqEKNgQ@mail.gmail.com>

On 20 July 2012 14:07, Wayne Werner <wayne at waynewerner.com> wrote:
>>> At present i write programs using vi editor. I am interested to change to
>>> something else. My specific need is that i want to select a portion/small
>>> segment of my program (for eg. a nested loop) and then monitor processing
>>> time it takes for that portion while i run the program. By this i hope to
>>> find the segment that takes time and modify to achieve better speed. Can
>>> someone please share their experience.
>>
>> I don't think that what you want exists. As far as I know, even
>> full-featured IDEs (Integrated Development Environments) don't include
>> profiling of selected sections of code.
>
> But you could also write a fairly simple Python script to do that for you,
> without any type of IDE integration.

Your comment made me think that perhaps there's a decorator for this
type of thing, so I googled it and and it turns out there is:
http://mg.pov.lt/profilehooks/

More ideas in a similar vein:
http://stackoverflow.com/questions/5375624/a-decorator-that-profiles-a-method-call-and-logs-the-profiling-result
http://code.activestate.com/recipes/577817-profile-decorator/

HTH,

Walter

From suryak at live.com  Fri Jul 20 17:41:29 2012
From: suryak at live.com (Surya K)
Date: Fri, 20 Jul 2012 21:11:29 +0530
Subject: [Tutor] A simple AJAX script needed
Message-ID: <SNT130-W27D0B64A74C7D91AC71E5AA4D80@phx.gbl>


Here is what i need:
I want to create a HTML button.. when clicked (should only work for once) should invoke a python script. 

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120720/d2274c5f/attachment.html>

From alan.gauld at btinternet.com  Fri Jul 20 20:49:50 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 20 Jul 2012 19:49:50 +0100
Subject: [Tutor] Invalid Token Problem
In-Reply-To: <5009338B.1020206@gmail.com>
References: <CAF33E7YJT_QcsCo4-QZK3SGfwFW36=gcsMr07oJwA51K1E9Udw@mail.gmail.com>
	<5008AA28.6020807@davea.name> <5008AE03.20701@gmail.com>
	<jub162$q4k$1@dough.gmane.org> <5009338B.1020206@gmail.com>
Message-ID: <juc98e$9ve$1@dough.gmane.org>

On 20/07/12 11:31, Ross Wilson wrote:

> But what is the difference if I write incorrect code and *execute* it or
> write an the same code in a string and *eval()* it.  The result is the
> same whether eval() is used or not.  Same result, same risk.


No, a much bigger risk because you can manipulate your strings at run 
time before eval()ing them.

The potential exists to modify your string in an unexpected way (via 
buggy code or buggy input data) that results in a dangerous command
set being executed, even if not intended. You can't do that in plain 
code unless you are writing self modifying code - and that's even more 
dangerous than using eval()!

I don;t want to exaggerate the risk, it is a lot lower than allowing 
anyone to type in potentially malicious code but it is still a whole 
level more dangerous than typing in explicit code and executing it in 
the interpreter. It's important not to forget that it's not just 
stranger's strings that can cause problems in eval()/exec().

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Fri Jul 20 20:54:51 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 20 Jul 2012 19:54:51 +0100
Subject: [Tutor] Where do I start developing from?
In-Reply-To: <CAE7MaQb5e=L6Gk9ayea7X_9rin8WZ8divY4NyeoFKr78cLqsdg@mail.gmail.com>
References: <CAE7MaQb5e=L6Gk9ayea7X_9rin8WZ8divY4NyeoFKr78cLqsdg@mail.gmail.com>
Message-ID: <juc9hq$9ve$2@dough.gmane.org>

On 20/07/12 12:36, Santosh Kumar wrote:

> Now I feel bored to learn more and want to implement my knowledge
> somewhere. I have no idea what to do, I probably won't program a
> software from starch. I want to test how much I am up to.

Have you tried the Python Challenge yet?

Google it.

It will take you on a programming game that teaches a new skill at each 
level. If you complete all levels you'll be a fairly competent python 
programmer, ready to tackle bigger projects...

It should stop you being bored at the very least! :-)


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Fri Jul 20 20:57:46 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 20 Jul 2012 19:57:46 +0100
Subject: [Tutor] A simple AJAX script needed
In-Reply-To: <SNT130-W27D0B64A74C7D91AC71E5AA4D80@phx.gbl>
References: <SNT130-W27D0B64A74C7D91AC71E5AA4D80@phx.gbl>
Message-ID: <juc9na$do3$1@dough.gmane.org>

On 20/07/12 16:41, Surya K wrote:
> Here is what i need:
>
> I want to create a HTML button.. when clicked (should only work for
> once) should invoke a python script.

So what bit do you need help on?
Can you create a web page?
Can you create a form?
Can you script a button?

Can you write a Python web app?
- using which framework?

You don't need AJAX to do what you ask. Simple CGI would work or any 
other web framework. But many frameworks support Ajax. But all do it 
differently.

But very little of this has to do with learning Python itself.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From alan.gauld at btinternet.com  Fri Jul 20 21:01:29 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 20 Jul 2012 20:01:29 +0100
Subject: [Tutor] suggestion for an editor
In-Reply-To: <50091A37.2070402@pearwood.info>
References: <CA+WPOVOjRBZBseprSv9rrTxBi0NRzdGWNj3Zj5nvMge1W7T16A@mail.gmail.com>
	<50091A37.2070402@pearwood.info>
Message-ID: <juc9u9$fis$1@dough.gmane.org>

On 20/07/12 09:43, Steven D'Aprano wrote:

> I don't think that what you want exists. As far as I know, even
> full-featured IDEs (Integrated Development Environments) don't include
> profiling of selected sections of code.

There are a few commercial IDEs that can do this but I've only seen it 
for compiled languages and Smalltalk. And the IDEs were all megabucks (I 
think the one I used was about $3000/seat or $100k for a site license)

I don't know of anything like that for Python (or even for Java).

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From breamoreboy at yahoo.co.uk  Fri Jul 20 21:43:47 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 20 Jul 2012 20:43:47 +0100
Subject: [Tutor] Where do I start developing from?
In-Reply-To: <CAE7MaQb5e=L6Gk9ayea7X_9rin8WZ8divY4NyeoFKr78cLqsdg@mail.gmail.com>
References: <CAE7MaQb5e=L6Gk9ayea7X_9rin8WZ8divY4NyeoFKr78cLqsdg@mail.gmail.com>
Message-ID: <juccbh$sef$1@dough.gmane.org>

On 20/07/2012 12:36, Santosh Kumar wrote:
> Hello There,
>
> First time I came in contact with Python programming languages was
> nearly 1 year (I am learning this languages from my home). Prior to
> this I was good at PHP, HTML and CSS (this doesn't mean that I want to
> say that I wanted to become a web developer). I have read many books
> and tutorial that which were available for free
> <http://en.wikibooks.org/wiki/Non-Programmer%27s_Tutorial_for_Python_2.6>,
> <http://learnpythonthehardway.com>,
> <http://openbookproject.net/thinkcs/python/english2e/>.
>
> Now I feel bored to learn more and want to implement my knowledge
> somewhere. I have no idea what to do, I probably won't program a
> software from starch. I want to test how much I am up to. So, where
> can I start?
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

This should keep you off the streets 
http://mail.python.org/mailman/listinfo/core-mentorship

-- 
Cheers.

Mark Lawrence.


From bayespokerguy at gmail.com  Fri Jul 20 22:34:20 2012
From: bayespokerguy at gmail.com (Fred G)
Date: Fri, 20 Jul 2012 16:34:20 -0400
Subject: [Tutor] Rapidly Importing CSVs with Python into MySQL
Message-ID: <CAMg+7mb02A1+KW_a1w=kTO4+XSfozHSUs5TTD0J8epxnZCYbZg@mail.gmail.com>

Hi--

This question has to do with MySQL but is fundamentally a python question,
so apologies if it seems tangential initially...

I've written a long python file that creates all the tables in database UN.
 Here is the first few lines:
import MySQLdb as mysql
statement = """CREATE DATABASE UN"""
db = mysql.connect(this stuff is correct)
cursor = db.connect()
cursor.execute(statement)
sql = """CREATE TABLE UN.1(
         id VARCHAR(255),
         name VARCHAR(255),
         age INT
       )"""
cursor.execute(sql)
statement = """LOAD DATA LOCAL INFILE...(this stuff is correct)"""
db.commit()
db.close()

db = mysql.connect(this stuff is correct)
cursor = db.connect()
sql = """CREATE TABLE UN.2(
         id VARCHAR(255),
         fname VARCHAR(255),
         lname VARCHAR(255),
         age INT
       )"""
cursor.execute(sql)
statement = """LOAD DATA LOCAL INFILE...(this stuff is correct)"""
db.commit()
db.close()

UN.3, UN.4, UN.5...etc...

I have a lot of tables here and I have a few questions:
1) When I run this Python file, only the first table is created.  Why does
the interpreter think that the statements are done at this point? I tried
experimenting with taking out all the "db.close()" commands until the last
one, but that didn't work, either.  More importantly, how can I get it such
that I can press run all and all these tables are built?  I'm going to be
re-building these (and other database's tables) pretty frequently, so it's
pretty frustrating to have to copy and paste out of the .py file into the
idle editor every single statement...

2) Is there a shortkey command to run line-by-line in a file using IDLE in
Python 2.7? (Some languages have this capacity, but a Google search of
python was not too promising about this feature...)

Thanks so much
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120720/4980d04e/attachment-0001.html>

From wprins at gmail.com  Sat Jul 21 00:27:41 2012
From: wprins at gmail.com (Walter Prins)
Date: Fri, 20 Jul 2012 23:27:41 +0100
Subject: [Tutor] Rapidly Importing CSVs with Python into MySQL
In-Reply-To: <CAMg+7mb02A1+KW_a1w=kTO4+XSfozHSUs5TTD0J8epxnZCYbZg@mail.gmail.com>
References: <CAMg+7mb02A1+KW_a1w=kTO4+XSfozHSUs5TTD0J8epxnZCYbZg@mail.gmail.com>
Message-ID: <CANLXbfD5A7ezunyWoPW50Ck2V-PLwx1ERaT3kog5CLAT3RXkyg@mail.gmail.com>

Hi Fred,

On 20 July 2012 21:34, Fred G <bayespokerguy at gmail.com> wrote:
> Here is the first few lines:

Rather than paraphrase and omit details that might be relevant to the
problem you're having, please try to whittle down your code to a bare
test program that is runnable by us as-is that demonstrates your
problem.

<snip pseudocode>

There's nothing there to suggest that Python or MySQL should thing
that the statements are done, so I'm supposing some of the stuff you
think is correct may not in fact be.  (But this is all guessing also)

> I have a lot of tables here and I have a few questions:
> 1) When I run this Python file, only the first table is created.  Why does
> the interpreter think that the statements are done at this point? I tried
> experimenting with taking out all the "db.close()" commands until the last
> one, but that didn't work, either.

Have you tried moving some of the create statements to a prior section
and creating them with the same connection/cursor you use for the
first table that is created correctly?

> More importantly, how can I get it such
> that I can press run all and all these tables are built?  I'm going to be
> re-building these (and other database's tables) pretty frequently, so it's
> pretty frustrating to have to copy and paste out of the .py file into the
> idle editor every single statement...

Of course.  There's a bug or bugs and you/we need to fix it. :)

> 2) Is there a shortkey command to run line-by-line in a file using IDLE in
> Python 2.7? (Some languages have this capacity, but a Google search of
> python was not too promising about this feature...)

On my Python 2.7 installation (Activestate distribution, in case it
matters), you can run a program line by line in the debugger from Idle
as follows:
1) Open Idle
2) Open the file you want to debug. (File->Open, click filename etc.)
3) On the main interpreter window, click "Debug" then click
"Debugger".  The debugger window appears.
4) Click back to your program window, and press F5 to run it.  The
program is started on the first line in the debugger window and
displays the first line of the program about to be executed.
5) You can now click the buttons at the top to control the program
flow, e.g. "Go", "Step", "Over", "Out" and "Quit", which respectively
(I guess) runs the program without further interruption, steps to the
next line (into functions/methods if needed), steps to the next line
(over function calls if needed), steps to the next line (returning
from the current call, e.g. "out"), and quits the program.

As an aside, I normally use Eclipse with PyDev which has very good
Python debugger integration.  If you'd like to give Eclipse a whirl
sometime and need a hand getting Eclipse set up for Python development
I'd be happy to post a little howto. (It does mean you'll have to
download Java if you don't yet have it, as well as an Eclipse bundle
etc, all of which may perhaps make it less attractive for you, but if
you happen to have a reasonably fast internet connection it's not
*that* much of a hardship...)

Walter

From alan.gauld at btinternet.com  Sat Jul 21 10:18:46 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 21 Jul 2012 09:18:46 +0100
Subject: [Tutor] Rapidly Importing CSVs with Python into MySQL
In-Reply-To: <CAMg+7mb02A1+KW_a1w=kTO4+XSfozHSUs5TTD0J8epxnZCYbZg@mail.gmail.com>
References: <CAMg+7mb02A1+KW_a1w=kTO4+XSfozHSUs5TTD0J8epxnZCYbZg@mail.gmail.com>
Message-ID: <judol6$hq5$1@dough.gmane.org>

On 20/07/12 21:34, Fred G wrote:
> Hi--
>
> This question has to do with MySQL but is fundamentally a python
> question, so apologies if it seems tangential initially...
>
> I've written a long python file that creates all the tables in database

To be honest I'd normally do that by writing a SQL file and executing 
that directly on the database. I'm not sure how you do that in MySql but 
every database I've worked with had some kind of SQL interpreter that 
you could run SQL directly.

I use Python when I want to process data not when I want to create or 
modify tables

> import MySQLdb as mysql
> statement = """CREATE DATABASE UN"""
> db = mysql.connect(this stuff is correct)
> cursor = db.connect()
> cursor.execute(statement)
> sql = """CREATE TABLE UN.1(
>           id VARCHAR(255),
>           name VARCHAR(255),
>           age INT
>         )"""
> cursor.execute(sql)
> statement = """LOAD DATA LOCAL INFILE...(this stuff is correct)"""
> db.commit()
> db.close()

You shouldn't need to close the database connection each time.

> db = mysql.connect(this stuff is correct)
> cursor = db.connect()
> sql = """CREATE TABLE UN.2(
>           id VARCHAR(255),
>           fname VARCHAR(255),
>           lname VARCHAR(255),
>           age INT
>         )"""
> cursor.execute(sql)
> statement = """LOAD DATA LOCAL INFILE...(this stuff is correct)"""
> db.commit()
> db.close()
>
> I have a lot of tables here and I have a few questions:
> 1) When I run this Python file, only the first table is created.  Why
> does the interpreter think that the statements are done at this point?

You have a bug or MySql has some kind of restriction on your user account.

Do you get any error messages?
Try running the code outside IDLE and see if anything gets reported. 
Sometimes IDEs like IDLE will mask important messages.

> tried experimenting with taking out all the "db.close()" commands until
> the last one, but that didn't work, either.  More importantly, how can I
> get it such that I can press run all and all these tables are built?

You need to fix the problem but...
You shouldn't really run programs inside IDLE except when you are 
developing/testing them. IDLE is only meant for use when writing code. 
Executing it should be done from the OS using the raw python interpreter.

>   I'm going to be re-building these (and other database's tables) pretty
> frequently, so it's pretty frustrating to have to copy and paste out of
> the .py file into the idle editor every single statement...

Not sure what you mean there? The IDLE editor is where your .py file 
should be. I'm assuming you mean from the IDLE editor into the IDLE shell?

> 2) Is there a shortkey command to run line-by-line in a file using IDLE
> in Python 2.7? (Some languages have this capacity, but a Google search
> of python was not too promising about this feature...)

Yes, there is a debugger in IDLE. You can set break points and then step 
over or into functions as needed. But usually inserting a few print 
statements into your code is all you need to monitor progress.
For example try putting a print just before every execute() call to 
print out the actual SQL command string used.

That will show which statement was last executed and you can examine the 
SQL to check there is nothing wrong with it.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From sntshkmr60 at gmail.com  Sun Jul 22 07:58:11 2012
From: sntshkmr60 at gmail.com (Santosh Kumar)
Date: Sun, 22 Jul 2012 11:28:11 +0530
Subject: [Tutor] Why isn't my simple if elif script not working?
In-Reply-To: <CAJVMpopd25AewJrMEuvP+R4iRaNZ_TjEbEGkDWHZ1dv-0pfesQ@mail.gmail.com>
References: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
	<CAJVMpooLgscKopC706kECMJCD5Obs2nNnGP2mYbNyCAGdW_G9w@mail.gmail.com>
	<5006B594.4050005@pearwood.info>
	<CAJVMpopAEPqzL78RpZ2cCOv5P4zuErMctWFA0VspoRc_RgtwjQ@mail.gmail.com>
	<5006CBE2.3060000@pearwood.info>
	<CAJVMpopd25AewJrMEuvP+R4iRaNZ_TjEbEGkDWHZ1dv-0pfesQ@mail.gmail.com>
Message-ID: <CAE7MaQba+GhE20_bOuha1oYe=b36d-0S0RNGm2_YcHY8zGcJdg@mail.gmail.com>

In first half of this script:

prompt = raw_input("Can you tell me your name? ")
if prompt in ("Yes", "yes", "y", "Y"):
    name = raw_input("What's your name? ")
elif prompt in ("No", "no", "n", "N"):
    exit("OK, have nice day..")
else:
    prompt = name

if name == "Santosh":
    print "Hey! I have the same name."
elif name in ("John Cleese", "Michael Palin"):
    print "I have no preference about your name. Really!!"
else:
    print "You have a nice name."

I want that if anyone enter their name at the first prompt the name
should be stored in "name" variable, or else if they enter anyone of
("Yes", "yes", "y", "Y") they should be given another prompt where
they can enter their name. But currently when I enter name other than
("Yes", "yes", "y", "Y") it says that name is not defined. How can I
fix this?

From alan.gauld at btinternet.com  Sun Jul 22 10:35:26 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 22 Jul 2012 09:35:26 +0100
Subject: [Tutor] Why isn't my simple if elif script not working?
In-Reply-To: <CAE7MaQba+GhE20_bOuha1oYe=b36d-0S0RNGm2_YcHY8zGcJdg@mail.gmail.com>
References: <CAE7MaQb7_xKGvcHPStN5iEzj=+ZfAVd_26vZf59h5v2Au4bpRA@mail.gmail.com>
	<CAJVMpooLgscKopC706kECMJCD5Obs2nNnGP2mYbNyCAGdW_G9w@mail.gmail.com>
	<5006B594.4050005@pearwood.info>
	<CAJVMpopAEPqzL78RpZ2cCOv5P4zuErMctWFA0VspoRc_RgtwjQ@mail.gmail.com>
	<5006CBE2.3060000@pearwood.info>
	<CAJVMpopd25AewJrMEuvP+R4iRaNZ_TjEbEGkDWHZ1dv-0pfesQ@mail.gmail.com>
	<CAE7MaQba+GhE20_bOuha1oYe=b36d-0S0RNGm2_YcHY8zGcJdg@mail.gmail.com>
Message-ID: <juge0e$1fi$1@dough.gmane.org>

On 22/07/12 06:58, Santosh Kumar wrote:

> prompt = raw_input("Can you tell me your name? ")
> if prompt in ("Yes", "yes", "y", "Y"):
>      name = raw_input("What's your name? ")
> elif prompt in ("No", "no", "n", "N"):
>      exit("OK, have nice day..")
> else:
>      prompt = name

Don't you want

        name = prompt?


> I want that if anyone enter their name at the first prompt the name
> should be stored in "name" variable, or else if they enter anyone of
> ("Yes", "yes", "y", "Y") they should be given another prompt where
> they can enter their name. But currently when I enter name other than
> ("Yes", "yes", "y", "Y") it says that name is not defined. How can I
> fix this?

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From gnj091405 at gmail.com  Mon Jul 23 00:33:19 2012
From: gnj091405 at gmail.com (Gregory Lund)
Date: Sun, 22 Jul 2012 15:33:19 -0700
Subject: [Tutor] How do I go from a textFile.txt to a list [] AND from a
 textFile.txt to a dictionary of dictionaries?
Message-ID: <CAK=ob9x_RXMnqmte9PqQvgWKw9AyxoyipfRGm1pHPqHfyxJU6A@mail.gmail.com>

Only my second try to post to this list.
This is a dumb question with a simple (I presume) solution...
but...
how do I take a txt file with semicolon separated values and create a
dictionary and a list?
( in 2.6 because I'm working with ArcGIS.)

text file is as follows: (saved as text_data.txt) (it's looooonger,
this is just a sample)

ObjID;Xval;Yval;LineID;PolyID
0;1279027.0;246427.9375;0;1
1;1279069.625;246433.234375;0;1
2;1279091.0;246435.046875;1;1
3;1279112.5;246436.3125;1;1
4;1279134.0;246437.0;2;1
5;1279176.875;246436.71875;2;1
6;1279198.375;246435.734375;3;1
7;1279219.75;246434.1875;3;1

I'd like to create a list from said text data...
and a dictionary of dictionaries from said data.

Eventually I will create a point, line and polygon shapefiles from this file.
With the data above, it would be an 8 point pnt file, a 3 line line
file and 1 polygon. (point 7 connecting back to point 0)

Right now, I'm just trying to get started by creating a list and a
dictionary (to see which I want to use when creating my shapefiles.)

And, I need to do it in 2.6 because I'm working with ArcGIS.

Thanks in advance for your thoughts?

From breamoreboy at yahoo.co.uk  Mon Jul 23 00:47:49 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 22 Jul 2012 23:47:49 +0100
Subject: [Tutor] How do I go from a textFile.txt to a list [] AND from a
 textFile.txt to a dictionary of dictionaries?
In-Reply-To: <CAK=ob9x_RXMnqmte9PqQvgWKw9AyxoyipfRGm1pHPqHfyxJU6A@mail.gmail.com>
References: <CAK=ob9x_RXMnqmte9PqQvgWKw9AyxoyipfRGm1pHPqHfyxJU6A@mail.gmail.com>
Message-ID: <juhvs0$ftr$1@dough.gmane.org>

On 22/07/2012 23:33, Gregory Lund wrote:
> Only my second try to post to this list.
> This is a dumb question with a simple (I presume) solution...
> but...
> how do I take a txt file with semicolon separated values and create a
> dictionary and a list?
> ( in 2.6 because I'm working with ArcGIS.)
>
> text file is as follows: (saved as text_data.txt) (it's looooonger,
> this is just a sample)
>
> ObjID;Xval;Yval;LineID;PolyID
> 0;1279027.0;246427.9375;0;1
> 1;1279069.625;246433.234375;0;1
> 2;1279091.0;246435.046875;1;1
> 3;1279112.5;246436.3125;1;1
> 4;1279134.0;246437.0;2;1
> 5;1279176.875;246436.71875;2;1
> 6;1279198.375;246435.734375;3;1
> 7;1279219.75;246434.1875;3;1
>
> I'd like to create a list from said text data...
> and a dictionary of dictionaries from said data.
>
> Eventually I will create a point, line and polygon shapefiles from this file.
> With the data above, it would be an 8 point pnt file, a 3 line line
> file and 1 polygon. (point 7 connecting back to point 0)
>
> Right now, I'm just trying to get started by creating a list and a
> dictionary (to see which I want to use when creating my shapefiles.)
>
> And, I need to do it in 2.6 because I'm working with ArcGIS.
>
> Thanks in advance for your thoughts?
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

Your starter for 10 is the csv module specifically 
http://docs.python.org/library/csv.html#csv-fmt-params

-- 
Cheers.

Mark Lawrence.


From wprins at gmail.com  Mon Jul 23 01:49:12 2012
From: wprins at gmail.com (Walter Prins)
Date: Mon, 23 Jul 2012 00:49:12 +0100
Subject: [Tutor] How do I go from a textFile.txt to a list [] AND from a
 textFile.txt to a dictionary of dictionaries?
In-Reply-To: <CAK=ob9x_RXMnqmte9PqQvgWKw9AyxoyipfRGm1pHPqHfyxJU6A@mail.gmail.com>
References: <CAK=ob9x_RXMnqmte9PqQvgWKw9AyxoyipfRGm1pHPqHfyxJU6A@mail.gmail.com>
Message-ID: <CANLXbfDT-T7dcvm=6pxgiLhBr6ks_EEOju6TAWWd34BhLz4UkA@mail.gmail.com>

Hi Greg,

Try this (next time try solving yourself first and post what you've
done and specific problems you're having trouble with.):

import csv

# read data using csv reader...
fileobj = open('text_data.txt')
csvreader = csv.reader(fileobj, delimiter=';')
csvreader.next() # skip header

# store lines indexed by line id in dict called polylines
# lines themselves are represented by lists.
polylines = {}

# store polygons made up of lines in dict called polygons
polygons = {}

# loop through point data from csvreader and unpack into
# dicts/lists
for line in csvreader:
    #unpack lineid and polyid
    lineid = line[3]
    polyid = line[4]

    # get the point data as list by slicing
    # first entry in list is point id.
    # maybe make points into a dict as well?
    point = line[0:3]

    # store the point in the correct line list:
    # first create an empty list if needed then append
    polylines[lineid] = polylines.get(lineid, [])
    polylines[lineid].append(point)

    # store the line in the correct polygon dict:
    # first create empty dict if needed then add
    # if the line not already part of the polygon.
    polygons[polyid] = polygons.get(polyid, {})
    if not lineid in polygons[polyid]:
        polygons[polyid][lineid] = polylines[lineid]

# check the results with some pretty printing:
import pprint
pp = pprint.PrettyPrinter(indent=4)

print "lines read from csv file:"
pp.pprint(polylines)

print "polygons read from csv file:"
pp.pprint(polygons)

Walter

From torkamani at gmail.com  Mon Jul 23 21:55:49 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Mon, 23 Jul 2012 15:55:49 -0400
Subject: [Tutor] Original indices after Sorting
Message-ID: <CAEmF0k6ydbG4vLJfggk_EVjV69hTRsU+fcmHtpv7QmsZWtQZ5Q@mail.gmail.com>

Hi every one,
How can we get the indices of values in the original list after sorting a
list?

for example:

(Pdb) A=[ 1, -1, 0, 7, 9, 1.3, 2.9 ]
(Pdb) A.sort()
(Pdb) A
[-1, 0, 1, 1.3, 2.9, 7, 9]
(Pdb)


Now I want to have the original indices of the sorted list, i.e:

[1, 2, 0, 5, 6, 3, 4]


If you know Matlab, in Matlab you'll write:

[B,L] = sort(A)

Then L, is what I'm refering to.

A
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120723/fced2627/attachment.html>

From torkamani at gmail.com  Mon Jul 23 22:02:37 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Mon, 23 Jul 2012 16:02:37 -0400
Subject: [Tutor] Original indices after Sorting
In-Reply-To: <CAEmF0k6ydbG4vLJfggk_EVjV69hTRsU+fcmHtpv7QmsZWtQZ5Q@mail.gmail.com>
References: <CAEmF0k6ydbG4vLJfggk_EVjV69hTRsU+fcmHtpv7QmsZWtQZ5Q@mail.gmail.com>
Message-ID: <CAEmF0k6frAVJEqZap13HtVUxGMhQ_mXs=Qn8MGp9QSat48pw0Q@mail.gmail.com>

By the way, I myself, have this solution:

How can we get the indices of values in the original list after sorting a
> list?
>
>
(Pdb) A=[ 1, -1, 0, 7, 9, 1.3, 2.9 ]
(Pdb) sorted(zip(A, range(len(A))), key = lambda x: x[0])
[(-1, 1), (0, 2), (1, 0), (1.3, 5), (2.9, 6), (7, 3), (9, 4)]


But compared to what Matlab neatly does, it's too ugly:


> If you know Matlab, in Matlab you'll write:
>
> [B,L] = sort(A)
>
>
A
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120723/e6a74969/attachment.html>

From tvssarma.omega9 at gmail.com  Mon Jul 23 22:05:59 2012
From: tvssarma.omega9 at gmail.com (Sarma Tangirala)
Date: Tue, 24 Jul 2012 01:35:59 +0530
Subject: [Tutor] Original indices after Sorting
In-Reply-To: <CAEmF0k6ydbG4vLJfggk_EVjV69hTRsU+fcmHtpv7QmsZWtQZ5Q@mail.gmail.com>
References: <CAEmF0k6ydbG4vLJfggk_EVjV69hTRsU+fcmHtpv7QmsZWtQZ5Q@mail.gmail.com>
Message-ID: <CABFCkKQ=1Hp78qerdJ9Vkd+vAEL0=Avu1DHLEtfLD4vrHbnNfQ@mail.gmail.com>

On 24 July 2012 01:25, Ali Torkamani <torkamani at gmail.com> wrote:

> Hi every one,
> How can we get the indices of values in the original list after sorting a
> list?
>
> for example:
>
> (Pdb) A=[ 1, -1, 0, 7, 9, 1.3, 2.9 ]
> (Pdb) A.sort()
> (Pdb) A
> [-1, 0, 1, 1.3, 2.9, 7, 9]
> (Pdb)
>
>
> Now I want to have the original indices of the sorted list, i.e:
>
> [1, 2, 0, 5, 6, 3, 4]
>
>
>
This should be of help ...
http://stackoverflow.com/questions/7851077/how-to-return-index-of-a-sorted-list...
 it uses the sorted method to sort a list rather than A.sort()


-- 
An monkey typed up this email. Please excuse him if he made a stupid error!
  .
    .
. . .
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120724/b29ade6d/attachment.html>

From torkamani at gmail.com  Mon Jul 23 22:12:48 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Mon, 23 Jul 2012 16:12:48 -0400
Subject: [Tutor] Original indices after Sorting
In-Reply-To: <CABFCkKQ=1Hp78qerdJ9Vkd+vAEL0=Avu1DHLEtfLD4vrHbnNfQ@mail.gmail.com>
References: <CAEmF0k6ydbG4vLJfggk_EVjV69hTRsU+fcmHtpv7QmsZWtQZ5Q@mail.gmail.com>
	<CABFCkKQ=1Hp78qerdJ9Vkd+vAEL0=Avu1DHLEtfLD4vrHbnNfQ@mail.gmail.com>
Message-ID: <CAEmF0k5GbbuH+sMX+b_Mg7XDzUb42zDq_xXzbCwQBef6C+AAPQ@mail.gmail.com>

I found the solution, we should use numpy's argsort(...):


How can we get the indices of values in the original list after sorting a
> list?
>
>>
>> for example:
>>
>> (Pdb) A=[ 1, -1, 0, 7, 9, 1.3, 2.9 ]
>> (Pdb) A.sort()
>> (Pdb) A
>> [-1, 0, 1, 1.3, 2.9, 7, 9]
>> (Pdb)
>>
>>
>> Now I want to have the original indices of the sorted list, i.e:
>>
>> [1, 2, 0, 5, 6, 3, 4]
>>
>>
(Pdb)  A=[ 1, -1, 0, 7, 9, 1.3, 2.9 ]
(Pdb) sort_index = numpy.argsort(A)
(Pdb) sort_index
array([1, 2, 0, 5, 6, 3, 4])
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120723/10f17bf1/attachment.html>

From torkamani at gmail.com  Mon Jul 23 23:11:40 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Mon, 23 Jul 2012 17:11:40 -0400
Subject: [Tutor] Counting the consistent rankings
Message-ID: <CAEmF0k5nq8gpM987tme3Z6cDjetgow48qyJHj=8pAsp6tq=hvw@mail.gmail.com>

Hi All,

I want to count how many  (what percentage)  of partial orders are
identical in two lists.  (For example let L1 be the list of ground truth
scores and L2 be the estimated scores for a list of students, I want to
check how many (what percentage) of pairwise orderings (rankings) are
preserved)

I have written the following (inefficient) code, is there anyway in Python
to write it more efficiently?


    S=0;
    A=0;
    n=len(L1)
    for i in range(n):
        for j in range(i+1,n,1):
            A+=1;
            if (L1[i]>L1[j] and L2[i]>L2[j]) or (L1[i]<L1[j] and
L2[i]<L2[j]) or (L1[i]==L1[j] and L2[i]==L2[j]):
                S+=1

    print(100*float(S)/A)


Thanks,

A
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120723/5d092187/attachment.html>

From torkamani at gmail.com  Mon Jul 23 23:19:33 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Mon, 23 Jul 2012 17:19:33 -0400
Subject: [Tutor] Counting the consistent rankings
In-Reply-To: <CAEmF0k5nq8gpM987tme3Z6cDjetgow48qyJHj=8pAsp6tq=hvw@mail.gmail.com>
References: <CAEmF0k5nq8gpM987tme3Z6cDjetgow48qyJHj=8pAsp6tq=hvw@mail.gmail.com>
Message-ID: <CAEmF0k6PDbgqbnaWFtvXKg8ZwPqDALnq7=TPhS8VFyxWRC_3bA@mail.gmail.com>

>
>     S=0;
>     A=0;
>     n=len(L1)
>     for i in range(n):
>         for j in range(i+1,n,1):
>             A+=1;
>             if (L1[i]>L1[j] and L2[i]>L2[j]) or (L1[i]<L1[j] and
> L2[i]<L2[j]) or (L1[i]==L1[j] and L2[i]==L2[j]):
>                 S+=1
>
>     print(100*float(S)/A)
>

In this code, A is equal to (len(L1)*( len(L1)-1)/2), but I had left it
this way for later sanity check.


>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120723/dfc6d2c7/attachment.html>

From alan.gauld at btinternet.com  Mon Jul 23 23:40:12 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 23 Jul 2012 22:40:12 +0100
Subject: [Tutor] Original indices after Sorting
In-Reply-To: <CAEmF0k6ydbG4vLJfggk_EVjV69hTRsU+fcmHtpv7QmsZWtQZ5Q@mail.gmail.com>
References: <CAEmF0k6ydbG4vLJfggk_EVjV69hTRsU+fcmHtpv7QmsZWtQZ5Q@mail.gmail.com>
Message-ID: <jukgbs$his$1@dough.gmane.org>

On 23/07/12 20:55, Ali Torkamani wrote:
> Hi every one,
> How can we get the indices of values in the original list after sorting
> a list?

create a sorted copy

L1 = [...original data...]
L2 = sorted(L1)

Now you can compare indexes except if there are duplicate elements
and you need to know which specific copy is where. In that case
I think you need to use id() but even that fails for small ints and 
other cached values.

If the data is too large to hold two copies you could create a list
of tuples with the original indexes:

L2 = []
n = -1
while L1:
    L2.append( (n,L1.pop()) )
    n -= 1

This creates L2 with a reversed L1 using the negative index of the 
original list. A bit clunky, I'm sure there is a neater way but it seems 
to work. But hopefully memory is not so tight you need to do this!


HTH

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




From torkamani at gmail.com  Mon Jul 23 23:48:52 2012
From: torkamani at gmail.com (Ali Torkamani)
Date: Mon, 23 Jul 2012 17:48:52 -0400
Subject: [Tutor] Original indices after Sorting
In-Reply-To: <jukgbs$his$1@dough.gmane.org>
References: <CAEmF0k6ydbG4vLJfggk_EVjV69hTRsU+fcmHtpv7QmsZWtQZ5Q@mail.gmail.com>
	<jukgbs$his$1@dough.gmane.org>
Message-ID: <CAEmF0k64irEjGuwqSxN8z8bKa4WyisriXGHa08YDF3BoqUFV1Q@mail.gmail.com>

On Mon, Jul 23, 2012 at 5:40 PM, Alan Gauld <alan.gauld at btinternet.com>wrote:

>
> create a sorted copy
>
> L1 = [...original data...]
> L2 = sorted(L1)
>
> Now you can compare indexes except if there are duplicate elements
> and you need to know which specific copy is where. In that case
> I think you need to use id() but even that fails for small ints and other
> cached values.
>
> If the data is too large to hold two copies you could create a list
> of tuples with the original indexes:
>
> L2 = []
> n = -1
> while L1:
>    L2.append( (n,L1.pop()) )
>    n -= 1
>
> This creates L2 with a reversed L1 using the negative index of the
> original list. A bit clunky, I'm sure there is a neater way but it seems to
> work. But hopefully memory is not so tight you need to do this!
>

 sort_index = numpy.argsort(L1) solves it, and does not need such
Mumbo-Jumping!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120723/f9130311/attachment.html>

From emile at fenx.com  Tue Jul 24 00:12:37 2012
From: emile at fenx.com (Emile van Sebille)
Date: Mon, 23 Jul 2012 15:12:37 -0700
Subject: [Tutor] Counting the consistent rankings
In-Reply-To: <CAEmF0k5nq8gpM987tme3Z6cDjetgow48qyJHj=8pAsp6tq=hvw@mail.gmail.com>
References: <CAEmF0k5nq8gpM987tme3Z6cDjetgow48qyJHj=8pAsp6tq=hvw@mail.gmail.com>
Message-ID: <juki4c$tsu$1@dough.gmane.org>

On 7/23/2012 2:11 PM Ali Torkamani said...
> Hi All,
>
> I want to count how many  (what percentage)  of partial orders are
> identical in two lists.

You may find SequenceMatcher from module difflib helpful.  See 
http://docs.python.org/library/difflib.html for info.

Emile




> (For example let L1 be the list of ground truth
> scores and L2 be the estimated scores for a list of students, I want to
> check how many (what percentage) of pairwise orderings (rankings) are
> preserved)
>
> I have written the following (inefficient) code, is there anyway in
> Python to write it more efficiently?
>
>
>      S=0;
>      A=0;
>      n=len(L1)
>      for i in range(n):
>          for j in range(i+1,n,1):
>              A+=1;
>              if (L1[i]>L1[j] and L2[i]>L2[j]) or (L1[i]<L1[j] and
> L2[i]<L2[j]) or (L1[i]==L1[j] and L2[i]==L2[j]):
>                  S+=1
>
>      print(100*float(S)/A)
>
>
> Thanks,
>
> A
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>




From eryksun at gmail.com  Tue Jul 24 04:52:19 2012
From: eryksun at gmail.com (eryksun)
Date: Mon, 23 Jul 2012 22:52:19 -0400
Subject: [Tutor] Original indices after Sorting
In-Reply-To: <CAEmF0k6frAVJEqZap13HtVUxGMhQ_mXs=Qn8MGp9QSat48pw0Q@mail.gmail.com>
References: <CAEmF0k6ydbG4vLJfggk_EVjV69hTRsU+fcmHtpv7QmsZWtQZ5Q@mail.gmail.com>
	<CAEmF0k6frAVJEqZap13HtVUxGMhQ_mXs=Qn8MGp9QSat48pw0Q@mail.gmail.com>
Message-ID: <CACL+1avHOdOfHodgy8HL6zMUrtYPhA9EgHGzafvSA5u3wZ0c_A@mail.gmail.com>

On Mon, Jul 23, 2012 at 4:02 PM, Ali Torkamani <torkamani at gmail.com> wrote:
> By the way, I myself, have this solution:
>
>> How can we get the indices of values in the original list after sorting a
>> list?
>>
>
> (Pdb) A=[ 1, -1, 0, 7, 9, 1.3, 2.9 ]
> (Pdb) sorted(zip(A, range(len(A))), key = lambda x: x[0])
> [(-1, 1), (0, 2), (1, 0), (1.3, 5), (2.9, 6), (7, 3), (9, 4)]

Here's a variation:

from operator import itemgetter

def sortix(seq):
    L, B = zip(*sorted(enumerate(seq), key=itemgetter(1)))
    return B, L

Your original result is shaped Nx2. zip(*seq) is a quick way to
transpose that to 2xN. The * operator turns each item into an argument
of zip.

Example:

>>> A = [1, -1, 0, 7, 9, 1.3, 2.9]
>>> B, L = sortix(A)
>>> B
(-1, 0, 1, 1.3, 2.9, 7, 9)
>>> L
(1, 2, 0, 5, 6, 3, 4)

If you need lists, you can use map:

>>> B, L = map(list, sortix(A))
>>> B
[-1, 0, 1, 1.3, 2.9, 7, 9]
>>> L
[1, 2, 0, 5, 6, 3, 4]

From fomcl at yahoo.com  Tue Jul 24 11:18:43 2012
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Tue, 24 Jul 2012 02:18:43 -0700 (PDT)
Subject: [Tutor] measuring the start up time of an event-driven program
Message-ID: <1343121523.62844.YahooMailNeo@web110708.mail.gq1.yahoo.com>

Hi,

I would like to test how long it takes for two versions of the same program to start up and be ready to receive commands. The program is SPSS version-very-old vs. SPSS version-latest.

Normally I'd just fire the program up in a subprocess and measure the time before and after finishing. But this kind of program is never finished. It's looping until infinity and waiting for events/commands. I tried wrapping it in a "while True" loop, and break out of the loop and terminate the program (using ctypes) if the retcode of the process is equal to zero. But that doesn't work. 


I know that, in case of spss, there is a Python api available, but this would measure the start-up time of spss without the graphical interface, only the backend. That's not what I want. I want to know how long the user on average needs to wait before he can start doing analyses in SPSS. If it takes way much longer in the new version, the user might be more inclined not to close the program after use, which may lead to a lack of concurrent licenses.

?
Thanks!


Regards,
Albert-Jan


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a 
fresh water system, and public health, what have the Romans ever done for us?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120724/4cb8863e/attachment.html>

From steve at pearwood.info  Tue Jul 24 11:48:06 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 24 Jul 2012 19:48:06 +1000
Subject: [Tutor] measuring the start up time of an event-driven program
In-Reply-To: <1343121523.62844.YahooMailNeo@web110708.mail.gq1.yahoo.com>
References: <1343121523.62844.YahooMailNeo@web110708.mail.gq1.yahoo.com>
Message-ID: <20120724094806.GB25953@ando>

On Tue, Jul 24, 2012 at 02:18:43AM -0700, Albert-Jan Roskam wrote:
> Hi,
> 
> I would like to test how long it takes for two versions of the same 
> program to start up and be ready to receive commands. The program is 
> SPSS version-very-old vs. SPSS version-latest.

I don't think this is a Python question. I think this is a "what tools 
does my operating system provide for fine testing of process startup 
time?" question.

I'm not an expert, but I suspect that the answer will be, "none". Which 
OS are you using?

If the SPSS app has the ability to run commands specified from a 
script, and then automatically exit, perhaps you can approximate 
start-up time as "start-up, run an empty script, and exit" time, 
assuming that running the script and exiting will be negligible.

Or, if the app is slow enough (I'm looking at you Chrome, fastest way to 
browse the web my arse) perhaps you could just time it with a 
stop-watch.


> I want to know how long the user on average needs to wait before he 
> can start doing analyses in SPSS. If it takes way much longer in the 
> new version, the user might be more inclined not to close the program 
> after use, which may lead to a lack of concurrent licenses.

If you have to measure the difference to notice the difference, the 
average user won't notice the difference.



-- 
Steven

From fomcl at yahoo.com  Tue Jul 24 12:28:06 2012
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Tue, 24 Jul 2012 03:28:06 -0700 (PDT)
Subject: [Tutor] measuring the start up time of an event-driven program
In-Reply-To: <20120724094806.GB25953@ando>
References: <1343121523.62844.YahooMailNeo@web110708.mail.gq1.yahoo.com>
	<20120724094806.GB25953@ando>
Message-ID: <1343125686.6453.YahooMailNeo@web110704.mail.gq1.yahoo.com>

From: Steven D'Aprano <steve at pearwood.info>

To: tutor at python.org 
>Sent: Tuesday, July 24, 2012 11:48 AM
>Subject: Re: [Tutor] measuring the start up time of an event-driven program
> 
>On Tue, Jul 24, 2012 at 02:18:43AM -0700, Albert-Jan Roskam wrote:
>> Hi,
>> 
>> I would like to test how long it takes for two versions of the same 
>> program to start up and be ready to receive commands. The program is 
>> SPSS version-very-old vs. SPSS version-latest.
>
>I don't think this is a Python question. I think this is a "what tools 
>does my operating system provide for fine testing of process startup 
>time?" question.
>
>I'm not an expert, but I suspect that the answer will be, "none". Which 
>OS are you using?
>===> I am using Windows 7, but I'd find it interesting to hear about Linux tools too (though I couldn't use them in this particular case). Would be cool if such a tool could differentiate between the various processes involved. In this case stats.exe (frontend) and spssengine.exe (backend).
>
>If the SPSS app has the ability to run commands specified from a 
>script, and then automatically exit, perhaps you can approximate 
>start-up time as "start-up, run an empty script, and exit" time, 
>assuming that running the script and exiting will be negligible.
>
>===> Well, I actually do that too. But the full-fledged app has a GUI programmed in (I think) C++ (old version) or Java (new version). The latter is markedly slower. People work with the GUI based spss all the time. Only when their code is final, people *might* use the scripting facility.
>
>Or, if the app is slow enough (I'm looking at you Chrome, fastest way to 
>browse the web my arse) perhaps you could just time it with a 
>stop-watch.
>
>===> ;-))) Yeah, good idea (I mean the stop-watch, not Chrome --I don't like Google's information obesity).
>
>> I want to know how long the user on average needs to wait before he 
>> can start doing analyses in SPSS. If it takes way much longer in the 
>> new version, the user might be more inclined not to close the program 
>> after use, which may lead to a lack of concurrent licenses.
>
>If you have to measure the difference to notice the difference, the 
>average user won't notice the difference.
>
>===> Good point. But it is noticeable, and now I'd like to quantify that. Something like: "ceteris paribus it takes 2.2 times longer to start up the new version, as compared with the old version"
>
>==> And of course: Thanks!
>
>-- 
>Steven
>_______________________________________________
>Tutor maillist? -? Tutor at python.org
>To unsubscribe or change subscription options:
>http://mail.python.org/mailman/listinfo/tutor
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120724/b864cee1/attachment.html>

From mi.janssen at gmail.com  Tue Jul 24 13:28:30 2012
From: mi.janssen at gmail.com (=?ISO-8859-1?Q?Michael_Jan=DFen?=)
Date: Tue, 24 Jul 2012 13:28:30 +0200
Subject: [Tutor] measuring the start up time of an event-driven program
In-Reply-To: <1343121523.62844.YahooMailNeo@web110708.mail.gq1.yahoo.com>
References: <1343121523.62844.YahooMailNeo@web110708.mail.gq1.yahoo.com>
Message-ID: <CAKhoyxF8HW2MkZ=d-AejY34zrkCcTV3rPdrn1u0exb_LyD3ubQ@mail.gmail.com>

On 24 July 2012 11:18, Albert-Jan Roskam <fomcl at yahoo.com> wrote:

> I would like to test how long it takes for two versions of the same
> program to start up and be ready to receive commands. The program is SPSS
> version-very-old vs. SPSS version-latest.
>
> Normally I'd just fire the program up in a subprocess and measure the time
> before and after finishing. But this kind of program is never finished.
> It's looping until infinity and waiting for events/commands. I tried
> wrapping it in a "while True" loop, and break out of the loop and terminate
> the program (using ctypes) if the retcode of the process is equal to zero.
> But that doesn't work.
>

So, the question is, what defines your end point for timing. I'd say, you
could either check if the program *looks* like being ready (
http://sikuli.org/ could help here) or if it doesn't use up cpu anymore.

Both ways, I'd except a fair amount of overhead and for the cpu-usage way
bogus results also. Both problems could perhaps be addressed by enough
timing runs.

cu, Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120724/75a26117/attachment.html>

From stefan_ml at behnel.de  Tue Jul 24 14:29:00 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 24 Jul 2012 14:29:00 +0200
Subject: [Tutor] measuring the start up time of an event-driven program
In-Reply-To: <1343121523.62844.YahooMailNeo@web110708.mail.gq1.yahoo.com>
References: <1343121523.62844.YahooMailNeo@web110708.mail.gq1.yahoo.com>
Message-ID: <jum4eb$omd$1@dough.gmane.org>

Albert-Jan Roskam, 24.07.2012 11:18:
> I would like to test how long it takes for two versions of the same
> program to start up and be ready to receive commands. The program is
> SPSS version-very-old vs. SPSS version-latest.
> 
> Normally I'd just fire the program up in a subprocess and measure the
> time before and after finishing. But this kind of program is never
> finished. It's looping until infinity and waiting for events/commands. I
> tried wrapping it in a "while True" loop, and break out of the loop and
> terminate the program (using ctypes) if the retcode of the process is
> equal to zero. But that doesn't work.

Is it really looping or is it just sitting and waiting for input? You might
be able to provide some input that makes it terminate.

Stefan


From zhenzhen.qi at gmail.com  Wed Jul 25 00:51:26 2012
From: zhenzhen.qi at gmail.com (Zhenzhen)
Date: Tue, 24 Jul 2012 15:51:26 -0700
Subject: [Tutor] where is my MySQL database
Message-ID: <CAHOFYs4p_JyPPZTZ9bDVs0anv0oujcWKaEjyAMkNX29Ekq5i1w@mail.gmail.com>

hi, I'm working on setting up my database for django by editing the
following file:
mysite/settings.py

for the field "name", I am suppose to put down the full path of my mySQL
database.

I did create a few databases in MySQL, however, I don't seem to be able to
find them anywhere.

I have tried 'grep datadir /etc/my.cnf' but found out my.cnf file doesn't
even exist in my /etc/ directory.

Is there something wrong with mySQL installation? Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120724/1cee2098/attachment.html>

From alan.gauld at btinternet.com  Wed Jul 25 02:09:17 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 25 Jul 2012 01:09:17 +0100
Subject: [Tutor] where is my MySQL database
In-Reply-To: <CAHOFYs4p_JyPPZTZ9bDVs0anv0oujcWKaEjyAMkNX29Ekq5i1w@mail.gmail.com>
References: <CAHOFYs4p_JyPPZTZ9bDVs0anv0oujcWKaEjyAMkNX29Ekq5i1w@mail.gmail.com>
Message-ID: <jundfd$dav$1@dough.gmane.org>

On 24/07/12 23:51, Zhenzhen wrote:
> hi, I'm working on setting up my database for django by editing the
> following file:
> mysite/settings.py
>
> for the field "name", I am suppose to put down the full path of my mySQL
> database.


This doesn't have much to do with learing Python, or even Django...

You might get a better response from a MySql mailing list or forum.
OTOH There are MySql users here too so you may get lucky.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/


From mi.janssen at gmail.com  Wed Jul 25 03:58:43 2012
From: mi.janssen at gmail.com (=?ISO-8859-1?Q?Michael_Jan=DFen?=)
Date: Wed, 25 Jul 2012 03:58:43 +0200
Subject: [Tutor] where is my MySQL database
In-Reply-To: <CAHOFYs4p_JyPPZTZ9bDVs0anv0oujcWKaEjyAMkNX29Ekq5i1w@mail.gmail.com>
References: <CAHOFYs4p_JyPPZTZ9bDVs0anv0oujcWKaEjyAMkNX29Ekq5i1w@mail.gmail.com>
Message-ID: <CAKhoyxEO-tBetAWQ_tuGqaTLdupGKbJJ3AkuyYjap9nRjjKK8g@mail.gmail.com>

On 25 July 2012 00:51, Zhenzhen <zhenzhen.qi at gmail.com> wrote:

> hi, I'm working on setting up my database for django by editing the
> following file:
> mysite/settings.py
>
> for the field "name", I am suppose to put down the full path of my mySQL
> database.
>

for mysql, it needs to be the database name, specified in the CREATE TABLE
statement. With sqlite3, it would be indeed the filename (an non-existent
filename, sqlite3 would create the file)


> I did create a few databases in MySQL, however, I don't seem to be able
> to find them anywhere.
>
> I have tried 'grep datadir /etc/my.cnf' but found out my.cnf file doesn't
> even exist in my /etc/ directory.
>

for me it is /etc/mysql/my.cnf

cu, Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120725/726ed63f/attachment.html>

From mnickey at gmail.com  Wed Jul 25 07:19:59 2012
From: mnickey at gmail.com (M Nickey)
Date: Tue, 24 Jul 2012 22:19:59 -0700
Subject: [Tutor] Recreating the help module
Message-ID: <B7859D6F-A8E1-486D-B770-15344F3F41F1@gmail.com>

Hey all,

I'm trying to recreate the 'help' on various modules that are available. So far, I have a bit of code and it seems to be working for the most part. I can get the modules available but I also want to be able to print the information that is available for each module.

Current output:
['__add__', '__class__', '__contains__', ... 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.

Desired result:
['__add__', '__class__', '__contains__', ... 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']

atof(s):
Return the floating point number represented by the st
atof(s) -> float

I'm close (I think) but I just can figure out how to grab that part of the documentation. Any advice?


Code:
import os
import string
import inspect

def getDirName():
    modList = []
    manPage = 'string'       #used as a place holder for now...           
    cwd = dir(manPage)
    print cwd
    info = inspect.getdoc(cwd[35])          
    print info
    return

if __name__ == '__main__':
    getDirName()

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120724/12a4b67d/attachment.html>

From __peter__ at web.de  Wed Jul 25 08:47:28 2012
From: __peter__ at web.de (Peter Otten)
Date: Wed, 25 Jul 2012 08:47:28 +0200
Subject: [Tutor] Recreating the help module
References: <B7859D6F-A8E1-486D-B770-15344F3F41F1@gmail.com>
Message-ID: <juo4pp$ta$1@dough.gmane.org>

M Nickey wrote:

> Hey all,
> 
> I'm trying to recreate the 'help' on various modules that are available.
> So far, I have a bit of code and it seems to be working for the most part.
> I can get the modules available but I also want to be able to print the
> information that is available for each module.
> 
> Current output:
> ['__add__', '__class__', '__contains__', ... 'partition', 'replace',
> ['rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split',
> ['splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate',
> ['upper', 'zfill']
> str(object) -> string
> 
> Return a nice string representation of the object.
> If the argument is a string, the return value is the same object.
> 
> Desired result:
> ['__add__', '__class__', '__contains__', ... 'partition', 'replace',
> ['rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split',
> ['splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate',
> ['upper', 'zfill']
> 
> atof(s):
> Return the floating point number represented by the st
> atof(s) -> float
> 
> I'm close (I think) but I just can figure out how to grab that part of the
> documentation. Any advice?

I'm guessing you want the functions in the string module rather than the 
methods of the str type? 

If so: you can get the module object from the name with the __import__() 
builtin function:

> Code:
> import os
> import string
> import inspect
> 
> def getDirName():
>     modList = []

      module_name = "string"
      module = __import__(module_name) 
      cwd = dir(module)

>     print cwd
>     info = inspect.getdoc(cwd[35])
>     print info
>     return
> 
> if __name__ == '__main__':
>     getDirName()

There's one complication with __import__(), it always gives you the toplevel 
package, e. g. to get the os.path module you have to do

package = __import__("os.path")
module = getattr(package, "path")

You might also take a look into the source of the pydoc module.


From fomcl at yahoo.com  Wed Jul 25 12:28:50 2012
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Wed, 25 Jul 2012 03:28:50 -0700 (PDT)
Subject: [Tutor] measuring the start up time of an event-driven program
In-Reply-To: <jum4eb$omd$1@dough.gmane.org>
References: <1343121523.62844.YahooMailNeo@web110708.mail.gq1.yahoo.com>
	<jum4eb$omd$1@dough.gmane.org>
Message-ID: <1343212130.80977.YahooMailNeo@web110714.mail.gq1.yahoo.com>

Albert-Jan Roskam, 24.07.2012 11:18:
>> I would like to test how long it takes for two versions of the same
>> program to start up and be ready to receive commands. The program is
>> SPSS version-very-old vs. SPSS version-latest.
>> 
>> Normally I'd just fire the program up in a subprocess and measure the
>> time before and after finishing. But this kind of program is never
>> finished. It's looping until infinity and waiting for events/commands. I
>> tried wrapping it in a "while True" loop, and break out of the loop and
>> terminate the program (using ctypes) if the retcode of the process is
>> equal to zero. But that doesn't work.
>
>Is it really looping or is it just sitting and waiting for input? You might
>be able to provide some input that makes it terminate.
>
>Stefan
>
>===> Good idea. And thanks everybody for replying. I solved it by using SendKeys-ctypes-0.2.
>I couldn't use pywin32 -- no rights to install this :-((
>Here's a fragment (it won't work on an english locale, I think. Replace 'u' with 'r' (for "Run...").
>??? start = time.clock()
>??? if windows_version == "Windows 2000" and spss_version == 14:
>??????? SendKeys.SendKeys("""{LWIN}
>??????????? {PAUSE 0.25}
>??????????? u
>??????????? %s{ENTER}
>??????????? {PAUSE %2.1f}
>??????????? %%{TAB 2}
>??????????? %%{F4}""" % (spss_path, PAUSE), with_spaces=True)
>??? elif windows_version == "Windows 7":
>??????? SendKeys.SendKeys("""{LWIN}
>??????????? {PAUSE 0.25}
>??????????? %s{ENTER}
>??????????? {PAUSE %2.1f}
>??????????? %%{TAB}
>??????????? %%{F4}""" % (spss_path, PAUSE), with_spaces=True)
>??? end = time.clock()
>??? duration = (end - start) 

________________________________
From: Stefan Behnel <stefan_ml at behnel.de>
To: tutor at python.org 
Sent: Tuesday, July 24, 2012 2:29 PM
Subject: Re: [Tutor] measuring the start up time of an event-driven program
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120725/83bd1d7f/attachment.html>

From leamhall at gmail.com  Wed Jul 25 12:48:00 2012
From: leamhall at gmail.com (Leam Hall)
Date: Wed, 25 Jul 2012 06:48:00 -0400
Subject: [Tutor] Getting name of Widget for event.widget?
Message-ID: <500FCEE0.2080701@gmail.com>

Note that this is for an on-line class so I'd appreciate pointers to 
what I need to read or think about more than the answer out right.

Using Python 3 on Linux, is there a way to get the name of a widget 
instead of numeric representations in event.widget?

What I tried were a few variations of:

     def small_frame_handler1(self, event):
         print(event.widget, " clicked at ", event.x, event.y)

and got:

     .140937484.144713004  clicked at  350 39

Called by:

        frame1 = Frame(self, bg="blue")
        frame1.grid(row=0, column=0, rowspan=1, columnspan=2, sticky=ALL)
        frame1.bind("<Button-1>", self.small_frame_handler1)

I tried to send parameters in the "bind" but kept getting argument 
mis-matches. There are two frames and I'd like to be able to put the 
name of the frame in the print without having to have a separate method 
for each one. This is inside a class, if that makes a difference.

Thanks!

Leam

From joel.goldstick at gmail.com  Wed Jul 25 15:40:16 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Wed, 25 Jul 2012 09:40:16 -0400
Subject: [Tutor] where is my MySQL database
In-Reply-To: <CAKhoyxEO-tBetAWQ_tuGqaTLdupGKbJJ3AkuyYjap9nRjjKK8g@mail.gmail.com>
References: <CAHOFYs4p_JyPPZTZ9bDVs0anv0oujcWKaEjyAMkNX29Ekq5i1w@mail.gmail.com>
	<CAKhoyxEO-tBetAWQ_tuGqaTLdupGKbJJ3AkuyYjap9nRjjKK8g@mail.gmail.com>
Message-ID: <CAPM-O+zrr2PAuzp0Y738Gk8yfNm-LpARtYZT_FMSPMrK9gnmAw@mail.gmail.com>

On Tue, Jul 24, 2012 at 9:58 PM, Michael Jan?en <mi.janssen at gmail.com> wrote:
> On 25 July 2012 00:51, Zhenzhen <zhenzhen.qi at gmail.com> wrote:
>>
>> hi, I'm working on setting up my database for django by editing the
>> following file:
>> mysite/settings.py
>>
>> for the field "name", I am suppose to put down the full path of my mySQL
>> database.
>
That is not true.  You need to read the tutorial at
https://docs.djangoproject.com/en/dev/intro/tutorial01/#database-setup

You new the path only if you are using sqlite3, which you are not.

>
> for mysql, it needs to be the database name, specified in the CREATE TABLE
> statement. With sqlite3, it would be indeed the filename (an non-existent
> filename, sqlite3 would create the file)
>
>>
>> I did create a few databases in MySQL, however, I don't seem to be able to
>> find them anywhere.
>>
>> I have tried 'grep datadir /etc/my.cnf' but found out my.cnf file doesn't
>> even exist in my /etc/ directory.
>
>
> for me it is /etc/mysql/my.cnf
>
> cu, Michael
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick

From lipskathekat at yahoo.co.uk  Wed Jul 25 17:11:58 2012
From: lipskathekat at yahoo.co.uk (Lipska TheCat)
Date: Wed, 25 Jul 2012 16:11:58 +0100 (BST)
Subject: [Tutor] What on earth is happening here ???
Message-ID: <1343229118.88627.YahooMailNeo@web133002.mail.ir2.yahoo.com>

Pythoners

I have a class
The point of this class is to provide an uncluttered interface
to Document Creation


from xml.dom.minidom import getDOMImplementation

class NpDOMDocumentFactory:
??? 
??? """A DOM Document Factory convenience class"""? 

??? 

??? # make these private "by convention"
??? __DOMImplementation = getDOMImplementation()
??? __defaultNamespaceUri = "http://nuldomain.com/"
??? __defaultQualifiedName = "root"
??? __defaultDoctype = __DOMImplementation.createDocumentType("HTML", 
??????????????????????????????????????????????????????????? "-//W3C//DTD HTML 4.01//EN", 
??????????????????????????????????????????????????????????? "http://www.w3.org/TR/html4/strict.dtd")
??? #params required for the createDocument method on DOMImplementation??? 
??? #DOMImplementation.createDocument(namespaceUri, qualifiedName, doctype)
??????? 
??? #get a default document with the root element initialised to <root>
??? def getDOMDocument(self):
??????? print("no args")
??????? return self.__DOMImplementation.createDocument(None, "root", None)

??? #allow a user to specify the namespaceUri node name
??? def getDOMDocument(self, namespaceUri=__defaultNamespaceUri):
??????? return self.__DOMImplementation.createDocument(namespaceUri, "root", None) 


??? #allow a user to specify the namespaceUri and a qualifiedName
??? def getDOMDocument(self, namespaceUri=__defaultNamespaceUri, qualifiedName=__defaultQualifiedName):
??????? return self.__DOMImplementation.createDocument(namespaceUri, qualifiedName, None) 

??? 
??? #allow a user to specify the namespaceUri, a qualifiedName and a doctype
??? def getDOMDocument(self, namespaceUri=__defaultNamespaceUri, qualifiedName=__defaultQualifiedName, docType=__defaultDoctype):
??????? print("3 args")
??????? return self.__DOMImplementation.createDocument(namespaceUri, qualifiedName, docType) 



#end NpDOMDocumentFactory

factory = NpDOMDocumentFactory()

print(factory.getDOMDocument().toxml())

when I pass this to python I get the following

lipska at ubuntu:~/python/dev/classes/com/nuldomain/xml$ python3.2 NpDOMDocumentFactory.py
3 args
<?xml version="1.0" ?><!DOCTYPE HTML? PUBLIC '-//W3C//DTD HTML 4.01//EN'? 'http://www.w3.org/TR/html4/strict.dtd'><root/>

I have absolutely no idea why a method that requires three arguments is being called when I intend the "no args" method to be called

I'm not asking for a critique of my code and I realise that something spooky
is going on with the doctype stuff I just need to understand why the 3 arg method is being called instead of the 

0 arg method as expected

Can someone enlighten me ... please, thanks


Lipska

From malaclypse2 at gmail.com  Wed Jul 25 17:26:56 2012
From: malaclypse2 at gmail.com (Jerry Hill)
Date: Wed, 25 Jul 2012 11:26:56 -0400
Subject: [Tutor] What on earth is happening here ???
In-Reply-To: <1343229118.88627.YahooMailNeo@web133002.mail.ir2.yahoo.com>
References: <1343229118.88627.YahooMailNeo@web133002.mail.ir2.yahoo.com>
Message-ID: <CADwdpyaQth23hYwpjuJ83BjaqBmyfkVB+A8KHy8LWuqK553sPw@mail.gmail.com>

On Wed, Jul 25, 2012 at 11:11 AM, Lipska TheCat
<lipskathekat at yahoo.co.uk> wrote:
>     def getDOMDocument(self):

This defines the method getDOMDocument().

>     def getDOMDocument(self, namespaceUri=__defaultNamespaceUri):

This defines the method getDOMDocument(), replacing the previous definition.

>     def getDOMDocument(self, namespaceUri=__defaultNamespaceUri, qualifiedName=__defaultQualifiedName):

This defines the method getDOMDocument() a third time, replacing the
second definition.


> I have absolutely no idea why a method that requires three arguments is being called when I intend the "no args" method to be called
>
> I'm not asking for a critique of my code and I realise that something spooky
> is going on with the doctype stuff I just need to understand why the 3 arg method is being called instead of the

In python there can only ever be one object bound to a particular
name.  You cannot have three different functions that share the same
name, but have different signatures.  Instead, define a single method
and use default arguments to supply the defaults if someone doesn't
pass in a value.

-- 
Jerry

From __peter__ at web.de  Wed Jul 25 18:27:06 2012
From: __peter__ at web.de (Peter Otten)
Date: Wed, 25 Jul 2012 18:27:06 +0200
Subject: [Tutor] Getting name of Widget for event.widget?
References: <500FCEE0.2080701@gmail.com>
Message-ID: <jup6oi$446$1@dough.gmane.org>

Leam Hall wrote:

> Note that this is for an on-line class so I'd appreciate pointers to
> what I need to read or think about more than the answer out right.
> 
> Using Python 3 on Linux, is there a way to get the name of a widget
> instead of numeric representations in event.widget?
> 
> What I tried were a few variations of:
> 
>      def small_frame_handler1(self, event):
>          print(event.widget, " clicked at ", event.x, event.y)
> 
> and got:
> 
>      .140937484.144713004  clicked at  350 39

The above is actually a name in Tcl, automatically generated by tkinter.
 
> Called by:
> 
>         frame1 = Frame(self, bg="blue")
>         frame1.grid(row=0, column=0, rowspan=1, columnspan=2, sticky=ALL)
>         frame1.bind("<Button-1>", self.small_frame_handler1)
> 
> I tried to send parameters in the "bind" but kept getting argument
> mis-matches. There are two frames and I'd like to be able to put the
> name of the frame in the print without having to have a separate method
> for each one. This is inside a class, if that makes a difference.

You can either provide a custom name (tkinter will still add a leading dot)

import tkinter as tk

root = tk.Tk()

def frame_handler(event):
    print(event.widget)

for row, color in enumerate(["red", "blue"]):
    frame = tk.Frame(root, bg=color, name=color + "_frame")
    frame.grid(row=row, column=0, sticky="nsew")
    frame.bind("<Button-1>", frame_handler)
    root.rowconfigure(row, weight=1)
root.columnconfigure(0, weight=1)
root.geometry("200x200+100+200")
root.mainloop()

set a custom attribute

...
def frame_handler(event):
    print(event.widget.my_name)

for row, color in enumerate(["red", "blue"]):
    frame = tk.Frame(root, bg=color)
    frame.my_name = color + "_frame"
    frame.grid(row=row, column=0, sticky="nsew")
    frame.bind("<Button-1>", frame_handler)
    root.rowconfigure(row, weight=1)
...

or use a more general mechanism with functools.partial:

import tkinter as tk
from functools import partial

root = tk.Tk()

def frame_handler(event, widget_name):
    print(widget_name)

for row, color in enumerate(["red", "blue"]):
    frame = tk.Frame(root, bg=color)
    frame.grid(row=row, column=0, sticky="nsew")
    frame.bind("<Button-1>", partial(frame_handler, widget_name=color + 
"_frame"))
...



From ramit.prasad at jpmorgan.com  Wed Jul 25 19:53:35 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Wed, 25 Jul 2012 17:53:35 +0000
Subject: [Tutor] What on earth is happening here ???
In-Reply-To: <1343229118.88627.YahooMailNeo@web133002.mail.ir2.yahoo.com>
References: <1343229118.88627.YahooMailNeo@web133002.mail.ir2.yahoo.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741658C242@SCACMX008.exchad.jpmchase.net>

> I have a class
> The point of this class is to provide an uncluttered interface
> to Document Creation
> 
> 
> from xml.dom.minidom import getDOMImplementation
> 
> class NpDOMDocumentFactory:
> 
>     """A DOM Document Factory convenience class"""
> 
> 
> 
>     # make these private "by convention"
>     __DOMImplementation = getDOMImplementation()
>     __defaultNamespaceUri = "http://nuldomain.com/"
>     __defaultQualifiedName = "root"
>     __defaultDoctype = __DOMImplementation.createDocumentType("HTML",
>                                                             "-//W3C//DTD HTML
> 4.01//EN",

Trying to make things "private" is a throwback to Java. In Python
the tendency is to just leave everything "public". And "private" is with
one underbar/underscore.

> 
> "http://www.w3.org/TR/html4/strict.dtd")
>     #params required for the createDocument method on DOMImplementation
>     #DOMImplementation.createDocument(namespaceUri, qualifiedName, doctype)
> 
>     #get a default document with the root element initialised to <root>
>     def getDOMDocument(self):
>         print("no args")
>         return self.__DOMImplementation.createDocument(None, "root", None)
> 
>     #allow a user to specify the namespaceUri node name
>     def getDOMDocument(self, namespaceUri=__defaultNamespaceUri):
>         return self.__DOMImplementation.createDocument(namespaceUri, "root",
> None)
>
> 
>     #allow a user to specify the namespaceUri and a qualifiedName
>     def getDOMDocument(self, namespaceUri=__defaultNamespaceUri,
> qualifiedName=__defaultQualifiedName):
>         return self.__DOMImplementation.createDocument(namespaceUri,
> qualifiedName, None)
> 
> 
>     #allow a user to specify the namespaceUri, a qualifiedName and a doctype
>     def getDOMDocument(self, namespaceUri=__defaultNamespaceUri,
> qualifiedName=__defaultQualifiedName, docType=__defaultDoctype):
>         print("3 args")
>         return self.__DOMImplementation.createDocument(namespaceUri,
> qualifiedName, docType)
> 
> 
> 
> #end NpDOMDocumentFactory
> 
> factory = NpDOMDocumentFactory()
> 
> print(factory.getDOMDocument().toxml())
> 
> when I pass this to python I get the following
> 
> lipska at ubuntu:~/python/dev/classes/com/nuldomain/xml$ python3.2
> NpDOMDocumentFactory.py
> 3 args
> <?xml version="1.0" ?><!DOCTYPE HTML  PUBLIC '-//W3C//DTD HTML 4.01//EN'
> 'http://www.w3.org/TR/html4/strict.dtd'><root/>
> 
> I have absolutely no idea why a method that requires three arguments is being
> called when I intend the "no args" method to be called
>

Python does not have polymorphism in the same manner as Java.
Instead of creating different methods with a different number of
arguments, you create one method with "optional" arguments, i.e.
"keyword" arguments. 

def function( something=None ):
	print(something)

This creates a function where the argument 'something' is optional.
If nothing is passed in, then something will default to None. If 
something is passed in as either a positional or keyword argument--
func('test') or func(something='test')--then the value of something
will be whatever is passed in.

Now applying that to your method I see you have three different
return statements.

return self.__DOMImplementation.createDocument(namespaceUri, 
    qualifiedName, docType)
return self.__DOMImplementation.createDocument(namespaceUri,
    qualifiedName, None)
return self.__DOMImplementation.createDocument(namespaceUri, 
    "root", None)


So I will use the method definition.

def getDOMDocument(self, namespaceUri=__defaultNamespaceUri,
 qualifiedName=__defaultQualifiedName, docType=__defaultDoctype):
         return self.__DOMImplementation.createDocument(namespaceUri,
             qualifiedName, docType)


Now in order to get the same returns as above I will call 
it with the following options.

self.getDOMDocument(namespace_url, qualified_name, doc_type )
self.getDOMDocument(namespace_url, qualified_name, docType=None )
self.getDOMDocument(namespace_url, docType=None, qualifiedName="root" )


The last line can also be written entirely as positional arguments.
self.getDOMDocument(namespace_url, "root", None )

Or if you want to use all the default arguments
self.getDOMDocument()



Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From alan.gauld at btinternet.com  Wed Jul 25 22:02:47 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 25 Jul 2012 21:02:47 +0100
Subject: [Tutor] What on earth is happening here ???
In-Reply-To: <1343229118.88627.YahooMailNeo@web133002.mail.ir2.yahoo.com>
References: <1343229118.88627.YahooMailNeo@web133002.mail.ir2.yahoo.com>
Message-ID: <jupjd6$e9n$1@dough.gmane.org>

On 25/07/12 16:11, Lipska TheCat wrote:

> from xml.dom.minidom import getDOMImplementation

I'd probably start by saying that I suspect elemtTree will be easier to 
use than minidom, but if you must....


> class NpDOMDocumentFactory:
>      # make these private "by convention"
>      __DOMImplementation = getDOMImplementation()
>      __defaultNamespaceUri = "http://nuldomain.com/"
>      __defaultQualifiedName = "root"
>      __defaultDoctype = __DOMImplementation.createDocumentType("HTML",

Its not C++ or Java, Python doesn't usually need "private" definitions...


>      def getDOMDocument(self):
>      def getDOMDocument(self, namespaceUri=__defaultNamespaceUri):
>      def getDOMDocument(self, namespaceUri=__defaultNamespaceUri, qualifiedName=__defaultQualifiedName):
 >      def getDOMDocument(self,
                           namespaceUri=__defaultNamespaceUri,
                           qualifiedName=__defaultQualifiedName,
                           docType=__defaultDoctype):

These definitions all define the same object. There is no function 
overloading in Python. You can often fake it with suitable use of 
default arguments. This looks likely here.

If you can't use defaulted values you can write helper functions and 
call the main method with a tuple of arguments and switch on the length 
of tuple.

Or you can use the argument expansion mechanism to pass unlimited 
numbers of arguments, or a set of keyword arguments. Lots of options.

> ...I just need to understand why the 3 arg method is being called
 > instead of the 0 arg method as expected

Because in Python the function name alone is the identifier and so the 
last version defined is the version used.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/


From steve at pearwood.info  Thu Jul 26 03:34:21 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 26 Jul 2012 11:34:21 +1000
Subject: [Tutor] What on earth is happening here ???
In-Reply-To: <jupjd6$e9n$1@dough.gmane.org>
References: <1343229118.88627.YahooMailNeo@web133002.mail.ir2.yahoo.com>
	<jupjd6$e9n$1@dough.gmane.org>
Message-ID: <50109E9D.8000303@pearwood.info>

Alan Gauld wrote:

>>      def getDOMDocument(self):
>>      def getDOMDocument(self, namespaceUri=__defaultNamespaceUri):
>>      def getDOMDocument(self, namespaceUri=__defaultNamespaceUri, 
>> qualifiedName=__defaultQualifiedName):
>  >      def getDOMDocument(self,
>                           namespaceUri=__defaultNamespaceUri,
>                           qualifiedName=__defaultQualifiedName,
>                           docType=__defaultDoctype):
> 
> These definitions all define the same object. There is no function 
> overloading in Python. You can often fake it with suitable use of 
> default arguments. This looks likely here.

Another alternative is the generics implementation from pkgutil:

from pkgutil import simplegeneric

See the source code for details -- simplegeneric is public but undocumented. 
 From time to time people make noise about documenting it and moving it into 
its own module, but this is volunteer-driven and it hasn't happened yet.



-- 
Steven


From steve at pearwood.info  Thu Jul 26 03:29:33 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 26 Jul 2012 11:29:33 +1000
Subject: [Tutor] Private attributes [was Re: What on earth is happening here
 ???]
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF4741658C242@SCACMX008.exchad.jpmchase.net>
References: <1343229118.88627.YahooMailNeo@web133002.mail.ir2.yahoo.com>
	<5B80DD153D7D744689F57F4FB69AF4741658C242@SCACMX008.exchad.jpmchase.net>
Message-ID: <50109D7D.8030800@pearwood.info>

Prasad, Ramit wrote:

> Trying to make things "private" is a throwback to Java. In Python
> the tendency is to just leave everything "public". And "private" is with
> one underbar/underscore.

Yes, and no.

It is true that Python encourages a "consenting adults" philosophy, where 
private attributes are not encouraged. But they're not exactly *discouraged* 
either.

In my opinion, private attributes are best used to clearly mark methods or 
functions as *implementation details*, not for access control. Python will 
give you no help whatsoever if you want to restrict access to a method or 
other object -- there are no hidden or secret names in pure Python code.

But for the purpose of marking methods as private in the sense of "not part of 
the public API", the single leading underscore convention is more than strong 
enough. Unless otherwise documented as safe to use[1], a single leading 
underscore means "if you use this, you're going to have a bad time".

Double leading underscores are different. They do double duty as marking the 
method as private, and to tell the compiler to mangle their name so as 
(almost) avoid *accidental* name clashes in subclasses. Since the caller knows 
the name mangling algorithm, Python cannot prevent deliberate name clashes. 
Nor should it -- again, the Python philosophy here is that we're all adults, 
and if a subclass wants to override a private method, who are we to say it 
shouldn't?

In general, I recommend that beginners avoid double leading underscore methods 
until they are no longer beginners, because (1) they can be confusing to 
people still trying to learn the language; (2) you aren't going to need them; 
and (3) if you do need them, you can add them in later.



[1] E.g. the collections.namedtuple API defines a number of single underscore 
names as public. They have a single underscore to avoid clashing with field 
names the caller may define.



-- 
Steven

From bouncingcats at gmail.com  Thu Jul 26 15:48:58 2012
From: bouncingcats at gmail.com (David)
Date: Thu, 26 Jul 2012 23:48:58 +1000
Subject: [Tutor] How to use introspection to discover parameters?
Message-ID: <CAMPXz=qrOZ-KuTE9ZX04kUEU3u8yA2-nFZYK3HnLZY5PuvFswQ@mail.gmail.com>

Please help me understand the following (I have python 2.6.2) ...

Because I want to write nice docstrings, I read PEP257 where it says:
"The one-line docstring should NOT be a "signature" reiterating the
function/method parameters (which can be obtained by introspection)."

I am understanding this sentence to mean: don't put function parameters in my
own docstrings, because they are readily obtained some other way. This other
way is apparently called "introspection", but how to actually do it is a
mystery that I cannot find documented. If the following guess is wrong, I will
be grateful if someone would just show a simple example of how this is done.

If I was to attempt a guess: does it just mean using the help() builtin in the
interpreter? I know about help() from books and tutorials. In the packaged html
python docs though, use of help() is hardly described and never associated with
the word "introspection".

My reasoning for this guess is explained in the rest of this email, which sets
out what I have tried before asking here:

I searched the installed html docs for "introspection".
I disregard most hits as too specific, except these:

	On Page 2. "Defining new types" it says:
	"An application can use the introspection API ..."

	and under Glossary it says, in the docstring entry:
	"Since [the docstring] is available via introspection, it is the
	canonical place for documentation of the object."

So I find no clue there how to do "introspection" to obtain function/method
parameters, except that there is an API for it, somewhere.

I googled and found various, eg:
http://www.ibm.com/developerworks/library/l-pyint/index.html
It also does not explain how to do "introspection" to obtain function/method
parameters.

Next, I tried looking somewhere arbitrary in the python libraries hoping to see
PEP257-compliant practice.I think that I need to find some code that is written
in python, not C, because PEP257 also says: "docstring [with parameters] is
only appropriate for C functions (such as built-ins), where introspection is
not possible."

I arbitrarily choose the "logging" module:

>>> import logging
>>> logging.__file__
'/usr/lib/python2.6/logging/__init__.pyc'

> ls -1 /usr/lib/python2.6/logging/*.py
/usr/lib/python2.6/logging/config.py
/usr/lib/python2.6/logging/handlers.py
/usr/lib/python2.6/logging/__init__.py

so I assume the "logging" module is not written in C, and therefore will
contain suitable examples. And I try this:

>>> help(logging.log)
Help on function log in module logging:

log(level, msg, *args, **kwargs)
    Log 'msg % args' with the integer severity 'level' on the root logger.

>>> logging.log.__doc__
"\n    Log 'msg % args' with the integer severity 'level' on the root
logger.\n    "
>>>

So I look at that and guess: is PEP257 actually attempting to say
"dont reiterate the function/method parameters in the docstring, because help()
displays them in line 3 of its output".
If it does mean this, it would be a lot clearer if it just said so!

A related question:
help() seems to do the introspection for me. Does python allow me to do it in
my own code? Specifically, how might I write my own function to mimic line 3 of
help(), appearing like so:

>>> my_function(logging.log)
"log(level, msg, *args, **kwargs)"

If I knew how to do that, it might help me understand how to do "introspection"
better.

One last thing. When looking for answers, I found this page which seems related:
http://stackoverflow.com/questions/2536879/python-introspection-how-to-get-varnames-of-class-methods

There is a comment by S Lott who possibly is the author of "Building Skills In
Python": "Please ... explain why you need introspection and why you can't
simply read the source."

Ummm ... I'm not sure what to make of that!! Because I've read that this
instrospection thing is supposed to be an important feature of Python.
Clarifications welcome :)

From malaclypse2 at gmail.com  Thu Jul 26 16:39:36 2012
From: malaclypse2 at gmail.com (Jerry Hill)
Date: Thu, 26 Jul 2012 10:39:36 -0400
Subject: [Tutor] How to use introspection to discover parameters?
In-Reply-To: <CAMPXz=qrOZ-KuTE9ZX04kUEU3u8yA2-nFZYK3HnLZY5PuvFswQ@mail.gmail.com>
References: <CAMPXz=qrOZ-KuTE9ZX04kUEU3u8yA2-nFZYK3HnLZY5PuvFswQ@mail.gmail.com>
Message-ID: <CADwdpyacaNAX0ST7bmjPNND0veh8mZf2rYZFCRJ0DOKK8EYkVw@mail.gmail.com>

On Thu, Jul 26, 2012 at 9:48 AM, David <bouncingcats at gmail.com> wrote:
> A related question:
> help() seems to do the introspection for me. Does python allow me to do it in
> my own code? Specifically, how might I write my own function to mimic line 3 of
> help(), appearing like so:
>
>>>> my_function(logging.log)
> "log(level, msg, *args, **kwargs)"
>
> If I knew how to do that, it might help me understand how to do "introspection"
> better.

The inspect module probably has all the tools you need:
http://docs.python.org/library/inspect.html .  In particular,
inspect.getargspec() will get you the argument specification of a
function, like this:

>>> inspect.getargspec(logging.log)
ArgSpec(args=['level', 'msg'], varargs='args', keywords='kwargs', defaults=None)
>>>

Then you just have to format those results the way you like.

Also, the help object itself is written in python.  You can look at
the source in pydoc.py
(http://hg.python.org/cpython/file/2.7/Lib/pydoc.py) to see exactly
what it does.

-- 
Jerry

From steve at pearwood.info  Thu Jul 26 17:02:33 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 27 Jul 2012 01:02:33 +1000
Subject: [Tutor] How to use introspection to discover parameters?
In-Reply-To: <CAMPXz=qrOZ-KuTE9ZX04kUEU3u8yA2-nFZYK3HnLZY5PuvFswQ@mail.gmail.com>
References: <CAMPXz=qrOZ-KuTE9ZX04kUEU3u8yA2-nFZYK3HnLZY5PuvFswQ@mail.gmail.com>
Message-ID: <50115C09.8020508@pearwood.info>

David wrote:
> Please help me understand the following (I have python 2.6.2) ...
> 
> Because I want to write nice docstrings, I read PEP257 where it says:
> "The one-line docstring should NOT be a "signature" reiterating the
> function/method parameters (which can be obtained by introspection)."

As a general rule, yes, that is right -- your doc strings should not merely 
repeat the function signature in the first line. The help() function 
automatically determines the signature for you. For example, if I define this 
function with a docstring, and then call help() on it:

py> def spam(n):
...     """Return Spam Spam Spam Glorious SPAM!!!"""
...     return "spam"*n
...
py> help(spam)


I get this help text displayed:


     Help on function spam in module __main__:

     spam(n)
         Return Spam Spam Spam Glorious SPAM!!!


Notice that the function signature, "spam(n)", is displayed by help() even 
though I didn't include it in the docstring. That is because help() uses 
introspection to discover the signature.

See more detail below.


> I am understanding this sentence to mean: don't put function parameters in my
> own docstrings, because they are readily obtained some other way. This other
> way is apparently called "introspection", but how to actually do it is a
> mystery that I cannot find documented. If the following guess is wrong, I will
> be grateful if someone would just show a simple example of how this is done.

You don't have to do a thing unless you are writing your own help system. The 
standard help() function already does it.

In case it isn't obvious, "introspection" in programming refers to the ability 
of your program to inspect its own components (in this case, functions) to 
find out what they are and what they can do. Python has a couple of built-in 
functions for introspection, plus an entire module of extra functions.

Built-ins include: dir, vars, globals, locals, hasattr, type, help

[Note: technically, help is not a built-in. It is added to the built-ins at 
runtime. Some systems may disable that, e.g. on embedded systems with limited 
memory.]

Plus the inspect module, which contains many other advanced introspection 
tools. You can read about the inspect module here:

http://docs.python.org/library/inspect.html
http://hg.python.org/cpython/file/2.7/Lib/inspect.py


[...]
> So I find no clue there how to do "introspection" to obtain function/method
> parameters, except that there is an API for it, somewhere.


Using the same function spam() I defined above:

py> import inspect
py> inspect.getargspec(spam)
ArgSpec(args=['n'], varargs=None, keywords=None, defaults=None)


And here is one way to use it to print the function signature. First we grab 
the argument spec, then we concatenate it after the function name.

py> spec = inspect.getargspec(spam)
py> spam.__name__ + inspect.formatargspec(*spec)
'spam(n)'


> So I look at that and guess: is PEP257 actually attempting to say
> "dont reiterate the function/method parameters in the docstring, because help()
> displays them in line 3 of its output".

Right!

> If it does mean this, it would be a lot clearer if it just said so!

Perhaps so. You might consider putting in a request for a documentation change 
on the bug tracker.


> A related question:
> help() seems to do the introspection for me. Does python allow me to do it in
> my own code? Specifically, how might I write my own function to mimic line 3 of
> help(), appearing like so:
> 
>>>> my_function(logging.log)
> "log(level, msg, *args, **kwargs)"
> 
> If I knew how to do that, it might help me understand how to do "introspection"
> better.


The help() function is written in pure Python. It's a big, powerful function, 
but if you want to read it, you can do so:

This is where it is added to the built-ins:

http://hg.python.org/cpython/file/2.7/Lib/site.py

And this is where the actual help functionality is made:

http://hg.python.org/cpython/file/2.7/Lib/pydoc.py


> One last thing. When looking for answers, I found this page which seems related:
> http://stackoverflow.com/questions/2536879/python-introspection-how-to-get-varnames-of-class-methods
> 
> There is a comment by S Lott who possibly is the author of "Building Skills In
> Python": "Please ... explain why you need introspection and why you can't
> simply read the source."

What an asinine comment. Where shall we start?

1) Source code is not always available. Even when available, it can sometimes 
be covered by a license which makes it professional suicide to read the code, 
because then anything you write afterwards may be legally deemed to be 
"copied" from the source code you read.

2) Even when you have the source code to read, reading the source can be much, 
much harder than using introspection. The pydoc module is over 2000 lines of 
code, the decimal module is over 4000 lines of code. I shudder to think how 
many thousands of lines of code, spread over who knows how many files, the 
numpy library would be. Ten? Twenty? And it is written in a mix of Python, C 
and Fortran. I wouldn't even know where to begin reading the source code for 
information on (say) numpy.ufunc. But with introspection, it's easy:

py> dir(numpy.ufunc)
['__call__', '__class__', '__delattr__', '__doc__', '__format__', 
'__getattribute__', '__hash__', '__init__', '__name__', '__new__', 
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', 
'__str__', '__subclasshook__', 'accumulate', 'identity', 'nargs', 'nin', 
'nout', 'ntypes', 'outer', 'reduce', 'reduceat', 'signature', 'types']

And now I know what methods and attributes ufunc has.

3) Even if the source code is short and sweet and easy to understand, this is 
not always useful. You still have to get that knowledge into your program, 
otherwise it is useless.

Consider our friend the help() function again. You can call it on a class, and 
it will show you help for each method in the class, one after the other. How 
does S Lott think that help() knows what methods the class has? Does he think 
that the person who programmed help() simply wrote up a giant list of every 
existing class and their methods, generated from reading the source code?

Of course not. That would be idiotic. Not only is it time-consuming and 
wasteful, not only does it duplicate work already done, but it is fragile: the 
slightest change to the class, and the list of methods may become out of date.

Instead, help() uses introspection to inspect the class programmatically, 
generate a list of methods on the fly, and display help for each one. You 
simply cannot do anything even remotely similar by reading the source code.


> Ummm ... I'm not sure what to make of that!! Because I've read that this
> instrospection thing is supposed to be an important feature of Python.


If you are ever unlucky enough to use a programming language without 
introspection, you'll soon come to miss it.

Introspection is a powerful technique. For example, I'm currently writing an 
experimental module which uses introspection to *automatically* create 
__repr__ methods for classes. (Well, mostly automatically -- at worst, the 
caller can drop a few extra hints.)





-- 
Steven


From ramit.prasad at jpmorgan.com  Thu Jul 26 17:26:16 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Thu, 26 Jul 2012 15:26:16 +0000
Subject: [Tutor] Private attributes [was Re: What on earth is happening
 here ???]
In-Reply-To: <50109D7D.8030800@pearwood.info>
References: <1343229118.88627.YahooMailNeo@web133002.mail.ir2.yahoo.com>
	<5B80DD153D7D744689F57F4FB69AF4741658C242@SCACMX008.exchad.jpmchase.net>
	<50109D7D.8030800@pearwood.info>
Message-ID: <5B80DD153D7D744689F57F4FB69AF4741658DD0F@SCACMX008.exchad.jpmchase.net>

> > Trying to make things "private" is a throwback to Java. In Python
> > the tendency is to just leave everything "public". And "private" is with
> > one underbar/underscore.
[snip]
> In general, I recommend that beginners avoid double leading underscore methods
> until they are no longer beginners, because (1) they can be confusing to
> people still trying to learn the language; (2) you aren't going to need them;
> and (3) if you do need them, you can add them in later.

Yeah, that is what I was trying to get across.

Ramit
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From bouncingcats at gmail.com  Thu Jul 26 17:43:45 2012
From: bouncingcats at gmail.com (David)
Date: Fri, 27 Jul 2012 01:43:45 +1000
Subject: [Tutor] How to use introspection to discover parameters?
In-Reply-To: <50115C09.8020508@pearwood.info>
References: <CAMPXz=qrOZ-KuTE9ZX04kUEU3u8yA2-nFZYK3HnLZY5PuvFswQ@mail.gmail.com>
	<50115C09.8020508@pearwood.info>
Message-ID: <CAMPXz=pJ_6QY39bY=C8hRcBasjWw9uGfbndsSw=NnbCP20mtPg@mail.gmail.com>

On 27/07/2012, Jerry Hill <malaclypse2 at gmail.com> wrote:
...
>>>> inspect.getargspec(logging.log)
> ArgSpec(args=['level', 'msg'], varargs='args', keywords='kwargs',
> defaults=None)
...
> Also, the help object itself is written in python.  You can look at
> the source in pydoc.py

On 27/07/2012, Steven D'Aprano <steve at pearwood.info> wrote:
>
> find out what they are and what they can do. Python has a couple of built-in
> functions for introspection, plus an entire module of extra functions.
> Built-ins include: dir, vars, globals, locals, hasattr, type, help
...
> Plus the inspect module, which contains many other advanced introspection
...
> This is where it is added to the built-ins:
...
> And this is where the actual help functionality is made:
...
> 1) Source code is not always available. Even when available, it can
...
> 2) Even when you have the source code to read, reading the source can be
...
> 3) Even if the source code is short and sweet and easy to understand, this
...

Thanks for the great and comprehensive answers! I understand much better it now.
When I saw that help() was a builtin, I assumed it was written in C. So it
helped that you pointed me to pydoc and inspect.
In pydoc I found this:

        if inspect.isfunction(object):
            args, varargs, varkw, defaults = inspect.getargspec(object)

I didn't realise that so much of a "builtin" help() would be visible to me
in *.py files. I naively assumed "builtin" meant compiled-from-C.

> You might consider putting in a request for a documentation
> change on the bug tracker.

Will do.

Assistance much appreciated!

From bfishbein79 at gmail.com  Thu Jul 26 20:06:21 2012
From: bfishbein79 at gmail.com (Benjamin Fishbein)
Date: Thu, 26 Jul 2012 13:06:21 -0500
Subject: [Tutor] going rate for python tutoring?
Message-ID: <37C29E8C-2807-4FF0-8913-A7935F27EBD6@gmail.com>

Hello,
I'm trying to learn "posting" and am having a very hard time at it. I'm going to hire someone to give me a private lesson, so I put up a notice on Northwestern's CS bulletin board--probably a student there'll be able to help.
How much money should I offer? I haven't the foggiest notion what the going rate is for private computer tutoring.
Thanks,
Ben


From alan.gauld at btinternet.com  Fri Jul 27 02:07:26 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 27 Jul 2012 01:07:26 +0100
Subject: [Tutor] going rate for python tutoring?
In-Reply-To: <37C29E8C-2807-4FF0-8913-A7935F27EBD6@gmail.com>
References: <37C29E8C-2807-4FF0-8913-A7935F27EBD6@gmail.com>
Message-ID: <jusm3u$qnu$1@dough.gmane.org>

On 26/07/12 19:06, Benjamin Fishbein wrote:

> I'm trying to learn "posting"

Can you explain what 'posting' is?
I've no idea what you mean. Are you talking about web transactions?
Or using message boards? Or something else?

>  I haven't the foggiest notion what the going rate is for
 > private computer tutoring.

Me neither but are there any other adverts you could compare to?
Meantime you could just ask your questions here for free!

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/


From bfishbein79 at gmail.com  Fri Jul 27 23:47:41 2012
From: bfishbein79 at gmail.com (Benjamin Fishbein)
Date: Fri, 27 Jul 2012 16:47:41 -0500
Subject: [Tutor] getting results from encoded data i sent to website
Message-ID: <D5754493-625B-42EE-9050-BB228A6308EB@gmail.com>

I'm trying to learn encoding, and I've had a bit of success. I managed to input something to the text box on Python's home page and I got the result I expected. But I can't figure out why this program I wrote isn't working; I think I did everything right.
I'm inputting the isbn of Cat in the Hat and trying to get the prices for it on half.com, the Ebay site for books.
Here's my code:

### This program tries to check the prices for "The Cat in the Hat" on http://half.com

#import modules
import urllib
import urllib2

#make variables
action="http://sell.half.ebay.com/ws/eBayISAPI.dll?HalfInstantSaleHub&action=search"
name="isbnupcnumbers"
cat_in_hat_isbn="039480001X"

#get stuff encoded
data={name:cat_in_hat_isbn}
encoded_data=urllib.urlencode(data)

#send encoded stuff to website
content=urllib2.urlopen(action, encoded_data)

#look at results
print content.readlines()

#next I'll parse out the data I want; that I already know how to do.

The program seems to be working until "content" is printed. Then it's a lot of strange mumbo-jumbo, quite different from the results I get if I put in the isbn manually.
Do you know where I went wrong?
Thanks,
Ben


From ramit.prasad at jpmorgan.com  Sat Jul 28 00:33:54 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Fri, 27 Jul 2012 22:33:54 +0000
Subject: [Tutor] getting results from encoded data i sent to website
In-Reply-To: <D5754493-625B-42EE-9050-BB228A6308EB@gmail.com>
References: <D5754493-625B-42EE-9050-BB228A6308EB@gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416590604@SCACMX008.exchad.jpmchase.net>

> I'm trying to learn encoding, and I've had a bit of success. I managed to
> input something to the text box on Python's home page and I got the result I
> expected. But I can't figure out why this program I wrote isn't working; I
> think I did everything right.
> I'm inputting the isbn of Cat in the Hat and trying to get the prices for it
> on half.com, the Ebay site for books.
> Here's my code:
> 
> ### This program tries to check the prices for "The Cat in the Hat" on
> http://half.com
> 
> #import modules
> import urllib
> import urllib2
> 
> #make variables
> action="http://sell.half.ebay.com/ws/eBayISAPI.dll?HalfInstantSaleHub&action=s
> earch"
> name="isbnupcnumbers"
> cat_in_hat_isbn="039480001X"
> 
> #get stuff encoded
> data={name:cat_in_hat_isbn}
> encoded_data=urllib.urlencode(data)
> 
> #send encoded stuff to website
> content=urllib2.urlopen(action, encoded_data)
> 
> #look at results
> print content.readlines()
> 
> #next I'll parse out the data I want; that I already know how to do.
> 
> The program seems to be working until "content" is printed. Then it's a lot of
> strange mumbo-jumbo, quite different from the results I get if I put in the
> isbn manually.

What do you mean by "mumbo-jumbo"? Can you provide a small sample?
How do you "put in the isbn manually"?

> Do you know where I went wrong?

I ran the program and printed the first 5-10 lines. It seemed like 
valid HTML code as I skimmed it. What output did you expect?


Ramit
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From alan.gauld at btinternet.com  Sat Jul 28 02:35:10 2012
From: alan.gauld at btinternet.com (ALAN GAULD)
Date: Sat, 28 Jul 2012 01:35:10 +0100 (BST)
Subject: [Tutor] going rate for python tutoring?
In-Reply-To: <0344A430-A063-4B47-8F86-E0E49CECE03D@gmail.com>
References: <37C29E8C-2807-4FF0-8913-A7935F27EBD6@gmail.com>
	<jusm3u$qnu$1@dough.gmane.org>
	<0344A430-A063-4B47-8F86-E0E49CECE03D@gmail.com>
Message-ID: <1343435710.34417.YahooMailNeo@web87703.mail.ir2.yahoo.com>

CC'ing the list to broaden the scope.

?
Alan Gauld
Author of the Learn To Program website
http://www.alan-g.me.uk/



>________________________________
> From: Benjamin Fishbein <bfishbein79 at gmail.com>
>To: Alan Gauld <alan.gauld at btinternet.com> 
>Sent: Friday, 27 July 2012, 20:53
>Subject: Re: [Tutor] going rate for python tutoring?
> 
>By posting I mean entering text to a text box in a website. For example, entering a username in the username box, then going to the password box and entering a password, then returning the text from the resulting page. The tutorials online about how to do this are quite confusing.
>OK, do you understand how web pages work. In particular do 

you understand http and CGI protocols and mechanisms? 

If so what part of the process are you having trouble with?

Can you read the page and locate the field you want to access for example?

Beautiful Soup may be helpful here if you haven't already discovered it. 
Although there are modules in the Python library that you can use too.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120728/4b482fd9/attachment.html>

From TTabern at ddti.net  Sat Jul 28 03:38:09 2012
From: TTabern at ddti.net (Todd Tabern)
Date: Sat, 28 Jul 2012 01:38:09 +0000
Subject: [Tutor] Search and replace text in XML file?
Message-ID: <m8nuw5kdbnncxl2wi08m7qj4.1343439421277@email.android.com>

I'm looking to search an entire XML file for specific text and replace that text, while maintaining the structure of the XML file. The text occurs within multiple nodes throughout the file. 
I basically need to replace every occurrence C:\Program Files with C:\Program Files (x86), regardless of location. For example, that text appears within: 
<URL>C:\Program Files\\Map Data\Road_Centerlines.shp</URL> 
and also within: 
<RoutingIndexPathName>C:\Program Files\Templates\RoadNetwork.rtx</RoutingIndexPathName> 
...among others. 
I've tried some non-python methods and they all ruined the XML structure. I've been Google searching all day and can only seem to find solutions that look for a specific node and replace the whole string between the tags. 
I've been looking at using minidom to achieve this but I just can't seem to figure out the right method. 
My end goal, once I have working code, is to compile an exe that can work on machines without python, allowing a user can click in order to perform the XML modification. 
Thanks in advance. 

From bodsda at googlemail.com  Sat Jul 28 09:22:18 2012
From: bodsda at googlemail.com (Bod Soutar)
Date: Sat, 28 Jul 2012 08:22:18 +0100
Subject: [Tutor] Search and replace text in XML file?
In-Reply-To: <m8nuw5kdbnncxl2wi08m7qj4.1343439421277@email.android.com>
References: <m8nuw5kdbnncxl2wi08m7qj4.1343439421277@email.android.com>
Message-ID: <CAG6Bxke8pRdQGhRgZEE5TA3HmSED=_BnE9d4vYE8nH6shD47Gg@mail.gmail.com>

On Jul 28, 2012 2:39 AM, "Todd Tabern" <TTabern at ddti.net> wrote:
>
> I'm looking to search an entire XML file for specific text and replace
that text, while maintaining the structure of the XML file. The text occurs
within multiple nodes throughout the file.
> I basically need to replace every occurrence C:\Program Files with
C:\Program Files (x86), regardless of location. For example, that text
appears within:
> <URL>C:\Program Files\\Map Data\Road_Centerlines.shp</URL>
> and also within:
> <RoutingIndexPathName>C:\Program
Files\Templates\RoadNetwork.rtx</RoutingIndexPathName>
> ...among others.
> I've tried some non-python methods and they all ruined the XML structure.
I've been Google searching all day and can only seem to find solutions that
look for a specific node and replace the whole string between the tags.
> I've been looking at using minidom to achieve this but I just can't seem
to figure out the right method.
> My end goal, once I have working code, is to compile an exe that can work
on machines without python, allowing a user can click in order to perform
the XML modification.
> Thanks in advance.

I'm not sure what you have tried already, but this should be as simple as
reading the file, replacing the strings and then writing the file. Because
you don't care about just a specific entry, you don't really need to be
concerned that its an XML file.

If you continue having formatting problems, send us a sample and the code
you've tried.

Bodsda
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120728/df1349d5/attachment-0001.html>

From alan.gauld at btinternet.com  Sat Jul 28 10:45:24 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 28 Jul 2012 09:45:24 +0100
Subject: [Tutor] Search and replace text in XML file?
In-Reply-To: <m8nuw5kdbnncxl2wi08m7qj4.1343439421277@email.android.com>
References: <m8nuw5kdbnncxl2wi08m7qj4.1343439421277@email.android.com>
Message-ID: <jv08r4$ace$1@dough.gmane.org>

On 28/07/12 02:38, Todd Tabern wrote:
> I'm looking to search an entire XML file for specific text and replace that text,
 > while maintaining the structure of the XML file.

Do you mean the physical layout of the file or the technical XML 
structure? I'm assuming its the latter? If it's the former just
use 'tidy' to reformat the file.

> I basically need to replace every occurrence C:\Program Files with C:\Program Files (x86),
 > regardless of location. ...
> I've tried some non-python methods and they all ruined the XML structure.

Can you give examples of what you tried? I'd have gone for sed for a job 
like this.

 > I've been Google searching all day and can only seem
> to find solutionsthat look for a specific node
 > and replace the whole string between the tags.

Because that's usually the requirement, but generally
you can extract the existing string and do the substitution
and then write back the new version.

> I've been looking at using minidom to achieve this

I'd consider element tree if you must do it via a parser, but it sounds 
like you don't need that, sed or simple string replacements should 
suffice here.

> compile an exe that can work on machines without python,

You can do that with Python but its not my favourite approach, you'd be 
better with a sed based solution. (You can get GNU sed for Windowss and 
it already is installed on MacOS/Linux boxes.)

HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/


From htdatcse at gmail.com  Sat Jul 28 11:34:36 2012
From: htdatcse at gmail.com (Dat Huynh)
Date: Sat, 28 Jul 2012 17:34:36 +0800
Subject: [Tutor] Encoding error when reading text files in Python 3
Message-ID: <CAPw=odj5GdiyOVrwF6FUgx27-rSjyvyGHkTOPQ2z=jmtePWD1g@mail.gmail.com>

Dear all,

I have written a simple application by Python to read data from text files.

Current I have both Python version 2.7.2 and Python 3.2.3 on my laptop.
I don't know why it does not run on Python version 3 while it runs
well on Python 2.

Could you please tell me how I can run it on python 3?
Following is my Python code.

 ------------------------------
   for subdir, dirs, files in os.walk(rootdir):

        for file in files:

            print("Processing [" +file +"]...\n" )

            f = open(rootdir+file, 'r')

            data = f.read()

            f.close()

            print(data)
------------------------------

This is the error message:

------------------------------
Traceback (most recent call last):

  File "/Users/dathuynh/Documents/workspace/PyTest/MyParser.py", line
53, in <module>

    main()

  File "/Users/dathuynh/Documents/workspace/PyTest/MyParser.py", line
20, in main

    data = f.read()

  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/encodings/ascii.py",
line 26, in decode

    return codecs.ascii_decode(input, self.errors)[0]

UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position
4980: ordinal not in range(128)
------------------------------

Thank you very much for your help.

Sincerely,
Dat Huynh.

From steve at pearwood.info  Sat Jul 28 12:09:28 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 28 Jul 2012 20:09:28 +1000
Subject: [Tutor] Encoding error when reading text files in Python 3
In-Reply-To: <CAPw=odj5GdiyOVrwF6FUgx27-rSjyvyGHkTOPQ2z=jmtePWD1g@mail.gmail.com>
References: <CAPw=odj5GdiyOVrwF6FUgx27-rSjyvyGHkTOPQ2z=jmtePWD1g@mail.gmail.com>
Message-ID: <5013BA58.1040404@pearwood.info>

Dat Huynh wrote:
> Dear all,
> 
> I have written a simple application by Python to read data from text files.
> 
> Current I have both Python version 2.7.2 and Python 3.2.3 on my laptop.
> I don't know why it does not run on Python version 3 while it runs
> well on Python 2.

Python 2 is more forgiving of beginner errors when dealing with text and 
bytes, but makes it harder to deal with text correctly.

Python 3 makes it easier to deal with text correctly, but is less forgiving.

When you read from a file in Python 2, it will give you *something*, even if 
it is the wrong thing. It will not give an decoding error, even if the text 
you are reading is not valid text. It will just give you junk bytes, sometimes 
known as moji-bake.

Python 3 no longer does that. It tells you when there is a problem, so you can 
fix it.


> Could you please tell me how I can run it on python 3?
> Following is my Python code.
> 
>  ------------------------------
>    for subdir, dirs, files in os.walk(rootdir):
>         for file in files:
>             print("Processing [" +file +"]...\n" )
>             f = open(rootdir+file, 'r')
>             data = f.read()
>             f.close()
>             print(data)
> ------------------------------
> 
> This is the error message:
[...]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position
> 4980: ordinal not in range(128)


This tells you that you are reading a non-ASCII file but haven't told Python 
what encoding to use, so by default Python uses ASCII.

Do you know what encoding the file is?

Do you understand about Unicode text and bytes? If not, I suggest you read 
this article:

http://www.joelonsoftware.com/articles/Unicode.html


In Python 3, you can either tell Python what encoding to use:

f = open(rootdir+file, 'r', encoding='utf8')  # for example

or you can set an error handler:

f = open(rootdir+file, 'r', errors='ignore')  # for example

or both

f = open(rootdir+file, 'r', encoding='ascii', errors='replace')


You can see the list of encodings and error handlers here:

http://docs.python.org/py3k/library/codecs.html


Unfortunately, Python 2 does not support this using the built-in open 
function. Instead, you have to uses codecs.open instead of the built-in open, 
like this:

import codecs
f = codecs.open(rootdir+file, 'r', encoding='utf8')  # for example

which fortunately works in both Python 2 or 3.


Or you can read the file in binary mode, and then decode it into text:

f = open(rootdir+file, 'rb')
data = f.read()
f.close()
text = data.decode('cp866', 'replace')
print(text)


If you don't know the encoding, you can try opening the file in Firefox or 
Internet Explorer and see if they can guess it, or you can use the chardet 
library in Python.

http://pypi.python.org/pypi/chardet

Or if you don't care about getting moji-bake, you can pretend that the file is 
encoded using Latin-1. That will pretty much read anything, although what it 
gives you may be junk.



-- 
Steven

From breamoreboy at yahoo.co.uk  Sat Jul 28 12:25:30 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 28 Jul 2012 11:25:30 +0100
Subject: [Tutor] Search and replace text in XML file?
In-Reply-To: <m8nuw5kdbnncxl2wi08m7qj4.1343439421277@email.android.com>
References: <m8nuw5kdbnncxl2wi08m7qj4.1343439421277@email.android.com>
Message-ID: <jv0emn$eda$1@dough.gmane.org>

On 28/07/2012 02:38, Todd Tabern wrote:
> I'm looking to search an entire XML file for specific text and replace that text, while maintaining the structure of the XML file. The text occurs within multiple nodes throughout the file.
> I basically need to replace every occurrence C:\Program Files with C:\Program Files (x86), regardless of location. For example, that text appears within:
> <URL>C:\Program Files\\Map Data\Road_Centerlines.shp</URL>
> and also within:
> <RoutingIndexPathName>C:\Program Files\Templates\RoadNetwork.rtx</RoutingIndexPathName>
> ...among others.
> I've tried some non-python methods and they all ruined the XML structure. I've been Google searching all day and can only seem to find solutions that look for a specific node and replace the whole string between the tags.
> I've been looking at using minidom to achieve this but I just can't seem to figure out the right method.
> My end goal, once I have working code, is to compile an exe that can work on machines without python, allowing a user can click in order to perform the XML modification.
> Thanks in advance.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

Did you really have to ask the same question on two separate Python 
mailing lists and only 15 minutes apart?

-- 
Cheers.

Mark Lawrence.


From htdatcse at gmail.com  Sat Jul 28 12:45:47 2012
From: htdatcse at gmail.com (Dat Huynh)
Date: Sat, 28 Jul 2012 18:45:47 +0800
Subject: [Tutor] Encoding error when reading text files in Python 3
In-Reply-To: <5013BA58.1040404@pearwood.info>
References: <CAPw=odj5GdiyOVrwF6FUgx27-rSjyvyGHkTOPQ2z=jmtePWD1g@mail.gmail.com>
	<5013BA58.1040404@pearwood.info>
Message-ID: <CAPw=odian5_MYMudR+OiaWstgoG9i+zDoSk-0rRy1arxif1G0g@mail.gmail.com>

I change my code and it runs on Python 3 now.

           f = open(rootdir+file, 'rb')
          data = f.read().decode('utf8', 'ignore')

Thank you very much.
Sincerely,
Dat.




On Sat, Jul 28, 2012 at 6:09 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> Dat Huynh wrote:
>>
>> Dear all,
>>
>> I have written a simple application by Python to read data from text
>> files.
>>
>> Current I have both Python version 2.7.2 and Python 3.2.3 on my laptop.
>> I don't know why it does not run on Python version 3 while it runs
>> well on Python 2.
>
>
> Python 2 is more forgiving of beginner errors when dealing with text and
> bytes, but makes it harder to deal with text correctly.
>
> Python 3 makes it easier to deal with text correctly, but is less forgiving.
>
> When you read from a file in Python 2, it will give you *something*, even if
> it is the wrong thing. It will not give an decoding error, even if the text
> you are reading is not valid text. It will just give you junk bytes,
> sometimes known as moji-bake.
>
> Python 3 no longer does that. It tells you when there is a problem, so you
> can fix it.
>
>
>
>> Could you please tell me how I can run it on python 3?
>> Following is my Python code.
>>
>>  ------------------------------
>>    for subdir, dirs, files in os.walk(rootdir):
>>         for file in files:
>>             print("Processing [" +file +"]...\n" )
>>             f = open(rootdir+file, 'r')
>>             data = f.read()
>>             f.close()
>>             print(data)
>> ------------------------------
>>
>> This is the error message:
>
> [...]
>
>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position
>> 4980: ordinal not in range(128)
>
>
>
> This tells you that you are reading a non-ASCII file but haven't told Python
> what encoding to use, so by default Python uses ASCII.
>
> Do you know what encoding the file is?
>
> Do you understand about Unicode text and bytes? If not, I suggest you read
> this article:
>
> http://www.joelonsoftware.com/articles/Unicode.html
>
>
> In Python 3, you can either tell Python what encoding to use:
>
> f = open(rootdir+file, 'r', encoding='utf8')  # for example
>
> or you can set an error handler:
>
> f = open(rootdir+file, 'r', errors='ignore')  # for example
>
> or both
>
> f = open(rootdir+file, 'r', encoding='ascii', errors='replace')
>
>
> You can see the list of encodings and error handlers here:
>
> http://docs.python.org/py3k/library/codecs.html
>
>
> Unfortunately, Python 2 does not support this using the built-in open
> function. Instead, you have to uses codecs.open instead of the built-in
> open, like this:
>
> import codecs
> f = codecs.open(rootdir+file, 'r', encoding='utf8')  # for example
>
> which fortunately works in both Python 2 or 3.
>
>
> Or you can read the file in binary mode, and then decode it into text:
>
> f = open(rootdir+file, 'rb')
> data = f.read()
> f.close()
> text = data.decode('cp866', 'replace')
> print(text)
>
>
> If you don't know the encoding, you can try opening the file in Firefox or
> Internet Explorer and see if they can guess it, or you can use the chardet
> library in Python.
>
> http://pypi.python.org/pypi/chardet
>
> Or if you don't care about getting moji-bake, you can pretend that the file
> is encoded using Latin-1. That will pretty much read anything, although what
> it gives you may be junk.
>
>
>
> --
> Steven
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From fal at libero.it  Sat Jul 28 17:12:57 2012
From: fal at libero.it (Francesco Loffredo)
Date: Sat, 28 Jul 2012 17:12:57 +0200
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
Message-ID: <50140179.2080306@libero.it>

Il 19/07/2012 19:33, PyProg PyProg ha scritto:
> Hi all,
>
> I would get a new list as:
>
> [(0, '3eA', 'Dupont', 'Juliette', '11.0/10.0', '4.0/5.0', '17.5/30.0',
> '3.0/5.0', '4.5/10.0', '35.5/60.0'), (1, '3eA', 'Pop', 'Iggy',
> '12.0/10.0', '3.5/5.0', '11.5/30.0', '4.0/5.0', '5.5/10.0',
> '7.5/10.0', '40.5/60.0')]
>
> ... from this one:
>
> [(0, '3eA', 'Dupont', 'Juliette', 0, 11.0, 10.0), (0, '3eA', 'Dupont',
> 'Juliette', 1, 4.0, 5.0), (0, '3eA', 'Dupont', 'Juliette', 2, 17.5,
> 30.0), (0, '3eA', 'Dupont', 'Juliette', 3, 3.0, 5.0), (0, '3eA',
> 'Dupont', 'Juliette', 4, 4.5, 10.0), (0, '3eA', 'Dupont', 'Juliette',
> 5, 35.5, 60.0), (1, '3eA', 'Pop', 'Iggy', 0, 12.0, 10.0), (1, '3eA',
> 'Pop', 'Iggy', 1, 3.5, 5.0), (1, '3eA', 'Pop', 'Iggy', 2, 11.5, 30.0),
> (1, '3eA', 'Pop', 'Iggy', 3, 4.0, 5.0), (1, '3eA', 'Pop', 'Iggy', 4,
> 5.5, 10.0), (1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0)]
>
> How to make that ? I'm looking for but for now I can't do it.
>
> Thanks in advance.
>
> a+
>
I had to study carefully your present and desired lists, and I 
understood what follows (please, next time explain !):
- each 7-tuple in your present list is a record for some measure 
relative to a person. Its fields are as follows:
     - field 0: code (I think you want that in growing order)
     - field 1: group code (could be a class or a group to which both of 
your example persons belong)
     - fields 2, 3: surname and name of the person
     - field 4: progressive number of the measure (these are in order 
already, but I think you want to enforce this) that you want to exclude 
from the output list while keeping the order
     - field 5, 6: numerator and denominator of a ratio that is the 
measure. you want the ratio to be written as a single string: "%s/%s" % 
field5, field6

Taking for granted this structure and my educated guesses about what you 
didn't tell us, here's my solution:

def flatten(inlist)
     """
       takes PyProg PyProg's current list and returns his/her desired one,
       given my guesses about the structure of inlist and the desired 
result.
     """
     tempdict = {}
     for item in inlist:
         if len(item) != 7:
             print "Item errato: \n", item
         id = tuple(item[:4])
         progr = item[4]
         payload = "%s/%s" % item[5:]
         if id in tempdict:
            tempdict[id].extend([(progr, payload)])
         else:
            tempdict[id] = [(progr, payload)]
     for item in tempdict:
         tempdict[item].sort() # so we set payloads in progressive 
order, if they aren't already
     # print "Temporary Dict: ", tempdict
     tmplist2 = []
     for item in tempdict:
         templist = []
         templist.extend(item)
         templist.extend(tempdict[item])
         tmplist2.append(tuple(templist))
     tmplist2.sort()# so we set IDs in order
     # print "Temporary List: ", tmplist2
     outlist = []
     for item in tmplist2:
         templist = []
         if isinstance(item, tuple):
            for subitem in item:
                if isinstance(subitem, tuple):
                   templist.append(subitem[1])
                else:
                   templist.append(subitem)
            outlist.append(tuple(templist))
         else:
            outlist.append(item)
     # print "\nOutput List: ", outlist
     return outlist

From fal at libero.it  Sat Jul 28 18:29:20 2012
From: fal at libero.it (Francesco Loffredo)
Date: Sat, 28 Jul 2012 18:29:20 +0200
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <50140179.2080306@libero.it>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
Message-ID: <50141360.6030606@libero.it>

Il 28/07/2012 17:12, Francesco Loffredo ha scritto:
> Il 19/07/2012 19:33, PyProg PyProg ha scritto:
>> Hi all,
>>
>> I would get a new list as:
>>
>> [(0, '3eA', 'Dupont', 'Juliette', '11.0/10.0', '4.0/5.0', '17.5/30.0',
>> '3.0/5.0', '4.5/10.0', '35.5/60.0'), (1, '3eA', 'Pop', 'Iggy',
>> '12.0/10.0', '3.5/5.0', '11.5/30.0', '4.0/5.0', '5.5/10.0',
>> '7.5/10.0', '40.5/60.0')]
>>
>> ... from this one:
>>
>> [(0, '3eA', 'Dupont', 'Juliette', 0, 11.0, 10.0), (0, '3eA', 'Dupont',
>> 'Juliette', 1, 4.0, 5.0), (0, '3eA', 'Dupont', 'Juliette', 2, 17.5,
>> 30.0), (0, '3eA', 'Dupont', 'Juliette', 3, 3.0, 5.0), (0, '3eA',
>> 'Dupont', 'Juliette', 4, 4.5, 10.0), (0, '3eA', 'Dupont', 'Juliette',
>> 5, 35.5, 60.0), (1, '3eA', 'Pop', 'Iggy', 0, 12.0, 10.0), (1, '3eA',
>> 'Pop', 'Iggy', 1, 3.5, 5.0), (1, '3eA', 'Pop', 'Iggy', 2, 11.5, 30.0),
>> (1, '3eA', 'Pop', 'Iggy', 3, 4.0, 5.0), (1, '3eA', 'Pop', 'Iggy', 4,
>> 5.5, 10.0), (1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0)]
>>
>> How to make that ? I'm looking for but for now I can't do it.
>>
>> Thanks in advance.
>>
>> a+
>>
> I had to study carefully your present and desired lists, and I 
> understood what follows (please, next time explain !):
> - each 7-tuple in your present list is a record for some measure 
> relative to a person. Its fields are as follows:
>     - field 0: code (I think you want that in growing order)
>     - field 1: group code (could be a class or a group to which both 
> of your example persons belong)
>     - fields 2, 3: surname and name of the person
>     - field 4: progressive number of the measure (these are in order 
> already, but I think you want to enforce this) that you want to 
> exclude from the output list while keeping the order
>     - field 5, 6: numerator and denominator of a ratio that is the 
> measure. you want the ratio to be written as a single string: "%s/%s" 
> % field5, field6
>
> Taking for granted this structure and my educated guesses about what 
> you didn't tell us, here's my solution:
>
> def flatten(inlist)
>     """
>       takes PyProg PyProg's current list and returns his/her desired one,
>       given my guesses about the structure of inlist and the desired 
> result.
>     """
>     tempdict = {}
>     for item in inlist:
>         if len(item) != 7:
>             print "Item errato: \n", item
>         id = tuple(item[:4])
>         progr = item[4]
>         payload = "%s/%s" % item[5:]
>         if id in tempdict:
>            tempdict[id].extend([(progr, payload)])
>         else:
>            tempdict[id] = [(progr, payload)]
>     for item in tempdict:
>         tempdict[item].sort() # so we set payloads in progressive 
> order, if they aren't already
>     # print "Temporary Dict: ", tempdict
>     tmplist2 = []
>     for item in tempdict:
>         templist = []
>         templist.extend(item)
>         templist.extend(tempdict[item])
>         tmplist2.append(tuple(templist))
>     tmplist2.sort()# so we set IDs in order
>     # print "Temporary List: ", tmplist2
>     outlist = []
>     for item in tmplist2:
>         templist = []
>         if isinstance(item, tuple):
>            for subitem in item:
>                if isinstance(subitem, tuple):
>                   templist.append(subitem[1])
>                else:
>                   templist.append(subitem)
>            outlist.append(tuple(templist))
>         else:
>            outlist.append(item)
>     # print "\nOutput List: ", outlist
>     return outlist
>
ok, as usual when I look again at something I wrote, I found some little 
mistakes. Here's my errata corrige:

1- of course, a function definition must end with a colon...
    line 1:
def flatten(inlist):

2- sorry, English is not my first language...
    line 9:
              print "Item length wrong!\n", item

3- I didn't insert a break statement after line 9, but if inlist 
contained a wrong item it would be nice to do something more than simply 
tell the user, for example we could skip that item, or trim / pad it, or 
stop the execution, or raise an exception... I just told it to the 
unsuspecting user, and this may very probably lead to some exception in 
a later point, or (much worse) to wrong results. So:
    line 8-9:
         if len(item) != 7:
              print "Item length wrong!\n", item
              raise ValueError("item length != 7")


... now I feel better ... but I must avoid reading my function again, or 
I'll find some more bugs!

Francesco

From steve at pearwood.info  Sat Jul 28 19:43:19 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 29 Jul 2012 03:43:19 +1000
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <50141360.6030606@libero.it>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>	<50140179.2080306@libero.it>
	<50141360.6030606@libero.it>
Message-ID: <501424B7.1000300@pearwood.info>

Francesco Loffredo wrote:

> but I must avoid reading my function again, or 
> I'll find some more bugs!

Perhaps you should run your function, and test it.

Finding bugs is not the problem. Once you find them, you can fix them. It is 
the bugs that you don't know about that is the problem.


-- 
Steven

From TTabern at ddti.net  Sat Jul 28 19:53:38 2012
From: TTabern at ddti.net (Todd Tabern)
Date: Sat, 28 Jul 2012 17:53:38 +0000
Subject: [Tutor] Topic #2 of Tutor Digest
Message-ID: <o4yr0wcpskdo08vfc0qotcv3.1343497635110@email.android.com>

Mark Lawrence: Yes, I did... I kept encountering errors when trying to post the first time. I didn't think my question went through, so I tried this one.
Even if I were to purposefully ask the question in multiple places, why does that concern you? I wasn't aware that asking for help in multiple places is forbidden.
I'm sorry that it offended you so much that you felt the need to respond in that manner instead of providing assistance...

Cheers

tutor-request at python.org wrote:


Send Tutor mailing list submissions to
        tutor at python.org

To subscribe or unsubscribe via the World Wide Web, visit
        http://mail.python.org/mailman/listinfo/tutor
or, via email, send a message with subject or body 'help' to
        tutor-request at python.org

You can reach the person managing the list at
        tutor-owner at python.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Tutor digest..."


Today's Topics:

   1. Re: Encoding error when reading text files in Python 3
      (Steven D'Aprano)
   2. Re: Search and replace text in XML file? (Mark Lawrence)
   3. Re: Encoding error when reading text files in Python 3 (Dat Huynh)
   4. Re: Flatten a list in tuples and remove doubles
      (Francesco Loffredo)
   5. Re: Flatten a list in tuples and remove doubles
      (Francesco Loffredo)


----------------------------------------------------------------------

Message: 1
Date: Sat, 28 Jul 2012 20:09:28 +1000
From: Steven D'Aprano <steve at pearwood.info>
To: tutor at python.org
Subject: Re: [Tutor] Encoding error when reading text files in Python
        3
Message-ID: <5013BA58.1040404 at pearwood.info>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Dat Huynh wrote:
> Dear all,
>
> I have written a simple application by Python to read data from text files.
>
> Current I have both Python version 2.7.2 and Python 3.2.3 on my laptop.
> I don't know why it does not run on Python version 3 while it runs
> well on Python 2.

Python 2 is more forgiving of beginner errors when dealing with text and
bytes, but makes it harder to deal with text correctly.

Python 3 makes it easier to deal with text correctly, but is less forgiving.

When you read from a file in Python 2, it will give you *something*, even if
it is the wrong thing. It will not give an decoding error, even if the text
you are reading is not valid text. It will just give you junk bytes, sometimes
known as moji-bake.

Python 3 no longer does that. It tells you when there is a problem, so you can
fix it.


> Could you please tell me how I can run it on python 3?
> Following is my Python code.
>
>  ------------------------------
>    for subdir, dirs, files in os.walk(rootdir):
>         for file in files:
>             print("Processing [" +file +"]...\n" )
>             f = open(rootdir+file, 'r')
>             data = f.read()
>             f.close()
>             print(data)
> ------------------------------
>
> This is the error message:
[...]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position
> 4980: ordinal not in range(128)


This tells you that you are reading a non-ASCII file but haven't told Python
what encoding to use, so by default Python uses ASCII.

Do you know what encoding the file is?

Do you understand about Unicode text and bytes? If not, I suggest you read
this article:

http://www.joelonsoftware.com/articles/Unicode.html


In Python 3, you can either tell Python what encoding to use:

f = open(rootdir+file, 'r', encoding='utf8')  # for example

or you can set an error handler:

f = open(rootdir+file, 'r', errors='ignore')  # for example

or both

f = open(rootdir+file, 'r', encoding='ascii', errors='replace')


You can see the list of encodings and error handlers here:

http://docs.python.org/py3k/library/codecs.html


Unfortunately, Python 2 does not support this using the built-in open
function. Instead, you have to uses codecs.open instead of the built-in open,
like this:

import codecs
f = codecs.open(rootdir+file, 'r', encoding='utf8')  # for example

which fortunately works in both Python 2 or 3.


Or you can read the file in binary mode, and then decode it into text:

f = open(rootdir+file, 'rb')
data = f.read()
f.close()
text = data.decode('cp866', 'replace')
print(text)


If you don't know the encoding, you can try opening the file in Firefox or
Internet Explorer and see if they can guess it, or you can use the chardet
library in Python.

http://pypi.python.org/pypi/chardet

Or if you don't care about getting moji-bake, you can pretend that the file is
encoded using Latin-1. That will pretty much read anything, although what it
gives you may be junk.



--
Steven


------------------------------

Message: 2
Date: Sat, 28 Jul 2012 11:25:30 +0100
From: Mark Lawrence <breamoreboy at yahoo.co.uk>
To: tutor at python.org
Subject: Re: [Tutor] Search and replace text in XML file?
Message-ID: <jv0emn$eda$1 at dough.gmane.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

On 28/07/2012 02:38, Todd Tabern wrote:
> I'm looking to search an entire XML file for specific text and replace that text, while maintaining the structure of the XML file. The text occurs within multiple nodes throughout the file.
> I basically need to replace every occurrence C:\Program Files with C:\Program Files (x86), regardless of location. For example, that text appears within:
> <URL>C:\Program Files\\Map Data\Road_Centerlines.shp</URL>
> and also within:
> <RoutingIndexPathName>C:\Program Files\Templates\RoadNetwork.rtx</RoutingIndexPathName>
> ...among others.
> I've tried some non-python methods and they all ruined the XML structure. I've been Google searching all day and can only seem to find solutions that look for a specific node and replace the whole string between the tags.
> I've been looking at using minidom to achieve this but I just can't seem to figure out the right method.
> My end goal, once I have working code, is to compile an exe that can work on machines without python, allowing a user can click in order to perform the XML modification.
> Thanks in advance.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

Did you really have to ask the same question on two separate Python
mailing lists and only 15 minutes apart?

--
Cheers.

Mark Lawrence.



------------------------------

Message: 3
Date: Sat, 28 Jul 2012 18:45:47 +0800
From: Dat Huynh <htdatcse at gmail.com>
To: tutor at python.org
Subject: Re: [Tutor] Encoding error when reading text files in Python
        3
Message-ID:
        <CAPw=odian5_MYMudR+OiaWstgoG9i+zDoSk-0rRy1arxif1G0g at mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

I change my code and it runs on Python 3 now.

           f = open(rootdir+file, 'rb')
          data = f.read().decode('utf8', 'ignore')

Thank you very much.
Sincerely,
Dat.




On Sat, Jul 28, 2012 at 6:09 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> Dat Huynh wrote:
>>
>> Dear all,
>>
>> I have written a simple application by Python to read data from text
>> files.
>>
>> Current I have both Python version 2.7.2 and Python 3.2.3 on my laptop.
>> I don't know why it does not run on Python version 3 while it runs
>> well on Python 2.
>
>
> Python 2 is more forgiving of beginner errors when dealing with text and
> bytes, but makes it harder to deal with text correctly.
>
> Python 3 makes it easier to deal with text correctly, but is less forgiving.
>
> When you read from a file in Python 2, it will give you *something*, even if
> it is the wrong thing. It will not give an decoding error, even if the text
> you are reading is not valid text. It will just give you junk bytes,
> sometimes known as moji-bake.
>
> Python 3 no longer does that. It tells you when there is a problem, so you
> can fix it.
>
>
>
>> Could you please tell me how I can run it on python 3?
>> Following is my Python code.
>>
>>  ------------------------------
>>    for subdir, dirs, files in os.walk(rootdir):
>>         for file in files:
>>             print("Processing [" +file +"]...\n" )
>>             f = open(rootdir+file, 'r')
>>             data = f.read()
>>             f.close()
>>             print(data)
>> ------------------------------
>>
>> This is the error message:
>
> [...]
>
>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position
>> 4980: ordinal not in range(128)
>
>
>
> This tells you that you are reading a non-ASCII file but haven't told Python
> what encoding to use, so by default Python uses ASCII.
>
> Do you know what encoding the file is?
>
> Do you understand about Unicode text and bytes? If not, I suggest you read
> this article:
>
> http://www.joelonsoftware.com/articles/Unicode.html
>
>
> In Python 3, you can either tell Python what encoding to use:
>
> f = open(rootdir+file, 'r', encoding='utf8')  # for example
>
> or you can set an error handler:
>
> f = open(rootdir+file, 'r', errors='ignore')  # for example
>
> or both
>
> f = open(rootdir+file, 'r', encoding='ascii', errors='replace')
>
>
> You can see the list of encodings and error handlers here:
>
> http://docs.python.org/py3k/library/codecs.html
>
>
> Unfortunately, Python 2 does not support this using the built-in open
> function. Instead, you have to uses codecs.open instead of the built-in
> open, like this:
>
> import codecs
> f = codecs.open(rootdir+file, 'r', encoding='utf8')  # for example
>
> which fortunately works in both Python 2 or 3.
>
>
> Or you can read the file in binary mode, and then decode it into text:
>
> f = open(rootdir+file, 'rb')
> data = f.read()
> f.close()
> text = data.decode('cp866', 'replace')
> print(text)
>
>
> If you don't know the encoding, you can try opening the file in Firefox or
> Internet Explorer and see if they can guess it, or you can use the chardet
> library in Python.
>
> http://pypi.python.org/pypi/chardet
>
> Or if you don't care about getting moji-bake, you can pretend that the file
> is encoded using Latin-1. That will pretty much read anything, although what
> it gives you may be junk.
>
>
>
> --
> Steven
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor


------------------------------

Message: 4
Date: Sat, 28 Jul 2012 17:12:57 +0200
From: Francesco Loffredo <fal at libero.it>
To: tutor at python.org
Subject: Re: [Tutor] Flatten a list in tuples and remove doubles
Message-ID: <50140179.2080306 at libero.it>
Content-Type: text/plain; charset=windows-1251; format=flowed

Il 19/07/2012 19:33, PyProg PyProg ha scritto:
> Hi all,
>
> I would get a new list as:
>
> [(0, '3eA', 'Dupont', 'Juliette', '11.0/10.0', '4.0/5.0', '17.5/30.0',
> '3.0/5.0', '4.5/10.0', '35.5/60.0'), (1, '3eA', 'Pop', 'Iggy',
> '12.0/10.0', '3.5/5.0', '11.5/30.0', '4.0/5.0', '5.5/10.0',
> '7.5/10.0', '40.5/60.0')]
>
> ... from this one:
>
> [(0, '3eA', 'Dupont', 'Juliette', 0, 11.0, 10.0), (0, '3eA', 'Dupont',
> 'Juliette', 1, 4.0, 5.0), (0, '3eA', 'Dupont', 'Juliette', 2, 17.5,
> 30.0), (0, '3eA', 'Dupont', 'Juliette', 3, 3.0, 5.0), (0, '3eA',
> 'Dupont', 'Juliette', 4, 4.5, 10.0), (0, '3eA', 'Dupont', 'Juliette',
> 5, 35.5, 60.0), (1, '3eA', 'Pop', 'Iggy', 0, 12.0, 10.0), (1, '3eA',
> 'Pop', 'Iggy', 1, 3.5, 5.0), (1, '3eA', 'Pop', 'Iggy', 2, 11.5, 30.0),
> (1, '3eA', 'Pop', 'Iggy', 3, 4.0, 5.0), (1, '3eA', 'Pop', 'Iggy', 4,
> 5.5, 10.0), (1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0)]
>
> How to make that ? I'm looking for but for now I can't do it.
>
> Thanks in advance.
>
> a+
>
I had to study carefully your present and desired lists, and I
understood what follows (please, next time explain !):
- each 7-tuple in your present list is a record for some measure
relative to a person. Its fields are as follows:
     - field 0: code (I think you want that in growing order)
     - field 1: group code (could be a class or a group to which both of
your example persons belong)
     - fields 2, 3: surname and name of the person
     - field 4: progressive number of the measure (these are in order
already, but I think you want to enforce this) that you want to exclude
from the output list while keeping the order
     - field 5, 6: numerator and denominator of a ratio that is the
measure. you want the ratio to be written as a single string: "%s/%s" %
field5, field6

Taking for granted this structure and my educated guesses about what you
didn't tell us, here's my solution:

def flatten(inlist)
     """
       takes PyProg PyProg's current list and returns his/her desired one,
       given my guesses about the structure of inlist and the desired
result.
     """
     tempdict = {}
     for item in inlist:
         if len(item) != 7:
             print "Item errato: \n", item
         id = tuple(item[:4])
         progr = item[4]
         payload = "%s/%s" % item[5:]
         if id in tempdict:
            tempdict[id].extend([(progr, payload)])
         else:
            tempdict[id] = [(progr, payload)]
     for item in tempdict:
         tempdict[item].sort() # so we set payloads in progressive
order, if they aren't already
     # print "Temporary Dict: ", tempdict
     tmplist2 = []
     for item in tempdict:
         templist = []
         templist.extend(item)
         templist.extend(tempdict[item])
         tmplist2.append(tuple(templist))
     tmplist2.sort()# so we set IDs in order
     # print "Temporary List: ", tmplist2
     outlist = []
     for item in tmplist2:
         templist = []
         if isinstance(item, tuple):
            for subitem in item:
                if isinstance(subitem, tuple):
                   templist.append(subitem[1])
                else:
                   templist.append(subitem)
            outlist.append(tuple(templist))
         else:
            outlist.append(item)
     # print "\nOutput List: ", outlist
     return outlist


------------------------------

Message: 5
Date: Sat, 28 Jul 2012 18:29:20 +0200
From: Francesco Loffredo <fal at libero.it>
To: tutor at python.org
Subject: Re: [Tutor] Flatten a list in tuples and remove doubles
Message-ID: <50141360.6030606 at libero.it>
Content-Type: text/plain; charset=windows-1251; format=flowed

Il 28/07/2012 17:12, Francesco Loffredo ha scritto:
> Il 19/07/2012 19:33, PyProg PyProg ha scritto:
>> Hi all,
>>
>> I would get a new list as:
>>
>> [(0, '3eA', 'Dupont', 'Juliette', '11.0/10.0', '4.0/5.0', '17.5/30.0',
>> '3.0/5.0', '4.5/10.0', '35.5/60.0'), (1, '3eA', 'Pop', 'Iggy',
>> '12.0/10.0', '3.5/5.0', '11.5/30.0', '4.0/5.0', '5.5/10.0',
>> '7.5/10.0', '40.5/60.0')]
>>
>> ... from this one:
>>
>> [(0, '3eA', 'Dupont', 'Juliette', 0, 11.0, 10.0), (0, '3eA', 'Dupont',
>> 'Juliette', 1, 4.0, 5.0), (0, '3eA', 'Dupont', 'Juliette', 2, 17.5,
>> 30.0), (0, '3eA', 'Dupont', 'Juliette', 3, 3.0, 5.0), (0, '3eA',
>> 'Dupont', 'Juliette', 4, 4.5, 10.0), (0, '3eA', 'Dupont', 'Juliette',
>> 5, 35.5, 60.0), (1, '3eA', 'Pop', 'Iggy', 0, 12.0, 10.0), (1, '3eA',
>> 'Pop', 'Iggy', 1, 3.5, 5.0), (1, '3eA', 'Pop', 'Iggy', 2, 11.5, 30.0),
>> (1, '3eA', 'Pop', 'Iggy', 3, 4.0, 5.0), (1, '3eA', 'Pop', 'Iggy', 4,
>> 5.5, 10.0), (1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0)]
>>
>> How to make that ? I'm looking for but for now I can't do it.
>>
>> Thanks in advance.
>>
>> a+
>>
> I had to study carefully your present and desired lists, and I
> understood what follows (please, next time explain !):
> - each 7-tuple in your present list is a record for some measure
> relative to a person. Its fields are as follows:
>     - field 0: code (I think you want that in growing order)
>     - field 1: group code (could be a class or a group to which both
> of your example persons belong)
>     - fields 2, 3: surname and name of the person
>     - field 4: progressive number of the measure (these are in order
> already, but I think you want to enforce this) that you want to
> exclude from the output list while keeping the order
>     - field 5, 6: numerator and denominator of a ratio that is the
> measure. you want the ratio to be written as a single string: "%s/%s"
> % field5, field6
>
> Taking for granted this structure and my educated guesses about what
> you didn't tell us, here's my solution:
>
> def flatten(inlist)
>     """
>       takes PyProg PyProg's current list and returns his/her desired one,
>       given my guesses about the structure of inlist and the desired
> result.
>     """
>     tempdict = {}
>     for item in inlist:
>         if len(item) != 7:
>             print "Item errato: \n", item
>         id = tuple(item[:4])
>         progr = item[4]
>         payload = "%s/%s" % item[5:]
>         if id in tempdict:
>            tempdict[id].extend([(progr, payload)])
>         else:
>            tempdict[id] = [(progr, payload)]
>     for item in tempdict:
>         tempdict[item].sort() # so we set payloads in progressive
> order, if they aren't already
>     # print "Temporary Dict: ", tempdict
>     tmplist2 = []
>     for item in tempdict:
>         templist = []
>         templist.extend(item)
>         templist.extend(tempdict[item])
>         tmplist2.append(tuple(templist))
>     tmplist2.sort()# so we set IDs in order
>     # print "Temporary List: ", tmplist2
>     outlist = []
>     for item in tmplist2:
>         templist = []
>         if isinstance(item, tuple):
>            for subitem in item:
>                if isinstance(subitem, tuple):
>                   templist.append(subitem[1])
>                else:
>                   templist.append(subitem)
>            outlist.append(tuple(templist))
>         else:
>            outlist.append(item)
>     # print "\nOutput List: ", outlist
>     return outlist
>
ok, as usual when I look again at something I wrote, I found some little
mistakes. Here's my errata corrige:

1- of course, a function definition must end with a colon...
    line 1:
def flatten(inlist):

2- sorry, English is not my first language...
    line 9:
              print "Item length wrong!\n", item

3- I didn't insert a break statement after line 9, but if inlist
contained a wrong item it would be nice to do something more than simply
tell the user, for example we could skip that item, or trim / pad it, or
stop the execution, or raise an exception... I just told it to the
unsuspecting user, and this may very probably lead to some exception in
a later point, or (much worse) to wrong results. So:
    line 8-9:
         if len(item) != 7:
              print "Item length wrong!\n", item
              raise ValueError("item length != 7")


... now I feel better ... but I must avoid reading my function again, or
I'll find some more bugs!

Francesco


------------------------------

_______________________________________________
Tutor maillist  -  Tutor at python.org
http://mail.python.org/mailman/listinfo/tutor


End of Tutor Digest, Vol 101, Issue 99
**************************************

From emile at fenx.com  Sat Jul 28 20:34:12 2012
From: emile at fenx.com (Emile van Sebille)
Date: Sat, 28 Jul 2012 11:34:12 -0700
Subject: [Tutor] Topic #2 of Tutor Digest
In-Reply-To: <o4yr0wcpskdo08vfc0qotcv3.1343497635110@email.android.com>
References: <o4yr0wcpskdo08vfc0qotcv3.1343497635110@email.android.com>
Message-ID: <jv1bb3$ppj$1@dough.gmane.org>

On 7/28/2012 10:53 AM Todd Tabern said...
> Even if I were to purposefully ask the question in multiple places, why does that concern you?  wasn't aware that asking for help in multiple places is forbidden.

It's not forbidden -- simply disrespectful.  We all volunteer our 
available time to respond to questions, and most of us tend to follow 
multiple lists, so by asking on both you'll waste the time of those 
responders unaware that an appropraite response has already been 
provided, although in an alternate forum.  I generally don't even read 
threads where I recognize that a respected or known respondant has 
already replied.  When I see a question posed on multiple forums I note 
the poster and tend to disregard their future posts.  When it continues 
you'll start seeing *plonk* as potential respondnts add you to their 
kill file.

See http://www.catb.org/~esr/faqs/smart-questions.html#forum in 
particular, but the entire article should be committed to memory.

Emile


From eryksun at gmail.com  Sat Jul 28 20:41:21 2012
From: eryksun at gmail.com (eryksun)
Date: Sat, 28 Jul 2012 14:41:21 -0400
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <50140179.2080306@libero.it>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
Message-ID: <CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>

On Sat, Jul 28, 2012 at 11:12 AM, Francesco Loffredo <fal at libero.it> wrote:
>
> I had to study carefully your present and desired lists, and I understood
> what follows (please, next time explain !):
> - each 7-tuple in your present list is a record for some measure relative to
> a person. Its fields are as follows:
>     - field 0: code (I think you want that in growing order)
>     - field 1: group code (could be a class or a group to which both of your
> example persons belong)
>     - fields 2, 3: surname and name of the person
>     - field 4: progressive number of the measure (these are in order
> already, but I think you want to enforce this) that you want to exclude from
> the output list while keeping the order
>     - field 5, 6: numerator and denominator of a ratio that is the measure.
> you want the ratio to be written as a single string: "%s/%s" % field5,
> field6

This looks like a good problem for itertools.groupby. My solution
below needs error checking and testing, but this is where I'd start:

data = [
  (0, '3eA', 'Dupont', 'Juliette', 0, 11.0, 10.0),
  (0, '3eA', 'Dupont', 'Juliette', 1, 4.0, 5.0),
  (0, '3eA', 'Dupont', 'Juliette', 2, 17.5, 30.0),
  (0, '3eA', 'Dupont', 'Juliette', 3, 3.0, 5.0),
  (0, '3eA', 'Dupont', 'Juliette', 4, 4.5, 10.0),
  (0, '3eA', 'Dupont', 'Juliette', 5, 35.5, 60.0),
  (1, '3eA', 'Pop', 'Iggy', 0, 12.0, 10.0),
  (1, '3eA', 'Pop', 'Iggy', 1, 3.5, 5.0),
  (1, '3eA', 'Pop', 'Iggy', 2, 11.5, 30.0),
  (1, '3eA', 'Pop', 'Iggy', 3, 4.0, 5.0),
  (1, '3eA', 'Pop', 'Iggy', 4, 5.5, 10.0),
  (1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0),
]

from operator import itemgetter
from itertools import groupby

#first sort by keyfunc, then group by it
keyfunc = itemgetter(0,1,2,3)
groups = groupby(sorted(data, key=keyfunc), keyfunc)

result = []
for group, records in groups:
    temp = tuple('%s/%s' % r[5:] for r in sorted(records, key=itemgetter(4)))
    result.append(group + temp)

>>> result
[(0, '3eA', 'Dupont', 'Juliette', '11.0/10.0', '4.0/5.0', '17.5/30.0',
'3.0/5.0', '4.5/10.0', '35.5/60.0'), (1, '3eA', 'Pop', 'Iggy',
'12.0/10.0', '3.5/5.0', '11.5/30.0', '4.0/5.0', '5.5/10.0',
'40.5/60.0')]

From fal at libero.it  Sat Jul 28 23:35:10 2012
From: fal at libero.it (Francesco Loffredo)
Date: Sat, 28 Jul 2012 23:35:10 +0200
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <501424B7.1000300@pearwood.info>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>	<50140179.2080306@libero.it>
	<50141360.6030606@libero.it> <501424B7.1000300@pearwood.info>
Message-ID: <50145B0E.4010409@libero.it>

Il 28/07/2012 19:43, Steven D'Aprano wrote:
> Francesco Loffredo wrote:
>
>> but I must avoid reading my function again, or I'll find some more bugs!
>
> Perhaps you should run your function, and test it.
Of course I did. Just not as thoroughly as I would if this were a job 
commitment. Unfortunately, I don't know anything about the possible 
contents of PyProg's list, so my testing can only be partial.
I think my function answers the question, though.
>
> Finding bugs is not the problem. Once you find them, you can fix them. 
> It is the bugs that you don't know about that is the problem.
Absolutely. Ignorance is Nr.1 cause of software failure. But I'm not 
sure that a complete, fully tested and robust function would be proper 
in this tutoring environment, unless the OP had asked about unit testing 
or proven software reliability. The many lines of code needed to take 
care of all possible input cases could even make more difficult for 
him/her to understand how that function solves the problem. At first, I 
wrote flatten(inlist) without any error testing. Then I added a couple 
of lines, just to show that input control is a Good Thing. And every 
time I looked at it, some more controls asked for being coded... but I 
wanted to stop at some point. If you'd like to show us what can be done 
to really make it rock-solid, feel free and welcome! ;-))

Francesco

From fal at libero.it  Sun Jul 29 01:12:32 2012
From: fal at libero.it (Francesco Loffredo)
Date: Sun, 29 Jul 2012 01:12:32 +0200
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
Message-ID: <501471E0.8020604@libero.it>

Il 28/07/2012 20:41, eryksun wrote:
> On Sat, Jul 28, 2012 at 11:12 AM, Francesco Loffredo<fal at libero.it>  wrote:
>> I had to study carefully your present and desired lists, and I understood
>> what follows (please, next time explain !):
>> - each 7-tuple in your present list is a record for some measure relative to
>> a person. Its fields are as follows:
>>      - field 0: code (I think you want that in growing order)
>>      - field 1: group code (could be a class or a group to which both of your
>> example persons belong)
>>      - fields 2, 3: surname and name of the person
>>      - field 4: progressive number of the measure (these are in order
>> already, but I think you want to enforce this) that you want to exclude from
>> the output list while keeping the order
>>      - field 5, 6: numerator and denominator of a ratio that is the measure.
>> you want the ratio to be written as a single string: "%s/%s" % field5,
>> field6
> This looks like a good problem for itertools.groupby. My solution
> below needs error checking and testing, but this is where I'd start:
>
> data = [
>    (0, '3eA', 'Dupont', 'Juliette', 0, 11.0, 10.0),
>    (0, '3eA', 'Dupont', 'Juliette', 1, 4.0, 5.0),
>    (0, '3eA', 'Dupont', 'Juliette', 2, 17.5, 30.0),
>    (0, '3eA', 'Dupont', 'Juliette', 3, 3.0, 5.0),
>    (0, '3eA', 'Dupont', 'Juliette', 4, 4.5, 10.0),
>    (0, '3eA', 'Dupont', 'Juliette', 5, 35.5, 60.0),
>    (1, '3eA', 'Pop', 'Iggy', 0, 12.0, 10.0),
>    (1, '3eA', 'Pop', 'Iggy', 1, 3.5, 5.0),
>    (1, '3eA', 'Pop', 'Iggy', 2, 11.5, 30.0),
>    (1, '3eA', 'Pop', 'Iggy', 3, 4.0, 5.0),
>    (1, '3eA', 'Pop', 'Iggy', 4, 5.5, 10.0),
>    (1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0),
> ]
>
> from operator import itemgetter
> from itertools import groupby
>
> #first sort by keyfunc, then group by it
> keyfunc = itemgetter(0,1,2,3)
> groups = groupby(sorted(data, key=keyfunc), keyfunc)
>
> result = []
> for group, records in groups:
>      temp = tuple('%s/%s' % r[5:] for r in sorted(records, key=itemgetter(4)))
>      result.append(group + temp)
>
>>>> result
> [(0, '3eA', 'Dupont', 'Juliette', '11.0/10.0', '4.0/5.0', '17.5/30.0',
> '3.0/5.0', '4.5/10.0', '35.5/60.0'), (1, '3eA', 'Pop', 'Iggy',
> '12.0/10.0', '3.5/5.0', '11.5/30.0', '4.0/5.0', '5.5/10.0',
> '40.5/60.0')]
>
Hmmmm... it happened again. I spend some time and effort to solve a 
problem, I feel like I'm almost a Real Python Programmer... and someone 
spoils my pride showing me some standard module whose name I barely 
remember, that can solve that same problem in a few lines...

Hey! That's a GREAT solution, Eryksun! Nothing wrong with you, really!

Every time this happens, I have to admit that I'm a newbie and I've 
still got a lot to learn about Python. Especially about its wonderful 
standard library. Better than Apple's App Store: for anything you can 
think, there's a Module. Problem is, I still can't readily recall which 
to use for a given problem.
My bad, now I'll RTFM again and I will study very carefully the operator 
and itertools modules. Who knows, maybe in a few decades I'll be able to 
say "This looks like a good problem for Module X" too.

Francesco


From alan.gauld at btinternet.com  Sun Jul 29 01:49:27 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 29 Jul 2012 00:49:27 +0100
Subject: [Tutor] Topic #2 of Tutor Digest
In-Reply-To: <o4yr0wcpskdo08vfc0qotcv3.1343497635110@email.android.com>
References: <o4yr0wcpskdo08vfc0qotcv3.1343497635110@email.android.com>
Message-ID: <jv1tq7$d8s$1@dough.gmane.org>

On 28/07/12 18:53, Todd Tabern wrote:

> Even if I were to purposefully ask the question in multiple places,
 > why does that concern you? I wasn't aware that asking for
 > help in multiple places is forbidden.

Its not. But it is considered bad Netiquette. Its also likely
to get you noticed as a "nuisance" and someone to be ignored
by the more experienced news users - usually the very ones
you want to be answering your questions!

Some of the reasons it's bad are:
- often the same people read all of the groups you choose
   to post in and so they read the same message multiple
   times - which is tedious, especially if they have already replied.
- often the person replying unwittingly sends to all of the
   forums, further compounding the issue above.
- if they don't send to all forums, multiple people from
   different forums duplicate the solution, wasting everyone's time.
- It uses up extra bandwidth which for smartphone users
   (and many third world users on dial up) may mean extra
   cash payments to read duplicate messages.

So its better to try the most likely group and only if you
don't get a reply after a day or two try another.

> I'm sorry that it offended you so much that you felt the
 > need to respond in that manner instead of
> providing assistance...

He was providing assistance, he was pointing out that
you had breached good practice for posting on internet
fora... :-)

Speaking of which....

> tutor-request at python.org wrote:
>
>
> Send Tutor mailing list submissions to
>          tutor at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
< .... snip ....>
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."

Please follow those instructions...

And also please don't post the entire digest. Trim it back to the bit 
that is relevant - see the last point above about consideration for 
other users bandwidth limitations.

HTH,

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/


From alan.gauld at btinternet.com  Sun Jul 29 01:53:49 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 29 Jul 2012 00:53:49 +0100
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <501471E0.8020604@libero.it>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it>
Message-ID: <jv1u2d$epi$1@dough.gmane.org>

On 29/07/12 00:12, Francesco Loffredo wrote:

> Every time this happens, I have to admit that I'm a newbie and I've
> still got a lot to learn about Python. Especially about its wonderful
> standard library.

Don't worry, I've been using Python for 15 years and there are
plenty modules I haven't explored yet - and I must admit itertools
is one that I really should get to grips with but never seem to
find the time!

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/


From d at davea.name  Sun Jul 29 03:42:36 2012
From: d at davea.name (Dave Angel)
Date: Sat, 28 Jul 2012 21:42:36 -0400
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <501471E0.8020604@libero.it>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it>
Message-ID: <5014950C.5060203@davea.name>

On 07/28/2012 07:12 PM, Francesco Loffredo wrote:
> Il 28/07/2012 20:41, eryksun wrote:
>> On Sat, Jul 28, 2012 at 11:12 AM, Francesco Loffredo<fal at libero.it> 
>> wrote:
>>> I had to study carefully your present and desired lists, and I
>>> understood
>>> what follows (please, next time explain !):
>>> - each 7-tuple in your present list is a record for some measure
>>> relative to
>>> a person. Its fields are as follows:
>>>      - field 0: code (I think you want that in growing order)
>>>      - field 1: group code (could be a class or a group to which
>>> both of your
>>> example persons belong)
>>>      - fields 2, 3: surname and name of the person
>>>      - field 4: progressive number of the measure (these are in order
>>> already, but I think you want to enforce this) that you want to
>>> exclude from
>>> the output list while keeping the order
>>>      - field 5, 6: numerator and denominator of a ratio that is the
>>> measure.
>>> you want the ratio to be written as a single string: "%s/%s" % field5,
>>> field6
>> This looks like a good problem for itertools.groupby. My solution
>> below needs error checking and testing, but this is where I'd start:
>>
>> data = [
>>    (0, '3eA', 'Dupont', 'Juliette', 0, 11.0, 10.0),
>>    (0, '3eA', 'Dupont', 'Juliette', 1, 4.0, 5.0),
>>    (0, '3eA', 'Dupont', 'Juliette', 2, 17.5, 30.0),
>>    (0, '3eA', 'Dupont', 'Juliette', 3, 3.0, 5.0),
>>    (0, '3eA', 'Dupont', 'Juliette', 4, 4.5, 10.0),
>>    (0, '3eA', 'Dupont', 'Juliette', 5, 35.5, 60.0),
>>    (1, '3eA', 'Pop', 'Iggy', 0, 12.0, 10.0),
>>    (1, '3eA', 'Pop', 'Iggy', 1, 3.5, 5.0),
>>    (1, '3eA', 'Pop', 'Iggy', 2, 11.5, 30.0),
>>    (1, '3eA', 'Pop', 'Iggy', 3, 4.0, 5.0),
>>    (1, '3eA', 'Pop', 'Iggy', 4, 5.5, 10.0),
>>    (1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0),
>> ]
>>
>> from operator import itemgetter
>> from itertools import groupby
>>
>> #first sort by keyfunc, then group by it
>> keyfunc = itemgetter(0,1,2,3)
>> groups = groupby(sorted(data, key=keyfunc), keyfunc)
>>
>> result = []
>> for group, records in groups:
>>      temp = tuple('%s/%s' % r[5:] for r in sorted(records,
>> key=itemgetter(4)))
>>      result.append(group + temp)
>>
>>>>> result
>> [(0, '3eA', 'Dupont', 'Juliette', '11.0/10.0', '4.0/5.0', '17.5/30.0',
>> '3.0/5.0', '4.5/10.0', '35.5/60.0'), (1, '3eA', 'Pop', 'Iggy',
>> '12.0/10.0', '3.5/5.0', '11.5/30.0', '4.0/5.0', '5.5/10.0',
>> '40.5/60.0')]
>>
> Hmmmm... it happened again. I spend some time and effort to solve a
> problem, I feel like I'm almost a Real Python Programmer... and
> someone spoils my pride showing me some standard module whose name I
> barely remember, that can solve that same problem in a few lines...
>
> Hey! That's a GREAT solution, Eryksun! Nothing wrong with you, really!
>
> Every time this happens, I have to admit that I'm a newbie and I've
> still got a lot to learn about Python. Especially about its wonderful
> standard library. Better than Apple's App Store: for anything you can
> think, there's a Module. Problem is, I still can't readily recall
> which to use for a given problem.
> My bad, now I'll RTFM again and I will study very carefully the
> operator and itertools modules. Who knows, maybe in a few decades I'll
> be able to say "This looks like a good problem for Module X" too.
>
> Francesco
>

You might find it enlightening to look up:

    http://www.doughellmann.com/PyMOTW/

which explores the Pythons standard library.

-- 

DaveA


From eryksun at gmail.com  Sun Jul 29 08:53:54 2012
From: eryksun at gmail.com (eryksun)
Date: Sun, 29 Jul 2012 02:53:54 -0400
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <501471E0.8020604@libero.it>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it>
Message-ID: <CACL+1avg319oA5DZq0YeP_cCaLYo_adfMXs+B45HMpvnWRfZaw@mail.gmail.com>

On Sat, Jul 28, 2012 at 7:12 PM, Francesco Loffredo <fal at libero.it> wrote:
>
> My bad, now I'll RTFM again and I will study very carefully the operator and
> itertools modules.

I forgot to mention a gotcha about groupby's implementation. The
grouby object and the yielded _grouper objects share a single
iterator. Here's a (somewhat contrived) example of a mistake:

>>> groups = groupby(sorted(data, key=keyfunc), keyfunc)
>>> groups = list(groups)  #DON'T DO THIS
>>> groups
[((0, '3eA', 'Dupont', 'Juliette'), <itertools._grouper object at 0xb74734ec>),
 ((1, '3eA', 'Pop', 'Iggy'), <itertools._grouper object at 0xb74735cc>)]

>>> list(groups[0][1])  #EMPTY
[]
>>> list(groups[1][1])  #ONLY THE LAST ITEM
[(1, '3eA', 'Pop', 'Iggy', 5, 40.5, 60.0)]

From breamoreboy at yahoo.co.uk  Sun Jul 29 12:38:58 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 29 Jul 2012 11:38:58 +0100
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <jv1u2d$epi$1@dough.gmane.org>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it> <jv1u2d$epi$1@dough.gmane.org>
Message-ID: <jv33rs$5l4$1@dough.gmane.org>

On 29/07/2012 00:53, Alan Gauld wrote:
> On 29/07/12 00:12, Francesco Loffredo wrote:
>
>> Every time this happens, I have to admit that I'm a newbie and I've
>> still got a lot to learn about Python. Especially about its wonderful
>> standard library.
>
> Don't worry, I've been using Python for 15 years and there are
> plenty modules I haven't explored yet - and I must admit itertools
> is one that I really should get to grips with but never seem to
> find the time!
>

You're nuts :)  Itertools to me is the Swiss Army Knife of the standard 
library, I've used it umpteen times.  Even the recipes are on PyPi.  The 
investment of a little time studying it will easily be repaid in the 
long term.

-- 
Cheers.

Mark Lawrence.


From __peter__ at web.de  Sun Jul 29 13:21:10 2012
From: __peter__ at web.de (Peter Otten)
Date: Sun, 29 Jul 2012 13:21:10 +0200
Subject: [Tutor] Flatten a list in tuples and remove doubles
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it> <jv1u2d$epi$1@dough.gmane.org>
	<jv33rs$5l4$1@dough.gmane.org>
Message-ID: <jv36an$lir$1@dough.gmane.org>

Mark Lawrence wrote:

> On 29/07/2012 00:53, Alan Gauld wrote:
>> On 29/07/12 00:12, Francesco Loffredo wrote:
>>
>>> Every time this happens, I have to admit that I'm a newbie and I've
>>> still got a lot to learn about Python. Especially about its wonderful
>>> standard library.
>>
>> Don't worry, I've been using Python for 15 years and there are
>> plenty modules I haven't explored yet - and I must admit itertools
>> is one that I really should get to grips with but never seem to
>> find the time!
>>
> 
> You're nuts :)  Itertools to me is the Swiss Army Knife of the standard
> library, I've used it umpteen times.  Even the recipes are on PyPi.  The
> investment of a little time studying it will easily be repaid in the
> long term.

If you don't have to deal with large datasets many of its functions can 
easily be emulated with lists and loops though. As an example here's the 
grouping with a plain vanilla dict:

groups = {}
for item in data:
    groups.setdefault(item[:4], []).append(item[-2:])

result = [key + tuple("{}/{}".format(*v) for v in values) for key, values in 
groups.items()]



From alan.gauld at btinternet.com  Sun Jul 29 15:16:37 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 29 Jul 2012 14:16:37 +0100
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <jv36an$lir$1@dough.gmane.org>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it> <jv1u2d$epi$1@dough.gmane.org>
	<jv33rs$5l4$1@dough.gmane.org> <jv36an$lir$1@dough.gmane.org>
Message-ID: <jv3d3l$2q5$1@dough.gmane.org>

On 29/07/12 12:21, Peter Otten wrote:

>>> plenty modules I haven't explored yet - and I must admit itertools
>>> is one that I really should get to grips with...

>> You're nuts :)  Itertools to me is the Swiss Army Knife
>
> If you don't have to deal with large datasets many of its functions can
> easily be emulated with lists and loops though.

Exactly. and if I deal with large data its usually in a SQL database and 
I can use SQL to do the hard work. So the groupby type functions are for 
me a SQL feature.

But I know thee itertools has a lot of powerful stuff in it, I just 
haven't had a big enough itch to investige it yet. :-)

And at the moment I'm immersing myself in JQuery so it will be a while 
before itertools gets a look in!

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/


From breamoreboy at yahoo.co.uk  Sun Jul 29 16:21:19 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 29 Jul 2012 15:21:19 +0100
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <jv3d3l$2q5$1@dough.gmane.org>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it> <jv1u2d$epi$1@dough.gmane.org>
	<jv33rs$5l4$1@dough.gmane.org> <jv36an$lir$1@dough.gmane.org>
	<jv3d3l$2q5$1@dough.gmane.org>
Message-ID: <jv3gsa$pf$2@dough.gmane.org>

On 29/07/2012 14:16, Alan Gauld wrote:
> On 29/07/12 12:21, Peter Otten wrote:
>
>>>> plenty modules I haven't explored yet - and I must admit itertools
>>>> is one that I really should get to grips with...
>
>>> You're nuts :)  Itertools to me is the Swiss Army Knife
>>
>> If you don't have to deal with large datasets many of its functions can
>> easily be emulated with lists and loops though.
>
> Exactly. and if I deal with large data its usually in a SQL database and
> I can use SQL to do the hard work. So the groupby type functions are for
> me a SQL feature.
>
> But I know thee itertools has a lot of powerful stuff in it, I just
> haven't had a big enough itch to investige it yet. :-)
>
> And at the moment I'm immersing myself in JQuery so it will be a while
> before itertools gets a look in!
>

Reminds me of the Welsh selectors, two were wise and chose Dai, but 
three were foolish and chose Gareth.  That should lead to some head 
scratching!!!

-- 
Cheers.

Mark Lawrence.


From alan.gauld at btinternet.com  Sun Jul 29 19:32:15 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 29 Jul 2012 18:32:15 +0100
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <jv3gsa$pf$2@dough.gmane.org>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it> <jv1u2d$epi$1@dough.gmane.org>
	<jv33rs$5l4$1@dough.gmane.org> <jv36an$lir$1@dough.gmane.org>
	<jv3d3l$2q5$1@dough.gmane.org> <jv3gsa$pf$2@dough.gmane.org>
Message-ID: <jv3s2v$4q6$1@dough.gmane.org>

On 29/07/12 15:21, Mark Lawrence wrote:

> Reminds me of the Welsh selectors, two were wise and chose Dai, but
> three were foolish and chose Gareth.  That should lead to some head
> scratching!!!

I'm scratching, but it sounds like a throwback to the 1980's
national rugby team... Dai Jones and Gareth Edwards?


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/


From breamoreboy at yahoo.co.uk  Sun Jul 29 20:27:37 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 29 Jul 2012 19:27:37 +0100
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <jv3s2v$4q6$1@dough.gmane.org>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it> <jv1u2d$epi$1@dough.gmane.org>
	<jv33rs$5l4$1@dough.gmane.org> <jv36an$lir$1@dough.gmane.org>
	<jv3d3l$2q5$1@dough.gmane.org> <jv3gsa$pf$2@dough.gmane.org>
	<jv3s2v$4q6$1@dough.gmane.org>
Message-ID: <jv3va9$r3q$1@dough.gmane.org>

On 29/07/2012 18:32, Alan Gauld wrote:
> On 29/07/12 15:21, Mark Lawrence wrote:
>
>> Reminds me of the Welsh selectors, two were wise and chose Dai, but
>> three were foolish and chose Gareth.  That should lead to some head
>> scratching!!!
>
> I'm scratching, but it sounds like a throwback to the 1980's
> national rugby team... Dai Jones and Gareth Edwards?
>
>

1970's actually. Gareth Edwards of course, but Dai was used by Max Boyce 
the same way an English person would use John.  It was taken from the 
Max Boyce version of "Deck of Cards".

-- 
Cheers.

Mark Lawrence.


From eryksun at gmail.com  Sun Jul 29 21:52:09 2012
From: eryksun at gmail.com (eryksun)
Date: Sun, 29 Jul 2012 15:52:09 -0400
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <jv36an$lir$1@dough.gmane.org>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it> <jv1u2d$epi$1@dough.gmane.org>
	<jv33rs$5l4$1@dough.gmane.org> <jv36an$lir$1@dough.gmane.org>
Message-ID: <CACL+1avQwCwet2dVTd72w-uUFpYiDBoBMARVOJHq8DGT+2orUw@mail.gmail.com>

On Sun, Jul 29, 2012 at 7:21 AM, Peter Otten <__peter__ at web.de> wrote:
>
> If you don't have to deal with large datasets many of its functions can
> easily be emulated with lists and loops though. As an example here's the
> grouping with a plain vanilla dict:
>
> groups = {}
> for item in data:
>     groups.setdefault(item[:4], []).append(item[-2:])
>
> result = [key + tuple("{}/{}".format(*v) for v in values) for key, values in
> groups.items()]

Or use a defaultdict:

from collections import defaultdict

groups = defaultdict(list)
for item in data:
    groups[item[:4]].append(item[4:])

result = []
for key in sorted(groups):
    groups[key].sort()
    result.append(key + tuple('%s/%s' % v[1:] for v in groups[key]))

From __peter__ at web.de  Sun Jul 29 22:29:10 2012
From: __peter__ at web.de (Peter Otten)
Date: Sun, 29 Jul 2012 22:29:10 +0200
Subject: [Tutor] Flatten a list in tuples and remove doubles
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it> <jv1u2d$epi$1@dough.gmane.org>
	<jv33rs$5l4$1@dough.gmane.org> <jv36an$lir$1@dough.gmane.org>
	<CACL+1avQwCwet2dVTd72w-uUFpYiDBoBMARVOJHq8DGT+2orUw@mail.gmail.com>
Message-ID: <jv46e4$j1e$1@dough.gmane.org>

eryksun wrote:

> On Sun, Jul 29, 2012 at 7:21 AM, Peter Otten <__peter__ at web.de> wrote:
>>
>> If you don't have to deal with large datasets many of its functions can
>> easily be emulated with lists and loops though. As an example here's the
>> grouping with a plain vanilla dict:
>>
>> groups = {}
>> for item in data:
>>     groups.setdefault(item[:4], []).append(item[-2:])
>>
>> result = [key + tuple("{}/{}".format(*v) for v in values) for key, values
>> in groups.items()]
> 
> Or use a defaultdict:
> 
> from collections import defaultdict

Yes, I would have used that, but I wanted to show that there was a 
reasonable solution that did not rely on any libraries.

> groups = defaultdict(list)
> for item in data:
>     groups[item[:4]].append(item[4:])
> 
> result = []
> for key in sorted(groups):
>     groups[key].sort()
>     result.append(key + tuple('%s/%s' % v[1:] for v in groups[key]))

If you have to sort your data anyway you can sort it first and then apply 
itertools.groupby()...


From fal at libero.it  Mon Jul 30 01:10:22 2012
From: fal at libero.it (Francesco Loffredo)
Date: Mon, 30 Jul 2012 01:10:22 +0200
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <5014950C.5060203@davea.name>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it> <5014950C.5060203@davea.name>
Message-ID: <5015C2DE.7080209@libero.it>

Il 29/07/2012 03:42, Dave Angel wrote:
> On 07/28/2012 07:12 PM, Francesco Loffredo wrote:
> <snip>
> You might find it enlightening to look up:
>
>      http://www.doughellmann.com/PyMOTW/
>
> which explores the Pythons standard library.
>
Site promptly visited, PDF downloaded, started reading. It really seems 
a thorough yet handy reference. THANKS, DAVE!

Francesco

From eryksun at gmail.com  Mon Jul 30 01:51:41 2012
From: eryksun at gmail.com (eryksun)
Date: Sun, 29 Jul 2012 19:51:41 -0400
Subject: [Tutor] Flatten a list in tuples and remove doubles
In-Reply-To: <jv46e4$j1e$1@dough.gmane.org>
References: <CAC0U9k6W2MO8-qy+xCZbaGZzN-NFkcr5tt9rk2ocDTYiOxPc9w@mail.gmail.com>
	<50140179.2080306@libero.it>
	<CACL+1at8Pbu00fU8XsHdt0ALvfGtY5Ljr8aLrE_=XVU4gogmmw@mail.gmail.com>
	<501471E0.8020604@libero.it> <jv1u2d$epi$1@dough.gmane.org>
	<jv33rs$5l4$1@dough.gmane.org> <jv36an$lir$1@dough.gmane.org>
	<CACL+1avQwCwet2dVTd72w-uUFpYiDBoBMARVOJHq8DGT+2orUw@mail.gmail.com>
	<jv46e4$j1e$1@dough.gmane.org>
Message-ID: <CACL+1asz1veMby2Z+ra5dJgUcRBgcn2mDHO0Vx-gfLRxDo-rbQ@mail.gmail.com>

On Sun, Jul 29, 2012 at 4:29 PM, Peter Otten <__peter__ at web.de> wrote:
>
> If you have to sort your data anyway you can sort it first and then apply
> itertools.groupby()...

That was my thinking. I wrote it with groupby (see link below) because
Francesco described field 0 as "in growing order".

http://mail.python.org/pipermail/tutor/2012-July/090607.html

I see now the 2nd call to sorted() in the tuple generator was
redundant. I should have just sorted the data without the key function
to order within each group by field 4 in the first pass:

keyfunc = itemgetter(0,1,2,3)
groups = groupby(sorted(data), keyfunc)

result = []
for group, records in groups:
    temp = tuple('%s/%s' % r[5:] for r in records)
    result.append(group + temp)

In general though you have to sort and group by the same key.

From emailkgnow at gmail.com  Mon Jul 30 15:57:18 2012
From: emailkgnow at gmail.com (Khalid Al-Ghamdi)
Date: Mon, 30 Jul 2012 16:57:18 +0300
Subject: [Tutor] sqlite3 Dilemma
Message-ID: <CABM2kur8rW3gSit1yUUEY-RsAN0QnQze_NZ5at8cK-Kx7aDbrw@mail.gmail.com>

Hi All,

I am a teacher at an industrial training center. We have to conduct
hands-on exams (fixing pumps, etc). I review all the test schedules for all
the students (made by other teachers) to make sure there are no clashes due
to lack of ample supply of equipment or tools. Meaning no two trainees are
to have the same test at the same time. If this is the case, then the date
or time or test code have to be changed to avoid the conflict.

I have the schedule in csv format. I have been able to export it to sqlite3
as seen below and I have found all the conflicts, too.

My dilemma is how to go about making sqlite3 alter the date, time, or exam
code if another trainee is scheduled at the same time and date with the
same test code.

What is the proper approach to handle this?

I'm using python 3 on windows.

Below is how far I've gotten.

Thanks


   1. import csv, sqlite3
   2.
   3.
   4. conn = sqlite3.connect(":memory:") #unless this is ":memory:" this
   database will be created in your CWD which is the resolver folder
   5. c = conn.cursor()
   6. c.execute("create table test (teams integer, sn integer, badge
   integer ,name text, grp integer,\
   7. major text, track text, stage text,  tc text, subject, text, course
   text, ws text, date text, \
   8. time text, proctor text, code text, no integer)")
   9.
   10. reader = csv.reader(open("final.csv", "r"))
   11.
   12. for (teams, sn, badge , name, grp, major, track, stage, tc, subject,
    course, ws,\
   13.     date, time, proctor, code, no) in reader:
   14.     c.execute('INSERT INTO test \
   15.    (teams, sn, badge, name, grp, major, track, stage,tc, subject,
   course, ws, date, time, proctor, code, no)\
   16.    VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)',
   17.     (teams, sn, badge, name, grp, major, track, stage, tc,subject,
    course, ws, date, time, proctor,code, no))
   18.
   19. c.execute("select code, date, time, count (*) from test group by
   code, date, time having count(*)>1")
   20. c1_list=list(c.fetchall())
   21. print(c1_list)
   22. print("your table has {} clashes".format(len(c1_list)))
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/1917f31c/attachment.html>

From wprins at gmail.com  Mon Jul 30 16:19:34 2012
From: wprins at gmail.com (Walter Prins)
Date: Mon, 30 Jul 2012 15:19:34 +0100
Subject: [Tutor] sqlite3 Dilemma
In-Reply-To: <CABM2kur8rW3gSit1yUUEY-RsAN0QnQze_NZ5at8cK-Kx7aDbrw@mail.gmail.com>
References: <CABM2kur8rW3gSit1yUUEY-RsAN0QnQze_NZ5at8cK-Kx7aDbrw@mail.gmail.com>
Message-ID: <CANLXbfDAnYUFrJ1zKF9ZU2OfWcBw+ZbZ+MBhKBRP7GSN0Q8zKw@mail.gmail.com>

On 30 July 2012 14:57, Khalid Al-Ghamdi <emailkgnow at gmail.com> wrote:
> Hi All,
>
> I am a teacher at an industrial training center. We have to conduct hands-on
> exams (fixing pumps, etc). I review all the test schedules for all the
> students (made by other teachers) to make sure there are no clashes due to
> lack of ample supply of equipment or tools. Meaning no two trainees are to
> have the same test at the same time. If this is the case, then the date or
> time or test code have to be changed to avoid the conflict.

How exactly is the date or time changed, and by who?  Is the process
always the same or is there discretion involved?

> I have the schedule in csv format. I have been able to export it to sqlite3
> as seen below and I have found all the conflicts, too.
>
> My dilemma is how to go about making sqlite3 alter the date, time, or exam
> code if another trainee is scheduled at the same time and date with the same
> test code.
>
> What is the proper approach to handle this?

It depends on the current process of managing collisions.  If this is
discretionary, then the approach for the program should be one where
your program identifies the collisions, identifies suitable
alternatives and then requests the user to supply the alternate
date/time to resolve the conflict, repeating this for every conflict.
If the collision management rules can be expressed entirely
objectively then it might perhaps be fully automated.  There may also
be a "best effort" type middle road where many/most cases can be
automatically handled while some will require operator input.

By the way, nice job using SQLite to do set based operations.  To move
toward a solution, thinking off the top of my head, you will need to
probably do something along the following lines:
1) For each collision detected in your previous work, find the details
for that specific collision.
2) Determine which of the entries will keep the slot and remove it
from the list of colliding entries
3) Find alternate slots for each of the remaining entries.

3) Will involve something like:
3.1) Determine list of suitable/available slots, suitably ordered
3.2) Update the colliding entry with a new selected date/time
3.3) Repeat for each collision.

So 1) would involve another query, 2) involve input from the user, 3)
likely involves 2 more queries, one for retrieving available slots and
one for updating with a new date/time

Finally I imagine you may want to write the entire lot back out to CSV
(not sure if that's your goal or not.)

Anyway, that's just a few thoughts off the top of my head.

Walter

From victoriahomsy at yahoo.com  Mon Jul 30 18:05:02 2012
From: victoriahomsy at yahoo.com (Victoria Homsy)
Date: Mon, 30 Jul 2012 17:05:02 +0100 (BST)
Subject: [Tutor] Help please!
Message-ID: <1343664302.13185.YahooMailNeo@web29506.mail.ird.yahoo.com>

Hi! I am a new Python user, and would really appreciate some help. My code is as follows:

from sys import argvs
script, mum_mood, dad_mood = argvs

# my own function
def dad_and_mum_mood(mum_mood, dad_mood):
print "If both mum and dad are in a good mood, all is good."
print "If one is and one isn't, all is good."
print "If both are in a bad mood, not so good."
print "Mum is in a %s mood" % (mum_mood)
print "Dad is in a %s mood" % (dad_mood)
print "Where does that leave us?"

?
dad_and_mum_mood(mum_mood, dad_mood)


I am just trying to get get the information mum_mood and dad_mood from the argvs (written into the command line), but I get the error "ImportError: cannot import name argvs".

Do you have any idea why this wouldn't be working? Thank you so much for helping me.?

Kind regards,?

Victoria?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/9c0a3506/attachment-0001.html>

From punchagan at gmail.com  Mon Jul 30 18:10:31 2012
From: punchagan at gmail.com (Puneeth Chaganti)
Date: Mon, 30 Jul 2012 21:40:31 +0530
Subject: [Tutor] Help please!
In-Reply-To: <1343664302.13185.YahooMailNeo@web29506.mail.ird.yahoo.com>
References: <1343664302.13185.YahooMailNeo@web29506.mail.ird.yahoo.com>
Message-ID: <CALnw1fSd-Vt_rpQx1kC9MUN6OMULAkx-uuu_dLFMYBPbq=QAtA@mail.gmail.com>

On Mon, Jul 30, 2012 at 9:35 PM, Victoria Homsy <victoriahomsy at yahoo.com> wrote:
> Hi! I am a new Python user, and would really appreciate some help. My code
> is as follows:
>
> from sys import argvs
> script, mum_mood, dad_mood = argvs
>
> # my own function
> def dad_and_mum_mood(mum_mood, dad_mood):
> print "If both mum and dad are in a good mood, all is good."
> print "If one is and one isn't, all is good."
> print "If both are in a bad mood, not so good."
> print "Mum is in a %s mood" % (mum_mood)
> print "Dad is in a %s mood" % (dad_mood)
> print "Where does that leave us?"
>
>
> dad_and_mum_mood(mum_mood, dad_mood)
>
>
> I am just trying to get get the information mum_mood and dad_mood from the
> argvs (written into the command line), but I get the error "ImportError:
> cannot import name argvs".

The problem is exactly what the error message says.  argvs should
really be argv.

HTH,
Puneeth

From emailkgnow at gmail.com  Mon Jul 30 18:28:20 2012
From: emailkgnow at gmail.com (Khalid Al-Ghamdi)
Date: Mon, 30 Jul 2012 19:28:20 +0300
Subject: [Tutor] sqlite3 Dilemma
In-Reply-To: <CANLXbfDAnYUFrJ1zKF9ZU2OfWcBw+ZbZ+MBhKBRP7GSN0Q8zKw@mail.gmail.com>
References: <CABM2kur8rW3gSit1yUUEY-RsAN0QnQze_NZ5at8cK-Kx7aDbrw@mail.gmail.com>
	<CANLXbfDAnYUFrJ1zKF9ZU2OfWcBw+ZbZ+MBhKBRP7GSN0Q8zKw@mail.gmail.com>
Message-ID: <CABM2kuqX7ZD35p22Ki5_XQ=r8tCXuW-0BQ064CMgkTpCDNq-+w@mail.gmail.com>

Thanks for your input Walter. Are there any techniques to achieve this via
code or sql you share. Because I was thinking along the same you mentioned
but didnt knwo how to go about it.

?????? ???????? ?? ?????? ????? ??? ?? Walter Prins wprins at gmail.com:

> On 30 July 2012 14:57, Khalid Al-Ghamdi <emailkgnow at gmail.com<javascript:;>>
> wrote:
> > Hi All,
> >
> > I am a teacher at an industrial training center. We have to conduct
> hands-on
> > exams (fixing pumps, etc). I review all the test schedules for all the
> > students (made by other teachers) to make sure there are no clashes due
> to
> > lack of ample supply of equipment or tools. Meaning no two trainees are
> to
> > have the same test at the same time. If this is the case, then the date
> or
> > time or test code have to be changed to avoid the conflict.
>
> How exactly is the date or time changed, and by who?  Is the process
> always the same or is there discretion involved?
>
> > I have the schedule in csv format. I have been able to export it to
> sqlite3
> > as seen below and I have found all the conflicts, too.
> >
> > My dilemma is how to go about making sqlite3 alter the date, time, or
> exam
> > code if another trainee is scheduled at the same time and date with the
> same
> > test code.
> >
> > What is the proper approach to handle this?
>
> It depends on the current process of managing collisions.  If this is
> discretionary, then the approach for the program should be one where
> your program identifies the collisions, identifies suitable
> alternatives and then requests the user to supply the alternate
> date/time to resolve the conflict, repeating this for every conflict.
> If the collision management rules can be expressed entirely
> objectively then it might perhaps be fully automated.  There may also
> be a "best effort" type middle road where many/most cases can be
> automatically handled while some will require operator input.
>
> By the way, nice job using SQLite to do set based operations.  To move
> toward a solution, thinking off the top of my head, you will need to
> probably do something along the following lines:
> 1) For each collision detected in your previous work, find the details
> for that specific collision.
> 2) Determine which of the entries will keep the slot and remove it
> from the list of colliding entries
> 3) Find alternate slots for each of the remaining entries.
>
> 3) Will involve something like:
> 3.1) Determine list of suitable/available slots, suitably ordered
> 3.2) Update the colliding entry with a new selected date/time
> 3.3) Repeat for each collision.
>
> So 1) would involve another query, 2) involve input from the user, 3)
> likely involves 2 more queries, one for retrieving available slots and
> one for updating with a new date/time
>
> Finally I imagine you may want to write the entire lot back out to CSV
> (not sure if that's your goal or not.)
>
> Anyway, that's just a few thoughts off the top of my head.
>
> Walter
> _______________________________________________
> Tutor maillist  -  Tutor at python.org <javascript:;>
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/7dba9a78/attachment.html>

From oberoc at gmail.com  Mon Jul 30 18:56:08 2012
From: oberoc at gmail.com (Tino Dai)
Date: Mon, 30 Jul 2012 12:56:08 -0400
Subject: [Tutor] finally without try or except
Message-ID: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>

Hi!

     Is there anyway to execute a block of code at the end of a program in
2.6 regardless of what happened before eg exiting normally or died because
of an exception?
I was thinking about maybe a free standing finally code block or a
decorator.

Any hints?
Tino
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/c5d679b9/attachment.html>

From marc.tompkins at gmail.com  Mon Jul 30 19:16:22 2012
From: marc.tompkins at gmail.com (Marc Tompkins)
Date: Mon, 30 Jul 2012 10:16:22 -0700
Subject: [Tutor] finally without try or except
In-Reply-To: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
Message-ID: <CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>

On Mon, Jul 30, 2012 at 9:56 AM, Tino Dai <oberoc at gmail.com> wrote:

> Hi!
>
>      Is there anyway to execute a block of code at the end of a program in
> 2.6 regardless of what happened before eg exiting normally or died because
> of an exception?
> I was thinking about maybe a free standing finally code block or a
> decorator.
>

How about calling the entire program in a module and putting the call to it
inside a try/except block?  Then you can have your cleanup code in a
finally block at the end of all that.

Exceptions are meant to be caught.  I think what you're asking is "is there
any way to make Python ignore any bad stuff and keep running my program?"
The answer is yes: catch your exceptions.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/59422048/attachment.html>

From oberoc at gmail.com  Mon Jul 30 19:23:45 2012
From: oberoc at gmail.com (Tino Dai)
Date: Mon, 30 Jul 2012 13:23:45 -0400
Subject: [Tutor] finally without try or except
In-Reply-To: <CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>
Message-ID: <CAOu0yXbK+yun0mJ+3PScjcMsN_HsDm9Pkm6647Cn=2QAs0Nm5g@mail.gmail.com>

On Mon, Jul 30, 2012 at 1:16 PM, Marc Tompkins <marc.tompkins at gmail.com>wrote:

> On Mon, Jul 30, 2012 at 9:56 AM, Tino Dai <oberoc at gmail.com> wrote:
>
>> Hi!
>>
>>      Is there anyway to execute a block of code at the end of a program
>> in 2.6 regardless of what happened before eg exiting normally or died
>> because of an exception?
>> I was thinking about maybe a free standing finally code block or a
>> decorator.
>>
>
> How about calling the entire program in a module and putting the call to
> it inside a try/except block?  Then you can have your cleanup code in a
> finally block at the end of all that.
>
> Exceptions are meant to be caught.  I think what you're asking is "is
> there any way to make Python ignore any bad stuff and keep running my
> program?"  The answer is yes: catch your exceptions.
>

Actually, what I'm doing is keeping a pending item log in memory as an
array and then saving it to the DB at the end of the program, but what
happens if the user hits ctrl-c, then the pending items array is lost.
That's the use case that I'm looking for a solution to.

-Tino
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/fb7845d4/attachment-0001.html>

From ramit.prasad at jpmorgan.com  Mon Jul 30 19:44:54 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Mon, 30 Jul 2012 17:44:54 +0000
Subject: [Tutor] finally without try or except
In-Reply-To: <CAOu0yXbK+yun0mJ+3PScjcMsN_HsDm9Pkm6647Cn=2QAs0Nm5g@mail.gmail.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>
	<CAOu0yXbK+yun0mJ+3PScjcMsN_HsDm9Pkm6647Cn=2QAs0Nm5g@mail.gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416593EDA@SCACMX008.exchad.jpmchase.net>

> Actually, what I'm doing is keeping a pending item log in memory as an array
> and then saving it to the DB at the end of the program, but what happens if
> the user hits ctrl-c, then the pending items array is lost. That's the use
> case that I'm looking for a solution to.

http://docs.python.org/library/exceptions.html#exceptions.KeyboardInterrupt

Ctrl-c generates a KeyboardInterrupt exception which you can catch with a
try-except clause. Of course, if the user hits ctrl-c again while 
executing the "except" portion of the python code, it will generate another
exception. The second exception will not be caught without another explicit
try-except clause and ad infinitum for the third press of ctrl-c.

You probably want to rethink your process. You can write the item log
to the db in the except clause but this can be buggy as a second ctrl-c 
will stop it. Often users will repeatedly hit ctrl-c until the program 
stops. Alternatively, you can store the item log in the db and update it 
as the program progresses. This will be the most robust as it will
also work for cases where the program is terminated without the use of 
the keyboard (i.e. kill -9, task manager, computer reboot, etc.)  but
it might also slow the program down. You could also try writing the
item log to a local file using pickle or csv.



Ramit


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From breamoreboy at yahoo.co.uk  Mon Jul 30 19:52:39 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 30 Jul 2012 18:52:39 +0100
Subject: [Tutor] finally without try or except
In-Reply-To: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
Message-ID: <jv6hjp$ukp$1@dough.gmane.org>

On 30/07/2012 17:56, Tino Dai wrote:
> Hi!
>
>       Is there anyway to execute a block of code at the end of a program in
> 2.6 regardless of what happened before eg exiting normally or died because
> of an exception?
> I was thinking about maybe a free standing finally code block or a
> decorator.
>
> Any hints?
> Tino
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>

Sorry I'm not completely sure what you're asking for but will this help 
http://docs.python.org/library/atexit.html ?

-- 
Cheers.

Mark Lawrence.


From __peter__ at web.de  Mon Jul 30 20:17:11 2012
From: __peter__ at web.de (Peter Otten)
Date: Mon, 30 Jul 2012 20:17:11 +0200
Subject: [Tutor] sqlite3 Dilemma
References: <CABM2kur8rW3gSit1yUUEY-RsAN0QnQze_NZ5at8cK-Kx7aDbrw@mail.gmail.com>
Message-ID: <jv6j2i$eng$1@dough.gmane.org>

Khalid Al-Ghamdi wrote:

> I am a teacher at an industrial training center. We have to conduct
> hands-on exams (fixing pumps, etc). I review all the test schedules for
> all the students (made by other teachers) to make sure there are no
> clashes due to lack of ample supply of equipment or tools. Meaning no two
> trainees are to have the same test at the same time. If this is the case,
> then the date or time or test code have to be changed to avoid the
> conflict.
> 
> I have the schedule in csv format. I have been able to export it to
> sqlite3 as seen below and I have found all the conflicts, too.
> 
> My dilemma is how to go about making sqlite3 alter the date, time, or exam
> code if another trainee is scheduled at the same time and date with the
> same test code.
> 
> What is the proper approach to handle this?

I think what you have is a "constraint satisfaction problem". There are 
packages in Python that address it, but I have no experience with them and 
therefore cannot make any recommendations. This one

http://pypi.python.org/pypi/constraint/

...has an example similar to your problem which may help you get started:

http://www.logilab.org/3441

> I'm using python 3 on windows.

I see no mention of Python 3 on their site, so you probably have to resort 
to Python 2 should you end up with the solver mentioned above.

> Below is how far I've gotten.

I'd forget the database for the moment, but keep the code to verify the 
solution once you have arrived at it through other means.


From fomcl at yahoo.com  Mon Jul 30 20:44:19 2012
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Mon, 30 Jul 2012 11:44:19 -0700 (PDT)
Subject: [Tutor] finally without try or except
In-Reply-To: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
Message-ID: <1343673859.91525.YahooMailNeo@web110711.mail.gq1.yahoo.com>

From: Tino Dai <oberoc at gmail.com>

To: nz*tutor pythonzzz <tutor at python.org> 
>Sent: Monday, July 30, 2012 6:56 PM
>Subject: [Tutor] finally without try or except
> 
>
>Hi!
>
>
>? ? ?Is there anyway to execute a block of code at the end of a program in 2.6 regardless of what happened before eg exiting normally or died because of an exception?
>I was thinking about maybe a free standing finally code block or a decorator.?
>
>
>Any hints?
>Tino
>
>===> Hi, perhaps it's overkill, but you could define your own __exit__ (and __enter__) method so you can use a 'with' statement (context manager): http://docs.python.org/release/2.5.2/lib/typecontextmanager.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/a78d5141/attachment.html>

From oberoc at gmail.com  Mon Jul 30 21:07:40 2012
From: oberoc at gmail.com (Tino Dai)
Date: Mon, 30 Jul 2012 15:07:40 -0400
Subject: [Tutor] finally without try or except
In-Reply-To: <1343673859.91525.YahooMailNeo@web110711.mail.gq1.yahoo.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<1343673859.91525.YahooMailNeo@web110711.mail.gq1.yahoo.com>
Message-ID: <CAOu0yXZqN2f0phLN_m+-+Kqq=Gx8BHXZZoB+j6cv+GDHJ8TnFg@mail.gmail.com>

On Mon, Jul 30, 2012 at 2:44 PM, Albert-Jan Roskam <fomcl at yahoo.com> wrote:

> *From:* Tino Dai <oberoc at gmail.com>
>
> *To:* nz*tutor pythonzzz <tutor at python.org>
> *Sent:* Monday, July 30, 2012 6:56 PM
> *Subject:* [Tutor] finally without try or except
>
> Hi!
>
>      Is there anyway to execute a block of code at the end of a program in
> 2.6 regardless of what happened before eg exiting normally or died because
> of an exception?
> I was thinking about maybe a free standing finally code block or a
> decorator.
>
> Any hints?
> Tino
>
> ===> Hi, perhaps it's overkill, but you could define your own __exit__
> (and __enter__) method so you can use a 'with' statement (context manager):
> http://docs.python.org/release/2.5.2/lib/typecontextmanager.html
>
> That's actually I good idea, and will take a look at this because it will
expand my python knowledge. I'm not sure if I want to go down this path
because of the above stated reasons. -T
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/a6198a27/attachment-0001.html>

From oberoc at gmail.com  Mon Jul 30 21:10:08 2012
From: oberoc at gmail.com (Tino Dai)
Date: Mon, 30 Jul 2012 15:10:08 -0400
Subject: [Tutor] finally without try or except
In-Reply-To: <jv6hjp$ukp$1@dough.gmane.org>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<jv6hjp$ukp$1@dough.gmane.org>
Message-ID: <CAOu0yXakoVXu23bCg0L7CT-pSBtuYNRAVCNNzCnSQ5tyKQypaA@mail.gmail.com>

On Mon, Jul 30, 2012 at 1:52 PM, Mark Lawrence <breamoreboy at yahoo.co.uk>wrote:

> On 30/07/2012 17:56, Tino Dai wrote:
>
>> Hi!
>>
>>       Is there anyway to execute a block of code at the end of a program
>> in
>> 2.6 regardless of what happened before eg exiting normally or died because
>> of an exception?
>> I was thinking about maybe a free standing finally code block or a
>> decorator.
>>
>> Any hints?
>> Tino
>>
>>
>>
>> ______________________________**_________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>>
>>
> Sorry I'm not completely sure what you're asking for but will this help
> http://docs.python.org/**library/atexit.html<http://docs.python.org/library/atexit.html>?
>
> I think this might be what I'm looking for. But for about 2 minutes, I was
like a-texit....what does that have to do with....oooohhhh it's at exit.
Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/26e3abbc/attachment.html>

From oberoc at gmail.com  Mon Jul 30 21:20:06 2012
From: oberoc at gmail.com (Tino Dai)
Date: Mon, 30 Jul 2012 15:20:06 -0400
Subject: [Tutor] finally without try or except
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF47416593EDA@SCACMX008.exchad.jpmchase.net>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>
	<CAOu0yXbK+yun0mJ+3PScjcMsN_HsDm9Pkm6647Cn=2QAs0Nm5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416593EDA@SCACMX008.exchad.jpmchase.net>
Message-ID: <CAOu0yXaAzh7L9X7y0CRVc9ZZbNDZsK_iNQg6cADiXakxNp+ZKA@mail.gmail.com>

On Mon, Jul 30, 2012 at 1:44 PM, Prasad, Ramit <ramit.prasad at jpmorgan.com>wrote:

> > Actually, what I'm doing is keeping a pending item log in memory as an
> array
> > and then saving it to the DB at the end of the program, but what happens
> if
> > the user hits ctrl-c, then the pending items array is lost. That's the
> use
> > case that I'm looking for a solution to.
>
> http://docs.python.org/library/exceptions.html#exceptions.KeyboardInterrupt
>
> Ctrl-c generates a KeyboardInterrupt exception which you can catch with a
> try-except clause. Of course, if the user hits ctrl-c again while
> executing the "except" portion of the python code, it will generate another
> exception. The second exception will not be caught without another explicit
> try-except clause and ad infinitum for the third press of ctrl-c.
>

Yes, but that would involve surrounding the entire method with a try except
finally block. I was
told by the Python-Guru-In Residence that shouldn't emulate Java code in
Python, and that was
generally bad programming practice (no flame war intended)

>
> You probably want to rethink your process. You can write the item log
> to the db in the except clause but this can be buggy as a second ctrl-c
> will stop it.


I want to do this in the finally block to insure that the records would be
written out to the
DB, hence my original question

Often users will repeatedly hit ctrl-c until the program
> stops. Alternatively, you can store the item log in the db and update it
> as the program progresses.


This is how the program was originally structured, but we found performance
problems with in.
It's using Django and an Oracle DB, which is notoriously bad for single
record read/writes. So, I
figured why don't we append them all to an array, and write them out at the
end of the program.


> This will be the most robust as it will
> also work for cases where the program is terminated without the use of
> the keyboard (i.e. kill -9, task manager, computer reboot, etc.)  but
> it might also slow the program down. You could also try writing the
> item log to a local file using pickle or csv.
>
>  -Thanks,
Tino
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/b6230f76/attachment.html>

From alan.gauld at btinternet.com  Mon Jul 30 21:24:46 2012
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 30 Jul 2012 20:24:46 +0100
Subject: [Tutor] finally without try or except
In-Reply-To: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
Message-ID: <jv6n1t$dkh$1@dough.gmane.org>

On 30/07/12 17:56, Tino Dai wrote:

>       Is there anyway to execute a block of code at the end of a program
> in 2.6 regardless of what happened before eg exiting normally or died
> because of an exception?

Yes, try/finally.
That's what it's there for.

> I was thinking about maybe a free standing finally code block or a
> decorator.

is

try:

really too much work?


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/


From steve at alchemy.com  Mon Jul 30 20:55:37 2012
From: steve at alchemy.com (Steve Willoughby)
Date: Mon, 30 Jul 2012 11:55:37 -0700
Subject: [Tutor] finally without try or except
In-Reply-To: <jv6hjp$ukp$1@dough.gmane.org>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<jv6hjp$ukp$1@dough.gmane.org>
Message-ID: <5016D8A9.7040804@alchemy.com>

On 30-Jul-12 10:52, Mark Lawrence wrote:
> On 30/07/2012 17:56, Tino Dai wrote:
>>       Is there anyway to execute a block of code at the end of a
>> program in
>> 2.6 regardless of what happened before eg exiting normally or died
>> because
>> of an exception?

> Sorry I'm not completely sure what you're asking for but will this help
> http://docs.python.org/library/atexit.html ?

I don't think that will cover the use cases the OP asked about.  I 
suppose you could do something like

try:
    main()
finally:
    save_everything()

But I think it would be better to re-think your design.  Why must the 
final database commit (or whatever it is) happen at the end and not 
along the way?  Are there critical subsets of the code you could protect 
more locally and save after they're done?

Are you certain that the data you'll save will be accurate if the 
program crashed or was interrupted in the middle of something?

I think the block above will do what you're asking for, but I'm not sure 
what you're asking for is what will help you best.




-- 
Steve Willoughby / steve at alchemy.com
"A ship in harbor is safe, but that is not what ships are built for."
PGP Fingerprint 4615 3CCE 0F29 AE6C 8FF4 CA01 73FE 997A 765D 696C

From swiftone at swiftone.org  Mon Jul 30 21:35:26 2012
From: swiftone at swiftone.org (Brett Ritter)
Date: Mon, 30 Jul 2012 12:35:26 -0700
Subject: [Tutor] finally without try or except
In-Reply-To: <CAOu0yXaAzh7L9X7y0CRVc9ZZbNDZsK_iNQg6cADiXakxNp+ZKA@mail.gmail.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>
	<CAOu0yXbK+yun0mJ+3PScjcMsN_HsDm9Pkm6647Cn=2QAs0Nm5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416593EDA@SCACMX008.exchad.jpmchase.net>
	<CAOu0yXaAzh7L9X7y0CRVc9ZZbNDZsK_iNQg6cADiXakxNp+ZKA@mail.gmail.com>
Message-ID: <CAMb349wLAp4UCW4EF9L78LGwZ7_QrsHiN9UbkHHd=bqk65YQHQ@mail.gmail.com>

On Mon, Jul 30, 2012 at 12:20 PM, Tino Dai <oberoc at gmail.com> wrote:
> Yes, but that would involve surrounding the entire method with a try except
> finally block. I was
> told by the Python-Guru-In Residence that shouldn't emulate Java code in
> Python, and that was
> generally bad programming practice (no flame war intended)

It is true that you should use the idioms of each language in their
domain.  That doesn't mean you should ignore language features
entirely.  Wrapping everything in a single try/finally to handle this
exception is as intended and not emulating the Java style of wrapping
everything everywhere (be that good or bad).

-- 
Brett Ritter / SwiftOne
swiftone at swiftone.org

From oberoc at gmail.com  Mon Jul 30 21:51:38 2012
From: oberoc at gmail.com (Tino Dai)
Date: Mon, 30 Jul 2012 15:51:38 -0400
Subject: [Tutor] finally without try or except
In-Reply-To: <CAMb349wLAp4UCW4EF9L78LGwZ7_QrsHiN9UbkHHd=bqk65YQHQ@mail.gmail.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>
	<CAOu0yXbK+yun0mJ+3PScjcMsN_HsDm9Pkm6647Cn=2QAs0Nm5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416593EDA@SCACMX008.exchad.jpmchase.net>
	<CAOu0yXaAzh7L9X7y0CRVc9ZZbNDZsK_iNQg6cADiXakxNp+ZKA@mail.gmail.com>
	<CAMb349wLAp4UCW4EF9L78LGwZ7_QrsHiN9UbkHHd=bqk65YQHQ@mail.gmail.com>
Message-ID: <CAOu0yXbijNmyPH5bf_9VTZ-GK362V6867VM=vNZbz-SMe4P0Gg@mail.gmail.com>

On Mon, Jul 30, 2012 at 3:35 PM, Brett Ritter <swiftone at swiftone.org> wrote:

> On Mon, Jul 30, 2012 at 12:20 PM, Tino Dai <oberoc at gmail.com> wrote:
> > Yes, but that would involve surrounding the entire method with a try
> except
> > finally block. I was
> > told by the Python-Guru-In Residence that shouldn't emulate Java code in
> > Python, and that was
> > generally bad programming practice (no flame war intended)
>
> It is true that you should use the idioms of each language in their
> domain.  That doesn't mean you should ignore language features
> entirely.  Wrapping everything in a single try/finally to handle this
> exception is as intended and not emulating the Java style of wrapping
> everything everywhere (be that good or bad).
>
>
I just remember that there was a performance hit or something for doing
that.
Does that sound familiar to anyone?



> --
> Brett Ritter / SwiftOne
> swiftone at swiftone.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/f8ba9018/attachment.html>

From victoriahomsy at yahoo.com  Mon Jul 30 21:52:45 2012
From: victoriahomsy at yahoo.com (Victoria Homsy)
Date: Mon, 30 Jul 2012 20:52:45 +0100 (BST)
Subject: [Tutor] Help please!
Message-ID: <1343677965.46260.YahooMailNeo@web29503.mail.ird.yahoo.com>

Hello all! I have a very simple question but I'm very new to python. Could you describe to me what the following piece of Python code says in English please??

def print_a_line(line_count, f):
print line_count, f.readline()

I understand that line_count counts the number of lines in the Python prog, and that we are creating a function, and that 'f' stands for file etc, but I can't get my head around what the function does.

Many thanks in advance!

Best,?

Victoria
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/a441b810/attachment.html>

From eryksun at gmail.com  Mon Jul 30 22:01:05 2012
From: eryksun at gmail.com (eryksun)
Date: Mon, 30 Jul 2012 16:01:05 -0400
Subject: [Tutor] finally without try or except
In-Reply-To: <CAOu0yXakoVXu23bCg0L7CT-pSBtuYNRAVCNNzCnSQ5tyKQypaA@mail.gmail.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<jv6hjp$ukp$1@dough.gmane.org>
	<CAOu0yXakoVXu23bCg0L7CT-pSBtuYNRAVCNNzCnSQ5tyKQypaA@mail.gmail.com>
Message-ID: <CACL+1auRd6jaujVuaB6ueKd=EkhCapNGmpj4iDFXKGSfoyMTxA@mail.gmail.com>

On Mon, Jul 30, 2012 at 3:10 PM, Tino Dai <oberoc at gmail.com> wrote:
> On Mon, Jul 30, 2012 at 1:52 PM, Mark Lawrence <breamoreboy at yahoo.co.uk>
> wrote:
>
>> Sorry I'm not completely sure what you're asking for but will this help
>> http://docs.python.org/library/atexit.html ?
>>
> I think this might be what I'm looking for. But for about 2 minutes, I was
> like a-texit....what does that have to do with....oooohhhh it's at exit.

You might want to install a SIGINT (ctrl-c) signal handler. If you
manually handle the SIGINT signal, you can ignore it after the first
signal (users tend to press ctrl-c repeatedly). You could also apply
this to a regular try/finally approach instead of using atexit. If you
choose to use atexit, don't expect magic compared to try/finally. If
the OS kills the process or the interpreter crashes, your atexit
function won't run. I'd periodically save the data (probably based on
both time and quantity) for a long-run process.

For example:

import signal

def sigint(signum, stack):
    #dummy handler to ignore future SIGINT signals
    dummy = lambda si, st: None
    signal.signal(signal.SIGINT, dummy)
    raise KeyboardInterrupt

signal.signal(signal.SIGINT, sigint)

def save_data():
    print "Saving"
    for k in xrange(1<<24):
        pass
    print "\nSaved {0} entries".format(k + 1)

try:
    print "Waiting for signal..."  #press ctrl-c
    signal.pause()
except:
    pass  #At least log here
finally:
    save_data()

The original SIGINT handler is signal.default_int_handler in case you
need to restore it.

From emailkgnow at gmail.com  Mon Jul 30 22:20:07 2012
From: emailkgnow at gmail.com (Khalid Al-Ghamdi)
Date: Mon, 30 Jul 2012 23:20:07 +0300
Subject: [Tutor] sqlite3 Dilemma
In-Reply-To: <jv6j2i$eng$1@dough.gmane.org>
References: <CABM2kur8rW3gSit1yUUEY-RsAN0QnQze_NZ5at8cK-Kx7aDbrw@mail.gmail.com>
	<jv6j2i$eng$1@dough.gmane.org>
Message-ID: <CABM2kuqxcoUyyWwi3=FYMndSa_d9SdXwqnKKNcNUSZSu0sO2cQ@mail.gmail.com>

Damn! So its so serious they have a name for it! No wonder I was stumped
...

Any way thanks you guys have been super helpful.

?????? ???????? ?? ?????? ????? ??? ?? Peter Otten
__peter__ at web.de<javascript:_e({}, 'cvml', 'peter__ at web.de');>
:

> Khalid Al-Ghamdi wrote:
>
> > I am a teacher at an industrial training center. We have to conduct
> > hands-on exams (fixing pumps, etc). I review all the test schedules for
> > all the students (made by other teachers) to make sure there are no
> > clashes due to lack of ample supply of equipment or tools. Meaning no two
> > trainees are to have the same test at the same time. If this is the case,
> > then the date or time or test code have to be changed to avoid the
> > conflict.
> >
> > I have the schedule in csv format. I have been able to export it to
> > sqlite3 as seen below and I have found all the conflicts, too.
> >
> > My dilemma is how to go about making sqlite3 alter the date, time, or
> exam
> > code if another trainee is scheduled at the same time and date with the
> > same test code.
> >
> > What is the proper approach to handle this?
>
> I think what you have is a "constraint satisfaction problem". There are
> packages in Python that address it, but I have no experience with them and
> therefore cannot make any recommendations. This one
>
> http://pypi.python.org/pypi/constraint/
>
> ...has an example similar to your problem which may help you get started:
>
> http://www.logilab.org/3441
>
> > I'm using python 3 on windows.
>
> I see no mention of Python 3 on their site, so you probably have to resort
> to Python 2 should you end up with the solver mentioned above.
>
> > Below is how far I've gotten.
>
> I'd forget the database for the moment, but keep the code to verify the
> solution once you have arrived at it through other means.
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120730/b1861ab2/attachment.html>

From emile at fenx.com  Mon Jul 30 22:29:22 2012
From: emile at fenx.com (Emile van Sebille)
Date: Mon, 30 Jul 2012 13:29:22 -0700
Subject: [Tutor] Help please!
In-Reply-To: <1343677965.46260.YahooMailNeo@web29503.mail.ird.yahoo.com>
References: <1343677965.46260.YahooMailNeo@web29503.mail.ird.yahoo.com>
Message-ID: <jv6qr5$fs9$1@dough.gmane.org>

On 7/30/2012 12:52 PM Victoria Homsy said...
> Hello all! I have a very simple question but I'm very new to python.
> Could you describe to me what the following piece of Python code says in
> English please?
>
> def print_a_line(line_count, f):
>     print line_count, f.readline()


This function accepts two passed parameters, then prints the first, and 
reads and prints one line from the second.  Obviously, to work the first 
must be printable, and the second must provide a readline method.

*IF* you believe that the names selected accurately represent the intent 
of the function, you may come to the type of conclusions you reach below.

Emile



>
> I understand that line_count counts the number of lines in the Python
> prog, and that we are creating a function, and that 'f' stands for file
> etc, but I can't get my head around what the function does.
>
> Many thanks in advance!
>
> Best,
>
> Victoria
>


From ramit.prasad at jpmorgan.com  Mon Jul 30 22:37:58 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Mon, 30 Jul 2012 20:37:58 +0000
Subject: [Tutor] getting results from encoded data i sent to website
In-Reply-To: <38C169C9-AA4B-4D4E-AD63-AD334C6DC0A1@gmail.com>
References: <D5754493-625B-42EE-9050-BB228A6308EB@gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416590604@SCACMX008.exchad.jpmchase.net>
	<38C169C9-AA4B-4D4E-AD63-AD334C6DC0A1@gmail.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416594259@SCACMX008.exchad.jpmchase.net>

Please always respond to the list (or at least CC it) and 
not the individual person.

> By "manually" I mean when I type the isbn into the text box and hit enter. The
> page that the browser shows has this html code (I just pasted the start of
> it):
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"><html>
> <head>
> <!--eBay V3- msxml 4.0 XXXXXXXXXXXXXXXXXXXXXXXXXX-->
> <!--srcId: HalfCommonPage-->
> <title>Half.com: The Cat in the Hat by Dr. Seuss (1957,
> Hardcover)(9780394800011): Dr. Seuss: Books </title>
> <meta name="google-site-verification"
> content="8kHr3jd3Z43q1ovwo0KVgo_NZKIEMjthBxti8m8fYTg">
> <meta name="keywords" content="the cat in the hat by dr. seuss 1957,
> hardcover, dr. seuss, 039480001x 9780394800011, random house children's books,
> hardcover">
> <meta name="description" content="When Mom's away, the mischievous Cat in the
> Hat comes to play--and turns the house into a total ...,($0.75),Dr.
> Seuss,1957,Random House Children's Books,9780394800011">
> <meta name="copyright" content="Copyright 1995-2008 eBay.com">
> <meta name="robots" content="follow,index">
> <meta name="revisit-after" content="7 days">
> <meta http-equiv="expires" content="0">
> <meta http-equiv="pragma" content="no-cache">
> <meta http-equiv="cache-control" content="no-cache,no-store,must-revalidate">
> <meta http-equiv="content-language" content="en">
> <meta http-equiv="content-type" content="text/html; charset=UTF-8">
> <meta property="og:title" content="The Cat in the Hat by Dr. Seuss (1957,
> Hardcover)">
> <meta property="og:description" content="Half.com (Best Price $0.75):When
> Mom's away, the mischievous Cat in the Hat comes to play--and turns the house
> into a total uproar. But the Cat manages to make things right in the end, not
> a split second before Mom returns. Dr. Seuss's">
> <meta property="og:image"
> content="http://i.ebayimg.com/03/!!eCcIY!!2M~$(KGrHqV,!iUE0GshyQ4nBNRUo50i8g~~
> _7.JPG?set_id=89040003C1">
> <meta property="og:site_name" content="Half.com">
> <meta property="og:type" content="product">
> <meta property="fb:app_id" content="102628213125203"><script
> type="text/javascript" language="JavaScript1.1">includeHost =
> 'http://include.ebaystatic.com/';</script><script
> src="http://include.ebaystatic.com/js/e783/us/rover_e7836us.js">
> </script><script language="JavaScript" type="text/javascript"><!--
> 				var s_pageName = "info-refresh/Books";
> 
> 				  var rover = new Rover();
> 				  var ns = rover.createNSTracker();
> 
> 					ns.setSvrGMT(1343591318016);
> 
> 					var RoverDomainBaseUrl =
> "http://rover.ebay.com";
> 
> 
> 				  ns.hasTpimCookielet();
> 
> 

> But when I run the run the program, it returns this:


> ['<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
> "http://www.w3.org/TR/html4/loose.dtd"><html><head><meta http-equiv="Content-
> Type" content="text/html; charset=UTF-8"><script>var pageHasRtmPlacements =
> true;</script><style type="text/css">body,.g-std{font-
> family:Arial,Helvetica,sans-serif;font-
> size:small}form{margin:0;padding:0}a:active,a:link,.g-novisited
> a:visited{color:#00c;text-decoration:none}a:visited{color:#909;text-
> decoration:none}.g-b{font-weight:bold}.g-em{color:#090;font-weight:bold}.g-
> err{color:#f00}.g-hlp{color:#666}.g-pipe{color:#99f}.g-txtBx,.g-btn,.g-
> nav{font-family:Verdana;font-size:x-small}.g-txtBxHlp{color:#666;font-
> family:Verdana;font-size:x-small}.g-m0{margin:0}a:hover{text-
> decoration:underline}.g-i{font-style:italic}.g-bi{font-weight:bold;font-
> style:italic}.g-dft{font-weight:normal}.g-dfti{font-weight:normal;font-
> style:italic}.g-s{font-size:small}.g-xs{font-size:x-small}.g-m{font-
> size:medium}.g-l{font-size:large}.g-xl{font-size:x-large}.g-hdn{height:0;line-
> height:0;overflow:hidden;width:0;position:absolute;font-size:0;z-index:-
> 1;outline:none}#CentralArea{clear:both!important;width:980px!important;margin:
> 0 auto!important}#footer_wrapper{clear:both!important;margin:0
> auto!important;width:100%!important}.iss-
> rtmw{width:980px!important;height:320px;margin:0 auto}.iss-wrp{margin:0
> auto;position:relative;width:980px!important;height:388px;overflow:hidden;bord
> er:1px solid #ccc}.iss-srcd1{width:820px}.iss-srcd2{width:850px;margin-
> left:70px}.iss-txtd{font-size:24px;font-
> family:tahoma;color:#333;position:absolute;left:240px;top:180px}.iss-
> txts1{font-size:24px;font-family:tahoma;color:#333}.iss-txts2{font-
> size:24px;font-family:tahoma;color:#f18113}.iss-txtbtdiv1{padding-
> left:30px;width:750px}.iss-textInput1{width:475px;border-width:1px;border-
> style:solid;border-color:#f00;font-size:13px;font-
> family:Arial;color:#333}.iss-textInput2{width:475px;font-
> size:13px;color:#333;font-family:Arial}.iss-inlineErrDiv1{font-
> size:11px;color:#f00;alignment-adjust:left;margin-bottom:3px}.iss-
> innerHelpSpan1{font-size:11px;color:#333;width:260px;margin-top:5px}.iss-
> title{width:100%!important}.iss-titleDiv{font-family:tahoma;font-
> size:32px;color:#333;width:970px!important;margin:0 auto}.iss-linkTd1{padding-
> top:30px;font-size:13px}.iss-linkTd2{width:810px}.iss-innerHelpSpan2{font-
> size:11px;color:#333;width:260px}.iss-searchDiv{width:820px;margin-
> bottom:80px}.iss-div1{border:1px solid
> #dedede;width:315px;height:320px;margin:7px 0 0 0!important}.iss-
> div2{border:1px solid #dedede;width:315px;height:320px;margin-
> left:10px!important;margin-top:7px!important}.iss-div3{border:1p

> That's the "mumbo-jumbo" I was talking about.
> I'm not sure why the program isn't returning what I expect.

Well, when comparing ebay.com between urllib and the browser 
the code was the same from the first few lines. So I would
assume it has to do with the way you are "filling in" the
text box data. Most of that "mumbo-jumbo" looks like a CSS 
definition. I only have a passing knowledge of urllib(2)
so maybe someone else on the list can help out.

From what I understand doing this all manually can be done
via the Python standard library, but you may want to look
at a couple packages that are written to help. I have not
used either before, but they have been recommended on
the list.

BeautifulSoup is for parsing HTML and a much better option
than the instinct to use regular expressions (which I see
regularly). Mechanize will help deal with browsing and
I think with textbox entry. 

http://www.crummy.com/software/BeautifulSoup/ 
http://wwwsearch.sourceforge.net/mechanize/

<p><span STYLE="color :#000000; font-size: 8pt; background-color :#FFFFFF">
This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email.  
</span></p>

From ramit.prasad at jpmorgan.com  Mon Jul 30 22:18:00 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Mon, 30 Jul 2012 20:18:00 +0000
Subject: [Tutor] Help please!
In-Reply-To: <1343677965.46260.YahooMailNeo@web29503.mail.ird.yahoo.com>
References: <1343677965.46260.YahooMailNeo@web29503.mail.ird.yahoo.com>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416594211@SCACMX008.exchad.jpmchase.net>

> Hello all! I have a very simple question but I'm very new to python. Could you
> describe to me what the following piece of Python code says in English
> please?

Welcome to the list and Python! When posting code in the future I recommend
posting in plain text and not "rich" text or HTML. If you are using
Yahoo! via web interface you can do this by the following link.
http://www.emailquestions.com/yahoo-mail/4446-switch-yahoo-mail-between-rich-text-plain-text.html


> 
> def print_a_line(line_count, f):
> 	print line_count, f.readline()
> 
> I understand that line_count counts the number of lines in the Python prog,
> and that we are creating a function, and that 'f' stands for file etc, but I
> can't get my head around what the function does.

line_count in this function can be anything. It is not actually
doing any counting or any work. All this function does is print whatever
information is in line_count (I assume it would be the current line number), 
then it retrieves exactly one line from the file f and then prints that
line.

Let us assume that the following multi-line string is what is contained in
file f. Some examples of what this function would print are shown below.

'''This is the first line. 
And this is the second line.
'''

>>> print_a_line(1, f )
1 This is the first line.
>>> print_a_line('line 2:', f )
line 2: And this is the second line.


Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From steve at pearwood.info  Tue Jul 31 03:01:53 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 31 Jul 2012 11:01:53 +1000
Subject: [Tutor] finally without try or except
In-Reply-To: <CAOu0yXaAzh7L9X7y0CRVc9ZZbNDZsK_iNQg6cADiXakxNp+ZKA@mail.gmail.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>	<CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>	<CAOu0yXbK+yun0mJ+3PScjcMsN_HsDm9Pkm6647Cn=2QAs0Nm5g@mail.gmail.com>	<5B80DD153D7D744689F57F4FB69AF47416593EDA@SCACMX008.exchad.jpmchase.net>
	<CAOu0yXaAzh7L9X7y0CRVc9ZZbNDZsK_iNQg6cADiXakxNp+ZKA@mail.gmail.com>
Message-ID: <50172E81.2040206@pearwood.info>

Tino Dai wrote:

> Yes, but that would involve surrounding the entire method with a try except
> finally block. I was
> told by the Python-Guru-In Residence that shouldn't emulate Java code in
> Python, and that was
> generally bad programming practice (no flame war intended)

And that is correct. It is *generally* bad programming practice. But that 
doesn't mean it is *always* bad programming practice.

*This* is terrible practice:

try:
     main()
except:
     print("an error occurred")
     # but you'll never know what it was, or be able to fix it ha ha!


But this is perfectly reasonable:

try:
     main()
finally:
     cleanup()



>> You probably want to rethink your process. You can write the item log
>> to the db in the except clause but this can be buggy as a second ctrl-c
>> will stop it.

If you want to be robust, it is best not to try to beat the database. That 
means you should write to the database as soon as you can, as often as you 
need to, and let the database do what it does best: reliable transaction 
storage. Any decent database will guarantee ACID compliance (atomicity, 
consistency, isolation, durability).

But sometimes you need to compromise on robustness for speed or convenience.


[...]
> This is how the program was originally structured, but we found performance
> problems with in.
> It's using Django and an Oracle DB, which is notoriously bad for single
> record read/writes. So, I
> figured why don't we append them all to an array, and write them out at the
> end of the program.

I suggest you periodically check the item log, and if there are more than 
(say) 20 items, you write them to the database and clear the temporary list. 
That way, the number of item logs should never get too large, and the 
performance shouldn't suffer too greatly. You will need to tweak that number 
to be more or less depending on how poorly the Oracle DB performs.

Then, at the very end of the program, you write whatever items are left in a 
finally clause, or atexit. That way, the finally clause should be nice and 
speedy and the user is unlikely to hit Ctrl-C a second time.

>> This will be the most robust as it will
>> also work for cases where the program is terminated without the use of
>> the keyboard (i.e. kill -9, task manager, computer reboot, etc.)  but

That unfortunately is not so. kill -9 does *not* send a signal or raise an 
exception. It just kills the process dead, instantly and without warning.

Likewise for cases when somebody trips over the power cord and disconnects the 
server from the power. (Really paranoid sys admins insist on having two power 
supplies connected to two different UPSes for their servers.)



>> it might also slow the program down. You could also try writing the
>> item log to a local file using pickle or csv.

Better is to use the logging module rather than to reinvent the wheel.

Do your item logs actually need to go in the database? Perhaps you could just 
write them to a local log file and leave the database for more critical records.



-- 
Steven


From steve at pearwood.info  Tue Jul 31 03:03:54 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 31 Jul 2012 11:03:54 +1000
Subject: [Tutor] finally without try or except
In-Reply-To: <CAOu0yXbijNmyPH5bf_9VTZ-GK362V6867VM=vNZbz-SMe4P0Gg@mail.gmail.com>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>	<CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>	<CAOu0yXbK+yun0mJ+3PScjcMsN_HsDm9Pkm6647Cn=2QAs0Nm5g@mail.gmail.com>	<5B80DD153D7D744689F57F4FB69AF47416593EDA@SCACMX008.exchad.jpmchase.net>	<CAOu0yXaAzh7L9X7y0CRVc9ZZbNDZsK_iNQg6cADiXakxNp+ZKA@mail.gmail.com>	<CAMb349wLAp4UCW4EF9L78LGwZ7_QrsHiN9UbkHHd=bqk65YQHQ@mail.gmail.com>
	<CAOu0yXbijNmyPH5bf_9VTZ-GK362V6867VM=vNZbz-SMe4P0Gg@mail.gmail.com>
Message-ID: <50172EFA.2040804@pearwood.info>

Tino Dai wrote:

> I just remember that there was a performance hit or something for doing
> that.
> Does that sound familiar to anyone?

Yes, and no.

Creating a try block in Python is lightweight and fast. It is about as fast as 
the "pass" statement, that is, pretty damn fast and more or less faster than 
doing *anything* at all. But technically, yes, there is overhead to try.

(This is in contrast to Java, where just creating a try block is expensive.)

But *catching* the exception is expensive. Not hugely expensive, it isn't like 
it takes a second or two, but it does take maybe ten microseconds[1] or so. If 
you're doing a LOT of exception-catching, all those microseconds can add up.

But for the use-case you're talking about, who cares if your program takes an 
extra ten microseconds, or even a millisecond? You're never going to notice.





[1] Number plucked out of thin air.


-- 
Steven


From aclark at aclark.net  Tue Jul 31 05:49:24 2012
From: aclark at aclark.net (Alex Clark)
Date: Mon, 30 Jul 2012 23:49:24 -0400
Subject: [Tutor] ANN: pythonpackages.com beta
Message-ID: <jv7kk3$tu7$1@dough.gmane.org>

Hi Python Tutor folks,


I am reaching out to various Python-related programming communities in
order to offer new help packaging your software.

If you have ever struggled with packaging and releasing Python software
(e.g. to PyPI), please check out this service:


- http://pythonpackages.com


The basic idea is to automate packaging by checking out code, testing,
and uploading (e.g. to PyPI) all through the web, as explained in this
introduction:


- http://docs.pythonpackages.com/en/latest/introduction.html


Also, I will be available to answer your Python packaging questions most
days/nights in #pythonpackages on irc.freenode.net. Hope to meet/talk
with all of you soon.



Alex



-- 
Alex Clark ? http://pythonpackages.com/ONE_CLICK


From ttmticdi at gmail.com  Tue Jul 31 11:52:07 2012
From: ttmticdi at gmail.com (ttmticdi .)
Date: Tue, 31 Jul 2012 12:52:07 +0300
Subject: [Tutor] Help please!
In-Reply-To: <CALnw1fSd-Vt_rpQx1kC9MUN6OMULAkx-uuu_dLFMYBPbq=QAtA@mail.gmail.com>
References: <1343664302.13185.YahooMailNeo@web29506.mail.ird.yahoo.com>
	<CALnw1fSd-Vt_rpQx1kC9MUN6OMULAkx-uuu_dLFMYBPbq=QAtA@mail.gmail.com>
Message-ID: <CAOToOwYsSG+3ksSZed1Uay0MXGdHu6DLC6AOxCriUG2Tos4ocQ@mail.gmail.com>

> > print "Mum is in a %s mood" % (mum_mood)
> > print "Dad is in a %s mood" % (dad_mood)
>
>


Hi Victoria!

Since you have only one format character in the strings above there is
no need to surround the variables mum_mood and dad_mood with
parenthesis.
You only do that when you have multiple formats in your string that
print multiple variables.

Ex:

print "Hi %s! You like %s and %s" (user_name, x, y)



>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor

From joel.goldstick at gmail.com  Tue Jul 31 13:36:54 2012
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 31 Jul 2012 07:36:54 -0400
Subject: [Tutor] Help please!
In-Reply-To: <CAOToOwYsSG+3ksSZed1Uay0MXGdHu6DLC6AOxCriUG2Tos4ocQ@mail.gmail.com>
References: <1343664302.13185.YahooMailNeo@web29506.mail.ird.yahoo.com>
	<CALnw1fSd-Vt_rpQx1kC9MUN6OMULAkx-uuu_dLFMYBPbq=QAtA@mail.gmail.com>
	<CAOToOwYsSG+3ksSZed1Uay0MXGdHu6DLC6AOxCriUG2Tos4ocQ@mail.gmail.com>
Message-ID: <CAPM-O+zB6Z7bcJbu9zCWv3eFufGN=kD972eW_Jw68vuFL1fHGA@mail.gmail.com>

On Tue, Jul 31, 2012 at 5:52 AM, ttmticdi . <ttmticdi at gmail.com> wrote:
>> > print "Mum is in a %s mood" % (mum_mood)
>> > print "Dad is in a %s mood" % (dad_mood)
>>
>>
>
>
> Hi Victoria!
>
> Since you have only one format character in the strings above there is
> no need to surround the variables mum_mood and dad_mood with
> parenthesis.
> You only do that when you have multiple formats in your string that
> print multiple variables.
>
> Ex:
>
No!
> print "Hi %s! You like %s and %s" (user_name, x, y)
>
Yes!
  print "Hi %s! You like %s and %s" % (user_name, x, y)
>
>
>>
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/mailman/listinfo/tutor
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor



-- 
Joel Goldstick

From ttmticdi at gmail.com  Tue Jul 31 13:45:24 2012
From: ttmticdi at gmail.com (ttmticdi .)
Date: Tue, 31 Jul 2012 14:45:24 +0300
Subject: [Tutor] Help please!
In-Reply-To: <CAPM-O+zB6Z7bcJbu9zCWv3eFufGN=kD972eW_Jw68vuFL1fHGA@mail.gmail.com>
References: <1343664302.13185.YahooMailNeo@web29506.mail.ird.yahoo.com>
	<CALnw1fSd-Vt_rpQx1kC9MUN6OMULAkx-uuu_dLFMYBPbq=QAtA@mail.gmail.com>
	<CAOToOwYsSG+3ksSZed1Uay0MXGdHu6DLC6AOxCriUG2Tos4ocQ@mail.gmail.com>
	<CAPM-O+zB6Z7bcJbu9zCWv3eFufGN=kD972eW_Jw68vuFL1fHGA@mail.gmail.com>
Message-ID: <CAOToOwZXLpGHnDRYChROXXC1zgpLNwkYUvdDmwd2C9T9CD8YoA@mail.gmail.com>

>> Ex:
>>
> No!
>> print "Hi %s! You like %s and %s" (user_name, x, y)
>>
> Yes!
>   print "Hi %s! You like %s and %s" % (user_name, x, y)
>>
>>
>>>

Forgot the interpolation operator(%). Thank you very much Joel for
correcting me.

Regards,
ttmticdi.

From learner404 at gmail.com  Tue Jul 31 14:02:26 2012
From: learner404 at gmail.com (learner404)
Date: Tue, 31 Jul 2012 14:02:26 +0200
Subject: [Tutor] Pydev + Remote System Explorer (RSE) : problems with local
 imports when working localy
Message-ID: <CAKib=N4uVHwaRU=CkYwA0sDnQBxHPk8o-OUyWer-2vE7Ua7nXA@mail.gmail.com>

Hello List,

I'm very happy with PyDev plugin for Eclipse as a Python IDE (I'm on
Eclipse 3.7, WinXP with latest Pydev).
Recently I've installed the Remote System Explorer plugin for Eclipse which
adds the ability to edit remote files on a server when I need it (awesome).

Unfortunately it's messing my ability to work on some Python
files locally on my PC (those with local imports):
File "C:\Documents and
Settings\myaccount\workspace\RemoteSystemsTempFiles\LOCALHOST\c\Documents
and Settings\myaccount\workspace\myprojet\myfile.py", line 50, in ?
ImportError: No module named mymodule
(it works only if I "python myfile.py" in the command shell)

I know some people of the list are using Eclipse and maybe they use RSE too.

Do you know if I can easily deactivate RSE so I can run a python script
normally (without the *localhost* inserting in the path) ?
Is there a configuration to do to keep both ?

I've tried to switch form perspective "remote systems" to perspective
"pydev" but I have the same problem.

Thanks.

francois
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120731/0a4794bb/attachment.html>

From amonroe at columbus.rr.com  Tue Jul 31 14:26:49 2012
From: amonroe at columbus.rr.com (R. Alan Monroe)
Date: Tue, 31 Jul 2012 08:26:49 -0400
Subject: [Tutor] Pydev + Remote System Explorer (RSE) : problems with
	local imports when working localy
In-Reply-To: <CAKib=N4uVHwaRU=CkYwA0sDnQBxHPk8o-OUyWer-2vE7Ua7nXA@mail.gmail.com>
References: <CAKib=N4uVHwaRU=CkYwA0sDnQBxHPk8o-OUyWer-2vE7Ua7nXA@mail.gmail.com>
Message-ID: <608022907.20120731082649@columbus.rr.com>

Hello learner404,

Tuesday, July 31, 2012, 8:02:26 AM, you wrote:

> Hello List,

> I'm very happy with PyDev plugin for Eclipse as a Python IDE (I'm
> on Eclipse 3.7, WinXP with latest Pydev).?
> File "C:\Documents and
> Settings\myaccount\workspace\RemoteSystemsTempFiles\LOCALHOST\c\Documents
> and Settings\myaccount\workspace\myprojet\myfile.py", line 50, in ?
> ImportError: No module named mymodule

Did you trying using forward slashes / rather than backslashes \ ?

Alan


From learner404 at gmail.com  Tue Jul 31 15:28:50 2012
From: learner404 at gmail.com (learner404)
Date: Tue, 31 Jul 2012 15:28:50 +0200
Subject: [Tutor] Pydev + Remote System Explorer (RSE) : problems with
 local imports when working localy
In-Reply-To: <CAKib=N4uVHwaRU=CkYwA0sDnQBxHPk8o-OUyWer-2vE7Ua7nXA@mail.gmail.com>
References: <CAKib=N4uVHwaRU=CkYwA0sDnQBxHPk8o-OUyWer-2vE7Ua7nXA@mail.gmail.com>
Message-ID: <CAKib=N784hq3sEa-iS6NprzoLoeC1mwDDp1UWzNYC-0TtsV8hg@mail.gmail.com>

hmmm ... menu "window">"Reset perspective" and "close all perspectives"
seems to have fix the problem.
Sorry for unnecessary mail.

On Tue, Jul 31, 2012 at 2:02 PM, learner404 <learner404 at gmail.com> wrote:

> Hello List,
>
> I'm very happy with PyDev plugin for Eclipse as a Python IDE (I'm on
> Eclipse 3.7, WinXP with latest Pydev).
> Recently I've installed the Remote System Explorer plugin for Eclipse
> which adds the ability to edit remote files on a server when I need it
> (awesome).
>
> Unfortunately it's messing my ability to work on some Python
> files locally on my PC (those with local imports):
> File "C:\Documents and
> Settings\myaccount\workspace\RemoteSystemsTempFiles\LOCALHOST\c\Documents
> and Settings\myaccount\workspace\myprojet\myfile.py", line 50, in ?
> ImportError: No module named mymodule
> (it works only if I "python myfile.py" in the command shell)
>
> I know some people of the list are using Eclipse and maybe they use RSE
> too.
>
> Do you know if I can easily deactivate RSE so I can run a python script
> normally (without the *localhost* inserting in the path) ?
> Is there a configuration to do to keep both ?
>
> I've tried to switch form perspective "remote systems" to perspective
> "pydev" but I have the same problem.
>
> Thanks.
>
> francois
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120731/646db193/attachment.html>

From oberoc at gmail.com  Tue Jul 31 16:32:05 2012
From: oberoc at gmail.com (Tino Dai)
Date: Tue, 31 Jul 2012 10:32:05 -0400
Subject: [Tutor] ImportError
Message-ID: <CAOu0yXYUwOuUXu8QD_aykmb5ZM0gBZPde6xmZH30dUYXF0C-1A@mail.gmail.com>

Hi All,

       I have been banging my head against a wall trying to figure it out.
I'm getting a ImportError on a
class that I know exists. I'm wondering if there is some dark corner of the
import mechanism that
I don't understand.

-Tino

Traceback (most recent call last):
  File "manage.py", line 14, in <module>
    execute_manager(settings)
  File
"/usr/local/lib/python2.6/dist-packages/Django-1.4-py2.6.egg/django/core/management/__init__.py",
line 459, in execute_manager
    utility.execute()
  File
"/usr/local/lib/python2.6/dist-packages/Django-1.4-py2.6.egg/django/core/management/__init__.py",
line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File
"/usr/local/lib/python2.6/dist-packages/Django-1.4-py2.6.egg/django/core/management/__init__.py",
line 261, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File
"/usr/local/lib/python2.6/dist-packages/Django-1.4-py2.6.egg/django/core/management/__init__.py",
line 69, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File
"/usr/local/lib/python2.6/dist-packages/Django-1.4-py2.6.egg/django/utils/importlib.py",
line 35, in import_module
    __import__(name)
  File
"/home/tdai/ProjectOne-TNT/leg_apps/../leg_apps/etl/management/commands/driver.py",
line 22, in <module>
    from etl.transfers.bill_subject import Bill_Subject
  File "/home/tdai/ProjectOne-TNT/leg_apps/etl/transfers/__init__.py", line
8, in <module>
    from api import models
  File "/home/tdai/ProjectOne-TNT/leg_apps/api/models.py", line 20, in
<module>
    from etl.transfers import eastern
ImportError: cannot import name eastern

< etl/transfers/__init__.py >
existing_pending_items = {}
new_pending_items = []

class eastern(object):
    @staticmethod
    def localize(theDate):
        if theDate:
            return easternTz.localize(theDate)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120731/3beb6180/attachment-0001.html>

From mail at timgolden.me.uk  Tue Jul 31 16:39:16 2012
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 31 Jul 2012 15:39:16 +0100
Subject: [Tutor] ImportError
In-Reply-To: <CAOu0yXYUwOuUXu8QD_aykmb5ZM0gBZPde6xmZH30dUYXF0C-1A@mail.gmail.com>
References: <CAOu0yXYUwOuUXu8QD_aykmb5ZM0gBZPde6xmZH30dUYXF0C-1A@mail.gmail.com>
Message-ID: <5017EE14.4010406@timgolden.me.uk>

On 31/07/2012 15:32, Tino Dai wrote:
> Hi All,
> 
>        I have been banging my head against a wall trying to figure it
> out. I'm getting a ImportError on a 
> class that I know exists. I'm wondering if there is some dark corner of
> the import mechanism that

Try running python with the -v parameter: and then "from etl.transfers
import eastern". That will at least confirm the file which Python is
reading for the import.

TJG


From oberoc at gmail.com  Tue Jul 31 16:44:29 2012
From: oberoc at gmail.com (Tino Dai)
Date: Tue, 31 Jul 2012 10:44:29 -0400
Subject: [Tutor] finally without try or except
In-Reply-To: <50172E81.2040206@pearwood.info>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>
	<CAOu0yXbK+yun0mJ+3PScjcMsN_HsDm9Pkm6647Cn=2QAs0Nm5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416593EDA@SCACMX008.exchad.jpmchase.net>
	<CAOu0yXaAzh7L9X7y0CRVc9ZZbNDZsK_iNQg6cADiXakxNp+ZKA@mail.gmail.com>
	<50172E81.2040206@pearwood.info>
Message-ID: <CAOu0yXadVv9gjbQjf1LKRcXPiwAu4j7K4RF_sGrCQukP4fgE9A@mail.gmail.com>

On Mon, Jul 30, 2012 at 9:01 PM, Steven D'Aprano <steve at pearwood.info>wrote:

> If you want to be robust, it is best not to try to beat the database. That
> means you should write to the database as soon as you can, as often as you
> need to, and let the database do what it does best: reliable transaction
> storage. Any decent database will guarantee ACID compliance (atomicity,
> consistency, isolation, durability).
>

> But sometimes you need to compromise on robustness for speed or
> convenience.
>
>
> [...]
>
>  This is how the program was originally structured, but we found
>> performance
>> problems with in.
>> It's using Django and an Oracle DB, which is notoriously bad for single
>> record read/writes. So, I
>> figured why don't we append them all to an array, and write them out at
>> the
>> end of the program.
>>
>
> I suggest you periodically check the item log, and if there are more than
> (say) 20 items, you write them to the database and clear the temporary
> list. That way, the number of item logs should never get too large, and the
> performance shouldn't suffer too greatly. You will need to tweak that
> number to be more or less depending on how poorly the Oracle DB performs.
>
> Then, at the very end of the program, you write whatever items are left in
> a finally clause, or atexit. That way, the finally clause should be nice
> and speedy and the user is unlikely to hit Ctrl-C a second time.
>
>
I think that is the process I'm going to take. It doesn't seem as elegant,
but I think it's a good compromise.


>
>  This will be the most robust as it will
>>> also work for cases where the program is terminated without the use of
>>> the keyboard (i.e. kill -9, task manager, computer reboot, etc.)  but
>>>
>>
> That unfortunately is not so. kill -9 does *not* send a signal or raise an
> exception. It just kills the process dead, instantly and without warning.
>
> Likewise for cases when somebody trips over the power cord and disconnects
> the server from the power. (Really paranoid sys admins insist on having two
> power supplies connected to two different UPSes for their servers.)
>
>
>
>
>  it might also slow the program down. You could also try writing the
>>> item log to a local file using pickle or csv.
>>>
>>
> Better is to use the logging module rather than to reinvent the wheel.
>
> Do your item logs actually need to go in the database? Perhaps you could
> just write them to a local log file and leave the database for more
> critical records.
>
>
Yes for reprocessing. :(


-Tino
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120731/1c582888/attachment.html>

From ramit.prasad at jpmorgan.com  Tue Jul 31 17:23:23 2012
From: ramit.prasad at jpmorgan.com (Prasad, Ramit)
Date: Tue, 31 Jul 2012 15:23:23 +0000
Subject: [Tutor] finally without try or except
In-Reply-To: <50172E81.2040206@pearwood.info>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>
	<CAOu0yXbK+yun0mJ+3PScjcMsN_HsDm9Pkm6647Cn=2QAs0Nm5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416593EDA@SCACMX008.exchad.jpmchase.net>
	<CAOu0yXaAzh7L9X7y0CRVc9ZZbNDZsK_iNQg6cADiXakxNp+ZKA@mail.gmail.com>
	<50172E81.2040206@pearwood.info>
Message-ID: <5B80DD153D7D744689F57F4FB69AF47416595E98@SCACMX008.exchad.jpmchase.net>

[snip]

> >> This will be the most robust as it will
> >> also work for cases where the program is terminated without the use of
> >> the keyboard (i.e. kill -9, task manager, computer reboot, etc.)  but
> 
> That unfortunately is not so. kill -9 does *not* send a signal or raise an
> exception. It just kills the process dead, instantly and without warning.

Does a plain kill send a signal or do you need to use an option
like SIGHUP?

[snip]
> 
> >> it might also slow the program down. You could also try writing the
> >> item log to a local file using pickle or csv.
> 
> Better is to use the logging module rather than to reinvent the wheel.
>

I was under the impression it was not logging in the traditional
sense of the word, but more like storable "status" so that reprocessing
can occur. I could be wrong though.

Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

From hugo.yoshi at gmail.com  Tue Jul 31 17:49:22 2012
From: hugo.yoshi at gmail.com (Hugo Arts)
Date: Tue, 31 Jul 2012 17:49:22 +0200
Subject: [Tutor] finally without try or except
In-Reply-To: <5B80DD153D7D744689F57F4FB69AF47416595E98@SCACMX008.exchad.jpmchase.net>
References: <CAOu0yXatZN=2vCMkaQnyQphsk+0O-1UZE6Myx8UfcEy9eb1MYw@mail.gmail.com>
	<CAKK8jXbabx-62J4+Xny-xYKKA1mJEQxCovBs9LZREcpj8sQdng@mail.gmail.com>
	<CAOu0yXbK+yun0mJ+3PScjcMsN_HsDm9Pkm6647Cn=2QAs0Nm5g@mail.gmail.com>
	<5B80DD153D7D744689F57F4FB69AF47416593EDA@SCACMX008.exchad.jpmchase.net>
	<CAOu0yXaAzh7L9X7y0CRVc9ZZbNDZsK_iNQg6cADiXakxNp+ZKA@mail.gmail.com>
	<50172E81.2040206@pearwood.info>
	<5B80DD153D7D744689F57F4FB69AF47416595E98@SCACMX008.exchad.jpmchase.net>
Message-ID: <CAJmBOfktEiFK5Prn99drOB91DBSMmoJsFkdziBcpXZaggq4Svg@mail.gmail.com>

On Tue, Jul 31, 2012 at 5:23 PM, Prasad, Ramit <ramit.prasad at jpmorgan.com>wrote:

> [snip]
>
> > >> This will be the most robust as it will
> > >> also work for cases where the program is terminated without the use of
> > >> the keyboard (i.e. kill -9, task manager, computer reboot, etc.)  but
> >
> > That unfortunately is not so. kill -9 does *not* send a signal or raise
> an
> > exception. It just kills the process dead, instantly and without warning.
>
> Does a plain kill send a signal or do you need to use an option
> like SIGHUP?
>
>
kill is supposed to always send a signal. Without options it will send the
process SIGTERM, which indicates to the process it should terminate. kill
-9 sends SIGKILL, which is kind of special since it is a signal that never
really arrives at the process itself. When the kernel notices the SIGKILL
it will just kill the process.

Hugo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120731/04ae1c93/attachment.html>

From eryksun at gmail.com  Tue Jul 31 18:26:14 2012
From: eryksun at gmail.com (eryksun)
Date: Tue, 31 Jul 2012 12:26:14 -0400
Subject: [Tutor] ImportError
In-Reply-To: <CAOu0yXYUwOuUXu8QD_aykmb5ZM0gBZPde6xmZH30dUYXF0C-1A@mail.gmail.com>
References: <CAOu0yXYUwOuUXu8QD_aykmb5ZM0gBZPde6xmZH30dUYXF0C-1A@mail.gmail.com>
Message-ID: <CACL+1asbMGw8RS2DmvwM7ddNFsr8x7bVv_2YxQeKaf1iwPz6kA@mail.gmail.com>

On Tue, Jul 31, 2012 at 10:32 AM, Tino Dai <oberoc at gmail.com> wrote:
>
>   File "/home/tdai/ProjectOne-TNT/leg_apps/etl/transfers/__init__.py", line
> 8, in <module>
>     from api import models
>   File "/home/tdai/ProjectOne-TNT/leg_apps/api/models.py", line 20, in
> <module>
>     from etl.transfers import eastern
> ImportError: cannot import name eastern

It looks to me like you have a circular import problem. Before the
etl.transfers package defines eastern it attempts to import
api.models, but executing the latter tries and fails to import (the as
yet undefined) eastern from etl.transfers. If this is the problem, you
could refactor the cyclic dependency into its own module, or move the
import to after eastern is defined, or do the import in
functions/methods that run after the modules have been loaded, etc.

From dfjennings at gmail.com  Tue Jul 31 19:42:34 2012
From: dfjennings at gmail.com (Don Jennings)
Date: Tue, 31 Jul 2012 13:42:34 -0400
Subject: [Tutor] finally without try or except
In-Reply-To: <mailman.9399.1343752016.4696.tutor@python.org>
References: <mailman.9399.1343752016.4696.tutor@python.org>
Message-ID: <0FE258E1-C367-4995-B4F2-9FA40C31627F@gmail.com>


On Jul 31, 2012, at 12:26 PM, tutor-request at python.org wrote:

> Message: 2
> Date: Tue, 31 Jul 2012 10:44:29 -0400
> From: Tino Dai <oberoc at gmail.com>
> To: "Steven D'Aprano" <steve at pearwood.info>
> Cc: "tutor at python.org" <tutor at python.org>
> Subject: Re: [Tutor] finally without try or except
> Message-ID:
> 	<CAOu0yXadVv9gjbQjf1LKRcXPiwAu4j7K4RF_sGrCQukP4fgE9A at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> On Mon, Jul 30, 2012 at 9:01 PM, Steven D'Aprano <steve at pearwood.info>wrote:
> 
>> If you want to be robust, it is best not to try to beat the database. That
>> means you should write to the database as soon as you can, as often as you
>> need to, and let the database do what it does best: reliable transaction
>> storage. Any decent database will guarantee ACID compliance (atomicity,
>> consistency, isolation, durability).
>> 
> 
>> But sometimes you need to compromise on robustness for speed or
>> convenience.
>> 
>> 
>> [...]
>> 
>> This is how the program was originally structured, but we found
>>> performance
>>> problems with in.
>>> It's using Django and an Oracle DB, which is notoriously bad for single
>>> record read/writes. So, I
>>> figured why don't we append them all to an array, and write them out at
>>> the
>>> end of the program.
>>> 
>> 
>> I suggest you periodically check the item log, and if there are more than
>> (say) 20 items, you write them to the database and clear the temporary
>> list. That way, the number of item logs should never get too large, and the
>> performance shouldn't suffer too greatly. You will need to tweak that
>> number to be more or less depending on how poorly the Oracle DB performs.
>> 
>> Then, at the very end of the program, you write whatever items are left in
>> a finally clause, or atexit. That way, the finally clause should be nice
>> and speedy and the user is unlikely to hit Ctrl-C a second time.
>> 
>> 
> I think that is the process I'm going to take. It doesn't seem as elegant,
> but I think it's a good compromise.


Another option might be to write each item to a different database which has better performance for single record writes?and, I'm betting someone on this list will have an excellent recommendation for such a DB. Then, you could batch the updates to the Oracle DB independently of the main program (the one related to your question).

Take care,
Don
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20120731/c0ef01d2/attachment.html>

attachments/20120731/c0ef01d2/attachment.html>