From marcus.luetolf at bluewin.ch  Fri Jan  1 05:21:15 2016
From: marcus.luetolf at bluewin.ch (=?utf-8?Q?marcus_l=C3=BCtolf?=)
Date: Fri, 1 Jan 2016 11:21:15 +0100
Subject: [Tutor] trouble with beautiful soup
In-Reply-To: <n63q49$3ru$1@ger.gmane.org>
References: <270401d13e6f$8e92c390$abb84ab0$@bluewin.ch>
 <CAKs9Esvkndvp1Qmi5oTaF=rAB2CDXrCe68L28=8LPPed+mNvsg@mail.gmail.com>
 <013601d13f10$6636d070$32a47150$@bluewin.ch>
 <CANLXbfAcjTqqHHwa3q7yHGrvVs2FZMLUcmmqVb3zGVAo=GyooQ@mail.gmail.com>
 <CANLXbfDKXgHzK_DjGA0e_mHG9raFPn8vYvj05sGZjxKZG78jYQ@mail.gmail.com>
 <0cf801d13fcf$e48d7130$ada85390$@bluewin.ch>
 <CAGZAPF7eR28MPLNThYB67mccMNYg_R3zbPTeCWGDPUAurEtQoQ@mail.gmail.com>
 <0d5701d14194$a821fce0$f865f6a0$@bluewin.ch> <n5s274$829$1@ger.gmane.org>
 <36d601d143cd$eb23e590$c16bb0b0$@bluewin.ch> <n63q49$3ru$1@ger.gmane.org>
Message-ID: <ef9b01d1447e$28a14240$79e3c6c0$@bluewin.ch>

Dear Pythonistas
Hi Alan,

yes, I typed it wrong(!!!!), "culpa mea". But the problem ist not quite solved, sorry to bother you further:
Althoug I get a postive message from the command window:

c:\Python27\Scripts>pip install beautifulsoup
Collecting beautifulsoup
  Downloading BeautifulSoup-3.2.1.tar.gz
Building wheels for collected packages: beautifulsoup
  Running setup.py bdist_wheel for beautifulsoup
  Stored in directory: C:\Users\marcus\AppData\Local\pip\Cache\wheels\0b\ca\15\397d8ca872cdde7f65c19b5cb26a67d14fd7ecdf13741b135a
Successfully built beautifulsoup
Installing collected packages: beautifulsoup
Successfully installed beautifulsoup-3.2.1

c:\Python27\Scripts>

But I get a traceback if I try to run beautifulsoup in Python:

Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec  5 2015, 20:32:19) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import beautifulsoup

Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import beutifulsoup
ImportError: No module named beautifulsoup
>>>

I get the same traceback even with wrong typing like 'beutifulsoup'.
That didn't happen on my laptop.
By the way, I just noticed in the first line of the shell it says  .....on win32. I'm running a 64bit w10 OS.
????

Thanks and regards, Marcus.
-----------------------------------------------------------------------------------------------------------------------------------------------


-----Urspr?ngliche Nachricht-----
Von: Tutor [mailto:tutor-bounces+marcus.luetolf=bluewin.ch at python.org] Im Auftrag von Alan Gauld
Gesendet: Donnerstag, 31. Dezember 2015 18:54
An: tutor at python.org
Betreff: Re: [Tutor] trouble with beautiful soup

On 31/12/15 13:19, marcus l?tolf wrote:

> c:\Python27\Scripts>pip install beutifulsoup

Is that really how you spell it?
Although I'd expect a different error if it isn't....

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist  -  Tutor at python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren gepr?ft.
https://www.avast.com/antivirus


From alan.gauld at btinternet.com  Fri Jan  1 06:24:35 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 1 Jan 2016 11:24:35 +0000
Subject: [Tutor] trouble with beautiful soup
In-Reply-To: <ef9b01d1447e$28a14240$79e3c6c0$@bluewin.ch>
References: <270401d13e6f$8e92c390$abb84ab0$@bluewin.ch>
 <CAKs9Esvkndvp1Qmi5oTaF=rAB2CDXrCe68L28=8LPPed+mNvsg@mail.gmail.com>
 <013601d13f10$6636d070$32a47150$@bluewin.ch>
 <CANLXbfAcjTqqHHwa3q7yHGrvVs2FZMLUcmmqVb3zGVAo=GyooQ@mail.gmail.com>
 <CANLXbfDKXgHzK_DjGA0e_mHG9raFPn8vYvj05sGZjxKZG78jYQ@mail.gmail.com>
 <0cf801d13fcf$e48d7130$ada85390$@bluewin.ch>
 <CAGZAPF7eR28MPLNThYB67mccMNYg_R3zbPTeCWGDPUAurEtQoQ@mail.gmail.com>
 <0d5701d14194$a821fce0$f865f6a0$@bluewin.ch> <n5s274$829$1@ger.gmane.org>
 <36d601d143cd$eb23e590$c16bb0b0$@bluewin.ch> <n63q49$3ru$1@ger.gmane.org>
 <ef9b01d1447e$28a14240$79e3c6c0$@bluewin.ch>
Message-ID: <568661F3.4040402@btinternet.com>

On 01/01/16 10:21, marcus l?tolf wrote:
> But I get a traceback if I try to run beautifulsoup in Python:
>
> Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec  5 2015, 20:32:19) [MSC v.1500 32 bit (Intel)] on win32
> Type "copyright", "credits" or "license()" for more information.

> By the way, I just noticed in the first line of the shell it says
> .....on win32. I'm running a 64bit w10 OS.

Looks like you installed 32 bit Python 2.7. That will run on 64bit OS
but not optimally.
If you re-install 2.7 with the 64 bit installer that should work.

Not sure about the import problem. Are you sure the imported
module/package name is beautifulsoup rather than bs or similar?


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


From dyoo at hashcollision.org  Fri Jan  1 13:17:28 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Fri, 1 Jan 2016 10:17:28 -0800
Subject: [Tutor] trouble with beautiful soup
In-Reply-To: <568661F3.4040402@btinternet.com>
References: <270401d13e6f$8e92c390$abb84ab0$@bluewin.ch>
 <CAKs9Esvkndvp1Qmi5oTaF=rAB2CDXrCe68L28=8LPPed+mNvsg@mail.gmail.com>
 <013601d13f10$6636d070$32a47150$@bluewin.ch>
 <CANLXbfAcjTqqHHwa3q7yHGrvVs2FZMLUcmmqVb3zGVAo=GyooQ@mail.gmail.com>
 <CANLXbfDKXgHzK_DjGA0e_mHG9raFPn8vYvj05sGZjxKZG78jYQ@mail.gmail.com>
 <0cf801d13fcf$e48d7130$ada85390$@bluewin.ch>
 <CAGZAPF7eR28MPLNThYB67mccMNYg_R3zbPTeCWGDPUAurEtQoQ@mail.gmail.com>
 <0d5701d14194$a821fce0$f865f6a0$@bluewin.ch> <n5s274$829$1@ger.gmane.org>
 <36d601d143cd$eb23e590$c16bb0b0$@bluewin.ch> <n63q49$3ru$1@ger.gmane.org>
 <ef9b01d1447e$28a14240$79e3c6c0$@bluewin.ch> <568661F3.4040402@btinternet.com>
Message-ID: <CAGZAPF7MwFGR=h14P702JCmKywB==n4tZbw5QiXvOOqbW6J_wA@mail.gmail.com>

According to the documentation for Beautiful Soup 3,

    http://www.crummy.com/software/BeautifulSoup/bs3/documentation.html

you must use one of the following:


    from BeautifulSoup import BeautifulSoup          # For processing HTML

OR

    from BeautifulSoup import BeautifulStoneSoup     # For processing XML

OR

    import BeautifulSoup                             # To get everything


Note that capitalization matters.

From marcus.luetolf at bluewin.ch  Fri Jan  1 07:04:54 2016
From: marcus.luetolf at bluewin.ch (=?utf-8?Q?marcus_l=C3=BCtolf?=)
Date: Fri, 1 Jan 2016 13:04:54 +0100
Subject: [Tutor] trouble with beautiful soup
In-Reply-To: <568661F3.4040402@btinternet.com>
References: <270401d13e6f$8e92c390$abb84ab0$@bluewin.ch>
 <CAKs9Esvkndvp1Qmi5oTaF=rAB2CDXrCe68L28=8LPPed+mNvsg@mail.gmail.com>
 <013601d13f10$6636d070$32a47150$@bluewin.ch>
 <CANLXbfAcjTqqHHwa3q7yHGrvVs2FZMLUcmmqVb3zGVAo=GyooQ@mail.gmail.com>
 <CANLXbfDKXgHzK_DjGA0e_mHG9raFPn8vYvj05sGZjxKZG78jYQ@mail.gmail.com>
 <0cf801d13fcf$e48d7130$ada85390$@bluewin.ch>
 <CAGZAPF7eR28MPLNThYB67mccMNYg_R3zbPTeCWGDPUAurEtQoQ@mail.gmail.com>
 <0d5701d14194$a821fce0$f865f6a0$@bluewin.ch> <n5s274$829$1@ger.gmane.org>
 <36d601d143cd$eb23e590$c16bb0b0$@bluewin.ch> <n63q49$3ru$1@ger.gmane.org>
 <ef9b01d1447e$28a14240$79e3c6c0$@bluewin.ch>
 <568661F3.4040402@btinternet.com>
Message-ID: <009d01d1448c$a34a49c0$e9dedd40$@bluewin.ch>

Thanks, I reinstalled the 64bit version but get the same traceback in Python Shell, also usin ..from bs4.
Regards, Marcus

-----Urspr?ngliche Nachricht-----
Von: Alan Gauld [mailto:alan.gauld at btinternet.com] 
Gesendet: Freitag, 1. Januar 2016 12:25
An: marcus l?tolf <marcus.luetolf at bluewin.ch>; tutor at python.org
Betreff: Re: AW: [Tutor] trouble with beautiful soup

On 01/01/16 10:21, marcus l?tolf wrote:
> But I get a traceback if I try to run beautifulsoup in Python:
>
> Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec  5 2015, 20:32:19) [MSC 
> v.1500 32 bit (Intel)] on win32 Type "copyright", "credits" or "license()" for more information.

> By the way, I just noticed in the first line of the shell it says 
> .....on win32. I'm running a 64bit w10 OS.

Looks like you installed 32 bit Python 2.7. That will run on 64bit OS but not optimally.
If you re-install 2.7 with the 64 bit installer that should work.

Not sure about the import problem. Are you sure the imported module/package name is beautifulsoup rather than bs or similar?


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren gepr?ft.
https://www.avast.com/antivirus


From robertvstepp at gmail.com  Fri Jan  1 15:41:49 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Fri, 1 Jan 2016 14:41:49 -0600
Subject: [Tutor] OT: How best to use Git for multi-directory projects?
In-Reply-To: <alpine.LSU.2.11.1512292050250.20208@qnttre.jbaqresebt.arg>
References: <CANDiX9JeQ2X5xfvo_1xYne6-KcCpzSao5NV=ovWi-quR0xPZpw@mail.gmail.com>
 <alpine.LSU.2.11.1512292050250.20208@qnttre.jbaqresebt.arg>
Message-ID: <CANDiX9JY0eeoTEtjiJ+nv2aD+1rBvLWvRz-c8O2QwhgppCTW_g@mail.gmail.com>

On Tue, Dec 29, 2015 at 11:32 PM, Martin A. Brown <martin at linux-ip.net> wrote:
>
> Hello there boB,
>
> Hey, wait a second!  What time zone are you in?  You can't have 2016
> resolutions already.  Not even the New Zealanders are there yet!

Santa gave me a peek into you know who's time machine!

>>My thoughts are that each project should have its own .git file.
>
> N.B. this is not a Python question, but ... do you mean .git
> directory?

My bad!  Of course, you're right!

>>I WILL study--in parallel--the following three books from cover to
>>cover, doing ALL non-trivial exercises:
>>    "Think Python, 2nd Edition" by Allen Downey
>>    "Introduction to Computation and Programming Using Python, Revised and Expanded Edition" by John V. Guttag
>>    "Python Crash Course" by Eric Matthes
>
> (Side note:  I do not know any of these three books.)

I have now acquired so many Python books, that I need to stop
dithering around and actually study one or more thoroughly, while
doing actual programming.  The three above may or may not be best
choices, but surely they're better than my current Easter-egging
approach to Python?!?

The first book is available both in print and online.  Its Python
version started out as "How to Think Like a Computer Scientist:
Learning with Python" and now has been updated for Python 3.  Its
online version can be found at:
http://www.greenteapress.com/thinkpython/

The second book is loosely the course text for MIT's Open Courseware
course, Introduction to Computer Science and Programming at:
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-00sc-introduction-to-computer-science-and-programming-spring-2011/index.htm

The third just came out in print and I just liked how the author wrote.

> Reading cover to cover is one way to go about it.  I think that it
> helps to balance such thorough and complete study of books with the
> direct experience of writing software.

Hopefully these books and MIT's course will have challenging problem
sets for me to do.

> Also, I might suggest one other reference and technique--it is more
> a generic strategy for continuous learning, here applied to
> increasing your Python proficiency.
>
> Pick one standard library module per week (or day) and read the
> whole page of documentation.  Pick a module that is relevant to some
> problem you are solving and study each function of the module.

This is a really good suggestion!

> I'm accustomed to keeping a single git repository for each distinct
> project or program I'm working on.  In the context of Python, I
> create a separate git repository for each distribution that I may
> release.  (So, if I am going to write a new "setup.py", then it is
> time for a new git repository.)
>
> To apply this approach to your case, I would make a different git
> repository for each StudyBook.

Looks like I was thinking along the correct lines for a change.
Thanks for taking the time to respond!

>
> P.S. Two questions:  should I buy some some YHOO stock and should I
>    sell my Euros?

Alas!  The new year is upon us and I have been tardy catching up on my
email reading...

Happy New Year!
-- 
boB

From robertvstepp at gmail.com  Fri Jan  1 23:26:11 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Fri, 1 Jan 2016 22:26:11 -0600
Subject: [Tutor] What potential problems might I encounter installing
 Anaconda?
Message-ID: <CANDiX9KjWCQab__0yhYyX4dqmN8SkXuiNTxhMyXFU24cQnLKOA@mail.gmail.com>

Currently I have one version of Python installed, v. 3.4.4.  My OS is
W7-64bit.  I use Vim 7.4 for my editing.  Two of the textbooks I have
started will use several Python scientific computing packages.
Anaconda seems the way to go to get them all (plus others) in one fell
swoop.  Questions:

1) At this time I have no need for multiple versions of Python on my
home PC.  Would it be advisable for me to uninstall my existing v.
3.4.4 as the Anaconda distribution I am looking at uses v. 3.5?

2) Will installing Anaconda cause any problems with my existing Vim
7.4 installation?  At this stage of my learning, I have not installed
any Python-related plugins in Vim.  However, I do value the existing
syntax highlighting Vim does for Python out of the box.  Will I need
to reinstall Vim after installing Anaconda for Vim to be able to find
Python?

3) Is there anything else I need to be aware of regarding Anaconda?

4) I have only tonight started looking into this.  My initial
impression is that updates and new Python package installations are
done in Anaconda with a "conda" command.  Does this mean that pip
installations can no longer be done?

TIA!
Happy New Year!

-- 
boB

From robertvstepp at gmail.com  Sat Jan  2 01:46:01 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 2 Jan 2016 00:46:01 -0600
Subject: [Tutor] antigravity?!?
Message-ID: <CANDiX9Km7dS6E_3gRfAJSiNA+h8zcNR_3OBO5Nz5QRvFJ4hF4w@mail.gmail.com>

Ok, you Python developers (If any follow the Tutor list.) are a quirky
lot!  Thinking about Martin's suggestion to explore the standard
library, I typed "py" in my W7 Command Prompt, followed by "help()"
and then modules to see what modules I had installed on my system.
While doing so, I was intrigued by one called, "antigravity".  Hmm.
So I next typed "antigravity" to see what this thing was.  I was
immediately startled!  My Chrome browser, open next to the Command
Prompt window, suddenly went to the xkcd web site!  Huh?  Well, that
comic explains "antigravity".  But it does not explain the rest of the
help entry:

NAME
    antigravity

FUNCTIONS
    geohash(latitude, longitude, datedow)
        Compute geohash() using the Munroe algorithm.

        >>> geohash(37.421542, -122.085589, b'2005-05-26-10458.68')
        37.857713 -122.544543

FILE
    c:\python34\lib\antigravity.py

Out of ever growing curiosity I went to the official Python docs site.
Sure enough, no module named "antigravity" is listed.

What else awaits as I explore Python???  I sure hope I don't stumble
across one named something like "clean_windows"!!!

Happy New Year!
-- 
boB

From alan.gauld at btinternet.com  Sat Jan  2 04:32:57 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 2 Jan 2016 09:32:57 +0000
Subject: [Tutor] What potential problems might I encounter installing
 Anaconda?
In-Reply-To: <CANDiX9KjWCQab__0yhYyX4dqmN8SkXuiNTxhMyXFU24cQnLKOA@mail.gmail.com>
References: <CANDiX9KjWCQab__0yhYyX4dqmN8SkXuiNTxhMyXFU24cQnLKOA@mail.gmail.com>
Message-ID: <n685g9$m6c$1@ger.gmane.org>

On 02/01/16 04:26, boB Stepp wrote:

> 1) At this time I have no need for multiple versions of Python on my
> home PC.  Would it be advisable for me to uninstall my existing v.
> 3.4.4 as the Anaconda distribution I am looking at uses v. 3.5?

It shouldn';t make any difference. Anaconda and v3.4 can coexist.
But removing the older distro should be fine too, it just makes anaconda
your default for all Python scripts.

> 2) Will installing Anaconda cause any problems with my existing Vim
> 7.4 installation?  

No.
> any Python-related plugins in Vim.  However, I do value the existing
> syntax highlighting Vim does for Python out of the box.  

The syntax highlighting is not powered by Python, its simply
a bunch of regex patterns. Depending on your vim there may be
a Python interpreter to execute Python macros but I believe
that's implemented as a library inside Vim it does not use
the system interpreter. So even if you deleted all Pythons
from your PC vim would not be impacted.


> to reinstall Vim after installing Anaconda for Vim to be able to find
> Python?

No.

> 3) Is there anything else I need to be aware of regarding Anaconda?
> 4) ...that updates and new Python package installations are
> done in Anaconda with a "conda" command.  Does this mean that pip
> installations can no longer be done?


Sorry I can't answer that, you'll need an Anaconda user I guess.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Sat Jan  2 06:12:00 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 2 Jan 2016 22:12:00 +1100
Subject: [Tutor] antigravity?!?
In-Reply-To: <CANDiX9Km7dS6E_3gRfAJSiNA+h8zcNR_3OBO5Nz5QRvFJ4hF4w@mail.gmail.com>
References: <CANDiX9Km7dS6E_3gRfAJSiNA+h8zcNR_3OBO5Nz5QRvFJ4hF4w@mail.gmail.com>
Message-ID: <20160102111159.GY23700@ando.pearwood.info>

On Sat, Jan 02, 2016 at 12:46:01AM -0600, boB Stepp wrote:
> Ok, you Python developers (If any follow the Tutor list.) are a quirky
> lot!  Thinking about Martin's suggestion to explore the standard
> library, I typed "py" in my W7 Command Prompt, followed by "help()"
> and then modules to see what modules I had installed on my system.
> While doing so, I was intrigued by one called, "antigravity".  Hmm.
> So I next typed "antigravity" to see what this thing was.  I was
> immediately startled!  My Chrome browser, open next to the Command
> Prompt window, suddenly went to the xkcd web site!  Huh?  Well, that
> comic explains "antigravity".  But it does not explain the rest of the
> help entry:

Geohashing is explained by one of the other XKCD cartoons. You'll need 
to google for it, I don't remember what it does.

[...]
> What else awaits as I explore Python??? 

antigravity is an Easter Egg.

You can also try:

import this

from __future__ import braces

from __future__ import barry_as_FLUFL


(the last one is *very* subtle, it might take you a while to work out 
what it does).


There may be a few others, I don't remember.


-- 
Steve

From marcus.luetolf at bluewin.ch  Sat Jan  2 04:37:02 2016
From: marcus.luetolf at bluewin.ch (=?utf-8?Q?marcus_l=C3=BCtolf?=)
Date: Sat, 2 Jan 2016 10:37:02 +0100
Subject: [Tutor] trouble with beautiful soup
In-Reply-To: <CAGZAPF7MwFGR=h14P702JCmKywB==n4tZbw5QiXvOOqbW6J_wA@mail.gmail.com>
References: <270401d13e6f$8e92c390$abb84ab0$@bluewin.ch>
 <CAKs9Esvkndvp1Qmi5oTaF=rAB2CDXrCe68L28=8LPPed+mNvsg@mail.gmail.com>
 <013601d13f10$6636d070$32a47150$@bluewin.ch>
 <CANLXbfAcjTqqHHwa3q7yHGrvVs2FZMLUcmmqVb3zGVAo=GyooQ@mail.gmail.com>
 <CANLXbfDKXgHzK_DjGA0e_mHG9raFPn8vYvj05sGZjxKZG78jYQ@mail.gmail.com>
 <0cf801d13fcf$e48d7130$ada85390$@bluewin.ch>
 <CAGZAPF7eR28MPLNThYB67mccMNYg_R3zbPTeCWGDPUAurEtQoQ@mail.gmail.com>
 <0d5701d14194$a821fce0$f865f6a0$@bluewin.ch> <n5s274$829$1@ger.gmane.org>
 <36d601d143cd$eb23e590$c16bb0b0$@bluewin.ch> <n63q49$3ru$1@ger.gmane.org>
 <ef9b01d1447e$28a14240$79e3c6c0$@bluewin.ch>
 <568661F3.4040402@btinternet.com>
 <CAGZAPF7MwFGR=h14P702JCmKywB==n4tZbw5QiXvOOqbW6J_wA@mail.gmail.com>
Message-ID: <b82e01d14541$25d01640$717042c0$@bluewin.ch>


-----Urspr?ngliche Nachricht-----
Von: Danny Yoo [mailto:dyoo at hashcollision.org] 
Gesendet: Freitag, 1. Januar 2016 19:17
An: Alan Gauld <alan.gauld at btinternet.com>
Cc: marcus l?tolf <marcus.luetolf at bluewin.ch>; Python Tutor Mailing List <tutor at python.org>
Betreff: Re: [Tutor] trouble with beautiful soup

According to the documentation for Beautiful Soup 3,

    http://www.crummy.com/software/BeautifulSoup/bs3/documentation.html

you must use one of the following:


    from BeautifulSoup import BeautifulSoup          # For processing HTML

OR

    from BeautifulSoup import BeautifulStoneSoup     # For processing XML

OR

    import BeautifulSoup                             # To get everything


Note that capitalization matters.


Yes !!! Thank you all again, case closed, Marcus.


---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren gepr?ft.
https://www.avast.com/antivirus


From katye2007 at gmail.com  Sat Jan  2 07:45:30 2016
From: katye2007 at gmail.com (yehudak .)
Date: Sat, 2 Jan 2016 14:45:30 +0200
Subject: [Tutor] Hi there,
Message-ID: <CAE3ie42ZXR6XRXCORHgF61+fP4-WR7VFj5k=wkT5JRVeFm6v3Q@mail.gmail.com>

I'm trying to write a Python 3.5 program to find how many trailing zeros
are in 100! (factorial of 100).
I downloaded factorial from Math module, but all my efforts to solve the
problem failed.

I know the mathematical way to solve it (resulting in 24), but I want a
Python solution.

Thank you.

From vrkratheesh at live.com  Sat Jan  2 09:33:11 2016
From: vrkratheesh at live.com (Ratheesh kumar)
Date: Sat, 2 Jan 2016 20:03:11 +0530
Subject: [Tutor] Difference between rounding off and typecasting to int
Message-ID: <SNT148-W443CAB52FD1F758A816C7AD8F00@phx.gbl>

Hii everyone, Today i was just solving a problem in hacker-rank. A simple one to calculate the tip and tax for the meal. The resultant answer should be rounded off. I first wrote the code as below:
m=float(input())
x=int(input())
t=int(input())
tip=(m*x)/100
tax=(m*t)/100
total=m+tip+tax
print("The final price of the meal is $",end='')
print(int(total),end='')
print('.')
It passed three test-cases but the last test continued to fail. Then i came to know that the problem was with rounding off. Then i edited my code to the following:
m=float(input())
x=int(input())
t=int(input())
tip=(m*x)/100
tax=(m*t)/100
total=m+tip+tax
print("The final price of the meal is $",end='')
print(round(total),end='')
print('.')Finally all test-cases passed. But I can't get to understand what round() did int() cant't do to the variable total. Can anyone pinpoint the change. Thank you.











 		 	   		  

From alan.gauld at btinternet.com  Sat Jan  2 10:17:43 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 2 Jan 2016 15:17:43 +0000
Subject: [Tutor] Difference between rounding off and typecasting to int
In-Reply-To: <SNT148-W443CAB52FD1F758A816C7AD8F00@phx.gbl>
References: <SNT148-W443CAB52FD1F758A816C7AD8F00@phx.gbl>
Message-ID: <n68pmm$ev3$1@ger.gmane.org>

On 02/01/16 14:33, Ratheesh kumar wrote:
> But I can't get to understand what round() did int() cant't do 

With this kind of question its best to ask the interpreter:

>>> int(12.1)
12
>>> int(12.9)
12
>>> round(12.1)
12
>>> round(12.9)
13
>>>

Does that make it clearer?

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Sat Jan  2 10:21:45 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 2 Jan 2016 15:21:45 +0000
Subject: [Tutor] Hi there,
In-Reply-To: <CAE3ie42ZXR6XRXCORHgF61+fP4-WR7VFj5k=wkT5JRVeFm6v3Q@mail.gmail.com>
References: <CAE3ie42ZXR6XRXCORHgF61+fP4-WR7VFj5k=wkT5JRVeFm6v3Q@mail.gmail.com>
Message-ID: <n68pu8$j8i$1@ger.gmane.org>

On 02/01/16 12:45, yehudak . wrote:


> I know the mathematical way to solve it (resulting in 24), but I want a
> Python solution.

Show us your code.

Usually "the mathematical way to do it" works in Python too.
Although there will likely be other ways that may sometimes
run faster or easier to code.

But until we see your code we can't comment on what you
are doing.

One way to try would be continually dividing by 10 until you
get a non-zero remainder. Count the number of divisions
necessary. Hint: The divmod() function may help here.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Sat Jan  2 10:47:38 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 3 Jan 2016 02:47:38 +1100
Subject: [Tutor] Hi there,
In-Reply-To: <CAE3ie42ZXR6XRXCORHgF61+fP4-WR7VFj5k=wkT5JRVeFm6v3Q@mail.gmail.com>
References: <CAE3ie42ZXR6XRXCORHgF61+fP4-WR7VFj5k=wkT5JRVeFm6v3Q@mail.gmail.com>
Message-ID: <20160102154737.GB23700@ando.pearwood.info>

On Sat, Jan 02, 2016 at 02:45:30PM +0200, yehudak . wrote:
> I'm trying to write a Python 3.5 program to find how many trailing zeros
> are in 100! (factorial of 100).
> I downloaded factorial from Math module, but all my efforts to solve the
> problem failed.
> 
> I know the mathematical way to solve it (resulting in 24), but I want a
> Python solution.

Here are the steps needed:

http://www.purplemath.com/modules/factzero.htm

Try writing some Python code for this, and if you have trouble, show us 
your code and we'll help. But you have to write it first -- we won't do 
your homework for you.



-- 
Steve

From katye2007 at gmail.com  Sun Jan  3 07:27:01 2016
From: katye2007 at gmail.com (yehudak .)
Date: Sun, 3 Jan 2016 14:27:01 +0200
Subject: [Tutor] To FORMAT or not to
Message-ID: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>

Hi there,
In a program I wrote the following line (Python 3.5):

print("You've visited", island, '&', new + ".")

A programmer told me that it's a bad habit, and I should have used instead:

print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))

May I understand why?

From kwpolska at gmail.com  Sun Jan  3 08:04:22 2016
From: kwpolska at gmail.com (Chris Warrick)
Date: Sun, 3 Jan 2016 14:04:22 +0100
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
Message-ID: <CAMw+j7+RTGqSFOQg=Fohx0xohV8ZFc3JJSn3GES2eMz-_TRCPw@mail.gmail.com>

On 3 January 2016 at 13:27, yehudak . <katye2007 at gmail.com> wrote:
> Hi there,
> In a program I wrote the following line (Python 3.5):
>
> print("You've visited", island, '&', new + ".")
>
> A programmer told me that it's a bad habit, and I should have used instead:
>
> print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
>
> May I understand why?
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

The programmer was not very intelligent in his use of str.format in
the first place. A more sensible way to write this is:

print("You've visited {0} & {1}.".format(island, new))

Formatting with constant strings is pointless, just include it in the
original input. However, string formatting is not.

Here are a couple of reasons:
* String formatting works everywhere, but this syntax is specific to
print() ? if you use something else, you might end up producing faulty
code
* The corrected string formatting usage is more readable than the
original print()
* String concatenation with + requires that all arguments are strings,
which is even less readable
* With string formatting, you can apply special formatting to your
inputs (eg. set width, number precision?), which is hard or impossible
with print()
* Using print() with commas adds spaces between all entries, which
might look bad (and it does in this example); the only way to prevent
that is by setting `sep=`, but then you need to remember about a space
after "visited" and around the ampersand?
* Easy to localize (translate into different languages), which is
generally impossible with any of the other options (some languages
might rearrange the sentence!)

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16

From francois.dion at gmail.com  Sun Jan  3 08:07:06 2016
From: francois.dion at gmail.com (Francois Dion)
Date: Sun, 3 Jan 2016 08:07:06 -0500
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
Message-ID: <CAOLi1KAabyvxJUNgQB1NoOnv__r2Xi_e4Zq=z6FOvtRxqdBriw@mail.gmail.com>

The answer is neither. The second shows the intent in part but doesn't
quite get it right.

The intent is to have a string template and insert values in that template:

print("You've visited {} & {}.".format(island, new)

This is totally clear what is going to happen. I'm not relying on the
behaviour of print() to format my string. Format does the formating, print
the printing :)

This separation of concern is a basic building block of good code and is
seen at various scale levels. At a higher level, it is seen in concepts
like MVC.

Another thing you do by having your string separated, is you could have
them defined elsewhere and have, say, a version in english and a version in
french.

I;m sure you can see the value.

Francois

On Sun, Jan 3, 2016 at 7:27 AM, yehudak . <katye2007 at gmail.com> wrote:

> Hi there,
> In a program I wrote the following line (Python 3.5):
>
> print("You've visited", island, '&', new + ".")
>
> A programmer told me that it's a bad habit, and I should have used instead:
>
> print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
>
> May I understand why?
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
raspberry-python.blogspot.com - www.pyptug.org - www.3DFutureTech.info -
@f_dion

From __peter__ at web.de  Sun Jan  3 08:09:24 2016
From: __peter__ at web.de (Peter Otten)
Date: Sun, 03 Jan 2016 14:09:24 +0100
Subject: [Tutor] To FORMAT or not to
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
Message-ID: <n6b6i7$g37$1@ger.gmane.org>

yehudak . wrote:

> Hi there,
> In a program I wrote the following line (Python 3.5):
> 
> print("You've visited", island, '&', new + ".")
> 
> A programmer told me that it's a bad habit, and I should have used
> instead:
> 
> print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
> 
> May I understand why?

I don't see the benefits either. If anything I'd move the constants into the 
format string:

print("You've visited {island}, & {new}.".format(island=island, new=new))

If you use the same names in your format string and your code it should 
suffice to read the format string to get an idea of what will be printed.

In future versions of Python you can simplify it to

print(f"You've visited {island}, & {new}.")

https://www.python.org/dev/peps/pep-0498/


From francois.dion at gmail.com  Sun Jan  3 08:10:09 2016
From: francois.dion at gmail.com (Francois Dion)
Date: Sun, 3 Jan 2016 08:10:09 -0500
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <CAMw+j7+RTGqSFOQg=Fohx0xohV8ZFc3JJSn3GES2eMz-_TRCPw@mail.gmail.com>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <CAMw+j7+RTGqSFOQg=Fohx0xohV8ZFc3JJSn3GES2eMz-_TRCPw@mail.gmail.com>
Message-ID: <CAOLi1KB8s53RDKbiSFB-B6_-T7JivZ96DAwFYM+Z+6DDQnYtsQ@mail.gmail.com>

And as Chris points out, if there is any possibility that the words will be
in a different order in a different language, use {0}, {1} instead of {}.


Francois

On Sun, Jan 3, 2016 at 8:04 AM, Chris Warrick <kwpolska at gmail.com> wrote:

> On 3 January 2016 at 13:27, yehudak . <katye2007 at gmail.com> wrote:
> > Hi there,
> > In a program I wrote the following line (Python 3.5):
> >
> > print("You've visited", island, '&', new + ".")
> >
> > A programmer told me that it's a bad habit, and I should have used
> instead:
> >
> > print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
> >
> > May I understand why?
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > https://mail.python.org/mailman/listinfo/tutor
>
> The programmer was not very intelligent in his use of str.format in
> the first place. A more sensible way to write this is:
>
> print("You've visited {0} & {1}.".format(island, new))
>
> Formatting with constant strings is pointless, just include it in the
> original input. However, string formatting is not.
>
> Here are a couple of reasons:
> * String formatting works everywhere, but this syntax is specific to
> print() ? if you use something else, you might end up producing faulty
> code
> * The corrected string formatting usage is more readable than the
> original print()
> * String concatenation with + requires that all arguments are strings,
> which is even less readable
> * With string formatting, you can apply special formatting to your
> inputs (eg. set width, number precision?), which is hard or impossible
> with print()
> * Using print() with commas adds spaces between all entries, which
> might look bad (and it does in this example); the only way to prevent
> that is by setting `sep=`, but then you need to remember about a space
> after "visited" and around the ampersand?
> * Easy to localize (translate into different languages), which is
> generally impossible with any of the other options (some languages
> might rearrange the sentence!)
>
> --
> Chris Warrick <https://chriswarrick.com/>
> PGP: 5EAAEA16
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
raspberry-python.blogspot.com - www.pyptug.org - www.3DFutureTech.info -
@f_dion

From alan.gauld at btinternet.com  Sun Jan  3 08:12:41 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 3 Jan 2016 13:12:41 +0000
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
Message-ID: <n6b6o8$isq$1@ger.gmane.org>

On 03/01/16 12:27, yehudak . wrote:
> Hi there,
> In a program I wrote the following line (Python 3.5):
> 
> print("You've visited", island, '&', new + ".")
> 
> A programmer told me that it's a bad habit, and I should have used instead:
> 
> print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
> 
> May I understand why?

There are several reasons although your technique is far from
the worst way of doing things. And the format string here would probably
be better written as:

print("You've visited {0} & {2}.".format(island, new))

ie only put the variables as placeholders.

Why is it better?
1) It is slightly more performant. String addition and
concatenation are relatively slow processes in Python.
Formatting will usually be slightly faster. This is not
a good reason in itself (and in your case with a single
print it's probably irrelevant) but it's one factor. It
does matter more if you are printing inside a loop with
many variables and long strings. (For example assembling
a web page). So it's a good habit to adopt.

2) It improves consistency. You can store the format
string as a variable and then print it out many times
from different parts of your code and it will always
be the same. This makes it much easier to maintain
your program. For example:

    fmtString = "You've visited {0} & {2}."
    if foo:
      print(fmtString.format(foo,bar))
    else:
      print(fmtString.format(baz,bad))

Now if you want to change the message you only need to
change the text in one place rather than searching your
code to do an edit. Also the user sees exactly the same
formatting, no extra spaces in one message compared
to another for example.

3) formatting provides many extra features to control
justification, leading and spacing. This is especially
important for numeric output where you can define the
number of decimal places, whether a leading sign is
included, padding with zeros etc, etc. You may not
need that initially but if you do have to add it in,
it's trivial with a format string but much more work
if you have lots of hard coded messages all over your
code.

There may be other reasons too but that should give
you some ideas.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From katye2007 at gmail.com  Sun Jan  3 09:40:05 2016
From: katye2007 at gmail.com (yehudak .)
Date: Sun, 3 Jan 2016 16:40:05 +0200
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <CAOLi1KB8s53RDKbiSFB-B6_-T7JivZ96DAwFYM+Z+6DDQnYtsQ@mail.gmail.com>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <CAMw+j7+RTGqSFOQg=Fohx0xohV8ZFc3JJSn3GES2eMz-_TRCPw@mail.gmail.com>
 <CAOLi1KB8s53RDKbiSFB-B6_-T7JivZ96DAwFYM+Z+6DDQnYtsQ@mail.gmail.com>
Message-ID: <CAE3ie43fSG+qTu__r74kAwyaM0mTCtv0crqa64-UqY-1LdnasA@mail.gmail.com>

Important point. Thanks again.
Yehuda

On Sun, Jan 3, 2016 at 3:10 PM, Francois Dion <francois.dion at gmail.com>
wrote:

> And as Chris points out, if there is any possibility that the words will
> be in a different order in a different language, use {0}, {1} instead of {}.
>
>
> Francois
>
> On Sun, Jan 3, 2016 at 8:04 AM, Chris Warrick <kwpolska at gmail.com> wrote:
>
>> On 3 January 2016 at 13:27, yehudak . <katye2007 at gmail.com> wrote:
>> > Hi there,
>> > In a program I wrote the following line (Python 3.5):
>> >
>> > print("You've visited", island, '&', new + ".")
>> >
>> > A programmer told me that it's a bad habit, and I should have used
>> instead:
>> >
>> > print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
>> >
>> > May I understand why?
>> > _______________________________________________
>> > Tutor maillist  -  Tutor at python.org
>> > To unsubscribe or change subscription options:
>> > https://mail.python.org/mailman/listinfo/tutor
>>
>> The programmer was not very intelligent in his use of str.format in
>> the first place. A more sensible way to write this is:
>>
>> print("You've visited {0} & {1}.".format(island, new))
>>
>> Formatting with constant strings is pointless, just include it in the
>> original input. However, string formatting is not.
>>
>> Here are a couple of reasons:
>> * String formatting works everywhere, but this syntax is specific to
>> print() ? if you use something else, you might end up producing faulty
>> code
>> * The corrected string formatting usage is more readable than the
>> original print()
>> * String concatenation with + requires that all arguments are strings,
>> which is even less readable
>> * With string formatting, you can apply special formatting to your
>> inputs (eg. set width, number precision?), which is hard or impossible
>> with print()
>> * Using print() with commas adds spaces between all entries, which
>> might look bad (and it does in this example); the only way to prevent
>> that is by setting `sep=`, but then you need to remember about a space
>> after "visited" and around the ampersand?
>> * Easy to localize (translate into different languages), which is
>> generally impossible with any of the other options (some languages
>> might rearrange the sentence!)
>>
>> --
>> Chris Warrick <https://chriswarrick.com/>
>> PGP: 5EAAEA16
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
>
>
> --
> raspberry-python.blogspot.com - www.pyptug.org - www.3DFutureTech.info -
> @f_dion
>

From katye2007 at gmail.com  Sun Jan  3 09:35:24 2016
From: katye2007 at gmail.com (yehudak .)
Date: Sun, 3 Jan 2016 16:35:24 +0200
Subject: [Tutor] Tutor Digest, Vol 143, Issue 4
In-Reply-To: <mailman.1549.1451826774.11928.tutor@python.org>
References: <mailman.1549.1451826774.11928.tutor@python.org>
Message-ID: <CAE3ie42D7RDJX2OY6MMFKGtmLQVuTm=J=DxQmY+5JALT_rWJNg@mail.gmail.com>

Thank you ALL for your kind help.

Yehuda

On Sun, Jan 3, 2016 at 3:12 PM, <tutor-request at python.org> wrote:

> Send Tutor mailing list submissions to
>         tutor at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.python.org/mailman/listinfo/tutor
> or, via email, send a message with subject or body 'help' to
>         tutor-request at python.org
>
> You can reach the person managing the list at
>         tutor-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."
>
>
> Today's Topics:
>
>    1. To FORMAT or not to (yehudak .)
>    2. Re: To FORMAT or not to (Chris Warrick)
>    3. Re: To FORMAT or not to (Francois Dion)
>    4. Re: To FORMAT or not to (Peter Otten)
>    5. Re: To FORMAT or not to (Francois Dion)
>    6. Re: To FORMAT or not to (Alan Gauld)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 3 Jan 2016 14:27:01 +0200
> From: "yehudak ." <katye2007 at gmail.com>
> To: tutor at python.org
> Subject: [Tutor] To FORMAT or not to
> Message-ID:
>         <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=
> Se6sLQ at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Hi there,
> In a program I wrote the following line (Python 3.5):
>
> print("You've visited", island, '&', new + ".")
>
> A programmer told me that it's a bad habit, and I should have used instead:
>
> print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
>
> May I understand why?
>
>
> ------------------------------
>
> Message: 2
> Date: Sun, 3 Jan 2016 14:04:22 +0100
> From: Chris Warrick <kwpolska at gmail.com>
> To: "yehudak ." <katye2007 at gmail.com>
> Cc: tutor at python.org
> Subject: Re: [Tutor] To FORMAT or not to
> Message-ID:
>         <CAMw+j7+RTGqSFOQg=
> Fohx0xohV8ZFc3JJSn3GES2eMz-_TRCPw at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> On 3 January 2016 at 13:27, yehudak . <katye2007 at gmail.com> wrote:
> > Hi there,
> > In a program I wrote the following line (Python 3.5):
> >
> > print("You've visited", island, '&', new + ".")
> >
> > A programmer told me that it's a bad habit, and I should have used
> instead:
> >
> > print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
> >
> > May I understand why?
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > https://mail.python.org/mailman/listinfo/tutor
>
> The programmer was not very intelligent in his use of str.format in
> the first place. A more sensible way to write this is:
>
> print("You've visited {0} & {1}.".format(island, new))
>
> Formatting with constant strings is pointless, just include it in the
> original input. However, string formatting is not.
>
> Here are a couple of reasons:
> * String formatting works everywhere, but this syntax is specific to
> print() ? if you use something else, you might end up producing faulty
> code
> * The corrected string formatting usage is more readable than the
> original print()
> * String concatenation with + requires that all arguments are strings,
> which is even less readable
> * With string formatting, you can apply special formatting to your
> inputs (eg. set width, number precision?), which is hard or impossible
> with print()
> * Using print() with commas adds spaces between all entries, which
> might look bad (and it does in this example); the only way to prevent
> that is by setting `sep=`, but then you need to remember about a space
> after "visited" and around the ampersand?
> * Easy to localize (translate into different languages), which is
> generally impossible with any of the other options (some languages
> might rearrange the sentence!)
>
> --
> Chris Warrick <https://chriswarrick.com/>
> PGP: 5EAAEA16
>
>
> ------------------------------
>
> Message: 3
> Date: Sun, 3 Jan 2016 08:07:06 -0500
> From: Francois Dion <francois.dion at gmail.com>
> To: "yehudak ." <katye2007 at gmail.com>
> Cc: "tutor at python.org" <tutor at python.org>
> Subject: Re: [Tutor] To FORMAT or not to
> Message-ID:
>         <CAOLi1KAabyvxJUNgQB1NoOnv__r2Xi_e4Zq=
> z6FOvtRxqdBriw at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> The answer is neither. The second shows the intent in part but doesn't
> quite get it right.
>
> The intent is to have a string template and insert values in that template:
>
> print("You've visited {} & {}.".format(island, new)
>
> This is totally clear what is going to happen. I'm not relying on the
> behaviour of print() to format my string. Format does the formating, print
> the printing :)
>
> This separation of concern is a basic building block of good code and is
> seen at various scale levels. At a higher level, it is seen in concepts
> like MVC.
>
> Another thing you do by having your string separated, is you could have
> them defined elsewhere and have, say, a version in english and a version in
> french.
>
> I;m sure you can see the value.
>
> Francois
>
> On Sun, Jan 3, 2016 at 7:27 AM, yehudak . <katye2007 at gmail.com> wrote:
>
> > Hi there,
> > In a program I wrote the following line (Python 3.5):
> >
> > print("You've visited", island, '&', new + ".")
> >
> > A programmer told me that it's a bad habit, and I should have used
> instead:
> >
> > print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
> >
> > May I understand why?
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > https://mail.python.org/mailman/listinfo/tutor
> >
>
>
>
> --
> raspberry-python.blogspot.com - www.pyptug.org - www.3DFutureTech.info -
> @f_dion
>
>
> ------------------------------
>
> Message: 4
> Date: Sun, 03 Jan 2016 14:09:24 +0100
> From: Peter Otten <__peter__ at web.de>
> To: tutor at python.org
> Subject: Re: [Tutor] To FORMAT or not to
> Message-ID: <n6b6i7$g37$1 at ger.gmane.org>
> Content-Type: text/plain; charset="ISO-8859-1"
>
> yehudak . wrote:
>
> > Hi there,
> > In a program I wrote the following line (Python 3.5):
> >
> > print("You've visited", island, '&', new + ".")
> >
> > A programmer told me that it's a bad habit, and I should have used
> > instead:
> >
> > print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
> >
> > May I understand why?
>
> I don't see the benefits either. If anything I'd move the constants into
> the
> format string:
>
> print("You've visited {island}, & {new}.".format(island=island, new=new))
>
> If you use the same names in your format string and your code it should
> suffice to read the format string to get an idea of what will be printed.
>
> In future versions of Python you can simplify it to
>
> print(f"You've visited {island}, & {new}.")
>
> https://www.python.org/dev/peps/pep-0498/
>
>
>
> ------------------------------
>
> Message: 5
> Date: Sun, 3 Jan 2016 08:10:09 -0500
> From: Francois Dion <francois.dion at gmail.com>
> To: Chris Warrick <kwpolska at gmail.com>
> Cc: "yehudak ." <katye2007 at gmail.com>, "tutor at python.org"
>         <tutor at python.org>
> Subject: Re: [Tutor] To FORMAT or not to
> Message-ID:
>         <
> CAOLi1KB8s53RDKbiSFB-B6_-T7JivZ96DAwFYM+Z+6DDQnYtsQ at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> And as Chris points out, if there is any possibility that the words will be
> in a different order in a different language, use {0}, {1} instead of {}.
>
>
> Francois
>
> On Sun, Jan 3, 2016 at 8:04 AM, Chris Warrick <kwpolska at gmail.com> wrote:
>
> > On 3 January 2016 at 13:27, yehudak . <katye2007 at gmail.com> wrote:
> > > Hi there,
> > > In a program I wrote the following line (Python 3.5):
> > >
> > > print("You've visited", island, '&', new + ".")
> > >
> > > A programmer told me that it's a bad habit, and I should have used
> > instead:
> > >
> > > print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
> > >
> > > May I understand why?
> > > _______________________________________________
> > > Tutor maillist  -  Tutor at python.org
> > > To unsubscribe or change subscription options:
> > > https://mail.python.org/mailman/listinfo/tutor
> >
> > The programmer was not very intelligent in his use of str.format in
> > the first place. A more sensible way to write this is:
> >
> > print("You've visited {0} & {1}.".format(island, new))
> >
> > Formatting with constant strings is pointless, just include it in the
> > original input. However, string formatting is not.
> >
> > Here are a couple of reasons:
> > * String formatting works everywhere, but this syntax is specific to
> > print() ? if you use something else, you might end up producing faulty
> > code
> > * The corrected string formatting usage is more readable than the
> > original print()
> > * String concatenation with + requires that all arguments are strings,
> > which is even less readable
> > * With string formatting, you can apply special formatting to your
> > inputs (eg. set width, number precision?), which is hard or impossible
> > with print()
> > * Using print() with commas adds spaces between all entries, which
> > might look bad (and it does in this example); the only way to prevent
> > that is by setting `sep=`, but then you need to remember about a space
> > after "visited" and around the ampersand?
> > * Easy to localize (translate into different languages), which is
> > generally impossible with any of the other options (some languages
> > might rearrange the sentence!)
> >
> > --
> > Chris Warrick <https://chriswarrick.com/>
> > PGP: 5EAAEA16
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > https://mail.python.org/mailman/listinfo/tutor
> >
>
>
>
> --
> raspberry-python.blogspot.com - www.pyptug.org - www.3DFutureTech.info -
> @f_dion
>
>
> ------------------------------
>
> Message: 6
> Date: Sun, 3 Jan 2016 13:12:41 +0000
> From: Alan Gauld <alan.gauld at btinternet.com>
> To: tutor at python.org
> Subject: Re: [Tutor] To FORMAT or not to
> Message-ID: <n6b6o8$isq$1 at ger.gmane.org>
> Content-Type: text/plain; charset=utf-8
>
> On 03/01/16 12:27, yehudak . wrote:
> > Hi there,
> > In a program I wrote the following line (Python 3.5):
> >
> > print("You've visited", island, '&', new + ".")
> >
> > A programmer told me that it's a bad habit, and I should have used
> instead:
> >
> > print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
> >
> > May I understand why?
>
> There are several reasons although your technique is far from
> the worst way of doing things. And the format string here would probably
> be better written as:
>
> print("You've visited {0} & {2}.".format(island, new))
>
> ie only put the variables as placeholders.
>
> Why is it better?
> 1) It is slightly more performant. String addition and
> concatenation are relatively slow processes in Python.
> Formatting will usually be slightly faster. This is not
> a good reason in itself (and in your case with a single
> print it's probably irrelevant) but it's one factor. It
> does matter more if you are printing inside a loop with
> many variables and long strings. (For example assembling
> a web page). So it's a good habit to adopt.
>
> 2) It improves consistency. You can store the format
> string as a variable and then print it out many times
> from different parts of your code and it will always
> be the same. This makes it much easier to maintain
> your program. For example:
>
>     fmtString = "You've visited {0} & {2}."
>     if foo:
>       print(fmtString.format(foo,bar))
>     else:
>       print(fmtString.format(baz,bad))
>
> Now if you want to change the message you only need to
> change the text in one place rather than searching your
> code to do an edit. Also the user sees exactly the same
> formatting, no extra spaces in one message compared
> to another for example.
>
> 3) formatting provides many extra features to control
> justification, leading and spacing. This is especially
> important for numeric output where you can define the
> number of decimal places, whether a leading sign is
> included, padding with zeros etc, etc. You may not
> need that initially but if you do have to add it in,
> it's trivial with a format string but much more work
> if you have lots of hard coded messages all over your
> code.
>
> There may be other reasons too but that should give
> you some ideas.
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> https://mail.python.org/mailman/listinfo/tutor
>
>
> ------------------------------
>
> End of Tutor Digest, Vol 143, Issue 4
> *************************************
>

From alan.gauld at btinternet.com  Sun Jan  3 10:43:19 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 3 Jan 2016 15:43:19 +0000
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <n6b6i7$g37$1@ger.gmane.org>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <n6b6i7$g37$1@ger.gmane.org>
Message-ID: <n6bfim$goi$1@ger.gmane.org>

On 03/01/16 13:09, Peter Otten wrote:

> In future versions of Python you can simplify it to
> 
> print(f"You've visited {island}, & {new}.")
> 
> https://www.python.org/dev/peps/pep-0498/

I hadn't seen that before. Interesting link, thanks.
I think I like it.... but only time and experience will tell :-)


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Sun Jan  3 20:13:13 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 4 Jan 2016 12:13:13 +1100
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
Message-ID: <20160104011313.GE23700@ando.pearwood.info>

On Sun, Jan 03, 2016 at 02:27:01PM +0200, yehudak . wrote:
> Hi there,
> In a program I wrote the following line (Python 3.5):
> 
> print("You've visited", island, '&', new + ".")
> 
> A programmer told me that it's a bad habit, and I should have used instead:
> 
> print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
> 
> May I understand why?


This is nonsense. There is nothing wrong with using print they way you 
did.

print is designed to solve simple problems, and you had a simple 
problem. format is designed to solve complicated problems. You can use 
format, or % string interpolation, to solve simple problems too, but why 
bother? These four lines all do more or less the same thing:


print("You've visited", island, '&', new + ".")
print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
print("You've visited {} & {}.".format(island, new))
print("You've visited %s & %s." % (island, new))


The second version is the LEAST sensible, it should be re-written as the 
third version. But apart from that minor difference, there's no 
practical difference between those lines. It is entirely a matter of 
personal taste which you use.

This doesn't mean that similar problems will also be a matter of 
personal taste. As the problem gets more complex, one or the other gets 
more appropriate. But there's nothing wrong with what you wrote.



-- 
Steve

From steve at pearwood.info  Sun Jan  3 21:03:22 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 4 Jan 2016 13:03:22 +1100
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <CAMw+j7+RTGqSFOQg=Fohx0xohV8ZFc3JJSn3GES2eMz-_TRCPw@mail.gmail.com>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <CAMw+j7+RTGqSFOQg=Fohx0xohV8ZFc3JJSn3GES2eMz-_TRCPw@mail.gmail.com>
Message-ID: <20160104020322.GF23700@ando.pearwood.info>

On Sun, Jan 03, 2016 at 02:04:22PM +0100, Chris Warrick wrote:

> Here are a couple of reasons:
> * String formatting works everywhere, but this syntax is specific to
> print() ? if you use something else, you might end up producing faulty
> code

That argument doesn't make sense to me. I think you mean semantics, 
not syntax, as print uses the same syntax as all Python functions. But 
the same might be said about EVERY function and method.

I might say, "Don't use format, because the semantics is specific to 
format -- if you use something else, you might end up producting faulty 
code". Well duh, of course it would, but if I used something else, I 
wouldn't give it arguments that format requires. The same applies to 
print, and literally every single other function and method.


> * The corrected string formatting usage is more readable than the
> original print()

That's a matter of opinion, and not one that I share.


> * String concatenation with + requires that all arguments are strings,
> which is even less readable

Again, that's a matter of opinion, and it's not a very credible opinion. 
You're arguing that using a complex mini-language with mysterious pairs 
of braces plus a method call:

    print("blah blah blah {}.".format(s))

is more readable than a simple concatenation:

    print("blah blah blah", s + ".")


I think not.


> * With string formatting, you can apply special formatting to your
> inputs (eg. set width, number precision?), which is hard or impossible
> with print()

*shrug*

Nobody says that format isn't capable of more advanced formatting 
control than print. That's format's purpose, after all, just as print's 
purpose is to print. But for the example given, when the formatting 
needed is trivial and well within the capabilities of print, there's no 
need to involve format.

The SpaceX "Falcon" rocket will take you to space, and a skate-board 
will take you around the block. If you strap a Falcon rocket to a 
skate-board, and lay it on the ground, you can also use a Falcon to go 
around the block. But why would you bother?

> * Using print() with commas adds spaces between all entries, which
> might look bad (and it does in this example); 

Say what? Adding spaces between the entries is exactly what is needed 
here, and it looks fine:

py> island = "Mothering Sunday Island"
py> new = "cuddled a tree-climbing octopus"
py> print("You've visited", island, '&', new + ".")
You've visited Mothering Sunday Island & cuddled a tree-climbing octopus.


> the only way to prevent
> that is by setting `sep=`, but then you need to remember about a space
> after "visited" and around the ampersand?

What? This argument makes no sense. There's no need to set sep='', the 
example works fine with the default single space separator.



> * Easy to localize (translate into different languages), which is
> generally impossible with any of the other options (some languages
> might rearrange the sentence!)

A translation engine might use format internally for *part* of the job, 
but the usual way to handle localisations is to call a function 
(conventionally named underscore) to do the work. That *might* involve 
using a template designed for format:

_("You've visited {} & {}.", island, new)

but could just as likely involve % or $ placeholders. That will allow 
both the template and the terms to be localised. In any case, it is not 
likely to involve a direct call to format for exactly the same reason 
that it isn't likely to involve a direct use of print's simple format 
control: it is too simple.

Not only can word-order change, but a single word that gets injected 
into one place of the template in Language A might end up being two 
terms that are injected into two widely-separated places in Language B:

# Language A
"blah blah blah blah {0} blah {1} blah blah", a, b

# Language B
"blah blah {1} blah blah blah {0} blah blah {2} blah blah", a, b, c

(and, by the way, the punctuation needs to be localised too). The 
localisation engine can deal with that complexity. Trying to manage 
translations yourself using just format cannot (or at least not easily).

But fundamentally, the problem with this argument is YAGNI ("You Aren't 
Going To Need It"). Don't over-engineer a simple script just in case 
someday you will need to localize it for arbitrary languages. You won't. 
And if you do, *that* is the time to rewrite the print call to include a 
localisation step, not now.



-- 
Steve

From steve at pearwood.info  Sun Jan  3 22:46:35 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 4 Jan 2016 14:46:35 +1100
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <n6b6o8$isq$1@ger.gmane.org>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <n6b6o8$isq$1@ger.gmane.org>
Message-ID: <20160104034632.GG23700@ando.pearwood.info>

On Sun, Jan 03, 2016 at 01:12:41PM +0000, Alan Gauld wrote:

> Why is it better?
> 1) It is slightly more performant. 

Consider that format has to build the entire output as a new string in 
advance:

"You've visited {0} & {2}.".format(island, new)

gets generated before being passed to print for printing:

print( result of above )

That involves copying each substring and building a single new string. 
Imagine if island and new were each 100MB long texts, you would end up 
with a third string, 200MB (plus a bit) in size, before print gets 
called. But by calling print directly, it can write each substring (and 
separator) to stdout one at a time, without needing to build up the 
entire output first.

But even for small strings, it has to parse the template, work out where 
the braces are, determine how big the new string will be, allocate that 
much memory (possibly causing a garbage collection pause), and copy the 
arguments into the new string.

I'm not so sure that all this will be faster than the simple-minded 
actions of print:

for each argument:
    send str(argument) to stdout;
    if this is not the last argument, send the separator

Arguing that format+print is more efficient than print on its own seems 
rather unlikely to be correct.


> 2) It improves consistency. You can store the format
> string as a variable and then print it out many times
> from different parts of your code and it will always
> be the same. This makes it much easier to maintain
> your program. For example:
> 
>     fmtString = "You've visited {0} & {2}."
>     if foo:
>       print(fmtString.format(foo,bar))
>     else:
>       print(fmtString.format(baz,bad))

I wouldn't write it like that. I'd write:


    if foo:
        args = (foo, bar)
    else:
        args = (baz, bad)
    print("You've visited {} & {}.".format(*args))


but now we're out of the realm of simple uses like the Original Poster's 
code. The original example gives us no reason to believe that we need to 
conditionally change the values being printed, or print it more than 
once.

There's nothing wrong with using format. But for something as simple as 
this example, you can equally use % or just call print directly. It 
isn't an accident that print comes with its own basic formatting 
control, it is a deliberate design choice. There's no *need* to 
over-complicate the situation and then invent some spurious 
justification for it:

"format will save 0.000001 second of time!"

"Maybe some day I'll need to translate this into Klingon!"

"Perhaps I will decide to print the same string from ten different 
places in the code!"

Well, sure, but right now, there's no reason to think that one 
microsecond will make any noticable difference, or that the program 
needs to include translation, or print the string more than once.

I think that, in this specific example, the only really good argument 
for using format is a matter of taste: "I prefer to use format, because 
I like it. It's more capable, so I use it more often and I am 
comfortable and happy using it. There's no strong argument *against* 
using it, so I prefer to use it."

Great, that's fair, can't argue with that. There's no need to invent 
justifications for format, or to say that format would be better in 
*some other situation*. (We don't have some other situation, we have 
this situation.) And if you disagree and prefer to just use print, 
that's fine too.


-- 
Steve

From alan.gauld at btinternet.com  Mon Jan  4 05:04:41 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 4 Jan 2016 10:04:41 +0000
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <20160104034632.GG23700@ando.pearwood.info>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <n6b6o8$isq$1@ger.gmane.org> <20160104034632.GG23700@ando.pearwood.info>
Message-ID: <n6dg3o$76c$1@ger.gmane.org>

On 04/01/16 03:46, Steven D'Aprano wrote:

>>     fmtString = "You've visited {0} & {2}."
>>     if foo:
>>       print(fmtString.format(foo,bar))
>>     else:
>>       print(fmtString.format(baz,bad))
> 
> I wouldn't write it like that. I'd write:
> 
>     if foo:
>         args = (foo, bar)
>     else:
>         args = (baz, bad)
>     print("You've visited {} & {}.".format(*args))

In this case yes but the point I was trying to illustrate
was that you often want to print the same message from
several different places in your code - even in different
functions for example. Using a format string helps keep
that consistent. The if/else construct was just the most
concise example I could think of.

> but now we're out of the realm of simple uses like the Original Poster's 
> code. 

Indeed, and I did point that out at the start of my message.
Apart from one string addition the OPs use of print was
pretty seamless. But he wanted to know why the other
programmer thought formatting was better, not whether we thought
formatting was better in his particular case. At least that's
how I read it.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From poojabhalode11 at gmail.com  Mon Jan  4 11:56:00 2016
From: poojabhalode11 at gmail.com (Pooja Bhalode)
Date: Mon, 4 Jan 2016 11:56:00 -0500
Subject: [Tutor] Lists+sorting
Message-ID: <CAK0ikxfHW+vAJ+gcgCssvqkS_H6nRdmq81C-kdW7XFYVHztTMw@mail.gmail.com>

Hi,
I wanted to check if I can write the following program in this manner as
well.

The problem is to merge the lists together and sort them.The solution they
have given is:

def linear_merge(list1, list2):


result = []

while len(list1) and len(list2):

if list1[0] < list2[0]:

result.append(list1.pop(0))

else:

result.append(list2.pop(0))


# Now tack on what's left

result.extend(list1)

result.extend(list2)

return result


So, is it correct if I write the function in this manner? This seems to
give me the correct solution though


def linear_merge(list1, list2):

for num in list2:

list1.append(num)

list1.sort()

# +++your code here+++

return list1


Thank you.

From katye2007 at gmail.com  Mon Jan  4 12:22:03 2016
From: katye2007 at gmail.com (yehudak .)
Date: Mon, 4 Jan 2016 19:22:03 +0200
Subject: [Tutor] Tutor Digest, Vol 143, Issue 6
In-Reply-To: <mailman.9.1451926802.12807.tutor@python.org>
References: <mailman.9.1451926802.12807.tutor@python.org>
Message-ID: <CAE3ie42pWfsbKkTv_tMmnVu4sacvvbDuZ4EoacHL0yHXJv6hug@mail.gmail.com>

Dear gentlemen.
This discussion gave me the pleasure of watching two programmers clash head
to head.

I got here a lot of knowledge that no book covers.

Thank you Alan & Steve.

Yehuda

On Mon, Jan 4, 2016 at 7:00 PM, <tutor-request at python.org> wrote:

> Send Tutor mailing list submissions to
>         tutor at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.python.org/mailman/listinfo/tutor
> or, via email, send a message with subject or body 'help' to
>         tutor-request at python.org
>
> You can reach the person managing the list at
>         tutor-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."
>
>
> Today's Topics:
>
>    1. Re: To FORMAT or not to (Steven D'Aprano)
>    2. Re: To FORMAT or not to (Steven D'Aprano)
>    3. Re: To FORMAT or not to (Steven D'Aprano)
>    4. Re: To FORMAT or not to (Alan Gauld)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 4 Jan 2016 12:13:13 +1100
> From: Steven D'Aprano <steve at pearwood.info>
> To: tutor at python.org
> Subject: Re: [Tutor] To FORMAT or not to
> Message-ID: <20160104011313.GE23700 at ando.pearwood.info>
> Content-Type: text/plain; charset=us-ascii
>
> On Sun, Jan 03, 2016 at 02:27:01PM +0200, yehudak . wrote:
> > Hi there,
> > In a program I wrote the following line (Python 3.5):
> >
> > print("You've visited", island, '&', new + ".")
> >
> > A programmer told me that it's a bad habit, and I should have used
> instead:
> >
> > print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
> >
> > May I understand why?
>
>
> This is nonsense. There is nothing wrong with using print they way you
> did.
>
> print is designed to solve simple problems, and you had a simple
> problem. format is designed to solve complicated problems. You can use
> format, or % string interpolation, to solve simple problems too, but why
> bother? These four lines all do more or less the same thing:
>
>
> print("You've visited", island, '&', new + ".")
> print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
> print("You've visited {} & {}.".format(island, new))
> print("You've visited %s & %s." % (island, new))
>
>
> The second version is the LEAST sensible, it should be re-written as the
> third version. But apart from that minor difference, there's no
> practical difference between those lines. It is entirely a matter of
> personal taste which you use.
>
> This doesn't mean that similar problems will also be a matter of
> personal taste. As the problem gets more complex, one or the other gets
> more appropriate. But there's nothing wrong with what you wrote.
>
>
>
> --
> Steve
>
>
> ------------------------------
>
> Message: 2
> Date: Mon, 4 Jan 2016 13:03:22 +1100
> From: Steven D'Aprano <steve at pearwood.info>
> To: tutor at python.org
> Subject: Re: [Tutor] To FORMAT or not to
> Message-ID: <20160104020322.GF23700 at ando.pearwood.info>
> Content-Type: text/plain; charset=utf-8
>
> On Sun, Jan 03, 2016 at 02:04:22PM +0100, Chris Warrick wrote:
>
> > Here are a couple of reasons:
> > * String formatting works everywhere, but this syntax is specific to
> > print() ? if you use something else, you might end up producing faulty
> > code
>
> That argument doesn't make sense to me. I think you mean semantics,
> not syntax, as print uses the same syntax as all Python functions. But
> the same might be said about EVERY function and method.
>
> I might say, "Don't use format, because the semantics is specific to
> format -- if you use something else, you might end up producting faulty
> code". Well duh, of course it would, but if I used something else, I
> wouldn't give it arguments that format requires. The same applies to
> print, and literally every single other function and method.
>
>
> > * The corrected string formatting usage is more readable than the
> > original print()
>
> That's a matter of opinion, and not one that I share.
>
>
> > * String concatenation with + requires that all arguments are strings,
> > which is even less readable
>
> Again, that's a matter of opinion, and it's not a very credible opinion.
> You're arguing that using a complex mini-language with mysterious pairs
> of braces plus a method call:
>
>     print("blah blah blah {}.".format(s))
>
> is more readable than a simple concatenation:
>
>     print("blah blah blah", s + ".")
>
>
> I think not.
>
>
> > * With string formatting, you can apply special formatting to your
> > inputs (eg. set width, number precision?), which is hard or impossible
> > with print()
>
> *shrug*
>
> Nobody says that format isn't capable of more advanced formatting
> control than print. That's format's purpose, after all, just as print's
> purpose is to print. But for the example given, when the formatting
> needed is trivial and well within the capabilities of print, there's no
> need to involve format.
>
> The SpaceX "Falcon" rocket will take you to space, and a skate-board
> will take you around the block. If you strap a Falcon rocket to a
> skate-board, and lay it on the ground, you can also use a Falcon to go
> around the block. But why would you bother?
>
> > * Using print() with commas adds spaces between all entries, which
> > might look bad (and it does in this example);
>
> Say what? Adding spaces between the entries is exactly what is needed
> here, and it looks fine:
>
> py> island = "Mothering Sunday Island"
> py> new = "cuddled a tree-climbing octopus"
> py> print("You've visited", island, '&', new + ".")
> You've visited Mothering Sunday Island & cuddled a tree-climbing octopus.
>
>
> > the only way to prevent
> > that is by setting `sep=`, but then you need to remember about a space
> > after "visited" and around the ampersand?
>
> What? This argument makes no sense. There's no need to set sep='', the
> example works fine with the default single space separator.
>
>
>
> > * Easy to localize (translate into different languages), which is
> > generally impossible with any of the other options (some languages
> > might rearrange the sentence!)
>
> A translation engine might use format internally for *part* of the job,
> but the usual way to handle localisations is to call a function
> (conventionally named underscore) to do the work. That *might* involve
> using a template designed for format:
>
> _("You've visited {} & {}.", island, new)
>
> but could just as likely involve % or $ placeholders. That will allow
> both the template and the terms to be localised. In any case, it is not
> likely to involve a direct call to format for exactly the same reason
> that it isn't likely to involve a direct use of print's simple format
> control: it is too simple.
>
> Not only can word-order change, but a single word that gets injected
> into one place of the template in Language A might end up being two
> terms that are injected into two widely-separated places in Language B:
>
> # Language A
> "blah blah blah blah {0} blah {1} blah blah", a, b
>
> # Language B
> "blah blah {1} blah blah blah {0} blah blah {2} blah blah", a, b, c
>
> (and, by the way, the punctuation needs to be localised too). The
> localisation engine can deal with that complexity. Trying to manage
> translations yourself using just format cannot (or at least not easily).
>
> But fundamentally, the problem with this argument is YAGNI ("You Aren't
> Going To Need It"). Don't over-engineer a simple script just in case
> someday you will need to localize it for arbitrary languages. You won't.
> And if you do, *that* is the time to rewrite the print call to include a
> localisation step, not now.
>
>
>
> --
> Steve
>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 4 Jan 2016 14:46:35 +1100
> From: Steven D'Aprano <steve at pearwood.info>
> To: tutor at python.org
> Subject: Re: [Tutor] To FORMAT or not to
> Message-ID: <20160104034632.GG23700 at ando.pearwood.info>
> Content-Type: text/plain; charset=us-ascii
>
> On Sun, Jan 03, 2016 at 01:12:41PM +0000, Alan Gauld wrote:
>
> > Why is it better?
> > 1) It is slightly more performant.
>
> Consider that format has to build the entire output as a new string in
> advance:
>
> "You've visited {0} & {2}.".format(island, new)
>
> gets generated before being passed to print for printing:
>
> print( result of above )
>
> That involves copying each substring and building a single new string.
> Imagine if island and new were each 100MB long texts, you would end up
> with a third string, 200MB (plus a bit) in size, before print gets
> called. But by calling print directly, it can write each substring (and
> separator) to stdout one at a time, without needing to build up the
> entire output first.
>
> But even for small strings, it has to parse the template, work out where
> the braces are, determine how big the new string will be, allocate that
> much memory (possibly causing a garbage collection pause), and copy the
> arguments into the new string.
>
> I'm not so sure that all this will be faster than the simple-minded
> actions of print:
>
> for each argument:
>     send str(argument) to stdout;
>     if this is not the last argument, send the separator
>
> Arguing that format+print is more efficient than print on its own seems
> rather unlikely to be correct.
>
>
> > 2) It improves consistency. You can store the format
> > string as a variable and then print it out many times
> > from different parts of your code and it will always
> > be the same. This makes it much easier to maintain
> > your program. For example:
> >
> >     fmtString = "You've visited {0} & {2}."
> >     if foo:
> >       print(fmtString.format(foo,bar))
> >     else:
> >       print(fmtString.format(baz,bad))
>
> I wouldn't write it like that. I'd write:
>
>
>     if foo:
>         args = (foo, bar)
>     else:
>         args = (baz, bad)
>     print("You've visited {} & {}.".format(*args))
>
>
> but now we're out of the realm of simple uses like the Original Poster's
> code. The original example gives us no reason to believe that we need to
> conditionally change the values being printed, or print it more than
> once.
>
> There's nothing wrong with using format. But for something as simple as
> this example, you can equally use % or just call print directly. It
> isn't an accident that print comes with its own basic formatting
> control, it is a deliberate design choice. There's no *need* to
> over-complicate the situation and then invent some spurious
> justification for it:
>
> "format will save 0.000001 second of time!"
>
> "Maybe some day I'll need to translate this into Klingon!"
>
> "Perhaps I will decide to print the same string from ten different
> places in the code!"
>
> Well, sure, but right now, there's no reason to think that one
> microsecond will make any noticable difference, or that the program
> needs to include translation, or print the string more than once.
>
> I think that, in this specific example, the only really good argument
> for using format is a matter of taste: "I prefer to use format, because
> I like it. It's more capable, so I use it more often and I am
> comfortable and happy using it. There's no strong argument *against*
> using it, so I prefer to use it."
>
> Great, that's fair, can't argue with that. There's no need to invent
> justifications for format, or to say that format would be better in
> *some other situation*. (We don't have some other situation, we have
> this situation.) And if you disagree and prefer to just use print,
> that's fine too.
>
>
> --
> Steve
>
>
> ------------------------------
>
> Message: 4
> Date: Mon, 4 Jan 2016 10:04:41 +0000
> From: Alan Gauld <alan.gauld at btinternet.com>
> To: tutor at python.org
> Subject: Re: [Tutor] To FORMAT or not to
> Message-ID: <n6dg3o$76c$1 at ger.gmane.org>
> Content-Type: text/plain; charset=utf-8
>
> On 04/01/16 03:46, Steven D'Aprano wrote:
>
> >>     fmtString = "You've visited {0} & {2}."
> >>     if foo:
> >>       print(fmtString.format(foo,bar))
> >>     else:
> >>       print(fmtString.format(baz,bad))
> >
> > I wouldn't write it like that. I'd write:
> >
> >     if foo:
> >         args = (foo, bar)
> >     else:
> >         args = (baz, bad)
> >     print("You've visited {} & {}.".format(*args))
>
> In this case yes but the point I was trying to illustrate
> was that you often want to print the same message from
> several different places in your code - even in different
> functions for example. Using a format string helps keep
> that consistent. The if/else construct was just the most
> concise example I could think of.
>
> > but now we're out of the realm of simple uses like the Original Poster's
> > code.
>
> Indeed, and I did point that out at the start of my message.
> Apart from one string addition the OPs use of print was
> pretty seamless. But he wanted to know why the other
> programmer thought formatting was better, not whether we thought
> formatting was better in his particular case. At least that's
> how I read it.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> https://mail.python.org/mailman/listinfo/tutor
>
>
> ------------------------------
>
> End of Tutor Digest, Vol 143, Issue 6
> *************************************
>

From alan.gauld at btinternet.com  Mon Jan  4 12:27:46 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 4 Jan 2016 17:27:46 +0000
Subject: [Tutor] Lists+sorting
In-Reply-To: <CAK0ikxfHW+vAJ+gcgCssvqkS_H6nRdmq81C-kdW7XFYVHztTMw@mail.gmail.com>
References: <CAK0ikxfHW+vAJ+gcgCssvqkS_H6nRdmq81C-kdW7XFYVHztTMw@mail.gmail.com>
Message-ID: <n6ea2h$kgh$1@ger.gmane.org>

On 04/01/16 16:56, Pooja Bhalode wrote:
> Hi,
> I wanted to check if I can write the following program in this manner as
> well.
> 

Can you resend in plain text please?
Your post lost all its formatting so its hard to read
or comment on.

> def linear_merge(list1, list2):
> result = []
> while len(list1) and len(list2):
> if list1[0] < list2[0]:
> result.append(list1.pop(0))
> else:
> result.append(list2.pop(0))
> 
> # Now tack on what's left
> 
> result.extend(list1)
> result.extend(list2)
> return result

...

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Mon Jan  4 12:35:18 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 4 Jan 2016 17:35:18 +0000
Subject: [Tutor] Tutor Digest, Vol 143, Issue 6
In-Reply-To: <CAE3ie42pWfsbKkTv_tMmnVu4sacvvbDuZ4EoacHL0yHXJv6hug@mail.gmail.com>
References: <mailman.9.1451926802.12807.tutor@python.org>
 <CAE3ie42pWfsbKkTv_tMmnVu4sacvvbDuZ4EoacHL0yHXJv6hug@mail.gmail.com>
Message-ID: <n6eagl$ta0$1@ger.gmane.org>

On 04/01/16 17:22, yehudak . wrote:
> Dear gentlemen.
> This discussion gave me the pleasure of watching two programmers clash head
> to head.

I'm not sure we were clashing. I actually agreed with most of what Steve
said. I simply clarified the intent of one aspect of his post.

To summarize:

For the specific case that you posted, where there is little or
no string concatenation or additional formatting to be done, there
is not much need to use format(). However, many programmers
(including the one you referred to in your initial post)
prefer it because it eases maintenance in bigger programs
where more specific and consistent output is required.

More subjectively, some people find a format style output easier
to read. Others prefer the comma separated list of a print
statement.

> I got here a lot of knowledge that no book covers.

That's what the tutor list is for ;-)

But please stop sending all the previously seen content from
the digest and please change the subject line, not least so
that searches of the list archive can find the messages.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From joel.goldstick at gmail.com  Mon Jan  4 12:45:42 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 4 Jan 2016 12:45:42 -0500
Subject: [Tutor] Lists+sorting
In-Reply-To: <n6ea2h$kgh$1@ger.gmane.org>
References: <CAK0ikxfHW+vAJ+gcgCssvqkS_H6nRdmq81C-kdW7XFYVHztTMw@mail.gmail.com>
 <n6ea2h$kgh$1@ger.gmane.org>
Message-ID: <CAPM-O+wMsygCk4CMtci=zcWQY1MzZj4t1XWc74OhCTT+YhaAPA@mail.gmail.com>

On Mon, Jan 4, 2016 at 12:27 PM, Alan Gauld <alan.gauld at btinternet.com>
wrote:

> On 04/01/16 16:56, Pooja Bhalode wrote:
> > Hi,
> > I wanted to check if I can write the following program in this manner as
> > well.
> >
>
> Can you resend in plain text please?
> Your post lost all its formatting so its hard to read
> or comment on.
>
> > def linear_merge(list1, list2):
> > result = []
> > while len(list1) and len(list2):
> > if list1[0] < list2[0]:
> > result.append(list1.pop(0))
> > else:
> > result.append(list2.pop(0))
> >
> > # Now tack on what's left
> >
> > result.extend(list1)
> > result.extend(list2)
> > return result
>
> Like Alan said, send in plain text.  Also, give a test case and your
results.  Your version uses the built in sort method.  The linear merge
doesn't use that, so the writer's are trying to teach you something about
sorting.

> ...
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From poojabhalode11 at gmail.com  Mon Jan  4 12:34:57 2016
From: poojabhalode11 at gmail.com (Pooja Bhalode)
Date: Mon, 4 Jan 2016 12:34:57 -0500
Subject: [Tutor] lists+sort
Message-ID: <CAK0ikxdzexV5v=u0DmbUVttANXeQezwJgr04bsrWYX0dQzp_1A@mail.gmail.com>

Hi, I wanted to check if this program can be used to merge the lists
together and sort them. This seems to work, but i wanted to check if there
are drawbacks in writing it in this manner.


My solution:


def linear_merge(list1, list2):

    for num in list2:

        list1.append(num)



    list1.sort()



  # +++your code here+++

    return list1



Whereas, their code is a bit different, I have posted it here.


def linear_merge(list1, list2):



  result = []

  # Look at the two lists so long as both are non-empty.

  # Take whichever element [0] is smaller.

  while len(list1) and len(list2):

    if list1[0] < list2[0]:

      result.append(list1.pop(0))

    else:

      result.append(list2.pop(0))


  # Now tack on what's left

  result.extend(list1)

  result.extend(list2)

  return result



Can you please tell me if there is a problem in the first code?

Thank you.

From poojabhalode11 at gmail.com  Mon Jan  4 12:50:39 2016
From: poojabhalode11 at gmail.com (Pooja Bhalode)
Date: Mon, 4 Jan 2016 12:50:39 -0500
Subject: [Tutor] Lists+sorting
In-Reply-To: <CAPM-O+wMsygCk4CMtci=zcWQY1MzZj4t1XWc74OhCTT+YhaAPA@mail.gmail.com>
References: <CAK0ikxfHW+vAJ+gcgCssvqkS_H6nRdmq81C-kdW7XFYVHztTMw@mail.gmail.com>
 <n6ea2h$kgh$1@ger.gmane.org>
 <CAPM-O+wMsygCk4CMtci=zcWQY1MzZj4t1XWc74OhCTT+YhaAPA@mail.gmail.com>
Message-ID: <CAK0ikxdPYhYAku=WBKEdp5UxXKHV-Z0bGTkzgg+zngeOpYMf1A@mail.gmail.com>

Hi,

Yes, I tried sending the mail again in plain text.
But I think I understood the difference there in using built-in sort
function and writing the code without using that.

Thank you


On Mon, Jan 4, 2016 at 12:45 PM, Joel Goldstick <joel.goldstick at gmail.com>
wrote:

> On Mon, Jan 4, 2016 at 12:27 PM, Alan Gauld <alan.gauld at btinternet.com>
> wrote:
>
> > On 04/01/16 16:56, Pooja Bhalode wrote:
> > > Hi,
> > > I wanted to check if I can write the following program in this manner
> as
> > > well.
> > >
> >
> > Can you resend in plain text please?
> > Your post lost all its formatting so its hard to read
> > or comment on.
> >
> > > def linear_merge(list1, list2):
> > > result = []
> > > while len(list1) and len(list2):
> > > if list1[0] < list2[0]:
> > > result.append(list1.pop(0))
> > > else:
> > > result.append(list2.pop(0))
> > >
> > > # Now tack on what's left
> > >
> > > result.extend(list1)
> > > result.extend(list2)
> > > return result
> >
> > Like Alan said, send in plain text.  Also, give a test case and your
> results.  Your version uses the built in sort method.  The linear merge
> doesn't use that, so the writer's are trying to teach you something about
> sorting.
>
> > ...
> >
> > --
> > Alan G
> > Author of the Learn to Program web site
> > http://www.alan-g.me.uk/
> > http://www.amazon.com/author/alan_gauld
> > Follow my photo-blog on Flickr at:
> > http://www.flickr.com/photos/alangauldphotos
> >
> >
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > https://mail.python.org/mailman/listinfo/tutor
> >
>
>
>
> --
> Joel Goldstick
> http://joelgoldstick.com/stats/birthdays
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From joel.goldstick at gmail.com  Mon Jan  4 14:26:15 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 4 Jan 2016 14:26:15 -0500
Subject: [Tutor] lists+sort
In-Reply-To: <CAK0ikxdzexV5v=u0DmbUVttANXeQezwJgr04bsrWYX0dQzp_1A@mail.gmail.com>
References: <CAK0ikxdzexV5v=u0DmbUVttANXeQezwJgr04bsrWYX0dQzp_1A@mail.gmail.com>
Message-ID: <CAPM-O+z5hCD_vs+Xge6j_JWKYFW0B8uKcL7-hBkdcS5Sd-XXVQ@mail.gmail.com>

On Mon, Jan 4, 2016 at 12:34 PM, Pooja Bhalode <poojabhalode11 at gmail.com>
wrote:

> Hi, I wanted to check if this program can be used to merge the lists
> together and sort them. This seems to work, but i wanted to check if there
> are drawbacks in writing it in this manner.
>
>
> My solution:
>
>
> def linear_merge(list1, list2):
>
>     for num in list2:
>
>         list1.append(num)
>
>
>
>     list1.sort()
>
> This could be more easily done with
list1.extend(list2)
list1.sort()

>
>
>   # +++your code here+++
>
>     return list1
>
>
>
> Whereas, their code is a bit different, I have posted it here.
>
>
> def linear_merge(list1, list2):
>
>
>
>   result = []
>
>   # Look at the two lists so long as both are non-empty.
>
>   # Take whichever element [0] is smaller.
>
>   while len(list1) and len(list2):
>
>     if list1[0] < list2[0]:
>
>       result.append(list1.pop(0))
>
>     else:
>
>       result.append(list2.pop(0))
>
>
>   # Now tack on what's left
>
>   result.extend(list1)
>
>   result.extend(list2)
>
>   return result
>
>
> I don't think this code will work if the original lists are unsorted to
begin with.  You should give some sample data and your results for both
methods

>
> Can you please tell me if there is a problem in the first code?
>
> Thank you.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From breamoreboy at yahoo.co.uk  Mon Jan  4 14:50:59 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 4 Jan 2016 19:50:59 +0000
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <n6b6o8$isq$1@ger.gmane.org>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <n6b6o8$isq$1@ger.gmane.org>
Message-ID: <n6eifp$2a7$1@ger.gmane.org>

On 03/01/2016 13:12, Alan Gauld wrote:
> On 03/01/16 12:27, yehudak . wrote:
>> Hi there,
>> In a program I wrote the following line (Python 3.5):
>>
>> print("You've visited", island, '&', new + ".")
>>
>> A programmer told me that it's a bad habit, and I should have used instead:
>>
>> print("You've visited {0} {1} {2}{3}".format(island, "&", new, "."))
>>
>> May I understand why?
>
> There are several reasons although your technique is far from
> the worst way of doing things. And the format string here would probably
> be better written as:
>
> print("You've visited {0} & {2}.".format(island, new))
>
> ie only put the variables as placeholders.
>
> Why is it better?
> 1) It is slightly more performant. String addition and
> concatenation are relatively slow processes in Python.
> Formatting will usually be slightly faster. This is not
> a good reason in itself (and in your case with a single
> print it's probably irrelevant) but it's one factor. It
> does matter more if you are printing inside a loop with
> many variables and long strings. (For example assembling
> a web page). So it's a good habit to adopt.
>
> 2) It improves consistency. You can store the format
> string as a variable and then print it out many times
> from different parts of your code and it will always
> be the same. This makes it much easier to maintain
> your program. For example:
>
>      fmtString = "You've visited {0} & {2}."
>      if foo:
>        print(fmtString.format(foo,bar))
>      else:
>        print(fmtString.format(baz,bad))
>
> Now if you want to change the message you only need to
> change the text in one place rather than searching your
> code to do an edit. Also the user sees exactly the same
> formatting, no extra spaces in one message compared
> to another for example.
>
> 3) formatting provides many extra features to control
> justification, leading and spacing. This is especially
> important for numeric output where you can define the
> number of decimal places, whether a leading sign is
> included, padding with zeros etc, etc. You may not
> need that initially but if you do have to add it in,
> it's trivial with a format string but much more work
> if you have lots of hard coded messages all over your
> code.
>
> There may be other reasons too but that should give
> you some ideas.
>
>

Three reasons for why it's better but it doesn't actually work as given.

 >>> island = "Isle Of Wight"
 >>> new = "Isle of Wong"
 >>> print("You've visited {0} & {2}.".format(island, new))
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
IndexError: tuple index out of range

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From bodsda at googlemail.com  Mon Jan  4 16:54:18 2016
From: bodsda at googlemail.com (Bod Soutar)
Date: Mon, 4 Jan 2016 21:54:18 +0000
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <n6eifp$2a7$1@ger.gmane.org>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <n6b6o8$isq$1@ger.gmane.org> <n6eifp$2a7$1@ger.gmane.org>
Message-ID: <CAG6BxkfuJLSP9BCrp_gj8XwXhPsSrVOPvfqh-NCbja6=fbvdDQ@mail.gmail.com>

On 4 January 2016 at 19:50, Mark Lawrence
>
> Three reasons for why it's better but it doesn't actually work as given.
>
>>>> island = "Isle Of Wight"
>>>> new = "Isle of Wong"
>>>> print("You've visited {0} & {2}.".format(island, new))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> IndexError: tuple index out of range
>
> --

Have you tried with {0} and {1}?

From breamoreboy at yahoo.co.uk  Mon Jan  4 17:13:50 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 4 Jan 2016 22:13:50 +0000
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <CAG6BxkfuJLSP9BCrp_gj8XwXhPsSrVOPvfqh-NCbja6=fbvdDQ@mail.gmail.com>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <n6b6o8$isq$1@ger.gmane.org> <n6eifp$2a7$1@ger.gmane.org>
 <CAG6BxkfuJLSP9BCrp_gj8XwXhPsSrVOPvfqh-NCbja6=fbvdDQ@mail.gmail.com>
Message-ID: <n6eqrj$7eu$1@ger.gmane.org>

On 04/01/2016 21:54, Bod Soutar via Tutor wrote:
> On 4 January 2016 at 19:50, Mark Lawrence
>>
>> Three reasons for why it's better but it doesn't actually work as given.
>>
>>>>> island = "Isle Of Wight"
>>>>> new = "Isle of Wong"
>>>>> print("You've visited {0} & {2}.".format(island, new))
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>> IndexError: tuple index out of range
>>
>> --
>
> Have you tried with {0} and {1}?

No, it would not have occurred to me without asking here first.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From dyoo at hashcollision.org  Mon Jan  4 17:35:10 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 4 Jan 2016 14:35:10 -0800
Subject: [Tutor] lists+sort
In-Reply-To: <CAK0ikxdzexV5v=u0DmbUVttANXeQezwJgr04bsrWYX0dQzp_1A@mail.gmail.com>
References: <CAK0ikxdzexV5v=u0DmbUVttANXeQezwJgr04bsrWYX0dQzp_1A@mail.gmail.com>
Message-ID: <CAGZAPF4fDSw2yE_L+zgP5k12KhSGcAQEmd0mnMij2K259t2quw@mail.gmail.com>

On Jan 4, 2016 11:00 AM, "Pooja Bhalode" <poojabhalode11 at gmail.com> wrote:
>
> Hi, I wanted to check if this program can be used to merge the lists
> together and sort them. This seems to work, but i wanted to check if there
> are drawbacks in writing it in this manner.

You may be missing some important details or misunderstanding a crucial
detail.

For example, the term "merge" in this context usually had a very specific
technical meaning.

Are your two input lists already sorted?

The use of the term "linear" in the function definition:

> def linear_merge(list1, list2):

has a particular meaning in terms of algorithmic complexity, and the first
implementation does not satisfy it: it does not perform linearly due to the
internal use of the sort().

If you have questions, please feel free to ask.

From fosiul at gmail.com  Mon Jan  4 17:29:01 2016
From: fosiul at gmail.com (Fosiul Alam)
Date: Mon, 4 Jan 2016 22:29:01 +0000
Subject: [Tutor] how to grep a word and make a dictionary from multiple
 lines.
Message-ID: <CAPBMF6xmae3LGaJRf+LaRBk1V2hiGCAtB9JNRScs5+-NoXPJJg@mail.gmail.com>

Hi Expert,
I am learning python to do some system admin code, i am still in the
process of learning but I need a help on bellow code in urgent, i will be
really greatfull for any help

Basically i wanted like this :-

grep a line which has 1:0:0:129 , and get the LUN number(i.e
360060165656565634348a739e511) for this path

so the Dictionalry will be :

dict = {'360060165656565634348a739e511': '1:0:0:129',
'3600601323h42h2k323asdf33511': 1:0:2:98};


How can i do this ?

any help will be really appreciable.

cat file.txt

mpathdz (360060165656565634348a739e511) dm-134 DGC,VRAID
size=200G features='0' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=130 status=active
| |- 0:0:0:129 sddz  128:16   active ready running
| `- 1:0:0:129 sdwd  69:656   active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 0:0:2:129 sdnd  70:496   active ready running
  `- 1:0:2:129 sdafg 68:864   active ready running
mpathcu (3600601323h42h2k323asdf33511) dm-103 DGC,VRAID
size=200G features='0' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=130 status=active
| |- 1:0:2:98  sdaeb 66:880   active ready running
| `- 0:0:2:98  sdly  69:256   active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 0:0:0:98  sdcu  70:32    active ready running
  `- 1:0:0:98  sduy  67:672   active ready running
mpathbp (36003434343eere8b69e411) dm-36 DGC,RAID 5
size=100G features='0' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=130 status=active
| |- 0:0:0:68  sdbq  68:64    active ready running
| `- 1:0:0:68  sdtu  65:704   active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 0:0:2:68  sdku  67:288   active ready running
  `- 1:0:2:68  sdacx 8:912    active ready running

From dyoo at hashcollision.org  Mon Jan  4 18:09:07 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 4 Jan 2016 15:09:07 -0800
Subject: [Tutor] how to grep a word and make a dictionary from multiple
 lines.
In-Reply-To: <CAPBMF6xmae3LGaJRf+LaRBk1V2hiGCAtB9JNRScs5+-NoXPJJg@mail.gmail.com>
References: <CAPBMF6xmae3LGaJRf+LaRBk1V2hiGCAtB9JNRScs5+-NoXPJJg@mail.gmail.com>
Message-ID: <CAGZAPF7CJG9kvtQfUgjPoKfNvZ24r2h=Luc96g2wZhsopnc_iw@mail.gmail.com>

On Mon, Jan 4, 2016 at 2:29 PM, Fosiul Alam <fosiul at gmail.com> wrote:
> Hi Expert,
> I am learning python to do some system admin code, i am still in the
> process of learning but I need a help on bellow code in urgent, i will be
> really greatfull for any help


Question: how would you do this task with regular Unix tools?  (i.e.
grep, cut,  and other shell tools)?



> grep a line which has 1:0:0:129 , and get the LUN number(i.e
> 360060165656565634348a739e511) for this path


You may want to look at the fileinput library, which automates
processes the lines of input files:

    https://docs.python.org/3.0/library/fileinput.html

along with string methods to check whether a line matches a given
criterion or not.  See the Python tutorial for examples:

    https://docs.python.org/3/tutorial/introduction.html#strings

For your case, you might find the 'in' operator helpful for checking
for substring match:

#################################
>>> 'hello' in 'this is a test hello world'
True
>>> 'hola' in 'this is a test hello world'
False
#################################

From alan.gauld at btinternet.com  Mon Jan  4 18:21:54 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 4 Jan 2016 23:21:54 +0000
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <n6eifp$2a7$1@ger.gmane.org>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <n6b6o8$isq$1@ger.gmane.org> <n6eifp$2a7$1@ger.gmane.org>
Message-ID: <n6euqh$71m$1@ger.gmane.org>

On 04/01/16 19:50, Mark Lawrence wrote:

> Three reasons for why it's better but it doesn't actually work as given.
> 
>  >>> island = "Isle Of Wight"
>  >>> new = "Isle of Wong"
>  >>> print("You've visited {0} & {2}.".format(island, new))
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> IndexError: tuple index out of range

Oops. That's what comes of using the OPs code and just
deleting the unnecessary markets. I forgot to change
the index. :-)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From david at graniteweb.com  Mon Jan  4 18:54:52 2016
From: david at graniteweb.com (David Rock)
Date: Mon, 4 Jan 2016 17:54:52 -0600
Subject: [Tutor] how to grep a word and make a dictionary from multiple
 lines.
In-Reply-To: <CAPBMF6xmae3LGaJRf+LaRBk1V2hiGCAtB9JNRScs5+-NoXPJJg@mail.gmail.com>
References: <CAPBMF6xmae3LGaJRf+LaRBk1V2hiGCAtB9JNRScs5+-NoXPJJg@mail.gmail.com>
Message-ID: <20160104235452.GM21994@raspberrypi>

* Fosiul Alam <fosiul at gmail.com> [2016-01-04 22:29]:
> Hi Expert,
> I am learning python to do some system admin code, i am still in the
> process of learning but I need a help on bellow code in urgent, i will be
> really greatfull for any help
> 
> Basically i wanted like this :-
> 
> grep a line which has 1:0:0:129 , and get the LUN number(i.e
> 360060165656565634348a739e511) for this path
> 
> so the Dictionalry will be :
> 
> dict = {'360060165656565634348a739e511': '1:0:0:129',
> '3600601323h42h2k323asdf33511': 1:0:2:98};
> 
> 
> How can i do this ?

based on your multipath output, one thing you have to take into account
is that the LUN value comes before the SCSI value.  You are going to
have to walk the file and for each mpath line, create a dict, then add
your SCSI values to that entry.  You will have to find a way to break
out of your logic when the next mpath is seen.

You will also want to think about what to do with all your LUNs.  You
have four per mpath, not one.  Do you want to capture only one, or do
you need all of them?

-- 
David Rock
david at graniteweb.com

From joel.goldstick at gmail.com  Mon Jan  4 19:17:20 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 4 Jan 2016 19:17:20 -0500
Subject: [Tutor] lists+sort
In-Reply-To: <CAGZAPF4fDSw2yE_L+zgP5k12KhSGcAQEmd0mnMij2K259t2quw@mail.gmail.com>
References: <CAK0ikxdzexV5v=u0DmbUVttANXeQezwJgr04bsrWYX0dQzp_1A@mail.gmail.com>
 <CAGZAPF4fDSw2yE_L+zgP5k12KhSGcAQEmd0mnMij2K259t2quw@mail.gmail.com>
Message-ID: <CAPM-O+x22eh1TQymT_SPSAB=o2gy1c_QQoE=aJnQy7vWKzo86A@mail.gmail.com>

On Mon, Jan 4, 2016 at 5:35 PM, Danny Yoo <dyoo at hashcollision.org> wrote:

> On Jan 4, 2016 11:00 AM, "Pooja Bhalode" <poojabhalode11 at gmail.com> wrote:
> >
> > Hi, I wanted to check if this program can be used to merge the lists
> > together and sort them. This seems to work, but i wanted to check if
> there
> > are drawbacks in writing it in this manner.
>
> You may be missing some important details or misunderstanding a crucial
> detail.
>
> For example, the term "merge" in this context usually had a very specific
> technical meaning.
>
> Are your two input lists already sorted?
>
> The use of the term "linear" in the function definition:
>
> > def linear_merge(list1, list2):
>
> has a particular meaning in terms of algorithmic complexity, and the first
> implementation does not satisfy it: it does not perform linearly due to the
> internal use of the sort().
>
> If you have questions, please feel free to ask.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

You may also take a look at this link:
http://stackoverflow.com/questions/7237875/linear-merging-for-lists-in-python

It appears that the poster was going through Googles python tutorials

-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From steve at pearwood.info  Mon Jan  4 19:53:51 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 5 Jan 2016 11:53:51 +1100
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <n6eifp$2a7$1@ger.gmane.org>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <n6b6o8$isq$1@ger.gmane.org> <n6eifp$2a7$1@ger.gmane.org>
Message-ID: <20160105005351.GH23700@ando.pearwood.info>

On Mon, Jan 04, 2016 at 07:50:59PM +0000, Mark Lawrence wrote:
> On 03/01/2016 13:12, Alan Gauld wrote:

[snip unnecessary quoting]

> >There are several reasons although your technique is far from
> >the worst way of doing things. And the format string here would probably
> >be better written as:
> >
> >print("You've visited {0} & {2}.".format(island, new))
[...]

> Three reasons for why it's better but it doesn't actually work as given.

Mark, please don't bottom-post. That has all the disadvantages of 
top-posting, and none of the advantages. (In other words, it is *worse* 
than top-posting.) Instead, you should trim your quoting and post 
inline.


> >>> island = "Isle Of Wight"
> >>> new = "Isle of Wong"
> >>> print("You've visited {0} & {2}.".format(island, new))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> IndexError: tuple index out of range

I think that's an obvious typo that should be {1} instead of {2}.


-- 
Steve

From katye2007 at gmail.com  Mon Jan  4 19:37:27 2016
From: katye2007 at gmail.com (yehudak .)
Date: Tue, 5 Jan 2016 02:37:27 +0200
Subject: [Tutor] Swapping values
Message-ID: <CAE3ie41NHTOuuSV++Ov=OK2+6-5QAPpkG_Lo8SxdNJ-QSarvzw@mail.gmail.com>

In Python we can swap values between two variable a and b this way:

a = 3; b = 7
print(a, b)   # =====> 3  7

a, b = b, a  # swapping!
print(a, b)   # =====> 7  3

How does this work?

If I split the 'magic' line into:
a = b; b = a
without a temp variable I get:
print(a, b)   # =====> 7  7

Thank you,
Yehuda (Israel)

From steve at pearwood.info  Mon Jan  4 20:10:50 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 5 Jan 2016 12:10:50 +1100
Subject: [Tutor] Swapping values
In-Reply-To: <CAE3ie41NHTOuuSV++Ov=OK2+6-5QAPpkG_Lo8SxdNJ-QSarvzw@mail.gmail.com>
References: <CAE3ie41NHTOuuSV++Ov=OK2+6-5QAPpkG_Lo8SxdNJ-QSarvzw@mail.gmail.com>
Message-ID: <20160105011050.GI23700@ando.pearwood.info>

On Tue, Jan 05, 2016 at 02:37:27AM +0200, yehudak . wrote:
> In Python we can swap values between two variable a and b this way:
> 
> a = 3; b = 7
> print(a, b)   # =====> 3  7
> 
> a, b = b, a  # swapping!
> print(a, b)   # =====> 7  3
> 
> How does this work?

The right-hand side of the assignment is evaluated fully before the 
assignment happens. Think about how you might do the same:

# right-hand side: ... = b, a
- write down b
- write down a

# left-hand side: a, b = ...
- take the first number written down (was b) and assign it to a
- take the second number written down (was a) and assign it to b

except of course Python doesn't literally "write things down".

To be technical, Python uses a stack. We can look at the byte code 
generated:

py> import dis
py> code = compile("a, b = b, a", "", "single")
py> dis.dis(code)
  1           0 LOAD_NAME                0 (b)
              3 LOAD_NAME                1 (a)
              6 ROT_TWO
              7 STORE_NAME               1 (a)
             10 STORE_NAME               0 (b)
             13 LOAD_CONST               0 (None)
             16 RETURN_VALUE


The first two commands LOAD_NAME pushes the values of b and a onto the 
internal stack. ROT_TWO swaps them around, then STORE_NAME gets called 
twice to assign them to a and b.


> If I split the 'magic' line into:
> a = b; b = a
> without a temp variable I get:
> print(a, b)   # =====> 7  7

In this case, what you are doing is:

- let a equal the value that b has right now;
- let b equal the value that a has right now 
  (which is of course, the same as b)


so the second part (b = a) doesn't actually do anything. If you just 
write

    a = b

of course now a and b are the same.



-- 
Steve

From breamoreboy at yahoo.co.uk  Mon Jan  4 23:10:29 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 5 Jan 2016 04:10:29 +0000
Subject: [Tutor] To FORMAT or not to
In-Reply-To: <20160105005351.GH23700@ando.pearwood.info>
References: <CAE3ie40p8HDmcMNBQxYf9GSurTPkzWP+jF6=H+Jpxig=Se6sLQ@mail.gmail.com>
 <n6b6o8$isq$1@ger.gmane.org> <n6eifp$2a7$1@ger.gmane.org>
 <20160105005351.GH23700@ando.pearwood.info>
Message-ID: <n6ffoc$thf$1@ger.gmane.org>

Sorry no as that would often leave out data that I consider important. 
I have no interest in whether or not you agree with my opinion.

On 05/01/2016 00:53, Steven D'Aprano wrote:
> On Mon, Jan 04, 2016 at 07:50:59PM +0000, Mark Lawrence wrote:
>> On 03/01/2016 13:12, Alan Gauld wrote:
>
> [snip unnecessary quoting]
>
>>> There are several reasons although your technique is far from
>>> the worst way of doing things. And the format string here would probably
>>> be better written as:
>>>
>>> print("You've visited {0} & {2}.".format(island, new))
> [...]
>
>> Three reasons for why it's better but it doesn't actually work as given.
>
> Mark, please don't bottom-post. That has all the disadvantages of
> top-posting, and none of the advantages. (In other words, it is *worse*
> than top-posting.) Instead, you should trim your quoting and post
> inline.
>
>
>>>>> island = "Isle Of Wight"
>>>>> new = "Isle of Wong"
>>>>> print("You've visited {0} & {2}.".format(island, new))
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>> IndexError: tuple index out of range
>
> I think that's an obvious typo that should be {1} instead of {2}.
>
>


-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From dyoo at hashcollision.org  Tue Jan  5 01:06:27 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 4 Jan 2016 22:06:27 -0800
Subject: [Tutor] lists+sort
In-Reply-To: <CAPM-O+x22eh1TQymT_SPSAB=o2gy1c_QQoE=aJnQy7vWKzo86A@mail.gmail.com>
References: <CAK0ikxdzexV5v=u0DmbUVttANXeQezwJgr04bsrWYX0dQzp_1A@mail.gmail.com>
 <CAGZAPF4fDSw2yE_L+zgP5k12KhSGcAQEmd0mnMij2K259t2quw@mail.gmail.com>
 <CAPM-O+x22eh1TQymT_SPSAB=o2gy1c_QQoE=aJnQy7vWKzo86A@mail.gmail.com>
Message-ID: <CAGZAPF4xJ_=U5=AW4Q6bSuWRSPEikMPXq-gFk3GpLzLstdjBxQ@mail.gmail.com>

>
> You may also take a look at this link:
> http://stackoverflow.com/questions/7237875/linear-merging-for-lists-in-python
>
> It appears that the poster was going through Googles python tutorials

Hi Joel,

Ah.  Nice catch!  Yes, that looks like it.  It looks like this comes
from the material at https://developers.google.com/edu/python/, as
part of the list2.py exercises referenced by the
google-python-exercises.zip file in
https://developers.google.com/edu/python/set-up

Just as a warning: those materials assume that the participant already
knows programming and computer science theory, and so they probably
won't work too well for complete beginners.

From steve at pearwood.info  Tue Jan  5 06:03:53 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 5 Jan 2016 22:03:53 +1100
Subject: [Tutor] lists+sort
In-Reply-To: <CAK0ikxdzexV5v=u0DmbUVttANXeQezwJgr04bsrWYX0dQzp_1A@mail.gmail.com>
References: <CAK0ikxdzexV5v=u0DmbUVttANXeQezwJgr04bsrWYX0dQzp_1A@mail.gmail.com>
Message-ID: <20160105110352.GL23700@ando.pearwood.info>

On Mon, Jan 04, 2016 at 12:34:57PM -0500, Pooja Bhalode wrote:
> Hi, I wanted to check if this program can be used to merge the lists
> together and sort them. This seems to work, but i wanted to check if there
> are drawbacks in writing it in this manner.
> 
> My solution:
> 
> def linear_merge(list1, list2):
>     for num in list2:
>         list1.append(num)
>     list1.sort()
>     return list1

One small change:

def linear_merge(list1, list2):
    list1.extend(list2)
    list1.sort()
    return list1

is easier to write and faster. For use in an actual program, this method 
is perfectly acceptible, although this version modifies the first list 
in place rather than make a copy. If you want a copy:

def linear_merge(list1, list2):
    result = list1[:]
    result.extend(list2)
    result.sort()
    return result


Or more compact, but not quite as efficient:

def linear_merge(list1, list2):
    return sorted(list1 + list2)

You should play around with these and see if you can understand how they 
differ.

For a practice exercise, none of the above might be acceptible. Usually, 
when people talk about "merging" lists, they mean to avoid calling sort. 
Sorting, on average, takes time proportional to N*log N, where N is the 
number of items. So if you have 1000 items, sorting will take time 
proportional to 1000*(log 1000), or 3000. But "merging" should take time 
proportional to just N, or 1000. So in theory, merging may be faster.

So for practice exercises, it is often expected that you do not just 
add the two lists together and sort as we did above.

Let's look at their code:

> def linear_merge(list1, list2):
>   result = []
>   # Look at the two lists so long as both are non-empty.
>   # Take whichever element [0] is smaller.
>   while len(list1) and len(list2):
>     if list1[0] < list2[0]:
>       result.append(list1.pop(0))
>     else:
>       result.append(list2.pop(0))
>   # Now tack on what's left
>   result.extend(list1)
>   result.extend(list2)
>   return result

As you can see, there is no call to sort. (This does assume that each 
list is itself sorted, but that is normal for this type of question.) So 
the function starts with a new, empty, list and then it keeps looking at 
the two lists. It looks at the first item from each list, picks the 
smaller, and moves it to the new list. When one or the other list has 
run out of items, the function then does a quick copy of the rest of the 
items from the other.


However, while this might technically be a merge, it is actually *very* 
inefficient. It will actually take time proportional to N**2, due to the 
list.pop calls. Each time you pop an item from the *beginning* of a 
list, all the other items have to be moved back one space. So if you 
have 10 items in the list, and remove the first one, the remaining 9 
have to move; then you remove the first, now the remaining 8 have to 
move; and so on. If you don't understand this, don't worry about it too 
much, the important thing is that the answer given will perform very 
slowly for large lists with millions of items. Here's a better version:


def merge(lista, listb):
    # Assumes that both lista and listb are already sorted.
    new = []
    i = 0  # Index of an item in lista
    j = 0  # Index of an item in listb
    while i < len(lista) and j < len(listb):
        if lista[i] < listb[j]:
            new.append(lista[i])
            i += 1
        else:
            new.append(listb[j])
            j += 1
    new.extend(lista[i:])
    new.extend(listb[j:])
    return new



-- 
Steve

From __peter__ at web.de  Tue Jan  5 07:03:23 2016
From: __peter__ at web.de (Peter Otten)
Date: Tue, 05 Jan 2016 13:03:23 +0100
Subject: [Tutor] lists+sort
References: <CAK0ikxdzexV5v=u0DmbUVttANXeQezwJgr04bsrWYX0dQzp_1A@mail.gmail.com>
Message-ID: <n6gbeh$pvv$1@ger.gmane.org>

Pooja Bhalode wrote:

> Hi, I wanted to check if this program can be used to merge the lists
> together and sort them. This seems to work, but i wanted to check if there
> are drawbacks in writing it in this manner.

When you start out lists are the natural datatype to use, but as you get 
more experienced you'll often switch to generators and iterators. These 
allow you to defer calculations until they are actually necessary and to 
reduce memory footprint. For example:

>>> import heapq
>>> a = range(0, 10**100, 2)
>>> b = range(0, 10**100, 7)
>>> for item in heapq.merge(a, b):
...     print(item)
...     if item > 20:
...         break
... 
0
0
2
4
6
7
8
10
12
14
14
16
18
20
21

A list-based implementation would try to build a list with 

>>> 10**100//2 + 10**100//7
6428571428571428571428571428571428571428571428571428571428571428571428571428571428571428571428571428

items. CPython can't even determine the length of such a list:

>>> len(range(0, 10**100, 2))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C ssize_t

Anyway, heapq.merge() is written in Python, so you can easily take a look:

https://hg.python.org/cpython/file/737efcadf5a6/Lib/heapq.py#l349





From fosiul at gmail.com  Tue Jan  5 04:46:09 2016
From: fosiul at gmail.com (Fosiul Alam)
Date: Tue, 5 Jan 2016 09:46:09 +0000
Subject: [Tutor] how to grep a word and make a dictionary from multiple
 lines.
In-Reply-To: <20160104235452.GM21994@raspberrypi>
References: <CAPBMF6xmae3LGaJRf+LaRBk1V2hiGCAtB9JNRScs5+-NoXPJJg@mail.gmail.com>
 <20160104235452.GM21994@raspberrypi>
Message-ID: <CAPBMF6y5kt8nc=ydTydmJ3Ps2=c8HDdop4LgOvjpdaaN=GvzSw@mail.gmail.com>

Hi
so the logic will be like bellow :-

a)Start to read the file,
b)start a line which start with  "3600601"
c) Iterate through the line till i see another line which starts with
"3600601"
d) read the last line before 3600601
e)cut all the path and create a dictionary which will have one key (LUN ID
3600601 ) and multiple values (PATH
*0:0:0:129,**1:0:0:129,*0:0:2:129,1:0:2:129)

f) then look up each dictinary by Values(*0:0:0:129) * to get the required
LUN ID

does the logic sound right ?

mpathdz (360060165656565634348a739e511) dm-134 DGC,VRAID
size=200G features='0' hwhandler='1 alua' wp=rw
|*-+- policy='round-robin 0' prio=130 status=active
*|* |- 0:0:0:129 sddz  128:16   active ready running
*|* `- 1:0:0:129 sdwd  69:656   active ready running
*`-+- policy='round-robin 0' prio=10 status=enabled
  |- 0:0:2:129 sdnd  70:496   active ready running
  `- 1:0:2:129 sdafg 68:864   active ready running
mpathcu (3600601323h42h2k323asdf33511) dm-103 DGC,VRAID
size=200G features='0' hwhandler='1 alua' wp=rw
|*-+- policy='round-robin 0' prio=130 status=active
*|* |- 1:0:2:98  sdaeb 66:880   active ready running
*|* `- 0:0:2:98  sdly  69:256   active ready running
*`-+- policy='round-robin 0' prio=10 status=enabled
  |- 0:0:0:98  sdcu  70:32    active ready running
  `- 1:0:0:98  sduy  67:672   active ready running
mpathbp (36003434343eere8b69e411) dm-36 DGC,RAID 5
size=100G features='0' hwhandler='1 alua' wp=rw
|*-+- policy='round-robin 0' prio=130 status=active
*|* |- 0:0:0:68  sdbq  68:64    active ready running
*|* `- 1:0:0:68  sdtu  65:704   active ready running
*`-+- policy='round-robin 0' prio=10 status=enabled
  |- 0:0:2:68  sdku  67:288   active ready running
  `- 1:0:2:68  sdacx 8:912    active ready running



On Mon, Jan 4, 2016 at 11:54 PM, David Rock <david at graniteweb.com> wrote:

> * Fosiul Alam <fosiul at gmail.com> [2016-01-04 22:29]:
> > Hi Expert,
> > I am learning python to do some system admin code, i am still in the
> > process of learning but I need a help on bellow code in urgent, i will be
> > really greatfull for any help
> >
> > Basically i wanted like this :-
> >
> > grep a line which has 1:0:0:129 , and get the LUN number(i.e
> > 360060165656565634348a739e511) for this path
> >
> > so the Dictionalry will be :
> >
> > dict = {'360060165656565634348a739e511': '1:0:0:129',
> > '3600601323h42h2k323asdf33511': 1:0:2:98};
> >
> >
> > How can i do this ?
>
> based on your multipath output, one thing you have to take into account
> is that the LUN value comes before the SCSI value.  You are going to
> have to walk the file and for each mpath line, create a dict, then add
> your SCSI values to that entry.  You will have to find a way to break
> out of your logic when the next mpath is seen.
>
> You will also want to think about what to do with all your LUNs.  You
> have four per mpath, not one.  Do you want to capture only one, or do
> you need all of them?
>
> --
> David Rock
> david at graniteweb.com
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Regards
Fosiul Alam

From katye2007 at gmail.com  Tue Jan  5 07:02:38 2016
From: katye2007 at gmail.com (yehudak .)
Date: Tue, 5 Jan 2016 14:02:38 +0200
Subject: [Tutor] Thank you Steve.
Message-ID: <CAE3ie4060wP0CRfqDpjvYBJUcDL468TB4p8BwViZUReC_r=UoA@mail.gmail.com>

Does this 'trick' work also in other programming languages?

Yehuda

From alan.gauld at btinternet.com  Tue Jan  5 08:28:52 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 5 Jan 2016 13:28:52 +0000
Subject: [Tutor] Swapping values
In-Reply-To: <CAE3ie41NHTOuuSV++Ov=OK2+6-5QAPpkG_Lo8SxdNJ-QSarvzw@mail.gmail.com>
References: <CAE3ie41NHTOuuSV++Ov=OK2+6-5QAPpkG_Lo8SxdNJ-QSarvzw@mail.gmail.com>
Message-ID: <n6ggej$c5h$1@ger.gmane.org>

On 05/01/16 00:37, yehudak . wrote:
> In Python we can swap values between two variable a and b this way:
> 
> a = 3; b = 7
> print(a, b)   # =====> 3  7
> 
> a, b = b, a  # swapping!
> print(a, b)   # =====> 7  3
> 
> How does this work?

Steven has given you a detailed answer showing how Python does it
at the low level. Conceptually there is another way to see it.

You are not actually swapping the values.
You are assigning members of a tuple. For example Python allows you
to do this:

a,b,c,d = (1,2,3,4)

And a = 1, b=2 etc.

ie Python is effectively doing the assignments:

t = (1,2,3,4)
a=t[0]
b=t[1]
etc...

Now the parens above are purely to highlight that the RHS is a tuple but
it can quite correctly be written without:

a,b,c,d = 1,2,3,4

And still a = 1, b=2 etc.
This is a fairly common idiom for initializing multiple variables.

So when in your example you write

a,b = b,a

What you are doing is just a specific case of the more general
tuple unpacking seen above, but limited to two members.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Tue Jan  5 08:30:36 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 5 Jan 2016 13:30:36 +0000
Subject: [Tutor] Thank you Steve.
In-Reply-To: <CAE3ie4060wP0CRfqDpjvYBJUcDL468TB4p8BwViZUReC_r=UoA@mail.gmail.com>
References: <CAE3ie4060wP0CRfqDpjvYBJUcDL468TB4p8BwViZUReC_r=UoA@mail.gmail.com>
Message-ID: <n6gghr$c5h$2@ger.gmane.org>

On 05/01/16 12:02, yehudak . wrote:
> Does this 'trick' work also in other programming languages?

Not generally, but I'm sure there are a few specific cases.
It really depends on whether they support unpacking
of data structures.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From richkappler at gmail.com  Tue Jan  5 10:31:28 2016
From: richkappler at gmail.com (richard kappler)
Date: Tue, 5 Jan 2016 10:31:28 -0500
Subject: [Tutor] multipart socket streaming problem: the socket
Message-ID: <CAG7edPEDSk4FdPxpD0GNnED2mPsm1Ba=0qhLhPtuxObZVV3htw@mail.gmail.com>

This is a continuation of the thread 'reading an input stream' I had to
walk away from for a few days due to the holidays and then other work
considerations, and I figured it best to break my confusion into separate
chunks, I hope that's appropriate. In short, my script needs to read a
stream of xml data from a socket (port 2008), the data coming in from as
many as 30 different machines, but usually 4 or less, as many as 3 messages
per second from each machine at times, messages block format delimited by
stx(\x02) and etx (\x03), send the data in those blocks to a parser
(already built using lxml and an xslt file) and send it out to splunk using
a native 'event writer'.

Here's what I have thus far for the socket, it works though has not been
tested for multiple connections yet, how should it be improved to be more
Pythonic, robust and efficient?:

#!/usr/bin/env python

import socket

# receive socket
sockrx = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sockrx_address = ('', 2008)
print 'opening sockrx on %s port %s' % sockrx_address
try:
    sockrx.bind(sockrx_address)
except socket.error as msg:
    print 'Bind failed. Error code: ' + str(msg)
    sys.exit()

sockrx.listen(5)
print 'listening'

# wait for a connection
connection, client_address = sockrx.accept()
data = connection.recv(8192)

I have questions including:
- Should I increase the 5 connection limit to 30+ as it may be listening to
that many machines?
- Buffer size is set at 8192, yet the messages may be much larger, can I
safely increase that? Should I?
- I'm presuming that the OS handles assembly of a message that comes in
more than one packet using the TCP/IP protocols, but is that true?

regards, Richard

From richkappler at gmail.com  Tue Jan  5 10:38:51 2016
From: richkappler at gmail.com (richard kappler)
Date: Tue, 5 Jan 2016 10:38:51 -0500
Subject: [Tutor] multipart socket streaming problem: reading the stream
Message-ID: <CAG7edPHE3SU8E9=AgP4_MPu5EED3i457a-nFYyO5jO1gW+46Gw@mail.gmail.com>

This is a continuation of the thread 'reading an input stream' I had to
walk away from for a few days due to the holidays and then other work
considerations, and I figured it best to break my confusion into separate
chunks, I hope that's appropriate. In short, my script needs to read a
stream of xml data from a socket (port 2008), the data coming in from as
many as 30 different machines, but usually 4 or less, as many as 3 messages
per second from each machine at times, messages block format delimited by
stx(\x02) and etx (\x03), send the data in those blocks to a parser
(already built using lxml and an xslt file) and send it out to splunk using
a native 'event writer'.

I am a bit lost in the woods on 'reading' the stream. My first attempt
tried to read the stream directly (from the buffer 'I think'):

#!/usr/bin/env python


import socket

import lxml.etree as ET


# receive socket

sockrx = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

sockrx_address = ('', 2008)

print 'opening sockrx on %s port %s' % sockrx_address

try:

    sockrx.bind(sockrx_address)

except socket.error as msg:

    print 'Bind failed. Error code: ' + str(msg)

    sys.exit()


sockrx.listen(5)

print 'listening'


while True:

    # wait for a connection

    connection, client_address = sockrx.accept()

    data = connection.recv(8192)

    if data:

        print 'receiving data'

        f1 = open('parser.out', 'a')

        xslt = ET.parse('stack13.xsl')

        dom = ET.parse(data)

        transform = ET.XSLT(xslt)

        newdom = transform(dom)

        f1.write(str(newdom))



# close sockrx

connection.close()


f1.close()


This did not work:

Traceback (most recent call last):

  File "streamer-01.py", line 27, in <module>

    dom = ET.parse(data)

  File "lxml.etree.pyx", line 3239, in lxml.etree.parse
(src/lxml/lxml.etree.c:69955)

  File "parser.pxi", line 1748, in lxml.etree._parseDocument
(src/lxml/lxml.etree.c:102066)

  File "parser.pxi", line 1774, in lxml.etree._parseDocumentFromURL
(src/lxml/lxml.etree.c:102330)

  File "parser.pxi", line 1678, in lxml.etree._parseDocFromFile
(src/lxml/lxml.etree.c:101365)

  File "parser.pxi", line 1110, in lxml.etree._BaseParser._parseDocFromFile
(src/lxml/lxml.etree.c:96817)

  File "parser.pxi", line 582, in
lxml.etree._ParserContext._handleParseResultDoc
(src/lxml/lxml.etree.c:91275)

  File "parser.pxi", line 683, in lxml.etree._handleParseResult
(src/lxml/lxml.etree.c:92461)

  File "parser.pxi", line 620, in lxml.etree._raiseParseError
(src/lxml/lxml.etree.c:91722)

IOError: Error reading file

and then the error message printed out the entire file to screen, which I
won't here  for brevity.


I was, however, able to receive the the stream as above, write it to a
file, then read the file to the parser and it worked. I tired using
makefile() but that did not work either. I'm looking for some general
direction here, more specific questions to follow.

regards, Richard

From richkappler at gmail.com  Tue Jan  5 10:42:59 2016
From: richkappler at gmail.com (richard kappler)
Date: Tue, 5 Jan 2016 10:42:59 -0500
Subject: [Tutor] multipart socket streaming problem: the delimiters
Message-ID: <CAG7edPEZ-ysvO3VyHThPeRLj1nG0w==Zu8HFjoKKHwmYN_gAYA@mail.gmail.com>

This is a continuation of the thread 'reading an input stream' I had to
walk away from for a few days due to the holidays and then other work
considerations, and I figured it best to break my confusion into separate
chunks, I hope that's appropriate. In short, my script needs to read a
stream of xml data from a socket (port 2008), the data coming in from as
many as 30 different machines, but usually 4 or less, as many as 3 messages
per second from each machine at times, messages block format delimited by
stx(\x02) and etx (\x03), send the data in those blocks to a parser
(already built using lxml and an xslt file) and send it out to splunk using
a native 'event writer'.

Once I figure out how to get the messages in and read in general (my
current test file has only one message) I'll need to figure out how to pull
them, one at a time (one message from stx to etx) and send to the parser. I
am totally lost here. Once I can get the data read from stream rather from
file, how do I write the part of the script that recognizes block format?
I've looked for libraries but am apparently not writing a good search, as I
always get sites that discuss blocking and non-blocking sockets, not block
format messages. Is there a library or tutorial out there anyone knows of
that might help with this?

regards, Richard

From richkappler at gmail.com  Tue Jan  5 10:46:08 2016
From: richkappler at gmail.com (richard kappler)
Date: Tue, 5 Jan 2016 10:46:08 -0500
Subject: [Tutor] multipart socket streaming problem: threading??
Message-ID: <CAG7edPGmdFQm+Rvwcxzvh_e-AwAB7LnBrVosxsuw0ZeU+W49Tw@mail.gmail.com>

This is a continuation of the thread 'reading an input stream' I had to
walk away from for a few days due to the holidays and then other work
considerations, and I figured it best to break my confusion into separate
chunks, I hope that's appropriate. In short, my script needs to read a
stream of xml data from a socket (port 2008), the data coming in from as
many as 30 different machines, but usually 4 or less, as many as 3 messages
per second from each machine at times, messages block format delimited by
stx(\x02) and etx (\x03), send the data in those blocks to a parser
(already built using lxml and an xslt file) and send it out to splunk using
a native 'event writer'.

And finally, for now, I suspect I'll need to run each 'connection', that is
the data from each sending machine on a separate thread. I'm not well
versed in threading, but have read some of the documentation and it seems
pretty straight forward, if I have questions I will of course ask, but the
preliminary question is: Do I in fact need to do this? Is threading the way
to go or is just running all the data through the same 'pipe'
feasible/Pythonic/efficient?

regards, Richard

From david.rock at gmail.com  Tue Jan  5 13:07:47 2016
From: david.rock at gmail.com (David Rock)
Date: Tue, 5 Jan 2016 12:07:47 -0600
Subject: [Tutor] how to grep a word and make a dictionary from multiple
 lines.
In-Reply-To: <CAPBMF6y5kt8nc=ydTydmJ3Ps2=c8HDdop4LgOvjpdaaN=GvzSw@mail.gmail.com>
References: <CAPBMF6xmae3LGaJRf+LaRBk1V2hiGCAtB9JNRScs5+-NoXPJJg@mail.gmail.com>
 <20160104235452.GM21994@raspberrypi>
 <CAPBMF6y5kt8nc=ydTydmJ3Ps2=c8HDdop4LgOvjpdaaN=GvzSw@mail.gmail.com>
Message-ID: <20160105180747.GN21994@raspberrypi>

* Fosiul Alam <fosiul at gmail.com> [2016-01-05 09:46]:
> Hi
> so the logic will be like bellow :-
> 
> a)Start to read the file,
> b)start a line which start with  "3600601"
> c) Iterate through the line till i see another line which starts with
> "3600601"
> d) read the last line before 3600601
> e)cut all the path and create a dictionary which will have one key (LUN ID
> 3600601 ) and multiple values (PATH
> *0:0:0:129,**1:0:0:129,*0:0:2:129,1:0:2:129)
> 
> f) then look up each dictinary by Values(*0:0:0:129) * to get the required
> LUN ID
> 
> does the logic sound right ?
> 
> mpathdz (360060165656565634348a739e511) dm-134 DGC,VRAID
> size=200G features='0' hwhandler='1 alua' wp=rw
> |*-+- policy='round-robin 0' prio=130 status=active
> *|* |- 0:0:0:129 sddz  128:16   active ready running
> *|* `- 1:0:0:129 sdwd  69:656   active ready running
> *`-+- policy='round-robin 0' prio=10 status=enabled
>   |- 0:0:2:129 sdnd  70:496   active ready running
>   `- 1:0:2:129 sdafg 68:864   active ready running
> mpathcu (3600601323h42h2k323asdf33511) dm-103 DGC,VRAID
> size=200G features='0' hwhandler='1 alua' wp=rw
> |*-+- policy='round-robin 0' prio=130 status=active
> *|* |- 1:0:2:98  sdaeb 66:880   active ready running
> *|* `- 0:0:2:98  sdly  69:256   active ready running
> *`-+- policy='round-robin 0' prio=10 status=enabled
>   |- 0:0:0:98  sdcu  70:32    active ready running
>   `- 1:0:0:98  sduy  67:672   active ready running
> mpathbp (36003434343eere8b69e411) dm-36 DGC,RAID 5
> size=100G features='0' hwhandler='1 alua' wp=rw
> |*-+- policy='round-robin 0' prio=130 status=active
> *|* |- 0:0:0:68  sdbq  68:64    active ready running
> *|* `- 1:0:0:68  sdtu  65:704   active ready running
> *`-+- policy='round-robin 0' prio=10 status=enabled
>   |- 0:0:2:68  sdku  67:288   active ready running
>   `- 1:0:2:68  sdacx 8:912    active ready running

If that's what you need, then that would probably work (more or less).
A few issues to make sure you address:

1. not all multipath entries start with "3600601".  mpathbp, for
example, is "3600343".  You would probably be better off looking for
"mpath"

2. looking up by value would work, but maybe you should do it the other
direction.  Your SCSI values will be unique, so use that as the key


a "simpler" logic is probably:

a) read file
b) store the LUN value from your mpath line
c) iterate over lines and find SCSI key, then apply the LUN as the value
for the key
d) when you hit a new mpath line, simply update your LUN value and keep
going

Then you can look up your LUN by SCSI key directly.

-- 
David Rock
david at graniteweb.com

From cegarcia0323 at gmail.com  Tue Jan  5 12:27:20 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Tue, 5 Jan 2016 12:27:20 -0500
Subject: [Tutor] Help with loops
Message-ID: <CABV0pWzxxE2ppYzOttrb-xHC_bTz14U=qbkJ07jbET_R6HJ7oQ@mail.gmail.com>

I need to create a loop to print out the most common phrases with the
counts 5 or greater and the rest to be bucketed into other category. How do
I write the loop to print out the common phrases that have counts of 5 or
more? This is my code so far:

import csv#from sys import argvfrom collections import defaultdictfrom
collections import Counter#script, filename = argv
data = defaultdict(list)
class dictionary:
    with open ('practice.csv', 'rb') as f:
       reader = csv.reader(f)
       #text_file = open("output.txt", "w")
       next(reader, None)
       for row in reader:
          data[row[2]].append(row[3])

       #text_file.write("%r" % data)

       #text_file.close()

   #print(data)

   text_file = open("count.txt", "w")
   data_count = Counter()
   for d in data.values():
      data_count += Counter(d)
   print data_count.most_common(5)
   text_file.write("%r" % data_count.most_common(5))
   text_file.close()

From joel.goldstick at gmail.com  Tue Jan  5 14:13:51 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 5 Jan 2016 14:13:51 -0500
Subject: [Tutor] Help with loops
In-Reply-To: <CABV0pWzxxE2ppYzOttrb-xHC_bTz14U=qbkJ07jbET_R6HJ7oQ@mail.gmail.com>
References: <CABV0pWzxxE2ppYzOttrb-xHC_bTz14U=qbkJ07jbET_R6HJ7oQ@mail.gmail.com>
Message-ID: <CAPM-O+zW8nFsKUKP_OWOaTC6_XbotO0w_WxFKmWLFS2M9TUUYw@mail.gmail.com>

On Tue, Jan 5, 2016 at 12:27 PM, Chelsea G <cegarcia0323 at gmail.com> wrote:

> I need to create a loop to print out the most common phrases with the
> counts 5 or greater and the rest to be bucketed into other category. How do
> I write the loop to print out the common phrases that have counts of 5 or
> more? This is my code so far:
>

Can you explain what you mean by common phrases with a count of 5 or more?
Show some same input file data here to better explain your assignment.

>
> import csv#from sys import argvfrom collections import defaultdictfrom
> collections import Counter#script, filename = argv
> data = defaultdict(list)
> class dictionary:
>
I think this looks like function code -- not a class

>     with open ('practice.csv', 'rb') as f:
>        reader = csv.reader(f)
>        #text_file = open("output.txt", "w")
>        next(reader, None)
>        for row in reader:
>           data[row[2]].append(row[3])
>
> why is row[2] and row[3] the only data you want to collect?  You defined
data outside of your function -- not good.

>        #text_file.write("%r" % data)
>
>        #text_file.close()
>
>    #print(data)
>
>    text_file = open("count.txt", "w")
>    data_count = Counter()
>    for d in data.values():
>       data_count += Counter(d)
>    print data_count.most_common(5)
>    text_file.write("%r" % data_count.most_common(5))
>    text_file.close()
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

This code is quite a mess.  That's ok.  Try to write code to just read the
file first.  Work from there

-- 
Joel Goldstick
http://joelgoldstick.com:8000/stats/birthdays
<http://joelgoldstick.com/stats/birthdays>

From katye2007 at gmail.com  Tue Jan  5 14:47:51 2016
From: katye2007 at gmail.com (yehudak .)
Date: Tue, 5 Jan 2016 21:47:51 +0200
Subject: [Tutor] Python 3.5.1 64 bit?
Message-ID: <CAE3ie43G4bK8bVUNC3as7OKx8HvYaNZA7-kWXNh_zJdgebZNhA@mail.gmail.com>

I'm using version 3.5 of Python and want to upgrade. The download site
doesn't mention whether it's 32 or 64 bytes. Any info if 64 bytes is
available?

Also, when I upgrade, does the newer version over-ride the old one or I
have first to remove the older version?

Thanks,
Yehuda

From alan.gauld at btinternet.com  Tue Jan  5 18:50:09 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 5 Jan 2016 23:50:09 +0000
Subject: [Tutor] Python 3.5.1 64 bit?
In-Reply-To: <CAE3ie43G4bK8bVUNC3as7OKx8HvYaNZA7-kWXNh_zJdgebZNhA@mail.gmail.com>
References: <CAE3ie43G4bK8bVUNC3as7OKx8HvYaNZA7-kWXNh_zJdgebZNhA@mail.gmail.com>
Message-ID: <n6hkrg$ok4$1@ger.gmane.org>

On 05/01/16 19:47, yehudak . wrote:
> I'm using version 3.5 of Python and want to upgrade. 

3.5 is the latest stream. If you are already using it why
do you want to upgrade? Its easy to get caught up in a frenzy
of chasing the latest dot release but unless there is a
specific bug fix or feature you need there is rarely any
good reason. Personally I haven't got around to moving
from 3.4 yet and on my Windows  7 PC I'm still on v3.3.

> The download site
> doesn't mention whether it's 32 or 64 bytes. Any info if 64 bytes is
> available?

I'm pretty sure there will be a 64bit version on there.
Assuming you are using Windows I seee one labelled

Windows x86-64 executable installer

That looks like 64 bit to me.

The one that says

Windows x86 executable installer

is the 32 bit version.

> Also, when I upgrade, does the newer version over-ride the old one or I
> have first to remove the older version?

They generally leave the old version around - you may have
a need for both (for example to test your codes compatibility
with both versions).


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From akleider at sonic.net  Tue Jan  5 23:58:42 2016
From: akleider at sonic.net (Alex Kleider)
Date: Tue, 05 Jan 2016 20:58:42 -0800
Subject: [Tutor] =?utf-8?q?method=2C_type=3F?=
Message-ID: <333654cd965fbd66934c6597ac37932d@sonic.net>

#!/usr/bin/env python3
# OS: Ubuntu 10.4LTS

# My code:

class JournalLineItem(object):
     """
     """

     def __init__(self, account, debit_or_credit, amount):
         self.account = account
         self.debit_or_credit = debit_or_credit
         self.amount = float(amount)

     def show(self):
         return ("ACNT: {}  ${:0.2f} {}".
             format(self.account, self.amount, self.debit_or_credit))

     def get_line_item(text):
         return JournalLineItem(*text.split())

def test():
     print(
     JournalLineItem.get_line_item("2435 Dr 25.33").show())

if __name__ == "__main__":
     test()

     myquestion = """
What kind of a method/function is get_line_item?
 From what I've read (and not fully understood)
static methods and class methods must have
@staticmethod and @classmethod on the line above them.
get_line_item works as I wanted but it's clearly not the
usual type of method and I don't know how to categorize it.
It's an instance creator- is there a better term?
Is this 'Pythonic' code?
"""
     as_always = """Thanks,

     Alex Kleider"""
~

From cs at zip.com.au  Wed Jan  6 00:40:35 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 6 Jan 2016 16:40:35 +1100
Subject: [Tutor] method, type?
In-Reply-To: <333654cd965fbd66934c6597ac37932d@sonic.net>
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
Message-ID: <20160106054035.GA20551@cskk.homeip.net>

On 05Jan2016 20:58, Alex Kleider <akleider at sonic.net> wrote:
>#!/usr/bin/env python3
># OS: Ubuntu 10.4LTS
>
># My code:
>
>class JournalLineItem(object):
>    """
>    """
>
>    def __init__(self, account, debit_or_credit, amount):
>        self.account = account
>        self.debit_or_credit = debit_or_credit
>        self.amount = float(amount)
>
>    def show(self):
>        return ("ACNT: {}  ${:0.2f} {}".
>            format(self.account, self.amount, self.debit_or_credit))
>
>    def get_line_item(text):
>        return JournalLineItem(*text.split())
>
>def test():
>    print(
>    JournalLineItem.get_line_item("2435 Dr 25.33").show())
>
>if __name__ == "__main__":
>    test()
>
>    myquestion = """
>
>What kind of a method/function is get_line_item?

As written, it is an instance menthod that _thinks_ it is a static method. Not 
that that was what you intended :-)

>From what I've read (and not fully understood)
>static methods and class methods must have
>@staticmethod and @classmethod on the line above them.

As written, you could put @staticmethod above it - it is an ordinary function 
that resides in the class to associate its functionality with the class.

However, what you are probably better off with is @classmethod, for reasons I 
will explain below.

Instance methods expect their first parameter to be the instance (self) and 
class methods expect their first parameter to be the class. This is arranged 
implicitly when you call the method via an instance or class.

>get_line_item works as I wanted but it's clearly not the
>usual type of method and I don't know how to categorize it.

It is confused. There is nothing in the Python language that requires the 
instance to be called "self", that is just convention. So your method thinks 
that the instance is "text", and works on that. So you could do this:

  jli = JournalLineItem(account, debit_or_credit, amount)
  jli.get_line_item()

and the call would go ahead, but fail because a JournalLineItem has no .split 
method. And you can't do this:

  jli = JournalLineItem(account, debit_or_credit, amount)
  jli.get_line_item("2435 Dr 25.33")

because, since it is an instance method, Python implicitly passes in the 
instance (jli) as the first parameter, so the call is equivalent to:

  JournalLineItem.get_line_item(jil, "2435 Dr 25.33")

but the function only expected a single parameter. Badness ensues.

>It's an instance creator- is there a better term?

Generally these are called factories. Normally you would either write this as a 
class method:

  @classmethod
  def get_line_item(cls, text):
    return cls(*text.split())

which will get JournalLineItem when you call it as in your example:

  item = JournalLineItem.get_line_item("2435 Dr 25.33")

but will get the right subclass if you subclass JournalLineItem:

  class SpecialJournalLineItem(JournalLineItem):
    ...

  special_item = SpecialJournalLineItem.get_line_item("2435 Dr 25.33")

Alternatively, if you never expect to subclass this, you could just declare it 
as a normal function outside the class entirely:

  def get_line_item(cls, text):
    return JournalLineItem(*text.split())

Often these functions or methods are called from_blah, so the outside-the-class 
function might be called "JournalLineItem_from_text", and the inside the class 
@classmethod one might be called "from_text". So you might write:

  jli = JournalLineItem.from_text("2435 Dr 25.33")

which reads well and allows you to subclass JournalLineItem later. Therefore I 
would recommend the @classmethod approach. And stylisticly, I would put that up 
the top, just under __init__.

Hoping this helps rather than confuses,
Cameron Simpson <cs at zip.com.au>

From eryksun at gmail.com  Wed Jan  6 01:32:05 2016
From: eryksun at gmail.com (eryk sun)
Date: Wed, 6 Jan 2016 00:32:05 -0600
Subject: [Tutor] Python 3.5.1 64 bit?
In-Reply-To: <n6hkrg$ok4$1@ger.gmane.org>
References: <CAE3ie43G4bK8bVUNC3as7OKx8HvYaNZA7-kWXNh_zJdgebZNhA@mail.gmail.com>
 <n6hkrg$ok4$1@ger.gmane.org>
Message-ID: <CACL+1asBe1TkoM4O8XHZ0Hg-vF4SXOnVmYqOREfzD=n_oegSog@mail.gmail.com>

On Tue, Jan 5, 2016 at 5:50 PM, Alan Gauld <alan.gauld at btinternet.com> wrote:
> On 05/01/16 19:47, yehudak . wrote:
>> I'm using version 3.5 of Python and want to upgrade.
>
> 3.5 is the latest stream. If you are already using it why
> do you want to upgrade? Its easy to get caught up in a frenzy
> of chasing the latest dot release but unless there is a
> specific bug fix or feature you need there is rarely any
> good reason. Personally I haven't got around to moving
> from 3.4 yet and on my Windows  7 PC I'm still on v3.3.

Review the release notes [1] to determine whether you need to upgrade.
For example, 3.5.1 has the following fixes that are specific to
Windows:

Issue #19143: platform module now reads Windows version from
              kernel32.dll to avoid compatibility shims.
Issue #25112: py.exe launcher is missing icons
Issue #25118: Fix a regression of Python 3.5.0 in os.waitpid() on
              Windows.
Issue #25213: Restores requestedExecutionLevel to manifest to
              disable UAC virtualization.
Issue #25361: Disables use of SSE2 instructions in Windows 32-bit
              build
Issue #25450: Updates shortcuts to start Python in installation
              directory.

Issues 25213 and 25361 only affect 32-bit Python.

It also fixes several issues with the new installer:

Issue #25081: Makes Back button in installer go back to upgrade
              page when upgrading.
Issue #25089: Adds logging to installer for case where launcher is
              not selected on upgrade.
Issue #25091: Increases font size of the installer.
Issue #25102: Windows installer does not precompile for -O or -OO.
Issue #25126: Clarifies that the non-web installer will download
              some components.
Issue #25143: Improves installer error messages for unsupported
              platforms.
Issue #25163: Display correct directory in installer when using
              non-default settings.
Issue #25164: Changes default all-users install directory to match
              per-user directory.
Issue #25165: Windows uninstallation should not remove launcher if
              other versions remain
Issue #25715: Python 3.5.1 installer shows wrong upgrade path and
              incorrect logic for launcher detection.

[1]: https://docs.python.org/3.5/whatsnew/changelog.html

>> Also, when I upgrade, does the newer version over-ride the old one or I
>> have first to remove the older version?
>
> They generally leave the old version around - you may have
> a need for both (for example to test your codes compatibility
> with both versions).

On Windows, updating to a newer micro release (i.e. major.minor.micro)
replaces the previous version. However, if you have the 64-bit version
of 3.5.0 installed for all users, you can install the 32-bit version
of 3.5.1 for the current user without affecting the system
installation.

From akleider at sonic.net  Wed Jan  6 02:25:34 2016
From: akleider at sonic.net (Alex Kleider)
Date: Tue, 05 Jan 2016 23:25:34 -0800
Subject: [Tutor] =?utf-8?q?method=2C_type=3F?=
In-Reply-To: <20160106054035.GA20551@cskk.homeip.net>
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
 <20160106054035.GA20551@cskk.homeip.net>
Message-ID: <01bec9f8e4f88f194c2cea6f5adbe5ac@sonic.net>


> Hoping this helps rather than confuses,
> Cameron Simpson <cs at zip.com.au>

It is no more confusing than what I had already read about static and 
class methods.
I guess I was hoping for an easy explanation but such a thing probably 
doesn't exist.
I'll have to slog through the explanation.

Thank you for taking the trouble to help.  It's much appreciated.

Alex



From eryksun at gmail.com  Wed Jan  6 03:01:30 2016
From: eryksun at gmail.com (eryk sun)
Date: Wed, 6 Jan 2016 02:01:30 -0600
Subject: [Tutor] Python 3.5.1 64 bit?
In-Reply-To: <CAE3ie40WFtQk4ft9SSFw=ZE4xnM_8sy9XmUTu8AXyXk6XU=PjA@mail.gmail.com>
References: <CAE3ie43G4bK8bVUNC3as7OKx8HvYaNZA7-kWXNh_zJdgebZNhA@mail.gmail.com>
 <n6hkrg$ok4$1@ger.gmane.org>
 <CACL+1asBe1TkoM4O8XHZ0Hg-vF4SXOnVmYqOREfzD=n_oegSog@mail.gmail.com>
 <CAE3ie40WFtQk4ft9SSFw=ZE4xnM_8sy9XmUTu8AXyXk6XU=PjA@mail.gmail.com>
Message-ID: <CACL+1asfkqEQ1FAUpd2jcemB8-m63C7mMdT52KnGk96MsTrHOQ@mail.gmail.com>

On Wed, Jan 6, 2016 at 1:06 AM, yehudak . <katye2007 at gmail.com> wrote:
> What's wrong with upgrading to newer version?

It depends on how many systems and virtual environments that you're
upgrading. It shouldn't be an issue if it's just a new micro release
for your own development machine. If you come across a regression, you
can just downgrade to the previous version. That said, there's the old
adage that if something isn't broken (for your needs), then you
shouldn't 'fix' it.

Installing a new minor release (e.g. 3.5 to 3.6) is more work, since
you have to reinstall all packages and rebuild extension modules for
source packages. But it's worth it to stay current with the evolution
of the language. On Windows, make sure to set the PY_PYTHON3
environment variable to the version you want as the default "python3".

From __peter__ at web.de  Wed Jan  6 03:19:07 2016
From: __peter__ at web.de (Peter Otten)
Date: Wed, 06 Jan 2016 09:19:07 +0100
Subject: [Tutor] method, type?
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
Message-ID: <n6iils$kq7$1@ger.gmane.org>

Alex Kleider wrote:

> #!/usr/bin/env python3
> # OS: Ubuntu 10.4LTS
> 
> # My code:
> 
> class JournalLineItem(object):
>      """
>      """
> 
>      def __init__(self, account, debit_or_credit, amount):
>          self.account = account
>          self.debit_or_credit = debit_or_credit
>          self.amount = float(amount)
> 
>      def show(self):
>          return ("ACNT: {}  ${:0.2f} {}".
>              format(self.account, self.amount, self.debit_or_credit))
> 
>      def get_line_item(text):
>          return JournalLineItem(*text.split())
> 
> def test():
>      print(
>      JournalLineItem.get_line_item("2435 Dr 25.33").show())
> 
> if __name__ == "__main__":
>      test()
> 
>      myquestion = """
> What kind of a method/function is get_line_item?

Let's see:

>>> class Class:
...     def method(*args): print("args passed:", args)
... 
>>> Class.method
<function Class.method at 0x7fea4219b510>
>>> Class.method(42)
args passed: (42,)

When you call it

Class.method(...)

in Python 3 method is just an ordinary function (in Python 2 it was an 
"unbound method").

>>> inst = Class()
>>> inst.method
<bound method Class.method of <__main__.Class object at 0x7fea421a9828>>
>>> inst.method(42)
args passed: (<__main__.Class object at 0x7fea421a9828>, 42)

When you call it

inst = Class(...)
inst.method(...)

method is a "bound method", i. e. it implicitly adds inst as the first 
argument. Therefore

item = JournalLineItem(...)
item.get_line_text(text) # function gets two args

will fail. As your get_line_item() is actually and "alternative constructor" 
you should make it a class method for it to pass the actual class as the 
first argument with both instances and the class.

@classmethod
def get_line_item(cls, text):
    return cls(*text.split())

When you use that argument to create the instances subclasses then will 
automatically create subclass instances, i. e.

class Modified(JournalLineItem):
   pass

assert isinstance(Modified.get_line_text(...), Modified)

Finally a little table:

invoked with | @staticmethod  | @classmethod    | no decorator
------------------------------------------------------------------
class        | args unchanged | class as 1st arg | args unchanged
instance     | args unchanged | class as 1st arg | inst as 1st arg



From katye2007 at gmail.com  Wed Jan  6 02:06:44 2016
From: katye2007 at gmail.com (yehudak .)
Date: Wed, 6 Jan 2016 09:06:44 +0200
Subject: [Tutor] Python 3.5.1 64 bit?
In-Reply-To: <CACL+1asBe1TkoM4O8XHZ0Hg-vF4SXOnVmYqOREfzD=n_oegSog@mail.gmail.com>
References: <CAE3ie43G4bK8bVUNC3as7OKx8HvYaNZA7-kWXNh_zJdgebZNhA@mail.gmail.com>
 <n6hkrg$ok4$1@ger.gmane.org>
 <CACL+1asBe1TkoM4O8XHZ0Hg-vF4SXOnVmYqOREfzD=n_oegSog@mail.gmail.com>
Message-ID: <CAE3ie40WFtQk4ft9SSFw=ZE4xnM_8sy9XmUTu8AXyXk6XU=PjA@mail.gmail.com>

eryk,
Thank you.
What's wrong with upgrading to newer version?

Yehuda

On Wed, Jan 6, 2016 at 8:32 AM, eryk sun <eryksun at gmail.com> wrote:

> On Tue, Jan 5, 2016 at 5:50 PM, Alan Gauld <alan.gauld at btinternet.com>
> wrote:
> > On 05/01/16 19:47, yehudak . wrote:
> >> I'm using version 3.5 of Python and want to upgrade.
> >
> > 3.5 is the latest stream. If you are already using it why
> > do you want to upgrade? Its easy to get caught up in a frenzy
> > of chasing the latest dot release but unless there is a
> > specific bug fix or feature you need there is rarely any
> > good reason. Personally I haven't got around to moving
> > from 3.4 yet and on my Windows  7 PC I'm still on v3.3.
>
> Review the release notes [1] to determine whether you need to upgrade.
> For example, 3.5.1 has the following fixes that are specific to
> Windows:
>
> Issue #19143: platform module now reads Windows version from
>               kernel32.dll to avoid compatibility shims.
> Issue #25112: py.exe launcher is missing icons
> Issue #25118: Fix a regression of Python 3.5.0 in os.waitpid() on
>               Windows.
> Issue #25213: Restores requestedExecutionLevel to manifest to
>               disable UAC virtualization.
> Issue #25361: Disables use of SSE2 instructions in Windows 32-bit
>               build
> Issue #25450: Updates shortcuts to start Python in installation
>               directory.
>
> Issues 25213 and 25361 only affect 32-bit Python.
>
> It also fixes several issues with the new installer:
>
> Issue #25081: Makes Back button in installer go back to upgrade
>               page when upgrading.
> Issue #25089: Adds logging to installer for case where launcher is
>               not selected on upgrade.
> Issue #25091: Increases font size of the installer.
> Issue #25102: Windows installer does not precompile for -O or -OO.
> Issue #25126: Clarifies that the non-web installer will download
>               some components.
> Issue #25143: Improves installer error messages for unsupported
>               platforms.
> Issue #25163: Display correct directory in installer when using
>               non-default settings.
> Issue #25164: Changes default all-users install directory to match
>               per-user directory.
> Issue #25165: Windows uninstallation should not remove launcher if
>               other versions remain
> Issue #25715: Python 3.5.1 installer shows wrong upgrade path and
>               incorrect logic for launcher detection.
>
> [1]: https://docs.python.org/3.5/whatsnew/changelog.html
>
> >> Also, when I upgrade, does the newer version over-ride the old one or I
> >> have first to remove the older version?
> >
> > They generally leave the old version around - you may have
> > a need for both (for example to test your codes compatibility
> > with both versions).
>
> On Windows, updating to a newer micro release (i.e. major.minor.micro)
> replaces the previous version. However, if you have the 64-bit version
> of 3.5.0 installed for all users, you can install the 32-bit version
> of 3.5.1 for the current user without affecting the system
> installation.
>

From katye2007 at gmail.com  Wed Jan  6 02:13:52 2016
From: katye2007 at gmail.com (yehudak .)
Date: Wed, 6 Jan 2016 09:13:52 +0200
Subject: [Tutor] Thanks Alan.
Message-ID: <CAE3ie42iGp+1AHK7kidykcHq-dvPWEDffUrOquKkVEk8B0nHEA@mail.gmail.com>

I'll look into it.
Yehuda

From katye2007 at gmail.com  Wed Jan  6 03:14:15 2016
From: katye2007 at gmail.com (yehudak .)
Date: Wed, 6 Jan 2016 10:14:15 +0200
Subject: [Tutor] Python 3.5.1 64 bit?
In-Reply-To: <CACL+1asfkqEQ1FAUpd2jcemB8-m63C7mMdT52KnGk96MsTrHOQ@mail.gmail.com>
References: <CAE3ie43G4bK8bVUNC3as7OKx8HvYaNZA7-kWXNh_zJdgebZNhA@mail.gmail.com>
 <n6hkrg$ok4$1@ger.gmane.org>
 <CACL+1asBe1TkoM4O8XHZ0Hg-vF4SXOnVmYqOREfzD=n_oegSog@mail.gmail.com>
 <CAE3ie40WFtQk4ft9SSFw=ZE4xnM_8sy9XmUTu8AXyXk6XU=PjA@mail.gmail.com>
 <CACL+1asfkqEQ1FAUpd2jcemB8-m63C7mMdT52KnGk96MsTrHOQ@mail.gmail.com>
Message-ID: <CAE3ie40uN7GzxXryT4xZTRa9SinnuuDi5TbAehtrzjY5c==nDg@mail.gmail.com>

Thanks eryk.
I've downloaded and installed 3.5.1 64 bit, as Alan suggested.
So far no problem. I can anytime return to 3.5 if it will be needed.

Yehuda

On Wed, Jan 6, 2016 at 10:01 AM, eryk sun <eryksun at gmail.com> wrote:

> On Wed, Jan 6, 2016 at 1:06 AM, yehudak . <katye2007 at gmail.com> wrote:
> > What's wrong with upgrading to newer version?
>
> It depends on how many systems and virtual environments that you're
> upgrading. It shouldn't be an issue if it's just a new micro release
> for your own development machine. If you come across a regression, you
> can just downgrade to the previous version. That said, there's the old
> adage that if something isn't broken (for your needs), then you
> shouldn't 'fix' it.
>
> Installing a new minor release (e.g. 3.5 to 3.6) is more work, since
> you have to reinstall all packages and rebuild extension modules for
> source packages. But it's worth it to stay current with the evolution
> of the language. On Windows, make sure to set the PY_PYTHON3
> environment variable to the version you want as the default "python3".
>

From alan.gauld at btinternet.com  Wed Jan  6 04:52:00 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 6 Jan 2016 09:52:00 +0000
Subject: [Tutor] Python 3.5.1 64 bit?
In-Reply-To: <CAE3ie40WFtQk4ft9SSFw=ZE4xnM_8sy9XmUTu8AXyXk6XU=PjA@mail.gmail.com>
References: <CAE3ie43G4bK8bVUNC3as7OKx8HvYaNZA7-kWXNh_zJdgebZNhA@mail.gmail.com>
 <n6hkrg$ok4$1@ger.gmane.org>
 <CACL+1asBe1TkoM4O8XHZ0Hg-vF4SXOnVmYqOREfzD=n_oegSog@mail.gmail.com>
 <CAE3ie40WFtQk4ft9SSFw=ZE4xnM_8sy9XmUTu8AXyXk6XU=PjA@mail.gmail.com>
Message-ID: <n6io3v$amg$1@ger.gmane.org>

On 06/01/16 07:06, yehudak . wrote:
> eryk,
> Thank you.
> What's wrong with upgrading to newer version?

There's nothing wrong with it. It's just more work and for
minor releases often for very little gain. So unless you
know why you are upgrading you can probably spend your
time more productively doing other things.

If you are a pro then it might also throw your development
environment out of sync with your target environment
which can result in subtle bugs. But I don't think
that's an issue for you yet.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Wed Jan  6 04:59:49 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 6 Jan 2016 09:59:49 +0000
Subject: [Tutor] method, type?
In-Reply-To: <333654cd965fbd66934c6597ac37932d@sonic.net>
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
Message-ID: <n6ioik$hq7$1@ger.gmane.org>

On 06/01/16 04:58, Alex Kleider wrote:

> class JournalLineItem(object):
>      def __init__(self, account, debit_or_credit, amount):
>          self.account = account
>          self.debit_or_credit = debit_or_credit
>          self.amount = float(amount)
> 
...
>      def get_line_item(text):
>          return JournalLineItem(*text.split())

> What kind of a method/function is get_line_item?

As it stands its not.
As its intended to be, its a named constructor.
But Python doesn't support named constructors so instead it
could be a class method - that is a method of the class
rather than of a particular instance.

But it should probably be a factory function. That is,
defined at the module level and not part of the class
at all. Not everything in OOP needs to be in a class.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Wed Jan  6 05:49:29 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 6 Jan 2016 21:49:29 +1100
Subject: [Tutor] method, type?
In-Reply-To: <333654cd965fbd66934c6597ac37932d@sonic.net>
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
Message-ID: <20160106104929.GM23700@ando.pearwood.info>

On Tue, Jan 05, 2016 at 08:58:42PM -0800, Alex Kleider wrote:

> class JournalLineItem(object):
>     def __init__(self, account, debit_or_credit, amount):
>         self.account = account
>         self.debit_or_credit = debit_or_credit
>         self.amount = float(amount)
>     def show(self):
>         return ("ACNT: {}  ${:0.2f} {}".
>             format(self.account, self.amount, self.debit_or_credit))
>     def get_line_item(text):
>         return JournalLineItem(*text.split())
> 
> def test():
>     print(
>     JournalLineItem.get_line_item("2435 Dr 25.33").show())


> What kind of a method/function is get_line_item?

In Python 3, it's a regular function. In Python 2, it's a broken method 
that won't work.

Some background information: methods and functions are different 
kinds of things, but methods are constructed from functions as 
needed. So when you define a method inside a class:

class Spam:
    def method(self, arg): ...

you're actually defining an ordinary function object. All the magic 
takes place when you go to use it. The rules changed slightly in Python 
3, so I'll start with Python 2.

Normal use is to create an instance, then call the method:

instance = Spam()
instance.method(x)

At this point, calling instance.method extracts the function object out 
of the class, converts it into a "bound method", and calls that method. 

("Bound" in this context only means that the method knows what instance 
it will get as self.)

If you extract the method from the class instead, you get an "unbound 
method" which means it doesn't have an instance applied to it, so you 
have to provide one yourself:

Spam.method(instance, x)


We can see that the class actually stores a regular function object, 
which is then automatically converted into methods as required:

py> Spam.__dict__['method']  
<function method at 0xb7c0f79c>
py> Spam().method
<bound method Spam.method of <__main__.Spam instance at 0xb7c0e22c>>
py> Spam.method
<unbound method Spam.method>

The magic that makes this work is called the Descriptor protocol, and it 
is responsible for all sorts of goodies, like properties, staticmethods, 
classmethods, and more.

So, that was the situation in Python 2. When Python 3 came about, people 
realised that there actually isn't anything special about unbound 
methods. They're just functions, and so in Python 3 extracting a method 
off the class just returns the regular function, with no changes made:

py> Spam.__dict__['method']
<function Spam.method at 0xb7ade14c>
py> Spam().method
<bound method Spam.method of <__main__.Spam object at 0xb7b5a86c>>
py> Spam.method
<function Spam.method at 0xb7ade14c>


Your JournalLineItem class takes advantage of that fact. When you go to 
use the get_line_item "method", you extract it from the class:

    JournalLineItem.get_line_item

which returns an ordinary function, as if it were defined outside of a 
class. You then provide an argument:

    JournalLineItem.get_line_item("2435 Dr 25.33")

which gets assigned to the parameter "text" and various things happen.

Although this is legal code, it should be avoided because:

(1) It is confusing and weird.

(2) If you try calling the method from an instance, bad things happen:

py> x = JournalLineItem("account", True, 100)
py> x.get_line_item("2435 Dr 25.33")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: get_line_item() takes 1 positional argument but 2 were given


This is how get_line_item should be written:

    @classmethod
    def get_line_item(cls, text):
        return cls(*text.split())

which also has the advantage that if you subclass JournalLineItem, the 
method will continue to work correctly.


> From what I've read (and not fully understood)
> static methods and class methods must have
> @staticmethod and @classmethod on the line above them.

In simple terms, correct.

To be technical, not quite, there are other, less convenient but 
sometimes useful, ways to create static and classmethods, but you don't 
need to worry about those if you don't want. (I'll explain if you ask.)


> get_line_item works as I wanted but it's clearly not the
> usual type of method and I don't know how to categorize it.
> It's an instance creator- is there a better term?
> Is this 'Pythonic' code?

The usual term for this is "alternate constructor", or just 
"constructor". It constructs an instance of the class, you see, but it 
is not the standard one, __init__.

(To be precise, __init__ is the initiator, __new__ is the actual 
constructor, but most of the time you don't write __new__, and it calls 
__init__ by default.)

Is it Pythonic? As written, hell no! It's a mess! (No offense intended.) 
It confused me, for a while, I really thought it wouldn't work at all 
and was somewhat surprised to see that it did actually work. (Once I saw 
that it worked, in hindsight it was obvious why it worked.)

But the principle is sound. For example, dicts have an alternate 
constructor method:

    dict.fromkeys

which creates a new dict from a collection of keys.

So I recommend you re-write the method to the version I suggested, and 
then you can happily use it secure in the knowledge that not only does 
it work, but it works in a Pythonic way.




-- 
Steve

From steve at pearwood.info  Wed Jan  6 09:46:08 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 7 Jan 2016 01:46:08 +1100
Subject: [Tutor] method, type?
In-Reply-To: <n6ioik$hq7$1@ger.gmane.org>
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
 <n6ioik$hq7$1@ger.gmane.org>
Message-ID: <20160106144608.GA10854@ando.pearwood.info>

On Wed, Jan 06, 2016 at 09:59:49AM +0000, Alan Gauld wrote:
> On 06/01/16 04:58, Alex Kleider wrote:
> 
> > class JournalLineItem(object):
> >      def __init__(self, account, debit_or_credit, amount):
> >          self.account = account
> >          self.debit_or_credit = debit_or_credit
> >          self.amount = float(amount)
> > 
> ...
> >      def get_line_item(text):
> >          return JournalLineItem(*text.split())
> 
> > What kind of a method/function is get_line_item?
> 
> As it stands its not.

Actually, it is a function, because `def` ALWAYS creates a function.


> As its intended to be, its a named constructor.
> But Python doesn't support named constructors so instead it
> could be a class method - that is a method of the class
> rather than of a particular instance.

I don't understand what you mean by "Python doesn't support named 
constructors". It seems to me that this is the *only* sort of 
constructor that Python supports.

As I understand it, "named constructor" comes from the C++ world, where 
functions are matched not just by name, but by parameters as well.

https://isocpp.org/wiki/faq/ctors#named-ctor-idiom


Constructors are named the same as the class by default. So, using 
Python syntax for simplicity, this is how you would define two 
constructors for a class called Spam, one that takes a single argument, 
and the other which takes two arguments:


class Spam:
    def Spam(x):
       # create a new Spam instance from one argument, x

    def Spam(x, y):
       # create a new Spam instance from two arguments, x and y


When you go to create an instance, you have a choice of saying:

x = Spam("arg")  # calls the one argument constructor
y = Spam(23, 99)  # calls the two argument constructor

and the C++ compiler will automatically choose the right constructor 
method.

But what if you need two constructors with the same number and type of 
arguments? For instance, you might have a Coordinate object, which takes 
two integers representing the position relative to the entire screen, or 
two integers representing the position relative to the current window. 
The C++ compiler cannot distinguish those two cases, and would give an 
error.

The solution is to use constructors with different names, which C++ 
calls "named constructors".

But note that Python DOES NOT support the standard name-of-class kind of 
constructor that C++ uses. In Python, we would write __new__ or 
__init__, not Spam, but that's just a cosmetic difference. The big 
difference is that we cannot write two methods with the same name and 
have Python keep them both. The second method would overwrite the first. 
So the only way to have two different constructors is to give them 
different names.

Hence all constructors in Python are "named constructors".


> But it should probably be a factory function. That is,
> defined at the module level and not part of the class
> at all. Not everything in OOP needs to be in a class.

It would be reasonable to make this a factory function declared in the 
global module level. But I think making it a classmethod is better. That 
keeps it encapsulated with the class, and ensures that if you subclass 
the class, you automatically get a valid constructor as well.



-- 
Steve

From katye2007 at gmail.com  Wed Jan  6 09:57:36 2016
From: katye2007 at gmail.com (yehudak .)
Date: Wed, 6 Jan 2016 16:57:36 +0200
Subject: [Tutor] Tutor Digest, Vol 143, Issue 15
In-Reply-To: <mailman.829.1452091581.2304.tutor@python.org>
References: <mailman.829.1452091581.2304.tutor@python.org>
Message-ID: <CAE3ie423A+M9ssTmrd7jN0p4qf9xwyO0NSQWcjW742cwJVe4Ug@mail.gmail.com>

Not a question, just ranting.

My 2 commonest coding errors are forgetting the : before a block, and
mismatch between the number of ('s and )'s. The error message is a simple
'Syntax Error', but the line quoted is not the offending line. I would love
a much more explicit 'syntax error'.

Anyone here in good relations with Guido van Rossum?

Yehuda

On Wed, Jan 6, 2016 at 4:46 PM, <tutor-request at python.org> wrote:

> Send Tutor mailing list submissions to
>         tutor at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.python.org/mailman/listinfo/tutor
> or, via email, send a message with subject or body 'help' to
>         tutor-request at python.org
>
> You can reach the person managing the list at
>         tutor-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."
>
>
> Today's Topics:
>
>    1. Thanks Alan. (yehudak .)
>    2. Re: Python 3.5.1 64 bit? (yehudak .)
>    3. Re: Python 3.5.1 64 bit? (Alan Gauld)
>    4. Re: method, type? (Alan Gauld)
>    5. Re: method, type? (Steven D'Aprano)
>    6. Re: method, type? (Steven D'Aprano)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 6 Jan 2016 09:13:52 +0200
> From: "yehudak ." <katye2007 at gmail.com>
> To: Wally Cow <alan.gauld at btinternet.com>, tutor at python.org
> Subject: [Tutor] Thanks Alan.
> Message-ID:
>         <
> CAE3ie42iGp+1AHK7kidykcHq-dvPWEDffUrOquKkVEk8B0nHEA at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> I'll look into it.
> Yehuda
>
>
> ------------------------------
>
> Message: 2
> Date: Wed, 6 Jan 2016 10:14:15 +0200
> From: "yehudak ." <katye2007 at gmail.com>
> To: eryk sun <eryksun at gmail.com>, Wally Cow
>         <alan.gauld at btinternet.com>
> Cc: tutor at python.org
> Subject: Re: [Tutor] Python 3.5.1 64 bit?
> Message-ID:
>         <CAE3ie40uN7GzxXryT4xZTRa9SinnuuDi5TbAehtrzjY5c==
> nDg at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Thanks eryk.
> I've downloaded and installed 3.5.1 64 bit, as Alan suggested.
> So far no problem. I can anytime return to 3.5 if it will be needed.
>
> Yehuda
>
> On Wed, Jan 6, 2016 at 10:01 AM, eryk sun <eryksun at gmail.com> wrote:
>
> > On Wed, Jan 6, 2016 at 1:06 AM, yehudak . <katye2007 at gmail.com> wrote:
> > > What's wrong with upgrading to newer version?
> >
> > It depends on how many systems and virtual environments that you're
> > upgrading. It shouldn't be an issue if it's just a new micro release
> > for your own development machine. If you come across a regression, you
> > can just downgrade to the previous version. That said, there's the old
> > adage that if something isn't broken (for your needs), then you
> > shouldn't 'fix' it.
> >
> > Installing a new minor release (e.g. 3.5 to 3.6) is more work, since
> > you have to reinstall all packages and rebuild extension modules for
> > source packages. But it's worth it to stay current with the evolution
> > of the language. On Windows, make sure to set the PY_PYTHON3
> > environment variable to the version you want as the default "python3".
> >
>
>
> ------------------------------
>
> Message: 3
> Date: Wed, 6 Jan 2016 09:52:00 +0000
> From: Alan Gauld <alan.gauld at btinternet.com>
> To: tutor at python.org
> Subject: Re: [Tutor] Python 3.5.1 64 bit?
> Message-ID: <n6io3v$amg$1 at ger.gmane.org>
> Content-Type: text/plain; charset=utf-8
>
> On 06/01/16 07:06, yehudak . wrote:
> > eryk,
> > Thank you.
> > What's wrong with upgrading to newer version?
>
> There's nothing wrong with it. It's just more work and for
> minor releases often for very little gain. So unless you
> know why you are upgrading you can probably spend your
> time more productively doing other things.
>
> If you are a pro then it might also throw your development
> environment out of sync with your target environment
> which can result in subtle bugs. But I don't think
> that's an issue for you yet.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
> ------------------------------
>
> Message: 4
> Date: Wed, 6 Jan 2016 09:59:49 +0000
> From: Alan Gauld <alan.gauld at btinternet.com>
> To: tutor at python.org
> Subject: Re: [Tutor] method, type?
> Message-ID: <n6ioik$hq7$1 at ger.gmane.org>
> Content-Type: text/plain; charset=utf-8
>
> On 06/01/16 04:58, Alex Kleider wrote:
>
> > class JournalLineItem(object):
> >      def __init__(self, account, debit_or_credit, amount):
> >          self.account = account
> >          self.debit_or_credit = debit_or_credit
> >          self.amount = float(amount)
> >
> ...
> >      def get_line_item(text):
> >          return JournalLineItem(*text.split())
>
> > What kind of a method/function is get_line_item?
>
> As it stands its not.
> As its intended to be, its a named constructor.
> But Python doesn't support named constructors so instead it
> could be a class method - that is a method of the class
> rather than of a particular instance.
>
> But it should probably be a factory function. That is,
> defined at the module level and not part of the class
> at all. Not everything in OOP needs to be in a class.
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
> ------------------------------
>
> Message: 5
> Date: Wed, 6 Jan 2016 21:49:29 +1100
> From: Steven D'Aprano <steve at pearwood.info>
> To: tutor at python.org
> Subject: Re: [Tutor] method, type?
> Message-ID: <20160106104929.GM23700 at ando.pearwood.info>
> Content-Type: text/plain; charset=us-ascii
>
> On Tue, Jan 05, 2016 at 08:58:42PM -0800, Alex Kleider wrote:
>
> > class JournalLineItem(object):
> >     def __init__(self, account, debit_or_credit, amount):
> >         self.account = account
> >         self.debit_or_credit = debit_or_credit
> >         self.amount = float(amount)
> >     def show(self):
> >         return ("ACNT: {}  ${:0.2f} {}".
> >             format(self.account, self.amount, self.debit_or_credit))
> >     def get_line_item(text):
> >         return JournalLineItem(*text.split())
> >
> > def test():
> >     print(
> >     JournalLineItem.get_line_item("2435 Dr 25.33").show())
>
>
> > What kind of a method/function is get_line_item?
>
> In Python 3, it's a regular function. In Python 2, it's a broken method
> that won't work.
>
> Some background information: methods and functions are different
> kinds of things, but methods are constructed from functions as
> needed. So when you define a method inside a class:
>
> class Spam:
>     def method(self, arg): ...
>
> you're actually defining an ordinary function object. All the magic
> takes place when you go to use it. The rules changed slightly in Python
> 3, so I'll start with Python 2.
>
> Normal use is to create an instance, then call the method:
>
> instance = Spam()
> instance.method(x)
>
> At this point, calling instance.method extracts the function object out
> of the class, converts it into a "bound method", and calls that method.
>
> ("Bound" in this context only means that the method knows what instance
> it will get as self.)
>
> If you extract the method from the class instead, you get an "unbound
> method" which means it doesn't have an instance applied to it, so you
> have to provide one yourself:
>
> Spam.method(instance, x)
>
>
> We can see that the class actually stores a regular function object,
> which is then automatically converted into methods as required:
>
> py> Spam.__dict__['method']
> <function method at 0xb7c0f79c>
> py> Spam().method
> <bound method Spam.method of <__main__.Spam instance at 0xb7c0e22c>>
> py> Spam.method
> <unbound method Spam.method>
>
> The magic that makes this work is called the Descriptor protocol, and it
> is responsible for all sorts of goodies, like properties, staticmethods,
> classmethods, and more.
>
> So, that was the situation in Python 2. When Python 3 came about, people
> realised that there actually isn't anything special about unbound
> methods. They're just functions, and so in Python 3 extracting a method
> off the class just returns the regular function, with no changes made:
>
> py> Spam.__dict__['method']
> <function Spam.method at 0xb7ade14c>
> py> Spam().method
> <bound method Spam.method of <__main__.Spam object at 0xb7b5a86c>>
> py> Spam.method
> <function Spam.method at 0xb7ade14c>
>
>
> Your JournalLineItem class takes advantage of that fact. When you go to
> use the get_line_item "method", you extract it from the class:
>
>     JournalLineItem.get_line_item
>
> which returns an ordinary function, as if it were defined outside of a
> class. You then provide an argument:
>
>     JournalLineItem.get_line_item("2435 Dr 25.33")
>
> which gets assigned to the parameter "text" and various things happen.
>
> Although this is legal code, it should be avoided because:
>
> (1) It is confusing and weird.
>
> (2) If you try calling the method from an instance, bad things happen:
>
> py> x = JournalLineItem("account", True, 100)
> py> x.get_line_item("2435 Dr 25.33")
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: get_line_item() takes 1 positional argument but 2 were given
>
>
> This is how get_line_item should be written:
>
>     @classmethod
>     def get_line_item(cls, text):
>         return cls(*text.split())
>
> which also has the advantage that if you subclass JournalLineItem, the
> method will continue to work correctly.
>
>
> > From what I've read (and not fully understood)
> > static methods and class methods must have
> > @staticmethod and @classmethod on the line above them.
>
> In simple terms, correct.
>
> To be technical, not quite, there are other, less convenient but
> sometimes useful, ways to create static and classmethods, but you don't
> need to worry about those if you don't want. (I'll explain if you ask.)
>
>
> > get_line_item works as I wanted but it's clearly not the
> > usual type of method and I don't know how to categorize it.
> > It's an instance creator- is there a better term?
> > Is this 'Pythonic' code?
>
> The usual term for this is "alternate constructor", or just
> "constructor". It constructs an instance of the class, you see, but it
> is not the standard one, __init__.
>
> (To be precise, __init__ is the initiator, __new__ is the actual
> constructor, but most of the time you don't write __new__, and it calls
> __init__ by default.)
>
> Is it Pythonic? As written, hell no! It's a mess! (No offense intended.)
> It confused me, for a while, I really thought it wouldn't work at all
> and was somewhat surprised to see that it did actually work. (Once I saw
> that it worked, in hindsight it was obvious why it worked.)
>
> But the principle is sound. For example, dicts have an alternate
> constructor method:
>
>     dict.fromkeys
>
> which creates a new dict from a collection of keys.
>
> So I recommend you re-write the method to the version I suggested, and
> then you can happily use it secure in the knowledge that not only does
> it work, but it works in a Pythonic way.
>
>
>
>
> --
> Steve
>
>
> ------------------------------
>
> Message: 6
> Date: Thu, 7 Jan 2016 01:46:08 +1100
> From: Steven D'Aprano <steve at pearwood.info>
> To: tutor at python.org
> Subject: Re: [Tutor] method, type?
> Message-ID: <20160106144608.GA10854 at ando.pearwood.info>
> Content-Type: text/plain; charset=us-ascii
>
> On Wed, Jan 06, 2016 at 09:59:49AM +0000, Alan Gauld wrote:
> > On 06/01/16 04:58, Alex Kleider wrote:
> >
> > > class JournalLineItem(object):
> > >      def __init__(self, account, debit_or_credit, amount):
> > >          self.account = account
> > >          self.debit_or_credit = debit_or_credit
> > >          self.amount = float(amount)
> > >
> > ...
> > >      def get_line_item(text):
> > >          return JournalLineItem(*text.split())
> >
> > > What kind of a method/function is get_line_item?
> >
> > As it stands its not.
>
> Actually, it is a function, because `def` ALWAYS creates a function.
>
>
> > As its intended to be, its a named constructor.
> > But Python doesn't support named constructors so instead it
> > could be a class method - that is a method of the class
> > rather than of a particular instance.
>
> I don't understand what you mean by "Python doesn't support named
> constructors". It seems to me that this is the *only* sort of
> constructor that Python supports.
>
> As I understand it, "named constructor" comes from the C++ world, where
> functions are matched not just by name, but by parameters as well.
>
> https://isocpp.org/wiki/faq/ctors#named-ctor-idiom
>
>
> Constructors are named the same as the class by default. So, using
> Python syntax for simplicity, this is how you would define two
> constructors for a class called Spam, one that takes a single argument,
> and the other which takes two arguments:
>
>
> class Spam:
>     def Spam(x):
>        # create a new Spam instance from one argument, x
>
>     def Spam(x, y):
>        # create a new Spam instance from two arguments, x and y
>
>
> When you go to create an instance, you have a choice of saying:
>
> x = Spam("arg")  # calls the one argument constructor
> y = Spam(23, 99)  # calls the two argument constructor
>
> and the C++ compiler will automatically choose the right constructor
> method.
>
> But what if you need two constructors with the same number and type of
> arguments? For instance, you might have a Coordinate object, which takes
> two integers representing the position relative to the entire screen, or
> two integers representing the position relative to the current window.
> The C++ compiler cannot distinguish those two cases, and would give an
> error.
>
> The solution is to use constructors with different names, which C++
> calls "named constructors".
>
> But note that Python DOES NOT support the standard name-of-class kind of
> constructor that C++ uses. In Python, we would write __new__ or
> __init__, not Spam, but that's just a cosmetic difference. The big
> difference is that we cannot write two methods with the same name and
> have Python keep them both. The second method would overwrite the first.
> So the only way to have two different constructors is to give them
> different names.
>
> Hence all constructors in Python are "named constructors".
>
>
> > But it should probably be a factory function. That is,
> > defined at the module level and not part of the class
> > at all. Not everything in OOP needs to be in a class.
>
> It would be reasonable to make this a factory function declared in the
> global module level. But I think making it a classmethod is better. That
> keeps it encapsulated with the class, and ensures that if you subclass
> the class, you automatically get a valid constructor as well.
>
>
>
> --
> Steve
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> https://mail.python.org/mailman/listinfo/tutor
>
>
> ------------------------------
>
> End of Tutor Digest, Vol 143, Issue 15
> **************************************
>

From joel.goldstick at gmail.com  Wed Jan  6 10:44:58 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Wed, 6 Jan 2016 10:44:58 -0500
Subject: [Tutor] Tutor Digest, Vol 143, Issue 15
In-Reply-To: <CAE3ie423A+M9ssTmrd7jN0p4qf9xwyO0NSQWcjW742cwJVe4Ug@mail.gmail.com>
References: <mailman.829.1452091581.2304.tutor@python.org>
 <CAE3ie423A+M9ssTmrd7jN0p4qf9xwyO0NSQWcjW742cwJVe4Ug@mail.gmail.com>
Message-ID: <CAPM-O+xDrh5xLTv=rrrm5XvU3FXzA0+KfFFPyViAUqaQ-zmhkg@mail.gmail.com>

On Wed, Jan 6, 2016 at 9:57 AM, yehudak . <katye2007 at gmail.com> wrote:

> Not a question, just ranting.
>
> My 2 commonest coding errors are forgetting the : before a block, and
> mismatch between the number of ('s and )'s. The error message is a simple
> 'Syntax Error', but the line quoted is not the offending line. I would love
> a much more explicit 'syntax error'.
>
> Anyone here in good relations with Guido van Rossum?
>
> Yehuda
>
> I'll see your rant and raise you one more:

Please don't respond to a journal posting without deleting everything
except what is relevant to your question.

The reason you don't get the error line correctly, is that the parser
doesn't know it is an error until it later encounters something confusing.
When you don't see the error on the line that was given, look up a line or
two.

From katye2007 at gmail.com  Wed Jan  6 13:23:59 2016
From: katye2007 at gmail.com (yehudak .)
Date: Wed, 6 Jan 2016 20:23:59 +0200
Subject: [Tutor] Teaching Python
Message-ID: <CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw@mail.gmail.com>

My grandson Guy (8th grader) is learning Python at school. That's what made
me teach myself Python programming as well.
Yesterday he asked my help in his homework:

Write a FOR-loop that prints all numbers up to 1000000

Thank you, the gods of Ctrl+C

Yehuda

From alan.gauld at btinternet.com  Wed Jan  6 13:57:31 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 6 Jan 2016 18:57:31 +0000
Subject: [Tutor] method, type?
In-Reply-To: <20160106144608.GA10854@ando.pearwood.info>
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
 <n6ioik$hq7$1@ger.gmane.org> <20160106144608.GA10854@ando.pearwood.info>
Message-ID: <n6jo2r$9op$1@ger.gmane.org>

Second attempt, my PC crashed just as I was sending the first
one. You might see two similar posts, if so apologies...

On 06/01/16 14:46, Steven D'Aprano wrote:
> I don't understand what you mean by "Python doesn't support named 
> constructors". It seems to me that this is the *only* sort of 
> constructor that Python supports.

No, Python constructors have names(new/init) but they are not
used explicitly (ie. by the client code)to create objects. By
contrast languages like Delphi, Smalltalk, Objective C,
some Lisps and, I think, Eiffel (and others?) all require an
explicit call of a constructor method and there may be
multiple constructors per class each with different names
(usually describing how the construction occurs or the
nature of the object constructed).

For example in Smalltalk:

obj := MyClass new.  # new is the normal choice of name
obj2 := Date today.
obj3 := Date new.
obj4 := Time now.

All of these methods (new, today, now) are constructors(*) but
with explicit names that tells you something about the kind
of instance created.

(*)As I understand it, Smalltalk constructors are just class
methods allocated to a constructor category. But the category
is just an organisational device used by the IDE/library
it doesn't actually change the code in any way.

> As I understand it, "named constructor" comes from the C++ world, where 
> functions are matched not just by name, but by parameters as well.

That's overloading which uses the same name but with different
numbers/types of parameters and/or return type. The problem with
that is that in C++ constructors must have the name of the class.
So if you want multiple constructors that all take a single
string as their paramater then it gets tricky.

> two integers representing the position relative to the entire screen, or 
> two integers representing the position relative to the current window. 
> The C++ compiler cannot distinguish those two cases, and would give an 
> error.

Exactly so.

> The solution is to use constructors with different names, which C++ 
> calls "named constructors".

I stopped using C++ around v2 and it didn't have such a feature.
Maybe its been added since. If so that's good to know. (Time
for some googling methinks...)

> So the only way to have two different constructors is to give them 
> different names.
> 
> Hence all constructors in Python are "named constructors".

But they are not native constructors such as those used in
Smalltalk etc. They have to be factory methods that call new/init
under the covers. And that's how most OOP languages that don't
support named constructors get round it. (Although interestingly,
Objective C refers to all constructors as 'factory methods', even
the 'default' ones.)

> It would be reasonable to make this a factory function declared in the 
> global module level. But I think making it a classmethod is better. 

I won't put up much of an argument here. In C++ or Java I'd
definitely say use a class method. But it seems to me that I've
seen more factory functions in modules than class method factories.
But that's not been a detailed analysis just a gut feel for what
seems idiomatic in Python. It also seems easier for beginners
to grasp than the more esoteric notion of class methods.

> ... ensures that if you subclass the class, you automatically
> get a valid constructor as well.

Wouldn't it return an instance of the superclass rather than
the sub class? You'd need to override it wouldn't you?


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Wed Jan  6 14:04:24 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 6 Jan 2016 19:04:24 +0000
Subject: [Tutor] Teaching Python
In-Reply-To: <CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw@mail.gmail.com>
References: <CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw@mail.gmail.com>
Message-ID: <n6jofo$g4a$1@ger.gmane.org>

On 06/01/16 18:23, yehudak . wrote:
> My grandson Guy (8th grader) is learning Python at school. That's what made
> me teach myself Python programming as well.
> Yesterday he asked my help in his homework:
> 
> Write a FOR-loop that prints all numbers up to 1000000

And the question is?
Python has a for loop and a print function.
It can easily handle numbers up to 1000000.
You probably need to look at the range() function too.

If you are having difficulty with any of those concepts
let us know and ideally show us the code that fails.

Alternatively if you are asking about what we think of
the homework question itself, then I'd say it seems
reasonable for a beginner. The only thing I'd gripe
about is the use of uppercase FOR.  That's because
Python is case sensitive and its for loop is
lowercase so the FOR could confuse some students.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From __peter__ at web.de  Wed Jan  6 15:40:26 2016
From: __peter__ at web.de (Peter Otten)
Date: Wed, 06 Jan 2016 21:40:26 +0100
Subject: [Tutor] Teaching Python
References: <CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw@mail.gmail.com>
Message-ID: <n6ju3r$e38$1@ger.gmane.org>

yehudak . wrote:

> My grandson Guy (8th grader) is learning Python at school. That's what
> made me teach myself Python programming as well.
> Yesterday he asked my help in his homework:
> 
> Write a FOR-loop that prints all numbers up to 1000000
> 
> Thank you, the gods of Ctrl+C

Did you mean to imply that it took too long?

It is likely that the terminal (emulation) is to blame, and not the python 
interpreter:

$ time python3 -c 'for i in range(10**6): print(i)' > /dev/null

real    0m1.562s
user    0m1.525s
sys     0m0.033s

Under 2 seconds on quite old hardware. Without redirection in Konsole (KDE's 
terminal emulator):

$ time python3 -c 'for i in range(10**6): print(i)'
[...]
999997
999998
999999

real    0m16.167s
user    0m5.258s
sys     0m5.512s

But wait -- you ran the code in IDLE, right?
Then book it under lessons learned ;)


From dyoo at hashcollision.org  Wed Jan  6 15:42:52 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Wed, 6 Jan 2016 12:42:52 -0800
Subject: [Tutor] Teaching Python
In-Reply-To: <CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw@mail.gmail.com>
References: <CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw@mail.gmail.com>
Message-ID: <CAGZAPF5yHAqqXsDDXg3OtKtuV=EUqtDo_rhOv=7g45=V_h2ZMg@mail.gmail.com>

On Wed, Jan 6, 2016 at 10:23 AM, yehudak . <katye2007 at gmail.com> wrote:
> My grandson Guy (8th grader) is learning Python at school. That's what
made
> me teach myself Python programming as well.
> Yesterday he asked my help in his homework:
>
> Write a FOR-loop that prints all numbers up to 1000000

Try to say explicitly where you think you're getting stuck. There are a few
general problem-solving strategies that you can try to apply, in the spirit
of Polya's How To Solve It:
http://www.amazon.com/How-Solve-It-Mathematical-Princeton/dp/069111966X.

* Do you understand the question being asked? Do you understand all the
technical "terms" that the question is using? This is actually a really
important one: if you don't understand the terms, there's very little hope
to get anything useful out of the experience.

- Do you know what a for loop is? Have you seen any examples of for loops?
- Do you know what it means to print a number? Have you seen any examples
of this?

* Do you know what the expected output is supposed to look like?

* Have you tackled problems that are similar to this one before?

* Do you know how to solve a simpler version of the problem? Say, from
changing "1000000" to "10"?

From tahir.hafiz at gmail.com  Wed Jan  6 15:59:34 2016
From: tahir.hafiz at gmail.com (Tahir Hafiz)
Date: Wed, 6 Jan 2016 20:59:34 +0000
Subject: [Tutor] Teaching Python
In-Reply-To: <CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw@mail.gmail.com>
References: <CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw@mail.gmail.com>
Message-ID: <CALmb6fvk+hd7ShHts2Qf6zOP1RzNwQo4zU6nVs390Z1ze=jSeA@mail.gmail.com>

Yes that is a huge for loop.
I can do it like this but I won't try:
>>> for num in list(range(1000000)):
>>>   print(num)

Sort of strange that the school wanted so much looping for your grandson,
they could have picked a lower number to demonstrate a for loop.

By the way there is the tqdm module which will show the time of a for loop
in a nice way:
https://github.com/noamraph/tqdm


Cheers,
Tahir







On Wed, Jan 6, 2016 at 6:23 PM, yehudak . <katye2007 at gmail.com> wrote:

> My grandson Guy (8th grader) is learning Python at school. That's what made
> me teach myself Python programming as well.
> Yesterday he asked my help in his homework:
>
> Write a FOR-loop that prints all numbers up to 1000000
>
> Thank you, the gods of Ctrl+C
>
> Yehuda
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From katye2007 at gmail.com  Wed Jan  6 16:04:38 2016
From: katye2007 at gmail.com (yehudak .)
Date: Wed, 6 Jan 2016 23:04:38 +0200
Subject: [Tutor] Teaching Python
In-Reply-To: <CALmb6fvk+hd7ShHts2Qf6zOP1RzNwQo4zU6nVs390Z1ze=jSeA@mail.gmail.com>
References: <CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw@mail.gmail.com>
 <CALmb6fvk+hd7ShHts2Qf6zOP1RzNwQo4zU6nVs390Z1ze=jSeA@mail.gmail.com>
Message-ID: <CAE3ie40wwA7yUY_dS20+iqKv35q0QCb7npKPG1VY5OrSiTncGw@mail.gmail.com>

Ahlan Tahir, That's EXACTLY my point.

Thank you,
Yehuda

On Wed, Jan 6, 2016 at 10:59 PM, Tahir Hafiz <tahir.hafiz at gmail.com> wrote:

> Yes that is a huge for loop.
> I can do it like this but I won't try:
> >>> for num in list(range(1000000)):
> >>>   print(num)
>
> Sort of strange that the school wanted so much looping for your grandson,
> they could have picked a lower number to demonstrate a for loop.
>
> By the way there is the tqdm module which will show the time of a for loop
> in a nice way:
> https://github.com/noamraph/tqdm
>
>
> Cheers,
> Tahir
>
>
>
>
>
>
>
> On Wed, Jan 6, 2016 at 6:23 PM, yehudak . <katye2007 at gmail.com> wrote:
>
>> My grandson Guy (8th grader) is learning Python at school. That's what
>> made
>> me teach myself Python programming as well.
>> Yesterday he asked my help in his homework:
>>
>> Write a FOR-loop that prints all numbers up to 1000000
>>
>> Thank you, the gods of Ctrl+C
>>
>> Yehuda
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
>

From katye2007 at gmail.com  Wed Jan  6 16:07:52 2016
From: katye2007 at gmail.com (yehudak .)
Date: Wed, 6 Jan 2016 23:07:52 +0200
Subject: [Tutor] Tutor Digest, Vol 143, Issue 17
In-Reply-To: <mailman.968.1452113982.2304.tutor@python.org>
References: <mailman.968.1452113982.2304.tutor@python.org>
Message-ID: <CAE3ie430VeD+1W7r=Zj8wT7ndJ50HxSi9gDJJAzgeuqLQ752EA@mail.gmail.com>

Alan, Tahir & friends,
My post was not a question, merely an observation.
I hope there's no objection to my mail.

Yehuda

On Wed, Jan 6, 2016 at 10:59 PM, <tutor-request at python.org> wrote:

> Send Tutor mailing list submissions to
>         tutor at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.python.org/mailman/listinfo/tutor
> or, via email, send a message with subject or body 'help' to
>         tutor-request at python.org
>
> You can reach the person managing the list at
>         tutor-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."
>
>
> Today's Topics:
>
>    1. Teaching Python (yehudak .)
>    2. Re: method, type? (Alan Gauld)
>    3. Re: Teaching Python (Alan Gauld)
>    4. Re: Teaching Python (Peter Otten)
>    5. Re: Teaching Python (Danny Yoo)
>    6. Re: Teaching Python (Tahir Hafiz)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 6 Jan 2016 20:23:59 +0200
> From: "yehudak ." <katye2007 at gmail.com>
> To: Tutor at python.org
> Subject: [Tutor] Teaching Python
> Message-ID:
>         <
> CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> My grandson Guy (8th grader) is learning Python at school. That's what made
> me teach myself Python programming as well.
> Yesterday he asked my help in his homework:
>
> Write a FOR-loop that prints all numbers up to 1000000
>
> Thank you, the gods of Ctrl+C
>
> Yehuda
>
>
> ------------------------------
>
> Message: 2
> Date: Wed, 6 Jan 2016 18:57:31 +0000
> From: Alan Gauld <alan.gauld at btinternet.com>
> To: tutor at python.org
> Subject: Re: [Tutor] method, type?
> Message-ID: <n6jo2r$9op$1 at ger.gmane.org>
> Content-Type: text/plain; charset=utf-8
>
> Second attempt, my PC crashed just as I was sending the first
> one. You might see two similar posts, if so apologies...
>
> On 06/01/16 14:46, Steven D'Aprano wrote:
> > I don't understand what you mean by "Python doesn't support named
> > constructors". It seems to me that this is the *only* sort of
> > constructor that Python supports.
>
> No, Python constructors have names(new/init) but they are not
> used explicitly (ie. by the client code)to create objects. By
> contrast languages like Delphi, Smalltalk, Objective C,
> some Lisps and, I think, Eiffel (and others?) all require an
> explicit call of a constructor method and there may be
> multiple constructors per class each with different names
> (usually describing how the construction occurs or the
> nature of the object constructed).
>
> For example in Smalltalk:
>
> obj := MyClass new.  # new is the normal choice of name
> obj2 := Date today.
> obj3 := Date new.
> obj4 := Time now.
>
> All of these methods (new, today, now) are constructors(*) but
> with explicit names that tells you something about the kind
> of instance created.
>
> (*)As I understand it, Smalltalk constructors are just class
> methods allocated to a constructor category. But the category
> is just an organisational device used by the IDE/library
> it doesn't actually change the code in any way.
>
> > As I understand it, "named constructor" comes from the C++ world, where
> > functions are matched not just by name, but by parameters as well.
>
> That's overloading which uses the same name but with different
> numbers/types of parameters and/or return type. The problem with
> that is that in C++ constructors must have the name of the class.
> So if you want multiple constructors that all take a single
> string as their paramater then it gets tricky.
>
> > two integers representing the position relative to the entire screen, or
> > two integers representing the position relative to the current window.
> > The C++ compiler cannot distinguish those two cases, and would give an
> > error.
>
> Exactly so.
>
> > The solution is to use constructors with different names, which C++
> > calls "named constructors".
>
> I stopped using C++ around v2 and it didn't have such a feature.
> Maybe its been added since. If so that's good to know. (Time
> for some googling methinks...)
>
> > So the only way to have two different constructors is to give them
> > different names.
> >
> > Hence all constructors in Python are "named constructors".
>
> But they are not native constructors such as those used in
> Smalltalk etc. They have to be factory methods that call new/init
> under the covers. And that's how most OOP languages that don't
> support named constructors get round it. (Although interestingly,
> Objective C refers to all constructors as 'factory methods', even
> the 'default' ones.)
>
> > It would be reasonable to make this a factory function declared in the
> > global module level. But I think making it a classmethod is better.
>
> I won't put up much of an argument here. In C++ or Java I'd
> definitely say use a class method. But it seems to me that I've
> seen more factory functions in modules than class method factories.
> But that's not been a detailed analysis just a gut feel for what
> seems idiomatic in Python. It also seems easier for beginners
> to grasp than the more esoteric notion of class methods.
>
> > ... ensures that if you subclass the class, you automatically
> > get a valid constructor as well.
>
> Wouldn't it return an instance of the superclass rather than
> the sub class? You'd need to override it wouldn't you?
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
> ------------------------------
>
> Message: 3
> Date: Wed, 6 Jan 2016 19:04:24 +0000
> From: Alan Gauld <alan.gauld at btinternet.com>
> To: tutor at python.org
> Subject: Re: [Tutor] Teaching Python
> Message-ID: <n6jofo$g4a$1 at ger.gmane.org>
> Content-Type: text/plain; charset=utf-8
>
> On 06/01/16 18:23, yehudak . wrote:
> > My grandson Guy (8th grader) is learning Python at school. That's what
> made
> > me teach myself Python programming as well.
> > Yesterday he asked my help in his homework:
> >
> > Write a FOR-loop that prints all numbers up to 1000000
>
> And the question is?
> Python has a for loop and a print function.
> It can easily handle numbers up to 1000000.
> You probably need to look at the range() function too.
>
> If you are having difficulty with any of those concepts
> let us know and ideally show us the code that fails.
>
> Alternatively if you are asking about what we think of
> the homework question itself, then I'd say it seems
> reasonable for a beginner. The only thing I'd gripe
> about is the use of uppercase FOR.  That's because
> Python is case sensitive and its for loop is
> lowercase so the FOR could confuse some students.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
> ------------------------------
>
> Message: 4
> Date: Wed, 06 Jan 2016 21:40:26 +0100
> From: Peter Otten <__peter__ at web.de>
> To: tutor at python.org
> Subject: Re: [Tutor] Teaching Python
> Message-ID: <n6ju3r$e38$1 at ger.gmane.org>
> Content-Type: text/plain; charset="ISO-8859-1"
>
> yehudak . wrote:
>
> > My grandson Guy (8th grader) is learning Python at school. That's what
> > made me teach myself Python programming as well.
> > Yesterday he asked my help in his homework:
> >
> > Write a FOR-loop that prints all numbers up to 1000000
> >
> > Thank you, the gods of Ctrl+C
>
> Did you mean to imply that it took too long?
>
> It is likely that the terminal (emulation) is to blame, and not the python
> interpreter:
>
> $ time python3 -c 'for i in range(10**6): print(i)' > /dev/null
>
> real    0m1.562s
> user    0m1.525s
> sys     0m0.033s
>
> Under 2 seconds on quite old hardware. Without redirection in Konsole
> (KDE's
> terminal emulator):
>
> $ time python3 -c 'for i in range(10**6): print(i)'
> [...]
> 999997
> 999998
> 999999
>
> real    0m16.167s
> user    0m5.258s
> sys     0m5.512s
>
> But wait -- you ran the code in IDLE, right?
> Then book it under lessons learned ;)
>
>
>
> ------------------------------
>
> Message: 5
> Date: Wed, 6 Jan 2016 12:42:52 -0800
> From: Danny Yoo <dyoo at hashcollision.org>
> To: "yehudak ." <katye2007 at gmail.com>
> Cc: Python Tutor Mailing List <Tutor at python.org>
> Subject: Re: [Tutor] Teaching Python
> Message-ID:
>         <CAGZAPF5yHAqqXsDDXg3OtKtuV=EUqtDo_rhOv=7g45=
> V_h2ZMg at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> On Wed, Jan 6, 2016 at 10:23 AM, yehudak . <katye2007 at gmail.com> wrote:
> > My grandson Guy (8th grader) is learning Python at school. That's what
> made
> > me teach myself Python programming as well.
> > Yesterday he asked my help in his homework:
> >
> > Write a FOR-loop that prints all numbers up to 1000000
>
> Try to say explicitly where you think you're getting stuck. There are a few
> general problem-solving strategies that you can try to apply, in the spirit
> of Polya's How To Solve It:
> http://www.amazon.com/How-Solve-It-Mathematical-Princeton/dp/069111966X.
>
> * Do you understand the question being asked? Do you understand all the
> technical "terms" that the question is using? This is actually a really
> important one: if you don't understand the terms, there's very little hope
> to get anything useful out of the experience.
>
> - Do you know what a for loop is? Have you seen any examples of for loops?
> - Do you know what it means to print a number? Have you seen any examples
> of this?
>
> * Do you know what the expected output is supposed to look like?
>
> * Have you tackled problems that are similar to this one before?
>
> * Do you know how to solve a simpler version of the problem? Say, from
> changing "1000000" to "10"?
>
>
> ------------------------------
>
> Message: 6
> Date: Wed, 6 Jan 2016 20:59:34 +0000
> From: Tahir Hafiz <tahir.hafiz at gmail.com>
> To: "yehudak ." <katye2007 at gmail.com>
> Cc: tutor <Tutor at python.org>
> Subject: Re: [Tutor] Teaching Python
> Message-ID:
>         <CALmb6fvk+hd7ShHts2Qf6zOP1RzNwQo4zU6nVs390Z1ze=
> jSeA at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Yes that is a huge for loop.
> I can do it like this but I won't try:
> >>> for num in list(range(1000000)):
> >>>   print(num)
>
> Sort of strange that the school wanted so much looping for your grandson,
> they could have picked a lower number to demonstrate a for loop.
>
> By the way there is the tqdm module which will show the time of a for loop
> in a nice way:
> https://github.com/noamraph/tqdm
>
>
> Cheers,
> Tahir
>
>
>
>
>
>
>
> On Wed, Jan 6, 2016 at 6:23 PM, yehudak . <katye2007 at gmail.com> wrote:
>
> > My grandson Guy (8th grader) is learning Python at school. That's what
> made
> > me teach myself Python programming as well.
> > Yesterday he asked my help in his homework:
> >
> > Write a FOR-loop that prints all numbers up to 1000000
> >
> > Thank you, the gods of Ctrl+C
> >
> > Yehuda
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > https://mail.python.org/mailman/listinfo/tutor
> >
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> https://mail.python.org/mailman/listinfo/tutor
>
>
> ------------------------------
>
> End of Tutor Digest, Vol 143, Issue 17
> **************************************
>

From tahir.hafiz at gmail.com  Wed Jan  6 16:15:26 2016
From: tahir.hafiz at gmail.com (Tahir Hafiz)
Date: Wed, 6 Jan 2016 21:15:26 +0000
Subject: [Tutor] Tutor Digest, Vol 143, Issue 17
In-Reply-To: <CAE3ie430VeD+1W7r=Zj8wT7ndJ50HxSi9gDJJAzgeuqLQ752EA@mail.gmail.com>
References: <mailman.968.1452113982.2304.tutor@python.org>
 <CAE3ie430VeD+1W7r=Zj8wT7ndJ50HxSi9gDJJAzgeuqLQ752EA@mail.gmail.com>
Message-ID: <CALmb6fsSMQeB=Br7znAvuaoqQ8iU3J4QY+pDz26Qpi2yO7MwOA@mail.gmail.com>

On Wed, Jan 6, 2016 at 9:07 PM, yehudak . <katye2007 at gmail.com> wrote:

> Alan, Tahir & friends,
> My post was not a question, merely an observation.
> I hope there's no objection to my mail.
>
> Yehuda
>
> On Wed, Jan 6, 2016 at 10:59 PM, <tutor-request at python.org> wrote:
>
> > Send Tutor mailing list submissions to
> >         tutor at python.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >         https://mail.python.org/mailman/listinfo/tutor
> > or, via email, send a message with subject or body 'help' to
> >         tutor-request at python.org
> >
> > You can reach the person managing the list at
> >         tutor-owner at python.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of Tutor digest..."
> >
> >
> > Today's Topics:
> >
> >    1. Teaching Python (yehudak .)
> >    2. Re: method, type? (Alan Gauld)
> >    3. Re: Teaching Python (Alan Gauld)
> >    4. Re: Teaching Python (Peter Otten)
> >    5. Re: Teaching Python (Danny Yoo)
> >    6. Re: Teaching Python (Tahir Hafiz)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Wed, 6 Jan 2016 20:23:59 +0200
> > From: "yehudak ." <katye2007 at gmail.com>
> > To: Tutor at python.org
> > Subject: [Tutor] Teaching Python
> > Message-ID:
> >         <
> > CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw at mail.gmail.com>
> > Content-Type: text/plain; charset=UTF-8
> >
> > My grandson Guy (8th grader) is learning Python at school. That's what
> made
> > me teach myself Python programming as well.
> > Yesterday he asked my help in his homework:
> >
> > Write a FOR-loop that prints all numbers up to 1000000
> >
> > Thank you, the gods of Ctrl+C
> >
> > Yehuda
> >
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Wed, 6 Jan 2016 18:57:31 +0000
> > From: Alan Gauld <alan.gauld at btinternet.com>
> > To: tutor at python.org
> > Subject: Re: [Tutor] method, type?
> > Message-ID: <n6jo2r$9op$1 at ger.gmane.org>
> > Content-Type: text/plain; charset=utf-8
> >
> > Second attempt, my PC crashed just as I was sending the first
> > one. You might see two similar posts, if so apologies...
> >
> > On 06/01/16 14:46, Steven D'Aprano wrote:
> > > I don't understand what you mean by "Python doesn't support named
> > > constructors". It seems to me that this is the *only* sort of
> > > constructor that Python supports.
> >
> > No, Python constructors have names(new/init) but they are not
> > used explicitly (ie. by the client code)to create objects. By
> > contrast languages like Delphi, Smalltalk, Objective C,
> > some Lisps and, I think, Eiffel (and others?) all require an
> > explicit call of a constructor method and there may be
> > multiple constructors per class each with different names
> > (usually describing how the construction occurs or the
> > nature of the object constructed).
> >
> > For example in Smalltalk:
> >
> > obj := MyClass new.  # new is the normal choice of name
> > obj2 := Date today.
> > obj3 := Date new.
> > obj4 := Time now.
> >
> > All of these methods (new, today, now) are constructors(*) but
> > with explicit names that tells you something about the kind
> > of instance created.
> >
> > (*)As I understand it, Smalltalk constructors are just class
> > methods allocated to a constructor category. But the category
> > is just an organisational device used by the IDE/library
> > it doesn't actually change the code in any way.
> >
> > > As I understand it, "named constructor" comes from the C++ world, where
> > > functions are matched not just by name, but by parameters as well.
> >
> > That's overloading which uses the same name but with different
> > numbers/types of parameters and/or return type. The problem with
> > that is that in C++ constructors must have the name of the class.
> > So if you want multiple constructors that all take a single
> > string as their paramater then it gets tricky.
> >
> > > two integers representing the position relative to the entire screen,
> or
> > > two integers representing the position relative to the current window.
> > > The C++ compiler cannot distinguish those two cases, and would give an
> > > error.
> >
> > Exactly so.
> >
> > > The solution is to use constructors with different names, which C++
> > > calls "named constructors".
> >
> > I stopped using C++ around v2 and it didn't have such a feature.
> > Maybe its been added since. If so that's good to know. (Time
> > for some googling methinks...)
> >
> > > So the only way to have two different constructors is to give them
> > > different names.
> > >
> > > Hence all constructors in Python are "named constructors".
> >
> > But they are not native constructors such as those used in
> > Smalltalk etc. They have to be factory methods that call new/init
> > under the covers. And that's how most OOP languages that don't
> > support named constructors get round it. (Although interestingly,
> > Objective C refers to all constructors as 'factory methods', even
> > the 'default' ones.)
> >
> > > It would be reasonable to make this a factory function declared in the
> > > global module level. But I think making it a classmethod is better.
> >
> > I won't put up much of an argument here. In C++ or Java I'd
> > definitely say use a class method. But it seems to me that I've
> > seen more factory functions in modules than class method factories.
> > But that's not been a detailed analysis just a gut feel for what
> > seems idiomatic in Python. It also seems easier for beginners
> > to grasp than the more esoteric notion of class methods.
> >
> > > ... ensures that if you subclass the class, you automatically
> > > get a valid constructor as well.
> >
> > Wouldn't it return an instance of the superclass rather than
> > the sub class? You'd need to override it wouldn't you?
> >
> >
> > --
> > Alan G
> > Author of the Learn to Program web site
> > http://www.alan-g.me.uk/
> > http://www.amazon.com/author/alan_gauld
> > Follow my photo-blog on Flickr at:
> > http://www.flickr.com/photos/alangauldphotos
> >
> >
> >
> >
> > ------------------------------
> >
> > Message: 3
> > Date: Wed, 6 Jan 2016 19:04:24 +0000
> > From: Alan Gauld <alan.gauld at btinternet.com>
> > To: tutor at python.org
> > Subject: Re: [Tutor] Teaching Python
> > Message-ID: <n6jofo$g4a$1 at ger.gmane.org>
> > Content-Type: text/plain; charset=utf-8
> >
> > On 06/01/16 18:23, yehudak . wrote:
> > > My grandson Guy (8th grader) is learning Python at school. That's what
> > made
> > > me teach myself Python programming as well.
> > > Yesterday he asked my help in his homework:
> > >
> > > Write a FOR-loop that prints all numbers up to 1000000
> >
> > And the question is?
> > Python has a for loop and a print function.
> > It can easily handle numbers up to 1000000.
> > You probably need to look at the range() function too.
> >
> > If you are having difficulty with any of those concepts
> > let us know and ideally show us the code that fails.
> >
> > Alternatively if you are asking about what we think of
> > the homework question itself, then I'd say it seems
> > reasonable for a beginner. The only thing I'd gripe
> > about is the use of uppercase FOR.  That's because
> > Python is case sensitive and its for loop is
> > lowercase so the FOR could confuse some students.
> >
> > --
> > Alan G
> > Author of the Learn to Program web site
> > http://www.alan-g.me.uk/
> > http://www.amazon.com/author/alan_gauld
> > Follow my photo-blog on Flickr at:
> > http://www.flickr.com/photos/alangauldphotos
> >
> >
> >
> >
> > ------------------------------
> >
> > Message: 4
> > Date: Wed, 06 Jan 2016 21:40:26 +0100
> > From: Peter Otten <__peter__ at web.de>
> > To: tutor at python.org
> > Subject: Re: [Tutor] Teaching Python
> > Message-ID: <n6ju3r$e38$1 at ger.gmane.org>
> > Content-Type: text/plain; charset="ISO-8859-1"
> >
> > yehudak . wrote:
> >
> > > My grandson Guy (8th grader) is learning Python at school. That's what
> > > made me teach myself Python programming as well.
> > > Yesterday he asked my help in his homework:
> > >
> > > Write a FOR-loop that prints all numbers up to 1000000
> > >
> > > Thank you, the gods of Ctrl+C
> >
> > Did you mean to imply that it took too long?
> >
> > It is likely that the terminal (emulation) is to blame, and not the
> python
> > interpreter:
> >
> > $ time python3 -c 'for i in range(10**6): print(i)' > /dev/null
> >
> > real    0m1.562s
> > user    0m1.525s
> > sys     0m0.033s
> >
> > Under 2 seconds on quite old hardware. Without redirection in Konsole
> > (KDE's
> > terminal emulator):
> >
> > $ time python3 -c 'for i in range(10**6): print(i)'
> > [...]
> > 999997
> > 999998
> > 999999
> >
> > real    0m16.167s
> > user    0m5.258s
> > sys     0m5.512s
> >
> > But wait -- you ran the code in IDLE, right?
> > Then book it under lessons learned ;)
> >
> >
> >
> > ------------------------------
> >
> > Message: 5
> > Date: Wed, 6 Jan 2016 12:42:52 -0800
> > From: Danny Yoo <dyoo at hashcollision.org>
> > To: "yehudak ." <katye2007 at gmail.com>
> > Cc: Python Tutor Mailing List <Tutor at python.org>
> > Subject: Re: [Tutor] Teaching Python
> > Message-ID:
> >         <CAGZAPF5yHAqqXsDDXg3OtKtuV=EUqtDo_rhOv=7g45=
> > V_h2ZMg at mail.gmail.com>
> > Content-Type: text/plain; charset=UTF-8
> >
> > On Wed, Jan 6, 2016 at 10:23 AM, yehudak . <katye2007 at gmail.com> wrote:
> > > My grandson Guy (8th grader) is learning Python at school. That's what
> > made
> > > me teach myself Python programming as well.
> > > Yesterday he asked my help in his homework:
> > >
> > > Write a FOR-loop that prints all numbers up to 1000000
> >
> > Try to say explicitly where you think you're getting stuck. There are a
> few
> > general problem-solving strategies that you can try to apply, in the
> spirit
> > of Polya's How To Solve It:
> > http://www.amazon.com/How-Solve-It-Mathematical-Princeton/dp/069111966X.
> >
> > * Do you understand the question being asked? Do you understand all the
> > technical "terms" that the question is using? This is actually a really
> > important one: if you don't understand the terms, there's very little
> hope
> > to get anything useful out of the experience.
> >
> > - Do you know what a for loop is? Have you seen any examples of for
> loops?
> > - Do you know what it means to print a number? Have you seen any examples
> > of this?
> >
> > * Do you know what the expected output is supposed to look like?
> >
> > * Have you tackled problems that are similar to this one before?
> >
> > * Do you know how to solve a simpler version of the problem? Say, from
> > changing "1000000" to "10"?
> >
> >
> > ------------------------------
> >
> > Message: 6
> > Date: Wed, 6 Jan 2016 20:59:34 +0000
> > From: Tahir Hafiz <tahir.hafiz at gmail.com>
> > To: "yehudak ." <katye2007 at gmail.com>
> > Cc: tutor <Tutor at python.org>
> > Subject: Re: [Tutor] Teaching Python
> > Message-ID:
> >         <CALmb6fvk+hd7ShHts2Qf6zOP1RzNwQo4zU6nVs390Z1ze=
> > jSeA at mail.gmail.com>
> > Content-Type: text/plain; charset=UTF-8
> >
> > Yes that is a huge for loop.
> > I can do it like this but I won't try:
> > >>> for num in list(range(1000000)):
> > >>>   print(num)
> >
> > Sort of strange that the school wanted so much looping for your grandson,
> > they could have picked a lower number to demonstrate a for loop.
> >
> > By the way there is the tqdm module which will show the time of a for
> loop
> > in a nice way:
> > https://github.com/noamraph/tqdm
> >
> >
> > Cheers,
> > Tahir
> >
> >
> >
> >
> >
> >
> >
> > On Wed, Jan 6, 2016 at 6:23 PM, yehudak . <katye2007 at gmail.com> wrote:
> >
> > > My grandson Guy (8th grader) is learning Python at school. That's what
> > made
> > > me teach myself Python programming as well.
> > > Yesterday he asked my help in his homework:
> > >
> > > Write a FOR-loop that prints all numbers up to 1000000
> > >
> > > Thank you, the gods of Ctrl+C
> > >
> > > Yehuda
> > > _______________________________________________
> > > Tutor maillist  -  Tutor at python.org
> > > To unsubscribe or change subscription options:
> > > https://mail.python.org/mailman/listinfo/tutor
> > >
> >
> >
> > ------------------------------
> >
> > Subject: Digest Footer
> >
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > https://mail.python.org/mailman/listinfo/tutor
> >
> >
> > ------------------------------
> >
> > End of Tutor Digest, Vol 143, Issue 17
> > **************************************
> >
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor




That is okay, everyone is very friendly here. No worries.

From fosiul at gmail.com  Wed Jan  6 15:32:28 2016
From: fosiul at gmail.com (Fosiul Alam)
Date: Wed, 6 Jan 2016 20:32:28 +0000
Subject: [Tutor] How to get Key from multiple value dictionary search
Message-ID: <CAPBMF6yKehHouD_wdWDFGmWf6eWwj4dmQhmkorwr_dpXyD2kyg@mail.gmail.com>

Hi All,
I refer to this website
https://www.codecademy.com/forum_questions/50721fce7c7091000201e56a

dic = {'key1': ["value1",  "value2"],
       'key2': ["value77", "something"] }

what I wanted to learn is :

a) how do i get the key only by providing value i.e value77 ?

b) Please help to break the code(instead of one liner) so that i can
understand properly


Thanks.

From cs at zip.com.au  Wed Jan  6 18:00:58 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 7 Jan 2016 10:00:58 +1100
Subject: [Tutor] method, type?
In-Reply-To: <n6jo2r$9op$1@ger.gmane.org>
References: <n6jo2r$9op$1@ger.gmane.org>
Message-ID: <20160106230058.GA80800@cskk.homeip.net>

On 06Jan2016 18:57, ALAN GAULD <alan.gauld at btinternet.com> wrote:
>On 06/01/16 14:46, Steven D'Aprano wrote:
>> It would be reasonable to make this a factory function declared in the
>> global module level. But I think making it a classmethod is better.
>
>I won't put up much of an argument here. In C++ or Java I'd
>definitely say use a class method. But it seems to me that I've
>seen more factory functions in modules than class method factories.
>But that's not been a detailed analysis just a gut feel for what
>seems idiomatic in Python. It also seems easier for beginners
>to grasp than the more esoteric notion of class methods.

Most of my factories are global functions, but that is mere historic artifact.  
As a direct consequence of explaining this to Alex Kleider I'm moving 
aggressively to using @classmethod for this, naming them ".from_*" unless I 
have some amazing reason not to.

>> ... ensures that if you subclass the class, you automatically
>> get a valid constructor as well.
>
>Wouldn't it return an instance of the superclass rather than
>the sub class? You'd need to override it wouldn't you?

No, when you call:

  SubClass.from_blah(...)

where from_blah is defined in the superclass, the leading "cls" parameter is 
the subclass because you invoked it through the subclass, just as you get the 
leading "self" instance parameter when you invoke an instance method through 
the instance. That's one reason that @classmethod is so nice here.

Cheers,
Cameron Simpson <cs at zip.com.au>

From martin at linux-ip.net  Wed Jan  6 18:48:00 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Wed, 6 Jan 2016 15:48:00 -0800
Subject: [Tutor] How to get Key from multiple value dictionary search
In-Reply-To: <CAPBMF6yKehHouD_wdWDFGmWf6eWwj4dmQhmkorwr_dpXyD2kyg@mail.gmail.com>
References: <CAPBMF6yKehHouD_wdWDFGmWf6eWwj4dmQhmkorwr_dpXyD2kyg@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1601061459060.3141@znpeba.jbaqresebt.arg>


Hello there,

>I refer to this website
>https://www.codecademy.com/forum_questions/50721fce7c7091000201e56a
>
>dic = {'key1': ["value1",  "value2"],
>       'key2': ["value77", "something"] }
>
>what I wanted to learn is :
>
>a) how do i get the key only by providing value i.e value77 ?

N.B.  First, see below for more discussion about complexity of the 
data you have in this sample dictionary. (As a result,) Second, I 
have revised your question to

How do I get the keys [pl.] by providing only the value, i.e. 'value77'?

  >>> [k for k, v in dic.iteritems() if 'value77' in v]
  ['key2']

The above is very similar to the example in the codecademy.com link, 
but it returns the key instead of simply showing a True or False.

>b) Please help to break the code(instead of one liner) so that i 
>can understand properly

Yes, of course.  That list comprehension syntax is dense.  There are 
many descriptions of list comprehension and even the link you 
provided includes a reference to the Python documentation.  Here it 
is (again) for anybod else following along:

  https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions

If you do not understand list comprehensions yet, it is always a 
good exercise to see if you can write the same thing with a loop.  
Here's my loop to do the same thing.  Note that I am creating a list 
variable with no entries before the loop.

    l = list()
    for k, v in dic.iteritems():
        if 'value77' in v:
            l.append(k)
    return l

I like writing little functions (the function name here is not very 
creative, though). Anyway, here's a little function that you can 
call that will return the list of all keys whose values contain the 
item of interest:

  def get_all_keys_if_value(d, sought):
      l = list()
      for k, v in d.iteritems():
          if sought in v:
              l.append(k)
      return l

You can call this function like:

  keys_of_interest = get_all_keys_if_value(dic, 'value77')

(Detour begins)

If you are confused about dic.iteritems(), then go to a Python 
console and observe the different results for these:

   >>> dic.keys()
  ['key2', 'key1']
  >>> dic.values()
  [['value77', 'something'], ['value1', 'value2']]
  >>> list(dic.iteritems())
  [('key2', ['value77', 'something']), ('key1', ['value1', 'value2'])]

(Return from detour)

Now, I would like to address the complexity of your data structure.  
There is the following guarantee for a dictionary:  There can be 
only one single entry for each unique key.  In short, the keys are 
always unique.

Your question (in this case) is about searching through the values.  
There's no easy way to lookup based on the values, only keys.

You might be tempted to build a new dictionary (invert yours) and 
use each of the strings, extracted from the lists (dictionary 
values) as the keys in the new dictionary.  It would work for you in 
this case.  The result would be:

  invdic = {'something': 'key2',
   'value1': 'key1',
   'value2': 'key1',
   'value77': 'key2'}

Then you could find the key for 'value77' very simply:

  invdic['value77']  # -- this would be 'key2'

As I mentioned, this would work just fine in your data example, but 
here be dragons!

Your data structure represents a many-to-many data set.  Each 
dictionary value contains a Python list of strings.

It is possible that every single key could point to a list that 
contains the string 'value77'.
  
For example, what happens when somebody adds a new key 'key3' to 
your dictionary and then the dictionary looks like this:

  newdic = {'key1': ['value1', 'value2'],
         'key2': ['value77', 'something'],
         'key3': ['value77'],
         'key4': ['value77', 'value1']}

What would you expect to be returned here:

  keys_of_interest = get_all_keys_if_value(newdic, 'value77')

Well, they should be:

  ['key3', 'key2', 'key4']  # -- keys_of_interest

I may have gone a bit far afield from your original question, but I 
hope you see why I identified the point about the dictionary value 
as a list.  This means the answer could contain anywhere from zero 
keys (the value is not present in any list in the dictionary values) 
OR all keys (the value is present in every list in the dictionary 
values).  {Whew.}

Good luck and have fun,

-Martin

-- 
Martin A. Brown
http://linux-ip.net/

From alan.gauld at btinternet.com  Wed Jan  6 19:06:17 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 7 Jan 2016 00:06:17 +0000
Subject: [Tutor] method, type?
In-Reply-To: <20160106230058.GA80800@cskk.homeip.net>
References: <n6jo2r$9op$1@ger.gmane.org>
 <20160106230058.GA80800@cskk.homeip.net>
Message-ID: <n6ka5p$d9o$1@ger.gmane.org>

On 06/01/16 23:00, Cameron Simpson wrote:

>>> ... ensures that if you subclass the class, you automatically
>>> get a valid constructor as well.
>>
>> Wouldn't it return an instance of the superclass rather than
>> the sub class? You'd need to override it wouldn't you?
> 
> No, when you call:
> 
>   SubClass.from_blah(...)
> 
> where from_blah is defined in the superclass, the leading "cls" parameter is 
> the subclass because you invoked it through the subclass, 

Yes, but my point is that constructors by their nature tend
to be very  specific.

For example a constructor that fetches data from a database
will embed a SQL query that is table specific and so may be
completely wrong for a sub class (unless it somehow shares
the superclass's table). Similarly reading from a network it
will only pull the fields necessary for the superclass from
the message. Indeed the trigger message it sends to the
server to initiate the transfer may well have the class
details included. So although you might wind up with an
object whose type is subclass, its details may be totally
wrong or even fail to operate (if the table has no
corresponding ID for example).

It would certainly work for a few cases but I suspect
most real-world constructors will fail unless overridden.
Other class methods are much more likely to work OK. (For
example those maintaining a cache of instances or simply
keeping a count) I'm just dubious about constructors
because they are so specific about where/how they obtain
their data. (Unless you get very clever about how you
use cls to access meta-data, or maintain some kind of
config mapping (possibly as class attributes) to tell
the constructor what to do.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Wed Jan  6 19:24:34 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 7 Jan 2016 00:24:34 +0000
Subject: [Tutor] method, type?
In-Reply-To: <n6jo2r$9op$1@ger.gmane.org>
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
 <n6ioik$hq7$1@ger.gmane.org> <20160106144608.GA10854@ando.pearwood.info>
 <n6jo2r$9op$1@ger.gmane.org>
Message-ID: <n6kb82$tdt$1@ger.gmane.org>

On 06/01/16 18:57, Alan Gauld wrote:

>> The solution is to use constructors with different names, which C++ 
>> calls "named constructors".
> 
> I stopped using C++ around v2 and it didn't have such a feature.
> Maybe its been added since. If so that's good to know. (Time
> for some googling methinks...)
> 

OK, I Googled. It seems named constructors is just an
idiom not an actual language feature. Pity :-(

And the idiom consists of two things:
1) Providing public class (static) factory methods
2) Making the actual constructor(s) protected

This has a similar feel to named constructors and is
identical to how you would do it in Python (except
for the protected bit) or Java. It's not a true
constructor function it's just a factory (which calls
the true constructors).

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Wed Jan  6 19:31:31 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 7 Jan 2016 11:31:31 +1100
Subject: [Tutor] method, type?
In-Reply-To: <n6jo2r$9op$1@ger.gmane.org>
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
 <n6ioik$hq7$1@ger.gmane.org> <20160106144608.GA10854@ando.pearwood.info>
 <n6jo2r$9op$1@ger.gmane.org>
Message-ID: <20160107003131.GB10854@ando.pearwood.info>

On Wed, Jan 06, 2016 at 06:57:31PM +0000, Alan Gauld wrote:

> On 06/01/16 14:46, Steven D'Aprano wrote:
> > I don't understand what you mean by "Python doesn't support named 
> > constructors". It seems to me that this is the *only* sort of 
> > constructor that Python supports.
> 
> No, Python constructors have names(new/init) but they are not
> used explicitly (ie. by the client code)to create objects.

The default new/init constructor isn't, but any additional ones are. The 
classic example is dict.fromkeys. Other examples include:

    Decimal.from_float 
    fractions.from_float
    fractions.from_decimal
    datetime.fromordinal
    datetime.fromtimestamp



> By contrast languages like Delphi, Smalltalk, Objective C,
> some Lisps and, I think, Eiffel (and others?) all require an
> explicit call of a constructor method 

The fact that in Python the default constructor is called via the class 
name is, in my opinion, just a cosmetic difference: MyClass(x) versus 
MyClass.new(arg) is not that big a difference. It just means that 
one name is implicit.

Contrast that to the C++ style where the compiler can do multiple 
dispatch by the number and type of arguments:

# Using Python syntax again
class Spam:
    # four constructors
    def Spam(afloat): ...
    def Spam(astring): ...
    def Spam(inta): ...
    def Spam(inta, intb): ...

compared to the Python/Smalltalk/Delphi etc style where they all have to 
be named differently:

class Spam:
    def __new__(cls, afloat): ...
    def from_word(cls, astring): ...
    def from_count(cls, inta): ...
    def from_ratio(cls, inta, intb): ...


But if you want to insist that the default Python constructor of __new__ 
plus __init__ doesn't qualify as a "named constructor" because it is 
called implicitly rather than explicitly, I won't argue. I'll just say 
that Python has named constructors for the *alternate* (non-default) 
constructors :-)



> and there may be
> multiple constructors per class each with different names
> (usually describing how the construction occurs or the
> nature of the object constructed).

Exactly like in Python :-)


> > So the only way to have two different constructors is to give them 
> > different names.
> > 
> > Hence all constructors in Python are "named constructors".
> 
> But they are not native constructors such as those used in
> Smalltalk etc. They have to be factory methods that call new/init
> under the covers.

No they don't. It is *most common* for alternate constructors to hand 
over the actual work of building the instance to __new__, because DRY. 
Hence:

class Spam(object):
    def __new__(cls, alist):
        # do the real work here

    @classmethod
    def from_string(cls, astring):
        alist = astring.split()
        return cls(alist)


But that's not a hard rule. Any method can manually create an instance, 
well, any *new-style class* method. (Classic classes from Python 2 are 
different, because they can't have constructors except for the default.)

class Eggs(object):
    @classmethod
    def this_is_my_constructor(cls):
        instance = object.__new__(cls)
        instance.initiate_me()
        return instance
    def initiate_me(self):
        print("Initiating...")
        self.style = "hard boiled"
    # And just to prove that the default constructor isn't used:
    def __new__(cls):
        raise TypeError


e = Eggs.this_is_my_constructor()


So while you are right that most alternate constructors end up calling 
the default __new__, that's not compulsory. It's just convenient.


> > ... ensures that if you subclass the class, you automatically
> > get a valid constructor as well.
> 
> Wouldn't it return an instance of the superclass rather than
> the sub class? You'd need to override it wouldn't you?

Not if you write it correctly :-)

If you hard-code the name of the superclass into the method, then you 
will always get the superclass. That is bad.

# Don't do this, this is wrong.
class Spam(object):
    @classmethod
    def from_eggs(cls, eggs):
        args = process(eggs)
        return Spam(args)


# Do this instead.
class Spam(object):
    @classmethod
    def from_eggs(cls, eggs):
        args = process(eggs)
        return cls(args)




-- 
Steve

From steve at pearwood.info  Wed Jan  6 19:43:18 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 7 Jan 2016 11:43:18 +1100
Subject: [Tutor] Teaching Python
In-Reply-To: <n6jofo$g4a$1@ger.gmane.org>
References: <CAE3ie43GPx6uD5H8CQFsj-L1TfRwwHLCAs0jbaq02wV-c+V1Lw@mail.gmail.com>
 <n6jofo$g4a$1@ger.gmane.org>
Message-ID: <20160107004318.GC10854@ando.pearwood.info>

On Wed, Jan 06, 2016 at 07:04:24PM +0000, Alan Gauld wrote:
> On 06/01/16 18:23, yehudak . wrote:
> > My grandson Guy (8th grader) is learning Python at school. That's what made
> > me teach myself Python programming as well.
> > Yesterday he asked my help in his homework:
> > 
> > Write a FOR-loop that prints all numbers up to 1000000
> 
> And the question is?

I don't think that Yehudak had a question, I think he is just sharing 
his experiences.

(I think Yehudak is a male name, apologies in advance if I got that 
wrong.)


> Python has a for loop and a print function.
> It can easily handle numbers up to 1000000.
> You probably need to look at the range() function too.
> 
> If you are having difficulty with any of those concepts
> let us know and ideally show us the code that fails.

I'm guessing that Yehudak is griping about how slow it is and how much 
pointless output it generates. I agree: on my computer, just looping 
from 1 to a million takes 0.2 seconds, but printing out the numbers not 
only generates a million lines of output, but takes 100 times longer: 20 
seconds.



-- 
Steve

From steve at pearwood.info  Wed Jan  6 20:00:22 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 7 Jan 2016 12:00:22 +1100
Subject: [Tutor] method, type?
In-Reply-To: <n6ka5p$d9o$1@ger.gmane.org>
References: <n6jo2r$9op$1@ger.gmane.org>
 <20160106230058.GA80800@cskk.homeip.net> <n6ka5p$d9o$1@ger.gmane.org>
Message-ID: <20160107010022.GD10854@ando.pearwood.info>

On Thu, Jan 07, 2016 at 12:06:17AM +0000, Alan Gauld wrote:
> On 06/01/16 23:00, Cameron Simpson wrote:
> 
> >>> ... ensures that if you subclass the class, you automatically
> >>> get a valid constructor as well.
> >>
> >> Wouldn't it return an instance of the superclass rather than
> >> the sub class? You'd need to override it wouldn't you?
> > 
> > No, when you call:
> > 
> >   SubClass.from_blah(...)
> > 
> > where from_blah is defined in the superclass, the leading "cls" parameter is 
> > the subclass because you invoked it through the subclass, 
> 
> Yes, but my point is that constructors by their nature tend
> to be very  specific.

That surely depends on the class.

> For example a constructor that fetches data from a database
> will embed a SQL query that is table specific and so may be
> completely wrong for a sub class (unless it somehow shares
> the superclass's table).

How many classes fetch data from a database? I accept that your 
experience may be different from mine, but my experience tells me that 
this is pretty unusual.

In any case, if the alternate constructor calls the default constructor, 
you may only need to override the default, not both.

The point is that while it is true that the alternate constructors *may* 
need to be overriden, sometimes they don't if you write them well. But 
if you use a global, top-level factory function, you have little choice 
but to create a separate factory function for every subclass.

I suppose you might make the class an explicit argument of the factory:

def factory(x, y, z, theclass=Spam):
    return theclass(x+y+z)

but a class method seems more natural for Python.



-- 
Steve

From cs at zip.com.au  Wed Jan  6 20:28:02 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 7 Jan 2016 12:28:02 +1100
Subject: [Tutor] method, type?
In-Reply-To: <n6ka5p$d9o$1@ger.gmane.org>
References: <n6ka5p$d9o$1@ger.gmane.org>
Message-ID: <20160107012802.GA91934@cskk.homeip.net>

On 07Jan2016 00:06, ALAN GAULD <alan.gauld at btinternet.com> wrote:
>On 06/01/16 23:00, Cameron Simpson wrote:
>
>>>> ... ensures that if you subclass the class, you automatically
>>>> get a valid constructor as well.
>>>
>>> Wouldn't it return an instance of the superclass rather than
>>> the sub class? You'd need to override it wouldn't you?
>>
>> No, when you call:
>>
>>   SubClass.from_blah(...)
>>
>> where from_blah is defined in the superclass, the leading "cls" parameter is
>> the subclass because you invoked it through the subclass,
>
>Yes, but my point is that constructors by their nature tend
>to be very  specific.

That wasn't what you said; because cls comes in as a parameter you will 
inherently get the subclass.

To your point: I don't think factory methods are necessarily very different in 
a subclass. Sometimes?  Often?  I think it is pretty variable.  In principle 
factory functions need not vary much more than any other method.  And like any 
other method, if it is special then it needs an override.

>For example a constructor that fetches data from a database
>will embed a SQL query that is table specific and so may be
>completely wrong for a sub class (unless it somehow shares
>the superclass's table).

Or if the table name (or table definition from some mapping) is a parameter of 
the subclass. I would try to make it so myself.

>Similarly reading from a network it
>will only pull the fields necessary for the superclass from
>the message. Indeed the trigger message it sends to the
>server to initiate the transfer may well have the class
>details included. So although you might wind up with an
>object whose type is subclass, its details may be totally
>wrong or even fail to operate (if the table has no
>corresponding ID for example).

Shrug. So in these cases you either need to have per-class parameters or have 
to override the factory methods as required. Like anything else. I don't think 
this is an argument against @classmethod factory functions, merely an argument 
against inattention when subclassing.

>It would certainly work for a few cases but I suspect
>most real-world constructors will fail unless overridden.
>Other class methods are much more likely to work OK. (For
>example those maintaining a cache of instances or simply
>keeping a count) I'm just dubious about constructors
>because they are so specific about where/how they obtain
>their data. (Unless you get very clever about how you
>use cls to access meta-data, or maintain some kind of
>config mapping (possibly as class attributes) to tell
>the constructor what to do.

I agree it won't also work, or work smoothly. But parameters or metadata from 
the class/subclass may be quite viable - that is the great convenience of the 
cls parameter to class methods. Once all this stuff starts getting too hard to 
look after you're probably in either don't-subclass territory, or providing 
common methods/mechanisms with mixins (i.e. a lookalike class instead of a 
subclass).

Cheers,
Cameron Simpson <cs at zip.com.au>

From brianna.mcgee2 at gmail.com  Wed Jan  6 17:57:26 2016
From: brianna.mcgee2 at gmail.com (Brianna McGee)
Date: Wed, 6 Jan 2016 14:57:26 -0800
Subject: [Tutor] help with existing code
Message-ID: <CAPu3Tkm-wZYe7adxUP+P8pCNh7dpE-KhPsZsS-xibLOMp-GvOg@mail.gmail.com>

I am a new to programming and have started out with modifying existing code
to suit my purposes in my Developmental Psychology lab. The script worked
fine previous to my fiddling with it, but now that I am trying to modify
the x and y locations based upon a text file with existing x and y
locations it will run but won't create the jpegs. I've attached an example
of the text file. HELP! Please let me know if you need to see the rest of
the code.

 if setsize == 4:

                locations = open('X_Y_locations_SS4.txt', 'r')

                i = 0;
                for line in locations:
                    i = i+1;

                    if i % 2 != 0:

                        pointArray = line.split(",", 8);
                        x1 = int(pointArray[0]);
                        y1 = int(pointArray[1]);
                        x2 = int(pointArray[2]);
                        y2 = int(pointArray[3]);
                        x3 = int(pointArray[4]);
                        y3 = int(pointArray[5]);
                        x4 = int(pointArray[6]);
                        y4 = int(pointArray[7]);

   Thank you in advance!
-------------- next part --------------
1454,657,530,794,410,464,1130,289,
111,107,820,512,634,671,693,577,
140,676,1463,868,90,312,403,602,
1088,963,817,542,147,300,565,561,
1471,595,1150,861,1085,979,101,110,
1395,334,402,209,893,146,407,196,
1153,626,575,287,1079,759,1230,791,
1236,759,157,645,1026,558,701,240,
1407,213,81,723,249,256,1476,456,
897,712,270,578,1565,474,1133,459,
1398,151,533,679,1546,788,672,922,
1101,450,815,336,996,473,960,121,
604,770,927,685,1338,194,1362,700,
880,145,1106,605,1250,500,456,803,
577,338,164,617,1027,886,679,233,
1559,381,1026,918,1552,698,967,608,
566,737,1061,330,1432,473,1440,725,
495,715,806,870,1075,517,1050,89,
1582,722,1033,83,510,503,595,245,
1156,544,915,699,630,383,725,847,
809,752,553,159,568,804,527,550,
1162,667,615,528,180,891,165,653,
304,553,517,909,1290,303,533,293,
250,450,1444,166,527,413,314,888,
231,852,696,533,842,139,1128,121,

From katsu-t at selene.is.dream.jp  Wed Jan  6 18:19:05 2016
From: katsu-t at selene.is.dream.jp (Katsuyoshi Takahashi)
Date: Thu, 07 Jan 2016 08:19:05 +0900
Subject: [Tutor] wx.CallLater doesn't work in subprocess
Message-ID: <568DA0E9.4000709@selene.is.dream.jp>

Hello everyone,

I'm trying make a the Simple Game program with GUI using wxPython. I 
used multiprocessing library in Python version 3.4. But, wx.CallLater 
Class doesn't work in subprocess.

classLOCthink(object):
     def__init__(self):
         super(LOCthink,self).__init__()

     defengine_think(self,queue_send,queue_command):

         importwx

         self.queue_command = queue_command
         self.queue_send = queue_send
         self.coord =0

         self.command = wx.CallLater(100,self.command_process)

     defcommand_process(self):
         fordummyinrange(10):
             self.queue_send.put_nowait(str(dummy))
         self.command.Start(100)

When I start Process at 'engine_think' function, then error occur with 
this message,

     self.command = wx.CallLater(100, self.command_process)
AttributeError: 'function' object has no attribute 'command_process'

In main process, I use same wx.CallAfter class and it works.
Is wx.CallLater not be able to use in subprocess? or I write wrong code 
in this class?


From __peter__ at web.de  Thu Jan  7 05:59:57 2016
From: __peter__ at web.de (Peter Otten)
Date: Thu, 07 Jan 2016 11:59:57 +0100
Subject: [Tutor] help with existing code
References: <CAPu3Tkm-wZYe7adxUP+P8pCNh7dpE-KhPsZsS-xibLOMp-GvOg@mail.gmail.com>
Message-ID: <n6lgff$5ot$1@ger.gmane.org>

Brianna McGee wrote:

> I am a new to programming and have started out with modifying existing
> code to suit my purposes in my Developmental Psychology lab. The script
> worked fine previous to my fiddling with it, but now that I am trying to
> modify the x and y locations based upon a text file with existing x and y
> locations it will run but won't create the jpegs. I've attached an example
> of the text file. HELP! Please let me know if you need to see the rest of
> the code.
> 
>  if setsize == 4:
> 
>                 locations = open('X_Y_locations_SS4.txt', 'r')
> 
>                 i = 0;
>                 for line in locations:
>                     i = i+1;
> 
>                     if i % 2 != 0:
> 
>                         pointArray = line.split(",", 8);
>                         x1 = int(pointArray[0]);
>                         y1 = int(pointArray[1]);
>                         x2 = int(pointArray[2]);
>                         y2 = int(pointArray[3]);
>                         x3 = int(pointArray[4]);
>                         y3 = int(pointArray[5]);
>                         x4 = int(pointArray[6]);
>                         y4 = int(pointArray[7]);
> 
>    Thank you in advance!

I'm sorry, the chunk of code you present is not sufficient to diagnose your 
problem. 

Is setsize actually 4?
Does the inner loop run?

You can find out by adding print(...) function calls to the code (print 
statements if you are using python 2). If that works as you wish:

What did the code you replaced with the above look like? 
How are x1,y1,...,y4 used?

Give us a bit more context.




From oscar.j.benjamin at gmail.com  Thu Jan  7 10:35:39 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 7 Jan 2016 15:35:39 +0000
Subject: [Tutor] Tutor Digest, Vol 143, Issue 17
In-Reply-To: <CAE3ie430VeD+1W7r=Zj8wT7ndJ50HxSi9gDJJAzgeuqLQ752EA@mail.gmail.com>
References: <mailman.968.1452113982.2304.tutor@python.org>
 <CAE3ie430VeD+1W7r=Zj8wT7ndJ50HxSi9gDJJAzgeuqLQ752EA@mail.gmail.com>
Message-ID: <CAHVvXxS1nr0MSGZWk6coOgxBz0hMue5rj4gMtmyc1yD146R6Mg@mail.gmail.com>

On 6 January 2016 at 21:07, yehudak . <katye2007 at gmail.com> wrote:
>
> Alan, Tahir & friends,
> My post was not a question, merely an observation.
> I hope there's no objection to my mail.

Hi Yehudak,

Please don't reply to the digest messages. The email you have sent is
really long as it contains all of the messages sent in one day on the
tutor list (below the small part that you wrote). If you do want to
reply to a message that you see in the digest please trim from the
digest email so that only the message you are responding to remains.

Better yet: don't use the digest email. I have all of the tutor emails
come to my gmail account and then I have a filter set up in gmail so
that they go to a special folder. This means that they don't clutter
up my inbox and there's no need to subscribe to the digest. You can
change your subscription settings here:
https://mail.python.org/mailman/listinfo/tutor

--
Oscar

From james at uplinkzero.com  Thu Jan  7 10:48:12 2016
From: james at uplinkzero.com (James Chapman)
Date: Thu, 7 Jan 2016 15:48:12 +0000
Subject: [Tutor] reading an input stream
In-Reply-To: <CAG7edPFKyb1iC0ChQJAMpNRwE75nDQSM9NOakQ+As01nnutTQQ@mail.gmail.com>
References: <CAG7edPHOoqWVxfYsx-KDCo7JPxdSRZKBAE_yrWeRBh9dvazTvA@mail.gmail.com>
 <CAGZAPF47Y0gfzeKpjkn3nxa4PD6RRoujnvnJpmLXugq7cx=fPg@mail.gmail.com>
 <CAG7edPFKyb1iC0ChQJAMpNRwE75nDQSM9NOakQ+As01nnutTQQ@mail.gmail.com>
Message-ID: <CAHvkzykeYQ_G-8BpKEHzcr-o6Rq+a188ptuiTJwnoDpnB=B7Dg@mail.gmail.com>

Hi Richard

There are a number of considerations you need to take into account here.

Raw sockets is almost never the right solution, while a basic socket to
socket connection is easy enough to program, handling failure and
concurrency can very quickly make the solution a lot more complex than it
needs to be, so perhaps you could supply more information? (I realise I'm
venturing outside the realm of learning python, but I'm a pedant for doing
things right).

You said you need to read XML in from a socket connection. You've not
mentioned what's generating the data? Is that data sent over HTTP in which
case is this part of a SOAP or REST API? Is the data being generated by
something you've written or a 3rd party software package? Is REST an
option? Is there a reason to serialise to XML? (If I was performing the
serialisation I would go with JSON if being human readable was a
requirement. )

If the method of receiving that data is optional, have you considered using
something like AMQP (RabbitMQ) which would eliminate your need to support
concurrency? It would also handle failure well.

James


--
James

On 29 December 2015 at 20:14, richard kappler <richkappler at gmail.com> wrote:

> Sorry it took so long to respond, just getting back from the holidays. You
> all have given me much to think about. I've read all the messages through
> once, now I need to go trough them again and try to apply the ideas. I'll
> be posting other questions as I run into problems. BTW, Danny, best
> explanation of generators I've heard, well done and thank you.
>
> regards, Richard
>
> On Thu, Dec 24, 2015 at 4:54 PM, Danny Yoo <dyoo at hashcollision.org> wrote:
>
> > > I think what I need to do would be analogous to (pardon if I'm using
> the
> > > wrong terminology, at this poing in the discussion I am officially out
> of
> > > my depth) sending the input stream to a buffer(s) until  the ETX for
> that
> > > message comes in, shoot the buffer contents to the parser while
> accepting
> > > the next STX + message fragment into the buffer, or something
> analogous.
> >
> > Yes, I agree.  It sounds like you have one process read the socket and
> > collect chunks of bytes delimited by the STX markers.  It can then
> > send those chunks to the XML parser.
> >
> >
> > We can imagine one process that reads the socket and spits out a list
> > of byte chunks:
> >
> >     chunks = readDelimitedChunks(socket)
> >
> > and another process that parses those chunks and does something with
> them:
> >
> >     for chunk in chunks:
> >         ....
> >
> >
> > It would be nice if we could organize the program like this.  But one
> > problem is that chunks might not be finite!  The socket might keep on
> > returning bytes.  If it keeps returning bytes, we can't possibly
> > return a finite list of the chunked bytes.
> >
> >
> > What we really want is something like:
> >
> >     chunkStream = readDelimitedChunks(socket)
> >     for chunk in chunkStream:
> >         ....
> >
> > where chunkStream is itself like a socket: it should be something that
> > we can repeatedly read from as if it were potentially infinite.
> >
> >
> > We can actually do this, and it isn't too bad.  There's a mechanism in
> > Python called a generator that allows us to write function-like things
> > that consume streams of input and produce streams of output.  Here's a
> > brief introduction to them.
> >
> > For example, here's a generator that knows how to produce an infinite
> > stream of numbers:
> >
> > ##############
> > def nums():
> >     n = 0
> >     while True:
> >         yield n
> >         n += 1
> > ##############
> >
> > What distinguishes a generator from a regular function?  The use of
> > "yield".  A "yield" is like a return, but rather than completely
> > escape out of the function with the return value, this generator will
> > remember what it was doing  at that time.  Why?  Because it can
> > *resume* itself when we try to get another value out of the generator.
> >
> > Let's try it out:
> >
> > #####################
> >
> > >>> numStream = nums()
> > >>> numStream.next()
> > 0
> > >>> numStream.next()
> > 1
> > >>> numStream.next()
> > 2
> > >>> numStream.next()
> > 3
> > >>> numStream.next()
> > 4
> > #####################
> >
> > Every next() we call on a generator will restart it from where it left
> > off, until it reaches its next "yield".  That's how we get this
> > generator to return an infinite sequence of things.
> >
> >
> > That's how we produce infinite sequences.  And we can write another
> > generator that knows how to take a stream of numbers, and square each
> > one.
> >
> > ########################
> > def squaring(stream):
> >     for n in stream:
> >         yield n
> > ########################
> >
> >
> > Let's try it.
> >
> >
> > ########################
> >
> > >>> numStream = nums()
> > >>> squaredNums = squaring(numStream)
> > >>> squaredNums.next()
> > 0
> > >>> squaredNums.next()
> > 1
> > >>> squaredNums.next()
> > 4
> > >>> squaredNums.next()
> > 9
> > >>> squaredNums.next()
> > 16
> > ########################
> >
> >
> > If you have experience with other programming languages, you may have
> > heard of the term "co-routine".  What we're doing with this should be
> > reminiscent of coroutine-style programming.  We have one generator
> > feeding input into the other, with program control bouncing back and
> > forth between the generators as necessary.
> >
> >
> > So that's a basic idea of generators.  It lets us write processes that
> > can deal with and produce streams of data.  In the context of sockets,
> > this is particularly helpful, because sockets can be considered a
> > stream of bytes.
> >
> >
> > Here's another toy example that's closer to the problem you're trying
> > to solve.  Let's say that we're working on a program to alphabetize
> > the words of a sentence.  Very useless, of course.  :P  We might pass
> > it in the input:
> >
> >     this
> >     is
> >     a
> >     test
> >     of
> >     the
> >     emergency
> >     broadcast
> >     system
> >
> > and expect to get back the following sentence:
> >
> >      hist
> >      is
> >      a
> >      estt
> >      fo
> >      eht
> >      ceeegmnry
> >      aabcdorst
> >      emssty
> >
> > We can imagine one process doing chunking, going from a sequence of
> > characters to a sequence of words:
> >
> > ###########################################
> > def extract_words(seq):
> >     """Yield the words in a sequence of characters."""
> >     buffer = []
> >     for ch in seq:
> >         if ch.isalpha():
> >             buffer.append(ch)
> >         elif buffer:
> >             yield ''.join(buffer)
> >             del buffer[:]
> >     # If we hit the end of the buffer, we still might
> >     # need to yield one more result.
> >     if buffer:
> >         yield ''.join(buffer)
> > ###########################################
> >
> >
> > and a function that transforms words to their munged counterpart:
> >
> > #########################
> > def transform(word):
> >     """"Munges a word into its alphabetized form."""
> >     chars = list(word)
> >     chars.sort()
> >     return ''.join(chars)
> > #########################
> >
> > This forms the major components of a program that can do the munging
> > on a file... or a socket!
> >
> >
> > Here's the complete example:
> >
> >
> > #############################################
> > import sys
> >
> > def extract_words(seq):
> >     """Yield the words in a sequence of characters."""
> >     buffer = []
> >     for ch in seq:
> >         if ch.isalpha():
> >             buffer.append(ch)
> >         elif buffer:
> >             yield ''.join(buffer)
> >             del buffer[:]
> >     # If we hit the end of the buffer, we still might
> >     # need to yield one more result.
> >     if buffer:
> >         yield ''.join(buffer)
> >
> > def transform(word):
> >     """"Munges a word into its alphabetized form."""
> >    chars = list(word)
> >     chars.sort()
> >     return ''.join(chars)
> >
> >
> > def as_byte_seq(f):
> >     """Return the bytes of the file-like object f as a
> >     sequence."""
> >     while True:
> >         ch = f.read(1)
> >         if not ch: break
> >         yield ch
> >
> >
> > if __name__ == '__main__':
> >     for word in extract_words(as_byte_seq(sys.stdin)):
> >         print(transform(word))
> > ############################################
> >
> >
> >
> > If you have questions, please feel free to ask.  Good luck!
> >
>
>
>
> --
>
> All internal models of the world are approximate. ~ Sebastian Thrun
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From richkappler at gmail.com  Thu Jan  7 11:23:35 2016
From: richkappler at gmail.com (richard kappler)
Date: Thu, 7 Jan 2016 11:23:35 -0500
Subject: [Tutor] reading an input stream
In-Reply-To: <CAHvkzykeYQ_G-8BpKEHzcr-o6Rq+a188ptuiTJwnoDpnB=B7Dg@mail.gmail.com>
References: <CAG7edPHOoqWVxfYsx-KDCo7JPxdSRZKBAE_yrWeRBh9dvazTvA@mail.gmail.com>
 <CAGZAPF47Y0gfzeKpjkn3nxa4PD6RRoujnvnJpmLXugq7cx=fPg@mail.gmail.com>
 <CAG7edPFKyb1iC0ChQJAMpNRwE75nDQSM9NOakQ+As01nnutTQQ@mail.gmail.com>
 <CAHvkzykeYQ_G-8BpKEHzcr-o6Rq+a188ptuiTJwnoDpnB=B7Dg@mail.gmail.com>
Message-ID: <CAG7edPEggUvwvUHpat8ovkGvRO3RudeHbz9JYKWqd2+f5Jx7Ew@mail.gmail.com>

Hi James,

I've actually come a ways since this message was posted, but still have
some related struggles.


>Raw sockets is almost never the right solution, while a basic socket to
socket connection is easy enough to program, handling failure and
concurrency can very quickly make >the solution a lot more complex than it
needs to be, so perhaps you could supply more information? (I realise I'm
venturing outside the realm of learning python, but I'm a >pedant for doing
things right).

>You said you need to read XML in from a socket connection. You've not
mentioned what's generating the data? Is that data sent over HTTP in which
case is this part of a SOAP >or REST API? Is the data being generated by
something you've written or a 3rd party software package? Is REST an
option? Is there a reason to serialise to XML? (If I was >performing the
serialisation I would go with JSON if being human readable was a
requirement. )

The method of receiving data is neither optional nor under our control. The
XML is generated by a camera tunnel and contains data about the packages
that pass through the tunnel. It comes in over the network (tcp) to a
specific port (2008). This is all within an internal vpn so security, for
the most part, is not an issue, though efficiency is. This one script can
have as many as 30+ connections to it. The xml messages have STX (\x02) and
ETX (\x03) bookending them and come as fast as 3-4 per second per
connection. At the moment we're in the initial dev stages. We have a
general framework decided upon, now are trying to work out the code. Here's
what works thus far (sorta-kinda):

#!/usr/bin/env python

import socket
import lxml.etree as ET

def dataParse(data):
    print 'parsing'
    xslt = ET.parse('stack13.xsl')
    dom = ET.XML(data)
    transform = ET.XSLT(xslt)
    newdom = transform(dom)
    f1.write(str(newdom))

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock_addr = ('', 2008)
sock.bind(sock_addr)
sock.listen(40)
print 'listening'

f1 = open('parser.out', 'a')
print "opening parser.out"

while True:
    # wait for a connection
    connection, client_address = sock.accept()
    while True:
        data = connection.recv(8192)
        if data:
            dataParse(data)

f1.close()

There is a tunnel simulator in our test environment that reads from a file,
each message on a separate line, and sends each line out through the socket
to the above. I say 'sorta-kinda' because this only works if there is a
time delay between each line/message being sent. If there is no time delay,
the dataParser throws a

lxml.etree.XMLSyntaxError: Extra content at the end of the document, line
2, column 1

error. This makes sense as the parser is currently just reading whatever is
in the buffer and the messages being sent DO NOT currently have the STX/ETX
on them. We are currently working on several fronts: changing the test
tunnelSim file to include STX/ETX; making the server multi-threaded, one
thread per incoming connection (working with numerous examples found on the
web); setting up a buffer within each thread into which 'data' will go in
one end (data += buffer?) and comes out the other - using a generator that
reads from STX to ETX to take a full 'message' out of the buffer that will
then be yielded to dataParse; and sending the parsed data out to via a
splunk event writer.

Or at least that's the idea. We're still far away from figuring it out, but
moving closer.

regards, Richard

From james at uplinkzero.com  Thu Jan  7 12:07:03 2016
From: james at uplinkzero.com (James Chapman)
Date: Thu, 7 Jan 2016 17:07:03 +0000
Subject: [Tutor] reading an input stream
In-Reply-To: <CAG7edPEggUvwvUHpat8ovkGvRO3RudeHbz9JYKWqd2+f5Jx7Ew@mail.gmail.com>
References: <CAG7edPHOoqWVxfYsx-KDCo7JPxdSRZKBAE_yrWeRBh9dvazTvA@mail.gmail.com>
 <CAGZAPF47Y0gfzeKpjkn3nxa4PD6RRoujnvnJpmLXugq7cx=fPg@mail.gmail.com>
 <CAG7edPFKyb1iC0ChQJAMpNRwE75nDQSM9NOakQ+As01nnutTQQ@mail.gmail.com>
 <CAHvkzykeYQ_G-8BpKEHzcr-o6Rq+a188ptuiTJwnoDpnB=B7Dg@mail.gmail.com>
 <CAG7edPEggUvwvUHpat8ovkGvRO3RudeHbz9JYKWqd2+f5Jx7Ew@mail.gmail.com>
Message-ID: <CAHvkzykrRLB4ZSQ63pnhsWnvFC7eoyDRzSUFF1_wMDSxW97z1w@mail.gmail.com>

?Ah yes, OK.

STX and ETX are control characters ?as you've probably worked out. Are
there other control characters? Ideally you should have some kind of
message header with the size of the data to be received. You'd then know
how big the receiving buffer needs to be rather than having a fixed and
presumably maximum size of incoming message buffer size.

>From an architectural POV I'd have a few listener threads that upon receipt
would spawn (or take from a pool is a better approach) a worker thread to
process the received data.

Some python snippets that may/may not be of help in my github repo here
<https://github.com/James-Chapman/python-code-snippets>. There are
unfortunately no raw socket code snippets in there.

I'd be keen to see the raw socket data (stripped of any actual xml content
and anything other identifying indicators) as I'm assuming there's more
than just...

\x02
<?xml version="1.0" encoding="utf-8"?>
<foo>
  <bar></bar>
</foo>
\x03


-J

From richkappler at gmail.com  Thu Jan  7 12:02:17 2016
From: richkappler at gmail.com (richard kappler)
Date: Thu, 7 Jan 2016 12:02:17 -0500
Subject: [Tutor] looping generator
Message-ID: <CAG7edPETDnBS1RXoFio8Xa3qebT5C_PY-8Wygpx=gny-Z2+YEg@mail.gmail.com>

I have a stream of incoming xml data. I can receive the data, parse the
data, etc, so long as I don't get fancy and I have a miniscule delay in
between each message. If I get rid of the time delay, which I need to, I
need the script to continuously process the incoming messages. Here's what
I have:

#!/usr/bin/env python

import socket
import lxml.etree as ET

def dataRecv(connection):
    print 'receiving'
    while True:
        data = connection.recv(65536)
        while True:
            print "writing to data.in"
            f2.write(data)
            start = data.find('\x02')
            end = data.find('\x03')
            message = data[start+1:end]
            print "writing to messages.out"
            f3.write(message)
            yield message

def dataParse(message):
    print 'parsing'
    xslt = ET.parse('stack13.xsl')
    dom = ET.XML(message)
    transform = ET.XSLT(xslt)
    newdom = transform(dom)
    f1.write(str(newdom))


sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock_addr = ('', 2008)
#data = sock.makefile('r')
sock.bind(sock_addr)
sock.listen(5)
print 'listening'

f1 = open('parser.out', 'a')
print "opening parser.out"
f2 = open('data.in', 'a')
print "opening data.in"
f3 = open('messages.out', 'a')
print "opening messages.out"

while True:
    # wait for a connection
    connection, client_address = sock.accept()
    q = dataRecv(connection)
    dataParse(q.next())

# close sockrx
#connection.close()

f1.close()


In the dataRecv function, I have tried (where you see while True) if data,
while data and while True. Regardless, it doesn't loop, it receives al ten
messages from the test file being sent, but only processes the first
message then stops (not exits). I feel like I'm missing something obvious
but can't find it.

regards, Richard

From richkappler at gmail.com  Thu Jan  7 12:14:58 2016
From: richkappler at gmail.com (richard kappler)
Date: Thu, 7 Jan 2016 12:14:58 -0500
Subject: [Tutor] reading an input stream
In-Reply-To: <CAHvkzykrRLB4ZSQ63pnhsWnvFC7eoyDRzSUFF1_wMDSxW97z1w@mail.gmail.com>
References: <CAG7edPHOoqWVxfYsx-KDCo7JPxdSRZKBAE_yrWeRBh9dvazTvA@mail.gmail.com>
 <CAGZAPF47Y0gfzeKpjkn3nxa4PD6RRoujnvnJpmLXugq7cx=fPg@mail.gmail.com>
 <CAG7edPFKyb1iC0ChQJAMpNRwE75nDQSM9NOakQ+As01nnutTQQ@mail.gmail.com>
 <CAHvkzykeYQ_G-8BpKEHzcr-o6Rq+a188ptuiTJwnoDpnB=B7Dg@mail.gmail.com>
 <CAG7edPEggUvwvUHpat8ovkGvRO3RudeHbz9JYKWqd2+f5Jx7Ew@mail.gmail.com>
 <CAHvkzykrRLB4ZSQ63pnhsWnvFC7eoyDRzSUFF1_wMDSxW97z1w@mail.gmail.com>
Message-ID: <CAG7edPGgixvLfeDt7ChCWAuaQMk4tA_++WM85yTYB1Qt6AQ-Yw@mail.gmail.com>

On Thu, Jan 7, 2016 at 12:07 PM, James Chapman <james at uplinkzero.com> wrote:

> ?Ah yes, OK.
>
> STX and ETX are control characters ?as you've probably worked out. Are
> there other control characters? Ideally you should have some kind of
> message header with the size of the data to be received. You'd then know
> how big the receiving buffer needs to be rather than having a fixed and
> presumably maximum size of incoming message buffer size.
>

no message header, that would make things too easy, just, as you say below,
\x02
<?xml version="1.0" encoding="utf-8"?>
<foo>
  <bar></bar>
</foo>
\x03

>
> From an architectural POV I'd have a few listener threads that upon
> receipt would spawn (or take from a pool is a better approach) a worker
> thread to process the received data.
>

That's the plan, if I'm understanding you correctly. We've brainstormed the
threading, haven't written any of it yet.

>
> Some python snippets that may/may not be of help in my github repo here
> <http://t.sidekickopen32.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJN7t5XZsd0QKqW3Mybng63BzGHW4X9JQW56dCWrf5cNnfb02?t=https%3A%2F%2Fgithub.com%2FJames-Chapman%2Fpython-code-snippets&si=5087565715079168&pi=8bcff193-2a91-49f6-ef04-f3f411fbcd45>.
> There are unfortunately no raw socket code snippets in there.
>

Thanks, I'll take a look!

>
> I'd be keen to see the raw socket data (stripped of any actual xml content
> and anything other identifying indicators) as I'm assuming there's more
> than just...
>
> \x02
> <?xml version="1.0" encoding="utf-8"?>
> <foo>
>   <bar></bar>
> </foo>
> \x03
>

See above.

regards, Richard

From sarah.a.rasco at gmail.com  Thu Jan  7 08:40:49 2016
From: sarah.a.rasco at gmail.com (Sarah Rasco)
Date: Thu, 7 Jan 2016 08:40:49 -0500
Subject: [Tutor] Syntax error
Message-ID: <CANu3AgpiqTB7OfAX2jc0VtzNWtSNP5MgeAhncVTkBoM5xgMHhg@mail.gmail.com>

Hello,

I'm new to programming and was told that Python would be a good language to
start with. I downloaded version 3.5.1, and I have Windows 10.

In IDLE, I typed print ("Hello, world!") and hit enter, and it returned the
message. I saved the file as hello.py in C:\python. Then, when I tried to
run it in IDLE, I got a syntax error and it highlighted the '5' in the
prompt 'python 3.5.1'.

I also tried to run it in my windows command prompt. I put in cd C:\python
and it gave me the python prompt. Then, when I tried to open the file by
typing python hello.py, I was given a syntax error again. Does anyone have
any suggestions as to what the problem could be?

Thank you!

From joel.goldstick at gmail.com  Thu Jan  7 13:01:15 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Thu, 7 Jan 2016 13:01:15 -0500
Subject: [Tutor] Syntax error
In-Reply-To: <CANu3AgpiqTB7OfAX2jc0VtzNWtSNP5MgeAhncVTkBoM5xgMHhg@mail.gmail.com>
References: <CANu3AgpiqTB7OfAX2jc0VtzNWtSNP5MgeAhncVTkBoM5xgMHhg@mail.gmail.com>
Message-ID: <CAPM-O+zXD159Bq4h_cz1LXwW6vGXeOsfGu6et2KhYt=Pp5V2ag@mail.gmail.com>

On Thu, Jan 7, 2016 at 8:40 AM, Sarah Rasco <sarah.a.rasco at gmail.com> wrote:

> Hello,
>
> I'm new to programming and was told that Python would be a good language to
> start with. I downloaded version 3.5.1, and I have Windows 10.
>
> In IDLE, I typed print ("Hello, world!") and hit enter, and it returned the
> message. I saved the file as hello.py in C:\python. Then, when I tried to
> run it in IDLE, I got a syntax error and it highlighted the '5' in the
> prompt 'python 3.5.1'.
>
> I also tried to run it in my windows command prompt. I put in cd C:\python
> and it gave me the python prompt. Then, when I tried to open the file by
> typing python hello.py, I was given a syntax error again. Does anyone have
> any suggestions as to what the problem could be?
>
> Thank you!
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

Welcome Sarah.

Can you copy and paste the traceback (error message) that you get when you
run your code.  Also, copy and paste your complete code text.

-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From alan.gauld at btinternet.com  Thu Jan  7 13:05:50 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 7 Jan 2016 18:05:50 +0000
Subject: [Tutor] Syntax error
In-Reply-To: <CANu3AgpiqTB7OfAX2jc0VtzNWtSNP5MgeAhncVTkBoM5xgMHhg@mail.gmail.com>
References: <CANu3AgpiqTB7OfAX2jc0VtzNWtSNP5MgeAhncVTkBoM5xgMHhg@mail.gmail.com>
Message-ID: <n6m9du$era$1@ger.gmane.org>

On 07/01/16 13:40, Sarah Rasco wrote:

> In IDLE, I typed print ("Hello, world!") and hit enter, and it returned the
> message. I saved the file as hello.py in C:\python. Then, when I tried to
> run it in IDLE, I got a syntax error and it highlighted the '5' in the
> prompt 'python 3.5.1'.

I suspect you have accidentally saved the session instead of your program.
To save a program you need to go to File->New menu and
select a Python script.
That opens a new editor window (No >>> prompt).
Type your code (the print line in this case) into the editor window.
Now use File-> SaveAs to save your file as whatever you want to
call it (hello.py in this case)
Now you can either use the Run menu or execute it from the CMD box


The File->Save from the Python shell window saves everything
you have typed and that Python has printed. That is occasionally
but not often useful. For program code you need a new editor file


HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From richkappler at gmail.com  Thu Jan  7 13:08:04 2016
From: richkappler at gmail.com (richard kappler)
Date: Thu, 7 Jan 2016 13:08:04 -0500
Subject: [Tutor] Syntax error
In-Reply-To: <CAPM-O+zXD159Bq4h_cz1LXwW6vGXeOsfGu6et2KhYt=Pp5V2ag@mail.gmail.com>
References: <CANu3AgpiqTB7OfAX2jc0VtzNWtSNP5MgeAhncVTkBoM5xgMHhg@mail.gmail.com>
 <CAPM-O+zXD159Bq4h_cz1LXwW6vGXeOsfGu6et2KhYt=Pp5V2ag@mail.gmail.com>
Message-ID: <CAG7edPGDvXwavvZ5z9vAODp06VZ-ZfKaQzb+kaE7HD_ayz0Yig@mail.gmail.com>

Hi Sarah!

instead of 'python hello.py', try

>>>import hello.py

using python hello.py works from the Linux command line (presumably Windows
as well) and it starts python then runs the hello.py script. From within
the python interpreter, you import the file and it executes.

HTH, Richard

On Thu, Jan 7, 2016 at 1:01 PM, Joel Goldstick <joel.goldstick at gmail.com>
wrote:

> On Thu, Jan 7, 2016 at 8:40 AM, Sarah Rasco <sarah.a.rasco at gmail.com>
> wrote:
>
> > Hello,
> >
> > I'm new to programming and was told that Python would be a good language
> to
> > start with. I downloaded version 3.5.1, and I have Windows 10.
> >
> > In IDLE, I typed print ("Hello, world!") and hit enter, and it returned
> the
> > message. I saved the file as hello.py in C:\python. Then, when I tried to
> > run it in IDLE, I got a syntax error and it highlighted the '5' in the
> > prompt 'python 3.5.1'.
> >
> > I also tried to run it in my windows command prompt. I put in cd
> C:\python
> > and it gave me the python prompt. Then, when I tried to open the file by
> > typing python hello.py, I was given a syntax error again. Does anyone
> have
> > any suggestions as to what the problem could be?
> >
> > Thank you!
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > https://mail.python.org/mailman/listinfo/tutor
> >
>
> Welcome Sarah.
>
> Can you copy and paste the traceback (error message) that you get when you
> run your code.  Also, copy and paste your complete code text.
>
> --
> Joel Goldstick
> http://joelgoldstick.com/stats/birthdays
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 

All internal models of the world are approximate. ~ Sebastian Thrun

From martin at linux-ip.net  Thu Jan  7 13:15:00 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Thu, 7 Jan 2016 10:15:00 -0800
Subject: [Tutor] looping generator
In-Reply-To: <CAG7edPETDnBS1RXoFio8Xa3qebT5C_PY-8Wygpx=gny-Z2+YEg@mail.gmail.com>
References: <CAG7edPETDnBS1RXoFio8Xa3qebT5C_PY-8Wygpx=gny-Z2+YEg@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1601070915270.3141@znpeba.jbaqresebt.arg>


Hi there Richard,

>I have a stream of incoming xml data. I can receive the data, parse 
>the data, etc, so long as I don't get fancy and I have a miniscule 
>delay in between each message. If I get rid of the time delay, 
>which I need to, I need the script to continuously process the 
>incoming messages. Here's what I have:

To begin, I have a suggestion that is not specifically a Python 
suggestion.  I have read several of your prior emails describing 
your problem.

If I were faced with the problem of receiving and processing data 
from remote "dumb" nodes, I would separate the software components 
into, at least, two distinct pieces.

  1. Trust the filesystem.  Write one software component that 
     receives the data from the wire and writes it out to a 
     configurable directory.  If, for whatever reason, you lose the
     data, the performance of the rest of the system does not 
     matter.  Thus, capturing the data is the most important first 
     step.  Use your system's daemonization tools to run this 
     service.

  2. Improve performance of the parsing and processing tools.
     Teach the tools how to read the data stored in the filesystem 
     and iteratively locate hot spots, performance issues, parsing 
     problems or data shortcomings.

Here are a few disorganized thoughts about why and how to do it this 
way:

  * the network listener becomes much simpler since it will not 
    parse, and will only write out to disk

  * let's assume each XML chunk is about 128k and you have 30 data 
    sources and are receiving 4 chunks per second from each; total 
    data volume is 1.5MiB, easily able to be received and written to 
    disk on modern hardware

  * you could segregate the XML chunks also by data source (and 
    maybe also time), writing out each chunk into the filesytem; if 
    you break each message into its own file, that would be a large 
    number of files (with attendant open() and close() costs), so
    perhaps writing out a new file every minute or fifteen minutes;
    here's a possible file naming scheme

        received/2016/01/0000-10.143.17.227.data
        received/2016/01/0100-10.143.17.227.data
        received/2016/01/0200-10.143.17.227.data
          ...
        received/2016/01/2300-10.143.17.227.data

    that would leave you with about 720 files per daily directory, 
    something that is eminently manageable for modern filesystems 
    (and for any pesky humans who happen to be wandering around)

  * if you write out the stream of data to the filesystem, your 
    network listener need only locate the \x02 byte and the \x03 
    byte--it could ensure that every file it wrote contained a first 
    byte of \x02 and a final byte of \x03

  * you can independently upgrade the parsing and processing tools 
    and the data recording service

  * if you retain these files, you can "replay" the past (errors, 
    bursts, reprocessing); alternatively simply delete the files 
    after they are processed for downstream consumers

  * separating the responsibilities of each software component also 
    simplifies your diagnosis and software authorship process; first 
    you can make sure that you are recording the data properly; once 
    that is done, you can start to process your data, moving along 
    to performance questions next

Now, below, I have a few Python-specific points or questions:

>#!/usr/bin/env python
>
>import socket
>import lxml.etree as ET
>
>def dataRecv(connection):
>    print 'receiving'
>    while True:
>        data = connection.recv(65536)
>        while True:
>            print "writing to data.in"
>            f2.write(data)
>            start = data.find('\x02')
>            end = data.find('\x03')
>            message = data[start+1:end]
>            print "writing to messages.out"
>            f3.write(message)
>            yield message

You do not define f2 and f3 until below.  If you are going to do 
this, pass them into the function f2 and f3.  I.e.
  
  def dataRecv(connection, f2, f3):
       ....
  
  while True:
      # wait for a connection
      connection, client_address = sock.accept()
      q = dataRecv(connection, f2, f3)
  

>def dataParse(message):
>    print 'parsing'
>    xslt = ET.parse('stack13.xsl')
>    dom = ET.XML(message)
>    transform = ET.XSLT(xslt)
>    newdom = transform(dom)
>    f1.write(str(newdom))
>
>
>sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>sock_addr = ('', 2008)
>#data = sock.makefile('r')
>sock.bind(sock_addr)
>sock.listen(5)
>print 'listening'
>
>f1 = open('parser.out', 'a')
>print "opening parser.out"
>f2 = open('data.in', 'a')
>print "opening data.in"
>f3 = open('messages.out', 'a')
>print "opening messages.out"
>
>while True:
>    # wait for a connection
>    connection, client_address = sock.accept()
>    q = dataRecv(connection)
>    dataParse(q.next())
>
># close sockrx
>#connection.close()
>
>f1.close()
>
>

By the way, keep on breaking these things into functions!  This is 
the way to go.

>In the dataRecv function, I have tried (where you see while True) 
>if data, while data and while True. Regardless, it doesn't loop, it 
>receives al ten messages from the test file being sent, but only 
>processes the first message then stops (not exits). I feel like I'm 
>missing something obvious but can't find it.

The problem starts in your dataRecv function:

    def dataRecv(connection):
        print 'receiving'
        while True:
            data = connection.recv(65536)      # -- A: receive all data
            while True:
                print "writing to data.in"
                f2.write(data)
                start = data.find('\x02')
                end = data.find('\x03')
                message = data[start+1:end]
                print "writing to messages.out"
                f3.write(message)
                yield message                  # -- B: yield one message

    while True:
        # wait for a connection
        connection, client_address = sock.accept()  # -- D: wait on network
        q = dataRecv(connection)
        dataParse(q.next())                    # -- C: process one message

Consider what happens when you read 64k bytes into the variable 
called 'data'.

  A: This probably reads all of the data at once (all ten messages).
     You then locate the first \x02 and then the following \x03.  (N.B. 
     You are also assuming that they will occur in that order in your 
     data; they might not.)
  B: Then you are happy you have identified the first message.
     You yield it, which is now handled by the dataParse function.
  C: Now, you take that message and parse it.
  D: And, we go back to sock.accept(), leaving all of that unprocessed data
     in the variable 'data' in the dataRecv function.

Specifically, your problem is about breaking the data apart and 
using it all.  You might benefit from studying techniques for 
breaking a text apart by paragraph.  Think about how this applies to 
your problem:

  http://code.activestate.com/recipes/66063-read-a-text-file-by-paragraph/#c1

N.B.  The code example may not be utterly perfect, but it is 
precisely the same problem that you are having.

Good luck and enjoy,

-Martin

-- 
Martin A. Brown
http://linux-ip.net/

From alan.gauld at btinternet.com  Thu Jan  7 13:16:20 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 7 Jan 2016 18:16:20 +0000
Subject: [Tutor] looping generator
In-Reply-To: <CAG7edPETDnBS1RXoFio8Xa3qebT5C_PY-8Wygpx=gny-Z2+YEg@mail.gmail.com>
References: <CAG7edPETDnBS1RXoFio8Xa3qebT5C_PY-8Wygpx=gny-Z2+YEg@mail.gmail.com>
Message-ID: <n6ma1k$ov1$1@ger.gmane.org>

On 07/01/16 17:02, richard kappler wrote:

> def dataRecv(connection):
>     print 'receiving'
>     while True:
>         data = connection.recv(65536)
>         while True:
>             print "writing to data.in"
>             f2.write(data)
>             start = data.find('\x02')
>             end = data.find('\x03')
>             message = data[start+1:end]
>             print "writing to messages.out"
>             f3.write(message)
>             yield message

There is something a bit weird going on here.
You are using yield to exit the function.
But the yield is inside an infinite loop with
no other break mechanism.
But that infinite loop is inside another infinite loop.
I don't see how you can ever complete the inner loop
so that it goes back to the outer loop.

Remember that when you use yield the function freezes.
Next time you call the function (in your outermost
while True loop) the function resumes just after the
yield - which is your innermost while loop.
How does it ever get back to the outer loop in
your function to read more data?

I think you need to rethink, and probably simplify,
the logic.

Or maybe I'm just missing something...

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From richkappler at gmail.com  Thu Jan  7 13:21:10 2016
From: richkappler at gmail.com (richard kappler)
Date: Thu, 7 Jan 2016 13:21:10 -0500
Subject: [Tutor] looping generator
In-Reply-To: <CAG7edPETDnBS1RXoFio8Xa3qebT5C_PY-8Wygpx=gny-Z2+YEg@mail.gmail.com>
References: <CAG7edPETDnBS1RXoFio8Xa3qebT5C_PY-8Wygpx=gny-Z2+YEg@mail.gmail.com>
Message-ID: <CAG7edPF91M1s3X7jc6CxFGytcpT6gc+obRdGfMbwyYokSYRm+A@mail.gmail.com>

We got it working, here's the code for critique and guidance on potential
improvement. All that's left to do is the threading, which seems fairly
trivial. We will spawn the threads between the     connection,
client_address = sock.accept() statement and the while True: that follows
it, putting everything under the while True: but before the f1.close() as a
function that is the thread target. Please note, during your code review,
that the socket is intentionally left open, this will run 24/7/365 and must
be listening as such. Also, f1 (parser out) is for dev only and will be
replaced with a call to splunk's event writer. I look forward to your
comments.

regards, Richard

#!/usr/bin/env python

import socket
import lxml.etree as ET

stx = '\x02'
etx = '\x03'

def waiting_for_stx(buf):
    start = buf.find(stx)
    if start >= 0:
        return (buf[start + 1:], waiting_for_etx, True)
    else:
        return ('', waiting_for_stx, False)

def waiting_for_etx(buf):
    end = buf.find(etx)
    if (end < 0):
        return (buf, waiting_for_etx, False)
    else:
        messagetoprocess = buf[:end]
        dataParse(messagetoprocess)
        return (buf[end + 1:], waiting_for_stx, True)

def dataParse(message):
    print 'parsing'
    xslt = ET.parse('stack13.xsl')
    dom = ET.XML(message)
    transform = ET.XSLT(xslt)
    newdom = transform(dom)
    f1.write(str(newdom))


sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock_addr = ('', 2008)
sock.bind(sock_addr)
sock.listen(5)
print 'listening'

f1 = open('parser.out', 'a')
print "opening parser.out"

curr_state = waiting_for_stx
buf = ''

while True:
    # wait for a connection
    connection, client_address = sock.accept()
    while True:
        data_chunks = connection.recv(8192)
        for chunk in data_chunks:
            buf += chunk
            stillProcessing = True
            while stillProcessing:
                (buf, curr_state, stillProcessing) = curr_state(buf)

f1.close()


On Thu, Jan 7, 2016 at 12:02 PM, richard kappler <richkappler at gmail.com>
wrote:

> I have a stream of incoming xml data. I can receive the data, parse the
> data, etc, so long as I don't get fancy and I have a miniscule delay in
> between each message. If I get rid of the time delay, which I need to, I
> need the script to continuously process the incoming messages. Here's what
> I have:
>
> #!/usr/bin/env python
>
> import socket
> import lxml.etree as ET
>
> def dataRecv(connection):
>     print 'receiving'
>     while True:
>         data = connection.recv(65536)
>         while True:
>             print "writing to data.in"
>             f2.write(data)
>             start = data.find('\x02')
>             end = data.find('\x03')
>             message = data[start+1:end]
>             print "writing to messages.out"
>             f3.write(message)
>             yield message
>
> def dataParse(message):
>     print 'parsing'
>     xslt = ET.parse('stack13.xsl')
>     dom = ET.XML(message)
>     transform = ET.XSLT(xslt)
>     newdom = transform(dom)
>     f1.write(str(newdom))
>
>
> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> sock_addr = ('', 2008)
> #data = sock.makefile('r')
> sock.bind(sock_addr)
> sock.listen(5)
> print 'listening'
>
> f1 = open('parser.out', 'a')
> print "opening parser.out"
> f2 = open('data.in', 'a')
> print "opening data.in"
> f3 = open('messages.out', 'a')
> print "opening messages.out"
>
> while True:
>     # wait for a connection
>     connection, client_address = sock.accept()
>     q = dataRecv(connection)
>     dataParse(q.next())
>
> # close sockrx
> #connection.close()
>
> f1.close()
>
>
> In the dataRecv function, I have tried (where you see while True) if data,
> while data and while True. Regardless, it doesn't loop, it receives al ten
> messages from the test file being sent, but only processes the first
> message then stops (not exits). I feel like I'm missing something obvious
> but can't find it.
>
> regards, Richard
>

From richkappler at gmail.com  Thu Jan  7 13:29:05 2016
From: richkappler at gmail.com (richard kappler)
Date: Thu, 7 Jan 2016 13:29:05 -0500
Subject: [Tutor] looping generator
In-Reply-To: <alpine.LSU.2.11.1601070915270.3141@znpeba.jbaqresebt.arg>
References: <CAG7edPETDnBS1RXoFio8Xa3qebT5C_PY-8Wygpx=gny-Z2+YEg@mail.gmail.com>
 <alpine.LSU.2.11.1601070915270.3141@znpeba.jbaqresebt.arg>
Message-ID: <CAG7edPGjKtRRNAiqkx1OgZCLT-H=AVYeBq2HYq3S3wji=Z=5WA@mail.gmail.com>

Martin, the suggestion you provide was the first I thought of days ago when
we first started this project, however the constraints preclude us writing
to file. I am reading everything your wrote though, and looking through the
links you provided. Your assistance is sincerely appreciated.

regards, Richard

On Thu, Jan 7, 2016 at 1:15 PM, Martin A. Brown <martin at linux-ip.net> wrote:

>
> Hi there Richard,
>
> >I have a stream of incoming xml data. I can receive the data, parse
> >the data, etc, so long as I don't get fancy and I have a miniscule
> >delay in between each message. If I get rid of the time delay,
> >which I need to, I need the script to continuously process the
> >incoming messages. Here's what I have:
>
> To begin, I have a suggestion that is not specifically a Python
> suggestion.  I have read several of your prior emails describing
> your problem.
>
> If I were faced with the problem of receiving and processing data
> from remote "dumb" nodes, I would separate the software components
> into, at least, two distinct pieces.
>
>   1. Trust the filesystem.  Write one software component that
>      receives the data from the wire and writes it out to a
>      configurable directory.  If, for whatever reason, you lose the
>      data, the performance of the rest of the system does not
>      matter.  Thus, capturing the data is the most important first
>      step.  Use your system's daemonization tools to run this
>      service.
>
>   2. Improve performance of the parsing and processing tools.
>      Teach the tools how to read the data stored in the filesystem
>      and iteratively locate hot spots, performance issues, parsing
>      problems or data shortcomings.
>
> Here are a few disorganized thoughts about why and how to do it this
> way:
>
>   * the network listener becomes much simpler since it will not
>     parse, and will only write out to disk
>
>   * let's assume each XML chunk is about 128k and you have 30 data
>     sources and are receiving 4 chunks per second from each; total
>     data volume is 1.5MiB, easily able to be received and written to
>     disk on modern hardware
>
>   * you could segregate the XML chunks also by data source (and
>     maybe also time), writing out each chunk into the filesytem; if
>     you break each message into its own file, that would be a large
>     number of files (with attendant open() and close() costs), so
>     perhaps writing out a new file every minute or fifteen minutes;
>     here's a possible file naming scheme
>
>         received/2016/01/0000-10.143.17.227.data
>         received/2016/01/0100-10.143.17.227.data
>         received/2016/01/0200-10.143.17.227.data
>           ...
>         received/2016/01/2300-10.143.17.227.data
>
>     that would leave you with about 720 files per daily directory,
>     something that is eminently manageable for modern filesystems
>     (and for any pesky humans who happen to be wandering around)
>
>   * if you write out the stream of data to the filesystem, your
>     network listener need only locate the \x02 byte and the \x03
>     byte--it could ensure that every file it wrote contained a first
>     byte of \x02 and a final byte of \x03
>
>   * you can independently upgrade the parsing and processing tools
>     and the data recording service
>
>   * if you retain these files, you can "replay" the past (errors,
>     bursts, reprocessing); alternatively simply delete the files
>     after they are processed for downstream consumers
>
>   * separating the responsibilities of each software component also
>     simplifies your diagnosis and software authorship process; first
>     you can make sure that you are recording the data properly; once
>     that is done, you can start to process your data, moving along
>     to performance questions next
>
> Now, below, I have a few Python-specific points or questions:
>
> >#!/usr/bin/env python
> >
> >import socket
> >import lxml.etree as ET
> >
> >def dataRecv(connection):
> >    print 'receiving'
> >    while True:
> >        data = connection.recv(65536)
> >        while True:
> >            print "writing to data.in"
> >            f2.write(data)
> >            start = data.find('\x02')
> >            end = data.find('\x03')
> >            message = data[start+1:end]
> >            print "writing to messages.out"
> >            f3.write(message)
> >            yield message
>
> You do not define f2 and f3 until below.  If you are going to do
> this, pass them into the function f2 and f3.  I.e.
>
>   def dataRecv(connection, f2, f3):
>        ....
>
>   while True:
>       # wait for a connection
>       connection, client_address = sock.accept()
>       q = dataRecv(connection, f2, f3)
>
>
> >def dataParse(message):
> >    print 'parsing'
> >    xslt = ET.parse('stack13.xsl')
> >    dom = ET.XML(message)
> >    transform = ET.XSLT(xslt)
> >    newdom = transform(dom)
> >    f1.write(str(newdom))
> >
> >
> >sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> >sock_addr = ('', 2008)
> >#data = sock.makefile('r')
> >sock.bind(sock_addr)
> >sock.listen(5)
> >print 'listening'
> >
> >f1 = open('parser.out', 'a')
> >print "opening parser.out"
> >f2 = open('data.in', 'a')
> >print "opening data.in"
> >f3 = open('messages.out', 'a')
> >print "opening messages.out"
> >
> >while True:
> >    # wait for a connection
> >    connection, client_address = sock.accept()
> >    q = dataRecv(connection)
> >    dataParse(q.next())
> >
> ># close sockrx
> >#connection.close()
> >
> >f1.close()
> >
> >
>
> By the way, keep on breaking these things into functions!  This is
> the way to go.
>
> >In the dataRecv function, I have tried (where you see while True)
> >if data, while data and while True. Regardless, it doesn't loop, it
> >receives al ten messages from the test file being sent, but only
> >processes the first message then stops (not exits). I feel like I'm
> >missing something obvious but can't find it.
>
> The problem starts in your dataRecv function:
>
>     def dataRecv(connection):
>         print 'receiving'
>         while True:
>             data = connection.recv(65536)      # -- A: receive all data
>             while True:
>                 print "writing to data.in"
>                 f2.write(data)
>                 start = data.find('\x02')
>                 end = data.find('\x03')
>                 message = data[start+1:end]
>                 print "writing to messages.out"
>                 f3.write(message)
>                 yield message                  # -- B: yield one message
>
>     while True:
>         # wait for a connection
>         connection, client_address = sock.accept()  # -- D: wait on network
>         q = dataRecv(connection)
>         dataParse(q.next())                    # -- C: process one message
>
> Consider what happens when you read 64k bytes into the variable
> called 'data'.
>
>   A: This probably reads all of the data at once (all ten messages).
>      You then locate the first \x02 and then the following \x03.  (N.B.
>      You are also assuming that they will occur in that order in your
>      data; they might not.)
>   B: Then you are happy you have identified the first message.
>      You yield it, which is now handled by the dataParse function.
>   C: Now, you take that message and parse it.
>   D: And, we go back to sock.accept(), leaving all of that unprocessed data
>      in the variable 'data' in the dataRecv function.
>
> Specifically, your problem is about breaking the data apart and
> using it all.  You might benefit from studying techniques for
> breaking a text apart by paragraph.  Think about how this applies to
> your problem:
>
>
> http://code.activestate.com/recipes/66063-read-a-text-file-by-paragraph/#c1
>
> N.B.  The code example may not be utterly perfect, but it is
> precisely the same problem that you are having.
>
> Good luck and enjoy,
>
> -Martin
>
> --
> Martin A. Brown
> http://linux-ip.net/
>



-- 

All internal models of the world are approximate. ~ Sebastian Thrun

From richkappler at gmail.com  Thu Jan  7 13:31:15 2016
From: richkappler at gmail.com (richard kappler)
Date: Thu, 7 Jan 2016 13:31:15 -0500
Subject: [Tutor] looping generator
In-Reply-To: <n6ma1k$ov1$1@ger.gmane.org>
References: <CAG7edPETDnBS1RXoFio8Xa3qebT5C_PY-8Wygpx=gny-Z2+YEg@mail.gmail.com>
 <n6ma1k$ov1$1@ger.gmane.org>
Message-ID: <CAG7edPHVjLH3uf6N6m5a51-uAYG-CkGC8xs11Urc9xo63M9PSA@mail.gmail.com>

Alan, have you ever actually been guilty of 'missing something'? :-) Not
that I've seen in my years on the list! We went through pretty much the
same thought process you put in your message in disecting our original code
and came up with something that works, which I sent under separate cover at
about the same time you sent this. I hope you will give us a critique on it.

regards, Richard

On Thu, Jan 7, 2016 at 1:16 PM, Alan Gauld <alan.gauld at btinternet.com>
wrote:

> On 07/01/16 17:02, richard kappler wrote:
>
> > def dataRecv(connection):
> >     print 'receiving'
> >     while True:
> >         data = connection.recv(65536)
> >         while True:
> >             print "writing to data.in"
> >             f2.write(data)
> >             start = data.find('\x02')
> >             end = data.find('\x03')
> >             message = data[start+1:end]
> >             print "writing to messages.out"
> >             f3.write(message)
> >             yield message
>
> There is something a bit weird going on here.
> You are using yield to exit the function.
> But the yield is inside an infinite loop with
> no other break mechanism.
> But that infinite loop is inside another infinite loop.
> I don't see how you can ever complete the inner loop
> so that it goes back to the outer loop.
>
> Remember that when you use yield the function freezes.
> Next time you call the function (in your outermost
> while True loop) the function resumes just after the
> yield - which is your innermost while loop.
> How does it ever get back to the outer loop in
> your function to read more data?
>
> I think you need to rethink, and probably simplify,
> the logic.
>
> Or maybe I'm just missing something...
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 

All internal models of the world are approximate. ~ Sebastian Thrun

From alan.gauld at btinternet.com  Thu Jan  7 13:39:09 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 7 Jan 2016 18:39:09 +0000
Subject: [Tutor] looping generator
In-Reply-To: <CAG7edPHVjLH3uf6N6m5a51-uAYG-CkGC8xs11Urc9xo63M9PSA@mail.gmail.com>
References: <CAG7edPETDnBS1RXoFio8Xa3qebT5C_PY-8Wygpx=gny-Z2+YEg@mail.gmail.com>
 <n6ma1k$ov1$1@ger.gmane.org>
 <CAG7edPHVjLH3uf6N6m5a51-uAYG-CkGC8xs11Urc9xo63M9PSA@mail.gmail.com>
Message-ID: <568EB0CD.3090502@btinternet.com>

On 07/01/16 18:31, richard kappler wrote:
> Alan, have you ever actually been guilty of 'missing something'? :-)

Actually quite often.
Usually when its late at night(tired) or early morning(no coffee)
or I'm rushing to go someplace.

But it happens quite a lot, Usually Steven or Peter or someone
will jump in and show me the error of my ways.  ;-)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


From richkappler at gmail.com  Thu Jan  7 14:26:21 2016
From: richkappler at gmail.com (richard kappler)
Date: Thu, 7 Jan 2016 14:26:21 -0500
Subject: [Tutor] basic threading question
Message-ID: <CAG7edPGXAXQspzgAQLV3QsgwRgDuaUsqJu+YpvyCvTwzNqCnBA@mail.gmail.com>

See previous posts on 'looping generator' for details about the code and
project.

The brief version, I am reading and parsing a data stream through a socket,
several actually. Each new connection spawns a thread that reads and
parses. Should the client close, I want the thread to terminate. Everything
I've read says you don't kill threads, but if the client closes, the socket
closes, and the thread is just sitting there, hanging.

If the socket.recv returns 0 bytes, the docs tell me that means the client
closed and therefore the server socket closes as well. If I do something in
the thread target function like:

        data_chunks = connection.recv(8192)
        if len(data_chunks) == 0:
             break

len(data_chunks)==0 tells me the socket is closed, but does the break kill
the thread? How do I prove that if it does?

regards, Richard

From akleider at sonic.net  Thu Jan  7 14:42:18 2016
From: akleider at sonic.net (Alex Kleider)
Date: Thu, 07 Jan 2016 11:42:18 -0800
Subject: [Tutor] =?utf-8?q?method=2C_type=3F?=
In-Reply-To: <20160106054035.GA20551@cskk.homeip.net>
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
 <20160106054035.GA20551@cskk.homeip.net>
Message-ID: <68ba8bcc3e673e72233336e4c881a70d@sonic.net>

Thank you to all who contributed to this thread.
It has helped me immensely and I enjoyed some of the spirited 
discussion.

Some of my notes follow (in case corrections are in order:-)

     my_notes = """

     @staticmethod
     def s_method(param_but_no_self_or_cls):
         # An ordinary function that resides in the class to associate
         # its functionality with the class.
         pass

Instance methods expect 'self'.    | arranged implicitly when called
Class methods expect 'cls'.        | via instance or class.

"factory" methods (typically called '.from_*') can be:
     1. a normal function outside the class, or
     2. a class method (would allow subclassing.)
     "alternative constructor" (what Petter Otten and Steven DAprano
     call it,) would be best placed immediately after __init__.
     Alan Gauld indicates that as initially written (without
     '@staticmethod') "it is not" ?a method/function?
     but a 'named constructor' which is not supported by Python, so
     it could be a 'class method.'  He recommends making it a factory
     function (defined at the module level, outside the class.)
     Steven DAprano calls it a Python3 regular function/ a Python2
     broken method and mentions the Descriptor protocol and how
     'methods' are initially simply functions that are then converted
     to methods (bound) as required. In my case it would be an
     'unbound' method which works in 3 but not in Python2.

Cameron Simpson indicated that putting @staticmethod above my 'method'
would be OK (although not preferred.)  Present or absent, my method
still functions the same way.
The table provided by Peter Otten (very helpful:)
-----------------------------------------------------------------
invoked with | @staticmethod  | @classmethod    | no decorator
------------------------------------------------------------------
class        | args unchanged | class as 1st arg | args unchanged
instance     | args unchanged | class as 1st arg | inst as 1st arg
-------------------------------------------------------------------
It suggests that use of the @staticmethod serves to protect one should
the 'method' be called via an instance rather than the class.  Has it
any other effect?

"""


From cs at zip.com.au  Thu Jan  7 16:52:14 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 8 Jan 2016 08:52:14 +1100
Subject: [Tutor] reading an input stream
In-Reply-To: <CAG7edPGgixvLfeDt7ChCWAuaQMk4tA_++WM85yTYB1Qt6AQ-Yw@mail.gmail.com>
References: <CAG7edPGgixvLfeDt7ChCWAuaQMk4tA_++WM85yTYB1Qt6AQ-Yw@mail.gmail.com>
Message-ID: <20160107215214.GA15037@cskk.homeip.net>

On 07Jan2016 12:14, richard kappler <richkappler at gmail.com> wrote:
>On Thu, Jan 7, 2016 at 12:07 PM, James Chapman <james at uplinkzero.com> wrote:
>> From an architectural POV I'd have a few listener threads that upon
>> receipt would spawn (or take from a pool is a better approach) a worker
>> thread to process the received data.

As would I.

>That's the plan, if I'm understanding you correctly. We've brainstormed the
>threading, haven't written any of it yet.

The code you've posted should be fine for testing a single connection.

I'd be doing 2 things to what you posted, myself:

  - use plain old .read to collect the data and assemble the XML packets

  - decouple your XML parsing from the collection and packet parsing

To the first, I suspect that when you have our packets arriving rapidly you are 
either dropping data because the data overflows your 8192 recv size or you're 
getting multiple logical packets stuffed into a buffer:

  recv #1:
    \x02xml...\x03\x02partial-xml

  recv #2:
    tail-of-previous-xml\x03\x02more-xml...

which would definitiely get your XML parser unhappy.

Instead, gather the data progressively and emit XML chunks. You've got a TCP 
stream - the TCPServer class will do an accept and handle you an _unbuffered_ 
binary stream file from which you can just .read(), ignoring any arbitrary 
"packet" sizes.  For example (totally untested) using a generator:

  def xml_extractor(fp):
    ''' Read a continuous stream of bytes from `fp`, yield bytes to be parsed 
    elsewhere. An arbitrary size of 8192 bytes is used to request more data; it 
    doesn't say anything about any underlying network packet size.
    '''
    # a (buffer, offset) pair of ungathered data
    buffer = b''
    offset = 0
    # locate start of XML chunk
    while True:
      if offset >= len(buffer):
        buffer = fp.read1(8192)
        offset = 0
        if not buffer:
          # EOF: exit generator
          return
      # examine the next byte
      b = buffer[offset]
      offset += 1
      if b == 0x02:
        # opening delimiter
        break
      warning("discard byte 0x%02x", b)
    # gather XML chunk
    chunks = []
    while True:
      endpos = buffer.find(b'\x03', offset)
      if endpos < 0:
        # no delimiter, collect entire chunk
        chunks.append(buffer[offset:])
        offset = len(buffer)
      else:
        # collect up to the delimiter
        chunks.append(buffer[offset:endpos])
        offset = endpos + 1
        break
      # keep collecting...
      if offset >= len(buffer):
        buffer = fp.read1(8192)
        offset = 0
        if not buffer:
          error("EOF: incomplete final XML packet found: %r", b''.join(chunks))
          return
    # yield the XML bytes
    yield b''.join(chunks)
    chunks = None   # release chunks so memory can be freed promptly

This reads bytes into a buffer and locates the 0x02...0x03 boundaries and 
yields the bytes in between. Then your main stream decoder just looks like 
this:

  for xml_bytes in xml_extractor(fp):
    # decode the bytes into a str
    xml_s = xml_bytes.decode('utf-8')
    ... pass xml_s to your XML parser ...

All of this presumes you have a binary file-like object reading from your TCP 
stream. And since we're suggesting you spawn a Thread per connection, I'm 
suggesting you use the TCPServer class from the socketserver module, using its 
ThreadingMixin. That gets you a threading TCP server.

Query: do the cameras connect to you, or do you connect to the cameras? I'm 
presuming the former.

So the surround framework would create a TCPServer instance listening on your 
ip:port, and have a handler method which is given a "request" parameter by 
TCPServer. That object has a .rfile property which is a read-only binary stream 
for reading from the socket, and _that_ is what we refer to as `fp` in the code 
above.

Setting up the TCPServer is pretty simple. Lifting the essential bits from some 
code of my own (again, untested):

  from socketserver import TCPServer, ThreadingMixIn, StreamRequestHandler

  class MyServer(ThreadingMixIn, TCPServer):
    def __init__(self, bind_addr):
      TCPServer.__init__(self, bind_addr, MyRequestHandler)

  class MyRequestHandler(StreamRequestHandler):
    def handle(self):
      fp = self.rfile
      for xml_bytes in xml_extractor(fp):
        # decode the bytes into a str
        xml_s = xml_bytes.decode('utf-8')
        ... pass xml_s to your XML parser ...

  # start the server
  S = MyServer( ("hostname", 9999) )
  S.serve_forever()

One critical bit in the above is the use of .read1() in the xml_extractor 
function: that calls the underlying stream's .read() method at most once, so 
that it behaves like a UNIX read() call and may return a "short" read - less 
than the maximum supplied. This is what you need to return data as soon as it 
is received. By contrast, the traditional Python .read() call will try to 
gather bytes until it has the amount asked for, which means that it will block.  
You definitely need read1() for this kind of work.

Cheers,
Cameron Simpson <cs at zip.com.au>

From cs at zip.com.au  Thu Jan  7 17:07:58 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 8 Jan 2016 09:07:58 +1100
Subject: [Tutor] reading an input stream
In-Reply-To: <20160107215214.GA15037@cskk.homeip.net>
References: <20160107215214.GA15037@cskk.homeip.net>
Message-ID: <20160107220758.GA76724@cskk.homeip.net>

On 08Jan2016 08:52, Cameron Simpson <cs at zip.com.au> wrote:
[...]
>Instead, gather the data progressively and emit XML chunks. You've got a TCP 
>stream - the TCPServer class will do an accept and handle you an _unbuffered_ 
>binary stream file from which you can just .read(), ignoring any arbitrary 
>"packet" sizes.  For example (totally untested) using a generator:
[...]

Just a few followup remarks:

This is all Python 3, where bytes and strings are cleanly separated. You've got 
a binary stream with binary delimiters, so we're reading binary data and 
returning the binary XML in between. We separately decode this into a string 
for handing to your XML parser. Just avoid Python 2 altogether; this can all be 
done in Python 2 but it is not as clean, and more confusing.

The socketserver module is... annoyingly vague about what the .rfile property 
gets you. It says a "a file-like object". That should be a nice io.BytesIO 
subclass with a .read1() method, but conceivably it is not. I'm mentioning this 
because I've noticed that the code I lifted the TCPServer setup from seems to 
make a BytesIO from whole cloth by doing:

  fp = os.fdopen(os.dup(request.fileno()),"rb")

You'd hope that isn't necessary here, and that request.rfile is a nice BytesIO 
already.

In xml_extractor, the "# locate start of XML chunk" loop could be better by 
using .find exactly as in the "# gather XML chunk"; I started with .read(1) 
instead of .read1(8192), which is why it does things byte by byte.

Cheers,
Cameron Simpson <cs at zip.com.au>

From alan.gauld at btinternet.com  Thu Jan  7 17:08:00 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 7 Jan 2016 22:08:00 +0000
Subject: [Tutor] basic threading question
In-Reply-To: <CAG7edPGXAXQspzgAQLV3QsgwRgDuaUsqJu+YpvyCvTwzNqCnBA@mail.gmail.com>
References: <CAG7edPGXAXQspzgAQLV3QsgwRgDuaUsqJu+YpvyCvTwzNqCnBA@mail.gmail.com>
Message-ID: <n6mnk1$1ph$1@ger.gmane.org>

On 07/01/16 19:26, richard kappler wrote:

> several actually. Each new connection spawns a thread that reads and
> parses. 

First question. Do you need to read AND parse in the thread. Could you
not read the raw data and send that to a parsing thread? Usually reading
the data won't be a problem (especially if its all coming through the
same serial connection as you seem to suggest).

> Should the client close, I want the thread to terminate. 

Second: I've lost the thread (pardon the pun) on your architecture.
Which process is the client? Is that the thing sending the
messages to your code? Or is it the process spawning the threads?

> I've read says you don't kill threads, 

I'm no threading expert, and especially in Python. But I thought
there was some kind of "kill all threads" signal that could be
called?

> If the socket.recv returns 0 bytes, the docs tell me that means the client
> closed and therefore the server socket closes as well. 

Yes, but that's the secondary socket set up post the initial
connection. Are you connecting to the server(and thus are
the client?) or are you the server that the cameras(?) are
connecting to?

Hopefully others have been following this more closely, but I
need a wee reminder of the architecture. Which process is
serving what, and which is connecting, and sending what?

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From richkappler at gmail.com  Thu Jan  7 17:22:58 2016
From: richkappler at gmail.com (richard kappler)
Date: Thu, 7 Jan 2016 17:22:58 -0500
Subject: [Tutor] reading an input stream
In-Reply-To: <20160107220758.GA76724@cskk.homeip.net>
References: <20160107215214.GA15037@cskk.homeip.net>
 <20160107220758.GA76724@cskk.homeip.net>
Message-ID: <CAG7edPERtS-8B6_f7SgCCmPYmHOBmEeRuxiAaRSc3x7Zp5br=g@mail.gmail.com>

On Thu, Jan 7, 2016 at 5:07 PM, Cameron Simpson <cs at zip.com.au> wrote:

>
> Just a few followup remarks:
>
> This is all Python 3, where bytes and strings are cleanly separated.
> You've got a binary stream with binary delimiters, so we're reading binary
> data and returning the binary XML in between. We separately decode this
> into a string for handing to your XML parser. Just avoid Python 2
> altogether; this can all be done in Python 2 but it is not as clean, and
> more confusing.
>

Love to, can't. Splunk uses 2.7 so that's what we have to work with. That
will not change in the forseeable future. Doing other homework right now,
but will more closely review this and the other posts that have come in
since I left work later tonight or first thing in the morning.

regards, Richard

From cs at zip.com.au  Thu Jan  7 17:25:40 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 8 Jan 2016 09:25:40 +1100
Subject: [Tutor] reading an input stream
In-Reply-To: <CAG7edPERtS-8B6_f7SgCCmPYmHOBmEeRuxiAaRSc3x7Zp5br=g@mail.gmail.com>
References: <CAG7edPERtS-8B6_f7SgCCmPYmHOBmEeRuxiAaRSc3x7Zp5br=g@mail.gmail.com>
Message-ID: <20160107222540.GA6813@cskk.homeip.net>

On 07Jan2016 17:22, richard kappler <richkappler at gmail.com> wrote:
>On Thu, Jan 7, 2016 at 5:07 PM, Cameron Simpson <cs at zip.com.au> wrote:
>
>>
>> Just a few followup remarks:
>>
>> This is all Python 3, where bytes and strings are cleanly separated.
>> You've got a binary stream with binary delimiters, so we're reading binary
>> data and returning the binary XML in between. We separately decode this
>> into a string for handing to your XML parser. Just avoid Python 2
>> altogether; this can all be done in Python 2 but it is not as clean, and
>> more confusing.
>>
>
>Love to, can't. Splunk uses 2.7 so that's what we have to work with. That
>will not change in the forseeable future. Doing other homework right now,
>but will more closely review this and the other posts that have come in
>since I left work later tonight or first thing in the morning.

Ok. You should still be ok, but things like bs[0] == 0x02 will need to be bs[0] 
== '\x02' and so forth, because you get str objects back from reads.

The rest of the suggested code should still broadly work.

Cheers,
Cameron Simpson <cs at zip.com.au>

From cs at zip.com.au  Thu Jan  7 17:30:09 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 8 Jan 2016 09:30:09 +1100
Subject: [Tutor] basic threading question
In-Reply-To: <CAG7edPGXAXQspzgAQLV3QsgwRgDuaUsqJu+YpvyCvTwzNqCnBA@mail.gmail.com>
References: <CAG7edPGXAXQspzgAQLV3QsgwRgDuaUsqJu+YpvyCvTwzNqCnBA@mail.gmail.com>
Message-ID: <20160107223009.GA65418@cskk.homeip.net>

On 07Jan2016 14:26, richard kappler <richkappler at gmail.com> wrote:
>See previous posts on 'looping generator' for details about the code and
>project.
>
>The brief version, I am reading and parsing a data stream through a socket,
>several actually. Each new connection spawns a thread that reads and
>parses. Should the client close, I want the thread to terminate. Everything
>I've read says you don't kill threads, but if the client closes, the socket
>closes, and the thread is just sitting there, hanging.
>
>If the socket.recv returns 0 bytes, the docs tell me that means the client
>closed and therefore the server socket closes as well. If I do something in
>the thread target function like:
>
>        data_chunks = connection.recv(8192)
>        if len(data_chunks) == 0:
>             break
>
>len(data_chunks)==0 tells me the socket is closed, but does the break kill
>the thread? How do I prove that if it does?

The break doesn't kill the thread, but if the recv loop is all your thread's 
main function does, then sure: when the function exits, the thread exits. For 
example:

  T = Thread(target=gather_func)
  T.start()
  ...

  def gather_func():
    while True:
      data_chunks = connection.recv(8192)
      if len(data_chunks) == 0:
        break
      ... do stuff with data_chunks ...

Then you break exits the loop. Then the function returns since there's no more 
code in the function after the loop. And then the Thread exits because the 
function it was running has finished. There is no need to kill the Thread here.

You can wait for the Thread with T.wait() and probe it with T.is_alive(); see 
the threading module documentation.

BTW, I still recommend you drop use of .recv() and use .read1() to assemble 
packets. See your main discussion.

Cheers,
Cameron Simpson <cs at zip.com.au>

From steve at pearwood.info  Thu Jan  7 18:19:57 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 8 Jan 2016 10:19:57 +1100
Subject: [Tutor] method, type?
In-Reply-To: <68ba8bcc3e673e72233336e4c881a70d@sonic.net>
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
 <20160106054035.GA20551@cskk.homeip.net>
 <68ba8bcc3e673e72233336e4c881a70d@sonic.net>
Message-ID: <20160107231939.GG10854@ando.pearwood.info>

On Thu, Jan 07, 2016 at 11:42:18AM -0800, Alex Kleider wrote:
> Thank you to all who contributed to this thread.
> It has helped me immensely and I enjoyed some of the spirited 
> discussion.
> 
> Some of my notes follow (in case corrections are in order:-)
> 
>     my_notes = """
> 
>     @staticmethod
>     def s_method(param_but_no_self_or_cls):
>         # An ordinary function that resides in the class to associate
>         # its functionality with the class.
>         pass

s_method is not a function, but a static method, since you have used the 
staticmethod decorator.

(And good news: the term "static method" in Python has *nothing* to do 
with the same term as used in Java and C++. Yay.)

If you are confused, it's not surprising, as you are playing on the 
edges of Python's internals where most programmers fear to tread :-)

Ultimately, the final arbiter of "what is this thing?" is to ask type. 
So let's have a look at what happens when we create a method inside a 
class, using Python 3:

py> class Test:
...     def foo():  # intentionally given no parameters
...             pass
...     print("during class definition time", type(foo))
...
during class definition time <class 'function'>
py> print("access from the class", type(Test.foo))
access from the class <class 'function'>
py> print("access from the instance", type(Test().foo))
access from the instance <class 'method'>


So while the class is still being defined, "foo" is an ordinary 
function. def ALWAYS creates a function, no exceptions.

When you access it via the class, Test.foo, you also get a function. 
(This bit is new to Python 3 -- previously, you got an unbound method 
object, which is essentially like a function but with restrictions on 
the first argument.)

When you access it via the instance, Test().foo, you get a method. 
What's happening here? This is the magic of the descriptor protocol, 
which you don't have to understand (consider it a bit more advanced than 
classes, but less advanced than metaclasses). For now, just think if it 
like this:

When you access instance.foo, Python creates a method object that 
automatically knows the instance it was called from, and the function 
it was built from, so that it can automatically provide the "self" 
parameter when you call the method.


What happens when you use the staticmethod (or classmethod) decorator?

py> class Test:
...     @staticmethod
...     def foo():
...             pass
...     print(type(foo))
...
<class 'staticmethod'>

So right from the word go, the function foo gets converted to a 
staticmethod object. Now for one of Python's worst-kept secrets:

staticmethod only exists to prevent Python from automatically converting 
functions into (regular) methods when you access them from the instance.

Remember how I showed that instance.foo takes the underlying foo 
function, converts it into a method, and binds the instance as the first 
parameter "self"? To prevent that behaviour, you have to fool Python 
into thinking that foo is not a function. The way to do that is to turn 
it into a staticmethod, which is (almost) exactly like a function but 
doesn't have the magic turn-into-a-method behaviour of functions.

Although there are slight differences, you should consider that static 
methods in Python are semantically equivalent to top-level functions 
except that they live inside a class.

Since staticmethods don't know what class they come from, there are very 
few reasons to justify using a staticmethod. At one point, Guido wrote 
that there were no known uses for staticmethod outside of Python's test 
suite. Most times you think you want to use staticmethod, chances are 
high that you either want a class method or a global function.

 
> Instance methods expect 'self'.    | arranged implicitly when called
> Class methods expect 'cls'.        | via instance or class.

Correct.


> "factory" methods (typically called '.from_*') can be:
>     1. a normal function outside the class, or
>     2. a class method (would allow subclassing.)

"Factory methods" just means a method which you, the creator or author, 
thinks of as a factory. What's a factory? A function or method which 
takes a bunch of arguments and creates something useful.

It's a pretty vague definition, because it's a pretty vague term. The 
idea is, you have things that you use directly (ints, strings, lists, 
widgets...) and you have things which you use to create those things. 
That's a factory.

In my opinion, the *only* use of the term "factory" which adds more than 
it takes away in confusion is "factory function", meaning a function 
which creates and returns a new function. But that's not what we're 
talking about here. In this case, I don't think the term "factory" is 
useful. It just makes things seem more "Enterprisey" and Java-esque than 
it need be.

A little bit less vague is the idea of a method inside a class which 
creates an instance of that same class. That's a constructor. The 
default constructor in Python is the combination of special methods 
__new__ and __init__, so any other such method (typically called 
"from_something") is an "alternate constructor".

As for our discussions of "named constructors", I don't think they're 
relevant because they're all named constructors in Python. (With the 
possible exception of the default, depending on what you mean by a named 
constructor.)


>     "alternative constructor" (what Petter Otten and Steven DAprano
>     call it,) 

Alternative in the sense of "not the default", that is all.


>     would be best placed immediately after __init__.

*shrug* 

It doesn't matter where you put it inside the class. That is entirely a 
matter of personal taste.


>     Alan Gauld indicates that as initially written (without
>     '@staticmethod') "it is not" ?a method/function?

With respect to Alan, I think he is factually wrong. As I said, the 
final arbiter of what a thing is is the type() function, and that tells 
us that if you don't use a decorator, def always creates a function.

(Technically even if you use a decorator that is still true, but it's 
just harder to spot.)


>     but a 'named constructor' which is not supported by Python, 

But they are :-)


>     so it could be a 'class method.'

But it isn't :-)


>     He recommends making it a factory
>     function (defined at the module level, outside the class.)

That's a matter of personal taste, and one which I happen to disagree 
with. Look at the design of the built-in classes like dict. We have 
dict.fromkeys(), not a global fromkeys() function.


>     Steven DAprano calls it a Python3 regular function/ a Python2
>     broken method and mentions the Descriptor protocol and how
>     'methods' are initially simply functions that are then converted
>     to methods (bound) as required. In my case it would be an
>     'unbound' method which works in 3 but not in Python2.
> 
> Cameron Simpson indicated that putting @staticmethod above my 'method'
> would be OK (although not preferred.)  Present or absent, my method
> still functions the same way.

Only because you're just calling it from the class. As soon as you 
create an instance and call the method from that, you'll see why it 
is broken :-)



> The table provided by Peter Otten (very helpful:)
> -----------------------------------------------------------------
> invoked with | @staticmethod  | @classmethod    | no decorator
> ------------------------------------------------------------------
> class        | args unchanged | class as 1st arg | args unchanged
> instance     | args unchanged | class as 1st arg | inst as 1st arg
> -------------------------------------------------------------------
> It suggests that use of the @staticmethod serves to protect one should
> the 'method' be called via an instance rather than the class.  Has it
> any other effect?

Yes, to confuse people into thinking they should be using staticmethod 
when what they really should use is classmethod :-)



-- 
Steve

From alan.gauld at btinternet.com  Thu Jan  7 19:18:03 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 8 Jan 2016 00:18:03 +0000
Subject: [Tutor] method, type?
In-Reply-To: <20160107231939.GG10854@ando.pearwood.info>
References: <333654cd965fbd66934c6597ac37932d@sonic.net>
 <20160106054035.GA20551@cskk.homeip.net>
 <68ba8bcc3e673e72233336e4c881a70d@sonic.net>
 <20160107231939.GG10854@ando.pearwood.info>
Message-ID: <n6mv7r$aej$1@ger.gmane.org>

On 07/01/16 23:19, Steven D'Aprano wrote:

> "Factory methods" just means a method which you, the creator or author, 
> thinks of as a factory. What's a factory? A function or method which 
> takes a bunch of arguments and creates something useful.

In classic OOP a factory method is more specific than that. It's
a method that returns an instance and is part of the language
definition in several older OOP languages - Objective C being
the best known.

A factory function however is any old function that returns
any old kind of object/record/function or whatever.

> more "Enterprisey" and Java-esque than it need be.

It pre-dates Java by a long way and has little to do with
anything enterprisey. (Except that enterprises tend to
build/own real physical factories! :-)

>>     Alan Gauld indicates that as initially written (without
>>     '@staticmethod') "it is not" ?a method/function?
> 
> With respect to Alan, I think he is factually wrong. 

Me too. Until your first post about this I hadn't realized
that v3 makes them all functions.

So I saw the original code as being a method definition
(it had a 'self' parameter, but it was treated as text
which would cause it to fail if used as a method. And
because it was defined as a method it couldn't be used
as a function.) It seems that's no longer true in v3 and
I'll need to do some playing around to better understand
the significance of the change.


>>     He recommends making it a factory
>>     function (defined at the module level, outside the class.)
> 
> That's a matter of personal taste, and one which I happen to disagree 

It was based on what I perceived as the Python idiomatic style.
It seems there are more classmethod factories around than
I thought so, on that basis, I'd revert to classmethod as
the preferred style.

>> The table provided by Peter Otten (very helpful:)
>> -----------------------------------------------------------------
>> invoked with | @staticmethod  | @classmethod    | no decorator
>> ------------------------------------------------------------------
>> class        | args unchanged | class as 1st arg | args unchanged
>> instance     | args unchanged | class as 1st arg | inst as 1st arg
>> -------------------------------------------------------------------

I missed this one somehow. Nice table Peter.

My only point of difference here, I think, is the definition
of a constructor. I consider a constructor to be the creator
of object instances, which makes the only default Python
constructor the __new__() since the __init__() is only an
initializer. And most factory methods simply use the
default __new__() for construction then provide alternative
initialization. However, there was one post that suggested that
__new__() could be bypassed - although it still seemed to
rely on an explicit call to object.__new__() So I need to
do yet more reading on that score too.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Thu Jan  7 19:23:16 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 8 Jan 2016 00:23:16 +0000
Subject: [Tutor] basic threading question
In-Reply-To: <CAG7edPGXAXQspzgAQLV3QsgwRgDuaUsqJu+YpvyCvTwzNqCnBA@mail.gmail.com>
References: <CAG7edPGXAXQspzgAQLV3QsgwRgDuaUsqJu+YpvyCvTwzNqCnBA@mail.gmail.com>
Message-ID: <n6mvhl$f0n$1@ger.gmane.org>

On 07/01/16 19:26, richard kappler wrote:

> The brief version, I am reading and parsing a data stream through a socket,
> several actually. Each new connection spawns a thread that reads and
> parses. Should the client close, I want the thread to terminate. 

Just another thought. Have you looked at asyncore yet? It seems
ideally suited to your usecase and avoids all the threading
problems (or more accurately lets Python deal with it invisibly)

I mentioned it right at the start of your project but haven't
brought it up again. Now seems like a good time. It's a bit
like Node.JS for Python. It allows you to add jobs to an
asynchronous event loop which then processes those events
in the background (using a thread pool I believe). It looks
like a good match to your use case.

The docs provide several examples and there are several
tutorials online.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Thu Jan  7 19:27:07 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 8 Jan 2016 00:27:07 +0000
Subject: [Tutor] basic threading question
In-Reply-To: <n6mvhl$f0n$1@ger.gmane.org>
References: <CAG7edPGXAXQspzgAQLV3QsgwRgDuaUsqJu+YpvyCvTwzNqCnBA@mail.gmail.com>
 <n6mvhl$f0n$1@ger.gmane.org>
Message-ID: <n6mvor$f0n$2@ger.gmane.org>

On 08/01/16 00:23, Alan Gauld wrote:

> Just another thought. Have you looked at asyncore yet? It seems
> ideally suited to your usecase and avoids all the threading
> problems (or more accurately lets Python deal with it invisibly)

OK, I just saw another message in another thread that says you are stuck
on 2.7. In that case forget asyncore it's 3.4+ only...

Pity.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From cs at zip.com.au  Thu Jan  7 19:54:13 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 8 Jan 2016 11:54:13 +1100
Subject: [Tutor] method, type?
In-Reply-To: <20160107231939.GG10854@ando.pearwood.info>
References: <20160107231939.GG10854@ando.pearwood.info>
Message-ID: <20160108005413.GA86260@cskk.homeip.net>

On 08Jan2016 10:19, Steven D'Aprano <steve at pearwood.info> wrote:
>[...]
>> "factory" methods (typically called '.from_*') can be:

Maybe I should have said "often" instead of "typically", if I said "typically".  
I think they read well that way and there are several stdlib functions named 
this way as a precedent. I'm aiming for the notion "make a Foo from this or 
that or something else (3 distinct methods/functions, all taking arguments a 
bit different from the core __init__)".

>>     1. a normal function outside the class, or
>>     2. a class method (would allow subclassing.)
>
>"Factory methods" just means a method which you, the creator or author,
>thinks of as a factory. What's a factory? A function or method which
>takes a bunch of arguments and creates something useful.
>
>It's a pretty vague definition, because it's a pretty vague term.

It's mostly vague if you take the "everything is an object in Python" stance.  
Which is strictly true, but it is often useful to be thinking of a factory 
function as an alternative to the bare ClassName(args-for-__init__) 
constructor, where those arguments are not convenient. Such as Alex's 
JournalLineItem construction from a line of text.

[...]
>>     "alternative constructor" (what Petter Otten and Steven DAprano
>>     call it,)
>
>Alternative in the sense of "not the default", that is all.
>
>>     would be best placed immediately after __init__.
>
>*shrug*
>It doesn't matter where you put it inside the class. That is entirely a
>matter of personal taste.

I suggested this as a style thing (of course, inherently a matter of personal 
taste:-) I like functions with the same purpose to be textually close together.

[...]
>>     [Alan Gauld] recommends making it a factory
>>     function (defined at the module level, outside the class.)
>
>That's a matter of personal taste, and one which I happen to disagree
>with. Look at the design of the built-in classes like dict. We have
>dict.fromkeys(), not a global fromkeys() function.

I'm largely with Steven here rather than Alan, partly because a classmethod 
subclasses nicely (with all the caveats Alan alluded to - if you do this then 
your subclasses _may_ need to override the extra constructor just as they may 
need to override other methods), and partly because it keeps the constructor 
inside the class definition, which I find conceptually tidier.

>>     Steven DAprano calls it a Python3 regular function/ a Python2
>>     broken method and mentions the Descriptor protocol and how
>>     'methods' are initially simply functions that are then converted
>>     to methods (bound) as required. In my case it would be an
>>     'unbound' method which works in 3 but not in Python2.
>>
>> Cameron Simpson indicated that putting @staticmethod above my 'method'
>> would be OK (although not preferred.)  Present or absent, my method
>> still functions the same way.
>
>Only because you're just calling it from the class. As soon as you
>create an instance and call the method from that, you'll see why it
>is broken :-)

Aye. While we're on what staticmethod and classmethod accomplish, we could stop 
treating them like magic. Have you (alex) written any decorators? They are 
functions which accept a function definition and return a wrapper function with 
tweaked behaviour. So the notation:

  @foo
  def func1(blah):

defines "func1" and then calls "foo(func1)". "foo" returns a new function 
definition, and the class binds _that_ definition to its "func1" method.

So...

>> The table provided by Peter Otten (very helpful:)
>> -----------------------------------------------------------------
>> invoked with | @staticmethod  | @classmethod    | no decorator
>> ------------------------------------------------------------------
>> class        | args unchanged | class as 1st arg | args unchanged
>> instance     | args unchanged | class as 1st arg | inst as 1st arg
>> -------------------------------------------------------------------
>> It suggests that use of the @staticmethod serves to protect one should
>> the 'method' be called via an instance rather than the class.  Has it
>> any other effect?
>
>Yes, to confuse people into thinking they should be using staticmethod
>when what they really should use is classmethod :-)

Now consider what @staticmethod achieves: it causes a normal method to be 
called as though it were a global function i.e. without the normally implied 
"self" parameter. So we could write our own:

  def staticmethod(func):
    def method(self, *a, **kw):
      return func(*a, **kw)
  return method

As described above, this effectively installs the "method" function as the 
class's actual method, and that function's whole purpose is simply to _discard_ 
the self parameter and call the original function without "self".

Once that makes sense, you can them imagine writing @classmethod similarly:

  def classmethod(func):
    def method(self, *a, **kw):
      return func(type(self), *a, **kw)
  return method

This version discards "self" but passes in its type (== its class).

Now, both of these examples above are actually simplifications of what Python's 
inbuilt @staticmethod and @classmethod decorators do but they show that this 
isn't magic: it has simple and concrete actions with well defined effects.

Cheers,
Cameron Simpson <cs at zip.com.au>

From cs at zip.com.au  Thu Jan  7 20:02:44 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 8 Jan 2016 12:02:44 +1100
Subject: [Tutor] method, type?
In-Reply-To: <n6mv7r$aej$1@ger.gmane.org>
References: <n6mv7r$aej$1@ger.gmane.org>
Message-ID: <20160108010244.GA83336@cskk.homeip.net>

On 08Jan2016 00:18, ALAN GAULD <alan.gauld at btinternet.com> wrote:
>My only point of difference here, I think, is the definition
>of a constructor. I consider a constructor to be the creator
>of object instances, which makes the only default Python
>constructor the __new__() since the __init__() is only an
>initializer.

Me too. I was deliberately avoiding the term "constructor", then let myself get 
sucked into using it just now because you get a new instance of your target 
class out of the factory function/method. But they are better thought of as 
wrappers for the class' real constructor.

For this sloppiness, my apologies,
Cameron Simpson <cs at zip.com.au>

From sarah.a.rasco at gmail.com  Thu Jan  7 13:15:42 2016
From: sarah.a.rasco at gmail.com (Sarah Rasco)
Date: Thu, 7 Jan 2016 13:15:42 -0500
Subject: [Tutor] Syntax error
In-Reply-To: <CANu3AgpiqTB7OfAX2jc0VtzNWtSNP5MgeAhncVTkBoM5xgMHhg@mail.gmail.com>
References: <CANu3AgpiqTB7OfAX2jc0VtzNWtSNP5MgeAhncVTkBoM5xgMHhg@mail.gmail.com>
Message-ID: <CANu3AgqeB0V0mRvqJbO-A5S9xNA4pJbJ2qfEuuj1UE5rTGmgXw@mail.gmail.com>

Alan - I realized I did that right after I sent this email. However, I
can't run it in the windows or python prompts.

Here is my document:
[image: Inline image 1]
When I try to run it in the windows prompt I get:
[image: Inline image 2]

Or in the python prompt:
[image: Inline image 3]

And the file is definitely in there...
[image: Inline image 4]



On Thu, Jan 7, 2016 at 8:40 AM, Sarah Rasco <sarah.a.rasco at gmail.com> wrote:

> Hello,
>
> I'm new to programming and was told that Python would be a good language
> to start with. I downloaded version 3.5.1, and I have Windows 10.
>
> In IDLE, I typed print ("Hello, world!") and hit enter, and it returned
> the message. I saved the file as hello.py in C:\python. Then, when I tried
> to run it in IDLE, I got a syntax error and it highlighted the '5' in the
> prompt 'python 3.5.1'.
>
> I also tried to run it in my windows command prompt. I put in cd C:\python
> and it gave me the python prompt. Then, when I tried to open the file by
> typing python hello.py, I was given a syntax error again. Does anyone have
> any suggestions as to what the problem could be?
>
> Thank you!
>

From cs at zip.com.au  Thu Jan  7 21:46:02 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 8 Jan 2016 13:46:02 +1100
Subject: [Tutor] Syntax error
In-Reply-To: <CANu3AgqeB0V0mRvqJbO-A5S9xNA4pJbJ2qfEuuj1UE5rTGmgXw@mail.gmail.com>
References: <CANu3AgqeB0V0mRvqJbO-A5S9xNA4pJbJ2qfEuuj1UE5rTGmgXw@mail.gmail.com>
Message-ID: <20160108024602.GA90210@cskk.homeip.net>

On 07Jan2016 13:15, Sarah Rasco <sarah.a.rasco at gmail.com> wrote:
>Alan - I realized I did that right after I sent this email. However, I
>can't run it in the windows or python prompts.
>
>Here is my document:
>[image: Inline image 1]
>When I try to run it in the windows prompt I get:
>[image: Inline image 2]
[...]

Hi Sarah,

The tutor list and the main python-list strips attachments, so we cannot see 
your images.  Please cut/paste the text as text instead of using screenshots.

>> I also tried to run it in my windows command prompt. I put in cd C:\python
>> and it gave me the python prompt.

I think you're misreading the Windows command prompt which recites the current 
folder. This:

  C:\python>

is still the Window command prompt. It is also probably not where you should do 
your python work unless you personally made this folder specially. Where did 
you save your "hello.py" file? That is probably where you should cd.

>>Then, when I tried to open the file by
>> typing python hello.py, I was given a syntax error again. Does anyone have
>> any suggestions as to what the problem could be?

Please cut/paste the text of your hello.py file and a complete transcript of 
the syntax error (all the lines).

Thanks,
Cameron Simpson <cs at zip.com.au>

From steve at pearwood.info  Thu Jan  7 23:24:16 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 8 Jan 2016 15:24:16 +1100
Subject: [Tutor] method, type?
In-Reply-To: <20160108010244.GA83336@cskk.homeip.net>
References: <n6mv7r$aej$1@ger.gmane.org>
 <20160108010244.GA83336@cskk.homeip.net>
Message-ID: <20160108042416.GH10854@ando.pearwood.info>

On Fri, Jan 08, 2016 at 12:02:44PM +1100, Cameron Simpson wrote:
> On 08Jan2016 00:18, ALAN GAULD <alan.gauld at btinternet.com> wrote:
> >My only point of difference here, I think, is the definition
> >of a constructor. I consider a constructor to be the creator
> >of object instances, which makes the only default Python
> >constructor the __new__() since the __init__() is only an
> >initializer.
> 
> Me too. I was deliberately avoiding the term "constructor", then let myself 
> get sucked into using it just now because you get a new instance of your 
> target class out of the factory function/method. But they are better 
> thought of as wrappers for the class' real constructor.

I maintain that "constructor" for the most part has to be understood of 
a statement of intention, not a hard definition. Apart from __new__ 
itself, which genuinely is special, "constructor" in a language like 
Python refers to the intention of creating new instances, as opposed to 
"doing some work".

For example, consider something like str.upper(). It returns a new 
string object. Does that make it a constructor? No, because the intent 
is to do work on a string (convert it to uppercase), and creating a new 
string is just a means to an end.

As I showed in a previous email, it is not necessary for a constructor 
to call __new__. It can do the work of creating a new instance itself, 
if the author chooses. But generally, that's just duplicating effort.

And likewise, even though __new__ is special, it doesn't have to be used 
as a constructor. Python supports classes with no default constructor, 
or one that returns something other than an instance of the class. Just 
arrange matters for __new__ to return something other than an instance 
of its own class:

class MyClass:
    def __new__(cls):
        return "Surprise!"

Surprising though this is, it is actually allowed, and the Python 
interpreter takes special care to ensure that it works correctly.

(In detail, when you call a class x = MyClass(), that calls __new__. If 
__new__ returns an instance of MyClass, Python then automatically calls 
__init__ as well. But if it is an instance of something else, then 
Python skips calling __init__ so as to prevent it being called twice.)

So my rule of thumb is, assuming that there's nothing funny going on:

(1) __new__ is the default constructor;

(2) __init__ is the initialiser, but sometimes we're lazy and call it 
the constructor;

(3) If you see a CLASS method called "from_foo" or similar, which 
returns an instance of the class, that's probably an alternate 
(non-default) constructor;

(4) But we usually don't count methods which take an instance of the 
class, and transform or copy them in some way as "constructors" even if 
they do in fact construct a new instance.



-- 
Steve

From alan.gauld at btinternet.com  Fri Jan  8 05:09:25 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 8 Jan 2016 10:09:25 +0000
Subject: [Tutor] method, type?
In-Reply-To: <20160108042416.GH10854@ando.pearwood.info>
References: <n6mv7r$aej$1@ger.gmane.org>
 <20160108010244.GA83336@cskk.homeip.net>
 <20160108042416.GH10854@ando.pearwood.info>
Message-ID: <n6o1sl$edg$1@ger.gmane.org>

On 08/01/16 04:24, Steven D'Aprano wrote:

> I maintain that "constructor" for the most part has to be understood of 
> a statement of intention, not a hard definition. Apart from __new__ 
> itself, which genuinely is special, "constructor" in a language like 
> Python refers to the intention of creating new instances, as opposed to 
> "doing some work".

I think the question here is whether we are talking about the
specifics of how Python does things or about the meaning of
the terms in the wider OOP community. I'd argue that in wider OOP
circles constructor is a much more specific term but in Python
it is more about intent. (The init case is a good example where
we generally, and sloppily, refer to init as a constructor when
in fact its only an initialiser)

>From the point of view of the tutor list we have a slight dilemma
because we are here to teach both Python the language as well as
wider programming skills.

A thread like this exposes where Python's specific way of doing
things is at odds with the wider practice and terminology and
probably confuses noobies (sorry folks) but at the same time
throws up a lot of interesting details about just how Python
does its particular brand of OOP magic.

> class MyClass:
>     def __new__(cls):
>         return "Surprise!"
> 
> Surprising though this is, it is actually allowed,

And again this is one of the differences between Python and
the vast majority of OOP languages.  In most cases constructors
are not allowed to explicitly return anything because the language
implicitly returns an instance (or occasionally throws an
exception). But that's because those languages have a very
specific meaning for constructor, often with different syntax
or idioms from normal methods. Whereas, as you've pointed
out, Python's construction mechanism is much more open and
consistent making the meaning of 'constructor' more loosely
defined.

> (1) __new__ is the default constructor;
> 
> (2) __init__ is the initialiser, but sometimes we're lazy and call it 
> the constructor;
> 
> (3) If you see a CLASS method called "from_foo" or similar, which 
> returns an instance of the class, that's probably an alternate 
> (non-default) constructor;
> 
> (4) But we usually don't count methods which take an instance of the 
> class, and transform or copy them in some way as "constructors" even if 
> they do in fact construct a new instance.

Yep. I'll buy that for when we are talking about Python
In the wider OOP world there are more definite distinctions
in the terms.

The good news is that for working programmers most of
these distinctions are subtle enough, or theoretical enough,
not to matter. You create objects by either
a) calling an established default constructor protocol or
b) by invoking a class method or factory function.

Coming back to the original theme of this thread it looks like
we are coming to the conclusion that, at least when applied
to objects, factory methods/functions can also be called
constructors. (And from my foray into the "named constructor"
idiom of C++ those guys are allowing such latitude in terminology
too.) So the only place where confusion would exist is in those
languages (such as objective C or Object Pascal) where constructor
or 'factory method' have a specific syntax and meaning in the
language.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From shlyoko at gmail.com  Fri Jan  8 10:47:51 2016
From: shlyoko at gmail.com (Emil Natan)
Date: Fri, 8 Jan 2016 15:47:51 +0000
Subject: [Tutor] new line to list of strings send by email
Message-ID: <CAG=4S2D8AiDnRmT5gPqx2anX-j6VH5JEZeTcDZeR+Ephw5X2Vg@mail.gmail.com>

Hello list,

I have a function which receives a string and sends it as a body of an
email.

It is a part of a program which does certain checks on network
infrastructure. When a check fails I append error message to a
error_collector list:


if self.check_axfr_refused(ip):
     error_collector.append('%s:%s AXFR test for %s FAILED' %
                                           (ns, ip, self.domainname))

At the  end I send the alert like this:

if len(error_collector) != 0:
        email_body = str(error_collector)
        email_alert(email_body)

The problem is the resulted email (expectedly) has the alert message as one
long string.

['pdns6.ultradns.co.uk.:204.74.115.1 AXFR test for amazon.com FAILED',
'pdns6.ultradns.co.uk.:2610:a1:1017::1 AXFR test for amazon.com FAILED',
'ns4.p31.dynect.net.:204.13.251.31 AXFR test for amazon.com FAILED',...]

I tried adding '\n' to end of each string error_collector collects, but
then these were simply added to the resulted email.

What I want to achieve is that each collected error is shown on a separate
line in the email. Any advice will be well appreciated.

Here is the email sending function if in interest:

def email_alert(message, recipient=DEFAULT_RECIPIENT, subject_prefix=''):
    ''' Send email alert. '''
    # check if we are running in quiet mode
    if QUIET.lower() == 'yes':
        return
    msg = MIMEText(message)
    msg['From'] = SENDER
    msg['To'] = recipient
    msg['Subject'] = subject_prefix + SUBJECT

    s = smtplib.SMTP(SMTP_SERVER)
    s.send_message(msg)
    s.quit()

Emil

From sebastian_cheung at yahoo.com  Fri Jan  8 09:04:10 2016
From: sebastian_cheung at yahoo.com (sebastian cheung)
Date: Fri, 8 Jan 2016 14:04:10 +0000 (UTC)
Subject: [Tutor] get aws path from argParser
References: <440720212.2122942.1452261850390.JavaMail.yahoo.ref@mail.yahoo.com>
Message-ID: <440720212.2122942.1452261850390.JavaMail.yahoo@mail.yahoo.com>

* take an s3 destination path as an argument optionally containing the string ++DATE++ as a placeholder (e.g.?s3://my-bucket/objects/++DATE++/,?s3://my-bucket/objects/++DATE++/file-++DATE++.txt and?s3://my-bucket/objects/ should all be valid)
I already have something for something more simple, but for s3 maybe use awscli etc? Thanks Seb
def dateType(string):
    """
    Convert a date string to a date object
    """
    try:
        date = datetime.datetime.strptime(string, '%Y-%m-%d').date()
    except ValueError:
        msg = "%r is not a valid date" % string
        raise argparse.ArgumentTypeError(msg)
    return date


def is_valid_file(parser, arg):
    if not os.path.exists(arg):
        parser.error("The file %s does not exist!" % arg)
    else:
        return open(arg, 'r')parser = argparse.ArgumentParser(
        description="Take CLI options called start-date and end-date, which must be formatted YYYY-MM-DD. "
                    "These should default to today if not supplied",
        epilog="See http://bitbucket.org/niceseb/ for details about the Project Time Tracker.")
parser.add_argument('-e', '--end-date', metavar='DATE', type=dateType, default=datetime.date.today(),
                    help='the date tracking data should start at, inclusive in the format YYYY-MM-DD (defaults to today)')
parser.add_argument('-s', '--start-date', metavar='DATE', type=dateType, default=datetime.date.today(),
                    help='the date tracking data should end at,   inclusive in the format YYYY-MM-DD (defaults to today)')
parser.add_argument('-v', action='version', version='%(prog)s 1.0')
parser.add_argument('-i', dest="filename", required=False, help="input file name", metavar="FILE",
                    type=lambda x: is_valid_file(parser, x))

From bgailer at gmail.com  Fri Jan  8 12:04:00 2016
From: bgailer at gmail.com (Bob Gailer)
Date: Fri, 8 Jan 2016 12:04:00 -0500
Subject: [Tutor] new line to list of strings send by email
In-Reply-To: <CAG=4S2D8AiDnRmT5gPqx2anX-j6VH5JEZeTcDZeR+Ephw5X2Vg@mail.gmail.com>
References: <CAG=4S2D8AiDnRmT5gPqx2anX-j6VH5JEZeTcDZeR+Ephw5X2Vg@mail.gmail.com>
Message-ID: <CAP1rxO79Qc_dSqAdtDpDbSQ99EgGLOQ4Bj_bhVxFREzPKZj4Jw@mail.gmail.com>

On Jan 8, 2016 11:03 AM, "Emil Natan" <shlyoko at gmail.com> wrote:
>
> Hello list,
>
> I have a function which receives a string and sends it as a body of an
> email.
>
> It is a part of a program which does certain checks on network
> infrastructure. When a check fails I append error message to a
> error_collector list:
>
>
> if self.check_axfr_refused(ip):
>      error_collector.append('%s:%s AXFR test for %s FAILED' %
>                                            (ns, ip, self.domainname))
>
> At the  end I send the alert like this:
>
> if len(error_collector) != 0:
>         email_body = str(error_collector)
>         email_alert(email_body)
Instead of str( str(error_collector) ) try '\n'.join(error_collector)

From alan.gauld at btinternet.com  Fri Jan  8 13:45:53 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 8 Jan 2016 18:45:53 +0000
Subject: [Tutor] get aws path from argParser
In-Reply-To: <440720212.2122942.1452261850390.JavaMail.yahoo@mail.yahoo.com>
References: <440720212.2122942.1452261850390.JavaMail.yahoo.ref@mail.yahoo.com>
 <440720212.2122942.1452261850390.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <n6p051$k36$1@ger.gmane.org>

On 08/01/16 14:04, sebastian cheung via Tutor wrote:
> * take an s3 destination path as an argument optionally containing the string ++DATE++ as a placeholder (e.g. s3://my-bucket/objects/++DATE++/, s3://my-bucket/objects/++DATE++/file-++DATE++.txt and s3://my-bucket/objects/ should all be valid)
> I already have something for something more simple, but for s3 maybe use awscli etc? Thanks Seb

I have no idea what you are asking about (other than I
assume its something related to AWS?). This is the python
tutor list for answering questions about the Python language
and its standard library. Did you mean to post here?

If so you need to give us a bit more information about
what you are doing and what exactly you want help with.

> def dateType(string):
>     """
>     Convert a date string to a date object
>     """
>     try:
>         date = datetime.datetime.strptime(string, '%Y-%m-%d').date()
>     except ValueError:
>         msg = "%r is not a valid date" % string
>         raise argparse.ArgumentTypeError(msg)
>     return date
> 
> 
> def is_valid_file(parser, arg):
>     if not os.path.exists(arg):
>         parser.error("The file %s does not exist!" % arg)
>     else:
>         return open(arg, 'r')parser = argparse.ArgumentParser(
>         description="Take CLI options called start-date and end-date, which must be formatted YYYY-MM-DD. "
>                     "These should default to today if not supplied",
>         epilog="See http://bitbucket.org/niceseb/ for details about the Project Time Tracker.")
> parser.add_argument('-e', '--end-date', metavar='DATE', type=dateType, default=datetime.date.today(),
>                     help='the date tracking data should start at, inclusive in the format YYYY-MM-DD (defaults to today)')
> parser.add_argument('-s', '--start-date', metavar='DATE', type=dateType, default=datetime.date.today(),
>                     help='the date tracking data should end at,   inclusive in the format YYYY-MM-DD (defaults to today)')
> parser.add_argument('-v', action='version', version='%(prog)s 1.0')
> parser.add_argument('-i', dest="filename", required=False, help="input file name", metavar="FILE",
>                     type=lambda x: is_valid_file(parser, x))


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From badouglas at gmail.com  Fri Jan  8 14:07:36 2016
From: badouglas at gmail.com (bruce)
Date: Fri, 8 Jan 2016 14:07:36 -0500
Subject: [Tutor] idle??
Message-ID: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>

Hey guys/gals - list readers

Recently came across someone here mentioning IDLE!! -- not knowing
this. I hit google for a look.

Is IDLE essentially an ide for doing py dev? I see there's a
windows/linux (rpms) for it.

I'm running py.. I normally do $$python to pop up the py env for quick
tests.. and of course run my test scripts/apps from the cmdline via
./foo.py...

So, where does IDLE fit into this....

Thanks

(and yeah, I know I could continue to look at google, and even install
the rpms to really check it out!!)

tia!!

From alan.gauld at btinternet.com  Fri Jan  8 18:42:04 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 8 Jan 2016 23:42:04 +0000
Subject: [Tutor] idle??
In-Reply-To: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
References: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
Message-ID: <n6phgc$a26$1@ger.gmane.org>

On 08/01/16 19:07, bruce wrote:

> Is IDLE essentially an ide for doing py dev? I see there's a
> windows/linux (rpms) for it.

Yes, its the official IDE for Python.

There is an "unofficial" version called xidle which tends
to get a lot of the new stuff before it makes it into the
official release. For a long time not much happened with
IDLE but recently there has been a bunch of activity so
I'm hopeful we may soon see some new features appearing.

> So, where does IDLE fit into this....

It incorporates a shell window where you can type commands
and you can create blank editor windows(with syntax
highlighting etc etc) from which you can save files,
run them, debug them etc.

There are some YouTube and ShowMeDo videos around and
Danny Yoo has a short tutorial that is quite old but
still pretty much applicable.

There is official documentation on the python.org
website too.

Finally, it's not universally loved and definitely has
some quirks but it's adequate for getting started,
definitely better than notepad, say, on Windows.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From badouglas at gmail.com  Fri Jan  8 19:11:01 2016
From: badouglas at gmail.com (bruce)
Date: Fri, 8 Jan 2016 19:11:01 -0500
Subject: [Tutor] idle??
In-Reply-To: <n6phgc$a26$1@ger.gmane.org>
References: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
 <n6phgc$a26$1@ger.gmane.org>
Message-ID: <CAP16ngpTHjC3CRqMUoyDW=f=gzrsb_2bE1o1V6fwE9Yu9uESOw@mail.gmail.com>

Thanks Alan...

So, as an IDE/shell.. I assume it's not quite Eclipse, butallows you
to do reasonable editing/snyax tracking/etc.. as well as run apps
within the window/shell.. I assume breakpoints as well, and a good
chunk of the rest of the usual IDE functions...

What about function completion? Where I type a function.. and it
displays a "list" of potential function/defs ? Does it provide
"function" or item hoovering. where cursor can be placed of a
function/item and information about the func, or item
(type/struct/etc..) is displayed?

Thanks again' much appreciated!!




On Fri, Jan 8, 2016 at 6:42 PM, Alan Gauld <alan.gauld at btinternet.com> wrote:
> On 08/01/16 19:07, bruce wrote:
>
>> Is IDLE essentially an ide for doing py dev? I see there's a
>> windows/linux (rpms) for it.
>
> Yes, its the official IDE for Python.
>
> There is an "unofficial" version called xidle which tends
> to get a lot of the new stuff before it makes it into the
> official release. For a long time not much happened with
> IDLE but recently there has been a bunch of activity so
> I'm hopeful we may soon see some new features appearing.
>
>> So, where does IDLE fit into this....
>
> It incorporates a shell window where you can type commands
> and you can create blank editor windows(with syntax
> highlighting etc etc) from which you can save files,
> run them, debug them etc.
>
> There are some YouTube and ShowMeDo videos around and
> Danny Yoo has a short tutorial that is quite old but
> still pretty much applicable.
>
> There is official documentation on the python.org
> website too.
>
> Finally, it's not universally loved and definitely has
> some quirks but it's adequate for getting started,
> definitely better than notepad, say, on Windows.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From alan.gauld at btinternet.com  Fri Jan  8 19:50:45 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 9 Jan 2016 00:50:45 +0000
Subject: [Tutor] idle??
In-Reply-To: <CAP16ngpTHjC3CRqMUoyDW=f=gzrsb_2bE1o1V6fwE9Yu9uESOw@mail.gmail.com>
References: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
 <n6phgc$a26$1@ger.gmane.org>
 <CAP16ngpTHjC3CRqMUoyDW=f=gzrsb_2bE1o1V6fwE9Yu9uESOw@mail.gmail.com>
Message-ID: <n6plh6$37q$1@ger.gmane.org>

On 09/01/16 00:11, bruce wrote:

> So, as an IDE/shell.. I assume it's not quite Eclipse, butallows you
> to do reasonable editing/snyax tracking/etc.. as well as run apps
> within the window/shell.. I assume breakpoints as well, and a good
> chunk of the rest of the usual IDE functions...

Exactly.

> What about function completion? Where I type a function.. and it
> displays a "list" of potential function/defs ? 

Yes.

> Does it provide "function" or item hoovering. where cursor 
> can be placed of a function/item and
> information about the func, or item
> (type/struct/etc..) is displayed?

I don't think so but there is a primitive class browser.

Also idleX has something called a code context that shows
things like the method definition within its class.

To be honest I don't use IDLE that much any more and if I do
its idleX. Mostly I just use vim and a command shell and
for debugging winpdb (in the rare cases where print and
the >>> prompt are insufficient.)

The other important thing is that it is all written in
Python, and Tkinter as the GUI, so its a good source of
sample code.

But the easiest thing is just install the package and
have a play! :-)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From wombingsac at gmail.com  Fri Jan  8 20:56:09 2016
From: wombingsac at gmail.com (Whom Isac)
Date: Sat, 9 Jan 2016 11:56:09 +1000
Subject: [Tutor] Not sure why the code is giving weird result?
Message-ID: <CADXqDX8dzW+7-bL+yg3HeDW0hUMgnv61JoWd53Sktd30cvMFzg@mail.gmail.com>

Hi, today I tried to help with one of the request in Python tutor about
trailing zeros -6days old.
I don't know why my code is incrementing in number.
Here is the code:

def factorial():
    print("Here you will put your factorial")
    factVar=int(input("Please input what number to be factorial: \t"))
    TValue=0
    for i in range(0,factVar+1):
        TValue+=i*i+1
    print("Your total factorial: "+str(TValue))
    return TValue


def trailing_zeros():
   value=factorial()
   dvdVal=1
   divider=1
   totalTrailingZeroes=0

   while dvdVal !=0:
       try:
           answer=0
           answer=int(value/5**(divider))
           totalTrailingZeroes+=answer
           while answer !=0:
               if answer >0:
                   answer=int(value/5**(divider+1))
                   newanswer=round(answer,1)
                   totalTrailingZeroes+=newanswer
               elif answer <=0:
                   dvdVal=0
           print(str(TrailingZeroes))


       except Exception:
           print("Sorry About that")


trailing_zeros()



Here is what I tried After ward


###And here is why I tried Afterwards

def factorial():
    print("Here you will put your factorial")
    factVar=int(input("Please input what number to be factorial: \t"))
    TValue=0
    for i in range(0,factVar+1):
        TValue+=i*i+1
    print("Your total factorial: "+str(TValue))

    value=TValue
    dvdVal=1
    divider=1
    totalTrailingZeroes=0

    answer=int(value/5**(divider))
    totalTrailingZeroes+=answer
    newanswer=round(answer,1)
    while newanswer !=0:
        if newanswer >0:
            answer=int(value/(5**(divider+1)))
            newanswer=round(answer,1)
            totalTrailingZeroes+=newanswer
            print(str(totalTrailingZeroes))
        else:
            newanswer=0
            print(str(TrailingZeroes))
factorial()

"""
def trailing_zeros():
   value=factorial()
   dvdVal=1
   divider=1
   totalTrailingZeroes=0

   while dvdVal !=0:
       try:
           answer=0
           answer=value/5**(divider)
           totalTrailingZeroes+=answer
           while answer !=0:
               if answer >0:
                   answer=value/5**(divider+1)
                   newanswer=round(answer,1)
                   totalTrailingZeroes+=newanswer
               elif answer <=0:
                   dvdVal=0
           print(str(TrailingZeroes))


       except Exception:
           print("Sorry About that")


trailing_zeros()

"""

From wombingsac at gmail.com  Fri Jan  8 21:01:35 2016
From: wombingsac at gmail.com (Whom Isac)
Date: Sat, 9 Jan 2016 12:01:35 +1000
Subject: [Tutor] Creating a webcrawler
Message-ID: <CADXqDX8L1UsZNUOTR9oh-WBoOcdsqYqRWzMMcLroHr-c7tGPug@mail.gmail.com>

Hi I want to create a web-crawler but dont have any lead to choose any
module. I have came across the Jsoup but I am not familiar with how to use
it in 3.5 as I tried looking at a similar web crawler codes from 3.4 dev
version.
I just want to build that crawler to crawl through a javascript enable site
and automatically detect a download link (for video file)
.
And should I be using pickles to write the data in the text file/ save file.
Thanks

From __peter__ at web.de  Sat Jan  9 05:03:40 2016
From: __peter__ at web.de (Peter Otten)
Date: Sat, 09 Jan 2016 11:03:40 +0100
Subject: [Tutor] Not sure why the code is giving weird result?
References: <CADXqDX8dzW+7-bL+yg3HeDW0hUMgnv61JoWd53Sktd30cvMFzg@mail.gmail.com>
Message-ID: <n6qltt$55b$1@ger.gmane.org>

Whom Isac wrote:

> Hi, today I tried to help with one of the request in Python tutor about
> trailing zeros -6days old.
> I don't know why my code is incrementing in number.
> Here is the code:
> 
> def factorial():
>     print("Here you will put your factorial")
>     factVar=int(input("Please input what number to be factorial: \t"))
>     TValue=0
>     for i in range(0,factVar+1):
>         TValue+=i*i+1

That's not the "factorial". Look up its definition, change the loop 
accordingly, and if the result still isn't correct put 

          print(TValue)

in the for loop to ensure that it calculates what you think it does.

>           answer=int(value/5**(divider))

Just a general remark as I haven't checked the complete function. Python has 
an integer division operator that allows to write this

answer = value // 5**divider

This gives the correct answer where the use of float in intermediate values 
introduces errors. Compare:

>>> int(10**24 / 5)
199999999999999983222784
>>> 10**24 // 5
200000000000000000000000



From kwpolska at gmail.com  Sat Jan  9 05:38:53 2016
From: kwpolska at gmail.com (Chris Warrick)
Date: Sat, 9 Jan 2016 11:38:53 +0100
Subject: [Tutor] idle??
In-Reply-To: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
References: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
Message-ID: <CAMw+j7Kb42xzHoFRoXddjRAk7OQv69g-WVEKiw+weMNSpb49-w@mail.gmail.com>

On 8 January 2016 at 20:07, bruce <badouglas at gmail.com> wrote:
> Hey guys/gals - list readers
>
> Recently came across someone here mentioning IDLE!! -- not knowing
> this. I hit google for a look.
>
> Is IDLE essentially an ide for doing py dev? I see there's a
> windows/linux (rpms) for it.
>
> I'm running py.. I normally do $$python to pop up the py env for quick
> tests.. and of course run my test scripts/apps from the cmdline via
> ./foo.py...
>
> So, where does IDLE fit into this....

IDLE is a sad little ?IDE?, which is really ugly, because it?s written
in Tk. It lacks many IDE features. It comes with a really basic
debugger (that doesn?t even highlight the line that is being currently
executed?), function signature hinting, and some code completion.

And it doesn?t even do something as basic as line numbering.

Pretty much anything is better than IDLE. I recommend using vim with
the python-mode plugin and YouCompleteMe. The Atom editor can also be
a good Python environment. For fans of full-blown IDEs, there?s
PyCharm.

For experiments, IPython, Jupyter (aka IPython Notebook) or bpython
should be used. They are more capable than the basic interpreter, and
even have more features than IDLE.

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16

From mcshizney at hotmail.co.uk  Sat Jan  9 08:17:55 2016
From: mcshizney at hotmail.co.uk (Lawrence Lorenzo)
Date: Sat, 9 Jan 2016 13:17:55 +0000
Subject: [Tutor] Some error that you may find funny but I can't fix.
Message-ID: <DUB113-W28D126570E1C2968D0490FE6F70@phx.gbl>




Hey, I am very new to the ways of python and am currently experiencing this error. This program is just a novice project set by my school to create an adventure game however I am having issues with being able to set skill points (500) to the users desired skills. Here is what I have done and the error it gives me in the email.


 		 	   		  

From steve at pearwood.info  Sat Jan  9 08:59:18 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 10 Jan 2016 00:59:18 +1100
Subject: [Tutor] Some error that you may find funny but I can't fix.
In-Reply-To: <DUB113-W28D126570E1C2968D0490FE6F70@phx.gbl>
References: <DUB113-W28D126570E1C2968D0490FE6F70@phx.gbl>
Message-ID: <20160109135918.GM10854@ando.pearwood.info>

Hi Lawrence, and welcome!


On Sat, Jan 09, 2016 at 01:17:55PM +0000, Lawrence Lorenzo wrote:
> 
> Hey, I am very new to the ways of python and am currently experiencing 
> this error. This program is just a novice project set by my school to 
> create an adventure game however I am having issues with being able to 
> set skill points (500) to the users desired skills. Here is what I 
> have done and the error it gives me in the email.

Unfortunately you seem to have forgotten to include the code or error. 
Assuming your code is not too big (say, no more than one or two 
hundred lines), please copy and paste both the code and the full error 
into the body of your email.

Please make sure you turn off "Rich Text" or HTML mail, as that often 
messes up the code and makes it really hard to understand.

Thanks,


-- 
Steven

From alan.gauld at btinternet.com  Sat Jan  9 09:02:45 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 9 Jan 2016 14:02:45 +0000
Subject: [Tutor] Creating a webcrawler
In-Reply-To: <CADXqDX8L1UsZNUOTR9oh-WBoOcdsqYqRWzMMcLroHr-c7tGPug@mail.gmail.com>
References: <CADXqDX8L1UsZNUOTR9oh-WBoOcdsqYqRWzMMcLroHr-c7tGPug@mail.gmail.com>
Message-ID: <n6r3u5$vo8$1@ger.gmane.org>

On 09/01/16 02:01, Whom Isac wrote:
> Hi I want to create a web-crawler but dont have any lead to choose any
> module. I have came across the Jsoup but I am not familiar with how to use
> it in 3.5 as I tried looking at a similar web crawler codes from 3.4 dev
> version.

I don't know Jsoup and have no idea about how it works with 3.5.
However there are some modules in the standard library you can
use including htmlib, urllib and so on.

Beautiful soup is good at parsing badly constructed html and
etree is good for xml/xhtml.

Requests is also a good bet for working with http requests.

> I just want to build that crawler to crawl through a javascript enable site
> and automatically detect a download link (for video file)

Depending on what exactly the Javascript does it might not
be possible (at least not directly) Many modern sites simply
load up the document structure before calling a Javascript
function to fetch all the data (including inks and images)
from a server via JSON.

If that's what your site does you'll need to find the call to
the server and emulate it from Python.

> And should I be using pickles to write the data in the text file/ save file.

You could. You could also use a database such as SQLite.
It really depends on what you plan on doing with it after
you save it.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Sat Jan  9 09:20:21 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 10 Jan 2016 01:20:21 +1100
Subject: [Tutor] Not sure why the code is giving weird result?
In-Reply-To: <CADXqDX8dzW+7-bL+yg3HeDW0hUMgnv61JoWd53Sktd30cvMFzg@mail.gmail.com>
References: <CADXqDX8dzW+7-bL+yg3HeDW0hUMgnv61JoWd53Sktd30cvMFzg@mail.gmail.com>
Message-ID: <20160109142021.GN10854@ando.pearwood.info>

On Sat, Jan 09, 2016 at 11:56:09AM +1000, Whom Isac wrote:

"Not sure why the code is giving weird result?"

Neither are we. What do you mean "weird results"? Which code did you 
run? (You give two versions.) What result did you expect, and what 
result did you get?


But I can see one horrible mistake in your code:

>        try:
>            [code]
>        except Exception:
>            print("Sorry About that")

That's the WORST possible thing you can do as a programmer. That's like 
poking your eyes out with sharp stick, and then complaining that you 
can't see what it going on. Of course you can't see what is going on, 
you've blinded yourself.

Instead of Python showing you the error, you get a silly apology "Sorry 
about that". Sorry about what? Nobody knows! Why did it happen? It is 
impossible to say!

Step 1: 

Get rid of the "try...except" block. try...except is very useful, but 
here it is just hurting you. It is making it impossible for you to debug 
your code, since you can't see where the bugs are.



> def factorial():
>     print("Here you will put your factorial")
>     factVar=int(input("Please input what number to be factorial: \t"))
>     TValue=0
>     for i in range(0,factVar+1):
>         TValue+=i*i+1
>     print("Your total factorial: "+str(TValue))

I don't think that calculates what you think it calculates. If you enter 
5, the calculation will be:

TValue = 0
Add 0*0 + 1 = 1 gives TValue = 1
Add 1*1 + 1 = 2 gives TValue = 3
Add 2*2 + 1 = 5 gives TValue = 8
Add 3*3 + 1 = 10 gives TValue = 18
Add 4*4 + 1 = 17 gives TValue = 35
Add 5*5 + 1 = 26 gives TValue = 61

which is a little bit more than half the correct value, 5! = 120.

Try this instead:

from math import factorial
print(factorial(5))


I don't understand the rest of your code. In your own words, step by 
step as if you were writing out a recipe, what is it supposed to be 
doing?


>     value=TValue
>     dvdVal=1
>     divider=1
>     totalTrailingZeroes=0
> 
>     answer=int(value/5**(divider))
>     totalTrailingZeroes+=answer
>     newanswer=round(answer,1)
>     while newanswer !=0:
>         if newanswer >0:
>             answer=int(value/(5**(divider+1)))
>             newanswer=round(answer,1)
>             totalTrailingZeroes+=newanswer
>             print(str(totalTrailingZeroes))
>         else:
>             newanswer=0
>             print(str(TrailingZeroes))


By the way, you don't need any of those calls to "round()". They don't 
do anything. Since "answer" is already an int, you call round on an int, 
which rounds it to give exactly the same number:

py> answer = 42
py> newanswer = round(answer, 1)
py> newanswer
42

So you can save time and effort by just saying:

newanswer = answer

But why bother? What's the purpose of having two variables "answer" and 
"newanswer"?



-- 
Steve

From steve at pearwood.info  Sat Jan  9 09:36:44 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 10 Jan 2016 01:36:44 +1100
Subject: [Tutor] Creating a webcrawler
In-Reply-To: <CADXqDX8L1UsZNUOTR9oh-WBoOcdsqYqRWzMMcLroHr-c7tGPug@mail.gmail.com>
References: <CADXqDX8L1UsZNUOTR9oh-WBoOcdsqYqRWzMMcLroHr-c7tGPug@mail.gmail.com>
Message-ID: <20160109143644.GO10854@ando.pearwood.info>

On Sat, Jan 09, 2016 at 12:01:35PM +1000, Whom Isac wrote:
> Hi I want to create a web-crawler but dont have any lead to choose any
> module. I have came across the Jsoup but I am not familiar with how to use
> it in 3.5 as I tried looking at a similar web crawler codes from 3.4 dev
> version.
> I just want to build that crawler to crawl through a javascript enable site
> and automatically detect a download link (for video file)
> .

I admire your enthusiasm, but you have set yourself a HUGELY complicated 
project.

If you just want to extract some videos, you might find this 
existing tool (written in Python!) helpful:

http://rg3.github.io/youtube-dl/



> And should I be using pickles to write the data in the text file/ save file.

No.



-- 
Steve

From badouglas at gmail.com  Sat Jan  9 11:33:09 2016
From: badouglas at gmail.com (bruce)
Date: Sat, 9 Jan 2016 11:33:09 -0500
Subject: [Tutor] Creating a webcrawler
In-Reply-To: <CADXqDX8L1UsZNUOTR9oh-WBoOcdsqYqRWzMMcLroHr-c7tGPug@mail.gmail.com>
References: <CADXqDX8L1UsZNUOTR9oh-WBoOcdsqYqRWzMMcLroHr-c7tGPug@mail.gmail.com>
Message-ID: <CAP16ngohTo5kyxVyRcraq_hXysHfSsTqZmiW4dN=mUDi30vq=Q@mail.gmail.com>

Hi Isac.

I'm not going to get into the pythonic stuff.. People on the list are
way better than I.  I've been doing a chunk of crawling, it's not too
bad, depending on what you're trying to accomplish and the site you're
targeting.

So, no offense, but I'm going to treat you like a 6 year old (google
it - from a movie!)

You need to back up, and analyze the site/pages/structure you're going
after. Use the tools - firefox - livehttpheaders/nettraffic/etc..
  -you want to be able to see what the exchange is between the
client/browser, as well as the server..
  -often, this gives you the clues/insite to crafting the request from
your client back to the server for the item/data you're going for...

Once you've gotten that together, setup the basic process with
wget/curl etc to get a feel for any weird issues - cert issues?
-security issues - are cookies required - etc.. A good deal of this
stuff can be resolved/checked out at this level, without jumping into
coding..

Once you're comfortable at this point, you can crank out some simple
code to go after the site you're targeting.

In the event you really have a javascript/dynamic site that you can't
handle in any other manner, you're going to need to go use a 'headless
browser' process.

There are a number of headless browser projects - I think most run on
the webit codebase (don't quote me). Casper/phantomjs, there are also
pythonic implementations as well...

So, there you go, should/hopefully this will get you on your way!



On Fri, Jan 8, 2016 at 9:01 PM, Whom Isac <wombingsac at gmail.com> wrote:
> Hi I want to create a web-crawler but dont have any lead to choose any
> module. I have came across the Jsoup but I am not familiar with how to use
> it in 3.5 as I tried looking at a similar web crawler codes from 3.4 dev
> version.
> I just want to build that crawler to crawl through a javascript enable site
> and automatically detect a download link (for video file)
> .
> And should I be using pickles to write the data in the text file/ save file.
> Thanks
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From ashish.makani at gmail.com  Sat Jan  9 12:46:42 2016
From: ashish.makani at gmail.com (ashish makani)
Date: Sat, 9 Jan 2016 23:16:42 +0530
Subject: [Tutor] get aws path from argParser
In-Reply-To: <n6p051$k36$1@ger.gmane.org>
References: <440720212.2122942.1452261850390.JavaMail.yahoo.ref@mail.yahoo.com>
 <440720212.2122942.1452261850390.JavaMail.yahoo@mail.yahoo.com>
 <n6p051$k36$1@ger.gmane.org>
Message-ID: <CAEX1urjcUc-Z=u+Py2j=btUW+_U7Z9bSKo72scvxxyCA1gVJCQ@mail.gmail.com>

+1 to what Alan said.

Its not clear what you are asking & if you are asking something at all.

( to me, it seems like you are answering someone's query & accidentally
posted here on the python tutor mailing list)

sent from mobile device ;
excuse typos & auto-correct errors
On Jan 9, 2016 00:17, "Alan Gauld" <alan.gauld at btinternet.com> wrote:

> On 08/01/16 14:04, sebastian cheung via Tutor wrote:
> > * take an s3 destination path as an argument optionally containing the
> string ++DATE++ as a placeholder (e.g. s3://my-bucket/objects/++DATE++/,
> s3://my-bucket/objects/++DATE++/file-++DATE++.txt and
> s3://my-bucket/objects/ should all be valid)
> > I already have something for something more simple, but for s3 maybe use
> awscli etc? Thanks Seb
>
> I have no idea what you are asking about (other than I
> assume its something related to AWS?). This is the python
> tutor list for answering questions about the Python language
> and its standard library. Did you mean to post here?
>
> If so you need to give us a bit more information about
> what you are doing and what exactly you want help with.
>
> > def dateType(string):
> >     """
> >     Convert a date string to a date object
> >     """
> >     try:
> >         date = datetime.datetime.strptime(string, '%Y-%m-%d').date()
> >     except ValueError:
> >         msg = "%r is not a valid date" % string
> >         raise argparse.ArgumentTypeError(msg)
> >     return date
> >
> >
> > def is_valid_file(parser, arg):
> >     if not os.path.exists(arg):
> >         parser.error("The file %s does not exist!" % arg)
> >     else:
> >         return open(arg, 'r')parser = argparse.ArgumentParser(
> >         description="Take CLI options called start-date and end-date,
> which must be formatted YYYY-MM-DD. "
> >                     "These should default to today if not supplied",
> >         epilog="See http://bitbucket.org/niceseb/ for details about the
> Project Time Tracker.")
> > parser.add_argument('-e', '--end-date', metavar='DATE', type=dateType,
> default=datetime.date.today(),
> >                     help='the date tracking data should start at,
> inclusive in the format YYYY-MM-DD (defaults to today)')
> > parser.add_argument('-s', '--start-date', metavar='DATE', type=dateType,
> default=datetime.date.today(),
> >                     help='the date tracking data should end at,
>  inclusive in the format YYYY-MM-DD (defaults to today)')
> > parser.add_argument('-v', action='version', version='%(prog)s 1.0')
> > parser.add_argument('-i', dest="filename", required=False, help="input
> file name", metavar="FILE",
> >                     type=lambda x: is_valid_file(parser, x))
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From katye2007 at gmail.com  Sat Jan  9 16:16:22 2016
From: katye2007 at gmail.com (yehudak .)
Date: Sat, 9 Jan 2016 23:16:22 +0200
Subject: [Tutor] Hi Tutor
Message-ID: <CAE3ie42E9PuR-QA1Pm6Nckf+=K0FAeUZXA-0GJh+ys467X4dPw@mail.gmail.com>

I wrote this short program for my grandson:

from random import sample

soups = ['Onion soup', 'Veggie soup', 'Chicken soup', 'Corn soup']
salads = ['Veggie', 'Onion', 'Cabbage', 'Lettuce', 'Caesar', 'Tomato']
main = ['Crab cake', 'Catfish', 'Ribs', 'Chopped liver', 'Meat balls']
beverage = ['Wine', 'Rum', 'Lemonade', 'Red bull', 'Margarita', 'Jin']

def dish(soups):
    return (sample(soups, 1))

print('Soup:\t\t', dish(soups))
print('Salad:\t\t', dish(salads))
print('Main dish:\t', dish(main))
print('Beverage:\t', dish(beverage))

A possible output could be:

Soup: ['Chicken soup']
Salad: ['Caesar']
Main dish: ['Meat balls']
Beverage: ['Wine']

How do I get rid from the square brackets and the quotation marks in the
output?

Thank you.

From __peter__ at web.de  Sat Jan  9 16:36:49 2016
From: __peter__ at web.de (Peter Otten)
Date: Sat, 09 Jan 2016 22:36:49 +0100
Subject: [Tutor] Printing a list without square brackets, was Re: Hi Tutor
References: <CAE3ie42E9PuR-QA1Pm6Nckf+=K0FAeUZXA-0GJh+ys467X4dPw@mail.gmail.com>
Message-ID: <n6ruhk$ej2$1@ger.gmane.org>

yehudak . wrote:

> I wrote this short program for my grandson:

Nothing against a little help here and there, but solving a problem can be 
fun, and you are taking away some of that fun. 

Does your grandson speak English? You might encourage him to post here 
himself. We bite, but only grandpas ;)
 
> from random import sample
> 
> soups = ['Onion soup', 'Veggie soup', 'Chicken soup', 'Corn soup']
> salads = ['Veggie', 'Onion', 'Cabbage', 'Lettuce', 'Caesar', 'Tomato']
> main = ['Crab cake', 'Catfish', 'Ribs', 'Chopped liver', 'Meat balls']
> beverage = ['Wine', 'Rum', 'Lemonade', 'Red bull', 'Margarita', 'Jin']
> 
> def dish(soups):
>     return (sample(soups, 1))
> 
> print('Soup:\t\t', dish(soups))
> print('Salad:\t\t', dish(salads))
> print('Main dish:\t', dish(main))
> print('Beverage:\t', dish(beverage))
> 
> A possible output could be:
> 
> Soup: ['Chicken soup']
> Salad: ['Caesar']
> Main dish: ['Meat balls']
> Beverage: ['Wine']
> 
> How do I get rid from the square brackets and the quotation marks in the
> output?

random.sample(items, n) returns a list of length n. As your sample size is 
one you could use random.choice(items) instead.

>>> import random
>>> soups = ['Onion soup', 'Veggie soup', 'Chicken soup', 'Corn soup']
>>> print("Soup:", random.choice(soups))
Soup: Chicken soup

When you actually want to print multiple strings you can preprocess the list 
with the str.join() method:

>>> print("Two soups:", ", ".join(random.sample(soups, 2)))
Two soups: Veggie soup, Chicken soup



From martin at linux-ip.net  Sat Jan  9 16:56:47 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Sat, 9 Jan 2016 13:56:47 -0800
Subject: [Tutor] Hi Tutor
In-Reply-To: <CAE3ie42E9PuR-QA1Pm6Nckf+=K0FAeUZXA-0GJh+ys467X4dPw@mail.gmail.com>
References: <CAE3ie42E9PuR-QA1Pm6Nckf+=K0FAeUZXA-0GJh+ys467X4dPw@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1601091354020.11287@qnttre.jbaqresebt.arg>


>I wrote this short program for my grandson:
>
>from random import sample
>
>soups = ['Onion soup', 'Veggie soup', 'Chicken soup', 'Corn soup']
>salads = ['Veggie', 'Onion', 'Cabbage', 'Lettuce', 'Caesar', 'Tomato']
>main = ['Crab cake', 'Catfish', 'Ribs', 'Chopped liver', 'Meat balls']
>beverage = ['Wine', 'Rum', 'Lemonade', 'Red bull', 'Margarita', 'Jin']
>
>def dish(soups):
>    return (sample(soups, 1))
>
>print('Soup:\t\t', dish(soups))
>print('Salad:\t\t', dish(salads))
>print('Main dish:\t', dish(main))
>print('Beverage:\t', dish(beverage))
>
>A possible output could be:
>
>Soup: ['Chicken soup']
>Salad: ['Caesar']
>Main dish: ['Meat balls']
>Beverage: ['Wine']
>
>How do I get rid from the square brackets and the quotation marks 
>in the output?

There are many possible answers to this question.  Here's my answer:

  from random import choice

  def dish(options):
      return choice(options)

Then, the function dish() will return exactly one element from the 
options.  Since each of soup, salads, main and beverage are lists 
with string elements, the dish() function will return a string.

I would like to have some Onion soup, the Crab cake, Rum and a 
Caesar, please.

Good luck,

-Martin

-- 
Martin A. Brown
http://linux-ip.net/

From mcshizney at hotmail.co.uk  Sat Jan  9 14:49:59 2016
From: mcshizney at hotmail.co.uk (Lawrence Lorenzo)
Date: Sat, 9 Jan 2016 19:49:59 +0000
Subject: [Tutor] Some error that you may find funny but I can't fix.
In-Reply-To: <DUB113-W28D126570E1C2968D0490FE6F70@phx.gbl>
References: <DUB113-W28D126570E1C2968D0490FE6F70@phx.gbl>
Message-ID: <DUB113-W6590660C226989813F7BAE6F70@phx.gbl>



From: mcshizney at hotmail.co.uk
To: tutor at python.org
Subject: Some error that you may find funny but I can't fix.
Date: Sat, 9 Jan 2016 13:17:55 +0000







Hey, I am very new to the ways of python and am currently experiencing this error. This program is just a novice project set by my school to create an adventure game however I am having issues with being able to set skill points (500) to the users desired skills. Here is what I have done and the error it gives me in the email.

RE: [Tutor] Some error that you may find funny but I can't fix.Lawrence Lorenzo  16:35 To: Steven D'Aprano> Date: Sun, 10 Jan 2016 00:59:18 +1100> From: steve at pearwood.info> To: tutor at python.org> CC: mcshizney at hotmail.co.uk> Subject: Re: [Tutor] Some error that you may find funny but I can't fix.> > Hi Lawrence, and welcome!> > > On Sat, Jan 09, 2016 at 01:17:55PM +0000, Lawrence Lorenzo wrote:> > > > Hey, I am very new to the ways of python and am currently experiencing > > this error. This program is just a novice project set by my school to > > create an adventure game however I am having issues with being able to > > set skill points (500) to the users desired skills. Here is what I > > have done and the error it gives me in the email.> > Unfortunately you seem to have forgotten to include the code or error. > Assuming your code is not too big (say, no more than one or two > hundred lines), please copy and paste both the code and the full error > into the body of your email.> > Please make sure you turn off "Rich Text" or HTML mail, as that often > messes up the code and makes it really hard to understand.> > Thanks,> > > -- > Stevenimport randomimport timeimport math#the player and NPC class.class char(object): #character attributes    def __init__(self, name, health, attack, rng, magic, speed):        self.name = name        self.health = health        self.attack = attack        self.speed = rng        self.magic = magic        self.speed =speedskillpoints = 500print(" You have 500 skill points to spend on character development so use them wisely.")print("There are 5 skills ""health, attack, range, magic and speed"" which you can decide to spend sillpoints on.")print("Each skill has a weakness apart from speed which determines who attacks first in a battle and if you can flee.")print("You may want to enforce a single ability rather than have multiple weaker abilities.")print("note that melee beats range, range beats magic, magic beats melee. If you have the same skill points in 2 skills then you won't have a weakness.")time.sleep(1)count = 500while (count) > 0:    name = input("Please enter a character name. ")    health = int(input("Enter a number for the ammount of points you would like to designate to your characters health. Remember you only have 500 and have 5 skills to set. "))        (count) = count - health    attack = int(input("Enter a number for the ammount of points you would like to designate to your characters attack. You only have ", count, " remaining and 4 skills to set. "))        (count) = count - attack    rng = int(input("enter a number for the ammount of points you would like to designate to your characters range. You only have ", count, " remaining and 3 skills to set. "))        (count) = count - rng    magic = int(input("enter a number for the ammount of points you would like to designate to your characters magic. You only have ", count," remaining and 2 skills to set "))        (count) = count - magic    print ("Your character speed has been set at (", count, ")")        speed = (count)    (count) = 0                                                     print ("" + name + "your health has been set to " (health))print ("" + name + "your attack has been set to " (attack))print ("" + name + "your range has been set to " (rng))print ("" + name + "your magic has been set to " (magic))print ("" + name + "your speed has been set to " (speed))player = [()]            Traceback (most recent call last):  File "C:\Users\mcshizney\Desktop\adventuregame.py", line 29, in <module>    attack = int(input("Enter a number for the ammount of points you would like to designate to your characters attack. You only have ", count, " remaining and 4 skills to set. "))TypeError: input expected at most 1 arguments, got 3'Code not done obviously however here is the error and code.
 		 	   		   		 	   		  

From mcshizney at hotmail.co.uk  Sat Jan  9 18:51:22 2016
From: mcshizney at hotmail.co.uk (Lawrence Lorenzo)
Date: Sat, 9 Jan 2016 23:51:22 +0000
Subject: [Tutor] Python error that you may find funny but I don't get. :)
Message-ID: <DUB113-W55707C2F7F608E023CAEE6E6F70@phx.gbl>

Hey, I am very new to the ways of python and am currently experiencing this error. This program is just a novice project set by my school to create an adventure game however I am having issues with being able to set skill points (500) to the users desired skills. Here is what I have done and the error it gives me in the email.



import randomimport timeimport math
#the player and NPC class.class char(object): #character attributes    def __init__(self, name, health, attack, rng, magic, speed):        self.name = name        self.health = health        self.attack = attack        self.speed = rng        self.magic = magic        self.speed =speedskillpoints = 500print(" You have 500 skill points to spend on character development so use them wisely.")print("There are 5 skills ""health, attack, range, magic and speed"" which you can decide to spend sillpoints on.")print("Each skill has a weakness apart from speed which determines who attacks first in a battle and if you can flee.")print("You may want to enforce a single ability rather than have multiple weaker abilities.")print("note that melee beats range, range beats magic, magic beats melee. If you have the same skill points in 2 skills then you won't have a weakness.")time.sleep(1)
count = 500
while (count) > 0:    name = input("Please enter a character name. ")    health = int(input("Enter a number for the ammount of points you would like to designate to your characters health. Remember you only have 500 and have 5 skills to set. "))        (count) = count - health    attack = int(input("Enter a number for the ammount of points you would like to designate to your characters attack. You only have ", count, " remaining and 4 skills to set. "))        (count) = count - attack    rng = int(input("enter a number for the ammount of points you would like to designate to your characters range. You only have ", count, " remaining and 3 skills to set. "))        (count) = count - rng    magic = int(input("enter a number for the ammount of points you would like to designate to your characters magic. You only have ", count," remaining and 2 skills to set "))        (count) = count - magic    print ("Your character speed has been set at (", count, ")")        speed = (count)    (count) = 0                                                     

print ("" + name + "your health has been set to " (health))print ("" + name + "your attack has been set to " (attack))print ("" + name + "your range has been set to " (rng))print ("" + name + "your magic has been set to " (magic))print ("" + name + "your speed has been set to " (speed))

player = [()]            


Traceback (most recent call last):  File "C:\Users\mcshizney\Desktop\adventuregame.py", line 29, in <module>    attack = int(input("Enter a number for the ammount of points you would like to designate to your characters attack. You only have ", count, " remaining and 4 skills to set. "))TypeError: input expected at most 1 arguments, got 3'



Code not done obviously however here is the error and code.
 		 	   		  

From preciousakams at yahoo.com  Sat Jan  9 14:54:57 2016
From: preciousakams at yahoo.com (precious akams)
Date: Sat, 9 Jan 2016 19:54:57 +0000 (UTC)
Subject: [Tutor] PLEASE I NEED HELP URGENTLY
References: <911299850.2554269.1452369297130.JavaMail.yahoo.ref@mail.yahoo.com>
Message-ID: <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>

PLEASE I NEED A LITTLE HELP .
I can figure out what Iam missing in this project

Create a class called BankAccount
.Create a constructor that takes in an integer and assigns this to a `balance` property.
.Create a method called `deposit` that takes in cash deposit amount and updates the balance accordingly.
.Create a method called `withdraw` that takes in cash withdrawal amount and updates the balance accordingly. if amount is greater than balance return `"invalid transaction"`
.Create a subclass MinimumBalanceAccount of the BankAccount class

THIS IS MY SOLUTION

class BankAccount:
def_init_(self, initial_amount):
self.balance=initial_amount

def deposit (self, amount):
self.balance+=amount

def withdraw (self, amount):
if self.balance>=amount:
return ('invalid transaction')

class MinimumBalanceAccount(BankAccount):
def _init_(self):
BankAccount_init_(self)

THIS IS THE ERROR MESSAGE I GOT

Internal Error: runTests aborted: TestOutcomeEvent(handled=False, test=, result=, outcome='error', exc_info=(, TypeError('this constructor takes no arguments',), ), reason=None, expected=False, shortLabel=None, longLabel=None) is not JSON serializable

From david at graniteweb.com  Sat Jan  9 19:41:08 2016
From: david at graniteweb.com (David Rock)
Date: Sat, 9 Jan 2016 18:41:08 -0600
Subject: [Tutor] PLEASE I NEED HELP URGENTLY
In-Reply-To: <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
References: <911299850.2554269.1452369297130.JavaMail.yahoo.ref@mail.yahoo.com>
 <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <20160110004108.GC29994@raspberrypi>

* precious akams via Tutor <tutor at python.org> [2016-01-09 19:54]:
> PLEASE I NEED A LITTLE HELP .
> I can figure out what Iam missing in this project
> 
> Create a class called BankAccount
> .Create a constructor that takes in an integer and assigns this to a `balance` property.
> .Create a method called `deposit` that takes in cash deposit amount and updates the balance accordingly.
> .Create a method called `withdraw` that takes in cash withdrawal amount and updates the balance accordingly. if amount is greater than balance return `"invalid transaction"`
> .Create a subclass MinimumBalanceAccount of the BankAccount class
> 
> THIS IS MY SOLUTION
> 
> class BankAccount:
> def_init_(self, initial_amount):
> self.balance=initial_amount
> 
> def deposit (self, amount):
> self.balance+=amount
> 
> def withdraw (self, amount):
> if self.balance>=amount:
> return ('invalid transaction')
> 
> class MinimumBalanceAccount(BankAccount):
> def _init_(self):
> BankAccount_init_(self)
> 
> THIS IS THE ERROR MESSAGE I GOT
> 
> Internal Error: runTests aborted: TestOutcomeEvent(handled=False, test=, result=, outcome='error', exc_info=(, TypeError('this constructor takes no arguments',), ), reason=None, expected=False, shortLabel=None, longLabel=None) is not JSON serializable

This doesn't appear to be all of your code.  What's the code you are running that actually generates the traceback?

-- 
David Rock
david at graniteweb.com

From alan.gauld at btinternet.com  Sat Jan  9 20:07:58 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 10 Jan 2016 01:07:58 +0000
Subject: [Tutor] PLEASE I NEED HELP URGENTLY
In-Reply-To: <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
References: <911299850.2554269.1452369297130.JavaMail.yahoo.ref@mail.yahoo.com>
 <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <n6sate$qjl$1@ger.gmane.org>

On 09/01/16 19:54, precious akams via Tutor wrote:
> PLEASE I NEED A LITTLE HELP .
> I can figure out what Iam missing in this project
> 

Please post in plain text otherwise the code formatting
gets messed up.

> class BankAccount:
> def_init_(self, initial_amount):
> self.balance=initial_amount

init should have two underscores on each side.
ie
__init__
not
_init_

> TypeError('this constructor takes no arguments'

I'm not sure how you are running your code but thats
not the usual error message format. If posting again
please include details of your OS, python version and
any IDE you are using.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Sat Jan  9 20:09:26 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 10 Jan 2016 01:09:26 +0000
Subject: [Tutor] Some error that you may find funny but I can't fix.
In-Reply-To: <DUB113-W6590660C226989813F7BAE6F70@phx.gbl>
References: <DUB113-W28D126570E1C2968D0490FE6F70@phx.gbl>
 <DUB113-W6590660C226989813F7BAE6F70@phx.gbl>
Message-ID: <n6sb06$qjl$2@ger.gmane.org>

On 09/01/16 19:49, Lawrence Lorenzo wrote:

> Hey, I am very new to the ways of python and am currently experiencing this error. This program is just a novice project set by my school to create an adventure game however I am having issues with being able to set skill points (500) to the users desired skills. Here is what I have done and the error it gives me in the email.

Can you post in plain text please?
Otherwise all your code gets messed up making it
impossible to guess what might be wrong.

> RE: [Tutor] Some error that you may find funny but I can't fix.Lawrence Lorenzo  16:35 To: Steven D'Aprano> Date: Sun, 10 Jan 2016 00:59:18 +1100> From: steve at pearwood.info> To: tutor at python.org> CC: mcshizney at hotmail.co.uk> Subject: Re: [Tutor] Some error that you may find funny but I can't fix.> > Hi Lawrence, and welcome!> > > On Sat, Jan 09, 2016 at 01:17:55PM +0000, Lawrence Lorenzo wrote:> > > > Hey, I am very new to the ways of python and am currently experiencing > > this error. This program is just a novice project set by my school to > > create an adventure game however I am having issues with being able to > > set skill points (500) to the users desired skills. Here is what I > > have done and the error it gives me in the email.> > Unfortunately you seem to have forgotten to inc
>  lude the code or error. > Assuming your code is not too big (say, no more than one or two > hundred lines), please copy and paste both the code and the full error > into the body of your email.> > P
>  lease make sure you turn off "Rich Text" or HTML mail, as that often > messes up the code and makes it really hard to understand.> > Thanks,> > > -- > Stevenimport randomimport timeimport math#the player and NPC class.class char(object): #character attributes    def __init__(self, name, health, attack, rng, magic, speed):        self.name = name        self.health = health        self.attack = attack        self.speed = rng        self.magic = magic        self.speed =speedskillpoints = 500print(" You have 500 skill points to spend on character development so use them wisely.")print("There are 5 skills ""health, attack, range, magic and speed"" which you can decide to spend sillpoints on.")print("Each skill has a weakness apart from speed which determines who attacks first in a battle and
>   if you can flee.")print("You may want to enforce a single ability rather than have multiple weaker abilities.")print("note that melee beats range, range beats magic, magic beats melee. If you have 
>  the same skill points in 2 skills then you won't have a weakness.")time.sleep(1)count = 500while (count) > 0:    name = input("Please enter a character name. ")    health = int(input("Enter a number for the ammount of points you would like to designate to your characters health. Remember you only have 500 and have 5 skills to set. "))        (count) = count - health    attack = int(input("Enter a number for the ammount of points you would like to designate to your characters attack. You only have ", count, " remaining and 4 skills to set. "))        (count) = count - attack    rng = int(input("enter a number for the ammount of points you would like to designate to your characters range. You only have ", count, " remaining and 3 skills to set. "))        (count) = count - rng    magic = in


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From sayz at bil.omu.edu.tr  Sat Jan  9 20:55:54 2016
From: sayz at bil.omu.edu.tr (=?UTF-8?B?U2VmYSBZxLFsZMSxeg==?=)
Date: Sun, 10 Jan 2016 03:55:54 +0200
Subject: [Tutor] PLEASE I NEED HELP URGENTLY
In-Reply-To: <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
References: <911299850.2554269.1452369297130.JavaMail.yahoo.ref@mail.yahoo.com>
 <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <CAEPLMOWo-D1FGRMe5EkmKaXdadWvtMbLi7w9yYw18zcqfAnKSQ@mail.gmail.com>

2016-01-09 21:54 GMT+02:00 precious akams via Tutor <tutor at python.org>:
> PLEASE I NEED A LITTLE HELP .

http://www.catb.org/esr/faqs/smart-questions.html

-- 
sayz

From sjeik_appie at hotmail.com  Sun Jan 10 06:53:22 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 10 Jan 2016 11:53:22 +0000
Subject: [Tutor] Question about the memory manager
Message-ID: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>

Hi,

I just found a neat trick to free up an emergency stash of memory in a funtion that overrides sys.excepthook. The rationale is that all exceptions, including MemoryErrors will be logged.
The code is below. My question: is that memory *guaranteed* to be freed right after the 'del' statement? Or should one call gc.collect to be really sure?

rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
def handle_exception(e):
global rainydayfund
del rainydayfund
... etc, etc ...
http://stackoverflow.com/questions/1235349/python-how-can-i-handle-any-unhandled-exception-in-an-alternative-way

Thanks!

Albert-Jan
 		 	   		  

From steve at pearwood.info  Sun Jan 10 11:16:21 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 11 Jan 2016 03:16:21 +1100
Subject: [Tutor] Question about the memory manager
In-Reply-To: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
Message-ID: <20160110161621.GV10854@ando.pearwood.info>

On Sun, Jan 10, 2016 at 11:53:22AM +0000, Albert-Jan Roskam wrote:
> Hi,
> 
> I just found a neat trick to free up an emergency stash of memory in a 
> funtion that overrides sys.excepthook.

> rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
> def handle_exception(e):
> global rainydayfund
> del rainydayfund
> ... etc, etc ...

I was going to write a scornful email about how useless this would be. I 
still think it's useless, but I see that the idea comes from Alex 
Martelli, who normally knows what he is talking about, so that makes me 
pause and think and perhaps do some experiments before commenting.

Even the best programmer can have some weird idiosyncratic 
superstitions, but only a fool would assume that Alex Martelli has got 
it wrong without doing some careful investigation. So let me get back to 
you :-)


-- 
Steve

From tim.peters at gmail.com  Sun Jan 10 11:54:10 2016
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 10 Jan 2016 10:54:10 -0600
Subject: [Tutor] Question about the memory manager
In-Reply-To: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
Message-ID: <CAExdVN=Fpw7x0pMRNT+JCFsq4PoWWgrooMU3qTjqpZ7-nqrWYQ@mail.gmail.com>

[Albert-Jan Roskam <sjeik_appie at hotmail.com>]
> I just found a neat trick to free up an emergency stash of memory in
> a funtion that overrides sys.excepthook. The rationale is that all
> exceptions, including MemoryErrors will be logged.
> The code is below. My question: is that memory *guaranteed* to be
> freed right after the 'del' statement? Or should one call gc.collect to
> be really sure?
>
> rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
> def handle_exception(e):
>     global rainydayfund
>     del rainydayfund
> ... etc, etc ...
> http://stackoverflow.com/questions/1235349/python-how-can-i-handle-any-unhandled-exception-in-an-alternative-way

This works fine in all versions of CPython (the C implementation of
Python distributed by python.org) to date.  That's because:

1. All versions of CPython rely primarily on reference counting (`gc`
is only needed to reclaim garbage containing reference cycles).  An
object is released immediately when its reference count falls to 0.

2. There is only one reference to the big list there (via the global
`raindydayfund`), so the memory becomes garbage immediately upon
executing the `del`.

3. Similarly, that giant list holds the only references to the masses
of distinct empty lists it contains, so they also become garbage
immediately upon the giant list becoming garbage.

4. CPython doesn't happen to stick garbage lists in, e.g., some
internal free list reusable only for new list objects - it actually
releases the memory for garbage lists.  Kinda ;-)

#2 and #3 are necessarily true.  #1 is true in CPython, but not in all
implementations of Python.

#4 is where things _might_ change even in CPython, but it's very
unlikely to change.  As is, it would take a small book to flesh out
what "Kinda ;-)" means, exactly.  Memory management is complex, with
many layers, involving many details.

If you can live with all that, I'd suggest a more straightforward way
of setting it up, like:

rainydayfund  = b"x" * N

where `N` is the number of bytes you want to reserve.  That is, create
a giant bytestring containing the number of "emergency bytes" you
need.  If N is large enough, that will avoid CPython's "small object
allocator" and CPython's "arena allocator", getting the memory
directly from (and returning the memory directly to) the OS.  The
fewer layers that get involved, the fewer layers that _may_ surprise
you by changing behavior in the future.

From __peter__ at web.de  Sun Jan 10 12:29:06 2016
From: __peter__ at web.de (Peter Otten)
Date: Sun, 10 Jan 2016 18:29:06 +0100
Subject: [Tutor] Question about the memory manager
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
Message-ID: <n6u4d2$2na$1@ger.gmane.org>

Albert-Jan Roskam wrote:

> Hi,
> 
> I just found a neat trick to free up an emergency stash of memory in a
> funtion that overrides sys.excepthook. The rationale is that all
> exceptions, including MemoryErrors will be logged. The code is below. My
> question: is that memory *guaranteed* to be freed right after the 'del'
> statement? Or should one call gc.collect to be really sure?
> 
> rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
> def handle_exception(e):
> global rainydayfund
> del rainydayfund
> ... etc, etc ...
> http://stackoverflow.com/questions/1235349/python-how-can-i-handle-any-unhandled-exception-in-an-alternative-way

I must admit that this looks rather strange to me, but I don't see any 
problem why it wouldn't work. gc.collect() only helps with circular 
dependencies, and there aren't any in your "rainy day fund".

Regarding the use of sys.excepthook I'd rather (SO voting be damned!) take 
the obvious approach as suggested by Vinay Sajip, i. e.

try:
    main()
except:
    # do what you have to do

instead of rewriting the hook. If potential memory resource hogs are 
confined within main() with no references from any module namespace you 
should have enough memory availaible to run the except suite without the 
need for a rainy day fund. If you know about a specific global name that 
references a consumer of a lot of memory, an in-memory db, say, then why not 
handle the memory error there or at least use it in lieu of a dedicated 
dummy? I. e.

in_memory_db = None
try:
    try:
        main()
    finally:
        del in_memory_db
except:
    # do what you have to do
    



From mcshizney at hotmail.co.uk  Sun Jan 10 10:21:15 2016
From: mcshizney at hotmail.co.uk (Lawrence Lorenzo)
Date: Sun, 10 Jan 2016 15:21:15 +0000
Subject: [Tutor] (no subject)
Message-ID: <DUB113-W592CFBDAD5481F899D3C32E6C80@phx.gbl>

import randomimport timeimport math
#the player and NPC class.class char(object): #character attributes    def __init__(self, name, health, attack, rng, magic, speed):        self.name = name        self.health = health        self.attack = attack        self.speed = rng        self.magic = magic        self.speed =speedskillpoints = 500print(" You have 500 skill points to spend on character development so use them wisely.")print("There are 5 skills ""health, attack, range, magic and speed"" which you can decide to spend sillpoints on.")print("Each skill has a weakness apart from speed which determines who attacks first in a battle and if you can flee.")print("You may want to enforce a single ability rather than have multiple weaker abilities.")print("note that melee beats range, range beats magic, magic beats melee. If you have the same skill points in 2 skills then you won't have a weakness.")time.sleep(1)
count = 500
while (count) > 0:    name = input("Please enter a character name. ")    health = int(input("Enter a number for the ammount of points you would like to designate to your characters health. Remember you only have 500 and have 5 skills to set. "))        (count) = count - health    attack = int(input("Enter a number for the ammount of points you would like to designate to your characters attack. You only have ", count, " remaining and 4 skills to set. "))        (count) = count - attack    rng = int(input("enter a number for the ammount of points you would like to designate to your characters range. You only have ", count, " remaining and 3 skills to set. "))        (count) = count - rng    magic = int(input("enter a number for the ammount of points you would like to designate to your characters magic. You only have ", count," remaining and 2 skills to set "))        (count) = count - magic    print ("Your character speed has been set at (", count, ")")        speed = (count)    (count) = 0                                                     

print ("" + name + "your health has been set to " (health))print ("" + name + "your attack has been set to " (attack))print ("" + name + "your range has been set to " (rng))print ("" + name + "your magic has been set to " (magic))print ("" + name + "your speed has been set to " (speed))

player = [()]



The error is: 
 You have 500 skill points to spend on character development so use them wisely.There are 5 skills health, attack, range, magic and speed which you can decide to spend sillpoints on.Each skill has a weakness apart from speed which determines who attacks first in a battle and if you can flee.You may want to enforce a single ability rather than have multiple weaker abilities.note that melee beats range, range beats magic, magic beats melee. If you have the same skill points in 2 skills then you won't have a weakness.Please enter a character name. bobEnter a number for the ammount of points you would like to designate to your characters health. Remember you only have 500 and have 5 skills to set. 40

Traceback (most recent call last):  File "C:\Users\mcshizney\Desktop\adventuregame.py", line 29, in <module>    attack = int(input("Enter a number for the ammount of points you would like to designate to your characters attack. You only have ", count, " remaining and 4 skills to set. "))TypeError: input expected at most 1 arguments, got 3             		 	   		  

From __peter__ at web.de  Sun Jan 10 13:42:19 2016
From: __peter__ at web.de (Peter Otten)
Date: Sun, 10 Jan 2016 19:42:19 +0100
Subject: [Tutor] (no subject)
References: <DUB113-W592CFBDAD5481F899D3C32E6C80@phx.gbl>
Message-ID: <n6u8mf$3np$1@ger.gmane.org>

Lawrence Lorenzo wrote:

> import randomimport timeimport math
> #the player and NPC class.class char(object): #character attributes    def
> #__init__(self, name, health, attack, rng, magic, speed):        self.name
> #= name        self.health = health        self.attack = attack       
> #self.speed = rng        self.magic = magic        self.speed

[snip]

> You have 500 skill points to spend on character development so use them
> wisely.There are 5 skills health, attack, range, magic and 

Hi Lawrence!

You have 500 goodwill points provided by those who help newbies on the tutor 
mailing list. Use them wisely by picking a proper subject line and sending 
the message body in plain text so that python code whose meaning depends 
heavily on its format can be read without tearing one's hear out.

Thank you.


From alan.gauld at btinternet.com  Sun Jan 10 13:49:25 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 10 Jan 2016 18:49:25 +0000
Subject: [Tutor] (no subject)
In-Reply-To: <DUB113-W592CFBDAD5481F899D3C32E6C80@phx.gbl>
References: <DUB113-W592CFBDAD5481F899D3C32E6C80@phx.gbl>
Message-ID: <n6u93l$9v3$1@ger.gmane.org>

On 10/01/16 15:21, Lawrence Lorenzo wrote:

Please use plain text to send mail. Your formatting
makes the code unreadable.

Thankfully the error does all the work for us...

> import randomimport timeimport math
> #the player and NPC class.class char(object): #character attributes    def __init__(self, name, health, attack, rng, magic, speed):        self.name = name        self.health = health        self.attack = attack        self.speed = rng        self.magic = magic        self.speed =speedskillpoints = 500print(" You have 500 skill points to spend on character development so use them wisely.")print("There are 5 skills ""health, attack, range, magic and speed"" which you can decide to spend sillpoints on.")print("Each skill has a weakness apart from speed which determines who attacks first in a battle and if you can flee.")print("You may want to enforce a single ability rather than have multiple weaker abilities.")print("note that melee beats range, range beats magic, magic beats melee. If you 
>  have the same skill points in 2 skills then you won't have a weakness.")time.sleep(1)
> count = 500

<snip for brevity...>

> The error is: 
>  You have 500 skill points to spend on character development so use them wisely....


> Traceback (most recent call last):  
File "C:\Users\mcshizney\Desktop\adventuregame.py", line 29, in <module>
attack = int(input(
"Enter a number for the ammount of points you would like to designate to
your characters attack. You only have ",
count, " remaining and 4 skills to set. "))
TypeError: input expected at most 1 arguments, got 3             		 	   		


As it says you can only have one argument to input.
You need to construct your prompt outside the call then use that:

prompt = "Enter a number for the amount of points you would like to
designate to your characters attack. You only have " + count +
" remaining and 4 skills to set."

attack = int(input(prompt))

or similar.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From james at uplinkzero.com  Mon Jan 11 06:44:10 2016
From: james at uplinkzero.com (James Chapman)
Date: Mon, 11 Jan 2016 11:44:10 +0000
Subject: [Tutor] Question about the memory manager
Message-ID: <CAHvkzynngeV5GYs-0C2rexbj1=kZR+wv=F4EUGoezM3YsdSkaw@mail.gmail.com>

If you read the comment that goes with the code snippet pasted in the
original email it makes far more sense as the author is talking
specifically about out of memory errors...


"You already got excellent answers, I just wanted to add one more tip
that's served me well over the years in a variety of language for the
specific problem "how to cleanly diagnose, log, etc, out of memory
errors?". Problem is, if your code gets control before enough objects
have been destroyed and their memory recycled, memory might be too
tight to do propert logging, gui work, etc, etc -- how do we ensure
this doesn't happen?

Answer: build an emergency stash so you know you can spend it in such
emergencies:

rainydayfund = [[] for x in xrange(16*1024)]  # or however much you need

def handle_exception(e):
  global rainydayfund
  del rainydayfund
  ... etc, etc ...

" - Alex Martelli


--
James

From alan.gauld at btinternet.com  Mon Jan 11 07:15:39 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 11 Jan 2016 12:15:39 +0000
Subject: [Tutor] Question about the memory manager
In-Reply-To: <20160110161621.GV10854@ando.pearwood.info>
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
 <20160110161621.GV10854@ando.pearwood.info>
Message-ID: <n706db$1es$1@ger.gmane.org>

On 10/01/16 16:16, Steven D'Aprano wrote:

>> rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
>> def handle_exception(e):
>> global rainydayfund
>> del rainydayfund
>> ... etc, etc ...
> 
> I was going to write a scornful email about how useless this would be.

Me too.

> still think it's useless, but I see that the idea comes from Alex 
> Martelli, who normally knows what he is talking about, so that makes me 
> pause and think and perhaps do some experiments before commenting.

Me too. :-)

But I think that it definitely is heavily OS dependent.
It should work in most *nix environments the first time
you call the function. But on second call I'd expect
all bets to be off. And in most real-time OS's memory
goes right back to the OS pool - but even there it
would probably be available I guess, at least the first
time.

Maybe the theory is that if you get a memory error the
only sensible thing is just to log it and exit. In which
case you only ever call this once.

Steven, Did you try any experiments? I'm struggling
to come up with a reliable test scenario.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From oscar.j.benjamin at gmail.com  Mon Jan 11 10:10:47 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 11 Jan 2016 15:10:47 +0000
Subject: [Tutor] Question about the memory manager
In-Reply-To: <n706db$1es$1@ger.gmane.org>
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
 <20160110161621.GV10854@ando.pearwood.info>
 <n706db$1es$1@ger.gmane.org>
Message-ID: <CAHVvXxSPbVdWJNyuF9nMmFV81nbCKfbVzxnCD3Bcv2waaj+2kA@mail.gmail.com>

On 11 January 2016 at 12:15, Alan Gauld <alan.gauld at btinternet.com> wrote:
>
> But I think that it definitely is heavily OS dependent.
> It should work in most *nix environments the first time
> you call the function. But on second call I'd expect
> all bets to be off. And in most real-time OS's memory
> goes right back to the OS pool - but even there it
> would probably be available I guess, at least the first
> time.
>
> Maybe the theory is that if you get a memory error the
> only sensible thing is just to log it and exit. In which
> case you only ever call this once.
>
> Steven, Did you try any experiments? I'm struggling
> to come up with a reliable test scenario.

I can't even work out how you trigger a MemoryError on Linux (apart
from just raising one). I've tried a few ways to make the system run
out of memory and it just borks the system rather than raise any error
- I can only interrupt it with REISUB.

Here's a simple one:

$ python -c 'x = []; x.append(iter(x))'

(Make sure you save all your work before trying that!)

--
Oscar

From __peter__ at web.de  Mon Jan 11 10:40:14 2016
From: __peter__ at web.de (Peter Otten)
Date: Mon, 11 Jan 2016 16:40:14 +0100
Subject: [Tutor] Question about the memory manager
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
 <20160110161621.GV10854@ando.pearwood.info> <n706db$1es$1@ger.gmane.org>
 <CAHVvXxSPbVdWJNyuF9nMmFV81nbCKfbVzxnCD3Bcv2waaj+2kA@mail.gmail.com>
Message-ID: <n70icu$cm6$1@ger.gmane.org>

Oscar Benjamin wrote:

> On 11 January 2016 at 12:15, Alan Gauld <alan.gauld at btinternet.com> wrote:
>>
>> But I think that it definitely is heavily OS dependent.
>> It should work in most *nix environments the first time
>> you call the function. But on second call I'd expect
>> all bets to be off. And in most real-time OS's memory
>> goes right back to the OS pool - but even there it
>> would probably be available I guess, at least the first
>> time.
>>
>> Maybe the theory is that if you get a memory error the
>> only sensible thing is just to log it and exit. In which
>> case you only ever call this once.
>>
>> Steven, Did you try any experiments? I'm struggling
>> to come up with a reliable test scenario.
> 
> I can't even work out how you trigger a MemoryError on Linux (apart
> from just raising one). I've tried a few ways to make the system run
> out of memory and it just borks the system rather than raise any error
> - I can only interrupt it with REISUB.
> 
> Here's a simple one:
> 
> $ python -c 'x = []; x.append(iter(x))'
> 
> (Make sure you save all your work before trying that!)

You can set the interpreter on a diet:

$ ulimit -v 22000
$ python -c 'print "x"'
x
$ python -c 'print "x"*10**6'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
MemoryError



From oscar.j.benjamin at gmail.com  Mon Jan 11 11:02:09 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 11 Jan 2016 16:02:09 +0000
Subject: [Tutor] Question about the memory manager
In-Reply-To: <n70icu$cm6$1@ger.gmane.org>
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
 <20160110161621.GV10854@ando.pearwood.info>
 <n706db$1es$1@ger.gmane.org>
 <CAHVvXxSPbVdWJNyuF9nMmFV81nbCKfbVzxnCD3Bcv2waaj+2kA@mail.gmail.com>
 <n70icu$cm6$1@ger.gmane.org>
Message-ID: <CAHVvXxQR=WxFijK4zGNoyw1g0PUD5qBJfyEHsw7LEwxXDzn2kA@mail.gmail.com>

On 11 January 2016 at 15:40, Peter Otten <__peter__ at web.de> wrote:
>> I can't even work out how you trigger a MemoryError on Linux (apart
>> from just raising one). I've tried a few ways to make the system run
>> out of memory and it just borks the system rather than raise any error
>> - I can only interrupt it with REISUB.
>>
>> Here's a simple one:
>>
>> $ python -c 'x = []; x.append(iter(x))'
>>
>> (Make sure you save all your work before trying that!)
>
> You can set the interpreter on a diet:
>
> $ ulimit -v 22000
> $ python -c 'print "x"'
> x
> $ python -c 'print "x"*10**6'
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
> MemoryError

This didn't initially work for me:

$ ulimit -v 22000
$ python -c 'print "x"*10**6'
python: error while loading shared libraries: libc.so.6: failed to map
segment from shared object: Cannot allocate memory

I guess that limit's too low so (in a new terminal):

$ ulimit -v 50000
$ python -c '"x"*10**8'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
MemoryError
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line
66, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python2.7/dist-packages/apport/__init__.py", line 1,
in <module>
    from apport.report import Report
  File "/usr/lib/python2.7/dist-packages/apport/report.py", line 20, in <module>
    import apport.fileutils
  File "/usr/lib/python2.7/dist-packages/apport/fileutils.py", line
22, in <module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python2.7/dist-packages/apport/packaging_impl.py",
line 20, in <module>
    import apt
  File "/usr/lib/python2.7/dist-packages/apt/__init__.py", line 21, in <module>
    import apt_pkg
ImportError: /usr/lib/python2.7/dist-packages/apt_pkg.so: failed to
map segment from shared object: Cannot allocate memory

Original exception was:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
MemoryError

So it seems that it ran out of memory and sys.excepthook failed
because of insufficient memory which is the situation Alex Martelli
described. However:

$ python -c 'raise ValueError'
Traceback (most recent call last):
  ...
ImportError: /usr/lib/python2.7/dist-packages/apt_pkg.so: failed to
map segment from shared object: Cannot allocate memory

Original exception was:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError

It seems I need a ulimit of 60000 to get this to work properly:

$ ulimit -v 60000
$ python -c 'raise ValueError'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError
$ python -c '"x"*10**8'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
MemoryError

--
Oscar

From PNiewosz at slb.com  Mon Jan 11 11:12:32 2016
From: PNiewosz at slb.com (Patrycja Niewosz)
Date: Mon, 11 Jan 2016 16:12:32 +0000
Subject: [Tutor] Thread output to GUI Text Field
Message-ID: <2B14D1AD04128B4AA0B29D4096FC649A0161F7FB33@NL0230MBX03N2.DIR.slb.com>

Hi All,

I use commands,

        server_address = ('134.32.45.9', 10000)
        print ('starting up on port '% server_address)
        sock.bind(server_address)

the message pop ups in the window command when I run full code, however I would like to print this message in the main GUI, where I have defined

tcpControlGrid.addWidget(QtGui.QLabel('TCP Server Messages'),3,0)
tcpControlGrid.addWidget(self.tcpControlMessages,4,0)

self.tcpControlMessages = QtGui.QPlainTextEdit()

How can I transfer messages from above code directly to the GUI interface, so that I can see on main window.


Many thanks for help.
Patrycja

From secmailinglist at gmail.com  Mon Jan 11 07:51:07 2016
From: secmailinglist at gmail.com (Rene Werner)
Date: Mon, 11 Jan 2016 13:51:07 +0100
Subject: [Tutor] Organizing files
Message-ID: <CAC3VOCfCvW2p-p_8yP+av+N1fZ2cUmSvWxihyC7vLYMXq_EDkg@mail.gmail.com>

Hello list,

right now I am working on a couple of programming-related challenges. The
challenges are sorted into problem sets, where each set contains a number
of problems.

Because a lot of these problems rely on code that is often the same, I have
put these parts into a seperate file, util.py, and I simply do

from util import *

in each solution. I have my solutions organized in different folders, one
for each problem set:

set1/prob1.py
set1/prob2.py
set2/prob3.py

and so on. So far, I keep symlinking util.py in each set folder, so e.g.
set1/util.py is a symlink to ../util.py. The same goes for set2, set3 and
so on. This works, but I get the feeling that it is clumsy and could be
done better.

What is the usual way in python do realize this in a way so that I don't
have to symlink util.py for every problem set, but still have it available?

Thank you very much for your time.
Kind regards

M.M.

From steven.molter at gmail.com  Mon Jan 11 11:40:43 2016
From: steven.molter at gmail.com (Steven Molter)
Date: Mon, 11 Jan 2016 10:40:43 -0600
Subject: [Tutor] 12 year old starting to code
Message-ID: <CAFkFBpw2x0xAs1iPahBfUo7ZqOe5-i55MNfOxfwxTVSQxQGhgQ@mail.gmail.com>

I am homeschooling my son Spencer, and he is interested in learning how to
write code.  We purchased a book by DK called "Help Your Kids With Computer
Coding".  We have done the section on scratch, and now have started the
Python section.  The book was written using IDLE, but with the newest
version of Python, I am unable to get the IDLE shell running.

If someone could help me get the IDLE shell running that would be a great
start, but any advice on how to help my son learn to code would be greatly
appreciated.

Thanks to all,
Steve Molter

From alan.gauld at btinternet.com  Mon Jan 11 13:33:27 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 11 Jan 2016 18:33:27 +0000
Subject: [Tutor] 12 year old starting to code
In-Reply-To: <CAFkFBpw2x0xAs1iPahBfUo7ZqOe5-i55MNfOxfwxTVSQxQGhgQ@mail.gmail.com>
References: <CAFkFBpw2x0xAs1iPahBfUo7ZqOe5-i55MNfOxfwxTVSQxQGhgQ@mail.gmail.com>
Message-ID: <n70shn$dg$1@ger.gmane.org>

On 11/01/16 16:40, Steven Molter wrote:

> Python section.  The book was written using IDLE, but with the newest
> version of Python, I am unable to get the IDLE shell running.

Which OS do you use? (and which specific Python version?)

If it's Windows you may prefer the ActiveState.com
distribution which includes some Windoze friendly extras,
including a different (and better) IDE called Pythonwin.

If its not Windows then you may need to install a separate
package on your system or get a specific download with IDLE
(or Tkinter which is what IDLE is built from) included.

> If someone could help me get the IDLE shell running that would be a great
> start, but any advice on how to help my son learn to code would be greatly
> appreciated.

Since you have a book just follow that and if you run into
things that don;t work as expected ask here.

Please always post full error messages(cut n paste) along
with the code.

And post in plain text since otherwise the code format
gets messed up.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Mon Jan 11 13:36:54 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 11 Jan 2016 18:36:54 +0000
Subject: [Tutor] Thread output to GUI Text Field
In-Reply-To: <2B14D1AD04128B4AA0B29D4096FC649A0161F7FB33@NL0230MBX03N2.DIR.slb.com>
References: <2B14D1AD04128B4AA0B29D4096FC649A0161F7FB33@NL0230MBX03N2.DIR.slb.com>
Message-ID: <n70so5$3s2$1@ger.gmane.org>

On 11/01/16 16:12, Patrycja Niewosz wrote:
> Hi All,
> 
> I use commands,
> 
>         server_address = ('134.32.45.9', 10000)
>         print ('starting up on port '% server_address)
>         sock.bind(server_address)
> 
> the message pop ups in the window command when I run full code, however I would like to print this message in the main GUI, where I have defined
> 
> tcpControlGrid.addWidget(QtGui.QLabel('TCP Server Messages'),3,0)
> tcpControlGrid.addWidget(self.tcpControlMessages,4,0)
> 
> self.tcpControlMessages = QtGui.QPlainTextEdit()
> 
> How can I transfer messages from above code directly to the GUI interface, so that I can see on main window.

Since this is obviously a PyQt issue you would be best asking on a
Qt or SIDE forum. However most UIs have a Text control(*) that you
can insert text into. So look for a method of your control
called something like insert() or put() or maybe even write().

(*)I'm assuming your QtGui.QPlainTextEdit() is the Qt version.
What methods does it have?

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From akleider at sonic.net  Mon Jan 11 14:11:50 2016
From: akleider at sonic.net (Alex Kleider)
Date: Mon, 11 Jan 2016 11:11:50 -0800
Subject: [Tutor] Organizing files
In-Reply-To: <CAC3VOCfCvW2p-p_8yP+av+N1fZ2cUmSvWxihyC7vLYMXq_EDkg@mail.gmail.com>
References: <CAC3VOCfCvW2p-p_8yP+av+N1fZ2cUmSvWxihyC7vLYMXq_EDkg@mail.gmail.com>
Message-ID: <f50264deb84b6e911fc32a9d60a7f95c@sonic.net>

On 2016-01-11 04:51, Rene Werner wrote:
> Hello list,
> 
> right now I am working on a couple of programming-related challenges. 
> The
> challenges are sorted into problem sets, where each set contains a 
> number
> of problems.
> 
> Because a lot of these problems rely on code that is often the same, I 
> have
> put these parts into a seperate file, util.py, and I simply do
> 
> from util import *
> 
> in each solution. I have my solutions organized in different folders, 
> one
> for each problem set:
> 
> set1/prob1.py
> set1/prob2.py
> set2/prob3.py
> 
> and so on. So far, I keep symlinking util.py in each set folder, so 
> e.g.
> set1/util.py is a symlink to ../util.py. The same goes for set2, set3 
> and
> so on. This works, but I get the feeling that it is clumsy and could be
> done better.
> 
> What is the usual way in python do realize this in a way so that I 
> don't
> have to symlink util.py for every problem set, but still have it 
> available?
> 
> Thank you very much for your time.
> Kind regards
> 
> M.M.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

I used to do it with symlinks as well but what I do now I believe
is the preferred way.

I have all my Python work under a ~/Py directory (in my /home/alex 
folder)
For each project I create a subfolder there: ~/Py/CurrentProject, or 
~/Py/CP for short.
Under ~/Py/CP you can put all your other code organized in what ever way 
suits you best.
The CP directory and every directory under it should contain an empty 
file called __init__.py.
For example:
     ~/Py/CP
         __init__.py
         src
             __init__.py
             main.py
         tests
             __init__.py
             test.py

test.py can import main.py with the following import statement:
import CP.src.main as main
as long as ~/Py is in your PYTHONPATH environment variable.
This can be set dynamically in your code but I do it by having the 
following in my ~/.bashrc file:
export PYTHONPATH="${PYTHONPATH:+$PYTHONPATH:}/home/alex/Py"

I'm submitting this not because I am an expert, but because this is the 
way I've learned to do it from postings on this list (for which I would 
like again to express gratitude) and would be interested in hearing if I 
got it right.  So in a sense, it is a question rather than an answer.
ak

From cs at zip.com.au  Mon Jan 11 15:03:53 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 12 Jan 2016 07:03:53 +1100
Subject: [Tutor] Organizing files
In-Reply-To: <CAC3VOCfCvW2p-p_8yP+av+N1fZ2cUmSvWxihyC7vLYMXq_EDkg@mail.gmail.com>
References: <CAC3VOCfCvW2p-p_8yP+av+N1fZ2cUmSvWxihyC7vLYMXq_EDkg@mail.gmail.com>
Message-ID: <20160111200353.GA59853@cskk.homeip.net>

On 11Jan2016 13:51, Rene Werner <secmailinglist at gmail.com> wrote:
>right now I am working on a couple of programming-related challenges. The
>challenges are sorted into problem sets, where each set contains a number
>of problems.
>
>Because a lot of these problems rely on code that is often the same, I have
>put these parts into a seperate file, util.py, and I simply do
>
>from util import *

Remark: you are better off importing only specific names:

  from util import this, that, the_other

otherwise you pollute your namespace to lots of junk. While you've only got one 
module your version works, but it stops being useful pretty soon after you 
start importing more things.

>in each solution. I have my solutions organized in different folders, one
>for each problem set:
>
>set1/prob1.py
>set1/prob2.py
>set2/prob3.py
>
>and so on. So far, I keep symlinking util.py in each set folder, so e.g.
>set1/util.py is a symlink to ../util.py. The same goes for set2, set3 and
>so on. This works, but I get the feeling that it is clumsy and could be
>done better.
>
>What is the usual way in python do realize this in a way so that I don't
>have to symlink util.py for every problem set, but still have it available?

The normal way is to have your own module library and modify $PYTHONPATH to 
consult it. You might have a $HOME/python_modules directory with util.py inside 
it (or whatever other modules). Put:

  PYTHONPATH=$HOME/python_modules
  export PYTHONPATH

in your environment. Then Python will know to look in your python_modules 
directory for modules ahead of other places and you will not need the symlinks 
(which are fragile anyway).

Cheers,
Cameron Simpson <cs at zip.com.au>

From alan.gauld at btinternet.com  Mon Jan 11 19:11:12 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 12 Jan 2016 00:11:12 +0000
Subject: [Tutor] 12 year old starting to code
In-Reply-To: <CAFkFBpyi0S1xuHbw-8TeL1BOgvywr_q-RBRT-9Qbs89Ry6F04Q@mail.gmail.com>
References: <CAFkFBpw2x0xAs1iPahBfUo7ZqOe5-i55MNfOxfwxTVSQxQGhgQ@mail.gmail.com>
 <n70shn$dg$1@ger.gmane.org>
 <CAFkFBpyi0S1xuHbw-8TeL1BOgvywr_q-RBRT-9Qbs89Ry6F04Q@mail.gmail.com>
Message-ID: <569444A0.9060202@btinternet.com>

CCd to list for info

On 11/01/16 22:55, Steven Molter wrote:
> I'm running Python 3.5.1 on windows 10.

OK, It should be in C:\Python35\Lib\idelib

Look for a file called idle.bat and create a shortcut to
that from your menu or desktop.

If you aren't sure where the start menu items are, open the
All Application->Python submenu, pick any entry and left-click.
Select More-> open file location...

A Windows explorer window will open and you just drag
your new shortcut into that folder, rename it to whatever
you want (My Activestate version says "IDLE(Python GUI)" )


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


From alan.gauld at btinternet.com  Tue Jan 12 04:44:53 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 12 Jan 2016 09:44:53 +0000
Subject: [Tutor] Fwd: RE:  Thread output to GUI Text Field
In-Reply-To: <2B14D1AD04128B4AA0B29D4096FC649A0161F7FC4C@NL0230MBX03N2.DIR.slb.com>
References: <2B14D1AD04128B4AA0B29D4096FC649A0161F7FC4C@NL0230MBX03N2.DIR.slb.com>
Message-ID: <5694CB15.9060706@btinternet.com>


Forwarding to list.
Please use Reply-All or Rely to List when responding to tutor messages.

You didn't mention earlier that you were using threads!
That significantly complicates matters.


-------- Forwarded Message --------
Subject: 	RE: [Tutor] Thread output to GUI Text Field
Date: 	Tue, 12 Jan 2016 08:52:42 +0000
From: 	Patrycja Niewosz <PNiewosz at slb.com>
To: 	Alan Gauld <alan.gauld at btinternet.com>



Hi Alan,

Thanks for a reply.
I have found that I need to use slots and signal to pass data between Threads and hence 


class Panel(QtGui.QWidget):
def __init__(self,master,parent=None):
.....
.....
self.initUI()


def initUI(self):
#Build layout
self.tcpControlMessages = QtGui.QPlainTextEdit()

def startThreadedProcess(self):
        self. tcpControlMessage.clear()
        self.thread.start()

@QtCore.Slot()
 def updateMessages(self, message):
        print message
        self. tcpControlMessages.appendPlainText(message)


Patrycja


-----Original Message-----
From: Tutor [mailto:tutor-bounces+pniewosz=slb.com at python.org] On Behalf Of Alan Gauld
Sent: Monday, January 11, 2016 6:37 PM
To: tutor at python.org
Subject: Re: [Tutor] Thread output to GUI Text Field

On 11/01/16 16:12, Patrycja Niewosz wrote:
> Hi All,
> 
> I use commands,
> 
>         server_address = ('134.32.45.9', 10000)
>         print ('starting up on port '% server_address)
>         sock.bind(server_address)
> 
> the message pop ups in the window command when I run full code, 
> however I would like to print this message in the main GUI, where I 
> have defined
> 
> tcpControlGrid.addWidget(QtGui.QLabel('TCP Server Messages'),3,0)
> tcpControlGrid.addWidget(self.tcpControlMessages,4,0)
> 
> self.tcpControlMessages = QtGui.QPlainTextEdit()
> 
> How can I transfer messages from above code directly to the GUI interface, so that I can see on main window.

Since this is obviously a PyQt issue you would be best asking on a Qt or SIDE forum. However most UIs have a Text control(*) that you can insert text into. So look for a method of your control called something like insert() or put() or maybe even write().

(*)I'm assuming your QtGui.QPlainTextEdit() is the Qt version.
What methods does it have?

--
Alan G
Author of the Learn to Program web site
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.alan-2Dg.me.uk_&d=CwICAg&c=uGuXJ43KPkPWEl2imVFDmZQlhQUET7pVRA2PDIOxgqw&r=VlCD52JbZNT0daOyAL7vAA&m=lMWz8Zwkrgc7CBzDOXEXkb8oTuIeuRsGgutRf32psIc&s=utKS1TTBmN6wmUJ4SnWJcRbNjIddYO09HyTV2cQUwj8&e=
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.amazon.com_author_alan-5Fgauld&d=CwICAg&c=uGuXJ43KPkPWEl2imVFDmZQlhQUET7pVRA2PDIOxgqw&r=VlCD52JbZNT0daOyAL7vAA&m=lMWz8Zwkrgc7CBzDOXEXkb8oTuIeuRsGgutRf32psIc&s=jXyzhQxcanJnO9e1u-yPXFqlRSgVfDu-CVVnYHsPrOM&e=
Follow my photo-blog on Flickr at:
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.flickr.com_photos_alangauldphotos&d=CwICAg&c=uGuXJ43KPkPWEl2imVFDmZQlhQUET7pVRA2PDIOxgqw&r=VlCD52JbZNT0daOyAL7vAA&m=lMWz8Zwkrgc7CBzDOXEXkb8oTuIeuRsGgutRf32psIc&s=zDdjUw8RFbWAtyLe1ywjgC40UayySD5H6fzQAlZN5N8&e= 


_______________________________________________
Tutor maillist  -  Tutor at python.org
To unsubscribe or change subscription options:
https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman_listinfo_tutor&d=CwICAg&c=uGuXJ43KPkPWEl2imVFDmZQlhQUET7pVRA2PDIOxgqw&r=VlCD52JbZNT0daOyAL7vAA&m=lMWz8Zwkrgc7CBzDOXEXkb8oTuIeuRsGgutRf32psIc&s=afbUD2-GlmdFUE2rIbOMtKiOxKEZSqDi0Z85FSoldpM&e= 




From alan.gauld at btinternet.com  Tue Jan 12 06:32:17 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 12 Jan 2016 11:32:17 +0000
Subject: [Tutor] Fwd: RE: Thread output to GUI Text Field
In-Reply-To: <5694CB15.9060706@btinternet.com>
References: <2B14D1AD04128B4AA0B29D4096FC649A0161F7FC4C@NL0230MBX03N2.DIR.slb.com>
 <5694CB15.9060706@btinternet.com>
Message-ID: <n72o81$a3q$1@ger.gmane.org>

On 12/01/16 09:44, Alan Gauld wrote:

> You didn't mention earlier that you were using threads!
> That significantly complicates matters.

Apologies, I just noticed that the subject line does,
although it wasn't clear that you were trying to output
between threads.

> I have found that I need to use slots and signal to pass data between Threads and hence 

Does that mean you have solved the problem?
Or are you still looking for help?
If so what exactly do you want to know?
Do you have some code that doesn't work for example?

> class Panel(QtGui.QWidget):
> def __init__(self,master,parent=None):
> .....
> .....
> self.initUI()
> 
> 
> def initUI(self):
> #Build layout
> self.tcpControlMessages = QtGui.QPlainTextEdit()
> 
> def startThreadedProcess(self):
>         self. tcpControlMessage.clear()
>         self.thread.start()
> 
> @QtCore.Slot()
>  def updateMessages(self, message):
>         print message
>         self. tcpControlMessages.appendPlainText(message)

We can help with the threading aspects but for Qt specifics
you will probably do better on a PyQt forum.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From cegarcia0323 at gmail.com  Tue Jan 12 15:36:47 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Tue, 12 Jan 2016 15:36:47 -0500
Subject: [Tutor] HELP!!
Message-ID: <CABV0pWzgnoWsg1r0uqFNb2yeP68nT5dtAUL5Sh5PyTZ=xb_qEw@mail.gmail.com>

So I am importing a csv file to textfile and the first column is product
names and the second is a description of each. I am trying to find the sum
of all descriptions using sum(c.values)). Here is my code so far:

import csv
import json
import sys
#from sys import argv
from collections import defaultdict
from collections import Counter
#script, filename = argv
data = defaultdict(list)
counted_data = defaultdict(list)
grouped_data = defaultdict(list)

class dictionary:
with open ('weekly_20160102.csv', 'rb') as f:
reader = csv.reader(f)
next(reader, None)
for row in reader:
data[row[2]].append(row[3]) #this appends the description(value) to the
product name(key)
#new_item = {}
for key in data.keys():
#print key
c = Counter(data[key])
for value in c:
#print c[value]
if c[value] >= 5:
print key, ':', value, ':', c[value]
elif c[value] <= 1:
print 'other', ':', sum(c.values())


Please help!!

From joel.goldstick at gmail.com  Tue Jan 12 17:56:44 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 12 Jan 2016 17:56:44 -0500
Subject: [Tutor] HELP!!
In-Reply-To: <CABV0pWzgnoWsg1r0uqFNb2yeP68nT5dtAUL5Sh5PyTZ=xb_qEw@mail.gmail.com>
References: <CABV0pWzgnoWsg1r0uqFNb2yeP68nT5dtAUL5Sh5PyTZ=xb_qEw@mail.gmail.com>
Message-ID: <CAPM-O+zotHp0rYH9W8LiWeXnUzhJ-OECYO=_5NipOS=gphZU=Q@mail.gmail.com>

On Tue, Jan 12, 2016 at 3:36 PM, Chelsea G <cegarcia0323 at gmail.com> wrote:

> So I am importing a csv file to textfile and the first column is product
> names and the second is a description of each. I am trying to find the sum
> of all descriptions using sum(c.values)). Here is my code so far:
>
> import csv
> import json
> import sys
> #from sys import argv
> from collections import defaultdict
> from collections import Counter
> #script, filename = argv
> data = defaultdict(list)
> counted_data = defaultdict(list)
> grouped_data = defaultdict(list)
>
> class dictionary:
> with open ('weekly_20160102.csv', 'rb') as f:
> reader = csv.reader(f)
> next(reader, None)
> for row in reader:
> data[row[2]].append(row[3]) #this appends the description(value) to the
> product name(key)
> #new_item = {}
> for key in data.keys():
> #print key
> c = Counter(data[key])
> for value in c:
> #print c[value]
> if c[value] >= 5:
> print key, ':', value, ':', c[value]
> elif c[value] <= 1:
> print 'other', ':', sum(c.values())
>
>
> Please help!!
>

First of all, turn off your rich text mail editor -- we can only help if we
see the indentation.  Send Text only to this list.
You define a class, but never instantiate it.  what is Counter()?  Where is
it defined?

This is awfully complicated.  If your csv file looks like this:

name1, 5
name3, 54
name2, 3
name1, 4

you can read the file line by line, split on , and add the second value to
a dict with the first value as a key.  You will need to check for the
exception of their being no key with that value (the first time it appears).

But first, resend in plain text

> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From dyoo at hashcollision.org  Tue Jan 12 18:45:56 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Tue, 12 Jan 2016 15:45:56 -0800
Subject: [Tutor] HELP!!
In-Reply-To: <CABV0pWzgnoWsg1r0uqFNb2yeP68nT5dtAUL5Sh5PyTZ=xb_qEw@mail.gmail.com>
References: <CABV0pWzgnoWsg1r0uqFNb2yeP68nT5dtAUL5Sh5PyTZ=xb_qEw@mail.gmail.com>
Message-ID: <CAGZAPF47ucczSFy8mSAV_Wmf2-KbVkXP0B8Ekjyp7B+gz_G-qQ@mail.gmail.com>

On Tue, Jan 12, 2016 at 12:36 PM, Chelsea G <cegarcia0323 at gmail.com> wrote:
> So I am importing a csv file to textfile and the first column is product
> names and the second is a description of each. I am trying to find the sum
> of all descriptions using sum(c.values)). Here is my code so far:

[code cut]

Code is sometimes useful, but it's also very important to express what
you see is going wrong.  Or, at the very least, try to explain the
problem as best as possible.

At the moment, all you've said is, effectively:

    1.  I have this program.
    2.  I need help!!!

Here's why these two pieces are often not enough.

1.  If your program can be executed by us, it may not be obvious or
clear that there's anything wrong with the program.  If your program
doesn't run, that's also no guarantee that we can do anything
effective.

For example, if we don't have your input file 'weekly_20160102.csv',
we can't determine whether your input is well-formed or not: perhaps
the input itself is wrong, and your program might be perfectly ok?

Or maybe it runs, but returns the wrong output.  How could we tell,
though?  What it means to have "wrong" output is something that's in
your head, and because we are not mind readers, we may not know what
"right" output looks like.

Or maybe it doesn't run.  In that case, if you show us the error, we
have an opportunity to explain what the error is trying to say, so
that you can understand what it means.  If you show us the error, then
we're more assured that we both are looking at the same thing.


The key thing here is: when you're asking for help on a program, try
to state up front what correct output looks like.  If the input is
big, then take a subset of the input, simplify it if necessary, and
present that to us.  Finally, present what you want the expected
result to look like.  Concrete is best.  When you say: " I am trying
to find the sum of all descriptions", I have to truthfully say that I
have no idea what this means.  Try expressing the idea concretely in
terms of what the input looks like, and what the output looks like.


2.  I know this will sound funny, but desperation doesn't work very
well as a strategy to get complicated things done.  Try to calm down.

If one is in "panic mode", then that person's brain is in a
flight-or-flight reactive state.  That would be productive if one
could run physically fast enough to get a program fixed.  But
programming is just not athletic.  Programming, and learning in
general, is something that's easier to do when you're not struggling
with immediate survival.

I'm not joking: I'm being very serious.  Please try to de-escalate the
stakes: don't make this a life-or-death situation.


Good luck!

From alan.gauld at btinternet.com  Tue Jan 12 18:57:09 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 12 Jan 2016 23:57:09 +0000
Subject: [Tutor] HELP!!
In-Reply-To: <CABV0pWzgnoWsg1r0uqFNb2yeP68nT5dtAUL5Sh5PyTZ=xb_qEw@mail.gmail.com>
References: <CABV0pWzgnoWsg1r0uqFNb2yeP68nT5dtAUL5Sh5PyTZ=xb_qEw@mail.gmail.com>
Message-ID: <n743sm$4sk$1@ger.gmane.org>

On 12/01/16 20:36, Chelsea G wrote:

> data = defaultdict(list)
> 
> class dictionary:
>    with open ('weekly_20160102.csv', 'rb') as f:
>       reader = csv.reader(f)
>       next(reader, None)
>       for row in reader:
>           data[row[2]].append(row[3]) 
>       for key in data.keys():
>           c = Counter(data[key])
>           for value in c:
>              if c[value] >= 5:
>                 print key, ':', value, ':', c[value]
>              elif c[value] <= 1:
>                 print 'other', ':', sum(c.values())

I think that's what your code was supposed to look like.
Am I right?

If so you have all the code inside a class definition.
But you do nothing with the class and it certainly doesn't
define a dictionary. Its not clear what the class line is for.

That aside, This implies the data format has 4 columns
and you are throwing away the first two and using
the third as a key to find the fourth.
Is that correct? It's certainly not what your introductory
paragraph said.

But I don't understand what you think the Counter is doing?
You are counting how many there are of each key, but a dict
can only have unique keys so it must always be 1.

Then you iterate over the Counter and none of that makes
any sense. I can't even think what you are trying to do.

Can you confirm the code layout - ideally by posting a plain
text version. And explain how you think it is supposed to
work.

Also please show us a sample (3-10 lines) of input data
and what exactly you expect the output to be.

Thanks,

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From sjeik_appie at hotmail.com  Wed Jan 13 03:01:52 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Wed, 13 Jan 2016 08:01:52 +0000
Subject: [Tutor] Question about the memory manager
In-Reply-To: <n6u4d2$2na$1@ger.gmane.org>
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>,
 <n6u4d2$2na$1@ger.gmane.org>
Message-ID: <DUB123-W36EB7F3203E4975496D2E983CB0@phx.gbl>

> To: tutor at python.org
> From: __peter__ at web.de
> Date: Sun, 10 Jan 2016 18:29:06 +0100
> Subject: Re: [Tutor] Question about the memory manager
> 
> Albert-Jan Roskam wrote:
> 
> > Hi,
> > 
> > I just found a neat trick to free up an emergency stash of memory in a
> > funtion that overrides sys.excepthook. The rationale is that all
> > exceptions, including MemoryErrors will be logged. The code is below. My
> > question: is that memory *guaranteed* to be freed right after the 'del'
> > statement? Or should one call gc.collect to be really sure?
> > 
> > rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
> > def handle_exception(e):
> > global rainydayfund
> > del rainydayfund
> > ... etc, etc ...
> > http://stackoverflow.com/questions/1235349/python-how-can-i-handle-any-unhandled-exception-in-an-alternative-way
> 
> I must admit that this looks rather strange to me, but I don't see any 
> problem why it wouldn't work. gc.collect() only helps with circular 
> dependencies, and there aren't any in your "rainy day fund".
> 
> Regarding the use of sys.excepthook I'd rather (SO voting be damned!) take 
> the obvious approach as suggested by Vinay Sajip, i. e.
> 
> try:
>     main()
> except:
>     # do what you have to do
> 
> instead of rewriting the hook. If potential memory resource hogs are 
> confined within main() with no references from any module namespace you 
> should have enough memory availaible to run the except suite without the 
> need for a rainy day fund. If you know about a specific global name that 
> references a consumer of a lot of memory, an in-memory db, say, then why not 
> handle the memory error there or at least use it in lieu of a dedicated 
> dummy? I. e.
> 
> in_memory_db = None
> try:
>     try:
>         main()
>     finally:
>         del in_memory_db
> except:
>     # do what you have to do

Hi all,

Thanks for your replies. Initially my main concern was to also log fatal/unhandled exceptions. The database MemoryError was also I was also an issue I was just facing, so Martelli's approach seemed nice. Meanwhile I found out I did not only delete "TOP 100" (which I used to debug my code) in my SELECT query, but also DISTINCT. After I corrected this MemoryErrors are no longer an issue with this function. :-) Still, I always wonder how close I am to a MemoryError (I am on a tiny Win 7 32 VM, with Python 2.7). I sometimes check the Task Manager to see how much RAM is in use. Is there a Python way to do something similar?

Best wishes,
Albert-Jan


 		 	   		  

From sjeik_appie at hotmail.com  Wed Jan 13 03:11:11 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Wed, 13 Jan 2016 08:11:11 +0000
Subject: [Tutor] Question about the memory manager
In-Reply-To: <CAExdVN=Fpw7x0pMRNT+JCFsq4PoWWgrooMU3qTjqpZ7-nqrWYQ@mail.gmail.com>
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>,
 <CAExdVN=Fpw7x0pMRNT+JCFsq4PoWWgrooMU3qTjqpZ7-nqrWYQ@mail.gmail.com>
Message-ID: <DUB123-W1156BF8488894A8EC1523783CB0@phx.gbl>

> From: tim.peters at gmail.com
> Date: Sun, 10 Jan 2016 10:54:10 -0600
> Subject: Re: [Tutor] Question about the memory manager
> To: sjeik_appie at hotmail.com
> CC: tutor at python.org
> 
> [Albert-Jan Roskam <sjeik_appie at hotmail.com>]
> > I just found a neat trick to free up an emergency stash of memory in
> > a funtion that overrides sys.excepthook. The rationale is that all
> > exceptions, including MemoryErrors will be logged.
> > The code is below. My question: is that memory *guaranteed* to be
> > freed right after the 'del' statement? Or should one call gc.collect to
> > be really sure?
> >
> > rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
> > def handle_exception(e):
> >     global rainydayfund
> >     del rainydayfund
> > ... etc, etc ...
> > http://stackoverflow.com/questions/1235349/python-how-can-i-handle-any-unhandled-exception-in-an-alternative-way
> 
> This works fine in all versions of CPython (the C implementation of
> Python distributed by python.org) to date.  That's because:
> 
> 1. All versions of CPython rely primarily on reference counting (`gc`
> is only needed to reclaim garbage containing reference cycles).  An
> object is released immediately when its reference count falls to 0.
> 
> 2. There is only one reference to the big list there (via the global
> `raindydayfund`), so the memory becomes garbage immediately upon
> executing the `del`.
> 
> 3. Similarly, that giant list holds the only references to the masses
> of distinct empty lists it contains, so they also become garbage
> immediately upon the giant list becoming garbage.
> 
> 4. CPython doesn't happen to stick garbage lists in, e.g., some
> internal free list reusable only for new list objects - it actually
> releases the memory for garbage lists.  Kinda ;-)
> 
> #2 and #3 are necessarily true.  #1 is true in CPython, but not in all
> implementations of Python.
> 
> #4 is where things _might_ change even in CPython, but it's very
> unlikely to change.  As is, it would take a small book to flesh out
> what "Kinda ;-)" means, exactly.  Memory management is complex, with
> many layers, involving many details.
> 
> If you can live with all that, I'd suggest a more straightforward way
> of setting it up, like:
> 
> rainydayfund  = b"x" * N
> 
> where `N` is the number of bytes you want to reserve.  That is, create
> a giant bytestring containing the number of "emergency bytes" you
> need.  If N is large enough, that will avoid CPython's "small object
> allocator" and CPython's "arena allocator", getting the memory
> directly from (and returning the memory directly to) the OS.  The
> fewer layers that get involved, the fewer layers that _may_ surprise
> you by changing behavior in the future.

Hi Tim,

Thank you! Can you recommend a document or website that CPython's memory manager?
Might be interesting and perhaps useful to know a bit more about the details. Perhaps this knowledge might sometimes help writing faster code?

Best wishes,
Albert-Jan

PS:

albertjan at debian:~$ python -c "import this" 
The Zen of Python, by Tim Peters

...
...
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.

--> Nope not even then. Perhaps if my name were Guido? :-)



 		 	   		  

From eryksun at gmail.com  Wed Jan 13 05:31:58 2016
From: eryksun at gmail.com (eryk sun)
Date: Wed, 13 Jan 2016 04:31:58 -0600
Subject: [Tutor] Question about the memory manager
In-Reply-To: <DUB123-W36EB7F3203E4975496D2E983CB0@phx.gbl>
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
 <n6u4d2$2na$1@ger.gmane.org>
 <DUB123-W36EB7F3203E4975496D2E983CB0@phx.gbl>
Message-ID: <CACL+1asnWjQ0-zs=7B0Q9nZHU4JN_8ANZ_TEYZUGATXqq_oegg@mail.gmail.com>

On Wed, Jan 13, 2016 at 2:01 AM, Albert-Jan Roskam
<sjeik_appie at hotmail.com> wrote:
> I sometimes check the Task Manager to see how much RAM is in use. Is there a Python way to do something similar?

Use the psutil module to solve this problem across platforms. For
Windows only, you can use ctypes to call GetProcessMemoryInfo. Here's
a function that returns the total working set size (including shared
pages) and the private working set.

    import ctypes
    from ctypes import wintypes
    import collections

    kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
    psapi = ctypes.WinDLL('psapi', use_last_error=True)

    wintypes.SIZE_T = ctypes.c_size_t

    class PROCESS_MEMORY_COUNTERS(ctypes.Structure):
        _fields_ = (('cb',                         wintypes.DWORD),
                    ('PageFaultCount',             wintypes.DWORD),
                    ('PeakWorkingSetSize',         wintypes.SIZE_T),
                    ('WorkingSetSize',             wintypes.SIZE_T),
                    ('QuotaPeakPagedPoolUsage',    wintypes.SIZE_T),
                    ('QuotaPagedPoolUsage',        wintypes.SIZE_T),
                    ('QuotaPeakNonPagedPoolUsage', wintypes.SIZE_T),
                    ('QuotaNonPagedPoolUsage',     wintypes.SIZE_T),
                    ('PagefileUsage',              wintypes.SIZE_T),
                    ('PeakPagefileUsage',          wintypes.SIZE_T))
        def __init__(self, *args, **kwds):
            super(PROCESS_MEMORY_COUNTERS, self).__init__(*args, **kwds)
            self.cb = ctypes.sizeof(self)

    class PROCESS_MEMORY_COUNTERS_EX(PROCESS_MEMORY_COUNTERS):
        _fields_ = (('PrivateUsage', wintypes.SIZE_T),)

    PPROCESS_MEMORY_COUNTERS = ctypes.POINTER(PROCESS_MEMORY_COUNTERS)

    def check_bool(result, func, args):
        if not result:
            raise ctypes.WinError(ctypes.get_last_error())
        return args

    kernel32.GetCurrentProcess.restype = wintypes.HANDLE

    psapi.GetProcessMemoryInfo.errcheck = check_bool
    psapi.GetProcessMemoryInfo.argtypes = (wintypes.HANDLE,
                                           PPROCESS_MEMORY_COUNTERS,
                                           wintypes.DWORD)

    MemoryUsage = collections.namedtuple('MemoryUsage', 'total private')

    def get_memory_usage():
        info = PROCESS_MEMORY_COUNTERS_EX()
        psapi.GetProcessMemoryInfo(kernel32.GetCurrentProcess(),
                                   ctypes.byref(info),
                                   ctypes.sizeof(info))
        if info.PrivateUsage:
            return MemoryUsage(info.WorkingSetSize, info.PrivateUsage)
        else:
            # prior to Windows 7.
            return MemoryUsage(info.WorkingSetSize, info.PagefileUsage)

From breamoreboy at yahoo.co.uk  Wed Jan 13 09:51:32 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 Jan 2016 14:51:32 +0000
Subject: [Tutor] 12 year old starting to code
In-Reply-To: <569444A0.9060202@btinternet.com>
References: <CAFkFBpw2x0xAs1iPahBfUo7ZqOe5-i55MNfOxfwxTVSQxQGhgQ@mail.gmail.com>
 <n70shn$dg$1@ger.gmane.org>
 <CAFkFBpyi0S1xuHbw-8TeL1BOgvywr_q-RBRT-9Qbs89Ry6F04Q@mail.gmail.com>
 <569444A0.9060202@btinternet.com>
Message-ID: <n75oah$daq$1@ger.gmane.org>

On 12/01/2016 00:11, Alan Gauld wrote:
> CCd to list for info
>
> On 11/01/16 22:55, Steven Molter wrote:
>> I'm running Python 3.5.1 on windows 10.
>
> OK, It should be in C:\Python35\Lib\idelib
>

That's not the default for 3.5, see 
https://docs.python.org/3/using/windows.html

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From breamoreboy at yahoo.co.uk  Wed Jan 13 12:53:06 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 Jan 2016 17:53:06 +0000
Subject: [Tutor] PLEASE I NEED HELP URGENTLY
In-Reply-To: <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
References: <911299850.2554269.1452369297130.JavaMail.yahoo.ref@mail.yahoo.com>
 <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <n762ut$e77$1@ger.gmane.org>

On 09/01/2016 19:54, precious akams via Tutor wrote:
> PLEASE I NEED A LITTLE HELP .
> I can figure out what Iam missing in this project
>
> Create a class called BankAccount

This is beyond a joke.  The main mailing list is all ready being 
moderated because of the constant messages asking for help on this 
presumably homework question. Can these be stopped at source here please?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From PNiewosz at slb.com  Wed Jan 13 07:53:36 2016
From: PNiewosz at slb.com (Patrycja Niewosz)
Date: Wed, 13 Jan 2016 12:53:36 +0000
Subject: [Tutor] Fwd: RE: Thread output to GUI Text Field
In-Reply-To: <n72o81$a3q$1@ger.gmane.org>
References: <2B14D1AD04128B4AA0B29D4096FC649A0161F7FC4C@NL0230MBX03N2.DIR.slb.com>
 <5694CB15.9060706@btinternet.com> <n72o81$a3q$1@ger.gmane.org>
Message-ID: <2B14D1AD04128B4AA0B29D4096FC649A0161F80FC5@NL0230MBX03N2.DIR.slb.com>

Hi Alan,

The code is working now.
Thanks,
Patrycja

-----Original Message-----
From: Tutor [mailto:tutor-bounces+pniewosz=slb.com at python.org] On Behalf Of Alan Gauld
Sent: Tuesday, January 12, 2016 11:32 AM
To: tutor at python.org
Subject: Re: [Tutor] Fwd: RE: Thread output to GUI Text Field

On 12/01/16 09:44, Alan Gauld wrote:

> You didn't mention earlier that you were using threads!
> That significantly complicates matters.

Apologies, I just noticed that the subject line does, although it wasn't clear that you were trying to output between threads.

> I have found that I need to use slots and signal to pass data between 
> Threads and hence

Does that mean you have solved the problem?
Or are you still looking for help?
If so what exactly do you want to know?
Do you have some code that doesn't work for example?

> class Panel(QtGui.QWidget):
> def __init__(self,master,parent=None):
> .....
> .....
> self.initUI()
> 
> 
> def initUI(self):
> #Build layout
> self.tcpControlMessages = QtGui.QPlainTextEdit()
> 
> def startThreadedProcess(self):
>         self. tcpControlMessage.clear()
>         self.thread.start()
> 
> @QtCore.Slot()
>  def updateMessages(self, message):
>         print message
>         self. tcpControlMessages.appendPlainText(message)

We can help with the threading aspects but for Qt specifics you will probably do better on a PyQt forum.


--
Alan G
Author of the Learn to Program web site
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.alan-2Dg.me.uk_&d=CwICAg&c=uGuXJ43KPkPWEl2imVFDmZQlhQUET7pVRA2PDIOxgqw&r=VlCD52JbZNT0daOyAL7vAA&m=9SVNfjVnnKpqs4j0eWR3Iq0mcPb4UPVzbrZzb2E_iWU&s=CI27ak-lLYWB8L1KnBOekFfqjOthWxdNFshm-EUmeRg&e=
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.amazon.com_author_alan-5Fgauld&d=CwICAg&c=uGuXJ43KPkPWEl2imVFDmZQlhQUET7pVRA2PDIOxgqw&r=VlCD52JbZNT0daOyAL7vAA&m=9SVNfjVnnKpqs4j0eWR3Iq0mcPb4UPVzbrZzb2E_iWU&s=QRRuN9SddMDPZUwUPEfsTRKmBoSE907XCeCbt6_2rOY&e=
Follow my photo-blog on Flickr at:
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.flickr.com_photos_alangauldphotos&d=CwICAg&c=uGuXJ43KPkPWEl2imVFDmZQlhQUET7pVRA2PDIOxgqw&r=VlCD52JbZNT0daOyAL7vAA&m=9SVNfjVnnKpqs4j0eWR3Iq0mcPb4UPVzbrZzb2E_iWU&s=4b3hD5aM-gpp35f9pcsAjE5a_ZyDoCaxpynaDnVo7Hc&e= 


_______________________________________________
Tutor maillist  -  Tutor at python.org
To unsubscribe or change subscription options:
https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman_listinfo_tutor&d=CwICAg&c=uGuXJ43KPkPWEl2imVFDmZQlhQUET7pVRA2PDIOxgqw&r=VlCD52JbZNT0daOyAL7vAA&m=9SVNfjVnnKpqs4j0eWR3Iq0mcPb4UPVzbrZzb2E_iWU&s=NKRcyXW49oaftjEi7LkjX7UPa6HL45Rzv79L6qbsQRg&e= 

From alan.gauld at btinternet.com  Wed Jan 13 13:08:56 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 13 Jan 2016 18:08:56 +0000
Subject: [Tutor] 12 year old starting to code
In-Reply-To: <n75oah$daq$1@ger.gmane.org>
References: <CAFkFBpw2x0xAs1iPahBfUo7ZqOe5-i55MNfOxfwxTVSQxQGhgQ@mail.gmail.com>
 <n70shn$dg$1@ger.gmane.org>
 <CAFkFBpyi0S1xuHbw-8TeL1BOgvywr_q-RBRT-9Qbs89Ry6F04Q@mail.gmail.com>
 <569444A0.9060202@btinternet.com> <n75oah$daq$1@ger.gmane.org>
Message-ID: <n763rp$tm8$1@ger.gmane.org>

On 13/01/16 14:51, Mark Lawrence wrote:

>> OK, It should be in C:\Python35\Lib\idelib
> 
> That's not the default for 3.5, see 
> https://docs.python.org/3/using/windows.html

Thanks for catching that. I based it on
my ActiveState 3.4 install, but...

I never use the default install and usually
install ActiveState rather than python.org.
And I haven't moved to 3.5 yet...

enough excuses? :-)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Wed Jan 13 13:13:08 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 13 Jan 2016 18:13:08 +0000
Subject: [Tutor] PLEASE I NEED HELP URGENTLY
In-Reply-To: <n762ut$e77$1@ger.gmane.org>
References: <911299850.2554269.1452369297130.JavaMail.yahoo.ref@mail.yahoo.com>
 <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
 <n762ut$e77$1@ger.gmane.org>
Message-ID: <n7643l$2a6$1@ger.gmane.org>

On 13/01/16 17:53, Mark Lawrence wrote:

> This is beyond a joke.  The main mailing list is all ready being 
> moderated because of the constant messages asking for help on this 
> presumably homework question. Can these be stopped at source here please?

I'm not sure we should. This is the kind of thing tutor is
here for. The poster has provided his code and the error
and is puzzled by a very common beginners issue - the
use of dunder methods.

It seems like a legitimate tutor post.
I fully understand the main list not wanting to take them
but this feels like the right place, as far as I can tell.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From breamoreboy at yahoo.co.uk  Wed Jan 13 15:44:19 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 Jan 2016 20:44:19 +0000
Subject: [Tutor] PLEASE I NEED HELP URGENTLY
In-Reply-To: <n7643l$2a6$1@ger.gmane.org>
References: <911299850.2554269.1452369297130.JavaMail.yahoo.ref@mail.yahoo.com>
 <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
 <n762ut$e77$1@ger.gmane.org> <n7643l$2a6$1@ger.gmane.org>
Message-ID: <n76cvv$ou2$1@ger.gmane.org>

On 13/01/2016 18:13, Alan Gauld wrote:
> On 13/01/16 17:53, Mark Lawrence wrote:
>
>> This is beyond a joke.  The main mailing list is all ready being
>> moderated because of the constant messages asking for help on this
>> presumably homework question. Can these be stopped at source here please?
>
> I'm not sure we should. This is the kind of thing tutor is
> here for. The poster has provided his code and the error
> and is puzzled by a very common beginners issue - the
> use of dunder methods.
>
> It seems like a legitimate tutor post.
> I fully understand the main list not wanting to take them
> but this feels like the right place, as far as I can tell.
>

Quoting the main Python list

<quote>
On 13/01/2016 05:57, ifeanyioprah--- via Python-list wrote:

[... snip yet another homework dump with one more still held in
moderation ...]

At this point you're basically spamming this list. I won't allow any
more of your posts through unless they appear to be engaging with the
help shown to you (and others?) over the last few days.

TJG
</quote>

What do you want, blood?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From lac at openend.se  Wed Jan 13 15:47:43 2016
From: lac at openend.se (Laura Creighton)
Date: Wed, 13 Jan 2016 21:47:43 +0100
Subject: [Tutor] me, my arm, my availability ...
Message-ID: <201601132047.u0DKlhIc025984@theraft.openend.se>


I fell recently.  Ought to be nothing, but a small chip of bone, either an
existing one or one I just made is nicely wedged in the joint taking away
a whole lot of the ability of my arm to rotate in the elbow joint.  Or
hold my arm in a position that is usual for typing.  Plus,  now that the
sprain/swelling is more or less over, the pain, unfortunately is not.

The real downside is that my typing speed is down from 135-140 wpm
to 5-10 wmp.  At this rate, just getting my usual work done takes
overtime.

Seems like surgery is needed to fix this. 

So I wanted you all to know, no, I haven't forgotten you and no haven't
stopped caring.  I have just stopped being as __capable__ if you know
what I mean.

Please take care of yourselves and each other.  I will often be reading
even if typing is more than I can do right now.

Laura

ps -- (recent tutor discussion) I am with Alan and not with Mark.  I
am happy as anything when people post their not-quite-working code for
homework assignments here to tutor.  They aren't lazy bastards wanting
somebody to do their assignments for them, they want to learn why what
they are trying to do isn't working.  Sounds perfect for tutor to me.


From breamoreboy at yahoo.co.uk  Wed Jan 13 15:49:37 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 Jan 2016 20:49:37 +0000
Subject: [Tutor] idle??
In-Reply-To: <CAMw+j7Kb42xzHoFRoXddjRAk7OQv69g-WVEKiw+weMNSpb49-w@mail.gmail.com>
References: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
 <CAMw+j7Kb42xzHoFRoXddjRAk7OQv69g-WVEKiw+weMNSpb49-w@mail.gmail.com>
Message-ID: <n76d9t$1pn$1@ger.gmane.org>

On 09/01/2016 10:38, Chris Warrick wrote:
> On 8 January 2016 at 20:07, bruce <badouglas at gmail.com> wrote:
>> So, where does IDLE fit into this....
>
> IDLE is a sad little ?IDE?, which is really ugly, because it?s written
> in Tk. It lacks many IDE features. It comes with a really basic
> debugger (that doesn?t even highlight the line that is being currently
> executed?), function signature hinting, and some code completion.
>

Please ignore this drivel, he's spouted this before without giving any 
justification.  IDLE is perfectly adequate as a starter for Python.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From mail at timgolden.me.uk  Wed Jan 13 15:51:46 2016
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 13 Jan 2016 20:51:46 +0000
Subject: [Tutor] PLEASE I NEED HELP URGENTLY
In-Reply-To: <n76cvv$ou2$1@ger.gmane.org>
References: <911299850.2554269.1452369297130.JavaMail.yahoo.ref@mail.yahoo.com>
 <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
 <n762ut$e77$1@ger.gmane.org> <n7643l$2a6$1@ger.gmane.org>
 <n76cvv$ou2$1@ger.gmane.org>
Message-ID: <5696B8E2.30408@timgolden.me.uk>

On 13/01/2016 20:44, Mark Lawrence wrote:
> On 13/01/2016 18:13, Alan Gauld wrote:
>> On 13/01/16 17:53, Mark Lawrence wrote:
>>
>>> This is beyond a joke.  The main mailing list is all ready being
>>> moderated because of the constant messages asking for help on this
>>> presumably homework question. Can these be stopped at source here
>>> please?
>>
>> I'm not sure we should. This is the kind of thing tutor is
>> here for. The poster has provided his code and the error
>> and is puzzled by a very common beginners issue - the
>> use of dunder methods.
>>
>> It seems like a legitimate tutor post.
>> I fully understand the main list not wanting to take them
>> but this feels like the right place, as far as I can tell.
>>
>
> Quoting the main Python list
>
> <quote>
> On 13/01/2016 05:57, ifeanyioprah--- via Python-list wrote:
>
> [... snip yet another homework dump with one more still held in
> moderation ...]
>
> At this point you're basically spamming this list. I won't allow any
> more of your posts through unless they appear to be engaging with the
> help shown to you (and others?) over the last few days.
>
> TJG
> </quote>
>
> What do you want, blood?
>

Speaking as the list moderator in question over there: if I might 
moderate Mark's well-known zeal...

What started as a somewhat naive but fairly typical request for 
coursework-style help turned into an untenable situation with the OP (or 
apparently several OPs, some or all possibly sock-puppets) repeating 
variants on the same question or simple pleas for help again and again 
without apparently engaging with any of the help they were receiving.

What you choose to do on the Tutor list is entirely up to you. For now, 
on the main python-list, I've held the most egregious email address 
offenders for moderation.

TJG

From mail at timgolden.me.uk  Wed Jan 13 15:52:35 2016
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 13 Jan 2016 20:52:35 +0000
Subject: [Tutor] PLEASE I NEED HELP URGENTLY
In-Reply-To: <5696B8E2.30408@timgolden.me.uk>
References: <911299850.2554269.1452369297130.JavaMail.yahoo.ref@mail.yahoo.com>
 <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
 <n762ut$e77$1@ger.gmane.org> <n7643l$2a6$1@ger.gmane.org>
 <n76cvv$ou2$1@ger.gmane.org> <5696B8E2.30408@timgolden.me.uk>
Message-ID: <5696B913.1020502@timgolden.me.uk>

On 13/01/2016 20:51, Tim Golden wrote:
> Speaking as the list moderator in question over there: if I might
> moderate Mark's well-known zeal...

(Absolutely no pun intended!)

TJG

From breamoreboy at yahoo.co.uk  Wed Jan 13 15:52:49 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 Jan 2016 20:52:49 +0000
Subject: [Tutor] idle??
In-Reply-To: <n6phgc$a26$1@ger.gmane.org>
References: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
 <n6phgc$a26$1@ger.gmane.org>
Message-ID: <n76dfu$1pn$2@ger.gmane.org>

On 08/01/2016 23:42, Alan Gauld wrote:
> On 08/01/16 19:07, bruce wrote:
>
>> Is IDLE essentially an ide for doing py dev? I see there's a
>> windows/linux (rpms) for it.
>
> Yes, its the official IDE for Python.
>
> There is an "unofficial" version called xidle which tends
> to get a lot of the new stuff before it makes it into the
> official release. For a long time not much happened with
> IDLE but recently there has been a bunch of activity so
> I'm hopeful we may soon see some new features appearing.
>

Did you mean http://idlex.sourceforge.net/ ?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From breamoreboy at yahoo.co.uk  Wed Jan 13 15:59:43 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 Jan 2016 20:59:43 +0000
Subject: [Tutor] me, my arm, my availability ...
In-Reply-To: <201601132047.u0DKlhIc025984@theraft.openend.se>
References: <201601132047.u0DKlhIc025984@theraft.openend.se>
Message-ID: <n76dsr$bto$1@ger.gmane.org>

On 13/01/2016 20:47, Laura Creighton wrote:
>
> I fell recently.  Ought to be nothing, but a small chip of bone, either an
> existing one or one I just made is nicely wedged in the joint taking away
> a whole lot of the ability of my arm to rotate in the elbow joint.  Or
> hold my arm in a position that is usual for typing.  Plus,  now that the
> sprain/swelling is more or less over, the pain, unfortunately is not.
>
> The real downside is that my typing speed is down from 135-140 wpm
> to 5-10 wmp.  At this rate, just getting my usual work done takes
> overtime.
>
> Seems like surgery is needed to fix this.
>
> So I wanted you all to know, no, I haven't forgotten you and no haven't
> stopped caring.  I have just stopped being as __capable__ if you know
> what I mean.
>
> Please take care of yourselves and each other.  I will often be reading
> even if typing is more than I can do right now.
>
> Laura
>
> ps -- (recent tutor discussion) I am with Alan and not with Mark.  I
> am happy as anything when people post their not-quite-working code for
> homework assignments here to tutor.  They aren't lazy bastards wanting
> somebody to do their assignments for them, they want to learn why what
> they are trying to do isn't working.  Sounds perfect for tutor to me.
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

Hardly surprising that so many experienced Python people no longer want 
to give their time, as the bib fitting, spoon feeding, nappy changing 
politically correct nambies are taking over.  Don't do any research, 
don't tell us your OS, don't tell us your python version, but we'll 
still answer your question for free.  Yuck, this approach makes me puke.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From breamoreboy at yahoo.co.uk  Wed Jan 13 17:10:56 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 Jan 2016 22:10:56 +0000
Subject: [Tutor] 12 year old starting to code
In-Reply-To: <n763rp$tm8$1@ger.gmane.org>
References: <CAFkFBpw2x0xAs1iPahBfUo7ZqOe5-i55MNfOxfwxTVSQxQGhgQ@mail.gmail.com>
 <n70shn$dg$1@ger.gmane.org>
 <CAFkFBpyi0S1xuHbw-8TeL1BOgvywr_q-RBRT-9Qbs89Ry6F04Q@mail.gmail.com>
 <569444A0.9060202@btinternet.com> <n75oah$daq$1@ger.gmane.org>
 <n763rp$tm8$1@ger.gmane.org>
Message-ID: <n76i2c$gce$1@ger.gmane.org>

On 13/01/2016 18:08, Alan Gauld wrote:
> On 13/01/16 14:51, Mark Lawrence wrote:
>
>>> OK, It should be in C:\Python35\Lib\idelib
>>
>> That's not the default for 3.5, see
>> https://docs.python.org/3/using/windows.html
>
> Thanks for catching that. I based it on
> my ActiveState 3.4 install, but...
>
> I never use the default install and usually
> install ActiveState rather than python.org.
> And I haven't moved to 3.5 yet...
>
> enough excuses? :-)
>

No, I expect accuracy on a tutor mailing list from the moderator.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From starfas_s at yahoo.com  Wed Jan 13 17:25:29 2016
From: starfas_s at yahoo.com (Sam Starfas)
Date: Wed, 13 Jan 2016 22:25:29 +0000 (UTC)
Subject: [Tutor] Removing open bracket, content, close bracket content...
References: <1736270889.4518790.1452723929323.JavaMail.yahoo.ref@mail.yahoo.com>
Message-ID: <1736270889.4518790.1452723929323.JavaMail.yahoo@mail.yahoo.com>

Hi,I am new to Python and trying to create a script that will remove content wrapped in brackets.For example I want to remove from each line the following:[!L]?[20][!20+:2]etc....
But when I run my script I only get the last line of the content. It is correct as far as the output I want, but I don't understand why the other lines disappear.?
Here is my data:
[!L]KEXITSETUP=Exit Setup[20]KEXITPSSETUP=Exit PS Setup[20]KEXITCOLSETUP=Exit Color SetupKSERVERSETUP=Server Setup[!L]KPRINTERSETUP=Printer Setup[!L]KNETWORKSETUP=Network Setup[!L]KJOBLOGSETUP=Job Log Setup[!L]KLANGUAGESETUP=Language Setup[!L]KCHANGEPASSWRD=Change Password[!L]KCLRSRVR=Clear Server[!L]KFACTORYDFLT=Factory DefaultsSETUPB1=Setup[20+:2]KPSERROR=Print to ? ? ? ? ? ?PS Error[20+:2]PSERROR=Print to ? ? ? ? ? ?PS Error[20+:2]EFPSError=Print to ? ? ? ? ? ?PS Error

----------------------------------------------------------------Here is the script I have so far (remember I am new):
import re
with open("lcd_eng.dct") as f:? ? with open("out.txt", "w") as f1:? ? ? ? for line in f:? ? ? ? ? ? if f == 0 or not line.lstrip().startswith('#'):? ? ? ? ? ? ? ? f1.write(line)
# Converts the = sign to a , comma for generation of the .csv filef = open("out.txt",'r')filedata = f.read()f.close()newdata = filedata.replace("=",",")f = open("out2.txt",'w')f.write(newdata)f.close()
# Should strip out anything [...]?f1 = open("out2.txt",'r')newdata = f1.read()f1.close()newdata = re.sub(r'\[[^>]*\]', '', newdata)f1 = open("end.txt",'w')f1.write(newdata)f1.close()

The output I get is only the last line of the input file. Not sure why. I know the issue is in the search and replace, but I am not sure where.?
Thanks for any and all help.Very much appreciated.Sam

From dyoo at hashcollision.org  Wed Jan 13 18:10:32 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Wed, 13 Jan 2016 15:10:32 -0800
Subject: [Tutor] idle??
In-Reply-To: <CAMw+j7Kb42xzHoFRoXddjRAk7OQv69g-WVEKiw+weMNSpb49-w@mail.gmail.com>
References: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
 <CAMw+j7Kb42xzHoFRoXddjRAk7OQv69g-WVEKiw+weMNSpb49-w@mail.gmail.com>
Message-ID: <CAGZAPF5AZ32pwcgQBVYcopC+uyQ-CVOdoawrsa2J9+nNN-bSJQ@mail.gmail.com>

>> So, where does IDLE fit into this....
>
> IDLE is a sad little ?IDE?, which is really ugly, because it?s written
> in Tk. It lacks many IDE features. It comes with a really basic
> debugger (that doesn?t even highlight the line that is being currently
> executed?), function signature hinting, and some code completion.
>
> And it doesn?t even do something as basic as line numbering.

Hi Chris,

The quality of a beginner-level IDE might not necessarily be based on
the number of features it has.  For someone who's starting out, IDLE
is probably fine because it gets out of your way.  It lets you type
programs and evaluate them.  For a beginner, that might just be enough
to focus on learning the language.


(Aside: I've had the contrary experience with Eclipse, for example,
which is as full-featured as they come, but makes me feel like I'm
staring at the flight controls of a space shuttle, with all this stuff
about launchers and Luna and such.  I can get productive with it  It
takes my a long time to learn.  I suppose I could say the same thing
about Emacs.)


That is, many features might be a *distraction* from learning to
program.  Tools for beginners should be measured by criteria for
learning, and that might not match with the features we care about as
professional developers.  But maybe that's a controversial opinion.

I think IDLE is ok for what it's designed for: to provide a simple,
textual environment for writing and running simple Python programs.

From alan.gauld at btinternet.com  Wed Jan 13 18:29:38 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 13 Jan 2016 23:29:38 +0000
Subject: [Tutor] Removing open bracket, content, close bracket content...
In-Reply-To: <1736270889.4518790.1452723929323.JavaMail.yahoo@mail.yahoo.com>
References: <1736270889.4518790.1452723929323.JavaMail.yahoo.ref@mail.yahoo.com>
 <1736270889.4518790.1452723929323.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <n76ml2$p0j$1@ger.gmane.org>

On 13/01/16 22:25, Sam Starfas via Tutor wrote:

Hi Sam,

Could you repost using plain text please?
As you can see below the htnml/rich text option destroys the formatting
of the data and code making it near impossible to read.

> Hi,I am new to Python and trying to create a script that will remove content wrapped in brackets.For example I want to remove from each line the following:[!L] [20][!20+:2]etc....
> But when I run my script I only get the last line of the content. It is correct as far as the output I want, but I don't understand why the other lines disappear. 
> Here is my data:
> [!L]KEXITSETUP=Exit Setup[20]KEXITPSSETUP=Exit PS Setup[20]KEXITCOLSETUP=Exit Color SetupKSERVERSETUP=Server Setup[!L]KPRINTERSETUP=Printer Setup[!L]KNETWORKSETUP=Network Setup[!L]KJOBLOGSETUP=Job Log Setup[!L]KLANGUAGESETUP=Language Setup[!L]KCHANGEPASSWRD=Change Password[!L]KCLRSRVR=Clear Server[!L]KFACTORYDFLT=Factory DefaultsSETUPB1=Setup[20+:2]KPSERROR=Print to            PS Error[20+:2]PSERROR=Print to            PS Error[20+:2]EFPSError=Print to            PS Error
> 
> ----------------------------------------------------------------Here is the script I have so far (remember I am new):
> import re
> with open("lcd_eng.dct") as f:    with open("out.txt", "w") as f1:        for line in f:            if f == 0 or not line.lstrip().startswith('#'):                f1.write(line)
> # Converts the = sign to a , comma for generation of the .csv filef = open("out.txt",'r')filedata = f.read()f.close()newdata = filedata.replace("=",",")f = open("out2.txt",'w')f.write(newdata)f.close()
> # Should strip out anything [...] f1 = open("out2.txt",'r')newdata = f1.read()f1.close()newdata = re.sub(r'\[[^>]*\]', '', newdata)f1 = open("end.txt",'w')f1.write(newdata)f1.close()
> 

As a guess, the problems you are facing is probably that you are
throwing away (or overwriting) the results inside a loop somewhere
and only the last line gets kept. You probably need to create a list and
append each result to the list.

But without being able to read your code that's only a guess.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From adeadmarshal at gmail.com  Thu Jan 14 03:05:59 2016
From: adeadmarshal at gmail.com (Ali Moradi)
Date: Thu, 14 Jan 2016 11:35:59 +0330
Subject: [Tutor] Clickable listbox opening specific .txt files
Message-ID: <CAMh2k3bYQWfq1EfH0uxwTJDz_narCSq5oV19iO9vf6ijRPbQ+w@mail.gmail.com>

Hi, i want to open specific .txt files when each item on the listbox was
clicked! When i click on the first item for example it opens 1.txt and
shows it on the text widget on the right, and when i select the second item
on the list first it erases the text on the Text widget and then opens
2.txt file and so on till item 17 on the list.

Please modify my code i'm still beginner
Thanks a lot

paste.pound-python.org/show/uMUswu4YSmMY5XBPPgIy/

From cmgcomsol at gmail.com  Wed Jan 13 23:07:02 2016
From: cmgcomsol at gmail.com (CMG Thrissur)
Date: Thu, 14 Jan 2016 09:37:02 +0530
Subject: [Tutor] me, my arm, my availability ...
In-Reply-To: <201601132047.u0DKlhIc025984@theraft.openend.se>
References: <201601132047.u0DKlhIc025984@theraft.openend.se>
Message-ID: <56971EE6.8010101@gmail.com>

take care, get well soon,

regards

CMG

On Thursday 14 January 2016 02:17 AM, Laura Creighton wrote:
> I fell recently.  Ought to be nothing, but a small chip of bone, either an
> existing one or one I just made is nicely wedged in the joint taking away
> a whole lot of the ability of my arm to rotate in the elbow joint.  Or
> hold my arm in a position that is usual for typing.  Plus,  now that the
> sprain/swelling is more or less over, the pain, unfortunately is not.
>
> The real downside is that my typing speed is down from 135-140 wpm
> to 5-10 wmp.  At this rate, just getting my usual work done takes
> overtime.
>
> Seems like surgery is needed to fix this.
>
> So I wanted you all to know, no, I haven't forgotten you and no haven't
> stopped caring.  I have just stopped being as __capable__ if you know
> what I mean.
>
> Please take care of yourselves and each other.  I will often be reading
> even if typing is more than I can do right now.
>
> Laura
>
> ps -- (recent tutor discussion) I am with Alan and not with Mark.  I
> am happy as anything when people post their not-quite-working code for
> homework assignments here to tutor.  They aren't lazy bastards wanting
> somebody to do their assignments for them, they want to learn why what
> they are trying to do isn't working.  Sounds perfect for tutor to me.
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor


From unee0x at gmail.com  Wed Jan 13 22:56:40 2016
From: unee0x at gmail.com (kay Cee)
Date: Wed, 13 Jan 2016 22:56:40 -0500
Subject: [Tutor] Modularity
Message-ID: <488FA568-B4B5-4698-A226-9757C1AF9D2C@gmail.com>

Recently I've decided to write a pong style game with pygame; however, I can't seem to modularize the game successfully.

I'm looking to make individual modules for each class eg. Ball, Paddle,bounds, Game, Physics and Logic.
Used like this :
ball = Ball(), paddle = Paddle() and etc...
In a Game class...
Mainly, I get errors that say class is not defined..

My questions are what is the best way to achieve this and what is the best way to import yourClass?

So far I've tried:
Import class as class,
From class import*,
Import class



Sent from my iPhone

From alan.gauld at btinternet.com  Thu Jan 14 03:53:22 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 14 Jan 2016 08:53:22 +0000
Subject: [Tutor] Modularity
In-Reply-To: <488FA568-B4B5-4698-A226-9757C1AF9D2C@gmail.com>
References: <488FA568-B4B5-4698-A226-9757C1AF9D2C@gmail.com>
Message-ID: <n77nm3$s73$1@ger.gmane.org>

On 14/01/16 03:56, kay Cee wrote:

> I'm looking to make individual modules for each class eg. Ball, Paddle,bounds, Game, Physics and Logic.

That's not usually necessary in Python. It's often better
to group several related classes in a single module.
For example your Ball and Paddle and any other display
type things might go together in a module called, say,
pongwidgets.

> Used like this :
> ball = Ball(), paddle = Paddle() and etc...

If you put your Ball class into a module called ball
then you would need to do

import ball

myBall = ball.Ball()

you need to reference the imported module.

Alternatively you could use

from ball import Ball

myBall = Ball()

> Mainly, I get errors that say class is not defined..

It would help if you show us actual code and post
the full error message rather than summarizing things.

> So far I've tried:
> Import class as class,
> From class import*,
> Import class

Note that python is case sensitive so you need lower case.
That may just be your mail tool being too clever though...

The first and third versions are ok, the middle one is frowned on.
It's better to use the

from module import class

style I showed above if you only have (or use) a single
class per module

Post some code and some errors and we will be better
able to help.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Thu Jan 14 04:03:59 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 14 Jan 2016 09:03:59 +0000
Subject: [Tutor] Clickable listbox opening specific .txt files
In-Reply-To: <CAMh2k3bYQWfq1EfH0uxwTJDz_narCSq5oV19iO9vf6ijRPbQ+w@mail.gmail.com>
References: <CAMh2k3bYQWfq1EfH0uxwTJDz_narCSq5oV19iO9vf6ijRPbQ+w@mail.gmail.com>
Message-ID: <n77o9v$6ia$1@ger.gmane.org>

On 14/01/16 08:05, Ali Moradi wrote:
> Hi, i want to open specific .txt files when each item on the listbox was
> clicked! When i click on the first item for example it opens 1.txt and
> shows it on the text widget on the right, and when i select the second item
> on the list first it erases the text on the Text widget and then opens
> 2.txt file and so on till item 17 on the list.

So what does happen?

> Please modify my code i'm still beginner
> Thanks a lot
> 
> paste.pound-python.org/show/uMUswu4YSmMY5XBPPgIy/

We won't write your code for you but we will point out areas
for improvement. But... see below.

For the future:
It's best if you post the actual code in your mail
(using plain text please). We are happy to accept up
to, say, a hundred lines.

Also post full error messages, if any.

Finally tell us the OS and Python version.

Here is a slightly shortened version of your code:

########################
from Tkinter import *

root = Tk()
root.title("Renkontoj kun Diferenculoj")
root.iconbitmap(r"C:\python27\DLLs/py.ico")
root.resizable(0, 0)
frame1 = LabelFrame(root, height=300, width=400, text='Lecionoj')
frame2 = LabelFrame(root, height=300, width=400, text='Tekstoj')
frame1.grid(row=0, column=0)
frame2.grid(row=0, column=1)

def click(event):
    index = list.curselection()[0]
    file = open(r"C:\Users\deadmarshal\PycharmProjects\ali\1.txt").read()

list = Listbox(frame1)
list.insert(1, "Konati?u kun Kamila")
list.insert(2, "Sinjoro Johano")
list.insert(3, "Onklino Marta")
scroll = Scrollbar(frame1, orient=VERTICAL, command=list.yview)
scroll.grid(row=0, column=2, sticky='ns')
list.grid(row=0, column=1)
list.bind("<ButtonRelease-1>", click)
text = Text(frame2).grid()

root.mainloop()
############################

Notice that your click function does not do anything. It reads
the file then throws it away when the function exits. You need
to insert the file data into your text widget. (And probably
clear the text widget first.)

Finally, note that by using the name liust you are hiding a builtin
Python function, so you will no longer be able to convert things
to a list using list(). This doesn't mattter just now but may be
a problem later. Its best to avoid using names that Python already
uses. One way to do that is to add a descriptive modifier like
name_list, say.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From sjeik_appie at hotmail.com  Thu Jan 14 04:03:20 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Thu, 14 Jan 2016 09:03:20 +0000
Subject: [Tutor] Question about the memory manager
In-Reply-To: <DUB123-W1156BF8488894A8EC1523783CB0@phx.gbl>
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>, ,
 <CAExdVN=Fpw7x0pMRNT+JCFsq4PoWWgrooMU3qTjqpZ7-nqrWYQ@mail.gmail.com>,
 <DUB123-W1156BF8488894A8EC1523783CB0@phx.gbl>
Message-ID: <DUB123-W221738530921133AAECD0483CC0@phx.gbl>

D

> From: sjeik_appie at hotmail.com
> To: tim.peters at gmail.com
> Date: Wed, 13 Jan 2016 08:11:11 +0000
> Subject: Re: [Tutor] Question about the memory manager
> CC: tutor at python.org
> 
> > From: tim.peters at gmail.com
> > Date: Sun, 10 Jan 2016 10:54:10 -0600
> > Subject: Re: [Tutor] Question about the memory manager
> > To: sjeik_appie at hotmail.com
> > CC: tutor at python.org
> > 
> > [Albert-Jan Roskam <sjeik_appie at hotmail.com>]
> > > I just found a neat trick to free up an emergency stash of memory in
> > > a funtion that overrides sys.excepthook. The rationale is that all
> > > exceptions, including MemoryErrors will be logged.
> > > The code is below. My question: is that memory *guaranteed* to be
> > > freed right after the 'del' statement? Or should one call gc.collect to
> > > be really sure?
> > >
> > > rainydayfund = [[] for x in xrange(16*1024)] # or however much you need
> > > def handle_exception(e):
> > >     global rainydayfund
> > >     del rainydayfund
> > > ... etc, etc ...
> > > http://stackoverflow.com/questions/1235349/python-how-can-i-handle-any-unhandled-exception-in-an-alternative-way
> > 
> > This works fine in all versions of CPython (the C implementation of
> > Python distributed by python.org) to date.  That's because:
> > 
> > 1. All versions of CPython rely primarily on reference counting (`gc`
> > is only needed to reclaim garbage containing reference cycles).  An
> > object is released immediately when its reference count falls to 0.
> > 
> > 2. There is only one reference to the big list there (via the global
> > `raindydayfund`), so the memory becomes garbage immediately upon
> > executing the `del`.
> > 
> > 3. Similarly, that giant list holds the only references to the masses
> > of distinct empty lists it contains, so they also become garbage
> > immediately upon the giant list becoming garbage.
> > 
> > 4. CPython doesn't happen to stick garbage lists in, e.g., some
> > internal free list reusable only for new list objects - it actually
> > releases the memory for garbage lists.  Kinda ;-)
> > 
> > #2 and #3 are necessarily true.  #1 is true in CPython, but not in all
> > implementations of Python.
> > 
> > #4 is where things _might_ change even in CPython, but it's very
> > unlikely to change.  As is, it would take a small book to flesh out
> > what "Kinda ;-)" means, exactly.  Memory management is complex, with
> > many layers, involving many details.
> > 
> > If you can live with all that, I'd suggest a more straightforward way
> > of setting it up, like:
> > 
> > rainydayfund  = b"x" * N
> > 
> > where `N` is the number of bytes you want to reserve.  That is, create
> > a giant bytestring containing the number of "emergency bytes" you
> > need.  If N is large enough, that will avoid CPython's "small object
> > allocator" and CPython's "arena allocator", getting the memory
> > directly from (and returning the memory directly to) the OS.  The
> > fewer layers that get involved, the fewer layers that _may_ surprise
> > you by changing behavior in the future.
> 
> Hi Tim,
> 
> Thank you! Can you recommend a document or website that CPython's memory manager?
> Might be interesting and perhaps useful to know a bit more about the details. Perhaps this knowledge might sometimes help writing faster code?
> 

These two pages are quite nice. The author says the memory used by small objects is never returned to the OS, which may be problematic for long running processes. It appears that it is better to have a few big objects rather than many small ones, because memory is more likely to become fragmented with many deleted small objects (and there's no such thing as gc.defrag)
http://www.evanjones.ca/memoryallocator/
http://deeplearning.net/software/theano/tutorial/python-memory-management.html


> Best wishes,
> Albert-Jan
> 
> PS:
> 
> albertjan at debian:~$ python -c "import this" 
> The Zen of Python, by Tim Peters
> 
> ...
> ...
> There should be one-- and preferably only one --obvious way to do it.
> Although that way may not be obvious at first unless you're Dutch.
> 
> --> Nope not even then. Perhaps if my name were Guido? :-)
> 
> 
> 
>  		 	   		  
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
 		 	   		  

From james at uplinkzero.com  Thu Jan 14 05:01:04 2016
From: james at uplinkzero.com (James Chapman)
Date: Thu, 14 Jan 2016 10:01:04 +0000
Subject: [Tutor] Modularity
Message-ID: <CAHvkzy=j_DxOaFsTkJ2GLCPV3h7s4+i3bMbEkKTPH=3L2+OW-w@mail.gmail.com>

May I suggest: https://docs.python.org/2/tutorial/modules.html

In particular:
* https://docs.python.org/2/tutorial/modules.html#the-module-search-path
* https://docs.python.org/2/tutorial/modules.html#packages


Now the next bit of advice is likely to be controversial but I have
good reasons for it.

I like to import the top level module and use the full namespace in
the code, that way, when I come back to it in 3 years time I know
where each function call is coming from.

For example, lets say I had the following package (stolen from the
docs page linked above):


sound/                          Top-level package
      __init__.py               Initialize the sound package
      formats/                  Subpackage for file format conversions
              __init__.py
              wavread.py
              wavwrite.py
              aiffread.py
              aiffwrite.py
              auread.py
              auwrite.py
              ...
      effects/                  Subpackage for sound effects
              __init__.py
              echo.py
              surround.py
              reverse.py
              ...
      filters/                  Subpackage for filters
              __init__.py
              equalizer.py
              vocoder.py
              karaoke.py


I myself would import as follows

        import sound

Then in my code, the calls would look like:

        wave = sound.formats.waveread(someFile)
        aiffFile = sound.formats.aiffwrite(wave)
        auFile = sound.formats.auwrite(wave)


If I did:

        from sound import formats.*

Then the code would be

        wave = waveread(someFile)
        aiffFile = aiffwrite(wave)
        auFile = auwrite(wave)


The problem with the latter is, which module supplies waveread?
I have roughly 20 import statements where I'm importing something.*
which one of those modules supplies the function waveread?


A way around the above would be

        import sound.formats.waveread
        import sound.formats.aiffwrite
        import sound.formats.auwrite

Code would then be

        wave = waveread(someFile)
        aiffFile = aiffwrite(wave)
        auFile = auwrite(wave)



But what if we have various modules that implement a waveread function?

Then we'd have to start using

        import sound.formats.waveread as sfwaveread
        import some.other.waveread


Code is read more than it is written, so don't be lazy! Use the
namespaces in your code. You make it clear what you're doing it you
avoid clashing.


Finally, if you absolutely must be lazy, then import like this:

        from sound import formats as sf

        wave = sf.waveread(someFile)
        aiffFile = sf.aiffwrite(wave)
        auFile = sf.auwrite(wave)


There is nothing I hate more than being asked to change or fix someone
elses code when that programmer is lazy and feels he/she can produce a
solution quicker by reducing the amount of typing he/she has to do.
Well, that's a lie, there are things I hate more but this come close!


James

From wolf.halton at gmail.com  Thu Jan 14 05:24:25 2016
From: wolf.halton at gmail.com (Wolf Halton)
Date: Thu, 14 Jan 2016 05:24:25 -0500
Subject: [Tutor] idle??
In-Reply-To: <CAGZAPF5AZ32pwcgQBVYcopC+uyQ-CVOdoawrsa2J9+nNN-bSJQ@mail.gmail.com>
References: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
 <CAMw+j7Kb42xzHoFRoXddjRAk7OQv69g-WVEKiw+weMNSpb49-w@mail.gmail.com>
 <CAGZAPF5AZ32pwcgQBVYcopC+uyQ-CVOdoawrsa2J9+nNN-bSJQ@mail.gmail.com>
Message-ID: <F8B35EAE-4AAC-4390-B615-2BAC2B657158@gmail.com>

Idle is sufficient for a beginner. Better than notepad. It is cross platform so your environment looks the same on Linux Mac or What-have-you. Some people obsess over ide features to avoid thinking about the more important questions. Eclipse never helped me learn to write better py code. I have made several attempts to run eclipse, and I keep falling back to geany. Geany needed some tweaking at the beginning. Idle never did. 

Wolf Halton
Atlanta Cloud Technology
Cybersecurity & Disaster Recovery Solutions 
Mobile/Text 678-687-6104

--
Sent from my iPhone. Creative word completion courtesy of Apple, Inc. 

On Jan 13, 2016, at 18:10, Danny Yoo <dyoo at hashcollision.org> wrote:

>>> So, where does IDLE fit into this....
>> 
>> IDLE is a sad little ?IDE?, which is really ugly, because it?s written
>> in Tk. It lacks many IDE features. It comes with a really basic
>> debugger (that doesn?t even highlight the line that is being currently
>> executed?), function signature hinting, and some code completion.
>> 
>> And it doesn?t even do something as basic as line numbering.
> 
> Hi Chris,
> 
> The quality of a beginner-level IDE might not necessarily be based on
> the number of features it has.  For someone who's starting out, IDLE
> is probably fine because it gets out of your way.  It lets you type
> programs and evaluate them.  For a beginner, that might just be enough
> to focus on learning the language.
> 
> 
> (Aside: I've had the contrary experience with Eclipse, for example,
> which is as full-featured as they come, but makes me feel like I'm
> staring at the flight controls of a space shuttle, with all this stuff
> about launchers and Luna and such.  I can get productive with it  It
> takes my a long time to learn.  I suppose I could say the same thing
> about Emacs.)
> 
> 
> That is, many features might be a *distraction* from learning to
> program.  Tools for beginners should be measured by criteria for
> learning, and that might not match with the features we care about as
> professional developers.  But maybe that's a controversial opinion.
> 
> I think IDLE is ok for what it's designed for: to provide a simple,
> textual environment for writing and running simple Python programs.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From eryksun at gmail.com  Thu Jan 14 05:42:57 2016
From: eryksun at gmail.com (eryk sun)
Date: Thu, 14 Jan 2016 04:42:57 -0600
Subject: [Tutor] Question about the memory manager
In-Reply-To: <DUB123-W221738530921133AAECD0483CC0@phx.gbl>
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
 <CAExdVN=Fpw7x0pMRNT+JCFsq4PoWWgrooMU3qTjqpZ7-nqrWYQ@mail.gmail.com>
 <DUB123-W1156BF8488894A8EC1523783CB0@phx.gbl>
 <DUB123-W221738530921133AAECD0483CC0@phx.gbl>
Message-ID: <CACL+1assrpC7JY2g0FE9nLK=z8hcupskUAgM5y=2xn5gq19Efw@mail.gmail.com>

On Thu, Jan 14, 2016 at 3:03 AM, Albert-Jan Roskam
<sjeik_appie at hotmail.com> wrote:
>
> These two pages are quite nice. The author says the memory used by small objects is
> never returned to the OS, which may be problematic for long running processes.

The article by Evan Jones discusses a patch to enable releasing unused
arenas (i.e. "how the problem was fixed"). Starting with 2.5, unused
arenas do get released back to the heap. Here's the diff in which Tim
Peters merged in a "heavily altered derivative" of Evan's patch [1].

Also, 2.7 and 3.3 bypass C malloc/free and the process heap to instead
use mmap/munmap on POSIX when available. This avoids the heap
high-water mark problem. Similarly, 3.4 switched to using
VirtualAlloc/VirtualFree on Windows. 3.4 also introduced the
PyObjectArenaAllocator and associated C API functions [2] to allow
modifying the default allocators.

[1]: https://hg.python.org/cpython/diff/685849bd905c/Objects/obmalloc.c
[2]: https://docs.python.org/3/c-api/memory.html#customize-pyobject-arena-allocator

From oscar.j.benjamin at gmail.com  Thu Jan 14 06:10:33 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 14 Jan 2016 11:10:33 +0000
Subject: [Tutor] me, my arm, my availability ...
In-Reply-To: <201601132047.u0DKlhIc025984@theraft.openend.se>
References: <201601132047.u0DKlhIc025984@theraft.openend.se>
Message-ID: <CAHVvXxQJdhGQDAPuc9k_iZPtSNbnRDLbMWDTyHSzKX4hN=CPpA@mail.gmail.com>

On 13 January 2016 at 20:47, Laura Creighton <lac at openend.se> wrote:
>
> Seems like surgery is needed to fix this.
>
> So I wanted you all to know, no, I haven't forgotten you and no haven't
> stopped caring.  I have just stopped being as __capable__ if you know
> what I mean.
>
> Please take care of yourselves and each other.  I will often be reading
> even if typing is more than I can do right now.

I had honestly been wondering where you were. Glad to see that you're
not gone (although obviously not glad that you're injured!).

Wishing a speedy recovery.

--
Oscar

From s.shall at virginmedia.com  Thu Jan 14 08:18:32 2016
From: s.shall at virginmedia.com (Sydney Shall)
Date: Thu, 14 Jan 2016 13:18:32 +0000
Subject: [Tutor] idle??
In-Reply-To: <CAGZAPF5AZ32pwcgQBVYcopC+uyQ-CVOdoawrsa2J9+nNN-bSJQ@mail.gmail.com>
References: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
 <CAMw+j7Kb42xzHoFRoXddjRAk7OQv69g-WVEKiw+weMNSpb49-w@mail.gmail.com>
 <CAGZAPF5AZ32pwcgQBVYcopC+uyQ-CVOdoawrsa2J9+nNN-bSJQ@mail.gmail.com>
Message-ID: <5697A028.10401@virginmedia.com>

On 13/01/2016 23:10, Danny Yoo wrote:
>>> So, where does IDLE fit into this....
>>
>> IDLE is a sad little ?IDE?, which is really ugly, because it?s written
>> in Tk. It lacks many IDE features. It comes with a really basic
>> debugger (that doesn?t even highlight the line that is being currently
>> executed?), function signature hinting, and some code completion.
>>
>> And it doesn?t even do something as basic as line numbering.
>
> Hi Chris,
>
> The quality of a beginner-level IDE might not necessarily be based on
> the number of features it has.  For someone who's starting out, IDLE
> is probably fine because it gets out of your way.  It lets you type
> programs and evaluate them.  For a beginner, that might just be enough
> to focus on learning the language.
>
>
> (Aside: I've had the contrary experience with Eclipse, for example,
> which is as full-featured as they come, but makes me feel like I'm
> staring at the flight controls of a space shuttle, with all this stuff
> about launchers and Luna and such.  I can get productive with it  It
> takes my a long time to learn.  I suppose I could say the same thing
> about Emacs.)
>
>
> That is, many features might be a *distraction* from learning to
> program.  Tools for beginners should be measured by criteria for
> learning, and that might not match with the features we care about as
> professional developers.  But maybe that's a controversial opinion.
>
> I think IDLE is ok for what it's designed for: to provide a simple,
> textual environment for writing and running simple Python programs.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
I was a beginner who began with >idle< following advice from this list.
I found it invaluable for the reason Danny gives. More complex 
environments befuddled me, and distracted me from learning Python.
I do not use it any longer. I now use IPython which I like a lot.
But for beginners I would recommend it, speaking as a debutante myself.

I hope that Ma Laura gets better soon. But please take care and have it 
seen to or you will have arthritis later. I have never met nor 
corresponded with the fine woman, but I think she is >terrible> [in 
French, otherwise in English wonderful.]

-- 
Sydney

From adeadmarshal at gmail.com  Thu Jan 14 06:39:44 2016
From: adeadmarshal at gmail.com (Ali Moradi)
Date: Thu, 14 Jan 2016 15:09:44 +0330
Subject: [Tutor] Clickable listbox opening .txt files
Message-ID: <CAMh2k3bdcDHg+vL3mubAEypP69oX3iGfR47Jct2mraP=v-HuxQ@mail.gmail.com>

Hi, you said my click event grabs text and doesn't show it. So how do get
it to show on the Text widget? So that when i clicked on item 1, 1.txt
shows on the Text widget. I can't connect click event to Text widget

from Tkinter import *
root = Tk()
root.title("Renkontoj kun Diferenculoj")
root.iconbitmap(r"C:\python27\DLLs/py.ico")
root.resizable(0, 0)
frame1 = LabelFrame(root, height=300, width=400, text='Lecionoj')
frame2 = LabelFrame(root, height=300, width=400, text='Tekstoj')
frame1.grid(row=0, column=0) frame2.grid(row=0, column=1)
def click(event):
index = list.curselection()[0]
file = open(r"C:\Users\deadmarshal\PycharmProjects\ali\1.txt").read()
list = Listbox(frame1)
list.insert(1, "Konati?u kun Kamila") list.insert(2, "Sinjoro Johano")
list.insert(3, "Onklino Marta")
scroll = Scrollbar(frame1, orient=VERTICAL, command=list.yview)
scroll.grid(row=0, column=2, sticky='ns') list.grid(row=0, column=1)
list.bind("<ButtonRelease-1>", click)
text = Text(frame2).grid() root.mainloop()

From rheeyauppaal at gmail.com  Thu Jan 14 05:09:11 2016
From: rheeyauppaal at gmail.com (Rheeya Uppaal)
Date: Thu, 14 Jan 2016 15:39:11 +0530
Subject: [Tutor] Procedure to install dlib on windows?
Message-ID: <CAA7L8V5GpVw-x=q5mzGqNyeKJ5r2zHK4AmOFCsNa1qs7-+asOw@mail.gmail.com>

I'm an absolute beginner at python, but have been asked to download the
dlib library on my windows system. I have followed the instructions given
here: http://dlib.net/compile.html

I have already downloaded cmake and Visual Studio. On the Visual Studio
command prompt, on running "python setup.py install" I get this:

running install
running bdist_egg
running build
Detected Python architecture: 64bit
Removing build directory
C:\Python\PackagesForPython\dlib-18.18\./tools/python/build
Configuring cmake ...
CMake Error in CMakeLists.txt:
error: cmake configuration failed!

Could anyone help me out please? I've tried whatever I can think of.

Thank you! Cheers!

From wombingsac at gmail.com  Thu Jan 14 05:43:52 2016
From: wombingsac at gmail.com (Whom Isac)
Date: Thu, 14 Jan 2016 20:43:52 +1000
Subject: [Tutor] Creating a webcrawler
In-Reply-To: <CAP16ngohTo5kyxVyRcraq_hXysHfSsTqZmiW4dN=mUDi30vq=Q@mail.gmail.com>
References: <CADXqDX8L1UsZNUOTR9oh-WBoOcdsqYqRWzMMcLroHr-c7tGPug@mail.gmail.com>
 <CAP16ngohTo5kyxVyRcraq_hXysHfSsTqZmiW4dN=mUDi30vq=Q@mail.gmail.com>
Message-ID: <CADXqDX_4Gxnxf5ohbwck-grk8bTgmZcn+CxQw0RDG6i-73QQBA@mail.gmail.com>

Thanks guys for your replies. I actually tried playing with my browser but
getting a web crawler to select a video and fetch the video link was not
helpful or should I say very hard for me as I am just a beginner level
programmer and python was the first language I learnt. I also learnt
javascript, ruby and html, bootstrap, C# recently. I may try this same
project in future with more knowledge.

On Sun, Jan 10, 2016 at 2:33 AM, bruce <badouglas at gmail.com> wrote:

> Hi Isac.
>
> I'm not going to get into the pythonic stuff.. People on the list are
> way better than I.  I've been doing a chunk of crawling, it's not too
> bad, depending on what you're trying to accomplish and the site you're
> targeting.
>
> So, no offense, but I'm going to treat you like a 6 year old (google
> it - from a movie!)
>
> You need to back up, and analyze the site/pages/structure you're going
> after. Use the tools - firefox - livehttpheaders/nettraffic/etc..
>   -you want to be able to see what the exchange is between the
> client/browser, as well as the server..
>   -often, this gives you the clues/insite to crafting the request from
> your client back to the server for the item/data you're going for...
>
> Once you've gotten that together, setup the basic process with
> wget/curl etc to get a feel for any weird issues - cert issues?
> -security issues - are cookies required - etc.. A good deal of this
> stuff can be resolved/checked out at this level, without jumping into
> coding..
>
> Once you're comfortable at this point, you can crank out some simple
> code to go after the site you're targeting.
>
> In the event you really have a javascript/dynamic site that you can't
> handle in any other manner, you're going to need to go use a 'headless
> browser' process.
>
> There are a number of headless browser projects - I think most run on
> the webit codebase (don't quote me). Casper/phantomjs, there are also
> pythonic implementations as well...
>
> So, there you go, should/hopefully this will get you on your way!
>
>
>
> On Fri, Jan 8, 2016 at 9:01 PM, Whom Isac <wombingsac at gmail.com> wrote:
> > Hi I want to create a web-crawler but dont have any lead to choose any
> > module. I have came across the Jsoup but I am not familiar with how to
> use
> > it in 3.5 as I tried looking at a similar web crawler codes from 3.4 dev
> > version.
> > I just want to build that crawler to crawl through a javascript enable
> site
> > and automatically detect a download link (for video file)
> > .
> > And should I be using pickles to write the data in the text file/ save
> file.
> > Thanks
> > _______________________________________________
> > Tutor maillist  -  Tutor at python.org
> > To unsubscribe or change subscription options:
> > https://mail.python.org/mailman/listinfo/tutor
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From wombingsac at gmail.com  Thu Jan 14 05:58:13 2016
From: wombingsac at gmail.com (Whom Isac)
Date: Thu, 14 Jan 2016 20:58:13 +1000
Subject: [Tutor] creating a mspaint utility
Message-ID: <CADXqDX-AMogT6twNqdvJhWRZu9GjDVKeD+D-L0-JR==Bjjp8Sg@mail.gmail.com>

Hi, I was wondering if it is possible to make a similar drawing tool with
basic functionality to draw lines, circles or square with python canvas.
I know how to draw them with canvas very well but the problem is the way I
am getting my mouse positions (initial& final).
I did not think it would have been a difficult work but I have spent 3-4
hours and out of luck. I get my functions to give me the mouse position
while moving, when pressed, original position or current position. But
don't know how to store them as a value. Should I write a function to store
them inside of it. I don't know because I had tried doing just that
recursively and failed and unfortunately I erased that part in the process.
I am not a genius and is not used to tkinter very well (to use command
function or anything) to store a value.
Here is my code. I kind of left some reduntant codes to show my try and
fail situation. Thus my code is quite long.

__author__ = 'WHOM ISAC'
import tkinter as tk
from tkinter import *
import sys
#import time

#app GUI
app=tk.Tk()
app.title("MSPAINT By Shams")
app.geometry('400x450')

#
show_event=app.winfo_pointerxy()
(X,Y)=show_event

#Mouse events
def original_mouse_position():
    show_event=app.winfo_pointerxy()
    (X,Y)=show_event
    print("The mouse are on: X:{0} Y:{1}".format(X,Y))
    label0=Label(frame1,text="Original Position",
relief=RAISED).pack(side=TOP,anchor="ne")
    label=Label(frame1,text="X    Y",relief=GROOVE).pack(side=BOTTOM,
anchor="ne")
    label1=Label(frame1,text=str(show_event),
relief=SUNKEN).pack(side=BOTTOM, anchor="ne")
    return

# Continuous Mouse Movement

def motion(event):
    x, y = event.x, event.y
    currentMousePosition=(x,y)
    print('MousePos: X:{0} Y:{1}'.format(x, y))
    return currentMousePosition
###app.bind('<Motion>', motion)-->WORKS but disabled from running



#Mouse Update And Position
def mouse_position():
    show_event=app.winfo_pointerxy()
    (X,Y)=show_event
    if '<Motion>'!=show_event :
        show_event=app.winfo_pointerxy()
        (X,Y)=show_event
        print("Current mouse are on: X:{0} Y:{1}".format(X,Y))
    label2=Label(frame1,text=str(show_event),
relief=GROOVE).pack(side=RIGHT)

#app.bind(mouse_position(),'Show')

#Mouse pressed
def Mouse_pressed(event):
    print("Right Click has been pressed.")
    initialpos=(X,Y)
    initialpos=app.winfo_pointerxy()
    #finalpos=motion(event)
    """
    while initialpos!=(0,0):
        initialpos=app.winfo_pointerxy()                   #Explain me why
it does not work.Should not it work?it's logical to do/call recursive
        finalpos=(0,0)
        if initialpos !=finalpos:
            if Mouse_pressed(event):
                finalpos=app.winfo_pointerxy()
    print(initialpos)
    """
    print(initialpos)
    return initialpos


#Mouse coordination
"""
initial_pos=Mouse_pressed
print(initial_pos)
time.sleep(1)
final_pos=Mouse_pressed
print(final_pos)
"""

#SOME WIDGETS:
lbl0=Label(app, text="This is a program that I have build Using Python.
Please Use it.", fg='blue', font='Times 9 bold').pack(fill=BOTH,anchor='nw')
####Frame for Original Mouse Position
frame1=Frame(app, bg='red', width=2).pack(fill=BOTH,anchor='ne')




#Canvas tools
"""CanvasFrame=Frame(app, width=300, height=200)"""
canvas_GUI=Canvas(app, height=300, width=300, bg='white')
       #canvas_draw_tool=canvas_GUI.create_line(20,0,100,200)
canvas_draw_tool=canvas_GUI.create_line(20,0,(X,Y)) #I know it won't work
unless I could get the initial position and final position but I don't have
a clue

##CanvasFrame Binding
canvas_GUI.bind("<Button-1>", Mouse_pressed)
canvas_GUI.bind('<Motion>', motion)
canvas_GUI.pack()

#QUIT Function
def quit():
    print("Quit function has been called. So I am quitting.")
    sys.exit()



#CLEAR Function
def clear():
    canvas_GUI.delete("all")
    print("Everything has been flushed.")



#Buttons
Button(app, text='Quit', command=quit).pack(anchor='sw',side=LEFT)
Button(app, text='Clear', command=clear).pack(anchor='sw',side=LEFT)
Button(app, text='Show', command=mouse_position).pack(anchor='sw',side=LEFT)
"""
initial_pos=Label(app,text="Initial
pos:{}".format(app.winfo_pointerxy())).pack()
final_pos=Label(app,text="Final
pos:{}".format(app.winfo_pointerxy())).pack()
canvas_GUI.bind(Mouse_pressed,initial_pos)
canvas_GUI.bind(Mouse_pressed,final_pos)
"""
#Mainloop running
original_mouse_position()
app.mainloop()

From wombingsac at gmail.com  Thu Jan 14 07:06:16 2016
From: wombingsac at gmail.com (Whom Isac)
Date: Thu, 14 Jan 2016 22:06:16 +1000
Subject: [Tutor] PLEASE I NEED HELP URGENTLY
In-Reply-To: <5696B913.1020502@timgolden.me.uk>
References: <911299850.2554269.1452369297130.JavaMail.yahoo.ref@mail.yahoo.com>
 <911299850.2554269.1452369297130.JavaMail.yahoo@mail.yahoo.com>
 <n762ut$e77$1@ger.gmane.org> <n7643l$2a6$1@ger.gmane.org>
 <n76cvv$ou2$1@ger.gmane.org> <5696B8E2.30408@timgolden.me.uk>
 <5696B913.1020502@timgolden.me.uk>
Message-ID: <CADXqDX8wDL1JW_BFBSfJnQ4Kq_3MAL9Xp6GAiRh6N=RQvqD7YQ@mail.gmail.com>

@precious Akams.
I don't know if you have tried to look for any resources e.g. python
directory to help you with this small syntax error. And I don't see
any relevance information with your callback error message with your
code. How did you able to run it?? I have not been reading my mails in
a while so I just honestly did not wanted upset you with anymore
words.
I felt you should learn to take a little bit precaution while coding.
Here is my Solution which I don't think you are using to pass or to
have cheated your homework:
class BankAccount(object):
    def __init__(self, initial_amount):
        self.balance=initial_amount
    def deposit (self, amount):
        self.balance+=amount
    def withdraw (self, amount):
        if self.balance>=amount:
            return ('invalid transaction')
        else:
            #What are trying to do eg.
            self.balance=self.balance-amount
            return self.balance
            pass

MinimumBalanceAcc=BankAccount #---> This created an instances
#To create a subclass
class MinimumBalanceAccount(BankAccount):
    #You don't need to call the init function if you are just trying
to use Bank account functions
# add any other additional methods below:
#eg.
    def show_balance(self):
        print(str(self.balance))


On Thu, Jan 14, 2016 at 6:52 AM, Tim Golden <mail at timgolden.me.uk> wrote:
>
> On 13/01/2016 20:51, Tim Golden wrote:
>>
>> Speaking as the list moderator in question over there: if I might
>> moderate Mark's well-known zeal...
>
>
> (Absolutely no pun intended!)
>
>
> TJG
> _______________________________________________
> Tutor maillist? -? Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From kwpolska at gmail.com  Thu Jan 14 09:59:41 2016
From: kwpolska at gmail.com (Chris Warrick)
Date: Thu, 14 Jan 2016 15:59:41 +0100
Subject: [Tutor] idle??
In-Reply-To: <n76d9t$1pn$1@ger.gmane.org>
References: <CAP16ngpVZRMRw8W-00FQdsMUqZ9R=GnYCRfUVZZg3sbsMzLCXA@mail.gmail.com>
 <CAMw+j7Kb42xzHoFRoXddjRAk7OQv69g-WVEKiw+weMNSpb49-w@mail.gmail.com>
 <n76d9t$1pn$1@ger.gmane.org>
Message-ID: <CAMw+j7+q0Dc4S9JZ2hRtx9314axHePoQRBTyB92vJ-L4oyX-Mg@mail.gmail.com>

On 13 January 2016 at 21:49, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> On 09/01/2016 10:38, Chris Warrick wrote:
>>
>> On 8 January 2016 at 20:07, bruce <badouglas at gmail.com> wrote:
>>>
>>> So, where does IDLE fit into this....
>>
>>
>> IDLE is a sad little ?IDE?, which is really ugly, because it?s written
>> in Tk. It lacks many IDE features. It comes with a really basic
>> debugger (that doesn?t even highlight the line that is being currently
>> executed?), function signature hinting, and some code completion.
>>
>
> Please ignore this drivel, he's spouted this before without giving any
> justification.  IDLE is perfectly adequate as a starter for Python.

I?m sorry, which part of ?ugly? (which you cannot deny, it doesn?t
match the OS most of the time), ?no debugger line highlighting?, ?no
line numbering? (which is a crucial feature of any code editor!) is
not enough ?justification??

For learning, a text editor (that?s better than notepad.exe) is
enough. However, the OP explicitly asked for an IDE, and as such, they
should get a good one.

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16

From alan.gauld at btinternet.com  Thu Jan 14 10:01:03 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 14 Jan 2016 15:01:03 +0000
Subject: [Tutor] Procedure to install dlib on windows?
In-Reply-To: <CAA7L8V5GpVw-x=q5mzGqNyeKJ5r2zHK4AmOFCsNa1qs7-+asOw@mail.gmail.com>
References: <CAA7L8V5GpVw-x=q5mzGqNyeKJ5r2zHK4AmOFCsNa1qs7-+asOw@mail.gmail.com>
Message-ID: <n78d7g$7le$1@ger.gmane.org>

On 14/01/16 10:09, Rheeya Uppaal wrote:
> I'm an absolute beginner at python, but have been asked to download the
> dlib library on my windows system. I have followed the instructions given
> here: http://dlib.net/compile.html

Its not clear why you are doing this.
Who asked you to download it? Why do they want you to use it?
Do you have experience in C++ (which seems to be how DLib is
documented)

The reason I ask is that DLib looks like a fairly specialised
tool that overlaps quite a lot of the standard libraries that
come with Python. It's certainly not something a novice Python
user would normally be playing with.

This list is really for teaching people the python language
and standard library. To get support on DLib you are probably
better asking on the DLib forum. Failing that you might try
the main Python list.

But I'd check you really need it first.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From james at uplinkzero.com  Thu Jan 14 10:15:03 2016
From: james at uplinkzero.com (James Chapman)
Date: Thu, 14 Jan 2016 15:15:03 +0000
Subject: [Tutor] Procedure to install dlib on windows?
In-Reply-To: <n78d7g$7le$1@ger.gmane.org>
References: <CAA7L8V5GpVw-x=q5mzGqNyeKJ5r2zHK4AmOFCsNa1qs7-+asOw@mail.gmail.com>
 <n78d7g$7le$1@ger.gmane.org>
Message-ID: <CAHvkzym5GuC0TwBvf15S=enDM8N=TUTevjkiNja-wwR6Ks_F-g@mail.gmail.com>

>From one of the Python examples:

# COMPILING/INSTALLING THE DLIB PYTHON INTERFACE
#   You can install dlib using the command:
#       pip install dlib
#
#   Alternatively, if you want to compile dlib yourself then go into the dlib
#   root folder and run:
#       python setup.py install
#   or
#       python setup.py install --yes USE_AVX_INSTRUCTIONS

It looks like the pip installer will install a pre-compiled lib with
python API bindings for you.

From breamoreboy at yahoo.co.uk  Thu Jan 14 10:20:53 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 14 Jan 2016 15:20:53 +0000
Subject: [Tutor] Procedure to install dlib on windows?
In-Reply-To: <CAA7L8V5GpVw-x=q5mzGqNyeKJ5r2zHK4AmOFCsNa1qs7-+asOw@mail.gmail.com>
References: <CAA7L8V5GpVw-x=q5mzGqNyeKJ5r2zHK4AmOFCsNa1qs7-+asOw@mail.gmail.com>
Message-ID: <n78edi$ruq$1@ger.gmane.org>

On 14/01/2016 10:09, Rheeya Uppaal wrote:
> I'm an absolute beginner at python, but have been asked to download the
> dlib library on my windows system. I have followed the instructions given
> here: http://dlib.net/compile.html
>
> I have already downloaded cmake and Visual Studio. On the Visual Studio
> command prompt, on running "python setup.py install" I get this:
>
> running install
> running bdist_egg
> running build
> Detected Python architecture: 64bit
> Removing build directory
> C:\Python\PackagesForPython\dlib-18.18\./tools/python/build
> Configuring cmake ...
> CMake Error in CMakeLists.txt:
> error: cmake configuration failed!
>
> Could anyone help me out please? I've tried whatever I can think of.
>
> Thank you! Cheers!

It is available on pypi for 2.7 and 3.4 so from a cmd prompt all you 
need is:-

pip install dlib.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From james at uplinkzero.com  Thu Jan 14 10:27:57 2016
From: james at uplinkzero.com (James Chapman)
Date: Thu, 14 Jan 2016 15:27:57 +0000
Subject: [Tutor] Modularity
In-Reply-To: <CAHvkzy=j_DxOaFsTkJ2GLCPV3h7s4+i3bMbEkKTPH=3L2+OW-w@mail.gmail.com>
References: <CAHvkzy=j_DxOaFsTkJ2GLCPV3h7s4+i3bMbEkKTPH=3L2+OW-w@mail.gmail.com>
Message-ID: <CAHvkzy=kk44KsitrUihK5TLr0e-EURJb42nu2866goPuwr-Pxg@mail.gmail.com>

I should have re-read that last reply before hitting send. Apologies
for the poor sentence construction!

Something I forgot to highlight before which might be related to your
initial question.

If you have a file called sound.py which contained a class called
WavFile, if you imported just sound like this:

        import sound

Then your class constructor would be called like this:

        wavFile = sound.WavFile()


Importing the module doesn't import the class, for that you'd do

        from sound import WavFile

        wavFile = WavFile()

From breamoreboy at yahoo.co.uk  Thu Jan 14 11:26:54 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 14 Jan 2016 16:26:54 +0000
Subject: [Tutor] Modularity
In-Reply-To: <CAHvkzy=kk44KsitrUihK5TLr0e-EURJb42nu2866goPuwr-Pxg@mail.gmail.com>
References: <CAHvkzy=j_DxOaFsTkJ2GLCPV3h7s4+i3bMbEkKTPH=3L2+OW-w@mail.gmail.com>
 <CAHvkzy=kk44KsitrUihK5TLr0e-EURJb42nu2866goPuwr-Pxg@mail.gmail.com>
Message-ID: <n78i9b$une$1@ger.gmane.org>

On 14/01/2016 15:27, James Chapman wrote:
> I should have re-read that last reply before hitting send. Apologies
> for the poor sentence construction!
>
> Something I forgot to highlight before which might be related to your
> initial question.
>
> If you have a file called sound.py which contained a class called
> WavFile, if you imported just sound like this:
>
>          import sound
>
> Then your class constructor would be called like this:
>
>          wavFile = sound.WavFile()
>
>
> Importing the module doesn't import the class, for that you'd do
>
>          from sound import WavFile
>
>          wavFile = WavFile()

Without any context this is completely meaningless, so what the hell are 
you talking about?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From cfkaran2 at gmail.com  Thu Jan 14 06:35:59 2016
From: cfkaran2 at gmail.com (Cem Karan)
Date: Thu, 14 Jan 2016 06:35:59 -0500
Subject: [Tutor] me, my arm, my availability ...
In-Reply-To: <201601132047.u0DKlhIc025984@theraft.openend.se>
References: <201601132047.u0DKlhIc025984@theraft.openend.se>
Message-ID: <D20AF40E-ECC4-4DD4-A679-392008F6088E@gmail.com>


On Jan 13, 2016, at 3:47 PM, Laura Creighton <lac at openend.se> wrote:

> 
> I fell recently.  Ought to be nothing, but a small chip of bone, either an
> existing one or one I just made is nicely wedged in the joint taking away
> a whole lot of the ability of my arm to rotate in the elbow joint.  Or
> hold my arm in a position that is usual for typing.  Plus,  now that the
> sprain/swelling is more or less over, the pain, unfortunately is not.
> 
> The real downside is that my typing speed is down from 135-140 wpm
> to 5-10 wmp.  At this rate, just getting my usual work done takes
> overtime.
> 
> Seems like surgery is needed to fix this. 
> 
> So I wanted you all to know, no, I haven't forgotten you and no haven't
> stopped caring.  I have just stopped being as __capable__ if you know
> what I mean.
> 
> Please take care of yourselves and each other.  I will often be reading
> even if typing is more than I can do right now.
> 
> Laura
> 
> ps -- (recent tutor discussion) I am with Alan and not with Mark.  I
> am happy as anything when people post their not-quite-working code for
> homework assignments here to tutor.  They aren't lazy bastards wanting
> somebody to do their assignments for them, they want to learn why what
> they are trying to do isn't working.  Sounds perfect for tutor to me.

Good luck healing!  Hope you get better soon.  Surgery has gotten a WHOLE lot better recently, they did wonders for my knee a few years back.  With luck, it'll be more or less outpatient surgery.

Good luck,
Cem Karan

From alan.gauld at btinternet.com  Thu Jan 14 19:32:51 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 15 Jan 2016 00:32:51 +0000
Subject: [Tutor] Clickable listbox opening .txt files
In-Reply-To: <CAMh2k3bdcDHg+vL3mubAEypP69oX3iGfR47Jct2mraP=v-HuxQ@mail.gmail.com>
References: <CAMh2k3bdcDHg+vL3mubAEypP69oX3iGfR47Jct2mraP=v-HuxQ@mail.gmail.com>
Message-ID: <n79enj$gvf$1@ger.gmane.org>

On 14/01/16 11:39, Ali Moradi wrote:
> Hi, you said my click event grabs text and doesn't show it. So how do get
> it to show on the Text widget?

Your text widget is calloed text.
So simply insert the text into the widget using

text.insert(END,file)

at the end of your click handler.

> def click(event):
>   index = list.curselection()[0]
>   file = open(r"C:\Users\deadmarshal\PycharmProjects\ali\1.txt").read()

> list = Listbox(frame1)
> list.insert(1, "Konati?u kun Kamila") list.insert(2, "Sinjoro Johano")
> list.insert(3, "Onklino Marta")
> scroll = Scrollbar(frame1, orient=VERTICAL, command=list.yview)
> scroll.grid(row=0, column=2, sticky='ns') list.grid(row=0, column=1)
> list.bind("<ButtonRelease-1>", click)
> text = Text(frame2).grid() root.mainloop()

However one problem is that you are creating your widget and calling
grid() which returns None. You need to split the line into two:

text = Text(frame2)
text.grid()

HTH

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Thu Jan 14 19:50:32 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 15 Jan 2016 00:50:32 +0000
Subject: [Tutor] creating a mspaint utility
In-Reply-To: <CADXqDX-AMogT6twNqdvJhWRZu9GjDVKeD+D-L0-JR==Bjjp8Sg@mail.gmail.com>
References: <CADXqDX-AMogT6twNqdvJhWRZu9GjDVKeD+D-L0-JR==Bjjp8Sg@mail.gmail.com>
Message-ID: <n79foo$37e$1@ger.gmane.org>

On 14/01/16 10:58, Whom Isac wrote:
> Hi, I was wondering if it is possible to make a similar drawing tool with
> basic functionality to draw lines, circles or square with python canvas.

Yes of course and there are at least a couple of online tutorials
on how to do that. Google is your friend.

> I did not think it would have been a difficult work but I have spent 3-4
> hours and out of luck. I get my functions to give me the mouse position
> while moving, when pressed, original position or current position. But
> don't know how to store them as a value. 

Use variables just like any other value.
GUI work is always harder than you expect and ful;l of
frustrating details. 3-4 hours on a GUI project is not
long at all.

> I am not a genius and is not used to tkinter very well (to use command
> function or anything) to store a value.

Which tkinter tutorial are you reading?

> import tkinter as tk
> from tkinter import *
> import sys

> #app GUI
> app=tk.Tk()
> app.title("MSPAINT By Shams")
> app.geometry('400x450')
> 
> #
> show_event=app.winfo_pointerxy()
> (X,Y)=show_event
> 
> #Mouse events
> def original_mouse_position():
>     show_event=app.winfo_pointerxy()
>     (X,Y)=show_event
>     print("The mouse are on: X:{0} Y:{1}".format(X,Y))
>     label0=Label(frame1,text="Original Position",
> relief=RAISED).pack(side=TOP,anchor="ne")

pavck() and grid() both return None so if you want to store a reference
to the widget you MUST split it over two lines

widgetVar = Widget()
widgetVar.pack()   # or grid()

Otherwise your widgetVar will just store None.

>     label=Label(frame1,text="X    Y",relief=GROOVE).pack(side=BOTTOM,
> anchor="ne")
>     label1=Label(frame1,text=str(show_event),
> relief=SUNKEN).pack(side=BOTTOM, anchor="ne")
>     return

This is confusing because your function is called
original_mouse_position() but it does a lot more than
that - it creates labels and all sorts. You should
either split this into several smaller functions
or change the name to reflect what the function
actually does.


> # Continuous Mouse Movement
> 
> def motion(event):
>     x, y = event.x, event.y
>     currentMousePosition=(x,y)
>     print('MousePos: X:{0} Y:{1}'.format(x, y))
>     return currentMousePosition
> ###app.bind('<Motion>', motion)-->WORKS but disabled from running

You return the x,y coordinates but the values are not
used by Tkinter when it calls the function in
response to a Motion event. You probably want
to store them in global variables - remember
to use the global keyword.

> 
> #Mouse Update And Position
> def mouse_position():
>     show_event=app.winfo_pointerxy()
>     (X,Y)=show_event

You've used these two lines several times. Probably
better to make them into a function.

>     if '<Motion>'!=show_event :

This will always be true since you are comparing
a literal string ('<Motion>') with an (x,y) tuple.
They can never be equal.

>         show_event=app.winfo_pointerxy()
>         (X,Y)=show_event

and yet again you repeat these lines.

>         print("Current mouse are on: X:{0} Y:{1}".format(X,Y))
>     label2=Label(frame1,text=str(show_event),
> relief=GROOVE).pack(side=RIGHT)
> 
> #app.bind(mouse_position(),'Show')
> 
> #Mouse pressed
> def Mouse_pressed(event):
>     print("Right Click has been pressed.")
>     initialpos=(X,Y)
>     initialpos=app.winfo_pointerxy()
>     #finalpos=motion(event)
>     """
>     while initialpos!=(0,0):
>         initialpos=app.winfo_pointerxy()                   #Explain me why
> it does not work.Should not it work?it's logical to do/call recursive

There is nothing recursive going on.
You set initialpos to the mouse position in the 3rd line
of this function. You then loop round doing it until the
mouse is on position 0,0 - ie extreme top left.

>         finalpos=(0,0)
>         if initialpos !=finalpos:
>             if Mouse_pressed(event):
>                 finalpos=app.winfo_pointerxy()

I've no idea what you think this bit is doing.

>     print(initialpos)
>     """
>     print(initialpos)
>     return initialpos
> 

That's as far as I went because there are enough big issues
to fix already that finding more would not be useful.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From lordrip at gmail.com  Thu Jan 14 20:11:21 2016
From: lordrip at gmail.com (=?UTF-8?Q?Ricardo_Mart=C3=ADnez?=)
Date: Fri, 15 Jan 2016 02:11:21 +0100
Subject: [Tutor] I want to share a source that i wrote
Message-ID: <CAMtNk+stBov38kH=OS3VdLgDx2pyHhqzWkTrvB8+RkAwmY1exQ@mail.gmail.com>

Hi guys, i wrote a simply MySQL interpretar with Python / Tk and I want to
ask if can I share with the people in the list to discuss about the code
and maybe help to the community.

The source code has 200 lines and is written in a single file.

P.D. please, Alan Gauld be kind with my mistakes about code or about my
words, I speak Spanish natively and English a little bit.

P.D.2. Alan it's only a joke, sorry in advice!

From alan.gauld at btinternet.com  Fri Jan 15 03:33:25 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 15 Jan 2016 08:33:25 +0000
Subject: [Tutor] I want to share a source that i wrote
In-Reply-To: <CAMtNk+stBov38kH=OS3VdLgDx2pyHhqzWkTrvB8+RkAwmY1exQ@mail.gmail.com>
References: <CAMtNk+stBov38kH=OS3VdLgDx2pyHhqzWkTrvB8+RkAwmY1exQ@mail.gmail.com>
Message-ID: <n7aasl$ekt$1@ger.gmane.org>

On 15/01/16 01:11, Ricardo Mart?nez wrote:
> Hi guys, i wrote a simply MySQL interpretar with Python / Tk and I want to
> ask if can I share with the people in the list to discuss about the code
> and maybe help to the community.
> 
> The source code has 200 lines and is written in a single file.

That should be OK.

Anything over 200 lines is quite big so you could try using
a pastebin instead of posting it by mail, its up to you.
posting is more convenient to read and to respond to,
but pastebin has the advantage of syntax colouring etc.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Fri Jan 15 03:34:37 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 15 Jan 2016 08:34:37 +0000
Subject: [Tutor] Fwd: Re:  creating a mspaint utility
In-Reply-To: <CADXqDX-q6WKpRrQr81oLdPSzB-E0yOFP5aZAPK4DDkNJ+bTGTA@mail.gmail.com>
References: <CADXqDX-q6WKpRrQr81oLdPSzB-E0yOFP5aZAPK4DDkNJ+bTGTA@mail.gmail.com>
Message-ID: <5698AF1D.6090701@btinternet.com>

forwarding to tutor list... please use ReplyAll when responding to the list.

-------- Forwarded Message --------
Subject: 	Re: [Tutor] creating a mspaint utility
Date: 	Fri, 15 Jan 2016 16:07:36 +1000
From: 	Whom Isac <wombingsac at gmail.com>
To: 	Alan Gauld <alan.gauld at btinternet.com>



Hi, Alan Gauld, in regards to the response I got from you, I am using
python 3.4 while writing the code but normally use 3.5. And this is just
a practice tutorial that I thought about my self to make it because I
know GUI-canvas can let you draw if you know the coordinates. I thought
maybe you can get the mouse positions and put them in order to draw a
line,etc.
I see you identified some issues which I have problem with because I
know mistakes already but You see if you have run the code into an
interpreter or IDLE it would show you the mouse co-ordinates each time
you right-click on the canvas (not the tkinter, intentionally) to draw.
I did not want to use global variable because I am not used to this
function in python or any lambda function because it's quite hard. 

And for original Mouse position function:

>     label=Label(frame1,text="X    Y",relief=GROOVE).pack(side=BOTTOM,
> anchor="ne")
>     label1=Label(frame1,text=str(show_event),
> relief=SUNKEN).pack(side=BOTTOM, anchor="ne")
>     return

 I wanted to show the mouseposition under a label with X and Y (at the
GUI-startup). Then if mouse position changes then make another label
underneath with the current mouse position with MousePosition method:

>def mouse_position():
>    show_event=app.winfo_pointerxy()
>    (X,Y)=show_event
>    if '<Motion>'!=show_event :
>        show_event=app.winfo_pointerxy()
>        (X,Y)=show_event
>        print("Current mouse are on: X:{0} Y:{1}".format(X,Y))
>    label2=Label(frame1,text=str(show_event),
relief=GROOVE).pack(side=RIGHT)

And you are right that I have to call the same code to find the
mouse-position again and again. Because I think if you don't right the
variable first or inside the function it will not work.

Then again I don't know what got me but I thought that I could write a
function for mouse pressed which will print(Right Click has been
pressed.) and take a variable to store the mouse position. Now I think I
can modify the code to use separetely to find the coordinates but I
don't know how to do that. Because in each right click the initialpos
and finalpos of the mouse will change and they would have the same
value. Should not they??

That's why I got so confused and made  a plan to write a small code with
it so it will compare final position and initial position with this:
 >finalpos=(0,0)
>         if initialpos !=finalpos:
>             if Mouse_pressed(event):
>                 finalpos=app.winfo_pointerxy()

It's not possible for final position to be (0,0) which I know better so
it would make the function to store a new finalposition. This attempt
failed so I comment them out. This is the same trick I used for getting
the initialposition which you did not include. So no point in arguing
about who is wrong.

If you had tested the code you might have notice that each time I am
pressing the show button the command keep printing or making a label to
the right. How do I keep only a single label and delele the old label.
And each time I click on the canvas initialposition and finalposition
become the same value. And even though I made sure the label-Original
Mouse Position should be followed by(" X Y ") and the mouse position
underneath the order is not right when I run the GUI.

So any idea how should I store the initial and final position e.g.
inside a function or as global value.?? or An advice to what should I do
about mouse position functions? Should I reduce any of them? Is it a
problem to put labels inside them?

If you have not run the GUI yet, here are some of the snapshot I took:
 


?

On Fri, Jan 15, 2016 at 10:50 AM, Alan Gauld <alan.gauld at btinternet.com
<mailto:alan.gauld at btinternet.com>> wrote:

    On 14/01/16 10:58, Whom Isac wrote:
    > Hi, I was wondering if it is possible to make a similar drawing
    tool with
    > basic functionality to draw lines, circles or square with python
    canvas.

    Yes of course and there are at least a couple of online tutorials
    on how to do that. Google is your friend.

    > I did not think it would have been a difficult work but I have
    spent 3-4
    > hours and out of luck. I get my functions to give me the mouse
    position
    > while moving, when pressed, original position or current position. But
    > don't know how to store them as a value.

    Use variables just like any other value.
    GUI work is always harder than you expect and ful;l of
    frustrating details. 3-4 hours on a GUI project is not
    long at all.

    > I am not a genius and is not used to tkinter very well (to use command
    > function or anything) to store a value.

    Which tkinter tutorial are you reading?

    > import tkinter as tk
    > from tkinter import *
    > import sys

    > #app GUI
    > app=tk.Tk()
    > app.title("MSPAINT By Shams")
    > app.geometry('400x450')
    >
    > #
    > show_event=app.winfo_pointerxy()
    > (X,Y)=show_event
    >
    > #Mouse events
    > def original_mouse_position():
    >     show_event=app.winfo_pointerxy()
    >     (X,Y)=show_event
    >     print("The mouse are on: X:{0} Y:{1}".format(X,Y))
    >     label0=Label(frame1,text="Original Position",
    > relief=RAISED).pack(side=TOP,anchor="ne")

    pavck() and grid() both return None so if you want to store a reference
    to the widget you MUST split it over two lines

    widgetVar = Widget()
    widgetVar.pack()   # or grid()

    Otherwise your widgetVar will just store None.

    >     label=Label(frame1,text="X    Y",relief=GROOVE).pack(side=BOTTOM,
    > anchor="ne")
    >     label1=Label(frame1,text=str(show_event),
    > relief=SUNKEN).pack(side=BOTTOM, anchor="ne")
    >     return

    This is confusing because your function is called
    original_mouse_position() but it does a lot more than
    that - it creates labels and all sorts. You should
    either split this into several smaller functions
    or change the name to reflect what the function
    actually does.


    > # Continuous Mouse Movement
    >
    > def motion(event):
    >     x, y = event.x, event.y
    >     currentMousePosition=(x,y)
    >     print('MousePos: X:{0} Y:{1}'.format(x, y))
    >     return currentMousePosition
    > ###app.bind('<Motion>', motion)-->WORKS but disabled from running

    You return the x,y coordinates but the values are not
    used by Tkinter when it calls the function in
    response to a Motion event. You probably want
    to store them in global variables - remember
    to use the global keyword.

    >
    > #Mouse Update And Position
    > def mouse_position():
    >     show_event=app.winfo_pointerxy()
    >     (X,Y)=show_event

    You've used these two lines several times. Probably
    better to make them into a function.

    >     if '<Motion>'!=show_event :

    This will always be true since you are comparing
    a literal string ('<Motion>') with an (x,y) tuple.
    They can never be equal.

    >         show_event=app.winfo_pointerxy()
    >         (X,Y)=show_event

    and yet again you repeat these lines.

    >         print("Current mouse are on: X:{0} Y:{1}".format(X,Y))
    >     label2=Label(frame1,text=str(show_event),
    > relief=GROOVE).pack(side=RIGHT)
    >
    > #app.bind(mouse_position(),'Show')
    >
    > #Mouse pressed
    > def Mouse_pressed(event):
    >     print("Right Click has been pressed.")
    >     initialpos=(X,Y)
    >     initialpos=app.winfo_pointerxy()
    >     #finalpos=motion(event)
    >     """
    >     while initialpos!=(0,0):
    >         initialpos=app.winfo_pointerxy()                 
     #Explain me why
    > it does not work.Should not it work?it's logical to do/call recursive

    There is nothing recursive going on.
    You set initialpos to the mouse position in the 3rd line
    of this function. You then loop round doing it until the
    mouse is on position 0,0 - ie extreme top left.

    >         finalpos=(0,0)
    >         if initialpos !=finalpos:
    >             if Mouse_pressed(event):
    >                 finalpos=app.winfo_pointerxy()

    I've no idea what you think this bit is doing.

    >     print(initialpos)
    >     """
    >     print(initialpos)
    >     return initialpos
    >

    That's as far as I went because there are enough big issues
    to fix already that finding more would not be useful.

    --
    Alan G
    Author of the Learn to Program web site
    http://www.alan-g.me.uk/
    http://www.amazon.com/author/alan_gauld
    Follow my photo-blog on Flickr at:
    http://www.flickr.com/photos/alangauldphotos


    _______________________________________________
    Tutor maillist  -  Tutor at python.org <mailto:Tutor at python.org>
    To unsubscribe or change subscription options:
    https://mail.python.org/mailman/listinfo/tutor





From alan.gauld at btinternet.com  Fri Jan 15 07:20:55 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 15 Jan 2016 12:20:55 +0000
Subject: [Tutor] Fwd: Re: creating a mspaint utility
In-Reply-To: <5698AF1D.6090701@btinternet.com>
References: <CADXqDX-q6WKpRrQr81oLdPSzB-E0yOFP5aZAPK4DDkNJ+bTGTA@mail.gmail.com>
 <5698AF1D.6090701@btinternet.com>
Message-ID: <n7ao77$pfn$1@ger.gmane.org>

On 15/01/16 08:34, Alan Gauld wrote:

> maybe you can get the mouse positions and put them in order to draw a
> line,etc.

Yes, that's a valid approach to building a multi-segment line.
Normally you store the points in a list somewhere.

> I did not want to use global variable because I am not used to this
> function in python or any lambda function because it's quite hard. 

You can avoid lambdas but I think you need to use globals to
get this to work.

You are already using some global variables in your code,
you just need to identify which ones you want to change
in your functions. That's simply done using the global
keyword, so your function would look like:

def some_function(event):
   global widget_name
   etc...
   widget_name = new_value
   etc...

It is really quite easy and allows you to store information
from inside a function so that other functions can see it.

The alternative strategy involves creating classes and
objects but that is possibly too advanced for you
just now.

The problem with your existing code is that you are
creating the widgets inside functions. But when the
function ends you lose all references to those widgets
so you cannot modify them or even read them  in any
way. (Although the variables all currently refer to
None because you use pack() in the same line as
you create them.)

>> def mouse_position():
>>    show_event=app.winfo_pointerxy()
>>    (X,Y)=show_event
>>    if '<Motion>'!=show_event :
>>        show_event=app.winfo_pointerxy()
>>        (X,Y)=show_event
>>        print("Current mouse are on: X:{0} Y:{1}".format(X,Y))
>>    label2=Label(frame1,text=str(show_event),
> relief=GROOVE).pack(side=RIGHT)
> 
> And you are right that I have to call the same code to find the
> mouse-position again and again. 

Actually you don't really need a function there, I just
realized you can simplify it by just rewriting it as a
single line:

X,Y = app.winfo_pointerxy()

> Then again I don't know what got me but I thought that I could write a
> function for mouse pressed which will print(Right Click has been
> pressed.) and take a variable to store the mouse position. Now I think I
> can modify the code to use separetely to find the coordinates but I
> don't know how to do that. Because in each right click the initialpos
> and finalpos of the mouse will change and they would have the same
> value. Should not they??

Sorry, I'm not sure what you mean there.
Can you try explaining again please?

> If you had tested the code you might have notice that each time I am
> pressing the show button the command keep printing or making a label to
> the right. How do I keep only a single label and delele the old label.

You need to create a global variable and then update that label.

> So any idea how should I store the initial and final position e.g.
> inside a function or as global value.?? 

Anything that you do  in one function that you want to be visible
in another function (or even the same one called a second time)
will need to be stored in a global variable. Variables inside
functions get deleted as soon as the function exits so you cannot
refer to them again. Each time the function gets called it creates
a new set of variables it does not use the same ones as the last
time.

> about mouse position functions? Should I reduce any of them? Is it a
> problem to put labels inside them?

Its OK to create labels inside but they will always be new
labels unless you  use global variables.


[Note 1:
Strictly speaking there are some other ways to maintain data
between function calls but I'm pretty sure they are too
esoteric for your purposes. Globals or classes are by far
the simplest solutions

Note 2:
Technically the widgets you are creating do have references
outside the function, otherwise they would be destroyed, but
to reach them involves traversing the app object's containment
tree which is relatively complicated and error prone
]

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From wolfrage8765 at gmail.com  Fri Jan 15 10:39:31 2016
From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com)
Date: Fri, 15 Jan 2016 10:39:31 -0500
Subject: [Tutor] Fwd: Re: creating a mspaint utility
In-Reply-To: <n7ao77$pfn$1@ger.gmane.org>
References: <CADXqDX-q6WKpRrQr81oLdPSzB-E0yOFP5aZAPK4DDkNJ+bTGTA@mail.gmail.com>
 <5698AF1D.6090701@btinternet.com> <n7ao77$pfn$1@ger.gmane.org>
Message-ID: <CAOhNYvnv-a0fWHUQv1f+ah8qkKLgam5NTr77u1aKHatYjzE+Ng@mail.gmail.com>

Might I also recommend Kivy as the GUI. It has a nice tutorial that is
actually on this subject.
https://kivy.org/docs/tutorials/firstwidget.html

From wolfrage8765 at gmail.com  Fri Jan 15 10:56:09 2016
From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com)
Date: Fri, 15 Jan 2016 10:56:09 -0500
Subject: [Tutor] Modularity
In-Reply-To: <488FA568-B4B5-4698-A226-9757C1AF9D2C@gmail.com>
References: <488FA568-B4B5-4698-A226-9757C1AF9D2C@gmail.com>
Message-ID: <CAOhNYvnJ8yjo-RmCTpGeudSR7q3qRbKRa9MC41niC3Y6H=k9Kg@mail.gmail.com>

 Modularity of code is difficult for sure and is kind of an art. To do
it best you will have to read about and implement some of the Design
Patterns. There are many but some of the ones that I have leaned
towards over time are Composition over inheritance, Message Passing
Interfaces, Mediator, and Model View Controller. Each of these is a
large subject in there own right; and how to use each in your use case
takes care and consideration. You also will need to consider that
there is a learning curve associated with each design pattern so you
will loose some time as you try to learn the pattern before it will
begin to save you time by making your code less coupled.

From jsutar at gmail.com  Fri Jan 15 11:25:28 2016
From: jsutar at gmail.com (Jignesh Sutar)
Date: Fri, 15 Jan 2016 16:25:28 +0000
Subject: [Tutor] str.strip strange result...?
Message-ID: <CACvW2fxwvHHKW0WOHDiLSuGSmv6_uTK82HYUkQtgsCRP8fqC4Q@mail.gmail.com>

#python2.7

>>> s="V01_1"
>>> s.strip("_1")
'V0'


Wouldn't you expect the result to be "V01" ?


Cheers
Jignesh

From __peter__ at web.de  Fri Jan 15 11:37:25 2016
From: __peter__ at web.de (Peter Otten)
Date: Fri, 15 Jan 2016 17:37:25 +0100
Subject: [Tutor] str.strip strange result...?
References: <CACvW2fxwvHHKW0WOHDiLSuGSmv6_uTK82HYUkQtgsCRP8fqC4Q@mail.gmail.com>
Message-ID: <n7b787$ktu$1@ger.gmane.org>

Jignesh Sutar wrote:

> #python2.7
> 
>>>> s="V01_1"
>>>> s.strip("_1")
> 'V0'
> 
> 
> Wouldn't you expect the result to be "V01" ?

str.strip() doesn't strip off a suffix or prefix; its first argument is 
interpreted as a character set, i. e. as long as s ends/starts with any of 
the characters "_" or "1", remove that.

If you want to remove a suffix you have to write

if suffix and s.endswith(suffix):
    s = s[:-len(suffix)]


From wolfrage8765 at gmail.com  Fri Jan 15 12:06:38 2016
From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com)
Date: Fri, 15 Jan 2016 12:06:38 -0500
Subject: [Tutor] Message Passing & User Interfaces
Message-ID: <CAOhNYvm2dmfW8Mj+dgn6=FSTE3TzaJ81kvg7ke4PZ-W+yGvAJg@mail.gmail.com>

I realize this is a higher level question; so please direct me as
appropriate. I can not seem to find a good standard that is in
practical use for controlling user interfaces via a message passing.
Does any one have any links to such an example? I would be most
grateful. Thank you.

From cperry172 at hotmail.com  Fri Jan 15 11:49:55 2016
From: cperry172 at hotmail.com (Chad Perry)
Date: Fri, 15 Jan 2016 16:49:55 +0000
Subject: [Tutor] Hard Drive wipe
Message-ID: <SN1PR13MB0447E5351E6E506927138F02F4CD0@SN1PR13MB0447.namprd13.prod.outlook.com>

#The function for writing random data to the disk.
def random():
    print ""
    os.system("/sbin/fdisk -l")
    print ""
    print "Please choose a device to kill.  Remember if you want to"
    print "wipe the whole drive and not just a partition, you can"
    print "remove the number appended.  Example /dev/sdc1 becomes /dev/sdc ."
    print ""
    device=raw_input("Enter device: ")
    print ""
    count=input("How many times would you like to wipe the device? ")
    print ""
    print "Writing changes to disk.  All data on %s will be lost."%(device)
    print ""
    raw_input("Press Enter to continue, or Ctrl+C to exit: ")
    print ""
    lap=1
    for i in range(count):
        print "Processing wipe count %s of %s..."%(lap, count)
        os.system(("dd if=/dev/urandom of=%s")%(device))
        lap=lap+1

I need to know how to substitute for the drive letter for the following drives.

sad-sdp
also will need to wipe data from /dev/md1

I believe that the script is sound just sub's

From cperry172 at hotmail.com  Fri Jan 15 12:07:58 2016
From: cperry172 at hotmail.com (Chad Perry)
Date: Fri, 15 Jan 2016 17:07:58 +0000
Subject: [Tutor] Substitution function needed
Message-ID: <SN1PR13MB04474B9165EAABE6DDA4D2B4F4CD0@SN1PR13MB0447.namprd13.prod.outlook.com>

#The function for writing random data to the disk.
def random():
   print ""
   os.system("/sbin/fdisk -l")
   print ""
   print "Please choose a device to kill.  Remember if you want to"
   print "wipe the whole drive and not just a partition, you can"
   print "remove the number appended.  Example /dev/sdc1 becomes /dev/sdc ."
   print ""
   device=raw_input("Enter device: ")
   print ""
   count=input("How many times would you like to wipe the device? ")
   print ""
   print "Writing changes to disk.  All data on %s will be lost."%(device)
   print ""
   raw_input("Press Enter to continue, or Ctrl+C to exit: ")
   print ""
   lap=1
   for i in range(count):
       print "Processing wipe count %s of %s..."%(lap, count)
       os.system(("dd if=/dev/urandom of=%s")%(device))
       lap=lap+1

I need to know how to substitute for the drive letter for the following drives.

sad-sdp
also will need to wipe data from /dev/md1

I believe that the script is sound just sub's

From cegarcia0323 at gmail.com  Fri Jan 15 11:33:24 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Fri, 15 Jan 2016 11:33:24 -0500
Subject: [Tutor] Help!
Message-ID: <CABV0pWwtPaLEbsQpU70hD-ktBwfXrTYe8A0BSOLYU+vTSaKZtg@mail.gmail.com>

Hi,
So what I am working on is taking a csv file and only taking 2 columns from
the spreadsheet and out putting that to a text file. Then taking those two
columns and organize them by product(key) and outputting the
description(values) that are associated. Some products have a lot of
duplicate descriptions and I am trying to get the counts of those. I have a
piece of code that takes anything greater then 5 and prints that and also
anything 4 or less goes into an 'other' category with the counts.So what I
am trying to do now is import a csv and change it to a text file with the
same naming convention as the csv. Below is my functions code:

import csv
import json
import sys
from collections import defaultdict
from collections import Counter

class dictionary():
def __init__(self, filename):
self.dict = defaultdict(list)
self.counted_dict = defaultdict(list)
self.grouped_dict = defaultdict(list)
self.other_dict = defaultdict(list)
self.final_dict = defaultdict(list)
self.total_dict = defaultdict(list)
self.txt_output = " "
def populate_dict(self, filename):
with open (filename, 'rb') as f:
reader = csv.reader(f)
next(reader, None)
for row in reader:
self.dict[row[2]].append(row[3])
def total_counts(self):
for key in self.dict.keys():
total = 0
b = Counter(self.dict[key])
for value in b:
total += b[value]
self.total_dict.update({key: str(total)})
def all_counts(self):
data_count = Counter()
for key in self.dict.keys():
self.counted_dict.update({key: Counter(self.dict[key])})
def grouped_counts(self):
for key in self.dict.keys():
total = 0
c = Counter(self.dict[key])
for value in c:
if c[value] >= 5:
self.grouped_dict.update({value: key + ': ' + str(c[value])})
elif c[value] <= 4:
total += c[value]
self.other_dict.update({key: 'other: ' + str(total)})
self.final_dict = self.grouped_dict, self.other_dict, self.total_dict,
def txt_output(self, filename):
a = filename.split('.')
self.txt_output = a + '.txt'
print a
def json_output(self):
with open (self.txt_output.txt, 'w') as text_file:
json.dump(self.final_dict, text_file, sort_keys = True, indent = 4)



What I am having issues with is the def txt_output that is where I am
trying to take the .csv off and add the .txt but keep the filename the
same. For example, having a filename "weekly_20160102.csv" and then create
a txt filename with "weekly_20160102.txt" and have all the counts and
products in the text file. Is there any way to do this?

From cegarcia0323 at gmail.com  Fri Jan 15 08:14:34 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Fri, 15 Jan 2016 08:14:34 -0500
Subject: [Tutor] Help!
Message-ID: <CABV0pWzDpZ3DgmoLutGzXR3ybHhvc1QTWapjA-LbPGdgu1HE0Q@mail.gmail.com>

Hi,
So what I am working on is taking a csv file and only taking 2 columns from
the spreadsheet and out putting that to a text file. Then taking those two
columns and organize them by product(key) and outputting the
description(values) that are associated. Some products have a lot of
duplicate descriptions and I am trying to get the counts of those. I have a
piece of code that takes anything greater then 5 and prints that and also
anything 4 or less goes into an 'other' category with the counts.So what I
am trying to do now is import a csv and change it to a text file with the
same naming convention as the csv. Below is my functions code:

import csv
import json
import sys
from collections import defaultdict
from collections import Counter

class dictionary():
def __init__(self, filename):
self.dict = defaultdict(list)
self.counted_dict = defaultdict(list)
self.grouped_dict = defaultdict(list)
self.other_dict = defaultdict(list)
self.final_dict = defaultdict(list)
self.total_dict = defaultdict(list)
self.txt_output = " "
def populate_dict(self, filename):
with open (filename, 'rb') as f:
reader = csv.reader(f)
next(reader, None)
for row in reader:
self.dict[row[2]].append(row[3])
def total_counts(self):
for key in self.dict.keys():
total = 0
b = Counter(self.dict[key])
for value in b:
total += b[value]
self.total_dict.update({key: str(total)})
def all_counts(self):
data_count = Counter()
for key in self.dict.keys():
self.counted_dict.update({key: Counter(self.dict[key])})
def grouped_counts(self):
for key in self.dict.keys():
total = 0
c = Counter(self.dict[key])
for value in c:
if c[value] >= 5:
self.grouped_dict.update({value: key + ': ' + str(c[value])})
elif c[value] <= 4:
total += c[value]
self.other_dict.update({key: 'other: ' + str(total)})
self.final_dict = self.grouped_dict, self.other_dict, self.total_dict,
def txt_output(self, filename):
a = filename.split('.')
self.txt_output = a + '.txt'
print a
def json_output(self):
with open (self.txt_output.txt, 'w') as text_file:
json.dump(self.final_dict, text_file, sort_keys = True, indent = 4)



What I am having issues with is the def txt_output that is where I am
trying to take the .csv off and add the .txt but keep the filename the
same. For example, having a filename "weekly_20160102.csv" and then create
a txt filename with "weekly_20160102.txt" and have all the counts and
products in the text file. Is there any way to do this?

From breamoreboy at yahoo.co.uk  Fri Jan 15 11:33:50 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 15 Jan 2016 16:33:50 +0000
Subject: [Tutor] str.strip strange result...?
In-Reply-To: <CACvW2fxwvHHKW0WOHDiLSuGSmv6_uTK82HYUkQtgsCRP8fqC4Q@mail.gmail.com>
References: <CACvW2fxwvHHKW0WOHDiLSuGSmv6_uTK82HYUkQtgsCRP8fqC4Q@mail.gmail.com>
Message-ID: <n7b72c$i4q$1@ger.gmane.org>

On 15/01/2016 16:25, Jignesh Sutar wrote:
> #python2.7
>
>>>> s="V01_1"
>>>> s.strip("_1")
> 'V0'
>
> Wouldn't you expect the result to be "V01" ?
>
> Cheers
> Jignesh

No, never expect anything from a given programming language.  What did 
you not understand about this 
https://docs.python.org/3/library/stdtypes.html#str.strip when you read it?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From jsutar at gmail.com  Fri Jan 15 12:50:19 2016
From: jsutar at gmail.com (Jignesh Sutar)
Date: Fri, 15 Jan 2016 17:50:19 +0000
Subject: [Tutor] str.strip strange result...?
In-Reply-To: <n7b72c$i4q$1@ger.gmane.org>
References: <CACvW2fxwvHHKW0WOHDiLSuGSmv6_uTK82HYUkQtgsCRP8fqC4Q@mail.gmail.com>
 <n7b72c$i4q$1@ger.gmane.org>
Message-ID: <CACvW2fwY=C3G61=G0LBDdpz=sk1Z5T0Wp8udn9EsEGOPD76Xeg@mail.gmail.com>

Gotcha and thank you for the reminder to read the documentation. Very
clear, indeed.

Many thanks!
Cheers
Jignesh
On Fri, 15 Jan 2016 at 17:32, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:

> On 15/01/2016 16:25, Jignesh Sutar wrote:
> > #python2.7
> >
> >>>> s="V01_1"
> >>>> s.strip("_1")
> > 'V0'
> >
> > Wouldn't you expect the result to be "V01" ?
> >
> > Cheers
> > Jignesh
>
> No, never expect anything from a given programming language.  What did
> you not understand about this
> https://docs.python.org/3/library/stdtypes.html#str.strip when you read
> it?
>
> --
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
>
> Mark Lawrence
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From __peter__ at web.de  Fri Jan 15 12:50:57 2016
From: __peter__ at web.de (Peter Otten)
Date: Fri, 15 Jan 2016 18:50:57 +0100
Subject: [Tutor] Help!
References: <CABV0pWzDpZ3DgmoLutGzXR3ybHhvc1QTWapjA-LbPGdgu1HE0Q@mail.gmail.com>
Message-ID: <n7bbi2$453$1@ger.gmane.org>

Chelsea G wrote:

> What I am having issues with is the def txt_output that is where I am
> trying to take the .csv off and add the .txt but keep the filename the
> same. For example, having a filename "weekly_20160102.csv" and then create
> a txt filename with "weekly_20160102.txt" and have all the counts and
> products in the text file. Is there any way to do this?

The best approach is to solve small subproblems in a simple experimental 
file or in the interactive interpreter before you integrate the solution as 
a function that you have tested and found to be working as intended.

If you look around the os.path module you'll find the splitext() function 
which works like this:

>>> import os.path
>>> os.path.splitext("/foo/bar/baz.ext")
('/foo/bar/baz', '.ext')

To change the extension you can do

>>> os.path.splitext("/foo/bar/my.csv")[0] + ".txt"
'/foo/bar/my.txt'

Now put it into a function:

>>> def change_ext(filename, newext):
...     return os.path.splitext(filename) + newext
... 

Let's test:

>>> change_ext("/foo/bar/my.one", ".two")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in change_ext
TypeError: can only concatenate tuple (not "str") to tuple

Oops, I made an error. Do you spot it? Let's fix it:

>>> def change_ext(filename, newext):
...     return os.path.splitext(filename)[0] + newext
... 
>>> change_ext("/foo/bar/my.one", ".two")
'/foo/bar/my.two'

Ok, this seems to work. You can do more tests (e.g. how will multiple 
extensions like "archive.tar.gz" or missing extensions like "somescript" be 
handled) and once it works as intended integrate it into your script.


From alan.gauld at btinternet.com  Fri Jan 15 19:32:40 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 16 Jan 2016 00:32:40 +0000
Subject: [Tutor] Substitution function needed
In-Reply-To: <SN1PR13MB04474B9165EAABE6DDA4D2B4F4CD0@SN1PR13MB0447.namprd13.prod.outlook.com>
References: <SN1PR13MB04474B9165EAABE6DDA4D2B4F4CD0@SN1PR13MB0447.namprd13.prod.outlook.com>
Message-ID: <n7c338$sl2$1@ger.gmane.org>

On 15/01/16 17:07, Chad Perry wrote:
> #The function for writing random data to the disk.
> def random():
>    os.system("/sbin/fdisk -l")

>    device=raw_input("Enter device: ")
>    count=input("How many times would you like to wipe the device? ")
>    raw_input("Press Enter to continue, or Ctrl+C to exit: ")
>    lap=1
>    for i in range(count):
>        print "Processing wipe count %s of %s..."%(lap, count)
>        os.system(("dd if=/dev/urandom of=%s")%(device))
>        lap=lap+1
> 
> I need to know how to substitute for the drive letter for the following drives.
> 
> sad-sdp
> also will need to wipe data from /dev/md1

Sorry, it's not clear to me exactly what you want to substitute.
Can you provide sample input and output?
For example, if I input
/dev/sda
What do you want to happen?

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From robertvstepp at gmail.com  Fri Jan 15 22:30:57 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Fri, 15 Jan 2016 21:30:57 -0600
Subject: [Tutor] Is there a preference of "class MyClass:" or "class
 MyClass(object):" for Py3?
Message-ID: <CANDiX9KyoKEswOjTh4sjBiJC-hFPhOkw9SF-907SX1dG9sQBuQ@mail.gmail.com>

Pythonic style/preference question:  For strictly Python 3 code, is
there any preference for

class MyClass:
    pass

versus the more explicit

class MyClass(object):
    pass

?

TIA!

-- 
boB

From robertvstepp at gmail.com  Fri Jan 15 23:20:41 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Fri, 15 Jan 2016 22:20:41 -0600
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
Message-ID: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>

At https://docs.python.org/3.4/library/stdtypes.html#sequence-types-list-tuple-range
it states:

"s.insert(i, x) inserts x into s at the index given by i (same as s[i:i] = [x])"

I find this confusing.  First, at the interpreter, whenever I type in:

>>> things
[0, 'Hmm...', 3, 'WhackABunny', 6, '?']
>>> things[-1:-1]
[]
>>> things[0:0]
[]

I always get an empty list, which is actually what I was expecting, so
I do not see how s[i:i] can ever equal [x].

The second thing I find puzzling is the docs say x is inserted at
position i, while in the interpreter:

>>> help(list.insert)
Help on method_descriptor:

insert(...)
    L.insert(index, object) -- insert object before index

The "...insert object before index" makes sense to me, but "...inserts
x into s at the index given by i..." does not because:

>>> things.insert(-1, 'What the heck?!?')
>>> things
[0, 'Hmm...', 3, 'WhackABunny', 6, 'What the heck?!?', '?']

"...at the index..." to me would mean that 'What the heck?!?' should
become the last item in the list.  Again, the interpreter help gave
what I was expecting.

Am I just being dense or are the docs in this instance confusing?

-- 
boB

From cs at zip.com.au  Fri Jan 15 23:53:51 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 16 Jan 2016 15:53:51 +1100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
Message-ID: <20160116045351.GA59660@cskk.homeip.net>

On 15Jan2016 22:20, boB Stepp <robertvstepp at gmail.com> wrote:
>At https://docs.python.org/3.4/library/stdtypes.html#sequence-types-list-tuple-range
>it states:
>
>"s.insert(i, x) inserts x into s at the index given by i (same as s[i:i] = [x])"
>
>I find this confusing.  First, at the interpreter, whenever I type in:
>
>>>> things
>[0, 'Hmm...', 3, 'WhackABunny', 6, '?']
>>>> things[-1:-1]
>[]
>>>> things[0:0]
>[]
>
>I always get an empty list, which is actually what I was expecting, so
>I do not see how s[i:i] can ever equal [x].

It isn't an equality test (==), it is an assignent. It is saying "set the zero 
length sequence at index i to the one element sequence [x]".

>The second thing I find puzzling is the docs say x is inserted at
>position i, while in the interpreter:
>
>>>> help(list.insert)
>Help on method_descriptor:
>
>insert(...)
>    L.insert(index, object) -- insert object before index
>
>The "...insert object before index" makes sense to me, but "...inserts
>x into s at the index given by i..." does not because:

Personally I'd rather it said "insert object at index". For "before" I'd need 
something longer, like "insert object before the elements from index onward".

>>>> things.insert(-1, 'What the heck?!?')
>>>> things
>[0, 'Hmm...', 3, 'WhackABunny', 6, 'What the heck?!?', '?']
>
>"...at the index..." to me would mean that 'What the heck?!?' should
>become the last item in the list.  Again, the interpreter help gave
>what I was expecting.

To me it means "insert 'x' so that its index is 'i'".

>Am I just being dense or are the docs in this instance confusing?

They may be a bit confusing, though I do think you're misreading the "=" bit at 
the top.

Cheers,
Cameron Simpson <cs at zip.com.au>

From robertvstepp at gmail.com  Sat Jan 16 00:05:35 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Fri, 15 Jan 2016 23:05:35 -0600
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <20160116045351.GA59660@cskk.homeip.net>
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116045351.GA59660@cskk.homeip.net>
Message-ID: <CANDiX9KuSR2bGfS+m4h8=zgKoumun7B1jkyR36NkMUcE2bmV9A@mail.gmail.com>

On Fri, Jan 15, 2016 at 10:53 PM, Cameron Simpson <cs at zip.com.au> wrote:
> On 15Jan2016 22:20, boB Stepp <robertvstepp at gmail.com> wrote:

>> I always get an empty list, which is actually what I was expecting, so
>> I do not see how s[i:i] can ever equal [x].
>
>
> It isn't an equality test (==), it is an assignent. It is saying "set the
> zero length sequence at index i to the one element sequence [x]".

Ah!  That makes sense.


>>>>> things.insert(-1, 'What the heck?!?')
>>>>> things
>>
>> [0, 'Hmm...', 3, 'WhackABunny', 6, 'What the heck?!?', '?']
>>
>> "...at the index..." to me would mean that 'What the heck?!?' should
>> become the last item in the list.  Again, the interpreter help gave
>> what I was expecting.
>
>
> To me it means "insert 'x' so that its index is 'i'".

But that's my point!  In my example x (here 'What the heck?!?') is
*not* at index i (here, -1).  Instead it winds up at index -2.  But
this fits in perfectly with the interpreter help, since it winds up
*before* index i (-1).

-- 
boB

From cs at zip.com.au  Sat Jan 16 00:32:34 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 16 Jan 2016 16:32:34 +1100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <CANDiX9KuSR2bGfS+m4h8=zgKoumun7B1jkyR36NkMUcE2bmV9A@mail.gmail.com>
References: <CANDiX9KuSR2bGfS+m4h8=zgKoumun7B1jkyR36NkMUcE2bmV9A@mail.gmail.com>
Message-ID: <20160116053234.GA43201@cskk.homeip.net>

On 15Jan2016 23:05, boB Stepp <robertvstepp at gmail.com> wrote:
>On Fri, Jan 15, 2016 at 10:53 PM, Cameron Simpson <cs at zip.com.au> wrote:
>>>>>> things.insert(-1, 'What the heck?!?')
>>>>>> things
>>>
>>> [0, 'Hmm...', 3, 'WhackABunny', 6, 'What the heck?!?', '?']
>>>
>>> "...at the index..." to me would mean that 'What the heck?!?' should
>>> become the last item in the list.  Again, the interpreter help gave
>>> what I was expecting.
>>
>>
>> To me it means "insert 'x' so that its index is 'i'".
>
>But that's my point!  In my example x (here 'What the heck?!?') is
>*not* at index i (here, -1).  Instead it winds up at index -2.  But
>this fits in perfectly with the interpreter help, since it winds up
>*before* index i (-1).

Ah, but -1 isn't the "real" index. It is a convenient value for computing the 
real index if you want to figure things out from the end of the list instead of 
the start.  In your example above, the real index is 5.  As you would get from 
things.index('?') before the insert. So your insert really means:

  things.insert(5, 'What the heck?!?')

Cheers,
Cameron Simpson <cs at zip.com.au>

From robertvstepp at gmail.com  Sat Jan 16 00:40:59 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Fri, 15 Jan 2016 23:40:59 -0600
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <20160116053234.GA43201@cskk.homeip.net>
References: <CANDiX9KuSR2bGfS+m4h8=zgKoumun7B1jkyR36NkMUcE2bmV9A@mail.gmail.com>
 <20160116053234.GA43201@cskk.homeip.net>
Message-ID: <CANDiX9LfZGjAivsPet65a=QHMqQ13psP8=n4PevSLcUWF72JAA@mail.gmail.com>

On Fri, Jan 15, 2016 at 11:32 PM, Cameron Simpson <cs at zip.com.au> wrote:
> On 15Jan2016 23:05, boB Stepp <robertvstepp at gmail.com> wrote:
>>
>> On Fri, Jan 15, 2016 at 10:53 PM, Cameron Simpson <cs at zip.com.au> wrote:
>>>>>>>
>>>>>>> things.insert(-1, 'What the heck?!?')
>>>>>>> things
>>>>
>>>>
>>>> [0, 'Hmm...', 3, 'WhackABunny', 6, 'What the heck?!?', '?']
>>>>
>>>> "...at the index..." to me would mean that 'What the heck?!?' should
>>>> become the last item in the list.  Again, the interpreter help gave
>>>> what I was expecting.
>>>
>>>
>>>
>>> To me it means "insert 'x' so that its index is 'i'".
>>
>>
>> But that's my point!  In my example x (here 'What the heck?!?') is
>> *not* at index i (here, -1).  Instead it winds up at index -2.  But
>> this fits in perfectly with the interpreter help, since it winds up
>> *before* index i (-1).
>
>
> Ah, but -1 isn't the "real" index. It is a convenient value for computing
> the real index if you want to figure things out from the end of the list
> instead of the start.  In your example above, the real index is 5.  As you
> would get from things.index('?') before the insert. So your insert really
> means:
>
>  things.insert(5, 'What the heck?!?')

Or, actual index = len(things) + (-1) before I do the insert.

Another subtlety for negative indexing to file away!

Thanks, Cameron. That wraps up things nicely for me.



-- 
boB

From steve at pearwood.info  Sat Jan 16 00:47:42 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 16 Jan 2016 16:47:42 +1100
Subject: [Tutor] Is there a preference of "class MyClass:" or "class
 MyClass(object):" for Py3?
In-Reply-To: <CANDiX9KyoKEswOjTh4sjBiJC-hFPhOkw9SF-907SX1dG9sQBuQ@mail.gmail.com>
References: <CANDiX9KyoKEswOjTh4sjBiJC-hFPhOkw9SF-907SX1dG9sQBuQ@mail.gmail.com>
Message-ID: <20160116054742.GN10854@ando.pearwood.info>

On Fri, Jan 15, 2016 at 09:30:57PM -0600, boB Stepp wrote:

> Pythonic style/preference question:  For strictly Python 3 code, is
> there any preference for
> 
> class MyClass:
>     pass
> 
> versus the more explicit
> 
> class MyClass(object):
>     pass
> 
> ?

For *purely* Python 3 code, where your audience (readers, maintainers, 
developers etc) are all familiar with, and expect, Python 3 semantics, 
not really. Maybe if you are writing introspection code which explicitly 
works with the class MRO or base-classes, you might prefer to be 
explicit. Otherwise, do whatever you feel best. For quick and dirty 
throw-away code, feel free to leave out the base class.

If there's any chance that the code might be used in Python 2, or copied 
into a Python 2 module, or read by people expecting Python 2 semantics, 
then you ought to be explicit about the base class.


-- 
Steve

From robertvstepp at gmail.com  Sat Jan 16 01:10:42 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 16 Jan 2016 00:10:42 -0600
Subject: [Tutor] Is there a preference of "class MyClass:" or "class
 MyClass(object):" for Py3?
In-Reply-To: <20160116054742.GN10854@ando.pearwood.info>
References: <CANDiX9KyoKEswOjTh4sjBiJC-hFPhOkw9SF-907SX1dG9sQBuQ@mail.gmail.com>
 <20160116054742.GN10854@ando.pearwood.info>
Message-ID: <CANDiX9LSr4a1bGpcmjnCZtw9ViYmmK+ii+yjkWnf04SL8HhiJw@mail.gmail.com>

On Fri, Jan 15, 2016 at 11:47 PM, Steven D'Aprano <steve at pearwood.info> wrote:

>
> If there's any chance that the code might be used in Python 2, or copied
> into a Python 2 module, or read by people expecting Python 2 semantics,
> then you ought to be explicit about the base class.

Hmm.  I *try* to keep Python 3 at home and Python 2 at work (OT aside:
 Hey!  I no longer have to worry about Python 2.4.  I'm up to Python
2.6.4 on all platforms that I do projects for at work!).  But I can
imagine learning something studying at home:  "Hey!  What I just
learned will be really useful at work!  I'll just copy this class to
my thumb drive and ..."  Perhaps for all too easily confused boB
explicit might be better than implicit?  Seems I read that somewhere
recently ...

Thanks, Steve!

-- 
boB

From steve at pearwood.info  Sat Jan 16 06:54:11 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 16 Jan 2016 22:54:11 +1100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
Message-ID: <20160116115411.GO10854@ando.pearwood.info>

On Fri, Jan 15, 2016 at 10:20:41PM -0600, boB Stepp wrote:
> At https://docs.python.org/3.4/library/stdtypes.html#sequence-types-list-tuple-range
> it states:
> 
> "s.insert(i, x) inserts x into s at the index given by i (same as s[i:i] = [x])"
> 
> I find this confusing.

That's because it is confusing, unless you get the missing picture. The 
missing picture is how indexes are treated in Python. For the following, 
you will need to read my email using a fixed-width (monospaced) font, 
like Courier, otherwise the ASCII diagrams won't make any sense.

First, understand that indexes are treated two ways by Python. When you 
just give a single index, like mylist[3], the indexes line up with the 
items:


mylist = [ 100, 200, 300, 400, 500 ]
indexes:    ^    ^    ^    ^    ^
            0    1    2    3    4

This makes perfect sense: mylist[3] is the 3rd item in the list 
(starting from zero), namely 400.

But slices are slightly different. When you provide two indexes in a 
slice, they mark the gaps BETWEEN items:

mylist = [ 100, 200, 300, 400, 500 ]
indexes:  ^    ^    ^    ^    ^   ^
          0    1    2    3    4   5

and the slice cuts in the gaps, so for example, mylist[1:3] cuts in the 
space *just before* 200 and *just before* 400, giving you [200, 300] as 
the result.

I stress that these gaps aren't "real". It's not like Python allocates a 
list, and leaves a physical chunk of memory between each item. That 
would be just silly. You should consider these gaps to be infinitely 
thin spaces beween the consecutive items.

The important thing here is that when you slice, the list is cut 
*between* items, so slice index 1 comes immediately after the 0th item 
and immediately before the 1st item.

Now, if you're paying attention, you will realise that earlier I told a 
little fib. There's no need to say that Python treats the index 
differently for regular indexing (like mylist[3]). We just need a slight 
change in understanding:


mylist = [ 100, 200, 300, 400, 500 ]
indexes:  ^    ^    ^    ^    ^   ^
          0    1    2    3    4   5


Now the rule becomes:

- if you have mylist[3], return the next item starting at index 3 
  (in this case, 400);

- but if you have a slice like mylist[1:3], return the items
  starting at the first index (1) and ending at the second (3), 
  namely [200, 300].

So mylist[1] is similar to mylist[1:2] except that the first returns the 
item itself, namely 200, while the second returns a one-element list 
containing that item, namely [200].


The picture is a bit more complicated once you introduce a third value 
in the slice, like mylist[1:3:2], but the important thing to remember is 
that indexes mark the gaps BETWEEN items, not the items themselves.

Now, what happens with *negative* indexes?

mylist = [ 100, 200, 300, 400, 500 ]
indexes:  ^    ^    ^    ^    ^   ^
          -6   -5   -4   -3   -2  -1

mylist[-5:-2] will be [200, 300, 400]. Easy.


Now, keeping in mind the rule that slices cut *between* items, you 
should be able to explain why mylist[1:1] is the empty list [].


What happens when you assign to a slice? The rule is the same, except 
instead of returning the slice as a new list, you *replace* that slice 
with the list given. So to understand

mylist[1:3] = [777, 888, 999]

we first mark the gaps between items and underline the slice being 
replaced:

mylist = [ 100, 200, 300, 400, 500 ]
indexes:  ^    ^    ^    ^    ^   ^
          0    1    2    3    4   5
               -----------

and insert the new slice in its place (moving everything to the right 
over):

mylist = [ 100, 777, 888, 999, 400, 500 ]
indexes:  ^    ^    ^    ^    ^    ^   ^
          0    1    2    3    4    5   6


Now let's look at an insertion. We're told that 

mylist.insert(1, x)

is the same as mylist[1:1] = [x]. Let's see:


mylist = [ 100, 200, 300, 400, 500 ]
indexes:  ^    ^    ^    ^    ^   ^
          0    1    2    3    4   5
               -

Note the tiny underline that extends from index 1 to index 1. Since that 
doesn't extend to the next index, no items are removed. Now insert the 
new list [x] into that slice:

mylist = [ 100, x, 200, 300, 400, 500 ]
indexes:  ^    ^  ^    ^    ^    ^   ^
          0    1  2    3    4    5   6
               
And that's an insertion! The insertion takes place at position one, 
which means that x ends up just after the gap at position one.



> The second thing I find puzzling is the docs say x is inserted at
> position i, while in the interpreter:
> 
> >>> help(list.insert)
> Help on method_descriptor:
> 
> insert(...)
>     L.insert(index, object) -- insert object before index
> 
> The "...insert object before index" makes sense to me, but "...inserts
> x into s at the index given by i..." does not because:

The beauty of thinking about indexs as the gap between items is that it 
doesn't matter whether you insert "after" the gap or "before" the gap or 
"into" the gap, you get the same thing:

mylist = [ 100, x, 200, 300, 400, 500 ]
indexes:  ^    ^  ^    ^    ^    ^   ^
          0    1  2    3    4    5   6


whereas if you consider the indexes to point to the items themselves, 
then insert "before" position 1, "after" position 1 and "into" position 
1 are three different things:

mylist = [ 100, 200, 300, 400, 500 ]
indexes:    ^    ^    ^    ^    ^
            0    1    2    3    4

Insert *before* position 1: mylist = [ 100, x, 200, 300, 400, 500 ]
Insert *after* position 1:  mylist = [ 100, 200, x, 300, 400, 500 ]
Insert *into* position 1:   mylist = [ 100, x, 300, 400, 500 ]

This ambiguity doesn't occur when you think about indexes pointing at 
the gaps between items instead of the items themselves.



-- 
Steve

From __peter__ at web.de  Sat Jan 16 07:19:16 2016
From: __peter__ at web.de (Peter Otten)
Date: Sat, 16 Jan 2016 13:19:16 +0100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info>
Message-ID: <n7dcg5$61a$1@ger.gmane.org>

Steven D'Aprano wrote:

> But slices are slightly different. When you provide two indexes in a 
> slice, they mark the gaps BETWEEN items:

The other explanation that Python uses half-open intervals works for me. 

> Now, what happens with *negative* indexes?
> 
> mylist = [ 100, 200, 300, 400, 500 ]
> indexes:  ^    ^    ^    ^    ^   ^
>           -6   -5   -4   -3   -2  -1
> 
> mylist[-5:-2] will be [200, 300, 400]. Easy.

>>> mylist = [ 100, 200, 300, 400, 500 ]
>>> mylist[-5:-2]
[100, 200, 300]

Off by one, you picked the wrong gaps.

Slightly related is a problem that comes up in practice; you cannot specify 
"including the last item" with negative indices:

>>> for i in reversed(range(len(mylist))):
...     print(mylist[:-i])
... 
[100]
[100, 200]
[100, 200, 300]
[100, 200, 300, 400]
[]

A simple fix is

>>> for i in reversed(range(len(mylist))):
...     print(mylist[:-i or None])
... 
[100]
[100, 200]
[100, 200, 300]
[100, 200, 300, 400]
[100, 200, 300, 400, 500]

The hard part is to remember to test whenever a negative index is 
calculated.


From egeberkay12 at gmail.com  Sat Jan 16 10:51:15 2016
From: egeberkay12 at gmail.com (=?UTF-8?Q?Ege_Berkay_G=C3=BClcan?=)
Date: Sat, 16 Jan 2016 17:51:15 +0200
Subject: [Tutor] What is the square brackets about?
Message-ID: <CAG1v=aE0uLxhj05FGQ+kkpnH82Mtyj6JAz0iJ=v+o9hMWW6r8A@mail.gmail.com>

def get_(loc, thing):
    if loc==[]: return thing
    return get_(loc[1:], thing[loc[0]])

Hi I am new to Python and I would like to learn about these uses of square
brackets. I know that loc[1:] means loc list without the first element but
I do not know the meanings of loc==[] and thing[loc[0]].
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Bu
e-posta Avast taraf?ndan korunan vir?ss?z bir bilgisayardan g?nderilmi?tir.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

From alan.gauld at btinternet.com  Sat Jan 16 14:00:09 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 16 Jan 2016 19:00:09 +0000
Subject: [Tutor] What is the square brackets about?
In-Reply-To: <CAG1v=aE0uLxhj05FGQ+kkpnH82Mtyj6JAz0iJ=v+o9hMWW6r8A@mail.gmail.com>
References: <CAG1v=aE0uLxhj05FGQ+kkpnH82Mtyj6JAz0iJ=v+o9hMWW6r8A@mail.gmail.com>
Message-ID: <n7e3vp$vin$1@ger.gmane.org>

On 16/01/16 15:51, Ege Berkay G?lcan wrote:
> def get_(loc, thing):
>     if loc==[]: return thing
>     return get_(loc[1:], thing[loc[0]])
> 
> Hi I am new to Python and I would like to learn about these uses of square
> brackets. I know that loc[1:] means loc list without the first element but
> I do not know the meanings of loc==[] and thing[loc[0]].

[] means an empty list.
so

if loc==[]: return thing

tests whether loc is empty.

Python treats an empty list as being 'false' so the test
could just as easily have been written:

if loc:

thing[loc[0]]

is simply applying an index to thing.

loc[0] returns the value of the first item in loc.
lets assume that is 2

then
thing[loc[0]]
would be the same as
thing[2]

As a side note, this function looks very fragile since
it depends on thing having nested data structures that
match the indexes provided by loc.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From kwpolska at gmail.com  Sat Jan 16 14:02:33 2016
From: kwpolska at gmail.com (Chris Warrick)
Date: Sat, 16 Jan 2016 20:02:33 +0100
Subject: [Tutor] What is the square brackets about?
In-Reply-To: <CAG1v=aE0uLxhj05FGQ+kkpnH82Mtyj6JAz0iJ=v+o9hMWW6r8A@mail.gmail.com>
References: <CAG1v=aE0uLxhj05FGQ+kkpnH82Mtyj6JAz0iJ=v+o9hMWW6r8A@mail.gmail.com>
Message-ID: <CAMw+j7+LNzxoSA_R393v2pq3HQy98_2mozSU2gn25thT9Q=6zA@mail.gmail.com>

On 16 January 2016 at 16:51, Ege Berkay G?lcan <egeberkay12 at gmail.com> wrote:
> def get_(loc, thing):
>     if loc==[]: return thing
>     return get_(loc[1:], thing[loc[0]])
>
> Hi I am new to Python and I would like to learn about these uses of square
> brackets. I know that loc[1:] means loc list without the first element but
> I do not know the meanings of loc==[] and thing[loc[0]].

loc == [] checks ?if `loc` is equal to an empty list?. Note that this
is not a good way to do this. A much better way to spell this would
be:

    if not loc:
        return thing

thing[loc[0]] means ?check what the 0th element of `loc` (`loc[0]`)
is, and use it as an index for `thing` (`thing[?]`).

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16

From robertvstepp at gmail.com  Sat Jan 16 14:14:54 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 16 Jan 2016 13:14:54 -0600
Subject: [Tutor] What is the square brackets about?
In-Reply-To: <n7e3vp$vin$1@ger.gmane.org>
References: <CAG1v=aE0uLxhj05FGQ+kkpnH82Mtyj6JAz0iJ=v+o9hMWW6r8A@mail.gmail.com>
 <n7e3vp$vin$1@ger.gmane.org>
Message-ID: <CANDiX9J3M6CGRH9bqA_4bvpWVJReuNzwH6D+qRzYqBWDxdGcLg@mail.gmail.com>

On Sat, Jan 16, 2016 at 1:00 PM, Alan Gauld <alan.gauld at btinternet.com> wrote:

> As a side note, this function looks very fragile since
> it depends on thing having nested data structures that
> match the indexes provided by loc.

As Alan's response arrived, I was in the interpreter trying out this
function with a set of values which would enable it to work.  Perhaps
it will illustrate one thing it could do.  It is hard for me to
determine the original intent of the function as the names do not
provide much context.  Perhaps "loc" is short for "location" within
the "thing"?

>>> def get_(loc, thing):
if loc == []: return thing
return get_(loc[1:], thing[loc[0]])

>>> loc = [2]
>>> thing = ['cat', 'dog', 'mouse', 'tick']
>>> get_(loc, thing)
'mouse'

Notice that "thing" appears to work for any "thing" that can be
indexed.  For instance, if I now make "thing" a tuple, the function
will still behave:

>>> thing = ('cat', 'dog', 'mouse', 'tick')
>>> get_(loc, thing)
'mouse'

While learning I find it very helpful to either use IDLE or invoke the
Python interpreter in the shell and try these things out.  Once I get
it to work, then I play around with the syntax and deliberately try to
break things and see what sorts of errors are generated, figure out
the limits of what the syntax will allow, etc., until I feel I am
starting to understand what the original code does.

HTH,

-- 
boB

From robertvstepp at gmail.com  Sat Jan 16 14:35:12 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 16 Jan 2016 13:35:12 -0600
Subject: [Tutor] What is the square brackets about?
In-Reply-To: <CANDiX9J3M6CGRH9bqA_4bvpWVJReuNzwH6D+qRzYqBWDxdGcLg@mail.gmail.com>
References: <CAG1v=aE0uLxhj05FGQ+kkpnH82Mtyj6JAz0iJ=v+o9hMWW6r8A@mail.gmail.com>
 <n7e3vp$vin$1@ger.gmane.org>
 <CANDiX9J3M6CGRH9bqA_4bvpWVJReuNzwH6D+qRzYqBWDxdGcLg@mail.gmail.com>
Message-ID: <CANDiX9+dtbD26ocRYaU15UE1u+fb2EgCS-QYKjn6b4zy2DQiuA@mail.gmail.com>

On Sat, Jan 16, 2016 at 1:14 PM, boB Stepp <robertvstepp at gmail.com> wrote:

>
> While learning I find it very helpful to either use IDLE or invoke the
> Python interpreter in the shell and try these things out.  Once I get
> it to work, then I play around with the syntax and deliberately try to
> break things and see what sorts of errors are generated, figure out
> the limits of what the syntax will allow, etc., until I feel I am
> starting to understand what the original code does.

Continuing to play around with the code:

>>> loc = [0, 2, 8]
>>> get_(loc, thing)
Traceback (most recent call last):
  File "<pyshell#58>", line 1, in <module>
    get_(loc, thing)
  File "<pyshell#51>", line 3, in get_
    return get_(loc[1:], thing[loc[0]])
  File "<pyshell#51>", line 3, in get_
    return get_(loc[1:], thing[loc[0]])
  File "<pyshell#51>", line 3, in get_
    return get_(loc[1:], thing[loc[0]])
IndexError: string index out of range

and,

>>> loc = [0, 1]
>>> get_(loc, thing)
'a'

And so on.  Until you (and I) can understand why the function produces
these outputs with the given values of loc and thing, then we cannot
claim we understand what is going on.  So I encourage you to
thoroughly explore your sample code!



-- 
boB

From alan.gauld at btinternet.com  Sat Jan 16 15:33:12 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 16 Jan 2016 20:33:12 +0000
Subject: [Tutor] What is the square brackets about?
In-Reply-To: <CANDiX9+dtbD26ocRYaU15UE1u+fb2EgCS-QYKjn6b4zy2DQiuA@mail.gmail.com>
References: <CAG1v=aE0uLxhj05FGQ+kkpnH82Mtyj6JAz0iJ=v+o9hMWW6r8A@mail.gmail.com>
 <n7e3vp$vin$1@ger.gmane.org>
 <CANDiX9J3M6CGRH9bqA_4bvpWVJReuNzwH6D+qRzYqBWDxdGcLg@mail.gmail.com>
 <CANDiX9+dtbD26ocRYaU15UE1u+fb2EgCS-QYKjn6b4zy2DQiuA@mail.gmail.com>
Message-ID: <n7e9e9$tob$1@ger.gmane.org>

On 16/01/16 19:35, boB Stepp wrote:

> And so on.  Until you (and I) can understand why the function produces
> these outputs with the given values of loc and thing, then we cannot
> claim we understand what is going on.  So I encourage you to
> thoroughly explore your sample code!

The function plumbs the depths of thing according to the indices
supplies in loc. The shape of thing must match the length of loc.

So if loc = [1,2,3]

thing must be a sequence of sequences of sequences
where the final sequence os at least 4 items long.

That's why I said it was a very fragile function.
Any mismatch in the data is likely to give an
IndexError.

I'd strongly recommend wrapping the second line
in a try/except - at least to aid debugging.

def get_(loc, thing):
    if loc==[]: return thing
    try: return get_(loc[1:], thing[loc[0]])
    except IndexError:
         print "cannot access index", loc[0]," of", thing
         raise

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From robertvstepp at gmail.com  Sat Jan 16 17:39:09 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 16 Jan 2016 16:39:09 -0600
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <n7dcg5$61a$1@ger.gmane.org>
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info>
 <n7dcg5$61a$1@ger.gmane.org>
Message-ID: <CANDiX9Kfg5u-uRbarPZ07Vi0rv4XZc_pDZxq5e_PervOEok=DA@mail.gmail.com>

On Sat, Jan 16, 2016 at 6:19 AM, Peter Otten <__peter__ at web.de> wrote:
> Steven D'Aprano wrote:
>
>> But slices are slightly different. When you provide two indexes in a
>> slice, they mark the gaps BETWEEN items:
>
> The other explanation that Python uses half-open intervals works for me.
>
>> Now, what happens with *negative* indexes?
>>
>> mylist = [ 100, 200, 300, 400, 500 ]
>> indexes:  ^    ^    ^    ^    ^   ^
>>           -6   -5   -4   -3   -2  -1

So in this model of understanding negative list indexing, should it be:

mylist = [ 100, 200, 300, 400, 500 ]
          ^    ^    ^    ^    ^   ^
          -5   -4   -3   -2   -1  ?

Well, it has to be this; otherwise, the off-by-one error exist.  This
also continues to explain why

mylist.insert(-1, x)

inserts x *before* 500.  But in this model, what should go in the place of "?"?


> Slightly related is a problem that comes up in practice; you cannot specify
> "including the last item" with negative indices:

[...]

> A simple fix is
>
>>>> for i in reversed(range(len(mylist))):
> ...     print(mylist[:-i or None])
> ...
> [100]
> [100, 200]
> [100, 200, 300]
> [100, 200, 300, 400]
> [100, 200, 300, 400, 500]

OK, Peter, all was going smoothly in boB-land until you added your
"fix".  Adding "or None" has me addled!  I tried to clarify things in
the interpreter (I removed "reversed" so that I could deal only with
what I was finding confusing.):

>>> for i in range(len(mylist)):
        print(mylist[:-i])

[]
[100, 200, 300, 400]
[100, 200, 300]
[100, 200]
[100]

Then adding the "or None":

>>> for i in range(len(mylist)):
        print(mylist[:-i or None])

[100, 200, 300, 400, 500]
[100, 200, 300, 400]
[100, 200, 300]
[100, 200]
[100]

So far I've duplicated what you did without the reversed built-in.  So
I tried playing around:

>>> mylist[:0]
[]

This was expected as this is equivalent to mylist[0:0].

>>> mylist[:0 or None]
[100, 200, 300, 400, 500]

The critical portion of the for loop for me to understand, since it
results in [100, 200, 300, 400, 500] instead of the empty list.  But
what the heck is going on here?

>>> mylist[0 or None]
Traceback (most recent call last):
  File "<pyshell#117>", line 1, in <module>
    mylist[0 or None]
TypeError: list indices must be integers, not NoneType

And I am stuck.  I can't figure out why [:0 or None] is legal and what
it is actually doing, while [0 or None] is (a rather obvious)
TypeError.  Please illuminate my darkness!

> The hard part is to remember to test whenever a negative index is
> calculated.

I am assuming that this is relevant to what just came before, the use
of this "or None" check.  Is this correct?


-- 
boB

From robertvstepp at gmail.com  Sat Jan 16 17:53:37 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 16 Jan 2016 16:53:37 -0600
Subject: [Tutor] What is the square brackets about?
In-Reply-To: <n7e9e9$tob$1@ger.gmane.org>
References: <CAG1v=aE0uLxhj05FGQ+kkpnH82Mtyj6JAz0iJ=v+o9hMWW6r8A@mail.gmail.com>
 <n7e3vp$vin$1@ger.gmane.org>
 <CANDiX9J3M6CGRH9bqA_4bvpWVJReuNzwH6D+qRzYqBWDxdGcLg@mail.gmail.com>
 <CANDiX9+dtbD26ocRYaU15UE1u+fb2EgCS-QYKjn6b4zy2DQiuA@mail.gmail.com>
 <n7e9e9$tob$1@ger.gmane.org>
Message-ID: <CANDiX9+6k2dEbsC3hXguPZMXmVaVbSYrfUaer9MmU+0gNwgyVg@mail.gmail.com>

On Sat, Jan 16, 2016 at 2:33 PM, Alan Gauld <alan.gauld at btinternet.com> wrote:
> On 16/01/16 19:35, boB Stepp wrote:
>
>> And so on.  Until you (and I) can understand why the function produces
>> these outputs with the given values of loc and thing, then we cannot
>> claim we understand what is going on.  So I encourage you to
>> thoroughly explore your sample code!
>
> The function plumbs the depths of thing according to the indices
> supplies in loc. The shape of thing must match the length of loc.

The interesting part of this function for me was not the OP's original
questions, but why this particular use of a recursive function (Am I
using the proper terminology here?)?  Can someone provide a practical
use of this type of function where some more straightforward
searching/parsing approach would be ineffective?

-- 
boB

From adeadmarshal at gmail.com  Sat Jan 16 16:18:53 2016
From: adeadmarshal at gmail.com (Ali Moradi)
Date: Sun, 17 Jan 2016 00:48:53 +0330
Subject: [Tutor] Console application
Message-ID: <CAMh2k3a-EXQf7C=iBvraBMFDKbX1U_aBMeZJnhYjAi4ax7xPxw@mail.gmail.com>

Hi, i don't have any clue how to write a console program that shows a list
of options which could be chosen with keyboard and when one item was
selected, a text shows there.

I want to know, which things i need to write a program like that?

Input() , print, and .... What? :(

From lordrip at gmail.com  Sat Jan 16 18:27:00 2016
From: lordrip at gmail.com (=?UTF-8?Q?Ricardo_Mart=C3=ADnez?=)
Date: Sun, 17 Jan 2016 00:27:00 +0100
Subject: [Tutor] Source of MySQL Command Interpreter
Message-ID: <CAMtNk+tw=QQxL-xiPVZ+rNV_Wn-hZTej-mHN-YUUvtRVrKY_4w@mail.gmail.com>

Hi, i wrote a small APP to execute MySQL commands and retrieve to a Treeview

http://pastebin.com/v2C8kAu1

Share your comments and upgrades.

From alan.gauld at btinternet.com  Sat Jan 16 18:56:43 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 16 Jan 2016 23:56:43 +0000
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <CANDiX9Kfg5u-uRbarPZ07Vi0rv4XZc_pDZxq5e_PervOEok=DA@mail.gmail.com>
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info> <n7dcg5$61a$1@ger.gmane.org>
 <CANDiX9Kfg5u-uRbarPZ07Vi0rv4XZc_pDZxq5e_PervOEok=DA@mail.gmail.com>
Message-ID: <n7elbr$53n$1@ger.gmane.org>

On 16/01/16 22:39, boB Stepp wrote:

> So in this model of understanding negative list indexing, should it be:
> 
> mylist = [ 100, 200, 300, 400, 500 ]
>           ^    ^    ^    ^    ^   ^
>           -5   -4   -3   -2   -1  ?
> 
> Well, it has to be this; otherwise, the off-by-one error exist.  This
> also continues to explain why
> 
> mylist.insert(-1, x)
> 
> inserts x *before* 500.  But in this model, what should go in the place of "?"?

-0

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From ben+python at benfinney.id.au  Sat Jan 16 18:58:15 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 17 Jan 2016 10:58:15 +1100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info>
 <n7dcg5$61a$1@ger.gmane.org>
 <CANDiX9Kfg5u-uRbarPZ07Vi0rv4XZc_pDZxq5e_PervOEok=DA@mail.gmail.com>
Message-ID: <85y4bpnkko.fsf@benfinney.id.au>

boB Stepp <robertvstepp at gmail.com> writes:

> So in this model of understanding negative list indexing, should it be:
>
> mylist = [ 100, 200, 300, 400, 500 ]
>           ^    ^    ^    ^    ^   ^
>           -5   -4   -3   -2   -1  ?

For completeness, let's use the rest of the integers also::

              0    1    2    3    4    5
              ?    ?    ?    ?    ?    ?
    mylist = [ 100, 200, 300, 400, 500  ]
              ?    ?    ?    ?    ?    ?
             ?5   ?4   ?3   ?2   ?1    ?

> But in this model, what should go in the place of "?"?

You can use ?len(mylist)? for the index at the end of the sequence.

There isn't a negative number which will address that position; it isn't
needed, because there is already one obvious way :-)

-- 
 \            ?Without cultural sanction, most or all of our religious |
  `\          beliefs and rituals would fall into the domain of mental |
_o__)                                 disturbance.? ?John F. Schumaker |
Ben Finney


From ben+python at benfinney.id.au  Sat Jan 16 19:04:00 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 17 Jan 2016 11:04:00 +1100
Subject: [Tutor] Source of MySQL Command Interpreter
References: <CAMtNk+tw=QQxL-xiPVZ+rNV_Wn-hZTej-mHN-YUUvtRVrKY_4w@mail.gmail.com>
Message-ID: <85pox1nkb3.fsf@benfinney.id.au>

Ricardo Mart?nez <lordrip at gmail.com> writes:

> Hi, i wrote a small APP to execute MySQL commands and retrieve to a Treeview
>
> Share your comments and upgrades.

You're addressing this to the wrong forum. If you want to discuss code,
please post *small, complete* samples of code directly here in the forum
and ask specific questions.

Large code bases are not appropriate here; we are better geared to
discussing specific problems and concepts.

Links to code elsewhere are not appropriate here; this forum is geared
to discussing the code in context with the messages.

So please keep code examples small, and put them in your message for
discussion.

-- 
 \         ?I call him Governor Bush because that's the only political |
  `\              office he's ever held legally.? ?George Carlin, 2008 |
_o__)                                                                  |
Ben Finney


From alan.gauld at btinternet.com  Sat Jan 16 19:04:23 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 17 Jan 2016 00:04:23 +0000
Subject: [Tutor] What is the square brackets about?
In-Reply-To: <CANDiX9+6k2dEbsC3hXguPZMXmVaVbSYrfUaer9MmU+0gNwgyVg@mail.gmail.com>
References: <CAG1v=aE0uLxhj05FGQ+kkpnH82Mtyj6JAz0iJ=v+o9hMWW6r8A@mail.gmail.com>
 <n7e3vp$vin$1@ger.gmane.org>
 <CANDiX9J3M6CGRH9bqA_4bvpWVJReuNzwH6D+qRzYqBWDxdGcLg@mail.gmail.com>
 <CANDiX9+dtbD26ocRYaU15UE1u+fb2EgCS-QYKjn6b4zy2DQiuA@mail.gmail.com>
 <n7e9e9$tob$1@ger.gmane.org>
 <CANDiX9+6k2dEbsC3hXguPZMXmVaVbSYrfUaer9MmU+0gNwgyVg@mail.gmail.com>
Message-ID: <n7elq7$dl5$1@ger.gmane.org>

On 16/01/16 22:53, boB Stepp wrote:

>> The function plumbs the depths of thing according to the indices
>> supplies in loc. The shape of thing must match the length of loc.
> 
> The interesting part of this function for me was not the OP's original
> questions, but why this particular use of a recursive function (Am I
> using the proper terminology here?)?  

Yes it's a recursive function.


> Can someone provide a practical
> use of this type of function where some more straightforward
> searching/parsing approach would be ineffective?

In fact its much easier to write this using recursion that
any other method. Its just very difficult to use and so
needs some protection. The difficulty is in getting the
input data structures to match, once that's done the
function itself is very straightforward.

It's a way of quickly navigating a tree structure where the
route to the node (or subtree) of interest is given in
advance. Think of 'loc' as being a directory path broken
into its individual parts and 'thing' as being the file
system represented as a dictionary of dictionaries (for
the named indexing to work). [BTW I haven't tried that to
see if it works it's just a guess at a possible
application...] Of course, for a real filesystem we have
functions in the os module to do the work but that type
of data structure might be the target.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From ben+python at benfinney.id.au  Sat Jan 16 19:01:47 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 17 Jan 2016 11:01:47 +1100
Subject: [Tutor] Console application
References: <CAMh2k3a-EXQf7C=iBvraBMFDKbX1U_aBMeZJnhYjAi4ax7xPxw@mail.gmail.com>
Message-ID: <85twmdnkes.fsf@benfinney.id.au>

Ali Moradi <adeadmarshal at gmail.com> writes:

> Hi, i don't have any clue how to write a console program that shows a
> list of options which could be chosen with keyboard and when one item
> was selected, a text shows there.

A text shows where? Shows at the point of input, at the point of the
option, at some other point?

> I want to know, which things i need to write a program like that?

I think you might want to address specific points in the grid of
characters on the terminal. That's specific to each operating system, so
you'll need an API which knows how to tell the terminal what you mean.

The Python standard library includes such an API in the ?curses? library
<URL:https://docs.python.org/3/library/curses.html>. Try using that and
see whether it meets your needs.

-- 
 \        ?Telling pious lies to trusting children is a form of abuse, |
  `\                    plain and simple.? ?Daniel Dennett, 2010-01-12 |
_o__)                                                                  |
Ben Finney


From alan.gauld at btinternet.com  Sat Jan 16 19:08:04 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 17 Jan 2016 00:08:04 +0000
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <n7elbr$53n$1@ger.gmane.org>
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info> <n7dcg5$61a$1@ger.gmane.org>
 <CANDiX9Kfg5u-uRbarPZ07Vi0rv4XZc_pDZxq5e_PervOEok=DA@mail.gmail.com>
 <n7elbr$53n$1@ger.gmane.org>
Message-ID: <n7em14$dl5$2@ger.gmane.org>

On 16/01/16 23:56, Alan Gauld wrote:
> On 16/01/16 22:39, boB Stepp wrote:
> 
>> So in this model of understanding negative list indexing, should it be:
>>
>> mylist = [ 100, 200, 300, 400, 500 ]
>>           ^    ^    ^    ^    ^   ^
>>           -5   -4   -3   -2   -1  ?
>>
>> Well, it has to be this; otherwise, the off-by-one error exist.  This
>> also continues to explain why
>>
>> mylist.insert(-1, x)
>>
>> inserts x *before* 500.  But in this model, what should go in the place of "?"?
> 
> -0
> 

I should have added a :-/ to that in case it wasn't obvious...

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Sat Jan 16 19:17:17 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 17 Jan 2016 00:17:17 +0000
Subject: [Tutor] Console application
In-Reply-To: <CAMh2k3a-EXQf7C=iBvraBMFDKbX1U_aBMeZJnhYjAi4ax7xPxw@mail.gmail.com>
References: <CAMh2k3a-EXQf7C=iBvraBMFDKbX1U_aBMeZJnhYjAi4ax7xPxw@mail.gmail.com>
Message-ID: <n7emid$q2e$1@ger.gmane.org>

On 16/01/16 21:18, Ali Moradi wrote:
> Hi, i don't have any clue how to write a console program that shows a list
> of options which could be chosen with keyboard and when one item was
> selected, a text shows there.
> 
> I want to know, which things i need to write a program like that?
> 
> Input() , print, and .... What? :(


There are lots of ways to do this. The simplest is simply to
display a menu and ask the user to select an entry.

print('''
1 New
2 Edit
3 Delete
4 quit
''')
cmd = int(input('Enter an option number')

if cmd == 1:....
elif cmd == 2:....
etc

If the above explanation is still too complicated
then please come back with more questions

You can add loops and error traps if you wish.
You can put the menu inside a function. But I don't
know how much of that extra stuff you understand yet.

Another, more advanced/professional, option, which builds
applications that look like Python's help() function, is
to use the cmd module. There are examples in the documentation

Finally, if you are on *nix or MacOS you can use curses
to build a pseudo-GUI that works in a terminal. But
that is almost certainly too much for you at this stage.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Sat Jan 16 19:23:17 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 17 Jan 2016 00:23:17 +0000
Subject: [Tutor] Source of MySQL Command Interpreter
In-Reply-To: <85pox1nkb3.fsf@benfinney.id.au>
References: <CAMtNk+tw=QQxL-xiPVZ+rNV_Wn-hZTej-mHN-YUUvtRVrKY_4w@mail.gmail.com>
 <85pox1nkb3.fsf@benfinney.id.au>
Message-ID: <n7emtl$vqo$1@ger.gmane.org>

On 17/01/16 00:04, Ben Finney wrote:
> Ricardo Mart?nez <lordrip at gmail.com> writes:
> 
>> Hi, i wrote a small APP to execute MySQL commands and retrieve to a Treeview
>>
>> Share your comments and upgrades.
> 
> You're addressing this to the wrong forum. If you want to discuss code,
> please post *small, complete* samples of code directly here in the forum
> and ask specific questions.

To be fair to Ricardo, he did ask permission(15th Jan)
and I said it would be fine and, if the code was more
that 200 lines he could put it on a pastebin. It's 178
lines so he is well within limits.

And there are quite a few precedents for folks asking
for a critique of their code on the tutor list. It's
all part of the learning process - provided its not
too big.

Whether people take the time to read/comment is, of
course, up to them.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From akleider at sonic.net  Sat Jan 16 19:23:36 2016
From: akleider at sonic.net (Alex Kleider)
Date: Sat, 16 Jan 2016 16:23:36 -0800
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <n7em14$dl5$2@ger.gmane.org>
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info> <n7dcg5$61a$1@ger.gmane.org>
 <CANDiX9Kfg5u-uRbarPZ07Vi0rv4XZc_pDZxq5e_PervOEok=DA@mail.gmail.com>
 <n7elbr$53n$1@ger.gmane.org> <n7em14$dl5$2@ger.gmane.org>
Message-ID: <395cac4a143e9fbcd6913da704d025af@sonic.net>

On 2016-01-16 16:08, Alan Gauld wrote:
> On 16/01/16 23:56, Alan Gauld wrote:
>> On 16/01/16 22:39, boB Stepp wrote:
>> 
>>> So in this model of understanding negative list indexing, should it 
>>> be:
>>> 
>>> mylist = [ 100, 200, 300, 400, 500 ]
>>>           ^    ^    ^    ^    ^   ^
>>>           -5   -4   -3   -2   -1  ?
>>> 
>>> Well, it has to be this; otherwise, the off-by-one error exist.  This
>>> also continues to explain why
>>> 
>>> mylist.insert(-1, x)
>>> 
>>> inserts x *before* 500.  But in this model, what should go in the 
>>> place of "?"?
>> 
>> -0
>> 

alex at x301:~$ python3
Python 3.4.3 (default, Oct 14 2015, 20:33:09)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> mylist = [1, 2, 3, 4, 5]
>>> mylist[0:None]
[1, 2, 3, 4, 5]
>>> mylist[0:-0]
[]
>>> -0
0
>>> 

It appears that None provides a surrogate for -0 which itself evaluates 
to 0.


> 
> I should have added a :-/ to that in case it wasn't obvious...

It wasn't to me; could you please explain what you mean by ":-/" and/or 
where you should have added it?



From robertvstepp at gmail.com  Sat Jan 16 19:43:57 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 16 Jan 2016 18:43:57 -0600
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <d1fe5ab5accdbce6a40c18d97b0a1926@sonic.net>
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info>
 <n7dcg5$61a$1@ger.gmane.org>
 <CANDiX9Kfg5u-uRbarPZ07Vi0rv4XZc_pDZxq5e_PervOEok=DA@mail.gmail.com>
 <d1fe5ab5accdbce6a40c18d97b0a1926@sonic.net>
Message-ID: <CANDiX9JMbH1D-vG8YMN1xtMLWnTtcbdoyF=m53tKDyhKTwPHdg@mail.gmail.com>

Alex sent me this off-list.  I hope he does not mind me sharing part
of what he wrote on-list!

On Sat, Jan 16, 2016 at 4:57 PM, Alex Kleider <akleider at sonic.net> wrote:
> On 2016-01-16 14:39, boB Stepp wrote:
>
>
>>>>> mylist[:0 or None]
>>
>> [100, 200, 300, 400, 500]
>>
>> The critical portion of the for loop for me to understand, since it
>> results in [100, 200, 300, 400, 500] instead of the empty list.  But
>> what the heck is going on here?

[...]

> I guess when used in slices, None represents '-0'; clever really.
> A way to distinguish -0 from 0 which in simple arithmetic are one and the
> same.

I have no clue whether Alex's hypothesis is what actually goes on in
the implementation details, but things sure act like he is correct.
This led me to try:

>>> mylist[:None]
[100, 200, 300, 400, 500]

So, in effect, None is acting as a place holder for that final
position in slices.  Also, I would never have thought to be able to
use a logical "or" inside an index in Peter's "[:-i or None]".


-- 
boB

From cs at zip.com.au  Sat Jan 16 21:02:32 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 17 Jan 2016 13:02:32 +1100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <CANDiX9JMbH1D-vG8YMN1xtMLWnTtcbdoyF=m53tKDyhKTwPHdg@mail.gmail.com>
References: <CANDiX9JMbH1D-vG8YMN1xtMLWnTtcbdoyF=m53tKDyhKTwPHdg@mail.gmail.com>
Message-ID: <20160117020232.GA39293@cskk.homeip.net>

On 16Jan2016 18:43, boB Stepp <robertvstepp at gmail.com> wrote:
>This led me to try:
>
>>>> mylist[:None]
>[100, 200, 300, 400, 500]
>
>So, in effect, None is acting as a place holder for that final
>position in slices.  Also, I would never have thought to be able to
>use a logical "or" inside an index in Peter's "[:-i or None]".

Yah, like the default value for many missing parameters. When you don't need an 
expression after the ":" you can of course write:

  mylist[:]

much like writing a function "def f(x, y=None)"; None is a sentinel value - 
specially recognised as nor in the normal domain for that value.

Cheers,
Cameron Simpson <cs at zip.com.au>

Q: How does a hacker fix a function which doesn't work for all of the elements 
in its domain?
A: He changes the domain.
- Rich Wareham <rjw57 at hermes.cam.ac.uk>


From akleider at sonic.net  Sun Jan 17 01:42:01 2016
From: akleider at sonic.net (Alex Kleider)
Date: Sat, 16 Jan 2016 22:42:01 -0800
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <20160117020232.GA39293@cskk.homeip.net>
References: <CANDiX9JMbH1D-vG8YMN1xtMLWnTtcbdoyF=m53tKDyhKTwPHdg@mail.gmail.com>
 <20160117020232.GA39293@cskk.homeip.net>
Message-ID: <deac8c7745fce587e3c1e8ae6241d8f3@sonic.net>

On 2016-01-16 18:02, Cameron Simpson wrote:
> On 16Jan2016 18:43, boB Stepp <robertvstepp at gmail.com> wrote:
>> This led me to try:
>> 
>>>>> mylist[:None]
>> [100, 200, 300, 400, 500]
>> 
>> So, in effect, None is acting as a place holder for that final
>> position in slices.  Also, I would never have thought to be able to
>> use a logical "or" inside an index in Peter's "[:-i or None]".
> 
> Yah, like the default value for many missing parameters. When you
> don't need an expression after the ":" you can of course write:
> 
>  mylist[:]
> 
> much like writing a function "def f(x, y=None)"; None is a sentinel
> value - specially recognised as nor in the normal domain for that
> value.


Can you please clarify the last bit:
"specially recognised as nor in the normal domain for that value."

thanks,
Alex

From steve at pearwood.info  Sun Jan 17 02:38:42 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 17 Jan 2016 18:38:42 +1100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <n7dcg5$61a$1@ger.gmane.org>
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info> <n7dcg5$61a$1@ger.gmane.org>
Message-ID: <20160117073842.GP10854@ando.pearwood.info>

On Sat, Jan 16, 2016 at 01:19:16PM +0100, Peter Otten wrote:
> Steven D'Aprano wrote:
> 
> > But slices are slightly different. When you provide two indexes in a 
> > slice, they mark the gaps BETWEEN items:
> 
> The other explanation that Python uses half-open intervals works for me. 

Half-open at the start or the end? :-)

You're right, "half-open intervals" (open at the end) is also a good way 
of thinking about it. It certainly helps with things like range(), which 
don't involve slicing/cutting. You just memorize the the start value is 
included and the end value is not.

If you think of indexes falling between elements, and slicing along 
those gaps, then the natural consequence is a half-open interval. Take 
mylist[1:6] for example:


          0 1 2 3 4 5 6 7 8
mylist = [ a b c d e f g h ]
            |         |
        cut here    and here


By cutting *between* a and b, and f and g, you naturally get a half open 
interval: mylist[1] is included, but mylist[6] is not.

But if you think of indexes being aligned with the items themselves, you 
need to memorize a special rule "do I include the item or not?":


           0 1 2 3 4 5 6 7
mylist = [ a b c d e f g h ]
             |         |
          include   exclude
           this      this



> > Now, what happens with *negative* indexes?
> > 
> > mylist = [ 100, 200, 300, 400, 500 ]
> > indexes:  ^    ^    ^    ^    ^   ^
> >           -6   -5   -4   -3   -2  -1
> > 
> > mylist[-5:-2] will be [200, 300, 400]. Easy.
> 
> >>> mylist = [ 100, 200, 300, 400, 500 ]
> >>> mylist[-5:-2]
> [100, 200, 300]
> 
> Off by one, you picked the wrong gaps.

Oops, so I did. You're absolutely right.

mylist = [ 100, 200, 300, 400, 500 ]
indexes:  ^    ^    ^    ^    ^   ^
          -5   -4   -3   -2  -1   

The last index ought to be "-0" in some sense, but of course that's just 
zero which is the first index. So as you say:


> Slightly related is a problem that comes up in practice; you cannot specify 
> "including the last item" with negative indices:

But you can do so by leaving the end index blank:

py> mylist = [ 100, 200, 300, 400, 500 ]
py> mylist[-2:-1]
[400]
py> mylist[-2:0]
[]
py> mylist[-2:]
[400, 500]



-- 
Steve

From steve at pearwood.info  Sun Jan 17 03:33:38 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 17 Jan 2016 19:33:38 +1100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <CANDiX9Kfg5u-uRbarPZ07Vi0rv4XZc_pDZxq5e_PervOEok=DA@mail.gmail.com>
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info> <n7dcg5$61a$1@ger.gmane.org>
 <CANDiX9Kfg5u-uRbarPZ07Vi0rv4XZc_pDZxq5e_PervOEok=DA@mail.gmail.com>
Message-ID: <20160117083338.GQ10854@ando.pearwood.info>

On Sat, Jan 16, 2016 at 04:39:09PM -0600, boB Stepp wrote:

> So in this model of understanding negative list indexing, should it be:
> 
> mylist = [ 100, 200, 300, 400, 500 ]
>           ^    ^    ^    ^    ^   ^
>           -5   -4   -3   -2   -1  ?

Correct.

> Well, it has to be this; otherwise, the off-by-one error exist.  This
> also continues to explain why
> 
> mylist.insert(-1, x)
> 
> inserts x *before* 500.  But in this model, what should go in the place of "?"?

Slice syntax looks like this:

obj[start:end]  # two argument version
obj[start:end:step]  # three argument version

All three start, end, step are optional and have useful default values. 

start defaults to "the beginning of the object", or 0.

end defaults to "the end of the object", or len(obj).

step defaults to 1.

There's no negative value which you can give as the end argument to 
represent the very end of the list (or string, tuple, etc) but you can 
leave it blank, use None, the length of the object, or some enormously 
huge number you know is bigger than the list:

py> mylist[-2:sys.maxsize]
[400, 500]


> >>>> for i in reversed(range(len(mylist))):
> > ...     print(mylist[:-i or None])
> > ...
> > [100]
> > [100, 200]
> > [100, 200, 300]
> > [100, 200, 300, 400]
> > [100, 200, 300, 400, 500]
> 
> OK, Peter, all was going smoothly in boB-land until you added your
> "fix".  Adding "or None" has me addled!

-2 or None returns -2
-1 or None returns -1
0 or None returns None

What's going on here?

`or` returns the first value if it is "truthy", otherwise the second 
value. All integers *except zero* are truthy, but zero is like 
false, and will trigger `or` to return the second value, in 
this case None. So we have:

mylist[start:end or None]

is short-hand for:

if end:
    # end could be 1, -1, 2, -2, 3, -3 etc.
    mylist[start:end]
else:
    # end could be 0
    mylist[start:None]


> >>> mylist[0 or None]
> Traceback (most recent call last):
>   File "<pyshell#117>", line 1, in <module>
>     mylist[0 or None]
> TypeError: list indices must be integers, not NoneType


You can't use None as a lone index, only in a slice.




-- 
Steve

From alan.gauld at btinternet.com  Sun Jan 17 03:44:04 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 17 Jan 2016 08:44:04 +0000
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <395cac4a143e9fbcd6913da704d025af@sonic.net>
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info> <n7dcg5$61a$1@ger.gmane.org>
 <CANDiX9Kfg5u-uRbarPZ07Vi0rv4XZc_pDZxq5e_PervOEok=DA@mail.gmail.com>
 <n7elbr$53n$1@ger.gmane.org> <n7em14$dl5$2@ger.gmane.org>
 <395cac4a143e9fbcd6913da704d025af@sonic.net>
Message-ID: <n7fk8k$lu9$1@ger.gmane.org>

On 17/01/16 00:23, Alex Kleider wrote:

>> I should have added a :-/ to that in case it wasn't obvious...
> 
> It wasn't to me; could you please explain what you mean by ":-/" and/or 
> where you should have added it?

It's an emoticon
I usually use it as being tongue in cheek, but I see that
wikipedia suggests :-j for that...

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From cs at zip.com.au  Sun Jan 17 05:18:22 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 17 Jan 2016 21:18:22 +1100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <deac8c7745fce587e3c1e8ae6241d8f3@sonic.net>
References: <deac8c7745fce587e3c1e8ae6241d8f3@sonic.net>
Message-ID: <20160117101822.GA47709@cskk.homeip.net>

On 16Jan2016 22:42, Alex Kleider <akleider at sonic.net> wrote:
>On 2016-01-16 18:02, Cameron Simpson wrote:
>>much like writing a function "def f(x, y=None)"; None is a sentinel
>>value - specially recognised as nor in the normal domain for that
>>value.
>
>Can you please clarify the last bit:
>"specially recognised as nor in the normal domain for that value."

s/nor/not/

The domain of a function (or value) is the set of valid values it may take. The 
range is the set of values it may produce.

A sentinel value is a special value you may encounter in a data set (or as a 
value) which is _not_ part of the normal domain; often it indicates the end of 
a sequence (hence the name, like a border guard).  In Python the special value 
None is often used as a sentinel value.  Since all names have _some_ value, if 
you need to indicate that this name "isn't set" or "doesn't specify a valid 
value", you need a sentinal value, often "None".

So the common idiom for default values in Python function definitions:

  def func(a, b, c=None):
    if c is None:
      c = default-value-for-c
    ...

indicates that the parameters "a" and "b must be supplied, and "c" is optional.  
If not supplied it will have the value "None". This is a sentinel value so that 
"func" can distinguish a valid value from a value not supplied.

You can also use them to indicate the end of data in some sense. If you're 
looking at a text file as lines of printable characters, the newline character 
at the end of the line could be considered a sentinel. Also consider a Queue: 
there's no notion of "closed" in the stdlib version, so one might put a None on 
the queue to indicate to the consumer that there will be no more items.

It isn't always None. Sometimes you may want to pass None as normal data, or 
you have no control over what is passed around. In that case you might need to 
make a unique sentinel value entirely for your object. The normal way to do 
this is simply to make a new object:

  sentinel = object()

This is a minimal Python object which nobody else is using. Its value is 
nothing special (or even meaningful), so you merely want to check whether what 
you've got is that particular object, using "is". Untested example sketch:

  class SomethingLikeAQueue:

    def __init__(self,......):
      self.things = []
      # private value to use as a sentinel, unique per instance
      self._sentinel = object()
      ... whatever else ...

    def put(self, value):
      # public method to put a new value
      if value is self._sentinel:
        raise ValueError("you may not put the sentinel value")
      self._put(value)

    def _put(self, value):
      # private method accepting any value including the sentinel
      # we will use receipt of the sentinel to process the things and stop 
      # further acceptance of more things
      if value is self._sentinel:
        things = self.things
        self.things = None  # will make the .append blow up
        ... process things here maybe ...
      else:
        things.append(value)

    def close(self):
      # send the sentinel to indicate no more things
      self._put(self._sentinel)

Cheers,
Cameron Simpson <cs at zip.com.au>

From __peter__ at web.de  Sun Jan 17 05:34:20 2016
From: __peter__ at web.de (Peter Otten)
Date: Sun, 17 Jan 2016 11:34:20 +0100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info> <n7dcg5$61a$1@ger.gmane.org>
 <20160117073842.GP10854@ando.pearwood.info>
Message-ID: <n7fqne$rfp$1@ger.gmane.org>

Steven D'Aprano wrote:

>> Slightly related is a problem that comes up in practice; you cannot
>> specify "including the last item" with negative indices:
> 
> But you can do so by leaving the end index blank:

That's why the problem typically comes up when the stop index is a variable.


From __peter__ at web.de  Sun Jan 17 05:52:07 2016
From: __peter__ at web.de (Peter Otten)
Date: Sun, 17 Jan 2016 11:52:07 +0100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
References: <CANDiX9KqEmyHK0fH4rOwNEAuqDMyPK+bFp8p5XNDWH1LjDJNgw@mail.gmail.com>
 <20160116115411.GO10854@ando.pearwood.info> <n7dcg5$61a$1@ger.gmane.org>
 <CANDiX9Kfg5u-uRbarPZ07Vi0rv4XZc_pDZxq5e_PervOEok=DA@mail.gmail.com>
Message-ID: <n7frop$kvi$1@ger.gmane.org>

boB Stepp wrote:

>> The hard part is to remember to test whenever a negative index is
>> calculated.
> 
> I am assuming that this is relevant to what just came before, the use
> of this "or None" check.  Is this correct?

No, I mean that you always should test your code against the corner cases. 
For example a trivial and seemingly harmless function

def tail(items, size):
    return items[-size:]

should return an empty list with size=0 (you might get away with undefined 
behaviour for size<0). If you only have a test

class T(unittest.TestCase):
    def test_tail(self):
        self.assertEqual(tail("abcde", 2), "de")

your coverage tool might be happy, but you are still in for trouble. You 
need at least

self.assertEqual(tail("abcde", 0), "")

to be prepared for the 0 == -0 problem.


From bachir_bac at yahoo.com  Sun Jan 17 06:43:39 2016
From: bachir_bac at yahoo.com (Bachir Bachir)
Date: Sun, 17 Jan 2016 11:43:39 +0000 (UTC)
Subject: [Tutor] pandas data frame
References: <138407729.6187388.1453031019097.JavaMail.yahoo.ref@mail.yahoo.com>
Message-ID: <138407729.6187388.1453031019097.JavaMail.yahoo@mail.yahoo.com>

?Hello EverybodyI need to sort a dataframe according to a specific column the create new dataframes according to the sorted columns each new created dataframe should contain the list(set('the sorted element)any help please , i am new in python and pandas?thanks muchBachir

From alan.gauld at btinternet.com  Sun Jan 17 09:35:04 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 17 Jan 2016 14:35:04 +0000
Subject: [Tutor] pandas data frame
In-Reply-To: <138407729.6187388.1453031019097.JavaMail.yahoo@mail.yahoo.com>
References: <138407729.6187388.1453031019097.JavaMail.yahoo.ref@mail.yahoo.com>
 <138407729.6187388.1453031019097.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <n7g8qp$lmv$1@ger.gmane.org>

On 17/01/16 11:43, Bachir Bachir via Tutor wrote:
> i am new in python and pandas thanks muchBachir

Hi Bachir,

This list is for questions about core Python and its standard
library. Although pandas is part of some Python distributions
(eg anaconda) it's not really part of the standard library but
part of SciPy.

There is a dedicated support forum for SciPy and you might
get more responses posting there.

http://scipy.org/scipylib/mailing-lists.html

However, a few folks on this list do use pandas so you might
get lucky here. :-)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From __peter__ at web.de  Sun Jan 17 10:19:15 2016
From: __peter__ at web.de (Peter Otten)
Date: Sun, 17 Jan 2016 16:19:15 +0100
Subject: [Tutor] pandas data frame
References: <138407729.6187388.1453031019097.JavaMail.yahoo.ref@mail.yahoo.com>
 <138407729.6187388.1453031019097.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <n7gbdk$1ev$1@ger.gmane.org>

Bachir Bachir via Tutor wrote:

> Hello EverybodyI need to sort a dataframe according to a specific column
> the create new dataframes according to the sorted columns each new created
> dataframe should contain the list(set('the sorted element)any help please
> , i am new in python and pandas thanks muchBachir

Like this?

>>> df
   foo bar
0  3.3   b
1  2.2   b
2  7.5   a
3  1.1   c
4  4.7   a

[5 rows x 2 columns]
>>> df.groupby("bar").sum()
      foo
bar      
a    12.2
b     5.5
c     1.1

[3 rows x 1 columns]



From __peter__ at web.de  Sun Jan 17 11:27:42 2016
From: __peter__ at web.de (Peter Otten)
Date: Sun, 17 Jan 2016 17:27:42 +0100
Subject: [Tutor] Source of MySQL Command Interpreter
References: <CAMtNk+tw=QQxL-xiPVZ+rNV_Wn-hZTej-mHN-YUUvtRVrKY_4w@mail.gmail.com>
Message-ID: <n7gfe0$bio$1@ger.gmane.org>

Ricardo Mart?nez wrote:

> Hi, i wrote a small APP to execute MySQL commands and retrieve to a
> Treeview
> 
> http://pastebin.com/v2C8kAu1
> 
> Share your comments and upgrades.

Well, you have an application that works -- that's good. 
Do you have specific questions in mind? Then don't hesitate to ask.
Lacking that I can only give some general remarks:

(1) Read PEP8 and consider following it with your code.

(2) Write unittests to ensure that things that currently work keep working.

(3) Check code coverage to ensure that there is no dead code and no untested 
code.

(4) Consider dropping star imports like

from tkinter import *

in favour of

import tkinter as tk

or at least

from tkinter import name1, ..., nameN 

(5) You have one big do-it-all class. Some modularization would probably be 
good. To avoid modularizing into the blue I suggest that you write a version 
of your script that works with SQLite or PostgreSQL, and aim to put as much 
of the code into helper modules that can be used by both versions of your 
application.


From akleider at sonic.net  Sun Jan 17 13:49:43 2016
From: akleider at sonic.net (Alex Kleider)
Date: Sun, 17 Jan 2016 10:49:43 -0800
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <20160117101822.GA47709@cskk.homeip.net>
References: <deac8c7745fce587e3c1e8ae6241d8f3@sonic.net>
 <20160117101822.GA47709@cskk.homeip.net>
Message-ID: <1e6a8b24a1ed312e728fb320ef4dc037@sonic.net>


On 2016-01-17 02:18, Cameron Simpson wrote:
> On 16Jan2016 22:42, Alex Kleider <akleider at sonic.net> wrote:
>> On 2016-01-16 18:02, Cameron Simpson wrote:
>>> much like writing a function "def f(x, y=None)"; None is a sentinel
>>> value - specially recognised as nor in the normal domain for that
>>> value.
>> 
>> Can you please clarify the last bit:
>> "specially recognised as nor in the normal domain for that value."
> 
> s/nor/not/
> 
> The domain of a function (or value) is the set of valid values it may
> take. The range is the set of values it may produce.
> 
> A sentinel value is a special value you may encounter in a data set
> (or as a value) which is _not_ part of the normal domain; often it
> indicates the end of a sequence (hence the name, like a border guard).
>  In Python the special value None is often used as a sentinel value.
> Since all names have _some_ value, if you need to indicate that this
> name "isn't set" or "doesn't specify a valid value", you need a
> sentinal value, often "None".
> 
> So the common idiom for default values in Python function definitions:
> 
>  def func(a, b, c=None):
>    if c is None:
>      c = default-value-for-c
>    ...
> 
> indicates that the parameters "a" and "b must be supplied, and "c" is
> optional.  If not supplied it will have the value "None". This is a
> sentinel value so that "func" can distinguish a valid value from a
> value not supplied.
> 
> You can also use them to indicate the end of data in some sense. If
> you're looking at a text file as lines of printable characters, the
> newline character at the end of the line could be considered a
> sentinel. Also consider a Queue: there's no notion of "closed" in the
> stdlib version, so one might put a None on the queue to indicate to
> the consumer that there will be no more items.
> 
> It isn't always None. Sometimes you may want to pass None as normal
> data, or you have no control over what is passed around. In that case
> you might need to make a unique sentinel value entirely for your
> object. The normal way to do this is simply to make a new object:
> 
>  sentinel = object()
> 
> This is a minimal Python object which nobody else is using. Its value
> is nothing special (or even meaningful), so you merely want to check
> whether what you've got is that particular object, using "is".
> Untested example sketch:
> 
>  class SomethingLikeAQueue:
> 
>    def __init__(self,......):
>      self.things = []
>      # private value to use as a sentinel, unique per instance
>      self._sentinel = object()
>      ... whatever else ...
> 
>    def put(self, value):
>      # public method to put a new value
>      if value is self._sentinel:
>        raise ValueError("you may not put the sentinel value")
>      self._put(value)
> 
>    def _put(self, value):
>      # private method accepting any value including the sentinel
>      # we will use receipt of the sentinel to process the things and
> stop      # further acceptance of more things
>      if value is self._sentinel:
>        things = self.things
>        self.things = None  # will make the .append blow up
>        ... process things here maybe ...
>      else:
>        things.append(value)     #### ??? self.things.append(value)
> 
>    def close(self):
>      # send the sentinel to indicate no more things
>      self._put(self._sentinel)
> 
> Cheers,
> Cameron Simpson <cs at zip.com.au>

Thanks, Cameron; use of 'sentinel = object()' is a whole new (and 
useful) concept for me.
May I trouble you further by specifically asking about 's/nor/not/'- I 
don't get what that's about.
Has it to do with this 'nor': 
http://www.merriam-webster.com/dictionary/nor?
Sincerely,
Alex

ps Am I correct that towards the end of your code it should have been
     self.things.append(value)
(about 17 lines above?? )

From ben+python at benfinney.id.au  Sun Jan 17 14:27:32 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 18 Jan 2016 06:27:32 +1100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
References: <deac8c7745fce587e3c1e8ae6241d8f3@sonic.net>
 <20160117101822.GA47709@cskk.homeip.net>
 <1e6a8b24a1ed312e728fb320ef4dc037@sonic.net>
Message-ID: <85lh7onh0b.fsf@benfinney.id.au>

Alex Kleider <akleider at sonic.net> writes:

> May I trouble you further by specifically asking about 's/nor/not/'- I
> don't get what that's about.

He's using a common editor syntax (the ancient ?ed? editor's
?substitute? command, which is inherited by Unix ?sed? and ?vi?, among
others) to represent ?please replace the text ?nor? in that line, with
the text ?not? instead?.

-- 
 \      ?Do I believe in God? ? without clarification of a kind I have |
  `\    never seen, I don?t know whether I believe or don?t believe in |
_o__)                whatever a questioner has in mind.? ?Noam Chomsky |
Ben Finney


From cs at zip.com.au  Sun Jan 17 16:48:59 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 18 Jan 2016 08:48:59 +1100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <1e6a8b24a1ed312e728fb320ef4dc037@sonic.net>
References: <1e6a8b24a1ed312e728fb320ef4dc037@sonic.net>
Message-ID: <20160117214859.GA44826@cskk.homeip.net>

On 17Jan2016 10:49, Alex Kleider <akleider at sonic.net> wrote:
>>>Can you please clarify the last bit:
>>>"specially recognised as nor in the normal domain for that value."
>>
>>s/nor/not/
>
>May I trouble you further by specifically asking about 's/nor/not/'- I don't 
>get what that's about.

Ah. Ed, sed, vi, vim speak. Substitute: replace "nor" with "not". The "nor" is 
a typo. I meant "not in the normal domain".

>Has it to do with this 'nor': http://www.merriam-webster.com/dictionary/nor?

"Nor" is indeed a useful word, but it wasn't the word I intended.

[...]
>>It isn't always None. [...]
>>you might need to make a unique sentinel value entirely for your
>>object. The normal way to do this is simply to make a new object:
>>
>> sentinel = object()
>>
>>This is a minimal Python object which nobody else is using. Its value
>>is nothing special (or even meaningful), so you merely want to check
>>whether what you've got is that particular object, using "is".
>>Untested example sketch:
>>
>> class SomethingLikeAQueue:
>>
>>   def __init__(self,......):
>>     self.things = []
>>     # private value to use as a sentinel, unique per instance
>>     self._sentinel = object()
>>     ... whatever else ...
>>
>>   def put(self, value):
>>     # public method to put a new value
>>     if value is self._sentinel:
>>       raise ValueError("you may not put the sentinel value")
>>     self._put(value)
>>
>>   def _put(self, value):
>>     # private method accepting any value including the sentinel
>>     # we will use receipt of the sentinel to process the things and
>>stop      # further acceptance of more things
>>     if value is self._sentinel:
>>       things = self.things
>>       self.things = None  # will make the .append blow up
>>       ... process things here maybe ...
>>     else:
>>       things.append(value)     #### ??? self.things.append(value)
>>
>>   def close(self):
>>     # send the sentinel to indicate no more things
>>     self._put(self._sentinel)
[...]
>ps Am I correct that towards the end of your code it should have been
>    self.things.append(value)

Yes. I probably let myself be lazy because of the earlier "things = 
self.things" in the "true" branch of the "if", where I did it to keep the 
former value of "things" for processing before scrubbing self.things. Of 
course, that way lies buggy code.

I do occasionally pull various object attributes into local variables for 
performance and readability; when I do that it really should be right at the 
top of the function just after any parameter processing, thus:

  class Foo:
    def method(self, foo, bar):
      things = self.things
      for item in foo:
        things.append(item) # or whatever

There are usually two reasons I would do that ("things = self.things" at the 
top): (a) for readability if I'm going to be saying "things" a lot - 
"self.things" may be cumbersome/wordy, making the code very verbose or (b) for 
performance.

To the latter: saying "self.things" requires Python to look up the things 
attribute in "self" every time you use it; if you put it into a local variable 
then Python has direct access to it from the function scope - in CPython his is 
very efficient, and likely so in other Python implementations.

Don't forget that because both "self.things" and "things" refer to the same 
list object (in the example earlier) there's no need to have any final 
"self.things = things" because both are acting on the same list.

Cheers,
Cameron Simpson <cs at zip.com.au>

From lordrip at gmail.com  Sun Jan 17 13:27:40 2016
From: lordrip at gmail.com (=?UTF-8?Q?Ricardo_Mart=C3=ADnez?=)
Date: Sun, 17 Jan 2016 19:27:40 +0100
Subject: [Tutor] Question about grid layout
Message-ID: <CAMtNk+ub952WhX5VgzRn=PZfpafX6A0xWDd44SnGCtX+DtzT_A@mail.gmail.com>

Hi folks, first thanks to Peter and Alan for the comments about the
Interpreter, i really appreciate that.

The question that i have in mind is about grid layout, i have the below
code and i want to resize every widget when the user resize the main
windows.

"""
START
"""


import tkinter  as tk
import tkinter.ttk as ttk


class sqlConsole(ttk.Frame):
    def __init__(self):
        self.root = tk.Tk()
        self.width = "640"
        self.height = "480"

        ttk.Frame.__init__(self, self.root)
        self.createWidgets()
        self.layoutWidgets()

        self.root.mainloop()


    def createWidgets(self):
        self.frmFrame = ttk.Frame(self.root, width=self.width,
height=self.height)
        self.grdResult = ttk.Treeview(self.frmFrame)

        self.grdResult.column("#0", width=self.width)
        self.frmFrame.grid_propagate(0)


    def layoutWidgets(self):
        self.rowconfigure(0, weight=1)
        self.columnconfigure(0, weight=1)
        self.frmFrame.rowconfigure(0, weight=1)
        self.frmFrame.columnconfigure(0, weight=1, minsize=self.width)

        self.grdResult.grid(row=0, column=0, columnspan=5, padx=5, pady=5, \
            sticky="nsew")
        self.frmFrame.grid(sticky="nsew")


if __name__ == "__main__":
    app = sqlConsole()


"""
END
"""

The issue that i have is when i resize the window the Treeview won't, i
tried combinations of sticky as NWSE, or WS, but is not working for me,
and i think that i'm missing some instruction in the grid layout that is
avoiding the widget resize.

I've googled the issue but not finish to understand the "misterious ways of
the grid layout", sometimes i think that is like "the ways of the force".

Thanks in advice!

From sjeik_appie at hotmail.com  Sun Jan 17 17:14:53 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 17 Jan 2016 22:14:53 +0000
Subject: [Tutor] str.strip strange result...?
In-Reply-To: <n7b787$ktu$1@ger.gmane.org>
References: <CACvW2fxwvHHKW0WOHDiLSuGSmv6_uTK82HYUkQtgsCRP8fqC4Q@mail.gmail.com>,
 <n7b787$ktu$1@ger.gmane.org>
Message-ID: <DUB123-W474C8338133C5BB6E5B14083CF0@phx.gbl>

> To: tutor at python.org
> From: __peter__ at web.de
> Date: Fri, 15 Jan 2016 17:37:25 +0100
> Subject: Re: [Tutor] str.strip strange result...?
> 
> Jignesh Sutar wrote:
> 
> > #python2.7
> > 
> >>>> s="V01_1"
> >>>> s.strip("_1")
> > 'V0'
> > 
> > 
> > Wouldn't you expect the result to be "V01" ?
> 
> str.strip() doesn't strip off a suffix or prefix; its first argument is 
> interpreted as a character set, i. e. as long as s ends/starts with any of 
> the characters "_" or "1", remove that.
> 
> If you want to remove a suffix you have to write
> 
> if suffix and s.endswith(suffix):
>     s = s[:-len(suffix)]

Not sure which one is faster, but in this case I actually find a regex more readable (!):
>>> re.sub(r"_1$", "", "V01_1")
'V01'

Without the $ sign may also do the trick, but:
>>> re.sub(r"_1", "", "V01_1_1")
'V01'
>>> re.sub(r"_1$", "", "V01_1_1")
'V01_1'


 		 	   		  

From sjeik_appie at hotmail.com  Sun Jan 17 17:17:55 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 17 Jan 2016 22:17:55 +0000
Subject: [Tutor] Question about the memory manager
In-Reply-To: <CACL+1assrpC7JY2g0FE9nLK=z8hcupskUAgM5y=2xn5gq19Efw@mail.gmail.com>
References: <DUB123-W34A065D8E6F1483A25AE9983C80@phx.gbl>
 <CAExdVN=Fpw7x0pMRNT+JCFsq4PoWWgrooMU3qTjqpZ7-nqrWYQ@mail.gmail.com>,
 <DUB123-W1156BF8488894A8EC1523783CB0@phx.gbl>
 <DUB123-W221738530921133AAECD0483CC0@phx.gbl>,
 <CACL+1assrpC7JY2g0FE9nLK=z8hcupskUAgM5y=2xn5gq19Efw@mail.gmail.com>
Message-ID: <DUB123-W128DB34570D099886CC28B83CF0@phx.gbl>

> From: eryksun at gmail.com
> Date: Thu, 14 Jan 2016 04:42:57 -0600
> Subject: Re: [Tutor] Question about the memory manager
> To: tutor at python.org
> CC: sjeik_appie at hotmail.com
> 
> On Thu, Jan 14, 2016 at 3:03 AM, Albert-Jan Roskam
> <sjeik_appie at hotmail.com> wrote:
> >
> > These two pages are quite nice. The author says the memory used by small objects is
> > never returned to the OS, which may be problematic for long running processes.
> 
> The article by Evan Jones discusses a patch to enable releasing unused
> arenas (i.e. "how the problem was fixed"). Starting with 2.5, unused
> arenas do get released back to the heap. Here's the diff in which Tim
> Peters merged in a "heavily altered derivative" of Evan's patch [1].
> 
> Also, 2.7 and 3.3 bypass C malloc/free and the process heap to instead
> use mmap/munmap on POSIX when available. This avoids the heap
> high-water mark problem. Similarly, 3.4 switched to using
> VirtualAlloc/VirtualFree on Windows. 3.4 also introduced the
> PyObjectArenaAllocator and associated C API functions [2] to allow
> modifying the default allocators.
> 
> [1]: https://hg.python.org/cpython/diff/685849bd905c/Objects/obmalloc.c
> [2]: https://docs.python.org/3/c-api/memory.html#customize-pyobject-arena-allocator

Hi Eryk,

Thanks a lot for the info and the links. This is truly interesting to read about! Glad to know that the high-water mark problem is no longer relevant anymore in recent Python versions. Also, thank you for your suggestion about psutils (and ctypes).

Best wishes,
Albert-Jan
 		 	   		  

From alan.gauld at btinternet.com  Sun Jan 17 17:19:55 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 17 Jan 2016 22:19:55 +0000
Subject: [Tutor] Question about grid layout
In-Reply-To: <CAMtNk+ub952WhX5VgzRn=PZfpafX6A0xWDd44SnGCtX+DtzT_A@mail.gmail.com>
References: <CAMtNk+ub952WhX5VgzRn=PZfpafX6A0xWDd44SnGCtX+DtzT_A@mail.gmail.com>
Message-ID: <n7h42b$st4$1@ger.gmane.org>

On 17/01/16 18:27, Ricardo Mart?nez wrote:
> Hi folks, first thanks to Peter and Alan for the comments about the
> Interpreter, i really appreciate that.

I don't remember giving any comments about the interpreter,
but if I did you're welcome! :-)

> The question that i have in mind is about grid layout, i have the below
> code and i want to resize every widget when the user resize the main
> windows.

Personally I rarely use grid() unless it really is
a form that I'm creating. Usually I use pack() because
I find it more flexible. So I'd suggest that you pack()
the treeview into the frame then grid() the frame.

In the packer use the expand=True and fill=BOTH options.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Sun Jan 17 17:21:04 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 18 Jan 2016 09:21:04 +1100
Subject: [Tutor] Help!
In-Reply-To: <CABV0pWwtPaLEbsQpU70hD-ktBwfXrTYe8A0BSOLYU+vTSaKZtg@mail.gmail.com>
References: <CABV0pWwtPaLEbsQpU70hD-ktBwfXrTYe8A0BSOLYU+vTSaKZtg@mail.gmail.com>
Message-ID: <20160117222104.GS10854@ando.pearwood.info>

On Fri, Jan 15, 2016 at 11:33:24AM -0500, Chelsea G wrote:
> Hi,

[snip stuff which is, as far as I can see, irrelevant]

> What I am having issues with is the def txt_output that is where I am
> trying to take the .csv off and add the .txt but keep the filename the
> same. For example, having a filename "weekly_20160102.csv" and then create
> a txt filename with "weekly_20160102.txt" and have all the counts and
> products in the text file. Is there any way to do this?


This is how to change the file extension:

py> import os
py> filename = "weekly_20160102.csv"
py> newname = os.path.splitext(filename)[0] + ".txt"
py> print(newname)
weekly_20160102.txt


Once you have the new file name, ending with .txt, do you understand how 
to open it and write to it? For example, you might do this:

with open(newname, "w") as f:
    f.write("line 1\n")
    f.write("second line\n")
    f.write("last line\n")


When you exit the "with" block, the file will be automatically closed.

Remember to use \n (newline) at the end of each line.


-- 
Steve

From sjeik_appie at hotmail.com  Sun Jan 17 17:24:00 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 17 Jan 2016 22:24:00 +0000
Subject: [Tutor] me, my arm, my availability ...
In-Reply-To: <56971EE6.8010101@gmail.com>
References: <201601132047.u0DKlhIc025984@theraft.openend.se>,
 <56971EE6.8010101@gmail.com>
Message-ID: <DUB123-W259D255905161A18CE088883CF0@phx.gbl>



> To: tutor at python.org
> From: cmgcomsol at gmail.com
> Date: Thu, 14 Jan 2016 09:37:02 +0530
> Subject: Re: [Tutor] me, my arm, my availability ...
> 
> take care, get well soon,
> 
> regards
> 
> CMG
> 
> On Thursday 14 January 2016 02:17 AM, Laura Creighton wrote:
> > I fell recently.  Ought to be nothing, but a small chip of bone, either an
> > existing one or one I just made is nicely wedged in the joint taking away
> > a whole lot of the ability of my arm to rotate in the elbow joint.  Or
> > hold my arm in a position that is usual for typing.  Plus,  now that the
> > sprain/swelling is more or less over, the pain, unfortunately is not.
> >
> > The real downside is that my typing speed is down from 135-140 wpm
> > to 5-10 wmp.  At this rate, just getting my usual work done takes
> > overtime.
> >
> > Seems like surgery is needed to fix this.
> >
> > So I wanted you all to know, no, I haven't forgotten you and no haven't
> > stopped caring.  I have just stopped being as __capable__ if you know
> > what I mean.
> >
> > Please take care of yourselves and each other.  I will often be reading
> > even if typing is more than I can do right now.

awww, that sucks. Sorry to hear that. My daughter had something similar last year. Barely visible on x-rays, but very painful and it took weeks to heal.
Take care!

Albert-Jan
 		 	   		  

From steve at pearwood.info  Sun Jan 17 17:42:09 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 18 Jan 2016 09:42:09 +1100
Subject: [Tutor] Message Passing & User Interfaces
In-Reply-To: <CAOhNYvm2dmfW8Mj+dgn6=FSTE3TzaJ81kvg7ke4PZ-W+yGvAJg@mail.gmail.com>
References: <CAOhNYvm2dmfW8Mj+dgn6=FSTE3TzaJ81kvg7ke4PZ-W+yGvAJg@mail.gmail.com>
Message-ID: <20160117224209.GT10854@ando.pearwood.info>

On Fri, Jan 15, 2016 at 12:06:38PM -0500, wolfrage8765 at gmail.com wrote:

> I realize this is a higher level question; so please direct me as
> appropriate. I can not seem to find a good standard that is in
> practical use for controlling user interfaces via a message passing.
> Does any one have any links to such an example? I would be most
> grateful. Thank you.


Perhaps you would be interested in LiveCode:

http://livecode.com/

which uses such a system. It is not Python, but it is a complete IDE 
(Integrated Development Environment) based on Apple's Hypercard from the 
1990s, only considerably updated. The programming model is based on 
message passing to and from graphical elements of the user interface, 
for example buttons receive messages when you click on them, you write a 
handler for those messages you want to deal with, and send messages to 
other components.

I'm not aware of anything even remotely similar for Python. PythonCard 
came close, but I don't think it is still maintained.


-- 
Steve

From akleider at sonic.net  Sun Jan 17 17:50:27 2016
From: akleider at sonic.net (Alex Kleider)
Date: Sun, 17 Jan 2016 14:50:27 -0800
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <20160117214859.GA44826@cskk.homeip.net>
References: <1e6a8b24a1ed312e728fb320ef4dc037@sonic.net>
 <20160117214859.GA44826@cskk.homeip.net>
Message-ID: <9073f8bc5f18d7f99d7bfaa4ccb3aaf1@sonic.net>


Again, a personal thank you.
More often than not, when answering one thing, you teach me about
other things.  The 'thing' thing is only the latest.  Of course
I knew that using a name bound to a collection would effect the
contents of the collection but it would never have occurred to me
to use it to advantage as you describe.
Another "Ah, Ha!" experience.
Alex

On 2016-01-17 13:48, Cameron Simpson wrote:
> On 17Jan2016 10:49, Alex Kleider <akleider at sonic.net> wrote:
>>>> Can you please clarify the last bit:
>>>> "specially recognised as nor in the normal domain for that value."
>>> 
>>> s/nor/not/
>> 
>> May I trouble you further by specifically asking about 's/nor/not/'- I 
>> don't get what that's about.
> 
> Ah. Ed, sed, vi, vim speak. Substitute: replace "nor" with "not". The
> "nor" is a typo. I meant "not in the normal domain".
> 
>> Has it to do with this 'nor': 
>> http://www.merriam-webster.com/dictionary/nor?
> 
> "Nor" is indeed a useful word, but it wasn't the word I intended.
> 
> [...]
>>> It isn't always None. [...]
>>> you might need to make a unique sentinel value entirely for your
>>> object. The normal way to do this is simply to make a new object:
>>> 
>>> sentinel = object()
>>> 
>>> This is a minimal Python object which nobody else is using. Its value
>>> is nothing special (or even meaningful), so you merely want to check
>>> whether what you've got is that particular object, using "is".
>>> Untested example sketch:
>>> 
>>> class SomethingLikeAQueue:
>>> 
>>>   def __init__(self,......):
>>>     self.things = []
>>>     # private value to use as a sentinel, unique per instance
>>>     self._sentinel = object()
>>>     ... whatever else ...
>>> 
>>>   def put(self, value):
>>>     # public method to put a new value
>>>     if value is self._sentinel:
>>>       raise ValueError("you may not put the sentinel value")
>>>     self._put(value)
>>> 
>>>   def _put(self, value):
>>>     # private method accepting any value including the sentinel
>>>     # we will use receipt of the sentinel to process the things and
>>> stop      # further acceptance of more things
>>>     if value is self._sentinel:
>>>       things = self.things
>>>       self.things = None  # will make the .append blow up
>>>       ... process things here maybe ...
>>>     else:
>>>       things.append(value)     #### ??? self.things.append(value)
>>> 
>>>   def close(self):
>>>     # send the sentinel to indicate no more things
>>>     self._put(self._sentinel)
> [...]
>> ps Am I correct that towards the end of your code it should have been
>>    self.things.append(value)
> 
> Yes. I probably let myself be lazy because of the earlier "things =
> self.things" in the "true" branch of the "if", where I did it to keep
> the former value of "things" for processing before scrubbing
> self.things. Of course, that way lies buggy code.
> 
> I do occasionally pull various object attributes into local variables
> for performance and readability; when I do that it really should be
> right at the top of the function just after any parameter processing,
> thus:
> 
>  class Foo:
>    def method(self, foo, bar):
>      things = self.things
>      for item in foo:
>        things.append(item) # or whatever
> 
> There are usually two reasons I would do that ("things = self.things"
> at the top): (a) for readability if I'm going to be saying "things" a
> lot - "self.things" may be cumbersome/wordy, making the code very
> verbose or (b) for performance.
> 
> To the latter: saying "self.things" requires Python to look up the
> things attribute in "self" every time you use it; if you put it into a
> local variable then Python has direct access to it from the function
> scope - in CPython his is very efficient, and likely so in other
> Python implementations.
> 
> Don't forget that because both "self.things" and "things" refer to the
> same list object (in the example earlier) there's no need to have any
> final "self.things = things" because both are acting on the same list.
> 
> Cheers,
> Cameron Simpson <cs at zip.com.au>

From robertvstepp at gmail.com  Sun Jan 17 17:50:35 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sun, 17 Jan 2016 16:50:35 -0600
Subject: [Tutor] Should a "data" directory have a __init__.py file?
Message-ID: <CANDiX9LadZxP6Uc_-vWG+4xLYsh+FGVK4xfOvFgvvhMECVu48Q@mail.gmail.com>

Py 3.4.4; W7-64-bit

I've been reviewing all of the posts relevant to questions I asked
last year on Python project directory structures.  I was wondering
about a data directory nested under the program's top-level directory.
I can see no reason why I would need an __init__.py file for a data
directory.  Yes, files will be read there, but there will be no code
there.  Is this correct?

-- 
boB

From steve at pearwood.info  Sun Jan 17 17:56:53 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 18 Jan 2016 09:56:53 +1100
Subject: [Tutor] Substitution function needed
In-Reply-To: <SN1PR13MB04474B9165EAABE6DDA4D2B4F4CD0@SN1PR13MB0447.namprd13.prod.outlook.com>
References: <SN1PR13MB04474B9165EAABE6DDA4D2B4F4CD0@SN1PR13MB0447.namprd13.prod.outlook.com>
Message-ID: <20160117225653.GU10854@ando.pearwood.info>

On Fri, Jan 15, 2016 at 05:07:58PM +0000, Chad Perry wrote:

> I need to know how to substitute for the drive letter for the following drives.
> 
> sad-sdp
> also will need to wipe data from /dev/md1
> 
> I believe that the script is sound just sub's

I don't understand your question. Substitute what for the drive 
letter? Can you explain what you want more carefully?

If I understand your script correctly, the user types in the drive they 
want wiped. So if the user types "/dev/md1", that's the drive that will 
be wiped. If they type "/dev/sdc", /dev/sdc will be wiped.

(How do you stop the user from wiping the drive that contains your 
script? And the drive containing the root partition?)


-- 
Steve

From cs at zip.com.au  Sun Jan 17 18:27:00 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 18 Jan 2016 10:27:00 +1100
Subject: [Tutor] s.insert(i,
 x) explanation in docs for Python 3.4 confusing to me
In-Reply-To: <9073f8bc5f18d7f99d7bfaa4ccb3aaf1@sonic.net>
References: <9073f8bc5f18d7f99d7bfaa4ccb3aaf1@sonic.net>
Message-ID: <20160117232700.GA81483@cskk.homeip.net>

On 17Jan2016 14:50, Alex Kleider <akleider at sonic.net> wrote:
>Again, a personal thank you.

No worries.

>More often than not, when answering one thing, you teach me about
>other things.  The 'thing' thing is only the latest.  Of course
>I knew that using a name bound to a collection would effect the
>contents of the collection but it would never have occurred to me
>to use it to advantage as you describe.

Note that it shouldn't be overdone. Pointless optimisation is also a source of 
bugs and obfuscation.

As I remarked, I'll do this for readability if needed or for performance where 
"things" (or whatever) is going to be heavily accessed (eg in a busy loop), 
such that getting-it-from-the-instance might significantly affect the 
throughput. For many things you may as well not waste your time with it - the 
direct expression of the problem is worth it for maintainability.

It is largely if I'm doing something low level (where interpreted or high level 
languages like Python lose because their data abstraction or cost-of-operation 
may exceed the teensy teensy task being performed.

As an example, I'm rewriting some code right now that scans bytes objects for 
boundaries in the data. It is inherently a sequential "get a byte, update some 
numbers and consult some tables, get the next byte ...". It is also heavily 
used in the storage phase of this application: all new data is scanned this 
way.

In C one might be going:

  for (char *cp=buffer; buf_len > 0; cp++, buflen--) {
    b = *cp;
    ... do stuff with "b" ...
  }

That will be efficiently compiled to machine code and run flat out at your 
CPU's native speed. The Python equivalent, which would look a bit like this:

  for offset in range(len(buffer)):
    b = bs[offset]
    ... do stuff with "b" ...

or possibly:

  for offset, b in enumerate(buffer):
    ... do stuff with "b" ...

will inherently be slower because in Python "bs", "b" and "offset" may be any 
type, so Python must do all this through object methods, and CPython (for 
example) compiles to opcodes for a logial machine, which are themselves 
interpreted by C code to decide what to do. You can see that the ratio of 
"implement the Python operations" to the core "do something with a byte" stuff 
can potentially be very large.

At some point I may be writing a C extension for these very busy parts of the 
code but for now I am putting some effort into making the pure Python 
implementation as efficient as I can while keeping it correct. This is an 
occasion when it is worth expending significant effort on minimising 
indirection (using "things" instead of "self.things", etc), because that 
indirection has a cost that _in this case_ will be large compared to the core 
operations.

Cheers,
Cameron Simpson <cs at zip.com.au>

From robertvstepp at gmail.com  Sun Jan 17 18:37:39 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sun, 17 Jan 2016 17:37:39 -0600
Subject: [Tutor] How (and whether I should) use pkgutil.get_data()?
Message-ID: <CANDiX9Ki8p=Zx5tSXA0ZrYcpmqo2AjFrRTSqvrrv=hv1zQDYSA@mail.gmail.com>

>From "Python Cookbook, 3rd Ed." by David Beazley and Brian K. Jones,
the authors recommend using pkgutil.get_data() to obtain the contents
of data files.  Normally I would use "with open..." giving my expected
file's location, but the authors claim that using pkgutil.get_data()
is a more robust way to do things due to the differences in how
packages get installed and used.  I have not thought that far ahead on
distributing my work.  So far I have been using my projects on the
machines on which I develop them, or manually copy the source code to
another nearby machine, manually making any adjustments in my
program's "home" directory.  (Typically my main.py program has a
single variable that sets the home location for the project, so I only
need to manually change it in this one place.)

But these projects in development for my wife's classes where they
have a variety of OS's has me rethinking this approach.  The book's
authors also seem to recommend relative addressing for importing a
program's modules.  That sounds nice in terms of being independent of
where the project gets installed, but I don't find "dots" to be very
descriptive!  So, some questions:

1)  Is using pkgutil.get_data() the way I should be reading my data files?

2)  The book says that this method will return "...a byte string
containing the raw contents of the file." (From page 409) Can I do all
of the normal things on this returned object that I could do if I used
"with open..."?  Is it iterable?  [OK, I confess up front.  I am being
lazy here.  I *could* create a file and try this out to see what
happens, so I will understand if I get chided (or worse!).]

3)  Should I be using relative imports in my projects?

TIA!

-- 
boB

From sudiptod19 at gmail.com  Sun Jan 17 18:15:22 2016
From: sudiptod19 at gmail.com (sudipto manna)
Date: Sun, 17 Jan 2016 17:15:22 -0600
Subject: [Tutor] if request.method == 'GET': NameError: global name
 'request' is not defined
Message-ID: <CAERL5Syxrmg4CP2Q26Afx=P7EuEb8v=qUGjq6fZORBkebmJeGQ@mail.gmail.com>

Hi All,
I am running a python flask project for fetching the endpoint.

Please find the files attached.
Parent file: FlaskTest2.py
Endpoint Tester File: FlaskTendpointTester2.py

Process:
1. Execute the file FlaskTest2.py --> python FlaskTest2.py
# This will execute the file on the terminal and will be ready to listen
the requests coming RESTFUL API's written in python, s as to validate the
endpoint and checking the various methods like: GET, POST, PUT, DEETE etc.

2. Execute the FlaskTendpointTester2.py -- > python FlaskTendpointTester2.py
#This will open the terminal and indicate the local hit n which the
FlaskTest2.py is executed and will respond back wth all the response; like
the GET, POST, PUT, DELETE method will be called.

Issue#
When the endpoint file is executed its returning this error:
"if request.method == 'GET': NameError: global name 'request' is not
defined"

Here we are looking if the FlaskTest2.py file has any GET, POST, DELETE
method or not.

The error is coming on the terminal(on same Machine 2 working terminals are
opened) where the FlaskTest2.py is run.


Appreciate the help.

Regards,
Sudipto Manna

From alan.gauld at btinternet.com  Sun Jan 17 20:17:21 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 18 Jan 2016 01:17:21 +0000
Subject: [Tutor] if request.method == 'GET': NameError: global name
 'request' is not defined
In-Reply-To: <CAERL5Syxrmg4CP2Q26Afx=P7EuEb8v=qUGjq6fZORBkebmJeGQ@mail.gmail.com>
References: <CAERL5Syxrmg4CP2Q26Afx=P7EuEb8v=qUGjq6fZORBkebmJeGQ@mail.gmail.com>
Message-ID: <n7hef2$nt8$1@ger.gmane.org>

On 17/01/16 23:15, sudipto manna wrote:
> Hi All,
> I am running a python flask project for fetching the endpoint.
> 
> Please find the files attached.

OK, This is a text based mailing list so attachments tend
not to make it through the server. If they are not too long post them
here, or if they are bigger put them on a pastebin. However...

Flask is not part of the standard library so a little bit
off topic for this list, you may be better off asking on
the Flask support forum.

Howe er it is a common framework so somebody here might
be able to help.

> When the endpoint file is executed its returning this error:
> "if request.method == 'GET': NameError: global name 'request' is not
> defined"

We really need to see the full error trace and the full
context around the reported fault - ideally the full file.
The probability is that the error is telling the truth
and 'request' is not defined yet. Is it part of Flask?
Should you have prepended the module name?
Is the spelling/case correct?


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From ben+python at benfinney.id.au  Sun Jan 17 23:27:03 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 18 Jan 2016 15:27:03 +1100
Subject: [Tutor] Should a "data" directory have a __init__.py file?
References: <CANDiX9LadZxP6Uc_-vWG+4xLYsh+FGVK4xfOvFgvvhMECVu48Q@mail.gmail.com>
Message-ID: <85bn8jo6lk.fsf@benfinney.id.au>

boB Stepp <robertvstepp at gmail.com> writes:

> I can see no reason why I would need an __init__.py file for a data
> directory. Yes, files will be read there, but there will be no code
> there. Is this correct?

That is correct. A directory of files is accessed just fine using the
normal filesystem access features.

What's more, I would say that if a directory is not positively intended
to be a Python package, that directory *should not* contain any
?__init__.py? file.

The module import system will only recognise a directory as a ?package?
(Python's technical term for ?a point in the import hierarchy which
contains other things to import?) if that directory contains a file
named ?__init__.py?. If you do not need the directory to also be
recognised as a package, it should not have such a file.

See <URL:https://docs.python.org/3/reference/import.html#packages> for
the full details of how Python recognises and uses packages in the
import system.

Also be aware Python's use of ?package? conflicts with the broader
computing use of that term (?a unit of software to install on the
operating system?); but it's too late to change either of them,
unfortunately.

-- 
 \        ?The deepest sin against the human mind is to believe things |
  `\           without evidence.? ?Thomas Henry Huxley, _Evolution and |
_o__)                                                    Ethics_, 1893 |
Ben Finney


From dyoo at hashcollision.org  Mon Jan 18 03:29:41 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 18 Jan 2016 00:29:41 -0800
Subject: [Tutor] str.strip strange result...?
In-Reply-To: <DUB123-W474C8338133C5BB6E5B14083CF0@phx.gbl>
References: <CACvW2fxwvHHKW0WOHDiLSuGSmv6_uTK82HYUkQtgsCRP8fqC4Q@mail.gmail.com>
 <n7b787$ktu$1@ger.gmane.org> <DUB123-W474C8338133C5BB6E5B14083CF0@phx.gbl>
Message-ID: <CAGZAPF7Jn40PFntisMnRgGMymp-udAJDz+49Pg9bC2Rd3nsZjg@mail.gmail.com>

> Not sure which one is faster, but in this case I actually find a regex more readable (!):
>>>> re.sub(r"_1$", "", "V01_1")
> 'V01'


Hi Albert-Jan Roskam,

Here's a minor counterpoint to using regexes here: they're sometimes a
bit too powerful.

In this situation, a regex approach might be troublesome if parts of
the suffix can be interpreted as regular expression meta-characters.
For example, if we were trying to strip out a suffix like

    ".exe"

then if we were to try to do this with regexes, we might forget that
"." is a meta-character that acts as a wildcard for any single
character.  Therefore, a regexp-based solution for the general
suffix-removal problem is complicated because we'd need consider
escaping certain characters.  Not that this would be hard, but that
it's yet another detail we have to keep in our heads.

Another disadvantage is that, if the suffix is dynamically determined,
then there's an additional cost in "compiling" the regular expression:
building the pattern-matching machinery doesn't come for free.


For those reasons, I think the regexp approach here is a little bit of
overkill.  This kind of tradeoff is the sort of thing that reference
documentation will probably not mention.   That's why this list is
here, to share the experience of using these systems with other
beginners and learners.  Regexes are still very useful: just be aware
that they have sharp edges.


Good luck to you!

From sudiptod19 at gmail.com  Sun Jan 17 21:13:44 2016
From: sudiptod19 at gmail.com (sudipto manna)
Date: Sun, 17 Jan 2016 20:13:44 -0600
Subject: [Tutor] if request.method == 'GET': NameError: global name
 'request' is not defined
In-Reply-To: <n7hef2$nt8$1@ger.gmane.org>
References: <CAERL5Syxrmg4CP2Q26Afx=P7EuEb8v=qUGjq6fZORBkebmJeGQ@mail.gmail.com>
 <n7hef2$nt8$1@ger.gmane.org>
Message-ID: <CAERL5SyGdE-aaYddNLN+N52Y+__mppsd09eT8j_q_jeP_WqbBg@mail.gmail.com>

Here is the code snippet:

File#FlaskTest2.py

from flask import Flask

app = Flask(__name__)

#Make an app.route() decorator here
@app.route("/puppies/", methods = ['GET' , 'POST'])
def puppiesFunction():
  if request.method == 'GET':
  return getAllPuppies()

  elif request.method == 'POST':
   return makeANewPuppy()

File# FlaskTendpointTester2.py
import httplib2
import json
import sys

print "Running Endpoint Tester....\n"
address = raw_input("Please enter the address of the server you want to
access, \n If left blank the connection will be set to '
http://localhost:5000':   ")
if address == '':
address = 'http://localhost:5000'
#Making a GET Request
print "Making a GET Request for /puppies..."

Virtual machine is running on port:5000

and the error trace seen on when the endpoint code is executed on localhost
port:5000

Traceback (most recent call last):

  File
"/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py",
line 1836, in __call__

    return self.wsgi_app(environ, start_response)

  File
"/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py",
line 1820, in wsgi_app

    response = self.make_response(self.handle_exception(e))

  File
"/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py",
line 1403, in handle_exception

    reraise(exc_type, exc_value, tb)

  File
"/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py",
line 1817, in wsgi_app

    response = self.full_dispatch_request()

  File
"/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py",
line 1477, in full_dispatch_request

    rv = self.handle_user_exception(e)

  File
"/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py",
line 1381, in handle_user_exception

    reraise(exc_type, exc_value, tb)

  File
"/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py",
line 1475, in full_dispatch_request

    rv = self.dispatch_request()

  File
"/Users/sudiptomanna/anaconda2/lib/python2.7/site-packages/flask/app.py",
line 1461, in dispatch_request

    return self.view_functions[rule.endpoint](**req.view_args)

  File "/Users/sudiptomanna/fullstack/vagrant/PythonData/FlaskTest2.py",
line 8, in puppiesFunction

    if request.method == 'GET':

NameError: global name 'request' is not defined


Regards,

Sudipto Manna

On Sun, Jan 17, 2016 at 7:17 PM, Alan Gauld <alan.gauld at btinternet.com>
wrote:

> On 17/01/16 23:15, sudipto manna wrote:
> > Hi All,
> > I am running a python flask project for fetching the endpoint.
> >
> > Please find the files attached.
>
> OK, This is a text based mailing list so attachments tend
> not to make it through the server. If they are not too long post them
> here, or if they are bigger put them on a pastebin. However...
>
> Flask is not part of the standard library so a little bit
> off topic for this list, you may be better off asking on
> the Flask support forum.
>
> Howe er it is a common framework so somebody here might
> be able to help.
>
> > When the endpoint file is executed its returning this error:
> > "if request.method == 'GET': NameError: global name 'request' is not
> > defined"
>
> We really need to see the full error trace and the full
> context around the reported fault - ideally the full file.
> The probability is that the error is telling the truth
> and 'request' is not defined yet. Is it part of Flask?
> Should you have prepended the module name?
> Is the spelling/case correct?
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From dyoo at hashcollision.org  Mon Jan 18 04:03:45 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 18 Jan 2016 01:03:45 -0800
Subject: [Tutor] if request.method == 'GET': NameError: global name
 'request' is not defined
In-Reply-To: <CAERL5SyGdE-aaYddNLN+N52Y+__mppsd09eT8j_q_jeP_WqbBg@mail.gmail.com>
References: <CAERL5Syxrmg4CP2Q26Afx=P7EuEb8v=qUGjq6fZORBkebmJeGQ@mail.gmail.com>
 <n7hef2$nt8$1@ger.gmane.org>
 <CAERL5SyGdE-aaYddNLN+N52Y+__mppsd09eT8j_q_jeP_WqbBg@mail.gmail.com>
Message-ID: <CAGZAPF4NZAqAqeUtjxY1ye7pSRaejQuUN6px4fLXVSMG8Wvg4w@mail.gmail.com>

On Sun, Jan 17, 2016 at 6:13 PM, sudipto manna <sudiptod19 at gmail.com> wrote:
> Here is the code snippet:
>
> File#FlaskTest2.py
>
> from flask import Flask
>
> app = Flask(__name__)
>
> #Make an app.route() decorator here
> @app.route("/puppies/", methods = ['GET' , 'POST'])
> def puppiesFunction():
>   if request.method == 'GET':
>   return getAllPuppies()


Hi Sudipto,

I don't know the Flask framework here, but this code looks suspicious.
What is "request" in this puppiesFunction definition?  It's not a
parameter.  Where does it come from?

Is it a global?

    http://flask.pocoo.org/docs/0.10/quickstart/#accessing-request-data

... oh.  It is a module global.  That is not something I like
philosophically, but oh well, roll with it...


Do you have other working examples from prior code that used requests?
 If so, you may want to double check them.  You're missing a
definition that you need to add to your program.  Take a look at the
documentation: read the "The Request Object" section in the docs, and
you should see the missing line that you need to add to your program.


If you can't find it after you've read this quickstart documentation,
please feel free to ask the mailing list.  But you should be able to
see it: it's the first boxed piece of code at:

    http://flask.pocoo.org/docs/0.10/quickstart/#the-request-object


Good luck!

From alan.gauld at btinternet.com  Mon Jan 18 03:58:23 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 18 Jan 2016 08:58:23 +0000
Subject: [Tutor] if request.method == 'GET': NameError: global name
 'request' is not defined
In-Reply-To: <CAERL5SyGdE-aaYddNLN+N52Y+__mppsd09eT8j_q_jeP_WqbBg@mail.gmail.com>
References: <CAERL5Syxrmg4CP2Q26Afx=P7EuEb8v=qUGjq6fZORBkebmJeGQ@mail.gmail.com>
 <n7hef2$nt8$1@ger.gmane.org>
 <CAERL5SyGdE-aaYddNLN+N52Y+__mppsd09eT8j_q_jeP_WqbBg@mail.gmail.com>
Message-ID: <569CA92F.1030104@btinternet.com>

On 18/01/16 02:13, sudipto manna wrote:
> Here is the code snippet:
>
> File#FlaskTest2.py
>
> from flask import Flask
>
> app = Flask(__name__)
>
> #Make an app.route() decorator here
> @app.route("/puppies/", methods = ['GET' , 'POST'])
> def puppiesFunction():
>   if request.method == 'GET':
>   return getAllPuppies()
>
As suspected the error is correct. You do not have a request object
anywhere in your code. A quick Google tells me that it is part
of Flask. So you need to import request from your flask module:

from flask import Flask, request,,....# any other names you use

hth

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


From cs at zip.com.au  Mon Jan 18 04:55:45 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 18 Jan 2016 20:55:45 +1100
Subject: [Tutor] if request.method == 'GET': NameError: global name
 'request' is not defined
In-Reply-To: <CAERL5SyGdE-aaYddNLN+N52Y+__mppsd09eT8j_q_jeP_WqbBg@mail.gmail.com>
References: <CAERL5SyGdE-aaYddNLN+N52Y+__mppsd09eT8j_q_jeP_WqbBg@mail.gmail.com>
Message-ID: <20160118095545.GA92822@cskk.homeip.net>

On 17Jan2016 20:13, sudipto manna <sudiptod19 at gmail.com> wrote:
>Here is the code snippet:
>
>File#FlaskTest2.py
>
>from flask import Flask

You need to import "request" from flask as well:

  from flask import Flask, request

For others on this list: the Flask framework presents the current web request 
as a thread local global called "request"; simplifies writing handlers as the 
request information is readily available without having to pass it through 
function calls.

Cheers,
Cameron Simpson <cs at zip.com.au>

From __peter__ at web.de  Mon Jan 18 05:25:06 2016
From: __peter__ at web.de (Peter Otten)
Date: Mon, 18 Jan 2016 11:25:06 +0100
Subject: [Tutor] pandas data frame
References: <138407729.6187388.1453031019097.JavaMail.yahoo.ref@mail.yahoo.com>
 <138407729.6187388.1453031019097.JavaMail.yahoo@mail.yahoo.com>
 <n7gbdk$1ev$1@ger.gmane.org>
Message-ID: <n7iei3$o49$1@ger.gmane.org>

Bachir Bachir wrote:

[Bachir, please send your mails to the list, not to me. That way you 
increase the likelihood to get a good answer]

>>     On Sunday, January 17, 2016 4:20 PM, Peter Otten <__peter__ at web.de>
>> wrote:
>>  Bachir Bachir via Tutor wrote:
>>> Hello EverybodyI need to sort a dataframe according to a specific column
>>> the create new dataframes according to the sorted columns each new
>>> created dataframe should contain the list(set('the sorted element)any
>>> help please, i am new in python and pandas thanks muchBachir
>> 
>> Like this?
>> 
>> >>> df
>> 
>>   foo bar
>> 0  3.3  b
>> 1  2.2  b
>> 2  7.5  a
>> 3  1.1  c
>> 4  4.7  a
>> 
>> [5 rows x 2 columns]
>> 
>> >>> df.groupby("bar").sum()
>> 
>>       foo
>> bar     
>> a    12.2
>> b    5.5
>> c    1.1
>> 
>> [3 rows x 1 columns]

> Thanks Peter ,This way but i  want keep all the table elements only
> separate them  by a single column 

I'm sorry, I don't understand what you mean by "separate them by a single 
column". Can you give an example? What exactly should the sorted version of

>> >>> df
>> 
>>   foo bar
>> 0  3.3  b
>> 1  2.2  b
>> 2  7.5  a
>> 3  1.1  c
>> 4  4.7  a
>> 
>> [5 rows x 2 columns]

look like?


From oscar.j.benjamin at gmail.com  Mon Jan 18 06:03:36 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 18 Jan 2016 11:03:36 +0000
Subject: [Tutor] Should a "data" directory have a __init__.py file?
In-Reply-To: <85bn8jo6lk.fsf@benfinney.id.au>
References: <CANDiX9LadZxP6Uc_-vWG+4xLYsh+FGVK4xfOvFgvvhMECVu48Q@mail.gmail.com>
 <85bn8jo6lk.fsf@benfinney.id.au>
Message-ID: <CAHVvXxSbubz=c8Ju6QfyOUCoUwqu13LSoc=FGRsArU7kJa0nzw@mail.gmail.com>

On 18 January 2016 at 04:27, Ben Finney <ben+python at benfinney.id.au> wrote:
>
> The module import system will only recognise a directory as a ?package?
> (Python's technical term for ?a point in the import hierarchy which
> contains other things to import?) if that directory contains a file
> named ?__init__.py?. If you do not need the directory to also be
> recognised as a package, it should not have such a file.

That was the case prior to 3.3. However (I can't immediately test this
but) now a directory without an __init__.py can be an implicit
namespace package if it contains importable modules:

https://www.python.org/dev/peps/pep-0420/

--
Oscar

From oscar.j.benjamin at gmail.com  Mon Jan 18 06:12:12 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 18 Jan 2016 11:12:12 +0000
Subject: [Tutor] How (and whether I should) use pkgutil.get_data()?
In-Reply-To: <CANDiX9Ki8p=Zx5tSXA0ZrYcpmqo2AjFrRTSqvrrv=hv1zQDYSA@mail.gmail.com>
References: <CANDiX9Ki8p=Zx5tSXA0ZrYcpmqo2AjFrRTSqvrrv=hv1zQDYSA@mail.gmail.com>
Message-ID: <CAHVvXxT1P3bPk3_ZsV3t8VR4Pzty71t15TGPRmqawzo-q55gUA@mail.gmail.com>

On 17 January 2016 at 23:37, boB Stepp <robertvstepp at gmail.com> wrote:
> 1)  Is using pkgutil.get_data() the way I should be reading my data files?

Generally yes, although it may be unnecessary. The main purpose of
pkgutil.get_data is to transparently handle the case where your
packages are imported from a zip file. The initial motivation for this
was setuptools eggs but there are other reasons you might want to
import your code from a zip file.

> 2)  The book says that this method will return "...a byte string
> containing the raw contents of the file." (From page 409) Can I do all
> of the normal things on this returned object that I could do if I used
> "with open..."?  Is it iterable?  [OK, I confess up front.  I am being
> lazy here.  I *could* create a file and try this out to see what
> happens, so I will understand if I get chided (or worse!).]

Just try it out :)

> 3)  Should I be using relative imports in my projects?

I don't use relative imports. The supposed advantage of relative
imports is that you can easily move a package to a different location
in the import hierarchy. For example if you have a package called
stuff and the modules inside stuff use relative imports to access each
other then you could move stuff inside another package called things
and have it be a package called things.stuff without needing to change
the import lines.

Personally I think that fixing up a few import lines is no big deal.
I'd rather choose exactly where things is going to go in the import
hierarchy and be done with it. If I need to move it into another
package then I'll have to change the import statements inside things
but that's easy to do. So I prefer absolute imports since they're less
ambiguous.

--
Oscar

From bachir_bac at yahoo.com  Mon Jan 18 05:31:49 2016
From: bachir_bac at yahoo.com (Bachir Bachir)
Date: Mon, 18 Jan 2016 10:31:49 +0000 (UTC)
Subject: [Tutor] pandas data frame
In-Reply-To: <n7iei3$o49$1@ger.gmane.org>
References: <n7iei3$o49$1@ger.gmane.org>
Message-ID: <1384981652.6407201.1453113109860.JavaMail.yahoo@mail.yahoo.com>

?Hi Peter?Thank you much for your help its very appreciated ,bellow is an example of what i need. ? The main dataframe ( need to be separated into separate dataframes ?). The desired dataframes output correctly as i need it to be,Thanks much
Main dataframendx ? ? V_id ? ? ? ?Average ? ? ? ?Mean ? ? ? Peak?0 ? ? ? ? 1 ? ? ? ? ? ? ?3 ? ? ? ? ? ? ? ? ? ? ?2 ? ? ? ? ?51 ? ? ? ? 2 ? ? ? ? ? ? ?2 ? ? ? ? ? ? ? ? ? ? ?1 ? ? ? ? ?62 ? ? ? ? 3 ? ? ? ? ? ? ?4 ? ? ? ? ? ? ? ? ? ? 1 ? ? ? ? ?83 ? ? ? ? 1 ? ? ? ? ? ? ?2 ? ? ? ? ? ? ? ? ? ? ?2 ? ? ? ? ?74 ? ? ? ? 2 ? ? ? ? ? ? ?3 ? ? ? ? ? ? ? ? ? ? 3 ? ? ? ? ?65 ? ? ? ? 3 ? ? ? ? ? ? ?5 ? ? ? ? ? ? ? ? ? ? 3 ? ? ? ? ?46 ? ? ? ? 1 ? ? ? ? ? ? ?1 ? ? ? ? ? ? ? ? ? ? 1 ? ? ? ? ?87 ? ? ? ? 2 ? ? ? ? ? ? ?2 ? ? ? ? ? ? ? ? ? ? 5 ? ? ? ? ?108 ? ? ? ? 3 ? ? ? ? ? ? ?5 ? ? ? ? ? ? ? ? ? ? 5 ? ? ? ? ?99 ? ? ? ? 1 ? ? ? ? ? ? ?2 ? ? ? ? ? ? ? ? ? ? 5 ? ? ? ? ?1010 ? ? ? ?2 ? ? ? ? ? ? 5 ? ? ? ? ? ? ? ? ? ? 5 ? ? ? ? ?911 ? ? ? ?3 ? ? ? ? ? ? 4 ? ? ? ? ? ? ? ? ? ? 3 ? ? ? ? ?10
Dataframe-1ndx ? ? V_id ? Average ? ? Mean ? ?Peak?0 ? ? ? ? 1 ? ? ? ? ? 3 ? ? ? ? ? ? ?2 ? ? ? ? ?53 ? ? ? ? 1 ? ? ? ? ? 2 ? ? ? ? ? ? ?2 ? ? ? ? ?76 ? ? ? ? 1 ? ? ? ? ? 1 ? ? ? ? ? ? ?1 ? ? ? ? ?89 ? ? ? ? 1 ? ? ? ? ? 2 ? ? ? ? ? ? ?5 ? ? ? ? ?10
Dataframe-2ndx ? ? V_id ? ?Averag ? ?Mean ? ? Peak?1 ? ? ? ? ?2 ? ? ? ? ? 2 ? ? ? ? ? ? ?1 ? ? ? ? ?64 ? ? ? ? ?2 ? ? ? ? ? 3 ? ? ? ? ? ? ?3 ? ? ? ? ?67 ? ? ? ? ?2 ? ? ? ? ? 2 ? ? ? ? ? ? ?5 ? ? ? ? ?1010 ? ? ? ?2 ? ? ? ? ? 5 ? ? ? ? ? ? ?5 ? ? ? ? ?9
Dataframe-3ndx ? ? V_id ? ?Average ? ?Mean ? ?Peak?2 ? ? ? ? 3 ? ? ? ? ? 4 ? ? ? ? ? ? ?1 ? ? ? ? ?85 ? ? ? ? 3 ? ? ? ? ? 5 ? ? ? ? ? ? ?3 ? ? ? ? ?48 ? ? ? ? 3 ? ? ? ? ? 5 ? ? ? ? ? ? ?5 ? ? ? ? ?911 ? ? ? ?3 ? ? ? ? ? 4 ? ? ? ? ? ? ?3 ? ? ? ? ?10
Bachir

 

    On Monday, January 18, 2016 11:25 AM, Peter Otten <__peter__ at web.de> wrote:
 

 Bachir Bachir wrote:

[Bachir, please send your mails to the list, not to me. That way you 
increase the likelihood to get a good answer]

>>? ? On Sunday, January 17, 2016 4:20 PM, Peter Otten <__peter__ at web.de>
>> wrote:
>>? Bachir Bachir via Tutor wrote:
>>> Hello EverybodyI need to sort a dataframe according to a specific column
>>> the create new dataframes according to the sorted columns each new
>>> created dataframe should contain the list(set('the sorted element)any
>>> help please, i am new in python and pandas thanks muchBachir
>> 
>> Like this?
>> 
>> >>> df
>> 
>>? foo bar
>> 0? 3.3? b
>> 1? 2.2? b
>> 2? 7.5? a
>> 3? 1.1? c
>> 4? 4.7? a
>> 
>> [5 rows x 2 columns]
>> 
>> >>> df.groupby("bar").sum()
>> 
>>? ? ? foo
>> bar? ? 
>> a? ? 12.2
>> b? ? 5.5
>> c? ? 1.1
>> 
>> [3 rows x 1 columns]

> Thanks Peter ,This way but i? want keep all the table elements only
> separate them? by a single column 

I'm sorry, I don't understand what you mean by "separate them by a single 
column". Can you give an example? What exactly should the sorted version of

>> >>> df
>> 
>>? foo bar
>> 0? 3.3? b
>> 1? 2.2? b
>> 2? 7.5? a
>> 3? 1.1? c
>> 4? 4.7? a
>> 
>> [5 rows x 2 columns]

look like?

_______________________________________________
Tutor maillist? -? Tutor at python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


  

From deepaknedumpilly at gmail.com  Mon Jan 18 08:06:37 2016
From: deepaknedumpilly at gmail.com (Deepak Nn)
Date: Mon, 18 Jan 2016 18:36:37 +0530
Subject: [Tutor] Fwd:
In-Reply-To: <CAGSR2fGgk92SW0KKzPyX5WOTcTnNtToEjZ_tEVOzxH8EABNM3A@mail.gmail.com>
References: <CAGSR2fGbWt7bJYdein5Cw5+oBMw2Ad5L5j_DkK-nUWqW8TduJA@mail.gmail.com>
 <CAGSR2fFrONeTJJ_zQieLgni+kmmSMPMpwzBqrho+bHEzHiMQtw@mail.gmail.com>
 <CAGSR2fGgk92SW0KKzPyX5WOTcTnNtToEjZ_tEVOzxH8EABNM3A@mail.gmail.com>
Message-ID: <CAGSR2fHfbb1DnPHRiCEqYNBSBNNR7bS2yeUjCSe7Get1YgxAEA@mail.gmail.com>

---------- Forwarded message ----------
From: Deepak Nn <deepaknedumpilly at gmail.com>
Date: Mon, Jan 18, 2016 at 6:27 PM
Subject: Re:
To: tutor-owner at python.org


Please provide a python program to run a program (.exe) and get Hash
*exactly* as :

 160 106 182 190 228 64 68 207 248 109 67 88 41 .The username to be
used is admin
.The *password* is what to be found out .The hash provided is of the
correct password .Mostly the password will be *13 char long *and has a
small chance of being all alpha characters .*GOOGLE DOES NOT ALLOW TO SENT
.EXE FILE .*

Provide the where is change i have to make in the program to run the .exe
file that i have in my computer .

Since i can't sent the .exe file .i am sending the screen shorts of the
program .*The text file of the program is included and few screenshots of
 the the the result of the program is also included* .

From sjeik_appie at hotmail.com  Mon Jan 18 10:12:08 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Mon, 18 Jan 2016 15:12:08 +0000
Subject: [Tutor] str.strip strange result...?
In-Reply-To: <CAGZAPF7Jn40PFntisMnRgGMymp-udAJDz+49Pg9bC2Rd3nsZjg@mail.gmail.com>
References: <CACvW2fxwvHHKW0WOHDiLSuGSmv6_uTK82HYUkQtgsCRP8fqC4Q@mail.gmail.com>,
 <n7b787$ktu$1@ger.gmane.org>
 <DUB123-W474C8338133C5BB6E5B14083CF0@phx.gbl>,
 <CAGZAPF7Jn40PFntisMnRgGMymp-udAJDz+49Pg9bC2Rd3nsZjg@mail.gmail.com>
Message-ID: <DUB123-W2582DEAC3E4DBEA609D56D83C00@phx.gbl>

 
> From: dyoo at hashcollision.org
> Date: Mon, 18 Jan 2016 00:29:41 -0800
> Subject: Re: [Tutor] str.strip strange result...?
> To: sjeik_appie at hotmail.com
> CC: __peter__ at web.de; tutor at python.org
> 
> > Not sure which one is faster, but in this case I actually find a regex more readable (!):
> >>>> re.sub(r"_1$", "", "V01_1")
> > 'V01'
> 
> 
> Hi Albert-Jan Roskam,
> 
> Here's a minor counterpoint to using regexes here: they're sometimes a
> bit too powerful. Yes, certainly: https://xkcd.com/208/ :-)
> 
> In this situation, a regex approach might be troublesome if parts of
> the suffix can be interpreted as regular expression meta-characters.
> For example, if we were trying to strip out a suffix like
> 
>     ".exe"
> 
> then if we were to try to do this with regexes, we might forget that
> "." is a meta-character that acts as a wildcard for any single
> character.  Therefore, a regexp-based solution for the general
> suffix-removal problem is complicated because we'd need consider
> escaping certain characters.  Not that this would be hard, but that
> it's yet another detail we have to keep in our heads.
Agreed. Apart from backslashes, re.escape could also be really handy. But regexes are scary and intimidating for those who have never seen them before.
> Another disadvantage is that, if the suffix is dynamically determined,
> then there's an additional cost in "compiling" the regular expression:
> building the pattern-matching machinery doesn't come for free.
re.compile helps, though it always bothers me that I "need" to put them as a global in the top-level of my module (along with collections.namedtuple).Unless I use a class, where __init__ is a good place.
 
> For those reasons, I think the regexp approach here is a little bit of
> overkill.  This kind of tradeoff is the sort of thing that reference
> documentation will probably not mention.   That's why this list is
> here, to share the experience of using these systems with other
> beginners and learners.  Regexes are still very useful: just be aware
> that they have sharp edges.
 One other downside is that it takes only a couple of days/beers before one forgets the regex. But re.VERBOSE and named groups help a LOT.
 
> Good luck to you!
 		 	   		  

From anshu.kumar726 at gmail.com  Mon Jan 18 11:01:41 2016
From: anshu.kumar726 at gmail.com (Anshu Kumar)
Date: Mon, 18 Jan 2016 21:31:41 +0530
Subject: [Tutor] Simultaneous read and write on file
Message-ID: <CAMFBua64-h4w4fVFkW4Z8WfDHTu0L5LxnwfKe5WWYWucaUa=Nw@mail.gmail.com>

Hello Everyone,

I try below code in python 2.7.10 to first create and write into a file and
then read and write that file but what i get is just a  file with new
content.


>>> with open('test.txt', 'wb+') as f:
...     f.write('this is test file.')
...     f.write('ok!!!')
...
>>> with open('test.txt', 'wb+') as f:
...     a_str = f.read() + 'read the file'
...     f.seek(0)
...     f.write(a_str)
...


I have read in documentation that wb+ mode is for writing and reading. Am i
using wrong mode, should i use rb+ ?

Thanks and regards,
Anshu

From sudiptod19 at gmail.com  Mon Jan 18 10:59:33 2016
From: sudiptod19 at gmail.com (sudipto manna)
Date: Mon, 18 Jan 2016 09:59:33 -0600
Subject: [Tutor] if request.method == 'GET': NameError: global name
 'request' is not defined
In-Reply-To: <20160118095545.GA92822@cskk.homeip.net>
References: <CAERL5SyGdE-aaYddNLN+N52Y+__mppsd09eT8j_q_jeP_WqbBg@mail.gmail.com>
 <20160118095545.GA92822@cskk.homeip.net>
Message-ID: <CAERL5SwriswKwqSM_GT-BYEJRRStjKvxcDGZr274Fa9kDV2Fgw@mail.gmail.com>

Thanks All.
The import request was missing and the issue was resolved upon calling that
module and initializing it.

Thanks for the guidance.

Regards,
Sudipto Manna

On Mon, Jan 18, 2016 at 3:55 AM, Cameron Simpson <cs at zip.com.au> wrote:

> On 17Jan2016 20:13, sudipto manna <sudiptod19 at gmail.com> wrote:
>
>> Here is the code snippet:
>>
>> File#FlaskTest2.py
>>
>> from flask import Flask
>>
>
> You need to import "request" from flask as well:
>
>  from flask import Flask, request
>
> For others on this list: the Flask framework presents the current web
> request as a thread local global called "request"; simplifies writing
> handlers as the request information is readily available without having to
> pass it through function calls.
>
> Cheers,
> Cameron Simpson <cs at zip.com.au>
>

From alan.gauld at btinternet.com  Mon Jan 18 11:23:33 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 18 Jan 2016 16:23:33 +0000
Subject: [Tutor] Password decoding (was Re: Fwd:)
In-Reply-To: <CAGSR2fHfbb1DnPHRiCEqYNBSBNNR7bS2yeUjCSe7Get1YgxAEA@mail.gmail.com>
References: <CAGSR2fGbWt7bJYdein5Cw5+oBMw2Ad5L5j_DkK-nUWqW8TduJA@mail.gmail.com>
 <CAGSR2fFrONeTJJ_zQieLgni+kmmSMPMpwzBqrho+bHEzHiMQtw@mail.gmail.com>
 <CAGSR2fGgk92SW0KKzPyX5WOTcTnNtToEjZ_tEVOzxH8EABNM3A@mail.gmail.com>
 <CAGSR2fHfbb1DnPHRiCEqYNBSBNNR7bS2yeUjCSe7Get1YgxAEA@mail.gmail.com>
Message-ID: <n7j3i5$n5k$1@ger.gmane.org>

On 18/01/16 13:06, Deepak Nn wrote:

> Please provide a python program to run a program (.exe) and get Hash
> *exactly* as :
> 
>  160 106 182 190 228 64 68 207 248 109 67 88 41 .The username to be
> used is admin
> .The *password* is what to be found out .The hash provided is of the
> correct password .Mostly the password will be *13 char long *and has a
> small chance of being all alpha characters .*GOOGLE DOES NOT ALLOW TO SENT
> .EXE FILE .*
> 
> Provide the where is change i have to make in the program to run the .exe
> file that i have in my computer .
> 
> Since i can't sent the .exe file .i am sending the screen shorts of the
> program .*The text file of the program is included and few screenshots of
>  the the the result of the program is also included* .

The list server does not generally let attachments through.

Also, we will not do your homework for you, although we
will try to help you do it. Please show us what you have
tried so far and any error messages (in full).

Ask specific questions and you will get specific help.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Mon Jan 18 11:29:44 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 18 Jan 2016 16:29:44 +0000
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <CAMFBua64-h4w4fVFkW4Z8WfDHTu0L5LxnwfKe5WWYWucaUa=Nw@mail.gmail.com>
References: <CAMFBua64-h4w4fVFkW4Z8WfDHTu0L5LxnwfKe5WWYWucaUa=Nw@mail.gmail.com>
Message-ID: <n7j3to$tnr$1@ger.gmane.org>

On 18/01/16 16:01, Anshu Kumar wrote:
> Hello Everyone,
> 
> I try below code in python 2.7.10 to first create and write into a file and
> then read and write that file but what i get is just a  file with new
> content.
> 
> 
>>>> with open('test.txt', 'wb+') as f:
> ...     f.write('this is test file.')
> ...     f.write('ok!!!')
> ...
>>>> with open('test.txt', 'wb+') as f:
> ...     a_str = f.read() + 'read the file'
> ...     f.seek(0)
> ...     f.write(a_str)
> ...

You called f.seek(0) which puts the cursor right back to the start
of the file. Try writing without using the f.seek().

But since you only want to append, it would be better and safer to use
append mode instead.

The + modes are deceptively appealing but they are full of dangers
for precisely the reasons you have discovered(*). You very rarely
need them and you are better opening/closing the file and
using explicit modes to read/write.

(*)Another problem, apart from the risks of overwriting your
data,  is that the + modes will lock the file even while you
are just reading it, which might cause a problem with shared
files)

> I have read in documentation that wb+ mode is for writing and reading. Am i
> using wrong mode, should i use rb+ ?

No, you should probably be separately using 'r' to read and 'a'
to append.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From dyoo at hashcollision.org  Mon Jan 18 13:51:18 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 18 Jan 2016 10:51:18 -0800
Subject: [Tutor] if request.method == 'GET': NameError: global name
 'request' is not defined
In-Reply-To: <CAERL5SwriswKwqSM_GT-BYEJRRStjKvxcDGZr274Fa9kDV2Fgw@mail.gmail.com>
References: <CAERL5SyGdE-aaYddNLN+N52Y+__mppsd09eT8j_q_jeP_WqbBg@mail.gmail.com>
 <20160118095545.GA92822@cskk.homeip.net>
 <CAERL5SwriswKwqSM_GT-BYEJRRStjKvxcDGZr274Fa9kDV2Fgw@mail.gmail.com>
Message-ID: <CAGZAPF6FqKXYX9vZ9iOf64bP38rDF+nRDRjom9H_6TviuzS7XQ@mail.gmail.com>

On Mon, Jan 18, 2016 at 7:59 AM, sudipto manna <sudiptod19 at gmail.com> wrote:
> Thanks All.
> The import request was missing and the issue was resolved upon calling that
> module and initializing it.


Hi Sudipto Manna,


One last thing before you go on.  Do a small retrospective: now that
you know why things weren't working, look back at the original error
message again.


#######################################################
 File "/Users/sudiptomanna/fullstack/vagrant/PythonData/FlaskTest2.py",
line 8, in puppiesFunction

    if request.method == 'GET':

NameError: global name 'request' is not defined
#######################################################


Does the message in the error message make more sense to you now?



Best of wishes!

From dyoo at hashcollision.org  Mon Jan 18 13:59:26 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 18 Jan 2016 10:59:26 -0800
Subject: [Tutor] Fwd:
In-Reply-To: <CAGSR2fHfbb1DnPHRiCEqYNBSBNNR7bS2yeUjCSe7Get1YgxAEA@mail.gmail.com>
References: <CAGSR2fGbWt7bJYdein5Cw5+oBMw2Ad5L5j_DkK-nUWqW8TduJA@mail.gmail.com>
 <CAGSR2fFrONeTJJ_zQieLgni+kmmSMPMpwzBqrho+bHEzHiMQtw@mail.gmail.com>
 <CAGSR2fGgk92SW0KKzPyX5WOTcTnNtToEjZ_tEVOzxH8EABNM3A@mail.gmail.com>
 <CAGSR2fHfbb1DnPHRiCEqYNBSBNNR7bS2yeUjCSe7Get1YgxAEA@mail.gmail.com>
Message-ID: <CAGZAPF4xTLy2qwqscQRrZce7-4n5=KDWB=asSTYVybnm0uUW2A@mail.gmail.com>

> Please provide a python program to run a program (.exe) and get Hash
> *exactly* as :
>
>  160 106 182 190 228 64 68 207 248 109 67 88 41 .The username to be
> used is admin
> .The *password* is what to be found out .The hash provided is of the
> correct password .Mostly the password will be *13 char long *and has a
> small chance of being all alpha characters .


Hi Deepak,

This doesn't seem like a beginner-level question.  If I had a guess,
it sounds more like something out of a shady rent-a-coder kind of
thing.

Unfortunately, I don't think we can help with this.  Even if we did
have the technical expertise, I still don't think we should help on
this in the first place.  If I'm understanding the question correctly,
you're asking for brute password breaking, which goes against most
professional codes of conduct.  Example:
https://www.acm.org/about-acm/acm-code-of-ethics-and-professional-conduct.

From cs at zip.com.au  Mon Jan 18 15:43:18 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 19 Jan 2016 07:43:18 +1100
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <n7j3to$tnr$1@ger.gmane.org>
References: <n7j3to$tnr$1@ger.gmane.org>
Message-ID: <20160118204318.GA1200@cskk.homeip.net>

On 18Jan2016 16:29, ALAN GAULD <alan.gauld at btinternet.com> wrote:
>On 18/01/16 16:01, Anshu Kumar wrote:
>> I try below code in python 2.7.10 to first create and write into a file and
>> then read and write that file but what i get is just a  file with new
>> content.
>>
>>
>>>>> with open('test.txt', 'wb+') as f:
>> ...     f.write('this is test file.')
>> ...     f.write('ok!!!')
>> ...
>>>>> with open('test.txt', 'wb+') as f:
>> ...     a_str = f.read() + 'read the file'
>> ...     f.seek(0)
>> ...     f.write(a_str)
>> ...
>
>You called f.seek(0) which puts the cursor right back to the start
>of the file. Try writing without using the f.seek().

Agreed.

>But since you only want to append, it would be better and safer to use
>append mode instead.
>
>The + modes are deceptively appealing but they are full of dangers
>for precisely the reasons you have discovered(*). You very rarely
>need them and you are better opening/closing the file and
>using explicit modes to read/write.

But if he wants to mix the modes, he certainly should be experimenting. Having 
a file open for read and write is sometimes useful; I do it myself in certain 
circumstances.

Tip for new players: if you do any .write()s, remember to do a .flush() before 
doing a seek or a read - unlike the C stdio library where the flush is 
automatic in Python the io classes require you to flush written data if you 
read or seek. (You don't have to flush before close, close does that for you.)

>(*)Another problem, apart from the risks of overwriting your
>data,  is that the + modes will lock the file even while you
>are just reading it, which might cause a problem with shared
>files)

Only on Windows. On UNIX everything is fine unless you go out of your way to 
make things harder with locking.

>> I have read in documentation that wb+ mode is for writing and reading. Am i
>> using wrong mode, should i use rb+ ?
>
>No, you should probably be separately using 'r' to read and 'a'
>to append.

Disagree. As far as his question goes, "wb+" is a correct mode for what he is 
trying. Whether it is a sensible approach depends very much on what he is doing 
with his files.

He _may_ be safer opening the file twice - once for "rb" and once for "ab" - 
because he does not have to juggle the modes, but it isn't necessarily what he 
wants.

Cheers,
Cameron Simpson <cs at zip.com.au>

From alan.gauld at btinternet.com  Mon Jan 18 16:07:49 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 18 Jan 2016 21:07:49 +0000
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <20160118204318.GA1200@cskk.homeip.net>
References: <n7j3to$tnr$1@ger.gmane.org>
 <20160118204318.GA1200@cskk.homeip.net>
Message-ID: <n7jk75$iau$1@ger.gmane.org>

On 18/01/16 20:43, Cameron Simpson wrote:

>> The + modes are deceptively appealing but they are full of dangers
>> for precisely the reasons you have discovered(*). You very rarely
>> need them and you are better opening/closing the file and
>> using explicit modes to read/write.
> 
> But if he wants to mix the modes, he certainly should be experimenting. Having 
> a file open for read and write is sometimes useful; I do it myself in certain 
> circumstances.

Yes and so have I. Maybe twice in 30 years of programming.
It's sometimes necessary but it's much, much harder to get
right and very easy to get wrong, usually with data corruption
as a result.

So for a beginner I would never encourage it. For an
experienced programmer sure' if there is no other option
(and especially if you have fixed size records where
things get easier).

> Tip for new players: if you do any .write()s, remember to do a .flush() before 
> doing a seek or a read

That's exactly my point. There are so many things you have
to do extra when working in mixed mode. Too easy to treat
things like normal mode files and get it wrong. Experts
can do it and make it work, but mostly it's just not needed.

> Disagree. As far as his question goes, "wb+" is a correct mode for what he is 
> trying. Whether it is a sensible approach depends very much on what he is doing 
> with his files.

I'm not sure we know what he(?) is trying.
We only know he successfully overwrote his data and
that apparently was not his intention. There are use
cases where it makes sense but in most cases you can
get by just fine without.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From __peter__ at web.de  Mon Jan 18 16:29:02 2016
From: __peter__ at web.de (Peter Otten)
Date: Mon, 18 Jan 2016 22:29:02 +0100
Subject: [Tutor] Simultaneous read and write on file
References: <CAMFBua64-h4w4fVFkW4Z8WfDHTu0L5LxnwfKe5WWYWucaUa=Nw@mail.gmail.com>
Message-ID: <n7jleu$5af$1@ger.gmane.org>

Anshu Kumar wrote:

> Hello Everyone,
> 
> I try below code in python 2.7.10 to first create and write into a file
> and
> then read and write that file but what i get is just a  file with new
> content.
> 
> 
>>>> with open('test.txt', 'wb+') as f:
> ...     f.write('this is test file.')
> ...     f.write('ok!!!')
> ...
>>>> with open('test.txt', 'wb+') as f:
> ...     a_str = f.read() + 'read the file'
> ...     f.seek(0)
> ...     f.write(a_str)
> ...
> 
> 
> I have read in documentation that wb+ mode is for writing and reading. Am
> i using wrong mode, should i use rb+ ?

Quoting https://docs.python.org/2.7/library/functions.html#open

"""
note that 'w+' truncates the file.
"""

That's why you lose the file's current content, and, yes, "r+b" would avoid 
that.


From __peter__ at web.de  Mon Jan 18 16:51:20 2016
From: __peter__ at web.de (Peter Otten)
Date: Mon, 18 Jan 2016 22:51:20 +0100
Subject: [Tutor] pandas data frame
References: <n7iei3$o49$1@ger.gmane.org>
 <1384981652.6407201.1453113109860.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <n7jmop$ol7$1@ger.gmane.org>

Bachir Bachir via Tutor wrote:

> Hi Peter Thank you much for your help its very appreciated ,bellow is an
> example of what i need.   The main dataframe ( need to be separated into
> separate dataframes  ). The desired dataframes output correctly as i need
> it to be,Thanks much Main dataframendx     V_id        Average        Mean
>       Peak 0         1              3                      2          51  
>       2              2                      1          62         3       
>       4                     1          83         1              2        
>              2          74         2              3                     3 
>         65         3              5                     3          46     
>    1              1                     1          87         2           
>   2                     5          108         3              5           
>          5          99         1              2                     5     
>     1010        2             5                     5          911       
> 3             4                     3          10 Dataframe-1ndx     V_id 
>  Average     Mean    Peak 0         1           3              2         
> 53         1           2              2          76         1           1 
>             1          89         1           2              5          10
> Dataframe-2ndx     V_id    Averag    Mean     Peak 1          2          
> 2              1          64          2           3              3        
>  67          2           2              5          1010        2          
> 5              5          9 Dataframe-3ndx     V_id    Average    Mean   
> Peak 2         3           4              1          85         3         
>  5              3          48         3           5              5        
>  911        3           4              3          10 Bachir

I see that you got an answer over at comp.python.pydata where you managed to 
get the format a bit more readable ;)

I recommend that you go with Goyo's answer, but instead of the hacky use of
locals() 
 
> >>> locals().update((('dataframe_{}'.format(k), grouper.get_group(k))
> ...                  for k in grouper.groups))

I recommend that you put the groups into a normal dict

groups = {k: g.get_group(k) for k in g.groups}

and then access the partial data frames with

v_id = 2
groups[v_id]


From eryksun at gmail.com  Mon Jan 18 17:42:52 2016
From: eryksun at gmail.com (eryk sun)
Date: Mon, 18 Jan 2016 16:42:52 -0600
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <CAMFBua64-h4w4fVFkW4Z8WfDHTu0L5LxnwfKe5WWYWucaUa=Nw@mail.gmail.com>
References: <CAMFBua64-h4w4fVFkW4Z8WfDHTu0L5LxnwfKe5WWYWucaUa=Nw@mail.gmail.com>
Message-ID: <CACL+1asDoucPunrHxMNfUa1ntRhEpbD1HAaJCzGMkSyQUde4xw@mail.gmail.com>

On Mon, Jan 18, 2016 at 10:01 AM, Anshu Kumar <anshu.kumar726 at gmail.com> wrote:
> I have read in documentation that wb+ mode is for writing and reading. Am i
> using wrong mode, should i use rb+ ?

Use w+ to create a new file, opened with read and write access. Use r+
to open an existing file with read and write access. Unlike w+, r+
does not truncate the file and will not create a new file.

To clarify what Alan said, the "+" modes do not 'lock' the file. On
Windows you may experience a sharing violation if another program
doesn't share write access. This isn't quite the same as a file lock.
All open references to a file have to agree on read, write, and delete
sharing. For example, when Notepad saves a file, it tries to open a
handle that shares only read access. If this open succeeds, Notepad
knows that it's the only writer. (It doesn't care if there are
multiple readers.) On the other hand, if some other program has the
file open with write or delete access, Notepad's open will fail with a
sharing violation. Then it displays a message box saying it can't save
the file because another process has it open, and it offers to save
the file using a different name.

FYI, in terms of POSIX open flags [1], the file mode gets mapped as follows:

      |       flags       |   no +   |   +
    -----------------------------------------
    r |                   | O_RDONLY | O_RDWR
    w | O_CREAT, O_TRUNC  | O_WRONLY | O_RDWR
    a | O_CREAT, O_APPEND | O_WRONLY | O_RDWR

Python 3 only:

    x | O_CREAT, O_EXCL   | O_WRONLY | O_RDWR

In terms of Windows open dispositions and access modes [2], the file
mode gets mapped as follows:

      |  disposition  |      no +     |            +
    ---------------------------------------------------------------
    r | OPEN_EXISTING | GENERIC_READ  | GENERIC_READ, GENERIC_WRITE
    w | CREATE_ALWAYS | GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE
    a | OPEN_ALWAYS   | GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE

Python 3 only:

    x | CREATE_NEW    | GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE

O_APPEND entails an implicit seek to the end of a file before every
write. Note that Python uses the Windows CRT for file access, which
doesn't use the native FILE_APPEND_DATA access to implement O_APPEND.
Instead it opens the file for generic writing and manually updates the
file pointer. This may cause problems if the security descriptor only
allows appending to a file. In this case you'd have to call CreateFile
directly (e.g. via ctypes or win32api).

[1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html
[2]: https://msdn.microsoft.com/en-us/library/aa363858

From ma7841 at truman.edu  Mon Jan 18 20:15:11 2016
From: ma7841 at truman.edu (Michael Appiah Boachie)
Date: Mon, 18 Jan 2016 17:15:11 -0800
Subject: [Tutor] An Advice to a dedicated beginner
Message-ID: <6EA8C8C9-5973-4EA8-8B02-D2FE3B4EE916@truman.edu>

Hello Tutors,

I am a programming beginner. Throughout college, I have not properly grasp anything the professor?s teach but when i sat on my own with a couple of materials and videos, I have become very good at the basics of python and completing various beginner courses and projects with ease but however I have run into some kind of ?beginner wall? where I don?t know where or what to take on next. This is killing my excitement. I think this isn?t something new to experienced programmers to hear. That?s why I am asking for help. Please any advice would help a dedicated one here. I don?t know what to do next with the knowledge I have acquired. People keep saying ?get into open source? , ?do that and that?. I wish they actually knew how someone like me feel. There are so many videos, articles and materials to get you to know basics and also become a top expert but almost nothing on how to transition into that. That?s exactly how I?m feeling. 

Hoping to receive some kind words.

Michael

From alan.gauld at btinternet.com  Mon Jan 18 20:44:52 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 19 Jan 2016 01:44:52 +0000
Subject: [Tutor] An Advice to a dedicated beginner
In-Reply-To: <6EA8C8C9-5973-4EA8-8B02-D2FE3B4EE916@truman.edu>
References: <6EA8C8C9-5973-4EA8-8B02-D2FE3B4EE916@truman.edu>
Message-ID: <n7k4ek$6fj$1@ger.gmane.org>

On 19/01/16 01:15, Michael Appiah Boachie wrote:
> I am a programming beginner. ...
> when i sat on my own with a couple of materials and videos,
> I have become very good at the basics of python and
> completing various beginner courses and projects with ease 
> but however I have run into some kind of ?beginner wall?
> where I don?t know where or what to take on next.

The first thing to do is to find a concept or project
that you want to build. We can't really help you with
that, it has to come from you. But maybe there is a
routine task you perform that you could automate?

Say, renaming photos that you load from a camera?
Or bulk conversion of music files between formats?
Or you have a spreadsheet with some macros that you
could convert to an application, maybe using a database?
Or maybe it's a game that you like? Or you could
generate reports from some data source? Maybe from
a web site?

Once you have a project, you need to decide how to build
it, and what modules you will be using.

<PLUG>
That second action is where my last book comes in.
Python Projects is targeted at people like you, who know
the basic language, but now want to do something with it.
It covers: interacting with the OS, managing data,
building UIs and web sites and using the Python
test tools.

I try to avoid too much self promotion on this list
but it does sound like you might benefit from this one.
</PLUG>

What it doesn't cover much is how to design a program,
that's a much more abstract concept and there are
whole books on that topic. But, for a beginners project,
it shouldn't be a huge issue, the important thing
is to pick something and make a start.

If you get stuck come back here.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From ben+python at benfinney.id.au  Mon Jan 18 21:04:50 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 19 Jan 2016 13:04:50 +1100
Subject: [Tutor] An Advice to a dedicated beginner
References: <6EA8C8C9-5973-4EA8-8B02-D2FE3B4EE916@truman.edu>
 <n7k4ek$6fj$1@ger.gmane.org>
Message-ID: <857fj6nx31.fsf@benfinney.id.au>

Alan Gauld <alan.gauld at btinternet.com> writes:

> On 19/01/16 01:15, Michael Appiah Boachie wrote:
> > I am a programming beginner. ...
> > when i sat on my own with a couple of materials and videos,
> > I have become very good at the basics of python and
> > completing various beginner courses and projects with ease 
> > but however I have run into some kind of ?beginner wall?
> > where I don?t know where or what to take on next.
>
> The first thing to do is to find a concept or project
> that you want to build. We can't really help you with
> that, it has to come from you. But maybe there is a
> routine task you perform that you could automate?

That's a worthwhile recommendation.

I usually recommend something different: Pick an *existing* code base
that you're already using ? Python is used in a great many tools that
you probably rely on, so this shouldn't be difficult ? and get the
source code. Find small improvements to make, discuss them with peers
and on this forum, and with the developers of that code base.

Making small improvements to something you already use will avoid the
?blank slate? paralysis from having too many options be motivational,
while also being very motivational: you already can think of
improvements you would like to make, and getting them implemented will
be very satisfying.

It also focusses on an often-dismissed aspect of learning to program:
you need to program *with other people*, and your code is a means of
communicating with those people. Learning how to do that, and learning
(by doing) that there is no shame in starting small and unskilled, is a
necessary part of the craft.

-- 
 \           ?The long-term solution to mountains of waste is not more |
  `\      landfill sites but fewer shopping centres.? ?Clive Hamilton, |
_o__)                                                _Affluenza_, 2005 |
Ben Finney


From cs at zip.com.au  Mon Jan 18 22:55:47 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 19 Jan 2016 14:55:47 +1100
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <n7jk75$iau$1@ger.gmane.org>
References: <n7jk75$iau$1@ger.gmane.org>
Message-ID: <20160119035547.GA27237@cskk.homeip.net>

On 18Jan2016 21:07, ALAN GAULD <alan.gauld at btinternet.com> wrote:
>On 18/01/16 20:43, Cameron Simpson wrote:
>>> The + modes are deceptively appealing but they are full of dangers
>>> for precisely the reasons you have discovered(*). You very rarely
>>> need them and you are better opening/closing the file and
>>> using explicit modes to read/write.
>>
>> But if he wants to mix the modes, he certainly should be experimenting. Having
>> a file open for read and write is sometimes useful; I do it myself in certain
>> circumstances.
>
>Yes and so have I. Maybe twice in 30 years of programming.
>It's sometimes necessary but it's much, much harder to get
>right and very easy to get wrong, usually with data corruption
>as a result.

I may have done it a little more than that; I agree it is very rare. I may be 
biased because I was debugging exactly this last week. (Which itself is an 
argument against mixed rerad/write with one file - it was all my own code and I 
spent over a day chasing this because I was looking in the wrong spot).

>So for a beginner I would never encourage it. For an
>experienced programmer sure' if there is no other option
>(and especially if you have fixed size records where
>things get easier).
>
>> Tip for new players: if you do any .write()s, remember to do a .flush() before
>> doing a seek or a read
>
>That's exactly my point. There are so many things you have
>to do extra when working in mixed mode. Too easy to treat
>things like normal mode files and get it wrong. Experts
>can do it and make it work, but mostly it's just not needed.

Yes. You're write - for simplicity and reliability two distinct open file 
instances are much easier.

>> Disagree. As far as his question goes, "wb+" is a correct mode for what he is
>> trying. Whether it is a sensible approach depends very much on what he is doing
>> with his files.
>
>I'm not sure we know what he(?) is trying.
>We only know he successfully overwrote his data and
>that apparently was not his intention. There are use
>cases where it makes sense but in most cases you can
>get by just fine without.

Yeah. I think I was more narked by your not answering his "is wb+ correct" 
literally; it may be only rarely the reasonable course, but for what he was 
actually _asking_ wb+ is correct. He may not be doing the best design but as 
you say we don't know his actual use case.

For the narkiness I apologise.

Cheers,
Cameron Simpson <cs at zip.com.au>

From cs at zip.com.au  Mon Jan 18 22:57:22 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 19 Jan 2016 14:57:22 +1100
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <n7jleu$5af$1@ger.gmane.org>
References: <n7jleu$5af$1@ger.gmane.org>
Message-ID: <20160119035722.GA59381@cskk.homeip.net>

On 18Jan2016 22:29, Peter Otten <__peter__ at web.de> wrote:
>Anshu Kumar wrote:
>> I have read in documentation that wb+ mode is for writing and reading. Am
>> i using wrong mode, should i use rb+ ?
>
>Quoting https://docs.python.org/2.7/library/functions.html#open
>"""
>note that 'w+' truncates the file.
>"""
>That's why you lose the file's current content, and, yes, "r+b" would avoid
>that.

And I stand corrected; I should have paid more attention. Thanks!

Cheers,
Cameron Simpson <cs at zip.com.au>

From martin at linux-ip.net  Mon Jan 18 23:04:38 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Mon, 18 Jan 2016 20:04:38 -0800
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <CACL+1asDoucPunrHxMNfUa1ntRhEpbD1HAaJCzGMkSyQUde4xw@mail.gmail.com>
References: <CAMFBua64-h4w4fVFkW4Z8WfDHTu0L5LxnwfKe5WWYWucaUa=Nw@mail.gmail.com>
 <CACL+1asDoucPunrHxMNfUa1ntRhEpbD1HAaJCzGMkSyQUde4xw@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1601181959380.2025@znpeba.jbaqresebt.arg>


Hello,

>> I have read in documentation that wb+ mode is for writing and 
>> reading. Am i using wrong mode, should i use rb+ ?
>
>Use w+ to create a new file, opened with read and write access. Use 
>r+ to open an existing file with read and write access. Unlike w+, 
>r+ does not truncate the file and will not create a new file.

[snip]

>FYI, in terms of POSIX open flags [1], the file mode gets mapped as follows:
>
>      |       flags       |   no +   |   +
>    -----------------------------------------
>    r |                   | O_RDONLY | O_RDWR
>    w | O_CREAT, O_TRUNC  | O_WRONLY | O_RDWR
>    a | O_CREAT, O_APPEND | O_WRONLY | O_RDWR
>
>Python 3 only:
>
>    x | O_CREAT, O_EXCL   | O_WRONLY | O_RDWR
>
>In terms of Windows open dispositions and access modes [2], the file
>mode gets mapped as follows:
>
>      |  disposition  |      no +     |            +
>    ---------------------------------------------------------------
>    r | OPEN_EXISTING | GENERIC_READ  | GENERIC_READ, GENERIC_WRITE
>    w | CREATE_ALWAYS | GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE
>    a | OPEN_ALWAYS   | GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE
>
>Python 3 only:
>
>    x | CREATE_NEW    | GENERIC_WRITE | GENERIC_READ, GENERIC_WRITE

The above is a very handy chart.  Did you find this somewhere, eryk 
sun, or is this from your own knowledge and experience?  This might 
benefit an experienced Windows/POSIX user trying to understand 
open() in Python if it were available in the standard documentation.

Thanks for mapping this to common operating systems.  I had inferred 
this already, but this is a great summary.

-Martin

-- 
Martin A. Brown
http://linux-ip.net/

From martin at linux-ip.net  Mon Jan 18 23:41:10 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Mon, 18 Jan 2016 20:41:10 -0800
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <20160119035547.GA27237@cskk.homeip.net>
References: <n7jk75$iau$1@ger.gmane.org>
 <20160119035547.GA27237@cskk.homeip.net>
Message-ID: <alpine.LSU.2.11.1601181958090.2025@znpeba.jbaqresebt.arg>


Hi all,

>>>> The + modes are deceptively appealing but they are full of dangers
>>>> for precisely the reasons you have discovered(*).
>
>> Yes and so have I. Maybe twice in 30 years of programming. It's 
>> sometimes necessary but it's much, much harder to get right and 
>> very easy to get wrong, usually with data corruption as a result.

Yes.
Yes.
Yes.
Yes.
Yes.

> I may have done it a little more than that; I agree it is very 
> rare. I may be biased because I was debugging exactly this last 
> week. (Which itself is an argument against mixed rerad/write with 
> one file - it was all my own code and I spent over a day chasing 
> this because I was looking in the wrong spot).

Oh yes.  Ooof.  Today's decisions are tomorrow's albatross.

Speaking of which, I have an albatross in very good condition and 
I'm looking for a buyer.  Anybody interested in a short-tailed 
albatross with good manners.  Looking for a good home.

>> So for a beginner I would never encourage it. For an experienced 
>> programmer sure' if there is no other option (and especially if 
>> you have fixed size records where things get easier).
>>
>>> Tip for new players: if you do any .write()s, remember to do a 
>>> .flush() before doing a seek or a read
>>
>> That's exactly my point. There are so many things you have to do 
>> extra when working in mixed mode. Too easy to treat things like 
>> normal mode files and get it wrong. Experts can do it and make it 
>> work, but mostly it's just not needed.
>
> Yes. You're write - for simplicity and reliability two distinct 
> open file instances are much easier.

Yes, he's write [sic].  He writes a bunch!  ;)

[Homonyms mess me up when I'm typing, all sew.]

OK, a bit more seriously, I will add a thought or two.

Modern filesystems are beautiful.  They are fast, reliable and 
efficient.  Application software, e.g. Python, can be hairy (see the 
points of Alan and Cameron earlier in this thread).  Why not take 
advantage of filesystem atomicity, a feature guarantee to userspace 
from all (?) modern local filesystems.

Options:

  * If disk throughput is not a problem, then there's practically 
    nothing but a benefit to reading from input file, writing to 
    output file, closing both and renaming (effectively squashing 
    the original file)

      import os
      A = open('a', 'w')
      A.write('hammy')
      A.close()
      A = open('a', 'r')
      B = open('b', 'w')
      data = A.read()                 # -- processing handled
      data = data.replace('m', 'p')   #    here, until happy
      B.write(data)
      B.close()
      A.close()
      os.rename(B.name, A.name)       # -- atomic [0]

  * Alternative:  If disk throughput is a problem, this is an 
    argument for using a database system where this class of data 
    integrity problem has been solved for the application 
    developer.

I'd suggest measuring the amount of time it takes to read, rewrite 
and os.rename() the entire file before deciding you need to 
undertake the massive complexity of modifying an existing file in 
situ.

If you can avoid modifying an existing file, don't bother with it. 
You will likely bring yourself (and maybe even others) headache.  

For example, if another reader of the file comes along while you are 
performing your in situ modification magic tricks, they (and you) 
will have no guarantees about what data they will receive.  That 
will be left up to the operating system (i.e. kernel).

So, take control of the data back into your own hands by taking 
adavantage of the beauty of the filesystem.

Filesystem atomicity!

Good luck,

-Martin

 [0] Or just about as close as conceivably possible to atomic as you 
     can be guaranteed in userspace applications.

-- 
Martin A. Brown
http://linux-ip.net/

From cs at zip.com.au  Tue Jan 19 00:34:52 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 19 Jan 2016 16:34:52 +1100
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <alpine.LSU.2.11.1601181958090.2025@znpeba.jbaqresebt.arg>
References: <alpine.LSU.2.11.1601181958090.2025@znpeba.jbaqresebt.arg>
Message-ID: <20160119053452.GA30056@cskk.homeip.net>

On 18Jan2016 20:41, Martin A. Brown <martin at linux-ip.net> wrote:
>>> Yes and so have I. Maybe twice in 30 years of programming. [...]
>>
>> I may have done it a little more than that; I agree it is very
>> rare. I may be biased because I was debugging exactly this last
>> week. (Which itself is an argument against mixed rerad/write with
>> one file - it was all my own code and I spent over a day chasing
>> this because I was looking in the wrong spot).
>
>Oh yes.  Ooof.  Today's decisions are tomorrow's albatross.

Actually I have good reason to mix these in this instance, and now that it is 
debugged it is reliable and more efficient to boot.

[...]
>>>> Tip for new players: if you do any .write()s, remember to do a
>>>> .flush() before doing a seek or a read
>>>
>>> That's exactly my point. There are so many things you have to do
>>> extra when working in mixed mode. Too easy to treat things like
>>> normal mode files and get it wrong. Experts can do it and make it
>>> work, but mostly it's just not needed.
>>
>> Yes. You're write - for simplicity and reliability two distinct
>> open file instances are much easier.
>
>Yes, he's write [sic].  He writes a bunch!  ;)

Alas, I have a tendency to substitute homophones, or near homophones, when 
typing in a hurry. You'll see this in a bunch of my messages. More annoyingly, 
some are only visible when I reread a posted message instead of when I was 
proofreading prior to send.

>[Homonyms mess me up when I'm typing, all sew.]

Homonyms too.

Cheers,
Cameron Simpson <cs at zip.com.au>

From eryksun at gmail.com  Tue Jan 19 01:10:27 2016
From: eryksun at gmail.com (eryk sun)
Date: Tue, 19 Jan 2016 00:10:27 -0600
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <alpine.LSU.2.11.1601181959380.2025@znpeba.jbaqresebt.arg>
References: <CAMFBua64-h4w4fVFkW4Z8WfDHTu0L5LxnwfKe5WWYWucaUa=Nw@mail.gmail.com>
 <CACL+1asDoucPunrHxMNfUa1ntRhEpbD1HAaJCzGMkSyQUde4xw@mail.gmail.com>
 <alpine.LSU.2.11.1601181959380.2025@znpeba.jbaqresebt.arg>
Message-ID: <CACL+1atoKSVFVgSOdf2O+n9Gh_NmkdZKcLs5wEaxJe0kctFpsQ@mail.gmail.com>

On Mon, Jan 18, 2016 at 10:04 PM, Martin A. Brown <martin at linux-ip.net> wrote:
> The above is a very handy chart.  Did you find this somewhere, eryk
> sun, or is this from your own knowledge and experience?

The mapping to POSIX open flags is from a table in the POSIX fopen
spec [1] as well as the Windows CRT docs for fopen (see the equivalent
oflag table) [2]. The flags for "x" mode can be verified in the Python
3 source [3]. The "x" mode flags are also mentioned in GNU's libc docs
for fopen [4] and open-time flags [5].

The mapping from POSIX open flags to Windows create disposition and
access modes is from the CRT source for the _open function. I can't
provide a source link, but the source code is distributed with MSVC.
For VS2015 it's in lowio/open.cpp, decode_access_flags() and
decode_open_create_flags().

[1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/fopen.html
[2]: https://msdn.microsoft.com/en-us/library/yeby3zcb
[3]: https://hg.python.org/cpython/file/v3.5.1/Modules/_io/fileio.c#l302
[4]: https://www.gnu.org/software/libc/manual/html_node/Opening-Streams.html
[5]: https://www.gnu.org/software/libc/manual/html_node/Open_002dtime-Flags.html

From crusier at gmail.com  Tue Jan 19 02:53:07 2016
From: crusier at gmail.com (Crusier)
Date: Tue, 19 Jan 2016 15:53:07 +0800
Subject: [Tutor] Beautiful Soup
Message-ID: <CAC7HCj8LGM2bao1YpitBY8=9qjtXhEwsiczCHmcUzj2kKvH9kQ@mail.gmail.com>

Hi Python Tutors,

I am currently able to strip down to the string I want. However, I
have problems with the JSON script and I am not sure how to slice it
into a dictionary.

import urllib
import json
import requests

from bs4 import BeautifulSoup


url = 'https://bochk.etnet.com.hk/content/bochkweb/eng/quote_transaction_daily_history.php?code=6881\
&time=F&timeFrom=090000&timeTo=160000&turnover=S&sessionId=44c99b61679e019666f0570db51ad932&volMin=0&turnoverMin=0'

def web_scraper(url):

    response = requests.get(url)
    html = response.content
    soup = BeautifulSoup(html, 'lxml')

    stock1 = soup.findAll('script')[4].string
    stock2 = stock1.split()
    stock3 = stock2[3]
    # is stock3 sufficient to process as JSON or need further cleaning??

    text =  json.dumps(stock3)
    print(text)


web_scraper(url)

If it is possible, please give me some pointers. Thank you

Regards,
Henry

From anshu.kumar726 at gmail.com  Mon Jan 18 22:01:56 2016
From: anshu.kumar726 at gmail.com (Anshu Kumar)
Date: Tue, 19 Jan 2016 08:31:56 +0530
Subject: [Tutor] An Advice to a dedicated beginner
In-Reply-To: <6EA8C8C9-5973-4EA8-8B02-D2FE3B4EE916@truman.edu>
References: <6EA8C8C9-5973-4EA8-8B02-D2FE3B4EE916@truman.edu>
Message-ID: <CAMFBua4pLF3Z1Oqs5Mm9kq1zHwPA7s8kqXEiKY+S+iVim3va_g@mail.gmail.com>

Hey,

This sounds very usual and common. Your next step should be something that
fetches you feedback like go for job/ internship or participate in coding
contests. Try to look for some small projects on github.

Most importantly never loose zeal ....it will take you there.

Good luck
Anshu
On Jan 19, 2016 7:03 AM, "Michael Appiah Boachie" <ma7841 at truman.edu> wrote:

> Hello Tutors,
>
> I am a programming beginner. Throughout college, I have not properly grasp
> anything the professor?s teach but when i sat on my own with a couple of
> materials and videos, I have become very good at the basics of python and
> completing various beginner courses and projects with ease but however I
> have run into some kind of ?beginner wall? where I don?t know where or what
> to take on next. This is killing my excitement. I think this isn?t
> something new to experienced programmers to hear. That?s why I am asking
> for help. Please any advice would help a dedicated one here. I don?t know
> what to do next with the knowledge I have acquired. People keep saying ?get
> into open source? , ?do that and that?. I wish they actually knew how
> someone like me feel. There are so many videos, articles and materials to
> get you to know basics and also become a top expert but almost nothing on
> how to transition into that. That?s exactly how I?m feeling.
>
> Hoping to receive some kind words.
>
> Michael
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From anshu.kumar726 at gmail.com  Tue Jan 19 00:41:40 2016
From: anshu.kumar726 at gmail.com (Anshu Kumar)
Date: Tue, 19 Jan 2016 11:11:40 +0530
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <20160119053452.GA30056@cskk.homeip.net>
References: <alpine.LSU.2.11.1601181958090.2025@znpeba.jbaqresebt.arg>
 <20160119053452.GA30056@cskk.homeip.net>
Message-ID: <CAMFBua7K7VXsVLbQCXK0z9tQO6849w41m8P+_uaMMinqRbFEjA@mail.gmail.com>

Hello All,

So much Thanks for your response.

Here is my actual scenario. I have a csv file and it would already be
present. I need to read and remove some rows based on some logic. I have
written earlier two separate file opens which I think was nice and clean.

actual code:

with open(file_path, 'rb') as fr:
    for row in csv.DictReader(fr):
        #Skip for those segments which are part of overridden_ids
        if row['id'] not in overriden_ids:
            segments[row['id']] = {
                'id': row['id'],
                'attrib': json.loads(row['attrib']),
                'stl': json.loads(row['stl']),
                'meta': json.loads(row['meta']),
            }
#rewriting files with deduplicated segments
with open(file_path, 'wb') as fw:
    writer = csv.UnicodeWriter(fw)
    writer.writerow(["id", "attrib", "stl", "meta"])
    for seg in segments.itervalues():
        writer.writerow([seg['id'], json.dumps(seg["attrib"]),
json.dumps(seg["stl"]), json.dumps(seg["meta"])])


I have got review comments to improve this block by having just single
file open and minimum memory usage.


Thanks and Regards,

Anshu



On Tue, Jan 19, 2016 at 11:04 AM, Cameron Simpson <cs at zip.com.au> wrote:

> On 18Jan2016 20:41, Martin A. Brown <martin at linux-ip.net> wrote:
>
>> Yes and so have I. Maybe twice in 30 years of programming. [...]
>>>>
>>>
>>> I may have done it a little more than that; I agree it is very
>>> rare. I may be biased because I was debugging exactly this last
>>> week. (Which itself is an argument against mixed rerad/write with
>>> one file - it was all my own code and I spent over a day chasing
>>> this because I was looking in the wrong spot).
>>>
>>
>> Oh yes.  Ooof.  Today's decisions are tomorrow's albatross.
>>
>
> Actually I have good reason to mix these in this instance, and now that it
> is debugged it is reliable and more efficient to boot.
>
> [...]
>
>> Tip for new players: if you do any .write()s, remember to do a
>>>>> .flush() before doing a seek or a read
>>>>>
>>>>
>>>> That's exactly my point. There are so many things you have to do
>>>> extra when working in mixed mode. Too easy to treat things like
>>>> normal mode files and get it wrong. Experts can do it and make it
>>>> work, but mostly it's just not needed.
>>>>
>>>
>>> Yes. You're write - for simplicity and reliability two distinct
>>> open file instances are much easier.
>>>
>>
>> Yes, he's write [sic].  He writes a bunch!  ;)
>>
>
> Alas, I have a tendency to substitute homophones, or near homophones, when
> typing in a hurry. You'll see this in a bunch of my messages. More
> annoyingly, some are only visible when I reread a posted message instead of
> when I was proofreading prior to send.
>
> [Homonyms mess me up when I'm typing, all sew.]
>>
>
> Homonyms too.
>
> Cheers,
> Cameron Simpson <cs at zip.com.au>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From __peter__ at web.de  Tue Jan 19 04:00:52 2016
From: __peter__ at web.de (Peter Otten)
Date: Tue, 19 Jan 2016 10:00:52 +0100
Subject: [Tutor] Beautiful Soup
References: <CAC7HCj8LGM2bao1YpitBY8=9qjtXhEwsiczCHmcUzj2kKvH9kQ@mail.gmail.com>
Message-ID: <n7ku05$q2r$1@ger.gmane.org>

Crusier wrote:

> Hi Python Tutors,
> 
> I am currently able to strip down to the string I want. However, I
> have problems with the JSON script and I am not sure how to slice it
> into a dictionary.
> 
> import urllib
> import json
> import requests
> 
> from bs4 import BeautifulSoup
> 
> 
> url =
> 
'https://bochk.etnet.com.hk/content/bochkweb/eng/quote_transaction_daily_history.php?code=6881\
> 
&time=F&timeFrom=090000&timeTo=160000&turnover=S&sessionId=44c99b61679e019666f0570db51ad932&volMin=0&turnoverMin=0'
> 
> def web_scraper(url):
> 
>     response = requests.get(url)
>     html = response.content
>     soup = BeautifulSoup(html, 'lxml')
> 
>     stock1 = soup.findAll('script')[4].string
>     stock2 = stock1.split()
>     stock3 = stock2[3]
>     # is stock3 sufficient to process as JSON or need further cleaning??
> 
>     text =  json.dumps(stock3)
>     print(text)
> 
> 
> web_scraper(url)
> 
> If it is possible, please give me some pointers. Thank you

- You need json.loads(), not dumps() to convert text into a python data
  structure
- It looks like you have to remove a trailing ";" from stock3 for loads() to
  succeed



From alan.gauld at btinternet.com  Tue Jan 19 04:12:06 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 19 Jan 2016 09:12:06 +0000
Subject: [Tutor] Simultaneous read and write on file
In-Reply-To: <CAMFBua7K7VXsVLbQCXK0z9tQO6849w41m8P+_uaMMinqRbFEjA@mail.gmail.com>
References: <alpine.LSU.2.11.1601181958090.2025@znpeba.jbaqresebt.arg>
 <20160119053452.GA30056@cskk.homeip.net>
 <CAMFBua7K7VXsVLbQCXK0z9tQO6849w41m8P+_uaMMinqRbFEjA@mail.gmail.com>
Message-ID: <n7kul6$4ja$1@ger.gmane.org>

On 19/01/16 05:41, Anshu Kumar wrote:

> Here is my actual scenario. I have a csv file and it would already be
> present. I need to read and remove some rows based on some logic. I have
> written earlier two separate file opens which I think was nice and clean.

Yes, it looks straightforward. The only possible issue is that
it reads the entire input file in before writing the output
which could become a memory hog.

> with open(file_path, 'rb') as fr:
>     for row in csv.DictReader(fr):
>         #Skip for those segments which are part of overridden_ids
>         if row['id'] not in overriden_ids:
>             segments[row['id']] = {
>                 'id': row['id'],
>                 'attrib': json.loads(row['attrib']),
>                 'stl': json.loads(row['stl']),
>                 'meta': json.loads(row['meta']),
>             }
> #rewriting files with deduplicated segments
> with open(file_path, 'wb') as fw:
>     writer = csv.UnicodeWriter(fw)
>     writer.writerow(["id", "attrib", "stl", "meta"])
>     for seg in segments.itervalues():
>         writer.writerow([seg['id'], json.dumps(seg["attrib"]),
> json.dumps(seg["stl"]), json.dumps(seg["meta"])])
> 
> 
> I have got review comments to improve this block by having just single
> file open and minimum memory usage.

I'd ignore the advice to use a single file. One extra file
handle is insignificant in memory terms and the extra simplicity
two handles brings is worth far more.
What I would do is open both files at the start and instead
of creating the segments just write the data direct to the
output file. That will slash your memory footprint.

Contrast that with using a single file:
You need to read a line. check its length, seek back to
the beginning of the line.
Create the new output string. Check its length.
If it is the same length(miracles happen!) just write the line
if it is shorter than the original write the new line,
then write spaces to fill the gap.
If it is longer than the original - oh dear. If you write it you will
overwrite part of your next line. So you need to do a look ahead to grab
the next line of data before writing.
But now your next line has to compare against
data.length-overlap.length and if the new line
is longer than that repeat.
And if your new line is longer than two old lines it gets even worse.
On top of that you now have a file that is partially full of new style
data while the rest is old style. Anyone trying to read that will get
very confused.
And we haven't even considered what to do about the lines you
want to delete...

In short this is not a situation where + mode is a good idea.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From __peter__ at web.de  Tue Jan 19 04:52:20 2016
From: __peter__ at web.de (Peter Otten)
Date: Tue, 19 Jan 2016 10:52:20 +0100
Subject: [Tutor] Simultaneous read and write on file
References: <alpine.LSU.2.11.1601181958090.2025@znpeba.jbaqresebt.arg>
 <20160119053452.GA30056@cskk.homeip.net>
 <CAMFBua7K7VXsVLbQCXK0z9tQO6849w41m8P+_uaMMinqRbFEjA@mail.gmail.com>
Message-ID: <n7l10l$bes$1@ger.gmane.org>

Anshu Kumar wrote:

> Hello All,
> 
> So much Thanks for your response.
> 
> Here is my actual scenario. I have a csv file and it would already be
> present. I need to read and remove some rows based on some logic. I have
> written earlier two separate file opens which I think was nice and clean.
> 
> actual code:
> 
> with open(file_path, 'rb') as fr:
>     for row in csv.DictReader(fr):
>         #Skip for those segments which are part of overridden_ids
>         if row['id'] not in overriden_ids:

Oops typo; so probably not your actual code :(

>             segments[row['id']] = {
>                 'id': row['id'],
>                 'attrib': json.loads(row['attrib']),
>                 'stl': json.loads(row['stl']),
>                 'meta': json.loads(row['meta']),
>             }
> #rewriting files with deduplicated segments
> with open(file_path, 'wb') as fw:
>     writer = csv.UnicodeWriter(fw)
>     writer.writerow(["id", "attrib", "stl", "meta"])
>     for seg in segments.itervalues():
>         writer.writerow([seg['id'], json.dumps(seg["attrib"]),
> json.dumps(seg["stl"]), json.dumps(seg["meta"])])
> 
> 
> I have got review comments to improve this block by having just single
> file open and minimum memory usage.

Are the duplicate ids stored in overridden_ids or are they implicitly 
removed by overwriting them in

segments[row["id"]] = ...

? If the latter, does it matter whether the last or the first row with a 
given id is kept?


From alan.gauld at btinternet.com  Tue Jan 19 11:26:37 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 19 Jan 2016 16:26:37 +0000
Subject: [Tutor] Source of MySQL Command Interpreter
In-Reply-To: <CAMtNk+tw=QQxL-xiPVZ+rNV_Wn-hZTej-mHN-YUUvtRVrKY_4w@mail.gmail.com>
References: <CAMtNk+tw=QQxL-xiPVZ+rNV_Wn-hZTej-mHN-YUUvtRVrKY_4w@mail.gmail.com>
Message-ID: <n7lo3t$lb4$1@ger.gmane.org>

On 16/01/16 23:27, Ricardo Mart?nez wrote:
> Hi, i wrote a small APP to execute MySQL commands and retrieve to a Treeview

I finally got round to looking at this.

Here are a couple of comments.

I don't understand what the else part is supposed to do here:

    if self.cursor.description is not None:
         self.resultset = self.cursor.fetchall()
    else:
         print("DES: ",self.cursor.description,"\n")

Surely it only executes if description is None, in
which case what do you expect to print?

Also, why is this a second if/else when it's the same test?

     if self.cursor.description is not None:
         columns = [x[0] for x in self.cursor.description]
     else:
         columns = []

Why not just set the columns in the branches of the first test?

Also at the top of that function:

    def executeSQL(self, sqlstr, grdResult):
        if sqlstr is not None:
            self.txtCommand.clipboard_clear()
            self.txtCommand.clipboard_append(sqlstr)
            print("QUERY: ",sqlstr,"\n")
        if self.cursor is not None:
            try:
                self.cursor.execute(sqlstr)

If sqlstr is None you still try to execute it? Is that correct?

Finally you use the \ line continuation in a few places where
it is not needed because you are inside parens.

You can use as many newlines as you like inside parens:
eg.

def foo(bar,   # a watering hole
        baz,   # an English Barry
        bash,  # a shell
        bob):  # Blackadder's new servant
    pass


Sorry, not much time but those were just some quick observations.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From dyoo at hashcollision.org  Tue Jan 19 12:22:06 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Tue, 19 Jan 2016 09:22:06 -0800
Subject: [Tutor] Fwd:
In-Reply-To: <CAGSR2fEwFAK43Muj37aM326cRE68XfhbW7U+sV0pAE1PYJzybA@mail.gmail.com>
References: <CAGSR2fGbWt7bJYdein5Cw5+oBMw2Ad5L5j_DkK-nUWqW8TduJA@mail.gmail.com>
 <CAGSR2fFrONeTJJ_zQieLgni+kmmSMPMpwzBqrho+bHEzHiMQtw@mail.gmail.com>
 <CAGSR2fGgk92SW0KKzPyX5WOTcTnNtToEjZ_tEVOzxH8EABNM3A@mail.gmail.com>
 <CAGSR2fHfbb1DnPHRiCEqYNBSBNNR7bS2yeUjCSe7Get1YgxAEA@mail.gmail.com>
 <CAGZAPF4xTLy2qwqscQRrZce7-4n5=KDWB=asSTYVybnm0uUW2A@mail.gmail.com>
 <CAGSR2fEaDVGeftKuZBQPbUV6m=GLZaSWpEWKKr5Hq4tV6D9Erw@mail.gmail.com>
 <CAGSR2fEwFAK43Muj37aM326cRE68XfhbW7U+sV0pAE1PYJzybA@mail.gmail.com>
Message-ID: <CAGZAPF4u4tjhZ1TMYUAu+PNhGiA-=4Jphf1+z0idqw0HfFw19g@mail.gmail.com>

No.  This is definitely wrong.
On Jan 19, 2016 2:51 AM, "Deepak Nn" <deepaknedumpilly at gmail.com> wrote:

> Finding the answer is very important that's why for the competition .
>
> On Tue, Jan 19, 2016 at 4:19 PM, Deepak Nn <deepaknedumpilly at gmail.com>
> wrote:
>
>> This is for an online competition i am now participating in Amrita InCTF
>> Junior <https://junior.inctf.in/> .Please don't misunderstand and sent
>> me the code .
>>
>> On Tue, Jan 19, 2016 at 12:29 AM, Danny Yoo <dyoo at hashcollision.org>
>> wrote:
>>
>>> > Please provide a python program to run a program (.exe) and get Hash
>>> > *exactly* as :
>>> >
>>> >  160 106 182 190 228 64 68 207 248 109 67 88 41 .The username to be
>>> > used is admin
>>> > .The *password* is what to be found out .The hash provided is of the
>>> > correct password .Mostly the password will be *13 char long *and has a
>>> > small chance of being all alpha characters .
>>>
>>>
>>> Hi Deepak,
>>>
>>> This doesn't seem like a beginner-level question.  If I had a guess,
>>> it sounds more like something out of a shady rent-a-coder kind of
>>> thing.
>>>
>>> Unfortunately, I don't think we can help with this.  Even if we did
>>> have the technical expertise, I still don't think we should help on
>>> this in the first place.  If I'm understanding the question correctly,
>>> you're asking for brute password breaking, which goes against most
>>> professional codes of conduct.  Example:
>>> https://www.acm.org/about-acm/acm-code-of-ethics-and-professional-conduct
>>> .
>>>
>>
>>
>

From dyoo at hashcollision.org  Tue Jan 19 12:43:34 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Tue, 19 Jan 2016 09:43:34 -0800
Subject: [Tutor] Fwd:
In-Reply-To: <CAGZAPF4u4tjhZ1TMYUAu+PNhGiA-=4Jphf1+z0idqw0HfFw19g@mail.gmail.com>
References: <CAGSR2fGbWt7bJYdein5Cw5+oBMw2Ad5L5j_DkK-nUWqW8TduJA@mail.gmail.com>
 <CAGSR2fFrONeTJJ_zQieLgni+kmmSMPMpwzBqrho+bHEzHiMQtw@mail.gmail.com>
 <CAGSR2fGgk92SW0KKzPyX5WOTcTnNtToEjZ_tEVOzxH8EABNM3A@mail.gmail.com>
 <CAGSR2fHfbb1DnPHRiCEqYNBSBNNR7bS2yeUjCSe7Get1YgxAEA@mail.gmail.com>
 <CAGZAPF4xTLy2qwqscQRrZce7-4n5=KDWB=asSTYVybnm0uUW2A@mail.gmail.com>
 <CAGSR2fEaDVGeftKuZBQPbUV6m=GLZaSWpEWKKr5Hq4tV6D9Erw@mail.gmail.com>
 <CAGSR2fEwFAK43Muj37aM326cRE68XfhbW7U+sV0pAE1PYJzybA@mail.gmail.com>
 <CAGZAPF4u4tjhZ1TMYUAu+PNhGiA-=4Jphf1+z0idqw0HfFw19g@mail.gmail.com>
Message-ID: <CAGZAPF5S7sWifkOzS0mCEw9TPaMcO6DZf6vBmDrG_E0o_ad3Ag@mail.gmail.com>

My apologies for this ugliness.  Followup to the mailing list: I've
contacted the organizers of the Amrita InCTF competition and told them
that one of their members was trying to use us for cheat for answers.
I'll follow up if I hear back from the organizers.


Apparently, this is an endemic problem, if one can generalize from the
multiple posts the organizers have made about folks breaking the
rules:

    https://www.facebook.com/cybergurukulam

From bachir_bac at yahoo.com  Tue Jan 19 14:28:24 2016
From: bachir_bac at yahoo.com (Bachir Bachir)
Date: Tue, 19 Jan 2016 19:28:24 +0000 (UTC)
Subject: [Tutor] python plotting
References: <39615045.7245524.1453231705147.JavaMail.yahoo.ref@mail.yahoo.com>
Message-ID: <39615045.7245524.1453231705147.JavaMail.yahoo@mail.yahoo.com>

?Dear all,
I have some data taken at specific time of the day and i ?want to display those data according to the time,attached is the cvs file and the display output from python pandas . I used the following script?
?v2=read_csv('v2_12.dat') ?#data frame for v2
?v2.plot(kind='bar', x='Time_hhmmss', y='Av_phase',figsize=(12,1)) #display for v2 only
I want to see a gap on the display because there was no data recorded between ?08:20:56 ?and ? 14:55:33 ? ? ?but on my display i see them side by ?side?Is there any way to do ?this using python display optionsYour help is highly appreciated?Thanks much

From alan.gauld at btinternet.com  Tue Jan 19 17:55:44 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 19 Jan 2016 22:55:44 +0000
Subject: [Tutor] python plotting
In-Reply-To: <39615045.7245524.1453231705147.JavaMail.yahoo@mail.yahoo.com>
References: <39615045.7245524.1453231705147.JavaMail.yahoo.ref@mail.yahoo.com>
 <39615045.7245524.1453231705147.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <n7metg$jhe$1@ger.gmane.org>

On 19/01/16 19:28, Bachir Bachir via Tutor wrote:
>  Dear all,
> I have some data taken at specific time of the day and i  want to 
> display those data according to the time,
> attached is the cvs file and the display output

I assume you mean csv file?

> from python pandas . I used the following script 
>  v2=read_csv('v2_12.dat')  #data frame for v2
>  v2.plot(kind='bar', x='Time_hhmmss', y='Av_phase',figsize=(12,1)) #display for v2 only

You need to post in plain text for us to see the code properly.

> I want to see a gap on the display because there was 
> no data recorded between  08:20:56  and   14:55:33

This is mainly a SciPy question it would be better placed there.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From francois.dion at gmail.com  Tue Jan 19 22:28:46 2016
From: francois.dion at gmail.com (Francois Dion)
Date: Tue, 19 Jan 2016 22:28:46 -0500
Subject: [Tutor] python plotting
In-Reply-To: <39615045.7245524.1453231705147.JavaMail.yahoo@mail.yahoo.com>
References: <39615045.7245524.1453231705147.JavaMail.yahoo.ref@mail.yahoo.com>
 <39615045.7245524.1453231705147.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <CAOLi1KAP5EYnMB6MVW+6pxa8jcZHwF-mh4SQ8Bq_GBUS61KBmw@mail.gmail.com>

I'm guessing you loaded from pandas import *... It is better to import
pandas as pd, then use pd.read_csv. I also tend to name my data frames df
or a variation and time series as ts.

Speaking of series, If your data is not a series with a datetime type, then
it will be plotted as a categorical, meaning each different X value is
represented at a constant interval, in index order. You need to convert
your time string to a datetime. (pd.to_datetime is your friend).

Francois

On Tue, Jan 19, 2016 at 2:28 PM, Bachir Bachir via Tutor <tutor at python.org>
wrote:

>  Dear all,
> I have some data taken at specific time of the day and i  want to display
> those data according to the time,attached is the cvs file and the display
> output from python pandas . I used the following script
>  v2=read_csv('v2_12.dat')  #data frame for v2
>  v2.plot(kind='bar', x='Time_hhmmss', y='Av_phase',figsize=(12,1))
> #display for v2 only
> I want to see a gap on the display because there was no data recorded
> between  08:20:56  and   14:55:33      but on my display i see them side by
>  side Is there any way to do  this using python display optionsYour help is
> highly appreciated Thanks much
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
raspberry-python.blogspot.com - www.pyptug.org - www.3DFutureTech.info -
@f_dion

From sjeik_appie at hotmail.com  Wed Jan 20 08:33:17 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Wed, 20 Jan 2016 13:33:17 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
Message-ID: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>

Hi,

Like the subject says: Why is an OrderedDict not sliceable? (From the collections library). Was that an intentional omission, or a mistake? [1]

Background: I do not use OrderedDict very often, but I thought I could use it to look up street and city names using postcodes ([0-9]{4} [a-z]{2} format). I needed it to be ordered because I also wanted to be able to use bisect, which is needed when the postcode letters are missing. In short: a fast dict lookup for complete postcodes and less fast bisect lookup for in complete postcodes.

[1] http://stackoverflow.com/questions/30975339/slicing-a-python-ordereddict

Thanks!

Albert-Jan
 		 	   		  

From rheeyauppaal at gmail.com  Wed Jan 20 05:55:57 2016
From: rheeyauppaal at gmail.com (Rheeya Uppaal)
Date: Wed, 20 Jan 2016 16:25:57 +0530
Subject: [Tutor] import cv, not cv2
Message-ID: <CAA7L8V4pGxG+=JM5y69t=zUVZBcQZZZv7gd2TxFX7Wjh1TRHkg@mail.gmail.com>

After successfully installing dlib and cv2, I have now been asked to
install the cv library on Ubuntu.

>From what I have read, it seems OpenCV has discontinued support for cv and
only cv2 is used now. Am I correct? Regardless, does anyone know how I can
get this library?

I apologize in advance if my question is foolish.

Thank you! Cheers!

From alan.gauld at btinternet.com  Wed Jan 20 12:18:45 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 20 Jan 2016 17:18:45 +0000
Subject: [Tutor] import cv, not cv2
In-Reply-To: <CAA7L8V4pGxG+=JM5y69t=zUVZBcQZZZv7gd2TxFX7Wjh1TRHkg@mail.gmail.com>
References: <CAA7L8V4pGxG+=JM5y69t=zUVZBcQZZZv7gd2TxFX7Wjh1TRHkg@mail.gmail.com>
Message-ID: <n7ofhl$fms$1@ger.gmane.org>

On 20/01/16 10:55, Rheeya Uppaal wrote:
> After successfully installing dlib and cv2, I have now been asked to
> install the cv library on Ubuntu.
> 
> From what I have read, it seems OpenCV has discontinued support for cv and
> only cv2 is used now. Am I correct? Regardless, does anyone know how I can
> get this library?

This isn't really a Python language or standard library question.
You should try asking on the openCV support forum:

http://answers.opencv.org/questions/

They should be able to help you.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Wed Jan 20 20:00:29 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 21 Jan 2016 12:00:29 +1100
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
Message-ID: <20160121010029.GD4619@ando.pearwood.info>

On Wed, Jan 20, 2016 at 01:33:17PM +0000, Albert-Jan Roskam wrote:
> Hi,
> 
> Like the subject says: Why is an OrderedDict not sliceable? (From the 
> collections library). Was that an intentional omission, or a mistake? 
> [1]

Because slicing a dict makes no sense. A dict is a mapping, not a 
sequence.

d = OrderedDict()
d["cow"] = "a"
d["fox"] = "b"
d["ape"] = "c"
d["dog"] = "d"
d["bee"] = "e"

I can do a dict lookup on a key:

d["cow"]

What would a slice even mean? d[1:4] but 1, 2, 3 are not keys of the 
dict.



-- 
Steve

From robertvstepp at gmail.com  Wed Jan 20 22:42:29 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Wed, 20 Jan 2016 21:42:29 -0600
Subject: [Tutor] Why is the name "self" optional instead of mandatory?
Message-ID: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>

I'm whizzing along in "Python Crash Course" and am in the chapter on
classes.  Just to satisfy my suspicion that "self" is just a
placeholder for creating an object instance, I tried the following:

>>> class Dog(object):
        def __init__(this, name, age):
            this.name = name
            this.age = age
        def bark(this):
            print("Woof!  Woof!  Grrr!!!")
        def whoami(this):
            print("My name is", this.name.title(), "and I am",
                 this.age, "years old.")

>>> mydog = Dog('Spotty', 50)
>>> mydog.bark()
Woof!  Woof!  Grrr!!!

>>> mydog.whoami()
My name is Spotty and I am 50 years old.

And just to be really silly:

>>> class Cat(object):
        def __init__(MEOWWWW, name, age):
            MEOWWWW.name = name
            MEOWWWW.age = age
        def happy_cat(MEOWWWW):
            print("Zzzz ... purrrr ... zzzz")
        def whoami(MEOWWWW):
            print("My name is", MEOWWWW.name, "and I am", MEOWWWW.age,
                 "years old.  Now leave me be!  I'm very sleepy!!!")

>>> mycat = Cat('Callie', 7)
>>> mycat.happy_cat()
Zzzz ... purrrr ... zzzz
>>> mycat.whoami()
My name is Callie and I am 7 years old.  Now leave me be!  I'm very sleepy!!!

So I really only have one question:  Why not make Python's
*traditional* name, "self", mandatory?  Why give the programmer this
kind of choice?  [OK, that was two questions.]

TIA!
-- 
boB

From robertvstepp at gmail.com  Wed Jan 20 23:34:59 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Wed, 20 Jan 2016 22:34:59 -0600
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
Message-ID: <CANDiX9+U47s=pKQozVk6DCB9xHZSBtw84_H+jhnsswUGj31n1g@mail.gmail.com>

My intent was to deliberately introduce an error into my class definition:

>>> class Hmm(object):
        def __init__(self, sigh_type, sigh_strength):
            self.sigh_type = sigh_type
            self.sigh_strength = sigh_strength
        def snort(self):
            if self.sigh_strength == 'low':
                print("snort")
            elif self.sigh_strength == 'med':
                print("Snort!")
            elif self.sigh_strenght == 'high':
                print("SNORT!!!")
            else:
                print("Hmm...")
        def sigh():
            if self.sigh_type == 'quiet':
                print("pssssss")
            elif self.sigh_type == 'annoying':
                print("Whoosh!")
            elif self.sigh_type == 'loud':
                print("HEAVY SIGH!!!")
            else:
                print("HMM!!!")

I omitted "self" from the sigh() method to see what would happen plus
some other things.

>>> humdrum = Hmm('quiet', 'low')
>>> humdrum.snort()
snort
>>> humdrum.sigh_strength = 'med'
>>> humdrum.snort()
Snort!
>>> humdrum.sigh_strenght = 'high'
>>> humdrum.snort()
Snort!

At this point I wondered why my output was not "SNORT!!!".  Then I
noticed my typo.  But now I wonder why I did not get an error from
this typo?

>>> humdrum.sigh_strength = 'high'
>>> humdrum.snort()
SNORT!!!
>>> humdrum.sigh()
Traceback (most recent call last):
  File "<pyshell#232>", line 1, in <module>
    humdrum.sigh()
TypeError: sigh() takes 0 positional arguments but 1 was given

This was my original point in doing all of this, to see what would
result if I omitted "self".  I am pretty sure the error is because the
object instance gets automatically passed to the sigh() method, but by
leaving the "self" parameter out in the method definition, I have a
mismatch between what was defined (0 parameters) and what was passed
to the method (1 argument).

>>> humdrum.sigh_strenght
'high'

But what about this?  It seems like I can use the humdrum arguments
outside of the Hmm class and merrily define new variables.  Why is
this?  Is this potentially useful behavior?

-- 
boB

From cs at zip.com.au  Thu Jan 21 00:34:40 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 21 Jan 2016 16:34:40 +1100
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9+U47s=pKQozVk6DCB9xHZSBtw84_H+jhnsswUGj31n1g@mail.gmail.com>
References: <CANDiX9+U47s=pKQozVk6DCB9xHZSBtw84_H+jhnsswUGj31n1g@mail.gmail.com>
Message-ID: <20160121053440.GA68084@cskk.homeip.net>

On 20Jan2016 22:34, boB Stepp <robertvstepp at gmail.com> wrote:
>My intent was to deliberately introduce an error into my class definition:
>
>>>> class Hmm(object):
>        def __init__(self, sigh_type, sigh_strength):
>            self.sigh_type = sigh_type
>            self.sigh_strength = sigh_strength
>        def snort(self):
>            if self.sigh_strength == 'low':
>                print("snort")
>            elif self.sigh_strength == 'med':
>                print("Snort!")
>            elif self.sigh_strenght == 'high':
>                print("SNORT!!!")
>            else:
>                print("Hmm...")
>        def sigh():
>            if self.sigh_type == 'quiet':
>                print("pssssss")
>            elif self.sigh_type == 'annoying':
>                print("Whoosh!")
>            elif self.sigh_type == 'loud':
>                print("HEAVY SIGH!!!")
>            else:
>                print("HMM!!!")
>
>I omitted "self" from the sigh() method to see what would happen plus
>some other things.

Well... You've bound a function accepting no arguments to the "sigh" attribute 
of the class. Legal. Nonsensical perhaps, but legal.

>>>> humdrum = Hmm('quiet', 'low')
>>>> humdrum.snort()
>snort
>>>> humdrum.sigh_strength = 'med'
>>>> humdrum.snort()
>Snort!
>>>> humdrum.sigh_strenght = 'high'
>>>> humdrum.snort()
>Snort!
>
>At this point I wondered why my output was not "SNORT!!!".  Then I
>noticed my typo.  But now I wonder why I did not get an error from
>this typo?

Because your "if" statement matched the "med". So it never tried to look up 
"self.sigh_strenght".

>>>> humdrum.sigh_strength = 'high'
>>>> humdrum.snort()
>SNORT!!!

Again, as you expected, yes?

>>>> humdrum.sigh()
>Traceback (most recent call last):
>  File "<pyshell#232>", line 1, in <module>
>    humdrum.sigh()
>TypeError: sigh() takes 0 positional arguments but 1 was given
>
>This was my original point in doing all of this, to see what would
>result if I omitted "self".  I am pretty sure the error is because the
>object instance gets automatically passed to the sigh() method, but by
>leaving the "self" parameter out in the method definition, I have a
>mismatch between what was defined (0 parameters) and what was passed
>to the method (1 argument).

Correct.

>>>> humdrum.sigh_strenght
>'high'
>
>But what about this?  It seems like I can use the humdrum arguments
>outside of the Hmm class and merrily define new variables.  Why is
>this?  Is this potentially useful behavior?

"humdrum" is just an object. You can assign attibutes to it at any time.

The code executing inside the class is no different to the code outside the 
class; Python is a dynamic language and you can do this stuff at any time.

It isn't _specificly_ useful to assign an attribute long after its normal 
initialisation, but it can be quite useful. But consider your initialiser:

>        def __init__(self, sigh_type, sigh_strength):
>            self.sigh_type = sigh_type
>            self.sigh_strength = sigh_strength

By the time __init__ is called, the object already exists with a type/class and 
everything. All __init__ is doing is what you find unexpected later; defining 
new attribute values not there before. The only thing special about __init__ is 
that it is called automatically after an object is created. But that is all.

This is not a static language, and __init__ is not defining what 
fields/attributes the object possesses. It is merely setting some of them. It 
is executable code, not a static type definition.

Cheers,
Cameron Simpson <cs at zip.com.au>

From cs at zip.com.au  Thu Jan 21 00:40:41 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 21 Jan 2016 16:40:41 +1100
Subject: [Tutor] Why is the name "self" optional instead of mandatory?
In-Reply-To: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
References: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
Message-ID: <20160121054041.GA24425@cskk.homeip.net>

On 20Jan2016 21:42, boB Stepp <robertvstepp at gmail.com> wrote:
>I'm whizzing along in "Python Crash Course" and am in the chapter on
>classes.  Just to satisfy my suspicion that "self" is just a
>placeholder for creating an object instance,

No, it is a placeholder for a _preexiting_ object instance.

[...]
>So I really only have one question:  Why not make Python's
>*traditional* name, "self", mandatory?  Why give the programmer this
>kind of choice?  [OK, that was two questions.]

Why make it mandatory? What benefit would it bring? Remember, one can write 
nonsense or impossible to read gibberish in any language; most people don't try 
to. So to repeat my question: why make it mandatory?

Cheers,
Cameron Simpson <cs at zip.com.au>

From dyoo at hashcollision.org  Thu Jan 21 00:54:32 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Wed, 20 Jan 2016 21:54:32 -0800
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9+U47s=pKQozVk6DCB9xHZSBtw84_H+jhnsswUGj31n1g@mail.gmail.com>
References: <CANDiX9+U47s=pKQozVk6DCB9xHZSBtw84_H+jhnsswUGj31n1g@mail.gmail.com>
Message-ID: <CAGZAPF4QCu7tAx_jZuWJ342e30Q92Dnyb1Rr_nbbgTMQrs0cdg@mail.gmail.com>

>>>> humdrum.sigh_strenght = 'high'
>>>> humdrum.snort()
> Snort!
>
> At this point I wondered why my output was not "SNORT!!!".  Then I
> noticed my typo.  But now I wonder why I did not get an error from
> this typo?


Hi boB,


Just to be explicit: you are pointing out that:

     humdrum.sigh_strenght = 'high'

was a typo, and it would have been nice if it could be caught as an error.

If that's the case, then yes, I agree.  Unfortunately, this isn't a
runtime error because Python allows us to add arbitrary attributes to
objects: it's a deliberate design feature.  One justification for this
appears to be similar to that of dynamic typing: if we have good unit
tests, we should be able to catch these problems with our unit tests,
so why add a restriction?

(As a personal note: I'm not entirely convinced of the merit of this
particular freedom.  Misspelling is a really easy mistake to make, and
I want the language to catch my simple mistakes, if it's not too
onerous to do so.)


That being said, there are tools available that should be able to
catch many of these problems before runtime.

    http://www.pylint.org/
    http://pychecker.sourceforge.net/

These should raise warnings if you run it on your program.  You might
want to take a look at these!


Furthermore, there actually *is* a way to turn it into a runtime
error, though this is probably not beginner material.  If you're
interested, see material on "__slots__".
https://docs.python.org/3/reference/datamodel.html#slots.  I don't
think it's intended to be used to catch typos though: its main purpose
is to reduce memory usage, and it's use to catch misspellings is
incidental.


Hope this helps!

From dyoo at hashcollision.org  Thu Jan 21 01:15:52 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Wed, 20 Jan 2016 22:15:52 -0800
Subject: [Tutor] Why is the name "self" optional instead of mandatory?
In-Reply-To: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
References: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
Message-ID: <CAGZAPF4s0HAS45pmjQvKnMHZ31R7_Ob6qCSYg8WqFnJRfhjNSA@mail.gmail.com>

> So I really only have one question:  Why not make Python's
> *traditional* name, "self", mandatory?  Why give the programmer this
> kind of choice?  [OK, that was two questions.]


There are situations where it might not be mandatory.


This is definitely not beginner Python material, so I don't want to
delve into it too deeply.  Roughly: we can do things like "metaclass"
programming where your programs modify the rules that construct
classes.  In that way, it's possible to do fairly crazy things.  We
might even be able to make it optional to say "self" all the time, if
we modify the rules radically enough.

If you are really interested in this, see:
https://jakevdp.github.io/blog/2012/12/01/a-primer-on-python-metaclasses/
for an primer.

Basically, Python does almost everything at run-time.  The sorts of
things we'd expect to be static properties aren't static in Python.
Other languages tend to lock things down a bit more firmly, or at the
very least, do more things at compile-time.

Python programmers have enormous flexibility at run-time, at the cost
of making it really easy to make mistakes that aren't caught until
run-time.

From ben+python at benfinney.id.au  Thu Jan 21 01:57:36 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 21 Jan 2016 17:57:36 +1100
Subject: [Tutor] Why is the name "self" optional instead of mandatory?
References: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
Message-ID: <85k2n3mnbz.fsf@benfinney.id.au>

boB Stepp <robertvstepp at gmail.com> writes:

> So I really only have one question: Why not make Python's
> *traditional* name, "self", mandatory? Why give the programmer this
> kind of choice?

For the same reason that four-space indentation is not mandatory, yet
anyone who chooses a different indentation size needs to produce good
reasons why.

In both cases: Because nothing is broken, nor made especially ambiguous,
by choosing differently. And because the Python community of programmers
can be relied upon to enforce the convention, as with most other
conventions.

-- 
 \      ?When I was a little kid we had a sand box. It was a quicksand |
  `\           box. I was an only child... eventually.? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Thu Jan 21 02:01:43 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 21 Jan 2016 18:01:43 +1100
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
References: <CANDiX9+U47s=pKQozVk6DCB9xHZSBtw84_H+jhnsswUGj31n1g@mail.gmail.com>
Message-ID: <85fuxrmn54.fsf@benfinney.id.au>

boB Stepp <robertvstepp at gmail.com> writes:

> My intent was to deliberately introduce an error into my class definition:
>
> >>> class Hmm(object):
>         def __init__(self, sigh_type, sigh_strength):
>             self.sigh_type = sigh_type
>             self.sigh_strength = sigh_strength
>         def snort(self):
>             if self.sigh_strength == 'low':
>                 print("snort")
>             elif self.sigh_strength == 'med':
>                 print("Snort!")
>             elif self.sigh_strenght == 'high':
>                 print("SNORT!!!")
>             else:
>                 print("Hmm...")
>         def sigh():
>             if self.sigh_type == 'quiet':
>                 print("pssssss")
>             elif self.sigh_type == 'annoying':
>                 print("Whoosh!")
>             elif self.sigh_type == 'loud':
>                 print("HEAVY SIGH!!!")
>             else:
>                 print("HMM!!!")
>
> I omitted "self" from the sigh() method to see what would happen

What would happen, when? At the time of class definition, there are no
errors that I can see.

Python doesn't run every possible branch of your code ahead of time,
just to find out what it will do. Most errors will be discovered only by
encountering the combination of inputs that will trigger them at run
time.

For this and other reasons, it is highly recommended to develop unit
tests with your code, to make explicit assertions about exactly what
will trigger each branch of the code.

This, in turn, will lead to making your code simpler and less-branching
:-)

-- 
 \     ?Do unto others twenty-five percent better than you expect them |
  `\      to do unto you. (The twenty-five percent is [to correct] for |
_o__)                            error.)? ?Linus Pauling's Golden Rule |
Ben Finney


From sjeik_appie at hotmail.com  Thu Jan 21 04:02:29 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Thu, 21 Jan 2016 09:02:29 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <20160121010029.GD4619@ando.pearwood.info>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>,
 <20160121010029.GD4619@ando.pearwood.info>
Message-ID: <DUB123-W19128E125B4E854CCAADF283C30@phx.gbl>



> Date: Thu, 21 Jan 2016 12:00:29 +1100
> From: steve at pearwood.info
> To: tutor at python.org
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> 
> On Wed, Jan 20, 2016 at 01:33:17PM +0000, Albert-Jan Roskam wrote:
> > Hi,
> > 
> > Like the subject says: Why is an OrderedDict not sliceable? (From the 
> > collections library). Was that an intentional omission, or a mistake? 
> > [1]
> 
> Because slicing a dict makes no sense. A dict is a mapping, not a 
> sequence.

For a regular dict: yes, I agree that doesn't make sense, because a regular dict is unordered.
A collections.OrderedDict on the other hand..

> d = OrderedDict()
> d["cow"] = "a"
> d["fox"] = "b"
> d["ape"] = "c"
> d["dog"] = "d"
> d["bee"] = "e"
> 
> I can do a dict lookup on a key:
> 
> d["cow"]
> 
> What would a slice even mean? d[1:4] but 1, 2, 3 are not keys of the 
> dict.

Indexing would create ambiguity, e.g d[1] might mean
* return the value associated with key 1
* return the first value

But that's not true for slices, because they cannot be dictionary keys anyway:
>>> {slice(1, 2): None}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type

With islice it is easy to do it anyway, but I still find it strange that OrderedDict.__getitem__ does not do that already (sorry!)
>>> from itertools import islice
>>> from collections import OrderedDict
>>> od = OrderedDict({"a": 1, "b": 2})
>>> list(islice(od, 1, 2))
['b']


 		 	   		  

From ben+python at benfinney.id.au  Thu Jan 21 04:19:20 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 21 Jan 2016 20:19:20 +1100
Subject: [Tutor] Why is an OrderedDict not sliceable?
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
Message-ID: <857fj3mgrr.fsf@benfinney.id.au>

Albert-Jan Roskam <sjeik_appie at hotmail.com> writes:

> Why is an OrderedDict not sliceable?

Because slicing implies index access. The built-in ?dict? and
?collections.OrderedDict? both do not support indexed access::

    >>> import collections
    >>> foo = collections.OrderedDict([
    ...         ('a', 1), ('b', 2), ('c', 3), ('d', 4)])
    >>> foo[3]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    KeyError: 3

    >>> bar = dict([
    ...         ('a', 1), ('b', 2), ('c', 3), ('d', 4)])
    >>> bar[3]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    KeyError: 3

Index access with ?foo[index]? syntax, and slicing with
?foo[start_index:stop_index:step]? syntax, collide with the existing
key-access meaning of ?foo[key]? for a mapping.

-- 
 \      ?[I]t is impossible for anyone to begin to learn that which he |
  `\                thinks he already knows.? ?Epictetus, _Discourses_ |
_o__)                                                                  |
Ben Finney


From diliupg at gmail.com  Thu Jan 21 00:58:43 2016
From: diliupg at gmail.com (DiliupG)
Date: Thu, 21 Jan 2016 11:28:43 +0530
Subject: [Tutor] Why is the name "self" optional instead of mandatory?
In-Reply-To: <20160121054041.GA24425@cskk.homeip.net>
References: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
 <20160121054041.GA24425@cskk.homeip.net>
Message-ID: <CAMxbqSMPojOBfetG621SwLu9fV9Ov=y+VgAdHuYbRbNWcbp0KA@mail.gmail.com>

The answer to this will be an interesting read... :)
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
This
email has been sent from a virus-free computer protected by Avast.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Thu, Jan 21, 2016 at 11:10 AM, Cameron Simpson <cs at zip.com.au> wrote:

> On 20Jan2016 21:42, boB Stepp <robertvstepp at gmail.com> wrote:
>
>> I'm whizzing along in "Python Crash Course" and am in the chapter on
>> classes.  Just to satisfy my suspicion that "self" is just a
>> placeholder for creating an object instance,
>>
>
> No, it is a placeholder for a _preexiting_ object instance.
>
> [...]
>
>> So I really only have one question:  Why not make Python's
>> *traditional* name, "self", mandatory?  Why give the programmer this
>> kind of choice?  [OK, that was two questions.]
>>
>
> Why make it mandatory? What benefit would it bring? Remember, one can
> write nonsense or impossible to read gibberish in any language; most people
> don't try to. So to repeat my question: why make it mandatory?
>
> Cheers,
> Cameron Simpson <cs at zip.com.au>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Diliup Gabadamudalige

http://www.diliupg.com
http://soft.diliupg.com/

**********************************************************************************************
This e-mail is confidential. It may also be legally privileged. If you are
not the intended recipient or have received it in error, please delete it
and all copies from your system and notify the sender immediately by return
e-mail. Any unauthorized reading, reproducing, printing or further
dissemination of this e-mail or its contents is strictly prohibited and may
be unlawful. Internet communications cannot be guaranteed to be timely,
secure, error or virus-free. The sender does not accept liability for any
errors or omissions.
**********************************************************************************************

From esawiek at gmail.com  Wed Jan 20 23:22:14 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Wed, 20 Jan 2016 23:22:14 -0500
Subject: [Tutor] Extract several arrays from a large 2D array
Message-ID: <CA+ZkTxu4h0hScBbxnYxqwW-dOWazdsTxrnVh+BY=Q69Z6ptyug@mail.gmail.com>

Hi All--



I have a 2D array (2000, 4); an example is given abelow.  On the 1st column
I have 15 variables, on the 2nd 4 variables. Ignore column 3 for now. I
want a code that generate 4 arrays for each variable on the 1st column;
each array contains all the values from column 4. Then I want to find the
average of each array.



For example; pick the 1st variable in column 1 which is 1; then pick the 1st
variable on the 2nd column which is 5. Now I want an array that contains
all the values on the 4th column that match variable 1 on the 1st and
variable 5 on the 2nd column. I need to get the average of each array.





A             b             c              d

1              5              3              4

1              3              2              7

2              5              7              5

3              2              8              5

2              3              2              3


Thanks in advance?EKE

From alan.gauld at btinternet.com  Thu Jan 21 04:47:33 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 21 Jan 2016 09:47:33 +0000
Subject: [Tutor] Why is the name "self" optional instead of mandatory?
In-Reply-To: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
References: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
Message-ID: <n7q9fl$ug$1@ger.gmane.org>

On 21/01/16 03:42, boB Stepp wrote:

> So I really only have one question:  Why not make Python's
> *traditional* name, "self", mandatory?  Why give the programmer this
> kind of choice?  [OK, that was two questions.]

Because to do otherwise would introduce all sorts of extra
complexity into the standard function calling mechanism.
There is nothing super special about methods they are just
functions. You can call the parameters of a function anything
you like. To suddenly have the interpreter check the name
of the first positional parameter for the specific case
of a method makes no sense.

And it's very convenient to allow other names. When I'm
playing at the >>> prompt I often just substitute s for self
to save typing. When sharing Python code with C++ or Java
programmers I often use 'this' instead because they will
understand that.

Python gives us flexibility which we are expected to use
responsibly. It's a good thing.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Thu Jan 21 05:05:00 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 21 Jan 2016 10:05:00 +0000
Subject: [Tutor] Extract several arrays from a large 2D array
In-Reply-To: <CA+ZkTxu4h0hScBbxnYxqwW-dOWazdsTxrnVh+BY=Q69Z6ptyug@mail.gmail.com>
References: <CA+ZkTxu4h0hScBbxnYxqwW-dOWazdsTxrnVh+BY=Q69Z6ptyug@mail.gmail.com>
Message-ID: <n7qagc$ij1$1@ger.gmane.org>

On 21/01/16 04:22, Ek Esawi wrote:

> I have a 2D array (2000, 4); an example is given abelow.  On the 1st column
> I have 15 variables, on the 2nd 4 variables. 

Your terminology is a little confusing since variables in
Python normally means names. You have values in your columns.
So I suspect when you say variables you mean distinct values?

> want a code that generate 4 arrays for each variable on the 1st column;
> each array contains all the values from column 4. Then I want to find the
> average of each array.

This is very different to what you describe later in your example.

> For example; pick the 1st variable in column 1 which is 1; then pick the 1st
> variable on the 2nd column which is 5. Now I want an array that contains
> all the values on the 4th column that match variable 1 on the 1st and
> variable 5 on the 2nd column. I need to get the average of each array.

In this particular case you are describing a tuple key (colA,colB)
There may be more efficient solutions, but I would create a dictionary
using the tuple as key and appending the values on colD to a list.
You can then take the average of the list for each tuple.

So in pseudo code:

data = {}
for row in myArray:
    data.get((row[0],row[1]),[]).append row[3]

for key in data:
    print key,':',sum(data[key]/len(data[key])



HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From oscar.j.benjamin at gmail.com  Thu Jan 21 05:35:24 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 21 Jan 2016 10:35:24 +0000
Subject: [Tutor] Extract several arrays from a large 2D array
In-Reply-To: <CA+ZkTxu4h0hScBbxnYxqwW-dOWazdsTxrnVh+BY=Q69Z6ptyug@mail.gmail.com>
References: <CA+ZkTxu4h0hScBbxnYxqwW-dOWazdsTxrnVh+BY=Q69Z6ptyug@mail.gmail.com>
Message-ID: <CAHVvXxQ+Pb2PkoWP53CBYBj6mTWwdY7GLTyhhJQgso_QsFi9jQ@mail.gmail.com>

On 21 January 2016 at 04:22, Ek Esawi <esawiek at gmail.com> wrote:
> I have a 2D array (2000, 4);

Do you mean a numpy array? I'm going to assume that you do.

> an example is given abelow.  On the 1st column
> I have 15 variables, on the 2nd 4 variables. Ignore column 3 for now. I
> want a code that generate 4 arrays for each variable on the 1st column;
> each array contains all the values from column 4. Then I want to find the
> average of each array.
>
> For example; pick the 1st variable in column 1 which is 1; then pick the 1st
> variable on the 2nd column which is 5. Now I want an array that contains
> all the values on the 4th column that match variable 1 on the 1st and
> variable 5 on the 2nd column. I need to get the average of each array.
>
> A             b             c              d
>
> 1              5              3              4
>
> 1              3              2              7
>
> 2              5              7              5
>
> 3              2              8              5
>
> 2              3              2              3

I'll generate a numpy array with this data:

In [1]: import numpy as np

In [2]: M = np.array([[1,5,3,4],[1,3,2,7],[2,5,7,5],[3,2,8,5],[2,3,2,3]])

In [3]: M
Out[3]:
array([[1, 5, 3, 4],
       [1, 3, 2, 7],
       [2, 5, 7, 5],
       [3, 2, 8, 5],
       [2, 3, 2, 3]])

First we want to extract the rows where 1st and 2nd columns have 1 and
5 respectively:

In [5]: M[(M[:, 0] == 1) & (M[:, 1] == 5), :]
Out[5]: array([[1, 5, 3, 4]])

Then we want to get only the 4th column of that:

In [7]: M[(M[:, 0] == 1) & (M[:, 1] == 5), 3]
Out[7]: array([4])

And now we want the mean of that:

In [8]: M[(M[:, 0] == 1) & (M[:, 1] == 5), 3].mean()
Out[8]: 4.0

--
Oscar

From __peter__ at web.de  Thu Jan 21 05:49:15 2016
From: __peter__ at web.de (Peter Otten)
Date: Thu, 21 Jan 2016 11:49:15 +0100
Subject: [Tutor] Extract several arrays from a large 2D array
References: <CA+ZkTxu4h0hScBbxnYxqwW-dOWazdsTxrnVh+BY=Q69Z6ptyug@mail.gmail.com>
Message-ID: <n7qd3b$oog$1@ger.gmane.org>

Ek Esawi wrote:

> Hi All--

> I have a 2D array (2000, 4); an example is given abelow.  On the 1st
> column I have 15 variables, on the 2nd 4 variables. Ignore column 3 for
> now. I want a code that generate 4 arrays for each variable on the 1st
> column; each array contains all the values from column 4. Then I want to
> find the average of each array.

> For example; pick the 1st variable in column 1 which is 1; then pick the
> 1st variable on the 2nd column which is 5. Now I want an array that
> contains all the values on the 4th column that match variable 1 on the 1st
> and variable 5 on the 2nd column. I need to get the average of each array.
> 
> 
> A             b             c              d
> 
> 1              5              3              4
> 
> 1              3              2              7
> 
> 2              5              7              5
> 
> 3              2              8              5
> 
> 2              3              2              3

With pandas:

>>> df = pandas.DataFrame([
... [1, 5, 3, 4],
... [1, 5, 2, 2],
... [2, 5, 1, 2],
... [2, 3, 1, 2]],
... columns=["a", "b", "c", "d"])
>>> df.groupby(("a", "b"))["d"].mean()
a  b
1  5    3
2  3    2
   5    2
Name: d, dtype: int64



From __peter__ at web.de  Thu Jan 21 05:51:10 2016
From: __peter__ at web.de (Peter Otten)
Date: Thu, 21 Jan 2016 11:51:10 +0100
Subject: [Tutor] Extract several arrays from a large 2D array
References: <CA+ZkTxu4h0hScBbxnYxqwW-dOWazdsTxrnVh+BY=Q69Z6ptyug@mail.gmail.com>
 <n7qagc$ij1$1@ger.gmane.org>
Message-ID: <n7qd6u$oog$2@ger.gmane.org>

Alan Gauld wrote:

> So in pseudo code:
> 
> data = {}
> for row in myArray:
>     data.get((row[0],row[1]),[]).append row[3]

Replace get() with setdefault() to go from pseudo to Python code.



From steve at pearwood.info  Thu Jan 21 05:57:09 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 21 Jan 2016 21:57:09 +1100
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CAGZAPF4QCu7tAx_jZuWJ342e30Q92Dnyb1Rr_nbbgTMQrs0cdg@mail.gmail.com>
References: <CANDiX9+U47s=pKQozVk6DCB9xHZSBtw84_H+jhnsswUGj31n1g@mail.gmail.com>
 <CAGZAPF4QCu7tAx_jZuWJ342e30Q92Dnyb1Rr_nbbgTMQrs0cdg@mail.gmail.com>
Message-ID: <20160121105708.GE4619@ando.pearwood.info>

On Wed, Jan 20, 2016 at 09:54:32PM -0800, Danny Yoo wrote:

> Just to be explicit: you are pointing out that:
> 
>      humdrum.sigh_strenght = 'high'
> 
> was a typo, and it would have been nice if it could be caught as an error.
> 
> If that's the case, then yes, I agree.  Unfortunately, this isn't a
> runtime error because Python allows us to add arbitrary attributes to
> objects: it's a deliberate design feature.

Danny is correct. And it is a useful feature too. For instance, we can 
add attributes to functions:

def spam(x, y):
    ...

spam.extra_info = "whatever"

Other advantages of this way of doing things include that you aren't 
forced to put all your initialisation code in a single, special place, 
you can factor parts of it out into alternative methods:

class X:
   def __init__(self, x):
       self.setup(x)
   def setup(self, x):
       process(x)
       self.attribute = x


It also means that Python doesn't require a magic "undefined" value, 
like Javascript has, or require declarations ahead of time, like static 
languages such as Pascal and C require.

However, this power does have a cost, as Danny points out:

> (As a personal note: I'm not entirely convinced of the merit of this
> particular freedom.  Misspelling is a really easy mistake to make, and
> I want the language to catch my simple mistakes, if it's not too
> onerous to do so.)

True, but I think that on average, the freedom and power of not needing 
declarations far outweighs the cost of dealing with typos. But if you 
type with your elbows and nose, you may disagree with me *wink*


[...]
> Furthermore, there actually *is* a way to turn it into a runtime
> error, though this is probably not beginner material.  If you're
> interested, see material on "__slots__".
> https://docs.python.org/3/reference/datamodel.html#slots.  I don't
> think it's intended to be used to catch typos though: its main purpose
> is to reduce memory usage, and it's use to catch misspellings is
> incidental.

Agreed. __slots__ is certainly not intended as a way of catching 
misspellings, or prohibiting the addition of new attributes. But if you 
don't mind the scorn of your fellow coders, you can use it for that 
purpose.



-- 
Steve

From oscar.j.benjamin at gmail.com  Thu Jan 21 06:02:40 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 21 Jan 2016 11:02:40 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <857fj3mgrr.fsf@benfinney.id.au>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
 <857fj3mgrr.fsf@benfinney.id.au>
Message-ID: <CAHVvXxTkthTWEmJ6Eum=u1-eUT=ZdpF97TyJW3UY9PpuO6K7dg@mail.gmail.com>

On 21 January 2016 at 09:19, Ben Finney <ben+python at benfinney.id.au> wrote:
> Albert-Jan Roskam <sjeik_appie at hotmail.com> writes:
>
>> Why is an OrderedDict not sliceable?
>
> Because slicing implies index access. The built-in ?dict? and
> ?collections.OrderedDict? both do not support indexed access::

According to a narrow definition of indexed access. I would say that
d[k] is index access even if d is a dict and k a key.

Albert-Jan I guess what you want is this:

from collections import OrderedDict

class SliceableOrderedDict(OrderedDict):
    def __getitem__(self, sl):
        if isinstance(sl, slice):
            keys = list(self)
            keyslice = slice(
                None if sl.start is None else keys.index(sl.start),
                None if sl.stop is None else keys.index(sl.stop),
                sl.step
                )
            newitems = ((k, self[k]) for k in keys[keyslice])
            return SliceableOrderedDict(newitems)
        else:
            return super().__getitem__(sl)

I guess that the authors of OrderedDict just didn't really consider
this to be very useful. Apart from having ordered iteration
OrderedDict is not really that deeply thought out. There's a thread on
python-ideas about the inconsistent behaviour of the keys and values
views and equality comparison of OrderedDicts on python-ideas at the
moment:

https://mail.python.org/pipermail/python-ideas/2015-December/037472.html

--
Oscar

From steve at pearwood.info  Thu Jan 21 06:49:29 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 21 Jan 2016 22:49:29 +1100
Subject: [Tutor] Why is the name "self" optional instead of mandatory?
In-Reply-To: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
References: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
Message-ID: <20160121114922.GF4619@ando.pearwood.info>

On Wed, Jan 20, 2016 at 09:42:29PM -0600, boB Stepp wrote:

> So I really only have one question:  Why not make Python's
> *traditional* name, "self", mandatory?  Why give the programmer this
> kind of choice?  [OK, that was two questions.]

Why bother making it mandatory? That just makes more work for the 
compiler -- it has to decide that a function is inside a class, and 
therefore apply a restriction to the first argument. Most of the time, 
giving the programmer more freedom is less work.

And what happens if you do this?

class X:
    pass

def func(this):
    print("instance %r called method" % this)

X.method = func

And what are we supposed to do with classmethods and staticmethods? They 
shouldn't take a "self" argument at all, but they start off life as a 
regular function, just like ordinary instance methods.

And of course, there are Metaclasses. You might need a metaclass method 
that needs to deal with all three levels of the hierarchy: the 
metaclass, the class it creates, and the instance of that class.

class Meta(type):
    def metamethod(meta, cls, self):
        ...



-- 
Steve

From francois.dion at gmail.com  Thu Jan 21 08:16:07 2016
From: francois.dion at gmail.com (Francois Dion)
Date: Thu, 21 Jan 2016 08:16:07 -0500
Subject: [Tutor] Why is the name "self" optional instead of mandatory?
In-Reply-To: <20160121114922.GF4619@ando.pearwood.info>
References: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
 <20160121114922.GF4619@ando.pearwood.info>
Message-ID: <CAOLi1KD6Ei6j9PfAE_cz6K-MdCuG5suxsKHUPH9t8yARBkymUg@mail.gmail.com>

On Thu, Jan 21, 2016 at 6:49 AM, Steven D'Aprano <steve at pearwood.info>
 wrote:

> On Wed, Jan 20, 2016 at 09:42:29PM -0600, boB Stepp wrote:
>
> > So I really only have one question:  Why not make Python's
> > *traditional* name, "self", mandatory?  Why give the programmer this
> > kind of choice?  [OK, that was two questions.]


[answers ommited]

All great answers. Ben mentioned the four space tab convention as a similar
thing:  Python has a ton of conventions but doesn't enforce them. That is
not the job of the interpreter, at least the Python one, since it has never
enforced any convention.

Yes, individual programmers can enforce these conventions in whichever way
they want. Some see this as a bad thing for coding standards, but that is
easily addressed by tools like pylint. In an enterprise setting, the
pipeline of going from writing code to deploying it will go through a few
phases, including checking for PEP8 compliance, Pylint compliance (the
rules the team cares about), code complexity etc. With pylint, if you try
it on your code with this instead of self, you'll get this in particular:
"Method should have "self" as first argument (no-self-argument)"

Francois


-- 
raspberry-python.blogspot.com - www.pyptug.org - www.3DFutureTech.info -
@f_dion

From ben+python at benfinney.id.au  Thu Jan 21 12:00:10 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 22 Jan 2016 04:00:10 +1100
Subject: [Tutor] Why is an OrderedDict not sliceable?
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
 <857fj3mgrr.fsf@benfinney.id.au>
 <CAHVvXxTkthTWEmJ6Eum=u1-eUT=ZdpF97TyJW3UY9PpuO6K7dg@mail.gmail.com>
Message-ID: <8537tqna05.fsf@benfinney.id.au>

Oscar Benjamin <oscar.j.benjamin at gmail.com> writes:

> According to a narrow definition of indexed access. I would say that
> d[k] is index access even if d is a dict and k a key.

An index implies the ordinal position in a sequence. In a mapping, the
key is *not* referring to the position in a sequence, so is not a key.

So accessing an item in a mapping by key is not indexed access.

-- 
 \         ?When we pray to God we must be seeking nothing ? nothing.? |
  `\                                                ?Francis of Assisi |
_o__)                                                                  |
Ben Finney


From jvillar1 at andrew.cmu.edu  Thu Jan 21 11:51:09 2016
From: jvillar1 at andrew.cmu.edu (John Villarraga)
Date: Thu, 21 Jan 2016 11:51:09 -0500
Subject: [Tutor] How to call the path of an input file
Message-ID: <CAOJgn8qzw94u7CAA5Y5yRi5yyrcsAETuuQf78fBD1s145cWkmg@mail.gmail.com>

I would like to know what is the syntax to call the path of the input file.
Below, my code is calling the input file, but not the path.

Sorry for the inconvenience and thank you for your time.


import sys


path = sys.argv[1]


y = map(str.lower, path.split())

From ben+python at benfinney.id.au  Thu Jan 21 12:12:08 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 22 Jan 2016 04:12:08 +1100
Subject: [Tutor] Why is an OrderedDict not sliceable?
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
 <857fj3mgrr.fsf@benfinney.id.au>
 <CAHVvXxTkthTWEmJ6Eum=u1-eUT=ZdpF97TyJW3UY9PpuO6K7dg@mail.gmail.com>
 <8537tqna05.fsf@benfinney.id.au>
Message-ID: <85y4biluvr.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> Oscar Benjamin <oscar.j.benjamin at gmail.com> writes:
>
> > According to a narrow definition of indexed access. I would say that
> > d[k] is index access even if d is a dict and k a key.

The sense of ?index? implied is used consistently throughout Python
<URL:https://docs.python.org/3/glossary.html> to refer to the integer
ordinal position in a sequence.

It is not compatible with key access into a mapping.

> An index implies the ordinal position in a sequence. In a mapping, the
> key is *not* referring to the position in a sequence, so is not a key.

?the key ? is not an index?, I mean.

> So accessing an item in a mapping by key is not indexed access.

-- 
 \     ?Facts do not cease to exist because they are ignored.? ?Aldous |
  `\                                                            Huxley |
_o__)                                                                  |
Ben Finney


From martin at linux-ip.net  Thu Jan 21 12:12:29 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Thu, 21 Jan 2016 09:12:29 -0800
Subject: [Tutor] How to call the path of an input file
In-Reply-To: <CAOJgn8qzw94u7CAA5Y5yRi5yyrcsAETuuQf78fBD1s145cWkmg@mail.gmail.com>
References: <CAOJgn8qzw94u7CAA5Y5yRi5yyrcsAETuuQf78fBD1s145cWkmg@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1601210909370.2025@znpeba.jbaqresebt.arg>


Greetings John,

>I would like to know what is the syntax to call the path of the 
>input file. Below, my code is calling the input file, but not the 
>path.

>import sys
>path = sys.argv[1]

Are you looking for os.path.abspath() [0]?

  import os
  import sys
  path = os.path.abspath(sys.argv[1])

>y = map(str.lower, path.split())

Next question:  What exactly are you trying to do with that third 
line?  It looks confused.

Good luck,

-Martin

 [0] https://docs.python.org/3/library/os.path.html#os.path.abspath

-- 
Martin A. Brown
http://linux-ip.net/

From ben+python at benfinney.id.au  Thu Jan 21 12:14:49 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 22 Jan 2016 04:14:49 +1100
Subject: [Tutor] How to call the path of an input file
References: <CAOJgn8qzw94u7CAA5Y5yRi5yyrcsAETuuQf78fBD1s145cWkmg@mail.gmail.com>
Message-ID: <85twm6lura.fsf@benfinney.id.au>

John Villarraga <jvillar1 at andrew.cmu.edu> writes:

> I would like to know what is the syntax to call the path of the input
> file.

It's not clear to me what ?call the path of the input file? would mean.

A filesystem path is not ?callable? in a programming as you seem to be
using it.

> Below, my code is calling the input file, but not the path.

Can you please ensure your messages are plain text (not re-formatted or
marked up), to help your program code survive unmolested.

-- 
 \     ?Men never do evil so completely and cheerfully as when they do |
  `\        it from religious conviction.? ?Blaise Pascal (1623?1662), |
_o__)                                                   Pens?es, #894. |
Ben Finney


From steve at pearwood.info  Thu Jan 21 18:18:51 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 22 Jan 2016 10:18:51 +1100
Subject: [Tutor] How to call the path of an input file
In-Reply-To: <CAOJgn8qzw94u7CAA5Y5yRi5yyrcsAETuuQf78fBD1s145cWkmg@mail.gmail.com>
References: <CAOJgn8qzw94u7CAA5Y5yRi5yyrcsAETuuQf78fBD1s145cWkmg@mail.gmail.com>
Message-ID: <20160121231851.GG4619@ando.pearwood.info>

Hi John, and welcome. My responses interleaved between yours below.


On Thu, Jan 21, 2016 at 11:51:09AM -0500, John Villarraga wrote:

> I would like to know what is the syntax to call the path of the input file.

Python only has one syntax for calling anything, and that is to put 
parentheses (round brackets) after it, with any arguments needed inside 
the parens. So this is how you would call the path:

path()

But that's not going to do you any good, since path is surely going to 
be a string, and strings aren't callable: you'll just get a TypeError:


py> path = "directory/file.txt"
py> path()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'str' object is not callable


So would you like to clarify what you actually mean by "call the path"?



> Below, my code is calling the input file, but not the path.
> 
> Sorry for the inconvenience and thank you for your time.
> 
> 
> import sys
> path = sys.argv[1]
> y = map(str.lower, path.split())


I *think* what you mean is that you have a python script, let's call it 
"script.py", and you run that script like this:

python script.py


and now you want to add a path (a path to what? a file?) as a command 
line argument:

python script.py /some/directory/file.txt


And then what is supposed to happen?

I'm going to guess that you want to take the argument given:

path = "/some/directory/file.txt"

and do something to it, not sure what. Perhaps normalise the case?


import os
path = os.path.normcase(path)


should do what you want. You can read up on the functions available in 
the os.path module here:

For version 2:

https://docs.python.org/2/library/os.path.html


For version 3:

https://docs.python.org/3/library/os.path.html


-- 
Steve

From steve at pearwood.info  Thu Jan 21 19:00:00 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 22 Jan 2016 11:00:00 +1100
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
Message-ID: <20160122000000.GI4619@ando.pearwood.info>

Further thoughts on your question...


On Wed, Jan 20, 2016 at 01:33:17PM +0000, Albert-Jan Roskam wrote:
> Hi,
> 
> Like the subject says: Why is an OrderedDict not sliceable? (From the 
> collections library). Was that an intentional omission, or a mistake? 
> [1]
> 
> Background: I do not use OrderedDict very often, but I thought I could 
> use it to look up street and city names using postcodes ([0-9]{4} 
> [a-z]{2} format). I needed it to be ordered because I also wanted to 
> be able to use bisect, which is needed when the postcode letters are 
> missing. In short: a fast dict lookup for complete postcodes and less 
> fast bisect lookup for in complete postcodes.

I'm not sure I understand your use-case here.

You have postcodes that look like this:

"1234az"

Correct? Why do you want them *ordered*? 

I think you are confusing OrderedDict for a "Sorted Dict". OrderedDict 
doesn't keep the keys in sorted order, it keeps them in the order that 
they were inserted. So unless you are super-careful to insert the 
postcodes in sorted order, the order of them in the dict will be 
whatever order you insert them:

py> from collections import OrderedDict
py> d = OrderedDict()
py> d['1234az'] = "1 Smith Street"
py> d['9999zz'] = "991203 Short Street"
py> d['3456mx'] = "24 Hour Lane"
py> for key in d:
...     print(key, d[key])
...
1234az 1 Smith Street
9999zz 991203 Short Street
3456mx 24 Hour Lane


So even if OrderedDict supported slicing, that would not do what you 
think it does.


Also, you have a problem -- what happens if the incomplete postcode is 
missing the first digit? (I suppose for that case, you just have to do a 
slow linear search.) What about transposed digits or other errors? I'm 
glad I don't have to solve that problem!


Anyway, I suggest doing something like this:

(1) Keep the known postcodes in a regular dict, not an ordered dict.

(2) Once you have built the dict, then copy the keys and sort them:

postcodes = {
    '1234az': "1 Smith Street",
    '9999zz': "991203 Short Street",
    '3456mx': "24 Hour Lane",
    }

array = sorted(postcodes.keys())


(3) Each time you add a new postcode to the dict, use bisect to add it 
to the array as well. To ensure you never forget, use a helper function:

def insert(postcode, entry):
    if postcode in postcodes:
        # deal with duplicate/existing key
        ...
    else:
        postcodes[postcode] = entry
        bisect.insort(array, postcode)


Same for deleting.



-- 
Steve

From alan.gauld at btinternet.com  Thu Jan 21 19:12:18 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 22 Jan 2016 00:12:18 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <20160122000000.GI4619@ando.pearwood.info>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
 <20160122000000.GI4619@ando.pearwood.info>
Message-ID: <n7rs52$n3s$1@ger.gmane.org>

On 22/01/16 00:00, Steven D'Aprano wrote:

> Also, you have a problem -- what happens if the incomplete postcode is 
> missing the first digit? (I suppose for that case, you just have to do a 
> slow linear search.) 

Which is why a different solution may be better suited.
What about an in-memory SQLite table? Then you can use
a LIKE based select and let the library do the hard work.

Just a thought.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From breamoreboy at yahoo.co.uk  Thu Jan 21 16:02:03 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 21 Jan 2016 21:02:03 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
Message-ID: <n7rh0d$6tn$1@ger.gmane.org>

On 20/01/2016 13:33, Albert-Jan Roskam wrote:
> Hi,
>
> Like the subject says: Why is an OrderedDict not sliceable? (From the collections library). Was that an intentional omission, or a mistake? [1]

Plenty of answers on this all ready, but...

>
> Background: I do not use OrderedDict very often, but I thought I could use it to look up street and city names using postcodes ([0-9]{4} [a-z]{2} format). I needed it to be ordered because I also wanted to be able to use bisect, which is needed when the postcode letters are missing. In short: a fast dict lookup for complete postcodes and less fast bisect lookup for in complete postcodes.
>

You appear to be confusing ordered and sorted.  There is no way that you 
can use bisect on an OrderedDict unless it is sorted in the first place.

> [1] http://stackoverflow.com/questions/30975339/slicing-a-python-ordereddict
>
> Thanks!
>
> Albert-Jan
>   		 	   		

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From arjuns123 at gmail.com  Thu Jan 21 17:06:19 2016
From: arjuns123 at gmail.com (Arjun Srivatsa)
Date: Thu, 21 Jan 2016 23:06:19 +0100
Subject: [Tutor] No connection could be made because the target machine
 actively refused it (despite mongod running) and unable to insert data into
 MongoDB
Message-ID: <CAOnfC-Vv0uN1YsfXYeoV90he2CAjyi+7OEwZge2uOgwzNYiOfQ@mail.gmail.com>

Hello everyone,

I have been trying to insert data (a 'hello world' string) from PLC into
MongoDB using Python API (which pulls the data from PLC and pushes it into
MongoDB). I have been getting the error message '*line 222, in meth  return
getattr(self._sock,name)(*args) error: [Errno 10061] No connection could be
made because the target machine actively refused it*' despite having mongod
running in the Services background for the code I have written below. Also,
the server IP address on which MongoDB is present is 10.52.124.186 and
address of PLC (which I am using it on my PC) is 10.52.124.135. I am have
tried almost everything to sort it out and yet I haven't got a clue as to
how to get past it. Where am I going wrong?







#!/usr/bin/python


import socket

import socket

import pymongo

from pymongo import MongoClient

import datetime


# Connection to server (PLC) on port 27017

server = socket.socket()

host = '10.52.124.135'

port = 27017


server.connect((host, port))

print server.recv(1024)


server.close


#Connection to Client (Mongodb) on port 27017

IP = '10.52.124.186'

PORT = 27017

BUFFER_SIZE = 1024


client = MongoClient('10.52.124.186', 27017)

db = client.RXMMongoDB


s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

s.bind((IP, PORT))

s.listen(1)


#connections loop

while True:

conn, addr = s.accept()

print 'Connection address:',addr

try:

# read loop

while True:

data = server.recv(BUFFER_SIZE)


if not data:

break

# send to Mongo

mongodoc = { "data": data, "date" : datetime.datetime.utcnow() }

db.AAAA.insert(mongodoc)

finally:

conn.close()





-- 
Mit freundlichen Gr??en / Thanks and Regards,


*Arjun Srivatsa*

From esawiek at gmail.com  Thu Jan 21 23:27:34 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Thu, 21 Jan 2016 23:27:34 -0500
Subject: [Tutor] Extract several arrays from a large 2D array
Message-ID: <CA+ZkTxveDxUf86iSdhy5WodqKnwHx3LwSdtuym58g3Xb_rGymw@mail.gmail.com>

Thank you all for your help. I am a decent programmer in another language
but new to Python and I have some issues with a project I am working on.
Some suggested using pandas but I am barley starting on Numpy. The
suggestions were very helpful, however, I decided to replace the 2D array
with several single arrays b/c the arrays are of different data types. I
ran into another problems and am almost done but got stuck.



Part of my code is below. The question is how to put the variables for each
j into a 14 by 6 array by a statement at the end of this code. I was hoping
to get an array like this one below:

                     [2013 TT1 TT2 TT3 TT4 TT5 TT6]

                                                         [2012TT1 TT2 TT3
TT4 TT5 TT6]

                                                         .

                                                         .

                                                         [1999TT1 TT2 TT3
TT4 TT5 TT6]

for j in range(14)

    for i in range(200):

            if TYear[i]==2013-j:

                if TTrea[i]=='T1':

                    TT1+=TTemp[i]

                elif TTrea[i]=='T2':

                    TT2+=TTemp[i]

                elif TTrea[i]=='T3':

                    TT3+=TTemp[i]

                elif TTrea[i]=='T4':

                    TT4+=TTemp[i]

                elif TTrea[i]=='T5':

                    TT5+=TTemp[i]

                elif TTrea[i]=='T6':
                    TT6+=TTemp[i]

From alan.gauld at btinternet.com  Fri Jan 22 04:54:45 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 22 Jan 2016 09:54:45 +0000
Subject: [Tutor] No connection could be made because the target machine
 actively refused it (despite mongod running) and unable to insert data into
 MongoDB
In-Reply-To: <CAOnfC-Vv0uN1YsfXYeoV90he2CAjyi+7OEwZge2uOgwzNYiOfQ@mail.gmail.com>
References: <CAOnfC-Vv0uN1YsfXYeoV90he2CAjyi+7OEwZge2uOgwzNYiOfQ@mail.gmail.com>
Message-ID: <n7su95$d7e$1@ger.gmane.org>

On 21/01/16 22:06, Arjun Srivatsa wrote:
> Hello everyone,
> 
> I have been trying to insert data (a 'hello world' string) from PLC into
> MongoDB using Python API (which pulls the data from PLC and pushes it into
> MongoDB). I have been getting the error message '*line 222, in meth  return
> getattr(self._sock,name)(*args) error: [Errno 10061] No connection could be
> made because the target machine actively refused it*' despite having mongod
> running in the Services background for the code I have written below. Also,
> the server IP address on which MongoDB is present is 10.52.124.186 and
> address of PLC (which I am using it on my PC) is 10.52.124.135. I am have
> tried almost everything to sort it out and yet I haven't got a clue as to
> how to get past it. Where am I going wrong?

This looks more like a MongoDB issue than a Python one.

Can you connect from the same machine using the Mongo
command line tools and options?

You might have more success asking on a Mongo support forum/list.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From __peter__ at web.de  Fri Jan 22 05:03:43 2016
From: __peter__ at web.de (Peter Otten)
Date: Fri, 22 Jan 2016 11:03:43 +0100
Subject: [Tutor] Extract several arrays from a large 2D array
References: <CA+ZkTxveDxUf86iSdhy5WodqKnwHx3LwSdtuym58g3Xb_rGymw@mail.gmail.com>
Message-ID: <n7suq1$po5$1@ger.gmane.org>

Ek Esawi wrote:

> Thank you all for your help. I am a decent programmer in another language
> but new to Python and I have some issues with a project I am working on.
> Some suggested using pandas but I am barley starting on Numpy. The
> suggestions were very helpful, however, I decided to replace the 2D array
> with several single arrays b/c the arrays are of different data types. I
> ran into another problems and am almost done but got stuck.
> 
> 
> 
> Part of my code is below. The question is how to put the variables for
> each j into a 14 by 6 array by a statement at the end of this code. I was
> hoping to get an array like this one below:
> 
>                      [2013 TT1 TT2 TT3 TT4 TT5 TT6]
> 
>                                                          [2012TT1 TT2 TT3
> TT4 TT5 TT6]
> 
>                                                          .
> 
>                                                          .
> 
>                                                          [1999TT1 TT2 TT3
> TT4 TT5 TT6]
> 
> for j in range(14)
> 
>     for i in range(200):

As a rule of thumb, if you are iterating over individual array entries in 
numpy you are doing something wrong ;)

> 
>             if TYear[i]==2013-j:
> 
>                 if TTrea[i]=='T1':
> 
>                     TT1+=TTemp[i]
> 
>                 elif TTrea[i]=='T2':
> 
>                     TT2+=TTemp[i]
> 
>                 elif TTrea[i]=='T3':
> 
>                     TT3+=TTemp[i]
> 
>                 elif TTrea[i]=='T4':
> 
>                     TT4+=TTemp[i]
> 
>                 elif TTrea[i]=='T5':
> 
>                     TT5+=TTemp[i]
> 
>                 elif TTrea[i]=='T6':
>                     TT6+=TTemp[i]

This looks like you are manually building a pivot table. If you don't want 
to use a spreadsheet (like Excel or Libre/OpenOffice Calc) you can do it 
with pandas, too:

import pandas
import numpy
df = pandas.DataFrame(
    [
        [2013, "T1", 42],
        [2013, "T2", 12],
        [2012, "T1", 1],
        [2012, "T1", 2],
        [2012, "T2", 10],
        [2012, "T3", 11],
        [2012, "T4", 12],
        ],
    columns=["year", "t", "v"])

print(
    pandas.pivot_table(df, cols="t", rows="year", aggfunc=numpy.sum)
)

Here's a generic Python solution for educational purposes. It uses nested 
dicts to model the table. The outer dict is used for the rows, the inner 
dicts for the cells in a row. An extra set keeps track of the column labels.

data = [
        [2013, "T1", 42],
        [2013, "T2", 12],
        [2012, "T1", 1],
        [2012, "T1", 2],
        [2012, "T2", 10],
        [2012, "T3", 11],
        [2012, "T4", 12],
]

table = {}
columns = set()
for year, t, v in data:
    columns.add(t)
    row = table.setdefault(year, {})
    row[t] = row.get(t, 0) + v

columns = sorted(columns)
print("    ", *["{:>5}".format(c) for c in  columns])
for year, row in sorted(table.items(), reverse=True):
    print(year, *["{:5}".format(row.get(c, 0)) for c in columns])



From emile at fenx.com  Fri Jan 22 09:37:19 2016
From: emile at fenx.com (Emile van Sebille)
Date: Fri, 22 Jan 2016 06:37:19 -0800
Subject: [Tutor] No connection could be made because the target machine
 actively refused it (despite mongod running) and unable to insert data into
 MongoDB
In-Reply-To: <CAOnfC-Vv0uN1YsfXYeoV90he2CAjyi+7OEwZge2uOgwzNYiOfQ@mail.gmail.com>
References: <CAOnfC-Vv0uN1YsfXYeoV90he2CAjyi+7OEwZge2uOgwzNYiOfQ@mail.gmail.com>
Message-ID: <n7term$ohh$1@ger.gmane.org>

On 1/21/2016 2:06 PM, Arjun Srivatsa wrote:
> Hello everyone,
>
> I have been trying to insert data (a 'hello world' string) from PLC into
> MongoDB using Python API (which pulls the data from PLC and pushes it into
> MongoDB). I have been getting the error message '*line 222, in meth  return
> getattr(self._sock,name)(*args) error: [Errno 10061] No connection could be
> made because the target machine actively refused it*' despite having mongod
> running in the Services background for the code I have written below. Also,
> the server IP address on which MongoDB is present is 10.52.124.186 and
> address of PLC (which I am using it on my PC) is 10.52.124.135. I am have
> tried almost everything to sort it out and yet I haven't got a clue as to
> how to get past it. Where am I going wrong?


The error you're getting means that the firewall running on the mongod 
host machine is not properly configured to allow mongod traffic into it. 
  Talk to the machines firewall administrator.

Emile



From skirwin at outlook.com  Fri Jan 22 12:05:11 2016
From: skirwin at outlook.com (samuel kirwin)
Date: Fri, 22 Jan 2016 17:05:11 +0000
Subject: [Tutor] Tutor Digest, Vol 143, Issue 77
In-Reply-To: <mailman.7.1453482002.10668.tutor@python.org>
References: <mailman.7.1453482002.10668.tutor@python.org>
Message-ID: <BAY407-EAS1943352FC2BCEE11E45A21EAFC40@phx.gbl>

Semantic errors are when a program acts incorrectly and doesn't give a error, this came up yesterday.

Samuel Kirwin
Samvelk




On Fri, Jan 22, 2016 at 9:02 AM -0800, <tutor-request at python.org> wrote:





Send Tutor mailing list submissions to
        tutor at python.org

To subscribe or unsubscribe via the World Wide Web, visit
        https://mail.python.org/mailman/listinfo/tutor
or, via email, send a message with subject or body 'help' to
        tutor-request at python.org

You can reach the person managing the list at
        tutor-owner at python.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Tutor digest..."


Today's Topics:

   1. Re: No connection could be made because the target machine
      actively refused it (despite mongod running) and unable to insert
      data into MongoDB (Emile van Sebille)


----------------------------------------------------------------------

Message: 1
Date: Fri, 22 Jan 2016 06:37:19 -0800
From: Emile van Sebille <emile at fenx.com>
To: tutor at python.org
Subject: Re: [Tutor] No connection could be made because the target
        machine actively refused it (despite mongod running) and unable to
        insert data into MongoDB
Message-ID: <n7term$ohh$1 at ger.gmane.org>
Content-Type: text/plain; charset=utf-8; format=flowed

On 1/21/2016 2:06 PM, Arjun Srivatsa wrote:
> Hello everyone,
>
> I have been trying to insert data (a 'hello world' string) from PLC into
> MongoDB using Python API (which pulls the data from PLC and pushes it into
> MongoDB). I have been getting the error message '*line 222, in meth  return
> getattr(self._sock,name)(*args) error: [Errno 10061] No connection could be
> made because the target machine actively refused it*' despite having mongod
> running in the Services background for the code I have written below. Also,
> the server IP address on which MongoDB is present is 10.52.124.186 and
> address of PLC (which I am using it on my PC) is 10.52.124.135. I am have
> tried almost everything to sort it out and yet I haven't got a clue as to
> how to get past it. Where am I going wrong?


The error you're getting means that the firewall running on the mongod
host machine is not properly configured to allow mongod traffic into it.
  Talk to the machines firewall administrator.

Emile




------------------------------

Subject: Digest Footer

_______________________________________________
Tutor maillist  -  Tutor at python.org
https://mail.python.org/mailman/listinfo/tutor


------------------------------

End of Tutor Digest, Vol 143, Issue 77
**************************************

From starfas_s at yahoo.com  Fri Jan 22 13:16:11 2016
From: starfas_s at yahoo.com (Sam Starfas)
Date: Fri, 22 Jan 2016 18:16:11 +0000 (UTC)
Subject: [Tutor] Combine Scripts to Work Together...
In-Reply-To: <1997984413.9069447.1453486430709.JavaMail.yahoo@mail.yahoo.com>
References: <1997984413.9069447.1453486430709.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <1312612295.9052509.1453486571826.JavaMail.yahoo@mail.yahoo.com>




 Hi,I am new to Python, but learning. (Hopefully the layout is readable)
I am having trouble getting two scripts to work together. What I want to do is have the combinded script do the following:
1. Read through a directory of xml files, not a single file, but many xml files.2. Read each files contents.3. Pull out the <uicontrol> element content (words between open and close tag)
? ?* For example: <uicontrol>Sam was here.</uicontrol>
? ?* The script would pull out the content "Sam was here" and print it to a file.

My scripts so far:----------------------------------------------------------Script to get <uicontol> content----------------------------------------------------------import xml.etree.ElementTree as ET
tree = ET.parse('TEST.xml')
root = tree.getroot()for uicontrol in root.iter('uicontrol'):
???? print(uicontrol.text)
----------------------------------------------------------Script to read through the files:----------------------------------------------------------import sys
import glob
import errnopath ='/Users/sastarfa/Documents/Projects/Script-projects/Scripts/Pull-out-terms-between-elements/*.xml'?? files = glob.glob(path)??
for name in files: # 'file' is a builtin type, 'name' is a less-ambiguousvariable name.
???? try:
?????????? with open(name) as f: # Noneed to specify 'r': this is the default.
?????????? sys.stdout.write(f.read())

This is the area I get stuck. I can understand, write a script to run on one a single file you input into the script, but getting a script to run on a directory I am not understanding how to do this.
Thanks in advance for any and all help.Really appreciate the help.
Sam

  

From alan.gauld at btinternet.com  Fri Jan 22 13:32:41 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 22 Jan 2016 18:32:41 +0000
Subject: [Tutor] Semantic error (was: Re: Tutor Digest, Vol 143, Issue 77)
In-Reply-To: <BAY407-EAS1943352FC2BCEE11E45A21EAFC40@phx.gbl>
References: <mailman.7.1453482002.10668.tutor@python.org>
 <BAY407-EAS1943352FC2BCEE11E45A21EAFC40@phx.gbl>
Message-ID: <n7tsk9$t97$1@ger.gmane.org>

On 22/01/16 17:05, samuel kirwin wrote:
> Semantic errors are when a program acts incorrectly and doesn't give a error, this came up yesterday.
> 
> Samuel Kirwin

Hi Sam, Thanks for participating.

However, it will make your comments more meaningful if you
reply with the subject line set to whatever tread you were
responding to. Also if using the digest please delete all
irrelevant messages/content. That way we can see the context.
We've all seen them already and some of us pay by the byte.

As it is, the above statement just seems like a fairly random
comment. After all, the word semantic does not occur anywhere
within the glossary messages you forwarded.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From joel.goldstick at gmail.com  Fri Jan 22 13:48:07 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Fri, 22 Jan 2016 13:48:07 -0500
Subject: [Tutor] Combine Scripts to Work Together...
In-Reply-To: <1312612295.9052509.1453486571826.JavaMail.yahoo@mail.yahoo.com>
References: <1997984413.9069447.1453486430709.JavaMail.yahoo@mail.yahoo.com>
 <1312612295.9052509.1453486571826.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <CAPM-O+xEpm=cw1rbd96874RVDcf9fpTof7+F0T7BydDnum5ZNg@mail.gmail.com>

On Fri, Jan 22, 2016 at 1:16 PM, Sam Starfas via Tutor <tutor at python.org>
wrote:

>
>
>
>  Hi,I am new to Python, but learning. (Hopefully the layout is readable)
> I am having trouble getting two scripts to work together. What I want to
> do is have the combinded script do the following:
> 1. Read through a directory of xml files, not a single file, but many xml
> files.2. Read each files contents.3. Pull out the <uicontrol> element
> content (words between open and close tag)
>    * For example: <uicontrol>Sam was here.</uicontrol>
>    * The script would pull out the content "Sam was here" and print it to
> a file.
>
> My scripts so
> far:----------------------------------------------------------Script to get
> <uicontol>
> content----------------------------------------------------------import
> xml.etree.ElementTree as ET
> tree = ET.parse('TEST.xml')
> root = tree.getroot()for uicontrol in root.iter('uicontrol'):
>      print(uicontrol.text)
>

Make the above a function
def get_uicontrol(f):

> tree = ET.parse(f))
> root = tree.getroot()for uicontrol in root.iter('uicontrol'):
>      print(uicontrol.text)
>

----------------------------------------------------------Script to read
> through the
> files:----------------------------------------------------------import sys
> import glob
> import errnopath
> ='/Users/sastarfa/Documents/Projects/Script-projects/Scripts/Pull-out-terms-between-elements/*.xml'
> files = glob.glob(path)
> for name in files: # 'file' is a builtin type, 'name' is a
> less-ambiguousvariable name.
>      try:
>            with open(name) as f: # Noneed to specify 'r': this is the
> default.
>            sys.stdout.write(f.read())
>
>
Make the above a function:

def get_files(path):
    glob.glob(path)
    for name in files:
         try:
              with open(name) as f
              get_uicontrol(f)

then at the end of your file do this:

if __name__ == '__main__':
    get_files('/your/path/goes/here')

get_files will loop to open each file, then call get_uicontrol on that open
file.  get_uicontrol will find your tag, collect the text and print it

I haven't tested this, but it is at least close


> This is the area I get stuck. I can understand, write a script to run on
> one a single file you input into the script, but getting a script to run on
> a directory I am not understanding how to do this.
> Thanks in advance for any and all help.Really appreciate the help.
> Sam
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From joel.goldstick at gmail.com  Fri Jan 22 13:49:46 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Fri, 22 Jan 2016 13:49:46 -0500
Subject: [Tutor] Combine Scripts to Work Together...
In-Reply-To: <CAPM-O+xEpm=cw1rbd96874RVDcf9fpTof7+F0T7BydDnum5ZNg@mail.gmail.com>
References: <1997984413.9069447.1453486430709.JavaMail.yahoo@mail.yahoo.com>
 <1312612295.9052509.1453486571826.JavaMail.yahoo@mail.yahoo.com>
 <CAPM-O+xEpm=cw1rbd96874RVDcf9fpTof7+F0T7BydDnum5ZNg@mail.gmail.com>
Message-ID: <CAPM-O+wdS-VCSX6L-_9Pq0XB3wsRtU2FmkWFhSJsVnB05aWzkQ@mail.gmail.com>

On Fri, Jan 22, 2016 at 1:48 PM, Joel Goldstick <joel.goldstick at gmail.com>
wrote:

>
>
> On Fri, Jan 22, 2016 at 1:16 PM, Sam Starfas via Tutor <tutor at python.org>
> wrote:
>
>>
>>
>>
>>  Hi,I am new to Python, but learning. (Hopefully the layout is readable)
>> I am having trouble getting two scripts to work together. What I want to
>> do is have the combinded script do the following:
>> 1. Read through a directory of xml files, not a single file, but many xml
>> files.2. Read each files contents.3. Pull out the <uicontrol> element
>> content (words between open and close tag)
>>    * For example: <uicontrol>Sam was here.</uicontrol>
>>    * The script would pull out the content "Sam was here" and print it to
>> a file.
>>
>> My scripts so
>> far:----------------------------------------------------------Script to get
>> <uicontol>
>> content----------------------------------------------------------import
>> xml.etree.ElementTree as ET
>> tree = ET.parse('TEST.xml')
>> root = tree.getroot()for uicontrol in root.iter('uicontrol'):
>>      print(uicontrol.text)
>>
>
> Make the above a function
> def get_uicontrol(f):
>
>> tree = ET.parse(f))
>> root = tree.getroot()for uicontrol in root.iter('uicontrol'):
>>      print(uicontrol.text)
>>
>
> ----------------------------------------------------------Script to read
>> through the
>> files:----------------------------------------------------------import sys
>> import glob
>> import errnopath
>> ='/Users/sastarfa/Documents/Projects/Script-projects/Scripts/Pull-out-terms-between-elements/*.xml'
>> files = glob.glob(path)
>> for name in files: # 'file' is a builtin type, 'name' is a
>> less-ambiguousvariable name.
>>      try:
>>            with open(name) as f: # Noneed to specify 'r': this is the
>> default.
>>            sys.stdout.write(f.read())
>>
>>
> Make the above a function:
>
> def get_files(path):
>     glob.glob(path)
>     for name in files:
>          try:
>               with open(name) as f:
>                   get_uicontrol(f)
>
> oops, edited above two lines


> then at the end of your file do this:
>
> if __name__ == '__main__':
>     get_files('/your/path/goes/here')
>
> get_files will loop to open each file, then call get_uicontrol on that
> open file.  get_uicontrol will find your tag, collect the text and print it
>
> I haven't tested this, but it is at least close
>
>
>> This is the area I get stuck. I can understand, write a script to run on
>> one a single file you input into the script, but getting a script to run on
>> a directory I am not understanding how to do this.
>> Thanks in advance for any and all help.Really appreciate the help.
>> Sam
>>
>>
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
>
>
> --
> Joel Goldstick
> http://joelgoldstick.com/stats/birthdays
>



-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From akleider at sonic.net  Fri Jan 22 15:27:24 2016
From: akleider at sonic.net (Alex Kleider)
Date: Fri, 22 Jan 2016 12:27:24 -0800
Subject: [Tutor] mock
Message-ID: <3e75abcbcd65a7d11800d0214ab0ca72@sonic.net>

Some weeks (perhaps months) ago, I posted a question about testing
and got many responses but had trouble grasping the concepts
so I settled on the suggestion that I thought would be the easiest
to implement (using unittest.mock.) Here it is-

"""
from Peter Otten:
I find Ben's example instructive, but when you're just starting you
might prefer a simpler approach:

import unittest
from unittest import mock
import mycode

class TestCollectData(unittest.TestCase):
     def test(self):
         with mock.patch(
                 "builtins.input",
                 side_effect=["foo", "bar", "baz"]):
             self.assertEqual(
                 mycode.collect_data(),
                 dict(first="foo", last="bar", phone="baz"))

if __name__ == "__main__":
     unittest.main()
"""

I've successfully implemented mock.patch parameters but don't
understand how to implement the parameters for the assertEqual
call as it pertains to my particular situation.

My current code sort of does the job but determinating success or
failure is not really automated.

It's Python 3 on Ubuntu 14.4LTS.
My test code and the tested code follow.  Any suggestions would be
most welcome.  Thanks in advance.
Alex

Test Code:
#!../../venv/bin/python
# -*- coding: utf-8 -*-
# vim: set file encoding=utf-8
"""
Attempt to automate testing of menu module.
"""
import unittest
from unittest import mock
import menu

class MenuTests(unittest.TestCase):

     def test_create_new(self):
         with mock.patch("builtins.input",
                 side_effect=['1', 'testentity', '', '0']):
             menu.menu()

if __name__ == '__main__':  # code block to run the application
     unittest.main()


Tested code:

#!../../venv/bin/python
# -*- coding: utf-8 -*-
# vim: set file encoding=utf-8
"""
A simple menu model used to format a question
about how to test using unittest.mock.
"""

class Entities(object):
     """
     Keep track of the entities including a default.
     Expect only one instance which will be a global.
     """

     def __init__(self, list_of_entities, default=''):
         self._list = list_of_entities
         self.default = default

     def get_default(self):
         return self.default

     def reset_default(self, default=''):
         self.default = default

     def get_list(self):
         return self._list

     def add(self, new_entity, set_default=False):
         """
         Adds another entity returning it to signal success.
         Returns None if entity is unacceptable.
         """
         if (not new_entity in self._list
         and new_entity.isalnum()
         and new_entity[0:1].isalpha()):
             self._list.append(new_entity)
             if set_default:
                 self.reset_default(new_entity)
             return new_entity
         else:
             print("Failing to add an invalid entity: '{}'."
                         .format(new_entity))

     def get_new_entity(self):
         """
         Prompts user for a new entity which, if valid, is
         returned after being appended to list and set as default.
         Returns None if fails to create a new entity.
         """
         while True:
             new_entity = input("Pick name for new entity: ")
             if not new_entity: return
             if new_entity != self.add(new_entity, set_default=True):
                 continue
             else:
                 return new_entity

     def remove(self, entity2remove):
         if not entity2remove in self.get_list():
             print("Can't remove '{}': not in the list."
                         .format(entity2remove))
             return
         if entity2remove == self.get_default():
             self.reset_default()
         self._list = [entity for entity in self._list
                         if entity!=entity2remove]

     def show_listing(self):
         """
         Returns a string listing the available entities.
         Empty string if there are no entities.
         """
         return ''.join(["\n\t    {}".format(entity)
                         for entity in self.get_list()])

     def entity_choice(self, set_default=False):
         """
         Prompts the user to choose from the list.
         Returns a valid choice or None.
         Can optionally set the default to the chosen entity.
         """
         list_of_entities= self.get_list()
         if not list_of_entities:
             print("There are no entities from which to choose.")
             return
         default_line = ''
          if self.default:
             default_line = ("\n\t_: Default is '{}', just hit enter."
                                 .format(self.default))
         menu = ('\n'.join(["\t{}: {}".format(n, entity)
                 for (n, entity) in enumerate(list_of_entities, 1)])
                 + default_line)
         while True:
             option = input(
     """Choose one of the following:
     {}
     \t0: to exit.
     Pick an entity: """.format(menu))
             default = self.get_default()
             if (option=='' or option=='_') and default:
                 return default
             try:
                 option = int(option)
             except ValueError:
                 print("Invalid option: {}! (It must be an integer.)"
                         .format(option))
                 continue
             entity_list = self.get_list()
             if option in range(1, len(entity_list) + 1):
                 choice = entity_list[option - 1]
                 if set_default:
                     self.default = choice
                 return choice
             elif option == 0:
                 return None
             else:
                 print("Invalid entry- try again ('0' to quit.)")

def create_new(option, entities):
     """
     A main menu response function.
     """
     print("Picked '{}. Create a new entity.'".format(option))
     entity = entities.get_new_entity()
     if entity:
         print(
             "Entity '{}' successfully created (and set as default.)"
                 .format(entity))
         work_with(entity)
     else:
         print("Aborting entity creation.")

def choose_existing(option, entities):
     """
     A main menu response function.
     """
     print("Picked '{}'. Choose an entity."
             .format(option))
     choice = entities.entity_choice(set_default=True)
     if choice:
         work_with(choice)

def delete_option(option, entities):
     """
     A main menu response function.
     """
     print("Picked '{}. Delete an existing entity.'".format(option))
     while True:
         entity = entities.entity_choice()
         if not entity:
             print("Entity deletion aborted.")
             return
         y_n = input("About to delete entity '{}', ARE YOU SURE? "
                 .format(entity))
         if y_n and y_n[0] in 'Yy':
             print("Deleting entity '{}'.".format(entity))
             entities.remove(entity)
         else:
             print("No deletion being done.")
         break

def work_with(entity):
     """
     Provides an interface stub for once an entity has been selected.
     """
     _ = input("Stub of code to work with '{}' entity goes here."
                     .format(entity))

def menu():
     """
     Provides the top level user interface.
     """
     entities = Entities(["ent1", "ent2", "ent3"], "ent2")
     while True:
         listing = entities.show_listing()
         if listing:
             listing = (
"""\n(  Currently existing entities are: {}          )"""
                     .format(listing))
         option = input("""
Main Menu:{}
     1. Create a new entity.
     2. Choose an existing entity.
     9. Delete an entity.
     0. Exit
Choice: """
                         .format(listing))
         print("Main Menu choice: {}".format(option))
         if option in ('', '_', '0'):
             return
         try:
             option = int(option)
         except ValueError:
             print(
                 "Invalid main menu choice: {} (must be an integer.)"
                         .format(option))
             continue
         if option == 1:
             entity = create_new(option, entities)
         elif option == 2:
             entity = choose_existing(option, entities)
         elif option == 9:
             delete_option(option, entities)
             entity = ''
          else:
             print("BAD CHOICE '{}'- try again....".format(option))
             entity = None
         if entity:
             work_with(entity)

if __name__ == "__main__":
     menu()
~


From robertvstepp at gmail.com  Fri Jan 22 23:14:57 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Fri, 22 Jan 2016 22:14:57 -0600
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <20160121105708.GE4619@ando.pearwood.info>
References: <CANDiX9+U47s=pKQozVk6DCB9xHZSBtw84_H+jhnsswUGj31n1g@mail.gmail.com>
 <CAGZAPF4QCu7tAx_jZuWJ342e30Q92Dnyb1Rr_nbbgTMQrs0cdg@mail.gmail.com>
 <20160121105708.GE4619@ando.pearwood.info>
Message-ID: <CANDiX9LcTXwZdga7_pT_XcuuJW_Rb7bH+DS9gtMq6CurB5tkng@mail.gmail.com>

On Thu, Jan 21, 2016 at 4:57 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Wed, Jan 20, 2016 at 09:54:32PM -0800, Danny Yoo wrote:
>
>> Just to be explicit: you are pointing out that:
>>
>>      humdrum.sigh_strenght = 'high'
>>
>> was a typo, and it would have been nice if it could be caught as an error.
>>
>> If that's the case, then yes, I agree.  Unfortunately, this isn't a
>> runtime error because Python allows us to add arbitrary attributes to
>> objects: it's a deliberate design feature.
>
> Danny is correct. And it is a useful feature too. For instance, we can
> add attributes to functions:
>
> def spam(x, y):
>     ...
>
> spam.extra_info = "whatever"

A new thing that I did not suspect I could do.  This bothers me for two reasons:

    1)  It does not seem right adding attributes to functions outside
of its definition.

    2)  spam.extra_info appears to be global:

>>> spam.extra_info = "whatever"
>>> z = 'WOW!'
>>> def print_global_stuff():
        print('z =', z)
        print('spam.extra_info =', spam.extra_info)

>>> print_global_stuff()
z = WOW!
spam.extra_info = whatever

And I imagine I am being dense about something that is quite obvious
to you:  How is this a useful feature to have?  What does it give me
that is more useful than just saying something like:

just_another_global variable = "whatever"

?

And what bothered me about my original example that started this
thread is that when my typo

humdrum.sigh_strenght = 'high'

was accepted and did not generate an error, it seemed to mean to me
that I was violating my object's data encapsulation.  It just seems to
me that I should not be able to arbitrarily add new attributes from
outside the class definition that created my object.  That seems
similar to having a class Dog, to which from outside the class'
definition, I decide to add a new Dog attribute that all dogs in this
class can now have a tail sticking out of their noses.  I have no
qualms about subclassing Dog with a MutantDog subclass, but adding new
attributes from outside the class definition?  That just does not seem
right to at this point of my understanding.  As in your function
example, I am sure that I am missing something quite obvious, but I am
just not seeing it now.

Keep in mind, that I am just now this past week starting to study
classes.  All of my programming experience has been strictly
procedural Other than dabbling in Tkinter at work.).  I have no
pre-experience with Java or C++, other than casual reading, so any
preconceptions I have come from my procedural background ages ago in
FORTRAN.


> Other advantages of this way of doing things include that you aren't
> forced to put all your initialisation code in a single, special place,
> you can factor parts of it out into alternative methods:
>
> class X:
>    def __init__(self, x):
>        self.setup(x)
>    def setup(self, x):
>        process(x)
>        self.attribute = x
>

I have seen this before and agree that this is both useful and
desirable.  But I am comfortable with this because it is happening
*inside* the class definition.

> It also means that Python doesn't require a magic "undefined" value,
> like Javascript has, or require declarations ahead of time, like static
> languages such as Pascal and C require.

I have come to enjoy this feature of Python as well!

boB

From cs at zip.com.au  Sat Jan 23 00:04:50 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 23 Jan 2016 16:04:50 +1100
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9LcTXwZdga7_pT_XcuuJW_Rb7bH+DS9gtMq6CurB5tkng@mail.gmail.com>
References: <CANDiX9LcTXwZdga7_pT_XcuuJW_Rb7bH+DS9gtMq6CurB5tkng@mail.gmail.com>
Message-ID: <20160123050450.GA38370@cskk.homeip.net>

On 22Jan2016 22:14, boB Stepp <robertvstepp at gmail.com> wrote:
>On Thu, Jan 21, 2016 at 4:57 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> Danny is correct. And it is a useful feature too. For instance, we can
>> add attributes to functions:
>>
>> def spam(x, y):
>>     ...
>>
>> spam.extra_info = "whatever"
>
>A new thing that I did not suspect I could do.  This bothers me for two reasons:
>
>    1)  It does not seem right adding attributes to functions outside
>of its definition.

Have you tried to do it from inside? Anyway, a function is just another object. 
What do you think it should be particularly special?

>    2)  spam.extra_info appears to be global:
>
>>>> spam.extra_info = "whatever"
>>>> z = 'WOW!'
>>>> def print_global_stuff():
>        print('z =', z)
>        print('spam.extra_info =', spam.extra_info)

No, "spam" is global. So you can name it anywhere in that module; therefore you 
can name anything inside it. Again, like any other object.

>>>> print_global_stuff()
>z = WOW!
>spam.extra_info = whatever
>
>And I imagine I am being dense about something that is quite obvious
>to you:  How is this a useful feature to have?  What does it give me
>that is more useful than just saying something like:
>
>just_another_global variable = "whatever"

spam.extra_info = "blah"
snot = spam
print(snot.extra_info)

Consider if you passed "snot" (or "spam") to a function:

  def print_extra_info(obj):
    print(obj.extra_info)

This is not something you could do with just_another_global_variable.

Consider: this is information you want assicated with a specific object.  
Therefore it really _is_ an arribute of the object so that it can follow it 
around.

>And what bothered me about my original example that started this
>thread is that when my typo
>
>humdrum.sigh_strenght = 'high'
>
>was accepted and did not generate an error, it seemed to mean to me
>that I was violating my object's data encapsulation.  It just seems to
>me that I should not be able to arbitrarily add new attributes from
>outside the class definition that created my object.  That seems
>similar to having a class Dog, to which from outside the class'
>definition, I decide to add a new Dog attribute that all dogs in this
>class can now have a tail sticking out of their noses.  I have no
>qualms about subclassing Dog with a MutantDog subclass, but adding new
>attributes from outside the class definition?  That just does not seem
>right to at this point of my understanding.  As in your function
>example, I am sure that I am missing something quite obvious, but I am
>just not seeing it now.

All you're missing is realising that setting an attribute is not a special 
operation.

To take an anecdote from elsewhere:

  UNIX was not designed to stop you from doing stupid things, because that 
  would also stop you from doing clever things.   - Doug Gwyn

There's no need to prevent you setting "sigh_strenght". There probably is 
benefit in linting tools reporting to you about attributes which are set but 
apparently never used.

Cheers,
Cameron Simpson <cs at zip.com.au>

From robertvstepp at gmail.com  Sat Jan 23 00:10:39 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Fri, 22 Jan 2016 23:10:39 -0600
Subject: [Tutor] Why is the name "self" optional instead of mandatory?
In-Reply-To: <20160121114922.GF4619@ando.pearwood.info>
References: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
 <20160121114922.GF4619@ando.pearwood.info>
Message-ID: <CANDiX9JfY1ACZXwJBazz_tQXgEg--6n5Z26-NREgaEUoczyVfg@mail.gmail.com>

On Thu, Jan 21, 2016 at 5:49 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Wed, Jan 20, 2016 at 09:42:29PM -0600, boB Stepp wrote:
>
>> So I really only have one question:  Why not make Python's
>> *traditional* name, "self", mandatory?  Why give the programmer this
>> kind of choice?  [OK, that was two questions.]
>
> Why bother making it mandatory? That just makes more work for the
> compiler -- it has to decide that a function is inside a class, and
> therefore apply a restriction to the first argument. Most of the time,
> giving the programmer more freedom is less work.
>
> And what happens if you do this?
>
> class X:
>     pass
>
> def func(this):
>     print("instance %r called method" % this)
>
> X.method = func

Am I understanding this correctly?  It appears to me that you have
successfully added a method to class X from outside of class X!  If
this is the case, then this clarifies some things I was wondering
about in my other thread.

> And what are we supposed to do with classmethods and staticmethods? They
> shouldn't take a "self" argument at all, but they start off life as a
> regular function, just like ordinary instance methods.
>
> And of course, there are Metaclasses. You might need a metaclass method
> that needs to deal with all three levels of the hierarchy: the
> metaclass, the class it creates, and the instance of that class.
>
> class Meta(type):
>     def metamethod(meta, cls, self):
>         ...

I scanned the article that Danny gave a link to.  I am not ready to go
in depth into these concepts at this time, but I cannot help but
wonder about the possibilities these concepts open up?  At work I find
myself writing Python to generate correct scripting language commands
for the proprietary software I use at work.  What if I could write a
Python program which thoroughly captured the entire proprietary
scripting language into a much easier to use interface?  Their
scripting language has an underlying OO structure that is not exposed
to my prying eyes.  But *if* I could create a class structure that was
sufficiently similar to theirs that it seamlessly generated the
correct messages that their OOP style uses, then things might be much
easier for my projects.  Hmm...

I guess I am trying to wrap my mind around this incredible power and
flexibility that Python provides.  I thought I had asked a relatively
harmless question.  But it generated some strong responses!  It seemed
like "self" had a similar utility of use as "print" to me.  After all,
we can't redefine "print", can we?  But now I realize that I can do
exactly that if I so choose.  That is both fascinating and scary!

From robertvstepp at gmail.com  Sat Jan 23 02:52:26 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 23 Jan 2016 01:52:26 -0600
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <20160123050450.GA38370@cskk.homeip.net>
References: <CANDiX9LcTXwZdga7_pT_XcuuJW_Rb7bH+DS9gtMq6CurB5tkng@mail.gmail.com>
 <20160123050450.GA38370@cskk.homeip.net>
Message-ID: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>

On Fri, Jan 22, 2016 at 11:04 PM, Cameron Simpson <cs at zip.com.au> wrote:
> On 22Jan2016 22:14, boB Stepp <robertvstepp at gmail.com> wrote:

[...]

> Consider: this is information you want assicated with a specific object.
> Therefore it really _is_ an arribute of the object so that it can follow it
> around.
>
>> And what bothered me about my original example that started this
>> thread is that when my typo
>>
>> humdrum.sigh_strenght = 'high'
>>
>> was accepted and did not generate an error, it seemed to mean to me
>> that I was violating my object's data encapsulation.  It just seems to
>> me that I should not be able to arbitrarily add new attributes from
>> outside the class definition that created my object.  That seems
>> similar to having a class Dog, to which from outside the class'
>> definition, I decide to add a new Dog attribute that all dogs in this
>> class can now have a tail sticking out of their noses.  I have no
>> qualms about subclassing Dog with a MutantDog subclass, but adding new
>> attributes from outside the class definition?  That just does not seem
>> right to at this point of my understanding.  As in your function
>> example, I am sure that I am missing something quite obvious, but I am
>> just not seeing it now.
>
>
> All you're missing is realising that setting an attribute is not a special
> operation.

I guess no matter how new one is to OOP, one nevertheless brings one's
preconceptions, however malformed, into the learning process.  In my
case, one of mine was that once a class is coded, any given instance
of a class is forevermore responsible for managing its *internals*, so
that any change in these would be managed by the object when one of
its (meant to be) publicly available methods is called.  In the case
of attributes, I was thinking/expecting something like the following
would have to be done:

>>> class Dog(object):
        def __init__(self, name, breed):
            self.name = name
            self.breed = breed
            self.number_legs = 4
            self.number_tails = 1
            self.number_eyes = 2
            self.attributes = {
                'name': self.name,
                'breed': self.breed,
                'number of legs': self.number_legs,
                'number of tails': self.number_tails,
                'number of eyes': self.number_eyes}

        def add_attribute(self, new_attribute_name, new_attribute_value):
            self.new_attribute_name = new_attribute_name
            self.new_attribute_value = new_attribute_value
            self.attributes[self.new_attribute_name] = self.new_attribute_value

        def show_attributes(self):
            print("The object,", self.name, "has the following attributes:")
            for attribute_name, attribute_value in self.attributes.items():
                print(attribute_name, "=", attribute_value)

>>> our_dog = Dog('Copper', 'beagle')
>>> our_dog.show_attributes()
The object, Copper has the following attributes:
number of tails = 1
number of legs = 4
breed = beagle
name = Copper
number of eyes = 2
>>> our_dog.add_attribute('unique marking', 'blue right eye')
>>> our_dog.show_attributes()
The object, Copper has the following attributes:
number of legs = 4
breed = beagle
unique marking = blue right eye
number of tails = 1
name = Copper
number of eyes = 2

I suspect there are probably better ways to do what I am trying to
demonstrate [I am trying to use a dict to simulate adding new instance
variables after the fact.], perhaps with built-in functions that I
have not read about yet.  But I hope my code illustrates my previous
expectations.  I guess I was expecting that access to an object's
internals would be more strictly regulated.  But, thinking about it, I
already knew that even if name mangling is done for a method's name,
it can still be directly accessed from outside of that object's class
definition.

But if I am understanding everyone's comments correctly, Python allows
me write my classes in such a way, that public access to my object's
internals is controlled, in the sense that Python has coding
conventions that tell all of the quite intelligent, respectful,
consenting adults that might use my code how I intend that code to be
used.  And that I, despite my intentions, must respect that users of
my code may have very valid reasons for deviating from my intentions.
Have I captured the Pythonic spirit of what everyone has been trying
to tell me?

Thanks for your patience, now and in the future, as I try to learn OOP
in Python!

Cheers!
boB

From alan.gauld at btinternet.com  Sat Jan 23 04:10:40 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 23 Jan 2016 09:10:40 +0000
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9LcTXwZdga7_pT_XcuuJW_Rb7bH+DS9gtMq6CurB5tkng@mail.gmail.com>
References: <CANDiX9+U47s=pKQozVk6DCB9xHZSBtw84_H+jhnsswUGj31n1g@mail.gmail.com>
 <CAGZAPF4QCu7tAx_jZuWJ342e30Q92Dnyb1Rr_nbbgTMQrs0cdg@mail.gmail.com>
 <20160121105708.GE4619@ando.pearwood.info>
 <CANDiX9LcTXwZdga7_pT_XcuuJW_Rb7bH+DS9gtMq6CurB5tkng@mail.gmail.com>
Message-ID: <n7vg2g$9q2$1@ger.gmane.org>

On 23/01/16 04:14, boB Stepp wrote:

> humdrum.sigh_strenght = 'high'
> 
> was accepted and did not generate an error, it seemed to mean to me
> that I was violating my object's data encapsulation.  

In fact you were adding to its encapsulation(*).
Encapsulation means putting things in a box. You were adding to the
box's content therefore increasing its encapsulation. Python is one of a
few languages that allow you to do this, it's not a common feature.
Javascript is the other commonly known example.

(*)Some OO texts wrongly imply that encapsulation is about hiding
details. That's abstraction, or data hiding. These are related
but different qualities. encapsulation is a necessary feature
of OOP - it's what defines an object: having data and function
in a single box that you can then pass around. Data hiding
is an optional feature not necessary to OOP but useful in
preserving the integrity of the object.


> similar to having a class Dog, to which from outside the class'
> definition, I decide to add a new Dog attribute that all dogs in this
> class can now have a tail sticking out of their noses.  

No, you only add the attributes to an instance. Its like
giving your pet spaniel a radio beacon under its skin to track it.
Only your dog has that not all of the spaniels in the world.

> qualms about subclassing Dog with a MutantDog subclass, but adding new
> attributes from outside the class definition?  That just does not seem
> right to at this point of my understanding.  

We do it all the time in the real world. But its not that common
in programming, mainly because how do you tell the existing code
about the new attribute?


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From cs at zip.com.au  Sat Jan 23 04:30:48 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 23 Jan 2016 20:30:48 +1100
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
References: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
Message-ID: <20160123093048.GA36822@cskk.homeip.net>

On 23Jan2016 01:52, boB Stepp <robertvstepp at gmail.com> wrote:
>On Fri, Jan 22, 2016 at 11:04 PM, Cameron Simpson <cs at zip.com.au> wrote:
>> On 22Jan2016 22:14, boB Stepp <robertvstepp at gmail.com> wrote:
>> All you're missing is realising that setting an attribute is not a special
>> operation.
>
>I guess no matter how new one is to OOP, one nevertheless brings one's
>preconceptions, however malformed, into the learning process.  In my
>case, one of mine was that once a class is coded, any given instance
>of a class is forevermore responsible for managing its *internals*, so
>that any change in these would be managed by the object when one of
>its (meant to be) publicly available methods is called.

That is the pure OO way; and you can do things that way in Python, though you 
can't _prevent_ direct access.  However, it isn't the common way with Python; 
generally with a Python class there are public attributes with ordinary names; 
outsiders can consult them but generally should not change them unless the doco 
says that is ok. "Internal" attributes which outsiders should not touch are 
general givens names starting with an underscore so that outsiders can know.

>In the case
>of attributes, I was thinking/expecting something like the following
>would have to be done:
>
>>>> class Dog(object):
>        def __init__(self, name, breed):
>            self.name = name
>            self.breed = breed
>            self.number_legs = 4
>            self.number_tails = 1
>            self.number_eyes = 2
>            self.attributes = {
>                'name': self.name,
>                'breed': self.breed,
>                'number of legs': self.number_legs,
>                'number of tails': self.number_tails,
>                'number of eyes': self.number_eyes}
>
>        def add_attribute(self, new_attribute_name, new_attribute_value):
>            self.new_attribute_name = new_attribute_name
>            self.new_attribute_value = new_attribute_value
>            self.attributes[self.new_attribute_name] = self.new_attribute_value
>
>        def show_attributes(self):
>            print("The object,", self.name, "has the following attributes:")
>            for attribute_name, attribute_value in self.attributes.items():
>                print(attribute_name, "=", attribute_value)
>
>>>> our_dog = Dog('Copper', 'beagle')
>>>> our_dog.show_attributes()
>The object, Copper has the following attributes:
>number of tails = 1
>number of legs = 4
>breed = beagle
>name = Copper
>number of eyes = 2
>>>> our_dog.add_attribute('unique marking', 'blue right eye')
>>>> our_dog.show_attributes()
>The object, Copper has the following attributes:
>number of legs = 4
>breed = beagle
>unique marking = blue right eye
>number of tails = 1
>name = Copper
>number of eyes = 2
>
>I suspect there are probably better ways to do what I am trying to
>demonstrate [I am trying to use a dict to simulate adding new instance
>variables after the fact.],

You're aware that most objects have a .__dict__ attribute containing the 
attributes? It is a dict, btw. So:

  class O(object):
    pass

  >>> o=O()
  >>> o.x=1
  >>> o.__dict__
  {'x': 1}

>perhaps with built-in functions that I
>have not read about yet.  But I hope my code illustrates my previous
>expectations.  I guess I was expecting that access to an object's
>internals would be more strictly regulated.  But, thinking about it, I
>already knew that even if name mangling is done for a method's name,
>it can still be directly accessed from outside of that object's class
>definition.

Yeah. If you mean .__blah methods and attributes, I used to use them but 
generally don't these days. They're more an aid to avoiding collisions when 
subclassing to my mind, but they're not bulletproff and the implicit mangling 
makes for confusion in my mind.

>But if I am understanding everyone's comments correctly, Python allows
>me write my classes in such a way, that public access to my object's
>internals is controlled, in the sense that Python has coding
>conventions that tell all of the quite intelligent, respectful,
>consenting adults that might use my code how I intend that code to be
>used.  And that I, despite my intentions, must respect that users of
>my code may have very valid reasons for deviating from my intentions.
>Have I captured the Pythonic spirit of what everyone has been trying
>to tell me?

Pretty much.

Alan will take you up on doing purer OO practices in Python.

Cheers,
Cameron Simpson <cs at zip.com.au>

From __peter__ at web.de  Sat Jan 23 05:48:55 2016
From: __peter__ at web.de (Peter Otten)
Date: Sat, 23 Jan 2016 11:48:55 +0100
Subject: [Tutor] mock
References: <3e75abcbcd65a7d11800d0214ab0ca72@sonic.net>
Message-ID: <n7vlqp$a4c$1@ger.gmane.org>

Alex Kleider wrote:

> Some weeks (perhaps months) ago, I posted a question about testing
> and got many responses but had trouble grasping the concepts
> so I settled on the suggestion that I thought would be the easiest
> to implement (using unittest.mock.) Here it is-
> 
> """
> from Peter Otten:
> I find Ben's example instructive, but when you're just starting you
> might prefer a simpler approach:
> 
> import unittest
> from unittest import mock
> import mycode
> 
> class TestCollectData(unittest.TestCase):
>      def test(self):
>          with mock.patch(
>                  "builtins.input",
>                  side_effect=["foo", "bar", "baz"]):
>              self.assertEqual(
>                  mycode.collect_data(),
>                  dict(first="foo", last="bar", phone="baz"))
> 
> if __name__ == "__main__":
>      unittest.main()
> """
> 
> I've successfully implemented mock.patch parameters but don't
> understand how to implement the parameters for the assertEqual
> call as it pertains to my particular situation.

The biggest change when you adopt unit tests is not that you now test what 
you have, but that you write your code in such a way that it is easy to 
test. That means that you break it into small parts and that if at all 
possible you design these parts like pure functions, i. e. the behaviour of 
a part is completely determined by external input and only affects the 
result.

Hard to test:

def square_v1():
    x = int(input("give me a number "))
    return x * x

Easy to test:

def square_v2(x):
    return x * x

While you can test square_v1 by patching input() it is not something you 
would do if you can avoid it, and your actual code is even more complex, 
similar to

def square_kleider():
    while True:
        s = input("give me a number ")
        if not s:
            break
        x = int(s)
        print(x * x)

You can only test that by patching print() as well; there is no clean way to 
get data into the function and no clean way to get data out.

Now back to your actual code. A simple start would be to make entities a 
global:

def menu():
   global entities


Your test can then become

with mock.patch("builtins.input",
        side_effect=['1', 'testentity', '', '0']):
    menu.menu()
self.assert_("testentity" in menu.entities.get_list())

(Why isn't list an attribute by the way? Python is not Java...)

In the long run you should try to separate user interaction (print, input) 
and data processing to allow standard unit tests without mock for the 
latter. For every function or method the first question you should ask is:
Can I verify its correctness? Some people formalize this question by 
actually writing the test first.


From dyoo at hashcollision.org  Sat Jan 23 13:07:05 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Sat, 23 Jan 2016 10:07:05 -0800
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
References: <CANDiX9LcTXwZdga7_pT_XcuuJW_Rb7bH+DS9gtMq6CurB5tkng@mail.gmail.com>
 <20160123050450.GA38370@cskk.homeip.net>
 <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
Message-ID: <CAGZAPF5RjQ9XA1o=aKJU2Bjuo8VETaYhpMHJ=U37-4u3i0tdkw@mail.gmail.com>

> But if I am understanding everyone's comments correctly, Python allows
> me write my classes in such a way, that public access to my object's
> internals is controlled, in the sense that Python has coding
> conventions that tell all of the quite intelligent, respectful,
> consenting adults that might use my code how I intend that code to be
> used.


Yes, pretty much.

Another one of those things that distinguish Python is the lack of
"visibility" declarations to protect the internals of an
implementation from outside forces.  In many other languages, we'd use
keywords like "public" or "private" or "protected" to label a variable
or attribute as being accessible only by a few privileged clients.

But Python has very little of this as a built-in part of the language.
Guarding who gets to touch something is instead done by convention and
by a very thin mechanism of name-mangling via an uncommonly-seen
character, the underscore "_".

For a little more discussion about this, we can see the Style
Guidelines, around the part that talks about names:

    https://www.python.org/dev/peps/pep-0008/#descriptive-naming-styles

and we'll see a bunch of recommendations about using underscores for
names when we want to provide a scoping hint to others.

From robertvstepp at gmail.com  Sat Jan 23 13:55:10 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 23 Jan 2016 12:55:10 -0600
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <20160123093048.GA36822@cskk.homeip.net>
References: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
 <20160123093048.GA36822@cskk.homeip.net>
Message-ID: <CANDiX9+pe0Ggfo8DUP7AH0xtNOe-GMei95Gv=3G=rgdE-=pZdw@mail.gmail.com>

On Sat, Jan 23, 2016 at 3:30 AM, Cameron Simpson <cs at zip.com.au> wrote:
> On 23Jan2016 01:52, boB Stepp <robertvstepp at gmail.com> wrote:


>> I guess no matter how new one is to OOP, one nevertheless brings one's
>> preconceptions, however malformed, into the learning process.  In my
>> case, one of mine was that once a class is coded, any given instance
>> of a class is forevermore responsible for managing its *internals*, so
>> that any change in these would be managed by the object when one of
>> its (meant to be) publicly available methods is called.
>
>
> That is the pure OO way; and you can do things that way in Python, though
> you can't _prevent_ direct access.  However, it isn't the common way with
> Python; generally with a Python class there are public attributes with
> ordinary names; outsiders can consult them but generally should not change
> them unless the doco says that is ok...

I'd like to focus on this last sentence.  Are you suggesting that it
is "better" programming practice to code the class so that it has its
own publicly available methods to process its public attributes?  And
that it is both good practice and Pythonic to allow outsiders to
freely read public attributes as needed?


>> I suspect there are probably better ways to do what I am trying to
>> demonstrate [I am trying to use a dict to simulate adding new instance
>> variables after the fact.],
>
>
> You're aware that most objects have a .__dict__ attribute containing the
> attributes? It is a dict, btw. So:
>
>  class O(object):
>    pass
>
>  >>> o=O()
>  >>> o.x=1
>  >>> o.__dict__
>  {'x': 1}

The book I am currently working through, "Python Crash Course",
despite its title, is oriented towards beginners to programming.  It
does not cover dunder methods and attributes, other than __init__().
I am aware from following this list and reading portions of other
books, that they exist, but I have not gotten into the details of any
of them.  So rewriting my class to use .__dict__, I have gotten:

>>> class Dog(object):
        def __init__(self, name, breed):
            self.name = name
            self.breed = breed
            self.number_legs = 4
            self.number_tails = 1
            self.number_eyes = 2
        def show_attributes(self):
            print("The object,", self.name, "has the following attributes:")
            for attribute_name, attribute_value in self.__dict__.items():
                print(attribute_name, "=", attribute_value)

>>> our_dog = Dog('Copper', 'beagle')
>>> our_dog.show_attributes()
The object, Copper has the following attributes:
number_tails = 1
number_eyes = 2
breed = beagle
number_legs = 4
name = Copper
>>> our_dog.unique_marking = 'blue right eye'   # Adding a new attribute from outside the class definition.
>>> our_dog.show_attributes()
The object, Copper has the following attributes:
number_eyes = 2
breed = beagle
unique_marking = blue right eye
number_tails = 1
number_legs = 4
name = Copper

I have to say that this seems very simple and direct despite my
initial reluctance to assign new attributes to an object from outside
the class definition of that object.

I still would like to do this via a method, but I am currently stuck
on how to replace ??? with the attribute name I would like to insert:

class Dog(object):
    ...

    def add_attribute(self, attribute_name, attribute_value):
        self.??? = attribute_value

If I write something like:

unique_marking = 'blue right eye'
our_dog.add_attribute(unique_marking, 'blue right eye)

I cannot get ??? to be unique_marking.  Instead, show_attributes()
will give from .__dict__, 'attribute_name', instead of what I would
like to replace it with, unique_marking.  But I have not given up yet!
 But since I am struggling to accomplish this, that argues (my
ignorance of Python aside) strongly for the clear, simple, direct
approach above that I did in my rewrite.

>
> Alan will take you up on doing purer OO practices in Python.
>

An inside joke?  Jedi Master Alan, please instruct me in the Pythonic
ways of OO purity!

-- 
boB

From robertvstepp at gmail.com  Sat Jan 23 14:17:16 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 23 Jan 2016 13:17:16 -0600
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CAGZAPF5RjQ9XA1o=aKJU2Bjuo8VETaYhpMHJ=U37-4u3i0tdkw@mail.gmail.com>
References: <CANDiX9LcTXwZdga7_pT_XcuuJW_Rb7bH+DS9gtMq6CurB5tkng@mail.gmail.com>
 <20160123050450.GA38370@cskk.homeip.net>
 <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
 <CAGZAPF5RjQ9XA1o=aKJU2Bjuo8VETaYhpMHJ=U37-4u3i0tdkw@mail.gmail.com>
Message-ID: <CANDiX9KRNtGcXwwmmxBfdpdSP5bPPZBkN9xJu50Q-qZxBYPxZw@mail.gmail.com>

On Sat, Jan 23, 2016 at 12:07 PM, Danny Yoo <dyoo at hashcollision.org> wrote:

> But Python has very little of this as a built-in part of the language.
> Guarding who gets to touch something is instead done by convention and
> by a very thin mechanism of name-mangling via an uncommonly-seen
> character, the underscore "_".
>
> For a little more discussion about this, we can see the Style
> Guidelines, around the part that talks about names:
>
>     https://www.python.org/dev/peps/pep-0008/#descriptive-naming-styles
>
> and we'll see a bunch of recommendations about using underscores for
> names when we want to provide a scoping hint to others.

One thing I was unaware of:

"_single_leading_underscore : weak "internal use" indicator. E.g. from
M import * does not import objects whose name starts with an
underscore."

My current understanding is to avoid "from M import *", but it is good
to know that this style of import will not capture names of the form
"_name".


-- 
boB

From robertvstepp at gmail.com  Sat Jan 23 17:25:01 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 23 Jan 2016 16:25:01 -0600
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9+pe0Ggfo8DUP7AH0xtNOe-GMei95Gv=3G=rgdE-=pZdw@mail.gmail.com>
References: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
 <20160123093048.GA36822@cskk.homeip.net>
 <CANDiX9+pe0Ggfo8DUP7AH0xtNOe-GMei95Gv=3G=rgdE-=pZdw@mail.gmail.com>
Message-ID: <CANDiX9Lh+nvQFg2EQWz8FBVBfLdhtsvxY8ZQu-dPbRGHAK4_jQ@mail.gmail.com>

On Sat, Jan 23, 2016 at 12:55 PM, boB Stepp <robertvstepp at gmail.com> wrote:

> I still would like to do this via a method, but I am currently stuck
> on how to replace ??? with the attribute name I would like to insert:
>
> class Dog(object):
>     ...
>
>     def add_attribute(self, attribute_name, attribute_value):
>         self.??? = attribute_value
>
> If I write something like:
>
> unique_marking = 'blue right eye'
> our_dog.add_attribute(unique_marking, 'blue right eye)
>
> I cannot get ??? to be unique_marking.  Instead, show_attributes()
> will give from .__dict__, 'attribute_name', instead of what I would
> like to replace it with, unique_marking.  But I have not given up yet!

I think I now have this nuked out.  I am only just now realizing how
powerful .__dict__ is:

>>> class Dog(object):
        def __init__(self, name, breed):
            self.name = name
            self.breed = breed
            self.number_legs = 4
            self.number_tails = 1
            self.number_eyes = 2

        def add_attribute(self, attribute_name, attribute_value):
            self.__dict__[attribute_name] = attribute_value

        def show_attributes(self):
            print("The object,", self.name, "has the following attributes:")
            for attribute_name, attribute_value in self.__dict__.items():
                print(attribute_name, "=", attribute_value)

>>> our_dog = Dog('Copper', 'beagle')
>>> our_dog.show_attributes()
The object, Copper has the following attributes:
number_tails = 1
number_eyes = 2
breed = beagle
number_legs = 4
name = Copper
>>> our_dog.add_attribute('unique_marking', 'blue right eye')
>>> our_dog.show_attributes()
The object, Copper has the following attributes:
number_eyes = 2
breed = beagle
unique_marking = blue right eye
number_tails = 1
number_legs = 4
name = Copper
>>> our_dog.unique_marking = 'blood-shot eyes'    # Just to check that .unique_marking is a correct reference
>>> our_dog.show_attributes()
The object, Copper has the following attributes:
number_eyes = 2
breed = beagle
unique_marking = blood-shot eyes
number_tails = 1
number_legs = 4
name = Copper

Wow!  So much power for so little code!

boB

From cs at zip.com.au  Sat Jan 23 18:28:30 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 24 Jan 2016 10:28:30 +1100
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9+pe0Ggfo8DUP7AH0xtNOe-GMei95Gv=3G=rgdE-=pZdw@mail.gmail.com>
References: <CANDiX9+pe0Ggfo8DUP7AH0xtNOe-GMei95Gv=3G=rgdE-=pZdw@mail.gmail.com>
Message-ID: <20160123232830.GA26020@cskk.homeip.net>

On 23Jan2016 12:55, boB Stepp <robertvstepp at gmail.com> wrote:
>On Sat, Jan 23, 2016 at 3:30 AM, Cameron Simpson <cs at zip.com.au> wrote:
>> On 23Jan2016 01:52, boB Stepp <robertvstepp at gmail.com> wrote:
>>> I guess no matter how new one is to OOP, one nevertheless brings one's
>>> preconceptions, however malformed, into the learning process.  In my
>>> case, one of mine was that once a class is coded, any given instance
>>> of a class is forevermore responsible for managing its *internals*, so
>>> that any change in these would be managed by the object when one of
>>> its (meant to be) publicly available methods is called.
>>
>> That is the pure OO way; and you can do things that way in Python, though
>> you can't _prevent_ direct access.  However, it isn't the common way with
>> Python; generally with a Python class there are public attributes with
>> ordinary names; outsiders can consult them but generally should not change
>> them unless the doco says that is ok...
>
>I'd like to focus on this last sentence.  Are you suggesting that it
>is "better" programming practice to code the class so that it has its
>own publicly available methods to process its public attributes?

No, I'm suggesting that in a pure OO system, your only access to the internal 
state of an object is via .get_thing() methods and the only way to set them is 
via .set_thing(value) methods. In a system where outsiders cannot access 
internal attributes, that provides a completely opaque layer where the object 
mediates these actions to ensure correctness and where the internals are 
invisible, allowing a complete change of implementation without breaking the 
interface outsiders use.

>And
>that it is both good practice and Pythonic to allow outsiders to
>freely read public attributes as needed?

Generally yes. As the author of the class, you need to decide what should be 
visible (meaning "not have a leading underscore"). Of course it is _all_ 
visible, but when you give something a "public" name you are quietly implying 
to outsiders that this is stable, and future implementations will continue to 
preserve it.

You can be totally conservative of course and give all the internal state ._* 
names. But an object with no attributes is generally not as useful. Note that 
there's a grey area here: plenty of objects have methods which return values:

  class O(object):
    def __init__(self, name):
      self._name = name
    def name(self):
      return self._name

  o = O("foo")
  print(o.name())

so you can keep the state "private" while presenting useful information via 
methods. By having .name be a function, you are free to reimplement the class 
using anther mechanism and outsiders will not have to change how they use 
things:

  class O(object):
    def __init__(self, name):
      # make a row in a database associating our id with this name
      db.set_name(id(self), name)
    def name(self):
      # fetch the name back out of the database
      n = db.get_name_by_id(id(self))
      return n

>>> I suspect there are probably better ways to do what I am trying to
>>> demonstrate [I am trying to use a dict to simulate adding new instance
>>> variables after the fact.],
>>
>> You're aware that most objects have a .__dict__ attribute containing the
>> attributes? It is a dict, btw. So:
>>
>>  class O(object):
>>    pass
>>
>>  >>> o=O()
>>  >>> o.x=1
>>  >>> o.__dict__
>>  {'x': 1}
>
>The book I am currently working through, "Python Crash Course",
>despite its title, is oriented towards beginners to programming.  It
>does not cover dunder methods and attributes, other than __init__().
>I am aware from following this list and reading portions of other
>books, that they exist, but I have not gotten into the details of any
>of them.  So rewriting my class to use .__dict__, I have gotten:
>
>>>> class Dog(object):
>        def __init__(self, name, breed):
>            self.name = name
>            self.breed = breed
>            self.number_legs = 4
>            self.number_tails = 1
>            self.number_eyes = 2
>        def show_attributes(self):
>            print("The object,", self.name, "has the following attributes:")
>            for attribute_name, attribute_value in self.__dict__.items():
>                print(attribute_name, "=", attribute_value)
>
>>>> our_dog = Dog('Copper', 'beagle')
>>>> our_dog.show_attributes()
>The object, Copper has the following attributes:
>number_tails = 1
>number_eyes = 2
>breed = beagle
>number_legs = 4
>name = Copper
>>>> our_dog.unique_marking = 'blue right eye'   # Adding a new attribute from outside the class definition.
>>>> our_dog.show_attributes()
>The object, Copper has the following attributes:
>number_eyes = 2
>breed = beagle
>unique_marking = blue right eye
>number_tails = 1
>number_legs = 4
>name = Copper
>
>I have to say that this seems very simple and direct despite my
>initial reluctance to assign new attributes to an object from outside
>the class definition of that object.

Yes. No magic here, just exposed mechanism :-)

>I still would like to do this via a method, but I am currently stuck
>on how to replace ??? with the attribute name I would like to insert:
>
>class Dog(object):
>    ...
>
>    def add_attribute(self, attribute_name, attribute_value):
>        self.??? = attribute_value

self.__dict__[attribute_name] = attribute_value

>If I write something like:
>
>unique_marking = 'blue right eye'
>our_dog.add_attribute(unique_marking, 'blue right eye)
>
>I cannot get ??? to be unique_marking.  Instead, show_attributes()
>will give from .__dict__, 'attribute_name', instead of what I would
>like to replace it with, unique_marking.  But I have not given up yet!

The name is a string, so:

  our_dog.add_attribute('unique_marking', 'blue right eye')

> But since I am struggling to accomplish this, that argues (my
>ignorance of Python aside) strongly for the clear, simple, direct
>approach above that I did in my rewrite.
>
>> Alan will take you up on doing purer OO practices in Python.
>
>An inside joke?  Jedi Master Alan, please instruct me in the Pythonic
>ways of OO purity!

No, but Alan has a far better handle on the concrete definitions of what is 
purely OO practice and what is commonly mixed in. He is also a stronger 
advocate of pure OO approaches than I.

Cheers,
Cameron Simpson <cs at zip.com.au>

From cs at zip.com.au  Sat Jan 23 18:38:38 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 24 Jan 2016 10:38:38 +1100
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9Lh+nvQFg2EQWz8FBVBfLdhtsvxY8ZQu-dPbRGHAK4_jQ@mail.gmail.com>
References: <CANDiX9Lh+nvQFg2EQWz8FBVBfLdhtsvxY8ZQu-dPbRGHAK4_jQ@mail.gmail.com>
Message-ID: <20160123233838.GA50127@cskk.homeip.net>

On 23Jan2016 16:25, boB Stepp <robertvstepp at gmail.com> wrote:
>On Sat, Jan 23, 2016 at 12:55 PM, boB Stepp <robertvstepp at gmail.com> wrote:
>> I still would like to do this via a method, but I am currently stuck
>> on how to replace ??? with the attribute name I would like to insert:
>>
>> class Dog(object):
>>     ...
>>
>>     def add_attribute(self, attribute_name, attribute_value):
>>         self.??? = attribute_value
>>
>> If I write something like:
>>
>> unique_marking = 'blue right eye'
>> our_dog.add_attribute(unique_marking, 'blue right eye)
>>
>> I cannot get ??? to be unique_marking.  Instead, show_attributes()
>> will give from .__dict__, 'attribute_name', instead of what I would
>> like to replace it with, unique_marking.  But I have not given up yet!
>
>I think I now have this nuked out.  I am only just now realizing how
>powerful .__dict__ is:
[... working code ...]

>Wow!  So much power for so little code!

Sure. But also note that in the real world you will hardly ever work on 
.__dict__ directly. If I were collecting a lot of _arbitrary_ features like 
.unique_marking I would put them all in some internal attribute:

  class Dog(object):
    def __init__(self, name):
      self.name = name
      self.features = {}
    def add_feature(self, feature_name, feature_value):
      self.features[feature_name] = feature_value

(Note, deliberately avoiding the term "attribute" here.)

Also, Python provideds the functions setattr and getattr to _avoid_ directly 
accessing .__dict__:

  setattr(self, attribute_name, attribute_value)

because not all objects use .__dict__ to store this stuff. Generally the dunder 
names are to be avoided: not that they should never be used, but they tend to 
be mechanism which has "public" exposure elsewhere, eg __dict__ vs 
getattr/setattr. For exactly the same reason as other OO stuff: because the 
internal implementation is not always __dict__.

So you might access __dict__ internally for introspection, but you would more 
commonly use getattr (and outsiders should _always_ use getattr!) Or, of 
course, if the attribute name is known at coding time, with the better .name 
syntax:

  v = dog.__dict__['unique_feature']
  v = getattr(dog, 'unique_feature')
  v = dog.unique_feature

from least desired to most desired. BTW, when the attribute exists these all 
return the same thing, but when the attribute does no exist they all raise 
different kinds of exception.

Cheers,
Cameron Simpson <cs at zip.com.au>

From esawiek at gmail.com  Sat Jan 23 12:57:50 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Sat, 23 Jan 2016 12:57:50 -0500
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
Message-ID: <CA+ZkTxsuDNFxmTCbjcoUrZnKjq5aGf6jokusUPvjEL_ub3kQ5w@mail.gmail.com>

Hi All---



Sorry for posting again, but I have a problem that I tried several
different ways to solve w/o success. I approached the problem from one
angle and asked about it here; I got some good input using pandas, and
structured array, but I am new to python and not very familiar with either
to use at this moment.  I decided to go about in a different direction. I
am hoping for a simpler solution using Numpy.


I have a csv file with 4 columns and 2000 rows. There are 10 variables in
column 1 and 4 variables on each column, 2 and 3. I read the csv file and
converted it to arrays. The problem I ran into and could not resolve is
2-fold: (1) change the datatype for columns 1 and 4 to float and (2) then,
I want to use Numpy-or simpler method- to calculate the mean of the data
points on column 4 based on each variable on column 1 and column 2. Below
is my code and sample data file.



Here is part of my code:



import numpy as np

import csv



TMatrix=[]

np.set_printoptions(precision=2)



" Converting csv to lists "



with open('c:/Users/My Documents/AAA/temp1.csv') as temp:

    reader = csv.reader(temp, delimiter=',', quoting=csv.QUOTE_NONE)

    for row in reader:

        TMatrix.append(row)



" converting lists to arrays "

TMatrix=np.array(TMatrix)

TMatrix=np.array(4,TMatrix[1:,::],dtype='float,int,int,float')        #
this statement is not working



+++++++++++++++ This is a sample of my file +++++++++++++



['19' 'A4' 'B2' '2']

 ['19' 'A5' 'B1' '12']

 ['18' 'A5' 'B2' '121']]



Thanks in advance

EK Esawi

From alan.gauld at btinternet.com  Sat Jan 23 20:06:19 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 24 Jan 2016 01:06:19 +0000
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9KRNtGcXwwmmxBfdpdSP5bPPZBkN9xJu50Q-qZxBYPxZw@mail.gmail.com>
References: <CANDiX9LcTXwZdga7_pT_XcuuJW_Rb7bH+DS9gtMq6CurB5tkng@mail.gmail.com>
 <20160123050450.GA38370@cskk.homeip.net>
 <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
 <CAGZAPF5RjQ9XA1o=aKJU2Bjuo8VETaYhpMHJ=U37-4u3i0tdkw@mail.gmail.com>
 <CANDiX9KRNtGcXwwmmxBfdpdSP5bPPZBkN9xJu50Q-qZxBYPxZw@mail.gmail.com>
Message-ID: <n8182b$19h$1@ger.gmane.org>

On 23/01/16 19:17, boB Stepp wrote:

> "_single_leading_underscore : weak "internal use" indicator. E.g. from
> M import * does not import objects whose name starts with an
> underscore."
> 
> My current understanding is to avoid "from M import *", but it is good
> to know that this style of import will not capture names of the form
> "_name".

There is another way to control what gets exported from modules.
You can define a variable in the module called __all__ which
contains a list of names that you are happy to export.

Then if someone does from m import * only the names in
m.__all__ will actually be made  visible.

But that doesn't do anything to stop those names colliding
with similar names from another module, it just provides some
token privacy for names you specifically don't want exported.

hth
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Sat Jan 23 20:20:18 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 24 Jan 2016 01:20:18 +0000
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <20160123093048.GA36822@cskk.homeip.net>
References: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
 <20160123093048.GA36822@cskk.homeip.net>
Message-ID: <n818si$bo9$1@ger.gmane.org>

On 23/01/16 09:30, Cameron Simpson wrote:

> Alan will take you up on doing purer OO practices in Python.

In pure OO objects should only expose methods and the data
attributes should only be there to support the methods.
As such, nobody outside the object has any need to know
anything about it. And if you have data attributes that
no method uses that's a warning that they maybe are in
the wrong place! Likewise a class with no methods maybe
should just be a dict or tuple?

In the real world of course many objects exist primarily
as data stores - especially in business type apps. But
the principle remains that it's a very suspicious anti-pattern
to see code like:

anObj = myClass(...)
anX = anObj.X
anX = someFunc(anX)   # alarm bells here
anObj.X = anX

This strongly suggests that there's a bit of functionality
that should be inside myClass as a method going on inside
someFunc().

And of course someFunc may be a series of inline statements,
it might not be so conveniently evident as a function call.

So as a class designer, if a user of your class (who might
be yourself!)  insists that they need a setX style method
or wants public write access to an attribute you've got
to ask the question why? There is a strong chance they
really need a new method instead.

One of the features of OO programming is that large
monolithic algorithms/functions often get split into small
chunks distributed over several classes.

Finally, if you are really concerned about data access
control (but you have to consider why, and how big
a risk it is really) then don't forget you can define
properties in Python, which can be read-only, write-only,
or both. (Of course, in Python it's never bulletproof,
but it requires deliberate vandalism to subvert it.)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From robertvstepp at gmail.com  Sat Jan 23 22:07:17 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 23 Jan 2016 21:07:17 -0600
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <n818si$bo9$1@ger.gmane.org>
References: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
 <20160123093048.GA36822@cskk.homeip.net>
 <n818si$bo9$1@ger.gmane.org>
Message-ID: <CANDiX9LOUDS_OokCheK9AGWLuW_aR9nj04Fj34Fy6bcAc0QuZw@mail.gmail.com>

On Sat, Jan 23, 2016 at 7:20 PM, Alan Gauld <alan.gauld at btinternet.com> wrote:
> On 23/01/16 09:30, Cameron Simpson wrote:
>
>> Alan will take you up on doing purer OO practices in Python.
>
> In pure OO objects should only expose methods and the data
> attributes should only be there to support the methods.
> As such, nobody outside the object has any need to know
> anything about it...

Apparently what OOP knowledge I started out with agrees strongly with
what you say here.  But this level of *purity* appears to be
non-Pythonic in actual practice.  So, in your opinion, how far should
I go towards emulating pure OO objects?

> One of the features of OO programming is that large
> monolithic algorithms/functions often get split into small
> chunks distributed over several classes.

With functions, the ideal is that each function executes a single,
specific purpose.  With large classes, what should I use as a guide as
to when to split them into smaller ones?  My initial thought would be
whenever a portion of the larger class models nicely into its own
class, perhaps one that looks to be useful on its own in other
programming scenarios.  Am I close to hitting the nail on the head
here?

Thanks, Alan!


-- 
boB

From robertvstepp at gmail.com  Sat Jan 23 23:52:27 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 23 Jan 2016 22:52:27 -0600
Subject: [Tutor] Value of tracebacks to malicious attackers?
Message-ID: <CANDiX9L5QEnjiDmYv95-dm7NeifABb=4KmV842kwLktwagih_w@mail.gmail.com>

>From page 202 of "Python Crash Course":  "..., but it's also not a
good idea to let users see tracebacks.  Nontechnical users will be
confused by them, and in a malicious setting, attackers will learn
more than you want them to know from a traceback.  For example,
they'll know the name of your program file, and they'll see a part of
your code that isn't working properly.  A skilled attacker can
sometimes use this information to determine which kind of attacks to
use against your code."

How much concern do you give this in designing and implementing your
production code?  How far do you go in handling exceptions to ensure
that tracebacks cannot arise for a malicious user?  Is it even
possible to prevent this from happening?  I am highly doubtful that it
is possible to handle all possible exceptions in any reasonably
complex application.

Of course for many applications these concerns are probably
inconsequential.  I don't think I would be gravely concerned if
someone cracked into my ticktacktoe game deployed to some freebie
site.  OTOH, if I was deploying an online payment system, I would be
much more concerned.  Awaiting your collective wisdom...

TIA!
boB

From dyoo at hashcollision.org  Sun Jan 24 01:43:31 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Sat, 23 Jan 2016 22:43:31 -0800
Subject: [Tutor] Value of tracebacks to malicious attackers?
In-Reply-To: <CANDiX9L5QEnjiDmYv95-dm7NeifABb=4KmV842kwLktwagih_w@mail.gmail.com>
References: <CANDiX9L5QEnjiDmYv95-dm7NeifABb=4KmV842kwLktwagih_w@mail.gmail.com>
Message-ID: <CAGZAPF6Pse1zL3R54jgcjC1fgEz+r9M02efsazXJueS32KC8Zw@mail.gmail.com>

> How much concern do you give this in designing and implementing your
> production code?  How far do you go in handling exceptions to ensure
> that tracebacks cannot arise for a malicious user?


Hi boB,

We can plan ahead and develop a program to support different modes of
operation . A "debug" or "devel" mode, for example, might be free to
show debugging output to the user in the face of errors.  We can run
our systems in a "debug" mode, but when we deploy such a system to
production, we can change the error handling so that errors don't
spill indiscriminately to the outside world.

We do not usually want verbose programmer-level error messages to
reach lay audiences because it's very possible for someone without
prior knowledge to completely misinterpret what the error is saying.
People have the tendency to attribute malice or blame to messages
that, on a first glance, are accusatory.  "Error: blah blah process
killed blah blah" can be *threatening*, even if someone doesn't
understand all the words there.

So it's not just because we're considering malicious intent here: a
message's interpretation depends much upon the audience receiving the
message, and we have to keep that in mind.

But if there is malicious intent, yes, a traceback can give a lot of
information that we're rather not give freely.  Operating system,
version of the Python runtime, locations of files, etc.  That's all
valuable knowledge.



> Is it even
> possible to prevent this from happening?  I am highly doubtful that it
> is possible to handle all possible exceptions in any reasonably
> complex application.

If we don't handle an exception directly, then we delegate the
handling of that responsibility upward.  There's a "toplevel"
exception handler that handles all uncaught errors.  For Python, this
is the 'sys.excepthook' mechanism:

    https://docs.python.org/3.5/library/sys.html#sys.excepthook

which, by default, prints the exception out to standard error.  We can
think of other things we can do here instead, like quietly logging the
message to a text file for later perusal.  Or maybe it could send an
automated email to the developers.  Or perhaps even start ringing
pagers, if things are serious enough to warrant waking someone up in
the middle of the night.



> Of course for many applications these concerns are probably
> inconsequential.  I don't think I would be gravely concerned if
> someone cracked into my ticktacktoe game deployed to some freebie
> site.

It doesn't hurt too much to practice safe habits, and doing the "right
thing" here is often useful for its own sake, and not just because
it's less risky.  For example, we discourage folks from using eval()
under normal circumstances, not only because it's unsafe (though
that's the high-order bit there), but because it's often a crutch for
not learning to use tools like parsers, or understanding how to use
dictionaries for holding key/value pairs.

Also, the stakes are higher than one might realize at first, because
computation and network resources are not free: being able to compute
something is itself a valuable resource (as the Bitcoin folks know).

Think not just about the breaking of your own program.  Instead, think
about what happens if someone takes over the machine and starts using
it as part of a botnet.  That is often a purpose of a system
compromise: the attacker may not personally care about tic-tac-toe,
but they do care about having root privileges on a computer.



Hope that makes sense.  Best of wishes!

From robertvstepp at gmail.com  Sun Jan 24 02:32:06 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sun, 24 Jan 2016 01:32:06 -0600
Subject: [Tutor] Value of tracebacks to malicious attackers?
In-Reply-To: <CAGZAPF6Pse1zL3R54jgcjC1fgEz+r9M02efsazXJueS32KC8Zw@mail.gmail.com>
References: <CANDiX9L5QEnjiDmYv95-dm7NeifABb=4KmV842kwLktwagih_w@mail.gmail.com>
 <CAGZAPF6Pse1zL3R54jgcjC1fgEz+r9M02efsazXJueS32KC8Zw@mail.gmail.com>
Message-ID: <CANDiX9L85ZZMXgmKuEDjBA+rJM454XqR4f+9PkHVGH6aVmCW8A@mail.gmail.com>

On Sun, Jan 24, 2016 at 12:43 AM, Danny Yoo <dyoo at hashcollision.org> wrote:
>> How much concern do you give this in designing and implementing your
>> production code?  How far do you go in handling exceptions to ensure
>> that tracebacks cannot arise for a malicious user?

> We can plan ahead and develop a program to support different modes of
> operation . A "debug" or "devel" mode, for example, might be free to
> show debugging output to the user in the face of errors.  We can run
> our systems in a "debug" mode, but when we deploy such a system to
> production, we can change the error handling so that errors don't
> spill indiscriminately to the outside world.

Is there some standard way of implementing a debug mode?  If I were
thinking procedurally I would probably turn debug mode on and off with
a flag.  And I suppose one or more classes could be written to
accomplish this to provide coherent error processing for the entire
application.

>> Is it even
>> possible to prevent this from happening?  I am highly doubtful that it
>> is possible to handle all possible exceptions in any reasonably
>> complex application.
>
> If we don't handle an exception directly, then we delegate the
> handling of that responsibility upward.  There's a "toplevel"
> exception handler that handles all uncaught errors.  For Python, this
> is the 'sys.excepthook' mechanism:
>
>     https://docs.python.org/3.5/library/sys.html#sys.excepthook
>
> which, by default, prints the exception out to standard error.  We can
> think of other things we can do here instead, like quietly logging the
> message to a text file for later perusal.  Or maybe it could send an
> automated email to the developers.  Or perhaps even start ringing
> pagers, if things are serious enough to warrant waking someone up in
> the middle of the night.

Looking at the reference you referred me to:

sys.excepthook(type, value, traceback)
This function prints out a given traceback and exception to sys.stderr.

When an exception is raised and uncaught, the interpreter calls
sys.excepthook with three arguments, the exception class, exception
instance, and a traceback object. In an interactive session this
happens just before control is returned to the prompt; in a Python
program this happens just before the program exits. The handling of
such top-level exceptions can be customized by assigning another
three-argument function to sys.excepthook.

It is not at all clear to me how to make use of this information.
"...assigning another three-argument function to sys.excepthook" is
about as clear as mud to me right now.  Can you provide an example of
how to redirect uncaught, top-level exceptions to a file?  I'm still
searching for understandable examples, but I have not found any yet.
But I am quite tired now, so things will probably look more reasonable
after some sleep.

>> Of course for many applications these concerns are probably
>> inconsequential.  I don't think I would be gravely concerned if
>> someone cracked into my ticktacktoe game deployed to some freebie
>> site.

> Think not just about the breaking of your own program.  Instead, think
> about what happens if someone takes over the machine and starts using
> it as part of a botnet.  That is often a purpose of a system
> compromise: the attacker may not personally care about tic-tac-toe,
> but they do care about having root privileges on a computer.

This possibility occurred to me shortly after sending my email to the
list, that being sloppy could provide an entry point to whatever
server this hypothetical game was being hosted on.


-- 
boB

From alan.gauld at btinternet.com  Sun Jan 24 04:11:28 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 24 Jan 2016 09:11:28 +0000
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9LOUDS_OokCheK9AGWLuW_aR9nj04Fj34Fy6bcAc0QuZw@mail.gmail.com>
References: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
 <20160123093048.GA36822@cskk.homeip.net> <n818si$bo9$1@ger.gmane.org>
 <CANDiX9LOUDS_OokCheK9AGWLuW_aR9nj04Fj34Fy6bcAc0QuZw@mail.gmail.com>
Message-ID: <n824g0$pcb$1@ger.gmane.org>

On 24/01/16 03:07, boB Stepp wrote:

>> In pure OO objects should only expose methods and the data
>> attributes should only be there to support the methods.
> 
> Apparently what OOP knowledge I started out with agrees strongly with
> what you say here.  But this level of *purity* appears to be
> non-Pythonic in actual practice.  

Python allows you to be as pure as you want to be.
The choice is mainly in the hands of the consumer of
the class. They get to decide whether they want/need
to poke about with the internals. If a class is well
designed it should be possible to use its operations
without directly modifying its data. Reading data
directly in Python is usually considered ok.

But laziness or the need for speed (of development)
often corrupts purity.

> So, in your opinion, how far should
> I go towards emulating pure OO objects?

As above, I try to ensure that the class operations
do not require the user to poke values in (possibly
by providing parameters). I "allow/encourage" direct
access to read.

As an example

class Point:
  def __init__(self,x,y):
     self.x = x
     self.y = y
  def distance(self):
     return (self.x**2+self.y**2)**0.5
  def moveTo(self,new_x,new_y):
     self.x = x
     self.y = y

By providing the moveTo() method there should be
no reason for users to set x or y individually.
But I encourage direct reading by not providing
get() methods. In pure OOP I should provide a Coords()
method too.

>> monolithic algorithms/functions often get split into small
>> chunks distributed over several classes.
> 
> With functions, the ideal is that each function executes a single,
> specific purpose.  

Still true in OOP

> With large classes, what should I use as a guide as
> to when to split them into smaller ones?  

Is it a single thing? Or is it really trying to be multiple things?
Of course a large object may be composed of many smaller objects.
A model of a car may have wheels, engines, brakes etc.
And the code for the accelerate() method may delegate to
engine and wheel methods rather than directly reading the
engine data.

> class, perhaps one that looks to be useful on its own in other
> programming scenarios.  Am I close to hitting the nail on the head
> here?

Yes, re-use is also a good reason to ask should this be a
separate object? But reuse alone is not strong enough to
break up what is a single entity, although if its a
function you want you might find that you can extract it
into a mixin class (ie one that reflects a behaviour rather
than a noun and is usually inherited as part of a
multi-inheritance lattice) - such as loggable, or colored.

HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Sun Jan 24 04:17:10 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 24 Jan 2016 09:17:10 +0000
Subject: [Tutor] Value of tracebacks to malicious attackers?
In-Reply-To: <CANDiX9L5QEnjiDmYv95-dm7NeifABb=4KmV842kwLktwagih_w@mail.gmail.com>
References: <CANDiX9L5QEnjiDmYv95-dm7NeifABb=4KmV842kwLktwagih_w@mail.gmail.com>
Message-ID: <n824qm$u1r$1@ger.gmane.org>

On 24/01/16 04:52, boB Stepp wrote:

> How much concern do you give this in designing and implementing your
> production code?  How far do you go in handling exceptions to ensure
> that tracebacks cannot arise for a malicious user?

When I was writing commercial code that went into "the wild" we
had a policy to wrap the entire main program in a try/except
that logged all error output in a file (on the server). If it
was a PC desktop app you could email it to a support site
 - although I've never personally written commercial apps
other than server side.

Our main concern was to prevent spooking the user, but nowadays
the security risk is definitely valid too.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From __peter__ at web.de  Sun Jan 24 05:22:19 2016
From: __peter__ at web.de (Peter Otten)
Date: Sun, 24 Jan 2016 11:22:19 +0100
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
References: <CA+ZkTxsuDNFxmTCbjcoUrZnKjq5aGf6jokusUPvjEL_ub3kQ5w@mail.gmail.com>
Message-ID: <n828kt$k75$1@ger.gmane.org>

Ek Esawi wrote:

> Hi All---
> 
> 
> 
> Sorry for posting again, but I have a problem that I tried several
> different ways to solve w/o success. I approached the problem from one
> angle and asked about it here; I got some good input using pandas, and
> structured array, but I am new to python and not very familiar with either
> to use at this moment.  I decided to go about in a different direction. I
> am hoping for a simpler solution using Numpy.
> 
> 
> I have a csv file with 4 columns and 2000 rows. There are 10 variables in
> column 1 and 4 variables on each column, 2 and 3. I read the csv file and
> converted it to arrays. The problem I ran into and could not resolve is
> 2-fold: (1) change the datatype for columns 1 and 4 to float and (2) then,
> I want to use Numpy-or simpler method- to calculate the mean of the data
> points on column 4 based on each variable on column 1 and column 2. Below
> is my code and sample data file.
> 
> 
> 
> Here is part of my code:
> 
> 
> 
> import numpy as np
> 
> import csv
> 
> 
> 
> TMatrix=[]
> 
> np.set_printoptions(precision=2)
> 
> 
> 
> " Converting csv to lists "
> 
> 
> 
> with open('c:/Users/My Documents/AAA/temp1.csv') as temp:
> 
>     reader = csv.reader(temp, delimiter=',', quoting=csv.QUOTE_NONE)
> 
>     for row in reader:
> 
>         TMatrix.append(row)
> 
> 
> 
> " converting lists to arrays "
> 
> TMatrix=np.array(TMatrix)
> 
> TMatrix=np.array(4,TMatrix[1:,::],dtype='float,int,int,float')        #
> this statement is not working
> 
> 
> 
> +++++++++++++++ This is a sample of my file +++++++++++++
> 
> 
> 
> ['19' 'A4' 'B2' '2']
> 
>  ['19' 'A5' 'B1' '12']
> 
>  ['18' 'A5' 'B2' '121']]

How do you want to convert the second and third column to int? Are A4 and B2 
hex numbers? Then try

$ cat esawi.csv 
19,A4,B2,2
19,A5,B1,12
$ python3
Python 3.4.3 (default, Oct 14 2015, 20:28:29) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> def fromhex(s):
...     return int(s, 16)
... 
>>> numpy.genfromtxt("esawi.csv", delimiter=",", converters={1:fromhex, 
2:fromhex})
array([(19.0, 164, 178, 2.0), (19.0, 165, 177, 12.0)], 
      dtype=[('f0', '<f8'), ('f1', '<i8'), ('f2', '<i8'), ('f3', '<f8')])



From jeanpierreda at gmail.com  Sun Jan 24 07:13:10 2016
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sun, 24 Jan 2016 04:13:10 -0800
Subject: [Tutor] Value of tracebacks to malicious attackers?
In-Reply-To: <CAGZAPF6Pse1zL3R54jgcjC1fgEz+r9M02efsazXJueS32KC8Zw@mail.gmail.com>
References: <CANDiX9L5QEnjiDmYv95-dm7NeifABb=4KmV842kwLktwagih_w@mail.gmail.com>
 <CAGZAPF6Pse1zL3R54jgcjC1fgEz+r9M02efsazXJueS32KC8Zw@mail.gmail.com>
Message-ID: <CABicbJJapk=RTxpX80iTLUpq8qSk_OqvNJV5fUF4N7dpMx3KVw@mail.gmail.com>

Forgetting to hide tracebacks on a client-side app is not a huge deal
(other than being confusing and looking too serious), and definitely
doesn't matter for security. So I guess we're assuming that the Python
code is not on the user's machine.

On Sat, Jan 23, 2016 at 10:43 PM, Danny Yoo <dyoo at hashcollision.org> wrote:
> But if there is malicious intent, yes, a traceback can give a lot of
> information that we're rather not give freely.  Operating system,
> version of the Python runtime, locations of files, etc.  That's all
> valuable knowledge.

Verbose tracebacks (like from cgitb) will also included the values of
variables on the stack, which can include extremely sensitive data
like cryptographic keys.

For letting users report errors without compromising any data at all,
one can give them a hashed stack trace or error ID that a developer
can look up locally from an error DB. If there is no error DB, then an
encrypted stack trace works too.

>> Is it even
>> possible to prevent this from happening?  I am highly doubtful that it
>> is possible to handle all possible exceptions in any reasonably
>> complex application.

You can catch exceptions using a top-level try-except. If an exception
slips through (e.g. a bug in your setup/teardown code etc.), then your
app will crash, but the exception will not be given to the user (just
to stderr, which will get directed to logs), because the only
exceptions users see are exceptions you explicitly give to them.

(This isn't true in some environments. Avoid those.)

-- Devin

From esawiek at gmail.com  Sun Jan 24 09:09:17 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Sun, 24 Jan 2016 09:09:17 -0500
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
Message-ID: <CA+ZkTxsSHAhG4aEfkmgaAXziUhrqeO=C7aU9HwBHSjP+gYZUGA@mail.gmail.com>

Thanks for the input. Columns 2 and 3 are strings and i assume that they
don't need to be converted. Because all i need is to compute the mean for
data on column 4 based on each variable in column 1 and each in column 2..

BTW, is it possible to send you what i did off list? That way you see what
i am doing.


EKE

From __peter__ at web.de  Sun Jan 24 12:24:03 2016
From: __peter__ at web.de (Peter Otten)
Date: Sun, 24 Jan 2016 18:24:03 +0100
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
References: <CA+ZkTxsSHAhG4aEfkmgaAXziUhrqeO=C7aU9HwBHSjP+gYZUGA@mail.gmail.com>
Message-ID: <n831bo$37h$1@ger.gmane.org>

Ek Esawi wrote:

> Thanks for the input. Columns 2 and 3 are strings and i assume that they
> don't need to be converted. Because all i need is to compute the mean for
> data on column 4 based on each variable in column 1 and each in column 2..

I'm an amateur with numpy, and unfortunately my favourite search engine 
didn't come up with a numpy-specific way to group rows in a 2D array.

> BTW, is it possible to send you what i did off list? That way you see what
> i am doing.

No. I'm posting here to help you overcome problems you run into when 
learning Python (and sometimes the application of Python libraries), not to 
write ready-to-use code for you.



From alan.gauld at btinternet.com  Sun Jan 24 13:26:38 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 24 Jan 2016 18:26:38 +0000
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
In-Reply-To: <CA+ZkTxsSHAhG4aEfkmgaAXziUhrqeO=C7aU9HwBHSjP+gYZUGA@mail.gmail.com>
References: <CA+ZkTxsSHAhG4aEfkmgaAXziUhrqeO=C7aU9HwBHSjP+gYZUGA@mail.gmail.com>
Message-ID: <n8350u$q31$1@ger.gmane.org>

On 24/01/16 14:09, Ek Esawi wrote:

> BTW, is it possible to send you what i did off list? That way you see what
> i am doing.

The list is here so that everyone can benefit.
Can you post the code publicly? If so, and its not more
than say, 100-200 lines, you can just post it here.

But I will repeat that while Oscar and Peter and a few
others can help you with Numpy/SciPy there is a dedicated
forum where you will find many more experts willing and
eager to help you.

And I get the feeling that even if we resolve this issue
there will only be more to follow...

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From sjeik_appie at hotmail.com  Sun Jan 24 14:47:47 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 24 Jan 2016 19:47:47 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <n7rh0d$6tn$1@ger.gmane.org>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>,
 <n7rh0d$6tn$1@ger.gmane.org>
Message-ID: <DUB123-W36914C93BB77114F32CB1983C60@phx.gbl>

 
> To: tutor at python.org
> From: breamoreboy at yahoo.co.uk
> Date: Thu, 21 Jan 2016 21:02:03 +0000
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> 
> On 20/01/2016 13:33, Albert-Jan Roskam wrote:
> > Hi,
> >
> > Like the subject says: Why is an OrderedDict not sliceable? (From the collections library). Was that an intentional omission, or a mistake? [1]
> 
> Plenty of answers on this all ready, but...
> 
> >
> > Background: I do not use OrderedDict very often, but I thought I could use it to look up street and city names using postcodes ([0-9]{4} [a-z]{2} format). I needed it to be ordered because I also wanted to be able to use bisect, which is needed when the postcode letters are missing. In short: a fast dict lookup for complete postcodes and less fast bisect lookup for in complete postcodes.
> >
> 
> You appear to be confusing ordered and sorted.   You are correct. Is there a difference in the way those terms are used colloquially vs. in the field of Computer Science (Note: English is not my mother tongue)? Anyway, this page seems to suggest that "Ordered" means "gets messed up upon insertion, deletion, update: http://stackoverflow.com/questions/1084146/what-is-the-difference-between-an-ordered-and-a-sorted-collection   There is no way that you 
> can use bisect on an OrderedDict unless it is sorted in the first place.
 I fetch the data from a MS SQL Server with a query that goes something like SELECT DISTINCT pc_digits, pc_letters, house_number_from, house_number_to, street, city WHERE ... ORDER BY pc_digits, pc_letters.  
> > [1] http://stackoverflow.com/questions/30975339/slicing-a-python-ordereddict
> >
> > Thanks!
> >
> > Albert-Jan
> >   		 	   		
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence
> 
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
 		 	   		  

From sjeik_appie at hotmail.com  Sun Jan 24 15:08:13 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 24 Jan 2016 20:08:13 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <n7rs52$n3s$1@ger.gmane.org>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>,
 <20160122000000.GI4619@ando.pearwood.info>, <n7rs52$n3s$1@ger.gmane.org>
Message-ID: <DUB123-W43DF49F3924EDAEEF78E8983C60@phx.gbl>


 
> To: tutor at python.org
> From: alan.gauld at btinternet.com
> Date: Fri, 22 Jan 2016 00:12:18 +0000
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> 
> On 22/01/16 00:00, Steven D'Aprano wrote:
> 
> > Also, you have a problem -- what happens if the incomplete postcode is 
> > missing the first digit? (I suppose for that case, you just have to do a 
> > slow linear search.) 
If the postcode letters are missing, I look up the insertion point for the postcode digits. I then indeed use a linear search to loop through that postal code 4-digits area. Then I use the apartment number to filter out impossible street(s). Often postcodes for odd and even house numbers differ. So if, for instance, I have postcode digits 1234, no postcode letters, and apartment number 312, I can ignore all odd number ranges, and all ranges that do not include 312 (many streets have just a few dozen numbers). I then end up with a bunch of options, which I call candidates. Finally, I use difflib.SequenceMatcher to try and select a record of which the "dirty" street name, most closely resembles the standard street name in the list of candidates. I did not consider what to do when one postcode letter is missing, or when letters are transposed. I expect (and hope!) that the street/city cannot be derived and that this record needs to be reviewed manually. And in case you wondered: yes, it sucks that all these corrections need to be done "after the fact". A good reason to learn how to use Django or Flask to create a data entry screen and start with crisp clean data :-)
> Which is why a different solution may be better suited.
> What about an in-memory SQLite table? Then you can use
> a LIKE based select and let the library do the hard work.
Ouch, I considered marshall and pickle to cache the data, because running the query on MS SQL takes more time than I like. But I am embarrassed to say I did not even consider SQLite. I will reconsider this. Thank you! 
> Just a thought.
> 
> -- 
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
> 
> 
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
 		 	   		  

From sjeik_appie at hotmail.com  Sun Jan 24 15:15:33 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 24 Jan 2016 20:15:33 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <20160122000000.GI4619@ando.pearwood.info>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>,
 <20160122000000.GI4619@ando.pearwood.info>
Message-ID: <DUB123-W1029A65709189467D5128B83C60@phx.gbl>

> Date: Fri, 22 Jan 2016 11:00:00 +1100
> From: steve at pearwood.info
> To: tutor at python.org
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> 
> Further thoughts on your question...
> 
> 
> On Wed, Jan 20, 2016 at 01:33:17PM +0000, Albert-Jan Roskam wrote:
> > Hi,
> > 
> > Like the subject says: Why is an OrderedDict not sliceable? (From the 
> > collections library). Was that an intentional omission, or a mistake? 
> > [1]
> > 
> > Background: I do not use OrderedDict very often, but I thought I could 
> > use it to look up street and city names using postcodes ([0-9]{4} 
> > [a-z]{2} format). I needed it to be ordered because I also wanted to 
> > be able to use bisect, which is needed when the postcode letters are 
> > missing. In short: a fast dict lookup for complete postcodes and less 
> > fast bisect lookup for in complete postcodes.
> 
> I'm not sure I understand your use-case here.
> 
> You have postcodes that look like this:
> 
> "1234az"
> 
> Correct? Why do you want them *ordered*? 
> 
> I think you are confusing OrderedDict for a "Sorted Dict". OrderedDict 
> doesn't keep the keys in sorted order, it keeps them in the order that 
> they were inserted. So unless you are super-careful to insert the 
> postcodes in sorted order, the order of them in the dict will be 
> whatever order you insert them:

You are right. See also my reply to Mark Lawrence, who made a similar remark.

> py> from collections import OrderedDict
> py> d = OrderedDict()
> py> d['1234az'] = "1 Smith Street"
> py> d['9999zz'] = "991203 Short Street"
> py> d['3456mx'] = "24 Hour Lane"
> py> for key in d:
> ...     print(key, d[key])
> ...
> 1234az 1 Smith Street
> 9999zz 991203 Short Street
> 3456mx 24 Hour Lane
> 
> 
> So even if OrderedDict supported slicing, that would not do what you 
> think it does.


Oscar Benjamin's link about collections.OrderedDict.__eq__ (or rather, the fact that it's not reimplemented) scared the heck outta me :-)
Maybe this class should be avoided altogether?
 
> Also, you have a problem -- what happens if the incomplete postcode is 
> missing the first digit? (I suppose for that case, you just have to do a 
> slow linear search.) What about transposed digits or other errors? I'm 
> glad I don't have to solve that problem!
> 
> 
> Anyway, I suggest doing something like this:
> 
> (1) Keep the known postcodes in a regular dict, not an ordered dict.
> 
> (2) Once you have built the dict, then copy the keys and sort them:
> 
> postcodes = {
>     '1234az': "1 Smith Street",
>     '9999zz': "991203 Short Street",
>     '3456mx': "24 Hour Lane",
>     }
> 
> array = sorted(postcodes.keys())
> 
> 
> (3) Each time you add a new postcode to the dict, use bisect to add it 
> to the array as well. To ensure you never forget, use a helper function:

But I only *read* from the postcode table. I never insert any postcodes, nor do I delete any (ok, a few times a year I load new definitions from the database, because new houses, with new postcodes, will have been built).  See also my mail to Alan (today).

> def insert(postcode, entry):
>     if postcode in postcodes:
>         # deal with duplicate/existing key
>         ...
>     else:
>         postcodes[postcode] = entry
>         bisect.insort(array, postcode)
> 
> 
> Same for deleting.
> 
> 
> 
> -- 
> Steve
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

 		 	   		  

From sjeik_appie at hotmail.com  Sun Jan 24 15:23:51 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 24 Jan 2016 20:23:51 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <85y4biluvr.fsf@benfinney.id.au>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>,
 <857fj3mgrr.fsf@benfinney.id.au>,
 <CAHVvXxTkthTWEmJ6Eum=u1-eUT=ZdpF97TyJW3UY9PpuO6K7dg@mail.gmail.com>,
 <8537tqna05.fsf@benfinney.id.au>, <85y4biluvr.fsf@benfinney.id.au>
Message-ID: <DUB123-W12BCF9E5743A0AEE9A7F6883C60@phx.gbl>

> To: tutor at python.org
> From: ben+python at benfinney.id.au
> Date: Fri, 22 Jan 2016 04:12:08 +1100
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> 
> Ben Finney <ben+python at benfinney.id.au> writes:
> 
> > Oscar Benjamin <oscar.j.benjamin at gmail.com> writes:
> >
> > > According to a narrow definition of indexed access. I would say that
> > > d[k] is index access even if d is a dict and k a key.
> 
> The sense of ?index? implied is used consistently throughout Python
> <URL:https://docs.python.org/3/glossary.html> to refer to the integer
> ordinal position in a sequence.

I appear to have confused the terms "sorted" and "ordered" (see the email I just sent to Mark Lawrence).  My OrderedDict was sorted on its keys, because I defined the dict using the result of an SQL query that ended with ORDER BY <names of dict keys here>. So in that case I needed a kind of "chameleon" datatype: both a mapping and an indexing type [1]
[1] https://docs.python.org/2/reference/datamodel.html#object.__getitem__ 
 

> It is not compatible with key access into a mapping.
> 
> > An index implies the ordinal position in a sequence. In a mapping, the
> > key is *not* referring to the position in a sequence, so is not a key.
> 
> ?the key ? is not an index?, I mean.
> 
> > So accessing an item in a mapping by key is not indexed access.
> 
> -- 
>  \     ?Facts do not cease to exist because they are ignored.? ?Aldous |
>   `\                                                            Huxley |
> _o__)                                                                  |
> Ben Finney
> 
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

 		 	   		  

From sjeik_appie at hotmail.com  Sun Jan 24 15:29:00 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 24 Jan 2016 20:29:00 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <CAHVvXxTkthTWEmJ6Eum=u1-eUT=ZdpF97TyJW3UY9PpuO6K7dg@mail.gmail.com>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>,
 <857fj3mgrr.fsf@benfinney.id.au>,
 <CAHVvXxTkthTWEmJ6Eum=u1-eUT=ZdpF97TyJW3UY9PpuO6K7dg@mail.gmail.com>
Message-ID: <DUB123-W2700355B2C30B9FE5BFD6A83C60@phx.gbl>

> From: oscar.j.benjamin at gmail.com
> Date: Thu, 21 Jan 2016 11:02:40 +0000
> To: ben+python at benfinney.id.au
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
> CC: tutor at python.org
> 
> On 21 January 2016 at 09:19, Ben Finney <ben+python at benfinney.id.au> wrote:
> > Albert-Jan Roskam <sjeik_appie at hotmail.com> writes:
> >
> >> Why is an OrderedDict not sliceable?
> >
> > Because slicing implies index access. The built-in ?dict? and
> > ?collections.OrderedDict? both do not support indexed access::
> 
> According to a narrow definition of indexed access. I would say that
> d[k] is index access even if d is a dict and k a key.
> 
> Albert-Jan I guess what you want is this:
> 
> from collections import OrderedDict
> 
> class SliceableOrderedDict(OrderedDict):
>     def __getitem__(self, sl):
>         if isinstance(sl, slice):
>             keys = list(self)
>             keyslice = slice(
>                 None if sl.start is None else keys.index(sl.start),
>                 None if sl.stop is None else keys.index(sl.stop),
>                 sl.step
>                 )
>             newitems = ((k, self[k]) for k in keys[keyslice])
>             return SliceableOrderedDict(newitems)
>         else:
>             return super().__getitem__(sl)
 
 
That looks interesting. I will check this out tomorrow at work. If I read it correctly this is indeed exactly what I meant. Thank you!!

 
> I guess that the authors of OrderedDict just didn't really consider
> this to be very useful. Apart from having ordered iteration
> OrderedDict is not really that deeply thought out. There's a thread on
> python-ideas about the inconsistent behaviour of the keys and values
> views and equality comparison of OrderedDicts on python-ideas at the
> moment:
> 
> https://mail.python.org/pipermail/python-ideas/2015-December/037472.html

As I said in a prior email: That is SCARY! Should I just avoid OrderedDict like the plague?
 

> --
> Oscar
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

 		 	   		  

From sjeik_appie at hotmail.com  Sun Jan 24 15:35:36 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 24 Jan 2016 20:35:36 +0000
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
In-Reply-To: <n828kt$k75$1@ger.gmane.org>
References: <CA+ZkTxsuDNFxmTCbjcoUrZnKjq5aGf6jokusUPvjEL_ub3kQ5w@mail.gmail.com>,
 <n828kt$k75$1@ger.gmane.org>
Message-ID: <DUB123-W3F365EA5E70066A7DD88783C60@phx.gbl>

> To: tutor at python.org
> From: __peter__ at web.de
> Date: Sun, 24 Jan 2016 11:22:19 +0100
> Subject: Re: [Tutor] Change datatype for specific columns in an 2D array & computing the mean
<snip>
 
> How do you want to convert the second and third column to int? Are A4 and B2 
> hex numbers? Then try
> 
> $ cat esawi.csv 
> 19,A4,B2,2
> 19,A5,B1,12
> $ python3
> Python 3.4.3 (default, Oct 14 2015, 20:28:29) 
> [GCC 4.8.4] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import numpy
> >>> def fromhex(s):
> ...     return int(s, 16)
> ... 
> >>> numpy.genfromtxt("esawi.csv", delimiter=",", converters={1:fromhex, 
> 2:fromhex})
> array([(19.0, 164, 178, 2.0), (19.0, 165, 177, 12.0)], 
>       dtype=[('f0', '<f8'), ('f1', '<i8'), ('f2', '<i8'), ('f3', '<f8')])

Wow what a cool trick!! I never knew about 'converters'. And (probably not surprisingly) pandas.read_csv has it too:
http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html
 
 
 
 		 	   		  

From lapsap7+python at gmail.com  Sun Jan 24 14:42:06 2016
From: lapsap7+python at gmail.com (STF)
Date: Sun, 24 Jan 2016 20:42:06 +0100
Subject: [Tutor] Noob: nested if-clauses
Message-ID: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>

Hi,

I've just started to learn Python thru some online courses and websites.
They just teach very basic things.  I've got some questions about "if" that
I'm unable to find the answers.  So let me ask the newbie questions here.

Let's see the following instructions:
--------
if condition_A:
    instruction_1
    instruction_2
    if condition_B:
      instruction_3
      instruction_4
    instruction_5
else:
    instruction_6
--------

* How to make Pythom understand that instruction_4 is a part of condition_B
if-clause but not a direct instruction of condition_A if-clause?  And how
to make Python understand that instruction_5 is outside of condition_B
if-clause?  Just by the number of white spaces in front of every
instruction??

* How to make Python understand that "else" belongs to the first
condition_A if-clause, not to the immediate condition_B if-clause?

* Suppose I put four white spaces in front of instruction_1, and then "tab
key" in front of instruction_2, would this break things?  I ask so because
most intelligent text editors would insert automatically a tab in place of
4 white spaces after we press Enter on a line with 4 leading white spaces.

* Do I really need to keep the consistency of 4 white spaces?  Not one more
or one less?

Thanks in advance.

From joel.goldstick at gmail.com  Sun Jan 24 16:05:36 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Sun, 24 Jan 2016 16:05:36 -0500
Subject: [Tutor] Noob: nested if-clauses
In-Reply-To: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
References: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
Message-ID: <CAPM-O+xL__oaAmMXhv7=QH2nc-X+T=XbJjuhJAYKZsCjkne-PA@mail.gmail.com>

On Sun, Jan 24, 2016 at 2:42 PM, STF <lapsap7+python at gmail.com> wrote:

> Hi,
>
> I've just started to learn Python thru some online courses and websites.
> They just teach very basic things.  I've got some questions about "if" that
> I'm unable to find the answers.  So let me ask the newbie questions here.
>
> Let's see the following instructions:
> --------
> if condition_A:
>     instruction_1
>     instruction_2
>     if condition_B:
>       instruction_3
>       instruction_4
>     instruction_5
> else:
>     instruction_6
> --------
>
> * How to make Pythom understand that instruction_4 is a part of condition_B
> if-clause but not a direct instruction of condition_A if-clause?  And how
> to make Python understand that instruction_5 is outside of condition_B
> if-clause?  Just by the number of white spaces in front of every
> instruction??
>
> * How to make Python understand that "else" belongs to the first
> condition_A if-clause, not to the immediate condition_B if-clause?
>
> * Suppose I put four white spaces in front of instruction_1, and then "tab
> key" in front of instruction_2, would this break things?  I ask so because
> most intelligent text editors would insert automatically a tab in place of
> 4 white spaces after we press Enter on a line with 4 leading white spaces.
>
> * Do I really need to keep the consistency of 4 white spaces?  Not one more
> or one less?
>

yes.  you can use 1 or 2 or any number of spaces, but do spacing
consistently.  4 is recommended. Don't mix tabs and spaces

>
> Thanks in advance.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From alan.gauld at btinternet.com  Sun Jan 24 16:08:25 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 24 Jan 2016 21:08:25 +0000
Subject: [Tutor] Noob: nested if-clauses
In-Reply-To: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
References: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
Message-ID: <n83eg8$ukt$1@ger.gmane.org>

On 24/01/16 19:42, STF wrote:

> Let's see the following instructions:
> --------
> if condition_A:
>     instruction_1
>     instruction_2
>     if condition_B:
>       instruction_3
>       instruction_4
>     instruction_5
> else:
>     instruction_6
> --------
> 
> * How to make Pythom understand that instruction_4 is a part of condition_B
> if-clause but not a direct instruction of condition_A if-clause?  

You've done it above by the indentation.

> to make Python understand that instruction_5 is outside of condition_B
> if-clause?  Just by the number of white spaces in front of every
> instruction??

Yes, the indent level tells Python where the instruction should be.

> * How to make Python understand that "else" belongs to the first
> condition_A if-clause, not to the immediate condition_B if-clause?

Again you've done it already, just use the indent level.

> * Suppose I put four white spaces in front of instruction_1, and then "tab
> key" in front of instruction_2, would this break things?  

In Python 2 things are a wee bit flexible but in Python 3 less so.
But in general avoid mixing them, stick to spaces. Most Python
programmers set their text editor/IDE to convert tabs to
spaces(usually 4)

> most intelligent text editors would insert automatically a tab in place of
> 4 white spaces after we press Enter on a line with 4 leading white spaces.

Most can also be configured not to use tabs at all and
for Python that's better. Tell us your editor and somebody
can probably advise on optimum settings.

> * Do I really need to keep the consistency of 4 white spaces?  Not one more
> or one less?

No you can have as many or as few as you like in your own code,
just be consistent. 4 just happens to be esy to read. And its
the standard for library code so if you want to write some code
for the standard library you will need to use 4 spaces. In
the interpreter (>>>) I often only use 2 just to save typing.
But for production code I stick with 4 - not a problem since
the editor(vim) does most of the work for me.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From breamoreboy at yahoo.co.uk  Sun Jan 24 16:56:30 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 24 Jan 2016 21:56:30 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <DUB123-W12BCF9E5743A0AEE9A7F6883C60@phx.gbl>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
 <857fj3mgrr.fsf@benfinney.id.au>
 <CAHVvXxTkthTWEmJ6Eum=u1-eUT=ZdpF97TyJW3UY9PpuO6K7dg@mail.gmail.com>
 <8537tqna05.fsf@benfinney.id.au> <85y4biluvr.fsf@benfinney.id.au>
 <DUB123-W12BCF9E5743A0AEE9A7F6883C60@phx.gbl>
Message-ID: <n83hah$g12$1@ger.gmane.org>

On 24/01/2016 20:23, Albert-Jan Roskam wrote:
>
> I appear to have confused the terms "sorted" and "ordered" (see the email I just sent to Mark Lawrence).  My OrderedDict was sorted on its keys, because I defined the dict using the result of an SQL query that ended with ORDER BY <names of dict keys here>. So in that case I needed a kind of "chameleon" datatype: both a mapping and an indexing type [1]
> [1] https://docs.python.org/2/reference/datamodel.html#object.__getitem__
>

Can you use one of the containers here 
http://www.grantjenks.com/docs/sortedcontainers/ ?

If yes it's as simple as:-

pip install sortedcontainers

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From steve at pearwood.info  Sun Jan 24 20:40:49 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 25 Jan 2016 12:40:49 +1100
Subject: [Tutor] Why is the name "self" optional instead of mandatory?
In-Reply-To: <CANDiX9JfY1ACZXwJBazz_tQXgEg--6n5Z26-NREgaEUoczyVfg@mail.gmail.com>
References: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
 <20160121114922.GF4619@ando.pearwood.info>
 <CANDiX9JfY1ACZXwJBazz_tQXgEg--6n5Z26-NREgaEUoczyVfg@mail.gmail.com>
Message-ID: <20160125014048.GK4619@ando.pearwood.info>

On Fri, Jan 22, 2016 at 11:10:39PM -0600, boB Stepp wrote:
> On Thu, Jan 21, 2016 at 5:49 AM, Steven D'Aprano <steve at pearwood.info> wrote:

> > class X:
> >     pass
> >
> > def func(this):
> >     print("instance %r called method" % this)
> >
> > X.method = func
> 
> Am I understanding this correctly?  It appears to me that you have
> successfully added a method to class X from outside of class X!  If
> this is the case, then this clarifies some things I was wondering
> about in my other thread.

Yes, you are reading it correctly.

Of course, in practice you wouldn't do that as shown. Why write the 
method outside the class when you could write it inside the class? There 
are some good reasons for doing something similar though:

(1) Perhaps you have a whole family of classes that share the same 
method. Three traditional solutions to this are to use inheritence, a 
mixin class, or traits. But a fourth is to create your classes without 
the method, then dynamically add the extra, shared, method into each one 
afterwards.

Possibly by using a simple decorator:

def method(self, arg): ...

def decorate(cls):
    cls.method = method
    return cls

@decorate
class Spam: ...

This technique is even more powerful when the method you are injecting 
is *slightly different* each time. For that, you can use a closure, 
created *inside* the decorator function. Play around with this example 
and see if you can understand what is going on:


# Warning: this may expand your mind.

def decorate(number):
    # Create a closure over the number.
    def method(self, x):
        """Return x + %d."""
        return x + number
    method.__doc__ %= number
    # Create a decorator.
    def decorator(cls):
        # Inject the method into the class.
        cls.method = method
        return cls
    # Return the decorator so it can be used.
    return decorator

@decorate(5)
class AddFive:
    pass

@decorate(9)
class AddNine:
    pass



(2) Another reason for doing this may be that you have an existing class 
from some library that you have to use. You can't subclass it, because 
too much of your code already depends on using that specific class. In 
some languages, like Java, perhaps the library authors marked the class 
as unsubclassable. But you want to add a new method for your own use.

Here's an example of this:

http://stackoverflow.com/questions/13730924/java-adding-fields-and-methods-to-existing-class

You'll notice that the answers given don't really solve the problem, 
apart from a vague and scary-sounding suggestion to use "byte-code 
injection". A number of people suggest subclassing, but a comment 
from another person says the the question also applies to him, but in 
his case subclassing isn't practical.

In Python, we have two solutions:

Write a function, and use that. Instead of calling obj.new_method(), 
we simply have new_function(obj). This option is available to Java as 
well, but hardly anyone ever thinks of it because Java is the Kingdom of 
Nouns: 

http://steve-yegge.blogspot.com.au/2006/03/execution-in-kingdom-of-nouns.html

Or *monkey-patch* the class using a new method we create on the outside.

https://en.wikipedia.org/wiki/Monkey_patch

Monkey-patching is a powerful technique, but should be used with care. 
Overuse is considered harmful:

http://devblog.avdi.org/2008/02/23/why-monkeypatching-is-destroying-ruby/


[...]
> I guess I am trying to wrap my mind around this incredible power and
> flexibility that Python provides.  I thought I had asked a relatively
> harmless question.  But it generated some strong responses!  It seemed
> like "self" had a similar utility of use as "print" to me.  After all,
> we can't redefine "print", can we?  But now I realize that I can do
> exactly that if I so choose.  That is both fascinating and scary!

Indeed. And in Python 3, print is a regular function, which means it 
*can* be redefined by shadowing, or even by monkey-patching the 
built-in module.


-- 
Steve

From steve at pearwood.info  Sun Jan 24 21:21:30 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 25 Jan 2016 13:21:30 +1100
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9LcTXwZdga7_pT_XcuuJW_Rb7bH+DS9gtMq6CurB5tkng@mail.gmail.com>
References: <CANDiX9+U47s=pKQozVk6DCB9xHZSBtw84_H+jhnsswUGj31n1g@mail.gmail.com>
 <CAGZAPF4QCu7tAx_jZuWJ342e30Q92Dnyb1Rr_nbbgTMQrs0cdg@mail.gmail.com>
 <20160121105708.GE4619@ando.pearwood.info>
 <CANDiX9LcTXwZdga7_pT_XcuuJW_Rb7bH+DS9gtMq6CurB5tkng@mail.gmail.com>
Message-ID: <20160125022128.GL4619@ando.pearwood.info>

On Fri, Jan 22, 2016 at 10:14:57PM -0600, boB Stepp wrote:
> On Thu, Jan 21, 2016 at 4:57 AM, Steven D'Aprano <steve at pearwood.info> wrote:

> > def spam(x, y):
> >     ...
> >
> > spam.extra_info = "whatever"
> 
> A new thing that I did not suspect I could do.  This bothers me for two reasons:
> 
>     1)  It does not seem right adding attributes to functions outside
> of its definition.

Well, unfortunately there's no syntax for adding attributes to a 
function *inside* its definition. If you put it inside the body of the 
function, it won't get run until you call the function:

def spam(arg):
    spam.widget = some_widget

spam.widget  # Fails because the function hasn't been run yet.


If you put it before the function definition, the function doesn't 
exist yet:

spam.widget = some_widget  # Fails because spam doesn't exist.
def spam(arg): 
    ...

So the only thing left is to put it *after* the definition.

In practice, I would create a decorator that adds the extra attribute, 
so you can write:


def add_widget(the_widget):
    def decorator(func):
        func.widget = the_widget
        return func
    return decorator

@add_widget(some_widget)
def spam(arg):
    ...



 
>     2)  spam.extra_info appears to be global:

`spam` is global, but only because you created it in the global 
namespace. You could have put it in a class, or nested inside another 
function. `extra_info` is not global, it is attached firmly to `spam`, 
no different from the 30 or so existing attributes of functions:

py> dir(lambda:None)
['__annotations__', '__call__', '__class__', '__closure__', '__code__', 
'__defaults__', '__delattr__', '__dict__', '__dir__', '__doc__', 
'__eq__', '__format__', '__ge__', '__get__', '__getattribute__', 
'__globals__', '__gt__', '__hash__', '__init__', '__kwdefaults__', 
'__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', 
'__qualname__', '__reduce__', '__reduce_ex__', '__repr__', 
'__setattr__', '__sizeof__', '__str__', '__subclasshook__']

or for that matter nearly any object. Even None has 20+ attributes.

You'll notice that *by default* all of the function attributes are 
__dunder__ names, which mean that they are used (somewhere) by the 
Python implementation. E.g. the __code__ object contains the actual 
byte-code of the function. But you can add your own, if you have a 
reason to do so.

It would have been very easy for the core developers to prohibit the 
addition of arbitrary attributes to functions: just don't give functions 
a __dict__ attribute. And indeed, in the earliest versions of Python, 
such as version 1.5, functions didn't have a __dict__ and you couldn't 
add attributes to them. So it is a deliberate choice to allow adding 
attributes to functions, but not strings, ints, floats etc.


[...]
> And I imagine I am being dense about something that is quite obvious
> to you:  How is this a useful feature to have?  What does it give me
> that is more useful than just saying something like:
> 
> just_another_global variable = "whatever"
> 
> ?


It keeps the function's data close to the function. What if you have two 
functions, spam and eggs, that both want to claim the name "widget" for 
their internal data? Do they just stomp all over each other's data? Or 
do you call them "spam_widget" and "eggs_widget"?

If you can write spam_widget, what's wrong with writing spam.widget 
instead?



> And what bothered me about my original example that started this
> thread is that when my typo
> 
> humdrum.sigh_strenght = 'high'
> 
> was accepted and did not generate an error, it seemed to mean to me
> that I was violating my object's data encapsulation.

That's a matter of opinion.

Encapsulation just means that data and code that operates on that 
data are bundled together in some sense. Encapsulation isn't even 
*necessarily* to do with objects. Ancient 1970s BASICs even had a 
primitive form of encapsulation, with the DATA statement. You could 
encapsulate data in the same file as the BASIC code that operated on 
that data.

One question is whether objects are *open* or *closed* to modifications. 
Python defaults to them being open unless there are good reasons for 
them to be closed. Other languages, like Java, default to them being 
closed: you can't easily add new attributes to Java objects after 
creation.


> It just seems to
> me that I should not be able to arbitrarily add new attributes from
> outside the class definition that created my object.  That seems
> similar to having a class Dog, to which from outside the class'
> definition, I decide to add a new Dog attribute that all dogs in this
> class can now have a tail sticking out of their noses.

That would be:

lassie = Dog()
lassie.nose.tail = make_tail()

Yes, that would be a silly thing to do. But how about this?

lassie = Dog()
lassie.collar = Collar()

I don't think that collar should be an attribute of all dogs. I know 
that Gaspode wouldn't be caught dead wearing a collar:

http://wiki.lspace.org/mediawiki/index.php/Gaspode

and I'm pretty sure that neither White Fang nor Buck would have collars:

https://en.wikipedia.org/wiki/White_Fang
https://en.wikipedia.org/wiki/The_Call_of_the_Wild

"collar" is not an attribute of the Dog class, but it might be an 
attribute of individual dogs.



-- 
Steve

From steve at pearwood.info  Mon Jan 25 05:05:42 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 25 Jan 2016 21:05:42 +1100
Subject: [Tutor] Value of tracebacks to malicious attackers?
In-Reply-To: <CANDiX9L5QEnjiDmYv95-dm7NeifABb=4KmV842kwLktwagih_w@mail.gmail.com>
References: <CANDiX9L5QEnjiDmYv95-dm7NeifABb=4KmV842kwLktwagih_w@mail.gmail.com>
Message-ID: <20160125100542.GM4619@ando.pearwood.info>

On Sat, Jan 23, 2016 at 10:52:27PM -0600, boB Stepp wrote:
> From page 202 of "Python Crash Course":  "..., but it's also not a
> good idea to let users see tracebacks.
[...]
> How much concern do you give this in designing and implementing your
> production code?  

Me personally? Absolutely none at all, as my audience is (1) mostly me; 
(2) or other Python developers; (3) assumed to be reasonably technical; 
and (4) running the code on their own machine. There's nothing they can 
learn from the traceback that they don't already have access to.

But on occasions where I am writing for non-technical uses (i.e. an 
application rather than a library) I would handle it something like 
this in the main application:


if __name__ == '__main__':
    try:
        main()
    except KeyboardInterrupt:
        log.log("keyboard interrupt")
        sys.exit()
    except SystemExit as e:
        log.log(e)
        raise
    else Exception as e:
        log.log(e)
        # show a GUI alert, or at least print a message
        # to the screen
        display_unexpected_error(e)
        sys.exit(1)


or something like that. The point is to catch any unhandled exception, 
log it, possibly notify the user that something bad happened, and then 
exit. The traceback never gets shown.


-- 
Steve

From steve at pearwood.info  Mon Jan 25 05:24:01 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 25 Jan 2016 21:24:01 +1100
Subject: [Tutor] Why do I not get an error when I mistakenly type
 "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
In-Reply-To: <CANDiX9Lh+nvQFg2EQWz8FBVBfLdhtsvxY8ZQu-dPbRGHAK4_jQ@mail.gmail.com>
References: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
 <20160123093048.GA36822@cskk.homeip.net>
 <CANDiX9+pe0Ggfo8DUP7AH0xtNOe-GMei95Gv=3G=rgdE-=pZdw@mail.gmail.com>
 <CANDiX9Lh+nvQFg2EQWz8FBVBfLdhtsvxY8ZQu-dPbRGHAK4_jQ@mail.gmail.com>
Message-ID: <20160125102401.GN4619@ando.pearwood.info>

On Sat, Jan 23, 2016 at 04:25:01PM -0600, boB Stepp wrote:

> I think I now have this nuked out.  I am only just now realizing how
> powerful .__dict__ is:
[...]
>             self.__dict__[attribute_name] = attribute_value

Indeed, but generally speaking you hardly ever need to manually operate 
with a __dunder__ method or attribute. They're not quite private, but 
they are reserved, and normally you would use the public interface.

Instead of obj.__dict__[name], one should use one of the attribute 
functions:

getattr(obj, name)
setattr(obj, name, 999)
delattr(obj, name)

Instead of obj.__dict__, one should use vars(obj).

And of course, we never write obj.__len__() when we can write len(obj) 
instead. And so forth.



-- 
Steve

From oscar.j.benjamin at gmail.com  Mon Jan 25 07:22:05 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 25 Jan 2016 12:22:05 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <DUB123-W36914C93BB77114F32CB1983C60@phx.gbl>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
 <n7rh0d$6tn$1@ger.gmane.org>
 <DUB123-W36914C93BB77114F32CB1983C60@phx.gbl>
Message-ID: <CAHVvXxTHpupQqRUnuwuPp-hubyda9NoPAJ+wVJZi4OsTfef=Mw@mail.gmail.com>

On 24 January 2016 at 19:47, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
>>
>> You appear to be confusing ordered and sorted.

>   You are correct. Is there a difference in the way those terms are used colloquially vs. in the field of Computer Science (Note: English is not my mother tongue)? Anyway, this page seems to suggest that "Ordered" means "gets messed up upon insertion, deletion, update: http://stackoverflow.com/questions/1084146/what-is-the-difference-between-an-ordered-and-a-sorted-collection

An ordered collection will preserve its order if not mutated. If
mutated the operation will have a well defined effect on the order.
For example list.append(a) adds a new element and the new element will
always be at the end of the list regardless of its value. A list might
be in sorted order (i.e. after you have called the .sort() method) but
when you call append the new element will go at the end so that it is
no longer sorted.

A sorted collection has a sort key and ensures that it always has an
order that corresponds to sorting by that key. So when you add a new
element the container will somehow ensure that it ends up in the right
position to keep the container sorted.

If you use an ordered container initialised with sorted data (as you
seem to be doing) then the distinction is only important if you mutate
the container. Specifically in the case of an OrderedDict which you
want sorted by keys the distinction is only important if you add new
keys to the OrderedDict. Any new keys will be implicitly added at the
end of the OrderedDict's ordering (as if you called append on a list).
Removing a key with e.g. pop() is fine and changing the value
associate with a key is fine: only adding new keys will break the sort
order.

So if you're not adding new keys to the OrderedDict as you work then
there is no reason not to use it in this situation.

--
Oscar

From oscar.j.benjamin at gmail.com  Mon Jan 25 07:27:57 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 25 Jan 2016 12:27:57 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <DUB123-W2700355B2C30B9FE5BFD6A83C60@phx.gbl>
References: <DUB123-W397AA5C24C7295BB180EC083C20@phx.gbl>
 <857fj3mgrr.fsf@benfinney.id.au>
 <CAHVvXxTkthTWEmJ6Eum=u1-eUT=ZdpF97TyJW3UY9PpuO6K7dg@mail.gmail.com>
 <DUB123-W2700355B2C30B9FE5BFD6A83C60@phx.gbl>
Message-ID: <CAHVvXxRt3aOpuChDiHeRPCkAgBVYAZ+ignXtForRSb_MPefcnQ@mail.gmail.com>

On 24 January 2016 at 20:29, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
>> I guess that the authors of OrderedDict just didn't really consider
>> this to be very useful. Apart from having ordered iteration
>> OrderedDict is not really that deeply thought out. There's a thread on
>> python-ideas about the inconsistent behaviour of the keys and values
>> views and equality comparison of OrderedDicts on python-ideas at the
>> moment:
>>
>> https://mail.python.org/pipermail/python-ideas/2015-December/037472.html
>
> As I said in a prior email: That is SCARY! Should I just avoid OrderedDict
> like the plague?

No, OrderedDict works fine. Just in answer to your question about why
this particular feature isn't there:

1) The slicing thing is conceptually not that well defined. To me it
was obvious what you meant but I suspect that many people would find
it confusing or unexpected. (see e.g. the initial responses from Steve
and Ben in this thread)
2) The OrderedDict isn't really that deeply thought out beyond "I want
a dict that has a consistent iteration order".

--
Oscar

From oscar.j.benjamin at gmail.com  Mon Jan 25 07:35:29 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 25 Jan 2016 12:35:29 +0000
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
In-Reply-To: <n831bo$37h$1@ger.gmane.org>
References: <CA+ZkTxsSHAhG4aEfkmgaAXziUhrqeO=C7aU9HwBHSjP+gYZUGA@mail.gmail.com>
 <n831bo$37h$1@ger.gmane.org>
Message-ID: <CAHVvXxRA9aYiuYjM49EkH2Wz41-i=DOkBrPY+hKsG2quWHFCLg@mail.gmail.com>

On 24 January 2016 at 17:24, Peter Otten <__peter__ at web.de> wrote:
>
> I'm an amateur with numpy, and unfortunately my favourite search engine
> didn't come up with a numpy-specific way to group rows in a 2D array.

What do you mean by "group rows"?

I thought the OP's problem is really to filter rows which I already
showed how to do in numpy.

--
Oscar

From __peter__ at web.de  Mon Jan 25 08:14:01 2016
From: __peter__ at web.de (Peter Otten)
Date: Mon, 25 Jan 2016 14:14:01 +0100
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
References: <CA+ZkTxsSHAhG4aEfkmgaAXziUhrqeO=C7aU9HwBHSjP+gYZUGA@mail.gmail.com>
 <n831bo$37h$1@ger.gmane.org>
 <CAHVvXxRA9aYiuYjM49EkH2Wz41-i=DOkBrPY+hKsG2quWHFCLg@mail.gmail.com>
Message-ID: <n8572s$a93$1@ger.gmane.org>

Oscar Benjamin wrote:

> On 24 January 2016 at 17:24, Peter Otten <__peter__ at web.de> wrote:
>>
>> I'm an amateur with numpy, and unfortunately my favourite search engine
>> didn't come up with a numpy-specific way to group rows in a 2D array.
> 
> What do you mean by "group rows"?

Given a table you can specify columns as keys and in the simplest case one 
column where you apply an aggregate function over the sets of rows with the 
same key.

If I understand you correctly you are doing that for a single known key, 
whereas I considered finding the keys part of the task. In SQL you'd spell 
that

[prob 1]
select key1, key2, sum(value) from some_table group by key1, key2;
 
> I thought the OP's problem is really to filter rows which I already
> showed how to do in numpy.

You solve

[prob 2] 
select sum(value) from some_table where key1=? and key2=?;

You'll eventually get from [prob 2] to [prob 1], but you need a few lines of 
Python.


From oscar.j.benjamin at gmail.com  Mon Jan 25 09:49:26 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 25 Jan 2016 14:49:26 +0000
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
In-Reply-To: <n8572s$a93$1@ger.gmane.org>
References: <CA+ZkTxsSHAhG4aEfkmgaAXziUhrqeO=C7aU9HwBHSjP+gYZUGA@mail.gmail.com>
 <n831bo$37h$1@ger.gmane.org>
 <CAHVvXxRA9aYiuYjM49EkH2Wz41-i=DOkBrPY+hKsG2quWHFCLg@mail.gmail.com>
 <n8572s$a93$1@ger.gmane.org>
Message-ID: <CAHVvXxSfFfsjhYhH0D05zbJQtnc=Xj=b3Uzj_odJgTkNbegSZw@mail.gmail.com>

On 25 January 2016 at 13:14, Peter Otten <__peter__ at web.de> wrote:
>> What do you mean by "group rows"?
>
> Given a table you can specify columns as keys and in the simplest case one
> column where you apply an aggregate function over the sets of rows with the
> same key.
>
> If I understand you correctly you are doing that for a single known key,
> whereas I considered finding the keys part of the task. In SQL you'd spell
> that
>
> [prob 1]
> select key1, key2, sum(value) from some_table group by key1, key2;
>
>> I thought the OP's problem is really to filter rows which I already
>> showed how to do in numpy.
>
> You solve
>
> [prob 2]
> select sum(value) from some_table where key1=? and key2=?;
>
> You'll eventually get from [prob 2] to [prob 1], but you need a few lines of
> Python.

Oh okay. It wasn't clear to me that was what the OP wanted.

You can do it in numpy by combining a few pieces. First we define an array:

In [1]: import numpy as np

In [2]: a = np.array([[5, 1.0], [1, 3.0], [5, 2.0], [1, 4.0], [5, -1.0]])

In [3]: a
Out[3]:
array([[ 5.,  1.],
       [ 1.,  3.],
       [ 5.,  2.],
       [ 1.,  4.],
       [ 5., -1.]])

Now we need to sort the array:

In [4]: a = np.sort(a, axis=0)

In [5]: a
Out[5]:
array([[ 1., -1.],
       [ 1.,  1.],
       [ 5.,  2.],
       [ 5.,  3.],
       [ 5.,  4.]])

Now we can access the 2nd column easy:

In [6]: a[:, 1]
Out[6]: array([-1.,  1.,  2.,  3.,  4.])

But we want to split that column according to the first column. We can
use the split function if we know the indices and we can get them with
diff:

In [7]: np.diff(a[:, 0])
Out[7]: array([ 0.,  4.,  0.,  0.])

n [14]: np.nonzero(np.diff(a[:, 0]))[0]
Out[14]: array([1])

In [15]: indices = np.nonzero(np.diff(a[:, 0]))[0] + 1

In [16]: indices
Out[16]: array([2])

Now we can use these to split the second column of the sorted array:

In [17]: grouped = np.split(a[:, 1], indices)

In [18]: grouped
Out[18]: [array([-1.,  1.]), array([ 2.,  3.,  4.])]

In [19]: list(map(np.mean, grouped))
Out[19]: [0.0, 3.0]

It's not exactly straight-forward but numpy has all the primitives to
make this reasonably efficient. If we also want the list of keys then:

In [23]: a[np.concatenate([[0], indices]), 0]
Out[23]: array([ 1.,  5.])

Altogether:

import numpy as np

a = np.array([[5, 1.0], [1, 3.0], [5, 2.0], [1, 4.0], [5, -1.0]])

a = np.sort(a, axis = 0)
indices = np.nonzero(np.diff(a[:, 0]))[0] + 1
means = list(map(np.mean, np.split(a[:, 1], indices)))
keys = a[np.concatenate([[0], indices]), 0]
group_means = dict(zip(keys, means))

print(group_means) # {1.0: 0.0, 5.0: 3.0}


If you want to key on multiple columns use lexsort instead of sort and
sum the diff array along rows but otherwise it's the same principle.

Looks easier with pandas :)

--
Oscar

From steve at pearwood.info  Mon Jan 25 10:02:26 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 26 Jan 2016 02:02:26 +1100
Subject: [Tutor] What's pure OO? [was Re: Why do I not get an error when I
 mistakenly type ...]
In-Reply-To: <20160123093048.GA36822@cskk.homeip.net>
References: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
 <20160123093048.GA36822@cskk.homeip.net>
Message-ID: <20160125150225.GO4619@ando.pearwood.info>

On Sat, Jan 23, 2016 at 08:30:48PM +1100, Cameron Simpson wrote:

> That is the pure OO way; 

Is this the room for an argument? I'd like the full half hour please.

http://www.montypython.net/scripts/argument.php


Without wishing to single out Cameron specifically, I'd like to take 
exception to the way folks are tossing around the term "pure OO". I 
don't think that's a term that adds much light to the discussion, and I 
think it risks being understood as "real OO", a term which is downright 
harmful.

At the very least, people ought to define their terms. What on earth is 
"pure OO"?

If I were to guess, I would think of pure OO in one of two ways:

(1) All values in the language are objects.

That describes Python: everything in Python, modules, ints, strings, 
functions, even classes themselves, are objects. That makes Python as 
"pure" an object-oriented language as it is possible to get.

In comparison, Java fails miserably: unlike Python, but default Java 
treats numbers, strings, booleans as native "unboxed" values, not 
objects.

This is the definition of "pure OO" language given by Wikipedia:

https://en.wikipedia.org/wiki/Object-oriented_programming#OOP_languages


(2) Alternatively, "pure OO" might mean a language which uses only OO 
syntax: alist.len() not len(alist).

Obviously Python is not pure in that sense, but then (i) I don't know 
any language which is, most languages allow non-OO syntax at least for 
arithmetic; and (ii) syntax is the most superficial and unimportant part 
of Object Oriented Programming. 


(3) Others (but not I) might fall for the "No True Scotsman" fallacy and 
use "pure OO" to mean "proper OO", for whatever definition of "proper" 
they like. Unfortunately, or perhaps fortunately, OO covers a lot of 
ground, and very little is mandatory. Just as true Scotsmen do sometimes 
wear trousers, and eat porridge with honey and milk, so almost any 
feature of OOP is optional:

(a) Classes are optional; prototype-based languages like Javascript 
and Lua are no less OO than class-based languages.

(b) Subtyping and inheritence are optional. Some people like to 
reserve the term "object-based" for languages with objects but no 
inheritence. Others distinguish between languages with nominal 
subtyping, like C++ and Swift, and structural subtyping, like Ocaml and 
Go. (Python arguably has both, as duck-typing is a form of 
structural subtyping.)

I could go on, but suffice to say that I will strongly object (pun 
intended) to any suggestion that Python is not a "proper" or even "pure" 
OO language because it lacks certain features some other OOP languages 
provide.

For example, it's popular in certain circles to say that No True OO 
Language lacks information-hiding (sometimes wrongly called 
encapsulation). Since Python has no "private" keyword, Python cannot be 
a proper OOP language like Java, or so they say.

But Java's private variables are not that private:

http://tutorials.jenkov.com/java-reflection/private-fields-and-methods.html

and Python's cooperative information hiding ("just ignore anything 
starting with a single underscore, since that's private") is actually no 
less private than Java's.




-- 
Steve

From parinay.svnit at gmail.com  Mon Jan 25 11:09:40 2016
From: parinay.svnit at gmail.com (Parinay Mahakur)
Date: Mon, 25 Jan 2016 21:39:40 +0530
Subject: [Tutor] Need Help
Message-ID: <CAHX6rtd=t1adZP1LMqZLqr7PSSqNsAjZOc3LU5KKAvkwaMmtpQ@mail.gmail.com>

Hello Tutors,


I need a program that should enable me to read values from a large number
of ASCII files and then I have to plot desired values - In this file
headings for all columns are also given. The link for the file is -

http://jsoc.stanford.edu/SUM75/D780005879/S00000/hmi.rdVfitsf_fd15.2171.015.355.0.-67.5.-20.0.fit.out

From cegarcia0323 at gmail.com  Mon Jan 25 12:39:26 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Mon, 25 Jan 2016 12:39:26 -0500
Subject: [Tutor] Help!
Message-ID: <CABV0pWy9DoDkWq4fFPq8xnzdKwkucEmqcdReHqK87gdZ3OwDfA@mail.gmail.com>

Hi,
I am trying to create a keyword search, so that someone can type in  a key
phrase and then the output be the value with the key. I have some code
already done but having some trouble getting it to work.
import csv
import json
import sys
from collections import defaultdict
from collections import Counter


class dictionary():
def __init__(self):
self.dict = defaultdict(list)
self.counted_dict = defaultdict(list)
self.grouped_dict = defaultdict(list)
self.other_dict = defaultdict(list)
self.final_dict = defaultdict(list)
self.total_dict = defaultdict(list)
self.keyword_dict = defaultdict(list)
def populate_dict(self, filename):
with open (filename, 'rb') as f:
reader = csv.reader(f)
next(reader, None)
for row in reader:
self.dict[row[2]].append(row[3])
def total_counts(self):
for key in self.dict.keys():
total = 0
b = Counter(self.dict[key])
for value in b:
total += b[value]
self.total_dict.update({key: str(total)})
def all_counts(self):
data_count = Counter()
for key in self.dict.keys():
self.counted_dict.update({key: Counter(self.dict[key])})
def grouped_counts(self):
for key in self.dict.keys():
total = 0
c = Counter(self.dict[key])
for value in c:
if c[value] >= 5:
self.grouped_dict.update({value: key + ': ' + str(c[value])})
elif c[value] <= 4:
total += c[value]
self.other_dict.update({key: 'other: ' + str(total)})
self.final_dict = self.grouped_dict, self.other_dict, self.total_dict

def keyword_items(defaultdict, lookup):
defaultdict = {' Tool Issue': ['Project Tool'], 'All catalogs missing or
not updating': ['Design Tool']}
for value, key in defaultdict.items():
for v in value:
if lookup in v:
return key
def json_output(self):
with open ('testing.txt', 'w') as text_file:
json.dump(self.final_dict, text_file, sort_keys = True, indent = 4)

From lapsap7+python at gmail.com  Mon Jan 25 10:52:42 2016
From: lapsap7+python at gmail.com (STF)
Date: Mon, 25 Jan 2016 16:52:42 +0100
Subject: [Tutor] Noob: nested if-clauses
In-Reply-To: <n83eg8$ukt$1@ger.gmane.org>
References: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
 <n83eg8$ukt$1@ger.gmane.org>
Message-ID: <CANsMTYUn8k4WEwfp2Fg_77xoCE=56foiL32of+Q-w8mcymN44A@mail.gmail.com>

Thanks to Joel and Alan for replying.

On 24 January 2016 at 22:08, Alan Gauld <alan.gauld at btinternet.com> wrote:

> On 24/01/16 19:42, STF wrote:
>
> > Let's see the following instructions:
> > --------
> > if condition_A:
> >     instruction_1
> >     instruction_2
> >     if condition_B:
> >       instruction_3
> >       instruction_4
> >     instruction_5
> > else:
> >     instruction_6
> > --------
> >
> > * How to make Pythom understand that instruction_4 is a part of
> condition_B
> > if-clause but not a direct instruction of condition_A if-clause?
>
> You've done it above by the indentation.
>

It's a total fluke.  I put the indentation like this to *visually* help
myself understand what I was going to write.

In the Python tutorial that I was using, the author only told us to use
indentation, without emphasizing on the size of it.


> > to make Python understand that instruction_5 is outside of condition_B
> > if-clause?  Just by the number of white spaces in front of every
> > instruction??
>
> Yes, the indent level tells Python where the instruction should be.
>
> > * How to make Python understand that "else" belongs to the first
> > condition_A if-clause, not to the immediate condition_B if-clause?
>
> Again you've done it already, just use the indent level.
>
> > * Suppose I put four white spaces in front of instruction_1, and then
> "tab
> > key" in front of instruction_2, would this break things?
>
> In Python 2 things are a wee bit flexible but in Python 3 less so.
> But in general avoid mixing them, stick to spaces. Most Python
> programmers set their text editor/IDE to convert tabs to
> spaces(usually 4)
>
> > most intelligent text editors would insert automatically a tab in place
> of
> > 4 white spaces after we press Enter on a line with 4 leading white
> spaces.
>
> Most can also be configured not to use tabs at all and
> for Python that's better. Tell us your editor and somebody
> can probably advise on optimum settings.
>

As I'm a newbie, I'm mostly using Python IDLE but sometimes I would use
Programmer's Notepad.


>
> > * Do I really need to keep the consistency of 4 white spaces?  Not one
> more
> > or one less?
>
> No you can have as many or as few as you like in your own code,
> just be consistent. 4 just happens to be esy to read. And its
> the standard for library code so if you want to write some code
> for the standard library you will need to use 4 spaces. In
> the interpreter (>>>) I often only use 2 just to save typing.
> But for production code I stick with 4 - not a problem since
> the editor(vim) does most of the work for me.
>

Let me ask an alternative question.  Suppose I have something like this:
----

if condition_C:
    instruction_10
   instruction_11
     instruction_12
----
There are 4 spaces in front of instruction_10, 3 spaces in front of
instruction_11 and 5 spaces in front of instruction_12.

What would happen to instruction_11 and instruction_12?  Would Python
ignore them?  Or would they be considered instructions outside the
if-clause?

Thanks again.

From maheshdabhade05 at gmail.com  Mon Jan 25 12:53:52 2016
From: maheshdabhade05 at gmail.com (Mahesh Dabhade)
Date: Mon, 25 Jan 2016 23:23:52 +0530
Subject: [Tutor] Variation in game of life in python solution by numpy method
Message-ID: <CAGZNBRFPqsenSur-cV70gubLrVKA86kx+eRs5d0aRZH2eRLPJQ@mail.gmail.com>

Hi...I have problem in game of life...
But I have 4 states rather than just two states like live and dead...
These are
0 : bare earth
1: grass 2: prey 3: predator
And rules are
1 )if 2 surrounded by less than 2 of 1 then 2 becomes 1.....starvation
2)if 0 surrounded by more than 0 of 1 then 0 becomes 1
3)if 1 surrounded by more than 1 of 2 then 1 will be 0
4)if 3 surrounded by less than 2 of 1 then  3 becomes 0
Now I am not even able to count the neighbors....
Can u plzz help me out in counting neighbours...
N even better can u plzz send the program code....
It will be great help....
Thanks

From alan.gauld at btinternet.com  Mon Jan 25 15:16:05 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 25 Jan 2016 20:16:05 +0000
Subject: [Tutor] What's pure OO? [was Re: Why do I not get an error when
 I mistakenly type ...]
In-Reply-To: <20160125150225.GO4619@ando.pearwood.info>
References: <CANDiX9Lc1heAxQPNbmG0Q2ZXan3qyyKgb8oboSy98F_GMXvgJw@mail.gmail.com>
 <20160123093048.GA36822@cskk.homeip.net>
 <20160125150225.GO4619@ando.pearwood.info>
Message-ID: <n85vq5$nl5$1@ger.gmane.org>

On 25/01/16 15:02, Steven D'Aprano wrote:
> On Sat, Jan 23, 2016 at 08:30:48PM +1100, Cameron Simpson wrote:
> 
>> That is the pure OO way; 
> 
> Is this the room for an argument? I'd like the full half hour please.

Personally I see OOP as a style thing rather than a language
issue. And I do think there is some sort of a "pure" definition
in all the object model theory papers that abound on the subject
in CS departments around the planet. (But that still leaves a
lot of wriggle room due to differences in opinion among the gurus)

But like most "pure" approaches it's thoroughly impractical in
isolation. As a result no pure OOP language exists because
it would be unusable. (Just like with pure FP, another case
where cries of purity are rife.)

OTOH striving towards purity is no bad thing. There is usually
a good reason lurking in the background. Otherwise we'd all
still be programming with goto and globals and variable names
like A$... But structured programming too has its
purists... remember single exit points anyone? (Actually F#
does insist on those!)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Mon Jan 25 15:21:21 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 25 Jan 2016 20:21:21 +0000
Subject: [Tutor] Need Help
In-Reply-To: <CAHX6rtd=t1adZP1LMqZLqr7PSSqNsAjZOc3LU5KKAvkwaMmtpQ@mail.gmail.com>
References: <CAHX6rtd=t1adZP1LMqZLqr7PSSqNsAjZOc3LU5KKAvkwaMmtpQ@mail.gmail.com>
Message-ID: <n86041$nl5$2@ger.gmane.org>

On 25/01/16 16:09, Parinay Mahakur wrote:

> I need a program that should enable me to read values from a large number
> of ASCII files and then I have to plot desired values

Have you considered a spreadsheet like Excel?
You could write a couple of macros to read the files
and to generate the plots.

If you want to do it in Python then we need a lot more
details, and some visibility of the code you've written
so far.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Mon Jan 25 15:29:07 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 25 Jan 2016 20:29:07 +0000
Subject: [Tutor] Help!
In-Reply-To: <CABV0pWy9DoDkWq4fFPq8xnzdKwkucEmqcdReHqK87gdZ3OwDfA@mail.gmail.com>
References: <CABV0pWy9DoDkWq4fFPq8xnzdKwkucEmqcdReHqK87gdZ3OwDfA@mail.gmail.com>
Message-ID: <n860ij$4ks$1@ger.gmane.org>

On 25/01/16 17:39, Chelsea G wrote:
> Hi,
> I am trying to create a keyword search, so that someone can type in  a key
> phrase and then the output be the value with the key. 

I'm not completely clear what you mean by a key phrase?
Do you mean a phrase matching the keys in your dictionary?(In which case
its easy) or domyou mean a random phrase? In which case what should happen?

> I have some code
> already done but having some trouble getting it to work.

Sadly you haven't posted in plain text so its lost all indentation,
making it nearly unreadable. Can you repost in plain text please?

> class dictionary():
> def __init__(self):
> self.dict = defaultdict(list)
> self.counted_dict = defaultdict(list)
> self.grouped_dict = defaultdict(list)
> self.other_dict = defaultdict(list)
> self.final_dict = defaultdict(list)
> self.total_dict = defaultdict(list)
> self.keyword_dict = defaultdict(list)
> def populate_dict(self, filename):
> with open (filename, 'rb') as f:
> reader = csv.reader(f)
> next(reader, None)
> for row in reader:
> self.dict[row[2]].append(row[3])
...

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From danny.yoo at gmail.com  Mon Jan 25 15:33:29 2016
From: danny.yoo at gmail.com (Danny Yoo)
Date: Mon, 25 Jan 2016 12:33:29 -0800
Subject: [Tutor] Noob: nested if-clauses
In-Reply-To: <CANsMTYUn8k4WEwfp2Fg_77xoCE=56foiL32of+Q-w8mcymN44A@mail.gmail.com>
References: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
 <n83eg8$ukt$1@ger.gmane.org>
 <CANsMTYUn8k4WEwfp2Fg_77xoCE=56foiL32of+Q-w8mcymN44A@mail.gmail.com>
Message-ID: <CAGZAPF52OOSnPVzc2E3p15nF+eD_UCszAkis5YpjOCOuMJ91Tg@mail.gmail.com>

> if condition_C:
>     instruction_10
>    instruction_11
>      instruction_12
> ----
> There are 4 spaces in front of instruction_10, 3 spaces in front of
> instruction_11 and 5 spaces in front of instruction_12.
>
> What would happen to instruction_11 and instruction_12?  Would Python
> ignore them?  Or would they be considered instructions outside the
> if-clause?
>

You should get an error at program parse time, as the system keeps track of
the indentation used to begin new blocks.  If it sees an indent level that
is inconsistent with those beginnings, it should know to signal a parse
error.

Please feel free to ask questions.  Good luck!

> Thanks again.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From breamoreboy at yahoo.co.uk  Mon Jan 25 15:09:14 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 25 Jan 2016 20:09:14 +0000
Subject: [Tutor] Need Help
In-Reply-To: <CAHX6rtd=t1adZP1LMqZLqr7PSSqNsAjZOc3LU5KKAvkwaMmtpQ@mail.gmail.com>
References: <CAHX6rtd=t1adZP1LMqZLqr7PSSqNsAjZOc3LU5KKAvkwaMmtpQ@mail.gmail.com>
Message-ID: <n85vde$dov$1@ger.gmane.org>

On 25/01/2016 16:09, Parinay Mahakur wrote:
> Hello Tutors,
>
> I need a program that should enable me to read values from a large number
> of ASCII files and then I have to plot desired values - In this file
> headings for all columns are also given. The link for the file is -
>
> http://jsoc.stanford.edu/SUM75/D780005879/S00000/hmi.rdVfitsf_fd15.2171.015.355.0.-67.5.-20.0.fit.out

Start here https://docs.python.org/3/tutorial/index.html as we don't 
write code for you.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From breamoreboy at yahoo.co.uk  Mon Jan 25 15:11:23 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 25 Jan 2016 20:11:23 +0000
Subject: [Tutor] Variation in game of life in python solution by numpy
 method
In-Reply-To: <CAGZNBRFPqsenSur-cV70gubLrVKA86kx+eRs5d0aRZH2eRLPJQ@mail.gmail.com>
References: <CAGZNBRFPqsenSur-cV70gubLrVKA86kx+eRs5d0aRZH2eRLPJQ@mail.gmail.com>
Message-ID: <n85vhf$dov$2@ger.gmane.org>

On 25/01/2016 17:53, Mahesh Dabhade wrote:
> Hi...I have problem in game of life...
> But I have 4 states rather than just two states like live and dead...
> These are
> 0 : bare earth
> 1: grass 2: prey 3: predator
> And rules are
> 1 )if 2 surrounded by less than 2 of 1 then 2 becomes 1.....starvation
> 2)if 0 surrounded by more than 0 of 1 then 0 becomes 1
> 3)if 1 surrounded by more than 1 of 2 then 1 will be 0
> 4)if 3 surrounded by less than 2 of 1 then  3 becomes 0
> Now I am not even able to count the neighbors....
> Can u plzz help me out in counting neighbours...
> N even better can u plzz send the program code....

Please show us the code that you have so far as we do not write code for 
you.

> It will be great help....
> Thanks

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From alan.gauld at btinternet.com  Mon Jan 25 15:46:10 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 25 Jan 2016 20:46:10 +0000
Subject: [Tutor] Noob: nested if-clauses
In-Reply-To: <CANsMTYUn8k4WEwfp2Fg_77xoCE=56foiL32of+Q-w8mcymN44A@mail.gmail.com>
References: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
 <n83eg8$ukt$1@ger.gmane.org>
 <CANsMTYUn8k4WEwfp2Fg_77xoCE=56foiL32of+Q-w8mcymN44A@mail.gmail.com>
Message-ID: <n861ii$kih$1@ger.gmane.org>

On 25/01/16 15:52, STF wrote:

> It's a total fluke.  I put the indentation like this to *visually* help
> myself understand what I was going to write.

That's one of the good things about Python, if it looks right
it very often is right.

> In the Python tutorial that I was using, the author only told us to use
> indentation, without emphasizing on the size of it.

Quite right the amount is not important(syntactically at least) provided
you are consistent.

> As I'm a newbie, I'm mostly using Python IDLE but sometimes I would use
> Programmer's Notepad.

I don't know PN but IDLE will keep you right most of the time.

> Let me ask an alternative question.  Suppose I have something like this:
> ----
> 
> if condition_C:
>     instruction_10
>    instruction_11
>      instruction_12
> ----
> There are 4 spaces in front of instruction_10, 3 spaces in front of
> instruction_11 and 5 spaces in front of instruction_12.
> 
> What would happen to instruction_11 and instruction_12?  

One of the best things about Python is the interpreter.
Just try it and see. It's much faster than posting a question
here and you can be sure it's the correct answer! If you
don't understand what you see, then come here.

Just use some print statements or simple assignments
for example:

>>> if True:
...    print 'in the if'
...   print 'still here'
...      y = 5 * 6
...

what happens?

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Mon Jan 25 15:48:34 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 25 Jan 2016 20:48:34 +0000
Subject: [Tutor] Variation in game of life in python solution by numpy
 method
In-Reply-To: <CAGZNBRFPqsenSur-cV70gubLrVKA86kx+eRs5d0aRZH2eRLPJQ@mail.gmail.com>
References: <CAGZNBRFPqsenSur-cV70gubLrVKA86kx+eRs5d0aRZH2eRLPJQ@mail.gmail.com>
Message-ID: <n861n1$kih$2@ger.gmane.org>

On 25/01/16 17:53, Mahesh Dabhade wrote:

> Now I am not even able to count the neighbors....

Then start with that as a first step.

> Can u plzz help me out in counting neighbours...

We need to see what you are doing to be able to help

> N even better can u plzz send the program code....

No, we won't do your work for you, we only
try to help you do it.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From starfas_s at yahoo.com  Mon Jan 25 16:34:58 2016
From: starfas_s at yahoo.com (Sam Starfas)
Date: Mon, 25 Jan 2016 21:34:58 +0000 (UTC)
Subject: [Tutor] Can This Script Be Modified to Read Many Files?..
References: <1567077832.1118549.1453757698483.JavaMail.yahoo.ref@mail.yahoo.com>
Message-ID: <1567077832.1118549.1453757698483.JavaMail.yahoo@mail.yahoo.com>

Hi,I am very new to Python, but having fun learning.?
I need to have a script read all of the XML files contents that are in a directory, pull out the contents of an element, in my case <uicontrol>, and list them in an output file. I have this script that does exactly what I need. But in my beginning Python class we didn't go over reading all files, only one file at a time.?
Can the below script be modified to scan/read all of the XML files in a directory instead of just the file typed into the script? If so, how do I do this?
Thanks for all the help.Sam

import xml.etree.ElementTree as ETtree = ET.parse('TEST.xml') ? <-- Want to read all the files in directory, not type in the filename.root = tree.getroot()

for uicontrol in root.iter('uicontrol'):? ? print(uicontrol.text)

From alan.gauld at btinternet.com  Mon Jan 25 18:35:13 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 25 Jan 2016 23:35:13 +0000
Subject: [Tutor] Can This Script Be Modified to Read Many Files?..
In-Reply-To: <1567077832.1118549.1453757698483.JavaMail.yahoo@mail.yahoo.com>
References: <1567077832.1118549.1453757698483.JavaMail.yahoo.ref@mail.yahoo.com>
 <1567077832.1118549.1453757698483.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <n86bfh$l83$1@ger.gmane.org>

On 25/01/16 21:34, Sam Starfas via Tutor wrote:

> Can the below script be modified to scan/read all of the XML files in a directory

You could do it manually using

for file in glob.glob("*.xml"):

But you need to think about all the other possible file
endings too.

Or you could look at the fileinput module which takes a
different approach.

Finally if you need to process subdirectories too you
can use the os.walk() function

The Python docs describe how to use glob, fileinput
and os.walk.

HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From bgailer at gmail.com  Mon Jan 25 18:41:10 2016
From: bgailer at gmail.com (Bob Gailer)
Date: Mon, 25 Jan 2016 18:41:10 -0500
Subject: [Tutor] Can This Script Be Modified to Read Many Files?..
In-Reply-To: <1567077832.1118549.1453757698483.JavaMail.yahoo@mail.yahoo.com>
References: <1567077832.1118549.1453757698483.JavaMail.yahoo.ref@mail.yahoo.com>
 <1567077832.1118549.1453757698483.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <CAP1rxO4XZbf6amDGSdoHwC6qOtfj5rdd6mVB_nVYCpmXyT7Qfw@mail.gmail.com>

On Jan 25, 2016 6:26 PM, "Sam Starfas via Tutor" <tutor at python.org> wrote:
>
> Hi,I am very new to Python, but having fun learning.
> I need to have a script read all of the XML files contents that are in a
directory, pull out the contents of an element, in my case <uicontrol>, and
list them in an output file. I have this script that does exactly what I
need. But in my beginning Python class we didn't go over reading all files,
only one file at a time.
> Can the below script be modified to scan/read all of the XML files in a
directory instead of just the file typed into the script? If so, how do I
do this?
> Thanks for all the help.Sam
>
> import xml.etree.ElementTree as ETtree = ET.parse('TEST.xml')   <-- Want
to read all the files in directory, not type in the filename.root =
tree.getroot()
>
> for uicontrol in root.iter('uicontrol'):    print(uicontrol.text)
As you probably noticed your code got mangled this is probably due to the
email program you're using what you should do is have it send plain text so
the formatting will be preserved.
I hope it's obvious that you will need a loop to process multiple files so
I'm going to assume that what you want is how to get a list of files
meeting some criteria in a directory. I'm dictating into my cell phone
right now so I can't look some things up but my presumption is you will
need to use the glob function in the glob module to get the list of
filenames, then iterate over that list. Hope that helps.

From esawiek at gmail.com  Mon Jan 25 22:07:10 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Mon, 25 Jan 2016 22:07:10 -0500
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
Message-ID: <CA+ZkTxsE30T7JsGy_mkqWbP4uvwu0RBKTTdTX8pJSJnB23uoag@mail.gmail.com>

Thank you all for your patience and dedication to this noble effort-the
list. I actually completed my project using Excel and python; unfortunately
my python code is not a python-like at all b/c I used way too many loops
and if statements. Plus I really want to learn python b/c I am a decent
programmer in other languages.


Here is a copy of my code and the csv file. It?s similar to the one I
posted earlier. For now I am not even concerned with mean or other
computations. All I want is to be able read a csv file which has string,
float, integer, date, and etc. datatypes. These are the majority of file
types I work with. I want to transfer it to a python array with proper
datatype. For example on my file, I want the 1st column integer, the 4th
float, the 2nd and 3rd string and the 5th date. I spent a few hours looking
online and in python documentation and did not find anything simple for me
to use.


I tried the function posted here but it?s too complicated for me-you can
see what I know about python. If there is no simpler way to do this then
that?s OK. I will keep on reading and see what happens.

I even tried to do some of it manually but did not work, as you see in my
code. Thank you all for your continuous help and dedication to this great
site----EKE


++ Code++

import numpy as np

import csv



DataMat=[]

np.set_printoptions(precision=2)



with open('c:/Users/EK Esawi/My Documents/Python Scripts/TestFile1.csv') as
Test1:

    reader = csv.reader(Test1, delimiter=',')

#    next(Test1)

    for row in reader:

        DataMat.append(row)



DataMat=np.array(DataMat)



col1=np.array(DataMat[1::,0], dtype=int)

col2=np.array(DataMat[1::,1])

col3=np.array(DataMat[1::,1])

col4=np.array(DataMat[1::,3], dtype=float)

col5=np.array(DataMat[1::,4])



++ csv file++

AA

BB

CC

DD

EE

1

A1

B1

11.2

11/20/2011

2

A2

B2

2.5

10/21/2011

3

A3

B3

13.67

9/21/2011

4

A4

B4

14.2

8/22/2011

5

A5

B5

20

7/23/2011

From oscar.j.benjamin at gmail.com  Tue Jan 26 04:41:01 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 26 Jan 2016 09:41:01 +0000
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
In-Reply-To: <CA+ZkTxsE30T7JsGy_mkqWbP4uvwu0RBKTTdTX8pJSJnB23uoag@mail.gmail.com>
References: <CA+ZkTxsE30T7JsGy_mkqWbP4uvwu0RBKTTdTX8pJSJnB23uoag@mail.gmail.com>
Message-ID: <CAHVvXxQeME1ckHw7kPcMO3o9U4vkgpKke_-x0rFu1nJ123a-zw@mail.gmail.com>

On 26 January 2016 at 03:07, Ek Esawi <esawiek at gmail.com> wrote:
>
> Here is a copy of my code and the csv file.

> ++ csv file++
>
> AA
>
> BB
>
> CC
<snip>

This doesn't look like a csv file. Is that what the actual contents of
the csv file looks like if you open it in a *text editor*?

If this was a csv file I'd expect it to look something like:

AA,BB,CC,12/1/2015
DD,EE,FF,24/1/2015
...

--
Oscar

From steve at pearwood.info  Tue Jan 26 10:24:35 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 27 Jan 2016 02:24:35 +1100
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <DUB123-W36914C93BB77114F32CB1983C60@phx.gbl>
References: <n7rh0d$6tn$1@ger.gmane.org>
 <DUB123-W36914C93BB77114F32CB1983C60@phx.gbl>
Message-ID: <20160126152435.GT4619@ando.pearwood.info>

On Sun, Jan 24, 2016 at 07:47:47PM +0000, Albert-Jan Roskam wrote:

> > You appear to be confusing ordered and sorted.   
> 
> You are correct. Is there a difference in the way those terms are 
> used colloquially vs. in the field of Computer Science (Note: English 
> is not my mother tongue)? 

In ordinary English, "ordered" and "sorted" often are used to mean the 
same thing. People do often use sorted and ordered as interchangeable, 
but the definitions are slightly different:



ordered \ordered\ adj.
   1. having or evincing a systematic arrangement; especially,
      having elements succeeding in order according to rule; as,
      an ordered sequence; an ordered pair. Opposite of
      disordered or unordered. [Narrower terms:
      abecedarian, alphabetical; {consecutive, sequent,
      sequential, serial, successive ]
      [WordNet 1.5 +PJC]

   2. arranged in order.

Sort \Sort\, v. t. [imp. & p. p. Sorted; p. pr. & vb. n.
   Sorting.]
   1. To separate, and place in distinct classes or divisions,
      as things having different qualities; as, to sort cloths
      according to their colors; to sort wool or thread
      according to its fineness.
      [1913 Webster]


The way I would put it is that "sorted" means the items are ordered 
according to some specific rule or property of the items themselves, 
e.g. to sort your clothes by colour. "Ordered" is more general: it just 
means to have some order, which may be according to a rule or property, 
or it may be in whatever sequence the items happen to have.

Books on a shelf have some order, the order that they appear when you 
read them from left to right, regardless of whether they are sorted by 
author, title, height, colour or at random. Books jumbled up in a bag 
have no order.

Ordinary dicts are like books in a bag. You reach in and grab whatever 
book happens to come to hand first. OrderedDicts are like books on a 
shelf: you can systematically touch each book in order starting from the 
left, and new books are always added to the right.

-- 
Steve

From cegarcia0323 at gmail.com  Tue Jan 26 09:26:32 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Tue, 26 Jan 2016 09:26:32 -0500
Subject: [Tutor] Help!
Message-ID: <CABV0pWzC2k582s02E37qO6-QPj--aTNNEfCMZZvDFyXZU_vKXg@mail.gmail.com>

Hi,
I am working on a python script to automate reporting. And I am working on
creating a keyword search. For example, if I want to search for the word
Tool in the value and see what keys are associated with that. So say the
value I have to search is Tool World and I want to know what key is
associated with Tool World I search Tool World and it comes up with several
results like missing or not updating and catalog issue. I have the basic
code for it figured out but I have created my own dictionary for it called
mydict and put a few key and values in but I want my code to search the csv
file that I am importing and then take the info I am getting from the
search results and put it in its own text file. I have attached my document.
-------------- next part --------------
import csv 
import json 
import sys 
from collections import defaultdict 
from collections import Counter 


class dictionary(): 
	def __init__(self):
		self.dict = defaultdict(list)
		self.counted_dict = defaultdict(list)
		self.grouped_dict = defaultdict(list)
		self.other_dict = defaultdict(list)
		self.final_dict = defaultdict(list)
		self.total_dict = defaultdict(list)
		self.search_dict = defaultdict(list)
	
	
		
		
	def populate_dict(self, filename):
		with open (filename, 'rb') as f:
			reader = csv.reader(f)
			next(reader, None) 
			for row in reader:
				self.dict[row[2]].append(row[3])  
	
	def total_counts(self):
		for key in self.dict.keys():
			total = 0
			b = Counter(self.dict[key])
			for value in b:
				total += b[value]
				self.total_dict.update({key: str(total)})
		
	def all_counts(self):
		data_count = Counter()
		for key in self.dict.keys(): 
			self.counted_dict.update({key: Counter(self.dict[key])})
			
	
	def grouped_counts(self):
		for key in self.dict.keys():
			total = 0
			c = Counter(self.dict[key]) 
			for value in c:
				if c[value] >= 5:
					self.grouped_dict.update({value: key + ': ' + str(c[value])})
				
				elif c[value] <= 4:
					
					total += c[value]
					self.other_dict.update({key: 'other: ' + str(total)})
	
			self.final_dict = self.grouped_dict, self.other_dict, self.total_dict
			
	mydict = {'Project Tool Issue': ['CPO Project Tool'], 'All catalogs missing or not updating': ['20/20 Design Tool']}
	def search(mydict, lookup):
		for key, value in mydict.iteritems():
			for v in value:
				if lookup in v:
					#return key
					print key
					
	search(mydict, 'Tool')
		
			
	# def txt_output(self, filename):
		# a = filename.split('.')
		# self.txt_output = a + '.txt'
		# print a
	
	
		
		
			
	
	def json_output(self):
		with open ('testing.txt', 'w') as text_file:
			json.dump(self.final_dict, text_file, sort_keys = True, indent = 4)

From cegarcia0323 at gmail.com  Tue Jan 26 13:05:27 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Tue, 26 Jan 2016 13:05:27 -0500
Subject: [Tutor] help!
Message-ID: <CABV0pWyRu1GbyiHDztHG0dqx4E5Sou7oRd3QpsA64=Mx7ntKGw@mail.gmail.com>

Hi,
I am working on a python script to automate reporting. And I am working on
creating a keyword search. For example, if I want to search for the word
Tool in the value and see what keys are associated with that. So say the
value I have to search is Tool World and I want to know what key is
associated with Tool World I search Tool World and it comes up with several
results like missing or not updating and catalog issue. I have the basic
code for it figured out but I have created my own dictionary for it called
mydict and put a few key and values in but I want my code to search the csv
file that I am importing and then take the info I am getting from the
search results and put it in its own text file. Also, I dont want to have
to create the mydict line with the keywords I want to be able to type in a
value like Tool and search through the csv file and then output the results
to a text file. I have attached my code.
-------------- next part --------------
import csv 
import json 
import sys 
from collections import defaultdict 
from collections import Counter 


class dictionary(): 
	def __init__(self):
		self.dict = defaultdict(list)
		self.counted_dict = defaultdict(list)
		self.grouped_dict = defaultdict(list)
		self.other_dict = defaultdict(list)
		self.final_dict = defaultdict(list)
		self.total_dict = defaultdict(list)
		self.search_dict = defaultdict(list)
		
		
	def populate_dict(self, filename):
		with open (filename, 'rb') as f:
			reader = csv.reader(f)
			next(reader, None) 
			for row in reader:
				self.dict[row[2]].append(row[3]) 
				
				
	
	def total_counts(self):
		for key in self.dict.keys():
			total = 0
			b = Counter(self.dict[key])
			for value in b:
				total += b[value]
				self.total_dict.update({key: str(total)})
		
	def all_counts(self):
		data_count = Counter()
		for key in self.dict.keys(): 
			self.counted_dict.update({key: Counter(self.dict[key])})
			
	
	def grouped_counts(self):
		for key in self.dict.keys():
			total = 0
			c = Counter(self.dict[key]) 
			for value in c:
				if c[value] >= 5:
					self.grouped_dict.update({value: key + ': ' + str(c[value])})
				
				elif c[value] <= 4:
					
					total += c[value]
					self.other_dict.update({key: 'other: ' + str(total)})
	
			self.final_dict = self.grouped_dict, self.other_dict, self.total_dict
			
	mydict = {'Project Tool Issue': ['CPO Project Tool'], 'All catalogs missing or not updating': ['20/20 Design Tool']}
	def search(mydict, lookup):
		for key, value in mydict.iteritems():
			for v in value:
				if lookup in v:
					 #return key
					print key
					
	search(mydict, 'Tool')
		
			
	# def txt_output(self, filename):
		# a = filename.split('.')
		# self.txt_output = a + '.txt'
		# print a
	
	
		
		
			
	
	def json_output(self):
		with open ('scripttesting.txt', 'w') as text_file:
			json.dump(self.final_dict, text_file, sort_keys = True, indent = 4)

From esawiek at gmail.com  Tue Jan 26 09:39:00 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Tue, 26 Jan 2016 09:39:00 -0500
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
Message-ID: <CA+ZkTxvN51C38UgeAopaVDOJKFrQhDbhf+QMwLNyKVUx0vkJCQ@mail.gmail.com>

Sorry! but it's a csv file. I just copied it directly from an opened csv
file in excel. And you're correct it looks like what you wrote. EKE

From joel.goldstick at gmail.com  Tue Jan 26 14:48:56 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 26 Jan 2016 14:48:56 -0500
Subject: [Tutor] help!
In-Reply-To: <CABV0pWyRu1GbyiHDztHG0dqx4E5Sou7oRd3QpsA64=Mx7ntKGw@mail.gmail.com>
References: <CABV0pWyRu1GbyiHDztHG0dqx4E5Sou7oRd3QpsA64=Mx7ntKGw@mail.gmail.com>
Message-ID: <CAPM-O+y6V_bjFE0R5O=sDRABUUhz1uPKH9zBk8XiAf2wdxMUZA@mail.gmail.com>

On Tue, Jan 26, 2016 at 1:05 PM, Chelsea G <cegarcia0323 at gmail.com> wrote:

> Hi,
> I am working on a python script to automate reporting. And I am working on
> creating a keyword search. For example, if I want to search for the word
> Tool in the value and see what keys are associated with that. So say the
> value I have to search is Tool World and I want to know what key is
> associated with Tool World I search Tool World and it comes up with several
> results like missing or not updating and catalog issue. I have the basic
> code for it figured out but I have created my own dictionary for it called
> mydict and put a few key and values in but I want my code to search the csv
> file that I am importing and then take the info I am getting from the
> search results and put it in its own text file. Also, I dont want to have
> to create the mydict line with the keywords I want to be able to type in a
> value like Tool and ursearch through the csv file and then output the
> results
> to a text file. I have attached my code.
>
> Welcome!  You can paste your code in your question.  People can't or won't
read attachments here, so for better help try to cut the code to show the
issue or problem, then paste it.

I'm not quite sure I understand what you want to create.  Could you give a
snippet of a typical csv file that you want to examine?  And the format and
substance of the output you wish to create?

> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
>


-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From alan.gauld at btinternet.com  Tue Jan 26 15:08:19 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 26 Jan 2016 20:08:19 +0000
Subject: [Tutor] help!
In-Reply-To: <CABV0pWyRu1GbyiHDztHG0dqx4E5Sou7oRd3QpsA64=Mx7ntKGw@mail.gmail.com>
References: <CABV0pWyRu1GbyiHDztHG0dqx4E5Sou7oRd3QpsA64=Mx7ntKGw@mail.gmail.com>
Message-ID: <n88jnk$47k$1@ger.gmane.org>

On 26/01/16 18:05, Chelsea G wrote:

> creating a keyword search. For example, if I want to search for the word
> Tool in the value and see what keys are associated with that. So say the
> value I have to search is Tool World and I want to know what key is
> associated with Tool World I search Tool World and it comes up with several
> results like missing or not updating and catalog issue. 


> I have the basic code for it figured out

Not if its the code you attached you haven't.
It won't do anything because all of the code is inside
the class definition.  So until you create an instance
of the class nothing will happen.

And that's assuming there are no other errors, and I'm
pretty sure I can see at least 1 more.

> ... I want my code to search the csv
> file that I am importing and then take the info I am getting from the
> search results and put it in its own text file. Also, I dont want to have
> to create the mydict line with the keywords I want to be able to type in a
> value like Tool and search through the csv file and then output the results
> to a text file. I have attached my code.

That's all fine and dandy but do it one piece at a time,
don't try to make a huge program work all at once.
Pick one thing and get it working.
I suggest:
1) load the csv file into a dict and print out the result.
2) implement user entry of a dict item, print the result
3) implement the search, print the found items value(s)
4) output the results to a text file (BTW why have you
   JSON functions if you want a text file? Or do you
   really mean a JSON file?

Bite it off one bit at a time. It's much easier to modify
working code than to fix everything at once.

BTW you were lucky your attachment made it through.
The list server often loses them. Its not a lot of code
so just paste it into your mail messages. Its much easier
for us to see and respond to that way.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From oscar.j.benjamin at gmail.com  Tue Jan 26 18:30:47 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 26 Jan 2016 23:30:47 +0000
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
In-Reply-To: <CA+ZkTxvN51C38UgeAopaVDOJKFrQhDbhf+QMwLNyKVUx0vkJCQ@mail.gmail.com>
References: <CA+ZkTxvN51C38UgeAopaVDOJKFrQhDbhf+QMwLNyKVUx0vkJCQ@mail.gmail.com>
Message-ID: <CAHVvXxQda=_XLJOxBAHeCuYrOaYJ4dxUo-zf+QasSUkvvziuRg@mail.gmail.com>

On 26 Jan 2016 19:32, "Ek Esawi" <esawiek at gmail.com> wrote:
>
> Sorry! but it's a csv file. I just copied it directly from an opened csv
> file in excel. And you're correct it looks like what you wrote. EKE

Can you show exactly how it looks? I.e. open it in a text editor and paste
the first say 20 lines here.

--
Oscar

From cegarcia0323 at gmail.com  Tue Jan 26 15:57:51 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Tue, 26 Jan 2016 15:57:51 -0500
Subject: [Tutor] Help
Message-ID: <CABV0pWxSKNDcaHp+gNTwetg5HOdm+bErs1DM7NgRsd9buWgO1Q@mail.gmail.com>

Hi,
I am working on a python script to automate reporting. And I am working on
creating a keyword search. For example, if I want to search for the word
Tool in the value and see what keys are associated with that. So say the
value I have to search is Tool World and I want to know what key is
associated with Tool World I search Tool World and it comes up with several
results like missing or not updating and catalog issue. I have the basic
code for it figured out but I have created my own dictionary for it called
mydict and put a few key and values in but I want my code to search the csv
file that I am importing and then take the info I am getting from the
search results and put it in its own text file. Also, I dont want to have
to create the mydict line with the keywords I want to be able to type in a
value like Tool and search through the csv file and then output the results
to a text file. So I guess what I want to do is take the row 2 and row 3
from the csv file and output that to text file then from there create a
keyword search and output the results to another text file.

import csv
import json
import sys
from collections import defaultdict
from collections import Counter


class dictionary():
def __init__(self):
self.dict = defaultdict(list)
self.counted_dict = defaultdict(list)
self.grouped_dict = defaultdict(list)
self.other_dict = defaultdict(list)
self.final_dict = defaultdict(list)
self.total_dict = defaultdict(list)
self.search_dict = defaultdict(list)
mydict = defaultdict(list)
def populate_dict(self, filename):
with open (filename, 'rb') as f:
reader = csv.reader(f)
next(reader, None)
for row in reader:
self.dict[row[2]].append(row[3])
def total_counts(self):
for key in self.dict.keys():
total = 0
b = Counter(self.dict[key])
for value in b:
total += b[value]
self.total_dict.update({key: str(total)})
def all_counts(self):
data_count = Counter()
for key in self.dict.keys():
self.counted_dict.update({key: Counter(self.dict[key])})
def grouped_counts(self):
for key in self.dict.keys():
total = 0
c = Counter(self.dict[key])
for value in c:
if c[value] >= 5:
self.grouped_dict.update({value: key + ': ' + str(c[value])})
elif c[value] <= 4:
total += c[value]
self.other_dict.update({key: 'other: ' + str(total)})
self.final_dict = self.grouped_dict, self.other_dict, self.total_dict
#mydict = {'Project Tool Issue': ['CPO Project Tool'], 'All catalogs
missing or not updating': ['20/20 Design Tool']}
def search(mydict, lookup):
for key, value in mydict.iteritems():
for v in value:
if lookup in v:
#return key
print key
search(mydict, 'Tool')
# def txt_output(self, filename):
# a = filename.split('.')
# self.txt_output = a + '.txt'
# print a
def json_output(self):
with open ('scripttesting.txt', 'w') as text_file:
json.dump(self.final_dict, text_file, sort_keys = True, indent = 4)

From alan.gauld at btinternet.com  Tue Jan 26 19:44:56 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 27 Jan 2016 00:44:56 +0000
Subject: [Tutor] Help
In-Reply-To: <CABV0pWxSKNDcaHp+gNTwetg5HOdm+bErs1DM7NgRsd9buWgO1Q@mail.gmail.com>
References: <CABV0pWxSKNDcaHp+gNTwetg5HOdm+bErs1DM7NgRsd9buWgO1Q@mail.gmail.com>
Message-ID: <n893u7$su7$1@ger.gmane.org>

On 26/01/16 20:57, Chelsea G wrote:
> ...I have the basic code for it...

Sorry, when I said paste it in the message I should have
also pointed out that you need to ensure the mail tool
is sending plain text, otherwise we lose all indentation.

But this looks like the same code as before so I'm
guessing you haven't fixed those issues I raised
last time?

Alan G.

> import csv
> import json
> import sys
> from collections import defaultdict
> from collections import Counter
> 
> 
> class dictionary():
> def __init__(self):
> self.dict = defaultdict(list)
> self.counted_dict = defaultdict(list)
> self.grouped_dict = defaultdict(list)
> self.other_dict = defaultdict(list)
> self.final_dict = defaultdict(list)
> self.total_dict = defaultdict(list)
> self.search_dict = defaultdict(list)
> mydict = defaultdict(list)
> def populate_dict(self, filename):
> with open (filename, 'rb') as f:
> reader = csv.reader(f)
> next(reader, None)
> for row in reader:
> self.dict[row[2]].append(row[3])
...

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From robertvstepp at gmail.com  Tue Jan 26 23:03:01 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Tue, 26 Jan 2016 22:03:01 -0600
Subject: [Tutor] Help
In-Reply-To: <CABV0pWxSKNDcaHp+gNTwetg5HOdm+bErs1DM7NgRsd9buWgO1Q@mail.gmail.com>
References: <CABV0pWxSKNDcaHp+gNTwetg5HOdm+bErs1DM7NgRsd9buWgO1Q@mail.gmail.com>
Message-ID: <CANDiX9J3atGMdY00pV4AFkc5+7RR2edYwFnipThqMg-hK-sg8A@mail.gmail.com>

In case you cannot see what we are talking about in how your message
is not keeping its proper indentation, go to the Tutor Mail Archive
at:

https://www.mail-archive.com/tutor at python.org/msg73411.html

and perhaps you can better see the difficulty we are having reading
your message.

boB

From esawiek at gmail.com  Tue Jan 26 21:00:11 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Tue, 26 Jan 2016 21:00:11 -0500
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
Message-ID: <CA+ZkTxuFdi4h3v-A8FY5wawDqSgW-3-vPWwOiYGzLbv1YY-xXg@mail.gmail.com>

Here is the file.

AA,BB,CC,DD,EE

1,A1,B1,11.2,11/20/2011

2,A2,B2,2.5,10/21/2011

3,A3,B3,13.67,9/21/2011

4,A4,B4,14.2,8/22/2011

5,A5,B5,20,7/23/2011

From oscar.j.benjamin at gmail.com  Wed Jan 27 07:26:13 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 27 Jan 2016 12:26:13 +0000
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
In-Reply-To: <CA+ZkTxuFdi4h3v-A8FY5wawDqSgW-3-vPWwOiYGzLbv1YY-xXg@mail.gmail.com>
References: <CA+ZkTxuFdi4h3v-A8FY5wawDqSgW-3-vPWwOiYGzLbv1YY-xXg@mail.gmail.com>
Message-ID: <CAHVvXxQQnUhH2YqpBoVHcqEUry_Vf86KaUeu3urH9b4iN4W06g@mail.gmail.com>

On 27 January 2016 at 02:00, Ek Esawi <esawiek at gmail.com> wrote:
> Here is the file.
>
> AA,BB,CC,DD,EE
>
> 1,A1,B1,11.2,11/20/2011
>
> 2,A2,B2,2.5,10/21/2011
>
> 3,A3,B3,13.67,9/21/2011
>
> 4,A4,B4,14.2,8/22/2011
>
> 5,A5,B5,20,7/23/2011

That still doesn't look right to me. Is there really a blank line
between each line of the .csv file?

--
Oscar

From esawiek at gmail.com  Wed Jan 27 09:23:32 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Wed, 27 Jan 2016 09:23:32 -0500
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
Message-ID: <CA+ZkTxtwjYLyXK3bPF6VxXtmukqQQvqe2kmyrccxfvCGX3bG2w@mail.gmail.com>

Sorry for the mishap. There are no blank lines on the file. I just did not
delete the blank lines when i copied and pasted. Here it's again. EK


*AA,BB,CC,DD,EE*

*1,A1,B1,11.2,11/20/2011*

*2,A2,B2,2.5,10/21/2011*

*3,A3,B3,13.67,9/21/2011*

*4,A4,B4,14.2,8/22/2011*
*5,A5,B5,20,7/23/2011*

From esawiek at gmail.com  Wed Jan 27 18:00:37 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Wed, 27 Jan 2016 18:00:37 -0500
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
Message-ID: <CA+ZkTxswP=_WrgKowo8f2gYvgAhiTNFfw77pDx98_piFtoAWaQ@mail.gmail.com>

Ops..here is the text file.; previously i copied and pasted from either
Word or Excel.

AA,BB,CC,DD,EE
1,A1,B1,11.2,11/20/2011
2,A2,B2,2.5,10/21/2011
3,A3,B3,13.67,9/21/2011
4,A4,B4,14.2,8/22/2011
5,A5,B5,20,7/23/2011

From robertvstepp at gmail.com  Wed Jan 27 23:23:07 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Wed, 27 Jan 2016 22:23:07 -0600
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
Message-ID: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>

I am starting a new project and wish to do it from beginning to end
using TDD best practices.  This is also my practical foray for the
first time into OOP.  The application will have a primitive editor to
create and edit certain text files that the program will need.  The
first file the program will need is one to be called classifiers.txt,
which will be a list of grammar abbreviations that will be used while
classifying sentences.  The beginning of such a file might look like:

N    # Noun
V    # Verb
ADJ  # Adjective

So I started out with the following test_main.py (Later I will factor
out code into separate modules once I know what will make sense.  For
now I am starting out with a single program file, main.py.):

#!/usr/bin/env python3

import unittest

import grammar.main

class TestEditor(unittest.TestCase):
    """
    Test all methods of the Editor class.
    """

    def setUp(self):
        self.editor = grammar.main.Editor()

    def test_edit_classifiers(self):
        # ???


if __name__ == "__main__":
    unittest.main()

And the current code in main.py:

#!/usr/bin/env python3

class Editor(object):
    def __init__(self):
        pass

    def edit_classifiers(self):
        pass

if __name__ == "__main__":
    main()

At this moment I am stumped as to what sort of test to write for the
method edit_classifiers().

At this very beginning stage, what I would want to have happen is to
call edit_classifiers, it will try to open classifiers.txt, realize
that it does not exist yet, and offer to create this file for the user
to start entering the needed grammar classifiers.

I don't want to mess with what will become the program's *real*
classifiers.txt (And other needed text files to come, that will
likewise be editable.), so how do I simulate these various needed file
operations in a way that tests the actual program code, but without
touching the actual data files?

I was looking at "How to Mock Python's `open` function" at
http://www.ichimonji10.name/blog/6/, which looks potentially
promising, but looks complicated.  This leads me to look at
https://docs.python.org/3.4/library/unittest.mock.html, but right now
I am finding this heavy going, and it will probably take me some time
and further research before I feel I am really understand applying
this mock process to file ops.  But is this direction I need to
investigate?

TIA!

-- 
boB

From ben+python at benfinney.id.au  Wed Jan 27 23:57:23 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 28 Jan 2016 15:57:23 +1100
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
Message-ID: <85egd2i9n0.fsf@benfinney.id.au>

boB Stepp <robertvstepp at gmail.com> writes:

> I don't want to mess with what will become the program's *real*
> classifiers.txt (And other needed text files to come, that will
> likewise be editable.), so how do I simulate these various needed file
> operations in a way that tests the actual program code, but without
> touching the actual data files?

This is a common requirement: you want to test using fake filesystem
entries, but *only* for certain filesystem entries. Other filesystem
access should go through to the real filesystem as normal.

> I was looking at "How to Mock Python's `open` function" at
> http://www.ichimonji10.name/blog/6/, which looks potentially
> promising, but looks complicated.

Not only that, but filesystem access is especially problematic for
running unit test suites: the invocation of test cases will itself often
require real access to the filesystem, so fiddling with that inside your
test case will make test failures much more difficult to debug.

Common solutions include ?ehhh, don't bother making test doubles for the
filesystem then?, or ?let the tests access loads of real files and hang
the drain on speed?, or other unsatisfactory compromises.


Would you ? would other readers here ? be sufficiently interested to
work with me on refining a small library I wrote to address this
problem?

I haven't yet created a published project for it, but it would probably
benefit from real-world use in more than the single code base I needed
it for. If I could know there will be several people benefiting and
providing feedback, that would be motivation to work on it further.

-- 
 \      ?An expert is a man who has made all the mistakes which can be |
  `\                         made in a very narrow field.? ?Niels Bohr |
_o__)                                                                  |
Ben Finney


From dyoo at hashcollision.org  Thu Jan 28 00:24:23 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Wed, 27 Jan 2016 21:24:23 -0800
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
Message-ID: <CAGZAPF6WOehHk0DBmnj0xFcTvgZJdbpWZ73sEOcQWyDOd_wz1Q@mail.gmail.com>

> And the current code in main.py:
>
> #!/usr/bin/env python3
>
> class Editor(object):
>     def __init__(self):
>         pass
>
>     def edit_classifiers(self):
>         pass
>
> if __name__ == "__main__":
>     main()
>
> At this moment I am stumped as to what sort of test to write for the
> method edit_classifiers().
>
> At this very beginning stage, what I would want to have happen is to
> call edit_classifiers, it will try to open classifiers.txt, realize
> that it does not exist yet, and offer to create this file for the user
> to start entering the needed grammar classifiers.


You can make the file input/output interface a parameter of your editor.

###############################
class Editor(object):
    def __init__(self, filesystem):
        self.filesystem = filesystem
    ...
################################

Since it's an explicit parameter, we can pass in either something that
does it "for real" by using the built-in input output functions, or we
can "fake it", by providing something that's convenient for unit
tests.


What do we need out of an input/output interface?  Well, maybe a few
basic operations.  Let's say that we need to be able to do two things:


    1. Open files, which returns a "filelike" object.  If it can't
find the file, let's have it raise an IOError.

    2. Create new files, which returns a "filelike" object.

This is admittedly bare-bones, but let's demonstrate what this might
look like.  First, let's see what a real implementation might look
like:

################################
class RealFilesystem(object):
    def __init__(self):
        pass

    def open(self, filename):
        return open(filename, 'r')

    def create(self, filename):
        return open(filename, 'w')
################################

where we're just delegating the methods here to use the built-in
open() function from Python's standard library.

If we need to construct an editor that works with the real file
system, that's not too bad:

    editor = Editor(filesystem=RealFilesystem())


Now what about a test-friendly version of this?  This actually isn't
bad either; we can make judicious use of the StringIO class, which
represents in-memory streams:

###############################################
from StringIO import StringIO

class FakeFilesystem(object):
    """Simulate a very simple filesystem."""
    def __init__(self):
        self.filecontents = {}

    def _open_as_stringio(self, filename):
        filelike = StringIO(self.filecontents[filename])
        real_close = filelike.close
        def wrapped_close():
            self.filecontents[filename] = filelike.getvalue()
            real_close()
        filelike.close = wrapped_close
        return filelike

    def open(self, filename):
        if filename in self.filecontents:
            return self._open_as_stringio(filename)
        else:
            raise IOError, "Not found"

    def create(self, filename):
        self.filecontents[filename] = None
        return self._open_as_stringio(filename)
################################################

(I'm using Python 2.7; if you're on Python 3, substitute the initial
import statement with "from io import StringIO").


This is a class that will look approximately like a filesystem,
because we can "create" and "open" files, and it'll remember.  All of
this is in-memory, taking advantage of the StringIO library.  The
"tricky" part about this is that we need to watch when files close
down, because then we have to record what the file looked like, so
that next time we open the file, we can recall it.


Let's see how this works:

################################
>>> fs = FakeFilesystem()
>>> fs.open('hello')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "fake_filesystem.py", line 21, in open
    raise IOError, "Not found"
IOError: Not found
>>> h = fs.create('hello')
>>> h.write('hello world')
>>> h.close()
>>> h2 = fs.open('hello')
>>> h2.read()
'hello world'
>>> h2.close()
################################

It will remember.  Good.


So for our own unit tests, now we should be able to say something like this:

##################################################
    def test_foobar(self):
        fs = FakeFileSystem()
        fs.filecontents['classifiers.txt'] = """
something here to test what happens when classifiers exists.
"""
        e = Editor(filesystem=fs)
        # ... fill me in!
##################################################


Our test cases can control the environmental conditions.  As long as
the editor uses the file system exclusively through that parameter,
we're now able to freely swap out the real filesystem with the fake
one.

From alan.gauld at btinternet.com  Thu Jan 28 03:44:43 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 28 Jan 2016 08:44:43 +0000
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
Message-ID: <n8ckdr$p08$1@ger.gmane.org>

On 28/01/16 04:23, boB Stepp wrote:

> I don't want to mess with what will become the program's *real*
> classifiers.txt (And other needed text files to come, that will
> likewise be editable.), so how do I simulate these various needed file
> operations in a way that tests the actual program code, but without
> touching the actual data files?

Danny has shown you one way using a mocked filesystem.
But for your case can't you just specify a file location
as an environment variable or argv? That way you get the
advantage of using real files, which can be an important
factor in timing issues, especially if you plan on having
any concurrency going on. And it's simple to do...


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From oscar.j.benjamin at gmail.com  Thu Jan 28 05:28:17 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 28 Jan 2016 10:28:17 +0000
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
In-Reply-To: <CA+ZkTxswP=_WrgKowo8f2gYvgAhiTNFfw77pDx98_piFtoAWaQ@mail.gmail.com>
References: <CA+ZkTxswP=_WrgKowo8f2gYvgAhiTNFfw77pDx98_piFtoAWaQ@mail.gmail.com>
Message-ID: <CAHVvXxSN5DMai=xKYz8201MS_KJ6hwruZfF2riHfUtUqmS72Pw@mail.gmail.com>

On 27 January 2016 at 23:00, Ek Esawi <esawiek at gmail.com> wrote:
> Ops..here is the text file.; previously i copied and pasted from either
> Word or Excel.
>
>
> AA,BB,CC,DD,EE
> 1,A1,B1,11.2,11/20/2011
> 2,A2,B2,2.5,10/21/2011
> 3,A3,B3,13.67,9/21/2011
> 4,A4,B4,14.2,8/22/2011
> 5,A5,B5,20,7/23/2011

Finally! That's what I expect to see in a csv file. Do the first,
second and third columns just count up like that? I thought that you
were expecting some rows to have the same values in the second and
third columns.

I'm not sure that it's really worth using numpy for this. I'd just use
the csv module:

import csv

with open('test.csv') as csvfile:
    reader = csv.reader(csvfile)
    next(reader, None) # Ignore first line of file
    for line in reader:
        index, col2, col3, col4, date = line
        col4 = float(col4)  # Convert 4th column value to float
        print([col2, col3, col4])

Running this I get:

$ python3 test.py
['A1', 'B1', 11.2]
['A2', 'B2', 2.5]
['A3', 'B3', 13.67]
['A4', 'B4', 14.2]
['A5', 'B5', 20.0]

--
Oscar

From dyoo at hashcollision.org  Thu Jan 28 15:03:46 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Thu, 28 Jan 2016 12:03:46 -0800
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <n8ckdr$p08$1@ger.gmane.org>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
Message-ID: <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>

On Thu, Jan 28, 2016 at 12:44 AM, Alan Gauld <alan.gauld at btinternet.com> wrote:
> On 28/01/16 04:23, boB Stepp wrote:
>
>> I don't want to mess with what will become the program's *real*
>> classifiers.txt (And other needed text files to come, that will
>> likewise be editable.), so how do I simulate these various needed file
>> operations in a way that tests the actual program code, but without
>> touching the actual data files?
>
> Danny has shown you one way using a mocked filesystem.
> But for your case can't you just specify a file location
> as an environment variable or argv? That way you get the
> advantage of using real files, which can be an important
> factor in timing issues, especially if you plan on having
> any concurrency going on. And it's simple to do...


Just to emphasize what I think is an essential point: the basic
approach we're doing is here parameterization: to take something that
used to be hardcoded, and turn it into a parameter that allows us to
substitute with something else.

As Alan says, you can also parameterize in a different way: by the
directory location where files are being read.  Then you can use a
temporary directory for your unit tests, and prepare the testing
environment that way.  If you take this approach, the tempfile module
can help with this.

    https://docs.python.org/3.5/library/tempfile.html
    https://docs.python.org/3.5/library/tempfile.html#tempfile.TemporaryDirectory


The mocking approach is one where we're doing this parameterization at
a behavioral level.  When we started to program, we may have initially
thought that a parameter could only be numbers, since that's what
algebra traditionally uses as its domain.  When we program, we find
that domain of values expanded to a richer set, and not just to
inactive values like strings or dictionaries or images, but now we can
pass entire collections of behavior as a parameter.  That's one of the
lessons of OOP: values are not just inert data: they can define
dynamic behavior.

(Aside: this is somewhat why I think the topic of inheritance and
inheritance hierarchies are entirely the wrong things to focus on when
we're teaching OOP.  Those topics are often used as a shortcut
mechanism to do code sharing, and although that's convenient, my
opinion is that it misses the forest for the trees.  What I think
OOP's heart is beating is in the ability to parameterize behavior.)

From sjeik_appie at hotmail.com  Thu Jan 28 15:23:37 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Thu, 28 Jan 2016 20:23:37 +0000
Subject: [Tutor] lc_ctype and re.LOCALE
Message-ID: <DUB123-W20902A91F7061922CE53BD83DA0@phx.gbl>

Hi,

Out of curiosity, I wrote the throw-away script below to find a character that is classified (--> LC_CTYPE) as digit in one locale, but not in another.
I ran it with 5000 locale combinations in Python 2 but did not find any (somebody shut down my computer!). I just modified the code so it also 
runs in Python 3. Is this the correct way to find such locale-dependent regex matches?


albertjan at debian:~/Downloads$ uname -a && python --version && python3 --version
Linux debian 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u3 (2015-08-04) x86_64 GNU/Linux
Python 2.7.9
Python 3.3.4
albertjan at debian:~/Downloads$ cat lc_ctype.py 
# -*- coding: utf-8 -*-
"""
Find two locales where a different character classification causes a regex 
to match a given character in one locale, but fail in another.
This is to demonstrate the effect that re.LOCALE (in particular the LC_CTYPE
locale category) might have on locale-aware regexes like \w or \d. 
E.g., a character might be classified as digit in one locale but not in another.
"""

from __future__ import print_function, division
import subprocess
import locale
import itertools
import sys
import re

try:
??? xrange
except NameError:
??? xrange = range
??? unichr = chr
if sys.version_info.major> 2:
??? unicode = str

proc = subprocess.Popen("locale -a", stdout=subprocess.PIPE, shell=True)
locales = proc.communicate()
locales = sorted(locales[0].split(b"\n"))? # this is the list: http://pastebin.com/FVxUnrWK
if sys.version_info.major> 2:
??? locales = [loc.decode("utf-8") for loc in locales]
regex = re.compile(r"\d+", re.LOCALE)? # is this the correct place?

total = len(list(itertools.combinations(locales, 2)))

for n, (locale1, locale2) in enumerate(itertools.combinations(locales, 2), 1):

??? if not locale1 or not locale2:
??????? continue
??? 
??? if n % 10 == 0 or n == 1:
??????? sys.stdout.write(" %d (%3.2f%%) ... "? % (n, (n / total * 100) ))
??????? sys.stdout.flush()? # python 2 print *function* does not have flush param

??? for i in xrange(sys.maxunicode + 1):?? # 1114111
??????? s = unichr(i)? #.encode("utf8")
??????? try:
??????????? locale.setlocale(locale.LC_CTYPE, locale1)
??????????? m1 = bool(regex.match(s))
??????????? locale.setlocale(locale.LC_CTYPE, locale2)
??????????? m2 = bool(regex.match(s))
??????????? if m1 ^ m2:? # m1 != m2
??????????????? msg = ("@@ ordinal: %s | character: %s (%r) | " 
?????????????????????? " digit in locale '%s': %s | digit in locale '%s': %s ")
??????????????? print(msg % (i, unichr(i), unichr(i), locale1, m1, locale2, m2))
??????????????? break
??????? except locale.Error as e:
??????????? #print("Error: %s with %s and/or %s" % (e, locale1, locale2))
??????????? continue
??????? 
print("---Done---")


Thank you!

Albert-Jan
 		 	   		  

From sjeik_appie at hotmail.com  Thu Jan 28 15:31:13 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Thu, 28 Jan 2016 20:31:13 +0000
Subject: [Tutor] Why is an OrderedDict not sliceable?
In-Reply-To: <20160126152435.GT4619@ando.pearwood.info>
References: <n7rh0d$6tn$1@ger.gmane.org>,
 <DUB123-W36914C93BB77114F32CB1983C60@phx.gbl>,
 <20160126152435.GT4619@ando.pearwood.info>
Message-ID: <DUB123-W268C034B81989E051FD02283DA0@phx.gbl>



----------------------------------------
> Date: Wed, 27 Jan 2016 02:24:35 +1100
> From: steve at pearwood.info
> To: tutor at python.org
> Subject: Re: [Tutor] Why is an OrderedDict not sliceable?
>
> On Sun, Jan 24, 2016 at 07:47:47PM +0000, Albert-Jan Roskam wrote:
>
>>> You appear to be confusing ordered and sorted.
>>
>> You are correct. Is there a difference in the way those terms are
>> used colloquially vs. in the field of Computer Science (Note: English
>> is not my mother tongue)?
>
> In ordinary English, "ordered" and "sorted" often are used to mean the
> same thing. People do often use sorted and ordered as interchangeable,
> but the definitions are slightly different:
>
>
>
> ordered \ordered\ adj.
> 1. having or evincing a systematic arrangement; especially,
> having elements succeeding in order according to rule; as,
> an ordered sequence; an ordered pair. Opposite of
> disordered or unordered. [Narrower terms:
> abecedarian, alphabetical; {consecutive, sequent,
> sequential, serial, successive ]
> [WordNet 1.5 +PJC]
>
> 2. arranged in order.
>
> Sort \Sort\, v. t. [imp. & p. p. Sorted; p. pr. & vb. n.
> Sorting.]
> 1. To separate, and place in distinct classes or divisions,
> as things having different qualities; as, to sort cloths
> according to their colors; to sort wool or thread
> according to its fineness.
> [1913 Webster]
>
>
> The way I would put it is that "sorted" means the items are ordered
> according to some specific rule or property of the items themselves,
> e.g. to sort your clothes by colour. "Ordered" is more general: it just
> means to have some order, which may be according to a rule or property,
> or it may be in whatever sequence the items happen to have.
>
> Books on a shelf have some order, the order that they appear when you
> read them from left to right, regardless of whether they are sorted by
> author, title, height, colour or at random. Books jumbled up in a bag
> have no order.
>
> Ordinary dicts are like books in a bag. You reach in and grab whatever
> book happens to come to hand first. OrderedDicts are like books on a
> shelf: you can systematically touch each book in order starting from the
> left, and new books are always added to the right.

Thank you! That distinction is indeed quite subtle. With "ordered" I tend to think of "ordinal measurement level" or something like that. E.g., sort a list of heights, calculate the median, ntiles etc. But your description makes it a whole lot clearer. Sometimes analogies work better!

 		 	   		  

From matt.williams45.mw at gmail.com  Thu Jan 28 15:34:55 2016
From: matt.williams45.mw at gmail.com (Matt Williams)
Date: Thu, 28 Jan 2016 20:34:55 +0000
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
Message-ID: <CAFTVGQauXzrpPyB6zZdf5f6bGTpwO5X1CM0Wmp8dYH90EuOurg@mail.gmail.com>

This is a problem I have come up against often, and I don't think I have a
good answer, so if anyone else does, I would be glad to hear it!

I would be tempted to generate a 'test data.CSV" file, and run the tests on
that. It means that as you write the code, and find some edge cases, you
can alter your code and add the edge cases to the test data. That way, the
test data acts to cover the space of various oddities in your work.

I would be very keen to hear other ideas,

BW,
Matt

On Thu, 28 Jan 2016 20:12 Danny Yoo <dyoo at hashcollision.org> wrote:

> On Thu, Jan 28, 2016 at 12:44 AM, Alan Gauld <alan.gauld at btinternet.com>
> wrote:
> > On 28/01/16 04:23, boB Stepp wrote:
> >
> >> I don't want to mess with what will become the program's *real*
> >> classifiers.txt (And other needed text files to come, that will
> >> likewise be editable.), so how do I simulate these various needed file
> >> operations in a way that tests the actual program code, but without
> >> touching the actual data files?
> >
> > Danny has shown you one way using a mocked filesystem.
> > But for your case can't you just specify a file location
> > as an environment variable or argv? That way you get the
> > advantage of using real files, which can be an important
> > factor in timing issues, especially if you plan on having
> > any concurrency going on. And it's simple to do...
>
>
> Just to emphasize what I think is an essential point: the basic
> approach we're doing is here parameterization: to take something that
> used to be hardcoded, and turn it into a parameter that allows us to
> substitute with something else.
>
> As Alan says, you can also parameterize in a different way: by the
> directory location where files are being read.  Then you can use a
> temporary directory for your unit tests, and prepare the testing
> environment that way.  If you take this approach, the tempfile module
> can help with this.
>
>     https://docs.python.org/3.5/library/tempfile.html
>
> https://docs.python.org/3.5/library/tempfile.html#tempfile.TemporaryDirectory
>
>
> The mocking approach is one where we're doing this parameterization at
> a behavioral level.  When we started to program, we may have initially
> thought that a parameter could only be numbers, since that's what
> algebra traditionally uses as its domain.  When we program, we find
> that domain of values expanded to a richer set, and not just to
> inactive values like strings or dictionaries or images, but now we can
> pass entire collections of behavior as a parameter.  That's one of the
> lessons of OOP: values are not just inert data: they can define
> dynamic behavior.
>
> (Aside: this is somewhat why I think the topic of inheritance and
> inheritance hierarchies are entirely the wrong things to focus on when
> we're teaching OOP.  Those topics are often used as a shortcut
> mechanism to do code sharing, and although that's convenient, my
> opinion is that it misses the forest for the trees.  What I think
> OOP's heart is beating is in the ability to parameterize behavior.)
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From esawiek at gmail.com  Thu Jan 28 08:47:20 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Thu, 28 Jan 2016 08:47:20 -0500
Subject: [Tutor] Change datatype for specific columns in an 2D array &
 computing the mean
Message-ID: <CA+ZkTxsZECbAHz+SNKWsYvky61ir+tPzGp1Qqjd2_=mi6YThLA@mail.gmail.com>

Thank you SO MUCH Oscar and others. This is exactly what i wanted to get. I
think i can take it form here. I spent hours looking at different ways to
figure it out and i came close. I tried numpy.genfromtxt, numpy.loadtext,
json, etc.

I am at the moment learning how to open various data files with different
format such as csv, pdf, html, etc. and then convert them to arrays or
list.

Thanks again for your patience and help

EK

From alan.gauld at btinternet.com  Thu Jan 28 17:14:10 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 28 Jan 2016 22:14:10 +0000
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
Message-ID: <56AA92B2.3000609@btinternet.com>

On 28/01/16 20:03, Danny Yoo wrote:
> (Aside: this is somewhat why I think the topic of inheritance and
> inheritance
> hierarchies are entirely the wrong things to focus on when we're
> teaching OOP. 

Absolutely.
I always try to focus on the objects passing, and responding to, *messages*
as the key feature. Inheritance is a convenience, often abused to do code
reuse. The thing to focus on is polymorphism which just happens to be
implemented by inheritance heirarchies in most languages (although
not in python!)

One of the things I like about Objective C is the fact that it has a
separate
notation for sending messages to objects compared to calling a function(*).
It really emphasises the different concepts at work - sending a message
is not the same as simply calling a function, there is a layer of magic in
between!

>  What I think OOP's heart is beating is in the ability to parameterize
behavior.)

That's a by-product of encapsulation and polymorphism which are the
beating heart of pure OOP (sorry Steve ;-).

(*) In ObjectiveC:
result = [anObject  aMessage:aParameter];  // send a message to an object

result2 = aFunction(anObject, aParameter);  // call a function

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


From martin at linux-ip.net  Thu Jan 28 18:09:55 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Thu, 28 Jan 2016 15:09:55 -0800
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <CAFTVGQauXzrpPyB6zZdf5f6bGTpwO5X1CM0Wmp8dYH90EuOurg@mail.gmail.com>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
 <CAFTVGQauXzrpPyB6zZdf5f6bGTpwO5X1CM0Wmp8dYH90EuOurg@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1601281458350.2025@znpeba.jbaqresebt.arg>


Hello all,

[much snipped]

boB>> >> I don't want to mess with what will become the program's *real*
boB>> >> classifiers.txt (And other needed text files to come, that will
boB>> >> likewise be editable.), so how do I simulate these various needed file
boB>> >> operations in a way that tests the actual program code, but without
boB>> >> touching the actual data files?

Danny> As Alan says, you can also parameterize in a different way: by the
Danny> directory location where files are being read.  Then you can use a
Danny> temporary directory for your unit tests, and prepare the testing
Danny> environment that way.  If you take this approach, the tempfile module
Danny> can help with this.
Danny>
Danny>     https://docs.python.org/3.5/library/tempfile.html
Danny>     https://docs.python.org/3.5/library/tempfile.html#tempfile.TemporaryDirectory
Danny>

Alan>> > Danny has shown you one way using a mocked filesystem.
Alan>> > But for your case can't you just specify a file location
Alan>> > as an environment variable or argv? That way you get the
Alan>> > advantage of using real files, which can be an important
Alan>> > factor in timing issues, especially if you plan on having
Alan>> > any concurrency going on. And it's simple to do...

Matt>I would be tempted to generate a 'test data.CSV" file, and run the 
Matt>tests on that. It means that as you write the code, and find some 
Matt>edge cases, you can alter your code and add the edge cases to the 
Matt>test data. That way, the test data acts to cover the space of 
Matt>various oddities in your work.

I'll add one option to the mix, and summarize the other options I saw listed
earlier.

Option A (tempfile):

  Create a directory and copy your pristine tree into the directory
  and make the base directory for the important data files configurable.
  Also known as parameterization.

Option B (starting with Danny's sample code)

  Create the objects to emulate whatever filesystem behaviour you need.

Option C (use pyfakefs):

  Use pyfakefs, which acts like a filesystem for testing purposes.
  https://pypi.python.org/pypi/pyfakefs
  https://github.com/jmcgeheeiv/pyfakefs

Option D (use StringIO):

  If you are less concerned about filesystem interactions and really 
  just want an individual thingy that that behaves like a file, but 
  can be constructed in memory, use StringIO (or cStringIO).

Good luck!

-Martin

-- 
Martin A. Brown
http://linux-ip.net/

From robertvstepp at gmail.com  Thu Jan 28 23:20:56 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Thu, 28 Jan 2016 22:20:56 -0600
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <CAGZAPF6WOehHk0DBmnj0xFcTvgZJdbpWZ73sEOcQWyDOd_wz1Q@mail.gmail.com>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <CAGZAPF6WOehHk0DBmnj0xFcTvgZJdbpWZ73sEOcQWyDOd_wz1Q@mail.gmail.com>
Message-ID: <CANDiX9Kqns0K4kaBrKHv9cd5QqKN5yg4Nfy4QA1d3mMQzvsA8A@mail.gmail.com>

On Wed, Jan 27, 2016 at 11:24 PM, Danny Yoo <dyoo at hashcollision.org> wrote:

> You can make the file input/output interface a parameter of your editor.
>
> ###############################
> class Editor(object):
>     def __init__(self, filesystem):
>         self.filesystem = filesystem
>     ...
> ################################
>
> Since it's an explicit parameter, we can pass in either something that
> does it "for real" by using the built-in input output functions, or we
> can "fake it", by providing something that's convenient for unit
> tests.
>
>
> What do we need out of an input/output interface?  Well, maybe a few
> basic operations.  Let's say that we need to be able to do two things:
>
>
>     1. Open files, which returns a "filelike" object.  If it can't
> find the file, let's have it raise an IOError.
>
>     2. Create new files, which returns a "filelike" object.
>
> This is admittedly bare-bones, but let's demonstrate what this might
> look like.  First, let's see what a real implementation might look
> like:
>
> ################################
> class RealFilesystem(object):
>     def __init__(self):
>         pass
>
>     def open(self, filename):
>         return open(filename, 'r')
>
>     def create(self, filename):
>         return open(filename, 'w')
> ################################
>
> where we're just delegating the methods here to use the built-in
> open() function from Python's standard library.
>
> If we need to construct an editor that works with the real file
> system, that's not too bad:
>
>     editor = Editor(filesystem=RealFilesystem())

I was already planning on designing a class to handle my program's
file I/O.  I will probably need to add an append method, too.  Thanks
for giving my a sound starting point!


> Now what about a test-friendly version of this?  This actually isn't
> bad either; we can make judicious use of the StringIO class, which
> represents in-memory streams:
>
> ###############################################
> from StringIO import StringIO
>
> class FakeFilesystem(object):
>     """Simulate a very simple filesystem."""
>     def __init__(self):
>         self.filecontents = {}
>
>     def _open_as_stringio(self, filename):
>         filelike = StringIO(self.filecontents[filename])
>         real_close = filelike.close
>         def wrapped_close():
>             self.filecontents[filename] = filelike.getvalue()
>             real_close()
>         filelike.close = wrapped_close
>         return filelike
>
>     def open(self, filename):
>         if filename in self.filecontents:
>             return self._open_as_stringio(filename)
>         else:
>             raise IOError, "Not found"
>
>     def create(self, filename):
>         self.filecontents[filename] = None
>         return self._open_as_stringio(filename)
> ################################################
>
> (I'm using Python 2.7; if you're on Python 3, substitute the initial
> import statement with "from io import StringIO").

I was just scanning the docs on io.  A note relevant to IOError:

"Changed in version 3.3: Operations that used to raise IOError now
raise OSError, since IOError is now an alias of OSError."


> This is a class that will look approximately like a filesystem,
> because we can "create" and "open" files, and it'll remember.  All of
> this is in-memory, taking advantage of the StringIO library.  The
> "tricky" part about this is that we need to watch when files close
> down, because then we have to record what the file looked like, so
> that next time we open the file, we can recall it.
>
>
> Let's see how this works:
>
> ################################
>>>> fs = FakeFilesystem()
>>>> fs.open('hello')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "fake_filesystem.py", line 21, in open
>     raise IOError, "Not found"
> IOError: Not found
>>>> h = fs.create('hello')
>>>> h.write('hello world')
>>>> h.close()
>>>> h2 = fs.open('hello')
>>>> h2.read()
> 'hello world'
>>>> h2.close()
> ################################

I will need to read the io docs in detail, but I am wondering if the
"with open ..." context manager is still usable to handle simulated
file closing using this technique?

> So for our own unit tests, now we should be able to say something like this:
>
> ##################################################
>     def test_foobar(self):
>         fs = FakeFileSystem()
>         fs.filecontents['classifiers.txt'] = """
> something here to test what happens when classifiers exists.
> """
>         e = Editor(filesystem=fs)
>         # ... fill me in!
> ##################################################

You have given me a substantial hunk of meat to chew on, Danny!  Thank
you very much for your lucid explanation and examples!!  You have
given me a very solid starting point if I choose this route.  I may
very well need to experiment with all of the approaches mentioned in
this thread.  Much to learn!

-- 
boB

From robertvstepp at gmail.com  Thu Jan 28 23:32:05 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Thu, 28 Jan 2016 22:32:05 -0600
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <n8ckdr$p08$1@ger.gmane.org>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
Message-ID: <CANDiX9+YC7YO7TJjgxai_SN==MjoP8sLXsv98tuV8h+3VPn08w@mail.gmail.com>

On Thu, Jan 28, 2016 at 2:44 AM, Alan Gauld <alan.gauld at btinternet.com> wrote:

> Danny has shown you one way using a mocked filesystem.
> But for your case can't you just specify a file location
> as an environment variable or argv? That way you get the
> advantage of using real files, which can be an important
> factor in timing issues, especially if you plan on having
> any concurrency going on. And it's simple to do...

This type of approach was actually what first popped into my head.  It
is partially why I set up my project structure as grammar/grammar and
grammar/test, where my actual program code would be under grammar.  I
thought this way I could have an identically structured data/ folder
under both grammar/ and test/.  But it was not coming to me how to set
up class Editor(), so that it would seamlessly access the *correct*
filesystem, depending on whether I was running program code or test
code.  But amongst all the things Danny clarified, he really helped me
see how to do this in the context of writing classes.

But in choosing what approach to use, I had not considered the
possible importance of using real files in some instances, which you
allude to.

Thanks!

boB

From robertvstepp at gmail.com  Thu Jan 28 23:54:51 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Thu, 28 Jan 2016 22:54:51 -0600
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
Message-ID: <CANDiX9+GwJvzwmK6Wsd8SMf=f_xkbGNGNJNO-cHUA8YeDqeygQ@mail.gmail.com>

On Thu, Jan 28, 2016 at 2:03 PM, Danny Yoo <dyoo at hashcollision.org> wrote:

> Just to emphasize what I think is an essential point: the basic
> approach we're doing is here parameterization: to take something that
> used to be hardcoded, and turn it into a parameter that allows us to
> substitute with something else.

Thank you for the correct term for the idea I have been gradually
trying to understand better and implement.  Work has been slow the
past couple of weeks, so I have been trying to rework the last large
project I wrote within the past year to make it capable of evaluating
a broader range of treatment plans.  Even though I wrote this less
than a year ago, I am now seeing so many ways I can improve the
readability, compactness and efficiency with what I have learned
since.  Even though this was a strictly procedural program, I have
found that many of my functions can be rewritten to handle more than
what I originally intended.  This parameterization you speak of has
allowed me to eliminate several similar functions by generalizing
certain things that behave similarly.  And I am sure that I am missing
other opportunities for doing this!

> ...When we program, we find
> that domain of values expanded to a richer set, and not just to
> inactive values like strings or dictionaries or images, but now we can
> pass entire collections of behavior as a parameter.  That's one of the
> lessons of OOP: values are not just inert data: they can define
> dynamic behavior.

I am only now beginning to speculate on these new possibilities that
this paradigm may enable.  How *mere* bits can be assembled into ever
more complex, but coherent structures that can do so much continues to
astound me.  Old hat for you experts, but exciting new ground for me!

> (Aside: this is somewhat why I think the topic of inheritance and
> inheritance hierarchies are entirely the wrong things to focus on when
> we're teaching OOP.  Those topics are often used as a shortcut
> mechanism to do code sharing, and although that's convenient, my
> opinion is that it misses the forest for the trees.  What I think
> OOP's heart is beating is in the ability to parameterize behavior.)

In response to your aside, do you have a favorite text that teaches
OOP the way you feel it should be taught?  And is there a Python-based
version?


-- 
boB

From robertvstepp at gmail.com  Thu Jan 28 23:58:44 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Thu, 28 Jan 2016 22:58:44 -0600
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <56AA92B2.3000609@btinternet.com>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
 <56AA92B2.3000609@btinternet.com>
Message-ID: <CANDiX9LT5RNAQKj2A9Qs2SH_aXzSCnsZuYLVsK69siakGrCPwg@mail.gmail.com>

On Thu, Jan 28, 2016 at 4:14 PM, Alan Gauld <alan.gauld at btinternet.com> wrote:
> On 28/01/16 20:03, Danny Yoo wrote:
>> (Aside: this is somewhat why I think the topic of inheritance and
>> inheritance
>> hierarchies are entirely the wrong things to focus on when we're
>> teaching OOP.
>
> Absolutely.
> I always try to focus on the objects passing, and responding to, *messages*
> as the key feature. Inheritance is a convenience, often abused to do code
> reuse. The thing to focus on is polymorphism which just happens to be
> implemented by inheritance heirarchies in most languages (although
> not in python!)

I'll ask you the same question that I asked Danny:  Do you have a
favorite text which teaches OOP the way you feel it should be taught?
And if possible, Python-based?

boB

From robertvstepp at gmail.com  Fri Jan 29 00:03:27 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Thu, 28 Jan 2016 23:03:27 -0600
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <alpine.LSU.2.11.1601281458350.2025@znpeba.jbaqresebt.arg>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
 <CAFTVGQauXzrpPyB6zZdf5f6bGTpwO5X1CM0Wmp8dYH90EuOurg@mail.gmail.com>
 <alpine.LSU.2.11.1601281458350.2025@znpeba.jbaqresebt.arg>
Message-ID: <CANDiX9+LEv_K-0qyr87Rhi8rfQ19xv_556G5M8uoz3CwjovG8w@mail.gmail.com>

On Thu, Jan 28, 2016 at 5:09 PM, Martin A. Brown <martin at linux-ip.net> wrote:

> I'll add one option to the mix, and summarize the other options I saw listed
> earlier.
>
> Option A (tempfile):
>
>   Create a directory and copy your pristine tree into the directory
>   and make the base directory for the important data files configurable.
>   Also known as parameterization.
>
> Option B (starting with Danny's sample code)
>
>   Create the objects to emulate whatever filesystem behaviour you need.
>
> Option C (use pyfakefs):
>
>   Use pyfakefs, which acts like a filesystem for testing purposes.
>   https://pypi.python.org/pypi/pyfakefs
>   https://github.com/jmcgeheeiv/pyfakefs
>
> Option D (use StringIO):
>
>   If you are less concerned about filesystem interactions and really
>   just want an individual thingy that that behaves like a file, but
>   can be constructed in memory, use StringIO (or cStringIO).

Isn't option D what Danny was using to make option B?  Or are you
saying keep things even simpler?

boB

From martin at linux-ip.net  Fri Jan 29 00:45:51 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Thu, 28 Jan 2016 21:45:51 -0800
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <CANDiX9+LEv_K-0qyr87Rhi8rfQ19xv_556G5M8uoz3CwjovG8w@mail.gmail.com>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
 <CAFTVGQauXzrpPyB6zZdf5f6bGTpwO5X1CM0Wmp8dYH90EuOurg@mail.gmail.com>
 <alpine.LSU.2.11.1601281458350.2025@znpeba.jbaqresebt.arg>
 <CANDiX9+LEv_K-0qyr87Rhi8rfQ19xv_556G5M8uoz3CwjovG8w@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1601282144400.2025@znpeba.jbaqresebt.arg>


Hello there,

>> I'll add one option to the mix, and summarize the other options I saw listed
>> earlier.
>>
>> Option A (tempfile):
>>
>>   Create a directory and copy your pristine tree into the directory
>>   and make the base directory for the important data files configurable.
>>   Also known as parameterization.
>>
>> Option B (starting with Danny's sample code)
>>
>>   Create the objects to emulate whatever filesystem behaviour you need.
>>
>> Option C (use pyfakefs):
>>
>>   Use pyfakefs, which acts like a filesystem for testing purposes.
>>   https://pypi.python.org/pypi/pyfakefs
>>   https://github.com/jmcgeheeiv/pyfakefs
>>
>> Option D (use StringIO):
>>
>>   If you are less concerned about filesystem interactions and really
>>   just want an individual thingy that that behaves like a file, but
>>   can be constructed in memory, use StringIO (or cStringIO).
>
>Isn't option D what Danny was using to make option B?  Or are you
>saying keep things even simpler?

Oh dear--yes.  Apologies, Danny and boB.

To the doghouse with me.

-Martin

-- 
Martin A. Brown
http://linux-ip.net/

From ben+python at benfinney.id.au  Fri Jan 29 02:09:58 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 29 Jan 2016 18:09:58 +1100
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
 <CAFTVGQauXzrpPyB6zZdf5f6bGTpwO5X1CM0Wmp8dYH90EuOurg@mail.gmail.com>
 <alpine.LSU.2.11.1601281458350.2025@znpeba.jbaqresebt.arg>
Message-ID: <85r3h0hneh.fsf@benfinney.id.au>

"Martin A. Brown" <martin at linux-ip.net> writes:

> Option A (tempfile):
>
>   Create a directory and copy your pristine tree into the directory
>   and make the base directory for the important data files
>   configurable. Also known as parameterization.

I consider that a poor option; real files complicate the test setup and
cleanup, they are always going to be slower, and they can lead to false
positives and false negatives in the test results.

All of those will act as a disincentive to add lots of unit tests for
filesystem-related areas of your code; and I think that's a disincentive
we don't need.

> Option D (use StringIO):
>
>   If you are less concerned about filesystem interactions and really 
>   just want an individual thingy that that behaves like a file, but 
>   can be constructed in memory, use StringIO (or cStringIO).

These are great if you can get away with them. But many programs use not
just file objects, but also interrogate the filesystem: they use
`os.stat`, `os.path.exists`, builtin `open`, etc.

We need to be able to control not only the in-memory file objects, but
the filesystem access for our fake files that don't actually exist on
the real filesystem.

> Option C (use pyfakefs):
>
>   Use pyfakefs, which acts like a filesystem for testing purposes.
>   https://pypi.python.org/pypi/pyfakefs
>   https://github.com/jmcgeheeiv/pyfakefs

I have today published a different approach, which I designed for one
code base ad is to date its only user. I'm looking for more feedback.

    <URL:https://pypi.python.org/pypi/gajja/>

See the ?doc/tutorial.txt? for a worked example of how it can be useful.
(and I'm still writing more documentation for general use.)

-- 
 \           ?I do not believe in immortality of the individual, and I |
  `\        consider ethics to be an exclusively human concern with no |
_o__)  superhuman authority behind it.? ?Albert Einstein, letter, 1953 |
Ben Finney


From ben+python at benfinney.id.au  Fri Jan 29 02:19:44 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 29 Jan 2016 18:19:44 +1100
Subject: [Tutor] [ANN] Gajja 0.1 (was: How do I test file operations (Such
 as opening, reading, writing, etc.)?)
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
 <CAFTVGQauXzrpPyB6zZdf5f6bGTpwO5X1CM0Wmp8dYH90EuOurg@mail.gmail.com>
 <alpine.LSU.2.11.1601281458350.2025@znpeba.jbaqresebt.arg>
 <85r3h0hneh.fsf@benfinney.id.au>
Message-ID: <85fuxghmy7.fsf_-_@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> I have today published a different approach, which I designed for one
> code base ad is to date its only user. I'm looking for more feedback.
>
>     <URL:https://pypi.python.org/pypi/gajja/>

I forgot to mention the appropriate channels for feedback:

Please contact me at <ben+python at benfinney.id.au> if you have feedback
on this, or report an issue at the project's homepage
<URL:https://notabug.org/bignose/python-gajja>.

You can also discuss this on the thread I've opened at the main Python
forum <URL:https://mail.python.org/mailman/listinfo/python-list>.

-- 
 \          ?That's all very good in practice, but how does it work in |
  `\                                             *theory*?? ?anonymous |
_o__)                                                                  |
Ben Finney


From alan.gauld at btinternet.com  Fri Jan 29 04:44:19 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 29 Jan 2016 09:44:19 +0000
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <CANDiX9LT5RNAQKj2A9Qs2SH_aXzSCnsZuYLVsK69siakGrCPwg@mail.gmail.com>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
 <56AA92B2.3000609@btinternet.com>
 <CANDiX9LT5RNAQKj2A9Qs2SH_aXzSCnsZuYLVsK69siakGrCPwg@mail.gmail.com>
Message-ID: <n8fc9j$dks$1@ger.gmane.org>

On 29/01/16 04:58, boB Stepp wrote:

> I'll ask you the same question that I asked Danny:  Do you have a
> favorite text which teaches OOP the way you feel it should be taught?
> And if possible, Python-based?

I have three; depending on your level of interest :-)

1) OO Design by Grady Booch.
But it has to be the first edition - try your library.
He teaches it by way of projects, each in a different
language, showing how to adapt the pure OOP ideas to
the target language.(Smalltalk, Object Pascal, C++,
ADA and Lisp) (He also teaches his design notation
which you can almost ignore because UML has replaced it.)

2) OOA by Coad & Yourdon. A lightweight( intro to OO
concepts without much code at all. Purely at the
analysis level. They also did an OOD book which does
feature some code and focuses on how to partition
an app but the OOA book is better.

3) OO Software Construction by B Meyer (2nd edition
this time) The daddy of OOP books. Its massive
(1200+pages), very detailed and at quite an academic
level but covers every facet of OOP you can imagine.
But its in Eiffel - Meyers own language which is
possibly the best (as in clean and complete) software
engineering language ever invented but in practice
hardly ever used.

I'd start with Coad. But borrow from a library rather
than buy (unless you can get it for pennies on Amazon
marketplace etc). But Booch is probably better overall
for practical application of the ideas and discussing
how OOP applies at the code level. Myer is for the
pure theory side of things.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Fri Jan 29 07:35:56 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 29 Jan 2016 12:35:56 +0000
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
In-Reply-To: <85r3h0hneh.fsf@benfinney.id.au>
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
 <CAFTVGQauXzrpPyB6zZdf5f6bGTpwO5X1CM0Wmp8dYH90EuOurg@mail.gmail.com>
 <alpine.LSU.2.11.1601281458350.2025@znpeba.jbaqresebt.arg>
 <85r3h0hneh.fsf@benfinney.id.au>
Message-ID: <n8fmbc$9sp$1@ger.gmane.org>

On 29/01/16 07:09, Ben Finney wrote:

>>   Create a directory and copy your pristine tree into the directory
>>   and make the base directory for the important data files
>>   configurable. Also known as parameterization.
> 
> I consider that a poor option; real files complicate the test setup and
> cleanup, they are always going to be slower, and they can lead to false
> positives and false negatives in the test results.

There are pros and cons to all the options.

File simulation is faster and fine for unit testing functionality,
but not so good for testing the overall system. Simulated files
can lead to false positives precisely because they are faster
than real files. Recognising slow access times - especially
on big data sets - are a vital part of testing any application.

In an ideal world parameterize so that you use file simulation
for basic functional unit testing and then use a dummy real file
system for integration, performance and system testing (and
probably user testing too). Finally, acceptance testing will
likely have to run on the real (actual) filesystem.

Having configurable test environments is pretty much essential.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From eryksun at gmail.com  Fri Jan 29 08:14:07 2016
From: eryksun at gmail.com (eryk sun)
Date: Fri, 29 Jan 2016 07:14:07 -0600
Subject: [Tutor] lc_ctype and re.LOCALE
In-Reply-To: <DUB123-W20902A91F7061922CE53BD83DA0@phx.gbl>
References: <DUB123-W20902A91F7061922CE53BD83DA0@phx.gbl>
Message-ID: <CACL+1atbdevOg7_hsu9mh0u_ZfQ_A-U3GcLUbFZea2_nCsm=mQ@mail.gmail.com>

On Thu, Jan 28, 2016 at 2:23 PM, Albert-Jan Roskam
<sjeik_appie at hotmail.com> wrote:
> Out of curiosity, I wrote the throw-away script below to find a character that is classified
> (--> LC_CTYPE) as digit in one locale, but not in another.

The re module is the wrong tool for this. The re.LOCALE flag is only
for byte strings, and in this case only ASCII 0-9 are matched as
decimal digits. It doesn't call the isdigit() ctype function. Using
Unicode with re.LOCALE is wrong. The current locale doesn't affect the
meaning of a Unicode character. Starting with 3.6 doing this will
raise an exception.

The POSIX ctype functions such as isalnum and isdigit are limited to a
single code in the range 0-255 and EOF (-1). For UTF-8, the ctype
functions return 0 in the range 128-255 (i.e. lead bytes and trailing
bytes aren't characters). Even if this range has valid characters in a
given locale, it's meaningless to use a Unicode value from the Latin-1
block, unless the locale uses Latin-1 as its codeset.

Python 2's str uses the locale-aware isdigit() function. However, all
of the locales on my Linux system use UTF-8, so I have to switch to
Windows to demonstrate two locales that differ with respect to
isdigit(). You could use PyWin32 or ctypes to iterate over all the
locales known to Windows, if it mattered that much to you.

The English locale (codepage 1252) includes superscript digits 1, 2, and 3:

    >>> locale.setlocale(locale.LC_CTYPE, 'English_United Kingdom')
    'English_United Kingdom.1252'
    >>> [chr(x) for x in range(256) if chr(x).isdigit()]
    ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '\xb2', '\xb3', '\xb9']
    >>> unicodedata.name('\xb9'.decode('1252'))
    'SUPERSCRIPT ONE'
    >>> unicodedata.name('\xb2'.decode('1252'))
    'SUPERSCRIPT TWO'
    >>> unicodedata.name('\xb3'.decode('1252'))
    'SUPERSCRIPT THREE'

Note that using the re.LOCALE flag doesn't match these superscript digits:

    >>> re.findall(r'\d', '0123456789\xb2\xb3\xb9', re.LOCALE)
    ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']

The Windows Greek locale (codepage 1253) substitutes "?" for superscript 1:

    >>> locale.setlocale(locale.LC_CTYPE, 'Greek_Greece')
    'Greek_Greece.1253'
    >>> [chr(x) for x in range(256) if chr(x).isdigit()]
    ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '\xb2', '\xb3']

    >>> unicodedata.name('\xb9'.decode('1253'))
    'GREEK CAPITAL LETTER ETA WITH TONOS'

From sjeik_appie at hotmail.com  Fri Jan 29 09:50:57 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Fri, 29 Jan 2016 14:50:57 +0000
Subject: [Tutor] http://www.rmunn.com/sqlalchemy-tutorial/tutorial.html
Message-ID: <DUB123-W10694CBF8290DF8F622AE183DB0@phx.gbl>



from sqlalchemy import *

db = create_engine('sqlite:///joindemo.db')

db.echo = True

metadata = BoundMetaData(db)

users = Table('users', metadata, autoload=True)
emails = Table('emails', metadata, autoload=True)

# These are the empty classes that will become our data classes
class User(object):
    pass
class Email(object):
    pass

usermapper = mapper(User, users)
emailmapper = mapper(Email, emails)

session = create_session()

mary = session.query(User).selectfirst(users.c.name=='Mary')
mary.age += 1

session.flush()

fred = User()
fred.name = 'Fred'
fred.age = 37

print "About to flush() without a save()..."
session.flush()  # Will *not* save Fred's data yet

session.save(fred)
print "Just called save(). Now flush() will actually do something."
session.flush()  # Now Fred's data will be saved

session.delete(fred)
session.flush() 		 	   		  

From oscar.j.benjamin at gmail.com  Fri Jan 29 11:32:57 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Fri, 29 Jan 2016 16:32:57 +0000
Subject: [Tutor] lc_ctype and re.LOCALE
In-Reply-To: <DUB123-W20902A91F7061922CE53BD83DA0@phx.gbl>
References: <DUB123-W20902A91F7061922CE53BD83DA0@phx.gbl>
Message-ID: <CAHVvXxQEgZN=Dt-MAjAvaB=Bi5cvB2SxTtmJDx0bPZxwzVqPog@mail.gmail.com>

On 28 January 2016 at 20:23, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
>
> Out of curiosity, I wrote the throw-away script below to find a character that is classified (--> LC_CTYPE) as digit in one locale, but not in another.
> I ran it with 5000 locale combinations in Python 2 but did not find any (somebody shut down my computer!). I just modified the code so it also
> runs in Python 3. Is this the correct way to find such locale-dependent regex matches?

Eryk already gave you a better explanation of the locale stuff than I
could but I have a separate comment about the algorithmic performance
of your code (since you mentioned that it took a long time).

You're looping over all pairs of locales:

...
> for n, (locale1, locale2) in enumerate(itertools.combinations(locales, 2), >
...
>     for i in xrange(sys.maxunicode + 1):   # 1114111
>         s = unichr(i)  #.encode("utf8")
>         try:
>             locale.setlocale(locale.LC_CTYPE, locale1)
>             m1 = bool(regex.match(s))
>             locale.setlocale(locale.LC_CTYPE, locale2)
>             m2 = bool(regex.match(s))
>             if m1 ^ m2:  # m1 != m2

Suppose there are N locales and M is sys.maxunicode. The number of
pairs of locales is N*(N-1)/2 which grows like N**2. For each pair you
loop over M characters so the innermost loop body is repeated
something like M*N**2 times.

Assume that f(locale, c) is the function that gets e.g. m1 or m2 in
your code above. We can swap the loops around so that the outer loop
is over unicode characters. Then the inner loop can be over the
locales but we only loop over all N locales once rather than over all
N**2 pairs of locales. This looks like this:

    for c in unicode_chacters:
        matched = f(locales[0], c) # Check the first locale
        for locale in locales:
            assert all(f(locale, c) == matched for locale in locales)

This way you call f(locale, c) M*N times which if N is not small
should be a lot faster than M*N**2 times.

--
Oscar

From lapsap7+python at gmail.com  Fri Jan 29 12:41:05 2016
From: lapsap7+python at gmail.com (STF)
Date: Fri, 29 Jan 2016 18:41:05 +0100
Subject: [Tutor] Noob: nested if-clauses
In-Reply-To: <n861ii$kih$1@ger.gmane.org>
References: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
 <n83eg8$ukt$1@ger.gmane.org>
 <CANsMTYUn8k4WEwfp2Fg_77xoCE=56foiL32of+Q-w8mcymN44A@mail.gmail.com>
 <n861ii$kih$1@ger.gmane.org>
Message-ID: <CANsMTYVsGNh_RUagDg4t5kEcLg5FY_V28hOPGKs9uOWzHTZ_iQ@mail.gmail.com>

On 25 January 2016 at 21:46, Alan Gauld <alan.gauld at btinternet.com> wrote:

> On 25/01/16 15:52, STF wrote:
>
> > It's a total fluke.  I put the indentation like this to *visually* help
> > myself understand what I was going to write.
>
> That's one of the good things about Python, if it looks right
> it very often is right.
>

Actually, in the original example code I type on notepad, I was using
tabs.  But since I can't press Tab inside Gmail interface, I pressed spaces
instead.

My incomprehension is partially due to this YouTube video:
https://youtu.be/W1zOj2CI-KQ (@ 7:00) in which the author didn't insist on
"consistency".

Another reason is that, while tab is interpreted as 4 white spaces in
convention, it's shown as 8 white spaces in Notepad.  So when I opened some
source code, I have different numbers of leading white spaces, which lead
to my confusion.

Personally, I don't find this as a "good thing".  It rather recalls the
horrible dreams I have had when I was using Fortran!  In Fortran, we have
to deal with position of first characters to make things work.  IMO, making
a visual format an essential thing in programming is a very bad idea, if
it's not superficial.



> > In the Python tutorial that I was using, the author only told us to use
> > indentation, without emphasizing on the size of it.
>
> Quite right the amount is not important(syntactically at least) provided
> you are consistent.
>
> > As I'm a newbie, I'm mostly using Python IDLE but sometimes I would use
> > Programmer's Notepad.
>
> I don't know PN but IDLE will keep you right most of the time.
>
> > Let me ask an alternative question.  Suppose I have something like this:
> > ----
> >
> > if condition_C:
> >     instruction_10
> >    instruction_11
> >      instruction_12
> > ----
> > There are 4 spaces in front of instruction_10, 3 spaces in front of
> > instruction_11 and 5 spaces in front of instruction_12.
> >
> > What would happen to instruction_11 and instruction_12?
>
> One of the best things about Python is the interpreter.
> Just try it and see. It's much faster than posting a question
> here and you can be sure it's the correct answer! If you
> don't understand what you see, then come here.
>
> Just use some print statements or simple assignments
> for example:
>
> >>> if True:
> ...    print 'in the if'
> ...   print 'still here'
> ...      y = 5 * 6
> ...
>
> what happens?
>

OK, I have just tried it (instead of just reading source codes) and I see
what that gives.  IDLE is indeed easier to use than the "DOS-style" Python
command-line window.  How do you call this thing, BTW?  I'm unable to
understand how to navigate inside this thing.  I mean, when I open it, in
which folder am I in?  Suppose I have a Python file in
D:\mycode\abc\myfile.py.  How to run it?

Thx

From alan.gauld at btinternet.com  Fri Jan 29 15:13:46 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 29 Jan 2016 20:13:46 +0000
Subject: [Tutor] Noob: nested if-clauses
In-Reply-To: <CANsMTYVsGNh_RUagDg4t5kEcLg5FY_V28hOPGKs9uOWzHTZ_iQ@mail.gmail.com>
References: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
 <n83eg8$ukt$1@ger.gmane.org>
 <CANsMTYUn8k4WEwfp2Fg_77xoCE=56foiL32of+Q-w8mcymN44A@mail.gmail.com>
 <n861ii$kih$1@ger.gmane.org>
 <CANsMTYVsGNh_RUagDg4t5kEcLg5FY_V28hOPGKs9uOWzHTZ_iQ@mail.gmail.com>
Message-ID: <n8gh5q$d0c$1@ger.gmane.org>

On 29/01/16 17:41, STF wrote:

> Personally, I don't find this as a "good thing".  It rather recalls the
> horrible dreams I have had when I was using Fortran!  In Fortran, we have
> to deal with position of first characters to make things work.  IMO, making
> a visual format an essential thing in programming is a very bad idea, if
> it's not superficial.

I've never done Fortran but I did do COBOL which similarly is fussy
about spacing. But trust me, Python is much more flexible and friendly
in its  use of whitespace. It essentially just asks you to do what
you should be doing anyways.

And the way it works avoids most of the dangling else errors
you get in languages like C/Java etc (And I had >10 years
experience with those before discovering python.)

When I first heard that python was white-space sensitive I
thought, oh no! But now I see it as a huge strength of the
language. Once you get used to it you will find it helps
far more than it hinders - just avoid tabs and/or map your
tab key to spaces.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From ben+python at benfinney.id.au  Fri Jan 29 17:15:18 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 30 Jan 2016 09:15:18 +1100
Subject: [Tutor] How do I test file operations (Such as opening, reading,
 writing, etc.)?
References: <CANDiX9K_-m_CxGbMJO1hw=iUYXnJz69N6s5Fpcq556u221uvVQ@mail.gmail.com>
 <n8ckdr$p08$1@ger.gmane.org>
 <CAGZAPF5cn8AxzpDQE5fyhKw3XzK58-=jbogswBsGYOWGmPL2CQ@mail.gmail.com>
 <CAFTVGQauXzrpPyB6zZdf5f6bGTpwO5X1CM0Wmp8dYH90EuOurg@mail.gmail.com>
 <alpine.LSU.2.11.1601281458350.2025@znpeba.jbaqresebt.arg>
 <85r3h0hneh.fsf@benfinney.id.au> <n8fmbc$9sp$1@ger.gmane.org>
Message-ID: <8537tgghhl.fsf@benfinney.id.au>

Alan Gauld <alan.gauld at btinternet.com> writes:

> File simulation is faster and fine for unit testing functionality,
> but not so good for testing the overall system.

Yes, I'm only advocating fake files for unit tests which, more than all
other kinds of tests, need to be fast and numerous.

For testing larger parts of the system than a single code unit, real
files are better. But likewise, there should be fewer tests (because
complete-coverage unit tests should be much more numerous) so the slower
performance is not so much a problem.

-- 
 \                ?I got fired from my job the other day. They said my |
  `\          personality was weird. ? That's okay, I have four more.? |
_o__)                                       ?Bug-Eyed Earl, _Red Meat_ |
Ben Finney


From robertvstepp at gmail.com  Fri Jan 29 17:23:28 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Fri, 29 Jan 2016 16:23:28 -0600
Subject: [Tutor] Noob: nested if-clauses
In-Reply-To: <CANsMTYVsGNh_RUagDg4t5kEcLg5FY_V28hOPGKs9uOWzHTZ_iQ@mail.gmail.com>
References: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
 <n83eg8$ukt$1@ger.gmane.org>
 <CANsMTYUn8k4WEwfp2Fg_77xoCE=56foiL32of+Q-w8mcymN44A@mail.gmail.com>
 <n861ii$kih$1@ger.gmane.org>
 <CANsMTYVsGNh_RUagDg4t5kEcLg5FY_V28hOPGKs9uOWzHTZ_iQ@mail.gmail.com>
Message-ID: <CANDiX9KsYx7Pk94q617FVZEGmj7kqfzvif4GXwC7nj=j5FY1yA@mail.gmail.com>

On Fri, Jan 29, 2016 at 11:41 AM, STF <lapsap7+python at gmail.com> wrote:

> ...How do you call this thing, BTW?  I'm unable to
> understand how to navigate inside this thing.  I mean, when I open it, in
> which folder am I in?  Suppose I have a Python file in
> D:\mycode\abc\myfile.py.  How to run it?

IDLE presents itself in two ways.  Normally it defaults to opening up
as the Python interpreter where you see the ">>>" and can type in code
interactively.

Its other view can be accessed by going to the "File" menu and either
choosing "Open..." and browsing to the file you wish to open, such as
"D:\mycode\abc\myfile.py" you mention above, or by creating a new file
with the menu option "New File".  This will open a second window which
is an editor window that you can type in and edit Python code, such as
you were doing in Programmer's Notepad.  To run the code you have
either opened using "Open..." or that you have typed in, use the menu
"Run" and select "Run Module".

Having both of these windows open at the same time is quite handy!  As
you type code into the Editor, you may get stuck and not be sure as to
what you want to do next.  You can then go to the Interpreter window
and type in code snippets, exploring Python's behavior for the code
you are contemplating.  Once you are satisfied, back to the Editor and
enter your code.  Of course you can copy and paste between the two as
well.

Notice that the menu bar has differences between the two windows.  You
should explore the various menu options to see what you can do with
both forms of IDLE.  Don't forget to check out "Help > IDLE Help" for
more information on IDLE's options.  Also note that you can get to the
local Python language docs through "Help > Python Docs".

One more thing to note:  IDLE has shortcut keys.  As you explore the
menus you will see some of them listed next to the menu entries.  For
more go to "Options > Configure IDLE" and you will see a small tabbed
window.  If you go to the "Keys" tab, you can see (and edit) the key
bindings IDLE uses as well as available pre-configured key binding
styles.

Hopefully I'm answering the question(s) you are really asking and have
not gone off on a tangent!


-- 
boB

From gokoproject at gmail.com  Fri Jan 29 15:57:22 2016
From: gokoproject at gmail.com (John Wong)
Date: Fri, 29 Jan 2016 15:57:22 -0500
Subject: [Tutor] Noob: nested if-clauses
In-Reply-To: <n8gh5q$d0c$1@ger.gmane.org>
References: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
 <n83eg8$ukt$1@ger.gmane.org>
 <CANsMTYUn8k4WEwfp2Fg_77xoCE=56foiL32of+Q-w8mcymN44A@mail.gmail.com>
 <n861ii$kih$1@ger.gmane.org>
 <CANsMTYVsGNh_RUagDg4t5kEcLg5FY_V28hOPGKs9uOWzHTZ_iQ@mail.gmail.com>
 <n8gh5q$d0c$1@ger.gmane.org>
Message-ID: <CACCLA55x1OuKexzMonc+2g89qsvSEtVXWMpQdkWFS0t0KJZSoQ@mail.gmail.com>

On Fri, Jan 29, 2016 at 3:13 PM, Alan Gauld <alan.gauld at btinternet.com>
wrote:

>
> When I first heard that python was white-space sensitive I
> thought, oh no! But now I see it as a huge strength of the
> language. Once you get used to it you will find it helps
> far more than it hinders - just avoid tabs and/or map your
> tab key to spaces.
>
>
Just want to emphasize: use a tool that actually works for you and for the
programming tool. In general, any modern IDE is capable of doing the right
thing. NotePad++ is also capable, but avoid anything like WordPad or
NotePad, if they are still around...
Lastly, most projects I encounter do use 4-space, but some projects (and
notably Google projects internally) would use 2-space, so you almost never
have to worry about mixing 4-space and 2-space in your code. It just happen
that sometimes some people publish code under tab or 2 space and if you
just copy-paste you can run into issues, just FYI for the beginners.

Thanks.

John

From ryan at allwegot.net  Fri Jan 29 16:29:14 2016
From: ryan at allwegot.net (Ryan Smith)
Date: Fri, 29 Jan 2016 16:29:14 -0500
Subject: [Tutor] Why define a function inside a function?
Message-ID: <CADYLmeEcptkiA6e9AMNtjvoKAVqZyD79T9rDwh+WrQjsw6Hb2w@mail.gmail.com>

Hi all,

I am new to programming and python and had a question. I hope I
articulate this well enough so here it goes... I was following along
on a thread on this mailing list discussing how to test file I/O
operations. In one of the suggested solutions the following code was
used:


def _open_as_stringio(self, filename):
        filelike = StringIO(self.filecontents[filename])
        real_close = filelike.close
        def wrapped_close():
            self.filecontents[filename] = filelike.getvalue()
            real_close()
        filelike.close = wrapped_close
        return filelike

In trying to understand the logic behind the code for my own
edification, I was wondering what is the reasoning of defining a
function in side another function (for lack of a better phrase)?

From alan.gauld at btinternet.com  Fri Jan 29 20:06:51 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 30 Jan 2016 01:06:51 +0000
Subject: [Tutor] Noob: nested if-clauses
In-Reply-To: <CANsMTYVsGNh_RUagDg4t5kEcLg5FY_V28hOPGKs9uOWzHTZ_iQ@mail.gmail.com>
References: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
 <n83eg8$ukt$1@ger.gmane.org>
 <CANsMTYUn8k4WEwfp2Fg_77xoCE=56foiL32of+Q-w8mcymN44A@mail.gmail.com>
 <n861ii$kih$1@ger.gmane.org>
 <CANsMTYVsGNh_RUagDg4t5kEcLg5FY_V28hOPGKs9uOWzHTZ_iQ@mail.gmail.com>
Message-ID: <n8h2bb$l86$1@ger.gmane.org>

On 29/01/16 17:41, STF wrote:

> what that gives.  IDLE is indeed easier to use than the "DOS-style" Python
> command-line window.  

One thing I'd say is that you can tweak the DOS CMD window quite
a lot to make it better for IDLE(and other interpreters). But
IDLE is probably still better for most folks.

> How do you call this thing, BTW?  I'm unable to
> understand how to navigate inside this thing.  

I'm not totally clear what you mean here.
You start IDLE from the start menu or desktop shortcut.
That gets you to the interactive shell which is a version
of the python interpreter running inside idle

> I mean, when I open it, in which folder am I in?

You can find out with

>>> import os
>>> print ( os.getcwd() )

But its rarely important because you normally use
File->Open to open a file, or File->New to create a new
one and save it wherever you want).

Once you do that IDLE opens a new editor (as opposed
to shell) window. From there you can use the Run menu
to execute your code with the output appearing in the
original shell window.

> Suppose I have a Python file in
> D:\mycode\abc\myfile.py.  How to run it?

Within IDLE use File->Open to locate and load it.
Then use the Run->Riun Module menu command (shortcut F5)

Outside idle use

C:\SOME\PATH> python D:\mycode\abc\myfile.py

You can of course create a shortcut to do that.

or CD to the folder:

C:\SOME\PATH> CD  D:\mycode\abc
D:
D:\mycode\abc>

then type

D:\mycode\abc> python myfile.py

hth
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From ben+python at benfinney.id.au  Fri Jan 29 20:10:51 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 30 Jan 2016 12:10:51 +1100
Subject: [Tutor] Why define a function inside a function?
References: <CADYLmeEcptkiA6e9AMNtjvoKAVqZyD79T9rDwh+WrQjsw6Hb2w@mail.gmail.com>
Message-ID: <85a8nng9d0.fsf@benfinney.id.au>

Ryan Smith <ryan at allwegot.net> writes:

> In trying to understand the logic behind the code for my own
> edification, I was wondering what is the reasoning of defining a
> function in side another function (for lack of a better phrase)?

Sometimes a function is so specific to the workings of another function
that it would not make sense outside that context.

Other times, the information needed to define the function ? for
example, a default value for one of the parameters ? may be dynamically
determined within the enclosing function. The statement defining the
inner function would not work without that context.

Yet other times, the inner function may be simpler to express by making
use of the enclosing scope of the outer function. Names in the enclosing
function can be referenced in the inner function, without needing to
formally pass them as parameters. Defining the function at the point of
its use keeps the code readable, and keeps the function simpler.

Lastly, functions themselves are objects like any other in Python.
Defining one function inside another, and then using the defined
function object as a value to pass on for further processing, is a
paradigm known as ?functional programming? which Python has good support
for. This is how it's done.

-- 
 \        ?Like the creators of sitcoms or junk food or package tours, |
  `\         Java's designers were consciously designing a product for |
_o__)                       people not as smart as them.? ?Paul Graham |
Ben Finney


From steve at pearwood.info  Fri Jan 29 22:31:09 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 30 Jan 2016 14:31:09 +1100
Subject: [Tutor] Why define a function inside a function?
In-Reply-To: <CADYLmeEcptkiA6e9AMNtjvoKAVqZyD79T9rDwh+WrQjsw6Hb2w@mail.gmail.com>
References: <CADYLmeEcptkiA6e9AMNtjvoKAVqZyD79T9rDwh+WrQjsw6Hb2w@mail.gmail.com>
Message-ID: <20160130033109.GF4619@ando.pearwood.info>

On Fri, Jan 29, 2016 at 04:29:14PM -0500, Ryan Smith wrote:
> Hi all,
> 
> I am new to programming and python and had a question. I hope I
> articulate this well enough so here it goes... I was following along
> on a thread on this mailing list discussing how to test file I/O
> operations. In one of the suggested solutions the following code was
> used:
> 
> 
> def _open_as_stringio(self, filename):
>         filelike = StringIO(self.filecontents[filename])
>         real_close = filelike.close
>         def wrapped_close():
>             self.filecontents[filename] = filelike.getvalue()
>             real_close()
>         filelike.close = wrapped_close
>         return filelike

This function (actually a method) creates and returns a new object which 
behaves like an actual file. When that fake file's "close" method is 
called, it has to run some code which accesses the instance which 
created it (self). How does this fake file know which instance created 
it? It has to "remember" the value of self, somehow.

There are two usual ways for a function to be given access to a variable 
(in this case, self):

- pass it as a parameter to the function;
- access it as a global variable.

The first can't work, because fake_file.close() takes no arguments.

The second won't work safely, unless you can guarantee that there is 
only ever one fake file at a time. If there are two, they will both try 
to access different values of self in the same global, and bad things 
will happen. ("Global Variables Considered Harmful.")

But there's a third way, and that's called "a closure". And this is 
exactly what happens here. Let's look at the inner function more 
carefully:

def wrapped_close():
    self.filecontents[filename] = filelike.getvalue()
    real_close()


"wrapped_close" takes no arguments, but it references four variables 
inside the body:

- self
- filename
- filelike
- real_close

Each of those variables are defined in the surrounding function. So when 
Python creates the inner function, it records a reference to those outer 
variables inside the inner function. These are called "closures", and we 
say "the inner function closes over these four variables". (I don't know 
why the word "close" is used.) So the word "closure" can be used either 
for the inner function itself, or the internal hidden links between it 
and the variables.

In other words:

The inner function "wrapped_close" remembers the values of those four 
variables, as they were at the time the inner function was defined.

So by the time you eventually get hold of the fake file, and call its 
close method, that close method will remember which instance created the 
fake file, etc, and use those variables as needed.

Complicated? Well, a bit. Here's a simpler demonstration of a closure:


def factory(num):
    """Factory function that returns a new function that adds 
    num to whatever it is given."""
    def adder(x):
        return x + num


add1 = factory(1)
add2 = factory(2)
add17_25 = factory(17.25)

print(add1(100))
# => prints 101

print(add2(50))
# => prints 52

print(add17_25(5000))
# => prints 5017.25


Each of the "add" functions remember the value that "num" had, *at the 
time of creation*. So as far as add1 is concerned, num=1, even though 
add2 considers that num=2.

(Warning: be careful with mutable values like lists, since they can 
change value if you modify them. If this makes no sense to you, please 
ask, and I'll demonstrate what I mean.)


 
> In trying to understand the logic behind the code for my own
> edification, I was wondering what is the reasoning of defining a
> function in side another function (for lack of a better phrase)?

There are two common reasons for defining inner functions.

The first is a boring and not-very-exciting reason: you want to hide 
that function from anyone else, prevent them from using it, except for 
one specific function. So you put it inside that function:

def example(n):
    results = []
    def calc(x):
        # Pretend that calc is actually more complex.
        return x*100
    for i in range(1, n+1):
        results.append(calc(i))
    return results


example(3)
# => returns [100, 200, 300]


That's a fairly boring reason, and people rarely do that. Normally, you 
would just make calc an external function, perhaps named "_calc" so 
people know it is a private function that shouldn't be used.

The second reason is to make the inner function a closure, as shown 
above.



-- 
Steve

From steve at pearwood.info  Fri Jan 29 22:35:21 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 30 Jan 2016 14:35:21 +1100
Subject: [Tutor] Why is the name "self" optional instead of mandatory?
In-Reply-To: <CANDiX9JfY1ACZXwJBazz_tQXgEg--6n5Z26-NREgaEUoczyVfg@mail.gmail.com>
References: <CANDiX9JaRCZ_6QFyGhDnT_sudVBme0E+xfz+_o+QL9-gdv2nEg@mail.gmail.com>
 <20160121114922.GF4619@ando.pearwood.info>
 <CANDiX9JfY1ACZXwJBazz_tQXgEg--6n5Z26-NREgaEUoczyVfg@mail.gmail.com>
Message-ID: <20160130033521.GG4619@ando.pearwood.info>

On Fri, Jan 22, 2016 at 11:10:39PM -0600, boB Stepp wrote:

> Am I understanding this correctly?  It appears to me that you have
> successfully added a method to class X from outside of class X!  If
> this is the case, then this clarifies some things I was wondering
> about in my other thread.


If you are looking for a real-world example of why you might add a 
method to an instance, please look at the thread started by Ryan:

"Why define a function inside a function?"


for an example of adding a pseudo-method to a specific instance (not the 
entire class).

For technical reasons, it's not actually a method, it's a regular 
function, but the difference is not important.


-- 
Steve

From steve at pearwood.info  Fri Jan 29 22:38:11 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 30 Jan 2016 14:38:11 +1100
Subject: [Tutor] Why define a function inside a function?
In-Reply-To: <20160130033109.GF4619@ando.pearwood.info>
References: <CADYLmeEcptkiA6e9AMNtjvoKAVqZyD79T9rDwh+WrQjsw6Hb2w@mail.gmail.com>
 <20160130033109.GF4619@ando.pearwood.info>
Message-ID: <20160130033810.GH4619@ando.pearwood.info>

Oops, I screwed up!


On Sat, Jan 30, 2016 at 02:31:09PM +1100, Steven D'Aprano wrote:

> Complicated? Well, a bit. Here's a simpler demonstration of a closure:
> 
> 
> def factory(num):
>     """Factory function that returns a new function that adds 
>     num to whatever it is given."""
>     def adder(x):
>         return x + num


Of course that can't work, because I forgot to return the inner 
function.

def factory(num):
    def adder(x):
        return x + num
    return adder


will work better. Sorry for any confusion.



-- 
Steve

From alan.gauld at btinternet.com  Sat Jan 30 05:50:46 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 30 Jan 2016 10:50:46 +0000
Subject: [Tutor] Noob: nested if-clauses
In-Reply-To: <n8h2bb$l86$1@ger.gmane.org>
References: <CANsMTYXj5rQgwF92PbHLO8Wka352gmnKAtQo1mKg0C28qp74-g@mail.gmail.com>
 <n83eg8$ukt$1@ger.gmane.org>
 <CANsMTYUn8k4WEwfp2Fg_77xoCE=56foiL32of+Q-w8mcymN44A@mail.gmail.com>
 <n861ii$kih$1@ger.gmane.org>
 <CANsMTYVsGNh_RUagDg4t5kEcLg5FY_V28hOPGKs9uOWzHTZ_iQ@mail.gmail.com>
 <n8h2bb$l86$1@ger.gmane.org>
Message-ID: <n8i4i6$iav$1@ger.gmane.org>

On 30/01/16 01:06, Alan Gauld wrote:

> or CD to the folder:
> 
> C:\SOME\PATH> CD  D:\mycode\abc
> D:
> D:\mycode\abc>

Oops, that sequence should be:

C:\SOME\PATH> CD  D:\mycode\abc
C:\SOME\PATH> D:
D:\mycode\abc>

sorry about that.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From mcshizney at hotmail.co.uk  Sat Jan 30 09:28:50 2016
From: mcshizney at hotmail.co.uk (Lawrence Lorenzo)
Date: Sat, 30 Jan 2016 14:28:50 +0000
Subject: [Tutor] Finding the max value from a dictionary that does not
 exceed a variable's value.
Message-ID: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>

The problem I am facing is commented in the code. Thanks for any help in advance.===============================================================================================================#15 Change Return Program
#The user enters a cost and then the amount of money given. You should write a
#program that works out what denominations of change should be given in pounds, 50p,
#20p, 10p etc.#to do this the programme needs to divide the change ammount by 1.00 then 0.50 then 0.20 untill it reaches zero#Extensions:#1. The program will figure out the change for the American currency and the number
#of quarters, dimes, nickels, pennies needed for the change
#2. Make an automatic testing part of your program where it automatically generates a
#random price and an amount that you give the cashier. It then works out what change to
#give, and then tests that your program works by adding the change back onto the price of
#the item to prove your program works. It should flag an error if there are problems.
cost_of_item = int(float(input("how much money does the item cost?")))money_given = int(float(input("How much money is given?")))change = (money_given) - (cost_of_item)


#def coin_return(amount):coinvalues = {'?1':1.00, '50p':0.50,              '20p': 0.20, '10p': 0.10,              '5p': 0.05, '2p': 0.02,              '1p': 0.0}
while change != 0:    coinvalues = {'?1':1.00, '50p':0.50,                  '20p': 0.20, '10p': 0.10,                  '5p': 0.05, '2p': 0.02,                  '1p': 0.0}    answer = []    for coin_name, coin_value in coinvalues:        #take the maximum value in the dictionary that is not above the variable        #"change" and then append it into the answer list.                       ###################################################################            # I want to take the max value in the dictionary "coinvalues"     #            # that is the same as or lower than the variable "change".        #            # I have no Idea how to search through the coinvalues dictionary  #            # and find the value that is highest but does not exceed the      #             # value held in the variable "change".                            #            ###################################################################

==========================================================================================================              

 		 	   		  

From ryan at allwegot.net  Sat Jan 30 10:54:18 2016
From: ryan at allwegot.net (Ryan Smith)
Date: Sat, 30 Jan 2016 10:54:18 -0500
Subject: [Tutor] Why define a function inside a function?
In-Reply-To: <20160130033810.GH4619@ando.pearwood.info>
References: <CADYLmeEcptkiA6e9AMNtjvoKAVqZyD79T9rDwh+WrQjsw6Hb2w@mail.gmail.com>
 <20160130033109.GF4619@ando.pearwood.info>
 <20160130033810.GH4619@ando.pearwood.info>
Message-ID: <CADYLmeGanW-W2Ap-DVQ4Mm8nwpoaV6vRN7cHOtQkbDm0hP+7CQ@mail.gmail.com>

Thank you to both you and Ben for taking the time and answering my
question! The detailed explanation of the example code I provided
really helped and made things clear. As did the explanations about
closures and a simplified example. I definitely have a lot to learn,
but this was very educational nonetheless.





On Fri, Jan 29, 2016 at 10:38 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> Oops, I screwed up!
>
>
> On Sat, Jan 30, 2016 at 02:31:09PM +1100, Steven D'Aprano wrote:
>
>> Complicated? Well, a bit. Here's a simpler demonstration of a closure:
>>
>>
>> def factory(num):
>>     """Factory function that returns a new function that adds
>>     num to whatever it is given."""
>>     def adder(x):
>>         return x + num
>
>
> Of course that can't work, because I forgot to return the inner
> function.
>
> def factory(num):
>     def adder(x):
>         return x + num
>     return adder
>
>
> will work better. Sorry for any confusion.
>
>
>
> --
> Steve
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From thabo.msiza123 at gmail.com  Sat Jan 30 13:00:17 2016
From: thabo.msiza123 at gmail.com (Thabo Msiza)
Date: Sat, 30 Jan 2016 20:00:17 +0200
Subject: [Tutor] Import Erro:No module numpy
Message-ID: <CADKxj72zq_tDjW9YofWHrV7QrytWHEOSe67O0UsfENVudR6S7g@mail.gmail.com>

Whenever I try to run a matlab plot program (program copied from the
internet) I come up with a Error - "ImportError:No Module named numpy" but
I have installed numpy several times using "pip install...." and "conda
update"....please help

From steve at pearwood.info  Sat Jan 30 17:19:59 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 31 Jan 2016 09:19:59 +1100
Subject: [Tutor] Import Erro:No module numpy
In-Reply-To: <CADKxj72zq_tDjW9YofWHrV7QrytWHEOSe67O0UsfENVudR6S7g@mail.gmail.com>
References: <CADKxj72zq_tDjW9YofWHrV7QrytWHEOSe67O0UsfENVudR6S7g@mail.gmail.com>
Message-ID: <20160130221959.GA31806@ando.pearwood.info>

On Sat, Jan 30, 2016 at 08:00:17PM +0200, Thabo Msiza wrote:
> Whenever I try to run a matlab plot program (program copied from the
> internet) I come up with a Error - "ImportError:No Module named numpy" but
> I have installed numpy several times using "pip install...." and "conda
> update"....please help


Possibly you have Python installed twice, one with numpy, and the 
other without, and you're trying to run the version without numpy.

Or perhaps you have a typo. Please COPY AND PASTE the full traceback 
of the error, starting with the line "Traceback" to the end. Do not 
retype it from memory.

Or perhaps there is some other error? It's hard to tell without more 
information.


What happens when you run:

    pip install numpy


Please COPY AND PASTE the entire output of the command.

What version(s) of Python are you running? What operating system?



-- 
Steve

From danny.yoo at gmail.com  Sat Jan 30 17:26:00 2016
From: danny.yoo at gmail.com (Danny Yoo)
Date: Sat, 30 Jan 2016 14:26:00 -0800
Subject: [Tutor] Finding the max value from a dictionary that does not
 exceed a variable's value.
In-Reply-To: <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
 <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
Message-ID: <CAGZAPF5H56gXkvJ1hUQ0Wp87O9ZeMLo8xxuFZdbBHVEUKy2bOw@mail.gmail.com>

>
> Do you already know how to trailer the maximum or minimum of a list of
numbers?

Apologies.  Substitute the word "trailer" with "take".  I didn't check the
output of the spelling checker as carefully as I should have.

From dyoo at hashcollision.org  Sat Jan 30 17:24:17 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Sat, 30 Jan 2016 14:24:17 -0800
Subject: [Tutor] Finding the max value from a dictionary that does not
 exceed a variable's value.
In-Reply-To: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
Message-ID: <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>

On Jan 30, 2016 12:22 PM, "Lawrence Lorenzo" <mcshizney at hotmail.co.uk>
wrote:
>
> The problem I am facing is commented in the code.

Let me cut out the code, as frankly it's a *huge* distraction from your
question.  Here is the question you are asking:

---
I want to take the max value in the dictionary 'coinvalues'  that is the
same as or lower than the variable 'change'.  I have no idea how to search
through the 'coinvalues' dictionary and find the value that is highest but
does not exceed the value held in the variable 'change'.
---

Do you already know how to trailer the maximum or minimum of a list of
numbers?

From steve at pearwood.info  Sun Jan 31 00:01:22 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 31 Jan 2016 16:01:22 +1100
Subject: [Tutor] Finding the max value from a dictionary that does not
 exceed a variable's value.
In-Reply-To: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
Message-ID: <20160131050122.GE31806@ando.pearwood.info>

On Sat, Jan 30, 2016 at 02:28:50PM +0000, Lawrence Lorenzo wrote:

> #The user enters a cost and then the amount of money given. You should 
> #write a program that works out what denominations of change should be 
> #given in pounds, 50p, 20p, 10p etc.#to do this the programme needs to 
> #divide the change ammount by 1.00 then 0.50 then 0.20 untill it 
> #reaches zero

I wouldn't use floating point numbers for this. I would take the cost 
in pounds and multiply by 100 to get the cost in pence, then divide by 
100, 50, 20, 10 etc to find out how many coins are needed.

To do this, you should use divmod:

py> divmod(47, 50)  # how many 50p in 47p?
(0, 47)
py> divmod(47, 20)  # how many 20p in 47p? any remainder?
(2, 7)
py> divmod(7, 10)
(0, 7)
py> divmod(7, 5)  # Does the UK have 5p coins?
(1, 2)


divmod(a, b) is the same as:

(a//b, a%b)

(how many b's in a, and any remainder?)


Also, I don't think that using a dict is useful or necessary in this 
case. Just use a list:

coins = [100, 50, 20, 10, 5, 2, 1]  # number of pence

and test the change against each coin in turn:

for coin in coins:
    if change//coin > 0:
        ...



-- 
Steve

From mr.eightnoteight at gmail.com  Sun Jan 31 00:50:04 2016
From: mr.eightnoteight at gmail.com (srinivas devaki)
Date: Sun, 31 Jan 2016 11:20:04 +0530
Subject: [Tutor] Finding the max value from a dictionary that does not
 exceed a variable's value.
In-Reply-To: <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
 <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
Message-ID: <CACs7g=DhUdO98BwKQTnT3b4L386xtDXzVWPskxD+5mKpQC61_Q@mail.gmail.com>

On Sun, Jan 31, 2016 at 3:54 AM, Danny Yoo <dyoo at hashcollision.org> wrote:
> ---
> I want to take the max value in the dictionary 'coinvalues'  that is the
> same as or lower than the variable 'change'.  I have no idea how to search
> through the 'coinvalues' dictionary and find the value that is highest but
> does not exceed the value held in the variable 'change'.
> ---

although OP's problem doesn't need this, is there a better way achieve
this other than
using a balanced binary search tree.

From sjeik_appie at hotmail.com  Sun Jan 31 14:48:07 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 31 Jan 2016 19:48:07 +0000
Subject: [Tutor] lc_ctype and re.LOCALE
In-Reply-To: <CACL+1atbdevOg7_hsu9mh0u_ZfQ_A-U3GcLUbFZea2_nCsm=mQ@mail.gmail.com>
References: <DUB123-W20902A91F7061922CE53BD83DA0@phx.gbl>,
 <CACL+1atbdevOg7_hsu9mh0u_ZfQ_A-U3GcLUbFZea2_nCsm=mQ@mail.gmail.com>
Message-ID: <DUB123-W9784D4D10C69CFCED938D83DD0@phx.gbl>



> From: eryksun at gmail.com
> Date: Fri, 29 Jan 2016 07:14:07 -0600
> Subject: Re: [Tutor] lc_ctype and re.LOCALE
> To: tutor at python.org
> CC: sjeik_appie at hotmail.com
> 
> On Thu, Jan 28, 2016 at 2:23 PM, Albert-Jan Roskam
> <sjeik_appie at hotmail.com> wrote:
> > Out of curiosity, I wrote the throw-away script below to find a character that is classified
> > (--> LC_CTYPE) as digit in one locale, but not in another.
> 
> The re module is the wrong tool for this. The re.LOCALE flag is only
> for byte strings, and in this case only ASCII 0-9 are matched as
> decimal digits. It doesn't call the isdigit() ctype function. Using
> Unicode with re.LOCALE is wrong. 


Ok, good to know. In my original Python-2 version of the script I did convert the ordinal to a byte string, but it was still a utf-8 byte string.



> The current locale doesn't affect the
> meaning of a Unicode character. Starting with 3.6 doing this will
> raise an exception.


I find it strange that specifying either re.LOCALE or re.UNICODE is still the "special case". IMHO it is a historical anomaly that ASCII is the "normal case". Matching accented characters should not require any special flags.


> The POSIX ctype functions such as isalnum and isdigit are limited to a
> single code in the range 0-255 and EOF (-1). For UTF-8, the ctype
> functions return 0 in the range 128-255 (i.e. lead bytes and trailing
> bytes aren't characters). Even if this range has valid characters in a
> given locale, it's meaningless to use a Unicode value from the Latin-1
> block, unless the locale uses Latin-1 as its codeset.
> 
> Python 2's str uses the locale-aware isdigit() function. However, all
> of the locales on my Linux system use UTF-8, so I have to switch to
> Windows to demonstrate two locales that differ with respect to
> isdigit(). 


In other words: the LC_CTYPE is only relevant with codepage encodings?


You could use PyWin32 or ctypes to iterate over all the
> locales known to Windows, if it mattered that much to you.
> 
> The English locale (codepage 1252) includes superscript digits 1, 2, and 3:
> 
>     >>> locale.setlocale(locale.LC_CTYPE, 'English_United Kingdom')
>     'English_United Kingdom.1252'
>     >>> [chr(x) for x in range(256) if chr(x).isdigit()]
>     ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '\xb2', '\xb3', '\xb9']
>     >>> unicodedata.name('\xb9'.decode('1252'))
>     'SUPERSCRIPT ONE'
>     >>> unicodedata.name('\xb2'.decode('1252'))
>     'SUPERSCRIPT TWO'
>     >>> unicodedata.name('\xb3'.decode('1252'))
>     'SUPERSCRIPT THREE'


Is character classification also related to the compatibility form of unicode normalization?
>>> unicodedata.normalize("NFKD", u'\xb3')
u'3'

(see also http://unicode.org/reports/tr15/)


> Note that using the re.LOCALE flag doesn't match these superscript digits:
> 
>     >>> re.findall(r'\d', '0123456789\xb2\xb3\xb9', re.LOCALE)
>     ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']



Ok, now I am upset. I did not expect this at all! I would have expected the re results to be in line with the str.isdigit results. If LC_CTYPE is not relevant isn't "re.DIACRITIC"  a better name for the re.LOCALE flag?


 
> The Windows Greek locale (codepage 1253) substitutes "?" for superscript 1:
> 
>     >>> locale.setlocale(locale.LC_CTYPE, 'Greek_Greece')
>     'Greek_Greece.1253'
>     >>> [chr(x) for x in range(256) if chr(x).isdigit()]
>     ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '\xb2', '\xb3']
> 
>     >>> unicodedata.name('\xb9'.decode('1253'))
>     'GREEK CAPITAL LETTER ETA WITH TONOS'

Ok, I switched to Windows to see this with my own eyes. Checked the regex. Strange, but fun to know.

Thanks a lot for your thorough reply!
 		 	   		  

From sjeik_appie at hotmail.com  Sun Jan 31 14:56:21 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 31 Jan 2016 19:56:21 +0000
Subject: [Tutor] lc_ctype and re.LOCALE
In-Reply-To: <CAHVvXxQEgZN=Dt-MAjAvaB=Bi5cvB2SxTtmJDx0bPZxwzVqPog@mail.gmail.com>
References: <DUB123-W20902A91F7061922CE53BD83DA0@phx.gbl>,
 <CAHVvXxQEgZN=Dt-MAjAvaB=Bi5cvB2SxTtmJDx0bPZxwzVqPog@mail.gmail.com>
Message-ID: <DUB123-W7E1C08B90DA1C20ABF37783DD0@phx.gbl>


> From: oscar.j.benjamin at gmail.com
> Date: Fri, 29 Jan 2016 16:32:57 +0000
> Subject: Re: [Tutor] lc_ctype and re.LOCALE
> To: sjeik_appie at hotmail.com
> CC: tutor at python.org
> 
> On 28 January 2016 at 20:23, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
> >
> > Out of curiosity, I wrote the throw-away script below to find a character that is classified (--> LC_CTYPE) as digit in one locale, but not in another.
> > I ran it with 5000 locale combinations in Python 2 but did not find any (somebody shut down my computer!). I just modified the code so it also
> > runs in Python 3. Is this the correct way to find such locale-dependent regex matches?
> 
> Eryk already gave you a better explanation of the locale stuff than I
> could but I have a separate comment about the algorithmic performance
> of your code (since you mentioned that it took a long time).
> 
> You're looping over all pairs of locales:
> 
> ...
> > for n, (locale1, locale2) in enumerate(itertools.combinations(locales, 2), >
> ...
> >     for i in xrange(sys.maxunicode + 1):   # 1114111
> >         s = unichr(i)  #.encode("utf8")
> >         try:
> >             locale.setlocale(locale.LC_CTYPE, locale1)
> >             m1 = bool(regex.match(s))
> >             locale.setlocale(locale.LC_CTYPE, locale2)
> >             m2 = bool(regex.match(s))
> >             if m1 ^ m2:  # m1 != m2
> 
> Suppose there are N locales and M is sys.maxunicode. The number of
> pairs of locales is N*(N-1)/2 which grows like N**2. For each pair you
> loop over M characters so the innermost loop body is repeated
> something like M*N**2 times.
> 
> Assume that f(locale, c) is the function that gets e.g. m1 or m2 in
> your code above. We can swap the loops around so that the outer loop
> is over unicode characters. Then the inner loop can be over the
> locales but we only loop over all N locales once rather than over all
> N**2 pairs of locales. This looks like this:
> 
>     for c in unicode_chacters:
>         matched = f(locales[0], c) # Check the first locale
>         for locale in locales:
>             assert all(f(locale, c) == matched for locale in locales)
> 
> This way you call f(locale, c) M*N times which if N is not small
> should be a lot faster than M*N**2 times.

Hi Oscar,

Putting the Busiest Loop on the InsideWhen you have nested loops, think about which loop you want on the outside and which you want on the inside. Following is an example of a nested loop that can be improved:I blindly followed a code tuning tip I once read about in Code Complete (McConnel 2004; page 623 [1]):
" Putting the Busiest Loop on the Inside

When you have nested loops, think about which loop you want on the outside and
which you want on the inside. Following is an example of a nested loop that can be
improved:
...
The key to improving the loop is that the outer loop executes much more often than the
inner loop. Each time the loop executes, it has to initialize the loop index, increment it
on each pass through the loop, and check it after each pass" 

[1] https://khmerbamboo.files.wordpress.com/2014/09/code-complete-2nd-edition-v413hav.pdf

Your advice makes perfect sense, though. So McConnel's code tuning tip may be a rule-of-thumb, with exceptions, right?

Thanks!


 		 	   		  

From sjeik_appie at hotmail.com  Sun Jan 31 14:57:30 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 31 Jan 2016 19:57:30 +0000
Subject: [Tutor] http://www.rmunn.com/sqlalchemy-tutorial/tutorial.html
In-Reply-To: <DUB123-W10694CBF8290DF8F622AE183DB0@phx.gbl>
References: <DUB123-W10694CBF8290DF8F622AE183DB0@phx.gbl>
Message-ID: <DUB123-W415827074D434E253A341E83DD0@phx.gbl>


> Date: Fri, 29 Jan 2016 14:50:57 +0000
> Subject: [Tutor] http://www.rmunn.com/sqlalchemy-tutorial/tutorial.html

sorry, I did not intend to send this to Python Tutor (the link is awesome, though)
 		 	   		  

From sjeik_appie at hotmail.com  Sun Jan 31 16:41:37 2016
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 31 Jan 2016 21:41:37 +0000
Subject: [Tutor] lc_ctype and re.LOCALE
In-Reply-To: <DUB123-W7E1C08B90DA1C20ABF37783DD0@phx.gbl>
References: <DUB123-W20902A91F7061922CE53BD83DA0@phx.gbl>, ,
 <CAHVvXxQEgZN=Dt-MAjAvaB=Bi5cvB2SxTtmJDx0bPZxwzVqPog@mail.gmail.com>,
 <DUB123-W7E1C08B90DA1C20ABF37783DD0@phx.gbl>
Message-ID: <DUB123-W471A4FB999AA645D86DC2183DD0@phx.gbl>

> From: sjeik_appie at hotmail.com
> To: oscar.j.benjamin at gmail.com
> Date: Sun, 31 Jan 2016 19:56:21 +0000
> Subject: Re: [Tutor] lc_ctype and re.LOCALE
> CC: tutor at python.org
> 
> 
> > From: oscar.j.benjamin at gmail.com
> > Date: Fri, 29 Jan 2016 16:32:57 +0000
> > Subject: Re: [Tutor] lc_ctype and re.LOCALE
> > To: sjeik_appie at hotmail.com
> > CC: tutor at python.org
> > 
> > On 28 January 2016 at 20:23, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
> > >
> > > Out of curiosity, I wrote the throw-away script below to find a character that is classified (--> LC_CTYPE) as digit in one locale, but not in another.
> > > I ran it with 5000 locale combinations in Python 2 but did not find any (somebody shut down my computer!). I just modified the code so it also
> > > runs in Python 3. Is this the correct way to find such locale-dependent regex matches?
> > 
> > Eryk already gave you a better explanation of the locale stuff than I
> > could but I have a separate comment about the algorithmic performance
> > of your code (since you mentioned that it took a long time).
> > 
> > You're looping over all pairs of locales:
> > 
> > ...
> > > for n, (locale1, locale2) in enumerate(itertools.combinations(locales, 2), >
> > ...
> > >     for i in xrange(sys.maxunicode + 1):   # 1114111
> > >         s = unichr(i)  #.encode("utf8")
> > >         try:
> > >             locale.setlocale(locale.LC_CTYPE, locale1)
> > >             m1 = bool(regex.match(s))
> > >             locale.setlocale(locale.LC_CTYPE, locale2)
> > >             m2 = bool(regex.match(s))
> > >             if m1 ^ m2:  # m1 != m2
> > 
> > Suppose there are N locales and M is sys.maxunicode. The number of
> > pairs of locales is N*(N-1)/2 which grows like N**2. For each pair you
> > loop over M characters so the innermost loop body is repeated
> > something like M*N**2 times.
> > 
> > Assume that f(locale, c) is the function that gets e.g. m1 or m2 in
> > your code above. We can swap the loops around so that the outer loop
> > is over unicode characters. Then the inner loop can be over the
> > locales but we only loop over all N locales once rather than over all
> > N**2 pairs of locales. This looks like this:
> > 
> >     for c in unicode_chacters:
> >         matched = f(locales[0], c) # Check the first locale
> >         for locale in locales:
> >             assert all(f(locale, c) == matched for locale in locales)
> > 
> > This way you call f(locale, c) M*N times which if N is not small
> > should be a lot faster than M*N**2 times.
> 
> Hi Oscar,
> 

Oh, it seems NoScript or something messed up Hotmail. Here is what I intended to send:

I blindly followed a code tuning tip I once read about in Code Complete (McConnel 2004; page 623 [1]):
 " Putting the Busiest Loop on the Inside
 
 When you have nested loops, think about which loop you want on the outside and
 which you want on the inside. Following is an example of a nested loop that can be
 improved:
 ...
 The key to improving the loop is that the outer loop executes much more often than the
 inner loop. Each time the loop executes, it has to initialize the loop index, increment it
 on each pass through the loop, and check it after each pass" 
 
 [1] https://khmerbamboo.files.wordpress.com/2014/09/code-complete-2nd-edition-v413hav.pdf
 
 Your advice makes perfect sense, though. So McConnel's code tuning tip may be a rule-of-thumb, with exceptions, right?




> 
> Thanks!
> 
> 
>  		 	   		  
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor