From knnleow at gmail.com  Mon Feb  1 02:53:21 2016
From: knnleow at gmail.com (knnleow GOOGLE)
Date: Mon, 1 Feb 2016 15:53:21 +0800
Subject: [Tutor] 2016-02-01 Filter STRINGS in Log File and Pass as VARAIBLE
 within PYTHON script
Message-ID: <56AF0EF1.70906@gmail.com>

hello all,

trying out on how to port my unix shell script to python.
get more complicated than i expected.....: (
i am not familiar with the modules available in python.

anyone care to share how to better the clumsy approach below.

regards,
kuenn


                 timestamp02 = time.strftime("%Y-%m-%d-%H%M%S")
                 banIPaddressesFile = os.popen("cat 
/var/log/fail2ban.log| egrep ssh| egrep Ban| egrep " + myDate + "| awk 
\'{print $7}\'| sort -n| uniq >/tmp/banIPaddressesFile." + 
timestamp02).read()
                 banIPaddresses = open("/tmp/banIPaddressesFile." + 
timestamp02,mode="r",encoding="utf-8")
                 print("banIPaddresses:")
                 print(banIPaddresses)
                 for banIP in banIPaddresses:
                         #print("banIP:", banIP)
                         banIP1 = banIP.rstrip("\n")
                         print("BanIPaddress:", banIP1)
                         whoisIP = os.popen("whois -H " + banIP1 + " 
|egrep -i \"name|country|mail\" |sort -n |uniq").read()
                         print("whoisIP:", whoisIP)


From cs at zip.com.au  Mon Feb  1 05:50:41 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 1 Feb 2016 21:50:41 +1100
Subject: [Tutor] 2016-02-01 Filter STRINGS in Log File and Pass as
 VARAIBLE within PYTHON script
In-Reply-To: <56AF0EF1.70906@gmail.com>
References: <56AF0EF1.70906@gmail.com>
Message-ID: <20160201105041.GA97843@cskk.homeip.net>

On 01Feb2016 15:53, knnleow GOOGLE <knnleow at gmail.com> wrote:
>trying out on how to port my unix shell script to python.
>get more complicated than i expected.....: (
>i am not familiar with the modules available in python.
>anyone care to share how to better the clumsy approach below.
>regards,
>kuenn
>
>                timestamp02 = time.strftime("%Y-%m-%d-%H%M%S")
>                banIPaddressesFile = os.popen("cat 
>/var/log/fail2ban.log| egrep ssh| egrep Ban| egrep " + myDate + "| awk 
>\'{print $7}\'| sort -n| uniq >/tmp/banIPaddressesFile." + 
>timestamp02).read()

First up, this is still essentially a shell script. You're constructing a shell 
pipeline like this (paraphrased):

  cat >/var/log/fail2ban.log
  | egrep ssh
  | egrep Ban
  | egrep myDate
  | awk '{print $7}'
  | sort -n
  | uniq 
  >/tmp/banIPaddressesFile-timestamp

So really, you're doing almost nothing in Python. You're also writing 
intermediate results to a temporary filename, then reading from it. Unless you 
really need to keep that file around, you won't need that either.

Before I get into the Python side of things, there are a few small (small) 
criticisms of your shell script:

- it has a "useless cat"; this is a very common shell inefficiency there people 
  put "cat filename | filter1 | filter2 ..." when they could more cleanly just 
  go "filter1 <filename | filter2 | ..."

- you are searching for fixed strings; why are you using egrep? Just say "grep" 
  (or even "fgrep" if you're old school - you're new to this so I presume not)

- you're using "sort -n | uniq", presumably because uniq requires sorted input; 
  you are better off using "sort -un" here and skipping uniq. I'd also point 
  out that since these are IP addresses, "sort -n" doesn't really do what you 
  want here.

So, to the Python:

You seem to want to read the file /var/log/fail2ban.log and for certain 
specific lines, record column 7 which I gather from the rest of the code 
(below) is an IP address. I gather you just want one copy of each unique IP 
address.

So, to read lines of the file the standard idom goes:

  with open('/var/log/fail2ban.log') as fail_log:
    for line in fail_log:
      ... process lines here ...

You seem to be checking for two keywords and a date in the interesting lines.  
You can do this with a simple test:

  if 'ssh' in line and 'Ban' in line and myDate in line:

If you want the seventh column from the line (per your awk command) you can get 
it like this:

  words = line.split()
  word7 = words[6]

because Python arrays count form 0, therefore index 6 is the seventh word.

You want the unique IP addresses, so I suggest storing them all in a set and 
not bothering with a sort until some other time. So make an empty set before 
you read the file:

  ip_addrs = set()

and add each address to it for the lines you select:

  ip_addrs.add(word7)

After you have read the whole file you will have the desired addresses in the 
ip_addrs set.

Try to put all that together and come back with working code, or come back with 
completed but not working code and specific questions.

Cheers,
Cameron Simpson <cs at zip.com.au>

From oscar.j.benjamin at gmail.com  Mon Feb  1 07:27:35 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 1 Feb 2016 12:27:35 +0000
Subject: [Tutor] lc_ctype and re.LOCALE
In-Reply-To: <DUB123-W471A4FB999AA645D86DC2183DD0@phx.gbl>
References: <DUB123-W20902A91F7061922CE53BD83DA0@phx.gbl>
 <CAHVvXxQEgZN=Dt-MAjAvaB=Bi5cvB2SxTtmJDx0bPZxwzVqPog@mail.gmail.com>
 <DUB123-W7E1C08B90DA1C20ABF37783DD0@phx.gbl>
 <DUB123-W471A4FB999AA645D86DC2183DD0@phx.gbl>
Message-ID: <CAHVvXxSCtPwKxR_HLECZ5idpe08wbHQiozEZEv6H1QMEoaW4Gg@mail.gmail.com>

On 31 January 2016 at 21:41, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
>> >
>> >
>> > You're looping over all pairs of locales:
>> >
>> > Suppose there are N locales and M is sys.maxunicode. The number of
>> > pairs of locales is N*(N-1)/2 which grows like N**2. For each pair you
>> > loop over M characters so the innermost loop body is repeated
>> > something like M*N**2 times.
>> >
>> > Assume that f(locale, c) is the function that gets e.g. m1 or m2 in
>> > your code above. We can swap the loops around so that the outer loop
>> > is over unicode characters. Then the inner loop can be over the
>> > locales but we only loop over all N locales once rather than over all
>> > N**2 pairs of locales. This looks like this:
>> >
>> > for c in unicode_chacters:
>> > matched = f(locales[0], c) # Check the first locale
>> > for locale in locales:
>> > assert all(f(locale, c) == matched for locale in locales)
>> >
>> > This way you call f(locale, c) M*N times which if N is not small
>> > should be a lot faster than M*N**2 times.
>>
>
> I blindly followed a code tuning tip I once read about in Code Complete
> (McConnel 2004; page 623 [1]):
>  " Putting the Busiest Loop on the Inside
>
>  When you have nested loops, think about which loop you want on the outside
> and
>  which you want on the inside. Following is an example of a nested loop that
> can be
>  improved:
>  ...
>  The key to improving the loop is that the outer loop executes much more
> often than the
>  inner loop. Each time the loop executes, it has to initialize the loop
> index, increment it
>  on each pass through the loop, and check it after each pass"
>
>  [1]
> https://khmerbamboo.files.wordpress.com/2014/09/code-complete-2nd-edition-v413hav.pdf
>
>  Your advice makes perfect sense, though. So McConnel's code tuning tip may
> be a rule-of-thumb, with exceptions, right?

I don't really understand the text you've quoted from the book but I
guess it's referring to a method to choose which loop should be the
outer in a nested loop. The rationale depends on many things though so
it would have to be a rule of thumb. However I doubt that McConnel's
advice applies to the situation we have in this thread because I
wasn't suggesting to simply swap the loops around.

Let's consider what McConnel might mean. Suppose I have a nested loop
so it looks like:

    # case 1
    for n in range(N):
        func_n(n)
        for m in range(M):
            func_m(m)
            func_nm(n, m)

The functions func_n etc. might not really be functions. They are
standins for blocks of code that you might have in those positions.
The point is that func_n depends only on n and needs to be re-executed
each time n changes. Likewise func_m depends only on m and func_nm
depends on both.

Sow func_nm is always going to be called N*M times. Swapping the order
of the two loops won't change that. However func_n will be called N
times and func_m will be called N*M times.

Swapping the loops around we would have

    # case 2
    for m in range(M):
        func_m(m)
        for n in range(N):
            func_n(n)
            func_nm(n, m)

In this case we have M calls to func_m and N*M called to func_n. If
C() is the cost of calling a function then the total cost is

    case 1:  M*N*C(func_nm) + N*C(func_n) + N*M*C(func_m)
    case 2:  M*N*C(func_nm) + M*C(func_m) + N*M*C(func_n)

Since the first term is the same in both cases we can ignore it. Now
suppose that func_n and func_m have approximately equal cost. Let's
just say they each have a cost of 1 for simplicity. Then comparing the
second two terms we get

    case1: N + N*M
    case 2: M + N*M

Now which is bigger depends on which is bigger of N and M. So we can
say that if N < M then case 1 is better and if M < N case 2 is better.
In other words: the outer loop should be the one with fewer
iterations. This suggests that

    for n in range(3):
        for m in range(1000):

is better than

    for m in range(1000):
        for n in range(3):

Just looking at calls to the range function the first example calls
range 4 times where as the second one calls range 3001 times so maybe
that is correct. The effect is likely to be small though (especially
if there is any significant work in the inside the inner loop i.e.
func_nm is costly).

Now consider the case where func_n and func_m have wildly different
costs. Let's assume that func_n takes orders of magnitude more time
than func_m. For example func_n might load something from a database
and func_m might add two integers. In this case which is bigger out of
N and M is likely to be unimportant. func_n is costly and we must
minimise the number of times we do it: the outer loop should be the
loop that is most costly to iterate. Example:

    words = 'foo', 'bar', 'baz'

    with open('stuff.txt') as inputfile:
        for line in inputfile:
            for word in words:
                if word in line:
                    print(word, line)

Switching the order in that loop gives us:

    for word in words:
       with open('stuff.txt') as inputfile:
            for line in inputfile:
                if word in line:
                    print(word, line)

So although words is a shorter loop (let's assume file has many more
than 3 lines) we should still use the file as the outer loop. The
second version here reads the whole file 3 times which is pretty much
guaranteed to be more expensive than any gains from not looping over
words too many times.

So by this last logic your two loops are one over the locales and one
over the unicode characters. Looping over the characters is cheap. For
each locale you call setlocale() and I have no idea how expensive that
is: maybe it just stores a string somewhere or maybe it loads some
files from the filesystem and parses them or something. I'm sure that
changing locale is more expensive than iterating through unicode
characters but I couldn't say whether it's a massive difference or
not. In any case if changing locale is more costly AND there are many
more unicode characters than locales then both arguments above suggest
the following:

    for locale in locales:
       for character in unicode_characters:

However the question now is what do you put in the inner-most loop? We
want to *compare* all of the (locale, character) combinations across
locales with each character fixed. We'd have to store the results of
the calculations in some data structure e.g. for each locale we have a
list of results for each unicode character.

    results = {}
    for locale in locales:
       results[locale] = []
       for character in unicode_characters:
            results[locale][character] = f(locale, character)

It's possible to use a smaller data structure but that's not
important. The main thing is that since we want compare across locales
for a given character the problem naturally lends itself to looping
over locales with the character fixed. Swapping the loops around we no
longer need a data structure:

   for character in unicode_characters:
        val = f(locales[0], character)
        assert all(f(locale, character) == val for locale in locales)

However using a data-structure may be faster: it's common for there to
be a trade off between memory usage and computation time.

In any case NONE of the above applies to the code you actually showed
which looks like this:

    for locale1, locale2 in combinations(locales, 2):
        for i in xrange(sys.maxunicode + 1):
            assert(f(locale1, i) == f(locale2, i))

The difference here is that you are doing many more loop iterations
than necessary. In the argument about swapping loops above I assumed
that func_nm is always called N*M times. But that's not true in the
loop above: by looping over combinations you've made the outer loop a
loop with len(locales)**2 iterations instead of len(locales)
iterations. This means that your innermost function body is executed
N**2 * M times which dwarfs any optimisation that can come from
switching the order of the loops.

The reason is that each locale appears in N-1 locale pairs. So your
loop will see each locale, character combination N-1 times instead of
just once. McConnel's arguments are about higher-order (meaning less
important) effects. The number 1 rule of optimising loops is to
minimise the number of loop iterations; in the ideal case you would
somehow reduce the number of iterations to 0.

For this particular problem the minimum possible is all locale,
character combinations so you need to have at least N*M inner loop
iterations. I would always begin by thinking of a method that meets
that minimum.

--
Oscar

From oscar.j.benjamin at gmail.com  Mon Feb  1 07:47:02 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 1 Feb 2016 12:47:02 +0000
Subject: [Tutor] Finding the max value from a dictionary that does not
 exceed a variable's value.
In-Reply-To: <CACs7g=DhUdO98BwKQTnT3b4L386xtDXzVWPskxD+5mKpQC61_Q@mail.gmail.com>
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
 <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
 <CACs7g=DhUdO98BwKQTnT3b4L386xtDXzVWPskxD+5mKpQC61_Q@mail.gmail.com>
Message-ID: <CAHVvXxSTiusxdcR3078b+Q0c_chuFr_sHbddJYWkF71mia7b8Q@mail.gmail.com>

On 31 January 2016 at 05:50, srinivas devaki <mr.eightnoteight at gmail.com> wrote:
> On Sun, Jan 31, 2016 at 3:54 AM, Danny Yoo <dyoo at hashcollision.org> wrote:
>> ---
>> I want to take the max value in the dictionary 'coinvalues'  that is the
>> same as or lower than the variable 'change'.  I have no idea how to search
>> through the 'coinvalues' dictionary and find the value that is highest but
>> does not exceed the value held in the variable 'change'.
>> ---
>
> although OP's problem doesn't need this, is there a better way achieve
> this other than
> using a balanced binary search tree.

You would need to state all of the requirements for your data
structure. If coinvalues is constant then you can use a list and the
bisect module:

https://docs.python.org/3.5/library/bisect.html

That gives O(log(len(coinvalues))) lookup.

--
Oscar

From steve at pearwood.info  Mon Feb  1 08:29:53 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 2 Feb 2016 00:29:53 +1100
Subject: [Tutor] Finding the max value from a dictionary that does not
 exceed a variable's value.
In-Reply-To: <CAHVvXxSTiusxdcR3078b+Q0c_chuFr_sHbddJYWkF71mia7b8Q@mail.gmail.com>
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
 <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
 <CACs7g=DhUdO98BwKQTnT3b4L386xtDXzVWPskxD+5mKpQC61_Q@mail.gmail.com>
 <CAHVvXxSTiusxdcR3078b+Q0c_chuFr_sHbddJYWkF71mia7b8Q@mail.gmail.com>
Message-ID: <20160201132953.GG31806@ando.pearwood.info>

On Mon, Feb 01, 2016 at 12:47:02PM +0000, Oscar Benjamin wrote:
> On 31 January 2016 at 05:50, srinivas devaki <mr.eightnoteight at gmail.com> wrote:
> > On Sun, Jan 31, 2016 at 3:54 AM, Danny Yoo <dyoo at hashcollision.org> wrote:
> >> ---
> >> I want to take the max value in the dictionary 'coinvalues'  that is the
> >> same as or lower than the variable 'change'.  I have no idea how to search
> >> through the 'coinvalues' dictionary and find the value that is highest but
> >> does not exceed the value held in the variable 'change'.
> >> ---
> >
> > although OP's problem doesn't need this, is there a better way achieve
> > this other than
> > using a balanced binary search tree.
> 
> You would need to state all of the requirements for your data
> structure. If coinvalues is constant then you can use a list and the
> bisect module:
> 
> https://docs.python.org/3.5/library/bisect.html
> 
> That gives O(log(len(coinvalues))) lookup.


There are unlikely to be more than dozen distinct coins. Australia has 
$2, $1, 50?, 20?, 10?, 5? coins. Even if you include the obsolete 2? and 
1? coins, that's only eight. I believe the US has $1, 50? (half dollar), 
25? (quarter), 10? (dime), 5? (nickel), 1? (penny).

You're unlikely to beat a straight linear search with only a few coins 
like this. Binary search has much more overhead. Especially since the 
coin change problem doesn't really require a search: just iterate over 
each coin in turn.


-- 
Steve

From oscar.j.benjamin at gmail.com  Mon Feb  1 09:52:21 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 1 Feb 2016 14:52:21 +0000
Subject: [Tutor] Finding the max value from a dictionary that does not
 exceed a variable's value.
In-Reply-To: <20160201132953.GG31806@ando.pearwood.info>
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
 <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
 <CACs7g=DhUdO98BwKQTnT3b4L386xtDXzVWPskxD+5mKpQC61_Q@mail.gmail.com>
 <CAHVvXxSTiusxdcR3078b+Q0c_chuFr_sHbddJYWkF71mia7b8Q@mail.gmail.com>
 <20160201132953.GG31806@ando.pearwood.info>
Message-ID: <CAHVvXxSd=M_=wLubCaf5GCb-yfy4mr1U3qkqTbxZiV_waRojWQ@mail.gmail.com>

On 1 February 2016 at 13:29, Steven D'Aprano <steve at pearwood.info> wrote:
>> > although OP's problem doesn't need this, is there a better way achieve
>> > this other than
>> > using a balanced binary search tree.
>>
>> You would need to state all of the requirements for your data
>> structure. If coinvalues is constant then you can use a list and the
>> bisect module:
>>
>> https://docs.python.org/3.5/library/bisect.html
>>
>> That gives O(log(len(coinvalues))) lookup.
>
> There are unlikely to be more than dozen distinct coins. Australia has
> $2, $1, 50?, 20?, 10?, 5? coins. Even if you include the obsolete 2? and
> 1? coins, that's only eight. I believe the US has $1, 50? (half dollar),
> 25? (quarter), 10? (dime), 5? (nickel), 1? (penny).
>
> You're unlikely to beat a straight linear search with only a few coins
> like this. Binary search has much more overhead. Especially since the
> coin change problem doesn't really require a search: just iterate over
> each coin in turn.

It was acknowledged that "OP's problem doesn't need this" so I assume
the question was to think about it more generally somehow.

--
Oscar

From dyoo at hashcollision.org  Mon Feb  1 14:28:05 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 1 Feb 2016 11:28:05 -0800
Subject: [Tutor] Finding the max value from a dictionary that does not
 exceed a variable's value.
In-Reply-To: <CAHVvXxSd=M_=wLubCaf5GCb-yfy4mr1U3qkqTbxZiV_waRojWQ@mail.gmail.com>
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
 <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
 <CACs7g=DhUdO98BwKQTnT3b4L386xtDXzVWPskxD+5mKpQC61_Q@mail.gmail.com>
 <CAHVvXxSTiusxdcR3078b+Q0c_chuFr_sHbddJYWkF71mia7b8Q@mail.gmail.com>
 <20160201132953.GG31806@ando.pearwood.info>
 <CAHVvXxSd=M_=wLubCaf5GCb-yfy4mr1U3qkqTbxZiV_waRojWQ@mail.gmail.com>
Message-ID: <CAGZAPF72Yt2XXnEecBZe_fE6MQw8fsTdVch0radXJ+yaYd2PMQ@mail.gmail.com>

>
> It was acknowledged that "OP's problem doesn't need this" so I assume
> the question was to think about it more generally somehow.
>


While we're on wild tangents...

We should probably note, since it hasn't been mentioned yet, that the
generalized problem with arbitrary coin choices is classically known
as the change-making problem:

    https://en.wikipedia.org/wiki/Change-making_problem

which is one of the core examples used when teaching the
dynamic-programming algorithm technique.

The reason we don't need dynamic-programming for the original poster's
question is because the "greedy" algorithm works on US and modern UK
denominations, because the coin set is "canonical", in the sense
described in David Pearson's "A Polynomial-time Algorithm for the
Change-Making Problem".  More details in:
http://cs.stackexchange.com/questions/6552/when-can-a-greedy-algorithm-solve-the-coin-change-problem

From dyoo at hashcollision.org  Mon Feb  1 15:00:47 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 1 Feb 2016 12:00:47 -0800
Subject: [Tutor] Fwd: Finding the max value from a dictionary that does not
 exceed a variable's value.
In-Reply-To: <CAGZAPF7t+7qgwNYU8LX8+NtCUdOni0__MW2bOO+3PKZ-zt_z-g@mail.gmail.com>
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
 <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
 <CACs7g=DhUdO98BwKQTnT3b4L386xtDXzVWPskxD+5mKpQC61_Q@mail.gmail.com>
 <CAGZAPF7t+7qgwNYU8LX8+NtCUdOni0__MW2bOO+3PKZ-zt_z-g@mail.gmail.com>
Message-ID: <CAGZAPF4LWBe=cb2xcyP2VLd2mKZJr_v2QmoTEcyEfM7T6nvQ+w@mail.gmail.com>

Here's a response I sent to Srinivas yesterday to further explain why
a balanced binary tree is probably overkill for the "largest
denomination" selection problem.  (I didn't realize that I had not
sent the following to the list.)



---------- Forwarded message ----------
From: Danny Yoo <dyoo at hashcollision.org>
Date: Sun, Jan 31, 2016 at 12:02 AM
Subject: Re: [Tutor] Finding the max value from a dictionary that does
not exceed a variable's value.
To: srinivas devaki <mr.eightnoteight at gmail.com>


On Sat, Jan 30, 2016 at 9:50 PM, srinivas devaki
<mr.eightnoteight at gmail.com> wrote:
> On Sun, Jan 31, 2016 at 3:54 AM, Danny Yoo <dyoo at hashcollision.org> wrote:
>> ---
>> I want to take the max value in the dictionary 'coinvalues'  that is the
>> same as or lower than the variable 'change'.  I have no idea how to search
>> through the 'coinvalues' dictionary and find the value that is highest but
>> does not exceed the value held in the variable 'change'.
>> ---
>
> although OP's problem doesn't need this, is there a better way achieve
> this other than using a balanced binary search tree.


Hi Srinivas,


Given that we're only talking about a few choices here, we likely
don't even need any kind of special data structure here.  The data is
small, and simply considering each coin in turn is probably good
enough.  That is, just do a linear scan.  :P


---


If we really want to over-optimize, look into the bisect library.
That is, binary search is one way to make the search fast.

    https://docs.python.org/3.5/library/bisect.html


You mentioned balanced binary search tree.  I'm assuming that we might
pick a particular implementation, like red-black trees, or AVL trees,
or other structures.  In one sense, they might be good because they
would certainly let us pick the largest appropriate denomination
quickly.  But I'd argue that they would be vastly overkill, precisely
because they support operations that we just don't need for this
problem.

To be specific, a balanced binary tree supports changes over time,
such as insertions and deletions.  But note that the denominations in
the problem statement don't change!  Because we're not dealing with
with a collection that needs to change over time, we're not going to
do any dynamic editing operations.

All the machinery behind a balanced binary tree is there to support
dynamic insertions and deletions: if we're not doing any dynamic
insertions, all that machinery is superfluous.


... But if we really wanted to over-optimize, an alternative approach
to implement this "find the biggest appropriate denomination" question
might be to prepare, in advance, a table to help answer questions
fast.  For US denominations, the table might look something like this:

    LOOKUP_TABLE = [0, 1, 1, 1, 1, 5, 5, ...]

where I'll elicit the details since I don't want to just give a away
an answer to a homework question.  But if we understand the binary
search approach, we should understand the lookup table approach too,
as it's fundamentally even simpler.  It's just an array lookup!
Hilariously, though, the cost of setting up the lookup table is
probably a lot larger than the amount of work needed to solve the
problem with the direct manner.  The lookup table approach makes sense
only if it's used a lot.


So there are several crazy avenues we can take to over-optimize this
problem.  Just to make it clear: I think sticking to a simple linear
scan makes the most sense.  Everything else just seems to try to make
the problem harder than it deserves to be, akin to trying to take the
size of a rectangle via integration.

http://homepage.usask.ca/~blb230/Math_Comics/Calculus_Comic_files/image001.gif

From cegarcia0323 at gmail.com  Mon Feb  1 09:07:22 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Mon, 1 Feb 2016 09:07:22 -0500
Subject: [Tutor] Help with printing to text file
Message-ID: <CABV0pWy7dH1nD_G-t7Y+SJqZvXgpaJFLjMuBVD2iBcNGjDhPjA@mail.gmail.com>

Hi,

So I am trying to get my function search to print  in a text file, but I
can only get it to print to Powershell. I have tried several things to get
it to print in its own text file but nothing I have tried is working. Can
someone tell me what I am doing wrong?

import csvimport sysimport jsonfrom collections import defaultdictfrom
collections import Counter
data = defaultdict(list)
mydict = defaultdict(list)
class dictionary:
	def __init__(self):
		self.dict = defaultdict(list)
		self.counted_dict = defaultdict(list)
		self.grouped_dict = defaultdict(list)
		self.other_dict = defaultdict(list)
		self.final_dict = defaultdict(list)
		self.total_dict = defaultdict(list)
		self.search_dict = defaultdict(list)
		
		
	def populate_dict(self, filename):	
		with open('weekly_test.csv', 'rb') as f:
			reader = csv.reader(f)
			next(reader, None)
			for row in reader:
				self.dict[row[2]].append(row[3])
			
	
	def total_counts(self):
		for key in self.dict.keys():
			total = 0
			b = Counter(self.dict[key])
			for value in b:
				total += b[value]
				self.total_dict.update({key: str(total)})
		
	def all_counts(self):
		data_count = Counter()
		for key in self.dict.keys():
			self.counted_dict.update({key: Counter(self.dict[key])})
			
	
	def grouped_counts(self):
		for key in self.dict.keys():
			total = 0
			c = Counter(self.dict[key])
			for value in c:
				if c[value] >= 5:
					self.grouped_dict.update({value: key + ': ' + str(c[value])})
				
				elif c[value] <= 4:
					
					total += c[value]
					self.other_dict.update({key: 'other: ' + str(total)})
	
			self.final_dict = self.grouped_dict, self.other_dict, self.total_dict			
				
	def json_output(self):
		with open ('weekly2.txt', 'w') as text_file:
			json.dump(self.final_dict, text_file, sort_keys = True, indent = 4)
			
	def search(self, filename):
		with open('weekly_test.csv', 'r') as searchfile:
			for line in searchfile:
				if 'PBI 43125' in line:
					print line
		with open('testdoc.txt', 'w') as text_file
			text_file = searchfile

From evanlespaul at gmail.com  Mon Feb  1 09:22:20 2016
From: evanlespaul at gmail.com (Evan Sommer)
Date: Mon, 1 Feb 2016 09:22:20 -0500
Subject: [Tutor] Code Regress
Message-ID: <CAPNRz81WzFPgM6=LwKtkJ-3ZhDtv6H1g5Hsmse95tz+LUDH+-A@mail.gmail.com>

Hey again Alan!

Sorry it has been so long since I have been in contact.

I tried the code suggestion that you proposed in december, and while
it did count down time, it actually set me back in a way because the
display is not how I desired it to be.

The way the display looks in this program below is how I have been
asked to run it, and all that I need to change is the fact that the
program adds 2 separate windows when it changes color. I just need one
window. That is all I am trying to change. I realize that there may be
a different command to prevent that, which is what I am trying to
achieve, but I need the display settings to remain the same.

I'm not sure how to impose this display with the code you suggested
and make the program still run without errors popping up.

So if you know how to put the display on your suggested code in the
previous comment, then I would be much obliged, or if you can impose
your changes on to this code specifically I would be very grateful.

My engineering teacher isn't even sure how to do this, so you are my
only hope Alan!

Thank you for putting up with this madness!!

Evan


try:
    # Python2
    import Tkinter as tk
except ImportError:
    # Python3
    import tkinter as tk
import time
def count_down():
    # start with 4 minutes --> 240 seconds
    for t in range(240, 120, -1):
        # format as 2 digit integers, fills with zero to the left
        # divmod() gives minutes, seconds
        sf = "{:01d}:{:02d}".format(*divmod(t, 60))
        #print(sf)  # test
        time_str.set(sf)
        root.update()
        # delay one second
        time.sleep(1)# create root/main window
root = tk.Tk()
time_str = tk.StringVar()
# create the time display label, give it a large font
# label auto-adjusts to the font
label_font = ('helvetica', 100)
tk.Label(root, textvariable=time_str, font=label_font, bg='forest green',
         fg='white', relief='raised', bd=3).pack(fill='x', padx=5, pady=5)
 # start with 2 minutes --> 119 seconds
for t in range(240, 120, -1):
        # format as 2 digit integers, fills with zero to the left
        # divmod() gives minutes, seconds
        sf = "{:01d}:{:02d}".format(*divmod(t, 60))
        #print(sf)  # test
        time_str.set(sf)
        root.update()
        # delay one second
        time.sleep(1)
# create the time display label, give it a large font
# label auto-adjusts to the font
label_font = ('helvetica', 100)
tk.Label(root, textvariable=time_str, font=label_font, bg='gold',
         fg='white', relief='raised', bd=3).pack(fill='x', padx=5, pady=5)
 # start with 1 minutes --> 59 seconds
for t in range(120,60, -1):
        # format as 2 digit integers, fills with zero to the left
        # divmod() gives minutes, seconds
        sf = "{:01d}:{:02d}".format(*divmod(t, 60))
        #print(sf)  # test
        time_str.set(sf)
        root.update()
        # delay one second
        time.sleep(1)
# create the time display label, give it a large font
# label auto-adjusts to the font
label_font = ('helvetica', 100)
tk.Label(root, textvariable=time_str, font=label_font, bg='firebrick',
         fg='white', relief='raised', bd=3).pack(fill='x', padx=5, pady=5)
 # start with 4 minutes --> 240 seconds
for t in range(60,-1, -1):
        # format as 2 digit integers, fills with zero to the left
        # divmod() gives minutes, seconds
        sf = "{:01d}:{:02d}".format(*divmod(t, 60))
        #print(sf)  # test
        time_str.set(sf)
        root.update()
        # delay one second
        time.sleep(1)
# start the GUI event loop
root.mainloop()

From alan.gauld at btinternet.com  Mon Feb  1 15:41:31 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 1 Feb 2016 20:41:31 +0000
Subject: [Tutor] Help with printing to text file
In-Reply-To: <CABV0pWy7dH1nD_G-t7Y+SJqZvXgpaJFLjMuBVD2iBcNGjDhPjA@mail.gmail.com>
References: <CABV0pWy7dH1nD_G-t7Y+SJqZvXgpaJFLjMuBVD2iBcNGjDhPjA@mail.gmail.com>
Message-ID: <n8oftr$qsb$1@ger.gmane.org>

On 01/02/16 14:07, Chelsea G wrote:
> Hi,
> 
> So I am trying to get my function search to print  in a text file, but I
> can only get it to print to Powershell. I have tried several things to get
> it to print in its own text file but nothing I have tried is working. Can
> someone tell me what I am doing wrong?
> 

> class dictionary:
> 	...		
> 	def search(self, filename):
> 		with open('weekly_test.csv', 'r') as searchfile:
> 			for line in searchfile:
> 				if 'PBI 43125' in line:
> 					print line
> 		with open('testdoc.txt', 'w') as text_file
> 			text_file = searchfile

That's because print sends its output to the standard out.
You need to store your results somewhere (maybe in a list?) and
then write() those results to a file.

Note that searchfile is only open within the first 'with' section,
it is automatically closed when you leave the with block.
So assigning searchfile to textfile is never going top work.
Even if it did, searchfile is read-only so you couldn't write to it.

So to make it work you need to

1) replace the print line with a line that appends the result
   to a list.

2) replace the textfile assignment with a loop that writes
   each entry in your list to textfile. (You may need to
   append a newline \n at the end of each line)

hth
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From joel.goldstick at gmail.com  Mon Feb  1 15:45:31 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 1 Feb 2016 15:45:31 -0500
Subject: [Tutor] Help with printing to text file
In-Reply-To: <CABV0pWy7dH1nD_G-t7Y+SJqZvXgpaJFLjMuBVD2iBcNGjDhPjA@mail.gmail.com>
References: <CABV0pWy7dH1nD_G-t7Y+SJqZvXgpaJFLjMuBVD2iBcNGjDhPjA@mail.gmail.com>
Message-ID: <CAPM-O+ydO7VxJ1AcmtzVc-CU+JU-=bbA=TEXT_jCvEBaiOiiqw@mail.gmail.com>

On Mon, Feb 1, 2016 at 9:07 AM, Chelsea G <cegarcia0323 at gmail.com> wrote:

> Hi,
>
> So I am trying to get my function search to print  in a text file, but I
> can only get it to print to Powershell. I have tried several things to get
> it to print in its own text file but nothing I have tried is working. Can
> someone tell me what I am doing wrong?
>
> import csvimport sysimport jsonfrom collections import defaultdictfrom
> collections import Counter
> data = defaultdict(list)
> mydict = defaultdict(list)
> class dictionary:
>         def __init__(self):
>                 self.dict = defaultdict(list)
>                 self.counted_dict = defaultdict(list)
>                 self.grouped_dict = defaultdict(list)
>                 self.other_dict = defaultdict(list)
>                 self.final_dict = defaultdict(list)
>                 self.total_dict = defaultdict(list)
>                 self.search_dict = defaultdict(list)
>
>
>         def populate_dict(self, filename):
>                 with open('weekly_test.csv', 'rb') as f:
>                         reader = csv.reader(f)
>                         next(reader, None)
>                         for row in reader:
>                                 self.dict[row[2]].append(row[3])
>
>
>         def total_counts(self):
>                 for key in self.dict.keys():
>                         total = 0
>                         b = Counter(self.dict[key])
>                         for value in b:
>                                 total += b[value]
>                                 self.total_dict.update({key: str(total)})
>
>         def all_counts(self):
>                 data_count = Counter()
>                 for key in self.dict.keys():
>                         self.counted_dict.update({key:
> Counter(self.dict[key])})
>
>
>         def grouped_counts(self):
>                 for key in self.dict.keys():
>                         total = 0
>                         c = Counter(self.dict[key])
>                         for value in c:
>                                 if c[value] >= 5:
>                                         self.grouped_dict.update({value:
> key + ': ' + str(c[value])})
>
>                                 elif c[value] <= 4:
>
>                                         total += c[value]
>                                         self.other_dict.update({key:
> 'other: ' + str(total)})
>
>                         self.final_dict = self.grouped_dict,
> self.other_dict, self.total_dict
>
>         def json_output(self):
>                 with open ('weekly2.txt', 'w') as text_file:
>                         json.dump(self.final_dict, text_file, sort_keys =
> True, indent = 4)
>
>         def search(self, filename):
>                 with open('weekly_test.csv', 'r') as searchfile:
>                         for line in searchfile:
>                                 if 'PBI 43125' in line:
>                                         print line
>
i'm guessing above prints

>                 with open('testdoc.txt', 'w') as text_file
>                         text_file = searchfile
>

try something like this:

>         def search(self, filename):
>                 with open('weekly_test.csv', 'r') as searchfile:
>                     with open('testdoc.txt', 'w') as text_file
>                         for line in searchfile:
>                                 if 'PBI 43125' in line:
>                                         text_file.write(line + '\n'

_______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From alan.gauld at btinternet.com  Mon Feb  1 15:47:46 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 1 Feb 2016 20:47:46 +0000
Subject: [Tutor] Help with printing to text file
In-Reply-To: <CABV0pWy7dH1nD_G-t7Y+SJqZvXgpaJFLjMuBVD2iBcNGjDhPjA@mail.gmail.com>
References: <CABV0pWy7dH1nD_G-t7Y+SJqZvXgpaJFLjMuBVD2iBcNGjDhPjA@mail.gmail.com>
Message-ID: <n8og9i$1v9$1@ger.gmane.org>

On 01/02/16 14:07, Chelsea G wrote:

> So I am trying to get my function search to print  in a text file, but I
> can only get it to print to Powershell. I have tried several things to get
> it to print in its own text file but nothing I have tried is working. Can
> someone tell me what I am doing wrong?

I meant to add in my other reply that you could combine the
reading and writing into a single 'with' block and thus avoid
creating the intermediate list. That could save important
memory if searchfile is a big file.

You can open 2 files in a single with statement:

with open('foo.txt','r') as infile, open ('bar.txt','w') as outfile:
    for line in infile:
        if searchstring in line:
            outfile.write(line)


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From cs at zip.com.au  Mon Feb  1 18:28:58 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 2 Feb 2016 10:28:58 +1100
Subject: [Tutor] Help with printing to text file
In-Reply-To: <n8oftr$qsb$1@ger.gmane.org>
References: <n8oftr$qsb$1@ger.gmane.org>
Message-ID: <20160201232858.GA97975@cskk.homeip.net>

On 01Feb2016 20:41, ALAN GAULD <alan.gauld at btinternet.com> wrote:
>On 01/02/16 14:07, Chelsea G wrote:
>> So I am trying to get my function search to print  in a text file, but I
>> can only get it to print to Powershell. I have tried several things to get
>> it to print in its own text file but nothing I have tried is working. Can
>> someone tell me what I am doing wrong?
>>
>
>> class dictionary:
>> 	...		
>> 	def search(self, filename):
>> 		with open('weekly_test.csv', 'r') as searchfile:
>> 			for line in searchfile:
>> 				if 'PBI 43125' in line:
>> 					print line
>> 		with open('testdoc.txt', 'w') as text_file
>> 			text_file = searchfile
>
>That's because print sends its output to the standard out.
>You need to store your results somewhere (maybe in a list?) and
>then write() those results to a file.

Or, of course, open the file and tell print to write there:

  with open('weekly_test.csv', 'r') as searchfile:
    with open('testdoc.txt', 'w') as text_file:
      ......
        print(line, file=text_file)

It looks like you're using python 2. To use the modern print syntax above you 
need:

  from __future__ import print_function

at the top of your python program. Python 3 uses the "function" form of print, 
and you need the above to tell python 2 to do so.

One advantage of Alan's "put it in a list" approach is that you could separate 
the seaching of the CSV into one function returning a list, and then do 
whatever you want (eg print it to your file) as a wholy separate operation.

Cheers,
Cameron Simpson <cs at zip.com.au>

From steve at pearwood.info  Mon Feb  1 18:46:52 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 2 Feb 2016 10:46:52 +1100
Subject: [Tutor] Fwd: Finding the max value from a dictionary that does
 not exceed a variable's value.
In-Reply-To: <CAGZAPF4LWBe=cb2xcyP2VLd2mKZJr_v2QmoTEcyEfM7T6nvQ+w@mail.gmail.com>
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
 <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
 <CACs7g=DhUdO98BwKQTnT3b4L386xtDXzVWPskxD+5mKpQC61_Q@mail.gmail.com>
 <CAGZAPF7t+7qgwNYU8LX8+NtCUdOni0__MW2bOO+3PKZ-zt_z-g@mail.gmail.com>
 <CAGZAPF4LWBe=cb2xcyP2VLd2mKZJr_v2QmoTEcyEfM7T6nvQ+w@mail.gmail.com>
Message-ID: <20160201234652.GH31806@ando.pearwood.info>

On Mon, Feb 01, 2016 at 12:00:47PM -0800, Danny Yoo wrote:
> Here's a response I sent to Srinivas yesterday to further explain why
> a balanced binary tree is probably overkill for the "largest
> denomination" selection problem.  (I didn't realize that I had not
> sent the following to the list.)
[...]
> So there are several crazy avenues we can take to over-optimize this
> problem.  Just to make it clear: I think sticking to a simple linear
> scan makes the most sense.  Everything else just seems to try to make
> the problem harder than it deserves to be, akin to trying to take the
> size of a rectangle via integration.
> 
> http://homepage.usask.ca/~blb230/Math_Comics/Calculus_Comic_files/image001.gif

I'm glad you've forwarded the message to the list, because I love that 
comic. The clever thing is that it actually gets the maths right too. 
Some of the notation is a bit strange compared to what I'm used to 
(I've never seen anyone use a bare integral sign before, with no 
integrand), and I think he skipped a line, but that's definitely one to 
keep.


Thanks,



-- 
Steve

From steve at pearwood.info  Mon Feb  1 19:45:32 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 2 Feb 2016 11:45:32 +1100
Subject: [Tutor] Help with printing to text file
In-Reply-To: <n8oftr$qsb$1@ger.gmane.org>
References: <CABV0pWy7dH1nD_G-t7Y+SJqZvXgpaJFLjMuBVD2iBcNGjDhPjA@mail.gmail.com>
 <n8oftr$qsb$1@ger.gmane.org>
Message-ID: <20160202004532.GI31806@ando.pearwood.info>

On Mon, Feb 01, 2016 at 08:41:31PM +0000, Alan Gauld wrote:
> On 01/02/16 14:07, Chelsea G wrote:
> > Hi,
> > 
> > So I am trying to get my function search to print  in a text file, but I
> > can only get it to print to Powershell. I have tried several things to get
> > it to print in its own text file but nothing I have tried is working. Can
> > someone tell me what I am doing wrong?

> That's because print sends its output to the standard out.
> You need to store your results somewhere (maybe in a list?) and
> then write() those results to a file.

You don't even need to do that! print has a secret (well, not really a 
secret, but you would be amazed how few people know about it) option to 
print directly to an open file.

In Python 3 you write:

    print("Hello World!", file=output_file)

but in Python 2 you must use this ugly syntax instead:

    print >>output_file, "Hello World!"

output_file must be already opened for writing, of course.

So Chelsea's class would become something like this:


class dictionary:
	...
	def search(self, filename):
		with open('weekly_test.csv', 'r') as searchfile, open('testdoc.txt', 'w') as text_file:
			for line in searchfile:
				if 'PBI 43125' in line:
					print >>text_file, line



By the way, the argument "filename" is not used here. Is that 
intentional?


But perhaps an even better solution is to use the environment's file 
redirection. Powershell should understand > to mean "print to a file", 
so you can write:

    python myscript.py 

to have myscript print output directly to the terminal window, and then:

    python myscript.py > testdoc.txt

to redirect the output and write it to testdoc.txt instead. 



-- 
Steve

From __peter__ at web.de  Mon Feb  1 20:59:40 2016
From: __peter__ at web.de (Peter Otten)
Date: Tue, 02 Feb 2016 02:59:40 +0100
Subject: [Tutor] [OT] Calculus comic,
 was Re: Fwd: Finding the max value from a dictionary that does not
 exceed a variable's value.
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
 <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
 <CACs7g=DhUdO98BwKQTnT3b4L386xtDXzVWPskxD+5mKpQC61_Q@mail.gmail.com>
 <CAGZAPF7t+7qgwNYU8LX8+NtCUdOni0__MW2bOO+3PKZ-zt_z-g@mail.gmail.com>
 <CAGZAPF4LWBe=cb2xcyP2VLd2mKZJr_v2QmoTEcyEfM7T6nvQ+w@mail.gmail.com>
 <20160201234652.GH31806@ando.pearwood.info>
Message-ID: <n8p2ic$b3r$1@ger.gmane.org>

Steven D'Aprano wrote:

> On Mon, Feb 01, 2016 at 12:00:47PM -0800, Danny Yoo wrote:
>> Here's a response I sent to Srinivas yesterday to further explain why
>> a balanced binary tree is probably overkill for the "largest
>> denomination" selection problem.  (I didn't realize that I had not
>> sent the following to the list.)
> [...]
>> So there are several crazy avenues we can take to over-optimize this
>> problem.  Just to make it clear: I think sticking to a simple linear
>> scan makes the most sense.  Everything else just seems to try to make
>> the problem harder than it deserves to be, akin to trying to take the
>> size of a rectangle via integration.
>> 
>> 
http://homepage.usask.ca/~blb230/Math_Comics/Calculus_Comic_files/image001.gif
> 
> I'm glad you've forwarded the message to the list, because I love that
> comic. The clever thing is that it actually gets the maths right too.
> Some of the notation is a bit strange compared to what I'm used to
> (I've never seen anyone use a bare integral sign before, with no
> integrand), 

That's not a bare integral sign, that's a vertical bar as in formula (12) of

http://www.mathe-online.at/mathint/int/i.html

The page is in German, sorry; the operator seems to be called "evaluated at" 
in English.

> and I think he skipped a line, but that's definitely one to
> keep.



From robertvstepp at gmail.com  Tue Feb  2 00:48:17 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Mon, 1 Feb 2016 23:48:17 -0600
Subject: [Tutor] Code Regress
In-Reply-To: <CAPNRz81WzFPgM6=LwKtkJ-3ZhDtv6H1g5Hsmse95tz+LUDH+-A@mail.gmail.com>
References: <CAPNRz81WzFPgM6=LwKtkJ-3ZhDtv6H1g5Hsmse95tz+LUDH+-A@mail.gmail.com>
Message-ID: <CANDiX9KVNMYyq6CsDgatVNy0Y6LhEs3KjfkcFu2WDLWuU94O5Q@mail.gmail.com>

On Mon, Feb 1, 2016 at 8:22 AM, Evan Sommer <evanlespaul at gmail.com> wrote:

> I tried the code suggestion that you proposed in december, and while
> it did count down time, it actually set me back in a way because the
> display is not how I desired it to be.

I think you need to review all of Alan's previous suggestions.  If you
do not understand what he said, then you should ask a specific
question so that we can help you understand.

> The way the display looks in this program below is how I have been
> asked to run it, and all that I need to change is the fact that the
> program adds 2 separate windows when it changes color. I just need one
> window...

Each time you call tk.Lbl, you are creating another label.  Alan
suggested that you update the background property directly.  You can
do this using the config() method.  Look it up in the docs!  A start
in this direction might be:

your_label = tk.Label(root, ...)
your_label.pack(...)

And when you want to change the background color, use:

your_label.config(bg='whatever_color')

This will change that particular property because the Label object
supports the config() method.  This method is used *after* applying
pack to create your label.

The code you supplied did not have any vertical spacing and was rather
hard to read.  You should separate logical blocks of code with a blank
line (or possibly two).  It makes it much easier to read!

As far as I can tell you never call your function, count_down, and
essentially duplicate what it does in your main code.

Why don't you try putting everything in a single for loop and test
when it is time to change colors with an if-elif construct?  A
skeleton might be:

for t in range(240, -1, -1):
    if t == 120:
        your_label.config(bg='gold')

    elif t == 60:
        your_label.config(bg='firebrick')
    .
    .
    .

HTH!

boB

From cegarcia0323 at gmail.com  Tue Feb  2 10:37:48 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Tue, 2 Feb 2016 10:37:48 -0500
Subject: [Tutor] Help with error
Message-ID: <CABV0pWysrGqHFmmVe5GUcSKi0rcKsh=aPVxYGCKe-UB6q8ufYA@mail.gmail.com>

Hi,
When I run my code I am getting an error it says "Attribute Error:
Dictionary instance has no attribute 'search'. So the whole idea for my
code is to input a csv file and output results that we want. This specific
piece of code def search is to search the csv file for any keyword like
'issues' and output those results into a text file. I just need some help
with figuring out how to fix the error I am receiving.
This is my functions:

import csvimport jsonimport sysfrom collections import defaultdictfrom
collections import Counter
data = defaultdict(list)

upper_limit = 5
lower_limit = 4
class dictionary():
    def __init__(self):
        self.dict = defaultdict(list)
        self.counted_dict = defaultdict(list)
        self.grouped_dict = defaultdict(list)
        self.other_dict = defaultdict(list)
        self.final_dict = defaultdict(list)
        self.total_dict = defaultdict(list)
        self.search_dict = defaultdict(list)

    def populate_dict(self, filename):
        with open(filename, 'rb') as f:
            reader = csv.reader(f)
            next(reader, None)
            for row in reader:
                self.dict[row[2]].append(row[3])

    def all_counts(self):
        data_count = Counter()
        for key in self.dict.keys():
            self.counted_dict.update({key: Counter(self.dict[key])})

    def total_counts(self):
        for key in self.dict.keys():
            total = 0
            b = Counter(self.dict[key])
            for value in b:
                total += b[value]
                new_list = str(total)
                #self.total_dict.update({key: 'Total count for this
application: ' + str(total)})
                #self.total_dict.update({key: str(total)})
            self.total_dict[key].append(new_list)

    def grouped_counts(self):
        for key in self.dict.keys():
            total = 0
            c = Counter(self.dict[key])
            for value in c:
                if c[value] >= upper_limit:
                    new_list = value, str(c[value])
                    self.grouped_dict[key].append(new_list)
                elif c[value] <= lower_limit:
                    total += c[value]
                    self.other_dict.update({key: 'other: ' + str(total)})

        for d in (self.grouped_dict, self.other_dict, self.total_dict):
            for key, value in d.iteritems():
                self.final_dict[key].append(value)

    def json_output(self):
        with open('test.txt', 'w') as text_file:
            json.dump(self.final_dict, text_file, sort_keys = True, indent = 4)
			
	def search(self, filename):
		with open(filename, 'r') as searchfile, open('weekly_test.txt', 'w')
as search_results_file:
			for line in searchfile:
				if 'PBI 43125' in line:
					print >>search_results_file, line



And then I have another python file where I import my functions and
run the results. Here is the code for that:

import functions2

week_choice = raw_input("Previous Week or Current Week?")if
week_choice == "current week":
	data = functions2.dictionary()
	filename = raw_input("What file do you want to use?")
	data.populate_dict(filename)
	data.total_counts()
	data.grouped_counts()
	data.json_output()
	data.search(filename)
	elif week_choice == "previous week":
	previous_data = functions2.dictionary()
	filename = raw_input("What file do you want to use?")
	previous_data.populate_dict(filename)
	previous_data.total_counts()
	previous_data.grouped_counts()
	previous_data.json_output()
	previous_data.search(filename)
	else:
	print "That choice is not correct"


It says the error is with data.search(filename).. Not sure how to get
this working.

From knnleow at gmail.com  Tue Feb  2 07:13:31 2016
From: knnleow at gmail.com (knnleow GOOGLE)
Date: Tue, 2 Feb 2016 20:13:31 +0800
Subject: [Tutor] 2016-02-01 Filter STRINGS in Log File and Pass as
 VARAIBLE within PYTHON script
In-Reply-To: <20160201105041.GA97843@cskk.homeip.net>
References: <56AF0EF1.70906@gmail.com> <20160201105041.GA97843@cskk.homeip.net>
Message-ID: <56B09D6B.5060701@gmail.com>

hello Cameron,

thank you for the positive input. this is my new code.

NEW CODE
----------------
$ more fail2ban-banned-ipAddress.py
#VERSION CONTROL:
#2016-01-31  - Initial build by Kuenn Leow
#            - fail2ban package has to be installed
#            - fail2ban leverage on linux iptables to work
#2016-02-02 - modified with recommendation from Cameron Simpson
#
#FIXED MODULE IMPORT and FIXED ARGV IMPORT
import sys
import os
import subprocess
import time
import traceback

myArray = sys.argv

def checkInputs():
         if('-date' not in myArray):
                 #print(__doc__)
                 print('''

USAGE:    python fail2ban-banned-ipAddress.py -date <YYYY-MM-DD>
EXAMPLE:  python fail2ban-banned-ipAddress.py -date 2016-01-31
                 ''')
                 sys.exit(1)

def main():
         #START MAIN PROGRAM HERE!!!
         try:
                 checkInputs()
                 myDate = myArray[myArray.index('-date') + 1]
                 timestamp01 = time.strftime("%Y-%m-%d")
                 timestamp02 = time.strftime("%Y-%m-%d-%H%M%S")
                 wd01 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" + 
timestamp01)
                 wd02 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" + 
timestamp02)

                 #print(" ")
                 #print(40 * "-")
                 #print("START DEBUG Log of MAIN Defined VARIABLE")
                 #print(40 * "-")
                 #print("myDate: " + myDate)
                 #print(" ")
                 #print("timestamp01: " + timestamp01)
                 #print("timestamp01: " + timestamp02)
                 #print(" ")
                 #print("wd01: " + wd01)
                 #print("wd02: " + wd02)
                 #print(38 * "-")
                 #print("END DEBUG Log of MAIN Defined VARIABLE")
                 #print(38 * "-")
                 #print(" ")

                 print(" ")
                 with open("/var/log/fail2ban.log") as fail_log:
                         for line in fail_log:
                                 if("ssh" in line and "Ban" in line and 
myDate in line):
                                         words = line.split()
                                         banIP = words[6]
                                         print("banIP:" , banIP)
                                         whoisFile = os.popen("whois -H 
" + banIP + " |egrep -i \"name|country|mail\" |sort -u").read()
                                         print("whoisFile:", whoisFile)
         except KeyboardInterrupt:
                 print('Shutdown requested...exiting')
         except Exception:
                 traceback.print_exc(file=sys.stdout)
         sys.exit(0)
         #END MAIN PROGRAM HERE!!!

#START RUN main program/functions HERE!!!
if __name__ == "__main__":
         main()
#END RUN main program/functions HERE!!!





TEST RESULT:
-------------------

$ python ./fail2ban-banned-ipAddress.py  -date 2016-01-31

banIP: 183.3.202.109
whoisFile: abuse-mailbox:  anti-spam at ns.chinanet.cn.net
abuse-mailbox:  antispam_gdnoc at 189.cn
country:        CN
e-mail:         anti-spam at ns.chinanet.cn.net
e-mail:         gdnoc_HLWI at 189.cn
netname:        CHINANET-GD

banIP: 183.3.202.109
whoisFile: abuse-mailbox:  anti-spam at ns.chinanet.cn.net
abuse-mailbox:  antispam_gdnoc at 189.cn
country:        CN
e-mail:         anti-spam at ns.chinanet.cn.net
e-mail:         gdnoc_HLWI at 189.cn
netname:        CHINANET-GD

banIP: 27.75.97.233
whoisFile: abuse-mailbox:  hm-changed at vnnic.net.vn
country:        VN
e-mail:         hm-changed at vnnic.net.vn
e-mail:         tiennd at viettel.com.vn
e-mail:         truongpd at viettel.com.vn
netname:        Newass2011xDSLHN-NET
remarks:        For spamming matters, mail to tiennd at viettel.com.vn

banIP: 183.3.202.109
whoisFile: abuse-mailbox:  anti-spam at ns.chinanet.cn.net
abuse-mailbox:  antispam_gdnoc at 189.cn
country:        CN
e-mail:         anti-spam at ns.chinanet.cn.net
e-mail:         gdnoc_HLWI at 189.cn
netname:        CHINANET-GD

Cameron Simpson wrote:
> On 01Feb2016 15:53, knnleow GOOGLE <knnleow at gmail.com> wrote:
>> trying out on how to port my unix shell script to python.
>> get more complicated than i expected.....: (
>> i am not familiar with the modules available in python.
>> anyone care to share how to better the clumsy approach below.
>> regards,
>> kuenn
>>
>>                timestamp02 = time.strftime("%Y-%m-%d-%H%M%S")
>>                banIPaddressesFile = os.popen("cat 
>> /var/log/fail2ban.log| egrep ssh| egrep Ban| egrep " + myDate + "| 
>> awk \'{print $7}\'| sort -n| uniq >/tmp/banIPaddressesFile." + 
>> timestamp02).read()
>
> First up, this is still essentially a shell script. You're 
> constructing a shell pipeline like this (paraphrased):
>
>  cat >/var/log/fail2ban.log
>  | egrep ssh
>  | egrep Ban
>  | egrep myDate
>  | awk '{print $7}'
>  | sort -n
>  | uniq  >/tmp/banIPaddressesFile-timestamp
>
> So really, you're doing almost nothing in Python. You're also writing 
> intermediate results to a temporary filename, then reading from it. 
> Unless you really need to keep that file around, you won't need that 
> either.
>
> Before I get into the Python side of things, there are a few small 
> (small) criticisms of your shell script:
>
> - it has a "useless cat"; this is a very common shell inefficiency 
> there people  put "cat filename | filter1 | filter2 ..." when they 
> could more cleanly just  go "filter1 <filename | filter2 | ..."
>
> - you are searching for fixed strings; why are you using egrep? Just 
> say "grep"  (or even "fgrep" if you're old school - you're new to this 
> so I presume not)
>
> - you're using "sort -n | uniq", presumably because uniq requires 
> sorted input;  you are better off using "sort -un" here and skipping 
> uniq. I'd also point  out that since these are IP addresses, "sort -n" 
> doesn't really do what you  want here.
>
> So, to the Python:
>
> You seem to want to read the file /var/log/fail2ban.log and for 
> certain specific lines, record column 7 which I gather from the rest 
> of the code (below) is an IP address. I gather you just want one copy 
> of each unique IP address.
>
> So, to read lines of the file the standard idom goes:
>
>  with open('/var/log/fail2ban.log') as fail_log:
>    for line in fail_log:
>      ... process lines here ...
>
> You seem to be checking for two keywords and a date in the interesting 
> lines.  You can do this with a simple test:
>
>  if 'ssh' in line and 'Ban' in line and myDate in line:
>
> If you want the seventh column from the line (per your awk command) 
> you can get it like this:
>
>  words = line.split()
>  word7 = words[6]
>
> because Python arrays count form 0, therefore index 6 is the seventh 
> word.
>
> You want the unique IP addresses, so I suggest storing them all in a 
> set and not bothering with a sort until some other time. So make an 
> empty set before you read the file:
>
>  ip_addrs = set()
>
> and add each address to it for the lines you select:
>
>  ip_addrs.add(word7)
>
> After you have read the whole file you will have the desired addresses 
> in the ip_addrs set.
>
> Try to put all that together and come back with working code, or come 
> back with completed but not working code and specific questions.
>
> Cheers,
> Cameron Simpson <cs at zip.com.au>


From knnleow at gmail.com  Tue Feb  2 08:14:43 2016
From: knnleow at gmail.com (knnleow GOOGLE)
Date: Tue, 2 Feb 2016 21:14:43 +0800
Subject: [Tutor] 2016-02-01 Filter STRINGS in Log File and Pass as
 VARAIBLE within PYTHON script
In-Reply-To: <56B09D6B.5060701@gmail.com>
References: <56AF0EF1.70906@gmail.com>
 <20160201105041.GA97843@cskk.homeip.net> <56B09D6B.5060701@gmail.com>
Message-ID: <56B0ABC3.3000605@gmail.com>

Sorry, forget to make use of SET() ....... this is the new update.....
appreciate your advice if we can still optimized further...

$ more fail2ban-banned-ipAddress.py
#VERSION CONTROL:
#2016-01-31   - Initial build by Kuenn Leow
#                     - fail2ban package has to be installed
#                     - fail2ban leverage on linux iptables to work
#2016-0-02    - modified with recommandation from Carmeron Simpson

#FIXED MODULE IMPORT and FIXED ARGV IMPORT
import sys
import os
import subprocess
import time
import traceback

myArray = sys.argv

def checkInputs():
         if('-date' not in myArray):
                 #print(__doc__)
                 print('''

USAGE:    python fail2ban-banned-ipAddress.py -date <YYYY-MM-DD>
EXAMPLE:  python fail2ban-banned-ipAddress.py -date 2016-01-31
                 ''')
                 sys.exit(1)

def main():
         #START MAIN PROGRAM HERE!!!
         try:
                 checkInputs()
                 myDate = myArray[myArray.index('-date') + 1]
                 timestamp01 = time.strftime("%Y-%m-%d")
                 timestamp02 = time.strftime("%Y-%m-%d-%H%M%S")
                 wd01 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" + 
timestamp01)
                 wd02 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" + 
timestamp02)

                 #print(" ")
                 #print(40 * "-")
                 #print("START DEBUG Log of MAIN Defined VARIABLE")
                 #print(40 * "-")
                 #print("myDate: " + myDate)
                 #print(" ")
                 #print("timestamp01: " + timestamp01)
                 #print("timestamp02: " + timestamp02)
                 #print(" ")
                 #print("wd01: " + wd01)
                 #print("wd02: " + wd02)
                 #print(38 * "-")
                 #print("END DEBUG Log of MAIN Defined VARIABLE")
                 #print(38 * "-")
                 #print(" ")

                 # store all the BANNED IP in a SET
                 print(" ")
                 banIP_addrs = set()
                 with open("/var/log/fail2ban.log") as fail_log:
                         for line in fail_log:
                                 if("ssh" in line and "Ban" in line and 
myDate in line):
                                         words = line.split()
                                         word6 = words[6]
                                         print("word6:" , word6)
                                         banIP_addrs.add(word6)
                 print("banIP_addrs:" , banIP_addrs)

                 # LOOP through the SET and WHOIS
                 for i in banIP_addrs:
                         print("i:", i)
                         whoisVAR = os.popen("whois -H " + i + " |egrep 
-i \"name|country|mail\" |sort -u").read()
                         print("whoisVAR:", whoisVAR)

         except KeyboardInterrupt:
                 print('Shutdown requested...exiting')
         except Exception:
                 traceback.print_exc(file=sys.stdout)
         sys.exit(0)
         #END MAIN PROGRAM HERE!!!

#START RUN main program/functions HERE!!!
if __name__ == "__main__":
         main()
#END RUN main program/functions HERE!!!

TEST RESULT
-------------------
$ python ./fail2ban-banned-ipAddress.py  -date 2016-01-31

word6: 183.3.202.109
word6: 183.3.202.109
word6: 27.75.97.233
word6: 183.3.202.109
word6: 222.187.222.220
word6: 183.3.202.109
word6: 77.73.91.28
word6: 59.47.0.149
word6: 183.3.202.109
word6: 77.73.91.28
word6: 77.73.91.28
word6: 185.130.5.184
word6: 183.3.202.109
word6: 221.203.142.71
banIP_addrs: {'183.3.202.109', '59.47.0.149', '222.187.222.220', 
'77.73.91.28', '27.75.97.233', '221.203.142.71', '185.130.5.184'}
i: 183.3.202.109
whoisVAR: abuse-mailbox:  anti-spam at ns.chinanet.cn.net
abuse-mailbox:  antispam_gdnoc at 189.cn
country:        CN
e-mail:         anti-spam at ns.chinanet.cn.net
e-mail:         gdnoc_HLWI at 189.cn
netname:        CHINANET-GD

i: 59.47.0.149
whoisVAR: abuse-mailbox:  anti-spam at ns.chinanet.cn.net
country:        CN
e-mail:         anti-spam at ns.chinanet.cn.net
e-mail:         lnabuse at lntele.com
netname:        CHINANET-LN


From pedro.miguel at live.co.uk  Tue Feb  2 05:46:04 2016
From: pedro.miguel at live.co.uk (Pedro Miguel)
Date: Tue, 2 Feb 2016 10:46:04 +0000
Subject: [Tutor] How to write tests for main() function that does not return
 anything
Message-ID: <DUB111-W79627DF6E4B0404F741CB0A3DF0@phx.gbl>

Hi guys, I'm trying to test the code in the main() but I'm a bit unsure how to go about it since I'm not passing any arguments or even returning anything other then logging. For the purposes of the example I've shortened the tree statements in the function..
Can anyone point me in the right direction on how to test the logic below? Or perhaps give me an example on how to test the code below (that would be really appreciated). 
I've posted this question on Stackoverflow but the guys over there told me to mock it but no one provided an example (I'm fairly new on mocking)..and how could I mock the if statements or even the for loop?
script.py

from . import settings


def main():
    if settings.PATHS:  # list containing full paths to a directory of files
        paths = settings.PATHS
        for path in paths:
            data = read_file(path)
            modified_data = do_something_with_the_data_collected(data)
            write_to_new_file(modified_data)
    else:
        logger.warning("There are no files in {}".format(settings.FILES_DIRECTORY))

if __name__ == '__main__':
    main()
tests/file_tests.py
import unittest

from module.script import main


class FileManagerTests(unittest.TestCase):

    def test_main_func(self):
        main()  # ?? this is where I am stuck, should I just test 
                # that it logs correctly if certain data exists 
                # in settings file?

if __name__ == '__main__':
    unittest.main()

RegardsPedro+44(0)7549646235pedro.miguel at live.co.uk 		 	   		  

From joel.goldstick at gmail.com  Tue Feb  2 13:07:42 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 2 Feb 2016 13:07:42 -0500
Subject: [Tutor] Help with error
In-Reply-To: <CABV0pWysrGqHFmmVe5GUcSKi0rcKsh=aPVxYGCKe-UB6q8ufYA@mail.gmail.com>
References: <CABV0pWysrGqHFmmVe5GUcSKi0rcKsh=aPVxYGCKe-UB6q8ufYA@mail.gmail.com>
Message-ID: <CAPM-O+y4hy7ozX2b3SoxOVMsG1p0d3Crm2eJEAHxck-K3=_aFg@mail.gmail.com>

First, Please try to be more creative with your message subject line.
Help, isn't -- helpfull

Second.  Don't paraphrase the error, copy it from your actual terminal
screen and paste it here.

See below

On Tue, Feb 2, 2016 at 10:37 AM, Chelsea G <cegarcia0323 at gmail.com> wrote:

> Hi,
> When I run my code I am getting an error it says "Attribute Error:
> Dictionary instance has no attribute 'search'. So the whole idea for my
> code is to input a csv file and output results that we want. This specific
> piece of code def search is to search the csv file for any keyword like
> 'issues' and output those results into a text file. I just need some help
> with figuring out how to fix the error I am receiving.
> This is my functions:
>
> import csvimport jsonimport sysfrom collections import defaultdictfrom
> collections import Counter
> data = defaultdict(list)
>
> upper_limit = 5
> lower_limit = 4
> class dictionary():
>     def __init__(self):
>         self.dict = defaultdict(list)
>         self.counted_dict = defaultdict(list)
>         self.grouped_dict = defaultdict(list)
>         self.other_dict = defaultdict(list)
>         self.final_dict = defaultdict(list)
>         self.total_dict = defaultdict(list)
>         self.search_dict = defaultdict(list)
>
>     def populate_dict(self, filename):
>         with open(filename, 'rb') as f:
>             reader = csv.reader(f)
>             next(reader, None)
>             for row in reader:
>                 self.dict[row[2]].append(row[3])
>
>     def all_counts(self):
>         data_count = Counter()
>         for key in self.dict.keys():
>             self.counted_dict.update({key: Counter(self.dict[key])})
>
>     def total_counts(self):
>         for key in self.dict.keys():
>             total = 0
>             b = Counter(self.dict[key])
>             for value in b:
>                 total += b[value]
>                 new_list = str(total)
>                 #self.total_dict.update({key: 'Total count for this
> application: ' + str(total)})
>                 #self.total_dict.update({key: str(total)})
>             self.total_dict[key].append(new_list)
>
>     def grouped_counts(self):
>         for key in self.dict.keys():
>             total = 0
>             c = Counter(self.dict[key])
>             for value in c:
>                 if c[value] >= upper_limit:
>                     new_list = value, str(c[value])
>                     self.grouped_dict[key].append(new_list)
>                 elif c[value] <= lower_limit:
>                     total += c[value]
>                     self.other_dict.update({key: 'other: ' + str(total)})
>
>         for d in (self.grouped_dict, self.other_dict, self.total_dict):
>             for key, value in d.iteritems():
>                 self.final_dict[key].append(value)
>
>     def json_output(self):
>         with open('test.txt', 'w') as text_file:
>             json.dump(self.final_dict, text_file, sort_keys = True, indent
> = 4)
>
> Your search method below is indented so as to be a function within
json_output.  Remove the indentation so that it lines up with your other
class methods.


>         def search(self, filename):
>                 with open(filename, 'r') as searchfile,
> open('weekly_test.txt', 'w')
> as search_results_file:
>                         for line in searchfile:
>                                 if 'PBI 43125' in line:
>                                         print >>search_results_file, line
>
>
>
> And then I have another python file where I import my functions and
> run the results. Here is the code for that:
>
> import functions2
>
> week_choice = raw_input("Previous Week or Current Week?")if
> week_choice == "current week":
>         data = functions2.dictionary()
>         filename = raw_input("What file do you want to use?")
>         data.populate_dict(filename)
>         data.total_counts()
>         data.grouped_counts()
>         data.json_output()
>         data.search(filename)
>         elif week_choice == "previous week":
>         previous_data = functions2.dictionary()
>         filename = raw_input("What file do you want to use?")
>         previous_data.populate_dict(filename)
>         previous_data.total_counts()
>         previous_data.grouped_counts()
>         previous_data.json_output()
>         previous_data.search(filename)
>         else:
>         print "That choice is not correct"
>
>
> It says the error is with data.search(filename).. Not sure how to get
> this working.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From alan.gauld at btinternet.com  Tue Feb  2 16:21:59 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 2 Feb 2016 21:21:59 +0000
Subject: [Tutor] Help with error
In-Reply-To: <CABV0pWysrGqHFmmVe5GUcSKi0rcKsh=aPVxYGCKe-UB6q8ufYA@mail.gmail.com>
References: <CABV0pWysrGqHFmmVe5GUcSKi0rcKsh=aPVxYGCKe-UB6q8ufYA@mail.gmail.com>
Message-ID: <n8r6ln$o75$1@ger.gmane.org>

On 02/02/16 15:37, Chelsea G wrote:

> 'issues' and output those results into a text file. I just need some help
> with figuring out how to fix the error I am receiving.

Joel has already pointed out the indentation error in your class.

However there is another problem... but possibly not with your code.

> And then I have another python file where I import my functions and
> run the results. Here is the code for that:
> 
> import functions2
> 
> week_choice = raw_input("Previous Week or Current Week?")if
> week_choice == "current week":
> 	data = functions2.dictionary()
> 	filename = raw_input("What file do you want to use?")
> 	data.populate_dict(filename)
> 	data.total_counts()
> 	data.grouped_counts()
> 	data.json_output()
> 	data.search(filename)
> 	elif week_choice == "previous week":
> 	previous_data = functions2.dictionary()
> 	filename = raw_input("What file do you want to use?")
> 	previous_data.populate_dict(filename)
> 	previous_data.total_counts()
> 	previous_data.grouped_counts()
> 	previous_data.json_output()
> 	previous_data.search(filename)
> 	else:
> 	print "That choice is not correct"

The if, elif and else indentation is way off
Now that is probably due to emails being messed up,
but it makes it hard to give sensible feedback since
there could be bugs but we can't see them.sensibly...

Please be sure to send the complete error message
and not just summarize it and also use plain text,
especially when posting code.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From dyoo at hashcollision.org  Tue Feb  2 16:23:07 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Tue, 2 Feb 2016 13:23:07 -0800
Subject: [Tutor] Fwd: RE: Finding the max value from a dictionary that does
 not exceed a variable's value.
In-Reply-To: <CAGZAPF6PTHz5-JX4mHMB3-m-jhPqw67vJGpamc3XMKWGDJsY2w@mail.gmail.com>
References: <DUB113-W680874C1873491B353271AE6DC0@phx.gbl>
 <CAGZAPF61QaSHV8gCQ5h=-ZwKgVqWE9uaZQuFYGMwWuJ7DYe26A@mail.gmail.com>
 <CAGZAPF5H56gXkvJ1hUQ0Wp87O9ZeMLo8xxuFZdbBHVEUKy2bOw@mail.gmail.com>
 <DUB113-W1171E5AA32E289836777A99E6DF0@phx.gbl>
 <CAGZAPF6PTHz5-JX4mHMB3-m-jhPqw67vJGpamc3XMKWGDJsY2w@mail.gmail.com>
Message-ID: <CAGZAPF4cw8V1Hy8cp45i_kSACOyrRk+trw9rCnzV7a+P1613Xw@mail.gmail.com>

---------- Forwarded message ----------
From: "Danny Yoo" <danny.yoo at gmail.com>
Date: Feb 2, 2016 1:22 PM
Subject: RE: [Tutor] Finding the max value from a dictionary that does not
exceed a variable's value.
To: "Lawrence Lorenzo" <mcshizney at hotmail.co.uk>
Cc:


On Feb 2, 2016 10:34 AM, "Lawrence Lorenzo" <mcshizney at hotmail.co.uk> wrote:
>
>
> I can do it with a list, however I need to be able to print the change
amount in coin names. Thanks for your reply sorry I didn't respond earlier
I didn't see it until just now.
> Just to expand on what I'm trying with this is to print an arbitrary
value as 1p, 2p, 5p, 10p, 20p, 50p, ?1 in the least amount required so if
the change needed is ?2.37 then the program prints that much as ("?1, ?1,
20p, 10p, 5p, 2p")

Consider separating the process of computing the answer from presentation
of that answer.

That is: just because you have to print or the coin names at the end
doesn't mean you have to *compute* the change in terms of those string
names.

You can compute in terms of the numerical coin values first, and as a
separate step, translate the answer back in terms of coin names when
presenting the answer.

Think of a problem where someone asks you to add two Roman numerals
together.  It would be madness to try to do that without first changing the
representation into something more computer-friendly.  Same line of
reasoning. Choose the data representation to make the problem easy to
solve.  As long as we have a way to go back from that representation to
something that's human readable, were good to go.

From cs at zip.com.au  Tue Feb  2 18:36:40 2016
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 3 Feb 2016 10:36:40 +1100
Subject: [Tutor] 2016-02-01 Filter STRINGS in Log File and Pass as
 VARAIBLE within PYTHON script
In-Reply-To: <56B0ABC3.3000605@gmail.com>
References: <56B0ABC3.3000605@gmail.com>
Message-ID: <20160202233640.GA25744@cskk.homeip.net>

On 02Feb2016 21:14, knnleow GOOGLE <knnleow at gmail.com> wrote:
>Sorry, forget to make use of SET() ....... this is the new update.....
>appreciate your advice if we can still optimized further...

A few remarks, interleaved below.

>myArray = sys.argv

I would not make this a global. Instead, pass sys.argv to main at the bottom 
where you call main:

  main(sys.argv)

and set up main as:

  def main(argv):

>def checkInputs():
>        if('-date' not in myArray):
>                #print(__doc__)
>                print('''

Also pass argv to checkInputs, so:

  def checkInputs(argv):
    if '-date' not in argv:
      ....

BTW, this is not C or Perl, you don't need brackets around your "if" 
conditions.

Also, have a read of PEP 8:

  https://www.python.org/dev/peps/pep-0008/

Although it is the style guide for the standard library, most of its 
recommendations are followed by most Python code. I have in mind the section on 
function names; in Python it is common to name functions and variables using 
lowercase_with_underscores, so one would normally call "checkInputs" then name 
"check_inputs".

>USAGE:    python fail2ban-banned-ipAddress.py -date <YYYY-MM-DD>
>EXAMPLE:  python fail2ban-banned-ipAddress.py -date 2016-01-31
>                ''')
>                sys.exit(1)
>
>def main():

As remarked, go for:

  def main(argv):

>        #START MAIN PROGRAM HERE!!!
>        try:
>                checkInputs()

and then:

  check_inputs(argv)

>                myDate = myArray[myArray.index('-date') + 1]

It is better to properly examine the argument list.

For example (for something this simple), I often go:

  cmd = argv.pop(0)
  badopts = False
  if not argv:
    print("%s: missing options" % (cmd,), file=sys.stderr)
    badopts = True
  else:
    option = argv.pop(0)
    if option != '-date':
      print("%s: unrecognised option: %s" % (cmd, option)", file=sys.stderr)
      badopts = True
    elif not argv:
      print("%s: %s: missing date" % (cmd, option), file=sys.stderr)
      badopts = True
    else:
      my_date = argv.pop(0)
      ... check date for sanity here ...

  if argv:
    print("%s: extra arguments: %r" % (cmd, argv), file=sys.stderr)
    badopts = True

  if badopts:
    print(USAGE, file=sys.stderr)
    return 2

  ... proceed with main program here ...

See how it checks for options and their arguments, and has the scope to make 
many complaints before quitting?

If you have several options you will want to reach for a module like argparse, 
but your program has only one.

>                timestamp01 = time.strftime("%Y-%m-%d")
>                timestamp02 = time.strftime("%Y-%m-%d-%H%M%S")
>                wd01 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" + 
>timestamp01)
>                wd02 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" + 
>timestamp02)

You never use these. Also, these pathnames are very special looking; I would 
put them up the top as tunable constants (Python doesn't have constants, but it 
has an idiom of globals with UPPERCASE names for the same purpose).

[...]
>                # LOOP through the SET and WHOIS
>                for i in banIP_addrs:

You might do better to use a better name than "i", for example "ipaddr". More 
readable to others, and also to yourself later.

Also, you _may_ want to sort the addresses for reporting purposes (entirely 
your call), you could go:

  for ipaddr in sorted(banIP_addrs):

>                        print("i:", i)
>                        whoisVAR = os.popen("whois -H " + i + " |egrep 
>-i \"name|country|mail\" |sort -u").read()

Again, here you are running a shell pipeline which could be done in Python.  
Look at the subprocess module, and invoke:

  whois = subprocess.Popen(['whois', '-H', ipaddr], stdout=subprocess.PIPE)
  for line in whois:
    ... gather up name/country/mail as you did with the log file ...
  whois.wait()
  ... now print report ...

Cheers,
Cameron Simpson <cs at zip.com.au>

From cegarcia0323 at gmail.com  Tue Feb  2 12:46:07 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Tue, 2 Feb 2016 12:46:07 -0500
Subject: [Tutor] File extension change
Message-ID: <CABV0pWyUEwEG17W60t3e2YrNdyVZFsoY9UK3=ziWX8nyGEHCWA@mail.gmail.com>

Hi,
So what I am working on is taking a csv file and only taking 2 columns from
the spreadsheet and out putting that to a text file. Then taking those two
columns and organize them by product(key) and outputting the
description(values) that are associated. Some products have a lot of
duplicate descriptions and I am trying to get the counts of those. I have a
piece of code that takes anything greater then 5 and prints that and also
anything 4 or less goes into an 'other' category with the counts.So what I
am trying to do now is import a csv and change it to a text file with the
same naming convention as the csv. But I want the text file to have the
results from the json function. I want the text file that I am trying to
output to have the same name as the filename I am importing. The file that
I am inputting in the def populate_dict (usually the files are named
weekly_and the date...ex: weekly_20160102.csv) and then i want to run all
my code i have, but then in the def json_output instead of having the
filename which i have now is 'test.txt' I want to have weekly_20160102.txt
same naming convention as the csv file i am inputting. In my code I am soft
coding the filename so that the user is prompted to enter what file they
want to use.

import csvimport jsonimport sysimport osfrom collections import
defaultdictfrom collections import Counter

upper_limit = 5
lower_limit = 4

class dictionary():
    def __init__(self):
        self.dict = defaultdict(list)
        self.counted_dict = defaultdict(list)
        self.grouped_dict = defaultdict(list)
        self.other_dict = defaultdict(list)
        self.final_dict = defaultdict(list)
        self.total_dict = defaultdict(list)


    def populate_dict(self, filename):
        with open(filename, 'rb') as f:
            reader = csv.reader(f)
            next(reader, None)
            for row in reader:
                self.dict[row[2]].append(row[3])
		with open(filename, 'r') as searchfile, open('weekly_test.txt', 'w')
as search_results_file:
			for line in searchfile:
				if 'PBI 43125' in line:
					print >>search_results_file, line		
				

    def all_counts(self):
        data_count = Counter()
        for key in self.dict.keys():
            self.counted_dict.update({key: Counter(self.dict[key])})
			

    def total_counts(self):
        for key in self.dict.keys():
            total = 0
            b = Counter(self.dict[key])
            for value in b:
                total += b[value]
                new_list = str(total)
                #self.total_dict.update({key: 'Total count for this
application: ' + str(total)})
                #self.total_dict.update({key: str(total)})
            self.total_dict[key].append(new_list)

    def grouped_counts(self):
        for key in self.dict.keys():
            total = 0
            c = Counter(self.dict[key])
            for value in c:
                if c[value] >= upper_limit:
                    new_list = value, str(c[value])
                    self.grouped_dict[key].append(new_list)
                elif c[value] <= lower_limit:
                    total += c[value]
                    self.other_dict.update({key: 'other: ' + str(total)})

        for d in (self.grouped_dict, self.other_dict, self.total_dict):
            for key, value in d.iteritems():
                self.final_dict[key].append(value)
				
	

    def json_output(self):
		with open('test.txt', 'w') as text_file:
			json.dump(self.final_dict, text_file, sort_keys = True, indent = 4

From cegarcia0323 at gmail.com  Tue Feb  2 16:54:31 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Tue, 2 Feb 2016 16:54:31 -0500
Subject: [Tutor] date range
Message-ID: <CABV0pWxcoeMd7NQ6LgPVCaXYEMrVUxfR274p=APiQTCdxKY1yg@mail.gmail.com>

Hi,
So I am taking in a csv file with several rows and one of those rows in a
date row. I am trying to see if I can read in a csv file and search for a
certain date range like 1/2/2016 to 1/5/2016. Something like that or a
bigger date range. And then I want to ouput the results to either another
csv or to a textfile. I am not sure how to even beginning this and was
looking for some guidance on this.

From ben+python at benfinney.id.au  Tue Feb  2 20:31:05 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 03 Feb 2016 12:31:05 +1100
Subject: [Tutor] File extension change
References: <CABV0pWyUEwEG17W60t3e2YrNdyVZFsoY9UK3=ziWX8nyGEHCWA@mail.gmail.com>
Message-ID: <85y4b2d1gm.fsf@benfinney.id.au>

Chelsea G <cegarcia0323 at gmail.com> writes:

> import csvimport jsonimport sysimport osfrom collections import
> defaultdictfrom collections import Counter

Your email client has, probably without your consent, mangled the
content of your message.

Please ensure your email client does not attempt to apply pretty
formatting. Set it explicitly to send messages as ?plain text? only, so
your deliberate code formatting will not be mangled.

-- 
 \     ?Capitalism is the astounding belief that the most wickedest of |
  `\    men will do the most wickedest of things for the greatest good |
_o__)                               of everyone.? ?John Maynard Keynes |
Ben Finney


From ben+python at benfinney.id.au  Tue Feb  2 20:37:52 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 03 Feb 2016 12:37:52 +1100
Subject: [Tutor] date range
References: <CABV0pWxcoeMd7NQ6LgPVCaXYEMrVUxfR274p=APiQTCdxKY1yg@mail.gmail.com>
Message-ID: <85twlqd15b.fsf@benfinney.id.au>

Chelsea G <cegarcia0323 at gmail.com> writes:

> So I am taking in a csv file with several rows and one of those rows
> in a date row.

That doesn't make much sense, as a sepcification. CSV is by design a
data format in which every data row has the same structure: the fields
always have the same meaning, in the same order.

> I am trying to see if I can read in a csv file and search for a
> certain date range like 1/2/2016 to 1/5/2016.

You can, by breaking the problem into simpler problems:

* Get the rows into some collection of structured records. (The
  Python standard library's ?csv.DictReader? is your friend here.)

* Identify which field of the record is the data on which you want to
  filter. (For a collection of records you retrieved from reading a
  ?csv.DictReader?, you need to know which field name has the dates
  you're interested in.)

* Construct an expression that takes an arbitrary date value as found in
  that field, and interrogates it by some comparison that evaluates to
  True when the value is in your specified range and False otherwise.

* Construct a new collection of records, by iterating the full
  collection and only accumulating the ones which match your criteria.

Each of those is much simpler that the original problem. Solve each of
them separately, and it should then be much easier to put the solutions
together to solve the larger problem.

-- 
 \          ?Friendship is born at that moment when one person says to |
  `\    another, ?What! You too? I thought I was the only one!?? ?C.S. |
_o__)                                                            Lewis |
Ben Finney


From knnleow at gmail.com  Tue Feb  2 20:30:16 2016
From: knnleow at gmail.com (knnleow GOOGLE)
Date: Wed, 3 Feb 2016 09:30:16 +0800
Subject: [Tutor] 2016-02-03 Filter STRINGS from urllib and Pass as VARAIBLE
 within PYTHON script
In-Reply-To: <56B0ABC3.3000605@gmail.com>
References: <56AF0EF1.70906@gmail.com>
 <20160201105041.GA97843@cskk.homeip.net> <56B09D6B.5060701@gmail.com>
 <56B0ABC3.3000605@gmail.com>
Message-ID: <56B15828.2080107@gmail.com>

advice on how to filter VARIABLES from output below.

01. need to strip out
       BEGIN_STRINGS  b'{  and
       END_STRINGS }\n'

02. need to filter out these FIELDS
FIELD 1   "ip":"222.187.222.220"
FIELD 2   "country_code":"CN"
FIELD 3  "country_name":"China"
FIELD 6  "city":"Shanghai"
FIELD 8  "time_zone":"Asia/Shanghai"

03.  need to feed this into python script so as to generate a HTML File 
by date script was run. example "/fail2ban/2016-01-31.html"

i am doing this using shell script today. will like to get this working 
with python.

regards,
kuenn


SCRIPT:
                 banIP_addrs = set()
                 with open("/var/log/fail2ban.log") as fail_log:
                         for line in fail_log:
                                 if("Ban" in line and "fail2ban.actions: 
WARNING" in line and myDate in line):
                                         words = line.split()
                                         word6 = words[6]
                                         #print("word6:" , word6)
                                         banIP_addrs.add(word6)
                 #print("banIP_addrs:" , banIP_addrs)

                 for i in banIP_addrs:
                         #print("i:" , i)
                         myGeoip = 
urllib.request.urlopen("http://freegeoip.net/json/" + i).read()
                         print(myGeoip)

OUTPUT:

b'{"ip":"222.187.222.220","country_code":"CN","country_name":"China","region_code":"31","region_name":"Shanghai_Shi","city":"Shanghai","zip_code":"","time_zone":"Asia/Shanghai","latitude":31.0456,"longitude":121.3997,"metro_code":0}\n'
b'{"ip":"185.130.5.184","country_code":"LT","country_name":"Republic_of_Lithuania","region_code":"","region_name":"","city":"","zip_code":"","time_zone":"Europe/Vilnius","latitude":56,"longitude":24,"metro_code":0}\n'

From __peter__ at web.de  Wed Feb  3 04:18:13 2016
From: __peter__ at web.de (Peter Otten)
Date: Wed, 03 Feb 2016 10:18:13 +0100
Subject: [Tutor] How to write tests for main() function that does not
 return anything
References: <DUB111-W79627DF6E4B0404F741CB0A3DF0@phx.gbl>
Message-ID: <n8sgkm$jbu$1@ger.gmane.org>

Pedro Miguel wrote:

> Hi guys, I'm trying to test the code in the main() but I'm a bit unsure
> how to go about it since I'm not passing any arguments or even returning
> anything other then logging. For the purposes of the example I've
> shortened the tree statements in the function.. Can anyone point me in the
> right direction on how to test the logic below? Or perhaps give me an
> example on how to test the code below (that would be really appreciated).
> I've posted this question on Stackoverflow but the guys over there told me
> to mock it but no one provided an example (I'm fairly new on mocking)..and
> how could I mock the if statements or even the for loop? script.py
> 
> from . import settings
> 
> 
> def main():
>     if settings.PATHS:  # list containing full paths to a directory of
>     files
>         paths = settings.PATHS
>         for path in paths:
>             data = read_file(path)
>             modified_data = do_something_with_the_data_collected(data)
>             write_to_new_file(modified_data)
>     else:
>         logger.warning("There are no files in
>         {}".format(settings.FILES_DIRECTORY))
> 
> if __name__ == '__main__':
>     main()
> tests/file_tests.py
> import unittest
> 
> from module.script import main
> 
> 
> class FileManagerTests(unittest.TestCase):
> 
>     def test_main_func(self):
>         main()  # ?? this is where I am stuck, should I just test
>                 # that it logs correctly if certain data exists
>                 # in settings file?
> 
> if __name__ == '__main__':
>     unittest.main()
> 
> RegardsPedro+44(0)7549646235pedro.miguel at live.co.uk

Keep it simple and avoid mocking. Instead set up some test input data and a 
script that compares the actual with the expected output. If in addition you 
refactor the code in main in such a way that the building blocks can easily 
be tested with conventional unit tests you get better coverage for less 
effort. 

Here's how main() might look after refactoring:

def main():
    if not process_files(settings.PATHS):
        logger.warning(...)

def process_files(paths):
    has_files = False
    for path in paths:
        has_files = True  
        process_file(path)
    return has_files

def process_file(path):
    ...


From alan.gauld at btinternet.com  Wed Feb  3 04:25:11 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 3 Feb 2016 09:25:11 +0000
Subject: [Tutor] File extension change
In-Reply-To: <CABV0pWyUEwEG17W60t3e2YrNdyVZFsoY9UK3=ziWX8nyGEHCWA@mail.gmail.com>
References: <CABV0pWyUEwEG17W60t3e2YrNdyVZFsoY9UK3=ziWX8nyGEHCWA@mail.gmail.com>
Message-ID: <n8sh1m$qiv$1@ger.gmane.org>

On 02/02/16 17:46, Chelsea G wrote:

> ...but then in the def json_output instead of having the
> filename which i have now is 'test.txt' I want to have weekly_20160102.txt
> same naming convention as the csv file i am inputting. 

Take a look at the os.path module documentation.
There are several functions for manipulating
filenames including splitting and joining
the various constituent parts (basename,
extension, path etc).

You might decide to use os.rename() too.

hth
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Wed Feb  3 04:30:09 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 3 Feb 2016 09:30:09 +0000
Subject: [Tutor] date range
In-Reply-To: <CABV0pWxcoeMd7NQ6LgPVCaXYEMrVUxfR274p=APiQTCdxKY1yg@mail.gmail.com>
References: <CABV0pWxcoeMd7NQ6LgPVCaXYEMrVUxfR274p=APiQTCdxKY1yg@mail.gmail.com>
Message-ID: <n8shb1$v7m$1@ger.gmane.org>

On 02/02/16 21:54, Chelsea G wrote:

> date row. I am trying to see if I can read in a csv file and search for a
> certain date range like 1/2/2016 to 1/5/2016. 

Ben has given you some good advice on how to tackle the overall issue.

To deal with the dates you will probably want to use the time and/or
datetime modules. They can convert formatted strings into datetime
objects which can be compared to get a timedelta object representing the
difference in days, seconds and microseconds (although the last is not
very reliable and depends on OS acuracy)

hth
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From __peter__ at web.de  Wed Feb  3 04:34:42 2016
From: __peter__ at web.de (Peter Otten)
Date: Wed, 03 Feb 2016 10:34:42 +0100
Subject: [Tutor] 2016-02-03 Filter STRINGS from urllib and Pass as
 VARAIBLE within PYTHON script
References: <56AF0EF1.70906@gmail.com>
 <20160201105041.GA97843@cskk.homeip.net> <56B09D6B.5060701@gmail.com>
 <56B0ABC3.3000605@gmail.com> <56B15828.2080107@gmail.com>
Message-ID: <n8shjj$3ts$1@ger.gmane.org>

knnleow GOOGLE wrote:

> advice on how to filter VARIABLES from output below.
> 
> 01. need to strip out
>        BEGIN_STRINGS  b'{  and
>        END_STRINGS }\n'

In b'{...}' the b prefix indicates that you are dealing with a byte string.
You have to decode it and then you can use the json module to parse the 
data:

>>> import urllib.request, json
>>> data = urllib.request.urlopen(
... "http://freegeoip.net/json/222.187.222.220").read()
>>> data
b'{"ip":"222.187.222.220","country_code":"CN","country_name":"China","region_code":"31","region_name":"Shanghai 
Shi","city":"Shanghai","zip_code":"","time_zone":"Asia/Shanghai","latitude":31.0456,"longitude":121.3997,"metro_code":0}\n'
>>> data = data.decode()

If the above fails with a UnicodeDecodeError you have to provide the actual 
encoding.

>>> data
'{"ip":"222.187.222.220","country_code":"CN","country_name":"China","region_code":"31","region_name":"Shanghai 
Shi","city":"Shanghai","zip_code":"","time_zone":"Asia/Shanghai","latitude":31.0456,"longitude":121.3997,"metro_code":0}\n'

> 02. need to filter out these FIELDS
> FIELD 1   "ip":"222.187.222.220"
> FIELD 2   "country_code":"CN"
> FIELD 3  "country_name":"China"
> FIELD 6  "city":"Shanghai"
> FIELD 8  "time_zone":"Asia/Shanghai"

>>> data = json.loads(data)
>>> data["city"]
'Shanghai'
 
> 03.  need to feed this into python script so as to generate a HTML File
> by date script was run. example "/fail2ban/2016-01-31.html"

Instead of doing this manually you should pick one of the many templating 
languages out there.
 
> i am doing this using shell script today. will like to get this working
> with python.
> 
> regards,
> kuenn
> 
> 
> SCRIPT:
>                  banIP_addrs = set()
>                  with open("/var/log/fail2ban.log") as fail_log:
>                          for line in fail_log:
>                                  if("Ban" in line and "fail2ban.actions:
> WARNING" in line and myDate in line):
>                                          words = line.split()
>                                          word6 = words[6]
>                                          #print("word6:" , word6)
>                                          banIP_addrs.add(word6)
>                  #print("banIP_addrs:" , banIP_addrs)
> 
>                  for i in banIP_addrs:
>                          #print("i:" , i)
>                          myGeoip =
> urllib.request.urlopen("http://freegeoip.net/json/" + i).read()
>                          print(myGeoip)
> 
> OUTPUT:
> 
> 
b'{"ip":"222.187.222.220","country_code":"CN","country_name":"China","region_code":"31","region_name":"Shanghai_Shi","city":"Shanghai","zip_code":"","time_zone":"Asia/Shanghai","latitude":31.0456,"longitude":121.3997,"metro_code":0}\n'
> 
b'{"ip":"185.130.5.184","country_code":"LT","country_name":"Republic_of_Lithuania","region_code":"","region_name":"","city":"","zip_code":"","time_zone":"Europe/Vilnius","latitude":56,"longitude":24,"metro_code":0}\n'
> _______________________________________________



From alan.gauld at btinternet.com  Wed Feb  3 04:39:55 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 3 Feb 2016 09:39:55 +0000
Subject: [Tutor] 2016-02-03 Filter STRINGS from urllib and Pass as
 VARAIBLE within PYTHON script
In-Reply-To: <56B15828.2080107@gmail.com>
References: <56AF0EF1.70906@gmail.com>
 <20160201105041.GA97843@cskk.homeip.net> <56B09D6B.5060701@gmail.com>
 <56B0ABC3.3000605@gmail.com> <56B15828.2080107@gmail.com>
Message-ID: <n8shtb$8v1$1@ger.gmane.org>

On 03/02/16 01:30, knnleow GOOGLE wrote:
> advice on how to filter VARIABLES from output below.
> 
> 01. need to strip out
>        BEGIN_STRINGS  b'{  and
>        END_STRINGS }\n'

Are you sure those are actually part of the string and not just the
representation output by Python?
The b at the start usually means its a byte string and is not actually
present in the data. And the trailing \n is a newline character which
can be removed with the string rstrip() method.

Also the format of your data looks a lot like JSON?
If that's the case you may be better off using the json module
to parse it into Python objects. If you strip off the markers
you may be forcing yourself to write a complex string parser
when the json one could do the job for you.


> 
> 02. need to filter out these FIELDS
> FIELD 1   "ip":"222.187.222.220"
> FIELD 2   "country_code":"CN"
> FIELD 3  "country_name":"China"
> FIELD 6  "city":"Shanghai"
> FIELD 8  "time_zone":"Asia/Shanghai"
> 

If it is JSON data you will get a dictionary from which you
can read out (or ignore) any of the fields as you wish.

> 03.  need to feed this into python script so as to generate a HTML File 
> by date script was run. example "/fail2ban/2016-01-31.html"

I'm not sure what exactly you mean here. Can you elaborate?
What is the "this" that you want to "feed in" and what kind
of HTML file are you thinking of? Just a simple textual
list of entries? A table?? If you can store the data as
Python objects it is fairly easy to just print the strings
you want as needed.

> OUTPUT:
> 
> b'{"ip":"222.187.222.220","country_code":"CN",
> "country_name":"China","region_code":"31",
> "region_name":"Shanghai_Shi","city":"Shanghai",
> "zip_code":"","time_zone":"Asia/Shanghai",
> "latitude":31.0456,"longitude":121.3997,"metro_code":0}\n'


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Wed Feb  3 08:20:54 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 4 Feb 2016 00:20:54 +1100
Subject: [Tutor] How to write tests for main() function that does not
 return anything
In-Reply-To: <DUB111-W79627DF6E4B0404F741CB0A3DF0@phx.gbl>
References: <DUB111-W79627DF6E4B0404F741CB0A3DF0@phx.gbl>
Message-ID: <20160203132054.GM31806@ando.pearwood.info>

On Tue, Feb 02, 2016 at 10:46:04AM +0000, Pedro Miguel wrote:

> Hi guys, I'm trying to test the code in the main() but I'm a bit 
> unsure how to go about it since I'm not passing any arguments or even 
> returning anything other then logging. For the purposes of the example 
> I've shortened the tree statements in the function.. Can anyone point 
> me in the right direction on how to test the logic below? 

That's a great question!

I think the right answer is that testing main() is *not* a unit test, 
it's an integration test. You're testing that the application as a whole 
works the way you expect, rather than testing individual units of the 
application (functions, class, methods, modules).

Nevertheless, you could use unittest to test this. Suppose you have a 
script which reads some files as input, and then writes some files as 
output. You could use the unittest library to give you this integration 
test:



import unittest

class MainTest(unittest.TestCase):
    def setUp(self):
        # set up the environment needed for the test

    def test_main(self):
        result = os.system("python myscript.py")
        self.assertEqual(result, 0)
        # now check that the script's output is what you expect

    def tearDown(self):
        # clean up after the test



But I wouldn't include it in your unit testing module, I would write it 
as a separate test file, and probably wouldn't bother using the unittest 
library since it doesn't give you any real benefit here:

# myscript_integration_test.py

setup()
result = os.system("python myscript.py")
if result:
    print "Integration test failed!"
check_output()
teardown()



Of course, a real test will be more complex. It would need to test each 
of the applications command line options, test whether it works 
correctly with any environment variables it uses. Test whether it logs 
as expected, whether it handles bad input as expected, etc. It will 
probably be quite slow, so you don't want to run it as often as your 
unit tests and regression tests.

In my opinion, your unit tests should limit themselves to checking that 
main() actually exists. You shouldn't assume that 

from myscript import main
main()


will work correctly. For example, a lot of scripts or applications 
perform setup in the module, like this:

if __name__ == '__main__':
    import sys
    args = sys.argv[1:]
    process(args)
    setup_logging()
    sys.exit(main(args))


which is hard to unit test. (You could, I guess, use a strict rule that 
"main() is the only thing that appears inside the `if __name__...` 
block", and if that rule works for you, great, but for me it feels a bit 
too restrictive.


Hope this helps,



-- 
Steve

From esawiek at gmail.com  Wed Feb  3 10:29:49 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Wed, 3 Feb 2016 10:29:49 -0500
Subject: [Tutor] Enumerate vs DictReader object manipulation:
Message-ID: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>

Hi All





I have a code that reads a csv file via DictReader. I ran into a peculiar
problem. The python interpreter ignores the 2nd code. That is if I put the
reader iterator 1st, like the code below, the enumerate code is ignored; if
I put the enumerate code 1st, the reader code is ignored. I am curious to
know the nature of such behavior. EKE



Here  part of my code:



.

.

.

    reader = csv.DictReader(MyFile)

    for row in reader:

        list_values = list(row.values())

        print (list_values)



    for i,j in enumerate(reader):

        print(j)

From dyoo at hashcollision.org  Wed Feb  3 11:21:19 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Wed, 3 Feb 2016 08:21:19 -0800
Subject: [Tutor] Enumerate vs DictReader object manipulation:
In-Reply-To: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
References: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
Message-ID: <CAGZAPF4Ux8gcram92S0HUaEx6v782AujzaSgc5D0jt_skPxv_w@mail.gmail.com>

> I have a code that reads a csv file via DictReader. I ran into a peculiar
> problem. The python interpreter ignores the 2nd code. That is if I put the
> reader iterator 1st, like the code below, the enumerate code is ignored; if
> I put the enumerate code 1st, the reader code is ignored. I am curious to
> know the nature of such behavior. EKE


The csv.DictReader acts in a streaming way: once you've started
pulling from it, it won't rewind back.  That is, Python isn't ignoring
your commands: it's following them very precisely.  The only problem
is that in your second loop, the reader is already exhausted: the
second call to enumerate will not rewind it.  The reason that it works
in a streaming way is fundamentally because it's reading from a file,
and files act in a streaming way to accommodate situations where the
content is too large to hold all at once in memory.


You have a few options.  If the file is itself not too large, you can
keep the contents of the DictReader as an explicit list:

    rows = list(csv.DictReader(MyFile))
    ...

after which you've got the rows held all at once, and can iterate
through those rows at will.

If the file is large, you can effectively rewind by reopening the file
and the reader again.

#################################
MyFile = ...  # opening up MyFIle
reader = csv.DictReader(MyFile)
for row in reader: ...

MyFile = ... # opening up MyFile again
reader = csv.DictReader(MyFile)
for row in reader: ...
#################################

From alan.gauld at btinternet.com  Wed Feb  3 11:24:05 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 3 Feb 2016 16:24:05 +0000
Subject: [Tutor] Enumerate vs DictReader object manipulation:
In-Reply-To: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
References: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
Message-ID: <n8t9j5$sl4$1@ger.gmane.org>

On 03/02/16 15:29, Ek Esawi wrote:

>     reader = csv.DictReader(MyFile)
> 
>     for row in reader:
>         list_values = list(row.values())
>         print (list_values)
> 

At this point the reader has reached the end of the file.

>     for i,j in enumerate(reader):
>         print(j)

So you can't read any more using that same reader you
need to create a new reader. (Or reset the old one, but
I don't think that's possible)

But why are you using a DictReader when you immediately
convert the data into a list? Why not use the standard
reader which returns tuples. Or why not just use the dicts
that you have - usually that leads to more intuitive
and reliable code since you access the data by name
rather than  column number.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From ben+python at benfinney.id.au  Wed Feb  3 16:20:48 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 04 Feb 2016 08:20:48 +1100
Subject: [Tutor] How to write tests for main() function that does not
 return anything
References: <DUB111-W79627DF6E4B0404F741CB0A3DF0@phx.gbl>
 <20160203132054.GM31806@ando.pearwood.info>
Message-ID: <85powdcwy7.fsf@benfinney.id.au>

Steven D'Aprano <steve at pearwood.info> writes:

> I think the right answer is that testing main() is *not* a unit test,
> it's an integration test. You're testing that the application as a
> whole works the way you expect, rather than testing individual units
> of the application (functions, class, methods, modules).

Alternatively, if you can define what ?main? should do in isolation from
the rest of the program, then you *can* test it in unit tests.

That requires designing ?main? such that you can explicitly describe its
interactions with the rest of the program; inputs, outputs, side
effects, expected behaviour when any of them change.

That exercise may lead you to conclude ?wow, this actually does quite a
lot and it's difficult to describe? ? and then it's a short step to
realising you should re-design the function so it has a more narrow
specification :-)

-- 
 \            ?The idea that He would take his attention away from the |
  `\       universe in order to give me a bicycle with three speeds is |
_o__)  just so unlikely that I can't go along with it.? ?Quentin Crisp |
Ben Finney


From ben+python at benfinney.id.au  Wed Feb  3 16:24:59 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 04 Feb 2016 08:24:59 +1100
Subject: [Tutor] Enumerate vs DictReader object manipulation:
References: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
Message-ID: <85lh71cwr8.fsf@benfinney.id.au>

Ek Esawi <esawiek at gmail.com> writes:

> I have a code that reads a csv file via DictReader. I ran into a peculiar
> problem. The python interpreter ignores the 2nd code. That is if I put the
> reader iterator 1st, like the code below, the enumerate code is ignored; if
> I put the enumerate code 1st, the reader code is ignored.

You have discovered the difference between an iterable (an object you
can iterate over with ?for?), versus a sequence (an object whose items
remain in place and can be iterated many times).

Every sequence is an iterable, but not vice versa.

File objects are iterables, but not sequences. Each time you ask for the
next item, you can't ask the file object for that item again; it is
?consumed? by the act of iteration.

-- 
 \       ?When I was little, my grandfather used to make me stand in a |
  `\   closet for five minutes without moving. He said it was elevator |
_o__)                                        practice.? ?Steven Wright |
Ben Finney


From esawiek at gmail.com  Wed Feb  3 16:54:47 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Wed, 3 Feb 2016 16:54:47 -0500
Subject: [Tutor] Enumerate vs DictReader object manipulation:
Message-ID: <CA+ZkTxuEoPtqcCxLV+Ljt8zyWhBEZ7Xf9S-BFRJn1h93HX7kaw@mail.gmail.com>

Thank you all. The only reason i tried both ways is to experiment with
Python. They made sense to me and thought why not try them both. And i am
relatively new to Python.

Thanks again--EKE

From akleider at sonic.net  Wed Feb  3 21:47:05 2016
From: akleider at sonic.net (Alex Kleider)
Date: Wed, 03 Feb 2016 18:47:05 -0800
Subject: [Tutor] Enumerate vs DictReader object manipulation:
In-Reply-To: <85lh71cwr8.fsf@benfinney.id.au>
References: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
 <85lh71cwr8.fsf@benfinney.id.au>
Message-ID: <9c3ac2822b36b1962970e9808df93bbb@sonic.net>

On 2016-02-03 13:24, Ben Finney wrote:


> You have discovered the difference between an iterable (an object you
> can iterate over with ?for?), versus a sequence (an object whose items
> remain in place and can be iterated many times).
> 
> Every sequence is an iterable, but not vice versa.
> 
> File objects are iterables, but not sequences. Each time you ask for 
> the
> next item, you can't ask the file object for that item again; it is
> ?consumed? by the act of iteration.

How does a dict fit into this scheme?
Is it a sequence?  It is an iterable (in that for key in d: works 
although not in a predictable manner and for this reason I tend NOT to 
think of it as a sequence.)

From ben+python at benfinney.id.au  Wed Feb  3 22:21:53 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 04 Feb 2016 14:21:53 +1100
Subject: [Tutor] Enumerate vs DictReader object manipulation:
References: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
 <85lh71cwr8.fsf@benfinney.id.au>
 <9c3ac2822b36b1962970e9808df93bbb@sonic.net>
Message-ID: <85wpqlb1ny.fsf@benfinney.id.au>

Alex Kleider <akleider at sonic.net> writes:

> How does a dict fit into this scheme?
> Is it a sequence?

No, a dict is not a sequence. But it is a container: all its items
remain available and can be retrieved again and again, and you can
interrogate whether a value is one of the items in that container.

An instance of the built-in ?set? type is also a container and not a
sequence.

Containers are iterable too.

> It is an iterable (in that for key in d: works although not in a
> predictable manner and for this reason I tend NOT to think of it as a
> sequence.)

That's right, IMO.

-- 
 \       ?The generation of random numbers is too important to be left |
  `\                                    to chance.? ?Robert R. Coveyou |
_o__)                                                                  |
Ben Finney


From jmartiee at gmail.com  Wed Feb  3 23:52:55 2016
From: jmartiee at gmail.com (Johan Martinez)
Date: Wed, 3 Feb 2016 20:52:55 -0800
Subject: [Tutor] Syntax for list comps
Message-ID: <CA+jMyfrq4346-tdUYGVVXc7zwNp88bpS6vaHNgY7kedv=2tdJw@mail.gmail.com>

Where can I find syntax for list comps? I am confused how/whether they are
evaluated left-right or right-left. Consider following list flattening
example:

mx = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]

[value for row in mx for value in row]

[1, 2, 3, 4, 5, 6, 7, 8, 9]

It seems like 'for' loops are evaluated from left to right (returned 'row'
is passed to right side for), but return 'value' (from second for loop) is
not on right, but left side.

That's a bit confusing syntax for me. Sorry if I am not clear in explaining
it.

What is the reasoning behind this syntax?


-jM

From dyoo at hashcollision.org  Thu Feb  4 00:29:15 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Wed, 3 Feb 2016 21:29:15 -0800
Subject: [Tutor] Syntax for list comps
In-Reply-To: <CA+jMyfrq4346-tdUYGVVXc7zwNp88bpS6vaHNgY7kedv=2tdJw@mail.gmail.com>
References: <CA+jMyfrq4346-tdUYGVVXc7zwNp88bpS6vaHNgY7kedv=2tdJw@mail.gmail.com>
Message-ID: <CAGZAPF643G66kH+uxTnwQHBaG9Ymm++ghg6Jf-Tyrm0kRfLkvw@mail.gmail.com>

On Wed, Feb 3, 2016 at 8:52 PM, Johan Martinez <jmartiee at gmail.com> wrote:
> Where can I find syntax for list comps? I am confused how/whether they are
> evaluated left-right or right-left. Consider following list flattening
> example:
>
> mx = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>
> [value for row in mx for value in row]


I read this as the following parts:

###################
mx = [
 value                 # line 1
 for row in mx      # line 2
 for value in row   # line 3
]
###################

where I'm seeing three parts.  Internally, I'm thinking that this is
going to work like the following code:

################################
mx = []

for row in mx:                # from line 2
    for value in row:         # from line 3
        mx.append(value)  # from line 1
################################

where we represent the implicit "nesting" in the double loops in the
list comprehension as actual nesting here, and pieces get rearranged
from the list comprehension syntax into the above.



> That's a bit confusing syntax for me. Sorry if I am not clear in explaining
> it.

It is admittedly odd.  I believe the syntax used for list
comprehensions is adopted from a kind of mathematical syntax that
mathematicians use for expressing sets of values.

For example, if we were to express the set of odd numbers as a
mathematician, we might say:

                                        { 2x + 1  |  x ? N }

where N is extra fancy-looking, the '?' is the math symbol for "in",
and the bar "|" is there just to distinguish the left side from the
right side.  https://en.wikipedia.org/wiki/Set-builder_notation#Formal_set_builder_notation_sets

Now that you've seen what a mathematician writes to represent a set,
we can look back at Python list comprehensions: they bear a
resemblance to set builder notation.  We don't have fancy symbols
ready on our keyboards, so we use for loop keywords to compensate.  I
think that's the inspiration.


That being said, as soon as there are *two* loops nested in there, I
think list comprehensions are hard to read too.  When they get nested
like that, I prefer to express them as the code with explicit list
construction and calls to append().


Hope that makes sense!

From ben+python at benfinney.id.au  Thu Feb  4 00:31:03 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 04 Feb 2016 16:31:03 +1100
Subject: [Tutor] Syntax for list comps
References: <CA+jMyfrq4346-tdUYGVVXc7zwNp88bpS6vaHNgY7kedv=2tdJw@mail.gmail.com>
Message-ID: <85oabxavoo.fsf@benfinney.id.au>

Johan Martinez <jmartiee at gmail.com> writes:

> Where can I find syntax for list comps?

The same place as other syntax descriptions: in the language reference
<URL:https://docs.python.org/3/reference/expressions.html#displays-for-lists-sets-and-dictionaries>.

> I am confused how/whether they are evaluated left-right or right-left.

According to the documentation:

    [?] the elements of the new container are those that would be
    produced by considering each of the for or if clauses a block,
    nesting from left to right, and evaluating the expression to produce
    an element each time the innermost block is reached.

> That's a bit confusing syntax for me. Sorry if I am not clear in
> explaining it.

Does that help? If not, please try re-phrasing the question.

-- 
 \           ?The lift is being fixed for the day. During that time we |
  `\            regret that you will be unbearable.? ?hotel, Bucharest |
_o__)                                                                  |
Ben Finney


From matt.williams45.mw at gmail.com  Thu Feb  4 01:45:15 2016
From: matt.williams45.mw at gmail.com (Matt Williams)
Date: Thu, 04 Feb 2016 06:45:15 +0000
Subject: [Tutor] Enumerate vs DictReader object manipulation:
In-Reply-To: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
References: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
Message-ID: <CAFTVGQYtAJPJPgdoSqiLPwoNEa0rxRbme9nnNus6kyVYwypbJg@mail.gmail.com>

Just as a note - you are not the only person caught out by this - it is a
very common slip.

I wonder whether it would be worth adding a more explicit line about this
in the Python Docs?

Matt

On Wed, 3 Feb 2016 16:13 Ek Esawi <esawiek at gmail.com> wrote:

> Hi All
>
>
>
>
>
> I have a code that reads a csv file via DictReader. I ran into a peculiar
> problem. The python interpreter ignores the 2nd code. That is if I put the
> reader iterator 1st, like the code below, the enumerate code is ignored; if
> I put the enumerate code 1st, the reader code is ignored. I am curious to
> know the nature of such behavior. EKE
>
>
>
> Here  part of my code:
>
>
>
> .
>
> .
>
> .
>
>     reader = csv.DictReader(MyFile)
>
>     for row in reader:
>
>         list_values = list(row.values())
>
>         print (list_values)
>
>
>
>     for i,j in enumerate(reader):
>
>         print(j)
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From oscar.j.benjamin at gmail.com  Thu Feb  4 04:41:07 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 4 Feb 2016 09:41:07 +0000
Subject: [Tutor] Enumerate vs DictReader object manipulation:
In-Reply-To: <CAFTVGQYtAJPJPgdoSqiLPwoNEa0rxRbme9nnNus6kyVYwypbJg@mail.gmail.com>
References: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
 <CAFTVGQYtAJPJPgdoSqiLPwoNEa0rxRbme9nnNus6kyVYwypbJg@mail.gmail.com>
Message-ID: <CAHVvXxRoc=JWTstDxosxnFrkf2_PE1vEYUhfeQvgF-K=qf5qfg@mail.gmail.com>

On 4 February 2016 at 06:45, Matt Williams <matt.williams45.mw at gmail.com> wrote:
>
> Just as a note - you are not the only person caught out by this - it is a
> very common slip.
>
> I wonder whether it would be worth adding a more explicit line about this
> in the Python Docs?

Where in the docs would you put it and what would you want it to say?

--
Oscar

From oscar.j.benjamin at gmail.com  Thu Feb  4 04:46:41 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 4 Feb 2016 09:46:41 +0000
Subject: [Tutor] Enumerate vs DictReader object manipulation:
In-Reply-To: <85wpqlb1ny.fsf@benfinney.id.au>
References: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
 <85lh71cwr8.fsf@benfinney.id.au> <9c3ac2822b36b1962970e9808df93bbb@sonic.net>
 <85wpqlb1ny.fsf@benfinney.id.au>
Message-ID: <CAHVvXxSq2PmWW=3Nnt=imYh_iEB3UhCBQG_pg=Vj+EaDNzJ5MQ@mail.gmail.com>

On 4 February 2016 at 03:21, Ben Finney <ben+python at benfinney.id.au> wrote:
> Alex Kleider <akleider at sonic.net> writes:
>
>> How does a dict fit into this scheme?
>> Is it a sequence?
>
> No, a dict is not a sequence. But it is a container: all its items
> remain available and can be retrieved again and again, and you can
> interrogate whether a value is one of the items in that container.
>
> An instance of the built-in ?set? type is also a container and not a
> sequence.
>
> Containers are iterable too.

You can see an explanation of the different collection terminology here:
https://docs.python.org/2/library/collections.html#collections-abstract-base-classes

A dict is a Mapping and a set is a Set. Both also comes under the
categories Sized, Iterable, and Container.

--
Oscar

From alexgnuant at gmail.com  Thu Feb  4 07:49:39 2016
From: alexgnuant at gmail.com (Alexa kun)
Date: Thu, 4 Feb 2016 14:49:39 +0200
Subject: [Tutor] Hi Dear!
Message-ID: <CALWJDU7Z3iCSFKgAE9=QuPzh=rwvQRzGawz2TeqmXnrEc1NGbA@mail.gmail.com>

Hi Dear!
I newbie and read 2.1.2. Interactive Mode
https://docs.python.org/3/tutorial/interpreter.html

but when I type

>>> the_world_is_flat = True
>>> if the_world_is_flat:
...     print("Be careful not to fall off!")

I got answer

IndentationError: expected an indented block

[root at localhost /]# python3
Python 3.4.3 (default, Jun 29 2015, 12:15:26)
[GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> the_world_is_flat = True
>>> if the_world_is_flat:
... print(Be careful not to fall off!)
  File "<stdin>", line 2
    print(Be careful not to fall off!)
        ^
IndentationError: expected an indented block
>>>

I have installed Python3 in Linux Fedora 23
Please tell my why Python3 doesn't work?

Sincerely!
Alexander

From steve at pearwood.info  Thu Feb  4 10:12:45 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 5 Feb 2016 02:12:45 +1100
Subject: [Tutor] Hi Dear!
In-Reply-To: <CALWJDU7Z3iCSFKgAE9=QuPzh=rwvQRzGawz2TeqmXnrEc1NGbA@mail.gmail.com>
References: <CALWJDU7Z3iCSFKgAE9=QuPzh=rwvQRzGawz2TeqmXnrEc1NGbA@mail.gmail.com>
Message-ID: <20160204151244.GQ31806@ando.pearwood.info>

Hello Alexander, and welcome!

My answers are below, between your questions (starting with > quote 
marks).


On Thu, Feb 04, 2016 at 02:49:39PM +0200, Alexa kun wrote:
> Hi Dear!
> I newbie and read 2.1.2. Interactive Mode
> https://docs.python.org/3/tutorial/interpreter.html
> 
> but when I type
> 
> >>> the_world_is_flat = True
> >>> if the_world_is_flat:
> ...     print("Be careful not to fall off!")

Look carefully at the last line. Do you notice the indented space 
between the three dots ... and the print? You need to either press space 
at least once, or the TAB key, to indent the line.



> I got answer
> 
> IndentationError: expected an indented block

Here Python tells you exactly what the problem is. Python expected an 
indented block of text (at least one line, indented by at least one 
space), but you didn't indent the line.


Python expects:

if the_world_is_flat:
	print(Be careful not to fall off!)


but you typed:

if the_world_is_flat:
print(Be careful not to fall off!)


without the indentation. You should press the TAB key to indent, or the 
SPACE key at least once.



Also, one last comment:

> [root at localhost /]# python3

I see from this that you are running Python as the root superuser. This 
is VERY dangerous for a beginner (and even for an expert). As root, you 
can over-write essential system files. Which means that if you 
accidentally give the wrong Python commands, you could break your system 
and leave it in a broken state where it needs to be re-installed.

It is MUCH safer to experiment as the regular user. If the command 
prompt shows # then you are running as root. If it shows $ then you are 
running as a regular user.



-- 
Steve

From joel.goldstick at gmail.com  Thu Feb  4 10:39:36 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Thu, 4 Feb 2016 10:39:36 -0500
Subject: [Tutor] Hi Dear!
In-Reply-To: <CALWJDU7Z3iCSFKgAE9=QuPzh=rwvQRzGawz2TeqmXnrEc1NGbA@mail.gmail.com>
References: <CALWJDU7Z3iCSFKgAE9=QuPzh=rwvQRzGawz2TeqmXnrEc1NGbA@mail.gmail.com>
Message-ID: <CAPM-O+wDahsorBLJyDQSnFhiVXB_ayBgx0MbnH6Upc+FtCahZg@mail.gmail.com>

On Thu, Feb 4, 2016 at 7:49 AM, Alexa kun <alexgnuant at gmail.com> wrote:

> Hi Dear!
> I newbie and read 2.1.2. Interactive Mode
> https://docs.python.org/3/tutorial/interpreter.html
>
> but when I type
>
> >>> the_world_is_flat = True
> >>> if the_world_is_flat:
> ...     print("Be careful not to fall off!")
>
> I got answer
>
> IndentationError: expected an indented block
>
> [root at localhost /]# python3
> Python 3.4.3 (default, Jun 29 2015, 12:15:26)
> [GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>
> >>> the_world_is_flat = True
> >>> if the_world_is_flat:
> ... print(Be careful not to fall off!)
>   File "<stdin>", line 2
>     print(Be careful not to fall off!)
>         ^
> IndentationError: expected an indented block
> >>>
> There is something funny about your print function.  The three periods
> indicate that you are no longer in interactive mode.
>

See my session below


> I have installed Python3 in Linux Fedora 23
> Please tell my why Python3 doesn't work?
>
> Sincerely!
> Alexander
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

>>> flat = True
>>> if flat:
...     print("flat")
...
flat
>>>


-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From alan.gauld at btinternet.com  Thu Feb  4 12:22:54 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 4 Feb 2016 17:22:54 +0000
Subject: [Tutor] Hi Dear!
In-Reply-To: <CALWJDU7Z3iCSFKgAE9=QuPzh=rwvQRzGawz2TeqmXnrEc1NGbA@mail.gmail.com>
References: <CALWJDU7Z3iCSFKgAE9=QuPzh=rwvQRzGawz2TeqmXnrEc1NGbA@mail.gmail.com>
Message-ID: <n901dd$m1t$1@ger.gmane.org>

On 04/02/16 12:49, Alexa kun wrote:
> Hi Dear!

Hi.
Can I ask that in future you choose a subject line that reflects your
question?
For this case it might be "IndentationError" say.

> but when I type
> 
>>>> the_world_is_flat = True
>>>> if the_world_is_flat:
> ...     print("Be careful not to fall off!")

The problem is that is not what you typed.
And this is why we always ask for the full error
trace - thanks for including it.

>>>> if the_world_is_flat:
> ... print(Be careful not to fall off!)
>   File "<stdin>", line 2
>     print(Be careful not to fall off!)
>         ^
> IndentationError: expected an indented block

It says that there is an IndentationError which means that
Python is expecting to see a line starting in a different
place from where it does. In your case that means the line
after the 'if' is expected to be "indented". That is it
should have a few spaces in front of it (we usually
recommend 4 but Python doesn't care so long as there
is more than the preceding line).

The indentation is Python's way of telling which bits of
code need to be executed if the 'if' test is true. Anything
that is indented will be executed (or missed if the test
is false) as appropriate. The indentation needs to be the
same for all the indented lines.

ie

if foo > 42:
   print (foo)
   f = 666

is ok but

if foo > 42:
    print (foo)
  f00 = 666        # not enough spaces

won't work.

> Please tell my why Python3 doesn't work?

It's working just fine, you only need to give it
some space(s)... :-)

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From shlyoko at gmail.com  Thu Feb  4 09:36:51 2016
From: shlyoko at gmail.com (Emil Natan)
Date: Thu, 4 Feb 2016 16:36:51 +0200
Subject: [Tutor] Hi Dear!
In-Reply-To: <CALWJDU7Z3iCSFKgAE9=QuPzh=rwvQRzGawz2TeqmXnrEc1NGbA@mail.gmail.com>
References: <CALWJDU7Z3iCSFKgAE9=QuPzh=rwvQRzGawz2TeqmXnrEc1NGbA@mail.gmail.com>
Message-ID: <CAG=4S2CpYSjJAihgi7ZACLQNm=hmMUDoMybGVnnW3-KHpvwr8g@mail.gmail.com>

On Thu, Feb 4, 2016 at 2:49 PM, Alexa kun <alexgnuant at gmail.com> wrote:

> Hi Dear!
> I newbie and read 2.1.2. Interactive Mode
> https://docs.python.org/3/tutorial/interpreter.html
>
> but when I type
>
> >>> the_world_is_flat = True
> >>> if the_world_is_flat:
> ...     print("Be careful not to fall off!")
>
> I got answer
>
> IndentationError: expected an indented block
>
> [root at localhost /]# python3
> Python 3.4.3 (default, Jun 29 2015, 12:15:26)
> [GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>
> >>> the_world_is_flat = True
> >>> if the_world_is_flat:
> ... print(Be careful not to fall off!)
>   File "<stdin>", line 2
>     print(Be careful not to fall off!)
>         ^
> IndentationError: expected an indented block
> >>>
>
>

You should have the print function indented, usually by 4 spaces. This is
how Python knows which commands to be executed as part of the if block. So
this is what you'll make your code work:

 >>> the_world_is_flat = True
>>> if the_world_is_flat:
...     print("Be careful not to fall off!")
...
Be careful not to fall off!

The interpreter also tries to help you, it puts ... at the begging of the
line (instead of >>>) which means it expect some indentation.

Emil

I have installed Python3 in Linux Fedora 23
> Please tell my why Python3 doesn't work?
>
> Sincerely!
> Alexander
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From akleider at sonic.net  Thu Feb  4 13:57:04 2016
From: akleider at sonic.net (Alex Kleider)
Date: Thu, 04 Feb 2016 10:57:04 -0800
Subject: [Tutor] Enumerate vs DictReader object manipulation:
In-Reply-To: <CAHVvXxSq2PmWW=3Nnt=imYh_iEB3UhCBQG_pg=Vj+EaDNzJ5MQ@mail.gmail.com>
References: <CA+ZkTxudQyScpzeVxGwQMFGusVCV32xng6cPtPG7b4T9YzK1uA@mail.gmail.com>
 <85lh71cwr8.fsf@benfinney.id.au>
 <9c3ac2822b36b1962970e9808df93bbb@sonic.net>
 <85wpqlb1ny.fsf@benfinney.id.au>
 <CAHVvXxSq2PmWW=3Nnt=imYh_iEB3UhCBQG_pg=Vj+EaDNzJ5MQ@mail.gmail.com>
Message-ID: <dbe89f83ed2056424ff6ba6cb4daf096@sonic.net>

On 2016-02-04 01:46, Oscar Benjamin wrote:
> 
> You can see an explanation of the different collection terminology 
> here:
> https://docs.python.org/2/library/collections.html#collections-abstract-base-classes
> 
> A dict is a Mapping and a set is a Set. Both also comes under the
> categories Sized, Iterable, and Container.
> 
> --
> Oscar

Thanks for the clarification.  It caused me to go down the path a bit 
farther...
The following link might be helpful in this context:
http://blog.wachowicz.eu/?p=132

From tbrodle at hotmail.com  Thu Feb  4 18:26:26 2016
From: tbrodle at hotmail.com (Tom Brodle)
Date: Thu, 4 Feb 2016 16:26:26 -0700
Subject: [Tutor] Help needed
Message-ID: <BLU170-W546658256FE612B00C8EF2A9D10@phx.gbl>

I am very new  to Python.  I am just practicing code using a simple plug board.  The code is working as I want it to but the last message and the original looping makes me have questions.  Because the main code works I will just abbreviate it.

import GPIO
import time
Gpio setmode (BCM)
led_pin = 18
GPIO setup(led_pin, GPIO)

try:
        While true:
            Turn LED on
            sleep 5 seconds
            Turn LED off
            sleep .5 seconds  # program was initially designed to keep LED turning on and off and that worked.
                                           # Coming from the world of Basic I did/do not understand what made the program loop
                                           # back to while True: line of code.
                                           # I  wanted the LED to turn on and then off and stay off, so from some Python.org 
                                           # page I became aware of quit(code=None) and inserted it before the line Print("Cleaning up")                  quit(code=None)            # I put this line in to stop the looping back to While true:.                    
    Print("Cleaning up")
    GPIO.cleanup()           # I was told to put this line in but I do not know where it goes to do whatever. 
                                          #The code did turn the LED on once and then it did stay off, but it gave me a message that 
                                          # the  program is still running.  I did not think that programs actually did stop until the
                                          # power was removed.

Thanks
Tom
 		 	   		  

From joel.goldstick at gmail.com  Thu Feb  4 19:13:19 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Thu, 4 Feb 2016 19:13:19 -0500
Subject: [Tutor] Help needed
In-Reply-To: <BLU170-W546658256FE612B00C8EF2A9D10@phx.gbl>
References: <BLU170-W546658256FE612B00C8EF2A9D10@phx.gbl>
Message-ID: <CAPM-O+zYZDaMd8rpZT=oy6DEBJ15GxzuruEbpLkKWW63DwyCCw@mail.gmail.com>

On Thu, Feb 4, 2016 at 6:26 PM, Tom Brodle <tbrodle at hotmail.com> wrote:

> I am very new  to Python.  I am just practicing code using a simple plug
> board.  The code is working as I want it to but the last message and the
> original looping makes me have questions.  Because the main code works I
> will just abbreviate it.
>
> import GPIO
> import time
> Gpio setmode (BCM)
> led_pin = 18
> GPIO setup(led_pin, GPIO)
>
> try:
>         While true:
>

Actually you need:
            While True:

This means run forever.  In the while loop you need some test to break or
this loop will run forever.  You can press Ctl-C to quit the program



>             Turn LED on
>             sleep 5 seconds
>             Turn LED off
>             sleep .5 seconds  # program was initially designed to keep LED
> turning on and off and that worked.
>                                            # Coming from the world of
> Basic I did/do not understand what made the program loop
>                                            # back to while True: line of
> code.
>                                            # I  wanted the LED to turn on
> and then off and stay off, so from some Python.org
>                                            # page I became aware of
> quit(code=None) and inserted it before the line Print("Cleaning up")
>           quit(code=None)            # I put this line in to stop the
> looping back to While true:.
>     Print("Cleaning up")
>     GPIO.cleanup()           # I was told to put this line in but I do not
> know where it goes to do whatever.
>                                           #The code did turn the LED on
> once and then it did stay off, but it gave me a message that
>                                           # the  program is still
> running.  I did not think that programs actually did stop until the
>                                           # power was removed.
>
> Thanks
> Tom
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From alan.gauld at btinternet.com  Thu Feb  4 19:38:24 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 5 Feb 2016 00:38:24 +0000
Subject: [Tutor] Help needed
In-Reply-To: <BLU170-W546658256FE612B00C8EF2A9D10@phx.gbl>
References: <BLU170-W546658256FE612B00C8EF2A9D10@phx.gbl>
Message-ID: <n90qu0$otc$1@ger.gmane.org>

On 04/02/16 23:26, Tom Brodle wrote:

>         While true:
>             Turn LED on
>             sleep 5 seconds
>             Turn LED off
>             sleep .5 seconds  

> # Coming from the world of Basic I did/do not understand what made the program loop
> # back to while True: line of code.

Its the indentation
In BASIC you would write

WHILE TRUE
   TURN LED ON
   SLEEP 5s
   TURN LED OFF
   Sleep 0.5s
WEND

In Python the Wend is implicit in the indentation.

> # I  wanted the LED to turn on and then off and stay off, 

In that case you didn't want a while loop, you just
wanted a sequence:

Turn LED on
sleep 5 seconds
Turn LED off

> # page I became aware of quit(code=None) and inserted it 

That quits the entire program.
That's OK if that's what you want but if yyou need to do
something else after wards its not too helpful.

If you really want to break out of a 'while True' loop you
need to use the 'break' statement, usually inside some
kind of if statement

while True
    if some condition:
       break   # exit the loop
    do whatever the loop does


>     GPIO.cleanup()           
> # I was told to put this line in but I do not know where it goes

Probably after the loop - that is without any indentation.
You are best to put it in a 'finally' clause like this

try:
   while True:
       loop code here
finally:      # always executed
   GPIO.cleanup()

Then even if you hit an error condition that forces an exit
from the loop the finally is guaranteed to be executed.

> #The code did turn the LED on once and then it did stay off, 
> but it gave me a message that the  program is still running.

> I did not think that programs actually did stop until the
> power was removed.

Probably not because the while never exits.
BTW the LED is not really staying on from the look of things.
Its turning on for 5 seconds then off for 0.5s. Its just that
the off is too short to notice.

For more about Python loops you can check out my tutorial below.
(While loops are coverd about half way down the loops topic)
If you know BASIC the examples are nearly all repeated in
VBScript (a form of BASIC) so you should recognise the patterns.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From ml1 at bitbert.com  Thu Feb  4 21:03:14 2016
From: ml1 at bitbert.com (noopy)
Date: Fri, 05 Feb 2016 03:03:14 +0100
Subject: [Tutor] recursion
Message-ID: <593bcef92a05d78d3ef79e3bd37f3444@bitbert.com>

Hi,

I just cannot get my head around the following code, maybe someone could 
explain it to me.

def permutations(items):
     n = len(items)
     if n==0: yield []
     else:
         for i in range(len(items)):
             for cc in permutations(items[:i]+items[i+1:]):
                 yield [items[i]]+cc

for p in permutations(list("XYZ")):
     print(''.join(p))


I did not figure out how this works in the deepest level of the 
recursion. I mean, when I go down the recursion level
and do the part "for cc in permutations("Z")".

I have a problem with the following understanding:
when I am in the recursion and call "for cc in permutations("Z")", the 
next level of the recursion will do "for cc in permutations([])", which 
will yield an empty list.

But when "for cc in permutations([])" yields an empty list, why does 
"for cc in permutations("Z")" then actually have an item so that "yield 
[items[i]]+cc" will be executed?
So although "for cc in permutations("Z")" does have an item it does 
"yield [items[i]]+cc" - thats the confusion for me. I hope I could 
express myself clearly.

Maybe someone could enlighten me, I would be more than happy since I 
have been thinking for quite some time about this and it does not let me 
sleep, hehe.

thanks in advance,

noopy





From ben+python at benfinney.id.au  Fri Feb  5 03:34:28 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 05 Feb 2016 19:34:28 +1100
Subject: [Tutor] recursion
References: <593bcef92a05d78d3ef79e3bd37f3444@bitbert.com>
Message-ID: <85wpqja73f.fsf@benfinney.id.au>

noopy via Tutor <tutor at python.org> writes:

> But when "for cc in permutations([])" yields an empty list, why does
> "for cc in permutations("Z")" then actually have an item so that
> "yield [items[i]]+cc" will be executed?

Does this help::

    >>> i = 1
    >>> "Z"[:i] + "Z"[i+1:]
    'Z'

Can you explain why that would be the case?

-- 
 \         ?I think a good gift for the President would be a chocolate |
  `\   revolver. And since he's so busy, you'd probably have to run up |
_o__)              to him real quick and hand it to him.? ?Jack Handey |
Ben Finney


From alan.gauld at btinternet.com  Fri Feb  5 03:42:10 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 5 Feb 2016 08:42:10 +0000
Subject: [Tutor] recursion
In-Reply-To: <593bcef92a05d78d3ef79e3bd37f3444@bitbert.com>
References: <593bcef92a05d78d3ef79e3bd37f3444@bitbert.com>
Message-ID: <n91n91$q53$1@ger.gmane.org>

On 05/02/16 02:03, noopy via Tutor wrote:

> I just cannot get my head around the following code, maybe someone could 
> explain it to me.

I'll try but it is a little bit tricky.

> def permutations(items):
>      n = len(items)
>      if n==0: yield []
>      else:

I assume this bit is clear enough?

>          for i in range(len(items)):
>              for cc in permutations(items[:i]+items[i+1:]):
>                  yield [items[i]]+cc

This bit is where the tricky stuff happens.
The key is in the slicing of items.
items[:i]+items[i+1:] effectively removes item[i] from the list.
So the 'for cc' line is saying:
  find the permutations of everything except item[i].
  then add items[i] to each of those permutations.
So for a single element it returns that element plus the empty list.

The recursion then repeats that for the reduced list.
So using XYZ as in your code:
for the first iteration of XYZ
   the first level calls for permutations of YZ and
   the second level calls for permutations of Z followed by Y
It then yields X plus those permutations - XYZ, XZY
for the second iteration of XYZ
   the first level calls for permutations of XZ and
   the second level calls for permutations of Z followed by X
It then yields Y plus those permutations - YXZ, YZX
for the third iteration of XYZ
   the first level calls for permutations of XY and
   the second level calls for permutations of X followed by Y
It then yields Z plus those permutations - ZXY, ZYX

I may have the return orders wrong because I didn't actually run it.
But that's the basic concept.

> for p in permutations(list("XYZ")):
>      print(''.join(p))

And this loop just joins the list back into strings.

> I have a problem with the following understanding:
> when I am in the recursion and call "for cc in permutations("Z")", the 
> next level of the recursion will do "for cc in permutations([])", which 
> will yield an empty list.

Yes, but the previous level then yields Z plus an empty list.

> So although "for cc in permutations("Z")" does have an item it does 
> "yield [items[i]]+cc" - thats the confusion for me. I hope I could 
> express myself clearly.

It is confusing. Maybe the easiest way to see it would be to
run it in a debugger with a break point set on the for cc
loop and look at items each time it stops.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From ben+python at benfinney.id.au  Fri Feb  5 04:07:29 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 05 Feb 2016 20:07:29 +1100
Subject: [Tutor] recursion
References: <593bcef92a05d78d3ef79e3bd37f3444@bitbert.com>
 <n91n91$q53$1@ger.gmane.org>
Message-ID: <85lh6za5ke.fsf@benfinney.id.au>

Alan Gauld <alan.gauld at btinternet.com> writes:

> On 05/02/16 02:03, noopy via Tutor wrote:
>
> > def permutations(items):
> >      n = len(items)
> >      if n==0: yield []
> >      else:
>
> I assume this bit is clear enough?

I think it would be clearer without the needless opaque name ?n?.
Better::

    def permutations(items):
        if not items:
            # ?items? is empty (or is not a container).
            yield []
        else:
            for i in range(len(items)):
                ?

Binding a name that is used exactly once should be done only if the name
clarifies the purpose. An opaque name like ?n? is not helpful.

-- 
 \     ?Why doesn't Python warn that it's not 100% perfect? Are people |
  `\         just supposed to ?know? this, magically?? ?Mitya Sirenef, |
_o__)                                     comp.lang.python, 2012-12-27 |
Ben Finney


From __peter__ at web.de  Fri Feb  5 04:39:19 2016
From: __peter__ at web.de (Peter Otten)
Date: Fri, 05 Feb 2016 10:39:19 +0100
Subject: [Tutor] recursion
References: <593bcef92a05d78d3ef79e3bd37f3444@bitbert.com>
 <n91n91$q53$1@ger.gmane.org> <85lh6za5ke.fsf@benfinney.id.au>
Message-ID: <n91qk9$g75$1@ger.gmane.org>

Ben Finney wrote:

> Alan Gauld <alan.gauld at btinternet.com> writes:
> 
>> On 05/02/16 02:03, noopy via Tutor wrote:
>>
>> > def permutations(items):
>> >      n = len(items)
>> >      if n==0: yield []
>> >      else:
>>
>> I assume this bit is clear enough?
> 
> I think it would be clearer without the needless opaque name ?n?.
> Better::
> 
>     def permutations(items):
>         if not items:
>             # ?items? is empty (or is not a container).
>             yield []
>         else:
>             for i in range(len(items)):
>                 ?
> 
> Binding a name that is used exactly once should be done only if the name
> clarifies the purpose. An opaque name like ?n? is not helpful.

While you're at it you can also throw out the range(len(...)) construct:

 
>>> def p(items):
...     if items:
...         for i, item in enumerate(items):
...             for cc in p(items[:i] + items[i+1:]):
...                 yield [item] + cc
...     else:
...         yield []
... 
>>> list(p("xyz"))
[['x', 'y', 'z'], ['x', 'z', 'y'], ['y', 'x', 'z'], ['y', 'z', 'x'], ['z', 
'x', 'y'], ['z', 'y', 'x']]



From richkappler at gmail.com  Fri Feb  5 11:25:50 2016
From: richkappler at gmail.com (richard kappler)
Date: Fri, 5 Feb 2016 11:25:50 -0500
Subject: [Tutor] script follows a file into zip?
Message-ID: <CAG7edPH6oPeuUTU+J8Z2THs7zw1KOKfLqVAZJ_OqLPKmXGpJMw@mail.gmail.com>

I have a script that checks a file and if there are new additions to the
file, parses and sends the new additions. This log rolls over and zips at
midnight creating a new log of the same name. The problem is, when the file
rolls over, the new file seems to follow the old file the way the script is
currently written.

Current script (edited for brevity):
#######################################
from time import sleep

f1 = open('theFile.log', 'r')

f1.seek(0,2)
eof = f1.tell()

While True:
    try:
        #check file size to see if grown, set neweof
        f1.seek(0,2)
        neweof = f1.tell()
    except ValueError:
        f1 = open(rd1, 'r')
        f1.seek(0,2)
        neweof = f1.tell()

    #if the file is larger...
    if neweof > eof:
        do a bunch of stuff

        # update log.txt file size
        eof = neweof
        time.sleep(10)

    elif neweof < eof:
        # this resets eof at night when old log file zipped and new log
file started
        eof = 0
        time.sleep(10)

    elif neweof == eof:
         # file hasn't changed, do nothing
         time.sleep(10)
#############################################

The script works great all day until the logrotate at midnight. I would
expect to get the elif neweof < eof bit then, but my log shows I'm getting
elif neweof == eof for the rest of the night, which is what leads me to
believe that the script, as written, is following the zipped file, not
looking at the new file.

I have two thoughts on this, not sure which to use, or if something else
completely might be appropriate.
1. Instead of

f1 = open('theFile.log', 'r')

while True:
....

Would it make a difference if I used

with open('theFile.log', 'r') as f1:
    while True:
and so on

2. Get rid of the elif neweof == eof statement and just make it
else:
   pass

to send it back to the beginning of the loop (doesn't make sense to me, but
wanted to throw it out there).

regards, Richard
-- 

No matter where you go, there you are. - Buckaroo Banzai

From __peter__ at web.de  Fri Feb  5 12:04:33 2016
From: __peter__ at web.de (Peter Otten)
Date: Fri, 05 Feb 2016 18:04:33 +0100
Subject: [Tutor] script follows a file into zip?
References: <CAG7edPH6oPeuUTU+J8Z2THs7zw1KOKfLqVAZJ_OqLPKmXGpJMw@mail.gmail.com>
Message-ID: <n92kn3$8ss$1@ger.gmane.org>

richard kappler wrote:

> I have a script that checks a file and if there are new additions to the
> file, parses and sends the new additions. This log rolls over and zips at
> midnight creating a new log of the same name. The problem is, when the
> file rolls over, the new file seems to follow the old file the way the
> script is currently written.
> 
> Current script (edited for brevity):
> #######################################
> from time import sleep
> 
> f1 = open('theFile.log', 'r')
> 
> f1.seek(0,2)
> eof = f1.tell()
> 
> While True:
>     try:
>         #check file size to see if grown, set neweof
>         f1.seek(0,2)
>         neweof = f1.tell()
>     except ValueError:
>         f1 = open(rd1, 'r')
>         f1.seek(0,2)
>         neweof = f1.tell()
> 
>     #if the file is larger...
>     if neweof > eof:
>         do a bunch of stuff
> 
>         # update log.txt file size
>         eof = neweof
>         time.sleep(10)
> 
>     elif neweof < eof:
>         # this resets eof at night when old log file zipped and new log
> file started
>         eof = 0
>         time.sleep(10)
> 
>     elif neweof == eof:
>          # file hasn't changed, do nothing
>          time.sleep(10)
> #############################################
> 
> The script works great all day until the logrotate at midnight. I would
> expect to get the elif neweof < eof bit then, but my log shows I'm getting
> elif neweof == eof for the rest of the night, which is what leads me to
> believe that the script, as written, is following the zipped file, not
> looking at the new file.
> 
> I have two thoughts on this, not sure which to use, or if something else
> completely might be appropriate.
> 1. Instead of
> 
> f1 = open('theFile.log', 'r')
> 
> while True:
> ....
> 
> Would it make a difference if I used
> 
> with open('theFile.log', 'r') as f1:
>     while True:
> and so on

No. The problem is that you keep the file open. When the file is overwritten 
by another application only the entry in the file system is replaced while 
the underlying file (identified by its inode) is kept.

The file you see has now no entry in the file system and therefore could 
only be changed by you; as you only read it no more changes will happen.

To see the file that is  is currently registered under the name 
"theFile.log" you have to reopen:

while True:
    with open("theFile.log") as f1:
        ...

 
> 2. Get rid of the elif neweof == eof statement and just make it
> else:
>    pass
> 
> to send it back to the beginning of the loop (doesn't make sense to me,
> but wanted to throw it out there).
> 
> regards, Richard



From richkappler at gmail.com  Fri Feb  5 12:10:06 2016
From: richkappler at gmail.com (richard kappler)
Date: Fri, 5 Feb 2016 12:10:06 -0500
Subject: [Tutor] script follows a file into zip?
In-Reply-To: <n92kn3$8ss$1@ger.gmane.org>
References: <CAG7edPH6oPeuUTU+J8Z2THs7zw1KOKfLqVAZJ_OqLPKmXGpJMw@mail.gmail.com>
 <n92kn3$8ss$1@ger.gmane.org>
Message-ID: <CAG7edPHDCZhp1b+bH2oh0DAnj+SaMbfCm1P34s+hJcLMhRhJ+A@mail.gmail.com>

Ah, I see, makes perfect sense now that it's right under my nose. Thanks
Peter!

regards, Richard

On Fri, Feb 5, 2016 at 12:04 PM, Peter Otten <__peter__ at web.de> wrote:

> richard kappler wrote:
>
> > I have a script that checks a file and if there are new additions to the
> > file, parses and sends the new additions. This log rolls over and zips at
> > midnight creating a new log of the same name. The problem is, when the
> > file rolls over, the new file seems to follow the old file the way the
> > script is currently written.
> >
> > Current script (edited for brevity):
> > #######################################
> > from time import sleep
> >
> > f1 = open('theFile.log', 'r')
> >
> > f1.seek(0,2)
> > eof = f1.tell()
> >
> > While True:
> >     try:
> >         #check file size to see if grown, set neweof
> >         f1.seek(0,2)
> >         neweof = f1.tell()
> >     except ValueError:
> >         f1 = open(rd1, 'r')
> >         f1.seek(0,2)
> >         neweof = f1.tell()
> >
> >     #if the file is larger...
> >     if neweof > eof:
> >         do a bunch of stuff
> >
> >         # update log.txt file size
> >         eof = neweof
> >         time.sleep(10)
> >
> >     elif neweof < eof:
> >         # this resets eof at night when old log file zipped and new log
> > file started
> >         eof = 0
> >         time.sleep(10)
> >
> >     elif neweof == eof:
> >          # file hasn't changed, do nothing
> >          time.sleep(10)
> > #############################################
> >
> > The script works great all day until the logrotate at midnight. I would
> > expect to get the elif neweof < eof bit then, but my log shows I'm
> getting
> > elif neweof == eof for the rest of the night, which is what leads me to
> > believe that the script, as written, is following the zipped file, not
> > looking at the new file.
> >
> > I have two thoughts on this, not sure which to use, or if something else
> > completely might be appropriate.
> > 1. Instead of
> >
> > f1 = open('theFile.log', 'r')
> >
> > while True:
> > ....
> >
> > Would it make a difference if I used
> >
> > with open('theFile.log', 'r') as f1:
> >     while True:
> > and so on
>
> No. The problem is that you keep the file open. When the file is
> overwritten
> by another application only the entry in the file system is replaced while
> the underlying file (identified by its inode) is kept.
>
> The file you see has now no entry in the file system and therefore could
> only be changed by you; as you only read it no more changes will happen.
>
> To see the file that is  is currently registered under the name
> "theFile.log" you have to reopen:
>
> while True:
>     with open("theFile.log") as f1:
>         ...
>
>
> > 2. Get rid of the elif neweof == eof statement and just make it
> > else:
> >    pass
> >
> > to send it back to the beginning of the loop (doesn't make sense to me,
> > but wanted to throw it out there).
> >
> > regards, Richard
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 

No matter where you go, there you are. - Buckaroo Banzai

From bachir_bac at yahoo.com  Sat Feb  6 01:47:37 2016
From: bachir_bac at yahoo.com (Bachir Bachir)
Date: Sat, 6 Feb 2016 06:47:37 +0000 (UTC)
Subject: [Tutor] installing rtree on python 34
References: <1134170754.125425.1454741257334.JavaMail.yahoo.ref@mail.yahoo.com>
Message-ID: <1134170754.125425.1454741257334.JavaMail.yahoo@mail.yahoo.com>

?Dear all,I am using winpython 34 , could anyone help installing rtree please?It seems rtree loading libspatialindex , i donwnloaded the libspatial index but rtree still cant find the files locationthanks much for your helpsincerelyBachir

From alan.gauld at btinternet.com  Sat Feb  6 03:49:28 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 6 Feb 2016 08:49:28 +0000
Subject: [Tutor] installing rtree on python 34
In-Reply-To: <1134170754.125425.1454741257334.JavaMail.yahoo@mail.yahoo.com>
References: <1134170754.125425.1454741257334.JavaMail.yahoo.ref@mail.yahoo.com>
 <1134170754.125425.1454741257334.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <n94c2n$4le$1@ger.gmane.org>

On 06/02/16 06:47, Bachir Bachir via Tutor wrote:
>  Dear all,I am using winpython 34 , could anyone help installing rtree please?
> It seems rtree loading libspatialindex ,
> i donwnloaded the libspatial index but rtree still cant find
> the files locationthanks much for your helpsincerelyBachir

This is a bit off topic for the tutor list so you might get a better
response on an rtree or libspatial specific forum or, failing that, on
the general python mailing list/newsgroup.

However, it will help if you tell us how you are installing both rtree
and libspatial. Are you using binary installers or using pip? Show us
the exact command you typed.

Telling us the OS might help too, although I assume its some
kind of Windows.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From bachir_bac at yahoo.com  Sat Feb  6 04:35:25 2016
From: bachir_bac at yahoo.com (Bachir Bachir)
Date: Sat, 6 Feb 2016 09:35:25 +0000 (UTC)
Subject: [Tutor] installing rtree on python 34
In-Reply-To: <n94c2n$4le$1@ger.gmane.org>
References: <n94c2n$4le$1@ger.gmane.org>
Message-ID: <451246886.169748.1454751325210.JavaMail.yahoo@mail.yahoo.com>

Hello AlainI am using windows10 64 bits, my os.name is 'nt', I?used pip to ?install but its complaining about the spatialindex_c.dll? ? ?File "C:\Users\Bachir\AppData\Local\Temp\pip-build-td64lrth\rtree\rtree\core.py", line 101, in <module>
? ? ? ?raise OSError("could not find or load spatialindex_c.dll")? ?OSError: could not find or load spatialindex_c.dll
I downloaded rtree-0.8.2 and then run python setup.py install and i have the same message?
?PS C:\Users\Bachir\documents\Python Scripts\Rtree-0.8.2> python setup.py installTraceback (most recent call last):? File "setup.py", line 4, in <module>? ? import rtree? File "C:\Users\Bachir\documents\Python Scripts\Rtree-0.8.2\rtree\__init__.py", line 1, in <module>? ? from .index import Rtree? File "C:\Users\Bachir\documents\Python Scripts\Rtree-0.8.2\rtree\index.py", line 6, in <module>? ? from . import core? File "C:\Users\Bachir\documents\Python Scripts\Rtree-0.8.2\rtree\core.py", line 101, in <module>? ? raise OSError("could not find or load spatialindex_c.dll")OSError: could not find or load spatialindex_c.dllPS C:\Users\Bachir\documents\Python Scripts\Rtree-0.8.2> ?
i donwloaded the spatialindex dll files ' libspatialindex-1.8.1-win-msvc-2010-x64-x32.zip ' . this file contain both 32 ?and 64 bits , ?unzip and put in the installation folder, when installing using python setuo.py install it still complaining about the spatialindex dll fileThanks much?
Bachir?

 

    On Saturday, February 6, 2016 9:49 AM, Alan Gauld <alan.gauld at btinternet.com> wrote:
 

 On 06/02/16 06:47, Bachir Bachir via Tutor wrote:
>? Dear all,I am using winpython 34 , could anyone help installing rtree please?
> It seems rtree loading libspatialindex ,
> i donwnloaded the libspatial index but rtree still cant find
> the files locationthanks much for your helpsincerelyBachir

This is a bit off topic for the tutor list so you might get a better
response on an rtree or libspatial specific forum or, failing that, on
the general python mailing list/newsgroup.

However, it will help if you tell us how you are installing both rtree
and libspatial. Are you using binary installers or using pip? Show us
the exact command you typed.

Telling us the OS might help too, although I assume its some
kind of Windows.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist? -? Tutor at python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


  

From anubhav1691 at gmail.com  Sat Feb  6 11:07:32 2016
From: anubhav1691 at gmail.com (Anubhav Yadav)
Date: Sat, 6 Feb 2016 21:37:32 +0530
Subject: [Tutor] Need help with two similar test cases that I have written.
 One works and the other fails
Message-ID: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>

Hello Everyone,

I am trying to write a simple program that generated random numbers based
on some rules.
when my program is in the yellow state, it should generate the numbers
from 100.0-100.5
and 102.5-103.0. When my program is in the red state, it should generate
numbers in the range
less than 99.9 and greater than 103.1.

I have written two very simple functions, and two unittest.TestCase test
cases to test the two functions. I somehow feel that both my test cases
doesn't make any sense, yet one of it passes perfectly and other one does
not pass.

Here is my code:

import random
import unittest

red_temperature_min = 99.9
red_temperature_max = 103.0
normal_temperature_min = 101.0
normal_temperature_max = 102.0
yellow_temperature_min = 100.0
yellow_temperature_max = 103.0

def red_temperature_simulation():
    """
    Method to simulate the red temperature condition
    """
    choice = random.randint(0,1)
    if choice:
        return round(random.uniform(red_temperature_min - 5.0,
                                red_temperature_min))
    else:
        return round(random.uniform(red_temperature_max,
                                red_temperature_max + 5.0))

def yellow_temperature_simulation():
    """
    Method to simulate the yellow temperature condition
    """
    choice = random.randint(0,1)
    if choice:
        return round(random.uniform(yellow_temperature_min,
                                    normal_temperature_min - 0.5), 2)
    else:
        return round(random.uniform(normal_temperature_max + 0.5,
                                    yellow_temperature_max), 2)


class TestCase(unittest.TestCase):
    def test_red_temperature_simulation(self):
        """
        Method to test the red_temperature_simulation method
        """
        for i in range(100000):
            self.assertLess(red_temperature_simulation(), 100.0) and \
                self.assertGreater(red_temperature_simulation(), 103.0)

    def test_yellow_temperature_simulation(self):
        """
        Method to test the yellow_temperature_simulation method
        """
        for i in range(1000000):
            (self.assertGreaterEqual(yellow_temperature_simulation(),
                                 100.0)) and \
            (self.assertLessEqual(yellow_temperature_simulation(),
                              100.5)) and \
            (self.assertGreaterEqual(yellow_temperature_simulation(),
                                 102.5)) and \
            (self.assertLessEqual(yellow_temperature_simulation(), 103.0))



I try to test if a number 99.7 is less than 100.0 and at the same time I
also check if it's greater than 102.5. The sentence itself doesn't make
sense, but somehow the test case for yellow passes, but the test case for
red fails. I have also tried using and instead of or!

I want to test my code, is there a better way of doing it?

Thank You.

From danny.yoo at gmail.com  Sat Feb  6 16:34:29 2016
From: danny.yoo at gmail.com (Danny Yoo)
Date: Sat, 6 Feb 2016 13:34:29 -0800
Subject: [Tutor] Need help with two similar test cases that I have
 written. One works and the other fails
In-Reply-To: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>
References: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>
Message-ID: <CAGZAPF7-Q+R4NE+iKdN9rgFcOFUZbpvxa0tAUh-bDL7exXZ2Jg@mail.gmail.com>

On Feb 6, 2016 12:31 PM, "Anubhav Yadav" <anubhav1691 at gmail.com> wrote:
>
> Hello Everyone,
>
> I am trying to write a simple program that generated random numbers based
> on some rules.
> when my program is in the yellow state, it should generate the numbers
> from 100.0-100.5
> and 102.5-103.0. When my program is in the red state, it should generate
> numbers in the range
> less than 99.9 and greater than 103.1.
>
> I have written two very simple functions, and two unittest.TestCase test
> cases to test the two functions. I somehow feel that both my test cases
> doesn't make any sense, yet one of it passes perfectly and other one does
> not pass.

Hi Anubhav,

Ah!  The assert functions are meant to be used as statements, not as
composable expressions.  If you're familiar with the idea of side effects,
then you need to understand that you should be calling the assert functions
just for their side effects, not for their return value.

If you want to test two conditions, do them as  separate statements.  By
trying to compose them with 'and', the code actually means something
different due to boolean logic short circuiting.

If you have questions, please feel free to ask.  Good luck!

From robertvstepp at gmail.com  Sat Feb  6 16:54:52 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 6 Feb 2016 15:54:52 -0600
Subject: [Tutor] Need help with two similar test cases that I have
 written. One works and the other fails
In-Reply-To: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>
References: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>
Message-ID: <CANDiX9KF2GagmVNrq5+yvCTfxYCURab0FMv4Bmo2TraWcb1j2g@mail.gmail.com>

On Sat, Feb 6, 2016 at 10:07 AM, Anubhav Yadav <anubhav1691 at gmail.com> wrote:


> class TestCase(unittest.TestCase):
>     def test_red_temperature_simulation(self):
>         """
>         Method to test the red_temperature_simulation method
>         """
>         for i in range(100000):
>             self.assertLess(red_temperature_simulation(), 100.0) and \
>                 self.assertGreater(red_temperature_simulation(), 103.0)

Is this really what you want (And similarly in your other test
method.)?  If I am reading things correctly, you are calling
red_temperature_simulation *twice*, which will *usually* give you two
*separate* values, which you then attempt to compare.  Shouldn't you
first do:

for i in range(100000):
    value_to_test = red_temperature_simulation()
    (self.assertLess(value_to_test, 100.0) <Which logic op?>
        self.assertGreater(value_to_test, 103.0))
?

If my understanding is indeed correct, then I will leave it to you to
figure out which logic operator ("and" or "or") makes sense here!
~(:>))



-- 
boB

From robertvstepp at gmail.com  Sat Feb  6 17:00:34 2016
From: robertvstepp at gmail.com (boB Stepp)
Date: Sat, 6 Feb 2016 16:00:34 -0600
Subject: [Tutor] Need help with two similar test cases that I have
 written. One works and the other fails
In-Reply-To: <CANDiX9KF2GagmVNrq5+yvCTfxYCURab0FMv4Bmo2TraWcb1j2g@mail.gmail.com>
References: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>
 <CANDiX9KF2GagmVNrq5+yvCTfxYCURab0FMv4Bmo2TraWcb1j2g@mail.gmail.com>
Message-ID: <CANDiX9Lzsfe341Hsa-AY8U6YNB5E00GF36u0tFp8SqCLMTB_=w@mail.gmail.com>

On Sat, Feb 6, 2016 at 3:54 PM, boB Stepp <robertvstepp at gmail.com> wrote:
> On Sat, Feb 6, 2016 at 10:07 AM, Anubhav Yadav <anubhav1691 at gmail.com> wrote:

Just read Danny's reply after sending mine, which means that the
answer to my question:

> If my understanding is indeed correct, then I will leave it to you to
> figure out which logic operator ("and" or "or") makes sense here!
> ~(:>))

must be neither one!  Thanks, Danny!

However, I do believe (Until proven otherwise.) my first point is still valid.

-- 
boB

From danny.yoo at gmail.com  Sat Feb  6 18:02:51 2016
From: danny.yoo at gmail.com (Danny Yoo)
Date: Sat, 6 Feb 2016 15:02:51 -0800
Subject: [Tutor] recursion
In-Reply-To: <593bcef92a05d78d3ef79e3bd37f3444@bitbert.com>
References: <593bcef92a05d78d3ef79e3bd37f3444@bitbert.com>
Message-ID: <CAGZAPF4kq_uKfVFFk6_7qfk=J9OyBHOJvCdFGaDbsd8T0DVKeg@mail.gmail.com>

On Feb 5, 2016 12:07 AM, "noopy via Tutor" <tutor at python.org> wrote:
>
> Hi,
>
> I just cannot get my head around the following code, maybe someone could
explain it to me.
>
> def permutations(items):

When trying to understand a function (or in this case, a generator),
knowing the types of input and output can be helpful.  Also, knowing some
sample outputs for small inputs is also important.

Without even looking at the implementation, can you say in words what this
function is intended to do? What's the type of the input?  What's the type
of the output? And can you give a few concrete examples of what you'd like
it to do on "small" inputs?

(Trying to demonstrate a general problem strategy that works surprisingly
well, and that we actually use in practice.)

From dyoo at hashcollision.org  Sat Feb  6 20:43:41 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Sat, 6 Feb 2016 17:43:41 -0800
Subject: [Tutor] recursion
In-Reply-To: <593bcef92a05d78d3ef79e3bd37f3444@bitbert.com>
References: <593bcef92a05d78d3ef79e3bd37f3444@bitbert.com>
Message-ID: <CAGZAPF74zgCCYYF2e5fs_iJgiNAc6kqmbP-ZD+-p74Ptk1PMXA@mail.gmail.com>

On Thu, Feb 4, 2016 at 6:03 PM, noopy via Tutor <tutor at python.org> wrote:
> Hi,
>
> I just cannot get my head around the following code, maybe someone could
> explain it to me.

One thing to note: the function here is a generator, which is itself
an intermediate subject that's specific to Python.  Recursion is a
more general concept that's shared by a lot of other programming
languages.  Tackling generators at the same time as recursion may make
both subjects harder to learn at first.



You might want to consider a version of permutations that doesn't do
any generator-related stuff.  Here's a version of permutations that's
a regular function:

##############################################
def permutations(items):
    n = len(items)
    if n == 0:
        return [[]]
    else:
        allPerms = []
        for i in range(len(items)):
            for cc in permutations(items[:i] + items[i+1:]):
                allPerms.append([items[i]] + cc)
        return allPerms
##############################################

This version should be a bit easier to understand, since we don't have
to deal with generators at the same time.


Also, we often have the choice of defining what the function should do
for "small" base cases.  Zero isn't always the most comprehensible
case to start with.  If it helps to make the function simpler to
understand, start with a list of length 1 or 2.  You don't have to
make the function's basis something that doesn't make sense to you.

What should the answer to permutations(['x']) be, for example?

I think it should be:

    [['x']]

Do you agree?  Did you have something else in mind?


Another question:  What should the answer to permutations(['x', 'y']) be?


It's very easy to write code that's busy or using CPU cycles.  It's
harder to tell whether it's doing something right.  To solve that
problem, we should build up a repertoire of known good solutions:
they'll make great test cases.

From anubhav1691 at gmail.com  Sun Feb  7 01:11:24 2016
From: anubhav1691 at gmail.com (Anubhav Yadav)
Date: Sun, 7 Feb 2016 11:41:24 +0530
Subject: [Tutor] Need help with two similar test cases that I have
 written. One works and the other fails
In-Reply-To: <CAGZAPF7-Q+R4NE+iKdN9rgFcOFUZbpvxa0tAUh-bDL7exXZ2Jg@mail.gmail.com>
References: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>
 <CAGZAPF7-Q+R4NE+iKdN9rgFcOFUZbpvxa0tAUh-bDL7exXZ2Jg@mail.gmail.com>
Message-ID: <CA+Jf9AHRHEg0rU1_Jts2cjx0mNbwtB3sK8h=7gUA1uyEks7V6A@mail.gmail.com>

> Hi Anubhav,
>
> Ah!  The assert functions are meant to be used as statements, not as
> composable expressions.  If you're familiar with the idea of side effects,
> then you need to understand that you should be calling the assert functions
> just for their side effects, not for their return value.
>
> If you want to test two conditions, do them as  separate statements.  By
> trying to compose them with 'and', the code actually means something
> different due to boolean logic short circuiting.
>
> If you have questions, please feel free to ask.  Good luck!
>

Hi,

Thanks a lot for replying back. I should do something like this right?
truth_value = yellow_temperature_simulation() <= 100.5 and
yellow_temperature_simulation() >= 100.0
self.assertTrue(truth_value)

I just want to confirm if I get the concept right?

From __peter__ at web.de  Sun Feb  7 04:52:25 2016
From: __peter__ at web.de (Peter Otten)
Date: Sun, 07 Feb 2016 10:52:25 +0100
Subject: [Tutor] Need help with two similar test cases that I have
 written. One works and the other fails
References: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>
 <CAGZAPF7-Q+R4NE+iKdN9rgFcOFUZbpvxa0tAUh-bDL7exXZ2Jg@mail.gmail.com>
 <CA+Jf9AHRHEg0rU1_Jts2cjx0mNbwtB3sK8h=7gUA1uyEks7V6A@mail.gmail.com>
Message-ID: <n9744s$doc$1@ger.gmane.org>

Anubhav Yadav wrote:

>> Hi Anubhav,
>>
>> Ah!  The assert functions are meant to be used as statements, not as
>> composable expressions.  If you're familiar with the idea of side
>> effects, then you need to understand that you should be calling the
>> assert functions just for their side effects, not for their return value.
>>
>> If you want to test two conditions, do them as  separate statements.  By
>> trying to compose them with 'and', the code actually means something
>> different due to boolean logic short circuiting.
>>
>> If you have questions, please feel free to ask.  Good luck!
>>
> 
> Hi,
> 
> Thanks a lot for replying back. I should do something like this right?
> truth_value = yellow_temperature_simulation() <= 100.5 and
> yellow_temperature_simulation() >= 100.0
> self.assertTrue(truth_value)
> 
> I just want to confirm if I get the concept right?

No, you want to check all constrainsts for one value. The way you write it 
above you check if one value is below 100.5 and another is above 100.0.

In your previous post you state

> when my program is in the yellow state, it should generate the numbers
> from 100.0-100.5
> and 102.5-103.0

The "and" is a bit misleading as you want to allow values that are in the 
first *or* the second intrval. When you spell that in Python it becomes 

temperature = yellow_temperature_simulation()
self.assertTrue(
    (100.0 <= temperature < 100.5) 
    or (102.5 <= temperature < 103.0))

The extra parentheses aren't required.
If you don't want half-open intervals replace < with <=.


From anubhav1691 at gmail.com  Sun Feb  7 06:23:07 2016
From: anubhav1691 at gmail.com (Anubhav Yadav)
Date: Sun, 7 Feb 2016 16:53:07 +0530
Subject: [Tutor] Need help with two similar test cases that I have
 written. One works and the other fails
In-Reply-To: <n9744s$doc$1@ger.gmane.org>
References: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>
 <CAGZAPF7-Q+R4NE+iKdN9rgFcOFUZbpvxa0tAUh-bDL7exXZ2Jg@mail.gmail.com>
 <CA+Jf9AHRHEg0rU1_Jts2cjx0mNbwtB3sK8h=7gUA1uyEks7V6A@mail.gmail.com>
 <n9744s$doc$1@ger.gmane.org>
Message-ID: <CA+Jf9AFOrd7OOAQkDgF4DeQVZ7OQsE0dQUD_NJDSRHKW_1Vf7A@mail.gmail.com>

> > when my program is in the yellow state, it should generate the numbers
> > from 100.0-100.5
> > and 102.5-103.0
>
> The "and" is a bit misleading as you want to allow values that are in the
> first *or* the second intrval. When you spell that in Python it becomes
>
> temperature = yellow_temperature_simulation()
> self.assertTrue(
>     (100.0 <= temperature < 100.5)
>     or (102.5 <= temperature < 103.0))
>
>
I agree, That's the mistake I made. I fixed it as follows:

    def test_red_temperature_simulation(self):
        """
        Method to test the red_temperature_simulation method
        """
        for i in range(1000000):
            test_value = red_temperature_simulation()
            self.assertTrue((test_value < 100.0) or (test_value > 103.0),
                            msg="Test failed for {}".format(test_value))

And now everything works the way it should.

Thanks for all your help :)

-- 
Regards,
Anubhav Yadav
KPIT Technologies,
Pune.

From danny.yoo at gmail.com  Sun Feb  7 11:40:17 2016
From: danny.yoo at gmail.com (Danny Yoo)
Date: Sun, 7 Feb 2016 08:40:17 -0800
Subject: [Tutor] Need help with two similar test cases that I have
 written. One works and the other fails
In-Reply-To: <CAGZAPF7=nE9WtbvdORgVBUbJjDMo14gNmf-vL8S5m8N5AaXqdQ@mail.gmail.com>
References: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>
 <CAGZAPF7-Q+R4NE+iKdN9rgFcOFUZbpvxa0tAUh-bDL7exXZ2Jg@mail.gmail.com>
 <CA+Jf9AHRHEg0rU1_Jts2cjx0mNbwtB3sK8h=7gUA1uyEks7V6A@mail.gmail.com>
 <n9744s$doc$1@ger.gmane.org>
 <CA+Jf9AFOrd7OOAQkDgF4DeQVZ7OQsE0dQUD_NJDSRHKW_1Vf7A@mail.gmail.com>
 <CAGZAPF7=nE9WtbvdORgVBUbJjDMo14gNmf-vL8S5m8N5AaXqdQ@mail.gmail.com>
Message-ID: <CAGZAPF5EqNW+brmeD-EdWgAmaEu2zSOz0ubnJvrU1ghMjeKp9A@mail.gmail.com>

On Feb 7, 2016 8:38 AM, "Danny Yoo" <danny.yoo at gmail.com> wrote:
>
> :
> >
> >     def test_red_temperature_simulation(self):
> >         """
> >         Method to test the red_temperature_simulation method
> >         """
> >         for i in range(1000000):
> >             test_value = red_temperature_simulation()
> >             self.assertTrue((test_value < 100.0) or (test_value >
103.0),
> >                             msg="Test failed for {}".format(test_value))
> >
> > And now everything works the way it should.
> >
>
> Be careful.  Now you have a test that always succeeds.

No, I'm wrong.  Sorry about that: I misread the numerical range.

From danny.yoo at gmail.com  Sun Feb  7 11:38:28 2016
From: danny.yoo at gmail.com (Danny Yoo)
Date: Sun, 7 Feb 2016 08:38:28 -0800
Subject: [Tutor] Need help with two similar test cases that I have
 written. One works and the other fails
In-Reply-To: <CA+Jf9AFOrd7OOAQkDgF4DeQVZ7OQsE0dQUD_NJDSRHKW_1Vf7A@mail.gmail.com>
References: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>
 <CAGZAPF7-Q+R4NE+iKdN9rgFcOFUZbpvxa0tAUh-bDL7exXZ2Jg@mail.gmail.com>
 <CA+Jf9AHRHEg0rU1_Jts2cjx0mNbwtB3sK8h=7gUA1uyEks7V6A@mail.gmail.com>
 <n9744s$doc$1@ger.gmane.org>
 <CA+Jf9AFOrd7OOAQkDgF4DeQVZ7OQsE0dQUD_NJDSRHKW_1Vf7A@mail.gmail.com>
Message-ID: <CAGZAPF7=nE9WtbvdORgVBUbJjDMo14gNmf-vL8S5m8N5AaXqdQ@mail.gmail.com>

:
>
>     def test_red_temperature_simulation(self):
>         """
>         Method to test the red_temperature_simulation method
>         """
>         for i in range(1000000):
>             test_value = red_temperature_simulation()
>             self.assertTrue((test_value < 100.0) or (test_value > 103.0),
>                             msg="Test failed for {}".format(test_value))
>
> And now everything works the way it should.
>

Be careful.  Now you have a test that always succeeds.

From anubhav1691 at gmail.com  Sun Feb  7 14:03:27 2016
From: anubhav1691 at gmail.com (Anubhav Yadav)
Date: Mon, 8 Feb 2016 00:33:27 +0530
Subject: [Tutor] Need help with two similar test cases that I have
 written. One works and the other fails
In-Reply-To: <CAGZAPF5EqNW+brmeD-EdWgAmaEu2zSOz0ubnJvrU1ghMjeKp9A@mail.gmail.com>
References: <CA+Jf9AGs6FDXAkxRFEfMyN7__4q7OuwBy_uWKw2h7C1fkxMXSw@mail.gmail.com>
 <CAGZAPF7-Q+R4NE+iKdN9rgFcOFUZbpvxa0tAUh-bDL7exXZ2Jg@mail.gmail.com>
 <CA+Jf9AHRHEg0rU1_Jts2cjx0mNbwtB3sK8h=7gUA1uyEks7V6A@mail.gmail.com>
 <n9744s$doc$1@ger.gmane.org>
 <CA+Jf9AFOrd7OOAQkDgF4DeQVZ7OQsE0dQUD_NJDSRHKW_1Vf7A@mail.gmail.com>
 <CAGZAPF7=nE9WtbvdORgVBUbJjDMo14gNmf-vL8S5m8N5AaXqdQ@mail.gmail.com>
 <CAGZAPF5EqNW+brmeD-EdWgAmaEu2zSOz0ubnJvrU1ghMjeKp9A@mail.gmail.com>
Message-ID: <CA+Jf9AEGkgyZu2MsysSX2L+Amm9GmmG3D25UuAYN3Bos4oWX0Q@mail.gmail.com>

> > Be careful.  Now you have a test that always succeeds.
>
> No, I'm wrong.  Sorry about that: I misread the numerical range.
>

This is the first time I am ever writing unit tests, and I am glad I am
doing it.

Thanks a lot for all your help!.

From esawiek at gmail.com  Sun Feb  7 20:36:06 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Sun, 7 Feb 2016 20:36:06 -0500
Subject: [Tutor] genfromtxt vs. reading csv to a list or dictionary
Message-ID: <CA+ZkTxsZH7P9GXs43FV29PzHLfxcqB96fyC7F814pj036Lu1yg@mail.gmail.com>

Hi all?



I normally need to convert csv and text files to a Numpy array. I tried to
do the same thing using (1) reader=DictReader(MyFile), (2)
reader=csv.readre(MyFile), or (3) genfromtxt (MyFile ,??).  The first two
is after I open the file. They produce a list of lists, list of tuples or
list of dictionaries which later are converted to an array.


The question is which one is the fastest and/or most efficient most
flexible? The genformtext function is nice b/c you can setup formatting of
various column which I often need to do b/c my files contain both string
and numeric values (integer and float).



Thanks in advance--EKE

From dyoo at hashcollision.org  Tue Feb  9 01:16:42 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 8 Feb 2016 22:16:42 -0800
Subject: [Tutor] genfromtxt vs. reading csv to a list or dictionary
In-Reply-To: <CA+ZkTxsZH7P9GXs43FV29PzHLfxcqB96fyC7F814pj036Lu1yg@mail.gmail.com>
References: <CA+ZkTxsZH7P9GXs43FV29PzHLfxcqB96fyC7F814pj036Lu1yg@mail.gmail.com>
Message-ID: <CAGZAPF4H=GW2QLi5_pAAp4Qxh85ow8wq=Hhqw_WvBfmrnuctaw@mail.gmail.com>

> I normally need to convert csv and text files to a Numpy array. I tried to
> do the same thing using (1) reader=DictReader(MyFile), (2)
> reader=csv.readre(MyFile), or (3) genfromtxt (MyFile ,??).  The first two
> is after I open the file. They produce a list of lists, list of tuples or
> list of dictionaries which later are converted to an array.


If we're touching the hard drive as a part of input/output operations,
likely you won't need to worry about efficiency, especially for a
program dedicated to read files.

What I mean is, disk operations are *several orders of magnitude* more
expensive than most other non-I/O operations your program will
perform.  As long as we're reading and processing the input in a
non-crazy way, we should be ok.  ("Non-crazy": A small constant number
of passes over the input file, and if the file is very large, doesn't
try to read the whole file into memory at once).  I think all three of
your described processes will be non-crazy.

What will dominate the time a file-parsing program takes will almost
certainly be I/O-bound.  And you probably can't do anything to change
the physics of how disk platters spin.  This rough rule is sensitive
to context, and several of my assumptions may be wrong.  I'm assuming
a standard desktop environment on a single machine, with a physical
hard drive.  But maybe you have SSDs or some unusual storage that's
very fast or parallel.  If those assumptions are wrong, then yes, you
may need to be concerned about shaving off every last millisecond to
get performance.

How can we know for sure?  We can measure: a "profile" of a program
lets us see if time is truly being spent in non-I/O computation.  See:
https://docs.python.org/3.5/library/profile.html for more details.  If
you have versions of your reader for those three strategies, try
profiling them.

From cegarcia0323 at gmail.com  Tue Feb  9 10:34:18 2016
From: cegarcia0323 at gmail.com (Chelsea G)
Date: Tue, 9 Feb 2016 10:34:18 -0500
Subject: [Tutor] Help with date range
Message-ID: <CABV0pWwO89+VFs03np+ieeETL=8NVmwsknURSk5rstbStYuCBA@mail.gmail.com>

So what I am trying to do is take in a csv file and the format of the csv
file is:
something, something1, something2,something3, something4, something5,
something6, something7.
Then in the csv file I want to search for a certain range of dates like
1/3/2016 - 2/3/2016. I can get individual dates but not a range of dates. I
have an if elif statement to read row5 which is the date row. My if
statement is the initial pass at returning the values within the date range
and then my elif is the parameter and if there is no parameter passed in
then it returns all data. I am having some trouble trying to pass in a date
range parameter. The piece of code is under the def populate_dict function
the date_choice part. Here is the code:

import csvimport jsonimport sysimport osfrom collections import
defaultdictfrom collections import Counter


UPPER_LIMIT = 5
LOWER_LIMIT = 4
class dictionary():
    def __init__(self):
	self.dict = defaultdict(list)
	self.counted_dict = defaultdict(list)
	self.grouped_dict = defaultdict(list)
	self.total_dict = defaultdict(list)


    def populate_dict(self, filename, date_choice, key_choice):
        with open(filename, 'rb') as f:
            reader = csv.reader(f)
            next(reader, None) #
            for row in reader:
		if date_choice == row[5]:
			self.dict[row[2]].append(row[3])
		elif date_choice == "none":
			self.dict[row[2]].append(row[3])
		if key_choice == row[3]:
			self.dict[row[2]].append(row[3])
		elif key_choice == "none":
			self.dict[row[2]].append(row[3])   def all_counts(self):
        data_count = Counter()
        for key in self.dict.keys():
            self.counted_dict.update({key: Counter(self.dict[key])})
# returns the total counts for each application
    def total_counts(self):
        self.total_dict.update({'Application': 'Incident Count'})
        for key in self.dict.keys():
            total = 0
            b = Counter(self.dict[key])
            for value in b:
                total += b[value]
                self.total_dict.update({key: total})
# returns the counts of incidents if they are greater than or equal to
5, and groups the rest in an "other" category
    def grouped_counts(self):
        for key in self.dict.keys():
            total = 0
            c = Counter(self.dict[key])
            self.grouped_dict[key].append(['Description', 'Incident Count'])
            for value in c:
                if c[value] >= UPPER_LIMIT:
                    grouped_list = value, c[value]
                    self.grouped_dict[key].append(grouped_list)
                elif c[value] <= LOWER_LIMIT:
                    total += c[value]
            other_list = "other ", total
            self.grouped_dict[key].append(other_list)

From joel.goldstick at gmail.com  Tue Feb  9 13:58:03 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 9 Feb 2016 13:58:03 -0500
Subject: [Tutor] Help with date range
In-Reply-To: <CABV0pWwO89+VFs03np+ieeETL=8NVmwsknURSk5rstbStYuCBA@mail.gmail.com>
References: <CABV0pWwO89+VFs03np+ieeETL=8NVmwsknURSk5rstbStYuCBA@mail.gmail.com>
Message-ID: <CAPM-O+x59ciCwaWXOVs1wjOkVjpZdk7gyvr3NK5__QgEhwwM3A@mail.gmail.com>

On Tue, Feb 9, 2016 at 10:34 AM, Chelsea G <cegarcia0323 at gmail.com> wrote:

> So what I am trying to do is take in a csv file and the format of the csv
> file is:
> something, something1, something2,something3, something4, something5,
> something6, something7.
> Then in the csv file I want to search for a certain range of dates like
> 1/3/2016 - 2/3/2016. I can get individual dates but not a range of dates. I
> have an if elif statement to read row5 which is the date row. My if
> statement is the initial pass at returning the values within the date range
> and then my elif is the parameter and if there is no parameter passed in
> then it returns all data. I am having some trouble trying to pass in a date
> range parameter. The piece of code is under the def populate_dict function
> the date_choice part. Here is the code:
>
> import csvimport jsonimport sysimport osfrom collections import
> defaultdictfrom collections import Counter
>
>
> The above imports are a mess! Can you format them correctly?


> UPPER_LIMIT = 5
> LOWER_LIMIT = 4
> class dictionary():
>     def __init__(self):
>         self.dict = defaultdict(list)
>         self.counted_dict = defaultdict(list)
>         self.grouped_dict = defaultdict(list)
>         self.total_dict = defaultdict(list)
>
>
>     def populate_dict(self, filename, date_choice, key_choice):
>         with open(filename, 'rb') as f:
>             reader = csv.reader(f)
>             next(reader, None) #
>             for row in reader:
>                 if date_choice == row[5]:
>                         self.dict[row[2]].append(row[3])
>                 elif date_choice == "none":
>                         self.dict[row[2]].append(row[3])
>                 if key_choice == row[3]:
>                         self.dict[row[2]].append(row[3])
>                 elif key_choice == "none":
>                         self.dict[row[2]].append(row[3])   def
> all_counts(self):
>         data_count = Counter()
>         for key in self.dict.keys():
>             self.counted_dict.update({key: Counter(self.dict[key])})
> # returns the total counts for each application
>     def total_counts(self):
>         self.total_dict.update({'Application': 'Incident Count'})
>         for key in self.dict.keys():
>             total = 0
>             b = Counter(self.dict[key])
>             for value in b:
>                 total += b[value]
>                 self.total_dict.update({key: total})
> # returns the counts of incidents if they are greater than or equal to
> 5, and groups the rest in an "other" category
>     def grouped_counts(self):
>         for key in self.dict.keys():
>             total = 0
>             c = Counter(self.dict[key])
>             self.grouped_dict[key].append(['Description', 'Incident
> Count'])
>             for value in c:
>                 if c[value] >= UPPER_LIMIT:
>                     grouped_list = value, c[value]
>                     self.grouped_dict[key].append(grouped_list)
>                 elif c[value] <= LOWER_LIMIT:
>                     total += c[value]
>             other_list = "other ", total
>             self.grouped_dict[key].append(other_list)
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>


Its hard to tell what you want to do.  You should show a small (10 lines?)
input table, and show the results you want.

This is probably much easier to solve by importing csv to a sqlite
database, then running some simple date range queries.  Do you know any sql?

-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From martin at linux-ip.net  Tue Feb  9 14:37:04 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Tue, 9 Feb 2016 11:37:04 -0800
Subject: [Tutor] Help with date range
In-Reply-To: <CABV0pWwO89+VFs03np+ieeETL=8NVmwsknURSk5rstbStYuCBA@mail.gmail.com>
References: <CABV0pWwO89+VFs03np+ieeETL=8NVmwsknURSk5rstbStYuCBA@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1602091041200.2025@znpeba.jbaqresebt.arg>


Greetings Chelsea,

>So what I am trying to do is take in a csv file and the format of 
>the csv file is:

While I do not have a suggestion to you about the Python you have 
sent to the list, I have a few suggestions about how to approach the 
problem differently.  I hope that this may make your task easier.

>something, something1, something2,something3, something4, 
>something5, something6, something7. Then in the csv file I want to 
>search for a certain range of dates like 1/3/2016 - 2/3/2016. I can 
>get individual dates but not a range of dates.


Option 1 for date handling:

Consider converting your date column to a datetime object, which 
then can be used in comparisons (and sorting).

  >>> d0 = datetime.datetime.strptime('1/3/2016', '%m/%d/%Y')
  >>> d1 = datetime.datetime.strptime('2/3/2016', '%m/%d/%Y')
  >>> d0 > d1
  False
  >>> d0 < d1
  True

This is good for general date and time handling.  If you are only 
handling date,s then you may wish to use the strptime() function to 
parse your date and then choose to convert it to a date.  See also 
below my point about serialized (on disk) representation and memory 
representation.


Option 2 for date handling:

Consider converting your date to a time struct, which can then by 
used in comparisons and sorting.

  >>> t0 = time.strptime('1/3/2016', '%m/%d/%Y')
  >>> t1 = time.strptime('2/3/2016', '%m/%d/%Y')
  >>> t0 > t1
  False
  >>> t0 < t1
  True

The time library is good, too.  I like it because I have, 
historically, preferred the Epoch timestamp, which is how several 
classes of computers expose their time-tracking to applications.

  >>> time.time()
  1455044423.592147


>I have an if elif statement to read row5 which is the date row. My 
>if statement is the initial pass at returning the values within the 
>date range and then my elif is the parameter and if there is no 
>parameter passed in then it returns all data.

>I am having some trouble trying to pass in a date range parameter. 
>The piece of code is under the def populate_dict function the 
>date_choice part. Here is the code:

Even though I would separate the selection of record from the 
reading and conversion of the data (see more below), I think you 
want to do something more like this (to handle a range of time):

    def populate_dict(self, filename, date_choice, key_choice):
        # -- date_choice must be a list/tuple of datetime types (datetime, datetime)
        with open(filename, 'rb') as f:
            reader = csv.reader(f)
            next(reader, None) # -- skipping the header, eh?
            for row in reader:
                eventdate = datetime.datetime.strptime(row[5], '%m/%d/%Y')

                if date_choice is None:
                   self.dict[row[2]].append(row[3])
                elif date_choice[0] <= eventdate <= date_choice[1]:
                   self.dict[row[2]].append(row[3])

                if key_choice == row[3]:
                   self.dict[row[2]].append(row[3])
                elif key_choice == "none":
                   self.dict[row[2]].append(row[3])

Handling time ranges is always a wee bit tricky.  If you don't 
convert the data into some sort of convenient format before trying 
to apply the range criteria, you'll slowly go crazy trying to figure 
out what is wrong.

How much data do you have?  I'm assuming it is small(er than half a 
million rows).  If so, then, there's no harm in loading it all into 
memory and then filtering in (or out) the stuff you want.

But, most importantly, I have a general suggestion about separating 
the serialized (on-disk) format from the format you are using in 
memory.  I will make reference to another data handling library in 
Python, but mostly to illustrate the idea.

You can read CSV data sets with a tool called pandas [0].  There are 
many reasons it is popular, but I'll examine one feature.  If you 
load a CSV file using pandas, it tries to apply a bit of inference 
logic to each data type as it reads a file.  You said that the sixth 
column of your data is a date-formatted field.

If the pandas library can determine that the sixth field in every 
row can be converted to a datetime() type (or an int() or a float(), 
etc.), it will automatically convert that field.  This is useful 
because the serialized form is a string that looks like '1/3/2016' 
or '2016-02-09 11:13:06' or 'Tue Feb  9 19:13:24 UTC 2016'.

But, you as a human are not thinking "this is a string".  You are 
thinking of it as a date.  So, simply put:

  Serialized form:  a string representation that contains a date
  Form in memory:  a date, datetime, or time type (of some kind)

Again, if you are working with a small amount of data (like 
less than 50MB), don't be afraid to load it into memory.  You can 
make things more efficient later, if need be.  If you have a larger 
data set or you want to perform all sorts of different computations, 
sorting and grouping, you may find a SQLite database a good solution 
(as Joel Goldstick has just posted).

So, finally, if you are merely reading a CSV file, I might suggest 
that you convert the serialized representation of the data (a 
string) into an object or datatype in memory that allows you to 
perform the desired sorting, computation, calculation or 
range-finding.  Also, if you have a smaller block of code and data, 
we may find it easier to make specific suggestions.

And, one last general comment....  Unless you have a specific reason 
why not to do so, it's best to convert and decode inputs into a 
canonical memory representation as soon as possible when reading 
from disk/network.

Good luck,

-Martin

 [0] http://pandas.pydata.org/

-- 
Martin A. Brown
http://linux-ip.net/

From alan.gauld at btinternet.com  Tue Feb  9 19:38:02 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 10 Feb 2016 00:38:02 +0000
Subject: [Tutor] Help with date range
In-Reply-To: <CABV0pWwO89+VFs03np+ieeETL=8NVmwsknURSk5rstbStYuCBA@mail.gmail.com>
References: <CABV0pWwO89+VFs03np+ieeETL=8NVmwsknURSk5rstbStYuCBA@mail.gmail.com>
Message-ID: <n9e0p9$c3h$1@ger.gmane.org>

On 09/02/16 15:34, Chelsea G wrote:

> Then in the csv file I want to search for a certain range of dates like
> 1/3/2016 - 2/3/2016. I can get individual dates but not a range of dates. 

Can ytou simplify the problem momentaruily?
Can you create a list of dates then search that list for a
range of dates? That is forget about the csv file issues
and focus on the date problem? That might make it easier
to see where you are having the issues.

> have an if elif statement to read row5 which is the date row. My if
> statement is the initial pass at returning the values within the date range
> and then my elif is the parameter and if there is no parameter passed in
> then it returns all data. I am having some trouble trying to pass in a date
> range parameter. 

How are you trying to represent the date range? As two dates?
Or as date and a timedelta object? Or some thing else?(what?)

>     def populate_dict(self, filename, date_choice, key_choice):

Unfortunately in Python we can't tell much about the types
of your parameters from the declaration.


>         with open(filename, 'rb') as f:
>             reader = csv.reader(f)
>             next(reader, None) #
>             for row in reader:
> 		if date_choice == row[5]:
> 			self.dict[row[2]].append(row[3])

row[5] will be a string so this suggests your date_choice is
a string representation of a single date? That obviously cannot
be a range you need a start and end point. And then you need
to see if the target date lies between those two points. And
for that you need a way to compare them. Comparing dates as
a general problem is very, very difficult.

Fortunately the datetime module and the timedelta object
within it simplifies things for your use case.



-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From esawiek at gmail.com  Thu Feb 11 03:01:54 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Thu, 11 Feb 2016 03:01:54 -0500
Subject: [Tutor] genfromtxt vs. reading csv to a list or dictionary
Message-ID: <CA+ZkTxts8nGCHvES54EnkP6b0q=-1E0cy+K7LGOzcgeAZ6zkOQ@mail.gmail.com>

Thanks Danny! I realized that I should not be concerned with
speed/efficiency since I am a beginner in python and giving the data I work
with, speed is not that important now.


I am learning how to read files via several ways and transform them to
numpy arrays. I just thought genfromtxt provides many options and one could
use it to read and convert a file into 2d array with proper format; instead
of reading it then format it. But genfromtxt gives byte strings which i
could not convert to integer, float, string, etc.



Thanks again--EKE

From anshu.kumar726 at gmail.com  Thu Feb 11 08:01:03 2016
From: anshu.kumar726 at gmail.com (Anshu Kumar)
Date: Thu, 11 Feb 2016 18:31:03 +0530
Subject: [Tutor] mock file reader object
Message-ID: <CAMFBua6kWjKVvXxoLRZq4HYGXYJNGHDeMt5HPxFNdxbKvhKd6w@mail.gmail.com>

Hi,

I need to mock file reader object , i tried using @patch('__builtin__.open')
but it will patch my all file open readers.

More precisely i have code something like below

def restore_segments_and_lines_json(segments_file_path,lines_file_path,deleted_segment_ids):
    with open(segments_file_path, "rb") as segments_reader:
        segments = segments_reader.read()

    with open(lines_file_path, "rb") as lines_reader:
        lines = lines_readers.read()

    """ Do some processing with lines and segments"""

    logger.info("updating segments file")
    with open(segments_file_path, "wb") as segments_writer:
        segments_writer.write(segments, segments_writer)

    logger.info("updating lines file")
    with open(lines_file_path, "wb") as lines_writer:
        lines_writer.write(lines, lines_writer)


I need to mock two different file read and file write opens, can i get
some suggestion.


Thanks in advance,

Anshu

From rdswift at umes.edu  Thu Feb 11 13:03:57 2016
From: rdswift at umes.edu (Swift, Robert)
Date: Thu, 11 Feb 2016 13:03:57 -0500
Subject: [Tutor] Need help with audio manipulation
Message-ID: <CAK+5R0D1hNjk=qTbXCuoSsTz-ojvdxA0L72XSc=ZGKi29BNZgw@mail.gmail.com>

I was wondering how to write a code that would take a recorded audio and
place these values into a numpy array? I have written various codes that
can record and playback sound and work properly. I need the recorded audio
to be accessed from a numpy array so I can process the sound and manipulate
it to play back the inverse of the array for a noise cancelling affect. Any
ideas would be great.


Thanks,

Robert

From richkappler at gmail.com  Thu Feb 11 15:28:21 2016
From: richkappler at gmail.com (richard kappler)
Date: Thu, 11 Feb 2016 15:28:21 -0500
Subject: [Tutor] declare a variable inside a class
Message-ID: <CAG7edPG5B92=AbxaP-59TVf63qE=fXebMDcaxiNdhZ4zc+7QrQ@mail.gmail.com>

Trying to optimize the modular input we wrote for Splunk, the basic
structure (dictated by Splunk) is the entire script is a class with
numerous methods within. The modular input reads a tcp stream, starts a new
thread for each source, reads the messages from that source into a buffer,
checks for stx and etx, formats it using an xslt and sends the result out
to splunk.

The method that actually formats the messages opens the xslt file, then
uses it to format the message. Clever boy that I am, I figured it would be
better to open the xslt file once, rather than once per message, so I moved
the line

xslt = ('path/to/xslt.file')

out of the method it was in and up to the top level of the class, to wit:

class MyClass():

    xslt = ('path/to/xslt.file')

   def a_bunch-of-methods():

and so on.

Obviously this didn't work, when the formatting method was called, it
through an exception that the variable xslt wasn't defined. This is my
first not-a-toy try at OOP, I could use a clue.

regards, Richard

-- 

*Java is like Alzheimers; it starts slow and eventually, it takes away all
of your memory.*

From martin at linux-ip.net  Thu Feb 11 15:43:57 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Thu, 11 Feb 2016 12:43:57 -0800
Subject: [Tutor] declare a variable inside a class
In-Reply-To: <CAG7edPG5B92=AbxaP-59TVf63qE=fXebMDcaxiNdhZ4zc+7QrQ@mail.gmail.com>
References: <CAG7edPG5B92=AbxaP-59TVf63qE=fXebMDcaxiNdhZ4zc+7QrQ@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1602111238050.2025@znpeba.jbaqresebt.arg>


Hi there again Richard,

[snipped a bit of stuff]

>The method that actually formats the messages opens the xslt file, 
>then uses it to format the message. Clever boy that I am, I figured 
>it would be better to open the xslt file once, rather than once per 
>message, so I moved the line
>
>xslt = ('path/to/xslt.file')

Yes, this is a good idea.

>out of the method it was in and up to the top level of the class, to wit:
>
>class MyClass():
>
>    xslt = ('path/to/xslt.file')
>
>   def a_bunch-of-methods():

An impossible method name.  That would be a syntax error.

>and so on.
>
>Obviously this didn't work,

Why is it obvious?  What was obvious to me is the syntax error, but 
that does not appear to be what you are asking.

What was the error message?

>when the formatting method was called, it through an exception that 
>the variable xslt wasn't defined. This is my first not-a-toy try at 
>OOP, I could use a clue.

So, before addressing your question, could I gently recommend that 
you post your actual code (clean up any variables or data that you 
don't want the public to see), whether it is running or not, along 
with any error messages (pasted, please).

This is just a reminder, that this reduces the guesswork on the part 
of the members of the list.

I think your problem is simply not knowing the name of the variable 
you called 'xslt'.  Perhaps the below example helps?

-Martin

#! /usr/bin/python3

class MyClass(object):

    xslt = '/usr/share/nmap/nmap.xsl'

    def find_xslt(self):
        print(xslt)

    def oh_there_it_is(self):
        print(self.xslt)

    def redefine_in_instance(self):
        self.xslt = '/usr/share/sgml/X11/xorg.xsl'

if __name__ == '__main__':

    # -- instantiate the class
    c = MyClass()

    # -- this is what I think you were doing
    try:
        c.where_is_xslt()
    except AttributeError:
        pass

    # -- but, try locating the class attribute through self
    c.oh_there_it_is()

    # -- class attribute can be overridden in instances....
    c.redefine_in_instance()
    c.oh_there_it_is()

    # -- newly defined instances will get the default class attribute
    d = MyClass()
    d.oh_there_it_is()

-- 
Martin A. Brown
http://linux-ip.net/

From richkappler at gmail.com  Thu Feb 11 16:03:48 2016
From: richkappler at gmail.com (richard kappler)
Date: Thu, 11 Feb 2016 16:03:48 -0500
Subject: [Tutor] declare a variable inside a class
In-Reply-To: <alpine.LSU.2.11.1602111238050.2025@znpeba.jbaqresebt.arg>
References: <CAG7edPG5B92=AbxaP-59TVf63qE=fXebMDcaxiNdhZ4zc+7QrQ@mail.gmail.com>
 <alpine.LSU.2.11.1602111238050.2025@znpeba.jbaqresebt.arg>
Message-ID: <CAG7edPEyemPW98Cf5sg=9BK81n2fgNQsoRDiP-KT1gMdrpRk3Q@mail.gmail.com>

Thanks for the reply Martin, and in this instance I cannot post the actual
code (company rules). What I can do is say that with the xslt variable
defined within the formatter method, everything works, but when I pull it
out and put it in the upper level of the class, it gives me a traceback
that says the global variable xslt is not defined. Does that help?

regards, Richard

On Thu, Feb 11, 2016 at 3:43 PM, Martin A. Brown <martin at linux-ip.net>
wrote:

>
> Hi there again Richard,
>
> [snipped a bit of stuff]
>
> >The method that actually formats the messages opens the xslt file,
> >then uses it to format the message. Clever boy that I am, I figured
> >it would be better to open the xslt file once, rather than once per
> >message, so I moved the line
> >
> >xslt = ('path/to/xslt.file')
>
> Yes, this is a good idea.
>
> >out of the method it was in and up to the top level of the class, to wit:
> >
> >class MyClass():
> >
> >    xslt = ('path/to/xslt.file')
> >
> >   def a_bunch-of-methods():
>
> An impossible method name.  That would be a syntax error.
>
> >and so on.
> >
> >Obviously this didn't work,
>
> Why is it obvious?  What was obvious to me is the syntax error, but
> that does not appear to be what you are asking.
>
> What was the error message?
>
> >when the formatting method was called, it through an exception that
> >the variable xslt wasn't defined. This is my first not-a-toy try at
> >OOP, I could use a clue.
>
> So, before addressing your question, could I gently recommend that
> you post your actual code (clean up any variables or data that you
> don't want the public to see), whether it is running or not, along
> with any error messages (pasted, please).
>
> This is just a reminder, that this reduces the guesswork on the part
> of the members of the list.
>
> I think your problem is simply not knowing the name of the variable
> you called 'xslt'.  Perhaps the below example helps?
>
> -Martin
>
> #! /usr/bin/python3
>
> class MyClass(object):
>
>     xslt = '/usr/share/nmap/nmap.xsl'
>
>     def find_xslt(self):
>         print(xslt)
>
>     def oh_there_it_is(self):
>         print(self.xslt)
>
>     def redefine_in_instance(self):
>         self.xslt = '/usr/share/sgml/X11/xorg.xsl'
>
> if __name__ == '__main__':
>
>     # -- instantiate the class
>     c = MyClass()
>
>     # -- this is what I think you were doing
>     try:
>         c.where_is_xslt()
>     except AttributeError:
>         pass
>
>     # -- but, try locating the class attribute through self
>     c.oh_there_it_is()
>
>     # -- class attribute can be overridden in instances....
>     c.redefine_in_instance()
>     c.oh_there_it_is()
>
>     # -- newly defined instances will get the default class attribute
>     d = MyClass()
>     d.oh_there_it_is()
>
> --
> Martin A. Brown
> http://linux-ip.net/
>



-- 

*Java is like Alzheimers; it starts slow and eventually, it takes away all
of your memory.*

From __peter__ at web.de  Thu Feb 11 16:20:29 2016
From: __peter__ at web.de (Peter Otten)
Date: Thu, 11 Feb 2016 22:20:29 +0100
Subject: [Tutor] declare a variable inside a class
References: <CAG7edPG5B92=AbxaP-59TVf63qE=fXebMDcaxiNdhZ4zc+7QrQ@mail.gmail.com>
 <alpine.LSU.2.11.1602111238050.2025@znpeba.jbaqresebt.arg>
 <CAG7edPEyemPW98Cf5sg=9BK81n2fgNQsoRDiP-KT1gMdrpRk3Q@mail.gmail.com>
Message-ID: <n9itv2$ca$1@ger.gmane.org>

richard kappler wrote:

> Thanks for the reply Martin, and in this instance I cannot post the actual
> code (company rules). What I can do is say that with the xslt variable
> defined within the formatter method, everything works, but when I pull it
> out and put it in the upper level of the class, it gives me a traceback
> that says the global variable xslt is not defined. Does that help?

You need to qualify the class attribute with self to access it from within a 
method:

>>> class MyClass:
...     xslt = "/path/to/file"
...     def some_method(self):
...         print(self.xslt)
... 
>>> m = MyClass()
>>> m.some_method()
/path/to/file



From ben+python at benfinney.id.au  Thu Feb 11 16:32:08 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 12 Feb 2016 08:32:08 +1100
Subject: [Tutor] Mock filesystem operations only for specific test doubles
 (was: mock file reader object)
References: <CAMFBua6kWjKVvXxoLRZq4HYGXYJNGHDeMt5HPxFNdxbKvhKd6w@mail.gmail.com>
Message-ID: <85io1v6ihz.fsf@benfinney.id.au>

Anshu Kumar <anshu.kumar726 at gmail.com> writes:

> I need to mock file reader object , i tried using
> @patch('__builtin__.open') but it will patch my all file open readers.

For this reason I have written and published the Gajja library
<URL:https://pypi.python.org/pypi/gajja/>.

Its FileDouble objects will allow fine-grained control over exactly
which file accesses are mocked, leaving the rest to behave normally.

To date it is only used in one code base. I would be pleased to receive
feedback either to my email address or at the ?testing in Python? forum
<URL:http://lists.idyll.org/listinfo/testing-in-python>.

-- 
 \       ?I never forget a face, but in your case I'll be glad to make |
  `\                                      an exception.? ?Groucho Marx |
_o__)                                                                  |
Ben Finney


From alan.gauld at btinternet.com  Thu Feb 11 17:05:16 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 11 Feb 2016 22:05:16 +0000
Subject: [Tutor] declare a variable inside a class
In-Reply-To: <n9itv2$ca$1@ger.gmane.org>
References: <CAG7edPG5B92=AbxaP-59TVf63qE=fXebMDcaxiNdhZ4zc+7QrQ@mail.gmail.com>
 <alpine.LSU.2.11.1602111238050.2025@znpeba.jbaqresebt.arg>
 <CAG7edPEyemPW98Cf5sg=9BK81n2fgNQsoRDiP-KT1gMdrpRk3Q@mail.gmail.com>
 <n9itv2$ca$1@ger.gmane.org>
Message-ID: <n9j0ir$ahs$1@ger.gmane.org>

On 11/02/16 21:20, Peter Otten wrote:

>> defined within the formatter method, everything works, but when I pull it
>> out and put it in the upper level of the class, it gives me a traceback
>> that says the global variable xslt is not defined. Does that help?
> 
> You need to qualify the class attribute with self to access it from within a 
> method:

Or you could use the class name since it is a class attribute:

>>>> class MyClass:
> ...     xslt = "/path/to/file"
> ...     def some_method(self):
> ...         print(MyClass.xslt)

If you need the possibility of having different paths for
different instances you need to initialise it in
an __init__() method:

class MyClass:
   def __init__(self,path):
       self.xslt = path
   ...

instance1 = MyClass('/some/path/here')
instance2 = MyClass('/another/path/here')

The way you have it currently both instances share the same path.
But that may be what you want.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Thu Feb 11 17:09:21 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 11 Feb 2016 22:09:21 +0000
Subject: [Tutor] Need help with audio manipulation
In-Reply-To: <CAK+5R0D1hNjk=qTbXCuoSsTz-ojvdxA0L72XSc=ZGKi29BNZgw@mail.gmail.com>
References: <CAK+5R0D1hNjk=qTbXCuoSsTz-ojvdxA0L72XSc=ZGKi29BNZgw@mail.gmail.com>
Message-ID: <n9j0qg$e89$1@ger.gmane.org>

On 11/02/16 18:03, Swift, Robert wrote:
> I was wondering how to write a code that would take a recorded audio and
> place these values into a numpy array? I have written various codes that
> can record and playback sound and work properly. I need the recorded audio
> to be accessed from a numpy array so I can process the sound and manipulate
> it to play back the inverse of the array for a noise cancelling affect. Any
> ideas would be great.

There are numerous audio processing toolkits available,
some work with SciPy, others are independent.

There is a useful page here:

wiki.python.org/moin/PythonInMusic

You may find useful pointers there.

As a subject its a bit off topic for this list which is
about the standard library and language.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From martin at linux-ip.net  Thu Feb 11 18:23:48 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Thu, 11 Feb 2016 15:23:48 -0800
Subject: [Tutor] declare a variable inside a class
In-Reply-To: <CAG7edPEyemPW98Cf5sg=9BK81n2fgNQsoRDiP-KT1gMdrpRk3Q@mail.gmail.com>
References: <CAG7edPG5B92=AbxaP-59TVf63qE=fXebMDcaxiNdhZ4zc+7QrQ@mail.gmail.com>
 <alpine.LSU.2.11.1602111238050.2025@znpeba.jbaqresebt.arg>
 <CAG7edPEyemPW98Cf5sg=9BK81n2fgNQsoRDiP-KT1gMdrpRk3Q@mail.gmail.com>
Message-ID: <alpine.LSU.2.11.1602111516570.2025@znpeba.jbaqresebt.arg>


>Thanks for the reply Martin, and in this instance I cannot post the 
>actual code (company rules).

That's often the case.  I think most of us understand that you may 
not be able to post the original code.

>What I can do is say that with the xslt variable defined within the 
>formatter method, everything works, but when I pull it out and put 
>it in the upper level of the class, it gives me a traceback that 
>says the global variable xslt is not defined. Does that help?

I understand what you are saying, but cannot make a more specific 
suggestion, beyond the example I posted before, which included the 
most common reason that people have that problem when moving names 
out of a method into the enclosing class.

Summary: 'xslt' is not in the namespace you think it is in.

I (or others) may be able to help if you use a short, self-contained 
correct example of your problem.  You can change all of the variable 
names.  You can (and should) omit everything that is irrelevant.

When I'm trying to solve a problem like this, I usually crack open 
an editor and try to reproduce the problem in as few lines of code 
as I can.  This often helps me to see my own errors.

Good luck,

-Martin

-- 
Martin A. Brown
http://linux-ip.net/

From breamoreboy at yahoo.co.uk  Thu Feb 11 15:54:08 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 11 Feb 2016 20:54:08 +0000
Subject: [Tutor] declare a variable inside a class
In-Reply-To: <CAG7edPG5B92=AbxaP-59TVf63qE=fXebMDcaxiNdhZ4zc+7QrQ@mail.gmail.com>
References: <CAG7edPG5B92=AbxaP-59TVf63qE=fXebMDcaxiNdhZ4zc+7QrQ@mail.gmail.com>
Message-ID: <n9ise0$7ti$1@ger.gmane.org>

On 11/02/2016 20:28, richard kappler wrote:
> Trying to optimize the modular input we wrote for Splunk, the basic
> structure (dictated by Splunk) is the entire script is a class with
> numerous methods within. The modular input reads a tcp stream, starts a new
> thread for each source, reads the messages from that source into a buffer,
> checks for stx and etx, formats it using an xslt and sends the result out
> to splunk.
>
> The method that actually formats the messages opens the xslt file, then
> uses it to format the message. Clever boy that I am, I figured it would be
> better to open the xslt file once, rather than once per message, so I moved
> the line
>
> xslt = ('path/to/xslt.file')
>
> out of the method it was in and up to the top level of the class, to wit:
>
> class MyClass():
>
>      xslt = ('path/to/xslt.file')
>
>     def a_bunch-of-methods():
>
> and so on.
>
> Obviously this didn't work, when the formatting method was called, it
> through an exception that the variable xslt wasn't defined. This is my
> first not-a-toy try at OOP, I could use a clue.
>
> regards, Richard
>

I'd be inclined to put "xslt = ('path/to/xslt.file')" right back where 
it started, otherwise every instance of Myclass is limited to whatever 
that path is.

If you insist on keeping xslt at the class level, then you need 
something like:-

myclass = MyClass()
myclass.xslt ...

or:-

def aMethod(self):
     self.xslt ...

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From ntomasic.blyth at gmail.com  Thu Feb 11 17:56:00 2016
From: ntomasic.blyth at gmail.com (Nicholas Tomasic)
Date: Thu, 11 Feb 2016 17:56:00 -0500
Subject: [Tutor] Python for Grade 12 Calculus
Message-ID: <CA+zUy=kfVC7oUQH3J=Za_2UY1CzFO9B9M1ecwUW0gAqiQG3z1A@mail.gmail.com>

Hi,

I'm a 12th grade Calculus teacher at a school in Toronto, Ontario, and I'm
thinking about incorporating Python into an Independent Study Project for
two of my students. Both are passionate about coding with Python and I'm
thinking of asking them to produce something that can either a) calculate
derivatives of simple functions or b) assign 1 student the product rule,
one the quotient rule, and have them write a program that can tackle
derivatives in those forms.

The problem is: I know very little about Python.

Unfortunately, I don't have a specific question but I'm looking for any
valuable insights or suggestions.

I appreciate any information you can give me on the subject.

Thanks so much,

Nick Tomasic

From alan.gauld at btinternet.com  Thu Feb 11 19:15:39 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 12 Feb 2016 00:15:39 +0000
Subject: [Tutor] Python for Grade 12 Calculus
In-Reply-To: <CA+zUy=kfVC7oUQH3J=Za_2UY1CzFO9B9M1ecwUW0gAqiQG3z1A@mail.gmail.com>
References: <CA+zUy=kfVC7oUQH3J=Za_2UY1CzFO9B9M1ecwUW0gAqiQG3z1A@mail.gmail.com>
Message-ID: <n9j87a$ske$1@ger.gmane.org>

On 11/02/16 22:56, Nicholas Tomasic wrote:

> I'm a 12th grade Calculus teacher at a school in Toronto,

I'm from Scotland and so have no idea what that means,
but I take it that they are in high school rather than
at a college/university?

> two of my students. Both are passionate about coding with Python and I'm
> thinking of asking them to produce something that can either a) calculate
> derivatives of simple functions or b) assign 1 student the product rule,
> one the quotient rule, and have them write a program that can tackle
> derivatives in those forms.

Programming is usually better for numerical analysis than doing
formula translations (eg calculus). However there is a library,
SymPy, that is part of the SciPy bundle that allows such things.
If they don't mind downloading SciPy and learning a new package
it might do what you want.

Have a look at this web site for a few ideas of its potential:

http://www.sympy.org/en/features.html

There is a tutorial that might help too:

http://docs.sympy.org/latest/tutorial/index.html#tutorial

HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Thu Feb 11 19:28:57 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 12 Feb 2016 11:28:57 +1100
Subject: [Tutor] Python for Grade 12 Calculus
In-Reply-To: <CA+zUy=kfVC7oUQH3J=Za_2UY1CzFO9B9M1ecwUW0gAqiQG3z1A@mail.gmail.com>
References: <CA+zUy=kfVC7oUQH3J=Za_2UY1CzFO9B9M1ecwUW0gAqiQG3z1A@mail.gmail.com>
Message-ID: <20160212002856.GQ31806@ando.pearwood.info>

On Thu, Feb 11, 2016 at 05:56:00PM -0500, Nicholas Tomasic wrote:
> Hi,
> 
> I'm a 12th grade Calculus teacher at a school in Toronto, Ontario, and I'm
> thinking about incorporating Python into an Independent Study Project for
> two of my students. Both are passionate about coding with Python and I'm
> thinking of asking them to produce something that can either a) calculate
> derivatives of simple functions or b) assign 1 student the product rule,
> one the quotient rule, and have them write a program that can tackle
> derivatives in those forms.

Sounds like a very reasonable project.

An alternative is to give them a project to work with SymPy, which is a 
symbolic algebra package for Python. I presume that Yr 12 students have 
access to CAS calculators in Canada, is that right? Perhaps you could 
get them to write a comparison between the differentiation capabilites 
of their CAS and that of SymPy. The advantage to *you* is that this 
doesn't require any programming skill on your part, since your students 
will be submitting an essay rather than a computer program you will need 
to judge.

Another idea might be to get them to write a small program that randomly 
generates a simple polynomial, asks the user to differentiate it, and 
then checks their answer against what SymPy calculates.

http://www.sympy.org/en/index.html

 
> The problem is: I know very little about Python.

That would be a problem :-)

Do you have any programming experience at all? If it's just a matter 
that you aren't familiar with Python, I can suggest running through a 
tutorial or two, and then asking questions here.

If you have no programming experience at all, this may be a bit hard on 
you, since your students will know more about programming than you.

-- 
Steve

From esawiek at gmail.com  Thu Feb 11 22:45:35 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Thu, 11 Feb 2016 22:45:35 -0500
Subject: [Tutor] Python for Grade 12 Calculus
Message-ID: <CA+ZkTxto2=A6aMe9TnVuU5PKGeksOsDo7q7GkNEN2kQ4H9xxng@mail.gmail.com>

I am not sure of the level of your students but computing the Jacobin and
directional derivative where they can use both calculus and linear algebra;
numerical differentiation and integration would be another topic. There are
several books with nice numerical recipes that can be programmed using
python.

EKE

From steve at pearwood.info  Fri Feb 12 04:10:15 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 12 Feb 2016 20:10:15 +1100
Subject: [Tutor] Python for Grade 12 Calculus
In-Reply-To: <CA+ZkTxto2=A6aMe9TnVuU5PKGeksOsDo7q7GkNEN2kQ4H9xxng@mail.gmail.com>
References: <CA+ZkTxto2=A6aMe9TnVuU5PKGeksOsDo7q7GkNEN2kQ4H9xxng@mail.gmail.com>
Message-ID: <20160212091015.GS31806@ando.pearwood.info>

On Thu, Feb 11, 2016 at 10:45:35PM -0500, Ek Esawi wrote:
> I am not sure of the level of your students but computing the Jacobin and
> directional derivative where they can use both calculus and linear algebra;
> numerical differentiation and integration would be another topic. There are
> several books with nice numerical recipes that can be programmed using
> python.

I'm not sure what maths schooling is like in Canada, but if it's like 
Australia, that might be far beyond the capabilities of the typical Year 
12 student. I've been tutoring at a Yr 12 level for about 15 years now, 
and I have no idea what you mean by "Jacobin and directional derivative" 
:-)


-- 
Steve

From steve at pearwood.info  Fri Feb 12 04:48:17 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 12 Feb 2016 20:48:17 +1100
Subject: [Tutor] Python for Grade 12 Calculus
In-Reply-To: <CA+zUy=kfVC7oUQH3J=Za_2UY1CzFO9B9M1ecwUW0gAqiQG3z1A@mail.gmail.com>
References: <CA+zUy=kfVC7oUQH3J=Za_2UY1CzFO9B9M1ecwUW0gAqiQG3z1A@mail.gmail.com>
Message-ID: <20160212094816.GT31806@ando.pearwood.info>

On Thu, Feb 11, 2016 at 05:56:00PM -0500, Nicholas Tomasic wrote:
> Hi,
> 
> I'm a 12th grade Calculus teacher at a school in Toronto, Ontario, and I'm
> thinking about incorporating Python into an Independent Study Project for
> two of my students. Both are passionate about coding with Python and I'm
> thinking of asking them to produce something that can either a) calculate
> derivatives of simple functions or b) assign 1 student the product rule,
> one the quotient rule, and have them write a program that can tackle
> derivatives in those forms.

I've been thinking more about this, and Python does make it very easy. 
Here's a function which will differentiate a polynomial. You pass as 
argument a list of the coefficients, so:

5x^3 + x^2 - 7x + 5

gets passed as the list [5, 1, -7, 5].

def differentiate(coeffs):
    new_coeffs = []
    for coeff, power in zip(coeffs, range(len(coeffs) - 1, -1, -1)):
        print("term = {}*x^{}".format(coeff, power))
        new_coeff = coeff * power
        if power == 0:
            print("derivative of this term = {}".format(new_coeff))
        else:
            print("derivative of this term = {}*x^{}".format(new_coeff, power-1))
        new_coeffs.append(new_coeff)
    if new_coeffs[-1] != 0:
        msg = "expected derivative of constant term to be zero, but got {}"
        raise RuntimeError(msg.format(new_coeffs[-1]))
    return new_coeffs[0:-1]




And here is the output:

py> differentiate([5, 1, -7, 5])
term = 5*x^3
derivative of this term = 15*x^2
term = 1*x^2
derivative of this term = 2*x^1
term = -7*x^1
derivative of this term = -7*x^0
term = 5*x^0
derivative of this term = 0
[15, 2, -7]


And here is how we would use Sympy to do the same sort of thing:

py> import sympy
py> x = sympy.var("x")
py> f = 5*x**3 + x**2 - 7*x + 5
py> print f
5*x**3 + x**2 - 7*x + 5
py> sympy.diff(f, x)
15*x**2 + 2*x - 7



You will notice that Python uses * for multiplication and ** for powers, 
but in my "differentiate" function I used ^ in the output to represent 
powers. 


Sympy has many display options. For example:

py> sympy.pprint(f, use_unicode=False)
   3    2
5*x  + x  - 7*x + 5
py> sympy.pprint(f, use_unicode=True)
   3    2
5?x  + x  - 7?x + 5


More here:

http://docs.sympy.org/latest/tutorial/printing.html




-- 
Steve

From richkappler at gmail.com  Fri Feb 12 09:56:12 2016
From: richkappler at gmail.com (richard kappler)
Date: Fri, 12 Feb 2016 09:56:12 -0500
Subject: [Tutor] declare a variable inside a class
In-Reply-To: <07171D9A-3EF2-4E94-BE39-B127AA797E2E@gmail.com>
References: <CAG7edPG5B92=AbxaP-59TVf63qE=fXebMDcaxiNdhZ4zc+7QrQ@mail.gmail.com>
 <07171D9A-3EF2-4E94-BE39-B127AA797E2E@gmail.com>
Message-ID: <CAG7edPGp+e=HYtbF02xMLoFWOW6N9yYVbO8NxLdsidV_TVoAiw@mail.gmail.com>

On Fri, Feb 12, 2016 at 2:23 AM, Ian Winstanley <ianwinstanley8 at gmail.com>
wrote:

> To refer to a class variable you need to put the class name first for
> example MyClass.xslt
>

That indeed did the trick, thanks Ian.

regards, Richard

-- 

*Java is like Alzheimers; it starts slow and eventually, it takes away all
of your memory.*

From richkappler at gmail.com  Fri Feb 12 10:16:47 2016
From: richkappler at gmail.com (richard kappler)
Date: Fri, 12 Feb 2016 10:16:47 -0500
Subject: [Tutor] Buffer code review
Message-ID: <CAG7edPFF3j1J9VTdUjjq+S=UDKaJuenSK=h6Xu58WjzLegstZQ@mail.gmail.com>

I have a two script test env set to simulate a controller in production.
The controller creates xml data from a camera tunnel as packages roll
through it, sends these xml messages over tcp to a different machine.

To simulate this in our test environment, I take a log of all the xml
messages, read them out line by line with script1, which writes them to
another log with a time.sleep(0.1) between each line being read to simulate
the rate at which packages pass through the camera tunnel.

Script2 checks file size to see if file has new data, if so, reads the
data, adds an stx and etx, then sends it out over tcp to a machine with a
splunk instance for data collection and analysis.

I will append the script below, but here is the problem. I included a
buffer because there are times when Script2 is reading while Script1 is
writing, so Script2 may not get a full line/message as detected by matching
open/close tags. In that case, the partial line is put in the buffer until
the next read, of which the first line is appended to the buffer and, if
there are matching open/close tags, sent on and the buffer cleared.

Most (literally over 99%) of the time this works fine, but on rare
occasions something goes wrong and I lose a message. The only messages that
are lost are ones that have gone through the buffer. It's rare (26 of 907
through buffer failed, of 17,000+ total messages), but it's still a data
loss so I need to figure out what it is I'm doing not so well here.

code:
##################################################################
#!/usr/bin/env python

import socket
import time
from time import sleep
import logging

logging.basicConfig(format='%(asctime)s|%(levelname)s|%(message)s',
filename='streamer.log', level=logging.DEBUG)

rd1 = 'xmldata.log'
f1 = open(rd1, 'r')
#find initial end of file
f1.seek(0,2)
eof = f1.tell()
logging.info('eof = ' + str(eof))
f1.close()

f2 = open('streamerBufOUT.log', 'a')
f3 = open('bufOUT.log', 'a')

############# open socket ################
# transmit socket
socktx = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socktx_address = ('127.0.0.1', 2008)
logging.info('opening socktx on %s port %s' % socktx_address)
socktx.connect(socktx_address)
##########################################

#to read from beginning of file when starting script, uncomment
#eof = 0

i = 0
buf = ''
brokenLine = 0

def sendLine(line, i):
    s =  '\x02' + line[:-1] + '\x03'
    logging.info('sending line ' + str(i))
    print i
    socktx.send(s)  # send the entire line to the socket
    f2.write(s + '\n')
#    time.sleep(0.2)
    return

while True:
  with open(rd1, 'r') as f1:
     try:
        #check file size to see if grown, set neweof
        f1.seek(0,2)
        neweof = f1.tell()
        logging.info('check file size, set neweof at ' + str(neweof))
     except ValueError:
        f1 = open(rd1, 'r')
        f1.seek(0,2)
        neweof = f1.tell()
        logging.info('value error, neweof is ' + str(neweof))
     #if the file is larger...
     if neweof > eof:
        #go back to last position...
        f1.seek(eof)
        logging.info('neweof > eof, file has new lines, parsing')
# read new line in log.txt
        for line in f1:
            i += 1
            logging.debug('entering for line loop')
            #if a partial line is in the buffer
            if brokenLine == 1:
                i -= 1
                logging.debug('adding rest of line to buf')
                buf += line
                f3.write(buf + '\n')
                sendLine(buf, i)
                buf = ''
                brokenLine = 0
            #otherwise see if it's a full line
            elif '<objectdata ' in line:
                if '</objectdata>' in line:
                    logging.info('sending objectdata line ' + str(i))
                    brokenLine = 0
                    sendLine(line, i)
                else:
                    brokenLine = 1
                    buf += line
                    logging.info('broken line sending to buf line ' +
str(i))
                    f3.write(buf + '\n')
            elif '<objectinfo>' in line:
                if '</objectinfo>' in line:
                    logging.info('sending objectinfo line ' + str(i))
                    brokenLine = 0
                    sendLine(line, i)
                else:
                    brokenLine = 1
                    buf += line
                    logging.info('broken line sending to buf line ' +
str(i))
                    f3.write(buf + '\n')
            elif '<tracedata ' in line:
                if '</tracedata>' in line:
                    logging.info('sending tracedata line ' + str(i))
                    brokenLine = 0
                    sendLine(line, i)
                else:
                    brokenLine = 1
                    buf += line
                    logging.info('broken line sending to buf line ' +
str(i))
                    f3.write(buf + '\n')
            elif '<heartbeatdata ' in line:
                if '</heartbeatdata>' in line:
                    logging.info('sending heartbeatdata line ' + str(i))
                    brokenLine = 0
                    sendLine(line, i)
                else:
                    brokenLine = 1
                    buf += line
                    logging.info('broken line sending to buf line ' +
str(i))
                    f3.write(buf + '\n')
            else:
                logging.error('tag match fail line ' + str(i))
                buf = ''

        # update log.txt file size
        eof = neweof
        logging.info('eof set to neweof, sleeping 2, eof =' + str(eof))
        time.sleep(2)

     elif neweof < eof:
        # this resets eof at night when old log file zipped and new log
file started
        eof = 0
        logging.info('neweof < eof, set eof = 0, sleeping 2')
        time.sleep(2)

     elif neweof == eof:
        # file hasn't changed, do nothing
        logging.info('neweof = eof, file has not changed, sleeping 2')
        time.sleep(2)
#####################################################################

regards, Richard

-- 

*Java is like Alzheimers; it starts slow and eventually, it takes away all
of your memory.*

From dyoo at hashcollision.org  Fri Feb 12 11:33:45 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Fri, 12 Feb 2016 08:33:45 -0800
Subject: [Tutor] Python for Grade 12 Calculus
In-Reply-To: <CA+zUy=kfVC7oUQH3J=Za_2UY1CzFO9B9M1ecwUW0gAqiQG3z1A@mail.gmail.com>
References: <CA+zUy=kfVC7oUQH3J=Za_2UY1CzFO9B9M1ecwUW0gAqiQG3z1A@mail.gmail.com>
Message-ID: <CAGZAPF7DLWUCpv41-jk=44uzzQfbUvOAdN297qUPBLsA6_smnQ@mail.gmail.com>

On Thu, Feb 11, 2016 at 2:56 PM, Nicholas Tomasic
<ntomasic.blyth at gmail.com> wrote:
> Hi,
>
> I'm a 12th grade Calculus teacher at a school in Toronto, Ontario, and I'm
> thinking about incorporating Python into an Independent Study Project for
> two of my students. Both are passionate about coding with Python and I'm
> thinking of asking them to produce something that can either a) calculate
> derivatives of simple functions or b) assign 1 student the product rule,
> one the quotient rule, and have them write a program that can tackle
> derivatives in those forms.
>
> The problem is: I know very little about Python.
>
> Unfortunately, I don't have a specific question but I'm looking for any
> valuable insights or suggestions.


This may be one of the places where they can see more clearly that the
type of the derivative operator is:

    derivative: (number->number) -> (number->number)

That is, that the derivative is a function that consumes functions as
first-class values, and produces first-class values.  Calculus is
supposed to be one of the first places where they see a function whose
domain and range are not plain, inert numbers, but functions.

A numerical approach leads naturally to:
http://aroberge.blogspot.com/2005/04/computing-derivatives-using-python.html


You might also want to look at:

    https://mitpress.mit.edu/sicp/full-text/sicp/book/node39.html

for some inspiration: it shows in another language (Scheme) how to
approach symbolic differentiation from first principles.

From dyoo at hashcollision.org  Fri Feb 12 11:39:04 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Fri, 12 Feb 2016 08:39:04 -0800
Subject: [Tutor] Python for Grade 12 Calculus
In-Reply-To: <CAGZAPF7DLWUCpv41-jk=44uzzQfbUvOAdN297qUPBLsA6_smnQ@mail.gmail.com>
References: <CA+zUy=kfVC7oUQH3J=Za_2UY1CzFO9B9M1ecwUW0gAqiQG3z1A@mail.gmail.com>
 <CAGZAPF7DLWUCpv41-jk=44uzzQfbUvOAdN297qUPBLsA6_smnQ@mail.gmail.com>
Message-ID: <CAGZAPF7j8O+2qk9WNPU1PuJMy9sJZ4M2WWGXD=e3rLrj04z=_A@mail.gmail.com>

On Fri, Feb 12, 2016 at 8:33 AM, Danny Yoo <dyoo at hashcollision.org> wrote:
> On Thu, Feb 11, 2016 at 2:56 PM, Nicholas Tomasic
> <ntomasic.blyth at gmail.com> wrote:
>> Hi,
>>
>> I'm a 12th grade Calculus teacher at a school in Toronto, Ontario, and I'm
>> thinking about incorporating Python into an Independent Study Project for
>> two of my students. Both are passionate about coding with Python and I'm
>> thinking of asking them to produce something that can either a) calculate
>> derivatives of simple functions or b) assign 1 student the product rule,
>> one the quotient rule, and have them write a program that can tackle
>> derivatives in those forms.


Also, by the way, note that this is a fairly standard problem for a
programmer going for an interview:

    https://www.youtube.com/watch?v=2cf9xo1S134

From dyoo at hashcollision.org  Fri Feb 12 11:55:36 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Fri, 12 Feb 2016 08:55:36 -0800
Subject: [Tutor] Buffer code review
In-Reply-To: <CAG7edPFF3j1J9VTdUjjq+S=UDKaJuenSK=h6Xu58WjzLegstZQ@mail.gmail.com>
References: <CAG7edPFF3j1J9VTdUjjq+S=UDKaJuenSK=h6Xu58WjzLegstZQ@mail.gmail.com>
Message-ID: <CAGZAPF5h+HAusDV+NHft_U-7Vy4xL5zS0jNQ0PS=z-=KPA5UwQ@mail.gmail.com>

Hi Richard,

Make sure you understand the semantics of socket.send.  It does *not*
guarantee that it sends all the data, and the comment you have in your
program

> def sendLine(line, i):
>     s =  '\x02' + line[:-1] + '\x03'
>     logging.info('sending line ' + str(i))
>     print i
>     socktx.send(s)  # send the entire line to the socket
>     f2.write(s + '\n')
> #    time.sleep(0.2)
>     return

isn't actually true: depending on the conditions on the socket, it may
not send all the bytes in s.  See:
https://docs.python.org/2/library/socket.html#socket.socket.send for
more details.

From hulbert21 at gmail.com  Fri Feb 12 13:51:47 2016
From: hulbert21 at gmail.com (Brianna Hulbert)
Date: Fri, 12 Feb 2016 13:51:47 -0500
Subject: [Tutor] question
Message-ID: <CAEjf7Gs8HRw6Cc_n-bW709uCmPOsZXfBWavTXM41iqy4vqxZbA@mail.gmail.com>

Hi,

I am new to python and just downloaded it onto my computer. The window
comes up however there is no menu bar across the top that says file, edit,
format, run, options, etc. I was wondering how to get that to appear.

Thank you

From sinasareth at yahoo.com  Fri Feb 12 19:02:03 2016
From: sinasareth at yahoo.com (sina sareth)
Date: Sat, 13 Feb 2016 00:02:03 +0000 (UTC)
Subject: [Tutor] AWS /Django/Apache/Python/Github
References: <1770162617.3531361.1455321723966.JavaMail.yahoo.ref@mail.yahoo.com>
Message-ID: <1770162617.3531361.1455321723966.JavaMail.yahoo@mail.yahoo.com>

Hi ThereHow are you looking for good videos on the deploy apps with Python Framework.?I wonder if somebody has any good videos.?Thanks?
Sina

From wolfrage8765 at gmail.com  Fri Feb 12 20:37:56 2016
From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com)
Date: Fri, 12 Feb 2016 20:37:56 -0500
Subject: [Tutor] AWS /Django/Apache/Python/Github
In-Reply-To: <1770162617.3531361.1455321723966.JavaMail.yahoo@mail.yahoo.com>
References: <1770162617.3531361.1455321723966.JavaMail.yahoo.ref@mail.yahoo.com>
 <1770162617.3531361.1455321723966.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <CAOhNYvnoUKaAAf8pCd9rgVOmyr5Vj8vbHsN6qfusOiTwevyrqQ@mail.gmail.com>

What kind of apps do you want to deploy? Mobile; Web; Desktop; or Server?

From wolfrage8765 at gmail.com  Fri Feb 12 21:07:32 2016
From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com)
Date: Fri, 12 Feb 2016 21:07:32 -0500
Subject: [Tutor] question
In-Reply-To: <CAEjf7Gs8HRw6Cc_n-bW709uCmPOsZXfBWavTXM41iqy4vqxZbA@mail.gmail.com>
References: <CAEjf7Gs8HRw6Cc_n-bW709uCmPOsZXfBWavTXM41iqy4vqxZbA@mail.gmail.com>
Message-ID: <CAOhNYvnkARB257TiasogCBnCUE3Cd4qbBv=cx60UQnCG15b3bw@mail.gmail.com>

When you say "Window" are you referring to the Installer or Idle?
Which Operating System are you using and what version of Python did
you download? Did you run the installer that you downloaded?

From wolfrage8765 at gmail.com  Fri Feb 12 23:34:36 2016
From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com)
Date: Fri, 12 Feb 2016 23:34:36 -0500
Subject: [Tutor] AWS /Django/Apache/Python/Github
In-Reply-To: <E452C313-41FA-4825-922C-29BBCC58953D@yahoo.com>
References: <1770162617.3531361.1455321723966.JavaMail.yahoo.ref@mail.yahoo.com>
 <1770162617.3531361.1455321723966.JavaMail.yahoo@mail.yahoo.com>
 <CAOhNYvnoUKaAAf8pCd9rgVOmyr5Vj8vbHsN6qfusOiTwevyrqQ@mail.gmail.com>
 <E452C313-41FA-4825-922C-29BBCC58953D@yahoo.com>
Message-ID: <CAOhNYvnbqpeP7fQUMuf=9RnbB+3stFN3RNWtCSAUBW12UDe-8A@mail.gmail.com>

I recommend that you give Kivy a try for desktop and mobile give it a look
up on YouTube.
https://kivy.org/
http://m.youtube.com/watch?v=F7UKmK9eQLY

From kwpolska at gmail.com  Sat Feb 13 03:56:20 2016
From: kwpolska at gmail.com (Chris Warrick)
Date: Sat, 13 Feb 2016 09:56:20 +0100
Subject: [Tutor] AWS /Django/Apache/Python/Github
In-Reply-To: <1770162617.3531361.1455321723966.JavaMail.yahoo@mail.yahoo.com>
References: <1770162617.3531361.1455321723966.JavaMail.yahoo.ref@mail.yahoo.com>
 <1770162617.3531361.1455321723966.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <CAMw+j7LWfC=9vnD9U_+3_53SMynP61aR_ANj-Nvb18FW3h11eQ@mail.gmail.com>

On 13 February 2016 at 01:02, sina sareth via Tutor <tutor at python.org> wrote:
> Hi ThereHow are you looking for good videos on the deploy apps with Python Framework. I wonder if somebody has any good videos. Thanks
> Sina

I don?t have any videos, but I do have a comprehensive tutorial on
deploying a Python application on a Linux server:

https://chriswarrick.com/blog/2016/02/10/deploying-python-web-apps-with-nginx-and-uwsgi-emperor/

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16

From sinasareth at yahoo.com  Fri Feb 12 23:28:12 2016
From: sinasareth at yahoo.com (Sina Sareth)
Date: Fri, 12 Feb 2016 20:28:12 -0800
Subject: [Tutor] AWS /Django/Apache/Python/Github
In-Reply-To: <CAOhNYvnoUKaAAf8pCd9rgVOmyr5Vj8vbHsN6qfusOiTwevyrqQ@mail.gmail.com>
References: <1770162617.3531361.1455321723966.JavaMail.yahoo.ref@mail.yahoo.com>
 <1770162617.3531361.1455321723966.JavaMail.yahoo@mail.yahoo.com>
 <CAOhNYvnoUKaAAf8pCd9rgVOmyr5Vj8vbHsN6qfusOiTwevyrqQ@mail.gmail.com>
Message-ID: <E452C313-41FA-4825-922C-29BBCC58953D@yahoo.com>

Primary desktop and mobile 
Thanks 

Sent from my iPhone

> On Feb 12, 2016, at 5:37 PM, "wolfrage8765 at gmail.com" <wolfrage8765 at gmail.com> wrote:
> 
> What kind of apps do you want to deploy? Mobile; Web; Desktop; or Server?

From esawiek at gmail.com  Sat Feb 13 22:53:04 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Sat, 13 Feb 2016 22:53:04 -0500
Subject: [Tutor] genfromtx- date and time conversions
Message-ID: <CA+ZkTxsALpX2pcNF7GfKQ0xkVKb5pXfN=Vwt24x2=gQXoYxXhg@mail.gmail.com>

I am using genfromtxt to read a csv file with data types that include
string, float, integer, date, and time. I was able to accomplish such a
task using reader.


I thought there might be easier ways and the one I thought about is
genfromtxt. I have 2 questions: (1) Is there a function (method) to convert
time in H:M:S, format, e.g. 12:30:30 to decimal or 24 format e.g.12.508 w/o
writing my own function to convert it ? (2) Given a date of the form
12/20/2005, can such date in such format be converted with converters in
numpy using datetime to ?2005-20-15?? I had to reformat the date on my csv
file to the later form for python to read it correctly.



Thanks-EKE

From alan.gauld at btinternet.com  Sun Feb 14 04:06:16 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 14 Feb 2016 09:06:16 +0000
Subject: [Tutor] genfromtx- date and time conversions
In-Reply-To: <CA+ZkTxsALpX2pcNF7GfKQ0xkVKb5pXfN=Vwt24x2=gQXoYxXhg@mail.gmail.com>
References: <CA+ZkTxsALpX2pcNF7GfKQ0xkVKb5pXfN=Vwt24x2=gQXoYxXhg@mail.gmail.com>
Message-ID: <n9pg28$bb1$1@ger.gmane.org>

On 14/02/16 03:53, Ek Esawi wrote:

> genfromtxt. I have 2 questions: (1) Is there a function (method) to convert
> time in H:M:S, format, e.g. 12:30:30 to decimal or 24 format e.g.12.508 

The decimal format is not a common one but the best place to start
are the time and datetime (and calendar modules for leap years)
in the standard library. They will make writing your own
converter easier.

For time handling the time modsule includes the strptime/strftime
functions which can read/write string representations of times.

> writing my own function to convert it ? (2) Given a date of the form
> 12/20/2005, can such date in such format be converted with converters in
> numpy using datetime to ?2005-20-15?? I had to reformat the date on my csv
> file to the later form for python to read it correctly.

strptime/strftime can do that for you. You might find the
datetime module versions of the functions more convenient
for dates.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From paul.hermeneutic at gmail.com  Sun Feb 14 18:42:50 2016
From: paul.hermeneutic at gmail.com (paul.hermeneutic at gmail.com)
Date: Sun, 14 Feb 2016 16:42:50 -0700
Subject: [Tutor] tk Radiobutton example
Message-ID: <CAJ4+4aovewUOZTL8h1BuOP6UyxrUJmN-a1RHGt6E=MQW3znVcw@mail.gmail.com>

+I am working through "Python GUI Programming Cookbook" by Burkhard Meier.
https://www.packtpub.com/application-development/python-gui-programming-cookbook

On page 20, the variable curRad is set to a string then to a
Radiobutton. What is the purpose of setting curRad to a string?

for col in range(3): # 3
    curRad = 'rad' + str(col)
    curRad = tk.Radiobutton(win, text=colors[col], variable=radVar,
value=col, command=radCall)
    curRad.grid(column=col, row=5, sticky=tk.W)

From __peter__ at web.de  Mon Feb 15 04:48:34 2016
From: __peter__ at web.de (Peter Otten)
Date: Mon, 15 Feb 2016 10:48:34 +0100
Subject: [Tutor] tk Radiobutton example
References: <CAJ4+4aovewUOZTL8h1BuOP6UyxrUJmN-a1RHGt6E=MQW3znVcw@mail.gmail.com>
Message-ID: <n9s6tj$mdh$1@ger.gmane.org>

paul.hermeneutic at gmail.com wrote:

> +I am working through "Python GUI Programming Cookbook" by Burkhard Meier.
> https://www.packtpub.com/application-development/python-gui-programming-cookbook
> 
> On page 20, the variable curRad is set to a string then to a
> Radiobutton. What is the purpose of setting curRad to a string?
> 
> for col in range(3): # 3
>     curRad = 'rad' + str(col)
>     curRad = tk.Radiobutton(win, text=colors[col], variable=radVar,
> value=col, command=radCall)
>     curRad.grid(column=col, row=5, sticky=tk.W)

In the code above the line

>     curRad = 'rad' + str(col)

has no effect. You can safely remove it.

If you care you can check if this is already in the errata, and submit it 
here if it isn't:

https://www.packtpub.com/books/content/support/22858


From alan.gauld at btinternet.com  Mon Feb 15 04:50:54 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 15 Feb 2016 09:50:54 +0000
Subject: [Tutor] tk Radiobutton example
In-Reply-To: <CAJ4+4aovewUOZTL8h1BuOP6UyxrUJmN-a1RHGt6E=MQW3znVcw@mail.gmail.com>
References: <CAJ4+4aovewUOZTL8h1BuOP6UyxrUJmN-a1RHGt6E=MQW3znVcw@mail.gmail.com>
Message-ID: <n9s71u$oia$1@ger.gmane.org>

On 14/02/16 23:42, paul.hermeneutic at gmail.com wrote:

> On page 20, the variable curRad is set to a string then to a
> Radiobutton. What is the purpose of setting curRad to a string?
> 
> for col in range(3): # 3
>     curRad = 'rad' + str(col)
>     curRad = tk.Radiobutton(win, text=colors[col], variable=radVar,
> value=col, command=radCall)
>     curRad.grid(column=col, row=5, sticky=tk.W)

No purpose whatsoever, it looks like either a mistake or a typo.
Mostly a mistake. Is there a similar piece of code elsewhere in the book
that may have been cut n pasted and then badly edited?


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From cmgcomsol at gmail.com  Mon Feb 15 08:09:03 2016
From: cmgcomsol at gmail.com (CMG Thrissur)
Date: Mon, 15 Feb 2016 18:39:03 +0530
Subject: [Tutor] asyncio or threading
Message-ID: <56C1CDEF.3030803@gmail.com>

Hello,

I just wanted an opinion on the subject, asyncio and threading both seam 
to do the same job, but i feel threading is better because of the 
multiple ways i can control it.

Just want get the groups opinion based on there experience in using 
asyncio or threading in real life problems.

thanks

George

From alan.gauld at btinternet.com  Mon Feb 15 14:24:17 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 15 Feb 2016 19:24:17 +0000
Subject: [Tutor] asyncio or threading
In-Reply-To: <56C1CDEF.3030803@gmail.com>
References: <56C1CDEF.3030803@gmail.com>
Message-ID: <n9t8l1$raf$1@ger.gmane.org>

On 15/02/16 13:09, CMG Thrissur wrote:

> I just wanted an opinion on the subject, asyncio and threading both seam 
> to do the same job, but i feel threading is better because of the 
> multiple ways i can control it.

They do similar jobs but asyncio uses threading in the background
and I believe it's at OS level not Python native threading (which
has some significant issues).

However, the biggest differences are in the conceptual usage. asyncio is
intended primarily for event driven scenarios where something happens
that triggers a long running (eg. a hundred milliseconds or longer,
say). You add the task to the event loop and it gets processes as and
when it gets scheduled. You don't care and just wait for it to complete
after which it will call your callback function. So if you have a
scenario where tasks can be scheduled and forgotten about then asyncio
is ideal. The obvious use case is a server receiving
messages over a network and returning results when done.

If you want to do something in near real-time with a separate
thread where you comunicate between two (or more) threads then
asyncio is not such a good fit and conventional threading will
likely be better. But it's a lot harder to get right. With asyncio
all the hard threading bits are done for you.

Finally, if you have done any work with NodeJS using Javascript
you are likely to find asyncio a natural fit to your style.

HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From zachary.ware+pytut at gmail.com  Mon Feb 15 14:33:09 2016
From: zachary.ware+pytut at gmail.com (Zachary Ware)
Date: Mon, 15 Feb 2016 13:33:09 -0600
Subject: [Tutor] asyncio or threading
In-Reply-To: <56C1CDEF.3030803@gmail.com>
References: <56C1CDEF.3030803@gmail.com>
Message-ID: <CAKJDb-M33L-c_F46JffjJFVRft0BRaRsr5Mr4YiV3i0jYMy=sQ@mail.gmail.com>

On Feb 15, 2016 1:12 PM, "CMG Thrissur" <cmgcomsol at gmail.com> wrote:
>
> Hello,
>
> I just wanted an opinion on the subject, asyncio and threading both seam
to do the same job, but i feel threading is better because of the multiple
ways i can control it.
>
> Just want get the groups opinion based on there experience in using
asyncio or threading in real life problems.

This is more of a python-list subject, I think you would receive more and
better answers there (at the cost of a lot more posts to the thread that
are only tangentially related to your query).

To answer with my own vague impressions (based on experience, but not
analyzed in depth): asyncio is substantially easier to work with, while
threads take a lot more thought to make sure you aren't trampling on what
another thread is doing. It's also quite easy to use threading for just
some parts that need it, like a blocking function that may take a long time
to complete, while using asyncio for everything else (see
BaseEventLoop.run_in_executor()).

--
Zach
(On a phone)

From zachary.ware+pytut at gmail.com  Mon Feb 15 16:08:19 2016
From: zachary.ware+pytut at gmail.com (Zachary Ware)
Date: Mon, 15 Feb 2016 15:08:19 -0600
Subject: [Tutor] asyncio or threading
In-Reply-To: <n9t8l1$raf$1@ger.gmane.org>
References: <56C1CDEF.3030803@gmail.com>
	<n9t8l1$raf$1@ger.gmane.org>
Message-ID: <CAKJDb-OsuOxFN0qnvq3J54UjRF_37+a=WuL_zf+ewf3vSMdFkA@mail.gmail.com>

On Feb 15, 2016 1:25 PM, "Alan Gauld" <alan.gauld at btinternet.com> wrote:
>
> On 15/02/16 13:09, CMG Thrissur wrote:
>
> > I just wanted an opinion on the subject, asyncio and threading both seam
> > to do the same job, but i feel threading is better because of the
> > multiple ways i can control it.
>
> They do similar jobs but asyncio uses threading in the background
> and I believe it's at OS level not Python native threading (which
> has some significant issues).

This isn't quite accurate. asyncio only uses threading if you explicitly
tell it to use a thread for something, otherwise everything is run in
coroutines in the main thread, which are based on generators. Also, Python
threads are native OS threads, but the threading module does a bit of
management to allow you to keep track of them.

> However, the biggest differences are in the conceptual usage. asyncio is
> intended primarily for event driven scenarios where something happens
> that triggers a long running (eg. a hundred milliseconds or longer,
> say). You add the task to the event loop and it gets processes as and
> when it gets scheduled. You don't care and just wait for it to complete
> after which it will call your callback function. So if you have a
> scenario where tasks can be scheduled and forgotten about then asyncio
> is ideal. The obvious use case is a server receiving
> messages over a network and returning results when done.

This is not all asyncio can do. Callbacks are only one way of using it, the
other, more common method is to use coroutines such that most of your code
reads almost like regular, synchronous code.

> If you want to do something in near real-time with a separate
> thread where you comunicate between two (or more) threads then
> asyncio is not such a good fit and conventional threading will
> likely be better. But it's a lot harder to get right. With asyncio
> all the hard threading bits are done for you.

This doesn't quite apply, since asyncio doesn't use threads :).

> Finally, if you have done any work with NodeJS using Javascript
> you are likely to find asyncio a natural fit to your style.

I don't think this is quite true either; I've heard the JavaScript
equivalent referred to frequently as "callback hell", which is not the
idiomatic usage of asyncio. Callbacks are supported, because they are
useful in several instances, but they should not be the only (or even main,
usually) way asyncio is used.

--
Zach
(On a phone, apologies for brevity)

From tbrodle at hotmail.com  Mon Feb 15 19:00:50 2016
From: tbrodle at hotmail.com (Tom Brodle)
Date: Mon, 15 Feb 2016 17:00:50 -0700
Subject: [Tutor] What do each of these functions do?
Message-ID: <BLU170-W4243E1047FF1B2A36D6441A9AD0@phx.gbl>

I have requested help related to a program before, but did not see an answer.  I am new to the communicating with "tutor", IE I may have missed the answer.  This time I will simplify my inquiry.
My previous programming has been with "Basic", so please give me a break.
What do the following two lines of code do, individually,  and where in the code does the pointer or (program counter) end up after it has finished that line, assuming it is the last line of code in each of two programs?

GPIO.cleanup()

quit(code=None)

What document would answer my question?

Thanks



 		 	   		  

From alan.gauld at btinternet.com  Mon Feb 15 20:18:17 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 16 Feb 2016 01:18:17 +0000
Subject: [Tutor] asyncio or threading
In-Reply-To: <CAKJDb-OsuOxFN0qnvq3J54UjRF_37+a=WuL_zf+ewf3vSMdFkA@mail.gmail.com>
References: <56C1CDEF.3030803@gmail.com> <n9t8l1$raf$1@ger.gmane.org>
 <CAKJDb-OsuOxFN0qnvq3J54UjRF_37+a=WuL_zf+ewf3vSMdFkA@mail.gmail.com>
Message-ID: <n9ttco$v5m$1@ger.gmane.org>

On 15/02/16 21:08, Zachary Ware wrote:

> This isn't quite accurate. asyncio only uses threading if you explicitly
> tell it to use a thread for something, otherwise everything is run in
> coroutines in the main thread, 

Thanks for that, my asyncio tutorial was obviously seriously
one sided, it made no mention of coroutines but did promise
threading. But thanks to your reply I've now discovered Python
coroutines. Something I had completely missed until now.

> This is not all asyncio can do. Callbacks are only one way of using it, the
> other, more common method is to use coroutines such that most of your code
> reads almost like regular, synchronous code.

I'm not sure about "most common" since every single example of
asyncio I've seen so far has used callbacks not coroutines.
(Not that I've seen too many examples, but the ones I have seen
all used callbacks.)

>> Finally, if you have done any work with NodeJS using Javascript
>> you are likely to find asyncio a natural fit to your style.
> 
> I don't think this is quite true either; I've heard the JavaScript
> equivalent referred to frequently as "callback hell",

A slanderous comment that's not really deserved :-) NodeJS code
is IMHO, one of the better uses of Javascript (a language with
many warts!). It's certainly not too difficult to work with
for anyone familiar with callbacks in any other language.

> idiomatic usage of asyncio. Callbacks are supported, because they are
> useful in several instances, but they should not be the only (or even main,
> usually) way asyncio is used.

As a matter of interest which tutorials on asyncio cover this best?
I've only read one closely and it was all about callbacks but
it impressed me enough to become an asyncio fan. However I'm
certainly interested in finding out more on the coroutine front.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From dyoo at hashcollision.org  Mon Feb 15 20:42:01 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 15 Feb 2016 17:42:01 -0800
Subject: [Tutor] What do each of these functions do?
In-Reply-To: <BLU170-W4243E1047FF1B2A36D6441A9AD0@phx.gbl>
References: <BLU170-W4243E1047FF1B2A36D6441A9AD0@phx.gbl>
Message-ID: <CAGZAPF5fHdvaBiKt2bMaP19-oKNF1ozoox2xONM_Dk_uHRC7fw@mail.gmail.com>

On Mon, Feb 15, 2016 at 4:00 PM, Tom Brodle <tbrodle at hotmail.com> wrote:

> What do the following two lines of code do, individually,  and where in the code does the pointer or (program counter) end up after it has finished that line, assuming it is the last line of code in each of two programs?
>
> GPIO.cleanup()
>
> quit(code=None)


As in regular human language, the terms we use don't have a universal
meaning: they are sensitive to the context of the usage.  On
Python-tutor, we try to stay within the context of the Standard
Library, which comes provided when we use the Python language as a
basic part of the installation.


The unqualified "quit" probably refers to the built-in quit in
Python's standard library.  If we look at:

    https://docs.python.org/3.4/genindex-Q.html

we should eventually find the reference to:

    https://docs.python.org/3.4/library/constants.html#quit

which explains what the quit function does, and it matches
signature-wise with the usage you're showing us, so I think that's the
right "quit".  Please read the docs there and feel free to ask
questions if you need more clarification on its meaning.


Note that GPIO doesn't show up in the index of known modules or
functions in the Python Standard Library.  I looked through:

    https://docs.python.org/3.4/genindex-G.html

and it does not occur there.

One part of your question left out some crucial information: in what
context did you see these lines?  Knowing where you saw these lines
would provide a clue.

My best guess is that GPIO is defined by the Python libraries provided
by the Raspberry Pi project.  Is that true?  If that's the case,
please see:

    https://www.raspberrypi.org/documentation/usage/python/more.md
    https://sourceforge.net/p/raspberry-gpio-python/wiki/BasicUsage/

In particular, within the BasicUsage link, there's a section in the
raspberry-pi documentation that talks more about what GPIO.cleanup()
does.  But if that's the case, this is very domain-specific, and you
may want to ask Pi-related questions on a forum such as:

    https://www.raspberrypi.org/forums/viewforum.php?f=32

since the folks there will be familiar with the specific details of
the functions that Raspberry Pi introduces.

From dyoo at hashcollision.org  Mon Feb 15 21:24:50 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Mon, 15 Feb 2016 18:24:50 -0800
Subject: [Tutor] asyncio or threading
In-Reply-To: <56C1CDEF.3030803@gmail.com>
References: <56C1CDEF.3030803@gmail.com>
Message-ID: <CAGZAPF7hbQkFQrua3Qqn15xO41+-PuYvM8C=xO1XvkH9G3TLBg@mail.gmail.com>

> I just wanted an opinion on the subject, asyncio and threading both seem to
> do the same job, but i feel threading is better because of the multiple ways
> i can control it.
>
> Just want get the groups opinion based on there experience in using asyncio
> or threading in real life problems.


I just wanted to chime in: the decision here is more accurately
described as: "synchronous" vs "asynchronous" style.

In a synchronous programming model, functions may block if they need
to do work that takes a while to complete.  By "block", we expect
control flow to stay there until the long-running operation finishes.
One repercussion of this is that it's easy to express the idea of
doing a sequence of staged operations, each of which might take a
while to complete.  e.g.

    stage1Results = stage1()
    stage2(stage1Results)
    ....

An advantage of this approach is that it's fairly clear what the
sequence of events will be: stage1 gets called, and when it's done,
stage2 is next in line.

A disadvantage is that if we forget that functions can block, then we
may prevent our program from making quick progress if there's only a
single flow-of-control through our program.

A major role of threading in Python is to allow the user to create
multiple flows of control through a program, all running concurrently,
so that even if one thread of control is blocking on a long-running
function, this may not necessarily stop other flows of control from
making progress toward some goal.

---

In an asynchronous programming model, functions are not allowed to
block: they must return immediately, with progress deferred some time
later in the future.

In order to express the idea of doing a sequence of staged operations,
where subsequent stages might depend on the results of prior stages,
we need to express to the system "what to do next" when the
long-running process finally has finished.  And unlike in the blocking
case, it's not possible for us to just say:

    stage1Results = stage1_nonblocking()
    stage2_nonblocking(stage1Results)
    ....

because the value of stage1Results may not be ready yet!

So in an asynchronous programming model, we need to adjust our
programs to handle a world where no function is ever allowed to block
execution, and yet we want to sequence code such that a later stage
can depend on results of a previous stage.  And, of course, we need to
delay "what do do next" until the value is actually ready to be used.

One mechanism (but not the only one!) we can use to express the "what
to do next" is the "callback", which is a function that is to be
called when the progress from a call is complete.

In callback style, we'd express the staged pseudocode like this:

    def handleStage1Results(stage1Results):
        stage2_nonblocking(stage1Results)

    stage1_nonblocking(success=handleStage1Results)

where we pass function objects around to the function that will take a
long time to finish its work.  We expect our callback functions to be
"called back" later by some party after some point.  In many
asynchronous I/O systems, the responsible party is some event loop
that keeps track of what non-blocking calls are still making progress
and which ones are done.

An advantage of this approach is that we don't need multiple flows of
control to do concurrent programming.  Some language environments
don't provide a mechanism for creating multiple flows of control, and
hence are essentially forced into the asynchronous programming model
because they have no other choice.

A disadvantage of asynchronous programming is that it can twist the
structure of programs.  Not only does it make programs less easy to
read, but they can also be harder to debug.  Concretely: we might lose
useful, simple stack traces when errors occur, depending on how
asynchronous programs are implemented, because parts of the stack can
be eliminated since functions never block.


(It's not impossible to get good stack traces in an asynchronous
programming model.  It's just that it's something that we can't take
for granted.  See:
http://www.html5rocks.com/en/tutorials/developertools/async-call-stack/
for an explanation in the context of JavaScript.)

From alan.gauld at btinternet.com  Tue Feb 16 03:52:55 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 16 Feb 2016 08:52:55 +0000
Subject: [Tutor] What do each of these functions do?
In-Reply-To: <BLU170-W4243E1047FF1B2A36D6441A9AD0@phx.gbl>
References: <BLU170-W4243E1047FF1B2A36D6441A9AD0@phx.gbl>
Message-ID: <n9uo16$aqi$1@ger.gmane.org>

On 16/02/16 00:00, Tom Brodle wrote:
> I have requested help related to a program before, but did not see an answer. 

If it was the post on Feb 4th then you got 2 answers
(from Joel and myself.)

> My previous programming has been with "Basic", so please give me a break.
> What do the following two lines of code do, individually,  
> 
> GPIO.cleanup()

This is, I assume from a RaspberryPi program?
If so, its not really in the scope of this group, which is for
the core Python language and library. However, as the
name suggests, it cleans up the GPIO (General Purpose Input
Output) bus connection. That usually entails freeing memory
resources and the like, none of which you really need to
worry about provided you call the function.

> quit(code=None)

This basically quits the Python interpreter.

> and where in the code does the pointer or (program counter)
> end up after it has finished that line, assuming it is the
> last line of code in each of two programs?

That doesn't really make any sense in a Python context.
Python translates your code into a Pseudocode format
that is run inside the interpreter. The interpreter is
running on the native hardware and will be incrementing
the native program counter as it goes but that has very
little correspondence to your Python code.

All you really need to know is that the Python statements
execute in sequence as self contained units.

> What document would answer my question?

If you want to know about how the Python interpreter
works then I'm not sure. If you just want to know
more about the quit() and GPIO functions then
there are documents available. For quit it's in
the standard library docs. For GPIO it will be
somewhere on your Pi installation.

HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From cmgcomsol at gmail.com  Tue Feb 16 07:26:17 2016
From: cmgcomsol at gmail.com (CMG Thrissur)
Date: Tue, 16 Feb 2016 17:56:17 +0530
Subject: [Tutor] asyncio or threading
In-Reply-To: <CAGZAPF7hbQkFQrua3Qqn15xO41+-PuYvM8C=xO1XvkH9G3TLBg@mail.gmail.com>
References: <56C1CDEF.3030803@gmail.com>
 <CAGZAPF7hbQkFQrua3Qqn15xO41+-PuYvM8C=xO1XvkH9G3TLBg@mail.gmail.com>
Message-ID: <56C31569.2070706@gmail.com>

Thank you all, especially Danny for enlightening about "synchronous" vs 
"asynchronous" style.  Further the groups response has helped me in my 
decision making, and views about using asyncio.

George

From jkgreenl at uwo.ca  Mon Feb 15 16:09:15 2016
From: jkgreenl at uwo.ca (Joan K Greenlee)
Date: Mon, 15 Feb 2016 21:09:15 +0000
Subject: [Tutor] Fw: PURCHASE ORDER #201623387
In-Reply-To: <SN1PR11MB06399F84134B3DCC4B8021CDADAC0@SN1PR11MB0639.namprd11.prod.outlook.com>
References: <SN1PR11MB06399F84134B3DCC4B8021CDADAC0@SN1PR11MB0639.namprd11.prod.outlook.com>
Message-ID: <SN1PR11MB063976C1CB10406B85A747E3ADAC0@SN1PR11MB0639.namprd11.prod.outlook.com>





Kindly find the attach PURCHASE ORDER #201623387 for review.




[cid:a94e441b-8229-49b2-916c-664fa9cfa5e0]


SRG- 15/Feb/2016

TAG-Org.com: The Global organization for professional services and education.
TAGIUNI.com "Providers of worldwide online accredited education".
==================================================================================================

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: NEW PURCHASE ORDER.html
URL: <http://mail.python.org/pipermail/tutor/attachments/20160215/ce8dd6d3/attachment-0001.ksh>

From badouglas at gmail.com  Tue Feb 16 10:30:54 2016
From: badouglas at gmail.com (bruce)
Date: Tue, 16 Feb 2016 10:30:54 -0500
Subject: [Tutor] really basic - finding multiline chunk within larger chunk
Message-ID: <CAP16ngrdj_6f=M4wEc8bzWP217ehsSBO+1qiJ2z_MKfzpj6WKw@mail.gmail.com>

Hi.

I've got a test, where I have a chunk of text "a" and a subset of text
"s2a". the subset is multiline.

For some reason, can't seem to return true on the find. I've pasted in
http://fpaste.org/323521/, but the following is an example as well.
(not sure if the psuedo code listed actually shows the chunks of text
with the whitespace correctly.

Obviously, this is an "example" not meant to be real code!!

Thoughts on what I've screwed up?

Thanks

aa='''<TR>
                  <TD align="LEFT" valign="TOP"></TD>
                  <TD align="LEFT" valign="TOP"
class="row_SectionLite"><STRONG>Retail Price</STRONG></TD>
                  <TD valign="TOP" class="row_SectionLite"></TD>
                  <TD align="RIGHT" valign="TOP"
class="row_SectionLite">Less than $10</TD>
                  <TD align="RIGHT" valign="TOP"></TD>
                 </TR>

                 <TR>
                  <TD align="LEFT" valign="TOP"></TD>
                  <TD align="LEFT" valign="TOP"
class="row_SectionLite"><STRONG>Required</STRONG></TD>
                  <TD valign="TOP" class="row_SectionLite"></TD>
                  <TD align="RIGHT" valign="TOP" class="row_SectionLite">
                   Yes
                  </TD>
                  <TD align="RIGHT" valign="TOP"></TD>
                 </TR>

                 <TR>
                  <TD align="LEFT" valign="TOP"></TD>
                  <TD align="LEFT" valign="TOP"
class="row_SectionLite"><STRONG>Used During</STRONG></TD>
                  <TD valign="TOP" class="row_SectionLite"></TD>
                  <TD align="RIGHT" valign="TOP"
class="row_SectionLite">Full Term</TD>
                  <TD align="RIGHT" valign="TOP"></TD>
                 </TR>

                 <TR>
                  <TD align="LEFT" valign="TOP"></TD>
                  <TD align="LEFT" valign="TOP"
class="row_SectionLite"><STRONG>Copies on Reserve in
Libraries</STRONG></TD>
                  <TD valign="TOP" class="row_SectionLite"></TD>
                  <TD align="RIGHT" valign="TOP" class="row_SectionLite">
                   No
                  </TD>
                  <TD align="RIGHT" valign="TOP"></TD>
                 </TR>'''

-----------------------------------------------

as a test:

    s2a='''<TR>
                  <TD align="LEFT" valign="TOP"></TD>
                  <TD align="LEFT" valign="TOP"
class="row_SectionLite"><STRONG>Required</STRONG></TD>
                  <TD valign="TOP" class="row_SectionLite"></TD>
                  <TD align="RIGHT" valign="TOP" class="row_SectionLite">
                   Yes
                  </TD>
                  <TD align="RIGHT" valign="TOP"></TD>
                 </TR>'''



          if (aa.find(s2a)>0):
            print "here ppp \n"
          else:
            print "errrrrr \n"

          sys.exit()

From joel.goldstick at gmail.com  Tue Feb 16 11:14:58 2016
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 16 Feb 2016 11:14:58 -0500
Subject: [Tutor] really basic - finding multiline chunk within larger
 chunk
In-Reply-To: <CAP16ngrdj_6f=M4wEc8bzWP217ehsSBO+1qiJ2z_MKfzpj6WKw@mail.gmail.com>
References: <CAP16ngrdj_6f=M4wEc8bzWP217ehsSBO+1qiJ2z_MKfzpj6WKw@mail.gmail.com>
Message-ID: <CAPM-O+wC+87sj+TQnfBbfwEuPkCbooJBPnc2OfLWSHtRtKxL+g@mail.gmail.com>

On Tue, Feb 16, 2016 at 10:30 AM, bruce <badouglas at gmail.com> wrote:

> Hi.
>
> I've got a test, where I have a chunk of text "a" and a subset of text
> "s2a". the subset is multiline.
>
> For some reason, can't seem to return true on the find. I've pasted in
> http://fpaste.org/323521/, but the following is an example as well.
> (not sure if the psuedo code listed actually shows the chunks of text
> with the whitespace correctly.
>
> Obviously, this is an "example" not meant to be real code!!
>
> Thoughts on what I've screwed up?
>
> I have some thoughts!  First, the code you posted at the link doesn't
run.  Its usually a good thing to post example code if your real code is
large, and you can identify the problem.  Instead you post a long example
elsewhere that doesn't run.  So as to what was wrong, you had improper
indentation and some comments that weren't commented out.  When I fixed
those problems the code ran fine:

jcg at jcg:~$ python temp.py
here ppp 436


Its hard to tell what you want to do in the bigger picture, but this
probably isn't the right way to go.  Many people use BeautifulSoup to parse
html.   And where did you find that HTML?  Code like that should have died
19 years ago when CSS came on the seen.

> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick
http://joelgoldstick.com/stats/birthdays

From __peter__ at web.de  Tue Feb 16 14:10:49 2016
From: __peter__ at web.de (Peter Otten)
Date: Tue, 16 Feb 2016 20:10:49 +0100
Subject: [Tutor] really basic - finding multiline chunk within larger
 chunk
References: <CAP16ngrdj_6f=M4wEc8bzWP217ehsSBO+1qiJ2z_MKfzpj6WKw@mail.gmail.com>
Message-ID: <n9vs7r$2vq$1@ger.gmane.org>

bruce wrote:

> I've got a test, where I have a chunk of text "a" and a subset of text
> "s2a". the subset is multiline.
> 
> For some reason, can't seem to return true on the find. I've pasted in
> http://fpaste.org/323521/, but the following is an example as well.
> (not sure if the psuedo code listed actually shows the chunks of text
> with the whitespace correctly.
> 
> Obviously, this is an "example" not meant to be real code!!
> 
> Thoughts on what I've screwed up?

str.find() returns -1 when the substring is not found. Other return values 
including 0 mean that the substring was found -- the return value is the 
start index of the substring in the larger string:

>>> "foo bar".find("bar")
4
>>> "foo bar".find("foo")
0
>>> "foo bar".find("baz")
-1

If you only want to know if the substring occurs and do not need the start 
index the clearest test uses the in operator:

>>> "foo" in "foo bar"
True



From dyoo at hashcollision.org  Tue Feb 16 23:56:05 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Tue, 16 Feb 2016 20:56:05 -0800
Subject: [Tutor] really basic - finding multiline chunk within larger
 chunk
In-Reply-To: <CAP16ngrdj_6f=M4wEc8bzWP217ehsSBO+1qiJ2z_MKfzpj6WKw@mail.gmail.com>
References: <CAP16ngrdj_6f=M4wEc8bzWP217ehsSBO+1qiJ2z_MKfzpj6WKw@mail.gmail.com>
Message-ID: <CAGZAPF7fLYxE3T381FEHbpFGR3TvDkGYws9mqwDCvTW5-jhqAQ@mail.gmail.com>

Hi Bruce,

> For some reason, can't seem to return true on the find. I've pasted in
> http://fpaste.org/323521/, but the following is an example as well.
> (not sure if the psuedo code listed actually shows the chunks of text
> with the whitespace correctly.


As others have mentioned, pattern matching on tree structures by using
string comparison is a fragile approach, and the mismatch is probably
reflecting that fragility: it'll mismatch entirely even if a single
whilespace character is not the same as the pattern you're matching.


Looking at your input, I see that you're expecting to match the second
row of content:

>
>                  <TR>
>                   <TD align="LEFT" valign="TOP"></TD>
>                   <TD align="LEFT" valign="TOP"
> class="row_SectionLite"><STRONG>Required</STRONG></TD>
>                   <TD valign="TOP" class="row_SectionLite"></TD>
>                   <TD align="RIGHT" valign="TOP" class="row_SectionLite">
>                    Yes
>                   </TD>
>                   <TD align="RIGHT" valign="TOP"></TD>
>                  </TR>
>


where your pattern string is

>
>     s2a='''<TR>
>                   <TD align="LEFT" valign="TOP"></TD>
>                   <TD align="LEFT" valign="TOP"
> class="row_SectionLite"><STRONG>Required</STRONG></TD>
>                   <TD valign="TOP" class="row_SectionLite"></TD>
>                   <TD align="RIGHT" valign="TOP" class="row_SectionLite">
>                    Yes
>                   </TD>
>                   <TD align="RIGHT" valign="TOP"></TD>
>                  </TR>'''
>


Huh.  Unfortunately, I can't reproduce your failure: it appears to
match successfully for me.  Example:

    https://repl.it/Bn34/0


Can you reproduce this?

From dyoo at hashcollision.org  Wed Feb 17 00:02:44 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Tue, 16 Feb 2016 21:02:44 -0800
Subject: [Tutor] really basic - finding multiline chunk within larger
 chunk
In-Reply-To: <CAGZAPF7fLYxE3T381FEHbpFGR3TvDkGYws9mqwDCvTW5-jhqAQ@mail.gmail.com>
References: <CAP16ngrdj_6f=M4wEc8bzWP217ehsSBO+1qiJ2z_MKfzpj6WKw@mail.gmail.com>
 <CAGZAPF7fLYxE3T381FEHbpFGR3TvDkGYws9mqwDCvTW5-jhqAQ@mail.gmail.com>
Message-ID: <CAGZAPF5kwaDdtnH=C2jyJc5Vz3WvVMDGfEZ_5JiYf53cNTvJcw@mail.gmail.com>

(Ah, I see that Joel Goldstick also was able to do the search
successfully; sorry about missing your message Joel!)

From fosiul at gmail.com  Tue Feb 16 17:28:19 2016
From: fosiul at gmail.com (Fosiul Alam)
Date: Tue, 16 Feb 2016 22:28:19 +0000
Subject: [Tutor] How to calculate high value from multiple lines for each
 column
Message-ID: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>

Hi
I am very new to python, basically , I want to get the Maximum value for
each column

                  0.000           0.000           0.000        0
(0.0%)           0.000           0.600
                  0.000           3.000           6.000        1
(0.0%)           0.300           0.000
                  3.000           0.000           0.000        0
(0.0%)           0.000           0.000
                  5.000           0.000           0.000        1
(0.0%)           0.200           0.000
                  0.000           7.000           3.000        0
(0.0%)           0.000           1.000
                  0.000           0.000           0.000        0
(0.0%)           0.000           0.000
                  0.000           0.000           0.000        0
(0.0%)           0.000           0.000

So  maximum value for 1st column=5
      maximum value for 2nd column = 7
      maximum value for 3rd colun =6
       .......................

How can I do this ?
i will be really greatfull if some give any hits.

Thanks for the help.

From kamintaylor at hotmail.com  Tue Feb 16 17:32:15 2016
From: kamintaylor at hotmail.com (taylor hansen)
Date: Tue, 16 Feb 2016 14:32:15 -0800
Subject: [Tutor] (no subject)
Message-ID: <BLU437-SMTP192F27450EF527C9EBA861B2AD0@phx.gbl>

Hi, I have to use Python 2 for school and I can?t seem to get it to work properly. I have to type from myro import* and whenever I do that, it says that there is no module named myro. I have been trying to get this to work for a couple days now. 

Thank you, 
Taylor



From magyar1886 at gmail.com  Tue Feb 16 15:19:47 2016
From: magyar1886 at gmail.com (Marco Soldavini)
Date: Tue, 16 Feb 2016 21:19:47 +0100
Subject: [Tutor] Hello! Questions
Message-ID: <CAO6YOj5-hcjJSUZhgAQHex-8qyV7tjH9YZ5F1xfJi7weissXqw@mail.gmail.com>

Hi,
I am almost new to python and I am trying to build a not so easy app (but
very neat and useful) related to industrial automation.

Is this the right place to knock down problems one by one?
Basically my app has several interactions but the most important is reading
values from an embedded machine (better a PLC) via the OPC protocol.

I already tested this with some values of various kind (real, boolean,
strings). Basically each tag is read with a fixed sample time from a stream
opened in OPC and recorded onto an array in python.

Is it better I explain you the main picture? I am not here for spoon
feeding but just tips and suggestion and maybe solution to particular
problems I might find on the track.

Something about OPC is here: https://opcfoundation.org/about/what-is-opc/

The lib I am using is openopc.

My first question is about data types, data structures in general and how
to organize an efficient loop for recording data.

Basically something like

while (stop condition false)
read data

From magyar1886 at gmail.com  Tue Feb 16 15:21:58 2016
From: magyar1886 at gmail.com (Marco Soldavini)
Date: Tue, 16 Feb 2016 21:21:58 +0100
Subject: [Tutor] Hello! Questions
Message-ID: <CAO6YOj5df6PXCPCwV0Xk6=ay+b-MOG+QuRK-DK-XXsFuzHj8Kw@mail.gmail.com>

I hit the send button too early. anyway

Basically something like

while (stop condition false)
    read data
    write data into local array or something
    wait sample time


Thanks
marco

From alan.gauld at btinternet.com  Wed Feb 17 05:04:43 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 17 Feb 2016 10:04:43 +0000
Subject: [Tutor] (no subject)
In-Reply-To: <BLU437-SMTP192F27450EF527C9EBA861B2AD0@phx.gbl>
References: <BLU437-SMTP192F27450EF527C9EBA861B2AD0@phx.gbl>
Message-ID: <na1gjr$a8a$1@ger.gmane.org>

On 16/02/16 22:32, taylor hansen wrote:

> I have to type from myro import* and whenever I do that, 
> it says that there is no module named myro.

myro is not a standard module so you need to install it
from somewhere. Presumably your school can give you
directions on that?

For future reference please always include the full
error message in posts, do not just summarize it.
That makes debugging a lot easier for us.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Wed Feb 17 05:13:56 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 17 Feb 2016 10:13:56 +0000
Subject: [Tutor] Hello! Questions
In-Reply-To: <CAO6YOj5-hcjJSUZhgAQHex-8qyV7tjH9YZ5F1xfJi7weissXqw@mail.gmail.com>
References: <CAO6YOj5-hcjJSUZhgAQHex-8qyV7tjH9YZ5F1xfJi7weissXqw@mail.gmail.com>
Message-ID: <na1h54$q5f$1@ger.gmane.org>

On 16/02/16 20:19, Marco Soldavini wrote:

> Is this the right place to knock down problems one by one?

Yes, although we are limited in scope to Python language
and standard library so for anything specific to openopc
you will probably need to ask on its support forum/list.

> I already tested this with some values of various kind (real, boolean,
> strings). Basically each tag is read with a fixed sample time from a stream
> opened in OPC and recorded onto an array in python.

Please be precise when describing data types.
Python does have an array module but its more usual to
use the built-in list type for this kind of thing.

> Is it better I explain you the main picture? I am not here for spoon
> feeding but just tips and suggestion and maybe solution to particular
> problems I might find on the track.

Yes, that's fine. Your description so far seems sufficient,
if we need more context later we will ask for it.

> The lib I am using is openopc.
> 
> My first question is about data types, data structures in general and how
> to organize an efficient loop for recording data.

> while (stop condition false)
>    read data
>    write data into local array or something
>    wait sample time

That's a good start. What is the question?
You say you know how to read data fro openopc, so the
first loop line should be fine.
Storing into a list involves the append method:

myDataList.append(myData)

and the wait will probably be openopc specific
but if not you can use the time.sleep() function.


If you have anything more specific please ask.
Post your code as you go.
Include the full error text if you get any.
And tell us the Python version and OS too.

hth
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Wed Feb 17 05:22:03 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 17 Feb 2016 10:22:03 +0000
Subject: [Tutor] How to calculate high value from multiple lines for
 each column
In-Reply-To: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
References: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
Message-ID: <na1hkb$32l$1@ger.gmane.org>

On 16/02/16 22:28, Fosiul Alam wrote:
> Hi
> I am very new to python, basically , I want to get the Maximum value for
> each column
> 
>                   0.000           0.000           0.000        0
> (0.0%)           0.000           0.600
>                   0.000           3.000           6.000        1
> (0.0%)           0.300           0.000
>                   3.000           0.000           0.000        0
> (0.0%)           0.000           0.000
>                   5.000           0.000           0.000        1
...
> 
> So  maximum value for 1st column=5
>       maximum value for 2nd column = 7
>       maximum value for 3rd colun =6
>        .......................
> 
> How can I do this ?

The classical way to represent a table of data is using a list of lists

myData = [[],[],[],[]]  # a table with 4 columns

You can then read the data line by line and insert the values into your
lists.

for line in dataSource:
   fields = line.split()
   myData[0].append(fields[0])
   myData[1].append(fields[1])
   etc

Then at the end find the max() of each column

for col in myData:
    print "Max = " max(col)

There are more efficient ways to do it but that's probably the simplest.
You need to fill in quite a few blanks, such as how you read your data
source - is it a file or what?

Try it and come back if you hit problems.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From oscar.j.benjamin at gmail.com  Wed Feb 17 08:58:31 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 17 Feb 2016 13:58:31 +0000
Subject: [Tutor] asyncio or threading
In-Reply-To: <CAGZAPF7hbQkFQrua3Qqn15xO41+-PuYvM8C=xO1XvkH9G3TLBg@mail.gmail.com>
References: <56C1CDEF.3030803@gmail.com>
 <CAGZAPF7hbQkFQrua3Qqn15xO41+-PuYvM8C=xO1XvkH9G3TLBg@mail.gmail.com>
Message-ID: <CAHVvXxSFLML3QPQZDwvy73mLbNf+EhraRq=icVikmi1-KmSa+w@mail.gmail.com>

On 16 February 2016 at 02:24, Danny Yoo <dyoo at hashcollision.org> wrote:
>
> where we pass function objects around to the function that will take a
> long time to finish its work.  We expect our callback functions to be
> "called back" later by some party after some point.  In many
> asynchronous I/O systems, the responsible party is some event loop
> that keeps track of what non-blocking calls are still making progress
> and which ones are done.
>
> An advantage of this approach is that we don't need multiple flows of
> control to do concurrent programming.  Some language environments
> don't provide a mechanism for creating multiple flows of control, and
> hence are essentially forced into the asynchronous programming model
> because they have no other choice.
>
> A disadvantage of asynchronous programming is that it can twist the
> structure of programs.  Not only does it make programs less easy to
> read, but they can also be harder to debug.  Concretely: we might lose
> useful, simple stack traces when errors occur, depending on how
> asynchronous programs are implemented, because parts of the stack can
> be eliminated since functions never block.

Asyncio (as in the new stdlib module as opposed to the generic
concept) is explicitly designed with the intention that it would not
have these problems. Firstly it's designed to avoid callbacks so
actually your example would look like:

    stage1Results = await stage1_nonblocking()
    await stage2_nonblocking(stage1Results)

The await keyword implicitly suspends this coroutine allowing others
to continue (driven by the event loop). The idea is that you write
stuff that looks like normal synchronous code but the keywords await
and async are used to make it concurrent.

Also asyncio deliberately preserves stack traces. You should still see
them in full glory. The difference is just that you have several
concurrent stacks and execution switches between them each time you
hit an await call. Each individual stack still behaves like a
synchronous one though.

--
Oscar

From badouglas at gmail.com  Wed Feb 17 10:01:19 2016
From: badouglas at gmail.com (bruce)
Date: Wed, 17 Feb 2016 10:01:19 -0500
Subject: [Tutor] really basic - finding multiline chunk within larger
 chunk
In-Reply-To: <CAGZAPF5kwaDdtnH=C2jyJc5Vz3WvVMDGfEZ_5JiYf53cNTvJcw@mail.gmail.com>
References: <CAP16ngrdj_6f=M4wEc8bzWP217ehsSBO+1qiJ2z_MKfzpj6WKw@mail.gmail.com>
 <CAGZAPF7fLYxE3T381FEHbpFGR3TvDkGYws9mqwDCvTW5-jhqAQ@mail.gmail.com>
 <CAGZAPF5kwaDdtnH=C2jyJc5Vz3WvVMDGfEZ_5JiYf53cNTvJcw@mail.gmail.com>
Message-ID: <CAP16ngoDdev840iyx-c2qQy8aYd7BY+O_3Hx+otsHXA8Zpcmnw@mail.gmail.com>

hmm...

Ok. For some reason, it appears to be a whitespace issue, which is
what I thought.


The basic process that was used to get the subchunk to test for, was
to actually do a copy/cut/paste of the subtext from the master text,
and then to write the code to test.

Yeah, testing for "text" with whitespaces/multiline can be fragile.
And yeah, the text might have been from the 90s but that's irrelevant!

Thanks for confirming what I thought. Thanks also for the sample code as well.

I might just wind up stripping tabs/spaces and joining on space to pre
massage the content prior to handling it..

'ppreciate it guys/gals!


On Wed, Feb 17, 2016 at 12:02 AM, Danny Yoo <dyoo at hashcollision.org> wrote:
> (Ah, I see that Joel Goldstick also was able to do the search
> successfully; sorry about missing your message Joel!)

From lwaters at flinthill.org  Wed Feb 17 10:45:46 2016
From: lwaters at flinthill.org (Lisa Hasler Waters)
Date: Wed, 17 Feb 2016 10:45:46 -0500
Subject: [Tutor] invalid syntax error in Run Run Module
Message-ID: <CAF91wy8=mujSJsPA0sbENNFcRON7tv7gUUp=OVffUUg6H=FkEg@mail.gmail.com>

Dear Python Tutor List,

We (my students and myself, the teacher -- all of us are new to Python &
coding) keep getting an error message when we try to Run Run Module from
IDLE. Here are the details:

-We are using MacBooks, running OX 10.11, 64-bit
-We have downloaded Python 3.5.1
-When we try to select Run Run Module - we get the error message "invalid
syntax," which points to the number 5 in Python 3.5.1
-We followed the recommendation to install ActiveState -- but that has not
resolved the problem
-Here is the exact message:

Python 3.5.1 (v3.5.1:37a07cee5969, Dec  5 2015, 21:12:44)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "copyright", "credits" or "license()" for more information.
==== No Subprocess ====

WARNING: Running IDLE without a Subprocess is deprecated
and will be removed in a later version. See Help/IDLE Help
for details.

>>> WARNING: The version of Tcl/Tk (8.5.9) in use may be unstable.
Visit http://www.python.org/download/mac/tcltk/ for current information.

>>> print("welcome")
welcome
>>>

Thank you for any advice you can share!!!

-- 
Lisa Waters, PhD
Technology Integration
Flint Hill School

From alan.gauld at btinternet.com  Wed Feb 17 12:36:55 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 17 Feb 2016 17:36:55 +0000
Subject: [Tutor] asyncio or threading
In-Reply-To: <n9ttco$v5m$1@ger.gmane.org>
References: <56C1CDEF.3030803@gmail.com> <n9t8l1$raf$1@ger.gmane.org>
 <CAKJDb-OsuOxFN0qnvq3J54UjRF_37+a=WuL_zf+ewf3vSMdFkA@mail.gmail.com>
 <n9ttco$v5m$1@ger.gmane.org>
Message-ID: <na2b3n$iv8$1@ger.gmane.org>

On 16/02/16 01:18, Alan Gauld wrote:

Following up on my post. That's rarely a good thing :-(

> Thanks for that, my asyncio tutorial was obviously seriously
> one sided, it made no mention of coroutines but did promise
> threading. 

I went back and looked again, it did mention coroutines, I was obviously
just too excited about asyncio using callbacks to notice!

Ooops.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Wed Feb 17 12:43:59 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 17 Feb 2016 17:43:59 +0000
Subject: [Tutor] invalid syntax error in Run Run Module
In-Reply-To: <CAF91wy8=mujSJsPA0sbENNFcRON7tv7gUUp=OVffUUg6H=FkEg@mail.gmail.com>
References: <CAF91wy8=mujSJsPA0sbENNFcRON7tv7gUUp=OVffUUg6H=FkEg@mail.gmail.com>
Message-ID: <na2bgv$q8o$1@ger.gmane.org>

On 17/02/16 15:45, Lisa Hasler Waters wrote:

> -When we try to select Run Run Module - we get the error message "invalid
> syntax," which points to the number 5 in Python 3.5.1

OK, that probably means you are trying to run the Python shell
window which is the wrong thing to do.

Run module is for when you create a program in a new edit
window as a separate file.

Try this set of steps

1) open IDLE
a python shell window appears (with >>> prompt)
2) File->New File
3) Select python script, hit OK - this step is not always needed
- a new editor window opens (no >>> prompt)
4) Type

print("Hello world")

in the editor.

5) File->SaveAs and choose a name - hello.py, say
6) Run->Run Module

You should see Hello world appear in the shell window
with no errors.

The shell window is for interactively trying out code
snippets. The editor window(s) are for creating Python
programs that can be run directly from the OS command
line (or as icons on your desktop/finder).

HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From dyoo at hashcollision.org  Thu Feb 18 03:14:37 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Thu, 18 Feb 2016 00:14:37 -0800
Subject: [Tutor] really basic - finding multiline chunk within larger
 chunk
In-Reply-To: <CAP16ngoDdev840iyx-c2qQy8aYd7BY+O_3Hx+otsHXA8Zpcmnw@mail.gmail.com>
References: <CAP16ngrdj_6f=M4wEc8bzWP217ehsSBO+1qiJ2z_MKfzpj6WKw@mail.gmail.com>
 <CAGZAPF7fLYxE3T381FEHbpFGR3TvDkGYws9mqwDCvTW5-jhqAQ@mail.gmail.com>
 <CAGZAPF5kwaDdtnH=C2jyJc5Vz3WvVMDGfEZ_5JiYf53cNTvJcw@mail.gmail.com>
 <CAP16ngoDdev840iyx-c2qQy8aYd7BY+O_3Hx+otsHXA8Zpcmnw@mail.gmail.com>
Message-ID: <CAGZAPF45dmUZAcDDUQw-gTzydiMWL2pJZmtMhKRgTic295eYyg@mail.gmail.com>

On Wed, Feb 17, 2016 at 7:01 AM, bruce <badouglas at gmail.com> wrote:
> I might just wind up stripping tabs/spaces and joining on space to pre
> massage the content prior to handling it..

Hi Bruce,

I want to give a second recommendation to BeautifulSoup:

    http://www.crummy.com/software/BeautifulSoup/


In particular, it knows how to search HTML:

    http://www.crummy.com/software/BeautifulSoup/bs4/doc/#searching-the-tree

From lwaters at flinthill.org  Thu Feb 18 09:10:20 2016
From: lwaters at flinthill.org (Lisa Hasler Waters)
Date: Thu, 18 Feb 2016 09:10:20 -0500
Subject: [Tutor] invalid syntax error in Run Run Module
In-Reply-To: <na2bgv$q8o$1@ger.gmane.org>
References: <CAF91wy8=mujSJsPA0sbENNFcRON7tv7gUUp=OVffUUg6H=FkEg@mail.gmail.com>
 <na2bgv$q8o$1@ger.gmane.org>
Message-ID: <CAF91wy9JiTr5ERQLbSx0xvDqpQkR2KyBvDVgLX9MRSQ8nKfZmA@mail.gmail.com>

Alan,

Thank you so very much! This worked perfectly!

Now, if you don't mind and you have the time, can you please elaborate on
when we (we being Python novices) would use the *Shell* vs when it is best
to use the *Editor*?

Many, many, many thanks!

Lisa



On Wed, Feb 17, 2016 at 12:43 PM, Alan Gauld <alan.gauld at btinternet.com>
wrote:

> On 17/02/16 15:45, Lisa Hasler Waters wrote:
>
> > -When we try to select Run Run Module - we get the error message "invalid
> > syntax," which points to the number 5 in Python 3.5.1
>
> OK, that probably means you are trying to run the Python shell
> window which is the wrong thing to do.
>
> Run module is for when you create a program in a new edit
> window as a separate file.
>
> Try this set of steps
>
> 1) open IDLE
> a python shell window appears (with >>> prompt)
> 2) File->New File
> 3) Select python script, hit OK - this step is not always needed
> - a new editor window opens (no >>> prompt)
> 4) Type
>
> print("Hello world")
>
> in the editor.
>
> 5) File->SaveAs and choose a name - hello.py, say
> 6) Run->Run Module
>
> You should see Hello world appear in the shell window
> with no errors.
>
> The shell window is for interactively trying out code
> snippets. The editor window(s) are for creating Python
> programs that can be run directly from the OS command
> line (or as icons on your desktop/finder).
>
> HTH
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Lisa Waters, PhD
Technology Integration
Flint Hill School

From alan.gauld at btinternet.com  Thu Feb 18 13:16:04 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 18 Feb 2016 18:16:04 +0000
Subject: [Tutor] invalid syntax error in Run Run Module
In-Reply-To: <CAF91wy9JiTr5ERQLbSx0xvDqpQkR2KyBvDVgLX9MRSQ8nKfZmA@mail.gmail.com>
References: <CAF91wy8=mujSJsPA0sbENNFcRON7tv7gUUp=OVffUUg6H=FkEg@mail.gmail.com>
 <na2bgv$q8o$1@ger.gmane.org>
 <CAF91wy9JiTr5ERQLbSx0xvDqpQkR2KyBvDVgLX9MRSQ8nKfZmA@mail.gmail.com>
Message-ID: <56C60A64.7060606@btinternet.com>

On 18/02/16 14:10, Lisa Hasler Waters wrote:
> Alan,
>
> Thank you so very much! This worked perfectly! 
>
> Now, if you don't mind and you have the time, can you please elaborate
> on when we (we being Python novices) would use the /Shell/ vs when it
> is best to use the /Editor/? 
>

You need to use the editor any time you want to create a file that you
can run
outside of IDLE. Especially for anything more than a dozen lines long - its
much easier to edit in the editor than to try to recall all the lines
you typed
in the shell.

The >>> prompt is for experimenting, trying out ideas. Running the help()
command to read documentation etc. Anything interactive or temporary..

I often find when I'm working on a project that I'll be entering my code in
the editor but then want to find out how some module or function works.
I then switch to the shell, try it out interactively and once I'm
comfortable
switch back to the editor to create the final code.

Another thing the shell is good for is informal testing of modules. You can
import them at the >>> prompt and then call the various functions/classes
interactively.

Of course eventually you will want to learn about how to formally test code
too but for beginners the shell is often the best and most instructive and
fun  way to start with testing.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


From lwaters at flinthill.org  Thu Feb 18 13:28:45 2016
From: lwaters at flinthill.org (Lisa Hasler Waters)
Date: Thu, 18 Feb 2016 13:28:45 -0500
Subject: [Tutor] invalid syntax error in Run Run Module
In-Reply-To: <56C60A64.7060606@btinternet.com>
References: <CAF91wy8=mujSJsPA0sbENNFcRON7tv7gUUp=OVffUUg6H=FkEg@mail.gmail.com>
 <na2bgv$q8o$1@ger.gmane.org>
 <CAF91wy9JiTr5ERQLbSx0xvDqpQkR2KyBvDVgLX9MRSQ8nKfZmA@mail.gmail.com>
 <56C60A64.7060606@btinternet.com>
Message-ID: <CAF91wy9Kv6nVb2xRpNV=HZaP1EkD1sL53V7hTBnTSF+vWYZt_g@mail.gmail.com>

Brilliant! Thanks so much. Makes perfect sense. Onward and upward then!

On Thu, Feb 18, 2016 at 1:16 PM, Alan Gauld <alan.gauld at btinternet.com>
wrote:

> On 18/02/16 14:10, Lisa Hasler Waters wrote:
> > Alan,
> >
> > Thank you so very much! This worked perfectly!
> >
> > Now, if you don't mind and you have the time, can you please elaborate
> > on when we (we being Python novices) would use the /Shell/ vs when it
> > is best to use the /Editor/?
> >
>
> You need to use the editor any time you want to create a file that you
> can run
> outside of IDLE. Especially for anything more than a dozen lines long - its
> much easier to edit in the editor than to try to recall all the lines
> you typed
> in the shell.
>
> The >>> prompt is for experimenting, trying out ideas. Running the help()
> command to read documentation etc. Anything interactive or temporary..
>
> I often find when I'm working on a project that I'll be entering my code in
> the editor but then want to find out how some module or function works.
> I then switch to the shell, try it out interactively and once I'm
> comfortable
> switch back to the editor to create the final code.
>
> Another thing the shell is good for is informal testing of modules. You can
> import them at the >>> prompt and then call the various functions/classes
> interactively.
>
> Of course eventually you will want to learn about how to formally test code
> too but for beginners the shell is often the best and most instructive and
> fun  way to start with testing.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>


-- 
Lisa Waters, PhD
Technology Integration
Flint Hill School

From magyar1886 at gmail.com  Thu Feb 18 16:17:58 2016
From: magyar1886 at gmail.com (Marco Soldavini)
Date: Thu, 18 Feb 2016 22:17:58 +0100
Subject: [Tutor] Hello! Questions
In-Reply-To: <na1h54$q5f$1@ger.gmane.org>
References: <CAO6YOj5-hcjJSUZhgAQHex-8qyV7tjH9YZ5F1xfJi7weissXqw@mail.gmail.com>
 <na1h54$q5f$1@ger.gmane.org>
Message-ID: <CAO6YOj5SMqM38C07feX_iypWGpsB-opCikt15JX_N=jPLQQvEA@mail.gmail.com>

On Wed, Feb 17, 2016 at 11:13 AM, Alan Gauld <alan.gauld at btinternet.com>
 wrote:


> > My first question is about data types, data structures in general and how
> > to organize an efficient loop for recording data.
>
> > while (stop condition false)
> >    read data
> >    write data into local array or something
> >    wait sample time
>
> That's a good start. What is the question?
> You say you know how to read data fro openopc, so the
> first loop line should be fine.
> Storing into a list involves the append method:
>
> myDataList.append(myData)
>
> and the wait will probably be openopc specific
> but if not you can use the time.sleep() function.
>
>
> hth
> --
> Alan G



Ok here my main loop for gathering data (stripped off some code to make it
easier to read):


*# While loop - scanning and storing OPC values at scan rate **while *(abort
== 0):


*# ESC pressed?     **if *msvcrt.kbhit() *and *ord(msvcrt.getch()) == 27:
        abort = 1


*break     *
*# Server up     **if *opc.ping():


        *if *opc[*'.run_batch'*] == True *and *rec_started == False:

*# Setting arrays for variables             *bool1 = []
            ana1 = []
            ana2 = []
            ana3 = []
            ana4 = []
            rec_started = True

        *if *opc[*'.run_batch'*] == True *and *rec_started == True:

*# scan time             *time2 = time.time()
            dtime = time2 - time1

            *if *dtime > 2 *and *comm_alarm == False:
                dt = datetime.datetime.now()
                bool1.append((opc.read(*'.watchdog'*)[0],opc.read(
*'.watchdog'*)[1],dt))
                ana1.append((opc.read(*'.analog2'*)[0],opc.read(*'.analog2'*
)[1],dt))
                time1 = time2


    *else*:

*# scan time         *time2 = time.time()
        dtime = time2 - time1
        *if *dtime > 2:
            *print **"ERROR: OPC Server is down"*



As you can see I am using append to store data from opc.read command which
returns elements of string array.

Let's say I can have 30-40 variables (so 30-40 append instructions at every
cycle, with same scan rate).

Is this scalable when variables are getting bigger. What if this program
shall run for 2 hours and gather let's say 2000 elements in 40 arrays,
could this be a problem in term of computation?


Second question is I want the arguments in the opc.read command not to be
hard coded but coming from a configuration files.


You see the names of my arrays? bool1, ana1, ana2, etc... Is it possible to
derive number and names of these variables from an external files.

Let's say in this configuration file I can say I have to read 10 arrays or
20 arrays and then my program adjust the while cycle consequently.


Maybe an array of array where the second dimension is coming from the
config file.


Is this clear?


Version of python is 2.7.9 running on Windows 7.


Cheers,

Marco

From sarantisx at hotmail.com  Thu Feb 18 18:25:16 2016
From: sarantisx at hotmail.com (Xristos Sarantis)
Date: Fri, 19 Feb 2016 01:25:16 +0200
Subject: [Tutor] how to import python script in the django project
Message-ID: <DUB124-W463ADF80CEC09D7AE38ADBB6AF0@phx.gbl>

hello

newbie:
i want to import python script(with calculator functions) to django app who is the better method to do that and how to do that step by step?
i want to create a web framework where the users give values take somes results from the python scripts
1.first
import the script in folder app like this ?

blog
-views.py
-urls.py
-models.py
-my_script.py
-admin.py

and with some way to connect with app ?

2.second import python script into to html form ?

3.thrird using form to take inputs ?

4.convert(request,response) my python script into to views.py ?(i think so is the better but?my problem is how to connect 'input' and 'print' from my python script on the django i dont know how to do this)

5 or with some addons ?

my code fron script:
total_inputs= int(input("number of numbers to input: "))
   for i in range (total_inputs)
    Num = input("Enter a number: ")
    NList.append(int(Num))
    for i in NList:
     if i == 1:
         y=1
     elif (i>1)and(i<5):
        y=(i-1)+2
     .............
     a.append(y) 
    print(a) 		 	   		  

From alan.gauld at btinternet.com  Thu Feb 18 20:16:42 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 19 Feb 2016 01:16:42 +0000
Subject: [Tutor] Hello! Questions
In-Reply-To: <CAO6YOj5SMqM38C07feX_iypWGpsB-opCikt15JX_N=jPLQQvEA@mail.gmail.com>
References: <CAO6YOj5-hcjJSUZhgAQHex-8qyV7tjH9YZ5F1xfJi7weissXqw@mail.gmail.com>
 <na1h54$q5f$1@ger.gmane.org>
 <CAO6YOj5SMqM38C07feX_iypWGpsB-opCikt15JX_N=jPLQQvEA@mail.gmail.com>
Message-ID: <na5qdq$gs1$1@ger.gmane.org>

On 18/02/16 21:17, Marco Soldavini wrote:

> *# While loop - scanning and storing OPC values at scan rate 

**while *(abort == 0):
> 
> 
> *# ESC pressed?     **if *msvcrt.kbhit() *and *ord(msvcrt.getch()) == 27:
>         abort = 1
> 
> 

As you can see your code is all messed up.
You need to post in plain text format.

> Let's say I can have 30-40 variables (so 30-40 append instructions at every
> cycle, with same scan rate).

Its not clear what the scan rate actually is.
How many scans per second?

> Is this scalable when variables are getting bigger. 

The variables stay the same size it's the data that
gets bigger. But how scalable it is depends on how
much memory you have. And how fast your storage
devices are. And what else is running on your
system at the time.

> shall run for 2 hours and gather let's say 2000 elements in 40 arrays,
> could this be a problem in term of computation?

Yes it could, but it depends on what size the data is.
You need to do some basic math to calculate it out.

40 * 2000 = 80k items. If they are integers then its
4 bytes per item so 320Kbytes. Not too bad. If they
are 100 character strings then its into MB but on a
modern PC still not too bad. But if it's intended to
run in an embedded controller with only 1M of RAM
it might be a big issue.

> Second question is I want the arguments in the opc.read command not to be
> hard coded but coming from a configuration files.

OK. You might want to think about what that file
format would look like.

> You see the names of my arrays? bool1, ana1, ana2, etc... Is it possible to
> derive number and names of these variables from an external files.

It is but its usually a bad idea.
Better is to use a dictionary with your "variables"
as keys and your arrays as values. So your append
looks like

data = dict()
keyName = readFromFile()
value = readFromFile()

data[keyName].append(value)

> Let's say in this configuration file I can say I have to read 10 arrays or
> 20 arrays and then my program adjust the while cycle consequently.

Yes that's doable.

> Maybe an array of array where the second dimension is coming from the
> config file.

I'd use the dictionary approach rather than arrays of arrays.
(Which are probably lists of lists in Python.)


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Thu Feb 18 20:21:13 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 19 Feb 2016 01:21:13 +0000
Subject: [Tutor] how to import python script in the django project
In-Reply-To: <DUB124-W463ADF80CEC09D7AE38ADBB6AF0@phx.gbl>
References: <DUB124-W463ADF80CEC09D7AE38ADBB6AF0@phx.gbl>
Message-ID: <na5qm9$ke6$1@ger.gmane.org>

On 18/02/16 23:25, Xristos Sarantis wrote:
> hello

Hi.

> newbie:
> i want to import python script(with calculator functions) to django app 

This list is really for questions about the python language
and standard library so Django is a wee bit off topic.
However there are some Django users on the list so you
might get an answer.

But you should probably point future Django questions at
the Django support community. It is pretty active so
you shouldn't wait long for an answer.

https://www.djangoproject.com/community/

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From magyar1886 at gmail.com  Fri Feb 19 02:51:30 2016
From: magyar1886 at gmail.com (Marco Soldavini)
Date: Fri, 19 Feb 2016 08:51:30 +0100
Subject: [Tutor] Hello! Questions
In-Reply-To: <na5qdq$gs1$1@ger.gmane.org>
References: <CAO6YOj5-hcjJSUZhgAQHex-8qyV7tjH9YZ5F1xfJi7weissXqw@mail.gmail.com>
 <na1h54$q5f$1@ger.gmane.org>
 <CAO6YOj5SMqM38C07feX_iypWGpsB-opCikt15JX_N=jPLQQvEA@mail.gmail.com>
 <na5qdq$gs1$1@ger.gmane.org>
Message-ID: <etPan.56c6c982.440cac73.1923@MacBook-Pro-di-Marco.local>

Sorry, Here my code in plaintext

#While loop - scanning and storing OPC values at scan rate
while (abort == 0):

? ? # ESC pressed?
? ? if msvcrt.kbhit() and ord(msvcrt.getch()) == 27:
? ? ? ? abort = 1
? ? ? ? break

? ? # Server up
? ? if opc.ping():


? ? ? ? if opc['.run_batch'] == True and rec_started == False:
? ? ? ? ? ? # Setting arrays for variables
? ? ? ? ? ? bool1 = []
? ? ? ? ? ? ana1 = []
? ? ? ? ? ? ana2 = []
? ? ? ? ? ? ana3 = []
? ? ? ? ? ? ana4 = []
? ? ? ? ? ? rec_started = True

? ? ? ? if opc['.run_batch'] == True and rec_started == True:
? ? ? ? ? ? # scan time
? ? ? ? ? ? time2 = time.time()
? ? ? ? ? ? dtime = time2 - time1

? ? ? ? ? ? if dtime > 2 and comm_alarm == False:
? ? ? ? ? ? ? ? dt = datetime.datetime.now()
? ? ? ? ? ? ? ? bool1.append((opc.read('.watchdog')[0],opc.read('.watchdog')[1],dt))
? ? ? ? ? ? ? ? ana1.append((opc.read('.analog2')[0],opc.read('.analog2')[1],dt))
? ? ? ? ? ? ? ? time1 = time2
? ? ? ? ? ? ? ??

? ? else:
? ? ? ? # scan time
? ? ? ? time2 = time.time()
? ? ? ? dtime = time2 - time1
? ? ? ? if dtime > 2:
? ? ? ? ? ? print "ERROR: OPC Server is down?



Il giorno?19 febbraio 2016?@?02:18:05, Alan Gauld (alan.gauld at btinternet.com) ha scritto:
> Let's say I can have 30-40 variables (so 30-40 append instructions at every?
> cycle, with same scan rate).?

Its not clear what the scan rate actually is.?
How many scans per second??


yes scan per second. For example a new fetch every 2 second, so a new append to array every 5 second



> shall run for 2 hours and gather let's say 2000 elements in 40 arrays,?
> could this be a problem in term of computation??

Yes it could, but it depends on what size the data is.?
You need to do some basic math to calculate it out.?

40 * 2000 = 80k items. If they are integers then its?
4 bytes per item so 320Kbytes. Not too bad. If they?
are 100 character strings then its into MB but on a?
modern PC still not too bad. But if it's intended to?
run in an embedded controller with only 1M of RAM?
it might be a big issue.?


Pc for sure, I think with 8GB RAM. I?ll do some detailed calculations about that.

> Second question is I want the arguments in the opc.read command not to be?
> hard coded but coming from a configuration files.?

OK. You might want to think about what that file?
format would look like.?


I?d like to have an xml file or csv file to parse (another topic I wanna learn) and in this file I have a table defining my variables with a name and another field for description


> You see the names of my arrays? bool1, ana1, ana2, etc... Is it possible to?
> derive number and names of these variables from an external files.?

It is but its usually a bad idea.?
Better is to use a dictionary with your "variables"?
as keys and your arrays as values. So your append?
looks like?

data = dict()?
keyName = readFromFile()?
value = readFromFile()?

data[keyName].append(value)?
Ok so I will look more into dictionaries, it is like hash tables?





> Let's say in this configuration file I can say I have to read 10 arrays or?
> 20 arrays and then my program adjust the while cycle consequently.?

Yes that's doable.?

> Maybe an array of array where the second dimension is coming from the?
> config file.?

I'd use the dictionary approach rather than arrays of arrays.?
(Which are probably lists of lists in Python.)?


Thanks!
Marco

From alan.gauld at btinternet.com  Fri Feb 19 04:13:03 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 19 Feb 2016 09:13:03 +0000
Subject: [Tutor] Hello! Questions
In-Reply-To: <etPan.56c6c982.440cac73.1923@MacBook-Pro-di-Marco.local>
References: <CAO6YOj5-hcjJSUZhgAQHex-8qyV7tjH9YZ5F1xfJi7weissXqw@mail.gmail.com>
 <na1h54$q5f$1@ger.gmane.org>
 <CAO6YOj5SMqM38C07feX_iypWGpsB-opCikt15JX_N=jPLQQvEA@mail.gmail.com>
 <na5qdq$gs1$1@ger.gmane.org>
 <etPan.56c6c982.440cac73.1923@MacBook-Pro-di-Marco.local>
Message-ID: <na6mav$k9v$1@ger.gmane.org>

On 19/02/16 07:51, Marco Soldavini wrote:
> Sorry, Here my code in plaintext
> ...

thanks

>> Better is to use a dictionary with your "variables" 

>> data[keyName].append(value) 
> Ok so I will look more into dictionaries, it is like hash tables?

Exactly.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From __peter__ at web.de  Fri Feb 19 09:32:09 2016
From: __peter__ at web.de (Peter Otten)
Date: Fri, 19 Feb 2016 15:32:09 +0100
Subject: [Tutor] Hello! Questions
References: <CAO6YOj5-hcjJSUZhgAQHex-8qyV7tjH9YZ5F1xfJi7weissXqw@mail.gmail.com>
 <na1h54$q5f$1@ger.gmane.org>
 <CAO6YOj5SMqM38C07feX_iypWGpsB-opCikt15JX_N=jPLQQvEA@mail.gmail.com>
 <na5qdq$gs1$1@ger.gmane.org>
 <etPan.56c6c982.440cac73.1923@MacBook-Pro-di-Marco.local>
Message-ID: <na791c$d3f$1@ger.gmane.org>

Marco Soldavini wrote:

Random remarks about your code:

> #While loop - scanning and storing OPC values at scan rate
> while (abort == 0):

The loop continues to work if you change it to

while True:
 
>     # ESC pressed?
>     if msvcrt.kbhit() and ord(msvcrt.getch()) == 27:
>         abort = 1
>         break
> 
>     # Server up
>     if opc.ping():
> 
> 
>         if opc['.run_batch'] == True and rec_started == False:

In idiomatic Python you don't compare boolean values, you test:

          if opc['.run_batch'] and not rec_started:

>             # Setting arrays for variables
>             bool1 = []
>             ana1 = []
>             ana2 = []
>             ana3 = []
>             ana4 = []
>             rec_started = True
> 
>         if opc['.run_batch'] == True and rec_started == True:
>             # scan time
>             time2 = time.time()
>             dtime = time2 - time1
> 
>             if dtime > 2 and comm_alarm == False:
>                 dt = datetime.datetime.now()
>                 bool1.append((opc.read('.watchdog')
[0],opc.read('.watchdog')[1],dt))

Invoking opc.read(".watchdog") twice looks wrong, the two values might be 
out of sync. I suspect that you want

                  result = opc.read(".watchdog")
                  bool1.append((result[0], result[1], dt))

Also, after reading http://openopc.sourceforge.net/api.html I wonder if it 
wouldn't be better to go with the timestamp provided by the server

                  bool1.append(opc.read(".watchdog"))

With a slight modification of Alan's suggestion you could write to a list of 
dicts instead of a dict of lists like so:

# outside the loop
WANTED = [".watchdog", ".analog1", ".analog2]
data = []

# in the loop:
                 data.append({r[0]:r[1:] for r in opc.read(WANTED)})

>                 ana1.append((opc.read('.analog2')[0],opc.read('.analog2')
[1],dt))
>                 time1 = time2
>                 
> 
>     else:
>         # scan time
>         time2 = time.time()
>         dtime = time2 - time1
>         if dtime > 2:
>             print "ERROR: OPC Server is down?
> 



From magyar1886 at gmail.com  Fri Feb 19 14:41:31 2016
From: magyar1886 at gmail.com (Marco Soldavini)
Date: Fri, 19 Feb 2016 20:41:31 +0100
Subject: [Tutor] Hello! Questions
In-Reply-To: <na791c$d3f$1@ger.gmane.org>
References: <CAO6YOj5-hcjJSUZhgAQHex-8qyV7tjH9YZ5F1xfJi7weissXqw@mail.gmail.com>
 <na1h54$q5f$1@ger.gmane.org>
 <CAO6YOj5SMqM38C07feX_iypWGpsB-opCikt15JX_N=jPLQQvEA@mail.gmail.com>
 <na5qdq$gs1$1@ger.gmane.org>
 <etPan.56c6c982.440cac73.1923@MacBook-Pro-di-Marco.local>
 <na791c$d3f$1@ger.gmane.org>
Message-ID: <CAO6YOj4eCUqsjrsg-to+zjur7cuAviSWZpXRwzvaS7RfBP9jFA@mail.gmail.com>

On Fri, Feb 19, 2016 at 3:32 PM, Peter Otten <__peter__ at web.de> wrote:

>
>
> Also, after reading http://openopc.sourceforge.net/api.html I wonder if it
> wouldn't be better to go with the timestamp provided by the server
>
>                   bool1.append(opc.read(".watchdog"))
>

yes but my next step is to load some of this data up to SQL and the
timestamp provided by the opc was difficult to manage into a TIME field. I
manage to insert the appended data instead with the timestamp generated
with my code.

I'll work more on that to see if it is possible to get the original
timestamp. I don't require high precision just something around the second
is ok.


>
> With a slight modification of Alan's suggestion you could write to a list
> of
> dicts instead of a dict of lists like so:
>
> # outside the loop
> WANTED = [".watchdog", ".analog1", ".analog2]
> data = []
>
> # in the loop:
>                  data.append({r[0]:r[1:] for r in opc.read(WANTED)})
>
>
>
Thanks for the helpful hints. I have a ton of questions yet to come!

Marco

From paul at whoosh.cn  Sat Feb 20 04:15:58 2016
From: paul at whoosh.cn (Paul Z)
Date: Sat, 20 Feb 2016 17:15:58 +0800
Subject: [Tutor] a beginning question
Message-ID: <SNT407-EAS190744442C3E245ED0C4B3CD5A10@phx.gbl>

Hi all,

I receive the string messages from my Mobile via UDP as below:
A, 0.1, 0.6, 0.7
B, 0.2, 0.3, 0.8

I want to arrange them to two array as below:

a = (0.1, 0.6, 0.7)
b = (0.2, 0.3, 0.8)

all the number are float.

Thanks!

Paul Z


From alan.gauld at btinternet.com  Sat Feb 20 05:33:51 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 20 Feb 2016 10:33:51 +0000
Subject: [Tutor] a beginning question
In-Reply-To: <SNT407-EAS190744442C3E245ED0C4B3CD5A10@phx.gbl>
References: <SNT407-EAS190744442C3E245ED0C4B3CD5A10@phx.gbl>
Message-ID: <na9fee$aev$1@ger.gmane.org>

On 20/02/16 09:15, Paul Z wrote:
> Hi all,
> 
> I receive the string messages from my Mobile via UDP as below:
> A, 0.1, 0.6, 0.7
> B, 0.2, 0.3, 0.8

I think we need a bit more detail.
How do you receive these messages? UDP requires some
kind of receiving service, so what is it that is
receiving these messages? Is it your python program?
Or is it some other bit of software?

Having received these strings what happens?
Are they stored in a variable? In a file? where?

> I want to arrange them to two array as below:
> 
> a = (0.1, 0.6, 0.7)
> b = (0.2, 0.3, 0.8)

What you've shown would be tuples in Python, not
arrays. We need to be precise about data types.
Do you want the data in a list, a tuple, an array?
Do you understand the differences?

Now, to try to answer your question based on the
most optimistic set of assumptions:
ie. You are receiving the messages in your python code
    and storing them as string variables called s1
    and s2 and you want the result to be a list of floats.

a = [float(n) for n in s1.split(',')]
b = [float(n) for n in s2.split(',')]

But I suspect that I'm being overly optimistic...
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From paul at whoosh.cn  Sat Feb 20 08:23:33 2016
From: paul at whoosh.cn (Paul Z)
Date: Sat, 20 Feb 2016 21:23:33 +0800
Subject: [Tutor] a beginning question
In-Reply-To: <SNT148-W50F68C969B78765613E1BDD5A10@phx.gbl>
References: <SNT407-EAS190744442C3E245ED0C4B3CD5A10@phx.gbl>,
 <na9fee$aev$1@ger.gmane.org>, <SNT148-W50F68C969B78765613E1BDD5A10@phx.gbl>
Message-ID: <SNT148-W46EF2F9783ABD3265DD8FDD5A10@phx.gbl>

Hi,

I writed some codes as the UDP messages:

import socket 
import random
from array import *

port = 8088
host = "localhost"

s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)

num = array('f')
x = 0
while x < 6:
??? num.append(random.random())
??? x += 1

a0 = str("%.2f"%num[0]) + ','
a1 = str("%.2f"%num[1]) + ','
a2 = str("%.2f"%num[2]) + ','
a3 = str("%.2f"%num[3]) + ','
a4 = str("%.2f"%num[4]) + ','
a5 = str("%.2f"%num[5]) + ','

msg1 = 'a,' + a0 + a1 + a2
msg1 = bytes(msg1, 'utf-8')
msg2 = 'b,' + a3 + a4 + a5
msg2 = bytes(msg2, 'utf-8')

s.sendto(msg1, (host, port))
s.sendto(msg2, (host, port))

and I receive the messages via:

import socket 
port = 8088
s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM) 
s.bind(("",port)) 
print('waiting on port:',port)
while True: 
? data,addr = s.recvfrom(1024) 
? print('reciveed:', data, "from", addr)

I want to arrange the messages to:

array1 = #the numbers which is start as 'a' in the messages 
array2 = #the numbers which is start as b in the messages

Thanks!
Paul Z 

?

----------------------------------------
> From: paul at whoosh.cn
> To: alan.gauld at btinternet.com
> CC: tutor at python.org
> Subject: RE: [Tutor] a beginning question
> Date: Sat, 20 Feb 2016 19:51:16 +0800
>
> Hi Alan,
>
> Thanks for your reply,
>
> My friend help me to write a andriod app which send UDP messages.
> And I writed some codes by using Python to receive the messages as below:
>
> import socket
> port = 8081
> s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
> s.bind(("",port))
> print('waiting on port:',port)
> while True:
>   data,addr = s.recvfrom(1024)
>   print('reciveed:', data, "from", addr)
>
> and I plan to re-pack them, and send them to an other computer and port.
> I think my code should judge the messages which is "a..." or "b..." first, then arrange them as below:
>
> a = (0.1, 0.6, 0.7)
> b = (0.2, 0.3, 0.8)
>
> the numbers in the example is variable.
> and The messages I recived from andriod app are some strings which is as the head 'a' or 'b', such as 'a, 0.1, 0.6, 0.7'.
>
> Thanks
> Paul Z
>
> ----------------------------------------
>> To: tutor at python.org
>> From: alan.gauld at btinternet.com
>> Date: Sat, 20 Feb 2016 10:33:51 +0000
>> Subject: Re: [Tutor] a beginning question
>>
>> On 20/02/16 09:15, Paul Z wrote:
>>> Hi all,
>>>
>>> I receive the string messages from my Mobile via UDP as below:
>>> A, 0.1, 0.6, 0.7
>>> B, 0.2, 0.3, 0.8
>>
>> I think we need a bit more detail.
>> How do you receive these messages? UDP requires some
>> kind of receiving service, so what is it that is
>> receiving these messages? Is it your python program?
>> Or is it some other bit of software?
>>
>> Having received these strings what happens?
>> Are they stored in a variable? In a file? where?
>>
>>> I want to arrange them to two array as below:
>>>
>>> a = (0.1, 0.6, 0.7)
>>> b = (0.2, 0.3, 0.8)
>>
>> What you've shown would be tuples in Python, not
>> arrays. We need to be precise about data types.
>> Do you want the data in a list, a tuple, an array?
>> Do you understand the differences?
>>
>> Now, to try to answer your question based on the
>> most optimistic set of assumptions:
>> ie. You are receiving the messages in your python code
>> and storing them as string variables called s1
>> and s2 and you want the result to be a list of floats.
>>
>> a = [float(n) for n in s1.split(',')]
>> b = [float(n) for n in s2.split(',')]
>>
>> But I suspect that I'm being overly optimistic...
>> --
>> Alan G
>> Author of the Learn to Program web site
>> http://www.alan-g.me.uk/
>> http://www.amazon.com/author/alan_gauld
>> Follow my photo-blog on Flickr at:
>> http://www.flickr.com/photos/alangauldphotos
>>
>>
>> _______________________________________________
>> Tutor maillist - Tutor at python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
 		 	   		  


From __peter__ at web.de  Sat Feb 20 11:04:15 2016
From: __peter__ at web.de (Peter Otten)
Date: Sat, 20 Feb 2016 17:04:15 +0100
Subject: [Tutor] a beginning question
References: <SNT407-EAS190744442C3E245ED0C4B3CD5A10@phx.gbl>
 <na9fee$aev$1@ger.gmane.org> <SNT148-W50F68C969B78765613E1BDD5A10@phx.gbl>
 <SNT148-W46EF2F9783ABD3265DD8FDD5A10@phx.gbl>
Message-ID: <naa2q0$amn$1@ger.gmane.org>

Paul Z wrote:

> Hi,
> 
> I writed some codes as the UDP messages:
> 
> import socket
> import random
> from array import *
> 
> port = 8088
> host = "localhost"
> 
> s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
> 
> num = array('f')
> x = 0
> while x < 6:
> num.append(random.random())
> x += 1
> 
> a0 = str("%.2f"%num[0]) + ','
> a1 = str("%.2f"%num[1]) + ','
> a2 = str("%.2f"%num[2]) + ','
> a3 = str("%.2f"%num[3]) + ','
> a4 = str("%.2f"%num[4]) + ','
> a5 = str("%.2f"%num[5]) + ','
> 
> msg1 = 'a,' + a0 + a1 + a2
> msg1 = bytes(msg1, 'utf-8')
> msg2 = 'b,' + a3 + a4 + a5
> msg2 = bytes(msg2, 'utf-8')
> 
> s.sendto(msg1, (host, port))
> s.sendto(msg2, (host, port))
> 
> and I receive the messages via:
> 
> import socket
> port = 8088
> s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
> s.bind(("",port))
> print('waiting on port:',port)
> while True:
> data,addr = s.recvfrom(1024)
> print('reciveed:', data, "from", addr)
> 
> I want to arrange the messages to:
> 
> array1 = #the numbers which is start as 'a' in the messages
> array2 = #the numbers which is start as b in the messages

You can put the lists into a dict. Then
lists["a"] accesses one and lists["b"] accesses the other list.

items1 = []
items2 = []
lists = {"a": items1, "b": items2}

while True: 
    data, addr = s.recvfrom(1024) 
    print('received:', data, "from", addr)

    parts = data.decode("utf-8").split(",")
    key = parts.pop(0)
    lists[key].extend(float(p) for p in parts if p.strip())

    print("items1:", items1)
    print("items2:", items2)

You can extend this approach to an arbitrary number of lists if you add them 
dynamically to the dict:

lists = {}
while True:
    ...
    parts = ...
    key = parts.pop(0)
    if key not in lists:
        lists[key] = []
    lists[key].extend(...)
    


From alan.gauld at btinternet.com  Sat Feb 20 11:16:56 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sat, 20 Feb 2016 16:16:56 +0000
Subject: [Tutor] a beginning question
In-Reply-To: <SNT148-W46EF2F9783ABD3265DD8FDD5A10@phx.gbl>
References: <SNT407-EAS190744442C3E245ED0C4B3CD5A10@phx.gbl>
 <na9fee$aev$1@ger.gmane.org> <SNT148-W50F68C969B78765613E1BDD5A10@phx.gbl>
 <SNT148-W46EF2F9783ABD3265DD8FDD5A10@phx.gbl>
Message-ID: <naa3ho$lvr$1@ger.gmane.org>

On 20/02/16 13:23, Paul Z wrote:
> Hi,
> 
> I writed some codes as the UDP messages:
> 
> import socket 
> import random
> from array import *
> 
> port = 8088
> host = "localhost"
> 
> s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
> 
> num = array('f')
> x = 0
> while x < 6:
>     num.append(random.random())
>     x += 1

You could shorten this to:

num = array.array('f',[random.random() for n in range(6)])

> a0 = str("%.2f"%num[0]) + ','
> a1 = str("%.2f"%num[1]) + ','
> a2 = str("%.2f"%num[2]) + ','
> a3 = str("%.2f"%num[3]) + ','
> a4 = str("%.2f"%num[4]) + ','
> a5 = str("%.2f"%num[5]) + ','
> 
> msg1 = 'a,' + a0 + a1 + a2
> msg1 = bytes(msg1, 'utf-8')
> msg2 = 'b,' + a3 + a4 + a5
> msg2 = bytes(msg2, 'utf-8')

This is a bit long winded, you could go direct to:

msg1 = bytes('a, %.2f, %.2f, %.2f' % (num[0],num[1],num[2]), 'utf-8')
msg2 = bytes('b, %.2f, %.2f, %.2f' % (num[3],num[4],num[5]), 'utf-8')

> s.sendto(msg1, (host, port))
> s.sendto(msg2, (host, port))
> 
> and I receive the messages via:
> 
> import socket 
> port = 8088
> s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM) 
> s.bind(("",port)) 
> print('waiting on port:',port)
> while True: 
>   data,addr = s.recvfrom(1024) 
>   print('reciveed:', data, "from", addr)
> 
> I want to arrange the messages to:
> 
> array1 = #the numbers which is start as 'a' in the messages 
> array2 = #the numbers which is start as b in the messages

OK, I'm not sure why you are using arrays? Do you have a
reason not to use plain lists? I'm going to assume not
because it keeps things simpler.

[If you need to create an array its easy enough once
you have the lists:
array1 = array.array('f',list1)
]

You first need to split the data by commas:

fields = data.split(',')

Then you need to check the first field for a or b

if fields[0] == 'a':
   list1 = [float(field) for field in field[1:]]
elif fields[0] == 'b':
   list2 = [float(field) for field in field[1:]]
else:
   raise ValueError 'Unknown first field in data'

Another way to do this would be to use a dictionary
based on the first field value:

received = {} # initialise dict
while True:
   data,addr = s.recvfrom(1024)
   fields = data.split(',')
   received[ fields[0] ] = [float(field) for field in field[1:]]

You can then access your data with

value = received['a'][2]  # sets value to 0.7 using your data


hth
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From woodworm at cybersmart.co.za  Sat Feb 20 11:25:03 2016
From: woodworm at cybersmart.co.za (Erol Gericke)
Date: Sat, 20 Feb 2016 18:25:03 +0200
Subject: [Tutor] How to run a python script
Message-ID: <56C8935F.2060905@cybersmart.co.za>

I want learn Python programming. Many years ago I taught myself BASIC so 
I have some programming experience.

I have  installed Python 2.7.11 and Notepad ++ on my Win 7 computer. I 
have also edited the PATH variable to include C:\Python27.

However I am unable to run any scripts.

When I type (into the COMMAND PROMPT window) python test.py or
python C:\python27\test.py      I get SyntaxError: invalid syntax

Until I solve this problem learning programming makes no sense.

Can you please help me, I would appreciate your help.

Thank you in advance,
Erol Gericke.
Somerse West. South Africa.

From kwpolska at gmail.com  Sat Feb 20 14:34:18 2016
From: kwpolska at gmail.com (Chris Warrick)
Date: Sat, 20 Feb 2016 20:34:18 +0100
Subject: [Tutor] How to run a python script
In-Reply-To: <56C8935F.2060905@cybersmart.co.za>
References: <56C8935F.2060905@cybersmart.co.za>
Message-ID: <CAMw+j7LaWw28N7RX9VCM+vn47bAku=X9S+v+hpRkTH7gSk--KQ@mail.gmail.com>

On 20 February 2016 at 17:25, Erol Gericke <woodworm at cybersmart.co.za> wrote:
> I want learn Python programming. Many years ago I taught myself BASIC so I
> have some programming experience.
>
> I have  installed Python 2.7.11 and Notepad ++ on my Win 7 computer. I have
> also edited the PATH variable to include C:\Python27.
>
> However I am unable to run any scripts.
>
> When I type (into the COMMAND PROMPT window) python test.py or
> python C:\python27\test.py      I get SyntaxError: invalid syntax
>
> Until I solve this problem learning programming makes no sense.
>
> Can you please help me, I would appreciate your help.
>
> Thank you in advance,
> Erol Gericke.
> Somerse West. South Africa.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

Please paste the full error message, and your complete source code.
Also, make sure you are running the commands in the regular command
prompt window, and not in a Python-specific one.

Also, your scripts should not live in C:\Python27, because they might
conflict with other things.

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16

From alan.gauld at btinternet.com  Sat Feb 20 19:30:49 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 21 Feb 2016 00:30:49 +0000
Subject: [Tutor] How to run a python script
In-Reply-To: <56C8935F.2060905@cybersmart.co.za>
References: <56C8935F.2060905@cybersmart.co.za>
Message-ID: <nab0fp$79g$1@ger.gmane.org>

On 20/02/16 16:25, Erol Gericke wrote:

> However I am unable to run any scripts.
> 
> When I type (into the COMMAND PROMPT window) python test.py or
> python C:\python27\test.py      I get SyntaxError: invalid syntax

As Chris has hinted it sounds like you are running the script
at the wrong command prompt.

In general the Python prompt >>> is used to enter code
snippets directly.

The OS command prompt (C:\Windows> or similar) is used
to run Python script files.

To get an OS command prompt hit WindowsKey->Run and
type CMD and click OK. (You can also create a desktop
shortcut if you prefer.)

Also, for future reference it will help if you send
(cut n paste) the full error message not just a summary.
It contains a lot of useful data that helps pinpoint
the issue, once you learn to read it.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From cperry172 at hotmail.com  Sat Feb 20 19:36:51 2016
From: cperry172 at hotmail.com (Chad Perry)
Date: Sun, 21 Feb 2016 00:36:51 +0000
Subject: [Tutor] How to run a python script
In-Reply-To: <nab0fp$79g$1@ger.gmane.org>
References: <56C8935F.2060905@cybersmart.co.za>,<nab0fp$79g$1@ger.gmane.org>
Message-ID: <SN1PR13MB0447F2AF00C22AC9106BC96EF4A20@SN1PR13MB0447.namprd13.prod.outlook.com>

What version of Python are you using? Have you tried an



apt-get -y update

yum -y update





Sincerely,

Chad Perry
cperry172 at hotmail.com
512.961.0291



From: Alan Gauld<mailto:alan.gauld at btinternet.com>
Sent: Saturday, February 20, 2016 6:32 PM
To: tutor at python.org<mailto:tutor at python.org>
Subject: Re: [Tutor] How to run a python script



On 20/02/16 16:25, Erol Gericke wrote:

> However I am unable to run any scripts.
>
> When I type (into the COMMAND PROMPT window) python test.py or
> python C:\python27\test.py      I get SyntaxError: invalid syntax

As Chris has hinted it sounds like you are running the script
at the wrong command prompt.

In general the Python prompt >>> is used to enter code
snippets directly.

The OS command prompt (C:\Windows> or similar) is used
to run Python script files.

To get an OS command prompt hit WindowsKey->Run and
type CMD and click OK. (You can also create a desktop
shortcut if you prefer.)

Also, for future reference it will help if you send
(cut n paste) the full error message not just a summary.
It contains a lot of useful data that helps pinpoint
the issue, once you learn to read it.

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist  -  Tutor at python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

From alan.gauld at btinternet.com  Sun Feb 21 03:48:22 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 21 Feb 2016 08:48:22 +0000
Subject: [Tutor] How to run a python script
In-Reply-To: <SN1PR13MB0447F2AF00C22AC9106BC96EF4A20@SN1PR13MB0447.namprd13.prod.outlook.com>
References: <56C8935F.2060905@cybersmart.co.za> <nab0fp$79g$1@ger.gmane.org>
 <SN1PR13MB0447F2AF00C22AC9106BC96EF4A20@SN1PR13MB0447.namprd13.prod.outlook.com>
Message-ID: <nabtkm$pv8$1@ger.gmane.org>

On 21/02/16 00:36, Chad Perry wrote:
> What version of Python are you using? Have you tried an
> apt-get -y update
> yum -y update


Since the OP appears to be running Windows I doubt
if that will help.

In fact, I doubt it would help with a syntax error
on Linux either.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From kwpolska at gmail.com  Sun Feb 21 04:10:47 2016
From: kwpolska at gmail.com (Chris Warrick)
Date: Sun, 21 Feb 2016 10:10:47 +0100
Subject: [Tutor] How to run a python script
In-Reply-To: <56C979E4.40500@cybersmart.co.za>
References: <56C8935F.2060905@cybersmart.co.za>
 <CAMw+j7LaWw28N7RX9VCM+vn47bAku=X9S+v+hpRkTH7gSk--KQ@mail.gmail.com>
 <56C979E4.40500@cybersmart.co.za>
Message-ID: <CAMw+j7L6m_qVbXVGCJbj2ax2vRW+AO2j_cv7RiSSn2yPiyE3Xw@mail.gmail.com>

Forwarding to mailing list, please use Reply All in the future.

On 21 February 2016 at 09:48, Erol Gericke <woodworm at cybersmart.co.za> wrote:
> Hi Chris,
>
> Thanks for your prompt reply, the problem has been solved!
> I was using the 'python' terminal not the 'DOS' terminal.
>
>  I have created a new directory to hold the *.py scripts.
>
> As I type with only two fingers I want to keep everything as short as
> possible.
> Is there a way to avoid the full path when running a script.
>
> Thanks again, you have revived my interest in Python.
>
> Regards,
> Erol Gericke
>
>
>> Please paste the full error message, and your complete source code.
>> Also, make sure you are running the commands in the regular command
>> prompt window, and not in a Python-specific one.
>>
>> Also, your scripts should not live in C:\Python27, because they might
>> conflict with other things.
>>
>

You can open command prompt in the directory of your scripts [0] or
put them in a directory that doesn?t require a lot of typing (C:\py
for example).

PS. it?s not the ?DOS? terminal, it?s the command prompt (cmd.exe).
DOS is not part of Windows NT/2000/XP and better, and rightfully so.

[0]: In Windows 8?/10, available from the File menu. Otherwise:
http://www.askvg.com/enable-open-command-window-here-option-in-context-menu-in-windows-vista/

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16

From woodworm at cybersmart.co.za  Sun Feb 21 04:20:38 2016
From: woodworm at cybersmart.co.za (Erol Gericke)
Date: Sun, 21 Feb 2016 11:20:38 +0200
Subject: [Tutor] How to run a python script
In-Reply-To: <CAMw+j7L6m_qVbXVGCJbj2ax2vRW+AO2j_cv7RiSSn2yPiyE3Xw@mail.gmail.com>
References: <56C8935F.2060905@cybersmart.co.za>
 <CAMw+j7LaWw28N7RX9VCM+vn47bAku=X9S+v+hpRkTH7gSk--KQ@mail.gmail.com>
 <56C979E4.40500@cybersmart.co.za>
 <CAMw+j7L6m_qVbXVGCJbj2ax2vRW+AO2j_cv7RiSSn2yPiyE3Xw@mail.gmail.com>
Message-ID: <56C98166.7020400@cybersmart.co.za>

Thanks again
Erol

On 2016/02/21 11:10 AM, Chris Warrick wrote:
> Forwarding to mailing list, please use Reply All in the future.
>
> On 21 February 2016 at 09:48, Erol Gericke <woodworm at cybersmart.co.za> wrote:
>> Hi Chris,
>>
>> Thanks for your prompt reply, the problem has been solved!
>> I was using the 'python' terminal not the 'DOS' terminal.
>>
>>   I have created a new directory to hold the *.py scripts.
>>
>> As I type with only two fingers I want to keep everything as short as
>> possible.
>> Is there a way to avoid the full path when running a script.
>>
>> Thanks again, you have revived my interest in Python.
>>
>> Regards,
>> Erol Gericke
>>
>>
>>> Please paste the full error message, and your complete source code.
>>> Also, make sure you are running the commands in the regular command
>>> prompt window, and not in a Python-specific one.
>>>
>>> Also, your scripts should not live in C:\Python27, because they might
>>> conflict with other things.
>>>
> You can open command prompt in the directory of your scripts [0] or
> put them in a directory that doesn?t require a lot of typing (C:\py
> for example).
>
> PS. it?s not the ?DOS? terminal, it?s the command prompt (cmd.exe).
> DOS is not part of Windows NT/2000/XP and better, and rightfully so.
>
> [0]: In Windows 8?/10, available from the File menu. Otherwise:
> http://www.askvg.com/enable-open-command-window-here-option-in-context-menu-in-windows-vista/
>


From codythetech at gmail.com  Sun Feb 21 14:32:42 2016
From: codythetech at gmail.com (Cody West)
Date: Sun, 21 Feb 2016 13:32:42 -0600
Subject: [Tutor] Finding line number by offset value.
Message-ID: <CA+Je6-X=7wGxkxw8NkFgpPFS0-r5p-nuVnoN0pBAm+HSqV_bkQ@mail.gmail.com>

Hi,

I'm using yara-python for some file scanning and I'm trying to take the
offset in the 'strings' field and turn it into a line number.

http://yara.readthedocs.org/en/latest/yarapython.html

Below is what I'm working with.

(48L, '$execution', 'eval(base64_decode')

I'm trying to take 48L, which I believe is the character number, and get
the line number from that.

I know how to count lines until I match a string, but this is a little
harder.

I've searched all over the tubes and I can't seem to find anything useful.
Help a beginner out?

Thanks for your time,
Cody W

From cperry172 at hotmail.com  Sun Feb 21 12:34:26 2016
From: cperry172 at hotmail.com (Chad Perry)
Date: Sun, 21 Feb 2016 17:34:26 +0000
Subject: [Tutor] How to run a python script
In-Reply-To: <CAMw+j7L6m_qVbXVGCJbj2ax2vRW+AO2j_cv7RiSSn2yPiyE3Xw@mail.gmail.com>
References: <56C8935F.2060905@cybersmart.co.za>
 <CAMw+j7LaWw28N7RX9VCM+vn47bAku=X9S+v+hpRkTH7gSk--KQ@mail.gmail.com>
 <56C979E4.40500@cybersmart.co.za>,
 <CAMw+j7L6m_qVbXVGCJbj2ax2vRW+AO2j_cv7RiSSn2yPiyE3Xw@mail.gmail.com>
Message-ID: <SN1PR13MB0447978CEE3B2C747DC44EB3F4A20@SN1PR13MB0447.namprd13.prod.outlook.com>

Can you share the knowledge on what you did. I love Python it's easy...to use most of the time

Sincerely,

Chad Perry
Cperry172 at hotmail.com
512.961.0291

> On Feb 21, 2016, at 3:11 AM, Chris Warrick <kwpolska at gmail.com> wrote:
> 
> Forwarding to mailing list, please use Reply All in the future.
> 
>> On 21 February 2016 at 09:48, Erol Gericke <woodworm at cybersmart.co.za> wrote:
>> Hi Chris,
>> 
>> Thanks for your prompt reply, the problem has been solved!
>> I was using the 'python' terminal not the 'DOS' terminal.
>> 
>> I have created a new directory to hold the *.py scripts.
>> 
>> As I type with only two fingers I want to keep everything as short as
>> possible.
>> Is there a way to avoid the full path when running a script.
>> 
>> Thanks again, you have revived my interest in Python.
>> 
>> Regards,
>> Erol Gericke
>> 
>> 
>>> Please paste the full error message, and your complete source code.
>>> Also, make sure you are running the commands in the regular command
>>> prompt window, and not in a Python-specific one.
>>> 
>>> Also, your scripts should not live in C:\Python27, because they might
>>> conflict with other things.
> 
> You can open command prompt in the directory of your scripts [0] or
> put them in a directory that doesn?t require a lot of typing (C:\py
> for example).
> 
> PS. it?s not the ?DOS? terminal, it?s the command prompt (cmd.exe).
> DOS is not part of Windows NT/2000/XP and better, and rightfully so.
> 
> [0]: In Windows 8?/10, available from the File menu. Otherwise:
> http://www.askvg.com/enable-open-command-window-here-option-in-context-menu-in-windows-vista/
> 
> -- 
> Chris Warrick <https://chriswarrick.com/>
> PGP: 5EAAEA16
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From cperry172 at hotmail.com  Sun Feb 21 12:35:40 2016
From: cperry172 at hotmail.com (Chad Perry)
Date: Sun, 21 Feb 2016 17:35:40 +0000
Subject: [Tutor] How to run a python script
In-Reply-To: <CAMw+j7L6m_qVbXVGCJbj2ax2vRW+AO2j_cv7RiSSn2yPiyE3Xw@mail.gmail.com>
References: <56C8935F.2060905@cybersmart.co.za>
 <CAMw+j7LaWw28N7RX9VCM+vn47bAku=X9S+v+hpRkTH7gSk--KQ@mail.gmail.com>
 <56C979E4.40500@cybersmart.co.za>,
 <CAMw+j7L6m_qVbXVGCJbj2ax2vRW+AO2j_cv7RiSSn2yPiyE3Xw@mail.gmail.com>
Message-ID: <SN1PR13MB044783B09AF852DE1E75A2F4F4A20@SN1PR13MB0447.namprd13.prod.outlook.com>

This is the shabang for Python

http://stackoverflow.com/questions/6908143/should-i-put-shebang-in-python-scripts-and-what-form-should-it-take

Sincerely,

Chad Perry
Cperry172 at hotmail.com
512.961.0291

> On Feb 21, 2016, at 3:11 AM, Chris Warrick <kwpolska at gmail.com> wrote:
> 
> Forwarding to mailing list, please use Reply All in the future.
> 
>> On 21 February 2016 at 09:48, Erol Gericke <woodworm at cybersmart.co.za> wrote:
>> Hi Chris,
>> 
>> Thanks for your prompt reply, the problem has been solved!
>> I was using the 'python' terminal not the 'DOS' terminal.
>> 
>> I have created a new directory to hold the *.py scripts.
>> 
>> As I type with only two fingers I want to keep everything as short as
>> possible.
>> Is there a way to avoid the full path when running a script.
>> 
>> Thanks again, you have revived my interest in Python.
>> 
>> Regards,
>> Erol Gericke
>> 
>> 
>>> Please paste the full error message, and your complete source code.
>>> Also, make sure you are running the commands in the regular command
>>> prompt window, and not in a Python-specific one.
>>> 
>>> Also, your scripts should not live in C:\Python27, because they might
>>> conflict with other things.
> 
> You can open command prompt in the directory of your scripts [0] or
> put them in a directory that doesn?t require a lot of typing (C:\py
> for example).
> 
> PS. it?s not the ?DOS? terminal, it?s the command prompt (cmd.exe).
> DOS is not part of Windows NT/2000/XP and better, and rightfully so.
> 
> [0]: In Windows 8?/10, available from the File menu. Otherwise:
> http://www.askvg.com/enable-open-command-window-here-option-in-context-menu-in-windows-vista/
> 
> -- 
> Chris Warrick <https://chriswarrick.com/>
> PGP: 5EAAEA16
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From street.sweeper at mailworks.org  Sun Feb 21 09:43:29 2016
From: street.sweeper at mailworks.org (street.sweeper at mailworks.org)
Date: Sun, 21 Feb 2016 09:43:29 -0500
Subject: [Tutor] library terminology and importing
Message-ID: <1456065809.4108495.527400970.7E7D498E@webmail.messagingengine.com>

Hello all,

I often use now() and strftime() from datetime, but it seems like I
can't import just those functions.  The os module allows me to import
like this:

from os.path import join,expanduser

but I get an error if I try

from datetime.datetime import now, strftime

But if I import all of os and datetime, I can use those functions by
writing the full 'path' 3 levels deep:

os.path.expanduser('~')
datetime.datetime.now()

Is there a way to import individual functions from datetime.datetime?


Also, is there proper terminology for each of the 3 sections of
os.path.expanduser('~') for example?  Such as

os - library (or module?)
path - ?
expanduser - function


Thanks!

From alan.gauld at btinternet.com  Sun Feb 21 20:41:42 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Mon, 22 Feb 2016 01:41:42 +0000
Subject: [Tutor] Finding line number by offset value.
In-Reply-To: <CA+Je6-X=7wGxkxw8NkFgpPFS0-r5p-nuVnoN0pBAm+HSqV_bkQ@mail.gmail.com>
References: <CA+Je6-X=7wGxkxw8NkFgpPFS0-r5p-nuVnoN0pBAm+HSqV_bkQ@mail.gmail.com>
Message-ID: <nadp0m$tt$1@ger.gmane.org>

On 21/02/16 19:32, Cody West wrote:
> I'm using yara-python for some file scanning and I'm trying to take the
> offset in the 'strings' field and turn it into a line number.

I know nothing about yara except that its some kind of
pattern matching engine. However...

> (48L, '$execution', 'eval(base64_decode')
> 
> I'm trying to take 48L, which I believe is the character number, and get
> the line number from that.

I'm not totally clear what you mean but, if it is that 48L
is the character count from the start of the file and you
want to know the line number then you need to count the
number of \n characters between the first and 48th
characters.

But thats depending on your line-end system of course,
there may be two characters on each EOL... It depends
on your OS/version and possibly the character encoding
too. And if it's a binary file, who knows, as there
won't really be any line endings. And, as I understand
it, yara is targeted at reading byte patterns from
binary files?

Are you sure you really need the line number?


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From ben+python at benfinney.id.au  Sun Feb 21 20:47:20 2016
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 22 Feb 2016 12:47:20 +1100
Subject: [Tutor] library terminology and importing
References: <1456065809.4108495.527400970.7E7D498E@webmail.messagingengine.com>
Message-ID: <85ziutzfbr.fsf@benfinney.id.au>

street.sweeper at mailworks.org writes:

> I get an error if I try
>
> from datetime.datetime import now, strftime

?datetime.datetime? is not a module, so you can not import objects from
it.

> But if I import all of os and datetime, I can use those functions by
> writing the full 'path' 3 levels deep:
>
> os.path.expanduser('~')
> datetime.datetime.now()

Yes. That's a good way to do it, because it makes your code explicit and
clear to read.

> Is there a way to import individual functions from datetime.datetime?

Don't try to do that. Namespaces are a honking good idea in Python, you
should not seek to avoid them.

-- 
 \         ?I may disagree with what you say, but I will defend to the |
  `\        death your right to mis-attribute this quote to Voltaire.? |
_o__)                   ?Avram Grumer, rec.arts.sf.written, 2000-05-30 |
Ben Finney


From steve at pearwood.info  Sun Feb 21 21:24:54 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 22 Feb 2016 13:24:54 +1100
Subject: [Tutor] Finding line number by offset value.
In-Reply-To: <nadp0m$tt$1@ger.gmane.org>
References: <CA+Je6-X=7wGxkxw8NkFgpPFS0-r5p-nuVnoN0pBAm+HSqV_bkQ@mail.gmail.com>
 <nadp0m$tt$1@ger.gmane.org>
Message-ID: <20160222022454.GF12028@ando.pearwood.info>

On Mon, Feb 22, 2016 at 01:41:42AM +0000, Alan Gauld wrote:
> On 21/02/16 19:32, Cody West wrote:

> > I'm trying to take 48L, which I believe is the character number, and get
> > the line number from that.
> 
> I'm not totally clear what you mean but, if it is that 48L
> is the character count from the start of the file and you
> want to know the line number then you need to count the
> number of \n characters between the first and 48th
> characters.
> 
> But thats depending on your line-end system of course,
> there may be two characters on each EOL... 

Provided your version of Python is built with "universal newline 
support", and nearly every Python is, then if you open the file in text 
mode, all end-of-lines are automatically converted to \n on reading.

If the file is small enough to read all at once, you can do this:

offset = 48
text = the_file.read(offset)
print text.count('\n')


to print a line number starting from 0.

If the file is too big to read all at once, you can do this:

# untested
running_total = 0
line_num = -1
offset = 48000000  # say
for line in the_file:
    running_total += len(line)
    line_num += 1
    if running_total >= offset:
        print line_num
        break
    

-- 
Steve

From martin at linux-ip.net  Sun Feb 21 21:40:19 2016
From: martin at linux-ip.net (Martin A. Brown)
Date: Sun, 21 Feb 2016 18:40:19 -0800
Subject: [Tutor] library terminology and importing
In-Reply-To: <1456065809.4108495.527400970.7E7D498E@webmail.messagingengine.com>
References: <1456065809.4108495.527400970.7E7D498E@webmail.messagingengine.com>
Message-ID: <alpine.LSU.2.11.1602211807220.2062@qnttre.jbaqresebt.arg>


Good morning,

I'm inverting the order of your questions, because I think the order of the
answers may help.

>But if I import all of os and datetime, I can use those functions by
>writing the full 'path' 3 levels deep:
>
>os.path.expanduser('~')
>datetime.datetime.now()

[... hold onto your hat, we'll get to datetime.datetime ...]

>from os.path import join,expanduser
>
>Also, is there proper terminology for each of the 3 sections of
>os.path.expanduser('~') for example?  Such as

Yes, there most certainly is; it's good that you are asking.  See below.

>os - library (or module?)
>path - ?
>expanduser - function

Here's how you could figure out what they are called.  Use 'type' to figure
it out:

  >>> import os
  >>> type(os)
  <class 'module'>

  >>> type(os.path)
  <class 'module'>

  >>> type(os.path.expanduser)
  <class 'function'>

Observe that the type of os.path.expanduser is function.

It is for this reason that you can importi (and use) the expanduser 
function by itself.  It can stand alone:

  >>> from os.path import expanduser
  >>> type(expanduser)
  <class 'function'>

Side note, for diagnostics, 'type' can be handy, also, for things 
like:

  >>> type('word')
  <class 'str'>
  >>> type(7)
  <class 'int'>

>I often use now() and strftime() from datetime, but it seems like I can't
>import just those functions.  The os module allows me to import like this:

Ok, so back to datetime...

  >>> type(datetime)
  <class 'module'>

This should not surpise you.  So, datetime is a module.  Good.

  >>> type(datetime.datetime)
  <class 'type'>

Oh-ho!  What is this one?  It's called 'type'?  Well, it's a Python 
class.  You can see it in the source code, if you look for the class 
definition of 'datetime' in the module 'datetime'.  I find mine in 
/usr/lib64/python3.4/datetime.py around line 1290ff.  Look for this:

  class datetime(date):
      """datetime(year, month, day[, hour[, minute[, second[, microsecond[,tzinfo]]]]])

Why am I pointing you to this?  Well, in particular, you should see the
following a few lines later (lines 1394 ff in my copy):

    @classmethod
    def now(cls, tz=None):
        "Construct a datetime from time.time() and optional time zone info."
        t = _time.time()
        return cls.fromtimestamp(t, tz)

If you wish, you can go look up the decorator @classmethod and what 
it does, but the main point I'm making here is that this is not a 
function!  It cannot be separated from the datetime class.  It is 
(in this case) an alternate constructor for a datetime object.  And, 
'type' will tell you so:

  >>> type(datetime.datetime.now)
  <class 'builtin_function_or_method'>

So, even though the name is available to you and callable, when you 
import the module datetime, you can't separate the classmethod 
called 'now()' from the datetime.datetime class.

>but I get an error if I try
>
>from datetime.datetime import now, strftime

If you are mostly interested in shortening your import statement, I have seen
people use this sort of technique:

  >>> from datetime import datetime as dt
  >>> now = dt.now()
  >>> now.strftime('%F-%T')
  '2016-02-21-18:30:37'

Good luck and enjoy,

-Martin

-- 
Martin A. Brown
http://linux-ip.net/

From __peter__ at web.de  Mon Feb 22 04:33:35 2016
From: __peter__ at web.de (Peter Otten)
Date: Mon, 22 Feb 2016 10:33:35 +0100
Subject: [Tutor] Finding line number by offset value.
References: <CA+Je6-X=7wGxkxw8NkFgpPFS0-r5p-nuVnoN0pBAm+HSqV_bkQ@mail.gmail.com>
 <nadp0m$tt$1@ger.gmane.org> <20160222022454.GF12028@ando.pearwood.info>
Message-ID: <naeklg$397$1@ger.gmane.org>

Steven D'Aprano wrote:

> On Mon, Feb 22, 2016 at 01:41:42AM +0000, Alan Gauld wrote:
>> On 21/02/16 19:32, Cody West wrote:
> 
>> > I'm trying to take 48L, which I believe is the character number, and
>> > get the line number from that.

The documentation isn't explicit, but

"""
with open('/foo/bar/my_file', 'rb') as f:
  matches = rules.match(data=f.read())
"""

suggests that the library operates on bytes, not characters.

>> I'm not totally clear what you mean but, if it is that 48L
>> is the character count from the start of the file and you
>> want to know the line number then you need to count the
>> number of \n characters between the first and 48th
>> characters.
>> 
>> But thats depending on your line-end system of course,
>> there may be two characters on each EOL...
> 
> Provided your version of Python is built with "universal newline
> support", and nearly every Python is, then if you open the file in text
> mode, all end-of-lines are automatically converted to \n on reading.

Be careful, *if* the numbers are byte offsets and you open the file in 
universal newlines mode or text mode your results will be unreliable.

> If the file is small enough to read all at once, you can do this:

> offset = 48
> text = the_file.read(offset)
> print text.count('\n')

It's the offset that matters, not the file size; the first 48 bytes of a 
terabyte file will easily fit into the memory of your Apple II ;)



From aneeque.khan at ericsson.com  Tue Feb 23 02:29:51 2016
From: aneeque.khan at ericsson.com (Aneeque Khan)
Date: Tue, 23 Feb 2016 07:29:51 +0000
Subject: [Tutor] How to source a shell script through python to set the
 environment variables.
Message-ID: <9543F11B7EBBF24CBFD5DFC01818F99013AFA8@ESESSMB105.ericsson.se>

>From the past week I am struggling to source a shell script through python script, to set the environment variables. Problem details are :-

I have a shell script named "env_test.sh", through which I set some environment variables by using command "source env_test.sh".
env_test.sh looks like this :-
                                                #!/bin/bash
                                                export OS_AK="Aneeque"
export OS_KHAN="KHAN"

echo "Please enter your Stack Password: "
read -sr OS_PWD_INPUT
export OS_PWD=$OS_PWD_INPUT

Now I want to execute the above script through python file named "ak_test.py", to set these variables and access them further in the code.
First I tried to this way :-
                                subprocess.call(". env_test.sh; env|grep OS_", shell=True, executable="/bin/bash")

                above command opens up new process and sets the environment for that process only due to this updated environment doesn`t reflect to the main process.

After this I tried to do this way :-
                                                import sys
from pipes import quote
from pprint import pprint

def update_env(script_path):
   if "--child" in sys.argv: # executed in the child environment
      pprint(dict(os.environ))
   else:
      python, script = quote(sys.executable), quote(sys.argv[0])
      os.execl("/bin/bash", "/bin/bash", "-c", "source %s; %s %s --child" % (script_path, python, script))

                                                update_env("env_test.sh")

                with this approach some variables set while others not.

Another approach used :-
def update_env2(script):
    #pipe1 = subprocess.Popen(". %s; env -0" % script, stdout=subprocess.PIPE, shell=True, executable="/bin/bash")
    pipe1 = subprocess.Popen(". %s; env" % script, stdout=subprocess.PIPE, shell=True, executable="/bin/bash")

    output = pipe1.communicate()[0]
    #env = dict((line.split("=", 1) for line in output.splitlines()))
    env = dict((line.split("=", 1) for line in output.split('\0')))

    os.environ.update(env)

                                update_env2("env_test.sh")

                this also not provides the desired output.

Regards,
Aneeque

From belcherjames21 at gmail.com  Mon Feb 22 21:35:02 2016
From: belcherjames21 at gmail.com (BelcherJ87)
Date: Mon, 22 Feb 2016 18:35:02 -0800 (PST)
Subject: [Tutor] =?utf-8?q?Creating_and_using_module_=E2=80=9Cfloat_object?=
 =?utf-8?q?_not_callable=E2=80=9D_-Python_3=2E5=2E1?=
Message-ID: <1456194902733-5184603.post@n6.nabble.com>

 


Complete novice here trying to learn. I am supposed to create a module which
I believe I have done here:
import math

def circum(x):
    return 2 * math.pi * x (format (",.3f"))

def area(x):
    return math.pi * x **2 (format (",.3f"))

I have named this module mycircle. Now I have to import this module and use
it to calculate the circumference and area of a circle. My code for this is:
import mycircle

def main():
    radius = float (input ("Please enter the radius of the circle: "))
    circumference = mycircle.circum (radius)
    area = mycircle.area (radius)
    print ("The circumference of the circle is ", format   
(mycircle.circum, ",.3f"),  sep="")
    print ("The area of the circle is ", format (mycircle.area, ",.3f"),
sep="")

main()

However, I am receiving the error:
 File "C:/Users/Jameson/Desktop/COP 1000/Chapter 5/54.py", line 26, in
<module>
main()
  File "C:/Users/Jameson/Desktop/COP 1000/Chapter 5/54.py", line 21, in main
circumference = mycircle.circum (radius)
File "C:/Users/Jameson/Desktop/COP 1000/Chapter 5\mycircle.py", line 4, in
circum
return 2 * math.pi * x (format (",.3f"))
TypeError: 'float' object is not callable

I can only assume that it is something dumb. Any help would be greatly
appreciated! I am stuck here. 
 



--
View this message in context: http://python.6.x6.nabble.com/Creating-and-using-module-float-object-not-callable-Python-3-5-1-tp5184603.html
Sent from the Python - tutor mailing list archive at Nabble.com.

From belcherjames21 at gmail.com  Mon Feb 22 22:12:27 2016
From: belcherjames21 at gmail.com (BelcherJ87)
Date: Mon, 22 Feb 2016 19:12:27 -0800 (PST)
Subject: [Tutor] 
 =?utf-8?q?Creating_and_using_module_=E2=80=9Cfloat_objec?=
 =?utf-8?q?t_not_callable=E2=80=9D_-Python_3=2E5=2E1?=
In-Reply-To: <1456194902733-5184603.post@n6.nabble.com>
References: <1456194902733-5184603.post@n6.nabble.com>
Message-ID: <1456197147410-5184607.post@n6.nabble.com>

Edit:

Solved with help from user Gill on Stack Overflow
<http://stackoverflow.com/questions/35567827/creating-and-using-module-float-object-not-callable-python-3-5-1>  



--
View this message in context: http://python.6.x6.nabble.com/Creating-and-using-module-float-object-not-callable-Python-3-5-1-tp5184603p5184607.html
Sent from the Python - tutor mailing list archive at Nabble.com.

From alan.gauld at btinternet.com  Tue Feb 23 05:46:25 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 23 Feb 2016 10:46:25 +0000
Subject: [Tutor] How to source a shell script through python to set the
 environment variables.
In-Reply-To: <9543F11B7EBBF24CBFD5DFC01818F99013AFA8@ESESSMB105.ericsson.se>
References: <9543F11B7EBBF24CBFD5DFC01818F99013AFA8@ESESSMB105.ericsson.se>
Message-ID: <nahda1$2s4$1@ger.gmane.org>

On 23/02/16 07:29, Aneeque Khan wrote:

> env_test.sh looks like this :-
> #!/bin/bash
> export OS_AK="Aneeque"
> export OS_KHAN="KHAN"
> 
> echo "Please enter your Stack Password: "
> read -sr OS_PWD_INPUT
> export OS_PWD=$OS_PWD_INPUT
> 
> Now I want to execute the above script through python file named "ak_test.py", 
> to set these variables and access them further in the code.

Can I first ask why you are trying to do it via a shell
script rather than using Python directly? Is this part
of a bigger workflow?

The problem is that virtually any method of running
a shell script will involve starting a separate process
and setting the variables in that process' environment.

So far as I can tell the only way to do what you want
would be to run your shell script first then launch
your python script from the shell script. That would
mean breaking your current python script in two, and
if you do a lot of work prior to launching the shell,
that's probably not practical.


> def update_env(script_path):
>    if "--child" in sys.argv: # executed in the child environment
>       pprint(dict(os.environ))
>    else:
>       python, script = quote(sys.executable), quote(sys.argv[0])
>       os.execl("/bin/bash", "/bin/bash", "-c", "source %s; %s %s --child" % (script_path, python, script))
> 
> with this approach some variables set while others not.

That surprises me since calling execl() overwrites the calling
process with the called process, so I would not expect you
to see anything in your calling script.

How are you testing the results?
Are you checking os.environ?
Or are you using os.getenv()?

> Another approach used :-
> def update_env2(script):
>     #pipe1 = subprocess.Popen(". %s; env -0" % script, stdout=subprocess.PIPE, shell=True, executable="/bin/bash")
>     pipe1 = subprocess.Popen(". %s; env" % script, stdout=subprocess.PIPE, shell=True, executable="/bin/bash")
> 
>     output = pipe1.communicate()[0]
>     #env = dict((line.split("=", 1) for line in output.splitlines()))
>     env = dict((line.split("=", 1) for line in output.split('\0')))
> 
>     os.environ.update(env)

While this could work you are not really reading the environment
variables you are simply reading the stdout. You could achieve
the same by parsing the script as a text file.

So can you explain why you need to load these values from a
shell script rather than setting os.environ directly?
Maybe we can solve the bigger issue.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Tue Feb 23 07:09:08 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 23 Feb 2016 12:09:08 +0000
Subject: [Tutor] Fwd: RE: How to source a shell script through python to set
 the environment variables.
In-Reply-To: <9543F11B7EBBF24CBFD5DFC01818F99013B068@ESESSMB105.ericsson.se>
References: <9543F11B7EBBF24CBFD5DFC01818F99013B068@ESESSMB105.ericsson.se>
Message-ID: <56CC4BE4.1070400@btinternet.com>

Forwarding to list.
Please always use ReplyAll (or replyList) when responding to tutor.


-------- Forwarded Message --------
Subject: 	RE: [Tutor] How to source a shell script through python to set
the environment variables.
Date: 	Tue, 23 Feb 2016 11:14:02 +0000
From: 	Aneeque Khan <aneeque.khan at ericsson.com>
To: 	Alan Gauld <alan.gauld at btinternet.com>



Please see my comments with [AK]

-----Original Message-----
From: Tutor [mailto:tutor-bounces+aneeque.khan=ericsson.com at python.org] On Behalf Of Alan Gauld
Sent: Tuesday, February 23, 2016 4:16 PM
To: tutor at python.org
Subject: Re: [Tutor] How to source a shell script through python to set the environment variables.

On 23/02/16 07:29, Aneeque Khan wrote:

> env_test.sh looks like this :-
> #!/bin/bash
> export OS_AK="Aneeque"
> export OS_KHAN="KHAN"
> 
> echo "Please enter your Stack Password: "
> read -sr OS_PWD_INPUT
> export OS_PWD=$OS_PWD_INPUT
> 
> Now I want to execute the above script through python file named 
> "ak_test.py", to set these variables and access them further in the code.

Can I first ask why you are trying to do it via a shell script rather than using Python directly? Is this part of a bigger workflow?

[AK] :- this is the part of bigger workflow, where I get the shell script as a output of some processing.
 
The problem is that virtually any method of running a shell script will involve starting a separate process and setting the variables in that process' environment.

So far as I can tell the only way to do what you want would be to run your shell script first then launch your python script from the shell script. That would mean breaking your current python script in two, and if you do a lot of work prior to launching the shell, that's probably not practical.

[AK] :- I can`t set the variables from outside the python script as our workflow requires to process no. of such shell scripts. We have to process each shell script to set the environment and work on further.
	You can assume that overall workflow have large no. of users each having its own values for certain environment variables. While performing any task for that particular user we require to set the environment first.


> def update_env(script_path):
>    if "--child" in sys.argv: # executed in the child environment
>       pprint(dict(os.environ))
>    else:
>       python, script = quote(sys.executable), quote(sys.argv[0])
>       os.execl("/bin/bash", "/bin/bash", "-c", "source %s; %s %s 
> --child" % (script_path, python, script))
> 
> with this approach some variables set while others not.

That surprises me since calling execl() overwrites the calling process with the called process, so I would not expect you to see anything in your calling script.

How are you testing the results?
Are you checking os.environ?
Or are you using os.getenv()?

[AK] :- I am checking like this :- 
	 os.getenv("OS_AK")
	os.getenv("OS_PWD")
and getting None as the result.

> Another approach used :-
> def update_env2(script):
>     #pipe1 = subprocess.Popen(". %s; env -0" % script, stdout=subprocess.PIPE, shell=True, executable="/bin/bash")
>     pipe1 = subprocess.Popen(". %s; env" % script, 
> stdout=subprocess.PIPE, shell=True, executable="/bin/bash")
> 
>     output = pipe1.communicate()[0]
>     #env = dict((line.split("=", 1) for line in output.splitlines()))
>     env = dict((line.split("=", 1) for line in output.split('\0')))
> 
>     os.environ.update(env)

While this could work you are not really reading the environment variables you are simply reading the stdout. You could achieve the same by parsing the script as a text file.

So can you explain why you need to load these values from a shell script rather than setting os.environ directly?
Maybe we can solve the bigger issue.

[AK] :- In workflow neither variable names nor their values are known in advance. We only came to know about these at run time through the generated shell scripts.
I have recently started with python, can you direct me how we can achieve this by parsing the shell as a text file.

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist  -  Tutor at python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor




From fosiul at gmail.com  Tue Feb 23 15:33:01 2016
From: fosiul at gmail.com (Fosiul Alam)
Date: Tue, 23 Feb 2016 20:33:01 +0000
Subject: [Tutor] How to calculate high value from multiple lines for
 each column
In-Reply-To: <na1hkb$32l$1@ger.gmane.org>
References: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
 <na1hkb$32l$1@ger.gmane.org>
Message-ID: <CAPBMF6yihq_9ZMNEHbWj4A8eE5Q253EffQXE-HZOnsqRe_T0ww@mail.gmail.com>

Hi Alan
Thanks for replying,
Yes, I am having some challanges .
this is the example file  :

so there are 2 dm (dm-30 and dm-31) wich 3 sample value

dm-30             1.47    36.48    2.82    0.66   270.84   148.56
240.96     0.06   44.99  18.27   6.36
dm-31             1.47    36.49    2.82    0.66   270.85   148.58
240.94     0.06   45.03  18.28   6.37
dm-30             0.00     0.00    0.00    0.00     0.00     0.00
0.00     0.00    0.00   2.00   1.00
dm-31             0.00     0.00    0.00    0.00     0.00     0.00
0.00     0.00    0.00   3.00   1.50
dm-30             0.00     0.00    0.00    0.00     0.00     0.00
0.00     0.00    0.00   0.40   0.50
dm-31             0.00     0.00    0.00    0.00     0.00     0.00
0.00     0.00    0.00   0.70   0.60

Basically I am interested with 10 and 11 ( last 2 column) with bellow 2 way

a) ignore the firsrt value for each dm  ( so line number 1 and 2 will
totally ignore)  : I am having trouble to do that
b) get the max value for clumn 10 and 11 for each dm (dm-30 and dm-31) : I
am having to do that



I went through this far : -

from operator import itemgetter



def _get_io():

        with open (DM,'r')as f:
                content=f.readlines()
        return content


s_value= dict()
u_value=dict()

if __name__ == '__main__':

        dm_data=_get_io()

        for line in dm_data:

                if fields[0] in s_value:
                        s_value[fields[0]].append(fields[10])
                else:
                        s_value[fields[0]]=[fields[10]]

                if fields[0] in u_value:
                        u_value[fields[0]].append(fields[11])
                else:
                        u_value[fields[0]]=[fields[11]]


        print s_value
        print u_value

Python output:=

{'dm-30': ['18.27', '2.00', '0.40'], 'dm-31': ['18.28', '3.00', '0.70']}
{'dm-30': ['6.36', '1.00', '0.50'], 'dm-31': ['6.37', '1.50', '0.60']}


I wanted  to combine s_value and u_value with each Max value like bellow
{'dm-30': ['2.0', '1.0'], 'dm-31': ['3.0', '1.50']}

or just get the Max value for each dm in s_value and u_value
like bellow
{'dm-30':2.00, 'dm-31': ['3.00]}
{'dm-30': [1.00], 'dm-31': [1.50]}

Any help will be really appreciable


On Wed, Feb 17, 2016 at 10:22 AM, Alan Gauld <alan.gauld at btinternet.com>
wrote:

> On 16/02/16 22:28, Fosiul Alam wrote:
> > Hi
> > I am very new to python, basically , I want to get the Maximum value for
> > each column
> >
> >                   0.000           0.000           0.000        0
> > (0.0%)           0.000           0.600
> >                   0.000           3.000           6.000        1
> > (0.0%)           0.300           0.000
> >                   3.000           0.000           0.000        0
> > (0.0%)           0.000           0.000
> >                   5.000           0.000           0.000        1
> ...
> >
> > So  maximum value for 1st column=5
> >       maximum value for 2nd column = 7
> >       maximum value for 3rd colun =6
> >        .......................
> >
> > How can I do this ?
>
> The classical way to represent a table of data is using a list of lists
>
> myData = [[],[],[],[]]  # a table with 4 columns
>
> You can then read the data line by line and insert the values into your
> lists.
>
> for line in dataSource:
>    fields = line.split()
>    myData[0].append(fields[0])
>    myData[1].append(fields[1])
>    etc
>
> Then at the end find the max() of each column
>
> for col in myData:
>     print "Max = " max(col)
>
> There are more efficient ways to do it but that's probably the simplest.
> You need to fill in quite a few blanks, such as how you read your data
> source - is it a file or what?
>
> Try it and come back if you hit problems.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Regards
Fosiul Alam

From alan.gauld at btinternet.com  Tue Feb 23 20:06:55 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Wed, 24 Feb 2016 01:06:55 +0000
Subject: [Tutor] How to calculate high value from multiple lines for
 each column
In-Reply-To: <CAPBMF6yihq_9ZMNEHbWj4A8eE5Q253EffQXE-HZOnsqRe_T0ww@mail.gmail.com>
References: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
 <na1hkb$32l$1@ger.gmane.org>
 <CAPBMF6yihq_9ZMNEHbWj4A8eE5Q253EffQXE-HZOnsqRe_T0ww@mail.gmail.com>
Message-ID: <56CD022F.9080700@btinternet.com>

On 23/02/16 20:33, Fosiul Alam wrote:
>
> so there are 2 dm (dm-30 and dm-31) wich 3 sample value
>
> dm-30             1.47    36.48    2.82    0.66   270.84   148.56  
> 240.96     0.06   44.99  18.27   6.36
> dm-31             1.47    36.49    2.82    0.66   270.85   148.58  
> 240.94     0.06   45.03  18.28   6.37
> dm-30             0.00     0.00    0.00    0.00     0.00     0.00    
> 0.00     0.00    0.00   2.00   1.00
> dm-31             0.00     0.00    0.00    0.00     0.00     0.00    
> 0.00     0.00    0.00   3.00   1.50
> dm-30             0.00     0.00    0.00    0.00     0.00     0.00    
> 0.00     0.00    0.00   0.40   0.50
> dm-31             0.00     0.00    0.00    0.00     0.00     0.00    
> 0.00     0.00    0.00   0.70   0.60
>
> Basically I am interested with 10 and 11 ( last 2 column) with bellow
> 2 way
>
> a) ignore the firsrt value for each dm  ( so line number 1 and 2 will
> totally ignore)  : I am having trouble to do that

We can do that at the end, just before getting the max().


> b) get the max value for clumn 10 and 11 for each dm (dm-30 and dm-31)
> : I am having to do that

> def _get_io():
>
>         with open (DM,'r')as f:
>                 content=f.readlines()
>         return content
>
>
> s_value= dict()
> u_value=dict()
>

You don't need two variables here, just one will do, lets just call it
values.


> if __name__ == '__main__':
>
>         dm_data=_get_io()
>        
>         for line in dm_data:
>                

You missed the line that split the fields

           fields = line.split()


>                 if fields[0] in s_value:
>                         s_value[fields[0]].append(fields[10])
>                 else:
>                         s_value[fields[0]]=[fields[10]]
>

We can tidy that up using the setdefault() method:

          data = values.setdefault(fields[0],[ [],[] ])     # return
lists if avail else empty lists
          data[0],append(float(fields[10])
          data[1],append(float(fields[11])

See if that gets you closer.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


From fosiul at gmail.com  Thu Feb 25 15:44:23 2016
From: fosiul at gmail.com (Fosiul Alam)
Date: Thu, 25 Feb 2016 20:44:23 +0000
Subject: [Tutor] How to calculate high value from multiple lines for
 each column
In-Reply-To: <56CD022F.9080700@btinternet.com>
References: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
 <na1hkb$32l$1@ger.gmane.org>
 <CAPBMF6yihq_9ZMNEHbWj4A8eE5Q253EffQXE-HZOnsqRe_T0ww@mail.gmail.com>
 <56CD022F.9080700@btinternet.com>
Message-ID: <CAPBMF6yTqynGpAi_3o4uRnPAo1g3ZaJS2yc1gQLUHQPirww1yg@mail.gmail.com>

Hi Alan
I am little bit confused about the statement,

bellow is from your previuos comments

>                 if fields[0] in s_value:
>                         s_value[fields[0]].append(fields[10])
>                 else:
>                         s_value[fields[0]]=[fields[10]]
>

We can tidy that up using the setdefault() method:

          data = values.setdefault(fields[0],[ [],[] ])     # return
lists if avail else empty lists
          data[0],append(float(fields[10])
          data[1],append(float(fields[11])

how ami I gettign the "values" ? as its not defiend anywhere, so how we
putting data into variables "values" ?

Thanks fror the help





On Wed, Feb 24, 2016 at 1:06 AM, Alan Gauld <alan.gauld at btinternet.com>
wrote:

> On 23/02/16 20:33, Fosiul Alam wrote:
> >
> > so there are 2 dm (dm-30 and dm-31) wich 3 sample value
> >
> > dm-30             1.47    36.48    2.82    0.66   270.84   148.56
> > 240.96     0.06   44.99  18.27   6.36
> > dm-31             1.47    36.49    2.82    0.66   270.85   148.58
> > 240.94     0.06   45.03  18.28   6.37
> > dm-30             0.00     0.00    0.00    0.00     0.00     0.00
> > 0.00     0.00    0.00   2.00   1.00
> > dm-31             0.00     0.00    0.00    0.00     0.00     0.00
> > 0.00     0.00    0.00   3.00   1.50
> > dm-30             0.00     0.00    0.00    0.00     0.00     0.00
> > 0.00     0.00    0.00   0.40   0.50
> > dm-31             0.00     0.00    0.00    0.00     0.00     0.00
> > 0.00     0.00    0.00   0.70   0.60
> >
> > Basically I am interested with 10 and 11 ( last 2 column) with bellow
> > 2 way
> >
> > a) ignore the firsrt value for each dm  ( so line number 1 and 2 will
> > totally ignore)  : I am having trouble to do that
>
> We can do that at the end, just before getting the max().
>
>
> > b) get the max value for clumn 10 and 11 for each dm (dm-30 and dm-31)
> > : I am having to do that
>
> > def _get_io():
> >
> >         with open (DM,'r')as f:
> >                 content=f.readlines()
> >         return content
> >
> >
> > s_value= dict()
> > u_value=dict()
> >
>
> You don't need two variables here, just one will do, lets just call it
> values.
>
>
> > if __name__ == '__main__':
> >
> >         dm_data=_get_io()
> >
> >         for line in dm_data:
> >
>
> You missed the line that split the fields
>
>            fields = line.split()
>
>
> >                 if fields[0] in s_value:
> >                         s_value[fields[0]].append(fields[10])
> >                 else:
> >                         s_value[fields[0]]=[fields[10]]
> >
>
> We can tidy that up using the setdefault() method:
>
>           data = values.setdefault(fields[0],[ [],[] ])     # return
> lists if avail else empty lists
>           data[0],append(float(fields[10])
>           data[1],append(float(fields[11])
>
> See if that gets you closer.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>


-- 
Regards
Fosiul Alam

From torral at telefonica.net  Thu Feb 25 11:13:38 2016
From: torral at telefonica.net (torral at telefonica.net)
Date: Thu, 25 Feb 2016 17:13:38 +0100
Subject: [Tutor] Dobut about a bug
Message-ID: <5D7A3EB480DF4EB9831C34EE14788AEF@UserPC>

Hellow I have downloaaded recently the python 3.5.1. It seems that there is a bug relating to version 3.5 and I want to know if it does afect the 3.5.1 version. Thanks in advance.

Regards

Carmen

From alan.gauld at btinternet.com  Thu Feb 25 17:01:54 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 25 Feb 2016 22:01:54 +0000
Subject: [Tutor] Dobut about a bug
In-Reply-To: <5D7A3EB480DF4EB9831C34EE14788AEF@UserPC>
References: <5D7A3EB480DF4EB9831C34EE14788AEF@UserPC>
Message-ID: <nantkh$v4p$1@ger.gmane.org>

On 25/02/16 16:13, torral at telefonica.net wrote:
> Hellow I have downloaaded recently the python 3.5.1. 
> It seems that there is a bug relating to version 3.5

There are always bugs in virtually all major software
releases. You need to be more specific about which
particular bug you are concerned about.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Thu Feb 25 17:04:10 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 25 Feb 2016 22:04:10 +0000
Subject: [Tutor] How to calculate high value from multiple lines for
 each column
In-Reply-To: <CAPBMF6yTqynGpAi_3o4uRnPAo1g3ZaJS2yc1gQLUHQPirww1yg@mail.gmail.com>
References: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
 <na1hkb$32l$1@ger.gmane.org>
 <CAPBMF6yihq_9ZMNEHbWj4A8eE5Q253EffQXE-HZOnsqRe_T0ww@mail.gmail.com>
 <56CD022F.9080700@btinternet.com>
 <CAPBMF6yTqynGpAi_3o4uRnPAo1g3ZaJS2yc1gQLUHQPirww1yg@mail.gmail.com>
Message-ID: <nantop$v4p$2@ger.gmane.org>

On 25/02/16 20:44, Fosiul Alam wrote:
> how ami I gettign the "values" ? as its not defiend
> anywhere, so how we putting data into variables "values" ?

Look more closely at my previous post.


>>> s_value= dict()
>>> u_value=dict()
>>>
>>
>> You don't need two variables here, just one will do, lets just call it
>> values.


>>           data = values.setdefault(fields[0],[ [],[] ])     
>>           data[0],append(float(fields[10])
>>           data[1],append(float(fields[11])

HTH

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Thu Feb 25 18:57:27 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 26 Feb 2016 10:57:27 +1100
Subject: [Tutor] Dobut about a bug
In-Reply-To: <5D7A3EB480DF4EB9831C34EE14788AEF@UserPC>
References: <5D7A3EB480DF4EB9831C34EE14788AEF@UserPC>
Message-ID: <20160225235726.GG12028@ando.pearwood.info>

On Thu, Feb 25, 2016 at 05:13:38PM +0100, torral at telefonica.net wrote:

> Hellow I have downloaaded recently the python 3.5.1. It seems that 
> there is a bug relating to version 3.5 and I want to know if it does 
> afect the 3.5.1 version. Thanks in advance.

Probably. There are dozens of open bugs listed in the bug tracker:

http://bugs.python.org/

Which bug are you referring to?

You can read the release notes to see what has been fixed in the 3.5.1 
release:

https://www.python.org/download/releases/3.5.1/

and the changelog:

https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-1-final



-- 
Steve

From breamoreboy at yahoo.co.uk  Thu Feb 25 17:09:42 2016
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 25 Feb 2016 22:09:42 +0000
Subject: [Tutor] Dobut about a bug
In-Reply-To: <5D7A3EB480DF4EB9831C34EE14788AEF@UserPC>
References: <5D7A3EB480DF4EB9831C34EE14788AEF@UserPC>
Message-ID: <nanu3v$8np$1@ger.gmane.org>

On 25/02/2016 16:13, torral at telefonica.net wrote:
> Hellow I have downloaaded recently the python 3.5.1. It seems that there is a bug relating to version 3.5 and I want to know if it does afect the 3.5.1 version. Thanks in advance.
>
> Regards
>
> Carmen

I am not aware of any bugs with Python 3.5.1 when compared to, 
presumably, 3.5.0.  Can you please specify your OS and what your 
perceived problem is.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From aneeque.khan at ericsson.com  Thu Feb 25 23:49:41 2016
From: aneeque.khan at ericsson.com (Aneeque Khan)
Date: Fri, 26 Feb 2016 04:49:41 +0000
Subject: [Tutor] Fwd: RE: How to source a shell script through python to
 set the environment variables.
In-Reply-To: <56CC4BE4.1070400@btinternet.com>
References: <9543F11B7EBBF24CBFD5DFC01818F99013B068@ESESSMB105.ericsson.se>
 <56CC4BE4.1070400@btinternet.com>
Message-ID: <9543F11B7EBBF24CBFD5DFC01818F99013C4A3@ESESSMB105.ericsson.se>

Hi guys, any suggestions on this.

-----Original Message-----
From: Tutor [mailto:tutor-bounces+aneeque.khan=ericsson.com at python.org] On Behalf Of Alan Gauld
Sent: Tuesday, February 23, 2016 5:39 PM
To: tutor
Subject: [Tutor] Fwd: RE: How to source a shell script through python to set the environment variables.

Forwarding to list.
Please always use ReplyAll (or replyList) when responding to tutor.


-------- Forwarded Message --------
Subject: 	RE: [Tutor] How to source a shell script through python to set
the environment variables.
Date: 	Tue, 23 Feb 2016 11:14:02 +0000
From: 	Aneeque Khan <aneeque.khan at ericsson.com>
To: 	Alan Gauld <alan.gauld at btinternet.com>



Please see my comments with [AK]

-----Original Message-----
From: Tutor [mailto:tutor-bounces+aneeque.khan=ericsson.com at python.org] On Behalf Of Alan Gauld
Sent: Tuesday, February 23, 2016 4:16 PM
To: tutor at python.org
Subject: Re: [Tutor] How to source a shell script through python to set the environment variables.

On 23/02/16 07:29, Aneeque Khan wrote:

> env_test.sh looks like this :-
> #!/bin/bash
> export OS_AK="Aneeque"
> export OS_KHAN="KHAN"
> 
> echo "Please enter your Stack Password: "
> read -sr OS_PWD_INPUT
> export OS_PWD=$OS_PWD_INPUT
> 
> Now I want to execute the above script through python file named 
> "ak_test.py", to set these variables and access them further in the code.

Can I first ask why you are trying to do it via a shell script rather than using Python directly? Is this part of a bigger workflow?

[AK] :- this is the part of bigger workflow, where I get the shell script as a output of some processing.
 
The problem is that virtually any method of running a shell script will involve starting a separate process and setting the variables in that process' environment.

So far as I can tell the only way to do what you want would be to run your shell script first then launch your python script from the shell script. That would mean breaking your current python script in two, and if you do a lot of work prior to launching the shell, that's probably not practical.

[AK] :- I can`t set the variables from outside the python script as our workflow requires to process no. of such shell scripts. We have to process each shell script to set the environment and work on further.
	You can assume that overall workflow have large no. of users each having its own values for certain environment variables. While performing any task for that particular user we require to set the environment first.


> def update_env(script_path):
>    if "--child" in sys.argv: # executed in the child environment
>       pprint(dict(os.environ))
>    else:
>       python, script = quote(sys.executable), quote(sys.argv[0])
>       os.execl("/bin/bash", "/bin/bash", "-c", "source %s; %s %s 
> --child" % (script_path, python, script))
> 
> with this approach some variables set while others not.

That surprises me since calling execl() overwrites the calling process with the called process, so I would not expect you to see anything in your calling script.

How are you testing the results?
Are you checking os.environ?
Or are you using os.getenv()?

[AK] :- I am checking like this :- 
	 os.getenv("OS_AK")
	os.getenv("OS_PWD")
and getting None as the result.

> Another approach used :-
> def update_env2(script):
>     #pipe1 = subprocess.Popen(". %s; env -0" % script, stdout=subprocess.PIPE, shell=True, executable="/bin/bash")
>     pipe1 = subprocess.Popen(". %s; env" % script, 
> stdout=subprocess.PIPE, shell=True, executable="/bin/bash")
> 
>     output = pipe1.communicate()[0]
>     #env = dict((line.split("=", 1) for line in output.splitlines()))
>     env = dict((line.split("=", 1) for line in output.split('\0')))
> 
>     os.environ.update(env)

While this could work you are not really reading the environment variables you are simply reading the stdout. You could achieve the same by parsing the script as a text file.

So can you explain why you need to load these values from a shell script rather than setting os.environ directly?
Maybe we can solve the bigger issue.

[AK] :- In workflow neither variable names nor their values are known in advance. We only came to know about these at run time through the generated shell scripts.
I have recently started with python, can you direct me how we can achieve this by parsing the shell as a text file.

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist  -  Tutor at python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor



_______________________________________________
Tutor maillist  -  Tutor at python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

From fosiul at gmail.com  Thu Feb 25 21:36:48 2016
From: fosiul at gmail.com (Fosiul Alam)
Date: Fri, 26 Feb 2016 02:36:48 +0000
Subject: [Tutor] How to calculate high value from multiple lines for
 each column
In-Reply-To: <nantop$v4p$2@ger.gmane.org>
References: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
 <na1hkb$32l$1@ger.gmane.org>
 <CAPBMF6yihq_9ZMNEHbWj4A8eE5Q253EffQXE-HZOnsqRe_T0ww@mail.gmail.com>
 <56CD022F.9080700@btinternet.com>
 <CAPBMF6yTqynGpAi_3o4uRnPAo1g3ZaJS2yc1gQLUHQPirww1yg@mail.gmail.com>
 <nantop$v4p$2@ger.gmane.org>
Message-ID: <CAPBMF6wenLA+wHS-fhWw2+x_cd+MqETDXW81-x+ewGbusotD_g@mail.gmail.com>

Hi Alan

Thanks for the reply, so bellow is the code, hope this is right ,

here 2 issues, i am having , not sure where i am doing wrong. really sorry
if i am doing something very silly .

1) the final data out put only has  one values  dm-31 but dm-30 is missing.
2) I need to assoicated values with dm name example dm-30 and dm-31 as
final result .

Thanks for your help .


for line in dm_data:
                fields= line.split()
                print fields
                data = values.setdefault(fields[0],[ [],[] ])     # return
                print data
                print "==="
                data[0].append(float(fields[10]))
                data[1].append(float(fields[11]))
        print "------"
        print data

bellow is the out put

['dm-30', '1.47', '36.48', '2.82', '0.66', '270.84', '148.56', '240.96',
'0.06', '44.99', '18.27', '6.36']
[[], []]
===
['dm-31', '1.47', '36.49', '2.82', '0.66', '270.85', '148.58', '240.94',
'0.06', '45.03', '18.28', '6.37']
[[], []]
===
['dm-30', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00',
'0.00', '2.00', '1.00']
[[18.27], [6.3600000000000003]]
===
['dm-31', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00',
'0.00', '3.00', '1.50']
[[18.280000000000001], [6.3700000000000001]]
===
['dm-30', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00',
'0.00', '0.40', '0.50']
[[18.27, 2.0], [6.3600000000000003, 1.0]]
===
['dm-31', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00',
'0.00', '0.70', '0.60']
[[18.280000000000001, 3.0], [6.3700000000000001, 1.5]]
===
------
[[18.280000000000001, 3.0, 0.69999999999999996], [6.3700000000000001, 1.5,
0.59999999999999998]]

On Thu, Feb 25, 2016 at 10:04 PM, Alan Gauld <alan.gauld at btinternet.com>
wrote:

> On 25/02/16 20:44, Fosiul Alam wrote:
> > how ami I gettign the "values" ? as its not defiend
> > anywhere, so how we putting data into variables "values" ?
>
> Look more closely at my previous post.
>
>
> >>> s_value= dict()
> >>> u_value=dict()
> >>>
> >>
> >> You don't need two variables here, just one will do, lets just call it
> >> values.
>
>
> >>           data = values.setdefault(fields[0],[ [],[] ])
> >>           data[0],append(float(fields[10])
> >>           data[1],append(float(fields[11])
>
> HTH
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Regards
Fosiul Alam

From unee0x at gmail.com  Thu Feb 25 23:59:21 2016
From: unee0x at gmail.com (kay Cee)
Date: Thu, 25 Feb 2016 23:59:21 -0500
Subject: [Tutor] Private members?
Message-ID: <9EA3259D-F89B-4C4A-8032-E3A17DDA7296@gmail.com>

Say I have a basic Circle class, for example:

class Circle:
    def __init__(self, radius):
        self.__radius = radius 

Does adding the double underscore make this member directly inaccessible to children of the Circle class?

Also, I'd like to know if there are any side effects to programming classes this way?


Sent from my iPhone

From alan.gauld at btinternet.com  Fri Feb 26 03:12:56 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 26 Feb 2016 08:12:56 +0000
Subject: [Tutor] How to calculate high value from multiple lines for
 each column
In-Reply-To: <CAPBMF6wenLA+wHS-fhWw2+x_cd+MqETDXW81-x+ewGbusotD_g@mail.gmail.com>
References: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
 <na1hkb$32l$1@ger.gmane.org>
 <CAPBMF6yihq_9ZMNEHbWj4A8eE5Q253EffQXE-HZOnsqRe_T0ww@mail.gmail.com>
 <56CD022F.9080700@btinternet.com>
 <CAPBMF6yTqynGpAi_3o4uRnPAo1g3ZaJS2yc1gQLUHQPirww1yg@mail.gmail.com>
 <nantop$v4p$2@ger.gmane.org>
 <CAPBMF6wenLA+wHS-fhWw2+x_cd+MqETDXW81-x+ewGbusotD_g@mail.gmail.com>
Message-ID: <56D00908.10009@btinternet.com>

On 26/02/16 02:36, Fosiul Alam wrote:
> 1) the final data out put only has  one values  dm-31 but dm-30 is
> missing.

That's because data is only ever the last iteration of the loop.
Your output is in values not data.
Try printing values at the end of the loop.

> 2) I need to assoicated values with dm name example dm-30 and dm-31 as
> final result .

You are already doing that.
Its working as expected.

> ['dm-30', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00',
> '0.00', '0.00', '0.40', '0.50']
> [[18.27, 2.0], [6.3600000000000003, 1.0]]
> ===
> ['dm-31', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00',
> '0.00', '0.00', '0.70', '0.60']
> [[18.280000000000001, 3.0], [6.3700000000000001, 1.5]]

Note that the data lines above reflect the correct results for both dm30
and dm31.

So values is storing both sets of data correctly.

After the loop try:

print "DM30: ", values['dm-30']
print "DM31:", values['dm-31']


HTH,

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


From alan.gauld at btinternet.com  Fri Feb 26 03:25:44 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 26 Feb 2016 08:25:44 +0000
Subject: [Tutor] Fwd: RE: How to source a shell script through python to
 set the environment variables.
In-Reply-To: <9543F11B7EBBF24CBFD5DFC01818F99013C4A3@ESESSMB105.ericsson.se>
References: <9543F11B7EBBF24CBFD5DFC01818F99013B068@ESESSMB105.ericsson.se>
 <56CC4BE4.1070400@btinternet.com>
 <9543F11B7EBBF24CBFD5DFC01818F99013C4A3@ESESSMB105.ericsson.se>
Message-ID: <56D00C08.6030200@btinternet.com>

On 26/02/16 04:49, Aneeque Khan wrote:
> The problem is that virtually any method of running a shell script
> will involve starting a separate process and setting the variables in
> that process' environment. So far as I can tell the only way to do
> what you want would be to run your shell script first then launch your
> python script from the shell script. That would mean breaking your
> current python script in two, and if you do a lot of work prior to
> launching the shell, that's probably not practical. 

Is it possible to adopt the approach described above?
Can you split your Python processing into two(or more)
parts, one before and one after the shell  script?
That will be the easiest way I suspect.

> I have recently started with python, can you direct me how we can achieve
> this by parsing the shell as a text file.

While parsing a shell script is not impossible, especially since you are
only
looking to trap the environment variables it's far from trivial.
Especially for
a beginner. I'd treat that as a last resort.

Also if the shell script does more than just set the variables its not
going
to work since you still need to run the script.

Do you have any control of the shell script content?
Or is that generated outside of your influence?
If you can control it you might be able to get the script to
dump it's environment vars to a temp file and read that.

The next question is what do you plan on doing with these once you
have them? They won't be of any value to the shell script or any other
script unless you set them again locally? Or are the just path locators?

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


From alan.gauld at btinternet.com  Fri Feb 26 03:36:03 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Fri, 26 Feb 2016 08:36:03 +0000
Subject: [Tutor] Private members?
In-Reply-To: <9EA3259D-F89B-4C4A-8032-E3A17DDA7296@gmail.com>
References: <9EA3259D-F89B-4C4A-8032-E3A17DDA7296@gmail.com>
Message-ID: <nap2pj$524$1@ger.gmane.org>

On 26/02/16 04:59, kay Cee wrote:
> Say I have a basic Circle class, for example:
> 
> class Circle:
>     def __init__(self, radius):
>         self.__radius = radius 
> 
> Does adding the double underscore make this member directly inaccessible to children of the Circle class?

The fastest way to answer that question would be for you to try
it in the interpreter!


>>> class C:
...   def __init__(s,v): s.__v = v
...
>>> class D(C):
...    def __init__(s,v): C.__init__(s,v)
...    def printv(s): print s.__v
...
>>> d = D(5)
>>> d.printv()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in printv
AttributeError: D instance has no attribute '_D__v'
>>>


> Also, I'd like to know if there are any side effects to programming classes this way?

Depends what you mean by side effects.
- The name gets mangled by the interpreter (actually by the
  compiler I think)
- The variable is not directly accessible so if you need
  access to the data you need to write accessors and
  setters and/or create a property.
- The code is not idiomatic Python since we mostly
  don't bother making things private.

But those are not really side-effects in a computing sense.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From steve at pearwood.info  Fri Feb 26 03:55:14 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 26 Feb 2016 19:55:14 +1100
Subject: [Tutor] Private members?
In-Reply-To: <9EA3259D-F89B-4C4A-8032-E3A17DDA7296@gmail.com>
References: <9EA3259D-F89B-4C4A-8032-E3A17DDA7296@gmail.com>
Message-ID: <20160226085514.GI12028@ando.pearwood.info>

On Thu, Feb 25, 2016 at 11:59:21PM -0500, kay Cee wrote:
> Say I have a basic Circle class, for example:
> 
> class Circle:
>     def __init__(self, radius):
>         self.__radius = radius 
> 
> Does adding the double underscore make this member directly inaccessible to children of the Circle class?

No, but it makes it harder to do so by accident and annoying to do so 
deliberately. In order to access a double-underscore attribute like 
__radius, the child needs to know the name of the class which defined 
it:

class MyCircle(Circle):
    def method(self):
        try:
            print(self.__radius)
        except AttributeError:
            print("__radius not found!")
        r = self._Circle__radius
        print("self._Circle__radius:", r)


And an example in use:

py> obj = MyCircle(999)
py> obj.method()
__radius not found!
self._Circle__radius: 999



What happens here is that when you refer to "self.__radius" inside a 
method, Python mangles the name to "self._CLASSNAME__radius" instead. 
(For whatever the class is actually named.) This happens automatically, 
and is intended as protection against accidental name clashes between 
the parent class and its children.


> Also, I'd like to know if there are any side effects to programming classes this way?

Yes, it's usually annoying and unnecessary.

The usual advice given is not to bother with double underscores unless 
you are really, really sure you need them.


-- 
Steve

From esawiek at gmail.com  Fri Feb 26 11:46:42 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Fri, 26 Feb 2016 11:46:42 -0500
Subject: [Tutor] Convert structured 1D array to 2D array
Message-ID: <CA+ZkTxv+e190w849ewDSiHjS+PbePbfOT-kwOrLRdrXE0HLthA@mail.gmail.com>

Hi All?



I used genfromtxt to read a file with multiple data types. The result was a
1D array of tuples; for example

[(1, 2, 3), (?a?, b?, ?c?), (?12/12/2009?, ?2/4/2014?, ?3/4/200)?]



I am trying to convert this structured array to a 2D array. Is this
possible where the data types in the 1D array are preserved?



Thanks in advance-EKE

From __peter__ at web.de  Fri Feb 26 13:28:58 2016
From: __peter__ at web.de (Peter Otten)
Date: Fri, 26 Feb 2016 19:28:58 +0100
Subject: [Tutor] Convert structured 1D array to 2D array
References: <CA+ZkTxv+e190w849ewDSiHjS+PbePbfOT-kwOrLRdrXE0HLthA@mail.gmail.com>
Message-ID: <naq5ha$s25$1@ger.gmane.org>

Ek Esawi wrote:

> I used genfromtxt to read a file with multiple data types. The result was
> a 1D array of tuples; for example
> 
> [(1, 2, 3), (?a?, b?, ?c?), (?12/12/2009?, ?2/4/2014?, ?3/4/200)?]
> 
> 
> 
> I am trying to convert this structured array to a 2D array. Is this
> possible where the data types in the 1D array are preserved?

You can specify object as the dtype for a numpy array:

>>> t = [(1, 2, 3), ('a', 'b', 'c'), ('12/12/2009', '2/4/2014', '3/4/2000')]
>>> numpy.array(t, dtype=object)
array([[1, 2, 3],
       ['a', 'b', 'c'],
       ['12/12/2009', '2/4/2014', '3/4/2000']], dtype=object)



From esawiek at gmail.com  Fri Feb 26 18:20:17 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Fri, 26 Feb 2016 18:20:17 -0500
Subject: [Tutor] Convert structured 1D array to 2D array
Message-ID: <CA+ZkTxtC74c4wjYPTQX-1tvNd-tv29g4+RbjR-t=1PTk_s+kbw@mail.gmail.com>

Thanks for the input. I tried your idea but i did not work; l got 1D array
of tuples; the same as the original array. I think b/c genfromtxt creates
an array and so it's already an array object-if i understand it correctly.
Your idea works well for a list which i tested but i want to an n x n 2D
array from 1D n x 1 tuples, each tuple has n elements,.

Thanks again-_EK

From beachkidken at gmail.com  Fri Feb 26 20:30:19 2016
From: beachkidken at gmail.com (Ken G.)
Date: Fri, 26 Feb 2016 20:30:19 -0500
Subject: [Tutor] Unable to get out of loop
Message-ID: <56D0FC2B.5060704@gmail.com>

I have been unable to get out of the following
loop. No matter what I entered, I can not get
get out of the loop. The only way I can stop
the routine is CTRL-C.

If an entry is either 1, 2 or 3, then I should
be able to proceed. I am sure the correct solution
is very simple but I don't see it.

Thanks.

print
side = ""
while side != "0" or side != "1" or side != "2":
     print "\t1 = right arm\t2 = left arm\t0 = exit program"
     print
     side = raw_input("\t")
     print
print
print "\tYou entered", side
print

From danny.yoo at gmail.com  Fri Feb 26 21:19:22 2016
From: danny.yoo at gmail.com (Danny Yoo)
Date: Fri, 26 Feb 2016 18:19:22 -0800
Subject: [Tutor] Unable to get out of loop
In-Reply-To: <56D0FC2B.5060704@gmail.com>
References: <56D0FC2B.5060704@gmail.com>
Message-ID: <CAGZAPF4CuNtG5LW7kKnyzC991M4CS-6nj6FuqRgZayp8W5xqig@mail.gmail.com>

On Feb 26, 2016 5:30 PM, "Ken G." <beachkidken at gmail.com> wrote:
>
> I have been unable to get out of the following
> loop. No matter what I entered, I can not get
> get out of the loop. The only way I can stop
> the routine is CTRL-C.

Yes: the condition on there code is unfortunately a tautology: it's always
going to be true.

For example, if side=0, although that fails the first sub-condition, it
succeeds in the other two, and because it uses the boolean 'or' operation,
the condition as a whole is successful.  That's why the condition is always
going to be true.

Consider the difference between boolean 'or' and boolean 'and' instead.

From danny.yoo at gmail.com  Fri Feb 26 21:24:23 2016
From: danny.yoo at gmail.com (Danny Yoo)
Date: Fri, 26 Feb 2016 18:24:23 -0800
Subject: [Tutor] Unable to get out of loop
In-Reply-To: <CAGZAPF4CuNtG5LW7kKnyzC991M4CS-6nj6FuqRgZayp8W5xqig@mail.gmail.com>
References: <56D0FC2B.5060704@gmail.com>
 <CAGZAPF4CuNtG5LW7kKnyzC991M4CS-6nj6FuqRgZayp8W5xqig@mail.gmail.com>
Message-ID: <CAGZAPF4DDNzt0s1KnUbEnr-Nz1sPS8_40H5R-RNMSoWJXYv3Fw@mail.gmail.com>

The last point I forgot to add: be careful about the "englishness" of 'and'
and 'or'.  The way we use them in day to day language among humans is
different than what those terms mean in computer programming.  It can be a
sticking point for some beginners, so just be aware of that nuance.

(And apologies for misspelling in messages: I'm hurriedly doing this on a
cell phone at the moment.  ;p)

Best of wishes!

From dyoo at hashcollision.org  Fri Feb 26 21:28:01 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Fri, 26 Feb 2016 18:28:01 -0800
Subject: [Tutor] Convert structured 1D array to 2D array
In-Reply-To: <CA+ZkTxv+e190w849ewDSiHjS+PbePbfOT-kwOrLRdrXE0HLthA@mail.gmail.com>
References: <CA+ZkTxv+e190w849ewDSiHjS+PbePbfOT-kwOrLRdrXE0HLthA@mail.gmail.com>
Message-ID: <CAGZAPF4CQ5n_rZDv5fYLko-YjO6poww2vRbHgyjSyvmvKbgn8g@mail.gmail.com>

On Feb 26, 2016 9:29 AM, "Ek Esawi" <esawiek at gmail.com> wrote:
>
> Hi All?
>
>
>
> I used genfromtxt to read a file with multiple data types. The result was
a
> 1D array of tuples; for example
>
> [(1, 2, 3), (?a?, b?, ?c?), (?12/12/2009?, ?2/4/2014?, ?3/4/200)?]
>
>
>
> I am trying to convert this structured array to a 2D array.

What would be the expected result for the case you're presenting above?
Let us know: that particular detail matters.

From steve at pearwood.info  Fri Feb 26 21:26:07 2016
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 27 Feb 2016 13:26:07 +1100
Subject: [Tutor] Unable to get out of loop
In-Reply-To: <56D0FC2B.5060704@gmail.com>
References: <56D0FC2B.5060704@gmail.com>
Message-ID: <20160227022606.GJ12028@ando.pearwood.info>

On Fri, Feb 26, 2016 at 08:30:19PM -0500, Ken G. wrote:

> side = ""
> while side != "0" or side != "1" or side != "2":

That will only exit if side == "0", "1" and "2" **at the same time**. 
Clearly that is impossible, so the condition is always true, and the 
while loop will never end.

Trying to reason about "reversed conditions" like that is hard, much 
much harder than reasoning about ordinary conditions. You should 
re-write it in terms of equality, then reverse the condition once:

while side not in ("0", "1", "2):
    ...


-- 
Steve

From rakeshsharma14 at hotmail.com  Fri Feb 26 22:04:50 2016
From: rakeshsharma14 at hotmail.com (rakesh sharma)
Date: Sat, 27 Feb 2016 03:04:50 +0000
Subject: [Tutor] Private members?
In-Reply-To: <9EA3259D-F89B-4C4A-8032-E3A17DDA7296@gmail.com>
References: <9EA3259D-F89B-4C4A-8032-E3A17DDA7296@gmail.com>
Message-ID: <BM1PR01MB01960B42167FB8E3A771F713C8B80@BM1PR01MB0196.INDPRD01.PROD.OUTLOOK.COM>

I believe in Python there are no private public concept. All are public by default

Sent from Outlook Mobile<https://aka.ms/blhgte>



On Fri, Feb 26, 2016 at 12:04 AM -0800, "kay Cee" <unee0x at gmail.com<mailto:unee0x at gmail.com>> wrote:

Say I have a basic Circle class, for example:

class Circle:
    def __init__(self, radius):
        self.__radius = radius

Does adding the double underscore make this member directly inaccessible to children of the Circle class?

Also, I'd like to know if there are any side effects to programming classes this way?


Sent from my iPhone
_______________________________________________
Tutor maillist  -  Tutor at python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

From __peter__ at web.de  Sat Feb 27 03:01:07 2016
From: __peter__ at web.de (Peter Otten)
Date: Sat, 27 Feb 2016 09:01:07 +0100
Subject: [Tutor] Convert structured 1D array to 2D array
References: <CA+ZkTxtC74c4wjYPTQX-1tvNd-tv29g4+RbjR-t=1PTk_s+kbw@mail.gmail.com>
Message-ID: <narl45$ptf$1@ger.gmane.org>

Ek Esawi wrote:

> Thanks for the input. I tried your idea but i did not work; l got 1D array
> of tuples; the same as the original array. I think b/c genfromtxt creates
> an array and so it's already an array object-if i understand it correctly.
> Your idea works well for a list which i tested but i want to an n x n 2D
> array from 1D n x 1 tuples, each tuple has n elements,.

Be creative, find a workaround like

a = [...]
b = numpy.array([tuple(row) for row in a], dtype=object)


Or try 

b = numpy.array(a, dtype=(object, object))


From __peter__ at web.de  Sat Feb 27 03:18:58 2016
From: __peter__ at web.de (Peter Otten)
Date: Sat, 27 Feb 2016 09:18:58 +0100
Subject: [Tutor] Convert structured 1D array to 2D array
References: <CA+ZkTxtC74c4wjYPTQX-1tvNd-tv29g4+RbjR-t=1PTk_s+kbw@mail.gmail.com>
 <narl45$ptf$1@ger.gmane.org>
Message-ID: <narm5j$9mr$1@ger.gmane.org>

Peter Otten wrote:

> Or try
> 
> b = numpy.array(a, dtype=(object, object))

That won't work, I accidentally tested it on an already converted array :(


From esawiek at gmail.com  Fri Feb 26 22:26:22 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Fri, 26 Feb 2016 22:26:22 -0500
Subject: [Tutor] Convert structured 1D array to 2D array
Message-ID: <CA+ZkTxsSDWHTnFqtruDEv1Jd5+oJe6X11+LqjNYqy_oYcTWAsA@mail.gmail.com>

The result i am looking for is a 2D array (matrix) instead of 1D array
of tuples. That's when i read a file using genfromtxt, it generated a 1D
array of n tuples (like the example shown below); each tuple has one row
from the original file. The file contains multiple data types and so i used
genfromtxt to specify data types for each column.


In another way, i just want to convert a 1D array of tuples to 2D array
where each tuple is converted multiple columns. Here is an example: the
output from genfromtxt is

[(1, 2, 3), (?a?, b?, ?c?), (?12/12/2009?, ?2/4/2014?, ?3/4/200?)]

I want this be converted to:

[[1,                         2,              3],

[?a?,                        b?,           ?c?],

[?12/12/2009?, ?2/4/2014?, ?3/4/200?]]




Thanks again--EKE

From __peter__ at web.de  Sat Feb 27 07:16:00 2016
From: __peter__ at web.de (Peter Otten)
Date: Sat, 27 Feb 2016 13:16 +0100
Subject: [Tutor] Convert structured 1D array to 2D array
References: <CA+ZkTxsSDWHTnFqtruDEv1Jd5+oJe6X11+LqjNYqy_oYcTWAsA@mail.gmail.com>
Message-ID: <nas420$fml$1@ger.gmane.org>

Ek Esawi wrote:

> The result i am looking for is a 2D array (matrix) instead of 1D array
> of tuples. That's when i read a file using genfromtxt, it generated a 1D
> array of n tuples (like the example shown below); each tuple has one row
> from the original file. The file contains multiple data types and so i
> used genfromtxt to specify data types for each column.
> 
> 
> In another way, i just want to convert a 1D array of tuples to 2D array
> where each tuple is converted multiple columns. Here is an example: the
> output from genfromtxt is
> 
> [(1, 2, 3), (?a?, b?, ?c?), (?12/12/2009?, ?2/4/2014?, ?3/4/200?)]

Can you provide the script (should be tiny) that produces this output?



From beachkidken at gmail.com  Sat Feb 27 07:36:53 2016
From: beachkidken at gmail.com (Ken G.)
Date: Sat, 27 Feb 2016 07:36:53 -0500
Subject: [Tutor] Unable to get out of loop [RESOLVING]
In-Reply-To: <20160227022606.GJ12028@ando.pearwood.info>
References: <56D0FC2B.5060704@gmail.com>
 <20160227022606.GJ12028@ando.pearwood.info>
Message-ID: <56D19865.9030505@gmail.com>

On 02/26/2016 09:26 PM, Steven D'Aprano wrote:
> On Fri, Feb 26, 2016 at 08:30:19PM -0500, Ken G. wrote:
>
>> side = ""
>> while side != "0" or side != "1" or side != "2":
> That will only exit if side == "0", "1" and "2" **at the same time**.
> Clearly that is impossible, so the condition is always true, and the
> while loop will never end.
>
> Trying to reason about "reversed conditions" like that is hard, much
> much harder than reasoning about ordinary conditions. You should
> re-write it in terms of equality, then reverse the condition once:
>
> while side not in ("0", "1", "2):
>      ...
>
>
Thanks Danny and Steven for your response. I will rewrite the code and 
try again. I had no idea I opened up a can of worms and now, I am 
putting them back. Thanks!

Ken



From emile at fenx.com  Sat Feb 27 17:41:19 2016
From: emile at fenx.com (Emile van Sebille)
Date: Sat, 27 Feb 2016 14:41:19 -0800
Subject: [Tutor] Convert structured 1D array to 2D array
In-Reply-To: <CA+ZkTxsSDWHTnFqtruDEv1Jd5+oJe6X11+LqjNYqy_oYcTWAsA@mail.gmail.com>
References: <CA+ZkTxsSDWHTnFqtruDEv1Jd5+oJe6X11+LqjNYqy_oYcTWAsA@mail.gmail.com>
Message-ID: <nat8nr$sli$1@ger.gmane.org>

On 2/26/2016 7:26 PM, Ek Esawi wrote:
> The result i am looking for is a 2D array (matrix) instead of 1D array
> of tuples. That's when i read a file using genfromtxt, it generated a 1D
> array of n tuples (like the example shown below); each tuple has one row
> from the original file. The file contains multiple data types and so i used
> genfromtxt to specify data types for each column.
>
>
> In another way, i just want to convert a 1D array of tuples to 2D array
> where each tuple is converted multiple columns. Here is an example: the
> output from genfromtxt is
>
> [(1, 2, 3), (?a?, b?, ?c?), (?12/12/2009?, ?2/4/2014?, ?3/4/200?)]
>
> I want this be converted to:
>
> [[1,                         2,              3],
>
> [?a?,                        b?,           ?c?],
>
> [?12/12/2009?, ?2/4/2014?, ?3/4/200?]]

Being that the white space in lists and tuples is irrelevant, this has 
simply converted the inner tuples to lists --

tuples = [(1,2,3),('a','b','c'),('12/12/2009','2/4/2014','3/4/200')]
lists =map(list,tuples)

Emile



From esawiek at gmail.com  Sat Feb 27 15:24:55 2016
From: esawiek at gmail.com (Ek Esawi)
Date: Sat, 27 Feb 2016 15:24:55 -0500
Subject: [Tutor] Convert structured 1D array to 2D array
Message-ID: <CA+ZkTxsrkptzndJL=vQ2Wn=FOK+EAdkLqq7Yge9HpcW6veU3Vg@mail.gmail.com>

Thanks Peter. It works now. I am trying to be creative but i am still
getting my way around pythony. EKE

From fosiul at gmail.com  Sat Feb 27 10:46:57 2016
From: fosiul at gmail.com (Fosiul Alam)
Date: Sat, 27 Feb 2016 15:46:57 +0000
Subject: [Tutor] How to calculate high value from multiple lines for
 each column
In-Reply-To: <56D00908.10009@btinternet.com>
References: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
 <na1hkb$32l$1@ger.gmane.org>
 <CAPBMF6yihq_9ZMNEHbWj4A8eE5Q253EffQXE-HZOnsqRe_T0ww@mail.gmail.com>
 <56CD022F.9080700@btinternet.com>
 <CAPBMF6yTqynGpAi_3o4uRnPAo1g3ZaJS2yc1gQLUHQPirww1yg@mail.gmail.com>
 <nantop$v4p$2@ger.gmane.org>
 <CAPBMF6wenLA+wHS-fhWw2+x_cd+MqETDXW81-x+ewGbusotD_g@mail.gmail.com>
 <56D00908.10009@btinternet.com>
Message-ID: <CAPBMF6yyFk2+F1951hGKKq4Uu-3J5bLnbucBnVton0DYdGes1w@mail.gmail.com>

Hi Alan
Thanks for the help.yes now i can see how it working, i am just stick in
small things
so this is what i have done ,

a) Can I combile 2 forloop into one ?
b) How can remove first value from for loop - step a ?


        for key in values:
                print "svtm-%s ,%s" % (key, values[key][0])

        for key in values:
                print "iostat-%s ,%s" % (key, values[key][1])
        #print data
        print "Remove First value", values['dm-30'][0].pop(0)
        print "SVTM DM30: ", max(values['dm-30'][0])
        print "Remove First VAlue for", values['dm-30'][1].pop(0)
        print "IOSTAT DM-30",max(values['dm-30'][1])


output

svtm-dm-30 ,[18.27, 2.0, 0.40000000000000002]
svtm-dm-31 ,[18.280000000000001, 3.0, 0.69999999999999996]
iostat-dm-30 ,[6.3600000000000003, 1.0, 0.5]
iostat-dm-31 ,[6.3700000000000001, 1.5, 0.59999999999999998]
Remove First value 18.27
SVTM DM30:  2.0
Remove First VAlue for 6.36
IOSTAT DM-30 1.0

Thanks for the help.


On Fri, Feb 26, 2016 at 8:12 AM, Alan Gauld <alan.gauld at btinternet.com>
wrote:

> On 26/02/16 02:36, Fosiul Alam wrote:
> > 1) the final data out put only has  one values  dm-31 but dm-30 is
> > missing.
>
> That's because data is only ever the last iteration of the loop.
> Your output is in values not data.
> Try printing values at the end of the loop.
>
> > 2) I need to assoicated values with dm name example dm-30 and dm-31 as
> > final result .
>
> You are already doing that.
> Its working as expected.
>
> > ['dm-30', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00',
> > '0.00', '0.00', '0.40', '0.50']
> > [[18.27, 2.0], [6.3600000000000003, 1.0]]
> > ===
> > ['dm-31', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00',
> > '0.00', '0.00', '0.70', '0.60']
> > [[18.280000000000001, 3.0], [6.3700000000000001, 1.5]]
>
> Note that the data lines above reflect the correct results for both dm30
> and dm31.
>
> So values is storing both sets of data correctly.
>
> After the loop try:
>
> print "DM30: ", values['dm-30']
> print "DM31:", values['dm-31']
>
>
> HTH,
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>


-- 
Regards
Fosiul Alam

From alan.gauld at btinternet.com  Sat Feb 27 19:30:26 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 28 Feb 2016 00:30:26 +0000
Subject: [Tutor] How to calculate high value from multiple lines for
 each column
In-Reply-To: <CAPBMF6yyFk2+F1951hGKKq4Uu-3J5bLnbucBnVton0DYdGes1w@mail.gmail.com>
References: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
 <na1hkb$32l$1@ger.gmane.org>
 <CAPBMF6yihq_9ZMNEHbWj4A8eE5Q253EffQXE-HZOnsqRe_T0ww@mail.gmail.com>
 <56CD022F.9080700@btinternet.com>
 <CAPBMF6yTqynGpAi_3o4uRnPAo1g3ZaJS2yc1gQLUHQPirww1yg@mail.gmail.com>
 <nantop$v4p$2@ger.gmane.org>
 <CAPBMF6wenLA+wHS-fhWw2+x_cd+MqETDXW81-x+ewGbusotD_g@mail.gmail.com>
 <56D00908.10009@btinternet.com>
 <CAPBMF6yyFk2+F1951hGKKq4Uu-3J5bLnbucBnVton0DYdGes1w@mail.gmail.com>
Message-ID: <natf32$muv$1@ger.gmane.org>

On 27/02/16 15:46, Fosiul Alam wrote:

> a) Can I combile 2 forloop into one ?

Yes of course.

      for key in values:
          print "svtm-%s ,%s" % (key, values[key][0])
          print "iostat-%s ,%s" % (key, values[key][1])


> b) How can remove first value from for loop - step a ?

>         print "Remove First value", values['dm-30'][0].pop(0)
>         print "SVTM DM30: ", max(values['dm-30'][0])

Instead of using pop() you can use slicing with [1:]:

    print "SVTM DM30: ", max(values['dm-30'][0][1:])
    print "IOSTAT DM-30",max(values['dm-30'][1][1:])


Does it all in one go.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From zachary.ware+pytut at gmail.com  Sun Feb 28 02:24:59 2016
From: zachary.ware+pytut at gmail.com (Zachary Ware)
Date: Sun, 28 Feb 2016 01:24:59 -0600
Subject: [Tutor] asyncio or threading
In-Reply-To: <n9ttco$v5m$1@ger.gmane.org>
References: <56C1CDEF.3030803@gmail.com> <n9t8l1$raf$1@ger.gmane.org>
 <CAKJDb-OsuOxFN0qnvq3J54UjRF_37+a=WuL_zf+ewf3vSMdFkA@mail.gmail.com>
 <n9ttco$v5m$1@ger.gmane.org>
Message-ID: <CAKJDb-NuWmU6xzs2bLY=JOvEJwv_FA=xjvLNVRfm_RbTb3PFoQ@mail.gmail.com>

My apologies for taking so long to reply here again, it's been a busy
couple of weeks.

On Mon, Feb 15, 2016 at 7:18 PM, Alan Gauld <alan.gauld at btinternet.com> wrote:
> On 15/02/16 21:08, Zachary Ware wrote:
>> This is not all asyncio can do. Callbacks are only one way of using it, the
>> other, more common method is to use coroutines such that most of your code
>> reads almost like regular, synchronous code.
>
> I'm not sure about "most common" since every single example of
> asyncio I've seen so far has used callbacks not coroutines.
> (Not that I've seen too many examples, but the ones I have seen
> all used callbacks.)

Could you point me towards some of those examples?

>> I don't think this is quite true either; I've heard the JavaScript
>> equivalent referred to frequently as "callback hell",
>
> A slanderous comment that's not really deserved :-)

Sorry; my information on this point is all secondhand (or worse) as
I've not used NodeJS and have somehow even avoided JavaScript as a
whole.  Take anything I say about JavaScript/NodeJS with a healthy
measure of salt :).  My impression was that the callback style
naturally leads to doing things where callbacks are chained several
layers deep, which makes things hard to read and takes concentrated
effort (slight though it may be) to avoid.

> As a matter of interest which tutorials on asyncio cover this best?
> I've only read one closely and it was all about callbacks but
> it impressed me enough to become an asyncio fan. However I'm
> certainly interested in finding out more on the coroutine front.

Frankly, I have yet to see a *good* asyncio tutorial, but haven't
looked very hard either.  I learned how to use it from the reference
docs (which are also somewhat lacking still, I had to fix some issues
in the docs as I was learning it) and trial-and-error, which actually
worked surprisingly well.

-- 
Zach

From alan.gauld at btinternet.com  Sun Feb 28 04:07:44 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 28 Feb 2016 09:07:44 +0000
Subject: [Tutor] asyncio or threading
In-Reply-To: <CAKJDb-NuWmU6xzs2bLY=JOvEJwv_FA=xjvLNVRfm_RbTb3PFoQ@mail.gmail.com>
References: <56C1CDEF.3030803@gmail.com> <n9t8l1$raf$1@ger.gmane.org>
 <CAKJDb-OsuOxFN0qnvq3J54UjRF_37+a=WuL_zf+ewf3vSMdFkA@mail.gmail.com>
 <n9ttco$v5m$1@ger.gmane.org>
 <CAKJDb-NuWmU6xzs2bLY=JOvEJwv_FA=xjvLNVRfm_RbTb3PFoQ@mail.gmail.com>
Message-ID: <naudd0$mt0$1@ger.gmane.org>

On 28/02/16 07:24, Zachary Ware wrote:

>> (Not that I've seen too many examples, but the ones I have seen
>> all used callbacks.)
> 
> Could you point me towards some of those examples?

As my followup post said they did actually mention coroutines
as well, I just didn't pick up on it at the time.

I suspect that's because call backs are in my programming DNA
since it's how most of my early C programming on embedded and
real time systems worked. So I saw async and callbacks and
thought "I've come home" :-)

> measure of salt :).  My impression was that the callback style
> naturally leads to doing things where callbacks are chained several
> layers deep, which makes things hard to read and takes concentrated
> effort (slight though it may be) to avoid.

It isn't hard to read if laid out sensibly, and you can define
separate functions for each call back rather than defining
them inline. But inline functions have their own style that
is no harder to read than, say, multi layer if/else statements
once you get used to them.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From fosiul at gmail.com  Sun Feb 28 06:37:05 2016
From: fosiul at gmail.com (Fosiul Alam)
Date: Sun, 28 Feb 2016 11:37:05 +0000
Subject: [Tutor] How to calculate high value from multiple lines for
 each column
In-Reply-To: <natf32$muv$1@ger.gmane.org>
References: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
 <na1hkb$32l$1@ger.gmane.org>
 <CAPBMF6yihq_9ZMNEHbWj4A8eE5Q253EffQXE-HZOnsqRe_T0ww@mail.gmail.com>
 <56CD022F.9080700@btinternet.com>
 <CAPBMF6yTqynGpAi_3o4uRnPAo1g3ZaJS2yc1gQLUHQPirww1yg@mail.gmail.com>
 <nantop$v4p$2@ger.gmane.org>
 <CAPBMF6wenLA+wHS-fhWw2+x_cd+MqETDXW81-x+ewGbusotD_g@mail.gmail.com>
 <56D00908.10009@btinternet.com>
 <CAPBMF6yyFk2+F1951hGKKq4Uu-3J5bLnbucBnVton0DYdGes1w@mail.gmail.com>
 <natf32$muv$1@ger.gmane.org>
Message-ID: <CAPBMF6y+oxV2540zXNjcpLWuEruNQrADEWEEMvL-uXawhR3YOg@mail.gmail.com>

Thanks Alan
bellow works perfect ..
and thanks you very much for all your support
I just need to do one thing which I should be able to do  my self,
hopefully will be able to finish the whole thing soon.
if i can i just ask you one more favour in related to this qustion.

for key in values:
                print "svtm-%s ,%s" % (key, max(values[key][0][1:]))
                print "iostat-%s ,%s" % (key, max(values[key][1][1:]))


On Sun, Feb 28, 2016 at 12:30 AM, Alan Gauld <alan.gauld at btinternet.com>
wrote:

> On 27/02/16 15:46, Fosiul Alam wrote:
>
> > a) Can I combile 2 forloop into one ?
>
> Yes of course.
>
>       for key in values:
>           print "svtm-%s ,%s" % (key, values[key][0])
>           print "iostat-%s ,%s" % (key, values[key][1])
>
>
> > b) How can remove first value from for loop - step a ?
>
> >         print "Remove First value", values['dm-30'][0].pop(0)
> >         print "SVTM DM30: ", max(values['dm-30'][0])
>
> Instead of using pop() you can use slicing with [1:]:
>
>     print "SVTM DM30: ", max(values['dm-30'][0][1:])
>     print "IOSTAT DM-30",max(values['dm-30'][1][1:])
>
>
> Does it all in one go.
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Regards
Fosiul Alam

From alan.gauld at btinternet.com  Sun Feb 28 10:19:16 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Sun, 28 Feb 2016 15:19:16 +0000
Subject: [Tutor] How to calculate high value from multiple lines for
 each column
In-Reply-To: <CAPBMF6y+oxV2540zXNjcpLWuEruNQrADEWEEMvL-uXawhR3YOg@mail.gmail.com>
References: <CAPBMF6zz4p0zQxQ8sCj379XyMto1xZ+TjbdJPLnVfA-MsY+Nyw@mail.gmail.com>
 <na1hkb$32l$1@ger.gmane.org>
 <CAPBMF6yihq_9ZMNEHbWj4A8eE5Q253EffQXE-HZOnsqRe_T0ww@mail.gmail.com>
 <56CD022F.9080700@btinternet.com>
 <CAPBMF6yTqynGpAi_3o4uRnPAo1g3ZaJS2yc1gQLUHQPirww1yg@mail.gmail.com>
 <nantop$v4p$2@ger.gmane.org>
 <CAPBMF6wenLA+wHS-fhWw2+x_cd+MqETDXW81-x+ewGbusotD_g@mail.gmail.com>
 <56D00908.10009@btinternet.com>
 <CAPBMF6yyFk2+F1951hGKKq4Uu-3J5bLnbucBnVton0DYdGes1w@mail.gmail.com>
 <natf32$muv$1@ger.gmane.org>
 <CAPBMF6y+oxV2540zXNjcpLWuEruNQrADEWEEMvL-uXawhR3YOg@mail.gmail.com>
Message-ID: <nav35j$ji1$1@ger.gmane.org>

On 28/02/16 11:37, Fosiul Alam wrote:

> if i can i just ask you one more favour in related to this qustion.
> 
> for key in values:
>                 print "svtm-%s ,%s" % (key, max(values[key][0][1:]))
>                 print "iostat-%s ,%s" % (key, max(values[key][1][1:]))
> 
> 

Sure, what is it?


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From oscar.j.benjamin at gmail.com  Sun Feb 28 14:46:50 2016
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sun, 28 Feb 2016 19:46:50 +0000
Subject: [Tutor] Convert structured 1D array to 2D array
In-Reply-To: <CA+ZkTxv+e190w849ewDSiHjS+PbePbfOT-kwOrLRdrXE0HLthA@mail.gmail.com>
References: <CA+ZkTxv+e190w849ewDSiHjS+PbePbfOT-kwOrLRdrXE0HLthA@mail.gmail.com>
Message-ID: <CAHVvXxQv88BBqjdCErv7rdP=WvEMEUheg5Jmq2Brs33sCJDaEQ@mail.gmail.com>

On 26 February 2016 at 16:46, Ek Esawi <esawiek at gmail.com> wrote:
>
> I used genfromtxt to read a file with multiple data types. The result was a
> 1D array of tuples; for example
>
> [(1, 2, 3), (?a?, b?, ?c?), (?12/12/2009?, ?2/4/2014?, ?3/4/200)?]
>
>
>
> I am trying to convert this structured array to a 2D array. Is this
> possible where the data types in the 1D array are preserved?

Eke you need to be more careful in the way that you describe your
problem. If you showed an example of the kind of file you're reading
and the *exact* code that you used to read it then we would have a
much better idea what the data structure is.

You use the word "array" in a way that is essentially incorrect in
Python. Python has many things that take the role of what would be
"arrays" in a language like C. The most basic and common ones are
lists and tuples. The data you show above is a list of tuples and not
a "1D array".

You say that you want a 2D array but actually it seems what you really
wanted was a list of lists instead of a list of tuples. If you get
your terminology right then you can easily search for the answers to
questions like this. For example I typed "python convert list of
tuples to list of lists" into google and the top hit was:
http://stackoverflow.com/questions/14831830/convert-a-list-of-tuples-to-a-list-of-lists

On that Stack Overflow page someone has asked the exact question that
you have asked here and been given two different answers. One is the
    list_of_lists = map(list, list_of_tuples)
answer presented by Emile which actually only works for Python 2. In
Python 3 the equivalent is
    list_of_lists = list(map(list, list_of_tuples))
The other answer from Stack Overflow is
    list_of_lists = [list(t) for t in list_of_tuples]
which works under Python 2 or 3.

Please try carefully to learn the correct terminology and it will help
you to ask the questions that you want to ask and to find the answers
that you want.

Finally I assume that what you sent was a small example of the kind of
data that you have. It looks to me as if you have you data transposed
from its natural ordering as a sequence of "records". If I change it
to

[(1, 'a', ?12/12/2009?),
 (2, 'b', ?2/4/2014?),
 (3, 'c', '3/4/200')]

then it looks like a more sensible data structure. This is now a list
of tuples but where each tuple represents a "record" (that's not a
Python type but rather a way of thinking about the data). You can
store a data structure like this in numpy in a 1D record array (the
word array is appropriate in when talking about "numpy arrays").

--
Oscar

From badouglas at gmail.com  Sun Feb 28 22:45:33 2016
From: badouglas at gmail.com (bruce)
Date: Sun, 28 Feb 2016 22:45:33 -0500
Subject: [Tutor] Py/selenium bindings
Message-ID: <CAP16ngqabo7_TtKEH5WMM1Wo=hGPF6yP4A0_4K+9nZ5P++bZ3Q@mail.gmail.com>

Hi.

This might be a bit beyond the group, but I figured no harm/no foul.

I'm looking to access a site that's generated via javascript. The jscript
of the site is invoked via the browser, generating the displayed content.

I'm testing using py/selenium bindings as docs indicate that the
py/selenium/PhantomJS browser (headless) combination should invoke the
jscript, and result in the required content.

However, I can't seem to generte anything other than the initial encrypted
page/content.

Any thoughts/comments would be useful.

The test script is:

#!/usr/bin/python
#-------------------------------------------------------------
#
#    FileName:
#        udel_sel.py
#
#
#-------------------------------------------------------------

#test python script
import subprocess
import re
import libxml2dom
import urllib
import urllib2
import sys, string
import time
import os
import os.path
from hashlib import sha1
from libxml2dom import Node
from libxml2dom import NodeList
import hashlib
import pycurl
import StringIO
import uuid
import simplejson
import copy
from selenium import webdriver



#----------------------------------------

if __name__ == "__main__":
# main app


  url="
http://udel.bncollege.com/webapp/wcs/stores/servlet/TBListView?storeId=37554&termMapping=Y&catalogId=10001&langId=-1&courseXml=%3C%3Fxml+version%3D%221.0%22%3F%3E%3Ctextbookorder%3E%3Cschool+id%3D%22289%22+%2F%3E%3Ccourses%3E%3Ccourse+num%3D%22200%22+dept%3D%22ACCT%22+sect%3D%22010%22+term%3D%222163%22%2F%3E%3C%2Fcourses%3E%3C%2Ftextbookorder%3E
"


  driver = webdriver.PhantomJS()
  driver.get(url)
  xx=driver.page_source

  print xx

  sys.exit()

---------------------------------------------------------------------------

From dyoo at hashcollision.org  Sun Feb 28 23:14:38 2016
From: dyoo at hashcollision.org (Danny Yoo)
Date: Sun, 28 Feb 2016 20:14:38 -0800
Subject: [Tutor] Py/selenium bindings
In-Reply-To: <CAP16ngqabo7_TtKEH5WMM1Wo=hGPF6yP4A0_4K+9nZ5P++bZ3Q@mail.gmail.com>
References: <CAP16ngqabo7_TtKEH5WMM1Wo=hGPF6yP4A0_4K+9nZ5P++bZ3Q@mail.gmail.com>
Message-ID: <CAGZAPF7meGP00W1vscvK1BQ36V=sPN+dC53QrRNUYSpcjBE_fw@mail.gmail.com>

On Sun, Feb 28, 2016 at 7:45 PM, bruce <badouglas at gmail.com> wrote:
> Hi.
>
> This might be a bit beyond the group, but I figured no harm/no foul.
>
> I'm looking to access a site that's generated via javascript. The jscript
> of the site is invoked via the browser, generating the displayed content.


I agree with you on this: it's a bit out of scope for
tutor at python.org.  I don't think we have enough domain knowledge here
to help effectively on this question.

Let me check the Selenium web site to see where you can ask questions
from them... I think you want to talk to the User Group mentioned in:
http://docs.seleniumhq.org/support/.

    https://groups.google.com/forum/#!forum/selenium-users

From cmgcomsol at gmail.com  Mon Feb 29 11:22:49 2016
From: cmgcomsol at gmail.com (CMG Thrissur)
Date: Mon, 29 Feb 2016 21:52:49 +0530
Subject: [Tutor] pyqt __del__ race conditions in subclassed form
Message-ID: <56D47059.6090900@gmail.com>

Hello,

This below given code is giving a race condition, i.e. sometimes gives 
the result as expected, sometimes gives error, sometimes just does 
nothing.  It is part of a subclassed qt form which i am using as main 
dialog.  This behaviour occurs when i close the dialog whereby ending 
the application.  what i want to achieve is that when the dialog is 
closed if any unsaved data, prompt the user.

Thanks

George


___________________________________________________________________________________________________________________________________

def __del__(self):
    if self.Saved==False:
       choice = PyQt4.QtGui.QMessageBox.question(self,'Warning','Data modified. Do you want to save?',
                                                 QMessageBox.Yes | QMessageBox.No, QMessageBox.Yes)
       print(type(choice), choice)
       if choice == QMessageBox.Yes:
          self.SaveData()

___________________________________________________________________________________________________________________________________ 


From don.woolfolk76 at gmail.com  Mon Feb 29 12:31:35 2016
From: don.woolfolk76 at gmail.com (Donald Woolfolk)
Date: Mon, 29 Feb 2016 10:31:35 -0700
Subject: [Tutor] Help with Python code
Message-ID: <01A66B59-5BAE-4B56-8BB7-A878252E5B52@gmail.com>

Hello, 

I am new to Python 3.5 and I am writing code to get data from a session api call and load that data into a csv file or a MySQL database. I am able to make the session call and also to connect to the MySQL db, but I am not able to convert the result (JSON) data. I would like assistance with this if possible.

Thanks,
Don

From soupoffideas at gmail.com  Mon Feb 29 12:15:46 2016
From: soupoffideas at gmail.com (Jack Potrashitel)
Date: Mon, 29 Feb 2016 09:15:46 -0800
Subject: [Tutor] Missing Standard Libraries on Python 3.5 for Mac
In-Reply-To: <CAKJDb-PjBpKbugwfJLc3ip1u77K=52P+jw0dskhU3jzcdo-FGg@mail.gmail.com>
References: <C70FE437-1C4C-4699-A3C7-698840C8B7E2@gmail.com>
 <CAKJDb-PjBpKbugwfJLc3ip1u77K=52P+jw0dskhU3jzcdo-FGg@mail.gmail.com>
Message-ID: <CAJFm=ADkGHuTnyfc7f1fzgCmuu7yYauziyQ795-Jz6hBvFs0pQ@mail.gmail.com>

I use anaconda . Since i dont have peoblems with libraries

???????, 3 ?????? 2015 ?. ???????????? Zachary Ware ???????:

> Hi Andrew,
>
> On Tue, Nov 3, 2015 at 11:20 AM, Andrew Machen
> <andrew.c.machen at gmail.com <javascript:;>> wrote:
> > Hi,
> >
> > I am new to Python, and I am trying to use a Python Standard Library,
> namely ?unicodedata?, however it appears to be missing (others are also
> missing) from the built-in libraries that come with Python 3.5 for Mac.
> >
> > I have searched the following directory, but have not found the library:
> >
> >         /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5
>
> Have you tried importing it?  If importing it works, you can then find
> where it lives by checking the __file__ attribute:
>
>     import unicodedata
>     print(unicodedata.__file__)
>
> > I have tried reinstalling Python 3.5 and also tried Python 3.4.3, and
> neither install the ?unicodedata? library. Is there anyway to install it
> manually?
>
> It should be included with your installation, assuming you're using
> the package from python.org.  If you are unable to import it, please
> raise an issue on the bug tracker at bugs.python.org.
>
> For the record, here's where unicodedata lives on my Mac:
>
> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/
> unicodedata.cpython-35m-darwin.so
>
> Note that unicodedata is implemented in C, so it does not have a .py
> extension.
>
> --
> Zach
> _______________________________________________
> Tutor maillist  -  Tutor at python.org <javascript:;>
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From alan.gauld at btinternet.com  Mon Feb 29 20:07:04 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 1 Mar 2016 01:07:04 +0000
Subject: [Tutor] Help with Python code
In-Reply-To: <01A66B59-5BAE-4B56-8BB7-A878252E5B52@gmail.com>
References: <01A66B59-5BAE-4B56-8BB7-A878252E5B52@gmail.com>
Message-ID: <nb2pvn$thg$1@ger.gmane.org>

On 29/02/16 17:31, Donald Woolfolk wrote:
> Hello, 
> 
> I am new to Python 3.5 and I am writing code to get data 
> from a session api call and load that data into a csv file
> or a MySQL database.

That's great, we can help with that, but...

> I am able to make the session call and also to connect to 
> the MySQL db, but I am not able to convert the result (JSON) data.

We can't read minds. You need to show us your code.
Show us your input and output data. Tell us where you are stuck.
Include full error messages where appropriate. Tell us your
OS and Python version too.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From alan.gauld at btinternet.com  Mon Feb 29 20:12:19 2016
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Tue, 1 Mar 2016 01:12:19 +0000
Subject: [Tutor] pyqt __del__ race conditions in subclassed form
In-Reply-To: <56D47059.6090900@gmail.com>
References: <56D47059.6090900@gmail.com>
Message-ID: <nb2q9i$4ci$1@ger.gmane.org>

On 29/02/16 16:22, CMG Thrissur wrote:
> This below given code is giving a race condition, i.e. sometimes gives 
> the result as expected, sometimes gives error, sometimes just does 
> nothing. 

Don't use the __del__ method for that kind of code, put it in
the close event handler. You don't know when the del() will be
called or what state your dialog object and data will be in.

> def __del__(self):
>     if self.Saved==False:
>        choice = PyQt4.QtGui.QMessageBox.question(self,'Warning','Data modified. Do you want to save?',
>                                                  QMessageBox.Yes | QMessageBox.No, QMessageBox.Yes)
>        print(type(choice), choice)
>        if choice == QMessageBox.Yes:
>           self.SaveData()


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From bfishbein79 at gmail.com  Mon Feb 29 23:32:39 2016
From: bfishbein79 at gmail.com (Benjamin Fishbein)
Date: Mon, 29 Feb 2016 22:32:39 -0600
Subject: [Tutor] saving webpage as webarchive
Message-ID: <1DB0625D-FDC3-4BAC-9154-70D57D31A25C@gmail.com>

This seems like it should be simple, but I can?t find any answer with the docs or google.
I?m on a Mac. OSX Yosemite 10.10.5
I want to save a webpage as a webarchive, and not just get the text.
I hope there?s a way to do it without saving all of the images separately.
And even if I do have to download them separately, then how would I combine everything into the HTM webarchive?

stuff = urllib.urlopen(url).read()
f = open(?file_name?, ?w?)
f.write(stuff)
f.close()

The file_name is just the the text.
I realize that the solution to this will require ?wb? due to the file type.
Other than that, I?m completely stuck.