From steve at pearwood.info  Mon Jun  1 00:09:10 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 01 Jun 2015 14:09:10 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <9ad6e2be-b3bf-4676-b486-82eab504fe64@googlegroups.com>
Message-ID: <556bdae7$0$12977$c3e8da3$5496439d@news.astraweb.com>

On Mon, 1 Jun 2015 10:29 am, Eddilbert Macharia wrote:

> 
> So what im getting from above reponse, everything in python is an object
> because the are instance of the metaclass type and also because they are
> subclasses of the class object ?

No.

Everything in Python is an object because Python has no "primitive"
or "unboxed" types, no machine values. This has nothing to do with
metaclasses. Even if Python had no metaclasses, it could still be true
that "everything is an object".

To summarise:

(1) In Python, "everything is an object" (that is, all values are objects)
is because Python does not include any primitive unboxed values. Anything
which can be used as a value (int, str, float, functions, modules, etc) are
implemented as objects.

(2) Python *also* has metaclasses. This does not follow from the above --
they are independent statements about the language. We could, if we want,
invent new languages:

- one where everything is an object, just like Python, but lacking
metaclasses;

- one which has metaclasses, just like Python, but not everything is an
object;

- and one which *neither* has metaclasses, *nor* everything is an object.

(Java is an example of the third one -- it has something kinda-sorta like
metaclasses, but it's just a tool for reflection, not the real thing.)


If that's enough, you can stop reading. But if you want more detail:


We can characterise various kinds of things in a programming language:

(1) Simple machine types, like bytes, integers, floats. These have no
internal structure, apart from individual bits. These are sometimes
called "machine primitives", and are the sorts of things you work with in
assembly language.

(2) Compound or structured machine types. These are made up of one or more
simple machine type, and include things like C structs and arrays.

(3) Objects, which are a more complex kind of compound type. More
importantly, objects involve a way of thinking about programming: some form
of inheritance, for example. Physically, an object is just a struct. But
it's a struct with a specific kind of meaning to the programming language,
which allows the language to provide inheritance, runtime types, and
various other goodies involved in Object Oriented Programming.

A language can choose to allow any or these, or all of these:

- assembly languages typically only support the most primitive, simple
machine types;

- languages like C and Pascal supports only simple and compound machine
types;

- some languages include both simple machine primitives and objects, such as
Java and Javascript;

- other languages don't offer any direct access to machine primitives, so
all values are objects, for example Python and Ruby.


To give a concrete example, consider a language with an "integer" type. In
C, an integer will be a machine primitive, a single, unstructured chunk of
memory with typically 32 bits, and that's all there is to it. In Python, an
integer will be an object, a large block of *structured* memory, typically
over 100 bits long, and different parts of that structure are used
internally to the interpreter.

A separate issue is whether or not classes are values. In some languages,
classes are *not* values: you cannot pass a class as argument to a
function, or assign it to a variable. In languages like these, classes are
not things (values), so "everything is an object" remains true even though
classes are not objects. Java is an example of that.

In Python, classes *are* variables. So we could have:

"In Python, everything is an object, or a class";

or

"In Python, everything is an object, including classes".


Which is true? The second is true: classes themselves are objects.

Finally, we have one last feature: does the language provide metaclasses?
Python does. Ruby, which is otherwise very similar to Python, does not:

In Python, everything is an object, and we have metaclasses.

In Ruby, everything is an object, but we don't have metaclasses.

http://ruby.dzone.com/news/ruby-doesnt-have-meta-classes



-- 
Steven



From kukki.kanchana at gmail.com  Mon Jun  1 00:09:59 2015
From: kukki.kanchana at gmail.com (Pythonista)
Date: Sun, 31 May 2015 21:09:59 -0700 (PDT)
Subject: Python Paramiko between Linux and Windows Server -- no output
 obtained from Windows for "dir" or "cd" commands
In-Reply-To: <mailman.256.1433121594.5151.python-list@python.org>
References: <b92b07c3-6d2a-4fc9-a780-5803b28dadc4@googlegroups.com>
 <mailman.242.1433074722.5151.python-list@python.org>
 <3a4707d8-46eb-4ba8-a6d9-3689b3c544f4@googlegroups.com>
 <mailman.256.1433121594.5151.python-list@python.org>
Message-ID: <dafa43a5-d722-4181-af6a-af1cc46abb6e@googlegroups.com>

On Sunday, May 31, 2015 at 6:20:19 PM UTC-7, Dennis Lee Bieber wrote:
> On Sun, 31 May 2015 13:12:24 -0700 (PDT), Pythonista
> <kukki.kanchana at gmail.com> declaimed the following:
> 
> >
> >Thanks Peter but I got no output from your suggestion either.!
> 
> 	From Windows viewpoint, I suspect EACH of the commands you send is
> being executed in a totally new process, with no memory of any commands
> sent previously.
> 
> 	As mentioned, "ipconfig" is a separate executable program file, and can
> be run without being tied to a Windows command shell. "dir", "cd", "copy",
> "del", and many similar functions are NOT actual programs but internal
> functions of the Windows command processor.
> 
> 	Given the mixed quoting used in this thread I'm not sure just what was
> tried or not.
> -- 
> 	Wulfraed                 Dennis Lee Bieber         AF6VN
>     wlfraed at ix.netcom.com    HTTP://wlfraed.home.netcom.com/

Hi Peter -

So I was trying "ipconfig" and "ls /path/to/dir" back to back. Apparently, I needed a sleep inbetween the two, I believe. But if "ls" was used individually, it works fine.

Thanks!


From steve at pearwood.info  Mon Jun  1 00:24:24 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 01 Jun 2015 14:24:24 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <c5d52ef0-e5fc-48b6-a30c-846772ff9618@googlegroups.com>
Message-ID: <556bde79$0$12982$c3e8da3$5496439d@news.astraweb.com>

On Mon, 1 Jun 2015 01:09 pm, Eddilbert Macharia wrote:

> Thank you for you responses guys.
> 
> So what I'm getting from above responses, everything in python is an
> object because the are instance of the metaclass type (from Steven
> response) and also because they are subclasses of the class object (from
> Marco and Terry response)? I'm having headache trying to understand the
> cyclic relationship that exit between the metaclass type, the object
> class, and the class type.
> 
> I'm trying to understand how python makes everything an object.is it
> because everything is an instance of the metaclass type or is it because
> everything is a subclass of object class.

Neither. You have the relationship confused.

Everything is an *instance* of the *class*, not the metaclass:

py> isinstance(42, type)  # type is the metaclass
False
py> isinstance(42, int)  # int is the class of 42
True


Most things are *not* subclasses of object:

py> issubclass(42, object)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: issubclass() arg 1 must be a class



> if its because of being subclass of object class, does that mean if the
> class object was named class pyobj. would that make everything python
> pyobj ?

No. "Object" is a standard term from computer science, Python could
rename "object" to "wibble", and we would still say "everything is an
object.

There are *two* (actually three) common meanings for the word "object" in
Python programming:

- the general meaning of object, as in Object Oriented Programming, shared
with hundreds of other languages;

- the specific, Python meaning, as in the built-in class called "object";

- and more rarely, as a synonym for "instance", as opposed to "class".


"Everything is an object" is understood in terms of Object Oriented
Programming. It was true way back in Python version 1.5, before the
built-in "object" class existed. It would still be true even if the
built-in "object" class was renamed to "wibble".

This:

py> isinstance(23, object)
True

is true because "object" is also the base class for everything in Python.
But that's a separate meaning.



[...]
> Is it safe to say that python creates the class object first using the
> type metaclass (I'm simplifying the metaclass for brevity).

No. type and object are special, and have to be bootstrapped into existence
by the interpreter. You cannot create type first, because type is an
object, so object needs to exist first. But you cannot create object first,
because object is a type, and so type has to exist.

You cannot solve this problem by creating one first, then the other. The
interpreter has to create them both.

[..]
> will i ever get this or is it a chicken and egg situation ?

Yes, exactly. You cannot break the cycle from Python code, it has to be done
deep inside the interpreter.




-- 
Steven



From larry at hastings.org  Mon Jun  1 00:37:59 2015
From: larry at hastings.org (Larry Hastings)
Date: Sun, 31 May 2015 21:37:59 -0700
Subject: [RELEASED] Python 3.5.0b2 is now available
Message-ID: <556BE1A7.1090101@hastings.org>



On behalf of the Python development community and the Python 3.5 release 
team, I'm relieved to announce the availability of Python 3.5.0b2.  
Python 3.5.0b1 had a major regression (see 
http://bugs.python.org/issue24285 for more information) and as such was 
not suitable for testing Python 3.5. Therefore we've made this extra 
beta release, only a week later.  Anyone trying Python 3.5.0b1 should 
switch immediately to testing with Python 3.5.0b2.

Python 3.5 has now entered "feature freeze".  By default new features 
may no longer be added to Python 3.5.  (However, there are a handful of 
features that weren't quite ready for Python 3.5.0 beta 2; these were 
granted exceptions to the freeze, and are scheduled to be added before 
beta 3.)

This is a preview release, and its use is not recommended for production 
settings.

Three important notes for Windows users about Python 3.5.0b2:

  * If installing Python 3.5.0b2 as a non-privileged user, you may need
    to escalate to administrator privileges to install an update to your
    C runtime libraries.
  * There is now a third type of Windows build for Python 3.5.  In
    addition to the conventional installer and the web-based installer,
    Python 3.5 now has an embeddable release designed to be deployed as
    part of a larger application's installer for apps using or extending
    Python.  During the 3.5 alpha releases, this was an executable
    installer; as of 3.5.0 beta 1 the embeddable build of Python is now
    shipped in a zip file.


You can find Python 3.5.0b2 here:

    https://www.python.org/downloads/release/python-350b2/

Happy hacking,


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150531/74487c9e/attachment.html>

From rxjwg98 at gmail.com  Mon Jun  1 00:46:31 2015
From: rxjwg98 at gmail.com (fl)
Date: Sun, 31 May 2015 21:46:31 -0700 (PDT)
Subject: Where is 'palindrome' defined?
Message-ID: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>

Hi,

When I search solution of reverse a string/number, I came across a short
function online:

>>> def palindrome(num):
	return str(num) == str(num)[::-1]

I thought that it is a general function. And with the following variable:

>>> a
'1234_5555'

>>> parlindrome(a)

Traceback (most recent call last):
  File "<pyshell#126>", line 1, in <module>
    parlindrome(a)
NameError: name 'parlindrome' is not defined


Then, I find that parlindrome is a special checking mirrored word.
I use Python 2.7.9. Why does the error message show   

name 'parlindrome' is not defined



Thanks,


From ethan at stoneleaf.us  Mon Jun  1 01:08:06 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sun, 31 May 2015 22:08:06 -0700
Subject: Using Python instead of Bash
In-Reply-To: <87h9qsnckl.fsf@Equus.decebal.nl>
References: <87h9qsnckl.fsf@Equus.decebal.nl>
Message-ID: <556BE8B6.4040408@stoneleaf.us>

Just for funsies, I did a slight rewrite using my scription[1] library:

--8<------------------------------------------------------------------------------
#!/usr/bin/python3
"""
Convert scanned images: contrast is increased, size is enlarged,
format is changed from jpeg to png.
"""

from __future__ import print_function   # only needed if running in 2
from glob import glob
from scription import *

@Command(
         threshold=Spec('cutoff value for white vs black', OPTION, type=int, default=66),
         verify=Spec('confirm conversion by viewing on screen', FLAG),
         )
def image_convert(verify, threshold):
     for count, input in enumerate(sorted(glob('*.JPG')), start=1):
         output = '{0:02d}.png'.format(count)
         # only prints if --verbose on commandline
         print('Going to convert {} to {}'.format(input, output))
         while True:
             # Execute automatically parses into a list
             attempt = Execute(
                     'convert -threshold {}% {} {}'.format(threshold, input, output),
                     interactive='echo',
                     )
             if attempt.returncode:
                 raise SystemExit(attempt.returncode)
             if not verify:
                 break
             attempt = Execute(
                     'display_the_png {}'.format(output),
                     interactive='echo',
                     )
             if attempt.returncode:
                 raise SystemExit(attempt.returncode)
             if get_response('Does the image look good?'):
                 break
             threshold = get_response("New threshold value:", type=int)
         # conversion looks good, or manual verification skipped
         print('going to print {0}'.format(output))
         attempt = Execute(
                 'lpr -o fit-to-page -o media=A4 {}'.format(output),
                 interactive='echo',
                 )
         if attempt.returncode:
             raise SystemExit(attempt.returncode)

Main()
--8<------------------------------------------------------------------------------

Opinions about the usability of the above script, both as the script writer and as the user, welcomed.

--
~Ethan~


[1] yes, everyone apparently writes their own command-line processor -- this one is mine.  ;)


From rxjwg98 at gmail.com  Mon Jun  1 01:15:21 2015
From: rxjwg98 at gmail.com (fl)
Date: Sun, 31 May 2015 22:15:21 -0700 (PDT)
Subject: What use for reversed()?
In-Reply-To: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>
References: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>
Message-ID: <b1d480da-13e2-4c03-b989-54245b092fdf@googlegroups.com>

On Sunday, May 31, 2015 at 12:59:47 PM UTC-7, Denis McMahon wrote:
> On Sun, 31 May 2015 12:40:19 -0700, fl wrote:
> reversed returns an iterator, not a list, so it returns the reversed list 
> of elements one at a time. You can use list() or create a list from 
> reversed and then join the result:
> 
> $ python
> Python 2.7.3 (default, Dec 18 2014, 19:10:20) 
> [GCC 4.6.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> "".join(list(reversed("fred")))
> 'derf'
> >>> "".join([x for x in reversed("fred")])
> 'derf'
> 
> So reversed can do it, but needs a little help
> 
> -- 
> Denis McMahon,

I follow your reply with these trials:

>>>>list_r=(reversed("fred"))
>>> list(list_r)
['d', 'e', 'r', 'f']
>>> list_r
<reversed object at 0x02B57F10>


I have searched about list, but I still don't know what list_r is.
It looks like an index in other language. 
What else can it be used besides list(list_r)?
I want to show list_r content. This is possibly an illegal question.

Thanks, 



From jpiitula at ling.helsinki.fi  Mon Jun  1 02:11:14 2015
From: jpiitula at ling.helsinki.fi (Jussi Piitulainen)
Date: Mon, 01 Jun 2015 09:11:14 +0300
Subject: What use for reversed()?
References: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>
 <b1d480da-13e2-4c03-b989-54245b092fdf@googlegroups.com>
Message-ID: <lf5bnh0ot4t.fsf@ling.helsinki.fi>

fl writes:

> On Sunday, May 31, 2015 at 12:59:47 PM UTC-7, Denis McMahon wrote:
>> On Sun, 31 May 2015 12:40:19 -0700, fl wrote:
>> reversed returns an iterator, not a list, so it returns the reversed
>> list of elements one at a time. You can use list() or create a list
>> from reversed and then join the result:

[snip]

> I follow your reply with these trials:
>
>  >>>>list_r=(reversed("fred"))
>  >>> list(list_r)
>  ['d', 'e', 'r', 'f']
>  >>> list_r
>  <reversed object at 0x02B57F10>
>
> I have searched about list, but I still don't know what list_r is.

It's an object that will produce elements on demand. The doc string
(Python 2.7.6) calls it an iterator: "reversed(sequence) -> reverse
iterator over values of the sequence".

Such objects "have state". Try to consume it *twice*, it'll be empty the
second time:

  >>> listr = reversed('fred')
  >>> listr
  <reversed object at 0xb70abc2c>
  >>> list(listr)
  ['d', 'e', 'r', 'f']
  >>> listr
  <reversed object at 0xb70abc2c>
  >>> list(listr)
  []

You can ask for the next element of the iterator:

  >>> listr = reversed('fred')
  >>> next(listr)
  'd'
  >>> list(listr)
  ['e', 'r', 'f']


> What else can it be used besides list(list_r)?

Piecemeal walking using next(list_r), as the source of elements in a for
loop, as a sequence argument to many functions that only need to walk it
once (but they will consume it!).

The suggested ''.join(reversed('fred')) is an example.


> I want to show list_r content. This is possibly an illegal question.

You could do this:

  >>> from __future__ import print_function
  >>> listr = reversed('fred')
  >>> for c in listr: print(c, end = '')
  ... else: print()
  ... 
  derf

But then listr won't have that content, or any content, any more. The
act of looking inside has changed listr :)

Such objects are quite powerful when used properly. The sequence they
produce can be larger than available memory because it need only exist
one element at a time.


From gherron at digipen.edu  Mon Jun  1 02:55:20 2015
From: gherron at digipen.edu (Gary Herron)
Date: Sun, 31 May 2015 23:55:20 -0700
Subject: Where is 'palindrome' defined?
In-Reply-To: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
Message-ID: <556C01D8.1070302@digipen.edu>

On 05/31/2015 09:46 PM, fl wrote:
> Hi,
>
> When I search solution of reverse a string/number, I came across a short
> function online:
>
>>>> def palindrome(num):
> 	return str(num) == str(num)[::-1]
>
> I thought that it is a general function. And with the following variable:

No, this  function is not built into Python because  ...  Well it's hard 
to say why.  It's not very general, or not useful to many programmers, 
or nobody's thought about it or made a case for including it in Python, etc.

But that should be no problem.  You can define it yourself (by entering 
the two line you have above).  Then it will be defined, and calling
     parlindrome('...')
will produce a result rather than an error.

Gary Herron


>
>>>> a
> '1234_5555'
>
>>>> parlindrome(a)
> Traceback (most recent call last):
>    File "<pyshell#126>", line 1, in <module>
>      parlindrome(a)
> NameError: name 'parlindrome' is not defined
>
>
> Then, I find that parlindrome is a special checking mirrored word.
> I use Python 2.7.9. Why does the error message show
>
> name 'parlindrome' is not defined
>
>
>
> Thanks,


-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418



From orgnut at yahoo.com  Mon Jun  1 02:56:52 2015
From: orgnut at yahoo.com (Larry Hudson)
Date: Sun, 31 May 2015 23:56:52 -0700
Subject: Where is 'palindrome' defined?
In-Reply-To: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
Message-ID: <7tOdnco9nqion_HInZ2dnUU7-dOdnZ2d@giganews.com>

On 05/31/2015 09:46 PM, fl wrote:
> Hi,
>
> When I search solution of reverse a string/number, I came across a short
> function online:
>
>>>> def palindrome(num):
> 	return str(num) == str(num)[::-1]
>
> I thought that it is a general function. And with the following variable:
>
>>>> a
> '1234_5555'
>
>>>> parlindrome(a)
>
> Traceback (most recent call last):
>    File "<pyshell#126>", line 1, in <module>
>      parlindrome(a)
> NameError: name 'parlindrome' is not defined
>
>
> Then, I find that parlindrome is a special checking mirrored word.
> I use Python 2.7.9. Why does the error message show
>
> name 'parlindrome' is not defined
>
>
>
> Thanks,
>
Don't be embarrassed, everybody does this:  You're blind to your own typos...

You define palindrome() then call parlindrome() -- with an extra 'r'.

      -=- Larry -=-



From dpalao.python at gmail.com  Mon Jun  1 02:58:10 2015
From: dpalao.python at gmail.com (David Palao)
Date: Mon, 1 Jun 2015 08:58:10 +0200
Subject: Where is 'palindrome' defined?
In-Reply-To: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
Message-ID: <CAKUKWzkNakR92Rhbx5LE31RaxAnsYxCiszzyiWAgVnoW3AjYwg@mail.gmail.com>

Hi,
Because "palindrome" != "parlindrome"?
Have you read the error message? Did you try to understand it?

Best

2015-06-01 6:46 GMT+02:00 fl <rxjwg98 at gmail.com>:
> Hi,
>
> When I search solution of reverse a string/number, I came across a short
> function online:
>
>>>> def palindrome(num):
>         return str(num) == str(num)[::-1]
>
> I thought that it is a general function. And with the following variable:
>
>>>> a
> '1234_5555'
>
>>>> parlindrome(a)
>
> Traceback (most recent call last):
>   File "<pyshell#126>", line 1, in <module>
>     parlindrome(a)
> NameError: name 'parlindrome' is not defined
>
>
> Then, I find that parlindrome is a special checking mirrored word.
> I use Python 2.7.9. Why does the error message show
>
> name 'parlindrome' is not defined
>
>
>
> Thanks,
> --
> https://mail.python.org/mailman/listinfo/python-list


From omar.aboumrad at gmail.com  Mon Jun  1 03:00:33 2015
From: omar.aboumrad at gmail.com (Omar Abou Mrad)
Date: Mon, 1 Jun 2015 10:00:33 +0300
Subject: Where is 'palindrome' defined?
In-Reply-To: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
Message-ID: <CAFwtXp0iaBX8NzD8SQ+4=u7BB5jUUYgwNE497A5BmnE2yWDj=Q@mail.gmail.com>

On Mon, Jun 1, 2015 at 7:46 AM, fl <rxjwg98 at gmail.com> wrote:

> Hi,
>
> When I search solution of reverse a string/number, I came across a short
> function online:
>
> >>> def palindrome(num):
>         return str(num) == str(num)[::-1]
>
> I thought that it is a general function. And with the following variable:
>
> >>> a
> '1234_5555'
>
> >>> parlindrome(a)
>
> Traceback (most recent call last):
>   File "<pyshell#126>", line 1, in <module>
>     parlindrome(a)
> NameError: name 'parlindrome' is not defined
>
> <snip>


Further to the mentioned, be careful with your spelling, the function name
as you've shown is "palindrome" but you're invoking it using "parlindrome".

Regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150601/c264c6b2/attachment.html>

From lac at openend.se  Mon Jun  1 03:11:43 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 09:11:43 +0200
Subject: Using Python instead of Bash
In-Reply-To: Message from Dennis Lee Bieber <wlfraed@ix.netcom.com> of "Sun,
 31 May 2015 20:53:49 -0400." <jbanma99kskmqmmt13ah3eusp744aigu9s@4ax.com>
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <mailman.247.1433084763.5151.python-list@python.org>
 <87382cn0bj.fsf@Equus.decebal.nl><jbanma99kskmqmmt13ah3eusp744aigu9s@4ax.com>
Message-ID: <201506010711.t517Bhtb014621@fido.openend.se>

On Sun, 31 May 2015 19:06:40 +0200, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>
>>By the way: because of the lpr my script only works on Linux (and
>>probably Unix and Apple). Can I rewrite it in such a way that it also
>>works with Windows? Someone else has to test it then, because I do not
>>have a Windows computer.

You may find it easier to have the windows computers learn how to use
lpr.  see: http://www.morovia.com/kb/Setting-LPR-Printing-Windows-78-10007.html
(or there are lots of guides out there.)

Laura


From rxjwg98 at gmail.com  Mon Jun  1 03:50:30 2015
From: rxjwg98 at gmail.com (fl)
Date: Mon, 1 Jun 2015 00:50:30 -0700 (PDT)
Subject: A simple print cannot run in Python Shell
Message-ID: <caa1cc9e-2f9b-423d-9298-a511180d15f7@googlegroups.com>

Hi,

When I try the following (They are saved in a main.py file)

#!/usr/bin/python
print r'C:\\nowhere'


It works as the tutorial, i.e. it echoes in a Windows 7
command console:


C:\\nowhere








When I run the following command in a Python 2.7.9 Shell on Windows 7,


print r'C:\\nowhere'



It has error:

>>> print r'C:\\nowhere'
SyntaxError: invalid syntax




What is the problem? Why does it behave different at .py file
and Python Shell?


Thanks, 


From orgnut at yahoo.com  Mon Jun  1 03:56:36 2015
From: orgnut at yahoo.com (Larry Hudson)
Date: Mon, 01 Jun 2015 00:56:36 -0700
Subject: Using Python instead of Bash
In-Reply-To: <87h9qsnckl.fsf@Equus.decebal.nl>
References: <87h9qsnckl.fsf@Equus.decebal.nl>
Message-ID: <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>

On 05/31/2015 05:42 AM, Cecil Westerhof wrote:
> I help someone that has problems reading. For this I take photo's of
> text, use convert from ImageMagick to make a good contrast (original
> paper is grey) and use lpr to print it a little bigger.
>
I''m wondering why you bother to take a photo, which then has to be adjusted for quality.  A 
screen-capture program is much easier and immediately gives you a perfect(?) starting image.

Linux Mint that I use has the program 'gnome-screenshot' (in the main menu under Accessories as 
'Screenshot').  It gives you the options to capture the whole screen, the current active window, 
or any arbitrary rectangular area of the screen.  It saves it as a .png and allows you to 
specify the path/filename to save it.  I'm sure there are many other equivalent programs 
available as well, and this one or others are likely available by default in other Linux 
distros.  Or easily installed if not.

Of course, this just gives you the original 'raw' image that you can then process further as 
necessary.  But this is _FAR_ easier and better quality than the round-about method of using a 
camera.

      -=- Larry -=-



From lac at openend.se  Mon Jun  1 03:58:27 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 09:58:27 +0200
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: Message from Chris Angelico <rosuav@gmail.com> of "Mon,
 01 Jun 2015 13:43:31 +1000."
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com><CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
Message-ID: <201506010758.t517wR9T015894@fido.openend.se>

If you are giving a talk about Decimal -- and trying to stamp out the
inappropriate use of floats you have to first inform people that
what they learned as 'decimals' as children was not floating point,
despite the fact that we write them the same way.

If I ever get the time machine, I am going back in time and demand that
floating point numbers be expressed as 12345:678 instead of 12345.678
because it would save so much trouble.  Never has the adage 'It's not
what you don't know, that bites you.  It's what you know that ain't so.'
been more apt.

I have done much better in speaking about this topic to a bunch of
incredulous people if you next explain that scientists really don't
care about accuracy in their calculations.  (This will surprise them).
Most scentific calculations have some real world measurement in them,
and for most real world measurements, if you are getting even 5 digits
of precision, you are doing really, really, well.  This means that
scientists are going to be throwing away all the extra digits they get
out a a floating point representation, so they don't have to care how
accurate they are.  As long as their results are good in the first 5,
it won't matter.  (Depending on time constraints, a review of
significant figures -- what they are and what they mean is good here.)

It is really hard to get the concept of Decimal across to people who
already have that concept in their mind, but think it is called Float.
You have to first teach them that they don't know anything about Float
and get them to reboot their brains before you can install this new
knowledge.  Otherwise their brains will just overwrite your new knowledge
with 'ah, just use a Float' as soon as you stop speaking.  Same day,
even.

Laura



From jpiitula at ling.helsinki.fi  Mon Jun  1 03:59:29 2015
From: jpiitula at ling.helsinki.fi (Jussi Piitulainen)
Date: Mon, 01 Jun 2015 10:59:29 +0300
Subject: A simple print cannot run in Python Shell
References: <caa1cc9e-2f9b-423d-9298-a511180d15f7@googlegroups.com>
Message-ID: <lf5382bq2ou.fsf@ling.helsinki.fi>

fl writes:

> When I run the following command in a Python 2.7.9 Shell on Windows 7,
>
> print r'C:\\nowhere'
>
> It has error:
>
>>>> print r'C:\\nowhere'
> SyntaxError: invalid syntax
>
> What is the problem? Why does it behave different at .py file
> and Python Shell?

Have you executed the following in that shell?

   from __future__ import print_function

That replaces the Python 2 print command with the Python 3 print
function and then you would get that error message.

Apologies if you've done that because I used it in one of my examples
about reversed(). I didn't mean to confuse - I've just never quite
learnt the Python 2 print syntax myself.

There is no "from __past__ import print_command", is there? To recover
the original functionality in that shell :)


From jpiitula at ling.helsinki.fi  Mon Jun  1 04:06:33 2015
From: jpiitula at ling.helsinki.fi (Jussi Piitulainen)
Date: Mon, 01 Jun 2015 11:06:33 +0300
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>
Message-ID: <lf5y4k3onsm.fsf@ling.helsinki.fi>

Larry Hudson writes:

> On 05/31/2015 05:42 AM, Cecil Westerhof wrote:
>> I help someone that has problems reading. For this I take photo's of
>> text, use convert from ImageMagick to make a good contrast (original
>> paper is grey) and use lpr to print it a little bigger.
>>
> I''m wondering why you bother to take a photo, which then has to be
> adjusted for quality.  A screen-capture program is much easier and
> immediately gives you a perfect(?) starting image.

"paper"


From steve+comp.lang.python at pearwood.info  Mon Jun  1 04:30:26 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 01 Jun 2015 18:30:26 +1000
Subject: A simple print cannot run in Python Shell
References: <caa1cc9e-2f9b-423d-9298-a511180d15f7@googlegroups.com>
Message-ID: <556c1826$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Monday 01 June 2015 17:50, fl wrote:

> Hi,
> 
> When I try the following (They are saved in a main.py file)
> 
> #!/usr/bin/python
> print r'C:\\nowhere'

Aside: Don't use raw strings for file names. They aren't intended for file 
names, and while they will *usually* work, some day you will try something 
like this:

    path = r'C:\\path\ending\with\slash\'

and then you will discover that raw strings aren't for file names.

So just get used to using forward slashes.



> It works as the tutorial, i.e. it echoes in a Windows 7
> command console:
> 
> 
> C:\\nowhere


How do you run it in Windows 7? The hashbang line

    #!/usr/bin/python

is for Linux and Unix, and won't work on Windows. So you must be doing 
something to execute the file. What is that?


> When I run the following command in a Python 2.7.9 Shell on Windows 7,
>
> print r'C:\\nowhere'
> 
> 
> 
> It has error:
> 
>>>> print r'C:\\nowhere'
> SyntaxError: invalid syntax

In the future, please copy and paste the *complete* traceback, as that may 
include more information. Syntax errors usually include a line pointing to 
the location of the error, or just past it:

py> print foo
  File "<stdin>", line 1
    prin foo
           ^
SyntaxError: invalid syntax


Notice the line with the ^ caret?



> What is the problem? Why does it behave different at .py file
> and Python Shell?

Are you absolutely sure the Python Shell is running Python 2.7? It looks 
like you are running Python 3. What does this display in the shell?

import sys
sys.version


Otherwise you must have run this at some point:

from __future__ import print_function


If you exit the shell, and start it up again, that should reset import.

If not, then there is a possibility that your version of Python is 
configured to do the __future__ import. Try this:

print
print_function

What do you see? If you get an error, copy and paste the *entire* traceback.


-- 
Steve



From mail at timgolden.me.uk  Mon Jun  1 04:46:50 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Mon, 01 Jun 2015 09:46:50 +0100
Subject: A simple print cannot run in Python Shell
In-Reply-To: <556c1826$0$13004$c3e8da3$5496439d@news.astraweb.com>
References: <caa1cc9e-2f9b-423d-9298-a511180d15f7@googlegroups.com>
 <556c1826$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <556C1BFA.6090809@timgolden.me.uk>

On 01/06/2015 09:30, Steven D'Aprano wrote:
> How do you run it in Windows 7? The hashbang line
> 
>     #!/usr/bin/python
> 
> is for Linux and Unix, and won't work on Windows. So you must be doing 
> something to execute the file. What is that?

Actually, it will for anywhere with a recent Python 3.x or where the
standalone launcher has been installed:

https://www.python.org/dev/peps/pep-0397/
(cf particularly the section "Python Script Launching")

Of course, I don't know if the OP does in fact have
the launcher installed, but if they do then the answer is: py.exe.

Even if the launcher isn't installed, Windows will happily run a .py
file via the file extension association, assuming that this has been
selected when installing Python (which it is, by default, ISTR).

TJG


From alister.nospam.ware at ntlworld.com  Mon Jun  1 05:07:16 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Mon, 1 Jun 2015 09:07:16 +0000 (UTC)
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>
 <lf5y4k3onsm.fsf@ling.helsinki.fi>
Message-ID: <mkh7c3$itb$1@speranza.aioe.org>

On Mon, 01 Jun 2015 11:06:33 +0300, Jussi Piitulainen wrote:

> Larry Hudson writes:
> 
>> On 05/31/2015 05:42 AM, Cecil Westerhof wrote:
>>> I help someone that has problems reading. For this I take photo's of
>>> text, use convert from ImageMagick to make a good contrast (original
>>> paper is grey) and use lpr to print it a little bigger.
>>>
>> I''m wondering why you bother to take a photo, which then has to be
>> adjusted for quality.  A screen-capture program is much easier and
>> immediately gives you a perfect(?) starting image.
> 
> "paper"

Have you looked at using OCR software combined with a scanner?
I have used tesseract in the past with very god results.


-- 
Eloquence is logic on fire.


From Cecil at decebal.nl  Mon Jun  1 05:11:27 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 01 Jun 2015 11:11:27 +0200
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>
Message-ID: <87r3pvlrnk.fsf@Equus.decebal.nl>

Op Monday 1 Jun 2015 09:56 CEST schreef Larry Hudson:

> On 05/31/2015 05:42 AM, Cecil Westerhof wrote:
>> I help someone that has problems reading. For this I take photo's
>> of text, use convert from ImageMagick to make a good contrast
>> (original paper is grey) and use lpr to print it a little bigger.
>>
> I''m wondering why you bother to take a photo, which then has to be
> adjusted for quality.  A screen-capture program is much easier and
> immediately gives you a perfect(?) starting image.

I am amazed about the technical progress: I did not know it was
possible to take a screen-shot from a booklet.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Mon Jun  1 05:20:17 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 01 Jun 2015 11:20:17 +0200
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
 <87vbf88k0n.fsf@elektro.pacujo.net>
Message-ID: <87mw0jlr8u.fsf@Equus.decebal.nl>

Op Monday 1 Jun 2015 00:22 CEST schreef Marko Rauhamaa:

> Cecil Westerhof <Cecil at decebal.nl>:
>
>> At the moment I have the following code:
>> os.chdir(directory)
>> for document in documents:
>> subprocess.Popen(['evince', document])
>>
>> With this I can open several documents at once. But there is no way
>> to know when those documents are going to be closed. This
>> could/will lead to zombie processes. (I run it on Linux.) What is
>> the best solution to circumvent this?
>>
>> I was thinking about putting all Popen instances in a list. And
>> then every five minutes walk through the list and check with poll
>> if the process has terminated. If it has it can be released from
>> the list. Of-course I need to synchronise those events. Is that a
>> good way to do it?
>
> If you don't care to know when child processes exit, you can simply
> ignore the SIGCHLD signal:
>
> import signal
> signal.signal(signal.SIGCHLD, signal.SIG_IGN)
>
> That will prevent zombies from appearing.

In this case I do not care. I just do not want to create zombie
processes.

It works. What I find kind of strange because
    https://docs.python.org/2/library/signal.html
says:
    signal.SIG_DFL

        This is one of two standard signal handling options; it will
        simply perform the default function for the signal. For
        example, on most systems the default action for SIGQUIT is to
        dump core and exit, while the default action for SIGCHLD is to
        simply ignore it.

    signal.SIG_IGN

        This is another standard signal handler, which will simply
        ignore the given signal.

As it looks it does not matter in this case, because when a process
terminates it will still communicate its exit status to Popen. But
what if I want for certain Popen signals SIG_IGN and others SIG_DFL.
How should I do that?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From marko at pacujo.net  Mon Jun  1 05:21:45 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 12:21:45 +0300
Subject: Where is 'palindrome' defined?
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
 <mailman.267.1433141892.5151.python-list@python.org>
Message-ID: <87h9qraimu.fsf@elektro.pacujo.net>

David Palao <dpalao.python at gmail.com>:

> Because "palindrome" != "parlindrome"?
> Have you read the error message? Did you try to understand it?

When you are starting with any new thing, even the simplest problems
look baffling. Once you have achieved a few successes, such error
messages start to make sense.

>>>>> def palindrome(num):
>>         return str(num) == str(num)[::-1]

BTW, this simplistic function is not very practical for serious,
real-life palindromes. It does work for the na?ve Finnish palindrome:

    saippuakauppias
    (= "a soap merchant")

However, it fails to detect:

    innostunut sonni
    (= "an enthusiastic bull")

let alone:

    Sota-apina nakataan raastimeen.
          Apelle pane emit.
      Saarnaa takanani paatos.

    (= "The war monkey will be chucked into a grater.
        Hand the pistils to father-in-law.
        Pathos does preach behind me.")


Marko


From Cecil at decebal.nl  Mon Jun  1 05:23:40 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 01 Jun 2015 11:23:40 +0200
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
 <mailman.258.1433125553.5151.python-list@python.org>
Message-ID: <87iob7lr37.fsf@Equus.decebal.nl>

Op Monday 1 Jun 2015 03:03 CEST schreef Cameron Simpson:

> On 31May2015 23:33, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> At the moment I have the following code:
>> os.chdir(directory)
>> for document in documents:
>> subprocess.Popen(['evince', document])
>>
>> With this I can open several documents at once. But there is no way
>> to know when those documents are going to be closed. This
>> could/will lead to zombie processes. (I run it on Linux.) What is
>> the best solution to circumvent this?
>
> The standard trick is to make the process a grandchild instead of a
> child.  Fork, kick off subprocess, exit (the forked child).
>
> But provided you will collect the children eventually then zombies
> are only untidy, not very resource wasteful. They are essentially
> just slots in the process table left around so that exit status can
> be collected; the resources associated with the full process
> (memory, open file, etc) have already been freed.

I do not like untidy. ;-)

But Marko already gave the solution:
    import signal
    signal.signal(signal.SIGCHLD, signal.SIG_IGN)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Mon Jun  1 05:25:05 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 01 Jun 2015 11:25:05 +0200
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <mailman.247.1433084763.5151.python-list@python.org>
 <87382cn0bj.fsf@Equus.decebal.nl>
 <mailman.257.1433121594.5151.python-list@python.org>
Message-ID: <87eglvlr0u.fsf@Equus.decebal.nl>

Op Monday 1 Jun 2015 02:53 CEST schreef Dennis Lee Bieber:

> On Sun, 31 May 2015 19:06:40 +0200, Cecil Westerhof
> <Cecil at decebal.nl> declaimed the following:
>
>>
>> By the way: because of the lpr my script only works on Linux (and
>> probably Unix and Apple). Can I rewrite it in such a way that it
>> also works with Windows? Someone else has to test it then, because
>> I do not have a Windows computer.
>
> 	Not easily... Either you install something like cygwin which MAY
> include LPR, or...
>
> 1) You invoke (instead of LPR) some application that accepts a
> command line argument telling it to do a print (I'm pretty sure that
> long ago M$ Word accepted a command line /p to print, but it seems
> that has gone away replaced with something uglier -- notepad /p
> <file> works for simple text files... I managed to get Acrobat
> Reader to open a PDF and bring up a print requester, but that path
> still required the user to hit a number of dialog windows)
>
> 2) You incorporate win32 system calls to manage a print job in all
> details (create bitmap, provide bitmap to print driver, etc.)

Well, at the moment I will not bother about it then.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From marko at pacujo.net  Mon Jun  1 05:36:21 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 12:36:21 +0300
Subject: What is considered an "advanced" topic in Python?
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
Message-ID: <87d21fahyi.fsf@elektro.pacujo.net>

Laura Creighton <lac at openend.se>:

> It is really hard to get the concept of Decimal across to people who
> already have that concept in their mind, but think it is called Float.

I can't remember running into this frustration.

However, I *constantly* run into engineers who don't understand what
zero means.


Marko


From marko at pacujo.net  Mon Jun  1 05:38:48 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 12:38:48 +0300
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>
 <87r3pvlrnk.fsf@Equus.decebal.nl>
Message-ID: <878uc3ahuf.fsf@elektro.pacujo.net>

Cecil Westerhof <Cecil at decebal.nl>:

> I am amazed about the technical progress: I did not know it was
> possible to take a screen-shot from a booklet.

Well, take a look at this video that demonstrates the possibilities of
the technology:

   https://www.youtube.com/watch?v=MOXQo7nURs0


Marko


From dpalao.python at gmail.com  Mon Jun  1 05:41:49 2015
From: dpalao.python at gmail.com (David Palao)
Date: Mon, 1 Jun 2015 11:41:49 +0200
Subject: Where is 'palindrome' defined?
In-Reply-To: <87h9qraimu.fsf@elektro.pacujo.net>
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
 <mailman.267.1433141892.5151.python-list@python.org>
 <87h9qraimu.fsf@elektro.pacujo.net>
Message-ID: <CAKUKWz=cj6yyHDrN_SRdBOX=Ear37HDGTFbOWqd1gOAFC+_-UQ@mail.gmail.com>

2015-06-01 11:21 GMT+02:00 Marko Rauhamaa <marko at pacujo.net>:
> David Palao <dpalao.python at gmail.com>:
>
>> Because "palindrome" != "parlindrome"?
>> Have you read the error message? Did you try to understand it?
>
> When you are starting with any new thing, even the simplest problems
> look baffling. Once you have achieved a few successes, such error
> messages start to make sense.
>

I didn't intend to be harsh. Sorry if I failed.
Anyway, for me the errors provided by python are mostly very useful.
When I forget to read them, or I'm simply too impatient to try to
understand them, I use to get very hard debugging sessions. The
contrary is often true.
What I tried to emphasize is the importance of paying careful
attention to the error messages. It is a good habit to start doing
this from the beginning.

Best


From rosuav at gmail.com  Mon Jun  1 05:45:31 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 1 Jun 2015 19:45:31 +1000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <201506010758.t517wR9T015894@fido.openend.se>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <rosuav@gmail.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se>
Message-ID: <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>

On Mon, Jun 1, 2015 at 5:58 PM, Laura Creighton <lac at openend.se> wrote:
> If you are giving a talk about Decimal -- and trying to stamp out the
> inappropriate use of floats you have to first inform people that
> what they learned as 'decimals' as children was not floating point,
> despite the fact that we write them the same way.
>
> If I ever get the time machine, I am going back in time and demand that
> floating point numbers be expressed as 12345:678 instead of 12345.678
> because it would save so much trouble.  Never has the adage 'It's not
> what you don't know, that bites you.  It's what you know that ain't so.'
> been more apt.

While I agree that there are problems with conflating float with "real
number", I don't know that decimal.Decimal is actually going to solve
that either; and using a colon as the decimal separator won't solve
anything (the world already has two - "." in the US and "," in Europe
- and adding a third with the same semantics of separating the
greater-than-unit from the sub-unit digits won't instantly give new
meaning to the way numbers are handled), so I would be whacking you
over the head when you invent that time machine XKCD 716 style.

decimal.Decimal() has its own peculiarities, ways in which it isn't
the same as real numbers, so what's really needed is a talk about both
of them (and fractions.Fraction for completeness) and how they all
have their uses. People need to grok that what computers do with
numbers is never quite what they're used to from grade school
(although it can be close; bignum integer arithmetic is pretty
reliable).

ChrisA


From rosuav at gmail.com  Mon Jun  1 06:04:32 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 1 Jun 2015 20:04:32 +1000
Subject: Best way to prevent zombie processes
In-Reply-To: <87mw0jlr8u.fsf@Equus.decebal.nl>
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
 <87vbf88k0n.fsf@elektro.pacujo.net>
 <87mw0jlr8u.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmpT9ZXvmo7xgSpOT+LJsgwbV+EC0NNr4RYSTbZ8ftBuXQ@mail.gmail.com>

On Mon, Jun 1, 2015 at 7:20 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> But
> what if I want for certain Popen signals SIG_IGN and others SIG_DFL.
> How should I do that?

You can't. A signal is a signal; you can't specify default handling
for some and not others. The only way is to actually handle them, and
then you can decide what to do on a per-process basis.

ChrisA


From marko at pacujo.net  Mon Jun  1 06:21:43 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 13:21:43 +0300
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
 <87vbf88k0n.fsf@elektro.pacujo.net> <87mw0jlr8u.fsf@Equus.decebal.nl>
 <mailman.277.1433153080.5151.python-list@python.org>
Message-ID: <87zj4j91ag.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> On Mon, Jun 1, 2015 at 7:20 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> But
>> what if I want for certain Popen signals SIG_IGN and others SIG_DFL.
>> How should I do that?
>
> You can't. A signal is a signal; you can't specify default handling
> for some and not others. The only way is to actually handle them, and
> then you can decide what to do on a per-process basis.

Signals are a very crude, old-school UNIX system programming concept.

One of their worst problems is their global scope. You can't
compartmentalize signals through ordinary means of encapsulation
(libraries, classes).

Linux is addressing this particular issue "as we speak:"

   This patch series introduces a new clone flag, CLONE_FD, which lets
   the caller handle child process exit notification via a file
   descriptor rather than SIGCHLD.

   <URL: https://lwn.net/Articles/636646/>

I guess it will be a while before this facility will be available to
Python programmers. For example, signalfd and eventfd were introduced in
2007 and 2010, respectively, but still aren't supported by Python.


Marko


From marko at pacujo.net  Mon Jun  1 06:25:21 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 13:25:21 +0300
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
 <87vbf88k0n.fsf@elektro.pacujo.net> <87mw0jlr8u.fsf@Equus.decebal.nl>
Message-ID: <87vbf7914e.fsf@elektro.pacujo.net>

Cecil Westerhof <Cecil at decebal.nl>:

> It works. What I find kind of strange because
>     https://docs.python.org/2/library/signal.html

You should not rely on Python documentation on Linux system specifics.
The wait(2) manual page states:

    POSIX.1-2001 specifies that if the disposition of SIGCHLD is set to
    SIG_IGN or the SA_NOCLDWAIT flag is set for SIGCHLD (see
    sigaction(2)), then children that terminate do not become zombies
    and a call to wait() or waitpid() will block until all children have
    terminated, and then fail with errno set to ECHILD. (The original
    POSIX standard left the behavior of setting SIGCHLD to SIG_IGN
    unspecified. Note that even though the default disposition of
    SIGCHLD is "ignore", explicitly setting the disposition to SIG_IGN
    results in different treatment of zombie process children.)


Marko


From lac at openend.se  Mon Jun  1 06:28:11 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 12:28:11 +0200
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: Message from Chris Angelico <rosuav@gmail.com> of "Mon,
 01 Jun 2015 19:45:31 +1000."
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <rosuav@gmail.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se><CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
Message-ID: <201506011028.t51ASBVG019928@fido.openend.se>

In a message of Mon, 01 Jun 2015 19:45:31 +1000, Chris Angelico writes:
>On Mon, Jun 1, 2015 at 5:58 PM, Laura Creighton <lac at openend.se> wrote:
>> If you are giving a talk about Decimal -- and trying to stamp out the
>> inappropriate use of floats you have to first inform people that
>> what they learned as 'decimals' as children was not floating point,
>> despite the fact that we write them the same way.
>>
>> If I ever get the time machine, I am going back in time and demand that
>> floating point numbers be expressed as 12345:678 instead of 12345.678
>> because it would save so much trouble.  Never has the adage 'It's not
>> what you don't know, that bites you.  It's what you know that ain't so.'
>> been more apt.
>
>While I agree that there are problems with conflating float with "real
>number", I don't know that decimal.Decimal is actually going to solve
>that either; and using a colon as the decimal separator won't solve
>anything (the world already has two - "." in the US and "," in Europe
>- and adding a third with the same semantics of separating the
>greater-than-unit from the sub-unit digits won't instantly give new
>meaning to the way numbers are handled), so I would be whacking you
>over the head when you invent that time machine XKCD 716 style.

You have missed my point.  What I want is for floats never to be
represented as '.' or ',' notation.  That way, when each naive
user writes his or her first program that deals with money, when
they look at their computer manual they will come to the section on
floating point numbers and they will all look like something they
have never seen before.  So they will read the section carefully
to see if this is what they want or need, and the section can nicely
say NEVER USE THIS FOR MONEY and they will know they are in the wrong
place.

The problem is that they take one look at a floating point number,
and think they know what it is from childhood, and use it.  They
don't know any better.  The notion that computer languages have
homophones, and just as the moles you have on your face and the
moles that burrow in your garden and the moles that you make out
of stone as a pier or a breakwater, numbers can work this way in
computers is deep, deep, deep in the 'this cannot be possible'
mindset of people who are learning to use computers.

I argued that when Decimal went into Python it should be called Money
(with an alias for Decimal for those who aren't using it for money) again
so that the people who don't know any better are more likely to end
up in the right place, but I lost that argument, too.

Laura



From breamoreboy at yahoo.co.uk  Mon Jun  1 06:34:08 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 01 Jun 2015 11:34:08 +0100
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <201506010758.t517wR9T015894@fido.openend.se>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com><CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se>
Message-ID: <mkhcf2$q9p$1@ger.gmane.org>

On 01/06/2015 08:58, Laura Creighton wrote:
> If you are giving a talk about Decimal -- and trying to stamp out the
> inappropriate use of floats you have to first inform people that
> what they learned as 'decimals' as children was not floating point,
> despite the fact that we write them the same way.
>
> If I ever get the time machine, I am going back in time and demand that
> floating point numbers be expressed as 12345:678 instead of 12345.678
> because it would save so much trouble.  Never has the adage 'It's not
> what you don't know, that bites you.  It's what you know that ain't so.'
> been more apt.
>
> I have done much better in speaking about this topic to a bunch of
> incredulous people if you next explain that scientists really don't
> care about accuracy in their calculations.  (This will surprise them).
> Most scentific calculations have some real world measurement in them,
> and for most real world measurements, if you are getting even 5 digits
> of precision, you are doing really, really, well.  This means that
> scientists are going to be throwing away all the extra digits they get
> out a a floating point representation, so they don't have to care how
> accurate they are.  As long as their results are good in the first 5,
> it won't matter.  (Depending on time constraints, a review of
> significant figures -- what they are and what they mean is good here.)
>
> It is really hard to get the concept of Decimal across to people who
> already have that concept in their mind, but think it is called Float.
> You have to first teach them that they don't know anything about Float
> and get them to reboot their brains before you can install this new
> knowledge.  Otherwise their brains will just overwrite your new knowledge
> with 'ah, just use a Float' as soon as you stop speaking.  Same day,
> even.
>
> Laura
>

In the wonderful world of numbers I believe that things are looking up. 
  I don't recall a new issue on the bug tracker for several months along 
the lines of "Python can't do arithmetic properly".

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Mon Jun  1 06:36:14 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 1 Jun 2015 20:36:14 +1000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <201506011028.t51ASBVG019928@fido.openend.se>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
Message-ID: <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>

On Mon, Jun 1, 2015 at 8:28 PM, Laura Creighton <lac at openend.se> wrote:
> You have missed my point.  What I want is for floats never to be
> represented as '.' or ',' notation.  That way, when each naive
> user writes his or her first program that deals with money, when
> they look at their computer manual they will come to the section on
> floating point numbers and they will all look like something they
> have never seen before.  So they will read the section carefully
> to see if this is what they want or need, and the section can nicely
> say NEVER USE THIS FOR MONEY and they will know they are in the wrong
> place.

The problem isn't the decimal separator, though, because floats can
have problems even without it (and can have no problems with a decimal
separator). If you want to distinguish "computer numbers" from "real
numbers", you'd do better to pick a different set of symbols for them
- or at least a different numerical base. If all literals were written
in octal, people would understand that there's something special going
on here. But would that really help?

ChrisA


From DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk  Mon Jun  1 06:44:22 2015
From: DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk (Dave Farrance)
Date: Mon, 01 Jun 2015 11:44:22 +0100
Subject: What is considered an "advanced" topic in Python?
References: <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com><CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <rosuav@gmail.com> of "Mon, 01 Jun 2015 13:43:31 +1000."
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
Message-ID: <dkdoma1th5idudfbglrj5me4f3bjtur2ed@4ax.com>

Laura Creighton <lac at openend.se> wrote:

>If you are giving a talk about Decimal -- and trying to stamp out the
>inappropriate use of floats you have to first inform people that
>what they learned as 'decimals' as children was not floating point,
>despite the fact that we write them the same way. ...

To be fair, prior to electronic computers, they were essentially
synonymous.  It's a bit like finance experts chiding people for mixing
"cost" with "price".

>>> Decimal('0.3')
Decimal('0.3')
>>> Decimal(0.3)
Decimal('0.299999999999999988897769753748434595763683319091796875')


From Cecil at decebal.nl  Mon Jun  1 06:44:34 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 01 Jun 2015 12:44:34 +0200
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>
 <lf5y4k3onsm.fsf@ling.helsinki.fi> <mkh7c3$itb$1@speranza.aioe.org>
Message-ID: <876177lncd.fsf@Equus.decebal.nl>

Op Monday 1 Jun 2015 11:07 CEST schreef alister:

> On Mon, 01 Jun 2015 11:06:33 +0300, Jussi Piitulainen wrote:
>
>> Larry Hudson writes:
>>
>>> On 05/31/2015 05:42 AM, Cecil Westerhof wrote:
>>>> I help someone that has problems reading. For this I take photo's
>>>> of text, use convert from ImageMagick to make a good contrast
>>>> (original paper is grey) and use lpr to print it a little bigger.
>>>>
>>> I''m wondering why you bother to take a photo, which then has to
>>> be adjusted for quality. A screen-capture program is much easier
>>> and immediately gives you a perfect(?) starting image.
>>
>> "paper"
>
> Have you looked at using OCR software combined with a scanner?
> I have used tesseract in the past with very god results.

Does not work because it is not just straight text, there are headings
with text floating around it.

Also the funny thing is that I first scanned it. But it gave several
problems. One of them was that it is a format between A4 and A5.
Taking pictures is faster and gives better results.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From marko at pacujo.net  Mon Jun  1 06:54:57 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 13:54:57 +0300
Subject: What is considered an "advanced" topic in Python?
References: <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <rosuav@gmail.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
 <dkdoma1th5idudfbglrj5me4f3bjtur2ed@4ax.com>
Message-ID: <87oakz8zr2.fsf@elektro.pacujo.net>

Dave Farrance <DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk>:

>>>> Decimal('0.3')
> Decimal('0.3')
>>>> Decimal(0.3)
> Decimal('0.299999999999999988897769753748434595763683319091796875')

At first I thought I had never dealt with money in Python nor made use
of Decimal. Then I remembered I *did* deal with money in one program and
wondered how I handled it. Sure enough:

        newdistr[clubacct] = decimal.Decimal("%.2f" % amount)


Marko


From lac at openend.se  Mon Jun  1 07:00:09 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 13:00:09 +0200
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: Message from Mark Lawrence <breamoreboy@yahoo.co.uk>
 of "Mon, 01 Jun 2015 11:34:08 +0100." <mkhcf2$q9p$1@ger.gmane.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com><CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se><mkhcf2$q9p$1@ger.gmane.org>
Message-ID: <201506011100.t51B09VN020803@fido.openend.se>

In a message of Mon, 01 Jun 2015 11:34:08 +0100, Mark Lawrence writes:

>In the wonderful world of numbers I believe that things are looking up. 
>  I don't recall a new issue on the bug tracker for several months along 
>the lines of "Python can't do arithmetic properly".

How Great to Hear!  <cheer cheer cheer>

But I wonder how much of that has to do with changing division?

OpenEnd (my company) makes bookkeeping systems for Swedish Ideal
Societies (closest English words 'non-profits' and 'clubs').  A
surprising number of our potential customers are relying on code
written more than 20 years ago, by somebody who is no longer part of
the Society which used floats for arithmetic.  Hard to make the sale
when the customer thinks your math is broken. So I am biased, but in
my little corner of the world, Float Confusion Reigns Supreme.

Laura



From lac at openend.se  Mon Jun  1 07:24:52 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 13:24:52 +0200
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: Message from Chris Angelico <rosuav@gmail.com> of "Mon,
 01 Jun 2015 20:36:14 +1000."
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se><CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
Message-ID: <201506011124.t51BOqO2021385@fido.openend.se>

In a message of Mon, 01 Jun 2015 20:36:14 +1000, Chris Angelico writes:
>The problem isn't the decimal separator, though, because floats can
>have problems even without it (and can have no problems with a decimal
>separator). If you want to distinguish "computer numbers" from "real
>numbers", you'd do better to pick a different set of symbols for them
>- or at least a different numerical base. If all literals were written
>in octal, people would understand that there's something special going
>on here. But would that really help?

Maybe _your_ brain needs some resetting, too. :)  You know too much so
have lost the grasp of how the world looks to the new programmer.

Problem:
I want to use a computer to add up a whole lot of money amounts so I can
figure out how much money to send some place.

This is an absolute, dead simple first computer program newbies write.
For a lot of people, this used to be the whole reason they bought a
computer in the first place.  Now they just want to do the same stuff
on their phone, which they have anyway.

And they immediately grab floating point numbers, and since adding
a whole lot of them in range of 'prices for stuff at the supermarket'
is one of the best ways to guarantee your will not get the correct,
exact amount you want for your answer, they don't get it.

You can pick any representation you want for float, as long as it
_isn't_ the same one as we use for money, and a whole lot of problems
will go away.  Because the problems are in the users' heads, and
no place else, and the problem is 'This looks familiar.  I will use it
an expect it to behave like I am used to.'  All human beings do this
all the time, so the way to prevent the problem is to make it look
less familiar.  But way back in time, you know.  Von Neumann recommended
against floating-point numbers for the 1951 IAS machine, arguing that
fixed-point arithmetic is preferable.  I agree, but, if John von Neumann
couldn't win that argument, then there is no way on earth I could.
So, if floating point is going in, at least we should represent them
differently.

But I am a bad arguer.

When incompatibilites were going into Python 3.0 I wanted

y = 1.3  to give you a decimal, not a float.

If you wanted a float you would have to write y = 1.3f or something.

I lost that one too.  I still think it would be great.

But, hell, I write accounting and bookkeeping systems.  Your milage
may vary. :)

Laura



From python at mrabarnett.plus.com  Mon Jun  1 07:48:42 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 01 Jun 2015 12:48:42 +0100
Subject: [RELEASED] Python 3.5.0b2 is now available
In-Reply-To: <556BE1A7.1090101@hastings.org>
References: <556BE1A7.1090101@hastings.org>
Message-ID: <556C469A.2070806@mrabarnett.plus.com>

On 2015-06-01 05:37, Larry Hastings wrote:
>
>
> On behalf of the Python development community and the Python 3.5 release
> team, I'm relieved to announce the availability of Python 3.5.0b2.
> Python 3.5.0b1 had a major regression (see
> http://bugs.python.org/issue24285 for more information) and as such was
> not suitable for testing Python 3.5. Therefore we've made this extra
> beta release, only a week later.  Anyone trying Python 3.5.0b1 should
> switch immediately to testing with Python 3.5.0b2.
>
> Python 3.5 has now entered "feature freeze".  By default new features
> may no longer be added to Python 3.5.  (However, there are a handful of
> features that weren't quite ready for Python 3.5.0 beta 2; these were
> granted exceptions to the freeze, and are scheduled to be added before
> beta 3.)
>
> This is a preview release, and its use is not recommended for production
> settings.
>
> Three important notes for Windows users about Python 3.5.0b2:
>
>   * If installing Python 3.5.0b2 as a non-privileged user, you may need
>     to escalate to administrator privileges to install an update to your
>     C runtime libraries.
>   * There is now a third type of Windows build for Python 3.5.  In
>     addition to the conventional installer and the web-based installer,
>     Python 3.5 now has an embeddable release designed to be deployed as
>     part of a larger application's installer for apps using or extending
>     Python.  During the 3.5 alpha releases, this was an executable
>     installer; as of 3.5.0 beta 1 the embeddable build of Python is now
>     shipped in a zip file.
>
>
> You can find Python 3.5.0b2 here:
>
>     https://www.python.org/downloads/release/python-350b2/
>
> Happy hacking,
>
>
> //arry/
>
>
I've just run "Windows x86 executable installer" and "Windows x86-64
executable installer".

While installing, the progress messages overwrote each other without
clearing the background each time.


From marko at pacujo.net  Mon Jun  1 07:57:02 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 14:57:02 +0300
Subject: What is considered an "advanced" topic in Python?
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org>
Message-ID: <87iob78wvl.fsf@elektro.pacujo.net>

Laura Creighton <lac at openend.se>:

> Von Neumann recommended against floating-point numbers for the 1951
> IAS machine, arguing that fixed-point arithmetic is preferable. I
> agree, but, if John von Neumann couldn't win that argument, then there
> is no way on earth I could. So, if floating point is going in, at
> least we should represent them differently.

Your problem is not fixed-point vs floating-point but related to the
radix. Base-10 floating-point would be perfect for you and base-2
fixed-point would demonstrate even worse rounding errors.

> But, hell, I write accounting and bookkeeping systems.

Ah, I see.

In 1951, decimal numbers would have done little good in the UK with the
pound divided into 20 shillings and the shilling into 12 pence. Maybe a
"Babylonian" module would have been perfect.


Marko


From edd.cowan at gmail.com  Mon Jun  1 08:03:23 2015
From: edd.cowan at gmail.com (Eddilbert Macharia)
Date: Mon, 1 Jun 2015 05:03:23 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
Message-ID: <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>

Thank you all who have taken there time to assist me.
Special thanks to Steven D'Aparo even though you didn't have two you have taken you time to go into details.

I think i kind of understand now. 

Instead of python having data types like int, string, e.t.c it has two primitive types which are class type and class object which are created by python interpreter during its setup .using this two classes python is able to create some more data types

In some context type means the data type an object is e.g. an int,string e.t.c.

In the new class style, type and class sort of mean the same thing.

**type type is used to create new types when used as a metaclass 
**type object is used to provide some default behavior and attributes to this types through inheritance

The interpreter makes sure of the following 

** The type type inherits from data type object which makes the following statements are true

>>> issubclass(type,object)
True # type inherits object as its parent

>>> isinstance(type,object) 
True # because of the inheritance 

>>> type.__bases__ 
(<class 'object'>,) # because of the inheritance 

>>> isinstance(type,type) 
True # because metaclass type was used to create this data type type

** The type object is created using the type type i.e. type object is an instance of type type. The type object does not subclass i.e. inherit from any other types. 

>>> isinstance(object,type) 
True # because metaclass type was used to create this object

>>> issubclass(object,type)
False # because object does not inherit from any other class

>>> object.__bases__
() # because object does not inherit from any other class



On Sunday, May 31, 2015 at 5:34:20 PM UTC+3, Eddilbert Macharia wrote:
> Hello All ,
> 
> I'm wrecking my head trying to understand. where the class object comes into play . 
> 
> Is it only meant to act as base class and does it mean there is an actual class called object in python which all the objects created using the class type inherit ?
> 
> i'm assuming the metaclass if simplified would look something like this :
> 
> type('dict', (object,),{})
> 
> And when we use the class type as a metaclass are we using the instance version of the class type or are we actually using the type class itself ?
> 
> Also when we say everything is an object in python, are we referring to the fact that everything is an instance of the class type or does it have to with the object class inherited ? 
> 
> As can be attested by using type() function as below :
> 
> >>> type(int)
> <class 'type'>
> >>> type(list)
> <class 'type'>
> >>> type(dict)
> <class 'type'>
> >>> type(type)
> <class 'type'>
> >>> type(object)
> <class 'type'>
> 
> From my understanding this means all of this are instances of the class type. which means the class type was used to create this instances.
> 
> Now if i look at the __bases__ of all this objects i get :
> 
> >>> type.__base__
> <class 'object'>
> >>> type.__bases__
> (<class 'object'>,)
> >>> dict.__bases__
> (<class 'object'>,)
> >>> list.__bases__
> (<class 'object'>,)
> >>> int.__bases__
> (<class 'object'>,)
> >>> object.__bases__
> ()
> 
> This tells me that all of this objects inherit from the class object which has nothing to do with them being instances.



From steve at pearwood.info  Mon Jun  1 08:07:41 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 01 Jun 2015 22:07:41 +1000
Subject: Zero [was Re: What is considered an "advanced" topic in Python?]
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
 <87d21fahyi.fsf@elektro.pacujo.net>
Message-ID: <556c4b0e$0$13003$c3e8da3$5496439d@news.astraweb.com>

On Mon, 1 Jun 2015 07:36 pm, Marko Rauhamaa wrote:

> However, I constantly run into engineers who don't understand what
> zero means.

Okay, I'll bite.

What does zero mean, and how do engineers misunderstand it?



-- 
Steven



From Cecil at decebal.nl  Mon Jun  1 08:16:21 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 01 Jun 2015 14:16:21 +0200
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
Message-ID: <871thvlj3e.fsf@Equus.decebal.nl>

Op Sunday 31 May 2015 23:33 CEST schreef Cecil Westerhof:

> At the moment I have the following code:
> os.chdir(directory)
> for document in documents:
> subprocess.Popen(['evince', document])
>
> With this I can open several documents at once. But there is no way
> to know when those documents are going to be closed. This could/will
> lead to zombie processes. (I run it on Linux.) What is the best
> solution to circumvent this?
>
> I was thinking about putting all Popen instances in a list. And then
> every five minutes walk through the list and check with poll if the
> process has terminated. If it has it can be released from the list.
> Of-course I need to synchronise those events. Is that a good way to
> do it?

With some investigation I decided on something completely different.

I made a class:
    import os
    import subprocess

    from os.path    import expanduser
    from threading  import Thread

    class DocumentsToShow:
        def show_documents(self):
            if self._desktop != -1:
                subprocess.check_call(['wmctrl', '-s', str(self._desktop - 1)])
            os.chdir(self._directory)
            for document in self._documents:
                Popen_without_zombie(['evince', document])

        def __init__(self, name, desktop, directory, documents):
            '''Initialise the class'''

            self._name      = name
            self._desktop   = desktop
            self._directory = expanduser(directory)
            self._documents = documents

And this class uses the following function:
    def Popen_without_zombie(command):
        p = subprocess.Popen(command)
        Thread(target = p.wait).start()

The class takes a name for the collection, which desktop to display it
on, the directory that contains the documents and a list of documents.
With show_documents they are displayed.

How about this way of solving it?


About the improvement of the class. Now it is always evince that is
used to open a document. But it would be nice if it was a little less
picky. What is the preferred way to solve this?
- Defining a set the user can choose from.
- Making it a string parameter and expecting the user to know what he
is doing.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From bc at freeuk.com  Mon Jun  1 08:24:22 2015
From: bc at freeuk.com (BartC)
Date: Mon, 01 Jun 2015 13:24:22 +0100
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mailman.282.1433157907.5151.python-list@python.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se><CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org>
Message-ID: <X9Yax.570951$nQ7.16389@fx42.am4>

On 01/06/2015 12:24, Laura Creighton wrote:

> Maybe _your_ brain needs some resetting, too. :)  You know too much so
> have lost the grasp of how the world looks to the new programmer.

Maybe the new programmer should learn then that binary floating point 
might not perform as they might expect.

I'm sure it wasn't long after I first started programming that I 
discovered the reason why code such as:

  for i:=0.0 until 10.0 step 0.1 ...     # (Algol 60)

might not always work properly.

I don't think using decimal magically solves all problems either:

  a=decimal.Decimal('0.0')
  b=decimal.Decimal('1.0')/decimal.Decimal('3.0')
  c=decimal.Decimal('10.0')

  while a<=c:
	print (a)
	a=a+b

This prints 9.999... as the final value rather than 10.0. And if changed to:

  while a!=c:
      .....

then this doesn't terminate.

> I want to use a computer to add up a whole lot of money amounts so I can
> figure out how much money to send some place.
>
> This is an absolute, dead simple first computer program newbies write.
> For a lot of people, this used to be the whole reason they bought a
> computer in the first place.

How many people who buy phones are actually going to write their own 
programs? Rather than download an app written by people who can code (or 
use the built-in calculator). (How do you even code in Python on a 
smartphone anyway?)

> And they immediately grab floating point numbers, and since adding
> a whole lot of them in range of 'prices for stuff at the supermarket'
> is one of the best ways to guarantee your will not get the correct,
> exact amount you want for your answer, they don't get it.

Also, how many items need to be added up before even 64-bit floating 
point values show a discrepancy, after rounding to whole cents? Or are 
we adding up billions of dollars and expecting a result to the exact cent?

> But I am a bad arguer.
>
> When incompatibilites were going into Python 3.0 I wanted
>
> y = 1.3  to give you a decimal, not a float.

1.3 yielding floating point is fairly universal. Probably there would be 
bigger problems if 1.3 silently gave you a decimal type.

> But, hell, I write accounting and bookkeeping systems.  Your milage
> may vary. :)

You can emulate decimal floating point with binary floating point, if 
the latter has some spare precision. Or an easy fix for the shopping 
problem is to work in whole cents, using either integer or floating 
point (although the latter limits you to some $50 trillion dollars for 
the total, which might be a problem if we ever get hyper-inflation again).

But if the numeric representation is crucial to your application, then 
surely you just create a custom type or class for that? (I think Python 
is quite good at this sort of customising; I lot of Python code I see 
now seems to consist of nothing else!)

-- 
Bartc



From bc at freeuk.com  Mon Jun  1 08:27:01 2015
From: bc at freeuk.com (BartC)
Date: Mon, 01 Jun 2015 13:27:01 +0100
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <87iob78wvl.fsf@elektro.pacujo.net>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>	<mailman.192.1432915802.5151.python-list@python.org>	<28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>	<CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>	<1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>	<CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>	<201506010758.t517wR9T015894@fido.openend.se>
 <rosuav@gmail.com>	<CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>	<201506011028.t51ASBVG019928@fido.openend.se>	<CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>	<mailman.282.1433157907.5151.python-list@python.org>
 <87iob78wvl.fsf@elektro.pacujo.net>
Message-ID: <qcYax.570952$nQ7.374467@fx42.am4>

On 01/06/2015 12:57, Marko Rauhamaa wrote:
> Laura Creighton <lac at openend.se>:
>
>> Von Neumann recommended against floating-point numbers for the 1951
>> IAS machine, arguing that fixed-point arithmetic is preferable. I
>> agree, but, if John von Neumann couldn't win that argument, then there
>> is no way on earth I could. So, if floating point is going in, at
>> least we should represent them differently.
>
> Your problem is not fixed-point vs floating-point but related to the
> radix. Base-10 floating-point would be perfect for you and base-2
> fixed-point would demonstrate even worse rounding errors.
>
>> But, hell, I write accounting and bookkeeping systems.
>
> Ah, I see.
>
> In 1951, decimal numbers would have done little good in the UK with the
> pound divided into 20 shillings and the shilling into 12 pence.

And pence divided into farthings and half-pennies. (With certain 
prestige goods prices in guineas (21 shillings) for good measure.)

-- 
Bartc



From python at mrabarnett.plus.com  Mon Jun  1 08:27:21 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 01 Jun 2015 13:27:21 +0100
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <87iob78wvl.fsf@elektro.pacujo.net>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org>
 <87iob78wvl.fsf@elektro.pacujo.net>
Message-ID: <556C4FA9.40403@mrabarnett.plus.com>

On 2015-06-01 12:57, Marko Rauhamaa wrote:
> Laura Creighton <lac at openend.se>:
>
>> Von Neumann recommended against floating-point numbers for the 1951
>> IAS machine, arguing that fixed-point arithmetic is preferable. I
>> agree, but, if John von Neumann couldn't win that argument, then there
>> is no way on earth I could. So, if floating point is going in, at
>> least we should represent them differently.
>
> Your problem is not fixed-point vs floating-point but related to the
> radix. Base-10 floating-point would be perfect for you and base-2
> fixed-point would demonstrate even worse rounding errors.
>
>> But, hell, I write accounting and bookkeeping systems.
>
> Ah, I see.
>
> In 1951, decimal numbers would have done little good in the UK with the
> pound divided into 20 shillings and the shilling into 12 pence. Maybe a
> "Babylonian" module would have been perfect.
>
Don't forget that there was also a ha'penny (half-penny).



From lac at openend.se  Mon Jun  1 08:52:09 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 14:52:09 +0200
Subject: Zero [was Re: What is considered an "advanced" topic in Python?]
In-Reply-To: Message from "Steven D'Aprano" <steve@pearwood.info> of "Mon,
 01 Jun 2015 22:07:41 +1000."
 <556c4b0e$0$13003$c3e8da3$5496439d@news.astraweb.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
 <87d21fahyi.fsf@elektro.pacujo.net><556c4b0e$0$13003$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <201506011252.t51Cq9uc023478@fido.openend.se>

In a message of Mon, 01 Jun 2015 22:07:41 +1000, "Steven D'Aprano" writes:
>On Mon, 1 Jun 2015 07:36 pm, Marko Rauhamaa wrote:
>
>> However, I constantly run into engineers who don't understand what
>> zero means.
>
>Okay, I'll bite.
>
>What does zero mean, and how do engineers misunderstand it?
>
>
>
>-- 
>Steven

I assume he meant 0 padding for fast fourier transforms, but I could
be wrong about that, too.

Laura



From DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk  Mon Jun  1 08:56:33 2015
From: DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk (Dave Farrance)
Date: Mon, 01 Jun 2015 13:56:33 +0100
Subject: Zero [was Re: What is considered an "advanced" topic in Python?]
References: <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
 <87d21fahyi.fsf@elektro.pacujo.net>
 <556c4b0e$0$13003$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <1jlomal1or023ian8rlaso1u5bnol52lag@4ax.com>

Steven D'Aprano <steve at pearwood.info> wrote:

>On Mon, 1 Jun 2015 07:36 pm, Marko Rauhamaa wrote:
>
>> However, I constantly run into engineers who don't understand what
>> zero means.
>
>Okay, I'll bite.
>
>What does zero mean, and how do engineers misunderstand it?

There are two hard things in computer science: cache invalidation,
naming things, and off-by-one errors.


From Cecil at decebal.nl  Mon Jun  1 09:03:52 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 01 Jun 2015 15:03:52 +0200
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl> <871thvlj3e.fsf@Equus.decebal.nl>
Message-ID: <87wpznk2br.fsf@Equus.decebal.nl>

Op Monday 1 Jun 2015 14:16 CEST schreef Cecil Westerhof:

> Op Sunday 31 May 2015 23:33 CEST schreef Cecil Westerhof:
>
>> At the moment I have the following code:
>> os.chdir(directory)
>> for document in documents:
>> subprocess.Popen(['evince', document])
>>
>> With this I can open several documents at once. But there is no way
>> to know when those documents are going to be closed. This
>> could/will lead to zombie processes. (I run it on Linux.) What is
>> the best solution to circumvent this?
>>
>> I was thinking about putting all Popen instances in a list. And
>> then every five minutes walk through the list and check with poll
>> if the process has terminated. If it has it can be released from
>> the list. Of-course I need to synchronise those events. Is that a
>> good way to do it?
>
> With some investigation I decided on something completely different.
>
> I made a class:
> import os
> import subprocess
>
> from os.path    import expanduser
> from threading  import Thread
>
> class DocumentsToShow:
> def show_documents(self):
> if self._desktop != -1:
> subprocess.check_call(['wmctrl', '-s', str(self._desktop - 1)])
> os.chdir(self._directory)
> for document in self._documents:
> Popen_without_zombie(['evince', document])
>
> def __init__(self, name, desktop, directory, documents):
> '''Initialise the class'''
>
> self._name      = name
> self._desktop   = desktop
> self._directory = expanduser(directory)
> self._documents = documents
>
> And this class uses the following function:
> def Popen_without_zombie(command):
> p = subprocess.Popen(command)
> Thread(target = p.wait).start()
>
> The class takes a name for the collection, which desktop to display
> it on, the directory that contains the documents and a list of
> documents. With show_documents they are displayed.
>
> How about this way of solving it?
>
>
> About the improvement of the class. Now it is always evince that is
> used to open a document. But it would be nice if it was a little
> less picky. What is the preferred way to solve this? - Defining a
> set the user can choose from. - Making it a string parameter and
> expecting the user to know what he is doing.

I let the user give the command to open the documents, but check if
the command exists:
    class DocumentsToShow:
        def show_documents(self):
            if self._desktop != -1:
                subprocess.check_call(['wmctrl', '-s', str(self._desktop - 1)])
            os.chdir(self._directory)
            for document in self._documents:
                Popen_without_zombie([self._command, document])

        def __init__(self, name, desktop, command, directory, documents):
            '''Initialise the class'''

            try:
                subprocess.check_call(['which', command])
            except subprocess.CalledProcessError:
                raise ValueError('Command \'{0}\' is not known'.format(command))

            self._name      = name
            self._desktop   = desktop
            self._command   = command
            self._directory = expanduser(directory)
            self._documents = documents


-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From skip.montanaro at gmail.com  Mon Jun  1 09:14:45 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 1 Jun 2015 08:14:45 -0500
Subject: Zero [was Re: What is considered an "advanced" topic in Python?]
In-Reply-To: <1jlomal1or023ian8rlaso1u5bnol52lag@4ax.com>
References: <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
 <87d21fahyi.fsf@elektro.pacujo.net>
 <556c4b0e$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <1jlomal1or023ian8rlaso1u5bnol52lag@4ax.com>
Message-ID: <CANc-5Uwba3DyMkEvaFLjzcE8+2qGYqTrjFp5DAiR5wPy9TOLbA@mail.gmail.com>

On Mon, Jun 1, 2015 at 7:56 AM, Dave Farrance <
DaveFarrance at omitthisyahooandthis.co.uk> wrote:
> There are two hard things in computer science: cache invalidation, naming
things, and off-by-one errors.

So it's 1 that engineers really don't understand? Just add 1 and you get
the correct number of hard things. <wink>

Skip

P.S., Dave, your "omitthis" and "andthis" kind of sucks for the rest of us.
And I just invalidated your attempts at
obscurity by replying to your correct email address. I suggest you just
omit that stuff going forward. Unfortunately,
I now I have a crap address in my Gmail contacts I have to hunt down and
remove. Maybe you should just install
a decent spam filter <http://www.spambayes.org/> or switch to Gmail, which
has a functioning spam filter (unlike Yahoo...)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150601/80e20074/attachment.html>

From marko at pacujo.net  Mon Jun  1 09:18:17 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 16:18:17 +0300
Subject: Zero [was Re: What is considered an "advanced" topic in Python?]
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
 <87d21fahyi.fsf@elektro.pacujo.net>
 <556c4b0e$0$13003$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87d21f8t46.fsf@elektro.pacujo.net>

Steven D'Aprano <steve at pearwood.info>:

> What does zero mean, and how do engineers misunderstand it?

Off the top of my head:

 * Code that does:

      if elements:
          for element in elements:
              ...

   instead of:

      for elements in elements:
          ...

 * C++ insists that all objects have nonzero sizes. This program:

   ====================================================================
   #include <stdio.h>
   struct S {};
   int main()
   {
       printf("%zd\n", sizeof(struct S));
       return 0;
   }
   ====================================================================

   prints out 1 when compiled with c++ but 0 when compiled with cc.

 * In C, it used to be illegal to define a struct without a dummy field
   or a zero-length array. At one point, that created irritating trouble
   for compilers that generated C.

 * malloc(0) is allowed to return NULL as a successful return value
   inviting application programming errors. Calling free(NULL) didn't
   use to be guaranteed to work.

 * Numerous shell commands assign special meaning to zero arguments.
   For example:

      $ ls a b c
      a b c
      $ ls a b
      a b
      $ ls a
      a
      $ ls
      a b c d e f g

      $ ln -s a b c
      # creates two links under c, which must be a directory
      $ ln -s c
      # creates one link under .

 * Wild-card expansion always produces at least one result:

      for src in *.py; do
          # executed even in the absence of a .py file
      done

 * Python insists that a block contain at least one statement even
   though technically, it is not necessary for syntactic reasons. Not a
   biggie but can cause a minor annoyance when commenting out a line.

 * Numerous timer implementations take 0 to mean infinity. For example,
   in Java:

       public final void wait(long timeout)
                throws InterruptedException

       [...] If timeout is zero, however, then real time is not taken
       into consideration and the thread simply waits until notified.

       <URL: https://docs.oracle.com/javase/7/docs/api/java/lan
       g/Object.html#wait%28long%29>

   A nasty surprise for those who'd naively expect zero to mean zero,
   especially if zero is the result of downward rounding.


Marko


From garabik-news-2005-05 at kassiopeia.juls.savba.sk  Mon Jun  1 09:18:49 2015
From: garabik-news-2005-05 at kassiopeia.juls.savba.sk (garabik-news-2005-05 at kassiopeia.juls.savba.sk)
Date: Mon, 1 Jun 2015 13:18:49 +0000 (UTC)
Subject: ANN: polynice 0.8 - a nice(1) like utility for throttling processes
Message-ID: <mailman.287.1433164934.5151.python-announce-list@python.org>

polynice is a nice(1)-like command line utility for unix systems to
throttle long running processes beyond what can be achieved by nice(1),
by repeatedly suspending and resuming the process.

It is written for python3, though there is some python2.6 & 2.7 compatibility.

Author:
Radovan Garab?k

URL:
http://kassiopeia.juls.savba.sk/~garabik/software/polynice.html

License:
GPL (v2)

Notable changes:

 * add '--mode random' parameter, to implement randomized throttling
 * add -n0 to disable throttling
 * python2.6 compatibility

-- 
 -----------------------------------------------------------
| Radovan Garab?k http://kassiopeia.juls.savba.sk/~garabik/ |
| __..--^^^--..__    garabik @ kassiopeia.juls.savba.sk     |
 -----------------------------------------------------------
Antivirus alert: file .signature infected by signature virus.
Hi! I'm a signature virus! Copy me into your signature file to help me spread!


From ben.usenet at bsb.me.uk  Mon Jun  1 09:22:49 2015
From: ben.usenet at bsb.me.uk (Ben Bacarisse)
Date: Mon, 01 Jun 2015 14:22:49 +0100
Subject: What is considered an "advanced" topic in Python?
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <rosuav@gmail.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <mailman.278.1433154504.5151.python-list@python.org>
Message-ID: <87zj4j4l7a.fsf@bsb.me.uk>

Laura Creighton <lac at openend.se> writes:

> In a message of Mon, 01 Jun 2015 19:45:31 +1000, Chris Angelico writes:
>>On Mon, Jun 1, 2015 at 5:58 PM, Laura Creighton <lac at openend.se> wrote:
>>> If you are giving a talk about Decimal -- and trying to stamp out the
>>> inappropriate use of floats you have to first inform people that
>>> what they learned as 'decimals' as children was not floating point,
>>> despite the fact that we write them the same way.

That may be focusing on the wrong aspect because what you learn in
school (at least in the UK) is not so dissimilar to floating point
arithmetic.  The differences between that and machine floating point are
that (a) machines typically use base 2 (so the set of fractions that can
be represented exactly is a surprise); (b) machines use fixed-size
floating point whereas at school the representation, whilst always
finite, is more elastic; and (c) the base you calculate in is the base
you are given the data in and also the base you have to give the result
in.

For example, when a pupil is asked to "calculate 1.5 x 10^3 + 1.01 x
10^2 to two decimal places" what they will do is very similar to what a
machine would do if it had decimal floating point.  The surprises that
come from using typical machine floating point come from two base
conversions and (possibly) from rounding at every stage rather than just
at the end.

<snip>
> You have missed my point.  What I want is for floats never to be
> represented as '.' or ',' notation.  That way, when each naive
> user writes his or her first program that deals with money, when
> they look at their computer manual they will come to the section on
> floating point numbers and they will all look like something they
> have never seen before.

I agree that the key in to learn enough about what is going on.  Maybe
the solution is to teach people fixed-width binary floating point
arithmetic in schools!

> So they will read the section carefully
> to see if this is what they want or need, and the section can nicely
> say NEVER USE THIS FOR MONEY and they will know they are in the wrong
> place.

I know what you mean by this but it still bothers me a bit!  My first
job was writing programs for an economist.  It was all about money but,
being macro economics, was all in floating-point.  (In those days you
could not solve sets of non-linear differential equations in reasonable
time using anything else, but even if you could there would be no
point.)

Even some kinds of accounting can be done using floating point.  You can
use "scaled integers" in floating point to get some advantages over
integer arithmetic.  You might get a wider range (~53 bits rather than
32 say) and you get very simple checks for many overflow conditions.
You can also arrange for a simple check that you've failed to round
correctly at some step -- any non-integer fractional part will usually
indicate this.

I'm not averse to a blanket warning as the simplest way to get the
message across to beginners, but it's slowly becoming a "fact" that
nothing to do with money can be done correctly using floating point.

<snip>
-- 
Ben.


From random832 at fastmail.us  Mon Jun  1 09:32:00 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Mon, 01 Jun 2015 09:32:00 -0400
Subject: Zero [was Re: What is considered an "advanced" topic in Python?]
In-Reply-To: <87d21f8t46.fsf@elektro.pacujo.net>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
 <87d21fahyi.fsf@elektro.pacujo.net>
 <556c4b0e$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <87d21f8t46.fsf@elektro.pacujo.net>
Message-ID: <1433165520.215894.283495561.33FB3BFD@webmail.messagingengine.com>

On Mon, Jun 1, 2015, at 09:18, Marko Rauhamaa wrote:
>       if elements:
>           for element in elements:
>               ...
> 
>    instead of:
> 
>       for elements in elements:

TypeError: 'NoneType' object is not iterable


From marko at pacujo.net  Mon Jun  1 09:32:12 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 16:32:12 +0300
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl> <871thvlj3e.fsf@Equus.decebal.nl>
Message-ID: <8761778sgz.fsf@elektro.pacujo.net>

Cecil Westerhof <Cecil at decebal.nl>:

>         Thread(target = p.wait).start()
>
> [...]
>
> How about this way of solving it?

It works.


Marko


From gheskett at wdtv.com  Mon Jun  1 09:49:35 2015
From: gheskett at wdtv.com (Gene Heskett)
Date: Mon, 1 Jun 2015 09:49:35 -0400
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <201506011124.t51BOqO2021385@fido.openend.se>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <201506011124.t51BOqO2021385@fido.openend.se>
Message-ID: <201506010949.35134.gheskett@wdtv.com>

On Monday 01 June 2015 07:24:52 Laura Creighton wrote:
> In a message of Mon, 01 Jun 2015 20:36:14 +1000, Chris Angelico writes:
> >The problem isn't the decimal separator, though, because floats can
> >have problems even without it (and can have no problems with a
> > decimal separator). If you want to distinguish "computer numbers"
> > from "real numbers", you'd do better to pick a different set of
> > symbols for them - or at least a different numerical base. If all
> > literals were written in octal, people would understand that there's
> > something special going on here. But would that really help?
>
> Maybe _your_ brain needs some resetting, too. :)  You know too much so
> have lost the grasp of how the world looks to the new programmer.
>
> Problem:
> I want to use a computer to add up a whole lot of money amounts so I
> can figure out how much money to send some place.
>
> This is an absolute, dead simple first computer program newbies write.
> For a lot of people, this used to be the whole reason they bought a
> computer in the first place.  Now they just want to do the same stuff
> on their phone, which they have anyway.
>
> And they immediately grab floating point numbers, and since adding
> a whole lot of them in range of 'prices for stuff at the supermarket'
> is one of the best ways to guarantee your will not get the correct,
> exact amount you want for your answer, they don't get it.
>
> You can pick any representation you want for float, as long as it
> _isn't_ the same one as we use for money, and a whole lot of problems
> will go away.  Because the problems are in the users' heads, and
> no place else, and the problem is 'This looks familiar.  I will use it
> an expect it to behave like I am used to.'  All human beings do this
> all the time, so the way to prevent the problem is to make it look
> less familiar.  But way back in time, you know.  Von Neumann
> recommended against floating-point numbers for the 1951 IAS machine,
> arguing that fixed-point arithmetic is preferable.  I agree, but, if
> John von Neumann couldn't win that argument, then there is no way on
> earth I could. So, if floating point is going in, at least we should
> represent them differently.
>
> But I am a bad arguer.
>
> When incompatibilites were going into Python 3.0 I wanted
>
> y = 1.3  to give you a decimal, not a float.
>
> If you wanted a float you would have to write y = 1.3f or something.
>
> I lost that one too.  I still think it would be great.
>
> But, hell, I write accounting and bookkeeping systems.  Your milage
> may vary. :)
>
> Laura

I may have gotten stoned and missed it, as I am not yet that familiar 
with python, which is why I am lurking on this list, hoping to absorb 
some of it by osmosis.

But IMO, any language that does not have the ability to set an fp number 
to a fixed number of digits to the right of the separator regardless of 
which , or . is used, needs one written.

The ability to guarantee that the output of a FIX(2) is zero to at least 
17 significant digits so that a zero coomparison is not non-zero because 
theres a 1 15 digits out in a 2 digit money format, is an absolute 
requirement.

I also write G-Code for cnc machinery, and that is something that is 
missing from common usage in most implementations of RS-274D, so we have 
to specifically set the seed value of the variables we use to 8 or more 
digits to the right of the decimal point.  Typically the machine can do, 
if calibrated correctly, cuts to .00025 inch accuracy, and to have to 
specify to fractions of an angstrom in order to make a while statement 
function as desired can be a gotcha.  I am used to it, so I do it 
automatically now, but it still bites the new bee getting started.

So teach me the error of my thinking.  Surely python has such a function.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>


From rosuav at gmail.com  Mon Jun  1 09:52:45 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 1 Jun 2015 23:52:45 +1000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <201506011124.t51BOqO2021385@fido.openend.se>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <201506011124.t51BOqO2021385@fido.openend.se>
Message-ID: <CAPTjJmqi48pFBYFj2yWOe-gNvjNSSnhYL1umj1Hr4ri6LdBPoQ@mail.gmail.com>

On Mon, Jun 1, 2015 at 9:24 PM, Laura Creighton <lac at openend.se> wrote:
> In a message of Mon, 01 Jun 2015 20:36:14 +1000, Chris Angelico writes:
>>The problem isn't the decimal separator, though, because floats can
>>have problems even without it (and can have no problems with a decimal
>>separator). If you want to distinguish "computer numbers" from "real
>>numbers", you'd do better to pick a different set of symbols for them
>>- or at least a different numerical base. If all literals were written
>>in octal, people would understand that there's something special going
>>on here. But would that really help?
>
> Maybe _your_ brain needs some resetting, too. :)  You know too much so
> have lost the grasp of how the world looks to the new programmer.
>
> Problem:
> I want to use a computer to add up a whole lot of money amounts so I can
> figure out how much money to send some place.
>
> And they immediately grab floating point numbers...

That part I do understand; floating-point is the one obvious way to do
these sorts of things. I'm fairly sure we agree that a novice *will*,
for better or for worse, stumble upon floats - in pretty much any
language.

The question is, what part of float is wrong? I don't think the
problem is with the decimal separator, and therefore laying them out
as "whole part, colon, fractional part" won't necessarily help. And it
can potentially hurt, by teaching people that computers are just
weird, and you have to speak a weird language to talk to them -
without understanding that there's a real difference behind this. It's
like the eternal debate about assignment and whether "x = x + 1" is
nonsense, with advocates preferring "x := x + 1" as being somehow
fundamentally different. It isn't. It's just a notational change, and
not even a huge one. (Though I do see the line of argument that it
should be "x <- x + 1" or something else that looks like an arrow.)

ChrisA


From Cecil at decebal.nl  Mon Jun  1 10:37:07 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 01 Jun 2015 16:37:07 +0200
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
 <871thvlj3e.fsf@Equus.decebal.nl> <8761778sgz.fsf@elektro.pacujo.net>
Message-ID: <87siabjy0c.fsf@Equus.decebal.nl>

Op Monday 1 Jun 2015 15:32 CEST schreef Marko Rauhamaa:

> Cecil Westerhof <Cecil at decebal.nl>:
>
>> Thread(target = p.wait).start()
>>
>> [...]
>>
>> How about this way of solving it?
>
> It works.

That I knew: I tested it before I posted it. What I mend is this
better, worse, or the same as working with signal. In my eye it is
much better, but maybe there is a reason that would make my line of
thinking wrong.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From df at see.replyto.invalid  Mon Jun  1 10:39:45 2015
From: df at see.replyto.invalid (Dave Farrance)
Date: Mon, 01 Jun 2015 15:39:45 +0100
Subject: Zero [was Re: What is considered an "advanced" topic in Python?]
References: <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
 <87d21fahyi.fsf@elektro.pacujo.net>
 <556c4b0e$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <1jlomal1or023ian8rlaso1u5bnol52lag@4ax.com>
 <mailman.286.1433164487.5151.python-list@python.org>
Message-ID: <t0qomalg8ev0rt6350l30ovhnpkr79aak1@4ax.com>

Skip Montanaro <skip.montanaro at gmail.com> wrote:

>P.S., Dave, your "omitthis" and "andthis" kind of sucks for the rest of us.
>And I just invalidated your attempts at
>obscurity by replying to your correct email address. I suggest you just
>omit that stuff going forward. Unfortunately,
>I now I have a crap address in my Gmail contacts I have to hunt down and
>remove. Maybe you should just install
>a decent spam filter <http://www.spambayes.org/> or switch to Gmail, which
>has a functioning spam filter (unlike Yahoo...)

I've had that as my Usenet address since 2000 when Usenet was the prime
spam harvesting source and the spammers were actively using
anti-address-munging techniques.  That's no longer so much of a problem,
given Usenet's obscurity for today's average Internet user, but I'm
still wary of posting email addresses directly to Usenet.  I've changed
the "from" address to end with ".invalid" which is the RFC way, and
should be stopped at the Email client, if that helps.

Since this is "comp.lang.python", I'm assuming that it should be treated
primarily as a Usenet group in which you'd normally reply to the group
-- rather than by personal email as you might do if this was primarily a
Listserv group.  I dunno what the group thinks about that.  A quick
header check tells me that the posters here are split about 50/50
between posting via Usenet or the list server.
  


From invalid at invalid.invalid  Mon Jun  1 10:51:42 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Mon, 1 Jun 2015 14:51:42 +0000 (UTC)
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
Message-ID: <mkhrhu$9d5$1@reader1.panix.com>

On 2015-05-31, Cecil Westerhof <Cecil at decebal.nl> wrote:

> I help someone that has problems reading. For this I take photo's of
> text, use convert from ImageMagick to make a good contrast (original
> paper is grey) and use lpr to print it a little bigger.
>
> Normally I would implement this in Bash, but I thought it a good idea
> to implement it in Python.

Why?  Is it difficult to do/maintain in Bash?

If you want to write Python, then you should write Python.  You're
still writing Bash, so you should probably do it in Bash.  If you just
want to invoke a set of external programs on a set of files, then Bash
is probably the right language to use: that's pretty much what Bash is
for: manipulating files by invoking other programs.

> This is my first try:
>     import glob
>     import subprocess
>
>     treshold = 66
>     count = 0
>     for input in sorted(glob.glob('*.JPG')):
>         count += 1
>         output = '{0:02d}.png'.format(count)
>         print('Going to convert {0} to {1}'.format(input, output))
>         p = subprocess.Popen(['convert', '-threshold', '{0}%'.format(treshold), input, output])
>         p.wait()
>         print('Going to print {0}'.format(output))
>         p = subprocess.Popen(['lpr', '-o', 'fit-to-page', '-o', 'media=A4', output])
>         p.wait()
>
> There have to be some improvements: display before printing,
> possibility to change threshold, ? But is this a good start, or
> should I do it differently?

If all the work is being done by making a series of calls to 
subprocess, then you should think about using bash instead.

If you really do want to do this in Python, you can use a Python
binding to the ImageMagick libraries:

  https://wiki.python.org/moin/ImageMagick
  http://www.imagemagick.org/download/python/
  http://stackoverflow.com/questions/7895278/can-i-access-imagemagick-api-with-python

Or maybe you can use Pillow:

  http://pillow.readthedocs.org/
  https://github.com/python-pillow/Pillow  
  https://python-pillow.github.io/
  
-- 
Grant Edwards               grant.b.edwards        Yow! Of course, you
                                  at               UNDERSTAND about the PLAIDS
                              gmail.com            in the SPIN CYCLE --


From invalid at invalid.invalid  Mon Jun  1 10:59:30 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Mon, 1 Jun 2015 14:59:30 +0000 (UTC)
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
 <87vbf88k0n.fsf@elektro.pacujo.net>
Message-ID: <mkhs0i$9d5$2@reader1.panix.com>

On 2015-05-31, Marko Rauhamaa <marko at pacujo.net> wrote:
> Cecil Westerhof <Cecil at decebal.nl>:
>
>> At the moment I have the following code:
>>     os.chdir(directory)
>>     for document in documents:
>>         subprocess.Popen(['evince', document])
>>
>> With this I can open several documents at once. But there is no way to
>> know when those documents are going to be closed. This could/will lead
>> to zombie processes. (I run it on Linux.) What is the best solution to
>> circumvent this?
>>
>> I was thinking about putting all Popen instances in a list. And then
>> every five minutes walk through the list and check with poll if the
>> process has terminated. If it has it can be released from the list.
>> Of-course I need to synchronise those events. Is that a good way to do
>> it?
>
> If you don't care to know when child processes exit, you can simply
> ignore the SIGCHLD signal:
>
>     import signal
>     signal.signal(signal.SIGCHLD, signal.SIG_IGN)
>
> That will prevent zombies from appearing.

Bravo!  I've been writing Unix apps for 30 years, and I did not know
that.  Is this something recent[1], or have I somehow managed to avoid
this useful bit of info for that long?

[1] "Recent" of course being rather subjective and highly
     age-dependent.

-- 
Grant Edwards               grant.b.edwards        Yow! How's it going in
                                  at               those MODULAR LOVE UNITS??
                              gmail.com            


From invalid at invalid.invalid  Mon Jun  1 11:02:11 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Mon, 1 Jun 2015 15:02:11 +0000 (UTC)
Subject: Where is 'palindrome' defined?
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
 <mailman.266.1433141733.5151.python-list@python.org>
Message-ID: <mkhs5j$9d5$3@reader1.panix.com>

On 2015-06-01, Gary Herron <gherron at digipen.edu> wrote:
> On 05/31/2015 09:46 PM, fl wrote:
>> Hi,
>>
>> When I search solution of reverse a string/number, I came across a short
>> function online:
>>
>>>>> def palindrome(num):
>> 	return str(num) == str(num)[::-1]
>>
>> I thought that it is a general function. And with the following variable:
>
> No, this  function is not built into Python because...

Because writing a palindrome predicate is something that is only ever
done as a homework exercise in an introduction to programming class.
If it were part of the standard library, Python would not be
appropriate for such a class.

-- 
Grant Edwards               grant.b.edwards        Yow! If Robert Di Niro
                                  at               assassinates Walter Slezak,
                              gmail.com            will Jodie Foster marry
                                                   Bonzo??


From lac at openend.se  Mon Jun  1 11:07:18 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 17:07:18 +0200
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: Message from Marko Rauhamaa <marko@pacujo.net>
 of "Mon, 01 Jun 2015 14:57:02 +0300." <87iob78wvl.fsf@elektro.pacujo.net>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org><87iob78wvl.fsf@elektro.pacujo.net>
Message-ID: <201506011507.t51F7IBl026868@fido.openend.se>

In a message of Mon, 01 Jun 2015 14:57:02 +0300, Marko Rauhamaa writes:
>In 1951, decimal numbers would have done little good in the UK with the
>pound divided into 20 shillings and the shilling into 12 pence. Maybe a
>"Babylonian" module would have been perfect.
>
>
>Marko

You are being facetious, but in point of fact, naive Brits who knew nothing
of neither accounting systems nor floating point for the most part got
things right when they bought their Sinclair computer in the early
1980s.

This is because their natural tendancy was to calculate all the pounds
separately, and then the shillings, separately, and then the pence.
(With Guineas and other odd stuff thrown in, when the needed them).
This meant that they kept 3+ legers at one time, and then, when they
were done calculating as one final step converted what they had into
its representation where you never had more than 100 pence or 12 shillings.
Thus, entirely by accident, they did their accounting in integers, not
decimals at all.

And this is, of course, the first thing that people who write real
systems that add money learn -- convert everything to pennies (or
whatever you call them) and do all your calculations in pennies, and
then as the final step express that in dollars and cents, or euros
and cents, or what have you.

The Brits still got in trouble when they needed to calculate things for their
4.2 per cent morgage, or decided to keep a running total of the sales
tax they were paying, but they at least did not grab the floating
number representation as the first thing off the shelf when they needed
money.

Laura


From mal at europython.eu  Mon Jun  1 11:08:57 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Mon, 01 Jun 2015 17:08:57 +0200
Subject: EuroPython 2015: Conference Dinner and Party
Message-ID: <556C7589.4020800@europython.eu>

We all know that good food, drinks and music are essential for a
perfect EuroPython conference and so we?ve arranged a nice cocktail
dinner with local food and a party for the evening on Wednesday,
July 22nd.

            *** EuroPython Conference Dinner and Party ***

        https://ep2015.europython.eu/en/events/social-event/

If you want to join the fun, please buy a social event ticket on the
same registration form you use to buy conference tickets. There?s a
new entry available now called ?Pyntxos Social Event? which you can
order using the form. Leave the conference tickets fields blank if you
only want to purchase social event tickets:

               https://ep2015.europython.eu/p3/cart/

We have 600 tickets available for the conference dinner and party, so
please book early. The ticket price includes food and drinks (until
midnight) for one person.

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From bc at freeuk.com  Mon Jun  1 11:17:42 2015
From: bc at freeuk.com (BartC)
Date: Mon, 01 Jun 2015 16:17:42 +0100
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mailman.290.1433166769.5151.python-list@python.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <201506011124.t51BOqO2021385@fido.openend.se>
 <mailman.290.1433166769.5151.python-list@python.org>
Message-ID: <rI_ax.603294$PJ4.511481@fx26.am4>

On 01/06/2015 14:52, Chris Angelico wrote:
 > It's
> like the eternal debate about assignment and whether "x = x + 1" is
> nonsense, with advocates preferring "x := x + 1" as being somehow
> fundamentally different. It isn't. It's just a notational change, and
> not even a huge one. (Though I do see the line of argument that it
> should be "x <- x + 1" or something else that looks like an arro'w.)

'x <- x + 1' already means something as an expression (whether x is less 
than (-x+1). 'x <= x + 1' has the same problem.

But I have used "=>" before,  for left-to-right assignment. (Mostly I 
use ":=")

-- 
Bartc




From denismfmcmahon at gmail.com  Mon Jun  1 11:17:49 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Mon, 1 Jun 2015 15:17:49 +0000 (UTC)
Subject: Where is 'palindrome' defined?
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
Message-ID: <mkht2s$hvk$2@dont-email.me>

On Sun, 31 May 2015 21:46:31 -0700, fl wrote:

>>>> def palindrome(num):

Note carefully the spelling(1): palindrome

>>>> parlindrome(a)

Note carefully the spelling(2): parlindrome

> NameError: name 'parlindrome' is not defined

Compare carefully spelling(1) and spelling(2).

palindrome is defined, pa_r_lindrome is not defined. The computer is not 
psychic yet, so it doesn't know that when you wrote pa_r_lindrome you 
meant palindrome, you have to spell it the same way every time for the 
computer to recognise that you mean the same name.

Now you also know that the error message:

"NameError: name '<something>' is not defined"

means that you might have spelled <something> differently on the line in 
the error message to the word you meant, so next time you see this error 
message you know to carefully check the spellings of function names and 
variables.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From alister.nospam.ware at ntlworld.com  Mon Jun  1 11:26:49 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Mon, 1 Jun 2015 15:26:49 +0000 (UTC)
Subject: What is considered an "advanced" topic in Python?
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org>
 <87iob78wvl.fsf@elektro.pacujo.net>
 <mailman.0.1433171249.13271.python-list@python.org>
Message-ID: <mkhtjp$a8e$1@speranza.aioe.org>

On Mon, 01 Jun 2015 17:07:18 +0200, Laura Creighton wrote:

> In a message of Mon, 01 Jun 2015 14:57:02 +0300, Marko Rauhamaa writes:
>>In 1951, decimal numbers would have done little good in the UK with the
>>pound divided into 20 shillings and the shilling into 12 pence. Maybe a
>>"Babylonian" module would have been perfect.
>>
>>
>>Marko
> 
> You are being facetious, but in point of fact, naive Brits who knew
> nothing of neither accounting systems nor floating point for the most
> part got things right when they bought their Sinclair computer in the
> early 1980s.
> 
> This is because their natural tendancy was to calculate all the pounds
> separately, and then the shillings, separately, and then the pence.
> (With Guineas and other odd stuff thrown in, when the needed them).
> This meant that they kept 3+ legers at one time, and then, when they
> were done calculating as one final step converted what they had into its
> representation where you never had more than 100 pence or 12 shillings.
> Thus, entirely by accident, they did their accounting in integers, not
> decimals at all.
> 
> And this is, of course, the first thing that people who write real
> systems that add money learn -- convert everything to pennies (or
> whatever you call them) and do all your calculations in pennies, and
> then as the final step express that in dollars and cents, or euros and
> cents, or what have you.
> 
> The Brits still got in trouble when they needed to calculate things for
> their 4.2 per cent morgage, or decided to keep a running total of the
> sales tax they were paying, but they at least did not grab the floating
> number representation as the first thing off the shelf when they needed
> money.
> 
> Laura

I don't think anyone programmed a Sinclair computer to use pre-decimal 
currency, we converted to decimal in 1971 (although the last pre-decimal 
coin did not go out of use untill 1993)   



-- 
If I traveled to the end of the rainbow
As Dame Fortune did intend,
Murphy would be there to tell me
The pot's at the other end.
		-- Bert Whitney


From denismfmcmahon at gmail.com  Mon Jun  1 11:30:32 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Mon, 1 Jun 2015 15:30:32 +0000 (UTC)
Subject: What is considered an "advanced" topic in Python?
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
Message-ID: <mkhtqo$hvk$3@dont-email.me>

On Fri, 29 May 2015 09:01:55 -0700, Mike Driscoll wrote:

> I've been asked on several occasions to write about intermediate or
> advanced topics in Python and I was wondering what the community
> considers to be "intermediate" or "advanced". I realize we're all
> growing in our abilities with the language, so this is going to be very
> subjective, but I am still curious what my fellow Python developers
> think about this topic.

Hmmm, in terms of learning about computer programming:

simplest: print "hello world"

then things get more advanced in steps:

(1) more instructions, but executed linearly, numbers and strings.

(2) conditional execution - if statement (including error trapping)

(3) loops, lists, dictionaries

(4) defining functions

(5) recursion

(6) sexy python stuff - things like list comprehensions, using iterators, 
importing modules etc

I think that's probably "basic python" covered in 6 steps (7 if you 
include hello world). Although you now have all the tools you need to 
write python code to do possibly very complex tasks, and thus to write 
very complex programs, you're using (IMO) basic python programming skills 
in doing so.

I guess that makes OOP / classes the advanced topic in my system.

I don't consider "using library x to do y" as advanced python, it's just 
gluing together existing functions with your own basic programming, no 
matter whether the library is for hardware IO, interfacing to a database 
etc.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From marko at pacujo.net  Mon Jun  1 11:39:05 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 18:39:05 +0300
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
 <87vbf88k0n.fsf@elektro.pacujo.net> <mkhs0i$9d5$2@reader1.panix.com>
Message-ID: <87mw0j8mli.fsf@elektro.pacujo.net>

Grant Edwards <invalid at invalid.invalid>:

> On 2015-05-31, Marko Rauhamaa <marko at pacujo.net> wrote:
>> If you don't care to know when child processes exit, you can simply
>> ignore the SIGCHLD signal:
>>
>>     import signal
>>     signal.signal(signal.SIGCHLD, signal.SIG_IGN)
>>
>> That will prevent zombies from appearing.
>
> Bravo!  I've been writing Unix apps for 30 years, and I did not know
> that.  Is this something recent[1], or have I somehow managed to avoid
> this useful bit of info for that long?

I wasn't aware of it ever having been any other way. However:

   POSIX.1-1990 disallowed setting the action for SIGCHLD to SIG_IGN.
   POSIX.1-2001 allows this possibility, so that ignoring SIGCHLD can be
   used to prevent the creation of zombies (see wait(2)). Nevertheless,
   the historical BSD and System V behaviors for ignoring SIGCHLD
   differ, so that the only completely portable method of ensuring that
   terminated children do not become zombies is to catch the SIGCHLD
   signal and perform a wait(2) or similar.

   <URL: http://man7.org/linux/man-pages/man2/sigaction.2.html>

If you possess the Stevens book, you'd probably find out how SIGC(H)LD
was treated in BSD and System V. I can't remember despite programming
for either environment.


Marko


From bc at freeuk.com  Mon Jun  1 11:42:50 2015
From: bc at freeuk.com (BartC)
Date: Mon, 01 Jun 2015 16:42:50 +0100
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mailman.0.1433171249.13271.python-list@python.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org><87iob78wvl.fsf@elektro.pacujo.net>
 <mailman.0.1433171249.13271.python-list@python.org>
Message-ID: <04%ax.914987$396.795703@fx12.am4>

On 01/06/2015 16:07, Laura Creighton wrote:
> In a message of Mon, 01 Jun 2015 14:57:02 +0300, Marko Rauhamaa writes:
>> In 1951, decimal numbers would have done little good in the UK with the
>> pound divided into 20 shillings and the shilling into 12 pence. Maybe a
>> "Babylonian" module would have been perfect.
>>
>>
>> Marko
>
> You are being facetious, but in point of fact, naive Brits who knew nothing
> of neither accounting systems nor floating point for the most part got
> things right when they bought their Sinclair computer in the early
> 1980s.
>
> This is because their natural tendancy was to calculate all the pounds
> separately, and then the shillings, separately, and then the pence.

Strangely I don't remember that. Although we did have to add up sums of 
money in three columns: pounds, shillings and pence, and carried as 
necessary from right to left.

> (With Guineas and other odd stuff thrown in, when the needed them).
> This meant that they kept 3+ legers at one time, and then, when they
> were done calculating as one final step converted what they had into
> its representation where you never had more than 100 pence or 12 shillings.

You mean 12 pence or 20 shillings?

> Thus, entirely by accident, they did their accounting in integers, not
> decimals at all.

> And this is, of course, the first thing that people who write real
> systems that add money learn -- convert everything to pennies (or
> whatever you call them) and do all your calculations in pennies, and
> then as the final step express that in dollars and cents, or euros
> and cents, or what have you.

When decimal currency came in, we had decimals *and* fractions! Because 
of the 1/2p coin. So this didn't quite work.

But it can work now, so we have no need of either binary or decimal 
floating point.

(Well, until you have to work in dual currencies which can mean that an 
amount of money that is exact in one currency, can only be approximate 
in the other, when both are expressed in the smallest currency unit - 
cent, penny or whatever.)

> The Brits still got in trouble when they needed to calculate things for their
> 4.2 per cent morgage, or decided to keep a running total of the sales
> tax they were paying, but they at least did not grab the floating
> number representation as the first thing off the shelf when they needed
> money.

At one time the choice was integer or floating point in many languages, 
unless you were specifically using a business language such as Cobol.

I think the Sinclair computer barely had integer types so the choice was 
even narrower.

-- 
Bartc


From lac at openend.se  Mon Jun  1 11:44:44 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 17:44:44 +0200
Subject: Using Python instead of Bash
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Mon, 01 Jun 2015 12:44:34 +0200." <876177lncd.fsf@Equus.decebal.nl>
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>
 <lf5y4k3onsm.fsf@ling.helsinki.fi>
 <mkh7c3$itb$1@speranza.aioe.org><876177lncd.fsf@Equus.decebal.nl>
Message-ID: <201506011544.t51FiiHo028015@fido.openend.se>

In a message of Mon, 01 Jun 2015 12:44:34 +0200, Cecil Westerhof writes:
>Also the funny thing is that I first scanned it. But it gave several
>problems. One of them was that it is a format between A4 and A5.
>Taking pictures is faster and gives better results.
>
>-- 
>Cecil Westerhof
>Senior Software Engineer
>LinkedIn: http://www.linkedin.com/in/cecilwesterhof

My irony detector may be on the fritz today, but, well, if you run into
some weird format between A4 and A5, you probably have a USA size.
see: http://en.wikipedia.org/wiki/Paper_size#North_American_paper_sizes

Most scanning programs have a way of specifying what paper size you want,
though those made in the USA have a natural tendancy to default to the
wierd American sizes.

Apologies if you already know all about this ...

Laura


From lac at openend.se  Mon Jun  1 11:51:49 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 17:51:49 +0200
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: Message from alister <alister.nospam.ware@ntlworld.com>
 of "Mon, 01 Jun 2015 15:26:49 -0000." <mkhtjp$a8e$1@speranza.aioe.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org>
 <87iob78wvl.fsf@elektro.pacujo.net>
 <mailman.0.1433171249.13271.python-list@python.org><mkhtjp$a8e$1@speranza.aioe.org>
Message-ID: <201506011551.t51FpnUp028244@fido.openend.se>

In a message of Mon, 01 Jun 2015 15:26:49 -0000, alister writes:
>I don't think anyone programmed a Sinclair computer to use pre-decimal 
>currency, we converted to decimal in 1971 (although the last pre-decimal 
>coin did not go out of use untill 1993)

Interesting.  Somebody sent me one that supposedly was written for
then a few years ago.  Maybe it was doing 'old style accounting'
for fun?  I will have to go ask that person about it.  Thank you
for letting me know.

Laura


From rosuav at gmail.com  Mon Jun  1 12:10:17 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 2 Jun 2015 02:10:17 +1000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <rI_ax.603294$PJ4.511481@fx26.am4>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <201506011124.t51BOqO2021385@fido.openend.se>
 <mailman.290.1433166769.5151.python-list@python.org>
 <rI_ax.603294$PJ4.511481@fx26.am4>
Message-ID: <CAPTjJmrXzta987vsgcr7FYZLJNYRsV7uW9HNWJi3zJh-qKT3OQ@mail.gmail.com>

On Tue, Jun 2, 2015 at 1:17 AM, BartC <bc at freeuk.com> wrote:
> On 01/06/2015 14:52, Chris Angelico wrote:
>> It's
>>
>> like the eternal debate about assignment and whether "x = x + 1" is
>> nonsense, with advocates preferring "x := x + 1" as being somehow
>> fundamentally different. It isn't. It's just a notational change, and
>> not even a huge one. (Though I do see the line of argument that it
>> should be "x <- x + 1" or something else that looks like an arro'w.)
>
>
> 'x <- x + 1' already means something as an expression (whether x is less
> than (-x+1). 'x <= x + 1' has the same problem.
>
> But I have used "=>" before,  for left-to-right assignment. (Mostly I use
> ":=")

In Python it does, yes; I'm talking about the language design
advocates. Some recommend a two-character ASCII notation like "<-" or
"<=", others prefer a single-character symbol eg "?" or "?", but
whatever it is, it will have no meaning in that language other than
assignment. And yes, I can see the value of using an arrow to indicate
assignment... but I don't really see a huge problem with using "=" to
mean assignment, given that people from a mathematical background will
have to grok the entire concept of temporal truth anyway. Whatever
symbol you use, it has to be explained.

ChrisA


From notMyMail at mail.not  Mon Jun  1 12:23:49 2015
From: notMyMail at mail.not (gm)
Date: Mon, 01 Jun 2015 18:23:49 +0200
Subject: lotto number generator
Message-ID: <mki0um$q97$1@l01news1.ot.hr>

Hi.
I am new to python so am still in learning phase.

I was thinking to make one program that will print out all possible 
combinations of 10 pairs. I think this is a good way for something 
"bigger" :-).

This is how this looks like:

1.) 1 2
2.) 1 2
3.) 1 2
4.) 1 2
5.) 1 2
6.) 1 2
7.) 1 2
8.) 1 2
9.) 1 2
10.) 1 2

So, i want to print out (in rows) each possible configuration but for 
all 10 pairs.

example:
1 1 1 2 2 2 2 1 1 1
2 2 2 1 1 1 1 2 2 2
1 1 2 2 1 1 2 2 1 1
etc.

What would be the best way to make something like this ?
Maybe some tutorial ?

GM


From Cecil at decebal.nl  Mon Jun  1 12:44:01 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 01 Jun 2015 18:44:01 +0200
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>
 <lf5y4k3onsm.fsf@ling.helsinki.fi> <mkh7c3$itb$1@speranza.aioe.org>
 <876177lncd.fsf@Equus.decebal.nl>
 <mailman.2.1433173497.13271.python-list@python.org>
Message-ID: <87oakzjs4u.fsf@Equus.decebal.nl>

Op Monday 1 Jun 2015 17:44 CEST schreef Laura Creighton:

> In a message of Mon, 01 Jun 2015 12:44:34 +0200, Cecil Westerhof
> writes:
>> Also the funny thing is that I first scanned it. But it gave
>> several problems. One of them was that it is a format between A4
>> and A5. Taking pictures is faster and gives better results.
>>
>> -- 
>> Cecil Westerhof
>> Senior Software Engineer
>> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>
> My irony detector may be on the fritz today, but, well, if you run
> into some weird format between A4 and A5, you probably have a USA
> size. see:
> http://en.wikipedia.org/wiki/Paper_size#North_American_paper_sizes

Nope, the dimensions are 177 mm x 227 mm.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From invalid at invalid.invalid  Mon Jun  1 13:02:14 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Mon, 1 Jun 2015 17:02:14 +0000 (UTC)
Subject: What is considered an "advanced" topic in Python?
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org>
 <87iob78wvl.fsf@elektro.pacujo.net>
 <mailman.0.1433171249.13271.python-list@python.org>
 <04%ax.914987$396.795703@fx12.am4>
Message-ID: <mki36m$bv3$1@reader1.panix.com>

On 2015-06-01, BartC <bc at freeuk.com> wrote:

> At one time the choice was integer or floating point in many languages, 
> unless you were specifically using a business language such as Cobol.

My recollection in the early days of home computers is that many BASIC
implementations had BCD floating point instead of binary.  Back then
most CPUs had instructions specifcally for dealing with BCD
represented with 4-bits per digit.  Dunno if they still do, I can't
even remember the last time I did calculations in BCD.

> I think the Sinclair computer barely had integer types so the choice
> was even narrower.

-- 
Grant Edwards               grant.b.edwards        Yow! I'm totally DESPONDENT
                                  at               over the LIBYAN situation
                              gmail.com            and the price of CHICKEN
                                                   ...


From python at mrabarnett.plus.com  Mon Jun  1 13:06:30 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 01 Jun 2015 18:06:30 +0100
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <201506011551.t51FpnUp028244@fido.openend.se>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org>
 <87iob78wvl.fsf@elektro.pacujo.net>
 <mailman.0.1433171249.13271.python-list@python.org><mkhtjp$a8e$1@speranza.aioe.org>
 <201506011551.t51FpnUp028244@fido.openend.se>
Message-ID: <556C9116.3000202@mrabarnett.plus.com>

On 2015-06-01 16:51, Laura Creighton wrote:
> In a message of Mon, 01 Jun 2015 15:26:49 -0000, alister writes:
>> I don't think anyone programmed a Sinclair computer to use
>> pre-decimal currency, we converted to decimal in 1971 (although the
>> last pre-decimal coin did not go out of use untill 1993)
>
> Interesting.  Somebody sent me one that supposedly was written for
> then a few years ago.  Maybe it was doing 'old style accounting' for
> fun?  I will have to go ask that person about it.  Thank you for
> letting me know.
>
1971 was also the year we started the switch over to the metric system.
It was going to be done gradually over a 10-year period. It's still a
work in progress...


From invalid at invalid.invalid  Mon Jun  1 13:07:30 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Mon, 1 Jun 2015 17:07:30 +0000 (UTC)
Subject: lotto number generator
References: <mki0um$q97$1@l01news1.ot.hr>
Message-ID: <mki3gi$bv3$2@reader1.panix.com>

On 2015-06-01, gm <notMyMail at mail.not> wrote:

> I was thinking to make one program that will print out all possible 
> combinations of 10 pairs.
[...]
> What would be the best way to make something like this?
> Maybe some tutorial ?

https://www.google.com/search?q=python+combinations

-- 
Grant Edwards               grant.b.edwards        Yow! A dwarf is passing out
                                  at               somewhere in Detroit!
                              gmail.com            


From ian.g.kelly at gmail.com  Mon Jun  1 13:13:39 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 1 Jun 2015 11:13:39 -0600
Subject: lotto number generator
In-Reply-To: <mki0um$q97$1@l01news1.ot.hr>
References: <mki0um$q97$1@l01news1.ot.hr>
Message-ID: <CALwzidk1Rg7oODLYvKC9JgLbbB8Kd3y-XWsUBv5s3qRvORV9Nw@mail.gmail.com>

On Mon, Jun 1, 2015 at 10:23 AM, gm <notMyMail at mail.not> wrote:
> Hi.
> I am new to python so am still in learning phase.
>
> I was thinking to make one program that will print out all possible
> combinations of 10 pairs. I think this is a good way for something "bigger"
> :-).
>
> This is how this looks like:
>
> 1.) 1 2
> 2.) 1 2
> 3.) 1 2
> 4.) 1 2
> 5.) 1 2
> 6.) 1 2
> 7.) 1 2
> 8.) 1 2
> 9.) 1 2
> 10.) 1 2
>
> So, i want to print out (in rows) each possible configuration but for all 10
> pairs.
>
> example:
> 1 1 1 2 2 2 2 1 1 1
> 2 2 2 1 1 1 1 2 2 2
> 1 1 2 2 1 1 2 2 1 1
> etc.
>
> What would be the best way to make something like this ?
> Maybe some tutorial ?

The Python tutorial is a good place to start:

    https://docs.python.org/3.4/tutorial/index.html

For your particular problem you'll probably want to use the
itertools.product function:

    https://docs.python.org/3.4/library/itertools.html#itertools.product

I wouldn't recommend printing out every possible combination to the
screen, though. That's over a million rows! Start with fewer pairs (4
is probably good), or write them to a file instead.


From marko at pacujo.net  Mon Jun  1 13:14:52 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 20:14:52 +0300
Subject: What is considered an "advanced" topic in Python?
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org>
 <87iob78wvl.fsf@elektro.pacujo.net>
 <mailman.0.1433171249.13271.python-list@python.org>
 <mkhtjp$a8e$1@speranza.aioe.org>
 <201506011551.t51FpnUp028244@fido.openend.se>
 <mailman.5.1433178401.13271.python-list@python.org>
Message-ID: <87bngz8i5v.fsf@elektro.pacujo.net>

MRAB <python at mrabarnett.plus.com>:

> 1971 was also the year we started the switch over to the metric
> system. It was going to be done gradually over a 10-year period. It's
> still a work in progress...

Same here in Finland. Minutes, hours and degrees persist as do calories,
teaspoons and carats.


Marko


From ian.g.kelly at gmail.com  Mon Jun  1 13:16:29 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 1 Jun 2015 11:16:29 -0600
Subject: lotto number generator
In-Reply-To: <CALwzidk1Rg7oODLYvKC9JgLbbB8Kd3y-XWsUBv5s3qRvORV9Nw@mail.gmail.com>
References: <mki0um$q97$1@l01news1.ot.hr>
 <CALwzidk1Rg7oODLYvKC9JgLbbB8Kd3y-XWsUBv5s3qRvORV9Nw@mail.gmail.com>
Message-ID: <CALwzid=rgDDyRrng4tJxRnsRE=RNYnMZ+0H9HUr2+7EB7UvO+g@mail.gmail.com>

On Mon, Jun 1, 2015 at 11:13 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Mon, Jun 1, 2015 at 10:23 AM, gm <notMyMail at mail.not> wrote:
>> Hi.
>> I am new to python so am still in learning phase.
>>
>> I was thinking to make one program that will print out all possible
>> combinations of 10 pairs. I think this is a good way for something "bigger"
>> :-).
>>
>> This is how this looks like:
>>
>> 1.) 1 2
>> 2.) 1 2
>> 3.) 1 2
>> 4.) 1 2
>> 5.) 1 2
>> 6.) 1 2
>> 7.) 1 2
>> 8.) 1 2
>> 9.) 1 2
>> 10.) 1 2
>>
>> So, i want to print out (in rows) each possible configuration but for all 10
>> pairs.
>>
>> example:
>> 1 1 1 2 2 2 2 1 1 1
>> 2 2 2 1 1 1 1 2 2 2
>> 1 1 2 2 1 1 2 2 1 1
>> etc.
>>
>> What would be the best way to make something like this ?
>> Maybe some tutorial ?
>
> The Python tutorial is a good place to start:
>
>     https://docs.python.org/3.4/tutorial/index.html
>
> For your particular problem you'll probably want to use the
> itertools.product function:
>
>     https://docs.python.org/3.4/library/itertools.html#itertools.product
>
> I wouldn't recommend printing out every possible combination to the
> screen, though. That's over a million rows! Start with fewer pairs (4
> is probably good), or write them to a file instead.

Er, actually it's just 1024, so that isn't so bad. You probably still
don't want all that in your terminal, though.


From breamoreboy at yahoo.co.uk  Mon Jun  1 13:35:45 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 01 Jun 2015 18:35:45 +0100
Subject: Zero [was Re: What is considered an "advanced" topic in Python?]
In-Reply-To: <CANc-5Uwba3DyMkEvaFLjzcE8+2qGYqTrjFp5DAiR5wPy9TOLbA@mail.gmail.com>
References: <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
 <87d21fahyi.fsf@elektro.pacujo.net>
 <556c4b0e$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <1jlomal1or023ian8rlaso1u5bnol52lag@4ax.com>
 <CANc-5Uwba3DyMkEvaFLjzcE8+2qGYqTrjFp5DAiR5wPy9TOLbA@mail.gmail.com>
Message-ID: <mki55j$bvb$1@ger.gmane.org>

On 01/06/2015 14:14, Skip Montanaro wrote:
>
> Maybe you should just install
> a decent spam filter <http://www.spambayes.org/> or switch to Gmail,
> which has a functioning spam filter (unlike Yahoo...)
>

Okay I'll bite, what's wrong with the Yahoo spam filter?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From dreamingforward at gmail.com  Mon Jun  1 13:35:58 2015
From: dreamingforward at gmail.com (Dr. Bigcock)
Date: Mon, 1 Jun 2015 10:35:58 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <556b4c52$0$13013$c3e8da3$5496439d@news.astraweb.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <556b4c52$0$13013$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5414e38c-ad39-4e5d-ad79-1c767e72cc3d@googlegroups.com>

On Sunday, May 31, 2015 at 1:01:03 PM UTC-5, Steven D'Aprano wrote:
> On Mon, 1 Jun 2015 12:34 am, Eddilbert Macharia wrote:
> 
> > Hello All ,
> > 
> > I'm wrecking my head trying to understand. where the class object comes
> > into play .
> 
> This is just the standard object-oriented concept of class and instances.
> Have you programmed in any other OO language? 

The issue here is:  which one is the object?  The class or the instance?

;^),

-m


From tyler at tysdomain.com  Mon Jun  1 13:36:27 2015
From: tyler at tysdomain.com (Littlefield, Tyler)
Date: Mon, 01 Jun 2015 13:36:27 -0400
Subject: replace parens update
Message-ID: <556C981B.3020102@tysdomain.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello all:
I've put a new version here:
https://dl.dropboxusercontent.com/u/10204868/replace_parens.py

It's for me, a lot easier to understand. It uses regex to replace
parens rather than using a state machine and conforms to the python
standards for naming and conventions.

HTH,
- -- 
Take care,
Ty
twitter: @sorressean
web:http://tysdomain.com
pubkey: http://tysdomain.com/files/pubkey.asc
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBAgAGBQJVbJgbAAoJEAdP60+BYxejGSoIAJUc3n7dL74MOc6APQ1gY+io
iQhMtmYEp2s5zjv7NP7i+ENWBys8+r7GDBghYW6xrV1JD24VuSFWi1Zya1RuRy2b
v95b+EH/ppHGva4jF71uBAVW4Rh3gdW0uqDrmW4FCdWhZaTATv3hlwG/M5QcFvjF
fDj/lXEB/zljqmf8Hdp3FuHj+QkwJwZihVOhHOHIiZ4zXMlT2J+M76LQKdHcMpFE
SJVaju5isJYEWXGJ9r/iJwgf87Y9fe20bWfqNw0MQtxjiApwo5wcvZshBz6vbLHg
ehKRD/Xz9yBSWnfTrSw0wkOE1uVk4cQlVLhBkP6WvfSf1UYKtXnf6zyNwILu+Rs=
=Ka2/
-----END PGP SIGNATURE-----


From breamoreboy at yahoo.co.uk  Mon Jun  1 13:45:18 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 01 Jun 2015 18:45:18 +0100
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mki36m$bv3$1@reader1.panix.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org>
 <87iob78wvl.fsf@elektro.pacujo.net>
 <mailman.0.1433171249.13271.python-list@python.org>
 <04%ax.914987$396.795703@fx12.am4> <mki36m$bv3$1@reader1.panix.com>
Message-ID: <mki5nh$m52$1@ger.gmane.org>

On 01/06/2015 18:02, Grant Edwards wrote:
> On 2015-06-01, BartC <bc at freeuk.com> wrote:
>
>> At one time the choice was integer or floating point in many languages,
>> unless you were specifically using a business language such as Cobol.
>
> My recollection in the early days of home computers is that many BASIC
> implementations had BCD floating point instead of binary.  Back then
> most CPUs had instructions specifcally for dealing with BCD
> represented with 4-bits per digit.  Dunno if they still do, I can't
> even remember the last time I did calculations in BCD.
>
>> I think the Sinclair computer barely had integer types so the choice
>> was even narrower.
>

Didn't Turbo C have compiler options to allow either BCD or fp?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lac at openend.se  Mon Jun  1 14:16:27 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 20:16:27 +0200
Subject: Somebody is trying to install python
Message-ID: <201506011816.t51IGR4s032445@fido.openend.se>

The person who is being told that their processor is incompatible is
back.  But replied to me, not the list.
reply to: https://mail.python.org/pipermail/python-list/2015-May/691557.html

Laura


------- Forwarded Message
Return-Path: <flowingtide2 at tiscali.co.uk>
From: "David Skardon" <flowingtide2 at tiscali.co.uk>
To: "Laura Creighton" <lac at openend.se>
References: <414E2DF84E3544769D2597ED4B2002BE at oldsalt> <201505291311.t4TDB3rM021314 at fido.openend.se>
Subject: Re: python
Content-Type: text/plain

trying to install a binary, opp system windows xp tried python 2 and three 
both failed said incompatible processor, my pc is 1 year old with an AMD 
6200 6 core processor
thanks
Dave 

------- End of Forwarded Message


From invalid at invalid.invalid  Mon Jun  1 14:23:31 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Mon, 1 Jun 2015 18:23:31 +0000 (UTC)
Subject: What is considered an "advanced" topic in Python?
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <mailman.282.1433157907.5151.python-list@python.org>
 <87iob78wvl.fsf@elektro.pacujo.net>
 <mailman.0.1433171249.13271.python-list@python.org>
 <04%ax.914987$396.795703@fx12.am4> <mki36m$bv3$1@reader1.panix.com>
 <mailman.10.1433180728.13271.python-list@python.org>
Message-ID: <mki7v3$hii$1@reader1.panix.com>

On 2015-06-01, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> On 01/06/2015 18:02, Grant Edwards wrote:
>> On 2015-06-01, BartC <bc at freeuk.com> wrote:
>>
>>> At one time the choice was integer or floating point in many languages,
>>> unless you were specifically using a business language such as Cobol.
>>
>> My recollection in the early days of home computers is that many BASIC
>> implementations had BCD floating point instead of binary.  Back then
>> most CPUs had instructions specifcally for dealing with BCD
>> represented with 4-bits per digit.  Dunno if they still do, I can't
>> even remember the last time I did calculations in BCD.
>>
>>> I think the Sinclair computer barely had integer types so the choice
>>> was even narrower.
>
> Didn't Turbo C have compiler options to allow either BCD or fp?

Probably.  BCD support was pretty widespread in the Pascal compilers I
remember for DOS and CP/M.  Binary floating point didn't get very
popular until HW support for it became more common.

-- 
Grant Edwards               grant.b.edwards        Yow! Now we can become
                                  at               alcoholics!
                              gmail.com            


From lac at openend.se  Mon Jun  1 14:42:58 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 20:42:58 +0200
Subject: Using Python instead of Bash
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Mon, 01 Jun 2015 18:44:01 +0200." <87oakzjs4u.fsf@Equus.decebal.nl>
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>
 <lf5y4k3onsm.fsf@ling.helsinki.fi> <mkh7c3$itb$1@speranza.aioe.org>
 <876177lncd.fsf@Equus.decebal.nl>
 <mailman.2.1433173497.13271.python-list@python.org><87oakzjs4u.fsf@Equus.decebal.nl>
Message-ID: <201506011842.t51IgwdZ000860@fido.openend.se>

In a message of Mon, 01 Jun 2015 18:44:01 +0200, Cecil Westerhof writes:
>Op Monday 1 Jun 2015 17:44 CEST schreef Laura Creighton:
>
>> In a message of Mon, 01 Jun 2015 12:44:34 +0200, Cecil Westerhof
>> writes:
>>> Also the funny thing is that I first scanned it. But it gave
>>> several problems. One of them was that it is a format between A4
>>> and A5. Taking pictures is faster and gives better results.
>>>
>>> -- 
>>> Cecil Westerhof
>>> Senior Software Engineer
>>> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>>
>> My irony detector may be on the fritz today, but, well, if you run
>> into some weird format between A4 and A5, you probably have a USA
>> size. see:
>> http://en.wikipedia.org/wiki/Paper_size#North_American_paper_sizes
>
>Nope, the dimensions are 177 mm x 227 mm.
>
>-- 
>Cecil Westerhof
>Senior Software Engineer
>LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>-- 
>https://mail.python.org/mailman/listinfo/python-list

Truly?  That's (very close to) 7 inch by 9 inch, 177.8 mm x 228.6 mm
and 7 by 9 is what pre-metric Britian called 'Small Post Quarto'.
I wonder if this merely a coincidence, or does some software really
still like this size?  How very weird.

Laura


From rosuav at gmail.com  Mon Jun  1 15:48:16 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 2 Jun 2015 05:48:16 +1000
Subject: Somebody is trying to install python
In-Reply-To: <201506011816.t51IGR4s032445@fido.openend.se>
References: <201506011816.t51IGR4s032445@fido.openend.se>
Message-ID: <CAPTjJmqGPnMH8meLuQ3pKCST8spW1UiSXjmRzrXu_VJ8gGNWKg@mail.gmail.com>

On Tue, Jun 2, 2015 at 4:16 AM, Laura Creighton <lac at openend.se> wrote:
> trying to install a binary, opp system windows xp tried python 2 and three
> both failed said incompatible processor, my pc is 1 year old with an AMD
> 6200 6 core processor
> thanks
> Dave

Hi David!

Do you know what edition of Windows you have installed? Presumably the
CPU you have there is a 64-bit one (also known as amd64 or x86_64),
but if you had a 32-bit Windows, you won't be able to run 64-bit
programs on it. With Windows XP, the 64-bit version was somewhat
unusual, if I recall correctly, so it's entirely possible you have a
32-bit Windows.

At very least, it's worth a try.

ChrisA


From Cecil at decebal.nl  Mon Jun  1 15:57:05 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 01 Jun 2015 21:57:05 +0200
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>
 <lf5y4k3onsm.fsf@ling.helsinki.fi> <mkh7c3$itb$1@speranza.aioe.org>
 <876177lncd.fsf@Equus.decebal.nl>
 <mailman.2.1433173497.13271.python-list@python.org>
 <87oakzjs4u.fsf@Equus.decebal.nl>
 <mailman.12.1433184186.13271.python-list@python.org>
Message-ID: <87h9qrjj72.fsf@Equus.decebal.nl>

Op Monday 1 Jun 2015 20:42 CEST schreef Laura Creighton:

> In a message of Mon, 01 Jun 2015 18:44:01 +0200, Cecil Westerhof
> writes:
>> Op Monday 1 Jun 2015 17:44 CEST schreef Laura Creighton:
>>
>>> In a message of Mon, 01 Jun 2015 12:44:34 +0200, Cecil Westerhof
>>> writes:
>>>> Also the funny thing is that I first scanned it. But it gave
>>>> several problems. One of them was that it is a format between A4
>>>> and A5. Taking pictures is faster and gives better results.
>>>>
>>>> -- 
>>>> Cecil Westerhof
>>>> Senior Software Engineer
>>>> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>>>
>>> My irony detector may be on the fritz today, but, well, if you run
>>> into some weird format between A4 and A5, you probably have a USA
>>> size. see:
>>> http://en.wikipedia.org/wiki/Paper_size#North_American_paper_sizes
>>
>> Nope, the dimensions are 177 mm x 227 mm.
>>
>> -- 
>> Cecil Westerhof
>> Senior Software Engineer
>> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>
> Truly?  That's (very close to) 7 inch by 9 inch, 177.8 mm x 228.6 mm
> and 7 by 9 is what pre-metric Britian called 'Small Post Quarto'.
> I wonder if this merely a coincidence, or does some software really
> still like this size?  How very weird.

Well, it is possible I did not measure correctly. ;-) It is a 32 page
booklet.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From python.list at tim.thechases.com  Mon Jun  1 16:29:30 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Mon, 1 Jun 2015 15:29:30 -0500
Subject: Pyton re module and POSIX equivalence classes
Message-ID: <20150601152930.77f0a8fb@bigbox.christie.dr>

Is Python supposed to support POSIX "equivalence classes"?  I tried
the following in Py2 and Py3:

  >>> re.sub('[[=a=]]', 'A', 'a?????', re.U)
  'a?????'

which suggests that it doesn't (I would have expected "AAAAAA" as the
result).

Is there a way to get this behavior?

I found that perl knows about them but treats them as an exception
for now[1].  Supposedly GNU awk (and other GNU POSIXish tools)
recognize character classes, as does vim.

Thanks,

-tkc



[1]
http://perldoc.perl.org/perlrecharclass.html







From dreamingforward at gmail.com  Mon Jun  1 16:38:57 2015
From: dreamingforward at gmail.com (TheDoctor)
Date: Mon, 1 Jun 2015 13:38:57 -0700 (PDT)
Subject: super
Message-ID: <6c339a78-28b8-4187-8f0f-0ea7147879d7@googlegroups.com>

Missive #02441:  super()

If you're going to use object composition, creating a super (no pun intended) class, you are forming an explicit object hierarchy (unlike mix-in style).  In this case, if the method exists in the parent class, you shouldn't give the user (or subclass) the choice of calling super because it will break the concept of PARENT.

The other VERY GOOD reason is that there's no question about initialization order.  A subclass can sometimes call other methods.  If those methods also are in the super class you have condition where you can't quite tell what is happening where.  When the hierarchy of calls is well-defined because you can rely on super call order, then there's *no problem*.

Please direct questions to my hand.

Mark


From lac at openend.se  Mon Jun  1 16:42:38 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 22:42:38 +0200
Subject: Using Python instead of Bash
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Mon, 01 Jun 2015 21:57:05 +0200." <87h9qrjj72.fsf@Equus.decebal.nl>
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>
 <lf5y4k3onsm.fsf@ling.helsinki.fi> <mkh7c3$itb$1@speranza.aioe.org>
 <876177lncd.fsf@Equus.decebal.nl>
 <mailman.2.1433173497.13271.python-list@python.org>
 <87oakzjs4u.fsf@Equus.decebal.nl>
 <mailman.12.1433184186.13271.python-list@python.org><87h9qrjj72.fsf@Equus.decebal.nl>
Message-ID: <201506012042.t51KgcHB003977@fido.openend.se>

In a message of Mon, 01 Jun 2015 21:57:05 +0200, Cecil Westerhof writes:
>> Truly?  That's (very close to) 7 inch by 9 inch, 177.8 mm x 228.6 mm
>> and 7 by 9 is what pre-metric Britian called 'Small Post Quarto'.
>> I wonder if this merely a coincidence, or does some software really
>> still like this size?  How very weird.
>
>Well, it is possible I did not measure correctly. ;-) It is a 32 page
>booklet.
>
>-- 
>Cecil Westerhof
>Senior Software Engineer
>LinkedIn: http://www.linkedin.com/in/cecilwesterhof

Ah, I have been misunderstanding the question.  It's the booklet itself
that is 7 by 9, not the rasters (or whatever) you get when you first
scan the thing.  Sorry about that.

Laura


From tjreedy at udel.edu  Mon Jun  1 16:57:45 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 01 Jun 2015 16:57:45 -0400
Subject: [RELEASED] Python 3.5.0b2 is now available
In-Reply-To: <556C469A.2070806@mrabarnett.plus.com>
References: <556BE1A7.1090101@hastings.org>
 <556C469A.2070806@mrabarnett.plus.com>
Message-ID: <mkih15$rtr$1@ger.gmane.org>

On 6/1/2015 7:48 AM, MRAB wrote:

> I've just run "Windows x86 executable installer" and "Windows x86-64
> executable installer".
>
> While installing, the progress messages overwrote each other without
> clearing the background each time.

I saw something like that too.  Can you open a tracker issue with 
steve.dower nosy?

-- 
Terry Jan Reedy



From Cecil at decebal.nl  Mon Jun  1 17:04:12 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 01 Jun 2015 23:04:12 +0200
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <6YCdnb5rQbmojfHInZ2dnUU7-LednZ2d@giganews.com>
 <lf5y4k3onsm.fsf@ling.helsinki.fi> <mkh7c3$itb$1@speranza.aioe.org>
 <876177lncd.fsf@Equus.decebal.nl>
 <mailman.2.1433173497.13271.python-list@python.org>
 <87oakzjs4u.fsf@Equus.decebal.nl>
 <mailman.12.1433184186.13271.python-list@python.org>
 <87h9qrjj72.fsf@Equus.decebal.nl>
 <mailman.14.1433191370.13271.python-list@python.org>
Message-ID: <87a8wjjg37.fsf@Equus.decebal.nl>

Op Monday 1 Jun 2015 22:42 CEST schreef Laura Creighton:

> In a message of Mon, 01 Jun 2015 21:57:05 +0200, Cecil Westerhof
> writes:
>>> Truly? That's (very close to) 7 inch by 9 inch, 177.8 mm x 228.6
>>> mm and 7 by 9 is what pre-metric Britian called 'Small Post
>>> Quarto'. I wonder if this merely a coincidence, or does some
>>> software really still like this size? How very weird.
>>
>> Well, it is possible I did not measure correctly. ;-) It is a 32
>> page booklet.
>>
>> -- 
>> Cecil Westerhof
>> Senior Software Engineer
>> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>
> Ah, I have been misunderstanding the question. It's the booklet
> itself that is 7 by 9, not the rasters (or whatever) you get when
> you first scan the thing. Sorry about that.

No problem.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From zachary.ware+pylist at gmail.com  Mon Jun  1 17:05:47 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Mon, 1 Jun 2015 16:05:47 -0500
Subject: [RELEASED] Python 3.5.0b2 is now available
In-Reply-To: <mkih15$rtr$1@ger.gmane.org>
References: <556BE1A7.1090101@hastings.org>
 <556C469A.2070806@mrabarnett.plus.com> <mkih15$rtr$1@ger.gmane.org>
Message-ID: <CAKJDb-PDt=ivesVZOF87mAD4YhTcKQGf7poPVjsetvU3YpCKxQ@mail.gmail.com>

On Mon, Jun 1, 2015 at 3:57 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 6/1/2015 7:48 AM, MRAB wrote:
>
>> I've just run "Windows x86 executable installer" and "Windows x86-64
>> executable installer".
>>
>> While installing, the progress messages overwrote each other without
>> clearing the background each time.
>
>
> I saw something like that too.  Can you open a tracker issue with
> steve.dower nosy?

There is already at least one open issue (though I don't have the number handy).

-- 
Zach


From vern.muhr at gmail.com  Mon Jun  1 17:12:37 2015
From: vern.muhr at gmail.com (vern.muhr at gmail.com)
Date: Mon, 1 Jun 2015 14:12:37 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
Message-ID: <3096a7a0-12dd-4e4a-a239-1be3766a067c@googlegroups.com>

On Sunday, May 31, 2015 at 7:34:20 AM UTC-7, Eddilbert Macharia wrote:
> Hello All ,
> 
> I'm wrecking my head trying to understand. where the class object comes into play . 
> 
> Is it only meant to act as base class and does it mean there is an actual class called object in python which all the objects created using the class type inherit ?
> 
> i'm assuming the metaclass if simplified would look something like this :
> 
> type('dict', (object,),{})
> 
> And when we use the class type as a metaclass are we using the instance version of the class type or are we actually using the type class itself ?
> 
> Also when we say everything is an object in python, are we referring to the fact that everything is an instance of the class type or does it have to with the object class inherited ? 
> 
> As can be attested by using type() function as below :
> 
> >>> type(int)
> <class 'type'>
> >>> type(list)
> <class 'type'>
> >>> type(dict)
> <class 'type'>
> >>> type(type)
> <class 'type'>
> >>> type(object)
> <class 'type'>
> 
> From my understanding this means all of this are instances of the class type. which means the class type was used to create this instances.
> 
> Now if i look at the __bases__ of all this objects i get :
> 
> >>> type.__base__
> <class 'object'>
> >>> type.__bases__
> (<class 'object'>,)
> >>> dict.__bases__
> (<class 'object'>,)
> >>> list.__bases__
> (<class 'object'>,)
> >>> int.__bases__
> (<class 'object'>,)
> >>> object.__bases__
> ()
> 
> This tells me that all of this objects inherit from the class object which has nothing to do with them being instances.

It may be helpful to try "dir(object)" to see what methods object provides to all those other classes.

Best regards, Vern


From tjreedy at udel.edu  Mon Jun  1 17:32:40 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 01 Jun 2015 17:32:40 -0400
Subject: Zero [was Re: What is considered an "advanced" topic in Python?]
In-Reply-To: <CANc-5Uwba3DyMkEvaFLjzcE8+2qGYqTrjFp5DAiR5wPy9TOLbA@mail.gmail.com>
References: <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
 <87d21fahyi.fsf@elektro.pacujo.net>
 <556c4b0e$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <1jlomal1or023ian8rlaso1u5bnol52lag@4ax.com>
 <CANc-5Uwba3DyMkEvaFLjzcE8+2qGYqTrjFp5DAiR5wPy9TOLbA@mail.gmail.com>
Message-ID: <mkij2k$2nj$1@ger.gmane.org>

On 6/1/2015 9:14 AM, Skip Montanaro wrote:

> remove. Maybe you should just install
> a decent spam filter <http://www.spambayes.org/> or switch to Gmail,
> which has a functioning spam filter (unlike Yahoo...)

For me, Yahoo's spam filter is comparable to Gmail's.  (My udel account 
is actually handled by gmail.)  Either spam has decreased recently or 
both have gotten better at putting fewer things in the junk box for 
checking and tossing away more things.

-- 
Terry Jan Reedy



From greenbay.graham at gmail.com  Mon Jun  1 17:33:56 2015
From: greenbay.graham at gmail.com (greenbay.graham at gmail.com)
Date: Mon, 1 Jun 2015 14:33:56 -0700 (PDT)
Subject: Using a particular python binary with venv
Message-ID: <ca4f39e1-f02a-403f-b113-82e25201a2cd@googlegroups.com>

According to this https://docs.python.org/3.4/library/venv.html#module-venv 'Each virtual environment has its own Python binary (allowing creation of environments with various Python versions)' 
So how would I create a virtual environment using the venv module that has a Python 2.7 binary?
tia
g


From ian.g.kelly at gmail.com  Mon Jun  1 17:38:56 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 1 Jun 2015 15:38:56 -0600
Subject: Using a particular python binary with venv
In-Reply-To: <ca4f39e1-f02a-403f-b113-82e25201a2cd@googlegroups.com>
References: <ca4f39e1-f02a-403f-b113-82e25201a2cd@googlegroups.com>
Message-ID: <CALwzid=-KUE3zs8Mkto=jEVHCwnv2ubNbyeG7Lg3RmsE_yRjww@mail.gmail.com>

On Mon, Jun 1, 2015 at 3:33 PM,  <greenbay.graham at gmail.com> wrote:
> According to this https://docs.python.org/3.4/library/venv.html#module-venv 'Each virtual environment has its own Python binary (allowing creation of environments with various Python versions)'
> So how would I create a virtual environment using the venv module that has a Python 2.7 binary?

I wouldn't recommend trying to create a venv virtual environment using
Python 2.7, because venv was only added to the standard library in
Python 3.3. Use the third-party virtualenv instead.


From carl at oddbird.net  Mon Jun  1 17:42:33 2015
From: carl at oddbird.net (Carl Meyer)
Date: Mon, 01 Jun 2015 15:42:33 -0600
Subject: Using a particular python binary with venv
In-Reply-To: <ca4f39e1-f02a-403f-b113-82e25201a2cd@googlegroups.com>
References: <ca4f39e1-f02a-403f-b113-82e25201a2cd@googlegroups.com>
Message-ID: <556CD1C9.1010007@oddbird.net>

On 06/01/2015 03:33 PM, greenbay.graham at gmail.com wrote:
> According to this 
> https://docs.python.org/3.4/library/venv.html#module-venv 'Each 
> virtual environment has its own Python binary (allowing creation of 
> environments with various Python versions)'
>
> So how would I create a virtual environment using the venv module 
> that has a Python 2.7 binary?

You can't. The venv module only exists in and supports Python 3.3+.

If you need to support earlier Django versions, you'll need to use the
older "virtualenv" library instead, which looks very similar from the
end-user side. See https://virtualenv.pypa.io/en/latest/

Carl

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20150601/697eb788/attachment.sig>

From lac at openend.se  Mon Jun  1 17:45:39 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 01 Jun 2015 23:45:39 +0200
Subject: Using a particular python binary with venv
In-Reply-To: Message from greenbay.graham@gmail.com of "Mon,
 01 Jun 2015 14:33:56 -0700."
 <ca4f39e1-f02a-403f-b113-82e25201a2cd@googlegroups.com>
References: <ca4f39e1-f02a-403f-b113-82e25201a2cd@googlegroups.com>
Message-ID: <201506012145.t51LjdWx005544@fido.openend.se>

In a message of Mon, 01 Jun 2015 14:33:56 -0700, greenbay.graham at gmail.com writ
es:
>According to this https://docs.python.org/3.4/library/venv.html#module-venv 'Each virtual environment has its own Python binary (allowing creation of environments with various Python versions)' 
>So how would I create a virtual environment using the venv module that has a Python 2.7 binary?
>tia
>g
>-- 
>https://mail.python.org/mailman/listinfo/python-list

#If you haven't already got it
pip install virtualenv

#Then

cd where_I_want_to_work
virtualenv -p /usr/bin/python2.7 venv
source venv/bin/activate
pip install packages_i_need

# when you are done
deactivate

to get back to your system python.

Laura



From dreamingforward at gmail.com  Mon Jun  1 17:45:57 2015
From: dreamingforward at gmail.com (TheDoctor)
Date: Mon, 1 Jun 2015 14:45:57 -0700 (PDT)
Subject: Language design
In-Reply-To: <5230ff66$0$29988$c3e8da3$5496439d@news.astraweb.com>
References: <522eb795$0$29999$c3e8da3$5496439d@news.astraweb.com>
 <mailman.270.1378935062.5461.python-list@python.org>
 <5230ff66$0$29988$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <a703cea2-ef76-4a47-81ba-3ddcbfcd2c06@googlegroups.com>

On Wednesday, September 11, 2013 at 6:40:22 PM UTC-5, Steven D'Aprano wrote:
> On Wed, 11 Sep 2013 14:30:54 -0700, Mark Janssen wrote:
> 
> > 1) It tried to make Object the parent of every class.  
> 
> Tried, and succeeded.

Oh?  How about:

class superdict(dict):  
    """I'm going to extend the dict type to include extra methods."""

class mixin():
   """Here we go."""

What is the parent of mixin?

> > No one's close enough to God to make that work.
> 
> Non-sequitor. One doesn't need to be close to a deity to have a single 
> root of the object hierarchy.

Loud voice:  Boom!

> > 4?) It allowed
> > [reference] variables to be used as dict keys.  This creates a parsing
> > difficulty for me, mentally.  Keys should be direct, hashable values,
> > not hidden in a variable name.
> 
> I don't even understand what you are talking about here. "[reference] 
> variables"? What does that mean?

Nevermind.  I was wrong on this one.  My mental lexer threw an error on this one whenever ppl used a variable as a key.

Cheer!

Mark


From greenbay.graham at gmail.com  Mon Jun  1 18:07:25 2015
From: greenbay.graham at gmail.com (greenbay.graham at gmail.com)
Date: Mon, 1 Jun 2015 15:07:25 -0700 (PDT)
Subject: Using a particular python binary with venv
In-Reply-To: <mailman.19.1433194962.13271.python-list@python.org>
References: <ca4f39e1-f02a-403f-b113-82e25201a2cd@googlegroups.com>
 <mailman.19.1433194962.13271.python-list@python.org>
Message-ID: <fc1b77f4-cb0e-4f03-b383-00bbd15c1a02@googlegroups.com>

On Tuesday, 2 June 2015 09:43:37 UTC+12, Carl Meyer  wrote:
> On 06/01/2015 03:33 PM, orotau wrote:
> > According to this 
> > https://docs.python.org/3.4/library/venv.html#module-venv 'Each 
> > virtual environment has its own Python binary (allowing creation of 
> > environments with various Python versions)'
> >
> > So how would I create a virtual environment using the venv module 
> > that has a Python 2.7 binary?
> 
> You can't. The venv module only exists in and supports Python 3.3+.
> 
> If you need to support earlier Django versions, you'll need to use the
> older "virtualenv" library instead, which looks very similar from the
> end-user side. See https://virtualenv.pypa.io/en/latest/
> 
> Carl

Thanks Carl
I am guessing that the documentation needs to be altered then. I will submit a documentation bug...
cheers
g


From carl at oddbird.net  Mon Jun  1 18:19:16 2015
From: carl at oddbird.net (Carl Meyer)
Date: Mon, 01 Jun 2015 16:19:16 -0600
Subject: Using a particular python binary with venv
In-Reply-To: <fc1b77f4-cb0e-4f03-b383-00bbd15c1a02@googlegroups.com>
References: <ca4f39e1-f02a-403f-b113-82e25201a2cd@googlegroups.com>
 <mailman.19.1433194962.13271.python-list@python.org>
 <fc1b77f4-cb0e-4f03-b383-00bbd15c1a02@googlegroups.com>
Message-ID: <556CDA64.7010805@oddbird.net>

On 06/01/2015 04:07 PM, greenbay.graham at gmail.com wrote:
> On Tuesday, 2 June 2015 09:43:37 UTC+12, Carl Meyer  wrote:
>> On 06/01/2015 03:33 PM, orotau wrote:
>>> According to this 
>>> https://docs.python.org/3.4/library/venv.html#module-venv 'Each 
>>> virtual environment has its own Python binary (allowing creation of 
>>> environments with various Python versions)'
>>>
>>> So how would I create a virtual environment using the venv module 
>>> that has a Python 2.7 binary?
>>
>> You can't. The venv module only exists in and supports Python 3.3+.
>>
>> If you need to support earlier Django versions, you'll need to use the
>> older "virtualenv" library instead, which looks very similar from the
>> end-user side. See https://virtualenv.pypa.io/en/latest/
> 
> Thanks Carl
> I am guessing that the documentation needs to be altered then. I will submit a documentation bug...

I think the documentation is accurate as written (you _can_ create
environments with various Python versions, they just have to be versions
that include and support the venv module), but a clarification on the
supported Python versions in that sentence seems reasonable to me.

Carl

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20150601/27cfc6c4/attachment.sig>

From greenbay.graham at gmail.com  Mon Jun  1 18:23:00 2015
From: greenbay.graham at gmail.com (greenbay.graham at gmail.com)
Date: Mon, 1 Jun 2015 15:23:00 -0700 (PDT)
Subject: Using a particular python binary with venv
In-Reply-To: <fc1b77f4-cb0e-4f03-b383-00bbd15c1a02@googlegroups.com>
References: <ca4f39e1-f02a-403f-b113-82e25201a2cd@googlegroups.com>
 <mailman.19.1433194962.13271.python-list@python.org>
 <fc1b77f4-cb0e-4f03-b383-00bbd15c1a02@googlegroups.com>
Message-ID: <34390cbb-bc88-4cf7-a6f8-1ed13d818912@googlegroups.com>

http://bugs.python.org/issue24356


From python at mrabarnett.plus.com  Mon Jun  1 18:28:10 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 01 Jun 2015 23:28:10 +0100
Subject: [RELEASED] Python 3.5.0b2 is now available
In-Reply-To: <mkih15$rtr$1@ger.gmane.org>
References: <556BE1A7.1090101@hastings.org>
 <556C469A.2070806@mrabarnett.plus.com> <mkih15$rtr$1@ger.gmane.org>
Message-ID: <556CDC7A.6070302@mrabarnett.plus.com>

On 2015-06-01 21:57, Terry Reedy wrote:
> On 6/1/2015 7:48 AM, MRAB wrote:
>
>> I've just run "Windows x86 executable installer" and "Windows x86-64
>> executable installer".
>>
>> While installing, the progress messages overwrote each other without
>> clearing the background each time.
>
> I saw something like that too.  Can you open a tracker issue with
> steve.dower nosy?
>
Already reported as issue 24344.



From breamoreboy at yahoo.co.uk  Mon Jun  1 18:48:54 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 01 Jun 2015 23:48:54 +0100
Subject: Pyton re module and POSIX equivalence classes
In-Reply-To: <20150601152930.77f0a8fb@bigbox.christie.dr>
References: <20150601152930.77f0a8fb@bigbox.christie.dr>
Message-ID: <mkingq$8b0$1@ger.gmane.org>

On 01/06/2015 21:29, Tim Chase wrote:
> Is Python supposed to support POSIX "equivalence classes"?  I tried
> the following in Py2 and Py3:
>
>    >>> re.sub('[[=a=]]', 'A', 'a?????', re.U)
>    'a?????'
>
> which suggests that it doesn't (I would have expected "AAAAAA" as the
> result).
>
> Is there a way to get this behavior?
>
> I found that perl knows about them but treats them as an exception
> for now[1].  Supposedly GNU awk (and other GNU POSIXish tools)
> recognize character classes, as does vim.
>
> Thanks,
>
> -tkc
>
> [1]
> http://perldoc.perl.org/perlrecharclass.html
>

I wouldn't know directly as I tend to avoid them like the plague, but if 
not are they in the "new" regex module, see 
https://pypi.python.org/pypi/regex/2015.05.28 and/or 
http://bugs.python.org/issue2636 ???

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From bc at freeuk.com  Mon Jun  1 18:59:09 2015
From: bc at freeuk.com (BartC)
Date: Mon, 01 Jun 2015 23:59:09 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
Message-ID: <3t5bx.755107$dj1.564752@fx03.am4>

On 01/06/2015 13:03, Eddilbert Macharia wrote:
> Thank you all who have taken there time to assist me.
> Special thanks to Steven D'Aparo even though you didn't have two you have taken you time to go into details.
>
> I think i kind of understand now.
>
> Instead of python having data types like int, string, e.t.c it has two primitive types which are class type and class object which are created by python interpreter during its setup .using this two classes python is able to create some more data types
>
> In some context type means the data type an object is e.g. an int,string e.t.c.
>
> In the new class style, type and class sort of mean the same thing.

I'm developing a new language along the lines of Python, perhaps a brief 
description of how things are done there might help. Or just give a 
different perspective.

Objects in this language are tagged: there's a code attached to each 
indicating what kind of data is stored. These codes are integers, or 
enumerations, for example:

  Int =    1
  String = 2
  List =   3
  Class =  4

And the following are examples of object instances with their tags:

a is (Int,    42)
b is (String, "ABCXYZ")
c is (List,   [10,20,30,40])
d is (Class,  2 or <String>)

The last one might be tricky to grasp: it's really just a number, but 
one that represents a class or type (or tag). If printed, it could 
display <String> rather than just 2. (And when used to do something with 
the class, the 2 might be an index into a set of tables.)

d is not /the/ class itself, but just a reference to it (this is 
pseudo-code not Python):

print (b)                 =>  ABCXYZ
print (typeof(b))         =>  String <2>

print (d)                 =>  String <2>
print (typeof(d))         =>  Class  <4>
print (typeof(typeof(d))) =>  Class  <4>

In my own language, the connection between class and type is hazy (I've 
only just added classes so it needs to be hazy until I understand them 
more).

'Type' includes everything, including all the built-in types such as the 
tags above, but also user-defined classes. In fact classes are the 
mechanism used to define new types. But with both designated by an 
integer within the same band (eg. built-int types 1 to 20, user-defined 
classes 21 and up), it is easier not to have a strong distinction ... at 
the moment.

I haven't a root class yet that is the base of all the others. I don't 
think it's necessary internally to make things work. But it might be a 
useful concept in the language. Calling it 'object' however might give 
rise to confusion as 'object' is informally used to refer to instances. 
(I've never used OO but have picked up some of the jargon!)

(This is almost certainly not how Python does things. Although the 
Python language doesn't go into details as to its implementation 
provided the behaviour is correct.)

-- 
Bartc


From bretedward at yahoo.com  Mon Jun  1 19:29:17 2015
From: bretedward at yahoo.com (Bret Edwards)
Date: Mon, 1 Jun 2015 23:29:17 +0000 (UTC)
Subject: Create on Win and run on Win or Mac
Message-ID: <789045946.3628039.1433201357640.JavaMail.yahoo@mail.yahoo.com>


I would like to create a Python (stand-alone executable) program on a Windows machine and deploy the finished product on other Windows machines (of varying Windows OS versions) and 
also on Mac machines (of varying Mac OS versions.) The program would present a GUI that the user would interact with and would only have simple computations. 

I looks like I could do the creation work twice (once for Windows and once for Mac) and then also create separate install packages. However, I would like to only do the creation 
work once and do it on a Windows machine and then be able to deploy to either Win or Mac. I understand that will mean separate intstallation packages due to the different 
environments. 

Is this possible? I do not think that it is, but asking just to be sure.

If so, what tools do I use to do the packaging? Do I use py2app for the Mac?
Thanks,
Bret

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150601/e7dc7646/attachment.html>

From python at mrabarnett.plus.com  Mon Jun  1 19:47:11 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 02 Jun 2015 00:47:11 +0100
Subject: Pyton re module and POSIX equivalence classes
In-Reply-To: <mkingq$8b0$1@ger.gmane.org>
References: <20150601152930.77f0a8fb@bigbox.christie.dr>
 <mkingq$8b0$1@ger.gmane.org>
Message-ID: <556CEEFF.4060003@mrabarnett.plus.com>

On 2015-06-01 23:48, Mark Lawrence wrote:
> On 01/06/2015 21:29, Tim Chase wrote:
>> Is Python supposed to support POSIX "equivalence classes"?  I tried
>> the following in Py2 and Py3:
>>
>>    >>> re.sub('[[=a=]]', 'A', 'a?????', re.U)
>>    'a?????'
>>
>> which suggests that it doesn't (I would have expected "AAAAAA" as the
>> result).
>>
>> Is there a way to get this behavior?
>>
>> I found that perl knows about them but treats them as an exception
>> for now[1].  Supposedly GNU awk (and other GNU POSIXish tools)
>> recognize character classes, as does vim.
>>
>> Thanks,
>>
>> -tkc
>>
>> [1]
>> http://perldoc.perl.org/perlrecharclass.html
>>
>
> I wouldn't know directly as I tend to avoid them like the plague, but if
> not are they in the "new" regex module, see
> https://pypi.python.org/pypi/regex/2015.05.28 and/or
> http://bugs.python.org/issue2636 ???
>
The regex module has POSIX character classes [[:alpha:]], but not
POSIX equivalence classes.



From tjreedy at udel.edu  Mon Jun  1 19:51:47 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 01 Jun 2015 19:51:47 -0400
Subject: Create on Win and run on Win or Mac
In-Reply-To: <789045946.3628039.1433201357640.JavaMail.yahoo@mail.yahoo.com>
References: <789045946.3628039.1433201357640.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <mkir7g$vms$1@ger.gmane.org>

On 6/1/2015 7:29 PM, Bret Edwards via Python-list wrote:
>
> I would like to create a Python (stand-alone executable) program on a
> Windows machine and deploy the finished product on other Windows
> machines (of varying Windows OS versions) and
> also on Mac machines (of varying Mac OS versions.) The program would
> present a GUI that the user would interact with and would only have
> simple computations.
>
> I looks like I could do the creation work twice (once for Windows and
> once for Mac) and then also create separate install packages. However, I
> would like to only do the creation
> work once and do it on a Windows machine and then be able to deploy to
> either Win or Mac. I understand that will mean separate intstallation
> packages due to the different
> environments.
>
> Is this possible? I do not think that it is, but asking just to be sure.
>
> If so, what tools do I use to do the packaging? Do I use py2app for the Mac?

renpy at renpy.org lets one develop script for a python-based visual 
novel, simulation game (or similar visual app) and deploy on windows, 
linux, and mac.  Push a button and its creates 3 installation packages. 
  I have no idea how it does so, but it is open source.

I just checked and it now also deploys to android 2.3+ and ios6+ (in beta).

-- 
Terry Jan Reedy



From dreamingforward at gmail.com  Mon Jun  1 20:10:21 2015
From: dreamingforward at gmail.com (TheDoctor)
Date: Mon, 1 Jun 2015 17:10:21 -0700 (PDT)
Subject: Language design
In-Reply-To: <52329db3$0$29988$c3e8da3$5496439d@news.astraweb.com>
References: <522eb795$0$29999$c3e8da3$5496439d@news.astraweb.com>
 <mailman.270.1378935062.5461.python-list@python.org>
 <5230ff66$0$29988$c3e8da3$5496439d@news.astraweb.com>
 <mailman.287.1378946951.5461.python-list@python.org>
 <52315238$0$29999$c3e8da3$5496439d@news.astraweb.com>
 <mailman.338.1379042609.5461.python-list@python.org>
 <52329db3$0$29988$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <47479653-fde1-4369-b958-8a42c8fcfeeb@googlegroups.com>

On Friday, September 13, 2013 at 12:08:04 AM UTC-5, Steven D'Aprano wrote:
> On Thu, 12 Sep 2013 20:23:21 -0700, Mark Janssen wrote:
> which would be silly. Only somebody who doesn't understand how 
> inheritance works in Python would do that. There's simply no need for it, 
> and in fact it would be actively harmful for larger hierarchies.
> 
> >>> But wait is it the "base" (at the bottom of the hierarchy) or is it
> >>> the "parent" at the top?  You see, you, like everyone else has been
> >>> using these terms loosely, confusing yourself.
> >>
> >> Depends on whether I'm standing on my head or not.
> >>
> >> Or more importantly, it depends on whether I visualise my hierarchy
> >> going top->down or bottom->up. Both are relevant, and both end up with
> >> the *exact same hierarchy* with only the direction reversed.
> > 
> > Ha,  "only the direction reversed".  That little directionality that
> > you're passing by so blithely is the difference between whether you're
> > talking about galaxies or atoms.
> 
> It makes no difference whether I write:
> 
>     atoms -> stars -> galaxies
> 
> or
> 
>     galaxies <- stars <- atoms
> 
> nor does it make any difference if I write the chain starting at the top 
> and pointing down, or at the bottom and pointing up.

Yes it does.  Ford IS-A Car, but Car IS-A Ford?  No.  Try reordering that one.

I see we've missed each other with our limitations to ASCII text, in ways that would have never happened had these conversations occurred in person.  Re-reading my texts, I see that I can easily confuse myself.

> "not someone wanting to understand the limitations of python..." -- are 
> you aware that I started this thread?

Yes, but did you?  LOL.

Mark


From dreamingforward at gmail.com  Mon Jun  1 20:24:26 2015
From: dreamingforward at gmail.com (TheDoctor)
Date: Mon, 1 Jun 2015 17:24:26 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
Message-ID: <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>

On Monday, June 1, 2015 at 7:03:49 AM UTC-5, Eddilbert Macharia wrote:
> I think i kind of understand now. 
> 
> Instead of python having data types like int, string, e.t.c it has two primitive types which are class type and class object which are created by python interpreter during its setup .using this two classes python is able to create some more data types

Eddibert, don't let this crowd (Terry, Stephen, et al.) confuse you.  They are as confused about it as you are.  

The truth is, that they've told a lie to themselves (everything is an object) to hide some details FROM THEMSELVES in their model.  They climb a ladder of ambiguity anytime there's a new problem in their model and call it a "type", or they pull out the "metaclass" term (because no one really knows that that is, so they feel *pretty* safe).

A type is not an object in the same way an instantiated type is an object -- anymore than a blueprint for a building is the building itself.

The reason this confusion appears is for the exact reason I said on an earlier thread:  Python does not define classes in it's grammar -- so it can't account for the difference between an [instantiated] object and it's definition/type.

Mark


From skybuck2000 at hotmail.com  Mon Jun  1 20:29:30 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 02:29:30 +0200
Subject: Parser needed.
Message-ID: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>

Hello,

I need some kind of parser and some kind of way to access the data contained 
in a file like the one below:

(text file):

http://www.skybuck.org/Games/StartrekOnline/Parser/SpaceFleetAlertEnemyExample.demo

I am interested in learning the entity numbers:

"EntityRef <number>"

and their positions:

"
Pos <x>, <z>, <y>
"

I am also interested in the creation of these entities, especially their 
type:

"entityTypeEnum <type>"

So basically the parser should produce a few lists:

EntityNumber = []
EntityType = []
EntityPositionX = []
EntityPositionY = []
EntityPositionZ = []

Any extra information about entities is welcome.

The parser should be able to parse a textfile of somewhere between 20.000 
lines to 50.000 lines in about 1 to 2 seconds.

My environment is SikuliX 1.1

Bye,
  Skybuck. 



From rosuav at gmail.com  Mon Jun  1 20:32:51 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 2 Jun 2015 10:32:51 +1000
Subject: Everything is an object in python - object class and type class
In-Reply-To: <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
Message-ID: <CAPTjJmpV-cndp=CtOGTOc5-+DsUV3Q6Jr9k74U+gwXzEWhetvw@mail.gmail.com>

On Tue, Jun 2, 2015 at 10:24 AM, TheDoctor <dreamingforward at gmail.com> wrote:
> On Monday, June 1, 2015 at 7:03:49 AM UTC-5, Eddilbert Macharia wrote:
>> I think i kind of understand now.
>>
>> Instead of python having data types like int, string, e.t.c it has two primitive types which are class type and class object which are created by python interpreter during its setup .using this two classes python is able to create some more data types
>
> Eddibert, don't let this crowd (Terry, Stephen, et al.) confuse you.  They are as confused about it as you are.
>
> The truth is, that they've told a lie to themselves (everything is an object) to hide some details FROM THEMSELVES in their model.  They climb a ladder of ambiguity anytime there's a new problem in their model and call it a "type", or they pull out the "metaclass" term (because no one really knows that that is, so they feel *pretty* safe).
>
> A type is not an object in the same way an instantiated type is an object -- anymore than a blueprint for a building is the building itself.

And don't let dreamingforward confuse you, because he is flat-out
wrong. An integer is an object, because you can manipulate it as one.
A function is an object. A dictionary is an object. And a type is an
object too. There is no significant difference here.

ChrisA


From rxjwg98 at gmail.com  Mon Jun  1 20:55:00 2015
From: rxjwg98 at gmail.com (fl)
Date: Mon, 1 Jun 2015 17:55:00 -0700 (PDT)
Subject: Where is 'palindrome' defined?
In-Reply-To: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
Message-ID: <888b51eb-0ede-4120-a914-aeec45d6366d@googlegroups.com>

On Sunday, May 31, 2015 at 9:46:56 PM UTC-7, fl wrote:
> Hi,
> 
> When I search solution of reverse a string/number, I came across a short
> function online:
> 
> >>> def palindrome(num):
> 	return str(num) == str(num)[::-1]
> 
> I thought that it is a general function. And with the following variable:
> 
> >>> a
> '1234_5555'
> 
> >>> parlindrome(a)
> 
> Traceback (most recent call last):
>   File "<pyshell#126>", line 1, in <module>
>     parlindrome(a)
> NameError: name 'parlindrome' is not defined
> 
> 
> Then, I find that parlindrome is a special checking mirrored word.
> I use Python 2.7.9. Why does the error message show   
> 
> name 'parlindrome' is not defined
> 
> 
> 
> Thanks,

Thanks, I realize that it was spelled wrong. Now, with the correct spelling
the result is a 'False'. I have expected it gives reversed string. Is the 
function behaves correct or not?


Thanks,




>>> a='1234'
>>> def palindrome(num):
	return str(num) == str(num)[::-1]
>>> palindrome(a)
False
>>> palindrome("fread")
False


From skybuck2000 at hotmail.com  Mon Jun  1 20:55:30 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 02:55:30 +0200
Subject: Parser needed.
In-Reply-To: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
Message-ID: <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>

I will try and help for as far as possible, maybe I will end up writing it 
myself in the process.

The first problem seems to be, how to read a textfile into python all at 
once... and then perhaps process them per line.

Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Mon Jun  1 20:56:46 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 02:56:46 +0200
Subject: Parser needed.
In-Reply-To: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
Message-ID: <a37c1$556cff43$5419aafe$35393@news.ziggo.nl>

Some further information about the demo file:

It seems to be split up into "creation sections"

and "update" sections.

The update sections contain the positions.

The update sections also contain a reference number to the created entities.

(There is one empty creation section but that's because I cut it out... so 
it can be ignored for now).

Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Mon Jun  1 21:01:18 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:01:18 +0200
Subject: Parser needed.
In-Reply-To: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
Message-ID: <e722$556d0054$5419aafe$37839@news.ziggo.nl>

This can help with parsing the C like bracketted data:

http://stackoverflow.com/questions/1651487/python-parsing-bracketed-blocks

algo:

"
For each string in the array:
    Find the first '{'. If there is none, leave that string alone.
    Init a counter to 0.
    For each character in the string:
        If you see a '{', increment the counter.
        If you see a '}', decrement the counter.
        If the counter reaches 0, break.
    Here, if your counter is not 0, you have invalid input (unbalanced 
brackets)
    If it is, then take the string from the first '{' up to the '}' that put 
the
     counter at 0, and that is a new element in your array."Not sure what 
this is, but more points: Or this pyparsing version:>>> from pyparsing 
import nestedExpr
>>> txt = "{ { a } { b } { { { c } } } }"
>>>
>>> nestedExpr('{','}').parseString(txt).asList()
[[['a'], ['b'], [[['c']]]]]
>>> Not sure if I want a data structure like that... not sure what data 
>>> structure it is anyway... seems to be a list in a list in a list... 
>>> hmm.... Bye,  Skybuck. 



From dreamingforward at gmail.com  Mon Jun  1 21:02:29 2015
From: dreamingforward at gmail.com (TheDoctor)
Date: Mon, 1 Jun 2015 18:02:29 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <mailman.28.1433205179.13271.python-list@python.org>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <mailman.28.1433205179.13271.python-list@python.org>
Message-ID: <3ee816c0-b3f3-4a0f-b567-40c435a53e8d@googlegroups.com>

On Monday, June 1, 2015 at 7:33:11 PM UTC-5, Chris Angelico wrote:
> On Tue, Jun 2, 2015 at 10:24 AM, TheDoctor <dreamingforward at gmail.com> wrote:
> > A type is not an object in the same way an instantiated type is an object -- anymore than a blueprint for a building is the building itself.
> 
> And don't let dreamingforward confuse you, because he is flat-out
> wrong. An integer is an object, because you can manipulate it as one.

An integer in Python *acts* like an object, in some narrow sense can indeed be said to BE one.  But no one uses them like that in practice (purity was trying to beat practicality in the case of ints) -- they use them like ints, with the extra super-nice shortcuts of having them arbitrarily-sized.  And that was available before the type-class unification.

> A function is an object. A dictionary is an object. 

Yep.

> And a type is an
> object too. 

OOPs.

Mark


From skybuck2000 at hotmail.com  Mon Jun  1 21:02:31 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:02:31 +0200
Subject: Parser needed.
In-Reply-To: <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
Message-ID: <37cda$556d009d$5419aafe$38529@news.ziggo.nl>

This will probably help:

http://stackoverflow.com/questions/14676265/how-to-read-text-file-into-a-list-or-array-with-python

text_file = open("filename.dat", "r")
lines = text_file.readlines()
print lines
print len(lines)
text_file.close() Usually I like to consult officals docs though...Bye, 
Skybuck. 



From skybuck2000 at hotmail.com  Mon Jun  1 21:03:46 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:03:46 +0200
Subject: Parser needed.
In-Reply-To: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
Message-ID: <aef84$556d00e7$5419aafe$39141@news.ziggo.nl>

>From examining the demo file it seems each bracketted data sections if 
pre-header/pre-fixed with it's name.

So it's somewhat similar to a C structure:

structure
{

}

Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Mon Jun  1 21:04:38 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:04:38 +0200
Subject: Parser needed.
In-Reply-To: <aef84$556d00e7$5419aafe$39141@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <aef84$556d00e7$5419aafe$39141@news.ziggo.nl>
Message-ID: <173$556d011c$5419aafe$39616@news.ziggo.nl>

Except for the first bracket... it has no structure name.

Perhaps the demo filename could be used as structure name or just be left 
empty.

Bye,
  Skybuck.


"Skybuck Flying"  wrote in message 
news:aef84$556d00e7$5419aafe$39141 at news.ziggo.nl...

>From examining the demo file it seems each bracketted data sections if
pre-header/pre-fixed with it's name.

So it's somewhat similar to a C structure:

structure
{

}

Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Mon Jun  1 21:07:06 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:07:06 +0200
Subject: Parser needed.
In-Reply-To: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
Message-ID: <3d933$556d01b0$5419aafe$40968@news.ziggo.nl>

I really like this python help file, it has served me well so far:

http://www.tutorialspoint.com/python/python_functions.htm

It's unusually better than the rest on the net ;)

It matches nicely Sikuli's python's capabilities.

Bye,
  Skybuck :)


From sohcahtoa82 at gmail.com  Mon Jun  1 21:07:16 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Mon, 1 Jun 2015 18:07:16 -0700 (PDT)
Subject: Where is 'palindrome' defined?
In-Reply-To: <888b51eb-0ede-4120-a914-aeec45d6366d@googlegroups.com>
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
 <888b51eb-0ede-4120-a914-aeec45d6366d@googlegroups.com>
Message-ID: <2698673b-3867-41dd-bd02-18a6c768cadf@googlegroups.com>

On Monday, June 1, 2015 at 5:55:14 PM UTC-7, fl wrote:
> On Sunday, May 31, 2015 at 9:46:56 PM UTC-7, fl wrote:
> > Hi,
> > 
> > When I search solution of reverse a string/number, I came across a short
> > function online:
> > 
> > >>> def palindrome(num):
> > 	return str(num) == str(num)[::-1]
> > 
> > I thought that it is a general function. And with the following variable:
> > 
> > >>> a
> > '1234_5555'
> > 
> > >>> parlindrome(a)
> > 
> > Traceback (most recent call last):
> >   File "<pyshell#126>", line 1, in <module>
> >     parlindrome(a)
> > NameError: name 'parlindrome' is not defined
> > 
> > 
> > Then, I find that parlindrome is a special checking mirrored word.
> > I use Python 2.7.9. Why does the error message show   
> > 
> > name 'parlindrome' is not defined
> > 
> > 
> > 
> > Thanks,
> 
> Thanks, I realize that it was spelled wrong. Now, with the correct spelling
> the result is a 'False'. I have expected it gives reversed string. Is the 
> function behaves correct or not?
> 
> 
> Thanks,
> 
> 
> 
> 
> >>> a='1234'
> >>> def palindrome(num):
> 	return str(num) == str(num)[::-1]
> >>> palindrome(a)
> False
> >>> palindrome("fread")
> False

If you want the reversed string, then just use a[::-1].  Your palindrome function simply returns a boolean telling you whether or not the string you gave it is a palindrome.


From skybuck2000 at hotmail.com  Mon Jun  1 21:07:51 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:07:51 +0200
Subject: Parser needed.
In-Reply-To: <3d933$556d01b0$5419aafe$40968@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <3d933$556d01b0$5419aafe$40968@news.ziggo.nl>
Message-ID: <ad05d$556d01dc$5419aafe$41358@news.ziggo.nl>

Link looks a bit odd, home index link:

Nice python documentation/tutorial/help:

http://www.tutorialspoint.com/python/index.htm

Bye,
  Skybuck.


From skybuck2000 at hotmail.com  Mon Jun  1 21:10:29 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:10:29 +0200
Subject: Parser needed.
In-Reply-To: <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
Message-ID: <9f113$556d027a$5419aafe$42776@news.ziggo.nl>

Readlines is indeed documented as reading EOF:

http://www.tutorialspoint.com/python/file_methods.htm

Bye,
  skybuck.

"Skybuck Flying"  wrote in message 
news:37cda$556d009d$5419aafe$38529 at news.ziggo.nl...

This will probably help:

http://stackoverflow.com/questions/14676265/how-to-read-text-file-into-a-list-or-array-with-python

text_file = open("filename.dat", "r")
lines = text_file.readlines()
print lines
print len(lines)
text_file.close() Usually I like to consult officals docs though...Bye,
Skybuck. 



From python at mrabarnett.plus.com  Mon Jun  1 21:10:54 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 02 Jun 2015 02:10:54 +0100
Subject: Where is 'palindrome' defined?
In-Reply-To: <888b51eb-0ede-4120-a914-aeec45d6366d@googlegroups.com>
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
 <888b51eb-0ede-4120-a914-aeec45d6366d@googlegroups.com>
Message-ID: <556D029E.9070407@mrabarnett.plus.com>

On 2015-06-02 01:55, fl wrote:
> On Sunday, May 31, 2015 at 9:46:56 PM UTC-7, fl wrote:
>> Hi,
>>
>> When I search solution of reverse a string/number, I came across a short
>> function online:
>>
>> >>> def palindrome(num):
>> 	return str(num) == str(num)[::-1]
>>
>> I thought that it is a general function. And with the following variable:
>>
>> >>> a
>> '1234_5555'
>>
>> >>> parlindrome(a)
>>
>> Traceback (most recent call last):
>>   File "<pyshell#126>", line 1, in <module>
>>     parlindrome(a)
>> NameError: name 'parlindrome' is not defined
>>
>>
>> Then, I find that parlindrome is a special checking mirrored word.
>> I use Python 2.7.9. Why does the error message show
>>
>> name 'parlindrome' is not defined
>>
>>
>>
>> Thanks,
>
> Thanks, I realize that it was spelled wrong. Now, with the correct spelling
> the result is a 'False'. I have expected it gives reversed string. Is the
> function behaves correct or not?
>
>
It compares 'num' as a string to the reverse of that to test whether 
it's a palindrome,
so, yes, it's behaving correctly.

Perhaps its purpose would have been clearer if it had been called
"is_palindrome".

>
>>>> a='1234'
>>>> def palindrome(num):
> 	return str(num) == str(num)[::-1]
>>>> palindrome(a)
> False
>>>> palindrome("fread")
> False
>



From torriem at gmail.com  Mon Jun  1 21:12:12 2015
From: torriem at gmail.com (Michael Torrie)
Date: Mon, 01 Jun 2015 19:12:12 -0600
Subject: Parser needed.
In-Reply-To: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
Message-ID: <556D02EC.6070802@gmail.com>

On 06/01/2015 06:29 PM, Skybuck Flying wrote:
> The parser should be able to parse a textfile of somewhere between 20.000 
> lines to 50.000 lines in about 1 to 2 seconds.
> 
> My environment is SikuliX 1.1

I don't have any inclination to examine your input files, but you could
certainly mock up a parser fairly quickly with the fantastic pyparsing
module (you'll have to install it from your distro's package manager, or
through pip.

Once you have figure out how to parse the data you can fill in your data
structures as you go along.  Pyparsing has pretty good docs; just google
for it.  Usually first I get pyparsing working on my input file, then I
add code to each of the steps along the way to actually do something
with the data that has been parsed.

You could parse it manually using regular expressions if the data is
fairly structured and regular.




From dreamingforward at gmail.com  Mon Jun  1 21:15:15 2015
From: dreamingforward at gmail.com (TheDoctor)
Date: Mon, 1 Jun 2015 18:15:15 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <mailman.28.1433205179.13271.python-list@python.org>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <mailman.28.1433205179.13271.python-list@python.org>
Message-ID: <e7de5af9-1775-4ace-a5a8-e1481a3cc910@googlegroups.com>

On Monday, June 1, 2015 at 7:33:11 PM UTC-5, Chris Angelico wrote:
> And a type is an
> object too. There is no significant difference here.

Let me make this clearer to you, Chris, because I don't want you to have to suck it too, like the rest of this community.

A type is not an object.  You see it as one, because you are MENTALLY lexing your own code on the screen.  But python does not define a type.  It defines certain primitives, like strings and integers.  But it doesn't define what an OBJECT is.  So you and everyone else are beating around the bush trying to define something that the LANGUAGE ITSELF doesn't define.  So, don't make a claim about it.  

In Python 2.7 type(type) is type (not object), but isinstance(type, object) is true -- so it is ambiguous, even contradictory, in the language.

Boom.  Suck it.

Mark


From skybuck2000 at hotmail.com  Mon Jun  1 21:16:17 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:16:17 +0200
Subject: Parser needed.
In-Reply-To: <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
 <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
Message-ID: <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>

So far this is what I got... I like to name things for what they are so 
FileObject I like better than something abstract/weird like "text file", my 
code:

BotDemoFolder = "C:\\Games\\Startrek Online\\Startrek Online\\Cryptic 
Studios\\Star Trek Online\\Live\\demos"
BotDemoFile = "SpaceFleetAlert.demo"

def Main():
DemoFilePath = BotDemoFolder + "\\" + BotDemoFile

FileObject = open( DemoFilePath, "r")

DemoLines = FileObject.readlines()

print DemoLines
print len(Demolines)

FileObject.close()
return

Main()

So far it took multiple seconds to print the demo lines...

Hopefully that's just a "printing" issue to the console and not a processing 
limitation of python.

I know python/sikulix can do about 500.000 if statements per second.

I shall have to add some timing code, to rule out sikulix startup time...

Hmmm..

Bye,
  Skybuck.




From skybuck2000 at hotmail.com  Mon Jun  1 21:19:16 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:19:16 +0200
Subject: Parser needed.
In-Reply-To: <mailman.31.1433207544.13271.python-list@python.org>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <mailman.31.1433207544.13271.python-list@python.org>
Message-ID: <52c36$556d048a$5419aafe$44775@news.ziggo.nl>

"Michael Torrie"  wrote in message 
news:mailman.31.1433207544.13271.python-list at python.org...

On 06/01/2015 06:29 PM, Skybuck Flying wrote:
> The parser should be able to parse a textfile of somewhere between 20.000
> lines to 50.000 lines in about 1 to 2 seconds.
>
> My environment is SikuliX 1.1

"
I don't have any inclination to examine your input files, but you could
certainly mock up a parser fairly quickly with the fantastic pyparsing
module (you'll have to install it from your distro's package manager, or
through pip.

Once you have figure out how to parse the data you can fill in your data
structures as you go along.  Pyparsing has pretty good docs; just google
for it.  Usually first I get pyparsing working on my input file, then I
add code to each of the steps along the way to actually do something
with the data that has been parsed.

You could parse it manually using regular expressions if the data is
fairly structured and regular.
"


It seems the data file is structure as:
{


<structure name>
{
   <SomeField> <SomeData>
    <another structure name>
   {
      <SomeField> <SomeData>
   }
}



}

Example:

{

createdEnts
{
  EntityRef 29294664
  ContainerID 65086
  EntitySendDistance 1500.000000
  entityTypeEnum ENTITYCRITTER

  CostumeV5
  {
   hReferencedCostume Loot_Space_Common_01
  }
}



}

How hard would it be to encode that into pyparser ?

Bye,
  Skybuck. 



From rxjwg98 at gmail.com  Mon Jun  1 21:26:57 2015
From: rxjwg98 at gmail.com (fl)
Date: Mon, 1 Jun 2015 18:26:57 -0700 (PDT)
Subject: How to use an iterator?
Message-ID: <0857360d-3934-4e92-8c28-5fb9732ec5da@googlegroups.com>

Hi,


I read the online tutorial on iterator:

https://docs.python.org/2/library/itertools.html

I have no idea on how to use this one:

itertools.count(start=0, step=1)

BTW, I am using Python 2.7.9 on Windows 7.



I even input the following:

def count(start=0, step=1):
    # count(10) --> 10 11 12 13 14 ...
    # count(2.5, 0.5) -> 2.5 3.0 3.5 ...
    n = start
    while True:
        yield n
        n += step


>>> xc=count(10)
>>> xc
<generator object count at 0x02BA9AD0>


How to use this generator? 


Thanks,


From skybuck2000 at hotmail.com  Mon Jun  1 21:27:34 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:27:34 +0200
Subject: Parser needed.
In-Reply-To: <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
 <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
 <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>
Message-ID: <e7e10$556d067b$5419aafe$48554@news.ziggo.nl>

Ok, so far so good, a little start has been made.

Text file is read into lines... I am not so sure if this is a good idea...

Maybe it's easier if the entire file is one gigant array of characters 
instead of fragmented lines.

However I don't know yet exactly how to read as one gigant array of 
characters.

I don't really like this line approach but maybe it's nice:

But I am not faced with a new problem: "How to process individually lines".

So this solution has created more problems than it solves :)

BotDemoFolder = "C:\\Games\\Startrek Online\\Startrek Online\\Cryptic 
Studios\\Star Trek Online\\Live\\demos"
BotDemoFile = "SpaceFleetAlert.demo"

import time

def ParseDemoLines( ParaLines ):
print "Parsing " + str( len(ParaLines) ) + " lines."

for LineIndex in range(0, len(ParaLines)):
  if "{" in ParaLines[LineIndex]: # how to process a line.. hmmm...
   print "yup"
return

def Main():
DemoFilePath = BotDemoFolder + "\\" + BotDemoFile

FileObject = open( DemoFilePath, "r")

DemoLines = FileObject.readlines()

ParseDemoLines( DemoLines )

# print DemoLines

FileObject.close()
return

print "program started"

Tick1 = time.time()
Main()
Tick2 = time.time()

Seconds = Tick2 - Tick1

print "Time in seconds: " + str(Seconds)

print "program finished"

Bye,
  Skybuck.




From skybuck2000 at hotmail.com  Mon Jun  1 21:29:03 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:29:03 +0200
Subject: Parser needed.
In-Reply-To: <e7e10$556d067b$5419aafe$48554@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
 <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
 <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>
 <e7e10$556d067b$5419aafe$48554@news.ziggo.nl>
Message-ID: <bf9f0$556d06d4$5419aafe$49289@news.ziggo.nl>

Since the file is probably ascii... not sure... I might come away with:

"
file.read([size])
Reads at most size bytes from the file (less if the read hits EOF before 
obtaining size bytes).

"

The doc does not mention is size is optionally... I will try and leave it 
out, see what happens, otherwise a big number will have to be given or so... 
or perhaps retrieve file size and add it.



Bye,

  Skybuck.






From rosuav at gmail.com  Mon Jun  1 21:30:34 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 2 Jun 2015 11:30:34 +1000
Subject: Everything is an object in python - object class and type class
In-Reply-To: <e7de5af9-1775-4ace-a5a8-e1481a3cc910@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <mailman.28.1433205179.13271.python-list@python.org>
 <e7de5af9-1775-4ace-a5a8-e1481a3cc910@googlegroups.com>
Message-ID: <CAPTjJmqQoMuU85pvAV8CW4JQhweq_6hDDnm0HMQwViu-47mOuw@mail.gmail.com>

On Tue, Jun 2, 2015 at 11:15 AM, TheDoctor <dreamingforward at gmail.com> wrote:
> Let me make this clearer to you, Chris, because I don't want you to have to suck it too, like the rest of this community.
>
> A type is not an object.  You see it as one, because you are MENTALLY lexing your own code on the screen.  But python does not define a type.  It defines certain primitives, like strings and integers.  But it doesn't define what an OBJECT is.  So you and everyone else are beating around the bush trying to define something that the LANGUAGE ITSELF doesn't define.  So, don't make a claim about it.
>
> In Python 2.7 type(type) is type (not object), but isinstance(type, object) is true -- so it is ambiguous, even contradictory, in the language.
>

I'm not saying this for dreamingforward's benefit, as he clearly isn't
interested in learning, but in case the OP is confused: Checking the
type of something doesn't tell you what it isn't. The isinstance check
is true, because type is an instance of type, which is a subclass of
object, therefore type is an instance of object. Similarly, you could
check this:

>>> issubclass(type, object)
True

So you can see that a type IS an object.

ChrisA


From skybuck2000 at hotmail.com  Mon Jun  1 21:31:07 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:31:07 +0200
Subject: Parser needed.
In-Reply-To: <bf9f0$556d06d4$5419aafe$49289@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
 <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
 <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>
 <e7e10$556d067b$5419aafe$48554@news.ziggo.nl>
 <bf9f0$556d06d4$5419aafe$49289@news.ziggo.nl>
Message-ID: <498a6$556d0751$5419aafe$50445@news.ziggo.nl>

Yes this will work:

DemoChars = FileObject.read()

I think this is a cleaner solution. EOL can be ignored and focusses on { } 
and stuff like that... when extracting information EOL could be used as 
well.

Bye,
  Skybuck. 



From torriem at gmail.com  Mon Jun  1 21:31:11 2015
From: torriem at gmail.com (Michael Torrie)
Date: Mon, 01 Jun 2015 19:31:11 -0600
Subject: Parser needed.
In-Reply-To: <52c36$556d048a$5419aafe$44775@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <mailman.31.1433207544.13271.python-list@python.org>
 <52c36$556d048a$5419aafe$44775@news.ziggo.nl>
Message-ID: <556D075F.1020605@gmail.com>

On 06/01/2015 07:19 PM, Skybuck Flying wrote:
> How hard would it be to encode that into pyparser ?

Check out the docs and you probably will get an idea.  The only real way
to find out is to try it.

Is this file from a certain program?  If so, it's possible someone has
already written a python library for reading it.



From skybuck2000 at hotmail.com  Mon Jun  1 21:34:10 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:34:10 +0200
Subject: Parser needed.
In-Reply-To: <498a6$556d0751$5419aafe$50445@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
 <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
 <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>
 <e7e10$556d067b$5419aafe$48554@news.ziggo.nl>
 <bf9f0$556d06d4$5419aafe$49289@news.ziggo.nl>
 <498a6$556d0751$5419aafe$50445@news.ziggo.nl>
Message-ID: <32bc3$556d0808$5419aafe$52078@news.ziggo.nl>

Nice char based code:

BotDemoFolder = "C:\\Games\\Startrek Online\\Startrek Online\\Cryptic 
Studios\\Star Trek Online\\Live\\demos"
BotDemoFile = "SpaceFleetAlert.demo"

import time

def ParseDemoLines( ParaLines ):
print "Parsing " + str( len(ParaLines) ) + " lines."

for LineIndex in range(0, len(ParaLines)):
  if "{" in ParaLines[LineIndex]: # how to process a line.. hmmm...
   print "yup"
return

def ParseDemoChars( ParaChars ):
print "Parsing " + str( len(ParaChars) ) + " chars."

for CharIndex in range(0, len(ParaChars)):
  if ParaChars[CharIndex] == "{":
   print "yup"
return

def Main():
DemoFilePath = BotDemoFolder + "\\" + BotDemoFile

FileObject = open( DemoFilePath, "r")

# DemoLines = FileObject.readlines()
# ParseDemoLines( DemoLines )

DemoChars = FileObject.read()
ParseDemoChars( DemoChars )

FileObject.close()
return

print "program started"

Tick1 = time.time()
Main()
Tick2 = time.time()

Seconds = Tick2 - Tick1

print "Time in seconds: " + str(Seconds)

print "program finished"






From rgaddi at technologyhighland.invalid  Mon Jun  1 21:34:37 2015
From: rgaddi at technologyhighland.invalid (Rob Gaddi)
Date: Tue, 2 Jun 2015 01:34:37 +0000 (UTC)
Subject: How to use an iterator?
References: <0857360d-3934-4e92-8c28-5fb9732ec5da@googlegroups.com>
Message-ID: <mkj17c$6fm$1@dont-email.me>

On Mon, 01 Jun 2015 18:26:57 -0700, fl wrote:

> Hi,
> 
> 
> I read the online tutorial on iterator:
> 
> https://docs.python.org/2/library/itertools.html
> 
> I have no idea on how to use this one:
> 
> itertools.count(start=0, step=1)
> 
> BTW, I am using Python 2.7.9 on Windows 7.
> 
> 
> 
> I even input the following:
> 
> def count(start=0, step=1):
>     # count(10) --> 10 11 12 13 14 ... # count(2.5, 0.5) -> 2.5 3.0 3.5
>     ...
>     n = start while True:
>         yield n n += step
> 
> 
>>>> xc=count(10)
>>>> xc
> <generator object count at 0x02BA9AD0>
> 
> 
> How to use this generator?
> 
> 
> Thanks,

Like this:

for xc in count(10):
    print(xc)

Then you hit Ctrl-C to kill the program because your generator is 
infinite, but the principle is sound.

Infinite iterators are of limited, but non-zero, use.  Generally you use 
them with some other function performing the bounding, such as:

for n, xc in zip(range(10, 0, -1), count(10)):
    print(n, xc)

range is a generator, similar to your count, but it's finite.  It'll 
output the numbers 10 through 1 in descending order.

zip is also a generator; it takes from each iterator input, bundles them 
together, and returns a tuple of all the inputs, until any one of the 
input iterators is exhausted.  Since range is finite, the zip becomes 
finite, and the infinitude of your generator isn't an issue.

You could of course turn this on its head and ask for

for n, xc in zip(count(0), someotheriterable):
    print(n, xc)

And use your count infinite iterator to enumerate the (hopefully finite) 
elements of someotheriterable.  This is common enough that the built-in 
enumerate function does exactly that.

Make sense yet?

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.


From skybuck2000 at hotmail.com  Mon Jun  1 21:35:48 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 03:35:48 +0200
Subject: Parser needed.
In-Reply-To: <mailman.33.1433208676.13271.python-list@python.org>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <mailman.31.1433207544.13271.python-list@python.org>
 <52c36$556d048a$5419aafe$44775@news.ziggo.nl>
 <mailman.33.1433208676.13271.python-list@python.org>
Message-ID: <25a48$556d086a$5419aafe$52911@news.ziggo.nl>

On 06/01/2015 07:19 PM, Skybuck Flying wrote:
> How hard would it be to encode that into pyparser ?

"
Check out the docs and you probably will get an idea.  The only real way
to find out is to try it.

Is this file from a certain program?  If so, it's possible someone has
already written a python library for reading it.
"

It's from a game called Star Trek Online, I think there is a C# parser for 
it.

The ammount of data I need from this file is very limited.

I don't want to spent too much time on a solution.

I have perhaps today to try and get a solution working ;)

So no time to learn complex pyparsers ?

If you can provide a simple example I might give it a shot ;)

Otherwise I try to wing my own.

Your help might be usefull later on if my own parser sux, fails or is too 
slow ;)

Bye,
  Skybuck. 



From joel.goldstick at gmail.com  Mon Jun  1 21:41:47 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Mon, 1 Jun 2015 21:41:47 -0400
Subject: Parser needed.
In-Reply-To: <498a6$556d0751$5419aafe$50445@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
 <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
 <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>
 <e7e10$556d067b$5419aafe$48554@news.ziggo.nl>
 <bf9f0$556d06d4$5419aafe$49289@news.ziggo.nl>
 <498a6$556d0751$5419aafe$50445@news.ziggo.nl>
Message-ID: <CAPM-O+z00Sc-U-RvE9y8gxeDZph+ZhgX4GWzwZQz0V29Lyf7HQ@mail.gmail.com>

On Mon, Jun 1, 2015 at 9:31 PM, Skybuck Flying <skybuck2000 at hotmail.com> wrote:
> Yes this will work:
>
> DemoChars = FileObject.read()
>
> I think this is a cleaner solution. EOL can be ignored and focusses on { }
> and stuff like that... when extracting information EOL could be used as
> well.
>
>
> Bye,
>  Skybuck.
> --
> https://mail.python.org/mailman/listinfo/python-list

This is kind of a q and a mailing list.  I'm baffled by all of your
posts, that seem to be a conversation with yourself.  Do you have a
question you need help with?

-- 
Joel Goldstick
http://joelgoldstick.com


From skybuck2000 at hotmail.com  Mon Jun  1 22:02:14 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 04:02:14 +0200
Subject: Parser needed.
In-Reply-To: <32bc3$556d0808$5419aafe$52078@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
 <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
 <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>
 <e7e10$556d067b$5419aafe$48554@news.ziggo.nl>
 <bf9f0$556d06d4$5419aafe$49289@news.ziggo.nl>
 <498a6$556d0751$5419aafe$50445@news.ziggo.nl>
 <32bc3$556d0808$5419aafe$52078@news.ziggo.nl>
Message-ID: <10414$556d0e9c$5419aafe$1840@news.ziggo.nl>

I am trying to rush it a bit but things are starting to go to hell.

SikuliX needs to be restarted continously to deal with bugs.

Plus Eclipse breakpoints cannot be set.

So both IDEs totally suck for any complex code development.

Plus I can already tell this is going to be insanely slow in python for some 
reason.

I am starting to have serious doubts for python as a parser platform:

This simple piece of code already takes 10 seconds for just 120 characters ? 
HOLYSHIT ?!

Bad sign.

BotDemoFolder = "C:\\Games\\Startrek Online\\Startrek Online\\Cryptic 
Studios\\Star Trek Online\\Live\\demos"
BotDemoFile = "SpaceFleetAlertEnemyExample.demo"


import time

def ParseDemoLines( ParaLines ):
print "Parsing " + str( len(ParaLines) ) + " lines."

for LineIndex in range(0, len(ParaLines)):
  if "{" in ParaLines[LineIndex]: # how to process a line.. hmmm...
   print "yup"
return

def ParseStructureOpen( ParaChars, CharIndex ):
# find eol before it and re-read characters until eol this is the structure 
name
while (CharIndex >= 0) and (CharIndex < len(ParaChars)) and 
(ParaChars[CharIndex] <> "\n"):
  print "CharIndex: " + str(CharIndex)
  CharIndex = CharIndex - 1


# re-read characters for structure name
StructureName = ""
while (CharIndex < len(ParaChars)) and (ParaChars[CharIndex] <> "\n"):
  print "CharIndex: " + str(CharIndex)
  StructureName = StructureName + ParaChars[CharIndex]
  CharIndex = CharIndex + 1

print StructureName
return

def ParseStructureClose( ParaChars, CharIndex ):
pass
return

def ParseDemoChars( ParaChars ):
print "Parsing " + str( len(ParaChars) ) + " chars."
for CharIndex in range(0, len(ParaChars)):
  print "Parsing char index: " + str(CharIndex)
  if ParaChars[CharIndex] == "{":
   ParseStructureOpen( ParaChars, CharIndex )
  elif ParaChars[CharIndex] == "}":
   ParseStructureClose( ParaChars, CharIndex )
return

def Main():
DemoFilePath = BotDemoFolder + "\\" + BotDemoFile

FileObject = open( DemoFilePath, "r")

# DemoLines = FileObject.readlines()
# ParseDemoLines( DemoLines )

DemoChars = FileObject.read()
ParseDemoChars( DemoChars )

FileObject.close()
return

print "program started"

Tick1 = time.time()
Main()
Tick2 = time.time()

Seconds = Tick2 - Tick1

print "Time in seconds: " + str(Seconds)

print "program finished"


Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Mon Jun  1 22:04:40 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 04:04:40 +0200
Subject: Parser needed.
In-Reply-To: <mailman.34.1433209315.13271.python-list@python.org>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
 <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
 <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>
 <e7e10$556d067b$5419aafe$48554@news.ziggo.nl>
 <bf9f0$556d06d4$5419aafe$49289@news.ziggo.nl>
 <498a6$556d0751$5419aafe$50445@news.ziggo.nl>
 <mailman.34.1433209315.13271.python-list@python.org>
Message-ID: <65043$556d0f2e$5419aafe$3149@news.ziggo.nl>



"Joel Goldstick"  wrote in message 
news:mailman.34.1433209315.13271.python-list at python.org...

On Mon, Jun 1, 2015 at 9:31 PM, Skybuck Flying <skybuck2000 at hotmail.com> 
wrote:
> Yes this will work:
>
> DemoChars = FileObject.read()
>
> I think this is a cleaner solution. EOL can be ignored and focusses on { }
> and stuff like that... when extracting information EOL could be used as
> well.
>
>
> Bye,
>  Skybuck.
> --
> https://mail.python.org/mailman/listinfo/python-list

"
This is kind of a q and a mailing list.  I'm baffled by all of your
posts, that seem to be a conversation with yourself.  Do you have a
question you need help with?
"

All of my postings contain things I run into.

Missing documentation, python limitations, and other problems.

Algo design considerations etc.

What I need help with is a parser for the file mentioned in the link.

So far it looks bad for python.

I may give the parser frameworks a try to see if they are speedier then 120 
chars per second ?! :P :)

Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Mon Jun  1 22:11:09 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 04:11:09 +0200
Subject: Parser needed.
In-Reply-To: <25a48$556d086a$5419aafe$52911@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <mailman.31.1433207544.13271.python-list@python.org>
 <52c36$556d048a$5419aafe$44775@news.ziggo.nl>
 <mailman.33.1433208676.13271.python-list@python.org>
 <25a48$556d086a$5419aafe$52911@news.ziggo.nl>
Message-ID: <9dd74$556d10b2$5419aafe$6564@news.ziggo.nl>

I tried running this in SikuliX 1.1

from pyparsing import Word, alphas

greet = Word( alphas ) + "," + Word( alphas ) + "!"
greeting = greet.parseString( "Hello, World!" )
print greeting

Doesn't seem to run pyparser module missing.

Another reason why not to use it, unfortunately.

Not sure if it's possible to somehow include it with SikuliX.

This might mean the end for this little experiment.

I think SikuliX/Jython might be too slow for parsing purposes which is kinda 
odd... since it can do 500.000 if statements last time I tested it... but 
when it comes to dealing with characters it seems to turn into a pile of 
shit ?

Hmmm.. maybe unicode processing has something to do with it :)

Perhaps I should write a small little test program to test this 
"python/jython" is a "piece of char shit"  theory.

Yes interesting test.

Bye,
  Skybuck.




From dreamingforward at gmail.com  Mon Jun  1 22:12:33 2015
From: dreamingforward at gmail.com (TheDoctor)
Date: Mon, 1 Jun 2015 19:12:33 -0700 (PDT)
Subject: Message passing syntax for objects | OOPv2
In-Reply-To: <518b3699$0$11120$c3e8da3@news.astraweb.com>
References: <CAMjeLr_eJf1Aboz=2+oti4TOV=oejUYVo7K4zks+Ck4CSHaqpg@mail.gmail.com>
 <5167F327.3030803@nedbatchelder.com>
 <CAMjeLr8+pr12+0mN0SOrtg9ZTkNq=PiqxWyqB8CxpJ0BT=sxJA@mail.gmail.com>
 <mailman.1475.1368066966.3114.python-list@python.org>
 <518b3699$0$11120$c3e8da3@news.astraweb.com>
Message-ID: <687e21ac-be50-4217-954d-ef37e203e366@googlegroups.com>

On Thursday, May 9, 2013 at 12:39:37 AM UTC-5, Steven D'Aprano wrote:
> On Wed, 08 May 2013 19:35:58 -0700, Mark Janssen wrote:
> 
> > Long story short: the lambda
> > calculus folks have to split from the Turing machine folks.
> >  These models of computation should not use the same language.  Their
> > computation models are too radically different.  
> 
> Their computation models are exactly equivalent.

No.  Church's thesis was a thesis.  So, yes in theory they can be made to be equivalent.  In practice, it's not.  Practicality beats purity, remember?

> This is like saying that Cartesian coordinates and polar coordinates are 
> so radically different that they cannot possibly both describe the same 
> space.

Good analogy, they in theory have the same descriptive potential.  In practice, you never do it.
 
Mark
Ho Hum...getting back to some old threads.


From rxjwg98 at gmail.com  Mon Jun  1 22:14:18 2015
From: rxjwg98 at gmail.com (fl)
Date: Mon, 1 Jun 2015 19:14:18 -0700 (PDT)
Subject: What use of string module?
Message-ID: <d3bf3182-d70e-4160-b600-733cbda748e6@googlegroups.com>

Hi,

I read the online help about string. It lists string constants, string 
formatting, template strings and string functions. After reading these,
I am still puzzled about how to use the string module. 

Could you show me a few example about this module?

Thanks


From skybuck2000 at hotmail.com  Mon Jun  1 22:20:49 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 04:20:49 +0200
Subject: Parser needed.
In-Reply-To: <9dd74$556d10b2$5419aafe$6564@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <mailman.31.1433207544.13271.python-list@python.org>
 <52c36$556d048a$5419aafe$44775@news.ziggo.nl>
 <mailman.33.1433208676.13271.python-list@python.org>
 <25a48$556d086a$5419aafe$52911@news.ziggo.nl>
 <9dd74$556d10b2$5419aafe$6564@news.ziggo.nl>
Message-ID: <40cd2$556d12f7$5419aafe$8907@news.ziggo.nl>

Test program shows, python/sikulix can handle about 3m chars per second so I 
must continue with a parser attempt.

# test char processing speed.
# test if sikulix 1.1, python, jython/whatever is a piece of shit when it 
comes to parsing dealing with characters, unicode processing bottleneck 
theory.

import time
import random

MaxChars = 3000000

MinChar = 33
MaxChar = 200

Chars = []

def GenerateChars():
vRandomChar = ""
for vIndex in range(0, MaxChars):
  vRandomChar = chr(random.randrange(MinChar,MaxChar))
  Chars.append(vRandomChar)
return

def ProcessChars():
Result = 0
for vIndex in range(0, MaxChars):
  if Chars[vIndex] == chr(45):
   Result = Result + 1
return Result

Tick1 = time.time()
GenerateChars()
Tick2 = time.time()
Seconds = Tick2 - Tick1
print "GenerateChars seconds: " + str(Seconds)

Tick1 = time.time()
ProcessChars()
Tick2 = time.time()
Seconds = Tick2 - Tick1
print "ProcessChars seconds: " + str(Seconds)



#print Chars




From skybuck2000 at hotmail.com  Mon Jun  1 22:22:02 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 04:22:02 +0200
Subject: Parser needed.
In-Reply-To: <10414$556d0e9c$5419aafe$1840@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
 <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
 <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>
 <e7e10$556d067b$5419aafe$48554@news.ziggo.nl>
 <bf9f0$556d06d4$5419aafe$49289@news.ziggo.nl>
 <498a6$556d0751$5419aafe$50445@news.ziggo.nl>
 <32bc3$556d0808$5419aafe$52078@news.ziggo.nl>
 <10414$556d0e9c$5419aafe$1840@news.ziggo.nl>
Message-ID: <7a209$556d133f$5419aafe$9322@news.ziggo.nl>

Test program shows, python/sikulix can handle about 3m chars per second so I 
must continue with a parser attempt.

Apperently there is some bottleneck/slowdown in posted code...

I may have to slow down a bit take more time... and perhaps use global vars 
or so... instead of parameters.

Maybe that will help, not sure.

Kinda strange...  how test program is super fast in other thread and this 
one is slow.

Maybe it's the printing that's slow... Hmm...

I shall test print speed instead to figure this out.

Bye,
  Skybuck.



From skybuck2000 at hotmail.com  Mon Jun  1 22:27:16 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 04:27:16 +0200
Subject: Parser needed.
In-Reply-To: <7a209$556d133f$5419aafe$9322@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
 <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
 <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>
 <e7e10$556d067b$5419aafe$48554@news.ziggo.nl>
 <bf9f0$556d06d4$5419aafe$49289@news.ziggo.nl>
 <498a6$556d0751$5419aafe$50445@news.ziggo.nl>
 <32bc3$556d0808$5419aafe$52078@news.ziggo.nl>
 <10414$556d0e9c$5419aafe$1840@news.ziggo.nl>
 <7a209$556d133f$5419aafe$9322@news.ziggo.nl>
Message-ID: <529c5$556d147a$5419aafe$12290@news.ziggo.nl>

Apperently print speed has some exponential time issue in SikuliX kinda 
weird.

But at least processing is high... however, debug console/print is only 
thing I got to diagnose problems...

Could get time-wise nasty... hmmm...

Perhaps I should make a small little parser in Delphi and port it to Python 
instead...

But it a bit risky if not using python only code hmm...

# test print speed.

# 1000 prints costs 3 seconds
# 10.000 prints costed 60 seconds.

# this is weird indeed, seems to be some kind of exponential growth.

import time

MaxPrints = 10000
def PrintMeARiver():
for vIndex in range(0, MaxPrints):
  print str(vIndex)
return

def Main():
PrintMeARiver()
return

print "program started"
Tick1 = time.time()
Main()
Tick2 = time.time()
Seconds = Tick2 - Tick1
print "Seconds: " + str(Seconds)
print "program finished"


Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Mon Jun  1 22:39:01 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 04:39:01 +0200
Subject: Parser needed.
In-Reply-To: <529c5$556d147a$5419aafe$12290@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <35ec1$556cfef8$5419aafe$34765@news.ziggo.nl>
 <37cda$556d009d$5419aafe$38529@news.ziggo.nl>
 <9f113$556d027a$5419aafe$42776@news.ziggo.nl>
 <f15f4$556d03d6$5419aafe$44206@news.ziggo.nl>
 <e7e10$556d067b$5419aafe$48554@news.ziggo.nl>
 <bf9f0$556d06d4$5419aafe$49289@news.ziggo.nl>
 <498a6$556d0751$5419aafe$50445@news.ziggo.nl>
 <32bc3$556d0808$5419aafe$52078@news.ziggo.nl>
 <10414$556d0e9c$5419aafe$1840@news.ziggo.nl>
 <7a209$556d133f$5419aafe$9322@news.ziggo.nl>
 <529c5$556d147a$5419aafe$12290@news.ziggo.nl>
Message-ID: <9f47f$556d173a$5419aafe$18556@news.ziggo.nl>

Perhaps the list.append operation is the cause of the slowdown, or perhaps 
my algo is flawed.

No idea yet with slowdown of parser code so far.

Instead of trying to debug everything going to add some time outputs to the 
whole thing to get to bottom of this ;)

Ok another sikuli failure... I just won't run the script... this is 
seriously getting on my tits...and I am seriously getting on your tits.

Time to nuke sikuli for now... and either give up on this idea...

Or try it in a more serious dev environment cause this sucks balls.

To bad this was a failure, bye bye.

Bye,
  Skybuck. 



From MrJean1 at gmail.com  Mon Jun  1 23:07:50 2015
From: MrJean1 at gmail.com (MrJean1)
Date: Mon, 1 Jun 2015 20:07:50 -0700 (PDT)
Subject: Create on Win and run on Win or Mac
In-Reply-To: <mailman.25.1433201701.13271.python-list@python.org>
References: <mailman.25.1433201701.13271.python-list@python.org>
Message-ID: <4ccaa7c0-a708-49da-9c75-d4a2ba9e851f@googlegroups.com>

See this thread for possible answers

<http://stackoverflow.com/questions/2933/how-can-i-create-a-directly-executable-cross-platform-gui-app-using-python>

/Jean


From rustompmody at gmail.com  Mon Jun  1 23:15:43 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 1 Jun 2015 20:15:43 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
Message-ID: <74537c3e-7b7e-4418-a14c-557874e6933f@googlegroups.com>

Two unrelated points:

On Tuesday, June 2, 2015 at 5:54:43 AM UTC+5:30, TheDoctor wrote:
> On Monday, June 1, 2015 at 7:03:49 AM UTC-5, Eddilbert Macharia wrote:
> > I think i kind of understand now. 
> > 
> > Instead of python having data types like int, string, e.t.c it has two primitive types which are class type and class object which are created by python interpreter during its setup .using this two classes python is able to create some more data types
> 
> Eddibert, don't let this crowd (Terry, Stephen, et al.) confuse you.  They are as confused about it as you are.  
> 
> The truth is, that they've told a lie to themselves (everything is an object) to hide some details FROM THEMSELVES in their model.  They climb a ladder of ambiguity anytime there's a new problem in their model and call it a "type", or they pull out the "metaclass" term (because no one really knows that that is, so they feel *pretty* safe).
> 
> A type is not an object in the same way an instantiated type is an object -- anymore than a blueprint for a building is the building itself.

Point 1

Yes.  You may be onto something here Mark [I assume that's your name].
Some very intelligent people have suggested that the sloppy use of 'is'
causes more misunderstanding than any useful communication.
See http://en.wikipedia.org/wiki/E-Prime.

And in "G?del, the Mind and the Laws of Physics" Roger Penrose draws a picture 
of Three worlds and the mysteries that connect them
http://sustainedreaction.yuku.com/topic/4931/Three-Worlds-Three-Mysteries#.VW0b9rzGI8o

1. The mathematical world
2. The mental world
3. The physical world

which means that the 'is' in each is different.

As enumerated here http://blog.languager.org/2015/03/cs-history-0.html
the history and very existence of CS has arisen from grapping with these questions.

Point 2: [unrelated to the content of the OP]
You may wish to consider whether your style and the names you keep assuming
and changing are helping you.  Best I can see many people have begun to killfile
you.


From gheskett at wdtv.com  Mon Jun  1 23:30:36 2015
From: gheskett at wdtv.com (Gene Heskett)
Date: Mon, 1 Jun 2015 23:30:36 -0400
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <6vtpmap39im9c9lfom4mfsqfuljrceki1s@4ax.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <201506010949.35134.gheskett@wdtv.com>
 <6vtpmap39im9c9lfom4mfsqfuljrceki1s@4ax.com>
Message-ID: <201506012330.36969.gheskett@wdtv.com>



On Monday 01 June 2015 20:33:50 Dennis Lee Bieber wrote:
> On Mon, 1 Jun 2015 09:49:35 -0400, Gene Heskett <gheskett at wdtv.com>
>
> declaimed the following:
> >But IMO, any language that does not have the ability to set an fp
> > number to a fixed number of digits to the right of the separator
> > regardless of which , or . is used, needs one written.
>
> 	That removes all modern hardware units using IEEE floating point
> standard, and pretty much all languages since (including) the first
> version of FORTRAN.
>
> 	Floating point numbers are x-significant digits (commonly x:7 for
> single precision and x:15 for double precision) with an exponent.
>
> 	Ada supports float and fixed point, but fixed point most easily
> visualized as an integer with a scaling factor.
>
> >The ability to guarantee that the output of a FIX(2) is zero to at
> > least 17 significant digits so that a zero coomparison is not
> > non-zero because theres a 1 15 digits out in a 2 digit money format,
> > is an absolute requirement.
>
> 	Use COBOL then... One used to have to go out of their way to get a
> "floating point" data type in COBOL... The common numeric type is
> packed BCD.
>
> 	Even M$ "money" datatype uses four decimal places even if only two
> are displayed to the user -- it allows for accumulation of fractions
> of a cent over time.
>
That is a far more restrictive interpretation than I had in mind.  What, 
in the case of g-code should be the result of looking at a double and 
seeing that rounding errors in incrementing a number origially set to 
zero adding 1.000 to it 21 times, have created say 21.00000001200873000. 
Then we do a 

while [number gt 0.000000]
number=number - 1.0000000
do stuff using that number
endwhile

But the loop then iterates an extra pass, because the 1200873000 is still 
there.  The numbers are all double's, but weren't initialized to a 
sufficient number of digits to the right of the . or ,

This is of course our own fault, caused by sloppy coding.  But specifying 
the value to 15 magnitudes more than the machine is capable of without 
spending weeks writing a screw compensation file to get that level of 
accuracy is counterproductive.  Its severe overkill IMO.  Any language 
ought to just throw away those rounding errors by filling the extra 
precision with NNNNN.nnnn000000000000000's. even if the original 
initialized value was only stated as 1.00000.


> --
> 	Wulfraed                 Dennis Lee Bieber         AF6VN
>     wlfraed at ix.netcom.com    HTTP://wlfraed.home.netcom.com/

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>


From breamoreboy at yahoo.co.uk  Mon Jun  1 23:37:58 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 02 Jun 2015 04:37:58 +0100
Subject: What use of string module?
In-Reply-To: <d3bf3182-d70e-4160-b600-733cbda748e6@googlegroups.com>
References: <d3bf3182-d70e-4160-b600-733cbda748e6@googlegroups.com>
Message-ID: <mkj8ep$3v7$1@ger.gmane.org>

On 02/06/2015 03:14, fl wrote:
> Hi,
>
> I read the online help about string. It lists string constants, string
> formatting, template strings and string functions. After reading these,
> I am still puzzled about how to use the string module.
>
> Could you show me a few example about this module?
>
> Thanks
>

I suggest you don't bother, it's effectively dead having been replaced 
by string methods.  See here 
https://docs.python.org/2/library/string.html#deprecated-string-functions and 
then compare to https://docs.python.org/3/library/string.html#module-string

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Mon Jun  1 23:42:39 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 02 Jun 2015 04:42:39 +0100
Subject: Parser needed.
In-Reply-To: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
Message-ID: <mkj8ni$7rv$1@ger.gmane.org>

On 02/06/2015 01:29, Skybuck Flying wrote:
> Hello,
>
> I need some kind of parser and some kind of way to access the data
> contained in a file like the one below:
>
> (text file):
>
> http://www.skybuck.org/Games/StartrekOnline/Parser/SpaceFleetAlertEnemyExample.demo
>
>
> I am interested in learning the entity numbers:
>
> "EntityRef <number>"
>
> and their positions:
>
> "
> Pos <x>, <z>, <y>
> "
>
> I am also interested in the creation of these entities, especially their
> type:
>
> "entityTypeEnum <type>"
>
> So basically the parser should produce a few lists:
>
> EntityNumber = []
> EntityType = []
> EntityPositionX = []
> EntityPositionY = []
> EntityPositionZ = []
>
> Any extra information about entities is welcome.
>
> The parser should be able to parse a textfile of somewhere between
> 20.000 lines to 50.000 lines in about 1 to 2 seconds.
>
> My environment is SikuliX 1.1
>
> Bye,
>   Skybuck.

See here http://nedbatchelder.com/text/python-parsers.html

Can you also please stop sending so many emails as I believe some people 
still pay by the byte.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Mon Jun  1 23:46:20 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 02 Jun 2015 04:46:20 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <74537c3e-7b7e-4418-a14c-557874e6933f@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <74537c3e-7b7e-4418-a14c-557874e6933f@googlegroups.com>
Message-ID: <mkj8ue$apd$1@ger.gmane.org>

On 02/06/2015 04:15, Rustom Mody wrote:
> Two unrelated points:
>
> On Tuesday, June 2, 2015 at 5:54:43 AM UTC+5:30, TheDoctor wrote:
>> On Monday, June 1, 2015 at 7:03:49 AM UTC-5, Eddilbert Macharia wrote:
>>> I think i kind of understand now.
>>>
>>> Instead of python having data types like int, string, e.t.c it has two primitive types which are class type and class object which are created by python interpreter during its setup .using this two classes python is able to create some more data types
>>
>> Eddibert, don't let this crowd (Terry, Stephen, et al.) confuse you.  They are as confused about it as you are.
>>
>> The truth is, that they've told a lie to themselves (everything is an object) to hide some details FROM THEMSELVES in their model.  They climb a ladder of ambiguity anytime there's a new problem in their model and call it a "type", or they pull out the "metaclass" term (because no one really knows that that is, so they feel *pretty* safe).
>>
>> A type is not an object in the same way an instantiated type is an object -- anymore than a blueprint for a building is the building itself.
>
> Point 1
>
> Yes.  You may be onto something here Mark [I assume that's your name].
> Some very intelligent people have suggested that the sloppy use of 'is'
> causes more misunderstanding than any useful communication.
> See http://en.wikipedia.org/wiki/E-Prime.
>

I thought it was our pot smoking hippy back under another name.  he 
seems to have more lives than a black cat and a phoenix combined.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From random832 at fastmail.us  Tue Jun  2 00:08:51 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 02 Jun 2015 00:08:51 -0400
Subject: Everything is an object in python - object class and type class
In-Reply-To: <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
Message-ID: <1433218131.807582.284294961.0C399E71@webmail.messagingengine.com>

On Mon, Jun 1, 2015, at 20:24, TheDoctor wrote:
> A type is not an object in the same way an instantiated type is an object
> -- anymore than a blueprint for a building is the building itself.

What is an object?

It's something you can call str or repr on. It's something you can
lookup attributes from its internal dictionary and call methods on. It's
something you can compare whether it's == to something else. It's
something you can pass as a parameter to a function

In many languages, you can't do any of these things (or their
equivalents for those languages) to a type. In others, you have to get a
special "class object" that isn't _really_ the same thing as the type
itself, via some special syntax like typeof(Foo) or Foo.class, and can
only work with it through magic methods in a certain module.

In python, you can. Why is this bad or confusing?


From random832 at fastmail.us  Tue Jun  2 00:13:46 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 02 Jun 2015 00:13:46 -0400
Subject: Everything is an object in python - object class and type class
In-Reply-To: <e7de5af9-1775-4ace-a5a8-e1481a3cc910@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <mailman.28.1433205179.13271.python-list@python.org>
 <e7de5af9-1775-4ace-a5a8-e1481a3cc910@googlegroups.com>
Message-ID: <1433218426.809236.284296905.5D73A280@webmail.messagingengine.com>

On Mon, Jun 1, 2015, at 21:15, TheDoctor wrote:
> In Python 2.7 type(type) is type (not object), but isinstance(type,
> object) is true -- so it is ambiguous, even contradictory, in the
> language.

isinstance(type, object) is true because object is the parent class of
type. issubclass(type, object) is also true, and that is why
isinstance(int, object) is true. There is no more contradiction than in
anything else being an instance of its own class, its parent class, that
class's parent class, and ultimately... well, object.

type in object.__subclasses__() is also true.
type.__base__ is object.

I could go on.


From rxjwg98 at gmail.com  Tue Jun  2 00:18:02 2015
From: rxjwg98 at gmail.com (fl)
Date: Mon, 1 Jun 2015 21:18:02 -0700 (PDT)
Subject: How to understand '_' in these tests?
Message-ID: <95087133-f7aa-47ee-9ac0-c6164f7d7e3b@googlegroups.com>

Hi,

I just know that '_' is the last result in the Python interpreter. I feel that
it is like 'ans' in Matlab. When I run the following commands. The result of
'_' are always '4'.


Because I have tried several commands, such as reversed('fred'), xx and rx,
 '4' are always there.

What is your explanation about the following?


Thanks,






>>> reversed('fred')
<reversed object at 0x02A8DD50>
>>> _
'4'
>>> _
'4'
>>> xx
33223
>>> _
'4'
>>> rx=reversed('fred')
>>> _
'4'
>>> xx
33223
>>> _
'4'
>>> 


From skybuck2000 at hotmail.com  Tue Jun  2 00:23:13 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 06:23:13 +0200
Subject: Parser needed.
In-Reply-To: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
Message-ID: <9c6eb$556d2fa6$5419aafe$8770@news.ziggo.nl>

Here is a parser for sto demo files written in C# maybe it can be of some 
use, check trunk folders:

http://sourceforge.net/p/stodemolauncher/code/HEAD/tree/

Bye,
  Skybuck. 



From random832 at fastmail.us  Tue Jun  2 00:23:49 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 02 Jun 2015 00:23:49 -0400
Subject: How to understand '_' in these tests?
In-Reply-To: <95087133-f7aa-47ee-9ac0-c6164f7d7e3b@googlegroups.com>
References: <95087133-f7aa-47ee-9ac0-c6164f7d7e3b@googlegroups.com>
Message-ID: <1433219029.811891.284306833.581BDB77@webmail.messagingengine.com>

On Tue, Jun 2, 2015, at 00:18, fl wrote:
> Hi,
> 
> I just know that '_' is the last result in the Python interpreter. I feel that
> it is like 'ans' in Matlab. When I run the following commands. The result of
> '_' are always '4'.
> 
> 
> Because I have tried several commands, such as reversed('fred'), xx and rx,
>  '4' are always there.
> 
> What is your explanation about the following?

The use of _ for the last result is a special feature of the REPL. It
does not work if there is a real variable called _.

Try del _ before trying to print _.


From skybuck2000 at hotmail.com  Tue Jun  2 00:29:31 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 06:29:31 +0200
Subject: Parser needed.
In-Reply-To: <9c6eb$556d2fa6$5419aafe$8770@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <9c6eb$556d2fa6$5419aafe$8770@news.ziggo.nl>
Message-ID: <81e68$556d3120$5419aafe$9437@news.ziggo.nl>

Together with this it's might be of some use:

http://codeconverter.sharpdevelop.net/SnippetConverter.aspx

It converts C# to Python pretty fast.

I'll give it a try to see if it can work.

Bye,
  Skybuck.


From skybuck2000 at hotmail.com  Tue Jun  2 00:39:49 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 06:39:49 +0200
Subject: Parser needed.
In-Reply-To: <81e68$556d3120$5419aafe$9437@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <9c6eb$556d2fa6$5419aafe$8770@news.ziggo.nl>
 <81e68$556d3120$5419aafe$9437@news.ziggo.nl>
Message-ID: <b352f$556d338a$5419aafe$10196@news.ziggo.nl>

Hmm this parser uses an interesting technique:

  if line.Contains("activePlayerRef"):
   self._activePlayerRef = 
Convert.ToInt64(line.Substring(line.LastIndexOf("activePlayerRef") + 16))

I'll see if python can do this as well... ;)

Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Tue Jun  2 00:45:16 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 06:45:16 +0200
Subject: Parser needed.
In-Reply-To: <b352f$556d338a$5419aafe$10196@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <9c6eb$556d2fa6$5419aafe$8770@news.ziggo.nl>
 <81e68$556d3120$5419aafe$9437@news.ziggo.nl>
 <b352f$556d338a$5419aafe$10196@news.ziggo.nl>
Message-ID: <a1ba9$556d34d1$5419aafe$10658@news.ziggo.nl>

Example for python:

MyString = "Hello World"

print MyString.rfind("World")
if MyString.rfind("World"):
print "yes"
else:
print "no"

Pretty cool.

Bye,
  Skybuck.


From rustompmody at gmail.com  Tue Jun  2 00:46:10 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 1 Jun 2015 21:46:10 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mailman.4.1433175021.13271.python-list@python.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
 <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <201506010758.t517wR9T015894@fido.openend.se>
 <CAPTjJmrvi0EPyhV49hmqPxi1ztcZHmCn8hKzHhR7Y8_bkd7ZCw@mail.gmail.com>
 <201506011028.t51ASBVG019928@fido.openend.se> <rosuav@gmail.com>
 <CAPTjJmrCR6LWApe7UUF9DBCcQe2+hsCYi+s203aENUv2mmvDAA@mail.gmail.com>
 <201506011124.t51BOqO2021385@fido.openend.se>
 <mailman.290.1433166769.5151.python-list@python.org>
 <rI_ax.603294$PJ4.511481@fx26.am4>
 <mailman.4.1433175021.13271.python-list@python.org>
Message-ID: <dea5462a-4904-4f86-a1a6-9caa0ee2c8c9@googlegroups.com>

On Monday, June 1, 2015 at 9:40:39 PM UTC+5:30, Chris Angelico wrote:
> On Tue, Jun 2, 2015 at 1:17 AM, BartC wrote:
> > On 01/06/2015 14:52, Chris Angelico wrote:
> >> It's
> >>
> >> like the eternal debate about assignment and whether "x = x + 1" is
> >> nonsense, with advocates preferring "x := x + 1" as being somehow
> >> fundamentally different. It isn't. It's just a notational change, and
> >> not even a huge one. (Though I do see the line of argument that it
> >> should be "x <- x + 1" or something else that looks like an arro'w.)
> >
> >
> > 'x <- x + 1' already means something as an expression (whether x is less
> > than (-x+1). 'x <= x + 1' has the same problem.
> >
> > But I have used "=>" before,  for left-to-right assignment. (Mostly I use
> > ":=")
> 
> In Python it does, yes; I'm talking about the language design
> advocates. Some recommend a two-character ASCII notation like "<-" or
> "<=", others prefer a single-character symbol eg "?" or "?", but
> whatever it is, it will have no meaning in that language other than
> assignment. And yes, I can see the value of using an arrow to indicate
> assignment... but I don't really see a huge problem with using "=" to
> mean assignment, given that people from a mathematical background will
> have to grok the entire concept of temporal truth anyway. Whatever
> symbol you use, it has to be explained.

Its not merely temporal truth but truth vs action and their wanton overloading.

In every (natural) language that I know (of)? declarative and imperative moods
are distinguished.  It does not require a PhD in English to see that
"Please sit down."
and
"It is raining."
differ in mood.
Imperative languages after Pascal (specially C and following) use a locution from
the one to denote a semantics in the other and make pickle of beginners' brains.

---------
? Except perhaps magic/mystic-speak wherein pronouncing a spell makes the heavens thunder.
Maybe I am just too old to have noticed that imperative programming is a paradigm of magic


From dreamingforward at gmail.com  Tue Jun  2 00:47:45 2015
From: dreamingforward at gmail.com (TheDoctor)
Date: Mon, 1 Jun 2015 21:47:45 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <74537c3e-7b7e-4418-a14c-557874e6933f@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <74537c3e-7b7e-4418-a14c-557874e6933f@googlegroups.com>
Message-ID: <f21fdb69-d90b-4d33-8ad7-c961ba3b28f0@googlegroups.com>

> > A type is not an object in the same way an instantiated type is an object -- anymore than a blueprint for a building is the building itself.
> 
> Point 1
> 
> Yes.  You may be onto something here Mark [I assume that's your name].
> Some very intelligent people have suggested that the sloppy use of 'is'
> causes more misunderstanding than any useful communication.
> See http://en.wikipedia.org/wiki/E-Prime.

Ah, yes.  Very good.  Probably relevant.

> And in "G?del, the Mind and the Laws of Physics" Roger Penrose draws a picture 
> of Three worlds and the mysteries that connect them
> http://sustainedreaction.yuku.com/topic/4931/Three-Worlds-Three-Mysteries#.VW0b9rzGI8o

Amazing you reference that site.  I used to interact with Ghost Dog who's referenced at the bottom of that post.  Are you familiar with sustainedaction.org?

> 1. The mathematical world
> 2. The mental world
> 3. The physical world
> 
> which means that the 'is' in each is different.

Yes.  That's good.

> As enumerated here http://blog.languager.org/2015/03/cs-history-0.html
> the history and very existence of CS has arisen from grapping with these questions.

Interesting, will read up more.

> Point 2: [unrelated to the content of the OP]
> You may wish to consider whether your style and the names you keep assuming
> and changing are helping you.  Best I can see many people have begun to killfile you.

Yes, I know.  The names change based on mood.  The canonical address remains the same.  People are dumb, but perhaps I am dumb for treating them as equals.  Perhaps that's not a fair assumption.  I thought it was polite.

It's quite clear, that there's a bit of religion here and when I come in and point it out, they pull out their Bible and cast me out.  

I get too weary to explain more.  You know how it is arguing to evangelical Christians.  Perhaps another time.

Mark, one of my real names.


From skybuck2000 at hotmail.com  Tue Jun  2 00:51:47 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 06:51:47 +0200
Subject: Parser needed.
In-Reply-To: <a1ba9$556d34d1$5419aafe$10658@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <9c6eb$556d2fa6$5419aafe$8770@news.ziggo.nl>
 <81e68$556d3120$5419aafe$9437@news.ziggo.nl>
 <b352f$556d338a$5419aafe$10196@news.ziggo.nl>
 <a1ba9$556d34d1$5419aafe$10658@news.ziggo.nl>
Message-ID: <bd780$556d3658$5419aafe$11125@news.ziggo.nl>

If some of this code could be made to work it could be of some use:

ParserEngine is the least important...

MessageParser is interesting.

class AbstractSectionsParser(AbstractParser):
# Provides a method to return a list of sections
def GetResult(self):
  # Returns the list with the found resources.
  pass

class AbstractParser(object):
# Must be implemented by objects that want to use the DemoFileParser.
# Provides an empty implementation for conveniance
def StartParsing(self, filename):
  # Is called before the first line is parsed.
  # "filename" Absolute path to the file being parsed
  pass

def NewLine(self, line, lineNumber):
  # Is called by the parser when it has fetched a new line
  # "line" Content of the line
  # "lineNumber" Current line number (0 index)
  pass

def HasAllInformation(self):
  # Is called before each line is read. When you return true, parsing
  # will stop prematurely.
  # returns true if this plugin does not want to  process any more lines
  return False

def StopParsing(self, lineCount):
  # Is called, when all lines have been read by the parser.
  # "lineCount" Total number of lines in the demo file.
  pass

class MessagesParser(AbstractSectionsParser):
# Parses a demo file for "Notifications"
# The list in which the parser stores the pprocessed messages
# The string we are looking for in parsed messages
# Number of open braces ("{" increments, "}" decrements)
# The number of open braces at the last encounter of "Messages"
# The line at which the last "Messages" section started
# Flag that indicates if the parser has currently entered a
# "Messages" section
# Flag that indicates if we are currently processing a Messages
# section that contains the messageSearchString
# The last visited message test
# The command identifier
# The time index of the message in seconds
# The entity ref for this message
def __init__(self, messageSearchString):
   # Creates a new MessageParser object
   # <param name="messageSearchString">The string to look for in parsed
  self._result = List[DemoSection]()
  self._braceLevel = 0
  self._lastMessagesBraceLevel = 0
  self._lastMessagesSectionStart = 0
  self._inMessagesSection = False
  self._inSearchSection = False
  self._messageText = ""
  self._command = -1
  self._time = -1
  self._entityRef = -1
  self._messageSearchString = messageSearchString

def NewLine(self, line, lineNumber):
   # Is called by the parser when it has fetched a new line
   # <param name="line">Content of the line</param>
   # <param name="lineNumber">Current line number (0 index)</param>
  if line.Trim().StartsWith("Messages"):
   self._lastMessagesSectionStart = lineNumber
   self._lastMessagesBraceLevel = self._braceLevel
   self._inMessagesSection = True
  if self._inMessagesSection and line.Trim().StartsWith("Time"):
   timeString = line.Substring(line.IndexOf("Time ") + 5)
   self._time = Double.Parse(timeString, 
System.Globalization.CultureInfo.GetCultureInfo("en-US").NumberFormat)
  if self._inMessagesSection and line.Trim().StartsWith("Message ") and 
line.Contains(self._messageSearchString):
   self._inSearchSection = True
   self._messageText = line.Substring(line.IndexOf("Message ") + 8)
  if self._inMessagesSection and line.Trim().StartsWith("Command"):
   self._command = Convert.ToInt32(line.Substring(line.IndexOf("Command ") + 
8))
  if self._inMessagesSection and line.Trim().StartsWith("EntityRef"):
   self._entityRef = Convert.ToInt32(line.Substring(line.IndexOf("EntityRef 
") + 10))
  if self._inSearchSection and line.Trim().StartsWith("}") and 
self._braceLevel - 1 == self._lastMessagesBraceLevel:
   self._inMessagesSection = False
   self._inSearchSection = False
   message = DemoMessage()
   message.StartLine = self._lastMessagesSectionStart
   message.EndLine = lineNumber + 1
   message.Message = self._messageText
   message.Time = self._time
   message.Command = self._command
   message.EntityRef = self._entityRef
   self._result.Add(message)
  if line.Trim().StartsWith("{"):
   self._braceLevel += 1
  if line.Trim().StartsWith("}"):
   self._braceLevel -= 1

def GetResult(self):
  # Returns the list of parsed messages
  # <returns>Returns a list of parsed messages</returns>
  return self._result


class ParserEngine(object):
def Parse(filename, plugin):
  """ <summary>
   Parses a demo file
   </summary>
   <param name="filename">The file to be parsed</param>
   <param name="plugin">The ParserPlugin to use</param>
   <returns>true if parsing was successful</returns>
  """
  # here we read the contents of the demo file line-by-line
  line = ""
  lineNumber = 0
  # only run if the file actually exists
  if File.Exists(filename):
   plugin.StartParsing(filename)
   # start reading file line by line
   file = None
   try:
    file = StreamReader(filename)
    # parse as long as we don't have all information and there are still 
lines to read
    while not plugin.HasAllInformation() and (line = file.ReadLine()) != 
None:
     plugin.NewLine(line, lineNumber)
     lineNumber += 1
    plugin.StopParsing(lineNumber)
   except Exception, e:
    # report exception
    MessageBox.Show("The demo file " + filename + " could not be parsed for 
meta information.\n" + "Please report the following error under \"Updates & 
Help\"\n" + e.Message, "Error Parsing Demo Directory", MessageBoxButtons.OK, 
MessageBoxIcon.Error)
   finally:
    # close file, no matter what
    if file != None:
     file.Close()
   # abort parsing, if exception occured
   if file == None:
    return False
  else:
   # report error, if file does not exist
   MessageBox.Show("The demo file " + filename + " does not exist.\n" + 
"Please report the this error under \"Updates & Help\"\n", "File Not Found", 
MessageBoxButtons.OK, MessageBoxIcon.Error)
   return False
  return True

Parse = staticmethod(Parse)

def Parse(fileContents, plugin):
  """ <summary>
   Parses a demo file in memory
   </summary>
   <param name="fileContents">The contents of a demo file to be 
parsed</param>
   <param name="plugin">The ParserPlugin to use</param>
   <returns>true if parsing was successful</returns>
  """ #, ToolStripProgressBar progressBar)
  if fileContents != None:
   # here we read the contents of the demo file line-by-line
   lineNumber = 0
   #progressBar.Maximum = fileContents.Count();
   #progressBar.Step = 1;
   enumerator = fileContents.GetEnumerator()
   plugin.StartParsing("no file")
   # parse as long as we don't have all information and there are still 
lines to read
   while not plugin.HasAllInformation() and enumerator.MoveNext():
    plugin.NewLine(enumerator.Current, lineNumber)
    lineNumber += 1
   #progressBar.PerformStep();
   plugin.StopParsing(lineNumber)
  else:
   # report error, if file does not exist
   MessageBox.Show("The demo file buffer was null.\n" + "Please report the 
this error under \"Updates & Help\"\n", "Demo File Buffer Null", 
MessageBoxButtons.OK, MessageBoxIcon.Error)
   return False
  return True

Parse = staticmethod(Parse)

Bye,
  Skybuck.



From skybuck2000 at hotmail.com  Tue Jun  2 00:53:52 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 06:53:52 +0200
Subject: Parser needed.
In-Reply-To: <bd780$556d3658$5419aafe$11125@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <9c6eb$556d2fa6$5419aafe$8770@news.ziggo.nl>
 <81e68$556d3120$5419aafe$9437@news.ziggo.nl>
 <b352f$556d338a$5419aafe$10196@news.ziggo.nl>
 <a1ba9$556d34d1$5419aafe$10658@news.ziggo.nl>
 <bd780$556d3658$5419aafe$11125@news.ziggo.nl>
Message-ID: <16767$556d36d6$5419aafe$11344@news.ziggo.nl>

Actually that's not true... messages not that interesting.

This C# parser does not parse what I want which is:

"updates"

and

"positions".

Thus I rejected this parser some days ago.

But it's technique could be handy.

Bye,
  Skybuck.


From dreamingforward at gmail.com  Tue Jun  2 00:54:47 2015
From: dreamingforward at gmail.com (TheDoctor)
Date: Mon, 1 Jun 2015 21:54:47 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <mailman.39.1433218133.13271.python-list@python.org>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <mailman.39.1433218133.13271.python-list@python.org>
Message-ID: <5a959fda-25ac-4cd8-a8e4-f0d10fbf0ed0@googlegroups.com>

On Monday, June 1, 2015 at 11:09:09 PM UTC-5, rand... at fastmail.us wrote:
> On Mon, Jun 1, 2015, at 20:24, TheDoctor wrote:
> > A type is not an object in the same way an instantiated type is an object
> > -- anymore than a blueprint for a building is the building itself.
> 
> What is an object?
> 
> It's something you can call str or repr on. It's something you can
> lookup attributes from its internal dictionary and call methods on. It's
> something you can compare whether it's == to something else. It's
> something you can pass as a parameter to a function

That's good material for relating to the issue. I wish I could justify my position better, but it gets into non-computational arguments.

> In many languages, you can't do any of these things (or their
> equivalents for those languages) to a type. In others, you have to get a
> special "class object" that isn't _really_ the same thing as the type
> itself, via some special syntax like typeof(Foo) or Foo.class, and can
> only work with it through magic methods in a certain module.
> 
> In python, you can. Why is this bad or confusing?

It breaks a conceptual barrier.  It's like the difference between seeing an ASCII character symbol and knowing the 8-bit binary digits.  One is a name, the other is the data.  That's an important distinction and they shouldn't be confused in the machine (which can't happen because the machine already creates that boundary).  In the case of the reader though, they can get confused -- they're not interchangeable.  They get "lexed" in the head very differently.

Mark


From rustompmody at gmail.com  Tue Jun  2 01:01:55 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 1 Jun 2015 22:01:55 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <f21fdb69-d90b-4d33-8ad7-c961ba3b28f0@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <74537c3e-7b7e-4418-a14c-557874e6933f@googlegroups.com>
 <f21fdb69-d90b-4d33-8ad7-c961ba3b28f0@googlegroups.com>
Message-ID: <96a55389-5d7a-4593-8e3d-b52e9ff8af87@googlegroups.com>

On Tuesday, June 2, 2015 at 10:17:55 AM UTC+5:30, TheDoctor wrote:

> 
> Mark, one of my real names.

Thanks -- that helps.

> People are dumb, but perhaps I am dumb for treating them as equals.  Perhaps 
> that's not a fair assumption.  

That doesn't.

> It's quite clear, that there's a bit of religion here and when I come in and point it out, they pull out their Bible and cast me out.  
> 
> I get too weary to explain more.  You know how it is arguing to evangelical Christians.  Perhaps another time.

Do realize that this is an internation list and so a word like 'religion'
with wildly different connotations in different cultures does not usually aid
communication.
[Sarcasm can communicate when there is a large shared culture else fails miserably]
eg Would it make sense to you if you were told that there are widespread
religions like Buddhism that are agnostic or Jainism that are strictly atheistic?


From rustompmody at gmail.com  Tue Jun  2 01:33:21 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 1 Jun 2015 22:33:21 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mailman.273.1433145516.5151.python-list@python.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com><CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>
 <mailman.273.1433145516.5151.python-list@python.org>
Message-ID: <aa758287-55f7-4255-8e96-9e2bf4907879@googlegroups.com>

On Monday, June 1, 2015 at 1:28:53 PM UTC+5:30, Laura Creighton wrote:
> If you are giving a talk about Decimal -- and trying to stamp out the
> inappropriate use of floats you have to first inform people that
> what they learned as 'decimals' as children was not floating point,
> despite the fact that we write them the same way.
> 
> If I ever get the time machine, I am going back in time and demand that
> floating point numbers be expressed as 12345:678 instead of 12345.678
> because it would save so much trouble.  Never has the adage 'It's not
> what you don't know, that bites you.  It's what you know that ain't so.'
> been more apt.
> 
> I have done much better in speaking about this topic to a bunch of
> incredulous people if you next explain that scientists really don't
> care about accuracy in their calculations.  (This will surprise them).
> Most scentific calculations have some real world measurement in them,
> and for most real world measurements, if you are getting even 5 digits
> of precision, you are doing really, really, well.  This means that
> scientists are going to be throwing away all the extra digits they get
> out a a floating point representation, so they don't have to care how
> accurate they are.  As long as their results are good in the first 5,
> it won't matter.  (Depending on time constraints, a review of
> significant figures -- what they are and what they mean is good here.)
> 
> It is really hard to get the concept of Decimal across to people who
> already have that concept in their mind, but think it is called Float.
> You have to first teach them that they don't know anything about Float
> and get them to reboot their brains before you can install this new
> knowledge.  Otherwise their brains will just overwrite your new knowledge
> with 'ah, just use a Float' as soon as you stop speaking.  Same day,
> even.
> 
> Laura

I'd say on the whole this *intention* is rightly directed.
Whether it would work in practice I am not sure.
You may like to look at haskell's defaulting mechanism which is about as
sophisticated as I have seen
https://www.haskell.org/tutorial/numbers.html [sect 10.4]
Basically: (the ':t' says give me the type of...)

Prelude> :t 4
4 :: Num a => a
Prelude> :t 1.0
1.0 :: Fractional a => a

What this says is that 4 is not an integer but any type that is in the Num class
Likewise 1.0 is not float but any of a family of 'Fractional' types.

Your ':' suggestion would work if Decimal and machine floats covered all needs.
Do you think that is possible/likely?
If not we would have to fall back to some 'defaulting' mechanism anyway.
[In all fairness while Haskell's default looks neat on paper it turns out to
be quite a nuisance in practice]


From rosuav at gmail.com  Tue Jun  2 01:42:41 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 2 Jun 2015 15:42:41 +1000
Subject: How to understand '_' in these tests?
In-Reply-To: <95087133-f7aa-47ee-9ac0-c6164f7d7e3b@googlegroups.com>
References: <95087133-f7aa-47ee-9ac0-c6164f7d7e3b@googlegroups.com>
Message-ID: <CAPTjJmrpWV8WUBK7Rg1YX0n9Xa=Dx2ZGreQDUE+Qjp-CUoTXQg@mail.gmail.com>

On Tue, Jun 2, 2015 at 2:18 PM, fl <rxjwg98 at gmail.com> wrote:
> Because I have tried several commands, such as reversed('fred'), xx and rx,
>  '4' are always there.
>
> What is your explanation about the following?
>
>
> Thanks,
>
>
>
>
>
>
>>>> reversed('fred')
> <reversed object at 0x02A8DD50>
>>>> _
> '4'
>>>> _
> '4'
>>>> xx
> 33223
>>>> _
> '4'
>>>> rx=reversed('fred')
>>>> _
> '4'
>>>> xx
> 33223
>>>> _
> '4'
>>>>

I think part of what you're seeing as odd here is that when you assign
an expression somewhere, _ doesn't get changed. This is because
there's no final evaluation getting printed out; I suppose it would be
reasonable to unset _ in those cases, but it's more useful to retain
it. The rest is what random832 pointed out, which is that the special
effect of _ can be shadowed just as anything else can. In fact, just
like most things that can be shadowed, it's actually found in the
builtins:

>>> import builtins
>>> 1 + 2
3
>>> builtins._
3

Same goes for other handy names; if you use "dir" to store a directory
name, you can retrieve the original dir() function as "builtins.dir",
or by unsetting your local name with "del dir".

ChrisA


From bretedward at yahoo.com  Tue Jun  2 01:51:43 2015
From: bretedward at yahoo.com (Bret Edwards)
Date: Tue, 2 Jun 2015 05:51:43 +0000 (UTC)
Subject: Create Web app on Win and run on Win or Mac
Message-ID: <123770791.3821937.1433224303626.JavaMail.yahoo@mail.yahoo.com>

I took a look at Ren'Py as suggested by a reply to my previous post entitled "Create on Win and run on Win or Mac." Thanks for your suggestion. Ren'Py looks pretty amazing!

Not sure that is a good route since my primary reason for this endeavor is learning Python scripting (secondarily to create the game) and Ren'Py uses it's own scripting language. 
Also not sure that Ren'Py supports Win 64-bit machines. 

Now I am switching course slightly and would like to write a Python based web app on a Windows machine, have it run locally and display the app in the browser. The finished product 
would be run on other Windows machines (of varying Windows OS versions) and also on Mac machines (of varying Mac OS versions.) The program would present a browser based GUI that the 
user would interact with (keying in integer data and clicking buttons) and would only have simple computations behind the scenes.

I would like to only do the creation work once and do it on a Windows machine (I do not have access to a Mac.) Then be able to run on both other Win machines and Mac machines with 
only doing a simple (or not any) installation and without requiring an existing Python installation. Also, I do not want the user to need to remain connected to the I-net to use the 
app. If they need to connect to the I-net initially to download it that would be OK, as long as they could disconnect immediately afterwards and still use the app. 

I am sure that I could do something in JavaScript and/or AngularJS, maybe a single page app, but I want to do it in Python since that is what I am trying to learn. It would be OK if 
it had a small amount of JavaScript, but most of it needs to Python. 

Is this possible? If so, what tools do I use? Can I still use Tkinter for a web app? Do I need to do (use) anything to create cross platform executables or would running in a 
browser make that step not required?

Sorry for the newbie question, but I am obviously looking for a light switch.
Thanks,Bret

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150602/addc6001/attachment.html>

From steve+comp.lang.python at pearwood.info  Tue Jun  2 02:04:20 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 02 Jun 2015 16:04:20 +1000
Subject: Religion [was Re: Everything is an object in python - object class
 and type class]
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <74537c3e-7b7e-4418-a14c-557874e6933f@googlegroups.com>
 <f21fdb69-d90b-4d33-8ad7-c961ba3b28f0@googlegroups.com>
 <96a55389-5d7a-4593-8e3d-b52e9ff8af87@googlegroups.com>
Message-ID: <556d4766$0$12918$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 02 June 2015 15:01, Rustom Mody wrote:

> eg Would it make sense to you if you were told that there are widespread
> religions like Buddhism that are agnostic or Jainism that are strictly
> atheistic?

No of course it wouldn't make sense. But nothing to do with religion, 
spirituality and superstition makes sense, the whole point of them is that 
they speak to the emotions, not logic. (You note that I am carefully not 
commenting on whether this is a good thing or not.)

Draw up two sets of overlapping axes, and label the vertical axes 
"Agnosticism / Gnosticism" and the horizontal axes "Supernatural / Natural". 
Belief systems can be found in all four quadrants. Agnostic religions are 
easy, they're just in the Supernatural+Agnostic quadrant. If you define 
religion to be merely any belief system, then even an atheist religion is 
understandable: it could be anything on the Natural half of the graph.

Personally, I consider that redefining religion to refer to belief systems 
which do not include supernatural divine gods is an abuse of language 
(except informally, as in "football is my religion" or "the religious war 
between Vi and Emacs users"). it's like the food processor that is 
advertised as being a "three speed food-processor" because there are three 
settings on the control: High, Low and OFF.


-- 
Steve



From breamoreboy at yahoo.co.uk  Tue Jun  2 02:11:36 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 02 Jun 2015 07:11:36 +0100
Subject: Create Web app on Win and run on Win or Mac
In-Reply-To: <123770791.3821937.1433224303626.JavaMail.yahoo@mail.yahoo.com>
References: <123770791.3821937.1433224303626.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <mkjher$cte$1@ger.gmane.org>

On 02/06/2015 06:51, Bret Edwards via Python-list wrote:
> I took a look at Ren'Py as suggested by a reply to my previous post
> entitled "Create on Win and run on Win or Mac." Thanks for your
> suggestion. Ren'Py looks pretty amazing!
>
> Not sure that is a good route since my primary reason for this endeavor
> is learning Python scripting (secondarily to create the game) and Ren'Py
> uses it's own scripting language.
> Also not sure that Ren'Py supports Win 64-bit machines.
>
> Now I am switching course slightly and would like to write a Python
> based web app on a Windows machine, have it run locally and display the
> app in the browser. The finished product
> would be run on other Windows machines (of varying Windows OS versions)
> and also on Mac machines (of varying Mac OS versions.) The program would
> present a browser based GUI that the
> user would interact with (keying in integer data and clicking buttons)
> and would only have simple computations behind the scenes.
>
> I would like to only do the creation work once and do it on a Windows
> machine (I do not have access to a Mac.) Then be able to run on both
> other Win machines and Mac machines with
> only doing a simple (or not any) installation and without requiring an
> existing Python installation. Also, I do not want the user to need to
> remain connected to the I-net to use the
> app. If they need to connect to the I-net initially to download it that
> would be OK, as long as they could disconnect immediately afterwards and
> still use the app.
>
> I am sure that I could do something in JavaScript and/or AngularJS,
> maybe a single page app, but I want to do it in Python since that is
> what I am trying to learn. It would be OK if
> it had a small amount of JavaScript, but most of it needs to Python.
>
> Is this possible? If so, what tools do I use? Can I still use Tkinter
> for a web app? Do I need to do (use) anything to create cross platform
> executables or would running in a
> browser make that step not required?
>
> Sorry for the newbie question, but I am obviously looking for a light
> switch.
>
> Thanks,
> Bret
>

https://wiki.python.org/moin/WebFrameworks is hopefully the correct 
voltage and frequency :)

I'll hazard a guess that your best bet for starters is the section 
"Popular Non Full-Stack Frameworks".

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rustompmody at gmail.com  Tue Jun  2 02:17:35 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 1 Jun 2015 23:17:35 -0700 (PDT)
Subject: Religion [was Re: Everything is an object in python - object
 class and type class]
In-Reply-To: <556d4766$0$12918$c3e8da3$5496439d@news.astraweb.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <74537c3e-7b7e-4418-a14c-557874e6933f@googlegroups.com>
 <f21fdb69-d90b-4d33-8ad7-c961ba3b28f0@googlegroups.com>
 <96a55389-5d7a-4593-8e3d-b52e9ff8af87@googlegroups.com>
 <556d4766$0$12918$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <8f91f88b-de16-46ad-8e2f-e9b28bd19fec@googlegroups.com>

On Tuesday, June 2, 2015 at 11:34:34 AM UTC+5:30, Steven D'Aprano wrote:
> On Tuesday 02 June 2015 15:01, Rustom Mody wrote:
> 
> > eg Would it make sense to you if you were told that there are widespread
> > religions like Buddhism that are agnostic or Jainism that are strictly
> > atheistic?
> 
> No of course it wouldn't make sense. But nothing to do with religion, 
> spirituality and superstition makes sense, the whole point of them is that 
> they speak to the emotions, not logic.

Emotions dont make sense?
In what sense of 'sense'?
If understanding rather than pronouncing is your intention this may help:
http://en.wikipedia.org/wiki/Triune_brain 
[You may remember a recent poster with the moniker 'PointedEars' in his name
making wild emotional outbursts.  So much for 20th century movie-mythology]

(You note that I am carefully not 
> commenting on whether this is a good thing or not.)
> 
> Draw up two sets of overlapping axes, and label the vertical axes 
> "Agnosticism / Gnosticism" and the horizontal axes "Supernatural / Natural". 
> Belief systems can be found in all four quadrants. Agnostic religions are 
> easy, they're just in the Supernatural+Agnostic quadrant. If you define 
> religion to be merely any belief system, then even an atheist religion is 
> understandable: it could be anything on the Natural half of the graph.
> 
> Personally, I consider that redefining religion to refer to belief systems 
> which do not include supernatural divine gods is an abuse of language 
> (except informally, as in "football is my religion" or "the religious war 
> between Vi and Emacs users"). it's like the food processor that is 
> advertised as being a "three speed food-processor" because there are three 
> settings on the control: High, Low and OFF.
> 

I am reminded of the TinTin(?) comic:
Telescope informs about a giant alien coming to invade from outer space.
Further examination of telescope shows 'alien' to be a bug (not the software variety) 
in the telescope.


From breamoreboy at yahoo.co.uk  Tue Jun  2 02:20:00 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 02 Jun 2015 07:20:00 +0100
Subject: Religion [was Re: Everything is an object in python - object
 class and type class]
In-Reply-To: <556d4766$0$12918$c3e8da3$5496439d@news.astraweb.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <74537c3e-7b7e-4418-a14c-557874e6933f@googlegroups.com>
 <f21fdb69-d90b-4d33-8ad7-c961ba3b28f0@googlegroups.com>
 <96a55389-5d7a-4593-8e3d-b52e9ff8af87@googlegroups.com>
 <556d4766$0$12918$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mkjhuj$kgd$1@ger.gmane.org>

On 02/06/2015 07:04, Steven D'Aprano wrote:
> On Tuesday 02 June 2015 15:01, Rustom Mody wrote:
>
>> eg Would it make sense to you if you were told that there are widespread
>> religions like Buddhism that are agnostic or Jainism that are strictly
>> atheistic?
>
> No of course it wouldn't make sense. But nothing to do with religion,
> spirituality and superstition makes sense, the whole point of them is that
> they speak to the emotions, not logic. (You note that I am carefully not
> commenting on whether this is a good thing or not.)
>
> Draw up two sets of overlapping axes, and label the vertical axes
> "Agnosticism / Gnosticism" and the horizontal axes "Supernatural / Natural".
> Belief systems can be found in all four quadrants. Agnostic religions are
> easy, they're just in the Supernatural+Agnostic quadrant. If you define
> religion to be merely any belief system, then even an atheist religion is
> understandable: it could be anything on the Natural half of the graph.
>
> Personally, I consider that redefining religion to refer to belief systems
> which do not include supernatural divine gods is an abuse of language
> (except informally, as in "football is my religion" or "the religious war
> between Vi and Emacs users"). it's like the food processor that is
> advertised as being a "three speed food-processor" because there are three
> settings on the control: High, Low and OFF.
>
>

If anybody wishes to discuss this can they please take it off list, as 
it surely has no place here when we're meant to be discussing the Python 
programming langauge.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From steve+comp.lang.python at pearwood.info  Tue Jun  2 02:23:11 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 02 Jun 2015 16:23:11 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <mailman.28.1433205179.13271.python-list@python.org>
 <e7de5af9-1775-4ace-a5a8-e1481a3cc910@googlegroups.com>
Message-ID: <556d4bd2$0$12899$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 02 June 2015 11:15, TheDoctor wrote:

> A type is not an object.

Yes it is.


py> isinstance(type, object)
True
py> isinstance(int, object)
True

> You see it as one, because you are MENTALLY
> lexing your own code on the screen.

No, I see it as one, because it is one.


> But python does not define a type.

Of course it does.

py> isinstance(int, type)  # int is a type
True
py> isinstance(42, type)  # 42 is not a type
False


> It defines certain primitives, like strings and integers.  But it doesn't
> define what an OBJECT is.

In one sense, it does: something is an object if it is an instance of type 
"object". 

In another sense, it doesn't, because the definition of "object" is bigger 
than Python and exists outside of Python. Python had objects before it had a 
built-in type called "object".


> So you and everyone else are beating around the
> bush trying to define something that the LANGUAGE ITSELF doesn't define. 

Incorrect.


> In Python 2.7 type(type) is type (not object), but isinstance(type,
> object) is true -- so it is ambiguous, even contradictory, in the
> language.

No, it is not ambiguous nor contradictory. It just means that types 
(classes), include type itself, are *both* types AND objects. Which is the 
whole point of saying that everything is an object! This is not true in all 
Object Oriented languages, for instance it is not true in Java, but it is 
true in Python.

Notice that we do NOT say that "everything is a type":

py> isinstance(42, type)
False

In Python, all types (classes) are objects, but not all objects (instances) 
are types.



-- 
Steve



From steve+comp.lang.python at pearwood.info  Tue Jun  2 02:30:17 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 02 Jun 2015 16:30:17 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <mailman.28.1433205179.13271.python-list@python.org>
 <3ee816c0-b3f3-4a0f-b567-40c435a53e8d@googlegroups.com>
Message-ID: <556d4d7b$0$11123$c3e8da3@news.astraweb.com>

On Tuesday 02 June 2015 11:02, TheDoctor wrote:

> On Monday, June 1, 2015 at 7:33:11 PM UTC-5, Chris Angelico wrote:
>> On Tue, Jun 2, 2015 at 10:24 AM, TheDoctor <dreamingforward at gmail.com>
>> wrote:
>> > A type is not an object in the same way an instantiated type is an
>> > object -- anymore than a blueprint for a building is the building
>> > itself.
>> 
>> And don't let dreamingforward confuse you, because he is flat-out
>> wrong. An integer is an object, because you can manipulate it as one.
> 
> An integer in Python *acts* like an object, in some narrow sense can
> indeed be said to BE one.

No, they *really are* objects. You can look at the source code of the Python 
interpreter to see how ints are implemented, and they are implemented as 
objects. You can see that they are bigger than a machine int:

py> sys.getsizeof(23)
14

That's 14 bytes. A machine int is typically 2 bytes (int16), 4 bytes (int32) 
or 8 bytes (int64).

If you use ctypes, I expect that you can investigate the internal structure 
of the int. If not in ctypes, then you can certainly do so from a C 
extension.



> But no one uses them like that in practice

Of course they do. ints have methods:

py> n = 23
py> n.to_bytes(5, "big")
b'\x00\x00\x00\x00\x17'


Is that object oriented enough for you?




-- 
Steve



From steve+comp.lang.python at pearwood.info  Tue Jun  2 03:12:13 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 02 Jun 2015 17:12:13 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
Message-ID: <556d574f$0$12905$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 02 June 2015 10:24, TheDoctor wrote:

> A type is not an object in the same way an instantiated type is an object
> -- anymore than a blueprint for a building is the building itself.

Nobody says that blueprints are *buildings*. But buildings are not the only 
kind of object that exists.

Plates and spoons and knives and forks are objects.
Cars and trucks and ships and planes are objects.
Books and shoes and maps and compasses are objects.
Buildings are objects.
And blueprints of buildings are objects too.


> The reason this confusion appears is for the exact reason I said on an
> earlier thread:  Python does not define classes in it's grammar

Of course it does. Otherwise "class K: pass" would give you a syntax error.


> -- so it
> can't account for the difference between an [instantiated] object and it's
> definition/type.

Of course it can and does. Python has a rich set of tools for introspecting 
values and determining whether they are instances or classes, what they are 
an instance of, and viewing the differences between the type and the class, 
starting with the humble human-readable string representation:

py> type(23)
<class 'int'>

and going on isinstance, issubclass, vars(), dir(), the inspect module, the 
dis module, and more.

The simplest test to tell whether something is an instance (an object) is:

isinstance(the_thing, object)

and that returns True *always*. The next test is to check whether it is a 
class (a type):

isinstance(the_thing, type)

which returns True only for classes and types. So, classes *are* objects, as 
well as being classes -- just as dogs are mammals, as well as being dogs.



-- 
Steve



From steve+comp.lang.python at pearwood.info  Tue Jun  2 03:19:11 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 02 Jun 2015 17:19:11 +1000
Subject: Language design
References: <522eb795$0$29999$c3e8da3$5496439d@news.astraweb.com>
 <mailman.270.1378935062.5461.python-list@python.org>
 <5230ff66$0$29988$c3e8da3$5496439d@news.astraweb.com>
 <mailman.287.1378946951.5461.python-list@python.org>
 <52315238$0$29999$c3e8da3$5496439d@news.astraweb.com>
 <mailman.338.1379042609.5461.python-list@python.org>
 <52329db3$0$29988$c3e8da3$5496439d@news.astraweb.com>
 <47479653-fde1-4369-b958-8a42c8fcfeeb@googlegroups.com>
Message-ID: <556d58f1$0$11122$c3e8da3@news.astraweb.com>

On Tuesday 02 June 2015 10:10, TheDoctor wrote:

> On Friday, September 13, 2013 at 12:08:04 AM UTC-5, Steven D'Aprano wrote:

Mark, you are digging up a conversation from nearly two years ago. 
Seriously?


>> It makes no difference whether I write:
>> 
>>     atoms -> stars -> galaxies
>> 
>> or
>> 
>>     galaxies <- stars <- atoms
>> 
>> nor does it make any difference if I write the chain starting at the top
>> and pointing down, or at the bottom and pointing up.
> 
> Yes it does.  Ford IS-A Car, but Car IS-A Ford?  No.  Try reordering that
> one.

Try again. Look at the direction of the arrows. Nobody denies that there is 
a difference between:

    atoms -> stars -> galaxies

    galaxies -> stars -> atoms

but that's not what my second example says. Look closely, and consider that 
sometimes we write "Mark's hat" and sometimes "the hat of Mark".


> I see we've missed each other with our limitations to ASCII text, in ways
> that would have never happened had these conversations occurred in person.
>  Re-reading my texts, I see that I can easily confuse myself.

Yes, I see that.


-- 
Steve



From steve+comp.lang.python at pearwood.info  Tue Jun  2 03:21:42 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 02 Jun 2015 17:21:42 +1000
Subject: What use of string module?
References: <d3bf3182-d70e-4160-b600-733cbda748e6@googlegroups.com>
Message-ID: <556d5986$0$11122$c3e8da3@news.astraweb.com>

On Tuesday 02 June 2015 12:14, fl wrote:

> Hi,
> 
> I read the online help about string. It lists string constants, string
> formatting, template strings and string functions. After reading these,
> I am still puzzled about how to use the string module.
> 
> Could you show me a few example about this module?


import string

c = "d"
if c in string.ascii_lowercase:
    print "character is an ASCII lower case letter"



-- 
Steve



From steve+comp.lang.python at pearwood.info  Tue Jun  2 03:25:32 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 02 Jun 2015 17:25:32 +1000
Subject: Language design
References: <522eb795$0$29999$c3e8da3$5496439d@news.astraweb.com>
 <mailman.270.1378935062.5461.python-list@python.org>
 <5230ff66$0$29988$c3e8da3$5496439d@news.astraweb.com>
 <a703cea2-ef76-4a47-81ba-3ddcbfcd2c06@googlegroups.com>
Message-ID: <556d5a6d$0$11122$c3e8da3@news.astraweb.com>

On Tuesday 02 June 2015 07:45, TheDoctor wrote:

> On Wednesday, September 11, 2013 at 6:40:22 PM UTC-5, Steven D'Aprano
> wrote:
>> On Wed, 11 Sep 2013 14:30:54 -0700, Mark Janssen wrote:
>> 
>> > 1) It tried to make Object the parent of every class.
>> 
>> Tried, and succeeded.
> 
> Oh?  How about:
> 
> class superdict(dict):
>     """I'm going to extend the dict type to include extra methods."""
> 
> class mixin():
>    """Here we go."""
> 
> What is the parent of mixin?

Why don't you try for yourself and see?


py> class mixin():
...     """Here we go."""
... 
py> mixin.__bases__
(<class 'object'>,)
py> mixin.__base__
<class 'object'>
py> mixin.__mro__
(<class '__main__.mixin'>, <class 'object'>)





-- 
Steve



From rosuav at gmail.com  Tue Jun  2 03:36:16 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 2 Jun 2015 17:36:16 +1000
Subject: Language design
In-Reply-To: <556d58f1$0$11122$c3e8da3@news.astraweb.com>
References: <522eb795$0$29999$c3e8da3$5496439d@news.astraweb.com>
 <mailman.270.1378935062.5461.python-list@python.org>
 <5230ff66$0$29988$c3e8da3$5496439d@news.astraweb.com>
 <mailman.287.1378946951.5461.python-list@python.org>
 <52315238$0$29999$c3e8da3$5496439d@news.astraweb.com>
 <mailman.338.1379042609.5461.python-list@python.org>
 <52329db3$0$29988$c3e8da3$5496439d@news.astraweb.com>
 <47479653-fde1-4369-b958-8a42c8fcfeeb@googlegroups.com>
 <556d58f1$0$11122$c3e8da3@news.astraweb.com>
Message-ID: <CAPTjJmpD2_NbHJMA9dM9rcHui0nfdZ1AQr6rGydCTZ0exiwf4w@mail.gmail.com>

On Tue, Jun 2, 2015 at 5:19 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> but that's not what my second example says. Look closely, and consider that
> sometimes we write "Mark's hat" and sometimes "the hat of Mark".

... and sometimes "the hat Mark's talking through", which appears to
put "hat" and "Mark's" the other way around, but is no less true.

ChrisA


From rustompmody at gmail.com  Tue Jun  2 03:44:11 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 2 Jun 2015 00:44:11 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <556d574f$0$12905$c3e8da3$5496439d@news.astraweb.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <556d574f$0$12905$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <3d06de71-a517-4cac-be12-a61813465735@googlegroups.com>

On Tuesday, June 2, 2015 at 12:42:31 PM UTC+5:30, Steven D'Aprano wrote:
> On Tuesday 02 June 2015 10:24, TheDoctor wrote:
> 
> > A type is not an object in the same way an instantiated type is an object
> > -- anymore than a blueprint for a building is the building itself.
> 
> Nobody says that blueprints are *buildings*. But buildings are not the only 
> kind of object that exists.
> 
> Plates and spoons and knives and forks are objects.
> Cars and trucks and ships and planes are objects.
> Books and shoes and maps and compasses are objects.
> Buildings are objects.
> And blueprints of buildings are objects too.

You are using 'are' as if they are all the same 'are'. Are they?
Consider the type int.
It is supposed to model mathematical integers.
ie 1 ? int (in some sense)

Consider the generator
def int2():
  i = 0
  yield i
  while True:
    yield i
    yield -i

1 ? int2 [Not to mention 1 ? int2()]

Consider range(10) (in python2 and 3)
And finally [1,2,3,4]

1 ? all these
Are the '?'s here same?  Similar?


From rosuav at gmail.com  Tue Jun  2 04:37:28 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 2 Jun 2015 18:37:28 +1000
Subject: Everything is an object in python - object class and type class
In-Reply-To: <3d06de71-a517-4cac-be12-a61813465735@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <556d574f$0$12905$c3e8da3$5496439d@news.astraweb.com>
 <3d06de71-a517-4cac-be12-a61813465735@googlegroups.com>
Message-ID: <CAPTjJmoPkbmOKim__WdOVvZRqY2Lbi5mpyUOCYBF0jyHPC-LOw@mail.gmail.com>

On Tue, Jun 2, 2015 at 5:44 PM, Rustom Mody <rustompmody at gmail.com> wrote:
>> Plates and spoons and knives and forks are objects.
>> Cars and trucks and ships and planes are objects.
>> Books and shoes and maps and compasses are objects.
>> Buildings are objects.
>> And blueprints of buildings are objects too.
>
> You are using 'are' as if they are all the same 'are'. Are they?
> Consider the type int.
> It is supposed to model mathematical integers.
> ie 1 ? int (in some sense)
>
> Consider the generator
> def int2():
>   i = 0
>   yield i
>   while True:
>     yield i
>     yield -i
>
> 1 ? int2 [Not to mention 1 ? int2()]
>
> Consider range(10) (in python2 and 3)
> And finally [1,2,3,4]
>
> 1 ? all these
> Are the '?'s here same?  Similar?

Yes, they are. Every one of them is asserting (or asking, depending on
your point of view) whether or not the instance to its left is a
member of the set to its right. The sets on the right are all
different, but the set membership operation is identical.

But even more so: The original collection of statements ("are
objects") all used the same RHS. In exactly the same senses, all of
those examples were testing for the same "objectness". (Although you
could argue that Steven's examples were talking about subclass
relationships rather than instance relationships, but really, an
instanceof relationship is a special case of subclass, where the class
on the left is defined by some form of object identity. Stating "All
books are objects" is a subclass relationship; stating "This book is
an object" is an instanceof relationship; both assertions use the same
meaning of "object(s)".)

All shoes are objects.
All ships are objects.
All lumps of sealing-wax are objects.
All cabbages are objects.
All kings are objects. (Some kings are also subjects, but not many.)

I can pick up a shoe, I can look at it, I can verify that it has
certain attributes common to all objects (location, mass, etc).
Superman can do the same with a ship (they're a bit big for me to pick
up). The actions you can perform on all objects are the attributes of
objects in general. Some subclasses of object may provide additional
actions (you can tip a king, for instance), but you can still do all
of these actions, so they're all the same type of object.

Note that not one iota of this has anything to do with Python's
documentation, its rules, its behaviour, or anything. Python follows
the natural expectations of our universe. The standard actions that
can be performed on Python objects are slightly different from the
standard actions that can be performed in Dungeons and Dragons, and
different again from the standard actions of the world we live in, but
they are no less applicable to all objects in that hierarchy:

* You can bind a name to an object (any object).
* You can pass an object as a function parameter.
* You can return an object from a function.
* You can reference objects in other objects' attributes.
* You can distinguish one object from another (that is, objects have
unique identities).

(In CPython, you can recognize an object by its GC header - an object
invariably has a reference count and so on. This is not a fundamental
attribute of objects, but it's a way to recognize them.)

Not everything that Python uses is an object. Syntax is not, itself,
manipulable; you can represent a syntax tree with either source code
(as a string) or an AST object, but syntax itself is not something you
can grasp. You can't assign "foo = def" in Python. But anything that
you can work with, in any way, is an object - and is an instance of
(some subclass of) the type 'object'. So, yes: They ARE all the same
meaning of "are" and "objects".

ChrisA


From chrismeek4542 at gmail.com  Tue Jun  2 05:16:28 2015
From: chrismeek4542 at gmail.com (chrismeek4542 at gmail.com)
Date: Tue, 2 Jun 2015 02:16:28 -0700 (PDT)
Subject: Knowledge Requirements for Machine Learning (SKLearn)
Message-ID: <33dfc8d4-bd98-4648-8d7d-ed16070dda23@googlegroups.com>

I would like to get into Machine learning. The problem I have is that my Math skills are sadly lacking. I went through SKlearn docs but it is beyond me. Could anyone recommended some resources to get up to speed with the subject?


From rustompmody at gmail.com  Tue Jun  2 05:18:06 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 2 Jun 2015 02:18:06 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <mailman.48.1433234251.13271.python-list@python.org>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <556d574f$0$12905$c3e8da3$5496439d@news.astraweb.com>
 <3d06de71-a517-4cac-be12-a61813465735@googlegroups.com>
 <mailman.48.1433234251.13271.python-list@python.org>
Message-ID: <08e0d883-95ce-4101-8330-3bc6832c8033@googlegroups.com>

On Tuesday, June 2, 2015 at 2:08:08 PM UTC+5:30, Chris Angelico wrote:
> On Tue, Jun 2, 2015 at 5:44 PM, Rustom Mody wrote:
> >> Plates and spoons and knives and forks are objects.
> >> Cars and trucks and ships and planes are objects.
> >> Books and shoes and maps and compasses are objects.
> >> Buildings are objects.
> >> And blueprints of buildings are objects too.
> >
> > You are using 'are' as if they are all the same 'are'. Are they?
> > Consider the type int.
> > It is supposed to model mathematical integers.
> > ie 1 ? int (in some sense)
> >
> > Consider the generator
> > def int2():
> >   i = 0
> >   yield i
> >   while True:
> >     yield i
> >     yield -i
> >
> > 1 ? int2 [Not to mention 1 ? int2()]
> >
> > Consider range(10) (in python2 and 3)
> > And finally [1,2,3,4]
> >
> > 1 ? all these
> > Are the '?'s here same?  Similar?
> 
> Yes, they are. Every one of them is asserting (or asking, depending on
> your point of view) whether or not the instance to its left is a
> member of the set to its right. The sets on the right are all
> different, but the set membership operation is identical.
> 
> But even more so: The original collection of statements ("are
> objects") all used the same RHS. In exactly the same senses, all of
> those examples were testing for the same "objectness". (Although you
> could argue that Steven's examples were talking about subclass
> relationships rather than instance relationships, but really, an
> instanceof relationship is a special case of subclass, where the class
> on the left is defined by some form of object identity. Stating "All
> books are objects" is a subclass relationship; stating "This book is
> an object" is an instanceof relationship; both assertions use the same
> meaning of "object(s)".)
> 
> All shoes are objects.
> All ships are objects.
> All lumps of sealing-wax are objects.
> All cabbages are objects.
> All kings are objects. (Some kings are also subjects, but not many.)
> 
> I can pick up a shoe, I can look at it, I can verify that it has
> certain attributes common to all objects (location, mass, etc).
> Superman can do the same with a ship (they're a bit big for me to pick
> up). The actions you can perform on all objects are the attributes of
> objects in general. Some subclasses of object may provide additional
> actions (you can tip a king, for instance), but you can still do all
> of these actions, so they're all the same type of object.
> 
> Note that not one iota of this has anything to do with Python's
> documentation, its rules, its behaviour, or anything. Python follows
> the natural expectations of our universe. The standard actions that
> can be performed on Python objects are slightly different from the
> standard actions that can be performed in Dungeons and Dragons, and
> different again from the standard actions of the world we live in, but
> they are no less applicable to all objects in that hierarchy:
> 
> * You can bind a name to an object (any object).
> * You can pass an object as a function parameter.
> * You can return an object from a function.
> * You can reference objects in other objects' attributes.
> * You can distinguish one object from another (that is, objects have
> unique identities).
> 
> (In CPython, you can recognize an object by its GC header - an object
> invariably has a reference count and so on. This is not a fundamental
> attribute of objects, but it's a way to recognize them.)
> 
> Not everything that Python uses is an object. Syntax is not, itself,
> manipulable; you can represent a syntax tree with either source code
> (as a string) or an AST object, but syntax itself is not something you
> can grasp. You can't assign "foo = def" in Python. But anything that
> you can work with, in any way, is an object - and is an instance of
> (some subclass of) the type 'object'. So, yes: They ARE all the same
> meaning of "are" and "objects".

Lets take this practically.


>>> isinstance(1,int)
True
>>> 1 in [1,2,3]
True

>>> def int2():
...   i = 0
...   while True:
...    yield i
...    i += 1

>>> 1 in int2()
True
>>> 

>>> 1 in range(10)
True

4 True-returning python expressions.  5 if you consider that range in python 2 
and 3 are quite different.

Do you consider the computational processes evoked (in the CPython sources) by
these 4 statements to be the same?

[Hint: What does
>>> "shoes" in int2()
return?

And try to take care to distinguish computational from ontological answer
]


From marko at pacujo.net  Tue Jun  2 05:23:42 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 02 Jun 2015 12:23:42 +0300
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <bb48ac94-c2ac-4874-9c95-7ac2280fd5bc@googlegroups.com>
 <556d574f$0$12905$c3e8da3$5496439d@news.astraweb.com>
 <3d06de71-a517-4cac-be12-a61813465735@googlegroups.com>
 <mailman.48.1433234251.13271.python-list@python.org>
Message-ID: <87zj4i79b5.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> On Tue, Jun 2, 2015 at 5:44 PM, Rustom Mody <rustompmody at gmail.com> wrote:
>> 1 ? all these
>> Are the '?'s here same?  Similar?
>
> Yes, they are. Every one of them is asserting (or asking, depending on
> your point of view) whether or not the instance to its left is a
> member of the set to its right. The sets on the right are all
> different, but the set membership operation is identical.

That is the risk of taking classes into Python. People start to obsess
about membership and ontological relationships. I think you should use
the type() function only for troubleshooting and isinstance() virtually
never.

I think objects should be understood only through their operational
semantics, ie, their methods. It doesn't matter what an object is; what
matters is what it does. Don't test it, just call it.

Yeah, I'm preaching to the choir:

   Alex Martelli made an early (2000) use of the term in a message to
   the comp.lang.python newsgroup:

       In other words, don't check whether it IS-a duck: check whether
       it QUACKS-like-a duck, WALKS-like-a duck, etc, etc, depending on
       exactly what subset of duck-like behaviour you need to play your
       language-games with.

   <URL: http://en.wikipedia.org/wiki/Duck_typing>


Marko

PS I have written code that violates my stated rule. For example, an
argument could be a string or a list of strings for convenience.
However, this kind of testing should not be performed on anything
created with the "class" keyword. IOW, some types *are* more primitive
than others -- in our minds and in practical use.


From lac at openend.se  Tue Jun  2 05:39:29 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 02 Jun 2015 11:39:29 +0200
Subject: What use of string module?
In-Reply-To: Message from fl <rxjwg98@gmail.com> of "Mon,
 01 Jun 2015 19:14:18 -0700."
 <d3bf3182-d70e-4160-b600-733cbda748e6@googlegroups.com>
References: <d3bf3182-d70e-4160-b600-733cbda748e6@googlegroups.com>
Message-ID: <201506020939.t529dThW023102@fido.openend.se>

In a message of Mon, 01 Jun 2015 19:14:18 -0700, fl writes:
>Hi,
>
>I read the online help about string. It lists string constants, string 
>formatting, template strings and string functions. After reading these,
>I am still puzzled about how to use the string module. 
>
>Could you show me a few example about this module?
>
>Thanks

A long time ago, strings didn't have the methods they had now.
So the string module was created, so that people could all have
a common way to do some really common things that people like
to do with strings.  Now the strings have object methods to
do the same thing, so you don't really want to use it.

It's there for backwards compatibility.  You don't want all the
code that exists out there and uses it to all stop working one
day because we now have a better way to do it. :)

Laura


From fabien.maussion at gmail.com  Tue Jun  2 06:16:39 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Tue, 02 Jun 2015 12:16:39 +0200
Subject: Knowledge Requirements for Machine Learning (SKLearn)
References: <33dfc8d4-bd98-4648-8d7d-ed16070dda23@googlegroups.com>
Message-ID: <mkjvq6$2v3$1@speranza.aioe.org>

On 06/02/2015 11:16 AM, chrismeek4542 at gmail.com wrote:
> I would like to get into Machine learning.

An Introduction to Statistical Learning:
http://www-bcf.usc.edu/~gareth/ISL/

and corresponding MOOC.

SKLearn and Statsmodels mailing lists:
http://dir.gmane.org/gmane.comp.python.scikit-learn
http://dir.gmane.org/gmane.comp.python.pystatsmodels

Cheers,

Fabien



From lac at openend.se  Tue Jun  2 06:27:20 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 02 Jun 2015 12:27:20 +0200
Subject: Knowledge Requirements for Machine Learning (SKLearn)
In-Reply-To: Message from chrismeek4542@gmail.com of "Tue,
 02 Jun 2015 02:16:28 -0700."
 <33dfc8d4-bd98-4648-8d7d-ed16070dda23@googlegroups.com>
References: <33dfc8d4-bd98-4648-8d7d-ed16070dda23@googlegroups.com>
Message-ID: <201506021027.t52ARKJP024580@fido.openend.se>

https://www.udacity.com/course/intro-to-machine-learning--ud120

Free online course using Python.

Laura


From edd.cowan at gmail.com  Tue Jun  2 06:36:59 2015
From: edd.cowan at gmail.com (Eddilbert Macharia)
Date: Tue, 2 Jun 2015 03:36:59 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
Message-ID: <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>

On Sunday, May 31, 2015 at 5:34:20 PM UTC+3, Eddilbert Macharia wrote:
> Hello All ,
> 
> I'm wrecking my head trying to understand. where the class object comes into play . 
> 
> Is it only meant to act as base class and does it mean there is an actual class called object in python which all the objects created using the class type inherit ?
> 
> i'm assuming the metaclass if simplified would look something like this :
> 
> type('dict', (object,),{})
> 
> And when we use the class type as a metaclass are we using the instance version of the class type or are we actually using the type class itself ?
> 
> Also when we say everything is an object in python, are we referring to the fact that everything is an instance of the class type or does it have to with the object class inherited ? 
> 
> As can be attested by using type() function as below :
> 
> >>> type(int)
> <class 'type'>
> >>> type(list)
> <class 'type'>
> >>> type(dict)
> <class 'type'>
> >>> type(type)
> <class 'type'>
> >>> type(object)
> <class 'type'>
> 
> From my understanding this means all of this are instances of the class type. which means the class type was used to create this instances.
> 
> Now if i look at the __bases__ of all this objects i get :
> 
> >>> type.__base__
> <class 'object'>
> >>> type.__bases__
> (<class 'object'>,)
> >>> dict.__bases__
> (<class 'object'>,)
> >>> list.__bases__
> (<class 'object'>,)
> >>> int.__bases__
> (<class 'object'>,)
> >>> object.__bases__
> ()
> 
> This tells me that all of this objects inherit from the class object which has nothing to do with them being instances.

you guys are just confusing me, you are going in loops, and still i have understood ,what makes everything in python an object.
hey is where i'm at :
*** type in python refers to data types e.g. int, str, boolean e.t.c. right ?

*** The interpreter creates two classes type and object when setting up a python environment. right ?

*** The creator (metaclass) of all data types (i.e. int,str) in python is the class type. right ?

>>> isinstance(int,type)
True

*** The instance of class type is a data type an instance of class type. right ?
>>> type(type)
<class 'type'>
>>> isinstance(type,type)
True


*** Class type gets some of its behavior from class object through inheritance.right ?

>>> issubclass(type,object)
True

*** instance of class object is type, in the sense it created using class type which inherits from class object.right ?

>>> isinstance(object,type)
True
>>> isinstance(object,object)
True

****so is it right to say, everything in python is an object because they are instance of the class type which inherits from class object ?



From bc at freeuk.com  Tue Jun  2 06:45:45 2015
From: bc at freeuk.com (BartC)
Date: Tue, 02 Jun 2015 11:45:45 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <3t5bx.755107$dj1.564752@fx03.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <3t5bx.755107$dj1.564752@fx03.am4>
Message-ID: <vPfbx.868703$pj.293442@fx16.am4>

On 01/06/2015 23:59, BartC wrote:

>> In the new class style, type and class sort of mean the same thing.
>
> I'm developing a new language along the lines of Python,

(After reading some of the rest of the sub-thread, I'm glad I reined in 
some of the proposed changes to my existing language to make it more 
Python-like.

If my simple, no-nonsense approach can avoid discussions about 
philosophy, religion and meta-physics (or whatever the hell the 
sub-thread was about) then that must be a good thing!)

-- 
Bartc



From breamoreboy at yahoo.co.uk  Tue Jun  2 07:11:44 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 02 Jun 2015 12:11:44 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
Message-ID: <mkk31j$bar$1@ger.gmane.org>

On 02/06/2015 11:36, Eddilbert Macharia wrote:
>
> you guys are just confusing me, you are going in loops, and still i have understood ,what makes everything in python an object.

After 14 or more years writing Python I can't really answer that and 
quite frankly it doesn't worry me as I can quite happily write working 
code, what with practicality beating purity and all that.  Does it worry 
you that much?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rustompmody at gmail.com  Tue Jun  2 07:16:16 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 2 Jun 2015 04:16:16 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <vPfbx.868703$pj.293442@fx16.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <3t5bx.755107$dj1.564752@fx03.am4>
 <vPfbx.868703$pj.293442@fx16.am4>
Message-ID: <c9932ea3-7124-4b0a-acdf-7eebdbf5fac5@googlegroups.com>

On Tuesday, June 2, 2015 at 4:16:02 PM UTC+5:30, BartC wrote:
> On 01/06/2015 23:59, BartC wrote:
> 
> >> In the new class style, type and class sort of mean the same thing.
> >
> > I'm developing a new language along the lines of Python,
> 
> (After reading some of the rest of the sub-thread, I'm glad I reined in 
> some of the proposed changes to my existing language to make it more 
> Python-like.
> 
> If my simple, no-nonsense approach can avoid discussions about 
> philosophy, religion and meta-physics (or whatever the hell the 
> sub-thread was about) then that must be a good thing!)

What is good that each one can only decide in his own private space.
What is historical is public and studying that history would show that
these questions are central to the existence of our field,
[Mark's views nowithstanding]
http://blog.languager.org/2015/03/cs-history-0.html


From mail.acdr at gmail.com  Tue Jun  2 07:26:42 2015
From: mail.acdr at gmail.com (acdr)
Date: Tue, 2 Jun 2015 13:26:42 +0200
Subject: for...else
Message-ID: <CANFcO8tOLorV1kuGJo6Vgb-e02EhC3NL2U7oiNRNF+b2s2M2ig@mail.gmail.com>

Hi,

Currently, in various places in my code, I have the equivalent of:

for x in it:
    if complicated_calculation_1():
        cleanup()
        break
    complicated_calculation_2()
    if complicated_calculation_3():
        cleanup()
        break

Obviously, I'm repeating myself by having two separate calls to
cleanup(). I can't really see a nicer way to do this. (Though I see
plenty of non-nice ways to do this, such as adding "broken = True" in
front of every "break", and then after the loop is done, have an "if
broken" section.) Other solutions that I'm not particularly fond of
can be found on stackexchange, where someone else is trying to do the
same thing:
http://stackoverflow.com/questions/3296044/opposite-of-python-for-else

I'm wondering if there is a demand for expanding the "for...else"
functionality to be expanded also have a block of code that only gets
called if the loop is broken out of. I.e.:

for x in it:
    ...
then:
    # "break" was called
    ...
else:
    # "break was not called
    ...


From steve at pearwood.info  Tue Jun  2 07:27:21 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 02 Jun 2015 21:27:21 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
Message-ID: <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>

On Tue, 2 Jun 2015 08:36 pm, Eddilbert Macharia wrote:

> you guys are just confusing me, you are going in loops, and still i have
> understood ,what makes everything in python an object. hey is where i'm at
> : *** type in python refers to data types e.g. int, str, boolean e.t.c.
> right ?

Yes. Also classes you create with the "class" keyword:

class K(object):
    ...

K is now a "type", just like int, str, list, object, etc.


> *** The interpreter creates two classes type and object when setting up a
> python environment. right ?

Many more than just two: it also creates list, str, dict, etc. But *first*
it has to create type and object. So you are correct.


> *** The creator (metaclass) of all data types (i.e. int,str) in python is
> the class type. right ?

Correct.

[Aside: I'm only talking about Python 3 here. In Python 2 there is also a
second hierarchy of classes, called "classic classes" or "old-style
classes", which are *not* subclasses of type. But let's just ignore them,
because they are gone in the most recent versions of Python.]


>>>> isinstance(int,type)
> True
> 
> *** The instance of class type is a data type an instance of class type.
> right ?
>>>> type(type)
> <class 'type'>

type has many instances, not just one.

Instances of int are individual ints, like 0, 1, 2, 3, ...

Instances of type are individual types, like int, dict, str, list, ...

But one of those many instances of type is, yes, type itself! So *one* of
the instances of type is type, which is also an instance of itself:

>>>> isinstance(type,type)
> True

Correct. This makes type special. Most types are *not* instances of
themselves:

py> isinstance(int, int)
False


> *** Class type gets some of its behavior from class object through
> inheritance.right ?
> 
>>>> issubclass(type,object)
> True

Correct.


> *** instance of class object is type, in the sense it created using class
> type which inherits from class object.right ?
> 
>>>> isinstance(object,type)
> True
>>>> isinstance(object,object)
> True
> 
> ****so is it right to say, everything in python is an object because they
> are instance of the class type which inherits from class object ?

No! That's not what we mean when we say "everything is an object".

Eddilbert, have you programmed in any other languages? It would help you
understand if you have.

"Object" has a general meaning in computer science and programming, it is a
compound data structure that is explicitly linked to a type which provides
functionality that operates on that data structure.

In the programming language C, *no* values are objects. C has types (int16,
uint32, bool, and many more) but no objects.

In the programming language Java, *some* values are objects, and some values
are not objects.

In the programming language Python, *all* values are objects, in the general
sense. That is what we mean by "Everything is an object".

Let's go back in time about 15 years or so. We're now using Python 1.5. In
Python 1.5, there is no built-in object, and type is just a function, not a
class:

>>> import sys
>>> print sys.version
1.5.2 (#1, Aug 27 2012, 09:09:18)  [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]
>>> object
Traceback (innermost last):
  File "<stdin>", line 1, in ?
NameError: object
>>> type
<built-in function type>


In Python 1.5, classes you create do not inherit from object, because object
does not exist! BUT even in Python 1.5, it is true that everything is an
object.

Remember, "object" can refer to two things:

- *specifically* the class called "object";

- the *general concept* of objects, from object oriented programming.


In Python 1.5:

- everything is an object [the general concept]

- nothing is an instance of the class called "object"


In Python 2:

- everything is an object [the general concept]

- some things, but not all things, are instances of the class
  called "object"


In Python 3:

- everything is an object [the general concept]

- everything is an instance of the class called "object"




-- 
Steven



From rustompmody at gmail.com  Tue Jun  2 07:40:33 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 2 Jun 2015 04:40:33 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <1d07e9d4-8314-4b5a-8f98-4174b89370ee@googlegroups.com>

On Tuesday, June 2, 2015 at 4:57:31 PM UTC+5:30, Steven D'Aprano wrote:
> On Tue, 2 Jun 2015 08:36 pm, Eddilbert Macharia wrote:
> 
> > you guys are just confusing me, you are going in loops, and still i have
> > understood ,what makes everything in python an object. hey is where i'm at
> > : *** type in python refers to data types e.g. int, str, boolean e.t.c.
> > right ?
> 
> Yes. Also classes you create with the "class" keyword:
> 
> class K(object):
>     ...
> 
> K is now a "type", just like int, str, list, object, etc.
> 
> 
> > *** The interpreter creates two classes type and object when setting up a
> > python environment. right ?
> 
> Many more than just two: it also creates list, str, dict, etc. But *first*
> it has to create type and object. So you are correct.
> 
> 
> > *** The creator (metaclass) of all data types (i.e. int,str) in python is
> > the class type. right ?
> 
> Correct.
> 
> [Aside: I'm only talking about Python 3 here. In Python 2 there is also a
> second hierarchy of classes, called "classic classes" or "old-style
> classes", which are *not* subclasses of type. But let's just ignore them,
> because they are gone in the most recent versions of Python.]
> 
> 
> >>>> isinstance(int,type)
> > True
> > 
> > *** The instance of class type is a data type an instance of class type.
> > right ?
> >>>> type(type)
> > <class 'type'>
> 
> type has many instances, not just one.

Not sure what you are trying to emphasize by the 'not just one'
In particular the issubclass relation is transitive
Whereas the isinstance is not

>>> isinstance(2,int)
True
>>> isinstance(int,type)
True
>>> isinstance(2,type)
False


From toddrjen at gmail.com  Tue Jun  2 07:49:09 2015
From: toddrjen at gmail.com (Todd)
Date: Tue, 2 Jun 2015 13:49:09 +0200
Subject: for...else
In-Reply-To: <CANFcO8tOLorV1kuGJo6Vgb-e02EhC3NL2U7oiNRNF+b2s2M2ig@mail.gmail.com>
References: <CANFcO8tOLorV1kuGJo6Vgb-e02EhC3NL2U7oiNRNF+b2s2M2ig@mail.gmail.com>
Message-ID: <CAFpSVp+pVYfhHfrS+aEeAMkXzkw9xKUZW0grGPf9s4=_fpiTUg@mail.gmail.com>

I think there is essentially zero chance of that.  My understanding is that
Guido regrets having "else" to begin with.

But this should work

broken = True
for x in it:
    if complicated_calculation_1():
        break
    complicated_calculation_2()
    if complicated_calculation_3():
        break
else:
    broken = False
if broken:
    cleanup()


On Tue, Jun 2, 2015 at 1:26 PM, acdr <mail.acdr at gmail.com> wrote:

> Hi,
>
> Currently, in various places in my code, I have the equivalent of:
>
> for x in it:
>     if complicated_calculation_1():
>         cleanup()
>         break
>     complicated_calculation_2()
>     if complicated_calculation_3():
>         cleanup()
>         break
>
> Obviously, I'm repeating myself by having two separate calls to
> cleanup(). I can't really see a nicer way to do this. (Though I see
> plenty of non-nice ways to do this, such as adding "broken = True" in
> front of every "break", and then after the loop is done, have an "if
> broken" section.) Other solutions that I'm not particularly fond of
> can be found on stackexchange, where someone else is trying to do the
> same thing:
> http://stackoverflow.com/questions/3296044/opposite-of-python-for-else
>
> I'm wondering if there is a demand for expanding the "for...else"
> functionality to be expanded also have a block of code that only gets
> called if the loop is broken out of. I.e.:
>
> for x in it:
>     ...
> then:
>     # "break" was called
>     ...
> else:
>     # "break was not called
>     ...
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150602/0671e880/attachment.html>

From jeanmichel at sequans.com  Tue Jun  2 08:01:03 2015
From: jeanmichel at sequans.com (Jean-Michel Pichavant)
Date: Tue, 2 Jun 2015 14:01:03 +0200 (CEST)
Subject: for...else
In-Reply-To: <CANFcO8tOLorV1kuGJo6Vgb-e02EhC3NL2U7oiNRNF+b2s2M2ig@mail.gmail.com>
Message-ID: <1831776353.1951415.1433246463232.JavaMail.root@sequans.com>

----- Original Message -----
> From: "acdr" <mail.acdr at gmail.com>
> To: python-list at python.org
> Sent: Tuesday, 2 June, 2015 1:26:42 PM
> Subject: for...else
> 
> Hi,
> 
> Currently, in various places in my code, I have the equivalent of:
> 
> for x in it:
>     if complicated_calculation_1():
>         cleanup()
>         break
>     complicated_calculation_2()
>     if complicated_calculation_3():
>         cleanup()
>         break

Hi

With the following layout, adding calculation steps is just a matter of adding a line

for x ion it:
  for calc, cbk in [
    (complicated_calculation_1, cleanup),
    (complicated_calculation_2, None),
    (complicated_calculation_3, cleanup),
  ]:
    if calc() and cbk:
      cbk()
      break
      
It might give you ideas.

JM


-- IMPORTANT NOTICE: 

The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

From chrismeek4542 at gmail.com  Tue Jun  2 08:17:40 2015
From: chrismeek4542 at gmail.com (chrismeek4542 at gmail.com)
Date: Tue, 2 Jun 2015 05:17:40 -0700 (PDT)
Subject: Knowledge Requirements for Machine Learning (SKLearn)
In-Reply-To: <33dfc8d4-bd98-4648-8d7d-ed16070dda23@googlegroups.com>
References: <33dfc8d4-bd98-4648-8d7d-ed16070dda23@googlegroups.com>
Message-ID: <1929839b-705e-4b1c-b6e2-641128ed8905@googlegroups.com>

Thank you both. Just what i was looking for


From rxjwg98 at gmail.com  Tue Jun  2 08:23:23 2015
From: rxjwg98 at gmail.com (fl)
Date: Tue, 2 Jun 2015 05:23:23 -0700 (PDT)
Subject: How to access the low digits of a list
Message-ID: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>

Hi,

I have a list:






>>> lines
['12', '42', '49', '156', '225', '36', '49', '164', '11181', '3100']



I want to access the last two digits. That is:

['12', '42', '49', '56', '25', '36', '49', '64', '81', '00']


When I try to use lines[3][0] is '1'
lines[3][1] is '5'
lines[3][2] is '6'


I don't know whether there is a way to know the length of lines[3]. Then, I
can use a -1 step to get the last two digits. Or, you may have much better 
ways to do that. Python is really too versatile I feel.

Thanks,


From __peter__ at web.de  Tue Jun  2 08:27:35 2015
From: __peter__ at web.de (Peter Otten)
Date: Tue, 02 Jun 2015 14:27:35 +0200
Subject: for...else
References: <CANFcO8tOLorV1kuGJo6Vgb-e02EhC3NL2U7oiNRNF+b2s2M2ig@mail.gmail.com>
Message-ID: <mkk7fp$m2h$1@ger.gmane.org>

acdr wrote:

> Hi,
> 
> Currently, in various places in my code, I have the equivalent of:
> 
> for x in it:
>     if complicated_calculation_1():
>         cleanup()
>         break
>     complicated_calculation_2()
>     if complicated_calculation_3():
>         cleanup()
>         break
> 
> Obviously, I'm repeating myself by having two separate calls to
> cleanup(). I can't really see a nicer way to do this. (Though I see
> plenty of non-nice ways to do this, such as adding "broken = True" in
> front of every "break", and then after the loop is done, have an "if
> broken" section.) Other solutions that I'm not particularly fond of
> can be found on stackexchange, where someone else is trying to do the
> same thing:
> http://stackoverflow.com/questions/3296044/opposite-of-python-for-else

Perhaps you like of these:

for x in it:
    if not complicated_calculation_1():
        complicated_calculation_2()
        if not complicated_calculation_3():
            continue
    cleanup()
    break

def f():
    for x in it:
        if complicated_calculation_1():
            break
        complicated_calculation_2()
        if complicated_calculation_3():
            break
    else:
        return
    cleanup()

> I'm wondering if there is a demand for expanding the "for...else"
> functionality to be expanded also have a block of code that only gets
> called if the loop is broken out of. I.e.:
> 
> for x in it:
>     ...
> then:
>     # "break" was called
>     ...
> else:
>     # "break was not called
>     ...

You may not like using a flag, but it's a really obvious approach and the 
situations where it's necessary are not very common. Apart from indentation-
based blocks Python is very middle-of-the-road, so I'd ask if there is an 
existing language that has such a feature, and if yes, is it used 
frequently? I suspect that there are many Python programmers that have never 
even used (for...else).

Python the language is already becoming too complicated for my taste; I 
would rather not add more constructs where there is no significant 
advantage. 




From joel.goldstick at gmail.com  Tue Jun  2 08:32:00 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 2 Jun 2015 08:32:00 -0400
Subject: How to access the low digits of a list
In-Reply-To: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>
References: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>
Message-ID: <CAPM-O+wNiws7XaegXGp+O5+av30u8mijuoZFA=FvMUp8KJe0YQ@mail.gmail.com>

On Tue, Jun 2, 2015 at 8:23 AM, fl <rxjwg98 at gmail.com> wrote:
> Hi,
>
> I have a list:
>
>
>
>
>
>
>>>> lines
> ['12', '42', '49', '156', '225', '36', '49', '164', '11181', '3100']
>
>
>
> I want to access the last two digits. That is:
>
> ['12', '42', '49', '56', '25', '36', '49', '64', '81', '00']
>
>
> When I try to use lines[3][0] is '1'
> lines[3][1] is '5'
> lines[3][2] is '6'
>
>
> I don't know whether there is a way to know the length of lines[3]. Then, I
> can use a -1 step to get the last two digits. Or, you may have much better
> ways to do that. Python is really too versatile I feel.
>
> Thanks,
> --
> https://mail.python.org/mailman/listinfo/python-list

You should read about slices

lines[3][-2:]  will give the last two characters in the 3rd group
(starting from 0)

The notation means to start with 2 from the end, and proceed to the end


-- 
Joel Goldstick
http://joelgoldstick.com


From stutzman at cat2.kjsl.com  Tue Jun  2 08:34:14 2015
From: stutzman at cat2.kjsl.com (Frank Stutzman)
Date: Tue, 2 Jun 2015 12:34:14 +0000 (UTC)
Subject: How to access the low digits of a list
References: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>
Message-ID: <mkk7s6$1iut$1@news.kjsl.com>

fl <rxjwg98 at gmail.com> wrote:

---snip----
>>>> lines
> ['12', '42', '49', '156', '225', '36', '49', '164', '11181', '3100']
> 
> 
> 
> I want to access the last two digits. That is:
> 
> ['12', '42', '49', '56', '25', '36', '49', '64', '81', '00']
> 
> 
> When I try to use lines[3][0] is '1'
> lines[3][1] is '5'
> lines[3][2] is '6'

Is there something wrong with using:
   lines[3][-2:]
   

-- 
Frank Stutzman



From rustompmody at gmail.com  Tue Jun  2 08:35:03 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 2 Jun 2015 05:35:03 -0700 (PDT)
Subject: How to access the low digits of a list
In-Reply-To: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>
References: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>
Message-ID: <18a4709f-47c8-41f7-87d5-08bbb00166b6@googlegroups.com>

On Tuesday, June 2, 2015 at 5:53:34 PM UTC+5:30, fl wrote:
> Hi,
> 
> I have a list:
> 
> 
> 
> 
> 
> 
> >>> lines
> ['12', '42', '49', '156', '225', '36', '49', '164', '11181', '3100']
> 
> 
> 
> I want to access the last two digits. That is:
> 
> ['12', '42', '49', '56', '25', '36', '49', '64', '81', '00']
> 
> 
> When I try to use lines[3][0] is '1'
> lines[3][1] is '5'
> lines[3][2] is '6'
> 
> 
> I don't know whether there is a way to know the length of lines[3]. Then, I
> can use a -1 step to get the last two digits. Or, you may have much better 
> ways to do that. Python is really too versatile I feel.

len(lines[3])  ??

You can do this: [Hope I am not doing your homework!]

>>> [(x[-2:] if len(x) > 2 else x)  for x in lines]
['12', '42', '49', '56', '25', '36', '49', '64', '81', '00']
>>> 

For that matter even this works
But I am not sure whats happening or that I like it

>>> [x[-2:]  for x in lines]
['12', '42', '49', '56', '25', '36', '49', '64', '81', '00']
>>> 


From swapnilrbhadade83 at gmail.com  Tue Jun  2 08:41:31 2015
From: swapnilrbhadade83 at gmail.com (Swapnil Bhadade)
Date: Tue, 2 Jun 2015 05:41:31 -0700 (PDT)
Subject: Framework / CMS for Banking
Message-ID: <8008a22e-81d6-4350-b569-2ef00de04883@googlegroups.com>

 I am want to know which are the best CMS / framework for building web banking / Financial / lending services ????
Best


From bc at freeuk.com  Tue Jun  2 08:49:32 2015
From: bc at freeuk.com (BartC)
Date: Tue, 02 Jun 2015 13:49:32 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <yDhbx.572527$oZ4.306159@fx38.am4>

On 02/06/2015 12:27, Steven D'Aprano wrote:

> "Object" has a general meaning in computer science and programming, it is a
> compound data structure that is explicitly linked to a type which provides
> functionality that operates on that data structure.
>
> In the programming language C, *no* values are objects. C has types (int16,
> uint32, bool, and many more) but no objects.

The C Standard has hundreds of references to 'object'.

C objects are also linked to a type but it doesn't need to be explicit 
/in the run-time data/ because it's a static language. The compiler 
knows all the types because there are explicit annotations in the source 
code.

> In the programming language Java, *some* values are objects, and some values
> are not objects.
>
> In the programming language Python, *all* values are objects, in the general
> sense. That is what we mean by "Everything is an object".

In a dynamically typed language such as Python, you need to be able to 
deal with values consistently whatever their type, simply because you 
can't tell what the types are from source code. (Not without a lot of 
work which Python doesn't attempt, although RPython might do so.) Example:

  print (a)

a might be the int 42, or a it might be a million-element list. So both 
are wrapped up as 'objects'.

Java is statically typed which makes it possible to treat instance 
variables differently without needing to examine their types at runtime. 
If you define 'object' in a certain way (eg. as boxed, tagged data), 
then it follows that some values don't need to be objects.

-- 
Bartc


From mail.acdr at gmail.com  Tue Jun  2 08:52:21 2015
From: mail.acdr at gmail.com (acdr)
Date: Tue, 2 Jun 2015 14:52:21 +0200
Subject: for...else
In-Reply-To: <1831776353.1951415.1433246463232.JavaMail.root@sequans.com>
References: <CANFcO8tOLorV1kuGJo6Vgb-e02EhC3NL2U7oiNRNF+b2s2M2ig@mail.gmail.com>
 <1831776353.1951415.1433246463232.JavaMail.root@sequans.com>
Message-ID: <CANFcO8sQ0BjHU+7i6XEpaygZLhhi2R0zh0enNk4zk=L8QT3=kg@mail.gmail.com>

That would work for my example, but it would only really work if all
the calculations are in a nice function. If I just have a block of
code, then I can't use it as easily. (E.g. replace
"complicated_calculation_2()" by multiple lines of code, which I don't
want to shove into a separate function.)

On 6/2/15, Jean-Michel Pichavant <jeanmichel at sequans.com> wrote:
> ----- Original Message -----
>> From: "acdr" <mail.acdr at gmail.com>
>> To: python-list at python.org
>> Sent: Tuesday, 2 June, 2015 1:26:42 PM
>> Subject: for...else
>>
>> Hi,
>>
>> Currently, in various places in my code, I have the equivalent of:
>>
>> for x in it:
>>     if complicated_calculation_1():
>>         cleanup()
>>         break
>>     complicated_calculation_2()
>>     if complicated_calculation_3():
>>         cleanup()
>>         break
>
> Hi
>
> With the following layout, adding calculation steps is just a matter of
> adding a line
>
> for x ion it:
>   for calc, cbk in [
>     (complicated_calculation_1, cleanup),
>     (complicated_calculation_2, None),
>     (complicated_calculation_3, cleanup),
>   ]:
>     if calc() and cbk:
>       cbk()
>       break
>
> It might give you ideas.
>
> JM
>
>
> -- IMPORTANT NOTICE:
>
> The contents of this email and any attachments are confidential and may also
> be privileged. If you are not the intended recipient, please notify the
> sender immediately and do not disclose the contents to any other person, use
> it for any purpose, or store or copy the information in any medium. Thank
> you.
>


From alex at duboaa.net  Tue Jun  2 09:09:34 2015
From: alex at duboaa.net (Alexis Dubois)
Date: Tue, 2 Jun 2015 06:09:34 -0700 (PDT)
Subject: Python.exe has stopped working
Message-ID: <8a6ed08d-7dd6-4a4a-a466-9d3ad98f8dff@googlegroups.com>

Hello !

I have this kind of message every time I quit my PyQt4 app whatever the method to quit is: a quit action menu, the windows "red cross", by quit(), close(), destroy(), deletelater(), ...

"python.exe has stopped working"
->Check online for a solution
->Close the program

I need to specify that the app is working nice, just have this message when quit.
Do you have an idea to help me understand this issue?

Thanks in advance
Best regards
Alexis


From marko at pacujo.net  Tue Jun  2 09:13:23 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 02 Jun 2015 16:13:23 +0300
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
Message-ID: <87siaa6yoc.fsf@elektro.pacujo.net>

BartC <bc at freeuk.com>:

> If you define 'object' in a certain way (eg. as boxed, tagged data),
> then it follows that some values don't need to be objects.

The word "object" really barely carries any meaning -- that's the point.
It's a Latin-based synonym of the Germanic "thing."

To say that everything is an object in Python simply means that the same
set of concepts applies to all data. The main distinguishing feature
between Python objects is mutability.

The utmost abstraction of everything into an object is the pinnacle of
understanding. The question is, should a beginning Python programmer
start the climb from the summit or from the root of the mountain. IOW,
does it help to start the Python journey by learning that numbers are
objects, given that the language gives lots of special treatment to
numbers (as well as strings, lists and functions, for example)?

I would imagine that the gentlest route to Python programming starts
without talking much about objects at all. Leave the discussion about
numbers as objects till after, say, multiple inheritance has been
covered. Then, things like strings as objects and regular expression
match objects can be brought in naturally, and expressions like

   ','.join(person.name
            for person in persons
            if person.is_underage())

no longer defy comprehension.


Marko


From jeanmichel at sequans.com  Tue Jun  2 09:46:55 2015
From: jeanmichel at sequans.com (Jean-Michel Pichavant)
Date: Tue, 2 Jun 2015 15:46:55 +0200 (CEST)
Subject: for...else
In-Reply-To: <CANFcO8sQ0BjHU+7i6XEpaygZLhhi2R0zh0enNk4zk=L8QT3=kg@mail.gmail.com>
Message-ID: <23156522.1965199.1433252815342.JavaMail.root@sequans.com>

----- Original Message -----
> From: "acdr" <mail.acdr at gmail.com>
> To: "Jean-Michel Pichavant" <jeanmichel at sequans.com>
> Cc: python-list at python.org
> Sent: Tuesday, 2 June, 2015 2:52:21 PM
> Subject: Re: for...else
> 
> That would work for my example, but it would only really work if all
> the calculations are in a nice function. 

You cannot blame me for considering the example you provided ;)

What about this:

class Cleanup(Exception): pass

try:
  for x in it:
    if c1():
      raise Cleanup()
    c2()
    if c3():
      raise Cleanup()
except Cleanup:
  #do the cleanup
   pass

If you can make c1 c3 raise themselves Cleanup, it's even better.

Now if you're able to write a cleanup function that can handle the case when there's nothing to clean, everything becomes crystal clear:

from contextlib import contextmanager

@contextmanager
def cleanup():
  yield
  # here do the cleaning
  print 'I am cleaning'

def do_the_job():
  if c1() : return
  c2()
  if c3() : return

with cleanup():
  do_the_job()

JM


-- IMPORTANT NOTICE: 

The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

From invalid at invalid.invalid  Tue Jun  2 09:48:43 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Tue, 2 Jun 2015 13:48:43 +0000 (UTC)
Subject: Parser needed.
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <mailman.31.1433207544.13271.python-list@python.org>
 <52c36$556d048a$5419aafe$44775@news.ziggo.nl>
Message-ID: <mkkc7r$je4$1@reader1.panix.com>

On 2015-06-02, Skybuck Flying <skybuck2000 at hotmail.com> wrote:

><structure name>
> {
>    <SomeField> <SomeData>
>     <another structure name>
>    {
>       <SomeField> <SomeData>
>    }
> }

IOW, it's almost, but not quite, JSON.

-- 
Grant Edwards               grant.b.edwards        Yow! My vaseline is
                                  at               RUNNING...
                              gmail.com            


From skip.montanaro at gmail.com  Tue Jun  2 09:58:54 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 2 Jun 2015 08:58:54 -0500
Subject: fork/exec & close file descriptors
In-Reply-To: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
Message-ID: <CANc-5UydTX0iw+8Ac2FeE_ziPtUMe=dargB46+gfQfCG2N1cOw@mail.gmail.com>

Reviving (and concluding) a thread I started a couple weeks ago, I asked:

> The basic fork/exec dance is not a problem, but how do I discover
> all the open file descriptors in the new child process to make sure
> they get closed? Do I simply start at fd 3 and call os.close() on
> everything up to some largish fd number?

I wanted this again today (for different reasons than before).
Googling for "python close all file descriptors" returned the os
module docs as the first hit, and lo and behold, what do I see
documented? os.closerange (new in 2.6):

    os.closerange(fd_low, fd_high)
    Close all file descriptors from fd_low (inclusive) to fd_high
    (exclusive), ignoring errors.

Guido's time machine strikes again...

Skip


From mail.acdr at gmail.com  Tue Jun  2 10:00:12 2015
From: mail.acdr at gmail.com (acdr)
Date: Tue, 2 Jun 2015 16:00:12 +0200
Subject: for...else
In-Reply-To: <23156522.1965199.1433252815342.JavaMail.root@sequans.com>
References: <CANFcO8sQ0BjHU+7i6XEpaygZLhhi2R0zh0enNk4zk=L8QT3=kg@mail.gmail.com>
 <23156522.1965199.1433252815342.JavaMail.root@sequans.com>
Message-ID: <CANFcO8t18csV0MMcE9hnYVN2mgrPi2Rco5K6Q+wuniVpfreYJw@mail.gmail.com>

The first solution in your e-mail (with a Cleanup exception) is
definitely very close, functionally, to what I want to accomplish. In
effect, it's the same structure as my original suggestion of
"for...then...else", except now it'd be "try: for...else...except".
That's workable. I can even cheat and rename the Cleanup class to
"Break" for clarity.

I'm guessing that there is not much support for functionality like
this to be built in, in a much less verbose manner, like the
"for...else" functionality? :P

On 6/2/15, Jean-Michel Pichavant <jeanmichel at sequans.com> wrote:
> ----- Original Message -----
>> From: "acdr" <mail.acdr at gmail.com>
>> To: "Jean-Michel Pichavant" <jeanmichel at sequans.com>
>> Cc: python-list at python.org
>> Sent: Tuesday, 2 June, 2015 2:52:21 PM
>> Subject: Re: for...else
>>
>> That would work for my example, but it would only really work if all
>> the calculations are in a nice function.
>
> You cannot blame me for considering the example you provided ;)
>
> What about this:
>
> class Cleanup(Exception): pass
>
> try:
>   for x in it:
>     if c1():
>       raise Cleanup()
>     c2()
>     if c3():
>       raise Cleanup()
> except Cleanup:
>   #do the cleanup
>    pass
>
> If you can make c1 c3 raise themselves Cleanup, it's even better.
>
> Now if you're able to write a cleanup function that can handle the case when
> there's nothing to clean, everything becomes crystal clear:
>
> from contextlib import contextmanager
>
> @contextmanager
> def cleanup():
>   yield
>   # here do the cleaning
>   print 'I am cleaning'
>
> def do_the_job():
>   if c1() : return
>   c2()
>   if c3() : return
>
> with cleanup():
>   do_the_job()
>
> JM
>
>
> -- IMPORTANT NOTICE:
>
> The contents of this email and any attachments are confidential and may also
> be privileged. If you are not the intended recipient, please notify the
> sender immediately and do not disclose the contents to any other person, use
> it for any purpose, or store or copy the information in any medium. Thank
> you.
>


From lac at openend.se  Tue Jun  2 10:05:55 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 02 Jun 2015 16:05:55 +0200
Subject: Framework / CMS for Banking
In-Reply-To: Message from Swapnil Bhadade <swapnilrbhadade83@gmail.com>
 of "Tue, 02 Jun 2015 05:41:31 -0700."
 <8008a22e-81d6-4350-b569-2ef00de04883@googlegroups.com>
References: <8008a22e-81d6-4350-b569-2ef00de04883@googlegroups.com>
Message-ID: <201506021405.t52E5tLa029778@fido.openend.se>

In a message of Tue, 02 Jun 2015 05:41:31 -0700, Swapnil Bhadade writes:
> I am want to know which are the best CMS / framework for building web banking / Financial / lending services ????
>Best
>-- 
>https://mail.python.org/mailman/listinfo/python-list

See this video.
https://ep2013.europython.eu/conference/talks/python-in-banks

Laura


From lac at openend.se  Tue Jun  2 10:12:59 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 02 Jun 2015 16:12:59 +0200
Subject: Python.exe has stopped working
In-Reply-To: Message from Alexis Dubois <alex@duboaa.net> of "Tue,
 02 Jun 2015 06:09:34 -0700."
 <8a6ed08d-7dd6-4a4a-a466-9d3ad98f8dff@googlegroups.com>
References: <8a6ed08d-7dd6-4a4a-a466-9d3ad98f8dff@googlegroups.com>
Message-ID: <201506021412.t52ECxHE030144@fido.openend.se>

In a message of Tue, 02 Jun 2015 06:09:34 -0700, Alexis Dubois writes:
>Hello !
>
>I have this kind of message every time I quit my PyQt4 app whatever the method to quit is: a quit action menu, the windows "red cross", by quit(), close(), destroy(), deletelater(), ...
>
>"python.exe has stopped working"
>->Check online for a solution
>->Close the program
>
>I need to specify that the app is working nice, just have this message when quit.
>Do you have an idea to help me understand this issue?
>
>Thanks in advance
>Best regards
>Alexis
>-- 
>https://mail.python.org/mailman/listinfo/python-list

You can override closeEvent

def closeEvent(self, event):
    QtGui.qApp.quit()
        event.ignore()

This just shuts it up, you may have more complicated things you want to do.

Laura





From ian.g.kelly at gmail.com  Tue Jun  2 10:19:39 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 2 Jun 2015 08:19:39 -0600
Subject: How to access the low digits of a list
In-Reply-To: <18a4709f-47c8-41f7-87d5-08bbb00166b6@googlegroups.com>
References: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>
 <18a4709f-47c8-41f7-87d5-08bbb00166b6@googlegroups.com>
Message-ID: <CALwzidmXw_NmLs_OO7GyxqZbCgVMcjuJCvH_kkOfU9JjMYCEGQ@mail.gmail.com>

On Tue, Jun 2, 2015 at 6:35 AM, Rustom Mody <rustompmody at gmail.com> wrote:
> For that matter even this works
> But I am not sure whats happening or that I like it
>
>>>> [x[-2:]  for x in lines]
> ['12', '42', '49', '56', '25', '36', '49', '64', '81', '00']

x[-2:] selects all items in the sequence with index i such that len(x)
- 2 <= i < len(x). For a sequence of length 2 or less, that's the
entire sequence.


From alex at duboaa.net  Tue Jun  2 10:29:16 2015
From: alex at duboaa.net (Alexis Dubois)
Date: Tue, 2 Jun 2015 07:29:16 -0700 (PDT)
Subject: Python.exe has stopped working
In-Reply-To: <mailman.62.1433254395.13271.python-list@python.org>
References: <8a6ed08d-7dd6-4a4a-a466-9d3ad98f8dff@googlegroups.com>
 <mailman.62.1433254395.13271.python-list@python.org>
Message-ID: <7c4cd02a-adfa-44c1-80f6-406b3447fb0c@googlegroups.com>

On Tuesday, June 2, 2015 at 4:13:27 PM UTC+2, Laura Creighton wrote:
> In a message of Tue, 02 Jun 2015 06:09:34 -0700, Alexis Dubois writes:
> >Hello !
> >
> >I have this kind of message every time I quit my PyQt4 app whatever the method to quit is: a quit action menu, the windows "red cross", by quit(), close(), destroy(), deletelater(), ...
> >
> >"python.exe has stopped working"
> >->Check online for a solution
> >->Close the program
> >
> >I need to specify that the app is working nice, just have this message when quit.
> >Do you have an idea to help me understand this issue?
> >
> >Thanks in advance
> >Best regards
> >Alexis
> >-- 
> >https://mail.python.org/mailman/listinfo/python-list
> 
> You can override closeEvent
> 
> def closeEvent(self, event):
>     QtGui.qApp.quit()
>         event.ignore()
> 
> This just shuts it up, you may have more complicated things you want to do.
> 
> Laura

Thanks Laura,

Unfortunately, my closeEvent is already override as this:

def closeEvent(self, event):        
        reply = QtGui.QMessageBox.question(self, 'Message',
            "Are you sure to quit HWL?", QtGui.QMessageBox.Yes | 
            QtGui.QMessageBox.No, QtGui.QMessageBox.No)

        if reply == QtGui.QMessageBox.Yes:
            app.quit()
        else:
            event.ignore()

Replacing app.quit() by QtGui.qApp.quit() doesn't change anything.


From lac at openend.se  Tue Jun  2 10:29:50 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 02 Jun 2015 16:29:50 +0200
Subject: for...else
In-Reply-To: Message from acdr <mail.acdr@gmail.com> of "Tue,
 02 Jun 2015 14:52:21 +0200."
 <CANFcO8sQ0BjHU+7i6XEpaygZLhhi2R0zh0enNk4zk=L8QT3=kg@mail.gmail.com>
References: <CANFcO8tOLorV1kuGJo6Vgb-e02EhC3NL2U7oiNRNF+b2s2M2ig@mail.gmail.com>
 <1831776353.1951415.1433246463232.JavaMail.root@sequans.com><CANFcO8sQ0BjHU+7i6XEpaygZLhhi2R0zh0enNk4zk=L8QT3=kg@mail.gmail.com>
Message-ID: <201506021429.t52ETocp030676@fido.openend.se>

You may be looking for dictionary dispatching.

You can translate the key into a callable.

def do_ping(self, arg):
    return 'Ping, {0}!'.format(arg)

def do_pong(self, arg):
    return 'Pong, {0}!'.format(arg)

dispatch = {
    'ping': do_ping,
    'pong': do_pong
	}

and then at some point do

    dispatch[command](arg)

It is useful to know that tuples make perfectly good dictionary keys,
in case you need to store more state somewhere.

Laura


From steve at pearwood.info  Tue Jun  2 11:08:26 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 03 Jun 2015 01:08:26 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1d07e9d4-8314-4b5a-8f98-4174b89370ee@googlegroups.com>
Message-ID: <556dc6ea$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Tue, 2 Jun 2015 09:40 pm, Rustom Mody wrote:

> On Tuesday, June 2, 2015 at 4:57:31 PM UTC+5:30, Steven D'Aprano wrote:
>> On Tue, 2 Jun 2015 08:36 pm, Eddilbert Macharia wrote:
[...]
>> > *** The instance of class type is a data type an instance of class
>> > type. right ?
>> >>>> type(type)
>> > <class 'type'>
>> 
>> type has many instances, not just one.
> 
> Not sure what you are trying to emphasize by the 'not just one'

The OP says "THE [emphasis added] instance of class type" -- type has more
than one instance.


> In particular the issubclass relation is transitive
> Whereas the isinstance is not

Why is that relevant?


-- 
Steven



From ian.g.kelly at gmail.com  Tue Jun  2 11:09:49 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 2 Jun 2015 09:09:49 -0600
Subject: Everything is an object in python - object class and type class
In-Reply-To: <3t5bx.755107$dj1.564752@fx03.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <7f287dcf-8d52-4ab8-9768-88d1f75da269@googlegroups.com>
 <3t5bx.755107$dj1.564752@fx03.am4>
Message-ID: <CALwzidnYwwFkFrxOCPZm9d357RZjRommAv0LfCnqdv2agpkwKQ@mail.gmail.com>

On Mon, Jun 1, 2015 at 4:59 PM, BartC <bc at freeuk.com> wrote:
> I'm developing a new language along the lines of Python, perhaps a brief
> description of how things are done there might help. Or just give a
> different perspective.
>
> Objects in this language are tagged: there's a code attached to each
> indicating what kind of data is stored. These codes are integers, or
> enumerations, for example:
>
>  Int =    1
>  String = 2
>  List =   3
>  Class =  4
>
> And the following are examples of object instances with their tags:
>
> a is (Int,    42)
> b is (String, "ABCXYZ")
> c is (List,   [10,20,30,40])
> d is (Class,  2 or <String>)
>
> The last one might be tricky to grasp: it's really just a number, but one
> that represents a class or type (or tag). If printed, it could display
> <String> rather than just 2. (And when used to do something with the class,
> the 2 might be an index into a set of tables.)
>
> d is not /the/ class itself, but just a reference to it (this is pseudo-code
> not Python):
>
> print (b)                 =>  ABCXYZ
> print (typeof(b))         =>  String <2>
>
> print (d)                 =>  String <2>
> print (typeof(d))         =>  Class  <4>
> print (typeof(typeof(d))) =>  Class  <4>
>
> In my own language, the connection between class and type is hazy (I've only
> just added classes so it needs to be hazy until I understand them more).
>
> 'Type' includes everything, including all the built-in types such as the
> tags above, but also user-defined classes. In fact classes are the mechanism
> used to define new types. But with both designated by an integer within the
> same band (eg. built-int types 1 to 20, user-defined classes 21 and up), it
> is easier not to have a strong distinction ... at the moment.
>
> I haven't a root class yet that is the base of all the others. I don't think
> it's necessary internally to make things work. But it might be a useful
> concept in the language. Calling it 'object' however might give rise to
> confusion as 'object' is informally used to refer to instances. (I've never
> used OO but have picked up some of the jargon!)
>
> (This is almost certainly not how Python does things. Although the Python
> language doesn't go into details as to its implementation provided the
> behaviour is correct.)

It sounds quite similar to me. In CPython at least, every object has a
direct pointer to its type rather than an array index (which is
essentially just an offset from a pointer).


From steve at pearwood.info  Tue Jun  2 11:11:27 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 03 Jun 2015 01:11:27 +1000
Subject: How to access the low digits of a list
References: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>
Message-ID: <556dc7a0$0$13009$c3e8da3$5496439d@news.astraweb.com>

On Tue, 2 Jun 2015 10:23 pm, fl wrote:

> I don't know whether there is a way to know the length of lines[3].

The same way as to know the length of anything else:

len(x)  # the length of x
len("hello")  # the length of "hello"
len(lines[3])  # the length of lines[3]

> Then, 
> I can use a -1 step to get the last two digits. Or, you may have much
> better ways to do that. Python is really too versatile I feel.

Last two digits of a string:

mystring = "123456"
mystring[-2:]


-- 
Steven



From jeanmichel at sequans.com  Tue Jun  2 11:15:54 2015
From: jeanmichel at sequans.com (Jean-Michel Pichavant)
Date: Tue, 2 Jun 2015 17:15:54 +0200 (CEST)
Subject: for...else
In-Reply-To: <CANFcO8t18csV0MMcE9hnYVN2mgrPi2Rco5K6Q+wuniVpfreYJw@mail.gmail.com>
Message-ID: <728516945.1982926.1433258154309.JavaMail.root@sequans.com>

----- Original Message -----
> From: "acdr" <mail.acdr at gmail.com>
> To: "Jean-Michel Pichavant" <jeanmichel at sequans.com>
> Cc: python-list at python.org
> Sent: Tuesday, 2 June, 2015 4:00:12 PM
> Subject: Re: for...else
> 
> The first solution in your e-mail (with a Cleanup exception) is
> definitely very close, functionally, to what I want to accomplish. In
> effect, it's the same structure as my original suggestion of
> "for...then...else", except now it'd be "try: for...else...except".
> That's workable. I can even cheat and rename the Cleanup class to
> "Break" for clarity.
> 
> I'm guessing that there is not much support for functionality like
> this to be built in, in a much less verbose manner, like the
> "for...else" functionality? :P

By the way please don't top post in this list.

You're right the first solution is not that helpfull. I quite dislike try blocks. But I dislike "for else" even more. While "if else"  is a correct english construct and means something, "for else " is not (correct me if I'm wrong).

"for else" works perfectly, I just can never remember on which condition the else block is processed.

JM


-- IMPORTANT NOTICE: 

The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

From marko at pacujo.net  Tue Jun  2 11:28:02 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 02 Jun 2015 18:28:02 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
Message-ID: <87lhg26sfx.fsf@elektro.pacujo.net>

Skip Montanaro <skip.montanaro at gmail.com>:

>     os.closerange(fd_low, fd_high)
>     Close all file descriptors from fd_low (inclusive) to fd_high
>     (exclusive), ignoring errors.
>
> Guido's time machine strikes again...

The only problem is that you don't know how high you need to go in
general.


Marko


From swapnilrbhadade83 at gmail.com  Tue Jun  2 11:39:48 2015
From: swapnilrbhadade83 at gmail.com (Swapnil)
Date: Tue, 2 Jun 2015 21:09:48 +0530
Subject: Framework / CMS for Banking
In-Reply-To: <201506021405.t52E5tLa029778@fido.openend.se>
References: <8008a22e-81d6-4350-b569-2ef00de04883@googlegroups.com>
 <201506021405.t52E5tLa029778@fido.openend.se>
Message-ID: <C5DD2B6B-C375-48CF-8991-F1D7DD90C0C3@gmail.com>


Respected: Thanks. Would you mind if , i ask you more detail Q?s on the framework. If not then any resources where i can i ask more Q?s ?

Best
Dr. Swapnil Bhadade
M.B.B.S
M.S
Yindu (India)
swapnilrbhadade83 at gmail.com.
???????????????????????????????????????????????????????????????????..
Please acknowledge receipt of this email.


The information in this e-mail is confidential and may be legally privileged. It is intended solely for the addressee. Access to this e-mail by anyone else is unauthorized. If you have received this communication in error, please address with the subject heading "Received in error," send to the original sender , then delete the e-mail and destroy any copies of it. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.



Please consider the environment before printing this email.
??????????????????????????????????????????????????????????????????????????...




> On 02-Jun-2015, at 19:35, Laura Creighton <lac at openend.se> wrote:
> 
> In a message of Tue, 02 Jun 2015 05:41:31 -0700, Swapnil Bhadade writes:
>> I am want to know which are the best CMS / framework for building web banking / Financial / lending services ????
>> Best
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
> 
> See this video.
> https://ep2013.europython.eu/conference/talks/python-in-banks
> 
> Laura



From skip.montanaro at gmail.com  Tue Jun  2 11:42:18 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 2 Jun 2015 10:42:18 -0500
Subject: fork/exec & close file descriptors
In-Reply-To: <87lhg26sfx.fsf@elektro.pacujo.net>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87lhg26sfx.fsf@elektro.pacujo.net>
Message-ID: <CANc-5UyRBuTzTC2u-FhVp0h0GVBSJ2bykEi_VkXfTxbUc0PzEA@mail.gmail.com>

On Tue, Jun 2, 2015 at 10:28 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
>
> The only problem is that you don't know how high you need to go in
> general.

Sure, but I didn't know anyway, so no matter what upper bound I choose
(or what function I choose/implement), it's just going to be a guess.
os.closerange just codifies the straightforward procedure.

Skip


From alain at universite-de-strasbourg.fr.invalid  Tue Jun  2 11:59:51 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Tue, 02 Jun 2015 17:59:51 +0200
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
Message-ID: <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>

Skip Montanaro <skip.montanaro at gmail.com> writes:

> Reviving (and concluding) a thread I started a couple weeks ago, I asked:
>
>> The basic fork/exec dance is not a problem, but how do I discover
>> all the open file descriptors in the new child process to make sure
>> they get closed? Do I simply start at fd 3 and call os.close() on
>> everything up to some largish fd number?
>
> I wanted this again today (for different reasons than before).
> Googling for "python close all file descriptors" returned the os
> module docs as the first hit, and lo and behold, what do I see
> documented? os.closerange (new in 2.6):
>
>     os.closerange(fd_low, fd_high)
>     Close all file descriptors from fd_low (inclusive) to fd_high
>     (exclusive), ignoring errors.

The close(2) manpage has the following warning on my Linux system:

| Not checking the return value of close() is a common  but  nevertheless
| serious  programming error.  It is quite possible that errors on a pre?
| vious write(2) operation are first reported at the final close().   Not
| checking the return value when closing the file may lead to silent loss
| of data.  This can especially be observed with NFS and with disk quota.
| 

(I haven't followed the thread, but if your problem is to make sure fds
are closed on exec, you may be better off using the... close-on-exec
flag. Or simply do the bookkeeping.)

-- Alain.


From Cecil at decebal.nl  Tue Jun  2 12:13:40 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 02 Jun 2015 18:13:40 +0200
Subject: Find in ipython3
Message-ID: <87y4k2hyvf.fsf@Equus.decebal.nl>

I am thinking about using ipython3 instead of bash. When I want to
find a file I can do the following:
    !find ~ -iname '*python*.pdf'
but is there a python way?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From rustompmody at gmail.com  Tue Jun  2 12:14:33 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 2 Jun 2015 09:14:33 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <556dc6ea$0$13004$c3e8da3$5496439d@news.astraweb.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1d07e9d4-8314-4b5a-8f98-4174b89370ee@googlegroups.com>
 <556dc6ea$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <9d580d9e-eda4-4441-a929-fd0275845561@googlegroups.com>

On Tuesday, June 2, 2015 at 8:38:42 PM UTC+5:30, Steven D'Aprano wrote:
> On Tue, 2 Jun 2015 09:40 pm, Rustom Mody wrote:
> 
> > On Tuesday, June 2, 2015 at 4:57:31 PM UTC+5:30, Steven D'Aprano wrote:
> >> On Tue, 2 Jun 2015 08:36 pm, Eddilbert Macharia wrote:
> [...]
> >> > *** The instance of class type is a data type an instance of class
> >> > type. right ?
> >> >>>> type(type)
> >> > <class 'type'>
> >> 
> >> type has many instances, not just one.
> > 
> > Not sure what you are trying to emphasize by the 'not just one'
> 
> The OP says "THE [emphasis added] instance of class type" -- type has more
> than one instance.
> 
> 

Ok

> > In particular the issubclass relation is transitive
> > Whereas the isinstance is not
> 
> Why is that relevant?

One way of construing the emphasized plural was what I pointed out as not the case
?x ? isinstance(x,t)
-- true or t = object but not t = type


From lac at openend.se  Tue Jun  2 12:20:22 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 02 Jun 2015 18:20:22 +0200
Subject: Framework / CMS for Banking
In-Reply-To: Message from Swapnil <swapnilrbhadade83@gmail.com> of "Tue,
 02 Jun 2015 21:09:48 +0530." <C5DD2B6B-C375-48CF-8991-F1D7DD90C0C3@gmail.com>
References: <8008a22e-81d6-4350-b569-2ef00de04883@googlegroups.com>
 <201506021405.t52E5tLa029778@fido.openend.se><C5DD2B6B-C375-48CF-8991-F1D7DD90C0C3@gmail.com>
Message-ID: <201506021620.t52GKMvf001204@fido.openend.se>

In a message of Tue, 02 Jun 2015 21:09:48 +0530, Swapnil writes:
>
>
>Respected: Thanks. Would you mind if , i ask you more detail Q?s on the framework. If not then any resources where i can i ask more Q?s ?
>
>Best
>Dr. Swapnil Bhadade

You can ask more questions here.  I just remembered that there was
a Europython talk on the very question you asked.  You might try
emailing the talk's author directly.

Laura


From tjreedy at udel.edu  Tue Jun  2 12:23:19 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 02 Jun 2015 12:23:19 -0400
Subject: Everything is an object in python - object class and type class
In-Reply-To: <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
Message-ID: <mkklam$e47$1@ger.gmane.org>

On 6/2/2015 6:36 AM, Eddilbert Macharia wrote:

> you guys are just confusing me, you are going in loops,

Ignore the troll who is trying to confuse you by slandering the rest of 
us.  I have been using python for 18 years, I believe. Current Python 
has a loop at the core

 >>> isinstance(type, object)
True
 >>> isinstance(object, type)
True

For using python, you hardly need to know this. (It obviously was not 
true before 'object' was added in 2.2.)  The interpreter creates this 
loop on startup with code similar to the following, except that it does 
so with classes rather than lists.

 >>> lob = []
 >>> l0 = []
 >>> l1 = [l0]
 >>> l0.append(l1)
 >>> l0 in l1
True
 >>> l1 in l0
True

 > and still i have understood ,what makes everything in python an object.

Guido's decision to make python be that way.

-- 
Terry Jan Reedy



From marko at pacujo.net  Tue Jun  2 12:24:58 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 02 Jun 2015 19:24:58 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87lhg26sfx.fsf@elektro.pacujo.net>
 <mailman.67.1433259746.13271.python-list@python.org>
Message-ID: <87eglu6pt1.fsf@elektro.pacujo.net>

Skip Montanaro <skip.montanaro at gmail.com>:

> On Tue, Jun 2, 2015 at 10:28 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
>>
>> The only problem is that you don't know how high you need to go in
>> general.
>
> Sure, but I didn't know anyway, so no matter what upper bound I choose
> (or what function I choose/implement), it's just going to be a guess.
> os.closerange just codifies the straightforward procedure.

Under linux, the cleanest way seems to be going through the files under
/proc/self/fd:

    def close_fds(leave_open=[0, 1, 2]):
        fds = os.listdir(b'/proc/self/fd')
        for fdn in fds:
            fd = int(fdn)
            if fd not in leave_open:
                os.close(fd)

No need for a upper bound.


Marko


From jon+usenet at unequivocal.co.uk  Tue Jun  2 12:37:15 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 2 Jun 2015 16:37:15 +0000 (UTC)
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87lhg26sfx.fsf@elektro.pacujo.net>
 <mailman.67.1433259746.13271.python-list@python.org>
 <87eglu6pt1.fsf@elektro.pacujo.net>
Message-ID: <slrnmmrn05.20b.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-02, Marko Rauhamaa <marko at pacujo.net> wrote:
> Skip Montanaro <skip.montanaro at gmail.com>:
>
>> On Tue, Jun 2, 2015 at 10:28 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
>>>
>>> The only problem is that you don't know how high you need to go in
>>> general.
>>
>> Sure, but I didn't know anyway, so no matter what upper bound I choose
>> (or what function I choose/implement), it's just going to be a guess.
>> os.closerange just codifies the straightforward procedure.
>
> Under linux, the cleanest way seems to be going through the files under
> /proc/self/fd:
>
>     def close_fds(leave_open=[0, 1, 2]):
>         fds = os.listdir(b'/proc/self/fd')
>         for fdn in fds:
>             fd = int(fdn)
>             if fd not in leave_open:
>                 os.close(fd)
>
> No need for a upper bound.

Or use the more generic code that I already posted in this thread:

  def closeall(min=0, max=4096, keep=()):
      """Close all open file descriptors except for the given exceptions.

      Any file descriptors below or equal to `min`, or in the set `keep`
      will not be closed. Any file descriptors above `max` *might* not be
      closed.
      """
      # First try /proc/self/pid
      try:
          for fd in os.listdir("/proc/self/fd"):
              try:
                  fd = int(fd)
              except ValueError:
                  continue
              if fd >= min and fd not in keep:
                  os.close(int(fd))
          return
      except OSError as exc:
          if exc[0] != errno.ENOENT:
              raise
      # If /proc was not available, fall back to closing a lot of descriptors.
      for fd in range(min, max):
          if fd not in keep:
              try:
                  os.close(fd)
              except OSError as exc:
                  if exc[0] != errno.EBADF:
                      raise

This function could use os.closerange(), but if the documentation is
correct and it ignores *all* errors and not just EBADF, then it
sounds like os.closerange() should not in fact ever be used for any
purpose.


From python.list at tim.thechases.com  Tue Jun  2 12:39:30 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Tue, 2 Jun 2015 11:39:30 -0500
Subject: What use of string module?
In-Reply-To: <mkj8ep$3v7$1@ger.gmane.org>
References: <d3bf3182-d70e-4160-b600-733cbda748e6@googlegroups.com>
 <mkj8ep$3v7$1@ger.gmane.org>
Message-ID: <20150602113930.4bf9d32e@bigbox.christie.dr>

On 2015-06-02 04:37, Mark Lawrence wrote:
> > I read the online help about string. It lists string constants,
> > string formatting, template strings and string functions. After
> > reading these, I am still puzzled about how to use the string
> > module.
> 
> I suggest you don't bother, it's effectively dead having been
> replaced by string methods.

While most of the functions in the string module are obsoleted by the
methods on strings, I still find the constants useful to save myself
from my own inability to type, possibly omitting letters from the
alphabet or doubling them up.  I'd much rather just reference
string.printable than try to recreate it (or its kin) myself.

-tkc





From python at mrabarnett.plus.com  Tue Jun  2 12:43:05 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 02 Jun 2015 17:43:05 +0100
Subject: Parser needed.
In-Reply-To: <a1ba9$556d34d1$5419aafe$10658@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <9c6eb$556d2fa6$5419aafe$8770@news.ziggo.nl>
 <81e68$556d3120$5419aafe$9437@news.ziggo.nl>
 <b352f$556d338a$5419aafe$10196@news.ziggo.nl>
 <a1ba9$556d34d1$5419aafe$10658@news.ziggo.nl>
Message-ID: <556DDD19.20202@mrabarnett.plus.com>

On 2015-06-02 05:45, Skybuck Flying wrote:
> Example for python:
>
> MyString = "Hello World"
>
> print MyString.rfind("World")
> if MyString.rfind("World"):
> print "yes"
> else:
> print "no"
>
> Pretty cool.
>
.rfind returns the index if found, -1 if not found.

"World".rfind("World") returns 0, which will be treated as false.

"foo".rfind("World") returns -1, which will be treated as true.



From steve at pearwood.info  Tue Jun  2 12:50:56 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 03 Jun 2015 02:50:56 +1000
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net> <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
 <87siagagpx.fsf@elektro.pacujo.net>
 <5567c840$0$12998$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <556ddef1$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Fri, 29 May 2015 12:00 pm, Steven D'Aprano wrote:

[...]
> in a language where classes are 
> themselves values, there is no reason why a class must be instantiated,
> particularly if you're only using a single instance of the class. Anyone
> ever come across a named design pattern that involves using classes
> directly without instantiating them?
> 
> I'm basically looking for a less inelegant term for "instanceless class"
> -- not so much a singleton as a zeroton.

C# has these, and calls them static classes.

https://msdn.microsoft.com/en-us/library/79b3xss3%28v=vs.100%29.aspx




-- 
Steven



From steve at pearwood.info  Tue Jun  2 13:00:50 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 03 Jun 2015 03:00:50 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
Message-ID: <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>

On Tue, 2 Jun 2015 10:49 pm, BartC wrote:

> On 02/06/2015 12:27, Steven D'Aprano wrote:
> 
>> "Object" has a general meaning in computer science and programming, it is
>> a compound data structure that is explicitly linked to a type which
>> provides functionality that operates on that data structure.
>>
>> In the programming language C, *no* values are objects. C has types
>> (int16, uint32, bool, and many more) but no objects.
> 
> The C Standard has hundreds of references to 'object'.

Are any of them objects in the sense of object oriented programming?

I'm pretty sure the answer to that is No. C is widely agreed to *not* be an
object oriented language, although obviously you can construct an OOP
system in C.

It is difficult to give a single, unconditional definition of "object" in
the object-oriented programming sense, because there are many subtle
variations. We can consume many hours in fruitless debates over an exact
formal definition of "object" in the sense of OOP, but I don't think that
is productive. But I think this should cover most of the common cases:


* An object-oriented programming language is one which provides objects 
  as a native data type;

* objects are structured entities which combine:

  - behaviour (methods)
  - state (data, value)
  - and identity (unique existence);

* the structure and behaviour of the object is inherited from a 
  class (blueprint or template), or a prototype (another object).


The important thing is that the language provides these as ready-to-use
features, complete with syntax and support from the compiler/interpreter. I
could write OOP-style code in C, or even assembly language, but only by
writing my own framework, then taking care to only write code within the
boundaries of that framework. Hence, C is not OOP: objects aren't built-in
to C, but have to be added.


> C objects are also linked to a type but it doesn't need to be explicit
> /in the run-time data/ because it's a static language. The compiler
> knows all the types because there are explicit annotations in the source
> code.

OOP can exist in both statically typed languages (such as Java, C++,
Objective C) and dynamically typed languages (such as Python, Javascript,
Ruby).


>> In the programming language Java, *some* values are objects, and some
>> values are not objects.
>>
>> In the programming language Python, *all* values are objects, in the
>> general sense. That is what we mean by "Everything is an object".
> 
> In a dynamically typed language such as Python, you need to be able to
> deal with values consistently whatever their type, simply because you
> can't tell what the types are from source code. (Not without a lot of
> work which Python doesn't attempt, although RPython might do so.) Example:
> 
>   print (a)
> 
> a might be the int 42, or a it might be a million-element list. So both
> are wrapped up as 'objects'.

Again, this is not relevant. Javascript is dynamically typed, but some
values are machine primitives and other values are objects. The interpreter
keeps track of this at runtime.

Java is similar, except it keeps track if the difference between boxed and
unboxed types at compile time.


> Java is statically typed which makes it possible to treat instance
> variables differently without needing to examine their types at runtime.
> If you define 'object' in a certain way (eg. as boxed, tagged data),
> then it follows that some values don't need to be objects.

I'm not sure what point you are trying to make here.



-- 
Steven



From skybuck2000 at hotmail.com  Tue Jun  2 13:09:26 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 2 Jun 2015 19:09:26 +0200
Subject: Parser needed.
In-Reply-To: <mailman.71.1433263397.13271.python-list@python.org>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <9c6eb$556d2fa6$5419aafe$8770@news.ziggo.nl>
 <81e68$556d3120$5419aafe$9437@news.ziggo.nl>
 <b352f$556d338a$5419aafe$10196@news.ziggo.nl>
 <a1ba9$556d34d1$5419aafe$10658@news.ziggo.nl>
 <mailman.71.1433263397.13271.python-list@python.org>
Message-ID: <bbe54$556de345$5419aafe$40024@news.ziggo.nl>



"MRAB"  wrote in message 
news:mailman.71.1433263397.13271.python-list at python.org...

On 2015-06-02 05:45, Skybuck Flying wrote:
> Example for python:
>
> MyString = "Hello World"
>
> print MyString.rfind("World")
> if MyString.rfind("World"):
> print "yes"
> else:
> print "no"
>
> Pretty cool.
>

"
.rfind returns the index if found, -1 if not found.

"World".rfind("World") returns 0, which will be treated as false.

"foo".rfind("World") returns -1, which will be treated as true.
"

Yes I agree, that code is not very clean.

Hence I added:

print MyString.rfind("World")

Just to figure out if it indeed prints an index.

Index 0 might be of use too, so checking for true/false like I did above 
should not be done.

Consider it incorrect code.

However I didn't care at the time cause I was in a hurry... I knew it was 
dirty but this post correct that and warns people not to write code like I 
did... just in case ! ;)

And also to let you know I am full aware of the shady nature of this code ! 
;) =D

Bye,
  Skybuck :)









From dreamingforward at gmail.com  Tue Jun  2 13:16:45 2015
From: dreamingforward at gmail.com (Dr. BigCock)
Date: Tue, 2 Jun 2015 10:16:45 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <556ddef1$0$13004$c3e8da3$5496439d@news.astraweb.com>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net> <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
 <87siagagpx.fsf@elektro.pacujo.net>
 <5567c840$0$12998$c3e8da3$5496439d@news.astraweb.com>
 <556ddef1$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <118e319a-792e-4ca8-bcf8-7b9e60b842a2@googlegroups.com>

On Tuesday, June 2, 2015 at 11:51:10 AM UTC-5, Steven D'Aprano wrote:
> On Fri, 29 May 2015 12:00 pm, Steven D'Aprano wrote:
> 
> [...]
> > in a language where classes are 
> > themselves values, there is no reason why a class must be instantiated,
> > particularly if you're only using a single instance of the class. Anyone
> > ever come across a named design pattern that involves using classes
> > directly without instantiating them?
> > 
> > I'm basically looking for a less inelegant term for "instanceless class"
> > -- not so much a singleton as a zeroton.

The only reason to have such uninstanced classed is for meta-class programming -- where you're designing classes *programmatically*, not by your interaction and typing in your source code as you normally would.

Otherwise, I'm suspicious that "there's no reason why a class must be instantiated".  It's like declaring a complex variable and never using it.

--m


From marko at pacujo.net  Tue Jun  2 13:19:53 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 02 Jun 2015 20:19:53 +0300
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net>
 <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
 <87siagagpx.fsf@elektro.pacujo.net>
 <5567c840$0$12998$c3e8da3$5496439d@news.astraweb.com>
 <556ddef1$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87mw0i6n9i.fsf@elektro.pacujo.net>

Steven D'Aprano <steve at pearwood.info>:

> On Fri, 29 May 2015 12:00 pm, Steven D'Aprano wrote:
>
> [...]
>> in a language where classes are 
>> themselves values, there is no reason why a class must be instantiated,
>> particularly if you're only using a single instance of the class. Anyone
>> ever come across a named design pattern that involves using classes
>> directly without instantiating them?
>> 
>> I'm basically looking for a less inelegant term for "instanceless class"
>> -- not so much a singleton as a zeroton.
>
> C# has these, and calls them static classes.

I guess Python has them, too, and calls them modules.


Marko


From nomail at invalid.com  Tue Jun  2 13:46:54 2015
From: nomail at invalid.com (ast)
Date: Tue, 2 Jun 2015 19:46:54 +0200
Subject: for...else
In-Reply-To: <mailman.64.1433255400.13271.python-list@python.org>
References: <CANFcO8tOLorV1kuGJo6Vgb-e02EhC3NL2U7oiNRNF+b2s2M2ig@mail.gmail.com>
 <1831776353.1951415.1433246463232.JavaMail.root@sequans.com><CANFcO8sQ0BjHU+7i6XEpaygZLhhi2R0zh0enNk4zk=L8QT3=kg@mail.gmail.com>
 <mailman.64.1433255400.13271.python-list@python.org>
Message-ID: <556dec13$0$3179$426a34cc@news.free.fr>


"Laura Creighton" <lac at openend.se> a ?crit dans le message de 
news:mailman.64.1433255400.13271.python-list at python.org...
> You may be looking for dictionary dispatching.
>
> You can translate the key into a callable.
>
> def do_ping(self, arg):
>    return 'Ping, {0}!'.format(arg)
>
> def do_pong(self, arg):
>    return 'Pong, {0}!'.format(arg)
>
> dispatch = {
>    'ping': do_ping,
>    'pong': do_pong
> }
>
> and then at some point do
>
>    dispatch[command](arg)
>
> It is useful to know that tuples make perfectly good dictionary keys,
> in case you need to store more state somewhere.
>
> Laura

hello

Why "self" is as parameter of functions do_ping and do_pong ?

thx




From bc at freeuk.com  Tue Jun  2 13:59:20 2015
From: bc at freeuk.com (BartC)
Date: Tue, 02 Jun 2015 18:59:20 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <%9mbx.608935$JH2.445461@fx11.am4>

On 02/06/2015 18:00, Steven D'Aprano wrote:
> On Tue, 2 Jun 2015 10:49 pm, BartC wrote:
>
>> On 02/06/2015 12:27, Steven D'Aprano wrote:

>>> In the programming language Java, *some* values are objects, and some
>>> values are not objects.
>>>
>>> In the programming language Python, *all* values are objects, in the
>>> general sense. That is what we mean by "Everything is an object".
>>
>> In a dynamically typed language such as Python, you need to be able to
>> deal with values consistently whatever their type, simply because you
>> can't tell what the types are from source code. (Not without a lot of
>> work which Python doesn't attempt, although RPython might do so.) Example:
>>
>>    print (a)
>>
>> a might be the int 42, or a it might be a million-element list. So both
>> are wrapped up as 'objects'.
>
> Again, this is not relevant. Javascript is dynamically typed, but some
> values are machine primitives and other values are objects. The interpreter
> keeps track of this at runtime.

Javascript primitives include Number and String.

What does Python allow to be done with its Number (int, etc) and String 
types that can't be done with their Javascript counterparts, that makes 
/them/ objects?

-- 
Bartc


From random832 at fastmail.us  Tue Jun  2 14:07:01 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 02 Jun 2015 14:07:01 -0400
Subject: Everything is an object in python - object class and type class
In-Reply-To: <%9mbx.608935$JH2.445461@fx11.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
Message-ID: <1433268421.1213011.284969081.46EF11EE@webmail.messagingengine.com>



On Tue, Jun 2, 2015, at 13:59, BartC wrote:
> On 02/06/2015 18:00, Steven D'Aprano wrote:
> > Again, this is not relevant. Javascript is dynamically typed, but some
> > values are machine primitives and other values are objects. The interpreter
> > keeps track of this at runtime.
> 
> Javascript primitives include Number and String.
> 
> What does Python allow to be done with its Number (int, etc) and String 
> types that can't be done with their Javascript counterparts, that makes 
> /them/ objects?

That's not really a fair question, because the Javascript object model
is so different from the Python one. The point is that they are handled
_the same_ as all other objects, not that they have some specific
capability.


From jon+usenet at unequivocal.co.uk  Tue Jun  2 14:10:38 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 2 Jun 2015 18:10:38 +0000 (UTC)
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
Message-ID: <slrnmmrsf7.20b.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-02, BartC <bc at freeuk.com> wrote:
> On 02/06/2015 18:00, Steven D'Aprano wrote:
>> Again, this is not relevant. Javascript is dynamically typed, but some
>> values are machine primitives and other values are objects. The interpreter
>> keeps track of this at runtime.
>
> Javascript primitives include Number and String.
>
> What does Python allow to be done with its Number (int, etc) and String 
> types that can't be done with their Javascript counterparts, that makes 
> /them/ objects?

The fact that they are, er, objects? They are instances of classes,
they have methods that you can call, you can create subclasses, etc?


From ned at nedbatchelder.com  Tue Jun  2 14:10:43 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 2 Jun 2015 11:10:43 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <%9mbx.608935$JH2.445461@fx11.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
Message-ID: <4d0ad8f5-d29f-442e-9374-5cda12153f88@googlegroups.com>

On Tuesday, June 2, 2015 at 1:59:37 PM UTC-4, BartC wrote:
> On 02/06/2015 18:00, Steven D'Aprano wrote:
> > On Tue, 2 Jun 2015 10:49 pm, BartC wrote:
> >
> >> On 02/06/2015 12:27, Steven D'Aprano wrote:
> 
> >>> In the programming language Java, *some* values are objects, and some
> >>> values are not objects.
> >>>
> >>> In the programming language Python, *all* values are objects, in the
> >>> general sense. That is what we mean by "Everything is an object".
> >>
> >> In a dynamically typed language such as Python, you need to be able to
> >> deal with values consistently whatever their type, simply because you
> >> can't tell what the types are from source code. (Not without a lot of
> >> work which Python doesn't attempt, although RPython might do so.) Example:
> >>
> >>    print (a)
> >>
> >> a might be the int 42, or a it might be a million-element list. So both
> >> are wrapped up as 'objects'.
> >
> > Again, this is not relevant. Javascript is dynamically typed, but some
> > values are machine primitives and other values are objects. The interpreter
> > keeps track of this at runtime.
> 
> Javascript primitives include Number and String.
> 
> What does Python allow to be done with its Number (int, etc) and String 
> types that can't be done with their Javascript counterparts, that makes 
> /them/ objects?

They have methods (not many, but a few):

>>> i, f = 1000001, 2.5
>>> i.bit_length()
20
>>> i.to_bytes(6, "big")
b'\x00\x00\x00\x0fBA'
>>> f.as_integer_ratio()
(5, 2)
>>> f.hex()
'0x1.4000000000000p+1'

--Ned.

> 
> -- 
> Bartc



From rxjwg98 at gmail.com  Tue Jun  2 14:14:10 2015
From: rxjwg98 at gmail.com (fl)
Date: Tue, 2 Jun 2015 11:14:10 -0700 (PDT)
Subject: Could you explain lambda function to me?
Message-ID: <323866d1-b117-4785-ae24-7d04c49bc063@googlegroups.com>

Hi,

I see the description of lambda at the online tutorial, but I cannot 
understand it. '42' is transferred to the function. What 'x' value should
be? I do not see it says that it is '0'. And, what is 'x'?








>>> def make_incrementor(n):
...     return lambda x: x + n
...
>>> f = make_incrementor(42)
>>> f(0)
42
>>> f(1)
43


The second lambda example is even more obscure to me:

>>> pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')]
>>> pairs.sort(key=lambda pair: pair[1])
>>> pairs
[(4, 'four'), (1, 'one'), (3, 'three'), (2, 'two')]


Could you explain 'key=lambda pair: pair[1]' to me?

Python grammar seems too succinct to me.


Thanks,


From lac at openend.se  Tue Jun  2 14:16:07 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 02 Jun 2015 20:16:07 +0200
Subject: for...else
In-Reply-To: Message from "ast" <nomail@invalid.com>
 of "Tue, 02 Jun 2015 19:46:54 +0200." <556dec13$0$3179$426a34cc@news.free.fr>
References: <CANFcO8tOLorV1kuGJo6Vgb-e02EhC3NL2U7oiNRNF+b2s2M2ig@mail.gmail.com>
 <1831776353.1951415.1433246463232.JavaMail.root@sequans.com><CANFcO8sQ0BjHU+7i6XEpaygZLhhi2R0zh0enNk4zk=L8QT3=kg@mail.gmail.com>
 <mailman.64.1433255400.13271.python-list@python.org><556dec13$0$3179$426a34cc@news.free.fr>
Message-ID: <201506021816.t52IG7LR004142@fido.openend.se>

In a message of Tue, 02 Jun 2015 19:46:54 +0200, "ast" writes:
>
>
>"Laura Creighton" <lac at openend.se> a ?crit dans le message de 
>news:mailman.64.1433255400.13271.python-list at python.org...
>> You may be looking for dictionary dispatching.
>>
>> You can translate the key into a callable.
>>
>> def do_ping(self, arg):
>>    return 'Ping, {0}!'.format(arg)
>>
>> def do_pong(self, arg):
>>    return 'Pong, {0}!'.format(arg)
>>
>> dispatch = {
>>    'ping': do_ping,
>>    'pong': do_pong
>> }
>>
>> and then at some point do
>>
>>    dispatch[command](arg)
>>
>> It is useful to know that tuples make perfectly good dictionary keys,
>> in case you need to store more state somewhere.
>>
>> Laura
>
>hello
>
>Why "self" is as parameter of functions do_ping and do_pong ?
>
>thx

yanked out of code I had lying around, without proper thought.
Sorry about that.

Laura



From ckaynor at zindagigames.com  Tue Jun  2 14:33:53 2015
From: ckaynor at zindagigames.com (Chris Kaynor)
Date: Tue, 2 Jun 2015 11:33:53 -0700
Subject: Could you explain lambda function to me?
In-Reply-To: <323866d1-b117-4785-ae24-7d04c49bc063@googlegroups.com>
References: <323866d1-b117-4785-ae24-7d04c49bc063@googlegroups.com>
Message-ID: <CALvWhxuSnCpF+-vqvhMGVPDjxv-p6toinT_nhZ_pZYcaQP2LLg@mail.gmail.com>

On Tue, Jun 2, 2015 at 11:14 AM, fl <rxjwg98 at gmail.com> wrote:

> Hi,
>
> I see the description of lambda at the online tutorial, but I cannot
> understand it. '42' is transferred to the function. What 'x' value should
> be? I do not see it says that it is '0'. And, what is 'x'?
>

The lambda keyword is merely another way to define a function.

As a general (and somewhat simplified rule),
    func = lambda <<1>>: <<2>>
is the same as
    def func(<<1>>):
        return <<2>>

The main difference is that lambda is an expression, while def is a
statement, which means you can use lambda is many places you cannot use
def, without adding extra lines of code.

>>> def make_incrementor(n):
> ...     return lambda x: x + n
> ...
> >>> f = make_incrementor(42)
> >>> f(0)
> 42
> >>> f(1)
> 43
>

Following the above rule, you can convert make_incrementor to look like:

def make_incrementor(n):
    def func(x):
        return x+n
    return func

That function, when called, will return a new function that takes one
argument, and adds it to the argument to make_incrementor.

n will be assigned what ever is passed into make_incrementor (42 in the
example code), and x will be what ever is passed into the returned function
(0 and 1 in the example).


> The second lambda example is even more obscure to me:
>
> >>> pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')]
> >>> pairs.sort(key=lambda pair: pair[1])
> >>> pairs
> [(4, 'four'), (1, 'one'), (3, 'three'), (2, 'two')]
>
>
> Could you explain 'key=lambda pair: pair[1]' to me?
>

In this case, you are passing a new "key" function into list.sort. Per the
documentation [1], the key function will be given each item in the input
list and should return the key to sort that item by. The lambda is defining
a function which takes one of those items (named pair inside the function),
and returns the second item (index 1) of it. list.sort then sorts the items
in the original list based on this key, whereby you get "four", "one",
"three", "two" sorted as strings, but the output will include the full
pairs.

This example could be rewritten as (again, following the general rule
mentioned at the top)"
>>> def keyFunc(pair):
>>>     return pair[1]
>>> pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')]
>>> pairs.sort(key=keyFunc)
>>> pairs
[(4, 'four'), (1, 'one'), (3, 'three'), (2, 'two')]

[1] https://docs.python.org/2/howto/sorting.html#key-functions


> Python grammar seems too succinct to me.
>
>
> Thanks,
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150602/10ba7765/attachment.html>

From jon+usenet at unequivocal.co.uk  Tue Jun  2 14:47:40 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 2 Jun 2015 18:47:40 +0000 (UTC)
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
Message-ID: <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-02, Dr. Bigcock <dreamingforward at gmail.com> wrote:
> It doesn't really do anything.  No one uses integers as objects.
> (Any dissenters?)

Yes. *Everyone* uses integers as objects. Containers such as
lists and dictionaries and tuples etc contain objects. If
integers weren't objects then you wouldn't be able to put them
in containers (and you'd end up with Java).


From nomail at invalid.com  Tue Jun  2 15:08:31 2015
From: nomail at invalid.com (ast)
Date: Tue, 2 Jun 2015 21:08:31 +0200
Subject: Could you explain lambda function to me?
In-Reply-To: <323866d1-b117-4785-ae24-7d04c49bc063@googlegroups.com>
References: <323866d1-b117-4785-ae24-7d04c49bc063@googlegroups.com>
Message-ID: <556dff36$0$3332$426a34cc@news.free.fr>


"fl" <rxjwg98 at gmail.com> a ?crit dans le message de 
news:323866d1-b117-4785-ae24-7d04c49bc063 at googlegroups.com...
> Hi,
>
>>>> def make_incrementor(n):
> ...     return lambda x: x + n
> ...
>>>> f = make_incrementor(42)
>>>> f(0)
> 42
>>>> f(1)
> 43

make_incrementor is a fonction which return a function !
and the returned function just add n to its input

it is equivalent to:

def make_incrementor(n):

    def inc(x):
        return x+n

    return inc

>
>
> The second lambda example is even more obscure to me:
>
>>>> pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')]
>>>> pairs.sort(key=lambda pair: pair[1])
>>>> pairs
> [(4, 'four'), (1, 'one'), (3, 'three'), (2, 'two')]
>
>
> Could you explain 'key=lambda pair: pair[1]' to me?

the sort is done on the values returned by the key function
key function "lambda pair: pair[1]" returns the second element of each pair ( , )

so the sort is done alphabetically on the strings 'one' 'two' 'three' 'four'




From jon+usenet at unequivocal.co.uk  Tue Jun  2 15:31:23 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 2 Jun 2015 19:31:23 +0000 (UTC)
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
 <914115ea-0f93-4929-b325-5b3f78f5d9f1@googlegroups.com>
Message-ID: <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-02, Dr. Bigcock <dreamingforward at gmail.com> wrote:
> On Tuesday, June 2, 2015 at 1:49:03 PM UTC-5, Jon Ribbens wrote:
>> On 2015-06-02, Dr. Bigcock <dreamingforward at gmail.com> wrote:
>> > It doesn't really do anything.  No one uses integers as objects.
>> > (Any dissenters?)
>> 
>> Yes. *Everyone* uses integers as objects. Containers such as
>> lists and dictionaries and tuples etc contain objects. If
>> integers weren't objects then you wouldn't be able to put them
>> in containers (and you'd end up with Java).
>
> Sorry.  I meant "object" in the sense of OOP:  something you might
> extend or make a derived class with.

I'm not sure you get to define which properties of objects you want
not to count.

> Your last claim, must not be true because integers were able to be
> placed in objects before the type/class unification with v2.6, I
> believe.

Unless I'm misremembering, before that they were still objects,
just not quite the same kind of objects as pure-Python ones.


From ian.g.kelly at gmail.com  Tue Jun  2 15:36:16 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 2 Jun 2015 13:36:16 -0600
Subject: Everything is an object in python - object class and type class
In-Reply-To: <4d0ad8f5-d29f-442e-9374-5cda12153f88@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <4d0ad8f5-d29f-442e-9374-5cda12153f88@googlegroups.com>
Message-ID: <CALwzidk7ffXVZkTbwsWxpBitMUKO-kms8QMOURsg=p1ATpjApA@mail.gmail.com>

On Tue, Jun 2, 2015 at 12:10 PM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> On Tuesday, June 2, 2015 at 1:59:37 PM UTC-4, BartC wrote:
>> Javascript primitives include Number and String.
>>
>> What does Python allow to be done with its Number (int, etc) and String
>> types that can't be done with their Javascript counterparts, that makes
>> /them/ objects?
>
> They have methods (not many, but a few):
>
>>>> i, f = 1000001, 2.5
>>>> i.bit_length()
> 20
>>>> i.to_bytes(6, "big")
> b'\x00\x00\x00\x0fBA'
>>>> f.as_integer_ratio()
> (5, 2)
>>>> f.hex()
> '0x1.4000000000000p+1'

To add a wrinkle to this discussion, Javascript numbers also have methods:

js> (24).toExponential(3)
"2.400e+1"

I believe this is accomplished by implicitly boxing the number in the
Number class when a method or property is accessed. This can be seen
with:

js> (24).toSource()
"(new Number(24))"

Note that "24" and "new Number(24)" are not equivalent.

js> 24 === 24
true
js> 24 === new Number(24)
false
js> typeof(24)
"number"
js> typeof(new Number(24))
"object"

But this is a bit of an implementation detail. So what distinguishes
Javascript primitives from objects? Steven listed "identity" as a
third property of objects upthread; that seems applicable here.


From ian.g.kelly at gmail.com  Tue Jun  2 15:37:02 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 2 Jun 2015 13:37:02 -0600
Subject: Everything is an object in python - object class and type class
In-Reply-To: <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <CALwzidnHLnnNoDysLre3WNRzs_w5E7+gVKLEEwd3ouStZjdnmg@mail.gmail.com>

On Tue, Jun 2, 2015 at 12:47 PM, Jon Ribbens
<jon+usenet at unequivocal.co.uk> wrote:
> On 2015-06-02, Dr. Bigcock <dreamingforward at gmail.com> wrote:
>> It doesn't really do anything.  No one uses integers as objects.
>> (Any dissenters?)
>
> Yes. *Everyone* uses integers as objects. Containers such as
> lists and dictionaries and tuples etc contain objects. If
> integers weren't objects then you wouldn't be able to put them
> in containers (and you'd end up with Java).

Javascript has no problem putting primitives into containers.

js> var arr = [0, 'test', new Object()];
js> typeof arr[0]
"number"
js> typeof arr[1]
"string"
js> typeof arr[2]
"object"


From ian.g.kelly at gmail.com  Tue Jun  2 15:50:11 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 2 Jun 2015 13:50:11 -0600
Subject: Everything is an object in python - object class and type class
In-Reply-To: <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
 <914115ea-0f93-4929-b325-5b3f78f5d9f1@googlegroups.com>
 <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <CALwzidnYFtTvsmJ8qOSM9jcXM=g1LdqW9WC6Nr=MPcn-RbMBPw@mail.gmail.com>

On Tue, Jun 2, 2015 at 1:31 PM, Jon Ribbens
<jon+usenet at unequivocal.co.uk> wrote:
> On 2015-06-02, Dr. Bigcock <dreamingforward at gmail.com> wrote:
>> On Tuesday, June 2, 2015 at 1:49:03 PM UTC-5, Jon Ribbens wrote:
>>> On 2015-06-02, Dr. Bigcock <dreamingforward at gmail.com> wrote:
>>> > It doesn't really do anything.  No one uses integers as objects.
>>> > (Any dissenters?)
>>>
>>> Yes. *Everyone* uses integers as objects. Containers such as
>>> lists and dictionaries and tuples etc contain objects. If
>>> integers weren't objects then you wouldn't be able to put them
>>> in containers (and you'd end up with Java).
>>
>> Sorry.  I meant "object" in the sense of OOP:  something you might
>> extend or make a derived class with.
>
> I'm not sure you get to define which properties of objects you want
> not to count.

Accepting for the sake of argument that "something to be subclassed"
is a reasonable definition of object, it should be pointed out that
anybody who works with bools in Python is using integers as objects.

The "Super Considered Harmful" essay also has a couple of examples of
subclasses of int, and even just googling "python subclass int"
demonstrates that there is plenty of interest in the subject.


From marko at pacujo.net  Tue Jun  2 16:05:45 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 02 Jun 2015 23:05:45 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <87eglt7u5i.fsf@elektro.pacujo.net>

Alain Ketterlin <alain at universite-de-strasbourg.fr.invalid>:

> The close(2) manpage has the following warning on my Linux system:
>
> | Not checking the return value of close() is a common but
> | nevertheless serious programming error. It is quite possible that
> | errors on a previous write(2) operation are first reported at the
> | final close(). Not checking the return value when closing the file
> | may lead to silent loss of data. This can especially be observed
> | with NFS and with disk quota.
> | 
>
> (I haven't followed the thread, but if your problem is to make sure
> fds are closed on exec, you may be better off using the...
> close-on-exec flag. Or simply do the bookkeeping.)

The quoted man page passage is a bit untenable.

First, if close() fails, what's a poor program to do? Try again? How
do you get rid of an obnoxious file descriptor? How would close-on-exec
help? Would exec*() fail?

What if an implicit close() fails on _exit(), will _exit() fail then?
(The man page doesn't allow it.)

The need to close all open file descriptors comes between fork() and
exec*(). The kernel (module) does not see the close() system call unless
the reference count drops to zero. Normally, those function calls
between fork() and exec*() are therefore no-ops.

However, there's no guarantee of that. So the parent process might get
to call close() before the child that is about to call exec*(). Then,
the parent would not get the error that the man page talks about.
Instead, the error goes to the child, which has no reasonable way of
dealing with the situation.

I think having NFS et al postpone their I/O errors till close() is
shifting the blame to the victim.


Marko


From rxjwg98 at gmail.com  Tue Jun  2 16:20:22 2015
From: rxjwg98 at gmail.com (fl)
Date: Tue, 2 Jun 2015 13:20:22 -0700 (PDT)
Subject: Please help on this sorted function
Message-ID: <37d9de72-b719-45a0-976c-441fc5898741@googlegroups.com>

Hi,

I try to learn sorted(). With the tutorial example:




>>> ff=sorted({1: 'D', 2: 'B', 3: 'B', 4: 'E', 5: 'A'})
>>> ff
[1, 2, 3, 4, 5]



I don't see what sorted does in this dictionary, i.e. the sequence of 
1..5 is unchanged. Could you explain it to me?


Thanks,


From rxjwg98 at gmail.com  Tue Jun  2 16:25:12 2015
From: rxjwg98 at gmail.com (fl)
Date: Tue, 2 Jun 2015 13:25:12 -0700 (PDT)
Subject: Please help on this sorted function
In-Reply-To: <37d9de72-b719-45a0-976c-441fc5898741@googlegroups.com>
References: <37d9de72-b719-45a0-976c-441fc5898741@googlegroups.com>
Message-ID: <70489d38-0848-44c4-9a4c-ba2e2e9aa027@googlegroups.com>

On Tuesday, June 2, 2015 at 1:20:40 PM UTC-7, fl wrote:
> Hi,
> 
> I try to learn sorted(). With the tutorial example:
> 
> 
> 
> 
> >>> ff=sorted({1: 'D', 2: 'B', 3: 'B', 4: 'E', 5: 'A'})
> >>> ff
> [1, 2, 3, 4, 5]
> 
> 
> 
> I don't see what sorted does in this dictionary, i.e. the sequence of 
> 1..5 is unchanged. Could you explain it to me?
> 
> 
> Thanks,

Excuse me. After a small modification, it can see the effect.


>>> ff=sorted({1: 'D', 2: 'B', 5: 'B', 4: 'E', 3: 'A'})
>>> ff
[1, 2, 3, 4, 5]


I am still new to Python. How to get the sorted dictionary output:

{1: 'D', 2: 'B', 3: 'A', 4: 'E', 5: 'B'}


From liik.joonas at gmail.com  Tue Jun  2 16:32:38 2015
From: liik.joonas at gmail.com (Joonas Liik)
Date: Tue, 2 Jun 2015 23:32:38 +0300
Subject: Please help on this sorted function
In-Reply-To: <37d9de72-b719-45a0-976c-441fc5898741@googlegroups.com>
References: <37d9de72-b719-45a0-976c-441fc5898741@googlegroups.com>
Message-ID: <CAB1GNpTHnFK1=TYj3JefcfbFesvrGyJgiTh5AmG+AYuDLhVcyQ@mail.gmail.com>

>>> ff=sorted({1: 'D', 2: 'B', 3: 'B', 4: 'E', 5: 'A'})
>>> ff
[1, 2, 3, 4, 5]

sorted({1: 'D', 2: 'B', 3: 'B', 4: 'E', 5: 'A'}) is equivalent to
sorted(iter({1: 'D', 2: 'B', 3: 'B', 4: 'E', 5: 'A'}))

and iter(dict) iterates over the dict keys, so when you do
iter({1: 'D', 2: 'B', 3: 'B', 4: 'E', 5: 'A'}) you essentially get
[1,2,3,4,5]
and sorted([1,2,3,4,5]) returns [1,2,3,4,5]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150602/5662edde/attachment.html>

From liik.joonas at gmail.com  Tue Jun  2 16:42:31 2015
From: liik.joonas at gmail.com (Joonas Liik)
Date: Tue, 2 Jun 2015 23:42:31 +0300
Subject: Please help on this sorted function
In-Reply-To: <70489d38-0848-44c4-9a4c-ba2e2e9aa027@googlegroups.com>
References: <37d9de72-b719-45a0-976c-441fc5898741@googlegroups.com>
 <70489d38-0848-44c4-9a4c-ba2e2e9aa027@googlegroups.com>
Message-ID: <CAB1GNpQBLpqbU82guUGwMng6xEToujTbCmkLW2FK=UYV-snTeA@mail.gmail.com>

my_dict = {1: 'D', 2: 'B', 3: 'A', 4: 'E', 5: 'B'}

# dict.items() returns an iterator that returns pairs of (key, value) pairs
# the key argument to sorted tells sorted what to sort by,
operator.itemgetter is a factory function , itemgetter(1)== lambda
iterable: iterable[1]
sorted_dict = sorted(my_dict.items(), key=itemgetter(1))

# at this moment sorted dict is a generator of key-value tuples in the
right order
sorted_dict = OrderedDict(sorted_dict) # turn the generator in to an actual
dict.

# notice: regular dicts are NOT ORDERED, you need a special type of dict to
preserve the order, hence OrderedDict
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150602/993e0e7e/attachment.html>

From gherron at digipen.edu  Tue Jun  2 17:00:00 2015
From: gherron at digipen.edu (Gary Herron)
Date: Tue, 02 Jun 2015 14:00:00 -0700
Subject: Please help on this sorted function
In-Reply-To: <37d9de72-b719-45a0-976c-441fc5898741@googlegroups.com>
References: <37d9de72-b719-45a0-976c-441fc5898741@googlegroups.com>
Message-ID: <556E1950.5090901@digipen.edu>

On 06/02/2015 01:20 PM, fl wrote:
> Hi,
>
> I try to learn sorted(). With the tutorial example:
>
>
>
>
>>>> ff=sorted({1: 'D', 2: 'B', 3: 'B', 4: 'E', 5: 'A'})
>>>> ff
> [1, 2, 3, 4, 5]
>
>
>
> I don't see what sorted does in this dictionary, i.e. the sequence of
> 1..5 is unchanged. Could you explain it to me?
>
>
> Thanks,

It's best to think of dictionaries as unordered collections of key/value 
pairs.  Dictionaries are not sequences, do not have any particular 
ordering, and in full generality *can't* be sorted in any sensible way.

For instance, this slightly odd (but perfectly legal) dictionary
     >>> d = {'a':123, 456:'b'}
can't be sorted
     >>> sorted(d)
     Traceback (most recent call last):
       File "<stdin>", line 1, in <module>
     TypeError: unorderable types: int() < str()
because it doesn't make sense to order/compare the two keys 'a' and 456.

If your dictionary is a little better behaved, say
     >>> d = {'a':123, 'b':456}
you may be able to sort the keys
     >>> sorted(d)
     ['a', 'b']
or the values
     >>> sorted(d.values())
     [123, 456]
or the key/value tuples (called items)
     >>> sorted(d.items())
     [('a', 123), ('b', 456)]
but each of those attempts to sort could fail on a general dictionary if 
the individual keys or values are not sortable.

There is also an implementation of a type of dictionary that remembers 
the order in which the items are *inserted*.  It's in the collections 
module and called OrderedDict.


Gary Herron






-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418



From alain at universite-de-strasbourg.fr.invalid  Tue Jun  2 17:06:55 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Tue, 02 Jun 2015 23:06:55 +0200
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
Message-ID: <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>

Marko Rauhamaa <marko at pacujo.net> writes:

> Alain Ketterlin <alain at universite-de-strasbourg.fr.invalid>:
>
>> The close(2) manpage has the following warning on my Linux system:
>>
>> | Not checking the return value of close() is a common but
>> | nevertheless serious programming error. It is quite possible that
>> | errors on a previous write(2) operation are first reported at the
>> | final close(). Not checking the return value when closing the file
>> | may lead to silent loss of data. This can especially be observed
>> | with NFS and with disk quota.
>> | 
>>
>> (I haven't followed the thread, but if your problem is to make sure
>> fds are closed on exec, you may be better off using the...
>> close-on-exec flag. Or simply do the bookkeeping.)
>
> The quoted man page passage is a bit untenable.
>
> First, if close() fails, what's a poor program to do?

Warn the user? Not assume everything went well?  It all depends on the
application, and what the file descriptor represents.

> Try again?

Could be a good idea on NFS or other kind of mounts.

> How do you get rid of an obnoxious file descriptor?

You don't, you check everything before closing the file, with fsync()
for example.

I've no idea what the OP's program was doing, so I'm not going to split
hairs. I can't imagine why one would like to mass-close an arbitrary set
of file descriptors, and I think APIs like os.closerange() are toxic and
an appeal to sloppy programming.

-- Alain.


From john_ladasky at sbcglobal.net  Tue Jun  2 17:14:42 2015
From: john_ladasky at sbcglobal.net (John Ladasky)
Date: Tue, 2 Jun 2015 14:14:42 -0700 (PDT)
Subject: Where is 'palindrome' defined?
In-Reply-To: <87h9qraimu.fsf@elektro.pacujo.net>
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
 <mailman.267.1433141892.5151.python-list@python.org>
 <87h9qraimu.fsf@elektro.pacujo.net>
Message-ID: <0600731d-d2b1-4045-967e-0c7e4fdcb4c7@googlegroups.com>

On Monday, June 1, 2015 at 2:22:02 AM UTC-7, Marko Rauhamaa wrote:

>     Sota-apina nakataan raastimeen.
>           Apelle pane emit.
>       Saarnaa takanani paatos.
> 
>     (= "The war monkey will be chucked into a grater.
>         Hand the pistils to father-in-law.
>         Pathos does preach behind me.")

With this post, you have convinced me that 1) Finnish is a very interesting, and even poetic, language; and that 2) eating the velvet off of reindeer antlers must have a very similar effect to LSD.  :^)




From invalid at invalid.invalid  Tue Jun  2 17:19:29 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Tue, 2 Jun 2015 21:19:29 +0000 (UTC)
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
 <914115ea-0f93-4929-b325-5b3f78f5d9f1@googlegroups.com>
 <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.78.1433274654.13271.python-list@python.org>
Message-ID: <mkl6l1$lle$1@reader1.panix.com>

On 2015-06-02, Ian Kelly <ian.g.kelly at gmail.com> wrote:

>>> Sorry.  I meant "object" in the sense of OOP:  something you might
>>> extend or make a derived class with.
>>
>> I'm not sure you get to define which properties of objects you want
>> not to count.
>
> Accepting for the sake of argument that "something to be subclassed"
> is a reasonable definition of object,

Huh?  You can't subclass an object.  You can subclass a Class.

-- 
Grant Edwards               grant.b.edwards        Yow! I want to so HAPPY,
                                  at               the VEINS in my neck STAND
                              gmail.com            OUT!!


From rxjwg98 at gmail.com  Tue Jun  2 17:27:21 2015
From: rxjwg98 at gmail.com (fl)
Date: Tue, 2 Jun 2015 14:27:21 -0700 (PDT)
Subject: Can Python function return multiple data?
Message-ID: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>

Hi,

I just see the tutorial says Python can return value in function, it does 
not say multiple data results return situation. In C, it is possible.
How about Python on a multiple data return requirement?


Thanks,


From rxjwg98 at gmail.com  Tue Jun  2 17:33:01 2015
From: rxjwg98 at gmail.com (fl)
Date: Tue, 2 Jun 2015 14:33:01 -0700 (PDT)
Subject: What is the difference between list() and list?
Message-ID: <3ada3275-68c9-421c-aa19-53c312c42b1f@googlegroups.com>

Hi,

I find the following results are interesting, but I don't know the difference
between list() and list.






>>> nums=list()
>>> nums
[]
>>> xx=list
>>> xx
<type 'list'>
>>> nums
[]
>>> print(xx)
<type 'list'>
>>> print(nums)
[]
>>> 



Could you tell me that?

Thanks,


From joel.goldstick at gmail.com  Tue Jun  2 17:35:51 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 2 Jun 2015 17:35:51 -0400
Subject: Can Python function return multiple data?
In-Reply-To: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
Message-ID: <CAPM-O+ze010hWfMnRzd0=e1t+pUaHi2-YBuinbNWqq4=N4BLFA@mail.gmail.com>

On Tue, Jun 2, 2015 at 5:27 PM, fl <rxjwg98 at gmail.com> wrote:
> Hi,
>
> I just see the tutorial says Python can return value in function, it does
> not say multiple data results return situation. In C, it is possible.
> How about Python on a multiple data return requirement?
>
>
> Thanks,
> --
> https://mail.python.org/mailman/listinfo/python-list

Yes!

def my_function(n):
    return n, n*2

a_number, its_double = my_function(3)

a_number will be 3, its_double will be 6

-- 
Joel Goldstick
http://joelgoldstick.com


From joel.goldstick at gmail.com  Tue Jun  2 17:38:19 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 2 Jun 2015 17:38:19 -0400
Subject: What is the difference between list() and list?
In-Reply-To: <3ada3275-68c9-421c-aa19-53c312c42b1f@googlegroups.com>
References: <3ada3275-68c9-421c-aa19-53c312c42b1f@googlegroups.com>
Message-ID: <CAPM-O+wi3v0nMPS8c2ya-cVjMtSs+xBQATVzJeXpF_pBV13iKQ@mail.gmail.com>

On Tue, Jun 2, 2015 at 5:33 PM, fl <rxjwg98 at gmail.com> wrote:
> Hi,
>
> I find the following results are interesting, but I don't know the difference
> between list() and list.
>
>
>
>
>
>
>>>> nums=list()
>>>> nums
> []
>>>> xx=list
>>>> xx
> <type 'list'>
>>>> nums
> []
>>>> print(xx)
> <type 'list'>
>>>> print(nums)
> []
>>>>
>
>
>
> Could you tell me that?

list is the name of a built-in function that returns a list object
when you set xx = list, you are giving that function a new name -- 'xx'
when you invoke the function:  xx() or list() it returns a list
>
> Thanks,
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Joel Goldstick
http://joelgoldstick.com


From sohcahtoa82 at gmail.com  Tue Jun  2 17:40:27 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Tue, 2 Jun 2015 14:40:27 -0700 (PDT)
Subject: Can Python function return multiple data?
In-Reply-To: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
Message-ID: <31ce682a-b4ab-46da-a9e4-ef7fc86d28c6@googlegroups.com>

On Tuesday, June 2, 2015 at 2:27:37 PM UTC-7, fl wrote:
> Hi,
> 
> I just see the tutorial says Python can return value in function, it does 
> not say multiple data results return situation. In C, it is possible.
> How about Python on a multiple data return requirement?
> 
> 
> Thanks,

You return a tuple, set, or other iterable.  For example:

def return_two_values():
    return 1, 2

a, b = return_two_values()
print a
print b

This would print:
1
2

Note though that when doing something like this, you have to be really careful that if you have multiple calls to `return` in your function, that they will ALL return the same number of values.  Otherwise, when the tuple/list/etc. is unpacked, you'll get an error.

def return_two_values():
    # ... do some stuff
    if someCondition:
        print "someCondition was true!"
        return 0
    return 1, 2

a, b = return_two_values()

Here, if someCondition ended up being False, then an exception would be thrown.

Keep in mind that the unpacking of the returned value into the variables `a` and `b` works with *ANY* iterable.  So if you returned 'abc' and unpacked it into three variables, then the first would contain 'a', the second 'b', and the third 'c'.

You can also just return a dictionary if you want to return multiple values.


From gordon at panix.com  Tue Jun  2 17:40:30 2015
From: gordon at panix.com (John Gordon)
Date: Tue, 2 Jun 2015 21:40:30 +0000 (UTC)
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
Message-ID: <mkl7se$dsi$1@reader1.panix.com>

In <3bbe49da-e989-4a8c-a8a9-75d3a786f508 at googlegroups.com> fl <rxjwg98 at gmail.com> writes:

> Hi,

> I just see the tutorial says Python can return value in function, it does 
> not say multiple data results return situation. In C, it is possible.
> How about Python on a multiple data return requirement?

Python functions can return lists, tuples, dictionaries, or any other
container-like object.

-- 
John Gordon                   A is for Amy, who fell down the stairs
gordon at panix.com              B is for Basil, assaulted by bears
                                -- Edward Gorey, "The Gashlycrumb Tinies"



From breamoreboy at yahoo.co.uk  Tue Jun  2 17:47:16 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 02 Jun 2015 22:47:16 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <CALwzidnYFtTvsmJ8qOSM9jcXM=g1LdqW9WC6Nr=MPcn-RbMBPw@mail.gmail.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
 <914115ea-0f93-4929-b325-5b3f78f5d9f1@googlegroups.com>
 <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>
 <CALwzidnYFtTvsmJ8qOSM9jcXM=g1LdqW9WC6Nr=MPcn-RbMBPw@mail.gmail.com>
Message-ID: <mkl897$fr6$1@ger.gmane.org>

On 02/06/2015 20:50, Ian Kelly wrote:
> On Tue, Jun 2, 2015 at 1:31 PM, Jon Ribbens
> <jon+usenet at unequivocal.co.uk> wrote:
>> On 2015-06-02, Dr. Bigcock <dreamingforward at gmail.com> wrote:
>>> On Tuesday, June 2, 2015 at 1:49:03 PM UTC-5, Jon Ribbens wrote:
>>>> On 2015-06-02, Dr. Bigcock <dreamingforward at gmail.com> wrote:
>>>>> It doesn't really do anything.  No one uses integers as objects.
>>>>> (Any dissenters?)
>>>>
>>>> Yes. *Everyone* uses integers as objects. Containers such as
>>>> lists and dictionaries and tuples etc contain objects. If
>>>> integers weren't objects then you wouldn't be able to put them
>>>> in containers (and you'd end up with Java).
>>>
>>> Sorry.  I meant "object" in the sense of OOP:  something you might
>>> extend or make a derived class with.
>>
>> I'm not sure you get to define which properties of objects you want
>> not to count.
>
> Accepting for the sake of argument that "something to be subclassed"
> is a reasonable definition of object, it should be pointed out that
> anybody who works with bools in Python is using integers as objects.
>
> The "Super Considered Harmful" essay also has a couple of examples of
> subclasses of int, and even just googling "python subclass int"
> demonstrates that there is plenty of interest in the subject.
>

The classic response to "Super Considered Harmful" for those who may be 
interested is 
https://rhettinger.wordpress.com/2011/05/26/super-considered-super/ and 
recently https://www.youtube.com/watch?v=EiOglTERPEo

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From gordon at panix.com  Tue Jun  2 17:49:20 2015
From: gordon at panix.com (John Gordon)
Date: Tue, 2 Jun 2015 21:49:20 +0000 (UTC)
Subject: What is the difference between list() and list?
References: <3ada3275-68c9-421c-aa19-53c312c42b1f@googlegroups.com>
Message-ID: <mkl8cv$dsi$2@reader1.panix.com>

In <3ada3275-68c9-421c-aa19-53c312c42b1f at googlegroups.com> fl <rxjwg98 at gmail.com> writes:

> I find the following results are interesting, but I don't know the difference
> between list() and list.

'list()' invokes the list class, which creates and returns a new list.
Since you haven't passed any arguments, the list is empty.

'list' refers to the list class itself.

Here is a more in-depth example, using functions instead of classes:

    def hello(name):
        return 'Hello'.

If you call this function, like so:

    greeting = hello()
    print greeting

You will get the output 'Hello'.

But, if you just REFER to the function, instead of actually CALLING it:

    greeting = hello
    print greeting

You will get this output:

    <function hello at 0xbb266bc4>

Because you omitted the double parentheses, you're getting the hello
function object itself, instead of the RESULT of that function.

-- 
John Gordon                   A is for Amy, who fell down the stairs
gordon at panix.com              B is for Basil, assaulted by bears
                                -- Edward Gorey, "The Gashlycrumb Tinies"



From marko at pacujo.net  Tue Jun  2 18:24:33 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 01:24:33 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <87a8wh7nq6.fsf@elektro.pacujo.net>

Alain Ketterlin <alain at universite-de-strasbourg.fr.invalid>:

> Marko Rauhamaa <marko at pacujo.net> writes:
>> First, if close() fails, what's a poor program to do?
>
> Warn the user? Not assume everything went well? It all depends on the
> application, and what the file descriptor represents.

The problem here is in the system call contract, which is broken.
There's no fix. The man page admonition is just hand-waving without
constructive advice.

>> Try again?
> Could be a good idea on NFS or other kind of mounts.

Maybe close() will fail for ever.

> I can't imagine why one would like to mass-close an arbitrary set of
> file descriptors,

That's standard practice before execking a file. Failure to do that can
seriously hurt the parent process. For example, the parent (or child)
will never read an EOF from file descriptor if its duplicate is open in
an unwitting child process. Also, the number of open files in the system
may grow over all limits or simply waste kernel resources.

Close-on-exec is nice, maybe. However, you don't have control over all
file descriptors. Loggers, high-level library calls and others open
files without the application programmer knowing or having direct
control over.

> and I think APIs like os.closerange() are toxic and an appeal to
> sloppy programming.

And you recommend what instead?


Marko


From marko at pacujo.net  Tue Jun  2 18:30:22 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 01:30:22 +0300
Subject: Where is 'palindrome' defined?
References: <83a279a3-133d-4a50-9af3-054233bcc6af@googlegroups.com>
 <mailman.267.1433141892.5151.python-list@python.org>
 <87h9qraimu.fsf@elektro.pacujo.net>
 <0600731d-d2b1-4045-967e-0c7e4fdcb4c7@googlegroups.com>
Message-ID: <8761757ngh.fsf@elektro.pacujo.net>

John Ladasky <john_ladasky at sbcglobal.net>:

> On Monday, June 1, 2015 at 2:22:02 AM UTC-7, Marko Rauhamaa wrote:
>
>>     Sota-apina nakataan raastimeen.
>>           Apelle pane emit.
>>       Saarnaa takanani paatos.
>> 
>>     (= "The war monkey will be chucked into a grater.
>>         Hand the pistils to father-in-law.
>>         Pathos does preach behind me.")
>
> With this post, you have convinced me that 1) Finnish is a very
> interesting, and even poetic, language; and that 2) eating the velvet
> off of reindeer antlers must have a very similar effect to LSD. :^)

The phonetic structure of Finnish is ideal for palindromes, unlike
English.


Marko


From marko at pacujo.net  Tue Jun  2 18:33:00 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 01:33:00 +0300
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
 <914115ea-0f93-4929-b325-5b3f78f5d9f1@googlegroups.com>
 <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.78.1433274654.13271.python-list@python.org>
 <mkl6l1$lle$1@reader1.panix.com>
Message-ID: <871tht7nc3.fsf@elektro.pacujo.net>

Grant Edwards <invalid at invalid.invalid>:

> On 2015-06-02, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> Accepting for the sake of argument that "something to be subclassed"
>> is a reasonable definition of object,
>
> Huh?  You can't subclass an object.  You can subclass a Class.

More to the point: you don't need classes for objects -- even in the
deepest OOP sense.

In Python, classes are little more than constructor functions.


Marko


From rosuav at gmail.com  Tue Jun  2 19:54:06 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 3 Jun 2015 09:54:06 +1000
Subject: fork/exec & close file descriptors
In-Reply-To: <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <CAPTjJmpujtP5AKdFJUSx_NS7R_zddR-nEiL6Srt5B8tM2D2vZQ@mail.gmail.com>

On Wed, Jun 3, 2015 at 7:06 AM, Alain Ketterlin
<alain at universite-de-strasbourg.fr.invalid> wrote:
> I've no idea what the OP's program was doing, so I'm not going to split
> hairs. I can't imagine why one would like to mass-close an arbitrary set
> of file descriptors, and I think APIs like os.closerange() are toxic and
> an appeal to sloppy programming.

When you fork, you get a duplicate referent to every open file in both
parent and child. Closing them all in the child is very common, as it
allows the parent to continue owning those file descriptors (so that
when you close it in the parent, the resource is really closed). One
notable example is with listening sockets; bind/listen in the parent,
then fork (maybe to handle a client), then terminate the parent
process. You now cannot restart the parent without aborting the child,
as the child now owns that listening socket (even if it never wants to
use it). There are some specific ways around this, but not on all OSes
(eg Linux only added support for SO_REUSEPORT in 3.9), and the best
way has always been to make sure the children don't hang onto the
listening socket. (There are other good reasons for doing this, too.)

ChrisA


From rosuav at gmail.com  Tue Jun  2 19:56:37 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 3 Jun 2015 09:56:37 +1000
Subject: Can Python function return multiple data?
In-Reply-To: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
Message-ID: <CAPTjJmp6B0hX5TDCTOT60e4jREeceJn5G95ZtwpbifqVKXhdLA@mail.gmail.com>

On Wed, Jun 3, 2015 at 7:27 AM, fl <rxjwg98 at gmail.com> wrote:
> I just see the tutorial says Python can return value in function, it does
> not say multiple data results return situation. In C, it is possible.
> How about Python on a multiple data return requirement?

Technically, neither C nor Python can return multiple values from a
single function call. In Python, the most common way to do this is to
return a tuple, which can then be unpacked; as other posts in this
thread have shown, this can look a lot like returning multiple values,
and it's pretty convenient. In C, the nearest equivalent is passing a
number of pointers as parameters, and having the function fill out
values. Python's model is a lot closer to what you're saying than C's
model is :)

ChrisA


From rosuav at gmail.com  Tue Jun  2 20:01:36 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 3 Jun 2015 10:01:36 +1000
Subject: Please help on this sorted function
In-Reply-To: <70489d38-0848-44c4-9a4c-ba2e2e9aa027@googlegroups.com>
References: <37d9de72-b719-45a0-976c-441fc5898741@googlegroups.com>
 <70489d38-0848-44c4-9a4c-ba2e2e9aa027@googlegroups.com>
Message-ID: <CAPTjJmqiEboP9DRXggvDZBPwiT0M2PbULge5ucsVe_Y9v+FqXQ@mail.gmail.com>

On Wed, Jun 3, 2015 at 6:25 AM, fl <rxjwg98 at gmail.com> wrote:
> I am still new to Python. How to get the sorted dictionary output:
>
> {1: 'D', 2: 'B', 3: 'A', 4: 'E', 5: 'B'}

Since dictionaries don't actually have any sort of order to them, the
best thing to do is usually to simply display it in order. And there's
a very handy function for doing that: a pretty-printer.

>>> import pprint
>>> pprint.pprint({1: 'D', 2: 'B', 5: 'B', 4: 'E', 3: 'A'})
{1: 'D', 2: 'B', 3: 'A', 4: 'E', 5: 'B'}

This one comes with Python, so you can use it as easily as that above
example. Or you could do it this way:

>>> from pprint import pprint
>>> pprint({1: 'D', 2: 'B', 5: 'B', 4: 'E', 3: 'A'})
{1: 'D', 2: 'B', 3: 'A', 4: 'E', 5: 'B'}

For a lot of Python data structures, this will give you a tidy and
human-readable display.

ChrisA


From ian.g.kelly at gmail.com  Tue Jun  2 20:49:17 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 2 Jun 2015 18:49:17 -0600
Subject: Everything is an object in python - object class and type class
In-Reply-To: <mkl897$fr6$1@ger.gmane.org>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
 <914115ea-0f93-4929-b325-5b3f78f5d9f1@googlegroups.com>
 <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>
 <CALwzidnYFtTvsmJ8qOSM9jcXM=g1LdqW9WC6Nr=MPcn-RbMBPw@mail.gmail.com>
 <mkl897$fr6$1@ger.gmane.org>
Message-ID: <CALwzidn-1fGGtA92T7xE+2wuGiEuC+ts=8+d2yad0A+wCZ9gTQ@mail.gmail.com>

On Tue, Jun 2, 2015 at 3:47 PM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> The classic response to "Super Considered Harmful" for those who may be
> interested is
> https://rhettinger.wordpress.com/2011/05/26/super-considered-super/ and
> recently https://www.youtube.com/watch?v=EiOglTERPEo

I feel slightly cheated. In the video he promises to show how to do
dependency injection using super, but in both of the examples given,
every instance of "super()" could simply be replaced with "self" and
the result would be the same. That's just taking advantage of the MRO,
not super.


From skip.montanaro at gmail.com  Tue Jun  2 21:21:06 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 2 Jun 2015 20:21:06 -0500
Subject: fork/exec & close file descriptors
In-Reply-To: <CAPTjJmpujtP5AKdFJUSx_NS7R_zddR-nEiL6Srt5B8tM2D2vZQ@mail.gmail.com>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <CAPTjJmpujtP5AKdFJUSx_NS7R_zddR-nEiL6Srt5B8tM2D2vZQ@mail.gmail.com>
Message-ID: <CANc-5UxG+ZvgAn5u+RvqpE3Y_gK5t4bMWM+SfoKNYV2NV=obuQ@mail.gmail.com>

Folks, I'm not interested in rehashing this. I don't know what all the
open file descriptors are. Some/many/most will have been opened by
underlying C++ libraries and will not have been opened by code at the
Python level. I do think the addition of an os.fsync() attempt on each
file descriptor before calling os.close() makes some sense to me.
Beyond that, I'm not sure there's more that can be done.

I sent my last message simply because I was surprised os.closerange()
existed. I have no idea why it didn't turn up in my original
searching.

Skip


From ian.g.kelly at gmail.com  Tue Jun  2 21:39:31 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 2 Jun 2015 19:39:31 -0600
Subject: should "self" be changed?
In-Reply-To: <87mw0i6n9i.fsf@elektro.pacujo.net>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net> <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
 <87siagagpx.fsf@elektro.pacujo.net>
 <5567c840$0$12998$c3e8da3$5496439d@news.astraweb.com>
 <556ddef1$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87mw0i6n9i.fsf@elektro.pacujo.net>
Message-ID: <CALwzid=CSgoea2CqvvYzGRNYhapAZ2aOkT9RJGK5Qi2D4QTkhw@mail.gmail.com>

On Tue, Jun 2, 2015 at 11:19 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Steven D'Aprano <steve at pearwood.info>:
>
>> On Fri, 29 May 2015 12:00 pm, Steven D'Aprano wrote:
>>
>> [...]
>>> in a language where classes are
>>> themselves values, there is no reason why a class must be instantiated,
>>> particularly if you're only using a single instance of the class. Anyone
>>> ever come across a named design pattern that involves using classes
>>> directly without instantiating them?
>>>
>>> I'm basically looking for a less inelegant term for "instanceless class"
>>> -- not so much a singleton as a zeroton.
>>
>> C# has these, and calls them static classes.
>
> I guess Python has them, too, and calls them modules.

Indeed. I find it amusing that C# has special syntax to work around
what is ostensibly a design feature -- that all code must be contained
in a class (or struct).

But then, the same practice also exists in Java, where there is no
specific syntax for it.


From kukki.kanchana at gmail.com  Tue Jun  2 22:37:21 2015
From: kukki.kanchana at gmail.com (Pythonista)
Date: Tue, 2 Jun 2015 19:37:21 -0700 (PDT)
Subject: Issuing commands using "exec_command()" of paramiko AND also sending
 commands together
Message-ID: <3603e97e-686a-47de-aff4-cabfcb930064@googlegroups.com>

Using paramiko's exec_command(), i would like to send a command, process its output and do it for several other commands. I notice that its not quick enough or something like that.

How would I handle that scenario AND also providing multiple commands together (there is 1 post on stackoverflow for issuing multiple commands but I am not sure if someone has tried it. It didnt work for me!

Thanks!


From edd.cowan at gmail.com  Wed Jun  3 00:16:48 2015
From: edd.cowan at gmail.com (Eddilbert Macharia)
Date: Tue, 2 Jun 2015 21:16:48 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>

On Tuesday, June 2, 2015 at 2:27:31 PM UTC+3, Steven D'Aprano wrote:
> On Tue, 2 Jun 2015 08:36 pm, Eddilbert Macharia wrote:
> 
> > you guys are just confusing me, you are going in loops, and still i have
> > understood ,what makes everything in python an object. hey is where i'm at
> > : *** type in python refers to data types e.g. int, str, boolean e.t.c.
> > right ?
> 
> Yes. Also classes you create with the "class" keyword:
> 
> class K(object):
>     ...
> 
> K is now a "type", just like int, str, list, object, etc.
> 
> 
> > *** The interpreter creates two classes type and object when setting up a
> > python environment. right ?
> 
> Many more than just two: it also creates list, str, dict, etc. But *first*
> it has to create type and object. So you are correct.
> 
> 
> > *** The creator (metaclass) of all data types (i.e. int,str) in python is
> > the class type. right ?
> 
> Correct.
> 
> [Aside: I'm only talking about Python 3 here. In Python 2 there is also a
> second hierarchy of classes, called "classic classes" or "old-style
> classes", which are *not* subclasses of type. But let's just ignore them,
> because they are gone in the most recent versions of Python.]
> 
> 
> >>>> isinstance(int,type)
> > True
> > 
> > *** The instance of class type is a data type an instance of class type.
> > right ?
> >>>> type(type)
> > <class 'type'>
> 
> type has many instances, not just one.
> 
> Instances of int are individual ints, like 0, 1, 2, 3, ...
> 
> Instances of type are individual types, like int, dict, str, list, ...
> 
> But one of those many instances of type is, yes, type itself! So *one* of
> the instances of type is type, which is also an instance of itself:
> 
> >>>> isinstance(type,type)
> > True
> 
> Correct. This makes type special. Most types are *not* instances of
> themselves:
> 
> py> isinstance(int, int)
> False
> 
> 
> > *** Class type gets some of its behavior from class object through
> > inheritance.right ?
> > 
> >>>> issubclass(type,object)
> > True
> 
> Correct.
> 
> 
> > *** instance of class object is type, in the sense it created using class
> > type which inherits from class object.right ?
> > 
> >>>> isinstance(object,type)
> > True
> >>>> isinstance(object,object)
> > True
> > 
> > ****so is it right to say, everything in python is an object because they
> > are instance of the class type which inherits from class object ?
> 
> No! That's not what we mean when we say "everything is an object".
> 
> Eddilbert, have you programmed in any other languages? It would help you
> understand if you have.

Sadly yes i have worked with java, and that is what is causing me so much grief.In java objects are instance of a class.pretty simple.

> 
> "Object" has a general meaning in computer science and programming, it is a
> compound data structure that is explicitly linked to a type which provides
> functionality that operates on that data structure.
> 
> In the programming language C, *no* values are objects. C has types (int16,
> uint32, bool, and many more) but no objects.
> 
> In the programming language Java, *some* values are objects, and some values
> are not objects.
> 
> In the programming language Python, *all* values are objects, in the general
> sense. That is what we mean by "Everything is an object".
> 
> Let's go back in time about 15 years or so. We're now using Python 1.5. In
> Python 1.5, there is no built-in object, and type is just a function, not a
> class:
> 
> >>> import sys
> >>> print sys.version
> 1.5.2 (#1, Aug 27 2012, 09:09:18)  [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]
> >>> object
> Traceback (innermost last):
>   File "<stdin>", line 1, in ?
> NameError: object
> >>> type
> <built-in function type>
> 
> 
> In Python 1.5, classes you create do not inherit from object, because object
> does not exist! BUT even in Python 1.5, it is true that everything is an
> object.
> 
> Remember, "object" can refer to two things:
> 
> - *specifically* the class called "object";
> 
> - the *general concept* of objects, from object oriented programming.
> 
> 
> In Python 1.5:
> 
> - everything is an object [the general concept]
> 
> - nothing is an instance of the class called "object"
> 
> 
> In Python 2:
> 
> - everything is an object [the general concept]
> 
> - some things, but not all things, are instances of the class
>   called "object"
> 
> 
> In Python 3:
> 
> - everything is an object [the general concept]
> 
> - everything is an instance of the class called "object"
> 
> 
> 
> 

I think then in python object and instance of a class are entirely different things.

in OOP and python - object is a representation of a real thing, or a concept .e.g a person,number and the concept of classes- which is the concept of create/representing other objects using a programming language to the machine.

class - This is what is used to create/represent objects in the machine using a programming language

class instance - This is the output of the classes this is a representation of an object.


> -- 
> Steven



From breamoreboy at yahoo.co.uk  Wed Jun  3 01:23:20 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 03 Jun 2015 06:23:20 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
Message-ID: <mkm30c$rfg$1@ger.gmane.org>

On 03/06/2015 05:16, Eddilbert Macharia wrote:
>
> Sadly yes i have worked with java, and that is what is causing me so much grief.In java objects are instance of a class.pretty simple.
>

You might like to read 
http://dirtsimple.org/2004/12/python-is-not-java.html and 
http://dirtsimple.org/2004/12/java-is-not-python-either.html

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From steve+comp.lang.python at pearwood.info  Wed Jun  3 03:05:56 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 03 Jun 2015 17:05:56 +1000
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net> <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
 <87siagagpx.fsf@elektro.pacujo.net>
 <5567c840$0$12998$c3e8da3$5496439d@news.astraweb.com>
 <556ddef1$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87mw0i6n9i.fsf@elektro.pacujo.net>
Message-ID: <556ea755$0$12995$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 03 June 2015 03:19, Marko Rauhamaa wrote:

> Steven D'Aprano <steve at pearwood.info>:
> 
>> On Fri, 29 May 2015 12:00 pm, Steven D'Aprano wrote:
>>
>> [...]
>>> in a language where classes are
>>> themselves values, there is no reason why a class must be instantiated,
>>> particularly if you're only using a single instance of the class. Anyone
>>> ever come across a named design pattern that involves using classes
>>> directly without instantiating them?
>>> 
>>> I'm basically looking for a less inelegant term for "instanceless class"
>>> -- not so much a singleton as a zeroton.
>>
>> C# has these, and calls them static classes.
> 
> I guess Python has them, too, and calls them modules.


Modules play a similar role -- after all, modules and classes are both 
namespaces. But:

- you can't (easily) use inheritance on a module to make a new module, 
  but you can use inheritance on a class; although I think C# prohibits
  that.

- you can't (easily) include more than one module in a single file.


-- 
Steve



From sreenath.cg at gmail.com  Wed Jun  3 03:07:40 2015
From: sreenath.cg at gmail.com (Sreenathan Nair)
Date: Wed, 3 Jun 2015 12:37:40 +0530
Subject: Issuing commands using "exec_command()" of paramiko AND also
 sending commands together
In-Reply-To: <3603e97e-686a-47de-aff4-cabfcb930064@googlegroups.com>
References: <3603e97e-686a-47de-aff4-cabfcb930064@googlegroups.com>
Message-ID: <CAC87=uPhEKJcHiuke7Go-wcT9nAYjzyYX=2vJWVbDty2h_omtA@mail.gmail.com>

Hi,

Could you be more specific about your problem? Perhaps an example of
something similar to what you're trying to do would be helpful.

Usually the process is to instantiate paramiko.SSHCLIENT, use the connect()
method with desired parameters and execute commands using the
exec_command(). If you'd like to process the output of the command
execution, then you would store the result of exec_command() into three
variables (it return a 3-tuple of Channel objects).

i.e com_stdin, com_stdout, com_stderr =
my_ssh_client_instance.exec_command("<command>")

The instance of SSHCLIENT is live until the close() method is called.
Meaning subsequent commands can be executed the same way.


On Wed, Jun 3, 2015 at 8:07 AM, Pythonista <kukki.kanchana at gmail.com> wrote:

> Using paramiko's exec_command(), i would like to send a command, process
> its output and do it for several other commands. I notice that its not
> quick enough or something like that.
>
> How would I handle that scenario AND also providing multiple commands
> together (there is 1 post on stackoverflow for issuing multiple commands
> but I am not sure if someone has tried it. It didnt work for me!
>
> Thanks!
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150603/a4ca3dec/attachment.html>

From alain at universite-de-strasbourg.fr.invalid  Wed Jun  3 03:11:19 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Wed, 03 Jun 2015 09:11:19 +0200
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <mailman.84.1433289254.13271.python-list@python.org>
Message-ID: <87h9qpl10o.fsf@universite-de-strasbourg.fr.invalid>

Chris Angelico <rosuav at gmail.com> writes:

> On Wed, Jun 3, 2015 at 7:06 AM, Alain Ketterlin
> <alain at universite-de-strasbourg.fr.invalid> wrote:
>> I've no idea what the OP's program was doing, so I'm not going to split
>> hairs. I can't imagine why one would like to mass-close an arbitrary set
>> of file descriptors, and I think APIs like os.closerange() are toxic and
>> an appeal to sloppy programming.
>
> When you fork, you get a duplicate referent to every open file in both
> parent and child. [...]

Thank you, I know this. What I mean is: what are the reasons that you
cannot access your file descriptors one by one? To me closing a range of
descriptors has absolutely no meaning, simply because ranges have no
meaning for file descriptors (they're not ordered in any way). What if
some library uses its own descriptors that happen to lie in your
"range"? Etc.

-- Alain.


From alain at universite-de-strasbourg.fr.invalid  Wed Jun  3 03:21:43 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Wed, 03 Jun 2015 09:21:43 +0200
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
Message-ID: <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>

Marko Rauhamaa <marko at pacujo.net> writes:

> Alain Ketterlin <alain at universite-de-strasbourg.fr.invalid>:
>
>> Marko Rauhamaa <marko at pacujo.net> writes:
>>> First, if close() fails, what's a poor program to do?
>>
>> Warn the user? Not assume everything went well? It all depends on the
>> application, and what the file descriptor represents.
>
> The problem here is in the system call contract, which is broken.
> There's no fix. The man page admonition is just hand-waving without
> constructive advice.
>
>>> Try again?
>> Could be a good idea on NFS or other kind of mounts.
>
> Maybe close() will fail for ever.

Your program has to deal with this, something is going wrong, it can't
just close and go on.

>> I can't imagine why one would like to mass-close an arbitrary set of
>> file descriptors,
>
> That's standard practice before execking a file. [...]

>> and I think APIs like os.closerange() are toxic and an appeal to
>> sloppy programming.
>
> And you recommend what instead?

Keeping them around and closing the ones you own, plus doing whatever is
necessary to have libraires (loggers, database connectors, etc.) finish
properly. And not taking the risk of messing with descriptors your
program is not responsible of.

-- Alain.


From marko at pacujo.net  Wed Jun  3 03:41:44 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 10:41:44 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <877frl6xxj.fsf@elektro.pacujo.net>

Alain Ketterlin <alain at universite-de-strasbourg.fr.invalid>:

> Marko Rauhamaa <marko at pacujo.net> writes:
>> Maybe close() will fail for ever.
>
> Your program has to deal with this, something is going wrong, it can't
> just close and go on.

Here's the deal: the child process is saddled with file descriptors it
never wanted in the first place. It can't decline them. Now you're
saying it can't even dispose of them.

The reason this has been allowed to go on is that everybody just closes
the file descriptors and ignores the possibility or repercussions of a
failure. I haven't read about horror stories of this failing.

I readily admit this is very dirty, but since the API doesn't offer a
clean alternative, there's nothing you can/should do about it.


Marko


From steve+comp.lang.python at pearwood.info  Wed Jun  3 04:08:54 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 03 Jun 2015 18:08:54 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
Message-ID: <556eb617$0$12919$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 03 June 2015 03:59, BartC wrote:

> Javascript primitives include Number and String.
> 
> What does Python allow to be done with its Number (int, etc) and String
> types that can't be done with their Javascript counterparts, that makes
> /them/ objects?

That's a good question, and I'm not sure whether or not the answer is 
"nothing, it's just an implementation detail".

I don't *think* it is an implementation detail, but I don't know enough 
about Javascript to be sure. I can see that you can include Numbers in an 
object without explicitly boxing them:

js> var foo = 23;
js> typeof foo;  
number
js> var bar = {1: foo};
js> typeof bar[1];
number


I can also see that you can explicitly box numbers inside objects:

js> var a = 42;
js> var b = new Number(42);
js> a == b;
true
js> a === b;
false
js> typeof a;
number
js> typeof b;
object


But I don't know enough to tell the practical differences between them.



-- 
Steve



From steve+comp.lang.python at pearwood.info  Wed Jun  3 04:13:34 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 03 Jun 2015 18:13:34 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
 <914115ea-0f93-4929-b325-5b3f78f5d9f1@googlegroups.com>
 <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.78.1433274654.13271.python-list@python.org>
 <mkl6l1$lle$1@reader1.panix.com> <871tht7nc3.fsf@elektro.pacujo.net>
 <fb0716a8-6470-48fe-b82b-b1587a486e04@googlegroups.com>
Message-ID: <556eb72f$0$12904$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 03 June 2015 08:49, Dr. Bigcock wrote:

> You need classes for objects.  Anything else, and you're confusing
> yourself.


Not quite.

https://en.wikipedia.org/wiki/Prototype-based_programming




-- 
Steve



From David.Aldrich at EMEA.NEC.COM  Wed Jun  3 04:19:23 2015
From: David.Aldrich at EMEA.NEC.COM (David Aldrich)
Date: Wed, 3 Jun 2015 08:19:23 +0000
Subject: What sort of data structure to use?
Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B97DECA@EX10MBX02.EU.NEC.COM>

Hi

I have written a Python utility that performs a certain activity on some predefined sets of files.  Here is the outline of what I have written:

# File Set A
pathA = 'pathA'
fileListA = ['fileA1.txt', 'fileA2.txt']

# File Set B
pathB = 'pathB'
fileListB = ['fileB1.txt', 'fileB2.txt', 'fileB3.txt']

myFunc1(pathA, fileListA)
myFunc2(pathA, fileListA)

myFunc1(pathB, fileListB)
myFunc2(pathB, fileListB)

I want to add more file sets, so I really want to add the sets to a list and iterate over the list, calling myFunc1 & myFunc2 for each item.

My question is: what sort of data structure could I use to organise this, given that I want to associate a set of files with each path and that, for each set, there is an arbitrary number of files?

Best regards

David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150603/41236d73/attachment.html>

From steve+comp.lang.python at pearwood.info  Wed Jun  3 04:21:08 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 03 Jun 2015 18:21:08 +1000
Subject: Please help on this sorted function
References: <37d9de72-b719-45a0-976c-441fc5898741@googlegroups.com>
 <70489d38-0848-44c4-9a4c-ba2e2e9aa027@googlegroups.com>
 <mailman.80.1433277754.13271.python-list@python.org>
Message-ID: <556eb8f6$0$13008$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 03 June 2015 06:42, Joonas Liik wrote:

> my_dict = {1: 'D', 2: 'B', 3: 'A', 4: 'E', 5: 'B'}
> 
> # dict.items() returns an iterator that returns pairs of (key, value)
> # pairs the key argument to sorted tells sorted what to sort by,
> operator.itemgetter is a factory function , itemgetter(1)== lambda
> iterable: iterable[1]
> sorted_dict = sorted(my_dict.items(), key=itemgetter(1))
> 
> # at this moment sorted dict is a generator of key-value tuples in the
> right order
> sorted_dict = OrderedDict(sorted_dict) # turn the generator in to an
> actual dict.
> 
> # notice: regular dicts are NOT ORDERED, you need a special type of dict
> # to
> preserve the order, hence OrderedDict

OrderedDicts preserve the *insertion order*, they don't sort the keys.


Ordinary dicts are unordered. The order you see is arbitrary and 
unpredictable:

py> d = {}
py> d['C'] = 1; d['A'] = 2; d['B'] = 3
py> d
{'A': 2, 'C': 1, 'B': 3}


Ordered dicts are ordered by insertion order:

py> from collections import OrderedDict
py> d = OrderedDict()
py> d['C'] = 1; d['A'] = 2; d['B'] = 3
py> d
OrderedDict([('C', 1), ('A', 2), ('B', 3)])


Python doesn't have a SortedDict, where the keys are kept in sorted order.



-- 
Steve



From steve+comp.lang.python at pearwood.info  Wed Jun  3 04:24:45 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 03 Jun 2015 18:24:45 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
 <914115ea-0f93-4929-b325-5b3f78f5d9f1@googlegroups.com>
 <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.78.1433274654.13271.python-list@python.org>
 <mkl6l1$lle$1@reader1.panix.com> <871tht7nc3.fsf@elektro.pacujo.net>
Message-ID: <556eb9cd$0$13008$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 03 June 2015 08:33, Marko Rauhamaa wrote:

> Grant Edwards <invalid at invalid.invalid>:
> 
>> On 2015-06-02, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>>> Accepting for the sake of argument that "something to be subclassed"
>>> is a reasonable definition of object,
>>
>> Huh?  You can't subclass an object.  You can subclass a Class.
> 
> More to the point: you don't need classes for objects -- even in the
> deepest OOP sense.

That part is true.

> In Python, classes are little more than constructor functions.

But that's not.

Classes give you an inheritance hierarchy. They also hold shared state, and 
behaviour for the instances.



-- 
Steve



From steve+comp.lang.python at pearwood.info  Wed Jun  3 04:32:42 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 03 Jun 2015 18:32:42 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
 <914115ea-0f93-4929-b325-5b3f78f5d9f1@googlegroups.com>
 <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <556ebbac$0$12929$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 03 June 2015 05:31, Jon Ribbens wrote:

> On 2015-06-02, Dr. Bigcock <dreamingforward at gmail.com> wrote:
>> On Tuesday, June 2, 2015 at 1:49:03 PM UTC-5, Jon Ribbens wrote:
>>> On 2015-06-02, Dr. Bigcock <dreamingforward at gmail.com> wrote:
>>> > It doesn't really do anything.  No one uses integers as objects.
>>> > (Any dissenters?)
>>> 
>>> Yes. *Everyone* uses integers as objects. Containers such as
>>> lists and dictionaries and tuples etc contain objects. If
>>> integers weren't objects then you wouldn't be able to put them
>>> in containers (and you'd end up with Java).
>>
>> Sorry.  I meant "object" in the sense of OOP:  something you might
>> extend or make a derived class with.
> 
> I'm not sure you get to define which properties of objects you want
> not to count.
> 
>> Your last claim, must not be true because integers were able to be
>> placed in objects before the type/class unification with v2.6, I
>> believe.
> 
> Unless I'm misremembering, before that they were still objects,
> just not quite the same kind of objects as pure-Python ones.

Correct. It was version 2.2, not 2.6, that unified built-in types with 
classes. Prior to that, Python had two distinct kinds of object, with 
separate hierarchies:


Types  (builtins, defined in C)
 +-- int
 +-- dict
 +-- str
 +-- list

Classes  (custom-made in Python using the class keyword)
 +-- Foo
 +-- Bar
      +-- FooBar
      +-  FooBarBaz


You could only subclass classes, not types. But *both* were kinds of 
objects. They were just separate, with slightly different characteristics.

Starting with 2.2, the old-style classic classes still existed, for 
backwards compatibility, but Python introduced a single base-class for the 
built-in types, called it "object", and enabled subclassing from pure-Python 
code:


Unified types/classes ("new-style classes")
 +-- object
     +-- int
         +-- MyInteger
     +-- dict
     +-- str
     +-- list
         +-- MyList
     +-- Spam

Classic Classes ("old-style classes")
[unchanged from above]


Finally, in Python 3, the classic classes were removed, and Python now has a 
single unified type/class hierarchy, with "object" at the root.


But regardless of whether Python had a single type hierarchy or two separate 
hierarchies, all values in Python were still implemented as objects.


-- 
Steve



From marko at pacujo.net  Wed Jun  3 04:57:33 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 11:57:33 +0300
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
 <914115ea-0f93-4929-b325-5b3f78f5d9f1@googlegroups.com>
 <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.78.1433274654.13271.python-list@python.org>
 <mkl6l1$lle$1@reader1.panix.com> <871tht7nc3.fsf@elektro.pacujo.net>
 <556eb9cd$0$13008$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <871tht6uf6.fsf@elektro.pacujo.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info>:

> On Wednesday 03 June 2015 08:33, Marko Rauhamaa wrote:
>> In Python, classes are little more than constructor functions.
>
> [...]
>
> Classes give you an inheritance hierarchy.

That's encapsulated in the constructor. From the class user's point of
view, it doesn't matter if the object derives its behavior through
inheritance or through reimplementation. From the class implementor's
point of view, inheritance is syntactic sugar.

> They also hold shared state, and behaviour for the instances.

That's the "little more" I was referring to. It is Python esoterics
that can safely be ignored.


Marko


From dpalao.python at gmail.com  Wed Jun  3 05:22:33 2015
From: dpalao.python at gmail.com (David Palao)
Date: Wed, 3 Jun 2015 11:22:33 +0200
Subject: What sort of data structure to use?
In-Reply-To: <41302A7145AC054FA7A96CFD03835A0A0B97DECA@EX10MBX02.EU.NEC.COM>
References: <41302A7145AC054FA7A96CFD03835A0A0B97DECA@EX10MBX02.EU.NEC.COM>
Message-ID: <CAKUKWzmU8xKpuL8HzzoOs266wSwpDZk+4nj0Q5cAJ=e4v2oOSw@mail.gmail.com>

2015-06-03 10:19 GMT+02:00 David Aldrich <David.Aldrich at emea.nec.com>:
> Hi
>
>
>
> I have written a Python utility that performs a certain activity on some
> predefined sets of files.  Here is the outline of what I have written:
>
>
>
> # File Set A
>
> pathA = ?pathA?
>
> fileListA = [?fileA1.txt?, ?fileA2.txt?]
>
>
>
> # File Set B
>
> pathB = ?pathB?
>
> fileListB = [?fileB1.txt?, ?fileB2.txt?, ?fileB3.txt?]
>
>
>
> myFunc1(pathA, fileListA)
>
> myFunc2(pathA, fileListA)
>
>
>
> myFunc1(pathB, fileListB)
>
> myFunc2(pathB, fileListB)
>
>
>
> I want to add more file sets, so I really want to add the sets to a list and
> iterate over the list, calling myFunc1 & myFunc2 for each item.
>
>
>
> My question is: what sort of data structure could I use to organise this,
> given that I want to associate a set of files with each path and that, for
> each set, there is an arbitrary number of files?
>
>
>
> Best regards
>
>
>
> David
>
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

Hi,
What about a for loop?
paths = [pathA, pathB, pathC]
functions = (myFunc1, myFunc2)
file_lists = [fileListA, fileListB,fileListC]
for path, file_list in zip(paths, file_lists):
    for f in functions:
        f(path, file_list)

Best


From cs at zip.com.au  Wed Jun  3 05:24:46 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 3 Jun 2015 19:24:46 +1000
Subject: What sort of data structure to use?
In-Reply-To: <41302A7145AC054FA7A96CFD03835A0A0B97DECA@EX10MBX02.EU.NEC.COM>
References: <41302A7145AC054FA7A96CFD03835A0A0B97DECA@EX10MBX02.EU.NEC.COM>
Message-ID: <20150603092446.GA53716@cskk.homeip.net>

On 03Jun2015 08:19, David Aldrich <David.Aldrich at EMEA.NEC.COM> wrote:
>I have written a Python utility that performs a certain activity on some predefined sets of files.  Here is the outline of what I have written:
>
># File Set A
>pathA = 'pathA'
>fileListA = ['fileA1.txt', 'fileA2.txt']
>
># File Set B
>pathB = 'pathB'
>fileListB = ['fileB1.txt', 'fileB2.txt', 'fileB3.txt']
>
>myFunc1(pathA, fileListA)
>myFunc2(pathA, fileListA)
>
>myFunc1(pathB, fileListB)
>myFunc2(pathB, fileListB)
>
>I want to add more file sets, so I really want to add the sets to a list and iterate over the list, calling myFunc1 & myFunc2 for each item.
>
>My question is: what sort of data structure could I use to organise this, given that I want to associate a set of files with each path and that, for each set, there is an arbitrary number of files?

Based on your description I would use a dict keyed on the path, whose values 
were a set of files. A set is a preprovided data type in Python. Look it up and 
use it.

There are other alternatives, but that would be a first attempt.

Cheers,
Cameron Simpson <cs at zip.com.au>


From dpalao.python at gmail.com  Wed Jun  3 05:24:59 2015
From: dpalao.python at gmail.com (David Palao)
Date: Wed, 3 Jun 2015 11:24:59 +0200
Subject: What sort of data structure to use?
In-Reply-To: <41302A7145AC054FA7A96CFD03835A0A0B97DECA@EX10MBX02.EU.NEC.COM>
References: <41302A7145AC054FA7A96CFD03835A0A0B97DECA@EX10MBX02.EU.NEC.COM>
Message-ID: <CAKUKWzmm5BYYsEMNSv-j=WPakNohKm28aDSmHv5SUbneFhJWug@mail.gmail.com>

2015-06-03 10:19 GMT+02:00 David Aldrich <David.Aldrich at emea.nec.com>:
> Hi
>
>
>
> I have written a Python utility that performs a certain activity on some
> predefined sets of files.  Here is the outline of what I have written:
>
>
>
> # File Set A
>
> pathA = ?pathA?
>
> fileListA = [?fileA1.txt?, ?fileA2.txt?]
>
>
>
> # File Set B
>
> pathB = ?pathB?
>
> fileListB = [?fileB1.txt?, ?fileB2.txt?, ?fileB3.txt?]
>
>
>
> myFunc1(pathA, fileListA)
>
> myFunc2(pathA, fileListA)
>
>
>
> myFunc1(pathB, fileListB)
>
> myFunc2(pathB, fileListB)
>
>
>
> I want to add more file sets, so I really want to add the sets to a list and
> iterate over the list, calling myFunc1 & myFunc2 for each item.
>
>
>
> My question is: what sort of data structure could I use to organise this,
> given that I want to associate a set of files with each path and that, for
> each set, there is an arbitrary number of files?
>
>
>
> Best regards
>
>
>
> David
>
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

And if you really want to link the path to the list of files, you
could define an object FileList (trivially) implementing the iterator
protocol.


From alister.nospam.ware at ntlworld.com  Wed Jun  3 05:38:36 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Wed, 3 Jun 2015 09:38:36 +0000 (UTC)
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net>
Message-ID: <mkmhus$1sr$1@speranza.aioe.org>

On Wed, 03 Jun 2015 10:41:44 +0300, Marko Rauhamaa wrote:

> Alain Ketterlin <alain at universite-de-strasbourg.fr.invalid>:
> 
>> Marko Rauhamaa <marko at pacujo.net> writes:
>>> Maybe close() will fail for ever.
>>
>> Your program has to deal with this, something is going wrong, it can't
>> just close and go on.
> 
> Here's the deal: the child process is saddled with file descriptors it
> never wanted in the first place. It can't decline them. Now you're
> saying it can't even dispose of them.
> 
No You cab dispose of them you just need to warn the user that the action 
did not complete correctly & there may be errors with the data.

Example What does your test editor do if you try to save a file back to a 
USB stick that has been removed? does it simply let you think the file 
has been successfully saved? i hope not.

> The reason this has been allowed to go on is that everybody just closes
> the file descriptors and ignores the possibility or repercussions of a
> failure. I haven't read about horror stories of this failing.
> 
> I readily admit this is very dirty, but since the API doesn't offer a
> clean alternative, there's nothing you can/should do about it.
> 
> 
> Marko





-- 
Never be afraid to tell the world who you are.
		-- Anonymous


From agn.91.91 at gmail.com  Wed Jun  3 05:47:16 2015
From: agn.91.91 at gmail.com (Amit Goutham)
Date: Wed, 3 Jun 2015 15:17:16 +0530
Subject: Calling Python Script from an SQL Proceudre
Message-ID: <CAJ5KB9jsYR0E+chPonkGAFkt1ihW0FSJ_3s0YXF_XRj7wmDMrw@mail.gmail.com>

Hi All,
I am trying to search on the Internet if i can call a Python Script from an
SQL Procedure.
All the information found on Internet is about connecting to a database
from Python through a Python script.But, i want the other way round.

Any Help will be appreciated

-- 
Thanks and Regards,
Amit Goutham
Ph:+91-8867801035
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150603/fe1e4165/attachment.html>

From bc at freeuk.com  Wed Jun  3 06:20:17 2015
From: bc at freeuk.com (BartC)
Date: Wed, 03 Jun 2015 11:20:17 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
Message-ID: <DxAbx.902026$_Q7.705597@fx28.am4>

On 03/06/2015 05:16, Eddilbert Macharia wrote:
> On Tuesday, June 2, 2015 at 2:27:31 PM UTC+3, Steven D'Aprano wrote:

>> Eddilbert, have you programmed in any other languages? It would help you
>> understand if you have.
>
> Sadly yes i have worked with java, and that is what is causing me so much grief.In java objects are instance of a class.pretty simple.

>> In Python 2:
>>
>> - everything is an object [the general concept]
>>
>> - some things, but not all things, are instances of the class
>>    called "object"
>>
>>
>> In Python 3:
>>
>> - everything is an object [the general concept]
>>
>> - everything is an instance of the class called "object"

> I think then in python object and instance of a class are entirely different things.
>
> in OOP and python - object is a representation of a real thing, or a concept .e.g a person,number and the concept of classes- which is the concept of create/representing other objects using a programming language to the machine.
>
> class - This is what is used to create/represent objects in the machine using a programming language
>
> class instance - This is the output of the classes this is a representation of an object.

I have a lot of trouble with this stuff too, as my ideas are decidedly 
old-fashioned. (Also I'm developing a language with some OO aspects 
without ever having used OO!)

But, it is mostly just jargon. If you go back to using 'variable' and 
'type', then it becomes a bit easier:

* A variable is an instance of some type.

And, that's pretty much it!

However, modern languages such as Python add a few more twists. So, if a 
static, compiled language, say, had these kinds of things:

- Module
- Record definition
- Enum type
- Label (for goto)
- Function
- Type

which are normally entities that only the compiler are concerned with, 
then these can now also be values that can be stored in variables! (One 
or two such languages might allow pointers to Functions or Labels, but 
that's about it.)

Exactly what it is about a Module, Record or Function that is stored in 
the variable is an implementation detail. The important thing is that 
you can use such as variable at runtime in the same way you might do at 
compile-time.

Python of course would subsume concepts such as Records or Enums into a 
class, as it seems to like to unify what it considers to be untidy, 
separate aspects of a language. Some of us however need things separated 
out again in order to understand them!

(This is a set of codes I'm using in a current compiler project, for a 
new language. Each denotes a separate kind of identifier in the input 
source:

	nullid = 0
	programid = 1
	moduleid = 2
	extmoduleid = 3
	classid = 4
	procid = 5
	staticid = 6
	constid = 7
	fieldid = 8
	genfieldid = 9
	enumid = 10
	paramid = 11
	frameid = 12
	varid = 13      # ?
	labelid = 14
	blockid = 15
	attribid = 16
	aliasid = 17

The language requires that each identifier is resolved at compile-time 
to one of the above. This makes it rather less dynamic than Python, 
where the bytecode compiler, as far as I know, only resolves a name as 
either global or local (and perhaps attribute, but I'm not an expert on 
its workings).

(The 'varid' code was for a unresolved top-level name, which I will 
probably remove as that was added when I'd intended to emulate Python 
more. But that was too much work (and too many speed-ups were no longer 
trivial). Some codes such as 'nullid' and 'programid' are only used 
internally.

'genfield' is a field (attribute) that can't be resolved, but the 
possibilities have been reduced to a small, finite set which  is 
resolved at load-time (in Python, the attribute could be anything, and 
you don't even know at runtime what it might be until you actually use 
the attribute.)

In this list, then 'static', 'frame', and 'param' denote what I'd call 
variables. A variable, at runtime, contains a value of a certain type 
(another list of codes). One of those types however is a Symbol: just a 
reference (a symbol table index) to a resolved name in the source code.

Just a different approach to things. But since the ultimate aim is to be 
able to write programs, not so different.)

-- 
Bartc


From kushal at locationd.net  Wed Jun  3 06:38:01 2015
From: kushal at locationd.net (Kushal Kumaran)
Date: Wed, 03 Jun 2015 16:08:01 +0530
Subject: Calling Python Script from an SQL Proceudre
In-Reply-To: <CAJ5KB9jsYR0E+chPonkGAFkt1ihW0FSJ_3s0YXF_XRj7wmDMrw@mail.gmail.com>
 (Amit Goutham's message of "Wed, 3 Jun 2015 15:17:16 +0530")
References: <CAJ5KB9jsYR0E+chPonkGAFkt1ihW0FSJ_3s0YXF_XRj7wmDMrw@mail.gmail.com>
Message-ID: <87iob56prq.fsf@carbon.locationd.net.>

Amit Goutham <agn.91.91 at gmail.com> writes:

> Hi All,
> I am trying to search on the Internet if i can call a Python Script from an
> SQL Procedure.
> All the information found on Internet is about connecting to a database
> from Python through a Python script.But, i want the other way round.
>
> Any Help will be appreciated
>

Which database are you using?

This is how to do it in postgresql, for example:
http://www.postgresql.org/docs/current/static/plpython.html

-- 
regards,
kushal


From rosuav at gmail.com  Wed Jun  3 06:38:29 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 3 Jun 2015 20:38:29 +1000
Subject: Everything is an object in python - object class and type class
In-Reply-To: <DxAbx.902026$_Q7.705597@fx28.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
 <DxAbx.902026$_Q7.705597@fx28.am4>
Message-ID: <CAPTjJmqNHFeNSnj9P1mvcFLXvNUiJy2OHMxpG4GPPXYT9j9-Dw@mail.gmail.com>

On Wed, Jun 3, 2015 at 8:20 PM, BartC <bc at freeuk.com> wrote:
> I have a lot of trouble with this stuff too, as my ideas are decidedly
> old-fashioned. (Also I'm developing a language with some OO aspects without
> ever having used OO!)
>
> But, it is mostly just jargon. If you go back to using 'variable' and
> 'type', then it becomes a bit easier:
>
> * A variable is an instance of some type.
>
> And, that's pretty much it!

If I have a list called "stuff" with three elements in it, is
"stuff[1]" a variable? What if I return that list from a function
called get_stuff()? Is get_stuff()[1] a variable? Because in Python,
get_stuff()[1] is certainly going to be an object.

ChrisA


From __peter__ at web.de  Wed Jun  3 06:59:13 2015
From: __peter__ at web.de (Peter Otten)
Date: Wed, 03 Jun 2015 12:59:13 +0200
Subject: What sort of data structure to use?
References: <41302A7145AC054FA7A96CFD03835A0A0B97DECA@EX10MBX02.EU.NEC.COM>
Message-ID: <mkmmm3$ott$1@ger.gmane.org>

David Aldrich wrote:

> Hi
> 
> I have written a Python utility that performs a certain activity on some
> predefined sets of files.  Here is the outline of what I have written:
> 
> # File Set A
> pathA = 'pathA'
> fileListA = ['fileA1.txt', 'fileA2.txt']
> 
> # File Set B
> pathB = 'pathB'
> fileListB = ['fileB1.txt', 'fileB2.txt', 'fileB3.txt']
> 
> myFunc1(pathA, fileListA)
> myFunc2(pathA, fileListA)
> 
> myFunc1(pathB, fileListB)
> myFunc2(pathB, fileListB)
> 
> I want to add more file sets, so I really want to add the sets to a list
> and iterate over the list, calling myFunc1 & myFunc2 for each item.
> 
> My question is: what sort of data structure could I use to organise this,
> given that I want to associate a set of files with each path and that, for
> each set, there is an arbitrary number of files?

I'd start simple and put (path, files) pairs into a list:

path_files_pairs = [
    ("pathA", ["fileA1.txt", "fileA2.txt", ...]),
    ("pathB", ["fileB1.txt", ...]),
]

for path, files in path_files_pairs:
    func1(path, files)
    func2(path, files)

You can always add complications later:

import glob
import os

class VirtualFileset:
    def __init__(self, folder, pattern):
        self.folder = folder
        self.pattern = pattern
    def __iter__(self):
        yield self.folder
        yield glob.glob(os.path.join(self.folder, self.pattern))

path_files_pairs = [
    ("pathA", ["fileA1.txt", "fileA2.txt", ...]),
    ("pathB", ["fileB1.txt", ...]),
    VirtualFileset("pathC", "*.py"), # all python files in directory "pathC"
]

for path, files in path_files_pairs:
    func1(path, files)
    func2(path, files)




From David.Aldrich at EMEA.NEC.COM  Wed Jun  3 07:07:50 2015
From: David.Aldrich at EMEA.NEC.COM (David Aldrich)
Date: Wed, 3 Jun 2015 11:07:50 +0000
Subject: What sort of data structure to use?
In-Reply-To: <mkmmm3$ott$1@ger.gmane.org>
References: <41302A7145AC054FA7A96CFD03835A0A0B97DECA@EX10MBX02.EU.NEC.COM>
 <mkmmm3$ott$1@ger.gmane.org>
Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B97E0C5@EX10MBX02.EU.NEC.COM>

Thanks very much for all the answers given to my question. They help me to think about the problem pythonically.

Best regards

David

> -----Original Message-----
> From: Python-list [mailto:python-list-
> bounces+david.aldrich=emea.nec.com at python.org] On Behalf Of Peter
> Otten
> Sent: 03 June 2015 11:59
> To: python-list at python.org
> Subject: Re: What sort of data structure to use?
> 
> David Aldrich wrote:
> 
> > Hi
> >
> > I have written a Python utility that performs a certain activity on
> > some predefined sets of files.  Here is the outline of what I have written:
> >
> > # File Set A
> > pathA = 'pathA'
> > fileListA = ['fileA1.txt', 'fileA2.txt']
> >
> > # File Set B
> > pathB = 'pathB'
> > fileListB = ['fileB1.txt', 'fileB2.txt', 'fileB3.txt']
> >
> > myFunc1(pathA, fileListA)
> > myFunc2(pathA, fileListA)
> >
> > myFunc1(pathB, fileListB)
> > myFunc2(pathB, fileListB)
> >
> > I want to add more file sets, so I really want to add the sets to a
> > list and iterate over the list, calling myFunc1 & myFunc2 for each item.
> >
> > My question is: what sort of data structure could I use to organise
> > this, given that I want to associate a set of files with each path and
> > that, for each set, there is an arbitrary number of files?
> 
> I'd start simple and put (path, files) pairs into a list:
> 
> path_files_pairs = [
>     ("pathA", ["fileA1.txt", "fileA2.txt", ...]),
>     ("pathB", ["fileB1.txt", ...]),
> ]
> 
> for path, files in path_files_pairs:
>     func1(path, files)
>     func2(path, files)
> 
> You can always add complications later:
> 
> import glob
> import os
> 
> class VirtualFileset:
>     def __init__(self, folder, pattern):
>         self.folder = folder
>         self.pattern = pattern
>     def __iter__(self):
>         yield self.folder
>         yield glob.glob(os.path.join(self.folder, self.pattern))
> 
> path_files_pairs = [
>     ("pathA", ["fileA1.txt", "fileA2.txt", ...]),
>     ("pathB", ["fileB1.txt", ...]),
>     VirtualFileset("pathC", "*.py"), # all python files in directory "pathC"
> ]
> 
> for path, files in path_files_pairs:
>     func1(path, files)
>     func2(path, files)
> 
> 
> --
> https://mail.python.org/mailman/listinfo/python-list
> 
> 
>  Click
> https://www.mailcontrol.com/sr/51ZWmSF1P47GX2PQPOmvUmaGI8Tu3yGr
> Vrr5Tv1xM3UP2MNyoKSTyt0rIsjE4onM5MUvmWbo6fT3KeH4!zzvzA==  to
> report this email as spam.


From bc at freeuk.com  Wed Jun  3 07:08:26 2015
From: bc at freeuk.com (BartC)
Date: Wed, 03 Jun 2015 12:08:26 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <mailman.99.1433328294.13271.python-list@python.org>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
 <DxAbx.902026$_Q7.705597@fx28.am4>
 <mailman.99.1433328294.13271.python-list@python.org>
Message-ID: <NeBbx.946747$wk1.662297@fx19.am4>

On 03/06/2015 11:38, Chris Angelico wrote:
> On Wed, Jun 3, 2015 at 8:20 PM, BartC <bc at freeuk.com> wrote:
>> I have a lot of trouble with this stuff too, as my ideas are decidedly
>> old-fashioned. (Also I'm developing a language with some OO aspects without
>> ever having used OO!)
>>
>> But, it is mostly just jargon. If you go back to using 'variable' and
>> 'type', then it becomes a bit easier:
>>
>> * A variable is an instance of some type.
>>
>> And, that's pretty much it!
>
> If I have a list called "stuff" with three elements in it, is
> "stuff[1]" a variable? What if I return that list from a function
> called get_stuff()? Is get_stuff()[1] a variable? Because in Python,
> get_stuff()[1] is certainly going to be an object.

Come on, we're trying to keep this simple.

To 'variable' and 'type', you might need to add 'value' to make it more 
complete. An old-fashioned program will be moving values around and 
constructing new ones. Some of them will be loaded from variables, and 
some might end up being stored in variables.

(With the obligatory twist in Python that variable names are not 
directly attached to their values, but via a 'string'. I can introduce a 
new term for what /is/ actually stored /with/ the variable, as it's got 
to be something unless Python works by magic, but I don't want to do that.)

You might call such a value an 'object'. The trouble is, Python also 
uses 'object' to mean the base class of all classes. And it seems to use 
it in a more abstract sense as well to mean pretty much everything. 
While other languages, such as C, use object in yet another way.

Which is where the term breaks down as it no longer helps in 
understanding. It's become meaningless.

-- 
Bartc




From bc at freeuk.com  Wed Jun  3 07:13:20 2015
From: bc at freeuk.com (BartC)
Date: Wed, 03 Jun 2015 12:13:20 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <DxAbx.902026$_Q7.705597@fx28.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
 <DxAbx.902026$_Q7.705597@fx28.am4>
Message-ID: <njBbx.721610$G53.453980@fx13.am4>

On 03/06/2015 11:20, BartC wrote:

> 'genfield' is a field (attribute) that can't be resolved, but the
> possibilities have been reduced to a small, finite set which  is
> resolved at load-time (in Python, the attribute could be anything, and
> you don't even know at runtime what it might be until you actually use
> the attribute.)

(That's not right. What I call 'genfield' is also resolved at runtime at 
the point of use. But the possibilities are very small (often just two), 
and must be an attribute the compiler knew about. Python-like open-ended 
attribute names have a separate mechanism, although I haven't 
implemented it yet...)

-- 
Bartc


From steve at pearwood.info  Wed Jun  3 08:07:47 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 03 Jun 2015 22:07:47 +1000
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
Message-ID: <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>

On Wed, 3 Jun 2015 07:38 pm, alister wrote:

> On Wed, 03 Jun 2015 10:41:44 +0300, Marko Rauhamaa wrote:
[...]
>> Here's the deal: the child process is saddled with file descriptors it
>> never wanted in the first place. It can't decline them. Now you're
>> saying it can't even dispose of them.
>> 
> No You cab dispose of them you just need to warn the user that the action
> did not complete correctly & there may be errors with the data.

*What* action? Pay closer attention to what Marko wrote:

"the child process ..."

How does the child process know what action didn't complete? What error
message are you going to display to the user?

"Error when closing file descriptor 123456"

What action do you think the user can take on seeing this error message?

 
> Example What does your test editor do if you try to save a file back to a
> USB stick that has been removed? does it simply let you think the file
> has been successfully saved? i hope not.


Correct me if I'm wrong, but don't you get an error when you try to *open* a
file on a missing USB stick?

There are at least three opportunities to get an error when writing a file:

* opening the file;
* writing to the file;
* closing the file.

It's not clear what conditions will lead to the first two succeeding but the
third failing, or what to do if you get such an error. If you don't know
what caused close() to fail, then you can't do anything about it, and if
you can't do anything about it, what's the point in reporting the error to
the user who will just get frustrated and nervous and not be able to do
anything about it either?

I'm sure that there are circumstances where an error when closing a file is
a clear and obvious fault that should be reported. But in the scenario that
Marko is describing, I'm not so sure that is the case.



-- 
Steven



From marko at pacujo.net  Wed Jun  3 08:08:51 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 15:08:51 +0300
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
 <DxAbx.902026$_Q7.705597@fx28.am4>
 <mailman.99.1433328294.13271.python-list@python.org>
 <NeBbx.946747$wk1.662297@fx19.am4>
Message-ID: <87fv6956zw.fsf@elektro.pacujo.net>

BartC <bc at freeuk.com>:

> To 'variable' and 'type', you might need to add 'value' to make it more
> complete.

'Value' and 'object' are indeed synonymous as long as you keep in mind
that:

    >>> -12 == -12
    True
    >>> -12 is -12
    False

IOW, the literal expression -12 happens to construct a fresh
value/object each time CPython parses it.

> You might call such a value an 'object'. The trouble is, Python also
> uses 'object' to mean the base class of all classes.

'object' is the class of all objects just like 'int' is the class of all
integers. So no trouble at all.

> And it seems to use it in a more abstract sense as well to mean pretty
> much everything. While other languages, such as C, use object in yet
> another way.
>
> Which is where the term breaks down as it no longer helps in
> understanding. It's become meaningless.

Correct! Abstractions are generalizations of specifics. You can't
understand the generalizations before being bored with the specifics
first.


Marko


From marco.buttu at gmail.com  Wed Jun  3 08:10:58 2015
From: marco.buttu at gmail.com (Marco Buttu)
Date: Wed, 03 Jun 2015 14:10:58 +0200
Subject: Everything is an object in python - object class and type class
In-Reply-To: <NeBbx.946747$wk1.662297@fx19.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
 <DxAbx.902026$_Q7.705597@fx28.am4>
 <mailman.99.1433328294.13271.python-list@python.org>
 <NeBbx.946747$wk1.662297@fx19.am4>
Message-ID: <556EEED2.5040402@oa-cagliari.inaf.it>

On 03/06/2015 13:08, BartC wrote:

> Come on, we're trying to keep this simple.

If we really want to keep it simple, we can take this starting point:

http://en.wikipedia.org/wiki/Object_(computer_science)

If we agree with the Wikipedia definition:

``In the class-based object-oriented programming paradigm, "object" 
refers to a particular instance of a class...``

than we can say in Python everything is an object, because everything is 
an instance of a class. And  of course, object is synonimous for instance:

 >>> (1, 2, 3).foo
Traceback (most recent call last):
    ...
AttributeError: 'tuple' object has no attribute 'foo'

-- 
Marco Buttu

INAF-Osservatorio Astronomico di Cagliari
Via della Scienza n. 5, 09047 Selargius (CA)
Phone: 070 711 80 217
Email: mbuttu at oa-cagliari.inaf.it



From alister.nospam.ware at ntlworld.com  Wed Jun  3 08:18:37 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Wed, 3 Jun 2015 12:18:37 +0000 (UTC)
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mkmrat$s26$1@speranza.aioe.org>

On Wed, 03 Jun 2015 22:07:47 +1000, Steven D'Aprano wrote:

> On Wed, 3 Jun 2015 07:38 pm, alister wrote:
> 
>> On Wed, 03 Jun 2015 10:41:44 +0300, Marko Rauhamaa wrote:
> [...]
>>> Here's the deal: the child process is saddled with file descriptors it
>>> never wanted in the first place. It can't decline them. Now you're
>>> saying it can't even dispose of them.
>>> 
>> No You cab dispose of them you just need to warn the user that the
>> action did not complete correctly & there may be errors with the data.
> 
> *What* action? Pay closer attention to what Marko wrote:
> 
> "the child process ..."
> 
> How does the child process know what action didn't complete? What error
> message are you going to display to the user?
> 
> "Error when closing file descriptor 123456"
> 
> What action do you think the user can take on seeing this error message?
> 
> 
>> Example What does your test editor do if you try to save a file back to
>> a USB stick that has been removed? does it simply let you think the
>> file has been successfully saved? i hope not.
> 
> 
> Correct me if I'm wrong, but don't you get an error when you try to
> *open* a file on a missing USB stick?
> 
> There are at least three opportunities to get an error when writing a
> file:
> 
> * opening the file;
> * writing to the file;
> * closing the file.
> 
> It's not clear what conditions will lead to the first two succeeding but
> the third failing, or what to do if you get such an error. If you don't
> know what caused close() to fail,

open file for appending data
write some data
remove mem stick
now try to close file

> then you can't do anything about it,
Correct you cant do anything about it.
> and if you can't do anything about it, what's the point in reporting the
> error to the user who will just get frustrated and nervous and not be
> able to do anything about it either?

Because they need to know they may have just lost important data whilst 
they still have a chance to do it again. finding out 6 months later that 
they had screwed up saving data on a multi-million pound account will not 
be good & quite rightly considered to be due to a bug in your software.


> 
> I'm sure that there are circumstances where an error when closing a file
> is a clear and obvious fault that should be reported. But in the
> scenario that Marko is describing, I'm not so sure that is the case.

from the scenario Marco is reporting I get the impression that he is 
having to interact with a system that is fundamentally flawed from the 
ground up.



-- 
Standards are crucial.  And the best thing about standards is: there are
so ____many to choose from!


From rosuav at gmail.com  Wed Jun  3 08:19:04 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 3 Jun 2015 22:19:04 +1000
Subject: Everything is an object in python - object class and type class
In-Reply-To: <NeBbx.946747$wk1.662297@fx19.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
 <DxAbx.902026$_Q7.705597@fx28.am4>
 <mailman.99.1433328294.13271.python-list@python.org>
 <NeBbx.946747$wk1.662297@fx19.am4>
Message-ID: <CAPTjJmqn51wpYptSr1C2pFyjHHiqzsBggC=3p+=U5h5Bf+yBEg@mail.gmail.com>

On Wed, Jun 3, 2015 at 9:08 PM, BartC <bc at freeuk.com> wrote:
> On 03/06/2015 11:38, Chris Angelico wrote:
>>
>> On Wed, Jun 3, 2015 at 8:20 PM, BartC <bc at freeuk.com> wrote:
>>>
>>> I have a lot of trouble with this stuff too, as my ideas are decidedly
>>> old-fashioned. (Also I'm developing a language with some OO aspects
>>> without
>>> ever having used OO!)
>>>
>>> But, it is mostly just jargon. If you go back to using 'variable' and
>>> 'type', then it becomes a bit easier:
>>>
>>> * A variable is an instance of some type.
>>>
>>> And, that's pretty much it!
>>
>>
>> If I have a list called "stuff" with three elements in it, is
>> "stuff[1]" a variable? What if I return that list from a function
>> called get_stuff()? Is get_stuff()[1] a variable? Because in Python,
>> get_stuff()[1] is certainly going to be an object.
>
>
> Come on, we're trying to keep this simple.
>
> To 'variable' and 'type', you might need to add 'value' to make it more
> complete. An old-fashioned program will be moving values around and
> constructing new ones. Some of them will be loaded from variables, and some
> might end up being stored in variables.
>
> (With the obligatory twist in Python that variable names are not directly
> attached to their values, but via a 'string'. I can introduce a new term for
> what /is/ actually stored /with/ the variable, as it's got to be something
> unless Python works by magic, but I don't want to do that.)
>
> You might call such a value an 'object'. The trouble is, Python also uses
> 'object' to mean the base class of all classes. And it seems to use it in a
> more abstract sense as well to mean pretty much everything. While other
> languages, such as C, use object in yet another way.
>
> Which is where the term breaks down as it no longer helps in understanding.
> It's become meaningless.

Right. My point is that it starts out meaningless, because "variable"
doesn't help with anything. "Value" might help, in which case what
you're saying is that every value in Python is an object (in contrast
to Java, where some aren't). But not all objects are referenced by
names, so "variable" isn't very helpful here.

ChrisA


From random832 at fastmail.us  Wed Jun  3 08:21:09 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 03 Jun 2015 08:21:09 -0400
Subject: fork/exec & close file descriptors
In-Reply-To: <87h9qpl10o.fsf@universite-de-strasbourg.fr.invalid>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <mailman.84.1433289254.13271.python-list@python.org>
 <87h9qpl10o.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <1433334069.1480416.285673681.2EA03CD8@webmail.messagingengine.com>

On Wed, Jun 3, 2015, at 03:11, Alain Ketterlin wrote:
> Thank you, I know this. What I mean is: what are the reasons that you
> cannot access your file descriptors one by one? To me closing a range of
> descriptors has absolutely no meaning, simply because ranges have no
> meaning for file descriptors (they're not ordered in any way). What if
> some library uses its own descriptors that happen to lie in your
> "range"? Etc.

The context in which this is useful is that you've just forked, and
you're about to exec. "Some library" isn't going to ever get back
control within the current process. Generally the range of file
descriptors you want to close is (e.g.) 3-Infinity, after you've already
got 0 1 and 2 pointing to where you want them (whatever redirected file
or pipe).


From marko at pacujo.net  Wed Jun  3 08:25:52 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 15:25:52 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <878uc1567j.fsf@elektro.pacujo.net>

Steven D'Aprano <steve at pearwood.info>:

> How does the child process know what action didn't complete? What
> error message are you going to display to the user?
>
> "Error when closing file descriptor 123456"
>
> What action do you think the user can take on seeing this error
> message?

Besides, even if you wanted to display that error message, you might not
be able to since maybe you have unknowingly closed fd 123433, which is
the hidden connection to the display server. So the program might just
crash in the attempt. Or the hidden fd is still open but won't tolerate
two processes sharing the connection.

You definitely must not use sys.stderr from the child process nor are
you allowed to exit through an exception but must call os._exit()
instead.

The child process is highly limited in what it can do between os.fork()
and os.execve(). To put it simply, it must not call any library
functions except system calls.

You might be able to reserve a special pipe to convey such errors to the
parent. Only don't rely on getting an EOF as closing the pipe might
fail...


Marko


From marko at pacujo.net  Wed Jun  3 08:27:19 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 15:27:19 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mkmrat$s26$1@speranza.aioe.org>
Message-ID: <874mmp5654.fsf@elektro.pacujo.net>

alister <alister.nospam.ware at ntlworld.com>:

> from the scenario Marco is reporting I get the impression that he is
> having to interact with a system that is fundamentally flawed from the
> ground up.

Well, yes. It's called linux, but it's not all bad. I just think that
man page was being sanctimonious.


Marko


From Cecil at decebal.nl  Wed Jun  3 08:27:52 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 03 Jun 2015 14:27:52 +0200
Subject: Retrying to send message
Message-ID: <87egltht87.fsf@Equus.decebal.nl>

I am using libturpial to post messages on Twitter. Sometimes I get a
libturpial.exceptions.ServiceOverCapacity.

It is a good idea to try it again then. For this I wrote the following
function:
    def send_message(account_id, message, max_tries, terminate_program):
        error_msg   = 'Something went wrong with: ' + message
        not_send    = True
        tries       = 0
        while not_send:
            try:
                Core().update_status(account_id, message)
            except libturpial.exceptions.ServiceOverCapacity:
                tries += 1
                print('Tried to send it {0} times'.format(tries))
                if tries >= max_tries:
                    terminate_program(error_msg)
                time.sleep(60)
            except:
                terminate_program(error_msg)
            else:
                not_send = False

Is this a reasonable way to implement this, or is another way better?
Well, maybe the timeout should be a parameter also. ;-)


Also the documentation of time.sleep says:
    The actual suspension time may be less than that requested because
    any caught signal will terminate the sleep() following execution
    of that signal?s catching routine.

My program does not much else as sending the message. So I think it is
not necessary to cover for this possibility. Are I assuming to much?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From random832 at fastmail.us  Wed Jun  3 08:54:10 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 03 Jun 2015 08:54:10 -0400
Subject: fork/exec & close file descriptors
In-Reply-To: <878uc1567j.fsf@elektro.pacujo.net>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <878uc1567j.fsf@elektro.pacujo.net>
Message-ID: <1433336050.1487936.285702649.20D7BAD4@webmail.messagingengine.com>

On Wed, Jun 3, 2015, at 08:25, Marko Rauhamaa wrote:
> Steven D'Aprano <steve at pearwood.info>:
> 
> > How does the child process know what action didn't complete? What
> > error message are you going to display to the user?
> 
> You definitely must not use sys.stderr from the child process nor are
> you allowed to exit through an exception but must call os._exit()
> instead.

Why does the child process need to report the error at all? The parent
process will find out naturally when *it* tries to close the same file
descriptor.


From alister.nospam.ware at ntlworld.com  Wed Jun  3 09:05:09 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Wed, 3 Jun 2015 13:05:09 +0000 (UTC)
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mkmrat$s26$1@speranza.aioe.org> <874mmp5654.fsf@elektro.pacujo.net>
Message-ID: <mkmu25$3p9$1@speranza.aioe.org>

On Wed, 03 Jun 2015 15:27:19 +0300, Marko Rauhamaa wrote:

> alister <alister.nospam.ware at ntlworld.com>:
> 
>> from the scenario Marco is reporting I get the impression that he is
>> having to interact with a system that is fundamentally flawed from the
>> ground up.
> 
> Well, yes. It's called linux, but it's not all bad. I just think that
> man page was being sanctimonious.
> 
> 
> Marko

I meant the program that is supplying your app with file handles willy- 
nilly without caring what happens to them



-- 
"...a most excellent barbarian ... Genghis Kahn!"
-- _Bill And Ted's Excellent Adventure_


From kavithabhaskaran2013 at gmail.com  Wed Jun  3 09:07:16 2015
From: kavithabhaskaran2013 at gmail.com (KR)
Date: Wed, 3 Jun 2015 06:07:16 -0700 (PDT)
Subject: Issuing commands using "exec_command()" of paramiko AND also
 sending commands together
In-Reply-To: <mailman.92.1433315397.13271.python-list@python.org>
References: <3603e97e-686a-47de-aff4-cabfcb930064@googlegroups.com>
 <mailman.92.1433315397.13271.python-list@python.org>
Message-ID: <49caa146-b407-4046-813d-05e02d929ef6@googlegroups.com>


Hi Sreenathan Nair:
import os, sys, 
import connectlibs as ssh
s = ssh.connect("xxx.xx.xx.xxx", "Admin", "Admin")
channel = s.invoke_shell()
channel.send("net use F: \\\\xyz.xy.xc.xa\\dir\n")
>>>32
channel.send("net use\n")
>>>7
channel.recv(500)
'Last login: Tue Jun  2 23:52:29 2015 from xxx.xx.xx.xx\r\r\n\x1b]0;~\x07\r\r\n\x1b[32mAdmin at WIN \x1b[33m~\x1b[0m\r\r\n$ net use F: \\\\xyz.xy.xc.xa\\dir\r\nSystem error 67 has occurred.\r\r\n\r\r\nThe network name cannot be found.\r\r\n\r\r\n\x1b]0;~\x07\r\r\n\x1b[32mAdmin at WIN \x1b[33m~\x1b[0m\r\r\n$ net use'
>>> 

Above is the what I have for code which is what I am encountering on my setup.

A. I need to be able to parse the o/p of 2 commands 1. "net use F: \\\\xyz.xy.xc.xa\\dir" and 2. "net use". 

B. When I use chan.recv(x) -> with certain 'x' bytes. What if I dont know what kind of a response I am expecting - I mean its not a usual response - if I give "x" bytes whereas in reality "x" could be 10000 bytes more than the ACTUAL number of bytes in which case Python hangs.

What do people use in such cases.

On Wednesday, June 3, 2015 at 12:10:08 AM UTC-7, Sreenathan Nair wrote:
> Hi,
> 
> 
> Could you be more specific about your problem? Perhaps an example of something similar to what you're trying to do would be helpful.
> 
> 
> Usually the process is to instantiate paramiko.SSHCLIENT, use the connect() method with desired parameters and execute commands using the exec_command(). If you'd like to process the output of the command execution, then you would store the result of exec_command() into three variables (it return a 3-tuple of Channel objects).
> 
> 
> i.e com_stdin, com_stdout, com_stderr = my_ssh_client_instance.exec_command("<command>")
> 
> 
> The instance of SSHCLIENT is live until the close() method is called. Meaning subsequent commands can be executed the same way.
> 
> 
> 
> 
> On Wed, Jun 3, 2015 at 8:07 AM, Pythonista?<kukki.k... at gmail.com>?wrote:
> Using paramiko's exec_command(), i would like to send a command, process its output and do it for several other commands. I notice that its not quick enough or something like that.
> 
> How would I handle that scenario AND also providing multiple commands together (there is 1 post on stackoverflow for issuing multiple commands but I am not sure if someone has tried it. It didnt work for me!
> 
> Thanks!
> --
> https://mail.python.org/mailman/listinfo/python-list



From marko at pacujo.net  Wed Jun  3 09:08:44 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 16:08:44 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <878uc1567j.fsf@elektro.pacujo.net>
 <mailman.104.1433336055.13271.python-list@python.org>
Message-ID: <87wpzl3pnn.fsf@elektro.pacujo.net>

random832 at fastmail.us:

> Why does the child process need to report the error at all? The parent
> process will find out naturally when *it* tries to close the same file
> descriptor.

That's not how it goes.

File descriptors are reference counted in the Linux kernel. Closes are
no-ops except for the last one that brings the reference count to zero.

If the parent should close the file before the child, no error is
returned to the parent.


Marko


From alain at universite-de-strasbourg.fr.invalid  Wed Jun  3 09:16:21 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Wed, 03 Jun 2015 15:16:21 +0200
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <mailman.84.1433289254.13271.python-list@python.org>
 <87h9qpl10o.fsf@universite-de-strasbourg.fr.invalid>
 <mailman.103.1433334072.13271.python-list@python.org>
Message-ID: <876175kk4a.fsf@universite-de-strasbourg.fr.invalid>

random832 at fastmail.us writes:

> On Wed, Jun 3, 2015, at 03:11, Alain Ketterlin wrote:
>> Thank you, I know this. What I mean is: what are the reasons that you
>> cannot access your file descriptors one by one? To me closing a range of
>> descriptors has absolutely no meaning, simply because ranges have no
>> meaning for file descriptors (they're not ordered in any way). What if
>> some library uses its own descriptors that happen to lie in your
>> "range"? Etc.
>
> The context in which this is useful is that you've just forked, and
> you're about to exec. "Some library" isn't going to ever get back
> control within the current process.

Any decent library will (on linux) use close-on-exec, or even have
fini/dtor functions to clean up. Any that does not is buggy. But of
course, if you shoot in their feet...

> Generally the range of file descriptors you want to close is (e.g.)
> 3-Infinity, after you've already got 0 1 and 2 pointing to where you
> want them (whatever redirected file or pipe).

Closing 3-... is meaningless, probably useless, and potentially harmful.

-- Alain.


From edd.cowan at gmail.com  Wed Jun  3 09:17:16 2015
From: edd.cowan at gmail.com (Eddilbert Macharia)
Date: Wed, 3 Jun 2015 06:17:16 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
Message-ID: <54921414-de65-4af3-bd60-7196a42ce85b@googlegroups.com>

Hello guys, 

After i read http://python-history.blogspot.com/2009/02/first-class-everything.html .

where Guidos says "One of my goals for Python was to make it so that all objects were "first class." By this, I meant that I wanted all objects that could be named in the language (e.g., integers, strings, functions, classes, modules, methods, etc.) to have equal status. That is, they can be assigned to variables, placed in lists, stored in dictionaries, passed as arguments, and so forth."

I think i was right to say:

*** in python object and instance of a class are entirely different things.

in OOP and python - object is a representation of a real thing, or a concept .e.g a person,number and the concept of classes- which is the concept of create/representing other objects using a programming language to the machine.

class - This is what is used to create/represent objects in the machine using a programming language

class instance - This is the output of the classes this is a representation of an object.


From random832 at fastmail.us  Wed Jun  3 09:21:19 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 03 Jun 2015 09:21:19 -0400
Subject: fork/exec & close file descriptors
In-Reply-To: <87wpzl3pnn.fsf@elektro.pacujo.net>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <878uc1567j.fsf@elektro.pacujo.net>
 <mailman.104.1433336055.13271.python-list@python.org>
 <87wpzl3pnn.fsf@elektro.pacujo.net>
Message-ID: <1433337679.1493926.285725793.69D6B33F@webmail.messagingengine.com>

On Wed, Jun 3, 2015, at 09:08, Marko Rauhamaa wrote:
> random832 at fastmail.us:
> 
> > Why does the child process need to report the error at all? The parent
> > process will find out naturally when *it* tries to close the same file
> > descriptor.
> 
> That's not how it goes.
> 
> File descriptors are reference counted in the Linux kernel. Closes are
> no-ops except for the last one that brings the reference count to zero.
> 
> If the parent should close the file before the child, no error is
> returned to the parent.

Why would the parent close it before the child? Your scenario doesn't
seem to have anything to do with how people actually use subprocesses.


From rosuav at gmail.com  Wed Jun  3 09:29:02 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 3 Jun 2015 23:29:02 +1000
Subject: Retrying to send message
In-Reply-To: <87egltht87.fsf@Equus.decebal.nl>
References: <87egltht87.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmrnmgexoYUZJiS-Y06MDdNa66XJrpc2draYR2hJ9Oio5w@mail.gmail.com>

On Wed, Jun 3, 2015 at 10:27 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>     def send_message(account_id, message, max_tries, terminate_program):
>         error_msg   = 'Something went wrong with: ' + message
>         not_send    = True
>         tries       = 0
>         while not_send:
>             try:
>                 Core().update_status(account_id, message)
>             except libturpial.exceptions.ServiceOverCapacity:
>                 tries += 1
>                 print('Tried to send it {0} times'.format(tries))
>                 if tries >= max_tries:
>                     terminate_program(error_msg)
>                 time.sleep(60)
>             except:
>                 terminate_program(error_msg)
>             else:
>                 not_send = False
>
> Is this a reasonable way to implement this, or is another way better?
> Well, maybe the timeout should be a parameter also. ;-)

I'd skip the not_send flag and do the logic thusly:

while True:
    try:
        update_status as above
        break
    except ServiceOverCapacity:
        as above

And I'd also skip the bare except clause. If you get any sort of
exception, whether it's a bug, a failure from libturpial, a network
error, or anything else, your code will just terminate with a bland
and useless message. Much better to simply let the exception bubble
up.

> Also the documentation of time.sleep says:
>     The actual suspension time may be less than that requested because
>     any caught signal will terminate the sleep() following execution
>     of that signal?s catching routine.
>
> My program does not much else as sending the message. So I think it is
> not necessary to cover for this possibility. Are I assuming to much?

I wouldn't worry too much about a signal cutting your sleep short; it
doesn't look to me as if "exactly sixty seconds" is all that crucial
here. If your program sleeps for 42.645 seconds and then gets woken up
by a signal, and you retry a bit sooner than you otherwise would have,
is it going to break anything? If not, just ignore that possibility.

ChrisA


From rosuav at gmail.com  Wed Jun  3 09:32:23 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 3 Jun 2015 23:32:23 +1000
Subject: fork/exec & close file descriptors
In-Reply-To: <1433337679.1493926.285725793.69D6B33F@webmail.messagingengine.com>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net>
 <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <878uc1567j.fsf@elektro.pacujo.net>
 <mailman.104.1433336055.13271.python-list@python.org>
 <87wpzl3pnn.fsf@elektro.pacujo.net>
 <1433337679.1493926.285725793.69D6B33F@webmail.messagingengine.com>
Message-ID: <CAPTjJmoTzNnDuoqVfbYOcRUtw3GDDqRVDaJxUa4F8xUFefk6tg@mail.gmail.com>

On Wed, Jun 3, 2015 at 11:21 PM,  <random832 at fastmail.us> wrote:
> On Wed, Jun 3, 2015, at 09:08, Marko Rauhamaa wrote:
>> random832 at fastmail.us:
>>
>> > Why does the child process need to report the error at all? The parent
>> > process will find out naturally when *it* tries to close the same file
>> > descriptor.
>>
>> That's not how it goes.
>>
>> File descriptors are reference counted in the Linux kernel. Closes are
>> no-ops except for the last one that brings the reference count to zero.
>>
>> If the parent should close the file before the child, no error is
>> returned to the parent.
>
> Why would the parent close it before the child? Your scenario doesn't
> seem to have anything to do with how people actually use subprocesses.

Write an editor that opens a file and holds it open until the user's
done with it. Have something that lets you shell out for whatever
reason. Then trigger the shell-out, and instantly SIGSTOP the child
process, before it does its work - or just have a really heavily
loaded system, so it can't get a time slice. Now close the file in the
UI, which results in the file being closed in the parent. Right, now
let the child run... and there it goes, closing the file.

Mightn't be a common situation (given that the amount of code executed
between forking and closing FDs is not going to be much, and isn't
going to involve reading from disk or anything), but it can certainly
happen.

ChrisA


From marko at pacujo.net  Wed Jun  3 09:33:49 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 16:33:49 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <878uc1567j.fsf@elektro.pacujo.net>
 <mailman.104.1433336055.13271.python-list@python.org>
 <87wpzl3pnn.fsf@elektro.pacujo.net>
Message-ID: <87oakw532a.fsf@elektro.pacujo.net>

Marko Rauhamaa <marko at pacujo.net>:

> random832 at fastmail.us:
>
>> Why does the child process need to report the error at all? The parent
>> process will find out naturally when *it* tries to close the same file
>> descriptor.
>
> That's not how it goes.
>
> File descriptors are reference counted in the Linux kernel. Closes are
> no-ops except for the last one that brings the reference count to zero.
>
> If the parent should close the file before the child, no error is
> returned to the parent.

First of all, it's not the descriptors that are refcounted -- it's the
files referred to by the descriptors.

However, I was also wrong about close() being a no-operation. Here's the
relevant kernel source code snippet:

========================================================================
int __close_fd(struct files_struct *files, unsigned fd)
{
	struct file *file;
	struct fdtable *fdt;

	spin_lock(&files->file_lock);
	fdt = files_fdtable(files);
	if (fd >= fdt->max_fds)
		goto out_unlock;
	file = fdt->fd[fd];
	if (!file)
		goto out_unlock;
	rcu_assign_pointer(fdt->fd[fd], NULL);
	__clear_close_on_exec(fd, fdt);
	__put_unused_fd(files, fd);
	spin_unlock(&files->file_lock);
	return filp_close(file, files);

out_unlock:
	spin_unlock(&files->file_lock);
	return -EBADF;
}

int filp_close(struct file *filp, fl_owner_t id)
{
	int retval = 0;

	if (!file_count(filp)) {
		printk(KERN_ERR "VFS: Close: file count is 0\n");
		return 0;
	}

	if (filp->f_op->flush)
		retval = filp->f_op->flush(filp, id);

	if (likely(!(filp->f_mode & FMODE_PATH))) {
		dnotify_flush(filp, id);
		locks_remove_posix(filp, id);
	}
	fput(filp);
	return retval;
}
========================================================================

What is revealed is that:

 1. The file descriptor is released regardless of the return value of
    close(2):

    __put_unused_fd(files, fd);

 2. The file object's refcount is decremented accordingly regardless of
    the return value of close(2).

 3. The return value reflects the success of the optional flush() method
    of the file object.

 4. The flush() method is called with each call to close(2), not only
    the last one.

IOW, os.close() closes the file even if it should report a failure. I
couldn't have guessed that behavior from the man page.

So the strategy you proposed is the right one: have the child process
ignore any possible errors from os.close(). The parent will have an
opportunity to deal with them.

And now Linux is back in the good graces, only the man page is
misleading.


Marko


From nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de  Wed Jun  3 09:56:53 2015
From: nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de (Thomas Rachel)
Date: Wed, 03 Jun 2015 15:56:53 +0200
Subject: Can Python function return multiple data?
In-Reply-To: <mailman.85.1433289398.13271.python-list@python.org>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
Message-ID: <mkn139$6hb$1@r01.glglgl.de>

Am 03.06.2015 um 01:56 schrieb Chris Angelico:

> and it's pretty convenient. In C, the nearest equivalent is passing a
> number of pointers as parameters, and having the function fill out
> values. Python's model is a lot closer to what you're saying than C's
> model is :)

At least, C functions can return structs...

Thomas


From ian.g.kelly at gmail.com  Wed Jun  3 09:57:47 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 3 Jun 2015 07:57:47 -0600
Subject: Everything is an object in python - object class and type class
In-Reply-To: <871tht6uf6.fsf@elektro.pacujo.net>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <yDhbx.572527$oZ4.306159@fx38.am4>
 <556de143$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <%9mbx.608935$JH2.445461@fx11.am4>
 <d7dfabae-a252-476e-aeef-a76e29be202a@googlegroups.com>
 <slrnmmrukl.20b.jon+usenet@frosty.unequivocal.co.uk>
 <914115ea-0f93-4929-b325-5b3f78f5d9f1@googlegroups.com>
 <slrnmms16k.20b.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.78.1433274654.13271.python-list@python.org>
 <mkl6l1$lle$1@reader1.panix.com> <871tht7nc3.fsf@elektro.pacujo.net>
 <556eb9cd$0$13008$c3e8da3$5496439d@news.astraweb.com>
 <871tht6uf6.fsf@elektro.pacujo.net>
Message-ID: <CALwzidk9MzPd83aY--LP5KsUbTDdCn1oON=tOFRTO+K8epGPeg@mail.gmail.com>

On Wed, Jun 3, 2015 at 2:57 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Steven D'Aprano <steve+comp.lang.python at pearwood.info>:
>
>> On Wednesday 03 June 2015 08:33, Marko Rauhamaa wrote:
>>> In Python, classes are little more than constructor functions.
>>
>> [...]
>>
>> Classes give you an inheritance hierarchy.
>
> That's encapsulated in the constructor.

Not entirely true.

>>> class A:
...   def say_hi(self):
...     print("Hello!")
...
>>> class B:
...   def say_hi(self):
...     print("G'day!")
...
>>> obj = A()
>>> obj.say_hi()
Hello!
>>> obj.__class__ = B
>>> obj.say_hi()
G'day!


From dav.kuhn at gmail.com  Wed Jun  3 10:33:04 2015
From: dav.kuhn at gmail.com (dav.kuhn at gmail.com)
Date: Wed, 3 Jun 2015 07:33:04 -0700 (PDT)
Subject: python 3.4 use python-gcm can't import
In-Reply-To: <9c330850-fd0d-4190-a924-5cd99fbaccd6@googlegroups.com>
References: <c6179d50-e8c4-43d4-896f-77d234e0f9ff@googlegroups.com>
 <mailman.13596.1409285341.18130.python-list@python.org>
 <9c330850-fd0d-4190-a924-5cd99fbaccd6@googlegroups.com>
Message-ID: <48616a02-4d2a-4ef4-8f10-c69e671e71a3@googlegroups.com>

Le vendredi 29 ao?t 2014 10:35:29 UTC+2, Frank Liou a ?crit?:

> and body is "b'" .....is empty
> 
> i'm so confused.don't know it work or not

Hi, you should read() before you close() the connection.

conn = http.client.HTTPConnection('android.googleapis.com')
conn.request('POST', '/gcm/send', jqs, headers)
res = conn.getresponse()
body = res.read()
conn.close()
print(body)
David


From rustompmody at gmail.com  Wed Jun  3 10:40:48 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Wed, 3 Jun 2015 07:40:48 -0700 (PDT)
Subject: for...else
In-Reply-To: <mailman.52.1433244404.13271.python-list@python.org>
References: <mailman.52.1433244404.13271.python-list@python.org>
Message-ID: <653fe238-5a27-4b42-9a67-3e4ace22bbd0@googlegroups.com>

On Tuesday, June 2, 2015 at 4:56:57 PM UTC+5:30, acdr wrote:
> Hi,
> 
> Currently, in various places in my code, I have the equivalent of:
> 
> for x in it:
>     if complicated_calculation_1():
>         cleanup()
>         break
>     complicated_calculation_2()
>     if complicated_calculation_3():
>         cleanup()
>         break
> 
> Obviously, I'm repeating myself by having two separate calls to
> cleanup(). I can't really see a nicer way to do this. (Though I see
> plenty of non-nice ways to do this, such as adding "broken = True" in
> front of every "break", and then after the loop is done, have an "if
> broken" section.) Other solutions that I'm not particularly fond of
> can be found on stackexchange, where someone else is trying to do the
> same thing:
> http://stackoverflow.com/questions/3296044/opposite-of-python-for-else
> 
> I'm wondering if there is a demand for expanding the "for...else"
> functionality to be expanded also have a block of code that only gets
> called if the loop is broken out of. I.e.:
> 
> for x in it:
>     ...
> then:
>     # "break" was called
>     ...
> else:
>     # "break was not called
>     ...

This question prompts a counter question whose answer Ive been hunting 
unsuccessfully for a while.

The well-known Bohm-Jacopini theorem says that any goto-based program can be
de-goto-ized using booleans:
http://en.wikipedia.org/wiki/Structured_program_theorem.

However there is a complementary theorem (due to Knuth??) that the exact details and sequence of tests performed in the unstructured and the structurized versions may be significantly different.
IOW the unstructured program can be significantly more efficient.

Does anyone know/have the reference to this other theorem?


From marko at pacujo.net  Wed Jun  3 10:43:26 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 17:43:26 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <878uc1567j.fsf@elektro.pacujo.net>
 <mailman.104.1433336055.13271.python-list@python.org>
 <87wpzl3pnn.fsf@elektro.pacujo.net>
 <87oakw532a.fsf@elektro.pacujo.net>
Message-ID: <87iob44zu9.fsf@elektro.pacujo.net>

Marko Rauhamaa <marko at pacujo.net>:

> So the strategy you proposed is the right one: have the child process
> ignore any possible errors from os.close(). The parent will have an
> opportunity to deal with them.
>
> And now Linux is back in the good graces, only the man page is
> misleading.

However, the child process needs to be prepared for os.close() to block
indefinitely because of an NFS problem or because SO_LINGER has been
specified by the parent, for example. Setting the close-on-exec flag
doesn't help there.


Marko


From marko at pacujo.net  Wed Jun  3 10:49:23 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 03 Jun 2015 17:49:23 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mkmrat$s26$1@speranza.aioe.org> <874mmp5654.fsf@elektro.pacujo.net>
 <mkmu25$3p9$1@speranza.aioe.org>
Message-ID: <87egls4zkc.fsf@elektro.pacujo.net>

alister <alister.nospam.ware at ntlworld.com>:

> I meant the program that is supplying your app with file handles
> willy- nilly without caring what happens to them

You seem to be advocating a strategy whereby the application keeps close
track of all file descriptors and closes them individually as needed.

Thing is, processes can be forked by library calls. For example, you
might have developed a class that converts a synchronous database API
into an asynchronous one. You'll have your object fork a helper process
that makes blocking calls while the object methods make sure never to
block the caller.

The application doesn't know the library would be starting a child
process. On the other hand, the library has no idea on what files the
application might have open. That's why the library traverses all file
descriptors and closes them categorically after forking.


Marko


From breamoreboy at yahoo.co.uk  Wed Jun  3 11:20:57 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 03 Jun 2015 16:20:57 +0100
Subject: most ambiguous line of code ever written?
Message-ID: <mkn60u$4k3$1@ger.gmane.org>

I wanted a piece of code to write alternatively to the left and right 
sides of a deque.  I remembered a stackoverflow question about toggling 
with an excellent answer from Raymond Hettinger.  So I now have this.

right = not right

Isn't that just beautiful?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From bc at freeuk.com  Wed Jun  3 12:00:11 2015
From: bc at freeuk.com (BartC)
Date: Wed, 03 Jun 2015 17:00:11 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <87fv6956zw.fsf@elektro.pacujo.net>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>	<48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>	<556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>	<1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>	<DxAbx.902026$_Q7.705597@fx28.am4>	<mailman.99.1433328294.13271.python-list@python.org>	<NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net>
Message-ID: <jwFbx.736675$m15.72884@fx45.am4>

On 03/06/2015 13:08, Marko Rauhamaa wrote:
> BartC <bc at freeuk.com>:
>
>> To 'variable' and 'type', you might need to add 'value' to make it more
>> complete.
>
> 'Value' and 'object' are indeed synonymous as long as you keep in mind
> that:
>
>      >>> -12 == -12
>      True
>      >>> -12 is -12
>      False
>
> IOW, the literal expression -12 happens to construct a fresh
> value/object each time CPython parses it.

That's a different matter. However, you appear to be wrong.

print (-12 is -12)

gives True. As does ("abc" is "abc"). I assume constructions for 
immutable values will do the same (([10,20,30] is [10,20,30]) gives 
False because the constructs are mutable, although it's difficult to see 
how in that form).

(This is on 2.7, 3.1 and PyPy. On 3.4.3, (-12 is -12) gives False as you 
say, although (12 is 12) gives True, so not even Python can make up its 
mind how it's supposed to work!)

3.4.3:

print (-12 is -12)                                     => False
print (12 is 12)                                       => True
print (20000000000000000000 is 20000000000000000000)   => True

The others all give True in all cases. It seems that older Python 
versions have a purer object model.

-- 
Bartc



From lac at openend.se  Wed Jun  3 12:13:19 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 03 Jun 2015 18:13:19 +0200
Subject: Calling Python Script from an SQL Proceudre
In-Reply-To: Message from Amit Goutham <agn.91.91@gmail.com> of "Wed,
 03 Jun 2015 15:17:16 +0530."
 <CAJ5KB9jsYR0E+chPonkGAFkt1ihW0FSJ_3s0YXF_XRj7wmDMrw@mail.gmail.com>
References: <CAJ5KB9jsYR0E+chPonkGAFkt1ihW0FSJ_3s0YXF_XRj7wmDMrw@mail.gmail.com>
Message-ID: <201506031613.t53GDJho003039@fido.openend.se>

In a message of Wed, 03 Jun 2015 15:17:16 +0530, Amit Goutham writes:
>Hi All,
>I am trying to search on the Internet if i can call a Python Script from an
>SQL Procedure.
>All the information found on Internet is about connecting to a database
>from Python through a Python script.But, i want the other way round.
>
>Any Help will be appreciated
>
>-- 
>Thanks and Regards,
>Amit Goutham
>Ph:+91-8867801035

You certainly can with postgreql, see
http://www.postgresql.org/docs/current/static/plpython.html

I don't know about other SQL servers.

Laura


From Cecil at decebal.nl  Wed Jun  3 12:15:16 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 03 Jun 2015 18:15:16 +0200
Subject: Retrying to send message
References: <87egltht87.fsf@Equus.decebal.nl>
 <mailman.108.1433338144.13271.python-list@python.org>
Message-ID: <876174ix9n.fsf@Equus.decebal.nl>

Op Wednesday 3 Jun 2015 15:29 CEST schreef Chris Angelico:

> On Wed, Jun 3, 2015 at 10:27 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> def send_message(account_id, message, max_tries,
>> terminate_program): error_msg = 'Something went wrong with: ' +
>> message not_send = True tries = 0 while not_send: try:
>> Core().update_status(account_id, message) except
>> libturpial.exceptions.ServiceOverCapacity: tries += 1 print('Tried
>> to send it {0} times'.format(tries)) if tries >= max_tries:
>> terminate_program(error_msg) time.sleep(60) except:
>> terminate_program(error_msg) else: not_send = False
>>
>> Is this a reasonable way to implement this, or is another way
>> better? Well, maybe the timeout should be a parameter also. ;-)
>
> I'd skip the not_send flag and do the logic thusly:
>
> while True:
> try:
> update_status as above
> break
> except ServiceOverCapacity:
> as above

Yes that is much better. I now made it:
    def send_message(account_id, message, max_tries, give_error, wait_time = 60):
        error_msg   = 'Something went wrong with: ' + message
        tries       = 0
        while True:
            try:
                Core().update_status(account_id, message)
                break
            except libturpial.exceptions.ServiceOverCapacity:
                tries += 1
                print('Tried to send it {0} times'.format(tries))
                if tries >= max_tries:
                    give_error(error_msg)
                    return
                time.sleep(wait_time)
            except:
                give_error(error_msg)
                return

> And I'd also skip the bare except clause. If you get any sort of
> exception, whether it's a bug, a failure from libturpial, a network
> error, or anything else, your code will just terminate with a bland
> and useless message. Much better to simply let the exception bubble
> up.

I kept the except. I like to see the message that went wrong. ;-)

Also, in my case give_error terminates the program, but it never hurts
to make functionality generic, so I added return statements. Not
break, because on error you should leave the function. Break would in
this instance do the same, but I find this neater.


>> Also the documentation of time.sleep says:
>> The actual suspension time may be less than that requested because
>> any caught signal will terminate the sleep() following execution
>> of that signal?s catching routine.
>>
>> My program does not much else as sending the message. So I think it
>> is not necessary to cover for this possibility. Are I assuming to
>> much?
>
> I wouldn't worry too much about a signal cutting your sleep short;
> it doesn't look to me as if "exactly sixty seconds" is all that
> crucial here. If your program sleeps for 42.645 seconds and then
> gets woken up by a signal, and you retry a bit sooner than you
> otherwise would have, is it going to break anything? If not, just
> ignore that possibility.

Correct, so I did not assume to much. :-D

Usually I check parameters. But I understood that is not the Python
way. For example a wait_time of 1, or negative is not useful. Also a
max_tries of 10 ** 12 would not be very useful. But I can put the
burden of the problem on the caller of the function?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From breamoreboy at yahoo.co.uk  Wed Jun  3 12:29:12 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 03 Jun 2015 17:29:12 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <jwFbx.736675$m15.72884@fx45.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>	<48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>	<556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>	<1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>	<DxAbx.902026$_Q7.705597@fx28.am4>	<mailman.99.1433328294.13271.python-list@python.org>	<NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net> <jwFbx.736675$m15.72884@fx45.am4>
Message-ID: <mkna0s$7rv$1@ger.gmane.org>

On 03/06/2015 17:00, BartC wrote:
> On 03/06/2015 13:08, Marko Rauhamaa wrote:
>> BartC <bc at freeuk.com>:
>>
>>> To 'variable' and 'type', you might need to add 'value' to make it more
>>> complete.
>>
>> 'Value' and 'object' are indeed synonymous as long as you keep in mind
>> that:
>>
>>      >>> -12 == -12
>>      True
>>      >>> -12 is -12
>>      False
>>
>> IOW, the literal expression -12 happens to construct a fresh
>> value/object each time CPython parses it.
>
> That's a different matter. However, you appear to be wrong.
>
> print (-12 is -12)
>
> gives True. As does ("abc" is "abc"). I assume constructions for
> immutable values will do the same (([10,20,30] is [10,20,30]) gives
> False because the constructs are mutable, although it's difficult to see
> how in that form).
>
> (This is on 2.7, 3.1 and PyPy. On 3.4.3, (-12 is -12) gives False as you
> say, although (12 is 12) gives True, so not even Python can make up its
> mind how it's supposed to work!)
>
> 3.4.3:
>
> print (-12 is -12)                                     => False
> print (12 is 12)                                       => True
> print (20000000000000000000 is 20000000000000000000)   => True
>
> The others all give True in all cases. It seems that older Python
> versions have a purer object model.
>

No, you don't understand how cPython does things.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From torriem at gmail.com  Wed Jun  3 12:36:49 2015
From: torriem at gmail.com (Michael Torrie)
Date: Wed, 03 Jun 2015 10:36:49 -0600
Subject: Everything is an object in python - object class and type class
In-Reply-To: <jwFbx.736675$m15.72884@fx45.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>	<48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>	<556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>	<1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>	<DxAbx.902026$_Q7.705597@fx28.am4>	<mailman.99.1433328294.13271.python-list@python.org>	<NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net> <jwFbx.736675$m15.72884@fx45.am4>
Message-ID: <556F2D21.3010905@gmail.com>

On 06/03/2015 10:00 AM, BartC wrote:
> The others all give True in all cases. It seems that older Python 
> versions have a purer object model.

No.  It's just an under-the-hood optimization that the interpreter is
making.  It's an implementation detail that you should never rely on.
It says nothing about the purity of the object model.  Immutable objects
can be optimized ("interred" is the word often used) by reusing the same
object over and over when the interpreter can.



From lac at openend.se  Wed Jun  3 12:57:42 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 03 Jun 2015 18:57:42 +0200
Subject: for...else
In-Reply-To: Message from Rustom Mody <rustompmody@gmail.com> of "Wed,
 03 Jun 2015 07:40:48 -0700."
 <653fe238-5a27-4b42-9a67-3e4ace22bbd0@googlegroups.com>
References: <mailman.52.1433244404.13271.python-list@python.org><653fe238-5a27-4b42-9a67-3e4ace22bbd0@googlegroups.com>
Message-ID: <201506031657.t53Gvg8t004378@fido.openend.se>

Are you looking for Knuth's paper Structured Programming with Goto Statements?
http://c2.com/cgi/wiki?StructuredProgrammingWithGoToStatements

I don't remember a theorem in there, but I haven't read it for decades,
so ...

Laura


From tjreedy at udel.edu  Wed Jun  3 13:05:31 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 03 Jun 2015 13:05:31 -0400
Subject: Everything is an object in python - object class and type class
In-Reply-To: <DxAbx.902026$_Q7.705597@fx28.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
 <DxAbx.902026$_Q7.705597@fx28.am4>
Message-ID: <mknc5r$jqq$1@ger.gmane.org>

On 6/3/2015 6:20 AM, BartC wrote:

> But, it is mostly just jargon. If you go back to using 'variable' and
> 'type', then it becomes a bit easier:
>
> * A variable is an instance of some type.

That is clear enough in itself, but be aware that many people use 
'variable' as a synonym for typeless names rather than typed values.

-- 
Terry Jan Reedy




From john_ladasky at sbcglobal.net  Wed Jun  3 13:13:15 2015
From: john_ladasky at sbcglobal.net (John Ladasky)
Date: Wed, 3 Jun 2015 10:13:15 -0700 (PDT)
Subject: most ambiguous line of code ever written?
In-Reply-To: <mailman.112.1433344889.13271.python-list@python.org>
References: <mailman.112.1433344889.13271.python-list@python.org>
Message-ID: <90349f7e-16ec-4d4e-a5a6-6f6fdd1034d2@googlegroups.com>

On Wednesday, June 3, 2015 at 8:21:43 AM UTC-7, Mark Lawrence wrote:

> right = not right
> 
> Isn't that just beautiful?

A few weeks ago, we were discussing the fact that, in Python 2, True = False, and False = True, were both legal statements in Python.  I remarked: "every politician's dream!"  This remark would apply equally well to your clever statement.


From tjreedy at udel.edu  Wed Jun  3 13:17:49 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 03 Jun 2015 13:17:49 -0400
Subject: Everything is an object in python - object class and type class
In-Reply-To: <jwFbx.736675$m15.72884@fx45.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>	<48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>	<556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>	<1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>	<DxAbx.902026$_Q7.705597@fx28.am4>	<mailman.99.1433328294.13271.python-list@python.org>	<NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net> <jwFbx.736675$m15.72884@fx45.am4>
Message-ID: <mkncst$g9$1@ger.gmane.org>

On 6/3/2015 12:00 PM, BartC wrote:

> That's a different matter. However, you appear to be wrong.
>
> print (-12 is -12)
>
> gives True. As does ("abc" is "abc"). I assume constructions for
> immutable values will do the same (([10,20,30] is [10,20,30]) gives
> False because the constructs are mutable, although it's difficult to see
> how in that form).
>
> (This is on 2.7, 3.1 and PyPy. On 3.4.3, (-12 is -12) gives False as you
> say, although (12 is 12) gives True, so not even Python can make up its
> mind how it's supposed to work!)

Implementation details not specified in the language definition are just 
that -- implementation details.  The 'is' operator. which is carefully 
defined in the reference, has three uses:
1. test whether an object is None, or some other specific, special object.
2. for implementation developers, test whether the implementation 
details, including optimizations, are as intended.
3. (least important, and often leading to confusion when people make 
assumptions and extrapolate limited results) for users, discover 
implementation details.

-- 
Terry Jan Reedy



From bc at freeuk.com  Wed Jun  3 13:26:47 2015
From: bc at freeuk.com (BartC)
Date: Wed, 03 Jun 2015 18:26:47 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <mailman.115.1433349421.13271.python-list@python.org>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>	<48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>	<556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>	<1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>	<DxAbx.902026$_Q7.705597@fx28.am4>	<mailman.99.1433328294.13271.python-list@python.org>	<NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net> <jwFbx.736675$m15.72884@fx45.am4>
 <mailman.115.1433349421.13271.python-list@python.org>
Message-ID: <vNGbx.932535$396.701770@fx12.am4>

On 03/06/2015 17:36, Michael Torrie wrote:
> On 06/03/2015 10:00 AM, BartC wrote:
>> The others all give True in all cases. It seems that older Python
>> versions have a purer object model.
>
> No.  It's just an under-the-hood optimization that the interpreter is
> making.  It's an implementation detail that you should never rely on.
> It says nothing about the purity of the object model.  Immutable objects
> can be optimized ("interred" is the word often used) by reusing the same
> object over and over when the interpreter can.

But (-12 is -12) yielding False was being used to illustrate why Value 
and Object didn't mean the same thing.

Are they in fact the same, or is there something else that can be done 
more reliably to show the difference?

(However, I feel the internal representation of values shouldn't matter. 
Each type of value will have its own documented behaviour.)

-- 
Bartc




From python at mrabarnett.plus.com  Wed Jun  3 14:12:05 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 03 Jun 2015 19:12:05 +0100
Subject: Retrying to send message
In-Reply-To: <876174ix9n.fsf@Equus.decebal.nl>
References: <87egltht87.fsf@Equus.decebal.nl>
 <mailman.108.1433338144.13271.python-list@python.org>
 <876174ix9n.fsf@Equus.decebal.nl>
Message-ID: <556F4375.7070309@mrabarnett.plus.com>

On 2015-06-03 17:15, Cecil Westerhof wrote:
> Op Wednesday 3 Jun 2015 15:29 CEST schreef Chris Angelico:
>
>> On Wed, Jun 3, 2015 at 10:27 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>> def send_message(account_id, message, max_tries,
>>> terminate_program): error_msg = 'Something went wrong with: ' +
>>> message not_send = True tries = 0 while not_send: try:
>>> Core().update_status(account_id, message) except
>>> libturpial.exceptions.ServiceOverCapacity: tries += 1 print('Tried
>>> to send it {0} times'.format(tries)) if tries >= max_tries:
>>> terminate_program(error_msg) time.sleep(60) except:
>>> terminate_program(error_msg) else: not_send = False
>>>
>>> Is this a reasonable way to implement this, or is another way
>>> better? Well, maybe the timeout should be a parameter also. ;-)
>>
>> I'd skip the not_send flag and do the logic thusly:
>>
>> while True:
>> try:
>> update_status as above
>> break
>> except ServiceOverCapacity:
>> as above
>
> Yes that is much better. I now made it:
>      def send_message(account_id, message, max_tries, give_error, wait_time = 60):
>          error_msg   = 'Something went wrong with: ' + message
>          tries       = 0
>          while True:
>              try:
>                  Core().update_status(account_id, message)
>                  break
>              except libturpial.exceptions.ServiceOverCapacity:
>                  tries += 1
>                  print('Tried to send it {0} times'.format(tries))
>                  if tries >= max_tries:
>                      give_error(error_msg)
>                      return
>                  time.sleep(wait_time)
>              except:
>                  give_error(error_msg)
>                  return
>
>> And I'd also skip the bare except clause. If you get any sort of
>> exception, whether it's a bug, a failure from libturpial, a network
>> error, or anything else, your code will just terminate with a bland
>> and useless message. Much better to simply let the exception bubble
>> up.
>
> I kept the except. I like to see the message that went wrong. ;-)
>
You're still swallowing unexpected exceptions.

If any exception apart from ServiceOverCapacity is raised, the bare
except will just call 'give_error' and then return.

If you try to interrupt the program (^C on Windows, ^D on Linux), that'll
also be ignored.

> Also, in my case give_error terminates the program, but it never hurts
> to make functionality generic, so I added return statements. Not
> break, because on error you should leave the function. Break would in
> this instance do the same, but I find this neater.
>
>
>>> Also the documentation of time.sleep says:
>>> The actual suspension time may be less than that requested because
>>> any caught signal will terminate the sleep() following execution
>>> of that signal?s catching routine.
>>>
>>> My program does not much else as sending the message. So I think it
>>> is not necessary to cover for this possibility. Are I assuming to
>>> much?
>>
>> I wouldn't worry too much about a signal cutting your sleep short;
>> it doesn't look to me as if "exactly sixty seconds" is all that
>> crucial here. If your program sleeps for 42.645 seconds and then
>> gets woken up by a signal, and you retry a bit sooner than you
>> otherwise would have, is it going to break anything? If not, just
>> ignore that possibility.
>
> Correct, so I did not assume to much. :-D
>
> Usually I check parameters. But I understood that is not the Python
> way. For example a wait_time of 1, or negative is not useful. Also a
> max_tries of 10 ** 12 would not be very useful. But I can put the
> burden of the problem on the caller of the function?
>



From davros at bellaliant.net  Wed Jun  3 14:22:15 2015
From: davros at bellaliant.net (John McKenzie)
Date: Wed, 03 Jun 2015 18:22:15 GMT
Subject: Keypress Input
Message-ID: <rBHbx.75089$MO7.40532@fx10.iad>


 Hello.

 Very new to Python and looking for some basic help.

 Would like a set-up where something happens when a key is pressed. Not 
propose a question, have the user type something, then hit return, then 
something happens, but just the R key is pressed, something happens, then 
something else happens if the B key is pressed, then a third thing 
happens if the G key is pressed.

 My research only served to confuse me. Firstly, I do not understand how 
it is possible for this to be a difficult thing not built into the system 
for any scripting language made within the last few decades. More to the 
point I am unclear on specific suggestions. Most of them seem to be for 
Windows only and I want this working on a Raspberry Pi. Saw getch but I 
am still confused if it is platform specific or not, or requires a module 
to be installed or not. Just get errors if I try to install getch using 
PIP.

 Other suggestions seemed to be overkill and confused me to due to my 
beginner level knowledge and the fact these suggestions have other, more 
complicated elements to them.

 I just want a button press on a device connected to a Raspberry Pi to 
trigger an action. If anyone can give me some guidance on this I would 
appreciate it.

 Thank you.



From ethan at stoneleaf.us  Wed Jun  3 14:28:54 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 03 Jun 2015 11:28:54 -0700
Subject: Retrying to send message
In-Reply-To: <876174ix9n.fsf@Equus.decebal.nl>
References: <87egltht87.fsf@Equus.decebal.nl>
 <mailman.108.1433338144.13271.python-list@python.org>
 <876174ix9n.fsf@Equus.decebal.nl>
Message-ID: <556F4766.5090500@stoneleaf.us>

On 06/03/2015 09:15 AM, Cecil Westerhof wrote:

> I kept the except. I like to see the message that went wrong. ;-)

That's fine, but then add a `raise` after you print the error so you can see the reason that message failed.

--
~Ethan~


From gherron at digipen.edu  Wed Jun  3 14:47:06 2015
From: gherron at digipen.edu (Gary Herron)
Date: Wed, 03 Jun 2015 11:47:06 -0700
Subject: Keypress Input
In-Reply-To: <rBHbx.75089$MO7.40532@fx10.iad>
References: <rBHbx.75089$MO7.40532@fx10.iad>
Message-ID: <556F4BAA.2070104@digipen.edu>

On 06/03/2015 11:22 AM, John McKenzie wrote:
>   Hello.
>
>   Very new to Python and looking for some basic help.
>
>   Would like a set-up where something happens when a key is pressed. Not
> propose a question, have the user type something, then hit return, then
> something happens, but just the R key is pressed, something happens, then
> something else happens if the B key is pressed, then a third thing
> happens if the G key is pressed.
>
>   My research only served to confuse me. Firstly, I do not understand how
> it is possible for this to be a difficult thing not built into the system
> for any scripting language made within the last few decades. More to the
> point I am unclear on specific suggestions. Most of them seem to be for
> Windows only and I want this working on a Raspberry Pi. Saw getch but I
> am still confused if it is platform specific or not, or requires a module
> to be installed or not. Just get errors if I try to install getch using
> PIP.

If you are using Python through a CLI (command line interface i.e., a 
shell), then in fact your request doesn't really make sense.  CLIs by 
their nature don't support that kind of interaction.

BUT don't despair.  Nearly every GIU framework on the planet has a 
Python interface, and they all allow for a window to be opened with 
event processing of your choice.

This are some good places to start:
     https://wiki.python.org/moin/GuiProgramming
     https://wiki.python.org/moin/GUI%20Programming%20in%20Python

Several of these (Tkinter and the curses module) are distributed with 
Python, so you should already have them installed.

Gary Herron



>
>   Other suggestions seemed to be overkill and confused me to due to my
> beginner level knowledge and the fact these suggestions have other, more
> complicated elements to them.
>
>   I just want a button press on a device connected to a Raspberry Pi to
> trigger an action. If anyone can give me some guidance on this I would
> appreciate it.
>
>   Thank you.
>


-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418



From lac at openend.se  Wed Jun  3 14:59:04 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 03 Jun 2015 20:59:04 +0200
Subject: Keypress Input
In-Reply-To: Message from John McKenzie <davros@bellaliant.net>
 of "Wed, 03 Jun 2015 18:22:15 +0000." <rBHbx.75089$MO7.40532@fx10.iad>
References: <rBHbx.75089$MO7.40532@fx10.iad>
Message-ID: <201506031859.t53Ix49H007428@fido.openend.se>

Tkinter runs on raspberry pi.

Get it installed, and then run this program.

from Tkinter import *
root = Tk()
prompt = 'Press any key. Remember to keep your mouse in the cyan box. '
lab = Label(root, text=prompt, width=len(prompt), bg='cyan')
lab.pack()

def key(event):
    msg = 'event.char is %r and event.keysym is %r' % (event.char, event.keysym)
        lab.config(text=msg)

root.bind_all('<Key>', key)
root.mainloop() 

Now you will have to bind the various keys to do what it is you want.
You need to read online dociumentation for tkinter to learn how to do
this, as well as how to use tkinter in general.

Laura


From bc at freeuk.com  Wed Jun  3 14:59:13 2015
From: bc at freeuk.com (BartC)
Date: Wed, 03 Jun 2015 19:59:13 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <mailman.114.1433348990.13271.python-list@python.org>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>	<48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>	<556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>	<1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>	<DxAbx.902026$_Q7.705597@fx28.am4>	<mailman.99.1433328294.13271.python-list@python.org>	<NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net> <jwFbx.736675$m15.72884@fx45.am4>
 <mailman.114.1433348990.13271.python-list@python.org>
Message-ID: <98Ibx.766729$kA6.22200@fx46.am4>

On 03/06/2015 17:29, Mark Lawrence wrote:
> On 03/06/2015 17:00, BartC wrote:
>> On 03/06/2015 13:08, Marko Rauhamaa wrote:
>>> BartC <bc at freeuk.com>:
>>>
>>>> To 'variable' and 'type', you might need to add 'value' to make it more
>>>> complete.
>>>
>>> 'Value' and 'object' are indeed synonymous as long as you keep in mind
>>> that:
>>>
>>>      >>> -12 == -12
>>>      True
>>>      >>> -12 is -12
>>>      False
>>>
>>> IOW, the literal expression -12 happens to construct a fresh
>>> value/object each time CPython parses it.
>>
>> That's a different matter. However, you appear to be wrong.
>>
>> print (-12 is -12)
>>
>> gives True.

> No, you don't understand how cPython does things.

Do I need to understand CPython to appreciate what difference there 
might be between variables, values and objects?

Does anyone need to understand CPython for anything?

-- 
Bartc


From gherron at digipen.edu  Wed Jun  3 15:15:32 2015
From: gherron at digipen.edu (Gary Herron)
Date: Wed, 03 Jun 2015 12:15:32 -0700
Subject: Keypress Input
In-Reply-To: <rBHbx.75089$MO7.40532@fx10.iad>
References: <rBHbx.75089$MO7.40532@fx10.iad>
Message-ID: <556F5254.7060200@digipen.edu>

On 06/03/2015 11:22 AM, John McKenzie wrote:
>   Hello.
>
>   Very new to Python and looking for some basic help.
>
>   Would like a set-up where something happens when a key is pressed. Not
> propose a question, have the user type something, then hit return, then
> something happens, but just the R key is pressed, something happens, then
> something else happens if the B key is pressed, then a third thing
> happens if the G key is pressed.
>
>   My research only served to confuse me. Firstly, I do not understand how
> it is possible for this to be a difficult thing not built into the system
> for any scripting language made within the last few decades. More to the
> point I am unclear on specific suggestions. Most of them seem to be for
> Windows only and I want this working on a Raspberry Pi. Saw getch but I
> am still confused if it is platform specific or not, or requires a module
> to be installed or not. Just get errors if I try to install getch using
> PIP.

If you are using Python through a CLI (command line interface i.e., a 
shell), then in fact your request doesn't really make sense.  CLIs by 
their nature don't support that kind of interaction.

BUT don't despair.  Nearly every GIU framework on the planet has a 
Python interface, and they all allow for a window to be opened with 
event processing of your choice.

This are some good places to start:
     https://wiki.python.org/moin/GuiProgramming
     https://wiki.python.org/moin/GUI%20Programming%20in%20Python

Several of these (Tkinter and the curses module) are distributed with 
Python, so you should already have them installed.

Gary Herron



>
>   Other suggestions seemed to be overkill and confused me to due to my
> beginner level knowledge and the fact these suggestions have other, more
> complicated elements to them.
>
>   I just want a button press on a device connected to a Raspberry Pi to
> trigger an action. If anyone can give me some guidance on this I would
> appreciate it.
>
>   Thank you.
>


-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418



From random832 at fastmail.us  Wed Jun  3 16:07:26 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 03 Jun 2015 16:07:26 -0400
Subject: fork/exec & close file descriptors
In-Reply-To: <CAPTjJmoTzNnDuoqVfbYOcRUtw3GDDqRVDaJxUa4F8xUFefk6tg@mail.gmail.com>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <878uc1567j.fsf@elektro.pacujo.net>
 <mailman.104.1433336055.13271.python-list@python.org>
 <87wpzl3pnn.fsf@elektro.pacujo.net>
 <1433337679.1493926.285725793.69D6B33F@webmail.messagingengine.com>
 <CAPTjJmoTzNnDuoqVfbYOcRUtw3GDDqRVDaJxUa4F8xUFefk6tg@mail.gmail.com>
Message-ID: <1433362046.1590533.286121265.066C282C@webmail.messagingengine.com>

On Wed, Jun 3, 2015, at 09:32, Chris Angelico wrote:
> Write an editor that opens a file and holds it open until the user's
> done with it. Have something that lets you shell out for whatever
> reason. Then trigger the shell-out, and instantly SIGSTOP the child
> process, before it does its work - or just have a really heavily
> loaded system, so it can't get a time slice. Now close the file in the
> UI, which results in the file being closed in the parent. Right, now
> let the child run... and there it goes, closing the file.

The parent should be waiting for the child process. If it shouldn't wait
for the command, then the child process should spawn a grandchild
process, after closing the file descriptors. This is how the text editor
I use actually works (more or less. In fact, the way to run a process it
won't wait for is to run it as a background shell command. If you STOP
the shell itself, the editor will be stuck waiting.)


From random832 at fastmail.us  Wed Jun  3 16:08:56 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 03 Jun 2015 16:08:56 -0400
Subject: fork/exec & close file descriptors
In-Reply-To: <87iob44zu9.fsf@elektro.pacujo.net>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <878uc1567j.fsf@elektro.pacujo.net>
 <mailman.104.1433336055.13271.python-list@python.org>
 <87wpzl3pnn.fsf@elektro.pacujo.net> <87oakw532a.fsf@elektro.pacujo.net>
 <87iob44zu9.fsf@elektro.pacujo.net>
Message-ID: <1433362136.1590644.286124761.15B5CFF8@webmail.messagingengine.com>

On Wed, Jun 3, 2015, at 10:43, Marko Rauhamaa wrote:
> However, the child process needs to be prepared for os.close() to block
> indefinitely because of an NFS problem or because SO_LINGER has been
> specified by the parent, for example. Setting the close-on-exec flag
> doesn't help there.

Out of curiosity, does exec block in this situation?


From breamoreboy at yahoo.co.uk  Wed Jun  3 16:37:26 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 03 Jun 2015 21:37:26 +0100
Subject: Retrying to send message
In-Reply-To: <556F4766.5090500@stoneleaf.us>
References: <87egltht87.fsf@Equus.decebal.nl>
 <mailman.108.1433338144.13271.python-list@python.org>
 <876174ix9n.fsf@Equus.decebal.nl> <556F4766.5090500@stoneleaf.us>
Message-ID: <mknoib$46i$1@ger.gmane.org>

On 03/06/2015 19:28, Ethan Furman wrote:
> On 06/03/2015 09:15 AM, Cecil Westerhof wrote:
>
>> I kept the except. I like to see the message that went wrong. ;-)
>
> That's fine, but then add a `raise` after you print the error so you can
> see the reason that message failed.
>
> --
> ~Ethan~

Why bother in the first place, especially when you're developing?  Far 
easier to let the exception bubble up, or are we talking cross purposes?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From mohan.mohta at gmail.com  Wed Jun  3 16:41:00 2015
From: mohan.mohta at gmail.com (Mohan Mohta)
Date: Wed, 3 Jun 2015 13:41:00 -0700 (PDT)
Subject: Multiple thread program problem
Message-ID: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>

Hello
I am trying to create multiple thread through the below program but I am getting an error 

#! /usr/bin/python
import os
import subprocess
import thread
import threading
from thread import start_new_thread

def proc(f) :
        com1="ssh -B "
        com2=line.strip('\n')
        com3= " uname -a"
        co=str("ssh -B ")+ str(com2) + str(" uname -a")
        subprocess.call(co,shell=True)
        print "----------------------------"
        return

f = open('/tmp/python/1')
for line in f:
        t=thread.start_new_thread(proc(f),())
        t.start()
f.close()
c = raw_input(" Type anything to quit")


Execution output:
Linux abc.myhomenetwork.com 2.6.18-348.25.1.el5 #1 SMP Thu Apr 10 06:32:45 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux
----------------------------
Traceback (most recent call last):
  File "./readfile1.py", line 19, in <module>
    t=thread.start_new_thread(proc(f),())
TypeError: first arg must be callable


From breamoreboy at yahoo.co.uk  Wed Jun  3 16:58:16 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 03 Jun 2015 21:58:16 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <98Ibx.766729$kA6.22200@fx46.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>	<48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>	<556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>	<1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>	<DxAbx.902026$_Q7.705597@fx28.am4>	<mailman.99.1433328294.13271.python-list@python.org>	<NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net> <jwFbx.736675$m15.72884@fx45.am4>
 <mailman.114.1433348990.13271.python-list@python.org>
 <98Ibx.766729$kA6.22200@fx46.am4>
Message-ID: <mknppc$o6l$1@ger.gmane.org>

On 03/06/2015 19:59, BartC wrote:

>
> Does anyone need to understand CPython for anything?
>

No you (plural) don't.  If people were to spend more time writing code 
and less time on hypothetical claptrap the amount of noise on this list 
would probably be reduced by 99%.

Then knock out those who are interested in things like rugby, with 
Haskell being an England international, my beautiful eight month old 
great niece Ruby, and various other things, we might even spend some 
time actually talking about the Python programming langauge.  Or have 
you all forgotten that that is what this list is about?  Hardly 
surprising that the bulk of the core devs left years ago, is it?

Oh, have to get in my obligatory mention of CORAL 66/250.  If you lot 
can waffle of about languages in which I've no interest on this list, 
I'll once again mention my poor old darling who gets little or no TLC.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From python at mrabarnett.plus.com  Wed Jun  3 16:59:47 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 03 Jun 2015 21:59:47 +0100
Subject: Multiple thread program problem
In-Reply-To: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
References: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
Message-ID: <556F6AC3.4030506@mrabarnett.plus.com>

On 2015-06-03 21:41, Mohan Mohta wrote:
> Hello
> I am trying to create multiple thread through the below program but I am getting an error
>
> #! /usr/bin/python
> import os
> import subprocess
> import thread
> import threading
> from thread import start_new_thread
>
> def proc(f) :
>          com1="ssh -B "
>          com2=line.strip('\n')
>          com3= " uname -a"
>          co=str("ssh -B ")+ str(com2) + str(" uname -a")
>          subprocess.call(co,shell=True)
>          print "----------------------------"
>          return
>
> f = open('/tmp/python/1')
> for line in f:
>          t=thread.start_new_thread(proc(f),())
>          t.start()
> f.close()
> c = raw_input(" Type anything to quit")
>
>
> Execution output:
> Linux abc.myhomenetwork.com 2.6.18-348.25.1.el5 #1 SMP Thu Apr 10 06:32:45 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux
> ----------------------------
> Traceback (most recent call last):
>    File "./readfile1.py", line 19, in <module>
>      t=thread.start_new_thread(proc(f),())
> TypeError: first arg must be callable
>
The first argument should be the function. You're calling proc(f) and 
the passing its result.

Also, start_new_thread starts the thread running immediately and
returns its identifier, an int.



From sam.raker at gmail.com  Wed Jun  3 17:01:01 2015
From: sam.raker at gmail.com (Sam Raker)
Date: Wed, 3 Jun 2015 14:01:01 -0700 (PDT)
Subject: Multiple thread program problem
In-Reply-To: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
References: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
Message-ID: <bd3b582f-20b5-477d-b7e3-32c4e73600d0@googlegroups.com>

proc(f) isn't a callable, it's whatever it returns. IIRC, you need to do something like 'start_new_thread(proc, (f,))'


From rustompmody at gmail.com  Wed Jun  3 17:08:54 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Wed, 3 Jun 2015 14:08:54 -0700 (PDT)
Subject: How to access the low digits of a list
In-Reply-To: <mailman.63.1433254823.13271.python-list@python.org>
References: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>
 <18a4709f-47c8-41f7-87d5-08bbb00166b6@googlegroups.com>
 <mailman.63.1433254823.13271.python-list@python.org>
Message-ID: <11005554-e89f-407c-8720-fae1bb3fced8@googlegroups.com>

On Tuesday, June 2, 2015 at 7:50:58 PM UTC+5:30, Ian wrote:
> On Tue, Jun 2, 2015 at 6:35 AM, Rustom Mody  wrote:
> > For that matter even this works
> > But I am not sure whats happening or that I like it
> >
> >>>> [x[-2:]  for x in lines]
> > ['12', '42', '49', '56', '25', '36', '49', '64', '81', '00']
> 
> x[-2:] selects all items in the sequence with index i such that len(x)
> - 2 <= i < len(x). For a sequence of length 2 or less, that's the
> entire sequence.

Thanks -- learn something
So it means that indices can give indexerror; slices cannot?
Seems fair enough put that way, but is visually counterintuitive


From marko at pacujo.net  Wed Jun  3 17:09:15 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 04 Jun 2015 00:09:15 +0300
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net> <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <878uc1567j.fsf@elektro.pacujo.net>
 <mailman.104.1433336055.13271.python-list@python.org>
 <87wpzl3pnn.fsf@elektro.pacujo.net>
 <87oakw532a.fsf@elektro.pacujo.net>
 <87iob44zu9.fsf@elektro.pacujo.net>
 <mailman.125.1433362139.13271.python-list@python.org>
Message-ID: <87vbf433es.fsf@elektro.pacujo.net>

random832 at fastmail.us:

> On Wed, Jun 3, 2015, at 10:43, Marko Rauhamaa wrote:
>> However, the child process needs to be prepared for os.close() to
>> block indefinitely because of an NFS problem or because SO_LINGER has
>> been specified by the parent, for example. Setting the close-on-exec
>> flag doesn't help there.
>
> Out of curiosity, does exec block in this situation?

I didn't try it, but it is apparent in the source code:

========================================================================
void do_close_on_exec(struct files_struct *files)
{
	unsigned i;
	struct fdtable *fdt;

	/* exec unshares first */
	spin_lock(&files->file_lock);
	for (i = 0; ; i++) {
		unsigned long set;
		unsigned fd = i * BITS_PER_LONG;
		fdt = files_fdtable(files);
		if (fd >= fdt->max_fds)
			break;
		set = fdt->close_on_exec[i];
		if (!set)
			continue;
		fdt->close_on_exec[i] = 0;
		for ( ; set ; fd++, set >>= 1) {
			struct file *file;
			if (!(set & 1))
				continue;
			file = fdt->fd[fd];
			if (!file)
				continue;
			rcu_assign_pointer(fdt->fd[fd], NULL);
			__put_unused_fd(files, fd);
			spin_unlock(&files->file_lock);
			filp_close(file, files);
			cond_resched();
			spin_lock(&files->file_lock);
		}

	}
	spin_unlock(&files->file_lock);
}

int filp_close(struct file *filp, fl_owner_t id)
{
	int retval = 0;

	if (!file_count(filp)) {
		printk(KERN_ERR "VFS: Close: file count is 0\n");
		return 0;
	}

	if (filp->f_op->flush)
		retval = filp->f_op->flush(filp, id);

	if (likely(!(filp->f_mode & FMODE_PATH))) {
		dnotify_flush(filp, id);
		locks_remove_posix(filp, id);
	}
	fput(filp);
	return retval;
}
========================================================================

Now, the kernel NFS code specifies a flush() method, which can block and
even fail.

Sockets don't have a flush() method. So closing a socket cannot fail.
However, fput(), which decrements the reference count, may block if
lingering has been specified for the socket.

So I wasn't all that wrong earlier after all: whoever closes a socket
last will linger. Thus, the parent (who wanted to linger) might zip
through closing a socket while the unwitting child process will suffer
the lingering delay before it gets to exec.

However, there's this comment in inet_release():

		/* [...]
		 * If the close is due to the process exiting, we never
		 * linger..
		 */


Marko


From rustompmody at gmail.com  Wed Jun  3 17:22:57 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Wed, 3 Jun 2015 14:22:57 -0700 (PDT)
Subject: Let's make Python into LISP
In-Reply-To: <63af3420-3c37-402e-b082-eb3a3e8345e4@googlegroups.com>
References: <63af3420-3c37-402e-b082-eb3a3e8345e4@googlegroups.com>
Message-ID: <03a4aa85-64c1-42e9-9fb7-ea47351b6afc@googlegroups.com>

On Wednesday, June 3, 2015 at 4:27:39 AM UTC+5:30, Dr. Bigcock wrote:
> We can make Python like LISP:
> 
> 1.  Make EVERYTHING the same kind of thing (call it "object").
> 2.  Let's make a lot of meta functions like super, instead of judicious use of interpreter impositions.
> 3. Forget *practicality*.  All hail *purity*.
> 
> Wait.  Someone's time machine broke.
> 
> Mark

Hi Mark

The way you keep bringing up Lisp, ?-calculus, etc completely without relevance
to anything, I wonder if you had a bad experience with someone teaching these
and thereabouts?

<Dark Secret>
Everyone knows that the average student is below average.
More true for the average teacher
</Dark Secret>

Yeah disasters (of all shapes and colors) strike and leave us scarred.
Isn't it more intelligent to pick up the pieces and move on than to keep 
putting salt on the scars?

PS If you de-expletivize your name, maybe more people will pay attention to
what you want to say


From bc at freeuk.com  Wed Jun  3 17:33:59 2015
From: bc at freeuk.com (BartC)
Date: Wed, 03 Jun 2015 22:33:59 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <mailman.127.1433365118.13271.python-list@python.org>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>	<48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>	<556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>	<1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>	<DxAbx.902026$_Q7.705597@fx28.am4>	<mailman.99.1433328294.13271.python-list@python.org>	<NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net> <jwFbx.736675$m15.72884@fx45.am4>
 <mailman.114.1433348990.13271.python-list@python.org>
 <98Ibx.766729$kA6.22200@fx46.am4>
 <mailman.127.1433365118.13271.python-list@python.org>
Message-ID: <cpKbx.674783$ob3.28981@fx47.am4>

On 03/06/2015 21:58, Mark Lawrence wrote:
> On 03/06/2015 19:59, BartC wrote:
>
>>
>> Does anyone need to understand CPython for anything?
>>
>
> No you (plural) don't.  If people were to spend more time writing code
> and less time on hypothetical claptrap the amount of noise on this list
> would probably be reduced by 99%.

Not so hypothetical in my case as I have to implement a lot of this stuff.

I'm also quite interested in how Python does things. If it's a good idea 
I'll copy it, if not I'll try and avoid it!

> Then knock out those who are interested in things like rugby, with
> Haskell being an England international, my beautiful eight month old
> great niece Ruby, and various other things, we might even spend some
> time actually talking about the Python programming langauge.  Or have
> you all forgotten that that is what this list is about?  Hardly
> surprising that the bulk of the core devs left years ago, is it?

(I'm accessing the 'list' (whatever that is) via usenet. Usenet itself 
is slowly dying. However I don't believe there weren't pointless 
discussions that went around in circles years ago too!)

-- 
Bartc


From rosuav at gmail.com  Wed Jun  3 17:35:28 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 4 Jun 2015 07:35:28 +1000
Subject: Can Python function return multiple data?
In-Reply-To: <mkn139$6hb$1@r01.glglgl.de>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
Message-ID: <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>

On Wed, Jun 3, 2015 at 11:56 PM, Thomas Rachel
<nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de>
wrote:
> Am 03.06.2015 um 01:56 schrieb Chris Angelico:
>
>> and it's pretty convenient. In C, the nearest equivalent is passing a
>> number of pointers as parameters, and having the function fill out
>> values. Python's model is a lot closer to what you're saying than C's
>> model is :)
>
>
> At least, C functions can return structs...

Oh, yes, I forgot about that. Thought that was C++ but not C, partly
because I never do it in either language. Although in a sense, a
struct is still a single "thing".

ChrisA


From breamoreboy at yahoo.co.uk  Wed Jun  3 17:41:33 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 03 Jun 2015 22:41:33 +0100
Subject: How to access the low digits of a list
In-Reply-To: <11005554-e89f-407c-8720-fae1bb3fced8@googlegroups.com>
References: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>
 <18a4709f-47c8-41f7-87d5-08bbb00166b6@googlegroups.com>
 <mailman.63.1433254823.13271.python-list@python.org>
 <11005554-e89f-407c-8720-fae1bb3fced8@googlegroups.com>
Message-ID: <mknsai$ta$1@ger.gmane.org>

On 03/06/2015 22:08, Rustom Mody wrote:
> On Tuesday, June 2, 2015 at 7:50:58 PM UTC+5:30, Ian wrote:
>> On Tue, Jun 2, 2015 at 6:35 AM, Rustom Mody  wrote:
>>> For that matter even this works
>>> But I am not sure whats happening or that I like it
>>>
>>>>>> [x[-2:]  for x in lines]
>>> ['12', '42', '49', '56', '25', '36', '49', '64', '81', '00']
>>
>> x[-2:] selects all items in the sequence with index i such that len(x)
>> - 2 <= i < len(x). For a sequence of length 2 or less, that's the
>> entire sequence.
>
> Thanks -- learn something
> So it means that indices can give indexerror; slices cannot?
> Seems fair enough put that way, but is visually counterintuitive
>

Are you seriously trying to say that you teach Python but don't 
understand a basic that is here 
https://docs.python.org/3/tutorial/introduction.html, "Slice indices 
have useful defaults; an omitted first index defaults to zero, an 
omitted second index defaults to the size of the string being sliced."?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Wed Jun  3 17:46:18 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 03 Jun 2015 22:46:18 +0100
Subject: Let's make Python into LISP
In-Reply-To: <03a4aa85-64c1-42e9-9fb7-ea47351b6afc@googlegroups.com>
References: <63af3420-3c37-402e-b082-eb3a3e8345e4@googlegroups.com>
 <03a4aa85-64c1-42e9-9fb7-ea47351b6afc@googlegroups.com>
Message-ID: <mknsje$5ai$1@ger.gmane.org>

On 03/06/2015 22:22, Rustom Mody wrote:
> On Wednesday, June 3, 2015 at 4:27:39 AM UTC+5:30, Dr. Bigcock wrote:
>> We can make Python like LISP:
>>
>> 1.  Make EVERYTHING the same kind of thing (call it "object").
>> 2.  Let's make a lot of meta functions like super, instead of judicious use of interpreter impositions.
>> 3. Forget *practicality*.  All hail *purity*.
>>
>> Wait.  Someone's time machine broke.
>>
>> Mark
>
> Hi Mark
>
> The way you keep bringing up Lisp, ?-calculus, etc completely without relevance
> to anything, I wonder if you had a bad experience with someone teaching these
> and thereabouts?
>
> <Dark Secret>
> Everyone knows that the average student is below average.
> More true for the average teacher
> </Dark Secret>
>
> Yeah disasters (of all shapes and colors) strike and leave us scarred.
> Isn't it more intelligent to pick up the pieces and move on than to keep
> putting salt on the scars?
>
> PS If you de-expletivize your name, maybe more people will pay attention to
> what you want to say
>

Please stop replying to this brain dead pot smoking hippy. My mail 
filters work perfectly to keep him away from my inbox, so I'm not 
interested in seeing his complete and utter garbage effectively 
forwarded on from somebody else, thank you.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Wed Jun  3 17:49:09 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 03 Jun 2015 22:49:09 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <cpKbx.674783$ob3.28981@fx47.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>	<48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>	<556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>	<1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>	<DxAbx.902026$_Q7.705597@fx28.am4>	<mailman.99.1433328294.13271.python-list@python.org>	<NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net> <jwFbx.736675$m15.72884@fx45.am4>
 <mailman.114.1433348990.13271.python-list@python.org>
 <98Ibx.766729$kA6.22200@fx46.am4>
 <mailman.127.1433365118.13271.python-list@python.org>
 <cpKbx.674783$ob3.28981@fx47.am4>
Message-ID: <mknsop$5ai$2@ger.gmane.org>

On 03/06/2015 22:33, BartC wrote:
> On 03/06/2015 21:58, Mark Lawrence wrote:
>
> Not so hypothetical in my case as I have to implement a lot of this stuff.
>
> I'm also quite interested in how Python does things. If it's a good idea
> I'll copy it, if not I'll try and avoid it!

Which implementation, cPython, Jython, IronPython...???

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Wed Jun  3 17:54:17 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 4 Jun 2015 07:54:17 +1000
Subject: fork/exec & close file descriptors
In-Reply-To: <1433362046.1590533.286121265.066C282C@webmail.messagingengine.com>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87pp5eksnc.fsf@universite-de-strasbourg.fr.invalid>
 <87eglt7u5i.fsf@elektro.pacujo.net>
 <87lhg1lt00.fsf@universite-de-strasbourg.fr.invalid>
 <87a8wh7nq6.fsf@elektro.pacujo.net>
 <87d21dl0jc.fsf@universite-de-strasbourg.fr.invalid>
 <877frl6xxj.fsf@elektro.pacujo.net>
 <mkmhus$1sr$1@speranza.aioe.org>
 <556eee15$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <878uc1567j.fsf@elektro.pacujo.net>
 <mailman.104.1433336055.13271.python-list@python.org>
 <87wpzl3pnn.fsf@elektro.pacujo.net>
 <1433337679.1493926.285725793.69D6B33F@webmail.messagingengine.com>
 <CAPTjJmoTzNnDuoqVfbYOcRUtw3GDDqRVDaJxUa4F8xUFefk6tg@mail.gmail.com>
 <1433362046.1590533.286121265.066C282C@webmail.messagingengine.com>
Message-ID: <CAPTjJmrHbzuJFLYSvz3u5Ld2F-UV=WNnGbzteKe1HBiH2QOOOQ@mail.gmail.com>

On Thu, Jun 4, 2015 at 6:07 AM,  <random832 at fastmail.us> wrote:
> On Wed, Jun 3, 2015, at 09:32, Chris Angelico wrote:
>> Write an editor that opens a file and holds it open until the user's
>> done with it. Have something that lets you shell out for whatever
>> reason. Then trigger the shell-out, and instantly SIGSTOP the child
>> process, before it does its work - or just have a really heavily
>> loaded system, so it can't get a time slice. Now close the file in the
>> UI, which results in the file being closed in the parent. Right, now
>> let the child run... and there it goes, closing the file.
>
> The parent should be waiting for the child process. If it shouldn't wait
> for the command, then the child process should spawn a grandchild
> process, after closing the file descriptors. This is how the text editor
> I use actually works (more or less. In fact, the way to run a process it
> won't wait for is to run it as a background shell command. If you STOP
> the shell itself, the editor will be stuck waiting.)

Really? I thought forking, execing, and not immediately waiting, was a
standard way to trigger an asynchronous action. My editor lets me
start something and then keep working in the editor, and see the
output from the command when it's ready. (Simple example: A "git push"
might take a long time if the network's slow, and I want to know if it
errors out, but most likely I'll just see the expected messages come
up and that's that.) The parent definitely shouldn't immediately wait
on the child; and it shouldn't disown the child via a second forking
because it wants to report on the child's completion. So no, I don't
think insta-waiting is right in all situations.

ChrisA


From breamoreboy at yahoo.co.uk  Wed Jun  3 17:56:28 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 03 Jun 2015 22:56:28 +0100
Subject: Can Python function return multiple data?
In-Reply-To: <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
Message-ID: <mknt6h$epv$1@ger.gmane.org>

On 03/06/2015 22:35, Chris Angelico wrote:
> On Wed, Jun 3, 2015 at 11:56 PM, Thomas Rachel
> <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de>
> wrote:
>> Am 03.06.2015 um 01:56 schrieb Chris Angelico:
>>
>>> and it's pretty convenient. In C, the nearest equivalent is passing a
>>> number of pointers as parameters, and having the function fill out
>>> values. Python's model is a lot closer to what you're saying than C's
>>> model is :)
>>
>>
>> At least, C functions can return structs...
>
> Oh, yes, I forgot about that. Thought that was C++ but not C, partly
> because I never do it in either language. Although in a sense, a
> struct is still a single "thing".
>
> ChrisA
>

Don't forget that C functions can accept structs as input.  Possibly not 
a good idea as I found out many years ago pre ANSIC when I forgot that 
little old ampersand, so the compiler didn't pick it up, but then with 
modern computers having so much memory who really cares if you burn a 
little bit of stack on structures rather than pointers to structures?

Now does Python pass by value or by reference?  Happily sits back and 
waits for 10**6 emails to arrive as this is discussed for the 10**6th time.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Wed Jun  3 18:00:13 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 4 Jun 2015 08:00:13 +1000
Subject: Retrying to send message
In-Reply-To: <876174ix9n.fsf@Equus.decebal.nl>
References: <87egltht87.fsf@Equus.decebal.nl>
 <mailman.108.1433338144.13271.python-list@python.org>
 <876174ix9n.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmrwzM=N_b_sYVoRMsVNQgWFNdUbSeMW==HNFnBGkYu1kQ@mail.gmail.com>

On Thu, Jun 4, 2015 at 2:15 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> And I'd also skip the bare except clause. If you get any sort of
>> exception, whether it's a bug, a failure from libturpial, a network
>> error, or anything else, your code will just terminate with a bland
>> and useless message. Much better to simply let the exception bubble
>> up.
>
> I kept the except. I like to see the message that went wrong. ;-)

In that case, there's an easier way to deal with it: just raise a
different exception, and let them chain. Something like this:

>>> def send_message(msg):
...     try: 1/0
...     except: raise FailedMessageException(msg)
...
>>> class FailedMessageException(Exception): pass
...
>>> send_message("Test")
Traceback (most recent call last):
  File "<stdin>", line 2, in send_message
ZeroDivisionError: division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in send_message
__main__.FailedMessageException: Test

But I'd still avoid the bare except, and use "except Exception:"
instead. I don't think you want to catch SystemExit in this way, for
instance. So here's how I'd write that code:

>>> def send_message(msg):
...     try: 1/0
...     except Exception as e: raise FailedMessageException(msg) from e
...
>>> send_message("Test")
Traceback (most recent call last):
  File "<stdin>", line 2, in send_message
ZeroDivisionError: division by zero

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in send_message
__main__.FailedMessageException: Test

(Also, the use of "from" here causes a slightly different wording,
which I think is more appropriate. But that's minor.)

You get the report of which message failed, plus you get the complete
traceback from the original exception. Much MUCH more useful.

ChrisA


From gherron at digipen.edu  Wed Jun  3 18:02:42 2015
From: gherron at digipen.edu (Gary Herron)
Date: Wed, 03 Jun 2015 15:02:42 -0700
Subject: Multiple thread program problem
In-Reply-To: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
References: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
Message-ID: <556F7982.90405@digipen.edu>

On 06/03/2015 01:41 PM, Mohan Mohta wrote:
> Hello
> I am trying to create multiple thread through the below program but I am getting an error
>
> #! /usr/bin/python
> import os
> import subprocess
> import thread
> import threading
> from thread import start_new_thread
>
> def proc(f) :
>          com1="ssh -B "
>          com2=line.strip('\n')
>          com3= " uname -a"
>          co=str("ssh -B ")+ str(com2) + str(" uname -a")
>          subprocess.call(co,shell=True)
>          print "----------------------------"
>          return
>
> f = open('/tmp/python/1')
> for line in f:
>          t=thread.start_new_thread(proc(f),())

You are calling the function f yourself, but what you want here is for 
the thread to call f once it's running.
So do

     t=thread.start_new_thread(proc, (f,)) # Procedure to call and a 
tuple of args for it.

>          t.start()

I don't think the thread has a start method.

> f.close()
> c = raw_input(" Type anything to quit")
>
>
> Execution output:
> Linux abc.myhomenetwork.com 2.6.18-348.25.1.el5 #1 SMP Thu Apr 10 06:32:45 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux
> ----------------------------
> Traceback (most recent call last):
>    File "./readfile1.py", line 19, in <module>
>      t=thread.start_new_thread(proc(f),())
> TypeError: first arg must be callable

You should probably also consider using the higher-level threading 
module rather than the lower level thread module.

(Also consider using Python3 instead of Python2.)

Gary Herron





-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418



From mohan.mohta at gmail.com  Wed Jun  3 18:23:50 2015
From: mohan.mohta at gmail.com (Mohan Mohta)
Date: Wed, 3 Jun 2015 15:23:50 -0700 (PDT)
Subject: Multiple thread program problem
In-Reply-To: <bd3b582f-20b5-477d-b7e3-32c4e73600d0@googlegroups.com>
References: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
 <bd3b582f-20b5-477d-b7e3-32c4e73600d0@googlegroups.com>
Message-ID: <7a358948-5929-46b5-a847-13e8b9bcf46e@googlegroups.com>

On Wednesday, June 3, 2015 at 4:01:13 PM UTC-5, Sam Raker wrote:
> proc(f) isn't a callable, it's whatever it returns. IIRC, you need to do something like 'start_new_thread(proc, (f,))'


If I execute something like 
t=thread.start_new_thread(proc,(f))

I get:

Traceback (most recent call last):
  File "./readfile1.py", line 19, in <module>
    t=thread.start_new_thread(proc,(f))
TypeError: 2nd arg must be a tuple


From sohcahtoa82 at gmail.com  Wed Jun  3 18:28:56 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Wed, 3 Jun 2015 15:28:56 -0700 (PDT)
Subject: Can Python function return multiple data?
In-Reply-To: <mailman.133.1433368610.13271.python-list@python.org>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
Message-ID: <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>

On Wednesday, June 3, 2015 at 2:57:00 PM UTC-7, Mark Lawrence wrote:
> On 03/06/2015 22:35, Chris Angelico wrote:
> > On Wed, Jun 3, 2015 at 11:56 PM, Thomas Rachel
> > <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de>
> > wrote:
> >> Am 03.06.2015 um 01:56 schrieb Chris Angelico:
> >>
> >>> and it's pretty convenient. In C, the nearest equivalent is passing a
> >>> number of pointers as parameters, and having the function fill out
> >>> values. Python's model is a lot closer to what you're saying than C's
> >>> model is :)
> >>
> >>
> >> At least, C functions can return structs...
> >
> > Oh, yes, I forgot about that. Thought that was C++ but not C, partly
> > because I never do it in either language. Although in a sense, a
> > struct is still a single "thing".
> >
> > ChrisA
> >
> 
> Don't forget that C functions can accept structs as input.  Possibly not 
> a good idea as I found out many years ago pre ANSIC when I forgot that 
> little old ampersand, so the compiler didn't pick it up, but then with 
> modern computers having so much memory who really cares if you burn a 
> little bit of stack on structures rather than pointers to structures?
> 
> Now does Python pass by value or by reference?  Happily sits back and 
> waits for 10**6 emails to arrive as this is discussed for the 10**6th time.
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence

People actually argue that Python passes by value?  This is easily proven wrong by passing a mutable object to a function and changing it within the function.


From liik.joonas at gmail.com  Wed Jun  3 18:33:39 2015
From: liik.joonas at gmail.com (Joonas Liik)
Date: Thu, 4 Jun 2015 01:33:39 +0300
Subject: Multiple thread program problem
In-Reply-To: <7a358948-5929-46b5-a847-13e8b9bcf46e@googlegroups.com>
References: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
 <bd3b582f-20b5-477d-b7e3-32c4e73600d0@googlegroups.com>
 <7a358948-5929-46b5-a847-13e8b9bcf46e@googlegroups.com>
Message-ID: <CAB1GNpQNCcr98YNHriw7rCSY3UyGszAMu8NeEemmQaA5FBC5vQ@mail.gmail.com>

You think "(f)" makes a tuple, but it does not.
the parentesis is not the tuple constructor, the comma is
try:
t=thread.start_new_thread(proc,(f,))
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150604/7779000b/attachment.html>

From ian.g.kelly at gmail.com  Wed Jun  3 18:36:21 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 3 Jun 2015 16:36:21 -0600
Subject: How to access the low digits of a list
In-Reply-To: <11005554-e89f-407c-8720-fae1bb3fced8@googlegroups.com>
References: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>
 <18a4709f-47c8-41f7-87d5-08bbb00166b6@googlegroups.com>
 <mailman.63.1433254823.13271.python-list@python.org>
 <11005554-e89f-407c-8720-fae1bb3fced8@googlegroups.com>
Message-ID: <CALwzidnwRmObOGhbu3GLqjNeQZx49R81YBpfNtrt90qbyw1u8Q@mail.gmail.com>

On Wed, Jun 3, 2015 at 3:08 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> On Tuesday, June 2, 2015 at 7:50:58 PM UTC+5:30, Ian wrote:
>> On Tue, Jun 2, 2015 at 6:35 AM, Rustom Mody  wrote:
>> > For that matter even this works
>> > But I am not sure whats happening or that I like it
>> >
>> >>>> [x[-2:]  for x in lines]
>> > ['12', '42', '49', '56', '25', '36', '49', '64', '81', '00']
>>
>> x[-2:] selects all items in the sequence with index i such that len(x)
>> - 2 <= i < len(x). For a sequence of length 2 or less, that's the
>> entire sequence.
>
> Thanks -- learn something
> So it means that indices can give indexerror; slices cannot?
> Seems fair enough put that way, but is visually counterintuitive

Yes. The rule I paraphrased above is stated at
https://docs.python.org/3/library/stdtypes.html#common-sequence-operations
-- scroll down to note 4. I don't know if there's anything that
clearly states that sequence slicing can't raise IndexError, but it is
at least implied by the above, and it is certainly true of all builtin
sequence types.


From bc at freeuk.com  Wed Jun  3 19:04:04 2015
From: bc at freeuk.com (BartC)
Date: Thu, 04 Jun 2015 00:04:04 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <mailman.131.1433368169.13271.python-list@python.org>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>	<48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>	<556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>	<1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>	<DxAbx.902026$_Q7.705597@fx28.am4>	<mailman.99.1433328294.13271.python-list@python.org>	<NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net> <jwFbx.736675$m15.72884@fx45.am4>
 <mailman.114.1433348990.13271.python-list@python.org>
 <98Ibx.766729$kA6.22200@fx46.am4>
 <mailman.127.1433365118.13271.python-list@python.org>
 <cpKbx.674783$ob3.28981@fx47.am4>
 <mailman.131.1433368169.13271.python-list@python.org>
Message-ID: <EJLbx.765869$qW.326303@fx20.am4>

On 03/06/2015 22:49, Mark Lawrence wrote:
> On 03/06/2015 22:33, BartC wrote:
>> On 03/06/2015 21:58, Mark Lawrence wrote:
>>
>> Not so hypothetical in my case as I have to implement a lot of this
>> stuff.
>>
>> I'm also quite interested in how Python does things. If it's a good idea
>> I'll copy it, if not I'll try and avoid it!
>
> Which implementation, cPython, Jython, IronPython...???

Mainly the language itself. But I've also been looking at the workings 
of CPython. (Also PyPy but obviously I'm not going to get anywhere 
there, although RPython sounds intriguing.)

(To be clear, I'm not implementing Python, but designing and 
implementing a separate language.

I did try tinkering with CPython, but the only thing I discovered, 
regarding its performance, is that the Windows version is probably 14% 
slower than it need be. I think due to using MS' C compiler which 
doesn't have gcc's computed gotos. But I think few people here use 
Windows so that's probably of little interest. It's not so easy to fix 
either.)

-- 
Bartc




From ethan at stoneleaf.us  Wed Jun  3 19:15:51 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 03 Jun 2015 16:15:51 -0700
Subject: Retrying to send message
In-Reply-To: <mknoib$46i$1@ger.gmane.org>
References: <87egltht87.fsf@Equus.decebal.nl>
 <mailman.108.1433338144.13271.python-list@python.org>
 <876174ix9n.fsf@Equus.decebal.nl> <556F4766.5090500@stoneleaf.us>
 <mknoib$46i$1@ger.gmane.org>
Message-ID: <556F8AA7.2000601@stoneleaf.us>

On 06/03/2015 01:37 PM, Mark Lawrence wrote:
> On 03/06/2015 19:28, Ethan Furman wrote:
>> On 06/03/2015 09:15 AM, Cecil Westerhof wrote:
>>
>>> I kept the except. I like to see the message that went wrong. ;-)
>>
>> That's fine, but then add a `raise` after you print the error so you can
>> see the reason that message failed.
>>
>
> Why bother in the first place, especially when you're developing?  Far easier to let the exception bubble up, or are we talking cross purposes?

I think Cecil means he wants to see which twitter message failed to send.  The rest of us would like to also have the error itself displayed.  ;)

--
~Ethan~


From mohan.mohta at gmail.com  Wed Jun  3 19:45:42 2015
From: mohan.mohta at gmail.com (Mohan Mohta)
Date: Wed, 3 Jun 2015 16:45:42 -0700 (PDT)
Subject: Multiple thread program problem
In-Reply-To: <mailman.136.1433370828.13271.python-list@python.org>
References: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
 <bd3b582f-20b5-477d-b7e3-32c4e73600d0@googlegroups.com>
 <7a358948-5929-46b5-a847-13e8b9bcf46e@googlegroups.com>
 <mailman.136.1433370828.13271.python-list@python.org>
Message-ID: <a51fd0dc-2aaf-491e-bfbf-e13d2a4df01f@googlegroups.com>

On Wednesday, June 3, 2015 at 5:34:31 PM UTC-5, Waffle wrote:
> You think "(f)" makes a tuple, but it does not.
> the parentesis is not the tuple constructor, the comma is
> try:
> t=thread.start_new_thread(proc,(f,))

Thanks for the pointer waffle.
The program executes now but still not the way I want it.
I think I will need to tweak it a bit as the code is executing with the same argument from the file /tmp/python/1 multiple times whereas it needs to be executed only ones but in parallel. Let me figure that out.


Once again thanks for all the help provided on this thread.



From sohcahtoa82 at gmail.com  Wed Jun  3 19:56:37 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Wed, 3 Jun 2015 16:56:37 -0700 (PDT)
Subject: Multiple thread program problem
In-Reply-To: <a51fd0dc-2aaf-491e-bfbf-e13d2a4df01f@googlegroups.com>
References: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
 <bd3b582f-20b5-477d-b7e3-32c4e73600d0@googlegroups.com>
 <7a358948-5929-46b5-a847-13e8b9bcf46e@googlegroups.com>
 <mailman.136.1433370828.13271.python-list@python.org>
 <a51fd0dc-2aaf-491e-bfbf-e13d2a4df01f@googlegroups.com>
Message-ID: <75bde6be-9b19-4bd2-8c8c-50cda0b61f9d@googlegroups.com>

On Wednesday, June 3, 2015 at 4:45:52 PM UTC-7, M2 wrote:
> On Wednesday, June 3, 2015 at 5:34:31 PM UTC-5, Waffle wrote:
> > You think "(f)" makes a tuple, but it does not.
> > the parentesis is not the tuple constructor, the comma is
> > try:
> > t=thread.start_new_thread(proc,(f,))
> 
> Thanks for the pointer waffle.
> The program executes now but still not the way I want it.
> I think I will need to tweak it a bit as the code is executing with the same argument from the file /tmp/python/1 multiple times whereas it needs to be executed only ones but in parallel. Let me figure that out.
> 
> 
> Once again thanks for all the help provided on this thread.

Check your usages of "line" and "f".  You have spots where you probably meant "line" instead of "f", and others where you have "f" where you probably meant "line".


From mohan.mohta at gmail.com  Wed Jun  3 20:04:29 2015
From: mohan.mohta at gmail.com (M2)
Date: Wed, 3 Jun 2015 17:04:29 -0700 (PDT)
Subject: Multiple thread program problem
In-Reply-To: <75bde6be-9b19-4bd2-8c8c-50cda0b61f9d@googlegroups.com>
References: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
 <bd3b582f-20b5-477d-b7e3-32c4e73600d0@googlegroups.com>
 <7a358948-5929-46b5-a847-13e8b9bcf46e@googlegroups.com>
 <mailman.136.1433370828.13271.python-list@python.org>
 <a51fd0dc-2aaf-491e-bfbf-e13d2a4df01f@googlegroups.com>
 <75bde6be-9b19-4bd2-8c8c-50cda0b61f9d@googlegroups.com>
Message-ID: <122a6965-5fe9-42bf-ba6d-393288121905@googlegroups.com>

On Wednesday, June 3, 2015 at 6:56:47 PM UTC-5, sohca... at gmail.com wrote:
> On Wednesday, June 3, 2015 at 4:45:52 PM UTC-7, M2 wrote:
> > On Wednesday, June 3, 2015 at 5:34:31 PM UTC-5, Waffle wrote:
> > > You think "(f)" makes a tuple, but it does not.
> > > the parentesis is not the tuple constructor, the comma is
> > > try:
> > > t=thread.start_new_thread(proc,(f,))
> > 
> > Thanks for the pointer waffle.
> > The program executes now but still not the way I want it.
> > I think I will need to tweak it a bit as the code is executing with the same argument from the file /tmp/python/1 multiple times whereas it needs to be executed only ones but in parallel. Let me figure that out.
> > 
> > 
> > Once again thanks for all the help provided on this thread.
> 
> Check your usages of "line" and "f".  You have spots where you probably meant "line" instead of "f", and others where you have "f" where you probably meant "line".

Here is my logic:
f is where the entire file is getting loaded
which is also passed as argument in the function proc
line has a single line from the file which is then stripped off the new line character and assigned to com2 variable which helps in using it in the subprocess.call



From cs at zip.com.au  Wed Jun  3 20:37:56 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 4 Jun 2015 10:37:56 +1000
Subject: Multiple thread program problem
In-Reply-To: <122a6965-5fe9-42bf-ba6d-393288121905@googlegroups.com>
References: <122a6965-5fe9-42bf-ba6d-393288121905@googlegroups.com>
Message-ID: <20150604003756.GA59209@cskk.homeip.net>

On 03Jun2015 17:04, M2 <mohan.mohta at gmail.com> wrote:
>On Wednesday, June 3, 2015 at 6:56:47 PM UTC-5, sohca... at gmail.com wrote:
>> On Wednesday, June 3, 2015 at 4:45:52 PM UTC-7, M2 wrote:
>> > On Wednesday, June 3, 2015 at 5:34:31 PM UTC-5, Waffle wrote:
>> > > You think "(f)" makes a tuple, but it does not.
>> > > the parentesis is not the tuple constructor, the comma is
>> > > try:
>> > > t=thread.start_new_thread(proc,(f,))
>> >
>> > Thanks for the pointer waffle.
>> > The program executes now but still not the way I want it.
>> > I think I will need to tweak it a bit as the code is executing with the same argument from the file /tmp/python/1 multiple times whereas it needs to be executed only ones but in parallel. Let me figure that out.
>> >
>> >
>> > Once again thanks for all the help provided on this thread.
>>
>> Check your usages of "line" and "f".  You have spots where you probably meant "line" instead of "f", and others where you have "f" where you probably meant "line".
>
>Here is my logic:
>f is where the entire file is getting loaded

In the main code, yes.

>which is also passed as argument in the function proc

But why? f is not using in proc. Only line is.

>line has a single line from the file which is then stripped off the new line character and assigned to com2 variable which helps in using it in the subprocess.call

That end is fine.

I would be passing only "line" to proc, not "f" at all.

Suggestion: move your main code into its own function. That will make all the 
variables in it "local". Your proc function is presently relying on "line" 
being global, which generally bad and a recipe for disaster in multithreaded 
code.

Moving the main code into its own function will (1) get rid of the global 
variables and (2) force you to consider exactly what you need to pass to 
"proc", and that will help reveal various logic issues.

Cheers,
Cameron Simpson <cs at zip.com.au>

>>>How do you blip the throttle and wave? Do you blip it real high, then wave
>>>before the revs drop back?
>>Blip = right hand; Wave = left hand.  Do both simultaneously.  QED.
>Doesnt this make the bike lurch forward thru the intersection?
Not if the disk lock is in place...
        - Dean Woodward <deanw at agora.rdrop.com>


From c.buhtz at posteo.jp  Wed Jun  3 21:08:51 2015
From: c.buhtz at posteo.jp (c.buhtz at posteo.jp)
Date: Thu, 4 Jun 2015 03:08:51 +0200
Subject: build a deb-package
Message-ID: <3m28BY1cgnzFpVh@dovecot04.posteo.de>

I am not sure where I am or where to go. ;)
So before asking about technical details I first want to know if it is
generally possible.

On Ubuntu when I install python-based applications (from a repository)
as deb-packages I can start them by just typing its name because the
py-file or a link to it are installed in bin-directory that is in the
$PATH.

I read a lot of wikis/howtows and couldn't get this done with my own
(simple hello world) application working with Python3.

E.g. I used setup.py bdist_rpm and alien to create a deb. But the
py-file is in dist-package and not in a bin-directory.

So is it possible to create a deb-file with a Python3-based application
in it that can be started over the $PATH variable?


From RussellJBrennan at gmail.com  Wed Jun  3 21:18:10 2015
From: RussellJBrennan at gmail.com (Russell Brennan)
Date: Wed, 3 Jun 2015 18:18:10 -0700
Subject: Argument Presence Checking via Identity or Boolean Operation?
Message-ID: <CAAKgPaGY_4yB7z=R-UO=s5TEWwEXZsE1nSEXw00sQPMfH_Xt2g@mail.gmail.com>

I'm going to x-post this to stackoverflow but...

When checking a method's arguments to see whether they were set, is it
pythonic to do an identity check:

def doThis(arg1, arg2=None):
  if arg2 is None:
    arg2 = myClass()


Or is it proper form to use a short-circuiting boolean:

def doThis(arg1, arg2=None):
    arg2 = arg2 or myClass()


In support of the former, PEP 8 states:

Comparisons to singletons like None should always be done with is or is not
, never the equality operators. Also, beware of writing if x when you
really mean if x is not None -- e.g. when testing whether a variable or
argument that defaults to None was set to some other value. The other value
might have a type (such as a container) that could be false in a boolean
context!


On the other hand, from the Google style guide:

Use the "implicit" false if at all possible. ...


But at the same time states...

Never use == or != to compare singletons like None. Use is or is not.


Does this apply to "None" since it evaluates to False always, and/or is a
boolean comparison equivalent to ==/!= under the hood?

Thanks much,
Russ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150603/a5b46e1c/attachment.html>

From rosuav at gmail.com  Wed Jun  3 21:52:16 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 4 Jun 2015 11:52:16 +1000
Subject: Can Python function return multiple data?
In-Reply-To: <69avma9djtmpmmncceh7euhv2b2ni5itd9@4ax.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <69avma9djtmpmmncceh7euhv2b2ni5itd9@4ax.com>
Message-ID: <CAPTjJmoA=oYeSLguEOQ8AxC7HMtPNegFS-keqk45jNy_zvU5GA@mail.gmail.com>

On Thu, Jun 4, 2015 at 11:30 AM, Dennis Lee Bieber
<wlfraed at ix.netcom.com> wrote:
> On Wed, 3 Jun 2015 15:28:56 -0700 (PDT), sohcahtoa82 at gmail.com declaimed
> the following:
>
>>
>>People actually argue that Python passes by value?  This is easily proven wrong by passing a mutable object to a function and changing it within the function.
>
>         Which only proves you passed a mutable object... Mutating the object
> means you've performed some operation that "goes inside the box".
>
>         Try assigning a new object to it rather than mutating it... IE; don't
> go inside the box, but try to change the box itself. Doesn't carry out --
> which is the traditional definition of pass-by-reference: assignment
> changes the passed-in-object.

So you've proven that Python doesn't pass by value, and that Python
doesn't pass by reference.

I'm still waiting for "pass-by-quantum-entanglement" to be a viable option.

ChrisA


From ned at nedbatchelder.com  Wed Jun  3 21:54:08 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 3 Jun 2015 18:54:08 -0700 (PDT)
Subject: Let's make Python into LISP
In-Reply-To: <mailman.132.1433368213.13271.python-list@python.org>
References: <63af3420-3c37-402e-b082-eb3a3e8345e4@googlegroups.com>
 <03a4aa85-64c1-42e9-9fb7-ea47351b6afc@googlegroups.com>
 <mailman.132.1433368213.13271.python-list@python.org>
Message-ID: <1a9422a3-a810-4567-a2d7-28188056a44d@googlegroups.com>

On Wednesday, June 3, 2015 at 5:50:24 PM UTC-4, Mark Lawrence wrote:
> On 03/06/2015 22:22, Rustom Mody wrote:
> > On Wednesday, June 3, 2015 at 4:27:39 AM UTC+5:30, Dr. Bigcock wrote:
> >> We can make Python like LISP:
> >>
> >> 1.  Make EVERYTHING the same kind of thing (call it "object").
> >> 2.  Let's make a lot of meta functions like super, instead of judicious use of interpreter impositions.
> >> 3. Forget *practicality*.  All hail *purity*.
> >>
> >> Wait.  Someone's time machine broke.
> >>
> >> Mark
> >
> > Hi Mark
> >
> > The way you keep bringing up Lisp, ?-calculus, etc completely without relevance
> > to anything, I wonder if you had a bad experience with someone teaching these
> > and thereabouts?
> >
> > <Dark Secret>
> > Everyone knows that the average student is below average.
> > More true for the average teacher
> > </Dark Secret>
> >
> > Yeah disasters (of all shapes and colors) strike and leave us scarred.
> > Isn't it more intelligent to pick up the pieces and move on than to keep
> > putting salt on the scars?
> >
> > PS If you de-expletivize your name, maybe more people will pay attention to
> > what you want to say
> >
> 
> Please stop replying to this brain dead pot smoking hippy. 

Please don't stoop to name-calling.  It isn't necessary.

--Ned.


From cs at zip.com.au  Wed Jun  3 22:54:47 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 4 Jun 2015 12:54:47 +1000
Subject: Find in ipython3
In-Reply-To: <87y4k2hyvf.fsf@Equus.decebal.nl>
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
Message-ID: <20150604025447.GA70852@cskk.homeip.net>

On 02Jun2015 18:13, Cecil Westerhof <Cecil at decebal.nl> wrote:
>I am thinking about using ipython3 instead of bash. When I want to
>find a file I can do the following:
>    !find ~ -iname '*python*.pdf'
>but is there a python way?

That succinct? Not out of the box, but something can easily be built on top of 
the os.walk() function.

Cheers,
Cameron Simpson <cs at zip.com.au>

Perl combines all of the worst aspects of BASIC, C and line noise.
        - Keith Packard <keithp at ncd.com>


From rustompmody at gmail.com  Wed Jun  3 22:59:01 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Wed, 3 Jun 2015 19:59:01 -0700 (PDT)
Subject: Let's make Python into LISP
In-Reply-To: <1a9422a3-a810-4567-a2d7-28188056a44d@googlegroups.com>
References: <63af3420-3c37-402e-b082-eb3a3e8345e4@googlegroups.com>
 <03a4aa85-64c1-42e9-9fb7-ea47351b6afc@googlegroups.com>
 <mailman.132.1433368213.13271.python-list@python.org>
 <1a9422a3-a810-4567-a2d7-28188056a44d@googlegroups.com>
Message-ID: <efd311f1-f86d-436f-acbb-beab4b170e36@googlegroups.com>

On Thursday, June 4, 2015 at 7:24:19 AM UTC+5:30, Ned Batchelder wrote:
> On Wednesday, June 3, 2015 at 5:50:24 PM UTC-4, Mark Lawrence wrote:
> > On 03/06/2015 22:22, Rustom Mody wrote:
> > > On Wednesday, June 3, 2015 at 4:27:39 AM UTC+5:30, Dr. Bigcock wrote:
> > >> We can make Python like LISP:
> > >>
> > >> 1.  Make EVERYTHING the same kind of thing (call it "object").
> > >> 2.  Let's make a lot of meta functions like super, instead of judicious use of interpreter impositions.
> > >> 3. Forget *practicality*.  All hail *purity*.
> > >>
> > >> Wait.  Someone's time machine broke.
> > >>
> > >> Mark
> > >
> > > Hi Mark
> > >
> > > The way you keep bringing up Lisp, ?-calculus, etc completely without relevance
> > > to anything, I wonder if you had a bad experience with someone teaching these
> > > and thereabouts?
> > >
> > > <Dark Secret>
> > > Everyone knows that the average student is below average.
> > > More true for the average teacher
> > > </Dark Secret>
> > >
> > > Yeah disasters (of all shapes and colors) strike and leave us scarred.
> > > Isn't it more intelligent to pick up the pieces and move on than to keep
> > > putting salt on the scars?
> > >
> > > PS If you de-expletivize your name, maybe more people will pay attention to
> > > what you want to say
> > >
> > 
> > Please stop replying to this brain dead pot smoking hippy. 
> 
> Please don't stoop to name-calling.  It isn't necessary.

Pot-Kettle?

Others have noted that Mark Lawrence's contributions here are in fringe-spam category:

https://mail.python.org/pipermail/python-list/2015-March/687314.html
https://mail.python.org/pipermail/python-list/2014-October/679909.html


From mohan.mohta at gmail.com  Wed Jun  3 22:59:23 2015
From: mohan.mohta at gmail.com (M2)
Date: Wed, 3 Jun 2015 19:59:23 -0700 (PDT)
Subject: Multiple thread program problem
In-Reply-To: <mailman.139.1433378292.13271.python-list@python.org>
References: <122a6965-5fe9-42bf-ba6d-393288121905@googlegroups.com>
 <mailman.139.1433378292.13271.python-list@python.org>
Message-ID: <bdd7aa19-1dca-4fc4-a998-c6fd6997375c@googlegroups.com>

On Wednesday, June 3, 2015 at 7:38:22 PM UTC-5, Cameron Simpson wrote:
> On 03Jun2015 17:04, M2 <mohan.mohta at gmail.com> wrote:
> >On Wednesday, June 3, 2015 at 6:56:47 PM UTC-5, sohca... at gmail.com wrote:
> >> On Wednesday, June 3, 2015 at 4:45:52 PM UTC-7, M2 wrote:
> >> > On Wednesday, June 3, 2015 at 5:34:31 PM UTC-5, Waffle wrote:
> >> > > You think "(f)" makes a tuple, but it does not.
> >> > > the parentesis is not the tuple constructor, the comma is
> >> > > try:
> >> > > t=thread.start_new_thread(proc,(f,))
> >> >
> >> > Thanks for the pointer waffle.
> >> > The program executes now but still not the way I want it.
> >> > I think I will need to tweak it a bit as the code is executing with the same argument from the file /tmp/python/1 multiple times whereas it needs to be executed only ones but in parallel. Let me figure that out.
> >> >
> >> >
> >> > Once again thanks for all the help provided on this thread.
> >>
> >> Check your usages of "line" and "f".  You have spots where you probably meant "line" instead of "f", and others where you have "f" where you probably meant "line".
> >
> >Here is my logic:
> >f is where the entire file is getting loaded
> 
> In the main code, yes.
> 
> >which is also passed as argument in the function proc
> 
> But why? f is not using in proc. Only line is.
> 
> >line has a single line from the file which is then stripped off the new line character and assigned to com2 variable which helps in using it in the subprocess.call
> 
> That end is fine.
> 
> I would be passing only "line" to proc, not "f" at all.
> 
> Suggestion: move your main code into its own function. That will make all the 
> variables in it "local". Your proc function is presently relying on "line" 
> being global, which generally bad and a recipe for disaster in multithreaded 
> code.
> 
> Moving the main code into its own function will (1) get rid of the global 
> variables and (2) force you to consider exactly what you need to pass to 
> "proc", and that will help reveal various logic issues.
> 
> Cheers,
> Cameron Simpson <cs at zip.com.au>
> 
> >>>How do you blip the throttle and wave? Do you blip it real high, then wave
> >>>before the revs drop back?
> >>Blip = right hand; Wave = left hand.  Do both simultaneously.  QED.
> >Doesnt this make the bike lurch forward thru the intersection?
> Not if the disk lock is in place...
>         - Dean Woodward <deanw at agora.rdrop.com>

Thanks Cameron.
I do not see the duplication in the execution now.
I do see it is not consistent by executing all the threads ; it might be due to the fact I am using 
        subprocess.call(co,shell=True) 
Per my understanding the above does not keep track of threads it just spawns a thread and leaves it there.
I might need to use the function start(), join() to ensure it picks up all the argument

For the record now my new code is 
#! /usr/bin/python
import os
import subprocess
import thread
import threading
import sys
from thread import start_new_thread
 
def proc(col) :
        subprocess.call(col,shell=True)
        return
 
f = open('/tmp/python/1')
for line in f:
        com1="ssh -B "
        com2=line.strip('\n')
        com3= " uname -a  "
        co=str("ssh -B ")+ str(com2) + str(" uname -a")
        t=thread.start_new_thread(proc,(co,))
f.close()


Thanks again for the help


From cs at zip.com.au  Thu Jun  4 00:42:22 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 4 Jun 2015 14:42:22 +1000
Subject: Multiple thread program problem
In-Reply-To: <bdd7aa19-1dca-4fc4-a998-c6fd6997375c@googlegroups.com>
References: <bdd7aa19-1dca-4fc4-a998-c6fd6997375c@googlegroups.com>
Message-ID: <20150604044222.GA3032@cskk.homeip.net>

On 03Jun2015 19:59, M2 <mohan.mohta at gmail.com> wrote:
>On Wednesday, June 3, 2015 at 7:38:22 PM UTC-5, Cameron Simpson wrote:
>> I would be passing only "line" to proc, not "f" at all.
>>
>> Suggestion: move your main code into its own function. That will make all the
>> variables in it "local". Your proc function is presently relying on "line"
>> being global, which generally bad and a recipe for disaster in multithreaded
>> code.
>>
>> Moving the main code into its own function will (1) get rid of the global
>> variables and (2) force you to consider exactly what you need to pass to
>> "proc", and that will help reveal various logic issues.
[...]
>
>Thanks Cameron.
>I do not see the duplication in the execution now.
>I do see it is not consistent by executing all the threads ; it might be due to the fact I am using
>        subprocess.call(co,shell=True)
>Per my understanding the above does not keep track of threads it just spawns a thread and leaves it there.

subprocess does not start a thread at all. It starts a new external process.  
Your variable "t" is the Thread.

>I might need to use the function start(), join() to ensure it picks up all the argument

Because you have called .start_new_thread, you do not need to call .start().

Whether you call .join() on the Thread later is up to you. When you call 
.join(), the caller will block until the Thread terminates. So you won't want 
to do that immediately if you hope to run several subprocesses in parallel with 
this setup.

Thank you for including your current code; most helpful. I have some other 
remarks, below the code which I'm keeping here for reference:

>For the record now my new code is
>#! /usr/bin/python
>import os
>import subprocess
>import thread
>import threading
>import sys
>from thread import start_new_thread
>
>def proc(col) :
>        subprocess.call(col,shell=True)
>        return
>
>f = open('/tmp/python/1')
>for line in f:
>        com1="ssh -B "
>        com2=line.strip('\n')
>        com3= " uname -a  "
>        co=str("ssh -B ")+ str(com2) + str(" uname -a")
>        t=thread.start_new_thread(proc,(co,))
>f.close()

First up, your open/read/close stuff is normally written like this in Python:

  with open('/tmp/python/1') as f:
    for line in f:
      ... loop body here ...

This is because the return from open() is in fact a "context manager", an 
object designed to work with the "with" statement. Specificly, the "exit" step 
of this particular context manager calls .close() for you. Aside from being 
more succinct, a context manager has the other advantage that the exit action 
is _always_ called when control leaves the scope of the "with". Consider this 
example function:

  def foo(filename):
    with open(filename) as f:
      for line in f:
        ... do stuff with line ...
        if "quit" in line:
          return

That will return from the function if the string "quit" occurs on one of the 
lines in the file, and not process any following lines. The important point 
here is that if you use "with" then the open file will be closed automatically 
when the return happens. With your original open/loop/close code, the "return" 
would bypass the .close() call, leaving the file open.

The next remark I would make is that while Threads are very nice (I use them a 
lot), if all your thread is doing is dispatching a subprocess then you do not 
need a thread. The separate process that is made by subprocess runs 
automatically, immediately, without waiting for your program. So you can invoke 
a bunch of subprocesses in parallel, and not go near a thread.

However, because you have invoked the subprocess with .call(), your "proc" 
function inherently waits for the subprocess to complete before returning and 
therefore you need a thread to do these in parallel.

The .call() function is a convenience function; the alternative is to use the 
.Popen() constructor directly:

  def proc(col):
    P = subprocess.Popen(col, shell=True)
    return P

Then you can have your main loop replace "t=..." with:

  P = proc(co)

At this point, proc starts the subprocess but does not wait for it (versus 
"call", which does the wait for you). So you can dispatch all these 
subprocesses in parallel. Then after your loop you can wait for them to finish 
at your leisure.

Cheers,
Cameron Simpson <cs at zip.com.au>

Nonsense. Space is blue, and birds fly through it.      - Heisenberg


From random832 at fastmail.us  Thu Jun  4 01:01:55 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 04 Jun 2015 01:01:55 -0400
Subject: Can Python function return multiple data?
In-Reply-To: <mknt6h$epv$1@ger.gmane.org>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mknt6h$epv$1@ger.gmane.org>
Message-ID: <1433394115.3590440.286441281.65226C5D@webmail.messagingengine.com>

On Wed, Jun 3, 2015, at 17:56, Mark Lawrence wrote:
> Now does Python pass by value or by reference?  Happily sits back and 
> waits for 10**6 emails to arrive as this is discussed for the 10**6th
> time.

Python's in that same awkward space as Java, where it technically passes
by value, but the values are usually pointers, so passing a mutable
object (such as a list) allows the function to modify the caller's state
in a way that many people associate exclusively with by-reference.

Have I hit all the high points?


From Cecil at decebal.nl  Thu Jun  4 01:09:53 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 04 Jun 2015 07:09:53 +0200
Subject: Find in ipython3
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.142.1433388453.13271.python-list@python.org>
Message-ID: <87vbf4giu6.fsf@Equus.decebal.nl>

Op Thursday 4 Jun 2015 04:54 CEST schreef Cameron Simpson:

> On 02Jun2015 18:13, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> I am thinking about using ipython3 instead of bash. When I want to
>> find a file I can do the following:
>> !find ~ -iname '*python*.pdf'
>> but is there a python way?
>
> That succinct? Not out of the box, but something can easily be built
> on top of the os.walk() function.

OK, I will write it. And something to implement du.

Thanks.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Thu Jun  4 01:13:23 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 04 Jun 2015 07:13:23 +0200
Subject: Retrying to send message
References: <87egltht87.fsf@Equus.decebal.nl>
 <mailman.108.1433338144.13271.python-list@python.org>
 <876174ix9n.fsf@Equus.decebal.nl> <556F4766.5090500@stoneleaf.us>
 <mknoib$46i$1@ger.gmane.org>
 <mailman.138.1433373373.13271.python-list@python.org>
Message-ID: <87r3psgioc.fsf@Equus.decebal.nl>

Op Thursday 4 Jun 2015 01:15 CEST schreef Ethan Furman:

> On 06/03/2015 01:37 PM, Mark Lawrence wrote:
>> On 03/06/2015 19:28, Ethan Furman wrote:
>>> On 06/03/2015 09:15 AM, Cecil Westerhof wrote:
>>>
>>>> I kept the except. I like to see the message that went wrong. ;-)
>>>
>>> That's fine, but then add a `raise` after you print the error so
>>> you can see the reason that message failed.
>>>
>>
>> Why bother in the first place, especially when you're developing?
>> Far easier to let the exception bubble up, or are we talking cross
>> purposes?
>
> I think Cecil means he wants to see which twitter message failed to
> send. The rest of us would like to also have the error itself
> displayed. ;)

Correct. I display the error in the function I call. I also quit the
program in the function I call. Is the correct thing to do, but if
someone just want to continue after displaying the error, that should
be possible also. That is why I choose to implement it this way.

I should produce some docstring. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From edd.cowan at gmail.com  Thu Jun  4 01:43:13 2015
From: edd.cowan at gmail.com (Eddilbert Macharia)
Date: Wed, 3 Jun 2015 22:43:13 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <fea7e1f3-6846-42e7-a119-89c2e022c43f@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
 <DxAbx.902026$_Q7.705597@fx28.am4>
 <mailman.99.1433328294.13271.python-list@python.org>
 <NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net> <jwFbx.736675$m15.72884@fx45.am4>
 <mailman.114.1433348990.13271.python-list@python.org>
 <98Ibx.766729$kA6.22200@fx46.am4>
 <mailman.127.1433365118.13271.python-list@python.org>
 <fea7e1f3-6846-42e7-a119-89c2e022c43f@googlegroups.com>
Message-ID: <83d7d7ff-82bb-4d09-a3b3-cf6028f088df@googlegroups.com>

can you please stick to the point.take your differences else where.please.....stay on target...some of us are learning... its annoying and tiring to have to read the insults and innuendo


From cs at zip.com.au  Thu Jun  4 01:43:39 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 4 Jun 2015 15:43:39 +1000
Subject: Find in ipython3
In-Reply-To: <87vbf4giu6.fsf@Equus.decebal.nl>
References: <87vbf4giu6.fsf@Equus.decebal.nl>
Message-ID: <20150604054339.GA79573@cskk.homeip.net>

On 04Jun2015 07:09, Cecil Westerhof <Cecil at decebal.nl> wrote:
>Op Thursday 4 Jun 2015 04:54 CEST schreef Cameron Simpson:
>
>> On 02Jun2015 18:13, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>> I am thinking about using ipython3 instead of bash. When I want to
>>> find a file I can do the following:
>>> !find ~ -iname '*python*.pdf'
>>> but is there a python way?
>>
>> That succinct? Not out of the box, but something can easily be built
>> on top of the os.walk() function.
>
>OK, I will write it. And something to implement du.
>Thanks.

Have fun.

For added points, do not forget that du knows about hardlinks and only counts a 
file once.  And that it counts the blocks field, not the byte size (think: 
sparse files).

Cheers,
Cameron Simpson <cs at zip.com.au>

Thermochromic ink is the Pet Rock ink of the New Millennium.
- overhead by WIRED at the Intelligent Printing conference Oct2006


From storchaka at gmail.com  Thu Jun  4 02:56:38 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 04 Jun 2015 09:56:38 +0300
Subject: Can Python function return multiple data?
In-Reply-To: <CAPTjJmp6B0hX5TDCTOT60e4jREeceJn5G95ZtwpbifqVKXhdLA@mail.gmail.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <CAPTjJmp6B0hX5TDCTOT60e4jREeceJn5G95ZtwpbifqVKXhdLA@mail.gmail.com>
Message-ID: <mkosr7$mb1$1@ger.gmane.org>

On 03.06.15 02:56, Chris Angelico wrote:
> On Wed, Jun 3, 2015 at 7:27 AM, fl <rxjwg98 at gmail.com> wrote:
>> I just see the tutorial says Python can return value in function, it does
>> not say multiple data results return situation. In C, it is possible.
>> How about Python on a multiple data return requirement?
>
> Technically, neither C nor Python can return multiple values from a
> single function call. In Python, the most common way to do this is to
> return a tuple, which can then be unpacked; as other posts in this
> thread have shown, this can look a lot like returning multiple values,
> and it's pretty convenient. In C, the nearest equivalent is passing a
> number of pointers as parameters, and having the function fill out
> values. Python's model is a lot closer to what you're saying than C's
> model is :)

Closer modeling of C's model in python is to pass an object or a list as 
an argument and set their attributes or elements in the function.

def f(a, b):
     a.x = 4.3
     a.y = 1.7
     b[:] = [45, 67]
     return 3




From mihamina.rakotomandimby at rktmb.org  Thu Jun  4 03:24:39 2015
From: mihamina.rakotomandimby at rktmb.org (Mihamina Rakotomandimby)
Date: Thu, 04 Jun 2015 10:24:39 +0300
Subject: build a deb-package
In-Reply-To: <3m28BY1cgnzFpVh@dovecot04.posteo.de>
References: <3m28BY1cgnzFpVh@dovecot04.posteo.de>
Message-ID: <556FFD37.8080602@rktmb.org>



On 06/04/2015 04:08 AM, c.buhtz at posteo.jp wrote:
> On Ubuntu when ...
> ... I used setup.py bdist_rpm and alien to create a deb.

You got it the hard way :-)
Why wont you try another way?
Here are some links from my bookmarks:
http://www.electricmonk.nl/log/2011/09/06/creating-simple-debian-packages/
https://wiki.debian.org/IntroDebianPackaging

Another solution is to subscribe to
https://lists.debian.org/debian-python/
https://lists.debian.org/debian-mentors/

Regards


From ben+python at benfinney.id.au  Thu Jun  4 03:39:56 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 04 Jun 2015 17:39:56 +1000
Subject: Argument Presence Checking via Identity or Boolean Operation?
References: <CAAKgPaGY_4yB7z=R-UO=s5TEWwEXZsE1nSEXw00sQPMfH_Xt2g@mail.gmail.com>
Message-ID: <85k2vjvs4z.fsf@benfinney.id.au>

Russell Brennan <RussellJBrennan at gmail.com> writes:

> I'm going to x-post this to stackoverflow but...
>
> When checking a method's arguments to see whether they were set, is it
> pythonic to do an identity check:
>
> def doThis(arg1, arg2=None):
>   if arg2 is None:
>     arg2 = myClass()

That is the Pythonic way to test whether an argument was set.

> In support of the former, PEP 8 states:
>
> Comparisons to singletons like None should always be done with is or
> is not , never the equality operators. Also, beware of writing if x
> when you really mean if x is not None -- e.g. when testing whether a
> variable or argument that defaults to None was set to some other
> value. The other value might have a type (such as a container) that
> could be false in a boolean context!

Yes. That's good advice.

> On the other hand, from the Google style guide:
>
> Use the "implicit" false if at all possible. ...

That's an over-statement. I'd say it is a bug in the document; ?if not
otherwise contradicted by the style guide? is better.

What they are likely warning against is code like this::

    if foo is False:
        # ?

which should instead be written::

    if not foo:
        # ?

If the guide is not explicit about that, it should be.

> But at the same time states...
>
> Never use == or != to compare singletons like None. Use is or is not.
>
>
> Does this apply to "None" since it evaluates to False always, and/or is a
> boolean comparison equivalent to ==/!= under the hood?

No, it applies to None because it is a singleton and designed to be a
sentinel value.

-- 
 \         ?A fine is a tax for doing wrong. A tax is a fine for doing |
  `\                                                 well.? ?anonymous |
_o__)                                                                  |
Ben Finney



From __peter__ at web.de  Thu Jun  4 04:07:51 2015
From: __peter__ at web.de (Peter Otten)
Date: Thu, 04 Jun 2015 10:07:51 +0200
Subject: Argument Presence Checking via Identity or Boolean Operation?
References: <CAAKgPaGY_4yB7z=R-UO=s5TEWwEXZsE1nSEXw00sQPMfH_Xt2g@mail.gmail.com>
Message-ID: <mkp10p$n0l$1@ger.gmane.org>

Russell Brennan wrote:

> I'm going to x-post this to stackoverflow but...
> 
> When checking a method's arguments to see whether they were set, is it
> pythonic to do an identity check:
> 
> def doThis(arg1, arg2=None):
>   if arg2 is None:
>     arg2 = myClass()
> 
> 
> Or is it proper form to use a short-circuiting boolean:
> 
> def doThis(arg1, arg2=None):
>     arg2 = arg2 or myClass()

When I read this I always have to stop and consider whether there are valid 
falsey arguments that could be passed as arg2.

An obvious example is

def do_this(arg=None):
    arg = arg or []
    arg.append(42)

a = []
do_this(a)
do_this(a)
print(a) # expected: [42, 42] actual output: []

Choosing the first approach is not just a matter of style, it avoids a 
source of subtle bugs.



From frank at chagford.com  Thu Jun  4 04:34:28 2015
From: frank at chagford.com (Frank Millman)
Date: Thu, 4 Jun 2015 10:34:28 +0200
Subject: Argument Presence Checking via Identity or Boolean Operation?
References: <CAAKgPaGY_4yB7z=R-UO=s5TEWwEXZsE1nSEXw00sQPMfH_Xt2g@mail.gmail.com>
 <mkp10p$n0l$1@ger.gmane.org>
Message-ID: <mkp2ie$glr$1@ger.gmane.org>


"Peter Otten" <__peter__ at web.de> wrote in message 
news:mkp10p$n0l$1 at ger.gmane.org...
> Russell Brennan wrote:
>
>> I'm going to x-post this to stackoverflow but...
>>
>> When checking a method's arguments to see whether they were set, is it
>> pythonic to do an identity check:
>>
>> def doThis(arg1, arg2=None):
>>   if arg2 is None:
>>     arg2 = myClass()
>>
>>
>> Or is it proper form to use a short-circuiting boolean:
>>
>> def doThis(arg1, arg2=None):
>>     arg2 = arg2 or myClass()
>
> When I read this I always have to stop and consider whether there are 
> valid
> falsey arguments that could be passed as arg2.
>
> An obvious example is
>
> def do_this(arg=None):
>    arg = arg or []
>    arg.append(42)
>
> a = []
> do_this(a)
> do_this(a)
> print(a) # expected: [42, 42] actual output: []
>
> Choosing the first approach is not just a matter of style, it avoids a
> source of subtle bugs.
>

I have a slight variation in that I want to keep a reference to the 
argument -

def __init__(self, arg=None):
    self.arg = arg or []

Based on your comment, I have changed it to -

def __init__(self, arg=None):
    self.arg = [] if arg is None else arg

Does this pass your 'stop and consider' test?

Frank Millman





From __peter__ at web.de  Thu Jun  4 05:28:45 2015
From: __peter__ at web.de (Peter Otten)
Date: Thu, 04 Jun 2015 11:28:45 +0200
Subject: Argument Presence Checking via Identity or Boolean Operation?
References: <CAAKgPaGY_4yB7z=R-UO=s5TEWwEXZsE1nSEXw00sQPMfH_Xt2g@mail.gmail.com>
 <mkp10p$n0l$1@ger.gmane.org> <mkp2ie$glr$1@ger.gmane.org>
Message-ID: <mkp5oe$55s$1@ger.gmane.org>

Frank Millman wrote:

> I have a slight variation in that I want to keep a reference to the
> argument -
> 
> def __init__(self, arg=None):
>     self.arg = arg or []
> 
> Based on your comment, I have changed it to -
> 
> def __init__(self, arg=None):
>     self.arg = [] if arg is None else arg
> 
> Does this pass your 'stop and consider' test?

Yes, that avoids the problem of the original code.




From lac at openend.se  Thu Jun  4 06:06:18 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 04 Jun 2015 12:06:18 +0200
Subject: Everything is an object in python - object class and type class
In-Reply-To: Message from BartC <bc@freeuk.com>
 of "Thu, 04 Jun 2015 00:04:04 +0100." <EJLbx.765869$qW.326303@fx20.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
 <DxAbx.902026$_Q7.705597@fx28.am4>
 <mailman.99.1433328294.13271.python-list@python.org>
 <NeBbx.946747$wk1.662297@fx19.am4> <87fv6956zw.fsf@elektro.pacujo.net>
 <jwFbx.736675$m15.72884@fx45.am4>
 <mailman.114.1433348990.13271.python-list@python.org>
 <98Ibx.766729$kA6.22200@fx46.am4>
 <mailman.127.1433365118.13271.python-list@python.org>
 <cpKbx.674783$ob3.28981@fx47.am4>
 <mailman.131.1433368169.13271.python-list@python.org><EJLbx.765869$qW.326303@fx20.am4>
Message-ID: <201506041006.t54A6IRk027702@fido.openend.se>

In a message of Thu, 04 Jun 2015 00:04:04 +0100, BartC writes:
>Mainly the language itself. But I've also been looking at the workings 
>of CPython. (Also PyPy but obviously I'm not going to get anywhere 
>there, although RPython sounds intriguing.)

Why not?  We built the thing for people like you who want to design
their own language.  This makes me sad.  What are we doing wrong
that you think you won't get anywhere there?

Laura



From lac at openend.se  Thu Jun  4 06:50:37 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 04 Jun 2015 12:50:37 +0200
Subject: Keypress Input
In-Reply-To: Message from Laura Creighton <lac@openend.se> of "Wed,
 03 Jun 2015 20:59:04 +0200." <201506031859.t53Ix49H007428@fido.openend.se>
References: <rBHbx.75089$MO7.40532@fx10.iad><201506031859.t53Ix49H007428@fido.openend.se>
Message-ID: <201506041050.t54AobBI029472@fido.openend.se>

In a message of Wed, 03 Jun 2015 20:59:04 +0200, Laura Creighton writes:
>Tkinter runs on raspberry pi.
>
>Get it installed, and then run this program.
>
>from Tkinter import *
>root = Tk()
>prompt = 'Press any key. Remember to keep your mouse in the cyan box. '
>lab = Label(root, text=prompt, width=len(prompt), bg='cyan')
>lab.pack()
>
>def key(event):
>    msg = 'event.char is %r and event.keysym is %r' % (event.char, event.keysym)
>        lab.config(text=msg)
>
>root.bind_all('<Key>', key)
>root.mainloop() 
>
>Now you will have to bind the various keys to do what it is you want.
>You need to read online dociumentation for tkinter to learn how to do
>this, as well as how to use tkinter in general.
>
>Laura


I was in a hurry last night.  What I mean is -- Python out of the box
is not in the business of detecting key input.  You normally use a
Gui kit for that.  There are lots of them, and they all do things
sort of the same but slightly differently.  So what you need is
a gui kit that runs on Raspberry Pi.  If you are already using a
gui, then it probably will do it for you; check its documentation.
However, if you aren't using one ...

I don't have a Raspberry Pi, but I have read that tkinter runs there.
We can check.  First make sure that tkinter is installed and run that
short program I posted (also included above). If it works -- every time
you press a key char, it is supposed to tell you what you typed -- then
you are in business.  I just bound pressing every key with telling you that
it got pressed.  You will need to bind particular keys to do particular
things, which you can read about in tkinter documentation about keybinding.

Tkinter is python wrapped around tk, so this page
http://www.tcl.tk/man/tcl8.6/TkCmd/keysyms.htm

lists all the keys you can bind.  Not every OS provides the ability to
bind every key.  You are advised to try the keys you want to bind with
Raspberry Pi -- not having one I cannot check this for you.

Laura


From nispray at gmail.com  Thu Jun  4 07:58:23 2015
From: nispray at gmail.com (Wesley)
Date: Thu, 4 Jun 2015 04:58:23 -0700 (PDT)
Subject: Get html DOM tree by only basic builtin moudles
Message-ID: <099a955d-134a-46d6-bdba-61ec2b1eb44f@googlegroups.com>

Hi guys,
  I know there are many modules(builtin or not, e.g. beautifulsoup,xml,lxml,htmlparser .etc) to parse html files and output the DOM tree. However, if there is any better way to get the DOM tree without using those html/xml related modules? I mean, just by some general standard modules, e.g. file operations, re module .etc

Input file is something like this:
<html> 
  <head> 
    <title>DOM Tree test</title> 
  </head> 
  <body> 
    <h1>Header 1</h1> 
    <p>Hello world!</p> 
  </body>
</html>

Need the dom tree or just something like:
html -- head -- title(DOM Tree test)
html -- body -- h1(Header 1)
html -- body -- p(Hello world!)

Thanks.
Wesley


From bc at freeuk.com  Thu Jun  4 08:01:00 2015
From: bc at freeuk.com (BartC)
Date: Thu, 04 Jun 2015 13:01:00 +0100
Subject: Everything is an object in python - object class and type class
In-Reply-To: <mailman.155.1433412387.13271.python-list@python.org>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
 <DxAbx.902026$_Q7.705597@fx28.am4>
 <mailman.99.1433328294.13271.python-list@python.org>
 <NeBbx.946747$wk1.662297@fx19.am4> <87fv6956zw.fsf@elektro.pacujo.net>
 <jwFbx.736675$m15.72884@fx45.am4>
 <mailman.114.1433348990.13271.python-list@python.org>
 <98Ibx.766729$kA6.22200@fx46.am4>
 <mailman.127.1433365118.13271.python-list@python.org>
 <cpKbx.674783$ob3.28981@fx47.am4>
 <mailman.131.1433368169.13271.python-list@python.org><EJLbx.765869$qW.326303@fx20.am4>
 <mailman.155.1433412387.13271.python-list@python.org>
Message-ID: <16Xbx.946577$396.102361@fx12.am4>

On 04/06/2015 11:06, Laura Creighton wrote:
> In a message of Thu, 04 Jun 2015 00:04:04 +0100, BartC writes:
>> Mainly the language itself. But I've also been looking at the workings
>> of CPython. (Also PyPy but obviously I'm not going to get anywhere
>> there, although RPython sounds intriguing.)
>
> Why not?  We built the thing for people like you who want to design
> their own language.

(I thought you wrote bookkeeping systems?)

> This makes me sad.  What are we doing wrong
> that you think you won't get anywhere there?

It's not that you're doing anything wrong. I'm just used to using my own 
languages and to implementing 100% of them myself. (I have one that does 
the job of C, and another that takes over the role of Python, which are 
the two I'd otherwise be using.)

In the case of the interpreted one, I write streamlined but otherwise 
unremarkable bytecode interpreters that can (when I add a bit of ASM) 
just about compete with PyPy on small benchmarks.

(It is this latter language that I am upgrading to be more dynamic, to 
make the comparisons fairer, and see if I can still maintain 
performance. But I do believe a lot can be achieved with a more careful 
language design.)

To make use of PyPy, I understand that I have to code my interpreter in 
RPython, with various hints, and it is the execution paths in this code 
that are somehow optimised at runtime.

However, I've seen at the video someone posted of the keynote talk about 
PyPy (https://youtu.be/l_HBRhcgeuQ), and it does look a rather 
intimidating process (apparently taking several hours to compile the 
smallest code tweak; currently it takes me about 1 second to try 
something new).

Also, I'm using Windows where even ordinary Python development doesn't 
really work; I had to use Ubuntu (a completely alien environment to me) 
to attempt compiling Cpython).

-- 
Bartc


From lac at openend.se  Thu Jun  4 09:01:35 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 04 Jun 2015 15:01:35 +0200
Subject: Get html DOM tree by only basic builtin moudles
In-Reply-To: Message from Wesley <nispray@gmail.com> of "Thu,
 04 Jun 2015 04:58:23 -0700."
 <099a955d-134a-46d6-bdba-61ec2b1eb44f@googlegroups.com>
References: <099a955d-134a-46d6-bdba-61ec2b1eb44f@googlegroups.com>
Message-ID: <201506041301.t54D1ZOT032529@fido.openend.se>

Elementtree is part of the Python standard library.  You are better off
using it than rolling your own.   (If you were one of the rare people who
have some very strange requirements that make you better off writing your
own, you wouldn't be asking us.  You'd already know.)

https://docs.python.org/2.7/library/xml.etree.elementtree.html#module-xml.etree.ElementTree
https://docs.python.org/3.4/library/xml.etree.elementtree.html


From steve at pearwood.info  Thu Jun  4 09:27:33 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 04 Jun 2015 23:27:33 +1000
Subject: How to access the low digits of a list
References: <efe9d48e-9aaf-4e68-aa26-c83da32e80f5@googlegroups.com>
 <18a4709f-47c8-41f7-87d5-08bbb00166b6@googlegroups.com>
 <mailman.63.1433254823.13271.python-list@python.org>
 <11005554-e89f-407c-8720-fae1bb3fced8@googlegroups.com>
Message-ID: <55705246$0$12991$c3e8da3$5496439d@news.astraweb.com>

On Thu, 4 Jun 2015 07:08 am, Rustom Mody wrote:

> So it means that indices can give indexerror; slices cannot?

If you write your own class with a __getitem__ method, you can have it do
anything you like, including raise an exception.

Built-in sequence types like list, str and tuple, however, allow slicing to
extend beyond the first and last indexes without error.



-- 
Steven



From palpandi111 at gmail.com  Thu Jun  4 09:36:29 2015
From: palpandi111 at gmail.com (Palpandi)
Date: Thu, 4 Jun 2015 06:36:29 -0700 (PDT)
Subject: Regular Expression
Message-ID: <c85bc324-37fe-469e-b3b1-b1d4e51bf7d8@googlegroups.com>

Hi All,

This is the case. To split "string2" from "string1_string2" I am using 
re.split('_', "string1_string2", 1)[1].

It is working fine for string "string1_string2" and output as "string2". But actually the problem is that if a sting is "__string1_string2" and the output is "_string1_string2". It is wrong.

How to fix this issue?


From steve at pearwood.info  Thu Jun  4 09:38:00 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 04 Jun 2015 23:38:00 +1000
Subject: Argument Presence Checking via Identity or Boolean Operation?
References: <mailman.149.1433401647.13271.python-list@python.org>
Message-ID: <557054b8$0$12983$c3e8da3$5496439d@news.astraweb.com>

On Thu, 4 Jun 2015 11:18 am, Russell Brennan wrote:

> I'm going to x-post this to stackoverflow but...
> 
> When checking a method's arguments to see whether they were set, is it
> pythonic to do an identity check:
> 
> def doThis(arg1, arg2=None):
>   if arg2 is None:
>     arg2 = myClass()
> 
> 
> Or is it proper form to use a short-circuiting boolean:
> 
> def doThis(arg1, arg2=None):
>     arg2 = arg2 or myClass()


Those two checks have different semantics, so the answer depends on what
behaviour you want.

Do you want *any* of None, 0, 0.0, [], (), {}, False, etc. to trigger the
myClass() behaviour? Then the right answer is to use a short-circuit
boolean. (And you probably should use False as the default for arg2.)

Do you want *only* None to trigger the myClass() behavious? Then the right
answer is to use the identity check.




-- 
Steven



From lac at openend.se  Thu Jun  4 09:39:00 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 04 Jun 2015 15:39:00 +0200
Subject: Everything is an object in python - object class and type class
In-Reply-To: Message from BartC <bc@freeuk.com>
 of "Thu, 04 Jun 2015 13:01:00 +0100." <16Xbx.946577$396.102361@fx12.am4>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
 <48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>
 <556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>
 <DxAbx.902026$_Q7.705597@fx28.am4>
 <mailman.99.1433328294.13271.python-list@python.org>
 <NeBbx.946747$wk1.662297@fx19.am4> <87fv6956zw.fsf@elektro.pacujo.net>
 <jwFbx.736675$m15.72884@fx45.am4>
 <mailman.114.1433348990.13271.python-list@python.org>
 <98Ibx.766729$kA6.22200@fx46.am4>
 <mailman.127.1433365118.13271.python-list@python.org>
 <cpKbx.674783$ob3.28981@fx47.am4>
 <mailman.131.1433368169.13271.python-list@python.org><EJLbx.765869$qW.326303@fx20.am4>
 <mailman.155.1433412387.13271.python-list@python.org><16Xbx.946577$396.102361@fx12.am4>
Message-ID: <201506041339.t54Dd0s4000899@fido.openend.se>

In a message of Thu, 04 Jun 2015 13:01:00 +0100, BartC writes:
>On 04/06/2015 11:06, Laura Creighton wrote:
>> In a message of Thu, 04 Jun 2015 00:04:04 +0100, BartC writes:
>>> Mainly the language itself. But I've also been looking at the workings
>>> of CPython. (Also PyPy but obviously I'm not going to get anywhere
>>> there, although RPython sounds intriguing.)
>>
>> Why not?  We built the thing for people like you who want to design
>> their own language.
>
>(I thought you wrote bookkeeping systems?)

Yes, but I have to do something for fun, now, too don't I?  There is only
so much fun you can get out of a bookkeeping system.

>> This makes me sad.  What are we doing wrong
>> that you think you won't get anywhere there?

snip

>To make use of PyPy, I understand that I have to code my interpreter in 
>RPython, with various hints, and it is the execution paths in this code 
>that are somehow optimised at runtime.
>
>However, I've seen at the video someone posted of the keynote talk about 
>PyPy (https://youtu.be/l_HBRhcgeuQ), and it does look a rather 
>intimidating process (apparently taking several hours to compile the 
>smallest code tweak; currently it takes me about 1 second to try 
>something new).

We have a way to run the tests without compiling the whole thing every
time.  Not quite the same, but not quite as horrible as you might imagine.
And, happy news, Armin has had a new idea.
https://mail.python.org/pipermail/pypy-dev/2015-February/013085.html
When this goes in, warmup times will get much faster, which will mean
that everyting goes a lot faster.  (We think.  In theory.  We will
see.)

>Also, I'm using Windows where even ordinary Python development doesn't 
>really work; I had to use Ubuntu (a completely alien environment to me) 
>to attempt compiling Cpython).

PyPy development on Windows has gotten a whole lot better since that
keynote -- what it took was getting some PyPy developers who use
windows.  But they are still in short supply.  And yes, it is probably
still easier for linux people.

But if one of your goals is better performance, well, python-list
isn't the place to find the people who spend most of their lives
thinking about such things.  Whereas pypy-dev is one such place.

>-- 
>Bartc
>-- 

At any rate, if there is any interest you can come visit the irc channel
on freenode, or send mail to pypy-dev, which isn't gatewayed to usenet
anywhere.  https://mail.python.org/mailman/listinfo/pypy-dev

You won't get philosophical notes about the meaning of objects
and the like there.  It's a pretty down-to-earth-and-here-is-the-code
and 'you are slow because you do X do Y instead' sort of place.  But
polite and friendly.

Laura


From larry.martell at gmail.com  Thu Jun  4 09:43:59 2015
From: larry.martell at gmail.com (Larry Martell)
Date: Thu, 4 Jun 2015 09:43:59 -0400
Subject: Regular Expression
In-Reply-To: <c85bc324-37fe-469e-b3b1-b1d4e51bf7d8@googlegroups.com>
References: <c85bc324-37fe-469e-b3b1-b1d4e51bf7d8@googlegroups.com>
Message-ID: <CACwCsY5u78W9x=7kDEOPXxsmL5kAuXpq72SOnqdueb6SQ6f4kA@mail.gmail.com>

On Thu, Jun 4, 2015 at 9:36 AM, Palpandi <palpandi111 at gmail.com> wrote:
>
> Hi All,
>
> This is the case. To split "string2" from "string1_string2" I am using
> re.split('_', "string1_string2", 1)[1].
>
> It is working fine for string "string1_string2" and output as "string2". But actually the problem is that if a sting is "__string1_string2" and the output is "_string1_string2". It is wrong.
>
> How to fix this issue?

"__string1_string2".split('_')[-1]


From steve at pearwood.info  Thu Jun  4 09:47:36 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 04 Jun 2015 23:47:36 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
Message-ID: <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>

On Thu, 4 Jun 2015 08:28 am, sohcahtoa82 at gmail.com wrote:

> On Wednesday, June 3, 2015 at 2:57:00 PM UTC-7, Mark Lawrence wrote:
[...]
>> Now does Python pass by value or by reference?  Happily sits back and
>> waits for 10**6 emails to arrive as this is discussed for the 10**6th
>> time.
 
> People actually argue that Python passes by value?  This is easily proven
> wrong by passing a mutable object to a function and changing it within the
> function.


You would be surprised how many people don't allow either facts or common
sense to get in the way of their preconceived notions.

The Python community has more or less reached consensus that Python uses the
calling convention usually called "pass by object" or "call by sharing"
(although it has a few other names), especially since the Python docs uses
the term.

But you still find a few people here and there who have been exposed to Java
foolishness, and will argue that Python is "pass by value, where the value
is an implementation dependent reference to the thing that you thought was
the value".

In other words, according to this Java philosophy, following `x = 23`, the
value of x is not 23 like any sane person would expect, but some invisible
and unknown, and unknowable, reference to 23.

As Fredrik Lundh (the Effbot) wrote:

    well, I guess you can, in theory, value an artificial number 
    assigned to an object as much as the object itself.

        "Joe, I think our son might be lost in the woods"
        "Don't worry, I have his social security number"

http://effbot.org/zone/call-by-object.htm


This may also be helpful:

http://import-that.dreamwidth.org/1130.html



-- 
Steven



From python.list at tim.thechases.com  Thu Jun  4 09:48:55 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Thu, 4 Jun 2015 08:48:55 -0500
Subject: Regular Expression
In-Reply-To: <c85bc324-37fe-469e-b3b1-b1d4e51bf7d8@googlegroups.com>
References: <c85bc324-37fe-469e-b3b1-b1d4e51bf7d8@googlegroups.com>
Message-ID: <20150604084855.27799bee@bigbox.christie.dr>

On 2015-06-04 06:36, Palpandi wrote:
> This is the case. To split "string2" from "string1_string2" I am
> using re.split('_', "string1_string2", 1)[1].
> 
> It is working fine for string "string1_string2" and output as
> "string2". But actually the problem is that if a sting is
> "__string1_string2" and the output is "_string1_string2". It is
> wrong.

Why use regular expressions to split a string on a constant?

Try

  for input in [
      "string1_string2",
      "__string1_string2",
      ]:
    value = input.rsplit('_', 1)[-1]
    assert value == "string2"

-tkc





From steve at pearwood.info  Thu Jun  4 09:54:24 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 04 Jun 2015 23:54:24 +1000
Subject: Regular Expression
References: <c85bc324-37fe-469e-b3b1-b1d4e51bf7d8@googlegroups.com>
Message-ID: <55705890$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Thu, 4 Jun 2015 11:36 pm, Palpandi wrote:

> Hi All,
> 
> This is the case. To split "string2" from "string1_string2" I am using
> re.split('_', "string1_string2", 1)[1].


There is absolutely no need to use the nuclear-powered bulldozer of regular
expressions to crack that tiny peanut. Strings have a perfectly useful
split method:

py> "string1_string2".split("_")
['string1', 'string2']


> It is working fine for string "string1_string2" and output as "string2".
> But actually the problem is that if a sting is "__string1_string2" and the
> output is "_string1_string2". It is wrong.

No, the output is correct. You tell Python to split on the *first*
underscore only, which is exactly what Python does:

py> re.split('_', "__string1_string2", 1)
['', '_string1_string2']


> How to fix this issue?


Again, this is a small problem, and regular expressions are not needed. Just
strip the underscores off the left, then split:


py> s = "__string1_string2"
py> s.lstrip("_").split("_")
['string1', 'string2']





-- 
Steven



From __peter__ at web.de  Thu Jun  4 10:00:30 2015
From: __peter__ at web.de (Peter Otten)
Date: Thu, 04 Jun 2015 16:00:30 +0200
Subject: Regular Expression
References: <c85bc324-37fe-469e-b3b1-b1d4e51bf7d8@googlegroups.com>
Message-ID: <mkpllv$9k1$1@ger.gmane.org>

Palpandi wrote:

> This is the case. To split "string2" from "string1_string2" I am using
> re.split('_', "string1_string2", 1)[1].
> 
> It is working fine for string "string1_string2" and output as "string2".
> But actually the problem is that if a sting is "__string1_string2" and the
> output is "_string1_string2". It is wrong.
> 
> How to fix this issue?

Use str.rpartion():

>>> "one_two__three".rpartition("_")[-1]
'three'




From agn.91.91 at gmail.com  Thu Jun  4 10:02:56 2015
From: agn.91.91 at gmail.com (Amit Goutham)
Date: Thu, 4 Jun 2015 19:32:56 +0530
Subject: Calling Python Script from an SQL Proceudre
In-Reply-To: <201506031613.t53GDJho003039@fido.openend.se>
References: <agn.91.91@gmail.com>
 <CAJ5KB9jsYR0E+chPonkGAFkt1ihW0FSJ_3s0YXF_XRj7wmDMrw@mail.gmail.com>
 <201506031613.t53GDJho003039@fido.openend.se>
Message-ID: <CAJ5KB9hwtKYsxc7+gd59ePj3d4k5jXjSC7CnuB=oPAkmFTev5w@mail.gmail.com>

Hi Laura,
Thanks a Lot for the reply.
I wanted to know if calling a Python script from SAP HANA database is
possible.

Thanks!
On Jun 3, 2015 9:43 PM, "Laura Creighton" <lac at openend.se> wrote:

> In a message of Wed, 03 Jun 2015 15:17:16 +0530, Amit Goutham writes:
> >Hi All,
> >I am trying to search on the Internet if i can call a Python Script from
> an
> >SQL Procedure.
> >All the information found on Internet is about connecting to a database
> >from Python through a Python script.But, i want the other way round.
> >
> >Any Help will be appreciated
> >
> >--
> >Thanks and Regards,
> >Amit Goutham
> >Ph:+91-8867801035
>
> You certainly can with postgreql, see
> http://www.postgresql.org/docs/current/static/plpython.html
>
> I don't know about other SQL servers.
>
> Laura
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150604/a83867dc/attachment.html>

From steve at pearwood.info  Thu Jun  4 10:11:30 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 00:11:30 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>	<48fc36e9-fa67-45d5-9864-0921b7e819ce@googlegroups.com>	<556d931a$0$12991$c3e8da3$5496439d@news.astraweb.com>	<1ad44d0f-8a98-4302-9391-51264aa258e5@googlegroups.com>	<DxAbx.902026$_Q7.705597@fx28.am4>	<mailman.99.1433328294.13271.python-list@python.org>	<NeBbx.946747$wk1.662297@fx19.am4>
 <87fv6956zw.fsf@elektro.pacujo.net> <jwFbx.736675$m15.72884@fx45.am4>
 <mailman.115.1433349421.13271.python-list@python.org>
 <vNGbx.932535$396.701770@fx12.am4>
Message-ID: <55705c93$0$12981$c3e8da3$5496439d@news.astraweb.com>

On Thu, 4 Jun 2015 03:26 am, BartC asked about Value and Object:

> Are they in fact the same, or is there something else that can be done
> more reliably to show the difference?

In *Python*, they are the same. All values are implemented as objects.

But, speaking generically, Value and Object are as different as Value and
String, or Value and Integer, or Value and List. Objects are a specific
kind of value. Values may be:

- simple machine-specific types like byte, int16, single (float);
- simple compound types like array, record or struct, string;
- more complex compound types, such as objects (in the Object Oriented
Programming sense).

An object in this sense is basically a record or struct with added language
semantics. So in a language like Pascal, for example, there are a bunch of
machine-specific simple unstructured types:

    Real, Boolean, Char, Integer, ...

plus machine-specific compound types:

    Array, Record, String, ...

plus Objects (at least for recent enough Pascal compilers). In Java, for
example, you have "unboxed primitives" for various integer types,
and "boxed" objects.

In Python, all values are Objects:

- ints are low-level integers wrapped in an Object;
- floats are low-level C doubles wrapped in an Object;
- strings are arrays of characters wrapped in an Object;

etc.


> (However, I feel the internal representation of values shouldn't matter.
> Each type of value will have its own documented behaviour.)

Well, sometimes it might matter. My own feeling is that languages shouldn't
lie to you. If it offers both low-level primitives for efficiency and
high-level objects for convenience, it should make the difference explicit,
like Java does, or Pascal, or C++. Javascript tries to hide the difference,
which works really well until it doesn't, and then you get weird behaviour:


js> arr = [false, Boolean(false), new Boolean(false)];
false,false,false
js> for (i = 0; i < arr.length; ++i) {
  > print(arr[i]);
  > if (arr[i]) {print("Surprise!")} }
false
false
false
Surprise!


If you can come up with a sensible justification for why Boolean(false) is
falsey, but new Boolean(false) is truthy, you're a better man than me.

 

-- 
Steven



From torriem at gmail.com  Thu Jun  4 10:16:58 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 04 Jun 2015 08:16:58 -0600
Subject: Can Python function return multiple data?
In-Reply-To: <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
Message-ID: <55705DDA.1040104@gmail.com>

On 06/03/2015 04:28 PM, sohcahtoa82 at gmail.com wrote:

> 
> People actually argue that Python passes by value?  This is easily
> proven wrong by passing a mutable object to a function and changing
> it within the function.

Sure but if you reassign the variable that was passed it, it has no
effect whatsoever on the caller's variable, mutable or not.  This is why
people argue for "pass by value."  Because of this confusion, we often
say, "pass by object."

For example

def foo(bar):
    bar.append(5)
    bar = 6 #bar is no longer referring to a list

a=[1,2,3,4]
foo(a)
print (a)
# prints [1,2,3,4,5]








From lac at openend.se  Thu Jun  4 10:21:15 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 04 Jun 2015 16:21:15 +0200
Subject: Regular Expression
In-Reply-To: Message from Palpandi <palpandi111@gmail.com> of "Thu,
 04 Jun 2015 06:36:29 -0700."
 <c85bc324-37fe-469e-b3b1-b1d4e51bf7d8@googlegroups.com>
References: <c85bc324-37fe-469e-b3b1-b1d4e51bf7d8@googlegroups.com>
Message-ID: <201506041421.t54ELFAC001945@fido.openend.se>

In a message of Thu, 04 Jun 2015 06:36:29 -0700, Palpandi writes:
>Hi All,
>
>This is the case. To split "string2" from "string1_string2" I am using 
>re.split('_', "string1_string2", 1)

And you shouldn't be.  The 3rd argument, 1 says stop after one match.

>It is working fine for string "string1_string2" and output as "string2". But actually the problem is that if a sting is "__string1_string2" and the output is "_string1_string2". It is wrong.
>
>How to fix this issue?

Depends on what you want.

Approach #1 - just use the string method, forget re, because you do not
need it.

>>>> "__string1_string2".split("_")
['', '', 'string1', 'string2']
>>>> "_string1_string2__".split("_")
['', 'string1', 'string2', '', '']

Approach #2 -- use re but with a fixed string (probably a bad idea,
you should be using approach 1 instead if you have a fixed string)

>>>> re.split('_', "__string1_string2")
['', '', 'string1', 'string2']
>>>> re.split('_', "__string1_string2__")
['', '', 'string1', 'string2', '', '']

Approach #3 - there is a real pattern here I want to use, the example
I posted to the list is a lot simpler than what I really want to do.
Ok, in this case we will match 'any number of underscores' for an
example.

>>>> p = re.compile('_*')
>>>> p.split("__string1_string2")
['', 'string1', 'string2']
>>>> p.split("__string1__string2__")
['', 'string1', 'string2', '']

Laura


From marko at pacujo.net  Thu Jun  4 10:25:46 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 04 Jun 2015 17:25:46 +0300
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87a8wf5z4l.fsf@elektro.pacujo.net>

Steven D'Aprano <steve at pearwood.info>:

> But you still find a few people here and there who have been exposed
> to Java foolishness, and will argue that Python is "pass by value,
> where the value is an implementation dependent reference to the thing
> that you thought was the value".

Why fight terminology? Definitions can't be proved right or wrong.

Anyway, I would say Python definitely is in the classic pass-by-value
camp. Here's a simple test:

   def f(x):
       x = 3

   y = 1
   f(y)
   print(y)

If it prints 1, it's pass by value. If it prints 3, it's pass by
reference.


Marko


From invalid at invalid.invalid  Thu Jun  4 10:27:59 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 4 Jun 2015 14:27:59 +0000 (UTC)
Subject: Find in ipython3
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.142.1433388453.13271.python-list@python.org>
 <87vbf4giu6.fsf@Equus.decebal.nl>
Message-ID: <mkpn9f$ghc$1@reader1.panix.com>

On 2015-06-04, Cecil Westerhof <Cecil at decebal.nl> wrote:
> Op Thursday 4 Jun 2015 04:54 CEST schreef Cameron Simpson:
>
>> On 02Jun2015 18:13, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>> I am thinking about using ipython3 instead of bash. When I want to
>>> find a file I can do the following:
>>> !find ~ -iname '*python*.pdf'
>>> but is there a python way?
>>
>> That succinct? Not out of the box, but something can easily be built
>> on top of the os.walk() function.
>
> OK, I will write it. And something to implement du.

Can't IPython just call the find and du utilities?

-- 
Grant Edwards               grant.b.edwards        Yow! Are we THERE yet?
                                  at               
                              gmail.com            


From invalid at invalid.invalid  Thu Jun  4 10:37:10 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 4 Jun 2015 14:37:10 +0000 (UTC)
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net>
Message-ID: <mkpnqm$gfn$1@reader1.panix.com>

On 2015-06-04, Marko Rauhamaa <marko at pacujo.net> wrote:
> Steven D'Aprano <steve at pearwood.info>:
>
>> But you still find a few people here and there who have been exposed
>> to Java foolishness, and will argue that Python is "pass by value,
>> where the value is an implementation dependent reference to the thing
>> that you thought was the value".
>
> Why fight terminology? Definitions can't be proved right or wrong.
>
> Anyway, I would say Python definitely is in the classic pass-by-value
> camp. Here's a simple test:
>
>    def f(x):
>        x = 3
>
>    y = 1
>    f(y)
>    print(y)
>
> If it prints 1, it's pass by value. If it prints 3, it's pass by
> reference.

Somebody else might just as honestly say that it's pass by reference:

def f(x):
    x[2] = 2;

x = ['a','b','c']
f(x)
print(x)

If it prints ['a','b','c'], it's pass by value.  If it's pass by
reference, it prints ['a', 'b', 2].

IMO, it's pass by reference.

But, discussing pass-by-this vs. pass-by-that without also discussing
the semantics of the assignment operator is rather pointless.  Not
that the pointlessness of an argument is going to slow down a
thread...

-- 
Grant Edwards               grant.b.edwards        Yow! I request a weekend in
                                  at               Havana with Phil Silvers!
                              gmail.com            


From marko at pacujo.net  Thu Jun  4 11:04:56 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 04 Jun 2015 18:04:56 +0300
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
Message-ID: <874mmn5xbb.fsf@elektro.pacujo.net>

Grant Edwards <invalid at invalid.invalid>:

>> Anyway, I would say Python definitely is in the classic pass-by-value
>> camp. Here's a simple test:
>>
>>    def f(x):
>>        x = 3
>>
>>    y = 1
>>    f(y)
>>    print(y)
>>
>> If it prints 1, it's pass by value. If it prints 3, it's pass by
>> reference.
>
> Somebody else might just as honestly say that it's pass by reference:

Yes, but that would be a shift from the 1970's notion:

   In pass-by-value, the actual parameter is evaluated. The value of the
   actual parameter is then stored in a new location allocated for the
   function parameter.

   [...]

   In pass-by-reference, the actual parameter must have an L-value. The
   L-value of the actual parameter is then bound to the formal
   parameter.

   [...]

    * A parameter in Pascal is normally passed by value. It is passed by
      reference, however, if the keyword var appears before the
      declaration of the formal parameter.

      procedure proc(in: Integer; var out: Real);

    * The only parameter-passing method in C is call-by-value; however,
      the effect of call-by-reference can be achieved using pointers. In
      C++ true call-by-reference is available using reference parameters.

   <URL: http://www.cl.cam.ac.uk/teaching/0910/ConceptsPL/Algol-Pasca
   l.pdf>

Pass by reference could easily be added to Python, too, if that were
deemed useful.


Marko


From Cecil at decebal.nl  Thu Jun  4 11:12:26 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 04 Jun 2015 17:12:26 +0200
Subject: Find in ipython3
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.142.1433388453.13271.python-list@python.org>
 <87vbf4giu6.fsf@Equus.decebal.nl> <mkpn9f$ghc$1@reader1.panix.com>
Message-ID: <87iob3h5id.fsf@Equus.decebal.nl>

Op Thursday 4 Jun 2015 16:27 CEST schreef Grant Edwards:

> On 2015-06-04, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> Op Thursday 4 Jun 2015 04:54 CEST schreef Cameron Simpson:
>>
>>> On 02Jun2015 18:13, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>>> I am thinking about using ipython3 instead of bash. When I want
>>>> to find a file I can do the following: !find ~ -iname
>>>> '*python*.pdf' but is there a python way?
>>>
>>> That succinct? Not out of the box, but something can easily be
>>> built on top of the os.walk() function.
>>
>> OK, I will write it. And something to implement du.
>
> Can't IPython just call the find and du utilities?

That is what
    !find ~ -iname '*python*.pdf'
does. But I do not find that aesthetically.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From elchino at cnn.cn  Thu Jun  4 11:37:20 2015
From: elchino at cnn.cn (ElChino)
Date: Thu, 04 Jun 2015 17:37:20 +0200
Subject: Can Python function return multiple data?
In-Reply-To: <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mkpr8v$1tp$2@dont-email.me>

Steven D'Aprano wrote:

 > But you still find a few people here and there who have been exposed to Java
> foolishness, and will argue that Python is "pass by value, where the value
> is an implementation dependent reference to the thing that you thought was
> the value".

To quote Niklaus Wirth (the father of Pascal, Modula-2 etc.),
"You may call me by name. But you may also call me by reference".




From skybuck2000 at hotmail.com  Thu Jun  4 12:01:44 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 4 Jun 2015 18:01:44 +0200
Subject: Parser needed.
In-Reply-To: <a1ba9$556d34d1$5419aafe$10658@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <9c6eb$556d2fa6$5419aafe$8770@news.ziggo.nl>
 <81e68$556d3120$5419aafe$9437@news.ziggo.nl>
 <b352f$556d338a$5419aafe$10196@news.ziggo.nl>
 <a1ba9$556d34d1$5419aafe$10658@news.ziggo.nl>
Message-ID: <ad067$55707666$5419aafe$54543@news.ziggo.nl>

Yes these string processing techniques will work very nicely and very fast:

cut and pasted an example but should work... now I developed it a bit 
further, bye ,bye.

BotDemoFolder = "C:\\Games\\Startrek Online\\Startrek Online\\Cryptic 
Studios\\Star Trek Online\\Live\\demos"
BotDemoFile = "SpaceFleetAlertEnemyExample.demo"


import time

def ProcessEntityRef():
global DemoChars
global CreatedEntsIndex

StartIndex = DemoChars.find("EntityRef", CreatedEntsIndex)
StopIndex = DemoChars.find("\n",StartIndex)

if StartIndex <> -1:
  Ref = DemoChars[StartIndex:StopIndex]
else:
  Ref = 0

return Ref

def ProcessCreatedEntities():
global DemoChars
global CreatedEntsIndex
CreatedEntsIndex = 0
while (CreatedEntsIndex <> -1): # and (CreatedEntsIndex < 100000):
  CreatedEntsIndex = DemoChars.find("createdEnts", CreatedEntsIndex)
  RefNumber = ProcessEntityRef()
  if CreatedEntsIndex <> -1:
   CreatedEntsIndex = CreatedEntsIndex + 1
  print "RefNumber: " + str(RefNumber)
return


def Main():
global DemoChars

DemoFilePath = BotDemoFolder + "\\" + BotDemoFile

FileObject = open( DemoFilePath, "r")

# DemoLines = FileObject.readlines()
# ParseDemoLines( DemoLines )

DemoChars = FileObject.read()

ProcessCreatedEntities()

# Index = DemoChars.find("createdEnts")

# print DemoChars[Index:Index+5]


# rfind

# ParseDemoChars( DemoChars )

FileObject.close()
return

print "program started"

Tick1 = time.time()
Main()
Tick2 = time.time()

Seconds = Tick2 - Tick1


print "program finished"


# Skybuck 



From skybuck2000 at hotmail.com  Thu Jun  4 13:02:51 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 4 Jun 2015 19:02:51 +0200
Subject: Parser needed.
In-Reply-To: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
Message-ID: <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>

Yeah... my first nice parser for this kind of stuff...

Python is really nice for this stuff...

Piece a cake.. now I just need to stuff it in some dictionary and I am done 
or so ;)

Though a dictionary might be hard to traverse in sequence...

A list is probably enough... assuming no duplicate entries for now...

And done ;) =D

Bye,
  Skybuck. 



From steve at pearwood.info  Thu Jun  4 13:11:48 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 03:11:48 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
Message-ID: <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 12:37 am, Grant Edwards wrote:

> On 2015-06-04, Marko Rauhamaa <marko at pacujo.net> wrote:

>> Anyway, I would say Python definitely is in the classic pass-by-value
>> camp. Here's a simple test:
>>
>>    def f(x):
>>        x = 3
>>
>>    y = 1
>>    f(y)
>>    print(y)
>>
>> If it prints 1, it's pass by value. If it prints 3, it's pass by
>> reference.

Wrong. Why do you [Marko] imagine that pass-by-value and pass-by-reference
are the only two options? Your test is insufficient to distinguish pass by
value from pass by sharing, or pass by reference from pass by name.

You need at least one more test to prove pass by value: you need to
demonstrate that the value bound to y is copied when passed to the
function. E.g. pass a mutable value (say, a list) and mutate it inside the
function. If the list in the outer scope is *not* mutated, then and only
then can you say it is pass by value.


> Somebody else might just as honestly say that it's pass by reference:

And they would be just as wrong.


> def f(x):
>     x[2] = 2;
> 
> x = ['a','b','c']
> f(x)
> print(x)
> 
> If it prints ['a','b','c'], it's pass by value.  If it's pass by
> reference, it prints ['a', 'b', 2].

Wrong. Why do you [Grant] imagine that pass by value and pass by reference
are the only two options? Your test is insufficient to distinguish pass by
reference from pass by name or pass by sharing, all three of which will
give the same behaviour for this specific test.

You need at least two more tests to demonstrate that this is pass by
reference. First you need to demonstrate that function f can rebind the
variable in the outer scope. E.g. take Marko's test above, if it is pass by
reference or pass by name it will print 3. (Hint: it doesn't.) Then you
need to demonstrate that it isn't pass by name. Despite the name, "pass by
name" doesn't actually require a variable name, it can also accept an
expression or literal. Pass by reference cannot.

def f(x):
    pass

f(23)

If that raises an exception, then it is pass by reference. (Hint: it
doesn't.)


> IMO, it's pass by reference.

It really isn't.


> But, discussing pass-by-this vs. pass-by-that without also discussing
> the semantics of the assignment operator is rather pointless.

No, that's a red-herring.

I don't know of any programming language where passing a value to a
function, and binding it to a name (or assigning it to a variable, if you
prefer) uses different mechanisms. If the language offers two or more
argument passing mechanisms, assignment uses one of them (e.g. Algol offers
pass by name and pass by value, and assignment is by value). If there is
any language where assignment uses one style and argument passing always
uses another, I've never come across it.


> Not 
> that the pointlessness of an argument is going to slow down a
> thread...

Clear thinking is pointless?

Well, that explains how you can claim that Python is pass by reference
immediately after Marko demonstrates that it cannot possibly be pass by
reference *wink*

 

-- 
Steven



From steve at pearwood.info  Thu Jun  4 13:14:12 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 03:14:12 +1000
Subject: Parser needed.
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
Message-ID: <55708764$0$13011$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 03:02 am, Skybuck Flying wrote:

> Yeah... my first nice parser for this kind of stuff...
> 
> Python is really nice for this stuff...
> 
> Piece a cake.. now I just need to stuff it in some dictionary and I am
> done or so ;)
> 
> Though a dictionary might be hard to traverse in sequence...

Use an ordered dict:

from collections import OrderedDict


-- 
Steven



From mohan.mohta at gmail.com  Thu Jun  4 13:20:47 2015
From: mohan.mohta at gmail.com (M2)
Date: Thu, 4 Jun 2015 10:20:47 -0700 (PDT)
Subject: Multiple thread program problem
In-Reply-To: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
References: <5eba09c9-aadb-46d9-b644-8ba775d8b97a@googlegroups.com>
Message-ID: <ff2d1eb5-95b7-4cfb-b259-c93ea244abc4@googlegroups.com>

On Wednesday, June 3, 2015 at 3:41:15 PM UTC-5, M2 wrote:
> Hello
> I am trying to create multiple thread through the below program but I am getting an error 
> 
> #! /usr/bin/python
> import os
> import subprocess
> import thread
> import threading
> from thread import start_new_thread
> 
> def proc(f) :
>         com1="ssh -B "
>         com2=line.strip('\n')
>         com3= " uname -a"
>         co=str("ssh -B ")+ str(com2) + str(" uname -a")
>         subprocess.call(co,shell=True)
>         print "----------------------------"
>         return
> 
> f = open('/tmp/python/1')
> for line in f:
>         t=thread.start_new_thread(proc(f),())
>         t.start()
> f.close()
> c = raw_input(" Type anything to quit")
> 
> 
> Execution output:
> Linux abc.myhomenetwork.com 2.6.18-348.25.1.el5 #1 SMP Thu Apr 10 06:32:45 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux
> ----------------------------
> Traceback (most recent call last):
>   File "./readfile1.py", line 19, in <module>
>     t=thread.start_new_thread(proc(f),())
> TypeError: first arg must be callable

Awesome Cameron.
It works the way I want it to work.

Thanks a lot guys.
Here is the new code:
#! /usr/bin/python
import os
import subprocess
import thread
import threading
from thread import start_new_thread

def proc(col) :
        P=subprocess.Popen(col, shell=True)
        return

f = open('/tmp/python/1')
for line in f:
        com1="ssh -B "
        com2=line.strip('\n')
        #com3= " /var/scripts/health/health.sh"
        com3= " uname -a "
        co=str("ssh -B ")+ str(com2) + str(com3)
        P=proc(co)
f.close()


Thanks a ton guys this should get me started on the greater program I am trying to write.


From skybuck2000 at hotmail.com  Thu Jun  4 13:23:17 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 4 Jun 2015 19:23:17 +0200
Subject: Parser needed.
In-Reply-To: <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
Message-ID: <74dd4$55708983$5419aafe$17465@news.ziggo.nl>

Very nice code almost done.

Now I am trying to do the code correctly and fast, thus using a dictionary, 
but I run into a little problem:

The dictionary is declared as:

DemoEntityRefIndex = {}

Pairs are added as:

DemoEntityRefIndex[Ref] = DemoEntityIndex

And now I try to retrieve the demo entit index as follows:

DemoEntityIndex = DemoEntityRefIndex[Ref]

But somehow it fails with error:

[error] script [ ParseDemoFile ] stopped with error in line 211
[error] KeyError ( EntityRef )
[error] --- Traceback --- error source first line: module ( function ) 
statement 146: main ( ProcessUpdateEntityRef ) DemoEntityIndex = 
DemoEntityRefIndex[Ref]

Hmm..

I may try dict.get maybe that ll help.

Bye,
  Skybuck.












From torriem at gmail.com  Thu Jun  4 13:25:55 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 04 Jun 2015 11:25:55 -0600
Subject: Parser needed.
In-Reply-To: <55708764$0$13011$c3e8da3$5496439d@news.astraweb.com>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <55708764$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <55708A23.9020704@gmail.com>

On 06/04/2015 11:14 AM, Steven D'Aprano wrote:
> On Fri, 5 Jun 2015 03:02 am, Skybuck Flying wrote:
> 
>> Yeah... my first nice parser for this kind of stuff...
>>
>> Python is really nice for this stuff...
>>
>> Piece a cake.. now I just need to stuff it in some dictionary and I am
>> done or so ;)
>>
>> Though a dictionary might be hard to traverse in sequence...
> 
> Use an ordered dict:
> 
> from collections import OrderedDict

I prefer a list of Dicts, or a list of objects you define.  That way it
keeps order, but you can have an arbitrary structure on each item



From random832 at fastmail.us  Thu Jun  4 13:26:15 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 04 Jun 2015 13:26:15 -0400
Subject: Can Python function return multiple data?
In-Reply-To: <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <1433438775.1905830.286978449.6430E548@webmail.messagingengine.com>

On Thu, Jun 4, 2015, at 09:47, Steven D'Aprano wrote:
> In other words, according to this Java philosophy, following `x = 23`,
> the
> value of x is not 23 like any sane person would expect, but some
> invisible
> and unknown, and unknowable, reference to 23.

Well, no, because, in Java, if the type of x is int, then the value
really is 23. If it's Integer, then it's a reference to a boxed Integer
object. Which is (thankfully) immutable. But isn't invisible, unknown,
or unknowable at all.

Of course, in CPython, the type of an object reference is PyObject *.
Which isn't invisible, unknown, or unknowable, either.

If the value really were 23, the "is vs ==" problem wouldn't exist.

x = 500
y = x+1
z = y-1

x and z (probably) point to two different objects.


From marko at pacujo.net  Thu Jun  4 13:30:02 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 04 Jun 2015 20:30:02 +0300
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <874mmn2xgl.fsf@elektro.pacujo.net>

Steven D'Aprano <steve at pearwood.info>:

>> On 2015-06-04, Marko Rauhamaa <marko at pacujo.net> wrote:
>>> If it prints 1, it's pass by value. If it prints 3, it's pass by
>>> reference.
>
> Wrong. Why do you [Marko] imagine that pass-by-value and
> pass-by-reference are the only two options?

It's a classic dichotomy. The example I provided illustrates the
distinction between the two.


Marko


From random832 at fastmail.us  Thu Jun  4 13:30:52 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 04 Jun 2015 13:30:52 -0400
Subject: Can Python function return multiple data?
In-Reply-To: <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <1433439052.1906911.286984889.678B8840@webmail.messagingengine.com>

On Thu, Jun 4, 2015, at 13:11, Steven D'Aprano wrote:
> You need at least one more test to prove pass by value: you need to
> demonstrate that the value bound to y is copied when passed to the
> function. E.g. pass a mutable value (say, a list) and mutate it inside
> the
> function. If the list in the outer scope is *not* mutated, then and only
> then can you say it is pass by value.

That's not true at all. Being able to mutate the object doesn't make it
not-pass-by-value any more than being able to use an int argument as an
index to mutate a global array makes the int not-pass-by-value.

a = [0]*16

def f(x, i):
    x[i] = 3

f(a, 0)

If x is not pass-by-value, then clearly i is also not pass-by-value,
despite integers being immutable. Being able to use an argument to cause
a side effect does not mean the argument has not been passed by value.

a hasn't changed, 0 hasn't changed, the only thing that has changed is
the list object, which is *not the same thing as a*.


From skybuck2000 at hotmail.com  Thu Jun  4 13:34:57 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 4 Jun 2015 19:34:57 +0200
Subject: Parser needed.
In-Reply-To: <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
Message-ID: <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>

get was tried but now new error somewhere else:

[error] TypeError ( list indices must be integers )
[error] --- Traceback --- error source first line: module ( function ) 
statement 133: main ( ProcessUpdateEntityDead ) 
DemoEntityDead[DemoEntityIndex] = Dead

Apperently the returned index from get is not an integer ?

Weird... so far dictionaries offer misery.

Bye,
  Skybuck. 



From torriem at gmail.com  Thu Jun  4 13:36:52 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 04 Jun 2015 11:36:52 -0600
Subject: Can Python function return multiple data?
In-Reply-To: <1433438775.1905830.286978449.6430E548@webmail.messagingengine.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <1433438775.1905830.286978449.6430E548@webmail.messagingengine.com>
Message-ID: <55708CB4.3030305@gmail.com>

On 06/04/2015 11:26 AM, random832 at fastmail.us wrote:
> Of course, in CPython, the type of an object reference is PyObject *.
> Which isn't invisible, unknown, or unknowable, either.
> 
> If the value really were 23, the "is vs ==" problem wouldn't exist.

Surely two objects can hold the same value, or represent the same value
(number), without *having* to be the same object.  I don't see why
people would assume, let alone demand that different objects
representing the same value be the same actual object.  Why should it
even matter? Just relax and write code.  I enjoy the forays into the the
technical, and even the pedantic, but this is getting a bit silly.


> x = 500
> y = x+1
> z = y-1
> 
> x and z (probably) point to two different objects.

Sure.  Python does not require objects representing numeric values to be
singletons.  And why worry anyway?  The "is" operator is to be used in
specific cases where identity is important (it's not important in the
case of numbers).

I'm sure an interpreter implementation could always guarantee that any
object representing an integer value is always the same one, if a person
thought it would make things faster.  And in fact cpython does inter
small values.  But given that the interpreter is computing an actual,
binary value under the hood during calculations, I can understand why
it's just as easy to wrap up the answer in an object and hand it back to
you, rather than try to guarantee a reference to an existing object of
the same value.


From skybuck2000 at hotmail.com  Thu Jun  4 13:38:44 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 4 Jun 2015 19:38:44 +0200
Subject: Parser needed.
In-Reply-To: <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
Message-ID: <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>

I feel my conclusion is a bit hasty... but using dictionaries is not easy 
that for sure.

Apperently the problem is

DemoEntityInde is none ?

But why would it be none ?

Hmmm strange... maybe some refs are not in there... hmmm...

Yeah could be... I cutted some stuff out... so I better check for none then 
for now ;)

So for now consider this solved, unless I return ! ;) :)

Bye,
  Skybuck.



From skybuck2000 at hotmail.com  Thu Jun  4 13:44:23 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 4 Jun 2015 19:44:23 +0200
Subject: Parser needed.
In-Reply-To: <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
Message-ID: <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>

Something strange happens with: 36044817

near the update section... for some reason it doesn't copy it properly...

Hmm...

Maybe a bug in output or an additional new line or maybe something wrong...

Hmm..

Bye,
  Skybuck.


From skybuck2000 at hotmail.com  Thu Jun  4 13:50:17 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 4 Jun 2015 19:50:17 +0200
Subject: Parser needed.
In-Reply-To: <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
Message-ID: <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>

Ok problem found.

The data contains:

  EntityRef EntityRef

So perhaps I screwed it up or perhaps the data is a bit bad.

I ll check on my web drive:

http://www.skybuck.org/Games/StartrekOnline/Parser/SpaceFleetAlertEnemyExample.demo

Firefox doesn't find it... so apperently I fucked up data a bit...

Ok so it's not a bug... thank god for that.

So this is not really a problem.

Now to redownload this file for now or so ;)

Bye,
  Skybuck. 



From random832 at fastmail.us  Thu Jun  4 13:51:49 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 04 Jun 2015 13:51:49 -0400
Subject: Can Python function return multiple data?
In-Reply-To: <55708CB4.3030305@gmail.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <1433438775.1905830.286978449.6430E548@webmail.messagingengine.com>
 <55708CB4.3030305@gmail.com>
Message-ID: <1433440309.1912699.286998297.7AA72484@webmail.messagingengine.com>

On Thu, Jun 4, 2015, at 13:36, Michael Torrie wrote:
> Surely two objects can hold the same value, or represent the same value
> (number), without *having* to be the same object.  I don't see why
> people would assume, let alone demand that different objects
> representing the same value be the same actual object.  Why should it
> even matter? Just relax and write code.  I enjoy the forays into the the
> technical, and even the pedantic, but this is getting a bit silly.

I think I've solved the terminology problem...

it's call-by-value, but the value of a variable, or an argument, or an
expression, is not the value of the object, but rather it is the object
(or, if you like, the id of the object.)

So it's call-by-value where the object is the value that is _really_
passed in. "The value of an object" is simply an equivalence
relationship (so, the value of [1, 2] is the same as the value of [1, 2]
until one of them changes) defined by the hash and == operators of the
object's class rather than having any meaning to the language.

Imagine you have a, b = [1, 2], [1, 2]. If a and b had the same value,
then you could substitute them - a[0] = 3 would have the same effect as
b[0] = 3.

Immutable objects confuse this, because there aren't any operations you
can do to one that would cause its value to be different from another
one that once had the same value, so people think the value of the
variable is the value of the object rather than the value of the
variable being *the object*.


From skybuck2000 at hotmail.com  Thu Jun  4 14:00:43 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 4 Jun 2015 20:00:43 +0200
Subject: Parser needed.
In-Reply-To: <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
 <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
Message-ID: <4252d$55709249$5419aafe$25285@news.ziggo.nl>

Well... I must say I am impressed:

Python parsers the file/info I want in just:

Seconds: 0.0139999389648

For +/- 20.000 lines of input data/text.

This makes it very usuable cool !

Now I try the bigger file:

+/- 285.000 lines of input data/text:

Seconds: 0.0929999351501

Very impressive !

I guess I was wrong about python...

Python pretty good at text processing wieee =D

Bye,
  Skybuck =D


From alain at universite-de-strasbourg.fr.invalid  Thu Jun  4 14:17:17 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Thu, 04 Jun 2015 20:17:17 +0200
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87r3prjq36.fsf@universite-de-strasbourg.fr.invalid>

Steven D'Aprano <steve at pearwood.info> writes:

[...]
> But you still find a few people here and there who have been exposed to Java
> foolishness, and will argue that Python is "pass by value, where the value
> is an implementation dependent reference to the thing that you thought was
> the value".

I find this clear and concise. Can you exhibit an example that would not
match this description?

> In other words, according to this Java philosophy, following `x = 23`, the
> value of x is not 23 like any sane person would expect, but some invisible
> and unknown, and unknowable, reference to 23.

No, Java doesn't work like that for primitive types (assuming that by
"Java" you mean the language and execution environment defined in
reference documents).

-- Alain.


From ian.g.kelly at gmail.com  Thu Jun  4 14:34:15 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 4 Jun 2015 12:34:15 -0600
Subject: Can Python function return multiple data?
In-Reply-To: <mknt6h$epv$1@ger.gmane.org>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mknt6h$epv$1@ger.gmane.org>
Message-ID: <CALwzidm4oA1b+3qdnm+ycJst6ag8P779mMjgSZBQ6Etgqke2kQ@mail.gmail.com>

On Wed, Jun 3, 2015 at 3:56 PM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> Now does Python pass by value or by reference?  Happily sits back and waits
> for 10**6 emails to arrive as this is discussed for the 10**6th time.

Troll.


From invalid at invalid.invalid  Thu Jun  4 14:38:19 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 4 Jun 2015 18:38:19 +0000 (UTC)
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mkq5ur$cj6$1@reader1.panix.com>

On 2015-06-04, Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, 5 Jun 2015 12:37 am, Grant Edwards wrote:
>> On 2015-06-04, Marko Rauhamaa <marko at pacujo.net> wrote:
>
>>> Anyway, I would say Python definitely is in the classic pass-by-value
>>> camp. Here's a simple test:
>>>
>>>    def f(x):
>>>        x = 3
>>>
>>>    y = 1
>>>    f(y)
>>>    print(y)
>>>
>>> If it prints 1, it's pass by value. If it prints 3, it's pass by
>>> reference.
>
> Wrong.

>> Somebody else might just as honestly say that it's pass by reference:
>
> And they would be just as wrong.

Yep, that was my point.  They're both wrong and yet people can be just
as honest when they argue one or the other.

>> def f(x):
>>     x[2] = 2;
>> 
>> x = ['a','b','c']
>> f(x)
>> print(x)
>> 
>> If it prints ['a','b','c'], it's pass by value.  If it's pass by
>> reference, it prints ['a', 'b', 2].
>
> Wrong.

I know.

>> But, discussing pass-by-this vs. pass-by-that without also discussing
>> the semantics of the assignment operator is rather pointless.
>
> No, that's a red-herring.

I don't think so.  The reason that many people seem to confused about
Python's argument passing is that they don't understand what
assignment does.

-- 
Grant Edwards               grant.b.edwards        Yow! Are we live or on
                                  at               tape?
                              gmail.com            


From torriem at gmail.com  Thu Jun  4 15:09:06 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 04 Jun 2015 13:09:06 -0600
Subject: Find in ipython3
In-Reply-To: <87y4k2hyvf.fsf@Equus.decebal.nl>
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
Message-ID: <5570A252.9040908@gmail.com>

On 06/02/2015 10:13 AM, Cecil Westerhof wrote:
> I am thinking about using ipython3 instead of bash. When I want to
> find a file I can do the following:
>     !find ~ -iname '*python*.pdf'
> but is there a python way?

No more than there is a bash-native way of doing find.  Bash scripts use
a myriad of external, utility programs like find, cut, awk, sed, etc to
do heavy lifting.  The same things work just fine in iPython.  Why do
you need a "python way?"  Besides, Python isn't really a shell scripting
language, and ipython seems more like a hack to me.  Bash, Zsh, etc are
all better suited to actual shell scripting because their syntaxs
integrate the spawning external processes and building pipes in a nice
way (if you can call bash syntax nice).  That's not to say Python can't
be used for system programming. It certainly can.  And it can replace
perl easily at fast text processing.  Generator expressions are the
single most powerful tool in this.

Why not use Python for what it's good for and say pipe the results of
find into your python script?  Reinventing find poorly isn't going to
buy you anything.



From torriem at gmail.com  Thu Jun  4 15:11:00 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 04 Jun 2015 13:11:00 -0600
Subject: Find in ipython3
In-Reply-To: <87iob3h5id.fsf@Equus.decebal.nl>
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.142.1433388453.13271.python-list@python.org>
 <87vbf4giu6.fsf@Equus.decebal.nl> <mkpn9f$ghc$1@reader1.panix.com>
 <87iob3h5id.fsf@Equus.decebal.nl>
Message-ID: <5570A2C4.4060103@gmail.com>

On 06/04/2015 09:12 AM, Cecil Westerhof wrote:
>> Can't IPython just call the find and du utilities?
> 
> That is what
>     !find ~ -iname '*python*.pdf'
> does. But I do not find that aesthetically.

Like I said, I find ipython to be hackish, but invoking find this way is
no more hackish than writing a 100 lines of python just to emulate it.
Using these existing external commands is more portable and more robust.

You could always write an ipython function that wraps the call to the
find command so you don't always have to see it.




From breamoreboy at yahoo.co.uk  Thu Jun  4 15:16:34 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 04 Jun 2015 20:16:34 +0100
Subject: Can Python function return multiple data?
In-Reply-To: <CALwzidm4oA1b+3qdnm+ycJst6ag8P779mMjgSZBQ6Etgqke2kQ@mail.gmail.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mknt6h$epv$1@ger.gmane.org>
 <CALwzidm4oA1b+3qdnm+ycJst6ag8P779mMjgSZBQ6Etgqke2kQ@mail.gmail.com>
Message-ID: <mkq86m$ht1$1@ger.gmane.org>

On 04/06/2015 19:34, Ian Kelly wrote:
> On Wed, Jun 3, 2015 at 3:56 PM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>> Now does Python pass by value or by reference?  Happily sits back and waits
>> for 10**6 emails to arrive as this is discussed for the 10**6th time.
>
> Troll.
>

True indeed.

As expected the "pass by value" versus the "pass by reference" brigade 
have come out of the stalls running very strongly, but as always the 
"pass by object" crew have arrived very late on the stand side and look 
as if they will win by a nose, unless a photo finish is called in which 
case there is a stewards' enquiry.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From python.list at tim.thechases.com  Thu Jun  4 15:17:23 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Thu, 4 Jun 2015 14:17:23 -0500
Subject: Find in ipython3
In-Reply-To: <5570A252.9040908@gmail.com>
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
	<5570A252.9040908@gmail.com>
Message-ID: <20150604141723.1840d1be@bigbox.christie.dr>

On 2015-06-04 13:09, Michael Torrie wrote:
> Why not use Python for what it's good for and say pipe the results
> of find into your python script?  Reinventing find poorly isn't
> going to buy you anything.

Until you port your app to Windows where find(1) is unavailable
natively ;-)

-tkc





From random832 at fastmail.us  Thu Jun  4 16:13:52 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 04 Jun 2015 16:13:52 -0400
Subject: Find in ipython3
In-Reply-To: <87y4k2hyvf.fsf@Equus.decebal.nl>
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
Message-ID: <1433448832.1944350.287109969.743267AE@webmail.messagingengine.com>

On Tue, Jun 2, 2015, at 12:13, Cecil Westerhof wrote:
> I am thinking about using ipython3 instead of bash. When I want to
> find a file I can do the following:
>     !find ~ -iname '*python*.pdf'
> but is there a python way?

Python really isn't a good substitute for a shell, but the normal python
way to do this task is:

import os, os.path, fnmatch

home = os.path.expanduser('~')  # only needed since you used ~
for dirpath, dirnames, filenames in os.walk(home):
    print(dirpath)
    for filename in filenames:
        if(fnmatch.fnmatch(filename.lower(), '*python*.pdf')):
            print(os.path.join(dirpath, filename))

Note that if you have filenames with invalid unicode characters (or any
non-ASCII characters at all on Windows) you may have to do additional
processing to the filename before printing it. And of course instead of
printing it you may want to store the filenames in a list for further
processing. But these are the basic building blocks.

I don't use ipython, so I don't know what it provides if anything to
make any of this easier.


From bc at freeuk.com  Thu Jun  4 16:52:42 2015
From: bc at freeuk.com (BartC)
Date: Thu, 04 Jun 2015 21:52:42 +0100
Subject: Can Python function return multiple data?
In-Reply-To: <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <wU2cx.757611$I81.286196@fx33.am4>

On 04/06/2015 18:11, Steven D'Aprano wrote:

>If there is
> any language where assignment uses one style and argument passing always
> uses another, I've never come across it.

My language does that. I'd be very surprised if it was the only one in 
existence that does so.

Assignments involve a deep copy. Argument passing is something 
in-between by-value and by-reference (depending also on the type of data 
involved). There is a also an actual by-reference mode (where a pointer 
is passed).


-- 
Bartc


From rosuav at gmail.com  Thu Jun  4 17:44:25 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 5 Jun 2015 07:44:25 +1000
Subject: Can Python function return multiple data?
In-Reply-To: <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net>
 <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmqnyPeLRcTOGPKNncbGutnPskpPNhAiuDUC8=G5rMu=ng@mail.gmail.com>

On Fri, Jun 5, 2015 at 3:11 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> You need at least one more test to prove pass by value: you need to
> demonstrate that the value bound to y is copied when passed to the
> function. E.g. pass a mutable value (say, a list) and mutate it inside the
> function. If the list in the outer scope is *not* mutated, then and only
> then can you say it is pass by value.

Minor quibble: This discussion is about semantics, not timing. A
language can use pass-by-value but with optimizations for the
read-only case.

$ cat array_argument.php
<?php
function readonly($arr) {if ($arr[1] != 1) echo "WRONG!\n";}
function readwrite($arr) {$arr[1] = 1;}

$array = array();
//Populate a big array
for ($i=0; $i<1000; ++$i) $array[$i] = $i;

//Now pass it repeatedly as an argument
$start = microtime(1);
for ($i=0; $i<100000000; ++$i) readonly($array);
$i/=1000000;
$t = (microtime(1)-$start);
echo "Read-only: $i million iterations in $t seconds\n";

$start = microtime(1);
for ($i=0; $i<100000; ++$i) readwrite($array);
$i/=1000;
$t = (microtime(1)-$start);
echo "Read-write: $i thousand iterations in $t seconds\n";

$ time php array_argument.php
Read-only: 100 million iterations in 11.251608133316 seconds
Read-write: 100 thousand iterations in 2.6384630203247 seconds

real 0m13.914s
user 0m13.780s
sys 0m0.000s


PHP passes arrays by value, but optimizes away the copy in the
read-only case - which means that array mutations can trigger
unexpected copying. But it's still pass-by-value, even though you can
appear to have the performance of pass-by-object/pass-by-reference.
Conceivably, a language might implement pass-by-value by having a
"shadow" that knows that it's "the same as that array only this one
got changed", although I can't imagine it'd be efficient for
real-world work. It'd still be legitimate pass-by-value though.

ChrisA


From rosuav at gmail.com  Thu Jun  4 18:00:54 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 5 Jun 2015 08:00:54 +1000
Subject: Calling Python Script from an SQL Proceudre
In-Reply-To: <CAJ5KB9hwtKYsxc7+gd59ePj3d4k5jXjSC7CnuB=oPAkmFTev5w@mail.gmail.com>
References: <agn.91.91@gmail.com>
 <CAJ5KB9jsYR0E+chPonkGAFkt1ihW0FSJ_3s0YXF_XRj7wmDMrw@mail.gmail.com>
 <201506031613.t53GDJho003039@fido.openend.se>
 <CAJ5KB9hwtKYsxc7+gd59ePj3d4k5jXjSC7CnuB=oPAkmFTev5w@mail.gmail.com>
Message-ID: <CAPTjJmqAYUmCwErhodKQru0OnDU2ZDc9PEz_DZ-tfS3O1Rzdpw@mail.gmail.com>

On Fri, Jun 5, 2015 at 12:02 AM, Amit Goutham <agn.91.91 at gmail.com> wrote:
> Hi Laura,
> Thanks a Lot for the reply.
> I wanted to know if calling a Python script from SAP HANA database is
> possible.

What you're asking for there is a SAP HANA feature, not a Python one.
You could possibly ask SAP about it, but based on my brother's
experiences with SAP, I suspect it's going be either very obvious or
not possible.

ChrisA


From steve at pearwood.info  Thu Jun  4 18:16:33 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 08:16:33 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
Message-ID: <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 03:26 am, random832 at fastmail.us wrote:

> On Thu, Jun 4, 2015, at 09:47, Steven D'Aprano wrote:
>> In other words, according to this Java philosophy, following `x = 23`,
>> the
>> value of x is not 23 like any sane person would expect, but some
>> invisible
>> and unknown, and unknowable, reference to 23.
> 
> Well, no, because, in Java, if the type of x is int, then the value
> really is 23. If it's Integer, then it's a reference to a boxed Integer
> object. Which is (thankfully) immutable. But isn't invisible, unknown,
> or unknowable at all.

I was talking about Python, hence the use of Python syntax <wink>

In Java, primitive types are passed by value, but objects are not, and to
the best of my knowledge, what I say below about Python also applies to
Java objects.


> Of course, in CPython, the type of an object reference is PyObject *.
> Which isn't invisible, unknown, or unknowable, either.

Well, I just tried to see that reference, so I would know what it is:

py> PyObject
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'PyObject' is not defined

Well, let me try another way:

py> x = 23
py> reference(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'reference' is not defined

Maybe it's a method?

py> x.getreference()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute 'getreference'

I tried another ~hundred~ ~dozen~ few possibilities, and they all failed
just as above. It seems that there is no way to see that object reference
from Python code, or know what it is. Hence, it is invisible, unknown and
unknowable.

What is visible and known is the *object itself*.

You have made the same mistake as the Java people: caring more about the
underlying implementation details rather than what's going on at the level
of Python code.


> If the value really were 23, the "is vs ==" problem wouldn't exist.

What "problem"? "is versus ==" is not a problem, it is a feature. The two
operators do two different things.

But I am *astonished* that you think that the value of x after x = 23 is not
23. If it's not 23, then what is it?

(Actually, I'm not that astonished. There are some kinds of foolishness that
only really smart people can fall for.)

Remember, you've tried to claim that it is not invisible or unknown, so you
must be able to see and know that value. So what is the value?


> x = 500
> y = x+1
> z = y-1
> 
> x and z (probably) point to two different objects.

Maybe they do, maybe they don't. That's not specified by the Python
language, and is an implementation detail. Other languages may make other
promises. Either way, that's irrelevant to the question at hand.




-- 
Steven



From steve at pearwood.info  Thu Jun  4 18:31:41 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 08:31:41 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <mailman.169.1433439055.13271.python-list@python.org>
Message-ID: <5570d1ce$0$12999$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 03:30 am, random832 at fastmail.us wrote:

> On Thu, Jun 4, 2015, at 13:11, Steven D'Aprano wrote:
>> You need at least one more test to prove pass by value: you need to
>> demonstrate that the value bound to y is copied when passed to the
>> function. E.g. pass a mutable value (say, a list) and mutate it inside
>> the
>> function. If the list in the outer scope is *not* mutated, then and only
>> then can you say it is pass by value.
> 
> That's not true at all. Being able to mutate the object doesn't make it
> not-pass-by-value any more than being able to use an int argument as an 
> index to mutate a global array makes the int not-pass-by-value.
> 
> a = [0]*16
> 
> def f(x, i):
>     x[i] = 3
> 
> f(a, 0)
> 
> If x is not pass-by-value, then clearly i is also not pass-by-value,
> despite integers being immutable. 

Neither are pass by value, and the results of running that code are
consistent with that fact. Nothing in the above code snippet shows pass by
value behaviour: neither the list a nor the literal 0 is copied. We know
that the list is not copied because the mutation operation inside the
function shows up on the outside (it affects the list bound to a). Nothing
in your code demonstrates that 0 is not copied, but this code snippet does:

x = 0  # or use any value you like
print(id(x))

def test(arg):
    print(id(arg))


test(x)



> Being able to use an argument to cause 
> a side effect does not mean the argument has not been passed by value.

Of course it does. The lack of side-effects is the defining characteristic
of pass by value: since the argument is copied, any direct mutations you
apply to it cannot affect the original.


> a hasn't changed,

Yes it has. It was [0,0,0,...0] and now it is [3,0,0,...0]. That's a
mutation to the value, and it is visible in the outer scope. Hence, a has
not been copied, which means it wasn't passed by value.


> 0 hasn't changed, the only thing that has changed is 
> the list object, which is *not the same thing as a*.

This is an interesting (and sometimes useful) distinction you are making,
between the value of a variable and the variable itself, but it is not
relevant to the discussion of pass by value.



-- 
Steven



From steve at pearwood.info  Thu Jun  4 18:37:45 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 08:37:45 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <874mmn2xgl.fsf@elektro.pacujo.net>
Message-ID: <5570d33a$0$12985$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 03:30 am, Marko Rauhamaa wrote:

> Steven D'Aprano <steve at pearwood.info>:
> 
>>> On 2015-06-04, Marko Rauhamaa <marko at pacujo.net> wrote:
>>>> If it prints 1, it's pass by value. If it prints 3, it's pass by
>>>> reference.
>>
>> Wrong. Why do you [Marko] imagine that pass-by-value and
>> pass-by-reference are the only two options?
> 
> It's a classic dichotomy. The example I provided illustrates the 
> distinction between the two.

No it isn't, and no it doesn't.

Call by name, call by [object] sharing, call by copy-restore, call by need,
call by macro expansion... where do these appear in your false dichotomy?


-- 
Steven



From steve at pearwood.info  Thu Jun  4 18:45:44 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 08:45:44 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87r3prjq36.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <5570d51a$0$13002$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 04:17 am, Alain Ketterlin wrote:

> Steven D'Aprano <steve at pearwood.info> writes:
> 
> [...]
>> But you still find a few people here and there who have been exposed to
>> Java foolishness, and will argue that Python is "pass by value, where the
>> value is an implementation dependent reference to the thing that you
>> thought was the value".
> 
> I find this clear and concise. Can you exhibit an example that would not
> match this description?
> 
>> In other words, according to this Java philosophy, following `x = 23`,
>> the value of x is not 23 like any sane person would expect, but some
>> invisible and unknown, and unknowable, reference to 23.
> 
> No, Java doesn't work like that for primitive types (assuming that by
> "Java" you mean the language and execution environment defined in
> reference documents).

Perhaps the fact that I used Python syntax was too ambiguous <wink> but I
was talking about Python. However, the same applies to Java, if you
substitute an object for the primitive value:

    Integer x = new Integer(23);

According to the Java philosophy, the value of x is not the object
Integer(23) like any sane person would expect, but some invisible and
unknown reference to that object.

Clear now?



-- 
Steven



From steve at pearwood.info  Thu Jun  4 18:52:34 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 08:52:34 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <mkq5ur$cj6$1@reader1.panix.com>
Message-ID: <5570d6b3$0$12993$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 04:38 am, Grant Edwards wrote:

>>> But, discussing pass-by-this vs. pass-by-that without also discussing
>>> the semantics of the assignment operator is rather pointless.
>>
>> No, that's a red-herring.
> 
> I don't think so. ?The reason that many people seem to confused about
> Python's argument passing is that they don't understand what
> assignment does.


I don't see the connection, but do explain, I'm interested.




-- 
Steven



From cs at zip.com.au  Thu Jun  4 18:58:00 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 5 Jun 2015 08:58:00 +1000
Subject: Multiple thread program problem
In-Reply-To: <ff2d1eb5-95b7-4cfb-b259-c93ea244abc4@googlegroups.com>
References: <ff2d1eb5-95b7-4cfb-b259-c93ea244abc4@googlegroups.com>
Message-ID: <20150604225800.GA31621@cskk.homeip.net>

On 04Jun2015 10:20, M2 <mohan.mohta at gmail.com> wrote:
>Awesome Cameron.
>It works the way I want it to work.

Glad to hear it. A few small remarks:

>Thanks a lot guys.
>Here is the new code:
[...]
>from thread import start_new_thread

You're not using this any more. You may want to tidy this up.

>def proc(col) :
>        P=subprocess.Popen(col, shell=True)
>        return

You are not returning P. But ...

>        co=str("ssh -B ")+ str(com2) + str(com3)
>        P=proc(co)

Here you are saving the return value as P (local to this function). My 
suggestion, I know. The point here is that in later code you might save all the 
P values and call P.wait() for them at some point after the loop, waiting for 
them to terminate. Or you may not care. Your call.

>f.close()

Consider adding the suggested "with" form. Shorter, easier to read, more 
reliable.

Cheers,
Cameron Simpson <cs at zip.com.au>

Ride fast
Die fast
Leave no usable organs
        - Tom Warner <tom at dfind.demon.co.uk>


From random832 at fastmail.us  Thu Jun  4 18:59:33 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 04 Jun 2015 18:59:33 -0400
Subject: Can Python function return multiple data?
In-Reply-To: <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <1433458773.339138.287240305.19445029@webmail.messagingengine.com>

On Thu, Jun 4, 2015, at 18:16, Steven D'Aprano wrote:
> Remember, you've tried to claim that it is not invisible or unknown, so
> you
> must be able to see and know that value. So what is the value?

It doesn't have to have a string representation to exist. But if you
really want one?

>>> object.__repr__(23)
'<int object at 0x10024af00>'


From rosuav at gmail.com  Thu Jun  4 19:04:51 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 5 Jun 2015 09:04:51 +1000
Subject: Can Python function return multiple data?
In-Reply-To: <5570d6b3$0$12993$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net>
 <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <mkq5ur$cj6$1@reader1.panix.com>
 <5570d6b3$0$12993$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmr-17ahN1dcz0+P4HBXiGiKFET51kKJPnU_QBu5tcYCbQ@mail.gmail.com>

On Fri, Jun 5, 2015 at 8:52 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, 5 Jun 2015 04:38 am, Grant Edwards wrote:
>
>>>> But, discussing pass-by-this vs. pass-by-that without also discussing
>>>> the semantics of the assignment operator is rather pointless.
>>>
>>> No, that's a red-herring.
>>
>> I don't think so.  The reason that many people seem to confused about
>> Python's argument passing is that they don't understand what
>> assignment does.
>
>
> I don't see the connection, but do explain, I'm interested.

The connection is that argument passing is, in Python and in many
other languages, a form of assignment, and follows all the same rules
as assignment follows. (It's a kind of magic assignment that spans two
namespaces, but that has nothing to do with pass-by-X questions.)
These two will exhibit the same behaviour:

# Option 1
x = some_object()
y = x
... do something with y ...
... compare y and x

# Option 2
def do_stuff(y):
    ... do something with y ...

x = some_object()
do_stuff(x)
... compare y and x


Python's object semantics go far deeper than just argument passing,
but everyone's confusions about pass-by-X are always about function
calls.

Whether this makes a real difference to anything is a separate question.

ChrisA


From cs at zip.com.au  Thu Jun  4 19:04:58 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 5 Jun 2015 09:04:58 +1000
Subject: Find in ipython3
In-Reply-To: <5570A252.9040908@gmail.com>
References: <5570A252.9040908@gmail.com>
Message-ID: <20150604230458.GA48443@cskk.homeip.net>

On 04Jun2015 13:09, Michael Torrie <torriem at gmail.com> wrote:
>Why not use Python for what it's good for and say pipe the results of
>find into your python script?  Reinventing find poorly isn't going to
>buy you anything.

And several others made similar disparaging remarks. I think you're all missing 
some of the point of Cecil's approach.

I suspect Cecil is a bit like me; he's writing various well understood simple 
external tools in Python as an aid to learning Python and its toolbox. As an 
added bonus one gets a deeper understanding of the nuances of the external 
tools as well, by debugging issues in implementing them.

The point isn't always to make the shortest route to a result; Cecil clearly 
already knows how to invoke the find command, for example. Sometimes the point 
is to understand the path to the result in more detail.

That may not be Cecil's objective, but I've certainly benefitted from writing 
my own version of things that already exist. Obviously you can't do that for 
everything, life is too short, but you can do it for things of interest and as 
a binus you can end up with a feature not available with the standard tool.

Cheers,
Cameron Simpson <cs at zip.com.au>

You should never bet against anything in science at odds of more than about
10^12 to 1.     - Ernest Rutherford


From steve at pearwood.info  Thu Jun  4 19:13:45 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 09:13:45 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <wU2cx.757611$I81.286196@fx33.am4>
Message-ID: <5570dbaa$0$12995$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 06:52 am, BartC wrote:

> On 04/06/2015 18:11, Steven D'Aprano wrote:
> 
>>If there is
>> any language where assignment uses one style and argument passing always
>> uses another, I've never come across it.
> 
> My language does that. I'd be very surprised if it was the only one in
> existence that does so.

I would be. That means that

func(x)

and 

tmp = x
func(tmp)

behave differently, and that would be very surprising to me (and, I think,
most people).

> Assignments involve a deep copy. Argument passing is something
> in-between by-value and by-reference (depending also on the type of data
> involved). 

When you say "something in-between ...", do you mean pass by sharing?


> There is a also an actual by-reference mode (where a pointer 
> is passed).

Passing a pointer is not necessarily by reference. Pass by sharing also uses
a pointer.


-- 
Steven



From stephenppraneel7 at gmail.com  Thu Jun  4 19:15:20 2015
From: stephenppraneel7 at gmail.com (stephenppraneel7 at gmail.com)
Date: Thu, 4 Jun 2015 16:15:20 -0700 (PDT)
Subject: How to inverse a particle emitter
Message-ID: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>

hey, i really need help, im a straight up beginner in scripting and i need to figure out how to make an inverted particle emitter using python in maya


From rgaddi at technologyhighland.invalid  Thu Jun  4 19:31:44 2015
From: rgaddi at technologyhighland.invalid (Rob Gaddi)
Date: Thu, 4 Jun 2015 23:31:44 +0000 (UTC)
Subject: How to inverse a particle emitter
References: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
Message-ID: <mkqn50$6fm$3@dont-email.me>

On Thu, 04 Jun 2015 16:15:20 -0700, stephenppraneel7 wrote:

> hey, i really need help, im a straight up beginner in scripting and i
> need to figure out how to make an inverted particle emitter using python
> in maya

This is why we can't have nice large hadron colliders.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.


From ian.g.kelly at gmail.com  Thu Jun  4 19:35:49 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 4 Jun 2015 17:35:49 -0600
Subject: How to inverse a particle emitter
In-Reply-To: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
References: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
Message-ID: <CALwzidnqTTYtQi6837Td6w5=6Vg+E3to=nMFTcqDoy1By7=8bw@mail.gmail.com>

On Thu, Jun 4, 2015 at 5:15 PM,  <stephenppraneel7 at gmail.com> wrote:
> hey, i really need help, im a straight up beginner in scripting and i need to figure out how to make an inverted particle emitter using python in maya

No idea. This sounds more like a Maya question than a Python question.
Maybe there is a Maya forum that would be a better place to direct
this question.


From stephenppraneel7 at gmail.com  Thu Jun  4 19:47:44 2015
From: stephenppraneel7 at gmail.com (stephenppraneel7 at gmail.com)
Date: Thu, 4 Jun 2015 16:47:44 -0700 (PDT)
Subject: How to inverse a particle emitter
In-Reply-To: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
References: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
Message-ID: <3a007fb2-d8a6-4a00-9567-90f49e82658b@googlegroups.com>

On Thursday, June 4, 2015 at 4:15:29 PM UTC-7, stephenp... at gmail.com wrote:
> hey, i really need help, im a straight up beginner in scripting and i need to figure out how to make an inverted particle emitter using python in maya

unfortunitly i have to make this using python not mel


From bc at freeuk.com  Thu Jun  4 20:16:23 2015
From: bc at freeuk.com (BartC)
Date: Fri, 05 Jun 2015 01:16:23 +0100
Subject: Can Python function return multiple data?
In-Reply-To: <5570dbaa$0$12995$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <wU2cx.757611$I81.286196@fx33.am4>
 <5570dbaa$0$12995$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <uT5cx.584043$oZ4.56869@fx38.am4>

On 05/06/2015 00:13, Steven D'Aprano wrote:
> On Fri, 5 Jun 2015 06:52 am, BartC wrote:
>
>> On 04/06/2015 18:11, Steven D'Aprano wrote:
>>
>>> If there is
>>> any language where assignment uses one style and argument passing always
>>> uses another, I've never come across it.
>>
>> My language does that. I'd be very surprised if it was the only one in
>> existence that does so.
>
> I would be. That means that
>
> func(x)
>
> and
>
> tmp = x
> func(tmp)
>
> behave differently,

Not as far as func() is concerned. But overall there is a difference 
because now tmp contains a copy of x. (Also, if x contains a list for 
example, func() can modify the copy in tmp, not in x. But this shouldn't 
be surprised because the code is different!)

  and that would be very surprising to me (and, I think,
> most people).
>
>> Assignments involve a deep copy. Argument passing is something
>> in-between by-value and by-reference (depending also on the type of data
>> involved).
>
> When you say "something in-between ...", do you mean pass by sharing?

No, it's a rather untidy mechanism which is not intended to be used when 
a parameter is modified (mutated), because the information passed is 
incomplete. But for read-access, or local assignment is used within the 
function, it works as expected.


>> There is a also an actual by-reference mode (where a pointer
>> is passed).
>
> Passing a pointer is not necessarily by reference. Pass by sharing also uses
> a pointer.

This can get tricky to explain. If variables (if you forgive me that 
term) are implemented actually as pointers, then when you pass that 
variable to a function, a copy of the pointer it contains is pushed (I 
think this is how CPython works).

In that case, it is not passed by reference, even though a pointer is 
used. Because the pointer doesn't point /at/ the variable, but with it. 
So if the variable contains a List, its pointer points to the list, and 
the function parameter points at the same list; it can change the list, 
but it can't make the variable point to something else.

When I use pass-by-reference then, regardless of whether variables 
already make use of pointers, I need to construct an extra pointer that 
points /at/ the variable (and box it in my case).

(Example of how I think CPython works:

  def fn():
     a = 5622
     b = [10,20,30]

For this purpose, a and b are locals, and they are allocated on some 
sort of stack, which gives them one pointer each. After those 
assignments, they might contain:

   a (pointer 1003)  ->    1003: [int 5622 ....]
   b (pointer 1007)  ->    1007: [list ........]

Now the following is executed:

     fn2(a)

A copy of a, (pointer 1003), is pushed, which points to the int. These 
are immutable anyway. It is impossible (AFAIK) to make a contain 
something else, such as a string, from outside fn().

The language could be extended:

     x = &a

x might now be:

   x (pointer 1012)  ->    1012: [Ref (pointer 1003)]

Now it is possible to call fn2(x), and inside fn2, some dereferencing is 
done (using explicit pointer ops, C-style):

      *x = "XYZ"        # this will change a)

-- 
Bartc


From gary.herron at islandtraining.com  Thu Jun  4 21:03:20 2015
From: gary.herron at islandtraining.com (Gary Herron)
Date: Thu, 04 Jun 2015 18:03:20 -0700
Subject: How to inverse a particle emitter
In-Reply-To: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
References: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
Message-ID: <5570F558.3050205@islandtraining.com>

On 06/04/2015 04:15 PM, stephenppraneel7 at gmail.com wrote:
> hey, i really need help, im a straight up beginner in scripting and i need to figure out how to make an inverted particle emitter using python in maya

Python is a programming language.
Maya is a modeling and animation application (or a Mesoamerican 
civilization).
An "inverted particle emitter" sounds like something you'd build in your 
backyard.

You are going to have to put in a *LOT* more time explaining how those 
three are connected in one project.

In truth, I can kind of guess what you want, (although I have no idea 
what "inverse" has to do with anything), but I won't waste my time doing 
so.  If you take the time to carefully explain what you want, then I'm 
sure you will find plenty of people here who will take the time to 
answer you.


Gary Herron


From ian.g.kelly at gmail.com  Thu Jun  4 21:10:06 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 4 Jun 2015 19:10:06 -0600
Subject: How to inverse a particle emitter
In-Reply-To: <3a007fb2-d8a6-4a00-9567-90f49e82658b@googlegroups.com>
References: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
 <3a007fb2-d8a6-4a00-9567-90f49e82658b@googlegroups.com>
Message-ID: <CALwzidm2T3bHnTTA+46eKCK_svBRw2bJawBOXmO4A0ns3yef8Q@mail.gmail.com>

On Thu, Jun 4, 2015 at 5:47 PM,  <stephenppraneel7 at gmail.com> wrote:
> On Thursday, June 4, 2015 at 4:15:29 PM UTC-7, stephenp... at gmail.com wrote:
>> hey, i really need help, im a straight up beginner in scripting and i need to figure out how to make an inverted particle emitter using python in maya
>
> unfortunitly i have to make this using python not mel

I'm sorry if I come across as dismissive, but as I stated before this
is not the place to be looking for expertise on Maya scripting,
regardless of what language you might happen to be using. A little bit
of googling turns up several useful-looking tutorials:

http://www.maya-python.com/
http://zurbrigg.com/maya-python/category/beginning-python-for-maya
https://www.youtube.com/watch?v=eXFGeZZbMzQ

as well as a web forum which looks to be quite active:

https://groups.google.com/forum/#!forum/python_inside_maya


From breamoreboy at yahoo.co.uk  Thu Jun  4 21:40:48 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 05 Jun 2015 02:40:48 +0100
Subject: Can Python function return multiple data?
In-Reply-To: <uT5cx.584043$oZ4.56869@fx38.am4>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <wU2cx.757611$I81.286196@fx33.am4>
 <5570dbaa$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <uT5cx.584043$oZ4.56869@fx38.am4>
Message-ID: <mkqun5$h2d$1@ger.gmane.org>

On 05/06/2015 01:16, BartC wrote:
> On 05/06/2015 00:13, Steven D'Aprano wrote:
>> On Fri, 5 Jun 2015 06:52 am, BartC wrote:
>>
>>> On 04/06/2015 18:11, Steven D'Aprano wrote:
>>>
>>>> If there is
>>>> any language where assignment uses one style and argument passing
>>>> always
>>>> uses another, I've never come across it.
>>>
>>> My language does that. I'd be very surprised if it was the only one in
>>> existence that does so.
>>
>> I would be. That means that
>>
>> func(x)
>>
>> and
>>
>> tmp = x
>> func(tmp)
>>
>> behave differently,
>
> Not as far as func() is concerned. But overall there is a difference
> because now tmp contains a copy of x. (Also, if x contains a list for
> example, func() can modify the copy in tmp, not in x. But this shouldn't
> be surprised because the code is different!)
>

Really?

 >>> x=[1,2,3]
 >>> tmp=x
 >>> def func(thing):
...     thing.clear()
...
 >>> func(tmp)
 >>> tmp
[]
 >>> x
[]

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From steve at pearwood.info  Thu Jun  4 21:48:38 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 11:48:38 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <wU2cx.757611$I81.286196@fx33.am4>
 <5570dbaa$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <uT5cx.584043$oZ4.56869@fx38.am4>
 <mailman.188.1433468465.13271.python-list@python.org>
Message-ID: <5570fff7$0$12992$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 11:40 am, Mark Lawrence wrote:

> On 05/06/2015 01:16, BartC wrote:
>> On 05/06/2015 00:13, Steven D'Aprano wrote:
>>> On Fri, 5 Jun 2015 06:52 am, BartC wrote:
>>>
>>>> On 04/06/2015 18:11, Steven D'Aprano wrote:
>>>>
>>>>> If there is
>>>>> any language where assignment uses one style and argument passing
>>>>> always
>>>>> uses another, I've never come across it.
>>>>
>>>> My language does that. I'd be very surprised if it was the only one in
>>>> existence that does so.

[...] 

> Really?

Probably. I can't be sure, because I've never used Bart's language. But
surely he has no reason to lie about his language, and I'm pretty sure you
can't disprove any claims about his language by running Python code.



-- 
Steven



From invalid at invalid.invalid  Thu Jun  4 22:02:09 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Fri, 5 Jun 2015 02:02:09 +0000 (UTC)
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <mkq5ur$cj6$1@reader1.panix.com>
 <5570d6b3$0$12993$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mkqvv1$779$1@reader1.panix.com>

On 2015-06-04, Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, 5 Jun 2015 04:38 am, Grant Edwards wrote:
>
>>>> But, discussing pass-by-this vs. pass-by-that without also discussing
>>>> the semantics of the assignment operator is rather pointless.
>>>
>>> No, that's a red-herring.
>> 
>> I don't think so. ??The reason that many people seem to confused
>> about Python's argument passing is that they don't understand what
>> assignment does.
>
> I don't see the connection, but do explain, I'm interested.

To many people the difference between pass-by-value and
pass-by-reference is what the effects are of assigning to the formal
parameter from within the function.  Using the model of "assignment"
they're used to using from FORTAN/C/C++ Python appears to behave
weirdly and inconsistently.

But, it's not really because the parameter passing mechnism is
something novel (it's not). It's because of what assignment does.
Python's parameter passing is quite simple: it's exactly the same as
an assignment: it binds a name within the funciton's namespacepace to
an object.  If you don't first understand that's what an assignment
does, then trying to explain the parameter passing using phrases
spelled 'pass-by-<whatever>' isn't going to help.

At least that's how it works in my head.  First, you have to
understand assignment.  Then you can understand what Python parameter
passing does.

Or to be a bit obtuse: Python parameters are passed by value, but all
values are references.

[I once had a CS prof who didn't know there was anything other than
pass-by-reference, but that's another story.]

--
Grant




From rustompmody at gmail.com  Thu Jun  4 22:07:51 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Thu, 4 Jun 2015 19:07:51 -0700 (PDT)
Subject: Can Python function return multiple data?
In-Reply-To: <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <586cafa7-1e9b-4199-acec-0c2bb36bbb54@googlegroups.com>

On Friday, June 5, 2015 at 3:46:47 AM UTC+5:30, Steven D'Aprano wrote:
> On Fri, 5 Jun 2015 03:26 am, random832 wrote:
> > If the value really were 23, the "is vs ==" problem wouldn't exist.
> 
> What "problem"? "is versus ==" is not a problem, it is a feature. The two
> operators do two different things.
> 
> But I am *astonished* that you think that the value of x after x = 23 is not
> 23. If it's not 23, then what is it?
> 
> (Actually, I'm not that astonished. There are some kinds of foolishness that
> only really smart people can fall for.)

Yeah some rather smart people... 

http://en.wikipedia.org/wiki/BLISS#Language_description
[See the part on 'reference to a variable']
In C in
x = y
x is l-value y is r-value and the choice of which attribute to select is
context-sensitive.
In this regimen it is uniform and straightforward, variable only ever
means l-value. r-value is obtained with a dot-operator '.x'
So the usual C's
x = y
in Bliss is 
x = .y

[BTW the lead to the wikipedia article describes Bliss as the best known
system programming language till C made its debut]

Also I believe Algol-68 similarly uniformed variable refs but in the opposite
direction [if I understand it right]
C's
x = y
becomes in Algol-68
ref x := y
[Admittedly the Algol-68 docs are too dense for me so I may be misunderstanding]

In any case my points

1. C's context sensitive variables is quite strange.
Even if (in 2015) its the universally acceptped norm there were more
intelligent/intelligible times...

2. If you think is vs == is not a problem, you are not paying attention to the
repeated arguments/confusions around that subject

[The cognoscenti argue, the ignoramuses are confused. Its probably the other
way round -- the ignorants' displayed confusion probably indicates the cognoscenti's real confusion

3. Functional programming choice to throw the baby out with the bathwater
-- no assignment, no mutation -- is extreme but not unfounded


From torriem at gmail.com  Thu Jun  4 22:17:44 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 04 Jun 2015 20:17:44 -0600
Subject: How to inverse a particle emitter
In-Reply-To: <3a007fb2-d8a6-4a00-9567-90f49e82658b@googlegroups.com>
References: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
 <3a007fb2-d8a6-4a00-9567-90f49e82658b@googlegroups.com>
Message-ID: <557106C8.5030006@gmail.com>

On 06/04/2015 05:47 PM, stephenppraneel7 at gmail.com wrote:
> On Thursday, June 4, 2015 at 4:15:29 PM UTC-7, stephenp... at gmail.com wrote:
>> hey, i really need help, im a straight up beginner in scripting and i need to figure out how to make an inverted particle emitter using python in maya
> 
> unfortunitly i have to make this using python not mel

Well, how would you do it in mel?  Perhaps you could then do it
similarly in Python.  But I suggest that before you can do anything in
Python you have to learn the language a little bit.  You should be able
to read Python code, for example, and tell what it does.  And know how
to implement basic algorithms (loops, if/then, variables, etc).



From torriem at gmail.com  Thu Jun  4 22:23:25 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 04 Jun 2015 20:23:25 -0600
Subject: Find in ipython3
In-Reply-To: <20150604230458.GA48443@cskk.homeip.net>
References: <5570A252.9040908@gmail.com>
 <20150604230458.GA48443@cskk.homeip.net>
Message-ID: <5571081D.2070507@gmail.com>

On 06/04/2015 05:04 PM, Cameron Simpson wrote:
> On 04Jun2015 13:09, Michael Torrie <torriem at gmail.com> wrote:
>> Why not use Python for what it's good for and say pipe the results of
>> find into your python script?  Reinventing find poorly isn't going to
>> buy you anything.
> 
> And several others made similar disparaging remarks. I think you're all missing 
> some of the point of Cecil's approach.

I take your point.  However I was not intending to make a disparaging
remark and certainly didn't expect my post to be taken that way. I've
been down this road before (doing shell scripting things in Python), and
it works pretty well for many things.  Was just sharing my experience is
all.

I don't mean to discourage exploration for exploration's sake.  By all
means have fun. Python certainly is a fun language.


From steve at pearwood.info  Thu Jun  4 22:37:28 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 12:37:28 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
Message-ID: <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 08:59 am, random832 at fastmail.us wrote:

> On Thu, Jun 4, 2015, at 18:16, Steven D'Aprano wrote:
>> Remember, you've tried to claim that it is not invisible or unknown, so
>> you
>> must be able to see and know that value. So what is the value?
> 
> It doesn't have to have a string representation to exist. But if you
> really want one?
> 
>>>> object.__repr__(23)
> '<int object at 0x10024af00>'

That's not a reference to the value. That's a string that describes the
object. If it returned

"twenty-three"

would you claim that the string "twenty three" is the value of x?

I think there is a simple, obvious, common-sense answer here:

* given `x = 23`, the value of x is 23;

* given `x = [1, 2]`, the value of x is the given list [1, 2];

* given `x = "Hello World", the value of x is the string "Hello World";

* given `x = object()`, the value of x is the given instance of type object.

That's so simple and obviously correct. Compare to:

* given `x = 23`, the value of x is not actually 23, but some invisible
reference to 23;

* given `x = [1, 2]`, the value of x is not actually [1, 2], but some
invisible reference to [1, 2]; 

etc. I'm sure you can extrapolate to the other examples.

The *only* reason for making this claim is to rationalise the belief that
passing x to a function is pass by value. Since the list [1, 2] is
obviously not copied, it becomes necessary to find something, anything,
which *is* copied, and identify that as "the value of x". Is there any
other context where you would want to say that the value of x isn't [1, 2]?


-- 
Steven



From rustompmody at gmail.com  Thu Jun  4 23:16:07 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Thu, 4 Jun 2015 20:16:07 -0700 (PDT)
Subject: Can Python function return multiple data?
In-Reply-To: <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>

On Friday, June 5, 2015 at 8:07:41 AM UTC+5:30, Steven D'Aprano wrote:
> On Fri, 5 Jun 2015 08:59 am, random832 wrote:
> 
> > On Thu, Jun 4, 2015, at 18:16, Steven D'Aprano wrote:
> >> Remember, you've tried to claim that it is not invisible or unknown, so
> >> you
> >> must be able to see and know that value. So what is the value?
> > 
> > It doesn't have to have a string representation to exist. But if you
> > really want one?
> > 
> >>>> object.__repr__(23)
> > '<int object at 0x10024af00>'
> 
> That's not a reference to the value. That's a string that describes the
> object. If it returned
> 
> "twenty-three"
> 
> would you claim that the string "twenty three" is the value of x?
> 
> I think there is a simple, obvious, common-sense answer here:
> 
> * given `x = 23`, the value of x is 23;
> 
> * given `x = [1, 2]`, the value of x is the given list [1, 2];
> 
> * given `x = "Hello World", the value of x is the string "Hello World";
> 
> * given `x = object()`, the value of x is the given instance of type object.
> 
> That's so simple and obviously correct. Compare to:
> 
> * given `x = 23`, the value of x is not actually 23, but some invisible
> reference to 23;
> 
> * given `x = [1, 2]`, the value of x is not actually [1, 2], but some
> invisible reference to [1, 2]; 
> 
> etc. I'm sure you can extrapolate to the other examples.
> 
> The *only* reason for making this claim is to rationalise the belief that
> passing x to a function is pass by value. Since the list [1, 2] is
> obviously not copied, it becomes necessary to find something, anything,
> which *is* copied, and identify that as "the value of x". Is there any
> other context where you would want to say that the value of x isn't [1, 2]?
> 

Sounds simple...
Until you realize that what the *is* is emphasizing [your emphasis] is not all
that clear.

Consider the C canonical linked-list example (of some type T):

struct node
{
   T elem;
   struct node *next;
};

Is the list in C to be identified with 'struct node'?

If you will see the usage pattern you would find that in most cases the
usage is for some p of type struct node*
To make it more explicit there are TWO mutually recursive types:

typedef struct node *nodeptr;
struct node
{
  T elem;
  nodeptr next;
}; 

And now if we ask which of node and nodeptr should we identify with the
list type, the answer is not quite clear.
- In all cases of usage we would find we need to use nodeptr
- Except for  "newp = (nodeptr)malloc(sizeof(struct node))"
- However nodeptr has no meaning other than referring (pointing) to 
   something... What?

My conclusion: C does not support lists at all in a first class way.
The C programmer needs to visualize/imagine them in the nebulous gap between
node and nodeptr.

The same situation obtains here in python:

The abstract platonic immutable list is non-existent in python
Use no mutation and the pretence that the list [1,2] *is* the list [1,2] works.

Start using mutation and the ground starts wobbling.
The only way to stay steady (that I know) in this wobbly world is to hold to
two concepts:
1. The abstract platonic immutable type -- good to think with, supported by 
python in only a half-assed way

2. The 'maybe-reference-maybe-value' actual mess

And to keep dancing between these


From cs at zip.com.au  Fri Jun  5 00:28:44 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 5 Jun 2015 14:28:44 +1000
Subject: Find in ipython3
In-Reply-To: <5571081D.2070507@gmail.com>
References: <5571081D.2070507@gmail.com>
Message-ID: <20150605042844.GA3212@cskk.homeip.net>

On 04Jun2015 20:23, Michael Torrie <torriem at gmail.com> wrote:
>On 06/04/2015 05:04 PM, Cameron Simpson wrote:
>> On 04Jun2015 13:09, Michael Torrie <torriem at gmail.com> wrote:
>>> Why not use Python for what it's good for and say pipe the results of
>>> find into your python script?  Reinventing find poorly isn't going to
>>> buy you anything.
>>
>> And several others made similar disparaging remarks. I think you're all missing
>> some of the point of Cecil's approach.
>
>I take your point.  However I was not intending to make a disparaging
>remark and certainly didn't expect my post to be taken that way.

And I should apologise for suggesting you were being disparaging to Cecil as 
such. I simply felt that these comments (don't bother reinventing the wheel, in 
various forms) gave a discouraging tone.

>I've
>been down this road before (doing shell scripting things in Python), and
>it works pretty well for many things.  Was just sharing my experience is
>all.
>I don't mean to discourage exploration for exploration's sake.  By all
>means have fun. Python certainly is a fun language.

Agreed.

Cheers,
Cameron Simpson <cs at zip.com.au>

Gabriel Genellina: See PEP 234 http://www.python.org/dev/peps/pep-0234/
Angus Rodgers:
  You've got to love a language whose documentation contains sentences
  beginning like this:
    "Among its chief virtues are the following four -- no, five -- no,
    six -- points: [...]"
from python-list at python.org


From rustompmody at gmail.com  Fri Jun  5 00:49:39 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Thu, 4 Jun 2015 21:49:39 -0700 (PDT)
Subject: Find in ipython3
In-Reply-To: <mailman.191.1433478539.13271.python-list@python.org>
References: <5571081D.2070507@gmail.com>
 <mailman.191.1433478539.13271.python-list@python.org>
Message-ID: <5b52f3f7-94dc-498f-aaf5-bdc2afefd3d3@googlegroups.com>

On Friday, June 5, 2015 at 9:59:22 AM UTC+5:30, Cameron Simpson wrote:
> On 04Jun2015 20:23, Michael Torrie  wrote:
> >On 06/04/2015 05:04 PM, Cameron Simpson wrote:
> >> On 04Jun2015 13:09, Michael Torrie  wrote:
> >>> Why not use Python for what it's good for and say pipe the results of
> >>> find into your python script?  Reinventing find poorly isn't going to
> >>> buy you anything.
> >>
> >> And several others made similar disparaging remarks. I think you're all missing
> >> some of the point of Cecil's approach.
> >
> >I take your point.  However I was not intending to make a disparaging
> >remark and certainly didn't expect my post to be taken that way.
> 
> And I should apologise for suggesting you were being disparaging to Cecil as 
> such. I simply felt that these comments (don't bother reinventing the wheel, in 
> various forms) gave a discouraging tone.
> 
> >I've
> >been down this road before (doing shell scripting things in Python), and
> >it works pretty well for many things.  Was just sharing my experience is
> >all.
> >I don't mean to discourage exploration for exploration's sake.  By all
> >means have fun. Python certainly is a fun language.

Chuck Moore the inventor of Forth and considered an all-time great among programmers
is in the opposite camp from the modern fad for "reuse done reinvent"
Cant find an original reference. Here is a quote:

| Chuck's philosophy is all about getting rid of everything that is not 
| absolutely necessary for the task at hand, paring both the problem and 
| solution down to the minimum. Compare this with the instinct that most 
| programmers have to generalize every problem and abstract code into 
| "reusable" modules, whether or not that code is ever actually reused. Chuck 
| might say that the best way to write maintainable code is to keep the 
| codebase small enough that it can be rewritten when the requirements change. 
| To program in Forth you have to fight against instincts drawn in from other languages.

https://news.ycombinator.com/item?id=4623770


From alain at universite-de-strasbourg.fr.invalid  Fri Jun  5 02:59:54 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Fri, 05 Jun 2015 08:59:54 +0200
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87r3prjq36.fsf@universite-de-strasbourg.fr.invalid>
 <5570d51a$0$13002$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87mw0ek5cl.fsf@universite-de-strasbourg.fr.invalid>

Steven D'Aprano <steve at pearwood.info> writes:

> On Fri, 5 Jun 2015 04:17 am, Alain Ketterlin wrote:
>
>> Steven D'Aprano <steve at pearwood.info> writes:
>> 
>> [...]
>>> But you still find a few people here and there who have been exposed to
>>> Java foolishness, and will argue that Python is "pass by value, where the
>>> value is an implementation dependent reference to the thing that you
>>> thought was the value".
>> 
>> I find this clear and concise. Can you exhibit an example that would not
>> match this description?
>> 
>>> In other words, according to this Java philosophy, following `x = 23`,
>>> the value of x is not 23 like any sane person would expect, but some
>>> invisible and unknown, and unknowable, reference to 23.
>> 
>> No, Java doesn't work like that for primitive types (assuming that by
>> "Java" you mean the language and execution environment defined in
>> reference documents).
>
> Perhaps the fact that I used Python syntax was too ambiguous <wink> but I
> was talking about Python. However, the same applies to Java, if you
> substitute an object for the primitive value:
>
>     Integer x = new Integer(23);
>
> According to the Java philosophy, the value of x is not the object
> Integer(23) like any sane person would expect,

Yes, clear (except for sanity).

> but some invisible and unknown reference to that object.

I think Python does exactly this (Grant Edwards has a good explanation
of it els ethread).

At least, that is how I understand it, and that is how I teach it, and I
have never met a single case where it failed to describe and explain the
actual behavior of the program.

That's why I was asking for an example to motivate your claim on sanity.

-- Alain.


From alain at universite-de-strasbourg.fr.invalid  Fri Jun  5 03:11:36 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Fri, 05 Jun 2015 09:11:36 +0200
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <mkq5ur$cj6$1@reader1.panix.com>
 <5570d6b3$0$12993$c3e8da3$5496439d@news.astraweb.com>
 <mkqvv1$779$1@reader1.panix.com>
Message-ID: <87iob2k4t3.fsf@universite-de-strasbourg.fr.invalid>

Grant Edwards <invalid at invalid.invalid> writes:

[...]
> Or to be a bit obtuse: Python parameters are passed by value, but all
> values are references.

Exactly, that's a perfect description. There's is no need for a new
name. As a corollary, all names (including "variables" and object
attributes) are references.

-- Alain.


From Cecil at decebal.nl  Fri Jun  5 03:17:31 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 05 Jun 2015 09:17:31 +0200
Subject: Find in ipython3
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.177.1433448836.13271.python-list@python.org>
Message-ID: <87bnguhbec.fsf@Equus.decebal.nl>

Op Thursday 4 Jun 2015 22:13 CEST schreef random:

> On Tue, Jun 2, 2015, at 12:13, Cecil Westerhof wrote:
>> I am thinking about using ipython3 instead of bash. When I want to
>> find a file I can do the following:
>> !find ~ -iname '*python*.pdf'
>> but is there a python way?
>
> Python really isn't a good substitute for a shell, but the normal
> python way to do this task is:
>
> import os, os.path, fnmatch
>
> home = os.path.expanduser('~')  # only needed since you used ~
> for dirpath, dirnames, filenames in os.walk(home):
> print(dirpath)
> for filename in filenames:
> if(fnmatch.fnmatch(filename.lower(), '*python*.pdf')):
> print(os.path.join(dirpath, filename))

I was already thinking along those lines. I made it:
    def find(directory, to_match):
        to_match = to_match.lower()
        results = []
        for dirpath, dirnames, filenames in os.walk(expanduser(directory)):
            for filename in filenames:
                if(fnmatch(filename.lower(), to_match)):
                    results.append(os.path.join(dirpath, filename))
        return results

> Note that if you have filenames with invalid unicode characters (or
> any non-ASCII characters at all on Windows) you may have to do
> additional processing to the filename before printing it. And of
> course instead of printing it you may want to store the filenames in
> a list for further processing. But these are the basic building
> blocks.

I have to look into it further. For one thing default the match should
be case dependent and an option used to make it independent.


> I don't use ipython, so I don't know what it provides if anything to
> make any of this easier.

I think it is useful to have it in Python also, so I should not use
ipython specific things.

In ?~/.ipython/profile_default/startup/00-init.ipy? I have:
    from utilDecebal import find

and now ?find('~', '*Python*.pdf')? gives what I want.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From alex at duboaa.net  Fri Jun  5 05:03:13 2015
From: alex at duboaa.net (Alexis Dubois)
Date: Fri, 5 Jun 2015 02:03:13 -0700 (PDT)
Subject: Python.exe has stopped working
In-Reply-To: <8a6ed08d-7dd6-4a4a-a466-9d3ad98f8dff@googlegroups.com>
References: <8a6ed08d-7dd6-4a4a-a466-9d3ad98f8dff@googlegroups.com>
Message-ID: <c5370bc0-00ee-49be-9ecd-2e25008a78c5@googlegroups.com>

Anyone else for an idea on that?


From auriocus at gmx.de  Fri Jun  5 05:15:31 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Fri, 05 Jun 2015 11:15:31 +0200
Subject: Python.exe has stopped working
In-Reply-To: <c5370bc0-00ee-49be-9ecd-2e25008a78c5@googlegroups.com>
References: <8a6ed08d-7dd6-4a4a-a466-9d3ad98f8dff@googlegroups.com>
 <c5370bc0-00ee-49be-9ecd-2e25008a78c5@googlegroups.com>
Message-ID: <mkrp99$b6d$1@dont-email.me>

Am 05.06.15 um 11:03 schrieb Alexis Dubois:
> Anyone else for an idea on that?
>
Well, it is a crash on exit. Looks like a memory error inside of PyQT. 
If you've got the time, you could run it inside of a debugger, or 
better, a memory checker like AppVerifier to find the culprit. These 
things are usually quite hard to diagnose, and unless someone has seen 
it here already, the fun starts now ;) Another possible reason might be 
mixing up DLLs from PyQT and another QT installation. You can find out 
by listing all loaded DLLs when the program is running, e.g. from a 
debugger or this tool: 
https://technet.microsoft.com/sysinternals/bb896653.aspx

	Christian


From marko at pacujo.net  Fri Jun  5 05:27:55 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 05 Jun 2015 12:27:55 +0300
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <mkq5ur$cj6$1@reader1.panix.com>
 <5570d6b3$0$12993$c3e8da3$5496439d@news.astraweb.com>
 <mkqvv1$779$1@reader1.panix.com>
 <87iob2k4t3.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <87y4jy4i90.fsf@elektro.pacujo.net>

Alain Ketterlin <alain at universite-de-strasbourg.fr.invalid>:

> Grant Edwards <invalid at invalid.invalid> writes:
>
> [...]
>> Or to be a bit obtuse: Python parameters are passed by value, but all
>> values are references.
>
> Exactly, that's a perfect description. There's is no need for a new
> name. As a corollary, all names (including "variables" and object
> attributes) are references.

We are all confusing each other because of the word "reference" means so
many things.

In Grant's explanation, a "reference" is a pointer, the invisible gluons
that bind, say, a variable to an object.

In Alain's sentence a "reference" is a variable or any other "lvalue"
that can be bound to an object.

Thus, the statement

    a[3] = 4

contains the reference[Alain] "a[3]". After the statement is executed,
the reference[Alain] contains a reference[Grant] to an int object 4.

Now, is it useful to make the distinction and use the verb "contain?"

Yes, it is. Consider these statements:

    a[0] = 4
    a[1] = 4
    a[2] = a[1]

Now, "a[0]", "a[1]" and "a[2]" are three separate references[Alain].
However between the three, they contain a maximum of two distinct
references[Grant], which can be ascertained with an "is" test:

    a[2] is a[1]
    => True

So while the references[Grant] are identical, the references[Alain] are
not. If I'm not mistaken, Python has no means of comparing the
identities of references[Alain].


Marko


From bc at freeuk.com  Fri Jun  5 05:51:01 2015
From: bc at freeuk.com (BartC)
Date: Fri, 05 Jun 2015 10:51:01 +0100
Subject: Can Python function return multiple data?
In-Reply-To: <mkpnqm$gfn$1@reader1.panix.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
Message-ID: <aiecx.889996$3i4.397133@fx29.am4>

On 04/06/2015 15:37, Grant Edwards wrote:
> On 2015-06-04, Marko Rauhamaa <marko at pacujo.net> wrote:
>> Steven D'Aprano <steve at pearwood.info>:
>>
>>> But you still find a few people here and there who have been exposed
>>> to Java foolishness, and will argue that Python is "pass by value,
>>> where the value is an implementation dependent reference to the thing
>>> that you thought was the value".
>>
>> Why fight terminology? Definitions can't be proved right or wrong.
>>
>> Anyway, I would say Python definitely is in the classic pass-by-value
>> camp. Here's a simple test:
>>
>>     def f(x):
>>         x = 3
>>
>>     y = 1
>>     f(y)
>>     print(y)
>>
>> If it prints 1, it's pass by value. If it prints 3, it's pass by
>> reference.
>
> Somebody else might just as honestly say that it's pass by reference:
>
> def f(x):
>      x[2] = 2;
>
> x = ['a','b','c']
> f(x)
> print(x)
>
> If it prints ['a','b','c'], it's pass by value.  If it's pass by
> reference, it prints ['a', 'b', 2].
>
> IMO, it's pass by reference.

No, it's not. Pass-by-reference specifically means (as Steven D'Aprano 
pointed out some weeks back in another thread) that you can directly 
change the caller's 'variable' so that it becomes something else, such 
as a string.

In your example, the caller's x is still a list. And the same list 
(assign it to y before the call. y will be ['a','b','2'] too, and (x is 
y) is True).

To modify, mutate or in any way update the data associated with the 
caller's argument, pass-by-reference is not needed. But it's not exactly 
pass-by-value either.

In a language like Python, such a limited choice would be too naive, 
because pointers and references are also used internally and 
transparently, and they help to confuse matters.

If you want to call it something, perhaps call it call-by-handle. 
Because a handle to the data is being passed. In the same way that if X 
is a handle to a file, a window, image, or any external resource, then:

   func(X)

might update the file, draw into the window or image or whatever, but 
no-one would be talking about whether that file or window is passed by 
value or by reference!

-- 
Bartc



From bc at freeuk.com  Fri Jun  5 06:06:50 2015
From: bc at freeuk.com (BartC)
Date: Fri, 05 Jun 2015 11:06:50 +0100
Subject: Can Python function return multiple data?
In-Reply-To: <5570fff7$0$12992$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <wU2cx.757611$I81.286196@fx33.am4>
 <5570dbaa$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <uT5cx.584043$oZ4.56869@fx38.am4>
 <mailman.188.1433468465.13271.python-list@python.org>
 <5570fff7$0$12992$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <%wecx.985112$wk1.522127@fx19.am4>

On 05/06/2015 02:48, Steven D'Aprano wrote:
> On Fri, 5 Jun 2015 11:40 am, Mark Lawrence wrote:
>
>> On 05/06/2015 01:16, BartC wrote:
>>> On 05/06/2015 00:13, Steven D'Aprano wrote:
>>>> On Fri, 5 Jun 2015 06:52 am, BartC wrote:
>>>>
>>>>> On 04/06/2015 18:11, Steven D'Aprano wrote:
>>>>>
>>>>>> If there is
>>>>>> any language where assignment uses one style and argument passing
>>>>>> always
>>>>>> uses another, I've never come across it.
>>>>>
>>>>> My language does that. I'd be very surprised if it was the only one in
>>>>> existence that does so.
>
> [...]
>
>> Really?
>
> Probably. I can't be sure, because I've never used Bart's language. But
> surely he has no reason to lie about his language, and I'm pretty sure you
> can't disprove any claims about his language by running Python code.

Well, my language does assignments differently.

Using tmp=x[:], the Python example will emulate the behaviour better.

(This is actually what I've always had trouble getting my head around. 
For example:

x=[10,20]
y=[x,x,x]
print (y)

gives:

   [[10,20],[10,20],[10,20]]

So far so good. But now, thousands of lines and minutes of runtime later 
so that everyone's forgotten that exactly y was from x:

  x[0]="Cat"
  print (y)

gives:

    [['Cat',20],['Cat',20],['Cat',20]]

WTF?!

Or:

   y[0][0]=99      # change one element, you think
   print (y)

=> [[99,20],[99,20],[99,20]]      # no, you change half of them!

Anyway, I am now upgrading my language to work the same way, *and* 
getting rid of explicit pointers, references and full pass-by-reference, 
because Python seems to manage without them. I'll find out in few weeks 
if I've made a big mistake!)

-- 
Bartc


From lac at openend.se  Fri Jun  5 06:54:47 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 05 Jun 2015 12:54:47 +0200
Subject: How to inverse a particle emitter
In-Reply-To: Message from stephenppraneel7@gmail.com of "Thu,
 04 Jun 2015 16:15:20 -0700."
 <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
References: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
Message-ID: <201506051054.t55Asmkd030114@fido.openend.se>

In a message of Thu, 04 Jun 2015 16:15:20 -0700, stephenppraneel7 at gmail.com wri
tes:
>hey, i really need help, im a straight up beginner in scripting and i need to figure out how to make an inverted particle emitter using python in maya
>-- 
>https://mail.python.org/mailman/listinfo/python-list
You need the maya-vi mailing list, not this one.
It's here:
https://lists.sourceforge.net/lists/listinfo/mayavi-users

Laura


From steve at pearwood.info  Fri Jun  5 07:06:22 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 21:06:22 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
Message-ID: <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 01:16 pm, Rustom Mody wrote:

> Consider the C canonical linked-list example (of some type T):
> 
> struct node
> {
>    T elem;
>    struct node *next;
> };
> 
> Is the list in C to be identified with 'struct node'?

Lists in C are not first class values. There is no one thing which can be
identified as the entire list, because the C language gives you the tools
to break any "list" abstraction.

In C, there is no single thing which refers to the entire list, *except by
convention*. There is *at least* one node which represents the head of the
list, followed by other nodes:


node->node->node->node->node
             ^
             |
node->node---

(ASCII diagram best viewed in a monospaced font)

But there is nothing which encapsulates the entire list, and indeed the
concept "the list" is ill-defined, as in the above diagram. We can use the
convention that "the list" means "the sequence of nodes, starting from this
node, which we assume is the head of the list", but that's about it. You
might even find some clever way of tracking which node is the head and
which are not in your own code, but C won't do it for you. You might refuse
to create such double-headed lists as shown above, but C won't enforce that
rule for you, you have to enforce it yourself. In short, C doesn't have the
concept of lists as first class values and so anything you do with it is
entirely in your own hands.


[...]
> My conclusion: C does not support lists at all in a first class way.

Exactly!


> The same situation obtains here in python:

Not so.

In Python, lists *are* first class values, and the Python language manages
all the book-keeping for you to ensure that when you look at a list, it is
the whole list, regardless of implementation. From your Python code, you
don't have to do any book-keeping, you don't have to be careful not to
create double-headed lists, or accidentally start in the middle of the
list, since Python simply doesn't allow you to do so from pure Python code,
unlike C. So in Python, we can always consider the list as a encapsulated
whole, regardless of how it is implemented.

(CPython documents that lists are actually arrays of pointers to objects,
but that's an implementation detail we can ignore.)


> The abstract platonic immutable list is non-existent in python

Just pretend that "immutable list" is spelled "tuple".


> Use no mutation and the pretence that the list [1,2] *is* the list [1,2]
> works.
> 
> Start using mutation and the ground starts wobbling.

Mutation is irrelevant here. Some objects are immutable, others are mutable,
but the language semantics are the same for both.

In particular, Python provides as a language feature two ways to compare
objects for identity. Python makes no promises as to when it will re-use
immutable objects, but it does promise that you can always test for re-use
by using `is` or `id()`. So there's no "wobbly ground" here. You can
*always* tell if (supposedly) two objects are in fact the same object or
not, since `is` cannot be shadowed or monkey-patched.


> The only way to stay steady (that I know) in this wobbly world is to hold
> to two concepts:
> 1. The abstract platonic immutable type -- good to think with, supported
> by python in only a half-assed way
> 
> 2. The 'maybe-reference-maybe-value' actual mess
> 
> And to keep dancing between these

All the words are in English, but I have no idea what you are trying to say
here.


-- 
Steven



From alain at universite-de-strasbourg.fr.invalid  Fri Jun  5 08:04:05 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Fri, 05 Jun 2015 14:04:05 +0200
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <87a8wf5z4l.fsf@elektro.pacujo.net> <mkpnqm$gfn$1@reader1.panix.com>
 <557086d6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <mkq5ur$cj6$1@reader1.panix.com>
 <5570d6b3$0$12993$c3e8da3$5496439d@news.astraweb.com>
 <mkqvv1$779$1@reader1.panix.com>
 <87iob2k4t3.fsf@universite-de-strasbourg.fr.invalid>
 <87y4jy4i90.fsf@elektro.pacujo.net>
Message-ID: <87eglqjr9m.fsf@universite-de-strasbourg.fr.invalid>

Marko Rauhamaa <marko at pacujo.net> writes:

> Alain Ketterlin <alain at universite-de-strasbourg.fr.invalid>:
>
>> Grant Edwards <invalid at invalid.invalid> writes:
>>
>> [...]
>>> Or to be a bit obtuse: Python parameters are passed by value, but all
>>> values are references.
>>
>> Exactly, that's a perfect description. There's is no need for a new
>> name. As a corollary, all names (including "variables" and object
>> attributes) are references.
>
> We are all confusing each other because of the word "reference" means so
> many things.
>
> In Grant's explanation, a "reference" is a pointer, the invisible gluons
> that bind, say, a variable to an object.
>
> In Alain's sentence a "reference" is a variable or any other "lvalue"
> that can be bound to an object.
>
> Thus, the statement
>
>     a[3] = 4
>
> contains the reference[Alain] "a[3]". After the statement is executed,
> the reference[Alain] contains a reference[Grant] to an int object 4.

You're right, my wording is imprecise, your interpretation is absolutely
correct.

> Now, is it useful to make the distinction and use the verb "contain?"
>
> Yes, it is. Consider these statements:
>
>     a[0] = 4
>     a[1] = 4
>     a[2] = a[1]
>
> Now, "a[0]", "a[1]" and "a[2]" are three separate references[Alain].
> However between the three, they contain a maximum of two distinct
> references[Grant], which can be ascertained with an "is" test:
>
>     a[2] is a[1]
>     => True
>
> So while the references[Grant] are identical, the references[Alain] are
> not. If I'm not mistaken, Python has no means of comparing the
> identities of references[Alain].

Right.

-- Alain.


From random832 at fastmail.us  Fri Jun  5 08:21:56 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 05 Jun 2015 08:21:56 -0400
Subject: Can Python function return multiple data?
In-Reply-To: <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <1433506916.2183854.287660969.64922BE6@webmail.messagingengine.com>

On Thu, Jun 4, 2015, at 22:37, Steven D'Aprano wrote:

> That's not a reference to the value. That's a string that describes the
> object.

Well, of course. None of these things are strings.

The normal human-readable representation of a reference is an arrow on a
diagram, pointing from a box that represents the variable to a box that
represents the object. But I can't exactly put that in a text-based
email.


From pap at nowhere.invalid  Fri Jun  5 08:46:22 2015
From: pap at nowhere.invalid (Paul Appleby)
Date: Fri, 5 Jun 2015 12:46:22 +0000 (UTC)
Subject: So what's happening here?
Message-ID: <pan.2015.06.05.12.46.21@nowhere.invalid>

I saw somewhere on the net that you can copy a list with slicing. So 
what's happening when I try it with a numpy array?

>>> a = numpy.array([1,2,3])
>>> b = a[:]
>>> a is b
False
>>> b[1] = 9
>>> a
array([1, 9, 3])


From fabien.maussion at gmail.com  Fri Jun  5 08:52:38 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Fri, 05 Jun 2015 14:52:38 +0200
Subject: So what's happening here?
References: <pan.2015.06.05.12.46.21@nowhere.invalid>
Message-ID: <mks62l$6gh$1@speranza.aioe.org>

On 06/05/2015 02:46 PM, Paul Appleby wrote:
> I saw somewhere on the net that you can copy a list with slicing. So
> what's happening when I try it with a numpy array?

Python lists and numpy arrays are NOT the same thing. This is one of the 
reasons why numpy was developed in the first place. Numpy uses views, 
not copies. There is however a .copy() method on numpy arrays

Fabien


From larry.martell at gmail.com  Fri Jun  5 08:55:09 2015
From: larry.martell at gmail.com (Larry Martell)
Date: Fri, 5 Jun 2015 08:55:09 -0400
Subject: So what's happening here?
In-Reply-To: <pan.2015.06.05.12.46.21@nowhere.invalid>
References: <pan.2015.06.05.12.46.21@nowhere.invalid>
Message-ID: <CACwCsY5A6qhCsp3Dn31Xuq9e=10qyJqJBO7=RxtuH2L+na_CfA@mail.gmail.com>

On Fri, Jun 5, 2015 at 8:46 AM, Paul Appleby <pap at nowhere.invalid> wrote:
> I saw somewhere on the net that you can copy a list with slicing. So
> what's happening when I try it with a numpy array?
>
>>>> a = numpy.array([1,2,3])
>>>> b = a[:]
>>>> a is b
> False
>>>> b[1] = 9
>>>> a
> array([1, 9, 3])

is is identity testing, == is equality testing

>>> a = numpy.array([1,2,3])
>>> b = a[:]
>>> a == b
array([ True,  True,  True], dtype=bool)
>>> id(a)
4510409872
>>> id(b)
4510410192


From toddrjen at gmail.com  Fri Jun  5 08:55:11 2015
From: toddrjen at gmail.com (Todd)
Date: Fri, 5 Jun 2015 14:55:11 +0200
Subject: So what's happening here?
In-Reply-To: <pan.2015.06.05.12.46.21@nowhere.invalid>
References: <pan.2015.06.05.12.46.21@nowhere.invalid>
Message-ID: <CAFpSVpLUNm-UEBomhqQFvfXbVVkE1tv60nk5pgH5EpuTy0dqdA@mail.gmail.com>

On Fri, Jun 5, 2015 at 2:46 PM, Paul Appleby <pap at nowhere.invalid> wrote:

> I saw somewhere on the net that you can copy a list with slicing. So
> what's happening when I try it with a numpy array?
>
> >>> a = numpy.array([1,2,3])
> >>> b = a[:]
> >>> a is b
> False
> >>> b[1] = 9
> >>> a
> array([1, 9, 3])
>
>
Numpy arrays are not lists, they are numpy arrays. They are two different
data types with different behaviors.  In lists, slicing is a copy.  In
numpy arrays, it is a view (a data structure representing some part of
another data structure).  You need to explicitly copy the numpy array using
the "copy" method to get a copy rather than a view:

>>> a = numpy.array([1,2,3])
>>> b = a.copy()
>>> a is b
False
>>> b[1] = 9
>>> a
array([1, 2, 3])

Here is how it works with lists:

>>> a = [1,2,3]
>>> b = a[:]
>>> a is b
False
>>> b[1] = 9
>>> a
[1, 2, 3]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150605/274cfcc4/attachment.html>

From __peter__ at web.de  Fri Jun  5 09:09:03 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 05 Jun 2015 15:09:03 +0200
Subject: So what's happening here?
References: <pan.2015.06.05.12.46.21@nowhere.invalid>
Message-ID: <mks71h$sfc$1@ger.gmane.org>

Paul Appleby wrote:

> I saw somewhere on the net that you can copy a list with slicing. So
> what's happening when I try it with a numpy array?
> 
>>>> a = numpy.array([1,2,3])
>>>> b = a[:]
>>>> a is b
> False
>>>> b[1] = 9
>>>> a
> array([1, 9, 3])

Copy or view -- have a look under the hood:

>>> a = numpy.array([1,2,3])
>>> v = a.view()
>>> c = a.copy()
>>> s = a[:]
>>> a.flags["OWNDATA"]
True
>>> v.flags["OWNDATA"]
False
>>> c.flags["OWNDATA"]
True
>>> s.flags["OWNDATA"]
False

You only get a copy if you ask for one; slicing produces a view.



From pap at nowhere.invalid  Fri Jun  5 09:11:13 2015
From: pap at nowhere.invalid (Paul Appleby)
Date: Fri, 5 Jun 2015 13:11:13 +0000 (UTC)
Subject: So what's happening here?
References: <pan.2015.06.05.12.46.21@nowhere.invalid>
 <mailman.196.1433508938.13271.python-list@python.org>
Message-ID: <pan.2015.06.05.13.11.13@nowhere.invalid>

On Fri, 05 Jun 2015 14:55:11 +0200, Todd wrote:

> Numpy arrays are not lists, they are numpy arrays. They are two
> different data types with different behaviors.  In lists, slicing is a
> copy.  In numpy arrays, it is a view (a data structure representing some
> part of another data structure).  You need to explicitly copy the numpy
> array using the "copy" method to get a copy rather than a view:

OK, thanks.  I see. 

(I'd have thought that id(a[1]) and id(b[1]) would be the same if they 
were the same element via different "views", but the id's seem to change 
according to rules that I can't fathom.)


From gary.herron at islandtraining.com  Fri Jun  5 09:23:53 2015
From: gary.herron at islandtraining.com (Gary Herron)
Date: Fri, 05 Jun 2015 06:23:53 -0700
Subject: So what's happening here?
In-Reply-To: <pan.2015.06.05.13.11.13@nowhere.invalid>
References: <pan.2015.06.05.12.46.21@nowhere.invalid>
 <mailman.196.1433508938.13271.python-list@python.org>
 <pan.2015.06.05.13.11.13@nowhere.invalid>
Message-ID: <5571A2E9.7050108@islandtraining.com>

On 06/05/2015 06:11 AM, Paul Appleby wrote:
> On Fri, 05 Jun 2015 14:55:11 +0200, Todd wrote:
>
>> Numpy arrays are not lists, they are numpy arrays. They are two
>> different data types with different behaviors.  In lists, slicing is a
>> copy.  In numpy arrays, it is a view (a data structure representing some
>> part of another data structure).  You need to explicitly copy the numpy
>> array using the "copy" method to get a copy rather than a view:
> OK, thanks.  I see.
>
> (I'd have thought that id(a[1]) and id(b[1]) would be the same if they
> were the same element via different "views", but the id's seem to change
> according to rules that I can't fathom.)
Nope.  It's odder than that.  a[1] is still a view into the inderlying 
numpy array, and your id is the id of that view. Each such index 
produces a new such view object.  Check this out:

 >>> import numpy
 >>> a = numpy.array([1,2,3])
 >>> id(a[1])
28392768
 >>> id(a[1])
28409872

This produces two different view of the same underlying object.

Gary Herron







From neng.zhou at gmail.com  Fri Jun  5 09:28:21 2015
From: neng.zhou at gmail.com (neng.zhou at gmail.com)
Date: Fri, 5 Jun 2015 06:28:21 -0700 (PDT)
Subject: GCJ Watersheds
Message-ID: <769ebe12-868c-4e29-8515-c2f68833bcdb@googlegroups.com>

I just posted a program in Picat for the GCJ Watersheds problem. I am wondering how many lines a Python program would require. 

https://groups.google.com/forum/#!topic/picat-lang/1QC3KWwfyA8 

Cheers, 
Neng-Fa 


From rustompmody at gmail.com  Fri Jun  5 09:29:10 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Fri, 5 Jun 2015 06:29:10 -0700 (PDT)
Subject: Can Python function return multiple data?
In-Reply-To: <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>

On Friday, June 5, 2015 at 4:36:35 PM UTC+5:30, Steven D'Aprano wrote:
> On Fri, 5 Jun 2015 01:16 pm, Rustom Mody wrote:
> > The abstract platonic immutable list is non-existent in python
> 
> Just pretend that "immutable list" is spelled "tuple".

Ok lets say I make no fuss about the need to 'pretend'.

And I try...

>>> a=[1,2,3]
>>> b=(a,a)
>>> a
[1, 2, 3]
>>> b
([1, 2, 3], [1, 2, 3])
>>> a.append(4)
>>> b
([1, 2, 3, 4], [1, 2, 3, 4])

> 
> 
> > Use no mutation and the pretence that the list [1,2] *is* the list [1,2]
> > works.
> > 
> > Start using mutation and the ground starts wobbling.
> 
> Mutation is irrelevant here. Some objects are immutable, others are mutable,
> but the language semantics are the same for both.
> 
> In particular, Python provides as a language feature two ways to compare
> objects for identity. Python makes no promises as to when it will re-use
> immutable objects, but it does promise that you can always test for re-use
> by using `is` or `id()`. So there's no "wobbly ground" here. You can
> *always* tell if (supposedly) two objects are in fact the same object or
> not, since `is` cannot be shadowed or monkey-patched.
> 
> 
> > The only way to stay steady (that I know) in this wobbly world is to hold
> > to two concepts:
> > 1. The abstract platonic immutable type -- good to think with, supported
> > by python in only a half-assed way
> > 
> > 2. The 'maybe-reference-maybe-value' actual mess
> > 
> > And to keep dancing between these
> 
> All the words are in English, but I have no idea what you are trying to say
> here.

It was an analogy.
C programmers use lists all right all the time.
However until they see something like python, they dont know that they never
REALLY saw lists. ie lists in python are more first-class than C.

Analogously immutable lists are more first-class than python lists.

Of course the language (we are conversing in) is itself very misleading.
We dont call a number an 'immutable' number. In
x = 3
x = x - 1
it is x that changes, not 3.

Whatever would/could it mean that 3 becomes 2?

If 3 becomes 2 does ? become 2.142? Do circles look different?
Does the normal curve (with a ? in the denominator) start wiggling?

What I am driving at is that immutable numbers are so deeply enshrined in our
culture that anything else is inconceivable.

Yet for lists one needs to spell out
immutable ? tuple
mutable ? list
This means that in our culture lists have not reached the first-class status 
that numbers have.  Python is just following the culture


From toddrjen at gmail.com  Fri Jun  5 09:31:07 2015
From: toddrjen at gmail.com (Todd)
Date: Fri, 5 Jun 2015 15:31:07 +0200
Subject: So what's happening here?
In-Reply-To: <pan.2015.06.05.13.11.13@nowhere.invalid>
References: <pan.2015.06.05.12.46.21@nowhere.invalid>
 <mailman.196.1433508938.13271.python-list@python.org>
 <pan.2015.06.05.13.11.13@nowhere.invalid>
Message-ID: <CAFpSVpLJGG2+4hmDAB7coj4goFs=Yigo68co1irCp1Ro4=G-hw@mail.gmail.com>

On Fri, Jun 5, 2015 at 3:11 PM, Paul Appleby <pap at nowhere.invalid> wrote:

> On Fri, 05 Jun 2015 14:55:11 +0200, Todd wrote:
>
> > Numpy arrays are not lists, they are numpy arrays. They are two
> > different data types with different behaviors.  In lists, slicing is a
> > copy.  In numpy arrays, it is a view (a data structure representing some
> > part of another data structure).  You need to explicitly copy the numpy
> > array using the "copy" method to get a copy rather than a view:
>
> OK, thanks.  I see.
>
> (I'd have thought that id(a[1]) and id(b[1]) would be the same if they
> were the same element via different "views", but the id's seem to change
> according to rules that I can't fathom.)
>
>
a[1] and b[1] are NOT views.   Slices are views, single elements are not.
They are numpy scalars, which are immutable, so returning a view wouldn't
make sense in such a case since you can't modify it anyway.

Taking a slice creates a new view.  It is looking at the same underlying
data, but it is a new object.  So "a[1:] is b[1:]" is False, as is "a[:] is
b".
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150605/0ac854ce/attachment.html>

From marko at pacujo.net  Fri Jun  5 09:37:44 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 05 Jun 2015 16:37:44 +0300
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <mailman.194.1433506919.13271.python-list@python.org>
Message-ID: <87bngu46on.fsf@elektro.pacujo.net>

random832 at fastmail.us:

> The normal human-readable representation of a reference is an arrow on
> a diagram, pointing from a box that represents the variable to a box
> that represents the object. But I can't exactly put that in a
> text-based email.

An *lvalue* is anything that can be assigned to:

   x = 3
   a[3] = 2
   d['hello'] = 'world'

An *rvalue* is anything that can be assigned to an lvalue.

In Python, every rvalue is a *reference* to an *object*.

So your first box is an lvalue, your arrow is an rvalue and your second
box is an object.

The words "lvalue" and "rvalue" smell like C, so Python people tend to
want to find loftier terms at the expense of clarity. Thus instead of
saying,

    The lvalue contains a reference to an object.

they prefer saying,

    The reference is bound to an object.

Unfortunately, that practice renders the meaning of a "reference"
ambiguous.


Marko


From toddrjen at gmail.com  Fri Jun  5 09:39:35 2015
From: toddrjen at gmail.com (Todd)
Date: Fri, 5 Jun 2015 15:39:35 +0200
Subject: So what's happening here?
In-Reply-To: <5571A2E9.7050108@islandtraining.com>
References: <pan.2015.06.05.12.46.21@nowhere.invalid>
 <mailman.196.1433508938.13271.python-list@python.org>
 <pan.2015.06.05.13.11.13@nowhere.invalid>
 <5571A2E9.7050108@islandtraining.com>
Message-ID: <CAFpSVp+h+soT3gydfBwwJvbFrQ0ARTatXruh-7q0x7P21_NQEA@mail.gmail.com>

On Fri, Jun 5, 2015 at 3:23 PM, Gary Herron <gary.herron at islandtraining.com>
wrote:

> On 06/05/2015 06:11 AM, Paul Appleby wrote:
>
>> On Fri, 05 Jun 2015 14:55:11 +0200, Todd wrote:
>>
>>  Numpy arrays are not lists, they are numpy arrays. They are two
>>> different data types with different behaviors.  In lists, slicing is a
>>> copy.  In numpy arrays, it is a view (a data structure representing some
>>> part of another data structure).  You need to explicitly copy the numpy
>>> array using the "copy" method to get a copy rather than a view:
>>>
>> OK, thanks.  I see.
>>
>> (I'd have thought that id(a[1]) and id(b[1]) would be the same if they
>> were the same element via different "views", but the id's seem to change
>> according to rules that I can't fathom.)
>>
> Nope.  It's odder than that.  a[1] is still a view into the inderlying
> numpy array, and your id is the id of that view. Each such index produces a
> new such view object.  Check this out:
>
> >>> import numpy
> >>> a = numpy.array([1,2,3])
> >>> id(a[1])
> 28392768
> >>> id(a[1])
> 28409872
>
> This produces two different view of the same underlying object.
>

a[1] and b[1] are not views:

>>> a[1].flags['OWNDATA']
True
>>> b[1].flags['OWNDATA']
True
>>> a[1:2].flags['OWNDATA']
False
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150605/0a7efc01/attachment.html>

From steve at pearwood.info  Fri Jun  5 09:51:55 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Jun 2015 23:51:55 +1000
Subject: So what's happening here?
References: <pan.2015.06.05.12.46.21@nowhere.invalid>
 <mailman.196.1433508938.13271.python-list@python.org>
 <pan.2015.06.05.13.11.13@nowhere.invalid>
Message-ID: <5571a97c$0$12975$c3e8da3$5496439d@news.astraweb.com>

On Fri, 5 Jun 2015 11:11 pm, Paul Appleby wrote:

> On Fri, 05 Jun 2015 14:55:11 +0200, Todd wrote:
> 
>> Numpy arrays are not lists, they are numpy arrays. They are two
>> different data types with different behaviors.  In lists, slicing is a
>> copy.  In numpy arrays, it is a view (a data structure representing some
>> part of another data structure).  You need to explicitly copy the numpy
>> array using the "copy" method to get a copy rather than a view:
> 
> OK, thanks.  I see.
> 
> (I'd have thought that id(a[1]) and id(b[1]) would be the same if they
> were the same element via different "views", but the id's seem to change
> according to rules that I can't fathom.)

They're the same element, but not the same object.

The id() function operates on an object, and returns some arbitrary ID
number for that object. The only thing promised about that ID number is
that for any two distinct objects existing at the same time, they will have
distinct IDs.

Now, let's see what happens when we extract elements from a regular Python
list:

py> a = [1, 2, 3]
py> x = a[0]
py> y = a[0]
py> x is y
True
py> id(x) == id(y)
True

This tells us that extracting the first (or zeroth, if you prefer) element
from the list gives us the same object each time.

Now let's try it with a numpy array:

py> import numpy as np
py> b = np.array([1, 2, 3])
py> x = b[0]
py> y = b[0]
py> x is y
False
py> id(x), id(y)
(149859472, 151810312)


The IDs are clearly different, therefore they are different objects. What's
going on?

The secret is that lists contain objects, so when you extract the zeroth
item using a[0], you get the same object each time. But numpy arrays do not
contain objects, they are a wrapper around a C array of machine ints.

(The numpy array itself is an object, but the elements of that array are
not.)

This is one of the reasons why numpy is so fast: it can bypass all the
high-level Python object-oriented machinery, and perform calculations using
high-speed, low-level C code taking advantage of unboxed machine primitive
values.

But when you extract an element using b[0], numpy has to give you an object.
(Python itself has no concept of low-level machine values.) So it grabs the
32-bit integer at offset 0, converts it into an object, and returns that.
When you do it again, numpy goes through the same process, and returns a
second object with the same numeric value. Hence, the IDs are different.



-- 
Steven



From gary.herron at islandtraining.com  Fri Jun  5 09:57:59 2015
From: gary.herron at islandtraining.com (Gary Herron)
Date: Fri, 05 Jun 2015 06:57:59 -0700
Subject: So what's happening here?
In-Reply-To: <CAFpSVp+h+soT3gydfBwwJvbFrQ0ARTatXruh-7q0x7P21_NQEA@mail.gmail.com>
References: <pan.2015.06.05.12.46.21@nowhere.invalid>
 <mailman.196.1433508938.13271.python-list@python.org>
 <pan.2015.06.05.13.11.13@nowhere.invalid>
 <5571A2E9.7050108@islandtraining.com>
 <CAFpSVp+h+soT3gydfBwwJvbFrQ0ARTatXruh-7q0x7P21_NQEA@mail.gmail.com>
Message-ID: <5571AAE7.4080004@islandtraining.com>

On 06/05/2015 06:39 AM, Todd wrote:
> On Fri, Jun 5, 2015 at 3:23 PM, Gary Herron 
> <gary.herron at islandtraining.com 
> <mailto:gary.herron at islandtraining.com>> wrote:
>
>     On 06/05/2015 06:11 AM, Paul Appleby wrote:
>
>         On Fri, 05 Jun 2015 14:55:11 +0200, Todd wrote:
>
>             Numpy arrays are not lists, they are numpy arrays. They
>             are two
>             different data types with different behaviors.  In lists,
>             slicing is a
>             copy.  In numpy arrays, it is a view (a data structure
>             representing some
>             part of another data structure).  You need to explicitly
>             copy the numpy
>             array using the "copy" method to get a copy rather than a
>             view:
>
>         OK, thanks.  I see.
>
>         (I'd have thought that id(a[1]) and id(b[1]) would be the same
>         if they
>         were the same element via different "views", but the id's seem
>         to change
>         according to rules that I can't fathom.)
>
>     Nope.  It's odder than that.  a[1] is still a view into the
>     inderlying numpy array, and your id is the id of that view. Each
>     such index produces a new such view object. Check this out:
>
>     >>> import numpy
>     >>> a = numpy.array([1,2,3])
>     >>> id(a[1])
>     28392768
>     >>> id(a[1])
>     28409872
>
>     This produces two different view of the same underlying object.
>
>
> a[1] and b[1] are not views:
>
> >>> a[1].flags['OWNDATA']
> True
> >>> b[1].flags['OWNDATA']
> True
> >>> a[1:2].flags['OWNDATA']
> False

Right.  My bad.  Each execution of a[1] creates a new numpy.int64 object 
with the value from the array.

 >>> type(a[1])
<class 'numpy.int64'>

Each execution of id(a[1]) creates an int64 object which is immediately 
used and then deleted.  Two successive executions of id(a[1]) may or may 
not reuse the same piece of memory, depending on what else is going on 
in memory.  Indeed when I produced the above example with id(a[1]), a 
third and fourth runs of id(a[1]) did indeed repeat 28409872, but they 
are all new creations of an int64 object which happen to use the same 
recently freed bit of memory.

Gary Herron







>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150605/d38d1505/attachment.html>

From steve at pearwood.info  Fri Jun  5 10:07:08 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 06 Jun 2015 00:07:08 +1000
Subject: Sorting in reverse is not the same as sorting then reversing
Message-ID: <5571ad0c$0$13005$c3e8da3$5496439d@news.astraweb.com>

Sorting in reverse does not give the same result as sorting then reversing.

It's easiest to see with a key function:


py> a = ['fox', 'dog', 'DOG', 'cat', 'ape']
py> b = a[:]
py> a.sort(key=str.lower, reverse=True)
py> b.sort(key=str.lower)
py> b.reverse()
py> a
['fox', 'dog', 'DOG', 'cat', 'ape']
py> b
['fox', 'DOG', 'dog', 'cat', 'ape']


Sorting in reverse keeps the initial order of any equal elements unchanged.
Sorting, then reversing, reverses them.


(Thanks to Tim Peters for the tip.)


-- 
Steven



From nobody at nowhere.invalid  Fri Jun  5 10:13:56 2015
From: nobody at nowhere.invalid (Nobody)
Date: Fri, 05 Jun 2015 15:13:56 +0100
Subject: So what's happening here?
References: <pan.2015.06.05.12.46.21@nowhere.invalid>
 <mailman.196.1433508938.13271.python-list@python.org>
 <pan.2015.06.05.13.11.13@nowhere.invalid>
Message-ID: <pan.2015.06.05.14.16.06.63000@nowhere.invalid>

On Fri, 05 Jun 2015 13:11:13 +0000, Paul Appleby wrote:

> (I'd have thought that id(a[1]) and id(b[1]) would be the same if they
> were the same element via different "views", but the id's seem to change
> according to rules that I can't fathom.)

First, a[1] and b[1] aren't views, they're scalars.

Second, different views on the same data are different objects, they just
share the same underlying data. Consider the case where the slice doesn't
cover the entire range:

> a = np.array([1,2,3])
> b = a[:2]
> a
array([1, 2, 3])
> b
array([1, 2])
> id(a)
139682716078288
> id(b)
139682716078368
> b[0] = 99
> a
array([99,  2,  3])
> b
array([99,  2])

The case where a slice *does* cover the entire range isn't special; the
resulting view is still a different object.



From invalid at invalid.invalid  Fri Jun  5 10:30:00 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Fri, 5 Jun 2015 14:30:00 +0000 (UTC)
Subject: How to inverse a particle emitter
References: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
Message-ID: <mksbp8$smn$1@reader1.panix.com>

On 2015-06-04, stephenppraneel7 at gmail.com <stephenppraneel7 at gmail.com> wrote:

> hey, i really need help, im a straight up beginner in scripting and i
> need to figure out how to make an inverted particle emitter using
> python in maya

1) Build a particle emitter

2) Turn it upside down

-- 
Grant Edwards               grant.b.edwards        Yow! HELLO, everybody,
                                  at               I'm a HUMAN!!
                              gmail.com            


From stefan_ml at behnel.de  Fri Jun  5 10:50:15 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 05 Jun 2015 16:50:15 +0200
Subject: Sorting in reverse is not the same as sorting then reversing
In-Reply-To: <5571ad0c$0$13005$c3e8da3$5496439d@news.astraweb.com>
References: <5571ad0c$0$13005$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mkscv8$6q2$1@ger.gmane.org>

Steven D'Aprano schrieb am 05.06.2015 um 16:07:
> Sorting in reverse does not give the same result as sorting then reversing.
> 
> It's easiest to see with a key function:
> 
> py> a = ['fox', 'dog', 'DOG', 'cat', 'ape']
> py> b = a[:]
> py> a.sort(key=str.lower, reverse=True)
> py> b.sort(key=str.lower)
> py> b.reverse()
> py> a
> ['fox', 'dog', 'DOG', 'cat', 'ape']
> py> b
> ['fox', 'DOG', 'dog', 'cat', 'ape']
> 
> Sorting in reverse keeps the initial order of any equal elements unchanged.
> Sorting, then reversing, reverses them.
> 
> (Thanks to Tim Peters for the tip.)

... and for implementing this in the first place. :)

For those of you who didn't know and now got interested, the relevant term
here is "stable sorting". It means that elements that compare equal keep
their relative order. That's a general property of Python's sort algorithm.
All that "reverse=True" does is to change "lower than" into "greater than"
and vice versa for elements that compare unequal. It does not change the
behaviour for elements that compare equal, which means that they keep the
same relative order in both cases (reversed/non-reversed).

Stefan




From __peter__ at web.de  Fri Jun  5 10:59:50 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 05 Jun 2015 16:59:50 +0200
Subject: Sorting in reverse is not the same as sorting then reversing
References: <5571ad0c$0$13005$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mksdh6$d38$1@ger.gmane.org>

Steven D'Aprano wrote:

> Sorting in reverse does not give the same result as sorting then
> reversing.
> 
> It's easiest to see with a key function:
> 
> 
> py> a = ['fox', 'dog', 'DOG', 'cat', 'ape']
> py> b = a[:]
> py> a.sort(key=str.lower, reverse=True)
> py> b.sort(key=str.lower)
> py> b.reverse()
> py> a
> ['fox', 'dog', 'DOG', 'cat', 'ape']
> py> b
> ['fox', 'DOG', 'dog', 'cat', 'ape']
> 
> 
> Sorting in reverse keeps the initial order of any equal elements
> unchanged. Sorting, then reversing, reverses them.

If there were no reverse flag you could reverse, then sort, then reverse 
again.

> (Thanks to Tim Peters for the tip.)




From skip.montanaro at gmail.com  Fri Jun  5 11:28:46 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Fri, 5 Jun 2015 10:28:46 -0500
Subject: Sorting in reverse is not the same as sorting then reversing
In-Reply-To: <mkscv8$6q2$1@ger.gmane.org>
References: <5571ad0c$0$13005$c3e8da3$5496439d@news.astraweb.com>
 <mkscv8$6q2$1@ger.gmane.org>
Message-ID: <CANc-5UyYhG_Jp2oaUUNsdQkK9LuLadpZ1bFQRp5XadNqYrOq1w@mail.gmail.com>

On Fri, Jun 5, 2015 at 9:50 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> [Stable sorting is] a general property of Python's sort algorithm.

And at times an extremely valuable property.

Skip


From alister.nospam.ware at ntlworld.com  Fri Jun  5 11:44:33 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Fri, 5 Jun 2015 15:44:33 +0000 (UTC)
Subject: How to inverse a particle emitter
References: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
Message-ID: <mksg51$vnj$1@speranza.aioe.org>

On Thu, 04 Jun 2015 16:15:20 -0700, stephenppraneel7 wrote:

> hey, i really need help, im a straight up beginner in scripting and i
> need to figure out how to make an inverted particle emitter using python
> in maya

Some very goo pointers from the Python team

https://www.youtube.com/watch?v=tNfGyIW7aHM




-- 
"When in doubt, print 'em out."
-- Karl's Programming Proverb 0x7


From me+python at ixokai.io  Fri Jun  5 12:24:22 2015
From: me+python at ixokai.io (Stephen Hansen)
Date: Fri, 05 Jun 2015 09:24:22 -0700
Subject: Python.exe has stopped working
In-Reply-To: <c5370bc0-00ee-49be-9ecd-2e25008a78c5@googlegroups.com>
References: <8a6ed08d-7dd6-4a4a-a466-9d3ad98f8dff@googlegroups.com>
 <c5370bc0-00ee-49be-9ecd-2e25008a78c5@googlegroups.com>
Message-ID: <1433521462.2317510.287871441.6EA45870@webmail.messagingengine.com>

On Fri, Jun 5, 2015, at 02:03 AM, Alexis Dubois wrote:
> Anyone else for an idea on that?

Sorry, I have no idea.

Have you tried asking on the PyQT mailing list where there is likely
more of a concentration of PyQT expertise?
http://www.riverbankcomputing.com/mailman/listinfo/pyqt

-- 
Stephen Hansen
  m e @ i x o k a i . i o


From nispray at gmail.com  Fri Jun  5 14:10:41 2015
From: nispray at gmail.com (Wesley)
Date: Fri, 5 Jun 2015 11:10:41 -0700 (PDT)
Subject: Get html DOM tree by only basic builtin moudles
In-Reply-To: <mailman.157.1433422910.13271.python-list@python.org>
References: <099a955d-134a-46d6-bdba-61ec2b1eb44f@googlegroups.com>
 <mailman.157.1433422910.13271.python-list@python.org>
Message-ID: <7317a823-5ed6-4078-886b-0d3caf897a84@googlegroups.com>

Hi Laura,
  Sure, I got special requirement that just parse html file into DOM tree, by only general basic modules, and based on my DOM tree structure, draft an bitmap.

  So, could you give me an direction how to get the DOM tree?
Currently, I just think out to use something like stack, I mean, maybe read the file line by line, adding to a stack data structure(list for example), and, then, got the parent/child relation .etc

I don't know if what I said is easy to achieve, I am just trying.
Any better suggestions will be great appreciated.

Thanks.
Wesley

> Elementtree is part of the Python standard library.  You are better off
> using it than rolling your own.   (If you were one of the rare people who
> have some very strange requirements that make you better off writing your
> own, you wouldn't be asking us.  You'd already know.)
> 
> https://docs.python.org/2.7/library/xml.etree.elementtree.html#module-xml.etree.ElementTree
> https://docs.python.org/3.4/library/xml.etree.elementtree.html



From sohcahtoa82 at gmail.com  Fri Jun  5 14:55:23 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Fri, 5 Jun 2015 11:55:23 -0700 (PDT)
Subject: Can Python function return multiple data?
In-Reply-To: <mailman.133.1433368610.13271.python-list@python.org>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
Message-ID: <b2b259e5-7c0c-4c2a-bd2b-7eea960db673@googlegroups.com>

On Wednesday, June 3, 2015 at 2:57:00 PM UTC-7, Mark Lawrence wrote:
> On 03/06/2015 22:35, Chris Angelico wrote:
> > On Wed, Jun 3, 2015 at 11:56 PM, Thomas Rachel
> > <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de>
> > wrote:
> >> Am 03.06.2015 um 01:56 schrieb Chris Angelico:
> >>
> >>> and it's pretty convenient. In C, the nearest equivalent is passing a
> >>> number of pointers as parameters, and having the function fill out
> >>> values. Python's model is a lot closer to what you're saying than C's
> >>> model is :)
> >>
> >>
> >> At least, C functions can return structs...
> >
> > Oh, yes, I forgot about that. Thought that was C++ but not C, partly
> > because I never do it in either language. Although in a sense, a
> > struct is still a single "thing".
> >
> > ChrisA
> >
> 
> Don't forget that C functions can accept structs as input.  Possibly not 
> a good idea as I found out many years ago pre ANSIC when I forgot that 
> little old ampersand, so the compiler didn't pick it up, but then with 
> modern computers having so much memory who really cares if you burn a 
> little bit of stack on structures rather than pointers to structures?
> 
> Now does Python pass by value or by reference?  Happily sits back and 
> waits for 10**6 emails to arrive as this is discussed for the 10**6th time.
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence

Boy, you really called it on the 10**6 emails thing.  This thread certainly exploded.


From ian.g.kelly at gmail.com  Fri Jun  5 15:48:55 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 5 Jun 2015 13:48:55 -0600
Subject: Get html DOM tree by only basic builtin moudles
In-Reply-To: <7317a823-5ed6-4078-886b-0d3caf897a84@googlegroups.com>
References: <099a955d-134a-46d6-bdba-61ec2b1eb44f@googlegroups.com>
 <mailman.157.1433422910.13271.python-list@python.org>
 <7317a823-5ed6-4078-886b-0d3caf897a84@googlegroups.com>
Message-ID: <CALwzidmAa6dY9pT=5gJdBosYnNjufBLbNCqcXNzmDOh5yAAqMA@mail.gmail.com>

On Fri, Jun 5, 2015 at 12:10 PM, Wesley <nispray at gmail.com> wrote:
> Hi Laura,
>   Sure, I got special requirement that just parse html file into DOM tree, by only general basic modules, and based on my DOM tree structure, draft an bitmap.
>
>   So, could you give me an direction how to get the DOM tree?
> Currently, I just think out to use something like stack, I mean, maybe read the file line by line, adding to a stack data structure(list for example), and, then, got the parent/child relation .etc
>
> I don't know if what I said is easy to achieve, I am just trying.
> Any better suggestions will be great appreciated.

If you want to recreate the same DOM structure that would be created
by a browser, the standardized algorithm to do so is very complicated,
but you can find it at
http://www.w3.org/TR/2011/WD-html5-20110113/parsing.html.

If you're not necessarily seeking perfect fidelity, I would encourage
you to try to find some way to incorporate beautifulsoup into your
project. It likely won't produce the same structure that a real
browser would, but it should do well enough to scrape from even badly
malformed html.

I recommend against using an XML parser, because HTML isn't XML, and
such a parser may choke even on perfectly valid HTML such as this:

<!DOCTYPE html>
<html>
  <head><title>Document</title></head>
  <body>
    First line
    <br>
    Second line
  </body>
</html>


From rosuav at gmail.com  Fri Jun  5 17:59:22 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 6 Jun 2015 07:59:22 +1000
Subject: Can Python function return multiple data?
In-Reply-To: <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
Message-ID: <CAPTjJmpYNuLG_Wrau4s6zB01qQfStopsY-Sfyio-SCjBP_zS0Q@mail.gmail.com>

On Fri, Jun 5, 2015 at 11:29 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> On Friday, June 5, 2015 at 4:36:35 PM UTC+5:30, Steven D'Aprano wrote:
>> On Fri, 5 Jun 2015 01:16 pm, Rustom Mody wrote:
>> > The abstract platonic immutable list is non-existent in python
>>
>> Just pretend that "immutable list" is spelled "tuple".
>
> Ok lets say I make no fuss about the need to 'pretend'.
>
> And I try...
>
>>>> a=[1,2,3]
>>>> b=(a,a)
>>>> a
> [1, 2, 3]
>>>> b
> ([1, 2, 3], [1, 2, 3])
>>>> a.append(4)
>>>> b
> ([1, 2, 3, 4], [1, 2, 3, 4])

Congrats! You just proved that an object can itself be immutable, but
can contain references to mutables. Ain't that awesome?

Did you have a point?

> It was an analogy.
> C programmers use lists all right all the time.
> However until they see something like python, they dont know that they never
> REALLY saw lists. ie lists in python are more first-class than C.

If you're talking about the things C programmers use all the time,
they're not linked lists, they're arrays - identified by base pointer
and element count. Linked lists are common in LISPy languages, but not
all that common in C, nor C-derived APIs. Most APIs designed for C
usage are either array-based or generator-based.

http://www.gnu.org/software/libc/manual/html_node/Scanning-Directory-Content.html
Array-based: scandir accepts a pointer-to-pointer, which it populates
with a pointer to freshly-allocated data, and it returns the number of
entries stashed there.

http://www.gnu.org/software/libc/manual/html_node/Opening-a-Directory.html
http://www.gnu.org/software/libc/manual/html_node/Reading_002fClosing-Directory.html
Generator-based: opendir returns a generator, readdir gives the next
result or NULL

http://www.gnu.org/software/libc/manual/html_node/Array-Sort-Function.html
Array-based: pass it a base pointer and a count (and an object size,
since it's completely generic), and it sorts the elements.

Your point is still broadly valid, though; these arrays are not
first-class objects. They do have their own little quirks; so long as
the original array isn't deallocated, you can efficiently work with
views by simply using a base pointer inside the original array and a
length that's shorter than the whole array. Very convenient if you
need it, but most certainly not first-class lists, and can cause
problems if you're not careful. Python lists truly are first-class
objects; "more first-class than C" in the sense that 1 is more true
than 0.

ChrisA


From nispray at gmail.com  Fri Jun  5 19:24:31 2015
From: nispray at gmail.com (Wesley)
Date: Fri, 5 Jun 2015 16:24:31 -0700 (PDT)
Subject: Get html DOM tree by only basic builtin moudles
In-Reply-To: <mailman.206.1433533780.13271.python-list@python.org>
References: <099a955d-134a-46d6-bdba-61ec2b1eb44f@googlegroups.com>
 <mailman.157.1433422910.13271.python-list@python.org>
 <7317a823-5ed6-4078-886b-0d3caf897a84@googlegroups.com>
 <mailman.206.1433533780.13271.python-list@python.org>
Message-ID: <33670551-7df5-487a-8b64-ea85dc165596@googlegroups.com>

> On Fri, Jun 5, 2015 at 12:10 PM, Wesley <nispray at gmail.com> wrote:
> > Hi Laura,
> >   Sure, I got special requirement that just parse html file into DOM tree, by only general basic modules, and based on my DOM tree structure, draft an bitmap.
> >
> >   So, could you give me an direction how to get the DOM tree?
> > Currently, I just think out to use something like stack, I mean, maybe read the file line by line, adding to a stack data structure(list for example), and, then, got the parent/child relation .etc
> >
> > I don't know if what I said is easy to achieve, I am just trying.
> > Any better suggestions will be great appreciated.
> 
> If you want to recreate the same DOM structure that would be created
> by a browser, the standardized algorithm to do so is very complicated,
> but you can find it at
> http://www.w3.org/TR/2011/WD-html5-20110113/parsing.html.
> 
> If you're not necessarily seeking perfect fidelity, I would encourage
> you to try to find some way to incorporate beautifulsoup into your
> project. It likely won't produce the same structure that a real
> browser would, but it should do well enough to scrape from even badly
> malformed html.
> 
> I recommend against using an XML parser, because HTML isn't XML, and
> such a parser may choke even on perfectly valid HTML such as this:
> 
> <!DOCTYPE html>
> <html>
>   <head><title>Document</title></head>
>   <body>
>     First line
>     <br>
>     Second line
>   </body>
> </html>

Hi,
  Hmm, it's really complex.
Currently, I don't need to involve all error handling,and assume html is well formatted, then, generate the DOM tree.

Html sample below:
<!DOCTYPE html>
<!-- saved from url=(0026)http://www.opera.com/about -->
<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  <meta name="description" content="Opera is an independent Scandinavian company that's been in the business of making web browsers since 1994. Read more about Opera Software here.">
  <title>About - Opera Software</title>
  <link rel="apple-touch-icon" sizes="57x57" href="http://d2jc9zwbrclgz3.cloudfront.net/static-heap/da/dafd15591b35d4f81ca96cf7de6582d705850ff0/apple-touch-icon-57x57.png">
</head>
<body screen_capture_injected="true"><div style="position: fixed; top: 0px; left: 0px; height: 0px; width: 0px; z-index: 9999999;"><div style="position: fixed; top: 100%; height: 0px;"><div style="position: relative;"></div></div></div>
<!-- Google Tag Manager -->
<nav class="business-menu">
  <ul>
    <li><a data-action-id="header_item" href="http://operamediaworks.com/">Opera Mediaworks</a></li>
  </ul>
</nav>
<main role="main" class="generic_landing_page">
<h1>Who we are, what we do</h1>  <figure class="visuals">
  <img src="./About - Opera Software_files/pro-kompaniyu.jpg" alt="" width="900" height="424">
</figure>  
<ul class="blocks col3">
<li>
<h3>Vision</h3>
<p>We strive to develop superior products and services for our users around the world, through state-of-the-art technology, innovation, leadership and partnerships.</p><p><a href="http://www.operasoftware.com/company/vision" target="_self">Find out more</a>.</p>
</li>
<li>
</ul>
</main>
<footer class="ns--hf">
<aside>
<div class="hf--extra">
  <h2 class="hf--visuallyhidden">Page language</h2>
  <div id="language" class="hf--language hf--hover-enabled hf--popup-container">
    <input id="language-toggle" class="hf--popup-toggle hf--visuallyhidden" type="checkbox" aria-haspopup="true">
    <label for="language-toggle" class="hf--popup-toggle-label" tabindex="0">
      <span class="hf--hide-overflow">
      <span class="">Select your language:</span>
      <span class="">English</span>
      </span>
    </label>
  </div>
</div>
</aside>
<div class="hf--meta hf--clearfix">
<small class="hf--company">Copyright ? 2014 Opera Software ASA. All rights reserved.
<a data-action-id="footer_item" href="http://www.opera.com/privacy">Privacy.</a> <a data-action-id="footer_item" href="http://www.opera.com/terms">Terms of Use.</a>
</small>
</div>
</footer>
</body></html>


From rosuav at gmail.com  Fri Jun  5 20:32:56 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 6 Jun 2015 10:32:56 +1000
Subject: How to inverse a particle emitter
In-Reply-To: <ebe4nal8th8thofbhduthjral8qlknmbl3@4ax.com>
References: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
 <mksbp8$smn$1@reader1.panix.com>
 <ebe4nal8th8thofbhduthjral8qlknmbl3@4ax.com>
Message-ID: <CAPTjJmo9u5WbO3Mx0HTRirUnu1LWgiYiWcfZ4QS3uX+brWPjJg@mail.gmail.com>

On Sat, Jun 6, 2015 at 10:10 AM, Dennis Lee Bieber
<wlfraed at ix.netcom.com> wrote:
>         That is in the same class as the "lunar lander" game on my college
> mainframe...
>
>         It did not do validity checking of inputs, with the result that one
> could do
>
> -10 lbs thrust at 180 degrees
>
> and GAIN fuel while decelerating.

Hah! The version I played didn't allow you to specify 180 degrees,
although it did allow negative burns. So you could tweak your fuel
supply by careful negative burning, but it required some balancing
work to actually come to a safe landing.

Of course, if you wanted to just quit out in a hurry, all you have to
do is a straight-down burn of -9999999, and bam, you're on the ground,
with a Captain Haddock style of "We shall end up by arriving, I
suppose".

ChrisA


From rustompmody at gmail.com  Fri Jun  5 23:20:17 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Fri, 5 Jun 2015 20:20:17 -0700 (PDT)
Subject: Can Python function return multiple data?
In-Reply-To: <mailman.207.1433541572.13271.python-list@python.org>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
Message-ID: <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>

On Saturday, June 6, 2015 at 3:30:23 AM UTC+5:30, Chris Angelico wrote:
> On Fri, Jun 5, 2015 at 11:29 PM, Rustom Mody wrote:
> > On Friday, June 5, 2015 at 4:36:35 PM UTC+5:30, Steven D'Aprano wrote:
> >> On Fri, 5 Jun 2015 01:16 pm, Rustom Mody wrote:
> >> > The abstract platonic immutable list is non-existent in python
> >>
> >> Just pretend that "immutable list" is spelled "tuple".
> >
> > Ok lets say I make no fuss about the need to 'pretend'.
> >
> > And I try...
> >
> >>>> a=[1,2,3]
> >>>> b=(a,a)
> >>>> a
> > [1, 2, 3]
> >>>> b
> > ([1, 2, 3], [1, 2, 3])
> >>>> a.append(4)
> >>>> b
> > ([1, 2, 3, 4], [1, 2, 3, 4])
> 
> Congrats! You just proved that an object can itself be immutable, but
> can contain references to mutables. Ain't that awesome?
> 
> Did you have a point?

[Under assumption you are not being facetious...]
The word immutuable happens to have existed in English before python.
I also happen to have used it before I knew of python
The two meanings do not match
I am surprised
Is that surprising?

As a parallel here is Dijkstra making fun of AI-ers use of the word
'intelligent'
 http://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD618.html


From random832 at fastmail.us  Fri Jun  5 23:28:00 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 05 Jun 2015 23:28:00 -0400
Subject: Can Python function return multiple data?
In-Reply-To: <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
Message-ID: <1433561280.1469545.288250601.0749B2DA@webmail.messagingengine.com>

On Fri, Jun 5, 2015, at 23:20, Rustom Mody wrote:
> The word immutuable happens to have existed in English before python.
> I also happen to have used it before I knew of python
> The two meanings do not match
> I am surprised
> Is that surprising?

They don't match only if you consider the objects a tuple references to
be part of the tuple.

You cannot change the reference. It will always point to the same list.


From rosuav at gmail.com  Fri Jun  5 23:49:44 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 6 Jun 2015 13:49:44 +1000
Subject: Can Python function return multiple data?
In-Reply-To: <1433561280.1469545.288250601.0749B2DA@webmail.messagingengine.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <1433561280.1469545.288250601.0749B2DA@webmail.messagingengine.com>
Message-ID: <CAPTjJmpt7fa3pNLdt6k0UezZjz8f2oCry0zqny2mJmdAWw88XQ@mail.gmail.com>

On Sat, Jun 6, 2015 at 1:28 PM,  <random832 at fastmail.us> wrote:
> On Fri, Jun 5, 2015, at 23:20, Rustom Mody wrote:
>> The word immutuable happens to have existed in English before python.
>> I also happen to have used it before I knew of python
>> The two meanings do not match
>> I am surprised
>> Is that surprising?
>
> They don't match only if you consider the objects a tuple references to
> be part of the tuple.
>
> You cannot change the reference. It will always point to the same list.

Precisely. I can use indelible ink to write a phone number on a piece
of paper, but that's no guarantee that the same person will always
answer that phone. Doesn't change the indelibility of the writing. If
you want something truly immutable, you have to be careful to
reference only other immutables. In Python, hashability is pretty much
that, but I don't think English has such a concept.

Even if there is a word in English with a slightly different meaning
from its Python meaning, is that so hard to believe? In common usage,
a "dictionary" is something which maps words to their meanings, is
ordered ("dictionary order" being a variant of alphabetical order
which takes into account non-alphabetic characters, diacriticals,
etc), and may have additional information that isn't strictly part of
the mapping. In Python, a "dictionary" is something which maps
hashable values to objects. That's pretty different, but clearly
connected. In many languages, "integer" actually means "integer less
than X" (eg X = 2**32), but people don't complain that that means
they're not integers.

In Python, immutables can contain references to mutables, and the
perceived behaviour of a tuple may be affected by those objects.
They're still immutable.

ChrisA


From steve at pearwood.info  Sat Jun  6 00:50:34 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 06 Jun 2015 14:50:34 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
Message-ID: <55727c1c$0$13000$c3e8da3$5496439d@news.astraweb.com>

On Sat, 6 Jun 2015 01:20 pm, Rustom Mody wrote:

> On Saturday, June 6, 2015 at 3:30:23 AM UTC+5:30, Chris Angelico wrote:

>> Congrats! You just proved that an object can itself be immutable, but
>> can contain references to mutables. Ain't that awesome?
>> 
>> Did you have a point?
> 
> [Under assumption you are not being facetious...]
> The word immutuable happens to have existed in English before python.
> I also happen to have used it before I knew of python
> The two meanings do not match
> I am surprised
> Is that surprising?

Yes, I am surprised that you are surprised. You have been a regular, and
prolific, contributor on this forum for some years now, teach Python, blog
about it. You're quite obviously well-read and experienced. How is it that
you are surprised by such a fundamental part of not just Python's object
model, but of real life objects too?

I suspect you are pretending to be surprised to make a rhetorical point.

Like many English words, "immutable" has a few meanings in plain English.
Bouvier's Law Dictionary included in 1856 this definition:

    IMMUTABLE. What cannot be removed, what is unchangeable.
    The laws of God peing perfect, are immutable, but no 
    human law can be so considered.

Clearly tuples can be removed. They are garbage-collected like any other
values in Python. If nothing else, you can turn the computer off, remove
the RAM, grind it down into the finest powder, and scatter it to the winds.
That surely is enough to remove the tuples <wink> 

So according to Bouvier's definition, tuples are not immutable. But I trust
that we can agree that Bouvier's definition is not useful here.

In object-oriented programming circles, including Python, the relevant
definition is that you cannot add or remove elements from the tuple once it
is instantiated. That is all. If those elements happen to be mutable
themselves, they don't cease to be mutable just because you happen to have
put them in a tuple. This does not happen:

    mylist = []
    mytuple = (None, 1, mylist)
    mylist.append(0)
    => raises an exception

The *tuple* is immutable, not the list. The elements of the list are not
elements of the tuple and don't participate in the guarantee of
tuple-immutability.


> As a parallel here is Dijkstra making fun of AI-ers use of the word
> 'intelligent'
>  http://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD618.html

Nice rant, but as is typical of so many rants by Dijkstra, it's overflowing
with smugness at his own cleverness, and very light on actual reasoning or
sense.


-- 
Steven



From rustompmody at gmail.com  Sat Jun  6 01:28:47 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Fri, 5 Jun 2015 22:28:47 -0700 (PDT)
Subject: Can Python function return multiple data?
In-Reply-To: <mailman.209.1433561282.13271.python-list@python.org>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
Message-ID: <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>

On Saturday, June 6, 2015 at 8:58:13 AM UTC+5:30, rand... at fastmail.us wrote:
> On Fri, Jun 5, 2015, at 23:20, Rustom Mody wrote:
> > The word immutuable happens to have existed in English before python.
> > I also happen to have used it before I knew of python
> > The two meanings do not match
> > I am surprised
> > Is that surprising?
> 
> They don't match only if you consider the objects a tuple references to
> be part of the tuple.
> 
> You cannot change the reference. It will always point to the same list.

You just repeated what Chris said, replacing 'immutable' with 'same'
There was a list: [1,2,3]
At some point that list is found to be(come) [1,2,3,4]
They dont look same to me.

IOW if immutable? is to have cognitive resonance with immutable? it needs to be
'deep-immutable' in analogy to deep-copy.

Anyways... All this is rather far from my original point
Python may or may not have genuine immutables and we may call skin-immutable 
as 'immutable'
We have muddled along thusly for the last 60 years and its called
'imperative programming'.

My point is that without a common conceptual basis there is neither
communication nor understanding.
That common base is usually called math/logic:
Your '1/2/3' matches mine
Your 'and/or/not' matches mine
They are immutable? therefore we can think and communicate with them

With [1,2,3] ([1,2],[1,2]) that is not the case
I regard that as unfortunate
[You are of course free to regard that as ok]


From rosuav at gmail.com  Sat Jun  6 01:29:08 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 6 Jun 2015 15:29:08 +1000
Subject: Can Python function return multiple data?
In-Reply-To: <55727c1c$0$13000$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <55727c1c$0$13000$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmruxBdCHumy2+LrdFs3hkSkq5378Od9SrG6n4qNJL1Ktw@mail.gmail.com>

On Sat, Jun 6, 2015 at 2:50 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> This does not happen:
>
>     mylist = []
>     mytuple = (None, 1, mylist)
>     mylist.append(0)
>     => raises an exception
>
> The *tuple* is immutable, not the list.

What you could have is a "FrozenList" (by analogy with frozenset),
something like this:

class FrozenList(tuple):
    def __new__(*a, **kw):
        self = tuple.__new__(*a, **kw)
        hash(self) # If error, disallow construction
        return self

That would raise the error at the moment of mytuple's creation, not at
mylist's mutation, but that's about as close as I can think of to a
"this truly must be immutable" object in Python. And of course, any
object can lie (or be mistaken) about its hashability and mutability:

>>> def adder(x, inc=1): return x+inc
...
>>> hash(adder)
-9223363254790714706
>>> FrozenList((adder,))
(<function adder at 0x7fcbbcf0aae8>,)
>>> descr = {adder: "Add 1 to a number"}

Looks immutable to me.

>>> adder.__defaults__=2,
>>> next(iter(descr))(6)
8

Oops.

ChrisA


From rustompmody at gmail.com  Sat Jun  6 01:32:51 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Fri, 5 Jun 2015 22:32:51 -0700 (PDT)
Subject: Can Python function return multiple data?
In-Reply-To: <55727c1c$0$13000$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <55727c1c$0$13000$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <afaa02b5-98be-401d-b399-185894c488cb@googlegroups.com>

On Saturday, June 6, 2015 at 10:20:49 AM UTC+5:30, Steven D'Aprano wrote:
> On Sat, 6 Jun 2015 01:20 pm, Rustom Mody wrote:
> > As a parallel here is Dijkstra making fun of AI-ers use of the word
> > 'intelligent'
> >  http://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD618.html
> 
> Nice rant, but as is typical of so many rants by Dijkstra, it's overflowing
> with smugness at his own cleverness, and very light on actual reasoning or
> sense.

The cleverness and smugness has made you miss the point:
Post-computers 'intelligent' has a new meaning on account of abuse of langauge


From rosuav at gmail.com  Sat Jun  6 01:43:37 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 6 Jun 2015 15:43:37 +1000
Subject: Can Python function return multiple data?
In-Reply-To: <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
Message-ID: <CAPTjJmrO6x_hOb_BrV7Dq4crMB5jfTaFbw+aon=+6BUcqKL6+A@mail.gmail.com>

On Sat, Jun 6, 2015 at 3:28 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> You just repeated what Chris said, replacing 'immutable' with 'same'
> There was a list: [1,2,3]
> At some point that list is found to be(come) [1,2,3,4]
> They dont look same to me.

"I'm going shopping, can you get me the shopping list please?"
*goes and fetches a piece of paper from the kitchen*
"Wait, this isn't the right list! This one has more things on it!"

The question of whether or not the thing fetched is indeed the
shopping list is independent of the items on it. The list has an
identity and it has a value (the items needed). If I hand you an empty
list on the basis that the shopping list you placed there last week
was empty, I've destroyed the value of the posted shopping list -
people have added things to it during the week, and they expect those
things to be on the list that gets referenced to make purchases.

ChrisA


From df at see.replyto.invalid  Sat Jun  6 02:52:21 2015
From: df at see.replyto.invalid (Dave Farrance)
Date: Sat, 06 Jun 2015 07:52:21 +0100
Subject: Can Python function return multiple data?
References: <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <55727c1c$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <afaa02b5-98be-401d-b399-185894c488cb@googlegroups.com>
Message-ID: <8155na90ll2vr6lnhppn19fo9vovrnmcrr@4ax.com>

Rustom Mody <rustompmody at gmail.com> wrote:

>On Saturday, June 6, 2015 at 10:20:49 AM UTC+5:30, Steven D'Aprano wrote:
>> On Sat, 6 Jun 2015 01:20 pm, Rustom Mody wrote:
>> > As a parallel here is Dijkstra making fun of AI-ers use of the word
>> > 'intelligent'
>> >  http://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD618.html
>> 
>> Nice rant, but as is typical of so many rants by Dijkstra, it's overflowing
>> with smugness at his own cleverness, and very light on actual reasoning or
>> sense.
>
>The cleverness and smugness has made you miss the point:
>Post-computers 'intelligent' has a new meaning on account of abuse of langauge

Hmm.  I've just read that and I have to say that I've no idea what point
the author was trying to make -- presumably because I'm not intelligent
enough.  I get that he's doing his best to make "Artificial
Intelligentsia" a pejorative term for some reason, but exactly what he's
got against this (imagined?) community or exactly what he imagines their
sin to be is hard to figure out.  I'll note that my own observation of
the common usage of the term "intelligent" in the mentioned context is
that you might indeed describe a terminal an "intelligent terminal" if
it has processor functions, but if you were to describe it as a
"terminal with artificial intelligence", then that would imply a whole
lot more -- which is indeed a language oddity, but that seems to be the
way it is now.


From liik.joonas at gmail.com  Sat Jun  6 05:24:45 2015
From: liik.joonas at gmail.com (Joonas Liik)
Date: Sat, 6 Jun 2015 12:24:45 +0300
Subject: [RELEASED] Python 3.5.0b2 is now available
In-Reply-To: <556CDC7A.6070302@mrabarnett.plus.com>
References: <556BE1A7.1090101@hastings.org>
 <556C469A.2070806@mrabarnett.plus.com> <mkih15$rtr$1@ger.gmane.org>
 <556CDC7A.6070302@mrabarnett.plus.com>
Message-ID: <CAB1GNpRYAPbrO5poQws-5ar25y8QaGdjW_-_z9HNoUEVqCTxuw@mail.gmail.com>

Perhaps its just me, but it seems to me that this release is mighty picky
about annotations.

More specifically everything is fine in the interactive interpreter but the
same code won't fly when run as a file.

example:
def some_function(my_arg:"my random annotation")->"my random return type
annotation":
    pass

..will fail with a syntax error when run from a file.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150606/9728605d/attachment.html>

From steve at pearwood.info  Sat Jun  6 05:56:57 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 06 Jun 2015 19:56:57 +1000
Subject: [RELEASED] Python 3.5.0b2 is now available
References: <556BE1A7.1090101@hastings.org>
 <556C469A.2070806@mrabarnett.plus.com> <mkih15$rtr$1@ger.gmane.org>
 <556CDC7A.6070302@mrabarnett.plus.com>
 <mailman.214.1433582688.13271.python-list@python.org>
Message-ID: <5572c3ea$0$12975$c3e8da3$5496439d@news.astraweb.com>

On Sat, 6 Jun 2015 07:24 pm, Joonas Liik wrote:

> Perhaps its just me, but it seems to me that this release is mighty picky
> about annotations.
> 
> More specifically everything is fine in the interactive interpreter but
> the same code won't fly when run as a file.
> 
> example:
> def some_function(my_arg:"my random annotation")->"my random return type
> annotation":
>     pass
> 
> ..will fail with a syntax error when run from a file.


Three likely possibilities, and one unlikely one:

(1) When you are running from a file, you are accidentally using the wrong
Python interpreter and trying to use Python 2 by mistake.

(2) When you are running from a file, you have invisible control characters
in the file, or the encoding is not set correctly, and are getting a
SyntaxError due to the control characters, nothing to do with annotations.

(3) There is some other syntax error in your file.

And the unlikely one:

(4) There's a bug in the interpreter.


If you would care to inspect the syntax error, you will see that when Python
finds a syntax error, it points at the offending token, or just after it,
using a ^ caret. That may give you some hint as to what the issue is.




-- 
Steven



From Cecil at decebal.nl  Sat Jun  6 05:57:54 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 06 Jun 2015 11:57:54 +0200
Subject: Find in ipython3
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.177.1433448836.13271.python-list@python.org>
 <87bnguhbec.fsf@Equus.decebal.nl>
Message-ID: <874mmlqhul.fsf@Equus.decebal.nl>

On Friday  5 Jun 2015 09:17 CEST, Cecil Westerhof wrote:

> I was already thinking along those lines. I made it:
> def find(directory, to_match):
> to_match = to_match.lower()
> results = []
> for dirpath, dirnames, filenames in os.walk(expanduser(directory)):
> for filename in filenames:
> if(fnmatch(filename.lower(), to_match)):
> results.append(os.path.join(dirpath, filename))
> return results

I have a slightly better variant:
    def find(directory, to_match, ignore_case = False):
        to_match =  to_match + r'$'
        if ignore_case:
            p = re.compile(to_match, re.IGNORECASE)
        else:
            p = re.compile(to_match)
        results = []
        for dirpath, dirnames, filenames in os.walk(expanduser(directory)):
            for filename in filenames:
                if p.match(filename):
                    results.append(os.path.join(dirpath, filename))
        return results

Default it works now case sensitive. But I now use regular expression.
That is a lot more efficient. The old version took 4.4 seconds and
this version takes 2.4 seconds. But the ?!find? version takes about
half a second. Why is this version so much less efficient?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sat Jun  6 06:10:14 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 06 Jun 2015 12:10:14 +0200
Subject: Find in ipython3
References: <5570A252.9040908@gmail.com>
 <mailman.183.1433459104.13271.python-list@python.org>
Message-ID: <87zj4dp2pl.fsf@Equus.decebal.nl>

On Friday  5 Jun 2015 01:04 CEST, Cameron Simpson wrote:

> On 04Jun2015 13:09, Michael Torrie <torriem at gmail.com> wrote:
>> Why not use Python for what it's good for and say pipe the results
>> of find into your python script? Reinventing find poorly isn't
>> going to buy you anything.
>
> And several others made similar disparaging remarks. I think you're
> all missing some of the point of Cecil's approach.

My first experience with this newsgroup was very positive. Lately I
found some remarks not really encouraging (to use a British way of
talking). I am glad with this remark.


> I suspect Cecil is a bit like me; he's writing various well
> understood simple external tools in Python as an aid to learning
> Python and its toolbox. As an added bonus one gets a deeper
> understanding of the nuances of the external tools as well, by
> debugging issues in implementing them.

Yes, this is part of what I am doing: learning to work with Python.

But it is more. If I use shell functions like:
    !find ~ -iname '*python*.pdf'
then I have to switch between different way of working. Now I can use:
    find('~', '.*python.*.pdf', True)
so I do not have to switch my thinking.

And I can use:
    results = find('~', '.*python.*.pdf', True)
to do something with it.

At the moment the problem is that my Python variant takes 5 times as
much time as the shell version. Not a real big problem, but something
to think about.

And as someone said: this will also work in Windows. No plus for me,
but maybe for someone else.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From lac at openend.se  Sat Jun  6 07:07:55 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 06 Jun 2015 13:07:55 +0200
Subject: Find in ipython3
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Sat, 06 Jun 2015 11:57:54 +0200." <874mmlqhul.fsf@Equus.decebal.nl>
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.177.1433448836.13271.python-list@python.org>
 <87bnguhbec.fsf@Equus.decebal.nl><874mmlqhul.fsf@Equus.decebal.nl>
Message-ID: <201506061107.t56B7tN3032685@fido.openend.se>

The !find version is C code optimised to do one thing, find files in
your directory structure, which happens to be what you want to do.
General regular expression matching is harder.

Carl Friedrich Bolz investigated regular expression algorithms and their
implementation to see if this is the sort of task that a JIT can improve.
He blogged about it in 2 posts (part1 and part2).  There are benchmarks
for part2.  Benchmarks in part2.

see:
http://morepypy.blogspot.se/2010/05/efficient-and-elegant-regular.html
http://morepypy.blogspot.se/2010/06/jit-for-regular-expression-matching.html

You may get faster results if you use Matthew Barnett's replacement
for re here: https://pypi.python.org/pypi/regex

You will get faster results if you build your IPython shell to use PyPy,
but I would still be very surprised if it beat the C program find.

Laura



From lac at openend.se  Sat Jun  6 07:39:49 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 06 Jun 2015 13:39:49 +0200
Subject: Python.exe has stopped working
In-Reply-To: Message from Christian Gollwitzer <auriocus@gmx.de>
 of "Fri, 05 Jun 2015 11:15:31 +0200." <mkrp99$b6d$1@dont-email.me>
References: <8a6ed08d-7dd6-4a4a-a466-9d3ad98f8dff@googlegroups.com>
 <c5370bc0-00ee-49be-9ecd-2e25008a78c5@googlegroups.com><mkrp99$b6d$1@dont-email.me>
Message-ID: <201506061139.t56BdnYd000985@fido.openend.se>

In a message of Fri, 05 Jun 2015 11:15:31 +0200, Christian Gollwitzer writes:
>Am 05.06.15 um 11:03 schrieb Alexis Dubois:
>> Anyone else for an idea on that?
>>
>Well, it is a crash on exit. Looks like a memory error inside of PyQT. 
>If you've got the time, you could run it inside of a debugger, or 
>better, a memory checker like AppVerifier to find the culprit. These 
>things are usually quite hard to diagnose, and unless someone has seen 
>it here already, the fun starts now ;) Another possible reason might be 
>mixing up DLLs from PyQT and another QT installation. You can find out 
>by listing all loaded DLLs when the program is running, e.g. from a 
>debugger or this tool: 
>https://technet.microsoft.com/sysinternals/bb896653.aspx
>
>	Christian
>-- 
>https://mail.python.org/mailman/listinfo/python-list

I don't have a windows machine, so I cannot do any more looking for this,
but this may be relevant.
http://stackoverflow.com/questions/20102419/pyqt-5-error-on-exit

Laura


From mal at europython.eu  Sat Jun  6 09:12:23 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Sat, 06 Jun 2015 15:12:23 +0200
Subject: EuroPython 2015: Please configure your tickets
Message-ID: <5572F1B7.8080801@europython.eu>

The EuroPython website supports buying tickets for other people
(friends, colleagues, etc.). As a result, it is necessary to ?assign?
the tickets you buy to either yourself or someone else. The assignment
process is explained below.

Please tell us your preferences
-------------------------------

The reason we?re interested in you applying this configuration as soon
as possible, is that the tickets include a number of preferences which
we need to know about in order to properly prepare the conference.

When assigning tickets you will fill some fields, telling us your
t-shirt size and cut (women or men style) and your diet
preferences. If you don?t provide this information, we cannot assure
we?ll have the right t-shirt available for you. Likewise, if you are
vegetarian or vegan, it is important to mark this in the preferences,
so that we can order enough vegetarian/vegan food.

How to edit the fields of the ticket
------------------------------------

Please see the blog post for full instructions and screenshots and
how the ticket assignment and preference editing works:

http://blog.europython.eu/post/120850074162/europython-2015-please-configure-your-tickets

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From am1r.ar3alan at gmail.com  Sat Jun  6 12:38:57 2015
From: am1r.ar3alan at gmail.com (Amir Arsalan)
Date: Sat, 6 Jun 2015 21:08:57 +0430
Subject: Can Python function return multiple data?
In-Reply-To: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
Message-ID: <CANNmrNbsRc1x3KjVUhBaefDxF+j_GcAc1NyKWMgyAXi0UMBNPA@mail.gmail.com>

you can use yield structure in python for multiple return. ex:

def func(a):
    yield a*2
    print "a*2"
    yield a*3
    print "a*3"
    ...

data = func(5) --> data = (10,15,... )

On Wed, Jun 3, 2015 at 1:57 AM, fl <rxjwg98 at gmail.com> wrote:

> Hi,
>
> I just see the tutorial says Python can return value in function, it does
> not say multiple data results return situation. In C, it is possible.
> How about Python on a multiple data return requirement?
>
>
> Thanks,
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150606/be6b33cf/attachment.html>

From rxjwg98 at gmail.com  Sat Jun  6 13:57:08 2015
From: rxjwg98 at gmail.com (fl)
Date: Sat, 6 Jun 2015 10:57:08 -0700 (PDT)
Subject: Can Python function return multiple data?
In-Reply-To: <mailman.218.1433608740.13271.python-list@python.org>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.218.1433608740.13271.python-list@python.org>
Message-ID: <ccc734ce-f9c7-43b6-a00b-7349c214dd0e@googlegroups.com>

On Saturday, June 6, 2015 at 9:39:19 AM UTC-7, Amir Arsalan wrote:
> you can use yield structure in python for multiple return. ex:
> 
> 
> def func(a):
> ? ? yield a*2
> ? ? print "a*2"
> ? ? yield a*3
> ? ? print "a*3"
> ? ? ...
> 
> 
> data = func(5) --> data = (10,15,... )
> 
> 
> On Wed, Jun 3, 2015 at 1:57 AM, fl <rxj... at gmail.com> wrote:
> Hi,
> 
> 
> 
> I just see the tutorial says Python can return value in function, it does
> 
> not say multiple data results return situation. In C, it is possible.
> 
> How about Python on a multiple data return requirement?
> 
> 
> 
> 
> 
> Thanks,
> 
> --
> 
> https://mail.python.org/mailman/listinfo/python-list

Excuse me. I input the following according to your idea, but I do not 
understand how to use it from the echo. It does not show how to use
the multiple output results. I am a new Python user. Please give a little
more explanation if you could.


Thanks,

>>> def func(a):
	yield a*2
	print "a*2"
	yield a*3
	print "a*3"

>>> data=func(5)
>>> data
<generator object func at 0x02B97260>
>>> list(data)
a*2
a*3
[10, 15]


From doc.mefisto at gmail.com  Sat Jun  6 14:09:37 2015
From: doc.mefisto at gmail.com (doc.mefisto at gmail.com)
Date: Sat, 6 Jun 2015 11:09:37 -0700 (PDT)
Subject: Embedded Python and C Callback functions
Message-ID: <daad7377-7a61-4e9e-a442-4ba48f7212b2@googlegroups.com>

Hi. I'm a newbie in python. But I want embed it in my C program.

There is such method of my class:
@staticmethod
def install_instr_callback(callback):
    # set hook for every change of PC
    m68k.set_instr_hook_callback(callback)

And in my C code there is such callback function:
static PyObject *my_instr_callback(PyObject *self);

I want call install_instr_callback from C with specifying of my callback. I do:
PyObject_CallMethod(myClass, "install_instr_callback", "(O)", my_instr_callback);
But python writes me: [b]null argument to internal routine[/b].

How can I fix this problem?


From davros at bellaliant.net  Sat Jun  6 14:28:29 2015
From: davros at bellaliant.net (John McKenzie)
Date: Sat, 06 Jun 2015 18:28:29 GMT
Subject: Keypress Input
References: <rBHbx.75089$MO7.40532@fx10.iad>
Message-ID: <hZGcx.45402$LN4.24632@fx02.iad>


 Laura and Gary, thank you for your replies. I have three physical 
buttons connected to a Kade device emulating a keyboard. These buttons 
control an LED light strip. So there is no screen, so a GUI did not cross 
my mind. I thought it made sense as it is easily done by other scripting 
languages. Thank you both for pointing my in the right direction.

 It turns out Tkinter is installed on Raspian and my Pi has it. Typing 
import tkinter into the Python interpreter gave me an error, then I 
corrected my spelling. The T should be upper case. No errors with "import 
Tkinter".

 Laura, thank you for typing up example code. I had to remove one indent 
on line 9, but after that it worked on my desktop. The Pi gave an error 
about Tkinter when I tried to run your code but I will work on that. In 
the meantime I will work my basic code out on the desktop and then move 
it over to the Pi, adapting it for and fixing Pi issues then.

 In my mind the Tkinter information I read on the web contradicts the 
examples given with the text, so obviously I am not getting it at all. 
Tkinter seems very confusing to me right now, but I think I just need to 
review the conceptual stuff again and keep trying. Also, I have your 
example, which I can experiment with.
 

 Thanks.













From tjreedy at udel.edu  Sat Jun  6 16:37:32 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 06 Jun 2015 16:37:32 -0400
Subject: [RELEASED] Python 3.5.0b2 is now available
In-Reply-To: <CAB1GNpRYAPbrO5poQws-5ar25y8QaGdjW_-_z9HNoUEVqCTxuw@mail.gmail.com>
References: <556BE1A7.1090101@hastings.org>
 <556C469A.2070806@mrabarnett.plus.com> <mkih15$rtr$1@ger.gmane.org>
 <556CDC7A.6070302@mrabarnett.plus.com>
 <CAB1GNpRYAPbrO5poQws-5ar25y8QaGdjW_-_z9HNoUEVqCTxuw@mail.gmail.com>
Message-ID: <mkvlne$hi$1@ger.gmane.org>

On 6/6/2015 5:24 AM, Joonas Liik wrote:
> Perhaps its just me, but it seems to me that this release is mighty
> picky about annotations.
>
> More specifically everything is fine in the interactive interpreter but
> the same code won't fly when run as a file.
>
> example:
> def some_function(my_arg:"my random annotation")->"my random return type
> annotation":
>      pass
>
> ..will fail with a syntax error when run from a file.

On Win7, 3.5b2, this runs fine both from Idle editor and command line. 
Following on Steven's post, add
import sys; print(sys.version)
to top of file, as I did.

-- 
Terry Jan Reedy



From lac at openend.se  Sat Jun  6 16:52:24 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 06 Jun 2015 22:52:24 +0200
Subject: Keypress Input
In-Reply-To: Message from John McKenzie <davros@bellaliant.net>
 of "Sat, 06 Jun 2015 18:28:29 +0000." <hZGcx.45402$LN4.24632@fx02.iad>
References: <rBHbx.75089$MO7.40532@fx10.iad><hZGcx.45402$LN4.24632@fx02.iad>
Message-ID: <201506062052.t56KqO09013475@fido.openend.se>

In a message of Sat, 06 Jun 2015 18:28:29 +0000, John McKenzie writes:
>
>
> Laura and Gary, thank you for your replies. I have three physical 
>buttons connected to a Kade device emulating a keyboard. These buttons 
>control an LED light strip. So there is no screen, so a GUI did not cross 
>my mind. I thought it made sense as it is easily done by other scripting 
>languages. Thank you both for pointing my in the right direction.
>
> It turns out Tkinter is installed on Raspian and my Pi has it. Typing 
>import tkinter into the Python interpreter gave me an error, then I 
>corrected my spelling. The T should be upper case. No errors with "import 
>Tkinter".
>
> Laura, thank you for typing up example code. I had to remove one indent 
>on line 9, but after that it worked on my desktop. The Pi gave an error 
>about Tkinter when I tried to run your code but I will work on that. In 
>the meantime I will work my basic code out on the desktop and then move 
>it over to the Pi, adapting it for and fixing Pi issues then.
>
> In my mind the Tkinter information I read on the web contradicts the 
>examples given with the text, so obviously I am not getting it at all. 
>Tkinter seems very confusing to me right now, but I think I just need to 
>review the conceptual stuff again and keep trying. Also, I have your 
>example, which I can experiment with.
> 
>
> Thanks.

You are most welcome.  Sorry if I mangled the indent when pasting it
into my mail.

http://infohost.nmt.edu/tcc/help/pubs/tkinter/web/index.html
is useful.
Especially http://infohost.nmt.edu/tcc/help/pubs/tkinter/web/key-names.html

So is http://effbot.org/tkinterbook/

Write back if you run into trouble.  The simple code I sent is just about
binding every key press to something that announces what you pressed.
You will have to bind particular keys, or sequence of keys to what
you want to do.  Write a small example to make sure that tkinter can
talk to your Kade device ok, because if it cannot we will have to
look harder at the problem.

Laura


From rosuav at gmail.com  Sat Jun  6 17:20:42 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 7 Jun 2015 07:20:42 +1000
Subject: Keypress Input
In-Reply-To: <hZGcx.45402$LN4.24632@fx02.iad>
References: <rBHbx.75089$MO7.40532@fx10.iad> <hZGcx.45402$LN4.24632@fx02.iad>
Message-ID: <CAPTjJmo1nXpcmod1gfiKvP0c00Tn=2HVyKVTVea2fj1VzR2QYQ@mail.gmail.com>

On Sun, Jun 7, 2015 at 4:28 AM, John McKenzie <davros at bellaliant.net> wrote:
>  It turns out Tkinter is installed on Raspian and my Pi has it. Typing
> import tkinter into the Python interpreter gave me an error, then I
> corrected my spelling. The T should be upper case. No errors with "import
> Tkinter".

Ah, that means your Python interpreter is the older (version 2) type,
rather than the newer (version 3). As of Python 3.0, the module is
named "tkinter". Use whichever one you have; there are a few other
changes inside the module, but the functionality's mostly the same.

ChrisA


From am1r.ar3alan at gmail.com  Sat Jun  6 17:55:47 2015
From: am1r.ar3alan at gmail.com (Amir Arsalan)
Date: Sun, 7 Jun 2015 02:25:47 +0430
Subject: Can Python function return multiple data?
In-Reply-To: <ccc734ce-f9c7-43b6-a00b-7349c214dd0e@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.218.1433608740.13271.python-list@python.org>
 <ccc734ce-f9c7-43b6-a00b-7349c214dd0e@googlegroups.com>
Message-ID: <CANNmrNYo7op51SpN3-z-5E4BafxODyn2DAUkU_02oE5PGTHfyA@mail.gmail.com>

yield structure can able to you that return list of return by nest in
function. for example you want send a value as enter value to function and
wanna after some calculate send value as return data. 2 way for this :

a) send data as a list
b) send data with yield


example:
a)
def func(a):
        a2 = a*2
        print "a*2"
        a3 = a*3
        print "a*3"
        return a2,a3
------------------------------------------------------------------------------------------
b)
def func(a):
        yield a*2
        print "a*2"
        yield a*3
        print "a*3"


if you dont understand tell me that i complete explain for u.

or send pm on hangouts,


Best,
Amir Arsalan

On Sat, Jun 6, 2015 at 10:27 PM, fl <rxjwg98 at gmail.com> wrote:

> On Saturday, June 6, 2015 at 9:39:19 AM UTC-7, Amir Arsalan wrote:
> > you can use yield structure in python for multiple return. ex:
> >
> >
> > def func(a):
> >     yield a*2
> >     print "a*2"
> >     yield a*3
> >     print "a*3"
> >     ...
> >
> >
> > data = func(5) --> data = (10,15,... )
> >
> >
> > On Wed, Jun 3, 2015 at 1:57 AM, fl <rxj... at gmail.com> wrote:
> > Hi,
> >
> >
> >
> > I just see the tutorial says Python can return value in function, it does
> >
> > not say multiple data results return situation. In C, it is possible.
> >
> > How about Python on a multiple data return requirement?
> >
> >
> >
> >
> >
> > Thanks,
> >
> > --
> >
> > https://mail.python.org/mailman/listinfo/python-list
>
> Excuse me. I input the following according to your idea, but I do not
> understand how to use it from the echo. It does not show how to use
> the multiple output results. I am a new Python user. Please give a little
> more explanation if you could.
>
>
> Thanks,
>
> >>> def func(a):
>         yield a*2
>         print "a*2"
>         yield a*3
>         print "a*3"
>
> >>> data=func(5)
> >>> data
> <generator object func at 0x02B97260>
> >>> list(data)
> a*2
> a*3
> [10, 15]
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150607/4f6b2619/attachment.html>

From steveburrus28 at gmail.com  Sat Jun  6 18:03:34 2015
From: steveburrus28 at gmail.com (Steve Burrus)
Date: Sat, 6 Jun 2015 15:03:34 -0700 (PDT)
Subject: Using "import math".
Message-ID: <d7f2716a-50a0-4a67-b506-bf3ac875482b@googlegroups.com>

I need some help/assistance with using the python "import math" function. Like I am trying to do a = math.sqrt(1000)
                  a.sqrt
but it fails. what am I doing wrong? Thanx for anyone's help.




From cs at zip.com.au  Sat Jun  6 18:14:08 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 7 Jun 2015 08:14:08 +1000
Subject: Using "import math".
In-Reply-To: <d7f2716a-50a0-4a67-b506-bf3ac875482b@googlegroups.com>
References: <d7f2716a-50a0-4a67-b506-bf3ac875482b@googlegroups.com>
Message-ID: <20150606221408.GA33621@cskk.homeip.net>

On 06Jun2015 15:03, Steve Burrus <steveburrus28 at gmail.com> wrote:
>I need some help/assistance with using the python "import math" function. Like I am trying to do a = math.sqrt(1000)
>                  a.sqrt
>but it fails. what am I doing wrong? Thanx for anyone's help.

Please post a tiny but complete example piece of code showing the problem so 
that others can run it. Please also post the error message; if it is a stack 
trace, include the full stack trace.

Thanks,
Cameron Simpson <cs at zip.com.au>

[The Press] had tried to contact usenet but that no one had been available
for comment.  That's odd, I thought, if there's one thing you can say about
usenet, its that there's usually someone available for comment.
        - Paul Taylor


From python at mrabarnett.plus.com  Sat Jun  6 18:35:42 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 06 Jun 2015 23:35:42 +0100
Subject: Using "import math".
In-Reply-To: <d7f2716a-50a0-4a67-b506-bf3ac875482b@googlegroups.com>
References: <d7f2716a-50a0-4a67-b506-bf3ac875482b@googlegroups.com>
Message-ID: <557375BE.2090005@mrabarnett.plus.com>

On 2015-06-06 23:03, Steve Burrus wrote:
> I need some help/assistance with using the python "import math" function. Like I am trying to do a = math.sqrt(1000)
>                    a.sqrt
> but it fails. what am I doing wrong? Thanx for anyone's help.
>
In what way does it fail?

If it printed an error message, copy and paste it in its entirety.



From fomcl at yahoo.com  Sat Jun  6 18:42:27 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Sat, 6 Jun 2015 15:42:27 -0700
Subject: Find in ipython3
In-Reply-To: <201506061107.t56B7tN3032685@fido.openend.se>
Message-ID: <1433630547.66316.YahooMailMobile@web163806.mail.gq1.yahoo.com>

&quot;To run any command at the system shell, simply prefix it with !&quot;
See: https://ipython.org/ipython-doc/dev/interactive/tutorial.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150606/6afec904/attachment.html>

From lac at openend.se  Sat Jun  6 20:31:38 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 07 Jun 2015 02:31:38 +0200
Subject: Find in ipython3
In-Reply-To: Message from Albert-Jan Roskam via Python-list
 <python-list@python.org> of "Sat, 06 Jun 2015 15:42:27 -0700."
 <1433630547.66316.YahooMailMobile@web163806.mail.gq1.yahoo.com>
References: <1433630547.66316.YahooMailMobile@web163806.mail.gq1.yahoo.com>
Message-ID: <201506070031.t570VcO4018891@fido.openend.se>

In a message of Sat, 06 Jun 2015 15:42:27 -0700, Albert-Jan Roskam via Python-l
ist writes:
>&quot;To run any command at the system shell, simply prefix it with !&quot;
>See: https://ipython.org/ipython-doc/dev/interactive/tutorial.html

Please don't top post.

2. He knows this.  He's doing this for fun and to learn things, and for
some aesthetic reason.


From torriem at gmail.com  Sun Jun  7 00:31:22 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sat, 06 Jun 2015 22:31:22 -0600
Subject: Keypress Input
In-Reply-To: <hZGcx.45402$LN4.24632@fx02.iad>
References: <rBHbx.75089$MO7.40532@fx10.iad> <hZGcx.45402$LN4.24632@fx02.iad>
Message-ID: <5573C91A.2060608@gmail.com>

On 06/06/2015 12:28 PM, John McKenzie wrote:
> 
>  Laura and Gary, thank you for your replies. I have three physical 
> buttons connected to a Kade device emulating a keyboard. These buttons 
> control an LED light strip. So there is no screen, so a GUI did not cross 
> my mind. I thought it made sense as it is easily done by other scripting 
> languages. Thank you both for pointing my in the right direction.

I assume this is on a Linux box of some kind, perhaps an embedded computer?

Actually you can indeed do what you want from a command-line program.
The curses module supports a function called "getch" which grabs keys
interactively  from whatever tty or ptty you are running the program on.
 You'll have to consult the docs on curses and how to use it in this
fashion.

The problem is you can't just ssh into your device and run the program
as it will fetch input from the terminal (and your remote keyboard)
which isn't what you want.  But if you can get it to run on the linux
console, it would work.  I have not idea how to do this in a
non-interactive startup fashion.

If your buttons are USB you might be able to read their status via
libusb but that gets complicated as Linux tries to grab it as a HID
keyboard.





From rustompmody at gmail.com  Sun Jun  7 00:35:30 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sat, 6 Jun 2015 21:35:30 -0700 (PDT)
Subject: Can Python function return multiple data?
In-Reply-To: <ccc734ce-f9c7-43b6-a00b-7349c214dd0e@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.218.1433608740.13271.python-list@python.org>
 <ccc734ce-f9c7-43b6-a00b-7349c214dd0e@googlegroups.com>
Message-ID: <e81b65c8-9615-4397-a66f-030a60db008e@googlegroups.com>

On Saturday, June 6, 2015 at 11:27:29 PM UTC+5:30, fl wrote:
> On Saturday, June 6, 2015 at 9:39:19 AM UTC-7, Amir Arsalan wrote:
> > you can use yield structure in python for multiple return. ex:
> > 
> > 
> > def func(a):
> > ? ? yield a*2
> > ? ? print "a*2"
> > ? ? yield a*3
> > ? ? print "a*3"
> > ? ? ...
> > 
> > 
> > data = func(5) --> data = (10,15,... )
> > 
> > 
> > On Wed, Jun 3, 2015 at 1:57 AM, fl wrote:
> > Hi,
> > 
> > 
> > 
> > I just see the tutorial says Python can return value in function, it does
> > 
> > not say multiple data results return situation. In C, it is possible.
> > 
> > How about Python on a multiple data return requirement?
> > 
> > 
> > 
> > 
> > 
> > Thanks,
> > 
> > --
> > 
> > https://mail.python.org/mailman/listinfo/python-list
> 
> Excuse me. I input the following according to your idea, but I do not 
> understand how to use it from the echo. It does not show how to use
> the multiple output results. I am a new Python user. Please give a little
> more explanation if you could.

You still have this question?
Ok apologies for our long rambles far from your question.

But Joel Goldstick gave right at the start a neat and complete answer. I repeat 
it with the tuples made more explicit

def my_function(n):
    return (n, n*2)   # return multiple vals as tuple

# somewhere, sometime else
(a_number, its_double) = my_function(3)  # receive the tuple

Is there something about it that you dont understand or find unsatisfactory?


From steve at pearwood.info  Sun Jun  7 01:33:13 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 15:33:13 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.218.1433608740.13271.python-list@python.org>
Message-ID: <5573d79a$0$12983$c3e8da3$5496439d@news.astraweb.com>

Hi Amir, and welcome!

On Sun, 7 Jun 2015 02:38 am, Amir Arsalan wrote:

> you can use yield structure in python for multiple return. ex:
> 
> def func(a):
>     yield a*2
>     print "a*2"
>     yield a*3
>     print "a*3"
>     ...
> 
> data = func(5) --> data = (10,15,... )

That's actually wrong. If you run:

data = func(5)

data does NOT equal the tuple (10, 15, ...) but it is a generator object.
Iterating over the generator will yield 10, 15 etc one item at a time.


Please remember that the Original Poster is a beginner. He is having trouble
with *basic* Python questions, like returning multiple items from a single
function call. Showing him advanced techniques like generators, especially
when the example you show is wrong, will just confuse him more.



-- 
Steven



From am1r.ar3alan at gmail.com  Sun Jun  7 01:45:51 2015
From: am1r.ar3alan at gmail.com (Amir Arsalan)
Date: Sun, 7 Jun 2015 10:15:51 +0430
Subject: Can Python function return multiple data?
In-Reply-To: <5573d79a$0$12983$c3e8da3$5496439d@news.astraweb.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.218.1433608740.13271.python-list@python.org>
 <5573d79a$0$12983$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CANNmrNYvvc2s9shCLq=rnMU-PhRApOYd23+_cRS0ugsGKUKYoA@mail.gmail.com>

Tnx Steven,

You are right completly.I try more subtilize

thanks for your hint.

On Sun, Jun 7, 2015 at 10:03 AM, Steven D'Aprano <steve at pearwood.info>
wrote:

> Hi Amir, and welcome!
>
> On Sun, 7 Jun 2015 02:38 am, Amir Arsalan wrote:
>
> > you can use yield structure in python for multiple return. ex:
> >
> > def func(a):
> >     yield a*2
> >     print "a*2"
> >     yield a*3
> >     print "a*3"
> >     ...
> >
> > data = func(5) --> data = (10,15,... )
>
> That's actually wrong. If you run:
>
> data = func(5)
>
> data does NOT equal the tuple (10, 15, ...) but it is a generator object.
> Iterating over the generator will yield 10, 15 etc one item at a time.
>
>
> Please remember that the Original Poster is a beginner. He is having
> trouble
> with *basic* Python questions, like returning multiple items from a single
> function call. Showing him advanced techniques like generators, especially
> when the example you show is wrong, will just confuse him more.
>
>
>
> --
> Steven
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150607/75ee58ed/attachment.html>

From steve at pearwood.info  Sun Jun  7 01:47:52 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 15:47:52 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.218.1433608740.13271.python-list@python.org>
 <ccc734ce-f9c7-43b6-a00b-7349c214dd0e@googlegroups.com>
Message-ID: <5573db09$0$12995$c3e8da3$5496439d@news.astraweb.com>

On Sun, 7 Jun 2015 03:57 am, fl wrote:

> Excuse me. I input the following according to your idea, but I do not
> understand how to use it from the echo. It does not show how to use
> the multiple output results. I am a new Python user. Please give a little
> more explanation if you could.
> 
> 
> Thanks,
> 
>>>> def func(a):
> yield a*2
> print "a*2"
> yield a*3
> print "a*3"


This is called a "generator function". It is a special type of function that
be stopped and restarted in the middle. Instead of "return", which exits
the function, you use "yield" which provides a result, then pauses the
function waiting to restart at the next line.

When you call x = func(5), it does NOT execute the body of the function.
Instead, it creates a "generator object", or just "generator". When you
display x, you see something like this:

<generator object func at 0x02B97260>

To run the body of func, we need to run x. There are a few different ways.
The usual way is to iterate over it:

for item in x:
    print "item:", item

This will print:

item: 10
a*2
item: 15
a*3

then stop. Do you understand why?

Once you have done this, the generator is exhausted. There are no more
values left, and so we're done with it.

Another way is to pass the generator to a function like list() or tuple().

Remember, because x is exhausted, you have to start with a new generator.
Just run x = func(5) again, and you will have a brand new fresh generator.
Now call list(x), the list function will iterate over the items yielded,
and assemble them into a list or tuple. "a*2" and "a*3" will be printed,
and the list will be [10, 15].


-- 
Steven



From steve at pearwood.info  Sun Jun  7 01:51:36 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 15:51:36 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
Message-ID: <5573dbe8$0$12995$c3e8da3$5496439d@news.astraweb.com>

On Sat, 6 Jun 2015 03:28 pm, Rustom Mody wrote:

> There was a list: [1,2,3]
> At some point that list is found to be(come) [1,2,3,4]
> They dont look same to me.

When you pour water into an empty bottle, does it turn into a different
bottle?

When you append items to a list (or remove them), it is still the same list.



-- 
Steven



From steve at pearwood.info  Sun Jun  7 01:53:45 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 15:53:45 +1000
Subject: Can Python function return multiple data?
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <55727c1c$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <afaa02b5-98be-401d-b399-185894c488cb@googlegroups.com>
Message-ID: <5573dc6a$0$12997$c3e8da3$5496439d@news.astraweb.com>

On Sat, 6 Jun 2015 03:32 pm, Rustom Mody wrote:

> On Saturday, June 6, 2015 at 10:20:49 AM UTC+5:30, Steven D'Aprano wrote:
>> On Sat, 6 Jun 2015 01:20 pm, Rustom Mody wrote:
>> > As a parallel here is Dijkstra making fun of AI-ers use of the word
>> > 'intelligent'
>> >  http://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD618.html
>> 
>> Nice rant, but as is typical of so many rants by Dijkstra, it's
>> overflowing with smugness at his own cleverness, and very light on actual
>> reasoning or sense.
> 
> The cleverness and smugness has made you miss the point:
> Post-computers 'intelligent' has a new meaning on account of abuse of
> langauge

Why do you think it is abuse of language?

The Oxford dictionary lists eight related but distinct meanings for
intelligence and seven for intelligent. Here is the definition from
Websters that Dijkstra rails against:

    able to perform some of the functions of a computer 
    <an ~ computer terminal>.


The Oxford dictionary describes a similar definition in more detail:

    Of a device, system or machine; able to vary its behaviour
    in response to varying situations, requirements, and past
    experience; spec. (esp. of a computer terminal) having its
    own data-processing capability, incorporating a microprocessor.


Clearly the meaning relates to dumb terminals versus intelligent terminals:
the dumb terminal is unable to vary its own behaviour in response to the
environment, but is reliant on a computer elsewhere, while the intelligent
terminal is able to vary its own behaviour and can perform some of the
functions of a computer. A perfectly sensible (even, dare I say it,
intelligent) meaning of the word.

Far from being an abuse of language, this is exactly the sort of thing that
language is made for.



-- 
Steven



From dieter at handshake.de  Sun Jun  7 02:02:33 2015
From: dieter at handshake.de (dieter)
Date: Sun, 07 Jun 2015 08:02:33 +0200
Subject: Embedded Python and C Callback functions
References: <daad7377-7a61-4e9e-a442-4ba48f7212b2@googlegroups.com>
Message-ID: <87wpzgrr7q.fsf@handshake.de>

doc.mefisto at gmail.com writes:

> Hi. I'm a newbie in python. But I want embed it in my C program.
>
> There is such method of my class:
> @staticmethod
> def install_instr_callback(callback):
>     # set hook for every change of PC
>     m68k.set_instr_hook_callback(callback)
>
> And in my C code there is such callback function:
> static PyObject *my_instr_callback(PyObject *self);
>
> I want call install_instr_callback from C with specifying of my callback. I do:
> PyObject_CallMethod(myClass, "install_instr_callback", "(O)", my_instr_callback);
> But python writes me: [b]null argument to internal routine[/b].

If this output really comes from "PyObject_CallMethod", then
some passed in argument is likely "null" - maybe "myclass"?

The Python-C interface is complicated. It might be advicible to
use e.g. "cython" to get the boilder plate code and some intricacies
taken care of. I have done this with "dm.xmlsec.binding". You
can find in its source also an example for the callback defined
in Python and called from "C".



From kwpolska at gmail.com  Sun Jun  7 02:12:48 2015
From: kwpolska at gmail.com (Chris Warrick)
Date: Sun, 7 Jun 2015 08:12:48 +0200
Subject: Using "import math".
In-Reply-To: <d7f2716a-50a0-4a67-b506-bf3ac875482b@googlegroups.com>
References: <d7f2716a-50a0-4a67-b506-bf3ac875482b@googlegroups.com>
Message-ID: <CAMw+j7Jk0tYCAWUqZ9zwrqxw-pXSdZOh4qBbd9to8NV_-WcA-A@mail.gmail.com>

On 7 Jun 2015 00:06, "Steve Burrus" <steveburrus28 at gmail.com> wrote:
>
> I need some help/assistance with using the python "import math" function.
Like I am trying to do a = math.sqrt(1000)
>                   a.sqrt
> but it fails. what am I doing wrong? Thanx for anyone's help.
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

Here's what your code did:

import math  # make the math module accessible
a = math.sqrt(1000)  # set `a` to the square root of 1000 (?31.62...)
a.sqrt  # get a reference to the sqrt method/attribute of a float, which
does not exist; moreover you are not calling the method/discarding the
value so it would be useless anyways

-- 
Chris Warrick <https://chriswarrick.com>
Sent from my Galaxy S3.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150607/b0247b69/attachment.html>

From Cecil at decebal.nl  Sun Jun  7 02:20:46 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 08:20:46 +0200
Subject: Find in ipython3
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.177.1433448836.13271.python-list@python.org>
 <87bnguhbec.fsf@Equus.decebal.nl> <874mmlqhul.fsf@Equus.decebal.nl>
 <mailman.215.1433588892.13271.python-list@python.org>
Message-ID: <87sia4ox8h.fsf@Equus.decebal.nl>

On Saturday  6 Jun 2015 13:07 CEST, Laura Creighton wrote:

> The !find version is C code optimised to do one thing, find files in
> your directory structure, which happens to be what you want to do.
> General regular expression matching is harder.
>
> Carl Friedrich Bolz investigated regular expression algorithms and
> their implementation to see if this is the sort of task that a JIT
> can improve. He blogged about it in 2 posts (part1 and part2). There
> are benchmarks for part2. Benchmarks in part2.
>
> see:
> http://morepypy.blogspot.se/2010/05/efficient-and-elegant-regular.html
> http://morepypy.blogspot.se/2010/06/jit-for-regular-expression-matching.html
>
> You may get faster results if you use Matthew Barnett's replacement
> for re here: https://pypi.python.org/pypi/regex
>
> You will get faster results if you build your IPython shell to use
> PyPy, but I would still be very surprised if it beat the C program
> find.

I have to look into that. But I prefer to write a version that can be
used by ?everyone?.

It is of-course not a very big program. The difference is significant,
but I do not use find that much. And if it is significant I still can
use the shell version.

There is no gain to get in standard Python? By switching from fnmatch
to re I got almost a speed gain of two. So I was wondering if I could
do more.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun  7 02:27:59 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 08:27:59 +0200
Subject: Testing random
Message-ID: <87oaksowwg.fsf@Equus.decebal.nl>

I wrote a very simple function to test random:
    def test_random(length, multiplier = 10000):
        number_list = length * [0]
        for i in range(length * multiplier):
            number_list[random.randint(0, length - 1)] += 1
        minimum = min(number_list)
        maximum = max(number_list)
        return (minimum, maximum, minimum / maximum)

When running:
    for i in range(1, 7):
        print(test_random(100, 10 ** i))
I get:
    (3, 17, 0.17647058823529413)
    (73, 127, 0.5748031496062992)
    (915, 1086, 0.8425414364640884)
    (9723, 10195, 0.9537027954879843)
    (99348, 100620, 0.9873583780560524)
    (997198, 1002496, 0.9947151908835546)

and when running:
    for i in range(1, 7):
        print(test_random(1000, 10 ** i))
I get:
    (2, 20, 0.1)
    (69, 138, 0.5)
    (908, 1098, 0.8269581056466302)
    (9684, 10268, 0.9431242695753799)
    (99046, 100979, 0.9808574059953059)
    (996923, 1003598, 0.9933489305478886)

It shows that when the first parameter increases the deviation
increases and when the second parameter increases the deviation
decreases. Exactly what you would expect. But what are the ranges you
would expect with a good random function. Then it could be used to
test a random function.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun  7 02:39:51 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 08:39:51 +0200
Subject: Function to show time to execute another function
Message-ID: <87k2vgowco.fsf@Equus.decebal.nl>

Sometimes I just want to know how much time a function takes, but at
the same time I also want the result of the function. For this I wrote
the following function:
    def time_test(function, *args):
        startTime   = time.time()
        results     = function(*args)
        endTime     = time.time()
        print('It took {0} seconds'.format(endTime - startTime))
        return results

I can do:
    time_test(test_random, 100, 10 ** 5)
This outputs:
    It took 17.01685857772827 seconds
and returns:
    (98592, 100833, 0.977775133140936)

When executing:
    time_test(test_random, 100, 10 ** 6)
it outputs:
    It took 165.26371836662292 seconds
and returns:
    (997103, 1002009, 0.9951038363926871)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From cs at zip.com.au  Sun Jun  7 03:38:23 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 7 Jun 2015 17:38:23 +1000
Subject: Find in ipython3
In-Reply-To: <87sia4ox8h.fsf@Equus.decebal.nl>
References: <87sia4ox8h.fsf@Equus.decebal.nl>
Message-ID: <20150607073823.GA34783@cskk.homeip.net>

On 07Jun2015 08:20, Cecil Westerhof <Cecil at decebal.nl> wrote:
>There is no gain to get in standard Python? By switching from fnmatch
>to re I got almost a speed gain of two. So I was wondering if I could
>do more.

Maybe write a few versions: one really dumb using filename == matchstring (like 
-name foo), one using filename.startswith(matchstring) (like -name '*foo'), one 
using matchstring in filename (like -name '*foo*') and your current one. See 
how much these affect the runtime. You _will_ need to make multiple identical 
runs (OSes do lots of caching, and other processes competing for I/O or the CPU 
will also perturb things).

One the topic of multiple runs, have a look at the timeit module, psecificly 
designed for testing code with multiple runs.

Cheers,
Cameron Simpson <cs at zip.com.au>


From breamoreboy at yahoo.co.uk  Sun Jun  7 03:39:20 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 07 Jun 2015 08:39:20 +0100
Subject: Function to show time to execute another function
In-Reply-To: <87k2vgowco.fsf@Equus.decebal.nl>
References: <87k2vgowco.fsf@Equus.decebal.nl>
Message-ID: <ml0sff$efl$1@ger.gmane.org>

On 07/06/2015 07:39, Cecil Westerhof wrote:
> Sometimes I just want to know how much time a function takes, but at
> the same time I also want the result of the function. For this I wrote
> the following function:
>      def time_test(function, *args):
>          startTime   = time.time()
>          results     = function(*args)
>          endTime     = time.time()
>          print('It took {0} seconds'.format(endTime - startTime))
>          return results
>
> I can do:
>      time_test(test_random, 100, 10 ** 5)
> This outputs:
>      It took 17.01685857772827 seconds
> and returns:
>      (98592, 100833, 0.977775133140936)
>
> When executing:
>      time_test(test_random, 100, 10 ** 6)
> it outputs:
>      It took 165.26371836662292 seconds
> and returns:
>      (997103, 1002009, 0.9951038363926871)
>

https://docs.python.org/3/library/timeit.html
https://docs.python.org/3/library/profile.html

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From PointedEars at web.de  Sun Jun  7 04:21:15 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 10:21:15 +0200
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
 <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <2c4d029c-8ea5-465b-8adc-6c35185bd150@googlegroups.com>
Message-ID: <2483375.eHyISxeWLQ@PointedEars.de>

Ned Batchelder wrote:

> On Saturday, May 23, 2015 at 9:01:29 AM UTC-4, Steven D'Aprano wrote:
>> On Sat, 23 May 2015 10:33 pm, Thomas 'PointedEars' Lahn wrote:
>> > If only characters were represented as sequences UTF-16 code units in
>> > ECMAScript implementations like JavaScript, there would not be a
>> > problem beyond the BMP;
>> 
>> Are you being sarcastic?
> 
> IIUC, Thomas' point is that *characters* should be sequences of
> codepoints, not that *strings* should be.

No, my point is that one character should be a sequence of code _units_ (for 
a code point value).  But in ECMAScript implementations (so far), a *code 
point value* equals a character, and that is a problem in ECMAScript because 
there the value range is limited to what can be encoded in 16 bit.  The 
problem starts beyond the BMP where 16 bit are no longer sufficient for a 
code sequence and code point value, and code sequence and code point value 
are no longer equal.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From Cecil at decebal.nl  Sun Jun  7 04:22:17 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 10:22:17 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
Message-ID: <87fv64orly.fsf@Equus.decebal.nl>

On Sunday  7 Jun 2015 09:39 CEST, Mark Lawrence wrote:

> On 07/06/2015 07:39, Cecil Westerhof wrote:
>> Sometimes I just want to know how much time a function takes, but
>> at the same time I also want the result of the function. For this I
>> wrote the following function: def time_test(function, *args):
>> startTime = time.time() results = function(*args) endTime =
>> time.time() print('It took {0} seconds'.format(endTime -
>> startTime)) return results
>>
>> I can do:
>> time_test(test_random, 100, 10 ** 5)
>> This outputs:
>> It took 17.01685857772827 seconds
>> and returns:
>> (98592, 100833, 0.977775133140936)
>>
>> When executing:
>> time_test(test_random, 100, 10 ** 6)
>> it outputs:
>> It took 165.26371836662292 seconds
>> and returns:
>> (997103, 1002009, 0.9951038363926871)
>>
>
> https://docs.python.org/3/library/timeit.html
> https://docs.python.org/3/library/profile.html

That only times the function. I explicitly mentioned I want both the
needed time AND the output.

Sadly the quality of the answers on this list is going down. Here I
get an alternative that does only half what I want and when writing an
alternative for ?!find? I am told I could use ?!find? (which only
works in ipython, not python and which also not works with Windows).

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From doc.mefisto at gmail.com  Sun Jun  7 04:37:23 2015
From: doc.mefisto at gmail.com (doc.mefisto at gmail.com)
Date: Sun, 7 Jun 2015 01:37:23 -0700 (PDT)
Subject: Embedded Python and C Callback functions
In-Reply-To: <mailman.229.1433656976.13271.python-list@python.org>
References: <daad7377-7a61-4e9e-a442-4ba48f7212b2@googlegroups.com>
 <mailman.229.1433656976.13271.python-list@python.org>
Message-ID: <49451412-92e4-4a22-9f2b-1d14ee35f836@googlegroups.com>

No, myclass is not null. I think my style of passing arguments is wrong.


From doc.mefisto at gmail.com  Sun Jun  7 04:56:47 2015
From: doc.mefisto at gmail.com (doc.mefisto at gmail.com)
Date: Sun, 7 Jun 2015 01:56:47 -0700 (PDT)
Subject: Embedded Python and C Callback functions
In-Reply-To: <mailman.229.1433656976.13271.python-list@python.org>
References: <daad7377-7a61-4e9e-a442-4ba48f7212b2@googlegroups.com>
 <mailman.229.1433656976.13271.python-list@python.org>
Message-ID: <620b003e-a6d1-4113-b6dc-ad3cf0eaeccd@googlegroups.com>

And I can't use Cython, because I have C++ module, and I have to use it.


From otlucaDELETE at DELETEyahoo.it  Sun Jun  7 05:06:10 2015
From: otlucaDELETE at DELETEyahoo.it (Luca Menegotto)
Date: Sun, 07 Jun 2015 11:06:10 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>	<mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl>
Message-ID: <ml11i1$31j$1@speranza.aioe.org>

Il 07/06/2015 10:22, Cecil Westerhof ha scritto:
 > That only times the function. I explicitly mentioned I want both the
 > needed time AND the output.
 >
 > Sadly the quality of the answers on this list is going down....

First of all, thank God it's a newsgroup, not a list.

Second, often the quality of an answer is deeply connected to the 
quality of a question. I've red your question, and I still have not 
clear what you want.

If you want a function that returns a result and a time elapsed, why 
don't you pack the time and the result in a tuple, a list or a 
dictionary? it takes 20 seconds, if you are slow.

Else if you want a function that prints time and result, why don't you 
print result more or less in the same manner you do with the time 
elapsed? it takes 20 seconds, if you are slow.

Else, can you explain exactly what do you want?


-- 
Ciao!
Luca



From doc.mefisto at gmail.com  Sun Jun  7 05:15:09 2015
From: doc.mefisto at gmail.com (doc.mefisto at gmail.com)
Date: Sun, 7 Jun 2015 02:15:09 -0700 (PDT)
Subject: Embedded Python and C Callback functions
In-Reply-To: <mailman.229.1433656976.13271.python-list@python.org>
References: <daad7377-7a61-4e9e-a442-4ba48f7212b2@googlegroups.com>
 <mailman.229.1433656976.13271.python-list@python.org>
Message-ID: <3726dccc-76cb-45a2-b0a8-e41ef300be96@googlegroups.com>

I'm trying to debug and find my error. It goes wrong when:
 PyObject *v;
            v = va_arg(*p_va, PyObject *);
            if (v != NULL) {
                if (*(*p_format - 1) != 'N')
                    Py_INCREF(v);
            }

it tries to PyINCREF to my passed callback.


From steve at pearwood.info  Sun Jun  7 05:28:46 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 19:28:46 +1000
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
Message-ID: <55740ecf$0$12978$c3e8da3$5496439d@news.astraweb.com>

On Sun, 7 Jun 2015 04:39 pm, Cecil Westerhof wrote:

> Sometimes I just want to know how much time a function takes, but at
> the same time I also want the result of the function. For this I wrote
> the following function:
>     def time_test(function, *args):
>         startTime   = time.time()
>         results     = function(*args)
>         endTime     = time.time()
>         print('It took {0} seconds'.format(endTime - startTime))
>         return results


There is a lot of subtlety in timing functions on modern day computers. What
you measure there includes the time taken to lookup the names "function"
and "args" (although that ought to be very quick) and to expand out *args
(not quite so quick). 

More importantly, it will also be less accurate on Windows systems, and may
include time during which the operating system is running other background
tasks. It makes no attempt to allow for whether code is in the CPU cache or
not. And what if the garbage collector happens to run in the middle of your
test?

Timing code these days is subtle and complicated!

Depending on what function actually does, those complications may, or may
not, make a real difference. If function() does a lot of work (say, at
least one second) then what I have said is probably completely irrelevant
and you can ignore it. For those cases, your function is perfectly fine,
although I prefer to use a with statement interactively. Of course I can
still call a function, but I don't *have* to call a function.

Here is a simple example:

http://code.activestate.com/recipes/577896-benchmark-code-with-the-with-statement/



But if your function takes less than, say, 1 millisecond, then your timing
results are probably just meaningless random numbers, affected more by the
other ten thousand processes running on your computer than by the Python
code itself.

In that case, you should learn how to use the timeit module. It's a little
complex, but worth it for timing small code snippets.


-- 
Steven



From Cecil at decebal.nl  Sun Jun  7 05:31:58 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 11:31:58 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
Message-ID: <87bngrq2y9.fsf@Equus.decebal.nl>

On Sunday  7 Jun 2015 08:39 CEST, Cecil Westerhof wrote:

> Sometimes I just want to know how much time a function takes, but at
> the same time I also want the result of the function. For this I
> wrote the following function: def time_test(function, *args):
> startTime = time.time() results = function(*args) endTime =
> time.time() print('It took {0} seconds'.format(endTime - startTime))
> return results
>
> I can do:
> time_test(test_random, 100, 10 ** 5)
> This outputs:
> It took 17.01685857772827 seconds
> and returns:
> (98592, 100833, 0.977775133140936)
>
> When executing:
> time_test(test_random, 100, 10 ** 6)
> it outputs:
> It took 165.26371836662292 seconds
> and returns:
> (997103, 1002009, 0.9951038363926871)

I improved a little on the function:
    def time_test(function, arguments, print_time = True):
        start_time  = time.time()
        results     = function(*arguments)
        end_time    = time.time()
        used_time   = end_time - start_time
        if print_time:
            print('It took {0} seconds'.format(used_time))
        else:
            results = (used_time, results)
        return results

Default the function still prints the time, but it also possible to
return the time with the results.

Also the arguments are given as a tuple now.

With the following function:
    def test_random(length, multiplier = 10000):
        number_list = length * [0]
        for i in range(length * multiplier):
            number_list[random.randint(0, length - 1)] += 1
        minimum = min(number_list)
        maximum = max(number_list)
        return (minimum, maximum, minimum / maximum)

The call:
    time_test(test_random, (100, 10 ** 5))
outputs:
    It took 15.95521855354309 seconds
and gives:
    (98870, 100810, 0.9807558773931158)

And the call:
    time_test(test_random, (100, 10 ** 5), False)
does not output anything and gives
    (15.926506280899048, (99146, 100779, 0.9837962273886425))

Of-course a new run will give a different result.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From lac at openend.se  Sun Jun  7 05:33:43 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 07 Jun 2015 11:33:43 +0200
Subject: Find in ipython3
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Sun, 07 Jun 2015 08:20:46 +0200." <87sia4ox8h.fsf@Equus.decebal.nl>
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.177.1433448836.13271.python-list@python.org>
 <87bnguhbec.fsf@Equus.decebal.nl> <874mmlqhul.fsf@Equus.decebal.nl>
 <mailman.215.1433588892.13271.python-list@python.org><87sia4ox8h.fsf@Equus.decebal.nl>
Message-ID: <201506070933.t579XhUE031296@fido.openend.se>

In a message of Sun, 07 Jun 2015 08:20:46 +0200, Cecil Westerhof writes:
>> You may get faster results if you use Matthew Barnett's replacement
>> for re here: https://pypi.python.org/pypi/regex
>>
>> You will get faster results if you build your IPython shell to use
>> PyPy, but I would still be very surprised if it beat the C program
>> find.
>
>I have to look into that. But I prefer to write a version that can be
>used by ?everyone?.

Well, everybody can download Matthew Barnett's regex, and get pypy,
and the claim is that ipython just works with pypy, and if it ever doesn't
the ipython team wants a bug report, so I am not sure what you mean by
'used by "everyone"' here that these don't have.

>There is no gain to get in standard Python? By switching from fnmatch
>to re I got almost a speed gain of two. So I was wondering if I could
>do more.

That's because speeding up regular expression matching is something
that people have put a significant amount of effort in, but under the
hood, so to speak.  Other modules are likely to be slower, as people
haven't gone to so much effort to make them fast.

You have reached the point where, if your python code is too slow, and
you don't want to use PyPy, people go grab Cython (or Boost or Swig, but
Cython is a whole lot easier and more fun to use) and make themselves
a C extension.  Which you can, of course, do as part of your voyage
of discovery.

But I am still betting that it won't perform as well as the linux utility
find. :)

Laura



From otlucaDELETE at DELETEyahoo.it  Sun Jun  7 05:44:27 2015
From: otlucaDELETE at DELETEyahoo.it (Luca Menegotto)
Date: Sun, 07 Jun 2015 11:44:27 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <55740ecf$0$12978$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <ml13pq$7su$1@speranza.aioe.org>

Il 07/06/2015 11:28, Steven D'Aprano ha scritto:

>
> But if your function takes less than, say, 1 millisecond, then your timing
> results are probably just meaningless random numbers, affected more by the
> other ten thousand processes running on your computer than by the Python
> code itself.

That's a good point. IMHO test the time execution of a function makes 
sense only if all these assetions are true:

- the function takes a long time to be executed;
- the execution is iterated a certain (large) number of times.

-- 
Ciao!
Luca



From lac at openend.se  Sun Jun  7 05:49:46 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 07 Jun 2015 11:49:46 +0200
Subject: Embedded Python and C Callback functions
In-Reply-To: Message from doc.mefisto@gmail.com of "Sun,
 07 Jun 2015 01:56:47 -0700."
 <620b003e-a6d1-4113-b6dc-ad3cf0eaeccd@googlegroups.com>
References: <daad7377-7a61-4e9e-a442-4ba48f7212b2@googlegroups.com>
 <mailman.229.1433656976.13271.python-list@python.org><620b003e-a6d1-4113-b6dc-ad3cf0eaeccd@googlegroups.com>
Message-ID: <201506070949.t579nkBP031867@fido.openend.se>

In a message of Sun, 07 Jun 2015 01:56:47 -0700, doc.mefisto at gmail.com writes:
>And I can't use Cython, because I have C++ module, and I have to use it.
>-- 
>https://mail.python.org/mailman/listinfo/python-list

Are you using Boost?
http://www.boost.org/doc/libs/1_58_0/libs/python/doc/

It handles lots of the setup for you.

Laura


From doc.mefisto at gmail.com  Sun Jun  7 05:51:02 2015
From: doc.mefisto at gmail.com (doc.mefisto at gmail.com)
Date: Sun, 7 Jun 2015 02:51:02 -0700 (PDT)
Subject: Embedded Python and C Callback functions
In-Reply-To: <mailman.234.1433670594.13271.python-list@python.org>
References: <daad7377-7a61-4e9e-a442-4ba48f7212b2@googlegroups.com>
 <mailman.229.1433656976.13271.python-list@python.org><620b003e-a6d1-4113-b6dc-ad3cf0eaeccd@googlegroups.com>
 <mailman.234.1433670594.13271.python-list@python.org>
Message-ID: <1c3a5e09-8b1a-4546-bf3b-e6130429dbe3@googlegroups.com>

> >And I can't use Cython, because I have C++ module, and I have to use it.
> >-- 
> >https://mail.python.org/mailman/listinfo/python-list
> 
> Are you using Boost?
> http://www.boost.org/doc/libs/1_58_0/libs/python/doc/
> 
> It handles lots of the setup for you.
> 
> Laura

No, I'm not using Boost. Pure Python.h


From steve at pearwood.info  Sun Jun  7 05:51:15 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 19:51:15 +1000
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
Message-ID: <55741414$0$12980$c3e8da3$5496439d@news.astraweb.com>

On Sun, 7 Jun 2015 04:39 pm, Cecil Westerhof wrote:

> Sometimes I just want to know how much time a function takes, but at
> the same time I also want the result of the function. For this I wrote
> the following function:
>     def time_test(function, *args):
>         startTime   = time.time()
>         results     = function(*args)
>         endTime     = time.time()
>         print('It took {0} seconds'.format(endTime - startTime))
>         return results

Oh, I forgot to mention... 

This has the disadvantage that it only prints the time, you cannot collect
it for further processing. (Well, not easily -- you could always shadow the
print built-in, then parse the string... ugggh, that's horrible.)

Using the Timer with statement I linked to earlier, you can easily collect
both the time and the calculated result:

with Timer(verbose=False) as t:
    do_this()
    do_that()
    result = something_else()

time_taken = t.interval
print(result, "took", time_taken)

Even the verbose=False flag is optional, if you leave it out the timing
result will be printed as usual, and you can still capture the time using
t.interval afterwards.

If you are interested, I have a more advanced version of that Timer which I
use almost every day. It works in Python 2.4 through 3.3, I expect it to
work in 3.4 and 3.5 as well, and is extensively tested in CPython and seems
to work in Jython and IronPython as well.


Oh, and to give you an idea of just how large the individual timing
fluctuations of small code snippets can be, here is an example running
under IronPython 2.6:

>>> from timer import Stopwatch
>>> with Stopwatch():
...     x = 12
...     y = x/3
...
time taken: 0.019135 seconds
>>> with Stopwatch():
...     x = 12
...     y = x/3
...
elapsed time is very small; consider using timeit.Timer for micro-timings of
small code snippets
time taken: 0.000114 seconds


The first run was over 160 times slower than the second run.



-- 
Steven



From breamoreboy at yahoo.co.uk  Sun Jun  7 06:16:30 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 07 Jun 2015 11:16:30 +0100
Subject: Function to show time to execute another function
In-Reply-To: <87fv64orly.fsf@Equus.decebal.nl>
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl>
Message-ID: <ml15m4$f44$1@ger.gmane.org>

On 07/06/2015 09:22, Cecil Westerhof wrote:
> On Sunday  7 Jun 2015 09:39 CEST, Mark Lawrence wrote:
>
>> On 07/06/2015 07:39, Cecil Westerhof wrote:
>>> Sometimes I just want to know how much time a function takes, but
>>> at the same time I also want the result of the function. For this I
>>> wrote the following function: def time_test(function, *args):
>>> startTime = time.time() results = function(*args) endTime =
>>> time.time() print('It took {0} seconds'.format(endTime -
>>> startTime)) return results
>>>
>>> I can do:
>>> time_test(test_random, 100, 10 ** 5)
>>> This outputs:
>>> It took 17.01685857772827 seconds
>>> and returns:
>>> (98592, 100833, 0.977775133140936)
>>>
>>> When executing:
>>> time_test(test_random, 100, 10 ** 6)
>>> it outputs:
>>> It took 165.26371836662292 seconds
>>> and returns:
>>> (997103, 1002009, 0.9951038363926871)
>>>
>>
>> https://docs.python.org/3/library/timeit.html
>> https://docs.python.org/3/library/profile.html
>
> That only times the function. I explicitly mentioned I want both the
> needed time AND the output.
>
> Sadly the quality of the answers on this list is going down. Here I
> get an alternative that does only half what I want and when writing an
> alternative for ?!find? I am told I could use ?!find? (which only
> works in ipython, not python and which also not works with Windows).
>

I suggest that you stop asking so many question here.  Get your cheque 
book and go for paid support.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From __peter__ at web.de  Sun Jun  7 06:27:05 2015
From: __peter__ at web.de (Peter Otten)
Date: Sun, 07 Jun 2015 12:27:05 +0200
Subject: Find in ipython3
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.177.1433448836.13271.python-list@python.org>
 <87bnguhbec.fsf@Equus.decebal.nl> <874mmlqhul.fsf@Equus.decebal.nl>
 <mailman.215.1433588892.13271.python-list@python.org>
 <87sia4ox8h.fsf@Equus.decebal.nl>
Message-ID: <ml16a9$qg8$1@ger.gmane.org>

Cecil Westerhof wrote:

> On Saturday  6 Jun 2015 13:07 CEST, Laura Creighton wrote:
> 
>> The !find version is C code optimised to do one thing, find files in
>> your directory structure, which happens to be what you want to do.
>> General regular expression matching is harder.
>>
>> Carl Friedrich Bolz investigated regular expression algorithms and
>> their implementation to see if this is the sort of task that a JIT
>> can improve. He blogged about it in 2 posts (part1 and part2). There
>> are benchmarks for part2. Benchmarks in part2.
>>
>> see:
>> http://morepypy.blogspot.se/2010/05/efficient-and-elegant-regular.html
>> http://morepypy.blogspot.se/2010/06/jit-for-regular-expression-matching.html
>>
>> You may get faster results if you use Matthew Barnett's replacement
>> for re here: https://pypi.python.org/pypi/regex
>>
>> You will get faster results if you build your IPython shell to use
>> PyPy, but I would still be very surprised if it beat the C program
>> find.
> 
> I have to look into that. But I prefer to write a version that can be
> used by ?everyone?.
> 
> It is of-course not a very big program. The difference is significant,
> but I do not use find that much. And if it is significant I still can
> use the shell version.
> 
> There is no gain to get in standard Python? By switching from fnmatch
> to re I got almost a speed gain of two. So I was wondering if I could
> do more.

Just wait for Python 3.5. The switch from os.listdir() to the (new) 
os.scandir() in the implementation of os.walk() is likely to improve the 
situation: 

$ cat findfiles.py                    
import fnmatch
import os
import re
import subprocess
import time


def find_re(root, pattern, ignore_case=False):
    match = re.compile(
        fnmatch.translate(pattern),
        re.IGNORECASE if ignore_case else 0).match

    results = []
    for path, _folders, files in os.walk(root):
        for filename in files:
            if match(filename):
                results.append(os.path.join(path, filename))
    return results


def find_sp(
        root, pattern, ignore_case=False,
        encoding="utf-8", errors="surrogateescape"):

    name_opt = "-iname" if ignore_case else "-name"
    matches = subprocess.Popen(
        ["find", root, name_opt, pattern, "-print0"], stdout=subprocess.PIPE
    ).communicate()[0].decode(encoding, errors=errors).split("\0")
    assert len(matches[-1]) == 0
    del matches[-1]
    return matches


def measure(f, *args):
    start = time.time()
    try:
        return f(*args)
    finally:
        end = time.time()
        print("{}{}".format(f.__name__, args), end - start)


def main():
    import argparse
    parser = argparse.ArgumentParser()
    parser.add_argument("root")
    parser.add_argument("pattern")
    parser.add_argument("-i", "--ignore-case", action="store_true")
    args = parser.parse_args()

    a = measure(find_re, args.root, args.pattern, args.ignore_case)
    b = measure(find_sp, args.root, args.pattern, args.ignore_case)
    measure(find_re, args.root, args.pattern, args.ignore_case)
    assert sorted(a) == sorted(b)
    print(len(a), "matches")


if __name__ == "__main__":
    main()

$ python3.4 findfiles.py . '*a*.PY' -i
find_re('.', '*a*.PY', True) 0.14614605903625488
find_sp('.', '*a*.PY', True) 0.043445587158203125
find_re('.', '*a*.PY', True) 0.16485309600830078
1454 matches

$ python3.5 findfiles.py . '*a*.PY' -i
find_re('.', '*a*.PY', True) 0.07263660430908203
find_sp('.', '*a*.PY', True) 0.04418468475341797
find_re('.', '*a*.PY', True) 0.07320952415466309
1454 matches




From PointedEars at web.de  Sun Jun  7 06:40:41 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 12:40:41 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
Message-ID: <1451048.pW9z17ilMA@PointedEars.de>

Cecil Westerhof wrote:

> I wrote a very simple function to test random:
>     def test_random(length, multiplier = 10000):
>         number_list = length * [0]
>         for i in range(length * multiplier):
>             number_list[random.randint(0, length - 1)] += 1
>         minimum = min(number_list)
>         maximum = max(number_list)
>         return (minimum, maximum, minimum / maximum)

As there is no guarantee that every number will occur randomly, using a 
dictionary at first should be more efficient than a list:

def test_random (length, multiplier=10000):
    number_list = {}
    for i in range(length * multiplier):
        r = random.randint(0, length - 1)
        number_list[r] = number_list[r] + 1 if r in number_list else 1
    values = number_list.values()
    minimum = min(values)
    maximum = max(values)
    return (minimum, maximum, minimum / maximum)

[?uple literals are going to be introduced with C# 7 ;-)]


Since Python uses indentation as syntax element, it is better not to indent 
Python code by default when posting in order to make it easier testable.  If 
necessary, code sections can be delimited in an unambiguous way that is 
syntactically correct by lines like

#------------------------------------------------------------------------

The line length then should not exceed 72 to 78 characters, for quoting.

Although the <HT> character can be used for indentation, 4 spaces appear to 
be a sensible, device-independent default (8 is too much for more complex 
snippets that need to fit within the conventional 80-characters-per-line 
limit).

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From jonas at wielicki.name  Sun Jun  7 06:41:53 2015
From: jonas at wielicki.name (Jonas Wielicki)
Date: Sun, 07 Jun 2015 12:41:53 +0200
Subject: Testing random
In-Reply-To: <87oaksowwg.fsf@Equus.decebal.nl>
References: <87oaksowwg.fsf@Equus.decebal.nl>
Message-ID: <55741FF1.3060807@wielicki.name>

On 07.06.2015 08:27, Cecil Westerhof wrote:
> I wrote a very simple function to test random:
>     def test_random(length, multiplier = 10000):
>         number_list = length * [0]
>         for i in range(length * multiplier):
>             number_list[random.randint(0, length - 1)] += 1
>         minimum = min(number_list)
>         maximum = max(number_list)
>         return (minimum, maximum, minimum / maximum)
> 
> When running:
>     for i in range(1, 7):
>         print(test_random(100, 10 ** i))
> I get:
>     (3, 17, 0.17647058823529413)
>     (73, 127, 0.5748031496062992)
>     (915, 1086, 0.8425414364640884)
>     (9723, 10195, 0.9537027954879843)
>     (99348, 100620, 0.9873583780560524)
>     (997198, 1002496, 0.9947151908835546)
> 
> and when running:
>     for i in range(1, 7):
>         print(test_random(1000, 10 ** i))
> I get:
>     (2, 20, 0.1)
>     (69, 138, 0.5)
>     (908, 1098, 0.8269581056466302)
>     (9684, 10268, 0.9431242695753799)
>     (99046, 100979, 0.9808574059953059)
>     (996923, 1003598, 0.9933489305478886)
> 
> It shows that when the first parameter increases the deviation
> increases and when the second parameter increases the deviation
> decreases. Exactly what you would expect. But what are the ranges you
> would expect with a good random function. 

Really depends on the number of samples. Appearantly, a good RNG would
come close to 1.0 for the ratio.

> Then it could be used to test a random function.

This is an interesting test (more interesting to me than it looks at the
first sight, and certainly better than what I had come up with), but
unfortunately, there is more to testing randomness.

The test clearly suggests that random functions should have a min/max
ratio of about 1.0. Think of a "random" function like this:

    def fake_random(minv, maxv, _cache={}):
        try:
            return next(_cache[minv, maxv])
        except (StopIteration, KeyError):
            iterator = iter(range(minv, maxv+1))
            _cache[minv, maxv] = iterator
            return next(iterator)

(if that is hard to read, I agree; it returns the sequence from minv to
maxv (inclusively) over and over again for equal minv and maxv. don?t
pass anything to _cache :), just call it like random.randint)

This gives a "perfect" ratio of 1.0, but is not very random. This kind
of function would probably be ruled out by a compression or correlation
test. If you want to dig deeper into the algorithms for random testing,
the dieharder test suite [1] is probably worth a look.

It all boils down to the use case. For some use cases, the
``fake_random`` might be good enough (unittesting would be such a case:
it is certainly uniformly distributed and allows full coverage of the
tested range), for others it would fail catastrophically (don?t generate
your cryptographic keys with this!).

cheers,
Jonas

   [1]: http://www.phy.duke.edu/~rgb/General/dieharder.php


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20150607/9fa6160b/attachment.sig>

From marko at pacujo.net  Sun Jun  7 07:02:00 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sun, 07 Jun 2015 14:02:00 +0300
Subject: [OT] Re: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl>
 <mailman.235.1433672222.13271.python-list@python.org>
Message-ID: <87pp57ye6v.fsf_-_@elektro.pacujo.net>

Mark Lawrence <breamoreboy at yahoo.co.uk>:

> Get your cheque book and go for paid support.

Are checks still in use in Britain? I thought only Americans still did
that.


Marko


From mail at timgolden.me.uk  Sun Jun  7 07:05:06 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Sun, 07 Jun 2015 12:05:06 +0100
Subject: Function to show time to execute another function
In-Reply-To: <ml15m4$f44$1@ger.gmane.org>
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl> <ml15m4$f44$1@ger.gmane.org>
Message-ID: <55742562.7080402@timgolden.me.uk>

On 07/06/2015 11:16, Mark Lawrence wrote:
> On 07/06/2015 09:22, Cecil Westerhof wrote:
>> That only times the function. I explicitly mentioned I want both the
>> needed time AND the output.
>>
>> Sadly the quality of the answers on this list is going down. Here I
>> get an alternative that does only half what I want and when writing an
>> alternative for ?!find? I am told I could use ?!find? (which only
>> works in ipython, not python and which also not works with Windows).
>>
>
> I suggest that you stop asking so many question here.  Get your cheque
> book and go for paid support.
>

Mark: that was too abrupt, even if the OP was being serious (and I 
suspect he may not have been, despite the absence of a smiley).

Cecil: I don't know if you really meant that seriously, but it certainly 
comes across as ungracious if you did. If you meant it half-humorously, 
then you need to throw in a smiley or a wink to show that you were joking.

TJG


From breamoreboy at yahoo.co.uk  Sun Jun  7 07:12:17 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 07 Jun 2015 12:12:17 +0100
Subject: [OT] Re: Function to show time to execute another function
In-Reply-To: <87pp57ye6v.fsf_-_@elektro.pacujo.net>
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl>
 <mailman.235.1433672222.13271.python-list@python.org>
 <87pp57ye6v.fsf_-_@elektro.pacujo.net>
Message-ID: <ml18uo$vfv$1@ger.gmane.org>

On 07/06/2015 12:02, Marko Rauhamaa wrote:
> Mark Lawrence <breamoreboy at yahoo.co.uk>:
>
>> Get your cheque book and go for paid support.
>
> Are checks still in use in Britain? I thought only Americans still did
> that.
>
> Marko
>

Cheques are still in use in Britain.  There was a move a year or so ago 
to get rid of them but too many people complained.  However I feel that 
they're doomed in the same way that cash is on the way out as reported 
here http://www.bbc.co.uk/news/business-32778196

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From random832 at fastmail.us  Sun Jun  7 07:20:52 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Sun, 07 Jun 2015 07:20:52 -0400
Subject: Is it a newsgroup or a list?
In-Reply-To: <ml11i1$31j$1@speranza.aioe.org>
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl> <ml11i1$31j$1@speranza.aioe.org>
Message-ID: <1433676052.3089941.288915321.4982B8F8@webmail.messagingengine.com>

On Sun, Jun 7, 2015, at 05:06, Luca Menegotto wrote:
> Il 07/06/2015 10:22, Cecil Westerhof ha scritto:
>  > That only times the function. I explicitly mentioned I want both the
>  > needed time AND the output.
>  >
>  > Sadly the quality of the answers on this list is going down....
> 
> First of all, thank God it's a newsgroup, not a list.
...
> -- 
> https://mail.python.org/mailman/listinfo/python-list

I've always thought of this as a list that is mirrored to a newsgroup,
not a newsgroup that is mirrored to a list. What is the true nature of
python-list (and others such as python-ideas)? What is it primarily,
what was it first, etc?


From steve at pearwood.info  Sun Jun  7 07:29:57 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 21:29:57 +1000
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl>
 <mailman.235.1433672222.13271.python-list@python.org>
Message-ID: <55742b36$0$12975$c3e8da3$5496439d@news.astraweb.com>

On Sun, 7 Jun 2015 08:16 pm, Mark Lawrence wrote:


> I suggest that you stop asking so many question here.  Get your cheque
> book and go for paid support.

Mark, that remark is uncalled for and completely out of line. Cecil is
perfectly entitled to ask questions here, and you are entitled to ignore
them. You're just pissed off because he rightly noted that your answer to
his question *doesn't solve his problem*.



-- 
Steven



From steve at pearwood.info  Sun Jun  7 07:42:05 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 21:42:05 +1000
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
 <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <2c4d029c-8ea5-465b-8adc-6c35185bd150@googlegroups.com>
 <2483375.eHyISxeWLQ@PointedEars.de>
Message-ID: <55742e0e$0$12980$c3e8da3$5496439d@news.astraweb.com>

On Sun, 7 Jun 2015 06:21 pm, Thomas 'PointedEars' Lahn wrote:

> Ned Batchelder wrote:
> 
>> On Saturday, May 23, 2015 at 9:01:29 AM UTC-4, Steven D'Aprano wrote:
>>> On Sat, 23 May 2015 10:33 pm, Thomas 'PointedEars' Lahn wrote:
>>> > If only characters were represented as sequences UTF-16 code units in
>>> > ECMAScript implementations like JavaScript, there would not be a
>>> > problem beyond the BMP;
>>> 
>>> Are you being sarcastic?
>> 
>> IIUC, Thomas' point is that *characters* should be sequences of
>> codepoints, not that *strings* should be.
> 
> No, my point is that one character should be a sequence of code _units_
> (for a code point value).  

I don't understand this sentence. "Code point value" doesn't appear to be
meaningful. "Code point" is a value in the Unicode codespace, informally "a
character" (but see below); code points can take on values in the range 0
to 1114111, usually written in hex as U+0000 to U+10FFFF.

"Code value" is an obsolete term for code unit, that is, the smallest chunk
of memory used to represent a code point. For example, UTF-8 uses 8-bit
code units, UTF-32 uses 32 bit code units.

But "code point value", I'm not sure what you mean by that. Consequently I
have no idea what you think a character should be. Is "Hello World" a
character? How about "?" or "?"?

The term "character" is problematic, because what counts as a character
depends on where you are and how the string is normalised. For example:

"ij" could be two characters, the letters i followed by j, or one, the 25th
letter of the Dutch language [and not even the Dutch agree on this];
conversely, "?" could be a single character, or a ligature of two
characters.

"?" (U+1E16 LATIN CAPITAL LETTER E WITH MACRON AND ACUTE) could be
considered one character, or three 'E\u0304\u0301', depending on whether it
is normalised or not.

So I'm afraid I do not understand your sentence.

Code point: http://www.unicode.org/glossary/#code_point

Code unit: http://www.unicode.org/glossary/#code_unit

Code value: http://www.unicode.org/glossary/#code_value


See also http://unicode.org/faq/char_combmark.html


> But in ECMAScript implementations (so far), a *code 
> point value* equals a character, and that is a problem in ECMAScript
> because
> there the value range is limited to what can be encoded in 16 bit.  The
> problem starts beyond the BMP where 16 bit are no longer sufficient for a
> code sequence and code point value, and code sequence and code point value
> are no longer equal.

This is no clearer.

I *think* what you are trying to say is that ECMAScript assumes that one
code point is always represented by a single code unit. So a sequence of
code points ABCD will be correctly interpreted as four "characters" so long
as each of those code points are in the BMP (i.e. between U+0000 and U+FFFF
inclusive), but *not* if they are from one of the supplementary planes.

This is the same problem that older Python "narrow builds" suffered from.
The solutions in Python was to use a wide-build (each code point is
represented by a single UTF-32 code unit, that is, four bytes) or to
upgrade to Python 3.3, which uses a compressed coding scheme where strings
are represented by either 1-byte per code point, 2-bytes per code point, or
4-bytes per code point, whichever is the minimum needed for that particular
string.

My opinion is that a programming language like Python or ECMAScript should
operate on *code points*. If we want to call them "characters" informally,
that should be allowed, but whenever there is ambiguity we should remember
we're dealing with code points. The implementation shouldn't matter:
compliant Python interpreters might choose to use UTF-8 internally, or
UTF-16, or UTF-32, or something else, and still agree on how many
characters a string contains. Normalisation is still an issue, of course,
but any decent Unicode implementation will include a way to normalise or
denormalise strings.

The question of graphemes (what "ordinary people" consider letters and
characters, e.g. "ch" is two letters to an English speaker but one letter
to a Czech speaker) should be left to libraries. It's a much harder problem
to solve in the full general case, requires localisation, and is overkill
for many string-processing tasks.



-- 
Steven



From steve at pearwood.info  Sun Jun  7 07:45:00 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 21:45:00 +1000
Subject: Is it a newsgroup or a list?
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl> <ml11i1$31j$1@speranza.aioe.org>
 <mailman.240.1433676054.13271.python-list@python.org>
Message-ID: <55742ebc$0$12980$c3e8da3$5496439d@news.astraweb.com>

On Sun, 7 Jun 2015 09:20 pm, random832 at fastmail.us wrote:

> On Sun, Jun 7, 2015, at 05:06, Luca Menegotto wrote:
>> Il 07/06/2015 10:22, Cecil Westerhof ha scritto:
>>  > That only times the function. I explicitly mentioned I want both the
>>  > needed time AND the output.
>>  >
>>  > Sadly the quality of the answers on this list is going down....
>> 
>> First of all, thank God it's a newsgroup, not a list.
> ...
>> --
>> https://mail.python.org/mailman/listinfo/python-list
> 
> I've always thought of this as a list that is mirrored to a newsgroup,
> not a newsgroup that is mirrored to a list. What is the true nature of
> python-list (and others such as python-ideas)? What is it primarily,
> what was it first, etc?

It's a mailing list mirrored to one official newsgroup, and at least one
unofficial newsgroup (gmane). It's also archived on various websites, plus
Google Groups.

As far as I know, python-list a.k.a. comp.lang.python is the only one of the
Python mailing lists with an official newsgroup mirror.

In *practice*, there's not much difference which comes first. Apart from
some spam, which gets filtered from the mailing list but not the newsgroup,
or vise versa, you can post to the mailing list or the newsgroup and the
message will show up on both.



-- 
Steven



From zachary.ware+pylist at gmail.com  Sun Jun  7 07:49:13 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Sun, 7 Jun 2015 06:49:13 -0500
Subject: Is it a newsgroup or a list?
In-Reply-To: <1433676052.3089941.288915321.4982B8F8@webmail.messagingengine.com>
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl> <ml11i1$31j$1@speranza.aioe.org>
 <1433676052.3089941.288915321.4982B8F8@webmail.messagingengine.com>
Message-ID: <CAKJDb-OmyfeG8-BQ8MNkVh=rYqfyyGCRyFe4sMH0DPcpHdfNsQ@mail.gmail.com>

On Sunday, June 7, 2015, <random832 at fastmail.us> wrote:

> On Sun, Jun 7, 2015, at 05:06, Luca Menegotto wrote:
> > Il 07/06/2015 10:22, Cecil Westerhof ha scritto:
> >  > That only times the function. I explicitly mentioned I want both the
> >  > needed time AND the output.
> >  >
> >  > Sadly the quality of the answers on this list is going down....
> >
> > First of all, thank God it's a newsgroup, not a list.
> ...
> > --
> > https://mail.python.org/mailman/listinfo/python-list
>
> I've always thought of this as a list that is mirrored to a newsgroup,
> not a newsgroup that is mirrored to a list. What is the true nature of
> python-list (and others such as python-ideas)? What is it primarily,
> what was it first, etc?
>

I believe the most accurate answer to the question in the subject is "yes".

--
Zach


-- 
Sent from Gmail Mobile
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150607/0659b6cd/attachment.html>

From rosuav at gmail.com  Sun Jun  7 07:51:47 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 7 Jun 2015 21:51:47 +1000
Subject: Testing random
In-Reply-To: <1451048.pW9z17ilMA@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
Message-ID: <CAPTjJmo5Oijun5-FChOUNFL3CYfy7Nq9cmgu-WP+B4LLYVb4xA@mail.gmail.com>

On Sun, Jun 7, 2015 at 8:40 PM, Thomas 'PointedEars' Lahn
<PointedEars at web.de> wrote:
> Cecil Westerhof wrote:
>
>> I wrote a very simple function to test random:
>>     def test_random(length, multiplier = 10000):
>>         number_list = length * [0]
>>         for i in range(length * multiplier):
>>             number_list[random.randint(0, length - 1)] += 1
>>         minimum = min(number_list)
>>         maximum = max(number_list)
>>         return (minimum, maximum, minimum / maximum)
>
> As there is no guarantee that every number will occur randomly, using a
> dictionary at first should be more efficient than a list:

Hmm, I'm not sure that's actually so. His code is aiming to get
'multiplier' values in each box; for any serious multiplier (he starts
with 10 in the main code), you can be fairly confident that every
number will come up at least once. The distribution of numbers won't
ever be perfectly even, but you'd expect it to be reasonably close. I
have a similar routine on my Dungeons & Dragons server; since a roll
of a twenty-sided dice is crucial to most of the game, I have a simple
tester that proves to people that the in-built dice roller is fair.
Its output looks like this:

> roll test
1: 10017
2: 10003
3: 9966
4: 9728
5: 10088
6: 9888
7: 10087
8: 9971
9: 10052
10: 10061
11: 10130
12: 9942
13: 10062
14: 10075
15: 10050
16: 9948
17: 9880
18: 10052
19: 9995
20: 10005
Standard deviation: 90.18 (0.90%)

This is about equivalent to test_random(20), and as you see, he and I
both picked a multiplier of 10K to use by default. (I call the
parameters "max" and "avg" rather than "length" and "multiplier", but
they have the exact same semantics.) The hard part is figuring out
what "looks reasonable"; a true RNG could legitimately produce nothing
but 7s for the entire run, it's just extremely unlikely.

ChrisA


From Cecil at decebal.nl  Sun Jun  7 07:53:18 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 13:53:18 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl> <ml11i1$31j$1@speranza.aioe.org>
Message-ID: <877frfpwep.fsf@Equus.decebal.nl>

On Sunday  7 Jun 2015 11:06 CEST, Luca Menegotto wrote:

> Il 07/06/2015 10:22, Cecil Westerhof ha scritto:
>> That only times the function. I explicitly mentioned I want both
>> the needed time AND the output.
>>
>> Sadly the quality of the answers on this list is going down....
>
> First of all, thank God it's a newsgroup, not a list.

My bad.


> Second, often the quality of an answer is deeply connected to the
> quality of a question. I've red your question, and I still have not
> clear what you want.

Well, it was not a question, it was sharing. I wanted something that
showed the needed time, but also the result. It was not a piece of
art, but something I thought could be useful to share.


> If you want a function that returns a result and a time elapsed, why
> don't you pack the time and the result in a tuple, a list or a
> dictionary? it takes 20 seconds, if you are slow.

I already did that. Sort of. I made a version that defaults to
printing the needed time and returning the result, but can me made to
put the time in a tuple with the result.

I must be very slow. :'-( It took more as 20 seconds.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun  7 08:03:11 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 14:03:11 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl>
 <mailman.235.1433672222.13271.python-list@python.org>
Message-ID: <873823pvy8.fsf@Equus.decebal.nl>

On Sunday  7 Jun 2015 12:16 CEST, Mark Lawrence wrote:

> I suggest that you stop asking so many question here. Get your
> cheque book and go for paid support.

First of all: it was not a question: I shared something I thought was
useful. You can disagree about it being useful, but there is in my
opinion no need for this kind of responses.

Giving a ?solution? that only does half of my solution is not very
useful. Also giving a solution I rejected/replaced is not useful. You
can disagree with the rejection/replacement, but giving it as an
alternative ?

Giving disparaging remarks will not improve the quality of this
newsgroup. You will get rid of my ?stupid? questions and shares. But I
found that some of the things that I thought would be of very slight
interest resulted in very interesting discussions. If I and other
people do not post those ?stupid? things anymore, we will lose those
gems.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From rosuav at gmail.com  Sun Jun  7 08:08:06 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 7 Jun 2015 22:08:06 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <55742e0e$0$12980$c3e8da3$5496439d@news.astraweb.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
 <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <2c4d029c-8ea5-465b-8adc-6c35185bd150@googlegroups.com>
 <2483375.eHyISxeWLQ@PointedEars.de>
 <55742e0e$0$12980$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmpt5TM0nKJdf4FWsGPvdRguxdQTTD9sAn87c91YGUkDyg@mail.gmail.com>

On Sun, Jun 7, 2015 at 9:42 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> My opinion is that a programming language like Python or ECMAScript should
> operate on *code points*. If we want to call them "characters" informally,
> that should be allowed, but whenever there is ambiguity we should remember
> we're dealing with code points. The implementation shouldn't matter:
> compliant Python interpreters might choose to use UTF-8 internally, or
> UTF-16, or UTF-32, or something else, and still agree on how many
> characters a string contains. Normalisation is still an issue, of course,
> but any decent Unicode implementation will include a way to normalise or
> denormalise strings.

If by "normalise" you mean the NF[K]C/NF[K]D composition and
decomposition, then yes, any decent Unicode library will provide that.
I'm not sure it's critical to string handling itself, though; and
Python defers the operation to the unicodedata module:

>>> s1 = "\N{LATIN SMALL LETTER A}\N{COMBINING ACUTE ACCENT}"
>>> s2 = "\N{LATIN SMALL LETTER A WITH ACUTE}"
>>> s1 == s2
False
>>> unicodedata.normalize("NFC", s1) == s2
True

It's a useful operation to be able to do, but I would never expect
that *string comparison* or other operations should automatically
normalize. (Unless you want to say that all strings are guaranteed to
be NFC/NFD normalized, such that s1 and s2 would actually be
identical, which I suppose is plausible. I'm not sure what the
advantage would be, though. And certainly you wouldn't want to
K-normalize strings automatically.)

> The question of graphemes (what "ordinary people" consider letters and
> characters, e.g. "ch" is two letters to an English speaker but one letter
> to a Czech speaker) should be left to libraries. It's a much harder problem
> to solve in the full general case, requires localisation, and is overkill
> for many string-processing tasks.

Yeah. The basic challenge to a beginning programmer, "reverse this
string", becomes rather tricky in the presence of natural language.

>>> s1 += "e"
>>> s1
'a?e'
>>> s1[::-1]
'e?a'

Oops.

But hey. It's easier to understand what went wrong here than, say, if
you reverse the bytes in a UTF-8 stream. Or the code units in a UTF-16
stream. If you're lucky, those would give you instant errors... if
you're not, well, who knows.

ChrisA


From rosuav at gmail.com  Sun Jun  7 08:13:54 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 7 Jun 2015 22:13:54 +1000
Subject: Find in ipython3
In-Reply-To: <ml16a9$qg8$1@ger.gmane.org>
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.177.1433448836.13271.python-list@python.org>
 <87bnguhbec.fsf@Equus.decebal.nl> <874mmlqhul.fsf@Equus.decebal.nl>
 <mailman.215.1433588892.13271.python-list@python.org>
 <87sia4ox8h.fsf@Equus.decebal.nl> <ml16a9$qg8$1@ger.gmane.org>
Message-ID: <CAPTjJmooEz7oto1S=2KZRt5pS0E0Q-t+xycbvus1D4RK0NXJoA@mail.gmail.com>

On Sun, Jun 7, 2015 at 8:27 PM, Peter Otten <__peter__ at web.de> wrote:
> Just wait for Python 3.5. The switch from os.listdir() to the (new)
> os.scandir() in the implementation of os.walk() is likely to improve the
> situation

Why wait? I've been using 3.5 for ages (and actually, my
/usr/local/bin/python3 now announces itself as 3.6), and the betas
have been available with all the regular installers. Python betas are
pretty stable, and apart from a few glitches with the installers on
Windows, I haven't heard any showstopper bugs. Aside from not
depending on them for your nuclear power plant safety systems, there's
not a lot that the betas can't be used for.

Mind you, a lot of the benefit of os.scandir() comes from its
behaviour across network mounts and such, which is why you're seeing
no more than about a 2:1 difference here. From what I gather, the
improvement across network can be simply amazing (because of the way
os.listdir has to stat everything separately).

ChrisA


From Cecil at decebal.nl  Sun Jun  7 08:14:29 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 14:14:29 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <55740ecf$0$12978$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87y4jvoguy.fsf@Equus.decebal.nl>

On Sunday  7 Jun 2015 11:28 CEST, Steven D'Aprano wrote:

> On Sun, 7 Jun 2015 04:39 pm, Cecil Westerhof wrote:
>
>> Sometimes I just want to know how much time a function takes, but
>> at the same time I also want the result of the function. For this I
>> wrote the following function: def time_test(function, *args):
>> startTime = time.time() results = function(*args) endTime =
>> time.time() print('It took {0} seconds'.format(endTime -
>> startTime)) return results
>
>
> There is a lot of subtlety in timing functions on modern day
> computers. What you measure there includes the time taken to lookup
> the names "function" and "args" (although that ought to be very
> quick) and to expand out *args (not quite so quick).
>
> More importantly, it will also be less accurate on Windows systems,
> and may include time during which the operating system is running
> other background tasks. It makes no attempt to allow for whether
> code is in the CPU cache or not. And what if the garbage collector
> happens to run in the middle of your test?
>
> Timing code these days is subtle and complicated!
>
> Depending on what function actually does, those complications may,
> or may not, make a real difference. If function() does a lot of work
> (say, at least one second) then what I have said is probably
> completely irrelevant and you can ignore it. For those cases, your
> function is perfectly fine, although I prefer to use a with
> statement interactively. Of course I can still call a function, but
> I don't *have* to call a function.

Next time I should give more background. ;-)

This function is only mend to get an indication of the needed time and
in my opinion is only useful from ate least 5 seconds. If you want
real measurements you should use timeit I think.


> Here is a simple example:
>
> http://code.activestate.com/recipes/577896-benchmark-code-with-the-with-statement/

That looks quite interesting. I think I will use that in my function.
I still think the function is handy, because you need less code with
it.


> But if your function takes less than, say, 1 millisecond, then your
> timing results are probably just meaningless random numbers,
> affected more by the other ten thousand processes running on your
> computer than by the Python code itself.
>
> In that case, you should learn how to use the timeit module. It's a
> little complex, but worth it for timing small code snippets.

I knew that. It is just to use before using the heavy guns (and also
wanting the result of the function). But I should be more explicit
next time.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun  7 08:27:02 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 14:27:02 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <55741414$0$12980$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87twujoga1.fsf@Equus.decebal.nl>

On Sunday  7 Jun 2015 11:51 CEST, Steven D'Aprano wrote:

> On Sun, 7 Jun 2015 04:39 pm, Cecil Westerhof wrote:
>
>> Sometimes I just want to know how much time a function takes, but
>> at the same time I also want the result of the function. For this I
>> wrote the following function: def time_test(function, *args):
>> startTime = time.time() results = function(*args) endTime =
>> time.time() print('It took {0} seconds'.format(endTime -
>> startTime)) return results
>
> Oh, I forgot to mention... 
>
> This has the disadvantage that it only prints the time, you cannot
> collect it for further processing. (Well, not easily -- you could
> always shadow the print built-in, then parse the string... ugggh,
> that's horrible.)

I already found this an omission and posted a better version.


> Using the Timer with statement I linked to earlier, you can easily
> collect both the time and the calculated result:
>
> with Timer(verbose=False) as t:
> do_this()
> do_that()
> result = something_else()
>
> time_taken = t.interval
> print(result, "took", time_taken)

Personally I find:
    time_test(test_random, (100, 10 ** 5))
better as:
    with Timer() as t:
        test_random(100, 10 ** 5)
also because I will do it in a loop with different values for the
second (and maybe the first) parameter. But that is probably personal
preference.


> If you are interested, I have a more advanced version of that Timer
> which I use almost every day. It works in Python 2.4 through 3.3, I
> expect it to work in 3.4 and 3.5 as well, and is extensively tested
> in CPython and seems to work in Jython and IronPython as well.

I would certainly be interested. I work with 3.4, so I could test it
for you. And maybe even build a 3.5 version and test it there also.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun  7 08:43:49 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 14:43:49 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl> <ml15m4$f44$1@ger.gmane.org>
 <mailman.238.1433675117.13271.python-list@python.org>
Message-ID: <87pp57ofi2.fsf@Equus.decebal.nl>

On Sunday  7 Jun 2015 13:05 CEST, Tim Golden wrote:

> On 07/06/2015 11:16, Mark Lawrence wrote:
>> On 07/06/2015 09:22, Cecil Westerhof wrote:
>>> That only times the function. I explicitly mentioned I want both
>>> the needed time AND the output.
>>>
>>> Sadly the quality of the answers on this list is going down. Here
>>> I get an alternative that does only half what I want and when
>>> writing an alternative for ?!find? I am told I could use ?!find?
>>> (which only works in ipython, not python and which also not works
>>> with Windows).
>>>
>>
>> I suggest that you stop asking so many question here. Get your
>> cheque book and go for paid support.
>>
>
> Mark: that was too abrupt, even if the OP was being serious (and I
> suspect he may not have been, despite the absence of a smiley).
>
> Cecil: I don't know if you really meant that seriously, but it
> certainly comes across as ungracious if you did. If you meant it
> half-humorously, then you need to throw in a smiley or a wink to
> show that you were joking.

Yes, I was serious, but maybe I should have used some ?honey?. When I
started posting questions here I was pleasantly surprised with the
very helpful responses, but lately I must say I see ?answers? that are
not only unhelpful, but potentially chasing people away. As I
explained in another post.

Maybe I am not very smart, but what is ungracious in pointing out that
in the reply on something I shared I think could be useful, the
?solution? does something else as what I shared?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From steve at pearwood.info  Sun Jun  7 08:52:51 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 22:52:51 +1000
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
Message-ID: <55743ea4$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Sun, 7 Jun 2015 04:27 pm, Cecil Westerhof wrote:

> I wrote a very simple function to test random:
>     def test_random(length, multiplier = 10000):
>         number_list = length * [0]
>         for i in range(length * multiplier):
>             number_list[random.randint(0, length - 1)] += 1
>         minimum = min(number_list)
>         maximum = max(number_list)
>         return (minimum, maximum, minimum / maximum)

Putting aside the timing aspects, your frequency calculations are not done
in a very Pythonic manner. A better way might be:


from collections import Counter
from random import randint

def test_random(length, multiplier = 10000):
    freq = Counter(
        randint(0, length - 1) for i in range(length * multiplier)
        )
    minimum = min(freq[i] for i in range(length))
    maximum = max(freq.values())
    return (minimum, maximum, minimum / maximum)


Note carefully the way I calculate the minimum value. That way, if by chance
some number has a frequency of zero, the minimum returned will be zero. For
the maximum, that doesn't matter, since any missing numbers will not be the
most frequent number.

Better than returning the min and max, you should return the counter object
itself. Before doing so, I make sure that any missing numbers are given a
value of zero.


def test_random(size, samples=1000000):
    """Return a sample of `samples` random numbers from 0 to size-1."""
    freq = Counter(randint(0, size-1) for i in range(samples))
    # Avoid missing numbers. (Likely for small samples.)
    freq.subtract(dict.fromkeys(range(size), 0))
    return freq


Now once you have the frequencies, you can look at many interesting
statistics, and verify that you have a valid sample.

py> f = test_random(100)
py> sum(f.values()) == 1000000  # Total number of samples is right?
True
py> sorted(f) == list(range(100))  # Any missing numbers?
True
py> max(f) - min(f)  # statistical range
99
py> min(f.values())  # Lowest frequency?
9775
py> max(f.values())  # And the highest?
10313

The ratio of those two frequencies isn't very interesting, but we can
calculate it:

py> min(f.values())/max(f.values())
0.9478328323475226

For a uniformly distributed population, that ratio will be 1.0 exactly, but
for any finite sample, it could be any number between 0 and 1. We expect
that if we have a large random sample, the ratio should approach 1, but
moderate deviations from that are expected. In fact, we should be surprised
if the ratio is exactly 1, and suspect shenanigans if it is.

We can extract the mode, and its frequency:

py> f.most_common(1)  # mode
[(85, 10313)]

In Python 3.4 or better, we can look at some more statistics:

py> import statistics
py> statistics.median(f.elements())
50.0
py> statistics.mean(f.elements())
49.531949
py> statistics.stdev(f.elements())
28.873178122988367

And verify that the mode is correct:

py> statistics.mode(f.elements())
85

We can see what happens if the data is fiddled with:

py> g = f.copy()
py> g[42] += g[23]
py> g[23] = 0
py> statistics.median(g.elements())
50.0
py> statistics.mean(g.elements())
49.7263
py> statistics.stdev(g.elements())
28.757647388343212
py> g.most_common(3)
[(42, 20258), (85, 10313), (56, 10208)]

The median is unchanged, the mean shifts slightly higher, and the standard
deviation increases. But as you can see, these are not particularly
powerful tests of randomness: only the mode shows an obvious deviation from
uniformity.


> But what are the ranges you
> would expect with a good random function. Then it could be used to
> test a random function.


Python's pseudo-random number generator is based on the Mersenne Twister.
This is one of the most high-quality and extensively tested PRNGs
available, with a period of 2**19937-1 before it repeats. Its randomness
properties are very good indeed. (Although of course that isn't to say that
there aren't bugs in the Python implementation.)

Mersenne Twister is is not suitable for cryptographic work, but apart from
that, it is pretty much as indistinguishable from random as any
deterministic computer program can be.



-- 
Steven



From auriocus at gmx.de  Sun Jun  7 08:53:22 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Sun, 07 Jun 2015 14:53:22 +0200
Subject: Testing random
In-Reply-To: <87oaksowwg.fsf@Equus.decebal.nl>
References: <87oaksowwg.fsf@Equus.decebal.nl>
Message-ID: <ml1epn$peo$1@dont-email.me>

Am 07.06.15 um 08:27 schrieb Cecil Westerhof:
> I wrote a very simple function to test random:
>      def test_random(length, multiplier = 10000):
>          number_list = length * [0]
>          for i in range(length * multiplier):
>              number_list[random.randint(0, length - 1)] += 1
>          minimum = min(number_list)
>          maximum = max(number_list)
>          return (minimum, maximum, minimum / maximum)
>

> It shows that when the first parameter increases the deviation
> increases and when the second parameter increases the deviation
> decreases. Exactly what you would expect. But what are the ranges you
> would expect with a good random function. Then it could be used to
> test a random function.

Random number generators (RNG or PRNG) are usually tested using similar 
methods, where the outcome can be assigned a probability distribution. 
For example, diehard is a famous sequence of tests for RNGs.

http://en.wikipedia.org/wiki/Diehard_tests

You are only checking for uniform distribution. This couldn't detect 
correlation, for instance if the RNG would just return increasing 
numbers. A better check for uniformness could be done by the chi square 
test or Kolmogorov-Smirnov. Then there are tables which relate the 
deviations to significance.

	Christian


From Cecil at decebal.nl  Sun Jun  7 08:58:40 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 14:58:40 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <55740ecf$0$12978$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87lhfvoetb.fsf@Equus.decebal.nl>

On Sunday  7 Jun 2015 11:28 CEST, Steven D'Aprano wrote:

> Here is a simple example:
>
> http://code.activestate.com/recipes/577896-benchmark-code-with-the-with-statement/
I use that now in my function:
#-----------------------------------------------------------------------
def time_test(function, arguments, print_time = True):
    t = Timer()
    with t:
        results     = function(*arguments)
    used_time = t.interval
    if print_time:
        print('It took {0} seconds'.format(used_time))
    else:
        results = (used_time, results)
    return results
#-----------------------------------------------------------------------

I find that neater. It also looks like the time value is a little
less, but I did not really measure it.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From lac at openend.se  Sun Jun  7 09:01:43 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 07 Jun 2015 15:01:43 +0200
Subject: Find in ipython3
In-Reply-To: Message from Peter Otten <__peter__@web.de>
 of "Sun, 07 Jun 2015 12:27:05 +0200." <ml16a9$qg8$1@ger.gmane.org>
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.177.1433448836.13271.python-list@python.org>
 <87bnguhbec.fsf@Equus.decebal.nl> <874mmlqhul.fsf@Equus.decebal.nl>
 <mailman.215.1433588892.13271.python-list@python.org>
 <87sia4ox8h.fsf@Equus.decebal.nl><ml16a9$qg8$1@ger.gmane.org>
Message-ID: <201506071301.t57D1hVq003587@fido.openend.se>

In a message of Sun, 07 Jun 2015 12:27:05 +0200, Peter Otten writes:
>> There is no gain to get in standard Python? By switching from fnmatch
>> to re I got almost a speed gain of two. So I was wondering if I could
>> do more.
>
>Just wait for Python 3.5. The switch from os.listdir() to the (new) 
>os.scandir() in the implementation of os.walk() is likely to improve the 
>situation:

Oh cool.  I had no idea.  Thank you for showing us this.
Laura



From lac at openend.se  Sun Jun  7 09:03:10 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 07 Jun 2015 15:03:10 +0200
Subject: Function to show time to execute another function
In-Reply-To: Message from Mark Lawrence <breamoreboy@yahoo.co.uk>
 of "Sun, 07 Jun 2015 11:16:30 +0100." <ml15m4$f44$1@ger.gmane.org>
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl><ml15m4$f44$1@ger.gmane.org>
Message-ID: <201506071303.t57D3AFI003645@fido.openend.se>

In a message of Sun, 07 Jun 2015 11:16:30 +0100, Mark Lawrence writes:
>I suggest that you stop asking so many question here.  Get your cheque 
>book and go for paid support.

Knock this off, please.  Some of us dearly like to teach
people who want to explore and learn things.

Laura



From steve at pearwood.info  Sun Jun  7 09:06:52 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 23:06:52 +1000
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <55743ea4$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <557441ed$0$13009$c3e8da3$5496439d@news.astraweb.com>

On Sun, 7 Jun 2015 10:52 pm, Steven D'Aprano wrote:

> The median is unchanged, the mean shifts slightly higher, and the standard
> deviation increases. But as you can see, these are not particularly
> powerful tests of randomness: only the mode shows an obvious deviation
> from uniformity.

Oh, I forgot: we can look at the frequencies themselves as well. If our
sample is absolutely perfectly distributed uniformly, then every element
will have a frequency of exactly 10000. We don't necessarily expect this
from a random sample, but if we're too far from that, we should be
concerned. Here are the results from the fair sample:

py> statistics.mean(f.values())
10000.0
py> statistics.median(f.values())
9995.0
py> statistics.stdev(f.values())
91.8027089673141

If I do the same thing with the biased sample, the standard deviation stands
out like a sore thumb:

py> statistics.mean(g.values())
10000.0
py> statistics.median(g.values())
9993.0
py> statistics.stdev(g.values())
1442.527341617181



-- 
Steven



From kwpolska at gmail.com  Sun Jun  7 09:10:33 2015
From: kwpolska at gmail.com (Chris Warrick)
Date: Sun, 7 Jun 2015 15:10:33 +0200
Subject: Is it a newsgroup or a list?
In-Reply-To: <55742ebc$0$12980$c3e8da3$5496439d@news.astraweb.com>
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl> <ml11i1$31j$1@speranza.aioe.org>
 <mailman.240.1433676054.13271.python-list@python.org>
 <55742ebc$0$12980$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAMw+j7LQa3qjRFZoRHOYOo8kVgDSVuQ9SLBL_UoWGEpbcc_2KQ@mail.gmail.com>

On Sun, Jun 7, 2015 at 1:45 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> As far as I know, python-list a.k.a. comp.lang.python is the only one of the
> Python mailing lists with an official newsgroup mirror.

comp.lang.python.announce also exists.

Sent via mailing list.

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16


From steve at pearwood.info  Sun Jun  7 09:16:22 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 23:16:22 +1000
Subject: Find in ipython3
References: <87y4k2hyvf.fsf@Equus.decebal.nl>
 <mailman.177.1433448836.13271.python-list@python.org>
 <87bnguhbec.fsf@Equus.decebal.nl> <874mmlqhul.fsf@Equus.decebal.nl>
 <mailman.215.1433588892.13271.python-list@python.org><87sia4ox8h.fsf@Equus.decebal.nl>
 <mailman.233.1433669631.13271.python-list@python.org>
Message-ID: <55744426$0$12986$c3e8da3$5496439d@news.astraweb.com>

On Sun, 7 Jun 2015 07:33 pm, Laura Creighton wrote:

> In a message of Sun, 07 Jun 2015 08:20:46 +0200, Cecil Westerhof writes:
>>> You may get faster results if you use Matthew Barnett's replacement
>>> for re here: https://pypi.python.org/pypi/regex
>>>
>>> You will get faster results if you build your IPython shell to use
>>> PyPy, but I would still be very surprised if it beat the C program
>>> find.
>>
>>I have to look into that. But I prefer to write a version that can be
>>used by ?everyone?.
> 
> Well, everybody can download Matthew Barnett's regex, and get pypy,
> and the claim is that ipython just works with pypy, and if it ever doesn't
> the ipython team wants a bug report, so I am not sure what you mean by
> 'used by "everyone"' here that these don't have.


Pedantically speaking, not everyone has Internet access, or even
electricity, but you know that :-)

More to the point, I think what Cecil might be trying to say is that he
wants to rely only on the standard library. Many people in schools, or
corporate environments, do not control what is on their computer and cannot
just download regex and pypy (not if they want to keep their job). There
may be processes to follow, forms to fill out, paperwork to file, approval
to be gained, and sometimes in a culture which is perplexed by, if not
actively hostile to, FOSS culture. ("What do you mean it's free? You mean
its pirated?")

For many of those people, they've already gone through the process to get
Python and the standard library approved. It might be part of the school's
SOE, or already installed on the corporate server, for example, so they can
use the standard library easily enough, but nothing else.



-- 
Steven



From steve at pearwood.info  Sun Jun  7 09:24:05 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 07 Jun 2015 23:24:05 +1000
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
 <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <2c4d029c-8ea5-465b-8adc-6c35185bd150@googlegroups.com>
 <2483375.eHyISxeWLQ@PointedEars.de>
 <55742e0e$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <mailman.243.1433678894.13271.python-list@python.org>
Message-ID: <557445f6$0$12997$c3e8da3$5496439d@news.astraweb.com>

On Sun, 7 Jun 2015 10:08 pm, Chris Angelico wrote:

> On Sun, Jun 7, 2015 at 9:42 PM, Steven D'Aprano <steve at pearwood.info>
> wrote:
>> My opinion is that a programming language like Python or ECMAScript
>> should operate on *code points*. If we want to call them "characters"
>> informally, that should be allowed, but whenever there is ambiguity we
>> should remember we're dealing with code points. The implementation
>> shouldn't matter: compliant Python interpreters might choose to use UTF-8
>> internally, or UTF-16, or UTF-32, or something else, and still agree on
>> how many characters a string contains. Normalisation is still an issue,
>> of course, but any decent Unicode implementation will include a way to
>> normalise or denormalise strings.
> 
> If by "normalise" you mean the NF[K]C/NF[K]D composition and
> decomposition, then yes, any decent Unicode library will provide that.

Dat's der bunny!

> I'm not sure it's critical to string handling itself, though; and
> Python defers the operation to the unicodedata module:
> 
>>>> s1 = "\N{LATIN SMALL LETTER A}\N{COMBINING ACUTE ACCENT}"
>>>> s2 = "\N{LATIN SMALL LETTER A WITH ACUTE}"
>>>> s1 == s2
> False
>>>> unicodedata.normalize("NFC", s1) == s2
> True
> 
> It's a useful operation to be able to do, but I would never expect
> that *string comparison* or other operations should automatically
> normalize.

I completely agree.

It might be convenient to have a string equality method that did
normalisation, but for most cases it would be unnecessary and slow. I think
that's the sort of thing which should be left to a subclass of str, and it
should normalise on construction.


> (Unless you want to say that all strings are guaranteed to 
> be NFC/NFD normalized, such that s1 and s2 would actually be
> identical, which I suppose is plausible. I'm not sure what the
> advantage would be, though. And certainly you wouldn't want to
> K-normalize strings automatically.)

I believe that filenames on Apple file systems (HFS+ if I remember
correctly) are guaranteed to be both normalised and correctly encoded as
UTF-8. If you could live in a purely Apple world, you'd have far fewer
filename hassles.



-- 
Steven



From __peter__ at web.de  Sun Jun  7 09:35:55 2015
From: __peter__ at web.de (Peter Otten)
Date: Sun, 07 Jun 2015 15:35:55 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <55743ea4$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <ml1hbr$ogu$1@ger.gmane.org>

Steven D'Aprano wrote:


>> I wrote a very simple function to test random:
>>     def test_random(length, multiplier = 10000):
>>         number_list = length * [0]
>>         for i in range(length * multiplier):
>>             number_list[random.randint(0, length - 1)] += 1
>>         minimum = min(number_list)
>>         maximum = max(number_list)
>>         return (minimum, maximum, minimum / maximum)
> 
> Putting aside the timing aspects, your frequency calculations are not done
> in a very Pythonic manner. 

I would agree if the frequency table were sparse, i. e. many indices with

number_list[index] == 0

but that's not the case with on average 10000 hits per index.

> A better way might be:

I'm too lazy to measure, but this will likely be a tad slower. Did you mean 
to convey this by "Putting aside the timing aspects"?

> from collections import Counter
> from random import randint
> 
> def test_random(length, multiplier = 10000):
>     freq = Counter(
>         randint(0, length - 1) for i in range(length * multiplier)
>         )
>     minimum = min(freq[i] for i in range(length))

How about

      if len(freq) < length:
          minimum = 0
      else:
          minimum = min(freq.values())

Not as an optimisation (replacing randint() with randrange() is probably 
more effective in that department), but because it's closer to being self-
explanatory.

>     maximum = max(freq.values())
>     return (minimum, maximum, minimum / maximum)
 



From mail at timgolden.me.uk  Sun Jun  7 09:57:55 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Sun, 07 Jun 2015 14:57:55 +0100
Subject: Is it a newsgroup or a list?
In-Reply-To: <1433676052.3089941.288915321.4982B8F8@webmail.messagingengine.com>
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl> <ml11i1$31j$1@speranza.aioe.org>
 <1433676052.3089941.288915321.4982B8F8@webmail.messagingengine.com>
Message-ID: <55744DE3.80908@timgolden.me.uk>

On 07/06/2015 12:20, random832 at fastmail.us wrote:
> On Sun, Jun 7, 2015, at 05:06, Luca Menegotto wrote:
>> Il 07/06/2015 10:22, Cecil Westerhof ha scritto:
>>   > That only times the function. I explicitly mentioned I want both the
>>   > needed time AND the output.
>>   >
>>   > Sadly the quality of the answers on this list is going down....
>>
>> First of all, thank God it's a newsgroup, not a list.
> ...
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
> I've always thought of this as a list that is mirrored to a newsgroup,
> not a newsgroup that is mirrored to a list. What is the true nature of
> python-list (and others such as python-ideas)? What is it primarily,
> what was it first, etc?
>

It's both: they're mirrored to each other and I consider both to be 
first class citizens. In strict chronology I believe comp.lang.python 
existed first, but python-list has been around for long enough that the 
distinction now is really quite academic.

I would not be surprised to find that a fair majority of people told off 
on this list for abusing newsgroup etiquette in one way or another not 
only have no idea that this is (also) a newsgroup but don't even know 
what a newsgroup *is*.

They may also not know that it's a mailing list, nor what that is, 
because they're coming via Google Groups -- which uses the Usenet 
gateway -- or through gmane -- which comes in through the mailing list 
-- or some other web mirror.


TJG


From otlucaDELETE at DELETEyahoo.it  Sun Jun  7 10:12:23 2015
From: otlucaDELETE at DELETEyahoo.it (Luca Menegotto)
Date: Sun, 07 Jun 2015 16:12:23 +0200
Subject: Is it a newsgroup or a list?
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl> <ml11i1$31j$1@speranza.aioe.org>
 <mailman.240.1433676054.13271.python-list@python.org>
 <55742ebc$0$12980$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <ml1jg6$bte$1@speranza.aioe.org>

Il 07/06/2015 13:45, Steven D'Aprano ha scritto:

> As far as I know, python-list a.k.a. comp.lang.python is the only one of the
> Python mailing lists with an official newsgroup mirror.

OK. So let me rephrase: Thank God the list is mirrired to a newsgroup...

-- 
Ciao!
Luca




From andrei.fokau at gmail.com  Sun Jun  7 10:31:28 2015
From: andrei.fokau at gmail.com (Andrei)
Date: Sun, 7 Jun 2015 07:31:28 -0700 (PDT)
Subject: Detect if specific Python.app instance is already running
Message-ID: <a3be1707-0eae-46fa-9921-e9c145816ad5@googlegroups.com>

Hello,

I am experimenting with OS X apps written in Python and need to detect if there is already an instance of Python.app running with certain script. My script modifies CFBundleName on-the-fly from "Python" to "MyApp" to change the app title in the menubar. If I start another instance and check CFBundleName of the running apps, it will only tell me the original value, i.e. "Python":

for app in NSWorkspace.sharedWorkspace().runningApplications():
    bundle = NSBundle.bundleWithURL_(app.bundleURL())
    name = bundle.infoDictionary().get('CFBundleName')
    if name in ('Python', 'MyApp'):
        print name  # => prints Python

So I need to find a way to mark a Python.app instance that runs MyApp script to be able to abort launching duplicate instances. Is there such way?

Thanks,
Andrei


From rosuav at gmail.com  Sun Jun  7 10:47:14 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 00:47:14 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <557445f6$0$12997$c3e8da3$5496439d@news.astraweb.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
 <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <2c4d029c-8ea5-465b-8adc-6c35185bd150@googlegroups.com>
 <2483375.eHyISxeWLQ@PointedEars.de>
 <55742e0e$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <mailman.243.1433678894.13271.python-list@python.org>
 <557445f6$0$12997$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmrTrQbZzarg0phQrmCLmBO4rdA6CLSy_wMWOkYbrnLNqg@mail.gmail.com>

On Sun, Jun 7, 2015 at 11:24 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> (Unless you want to say that all strings are guaranteed to
>> be NFC/NFD normalized, such that s1 and s2 would actually be
>> identical, which I suppose is plausible. I'm not sure what the
>> advantage would be, though. And certainly you wouldn't want to
>> K-normalize strings automatically.)
>
> I believe that filenames on Apple file systems (HFS+ if I remember
> correctly) are guaranteed to be both normalised and correctly encoded as
> UTF-8. If you could live in a purely Apple world, you'd have far fewer
> filename hassles.

Yep. Actually, there should be nothing stopping the next Linux file
system ("ext5" or whatever) from enforcing the same thing;
byte-oriented filename APIs would still work just fine, but you could
have some confidence that at least local file systems will normally be
decodable as UTF-8. Then the only time you'd have to worry about
encoding problems would be network or removable file systems - no
worrying about "what's the FS encoding", because it'll just be UTF-8.
(Hmm. Point of interest: What happens on a Mac if you network-mount
something that isn't Unicode? If the enforcement of UTF-8 and
normalization is done at the file system level, it's no different from
the current Linux situation, where basically anything goes.)

But that's file names, not strings in a program. I'm not sure that
mandating that strings be normalized is particularly useful, but on
the flip side, I'm not sure of any situation where it'd be majorly
problematic either. There are ambiguities in some encodings, and as
soon as you decode from them and re-encode, you've effectively folded
those ambiguities to some canonical form; if your language
automatically normalized strings, you'd just have the same effect of
folding. And then you could have encode methods that stipulate the
other form of normalization - say you NFD everything internally, you
could then have a method "a\u0301".encode("utf-8", combine=True) which
NFC normalizes prior to encoding (and would thus be C3 A1 instead of
61 CC 81). Are there any languages out there that work this way?

ChrisA


From random832 at fastmail.us  Sun Jun  7 10:58:52 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Sun, 07 Jun 2015 10:58:52 -0400
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <55742e0e$0$12980$c3e8da3$5496439d@news.astraweb.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
 <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <2c4d029c-8ea5-465b-8adc-6c35185bd150@googlegroups.com>
 <2483375.eHyISxeWLQ@PointedEars.de>
 <55742e0e$0$12980$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <1433689132.2946028.289007025.05F10917@webmail.messagingengine.com>

On Sun, Jun 7, 2015, at 07:42, Steven D'Aprano wrote:
> The question of graphemes (what "ordinary people" consider letters and
> characters, e.g. "ch" is two letters to an English speaker but one letter
> to a Czech speaker) should be left to libraries.

Do Czech speakers expect to be able to select and delete it as a single
unit and never have the cursor in the middle of it? If not, then this is
not really fundamentally the same thing as what we have with combining
characters or certain sequences of Indic letters.

Also, "should be left to libraries" isn't really a coherent statement
when we are talking about the design of the standard *library*.


From rosuav at gmail.com  Sun Jun  7 11:12:33 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 01:12:33 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <1433689132.2946028.289007025.05F10917@webmail.messagingengine.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
 <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <2c4d029c-8ea5-465b-8adc-6c35185bd150@googlegroups.com>
 <2483375.eHyISxeWLQ@PointedEars.de>
 <55742e0e$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <1433689132.2946028.289007025.05F10917@webmail.messagingengine.com>
Message-ID: <CAPTjJmr_9gFrg+rKfG2sSr0XB+5D5EA_8a4tGFuSqa7+GeqVuQ@mail.gmail.com>

On Mon, Jun 8, 2015 at 12:58 AM,  <random832 at fastmail.us> wrote:
> On Sun, Jun 7, 2015, at 07:42, Steven D'Aprano wrote:
>> The question of graphemes (what "ordinary people" consider letters and
>> characters, e.g. "ch" is two letters to an English speaker but one letter
>> to a Czech speaker) should be left to libraries.
>
> Do Czech speakers expect to be able to select and delete it as a single
> unit and never have the cursor in the middle of it? If not, then this is
> not really fundamentally the same thing as what we have with combining
> characters or certain sequences of Indic letters.

Not sure about Indic letters, but with combining characters, you *should*
select and delete a single unit containing a base character and all its
combining characters, and you should never have the cursor in the middle of
it. (Not everything gets this right; SciTE, though otherwise a decent text
editor, does allow the cursor to go inside combining characters.) But I
suspect that with the Czech "ch", like the Dutch "ij" and the German "oe"
(when it's not ?), should be treated as two separate characters.

Digression: English has seventy phonograms, which are what words are really
built out of. Digraphs like "th" and "sh", represent single sounds despite
being spelled with multiple letters - but nobody ever expects them to be
treated as single character units just because other languages spell them
"?" or "?". The alphabet of English includes "q", which is not a phonogram
on its own ("qu" is), and doesn't include all the digraphs, and any
character-based representation of English should correspondingly work with
letters, not phonograms.

I don't know Czech enough to be able to say whether "ch" is more like a
single letter or a phonogram, but even if it basically functions as a
letter, I suspect that treating it as two characters will be no surprise to
most people.

ChrisA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150608/a24cb599/attachment.html>

From gheskett at wdtv.com  Sun Jun  7 11:49:50 2015
From: gheskett at wdtv.com (Gene Heskett)
Date: Sun, 7 Jun 2015 11:49:50 -0400
Subject: Is it a newsgroup or a list?
In-Reply-To: <55744DE3.80908@timgolden.me.uk>
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <1433676052.3089941.288915321.4982B8F8@webmail.messagingengine.com>
 <55744DE3.80908@timgolden.me.uk>
Message-ID: <201506071149.50339.gheskett@wdtv.com>



On Sunday 07 June 2015 09:57:55 Tim Golden wrote:
> On 07/06/2015 12:20, random832 at fastmail.us wrote:
> > On Sun, Jun 7, 2015, at 05:06, Luca Menegotto wrote:
> >> Il 07/06/2015 10:22, Cecil Westerhof ha scritto:
> >>   > That only times the function. I explicitly mentioned I want
> >>   > both the needed time AND the output.
> >>   >
> >>   > Sadly the quality of the answers on this list is going down....
> >>
> >> First of all, thank God it's a newsgroup, not a list.
> >
> > ...
> >
> >> --
> >> https://mail.python.org/mailman/listinfo/python-list
> >
> > I've always thought of this as a list that is mirrored to a
> > newsgroup, not a newsgroup that is mirrored to a list. What is the
> > true nature of python-list (and others such as python-ideas)? What
> > is it primarily, what was it first, etc?
>
> It's both: they're mirrored to each other and I consider both to be
> first class citizens. In strict chronology I believe comp.lang.python
> existed first, but python-list has been around for long enough that
> the distinction now is really quite academic.

And I should point out that streaming to a newsgroup server at your local 
ISP, is something not often justified as it can quite easily use up the 
bandwidth your local ISP is buying.  Mine is one such as the newsgroups 
in total are often tying up 20+Gb/sec just for that one service.

In contrast, subbing to 50 mailing lists per customer translates to about 
1% of the bandwidth they are provisioned for.

If you were an ISP, how would you handle it? The mailing list version is 
a no brainer.

> I would not be surprised to find that a fair majority of people told
> off on this list for abusing newsgroup etiquette in one way or another
> not only have no idea that this is (also) a newsgroup but don't even
> know what a newsgroup *is*.

This is true. But newsgroups I believe predate a mailing list by perhaps 
6 months. I sat and watched the net come to life, dialing up Delphi at 
300 baud in the beginning, running up an LD bill often north of $100/mo 
because the access number was not a local call.  On a TRS-80 Color 
Computer in those days.

> They may also not know that it's a mailing list, nor what that is,
> because they're coming via Google Groups -- which uses the Usenet
> gateway -- or through gmane -- which comes in through the mailing list
> -- or some other web mirror.

You mentioned GoogleGroups, now go warsh yur mouth out with some of 
Grandma's Lye soap.  This list is 500% easier to read when they are 
filtered out. I still see the responses but they are at least formatted 
for readability.

> TJG

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>


From rustompmody at gmail.com  Sun Jun  7 11:49:52 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sun, 7 Jun 2015 08:49:52 -0700 (PDT)
Subject: Lawful != Mutable (was Can Python function return multiple data?)
In-Reply-To: <mailman.212.1433569420.13271.python-list@python.org>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
 <mailman.212.1433569420.13271.python-list@python.org>
Message-ID: <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>

On Saturday, June 6, 2015 at 11:13:52 AM UTC+5:30, Chris Angelico wrote:
> On Sat, Jun 6, 2015 at 3:28 PM, Rustom Mody  wrote:
> > You just repeated what Chris said, replacing 'immutable' with 'same'
> > There was a list: [1,2,3]
> > At some point that list is found to be(come) [1,2,3,4]
> > They dont look same to me.
> 
> "I'm going shopping, can you get me the shopping list please?"
> *goes and fetches a piece of paper from the kitchen*
> "Wait, this isn't the right list! This one has more things on it!"
> 
> The question of whether or not the thing fetched is indeed the
> shopping list is independent of the items on it. The list has an
> identity and it has a value (the items needed). If I hand you an empty
> list on the basis that the shopping list you placed there last week
> was empty, I've destroyed the value of the posted shopping list -
> people have added things to it during the week, and they expect those
> things to be on the list that gets referenced to make purchases.


Well that is a more useful metaphor than much of what I see being said here.
[Most of what I see being summarizable into:
This is the way python is
Python is Holy Writ
THOU SHALT NOT QUESTION
]

So thanks Chris for a non-pythonic metaphor.

May we look a little into it?

Is your shopping list really a list? If I have:

1. Bread
2. Eggs
3. Rice

Presumably that's the 'same' list as

1. Eggs
2. Bread
3. Rice

IOW if we have to make a python data-structure for it, the (python) list
["eggs", "bread", "rice"]
is overspecific. Whereas the set
set(["eggs", "bread", "rice"])
works better for the simple reason that python will conform with our informal
expectation of the two differently listed shopping lists being 
equal with the set data structure and not the list data structure.  ie
["eggs", "bread", "rice"] == ["bread", "eggs", "rice"] fails
whereas
set(["eggs", "bread", "rice"]) == set(["bread", "eggs", "rice"]) works

Is it only coincidentally related to the math law: 
{1,2,3} = {2,1,3}  ?

Now say we change the example slightly, from shopping list to
shopping itenary:
1. Grocer
2. Hardware
3. Clothes

and due to geography of the stores the above order is the best one.
[Yeah I am assuming the Travelling Buysman problem is a solved problem]


So I model that as
itinerary = ["grocer", "hardware", "clothes"]

O! I forgot... Need to cut my hair

itinerary[:2] + ["barber"] + itinerary[-1:]
['grocer', 'hardware', 'barber', 'clothes']
>>> 

Nice... Just one doubt
Do I need to parenthesize that expression??

I try
>>> itinerary[:2] + (["barber"] + itinerary[-1:])
['grocer', 'hardware', 'barber', 'clothes']
>>> (itinerary[:2] + ["barber"]) + itinerary[-1:]
['grocer', 'hardware', 'barber', 'clothes']
>>> 

Seems to be the same.
Nice...

But is it always so?
Or did it just happen by coincidence?

IOW: Does list-+ satisfy an associative LAW of the form:

(? a,b,c:list ? (a + b) + c = a + (b + c) ) ??


Now try it with mutation.

Some (wiseguy) told me that mutating data structures can be more efficient
than 'functional' ones.

Sounds good... Just not sure how/when associativity works/breaks.

Say we want to stuff a 4 between the 2 and the 3 in [1,2,3]
[Its a bit tiresome to keep typing 'grocer' etc]

>>> it = [1,2,3]
>>> it[:2]
[1, 2]
>>> it[:2].append(4)
>>> it
[1, 2, 3]

Ho Hum.
Try again

>>> it = [1,2,3]
>>> it = it[:2].append(4)
>>> it = it.append(it[-1])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'append'


Got to 'un-None' the #$@%%^ thing

>>> def app(a,b):
...   a.extend(b)
...   return a
... 

>>> a,b,c=[1,2],[3,4],[5,6]
>>> app(app(a,b),c)
[1, 2, 3, 4, 5, 6]
>>> a,b,c=[1,2],[3,4],[5,6]
>>> app(a,app(b,c))
[1, 2, 3, 4, 5, 6]

Looks right so far.
A few more checks...

>>> a,b=[1,2],[3,4]
>>> app(app(a,a),b)  # (a + a) + b
[1, 2, 1, 2, 3, 4]
>>> a,b=[1,2],[3,4]  # ok
>>> app(a,app(a,b))  # a + (a + b)
[1, 2, 3, 4, 1, 2, 3, 4]  # Whooops!


No I am not claiming that my pretence at noob struggles is very convincing.
Hopefully works enough to show that mutatory programming may be a lot of 'fun'
However imperative ? mutating  ? not-lawful

In Summary:

Correct programs are in accordance with math/logic laws
Incorrect programs are written by people ignorant of laws
Immutability is the bedrock of lawfulness -- If Newton had assumed
that between the time Kepler made his calculations and Newton recast them
into general form, the gravitional constant 'G' was changing, we would not
have physics or science as we know it
Properties like (python's version of) immutability just make stating of lawful
behavior unduly hard by asserting gibberish such as
[1,2,3] is [1,2,3,4].

[Just to be clear, I find python's notion of immutability sloppy but hard
to avoid whereas python's use of the 'is' operator for pointer-equality
a much bigger linguistic blunder. The two errors are closely related]

Piquant footnote: The big proponent of 'lawful?' programming is
Lambert Meertens, who happens to be the grandfather? of python.

? http://www.kestrel.edu/home/people/meertens/publications/papers/Algorithmics.pdf
? http://en.wikipedia.org/wiki/ABC_%28programming_language%29


From PointedEars at web.de  Sun Jun  7 11:51:20 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 17:51:20 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
Message-ID: <3158703.Lr4HFMbMOd@PointedEars.de>

Chris Angelico wrote:

> On Sun, Jun 7, 2015 at 8:40 PM, Thomas 'PointedEars' Lahn
> <PointedEars at web.de> wrote:
>> Cecil Westerhof wrote:
>>> I wrote a very simple function to test random:
>>>     def test_random(length, multiplier = 10000):
>>>         number_list = length * [0]
>>>         for i in range(length * multiplier):
>>>             number_list[random.randint(0, length - 1)] += 1
>>>         minimum = min(number_list)
>>>         maximum = max(number_list)
>>>         return (minimum, maximum, minimum / maximum)
>>
>> As there is no guarantee that every number will occur randomly, using a
>> dictionary at first should be more efficient than a list:
> 
> Hmm, I'm not sure that's actually so. His code is aiming to get
> 'multiplier' values in each box; for any serious multiplier (he starts
> with 10 in the main code), you can be fairly confident that every
> number will come up at least once. 

The wording shows a common misconception: that random distribution would 
mean that it is guaranteed or more probable that every element of the set 
will occur at least once.  It is another common misconception that 
increasing the number of trials would increase the probability of that 
happening.  But that is not so.

The law of large numbers only says that as you increase the number of 
trials, that the relative frequency *approaches* the probability for each 
value of the probability variable, or IOW ?the average of the results 
obtained from a large number of trials should be close to the expected 
value, and will *tend to become closer* as more trials are performed.? 
(<http://en.wikipedia.org/wiki/Law_of_large_numbers>; emphasis by me)

> [?] a true RNG could legitimately produce nothing but 7s for the entire 
> run, it's just extremely unlikely.

That reasoning is precisely a result of the misconception described above.  
Because people think that every value must occur, they do not think it 
possible that (much) repetition could occur with a (pseudo-)random 
generator, and when they want to mince words they say ?(extremely) unlikely? 
instead.  For example, when people see

  6 5 8 7 9 3 7 8 4 7 5 6 8 8 1 2 8 3 5 7 5 4 1 2 4 8 8 7 5 1

and are told that this is random sequence, they find it hard to believe.  
They think something like: ?Look at all those repeated 8, and ?7, 5? 
occurs twice.  4 occurs more often than 2, and there are much more 5s than 
1s.  That cannot be possibly be random!?

Yes, it *can*.  I have just produced it with

#------------------------------------------------------------------
import random
print(" ".join([str(random.randint(1, 9)) for i in range(0, 30)]))'
#------------------------------------------------------------------

But if you think this *Mersenne Twister* PRNG which ?generates numbers with 
nearly uniform distribution and a large period? is flawed, use a proper die 
as RNG, and a sheet of paper to record the outcomes, and do the experiment.  
The outcome is not going to be different.

If the distribution is even and the game is fair, every outcome *always* has 
the same probability of occurring.  As a result, every sequence of outcomes 
of equal length *always* has the same probability of occurring, and the 
probability for a particular sequence of equal length does _not_ increase or 
decrease based on the number of occurrences of previous outcomes.  Those are 
*independent* events.

See also: 
<http://www.teacherlink.org/content/math/interactive/probability/numbersense/misconceptions/home.html>, 
in particular the section ?Representativeness?

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From larry.martell at gmail.com  Sun Jun  7 11:56:11 2015
From: larry.martell at gmail.com (Larry Martell)
Date: Sun, 7 Jun 2015 11:56:11 -0400
Subject: Is it a newsgroup or a list?
In-Reply-To: <201506071149.50339.gheskett@wdtv.com>
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <1433676052.3089941.288915321.4982B8F8@webmail.messagingengine.com>
 <55744DE3.80908@timgolden.me.uk>
 <201506071149.50339.gheskett@wdtv.com>
Message-ID: <CACwCsY6WscqesOpmeJ+sUU0eLiz5xTeg2ffyxXhRoU7G5eR77g@mail.gmail.com>

On Sun, Jun 7, 2015 at 11:49 AM, Gene Heskett <gheskett at wdtv.com> wrote:
> You mentioned GoogleGroups, now go warsh yur mouth out with some of
> Grandma's Lye soap.  This list is 500% easier to read when they are
> filtered out. I still see the responses but they are at least formatted
> for readability.

Gene is very wise. Google Groups is evil.


From rosuav at gmail.com  Sun Jun  7 12:07:54 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 02:07:54 +1000
Subject: Lawful != Mutable (was Can Python function return multiple data?)
In-Reply-To: <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
 <mailman.212.1433569420.13271.python-list@python.org>
 <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>
Message-ID: <CAPTjJmrjG+0tH3aMQWREA7Vqj_sYnZc-fK_bgEH94zPJmZDqVQ@mail.gmail.com>

On Mon, Jun 8, 2015 at 1:49 AM, Rustom Mody <rustompmody at gmail.com> wrote:
> On Saturday, June 6, 2015 at 11:13:52 AM UTC+5:30, Chris Angelico wrote:
>> On Sat, Jun 6, 2015 at 3:28 PM, Rustom Mody  wrote:
>> > You just repeated what Chris said, replacing 'immutable' with 'same'
>> > There was a list: [1,2,3]
>> > At some point that list is found to be(come) [1,2,3,4]
>> > They dont look same to me.
>>
>> "I'm going shopping, can you get me the shopping list please?"
>> *goes and fetches a piece of paper from the kitchen*
>> "Wait, this isn't the right list! This one has more things on it!"
>>
>> The question of whether or not the thing fetched is indeed the
>> shopping list is independent of the items on it. The list has an
>> identity and it has a value (the items needed). If I hand you an empty
>> list on the basis that the shopping list you placed there last week
>> was empty, I've destroyed the value of the posted shopping list -
>> people have added things to it during the week, and they expect those
>> things to be on the list that gets referenced to make purchases.
>
>
> Well that is a more useful metaphor than much of what I see being said here.
> [Most of what I see being summarizable into:
> This is the way python is
> Python is Holy Writ
> THOU SHALT NOT QUESTION
> ]
>
> So thanks Chris for a non-pythonic metaphor.
>
> May we look a little into it?
>
> Is your shopping list really a list? If I have:
>
> 1. Bread
> 2. Eggs
> 3. Rice
>
> Presumably that's the 'same' list as
>
> 1. Eggs
> 2. Bread
> 3. Rice
>
> IOW if we have to make a python data-structure for it, the (python) list
> ["eggs", "bread", "rice"]
> is overspecific. Whereas the set
> set(["eggs", "bread", "rice"])
> works better for the simple reason that python will conform with our informal
> expectation of the two differently listed shopping lists being
> equal with the set data structure and not the list data structure.  ie
> ["eggs", "bread", "rice"] == ["bread", "eggs", "rice"] fails
> whereas
> set(["eggs", "bread", "rice"]) == set(["bread", "eggs", "rice"]) works

Well, yes and no. I used the term "list" because most people don't
talk about a "shopping set", and the list that's posted does have an
order to it; the effect of the list will be the same regardless of
order, but the order might be useful for curiosity purposes - the
items at the head of the list have been on it longer than the ones at
the tail, so they most likely will be more needed. For a comparison,
I'm sure you'd agree that the order of a program's arguments matters,
so you need a list and not a set; but if you type "cp spam eggs ham
target/" it's going to have the same net result as "cp eggs spam ham
target/", in that all three files will be copied to the same target
directory.

There are plenty of real-world lists where order turns out to not
matter, but they still retain that order. On any membership list, the
important thing is whether you're on it or not, but somewhere on a
sheet of paper, someone's going to be higher and someone's going to be
lower. A list of the different types of Cadbury Roses has to be sorted
one way or another; the Wikipedia page [1] mentions Cherry Heaven
above Turkish Delight, but my blog [2] has them the other way around -
is one of them somehow wrong? These are, in a way, sets masquerading
as lists, but given that many contexts don't really allow any
distinction, it's not usually considered to be a problem.

So you can call them sets if you like, but there's no real
significance either way.

ChrisA

[1] http://en.wikipedia.org/wiki/Cadbury_Roses
[2] http://rosuav.blogspot.com.au/2012/06/scientific-research-into-cadbury-roses.html


From rustompmody at gmail.com  Sun Jun  7 12:20:37 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sun, 7 Jun 2015 09:20:37 -0700 (PDT)
Subject: Lawful != Mutable (was Can Python function return multiple data?)
In-Reply-To: <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
 <mailman.212.1433569420.13271.python-list@python.org>
 <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>
Message-ID: <4fdc38b3-6b51-4f98-9ba3-42b265582fc0@googlegroups.com>

On Saturday, June 6, 2015 at 10:20:49 AM UTC+5:30, Steven D'Aprano wrote:
> On Sat, 6 Jun 2015 01:20 pm, Rustom Mody wrote:
> 
> > On Saturday, June 6, 2015 at 3:30:23 AM UTC+5:30, Chris Angelico wrote:
> 
> >> Congrats! You just proved that an object can itself be immutable, but
> >> can contain references to mutables. Ain't that awesome?
> >> 
> >> Did you have a point?
> > 
> > [Under assumption you are not being facetious...]
> > The word immutuable happens to have existed in English before python.
> > I also happen to have used it before I knew of python
> > The two meanings do not match
> > I am surprised
> > Is that surprising?
> 
> Yes, I am surprised that you are surprised. You have been a regular, and
> prolific, contributor on this forum for some years now, teach Python, blog
> about it. You're quite obviously well-read and experienced. How is it that
> you are surprised by such a fundamental part of not just Python's object
> model, but of real life objects too?
> 
> I suspect you are pretending to be surprised to make a rhetorical point.

Dunno why the fuss...
As someone who often asks for short-simple-self-contained examples,
I would have expected you to prefer clarity of communication over
fidelity of experience??

Anyways...

Did I(rusi) now(2015) find that specific example surprising??

No, but:

1. Questioners here regularly do show similar confusions.
2. I did go through similar decades ago when I first encountered programming
3. And most to the point, just add a little complexity and I (rusi-now)
am as confused as any noob.

A few weeks ago there was this example.
It completely knocked me.
ChrisA's exxplanation clarified.
Here is a reconstruction based on Chris clarification
[Chris do you remember the question?]

>>> t=([1,2],[3,4])
>>> t
([1, 2], [3, 4])
>>> t[1] = t[1].append(5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> t
([1, 2], [3, 4, 5])

> 
> Like many English words, "immutable" has a few meanings in plain English.
> Bouvier's Law Dictionary included in 1856 this definition:
> 
>     IMMUTABLE. What cannot be removed, what is unchangeable.
>     The laws of God peing perfect, are immutable, but no 
>     human law can be so considered.
> 
> Clearly tuples can be removed. They are garbage-collected like any other
> values in Python. If nothing else, you can turn the computer off, remove
> the RAM, grind it down into the finest powder, and scatter it to the winds.
> That surely is enough to remove the tuples <wink> 

Ok now rewrite that para above with 
s/tuple/numbers like 3 or 666/
So I put '3' on the ram and grind it to finest powder.
Have all trinities (of religious or secular variety) disappeared?
666 gone has the devil been banished from God's (or Steven's) universe?

> So according to Bouvier's definition, tuples are not immutable. But I trust
> that we can agree that Bouvier's definition is not useful here.

Actually its a very nice and apt quote.
Just add laws of mathematics to laws of God and it will be perfect.
[Or better remember that for Plato they were the same]


From rosuav at gmail.com  Sun Jun  7 12:25:21 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 02:25:21 +1000
Subject: Testing random
In-Reply-To: <3158703.Lr4HFMbMOd@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
Message-ID: <CAPTjJmo_+b0yiGfupDqXRE0Zbn_oss6gHvB+yMEKNp47cGixGg@mail.gmail.com>

On Mon, Jun 8, 2015 at 1:51 AM, Thomas 'PointedEars' Lahn
<PointedEars at web.de> wrote:
> Chris Angelico wrote:
>
>> On Sun, Jun 7, 2015 at 8:40 PM, Thomas 'PointedEars' Lahn
>> <PointedEars at web.de> wrote:
>>> Cecil Westerhof wrote:
>>>> I wrote a very simple function to test random:
>>>>     def test_random(length, multiplier = 10000):
>>>>         number_list = length * [0]
>>>>         for i in range(length * multiplier):
>>>>             number_list[random.randint(0, length - 1)] += 1
>>>>         minimum = min(number_list)
>>>>         maximum = max(number_list)
>>>>         return (minimum, maximum, minimum / maximum)
>>>
>>> As there is no guarantee that every number will occur randomly, using a
>>> dictionary at first should be more efficient than a list:
>>
>> Hmm, I'm not sure that's actually so. His code is aiming to get
>> 'multiplier' values in each box; for any serious multiplier (he starts
>> with 10 in the main code), you can be fairly confident that every
>> number will come up at least once.
>
> The wording shows a common misconception: that random distribution would
> mean that it is guaranteed or more probable that every element of the set
> will occur at least once.  It is another common misconception that
> increasing the number of trials would increase the probability of that
> happening.  But that is not so.

The greater the multiplier, the lower the chance that any element will
have no hits. With uniform distribution, a length of 10, and a
multiplier of 10, there are 100 attempts with a 90% chance each that
any given number will be avoided - which works out to 0.9**100 ==
2.6561398887587544e-05 probability that (say) there'll be no 4s in the
list. Invert that and raise to the 10th power, and you get a
probability of 0.9997344177567317 that there'll be at least one in
every bucket. Raise the multiplier to 100, and the probability of a
single whiff becomes 1.7478712517226947e-46; invert and raise to the
tenth power, and it becomes closer to certainty than IEEE double
precision can represent. Raise the length to 100 and the numbers get
lower again; with multiplier 10, probability 0.9956920878572284 of
having one in every bucket (that's a half a percent chance of a zero
anywhere), and at multiplier 100, still underflows to certainty.

But you'll notice that I wasn't actually talking about certainty here.
I was talking about confidence, at levels sufficient to make data-type
decisions on. Sure, there's no guarantee that every number will occur;
but if there's a 0.4% chance that any number will be omitted, I think
the list is going to work out more efficient.

You'll notice that some of the other posts have been concerned more
about correctness (for instance, using collections.Counter and then
making sure there's a zero slot for every element - otherwise empty
slots would be omitted), and then they _do_ acknowledge the chance
that something will underflow. But with the numbers the OP gave, I
would be fully satisfied with optimizing for the case where every
bucket gets at least something.

ChrisA


From steve at pearwood.info  Sun Jun  7 12:28:35 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 08 Jun 2015 02:28:35 +1000
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
 <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <2c4d029c-8ea5-465b-8adc-6c35185bd150@googlegroups.com>
 <2483375.eHyISxeWLQ@PointedEars.de>
 <55742e0e$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <mailman.251.1433689134.13271.python-list@python.org>
Message-ID: <55747134$0$13005$c3e8da3$5496439d@news.astraweb.com>

On Mon, 8 Jun 2015 12:58 am, random832 at fastmail.us wrote:

> On Sun, Jun 7, 2015, at 07:42, Steven D'Aprano wrote:
>> The question of graphemes (what "ordinary people" consider letters and
>> characters, e.g. "ch" is two letters to an English speaker but one letter
>> to a Czech speaker) should be left to libraries.
> 
> Do Czech speakers expect to be able to select and delete it as a single
> unit and never have the cursor in the middle of it?

You'd have to ask one. I expect the answer is No, because they're used to
using software written by English speakers who think that "ch" is two
letters.

Whether they would *like* to stick the cursor between the c and the h is a
different question to whether they would *expect* it.

There may even be words where "ch" counts as two letters, where the "c" is
at the end of one syllable but the "h" is the beginning of the next.
(That's certainly the case for Dutch "ij".) Natural language is *hard*.

But generally speaking, I expect that when Czech speakers are playing (say)
Scrabble, they would want to have a tile called "CH" which they can play as
a single letter.


> If not, then this is 
> not really fundamentally the same thing as what we have with combining
> characters or certain sequences of Indic letters.

I'll have to take your word for that.

> 
> Also, "should be left to libraries" isn't really a coherent statement
> when we are talking about the design of the standard *library*.

The language offers a certain view of strings, which is reflected in the
methods that strings have, and built-in functions that operate on strings.
Should len('ch') return 1 or 2? If you think that the language should treat
strings as sequences of graphemes, then you will answer "sometimes 1".
Maybe there is a global setting to set the locale

LANG = 'Cz'
len('ch')
=> returns 1

or an optional parameter that you can pass to len:

len('ch', lang='Cz')
=> returns 1

len('ch', lang='En')
=> returns 2

But if you think that the language should treat strings as sequences of code
points, as I do, then there's only one reasonable thing for len('ch') to
return, and that is 2. But *some library* (as opposed to the built-in str
type) can offer a grapheme view of strings:

from language_tools import Graphemes
g = Graphemes.fromstr('ch', lang='Cz', exceptions=['xchx', 'ychy'])
len(g)
=> returns 1

Do you still think this is incoherent?



-- 
Steven



From rosuav at gmail.com  Sun Jun  7 12:34:22 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 02:34:22 +1000
Subject: Lawful != Mutable (was Can Python function return multiple data?)
In-Reply-To: <4fdc38b3-6b51-4f98-9ba3-42b265582fc0@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
 <mailman.212.1433569420.13271.python-list@python.org>
 <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>
 <4fdc38b3-6b51-4f98-9ba3-42b265582fc0@googlegroups.com>
Message-ID: <CAPTjJmoPozJGKD+f=+uwjrh+EzVBhq4x-xXx8DT6TY+nuhdFYg@mail.gmail.com>

On Mon, Jun 8, 2015 at 2:20 AM, Rustom Mody <rustompmody at gmail.com> wrote:
> Ok now rewrite that para above with
> s/tuple/numbers like 3 or 666/
> So I put '3' on the ram and grind it to finest powder.
> Have all trinities (of religious or secular variety) disappeared?
> 666 gone has the devil been banished from God's (or Steven's) universe?

If you write down your phone number and give it to a girl you like,
and she burns that piece of paper, do you no longer have a phone
number? No, but she certainly doesn't have your phone number any more.
There's a difference between destroying a representation and
destroying the original entity. But Steven's point wasn't about how
easy/hard it is to destroy something; it was about a tuple's
immutability NOT being a prevention of its destruction, and therefore
it's not that sense of the word in which they're immutable. Similarly,
you could destroy a document on which God's laws are written (in the
original dictionary sense; if you don't want to believe in immutable
laws of a Deity, you can substitute in a law of physics - for
instance, the law that two objects with mass exert a force of
attraction on each other due to gravity), and it wouldn't destroy the
law itself.

ChrisA


From PointedEars at web.de  Sun Jun  7 12:36:00 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 18:36 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <55743ea4$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <mailman.247.1433684166.13271.python-list@python.org>
Message-ID: <1586765.9ka6YhQpJC@PointedEars.de>

Peter Otten wrote:

> Steven D'Aprano wrote:
>>> I wrote a very simple function to test random:
>>>     def test_random(length, multiplier = 10000):
>>>         number_list = length * [0]
>>>         for i in range(length * multiplier):
>>>             number_list[random.randint(0, length - 1)] += 1
>>>         minimum = min(number_list)
>>>         maximum = max(number_list)
>>>         return (minimum, maximum, minimum / maximum)
>> 
>> Putting aside the timing aspects, your frequency calculations are not
>> done in a very Pythonic manner.
> 
> I would agree if the frequency table were sparse, i. e. many indices with
> 
> number_list[index] == 0
> 
> but that's not the case with on average 10000 hits per index.

A common misconception.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From PointedEars at web.de  Sun Jun  7 12:36:22 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 18:36:22 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
Message-ID: <1656356.OjxBvjpJ5d@PointedEars.de>

Chris Angelico wrote:

> On Mon, Jun 8, 2015 at 1:51 AM, Thomas 'PointedEars' Lahn
> <PointedEars at web.de> wrote:
>> Chris Angelico wrote:
>>
>>> On Sun, Jun 7, 2015 at 8:40 PM, Thomas 'PointedEars' Lahn
>>> <PointedEars at web.de> wrote:
>>>> Cecil Westerhof wrote:
>>>>> I wrote a very simple function to test random:
>>>>>     def test_random(length, multiplier = 10000):
>>>>>         number_list = length * [0]
>>>>>         for i in range(length * multiplier):
>>>>>             number_list[random.randint(0, length - 1)] += 1
>>>>>         minimum = min(number_list)
>>>>>         maximum = max(number_list)
>>>>>         return (minimum, maximum, minimum / maximum)
>>>>
>>>> As there is no guarantee that every number will occur randomly, using a
>>>> dictionary at first should be more efficient than a list:
>>>
>>> Hmm, I'm not sure that's actually so. His code is aiming to get
>>> 'multiplier' values in each box; for any serious multiplier (he starts
>>> with 10 in the main code), you can be fairly confident that every
>>> number will come up at least once.
>>
>> The wording shows a common misconception: that random distribution would
>> mean that it is guaranteed or more probable that every element of the set
>> will occur at least once.  It is another common misconception that
>> increasing the number of trials would increase the probability of that
>> happening.  But that is not so.
> 
> The greater the multiplier, the lower the chance that any element will
> have no hits.

Wrong.

> [ex falso quodlibet]

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From rosuav at gmail.com  Sun Jun  7 12:44:42 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 02:44:42 +1000
Subject: Testing random
In-Reply-To: <1656356.OjxBvjpJ5d@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
Message-ID: <CAPTjJmraTawCXS7fc_LwFiEfpdMAq6e6JK+qvognFRgXt7iXtQ@mail.gmail.com>

On Mon, Jun 8, 2015 at 2:36 AM, Thomas 'PointedEars' Lahn
<PointedEars at web.de> wrote:
>> The greater the multiplier, the lower the chance that any element will
>> have no hits.
>
> Wrong.
>
>> [ex falso quodlibet]

Huh. Do you want to explain how, mathematically, I am wrong, or do you
want to join the RUE in my ignore list?

ChrisA


From __peter__ at web.de  Sun Jun  7 12:48:01 2015
From: __peter__ at web.de (Peter Otten)
Date: Sun, 07 Jun 2015 18:48:01 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <55743ea4$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <mailman.247.1433684166.13271.python-list@python.org>
 <1586765.9ka6YhQpJC@PointedEars.de>
Message-ID: <ml1sk2$189$1@ger.gmane.org>

Thomas 'PointedEars' Lahn wrote:

> Peter Otten wrote:
> 
>> Steven D'Aprano wrote:
>>>> I wrote a very simple function to test random:
>>>>     def test_random(length, multiplier = 10000):
>>>>         number_list = length * [0]
>>>>         for i in range(length * multiplier):
>>>>             number_list[random.randint(0, length - 1)] += 1
>>>>         minimum = min(number_list)
>>>>         maximum = max(number_list)
>>>>         return (minimum, maximum, minimum / maximum)
>>> 
>>> Putting aside the timing aspects, your frequency calculations are not
>>> done in a very Pythonic manner.
>> 
>> I would agree if the frequency table were sparse, i. e. many indices with
>> 
>> number_list[index] == 0
>> 
>> but that's not the case with on average 10000 hits per index.
> 
> A common misconception.

A pointless remark. Not counting the ears ;) 




From ian.g.kelly at gmail.com  Sun Jun  7 13:07:03 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sun, 7 Jun 2015 11:07:03 -0600
Subject: Testing random
In-Reply-To: <CAPTjJmraTawCXS7fc_LwFiEfpdMAq6e6JK+qvognFRgXt7iXtQ@mail.gmail.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <CAPTjJmraTawCXS7fc_LwFiEfpdMAq6e6JK+qvognFRgXt7iXtQ@mail.gmail.com>
Message-ID: <CALwzidmB6hawAjtvaGNwP+DRDD=wVAdsZnAZ6wJRSVGfMzFkWA@mail.gmail.com>

On Sun, Jun 7, 2015 at 10:44 AM, Chris Angelico <rosuav at gmail.com> wrote:
> On Mon, Jun 8, 2015 at 2:36 AM, Thomas 'PointedEars' Lahn
> <PointedEars at web.de> wrote:
>>> The greater the multiplier, the lower the chance that any element will
>>> have no hits.
>>
>> Wrong.
>>
>>> [ex falso quodlibet]
>
> Huh. Do you want to explain how, mathematically, I am wrong, or do you
> want to join the RUE in my ignore list?

My best speculation is that he's either objecting to the generality of
your statement (it's false if the probability of some element
occurring is zero or eventually degrades to zero), or misreading the
word "multiplier" to the conclusion that the value of each element is
being multiplied rather than the number of trials. Or trolling; I
suppose that's always an option too.


From chris at cdreimer.com  Sun Jun  7 13:17:10 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sun, 07 Jun 2015 10:17:10 -0700
Subject: Python Random vs. Cython C Rand for Dice Rolls
Message-ID: <55747C96.7090006@cdreimer.com>

Greetings,

I've revisited my misbegotten childhood by translating the programs from 
"BASIC Computer Games" by David H. Ahl into Python. This is mostly an 
exercise in unraveling spaghetti code with all those GOTO statements 
going all over the place. The dice program caught my attention in 
particular for a stripped down version in Python and Cython to test the 
random number generator functionality.

http://www.atariarchives.org/basicgames/showpage.php?page=57

Here are my scripts to roll a pair of dice 50,000,000 times (this number 
creates a noticeable delay on my 3GHz quad processor). The Python script 
uses random, the Cython script uses C rand. Besides the obvious speed 
difference, the results are quite different.

This is the Python script that takes ~197 seconds to complete.

import random, time

startTime = time.time()

f = [0] * 12

for i in range(50000000):

     a = random.randint(1,6)

     b = random.randint(1,6)

     f[(a + b) - 1] += 1

print "\nTOTAL SPOTS","\tNUMBER OF TIMES\n"

for i in range(1,12):

     print ' ' + str(i + 1), '\t\t ', f[i]

print '\n', time.time() - startTime


This the Cython script that is imported into a test scripts similar to 
the Python script that takes ~1.6 seconds to complete.

cdef extern from "stdlib.h":

     int c_libc_rand "rand"()

def roll(int x):

     cdef:

         int a, b, i

         int f[12]

     for i in range(x):

         a = c_libc_rand() % 6 + 1

         b = c_libc_rand() % 6 + 1

         f[(a + b) - 1] += 1

     return f

Here's the console output.

PS Z:\projects\programming\python\basic_games\fastdice> python slowdice.py

TOTAL SPOTS     NUMBER OF TIMES

  2                1388086
  3                2776286
  4                4165556
  5                5555869
  6                6940547
  7                8335864
  8                6945446
  9                5556470
  10               4169549
  11               2777972
  12               1388355

197.006999969
PS Z:\projects\programming\python\basic_games\fastdice> python 
test_fastdice.py

TOTAL SPOTS     NUMBER OF TIMES

  2                1389911
  3                -2144697697
  4                4168249
  5                35008856
  6                6944907
  7                512318212
  8                6945597
  9                342017362
  10               4167485
  11               2775806
  12               1388465

1.63799977303

The Python random shows a uniform bell curve with low numbers at the 
ends and the peak in the middle, which is similar to the text in the 
book for the BASIC program. The Cython C rand is over all the place 
(especially with a negative number), which reminds me how bad the random 
number generator was on my 1MHz C64 in the day.

Is there something in the Cython code that I need to change and/or find 
a better C random number generator?

Thanks,

Chris R.


From jimages123 at gmail.com  Sun Jun  7 13:18:59 2015
From: jimages123 at gmail.com (Jimages)
Date: Mon, 08 Jun 2015 01:18:59 +0800
Subject: Is there a mail list like python-list in c?
Message-ID: <55747D03.5090805@gmail.com>

Hi!
i know that the question may not be suitable. But i am learning python 
as well as c. And i find the list valuable. After google mail list about 
c. i don't find even one mail list t about programming in C. So could 
you recommend a mail list about programming in c?
BTW excuse my bad English.
Thanks.


From rosuav at gmail.com  Sun Jun  7 13:20:14 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 03:20:14 +1000
Subject: Testing random
In-Reply-To: <CALwzidmB6hawAjtvaGNwP+DRDD=wVAdsZnAZ6wJRSVGfMzFkWA@mail.gmail.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <CAPTjJmraTawCXS7fc_LwFiEfpdMAq6e6JK+qvognFRgXt7iXtQ@mail.gmail.com>
 <CALwzidmB6hawAjtvaGNwP+DRDD=wVAdsZnAZ6wJRSVGfMzFkWA@mail.gmail.com>
Message-ID: <CAPTjJmpFb4B-egOeJ9Qwghj4LiNBUL67tmzYjQO3bGqB7i0Njg@mail.gmail.com>

On Mon, Jun 8, 2015 at 3:07 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Sun, Jun 7, 2015 at 10:44 AM, Chris Angelico <rosuav at gmail.com> wrote:
>> On Mon, Jun 8, 2015 at 2:36 AM, Thomas 'PointedEars' Lahn
>> <PointedEars at web.de> wrote:
>>>> The greater the multiplier, the lower the chance that any element will
>>>> have no hits.
>>>
>>> Wrong.
>>>
>>>> [ex falso quodlibet]
>>
>> Huh. Do you want to explain how, mathematically, I am wrong, or do you
>> want to join the RUE in my ignore list?
>
> My best speculation is that he's either objecting to the generality of
> your statement (it's false if the probability of some element
> occurring is zero or eventually degrades to zero), or misreading the
> word "multiplier" to the conclusion that the value of each element is
> being multiplied rather than the number of trials. Or trolling; I
> suppose that's always an option too.

My first thought was your first option, which is why I specifically
responded with explanation about how data type selection can viably be
about expectations rather than certainties, but he responded with a
one-word "Wrong" so I must have been answering the wrong question.
I've no idea about the misreading of "multiplier". A fourth
possibility is that mathematics works differently for him and for us,
which I suppose is possible; when I visited sci.math a while ago, I
found some people for whom everything I'd learned in grade school was
clearly wrong, and they were doing their best to enlighten the world
about the new truths of mathematics that they'd found.

ChrisA


From rosuav at gmail.com  Sun Jun  7 13:23:24 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 03:23:24 +1000
Subject: Python Random vs. Cython C Rand for Dice Rolls
In-Reply-To: <55747C96.7090006@cdreimer.com>
References: <55747C96.7090006@cdreimer.com>
Message-ID: <CAPTjJmqq7LZPusE12mjrvywAZ3n=zo3Af28wtUkEVTz3Pga_7Q@mail.gmail.com>

On Mon, Jun 8, 2015 at 3:17 AM, C.D. Reimer <chris at cdreimer.com> wrote:
> This is the Python script that takes ~197 seconds to complete.
>
> import random, time
>
> startTime = time.time()
>
> f = [0] * 12
>
> for i in range(50000000):
>
>     a = random.randint(1,6)
>
>     b = random.randint(1,6)
>
>     f[(a + b) - 1] += 1
>
> print "\nTOTAL SPOTS","\tNUMBER OF TIMES\n"
>
> for i in range(1,12):
>
>     print ' ' + str(i + 1), '\t\t ', f[i]
>
> print '\n', time.time() - startTime

Before you go any further, can you just try this script, please, and
see how long it takes to run?

import random, time
startTime = time.time()
for i in range(50000000):
    pass
print '\n', time.time() - startTime

I know, seems a stupid thing to try, right? But you're using Python 2,
as evidenced by the print statements, and that means that range() is
constructing a 50M element list. It's entirely possible that that's a
significant part of your time - allocating all that memory, populating
it, and then disposing of it at the end (maybe).

Maybe it'll turn out not to be significant, but there's only one way
to find out.

ChrisA


From rosuav at gmail.com  Sun Jun  7 13:33:00 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 03:33:00 +1000
Subject: Python Random vs. Cython C Rand for Dice Rolls
In-Reply-To: <55747C96.7090006@cdreimer.com>
References: <55747C96.7090006@cdreimer.com>
Message-ID: <CAPTjJmo0Aw961q54BySZBSDdD8xqCPFjMezcytzWxrE0m6afTw@mail.gmail.com>

On Mon, Jun 8, 2015 at 3:17 AM, C.D. Reimer <chris at cdreimer.com> wrote:
> The Python random shows a uniform bell curve with low numbers at the ends
> and the peak in the middle, which is similar to the text in the book for the
> BASIC program. The Cython C rand is over all the place (especially with a
> negative number), which reminds me how bad the random number generator was
> on my 1MHz C64 in the day.

The negative result is a strong indicator that you're not seeing the
results of rand() here. While there is a potential for bias (check out
RAND_MAX, and consider that there may be some small bias there;
although on most modern systems, RAND_MAX is going to be high enough
that the bias from modulo 6 won't be highly visible), it's much MUCH
more likely that you're looking at uninitialized memory. I'm not
overly familiar with Cython, but can you simply add "={0}" to the
declaration to force it to be initialized to all zeroes? That's what
I'd do in raw C. Here's the same thing done in straight-forward C:

#include <stdlib.h>
#include <stdio.h>

void roll(int *f,int x)
{
    int i, a, b;
    for (i=0; i<x; ++i)
    {
        a = rand() % 6 + 1;
        b = rand() % 6 + 1;
        f[(a + b) - 1] += 1;
    }
}

int main()
{
    int f[12];
    int i;
    roll(f, 50000000);
    for (i=1;i<12;++i)
        printf("%-15d %d\n", i+1, f[i]);
    return 0;
}

rosuav at sikorsky:~$ time ./a.out
2               1387782
3               6972536
4               4170215
5               21326281
6               6945600
7               8334121
8               6943983
9               9750611
10              4168554
11              6972254
12              1389371

real 0m1.089s
user 0m1.088s
sys 0m0.000s

Similar weird results to yours. Replacing "int f[12];" with "int
f[12]={0};" produces a much more plausible bell curve:

rosuav at sikorsky:~$ time ./a.out
2               1387782
3               2776523
4               4170215
5               5552330
6               6945600
7               8334121
8               6943983
9               5554675
10              4168554
11              2776846
12              1389371

real 0m1.093s
user 0m1.088s
sys 0m0.000s

But I don't know Cython, so I don't know whether this would work as-is
or not. Worst case, toss in a quick initialize-to-zero loop above your
main loop, and you can at least verify correctness.

ChrisA


From chris at cdreimer.com  Sun Jun  7 13:36:31 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sun, 07 Jun 2015 10:36:31 -0700
Subject: Testing random
In-Reply-To: <CAPTjJmpFb4B-egOeJ9Qwghj4LiNBUL67tmzYjQO3bGqB7i0Njg@mail.gmail.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <CAPTjJmraTawCXS7fc_LwFiEfpdMAq6e6JK+qvognFRgXt7iXtQ@mail.gmail.com>
 <CALwzidmB6hawAjtvaGNwP+DRDD=wVAdsZnAZ6wJRSVGfMzFkWA@mail.gmail.com>
 <CAPTjJmpFb4B-egOeJ9Qwghj4LiNBUL67tmzYjQO3bGqB7i0Njg@mail.gmail.com>
Message-ID: <5574811F.1030500@cdreimer.com>

On 6/7/2015 10:20 AM, Chris Angelico wrote:
> A fourth
> possibility is that mathematics works differently for him and for us,
> which I suppose is possible; when I visited sci.math a while ago, I
> found some people for whom everything I'd learned in grade school was
> clearly wrong, and they were doing their best to enlighten the world
> about the new truths of mathematics that they'd found.

I had the unfortunate luck of taking "Harvard Calculus" in college 
(circa 1995). The textbook was nothing but word problems from end to 
end. The goal was to get the students away from symbolic thinking of 
traditional calculus into thinking about real world problems that uses 
calculus. (Never mind that the majority of students don't use calculus 
after they get out school.) I bailed out after two weeks. Taking the 
class at 7:30AM probably didn't help.

Chris R.


From chris at cdreimer.com  Sun Jun  7 13:59:53 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sun, 07 Jun 2015 10:59:53 -0700
Subject: Python Random vs. Cython C Rand for Dice Rolls
In-Reply-To: <CAPTjJmqq7LZPusE12mjrvywAZ3n=zo3Af28wtUkEVTz3Pga_7Q@mail.gmail.com>
References: <55747C96.7090006@cdreimer.com>
 <CAPTjJmqq7LZPusE12mjrvywAZ3n=zo3Af28wtUkEVTz3Pga_7Q@mail.gmail.com>
Message-ID: <55748699.1070009@cdreimer.com>

On 6/7/2015 10:23 AM, Chris Angelico wrote:
> Before you go any further, can you just try this script, please, and
> see how long it takes to run?
>
> import random, time
> startTime = time.time()
> for i in range(50000000):
>      pass
> print '\n', time.time() - startTime
>
> I know, seems a stupid thing to try, right? But you're using Python 2,
> as evidenced by the print statements, and that means that range() is
> constructing a 50M element list. It's entirely possible that that's a
> significant part of your time - allocating all that memory, populating
> it, and then disposing of it at the end (maybe).

PS Z:\projects\programming\python\basic_games\fastdice> python test_random_time.py

4.84700012207


So... I'm not looking at the problem on the Python side in the correct way?

In particular, I'm using the wrong container type?

I was thinking "array" (like the Cython version) when I put the Python 
script together. I got an error message at one point about indexing the 
"list" that threw me off for a while. Since it looks like an array, 
walks like an array, and quack likes array, I fixed the indexing problem 
like an array. A list != array? :)

Chris R.


From rosuav at gmail.com  Sun Jun  7 14:12:58 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 04:12:58 +1000
Subject: Python Random vs. Cython C Rand for Dice Rolls
In-Reply-To: <55748699.1070009@cdreimer.com>
References: <55747C96.7090006@cdreimer.com>
 <CAPTjJmqq7LZPusE12mjrvywAZ3n=zo3Af28wtUkEVTz3Pga_7Q@mail.gmail.com>
 <55748699.1070009@cdreimer.com>
Message-ID: <CAPTjJmoTJNYsB7BkHAPDsXmV2vUwQEBC8kOuzUxg_DphL833qw@mail.gmail.com>

On Mon, Jun 8, 2015 at 3:59 AM, C.D. Reimer <chris at cdreimer.com> wrote:
> On 6/7/2015 10:23 AM, Chris Angelico wrote:
>>
>> Before you go any further, can you just try this script, please, and
>> see how long it takes to run?
>>
>> import random, time
>> startTime = time.time()
>> for i in range(50000000):
>>      pass
>> print '\n', time.time() - startTime
>>
>> I know, seems a stupid thing to try, right? But you're using Python 2,
>> as evidenced by the print statements, and that means that range() is
>> constructing a 50M element list. It's entirely possible that that's a
>> significant part of your time - allocating all that memory, populating
>> it, and then disposing of it at the end (maybe).
>
>
> PS Z:\projects\programming\python\basic_games\fastdice> python
> test_random_time.py
>
> 4.84700012207
>
>
> So... I'm not looking at the problem on the Python side in the correct way?
>
> In particular, I'm using the wrong container type?

The only thing I was looking at here was range() vs xrange(). Try
using xrange here, and see how much faster your null loop is, and then
make the corresponding change to your real program and see if it's
equally faster. But since your main routine was taking ~200 seconds
and the null took ~5, it's not as big a deal as I thought; and a good
slab of that time will be spent on the null iteration, rather than the
list construction. (In fact, it's entirely possible you have a site.py
or something that goes "range = xrange".)

> I was thinking "array" (like the Cython version) when I put the Python
> script together. I got an error message at one point about indexing the
> "list" that threw me off for a while. Since it looks like an array, walks
> like an array, and quack likes array, I fixed the indexing problem like an
> array. A list != array? :)

The main difference between lists and arrays is that lists can grow
and shrink, which you're not using here. So I'd expect that, for the
purposes of this program, they're in the same space. You can use a
Python list the way you'd use a C array, generally. They can just do
ever so much more.

ChrisA


From PointedEars at web.de  Sun Jun  7 14:23:10 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 20:23:10 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
Message-ID: <5515034.V7dcXEWAvK@PointedEars.de>

Chris Angelico wrote:

> On Mon, Jun 8, 2015 at 2:36 AM, Thomas 'PointedEars' Lahn
> <PointedEars at web.de> wrote:
>>> The greater the multiplier, the lower the chance that any element will
>>> have no hits.
>> Wrong.
>>
>>> [ex falso quodlibet]
> 
> Huh. Do you want to explain how, mathematically, I am wrong, or do you
> want to join the RUE in my ignore list?

I already did; you have overlooked it.  In a nutshell, the probability of 

  1 1 1 1 1 1 1 1 1 1 1

is *the same* as that of

  1 2 3 4 5 6 7 8 9 1 2

and the same as that of

  8 3 6 3 1 2 6 8 2 1 6.

If the set to choose from is integer numbers from 1 to 9, then *each* of 
those sequences has *the same* probability (1?9)?? ? 3.1866355 ? 10???.
Repeating that random experiment one more time, the probability of

  1 1 1 1 1 1 1 1 1 1 1 2

is the same as of

  1 2 3 4 5 6 7 8 9 1 2 2

and the same as that of

  8 3 6 3 1 2 6 8 2 1 6 2

(that the next outcome is a 2): (1?9)?? ? 3.5407062 ? 10???.  And the 
probability of

  1 1 1 1 1 1 1 1 1 1 1 1

(only 1s) is the *same* as of

  1 1 1 1 1 1 1 1 1 1 1 2

(only 1s and one 2) and the same as of

  1 1 1 1 1 1 1 1 1 1 1 3

(only 1s and a 3) and of

  8 3 6 3 1 2 6 8 2 1 6 4

(that the next outcome is a 4, and there are no 5s at all).

AISB, those are *independent* events; the number of occurrences of an 
outcome before *does not matter* for the probability of the next one.  And 
so the probability of getting a certain number does _not_ change depending 
on the number of times you repeat the experiment.  It is always the same;
in this example, it is always 1?9.  And the probability of _not_ getting a 
certain number is always the same; in this example, it is always 1 ? 1?9 = 
8?9.


So, I am sorry to tell you this, but you do _not_ understand probability.  
And you *cannot* understand it intuitively, like you tried to.  Probability 
is counter-intuitive.

That should not surprise or worry you.  Because as I and others have pointed 
out in this thread, the majority of the human population, including the OP, 
does not understand probability ? until it has been explained to them.  I am 
not even sure I understand probability in every instance, or rather, I am 
sure that *intuitively* I do not.

It is why humans play the lottery using their ?lucky numbers? whereas other 
numbers have exactly the same probability of being drawn; why they think a  
baby boy ?is due? when they have two girls already; why they are surprised 
if two people in the room have the same birthday; why *in the short run* you 
can make a fortune by running a casino even if the tables are not rigged; 
why the ?gate three? trick works (for a while); why they pay money for 
horoscopes and truthsayers, not recognizing that their wording is so 
ambiguous that it is highly probable to apply to everyone, and why many of 
them believe in a deity or fate.

A big part of the reason is psychology, how the human mind works: The human 
mind has evolved to recognize patterns.  It is an evolutionary advantage to 
be able to recognize patterns, to tell a mammoth from a sabre-stooth lion at 
a distance, a poisonous from an edible plant, and an X from a U.  So humans 
try to see patterns everywhere, even in pure randomness: clouds, texts, 
numbers, you name it.  And when they are told that something is random, and 
they find patterns regardless, they do not believe that it is random.

An extension of that misconception is emphasized by an anecdote (which may 
be apocryphal) told about Richard Feynman (I heard it from Lawrence Krauss 
in ?A Universe from Nothing?; he can tell that in a much more funny way than 
I am able to reproduce it here [1]):

  Richard Feynman used to go up to people all the time and he?d say:

    ?You won?t believe what happend to me today!
     You won?t believe what happend to me today!?

  And people would say:

    ?What??

  And he would reply:

    ?Absolutely nothing!?

Because humans are evolutionary driven to look for patterns, and they 
believe that everything that happens to them is important and has 
significance in ?the greater scheme of things? (whose mere existence
is a misconception, too).

HTH

________
[1] Lawrence KRAUSS (2009). ?A Universe from Nothing?. The Mariott hotel,
    Burbank. 1:03:12.  <https://youtu.be/-EilZ4VY5Vs?t=3792>
-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From steve at pearwood.info  Sun Jun  7 14:24:14 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 08 Jun 2015 04:24:14 +1000
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
Message-ID: <55748c4f$0$13012$c3e8da3$5496439d@news.astraweb.com>

On Mon, 8 Jun 2015 01:51 am, Thomas 'PointedEars' Lahn wrote:

> Chris Angelico wrote:
> 
>> On Sun, Jun 7, 2015 at 8:40 PM, Thomas 'PointedEars' Lahn
>> <PointedEars at web.de> wrote:
>>> Cecil Westerhof wrote:
>>>> I wrote a very simple function to test random:
>>>>     def test_random(length, multiplier = 10000):
>>>>         number_list = length * [0]
>>>>         for i in range(length * multiplier):
>>>>             number_list[random.randint(0, length - 1)] += 1
>>>>         minimum = min(number_list)
>>>>         maximum = max(number_list)
>>>>         return (minimum, maximum, minimum / maximum)
>>>
>>> As there is no guarantee that every number will occur randomly, using a
>>> dictionary at first should be more efficient than a list:
>> 
>> Hmm, I'm not sure that's actually so. His code is aiming to get
>> 'multiplier' values in each box; for any serious multiplier (he starts
>> with 10 in the main code), you can be fairly confident that every
>> number will come up at least once.
> 
> The wording shows a common misconception: that random distribution would
> mean that it is guaranteed or more probable that every element of the set
> will occur at least once.  It is another common misconception that
> increasing the number of trials would increase the probability of that
> happening.  But that is not so.

You are badly mistaken.

Of course you are correct that a random distribution does not *guarantee*
that every element of the set is seen at least once. But as soon as you
call it "a common misconception" that increasing the number of trials
increases the probability of seeing every element, you are simply mistaken.
It's not a misconception at all, it is true. See below for calculations.


[...]
> If the distribution is even and the game is fair, every outcome *always*
> has
> the same probability of occurring.  As a result, every sequence of
> outcomes of equal length *always* has the same probability of occurring,
> and the probability for a particular sequence of equal length does _not_
> increase or
> decrease based on the number of occurrences of previous outcomes.  Those
> are *independent* events.

You are confused. Think about it more carefully.

To make it simple, let's randomly generate from the set 1, 2, 3, and do just
three samples. That's small enough to enumerate all the possibilities:

111 112 113 121 122 123 131 132 133
211 212 213 221 222 223 231 232 233
311 312 313 321 322 323 331 332 333

Each event (111 etc) has a probability of 1/27, but that's not important.
We're comparing the probability of all elements being seen, versus at least
one element being missed. I'm going to tag the events that contain all
three elements:

111 112 113 121 122 123* 131 132* 133
211 212 213* 221 222 223 231* 232 233
311 312* 313 321* 322 323 331 332 333

So there are six events out of 27, or 2/9, where all elements are seen in a
trial, versus 7/9 where at least one element will have a count of zero.

Putting it another way:

Pr(at least one element has zero count)
= Pr(1 has zero count) + Pr(2 has zero count) + Pr(3 has zero count)
  - Pr(1 and 2 both have zero counts) 
  - Pr(1 and 3 both have zero counts) 
  - Pr(2 and 3 both have zero counts)

= (2/3)**3 * 3 - (1/3)**3 * 3

which gives us 7/9 as confirmed by direct enumeration of all the
possibilities.

With four samples per trial:
Pr(at least one element has zero count) = 5/9

With five samples:
Pr(at least one element has zero count) = 31/81

Ten samples:
Pr(at least one element has zero count) = 341/6561 
or approximately 0.05

Twenty samples:
Pr(at least one element has zero count) = 349525/387420489
or approximately 0.0009

For 100 samples:
Pr(at least one element has zero count) = 7.4e-18 (approx)


which means that with 100 samples from the set (1, 2, 3), if you could run
one million trials every second, it would on average take you almost 4300
years to see a trial that had a zero count for one or more of the elements.

The probability of any element *not* showing up in a random trial with N
samples decreases rapidly as N increases. Elements with zero count are only
a factor when the number of samples is small, or the number of elements is
large.




-- 
Steven



From steve at pearwood.info  Sun Jun  7 14:28:46 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 08 Jun 2015 04:28:46 +1000
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
Message-ID: <55748d5f$0$13008$c3e8da3$5496439d@news.astraweb.com>

On Mon, 8 Jun 2015 02:36 am, Thomas 'PointedEars' Lahn wrote:

> Chris Angelico wrote:

>> The greater the multiplier, the lower the chance that any element will
>> have no hits.
> 
> Wrong.

No, Chris is correct. The "multiplier" increases the number of samples. The
larger the number of samples, the lower the chance that any element is not
represented.

If my previous post didn't convince you, consider an even simpler random
distribution: tossing a fair coin. The probability of getting a head is
exactly 1/2 whether you toss the coin once or a thousand times. But if you
toss the coin once, your chances of getting "no heads" is 1/2. If you toss
it a thousand times, your chances of getting "no heads" is 1/2**1000.



-- 
Steven



From steve at pearwood.info  Sun Jun  7 14:33:16 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 08 Jun 2015 04:33:16 +1000
Subject: Python Random vs. Cython C Rand for Dice Rolls
References: <mailman.262.1433697446.13271.python-list@python.org>
Message-ID: <55748e6d$0$13008$c3e8da3$5496439d@news.astraweb.com>

On Mon, 8 Jun 2015 03:17 am, C.D. Reimer wrote:

> Greetings,
> 
> I've revisited my misbegotten childhood by translating the programs from
> "BASIC Computer Games" by David H. Ahl into Python. This is mostly an
> exercise in unraveling spaghetti code with all those GOTO statements
> going all over the place. The dice program caught my attention in
> particular for a stripped down version in Python and Cython to test the
> random number generator functionality.
> 
> http://www.atariarchives.org/basicgames/showpage.php?page=57
> 
> Here are my scripts to roll a pair of dice 50,000,000 times (this number
> creates a noticeable delay on my 3GHz quad processor). The Python script
> uses random, the Cython script uses C rand. Besides the obvious speed
> difference, the results are quite different.

C rand is not even close to random. The technical term for it is "shite".


-- 
Steve



From rosuav at gmail.com  Sun Jun  7 14:40:30 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 04:40:30 +1000
Subject: Testing random
In-Reply-To: <55748d5f$0$13008$c3e8da3$5496439d@news.astraweb.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <55748d5f$0$13008$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmrwscQWWPMV_UK_Xj8Pjt2GGNqyUFMTwE0JCqXx49cu=A@mail.gmail.com>

On Mon, Jun 8, 2015 at 4:28 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> If my previous post didn't convince you, consider an even simpler random
> distribution: tossing a fair coin. The probability of getting a head is
> exactly 1/2 whether you toss the coin once or a thousand times. But if you
> toss the coin once, your chances of getting "no heads" is 1/2. If you toss
> it a thousand times, your chances of getting "no heads" is 1/2**1000.

Although... purity and reality do sometimes disagree. The concept of
"independent events" says that if I toss a coin 99 times and it comes
up heads every time, the probability that it'll come up heads again
the hundredth time is still 50-50. But the reality of cheats coins
says that the chances of the coin being a fair one and having been
tossed 99 times and come up heads every time is one in 1<<99, and
given that the mass of this planet is roughly 6E27 grams, that would
mean that you'd need about a thousand earths' worth of coins to get
that many... so the chances are much better that you're looking at
either an imbalanced coin or a crafty operator, and the probability is
much higher that it comes up heads :)

ChrisA


From chris at cdreimer.com  Sun Jun  7 14:40:37 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sun, 07 Jun 2015 11:40:37 -0700
Subject: Python Random vs. Cython C Rand for Dice Rolls
In-Reply-To: <CAPTjJmo0Aw961q54BySZBSDdD8xqCPFjMezcytzWxrE0m6afTw@mail.gmail.com>
References: <55747C96.7090006@cdreimer.com>
 <CAPTjJmo0Aw961q54BySZBSDdD8xqCPFjMezcytzWxrE0m6afTw@mail.gmail.com>
Message-ID: <55749025.8090501@cdreimer.com>

On 6/7/2015 10:33 AM, Chris Angelico wrote:
> The negative result is a strong indicator that you're not seeing the
> results of rand() here. While there is a potential for bias (check out
> RAND_MAX, and consider that there may be some small bias there;
> although on most modern systems, RAND_MAX is going to be high enough
> that the bias from modulo 6 won't be highly visible), it's much MUCH
> more likely that you're looking at uninitialized memory.

Here's the revised Cython code.

cdef extern from "stdlib.h":

     int c_libc_rand "rand"()

cdef int f[12]

     

def roll(int x):

     cdef:

         int a = 0, b = 0, i = 0

     for i in range(x):

         a = c_libc_rand() % 6 + 1

         b = c_libc_rand() % 6 + 1
         

         f[(a + b) - 1] += 1

     return f


Here's the new console output.

PS Z:\projects\programming\python\basic_games\fastdice> python test_fastdice.py

TOTAL SPOTS     NUMBER OF TIMES

  2                1389911

  3                2777722

  4                4168248

  5                5553632

  6                6944907

  7                8334670

  8                6945597

  9                5553557

  10               4167485

  11               2775806

  12               1388465

1.65599989891


I had to put the array definition outside of the function declaration, 
where it's automatically initialized to zero in the global space. I 
tried to initialize the array from inside the function but Cython threw 
up errors, either the C declaration wasn't correct or unable to convert 
from C to Python when updating the array elements. I'll figure out that 
problem later. The new version works as expected.

Thanks,

Chris R.


From dfnsonfsduifb at gmx.de  Sun Jun  7 14:51:20 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sun, 07 Jun 2015 20:51:20 +0200
Subject: Function to show time to execute another function
In-Reply-To: <87fv64orly.fsf@Equus.decebal.nl>
References: <87k2vgowco.fsf@Equus.decebal.nl>	<mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl>
Message-ID: <ml23r8$htm$1@news.albasani.net>

On 07.06.2015 10:22, Cecil Westerhof wrote:

> That only times the function. I explicitly mentioned I want both the
> needed time AND the output.

And you also posted your solution. I fail to find any question in your
original posting at all.

> Sadly the quality of the answers on this list is going down. 

Maybe you should start asking questions that people are able to
comprehend so that you get an answer that you like.

> Here I
> get an alternative that does only half what I want and when writing an
> alternative for ?!find? I am told I could use ?!find? (which only
> works in ipython, not python and which also not works with Windows).

Protip: Ditch the shitty attitude. I don't know if you're a jerk or not
but I know for sure that you sound like one. Makes it also much less
likely to get the answers you'd like.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From rosuav at gmail.com  Sun Jun  7 14:52:06 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 04:52:06 +1000
Subject: Testing random
In-Reply-To: <5515034.V7dcXEWAvK@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
Message-ID: <CAPTjJmrDVV40=oW1du-7MHhScSRdUV=awOaGOmU52Ov2PCti2g@mail.gmail.com>

On Mon, Jun 8, 2015 at 4:23 AM, Thomas 'PointedEars' Lahn
<PointedEars at web.de> wrote:
> If the set to choose from is integer numbers from 1 to 9, then *each* of
> those sequences has *the same* probability (1?9)?? ? 3.1866355 ? 10???.
>
> AISB, those are *independent* events; the number of occurrences of an
> outcome before *does not matter* for the probability of the next one.  And
> so the probability of getting a certain number does _not_ change depending
> on the number of times you repeat the experiment.  It is always the same;
> in this example, it is always 1?9.  And the probability of _not_ getting a
> certain number is always the same; in this example, it is always 1 ? 1?9 =
> 8?9.

Yes, the probability of not getting a certain number is always 8/9...
but the probability of not getting that number for an entire sequence
is 8/9 raised to the power of the length of the sequence, because they
are independent events that must *all* happen. In your case, with a
length of 11, that means there's about a 27% chance of any particular
number not happening. Looking at it the other way, that means there's
a 73% chance of seeing at least one N, for any N. So the probability
of seeing at least one of every digit is actually quite low - about
5.6% - because you're looking for nine digits out of eleven. (Note
that the numbers aren't quite exact here. The probabilities aren't
entirely independent. Calculating off the exact figures is left as an
exercise for the reader, but I don't expect they'll be more than a
percentage point different one way or the other.)

The chances of not seeing a 5 in any of 100 tosses would be virtually
nil. The chances of not seeing a 5 in any of 1000 tosses is so close
to zero that its inverse is indistinguishable from certainty with
Python floats (IEEE double precision):

>>> 1.0-(9/10)**1000 == 1.0
True

That's a pretty low number.

ChrisA


From rosuav at gmail.com  Sun Jun  7 14:53:24 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 04:53:24 +1000
Subject: Python Random vs. Cython C Rand for Dice Rolls
In-Reply-To: <55749025.8090501@cdreimer.com>
References: <55747C96.7090006@cdreimer.com>
 <CAPTjJmo0Aw961q54BySZBSDdD8xqCPFjMezcytzWxrE0m6afTw@mail.gmail.com>
 <55749025.8090501@cdreimer.com>
Message-ID: <CAPTjJmqf6m49Kvh6Rq8ZsyFrdsrBL3Gg29O3NkhtN6-Wq3rdoQ@mail.gmail.com>

On Mon, Jun 8, 2015 at 4:40 AM, C.D. Reimer <chris at cdreimer.com> wrote:
> PS Z:\projects\programming\python\basic_games\fastdice> python
> test_fastdice.py
>
> TOTAL SPOTS     NUMBER OF TIMES
>
>  2                1389911
>
>  3                2777722
>
>  4                4168248
>
>  5                5553632
>
>  6                6944907
>
>  7                8334670
>
>  8                6945597
>
>  9                5553557
>
>  10               4167485
>
>  11               2775806
>
>  12               1388465
>
> 1.65599989891
>
>
> I had to put the array definition outside of the function declaration, where
> it's automatically initialized to zero in the global space.

Yep, that's one valid way to force them all to zero (per C spec). And
now you're seeing reasonable values. The rand() function may not be
all that great, but it certainly isn't responsible for the bizarre
values you were seeing :)

ChrisA


From chris at cdreimer.com  Sun Jun  7 14:56:16 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sun, 07 Jun 2015 11:56:16 -0700
Subject: Python Random vs. Cython C Rand for Dice Rolls
In-Reply-To: <55748e6d$0$13008$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.262.1433697446.13271.python-list@python.org>
 <55748e6d$0$13008$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <557493D0.7090805@cdreimer.com>

On 6/7/2015 11:33 AM, Steven D'Aprano wrote:
> C rand is not even close to random. The technical term for it is "shite".

Looking through the BASIC book, I remembered all the tricks needed to 
get a half-way decent number generator on a 1MHz processor back in the 
day. Either the numbers start repeating after a while or get stuck on a 
particular number. That's one of the challenges of converting these old 
games from BASIC to Python. A lot of the stuff done in BASIC was meant 
to get around the hardware limitations of the early computers.

Chris R.


From jpiitula at ling.helsinki.fi  Sun Jun  7 15:08:46 2015
From: jpiitula at ling.helsinki.fi (Jussi Piitulainen)
Date: Sun, 07 Jun 2015 22:08:46 +0300
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
Message-ID: <lf5sia3bakh.fsf@ling.helsinki.fi>

Thomas 'PointedEars' Lahn writes:
> Chris Angelico wrote:
>
>> Huh. Do you want to explain how, mathematically, I am wrong, or do
>> you want to join the RUE in my ignore list?
>
> I already did; you have overlooked it.  In a nutshell, the probability
> of
>
>   1 1 1 1 1 1 1 1 1 1 1

There is one way to get those numbers in some order.

(11! / 11! = 1)

> is *the same* as that of
>
>   1 2 3 4 5 6 7 8 9 1 2

There are almost ten million ways to get those numbers in some order.

(11! / 2! / 2! = 9979200)

> and the same as that of
>
>   8 3 6 3 1 2 6 8 2 1 6.

There are more than four hundred thousand ways to get those numbers in
some order.

(11! / 2! / 2! / 2! / 3! / 2! = 415800)


From ericgorr at gmail.com  Sun Jun  7 15:13:33 2015
From: ericgorr at gmail.com (Eric)
Date: Sun, 7 Jun 2015 12:13:33 -0700 (PDT)
Subject: Built-in Python3 web server functionality - wsgiref & http.server
Message-ID: <4f0edbe7-c634-4c26-8e36-0f32c4238447@googlegroups.com>

I am trying to better understand the built-in python3 web server functionality and have just started reading the documentation.

There seem to be two options provided by Python 3.

1. 
wsgiref
https://docs.python.org/3/library/wsgiref.html

2.
http.server
https://docs.python.org/3/library/http.server.html


Both options appear to offer a simple way to get a simple web server up and running. It does appear that wsgiref is built on top of http.server. Is this correct? 

Would it be correct to say that wsgiref is a high(er?)-level framework for designing web-based applications where as http.server is operating at a lower level? While one could do everything with just http.server, if wsgiref was appropriate for a particular use case, it would be the path to follow.

Any insights into this topic would be appreciated.

Are there any good python 3 books covering this functionality?






From PointedEars at web.de  Sun Jun  7 15:29:37 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 21:29:37 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
Message-ID: <1583276.2lplL8rY5W@PointedEars.de>

Jussi Piitulainen wrote:

> Thomas 'PointedEars' Lahn writes:
>>   8 3 6 3 1 2 6 8 2 1 6.
> 
> There are more than four hundred thousand ways to get those numbers in
> some order.
> 
> (11! / 2! / 2! / 2! / 3! / 2! = 415800)

Fallacy.  Order is irrelevant here.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From Cecil at decebal.nl  Sun Jun  7 15:34:03 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 21:34:03 +0200
Subject: Python Random vs. Cython C Rand for Dice Rolls
References: <55747C96.7090006@cdreimer.com>
 <mailman.264.1433697806.13271.python-list@python.org>
Message-ID: <87d217nwic.fsf@Equus.decebal.nl>

On Sunday  7 Jun 2015 19:23 CEST, Chris Angelico wrote:

> On Mon, Jun 8, 2015 at 3:17 AM, C.D. Reimer <chris at cdreimer.com> wrote:
>> This is the Python script that takes ~197 seconds to complete.
>>
>> import random, time
>>
>> startTime = time.time()
>>
>> f = [0] * 12
>>
>> for i in range(50000000):
>>
>> a = random.randint(1,6)
>>
>> b = random.randint(1,6)
>>
>> f[(a + b) - 1] += 1
>>
>> print "\nTOTAL SPOTS","\tNUMBER OF TIMES\n"
>>
>> for i in range(1,12):
>>
>> print ' ' + str(i + 1), '\t\t ', f[i]
>>
>> print '\n', time.time() - startTime
>
> Before you go any further, can you just try this script, please, and
> see how long it takes to run?
>
> import random, time
> startTime = time.time()
> for i in range(50000000):
> pass
> print '\n', time.time() - startTime
>
> I know, seems a stupid thing to try, right? But you're using Python
> 2, as evidenced by the print statements, and that means that range()
> is constructing a 50M element list. It's entirely possible that
> that's a significant part of your time - allocating all that memory,
> populating it, and then disposing of it at the end (maybe).
>
> Maybe it'll turn out not to be significant, but there's only one way
> to find out.

I tried this in 2 and 3. In 3 it takes 3.13 seconds and 2 7.5 seconds.
A significant difference, but not in the whole of what he was trying
to do I think.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From PointedEars at web.de  Sun Jun  7 15:41:51 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 21:41:51 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <mailman.271.1433703133.13271.python-list@python.org>
Message-ID: <3006491.U2VKTQuU5p@PointedEars.de>

Chris Angelico wrote:

> On Mon, Jun 8, 2015 at 4:23 AM, Thomas 'PointedEars' Lahn
> <PointedEars at web.de> wrote:
>> If the set to choose from is integer numbers from 1 to 9, then *each* of
>> those sequences has *the same* probability (1?9)?? ? 3.1866355 ? 10???.
>>
>> AISB, those are *independent* events; the number of occurrences of an
>> outcome before *does not matter* for the probability of the next one. 
>> And so the probability of getting a certain number does _not_ change
>> depending on the number of times you repeat the experiment.  It is always 
>> the same; in this example, it is always 1?9.  And the probability of 
>> _not_ getting  certain number is always the same; in this example, it is 
>> always 1 ? 1?9 = 8?9.
> 
> Yes, the probability of not getting a certain number is always 8/9...
> but the probability of not getting that number for an entire sequence
> is 8/9 raised to the power of the length of the sequence

As it is for getting every number at least once, or only two, three, four, 
five, six, seven, or eight of them.  Each number has the probability of 1?9 
of being drawn, and for a sequence of n numbers drawn, the probability is 
(1?9)?, *no matter* which numbers are in it.

> because they are independent events that must *all* happen. [?]

As I already pointed out, your reasoning is purely *intuitive*, _not_ based 
on (correct) math, and (therefore) flawed.  And intuition lets you down 
here.

The correct, mathematical reasoning is: *Because* the events are 
*independent*, the probability of the sequence does not change depending on 
the number of occurrences of an event (outcome of a draw).  That all numbers 
occur is _not_ more likely than that only one number occurs, or two, three, 
and so on.  And that does _not_ depend on how many draws you take as there 
is an infinite reservoir for each number.

I do not think I can explain it to you better, and I am not going to waste 
my precious free time to repeat myself.  Follow the reference I gave.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From random832 at fastmail.us  Sun Jun  7 15:44:31 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Sun, 07 Jun 2015 15:44:31 -0400
Subject: Testing random
In-Reply-To: <1583276.2lplL8rY5W@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
Message-ID: <1433706271.3761975.289140817.4214AB20@webmail.messagingengine.com>

On Sun, Jun 7, 2015, at 15:29, Thomas 'PointedEars' Lahn wrote:
> Jussi Piitulainen wrote:
> 
> > Thomas 'PointedEars' Lahn writes:
> >>   8 3 6 3 1 2 6 8 2 1 6.
> > 
> > There are more than four hundred thousand ways to get those numbers in
> > some order.
> > 
> > (11! / 2! / 2! / 2! / 3! / 2! = 415800)
> 
> Fallacy.  Order is irrelevant here.

Huh? The fact that order is irrelevant is exactly _why_ what he said
_is_ relevant.


From andrei.fokau at gmail.com  Sun Jun  7 15:54:08 2015
From: andrei.fokau at gmail.com (Andrei)
Date: Sun, 7 Jun 2015 12:54:08 -0700 (PDT)
Subject: Detect if specific Python.app instance is already running
In-Reply-To: <a3be1707-0eae-46fa-9921-e9c145816ad5@googlegroups.com>
References: <a3be1707-0eae-46fa-9921-e9c145816ad5@googlegroups.com>
Message-ID: <11e093d5-b78e-4ac6-9a7f-649cb2c2c942@googlegroups.com>

Alright, I have had some development in http://stackoverflow.com/questions/30694560/detect-if-specific-python-app-instance-is-already-running and can prevent running multiple instances of the same app/script (by lockf), but I still need to identify which Python.app instance is running certain script. I guess it's more Cocoa-related question but I hope someone had same problem with tkinter etc.

Andrei


From PointedEars at web.de  Sun Jun  7 16:09:31 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 22:09:31 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
 <mailman.274.1433706275.13271.python-list@python.org>
Message-ID: <44804165.jDZYAAtjtx@PointedEars.de>

random832 at fastmail.us wrote:

> On Sun, Jun 7, 2015, at 15:29, Thomas 'PointedEars' Lahn wrote:
>> Jussi Piitulainen wrote:
>> > Thomas 'PointedEars' Lahn writes:
>> >>   8 3 6 3 1 2 6 8 2 1 6.
>> > 
>> > There are more than four hundred thousand ways to get those numbers in
>> > some order.
>> > 
>> > (11! / 2! / 2! / 2! / 3! / 2! = 415800)
>> Fallacy.  Order is irrelevant here.
> 
> Huh? The fact that order is irrelevant is exactly _why_ what he said
> _is_ relevant.

No.  AISB, those sequences all have the same probability:

<http://www.teacherlink.org/content/math/interactive/probability/numbersense/misconceptions/representativeness.html>
<http://www.teacherlink.org/content/math/interactive/probability/numbersense/misconceptions/recency.html>

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From PointedEars at web.de  Sun Jun  7 16:15:49 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 22:15:49 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <55743ea4$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <mailman.247.1433684166.13271.python-list@python.org>
 <1586765.9ka6YhQpJC@PointedEars.de>
 <mailman.260.1433695702.13271.python-list@python.org>
Message-ID: <2115433.UTq2kp7hT1@PointedEars.de>

Peter Otten wrote:

> Thomas 'PointedEars' Lahn wrote:
>> Peter Otten wrote: 
>>> Steven D'Aprano wrote:
>>>>> I wrote a very simple function to test random:
>>>>>     def test_random(length, multiplier = 10000):
>>>>>         number_list = length * [0]
>>>>>         for i in range(length * multiplier):
>>>>>             number_list[random.randint(0, length - 1)] += 1
>>>>>         minimum = min(number_list)
>>>>>         maximum = max(number_list)
>>>>>         return (minimum, maximum, minimum / maximum)
>>>> 
>>>> Putting aside the timing aspects, your frequency calculations are not
>>>> done in a very Pythonic manner.
>>> 
>>> I would agree if the frequency table were sparse, i. e. many indices
>>> with
>>> 
>>> number_list[index] == 0
>>> 
>>> but that's not the case with on average 10000 hits per index.
>> 
>> A common misconception.
> 
> A pointless remark. Not counting the ears ;)

The details are in my other follow-ups.  Go read them.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From stefan_ml at behnel.de  Sun Jun  7 16:32:08 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 07 Jun 2015 22:32:08 +0200
Subject: Embedded Python and C Callback functions
In-Reply-To: <620b003e-a6d1-4113-b6dc-ad3cf0eaeccd@googlegroups.com>
References: <daad7377-7a61-4e9e-a442-4ba48f7212b2@googlegroups.com>
 <mailman.229.1433656976.13271.python-list@python.org>
 <620b003e-a6d1-4113-b6dc-ad3cf0eaeccd@googlegroups.com>
Message-ID: <ml29o8$4in$1@ger.gmane.org>

doc.mefisto at gmail.com schrieb am 07.06.2015 um 10:56:
> And I can't use Cython, because I have C++ module, and I have to use it.

That's not a valid reason. Cython supports C++ code just fine.

http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html

Stefan




From Cecil at decebal.nl  Sun Jun  7 16:35:23 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 07 Jun 2015 22:35:23 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl> <ml23r8$htm$1@news.albasani.net>
Message-ID: <878ubvnto4.fsf@Equus.decebal.nl>

On Sunday  7 Jun 2015 20:51 CEST, Johannes Bauer wrote:

> On 07.06.2015 10:22, Cecil Westerhof wrote:
>
>> That only times the function. I explicitly mentioned I want both
>> the needed time AND the output.
>
> And you also posted your solution. I fail to find any question in
> your original posting at all.

That is because there was no question: I just wanted to share
something I thought that could be useful. If you would have taken the
trouble to read a little, you would have known that.


>> Sadly the quality of the answers on this list is going down. 
>
> Maybe you should start asking questions that people are able to
> comprehend so that you get an answer that you like.

I do not want answer I like, but an answer that is useful.


>> Here I get an alternative that does only half what I want and when
>> writing an alternative for ?!find? I am told I could use ?!find?
>> (which only works in ipython, not python and which also not works
>> with Windows).
>
> Protip: Ditch the shitty attitude. I don't know if you're a jerk or
> not but I know for sure that you sound like one. Makes it also much
> less likely to get the answers you'd like.

So if someone gives an answer that is completely useless it is a
shitty attitude when I point this out? Very interesting indeed.

I do not think I have to expect something useful from you, it looks
like you prefer name calling. Luckily there are a lot of people that
have a different attitude.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From random832 at fastmail.us  Sun Jun  7 16:41:19 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Sun, 07 Jun 2015 16:41:19 -0400
Subject: Testing random
In-Reply-To: <44804165.jDZYAAtjtx@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
 <mailman.274.1433706275.13271.python-list@python.org>
 <44804165.jDZYAAtjtx@PointedEars.de>
Message-ID: <1433709679.3772506.289167697.02C8F7CD@webmail.messagingengine.com>



On Sun, Jun 7, 2015, at 16:09, Thomas 'PointedEars' Lahn wrote:
> No.  AISB, those sequences all have the same probability:

Yes and the probability of getting _any_ of the sequences, is the sum of
the probabilities for each one of the sequences individually.


From PointedEars at web.de  Sun Jun  7 16:56:32 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 22:56:32 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
 <mailman.274.1433706275.13271.python-list@python.org>
 <44804165.jDZYAAtjtx@PointedEars.de>
 <mailman.276.1433709682.13271.python-list@python.org>
Message-ID: <1506818.lbhLiAFkUt@PointedEars.de>

random832 at fastmail.us wrote:

> On Sun, Jun 7, 2015, at 16:09, Thomas 'PointedEars' Lahn wrote:
>> No.  AISB, those sequences all have the same probability:
> 
> Yes and the probability of getting _any_ of the sequences, is the sum of
> the probabilities for each one of the sequences individually.

The probability of getting any of the sequences is precisely 1, and that is 
irrelevant to the fact that the probability of getting a particular sequence 
is irrelevant of the number of occurrences of a specific event as the 
probability variable is evenly distributed and the events are independent 
(of each other).

You really should follow the references and brush up your math.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From PointedEars at web.de  Sun Jun  7 16:59:04 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 07 Jun 2015 22:59:04 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
 <mailman.274.1433706275.13271.python-list@python.org>
 <44804165.jDZYAAtjtx@PointedEars.de>
 <mailman.276.1433709682.13271.python-list@python.org>
Message-ID: <12048809.77ogS5qXqA@PointedEars.de>

random832 at fastmail.us wrote:

> On Sun, Jun 7, 2015, at 16:09, Thomas 'PointedEars' Lahn wrote:
>> No.  AISB, those sequences all have the same probability:
> 
> Yes and the probability of getting _any_ of the sequences, is the sum of
> the probabilities for each one of the sequences individually.

The probability of getting any of the sequences is precisely 1, and that is 
irrelevant to the fact that the probability of getting a particular sequence 
is independent of the number of occurrences of a specific event as the 
probability variable is evenly distributed and the events are independent 
(of each other).

You really should follow the references and brush up your math.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From sreenath.cg at gmail.com  Sun Jun  7 17:17:24 2015
From: sreenath.cg at gmail.com (Sreenath Nair)
Date: Mon, 8 Jun 2015 02:47:24 +0530
Subject: Query regarding sys.stdout.write
Message-ID: <BLU402-EAS68B18DD7A88AF142BC67B7E3B00@phx.gbl>

Hi,

I have a general query about the following snippet:

import os
Import sys
for each_dir in os.listdir("/home/tmpuser"):
    full_path = os.path.join("/home/tmpuser", each_dir)
    sys.stdout.write("\r%s" % full_path)
    sys.stdout.flush()

The snippet is a simplified example of me trying to print to the same line by using carriage return. This is working fine. However, the issue is that if the previous line was longer than the current line being printed then there are characters leftover from the previous print. Like so:

Print no. 1: /home/tmpuser/somedir/somefile.ext
Print no. 2:/home/tmpuser/somefile.extmefile.ext

In case of the newly printed shorter line, the characters from the previously printed longer line are leftover... Is there any way to clear the previous print? While still being able to print to the same line?


Thank you for any help.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150608/aa16b1c1/attachment.html>

From denismfmcmahon at gmail.com  Sun Jun  7 17:18:09 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sun, 7 Jun 2015 21:18:09 +0000 (UTC)
Subject: How to inverse a particle emitter
References: <1bad33c0-a728-4345-883b-505f84d8ea9b@googlegroups.com>
Message-ID: <ml2ceh$an8$2@dont-email.me>

On Thu, 04 Jun 2015 16:15:20 -0700, stephenppraneel7 wrote:

> hey, i really need help, im a straight up beginner in scripting and i
> need to figure out how to make an inverted particle emitter using python
> in maya

An emitter of inverted particles? Or an inverted emitter?

Or an absorber / accepter / receiver of emitted particles?

I'm not quite sure what an inverted particle emitter is, but I don't 
think you can create one in software, you need lots of volts and magnets.

If you're trying to model some physical device using python as a 
scripting language inside some simulation environment, then the best 
place to look is probably forums dedicated to that simulation environment.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From ned at nedbatchelder.com  Sun Jun  7 17:21:48 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sun, 7 Jun 2015 14:21:48 -0700 (PDT)
Subject: Testing random
In-Reply-To: <5515034.V7dcXEWAvK@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
Message-ID: <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>

On Sunday, June 7, 2015 at 2:26:02 PM UTC-4, Thomas 'PointedEars' Lahn wrote:
> Chris Angelico wrote:
> 
> > On Mon, Jun 8, 2015 at 2:36 AM, Thomas 'PointedEars' Lahn
> > <PointedEars at web.de> wrote:
> >>> The greater the multiplier, the lower the chance that any element will
> >>> have no hits.
> >> Wrong.
> >>
> >>> [ex falso quodlibet]
> > 
> > Huh. Do you want to explain how, mathematically, I am wrong, or do you
> > want to join the RUE in my ignore list?
> 
> I already did; you have overlooked it.  In a nutshell, the probability of 
> 
>   1 1 1 1 1 1 1 1 1 1 1
> 
> is *the same* as that of
> 
>   1 2 3 4 5 6 7 8 9 1 2
> 
> and the same as that of
> 
>   8 3 6 3 1 2 6 8 2 1 6.
> 

You aren't agreeing because you are arguing about different things.
Thomas is talking about the relative probability of sequences of digits.
Chris is talking about the probability of a single digit never appearing
in the output.

Thomas: let's say I generate streams of N digits drawn randomly from 0-9.  
I then consider the probability of a zero *never appearing once* in my
stream.  Let's call that P(N).  Do you agree that as N increases, P(N)
decreases?

--Ned.


From Cecil at decebal.nl  Sun Jun  7 18:30:49 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 08 Jun 2015 00:30:49 +0200
Subject: Query regarding sys.stdout.write
References: <mailman.278.1433712448.13271.python-list@python.org>
Message-ID: <874mmjnobq.fsf@Equus.decebal.nl>

On Sunday  7 Jun 2015 23:17 CEST, Sreenath Nair wrote:

> I have a general query about the following snippet:
>
> import os
> Import sys
> for each_dir in os.listdir("/home/tmpuser"):
> full_path = os.path.join("/home/tmpuser", each_dir)
> sys.stdout.write("\r%s" % full_path)
> sys.stdout.flush()
>
> The snippet is a simplified example of me trying to print to the
> same line by using carriage return. This is working fine. However,
> the issue is that if the previous line was longer than the current
> line being printed then there are characters leftover from the
> previous print. Like so:
>
> Print no. 1: /home/tmpuser/somedir/somefile.ext
> Print no. 2:/home/tmpuser/somefile.extmefile.ext
>
> In case of the newly printed shorter line, the characters from the
> previously printed longer line are leftover... Is there any way to
> clear the previous print? While still being able to print to the
> same line?

Say that you know that a line will never be longer as 80 characters,
then you could do something like:
sys.stdout.write('\r{0:80}'.format(full_path))

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From cs at zip.com.au  Sun Jun  7 18:40:42 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 8 Jun 2015 08:40:42 +1000
Subject: Query regarding sys.stdout.write
In-Reply-To: <BLU402-EAS68B18DD7A88AF142BC67B7E3B00@phx.gbl>
References: <BLU402-EAS68B18DD7A88AF142BC67B7E3B00@phx.gbl>
Message-ID: <20150607224042.GA62552@cskk.homeip.net>

On 08Jun2015 02:47, Sreenath Nair <sreenath.cg at gmail.com> wrote:
>I have a general query about the following snippet:
>
>import os
>Import sys
>for each_dir in os.listdir("/home/tmpuser"):
>    full_path = os.path.join("/home/tmpuser", each_dir)
>    sys.stdout.write("\r%s" % full_path)
>    sys.stdout.flush()
>
>The snippet is a simplified example of me trying to print to the same line by using carriage return. This is working fine. However, the issue is that if the previous line was longer than the current line being printed then there are characters leftover from the previous print. Like so:
>
>Print no. 1: /home/tmpuser/somedir/somefile.ext
>Print no. 2:/home/tmpuser/somefile.extmefile.ext
>
>In case of the newly printed shorter line, the characters from the previously printed longer line are leftover... Is there any way to clear the previous print? While still being able to print to the same line?

Certainly. Keep track of the length of the previous line. Pad the new line with 
sufficient spaces to overwrite the old line, and then sufficient backspaces to 
get back to the end of the new line.

For added points, write a minimal updated that notices if the old line had a 
common prefix with the new line, and emit only the change.

Cheers,
Cameron Simpson <cs at zip.com.au>

The worst tyrannies were the ones where a governance required its own logic on
every embedded node. - Vernor Vinge


From cs at zip.com.au  Sun Jun  7 18:48:57 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 8 Jun 2015 08:48:57 +1000
Subject: Is there a mail list like python-list in c?
In-Reply-To: <55747D03.5090805@gmail.com>
References: <55747D03.5090805@gmail.com>
Message-ID: <20150607224857.GA94974@cskk.homeip.net>

On 08Jun2015 01:18, Jimages <jimages123 at gmail.com> wrote:
>i know that the question may not be suitable. But i am learning python
>as well as c. And i find the list valuable. After google mail list
>about c. i don't find even one mail list t about programming in C. So
>could you recommend a mail list about programming in c?

Well there is the comp.lang.c newsgroup. Not a mailing list, but similar.

>BTW excuse my bad English.

Your English is better than my own poor second language.

Cheers,
Cameron Simpson <cs at zip.com.au>

We're lost, but we're making good time.


From nad at acm.org  Sun Jun  7 19:07:46 2015
From: nad at acm.org (Ned Deily)
Date: Sun, 07 Jun 2015 16:07:46 -0700
Subject: Detect if specific Python.app instance is already running
References: <a3be1707-0eae-46fa-9921-e9c145816ad5@googlegroups.com>
 <11e093d5-b78e-4ac6-9a7f-649cb2c2c942@googlegroups.com>
Message-ID: <nad-7D8872.16074507062015@news.gmane.org>

In article <11e093d5-b78e-4ac6-9a7f-649cb2c2c942 at googlegroups.com>,
 Andrei <andrei.fokau at gmail.com> wrote:
> Alright, I have had some development in 
> http://stackoverflow.com/questions/30694560/detect-if-specific-python-app-inst
> ance-is-already-running and can prevent running multiple instances of the 
> same app/script (by lockf), but I still need to identify which Python.app 
> instance is running certain script. I guess it's more Cocoa-related question 
> but I hope someone had same problem with tkinter etc.

Can you give a more complete explanation of the goal, i.e. what you are 
trying to end up with here, rather than the steps you are taking to get 
there?  It's not at all clear to me what your goal is and the cumbersome 
steps you find you have to take suggest that you might be going down the 
wrong path.  OS X apps usually don't have to resort to such hacks.  If 
you haven't already, make sure you have looked at the "Information 
Property List Key Reference" for OS X (and iOS) apps.  For instance, key 
LSMultipleInstancesProhibited *might* be useful if you really only want 
to ensure that there is only instance of an app launched.

https://developer.apple.com/library/mac/documentation/General/Reference/I
nfoPlistKeyReference/Articles/LaunchServicesKeys.html

-- 
 Ned Deily,
 nad at acm.org



From minscheung at googlemail.com  Sun Jun  7 20:01:00 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Sun, 7 Jun 2015 17:01:00 -0700 (PDT)
Subject: Updated Python 2.7.10 and now causing XCode not to build new project?
Message-ID: <6ea1ae4d-fd3c-4216-a22b-ad3e495f9d47@googlegroups.com>

Updated my anaconda Python to 2.7.10 and now when I start my XCode new project i get:

env: python: No such file or directory
Command /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed with exit code 127

Anyone experienced this issue? 


From rosuav at gmail.com  Sun Jun  7 20:13:37 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 10:13:37 +1000
Subject: Built-in Python3 web server functionality - wsgiref & http.server
In-Reply-To: <4f0edbe7-c634-4c26-8e36-0f32c4238447@googlegroups.com>
References: <4f0edbe7-c634-4c26-8e36-0f32c4238447@googlegroups.com>
Message-ID: <CAPTjJmoNoMrcAxRJQa5CO1EOw1wv5qBZoMU=zyxtTTOyaNCOQQ@mail.gmail.com>

On Mon, Jun 8, 2015 at 5:13 AM, Eric <ericgorr at gmail.com> wrote:
> I am trying to better understand the built-in python3 web server functionality and have just started reading the documentation.
>
> There seem to be two options provided by Python 3.
>
> 1.
> wsgiref
> https://docs.python.org/3/library/wsgiref.html
>
> 2.
> http.server
> https://docs.python.org/3/library/http.server.html
>
>
> Both options appear to offer a simple way to get a simple web server up and running. It does appear that wsgiref is built on top of http.server. Is this correct?
>
> Would it be correct to say that wsgiref is a high(er?)-level framework for designing web-based applications where as http.server is operating at a lower level? While one could do everything with just http.server, if wsgiref was appropriate for a particular use case, it would be the path to follow.
>
> Any insights into this topic would be appreciated.
>
> Are there any good python 3 books covering this functionality?

What are you actually trying to accomplish here? At what level of
abstraction do you want to work?

http.server is pretty low level:
https://github.com/Rosuav/Yosemite/blob/master/Yosemite.py

But if you want a higher level, I would recommend using something like Flask:
https://github.com/Rosuav/MinstrelHall/blob/master/mh.py

Yosemite.py runs as a stand-alone program, listens on a port, and can
be controlled by a web browser. It's a web server. But if you look
through the code, you'll see how raw an interface it is; that's not a
big problem for Yosemite, as it's an extremely thin layer over a file
system, so it's basically pulling out a few special cases ("/stop"
etc) and then looking for something at the exact path it was given.

Minstrel Hall runs inside Apache, via Passenger. You can also run it
directly, in which case it makes use of an internal
bind/listen/accept/dispatch loop, so it can also be called a web
server. But it's much more clearly operating as a web site, with
separate top-level functions that clearly show what URLs they're
handling. There's support for parameterized URLs like
http://minstrelhall.com/campaign/6 (comes through to campaign(id=6)),
and heaps of other features that a simple example like MH won't show
off. If you want to make a web site that's still maintainable after
it's grown bigger than the ~300 lines of code that Yosemite.py has, I
strongly recommend using a framework like Flask.

On the flip side, if you want a quick-and-dirty program that shows you
a whole lot of extremely raw information about what the web browser is
trying to do, http.server is a very handy tool, and it doesn't require
any extra installations (Flask isn't part of the standard library, so
you'd need to pip install it).

High or low abstraction level?

ChrisA


From rosuav at gmail.com  Sun Jun  7 20:18:17 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 10:18:17 +1000
Subject: Query regarding sys.stdout.write
In-Reply-To: <BLU402-EAS68B18DD7A88AF142BC67B7E3B00@phx.gbl>
References: <BLU402-EAS68B18DD7A88AF142BC67B7E3B00@phx.gbl>
Message-ID: <CAPTjJmqnaO-MM230s1HwaHXWEyPdEi3Te1RQkfsi6D70T9ttHA@mail.gmail.com>

On Mon, Jun 8, 2015 at 7:17 AM, Sreenath Nair <sreenath.cg at gmail.com> wrote:
> I have a general query about the following snippet:
>
> import os
> Import sys
> for each_dir in os.listdir("/home/tmpuser"):
>     full_path = os.path.join("/home/tmpuser", each_dir)
>     sys.stdout.write("\r%s" % full_path)
>     sys.stdout.flush()
>
> The snippet is a simplified example of me trying to print to the same line
> by using carriage return. This is working fine. However, the issue is that
> if the previous line was longer than the current line being printed then
> there are characters leftover from the previous print. Like so:
>
> Print no. 1: /home/tmpuser/somedir/somefile.ext
> Print no. 2:/home/tmpuser/somefile.extmefile.ext
>
> In case of the newly printed shorter line, the characters from the
> previously printed longer line are leftover... Is there any way to clear the
> previous print? While still being able to print to the same line?

The most common solution is to print out some spaces to overwrite the
previous text. It's simple, straight-forward, and works on all
systems. But if your console is properly ANSI-compliant, you may be
able to simply append \33[K to clear to end of line:

https://github.com/Rosuav/LetMeKnow/blob/master/letmeknow.py#L172

No need to retain the previous line length, no need to guess at the
length required, and no lengthening the long lines (which annoyed me;
I had to make the console window X characters wider than it needed to
be, else the spaces would make it wrap). Give it a try, see what
happens; worst case, fall back on what Cameron suggested.

ChrisA


From steve at pearwood.info  Sun Jun  7 21:11:20 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 08 Jun 2015 11:11:20 +1000
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
Message-ID: <5574ebb9$0$13009$c3e8da3$5496439d@news.astraweb.com>

On Mon, 8 Jun 2015 04:23 am, Thomas 'PointedEars' Lahn wrote:

> Chris Angelico wrote:
> 
>> On Mon, Jun 8, 2015 at 2:36 AM, Thomas 'PointedEars' Lahn
>> <PointedEars at web.de> wrote:
>>>> The greater the multiplier, the lower the chance that any element will
>>>> have no hits.
>>> Wrong.
>>>
>>>> [ex falso quodlibet]
>> 
>> Huh. Do you want to explain how, mathematically, I am wrong, or do you
>> want to join the RUE in my ignore list?
> 
> I already did; you have overlooked it.  In a nutshell, the probability of
> 
>   1 1 1 1 1 1 1 1 1 1 1
> 
> is *the same* as that of
> 
>   1 2 3 4 5 6 7 8 9 1 2
> 
> and the same as that of
> 
>   8 3 6 3 1 2 6 8 2 1 6.

True, but irrelevant. What is irrelevant is that the probability of:

    1 1 1 1 1 1 1 1 1 1 1

is MUCH LESS than the probability of:

    1 2 3 4 5 6 7 8 9 1 2  or
    8 3 6 3 1 2 6 8 2 1 6  or
    9 3 1 5 5 4 7 1 6 8 1  or
    2 3 3 1 8 9 7 2 5 1 6  or
    7 8 9 4 2 7 7 1 3 8 2  or
    ...
    [enumerate millions of other possibilities]
    ...
    2 1 5 5 8 1 8 7 6 5 1 or
    3 4 9 5 2 8 4 2 3 2 6.


> So, I am sorry to tell you this, but you do _not_ understand probability.

It's certainly true that you don't.

> And you *cannot* understand it intuitively, like you tried to. 

Rubbish. Chris has got it right. Whether he did so "intuitively" or whether
he learned this after years of study is irrelevant, and you have no
evidence for which it is. You are jumping to wrong conclusions -- you are
so sure that you're so much smarter than everyone else that you've ASSUMED
that Chris must be wrong and *completely failed to pay attention* to what
he has actually said, responding to things he hasn't said.


> It is why humans play the lottery using their ?lucky numbers? whereas
> other numbers have exactly the same probability of being drawn; 
[snip more irrelevant examples]

None of these things are relevant in the slightest.


> An extension of that misconception is emphasized by an anecdote (which may
> be apocryphal) told about Richard Feynman (I heard it from Lawrence Krauss
> in ?A Universe from Nothing?; he can tell that in a much more funny way
> than I am able to reproduce it here [1]):
> 
>   Richard Feynman used to go up to people all the time and he?d say:
> 
>     ?You won?t believe what happend to me today!
>      You won?t believe what happend to me today!?
> 
>   And people would say:
> 
>     ?What??
> 
>   And he would reply:
> 
>     ?Absolutely nothing!?

And I wouldn't believe it either. *Something happened*. He got up. He put
his shoes on. He washed his face. These are all events, but they are not
memorable events. They are equivalent to a sample like:

    6 8 1 5 9 2 7 6 3 3 2

There are millions of ways to get a non-memorable sample like that, but only
nine ways to get a memorable sample like:

    2 2 2 2 2 2 2 2 2 2 2


Your mistake is that you are comparing that one memorable sample to any
*one* of the non-memorable ones, when you should be comparing the
probability of *any* memorable sample against *any* non-memorable ones.

Actually, your real mistake is hubris. *Nothing* in Chris' comments in this
thread should lead you to the conclusion he doesn't know what he is talking
about, but you're so full of the preconceived notion that he must be wrong
because he is not Thomas Lahn that you've actually made an astonishing
blunder and compounded it with laughable arrogance.



-- 
Steven



From steve at pearwood.info  Sun Jun  7 21:26:28 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 08 Jun 2015 11:26:28 +1000
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
 <mailman.274.1433706275.13271.python-list@python.org>
 <44804165.jDZYAAtjtx@PointedEars.de>
 <mailman.276.1433709682.13271.python-list@python.org>
 <1506818.lbhLiAFkUt@PointedEars.de>
Message-ID: <5574ef45$0$12981$c3e8da3$5496439d@news.astraweb.com>

On Mon, 8 Jun 2015 06:56 am, Thomas 'PointedEars' Lahn wrote:

> You really should follow the references and brush up your math.

You really should stop being patronising to everyone and pay attention to
what is actually being said.

If you won't believe us, and you won't believe mathematics, perhaps you will
believe a simulation.


import random

def trial(size):
    return [random.randint(1, 9) for i in range(size)]

def test(trial_size, test_size):
    every_element_seen = 0
    for i in range(test_size):
        s = set(trial(trial_size))
        if len(s) == 9:
            every_element_seen += 1
    return every_element_seen/test_size


msg = "Pr(every element seen at least once) in %d trials"
for size in range(10, 100, 5):
    print(msg % size, end=' ')
    pr = test(size, 1000)
    print(pr)


When I run this simulation, I get:

Pr(every element seen at least once) in 10 trials 0.007
Pr(every element seen at least once) in 15 trials 0.109
Pr(every element seen at least once) in 20 trials 0.349
Pr(every element seen at least once) in 25 trials 0.6
Pr(every element seen at least once) in 30 trials 0.765
Pr(every element seen at least once) in 35 trials 0.868
Pr(every element seen at least once) in 40 trials 0.917
Pr(every element seen at least once) in 45 trials 0.953
Pr(every element seen at least once) in 50 trials 0.976
Pr(every element seen at least once) in 55 trials 0.981
Pr(every element seen at least once) in 60 trials 0.996
Pr(every element seen at least once) in 65 trials 0.999
Pr(every element seen at least once) in 70 trials 0.997
Pr(every element seen at least once) in 75 trials 1.0
Pr(every element seen at least once) in 80 trials 0.998
Pr(every element seen at least once) in 85 trials 1.0
Pr(every element seen at least once) in 90 trials 1.0
Pr(every element seen at least once) in 95 trials 1.0


 

-- 
Steven



From random832 at fastmail.us  Sun Jun  7 21:34:38 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Sun, 07 Jun 2015 21:34:38 -0400
Subject: Testing random
In-Reply-To: <1506818.lbhLiAFkUt@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
 <mailman.274.1433706275.13271.python-list@python.org>
 <44804165.jDZYAAtjtx@PointedEars.de>
 <mailman.276.1433709682.13271.python-list@python.org>
 <1506818.lbhLiAFkUt@PointedEars.de>
Message-ID: <1433727278.3822814.289289041.222E2077@webmail.messagingengine.com>

On Sun, Jun 7, 2015, at 16:56, Thomas 'PointedEars' Lahn wrote:
> random832 at fastmail.us wrote:
> 
> > On Sun, Jun 7, 2015, at 16:09, Thomas 'PointedEars' Lahn wrote:
> >> No.  AISB, those sequences all have the same probability:
> > 
> > Yes and the probability of getting _any_ of the sequences, is the sum of
> > the probabilities for each one of the sequences individually.
> 
> The probability of getting any of the sequences is precisely 1

Any of the sequences *satisfying "contains at least one of each result"*
(or, in general, any assertion about the degree of coverage, for
example, covering at least two values being more likely than covering
only one value)

Flipping one coin has two possible sequences: H and T. The probability
of getting at least one H and at least one T is 0.

Flipping two coins has four possible sequences: HH, HT, TH, TT. The
probability of getting at least one H and at least one T is 50%, since
two of these sequences satisfies the condition.

Flipping three coins has eight possible sequences. The probability of
getting at least one H and at least one T is 75%, since six of the
sequences: THH, HTH, HHT, HTT, THT, TTH, all satisfy the condition.

Flipping four coins has sixteen possibile sequences. The probability of
getting at least one H and at least one T is 87.5%

Rolling 2d3 has nine possible sequences.
The probability of getting at least one of each value is 0.
The probability of getting at least one each of two values is 66.7%.
The probability of getting only one value is 33.3%.

Rolling 3d3 has 27 possible sequences.
Six of them have one of each value. The probability is 22.2%.
24 have at least two different values. That probability is 88.9%.
Only 3 have all of the same value. 11.1%.

Rolling 4d3 has 81 possible sequences.
36 of them have at least one of each value. The probability is 44.4%.
78 of them have at least two different values. The probability is 96.3%.
Still only 3 have all of the same value. 3.7%.

Rolling 5d3 has 243 possible sequences.
150 of them have at least one of each value. The probability is 61.7%
240 of them have at least two different values. 98.8%.
Still only 3 have all of the same value. 1.2%.


In general, as the number of trials increases, the probability of having
e.g. at least one of each value never _reaches_ 1, but it gets
arbitrarily close.


From steve at pearwood.info  Sun Jun  7 21:35:07 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 08 Jun 2015 11:35:07 +1000
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <55743ea4$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <mailman.247.1433684166.13271.python-list@python.org>
Message-ID: <5574f14c$0$13008$c3e8da3$5496439d@news.astraweb.com>

On Sun, 7 Jun 2015 11:35 pm, Peter Otten wrote:

> Steven D'Aprano wrote:
> 
> 
>>> I wrote a very simple function to test random:
>>>     def test_random(length, multiplier = 10000):
>>>         number_list = length * [0]
>>>         for i in range(length * multiplier):
>>>             number_list[random.randint(0, length - 1)] += 1
>>>         minimum = min(number_list)
>>>         maximum = max(number_list)
>>>         return (minimum, maximum, minimum / maximum)
>> 
>> Putting aside the timing aspects, your frequency calculations are not
>> done in a very Pythonic manner.
> 
> I would agree if the frequency table were sparse, i. e. many indices with
> 
> number_list[index] == 0
> 
> but that's not the case with on average 10000 hits per index.
> 
>> A better way might be:
> 
> I'm too lazy to measure, but this will likely be a tad slower. Did you
> mean to convey this by "Putting aside the timing aspects"?

No, I meant that using a Counter to count stuff is more Pythonic than using
a list.



>> from collections import Counter
>> from random import randint
>> 
>> def test_random(length, multiplier = 10000):
>>     freq = Counter(
>>         randint(0, length - 1) for i in range(length * multiplier)
>>         )
>>     minimum = min(freq[i] for i in range(length))
> 
> How about
> 
>       if len(freq) < length:
>           minimum = 0
>       else:
>           minimum = min(freq.values())

That's quite nice, and it's extensible to the case where I return the
counter:

if len(freq) != length:
    # At least one element never showed up at all.
    freq.subtract(dict.fromkeys(range(length), 0)


 
> Not as an optimisation (replacing randint() with randrange() is probably
> more effective in that department), but because it's closer to being self-
> explanatory.

I thought about replacing randint with randrange, but they are different
functions with different implementations, and could exhibit different
biases. So one might wish to test them separately for statistical
properties.



-- 
Steven



From rosuav at gmail.com  Sun Jun  7 21:42:49 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 11:42:49 +1000
Subject: Testing random
In-Reply-To: <1433727278.3822814.289289041.222E2077@webmail.messagingengine.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
 <mailman.274.1433706275.13271.python-list@python.org>
 <44804165.jDZYAAtjtx@PointedEars.de>
 <mailman.276.1433709682.13271.python-list@python.org>
 <1506818.lbhLiAFkUt@PointedEars.de>
 <1433727278.3822814.289289041.222E2077@webmail.messagingengine.com>
Message-ID: <CAPTjJmqzEeTefie0gbOv300003FCUsJnqJ0ZxdHKd29LdR4jkg@mail.gmail.com>

On Mon, Jun 8, 2015 at 11:34 AM,  <random832 at fastmail.us> wrote:
> In general, as the number of trials increases, the probability of having
> e.g. at least one of each value never _reaches_ 1, but it gets
> arbitrarily close.

And by "arbitrarily close", you mean any of:

* So close to 1.0 that IEEE double precision is unable to represent it
* So unlikely that you could perform one trial every nanosecond until
the heat death of the universe and still not expect to see it
* Less likely than that two files could accidentally collide on MD5,
SHA1, SHA256, and file size, simultaneously

I think all of the above are true of this case, though I have to guess
about the heat death of the universe.

ChrisA


From python at mrabarnett.plus.com  Sun Jun  7 21:49:59 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 08 Jun 2015 02:49:59 +0100
Subject: Testing random
In-Reply-To: <CAPTjJmqzEeTefie0gbOv300003FCUsJnqJ0ZxdHKd29LdR4jkg@mail.gmail.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
 <mailman.274.1433706275.13271.python-list@python.org>
 <44804165.jDZYAAtjtx@PointedEars.de>
 <mailman.276.1433709682.13271.python-list@python.org>
 <1506818.lbhLiAFkUt@PointedEars.de>
 <1433727278.3822814.289289041.222E2077@webmail.messagingengine.com>
 <CAPTjJmqzEeTefie0gbOv300003FCUsJnqJ0ZxdHKd29LdR4jkg@mail.gmail.com>
Message-ID: <5574F4C7.1060503@mrabarnett.plus.com>

On 2015-06-08 02:42, Chris Angelico wrote:
> On Mon, Jun 8, 2015 at 11:34 AM,  <random832 at fastmail.us> wrote:
>> In general, as the number of trials increases, the probability of having
>> e.g. at least one of each value never _reaches_ 1, but it gets
>> arbitrarily close.
>
> And by "arbitrarily close", you mean any of:
>
I believe the word is "asymptotic".

> * So close to 1.0 that IEEE double precision is unable to represent it
> * So unlikely that you could perform one trial every nanosecond until
> the heat death of the universe and still not expect to see it
> * Less likely than that two files could accidentally collide on MD5,
> SHA1, SHA256, and file size, simultaneously
>
> I think all of the above are true of this case, though I have to guess
> about the heat death of the universe.
>



From cs at zip.com.au  Sun Jun  7 21:56:25 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 8 Jun 2015 11:56:25 +1000
Subject: Query regarding sys.stdout.write
In-Reply-To: <CAPTjJmqnaO-MM230s1HwaHXWEyPdEi3Te1RQkfsi6D70T9ttHA@mail.gmail.com>
References: <CAPTjJmqnaO-MM230s1HwaHXWEyPdEi3Te1RQkfsi6D70T9ttHA@mail.gmail.com>
Message-ID: <20150608015625.GA74401@cskk.homeip.net>

On 08Jun2015 10:18, Chris Angelico <rosuav at gmail.com> wrote:
>On Mon, Jun 8, 2015 at 7:17 AM, Sreenath Nair <sreenath.cg at gmail.com> wrote:
>> I have a general query about the following snippet:
>>
>> import os
>> Import sys
>> for each_dir in os.listdir("/home/tmpuser"):
>>     full_path = os.path.join("/home/tmpuser", each_dir)
>>     sys.stdout.write("\r%s" % full_path)
>>     sys.stdout.flush()
>>
>> The snippet is a simplified example of me trying to print to the same line
>> by using carriage return. This is working fine. However, the issue is that
>> if the previous line was longer than the current line being printed then
>> there are characters leftover from the previous print. Like so:
>>
>> Print no. 1: /home/tmpuser/somedir/somefile.ext
>> Print no. 2:/home/tmpuser/somefile.extmefile.ext
>>
>> In case of the newly printed shorter line, the characters from the
>> previously printed longer line are leftover... Is there any way to clear the
>> previous print? While still being able to print to the same line?
>
>The most common solution is to print out some spaces to overwrite the
>previous text. It's simple, straight-forward, and works on all
>systems. But if your console is properly ANSI-compliant, you may be
>able to simply append \33[K to clear to end of line:

If you want do do this version portably, hook into the Python curses module and 
get the result of curses.tigetstr('el'). If that returns None, the terminal 
does not support clear-to-end-of-line and you should use the spaces-overwrite 
technique. But if it returns a string, you can write that string, whatever it 
is, to erase to the end of the line.

Most modern terminals are ANSI supersets, but the terminfo database has a huge 
suite of descriptions of terminals, and this will get your the right sequence, 
whatever that may be, for your current terminal.

Of course, reaching for the curses module is massive overkill in a sense, but 
it is the only way to consult terminfo in the Python stdlib.

See "man terminfo" for details on what terminal capabilities the terminfo 
database describes.

Cheers,
Cameron Simpson <cs at zip.com.au>

But I have to say, I "non-concur." (Non-concur was a term I learned at
IBM. IBM seemed to be a breeding ground for making up words and
phrases, turning verbs into nouns, etc. "Non-concur" seems to be for
those times when saying "I disagree" just isn't strong enough!)
        - Robert D. Seidman <robert at clark.net>


From random832 at fastmail.us  Sun Jun  7 21:57:03 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Sun, 07 Jun 2015 21:57:03 -0400
Subject: Testing random
In-Reply-To: <CAPTjJmqzEeTefie0gbOv300003FCUsJnqJ0ZxdHKd29LdR4jkg@mail.gmail.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
 <mailman.274.1433706275.13271.python-list@python.org>
 <44804165.jDZYAAtjtx@PointedEars.de>
 <mailman.276.1433709682.13271.python-list@python.org>
 <1506818.lbhLiAFkUt@PointedEars.de>
 <1433727278.3822814.289289041.222E2077@webmail.messagingengine.com>
 <CAPTjJmqzEeTefie0gbOv300003FCUsJnqJ0ZxdHKd29LdR4jkg@mail.gmail.com>
Message-ID: <1433728623.3827210.289304689.29C9F79F@webmail.messagingengine.com>

On Sun, Jun 7, 2015, at 21:42, Chris Angelico wrote:
> On Mon, Jun 8, 2015 at 11:34 AM,  <random832 at fastmail.us> wrote:
> > In general, as the number of trials increases, the probability of having
> > e.g. at least one of each value never _reaches_ 1, but it gets
> > arbitrarily close.
> 
> And by "arbitrarily close", you mean any of:
> 
> * So close to 1.0 that IEEE double precision is unable to represent it
> * So unlikely that you could perform one trial every nanosecond until
> the heat death of the universe and still not expect to see it
> * Less likely than that two files could accidentally collide on MD5,
> SHA1, SHA256, and file size, simultaneously
> 
> I think all of the above are true of this case, though I have to guess
> about the heat death of the universe.

Well, yes (assuming, as you say, that the heat death of the universe is
a finite time). More precisely, "arbitrarily close" is a mathematical
term which means that for any desired distance from the limit, there is
an input that will cause the result to get closer. It's entirely
possible to select a number of symbols and a number of trials that won't
get you there.

I also think there may have been some confusion with some of us using
"trial" to refer to each individual random number, and some of us at
some times using it to refer to each attempt to generate a sequence of a
given length.


From rosuav at gmail.com  Sun Jun  7 23:19:29 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Jun 2015 13:19:29 +1000
Subject: Query regarding sys.stdout.write
In-Reply-To: <20150608015625.GA74401@cskk.homeip.net>
References: <CAPTjJmqnaO-MM230s1HwaHXWEyPdEi3Te1RQkfsi6D70T9ttHA@mail.gmail.com>
 <20150608015625.GA74401@cskk.homeip.net>
Message-ID: <CAPTjJmrgnrp3zzbwa9OXrEzCLQOvUoMeVr9Z5Tudg+Xw8_wdNQ@mail.gmail.com>

On Mon, Jun 8, 2015 at 11:56 AM, Cameron Simpson <cs at zip.com.au> wrote:
> If you want do do this version portably, hook into the Python curses module
> and get the result of curses.tigetstr('el'). If that returns None, the
> terminal does not support clear-to-end-of-line and you should use the
> spaces-overwrite technique. But if it returns a string, you can write that
> string, whatever it is, to erase to the end of the line.
>
> Most modern terminals are ANSI supersets, but the terminfo database has a
> huge suite of descriptions of terminals, and this will get your the right
> sequence, whatever that may be, for your current terminal.

Fair point. I hard-coded it in LetMeKnow because I didn't know of a
way to access terminfo in Python; thanks for the tip. Although...

> Of course, reaching for the curses module is massive overkill in a sense,
> but it is the only way to consult terminfo in the Python stdlib.

... yeah, massive overkill.

> But I have to say, I "non-concur." (Non-concur was a term I learned at
> IBM. IBM seemed to be a breeding ground for making up words and
> phrases, turning verbs into nouns, etc. "Non-concur" seems to be for
> those times when saying "I disagree" just isn't strong enough!)
>        - Robert D. Seidman <robert at clark.net>

Heh. That is so true. My favourite piece of IBM-speak comes from the
OS/2 config.sys directive "REIPL=C", which causes fatal errors to be
logged to C:\POPUPLOG.OS2 ... of course.

http://rosuav.blogspot.com.au/2011/01/reiplophobia.html

ChrisA


From dfnsonfsduifb at gmx.de  Mon Jun  8 01:04:08 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Mon, 08 Jun 2015 07:04:08 +0200
Subject: Function to show time to execute another function
In-Reply-To: <878ubvnto4.fsf@Equus.decebal.nl>
References: <87k2vgowco.fsf@Equus.decebal.nl>	<mailman.231.1433662792.13271.python-list@python.org>	<87fv64orly.fsf@Equus.decebal.nl>
 <ml23r8$htm$1@news.albasani.net> <878ubvnto4.fsf@Equus.decebal.nl>
Message-ID: <ml37o8$qmq$1@news.albasani.net>

On 07.06.2015 22:35, Cecil Westerhof wrote:

>> And you also posted your solution. I fail to find any question in
>> your original posting at all.
> 
> That is because there was no question: I just wanted to share
> something I thought that could be useful. If you would have taken the
> trouble to read a little, you would have known that.

You actually didn't write that. You wrote "I have a problem. Here is my
solution. This is how it works." Okay, so what? If you would have taken
the time to write "Just wanted to share this" your intention would have
been obvious.

>>> Sadly the quality of the answers on this list is going down. 
>>
>> Maybe you should start asking questions that people are able to
>> comprehend so that you get an answer that you like.
> 
> I do not want answer I like, but an answer that is useful.

Sorry to break it to you, but this group does not revolve around you. An
answer that might not be useful to you might be useful for someone else
and it surely is useful to get a discussion.

>>> Here I get an alternative that does only half what I want and when
>>> writing an alternative for ?!find? I am told I could use ?!find?
>>> (which only works in ipython, not python and which also not works
>>> with Windows).
>>
>> Protip: Ditch the shitty attitude. I don't know if you're a jerk or
>> not but I know for sure that you sound like one. Makes it also much
>> less likely to get the answers you'd like.
> 
> So if someone gives an answer that is completely useless it is a
> shitty attitude when I point this out? Very interesting indeed.

Precisely. Especially the tone you used is proof of your really shitty
attitude.

> I do not think I have to expect something useful from you, it looks
> like you prefer name calling. Luckily there are a lot of people that
> have a different attitude.

Indeed you can expect nothing more from me and I purposely omitted the
way I do timing from this discussion. I do not aid people who fail to
recognize their major social dysfunction. Not even when they are coding
geniuses. Which, judging from your code snippet, you clearly aren't.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From dieter at handshake.de  Mon Jun  8 01:44:52 2015
From: dieter at handshake.de (dieter)
Date: Mon, 08 Jun 2015 07:44:52 +0200
Subject: Embedded Python and C Callback functions
References: <daad7377-7a61-4e9e-a442-4ba48f7212b2@googlegroups.com>
 <mailman.229.1433656976.13271.python-list@python.org>
 <3726dccc-76cb-45a2-b0a8-e41ef300be96@googlegroups.com>
Message-ID: <87k2vezrcb.fsf@handshake.de>

doc.mefisto at gmail.com writes:

> I'm trying to debug and find my error. It goes wrong when:
>  PyObject *v;
>             v = va_arg(*p_va, PyObject *);
>             if (v != NULL) {
>                 if (*(*p_format - 1) != 'N')
>                     Py_INCREF(v);
>             }
>
> it tries to PyINCREF to my passed callback.

This indicates that it expects the "callback" to be a Python object.

Check the C-API specification. It should tell you which parameters
are expected to be Python objects and which C objects.



From Cecil at decebal.nl  Mon Jun  8 02:32:22 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 08 Jun 2015 08:32:22 +0200
Subject: Function to show time to execute another function
References: <87k2vgowco.fsf@Equus.decebal.nl>
 <mailman.231.1433662792.13271.python-list@python.org>
 <87fv64orly.fsf@Equus.decebal.nl> <ml23r8$htm$1@news.albasani.net>
 <878ubvnto4.fsf@Equus.decebal.nl> <ml37o8$qmq$1@news.albasani.net>
Message-ID: <87mw0aww09.fsf@Equus.decebal.nl>

On Monday  8 Jun 2015 07:04 CEST, Johannes Bauer wrote:

> On 07.06.2015 22:35, Cecil Westerhof wrote:
>
>>> And you also posted your solution. I fail to find any question in
>>> your original posting at all.
>>
>> That is because there was no question: I just wanted to share
>> something I thought that could be useful. If you would have taken
>> the trouble to read a little, you would have known that.
>
> You actually didn't write that. You wrote "I have a problem. Here is
> my solution. This is how it works." Okay, so what? If you would have
> taken the time to write "Just wanted to share this" your intention
> would have been obvious.

Again: if you just had read other posts you would have found that I
acknowledged that.


>>>> Sadly the quality of the answers on this list is going down. 
>>>
>>> Maybe you should start asking questions that people are able to
>>> comprehend so that you get an answer that you like.
>>
>> I do not want answer I like, but an answer that is useful.
>
> Sorry to break it to you, but this group does not revolve around
> you. An answer that might not be useful to you might be useful for
> someone else and it surely is useful to get a discussion.

Where do you find that I think that the group revolves around me?

Well it could be useful for someone to write something about how to
pay less taxes. But I do not think it should be posted in this group.


>>>> Here I get an alternative that does only half what I want and
>>>> when writing an alternative for ?!find? I am told I could use
>>>> ?!find? (which only works in ipython, not python and which also
>>>> not works with Windows).
>>>
>>> Protip: Ditch the shitty attitude. I don't know if you're a jerk
>>> or not but I know for sure that you sound like one. Makes it also
>>> much less likely to get the answers you'd like.
>>
>> So if someone gives an answer that is completely useless it is a
>> shitty attitude when I point this out? Very interesting indeed.
>
> Precisely. Especially the tone you used is proof of your really
> shitty attitude.

Do you ever look into a mirror?


>> I do not think I have to expect something useful from you, it looks
>> like you prefer name calling. Luckily there are a lot of people
>> that have a different attitude.
>
> Indeed you can expect nothing more from me and I purposely omitted
> the way I do timing from this discussion. I do not aid people who
> fail to recognize their major social dysfunction. Not even when they
> are coding geniuses. Which, judging from your code snippet, you
> clearly aren't.

And again the name calling. You think you are very superior, don't
you? You have a problem with my tone, well I do not even get near you:
you clearly are the master. If I would like to become dysfunctional I
should treasure your postings.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From windowod at gmail.com  Mon Jun  8 03:08:49 2015
From: windowod at gmail.com (Yurij Alexandrovich)
Date: Mon, 8 Jun 2015 00:08:49 -0700 (PDT)
Subject: Please, rate the project
Message-ID: <bc418117-4f09-44e9-9f3e-755adaf2252b@googlegroups.com>

EnvTransfer - transfer your environment between computers using Yandex disk. https://github.com/deslum/envtransfer


From jbauer.usenet at gmail.com  Mon Jun  8 03:22:49 2015
From: jbauer.usenet at gmail.com (jbauer.usenet at gmail.com)
Date: Mon, 8 Jun 2015 00:22:49 -0700 (PDT)
Subject: Using ssl module over custom sockets
Message-ID: <9d27cf3f-3edc-4ff0-ba69-0fc588e049a6@googlegroups.com>

Hi group,

is it possible to use the ssl module using a custom transport? It appears to me as if currently the relationship between ssl.SSLSocket() and socket.socket() is pretty entangled.

Suppose I do have some kind of reliable transport (let's say RS232) and a connection that I have wrapped in a class. The things I can provide are send/recv methods, but nothing specific to UNIX sockets (getsockopt, etc). Could I use the Python ssl module to perform a SSL connection over such a line?

Something that I could always use as a workaround would be to open up a listening port locally in one thread and connection to that local port in a different thread, then forward packets. But that's pretty ugly and I'd like to avoid it.

Any hints appreciated,
Best regards,
Johannes


From narencr7 at gmail.com  Mon Jun  8 03:32:38 2015
From: narencr7 at gmail.com (naren)
Date: Mon, 8 Jun 2015 13:02:38 +0530
Subject: Memory error while using pandas dataframe
Message-ID: <CAB1mvkwG6-ryzsqdiHus-McRWy2QCh1f6+Xm-Q0KNU=YeGOA6Q@mail.gmail.com>

Memory Error while working with pandas dataframe.

Description of Environment Windows 7 python 3.4.2 32-bit version pandas
0.16.0

We are running into the error described below. Any help provided will be
sincerely appreciated.

We are able to read a 300MB Csv file into a dataframe using the read_csv
function. While working with the dataframe we ran into memory error. We
used the pd.Concat function to concatenate two dataframes. So we decided to
use chunksize for lazy reading. Chunking returns an object of type
TextFileReader.

http://pandas.pydata.org/pandas-docs/stable/io.html#iterating-through-files-chunk-by-chunk

We are able to iterate over this object once as a debugging measure. The
iterator gets exhausted after iterating once. So we are not able to convert
the TextFileReader object back into a dataframe, using the pd.concat
function.

Error

Traceback (most recent call last):
  File "psindia.py", line 60, in <module>
    data=pd.concat(tp,ignore_index=True)
  File "C:\Python34\lib\site-packages\pandas\tools\merge.py", line 754, in conca
t
    copy=copy)
  File "C:\Python34\lib\site-packages\pandas\tools\merge.py", line 799, in __ini
t__
    raise ValueError('All objects passed were None')ValueError: All
objects passed were None

Thanks for your time.

-- 
Narendran Elango
B.Tech(2014)
Department of Computer Engineering
National Institute of Technology Karnataka, Surathkal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150608/2708ce36/attachment.html>

From jpiitula at ling.helsinki.fi  Mon Jun  8 03:40:46 2015
From: jpiitula at ling.helsinki.fi (Jussi Piitulainen)
Date: Mon, 08 Jun 2015 10:40:46 +0300
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
Message-ID: <lf5r3pmejgh.fsf@ling.helsinki.fi>

Thomas 'PointedEars' Lahn writes:
> Jussi Piitulainen wrote:
>> Thomas 'PointedEars' Lahn writes:
>>
>>>   8 3 6 3 1 2 6 8 2 1 6.
>> 
>> There are more than four hundred thousand ways to get those numbers
>> in some order.
>> 
>> (11! / 2! / 2! / 2! / 3! / 2! = 415800)
>
> Fallacy.  Order is irrelevant here.

You need to consider every sequence that leads to the observed counts.
One of those sequences occurred. You don't know which.

When tossing herrings into a fixed number of boxes (each time picking
the number of one of the boxes), the proportion of the ways to hit every
box at least once increases with the number of herrings as follows.

Proportions of occupying sequences of tosses into 3 boxes:
 2 herrings:  0% == 0 out of 9
 3 herrings: 22% == 6 out of 27
 4 herrings: 44% == 36 out of 81
 5 herrings: 62% == 150 out of 243
 6 herrings: 74% == 540 out of 729
 7 herrings: 83% == 1806 out of 2187
 8 herrings: 88% == 5796 out of 6561
 9 herrings: 92% == 18150 out of 19683
10 herrings: 95% == 55980 out of 59049
11 herrings: 97% == 171006 out of 177147
12 herrings: 98% == 519156 out of 531441

That's counting the sequences of box numbers, so those proportions can
be interpreted as probabilities (of occupying every box) under the
standard assumption that the sequences are equiprobable. The final
distributions of counts aren't. (How is this even counter-intuitive?)

Code follows. Incidentally, I'm not feeling smart here. I made several
quite stupid mistakes during this little experiment before I was happy
with it. Note that this actually walks through a combinatorial
explosion, so do not substitute much bigger parameters.

from itertools import product
from collections import Counter

boxes = 3
print('Proportions of occupying sequences of tosses into {} boxes:'
      .format(boxes))
for herrings in range(2, 13):
    winnings = sum(len(Counter(p)) == boxes
                   for p in product(range(boxes),
                                    repeat = herrings))
    print('{:>2} herrings:'.format(herrings),
          '{:>3.0%}'.format(winnings / (boxes ** herrings)),
          '== {} out of {}'.format(winnings, boxes ** herrings))


From jeremy at jeremysanders.net  Mon Jun  8 03:43:33 2015
From: jeremy at jeremysanders.net (Jeremy Sanders)
Date: Mon, 08 Jun 2015 09:43:33 +0200
Subject: Python Random vs. Cython C Rand for Dice Rolls
References: <55747C96.7090006@cdreimer.com>
Message-ID: <ml3h1v$pjo$1@ger.gmane.org>

C.D. Reimer wrote:

> Is there something in the Cython code that I need to change and/or find
> a better C random number generator?

This may not be helpful, but numpy is pretty helpful for this sort of thing:

import numpy
import numpy.random

a=numpy.random.randint(1,6,50000000)
b=numpy.random.randint(1,6,50000000)

numpy.bincount(a+b-1)

array([      0, 1999229, 4000369, 5999372, 7999232, 9998769, 8003430,
       5998538, 4001160, 1999901])

This takes a few seconds on my system.

Jeremy




From narencr7 at gmail.com  Mon Jun  8 03:57:14 2015
From: narencr7 at gmail.com (narencr7 at gmail.com)
Date: Mon, 8 Jun 2015 00:57:14 -0700 (PDT)
Subject: Memory Error while using pandas dataframe
Message-ID: <4300c647-6c2d-4e5a-8404-ee7a04317a2f@googlegroups.com>

Memory Error while working with pandas dataframe.

Description of Environment Windows 7 python 3.4.2 32-bit version pandas 0.16.0

We are running into the error described below. Any help provided will be sincerely appreciated.

We are able to read a 300MB Csv file into a dataframe using the read_csv function. While working with the dataframe we ran into memory error. We used the pd.Concat function to concatenate two dataframes. So we decided to use chunksize for lazy reading. Chunking returns an object of type TextFileReader.

http://pandas.pydata.org/pandas-docs/stable/io.html#iterating-through-files-chunk-by-chunk

We are able to iterate over this object once as a debugging measure. The iterator gets exhausted after iterating once. So we are not able to convert the TextFileReader object back into a dataframe, using the pd.concat function.

Error

Traceback (most recent call last):
  File "psindia.py", line 60, in <module>
    data=pd.concat(tp,ignore_index=True)
  File "C:\Python34\lib\site-packages\pandas\tools\merge.py", line 754, in conca
t
    copy=copy)
  File "C:\Python34\lib\site-packages\pandas\tools\merge.py", line 799, in __ini
t__
    raise ValueError('All objects passed were None')
ValueError: All objects passed were None
Thanks for your time.


From andrei.fokau at gmail.com  Mon Jun  8 04:32:40 2015
From: andrei.fokau at gmail.com (Andrei)
Date: Mon, 8 Jun 2015 01:32:40 -0700 (PDT)
Subject: Detect if specific Python.app instance is already running
In-Reply-To: <mailman.281.1433718479.13271.python-list@python.org>
References: <a3be1707-0eae-46fa-9921-e9c145816ad5@googlegroups.com>
 <11e093d5-b78e-4ac6-9a7f-649cb2c2c942@googlegroups.com>
 <mailman.281.1433718479.13271.python-list@python.org>
Message-ID: <6651a781-abe1-4f25-b1f3-1f849776d060@googlegroups.com>

On Monday, June 8, 2015 at 1:08:07 AM UTC+2, Ned Deily wrote:
> In article <11e093d5-b78e-4ac6-9a7f-649cb2c2c942 at googlegroups.com>,
>  Andrei wrote:
> > Alright, I have had some development in 
> > http://stackoverflow.com/questions/30694560/detect-if-specific-python-app-inst
> > ance-is-already-running and can prevent running multiple instances of the 
> > same app/script (by lockf), but I still need to identify which Python.app 
> > instance is running certain script. I guess it's more Cocoa-related question 
> > but I hope someone had same problem with tkinter etc.
> 
> Can you give a more complete explanation of the goal, i.e. what you are 
> trying to end up with here, rather than the steps you are taking to get 
> there?  It's not at all clear to me what your goal is and the cumbersome 
> steps you find you have to take suggest that you might be going down the 
> wrong path.  OS X apps usually don't have to resort to such hacks.  If 
> you haven't already, make sure you have looked at the "Information 
> Property List Key Reference" for OS X (and iOS) apps.  For instance, key 
> LSMultipleInstancesProhibited *might* be useful if you really only want 
> to ensure that there is only instance of an app launched.
> 
> https://developer.apple.com/library/mac/documentation/General/Reference/I
> nfoPlistKeyReference/Articles/LaunchServicesKeys.html
> 
> -- 
>  Ned Deily


Hi Ned,
Thanks for your reply! The problem is a bit tricky as I see it. Multiple instances of Python.app needs to be run since it may run different tkinter apps etc. What I need is rather simple:

1. start app, mark the instance with tkinter app name
2. start another one, detect that the same instance is already running and identify it
3. focus on the first instance, close the second one

I have solved it with a lock file and a pid file (I guess I could use just one file for both jobs, but didn't have time to test it yet). I believe it could be done without files, i.e. by setting some public attributes on app instances. I could probably also use OS X key storage for this but it gets too similar to the file approach and I am not sure if it's any better.

I may be wrong in my thinking. If so, just let me know :)

Thanks,
Andrei


From albert at spenarnc.xs4all.nl  Mon Jun  8 07:21:54 2015
From: albert at spenarnc.xs4all.nl (Albert van der Horst)
Date: 08 Jun 2015 11:21:54 GMT
Subject: Rule of order for dot operators?
References: <55579886.3010001@cdreimer.com>
 <mailman.118.1431989304.17265.python-list@python.org>
Message-ID: <55757ad2$0$3063$e4fe514c@dreader35.news.xs4all.nl>

In article <mailman.118.1431989304.17265.python-list at python.org>,
Cameron Simpson  <cs at zip.com.au> wrote:
>On 16May2015 12:20, C.D. Reimer <chris at cdreimer.com> wrote:
>>title = slug.replace('-',' ').title()
>>This line also works if I switched the dot operators around.
>>title = slug.title().replace('-',' ')
>>
>>I'm reading the first example as character replacement first and title
>>capitalization second, and the second example as title capitalization
>>first and character replacement second.
>>
>>Does python perform the dot operators from left to right or according
>>to a rule of order (i.e., multiplication/division before add/subtract)?
>
>I've been thinking about the mindset that asks this question.
>
>I think the reason you needed to ask it was that you were thinking in terms of
>each .foo() operation acting on the original "slug". They do not.
>
>"slug" is an expression returning, of course, itself.
>
>"slug.title()" is an expression returning a new string which is a titlecased
>version of "slug".

Why is "slug.title" a valid decomposition of the total string>
(Or is it?)
What is the ()-brackets doing? Does it force the execution of title,
which gives something to be dotted onto slug etc. See below.

>
>"slug.title().replace(...)" is an expression which _operates on_ that new
>string, _not_ on "slug".
>
>In particular, each .foo() need not return a string - it might return anything,
>and the following .bah() will work on that anything.

I interpreted the question as about the associative of the
dot operator.

title = slug.title().replace('-',' ')

Does that mean
title = slug.( title().replace('-',' ') )
or

title = ( slug.( title()) .replace('-',' ')

or is it even    <slot>.<slot>.<slot> a ternary operator with
special syntax like we have in
    <slot> <= <slot> <= <slot>
lately.

It seems to me that you presuppose the answer.

>
>Cheers,
>Cameron Simpson <cs at zip.com.au>
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst



From felix at epepm.cupet.cu  Mon Jun  8 08:24:16 2015
From: felix at epepm.cupet.cu (felix)
Date: Mon, 08 Jun 2015 08:24:16 -0400
Subject: Lawful != Mutable (was Can Python function return multiple data?)
In-Reply-To: <4fdc38b3-6b51-4f98-9ba3-42b265582fc0@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
 <mailman.212.1433569420.13271.python-list@python.org>
 <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>
 <4fdc38b3-6b51-4f98-9ba3-42b265582fc0@googlegroups.com>
Message-ID: <55758970.1080005@epepm.cupet.cu>

El 07/06/15 12:20, Rustom Mody escribi?:
> On Saturday, June 6, 2015 at 10:20:49 AM UTC+5:30, Steven D'Aprano wrote:
>> On Sat, 6 Jun 2015 01:20 pm, Rustom Mody wrote:
>>
>>> On Saturday, June 6, 2015 at 3:30:23 AM UTC+5:30, Chris Angelico wrote:
>>>> Congrats! You just proved that an object can itself be immutable, but
>>>> can contain references to mutables. Ain't that awesome?
>>>>
>>>> Did you have a point?
>>> [Under assumption you are not being facetious...]
>>> The word immutuable happens to have existed in English before python.
>>> I also happen to have used it before I knew of python
>>> The two meanings do not match
>>> I am surprised
>>> Is that surprising?
>> Yes, I am surprised that you are surprised. You have been a regular, and
>> prolific, contributor on this forum for some years now, teach Python, blog
>> about it. You're quite obviously well-read and experienced. How is it that
>> you are surprised by such a fundamental part of not just Python's object
>> model, but of real life objects too?
>>
>> I suspect you are pretending to be surprised to make a rhetorical point.
> Dunno why the fuss...
> As someone who often asks for short-simple-self-contained examples,
> I would have expected you to prefer clarity of communication over
> fidelity of experience??
>
> Anyways...
>
> Did I(rusi) now(2015) find that specific example surprising??
>
> No, but:
>
> 1. Questioners here regularly do show similar confusions.
> 2. I did go through similar decades ago when I first encountered programming
> 3. And most to the point, just add a little complexity and I (rusi-now)
> am as confused as any noob.
>
> A few weeks ago there was this example.
> It completely knocked me.
> ChrisA's exxplanation clarified.
> Here is a reconstruction based on Chris clarification
> [Chris do you remember the question?]
>
>>>> t=([1,2],[3,4])
>>>> t
> ([1, 2], [3, 4])
>>>> t[1] = t[1].append(5)
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> TypeError: 'tuple' object does not support item assignment
>>>> t
> ([1, 2], [3, 4, 5])
>
>> Like many English words, "immutable" has a few meanings in plain English.
>> Bouvier's Law Dictionary included in 1856 this definition:
>>
>>      IMMUTABLE. What cannot be removed, what is unchangeable.
>>      The laws of God peing perfect, are immutable, but no
>>      human law can be so considered.
>>
>> Clearly tuples can be removed. They are garbage-collected like any other
>> values in Python. If nothing else, you can turn the computer off, remove
>> the RAM, grind it down into the finest powder, and scatter it to the winds.
>> That surely is enough to remove the tuples <wink>
> Ok now rewrite that para above with
> s/tuple/numbers like 3 or 666/
> So I put '3' on the ram and grind it to finest powder.
> Have all trinities (of religious or secular variety) disappeared?
> 666 gone has the devil been banished from God's (or Steven's) universe?
>
>> So according to Bouvier's definition, tuples are not immutable. But I trust
>> that we can agree that Bouvier's definition is not useful here.
> Actually its a very nice and apt quote.
> Just add laws of mathematics to laws of God and it will be perfect.
> [Or better remember that for Plato they were the same]
Interesting topic.
I'm a newbie to Python and programming.
I remembered that in the official Python tutorial I read this:

"...
TypeError: 'tuple' object does not support item assignment
 >>> # but they can contain mutable objects:
..."

Cheers.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150608/e092054c/attachment.html>

From steve at pearwood.info  Mon Jun  8 09:06:58 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 08 Jun 2015 23:06:58 +1000
Subject: Rule of order for dot operators?
References: <55579886.3010001@cdreimer.com>
 <mailman.118.1431989304.17265.python-list@python.org>
 <55757ad2$0$3063$e4fe514c@dreader35.news.xs4all.nl>
Message-ID: <55759373$0$13010$c3e8da3$5496439d@news.astraweb.com>

On Mon, 8 Jun 2015 09:21 pm, Albert van der Horst wrote:


> Why is "slug.title" a valid decomposition of the total string>
> (Or is it?)

I'm afraid I don't understand the question.


> What is the ()-brackets doing? Does it force the execution of title,
> which gives something to be dotted onto slug etc. See below.

slug.title looks up an attribute called "title" attached to the object
called "slug". In this case, slug is a string, so slug.title finds the
string title method:


py> slug = "hello"
py> slug.title
<built-in method title of str object at 0xb7b036e0>


Then the round brackets (parentheses) calls that method with no arguments,
which returns a new string:

py> slug.title()
'Hello'



> I interpreted the question as about the associative of the
> dot operator.

Technically, dot is not an operator. I believe that the docs call it a
delimiter, but in once sense it is more than that because it also performs
an attribute lookup. 

In any case, the order of applying dots is *strictly* left-to-right.




-- 
Steven



From dfnsonfsduifb at gmx.de  Mon Jun  8 11:25:12 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Mon, 08 Jun 2015 17:25:12 +0200
Subject: Using ssl module over custom sockets
In-Reply-To: <9d27cf3f-3edc-4ff0-ba69-0fc588e049a6@googlegroups.com>
References: <9d27cf3f-3edc-4ff0-ba69-0fc588e049a6@googlegroups.com>
Message-ID: <ml4c4o$p3k$1@news.albasani.net>

On 08.06.2015 09:22, jbauer.usenet at gmail.com wrote:

> Something that I could always use as a workaround would be to open up a listening port locally in one thread and connection to that local port in a different thread, then forward packets. But that's pretty ugly and I'd like to avoid it.

Didn't actually have to do that. My solution now is to use
socket.socketpair() and forward traffic from there. Works reasonably
well and isn't racy (unlike opening up a local TCP/IP port would be).

So I think this solves the problem (although it'd still be cool to have
an alternative way of specifying a send/recv function IMHO).

Cheers,
Johannes


-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From lac at openend.se  Mon Jun  8 13:33:07 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 08 Jun 2015 19:33:07 +0200
Subject: Python Random vs. Cython C Rand for Dice Rolls
In-Reply-To: Message from "Steven D'Aprano" <steve@pearwood.info> of "Mon,
 08 Jun 2015 04:33:16 +1000."
 <55748e6d$0$13008$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.262.1433697446.13271.python-list@python.org><55748e6d$0$13008$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <201506081733.t58HX7Ft006182@fido.openend.se>

Better C random number generator.
http://www.pcg-random.org/download.html

Laura



From lac at openend.se  Mon Jun  8 14:21:41 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 08 Jun 2015 20:21:41 +0200
Subject: Detect if specific Python.app instance is already running
In-Reply-To: Message from Andrei <andrei.fokau@gmail.com> of "Mon,
 08 Jun 2015 01:32:40 -0700."
 <6651a781-abe1-4f25-b1f3-1f849776d060@googlegroups.com>
References: <a3be1707-0eae-46fa-9921-e9c145816ad5@googlegroups.com>
 <11e093d5-b78e-4ac6-9a7f-649cb2c2c942@googlegroups.com>
 <mailman.281.1433718479.13271.python-list@python.org><6651a781-abe1-4f25-b1f3-1f849776d060@googlegroups.com>
Message-ID: <201506081821.t58ILfKF008970@fido.openend.se>

I needed to do something like this once.  What I needed was a way to
send a process a signal, and have it then spit out a huge amount
of stats about how long it had been running, how many page faults
it had suffered, and, goodness, I forget all the information that
was needed.  Lots.  So I just wrapped the code in something that
waited to see if it got the special signal from hell, and, when it
did spit out the information.

It was not pretty, but that is the sort of thing you do when somebody
comes by after you have written something with some new requirements
that basically mean you would have to rewrite the thing.

Of course, sending odd signals to random processes in your proc table,
asking them to identify themselves, runs the risk that the process
isn't set up to receive signals it doesn't expect gracefully, and may
fall over dead when you do so.  So I am not sure I can actually
recommend this technique, but it is one way to skin this particular cat.

Laura



From skip.montanaro at gmail.com  Mon Jun  8 15:09:37 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 8 Jun 2015 14:09:37 -0500
Subject: Cheat sheet for the new string formatting?
Message-ID: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>

I have so far ignored the new string formatting (you know, the stuff with
all the braces, dots and brackets that make Python strings look like Perl
code <wink>). I am still only using Python 2.7, but have recently started
forcing myself to use the print() function. I figure maybe I should also
start to come to grips with the fancy new string formatting.

Is there a cheat sheet around which shows some side-by-side examples of the
{}-style and printf-style? I didn't see anything with a few Google searches.

Thx,

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150608/39df17a5/attachment.html>

From steven at knight.cx  Mon Jun  8 16:05:58 2015
From: steven at knight.cx (Steven K Knight)
Date: Mon, 08 Jun 2015 20:05:58 +0000
Subject: Cheat sheet for the new string formatting?
In-Reply-To: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
References: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
Message-ID: <cdede6267d6e8c525061570e6c355142@mail.knight.cx>

June 8 2015 3:11 PM, "Skip Montanaro"  wrote:

	I have so far ignored the new string formatting (you know, the stuff with all the braces, dots and brackets that make Python strings look like Perl code ). I am still only using Python 2.7, but have recently started forcing myself to use the print() function. I figure maybe I should also start to come to grips with the fancy new string formatting.

Is there a cheat sheet around which shows some side-by-side examples of the {}-style and printf-style? I didn't see anything with a few Google searches.

Thx,

Skip

Hi,

I think?http://pyformat.info/ (http://pyformat.info/)?is what you're looking for.

Thanks,
?
--
Steven Knight
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150608/9deadf36/attachment.html>

From skip.montanaro at gmail.com  Mon Jun  8 16:08:58 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 8 Jun 2015 15:08:58 -0500
Subject: Cheat sheet for the new string formatting?
In-Reply-To: <cdede6267d6e8c525061570e6c355142@mail.knight.cx>
References: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
 <cdede6267d6e8c525061570e6c355142@mail.knight.cx>
Message-ID: <CANc-5UyedA-nxdbfrvwg83oMtYCd+RRqJ-drg9eHh4SnQUaR4w@mail.gmail.com>

On Mon, Jun 8, 2015 at 3:05 PM, Steven K Knight <steven at knight.cx> wrote:

> I think http://pyformat.info/ is what you're looking for.


Perfect, thanks!

S
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150608/56402357/attachment.html>

From skip.montanaro at gmail.com  Mon Jun  8 16:32:19 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 8 Jun 2015 15:32:19 -0500
Subject: Cheat sheet for the new string formatting?
In-Reply-To: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
References: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
Message-ID: <CANc-5UxhxqE2P_U11Ut98Re6Eq+siXTXcrbtG+w09yX9UpvkRw@mail.gmail.com>

This is counterintuitive:

>>> "{:.3}".format(-0.00666762259822)
'-0.00667'
>>> "{:.3f}".format(-0.00666762259822)
'-0.007'
>>> "%.3f" % -0.00666762259822
'-0.007'
>>> "{:.3s}".format(-0.00666762259822)
ValueError Unknown format code 's' for object of type 'float'

Why does the first form display five digits after the decimal point? Why
don't floats support "{:.Ns}"? (I know I can use "{!s}".)

This is using a freshly hg pulled and updated 2.7  branch.

Thx,

S
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150608/d19df931/attachment.html>

From kaiser.yann at gmail.com  Mon Jun  8 16:47:28 2015
From: kaiser.yann at gmail.com (Yann Kaiser)
Date: Mon, 08 Jun 2015 20:47:28 +0000
Subject: Cheat sheet for the new string formatting?
In-Reply-To: <CANc-5UxhxqE2P_U11Ut98Re6Eq+siXTXcrbtG+w09yX9UpvkRw@mail.gmail.com>
References: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
 <CANc-5UxhxqE2P_U11Ut98Re6Eq+siXTXcrbtG+w09yX9UpvkRw@mail.gmail.com>
Message-ID: <CANUJvPUh54Djtq8ig9BCHw5gVVYauBDT73nL6uONO_x9Zkj0Lg@mail.gmail.com>

It just means significant digits in the general format, which alternates
between 10-exponent notation and plain decimal notation.

https://docs.python.org/3.4/library/string.html#format-specification-mini-language

    >>> '{:.3}'.format(0.0000356785)
    '3.57e-05'
    >>> '{:.3}'.format(0.00356785)
    '0.00357'

On Mon, 8 Jun 2015 at 22:33 Skip Montanaro <skip.montanaro at gmail.com> wrote:

> This is counterintuitive:
>
> >>> "{:.3}".format(-0.00666762259822)
> '-0.00667'
> >>> "{:.3f}".format(-0.00666762259822)
> '-0.007'
> >>> "%.3f" % -0.00666762259822
> '-0.007'
> >>> "{:.3s}".format(-0.00666762259822)
> ValueError Unknown format code 's' for object of type 'float'
>
> Why does the first form display five digits after the decimal point? Why
> don't floats support "{:.Ns}"? (I know I can use "{!s}".)
>
> This is using a freshly hg pulled and updated 2.7  branch.
>
> Thx,
>
> S
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150608/b7c1f77c/attachment.html>

From storchaka at gmail.com  Mon Jun  8 16:48:25 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 08 Jun 2015 23:48:25 +0300
Subject: Cheat sheet for the new string formatting?
In-Reply-To: <CANc-5UxhxqE2P_U11Ut98Re6Eq+siXTXcrbtG+w09yX9UpvkRw@mail.gmail.com>
References: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
 <CANc-5UxhxqE2P_U11Ut98Re6Eq+siXTXcrbtG+w09yX9UpvkRw@mail.gmail.com>
Message-ID: <ml4v2q$jqv$1@ger.gmane.org>

On 08.06.15 23:32, Skip Montanaro wrote:
> This is counterintuitive:
>
>  >>> "{:.3}".format(-0.00666762259822)
> '-0.00667'
>  >>> "{:.3f}".format(-0.00666762259822)
> '-0.007'
>  >>> "%.3f" % -0.00666762259822
> '-0.007'
>  >>> "{:.3s}".format(-0.00666762259822)
> ValueError Unknown format code 's' for object of type 'float'
>
> Why does the first form display five digits after the decimal point? Why
> don't floats support "{:.Ns}"? (I know I can use "{!s}".)

"{:.3}" for floats is equivalent to "{:.3g}".

 >>> "{:.3g}".format(-0.00666762259822)
'-0.00667'
 >>> "%.3g" % -0.00666762259822
'-0.00667'

Format code 's' would produce incorrect and meaningless output for floats.

 >>> "{!s:.3}".format(-0.00666762259822)
'-0.'
 >>> "%.3s" % -0.00666762259822
'-0.'
 >>> "{!s:.3}".format(-0.0000666762259822)
'-6.'
 >>> "%.3s" % -0.0000666762259822
'-6.'




From random832 at fastmail.us  Mon Jun  8 16:52:04 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Mon, 08 Jun 2015 16:52:04 -0400
Subject: Cheat sheet for the new string formatting?
In-Reply-To: <CANc-5UxhxqE2P_U11Ut98Re6Eq+siXTXcrbtG+w09yX9UpvkRw@mail.gmail.com>
References: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
 <CANc-5UxhxqE2P_U11Ut98Re6Eq+siXTXcrbtG+w09yX9UpvkRw@mail.gmail.com>
Message-ID: <1433796724.3238010.290150233.58C2103F@webmail.messagingengine.com>

On Mon, Jun 8, 2015, at 16:32, Skip Montanaro wrote:
> This is counterintuitive:
> 
> >>> "{:.3}".format(-0.00666762259822)
> '-0.00667'
> >>> "{:.3f}".format(-0.00666762259822)
> '-0.007'
> >>> "%.3f" % -0.00666762259822
> '-0.007'
> >>> "{:.3s}".format(-0.00666762259822)
> ValueError Unknown format code 's' for object of type 'float'
> 
> Why does the first form display five digits after the decimal point?

Because it's three significant figures. Using .3 alone with a float is
equivalent to %.3g, not %.3f

> Why don't floats support "{:.Ns}"? (I know I can use "{!s}".)

Why would they? The old style didn't support %.Ns either. If you want a
width like "%50s" it's {!s:50}. But if you're using !s (or %s with the
old style) you don't get to specify the significant digits, only the
field width, so it's not clear what old-style code you're trying to find
an equivalent to.


From sturla.molden at gmail.com  Mon Jun  8 17:44:53 2015
From: sturla.molden at gmail.com (Sturla Molden)
Date: Mon, 08 Jun 2015 23:44:53 +0200
Subject: Python Random vs. Cython C Rand for Dice Rolls
In-Reply-To: <201506081733.t58HX7Ft006182@fido.openend.se>
References: <mailman.262.1433697446.13271.python-list@python.org><55748e6d$0$13008$c3e8da3$5496439d@news.astraweb.com>
 <201506081733.t58HX7Ft006182@fido.openend.se>
Message-ID: <ml52ck$ajj$1@ger.gmane.org>

On 08/06/15 19:33, Laura Creighton wrote:

> Better C random number generator.
> http://www.pcg-random.org/download.html

Or for something less minimalistic, just grab randomkit.c and 
randomkit.h from NumPy, which implements the same Mersenne Twister as 
Python. That is what I usually do to get fast random numbers from Cython

https://github.com/numpy/numpy/tree/master/numpy/random/mtrand


Sturla



From breamoreboy at yahoo.co.uk  Mon Jun  8 18:20:07 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 08 Jun 2015 23:20:07 +0100
Subject: Cheat sheet for the new string formatting?
In-Reply-To: <cdede6267d6e8c525061570e6c355142@mail.knight.cx>
References: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
 <cdede6267d6e8c525061570e6c355142@mail.knight.cx>
Message-ID: <ml54ev$bk0$1@ger.gmane.org>

On 08/06/2015 21:05, Steven K Knight wrote:
>
>
> June 8 2015 3:11 PM, "Skip Montanaro" <skip.montanaro at gmail.com
> <mailto:%22Skip%20Montanaro%22%20<skip.montanaro at gmail.com>>> wrote:
>
>     I have so far ignored the new string formatting (you know, the stuff
>     with all the braces, dots and brackets that make Python strings look
>     like Perl code <wink>). I am still only using Python 2.7, but have
>     recently started forcing myself to use the print() function. I
>     figure maybe I should also start to come to grips with the fancy new
>     string formatting.
>
>     Is there a cheat sheet around which shows some side-by-side examples
>     of the {}-style and printf-style? I didn't see anything with a few
>     Google searches.
>     Thx,
>     Skip
>
>
> Hi,
>
> I think http://pyformat.info/ is what you're looking for.
>
> Thanks,
> ?
> --
> Steven Knight <steven at knight.cx <mailto:steven at knight.cx>>
>

Many thanks for the link, just what the doctor ordered :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From skybuck2000 at hotmail.com  Mon Jun  8 22:30:13 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 9 Jun 2015 04:30:13 +0200
Subject: Parser needed.
In-Reply-To: <4252d$55709249$5419aafe$25285@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
 <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
 <4252d$55709249$5419aafe$25285@news.ziggo.nl>
Message-ID: <74396$55764fb5$5419aafe$6142@news.ziggo.nl>

I made it way too difficult on myself with that stupid dictionary bs...

What I really wanted was to know if the ref was already in the reflist.

Turns out python has a really nice simple operation for that:

   if not (Ref in EntityRef):
    EntityRef.append(Ref)

DONE ! =D

No need for dictionary.

Anyway... I am trying a more robust parser... because my own parser right 
now didn't work out for new inputs.

It almost worked except for first item... don't know what problem was.... 
maybe just this...

But I'll try and do it the usually way.

"Tokenize", "Parse" etc.

It's significantly slower though.

Maybe an idea for a new kind of parser could be:

Find all recgonized token keywords in one go... and stuff all find indexes 
into a list... and then perhaps sort the list...

and then use the sorted list... to return token found at that position...

and then start processing like that...

Benefit of this idea is less characters to compare... and could be done with 
just integer compare... or even lookup table processing.

Added benefit is still in-order which is nice.

For now I won't do it... cause I am not sure it would be an improvemt.

Another idea which might be an improvement is.

Parallel searching... ofcourse that might not be possible... though... multi 
core does it exist and threading too.

But to mimic parallel searching and to prevent problems.

A sliding window approach could be taken.

And perhaps items found that way in a certain buffer or so and then still 
added to processing list or so...

which kinda mimics parallel search... but just uses data cache nicely.

Though it's somewhat of a complex method... I will avoid it for now.

The split() routine is real nice... to get rid of fudd/white space... and 
just tokens which is nice.

So for now I will use that as my tokenizer ;) =D

and bracket level counting and sections and stuff like that yeah...

Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Mon Jun  8 22:31:01 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 9 Jun 2015 04:31:01 +0200
Subject: Parser needed.
In-Reply-To: <4252d$55709249$5419aafe$25285@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
 <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
 <4252d$55709249$5419aafe$25285@news.ziggo.nl>
Message-ID: <97a06$55764fe4$5419aafe$6212@news.ziggo.nl>

Oh I think I forgot to mention... parser is now getting close to 1 second... 
with tokenizer and such.

But I think this is still within acceptable performance level for now.

Bye,
  Skybuck.




From ryexander at gmail.com  Tue Jun  9 00:42:49 2015
From: ryexander at gmail.com (Ben Powers)
Date: Mon, 8 Jun 2015 22:42:49 -0600
Subject: Pyitect - Plugin architectural system for Python 3.0+ (feedback?)
Message-ID: <CAO-rxQsTP8k_nmaV80Zg2OZnbun040jpYG7_tLQpb-q22V3jSw@mail.gmail.com>

As importlib has been added in python 3 and up I decided to use it's
abilities to create a plugin system for truly modular development in
python.

Pyitect has the ability to drop in components and resolve dependencies.
Even load different versions of a dependency if two different libraries
require two different versions.

Pyitect gives each "plugin" it won private namespace to import the
components it needs. and nothing else.

Pyitect  organizes it self around the concept of a System. Each system can
scan and load it's own set of plugins, in each system plugins can be
enabled individually.

Pyitect even comes with a very simple event system that it uses internal to
fire event when plugins are found/loaded/ect for the developer to use.

I've written up What I think is a quite comprehensive readme to go with it.

Available on PyPi https://pypi.python.org/pypi?:action=display&name=pyitect
and Github https://github.com/Ryex/pyitect

I'm looking for feedback on what flaws the system may have and where I can
improve. I'm always open to pull requests

Looking forward to what the comunity thinks of this little project of mine.
Thanks for your input and criticism.

~Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150608/e39e954b/attachment.html>

From mal at europython.eu  Tue Jun  9 04:03:25 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Tue, 09 Jun 2015 10:03:25 +0200
Subject: EuroPython 2015: Educational Summit
Message-ID: <55769DCD.8020200@europython.eu>

We are proud to announce an educational summit at EuroPython
2015. This is being organized together with the Raspberry Pi
Foundation (https://www.raspberrypi.org/).


            *** EuroPython 2015 Educational Summit ***

    https://ep2015.europython.eu/en/events/educational-summit/


We will have Educational Summit focused talks, trainings, birds of a
feather sessions to debate and also Educational Sprints for the
building of education focused projects during the weekend.

Reduced ticket prices for teachers and kids
-------------------------------------------

To to facilitate attendance, teacher can attend the conference at the
reduced student rate. If you want to bring along kids, we also have
special tickets for them.

Spanish track
-------------

For those who don?t feel comfortable with English, we also provide
Spanish tracks to get you up to speed with Python.

Summary of the agenda
---------------------

 * July 21 - Trainings in Spanish (including an introduction to Python)
 * July 22 - Talks in Spanish
 * July 23 - Educational Summit Talks (English, all day)
             + Trainings (English, afternoon)
 * July 24 - relax, discuss, attend other conference talks
 * July 25 - Educational Summit Sprints
 * July 26 - Educational Summit Sprints

More information is available on the Education Summit page. If you
have questions, please write to helpdesk at europython.eu.

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From ryan.stuart.85 at gmail.com  Tue Jun  9 05:38:51 2015
From: ryan.stuart.85 at gmail.com (Ryan Stuart)
Date: Tue, 9 Jun 2015 19:38:51 +1000
Subject: Parser needed.
In-Reply-To: <74396$55764fb5$5419aafe$6142@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
 <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
 <4252d$55709249$5419aafe$25285@news.ziggo.nl>
 <74396$55764fb5$5419aafe$6142@news.ziggo.nl>
Message-ID: <CAM8kh5t-ShpDOdmNfcY4XQSnhNU5DnymWwXVs8tvV2WgFu4VXw@mail.gmail.com>

On Tue, Jun 9, 2015 at 12:30 PM, Skybuck Flying <skybuck2000 at hotmail.com>
wrote:

> Anyway... I am trying a more robust parser... because my own parser right
> now didn't work out for new inputs.
>

You should take a look at lrparsing:
https://pypi.python.org/pypi/lrparsing/1.0.11

Cheers


>
> It almost worked except for first item... don't know what problem was....
> maybe just this...
>
> But I'll try and do it the usually way.
>
> "Tokenize", "Parse" etc.
>
> It's significantly slower though.
>
> Maybe an idea for a new kind of parser could be:
>
> Find all recgonized token keywords in one go... and stuff all find indexes
> into a list... and then perhaps sort the list...
>
> and then use the sorted list... to return token found at that position...
>
> and then start processing like that...
>
> Benefit of this idea is less characters to compare... and could be done
> with just integer compare... or even lookup table processing.
>
> Added benefit is still in-order which is nice.
>
> For now I won't do it... cause I am not sure it would be an improvemt.
>
> Another idea which might be an improvement is.
>
> Parallel searching... ofcourse that might not be possible... though...
> multi core does it exist and threading too.
>
> But to mimic parallel searching and to prevent problems.
>
> A sliding window approach could be taken.
>
> And perhaps items found that way in a certain buffer or so and then still
> added to processing list or so...
>
> which kinda mimics parallel search... but just uses data cache nicely.
>
> Though it's somewhat of a complex method... I will avoid it for now.
>
> The split() routine is real nice... to get rid of fudd/white space... and
> just tokens which is nice.
>
> So for now I will use that as my tokenizer ;) =D
>
> and bracket level counting and sections and stuff like that yeah...
>
>
> Bye,
>  Skybuck.
> --
> https://mail.python.org/mailman/listinfo/python-list
>



-- 
Ryan Stuart, B.Eng
Software Engineer

ABN: 81-206-082-133
W: http://www.textisbeautiful.net/
M: +61-431-299-036
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150609/75da50f2/attachment.html>

From skip.montanaro at gmail.com  Tue Jun  9 08:15:48 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 9 Jun 2015 07:15:48 -0500
Subject: Cheat sheet for the new string formatting?
In-Reply-To: <1433796724.3238010.290150233.58C2103F@webmail.messagingengine.com>
References: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
 <CANc-5UxhxqE2P_U11Ut98Re6Eq+siXTXcrbtG+w09yX9UpvkRw@mail.gmail.com>
 <1433796724.3238010.290150233.58C2103F@webmail.messagingengine.com>
Message-ID: <CANc-5Uz-ti4cpjdWCMagc6hva0b8TQ949RNv6D+y9B39_wfQLA@mail.gmail.com>

Skip> Why don't floats support "{:.Ns}"? (I know I can use "{!s}".)

random832> Why would they? The old style didn't support %.Ns either.

Well, the old style does, though it appears the N is ignored:

>>> "%5s" % -0.00666762259822
'-0.00666762259822'

It doesn't raise an exception though.

(This is with Python 2.7.2. Haven't tried to see if that behavior
changed with 3.x.)

One thing which seems obvious now is that since format() delegates to
the individual types for formatting, much of the documentation of this
stuff must now be delegated to the individual types. However, I can't
find anything about the formatting syntax supported by
float.__format__.  Where might I find it documented that "{:.3}" is
synonymous with "%.3g"?

Skip


From skybuck2000 at hotmail.com  Tue Jun  9 08:20:46 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 9 Jun 2015 14:20:46 +0200
Subject: Parser needed.
In-Reply-To: <mailman.310.1433842760.13271.python-list@python.org>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
 <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
 <4252d$55709249$5419aafe$25285@news.ziggo.nl>
 <74396$55764fb5$5419aafe$6142@news.ziggo.nl>
 <mailman.310.1433842760.13271.python-list@python.org>
Message-ID: <48a90$5576da1e$5419aafe$34050@news.ziggo.nl>

Euhm...

My parser is already done... since today

Loving it too

Wrote it myself... based on the c# code technique explained somewhere in 
this thread too

Bye,
  Skybuck.




From random832 at fastmail.us  Tue Jun  9 08:29:23 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 09 Jun 2015 08:29:23 -0400
Subject: Cheat sheet for the new string formatting?
In-Reply-To: <CANc-5Uz-ti4cpjdWCMagc6hva0b8TQ949RNv6D+y9B39_wfQLA@mail.gmail.com>
References: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
 <CANc-5UxhxqE2P_U11Ut98Re6Eq+siXTXcrbtG+w09yX9UpvkRw@mail.gmail.com>
 <1433796724.3238010.290150233.58C2103F@webmail.messagingengine.com>
 <CANc-5Uz-ti4cpjdWCMagc6hva0b8TQ949RNv6D+y9B39_wfQLA@mail.gmail.com>
Message-ID: <1433852963.3476185.290722345.69E5EA76@webmail.messagingengine.com>

On Tue, Jun 9, 2015, at 08:15, Skip Montanaro wrote:
> Skip> Why don't floats support "{:.Ns}"? (I know I can use "{!s}".)
> 
> random832> Why would they? The old style didn't support %.Ns either.
> 
> Well, the old style does, though it appears the N is ignored:
> 
> >>> "%5s" % -0.00666762259822
> '-0.00666762259822'

There's no dot there. I was mistaken, though, it does work.

>>> "%.5s" % -0.00666762259822
'-0.00'

The equivalent new-style format is {!s:.5} - the reason is that !s is
fundamentally different from other formats since it performs a type
conversion rather than delegating to the argument's __format__ method.
Whereas _all_ old-style format specifiers did type conversions [see
'%.23f' % Decimal('0.1') vs '{:.23f}'.format(Decimal('0.1'))]

> One thing which seems obvious now is that since format() delegates to
> the individual types for formatting, much of the documentation of this
> stuff must now be delegated to the individual types. However, I can't
> find anything about the formatting syntax supported by
> float.__format__.  Where might I find it documented that "{:.3}" is
> synonymous with "%.3g"?

Where have you looked? Have you read
https://docs.python.org/3/library/string.html#formatspec ?

None    Similar to 'g', except that fixed-point notation, when used, has
at least one digit past the decimal point. The default precision is as
high as needed to represent the particular value. The overall effect is
to match the output of str() as altered by the other format modifiers.


From skip.montanaro at gmail.com  Tue Jun  9 09:13:41 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 9 Jun 2015 08:13:41 -0500
Subject: Cheat sheet for the new string formatting?
In-Reply-To: <1433852963.3476185.290722345.69E5EA76@webmail.messagingengine.com>
References: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
 <CANc-5UxhxqE2P_U11Ut98Re6Eq+siXTXcrbtG+w09yX9UpvkRw@mail.gmail.com>
 <1433796724.3238010.290150233.58C2103F@webmail.messagingengine.com>
 <CANc-5Uz-ti4cpjdWCMagc6hva0b8TQ949RNv6D+y9B39_wfQLA@mail.gmail.com>
 <1433852963.3476185.290722345.69E5EA76@webmail.messagingengine.com>
Message-ID: <CANc-5UymHNW7aLoHu19yufbQierKKXx-GSeqz3=_g5Q-94LFQw@mail.gmail.com>

On Tue, Jun 9, 2015 at 7:29 AM,  <random832 at fastmail.us> wrote:
> Where have you looked? Have you read
> https://docs.python.org/3/library/string.html#formatspec ?

Yes, but I missed the None section. I looked closely at 'g', but
didn't see anything like "this is the default". I will admit I was a
bit frustrated to see things like the documentation for the format()
builtin indicating that it calls value.__format__, but then couldn't
find any documentation in the numeric types for float.__format__. From
the documentation of the format() builtin: "The interpretation of
format_spec will depend on the type of the value argument...", which
suggested to me that at least the builtin types would have some stub
documentation of their __format__ method, even if all it did was
redirect the user to the section in the string doc:
https://docs.python.org/3.5/library/string.html#format-string-syntax

I'll open a doc issue about that. Thanks for the help.

Skip


From bushl at teachco.com  Tue Jun  9 10:19:01 2015
From: bushl at teachco.com (Leslie Bush)
Date: Tue, 9 Jun 2015 14:19:01 +0000
Subject: Permission to Showcase the Python Program
Message-ID: <D19C6E14.35A2%bushl@teachco.com>

I?m having trouble reaching an actual human at your organization as my emails get bounced back.

If someone could please see my email below and respond with an answer as soon as possible, I?d appreciate it.
--------------------------------------------------------------------------------------------------------------------------------------------------------



To Whom It May Concern:

My name is Leslie Bush and I am the Intellectual Property Coordinator for The Great Courses. We produce non-credit, college-level educational programs on DVD and electronic formats in a lecture series. The lectures are recorded and then sold to the general public for-profit. As a didactic tool for enhancing the programs, we include in the lectures visual elements illustrating works of art, people, events, locations, etc....

We are currently producing a course with Dr. John Keyser, Professor of Computer Science at the University of North Carolina Chapel Hill, entitled ?Computer Science for Everyone: Programming Concepts and Exercises?  The professor would like to use your Python program to showcase in the course. I am writing to see if we may have permission to do so. If permission is granted, we will have a copy of our license agreement sent over to your company?s authorizer for review and a signature.  Here are the details of our program:

Title: Computer Science for Everyone: Programming Concepts and Exercises
Author and Publisher: The Teaching Company
Language: All
Format: all electronic formats
Distribution: Worldwide
Print Run: Life of the product
Lecturer: Dr. John Keyser
Release date: Summer 2016
Price: Unknown

If you need further details or have questions or concerns, please do not hesitate to contact me.  Our address is 4840 Westfields Blvd. Chantilly, VA 20151. For more information about The Teaching Company you can look at our site at www.thegreatcourses.com

If you are interested in seeing any of the courses on our site please let me know and I?ll be happy to send you a copy. I look forward to working with you on this exciting course. Thank you for your assistance.

Sincerely,
Leslie Bush
Product Development Intellectual Property Coordinator
The Teaching Company/The Great Courses
4840 Westfields Blvd., Suite 500, Chantilly, VA 20151-2299
(703)774-1687 Direct
(703)502-4270 Fax
www.thegreatcourses.com



From breamoreboy at yahoo.co.uk  Tue Jun  9 11:42:24 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 09 Jun 2015 16:42:24 +0100
Subject: Cheat sheet for the new string formatting?
In-Reply-To: <CANc-5Uz-ti4cpjdWCMagc6hva0b8TQ949RNv6D+y9B39_wfQLA@mail.gmail.com>
References: <CANc-5Uyv8Ma3G-JfRh7PT-yf4QzJaWnqgyYN9c8=mrnGx_5yRQ@mail.gmail.com>
 <CANc-5UxhxqE2P_U11Ut98Re6Eq+siXTXcrbtG+w09yX9UpvkRw@mail.gmail.com>
 <1433796724.3238010.290150233.58C2103F@webmail.messagingengine.com>
 <CANc-5Uz-ti4cpjdWCMagc6hva0b8TQ949RNv6D+y9B39_wfQLA@mail.gmail.com>
Message-ID: <ml71h9$f2q$1@ger.gmane.org>

On 09/06/2015 13:15, Skip Montanaro wrote:
>
> One thing which seems obvious now is that since format() delegates to
> the individual types for formatting, much of the documentation of this
> stuff must now be delegated to the individual types. However, I can't
> find anything about the formatting syntax supported by
> float.__format__.  Where might I find it documented that "{:.3}" is
> synonymous with "%.3g"?
>

You won't, at least not that I'm aware of.  I think part of the problem 
is that there was a lot of confusion as to whether or not the old style 
formatting was to be deprecated.  See http://bugs.python.org/issue14123 
which refers to http://www.gossamer-threads.com/lists/python/dev/969817

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From ndbecker2 at gmail.com  Tue Jun  9 14:08:25 2015
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 09 Jun 2015 14:08:25 -0400
Subject: enhancement request: make py3 read/write py2 pickle format
Message-ID: <ml7a2p$hp$1@ger.gmane.org>

One of the most annoying problems with py2/3 interoperability is that the 
pickle formats are not compatible.  There must be many who, like myself, 
often use pickle format for data storage.

It certainly would be a big help if py3 could read/write py2 pickle format.  
You know, backward compatibility? 



From breamoreboy at yahoo.co.uk  Tue Jun  9 14:25:09 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 09 Jun 2015 19:25:09 +0100
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <ml7a2p$hp$1@ger.gmane.org>
References: <ml7a2p$hp$1@ger.gmane.org>
Message-ID: <ml7b2e$io1$1@ger.gmane.org>

On 09/06/2015 19:08, Neal Becker wrote:
> One of the most annoying problems with py2/3 interoperability is that the
> pickle formats are not compatible.  There must be many who, like myself,
> often use pickle format for data storage.
>
> It certainly would be a big help if py3 could read/write py2 pickle format.
> You know, backward compatibility?
>

http://bugs.python.org/issue13566

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lac at openend.se  Tue Jun  9 14:31:36 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 09 Jun 2015 20:31:36 +0200
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: Message from Neal Becker <ndbecker2@gmail.com>
 of "Tue, 09 Jun 2015 14:08:25 -0400." <ml7a2p$hp$1@ger.gmane.org>
References: <ml7a2p$hp$1@ger.gmane.org>
Message-ID: <201506091831.t59IVafp009018@fido.openend.se>

In a message of Tue, 09 Jun 2015 14:08:25 -0400, Neal Becker writes:
>One of the most annoying problems with py2/3 interoperability is that the 
>pickle formats are not compatible.  There must be many who, like myself, 
>often use pickle format for data storage.
>
>It certainly would be a big help if py3 could read/write py2 pickle format.  
>You know, backward compatibility? 
>
>-- 
>https://mail.python.org/mailman/listinfo/python-list

We have an issue about that.
https://bugs.python.org/issue13566

Go there and say you want it too. :)

Laura


From kwpolska at gmail.com  Tue Jun  9 14:35:02 2015
From: kwpolska at gmail.com (Chris Warrick)
Date: Tue, 9 Jun 2015 20:35:02 +0200
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <ml7a2p$hp$1@ger.gmane.org>
References: <ml7a2p$hp$1@ger.gmane.org>
Message-ID: <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>

On Tue, Jun 9, 2015 at 8:08 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
> One of the most annoying problems with py2/3 interoperability is that the
> pickle formats are not compatible.  There must be many who, like myself,
> often use pickle format for data storage.
>
> It certainly would be a big help if py3 could read/write py2 pickle format.
> You know, backward compatibility?

Don?t use pickle. It?s unsafe ? it executes arbitrary code, which
means someone can give you a pickle file that will delete all your
files or eat your cat.

Instead, use a safe format that has no ability to execute code, like
JSON. It will also work with other programming languages and
environments if you ever need to talk to anyone else.

But, FYI: there is backwards compatibility if you ask for it, in the
form of protocol versions. That?s all you should know ? again, don?t
use pickle.

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16


From zachary.ware+pylist at gmail.com  Tue Jun  9 14:38:11 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Tue, 9 Jun 2015 13:38:11 -0500
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <ml7a2p$hp$1@ger.gmane.org>
References: <ml7a2p$hp$1@ger.gmane.org>
Message-ID: <CAKJDb-MJ9iL5yy1bJ2JrOR0v8LEMzKL5qEkMOwERQKT1ijAEcQ@mail.gmail.com>

On Tue, Jun 9, 2015 at 1:08 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
> One of the most annoying problems with py2/3 interoperability is that the
> pickle formats are not compatible.  There must be many who, like myself,
> often use pickle format for data storage.
>
> It certainly would be a big help if py3 could read/write py2 pickle format.
> You know, backward compatibility?

Uhh...

$ python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> with open('test.pkl', 'wb') as f:
...  pickle.dump({'test': [1, 2, {3}]}, f)
...
>>> with open('test.pkl', 'rb') as f:
...  pickle.load(f)
...
{'test': [1, 2, set([3])]}
>>> ^D
$ python3
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 23 2015, 02:52:03)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> with open('test.pkl', 'rb') as f:
...  pickle.load(f)
...
{'test': [1, 2, {3}]}
>>> with open('test2.pkl', 'wb') as f:
...  pickle.dump(['test', {2: {3.4}}], f, protocol=2)
...
>>> with open('test2.pkl', 'rb') as f:
...  pickle.load(f)
...
['test', {2: {3.4}}]
>>> ^D
? ~
13:35 $ python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> with open('test2.pkl', 'rb') as f:
...  pickle.load(f)
...
[u'test', {2: set([3.4])}]
>>>

-- 
Zach


From storchaka at gmail.com  Tue Jun  9 15:51:00 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 09 Jun 2015 22:51:00 +0300
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <ml7a2p$hp$1@ger.gmane.org>
References: <ml7a2p$hp$1@ger.gmane.org>
Message-ID: <ml7g35$buf$1@ger.gmane.org>

On 09.06.15 21:08, Neal Becker wrote:
> One of the most annoying problems with py2/3 interoperability is that the
> pickle formats are not compatible.  There must be many who, like myself,
> often use pickle format for data storage.
>
> It certainly would be a big help if py3 could read/write py2 pickle format.
> You know, backward compatibility?

Pickle format is mostly compatible. What is your issue?




From storchaka at gmail.com  Tue Jun  9 15:56:19 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 09 Jun 2015 22:56:19 +0300
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <201506091831.t59IVafp009018@fido.openend.se>
References: <ml7a2p$hp$1@ger.gmane.org>
 <201506091831.t59IVafp009018@fido.openend.se>
Message-ID: <ml7gd3$hac$1@ger.gmane.org>

On 09.06.15 21:31, Laura Creighton wrote:
> We have an issue about that.
> https://bugs.python.org/issue13566
>
> Go there and say you want it too. :)

I afraid issue title is too general. :) The issue is only about one 
minor detail of py3 to py2 compatibility.



From jeanpierreda at gmail.com  Tue Jun  9 16:07:26 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 9 Jun 2015 13:07:26 -0700
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
Message-ID: <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>

There's a lot of subtle issues with pickle compatibility. e.g.
old-style vs new-style classes. It's kinda hard and it's better to
give up. I definitely agree it's better to use something else instead.
For example, we switched to using protocol buffers, which have much
better compatibility properties and are a bit more testable to boot
(since text format protobufs are always output in a canonical (sorted)
form.)

-- Devin

On Tue, Jun 9, 2015 at 11:35 AM, Chris Warrick <kwpolska at gmail.com> wrote:
> On Tue, Jun 9, 2015 at 8:08 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
>> One of the most annoying problems with py2/3 interoperability is that the
>> pickle formats are not compatible.  There must be many who, like myself,
>> often use pickle format for data storage.
>>
>> It certainly would be a big help if py3 could read/write py2 pickle format.
>> You know, backward compatibility?
>
> Don?t use pickle. It?s unsafe ? it executes arbitrary code, which
> means someone can give you a pickle file that will delete all your
> files or eat your cat.
>
> Instead, use a safe format that has no ability to execute code, like
> JSON. It will also work with other programming languages and
> environments if you ever need to talk to anyone else.
>
> But, FYI: there is backwards compatibility if you ask for it, in the
> form of protocol versions. That?s all you should know ? again, don?t
> use pickle.
>
> --
> Chris Warrick <https://chriswarrick.com/>
> PGP: 5EAAEA16
> --
> https://mail.python.org/mailman/listinfo/python-list


From hongyi.zhao at gmail.com  Tue Jun  9 18:21:20 2015
From: hongyi.zhao at gmail.com (Hongyi Zhao)
Date: Tue, 9 Jun 2015 22:21:20 +0000 (UTC)
Subject: Want to readout the control file of aria2.
Message-ID: <ml7osv$aog$2@aspen.stu.neva.ru>

Hi all,

I want to read out the control file of aria2, which is a binary file 
including the data format depicted here:

http://aria2.sourceforge.net/manual/en/html/technical-notes.html

Could you please give me some snipped codes for this purpose?

Regards
-- 
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.





-- 
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.


From joel.goldstick at gmail.com  Tue Jun  9 18:52:00 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 9 Jun 2015 18:52:00 -0400
Subject: Want to readout the control file of aria2.
In-Reply-To: <ml7osv$aog$2@aspen.stu.neva.ru>
References: <ml7osv$aog$2@aspen.stu.neva.ru>
Message-ID: <CAPM-O+xrG00zki5npGNPO63BE3qLqNABCe9WmCbZ13oJW55Q7A@mail.gmail.com>

On Tue, Jun 9, 2015 at 6:21 PM, Hongyi Zhao <hongyi.zhao at gmail.com> wrote:
> Hi all,
>
> I want to read out the control file of aria2, which is a binary file
> including the data format depicted here:
>
> http://aria2.sourceforge.net/manual/en/html/technical-notes.html
>
> Could you please give me some snipped codes for this purpose?
>
> Regards
> --
> .: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.
>
>
>
>
>
> --
> .: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.
> --
> https://mail.python.org/mailman/listinfo/python-list

I found this: http://aria.pasteur.fr/developers/aria-python-api

Does that help?


-- 
Joel Goldstick
http://joelgoldstick.com


From lac at openend.se  Tue Jun  9 18:58:41 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 10 Jun 2015 00:58:41 +0200
Subject: Want to readout the control file of aria2.
In-Reply-To: Message from Hongyi Zhao <hongyi.zhao@gmail.com>
 of "Tue, 09 Jun 2015 22:21:20 -0000." <ml7osv$aog$2@aspen.stu.neva.ru>
References: <ml7osv$aog$2@aspen.stu.neva.ru>
Message-ID: <201506092258.t59MwfRU015032@fido.openend.se>

No promises -- I never used this myself, but it looks like:

https://github.com/killuahzl/pyaria2

parses pyaria files for its own purposes, so you could steal code from there.

Laura



From rosuav at gmail.com  Tue Jun  9 19:06:27 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 10 Jun 2015 09:06:27 +1000
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
Message-ID: <CAPTjJmrS_3vYybXo_xsUrmQdimekyUcCd_+uzTh6H_4=JtqTbA@mail.gmail.com>

On Wed, Jun 10, 2015 at 6:07 AM, Devin Jeanpierre
<jeanpierreda at gmail.com> wrote:
> There's a lot of subtle issues with pickle compatibility. e.g.
> old-style vs new-style classes. It's kinda hard and it's better to
> give up. I definitely agree it's better to use something else instead.
> For example, we switched to using protocol buffers, which have much
> better compatibility properties and are a bit more testable to boot
> (since text format protobufs are always output in a canonical (sorted)
> form.)

Or use JSON, if your data fits within that structure. It's easy to
read and write, it's human-readable, and it's safe (no chance of
arbitrary code execution). Forcing yourself to use a format that can
basically be processed by ast.literal_eval() is a good discipline -
means you don't accidentally save/load too much.

ChrisA


From hongyi.zhao at gmail.com  Tue Jun  9 19:19:43 2015
From: hongyi.zhao at gmail.com (Hongyi Zhao)
Date: Tue, 9 Jun 2015 23:19:43 +0000 (UTC)
Subject: Want to readout the control file of aria2.
References: <ml7osv$aog$2@aspen.stu.neva.ru>
 <mailman.323.1433890323.13271.python-list@python.org>
Message-ID: <ml7saf$aog$3@aspen.stu.neva.ru>

On Tue, 09 Jun 2015 18:52:00 -0400, Joel Goldstick wrote:

> I found this: http://aria.pasteur.fr/developers/aria-python-api
> 
> Does that help?

The aria mentioned by your above links is completely different thing WRT 
the one I posted here ;-(

Regards
-- 
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.


From joel.goldstick at gmail.com  Tue Jun  9 19:21:30 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Tue, 9 Jun 2015 19:21:30 -0400
Subject: Want to readout the control file of aria2.
In-Reply-To: <ml7saf$aog$3@aspen.stu.neva.ru>
References: <ml7osv$aog$2@aspen.stu.neva.ru>
 <mailman.323.1433890323.13271.python-list@python.org>
 <ml7saf$aog$3@aspen.stu.neva.ru>
Message-ID: <CAPM-O+xaiLr9_eZ_FAPPz1SDPUHVpyc9pnM37ZmJUu_BXKYVZA@mail.gmail.com>

On Tue, Jun 9, 2015 at 7:19 PM, Hongyi Zhao <hongyi.zhao at gmail.com> wrote:
> On Tue, 09 Jun 2015 18:52:00 -0400, Joel Goldstick wrote:
>
>> I found this: http://aria.pasteur.fr/developers/aria-python-api
>>
>> Does that help?
>
> The aria mentioned by your above links is completely different thing WRT
> the one I posted here ;-(
>
> Regards
> --
> .: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.
> --
> https://mail.python.org/mailman/listinfo/python-list

Well, maybe Laura's link will be what you need?

-- 
Joel Goldstick
http://joelgoldstick.com


From hongyi.zhao at gmail.com  Tue Jun  9 19:29:43 2015
From: hongyi.zhao at gmail.com (Hongyi Zhao)
Date: Tue, 9 Jun 2015 23:29:43 +0000 (UTC)
Subject: Want to readout the control file of aria2.
References: <ml7osv$aog$2@aspen.stu.neva.ru>
 <mailman.324.1433890734.13271.python-list@python.org>
Message-ID: <ml7st6$aog$4@aspen.stu.neva.ru>

On Wed, 10 Jun 2015 00:58:41 +0200, Laura Creighton wrote:

> No promises -- I never used this myself, but it looks like:
> 
> https://github.com/killuahzl/pyaria2
> 
> parses pyaria files for its own purposes, so you could steal code from
> there.
> 
> Laura

Thanks, but it doesn't do the same thing that I want.

Regards
-- 
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.


From hongyi.zhao at gmail.com  Tue Jun  9 19:32:00 2015
From: hongyi.zhao at gmail.com (Hongyi Zhao)
Date: Tue, 9 Jun 2015 23:32:00 +0000 (UTC)
Subject: Want to readout the control file of aria2.
References: <ml7osv$aog$2@aspen.stu.neva.ru>
 <mailman.323.1433890323.13271.python-list@python.org>
 <ml7saf$aog$3@aspen.stu.neva.ru>
 <mailman.326.1433892098.13271.python-list@python.org>
Message-ID: <ml7t1g$aog$5@aspen.stu.neva.ru>

On Tue, 09 Jun 2015 19:21:30 -0400, Joel Goldstick wrote:

>> https://mail.python.org/mailman/listinfo/python-list
> 
> Well, maybe Laura's link will be what you need?

Not still.  Laura's link is a python script which interactively using 
aria2 for downloading via rpc.  I want to readout the binary control file 
of aria2 ( *.aria2 ).

Regards
-- 
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.


From torriem at gmail.com  Tue Jun  9 19:33:19 2015
From: torriem at gmail.com (Michael Torrie)
Date: Tue, 09 Jun 2015 17:33:19 -0600
Subject: Parser needed.
In-Reply-To: <48a90$5576da1e$5419aafe$34050@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
 <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
 <4252d$55709249$5419aafe$25285@news.ziggo.nl>
 <74396$55764fb5$5419aafe$6142@news.ziggo.nl>
 <mailman.310.1433842760.13271.python-list@python.org>
 <48a90$5576da1e$5419aafe$34050@news.ziggo.nl>
Message-ID: <557777BF.4010800@gmail.com>

On 06/09/2015 06:20 AM, Skybuck Flying wrote:
> Euhm...
> 
> My parser is already done... since today
> 
> Loving it too
> 
> Wrote it myself... based on the c# code technique explained somewhere in 
> this thread too

I'm glad you're having fun, and making good progress.  And it's good to
hear of success with Python.  However, I wouldn't really call this a
"thread;" more a "monologue."  I'm not sure list members really
appreciate the blow-by-blow description of a project, for the future.  I
know I find it rather noisy.  I don't think anyone has actually read any
of your messages (I'm pretty sure some have just blacklisted your
messages), except to determine you're trying to parse something rather
vague.  Might I suggest a blog is a more appropriate place to ruminate
over a project (and posting code and examples) and use the list for
questions and advice?


From hongyi.zhao at gmail.com  Tue Jun  9 19:45:35 2015
From: hongyi.zhao at gmail.com (Hongyi Zhao)
Date: Tue, 9 Jun 2015 23:45:35 +0000 (UTC)
Subject: Want to readout the control file of aria2.
References: <ml7osv$aog$2@aspen.stu.neva.ru>
 <mailman.324.1433890734.13271.python-list@python.org>
 <ml7st6$aog$4@aspen.stu.neva.ru>
Message-ID: <ml7tqv$aog$6@aspen.stu.neva.ru>

On Tue, 09 Jun 2015 23:29:43 +0000, Hongyi Zhao wrote:

> On Wed, 10 Jun 2015 00:58:41 +0200, Laura Creighton wrote:
> 
>> No promises -- I never used this myself, but it looks like:
>> 
>> https://github.com/killuahzl/pyaria2
>> 
>> parses pyaria files for its own purposes, so you could steal code from
>> there.
>> 
>> Laura
> 
> Thanks, but it doesn't do the same thing that I want.
> 
> Regards

Thanks again, based on your hints, I do a further search on the github.  
And meet the following one:

https://github.com/atmouse-/aria2-trunc/blob/master/aria2_cutfile.py

It seems this one is just do the things that I want. I'll read it further 
for confirmation.

Regards
-- 
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.


From irmen.NOSPAM at xs4all.nl  Tue Jun  9 20:17:10 2015
From: irmen.NOSPAM at xs4all.nl (Irmen de Jong)
Date: Wed, 10 Jun 2015 02:17:10 +0200
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <mailman.325.1433891190.13271.python-list@python.org>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
Message-ID: <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>

On 10-6-2015 1:06, Chris Angelico wrote:
> On Wed, Jun 10, 2015 at 6:07 AM, Devin Jeanpierre
> <jeanpierreda at gmail.com> wrote:
>> There's a lot of subtle issues with pickle compatibility. e.g.
>> old-style vs new-style classes. It's kinda hard and it's better to
>> give up. I definitely agree it's better to use something else instead.
>> For example, we switched to using protocol buffers, which have much
>> better compatibility properties and are a bit more testable to boot
>> (since text format protobufs are always output in a canonical (sorted)
>> form.)
> 
> Or use JSON, if your data fits within that structure. It's easy to
> read and write, it's human-readable, and it's safe (no chance of
> arbitrary code execution). Forcing yourself to use a format that can
> basically be processed by ast.literal_eval() is a good discipline -
> means you don't accidentally save/load too much.
> 
> ChrisA
> 

I made a specialized serializer for this, which is more expressive than JSON. It outputs
python literal expressions that can be directly parsed by ast.literal_eval(). You can
find it on pypi (https://pypi.python.org/pypi/serpent).  It's the default serializer of
Pyro, and it includes a Java and .NET version as well as an added bonus.


Irmen




From jeanpierreda at gmail.com  Tue Jun  9 20:47:53 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 9 Jun 2015 17:47:53 -0700
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
Message-ID: <CABicbJJ_ZxExukTcmFaE_ar1bYgW3xb5tOkG3BNdr5XcSKGQpg@mail.gmail.com>

Passing around data that can be put into ast.literal_eval is
synonymous with passing around data taht can be put into eval. It
sounds like a trap.

Other points against JSON / etc.: the lack of schema makes it easier
to stuff anything in there (not as easily as pickle, mind), and by
returning a plain dict, it becomes easier to require a field than to
allow a field to be missing, which is bad for robustness and bad for
data format migrations. (Protobuf (v3) has schemas and gives every
field a default value.)

For human readable serialized data, text format protocol buffers are
seriously underrated. (Relatedly: underdocumented, too.)

/me lifts head out of kool-aid and gasps for air

-- Devin

On Tue, Jun 9, 2015 at 5:17 PM, Irmen de Jong <irmen.NOSPAM at xs4all.nl> wrote:
> On 10-6-2015 1:06, Chris Angelico wrote:
>> On Wed, Jun 10, 2015 at 6:07 AM, Devin Jeanpierre
>> <jeanpierreda at gmail.com> wrote:
>>> There's a lot of subtle issues with pickle compatibility. e.g.
>>> old-style vs new-style classes. It's kinda hard and it's better to
>>> give up. I definitely agree it's better to use something else instead.
>>> For example, we switched to using protocol buffers, which have much
>>> better compatibility properties and are a bit more testable to boot
>>> (since text format protobufs are always output in a canonical (sorted)
>>> form.)
>>
>> Or use JSON, if your data fits within that structure. It's easy to
>> read and write, it's human-readable, and it's safe (no chance of
>> arbitrary code execution). Forcing yourself to use a format that can
>> basically be processed by ast.literal_eval() is a good discipline -
>> means you don't accidentally save/load too much.
>>
>> ChrisA
>>
>
> I made a specialized serializer for this, which is more expressive than JSON. It outputs
> python literal expressions that can be directly parsed by ast.literal_eval(). You can
> find it on pypi (https://pypi.python.org/pypi/serpent).  It's the default serializer of
> Pyro, and it includes a Java and .NET version as well as an added bonus.
>
>
> Irmen
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list


From rosuav at gmail.com  Tue Jun  9 21:03:49 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 10 Jun 2015 11:03:49 +1000
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <CABicbJJ_ZxExukTcmFaE_ar1bYgW3xb5tOkG3BNdr5XcSKGQpg@mail.gmail.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <CABicbJJ_ZxExukTcmFaE_ar1bYgW3xb5tOkG3BNdr5XcSKGQpg@mail.gmail.com>
Message-ID: <CAPTjJmrxxx3Gzjn_0YjKq_+YnotoKLorR=2XoXhQaJBB9+ekSA@mail.gmail.com>

On Wed, Jun 10, 2015 at 10:47 AM, Devin Jeanpierre
<jeanpierreda at gmail.com> wrote:
> Passing around data that can be put into ast.literal_eval is
> synonymous with passing around data taht can be put into eval. It
> sounds like a trap.

Except that it's hugely restricted. There are JSON parsing libraries
all over the place, and one of the points in favour of JSON is that it
can be dumped into JavaScript as-is and it evaluates correctly.
Doesn't mean you have to use eval() to parse it, and usually you
won't.

> Other points against JSON / etc.: the lack of schema makes it easier
> to stuff anything in there (not as easily as pickle, mind), and by
> returning a plain dict, it becomes easier to require a field than to
> allow a field to be missing, which is bad for robustness and bad for
> data format migrations. (Protobuf (v3) has schemas and gives every
> field a default value.)

This is true. But there are plenty of cases where you can manage on a
simple dictionary that maps keywords to values that are either
strings, numbers, or lists/dicts of same - and without any schema to
tell you what keywords are valid. It's simple, extensible, and scales
reasonably well to mid-sized use cases. Sure, the biggest cases (and
some of the smaller ones) benefit nicely from schema definitions, but
there's room to manage without.

ChrisA


From rustompmody at gmail.com  Tue Jun  9 22:08:32 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 9 Jun 2015 19:08:32 -0700 (PDT)
Subject: Parser needed.
In-Reply-To: <mailman.327.1433892811.13271.python-list@python.org>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
 <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
 <4252d$55709249$5419aafe$25285@news.ziggo.nl>
 <74396$55764fb5$5419aafe$6142@news.ziggo.nl>
 <mailman.310.1433842760.13271.python-list@python.org>
 <48a90$5576da1e$5419aafe$34050@news.ziggo.nl>
 <mailman.327.1433892811.13271.python-list@python.org>
Message-ID: <bf92c4ed-c616-458f-9d71-060aeb15cd46@googlegroups.com>

On Wednesday, June 10, 2015 at 5:04:17 AM UTC+5:30, Michael Torrie wrote:
> On 06/09/2015 06:20 AM, Skybuck Flying wrote:
> > Euhm...
> > 
> > My parser is already done... since today
> > 
> > Loving it too
> > 
> > Wrote it myself... based on the c# code technique explained somewhere in 
> > this thread too
> 
> I'm glad you're having fun, and making good progress.  And it's good to
> hear of success with Python.  However, I wouldn't really call this a
> "thread;" more a "monologue."  I'm not sure list members really
> appreciate the blow-by-blow description of a project, for the future.  I
> know I find it rather noisy.  I don't think anyone has actually read any
> of your messages (I'm pretty sure some have just blacklisted your
> messages), except to determine you're trying to parse something rather
> vague.  Might I suggest a blog is a more appropriate place to ruminate
> over a project (and posting code and examples) and use the list for
> questions and advice?

+1


From steve+comp.lang.python at pearwood.info  Tue Jun  9 23:52:11 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 10 Jun 2015 13:52:11 +1000
Subject: enhancement request: make py3 read/write py2 pickle format
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
Message-ID: <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 10 June 2015 10:47, Devin Jeanpierre wrote:

> Passing around data that can be put into ast.literal_eval is
> synonymous with passing around data taht can be put into eval. It
> sounds like a trap.

In what way?

literal_eval will cleanly and safely refuse to evaluate strings like:

    "len(None)"
    "100**100**100"
    "__import__('os').system('rm this')"


and so on, which makes it significantly safer when given untrusted data. I 
suppose that one might be able to perform a DOS attack by passing it:

    "1000 ... 0"

where the ... represents, say, a gigabyte of zeroes, but if an attacker has 
the ability to feed you gigabytes of data, they don't need literal_eval to 
DOS you.

If you can think of an actual attack against literal_eval, please tell us or 
report it, so it can be fixed.


> For human readable serialized data, text format protocol buffers are
> seriously underrated. (Relatedly: underdocumented, too.)

Ironically, literal_eval is designed to process text-format protocols using 
human-readable Python syntax for common data types like int, str, and dict.



-- 
Steve



From random832 at fastmail.us  Tue Jun  9 23:57:09 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 09 Jun 2015 23:57:09 -0400
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <1433908629.3534335.291461817.33072CF3@webmail.messagingengine.com>

On Tue, Jun 9, 2015, at 23:52, Steven D'Aprano wrote:
> > For human readable serialized data, text format protocol buffers are
> > seriously underrated. (Relatedly: underdocumented, too.)
> 
> Ironically, literal_eval is designed to process text-format protocols
> using 
> human-readable Python syntax for common data types like int, str, and
> dict.

"protocol buffers" is the name of a specific tool.


From rosuav at gmail.com  Wed Jun 10 00:00:55 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 10 Jun 2015 14:00:55 +1000
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <1433908629.3534335.291461817.33072CF3@webmail.messagingengine.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <1433908629.3534335.291461817.33072CF3@webmail.messagingengine.com>
Message-ID: <CAPTjJmqEOhMXmUUbj5CR9FX9+UqzOUY5YVo6wAxjGZ2pfnf3Lw@mail.gmail.com>

On Wed, Jun 10, 2015 at 1:57 PM,  <random832 at fastmail.us> wrote:
> On Tue, Jun 9, 2015, at 23:52, Steven D'Aprano wrote:
>> > For human readable serialized data, text format protocol buffers are
>> > seriously underrated. (Relatedly: underdocumented, too.)
>>
>> Ironically, literal_eval is designed to process text-format protocols
>> using
>> human-readable Python syntax for common data types like int, str, and
>> dict.
>
> "protocol buffers" is the name of a specific tool.

Yes, it is. But the point is that literal_eval, JSON, and other such
tools are _also_ text-format protocols that serialize to/from human
readable data. I'm not sure what the advantage of protocol buffers is,
but it's not like "human readable" is such a rarity. (It is still a
strike against pickle.)

ChrisA


From jeanpierreda at gmail.com  Wed Jun 10 00:48:40 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 9 Jun 2015 21:48:40 -0700
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CABicbJJChKiXGmUUL58z1JUHuJnV29h=bwmSmJe6pNE_rPDYUQ@mail.gmail.com>

On Tue, Jun 9, 2015 at 8:52 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Wednesday 10 June 2015 10:47, Devin Jeanpierre wrote:
>
>> Passing around data that can be put into ast.literal_eval is
>> synonymous with passing around data taht can be put into eval. It
>> sounds like a trap.
>
> In what way?

I misspoke, and instead of "synonymous", meant "also means".
(Implication, not equivalence.)

>> For human readable serialized data, text format protocol buffers are
>> seriously underrated. (Relatedly: underdocumented, too.)
>
> Ironically, literal_eval is designed to process text-format protocols using
> human-readable Python syntax for common data types like int, str, and dict.

"Protocol buffers" are a specific technology, not an abstract concept,
and literal_eval is not a great idea.

* the common serializer (repr) does not output a canonical form, and
  can serialize things in a way that they can't be deserialized
* there is no schema
* there is no well understood migration story for when the data you
  load and store changes
* it is not usable from other programming languages
* it encourages the use of eval when literal_eval becomes inconvenient
  or insufficient
* It is not particularly well specified or documented compared to the
  alternatives.
* The types you get back differ in python 2 vs 3

For most apps, the alternatives are better. Irmen's serpent library is
strictly better on every front, for example. (Except potentially
security, who knows.)

At least it's better than pickle, security wise. Reliability wise,
repr is a black hole, so no dice. :(

-- Devin


From steve+comp.lang.python at pearwood.info  Wed Jun 10 01:43:11 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 10 Jun 2015 15:43:11 +1000
Subject: enhancement request: make py3 read/write py2 pickle format
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.330.1433908633.13271.python-list@python.org>
Message-ID: <5577ce71$0$12998$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 10 June 2015 13:57, random832 at fastmail.us wrote:

> On Tue, Jun 9, 2015, at 23:52, Steven D'Aprano wrote:
>> > For human readable serialized data, text format protocol buffers are
>> > seriously underrated. (Relatedly: underdocumented, too.)
>> 
>> Ironically, literal_eval is designed to process text-format protocols
>> using
>> human-readable Python syntax for common data types like int, str, and
>> dict.
> 
> "protocol buffers" is the name of a specific tool.

It is? It sounds like a generic term for, you know, a buffer used by a 
protocol. I live and learn.

https://developers.google.com/protocol-buffers/docs/pythontutorial

You have to:

- write a data template, in a separate file; just don't call it a schema, 
because this isn't XML;

- don't forget the technically-optional-but-recommended (and required if you 
use other languages) "package" header, which is completely redundant in 
Python;

- run a separate compiler over that template, which will generate Python 
classes for you; just don't think that these classes are first class 
citizens that you can extend using inheritance, because they're not;

- import the generated module containing those classes;

- and now you have you're very own private pickle-like format, yay!


I'm sure that this has its uses for big, complex projects, but for 
lightweight needs, it seems over-engineered and unPythonic.

-- 
Steve



From info at egenix.com  Wed Jun 10 04:19:57 2015
From: info at egenix.com (eGenix Team: M.-A. Lemburg)
Date: Wed, 10 Jun 2015 10:19:57 +0200
Subject: ANN: eGenix mxODBC Plone/Zope Database Adapter 2.2.2
Message-ID: <5577F32D.3050607@egenix.com>

________________________________________________________________________
ANNOUNCING

                  mxODBC Plone/Zope Database Adapter

                            Version 2.2.2

              for the Plone CMS and Zope server platform

              Available for Plone 4.0-4.3 and Plone 5.0,
                        Zope 2.12 and 2.13, on
        Windows, Linux, Mac OS X, FreeBSD and other platforms

This announcement is also available on our web-site for online reading:
http://www.egenix.com/company/news/eGenix-mxODBC-Zope-DA-2.2.2-GA.html

________________________________________________________________________
INTRODUCTION

The eGenix mxODBC Zope DA allows you to easily connect your Zope or
Plone CMS installation to just about any database backend on the
market today, giving you the reliability of the commercially supported
eGenix product mxODBC and the flexibility of the ODBC standard as
middle-tier architecture.

The mxODBC Zope Database Adapter is highly portable, just like Zope
itself and provides a high performance interface to all your ODBC data
sources, using a single well-supported interface on Windows, Linux,
Mac OS X, FreeBSD and other platforms.

This makes it ideal for deployment in ZEO Clusters and Zope hosting
environments where stability and high performance are a top priority,
establishing an excellent basis and scalable solution for your Plone
CMS.

Product page:

    http://www.egenix.com/products/zope/mxODBCZopeDA/

________________________________________________________________________
NEWS

The 2.2.2 release of our mxODBC Zope/Plone Database Adapter product is
a patch level release of the popular ODBC database interface for Plone
and Zope. It includes these enhancements and fixes:

Driver Compatibility Enhancements
---------------------------------

 * Reenabled returning cursor.rowcount for FreeTDS >= 0.91. In
   previous versions, FreeTDS could return wrong data for .rowcount
   when using SELECTs.

Fixes
-----

 * Removed exists() built-in from mxODBC Zope DA's implicit addition
   of new built-ins via mxTools.

   This resolves a hard to track bug where the new built-in could
   potentially override the TAL python:exists function (in
   e.g. tal:condition="exists:something"). See this
   Products.CMFEditions fix for an example where the problem
   surfaced. This is a bug in TAL (it shouldn't give preference to
   built-ins over its own helpers), but we're providing the fix as
   easy work-around.

The complete list of changes is available on the mxODBC Zope DA
changelog page.

    http://www.egenix.com/products/zope/mxODBCZopeDA/changelog.html

mxODBC Zope DA 2.2.0 was released on 2014-12-11. Please see the mxODBC
Zope DA 2.2.0 release announcement for all the new features we have
added.

http://www.egenix.com/company/news/eGenix-mxODBC-Zope-DA-2.2.0-GA.html

For the full list of features, please see the mxODBC Zope DA feature
list:

    http://www.egenix.com/products/zope/mxODBCZopeDA/#Features

The complete list of changes is available on the mxODBC Zope DA
changelog page.


________________________________________________________________________
UPGRADING

Users are encouraged to upgrade to this latest mxODBC Plone/Zope
Database Adapter release to benefit from the new features and updated
ODBC driver support. We have taken special care not to introduce
backwards incompatible changes, making the upgrade experience as
smooth as possible.

For major and minor upgrade purchases, we will give out 20% discount
coupons going from mxODBC Zope DA 1.x to 2.2 and 50% coupons for
upgrades from mxODBC 2.x to 2.2. After upgrade, use of the original
license from which you upgraded is no longer permitted. Patch level
upgrades (e.g. 2.2.0 to 2.2.2) are always free of charge.

Please contact the eGenix.com Sales Team with your existing license
serials for details for an upgrade discount coupon.

If you want to try the new release before purchase, you can request
30-day evaluation licenses by visiting our web-site or writing to
sales at egenix.com, stating your name (or the name of the company) and
the number of eval licenses that you need.

    http://www.egenix.com/products/python/mxODBCZopeDA/#Evaluation

________________________________________________________________________
DOWNLOADS

Please visit the eGenix mxODBC Zope DA product page for downloads,
instructions on installation and documentation of the packages:

    http://www.egenix.com/company/products/zope/mxODBCZopeDA/

If you want to try the package, please jump straight to the download
instructions:

    http://www.egenix.com/products/zope/mxODBCZopeDA/#Download

Fully functional evaluation licenses for the mxODBC Zope DA are
available free of charge:

    http://www.egenix.com/products/zope/mxODBCZopeDA/#Evaluation

________________________________________________________________________
SUPPORT

Commercial support for this product is available directly from
eGenix.com.

Please see the support section of our website for details:

    http://www.egenix.com/services/support/

________________________________________________________________________
MORE INFORMATION

For more information on eGenix mxODBC Zope DA, licensing and download
instructions, please write to sales at egenix.com.

About eGenix (http://www.egenix.com/):

    eGenix is a software project, consulting and product company
    focusing on expert project services and professional quality
    products for companies, Python users and developers.

Enjoy,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 10 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From steve+comp.lang.python at pearwood.info  Wed Jun 10 05:36:56 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 10 Jun 2015 19:36:56 +1000
Subject: enhancement request: make py3 read/write py2 pickle format
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
Message-ID: <5578053a$0$11102$c3e8da3@news.astraweb.com>

On Wednesday 10 June 2015 14:48, Devin Jeanpierre wrote:

[...]
> and literal_eval is not a great idea.
> 
> * the common serializer (repr) does not output a canonical form, and
>   can serialize things in a way that they can't be deserialized

For literals, the canonical form is that understood by Python. I'm pretty 
sure that these have been stable since the days of Python 1.0, and will 
remain so pretty much forever:

ints: 12345
floats: 1.2345
strings: "spam"
None
True
False
lists, tuples, dicts and sets containing the above

There may be a few differences between Python 2 and 3, e.g. no set literal 
in Python 2, but in general the Python syntax is well-known and understood 
by anyone programming in Python.


> * there is no schema
> * there is no well understood migration story for when the data you
>   load and store changes

literal_eval is not a serialisation format itself. It is a primitive 
operation usable when serialising. E.g. you might write out a simple Unix-
style rc file of key:value pairs:


length=23.45
width=10.95
landscape=False

split on "=" and call literal_eval on the value.

This is a perfectly reasonable light-weight solution for simple 
serialisation needs.


> * it is not usable from other programming languages

That's okay, we're not writing in other programming languages :-)


> * it encourages the use of eval when literal_eval becomes inconvenient
>   or insufficient

I don't think so. I think that people who make the effort to import ast and 
call ast.literal_eval are fully aware of the dangers of eval and aren't 
silly enough to start using eval.


> * It is not particularly well specified or documented compared to the
>   alternatives.
> * The types you get back differ in python 2 vs 3

Doesn't matter. The type you *write* are different in Python 2 vs 3, so of 
course you do.


> For most apps, the alternatives are better. Irmen's serpent library is
> strictly better on every front, for example. (Except potentially
> security, who knows.)

Beyond simple needs, like rc files, literal_eval is not sufficient. You 
can't use it to deserialise arbitrary objects. That might be a feature, but 
if you need something more powerful than basic ints, floats, strings and a 
few others, literal_eval will not be powerful enough.

I think we are in violent agreement :-)

-- 
Steve



From ndbecker2 at gmail.com  Wed Jun 10 07:04:33 2015
From: ndbecker2 at gmail.com (Neal Becker)
Date: Wed, 10 Jun 2015 07:04:33 -0400
Subject: enhancement request: make py3 read/write py2 pickle format
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
Message-ID: <ml95k2$8ka$1@ger.gmane.org>

Chris Warrick wrote:

> On Tue, Jun 9, 2015 at 8:08 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
>> One of the most annoying problems with py2/3 interoperability is that the
>> pickle formats are not compatible.  There must be many who, like myself,
>> often use pickle format for data storage.
>>
>> It certainly would be a big help if py3 could read/write py2 pickle
>> format. You know, backward compatibility?
> 
> Don?t use pickle. It?s unsafe ? it executes arbitrary code, which
> means someone can give you a pickle file that will delete all your
> files or eat your cat.
> 
> Instead, use a safe format that has no ability to execute code, like
> JSON. It will also work with other programming languages and
> environments if you ever need to talk to anyone else.
> 
> But, FYI: there is backwards compatibility if you ask for it, in the
> form of protocol versions. That?s all you should know ? again, don?t
> use pickle.
> 

I believe a good native serialization system is essential for any modern 
programming language.  If pickle isn't it, we need something else that can 
serialize all language objects.  Or, are you saying, it's impossible to do 
this safely?



From robert.kern at gmail.com  Wed Jun 10 07:22:47 2015
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 10 Jun 2015 12:22:47 +0100
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <ml95k2$8ka$1@ger.gmane.org>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <ml95k2$8ka$1@ger.gmane.org>
Message-ID: <ml96m7$qf8$1@ger.gmane.org>

On 2015-06-10 12:04, Neal Becker wrote:
> Chris Warrick wrote:
>
>> On Tue, Jun 9, 2015 at 8:08 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
>>> One of the most annoying problems with py2/3 interoperability is that the
>>> pickle formats are not compatible.  There must be many who, like myself,
>>> often use pickle format for data storage.
>>>
>>> It certainly would be a big help if py3 could read/write py2 pickle
>>> format. You know, backward compatibility?
>>
>> Don?t use pickle. It?s unsafe ? it executes arbitrary code, which
>> means someone can give you a pickle file that will delete all your
>> files or eat your cat.
>>
>> Instead, use a safe format that has no ability to execute code, like
>> JSON. It will also work with other programming languages and
>> environments if you ever need to talk to anyone else.
>>
>> But, FYI: there is backwards compatibility if you ask for it, in the
>> form of protocol versions. That?s all you should know ? again, don?t
>> use pickle.
>
> I believe a good native serialization system is essential for any modern
> programming language.  If pickle isn't it, we need something else that can
> serialize all language objects.  Or, are you saying, it's impossible to do
> this safely?

By the very nature of the stated problem: serializing all language objects. 
Being able to construct any object, including instances of arbitrary classes, 
means that arbitrary code can be executed. All I have to do is make a pickle 
file for an object that claims that its constructor is shutil.rmtree().

This is fine in some use cases (e.g. wire format for otherwise-secured 
communication between two endpoints under your complete control), but it is 
worrying in others, like your use case of data storage (and presumably sharing).

Python 2/3 is also the least of your compatibility worries there. Refactor a 
class to a different module, or did one of your third-party dependencies do 
this? Poof! Your pickle files no longer work.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From rosuav at gmail.com  Wed Jun 10 07:23:48 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 10 Jun 2015 21:23:48 +1000
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <ml95k2$8ka$1@ger.gmane.org>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <ml95k2$8ka$1@ger.gmane.org>
Message-ID: <CAPTjJmoejU2Vi+eTUGrR_082TsGTr7HXf_sisLHo8URj+KX-Zw@mail.gmail.com>

On Wed, Jun 10, 2015 at 9:04 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
> I believe a good native serialization system is essential for any modern
> programming language.  If pickle isn't it, we need something else that can
> serialize all language objects.  Or, are you saying, it's impossible to do
> this safely?

It is indeed impossible to serialize _all_ objects safely. How do you,
for instance, serialize an open socket?

ChrisA


From marko at pacujo.net  Wed Jun 10 08:08:59 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 10 Jun 2015 15:08:59 +0300
Subject: enhancement request: make py3 read/write py2 pickle format
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <ml95k2$8ka$1@ger.gmane.org>
 <mailman.337.1433935377.13271.python-list@python.org>
Message-ID: <878ubr3gv8.fsf@elektro.pacujo.net>

Robert Kern <robert.kern at gmail.com>:

> By the very nature of the stated problem: serializing all language
> objects. Being able to construct any object, including instances of
> arbitrary classes, means that arbitrary code can be executed. All I
> have to do is make a pickle file for an object that claims that its
> constructor is shutil.rmtree().

You can't serialize/migrate arbitrary objects. Consider open TCP
connections, open files and other objects that extend outside the Python
VM. Also objects hold references to each other, leading to a huge
reference mesh.

For example:

   a.buddy = b
   b.buddy = a
   with open("a", "wb") as f: f.write(serialize(a))
   with open("b", "wb") as f: f.write(serialize(b))

   with open("a", "rb") as f: aa = deserialize(f.read())
   with open("b", "rb") as f: bb = deserialize(f.read())
   assert aa.buddy is bb


Marko


From python.list at tim.thechases.com  Wed Jun 10 09:28:12 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Wed, 10 Jun 2015 08:28:12 -0500
Subject: Python NBSP DWIM
Message-ID: <20150610082812.2ce887c3@bigbox.christie.dr>

str.split() doesn't seem to respect non-breaking space:

  Python 3.4.2 (default, Oct  8 2014, 10:45:20) 
  [GCC 4.9.1] on linux
  Type "help", "copyright", "credits" or "license" for more information.
  >>> print(repr("hello\N{NO-BREAK SPACE}world".split()))
  ['hello', 'world']

What's the purpose of a non-breaking space if it's treated like a
space for breaking/splitting purposes? :-)

Is this a bug?

-tkc






From random832 at fastmail.us  Wed Jun 10 09:38:39 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 10 Jun 2015 09:38:39 -0400
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <878ubr3gv8.fsf@elektro.pacujo.net>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <ml95k2$8ka$1@ger.gmane.org>
 <mailman.337.1433935377.13271.python-list@python.org>
 <878ubr3gv8.fsf@elektro.pacujo.net>
Message-ID: <1433943519.525527.291859417.54DFE170@webmail.messagingengine.com>

On Wed, Jun 10, 2015, at 08:08, Marko Rauhamaa wrote:
> You can't serialize/migrate arbitrary objects. Consider open TCP
> connections, open files and other objects that extend outside the Python
> VM. Also objects hold references to each other, leading to a huge
> reference mesh.
> 
> For example:
> 
>    a.buddy = b
>    b.buddy = a
>    with open("a", "wb") as f: f.write(serialize(a))
>    with open("b", "wb") as f: f.write(serialize(b))
> 
>    with open("a", "rb") as f: aa = deserialize(f.read())
>    with open("b", "rb") as f: bb = deserialize(f.read())
>    assert aa.buddy is bb

Of course, if you serialize a single dict with e.g. {'a': a, 'b': b},
you can expect (with advanced serialization tools, anyway  - I suspect
JSON will just make a mess or exceed maximum recursion depth)
result['a'].buddy is result['b']


From robert.kern at gmail.com  Wed Jun 10 09:52:29 2015
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 10 Jun 2015 14:52:29 +0100
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <878ubr3gv8.fsf@elektro.pacujo.net>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <ml95k2$8ka$1@ger.gmane.org>
 <mailman.337.1433935377.13271.python-list@python.org>
 <878ubr3gv8.fsf@elektro.pacujo.net>
Message-ID: <ml9fet$qkk$1@ger.gmane.org>

On 2015-06-10 13:08, Marko Rauhamaa wrote:
> Robert Kern <robert.kern at gmail.com>:
>
>> By the very nature of the stated problem: serializing all language
>> objects. Being able to construct any object, including instances of
>> arbitrary classes, means that arbitrary code can be executed. All I
>> have to do is make a pickle file for an object that claims that its
>> constructor is shutil.rmtree().
>
> You can't serialize/migrate arbitrary objects. Consider open TCP
> connections, open files and other objects that extend outside the Python
> VM.

Yes, yes, but that's really beside the point. Yes, there are some objects for 
which it doesn't even make sense to serialize. But my point is that even in this 
slightly smaller set of objects that *can* be serialized (and pickle currently 
does serialize), being able to serialize all of them entails arbitrary code 
execution to deserialize them. To allow people to write their own types that can 
be serialized, you have to let them specify arbitrary callables that will do the 
reconstruction. If you whitelist the possible reconstruction callables, you have 
greatly restricted the types that can participate in the serialization system.

> Also objects hold references to each other, leading to a huge
> reference mesh.
>
> For example:
>
>     a.buddy = b
>     b.buddy = a
>     with open("a", "wb") as f: f.write(serialize(a))
>     with open("b", "wb") as f: f.write(serialize(b))
>
>     with open("a", "rb") as f: aa = deserialize(f.read())
>     with open("b", "rb") as f: bb = deserialize(f.read())
>     assert aa.buddy is bb

Yeah, no one expects that to work. For example, if I deserialize the same string 
twice, you can't expect to get identical returned objects (as in, 
"deserialize(pickle) is deserialize(pickle)"). However, pickle does correctly 
handle fairly arbitrary reference graphs within the context of a single 
serialization, which is the most that can be asked of a serialization system. 
That isn't really a concern here.

 >>> class A(object):
...     pass
...
 >>> a = A()
 >>> b = A()
 >>> a.buddy = b
 >>> b.buddy = a
 >>> data = [a, b]
 >>> data[0].buddy is data[1]
True
 >>> data[1].buddy is data[0]
True
 >>> import cPickle
 >>> unpickled = cPickle.loads(cPickle.dumps(data))
 >>> unpickled[0].buddy is unpickled[1]
True
 >>> unpickled[1].buddy is unpickled[0]
True

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From nicholas.chammas at gmail.com  Wed Jun 10 10:11:53 2015
From: nicholas.chammas at gmail.com (Nicholas Chammas)
Date: Wed, 10 Jun 2015 10:11:53 -0400
Subject: Did the 3.4.4 docs get published early?
Message-ID: <CAOhmDzd-DTc2NP21LbyHZLrCL03RhwYo8jgZwfu4hULnbZ6-Mw@mail.gmail.com>

For example, here is a "New in version 3.4.4" method:

https://docs.python.org/3/library/asyncio-task.html#asyncio.ensure_future

However, the latest release appears to be 3.4.3:

https://www.python.org/downloads/

Is this normal, or did the 3.4.4 docs somehow get published early by
mistake?

Nick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150610/cd015ac7/attachment.html>

From breamoreboy at yahoo.co.uk  Wed Jun 10 10:17:21 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 10 Jun 2015 15:17:21 +0100
Subject: Python NBSP DWIM
In-Reply-To: <20150610082812.2ce887c3@bigbox.christie.dr>
References: <20150610082812.2ce887c3@bigbox.christie.dr>
Message-ID: <ml9gtr$ph2$1@ger.gmane.org>

On 10/06/2015 14:28, Tim Chase wrote:
> str.split() doesn't seem to respect non-breaking space:
>
>    Python 3.4.2 (default, Oct  8 2014, 10:45:20)
>    [GCC 4.9.1] on linux
>    Type "help", "copyright", "credits" or "license" for more information.
>    >>> print(repr("hello\N{NO-BREAK SPACE}world".split()))
>    ['hello', 'world']
>
> What's the purpose of a non-breaking space if it's treated like a
> space for breaking/splitting purposes? :-)
>
> Is this a bug?
>
> -tkc
>

IMNSHO yes.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From skip.montanaro at gmail.com  Wed Jun 10 10:28:24 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Wed, 10 Jun 2015 09:28:24 -0500
Subject: Python NBSP DWIM
In-Reply-To: <20150610082812.2ce887c3@bigbox.christie.dr>
References: <20150610082812.2ce887c3@bigbox.christie.dr>
Message-ID: <CANc-5UxaGMbY4BCDjuui4XXq=g2NKYXnYJAzgp9aNzDjOWZjyg@mail.gmail.com>

On Wed, Jun 10, 2015 at 8:28 AM, Tim Chase
<python.list at tim.thechases.com> wrote:
> Is this a bug?

Looks like it's been reported a few times with slightly different context:

https://bugs.python.org/issue6537
https://bugs.python.org/issue16623
https://bugs.python.org/issue20491
https://bugs.python.org/issue1390608

The couple times it's come up in the context of str.split, it's been
rejected, since the purpose of that method is to split words.

Skip


From breamoreboy at yahoo.co.uk  Wed Jun 10 10:40:27 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 10 Jun 2015 15:40:27 +0100
Subject: Did the 3.4.4 docs get published early?
In-Reply-To: <CAOhmDzd-DTc2NP21LbyHZLrCL03RhwYo8jgZwfu4hULnbZ6-Mw@mail.gmail.com>
References: <CAOhmDzd-DTc2NP21LbyHZLrCL03RhwYo8jgZwfu4hULnbZ6-Mw@mail.gmail.com>
Message-ID: <ml9i94$lbl$1@ger.gmane.org>

On 10/06/2015 15:11, Nicholas Chammas wrote:
> For example, here is a "New in version 3.4.4" method:
>
> https://docs.python.org/3/library/asyncio-task.html#asyncio.ensure_future
>
> However, the latest release appears to be 3.4.3:
>
> https://www.python.org/downloads/
>
> Is this normal, or did the 3.4.4 docs somehow get published early by
> mistake?
>
> Nick
>

I suspect that this is due to a trainee pilot being let loose too early 
with the time machine.  Failing that finger trouble when doing a commit. 
  Thinking about it more likely the former rather than the latter :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lac at openend.se  Wed Jun 10 11:03:58 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 10 Jun 2015 17:03:58 +0200
Subject: Python NBSP DWIM
In-Reply-To: Message from Skip Montanaro <skip.montanaro@gmail.com> of "Wed,
 10 Jun 2015 09:28:24 -0500."
 <CANc-5UxaGMbY4BCDjuui4XXq=g2NKYXnYJAzgp9aNzDjOWZjyg@mail.gmail.com>
References: <20150610082812.2ce887c3@bigbox.christie.dr><CANc-5UxaGMbY4BCDjuui4XXq=g2NKYXnYJAzgp9aNzDjOWZjyg@mail.gmail.com>
Message-ID: <201506101503.t5AF3w8J004902@fido.openend.se>

In a message of Wed, 10 Jun 2015 09:28:24 -0500, Skip Montanaro writes:
>On Wed, Jun 10, 2015 at 8:28 AM, Tim Chase
><python.list at tim.thechases.com> wrote:
>> Is this a bug?
>
>Looks like it's been reported a few times with slightly different context:
>
>https://bugs.python.org/issue6537
>https://bugs.python.org/issue16623
>https://bugs.python.org/issue20491
>https://bugs.python.org/issue1390608
>
>The couple times it's come up in the context of str.split, it's been
>rejected, since the purpose of that method is to split words.
>
>Skip

In these unicode days, this thinking may need to be revisited.  There
are many languages where whitespace does not separate words -- either
words aren't separated, or in Vietnamese, spaces separate syllables,
so entire words have spaces in them.

Laura



From zachary.ware+pylist at gmail.com  Wed Jun 10 11:05:24 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Wed, 10 Jun 2015 10:05:24 -0500
Subject: Did the 3.4.4 docs get published early?
In-Reply-To: <ml9i94$lbl$1@ger.gmane.org>
References: <CAOhmDzd-DTc2NP21LbyHZLrCL03RhwYo8jgZwfu4hULnbZ6-Mw@mail.gmail.com>
 <ml9i94$lbl$1@ger.gmane.org>
Message-ID: <CAKJDb-PXFzKPZobe_fTHnYYiWKsn2OJoY2Z7QwitRbeBKp8epQ@mail.gmail.com>

On Jun 10, 2015 9:41 AM, "Mark Lawrence" <breamoreboy at yahoo.co.uk> wrote:
>
> On 10/06/2015 15:11, Nicholas Chammas wrote:
>>
>> For example, here is a "New in version 3.4.4" method:
>>
>> https://docs.python.org/3/library/asyncio-task.html#asyncio.ensure_future
>>
>> However, the latest release appears to be 3.4.3:
>>
>> https://www.python.org/downloads/
>>
>> Is this normal, or did the 3.4.4 docs somehow get published early by
>> mistake?
>>
>> Nick
>>
>
> I suspect that this is due to a trainee pilot being let loose too early
with the time machine.  Failing that finger trouble when doing a commit.
Thinking about it more likely the former rather than the latter :)
>

Actually, it's just that the online docs reflect the latest documentation
from a particular branch of the source repository, since the docs are
continually improving and have no backwards compatibility constraints. This
does mean we sometimes have anomalies like this, though.

If you truly need the docs as they were at the time of release, they are
available, though I don't have a link handy on my phone.

--
Zach
(On a phone)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150610/95dbdb63/attachment.html>

From random832 at fastmail.us  Wed Jun 10 11:57:37 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 10 Jun 2015 11:57:37 -0400
Subject: Python NBSP DWIM
In-Reply-To: <201506101503.t5AF3w8J004902@fido.openend.se>
References: <20150610082812.2ce887c3@bigbox.christie.dr><CANc-5UxaGMbY4BCDjuui4XXq=g2NKYXnYJAzgp9aNzDjOWZjyg@mail.gmail.com>
 <201506101503.t5AF3w8J004902@fido.openend.se>
Message-ID: <1433951857.560592.291992697.365E8B44@webmail.messagingengine.com>

On Wed, Jun 10, 2015, at 11:03, Laura Creighton wrote:
> In these unicode days, this thinking may need to be revisited.  There
> are many languages where whitespace does not separate words -- either
> words aren't separated, or in Vietnamese, spaces separate syllables,
> so entire words have spaces in them.

Text wrapping for CJK scripts is another topic that might be worth
addressing in textwrap - words aren't space-separated, but there are
still rules about where you can place a line break. Generally these are
centered around preventing punctuation marks from being orphaned rather
than any attempt to algorithmically find word boundaries.

For the process called "Oikomi", while messing with kerning is not
strictly possible for monospaced text, it might be worthwhile in general
to have "preferred" and "maximum" line widths as parameters for
textwrap.

http://en.wikipedia.org/wiki/Line_breaking_rules_in_East_Asian_languages


From jason.swails at gmail.com  Wed Jun 10 12:54:50 2015
From: jason.swails at gmail.com (Jason Swails)
Date: Wed, 10 Jun 2015 12:54:50 -0400
Subject: Memory error while using pandas dataframe
In-Reply-To: <CAB1mvkwG6-ryzsqdiHus-McRWy2QCh1f6+Xm-Q0KNU=YeGOA6Q@mail.gmail.com>
References: <CAB1mvkwG6-ryzsqdiHus-McRWy2QCh1f6+Xm-Q0KNU=YeGOA6Q@mail.gmail.com>
Message-ID: <CAEk9e3oWa5zvAEnmX-WiXAY9O9mH9_0o+UKb+e7dRxDbqfEZ0w@mail.gmail.com>

On Mon, Jun 8, 2015 at 3:32 AM, naren <narencr7 at gmail.com> wrote:

> Memory Error while working with pandas dataframe.
>
> Description of Environment Windows 7 python 3.4.2 32-bit version pandas
> 0.16.0
>
> We are running into the error described below. Any help provided will be
> sincerely appreciated.
>
> We are able to read a 300MB Csv file into a dataframe using the read_csv
> function. While working with the dataframe we ran into memory error. We
> used the pd.Concat function to concatenate two dataframes. So we decided to
> use chunksize for lazy reading. Chunking returns an object of type
> TextFileReader.
>
>
> http://pandas.pydata.org/pandas-docs/stable/io.html#iterating-through-files-chunk-by-chunk
>
> We are able to iterate over this object once as a debugging measure. The
> iterator gets exhausted after iterating once. So we are not able to convert
> the TextFileReader object back into a dataframe, using the pd.concat
> function.
>
?It looks like you already figured out what your problem is.  The
TextFileReader is exhausted (i.e., at EOF), so you end up getting None from
it.?


?What is your question?  You want to be able to iterate through
TextFileReader again?

If so, try rewinding the file object that you passed to pd.concat.  If you
saved a reference to the file object, just call "seek(0)" on that object.
If you didn't, access it as the "f" attribute on the TextFileReader object
and call "seek(0)" on that instead.

That might work.  Otherwise, you should be more specific with your question
and provide a full segment of code that is as small as possible to
reproduce the error you're seeing.

HTH,
Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150610/f2374ac5/attachment.html>

From jonas at wielicki.name  Wed Jun 10 13:02:45 2015
From: jonas at wielicki.name (Jonas Wielicki)
Date: Wed, 10 Jun 2015 19:02:45 +0200
Subject: Did the 3.4.4 docs get published early?
In-Reply-To: <CAKJDb-PXFzKPZobe_fTHnYYiWKsn2OJoY2Z7QwitRbeBKp8epQ@mail.gmail.com>
References: <CAOhmDzd-DTc2NP21LbyHZLrCL03RhwYo8jgZwfu4hULnbZ6-Mw@mail.gmail.com>
 <ml9i94$lbl$1@ger.gmane.org>
 <CAKJDb-PXFzKPZobe_fTHnYYiWKsn2OJoY2Z7QwitRbeBKp8epQ@mail.gmail.com>
Message-ID: <55786DB5.1030106@wielicki.name>

On 10.06.2015 17:05, Zachary Ware wrote:
> On Jun 10, 2015 9:41 AM, "Mark Lawrence" <breamoreboy at yahoo.co.uk> wrote:
>>
>> On 10/06/2015 15:11, Nicholas Chammas wrote:
>>>
>>> For example, here is a "New in version 3.4.4" method:
>>>
>>> https://docs.python.org/3/library/asyncio-task.html#asyncio.ensure_future
>>>
>>> However, the latest release appears to be 3.4.3:
>>>
>>> https://www.python.org/downloads/
>>>
>>> Is this normal, or did the 3.4.4 docs somehow get published early by
>>> mistake?
>>>
>>> Nick
>>>
>>
>> I suspect that this is due to a trainee pilot being let loose too early
> with the time machine.  Failing that finger trouble when doing a commit.
> Thinking about it more likely the former rather than the latter :)
>>
> 
> Actually, it's just that the online docs reflect the latest documentation
> from a particular branch of the source repository, since the docs are
> continually improving and have no backwards compatibility constraints. This
> does mean we sometimes have anomalies like this, though.
> 
> If you truly need the docs as they were at the time of release, they are
> available, though I don't have a link handy on my phone.

You can (with javascript enabled) select the version for the docs at the
top right of the page. Also, just replacing the version number in the
URL works for the python 3 series (use 3.X even for python 3.0), even
farther back than the drop down menu allows.

regards,
jwi

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20150610/afc6e355/attachment.sig>

From PointedEars at web.de  Wed Jun 10 13:03:43 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Wed, 10 Jun 2015 19:03:43 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de> <lf5r3pmejgh.fsf@ling.helsinki.fi>
Message-ID: <16299015.icEhCTIHP8@PointedEars.de>

Jussi Piitulainen wrote:

> Thomas 'PointedEars' Lahn writes:
>> Jussi Piitulainen wrote:
>>> Thomas 'PointedEars' Lahn writes:
>>>>   8 3 6 3 1 2 6 8 2 1 6.
>>> 
>>> There are more than four hundred thousand ways to get those numbers
>>> in some order.
>>> 
>>> (11! / 2! / 2! / 2! / 3! / 2! = 415800)
>>
>> Fallacy.  Order is irrelevant here.
> 
> You need to consider every sequence that leads to the observed counts.

No, you need _not_, because ? I repeat ? the probability of getting a 
sequence of length n from a set of 9 numbers whereas the probability of 
picking a number is evenly distributed, is (1?9)? [(1/9)^n, or 1/9 to the 
nth, for those who do to see it because of lack of Unicode support at their 
system].  *Always.*  *No matter* which numbers are in it.  *No matter* in 
which order they are.  AISB, order is *irrelevant* here.  *Completely.*

This is _not_ a lottery box; you put the ball with the number on it *back 
into the box* after you have drawn it and before you draw a new one.

> One of those sequences occurred. You don't know which.

You do not have to.

> When tossing herrings [?]

Herrings are the key word here, indeed, and they are deep dark red.

> Code follows. Incidentally, I'm not feeling smart here. 

Good.  Because you should not feel smart in any way after ignoring all my 
explanations.

> [nonsense]

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From minscheung at googlemail.com  Wed Jun 10 13:05:57 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Wed, 10 Jun 2015 10:05:57 -0700 (PDT)
Subject: How to find number of whole weeks between dates?
Message-ID: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>

Say in 2014 April to May whole weeks would be 7th, 14th 28th April and  May would be 5th, 12th and 19th. So expecting 7 whole weeks in total


From steve at pearwood.info  Wed Jun 10 13:11:47 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 11 Jun 2015 03:11:47 +1000
Subject: Python NBSP DWIM
References: <20150610082812.2ce887c3@bigbox.christie.dr>
 <mailman.344.1433946513.13271.python-list@python.org>
Message-ID: <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com>

On Thu, 11 Jun 2015 12:28 am, Skip Montanaro wrote:

> On Wed, Jun 10, 2015 at 8:28 AM, Tim Chase
> <python.list at tim.thechases.com> wrote:
>> Is this a bug?
> 
> Looks like it's been reported a few times with slightly different context:
> 
> https://bugs.python.org/issue6537
> https://bugs.python.org/issue16623
> https://bugs.python.org/issue20491
> https://bugs.python.org/issue1390608
> 
> The couple times it's come up in the context of str.split, it's been
> rejected, since the purpose of that method is to split words.

That reasoning is ... strange. The whole point of the NBSP is specifically
*not* to split on it. If you wanted it to split, you would use a regular
space.

(Oh, and for the record, there are at least two non-breaking spaces in
Unicode, U+00A0 "NO-BREAK SPACE" and U+202F "NARROW NO-BREAK SPACE".)

http://www.unicode.org/charts/PDF/U0080.pdf
http://www.unicode.org/charts/PDF/U2000.pdf


Non-breaking spaces should be used for when you want to prevent
word-wrapping, and also for "open form" compound words:

http://grammar.ccc.commnet.edu/grammar/compounds.htm

textwrap should also treat NBSPs as non-spaces for the purposes of wrapping.

As a work-around, I think this should work:

- split the string on NBSPs;

- for substring returned, split normally;

- merge sub-substrings.


def split(s):
    """Split on whitespace, except NBSP.

    >>> split(u'hello world spam\\u00A0eggs cheese')
    [u'hello', u'world', u'spam\\xa0eggs', 'cheese']

    """
    words = []
    NBSP = u'\u00A0'
    substrings = s.split(NBSP)
    for i, sub in enumerate(substrings):
        parts = sub.split()
        if i == 0:
            words.extend(parts)
        else:
            words[-1] += NBSP + parts[0]
            words.extend(parts[1:])
    return words
        

-- 
Steven



From irmen.NOSPAM at xs4all.nl  Wed Jun 10 13:34:05 2015
From: irmen.NOSPAM at xs4all.nl (Irmen de Jong)
Date: Wed, 10 Jun 2015 19:34:05 +0200
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <5578053a$0$11102$c3e8da3@news.astraweb.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
Message-ID: <5578750e$0$2950$e4fe514c@news2.news.xs4all.nl>

On 10-6-2015 11:36, Steven D'Aprano wrote:
>> For most apps, the alternatives are better. Irmen's serpent library is
>> strictly better on every front, for example. (Except potentially
>> security, who knows.)
> 
> Beyond simple needs, like rc files, literal_eval is not sufficient. You 
> can't use it to deserialise arbitrary objects. That might be a feature, but 
> if you need something more powerful than basic ints, floats, strings and a 
> few others, literal_eval will not be powerful enough.

Just to have this off my chest:

I guess that "serialization format" is not the most correct term for what serpent does
(or in general, for the literal expressions that literal_eval accepts). Serpent doesn't
strive to (de)serialize everything perfectly. It is meant as a pythonic data transfer
format.

You can do this by explicitly mapping your application's object model to and from the
wire data format, or do it in a more pythonic way (IMO) and let python take care of most
of it automatically. Serpent is smart (I hope) about a number of non-primitive types. If
needed, use its hooks to teach it about types it doesn't readily recognize.
Yes, it does force you to reduce the arbitrary types you want to process to the set of
types that are accepted in a python literal expression. Thankfully lists, sets, tuples
and dicts are also among them.


Raison d'?tre for serpent is that I was looking for a safe pythonic alternative for
pickle, and with fewer limitations than Json.   I chose to use ast.literal_eval from the
standard library to do the "deserialization" for me, and so only had to build some code
to "serialize" object trees into python literal expressions :)

Regarding security: I simply trust the docstring of ast.literal_eval here;
"Safely evaluate an expression node or a string containing a Python expression. [...]"


Irmen




From marko at pacujo.net  Wed Jun 10 13:38:59 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 10 Jun 2015 20:38:59 +0300
Subject: How to find number of whole weeks between dates?
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
Message-ID: <877frbwjik.fsf@elektro.pacujo.net>

Sebastian M Cheung <minscheung at googlemail.com>:

> Say in 2014 April to May whole weeks would be 7th, 14th 28th April and
> May would be 5th, 12th and 19th. So expecting 7 whole weeks in total

This program gives you the number of days between two dates given in the
YYYY-MM-DD format:

========================================================================
#!/usr/bin/env python3

import sys

def gregorian_day_count(isodate):
    year, month, day = map(int, isodate.split('-'))
    a, b = divmod(12 * year + month - 3, 12)
    return (a * 365 + (a >> 2) - (a * 1311 >> 17) + (a * 1311 >> 19) +
            + (31306 * b + 722 >> 10))

def main():
    print(gregorian_day_count(sys.argv[2]) - gregorian_day_count(sys.argv[1]))

if __name__ == '__main__':
    main()
========================================================================

Divide the number by 7 and you have your answer.


Marko


From ian.g.kelly at gmail.com  Wed Jun 10 13:42:00 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 10 Jun 2015 11:42:00 -0600
Subject: How to find number of whole weeks between dates?
In-Reply-To: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
Message-ID: <CALwzidk=tXbTjE-LoKFmc=ObzcZJybK5c8zpCR7fg7jx3_hu7A@mail.gmail.com>

On Wed, Jun 10, 2015 at 11:05 AM, Sebastian M Cheung via Python-list
<python-list at python.org> wrote:
> Say in 2014 April to May whole weeks would be 7th, 14th 28th April and  May would be 5th, 12th and 19th. So expecting 7 whole weeks in total

>>> from datetime import date
>>> d1 = date(2014, 4, 7)
>>> d2 = date(2014, 5, 19)
>>> d2 - d1
datetime.timedelta(42)
>>> (d2 - d1).days
42
>>> (d2 - d1).days // 7
6


From marko at pacujo.net  Wed Jun 10 13:43:55 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 10 Jun 2015 20:43:55 +0300
Subject: How to find number of whole weeks between dates?
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
 <877frbwjik.fsf@elektro.pacujo.net>
Message-ID: <87381zwjac.fsf@elektro.pacujo.net>

Marko Rauhamaa <marko at pacujo.net>:

> This program gives you the number of days between two dates given in the
> YYYY-MM-DD format:

Sorry, couldn't resist.

It still does work, though.


Marko


From lac at openend.se  Wed Jun 10 13:50:28 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 10 Jun 2015 19:50:28 +0200
Subject: How to find number of whole weeks between dates?
In-Reply-To: Message from Marko Rauhamaa <marko@pacujo.net>
 of "Wed, 10 Jun 2015 20:38:59 +0300." <877frbwjik.fsf@elektro.pacujo.net>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com><877frbwjik.fsf@elektro.pacujo.net>
Message-ID: <201506101750.t5AHoSSu008696@fido.openend.se>

In a message of Wed, 10 Jun 2015 20:38:59 +0300, Marko Rauhamaa writes:
>Divide the number by 7 and you have your answer.
>

I am not sure that is what he wants -- If he gives us a start of Tuesday the
9th of June 2015 (yesterday) and an end of Thursday the 25th of June, that's
16 days.  But there is only one Monday-Friday week in there, the 14th-19th.

So if the OP wants an answer of 1 for such data, he may be interested in
the python calendar module https://docs.python.org/2/library/calendar.html

Laura




From sohcahtoa82 at gmail.com  Wed Jun 10 13:52:01 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Wed, 10 Jun 2015 10:52:01 -0700 (PDT)
Subject: Testing random
In-Reply-To: <16299015.icEhCTIHP8@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <lf5sia3bakh.fsf@ling.helsinki.fi> <1583276.2lplL8rY5W@PointedEars.de>
 <lf5r3pmejgh.fsf@ling.helsinki.fi> <16299015.icEhCTIHP8@PointedEars.de>
Message-ID: <e4a78e74-ca31-4000-9239-e14df9b47fd0@googlegroups.com>

On Wednesday, June 10, 2015 at 10:06:49 AM UTC-7, Thomas 'PointedEars' Lahn wrote:
> Jussi Piitulainen wrote:
> 
> > Thomas 'PointedEars' Lahn writes:
> >> Jussi Piitulainen wrote:
> >>> Thomas 'PointedEars' Lahn writes:
> >>>>   8 3 6 3 1 2 6 8 2 1 6.
> >>> 
> >>> There are more than four hundred thousand ways to get those numbers
> >>> in some order.
> >>> 
> >>> (11! / 2! / 2! / 2! / 3! / 2! = 415800)
> >>
> >> Fallacy.  Order is irrelevant here.
> > 
> > You need to consider every sequence that leads to the observed counts.
> 
> No, you need _not_, because ? I repeat ? the probability of getting a 
> sequence of length n from a set of 9 numbers whereas the probability of 
> picking a number is evenly distributed, is (1?9)? [(1/9)^n, or 1/9 to the 
> nth, for those who do to see it because of lack of Unicode support at their 
> system].  *Always.*  *No matter* which numbers are in it.  *No matter* in 
> which order they are.  AISB, order is *irrelevant* here.  *Completely.*
> 
> This is _not_ a lottery box; you put the ball with the number on it *back 
> into the box* after you have drawn it and before you draw a new one.
> 
> > One of those sequences occurred. You don't know which.
> 
> You do not have to.
> 
> > When tossing herrings [?]
> 
> Herrings are the key word here, indeed, and they are deep dark red.
> 
> > Code follows. Incidentally, I'm not feeling smart here. 
> 
> Good.  Because you should not feel smart in any way after ignoring all my 
> explanations.
> 
> > [nonsense]
> 
> -- 
> PointedEars
> 
> Twitter: @PointedEars2
> Please do not cc me. / Bitte keine Kopien per E-Mail.

To put it another way, let's simplify the problem.  You're rolling a pair of dice.  What are the chances that you'll see a pair of 3s?

Look at the list of possible roll combinations:

1 1     1 2     1 3     1 4     1 5     1 6
2 1     2 2     2 3     2 4     2 5     2 6
3 1     3 2     3 3     3 4     3 5     3 6
4 1     4 2     4 3     4 4     4 5     4 6
5 1     5 2     5 3     5 4     5 5     5 6
6 1     6 2     6 3     6 4     6 5     6 6

36 possible combinations.  Only one of them has a pair of 3s.  The answer is 1/36.

What about the chances of seeing 2 1?

Here's where I think you two are having such a huge disagreement.  Does order matter?  It depends what you're pulling random numbers out for.

The odds of seeing 2 1 are also only 1/36.  But if order doesn't matter in your application, then 1 2 is equivalent.  The odds of getting 2 1 OR 1 2 is 2/36, or 1/18.

But whether order matters or not, the chances of getting a pair of threes in two rolls is ALWAYS 1/36.

If this gets expanded to grabbing 10 random numbers between 1 and 9, then the chances of getting a sequence of 10 ones is still only (1/9)^10, *regardless of whether or not order matters*.  There are 9^10 possible sequences, but only *one* of these is all ones.

If order matters, then 7385941745 also has a (1/9)^10 chance of occurring.  Just because it isn't a memorable sequence doesn't give it a higher chance of happening.

If order DOESN'T matter, then 1344557789 would be equivalent, and the odds are higher.


From ian.g.kelly at gmail.com  Wed Jun 10 14:02:48 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 10 Jun 2015 12:02:48 -0600
Subject: Testing random
In-Reply-To: <16299015.icEhCTIHP8@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <lf5sia3bakh.fsf@ling.helsinki.fi> <1583276.2lplL8rY5W@PointedEars.de>
 <lf5r3pmejgh.fsf@ling.helsinki.fi> <16299015.icEhCTIHP8@PointedEars.de>
Message-ID: <CALwzidknZVYWNTK8KMbkZjiKQfmr7Q6XvHzL6DKoqN6y-AdpZw@mail.gmail.com>

On Wed, Jun 10, 2015 at 11:03 AM, Thomas 'PointedEars' Lahn
<PointedEars at web.de> wrote:
> Jussi Piitulainen wrote:
>
>> Thomas 'PointedEars' Lahn writes:
>>> Jussi Piitulainen wrote:
>>>> Thomas 'PointedEars' Lahn writes:
>>>>>   8 3 6 3 1 2 6 8 2 1 6.
>>>>
>>>> There are more than four hundred thousand ways to get those numbers
>>>> in some order.
>>>>
>>>> (11! / 2! / 2! / 2! / 3! / 2! = 415800)
>>>
>>> Fallacy.  Order is irrelevant here.
>>
>> You need to consider every sequence that leads to the observed counts.
>
> No, you need _not_, because ? I repeat ? the probability of getting a
> sequence of length n from a set of 9 numbers whereas the probability of
> picking a number is evenly distributed, is (1?9)? [(1/9)^n, or 1/9 to the
> nth, for those who do to see it because of lack of Unicode support at their
> system].  *Always.*  *No matter* which numbers are in it.  *No matter* in
> which order they are.  AISB, order is *irrelevant* here.  *Completely.*

Order is relevant because, for instance, there are n differently
ordered sequences that contain n-1 1s and one 2, while there is only
one sequence that contains n 1s. While each of those individual
sequences are indeed equiprobable, the overall probability of getting
a sequence that contains n-1 1s and one 2 is n times the probability
of getting a sequence that contains n 1s.

The context of this whole thread is about the probability of getting a
sequence where every number occurs at least once. The order that they
occur in doesn't matter, but the number of possible permutations does,
because every one of those permutations is a distinct sequence
contributing an equal amount to the total overall probability.

The probability of 123456789 and 111111111 are equal. The probability
of a sequence containing all nine numbers and a sequence containing
only 1s are *not* equal.


From joel.goldstick at gmail.com  Wed Jun 10 14:07:46 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Wed, 10 Jun 2015 14:07:46 -0400
Subject: How to find number of whole weeks between dates?
In-Reply-To: <201506101750.t5AHoSSu008696@fido.openend.se>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
 <marko@pacujo.net> <877frbwjik.fsf@elektro.pacujo.net>
 <201506101750.t5AHoSSu008696@fido.openend.se>
Message-ID: <CAPM-O+z92q-8hXtUU+Qgorw4tjci63ZknNL7H9yFrw9bDKmS7g@mail.gmail.com>

On Wed, Jun 10, 2015 at 1:50 PM, Laura Creighton <lac at openend.se> wrote:
> In a message of Wed, 10 Jun 2015 20:38:59 +0300, Marko Rauhamaa writes:
>>Divide the number by 7 and you have your answer.
>>
>
> I am not sure that is what he wants -- If he gives us a start of Tuesday the
> 9th of June 2015 (yesterday) and an end of Thursday the 25th of June, that's
> 16 days.  But there is only one Monday-Friday week in there, the 14th-19th.
>
> So if the OP wants an answer of 1 for such data, he may be interested in
> the python calendar module https://docs.python.org/2/library/calendar.html
>
> Laura
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

Find the number of weeks with the above method, then

>>> import datetime
end_date = datetime.datetime(2012, 3, 23)  // whatever your end date is
if end_date.weekday() != 5:
    number_of_complete _weeks -= 1

weekday returns 0 for monday, so 5 for Saturday


-- 
Joel Goldstick
http://joelgoldstick.com


From nicholas.chammas at gmail.com  Wed Jun 10 14:25:47 2015
From: nicholas.chammas at gmail.com (Nicholas Chammas)
Date: Wed, 10 Jun 2015 18:25:47 +0000
Subject: Did the 3.4.4 docs get published early?
In-Reply-To: <CAOhmDzd-DTc2NP21LbyHZLrCL03RhwYo8jgZwfu4hULnbZ6-Mw@mail.gmail.com>
References: <CAOhmDzd-DTc2NP21LbyHZLrCL03RhwYo8jgZwfu4hULnbZ6-Mw@mail.gmail.com>
Message-ID: <CAOhmDzd-Rz1sAZkvxu6548PSsYEgP5UFcCpZZG=908Led2iT-Q@mail.gmail.com>

Also, just replacing the version number in the URL works for the python 3
series (use 3.X even for python 3.0), even farther back than the drop down
menu allows.

This does not help in this case:

https://docs.python.org/3.4/library/asyncio-task.html#asyncio.ensure_future

Also, you cannot select the docs for a maintenance release, like 3.4.3.

Anyway, it?s not a big deal as long as significant changes are tagged
appropriately with notes like ?New in version NNN?, which they are.

Ideally, the docs would only show the latest changes for released versions
of Python, but since some changes (like the one I linked to) are introduced
in maintenance versions, it?s probably hard to separate them out into
separate branches.

Nick
?

On Wed, Jun 10, 2015 at 10:11 AM Nicholas Chammas <
nicholas.chammas at gmail.com> wrote:

> For example, here is a "New in version 3.4.4" method:
>
> https://docs.python.org/3/library/asyncio-task.html#asyncio.ensure_future
>
> However, the latest release appears to be 3.4.3:
>
> https://www.python.org/downloads/
>
> Is this normal, or did the 3.4.4 docs somehow get published early by
> mistake?
>
> Nick
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150610/97abf3da/attachment.html>

From random832 at fastmail.us  Wed Jun 10 14:26:00 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 10 Jun 2015 14:26:00 -0400
Subject: Testing random
In-Reply-To: <16299015.icEhCTIHP8@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de> <lf5r3pmejgh.fsf@ling.helsinki.fi>
 <16299015.icEhCTIHP8@PointedEars.de>
Message-ID: <1433960760.593495.292131873.0829E675@webmail.messagingengine.com>

On Wed, Jun 10, 2015, at 13:03, Thomas 'PointedEars' Lahn wrote:
> This is _not_ a lottery box; you put the ball with the number on it *back 
> into the box* after you have drawn it and before you draw a new one.

Yes, but getting a 2, putting it back, and getting a 1 is just as good
as getting a 1, putting it back, and getting a 2, so you have to add the
probability of those cases together to get the probability of getting at
least one 1 and at least one 2.


From breamoreboy at yahoo.co.uk  Wed Jun 10 15:48:19 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 10 Jun 2015 20:48:19 +0100
Subject: How to find number of whole weeks between dates?
In-Reply-To: <201506101750.t5AHoSSu008696@fido.openend.se>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com><877frbwjik.fsf@elektro.pacujo.net>
 <201506101750.t5AHoSSu008696@fido.openend.se>
Message-ID: <mla4ac$k4b$1@ger.gmane.org>

On 10/06/2015 18:50, Laura Creighton wrote:
> In a message of Wed, 10 Jun 2015 20:38:59 +0300, Marko Rauhamaa writes:
>> Divide the number by 7 and you have your answer.
>>
>
> I am not sure that is what he wants -- If he gives us a start of Tuesday the
> 9th of June 2015 (yesterday) and an end of Thursday the 25th of June, that's
> 16 days.  But there is only one Monday-Friday week in there, the 14th-19th.
>
> So if the OP wants an answer of 1 for such data, he may be interested in
> the python calendar module https://docs.python.org/2/library/calendar.html
>
> Laura
>
>

For those who wish to move into the 21st century the link is 
https://docs.python.org/3/library/calendar.html

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From jpiitula at ling.helsinki.fi  Wed Jun 10 16:00:52 2015
From: jpiitula at ling.helsinki.fi (Jussi Piitulainen)
Date: Wed, 10 Jun 2015 23:00:52 +0300
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de> <lf5r3pmejgh.fsf@ling.helsinki.fi>
 <16299015.icEhCTIHP8@PointedEars.de>
 <e4a78e74-ca31-4000-9239-e14df9b47fd0@googlegroups.com>
Message-ID: <lf5wpzb9vuz.fsf@ling.helsinki.fi>

sohcahtoa82 at gmail.com writes:

[...]

> Here's where I think you two are having such a huge disagreement.
> Does order matter?  It depends what you're pulling random numbers out
> for.
>
> The odds of seeing 2 1 are also only 1/36.  But if order doesn't
> matter in your application, then 1 2 is equivalent.  The odds of
> getting 2 1 OR 1 2 is 2/36, or 1/18.

[...]

I'm not sure what Thomas 'PointedEars' Lahn is talking about. It seems
to be something else than what others have been discussing.

Others have been discussing a record of the number of times that each
possible outcome came up in a sequence of random numbers. There is no
other record of the sequence. The number of drawings is much larger than
the number of possible outcomes. The subject line refers to testing
whether the record of counts is compatible with the drawings being
random in the usual sense: independent, with uniform distribution.

Someone pointed out that some numbers may not have occurred at all - I
think a piece of code needed modification - and so people have commented
on the probability of this happening ... and whether it depends on the
number of drawings.


From minscheung at googlemail.com  Wed Jun 10 16:11:04 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Wed, 10 Jun 2015 13:11:04 -0700 (PDT)
Subject: How to find number of whole weeks between dates?
In-Reply-To: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
Message-ID: <a650c18e-fa33-4139-bc68-e14b08273a8d@googlegroups.com>

On Wednesday, June 10, 2015 at 6:06:09 PM UTC+1, Sebastian M Cheung wrote:
> Say in 2014 April to May whole weeks would be 7th, 14th 28th April and  May would be 5th, 12th and 19th. So expecting 7 whole weeks in total

What I mean is given two dates I want to find WHOLE weeks, so if given the 2014 calendar and function has two inputs (4th and 5th month) then 7th, 14th, 21st and 28th from April with 28th April week carrying into May, and then 5th, 12th and 19th May to give total of 7 whole weeks, because 26th May is not a whole week and will not be counted.

Hope thats clear.


From breamoreboy at yahoo.co.uk  Wed Jun 10 16:26:01 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 10 Jun 2015 21:26:01 +0100
Subject: How to find number of whole weeks between dates?
In-Reply-To: <a650c18e-fa33-4139-bc68-e14b08273a8d@googlegroups.com>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
 <a650c18e-fa33-4139-bc68-e14b08273a8d@googlegroups.com>
Message-ID: <mla6h2$rd0$1@ger.gmane.org>

On 10/06/2015 21:11, Sebastian M Cheung via Python-list wrote:
> On Wednesday, June 10, 2015 at 6:06:09 PM UTC+1, Sebastian M Cheung wrote:
>> Say in 2014 April to May whole weeks would be 7th, 14th 28th April and  May would be 5th, 12th and 19th. So expecting 7 whole weeks in total
>
> What I mean is given two dates I want to find WHOLE weeks, so if given the 2014 calendar and function has two inputs (4th and 5th month) then 7th, 14th, 21st and 28th from April with 28th April week carrying into May, and then 5th, 12th and 19th May to give total of 7 whole weeks, because 26th May is not a whole week and will not be counted.
>
> Hope thats clear.
>

If you'd be kind enough to show the code that you've written and the 
precise reasons(s) that it doesn't work then we'll be delighted to point 
you in the right direction.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From nicholas.chammas at gmail.com  Wed Jun 10 16:28:12 2015
From: nicholas.chammas at gmail.com (Nicholas Chammas)
Date: Wed, 10 Jun 2015 20:28:12 +0000
Subject: Did the 3.4.4 docs get published early?
In-Reply-To: <CAOhmDzd-Rz1sAZkvxu6548PSsYEgP5UFcCpZZG=908Led2iT-Q@mail.gmail.com>
References: <CAOhmDzd-DTc2NP21LbyHZLrCL03RhwYo8jgZwfu4hULnbZ6-Mw@mail.gmail.com>
 <CAOhmDzd-Rz1sAZkvxu6548PSsYEgP5UFcCpZZG=908Led2iT-Q@mail.gmail.com>
Message-ID: <CAOhmDzc6d9WU1VAOVvFBg1QvWMmjwjswSVVo3=j-1bakkQ42tQ@mail.gmail.com>

Sorry, somehow the formatting in my previous email didn't come through
correctly.

This part was supposed to be in a quote block:

> Also, just replacing the version number in the URL works for the python 3
series
> (use 3.X even for python 3.0), even farther back than the drop down menu
allows.

Nick

On Wed, Jun 10, 2015 at 2:25 PM Nicholas Chammas <nicholas.chammas at gmail.com>
wrote:

> Also, just replacing the version number in the URL works for the python 3
> series (use 3.X even for python 3.0), even farther back than the drop down
> menu allows.
>
> This does not help in this case:
>
> https://docs.python.org/3.4/library/asyncio-task.html#asyncio.ensure_future
>
> Also, you cannot select the docs for a maintenance release, like 3.4.3.
>
> Anyway, it?s not a big deal as long as significant changes are tagged
> appropriately with notes like ?New in version NNN?, which they are.
>
> Ideally, the docs would only show the latest changes for released versions
> of Python, but since some changes (like the one I linked to) are introduced
> in maintenance versions, it?s probably hard to separate them out into
> separate branches.
>
> Nick
> ?
>
> On Wed, Jun 10, 2015 at 10:11 AM Nicholas Chammas <
> nicholas.chammas at gmail.com> wrote:
>
>> For example, here is a "New in version 3.4.4" method:
>>
>> https://docs.python.org/3/library/asyncio-task.html#asyncio.ensure_future
>>
>> However, the latest release appears to be 3.4.3:
>>
>> https://www.python.org/downloads/
>>
>> Is this normal, or did the 3.4.4 docs somehow get published early by
>> mistake?
>>
>> Nick
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150610/a4bcbfe2/attachment.html>

From ian.g.kelly at gmail.com  Wed Jun 10 16:33:22 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 10 Jun 2015 14:33:22 -0600
Subject: How to find number of whole weeks between dates?
In-Reply-To: <a650c18e-fa33-4139-bc68-e14b08273a8d@googlegroups.com>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
 <a650c18e-fa33-4139-bc68-e14b08273a8d@googlegroups.com>
Message-ID: <CALwzid=G3+cEC6X9GxQv6huvFr30oqHmjYNg4Hqy9PhmpxaW0g@mail.gmail.com>

On Wed, Jun 10, 2015 at 2:11 PM, Sebastian M Cheung via Python-list
<python-list at python.org> wrote:
> On Wednesday, June 10, 2015 at 6:06:09 PM UTC+1, Sebastian M Cheung wrote:
>> Say in 2014 April to May whole weeks would be 7th, 14th 28th April and  May would be 5th, 12th and 19th. So expecting 7 whole weeks in total
>
> What I mean is given two dates I want to find WHOLE weeks, so if given the 2014 calendar and function has two inputs (4th and 5th month) then 7th, 14th, 21st and 28th from April with 28th April week carrying into May, and then 5th, 12th and 19th May to give total of 7 whole weeks, because 26th May is not a whole week and will not be counted.

So the two "dates" being passed are actually months? The calendar
module already suggested should be useful for this.


From jeanpierreda at gmail.com  Wed Jun 10 18:10:25 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Wed, 10 Jun 2015 15:10:25 -0700
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <5578053a$0$11102$c3e8da3@news.astraweb.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
Message-ID: <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>

FWIW most of the objections below also apply to JSON, so this doesn't
just have to be about repr/literal_eval. I'm definitely a huge
proponent of widespread use of something like protocol buffers, both
for production code and personal hacky projects.

On Wed, Jun 10, 2015 at 2:36 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Wednesday 10 June 2015 14:48, Devin Jeanpierre wrote:
>
> [...]
>> and literal_eval is not a great idea.
>>
>> * the common serializer (repr) does not output a canonical form, and
>>   can serialize things in a way that they can't be deserialized
>
> For literals, the canonical form is that understood by Python. I'm pretty
> sure that these have been stable since the days of Python 1.0, and will
> remain so pretty much forever:

The problem is that there are two different ways repr might write out
a dict equal to {'a': 1, 'b': 2}. This can make tests brittle -- e.g.
it's why doctest fails badly at examples involving dictionaries. Text
format protocol buffers output everything sorted, so that you can do
textual diffs for compatibility tests and such.

At work, one thing we do in places is mock out services using "golden"
expected protobuf responses, so that you can test that the server
returns exactly that, and test what the client does with that,
separately. These are checked into perforce in text format.

>> * there is no schema
>> * there is no well understood migration story for when the data you
>>   load and store changes
>
> literal_eval is not a serialisation format itself. It is a primitive
> operation usable when serialising. E.g. you might write out a simple Unix-
> style rc file of key:value pairs:
>
-snip-
>
> split on "=" and call literal_eval on the value.
>
> This is a perfectly reasonable light-weight solution for simple
> serialisation needs.

I could spend a bunch of time writing yet another config file format,
or I could use text format protocol buffers, YAML, or TOML and call it
a day.

>> * it encourages the use of eval when literal_eval becomes inconvenient
>>   or insufficient
>
> I don't think so. I think that people who make the effort to import ast and
> call ast.literal_eval are fully aware of the dangers of eval and aren't
> silly enough to start using eval.

The problem is when you have your config file format using python
literals, and another programmer wants to deal with it and doesn't
look at your codebase, and things like that. When transferring data,
this can happen a lot, since you are often not the user of the data
you wrote, and you can't control how others consume it. They might use
eval even if you didn't mean for them to. For example, in JavaScript,
this was once a common problem for services exposing JSON, and it
still happens even now.

>> * It is not particularly well specified or documented compared to the
>>   alternatives.
>> * The types you get back differ in python 2 vs 3
>
> Doesn't matter. The type you *write* are different in Python 2 vs 3, so of
> course you do.

In a shared 2/3 codebase, if I write bytes I expect to get bytes, and
if I write unicode I expect to get unicode. (There is a third category
of thing, which should be bytes on 2.x and string on 3.x, but it's
probably best to handle that outside of the deserializer). If you
thread it through repr and literal_eval using different versions for
each, unicode in python 3 becomes bytes in python 2, and vice versa.
So it makes migrating to Python 3 even harder.

>> For most apps, the alternatives are better. Irmen's serpent library is
>> strictly better on every front, for example. (Except potentially
>> security, who knows.)
>
> Beyond simple needs, like rc files, literal_eval is not sufficient. You
> can't use it to deserialise arbitrary objects. That might be a feature, but
> if you need something more powerful than basic ints, floats, strings and a
> few others, literal_eval will not be powerful enough.

No, it is powerful enough. After all, JSON has the same limitations.
Protobuf only adds enums and structs to JSON's types, and it's
potentially the most-used serialization format in the world by
operations per second.

Serialization libraries/formats usually need handholding to serialize
complex Python objects into simple serializable types. [Except pickle,
and that's the very reason it's insecure (per previous discussion in
thread.)]

-- Devin


From tjreedy at udel.edu  Wed Jun 10 18:46:22 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 10 Jun 2015 18:46:22 -0400
Subject: Permission to Showcase the Python Program
In-Reply-To: <D19C6E14.35A2%bushl@teachco.com>
References: <D19C6E14.35A2%bushl@teachco.com>
Message-ID: <5578BE3E.30000@udel.edu>

On 6/9/2015 10:19 AM, Leslie Bush wrote:
> I?m having trouble reaching an actual human at your organization
 > as my emails get bounced back.

If you sent email to
psf at python.org <psf at python.org>
it should not have bounced, as according to
https://mail.python.org/mailman/listinfo/python-legal-sig
that is the 'legal email address'.

But I do not think it really matters for your purpose.

> My name is Leslie Bush and I am the Intellectual Property Coordinator
 > for The Great Courses. We produce non-credit, college-level educational
 > programs on DVD and electronic formats in a lecture series.
 > The lectures are recorded and then sold to the general public for-profit.
 > As a didactic tool for enhancing the programs, we include in the lectures
 > visual elements illustrating works of art, people, events, locations, 
etc....

I am a python core developer, a member of PSF, but otherwise have no 
official position.  As a parent, I am familiar with The Great Courses,
having used 2 for home schooling.  They were very helpful.

> We are currently producing a course with Dr. John Keyser, Professor of
 > Computer Science at the University of North Carolina Chapel Hill, 
entitled
 > ?Computer Science for Everyone: Programming Concepts and Exercises?
 > The professor would like to use your Python program to showcase in 
the course.

Great.  Most of us would consider it silly for him not to.  Most of us 
would also urge that he use Python 3 rather than Python 2, if he is not 
already.

 > I am writing to see if we may have permission to do so. If permission
 > is granted, we will have a copy of our license agreement sent over to
 > your company?s authorizer for review and a signature.

I am not a lawyer, but am 99.990% (and I am not exaggerating) sure that 
the Python license already gives you the permission you need. Our 
documentation page is at
https://docs.python.org/3/
Clicking the History and License of Python link takes you to
https://docs.python.org/3/license.html
Skip to Terms and conditions for accessing or otherwise using Python
and read PSF LICENSE AGREEMENT FOR PYTHON 3.4.3.  It is about as liberal 
as can be.  It was written by the PSF lawyer and is intended to be a 
blanket grant of permissiond, so that the PSF will not need to employ an 
'authorizer' to review and sign permissions. This is standard for 
open-source software.

Companies routinely use Python and write their own public and 
proprietary code.  People write and sell books about Python or that 
reference Python.  Universities teach courses that include Python. 
People post videos about Python.  We WANT people to do all of these 
things.  They all do it without further agreements and signatures beyond 
what is openly published.

I suspect that some companies paranoid about the possibility of lawsuit 
may have their own lawyer review the license agreement to make sure it 
means what it seems to say and that their use falls within the license. 
  You are free to do the same if worried.

-- 
Terry Jan Reedy





From tjreedy at udel.edu  Wed Jun 10 19:16:28 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 10 Jun 2015 19:16:28 -0400
Subject: Did the 3.4.4 docs get published early?
In-Reply-To: <CAOhmDzd-DTc2NP21LbyHZLrCL03RhwYo8jgZwfu4hULnbZ6-Mw@mail.gmail.com>
References: <CAOhmDzd-DTc2NP21LbyHZLrCL03RhwYo8jgZwfu4hULnbZ6-Mw@mail.gmail.com>
Message-ID: <mlaghi$oba$1@ger.gmane.org>

On 6/10/2015 10:11 AM, Nicholas Chammas wrote:
> For example, here is a "New in version 3.4.4" method:
>
> https://docs.python.org/3/library/asyncio-task.html#asyncio.ensure_future
>
> However, the latest release appears to be 3.4.3:
>
> https://www.python.org/downloads/
>
> Is this normal, or did the 3.4.4 docs somehow get published early by
> mistake?

The online x.y docs reflect the x.y branch in the repository.  New 
features are not normally added in an x.y.z maintenance release, which 
is normally bugfixes only.  However, asyncio is a new module in 3.4 and 
marked as 'provisional', which means subject to change during the 3.4 
series of releases. Idle is also exceptional in getting uncategorized 
changes in maintenance releases, so each x.y.z release needs a copy of 
the Idle doc chapter that is both up-to-date and frozen as of x.y.z. 
Making this happen is still a work-in-progress.

-- 
Terry Jan Reedy




From tjreedy at udel.edu  Wed Jun 10 19:25:33 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 10 Jun 2015 19:25:33 -0400
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
 <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>
Message-ID: <mlah2i$f7$1@ger.gmane.org>

On 6/10/2015 6:10 PM, Devin Jeanpierre wrote:

> The problem is that there are two different ways repr might write out
> a dict equal to {'a': 1, 'b': 2}. This can make tests brittle

Not if one compares objects rather than string representations of 
objects.  I am strongly of the view that code and tests should be 
written to directly compare objects as much as possible.

> it's why doctest fails badly at examples involving dictionaries.

or sets or addresses or object ids or locale-dependent strings or random 
numbers or values dependent on random numbers.

-- 
Terry Jan Reedy



From greg.ewing at canterbury.ac.nz  Wed Jun 10 19:30:23 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Thu, 11 Jun 2015 11:30:23 +1200
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <mailman.341.1433944371.13271.python-list@python.org>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <ml95k2$8ka$1@ger.gmane.org>
 <mailman.337.1433935377.13271.python-list@python.org>
 <878ubr3gv8.fsf@elektro.pacujo.net>
 <mailman.341.1433944371.13271.python-list@python.org>
Message-ID: <ctrvkhFfutdU1@mid.individual.net>

Robert Kern wrote:
> To allow people to write their own types that can be serialized, 
> you have to let them specify arbitrary callables that will do the 
> reconstruction. If you whitelist the possible reconstruction callables, 
> you have greatly restricted the types that can participate in the 
> serialization system.

If whitelisting a type is the *only* thing you need to
do to make it serialisable, I think that comes close
enough to the stated goal of being able to "serialise
all [potentially serialisable] language objects".

Having to be explicit about which types are deserialisable
is probably a good thing anyway. It gives you an opportunity
to specify the mapping between the external format and
class names, so that your serialised data doesn't contain
assumptions about implementation details of your program.

-- 
Greg


From jeanpierreda at gmail.com  Wed Jun 10 19:39:49 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Wed, 10 Jun 2015 16:39:49 -0700
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <mlah2i$f7$1@ger.gmane.org>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
 <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>
 <mlah2i$f7$1@ger.gmane.org>
Message-ID: <CABicbJL5EqxTasVMb8yALSSss5xexf=9_7i-BhC2t5a-TxRCFA@mail.gmail.com>

On Wed, Jun 10, 2015 at 4:25 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 6/10/2015 6:10 PM, Devin Jeanpierre wrote:
>
>> The problem is that there are two different ways repr might write out
>> a dict equal to {'a': 1, 'b': 2}. This can make tests brittle
>
>
> Not if one compares objects rather than string representations of objects.
> I am strongly of the view that code and tests should be written to directly
> compare objects as much as possible.

For serialization formats that always output the same string for the
same data (like text format protos), there is no practical difference
between the two, except that if you're comparing text, you can easily
supply a diff to update one to match the other.

-- Devin


From tjreedy at udel.edu  Wed Jun 10 19:46:10 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 10 Jun 2015 19:46:10 -0400
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <CABicbJL5EqxTasVMb8yALSSss5xexf=9_7i-BhC2t5a-TxRCFA@mail.gmail.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
 <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>
 <mlah2i$f7$1@ger.gmane.org>
 <CABicbJL5EqxTasVMb8yALSSss5xexf=9_7i-BhC2t5a-TxRCFA@mail.gmail.com>
Message-ID: <mlai96$eot$2@ger.gmane.org>

On 6/10/2015 7:39 PM, Devin Jeanpierre wrote:
> On Wed, Jun 10, 2015 at 4:25 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> On 6/10/2015 6:10 PM, Devin Jeanpierre wrote:
>>
>>> The problem is that there are two different ways repr might write out
>>> a dict equal to {'a': 1, 'b': 2}. This can make tests brittle

You commented about *tests*

>> Not if one compares objects rather than string representations of objects.
>> I am strongly of the view that code and tests should be written to directly
>> compare objects as much as possible.

I responded about *tests*

> For serialization formats that always output the same string for the
> same data (like text format protos), there is no practical difference
> between the two, except that if you're comparing text, you can easily
> supply a diff to update one to match the other.

Serialization is a different issue.

-- 
Terry Jan Reedy



From jeanpierreda at gmail.com  Wed Jun 10 19:48:07 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Wed, 10 Jun 2015 16:48:07 -0700
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <CABicbJL5EqxTasVMb8yALSSss5xexf=9_7i-BhC2t5a-TxRCFA@mail.gmail.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
 <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>
 <mlah2i$f7$1@ger.gmane.org>
 <CABicbJL5EqxTasVMb8yALSSss5xexf=9_7i-BhC2t5a-TxRCFA@mail.gmail.com>
Message-ID: <CABicbJJm15MxCFz9XcEqN8O-wkPQzOqE3fuj9f8T7GgK563z3w@mail.gmail.com>

On Wed, Jun 10, 2015 at 4:39 PM, Devin Jeanpierre
<jeanpierreda at gmail.com> wrote:
> On Wed, Jun 10, 2015 at 4:25 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> On 6/10/2015 6:10 PM, Devin Jeanpierre wrote:
>>
>>> The problem is that there are two different ways repr might write out
>>> a dict equal to {'a': 1, 'b': 2}. This can make tests brittle
>>
>>
>> Not if one compares objects rather than string representations of objects.
>> I am strongly of the view that code and tests should be written to directly
>> compare objects as much as possible.
>
> For serialization formats that always output the same string for the
> same data (like text format protos), there is no practical difference
> between the two, except that if you're comparing text, you can easily
> supply a diff to update one to match the other.

Ugh, there's also the fiddly difference between what goes in and what
you read. A serialized data structure might contain lots of data that
is ignored by the deserializer (in protobuf), or it might contain data
which can't be loaded by the deserializer or produces weird /
incorrect results. Being able to inspect and test the serialized data
separately from the deserialized data is useful in that regard, so
that you know where the failure lies, but it's sort of fuzzy.

Some examples of where this crops up: pickles after you've moved a
class, JSON encoders that try to be clever and output invalid JSON,
protocol buffers with unexpected fields.

Overall, though, the diff thing is probably the bigger reason everyone
wants to do this sort of thing with serialized data. If you do it
right and are principled about it, I don't see a problem with it.

-- Devin


From rosuav at gmail.com  Wed Jun 10 19:58:08 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 11 Jun 2015 09:58:08 +1000
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
 <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>
Message-ID: <CAPTjJmrUWPv9sAO-UZc+xkSatS=LyLBdSJzpcNqFo1ByDKMUHg@mail.gmail.com>

On Thu, Jun 11, 2015 at 8:10 AM, Devin Jeanpierre
<jeanpierreda at gmail.com> wrote:
> The problem is that there are two different ways repr might write out
> a dict equal to {'a': 1, 'b': 2}. This can make tests brittle -- e.g.
> it's why doctest fails badly at examples involving dictionaries. Text
> format protocol buffers output everything sorted, so that you can do
> textual diffs for compatibility tests and such.

With Python's JSON module [1], you can pass sort_keys=True to
stipulate that the keys be lexically ordered, which should make the
output "canonical". Pike's Standards.JSON.encode() [2] can take a flag
value to canonicalize the output, which currently has the same effect
(sort mappings by their indices). I did a quick check for Ruby and
didn't find anything in its standard library JSON module, but knowing
Ruby, it'll be available somewhere in a gem. A web search for 'perl
json' brought up a CPAN link [4] that has a canonicalize option for
sorting by keys. So that's three out of four definite, one uncertain,
where it's pretty easy to ensure that you get byte-for-byte identical
output from a JSON encoder.

Even though failing doctests are a separate problem, it's useful to
have canonical output. Your diffs get less noisy, for instance.
Coupled with a human-readability flag (eg "indent=4" in Python,
"Standards.JSON.HUMAN_READABLE" in Pike) that splits the result over
multiple lines, it can make a pretty easy to diff file. Definitely
worth doing... and definitely worth using a JSON encoder rather than
repr().

ChrisA

[1] https://docs.python.org/3/library/json.html#json.dump
[2] http://pike.lysator.liu.se/generated/manual/modref/ex/predef_3A_3A/Standards/JSON.html
[3] http://ruby-doc.org/stdlib-2.0.0/libdoc/json/rdoc/JSON.html
[4] http://search.cpan.org/~makamaka/JSON-2.90/lib/JSON.pm#PERL_-%3E_JSON


From jeanpierreda at gmail.com  Wed Jun 10 20:02:01 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Wed, 10 Jun 2015 17:02:01 -0700
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <mlai96$eot$2@ger.gmane.org>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
 <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>
 <mlah2i$f7$1@ger.gmane.org>
 <CABicbJL5EqxTasVMb8yALSSss5xexf=9_7i-BhC2t5a-TxRCFA@mail.gmail.com>
 <mlai96$eot$2@ger.gmane.org>
Message-ID: <CABicbJLdsD1akN5hMKsLL-3R=bmKRYdCMe15xfWVJE_sq_X8cA@mail.gmail.com>

On Wed, Jun 10, 2015 at 4:46 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 6/10/2015 7:39 PM, Devin Jeanpierre wrote:
>>
>> On Wed, Jun 10, 2015 at 4:25 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>>>
>>> On 6/10/2015 6:10 PM, Devin Jeanpierre wrote:
>>>
>>>> The problem is that there are two different ways repr might write out
>>>> a dict equal to {'a': 1, 'b': 2}. This can make tests brittle
>
>
> You commented about *tests*
>
>>> Not if one compares objects rather than string representations of
>>> objects.
>>> I am strongly of the view that code and tests should be written to
>>> directly
>>> compare objects as much as possible.
>
>
> I responded about *tests*
>
>> For serialization formats that always output the same string for the
>> same data (like text format protos), there is no practical difference
>> between the two, except that if you're comparing text, you can easily
>> supply a diff to update one to match the other.
>
>
> Serialization is a different issue.

Yes, tests of code that uses serialization (caching, RPCs, etc.).

I mentioned above a sort of test that divides tests of a client and
server along RPC boundaries by providing fake queries and responses,
and testing that those are the queries and responses given by the
client and server. This way you don't need to actually start the
client and server to test them both and their interactions. This is
one example, there are other uses, but they go along the same lines.
For example, one can also imagine testing that a serialized structure
is identical across version changes, so that it's guaranteed to be
forwards/backwards compatible. It is not enough to test that the
deserialized form is, because it might differ substantially, as long
as the communicated serialized structure is the same.

-- Devin


From rosuav at gmail.com  Wed Jun 10 20:09:30 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 11 Jun 2015 10:09:30 +1000
Subject: Python NBSP DWIM
In-Reply-To: <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com>
References: <20150610082812.2ce887c3@bigbox.christie.dr>
 <mailman.344.1433946513.13271.python-list@python.org>
 <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmqS-sx2yxrPAcN6iv625hUOQKpM8bqUqBrNfcHvyzm8AQ@mail.gmail.com>

On Thu, Jun 11, 2015 at 3:11 AM, Steven D'Aprano <steve at pearwood.info>
wrote:
> (Oh, and for the record, there are at least two non-breaking spaces in
> Unicode, U+00A0 "NO-BREAK SPACE" and U+202F "NARROW NO-BREAK SPACE".)
>
> http://www.unicode.org/charts/PDF/U0080.pdf
> http://www.unicode.org/charts/PDF/U2000.pdf

And U+FEFF "ZERO WIDTH NO-BREAK SPACE", notable because it's also used as
the byte-order mark (as its counterpart, U+FFFE, is unallocated). I've been
fighting with VLC Media Player over the font it uses for subtitles; for
some bizarre reason, that font represents U+FEFF not with zero pixels of
emptiness, but with a box containing the letters "ZWN" "BSP" on two lines.
Yeah, because that totally takes up zero width and looks like blank space.

ChrisA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150611/c5da01d8/attachment.html>

From random832 at fastmail.us  Wed Jun 10 20:47:54 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 10 Jun 2015 20:47:54 -0400
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <ctrvkhFfutdU1@mid.individual.net>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <ml95k2$8ka$1@ger.gmane.org>
 <mailman.337.1433935377.13271.python-list@python.org>
 <878ubr3gv8.fsf@elektro.pacujo.net>
 <mailman.341.1433944371.13271.python-list@python.org>
 <ctrvkhFfutdU1@mid.individual.net>
Message-ID: <1433983674.345609.292407865.4BF04DC6@webmail.messagingengine.com>

On Wed, Jun 10, 2015, at 19:30, Gregory Ewing wrote:
> If whitelisting a type is the *only* thing you need to
> do to make it serialisable, I think that comes close
> enough to the stated goal of being able to "serialise
> all [potentially serialisable] language objects".

IMO the serialization framework should handle this by providing your own
way to look them up (almost but not entirely unlike providing your own
globals table to eval) rather than by having a whitelist.


From random832 at fastmail.us  Wed Jun 10 21:02:16 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 10 Jun 2015 21:02:16 -0400
Subject: Python NBSP DWIM
In-Reply-To: <CAPTjJmqS-sx2yxrPAcN6iv625hUOQKpM8bqUqBrNfcHvyzm8AQ@mail.gmail.com>
References: <20150610082812.2ce887c3@bigbox.christie.dr>
 <mailman.344.1433946513.13271.python-list@python.org>
 <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmqS-sx2yxrPAcN6iv625hUOQKpM8bqUqBrNfcHvyzm8AQ@mail.gmail.com>
Message-ID: <1433984536.349589.292410417.0EAD7A6C@webmail.messagingengine.com>

On Wed, Jun 10, 2015, at 20:09, Chris Angelico wrote:
> And U+FEFF "ZERO WIDTH NO-BREAK SPACE", notable because it's also used as
> the byte-order mark (as its counterpart, U+FFFE, is unallocated). I've
> been
> fighting with VLC Media Player over the font it uses for subtitles; for
> some bizarre reason, that font represents U+FEFF not with zero pixels of
> emptiness, but with a box containing the letters "ZWN" "BSP" on two
> lines.
> Yeah, because that totally takes up zero width and looks like blank
> space.

As I understand it, the proper behavior is that the ZWNBSP that is the
byte order mark shall never appear in an in-memory representation of the
first line of a BOM-encoded file, or any other line of the concatenation
of two BOM-encoded files, but should "vanish" when the file is opened
and first read from. So it shouldn't be showing up in your subtitles
regardless of its rendering behavior.

The real world, needless to say, isn't so nice.

IIRC there's also a font in MS windows that uses various glyphs which
are zero-width, but are not blank, to represent ZWJ, ZWNJ, RLM, and LRM.
Good for seeing what is happening, bad for actually rendering text
that's intended to contain these characters. Though there's another
argument that ideally a rendering engine should not render any such
glyph unless something like "visible controls" has been selected (the
real world, again, isn't so nice, which is why most symbols intended for
visible control style rendering have their own distinct code points
rather than using those of the control characters they represent).


From rosuav at gmail.com  Wed Jun 10 21:09:30 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 11 Jun 2015 11:09:30 +1000
Subject: Python NBSP DWIM
In-Reply-To: <1433984536.349589.292410417.0EAD7A6C@webmail.messagingengine.com>
References: <20150610082812.2ce887c3@bigbox.christie.dr>
 <mailman.344.1433946513.13271.python-list@python.org>
 <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmqS-sx2yxrPAcN6iv625hUOQKpM8bqUqBrNfcHvyzm8AQ@mail.gmail.com>
 <1433984536.349589.292410417.0EAD7A6C@webmail.messagingengine.com>
Message-ID: <CAPTjJmoiQKWCzsPRXMc=+BdFcuykFLJHwUKvMyDH6dMWPu1HpA@mail.gmail.com>

On Thu, Jun 11, 2015 at 11:02 AM, <random832 at fastmail.us> wrote:
>
> On Wed, Jun 10, 2015, at 20:09, Chris Angelico wrote:
> > And U+FEFF "ZERO WIDTH NO-BREAK SPACE", notable because it's also used as
> > the byte-order mark (as its counterpart, U+FFFE, is unallocated). I've
> > been
> > fighting with VLC Media Player over the font it uses for subtitles; for
> > some bizarre reason, that font represents U+FEFF not with zero pixels of
> > emptiness, but with a box containing the letters "ZWN" "BSP" on two
> > lines.
> > Yeah, because that totally takes up zero width and looks like blank
> > space.
>
> As I understand it, the proper behavior is that the ZWNBSP that is the
> byte order mark shall never appear in an in-memory representation of the
> first line of a BOM-encoded file, or any other line of the concatenation
> of two BOM-encoded files, but should "vanish" when the file is opened
> and first read from. So it shouldn't be showing up in your subtitles
> regardless of its rendering behavior.

It's a perfectly valid character for other purposes; it's coming up in
the middle of pieces of text, which should be 100% legal. No, it's a
font problem.

ChrisA


From minscheung at googlemail.com  Wed Jun 10 22:01:59 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Wed, 10 Jun 2015 19:01:59 -0700 (PDT)
Subject: How to find number of whole weeks between dates?
In-Reply-To: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
Message-ID: <c62196d7-d1fb-4f0d-bb22-090aac862145@googlegroups.com>

yes just whole weeks given any two months, I did looked into calendar module but couldn't find specifically what i need.


From steve at pearwood.info  Wed Jun 10 22:26:05 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 11 Jun 2015 12:26:05 +1000
Subject: Python NBSP DWIM
References: <20150610082812.2ce887c3@bigbox.christie.dr>
 <mailman.344.1433946513.13271.python-list@python.org>
 <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <mailman.370.1433981374.13271.python-list@python.org>
Message-ID: <5578f1be$0$12979$c3e8da3$5496439d@news.astraweb.com>

On Thu, 11 Jun 2015 10:09 am, Chris Angelico wrote:

> On Thu, Jun 11, 2015 at 3:11 AM, Steven D'Aprano <steve at pearwood.info>
> wrote:
>> (Oh, and for the record, there are at least two non-breaking spaces in
>> Unicode, U+00A0 "NO-BREAK SPACE" and U+202F "NARROW NO-BREAK SPACE".)
>>
>> http://www.unicode.org/charts/PDF/U0080.pdf
>> http://www.unicode.org/charts/PDF/U2000.pdf
> 
> And U+FEFF "ZERO WIDTH NO-BREAK SPACE", 

No, despite the name, that is not a space character, it is a formatting
character. Due to Unicode's stability policy, the name is stuck forever,
but it should not be treated as a space character:

py> unicodedata.category(' ')
'Zs'
py> unicodedata.category('\u00A0')  # NBSP
'Zs'
py> unicodedata.category('\uFEFF')  # ZWNBSP
'Cf'


Ideally, outside of the BOM, you should never come across a ZWNBSP. You
should use U+2060 WORD JOINER instead. But if you do come across one
outside of the BOM, it should be treated as a legitimate non-space
character:

http://www.unicode.org/faq/utf_bom.html#bom6

Although ZWNBSP is a "default ignorable" code point, I believe that the font
is well within its rights to show it with a visible glyph:

    "Fonts can contain glyphs intended for visible display of 
    default ignorable code points that would otherwise be 
    rendered invisibly when not supported."

http://www.unicode.org/faq/unsup_char.html


> notable because it's also used as 
> the byte-order mark (as its counterpart, U+FFFE, is unallocated). I've 
> been fighting with VLC Media Player over the font it uses for subtitles;
> for some bizarre reason, that font represents U+FEFF not with zero pixels
> of emptiness, but with a box containing the letters "ZWN" "BSP" on two
> lines. Yeah, because that totally takes up zero width and looks like blank
> space.

Why do the subtitles contain ZWNBSP in the first place? Surely they're not
English subtitles?


-- 
Steven



From rosuav at gmail.com  Wed Jun 10 23:05:35 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 11 Jun 2015 13:05:35 +1000
Subject: Python NBSP DWIM
In-Reply-To: <5578f1be$0$12979$c3e8da3$5496439d@news.astraweb.com>
References: <20150610082812.2ce887c3@bigbox.christie.dr>
 <mailman.344.1433946513.13271.python-list@python.org>
 <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <mailman.370.1433981374.13271.python-list@python.org>
 <5578f1be$0$12979$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmoqBtYOBnmWHdxss+d5fFqFXMxU6EB_yOBYgX9JNMH4mg@mail.gmail.com>

On Thu, Jun 11, 2015 at 12:26 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> No, despite the name, that is not a space character, it is a formatting
> character. Due to Unicode's stability policy, the name is stuck forever,
> but it should not be treated as a space character:
>
> py> unicodedata.category(' ')
> 'Zs'
> py> unicodedata.category('\u00A0')  # NBSP
> 'Zs'
> py> unicodedata.category('\uFEFF')  # ZWNBSP
> 'Cf'
>
>
> Ideally, outside of the BOM, you should never come across a ZWNBSP. You
> should use U+2060 WORD JOINER instead. But if you do come across one
> outside of the BOM, it should be treated as a legitimate non-space
> character:
>
> http://www.unicode.org/faq/utf_bom.html#bom6
>
> Although ZWNBSP is a "default ignorable" code point, I believe that the font
> is well within its rights to show it with a visible glyph:
>
>     "Fonts can contain glyphs intended for visible display of
>     default ignorable code points that would otherwise be
>     rendered invisibly when not supported."
>
> http://www.unicode.org/faq/unsup_char.html

Huh. Okay, my bad. I was under the impression that it was supposed to
take up no width, as the name implies, but stability trumps logic
sometimes. Learn something new every day.

>> notable because it's also used as
>> the byte-order mark (as its counterpart, U+FFFE, is unallocated). I've
>> been fighting with VLC Media Player over the font it uses for subtitles;
>> for some bizarre reason, that font represents U+FEFF not with zero pixels
>> of emptiness, but with a box containing the letters "ZWN" "BSP" on two
>> lines. Yeah, because that totally takes up zero width and looks like blank
>> space.
>
> Why do the subtitles contain ZWNBSP in the first place? Surely they're not
> English subtitles?

No, they're not :) The character comes up in the Cantonese and
Japanese subs for Once Upon A December.

http://youtu.be/CEpcUeWP0bg
http://youtu.be/WFZAaHrHens

Possibly some others in the series as well. It may well be a fault in
the subtitles, but most programs I've seen don't show U+FEFF as a big
fat box.

ChrisA


From random832 at fastmail.us  Wed Jun 10 23:18:01 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 10 Jun 2015 23:18:01 -0400
Subject: Python NBSP DWIM
In-Reply-To: <CAPTjJmoqBtYOBnmWHdxss+d5fFqFXMxU6EB_yOBYgX9JNMH4mg@mail.gmail.com>
References: <20150610082812.2ce887c3@bigbox.christie.dr>
 <mailman.344.1433946513.13271.python-list@python.org>
 <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <mailman.370.1433981374.13271.python-list@python.org>
 <5578f1be$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmoqBtYOBnmWHdxss+d5fFqFXMxU6EB_yOBYgX9JNMH4mg@mail.gmail.com>
Message-ID: <1433992681.383068.292482185.5744B721@webmail.messagingengine.com>

On Wed, Jun 10, 2015, at 23:05, Chris Angelico wrote:
> http://youtu.be/CEpcUeWP0bg
> http://youtu.be/WFZAaHrHens

An example of the actual subtitle text would be more useful than a
youtube link to the video, since we're unlikely to be able to see what
context the character appears in if our client doesn't show it. (I don't
think the default youtube player does). And you haven't even included a
time code.


From torriem at gmail.com  Wed Jun 10 23:19:47 2015
From: torriem at gmail.com (Michael Torrie)
Date: Wed, 10 Jun 2015 21:19:47 -0600
Subject: How to find number of whole weeks between dates?
In-Reply-To: <a650c18e-fa33-4139-bc68-e14b08273a8d@googlegroups.com>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
 <a650c18e-fa33-4139-bc68-e14b08273a8d@googlegroups.com>
Message-ID: <5578FE53.6090906@gmail.com>

On 06/10/2015 02:11 PM, Sebastian M Cheung via Python-list wrote:
> On Wednesday, June 10, 2015 at 6:06:09 PM UTC+1, Sebastian M Cheung wrote:
>> Say in 2014 April to May whole weeks would be 7th, 14th 28th April and  May would be 5th, 12th and 19th. So expecting 7 whole weeks in total
> 
> What I mean is given two dates I want to find WHOLE weeks, so if given the 2014 calendar and function has two inputs (4th and 5th month) then 7th, 14th, 21st and 28th from April with 28th April week carrying into May, and then 5th, 12th and 19th May to give total of 7 whole weeks, because 26th May is not a whole week and will not be counted.
> 
> Hope thats clear.

I think Joel had the right idea.  First calculate the rough number of
weeks by taking the number of days between the date and divide by seven.
Then check to see what the start date's day of week is, and adjust the
rough week count down by one if it's not the first day of the week.  I'm
not sure if you have to check the end date's day of week or not.  I kind
of think checking the first one only is sufficient, but I could be
wrong.  You'll have to code it up and test it, which I assume you've
been doing up to this point, even though you haven't shared any code.


From steve at pearwood.info  Wed Jun 10 23:21:25 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 11 Jun 2015 13:21:25 +1000
Subject: enhancement request: make py3 read/write py2 pickle format
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
 <mailman.361.1433974273.13271.python-list@python.org>
Message-ID: <5578feb6$0$12984$c3e8da3$5496439d@news.astraweb.com>

On Thu, 11 Jun 2015 08:10 am, Devin Jeanpierre wrote:

[...]
>> For literals, the canonical form is that understood by Python. I'm pretty
>> sure that these have been stable since the days of Python 1.0, and will
>> remain so pretty much forever:
> 
> The problem is that there are two different ways repr might write out
> a dict equal to {'a': 1, 'b': 2}. This can make tests brittle -- e.g.
> it's why doctest fails badly at examples involving dictionaries. 

Only if they are badly written.

Yes, dicts are *less convenient* for doctests, but if they fail, the blame
is on the author of the tests themselves, not doctest.

Unordered output is not a problem for dicts, because dicts also have
unordered *input*. It doesn't matter whether you input {'a':1,'b':2} or
{'b':2,'a':1}, you will get the same dict either way.


[...]
> I could spend a bunch of time writing yet another config file format,
> or I could use text format protocol buffers, YAML, or TOML and call it
> a day.

Writing a rc parser is so trivial that it's almost easier to just write it
than it is to look up the APIs for YAML or JSON, to say nothing of the
rigmarole of defining a protocol buffer config file, compiling it,
importing the module, and using that.

def read(configfile):
    config = collections.OrderedDict()
    with open(configfile) as f:
        for line in f:
            line = line.strip()
            if line.startswith('#"): continue
            key, value = line.split("=", 1)
            key = key.rstrip()
            value = value.lstrip()
            config[key] = ast.literal_eval(value)
    return config


That's a basic, *but acceptable*, rc parser written in literally under a
minute. At the risk of ending up with egg on my face, I reckon that it's so
simple and so obviously correct that I can tell it works correctly without
even testing it. (Famous last words, huh?)

Unlike any of the richer, more powerful serialisation formats like YAML,
JSON, or protocol buffer, its not only human readable but human writable
too. By which I mean, while it is *possible* for a sufficiently motivated
person to write correctly formatted JSON, YAML or even XML, it's not really
something you would choose to do willingly. But Unix sys admins hand-edit
rc files every day.

But of course this also means it's less powerful and can deal with few types
of data. Power comes at a cost of complexity, and simplicity itself can be
a virtue. I wouldn't use JSON etc. for config files until I was sure that a
simpler INI or RC file wasn't sufficient for my needs.

Some how I have drifted away from serialisation in general to specifically
config files... never mind.


[...]
> The problem is when you have your config file format using python
> literals, and another programmer wants to deal with it and doesn't
> look at your codebase, and things like that. When transferring data,
> this can happen a lot, since you are often not the user of the data
> you wrote, and you can't control how others consume it. 

Not only can I not control how they consume it, but I don't care how they
consume it :-)

I hear what you are saying, and I don't disagree with it. I'm just standing
up for simplicity as a virtue when appropriate. If I'm writing a script to
save a bunch of values to pass to another script after some human editing,
it's faster for me to just write out the key:value pairs than it is to
learn how to use protocol buffer, deal with a separate compilation step,
etc. It's actually easier to write out, and read in, the key:values than to
use the configfile module. If you don't need multiple sections, default
values, or variable interpolation, even configparser is overkill.

But if I'm swapping data with others, or if I have to use a richer set of
types or functionality, then naturally I'm going to need something more
powerful, preferably something standard so I don't have to document the
internal format, just say "use XML with this schema" or whatever.


> They might use 
> eval even if you didn't mean for them to. For example, in JavaScript,
> this was once a common problem for services exposing JSON, and it
> still happens even now.

<shrug> If they choose to use eval, *that's not my fault*. You can't stop
them from deserialising your data and then passing any and all strings to
eval, so why should I be expected to stop them from something similar?


[...]
>> Beyond simple needs, like rc files, literal_eval is not sufficient. You
>> can't use it to deserialise arbitrary objects. That might be a feature,
>> but if you need something more powerful than basic ints, floats, strings
>> and a few others, literal_eval will not be powerful enough.
> 
> No, it is powerful enough. After all, JSON has the same limitations.

In the sense that you can build arbitrary objects from a combination of a
few basic types, yes, literal_eval is "powerful enough" if you are prepared
to re-invent JSON, YAML, or protocol buffer.

But I'm not talking about re-inventing what already exists. If I want JSON,
I'll use JSON, not spend weeks or months re-writing it from scratch. I
can't do this:

class MyClass:
    pass

a = MyClass()
serialised = repr(a)
b = ast.literal_eval(serialised)
assert a == b

which is what I mean when I say literal_eval isn't powerful enough to handle
arbitrary types. That's not a bug, that's a feature of literal_eval. It is
*designed* to have that limitation.




-- 
Steven



From steve at pearwood.info  Wed Jun 10 23:27:09 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 11 Jun 2015 13:27:09 +1000
Subject: Python NBSP DWIM
References: <20150610082812.2ce887c3@bigbox.christie.dr>
 <mailman.344.1433946513.13271.python-list@python.org>
 <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <mailman.370.1433981374.13271.python-list@python.org>
 <5578f1be$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <mailman.374.1433991937.13271.python-list@python.org>
Message-ID: <5579000e$0$12986$c3e8da3$5496439d@news.astraweb.com>

On Thu, 11 Jun 2015 01:05 pm, Chris Angelico wrote:
[...]
>> Why do the subtitles contain ZWNBSP in the first place? Surely they're
>> not English subtitles?
> 
> No, they're not :) The character comes up in the Cantonese and
> Japanese subs for Once Upon A December.
> 
> http://youtu.be/CEpcUeWP0bg
> http://youtu.be/WFZAaHrHens
> 
> Possibly some others in the series as well. It may well be a fault in
> the subtitles, but most programs I've seen don't show U+FEFF as a big
> fat box.

I think that for backwards compatibility, applications (or fonts) are
permitted to treat U+FEFF as a zero-width invisible character, so perhaps
you can raise a feature request with VLC.



-- 
Steven



From rosuav at gmail.com  Wed Jun 10 23:28:25 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 11 Jun 2015 13:28:25 +1000
Subject: Python NBSP DWIM
In-Reply-To: <1433992681.383068.292482185.5744B721@webmail.messagingengine.com>
References: <20150610082812.2ce887c3@bigbox.christie.dr>
 <mailman.344.1433946513.13271.python-list@python.org>
 <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <mailman.370.1433981374.13271.python-list@python.org>
 <5578f1be$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmoqBtYOBnmWHdxss+d5fFqFXMxU6EB_yOBYgX9JNMH4mg@mail.gmail.com>
 <1433992681.383068.292482185.5744B721@webmail.messagingengine.com>
Message-ID: <CAPTjJmpZqRkjQsJPp9ppJ8N6SwqT=Sg-HvHAjhefKZA8TOP7ZQ@mail.gmail.com>

On Thu, Jun 11, 2015 at 1:18 PM,  <random832 at fastmail.us> wrote:
> On Wed, Jun 10, 2015, at 23:05, Chris Angelico wrote:
>> http://youtu.be/CEpcUeWP0bg
>> http://youtu.be/WFZAaHrHens
>
> An example of the actual subtitle text would be more useful than a
> youtube link to the video, since we're unlikely to be able to see what
> context the character appears in if our client doesn't show it. (I don't
> think the default youtube player does). And you haven't even included a
> time code.

Unfortunately I can't really offer anything better, as the text I saw
was after a lot of processing (youtube-dl, then some other
post-processing), and I don't actually remember which file it was that
bugged me about this, now. But the subs/annotations (visible in the
default player if you turn on "Subtitles" down the bottom) do include
U+FEFF; in each case, it's on the very last line of the song, although
that's not where I remember it occurring.

ChrisA


From ian.g.kelly at gmail.com  Wed Jun 10 23:35:53 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 10 Jun 2015 21:35:53 -0600
Subject: How to find number of whole weeks between dates?
In-Reply-To: <c62196d7-d1fb-4f0d-bb22-090aac862145@googlegroups.com>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
 <c62196d7-d1fb-4f0d-bb22-090aac862145@googlegroups.com>
Message-ID: <CALwzid=rW8tvBubFvnYFN5dDSM2zJZZu_UaNwhgJ=uU8M1eT2g@mail.gmail.com>

On Wed, Jun 10, 2015 at 8:01 PM, Sebastian M Cheung via Python-list
<python-list at python.org> wrote:
> yes just whole weeks given any two months, I did looked into calendar module but couldn't find specifically what i need.

>>> cal.monthdays2calendar(2014, 4) + cal.monthdays2calendar(2014, 5)
[[(0, 0), (1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6)], [(7, 0),
(8, 1), (9, 2), (10, 3), (11, 4), (12, 5), (13, 6)], [(14, 0), (15,
1), (16, 2), (17, 3), (18, 4), (19, 5), (20, 6)], [(21, 0), (22, 1),
(23, 2), (24, 3), (25, 4), (26, 5), (27, 6)], [(28, 0), (29, 1), (30,
2), (0, 3), (0, 4), (0, 5), (0, 6)], [(0, 0), (0, 1), (0, 2), (1, 3),
(2, 4), (3, 5), (4, 6)], [(5, 0), (6, 1), (7, 2), (8, 3), (9, 4), (10,
5), (11, 6)], [(12, 0), (13, 1), (14, 2), (15, 3), (16, 4), (17, 5),
(18, 6)], [(19, 0), (20, 1), (21, 2), (22, 3), (23, 4), (24, 5), (25,
6)], [(26, 0), (27, 1), (28, 2), (29, 3), (30, 4), (31, 5), (0, 6)]]

You just need to:

1) Trim the first and last weeks off since they contain invalid dates.
2) Merge the overlapping last week of April and first week of May.
3) Count the resulting number of weeks in the list.

Alternatively, the dateutil.rrule module could probably be used to do
this fairly easily, but it's a third-party module and not part of the
standard library.

https://labix.org/python-dateutil


From rosuav at gmail.com  Wed Jun 10 23:36:15 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 11 Jun 2015 13:36:15 +1000
Subject: How to find number of whole weeks between dates?
In-Reply-To: <5578FE53.6090906@gmail.com>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
 <a650c18e-fa33-4139-bc68-e14b08273a8d@googlegroups.com>
 <5578FE53.6090906@gmail.com>
Message-ID: <CAPTjJmo4Mn=Gx5WUGmda7zL3a6hqxzMpySbxM3z6ZHBrZFEvhA@mail.gmail.com>

On Thu, Jun 11, 2015 at 1:19 PM, Michael Torrie <torriem at gmail.com> wrote:
> I think Joel had the right idea.  First calculate the rough number of
> weeks by taking the number of days between the date and divide by seven.
> Then check to see what the start date's day of week is, and adjust the
> rough week count down by one if it's not the first day of the week.  I'm
> not sure if you have to check the end date's day of week or not.  I kind
> of think checking the first one only is sufficient, but I could be
> wrong.  You'll have to code it up and test it, which I assume you've
> been doing up to this point, even though you haven't shared any code.

Alternatively, you could start by rounding the start date up to the
next week boundary, then round the end date down to the previous week
boundary, and then calculate from there. Something like this:

>>> start = datetime.date(2015, 1, 4)
>>> end = datetime.date(2015, 4, 2)
>>> start += datetime.timedelta(7-start.isoweekday())
>>> end -= datetime.timedelta(end.isoweekday() % 7)

Now both dates represent Sundays. If either already did, it hasn't been changed.

>>> (end - start).days//7
12

There are twelve complete Sunday-to-Sunday weeks (plus any loose days
either end) between the original dates.

Depending on your definition of "complete week", you may need to
adjust this code some.

ChrisA


From rosuav at gmail.com  Wed Jun 10 23:37:59 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 11 Jun 2015 13:37:59 +1000
Subject: Python NBSP DWIM
In-Reply-To: <5579000e$0$12986$c3e8da3$5496439d@news.astraweb.com>
References: <20150610082812.2ce887c3@bigbox.christie.dr>
 <mailman.344.1433946513.13271.python-list@python.org>
 <55786fd5$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <mailman.370.1433981374.13271.python-list@python.org>
 <5578f1be$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <mailman.374.1433991937.13271.python-list@python.org>
 <5579000e$0$12986$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmo45rL6KF+EnZ=Q0G=NhGhY3nSPV3hP-qo5TGJrN=DTqw@mail.gmail.com>

On Thu, Jun 11, 2015 at 1:27 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Thu, 11 Jun 2015 01:05 pm, Chris Angelico wrote:
> [...]
>>> Why do the subtitles contain ZWNBSP in the first place? Surely they're
>>> not English subtitles?
>>
>> No, they're not :) The character comes up in the Cantonese and
>> Japanese subs for Once Upon A December.
>>
>> http://youtu.be/CEpcUeWP0bg
>> http://youtu.be/WFZAaHrHens
>>
>> Possibly some others in the series as well. It may well be a fault in
>> the subtitles, but most programs I've seen don't show U+FEFF as a big
>> fat box.
>
> I think that for backwards compatibility, applications (or fonts) are
> permitted to treat U+FEFF as a zero-width invisible character, so perhaps
> you can raise a feature request with VLC.

Yeah. Well, like I said - learn something new every day. I didn't know
it wasn't a bug. (Though it'd still be a font issue, not a VLC one.
With other fonts, it comes up looking different, in some cases
invisible. Unfortunately, the fonts that look good aren't the fonts
that have glyphs for all characters, so I need to figure out why font
substitution isn't working right. But that's a separate issue.)

ChrisA


From ian.g.kelly at gmail.com  Wed Jun 10 23:45:32 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 10 Jun 2015 21:45:32 -0600
Subject: How to find number of whole weeks between dates?
In-Reply-To: <5578FE53.6090906@gmail.com>
References: <a2056385-f99b-4691-9a33-5def63216e9c@googlegroups.com>
 <a650c18e-fa33-4139-bc68-e14b08273a8d@googlegroups.com>
 <5578FE53.6090906@gmail.com>
Message-ID: <CALwzid=Bq8pderZxoRhrg6Uq=czy191fj0DyfvnGumOzeVgryQ@mail.gmail.com>

On Wed, Jun 10, 2015 at 9:19 PM, Michael Torrie <torriem at gmail.com> wrote:
> On 06/10/2015 02:11 PM, Sebastian M Cheung via Python-list wrote:
>> On Wednesday, June 10, 2015 at 6:06:09 PM UTC+1, Sebastian M Cheung wrote:
>>> Say in 2014 April to May whole weeks would be 7th, 14th 28th April and  May would be 5th, 12th and 19th. So expecting 7 whole weeks in total
>>
>> What I mean is given two dates I want to find WHOLE weeks, so if given the 2014 calendar and function has two inputs (4th and 5th month) then 7th, 14th, 21st and 28th from April with 28th April week carrying into May, and then 5th, 12th and 19th May to give total of 7 whole weeks, because 26th May is not a whole week and will not be counted.
>>
>> Hope thats clear.
>
> I think Joel had the right idea.  First calculate the rough number of
> weeks by taking the number of days between the date and divide by seven.
> Then check to see what the start date's day of week is, and adjust the
> rough week count down by one if it's not the first day of the week.  I'm
> not sure if you have to check the end date's day of week or not.  I kind
> of think checking the first one only is sufficient, but I could be
> wrong.  You'll have to code it up and test it, which I assume you've
> been doing up to this point, even though you haven't shared any code.

I don't think the logic is quite right. Consider:

>>> cal = calendar.TextCalendar()
>>> print(cal.formatmonth(2014, 6))
     June 2014
Mo Tu We Th Fr Sa Su
                   1
 2  3  4  5  6  7  8
 9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30

>>> date(2014, 7, 1) - date(2014, 6, 1)
datetime.timedelta(30)
>>> _.days // 7 - 1
3


From marko at pacujo.net  Thu Jun 11 00:08:46 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 11 Jun 2015 07:08:46 +0300
Subject: enhancement request: make py3 read/write py2 pickle format
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
 <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>
 <mlah2i$f7$1@ger.gmane.org>
 <CABicbJL5EqxTasVMb8yALSSss5xexf=9_7i-BhC2t5a-TxRCFA@mail.gmail.com>
 <mlai96$eot$2@ger.gmane.org>
 <mailman.369.1433980969.13271.python-list@python.org>
Message-ID: <87r3pivqcx.fsf@elektro.pacujo.net>

Devin Jeanpierre <jeanpierreda at gmail.com>:

> For example, one can also imagine testing that a serialized structure
> is identical across version changes, so that it's guaranteed to be
> forwards/backwards compatible. It is not enough to test that the
> deserialized form is, because it might differ substantially, as long
> as the communicated serialized structure is the same.

There are merits to canonical serialization formats, but that approach
to testing is far too simplistic. A test case should accept all observed
behavior that is correct.


Marko


From azrocknroll at gmail.com  Thu Jun 11 00:50:54 2015
From: azrocknroll at gmail.com (c me)
Date: Wed, 10 Jun 2015 21:50:54 -0700 (PDT)
Subject: New Python student needs help with execution
Message-ID: <77ccd04d-60fe-495c-877f-871a50c89cd2@googlegroups.com>

I installed 2.7.9 on a Win8.1 machine. The Coursera instructor did a simple install then executed Python from a file in which he'd put a simple hello world script.  My similar documents folder cannot see the python executable.  How do I make this work?


From jeanpierreda at gmail.com  Thu Jun 11 01:39:46 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Wed, 10 Jun 2015 22:39:46 -0700
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <5578feb6$0$12984$c3e8da3$5496439d@news.astraweb.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
 <mailman.361.1433974273.13271.python-list@python.org>
 <5578feb6$0$12984$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CABicbJ++qL0wSQ4A=LfZB-mS7COG1iyME+1fxVPJYadG_efDfg@mail.gmail.com>

Snipped aplenty.

On Wed, Jun 10, 2015 at 8:21 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Thu, 11 Jun 2015 08:10 am, Devin Jeanpierre wrote:
> [...]
>> I could spend a bunch of time writing yet another config file format,
>> or I could use text format protocol buffers, YAML, or TOML and call it
>> a day.
>
> Writing a rc parser is so trivial that it's almost easier to just write it
> than it is to look up the APIs for YAML or JSON, to say nothing of the
> rigmarole of defining a protocol buffer config file, compiling it,
> importing the module, and using that.
>
-snip
>
> That's a basic, *but acceptable*, rc parser written in literally under a
> minute. At the risk of ending up with egg on my face, I reckon that it's so
> simple and so obviously correct that I can tell it works correctly without
> even testing it. (Famous last words, huh?)

I won't try to egg you. That said, you have to write tests. Also,
everyone who uses it has to learn the format and API, and it may have
corner cases you aren't aware of, it has to get ported to python 3 if
you wrote it for python 2, the parsing errors are obscure and might
need improvement, and so on. There's a place for this, but I suspect
it is small compared to the place where it seemed like a good idea at
the time.

>>> Beyond simple needs, like rc files, literal_eval is not sufficient. You
>>> can't use it to deserialise arbitrary objects. That might be a feature,
>>> but if you need something more powerful than basic ints, floats, strings
>>> and a few others, literal_eval will not be powerful enough.
>>
>> No, it is powerful enough. After all, JSON has the same limitations.
>
> In the sense that you can build arbitrary objects from a combination of a
> few basic types, yes, literal_eval is "powerful enough" if you are prepared
> to re-invent JSON, YAML, or protocol buffer.
>
> But I'm not talking about re-inventing what already exists. If I want JSON,
> I'll use JSON, not spend weeks or months re-writing it from scratch. I
> can't do this:
>
> class MyClass:
>     pass
>
> a = MyClass()
> serialised = repr(a)
> b = ast.literal_eval(serialised)
> assert a == b

I don't understand. You can't do that in JSON, YAML, XML, or protocol
buffers, either. They only provide a small set of types, comparable to
(but smaller) than the set of types you get from literal_eval/repr.

-- Devin


From breamoreboy at yahoo.co.uk  Thu Jun 11 03:32:18 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 11 Jun 2015 08:32:18 +0100
Subject: New Python student needs help with execution
In-Reply-To: <77ccd04d-60fe-495c-877f-871a50c89cd2@googlegroups.com>
References: <77ccd04d-60fe-495c-877f-871a50c89cd2@googlegroups.com>
Message-ID: <mlbdic$vjo$1@ger.gmane.org>

On 11/06/2015 05:50, c me wrote:
> I installed 2.7.9 on a Win8.1 machine. The Coursera instructor did a simple install then executed Python from a file in which he'd put a simple hello world script.  My similar documents folder cannot see the python executable.  How do I make this work?
>

I'm not sure what you're asking but this 
https://docs.python.org/3/using/windows.html should help.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lac at openend.se  Thu Jun 11 04:52:50 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 11 Jun 2015 10:52:50 +0200
Subject: New Python student needs help with execution
In-Reply-To: Message from c me <azrocknroll@gmail.com> of "Wed,
 10 Jun 2015 21:50:54 -0700."
 <77ccd04d-60fe-495c-877f-871a50c89cd2@googlegroups.com>
References: <77ccd04d-60fe-495c-877f-871a50c89cd2@googlegroups.com>
Message-ID: <201506110852.t5B8qo0m000584@fido.openend.se>

In a message of Wed, 10 Jun 2015 21:50:54 -0700, c me writes:
>I installed 2.7.9 on a Win8.1 machine. The Coursera instructor did a simple install then executed Python from a file in which he'd put a simple hello world script.  My similar documents folder cannot see the python executable.  How do I make this work?
>-- 
>https://mail.python.org/mailman/listinfo/python-list

You need to set your PYTHONPATH.
Instructions here should help.
https://docs.python.org/2.7/using/windows.html

It is possible that you need to do some other things too, but again
that's the doc for it.

Laura


From steve+comp.lang.python at pearwood.info  Thu Jun 11 05:20:43 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 11 Jun 2015 19:20:43 +1000
Subject: enhancement request: make py3 read/write py2 pickle format
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
 <mailman.361.1433974273.13271.python-list@python.org>
 <5578feb6$0$12984$c3e8da3$5496439d@news.astraweb.com>
 <mailman.382.1434001230.13271.python-list@python.org>
Message-ID: <557952eb$0$21718$c3e8da3@news.astraweb.com>

On Thursday 11 June 2015 15:39, Devin Jeanpierre wrote:

>> But I'm not talking about re-inventing what already exists. If I want
>> JSON, I'll use JSON, not spend weeks or months re-writing it from
>> scratch. I can't do this:
>>
>> class MyClass:
>>pass
>>
>> a = MyClass()
>> serialised = repr(a)
>> b = ast.literal_eval(serialised)
>> assert a == b
> 
> I don't understand. You can't do that in JSON, YAML, XML, or protocol
> buffers, either. They only provide a small set of types, comparable to
> (but smaller) than the set of types you get from literal_eval/repr.

Well, what do people do when they want to serialise something like MyClass, 
but have to use (say) JSON rather than pickle?

I'd write a method to export enough information (as JSON) to reconstruct the 
instance, and another method to take that JSON and build an instance. If I'm 
going to do all that, *I would use JSON* rather than try to create my own 
format invented from scratch using only literal_eval.


Although... I suppose if I really wanted to be quick and dirty about it...


py> import ast
py> class MyClass(object): pass
... 
py> a = MyClass()
py> s = repr(a.__dict__)
py> b = object.__new__(MyClass)
py> b.__dict__ = ast.literal_eval(s)
py> b
<__main__.MyClass object at 0xb725218c>


;-)



-- 
Steve



From David.Aldrich at EMEA.NEC.COM  Thu Jun 11 06:10:23 2015
From: David.Aldrich at EMEA.NEC.COM (David Aldrich)
Date: Thu, 11 Jun 2015 10:10:23 +0000
Subject: A basic dictionary question
Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B980FE0@EX10MBX02.EU.NEC.COM>

Hi

I am fairly new to Python.  I am writing some code that uses a dictionary to store definitions of hardware registers.  Here is a small part of it:

import sys

register = {
    'address'  : 0x3001c,
    'fields' : {
        'FieldA' : {
            'range' : (31,20),
        },
        'FieldB' : {
            'range' : (19,16),
        },
    },
    'width' : 32
};

def main():
    fields = register['fields']
    for field, range_dir in fields:         <== This line fails
        range_dir = field['range']
        x,y = range_dir['range']
        print(x, y)

if __name__ == '__main__':
    main()

I want the code to print the range of bits of each field defined in the dictionary.

The output is:

Traceback (most recent call last):
  File "testdir.py", line 32, in <module>
    main()
  File "testdir.py", line 26, in main
    for field, range_dir in fields:
ValueError: too many values to unpack (expected 2)

Please will someone explain what I am doing wrong?

Also I would like to ask how I could print the ranges in the order they are defined.  Should I use a different dictionary class or could I add a field to the dictionary/list to achieve this?

Best regards

David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150611/1724352d/attachment.html>

From adebayo.abraham at gmail.com  Thu Jun 11 06:44:59 2015
From: adebayo.abraham at gmail.com (adebayo.abraham at gmail.com)
Date: Thu, 11 Jun 2015 03:44:59 -0700 (PDT)
Subject: PYTHON QUESTION
Message-ID: <029c49a7-7e82-4755-8631-6891afc663c5@googlegroups.com>

Help with this problem!

Temperature converter
Description

Write two functions that will convert temperatures back and forth from the Celsius and Fahrenheit temperature scales. The formulas for making the conversion are as follows:

  Tc=(5/9)*(Tf-32)
  Tf=(9/5)*Tc+32

where Tc is the Celsius temperature and Tf is the Fahrenheit temperature. More information and further descriptions of how to do the conversion can be found at this NASA Webpage. If you finish this assignment quickly, add a function to calculate the wind chill.
Input

Your program should ask the user to input a temperature and then which conversion they would like to perform.


From dpalao.python at gmail.com  Thu Jun 11 06:55:12 2015
From: dpalao.python at gmail.com (David Palao)
Date: Thu, 11 Jun 2015 12:55:12 +0200
Subject: PYTHON QUESTION
In-Reply-To: <029c49a7-7e82-4755-8631-6891afc663c5@googlegroups.com>
References: <029c49a7-7e82-4755-8631-6891afc663c5@googlegroups.com>
Message-ID: <CAKUKWz=UD4WEnMmr5zGfGDeXj0JDpgVz2qAUN-BiS7XbMFMm0w@mail.gmail.com>

2015-06-11 12:44 GMT+02:00  <adebayo.abraham at gmail.com>:
> Help with this problem!
>
> Temperature converter
> Description
>
> Write two functions that will convert temperatures back and forth from the Celsius and Fahrenheit temperature scales. The formulas for making the conversion are as follows:
>
>   Tc=(5/9)*(Tf-32)
>   Tf=(9/5)*Tc+32
>
> where Tc is the Celsius temperature and Tf is the Fahrenheit temperature. More information and further descriptions of how to do the conversion can be found at this NASA Webpage. If you finish this assignment quickly, add a function to calculate the wind chill.
> Input
>
> Your program should ask the user to input a temperature and then which conversion they would like to perform.
> --
> https://mail.python.org/mailman/listinfo/python-list

Hello,
While people here is kindly helping others, it doesn't work the way
you are posing it: we will not do your homeworks.
If you want some help, please, present some code that you wrote and
does not work, or a specific question about the problem. For instance,
how can I write a function in python?
BTW, probably your questions should go to the tutor mailing list...

Best


From storchaka at gmail.com  Thu Jun 11 07:11:25 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 11 Jun 2015 14:11:25 +0300
Subject: enhancement request: make py3 read/write py2 pickle format
In-Reply-To: <CAPTjJmrUWPv9sAO-UZc+xkSatS=LyLBdSJzpcNqFo1ByDKMUHg@mail.gmail.com>
References: <ml7a2p$hp$1@ger.gmane.org>
 <CAMw+j7K7JmRPo3PnzTE2rZEstdZRF085+VA=v1ieMMHk2Gp6mQ@mail.gmail.com>
 <CABicbJ+e02OKYi48kbjFFutyLUGgkEQdiPZqiB657AO+QKu11Q@mail.gmail.com>
 <mailman.325.1433891190.13271.python-list@python.org>
 <55778208$0$2899$e4fe514c@news2.news.xs4all.nl>
 <mailman.328.1433897321.13271.python-list@python.org>
 <5577b46d$0$12975$c3e8da3$5496439d@news.astraweb.com>
 <mailman.332.1433911768.13271.python-list@python.org>
 <5578053a$0$11102$c3e8da3@news.astraweb.com>
 <CABicbJJFURk+kwUO-wz8K5ZXY5TYUpqor1DLr0T30PaLkW5w4A@mail.gmail.com>
 <CAPTjJmrUWPv9sAO-UZc+xkSatS=LyLBdSJzpcNqFo1ByDKMUHg@mail.gmail.com>
Message-ID: <mlbqcu$i79$1@ger.gmane.org>

On 11.06.15 02:58, Chris Angelico wrote:
> On Thu, Jun 11, 2015 at 8:10 AM, Devin Jeanpierre
> <jeanpierreda at gmail.com> wrote:
>> The problem is that there are two different ways repr might write out
>> a dict equal to {'a': 1, 'b': 2}. This can make tests brittle -- e.g.
>> it's why doctest fails badly at examples involving dictionaries. Text
>> format protocol buffers output everything sorted, so that you can do
>> textual diffs for compatibility tests and such.
>
> With Python's JSON module [1], you can pass sort_keys=True to
> stipulate that the keys be lexically ordered, which should make the
> output "canonical". Pike's Standards.JSON.encode() [2] can take a flag
> value to canonicalize the output, which currently has the same effect
> (sort mappings by their indices). I did a quick check for Ruby and
> didn't find anything in its standard library JSON module, but knowing
> Ruby, it'll be available somewhere in a gem.

AFAIK Ruby's dicts are ordered. So the output is pretty stable.




From skybuck2000 at hotmail.com  Thu Jun 11 07:15:33 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 11 Jun 2015 13:15:33 +0200
Subject: Parser needed.
In-Reply-To: <mailman.327.1433892811.13271.python-list@python.org>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
 <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
 <4252d$55709249$5419aafe$25285@news.ziggo.nl>
 <74396$55764fb5$5419aafe$6142@news.ziggo.nl>
 <mailman.310.1433842760.13271.python-list@python.org>
 <48a90$5576da1e$5419aafe$34050@news.ziggo.nl>
 <mailman.327.1433892811.13271.python-list@python.org>
Message-ID: <efad0$55796dd4$5419aafe$46357@news.ziggo.nl>

Well it did help a little bit.

Somebody asked if there was already a parser for it.

I answered yes in C#.

So I took a closer look at it... and learned something from it.

Maybe I would have done that anyway... or maybe not...

Now we will never know... but I am happy that the parser is now ok, done and 
pretty easy extendable.

I think it's probably my first really good one.

I did do a little assembler before... but not sure if it was any good.

I also read before I started the thread... what "parse" means in dictionary.

That also helped me understand it a little bit better.

And also I read how it's usually done... tokens/tokenize etc... kinda 
already knew that... but still.

Turned out to be quite easy...

But at the start /always it seemed so difficult...

So any little bits of help/advice/information can help ! :)

Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Thu Jun 11 07:19:47 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 11 Jun 2015 13:19:47 +0200
Subject: I don't like the OO part of python. In particular the "self" keyword
 everywhere.
Message-ID: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>

Hello,

I don't like the object orientated part of Python.

The idea/prospect of having to write "self" everywhere... seems very 
horrorific and a huge time waster.

(Perhaps the module thing of python might help in future not sure about 
that).

What are your thoughts on the "self" thing/requirement.

I only want replies from expert programmers, cause we need a language for 
expert programmers...

Not noobies that need to be hand-held...

Personally I think I could do just fine with the "self" keyword everywhere.

So question is... can the python interpreter/compiler be written in such a 
way that self can be left out ?

In other words: Is there any hope... that this part of the language will be 
cleaned up some day ?

Are there any tricks to get rid of it ?

Maybe "white" like in Delphi ?

I haven't written much OO code yet in Python... and don't plan on doing it 
too...

Cause it looks hellish confusing... and clouded/clodded.

I think I have better things to do then to insert "self" everywhere...

It's almost like "self" masturbation  LOL.

Bye,
  Skybuck =D 



From skybuck2000 at hotmail.com  Thu Jun 11 07:21:17 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 11 Jun 2015 13:21:17 +0200
Subject: I don't like the OO part of python. In particular the "self"
 keyword everywhere.
Message-ID: <2c87e$55796f2c$5419aafe$47209@news.ziggo.nl>

Little typo corrected... it's a common typo I seem to make.

"with" had to be "without" see ***.

"Skybuck Flying"  wrote in message news:...

Hello,

I don't like the object orientated part of Python.

The idea/prospect of having to write "self" everywhere... seems very
horrorific and a huge time waster.

(Perhaps the module thing of python might help in future not sure about
that).

What are your thoughts on the "self" thing/requirement.

I only want replies from expert programmers, cause we need a language for
expert programmers...

Not noobies that need to be hand-held...

Personally I think I could do just fine without (***) the "self" keyword 
everywhere.

So question is... can the python interpreter/compiler be written in such a
way that self can be left out ?

In other words: Is there any hope... that this part of the language will be
cleaned up some day ?

Are there any tricks to get rid of it ?

Maybe "white" like in Delphi ?

I haven't written much OO code yet in Python... and don't plan on doing it
too...

Cause it looks hellish confusing... and clouded/clodded.

I think I have better things to do then to insert "self" everywhere...

It's almost like "self" masturbation  LOL.

Bye,
  Skybuck =D 



From skybuck2000 at hotmail.com  Thu Jun 11 07:22:45 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 11 Jun 2015 13:22:45 +0200
Subject: I don't like the OO part of python. In particular the "self"
 keyword everywhere.
In-Reply-To: <2c87e$55796f2c$5419aafe$47209@news.ziggo.nl>
References: <2c87e$55796f2c$5419aafe$47209@news.ziggo.nl>
Message-ID: <775a1$55796f84$5419aafe$47431@news.ziggo.nl>

Another typo corrected... see (*****)

"Skybuck Flying"  wrote in message 
news:2c87e$55796f2c$5419aafe$47209 at news.ziggo.nl...

Little typo corrected... it's a common typo I seem to make.

"with" had to be "without" see ***.

"Skybuck Flying"  wrote in message news:...

Hello,

I don't like the object orientated part of Python.

The idea/prospect of having to write "self" everywhere... seems very
horrorific and a huge time waster.

(Perhaps the module thing of python might help in future not sure about
that).

What are your thoughts on the "self" thing/requirement.

I only want replies from expert programmers, cause we need a language for
expert programmers...

Not noobies that need to be hand-held...

Personally I think I could do just fine without (***) the "self" keyword
everywhere.

So question is... can the python interpreter/compiler be written in such a
way that self can be left out ?

In other words: Is there any hope... that this part of the language will be
cleaned up some day ?

Are there any tricks to get rid of it ?

Maybe "with" (*****) like in Delphi ?

I haven't written much OO code yet in Python... and don't plan on doing it
too...

Cause it looks hellish confusing... and clouded/clodded.

I think I have better things to do then to insert "self" everywhere...

It's almost like "self" masturbation  LOL.

Bye,
  Skybuck =D



From skybuck2000 at hotmail.com  Thu Jun 11 07:27:18 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 11 Jun 2015 13:27:18 +0200
Subject: I don't like the OO part of python. In particular the "self"
 keyword everywhere.
In-Reply-To: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
References: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
Message-ID: <77496$55797095$5419aafe$48148@news.ziggo.nl>

Then again...

I also believe the highest goal for a programming language is "natural 
spoken" language.

If "self.somefield" equals 10 then...

Does have some understandable ring to it.

However... time constraints also have to be kept in mind.

In another words if the code looks like

begin of class section

if somefield equals 10 then...


end of class section

Should be pretty obvious that somefield belongs to class section...

So no need to specify self...

If I wanted to access a global variable I would use the existing "global" 
thing

global SomeField...

maybe if I wanted to use a local variable for routine:

local SomeField...

seems nicer... then having to use self everywhere...

Bye,
  Skybuck. 



From __peter__ at web.de  Thu Jun 11 07:28:37 2015
From: __peter__ at web.de (Peter Otten)
Date: Thu, 11 Jun 2015 13:28:37 +0200
Subject: A basic dictionary question
References: <41302A7145AC054FA7A96CFD03835A0A0B980FE0@EX10MBX02.EU.NEC.COM>
Message-ID: <mlbrd6$564$1@ger.gmane.org>

David Aldrich wrote:

> Hi
> 
> I am fairly new to Python.  I am writing some code that uses a dictionary
> to store definitions of hardware registers.  Here is a small part of it:
> 
> import sys
> 
> register = {
>     'address'  : 0x3001c,
>     'fields' : {
>         'FieldA' : {
>             'range' : (31,20),
>         },
>         'FieldB' : {
>             'range' : (19,16),
>         },
>     },
>     'width' : 32
> };
> 
> def main():
>     fields = register['fields']
>     for field, range_dir in fields:         <== This line fails
>         range_dir = field['range']
>         x,y = range_dir['range']
>         print(x, y)
> 
> if __name__ == '__main__':
>     main()
> 
> I want the code to print the range of bits of each field defined in the
> dictionary.
> 
> The output is:
> 
> Traceback (most recent call last):
>   File "testdir.py", line 32, in <module>
>     main()
>   File "testdir.py", line 26, in main
>     for field, range_dir in fields:
> ValueError: too many values to unpack (expected 2)
> 
> Please will someone explain what I am doing wrong?

for key in some_dict:
    ...

iterates over the keys of the dictionary, for (key, value) pairs you need

for key, value in some_dict.items():
    ...

 
> Also I would like to ask how I could print the ranges in the order they
> are defined.  Should I use a different dictionary class or could I add a
> field to the dictionary/list to achieve this?

Have a look at collections.OrderedDict:

https://docs.python.org/dev/library/collections.html#collections.OrderedDict

If you don't care about fast access by key you can also use a list of 
(key, value) pairs.





From python at mrabarnett.plus.com  Thu Jun 11 07:29:25 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 11 Jun 2015 12:29:25 +0100
Subject: A basic dictionary question
In-Reply-To: <41302A7145AC054FA7A96CFD03835A0A0B980FE0@EX10MBX02.EU.NEC.COM>
References: <41302A7145AC054FA7A96CFD03835A0A0B980FE0@EX10MBX02.EU.NEC.COM>
Message-ID: <55797115.4050001@mrabarnett.plus.com>

On 2015-06-11 11:10, David Aldrich wrote:
> Hi
>
> I am fairly new to Python.  I am writing some code that uses a
> dictionary to store definitions of hardware registers.  Here is a small
> part of it:
>
> import sys
>
> register = {
>
>      'address'  : 0x3001c,
>
>      'fields' : {
>
>          'FieldA' : {
>
>              'range' : (31,20),
>
>          },
>
>          'FieldB' : {
>
>              'range' : (19,16),
>
>          },
>
>      },
>
>      'width' : 32
>
> };
>
> def main():
>
>      fields = register['fields']
>
>      for field, range_dir in fields:         <== This line fails
>
>          range_dir = field['range']
>
>          x,y = range_dir['range']
>
>          print(x, y)
>
> if __name__ == '__main__':
>
>      main()
>
> I want the code to print the range of bits of each field defined in the
> dictionary.
>
> The output is:
>
> Traceback (most recent call last):
>
>    File "testdir.py", line 32, in <module>
>
>      main()
>
>    File "testdir.py", line 26, in main
>
>      for field, range_dir in fields:
>
> ValueError: too many values to unpack (expected 2)
>
> Please will someone explain what I am doing wrong?
>
You're iterating over the keys. What you want is to iterate over
"fields.items()" which gives the key/value pairs.

> Also I would like to ask how I could print the ranges in the order they
> are defined.  Should I use a different dictionary class or could I add a
> field to the dictionary/list to achieve this?
>
Dicts are unordered. Try 'OrderedDict' from the 'collections' module.



From rosuav at gmail.com  Thu Jun 11 07:36:32 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 11 Jun 2015 21:36:32 +1000
Subject: I don't like the OO part of python. In particular the "self"
 keyword everywhere.
In-Reply-To: <77496$55797095$5419aafe$48148@news.ziggo.nl>
References: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
 <77496$55797095$5419aafe$48148@news.ziggo.nl>
Message-ID: <CAPTjJmpmdFSDqS4+fKsz52ifcoE9ouc0fH8cFG-dC-wcBT564A@mail.gmail.com>

On Thu, Jun 11, 2015 at 9:27 PM, Skybuck Flying <skybuck2000 at hotmail.com> wrote:
> If I wanted to access a global variable I would use the existing "global"
> thing
>
> global SomeField...
>
> maybe if I wanted to use a local variable for routine:
>
> local SomeField...
>
> seems nicer... then having to use self everywhere...


From rosuav at gmail.com  Thu Jun 11 07:38:06 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 11 Jun 2015 21:38:06 +1000
Subject: I don't like the OO part of python. In particular the "self"
 keyword everywhere.
In-Reply-To: <77496$55797095$5419aafe$48148@news.ziggo.nl>
References: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
 <77496$55797095$5419aafe$48148@news.ziggo.nl>
Message-ID: <CAPTjJmrGmTAqAu8CKHUHnAgw50qO=9+_4PY1=bPRr2Ygj2xDUQ@mail.gmail.com>

On Thu, Jun 11, 2015 at 9:27 PM, Skybuck Flying <skybuck2000 at hotmail.com> wrote:
> If I wanted to access a global variable I would use the existing "global"
> thing
>
> global SomeField...
>
> maybe if I wanted to use a local variable for routine:
>
> local SomeField...
>
> seems nicer... then having to use self everywhere...

Oops, missent.

If you want this, how about simply declaring all function-local
variables, and having everything else be implicitly global? Or declare
(with data type) all globals, all class attributes, and all locals,
and then let the compiler figure out what you want? Because if you
want ECMAScript or C++, you know where to find them.

ChrisA


From adebayo.abraham at gmail.com  Thu Jun 11 08:03:45 2015
From: adebayo.abraham at gmail.com (Adebayo Abraham)
Date: Thu, 11 Jun 2015 05:03:45 -0700 (PDT)
Subject: I just need this question explained.
Message-ID: <135b6aa8-0761-4cda-b385-10b51d58465f@googlegroups.com>

I am not requesting for a solution. I just need the question explained. I am a beginner python developer and i do not know where to start from to solve this problem. So anybody, somebody: please explain this question. Am i to create a testcase or create the code to display a value?

Challenge: FitFam

Problem

The aerobics class begins. The trainer says, "Please position yourselves on the training mat so that each one of you has enough space to move your arms around freely, and not hit anybody else." People start milling around on the mat, trying to position themselves properly. Minutes pass, and finally the trainer is so annoyed that he asks you to write a program that will position all the people correctly, hoping it will be quicker than letting them figure it out for themselves!

You are given the dimensions (width and length) of the mat on which the class takes place. For every student, there is a circular area she has to have for herself, with radius equal to the reach of her arms. These circles can not intersect, though they can touch; and the center of each circle (where the student stands) has to be on the mat. Note that the arms can reach outside the mat. You know that there's plenty of space on the mat ? the area of the mat is at least five times larger than the total area of the circles required by all the people in the class. It will always be possible for all the people to position themselves as required.

Input

The first line of the input gives the number of test cases, T. T test cases follow. Each test case consists of two lines. The first line contains three integers: N, W and L, denoting the number of students, the width of the mat, and the length of the mat, respectively. The second line contains N integers ri, denoting the reach of the arms of the ith student.

Output

For each test case, output one line containing "Case #n: y", where n is the case number (starting from 1) and y is a string containing 2N numbers, each of which can be an integer or a real number: x1, y1, x2, y2, etc., where the pair (xi, yi) is the position where the ith student should stand (with 0 ? xi ? W and 0 ? yi ? L).

As there will likely be multiple ways to position the students on the mat, you may output any correct positioning; but remember that you may not submit an output file more than 200kB in size.

Limits

1 ? T ? 50.
1 ? W, L ? 109.
1 ? ri ? 105.
The area of the mat is at least 5 times larger than the total area of the circles:
5*?*(r12 + ... + rN2) ? W*L.
Small dataset

1 ? N ? 10.
Large dataset

1 ? N ? 103.
The total number of circles in all test cases will be ? 6000.
Sample


Input

Output

2
2 6 6
1 1
3 320 2
4 3 2
Case #1: 0.0 0.0 6.0 6.0
Case #2: 0.0 0.0 7.0 0.0 12.0 0.0



From python at mrabarnett.plus.com  Thu Jun 11 08:16:57 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 11 Jun 2015 13:16:57 +0100
Subject: I don't like the OO part of python. In particular the "self"
 keyword everywhere.
In-Reply-To: <77496$55797095$5419aafe$48148@news.ziggo.nl>
References: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
 <77496$55797095$5419aafe$48148@news.ziggo.nl>
Message-ID: <55797C39.4000309@mrabarnett.plus.com>

On 2015-06-11 12:27, Skybuck Flying wrote:
> Then again...
>
> I also believe the highest goal for a programming language is "natural
> spoken" language.
>
Natural language is full of ambiguities.

> If "self.somefield" equals 10 then...
>
> Does have some understandable ring to it.
>
> However... time constraints also have to be kept in mind.
>
> In another words if the code looks like
>
> begin of class section
>
Shouldn't that be "beginning of class section"?

> if somefield equals 10 then...
>
>
> end of class section
>
You should have a look at Cobol. If was designed with a more natural-
looking syntax so that business managers could write their own code.

It turned out that the managers didn't write code because programming
harder then they anticipated.

Then there's AppleScript. It also tries to have a more natural-looking
syntax, but the problem is that it's then not so clear what's legal.
For example, it allows "title of window" or "window's title". So what
is the title of the script's title? It's "title of window of me" or
"me's window's title". Yes, "me's", not "my". It's an example of the
"Uncanny Valley".

I prefer a language that doesn't look like a natural language, because
it isn't, that's not its purpose.

> Should be pretty obvious that somefield belongs to class section...
>
> So no need to specify self...
>
> If I wanted to access a global variable I would use the existing "global"
> thing
>
> global SomeField...
>
> maybe if I wanted to use a local variable for routine:
>
> local SomeField...
>
> seems nicer... then having to use self everywhere...
>
"then"? Should be "than"... (That seems to be happening more and more
these days...)

Anyway, the use of "self" is something that's not going to change.

If you don't like it, there are plenty of other programming languages
out there for you to try.



From dpalao.python at gmail.com  Thu Jun 11 08:19:11 2015
From: dpalao.python at gmail.com (David Palao)
Date: Thu, 11 Jun 2015 14:19:11 +0200
Subject: I just need this question explained.
In-Reply-To: <135b6aa8-0761-4cda-b385-10b51d58465f@googlegroups.com>
References: <135b6aa8-0761-4cda-b385-10b51d58465f@googlegroups.com>
Message-ID: <CAKUKWzmkjgPt+ko5Hm3vBAhtQS0OBAr1_3nyZ6JCcyqfiba09w@mail.gmail.com>

2015-06-11 14:03 GMT+02:00 Adebayo Abraham <adebayo.abraham at gmail.com>:
> I am not requesting for a solution. I just need the question explained. I am a beginner python developer and i do not know where to start from to solve this problem. So anybody, somebody: please explain this question. Am i to create a testcase or create the code to display a value?
>
> Challenge: FitFam
>
> Problem
>
> The aerobics class begins. The trainer says, "Please position yourselves on the training mat so that each one of you has enough space to move your arms around freely, and not hit anybody else." People start milling around on the mat, trying to position themselves properly. Minutes pass, and finally the trainer is so annoyed that he asks you to write a program that will position all the people correctly, hoping it will be quicker than letting them figure it out for themselves!
>
> You are given the dimensions (width and length) of the mat on which the class takes place. For every student, there is a circular area she has to have for herself, with radius equal to the reach of her arms. These circles can not intersect, though they can touch; and the center of each circle (where the student stands) has to be on the mat. Note that the arms can reach outside the mat. You know that there's plenty of space on the mat ? the area of the mat is at least five times larger than the total area of the circles required by all the people in the class. It will always be possible for all the people to position themselves as required.
>
> Input
>
> The first line of the input gives the number of test cases, T. T test cases follow. Each test case consists of two lines. The first line contains three integers: N, W and L, denoting the number of students, the width of the mat, and the length of the mat, respectively. The second line contains N integers ri, denoting the reach of the arms of the ith student.
>
> Output
>
> For each test case, output one line containing "Case #n: y", where n is the case number (starting from 1) and y is a string containing 2N numbers, each of which can be an integer or a real number: x1, y1, x2, y2, etc., where the pair (xi, yi) is the position where the ith student should stand (with 0 ? xi ? W and 0 ? yi ? L).
>
> As there will likely be multiple ways to position the students on the mat, you may output any correct positioning; but remember that you may not submit an output file more than 200kB in size.
>
> Limits
>
> 1 ? T ? 50.
> 1 ? W, L ? 109.
> 1 ? ri ? 105.
> The area of the mat is at least 5 times larger than the total area of the circles:
> 5*?*(r12 + ... + rN2) ? W*L.
> Small dataset
>
> 1 ? N ? 10.
> Large dataset
>
> 1 ? N ? 103.
> The total number of circles in all test cases will be ? 6000.
> Sample
>
>
> Input
>
> Output
>
> 2
> 2 6 6
> 1 1
> 3 320 2
> 4 3 2
> Case #1: 0.0 0.0 6.0 6.0
> Case #2: 0.0 0.0 7.0 0.0 12.0 0.0
>
> --
> https://mail.python.org/mailman/listinfo/python-list

Hello again,
It looks to me that you have to write the code that is able to read
the input described and produce the wished output.
You could organize the program like
1) read input
2) compute
3) print out result
Perhaps each part being a function.

Try something and tell us when you are blocked.

Best


From python at mrabarnett.plus.com  Thu Jun 11 08:20:25 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 11 Jun 2015 13:20:25 +0100
Subject: I just need this question explained.
In-Reply-To: <135b6aa8-0761-4cda-b385-10b51d58465f@googlegroups.com>
References: <135b6aa8-0761-4cda-b385-10b51d58465f@googlegroups.com>
Message-ID: <55797D09.6070408@mrabarnett.plus.com>

On 2015-06-11 13:03, Adebayo Abraham wrote:
> I am not requesting for a solution. I just need the question
> explained. I am a beginner python developer and i do not know where
> to start from to solve this problem. So anybody, somebody: please
> explain this question. Am i to create a testcase or create the code
> to display a value?
>
[snip]

You should create a program that will accept test cases and print the
results for each.


From joel.goldstick at gmail.com  Thu Jun 11 08:35:37 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Thu, 11 Jun 2015 08:35:37 -0400
Subject: Parser needed.
In-Reply-To: <efad0$55796dd4$5419aafe$46357@news.ziggo.nl>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
 <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
 <4252d$55709249$5419aafe$25285@news.ziggo.nl>
 <74396$55764fb5$5419aafe$6142@news.ziggo.nl>
 <mailman.310.1433842760.13271.python-list@python.org>
 <48a90$5576da1e$5419aafe$34050@news.ziggo.nl>
 <mailman.327.1433892811.13271.python-list@python.org>
 <efad0$55796dd4$5419aafe$46357@news.ziggo.nl>
Message-ID: <CAPM-O+wF=opshvFMBguLzkwpp_-Z1O1c=PjkrgGxLgdZNjmBBA@mail.gmail.com>

On Thu, Jun 11, 2015 at 7:15 AM, Skybuck Flying <skybuck2000 at hotmail.com> wrote:
> Well it did help a little bit.
>
> Somebody asked if there was already a parser for it.
>
> I answered yes in C#.
>
> So I took a closer look at it... and learned something from it.
>
> Maybe I would have done that anyway... or maybe not...
>
> Now we will never know... but I am happy that the parser is now ok, done and
> pretty easy extendable.
>
> I think it's probably my first really good one.
>
> I did do a little assembler before... but not sure if it was any good.
>
> I also read before I started the thread... what "parse" means in dictionary.
>
> That also helped me understand it a little bit better.
>
> And also I read how it's usually done... tokens/tokenize etc... kinda
> already knew that... but still.
>
> Turned out to be quite easy...
>
> But at the start /always it seemed so difficult...
>
> So any little bits of help/advice/information can help ! :)
>
>
> Bye,
>  Skybuck.
> --
> https://mail.python.org/mailman/listinfo/python-list


but you aren't asking questions.  You are having a conversation with
yourself on a public q/a list.  Its unpleasant
-- 
Joel Goldstick
http://joelgoldstick.com


From larry.martell at gmail.com  Thu Jun 11 08:38:01 2015
From: larry.martell at gmail.com (Larry Martell)
Date: Thu, 11 Jun 2015 08:38:01 -0400
Subject: Parser needed.
In-Reply-To: <CAPM-O+wF=opshvFMBguLzkwpp_-Z1O1c=PjkrgGxLgdZNjmBBA@mail.gmail.com>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
 <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
 <4252d$55709249$5419aafe$25285@news.ziggo.nl>
 <74396$55764fb5$5419aafe$6142@news.ziggo.nl>
 <mailman.310.1433842760.13271.python-list@python.org>
 <48a90$5576da1e$5419aafe$34050@news.ziggo.nl>
 <mailman.327.1433892811.13271.python-list@python.org>
 <efad0$55796dd4$5419aafe$46357@news.ziggo.nl>
 <CAPM-O+wF=opshvFMBguLzkwpp_-Z1O1c=PjkrgGxLgdZNjmBBA@mail.gmail.com>
Message-ID: <CACwCsY6MqRkShswEtB5E82_JB3hBggrEO-xwQ59ySRs-e0CeWA@mail.gmail.com>

On Thu, Jun 11, 2015 at 8:35 AM, Joel Goldstick
<joel.goldstick at gmail.com> wrote:
> but you aren't asking questions.  You are having a conversation with
> yourself on a public q/a list.  Its unpleasant

Well, he did mention masterbation in another post.


From joenbadello at gmx.com  Thu Jun 11 09:35:13 2015
From: joenbadello at gmx.com (Redacted)
Date: Thu, 11 Jun 2015 06:35:13 -0700 (PDT)
Subject: Redacted
Message-ID: <1ca87402-f6d4-4b9f-a291-dcc1858e2c45@googlegroups.com>

Redacted



From joenbadello at gmx.com  Thu Jun 11 09:36:19 2015
From: joenbadello at gmx.com (Redacted)
Date: Thu, 11 Jun 2015 06:36:19 -0700 (PDT)
Subject: Redacted
Message-ID: <8afc23c9-dba8-4f05-a1a3-bc3e46307e3e@googlegroups.com>

Redacted


From mewthree19 at hotmail.com  Thu Jun 11 09:53:55 2015
From: mewthree19 at hotmail.com (mewthree19 at hotmail.com)
Date: Thu, 11 Jun 2015 06:53:55 -0700 (PDT)
Subject: Writting Dialog to enter preset items from a combo list
Message-ID: <95b7c4d2-6afd-4694-b3c4-6e4f856b4617@googlegroups.com>

ok that subject is complex I known I am fairly new to python programming and I am using python 3.4.3 and the gui editor/creator boa Constructor and and another one what I can't think of as I type this will add later on as am typing this of public system and not the computer I do most of my programming on

I have wxPheonix installed for my python version.

I have it layed out like this 

COMBOLIST1 TEXTBOX1 COMBOLIST2 TEXTBOX2 COMBOLIST3
TEXTBOX3
COMBOLIST4 TEXTBOX4

after this I would like to put a button to add more textbox and combolist if needed

TEXTBOX5
COMBOLIST5

and then once its done it gets all the information from the textbox and combolist and prints it to a RFT/TXT section on the main form of my program

is this possible to do if so how 

if it is not please can you help me work out how to do something that would work



From kwpolska at gmail.com  Thu Jun 11 10:03:33 2015
From: kwpolska at gmail.com (Chris Warrick)
Date: Thu, 11 Jun 2015 16:03:33 +0200
Subject: New Python student needs help with execution
In-Reply-To: <201506110852.t5B8qo0m000584@fido.openend.se>
References: <azrocknroll@gmail.com>
 <77ccd04d-60fe-495c-877f-871a50c89cd2@googlegroups.com>
 <201506110852.t5B8qo0m000584@fido.openend.se>
Message-ID: <CAMw+j7JNksWJd99S3hqLyzP-xPA+FPKf1V3hs_yP-2aXj2Ji8Q@mail.gmail.com>

On Thu, Jun 11, 2015 at 10:52 AM, Laura Creighton <lac at openend.se> wrote:
> In a message of Wed, 10 Jun 2015 21:50:54 -0700, c me writes:
>>I installed 2.7.9 on a Win8.1 machine. The Coursera instructor did a simple install then executed Python from a file in which he'd put a simple hello world script.  My similar documents folder cannot see the python executable.  How do I make this work?
>>--
>>https://mail.python.org/mailman/listinfo/python-list
>
> You need to set your PYTHONPATH.
> Instructions here should help.
> https://docs.python.org/2.7/using/windows.html
>
> It is possible that you need to do some other things too, but again
> that's the doc for it.
>
> Laura
> --
> https://mail.python.org/mailman/listinfo/python-list

It?s actually %PATH%. %PYTHONPATH% does something different and is not
really useful to newcomers (especially since there are much better
ways to accomplish what it does)

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16


From torriem at gmail.com  Thu Jun 11 10:15:19 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 11 Jun 2015 08:15:19 -0600
Subject: I don't like the OO part of python. In particular the "self"
 keyword everywhere.
In-Reply-To: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
References: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
Message-ID: <557997F7.6070101@gmail.com>

On 06/11/2015 05:19 AM, Skybuck Flying wrote:
> I haven't written much OO code yet in Python... and don't plan on doing it 
> too...

Except that you already have written OO code in Python with your parser.
 Or at least code that interacts heavily with OO.  Anytime you call a
method on a string like split(), or worked with regular expressions, you
are using OO.  OO permeates Python.  You don't have to use OO design in
your programs, but you will always be working with objects and calling
methods on them.

But anyway, if you don't like "self," use a different name.  Like "skybuck."

class Bar:
    def foo(skybuck, a, b):
        skybuck.a = a
        skybuck.b = b

In other languages with an implicit self or this like C, I find the
ambiguities of scope highly problematic.  Method variables can shadow
instance variables.  It's also difficult for someone unfamiliar with the
code to differentiate between a local variable and an instance variable
as the naked variable name has no qualifier.  I've seen some projects
that apply a "m_" prefix to every instance variable so you can tell them
apart.  Sure that's less typing, but it is super ugly!  Explicitly
providing self not only makes things very clear, it also helps me when
I'm scrolling through code to identify which functions definitions are
bare functions and which are methods of a class.

In short, it's part of the Python language and will not be changing
anytime soon.  So no the interpreter couldn't be written in such a way
that the self can be left out without changing the language's semantics
and definition.



From thomas at mlynarczyk-webdesign.de  Thu Jun 11 11:32:55 2015
From: thomas at mlynarczyk-webdesign.de (Thomas Mlynarczyk)
Date: Thu, 11 Jun 2015 17:32:55 +0200
Subject: I don't like the OO part of python. In particular the "self"
 keyword everywhere.
In-Reply-To: <mailman.394.1434025019.13271.python-list@python.org>
References: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
 <77496$55797095$5419aafe$48148@news.ziggo.nl>
 <mailman.394.1434025019.13271.python-list@python.org>
Message-ID: <mlc9n7$5br$1@news.albasani.net>

On 11/06/15 14:16, MRAB wrote:
> harder then they anticipated.
-----------^
>> seems nicer... then having to use self everywhere...
>>
> "then"? Should be "than"... (That seems to be happening more and more
> these days...)

Indeed :-)

-- 
Ce n'est pas parce qu'ils sont nombreux ? avoir tort qu'ils ont raison!
(Coluche)


From rustompmody at gmail.com  Thu Jun 11 11:35:57 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Thu, 11 Jun 2015 08:35:57 -0700 (PDT)
Subject: Parser needed.
In-Reply-To: <mailman.398.1434026290.13271.python-list@python.org>
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
 <7cd38$557084b9$5419aafe$9688@news.ziggo.nl>
 <74dd4$55708983$5419aafe$17465@news.ziggo.nl>
 <ebb0e$55708c3f$5419aafe$19843@news.ziggo.nl>
 <cb46a$55708d22$5419aafe$20596@news.ziggo.nl>
 <8dfe$55708e76$5419aafe$21711@news.ziggo.nl>
 <7241f$55708fd7$5419aafe$23020@news.ziggo.nl>
 <4252d$55709249$5419aafe$25285@news.ziggo.nl>
 <74396$55764fb5$5419aafe$6142@news.ziggo.nl>
 <mailman.310.1433842760.13271.python-list@python.org>
 <48a90$5576da1e$5419aafe$34050@news.ziggo.nl>
 <mailman.327.1433892811.13271.python-list@python.org>
 <efad0$55796dd4$5419aafe$46357@news.ziggo.nl>
 <CAPM-O+wF=opshvFMBguLzkwpp_-Z1O1c=PjkrgGxLgdZNjmBBA@mail.gmail.com>
 <mailman.398.1434026290.13271.python-list@python.org>
Message-ID: <b128dc4b-ac60-4ad9-bf5a-db3296327167@googlegroups.com>

On Thursday, June 11, 2015 at 6:08:22 PM UTC+5:30, Larry.... at gmail.com wrote:
> On Thu, Jun 11, 2015 at 8:35 AM, Joel Goldstick wrote:
> > but you aren't asking questions.  You are having a conversation with
> > yourself on a public q/a list.  Its unpleasant
> 
> Well, he did mention masterbation in another post.

Er...

Those of us who happen to be teachers are getting pointed at by the misspelling+archaism combo above.

Thought python had no rogue pointers? <wink>


From subhabrata.banerji at gmail.com  Thu Jun 11 11:40:50 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Thu, 11 Jun 2015 08:40:50 -0700 (PDT)
Subject: Error in or
Message-ID: <3bf5dd52-c698-4392-be2b-5c0aeefc3cd3@googlegroups.com>

Dear Group,

In the following script,

  inp1=raw_input("PRINT YOUR INPUT:")
  if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*" in inp1):
  
  if write this it is working fine, but if I write 
   
  if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*" in inp1) or (''' " ''' in   inp1):
  
 the portion of (''' " ''' in   inp1) is not working.

If any one of the esteemed members may kindly suggest the error I am making.

I am using Python2.7+ on Windows 7 Professional. 

Apology for any indentation error. 

Regards,
Subhabrata Banerjee. 
         


From lac at openend.se  Thu Jun 11 11:48:17 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 11 Jun 2015 17:48:17 +0200
Subject: New Python student needs help with execution
In-Reply-To: Message from Chris Warrick <kwpolska@gmail.com> of "Thu,
 11 Jun 2015 16:03:33 +0200."
 <CAMw+j7JNksWJd99S3hqLyzP-xPA+FPKf1V3hs_yP-2aXj2Ji8Q@mail.gmail.com>
References: <azrocknroll@gmail.com>
 <77ccd04d-60fe-495c-877f-871a50c89cd2@googlegroups.com>
 <201506110852.t5B8qo0m000584@fido.openend.se><CAMw+j7JNksWJd99S3hqLyzP-xPA+FPKf1V3hs_yP-2aXj2Ji8Q@mail.gmail.com>
Message-ID: <201506111548.t5BFmH1g010145@fido.openend.se>

In a message of Thu, 11 Jun 2015 16:03:33 +0200, Chris Warrick writes:
>On Thu, Jun 11, 2015 at 10:52 AM, Laura Creighton <lac at openend.se> wrote:
>> In a message of Wed, 10 Jun 2015 21:50:54 -0700, c me writes:
>>>I installed 2.7.9 on a Win8.1 machine. The Coursera instructor did a simple install then executed Python from a file in which he'd put a simple hello world script.  My similar documents folder cannot see the python executable.  How do I make this work?
>>>--
>>>https://mail.python.org/mailman/listinfo/python-list
>>
>> You need to set your PYTHONPATH.
>> Instructions here should help.
>> https://docs.python.org/2.7/using/windows.html
>>
>> It is possible that you need to do some other things too, but again
>> that's the doc for it.
>>
>> Laura
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
>It?s actually %PATH%. %PYTHONPATH% does something different and is not
>really useful to newcomers (especially since there are much better
>ways to accomplish what it does)
>
>-- 
>Chris Warrick <https://chriswarrick.com/>
>PGP: 5EAAEA16

Sorry about that -- what you get for not using windows.  Thank you.

Laura



From ian.g.kelly at gmail.com  Thu Jun 11 11:49:58 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 11 Jun 2015 09:49:58 -0600
Subject: Error in or
In-Reply-To: <3bf5dd52-c698-4392-be2b-5c0aeefc3cd3@googlegroups.com>
References: <3bf5dd52-c698-4392-be2b-5c0aeefc3cd3@googlegroups.com>
Message-ID: <CALwzidn6uqRJjsNY2Am1vasVoFmq3NJrX==HS045iO3br_FVEQ@mail.gmail.com>

On Thu, Jun 11, 2015 at 9:40 AM,  <subhabrata.banerji at gmail.com> wrote:
>   if write this it is working fine, but if I write
>
>   if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*" in inp1) or (''' " ''' in   inp1):
>
>  the portion of (''' " ''' in   inp1) is not working.

Not working how? I copy-pasted the line and it appears to work fine.


From rustompmody at gmail.com  Thu Jun 11 12:39:23 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Thu, 11 Jun 2015 09:39:23 -0700 (PDT)
Subject: I don't like the OO part of python. In particular the "self"
 keyword everywhere.
In-Reply-To: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
References: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
Message-ID: <11e2ec7b-73ae-4afc-98f5-e2861a7cc781@googlegroups.com>

On Thursday, June 11, 2015 at 4:49:59 PM UTC+5:30, Skybuck Flying wrote:
> Hello,
> 
> I don't like the object orientated part of Python.
> 
> The idea/prospect of having to write "self" everywhere... seems very 
> horrorific and a huge time waster.
> 
> (Perhaps the module thing of python might help in future not sure about 
> that).
> 
> What are your thoughts on the "self" thing/requirement.
> 
> I only want replies from expert programmers, cause we need a language for 
> expert programmers...
> 
> Not noobies that need to be hand-held...
> 
> Personally I think I could do just fine with the "self" keyword everywhere.
> 
> So question is... can the python interpreter/compiler be written in such a 
> way that self can be left out ?
> 
> In other words: Is there any hope... that this part of the language will be 
> cleaned up some day ?
> 
> Are there any tricks to get rid of it ?
> 
> Maybe "white" like in Delphi ?
> 
> I haven't written much OO code yet in Python... and don't plan on doing it 
> too...
> 
> Cause it looks hellish confusing... and clouded/clodded.
> 
> I think I have better things to do then to insert "self" everywhere...
> 
> It's almost like "self" masturbation  LOL.
> 
> Bye,
>   Skybuck =D

I suggest you disentangle your question/crib along 3 axes:

1. Specific grumble about a specific feature -- in this case 'explicit-self'
2. General discomfort with a style/paradigm -- OO

2 itself can be bifurcated along 2 axes
2a. Hard when learning
2b. Painful/stupid for serious/real-world usage

That 2a and 2b were quite separate was well understood in the 70s and
80s when Pascal was widely known.  Today it seems increasingly
forgotten.  eg all the OOP texts that I studied wax endlessly
on circles and ellipses and rectangles and squares and other such
wonders of God's (or OO's) creations.
Today http://en.wikipedia.org/wiki/Circle-ellipse_problem suggests its not
so neat (or useful) as the OO aficionados imagine



From subhabrata.banerji at gmail.com  Thu Jun 11 12:39:29 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Thu, 11 Jun 2015 09:39:29 -0700 (PDT)
Subject: Error in or
In-Reply-To: <mailman.402.1434037847.13271.python-list@python.org>
References: <3bf5dd52-c698-4392-be2b-5c0aeefc3cd3@googlegroups.com>
 <mailman.402.1434037847.13271.python-list@python.org>
Message-ID: <6a171772-23a5-471d-a4e5-73f195c9055c@googlegroups.com>

On Thursday, June 11, 2015 at 9:20:59 PM UTC+5:30, Ian wrote:
> On Thu, Jun 11, 2015 at 9:40 AM,  
> >   if write this it is working fine, but if I write
> >
> >   if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*" in inp1) or (''' " ''' in   inp1):
> >
> >  the portion of (''' " ''' in   inp1) is not working.
> 
> Not working how? I copy-pasted the line and it appears to work fine.

Dear Sir,
Thank you for your kind reply. Nice to know your reply, but I am trying to send you my experiment, please see my results,

>>> def input1(n):
	inp1=raw_input("PRINT YOUR QUERY:")
	if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*" in inp1) or (''' " ''' in   inp1):
		print "FINE"

		
>>> input1(1)
PRINT YOUR QUERY:"Java"
>>> input1(1)
PRINT YOUR QUERY:"Obama in London"
>>> input1(1)
PRINT YOUR QUERY:Obama AND Bush
FINE
>>> input1(1)
PRINT YOUR QUERY:Obama OR Bush
FINE
>>> 

you may get better my problem. 

Regards,
Subhabrata Banerjee. 


From ian.g.kelly at gmail.com  Thu Jun 11 12:47:30 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 11 Jun 2015 10:47:30 -0600
Subject: Error in or
In-Reply-To: <6a171772-23a5-471d-a4e5-73f195c9055c@googlegroups.com>
References: <3bf5dd52-c698-4392-be2b-5c0aeefc3cd3@googlegroups.com>
 <mailman.402.1434037847.13271.python-list@python.org>
 <6a171772-23a5-471d-a4e5-73f195c9055c@googlegroups.com>
Message-ID: <CALwzidnt4yMEc3hac-fztLR_+b2=bvA1PZSw-nUpCaAUDSjF_w@mail.gmail.com>

On Thu, Jun 11, 2015 at 10:39 AM,  <subhabrata.banerji at gmail.com> wrote:
> On Thursday, June 11, 2015 at 9:20:59 PM UTC+5:30, Ian wrote:
>> On Thu, Jun 11, 2015 at 9:40 AM,
>> >   if write this it is working fine, but if I write
>> >
>> >   if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*" in inp1) or (''' " ''' in   inp1):
>> >
>> >  the portion of (''' " ''' in   inp1) is not working.
>>
>> Not working how? I copy-pasted the line and it appears to work fine.
>
> Dear Sir,
> Thank you for your kind reply. Nice to know your reply, but I am trying to send you my experiment, please see my results,
>
>>>> def input1(n):
>         inp1=raw_input("PRINT YOUR QUERY:")
>         if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*" in inp1) or (''' " ''' in   inp1):
>                 print "FINE"
>
>
>>>> input1(1)
> PRINT YOUR QUERY:"Java"
>>>> input1(1)
> PRINT YOUR QUERY:"Obama in London"
>>>> input1(1)
> PRINT YOUR QUERY:Obama AND Bush
> FINE
>>>> input1(1)
> PRINT YOUR QUERY:Obama OR Bush
> FINE
>>>>
>
> you may get better my problem.

The substring that you're looking for has spaces around the " symbol.
The example inputs that you gave don't have spaces around the "
symbols, so they don't contain the substring.  The triple quotes are
also unnecessary, though harmless -- it's not a multiline string, and
there are no ' symbols to escape in the string. Try replacing the
substring with just this: '"'.


From sohcahtoa82 at gmail.com  Thu Jun 11 12:55:32 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Thu, 11 Jun 2015 09:55:32 -0700 (PDT)
Subject: I don't like the OO part of python. In particular the "self"
 keyword everywhere.
In-Reply-To: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
References: <a91ad$55796ed2$5419aafe$46975@news.ziggo.nl>
Message-ID: <71ff3f39-9786-47b4-a7ad-5256078c28d9@googlegroups.com>

On Thursday, June 11, 2015 at 4:19:59 AM UTC-7, Skybuck Flying wrote:
> Hello,
> 
> I don't like the object orientated part of Python.
> 
> The idea/prospect of having to write "self" everywhere... seems very 
> horrorific and a huge time waster.
> 
> (Perhaps the module thing of python might help in future not sure about 
> that).
> 
> What are your thoughts on the "self" thing/requirement.
> 
> I only want replies from expert programmers, cause we need a language for 
> expert programmers...
> 
> Not noobies that need to be hand-held...
> 
> Personally I think I could do just fine with the "self" keyword everywhere.
> 
> So question is... can the python interpreter/compiler be written in such a 
> way that self can be left out ?
> 
> In other words: Is there any hope... that this part of the language will be 
> cleaned up some day ?
> 
> Are there any tricks to get rid of it ?
> 
> Maybe "white" like in Delphi ?
> 
> I haven't written much OO code yet in Python... and don't plan on doing it 
> too...
> 
> Cause it looks hellish confusing... and clouded/clodded.
> 
> I think I have better things to do then to insert "self" everywhere...
> 
> It's almost like "self" masturbation  LOL.
> 
> Bye,
>   Skybuck =D

We just had a huge thread about the "self" thing less than two weeks ago.


From subhabrata.banerji at gmail.com  Thu Jun 11 13:21:18 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Thu, 11 Jun 2015 10:21:18 -0700 (PDT)
Subject: Error in or
In-Reply-To: <mailman.403.1434041298.13271.python-list@python.org>
References: <3bf5dd52-c698-4392-be2b-5c0aeefc3cd3@googlegroups.com>
 <mailman.402.1434037847.13271.python-list@python.org>
 <6a171772-23a5-471d-a4e5-73f195c9055c@googlegroups.com>
 <mailman.403.1434041298.13271.python-list@python.org>
Message-ID: <6d09e002-974d-4823-b8bc-9a8cce12d4d6@googlegroups.com>

On Thursday, June 11, 2015 at 10:18:33 PM UTC+5:30, Ian wrote:
> On Thu, Jun 11, 2015 at 10:39 AM,   wrote:
> > On Thursday, June 11, 2015 at 9:20:59 PM UTC+5:30, Ian wrote:
> >> On Thu, Jun 11, 2015 at 9:40 AM,
> >> >   if write this it is working fine, but if I write
> >> >
> >> >   if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*" in inp1) or (''' " ''' in   inp1):
> >> >
> >> >  the portion of (''' " ''' in   inp1) is not working.
> >>
> >> Not working how? I copy-pasted the line and it appears to work fine.
> >
> > Dear Sir,
> > Thank you for your kind reply. Nice to know your reply, but I am trying to send you my experiment, please see my results,
> >
> >>>> def input1(n):
> >         inp1=raw_input("PRINT YOUR QUERY:")
> >         if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*" in inp1) or (''' " ''' in   inp1):
> >                 print "FINE"
> >
> >
> >>>> input1(1)
> > PRINT YOUR QUERY:"Java"
> >>>> input1(1)
> > PRINT YOUR QUERY:"Obama in London"
> >>>> input1(1)
> > PRINT YOUR QUERY:Obama AND Bush
> > FINE
> >>>> input1(1)
> > PRINT YOUR QUERY:Obama OR Bush
> > FINE
> >>>>
> >
> > you may get better my problem.
> 
> The substring that you're looking for has spaces around the " symbol.
> The example inputs that you gave don't have spaces around the "
> symbols, so they don't contain the substring.  The triple quotes are
> also unnecessary, though harmless -- it's not a multiline string, and
> there are no ' symbols to escape in the string. Try replacing the
> substring with just this: '"'.

Dear Sir,
Thank you. It seems working.
Regards,
Subhabrata Banerjee. 


From breamoreboy at yahoo.co.uk  Thu Jun 11 13:54:55 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 11 Jun 2015 18:54:55 +0100
Subject: Error in or
In-Reply-To: <6a171772-23a5-471d-a4e5-73f195c9055c@googlegroups.com>
References: <3bf5dd52-c698-4392-be2b-5c0aeefc3cd3@googlegroups.com>
 <mailman.402.1434037847.13271.python-list@python.org>
 <6a171772-23a5-471d-a4e5-73f195c9055c@googlegroups.com>
Message-ID: <mlci1p$q7q$1@ger.gmane.org>

On 11/06/2015 17:39, subhabrata.banerji at gmail.com wrote:
> On Thursday, June 11, 2015 at 9:20:59 PM UTC+5:30, Ian wrote:
>> On Thu, Jun 11, 2015 at 9:40 AM,
>>>    if write this it is working fine, but if I write
>>>
>>>    if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*" in inp1) or (''' " ''' in   inp1):
>>>
>>>   the portion of (''' " ''' in   inp1) is not working.
>>
>> Not working how? I copy-pasted the line and it appears to work fine.
>
> Dear Sir,
> Thank you for your kind reply. Nice to know your reply, but I am trying to send you my experiment, please see my results,
>
>>>> def input1(n):
> 	inp1=raw_input("PRINT YOUR QUERY:")
> 	if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*" in inp1) or (''' " ''' in   inp1):
> 		print "FINE"
>
> 		
>>>> input1(1)
> PRINT YOUR QUERY:"Java"
>>>> input1(1)
> PRINT YOUR QUERY:"Obama in London"
>>>> input1(1)
> PRINT YOUR QUERY:Obama AND Bush
> FINE
>>>> input1(1)
> PRINT YOUR QUERY:Obama OR Bush
> FINE
>>>>
>
> you may get better my problem.
>
> Regards,
> Subhabrata Banerjee.
>

In [6]: input1(1)
PRINT YOUR QUERY:Bush ANDES Obama
FINE

In [8]: input1(1)
PRINT YOUR QUERY:Bush ORDER Obama
FINE

Do you actually want this?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From minscheung at googlemail.com  Thu Jun 11 14:32:53 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Thu, 11 Jun 2015 11:32:53 -0700 (PDT)
Subject: How to pretty mathematical formulas in Python? Similar to Mathematica
 formats.
Message-ID: <b881edd8-55e7-4eea-88fd-7b3a094a295c@googlegroups.com>

How to pretty mathematical formulas in Python? Similar to Mathematica formats.

Are there good packages to prettify mathematica formulas in Python?


From random832 at fastmail.us  Thu Jun 11 14:54:47 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 11 Jun 2015 14:54:47 -0400
Subject: How to pretty mathematical formulas in Python? Similar to
 Mathematica formats.
In-Reply-To: <b881edd8-55e7-4eea-88fd-7b3a094a295c@googlegroups.com>
References: <b881edd8-55e7-4eea-88fd-7b3a094a295c@googlegroups.com>
Message-ID: <1434048887.945315.293214537.69EE7894@webmail.messagingengine.com>

On Thu, Jun 11, 2015, at 14:32, Sebastian M Cheung via Python-list
wrote:
> How to pretty mathematical formulas in Python? Similar to Mathematica
> formats.
> 
> Are there good packages to prettify mathematica formulas in Python?

>From a google search, apparently matplotlib has a module called mathtext
which uses a subset of LaTeX.

http://matplotlib.org/api/mathtext_api.html

Of course, you've got to put your formula in LaTeX format... which is
the same format used on Stack Exchange and Wikipedia, so you should be
able to find plenty of resources to learn about it.


From random832 at fastmail.us  Thu Jun 11 14:57:22 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 11 Jun 2015 14:57:22 -0400
Subject: How to pretty mathematical formulas in Python? Similar to
 Mathematica formats.
In-Reply-To: <1434048887.945315.293214537.69EE7894@webmail.messagingengine.com>
References: <b881edd8-55e7-4eea-88fd-7b3a094a295c@googlegroups.com>
 <1434048887.945315.293214537.69EE7894@webmail.messagingengine.com>
Message-ID: <1434049042.946107.293221161.4F610DA1@webmail.messagingengine.com>

On Thu, Jun 11, 2015, at 14:54, random832 at fastmail.us wrote:
> On Thu, Jun 11, 2015, at 14:32, Sebastian M Cheung via Python-list
> wrote:
> > How to pretty mathematical formulas in Python? Similar to Mathematica
> > formats.
> > 
> > Are there good packages to prettify mathematica formulas in Python?
> 
> From a google search, apparently matplotlib has a module called mathtext
> which uses a subset of LaTeX.
> 
> http://matplotlib.org/api/mathtext_api.html
> 
> Of course, you've got to put your formula in LaTeX format... which is
> the same format used on Stack Exchange and Wikipedia, so you should be
> able to find plenty of resources to learn about it.

I found another page on matplotlib's site with examples:
http://matplotlib.org/users/mathtext.html


From baloand at gmail.com  Thu Jun 11 14:57:41 2015
From: baloand at gmail.com (Andreas Balogh)
Date: Thu, 11 Jun 2015 20:57:41 +0200
Subject: Python 3.5.0b2 - permission error after pip upgrade
Message-ID: <mlcll0$o3b$1@dont-email.me>

I've installed a fresh copy of Python 3.5.0b2 and - as recommended - upgraded pip. I don't
understand the reason for the permission errors as I am owner and have full control for
the temporary directory created.

What can I do to fix this?


C:\Apps\Python35>pip install --upgrade pip
You are using pip version 7.0.1, however version 7.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pip
  Downloading pip-7.0.3-py2.py3-none-any.whl (1.1MB)
    100% |################################| 1.1MB 422kB/s
Installing collected packages: pip
  Found existing installation: pip 7.0.1
    Uninstalling pip-7.0.1:
      Successfully uninstalled pip-7.0.1
Exception:
Traceback (most recent call last):
  File "C:\Apps\Python35\lib\shutil.py", line 377, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied:
'C:\\Users\\Andreas\\AppData\\Local\\Temp\\pip-xf9kwi7g-uninstall\\apps\\python35\\scripts\\pip.exe'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Apps\Python35\lib\site-packages\pip\basecommand.py", line 223, in main
    status = self.run(options, args)
  File "C:\Apps\Python35\lib\site-packages\pip\commands\install.py", line 297, in run
    root=options.root_path,
  File "C:\Apps\Python35\lib\site-packages\pip\req\req_set.py", line 633, in install
    requirement.commit_uninstall()
  File "C:\Apps\Python35\lib\site-packages\pip\req\req_install.py", line 734, in
commit_uninstall
    self.uninstalled.commit()
  File "C:\Apps\Python35\lib\site-packages\pip\req\req_uninstall.py", line 153, in commit
    rmtree(self.save_dir)
  File "C:\Apps\Python35\lib\site-packages\pip\_vendor\retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "C:\Apps\Python35\lib\site-packages\pip\_vendor\retrying.py", line 212, in call
    raise attempt.get()
  File "C:\Apps\Python35\lib\site-packages\pip\_vendor\retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "C:\Apps\Python35\lib\site-packages\pip\_vendor\six.py", line 659, in reraise
    raise value
  File "C:\Apps\Python35\lib\site-packages\pip\_vendor\retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "C:\Apps\Python35\lib\site-packages\pip\utils\__init__.py", line 89, in rmtree
    onerror=rmtree_errorhandler)
  File "C:\Apps\Python35\lib\shutil.py", line 484, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Apps\Python35\lib\shutil.py", line 374, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Apps\Python35\lib\shutil.py", line 374, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Apps\Python35\lib\shutil.py", line 374, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Apps\Python35\lib\shutil.py", line 379, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Apps\Python35\lib\site-packages\pip\utils\__init__.py", line 101, in
rmtree_errorhandler
    func(path)
PermissionError: [WinError 5] Access is denied:
'C:\\Users\\Andreas\\AppData\\Local\\Temp\\pip-xf9kwi7g-uninstall\\apps\\python35\\scripts\\pip.exe'


-- 
baloand (at) gmail.com


From rosuav at gmail.com  Thu Jun 11 15:15:49 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 12 Jun 2015 05:15:49 +1000
Subject: Python 3.5.0b2 - permission error after pip upgrade
In-Reply-To: <mlcll0$o3b$1@dont-email.me>
References: <mlcll0$o3b$1@dont-email.me>
Message-ID: <CAPTjJmp0up1Db1RTi3z2TfWrZjZ-0pLiTNJMzG4Mf5K0ECp0Uw@mail.gmail.com>

On Fri, Jun 12, 2015 at 4:57 AM, Andreas Balogh <baloand at gmail.com> wrote:
> I've installed a fresh copy of Python 3.5.0b2 and - as recommended - upgraded pip. I don't
> understand the reason for the permission errors as I am owner and have full control for
> the temporary directory created.
>
> What can I do to fix this?
>
>
> C:\Apps\Python35>pip install --upgrade pip
>   File "C:\Apps\Python35\lib\shutil.py", line 377, in _rmtree_unsafe
>     os.unlink(fullname)
> PermissionError: [WinError 5] Access is denied:
> 'C:\\Users\\Andreas\\AppData\\Local\\Temp\\pip-xf9kwi7g-uninstall\\apps\\python35\\scripts\\pip.exe'

This is a problem with the way Windows locks running executables,
conflicting with the way pip wants to delete (unlink) the executable.
It appears to be a known issue by the pip people:

https://github.com/pypa/pip/issues/2773

One workaround is to skip the short-hand "pip" when upgrading pip
itself, and type out the command in full:

python -m pip install --upgrade pip

Just make sure you run the same Python (so you may need to run
"python3 -m pip etc", not sure) so it upgrades the right pip.

ChrisA


From zachary.ware+pylist at gmail.com  Thu Jun 11 15:41:05 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Thu, 11 Jun 2015 14:41:05 -0500
Subject: Python 3.5.0b2 - permission error after pip upgrade
In-Reply-To: <CAPTjJmp0up1Db1RTi3z2TfWrZjZ-0pLiTNJMzG4Mf5K0ECp0Uw@mail.gmail.com>
References: <mlcll0$o3b$1@dont-email.me>
 <CAPTjJmp0up1Db1RTi3z2TfWrZjZ-0pLiTNJMzG4Mf5K0ECp0Uw@mail.gmail.com>
Message-ID: <CAKJDb-NsFPuAiXrDOyhh+JdnQB+do5rFwXWjqQ-+EJd7iNo7Kw@mail.gmail.com>

On Thu, Jun 11, 2015 at 2:15 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Fri, Jun 12, 2015 at 4:57 AM, Andreas Balogh <baloand at gmail.com> wrote:
>> I've installed a fresh copy of Python 3.5.0b2 and - as recommended - upgraded pip. I don't
>> understand the reason for the permission errors as I am owner and have full control for
>> the temporary directory created.
>>
>> What can I do to fix this?
>>
>>
>> C:\Apps\Python35>pip install --upgrade pip
>>   File "C:\Apps\Python35\lib\shutil.py", line 377, in _rmtree_unsafe
>>     os.unlink(fullname)
>> PermissionError: [WinError 5] Access is denied:
>> 'C:\\Users\\Andreas\\AppData\\Local\\Temp\\pip-xf9kwi7g-uninstall\\apps\\python35\\scripts\\pip.exe'
>
> This is a problem with the way Windows locks running executables,
> conflicting with the way pip wants to delete (unlink) the executable.
> It appears to be a known issue by the pip people:
>
> https://github.com/pypa/pip/issues/2773
>
> One workaround is to skip the short-hand "pip" when upgrading pip
> itself, and type out the command in full:
>
> python -m pip install --upgrade pip
>
> Just make sure you run the same Python (so you may need to run
> "python3 -m pip etc", not sure) so it upgrades the right pip.

More accurately for Windows, you'll probably want 'py -3.5 -m pip ...'.

-- 
Zach


From duane.kaufman at gmail.com  Thu Jun 11 16:27:45 2015
From: duane.kaufman at gmail.com (TheSeeker)
Date: Thu, 11 Jun 2015 13:27:45 -0700 (PDT)
Subject: How to pretty mathematical formulas in Python? Similar to
 Mathematica formats.
In-Reply-To: <b881edd8-55e7-4eea-88fd-7b3a094a295c@googlegroups.com>
References: <b881edd8-55e7-4eea-88fd-7b3a094a295c@googlegroups.com>
Message-ID: <9856774e-0d0d-4bae-b9e3-6d8f3ae7b94d@googlegroups.com>

On Thursday, June 11, 2015 at 1:33:12 PM UTC-5, Sebastian M Cheung wrote:
> How to pretty mathematical formulas in Python? Similar to Mathematica formats.
> 
> Are there good packages to prettify mathematica formulas in Python?

Sympy (http://www.sympy.org/en/index.html) has some capabilities to pretty-print mathematics.

Duane


From minscheung at googlemail.com  Thu Jun 11 18:04:17 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Thu, 11 Jun 2015 15:04:17 -0700 (PDT)
Subject: How to pretty mathematical formulas in Python? Similar to
 Mathematica formats.
In-Reply-To: <b881edd8-55e7-4eea-88fd-7b3a094a295c@googlegroups.com>
References: <b881edd8-55e7-4eea-88fd-7b3a094a295c@googlegroups.com>
Message-ID: <59fafd20-efc7-43f9-93d7-339146686b34@googlegroups.com>

On Thursday, June 11, 2015 at 7:33:12 PM UTC+1, Sebastian M Cheung wrote:
> How to pretty mathematical formulas in Python? Similar to Mathematica formats.
> 
> Are there good packages to prettify mathematica formulas in Python?

Thanks Pythonistas


From minscheung at googlemail.com  Thu Jun 11 18:07:47 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Thu, 11 Jun 2015 15:07:47 -0700 (PDT)
Subject: XCode and Python
Message-ID: <7c1d1120-5afb-431c-9f90-457f5694c87c@googlegroups.com>

For some reason I cannot build now in XCode:

$ xcodebuild -find python
/Users/sebc/anaconda/bin/python

$python
Python 2.7.10 |Anaconda 2.2.0 (x86_64)| (default, May 28 2015, 17:04:42) 
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org

But XCode now simply say error

env: python: No such file or directory
Command /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed with exit code 127

I couldn't find anything relating XCode for iPhone related to Python build? I am using
anaconda Python

Anyone? 


From minscheung at googlemail.com  Thu Jun 11 18:09:05 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Thu, 11 Jun 2015 15:09:05 -0700 (PDT)
Subject: XCode and Python
In-Reply-To: <7c1d1120-5afb-431c-9f90-457f5694c87c@googlegroups.com>
References: <7c1d1120-5afb-431c-9f90-457f5694c87c@googlegroups.com>
Message-ID: <90a23bdc-7703-4397-b83a-92718ae10c48@googlegroups.com>

On Thursday, June 11, 2015 at 11:07:59 PM UTC+1, Sebastian M Cheung wrote:
> For some reason I cannot build now in XCode:
> 
> $ xcodebuild -find python
> /Users/sebc/anaconda/bin/python
> 
> $python
> Python 2.7.10 |Anaconda 2.2.0 (x86_64)| (default, May 28 2015, 17:04:42) 
> [GCC 4.2.1 (Apple Inc. build 5577)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> Anaconda is brought to you by Continuum Analytics.
> Please check out: http://continuum.io/thanks and https://binstar.org
> 
> But XCode now simply say error
> 
> env: python: No such file or directory
> Command /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed with exit code 127
> 
> I couldn't find anything relating XCode for iPhone related to Python build? I am using
> anaconda Python
> 
> Anyone?

Or I need to configure something in Xcode?


From chris at cdreimer.com  Thu Jun 11 18:30:20 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Thu, 11 Jun 2015 15:30:20 -0700
Subject: XCode and Python
In-Reply-To: <90a23bdc-7703-4397-b83a-92718ae10c48@googlegroups.com>
References: <7c1d1120-5afb-431c-9f90-457f5694c87c@googlegroups.com>
 <90a23bdc-7703-4397-b83a-92718ae10c48@googlegroups.com>
Message-ID: <557A0BFC.20505@cdreimer.com>

On 6/11/2015 3:09 PM, Sebastian M Cheung via Python-list wrote:
> Or I need to configure something in Xcode? 

Perhaps this link might help determine if the problem is with Xcode 
and/or Python.

http://stackoverflow.com/questions/5276967/python-in-xcode-6

Chris R.


From issacalby at gmail.com  Fri Jun 12 00:54:05 2015
From: issacalby at gmail.com (Alby Issac)
Date: Fri, 12 Jun 2015 10:24:05 +0530
Subject: Guidance for academic project
Message-ID: <CANJfs9N-3fZn6p1cgfbHY0JwjpeDCs0XT8doTeL58n78tNO7ww@mail.gmail.com>

Hi,

Am new to this group. Am a PG student in Computer Science  Engineering
from India. I have some doubts related to my academic research.

I just want to know that is it possible to implement a filter for view
updates in the database tables using python. and that filter will help to
reject false updates in the source database.

If it is possible, could anyone help to me do the same.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150612/08ffc18f/attachment.html>

From info at egenix.com  Fri Jun 12 04:25:38 2015
From: info at egenix.com (eGenix Team: M.-A. Lemburg)
Date: Fri, 12 Jun 2015 10:25:38 +0200
Subject: ANN: eGenix pyOpenSSL Distribution 0.13.9
Message-ID: <557A9782.6010807@egenix.com>

________________________________________________________________________
ANNOUNCING

                   eGenix.com pyOpenSSL Distribution

                            Version 0.13.9

            An easy-to-install and easy-to-use distribution
            of the pyOpenSSL Python interface for OpenSSL -
           available for Windows, Mac OS X and Unix platforms


This announcement is also available on our web-site for online reading:
http://www.egenix.com/company/news/eGenix-pyOpenSSL-Distribution-0.13.9.html

________________________________________________________________________
INTRODUCTION

The eGenix.com pyOpenSSL Distribution includes everything you need to
get started with SSL in Python.

It comes with an easy-to-use installer that includes the most recent
OpenSSL library versions in pre-compiled form, making your application
independent of OS provided OpenSSL libraries:

    http://www.egenix.com/products/python/pyOpenSSL/

pyOpenSSL is an open-source Python add-on that allows writing SSL/TLS-
aware network applications as well as certificate management tools:

    https://launchpad.net/pyopenssl/

OpenSSL is an open-source implementation of the SSL/TLS protocol:

    http://www.openssl.org/

________________________________________________________________________
NEWS

This new release of the eGenix.com pyOpenSSL Distribution includes the
following updates:

New in eGenix pyOpenSSL
-----------------------

 * Fixed a bug in the build process which resulted in the CA bundle
   files not get installed in the OpenSSL/ package dir.

 * Added a work-around for recent pip versions not showing the
   installer output, causing an apparently hanging installation
   process. The installer will now use a timeout when entering the
   crypto confirmation and report how to fix the problem (by using an
   environment variable EGENIX_CRYPTO_CONFIRM for confirmation).

 * Updated the Mozilla CA root bundle to version 2015-04-22.

 * Various minor fixes to the web installer to make installations on
   Linux and FreeBSD more robust, having pip uninstall not remove the
   .pyc/.pyo files, intermittent error causing a source installation
   in some rare cases.

New in OpenSSL
--------------

 * Updated included OpenSSL libraries from OpenSSL 1.0.1m to
   1.0.1n. See https://www.openssl.org/news/secadv_20150611.txt ?for a
   complete list of changes. The following fixes are relevant for
   pyOpenSSL applications:

   - CVE-2015-1788: Possible infinite loop during client
     authentication, which can be used for Denial of Service (DoS)
     attacks.

   - CVE-2015-1789: X509_cmp_time does not properly check the length
     of the ASN1_TIME string and can read a few bytes out of bounds,
     which can lead to a segmentation fault.

   - CVE-2015-1790: The PKCS#7 parsing code does not handle missing
     inner EncryptedContent correctly, which can lead to a NULL
     pointer dereference on parsing.

   - CVE-2015-1792: When verifying a signedData message the CMS code
     can enter an infinite loop if presented with an unknown hash
     function OID.

   - CVE-2015-1791: If a NewSessionTicket is received by a
     multi-threaded client when attempting to reuse a previous ticket
     then a race condition can occur potentially leading to a double
     free of the ticket data.

Please see the product changelog for the full set of changes.

    http://www.egenix.com/products/python/pyOpenSSL/changelog.html


pyOpenSSL / OpenSSL Binaries Included
-------------------------------------

In addition to providing sources, we make binaries available that
include both pyOpenSSL and the necessary OpenSSL libraries for all
supported platforms: Windows, Linux, Mac OS X and FreeBSD, for x86 and
x64.

To simplify installation, we have uploaded a web installer to PyPI
which will automatically choose the right binary for your platform, so
a simple

    pip install egenix-pyopenssl

will get you the package with OpenSSL libraries installed. Please see
our installation instructions for details:

    http://www.egenix.com/products/python/pyOpenSSL/#Installation

We have also added .egg-file distribution versions of our eGenix.com
pyOpenSSL Distribution for Windows, Linux and Mac OS X to the
available download options. These make setups using e.g. zc.buildout
and other egg-file based installers a lot easier.

________________________________________________________________________
DOWNLOADS

The download archives and instructions for installing the package can
be found at:

    http://www.egenix.com/products/python/pyOpenSSL/

________________________________________________________________________
UPGRADING

Before installing this version of pyOpenSSL, please make sure that
you uninstall any previously installed pyOpenSSL version. Otherwise,
you could end up not using the included OpenSSL libs.

_______________________________________________________________________
SUPPORT

Commercial support for these packages is available from eGenix.com.
Please see

    http://www.egenix.com/services/support/

for details about our support offerings.

________________________________________________________________________
MORE INFORMATION

For more information about the eGenix pyOpenSSL Distribution, licensing
and download instructions, please visit our web-site or write to
sales at egenix.com.

About eGenix (http://www.egenix.com/):

    eGenix is a software project, consulting and product company
    focusing on expert project services and professional quality
    products for companies, Python users and developers.

Enjoy,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 12 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2015-07-20: EuroPython 2015, Bilbao, Spain ...             38 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From steve+comp.lang.python at pearwood.info  Fri Jun 12 05:41:42 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 12 Jun 2015 09:41:42 GMT
Subject: Error in or
References: <3bf5dd52-c698-4392-be2b-5c0aeefc3cd3@googlegroups.com>
Message-ID: <557aa956$0$11125$c3e8da3@news.astraweb.com>

On Thu, 11 Jun 2015 08:40:50 -0700, subhabrata.banerji wrote:

>   if ("AND" in inp1) or ("OR" in inp1) or ("NOT" in inp1) or (">" in
>   inp1) or ("&" in inp1) or ("MAYBE" in inp1) or ("(" in inp1) or ("*"
>   in inp1) or (''' " ''' in   inp1):

This is better written as:

    if any(substr in inp1 for substr in 
           ['AND', 'OR', 'NOT', '>', '&', 'MAYBE', '(', '*', ' " ']):
        print 'FINE'


-- 
Steve


From marko at pacujo.net  Fri Jun 12 05:55:08 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 12 Jun 2015 12:55:08 +0300
Subject: Error in or
References: <3bf5dd52-c698-4392-be2b-5c0aeefc3cd3@googlegroups.com>
 <557aa956$0$11125$c3e8da3@news.astraweb.com>
Message-ID: <87k2v91car.fsf@elektro.pacujo.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info>:

> This is better written as:
>
>     if any(substr in inp1 for substr in 
>            ['AND', 'OR', 'NOT', '>', '&', 'MAYBE', '(', '*', ' " ']):
>         print 'FINE'

Or, equivalently:

   for substr in ['AND', 'OR', 'NOT', '>', '&', 'MAYBE', '(', '*', ' " ']:
       if substr in inp1:
           print('FINE')
           break


Marko


From hv at tbz-pariv.de  Fri Jun 12 07:12:52 2015
From: hv at tbz-pariv.de (=?ISO-8859-1?Q?Thomas_G=FCttler?=)
Date: Fri, 12 Jun 2015 04:12:52 -0700 (PDT)
Subject: Get classes from "self.MyClass" to improve subclassability
Message-ID: <4d336971-d986-4e21-85a2-ddc8ee995c7e@googlegroups.com>

Here is a snippet from the argparse module:

{{{
    def parse_known_args(self, args=None, namespace=None):
        ...
        # default Namespace built from parser defaults
        if namespace is None:
            namespace = Namespace() # < ======= my issue
}}}

I subclass from the class of the above snippet.

I would like to use a different Namespace class.

if the above could would use

    namespace = self.Namespace()

it would be very easy for me to inject a different Namespace class.

Yes, I have seen the "namespace" kwarg.

This is not the first time I would like the upstream code to make subclassing more fun.

Some months ago I asked myself how to call the "self.Namespace()" pattern:
 
http://stackoverflow.com/questions/27571848/name-of-design-pattern-get-class-from-class-level

The answer with the most upvotes states it is the Factory method pattern.

I prefer "self.Namespace()" to namespace kwargs.

What do you think? 


From rosuav at gmail.com  Fri Jun 12 07:28:31 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 12 Jun 2015 21:28:31 +1000
Subject: Get classes from "self.MyClass" to improve subclassability
In-Reply-To: <4d336971-d986-4e21-85a2-ddc8ee995c7e@googlegroups.com>
References: <4d336971-d986-4e21-85a2-ddc8ee995c7e@googlegroups.com>
Message-ID: <CAPTjJmpRiQbfEbTY1xHRdQbM5xGAsqP2jvmC6SMprnoaNQSSSQ@mail.gmail.com>

On Fri, Jun 12, 2015 at 9:12 PM, Thomas G?ttler <hv at tbz-pariv.de> wrote:
> I prefer "self.Namespace()" to namespace kwargs.
>
> What do you think?

Given that the namespace argument already exists, and you're proposing
a change, you'll need a much stronger justification than mere
preference. What's the downside of providing your own instance, rather
than injecting a class?

ChrisA


From minscheung at googlemail.com  Fri Jun 12 07:36:18 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Fri, 12 Jun 2015 04:36:18 -0700 (PDT)
Subject: Java to Python autoconverters
Message-ID: <4d3e5d71-72f2-47ea-b3f3-25630019ce83@googlegroups.com>

Are these available? Any good ones to recommend?


From steve+comp.lang.python at pearwood.info  Fri Jun 12 08:23:54 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 12 Jun 2015 12:23:54 GMT
Subject: Get classes from "self.MyClass" to improve subclassability
References: <4d336971-d986-4e21-85a2-ddc8ee995c7e@googlegroups.com>
Message-ID: <557acf5a$0$11125$c3e8da3@news.astraweb.com>

On Fri, 12 Jun 2015 04:12:52 -0700, Thomas G?ttler wrote:

> Here is a snippet from the argparse module:
> 
> {{{
>     def parse_known_args(self, args=None, namespace=None):
>         ...
>         # default Namespace built from parser defaults if namespace is
>         None:
>             namespace = Namespace() # < ======= my issue
> }}}
> 
> I subclass from the class of the above snippet.
> 
> I would like to use a different Namespace class.
> 
> if the above could would use
> 
>     namespace = self.Namespace()
> 
> it would be very easy for me to inject a different Namespace class.

Yes it would.

And here is how you do it, even when the parent class doesn't:

    class MySubclass(ParentClass):
        Namespace = Namespace
        def parse_known_args(self, args=None, namespace=None):
            if namespace is None:
                namespace = self.Namespace()
            # any other method overriding needed
            return super().parse_known_args(args, namespace)

In Python 2, you cannot use super(), you have to explicitly provide the 
arguments:

            return super(MySubclass,self).parse_known_args(args,namespace)


-- 
Steve


From hv at tbz-pariv.de  Fri Jun 12 08:47:08 2015
From: hv at tbz-pariv.de (=?ISO-8859-1?Q?Thomas_G=FCttler?=)
Date: Fri, 12 Jun 2015 05:47:08 -0700 (PDT)
Subject: Get classes from "self.MyClass" to improve subclassability
In-Reply-To: <557acf5a$0$11125$c3e8da3@news.astraweb.com>
References: <4d336971-d986-4e21-85a2-ddc8ee995c7e@googlegroups.com>
 <557acf5a$0$11125$c3e8da3@news.astraweb.com>
Message-ID: <78f8b0b9-eb7b-40a8-a90d-f2f47b36ad20@googlegroups.com>

Hi Steven,

I understand your solution. It is correct and works.

But the missing five characters "self." in the upstream code
produces a lot of more lines in the final result.

Regards,
  Thomas G?ttler


Am Freitag, 12. Juni 2015 14:24:06 UTC+2 schrieb Steven D'Aprano:
> On Fri, 12 Jun 2015 04:12:52 -0700, Thomas G?ttler wrote:
> 
> > Here is a snippet from the argparse module:
> > 
> > {{{
> >     def parse_known_args(self, args=None, namespace=None):
> >         ...
> >         # default Namespace built from parser defaults if namespace is
> >         None:
> >             namespace = Namespace() # < ======= my issue
> > }}}
> > 
> > I subclass from the class of the above snippet.
> > 
> > I would like to use a different Namespace class.
> > 
> > if the above could would use
> > 
> >     namespace = self.Namespace()
> > 
> > it would be very easy for me to inject a different Namespace class.
> 
> Yes it would.
> 
> And here is how you do it, even when the parent class doesn't:
> 
>     class MySubclass(ParentClass):
>         Namespace = Namespace
>         def parse_known_args(self, args=None, namespace=None):
>             if namespace is None:
>                 namespace = self.Namespace()
>             # any other method overriding needed
>             return super().parse_known_args(args, namespace)
> 
> In Python 2, you cannot use super(), you have to explicitly provide the 
> arguments:
> 
>             return super(MySubclass,self).parse_known_args(args,namespace)
> 
> 
> -- 
> Steve



From nestorjb at gmail.com  Fri Jun 12 08:51:46 2015
From: nestorjb at gmail.com (=?UTF-8?B?TsOpc3RvciBCb3Njw6Fu?=)
Date: Fri, 12 Jun 2015 08:21:46 -0430
Subject: Trying to configure Apache and Python 2.7 on Red Hat I get 403
 Forbidden
Message-ID: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>

Hi

I've been trying to configure Apache and Python 2.7 on Red Hat. I've tried
the different configurations i've seen on the web, I've given chmod -R 777
tu my python code but I still get 403 Forbidden and I don't get any errors.

Regards,

N?stor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150612/613cb174/attachment.html>

From stefan_ml at behnel.de  Fri Jun 12 08:51:56 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 12 Jun 2015 14:51:56 +0200
Subject: Java to Python autoconverters
In-Reply-To: <4d3e5d71-72f2-47ea-b3f3-25630019ce83@googlegroups.com>
References: <4d3e5d71-72f2-47ea-b3f3-25630019ce83@googlegroups.com>
Message-ID: <mlekle$3q3$1@ger.gmane.org>

Sebastian M Cheung via Python-list schrieb am 12.06.2015 um 13:36:
> Are these available? Any good ones to recommend?

I recommend not doing that. You'd end up with ugly and unidiomatic Python
code that's impossible to maintain, whereas you now (hopefully) have
somewhat idiomatic Java code that should be reasonably maintainable.

If you want to integrate Python code with Java code, take a look at Jython
instead. If that's not what you want, then feel free to unveil your intentions.

Stefan




From breamoreboy at yahoo.co.uk  Fri Jun 12 08:57:54 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 12 Jun 2015 13:57:54 +0100
Subject: Java to Python autoconverters
In-Reply-To: <4d3e5d71-72f2-47ea-b3f3-25630019ce83@googlegroups.com>
References: <4d3e5d71-72f2-47ea-b3f3-25630019ce83@googlegroups.com>
Message-ID: <mlel0s$9sl$1@ger.gmane.org>

On 12/06/2015 12:36, Sebastian M Cheung via Python-list wrote:
> Are these available? Any good ones to recommend?
>

Yes and no.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From grawburg at myglnc.com  Fri Jun 12 09:07:47 2015
From: grawburg at myglnc.com (Grawburg)
Date: Fri, 12 Jun 2015 09:07:47 -0400
Subject: os.system error returns
Message-ID: <2ebdfdf225c075b2f5ef350b06bc14f0@myglnc.com>

I have a piece of code written for a Raspberry Pi with no explanation for two of the lines -- and I can't find an explanation I understand.

Here are the lines:
if os.system('modprobe --first-time -q w1_gpio') ==0

if os.system('modprobe -q w1_gpio') == 256:



I know what the 'modprobe...' is, it's the 0 and the 256 I don't get. Where do these numbers come from?
I recognize they're some kind of error returns, but don't know what they mean.


Thanks,
Brian Grawburg






From ben+python at benfinney.id.au  Fri Jun 12 09:43:04 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 12 Jun 2015 23:43:04 +1000
Subject: os.system error returns
References: <2ebdfdf225c075b2f5ef350b06bc14f0@myglnc.com>
Message-ID: <85wpz9ghzr.fsf@benfinney.id.au>

Grawburg <grawburg at myglnc.com> writes:

> if os.system('modprobe --first-time -q w1_gpio') ==0
>
> if os.system('modprobe -q w1_gpio') == 256:
>
> I know what the 'modprobe...' is, it's the 0 and the 256 I don't get.
> Where do these numbers come from?

They are integer literals, they come from the source code.

The statements are comparing those integers to the return value from
?os.system?. The return value from ?os.system? is whatever was the child
process sets as its exit status.

> I recognize they're some kind of error returns, but don't know what
> they mean.

That's not up to Python, it's entirely set by the external program.

There is no standardisation of exit status values between different
programs. The best one can say is ?exit status 0 means success?.
Anything further is specific to particular programs and is not
universal.

You'll need to see the documentation for ?modprobe(1)? to find out what
its different exit status values mean.

-- 
 \       ?The Vatican is not a state.? a state must have people. There |
  `\    are no Vaticanians.? No-one gets born in the Vatican except by |
_o__)        an unfortunate accident.? ?Geoffrey Robertson, 2010-09-18 |
Ben Finney



From invalid at invalid.invalid  Fri Jun 12 09:48:40 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Fri, 12 Jun 2015 13:48:40 +0000 (UTC)
Subject: os.system error returns
References: <mailman.419.1434115180.13271.python-list@python.org>
Message-ID: <mlenvo$b5p$1@reader1.panix.com>

On 2015-06-12, Grawburg <grawburg at myglnc.com> wrote:
> I have a piece of code written for a Raspberry Pi with no explanation for two of the lines -- and I can't find an explanation I understand.
>
> Here are the lines:
> if os.system('modprobe --first-time -q w1_gpio') ==0
>
> if os.system('modprobe -q w1_gpio') == 256:
>
> I know what the 'modprobe...' is, it's the 0 and the 256 I don't get.
> Where do these numbers come from? I recognize they're some kind of
> error returns, but don't know what they mean.

In generail, Linux command line utilities return 0 to indicate
success/OK/true. They return non-zero to indicate failure/error/false.

The status you get back from os.system() on linux is a 16-bit number
with the the upper 8 bits the return value from the command and the
lower 8 bits the signal number that terminated the sub-process (0
means normal exit).

So os.system() will return 0 if the command it invoked executed
successfully and returned a 0, and os.system() returns 256 if the
command it invoked returned a 1.

-- 
Grant Edwards               grant.b.edwards        Yow! Did I say I was
                                  at               a sardine?  Or a bus???
                              gmail.com            


From __peter__ at web.de  Fri Jun 12 09:52:01 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 12 Jun 2015 15:52:01 +0200
Subject: os.system error returns
References: <2ebdfdf225c075b2f5ef350b06bc14f0@myglnc.com>
Message-ID: <mleo62$1nb$1@ger.gmane.org>

Grawburg wrote:

> I have a piece of code written for a Raspberry Pi with no explanation for
> two of the lines -- and I can't find an explanation I understand.
> 
> Here are the lines:
> if os.system('modprobe --first-time -q w1_gpio') ==0
> 
> if os.system('modprobe -q w1_gpio') == 256:
> 
> 
> 
> I know what the 'modprobe...' is, it's the 0 and the 256 I don't get.
> Where do these numbers come from? I recognize they're some kind of error
> returns, but don't know what they mean.

By convention a return value of 0 means that the invoked program terminated 
normally, non-zero returns indicate an error. For the details you have to 
look into the modprobe documentation.

The invoked program is free to return a value in the range 0...255 which 
then goes into the upper byte of the return value of os.system(). For 
example let's use python instead of modprobe:

>>> os.system("python -c 'exit(42)'")
10752

You don't recognize the 42? Then how about

>>> os.system("python -c 'exit(42)'") >> 8
42

Read

https://docs.python.org/2.7/library/os.html#os.system

for the details.



From invalid at invalid.invalid  Fri Jun 12 09:53:03 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Fri, 12 Jun 2015 13:53:03 +0000 (UTC)
Subject: os.system error returns
References: <2ebdfdf225c075b2f5ef350b06bc14f0@myglnc.com>
 <mailman.420.1434116599.13271.python-list@python.org>
Message-ID: <mleo7v$b5p$2@reader1.panix.com>

On 2015-06-12, Ben Finney <ben+python at benfinney.id.au> wrote:

> There is no standardisation of exit status values between different
> programs. The best one can say is ?exit status 0 means success?.
> Anything further is specific to particular programs and is not
> universal.
>
> You'll need to see the documentation for ?modprobe(1)? to find out what
> its different exit status values mean.

It's modprobe(8), and all the man page says is that it returns
non-zero if you try to remove or insert a module it can't find.

Explicitly checking for an os.system() return value of 1<<8 seems like
a pretty bad idea to me, since there's nothing in the modprobe docs
that gurantees it will return 1 under some particular conditions.

-- 
Grant Edwards               grant.b.edwards        Yow! YOU PICKED KARL
                                  at               MALDEN'S NOSE!!
                              gmail.com            


From ian.g.kelly at gmail.com  Fri Jun 12 09:54:17 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 12 Jun 2015 07:54:17 -0600
Subject: os.system error returns
In-Reply-To: <2ebdfdf225c075b2f5ef350b06bc14f0@myglnc.com>
References: <2ebdfdf225c075b2f5ef350b06bc14f0@myglnc.com>
Message-ID: <CALwzidk4RoM0owVcWWYGGVUzsGbYWvGzyLXMh=YixDA+sa1xqQ@mail.gmail.com>

On Jun 12, 2015 7:21 AM, "Grawburg" <grawburg at myglnc.com> wrote:
>
> I have a piece of code written for a Raspberry Pi with no explanation for
two of the lines -- and I can't find an explanation I understand.
>
> Here are the lines:
> if os.system('modprobe --first-time -q w1_gpio') ==0
>
> if os.system('modprobe -q w1_gpio') == 256:
>
>
>
> I know what the 'modprobe...' is, it's the 0 and the 256 I don't get.
Where do these numbers come from?
> I recognize they're some kind of error returns, but don't know what they
mean.

Exit code 0 traditionally means success. The exit status is two bytes, with
the low-order byte normally containing the exit code and the high-order
byte containing the signal that caused the program to exit. So 256 would
appear to mean that modprobe exited due to SIGHUP, although that seems an
odd thing to test for here so I may be wrong.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150612/5420715f/attachment.html>

From ian.g.kelly at gmail.com  Fri Jun 12 09:55:47 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 12 Jun 2015 07:55:47 -0600
Subject: os.system error returns
In-Reply-To: <CALwzidk4RoM0owVcWWYGGVUzsGbYWvGzyLXMh=YixDA+sa1xqQ@mail.gmail.com>
References: <2ebdfdf225c075b2f5ef350b06bc14f0@myglnc.com>
 <CALwzidk4RoM0owVcWWYGGVUzsGbYWvGzyLXMh=YixDA+sa1xqQ@mail.gmail.com>
Message-ID: <CALwzidnLXQuuO1uw+bX35piwBKoKOGongRc5GNVW+c5r2V4Bow@mail.gmail.com>

On Jun 12, 2015 7:54 AM, "Ian Kelly" <ian.g.kelly at gmail.com> wrote:
>
> On Jun 12, 2015 7:21 AM, "Grawburg" <grawburg at myglnc.com> wrote:
> >
> > I have a piece of code written for a Raspberry Pi with no explanation
for two of the lines -- and I can't find an explanation I understand.
> >
> > Here are the lines:
> > if os.system('modprobe --first-time -q w1_gpio') ==0
> >
> > if os.system('modprobe -q w1_gpio') == 256:
> >
> >
> >
> > I know what the 'modprobe...' is, it's the 0 and the 256 I don't get.
Where do these numbers come from?
> > I recognize they're some kind of error returns, but don't know what
they mean.
>
> Exit code 0 traditionally means success. The exit status is two bytes,
with the low-order byte normally containing the exit code and the
high-order byte containing the signal that caused the program to exit. So
256 would appear to mean that modprobe exited due to SIGHUP, although that
seems an odd thing to test for here so I may be wrong.

Per the last two posters I have it backward, which explains my confusion.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150612/7968e95f/attachment.html>

From ian.g.kelly at gmail.com  Fri Jun 12 09:58:56 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 12 Jun 2015 07:58:56 -0600
Subject: Java to Python autoconverters
In-Reply-To: <mlekle$3q3$1@ger.gmane.org>
References: <4d3e5d71-72f2-47ea-b3f3-25630019ce83@googlegroups.com>
 <mlekle$3q3$1@ger.gmane.org>
Message-ID: <CALwzidn2X2RT9ip1mTkzeYQ0FH_PgTN-5Gkv0JSpqvUcsYbY_g@mail.gmail.com>

On Jun 12, 2015 6:53 AM, "Stefan Behnel" <stefan_ml at behnel.de> wrote:
>
> Sebastian M Cheung via Python-list schrieb am 12.06.2015 um 13:36:
> > Are these available? Any good ones to recommend?
>
> I recommend not doing that. You'd end up with ugly and unidiomatic Python
> code that's impossible to maintain, whereas you now (hopefully) have
> somewhat idiomatic Java code that should be reasonably maintainable.
>
> If you want to integrate Python code with Java code, take a look at Jython
> instead. If that's not what you want, then feel free to unveil your
intentions.

I've also found Jpype useful in the past, allowing integration between
CPython and a JVM. I'm not sure if it's actively maintained, though.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150612/2edfa667/attachment.html>

From random832 at fastmail.us  Fri Jun 12 10:25:28 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 12 Jun 2015 10:25:28 -0400
Subject: os.system error returns
In-Reply-To: <CALwzidk4RoM0owVcWWYGGVUzsGbYWvGzyLXMh=YixDA+sa1xqQ@mail.gmail.com>
References: <2ebdfdf225c075b2f5ef350b06bc14f0@myglnc.com>
 <CALwzidk4RoM0owVcWWYGGVUzsGbYWvGzyLXMh=YixDA+sa1xqQ@mail.gmail.com>
Message-ID: <1434119128.1233979.293924985.63E81914@webmail.messagingengine.com>

On Fri, Jun 12, 2015, at 09:54, Ian Kelly wrote:

> Exit code 0 traditionally means success. The exit status is two bytes,
> with
> the low-order byte normally containing the exit code and the high-order
> byte containing the signal that caused the program to exit.

That's backwards. The signal (or 0 for a normal exit, or 0177 if the
process is stopped) is in the low seven bits of the low-order byte, the
core dump flag is in the high bit of the low-order byte and the exit
status or the stop signal is in the high-order byte.

I think you're confused because the _shell_ puts the exit status, or the
signal plus 128, in the $? variable, and many languages (but not python)
follow suit.


From torriem at gmail.com  Fri Jun 12 10:30:56 2015
From: torriem at gmail.com (Michael Torrie)
Date: Fri, 12 Jun 2015 08:30:56 -0600
Subject: Java to Python autoconverters
In-Reply-To: <4d3e5d71-72f2-47ea-b3f3-25630019ce83@googlegroups.com>
References: <4d3e5d71-72f2-47ea-b3f3-25630019ce83@googlegroups.com>
Message-ID: <557AED20.5020806@gmail.com>

On 06/12/2015 05:36 AM, Sebastian M Cheung via Python-list wrote:
> Are these available? Any good ones to recommend?

The only use case for such a program that I can think of is a compiler
that is just using another language as an intermediate step, and that
language is usually going to be compiled (for example, several compilers
for different languages emit C code) or some other virtual machine like
a Javascript engine.

I don't want to put this too harshly, but the job of translating from
one language to another, and making the code idiomatic and maintainable
is up to you the programmer.  If you are a programmer you should be able
to easily do this, even if you have not worked with the source language
before.



From random832 at fastmail.us  Fri Jun 12 10:46:36 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 12 Jun 2015 10:46:36 -0400
Subject: os.system error returns
In-Reply-To: <1434119128.1233979.293924985.63E81914@webmail.messagingengine.com>
References: <2ebdfdf225c075b2f5ef350b06bc14f0@myglnc.com>
 <CALwzidk4RoM0owVcWWYGGVUzsGbYWvGzyLXMh=YixDA+sa1xqQ@mail.gmail.com>
 <1434119128.1233979.293924985.63E81914@webmail.messagingengine.com>
Message-ID: <1434120396.1238674.293932785.44114184@webmail.messagingengine.com>

For completeness I will note that Windows is completely different. The
plain exit status (1 for typical command failures) appears in the
os.system result rather than a wait-encoded value. And, incidentally, an
MSVC program which calls abort() will return an exit status of 3. A
process that terminates another process with TerminateProcess can set
that process's exit status to an arbitrary 32-bit value. The MSVC exit
function limits the value to one byte, but the raw ExitProcess system
call supports any 32-bit value.

And, just for fun, two versions of "cat" on my system (one supplied with
Git, and one from gnuwin32) return 0x10200 and 0xC000013A respectively
when terminated by Ctrl-C. I think the first is some effort at an
alternate encoding that's compatible with plain exit statuses in the
low-order byte, and the second is the value of STATUS_CONTROL_C_EXIT (an
HRESULT).

I found some information on google suggesting that some unix-alike
layers use an exit status of 0xC00002NN to represent signals (with, once
again, plain exit status being a value of 0x00NN).

All of these values are available as the returned value from os.system.


From fabien.maussion at gmail.com  Fri Jun 12 11:00:09 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Fri, 12 Jun 2015 17:00:09 +0200
Subject: zip as iterator and bad/good practices
Message-ID: <mles66$sk2$1@speranza.aioe.org>

Folks,

I am developing a program which I'd like to be python 2 and 3 
compatible. I am still relatively new to python and I use primarily py3 
for development. Every once in a while I use a py2 interpreter to see if 
my tests pass through.

I just spent several hours tracking down a bug which was related to the 
fact that zip is an iterator in py3 but not in py2. Of course I did not 
know about that difference. I've found the izip() function which should 
do what I want, but that awful bug made me wonder: is it a bad practice 
to interactively modify the list you are iterating over?

I am computing mass fluxes along glacier branches ordered by 
hydrological order, i.e. branch i is guaranteed to flow in a branch 
later in that list. Branches are objects which have a pointer to the 
object they are flowing into.

In pseudo code:

for stuff, branch in zip(stuffs, branches):
	# compute flux
	...
	# add to the downstream branch
	id_branch = branches.index(branch.flows_to)
	branches[id_branch].property.append(stuff_i_computed)

So, all downstream branches in python2 where missing information from 
their tributaries. It is quite a dangerous code but I can't find a more 
elegant solution.

Thanks!

Fabien



From fabien.maussion at gmail.com  Fri Jun 12 11:05:19 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Fri, 12 Jun 2015 17:05:19 +0200
Subject: zip as iterator and bad/good practices
References: <mles66$sk2$1@speranza.aioe.org>
Message-ID: <mlesfd$sk2$2@speranza.aioe.org>

On 06/12/2015 05:00 PM, Fabien wrote:
> I've found the izip() function which should do what I want

I've just come accross a stackoverflow post where they recommend:

from future_builtins import zip

which is OK since I don't want to support versions <= 2.6


From ian.g.kelly at gmail.com  Fri Jun 12 11:26:30 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 12 Jun 2015 09:26:30 -0600
Subject: zip as iterator and bad/good practices
In-Reply-To: <mles66$sk2$1@speranza.aioe.org>
References: <mles66$sk2$1@speranza.aioe.org>
Message-ID: <CALwzidmFgEzF-ymVtye_-m9Q5yr_GX5fBsMBj_AyrPo-LhXQng@mail.gmail.com>

On Fri, Jun 12, 2015 at 9:00 AM, Fabien <fabien.maussion at gmail.com> wrote:
> Folks,
>
> I am developing a program which I'd like to be python 2 and 3 compatible. I
> am still relatively new to python and I use primarily py3 for development.
> Every once in a while I use a py2 interpreter to see if my tests pass
> through.
>
> I just spent several hours tracking down a bug which was related to the fact
> that zip is an iterator in py3 but not in py2. Of course I did not know
> about that difference. I've found the izip() function which should do what I
> want

If you're supporting both 2 and 3, you may want to look into using the
third-party "six" library, which provides utilities for writing
cross-compatible code.  Using the correct zip() function with six is
just:

    from six.moves import zip

> but that awful bug made me wonder: is it a bad practice to
> interactively modify the list you are iterating over?

Generally speaking, yes, it's bad practice to add or remove items
because this may result in items being visited more than once or not
at all. Modifying or replacing items however is usually not an issue.

> I am computing mass fluxes along glacier branches ordered by hydrological
> order, i.e. branch i is guaranteed to flow in a branch later in that list.
> Branches are objects which have a pointer to the object they are flowing
> into.
>
> In pseudo code:
>
> for stuff, branch in zip(stuffs, branches):
>         # compute flux
>         ...
>         # add to the downstream branch
>         id_branch = branches.index(branch.flows_to)
>         branches[id_branch].property.append(stuff_i_computed)

Er, I don't see the problem here. The branch object in the zip list
and the branch object in branches should be the *same* object, so the
downstream branch update should be reflected when you visit it later
in the iteration, regardless of whether zip returns a list or an iterator.

Tangentially, unless you're using id_branch for something else that
isn't shown here, is it really necessary to search the list for the
downstream branch when it looks like you already have a reference to
it? Could the above simply be replaced with:

    branch.flows_to.property.append(stuff_i_computed)


From fabien.maussion at gmail.com  Fri Jun 12 11:34:03 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Fri, 12 Jun 2015 17:34:03 +0200
Subject: zip as iterator and bad/good practices
References: <mles66$sk2$1@speranza.aioe.org>
 <mailman.428.1434122838.13271.python-list@python.org>
Message-ID: <mleu5a$1pf$1@speranza.aioe.org>

On 06/12/2015 05:26 PM, Ian Kelly wrote:
>> for stuff, branch in zip(stuffs, branches):
>> >         # compute flux
>> >         ...
>> >         # add to the downstream branch
>> >         id_branch = branches.index(branch.flows_to)
>> >         branches[id_branch].property.append(stuff_i_computed)
> Er, I don't see the problem here. The branch object in the zip list
> and the branch object in branches should be the*same*  object, so the
> downstream branch update should be reflected when you visit it later
> in the iteration, regardless of whether zip returns a list or an iterator.
>
> Tangentially, unless you're using id_branch for something else that
> isn't shown here, is it really necessary to search the list for the
> downstream branch when it looks like you already have a reference to
> it? Could the above simply be replaced with:
>
>      branch.flows_to.property.append(stuff_i_computed)

Thanks a lot for your careful reading! I overly simplified my example 
and indeed this line works fine. I was adding things to "stuffs" too, 
which is a list of lists... Sorry for the confusion!


From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Fri Jun 12 11:53:08 2015
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Fri, 12 Jun 2015 16:53:08 +0100
Subject: Passing new fields to an object
Message-ID: <mlev91$4ji$1@speranza.aioe.org>

I would like to do something like this:

class C:
	def __init__(self,**parms):
	...

c=C(f1=1,f2=None)

I want to have, for the object
	self.f1=1
	self.f2=None

for an arbitrary number of parameters.

What is the best way to achieve this?

Thanks



From fabien.maussion at gmail.com  Fri Jun 12 11:59:20 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Fri, 12 Jun 2015 17:59:20 +0200
Subject: zip as iterator and bad/good practices
References: <mles66$sk2$1@speranza.aioe.org>
 <mailman.428.1434122838.13271.python-list@python.org>
Message-ID: <mlevkm$4nt$1@speranza.aioe.org>

On 06/12/2015 05:26 PM, Ian Kelly wrote:
>> but that awful bug made me wonder: is it a bad practice to
>> >interactively modify the list you are iterating over?
> Generally speaking, yes, it's bad practice to add or remove items
> because this may result in items being visited more than once or not
> at all. Modifying or replacing items however is usually not an issue.
>

Thanks. In that case I was modifying items and needed them to be updated 
during the loop. I kept the solution as is and my tests pass in 2 and 3.

I will consider using six. Currently all my modules begin with:


from __future__ import division
try:
     from itertools import izip as zip
except ImportError:
     pass

Which might even become longer if I find other bugs ;-)

Fabien


From g.starck at gmail.com  Fri Jun 12 12:17:00 2015
From: g.starck at gmail.com (gst)
Date: Fri, 12 Jun 2015 09:17:00 -0700 (PDT)
Subject: Passing new fields to an object
In-Reply-To: <mlev91$4ji$1@speranza.aioe.org>
References: <mlev91$4ji$1@speranza.aioe.org>
Message-ID: <e120ad04-e4f1-4f2c-b004-0c70b85f4047@googlegroups.com>

Le vendredi 12 juin 2015 11:53:24 UTC-4, Paulo da Silva a ?crit?:
> I would like to do something like this:
> 
> class C:
> 	def __init__(self,**parms):
> 	...
> 
> c=C(f1=1,f2=None)
> 
> I want to have, for the object
> 	self.f1=1
> 	self.f2=None
> 
> for an arbitrary number of parameters.
> 
> What is the best way to achieve this?
> 
> Thanks

in the __init__, simply do:

    self.__dict__.update(**parms)

regards,

gst.


From __peter__ at web.de  Fri Jun 12 12:17:09 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 12 Jun 2015 18:17:09 +0200
Subject: Passing new fields to an object
References: <mlev91$4ji$1@speranza.aioe.org>
Message-ID: <mlf0m6$oui$1@ger.gmane.org>

Paulo da Silva wrote:

> I would like to do something like this:
> 
> class C:
> def __init__(self,**parms):
> ...
> 
> c=C(f1=1,f2=None)
> 
> I want to have, for the object
> self.f1=1
> self.f2=None
> 
> for an arbitrary number of parameters.
> 
> What is the best way to achieve this?

Use a dict ;)

While I'd recommend that you spell out the possible arguments here is one 
way to not obey my advice:

>>> import types
>>> class C(types.SimpleNamespace):
...     pass
... 
>>> c = C(f1=1, f2=None)
>>> c
C(f1=1, f2=None)

If you want to do it manually you can either use

for name, value in parms.items():
    setattr(self, name, value)

or the less general

self.__dict__.update(parms)

Again, if you don't know the names in advance the appropriate data structure 
is a dict.



From pkpearson at nowhere.invalid  Fri Jun 12 12:21:14 2015
From: pkpearson at nowhere.invalid (Peter Pearson)
Date: 12 Jun 2015 16:21:14 GMT
Subject: Guidance for academic project
References: <mailman.411.1434092329.13271.python-list@python.org>
Message-ID: <cu0f7qFjkeaU1@mid.individual.net>

On Fri, 12 Jun 2015 10:24:05 +0530, Alby Issac <issacalby at gmail.com> wrote:
>
> Am new to this group.
[snip]

Welcome.


> . . . I just want to know that is it possible to implement a filter
> for view updates in the database tables using python. and that filter
> will help to reject false updates in the source database.

To get useful advice, you might need to clarify the requirements.
Does the system allow someone to update the database *without* going
through the Python program?  What kind of test might the Python program
apply in order to recognize a false update -- for example, is falseness
a matter of the data in the update, or of the authorization of the person
making the update?

-- 
To email me, substitute nowhere->runbox, invalid->com.


From breamoreboy at yahoo.co.uk  Fri Jun 12 12:22:21 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 12 Jun 2015 17:22:21 +0100
Subject: zip as iterator and bad/good practices
In-Reply-To: <mles66$sk2$1@speranza.aioe.org>
References: <mles66$sk2$1@speranza.aioe.org>
Message-ID: <mlf108$vcv$1@ger.gmane.org>

On 12/06/2015 16:00, Fabien wrote:
> Folks,
>
> I am developing a program which I'd like to be python 2 and 3
> compatible. I am still relatively new to python and I use primarily py3
> for development. Every once in a while I use a py2 interpreter to see if
> my tests pass through.
>
> I just spent several hours tracking down a bug which was related to the
> fact that zip is an iterator in py3 but not in py2. Of course I did not
> know about that difference. I've found the izip() function which should
> do what I want, but that awful bug made me wonder: is it a bad practice
> to interactively modify the list you are iterating over?
>
> I am computing mass fluxes along glacier branches ordered by
> hydrological order, i.e. branch i is guaranteed to flow in a branch
> later in that list. Branches are objects which have a pointer to the
> object they are flowing into.
>
> In pseudo code:
>
> for stuff, branch in zip(stuffs, branches):
>      # compute flux
>      ...
>      # add to the downstream branch
>      id_branch = branches.index(branch.flows_to)
>      branches[id_branch].property.append(stuff_i_computed)
>
> So, all downstream branches in python2 where missing information from
> their tributaries. It is quite a dangerous code but I can't find a more
> elegant solution.
>
> Thanks!
>
> Fabien
>

Start here https://docs.python.org/3/howto/pyporting.html

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From mewthree19 at hotmail.com  Fri Jun 12 13:39:36 2015
From: mewthree19 at hotmail.com (Wayne Norman)
Date: Fri, 12 Jun 2015 10:39:36 -0700 (PDT)
Subject: Writting Dialog to enter preset items from a combo list
In-Reply-To: <95b7c4d2-6afd-4694-b3c4-6e4f856b4617@googlegroups.com>
References: <95b7c4d2-6afd-4694-b3c4-6e4f856b4617@googlegroups.com>
Message-ID: <a6f934b9-d013-4c92-b1f5-e5dbf84886b4@googlegroups.com>

ok I have wxFormBuilder as well and so could use some help with working on my project I would explain exactally what I am doing to and one who wishes to aid me with this.

for now I will has it has to do with a minecraft mod if you would like to known more I will put more here


From breamoreboy at yahoo.co.uk  Fri Jun 12 13:46:39 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 12 Jun 2015 18:46:39 +0100
Subject: Writting Dialog to enter preset items from a combo list
In-Reply-To: <a6f934b9-d013-4c92-b1f5-e5dbf84886b4@googlegroups.com>
References: <95b7c4d2-6afd-4694-b3c4-6e4f856b4617@googlegroups.com>
 <a6f934b9-d013-4c92-b1f5-e5dbf84886b4@googlegroups.com>
Message-ID: <mlf5ub$tnq$1@ger.gmane.org>

On 12/06/2015 18:39, Wayne Norman wrote:
> ok I have wxFormBuilder as well and so could use some help with working on my project I would explain exactally what I am doing to and one who wishes to aid me with this.
>
> for now I will has it has to do with a minecraft mod if you would like to known more I will put more here
>

You'd be better off putting your question(s) on the wxPython users list, 
see http://wiki.wxpython.org/wxPython-users

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From subhabrata.banerji at gmail.com  Fri Jun 12 13:52:19 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Fri, 12 Jun 2015 10:52:19 -0700 (PDT)
Subject: How may I Integrate Python Code with REST
Message-ID: <02f1571f-a497-4b21-99b5-b39a627bb5ea@googlegroups.com>

Dear Group,

I wrote a Python code. In the code there are two modules where we may insert data from outside. They are updating some training module and updating index. As a standalone code this is working fine. 

I need to port this code to REST. I tried to learn Flask. My Practice for Flask is okay. I can put,get,delete. 

But how may I proceed I am not getting much idea. I bit new in REST. 
If any one of the esteemed members may kindly provide an idea how may I proceed? 

Regards,
Subhabrata Banerjee.  


From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Fri Jun 12 14:26:09 2015
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Fri, 12 Jun 2015 19:26:09 +0100
Subject: Passing new fields to an object
References: <mlev91$4ji$1@speranza.aioe.org>
 <e120ad04-e4f1-4f2c-b004-0c70b85f4047@googlegroups.com>
Message-ID: <mlf87v$qmk$1@speranza.aioe.org>

On 12-06-2015 17:17, gst wrote:
> Le vendredi 12 juin 2015 11:53:24 UTC-4, Paulo da Silva a ?crit :

> in the __init__, simply do:
> 
>     self.__dict__.update(**parms)
> 
> regards,
> 
Ok. Thanks.




From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Fri Jun 12 14:34:09 2015
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Fri, 12 Jun 2015 19:34:09 +0100
Subject: Passing new fields to an object
References: <mlev91$4ji$1@speranza.aioe.org>
 <mailman.429.1434125838.13271.python-list@python.org>
Message-ID: <mlf8mu$ro2$1@speranza.aioe.org>

On 12-06-2015 17:17, Peter Otten wrote:
> Paulo da Silva wrote:
> 
...

> 
>>>> import types
>>>> class C(types.SimpleNamespace):
> ...     pass
> ... 
>>>> c = C(f1=1, f2=None)
>>>> c
> C(f1=1, f2=None)
> 

Thanks for all your explanations.
This solution works. Would you please detail a little on how it works?
Or just point me out some readings.
I am confused because types.SimpleNamespace seems to be a class!
>From docs ...:
class SimpleNamespace:
    def __init__(self, **kwargs):
        self.__dict__.update(kwargs)
    def __repr__(self):
        keys = sorted(self.__dict__)
        items = ("{}={!r}".format(k, self.__dict__[k]) for k in keys)
        return "{}({})".format(type(self).__name__, ", ".join(items))
    def __eq__(self, other):
        return self.__dict__ == other.__dict__



From nad at acm.org  Fri Jun 12 14:36:06 2015
From: nad at acm.org (Ned Deily)
Date: Fri, 12 Jun 2015 11:36:06 -0700
Subject: XCode and Python
References: <7c1d1120-5afb-431c-9f90-457f5694c87c@googlegroups.com>
 <90a23bdc-7703-4397-b83a-92718ae10c48@googlegroups.com>
Message-ID: <nad-456ACF.11360612062015@news.gmane.org>

In article <90a23bdc-7703-4397-b83a-92718ae10c48 at googlegroups.com>,
 Sebastian M Cheung via Python-list <python-list at python.org> wrote:
> On Thursday, June 11, 2015 at 11:07:59 PM UTC+1, Sebastian M Cheung wrote:
> > For some reason I cannot build now in XCode:
> > 
> > $ xcodebuild -find python
> > /Users/sebc/anaconda/bin/python
> > 
> > $python
> > Python 2.7.10 |Anaconda 2.2.0 (x86_64)| (default, May 28 2015, 17:04:42) 
> > [GCC 4.2.1 (Apple Inc. build 5577)] on darwin
> > Type "help", "copyright", "credits" or "license" for more information.
> > Anaconda is brought to you by Continuum Analytics.
> > Please check out: http://continuum.io/thanks and https://binstar.org
> > 
> > But XCode now simply say error
> > 
> > env: python: No such file or directory
> > Command 
> > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolcha
> > in/usr/bin/clang failed with exit code 127
> > 
> > I couldn't find anything relating XCode for iPhone related to Python build? 
> > I am using
> > anaconda Python
> > Anyone?
> Or I need to configure something in Xcode?

Wild guess: did you by any chance delete the Apple-supplied system 
python?  What is the result of:
$ /usr/bin/python -c 'import sys;print(sys.version)'

-- 
 Ned Deily,
 nad at acm.org



From sohcahtoa82 at gmail.com  Fri Jun 12 14:59:33 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Fri, 12 Jun 2015 11:59:33 -0700 (PDT)
Subject: How may I Integrate Python Code with REST
In-Reply-To: <02f1571f-a497-4b21-99b5-b39a627bb5ea@googlegroups.com>
References: <02f1571f-a497-4b21-99b5-b39a627bb5ea@googlegroups.com>
Message-ID: <1d454ab1-b589-4d0d-9d57-b00483082dcc@googlegroups.com>

On Friday, June 12, 2015 at 10:52:30 AM UTC-7, subhabrat... at gmail.com wrote:
> Dear Group,
> 
> I wrote a Python code. In the code there are two modules where we may insert data from outside. They are updating some training module and updating index. As a standalone code this is working fine. 
> 
> I need to port this code to REST. I tried to learn Flask. My Practice for Flask is okay. I can put,get,delete. 
> 
> But how may I proceed I am not getting much idea. I bit new in REST. 
> If any one of the esteemed members may kindly provide an idea how may I proceed? 
> 
> Regards,
> Subhabrata Banerjee.

It is slightly unclear what you're asking.  REST isn't a programming language or a module or anything like that.  It is a software architecture style.

Flask is often used for implementing a REST interface.  If you understand how to use Flask, you're basically there already.


From nad at acm.org  Fri Jun 12 15:08:35 2015
From: nad at acm.org (Ned Deily)
Date: Fri, 12 Jun 2015 12:08:35 -0700
Subject: Detect if specific Python.app instance is already running
References: <a3be1707-0eae-46fa-9921-e9c145816ad5@googlegroups.com>
 <11e093d5-b78e-4ac6-9a7f-649cb2c2c942@googlegroups.com>
 <mailman.281.1433718479.13271.python-list@python.org>
 <6651a781-abe1-4f25-b1f3-1f849776d060@googlegroups.com>
Message-ID: <nad-692841.12083512062015@news.gmane.org>

In article <6651a781-abe1-4f25-b1f3-1f849776d060 at googlegroups.com>,
 Andrei <andrei.fokau at gmail.com> wrote:

> On Monday, June 8, 2015 at 1:08:07 AM UTC+2, Ned Deily wrote:
> > In article <11e093d5-b78e-4ac6-9a7f-649cb2c2c942 at googlegroups.com>,
> >  Andrei wrote:
> > > Alright, I have had some development in 
> > > http://stackoverflow.com/questions/30694560/detect-if-specific-python-app-
> > > inst
> > > ance-is-already-running and can prevent running multiple instances of the 
> > > same app/script (by lockf), but I still need to identify which Python.app 
> > > instance is running certain script. I guess it's more Cocoa-related 
> > > question 
> > > but I hope someone had same problem with tkinter etc.
> > 
> > Can you give a more complete explanation of the goal, i.e. what you are 
> > trying to end up with here, rather than the steps you are taking to get 
> > there?  It's not at all clear to me what your goal is and the cumbersome 
> > steps you find you have to take suggest that you might be going down the 
> > wrong path.  OS X apps usually don't have to resort to such hacks.  If 
> > you haven't already, make sure you have looked at the "Information 
> > Property List Key Reference" for OS X (and iOS) apps.  For instance, key 
> > LSMultipleInstancesProhibited *might* be useful if you really only want 
> > to ensure that there is only instance of an app launched.
> > 
> > https://developer.apple.com/library/mac/documentation/General/Reference/I
> > nfoPlistKeyReference/Articles/LaunchServicesKeys.html
> > 
> > -- 
> >  Ned Deily
> 
> 
> Hi Ned,
> Thanks for your reply! The problem is a bit tricky as I see it. Multiple 
> instances of Python.app needs to be run since it may run different tkinter 
> apps etc. What I need is rather simple:
> 
> 1. start app, mark the instance with tkinter app name
> 2. start another one, detect that the same instance is already running and 
> identify it
> 3. focus on the first instance, close the second one
> 
> I have solved it with a lock file and a pid file (I guess I could use just 
> one file for both jobs, but didn't have time to test it yet). I believe it 
> could be done without files, i.e. by setting some public attributes on app 
> instances. I could probably also use OS X key storage for this but it gets 
> too similar to the file approach and I am not sure if it's any better.
> 
> I may be wrong in my thinking. If so, just let me know :)

If you have something that works for you, great!  A question that 
remains is: how are you launching all these apps?  You mention tkinter 
so presumably these are "gui" apps.  On OS X, native "gui" apps are 
expected to be packaged as app bundles and launched via Launch Services, 
e.g. by double-clicking in the Finder or by /usr/bin/open from the 
command line.  Framework installs of Python cheat a bit and provide a 
default Python.app bundle buried in the framework and do some trickery 
when python is started conventionally from the command line (e.g. via 
something like /usr/local/bin/python) to allow the process to be 
promoted to a gui app, if necessary, like when using Tk from tkinter.  
If you need gui apps, the most "Mac-thonic" approach would be to use 
something like py2app to produce each different app as a unique gui app 
bundle and then, assuming each app's Info.plist are defined properly, OS 
X will enforce that only one instance of each app is running at a time.  
But that works only if they are launched as apps rather than run as 
scripts from the Python interpreter.  Of course, this all may be 
overkill for what you are trying to do but dynamically modifying app 
bundles or some such certainly does not sound like a good idea for lots 
of reasons.  

In any case, good luck!

-- 
 Ned Deily,
 nad at acm.org



From __peter__ at web.de  Fri Jun 12 15:12:26 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 12 Jun 2015 21:12:26 +0200
Subject: Passing new fields to an object
References: <mlev91$4ji$1@speranza.aioe.org>
 <mailman.429.1434125838.13271.python-list@python.org>
 <mlf8mu$ro2$1@speranza.aioe.org>
Message-ID: <mlfaut$h4a$1@ger.gmane.org>

Paulo da Silva wrote:

> On 12-06-2015 17:17, Peter Otten wrote:
>> Paulo da Silva wrote:
>> 
> ...
> 
>> 
>>>>> import types
>>>>> class C(types.SimpleNamespace):
>> ...     pass
>> ...
>>>>> c = C(f1=1, f2=None)
>>>>> c
>> C(f1=1, f2=None)
>> 
> 
> Thanks for all your explanations.
> This solution works. Would you please detail a little on how it works?
> Or just point me out some readings.
> I am confused because types.SimpleNamespace seems to be a class!

It *is* a class, and by making C a subclass of SimpleNamespace C inherits 
the initialiser which does the actual work of updating the __dict__ of the C 
instance.

> From docs ...:

> class SimpleNamespace:
>     def __init__(self, **kwargs):
>         self.__dict__.update(kwargs)

The actual implementation is written in C (the language used to implement 
the CPython interpreter), but following the example in the docs you can make 
your own SimpleNamespace...

>>> class MySimpleNamespace:
...     def __init__(self, **parms): self.__dict__.update(parms)
... 
>>> m = MySimpleNamespace(a=1, b=2)
>>> m.a
1
>>> m.b
2

and when you subclass it the subclass inherits the behaviour:

>>> class C(MySimpleNamespace):
...     pass
... 
>>> c = C(x=10, y=20)
>>> c.x
10
>>> c.y
20




From malik.a.rumi at gmail.com  Fri Jun 12 15:39:08 2015
From: malik.a.rumi at gmail.com (Malik Rumi)
Date: Fri, 12 Jun 2015 12:39:08 -0700 (PDT)
Subject: Path, strings, and lines
Message-ID: <56ac4185-3d58-473a-ad40-99d9366a8bb3@googlegroups.com>

 I am trying to find a list of strings in a directory of files. Here is my code:

# -*- coding: utf-8 -*-
import os
import fileinput

s2 = os.listdir('/home/malikarumi/Projects/P5/shortstories')

with open('/home/malikarumi/Projects/P5/list_stories') as f:
    lines = f.readlines()

for line in lines:
     for item in fileinput.input(s2):
         if line in item:
            with open(line + '_list', 'a+') as l:
                l.append(filename(), filelineno(), line)

And here is my error message:

In [44]: %run algo_e3.py
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
/home/malikarumi/Projects/Pipeline/4 Transforms/algo_e3.py in <module>()
      9 
     10 for line in lines:
---> 11      for item in fileinput.input(s2):
     12          if line in item:
     13             with open(line + '_list', 'a+') as l:

/usr/lib/python3.4/fileinput.py in __next__(self)
    261             self._filelineno += 1
    262             return line
--> 263         line = self.readline()
    264         if not line:
    265             raise StopIteration

/usr/lib/python3.4/fileinput.py in readline(self)
    360                         self._file = self._openhook(self._filename, self._mode)
    361                     else:
--> 362                         self._file = open(self._filename, self._mode)
    363         self._buffer = self._file.readlines(self._bufsize)
    364         self._bufindex = 0

FileNotFoundError: [Errno 2] No such file or directory: 'THE LAND OF LOST TOYS~'

In trying to figure out what is wrong. I have run this same code, down to the 'for line in lines', but ending with a print statement, and that works fine. I am also able to get it to print a list of the names of the files in s2. But as soon as I add 'for file in fileinput....' things go sideways. Clearly the files exist, and I gave a full path. 

I see the pointers in the traceback, but I don't really grasp what they are telling me to do. 'line = self.readline()' and 'self._file = open(self._filename...' almost seems like fileinput is reading an instance of itself instead of s2.  I don't think that's right but it would explain why it can't find the file. 

I don't know what the tilde at the end of 'The Land of Lost Toys' is about. I don't think capitalization is an issue. All the files have all cap names to help me debug this from my list of strings. 

And if there is no such file or directory, how does Python know the correct name of at least this one file? My code does not give the names, all that is in the assigned variables. 

Thanks for your help. 


From lac at openend.se  Fri Jun 12 16:11:48 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 12 Jun 2015 22:11:48 +0200
Subject: Writting Dialog to enter preset items from a combo list
In-Reply-To: Message from Wayne Norman <mewthree19@hotmail.com> of "Fri,
 12 Jun 2015 10:39:36 -0700."
 <a6f934b9-d013-4c92-b1f5-e5dbf84886b4@googlegroups.com>
References: <95b7c4d2-6afd-4694-b3c4-6e4f856b4617@googlegroups.com><a6f934b9-d013-4c92-b1f5-e5dbf84886b4@googlegroups.com>
Message-ID: <201506122011.t5CKBm10024847@fido.openend.se>

In a message of Fri, 12 Jun 2015 10:39:36 -0700, Wayne Norman writes:
>ok I have wxFormBuilder as well and so could use some help with working on my project I would explain exactally what I am doing to and one who wishes to aid me with this.
>
>for now I will has it has to do with a minecraft mod if you would like to known more I will put more here
>-- 
>https://mail.python.org/mailman/listinfo/python-list

The wxpython mailing lists are over here.
http://www.wxpython.org/maillist.php

You may have better luck in such places -- I for one wish you well, but
never have used wxpython beyond testing it to see if I liked it better
than tkinter (and I didn't).

Laura


From ian.g.kelly at gmail.com  Fri Jun 12 16:30:34 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 12 Jun 2015 14:30:34 -0600
Subject: Path, strings, and lines
In-Reply-To: <56ac4185-3d58-473a-ad40-99d9366a8bb3@googlegroups.com>
References: <56ac4185-3d58-473a-ad40-99d9366a8bb3@googlegroups.com>
Message-ID: <CALwzidmWmNiHMG+37_mtiHN0Emo6WUGoBnK9bZyoex1g45ib9A@mail.gmail.com>

On Fri, Jun 12, 2015 at 1:39 PM, Malik Rumi <malik.a.rumi at gmail.com> wrote:
>  I am trying to find a list of strings in a directory of files. Here is my code:
>
> # -*- coding: utf-8 -*-
> import os
> import fileinput
>
> s2 = os.listdir('/home/malikarumi/Projects/P5/shortstories')

Note that the filenames that will be returned here are not fully
qualified: you'll just get filename.txt, not
/home/.../shortstories/filename.txt.

> for line in lines:
>      for item in fileinput.input(s2):

fileinput doesn't have the context of the directory that you listed
above, so it's just going to look in the current directory.

>          if line in item:
>             with open(line + '_list', 'a+') as l:
>                 l.append(filename(), filelineno(), line)

Although it's not the problem at hand, I think you'll find that you
need to qualify the filename() and filelineno() function calls with
the fileinput module.

> FileNotFoundError: [Errno 2] No such file or directory: 'THE LAND OF LOST TOYS~'

And here you can see that it's failing to find the file because it's
looking in the wrong directory. You can use the os.path.join function
to add the proper directory path to the filenames that you pass to
fileinput.

> I don't know what the tilde at the end of 'The Land of Lost Toys' is about.

The trailing ~ is a convention used by Emacs (and possibly other
editors) for files that it creates as backups.


From lac at openend.se  Fri Jun 12 16:34:34 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 12 Jun 2015 22:34:34 +0200
Subject: zip as iterator and bad/good practices
In-Reply-To: Message from Fabien <fabien.maussion@gmail.com>
 of "Fri, 12 Jun 2015 17:00:09 +0200." <mles66$sk2$1@speranza.aioe.org>
References: <mles66$sk2$1@speranza.aioe.org>
Message-ID: <201506122034.t5CKYYt0025588@fido.openend.se>

The real problem is removing things from lists when you are iterating
over them, not adding things to the end of lists.

Python 2.7.9 (default, Mar  1 2015, 12:57:24)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> mylist = [1,2,3]
>>> for i in mylist:
...     print i
...     mylist.remove(i)
...
1
3
>>> mylist
[2]

Most people expect 1 2 and 3 to get printed, and mylist to be empty at
the end of this loop.

Laura


From MrJean1 at gmail.com  Fri Jun 12 16:43:42 2015
From: MrJean1 at gmail.com (MrJean1)
Date: Fri, 12 Jun 2015 13:43:42 -0700 (PDT)
Subject: fork/exec & close file descriptors
In-Reply-To: <87lhg26sfx.fsf@elektro.pacujo.net>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <mailman.59.1433253538.13271.python-list@python.org>
 <87lhg26sfx.fsf@elektro.pacujo.net>
Message-ID: <a4327b83-6aaf-4252-9707-fbf796e0fc1e@googlegroups.com>

The subprocess module uses upper bound MAXFD which is defined as

<code>
try:
    MAXFD = os.sysconf("SC_OPEN_MAX")
except:
    MAXFD = 256
</code>

/Jean


From lac at openend.se  Fri Jun 12 16:46:39 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 12 Jun 2015 22:46:39 +0200
Subject: How may I Integrate Python Code with REST
In-Reply-To: Message from subhabrata.banerji@gmail.com of "Fri,
 12 Jun 2015 10:52:19 -0700."
 <02f1571f-a497-4b21-99b5-b39a627bb5ea@googlegroups.com>
References: <02f1571f-a497-4b21-99b5-b39a627bb5ea@googlegroups.com>
Message-ID: <201506122046.t5CKkdoI025960@fido.openend.se>

In a message of Fri, 12 Jun 2015 10:52:19 -0700, subhabrata.banerji at gmail.com w
rites:
>Dear Group,
>
>I wrote a Python code. In the code there are two modules where we may insert data from outside. They are updating some training module and updating index. As a standalone code this is working fine. 
>
>I need to port this code to REST. I tried to learn Flask. My Practice for Flask is okay. I can put,get,delete. 
>
>But how may I proceed I am not getting much idea. I bit new in REST. 
>If any one of the esteemed members may kindly provide an idea how may I proceed? 
>
>Regards,
>Subhabrata Banerjee.  

All you need now is post.  So it looks as if you have  all the bits you
need, but you don't know how to use them to design an API.

See if this helps.
http://blog.miguelgrinberg.com/post/designing-a-restful-api-with-python-and-flask

If not, you are probably at the stage of posting code here.
Here is my code that doesn't work.
Here is what happens when I run it.
This is what I wish it would do instead.

Right now we don't understand what it is that you don't understand, alas.
Code makes the problem obvious.

Laura



From PointedEars at web.de  Fri Jun 12 17:32:31 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Fri, 12 Jun 2015 23:32:31 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de> <lf5r3pmejgh.fsf@ling.helsinki.fi>
 <16299015.icEhCTIHP8@PointedEars.de>
 <mailman.354.1433959416.13271.python-list@python.org>
Message-ID: <1481051.eNy53rHVEB@PointedEars.de>

Ian Kelly wrote:

> The probability of 123456789 and 111111111 are equal. The probability
> of a sequence containing all nine numbers and a sequence containing
> only 1s are *not* equal.

There is a contradiction in that statement.  Can you find it?

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From alister.nospam.ware at ntlworld.com  Fri Jun 12 17:46:22 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Fri, 12 Jun 2015 21:46:22 +0000 (UTC)
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de> <lf5r3pmejgh.fsf@ling.helsinki.fi>
 <16299015.icEhCTIHP8@PointedEars.de>
 <mailman.354.1433959416.13271.python-list@python.org>
 <1481051.eNy53rHVEB@PointedEars.de>
Message-ID: <mlfjve$n5i$1@speranza.aioe.org>

On Fri, 12 Jun 2015 23:32:31 +0200, Thomas 'PointedEars' Lahn wrote:

> Ian Kelly wrote:
> 
>> The probability of 123456789 and 111111111 are equal. The probability
>> of a sequence containing all nine numbers and a sequence containing
>> only 1s are *not* equal.
> 
> There is a contradiction in that statement.  Can you find it?

i am afraid ther is not

there is only one probibility that will give 111111111 there are a huge 
number of combinations that contain all 9 digits

123456789 & 987654321 to name but 2

there are vastly more combinations that contain all 9 digits (9!) even 
though the chances of any one sequence is still the same as 111111111  


-- 
When in a hole, stop diging



From random832 at fastmail.us  Fri Jun 12 17:52:27 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 12 Jun 2015 17:52:27 -0400
Subject: Testing random
In-Reply-To: <1481051.eNy53rHVEB@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de> <lf5r3pmejgh.fsf@ling.helsinki.fi>
 <16299015.icEhCTIHP8@PointedEars.de>
 <mailman.354.1433959416.13271.python-list@python.org>
 <1481051.eNy53rHVEB@PointedEars.de>
Message-ID: <1434145947.1334637.294266905.3AD34266@webmail.messagingengine.com>

On Fri, Jun 12, 2015, at 17:32, Thomas 'PointedEars' Lahn wrote:
> Ian Kelly wrote:
> 
> > The probability of 123456789 and 111111111 are equal. The probability
> > of a sequence containing all nine numbers and a sequence containing
> > only 1s are *not* equal.
> 
> There is a contradiction in that statement.  Can you find it?

There is not.

The probability of 111111111 is ~0.000000002581, given each digit is
chosen from 1 to 9.

The probability of 123456789 is ~0.000000002581.
The probability of 123456798 is ~0.000000002581.
The probability of 123456879 is ~0.000000002581.
The probability of 123456897 is ~0.000000002581.
The probability of 123456978 is ~0.000000002581.
The probability of 123456987 is ~0.000000002581.
The probability of 123456897 is ~0.000000002581.
The probability of 123457689 is ~0.000000002581.
The probability of 123457698 is ~0.000000002581.
And so on, for a total of 362,880 combinations.

The probability of getting one of these combinations is ~0.000936656708

~0.000936656708 is not equal to ~0.000000002581. QED.

Your problem is that you _continually insist_ on interpreting
"probability of a member of this set" as meaning "probability of one
particular member of this set" rather than "probability of any member of
this set".

It's like claiming the probability of getting a digit from 1 to 3 when
you roll a six-sided die is ~0.1667 instead of 0.5, and playing stupid
when people point out why you're wrong. And it's not even defensible as
an honest misunderstanding because the linguistic ambiguity doesn't
exist in the original claim about the limit probability of full
coverage.


From ian.g.kelly at gmail.com  Fri Jun 12 18:00:26 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 12 Jun 2015 16:00:26 -0600
Subject: Testing random
In-Reply-To: <1481051.eNy53rHVEB@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <lf5sia3bakh.fsf@ling.helsinki.fi> <1583276.2lplL8rY5W@PointedEars.de>
 <lf5r3pmejgh.fsf@ling.helsinki.fi> <16299015.icEhCTIHP8@PointedEars.de>
 <mailman.354.1433959416.13271.python-list@python.org>
 <1481051.eNy53rHVEB@PointedEars.de>
Message-ID: <CALwzidm1nZFzKqnmgSsY4YPe0TqxrmF1QUA+ehwjGRUSNB6KMQ@mail.gmail.com>

On Fri, Jun 12, 2015 at 3:32 PM, Thomas 'PointedEars' Lahn
<PointedEars at web.de> wrote:
> Ian Kelly wrote:
>
>> The probability of 123456789 and 111111111 are equal. The probability
>> of a sequence containing all nine numbers and a sequence containing
>> only 1s are *not* equal.
>
> There is a contradiction in that statement.  Can you find it?

Yes. I phrased my statement as if I were addressing a rational
individual, in clear contradiction of the current evidence.

Seriously, if you reject even the statement I made above, in spite of
all the arguments that have been advanced in this thread, in spite of
the fact that this is very easy to demonstrate empirically, then I
don't think there's any fertile ground for discussion here.


From PointedEars at web.de  Fri Jun 12 18:09:18 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sat, 13 Jun 2015 00:09:18 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de> <lf5r3pmejgh.fsf@ling.helsinki.fi>
 <16299015.icEhCTIHP8@PointedEars.de>
 <mailman.354.1433959416.13271.python-list@python.org>
 <1481051.eNy53rHVEB@PointedEars.de>
 <mailman.442.1434146468.13271.python-list@python.org>
Message-ID: <4021290.WxLfFvsQIG@PointedEars.de>

Ian Kelly wrote:

> [?] Thomas 'PointedEars' Lahn [?] wrote:
>> Ian Kelly wrote:
>>> The probability of 123456789 and 111111111 are equal. The probability
>>> of a sequence containing all nine numbers and a sequence containing
>>> only 1s are *not* equal.d
>> There is a contradiction in that statement.  Can you find it?
> 
> Yes. I phrased my statement as if I were addressing a rational
> individual, in clear contradiction of the current evidence.
> 
> Seriously, if you reject even the statement I made above, in spite of
> all the arguments that have been advanced in this thread, in spite of
> the fact that this is very easy to demonstrate empirically, then I
> don't think there's any fertile ground for discussion here.

/Ad hominem/ when out of arguments.  How typical.

Do you deny that ?123456789? *is* ?a sequence containing all nine numbers? 
(digits, really), and that ?111111111? *is* ?a sequence containing only 1s??

Do you deny that therefore your second sentence contradicts the first one?

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From Seymore4Head at Hotmail.invalid  Fri Jun 12 18:20:34 2015
From: Seymore4Head at Hotmail.invalid (Seymore4Head)
Date: Fri, 12 Jun 2015 18:20:34 -0400
Subject: OT ish  Blocking telemarketers
Message-ID: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>

Is there a program what runs on Windows that uses a national blacklist
to block phone calls?


From ian.g.kelly at gmail.com  Fri Jun 12 18:36:39 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 12 Jun 2015 16:36:39 -0600
Subject: Testing random
In-Reply-To: <4021290.WxLfFvsQIG@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de>
 <lf5r3pmejgh.fsf@ling.helsinki.fi>
 <16299015.icEhCTIHP8@PointedEars.de>
 <mailman.354.1433959416.13271.python-list@python.org>
 <1481051.eNy53rHVEB@PointedEars.de>
 <mailman.442.1434146468.13271.python-list@python.org>
 <4021290.WxLfFvsQIG@PointedEars.de>
Message-ID: <CALwzid=ABq1i7TAtfNi-xMHmdofF3Fc0kKm60b75GZpQGtfNew@mail.gmail.com>

On Jun 12, 2015 4:16 PM, "Thomas 'PointedEars' Lahn" <PointedEars at web.de>
wrote:
>
> Ian Kelly wrote:
>
> > [?] Thomas 'PointedEars' Lahn [?] wrote:
> >> Ian Kelly wrote:
> >>> The probability of 123456789 and 111111111 are equal. The probability
> >>> of a sequence containing all nine numbers and a sequence containing
> >>> only 1s are *not* equal.d
> >> There is a contradiction in that statement.  Can you find it?
> >
> > Yes. I phrased my statement as if I were addressing a rational
> > individual, in clear contradiction of the current evidence.
> >
> > Seriously, if you reject even the statement I made above, in spite of
> > all the arguments that have been advanced in this thread, in spite of
> > the fact that this is very easy to demonstrate empirically, then I
> > don't think there's any fertile ground for discussion here.
>
> /Ad hominem/ when out of arguments.  How typical.

No, it's ad hominem on top of the other arguments, all of which you have
failed to refute.

> Do you deny that ?123456789? *is* ?a sequence containing all nine numbers?
> (digits, really), and that ?111111111? *is* ?a sequence containing only
1s??

No.

> Do you deny that therefore your second sentence contradicts the first one?

Yes. There are 9! - 1 *other* sequences that all contribute to "the
probability of a sequence containing all nine numbers".

I also think your explanation betrays that you understood my meaning
perfectly well and rather than engaging in honest discussion chose to
deliberately and obtusely misconstrue it in order to pick it apart in some
puerile form of one-upmanship.

I have no interest in wasting my time engaging with dishonest people.
Welcome to my kill file.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150612/704d70d3/attachment.html>

From sohcahtoa82 at gmail.com  Fri Jun 12 18:55:08 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Fri, 12 Jun 2015 15:55:08 -0700 (PDT)
Subject: Testing random
In-Reply-To: <4021290.WxLfFvsQIG@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <lf5sia3bakh.fsf@ling.helsinki.fi> <1583276.2lplL8rY5W@PointedEars.de>
 <lf5r3pmejgh.fsf@ling.helsinki.fi> <16299015.icEhCTIHP8@PointedEars.de>
 <mailman.354.1433959416.13271.python-list@python.org>
 <1481051.eNy53rHVEB@PointedEars.de>
 <mailman.442.1434146468.13271.python-list@python.org>
 <4021290.WxLfFvsQIG@PointedEars.de>
Message-ID: <bb47ddf4-7ca1-4b6f-affb-50e2f45825df@googlegroups.com>

On Friday, June 12, 2015 at 3:12:26 PM UTC-7, Thomas 'PointedEars' Lahn wrote:
> Ian Kelly wrote:
> 
> > [...] Thomas 'PointedEars' Lahn [...] wrote:
> >> Ian Kelly wrote:
> >>> The probability of 123456789 and 111111111 are equal. The probability
> >>> of a sequence containing all nine numbers and a sequence containing
> >>> only 1s are *not* equal.d
> >> There is a contradiction in that statement.  Can you find it?
> > 
> > Yes. I phrased my statement as if I were addressing a rational
> > individual, in clear contradiction of the current evidence.
> > 
> > Seriously, if you reject even the statement I made above, in spite of
> > all the arguments that have been advanced in this thread, in spite of
> > the fact that this is very easy to demonstrate empirically, then I
> > don't think there's any fertile ground for discussion here.
> 
> /Ad hominem/ when out of arguments.  How typical.
> 
> Do you deny that "123456789" *is* "a sequence containing all nine numbers" 
> (digits, really), and that "111111111" *is* "a sequence containing only 1s"?
> 
> Do you deny that therefore your second sentence contradicts the first one?
> 
> -- 
> PointedEars
> 
> Twitter: @PointedEars2
> Please do not cc me. / Bitte keine Kopien per E-Mail.

I'm struggling to see the contradiction here.

Yes, 123456789 is a sequence containing all nine numbers.  And as someone else said, 123456798 is also a sequence containing all numbers.  987654321 contains all numbers.  There are 9*8*7*6*5*4*3*2*1 (Usually expressed as "9!", and equal to 362,880) distinct ways to order nine numbers.

However, there is only ONE way to order a series of all 1s.

There's no contradiction here.


From random832 at fastmail.us  Fri Jun 12 18:57:44 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 12 Jun 2015 18:57:44 -0400
Subject: Testing random
In-Reply-To: <4021290.WxLfFvsQIG@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de> <lf5r3pmejgh.fsf@ling.helsinki.fi>
 <16299015.icEhCTIHP8@PointedEars.de>
 <mailman.354.1433959416.13271.python-list@python.org>
 <1481051.eNy53rHVEB@PointedEars.de>
 <mailman.442.1434146468.13271.python-list@python.org>
 <4021290.WxLfFvsQIG@PointedEars.de>
Message-ID: <1434149864.3172438.294305857.042D6711@webmail.messagingengine.com>

On Fri, Jun 12, 2015, at 18:09, Thomas 'PointedEars' Lahn wrote:
> Do you deny that ?123456789? *is* ?a sequence containing all nine
> numbers?

Do you deny that "123456798" *is* "a sequence containing all nine
numbers"?

Does this mean that "123456789" *is* "123456798" by the transitive
property? NO.

There are multiple different meanings of the word "is" here, and the
particular one that applies here does not mean it can be substituted for
the more general description in a sentence about the probability of
getting a member of the set of such sequences.


From sohcahtoa82 at gmail.com  Fri Jun 12 18:57:53 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Fri, 12 Jun 2015 15:57:53 -0700 (PDT)
Subject: OT ish  Blocking telemarketers
In-Reply-To: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
References: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
Message-ID: <a80b5330-0e18-4a96-ab3e-be243df48211@googlegroups.com>

On Friday, June 12, 2015 at 3:23:32 PM UTC-7, Seymore4Head wrote:
> Is there a program what runs on Windows that uses a national blacklist
> to block phone calls?

Are you talking about a Windows Phone?  Windows for a PC doesn't make phone calls unless that's a new feature that I don't know about.


From tjreedy at udel.edu  Fri Jun 12 19:08:43 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 12 Jun 2015 19:08:43 -0400
Subject: Get classes from "self.MyClass" to improve subclassability
In-Reply-To: <4d336971-d986-4e21-85a2-ddc8ee995c7e@googlegroups.com>
References: <4d336971-d986-4e21-85a2-ddc8ee995c7e@googlegroups.com>
Message-ID: <mlfor3$7jd$1@ger.gmane.org>

On 6/12/2015 7:12 AM, Thomas G?ttler wrote:
> Here is a snippet from the argparse module:
>
> {{{
>      def parse_known_args(self, args=None, namespace=None):
>          ...
>          # default Namespace built from parser defaults
>          if namespace is None:
>              namespace = Namespace() # < ======= my issue
> }}}
>
> I subclass from the class of the above snippet.
>
> I would like to use a different Namespace class.
>
> if the above could would use
>
>      namespace = self.Namespace()
>
> it would be very easy for me to inject a different Namespace class.

The default arg (None) for the namespace parameter of the 
parse_known_args is an attribute of the function, not of the class or 
instance thereof.  Unless the default Namespace is used elsewhere, this 
seems sensible.

In CPython, at least, and probably in pypy, you can change this 
attribute.  (But AFAIK, this is not guaranteed in all implementations.)

 >>> def f(n = 1): pass

 >>> f.__defaults__
(1,)
 >>> f.__defaults__ = (2,)
 >>> f.__defaults__
(2,)

So the following works

 >>> class C():
	def f(n=1): print(n)
	
 >>> class D(C):
	C.f.__defaults__ = (2,)
	
 >>> D.f()
2

Of course, this may or may not do more than you want.

 >>> C.f()
2

-- 
Terry Jan Reedy




From tjreedy at udel.edu  Fri Jun 12 19:27:44 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 12 Jun 2015 19:27:44 -0400
Subject: zip as iterator and bad/good practices
In-Reply-To: <mles66$sk2$1@speranza.aioe.org>
References: <mles66$sk2$1@speranza.aioe.org>
Message-ID: <mlfpuo$o2j$1@ger.gmane.org>

On 6/12/2015 11:00 AM, Fabien wrote:
> is it a bad practice
> to interactively modify the list you are iterating over?

One needs care.  Appending to the end of the list is OK, unless you 
append a billion items or so ;-)  Appending to the end of a queue while 
*removing* items from the front of the queue, where the queue resizes 
itself at the front as needed, is standard for breadth-first search.  A 
deque.Deque can be used for this.  Depth-first search appends to and 
deletes from the end (or top) of a stack, but this is NOT 
forward-iteration as implemented by Python iterators.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Fri Jun 12 19:43:08 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 12 Jun 2015 19:43:08 -0400
Subject: zip as iterator and bad/good practices
In-Reply-To: <201506122034.t5CKYYt0025588@fido.openend.se>
References: <mles66$sk2$1@speranza.aioe.org>
 <201506122034.t5CKYYt0025588@fido.openend.se>
Message-ID: <mlfqrk$4oo$1@ger.gmane.org>

On 6/12/2015 4:34 PM, Laura Creighton wrote:
> The real problem is removing things from lists when you are iterating
> over them, not adding things to the end of lists.

One needs to iterate backwards.

 >>> ints = [0, 1, 2, 2, 1, 4, 6, 5, 5]

 >>> for i in range(len(ints)-1, -1, -1):
	if ints[i] % 2:
		del ints[i]
	
 >>> ints
[0, 2, 2, 4, 6]

But using a list comp and, if necessary, copying the result back into 
the original list is much easier.

 >>> ints = [0, 1, 2, 2, 1, 4, 6, 5, 5]
 >>> ints[:] = [i for i in ints if not i % 2]
 >>> ints
[0, 2, 2, 4, 6]


-- 
Terry Jan Reedy



From rosuav at gmail.com  Fri Jun 12 19:47:28 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 13 Jun 2015 09:47:28 +1000
Subject: Path, strings, and lines
In-Reply-To: <56ac4185-3d58-473a-ad40-99d9366a8bb3@googlegroups.com>
References: <56ac4185-3d58-473a-ad40-99d9366a8bb3@googlegroups.com>
Message-ID: <CAPTjJmrFrf9XCVnPnKttNNF2ZDk-FZ_GNB74oDBTJV=Vbv0FGg@mail.gmail.com>

On Sat, Jun 13, 2015 at 5:39 AM, Malik Rumi <malik.a.rumi at gmail.com> wrote:
> for line in lines:
>      for item in fileinput.input(s2):
>          if line in item:
>             with open(line + '_list', 'a+') as l:
>                 l.append(filename(), filelineno(), line)

Ian's already answered your actual question, but I'll make one
separate comment. What you have here will open, append to, and close,
the list file for every single line that you find. If you're expecting
to find zero or very few lines, then that's fine, but if you expect
you might find a lot, this will be extremely slow. Much more efficient
would be to open the file once, and write to it every time - just
switch around the nesting a bit:

with open(line + '_list', 'a+') as l:
     for line in lines:
         for item in fileinput.input(s2):
             if line in item:
                l.append(filename(), filelineno(), line)

(Although you may want to rename your open file object, here; "l"
isn't a very useful name at the best of times, so I'd be inclined to
call it "log".)

ChrisA


From Seymore4Head at Hotmail.invalid  Fri Jun 12 20:00:29 2015
From: Seymore4Head at Hotmail.invalid (Seymore4Head)
Date: Fri, 12 Jun 2015 20:00:29 -0400
Subject: OT ish  Blocking telemarketers
References: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
 <a80b5330-0e18-4a96-ab3e-be243df48211@googlegroups.com>
Message-ID: <jismna5pcgirs813si12p4m5ig4ub6bc02@4ax.com>

On Fri, 12 Jun 2015 15:57:53 -0700 (PDT), sohcahtoa82 at gmail.com wrote:

>On Friday, June 12, 2015 at 3:23:32 PM UTC-7, Seymore4Head wrote:
>> Is there a program what runs on Windows that uses a national blacklist
>> to block phone calls?
>
>Are you talking about a Windows Phone?  Windows for a PC doesn't make phone calls unless that's a new feature that I don't know about.

I guess I should have said...........A program that runs on Windows
that uses a modem and a national blacklist to block phone calls.


From sohcahtoa82 at gmail.com  Fri Jun 12 20:02:24 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Fri, 12 Jun 2015 17:02:24 -0700 (PDT)
Subject: zip as iterator and bad/good practices
In-Reply-To: <mailman.447.1434152635.13271.python-list@python.org>
References: <mles66$sk2$1@speranza.aioe.org>
 <201506122034.t5CKYYt0025588@fido.openend.se>
 <mailman.447.1434152635.13271.python-list@python.org>
Message-ID: <e5d750bd-d7e3-4d93-9794-e9f16a4b40bd@googlegroups.com>

On Friday, June 12, 2015 at 4:44:08 PM UTC-7, Terry Reedy wrote:
> On 6/12/2015 4:34 PM, Laura Creighton wrote:
> > The real problem is removing things from lists when you are iterating
> > over them, not adding things to the end of lists.
> 
> One needs to iterate backwards.
> 
>  >>> ints = [0, 1, 2, 2, 1, 4, 6, 5, 5]
> 
>  >>> for i in range(len(ints)-1, -1, -1):
> 	if ints[i] % 2:
> 		del ints[i]
> 	
>  >>> ints
> [0, 2, 2, 4, 6]
> 
> But using a list comp and, if necessary, copying the result back into 
> the original list is much easier.
> 
>  >>> ints = [0, 1, 2, 2, 1, 4, 6, 5, 5]
>  >>> ints[:] = [i for i in ints if not i % 2]
>  >>> ints
> [0, 2, 2, 4, 6]
> 
> 
> -- 
> Terry Jan Reedy

On the second line of your final solution, is there any reason you're using `ints[:]` rather than just `ints`?


From rosuav at gmail.com  Fri Jun 12 20:24:32 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 13 Jun 2015 10:24:32 +1000
Subject: OT ish Blocking telemarketers
In-Reply-To: <jismna5pcgirs813si12p4m5ig4ub6bc02@4ax.com>
References: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
 <a80b5330-0e18-4a96-ab3e-be243df48211@googlegroups.com>
 <jismna5pcgirs813si12p4m5ig4ub6bc02@4ax.com>
Message-ID: <CAPTjJmr=CMSnfRwKup3H0+kLVa-xE8-=npxOKKYdNooWsB5zqg@mail.gmail.com>

On Sat, Jun 13, 2015 at 10:00 AM, Seymore4Head
<Seymore4Head at hotmail.invalid> wrote:
> On Fri, 12 Jun 2015 15:57:53 -0700 (PDT), sohcahtoa82 at gmail.com wrote:
>
>>On Friday, June 12, 2015 at 3:23:32 PM UTC-7, Seymore4Head wrote:
>>> Is there a program what runs on Windows that uses a national blacklist
>>> to block phone calls?
>>
>>Are you talking about a Windows Phone?  Windows for a PC doesn't make phone calls unless that's a new feature that I don't know about.
>
> I guess I should have said...........A program that runs on Windows
> that uses a modem and a national blacklist to block phone calls.

I'm not entirely sure how you'd use a modem to block phone calls. Do
you mean that the computer answers it before the human has to deal
with it? And then what does it do?

My anti-phone-spam policy is much simpler. I don't depend on
caller-id, I don't depend on blacklists... I just don't have a phone.
No matter what the incoming number is, it can't be for me, so I take
no notice. Look, ma! No spam! :)

ChrisA


From rosuav at gmail.com  Fri Jun 12 20:26:49 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 13 Jun 2015 10:26:49 +1000
Subject: zip as iterator and bad/good practices
In-Reply-To: <e5d750bd-d7e3-4d93-9794-e9f16a4b40bd@googlegroups.com>
References: <mles66$sk2$1@speranza.aioe.org>
 <201506122034.t5CKYYt0025588@fido.openend.se>
 <mailman.447.1434152635.13271.python-list@python.org>
 <e5d750bd-d7e3-4d93-9794-e9f16a4b40bd@googlegroups.com>
Message-ID: <CAPTjJmobVX-bPVeJXnWzYbTwZUie-d=YTAktiMdjxeAsMdTA8g@mail.gmail.com>

On Sat, Jun 13, 2015 at 10:02 AM,  <sohcahtoa82 at gmail.com> wrote:
>>  >>> ints = [0, 1, 2, 2, 1, 4, 6, 5, 5]
>>  >>> ints[:] = [i for i in ints if not i % 2]
>>  >>> ints
>> [0, 2, 2, 4, 6]
>>
>>
>> --
>> Terry Jan Reedy
>
> On the second line of your final solution, is there any reason you're using `ints[:]` rather than just `ints`?

If you use "ints = [...]", it rebinds the name ints to the new list.
If you use "ints[:] = [...]", it replaces the entire contents of the
list with the new list. The two are fairly similar if there are no
other references to that list, but the replacement matches the
mutation behaviour of remove().

def just_some(ints):
    ints[:] = [i for i in ints if not i % 2]

ChrisA


From sohcahtoa82 at gmail.com  Fri Jun 12 20:37:32 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Fri, 12 Jun 2015 17:37:32 -0700 (PDT)
Subject: OT ish  Blocking telemarketers
In-Reply-To: <jismna5pcgirs813si12p4m5ig4ub6bc02@4ax.com>
References: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
 <a80b5330-0e18-4a96-ab3e-be243df48211@googlegroups.com>
 <jismna5pcgirs813si12p4m5ig4ub6bc02@4ax.com>
Message-ID: <b949cbe1-19ba-41ed-a81d-f71f91da1477@googlegroups.com>

On Friday, June 12, 2015 at 5:03:25 PM UTC-7, Seymore4Head wrote:
> On Fri, 12 Jun 2015 15:57:53 -0700 (PDT), sohcahtoa82 at gmail.com wrote:
> 
> >On Friday, June 12, 2015 at 3:23:32 PM UTC-7, Seymore4Head wrote:
> >> Is there a program what runs on Windows that uses a national blacklist
> >> to block phone calls?
> >
> >Are you talking about a Windows Phone?  Windows for a PC doesn't make phone calls unless that's a new feature that I don't know about.
> 
> I guess I should have said...........A program that runs on Windows
> that uses a modem and a national blacklist to block phone calls.

On a home telephone, you can't block phone calls "client-side" so-to-speak.  At best, you can have a phone that can choose not to ring based on the number calling, but in my experience, most telemarketers block their caller ID from showing up so that wouldn't work anyways.  Also, any OTHER phones connected to the same line would still ring unless one of phones "answered" the call and then hung up, which is something a modem could do.

Are you in the USA?  Have you checked out the Do Not Call registry? https://www.donotcall.gov/


From sohcahtoa82 at gmail.com  Fri Jun 12 20:39:28 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Fri, 12 Jun 2015 17:39:28 -0700 (PDT)
Subject: zip as iterator and bad/good practices
In-Reply-To: <mailman.450.1434155212.13271.python-list@python.org>
References: <mles66$sk2$1@speranza.aioe.org>
 <201506122034.t5CKYYt0025588@fido.openend.se>
 <mailman.447.1434152635.13271.python-list@python.org>
 <e5d750bd-d7e3-4d93-9794-e9f16a4b40bd@googlegroups.com>
 <mailman.450.1434155212.13271.python-list@python.org>
Message-ID: <df28e29a-cbc9-48b9-bf30-8b1f311848c2@googlegroups.com>

On Friday, June 12, 2015 at 5:27:21 PM UTC-7, Chris Angelico wrote:
> On Sat, Jun 13, 2015 at 10:02 AM,  <sohcahtoa82 at gmail.com> wrote:
> >>  >>> ints = [0, 1, 2, 2, 1, 4, 6, 5, 5]
> >>  >>> ints[:] = [i for i in ints if not i % 2]
> >>  >>> ints
> >> [0, 2, 2, 4, 6]
> >>
> >>
> >> --
> >> Terry Jan Reedy
> >
> > On the second line of your final solution, is there any reason you're using `ints[:]` rather than just `ints`?
> 
> If you use "ints = [...]", it rebinds the name ints to the new list.
> If you use "ints[:] = [...]", it replaces the entire contents of the
> list with the new list. The two are fairly similar if there are no
> other references to that list, but the replacement matches the
> mutation behaviour of remove().
> 
> def just_some(ints):
>     ints[:] = [i for i in ints if not i % 2]
> 
> ChrisA

Ah that makes sense.  Thanks.


From steve+comp.lang.python at pearwood.info  Fri Jun 12 21:25:33 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 13 Jun 2015 01:25:33 GMT
Subject: Passing new fields to an object
References: <mlev91$4ji$1@speranza.aioe.org>
Message-ID: <557b868c$0$11125$c3e8da3@news.astraweb.com>

On Fri, 12 Jun 2015 16:53:08 +0100, Paulo da Silva wrote:

> I would like to do something like this:
> 
> class C:
> 	def __init__(self,**parms):
> 	...
> 
> c=C(f1=1,f2=None)
> 
> I want to have, for the object
> 	self.f1=1
> 	self.f2=None
> 
> for an arbitrary number of parameters.
> 
> What is the best way to achieve this?


Others have suggested that you update the instance dict with the keyword 
parameters:

    self.__dict__.update(parms)


But I suggest that you should be very careful with this technique, 
because it can lead to surprising problems and bugs in your code. If your 
class has methods (and what sort of class doesn't have methods?), this 
will override them and lead to mysterious failures in your code:

instance = C(a=1, b=2, method=3)
# much later
result = instance.method(args)  # will raise exception


You should use SimpleNamespace, as Peter suggests, but *not* subclass it. 
If you subclass it and add methods:

class C(SimpleNamespace):
    def foo(self, arg):
        print("called foo")


then you risk overriding foo method, as above. If you don't add methods, 
there is no need to subclass.

Instead, use composition: your class should *contain* a SimpleNamespace, 
not *be* one:

class C:
    def __init__(self, **param):
        self.ns = SimpleNamespace(param)
    def __getattr__(self, attrname):
        return getattr(self.ns, attrname)
    def foo(self, arg):
        print("called foo")


instance = C(a=1, b=2, foo=3)
# later
instance.foo("x")  # prints "called foo"


The special method __getattr__ only runs if the attribute name is not 
found in the usual way, so the method foo will continue to be found and 
not be overridden by the param foo.



-- 
Steve


From torriem at gmail.com  Fri Jun 12 21:47:18 2015
From: torriem at gmail.com (Michael Torrie)
Date: Fri, 12 Jun 2015 19:47:18 -0600
Subject: OT ish  Blocking telemarketers
In-Reply-To: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
References: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
Message-ID: <557B8BA6.4070505@gmail.com>

On 06/12/2015 04:20 PM, Seymore4Head wrote:
> Is there a program what runs on Windows that uses a national blacklist
> to block phone calls?

I'm sure you could install and use the Asterisk PBX software, and I bet
people have made scripts for it to block calls in this way.  You'll need
to take your question to the asterisk mailing list though.



From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Fri Jun 12 23:29:52 2015
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Sat, 13 Jun 2015 04:29:52 +0100
Subject: Passing new fields to an object
References: <mlev91$4ji$1@speranza.aioe.org>
 <mailman.429.1434125838.13271.python-list@python.org>
 <mlf8mu$ro2$1@speranza.aioe.org>
 <mailman.436.1434136357.13271.python-list@python.org>
Message-ID: <mlg83e$tnr$1@speranza.aioe.org>

On 12-06-2015 20:12, Peter Otten wrote:
> Paulo da Silva wrote:
> 
>> On 12-06-2015 17:17, Peter Otten wrote:
>>> Paulo da Silva wrote:
>>>
>> ...
...

> It *is* a class, and by making C a subclass of SimpleNamespace C inherits 
> the initialiser which does the actual work of updating the __dict__ of the C 
> instance.
> 

OK, OK ...
Somehow I "changed" my mind to think SimpleNamespace as an argument of
__init__ and not as a subclass. I should have looked better before
responding.
Sorry.

Thank you very much.



From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Fri Jun 12 23:33:29 2015
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Sat, 13 Jun 2015 04:33:29 +0100
Subject: Passing new fields to an object
References: <mlev91$4ji$1@speranza.aioe.org>
 <557b868c$0$11125$c3e8da3@news.astraweb.com>
Message-ID: <mlg8a7$u38$2@speranza.aioe.org>

On 13-06-2015 02:25, Steven D'Aprano wrote:
> On Fri, 12 Jun 2015 16:53:08 +0100, Paulo da Silva wrote:
> 
...

> 
> You should use SimpleNamespace, as Peter suggests, but *not* subclass it. 
> If you subclass it and add methods:
> 
> class C(SimpleNamespace):
>     def foo(self, arg):
>         print("called foo")
> 
> 
> then you risk overriding foo method, as above. If you don't add methods, 
> there is no need to subclass.
> 
> Instead, use composition: your class should *contain* a SimpleNamespace, 
> not *be* one:
> 
> class C:
>     def __init__(self, **param):
>         self.ns = SimpleNamespace(param)
>     def __getattr__(self, attrname):
>         return getattr(self.ns, attrname)
>     def foo(self, arg):
>         print("called foo")
> 
> 
> instance = C(a=1, b=2, foo=3)
> # later
> instance.foo("x")  # prints "called foo"
> 
> 
> The special method __getattr__ only runs if the attribute name is not 
> found in the usual way, so the method foo will continue to be found and 
> not be overridden by the param foo.

Always learning!

Thanks a lot.




From Seymore4Head at Hotmail.invalid  Fri Jun 12 23:47:10 2015
From: Seymore4Head at Hotmail.invalid (Seymore4Head)
Date: Fri, 12 Jun 2015 23:47:10 -0400
Subject: OT ish  Blocking telemarketers
References: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
 <a80b5330-0e18-4a96-ab3e-be243df48211@googlegroups.com>
 <jismna5pcgirs813si12p4m5ig4ub6bc02@4ax.com>
 <b949cbe1-19ba-41ed-a81d-f71f91da1477@googlegroups.com>
Message-ID: <0s9nna9srbfrm5k0edppcvr694g7vpka4h@4ax.com>

On Fri, 12 Jun 2015 17:37:32 -0700 (PDT), sohcahtoa82 at gmail.com wrote:

>On Friday, June 12, 2015 at 5:03:25 PM UTC-7, Seymore4Head wrote:
>> On Fri, 12 Jun 2015 15:57:53 -0700 (PDT), sohcahtoa82 at gmail.com wrote:
>> 
>> >On Friday, June 12, 2015 at 3:23:32 PM UTC-7, Seymore4Head wrote:
>> >> Is there a program what runs on Windows that uses a national blacklist
>> >> to block phone calls?
>> >
>> >Are you talking about a Windows Phone?  Windows for a PC doesn't make phone calls unless that's a new feature that I don't know about.
>> 
>> I guess I should have said...........A program that runs on Windows
>> that uses a modem and a national blacklist to block phone calls.
>
>On a home telephone, you can't block phone calls "client-side" so-to-speak.  At best, you can have a phone that can choose not to ring based on the number calling, but in my experience, most telemarketers block their caller ID from showing up so that wouldn't work anyways.  Also, any OTHER phones connected to the same line would still ring unless one of phones "answered" the call and then hung up, which is something a modem could do.
>
>Are you in the USA?  Have you checked out the Do Not Call registry? https://www.donotcall.gov/

Yes I have tried the DNCR.  It didn't help.  The calls are not coming
from the US although the caller ID says they are.


From malik.a.rumi at gmail.com  Sat Jun 13 00:48:02 2015
From: malik.a.rumi at gmail.com (Malik Rumi)
Date: Fri, 12 Jun 2015 21:48:02 -0700 (PDT)
Subject: Path, strings, and lines
In-Reply-To: <mailman.438.1434141083.13271.python-list@python.org>
References: <56ac4185-3d58-473a-ad40-99d9366a8bb3@googlegroups.com>
 <mailman.438.1434141083.13271.python-list@python.org>
Message-ID: <58a13725-92f7-407b-8444-71080ada1538@googlegroups.com>

On Friday, June 12, 2015 at 3:31:36 PM UTC-5, Ian wrote:
> On Fri, Jun 12, 2015 at 1:39 PM, Malik Rumi wrote:
> >  I am trying to find a list of strings in a directory of files. Here is my code:
> >
> > # -*- coding: utf-8 -*-
> > import os
> > import fileinput
> >
> > s2 = os.listdir('/home/malikarumi/Projects/P5/shortstories')
> 
> Note that the filenames that will be returned here are not fully
> qualified: you'll just get filename.txt, not
> /home/.../shortstories/filename.txt.
> 

Yes, that is what I want.

> > for line in lines:
> >      for item in fileinput.input(s2):
> 
> fileinput doesn't have the context of the directory that you listed
> above, so it's just going to look in the current directory.

Can you explain a little more what you mean by fileinput lacking the context of s4?

> 
> >          if line in item:
> >             with open(line + '_list', 'a+') as l:
> >                 l.append(filename(), filelineno(), line)
> 
> Although it's not the problem at hand, I think you'll find that you
> need to qualify the filename() and filelineno() function calls with
> the fileinput module.

By 'qualify', do you mean something like 
l.append(fileinput.filename())?

> 
> > FileNotFoundError: [Errno 2] No such file or directory: 'THE LAND OF LOST TOYS~'
> 
> And here you can see that it's failing to find the file because it's
> looking in the wrong directory. You can use the os.path.join function
> to add the proper directory path to the filenames that you pass to
> fileinput.

I tried new code:

# -*- coding: utf-8 -*-
import os
import fileinput


os.path.join('/Projects/Pipeline/4 Transforms', '/Projects/P5/shortstories/')
s2 = os.listdir('/Projects/P5/shortstories/')
for item in fileinput.input(s2):
     if 'penelope' in item:
        print(item)

But still got the same errors even though the assignment of the path variable seems to have worked:


In [51]: import os

In [52]: path = os.path.join('/Projects/Pipeline/4 Transforms', '/Projects/P5/shortstories')

In [53]: print(path)
/Projects/P5/shortstories

In [54]: %run algo_f3.py
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
/home/malikarumi/Projects/Pipeline/4 Transforms/algo_f3.py in <module>()
      5 
      6 os.path.join('/Projects/Pipeline/4 Transforms', '/Projects/P5/shortstories')
----> 7 s2 = os.listdir('/Projects/P5/shortstories')
      8 for item in fileinput.input(s2):
      9      if 'penelope' in item:

FileNotFoundError: [Errno 2] No such file or directory: '/Projects/P5/shortstories'

In [55]: os.listdir(path)
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-55-9f0bcdc47648> in <module>()
----> 1 os.listdir(path)

FileNotFoundError: [Errno 2] No such file or directory: '/Projects/P5/shortstories'

In [56]: %run algo_f4.py
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
/home/malikarumi/Projects/Pipeline/4 Transforms/algo_f4.py in <module>()
      5 
      6 os.path.join('/Projects/Pipeline/4 Transforms', '/Projects/P5/shortstories/')
----> 7 s2 = os.listdir('/Projects/P5/shortstories/')
      8 for item in fileinput.input(s2):
      9      if 'penelope' in item:

FileNotFoundError: [Errno 2] No such file or directory: '/Projects/P5/shortstories/'

Clearly, I don't understand os.path.join() although I have read about it and seen examples. In both https://www.youtube.com/watch?v=t5uRlE28F54  Python path.join, and listdir 
and in https://courses.cs.washington.edu/courses/cse140/13wi/file-interaction.html they are using a real, existing path to 'create' a path with os.join.path. So my immediate problem is that I don't see why that is necessary if they are dealing with an actual path. Why can't they just use the actual path? 

This is why I thought putting the absolute path in my code would solve this problem, but obviously not. Can you help me get a better grasp of how to use os.path.join, and why my refactored code still does not work? 
thanks.

> 
> > I don't know what the tilde at the end of 'The Land of Lost Toys' is about.
> 
> The trailing ~ is a convention used by Emacs (and possibly other
> editors) for files that it creates as backups.



From malik.a.rumi at gmail.com  Sat Jun 13 00:50:44 2015
From: malik.a.rumi at gmail.com (Malik Rumi)
Date: Fri, 12 Jun 2015 21:50:44 -0700 (PDT)
Subject: Path, strings, and lines
In-Reply-To: <mailman.448.1434152856.13271.python-list@python.org>
References: <56ac4185-3d58-473a-ad40-99d9366a8bb3@googlegroups.com>
 <mailman.448.1434152856.13271.python-list@python.org>
Message-ID: <66b8f307-2ab6-4aa1-ab1e-54548b0fdd06@googlegroups.com>

On Friday, June 12, 2015 at 6:48:18 PM UTC-5, Chris Angelico wrote:
> On Sat, Jun 13, 2015 at 5:39 AM, Malik Rumi wrote:
> > for line in lines:
> >      for item in fileinput.input(s2):
> >          if line in item:
> >             with open(line + '_list', 'a+') as l:
> >                 l.append(filename(), filelineno(), line)
> 
> Ian's already answered your actual question, but I'll make one
> separate comment. What you have here will open, append to, and close,
> the list file for every single line that you find. If you're expecting
> to find zero or very few lines, then that's fine, but if you expect
> you might find a lot, this will be extremely slow. Much more efficient
> would be to open the file once, and write to it every time - just
> switch around the nesting a bit:
> 
> with open(line + '_list', 'a+') as l:
>      for line in lines:
>          for item in fileinput.input(s2):
>              if line in item:
>                 l.append(filename(), filelineno(), line)
> 
> (Although you may want to rename your open file object, here; "l"
> isn't a very useful name at the best of times, so I'd be inclined to
> call it "log".)
> 
> ChrisA

Ok, I'll try that once I get out of my current thicket. ;-)


From jimages123 at gmail.com  Sat Jun 13 01:32:59 2015
From: jimages123 at gmail.com (jimages)
Date: Sat, 13 Jun 2015 13:32:59 +0800
Subject: zip as iterator and bad/good practices
In-Reply-To: <mles66$sk2$1@speranza.aioe.org>
References: <mles66$sk2$1@speranza.aioe.org>
Message-ID: <0EB39D35-E064-41E6-9EF0-FE61928F5F77@gmail.com>



> On Jun 12, 2015, at 11:00 PM, Fabien <fabien.maussion at gmail.com> wrote:
> but that awful bug made me wonder: is it a bad practice to interactively modify the list you are iterating over?
Yes.
I am a newbie. I also have been confused when I read the tutorial. It recommends make a copy before looping. Then I try.
#--------------------------
Test = [1, 2]
For i in Test:
    Test.append(i)
#--------------------------
But when i execute. The script does not end. I know there must something wrong. So I launch debugger and deserve the list after each loop.
And I see:
Loop 1: [ 1, 2, 1]
Loop 2: [ 1, 2, 1, 2]
Loop 3: [ 1, 2, 1, 2, 1]
Loop 4: [ 1, 2, 1, 2, 1, 2]
......
So you can see that loop will *never* end.
So I think you regard the 'i' as a pointer. After execute one loop the pointer repoints to next element , but at the same time you are appending element. So pointer will *never* repoints to the last element.
How to solve?
Change code to this
#--------------------------
Test = [1, 2]
For i in Test[:] :
    Test.append(i)
#--------------------------




From steve+comp.lang.python at pearwood.info  Sat Jun 13 03:17:23 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 13 Jun 2015 07:17:23 GMT
Subject: zip as iterator and bad/good practices
References: <mles66$sk2$1@speranza.aioe.org>
 <mailman.452.1434175078.13271.python-list@python.org>
Message-ID: <557bd903$0$11125$c3e8da3@news.astraweb.com>

On Sat, 13 Jun 2015 13:32:59 +0800, jimages wrote:

> I am a newbie. I also have been confused when I read the tutorial. It
> recommends make a copy before looping. Then I try.
> #--------------------------
> Test = [1, 2]
> For i in Test:
>     Test.append(i)
> #--------------------------

You don't make a copy of Test here. You could try this instead:

Test = [1, 2]
copy_test = Test[:]  # [:] makes a slice copy of the whole list
for i in copy_test:  # iterate over the copy
    Test.append(i)  # and append to the original

print(Test)


But an easier way is:

Test = [1, 2]
Test.extend(Test)
print(Test)


> But when i execute. The script does not end. I know there must something
> wrong. So I launch debugger and deserve the list after each loop. And I
> see:
> Loop 1: [ 1, 2, 1]
> Loop 2: [ 1, 2, 1, 2]
> Loop 3: [ 1, 2, 1, 2, 1]
> Loop 4: [ 1, 2, 1, 2, 1, 2]
> ......
> So you can see that loop will *never* end. So I think you regard the 'i'
> as a pointer.

i is not a pointer. It is just a variable that gets a value from the 
list, the same as:

    # first time through the loop
    i = Test[0]
    # second time through the loop
    i = Test[1]  # the second item


The for loop statement:

    for item in seq: ...

understands sequences, lists, and other iterables, not "item". item is 
just an ordinary variable, nothing special about it. The for statement 
takes the items in seq, one at a time, and assigns them to the variable 
"item". In English:

    for each item in seq ...

or to put it another way:

    get the first item of seq
    assign it to "item"
    process the block
    get the second item of seq
    assign it to "item"
    process the block
    get the third item of seq
    assign it to "item"
    process the block
    ...

and so on, until seq runs out of items. But if you keep appending items 
to the end, it will never run out.


> Change code to this
> #--------------------------
> Test = [1, 2]
> For i in Test[:] :
>     Test.append(i)
> #--------------------------


Yes, this will work.


-- 
Steve


From subhabrata.banerji at gmail.com  Sat Jun 13 03:23:43 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Sat, 13 Jun 2015 00:23:43 -0700 (PDT)
Subject: How may I Integrate Python Code with REST
In-Reply-To: <mailman.440.1434142009.13271.python-list@python.org>
References: <02f1571f-a497-4b21-99b5-b39a627bb5ea@googlegroups.com>
 <mailman.440.1434142009.13271.python-list@python.org>
Message-ID: <1040428a-64d0-4863-9cf0-be7baa5a0a01@googlegroups.com>

On Saturday, June 13, 2015 at 2:17:43 AM UTC+5:30, Laura Creighton wrote:
> In a message of Fri, 12 Jun 2015 10:52:19 -0700,  w
> rites:
> >Dear Group,
> >
> >I wrote a Python code. In the code there are two modules where we may insert data from outside. They are updating some training module and updating index. As a standalone code this is working fine. 
> >
> >I need to port this code to REST. I tried to learn Flask. My Practice for Flask is okay. I can put,get,delete. 
> >
> >But how may I proceed I am not getting much idea. I bit new in REST. 
> >If any one of the esteemed members may kindly provide an idea how may I proceed? 
> >
> >Regards,
> >Subhabrata Banerjee.  
> 
> All you need now is post.  So it looks as if you have  all the bits you
> need, but you don't know how to use them to design an API.
> 
> See if this helps.
> http://blog.miguelgrinberg.com/post/designing-a-restful-api-with-python-and-flask
> 
> If not, you are probably at the stage of posting code here.
> Here is my code that doesn't work.
> Here is what happens when I run it.
> This is what I wish it would do instead.
> 
> Right now we don't understand what it is that you don't understand, alas.
> Code makes the problem obvious.
> 
> Laura

Hi,

Thank you for your post. Yes I can PUT the data in JSON format but how to interact with the code I am thinking on that. My script snippet of Flask is as
follows,
>>> x3=requests.put('http://127.0.0.1:5000/todos/todo1', data={'task': 'We are trying to build API. API seems okay.'})
>>> x4=requests.get('http://127.0.0.1:5000/todos')
>>> x5=x4.text
>>> print x5
{
    "todo1": {
        "task": "We are trying to build API. API seems okay."
    }, 
    "todo2": {
        "task": "Change my brakepads"
    }, 
    "todo3": {
        "task": "Life is okay"
    }
}

Now, as I understand it I have to work out something around this PUT. 
As per my guess, 
1. I should work around this put if takes data from file.
2. The data I would get thus, I have to convert this JSON data (by import json or so), into text or the format my main code understands as input.

Am I thinking fine. If not please suggest. 

There is an interesting error occurs almost every other day port 5000
in 
requests.get('http://127.0.0.1:5000/todos') gives error everyother day.
I have stopped firewall, consulted senior administrators of system group but no error seems there. It works suddenly and does not work suddenly,
if you suggest how to fix this issue. 

I am using Python2.7+ on MS-Windows 7 Professional. I am using Microsoft Security Essentials as AntiVirus. 

Regards,
Subhabrata Banerjee. 


From __peter__ at web.de  Sat Jun 13 03:49:34 2015
From: __peter__ at web.de (Peter Otten)
Date: Sat, 13 Jun 2015 09:49:34 +0200
Subject: Passing new fields to an object
References: <mlev91$4ji$1@speranza.aioe.org>
 <557b868c$0$11125$c3e8da3@news.astraweb.com>
Message-ID: <mlgnai$fsp$1@ger.gmane.org>

Steven D'Aprano wrote:

> On Fri, 12 Jun 2015 16:53:08 +0100, Paulo da Silva wrote:
> 
>> I would like to do something like this:
>> 
>> class C:
>> def __init__(self,**parms):
>> ...
>> 
>> c=C(f1=1,f2=None)
>> 
>> I want to have, for the object
>> self.f1=1
>> self.f2=None
>> 
>> for an arbitrary number of parameters.
>> 
>> What is the best way to achieve this?
> 
> 
> Others have suggested that you update the instance dict with the keyword
> parameters:
> 
>     self.__dict__.update(parms)
> 
> 
> But I suggest that you should be very careful with this technique,
> because it can lead to surprising problems and bugs in your code. If your
> class has methods (and what sort of class doesn't have methods?), this
> will override them and lead to mysterious failures in your code:
> 
> instance = C(a=1, b=2, method=3)
> # much later
> result = instance.method(args)  # will raise exception
> 
> 
> You should use SimpleNamespace, as Peter suggests, but *not* subclass it.
> If you subclass it and add methods:
> 
> class C(SimpleNamespace):
>     def foo(self, arg):
>         print("called foo")
> 
> 
> then you risk overriding foo method, as above. If you don't add methods,
> there is no need to subclass.

I sometimes do it anyway if only to get a meaningful class name.

> Instead, use composition: your class should *contain* a SimpleNamespace,
> not *be* one:
> 
> class C:
>     def __init__(self, **param):
>         self.ns = SimpleNamespace(param)

**param

>     def __getattr__(self, attrname):
>         return getattr(self.ns, attrname)
>     def foo(self, arg):
>         print("called foo")
> 
> 
> instance = C(a=1, b=2, foo=3)
> # later
> instance.foo("x")  # prints "called foo"

>>> c = C(ns=42)
>>> assert c.ns == 42
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

Yes, it's less likely, but now that the OP has two convenient ways to 
produce a name clash I think it's time to repeat that there is one bullet-
proof approach: use a dict ;)

> The special method __getattr__ only runs if the attribute name is not
> found in the usual way, so the method foo will continue to be found and
> not be overridden by the param foo.




From breamoreboy at yahoo.co.uk  Sat Jun 13 03:53:58 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 13 Jun 2015 08:53:58 +0100
Subject: Testing random
In-Reply-To: <CALwzidm1nZFzKqnmgSsY4YPe0TqxrmF1QUA+ehwjGRUSNB6KMQ@mail.gmail.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de> <lf5sia3bakh.fsf@ling.helsinki.fi>
 <1583276.2lplL8rY5W@PointedEars.de> <lf5r3pmejgh.fsf@ling.helsinki.fi>
 <16299015.icEhCTIHP8@PointedEars.de>
 <mailman.354.1433959416.13271.python-list@python.org>
 <1481051.eNy53rHVEB@PointedEars.de>
 <CALwzidm1nZFzKqnmgSsY4YPe0TqxrmF1QUA+ehwjGRUSNB6KMQ@mail.gmail.com>
Message-ID: <mlgnj1$jjq$1@ger.gmane.org>

On 12/06/2015 23:00, Ian Kelly wrote:
> On Fri, Jun 12, 2015 at 3:32 PM, Thomas 'PointedEars' Lahn
> <PointedEars at web.de> wrote:
>> Ian Kelly wrote:
>>
>>> The probability of 123456789 and 111111111 are equal. The probability
>>> of a sequence containing all nine numbers and a sequence containing
>>> only 1s are *not* equal.
>>
>> There is a contradiction in that statement.  Can you find it?
>
> Yes. I phrased my statement as if I were addressing a rational
> individual, in clear contradiction of the current evidence.

Beautifully put, hence +1 QOTW.

>
> Seriously, if you reject even the statement I made above, in spite of
> all the arguments that have been advanced in this thread, in spite of
> the fact that this is very easy to demonstrate empirically, then I
> don't think there's any fertile ground for discussion here.
>

Congratulations Thomas 'PointedEars' Lahn, you've been promoted to my 
dream team.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From oscar.j.benjamin at gmail.com  Sat Jun 13 08:48:45 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 13 Jun 2015 13:48:45 +0100
Subject: zip as iterator and bad/good practices
In-Reply-To: <557bd903$0$11125$c3e8da3@news.astraweb.com>
References: <mles66$sk2$1@speranza.aioe.org>
 <mailman.452.1434175078.13271.python-list@python.org>
 <557bd903$0$11125$c3e8da3@news.astraweb.com>
Message-ID: <CAHVvXxQw7G3fhWEQNEsVq7YY=YN00CFFxW1bpB=4JneFtiv7Gw@mail.gmail.com>

On 13 June 2015 at 08:17, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Sat, 13 Jun 2015 13:32:59 +0800, jimages wrote:
>
>> I am a newbie. I also have been confused when I read the tutorial. It
>> recommends make a copy before looping. Then I try.
>> #--------------------------
>> Test = [1, 2]
>> For i in Test:
>>     Test.append(i)
>> #--------------------------
>
> You don't make a copy of Test here. You could try this instead:
>
> Test = [1, 2]
> copy_test = Test[:]  # [:] makes a slice copy of the whole list
> for i in copy_test:  # iterate over the copy
>     Test.append(i)  # and append to the original
>
> print(Test)
>
>
> But an easier way is:
>
> Test = [1, 2]
> Test.extend(Test)
> print(Test)

I can't see anything in the docs that specify the behaviour that
occurs here. If I change it to

    Test.extend(iter(Test))

then it borks my system in 1s after consuming 8GB of RAM (I recovered
with killall python in the tty).

According to the docs:
"""
list.extend(L)

Extend the list by appending all the items in the given list;
equivalent to a[len(a):] = L.
"""
https://docs.python.org/2/tutorial/datastructures.html#more-on-lists

The alternate form

    Test[len(Test):] = Test

is equivalent but

    Test[len(Test):] = iter(Test)

is not since it doesn't bork my system.

I looked here:
https://docs.python.org/2/library/stdtypes.html#mutable-sequence-types
but I don't see anything that specifies how self-referential slice
assignment should behave.

I checked under pypy and all behaviour is the same but I'm not sure if
this shouldn't be considered implementation-defined or undefined
behaviour. It's not hard to see how a rearrangement of the list.extend
method would lead to a change of behaviour and I can't see that the
current behaviour is really guaranteed by the language and in fact
it's inconsistent with the docs for list.extend.

As an aside they say that pypy is fast but it took about 10 times
longer than cpython to bork my system. :)

--
Oscar


From hamilton at nothere.com  Sat Jun 13 09:08:10 2015
From: hamilton at nothere.com (hamilton)
Date: Sat, 13 Jun 2015 07:08:10 -0600
Subject: OT ish  Blocking telemarketers
In-Reply-To: <0s9nna9srbfrm5k0edppcvr694g7vpka4h@4ax.com>
References: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
 <a80b5330-0e18-4a96-ab3e-be243df48211@googlegroups.com>
 <jismna5pcgirs813si12p4m5ig4ub6bc02@4ax.com>
 <b949cbe1-19ba-41ed-a81d-f71f91da1477@googlegroups.com>
 <0s9nna9srbfrm5k0edppcvr694g7vpka4h@4ax.com>
Message-ID: <mlh9ts$lqj$1@dont-email.me>

On 6/12/2015 9:47 PM, Seymore4Head wrote:

> Yes I have tried the DNCR.  It didn't help.  The calls are not coming
> from the US although the caller ID says they are.
>

So you want to "block" calls from a faked number ?!?

( do you have a good program to select lotto numbers?? :-)

On my cell phone, I have my ringer turned off and it only vibrates, so I 
get a chance to see the number calling.

If the number has already been logged into my phone, then it has a name 
attached to it.

If no name is displayed, I do not answer. I will google the number to 
see if that number has a history. ( see http://800notes.com )

If you still have a landline, check with your provider to see if they 
offer call blocking.

Years ago, USWEST had call blocking, but a limited amount of numbers 
could be entered. ( after talking with them about this, they told me 
that they had limited space per home phone number to store blocked 
numbers. )

DNCR is NOT the enemy, SCAMmers use what ever number they think they can 
get away with to make bogus calls.

Until the NSA makes their data available to the public, SCAM callers 
will continue.






From zhouwtlord at gmail.com  Sat Jun 13 09:17:37 2015
From: zhouwtlord at gmail.com (zhou weitao)
Date: Sat, 13 Jun 2015 21:17:37 +0800
Subject: Trying to configure Apache and Python 2.7 on Red Hat I get 403
 Forbidden
In-Reply-To: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>
References: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>
Message-ID: <CABD09x9JQrb0dUXf7JNbqJT7OsDDgM6mftU9PcBU8YFP4PXdvQ@mail.gmail.com>

selinux is causing this, I guess. Please try run *setenforce 1* to bypass
it firstly. If it works then google the related configuration.


from ex-redhatter

2015-06-12 20:51 GMT+08:00 N?stor Bosc?n <nestorjb at gmail.com>:

> Hi
>
> I've been trying to configure Apache and Python 2.7 on Red Hat. I've tried
> the different configurations i've seen on the web, I've given chmod -R 777
> tu my python code but I still get 403 Forbidden and I don't get any errors.
>
> Regards,
>
> N?stor
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150613/10883343/attachment.html>

From alister.nospam.ware at ntlworld.com  Sat Jun 13 10:42:33 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Sat, 13 Jun 2015 14:42:33 +0000 (UTC)
Subject: OT ish  Blocking telemarketers
References: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
 <mailman.451.1434160045.13271.python-list@python.org>
Message-ID: <mlhfgp$kml$1@speranza.aioe.org>

On Fri, 12 Jun 2015 19:47:18 -0600, Michael Torrie wrote:

> On 06/12/2015 04:20 PM, Seymore4Head wrote:
>> Is there a program what runs on Windows that uses a national blacklist
>> to block phone calls?
> 
> I'm sure you could install and use the Asterisk PBX software, and I bet
> people have made scripts for it to block calls in this way.  You'll need
> to take your question to the asterisk mailing list though.

& you will still be wasting your time.
the scumbags change numbers regularly & will also withhold it

it is like fighting the hydra, for every 1 you block 2 more appear 


-- 
Chemistry is applied theology.
		-- Augustus Stanley Owsley III


From torriem at gmail.com  Sat Jun 13 10:46:48 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sat, 13 Jun 2015 08:46:48 -0600
Subject: OT ish  Blocking telemarketers
In-Reply-To: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
References: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
Message-ID: <557C4258.6020902@gmail.com>

On 06/12/2015 04:20 PM, Seymore4Head wrote:
> Is there a program what runs on Windows that uses a national blacklist
> to block phone calls?

You could port your number into Google Voice and then forward that to
your cell phone or land line.  You can screen all calls if you want
(direct them to voicemail and listen in), and you can block numbers as
spam.



From torriem at gmail.com  Sat Jun 13 10:53:35 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sat, 13 Jun 2015 08:53:35 -0600
Subject: OT ish  Blocking telemarketers
In-Reply-To: <mlhfgp$kml$1@speranza.aioe.org>
References: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
 <mailman.451.1434160045.13271.python-list@python.org>
 <mlhfgp$kml$1@speranza.aioe.org>
Message-ID: <557C43EF.7080307@gmail.com>

On 06/13/2015 08:42 AM, alister wrote:
> On Fri, 12 Jun 2015 19:47:18 -0600, Michael Torrie wrote:
> 
>> On 06/12/2015 04:20 PM, Seymore4Head wrote:
>>> Is there a program what runs on Windows that uses a national blacklist
>>> to block phone calls?
>>
>> I'm sure you could install and use the Asterisk PBX software, and I bet
>> people have made scripts for it to block calls in this way.  You'll need
>> to take your question to the asterisk mailing list though.
> 
> & you will still be wasting your time.
> the scumbags change numbers regularly & will also withhold it
> 
> it is like fighting the hydra, for every 1 you block 2 more appear 

Sure but with a proper PBX, you can stop almost all telemarketers right
at the door with a touch-tone menu.  They don't have the time (money) to
wait if you don't answer right away.



From steve+comp.lang.python at pearwood.info  Sat Jun 13 12:16:29 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 13 Jun 2015 16:16:29 GMT
Subject: zip as iterator and bad/good practices
References: <mles66$sk2$1@speranza.aioe.org>
 <mailman.452.1434175078.13271.python-list@python.org>
 <557bd903$0$11125$c3e8da3@news.astraweb.com>
 <mailman.455.1434199754.13271.python-list@python.org>
Message-ID: <557c575d$0$11128$c3e8da3@news.astraweb.com>

On Sat, 13 Jun 2015 13:48:45 +0100, Oscar Benjamin wrote:

> On 13 June 2015 at 08:17, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:

>> But an easier way is:
>>
>> Test = [1, 2]
>> Test.extend(Test)
>> print(Test)
> 
> I can't see anything in the docs that specify the behaviour that occurs
> here. 

Neither do I, but there is a test for it:

        a.extend(a)
        self.assertEqual(a, self.type2test([0, 0, 1, 0, 0, 1]))

https://hg.python.org/cpython/file/a985b6455fde/Lib/test/list_tests.py

> If I change it to
> 
>     Test.extend(iter(Test))
> 
> then it borks my system in 1s after consuming 8GB of RAM (I recovered
> with killall python in the tty).

The reason that fails should be obvious: as new items keep getting added 
to Test, the iterator likewise sees more items to iterate over. I don't 
know if this is documented, but you can see what happens here:

py> L = [10, 20]
py> it = iter(L)
py> L.append(next(it)); print L
[10, 20, 10]
py> L.append(next(it)); print L
[10, 20, 10, 20]
py> L.append(next(it)); print L
[10, 20, 10, 20, 10]
py> L.append(next(it)); print L
[10, 20, 10, 20, 10, 20]


So as Test.extend tries to iterate over iter(Test), it just keeps growing 
as more items are added to Test.


-- 
Steven D'Aprano


From python at mrabarnett.plus.com  Sat Jun 13 14:25:17 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 13 Jun 2015 19:25:17 +0100
Subject: Path, strings, and lines
In-Reply-To: <58a13725-92f7-407b-8444-71080ada1538@googlegroups.com>
References: <56ac4185-3d58-473a-ad40-99d9366a8bb3@googlegroups.com>
 <mailman.438.1434141083.13271.python-list@python.org>
 <58a13725-92f7-407b-8444-71080ada1538@googlegroups.com>
Message-ID: <557C758D.3090207@mrabarnett.plus.com>

On 2015-06-13 05:48, Malik Rumi wrote:
> On Friday, June 12, 2015 at 3:31:36 PM UTC-5, Ian wrote:
>> On Fri, Jun 12, 2015 at 1:39 PM, Malik Rumi wrote:
>> >  I am trying to find a list of strings in a directory of files. Here is my code:
>> >
>> > # -*- coding: utf-8 -*-
>> > import os
>> > import fileinput
>> >
>> > s2 = os.listdir('/home/malikarumi/Projects/P5/shortstories')
>>
>> Note that the filenames that will be returned here are not fully
>> qualified: you'll just get filename.txt, not
>> /home/.../shortstories/filename.txt.
>>
>
> Yes, that is what I want.
>
>> > for line in lines:
>> >      for item in fileinput.input(s2):
>>
>> fileinput doesn't have the context of the directory that you listed
>> above, so it's just going to look in the current directory.
>
> Can you explain a little more what you mean by fileinput lacking the context of s4?
>
listdir returns the names of the files that are in the folder, not
their paths.

If you give fileinput only the names of the files, it'll assume they're
in the current folder (directory), which they (probably) aren't. You
need to give fileinput the complete _paths_ of the files, not just
their names.

>>
>> >          if line in item:
>> >             with open(line + '_list', 'a+') as l:
>> >                 l.append(filename(), filelineno(), line)
>>
>> Although it's not the problem at hand, I think you'll find that you
>> need to qualify the filename() and filelineno() function calls with
>> the fileinput module.
>
> By 'qualify', do you mean something like
> l.append(fileinput.filename())?
>
>>
>> > FileNotFoundError: [Errno 2] No such file or directory: 'THE LAND OF LOST TOYS~'
>>
>> And here you can see that it's failing to find the file because it's
>> looking in the wrong directory. You can use the os.path.join function
>> to add the proper directory path to the filenames that you pass to
>> fileinput.
>
> I tried new code:
>
> # -*- coding: utf-8 -*-
> import os
> import fileinput
>
>
os.join _returns its result.

> os.path.join('/Projects/Pipeline/4 Transforms', '/Projects/P5/shortstories/')
> s2 = os.listdir('/Projects/P5/shortstories/')

At this point, s2 contains a list of _names_.

You pass those names to fileinput.input, but where are they? In which
folder? It assumes they're in the current folder (directory), but
they're not!

> for item in fileinput.input(s2):
>       if 'penelope' in item:
>          print(item)
>
> But still got the same errors even though the assignment of the path variable seems to have worked:
>
[snip]

Try this:

     filenames = os.listdir('/Projects/P5/shortstories/')
     paths = [os.join('/Projects/P5/shortstories/', name) for name in names]
     for item in fileinput.input(paths):



From chris at cdreimer.com  Sat Jun 13 15:10:02 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sat, 13 Jun 2015 12:10:02 -0700
Subject: Is there a utility to tally function calls from other files?
Message-ID: <557C800A.6010107@cdreimer.com>

Greetings,

I'm converting 101 BASIC games from an old book 
(http://www.atariarchives.org/basicgames/) into Python scripts. After a 
dozen conversions, I've created a helper file for common functions 
(i.e., pick_card(), replay_game() and roll_dice()). The functions for 
card and dice games are obvious candidates for the helper file, but 
other functions that might be one offs that shouldn't be included.

Is there utility that will take the function names from the helper file 
and scan the other files to tally up how many times a particular 
function got called?

Or do I need to write a script?

Thanks,

Chris R.


From alister.nospam.ware at ntlworld.com  Sat Jun 13 15:12:28 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Sat, 13 Jun 2015 19:12:28 +0000 (UTC)
Subject: OT ish  Blocking telemarketers
References: <5kmmnapcrvm27vua2lt9to5q12ibm3d5of@4ax.com>
 <mailman.451.1434160045.13271.python-list@python.org>
 <mlhfgp$kml$1@speranza.aioe.org>
 <mailman.457.1434207220.13271.python-list@python.org>
Message-ID: <mlhvas$rau$1@speranza.aioe.org>

On Sat, 13 Jun 2015 08:53:35 -0600, Michael Torrie wrote:

> On 06/13/2015 08:42 AM, alister wrote:
>> On Fri, 12 Jun 2015 19:47:18 -0600, Michael Torrie wrote:
>> 
>>> On 06/12/2015 04:20 PM, Seymore4Head wrote:
>>>> Is there a program what runs on Windows that uses a national
>>>> blacklist to block phone calls?
>>>
>>> I'm sure you could install and use the Asterisk PBX software, and I
>>> bet people have made scripts for it to block calls in this way. 
>>> You'll need to take your question to the asterisk mailing list though.
>> 
>> & you will still be wasting your time.
>> the scumbags change numbers regularly & will also withhold it
>> 
>> it is like fighting the hydra, for every 1 you block 2 more appear
> 
> Sure but with a proper PBX, you can stop almost all telemarketers right
> at the door with a touch-tone menu.  They don't have the time (money) to
> wait if you don't answer right away.

I have considered that approach on many occasions but it would make life 
difficult for genuine callers.

Now I just play with them to waste there time
you would not believe how many are stupid enough to bite when you say "I 
don't deal with disreputable companies" :-)

the current record is 15 minutes for one wally who spent 15 minutes 
listening to silence while i left the phone off the hook in another room 
( & i had even told him that was what i was about to do)

that was 15 minutes he was not bothering someone else.
 




-- 
<Palisade> how are we going to pronounce '00 or '01 or '02 and so on?
<Deek> Say goodbye to the nineties, say hello to the naughties. :)


From g.rodola at gmail.com  Sat Jun 13 15:28:33 2015
From: g.rodola at gmail.com (Giampaolo Rodola')
Date: Sat, 13 Jun 2015 21:28:33 +0200
Subject: ANN: psutil 3.0.0 is out
Message-ID: <CAFYqXL97RXTWqxLmBNuL_o2eyK10a4YH9dTdE9=dKb0meTkuuQ@mail.gmail.com>

Hi there folks,
I'm pleased to announce release 3.0.0 release of psutil:
http://grodola.blogspot.com/2015/06/psutil-30.html

=== About ===

psutil (python system and process utilities) is a cross-platform library
for retrieving information on running processes and system utilization
(CPU, memory, disks, network) in Python. It is useful mainly for system
monitoring, profiling and limiting process resources and management of
running processes. It implements many functionalities offered by command
line tools such as: ps, top, lsof, netstat, ifconfig, who, df, kill, free,
nice, ionice, iostat, iotop, uptime, pidof, tty, taskset, pmap. It
currently supports Linux, Windows, OSX, FreeBSD and Sun Solaris, both
32-bit and 64-bit architectures, with Python versions from 2.4 to 3.4. Pypi
is also known to work.

What changed
===========

See detailed blog post:
http://grodola.blogspot.com/2015/06/psutil-30.html


Links
====

* Home page: https://github.com/giampaolo/psutil
* Downloads:
https://pypi.python.org/pypi?:action=display&name=psutil#downloads
* Documentation: http://pythonhosted.org/psutil/


-- 
Giampaolo - http://grodola.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150613/6a797c2c/attachment.html>

From rosuav at gmail.com  Sat Jun 13 15:31:22 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 14 Jun 2015 05:31:22 +1000
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: <557C800A.6010107@cdreimer.com>
References: <557C800A.6010107@cdreimer.com>
Message-ID: <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>

On Sun, Jun 14, 2015 at 5:10 AM, C.D. Reimer <chris at cdreimer.com> wrote:
> I'm converting 101 BASIC games from an old book
> (http://www.atariarchives.org/basicgames/) into Python scripts. After a
> dozen conversions, I've created a helper file for common functions (i.e.,
> pick_card(), replay_game() and roll_dice()). The functions for card and dice
> games are obvious candidates for the helper file, but other functions that
> might be one offs that shouldn't be included.
>
> Is there utility that will take the function names from the helper file and
> scan the other files to tally up how many times a particular function got
> called?

Can you guarantee that your function names are absolutely unique text
strings? If so, grep can do what you want. I don't have a huge Python
project to demo this on, but grep is language-agnostic.

rosuav at sikorsky:~/Gypsum$ grep gtksignal *.pike plugins/*.pike
globals.pike://Usage:
gtksignal(some_object,"some_signal",handler,arg,arg,arg) --> save that
object.
globals.pike:class gtksignal(object obj)
globals.pike: mi->signals=({gtksignal(mi->menuitem,"activate",menu_clicked)});
globals.pike: mi->signals=({gtksignal(mi->menuitem,"activate",menu_clicked)});
globals.pike: //a gtksignal object or the integer 0, though as of
20150103 nothing depends on this.
globals.pike: gtksignal(win->mainwindow,"delete_event",closewindow),
globals.pike: win->stock_close &&
gtksignal(win->stock_close,"clicked",closewindow),
globals.pike: win->signals+=({gtksignal(obj,parts[i+1..]*"_",this[key],arg,UNDEFINED,b4)});
globals.pike: if (actionbtn)
win->signals+=({gtksignal(win->pb_action,"clicked",action_callback)});
globals.pike: statustxt->signals=({gtksignal(statustxt->evbox,"button_press_event",mousedown)});
window.pike: //an additional argument to signal_connect/gtksignal
would provide that value here)
plugins/hpgraph.pike:
win->signals+=({gtksignal(win["color"+j+i],"clicked",update_color,i)});
plugins/hpgraph.pike:
info->signals=({gtksignal(info->evbox,"button_press_event",mousedown)});
plugins/rmimport.pike:
gtksignal(win->pb_selectall,"clicked",pb_select_click,1), //Same
handler for these, just an arg
plugins/rmimport.pike:
gtksignal(win->pb_selectnone,"clicked",pb_select_click,0),
//Technically, since the default is 0, I could call this
sig_pb_selectnone_clicked, but this way is clearer.
plugins/search.pike:
statustxt->signals=({gtksignal(statustxt->ef,"key_press_event",keypress,0,UNDEFINED,1)});
plugins/zoneinfo.pike:
win->signals+=({gtksignal(win[pfx+"_"+sfx],"changed",pfx=="Thresh"?convert_th:convert_rl,pfx)});

Each of those lines is one usage of the gtksignal() class/function.
One of them is actually a comment (in window.pike), and the first two
at the top of globals.pike are the class definition, with an
associated usage comment, but all the rest are actual uses. As long as
you can depend on names like roll_dice not occurring anywhere else,
this is a great way to find out what you need to know.

If you _can't_ rely on that, it's a bit harder. You could possibly
append an open parenthesis to it, to require classic function-call
syntax, but that may not necessarily be enough (depends how strict
your style guide is). You can do lexical analysis of a Python program
to see where the name is used in a function call, which might be
better. Or you could instrument the function to find out where it's
being called from, and then run every program and see how many of them
call it - something like this:

def pick_card():
    import __main__
    print("Called from %s"%__main__.__file__)
    # rest of pick_card's definition

Log that to a file instead of the screen, and you have yourself a
guaranteed log that shows which programs use that function. Of course,
that depends on actually running the program far enough to get to the
call, which might require interaction.

Depending on your requirements, it could be anywhere from easy to
hard. Good luck :)

ChrisA


From chris at cdreimer.com  Sat Jun 13 15:53:43 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sat, 13 Jun 2015 12:53:43 -0700
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
Message-ID: <557C8A47.8080608@cdreimer.com>

On 6/13/2015 12:31 PM, Chris Angelico wrote:
> Depending on your requirements, it could be anywhere from easy to
> hard. Good luck:)

I don't have  grep on my Windows machine. Writing a script might be easier.

Each file has a import statement for the helper file:

     from bg_helper import replay_game, roll_dice

I just need to grab the function names from the helper file, walk the 
directory, find the import line from each file, and parse each line for 
the function names to build the tally.

Thanks,

Chris R.



From rosuav at gmail.com  Sat Jun 13 15:58:28 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 14 Jun 2015 05:58:28 +1000
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: <557C8A47.8080608@cdreimer.com>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com>
Message-ID: <CAPTjJmozy+5Kua_ex3B4St+CfvD_7vVn3egNxJNzVXS3uWps-w@mail.gmail.com>

On Sun, Jun 14, 2015 at 5:53 AM, C.D. Reimer <chris at cdreimer.com> wrote:
> On 6/13/2015 12:31 PM, Chris Angelico wrote:
>>
>> Depending on your requirements, it could be anywhere from easy to
>> hard. Good luck:)
>
>
> I don't have  grep on my Windows machine. Writing a script might be easier.
>
> Each file has a import statement for the helper file:
>
>     from bg_helper import replay_game, roll_dice
>
> I just need to grab the function names from the helper file, walk the
> directory, find the import line from each file, and parse each line for the
> function names to build the tally.

Hmm, I think the Windows 'find' command can do the same sort of job.
Though it's not hard to grab a Windows port of grep and use that.
Should be easier than writing your own script.

ChrisA


From tjreedy at udel.edu  Sat Jun 13 16:02:37 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 13 Jun 2015 16:02:37 -0400
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: <557C8A47.8080608@cdreimer.com>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com>
Message-ID: <mli2a5$vvf$1@ger.gmane.org>

On 6/13/2015 3:53 PM, C.D. Reimer wrote:
> On 6/13/2015 12:31 PM, Chris Angelico wrote:
>> Depending on your requirements, it could be anywhere from easy to
>> hard. Good luck:)
>
> I don't have  grep on my Windows machine. Writing a script might be easier.

If you have a normal Python installation, you do.  Idle's Find in Files 
is a grep function with a gui interface.

-- 
Terry Jan Reedy



From lac at openend.se  Sat Jun 13 16:34:25 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 13 Jun 2015 22:34:25 +0200
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: Message from "C.D. Reimer" <chris@cdreimer.com>
 of "Sat, 13 Jun 2015 12:53:43 -0700." <557C8A47.8080608@cdreimer.com>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com><557C8A47.8080608@cdreimer.com>
Message-ID: <201506132034.t5DKYP2i025897@fido.openend.se>

In a message of Sat, 13 Jun 2015 12:53:43 -0700, "C.D. Reimer" writes:
>On 6/13/2015 12:31 PM, Chris Angelico wrote:
>> Depending on your requirements, it could be anywhere from easy to
>> hard. Good luck:)
>
>I don't have  grep on my Windows machine. Writing a script might be easier.

Does Idle run?

Because Idle's 'Find in Files' is grep with a gui around the outside.

(And if it doesn't, getting a python from python.org should fix that
particular problem.)

Laura


From chris at cdreimer.com  Sat Jun 13 16:41:07 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sat, 13 Jun 2015 13:41:07 -0700
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: <CAPTjJmozy+5Kua_ex3B4St+CfvD_7vVn3egNxJNzVXS3uWps-w@mail.gmail.com>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com>
 <CAPTjJmozy+5Kua_ex3B4St+CfvD_7vVn3egNxJNzVXS3uWps-w@mail.gmail.com>
Message-ID: <557C9563.6070405@cdreimer.com>

On 6/13/2015 12:58 PM, Chris Angelico wrote:
> Hmm, I think the Windows 'find' command can do the same sort of job.
> Though it's not hard to grab a Windows port of grep and use that.
> Should be easier than writing your own script.

Writing the script would be easier for me since I'll be cobbling 
together several scripts I've used before.

Thanks,

Chris R.


From chris at cdreimer.com  Sat Jun 13 16:45:10 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sat, 13 Jun 2015 13:45:10 -0700
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: <mli2a5$vvf$1@ger.gmane.org>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com> <mli2a5$vvf$1@ger.gmane.org>
Message-ID: <557C9656.5030205@cdreimer.com>

On 6/13/2015 1:02 PM, Terry Reedy wrote:
> If you have a normal Python installation, you do.  Idle's Find in 
> Files is a grep function with a gui interface.

I have Notepad++ on Windows. The find function for that program does the 
same thing. If I have a working Mac, Text Wrangler does the same thing 
as well.

Thanks,

Chris


From lac at openend.se  Sat Jun 13 16:59:33 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 13 Jun 2015 22:59:33 +0200
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: Message from "C.D. Reimer" <chris@cdreimer.com>
 of "Sat, 13 Jun 2015 13:45:10 -0700." <557C9656.5030205@cdreimer.com>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com>
 <mli2a5$vvf$1@ger.gmane.org><557C9656.5030205@cdreimer.com>
Message-ID: <201506132059.t5DKxXEF026509@fido.openend.se>

In a message of Sat, 13 Jun 2015 13:45:10 -0700, "C.D. Reimer" writes:
>On 6/13/2015 1:02 PM, Terry Reedy wrote:
>> If you have a normal Python installation, you do.  Idle's Find in 
>> Files is a grep function with a gui interface.
>
>I have Notepad++ on Windows. The find function for that program does the 
>same thing. If I have a working Mac, Text Wrangler does the same thing 
>as well.
>
>Thanks,
>
>Chris

Idle is written in pure python.  Steal from:
https://hg.python.org/cpython/file/74d182cf0187/Lib/idlelib/GrepDialog.py
if you do not have a copy locally.

Laura




From chris at cdreimer.com  Sat Jun 13 17:09:48 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sat, 13 Jun 2015 14:09:48 -0700
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: <201506132059.t5DKxXEF026509@fido.openend.se>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com>
 <mli2a5$vvf$1@ger.gmane.org><557C9656.5030205@cdreimer.com>
 <201506132059.t5DKxXEF026509@fido.openend.se>
Message-ID: <557C9C1C.4000801@cdreimer.com>

On 6/13/2015 1:59 PM, Laura Creighton wrote:
> Idle is written in pure python.  Steal from:
> https://hg.python.org/cpython/file/74d182cf0187/Lib/idlelib/GrepDialog.py
> if you do not have a copy locally.

I've been using Notepad++ (Windows) and TextWrangler (Mac) for over a 
decade now. Idle may be pure Python, but that's not enough for me to 
switch. :)

Thanks,

Chris R.


From breamoreboy at yahoo.co.uk  Sat Jun 13 17:30:15 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 13 Jun 2015 22:30:15 +0100
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: <557C8A47.8080608@cdreimer.com>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com>
Message-ID: <mli7di$g3j$1@ger.gmane.org>

On 13/06/2015 20:53, C.D. Reimer wrote:
> On 6/13/2015 12:31 PM, Chris Angelico wrote:
>> Depending on your requirements, it could be anywhere from easy to
>> hard. Good luck:)
>
> I don't have  grep on my Windows machine. Writing a script might be easier.
>

http://getgnuwin32.sourceforge.net/ might come in handy for you one day.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From steve+comp.lang.python at pearwood.info  Sat Jun 13 21:59:10 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 14 Jun 2015 01:59:10 GMT
Subject: Is there a utility to tally function calls from other files?
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com> <mli2a5$vvf$1@ger.gmane.org>
 <557C9656.5030205@cdreimer.com>
 <201506132059.t5DKxXEF026509@fido.openend.se>
 <mailman.469.1434229798.13271.python-list@python.org>
Message-ID: <557cdfee$0$11128$c3e8da3@news.astraweb.com>

On Sat, 13 Jun 2015 14:09:48 -0700, C.D. Reimer wrote:

> On 6/13/2015 1:59 PM, Laura Creighton wrote:
>> Idle is written in pure python.  Steal from:
>> https://hg.python.org/cpython/file/74d182cf0187/Lib/idlelib/
GrepDialog.py
>> if you do not have a copy locally.
> 
> I've been using Notepad++ (Windows) and TextWrangler (Mac) for over a
> decade now. Idle may be pure Python, but that's not enough for me to
> switch. :)


You don't have to switch for good. You can fire up IDLE to do this one 
job, then never use it again.



-- 
Steven D'Aprano


From lac at openend.se  Sat Jun 13 22:11:59 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 14 Jun 2015 04:11:59 +0200
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: Message from "Steven D'Aprano"
 <steve+comp.lang.python@pearwood.info>
 of "14 Jun 2015 01:59:10 +0000." <557cdfee$0$11128$c3e8da3@news.astraweb.com>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com> <mli2a5$vvf$1@ger.gmane.org>
 <557C9656.5030205@cdreimer.com> <201506132059.t5DKxXEF026509@fido.openend.se>
 <mailman.469.1434229798.13271.python-list@python.org><557cdfee$0$11128$c3e8da3@news.astraweb.com>
Message-ID: <201506140211.t5E2BxAh000784@fido.openend.se>

In a message of 14 Jun 2015 01:59:10 +0000, "Steven D'Aprano" writes:
>On Sat, 13 Jun 2015 14:09:48 -0700, C.D. Reimer wrote:
>
>> On 6/13/2015 1:59 PM, Laura Creighton wrote:
>>> Idle is written in pure python.  Steal from:
>>> https://hg.python.org/cpython/file/74d182cf0187/Lib/idlelib/
>GrepDialog.py
>>> if you do not have a copy locally.
>> 
>> I've been using Notepad++ (Windows) and TextWrangler (Mac) for over a
>> decade now. Idle may be pure Python, but that's not enough for me to
>> switch. :)
>
>
>You don't have to switch for good. You can fire up IDLE to do this one 
>job, then never use it again.
>

Or my idea was, if his plan was to write a script to grep through his
files, he could just go out and steal the code that comes with idle
where it is already written for him.

Of course, by now he may have already written one ...

Laura



From chris at cdreimer.com  Sat Jun 13 22:24:57 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sat, 13 Jun 2015 19:24:57 -0700
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: <201506140211.t5E2BxAh000784@fido.openend.se>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com> <mli2a5$vvf$1@ger.gmane.org>
 <557C9656.5030205@cdreimer.com> <201506132059.t5DKxXEF026509@fido.openend.se>
 <mailman.469.1434229798.13271.python-list@python.org><557cdfee$0$11128$c3e8da3@news.astraweb.com>
 <201506140211.t5E2BxAh000784@fido.openend.se>
Message-ID: <557CE5F9.3030004@cdreimer.com>

Here's my code and output for the curious. I'm sure someone will come up 
with a five-line version. :)

import os, time

def main():

     def_str = 'def '

     line_limit = 5

     call_tally = []

     import_str = 'from bg_helper import '

     path = 'scripts'

     startTime = time.time()

     

     # pull call names from helper file

     with open(os.path.join(path, 'bg_helper.py')) as f:

         for line in f:

         

             search = line.find(def_str)

             

             if search != -1:

             

                 call_tally.append([(line[len(def_str):].split('('))[0], 0])

     

     call_tally.sort()

     

     # walk through the scripts directory

     fileList = next(os.walk(path))[2]

     fileList.remove('bg_helper.py')

     fileList.remove('bg_helper.pyc')

     

     # check files for import statements

     for filename in fileList:

     

         i = 0

         

         with open(os.path.join(path, filename)) as f:

         

             for line in f:

             

                 if i < line_limit:

                 

                     search = line.find(import_str)

                 

                     if search != -1:

            

                         found = line[len(import_str):].strip('\n').split(', ')

                     

                         found.sort()

  

                         for found in enumerate(found):

                     

                             for tally in enumerate(call_tally):

                                 

                                 if tally[1][0] == found[1]:

                                 

                                    tally[1][1] += 1

                         

                     i += 1

                     

                 else:

                 

                     break

     # print the results

     print 'FUNCTION CALLS\t\tTALLY\tNOTES\n'

                 

     for tally in enumerate(call_tally):

     

         print tally[1][0],'\t\t', tally[1][1],

         

         if tally[1][1] == 0:

         

             print '\t** NO CALLS **'

             

         else:

         

             print ''

         

     print '\nEvaluated',len(fileList),'files in',

     

     print '{0:.1g}'.format(time.time() - startTime),'seconds\n'

         

                 

if __name__ == "__main__":

     main()



FUNCTION CALLS          TALLY   NOTES

card_faces              1

pick_card               1

replay_game             3

roll_dice               2

slot_faces              0       ** NO CALLS **

Evaluated 10 files in 0.08 seconds



From steve+comp.lang.python at pearwood.info  Sat Jun 13 22:53:15 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 14 Jun 2015 02:53:15 GMT
Subject: Is there a utility to tally function calls from other files?
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com> <mli2a5$vvf$1@ger.gmane.org>
 <557C9656.5030205@cdreimer.com>
 <201506132059.t5DKxXEF026509@fido.openend.se>
 <mailman.469.1434229798.13271.python-list@python.org>
 <557cdfee$0$11128$c3e8da3@news.astraweb.com>
 <mailman.471.1434247946.13271.python-list@python.org>
Message-ID: <557cec9b$0$11128$c3e8da3@news.astraweb.com>

On Sun, 14 Jun 2015 04:11:59 +0200, Laura Creighton wrote:

> In a message of 14 Jun 2015 01:59:10 +0000, "Steven D'Aprano" writes:
>>On Sat, 13 Jun 2015 14:09:48 -0700, C.D. Reimer wrote:
>>
>>> On 6/13/2015 1:59 PM, Laura Creighton wrote:
>>>> Idle is written in pure python.  Steal from:
>>>> https://hg.python.org/cpython/file/74d182cf0187/Lib/idlelib/
>>GrepDialog.py
>>>> if you do not have a copy locally.
>>> 
>>> I've been using Notepad++ (Windows) and TextWrangler (Mac) for over a
>>> decade now. Idle may be pure Python, but that's not enough for me to
>>> switch. :)
>>
>>
>>You don't have to switch for good. You can fire up IDLE to do this one
>>job, then never use it again.
>>
>>
> Or my idea was, if his plan was to write a script to grep through his
> files, he could just go out and steal the code that comes with idle
> where it is already written for him.

Well, yes, but since IDLE already contains that script, why not just use 
IDLE?


> Of course, by now he may have already written one ...

Indeed.



-- 
Steven D'Aprano


From lac at openend.se  Sun Jun 14 03:08:32 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 14 Jun 2015 09:08:32 +0200
Subject: Is there a utility to tally function calls from other files?
In-Reply-To: Message from "Steven D'Aprano"
 <steve+comp.lang.python@pearwood.info>
 of "14 Jun 2015 02:53:15 +0000." <557cec9b$0$11128$c3e8da3@news.astraweb.com>
References: <557C800A.6010107@cdreimer.com>
 <CAPTjJmqLOZKs8vzxsWfECK5ynmebAOXwq8rJkisUsFgyFQT2Fw@mail.gmail.com>
 <557C8A47.8080608@cdreimer.com> <mli2a5$vvf$1@ger.gmane.org>
 <557C9656.5030205@cdreimer.com> <201506132059.t5DKxXEF026509@fido.openend.se>
 <mailman.469.1434229798.13271.python-list@python.org>
 <557cdfee$0$11128$c3e8da3@news.astraweb.com>
 <mailman.471.1434247946.13271.python-list@python.org><557cec9b$0$11128$c3e8da3@news.astraweb.com>
Message-ID: <201506140708.t5E78Wvo007703@fido.openend.se>

In a message of 14 Jun 2015 02:53:15 +0000, "Steven D'Aprano" writes:
>Well, yes, but since IDLE already contains that script, why not just use 
>IDLE?

I don't know any way to tell Idle to run in the background, generate this
result, and hand it to me as a list, or a csv or whatever.  Have
I been missing something?  The nice thing about grep is that it doesn't
demand that you have an interactive experience with it in order to get
your results out.

Laura


From laurent.pointal at free.fr  Sun Jun 14 06:08:24 2015
From: laurent.pointal at free.fr (Laurent Pointal)
Date: Sun, 14 Jun 2015 12:08:24 +0200
Subject: PYTHON QUESTION
References: <029c49a7-7e82-4755-8631-6891afc663c5@googlegroups.com>
Message-ID: <557d5298$0$3293$426a34cc@news.free.fr>

adebayo.abraham at gmail.com wrote his student exercise as raw text:

> Help with this problem!
> 
> Temperature converter
> Description
> 
> Write two functions that will convert temperatures back and forth from the
> Celsius and Fahrenheit temperature scales. The formulas for making the
> conversion are as follows:
> 
>   Tc=(5/9)*(Tf-32)
>   Tf=(9/5)*Tc+32
> 
> where Tc is the Celsius temperature and Tf is the Fahrenheit temperature.
> More information and further descriptions of how to do the conversion can
> be found at this NASA Webpage. If you finish this assignment quickly, add
> a function to calculate the wind chill. Input
> 
> Your program should ask the user to input a temperature and then which
> conversion they would like to perform.

One link to help you: http://www.catb.org/esr/faqs/smart-questions.html

You have just to add the Python syntax for functions declaration and return 
value around your expressions (writing let as an exercise)? just care of a 
small difference between Python2 et Python3 with the division operator, 
which may makes you discover a tricky issue with integers manipulation 
(common to many programming languages).

A+



From rosuav at gmail.com  Sun Jun 14 06:24:13 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 14 Jun 2015 20:24:13 +1000
Subject: PyCon 2015 - were all the talks recorded?
Message-ID: <CAPTjJmp=kJA4Y7cVzKvu8wo+X-+aoEHx-O9coLc1Hk9NduUQgA@mail.gmail.com>

Geography prevented me from attending PyCon 2015, but I would have
loved to see some of the talks, particularly the one by Paul
Hildebrandt of Disney [1]. There's a Youtube channel for "PyCon 2015"
[2], but searching by speaker's name, "Disney", and "Inside the Hat"
brought no results. Must I assume that this talk was not recorded?
It'd be a great pity.

[1] https://us.pycon.org/2015/schedule/presentation/365/
[2] https://www.youtube.com/channel/UCgxzjK6GuOHVKR_08TT4hJQ

ChrisA


From ned at nedbatchelder.com  Sun Jun 14 08:35:43 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sun, 14 Jun 2015 05:35:43 -0700 (PDT)
Subject: PyCon 2015 - were all the talks recorded?
In-Reply-To: <mailman.477.1434277457.13271.python-list@python.org>
References: <mailman.477.1434277457.13271.python-list@python.org>
Message-ID: <f270c1c9-1732-4848-86ee-ceeb0efe35bf@googlegroups.com>

On Sunday, June 14, 2015 at 6:25:08 AM UTC-4, Chris Angelico wrote:
> Geography prevented me from attending PyCon 2015, but I would have
> loved to see some of the talks, particularly the one by Paul
> Hildebrandt of Disney [1]. There's a Youtube channel for "PyCon 2015"
> [2], but searching by speaker's name, "Disney", and "Inside the Hat"
> brought no results. Must I assume that this talk was not recorded?
> It'd be a great pity.

My understanding is that Disney does not allow talks by their employees
to be recorded.  This talk was not recorded.

--Ned.


From rosuav at gmail.com  Sun Jun 14 08:45:32 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 14 Jun 2015 22:45:32 +1000
Subject: PyCon 2015 - were all the talks recorded?
In-Reply-To: <f270c1c9-1732-4848-86ee-ceeb0efe35bf@googlegroups.com>
References: <mailman.477.1434277457.13271.python-list@python.org>
 <f270c1c9-1732-4848-86ee-ceeb0efe35bf@googlegroups.com>
Message-ID: <CAPTjJmr3v7DAwO808p7FjQ1RXnwOHxy51fEOv4PFhF2ggEyoKw@mail.gmail.com>

On Sun, Jun 14, 2015 at 10:35 PM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> On Sunday, June 14, 2015 at 6:25:08 AM UTC-4, Chris Angelico wrote:
>> Geography prevented me from attending PyCon 2015, but I would have
>> loved to see some of the talks, particularly the one by Paul
>> Hildebrandt of Disney [1]. There's a Youtube channel for "PyCon 2015"
>> [2], but searching by speaker's name, "Disney", and "Inside the Hat"
>> brought no results. Must I assume that this talk was not recorded?
>> It'd be a great pity.
>
> My understanding is that Disney does not allow talks by their employees
> to be recorded.  This talk was not recorded.

Aww :( Oh well. The costs of living in the southern hemisphere.

ChrisA


From davros at bellaliant.net  Mon Jun 15 01:15:31 2015
From: davros at bellaliant.net (John McKenzie)
Date: Mon, 15 Jun 2015 05:15:31 GMT
Subject: Keypress Input
References: <rBHbx.75089$MO7.40532@fx10.iad>
Message-ID: <Tbtfx.178924$Jv.102633@fx26.iad>


from Tkinter import *
from blinkstick import blinkstick

led = blinkstick.find_first()

timered = 0
timeyellow = 0
timeblue = 0

colour = None

root = Tk()
root.title('eFlag 1')



def red1(event):
    colour = 1

def yellow1(event):
    colour = 2

def blue1(event):
    colour = 3

root.bind_all('r', red1)
root.bind_all('b', blue1)
root.bind_all('y', yellow1)
root.mainloop()

while colour == None:
    led.pulse(red=0, green=255, blue=0, repeats=1, duration=5000, 
steps=50)

while colour == 1:
    led.pulse(red=255, green=0, blue=0, repeats=1, duration=3000, 
steps=50)
    timered += 1

while colour == 2:
    led.pulse(red=255, green=255, blue=0, repeats=1, duration=3000, 
steps=50)
    timeyellow += 1

while colour == 3:
    led.pulse(red=0, green=0, blue=255, repeats=1, duration=2000, 
steps=50)
    timeblue += 1


def exit_handler():
    print '\033[0;41;37mRed Team:\033[0m ', timered
    print '\033[0;43;30mYellow Time:\033[0m ', timeyellow
    print '\033[0;44;37mBlue Time:\033[0m ', timeblue
    flog = open('flag1log.text', 'a')
    flog.write(timestamp + '\n' + 'Red Team: ' + str(timered) + '\n' + 
'Yellow Team: ' + str(timeyellow) + '\n' + 'Blue Team: ' + str(timeblue) 
+ '\n')
    flog.close()
    atexit.register(exit_handler)




From davros at bellaliant.net  Mon Jun 15 01:23:58 2015
From: davros at bellaliant.net (John McKenzie)
Date: Mon, 15 Jun 2015 05:23:58 GMT
Subject: Keypress Input
References: <rBHbx.75089$MO7.40532@fx10.iad>
Message-ID: <Ojtfx.178941$Jv.137884@fx26.iad>


 Thank to the others who joined in and posted replies.

 Michael, your assumption is correct. To quote my original post, "and I 
want this working on a Raspberry Pi." Doing a superficial look at curses 
and getch it looks excessively complicated. I was under the impression it 
was not multi-platform and Linux was excluded. Searching for getch and 
raspberry pi on the web I see it is not and is available for Raspian.

 Worried about implementing it but I will consider it again. May spend 
time reading about during my treatment tomorrow as the hospital has wifi 
and Internet access and I am allowed to have my laptop there.


From minscheung at googlemail.com  Mon Jun 15 06:12:46 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Mon, 15 Jun 2015 03:12:46 -0700 (PDT)
Subject: python financial data cleaning
Message-ID: <b0cbc75c-cc0b-4f27-a8d6-e4c20c356e6d@googlegroups.com>

How to do financial data cleaning ? Say I assume a list of 1000 finance series data in myList = Open, High, Low and Close. For missing Close Price data, What is best practice to clean data in Python



From breamoreboy at yahoo.co.uk  Mon Jun 15 06:19:14 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 15 Jun 2015 11:19:14 +0100
Subject: python financial data cleaning
In-Reply-To: <b0cbc75c-cc0b-4f27-a8d6-e4c20c356e6d@googlegroups.com>
References: <b0cbc75c-cc0b-4f27-a8d6-e4c20c356e6d@googlegroups.com>
Message-ID: <mlm8re$22n$1@ger.gmane.org>

On 15/06/2015 11:12, Sebastian M Cheung via Python-list wrote:
> How to do financial data cleaning ? Say I assume a list of 1000 finance series data in myList = Open, High, Low and Close. For missing Close Price data, What is best practice to clean data in Python
>

http://pandas.pydata.org/

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From minscheung at googlemail.com  Mon Jun 15 06:59:33 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Mon, 15 Jun 2015 03:59:33 -0700 (PDT)
Subject: python financial data cleaning
In-Reply-To: <b0cbc75c-cc0b-4f27-a8d6-e4c20c356e6d@googlegroups.com>
References: <b0cbc75c-cc0b-4f27-a8d6-e4c20c356e6d@googlegroups.com>
Message-ID: <18f75f78-2a6b-4e3a-b3b8-3811b47ff3e4@googlegroups.com>

On Monday, June 15, 2015 at 11:13:07 AM UTC+1, Sebastian M Cheung wrote:
> How to do financial data cleaning ? Say I assume a list of 1000 finance series data in myList = Open, High, Low and Close. For missing Close Price data, What is best practice to clean data in Python

Thanks Mark just looking into Pandas now



Seb


From oscar.j.benjamin at gmail.com  Mon Jun 15 07:22:13 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 15 Jun 2015 12:22:13 +0100
Subject: Keypress Input
In-Reply-To: <Ojtfx.178941$Jv.137884@fx26.iad>
References: <rBHbx.75089$MO7.40532@fx10.iad> <Ojtfx.178941$Jv.137884@fx26.iad>
Message-ID: <CAHVvXxQ_=zHL2Joe=DZrZLgVcnOv6Av0maKpK5MuuUx2ajR3Kg@mail.gmail.com>

On 15 June 2015 at 06:23, John McKenzie <davros at bellaliant.net> wrote:
>
>  Thank to the others who joined in and posted replies.
>
>  Michael, your assumption is correct. To quote my original post, "and I
> want this working on a Raspberry Pi." Doing a superficial look at curses
> and getch it looks excessively complicated. I was under the impression it
> was not multi-platform and Linux was excluded. Searching for getch and
> raspberry pi on the web I see it is not and is available for Raspian.

I'm not sure what you mean but you can write a getch function for
Unixy systems using the tty module (I can't remember where I
originally borrowed this code from):

import sys, tty, termios

def getch():
    fd = sys.stdin.fileno()
    oldsettings = termios.tcgetattr(fd)
    try:
        tty.setraw(sys.stdin.fileno())
        c = sys.stdin.read(1)
    finally:
        termios.tcsetattr(fd, termios.TCSADRAIN, oldsettings)
    return c

while True:
    key = getch()
    print("key: '%c'  code: '%d'" % (key, ord(key)))
    if key == 'q':
        break

This puts the terminal in "raw mode" and waits for a key-press. Once a
key is pressed the terminal is restored to it's previous mode (most
likely not raw mode) and the character is returned. This is important
because once your program finishes you don't want the terminal to be
in raw mode. If the terminal goes weird you can usually fix it by
typing "reset" and pressing enter.

Note that going into raw mode has other implications such as not being
able to exit your program with ctrl-c or suspend with ctrl-z etc. You
can explicitly process those kinds of contrl keys with something like:

while True:
    key = getch()
    if 1 <= ord(key) <= 26:
        ctrl_key = chr(ord(key) + 64)
        print("ctrl-%c" % ctrl_key)
        if ctrl_key == 'C':
            break
    else:
        print("key: '%c'" % key)


--
Oscar


From lac at openend.se  Mon Jun 15 07:34:55 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 15 Jun 2015 13:34:55 +0200
Subject: python financial data cleaning
In-Reply-To: Message from Sebastian M Cheung via Python-list
 <python-list@python.org> of "Mon, 15 Jun 2015 03:12:46 -0700."
 <b0cbc75c-cc0b-4f27-a8d6-e4c20c356e6d@googlegroups.com>
References: <b0cbc75c-cc0b-4f27-a8d6-e4c20c356e6d@googlegroups.com>
Message-ID: <201506151134.t5FBYtZM014718@fido.openend.se>

I don't know anything about this program, and in particular how
complete it is, but worth a look
https://github.com/benjaminmgross/clean-fin-data

Laura


From subhabrata.banerji at gmail.com  Mon Jun 15 07:42:09 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Mon, 15 Jun 2015 04:42:09 -0700 (PDT)
Subject: Creating .exe file in Python
Message-ID: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>

Dear Group,

I am trying to learn how to create .exe file for Python. I tried to work around 
http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program went nice.
But if I try to make exe for larger programs with methods and classes I am getting error. 

If any one of the esteemed members may kindly suggest how to work out. 
I am using Python2.7+ on Windows 7 Professional. 

Regards,
Subhabrata Banerjee. 



From breamoreboy at yahoo.co.uk  Mon Jun 15 08:07:38 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 15 Jun 2015 13:07:38 +0100
Subject: Creating .exe file in Python
In-Reply-To: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
Message-ID: <mlmf6n$eim$1@ger.gmane.org>

On 15/06/2015 12:42, subhabrata.banerji at gmail.com wrote:
> Dear Group,
>
> I am trying to learn how to create .exe file for Python. I tried to work around
> http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program went nice.
> But if I try to make exe for larger programs with methods and classes I am getting error.
>
> If any one of the esteemed members may kindly suggest how to work out.
> I am using Python2.7+ on Windows 7 Professional.
>
> Regards,
> Subhabrata Banerjee.
>

I'm awfully sorry but both my main and backup crystal balls are being 
repaired due to overuse.  "I am getting error" is about as much use as a 
wet fart in a thunderstorm.  So given that I never much liked guessing 
games anyway as I'm not much good at them, how about telling us 
precisely what error you're getting.  In that way one or more of our 
most esteemed colleagues might be able to help.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lac at openend.se  Mon Jun 15 09:01:11 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 15 Jun 2015 15:01:11 +0200
Subject: Creating .exe file in Python
In-Reply-To: Message from subhabrata.banerji@gmail.com of "Mon,
 15 Jun 2015 04:42:09 -0700."
 <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
Message-ID: <201506151301.t5FD1Bld016883@fido.openend.se>

In a message of Mon, 15 Jun 2015 04:42:09 -0700, subhabrata.banerji at gmail.com w
rites:
>Dear Group,
>
>I am trying to learn how to create .exe file for Python. I tried to work around 
>http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program went nice.
>But if I try to make exe for larger programs with methods and classes I am getting error. 
>
>If any one of the esteemed members may kindly suggest how to work out. 
>I am using Python2.7+ on Windows 7 Professional. 
>
>Regards,
>Subhabrata Banerjee. 

1. Post the error.

2. I have always had better luck with PyInstaller than with py2exe
   https://github.com/pyinstaller/pyinstaller/wiki

Laura



From hv at tbz-pariv.de  Mon Jun 15 09:14:47 2015
From: hv at tbz-pariv.de (=?ISO-8859-1?Q?Thomas_G=FCttler?=)
Date: Mon, 15 Jun 2015 06:14:47 -0700 (PDT)
Subject: Get classes from "self.MyClass" to improve subclassability
In-Reply-To: <mailman.445.1434150574.13271.python-list@python.org>
References: <4d336971-d986-4e21-85a2-ddc8ee995c7e@googlegroups.com>
 <mailman.445.1434150574.13271.python-list@python.org>
Message-ID: <6531bd54-d3d5-4b6c-b182-4ac7addc9775@googlegroups.com>

Hi,

crazy. I develop python since several years. I was not aware, that you can 
change the defaults of kwargs. I am amazed, but won't use it :-)



Am Samstag, 13. Juni 2015 01:09:47 UTC+2 schrieb Terry Reedy:
> On 6/12/2015 7:12 AM, Thomas G?ttler wrote:
> > Here is a snippet from the argparse module:
> >
> > {{{
> >      def parse_known_args(self, args=None, namespace=None):
> >          ...
> >          # default Namespace built from parser defaults
> >          if namespace is None:
> >              namespace = Namespace() # < ======= my issue
> > }}}
> >
> > I subclass from the class of the above snippet.
> >
> > I would like to use a different Namespace class.
> >
> > if the above could would use
> >
> >      namespace = self.Namespace()
> >
> > it would be very easy for me to inject a different Namespace class.
> 
> The default arg (None) for the namespace parameter of the 
> parse_known_args is an attribute of the function, not of the class or 
> instance thereof.  Unless the default Namespace is used elsewhere, this 
> seems sensible.
> 
> In CPython, at least, and probably in pypy, you can change this 
> attribute.  (But AFAIK, this is not guaranteed in all implementations.)
> 
>  >>> def f(n = 1): pass
> 
>  >>> f.__defaults__
> (1,)
>  >>> f.__defaults__ = (2,)
>  >>> f.__defaults__
> (2,)
> 
> So the following works
> 
>  >>> class C():
> 	def f(n=1): print(n)
> 	
>  >>> class D(C):
> 	C.f.__defaults__ = (2,)
> 	
>  >>> D.f()
> 2
> 
> Of course, this may or may not do more than you want.
> 
>  >>> C.f()
> 2
> 
> -- 
> Terry Jan Reedy



From minscheung at googlemail.com  Mon Jun 15 09:42:04 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Mon, 15 Jun 2015 06:42:04 -0700 (PDT)
Subject: python financial data cleaning
In-Reply-To: <mailman.482.1434368106.13271.python-list@python.org>
References: <b0cbc75c-cc0b-4f27-a8d6-e4c20c356e6d@googlegroups.com>
 <mailman.482.1434368106.13271.python-list@python.org>
Message-ID: <907d123b-6eb0-4268-b82b-6388a0c2e7b5@googlegroups.com>

On Monday, June 15, 2015 at 12:35:18 PM UTC+1, Laura Creighton wrote:
> I don't know anything about this program, and in particular how
> complete it is, but worth a look
> https://github.com/benjaminmgross/clean-fin-data
> 
> Laura

Thanks Laura, I will check it out, but basically it is to clean financial series data as sometimes there are missing closing price data, misalignment of some sort etc 


From subhabrata.banerji at gmail.com  Mon Jun 15 09:42:48 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Mon, 15 Jun 2015 06:42:48 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.484.1434373285.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <mailman.484.1434373285.13271.python-list@python.org>
Message-ID: <c257d498-fb90-4206-9dee-72f80456fd9d@googlegroups.com>

On Monday, June 15, 2015 at 6:32:33 PM UTC+5:30, Laura Creighton wrote:
> In a message of Mon, 15 Jun 2015 04:42:09 -0700,  w
> rites:
> >Dear Group,
> >
> >I am trying to learn how to create .exe file for Python. I tried to work around 
> >http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program went nice.
> >But if I try to make exe for larger programs with methods and classes I am getting error. 
> >
> >If any one of the esteemed members may kindly suggest how to work out. 
> >I am using Python2.7+ on Windows 7 Professional. 
> >
> >Regards,
> >Subhabrata Banerjee. 
> 
> 1. Post the error.
> 
> 2. I have always had better luck with PyInstaller than with py2exe
>    https://github.com/pyinstaller/pyinstaller/wiki
> 
> Laura
Dear Group,

I wrote a script as NLQ3. py 

the code is written as, 

import nltk
import itertools 
def nlq3(n):
    inp=raw_input("Print Your Query:")
    tag=nltk.pos_tag(nltk.wordpunct_tokenize(inp))
    print "The Tagged Value Is:",tag
    noun=[word[0] for word in tag if 'NN' in word[1]]
    #print noun
    #COMBINATION OF ALL ELEMENTS
    for i in xrange(1,len(noun)+1):
        comb= list(itertools.combinations(noun,i))
        for i,v in enumerate(comb):
            #print v
            v1=list(v)
            print v1

I tried to call it as,
C:\Tutorial>python hello.py
...
as 

C:\Python27>python NLQ3.py

C:\Python27>

But I am not getting any output.

Next I tried to create setup.py as,

from distutils.core import setup
import py2exe
setup(console=['NLQ3.py'])

and trying to work out as,


C:\Python27>python setup.py install
running install
running build
running install_egg_info
Removing C:\Python27\Lib\site-packages\UNKNOWN-0.0.0-py2.7.egg-info
Writing C:\Python27\Lib\site-packages\UNKNOWN-0.0.0-py2.7.egg-info

C:\Python27>

Now, 
as I am giving 
C:\Python27>python setup.py py2exe

it is giving me the following result,
running py2exe
*** searching for required modules ***
*** parsing results ***
creating python loader for extension 'scipy.optimize.minpack2' (C:\Python27\lib\
site-packages\scipy\optimize\minpack2.pyd -> scipy.optimize.minpack2.pyd)
creating python loader for extension 'numpy.core.umath' (C:\Python27\lib\site-pa
ckages\numpy\core\umath.pyd -> numpy.core.umath.pyd)
creating python loader for extension '_elementtree' (C:\Python27\DLLs\_elementtr
ee.pyd -> _elementtree.pyd)
creating python loader for extension 'matplotlib._path' (C:\Python27\lib\site-pa
ckages\matplotlib\_path.pyd -> matplotlib._path.pyd)
creating python loader for extension 'numpy.fft.fftpack_lite' (C:\Python27\lib\s
ite-packages\numpy\fft\fftpack_lite.pyd -> numpy.fft.fftpack_lite.pyd)
creating python loader for extension 'numpy.lib._compiled_base' (C:\Python27\lib
\site-packages\numpy\lib\_compiled_base.pyd -> numpy.lib._compiled_base.pyd)
creating python loader for extension 'scipy.optimize._minpack' (C:\Python27\lib\
site-packages\scipy\optimize\_minpack.pyd -> scipy.optimize._minpack.pyd)
creating python loader for extension '_ctypes' (C:\Python27\DLLs\_ctypes.pyd ->
_ctypes.pyd)
creating python loader for extension 'scipy.sparse.linalg.isolve._iterative' (C:
\Python27\lib\site-packages\scipy\sparse\linalg\isolve\_iterative.pyd -> scipy.s
parse.linalg.isolve._iterative.pyd)
creating python loader for extension 'sklearn.linear_model.sgd_fast' (C:\Python2
7\lib\site-packages\sklearn\linear_model\sgd_fast.pyd -> sklearn.linear_model.sg
d_fast.pyd)
creating python loader for extension 'scipy.stats._rank' (C:\Python27\lib\site-p
ackages\scipy\stats\_rank.pyd -> scipy.stats._rank.pyd)
creating python loader for extension 'select' (C:\Python27\DLLs\select.pyd -> se
lect.pyd)
creating python loader for extension 'scipy.sparse.csgraph._reordering' (C:\Pyth
on27\lib\site-packages\scipy\sparse\csgraph\_reordering.pyd -> scipy.sparse.csgr
aph._reordering.pyd)
creating python loader for extension 'unicodedata' (C:\Python27\DLLs\unicodedata
.pyd -> unicodedata.pyd)
creating python loader for extension 'scipy.special.specfun' (C:\Python27\lib\si
te-packages\scipy\special\specfun.pyd -> scipy.special.specfun.pyd)
creating python loader for extension 'scipy.spatial.qhull' (C:\Python27\lib\site
-packages\scipy\spatial\qhull.pyd -> scipy.spatial.qhull.pyd)
creating python loader for extension 'scipy.interpolate._ppoly' (C:\Python27\lib
\site-packages\scipy\interpolate\_ppoly.pyd -> scipy.interpolate._ppoly.pyd)
creating python loader for extension 'scipy.sparse.csgraph._tools' (C:\Python27\
lib\site-packages\scipy\sparse\csgraph\_tools.pyd -> scipy.sparse.csgraph._tools
.pyd)
creating python loader for extension 'scipy.interpolate.dfitpack' (C:\Python27\l
ib\site-packages\scipy\interpolate\dfitpack.pyd -> scipy.interpolate.dfitpack.py
d)
creating python loader for extension 'scipy.linalg._fblas' (C:\Python27\lib\site
-packages\scipy\linalg\_fblas.pyd -> scipy.linalg._fblas.pyd)
creating python loader for extension 'scipy.optimize._zeros' (C:\Python27\lib\si
te-packages\scipy\optimize\_zeros.pyd -> scipy.optimize._zeros.pyd)
creating python loader for extension 'scipy.stats.mvn' (C:\Python27\lib\site-pac
kages\scipy\stats\mvn.pyd -> scipy.stats.mvn.pyd)
creating python loader for extension 'sklearn.__check_build._check_build' (C:\Py
thon27\lib\site-packages\sklearn\__check_build\_check_build.pyd -> sklearn.__che
ck_build._check_build.pyd)
creating python loader for extension 'scipy.linalg._flinalg' (C:\Python27\lib\si
te-packages\scipy\linalg\_flinalg.pyd -> scipy.linalg._flinalg.pyd)
creating python loader for extension 'numpy.core.scalarmath' (C:\Python27\lib\si
te-packages\numpy\core\scalarmath.pyd -> numpy.core.scalarmath.pyd)
creating python loader for extension '_tkinter' (C:\Python27\DLLs\_tkinter.pyd -
> _tkinter.pyd)
creating python loader for extension 'scipy.integrate.lsoda' (C:\Python27\lib\si
te-packages\scipy\integrate\lsoda.pyd -> scipy.integrate.lsoda.pyd)
creating python loader for extension 'matplotlib._delaunay' (C:\Python27\lib\sit
e-packages\matplotlib\_delaunay.pyd -> matplotlib._delaunay.pyd)
creating python loader for extension 'scipy.optimize._slsqp' (C:\Python27\lib\si
te-packages\scipy\optimize\_slsqp.pyd -> scipy.optimize._slsqp.pyd)
creating python loader for extension 'scipy.optimize._nnls' (C:\Python27\lib\sit
e-packages\scipy\optimize\_nnls.pyd -> scipy.optimize._nnls.pyd)
creating python loader for extension 'scipy.stats.vonmises_cython' (C:\Python27\
lib\site-packages\scipy\stats\vonmises_cython.pyd -> scipy.stats.vonmises_cython
.pyd)
creating python loader for extension '_hashlib' (C:\Python27\DLLs\_hashlib.pyd -
> _hashlib.pyd)
creating python loader for extension 'bz2' (C:\Python27\DLLs\bz2.pyd -> bz2.pyd)

creating python loader for extension 'sklearn.svm.libsvm_sparse' (C:\Python27\li
b\site-packages\sklearn\svm\libsvm_sparse.pyd -> sklearn.svm.libsvm_sparse.pyd)
creating python loader for extension '_ssl' (C:\Python27\DLLs\_ssl.pyd -> _ssl.p
yd)
creating python loader for extension '_sqlite3' (C:\Python27\DLLs\_sqlite3.pyd -
> _sqlite3.pyd)
creating python loader for extension 'scipy.integrate._odepack' (C:\Python27\lib
\site-packages\scipy\integrate\_odepack.pyd -> scipy.integrate._odepack.pyd)
creating python loader for extension 'matplotlib._tri' (C:\Python27\lib\site-pac
kages\matplotlib\_tri.pyd -> matplotlib._tri.pyd)
creating python loader for extension 'scipy.sparse.csgraph._min_spanning_tree' (
C:\Python27\lib\site-packages\scipy\sparse\csgraph\_min_spanning_tree.pyd -> sci
py.sparse.csgraph._min_spanning_tree.pyd)
creating python loader for extension 'numpy.random.mtrand' (C:\Python27\lib\site
-packages\numpy\random\mtrand.pyd -> numpy.random.mtrand.pyd)
creating python loader for extension 'sklearn.metrics.cluster.expected_mutual_in
fo_fast' (C:\Python27\lib\site-packages\sklearn\metrics\cluster\expected_mutual_
info_fast.pyd -> sklearn.metrics.cluster.expected_mutual_info_fast.pyd)
creating python loader for extension 'scipy.stats.futil' (C:\Python27\lib\site-p
ackages\scipy\stats\futil.pyd -> scipy.stats.futil.pyd)
creating python loader for extension 'scipy.optimize.moduleTNC' (C:\Python27\lib
\site-packages\scipy\optimize\moduleTNC.pyd -> scipy.optimize.moduleTNC.pyd)
creating python loader for extension 'scipy.integrate.vode' (C:\Python27\lib\sit
e-packages\scipy\integrate\vode.pyd -> scipy.integrate.vode.pyd)
creating python loader for extension 'scipy.sparse.linalg.dsolve._superlu' (C:\P
ython27\lib\site-packages\scipy\sparse\linalg\dsolve\_superlu.pyd -> scipy.spars
e.linalg.dsolve._superlu.pyd)
creating python loader for extension 'matplotlib._image' (C:\Python27\lib\site-p
ackages\matplotlib\_image.pyd -> matplotlib._image.pyd)
creating python loader for extension 'scipy.integrate._quadpack' (C:\Python27\li
b\site-packages\scipy\integrate\_quadpack.pyd -> scipy.integrate._quadpack.pyd)
creating python loader for extension 'scipy.linalg._flapack' (C:\Python27\lib\si
te-packages\scipy\linalg\_flapack.pyd -> scipy.linalg._flapack.pyd)
creating python loader for extension 'scipy.interpolate.interpnd' (C:\Python27\l
ib\site-packages\scipy\interpolate\interpnd.pyd -> scipy.interpolate.interpnd.py
d)
creating python loader for extension 'numpy.core._sort' (C:\Python27\lib\site-pa
ckages\numpy\core\_sort.pyd -> numpy.core._sort.pyd)
creating python loader for extension '_yaml' (C:\Python27\lib\site-packages\_yam
l.pyd -> _yaml.pyd)
creating python loader for extension 'scipy.special._ellip_harm_2' (C:\Python27\
lib\site-packages\scipy\special\_ellip_harm_2.pyd -> scipy.special._ellip_harm_2
.pyd)
creating python loader for extension 'scipy.integrate._dop' (C:\Python27\lib\sit
e-packages\scipy\integrate\_dop.pyd -> scipy.integrate._dop.pyd)
creating python loader for extension 'sklearn.svm.liblinear' (C:\Python27\lib\si
te-packages\sklearn\svm\liblinear.pyd -> sklearn.svm.liblinear.pyd)
creating python loader for extension 'scipy.sparse.csgraph._traversal' (C:\Pytho
n27\lib\site-packages\scipy\sparse\csgraph\_traversal.pyd -> scipy.sparse.csgrap
h._traversal.pyd)
creating python loader for extension 'matplotlib._png' (C:\Python27\lib\site-pac
kages\matplotlib\_png.pyd -> matplotlib._png.pyd)
creating python loader for extension 'matplotlib.backends._backend_agg' (C:\Pyth
on27\lib\site-packages\matplotlib\backends\_backend_agg.pyd -> matplotlib.backen
ds._backend_agg.pyd)
creating python loader for extension 'pyexpat' (C:\Python27\DLLs\pyexpat.pyd ->
pyexpat.pyd)
creating python loader for extension 'matplotlib.ttconv' (C:\Python27\lib\site-p
ackages\matplotlib\ttconv.pyd -> matplotlib.ttconv.pyd)
creating python loader for extension 'scipy.sparse.linalg.eigen.arpack._arpack'
(C:\Python27\lib\site-packages\scipy\sparse\linalg\eigen\arpack\_arpack.pyd -> s
cipy.sparse.linalg.eigen.arpack._arpack.pyd)
creating python loader for extension 'scipy.sparse._sparsetools' (C:\Python27\li
b\site-packages\scipy\sparse\_sparsetools.pyd -> scipy.sparse._sparsetools.pyd)
creating python loader for extension 'sklearn.linear_model.cd_fast' (C:\Python27
\lib\site-packages\sklearn\linear_model\cd_fast.pyd -> sklearn.linear_model.cd_f
ast.pyd)
creating python loader for extension 'scipy.spatial.ckdtree' (C:\Python27\lib\si
te-packages\scipy\spatial\ckdtree.pyd -> scipy.spatial.ckdtree.pyd)
creating python loader for extension 'scipy.sparse.csgraph._shortest_path' (C:\P
ython27\lib\site-packages\scipy\sparse\csgraph\_shortest_path.pyd -> scipy.spars
e.csgraph._shortest_path.pyd)
creating python loader for extension 'scipy.linalg._clapack' (C:\Python27\lib\si
te-packages\scipy\linalg\_clapack.pyd -> scipy.linalg._clapack.pyd)
creating python loader for extension 'scipy.spatial._distance_wrap' (C:\Python27
\lib\site-packages\scipy\spatial\_distance_wrap.pyd -> scipy.spatial._distance_w
rap.pyd)
creating python loader for extension 'scipy.interpolate._fitpack' (C:\Python27\l
ib\site-packages\scipy\interpolate\_fitpack.pyd -> scipy.interpolate._fitpack.py
d)
creating python loader for extension 'matplotlib._cntr' (C:\Python27\lib\site-pa
ckages\matplotlib\_cntr.pyd -> matplotlib._cntr.pyd)
creating python loader for extension 'sklearn.metrics.pairwise_fast' (C:\Python2
7\lib\site-packages\sklearn\metrics\pairwise_fast.pyd -> sklearn.metrics.pairwis
e_fast.pyd)
creating python loader for extension 'numpy.core.multiarray' (C:\Python27\lib\si
te-packages\numpy\core\multiarray.pyd -> numpy.core.multiarray.pyd)
creating python loader for extension 'scipy.sparse._csparsetools' (C:\Python27\l
ib\site-packages\scipy\sparse\_csparsetools.pyd -> scipy.sparse._csparsetools.py
d)
creating python loader for extension 'matplotlib.ft2font' (C:\Python27\lib\site-
packages\matplotlib\ft2font.pyd -> matplotlib.ft2font.pyd)
creating python loader for extension '_multiprocessing' (C:\Python27\DLLs\_multi
processing.pyd -> _multiprocessing.pyd)
creating python loader for extension 'scipy.special._ufuncs' (C:\Python27\lib\si
te-packages\scipy\special\_ufuncs.pyd -> scipy.special._ufuncs.pyd)
creating python loader for extension 'scipy.optimize._lbfgsb' (C:\Python27\lib\s
ite-packages\scipy\optimize\_lbfgsb.pyd -> scipy.optimize._lbfgsb.pyd)
creating python loader for extension 'numpy.linalg.lapack_lite' (C:\Python27\lib
\site-packages\numpy\linalg\lapack_lite.pyd -> numpy.linalg.lapack_lite.pyd)
creating python loader for extension 'scipy.linalg._cblas' (C:\Python27\lib\site
-packages\scipy\linalg\_cblas.pyd -> scipy.linalg._cblas.pyd)
creating python loader for extension '_socket' (C:\Python27\DLLs\_socket.pyd ->
_socket.pyd)
creating python loader for extension 'sklearn.utils.sparsefuncs_fast' (C:\Python
27\lib\site-packages\sklearn\utils\sparsefuncs_fast.pyd -> sklearn.utils.sparsef
uncs_fast.pyd)
creating python loader for extension 'scipy.stats.statlib' (C:\Python27\lib\site
-packages\scipy\stats\statlib.pyd -> scipy.stats.statlib.pyd)
creating python loader for extension 'sklearn.utils.seq_dataset' (C:\Python27\li
b\site-packages\sklearn\utils\seq_dataset.pyd -> sklearn.utils.seq_dataset.pyd)
creating python loader for extension 'sklearn.utils.arrayfuncs' (C:\Python27\lib
\site-packages\sklearn\utils\arrayfuncs.pyd -> sklearn.utils.arrayfuncs.pyd)
creating python loader for extension 'sklearn.utils._logistic_sigmoid' (C:\Pytho
n27\lib\site-packages\sklearn\utils\_logistic_sigmoid.pyd -> sklearn.utils._logi
stic_sigmoid.pyd)
creating python loader for extension 'sklearn.svm.libsvm' (C:\Python27\lib\site-
packages\sklearn\svm\libsvm.pyd -> sklearn.svm.libsvm.pyd)
creating python loader for extension 'numpy.core._dotblas' (C:\Python27\lib\site
-packages\numpy\core\_dotblas.pyd -> numpy.core._dotblas.pyd)
creating python loader for extension 'sklearn.utils.murmurhash' (C:\Python27\lib
\site-packages\sklearn\utils\murmurhash.pyd -> sklearn.utils.murmurhash.pyd)
creating python loader for extension 'sklearn.utils._random' (C:\Python27\lib\si
te-packages\sklearn\utils\_random.pyd -> sklearn.utils._random.pyd)
creating python loader for extension 'scipy.optimize._cobyla' (C:\Python27\lib\s
ite-packages\scipy\optimize\_cobyla.pyd -> scipy.optimize._cobyla.pyd)
*** finding dlls needed ***
error: [Errno 2] No such file or directory: 'MSVCP90.dll'

Please suggest how may I fix the issue. 

Regards,
Subhabrata. 









From chrismeek4542 at gmail.com  Mon Jun 15 09:45:53 2015
From: chrismeek4542 at gmail.com (chrismeek4542 at gmail.com)
Date: Mon, 15 Jun 2015 06:45:53 -0700 (PDT)
Subject: Pandas SQL Update
Message-ID: <d03cf991-e753-4ccf-9890-78befbe25edc@googlegroups.com>

I have an app that basically compares a old foxpro database to a MySQL database. If the time-stamp does not match up then it updates the MySQL. My question is, is there a more officiant way of doing the MySQL update. I am using pandas itterrows() then doing an update query on each iteration. There is 100,000 plus rows to iterate over so it takes some time


From subhabrata.banerji at gmail.com  Mon Jun 15 10:03:03 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Mon, 15 Jun 2015 07:03:03 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
Message-ID: <ba82704b-ad7e-4d2b-840b-6b4932d83ef9@googlegroups.com>

On Monday, June 15, 2015 at 5:12:24 PM UTC+5:30, subhabrat... at gmail.com wrote:
> Dear Group,
> 
> I am trying to learn how to create .exe file for Python. I tried to work around 
> http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program went nice.
> But if I try to make exe for larger programs with methods and classes I am getting error. 
> 
> If any one of the esteemed members may kindly suggest how to work out. 
> I am using Python2.7+ on Windows 7 Professional. 
> 
> Regards,
> Subhabrata Banerjee.

I am also experimenting around Pyinstaller, cx_Freeze and Inno Studio. But not finding ample examples and manuals. 
Regards,
Subhabrata. 


From tchappui at gmail.com  Mon Jun 15 10:29:44 2015
From: tchappui at gmail.com (Thierry Chappuis)
Date: Mon, 15 Jun 2015 16:29:44 +0200
Subject: Creating .exe file in Python
In-Reply-To: <ba82704b-ad7e-4d2b-840b-6b4932d83ef9@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <ba82704b-ad7e-4d2b-840b-6b4932d83ef9@googlegroups.com>
Message-ID: <1434378586966-ff85dac2-2f80fdde-b4e70f26@gmail.com>

Hi,

The question is why to you want to create an exe from your python project?

Setuptools is capable to create small .exe launchers in the Scripts dir of 
your
python install. These launchers start a python script and use the python
interpreter registered on your platform. That's pretty light and that's my
prefered solution.

If installing the Python interpreter is an issue for the end user, we can 
make
the installer do it for him. Installer programs like Inno Setup let you do 
it
quite easily.

Kind regards

Thierry


On lun., juin 15, 2015 at 4:10 PM, < subhabrata.banerji at gmail.com 
[subhabrata.banerji at gmail.com] > wrote:
On Monday, June 15, 2015 at 5:12:24 PM UTC+5:30, subhabrat... at gmail.com 
wrote:
 > Dear Group,
 >
 > I am trying to learn how to create .exe file for Python. I tried to work
around
 > http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program 
went
nice.
 > But if I try to make exe for larger programs with methods and classes I 
am
getting error.
 >
 > If any one of the esteemed members may kindly suggest how to work out.
 > I am using Python2.7+ on Windows 7 Professional.
 >
 > Regards,
 > Subhabrata Banerjee.

I am also experimenting around Pyinstaller, cx_Freeze and Inno Studio. But 
not
finding ample examples and manuals.
Regards,
Subhabrata.
--
https://mail.python.org/mailman/listinfo/python-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150615/900f5c40/attachment.html>

From subhabrata.banerji at gmail.com  Mon Jun 15 10:40:10 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Mon, 15 Jun 2015 07:40:10 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.485.1434378723.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <ba82704b-ad7e-4d2b-840b-6b4932d83ef9@googlegroups.com>
 <mailman.485.1434378723.13271.python-list@python.org>
Message-ID: <3d31bd2a-07e8-4ed4-82de-ad95571a5d5f@googlegroups.com>

On Monday, June 15, 2015 at 8:02:21 PM UTC+5:30, Thierry Chappuis wrote:
> Hi,
> 
> The question is why to you want to create an exe from your 
> python project?
> 
> Setuptools is capable to create small .exe launchers in the 
> Scripts dir of your python install. These launchers start a python script 
> and use the python interpreter registered on your platform. That's pretty 
> light and that's my prefered solution.
> 
> If installing the Python interpreter is an issue for the end 
> user, we can make the installer do it for him. Installer programs like Inno 
> Setup let you do it quite easily.
> 
> Kind regards
> 
> Thierry
> 
> 
> 
> On lun., juin 15, 2015 at 
> 4:10 PM,  <subhabrat... at gmail.com> 
> wrote:
> 
> 
> On Monday, June 15, 2015 at 
> 5:12:24 PM UTC+5:30, subhabrat... at gmail.com wrote:
> 
> 
> > Dear Group,
> 
> 
> > 
> 
> 
> > I am trying to learn how to create .exe file for Python. I tried to 
> work around 
> 
> 
> > http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program 
> went nice.
> 
> 
> > But if I try to make exe for larger programs with methods and classes 
> I am getting error. 
> 
> 
> > 
> 
> 
> > If any one of the esteemed members may kindly suggest how to work out. 
> 
> 
> 
> > I am using Python2.7+ on Windows 7 Professional. 
> 
> 
> > 
> 
> 
> > Regards,
> 
> 
> > Subhabrata Banerjee.
> 
> 
> 
> 
> 
> I am also experimenting around Pyinstaller, cx_Freeze and Inno Studio. But 
> not finding ample examples and manuals. 
> 
> 
> Regards,
> 
> 
> Subhabrata. 
> 
> 
> -- 
> 
> 
> https://mail.python.org/mailman/listinfo/python-list

Thank you for your kind reply. I have downloaded it, but no good tutorial. If you may send any. Regards, Subhabrata Banerjee. 


From lac at openend.se  Mon Jun 15 10:44:43 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 15 Jun 2015 16:44:43 +0200
Subject: Creating .exe file in Python
In-Reply-To: Message from subhabrata.banerji@gmail.com of "Mon,
 15 Jun 2015 06:42:48 -0700."
 <c257d498-fb90-4206-9dee-72f80456fd9d@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <mailman.484.1434373285.13271.python-list@python.org><c257d498-fb90-4206-9dee-72f80456fd9d@googlegroups.com>
Message-ID: <201506151444.t5FEihqY019262@fido.openend.se>

In a message of Mon, 15 Jun 2015 06:42:48 -0700, subhabrata.banerji at gmail.com w

>I wrote a script as NLQ3. py 
>
>the code is written as, 
>
>import nltk
>import itertools 
>def nlq3(n):
>    inp=raw_input("Print Your Query:")
>    tag=nltk.pos_tag(nltk.wordpunct_tokenize(inp))
>    print "The Tagged Value Is:",tag
>    noun=[word[0] for word in tag if 'NN' in word[1]]
>    #print noun
>    #COMBINATION OF ALL ELEMENTS
>    for i in xrange(1,len(noun)+1):
>        comb= list(itertools.combinations(noun,i))
>        for i,v in enumerate(comb):
>            #print v
>            v1=list(v)
>            print v1
>
>I tried to call it as,
>C:\Tutorial>python hello.py
>...
>as 
>
>C:\Python27>python NLQ3.py
>
>C:\Python27>
>
>But I am not getting any output.

You need to fix your script first, and package it later.
That you are not getting any output indicates that you have
a severe problem.  And you do.  There is no main function in
the code that you posted.  So I made 2 small changes.

First, you aren't using n in your nlq3.  So I changed the function
definition to it to.

def nlq3():

Then I added these lines to the very bottom of the file.

if __name__ == "__main__":
    nlq3()

My file is named nlq3.py and now python nlq3.py gives output.  This
is an improvement.  But the code is still wrong.

It gets to here:

 tag=nltk.pos_tag(nltk.wordpunct_tokenize(inp))

and nltk is very unhappy about this.


lac at smartwheels:~/python$ python nlq3.py
Print Your Query:hello
Traceback (most recent call last):
  File "nlq3.py", line 18, in <module>
    nlq3()
  File "nlq3.py", line 5, in nlq3
    tag=nltk.pos_tag(nltk.wordpunct_tokenize(inp))
  File "/usr/local/lib/python2.7/dist-packages/nltk/tag/__init__.py", line 103, in pos_tag
    tagger = load(_POS_TAGGER)
  File "/usr/local/lib/python2.7/dist-packages/nltk/data.py", line 781, in load
    opened_resource = _open(resource_url)
  File "/usr/local/lib/python2.7/dist-packages/nltk/data.py", line 895, in _open
    return find(path_, path + ['']).open()
  File "/usr/local/lib/python2.7/dist-packages/nltk/data.py", line 624, in find
    raise LookupError(resource_not_found)
LookupError: 
**********************************************************************
  Resource u'taggers/maxent_treebank_pos_tagger/english.pickle'
  not found.  Please use the NLTK Downloader to obtain the
  resource:  >>> nltk.download()
  Searched in:
    - '/home/lac/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
    - u''
**********************************************************************

I don't know enough about nltk to know how to fix this.  You will have
to read nltk docs for that.  But get the code to work, first and
package it later, ok?

Laura


From invalid at invalid.invalid  Mon Jun 15 11:22:52 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Mon, 15 Jun 2015 15:22:52 +0000 (UTC)
Subject: Keypress Input
References: <rBHbx.75089$MO7.40532@fx10.iad> <Ojtfx.178941$Jv.137884@fx26.iad>
 <mailman.481.1434367364.13271.python-list@python.org>
Message-ID: <mlmqkc$dnq$1@reader1.panix.com>

On 2015-06-15, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:

> Note that going into raw mode has other implications such as not
> being able to exit your program with ctrl-c or suspend with ctrl-z
> etc. You can explicitly process those kinds of contrl keys with
> something like:
>
> while True:
>     key = getch()
>     if 1 <= ord(key) <= 26:
>         ctrl_key = chr(ord(key) + 64)
>         print("ctrl-%c" % ctrl_key)
>         if ctrl_key == 'C':
>             break
>     else:
>         print("key: '%c'" % key)

It's probably better (at least on Linux) to just enable handling of
those characters in by setting the ISIG lflag:

    def getch():
        fd = sys.stdin.fileno()
        oldsettings = termios.tcgetattr(fd)
        try:
            tty.setraw(sys.stdin.fileno())
    
            # enable handling of ctrl-C, ctrl-Z, etc.            
            attr = termios.tcgetattr(fd)
            attr[3] |= termios.ISIG
            termios.tcsetattr(fd,termios.TCSANOW,attr)
    
            c = sys.stdin.read(1)
    
        finally:
            termios.tcsetattr(fd, termios.TCSADRAIN, oldsettings)
        return c

It would be a bit cleaner if the termios module supported the the
cfmakeraw(3) function, then you could do it this way and save a couple
of system calls:

    def getch():
        fd = sys.stdin.fileno()
        oldsettings = termios.tcgetattr(fd)
        try:
            newsettings = termios.makeraw(oldsettings)
            newsettings[3] |= termios.ISIG
            termios.tcsetattr(fd,termios.TCSANOW,newsettings)
            c = sys.stdin.read(1)
        finally:
            termios.tcsetattr(fd, termios.TCSADRAIN, oldsettings)
        return c

[I'm a bit surprised that after all these years using literal integers
to index into the attribute list is the "right" way.]
        
-- 
Grant Edwards               grant.b.edwards        Yow! Well, O.K.
                                  at               I'll compromise with my
                              gmail.com            principles because of
                                                   EXISTENTIAL DESPAIR!


From subhabrata.banerji at gmail.com  Mon Jun 15 12:11:51 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Mon, 15 Jun 2015 09:11:51 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
Message-ID: <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>

On Monday, June 15, 2015 at 5:12:24 PM UTC+5:30, subhabrat... at gmail.com wrote:
> Dear Group,
> 
> I am trying to learn how to create .exe file for Python. I tried to work around 
> http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program went nice.
> But if I try to make exe for larger programs with methods and classes I am getting error. 
> 
> If any one of the esteemed members may kindly suggest how to work out. 
> I am using Python2.7+ on Windows 7 Professional. 
> 
> Regards,
> Subhabrata Banerjee.

Dear Madam,

Thanks. Most of it is working except the last portion as I am giving 
python setup.py py2exe

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Admin>cd\

C:\>cd Python27

C:\Python27>python NLQ3.py
Print Your Query:Java
The Tagged Value Is: [('Java', 'NNP')]
['Java']

C:\Python27>python NLQ3.py
Print Your Query:Obama is the president of USA
The Tagged Value Is: [('Obama', 'NNP'), ('is', 'VBZ'), ('the', 'DT'), ('presiden
t', 'NN'), ('of', 'IN'), ('USA', 'NNP')]
['Obama']
['president']
['USA']
['Obama', 'president']
['Obama', 'USA']
['president', 'USA']
['Obama', 'president', 'USA']

C:\Python27>python setup.py install
running install
running build
running install_egg_info
Removing C:\Python27\Lib\site-packages\UNKNOWN-0.0.0-py2.7.egg-info
Writing C:\Python27\Lib\site-packages\UNKNOWN-0.0.0-py2.7.egg-info

C:\Python27>python setup.py py2exe
running py2exe
*** searching for required modules ***
*** parsing results ***
creating python loader for extension 'scipy.optimize.minpack2' (C:\Python27\lib\
site-packages\scipy\optimize\minpack2.pyd -> scipy.optimize.minpack2.pyd)
creating python loader for extension 'numpy.core.umath' (C:\Python27\lib\site-pa
ckages\numpy\core\umath.pyd -> numpy.core.umath.pyd)
creating python loader for extension '_elementtree' (C:\Python27\DLLs\_elementtr
ee.pyd -> _elementtree.pyd)
creating python loader for extension 'matplotlib._path' (C:\Python27\lib\site-pa
ckages\matplotlib\_path.pyd -> matplotlib._path.pyd)
creating python loader for extension 'numpy.fft.fftpack_lite' (C:\Python27\lib\s
ite-packages\numpy\fft\fftpack_lite.pyd -> numpy.fft.fftpack_lite.pyd)
creating python loader for extension 'numpy.lib._compiled_base' (C:\Python27\lib
\site-packages\numpy\lib\_compiled_base.pyd -> numpy.lib._compiled_base.pyd)
creating python loader for extension 'scipy.optimize._minpack' (C:\Python27\lib\
site-packages\scipy\optimize\_minpack.pyd -> scipy.optimize._minpack.pyd)
creating python loader for extension '_ctypes' (C:\Python27\DLLs\_ctypes.pyd ->
_ctypes.pyd)
creating python loader for extension 'scipy.sparse.linalg.isolve._iterative' (C:
\Python27\lib\site-packages\scipy\sparse\linalg\isolve\_iterative.pyd -> scipy.s
parse.linalg.isolve._iterative.pyd)
creating python loader for extension 'sklearn.linear_model.sgd_fast' (C:\Python2
7\lib\site-packages\sklearn\linear_model\sgd_fast.pyd -> sklearn.linear_model.sg
d_fast.pyd)
creating python loader for extension 'scipy.stats._rank' (C:\Python27\lib\site-p
ackages\scipy\stats\_rank.pyd -> scipy.stats._rank.pyd)
creating python loader for extension 'select' (C:\Python27\DLLs\select.pyd -> se
lect.pyd)
creating python loader for extension 'scipy.sparse.csgraph._reordering' (C:\Pyth
on27\lib\site-packages\scipy\sparse\csgraph\_reordering.pyd -> scipy.sparse.csgr
aph._reordering.pyd)
creating python loader for extension 'unicodedata' (C:\Python27\DLLs\unicodedata
.pyd -> unicodedata.pyd)
creating python loader for extension 'scipy.special.specfun' (C:\Python27\lib\si
te-packages\scipy\special\specfun.pyd -> scipy.special.specfun.pyd)
creating python loader for extension 'scipy.spatial.qhull' (C:\Python27\lib\site
-packages\scipy\spatial\qhull.pyd -> scipy.spatial.qhull.pyd)
creating python loader for extension 'scipy.interpolate._ppoly' (C:\Python27\lib
\site-packages\scipy\interpolate\_ppoly.pyd -> scipy.interpolate._ppoly.pyd)
creating python loader for extension 'scipy.sparse.csgraph._tools' (C:\Python27\
lib\site-packages\scipy\sparse\csgraph\_tools.pyd -> scipy.sparse.csgraph._tools
.pyd)
creating python loader for extension 'scipy.interpolate.dfitpack' (C:\Python27\l
ib\site-packages\scipy\interpolate\dfitpack.pyd -> scipy.interpolate.dfitpack.py
d)
creating python loader for extension 'scipy.linalg._fblas' (C:\Python27\lib\site
-packages\scipy\linalg\_fblas.pyd -> scipy.linalg._fblas.pyd)
creating python loader for extension 'scipy.optimize._zeros' (C:\Python27\lib\si
te-packages\scipy\optimize\_zeros.pyd -> scipy.optimize._zeros.pyd)
creating python loader for extension 'scipy.stats.mvn' (C:\Python27\lib\site-pac
kages\scipy\stats\mvn.pyd -> scipy.stats.mvn.pyd)
creating python loader for extension 'sklearn.__check_build._check_build' (C:\Py
thon27\lib\site-packages\sklearn\__check_build\_check_build.pyd -> sklearn.__che
ck_build._check_build.pyd)
creating python loader for extension 'scipy.linalg._flinalg' (C:\Python27\lib\si
te-packages\scipy\linalg\_flinalg.pyd -> scipy.linalg._flinalg.pyd)
creating python loader for extension 'numpy.core.scalarmath' (C:\Python27\lib\si
te-packages\numpy\core\scalarmath.pyd -> numpy.core.scalarmath.pyd)
creating python loader for extension '_tkinter' (C:\Python27\DLLs\_tkinter.pyd -
> _tkinter.pyd)
creating python loader for extension 'scipy.integrate.lsoda' (C:\Python27\lib\si
te-packages\scipy\integrate\lsoda.pyd -> scipy.integrate.lsoda.pyd)
creating python loader for extension 'matplotlib._delaunay' (C:\Python27\lib\sit
e-packages\matplotlib\_delaunay.pyd -> matplotlib._delaunay.pyd)
creating python loader for extension 'scipy.optimize._slsqp' (C:\Python27\lib\si
te-packages\scipy\optimize\_slsqp.pyd -> scipy.optimize._slsqp.pyd)
creating python loader for extension 'scipy.optimize._nnls' (C:\Python27\lib\sit
e-packages\scipy\optimize\_nnls.pyd -> scipy.optimize._nnls.pyd)
creating python loader for extension 'scipy.stats.vonmises_cython' (C:\Python27\
lib\site-packages\scipy\stats\vonmises_cython.pyd -> scipy.stats.vonmises_cython
.pyd)
creating python loader for extension '_hashlib' (C:\Python27\DLLs\_hashlib.pyd -
> _hashlib.pyd)
creating python loader for extension 'bz2' (C:\Python27\DLLs\bz2.pyd -> bz2.pyd)

creating python loader for extension 'sklearn.svm.libsvm_sparse' (C:\Python27\li
b\site-packages\sklearn\svm\libsvm_sparse.pyd -> sklearn.svm.libsvm_sparse.pyd)
creating python loader for extension '_ssl' (C:\Python27\DLLs\_ssl.pyd -> _ssl.p
yd)
creating python loader for extension '_sqlite3' (C:\Python27\DLLs\_sqlite3.pyd -
> _sqlite3.pyd)
creating python loader for extension 'scipy.integrate._odepack' (C:\Python27\lib
\site-packages\scipy\integrate\_odepack.pyd -> scipy.integrate._odepack.pyd)
creating python loader for extension 'matplotlib._tri' (C:\Python27\lib\site-pac
kages\matplotlib\_tri.pyd -> matplotlib._tri.pyd)
creating python loader for extension 'scipy.sparse.csgraph._min_spanning_tree' (
C:\Python27\lib\site-packages\scipy\sparse\csgraph\_min_spanning_tree.pyd -> sci
py.sparse.csgraph._min_spanning_tree.pyd)
creating python loader for extension 'numpy.random.mtrand' (C:\Python27\lib\site
-packages\numpy\random\mtrand.pyd -> numpy.random.mtrand.pyd)
creating python loader for extension 'sklearn.metrics.cluster.expected_mutual_in
fo_fast' (C:\Python27\lib\site-packages\sklearn\metrics\cluster\expected_mutual_
info_fast.pyd -> sklearn.metrics.cluster.expected_mutual_info_fast.pyd)
creating python loader for extension 'scipy.stats.futil' (C:\Python27\lib\site-p
ackages\scipy\stats\futil.pyd -> scipy.stats.futil.pyd)
creating python loader for extension 'scipy.optimize.moduleTNC' (C:\Python27\lib
\site-packages\scipy\optimize\moduleTNC.pyd -> scipy.optimize.moduleTNC.pyd)
creating python loader for extension 'scipy.integrate.vode' (C:\Python27\lib\sit
e-packages\scipy\integrate\vode.pyd -> scipy.integrate.vode.pyd)
creating python loader for extension 'scipy.sparse.linalg.dsolve._superlu' (C:\P
ython27\lib\site-packages\scipy\sparse\linalg\dsolve\_superlu.pyd -> scipy.spars
e.linalg.dsolve._superlu.pyd)
creating python loader for extension 'matplotlib._image' (C:\Python27\lib\site-p
ackages\matplotlib\_image.pyd -> matplotlib._image.pyd)
creating python loader for extension 'scipy.integrate._quadpack' (C:\Python27\li
b\site-packages\scipy\integrate\_quadpack.pyd -> scipy.integrate._quadpack.pyd)
creating python loader for extension 'scipy.linalg._flapack' (C:\Python27\lib\si
te-packages\scipy\linalg\_flapack.pyd -> scipy.linalg._flapack.pyd)
creating python loader for extension 'scipy.interpolate.interpnd' (C:\Python27\l
ib\site-packages\scipy\interpolate\interpnd.pyd -> scipy.interpolate.interpnd.py
d)
creating python loader for extension 'numpy.core._sort' (C:\Python27\lib\site-pa
ckages\numpy\core\_sort.pyd -> numpy.core._sort.pyd)
creating python loader for extension '_yaml' (C:\Python27\lib\site-packages\_yam
l.pyd -> _yaml.pyd)
creating python loader for extension 'scipy.special._ellip_harm_2' (C:\Python27\
lib\site-packages\scipy\special\_ellip_harm_2.pyd -> scipy.special._ellip_harm_2
.pyd)
creating python loader for extension 'scipy.integrate._dop' (C:\Python27\lib\sit
e-packages\scipy\integrate\_dop.pyd -> scipy.integrate._dop.pyd)
creating python loader for extension 'sklearn.svm.liblinear' (C:\Python27\lib\si
te-packages\sklearn\svm\liblinear.pyd -> sklearn.svm.liblinear.pyd)
creating python loader for extension 'scipy.sparse.csgraph._traversal' (C:\Pytho
n27\lib\site-packages\scipy\sparse\csgraph\_traversal.pyd -> scipy.sparse.csgrap
h._traversal.pyd)
creating python loader for extension 'matplotlib._png' (C:\Python27\lib\site-pac
kages\matplotlib\_png.pyd -> matplotlib._png.pyd)
creating python loader for extension 'matplotlib.backends._backend_agg' (C:\Pyth
on27\lib\site-packages\matplotlib\backends\_backend_agg.pyd -> matplotlib.backen
ds._backend_agg.pyd)
creating python loader for extension 'pyexpat' (C:\Python27\DLLs\pyexpat.pyd ->
pyexpat.pyd)
creating python loader for extension 'matplotlib.ttconv' (C:\Python27\lib\site-p
ackages\matplotlib\ttconv.pyd -> matplotlib.ttconv.pyd)
creating python loader for extension 'scipy.sparse.linalg.eigen.arpack._arpack'
(C:\Python27\lib\site-packages\scipy\sparse\linalg\eigen\arpack\_arpack.pyd -> s
cipy.sparse.linalg.eigen.arpack._arpack.pyd)
creating python loader for extension 'scipy.sparse._sparsetools' (C:\Python27\li
b\site-packages\scipy\sparse\_sparsetools.pyd -> scipy.sparse._sparsetools.pyd)
creating python loader for extension 'sklearn.linear_model.cd_fast' (C:\Python27
\lib\site-packages\sklearn\linear_model\cd_fast.pyd -> sklearn.linear_model.cd_f
ast.pyd)
creating python loader for extension 'scipy.spatial.ckdtree' (C:\Python27\lib\si
te-packages\scipy\spatial\ckdtree.pyd -> scipy.spatial.ckdtree.pyd)
creating python loader for extension 'scipy.sparse.csgraph._shortest_path' (C:\P
ython27\lib\site-packages\scipy\sparse\csgraph\_shortest_path.pyd -> scipy.spars
e.csgraph._shortest_path.pyd)
creating python loader for extension 'scipy.linalg._clapack' (C:\Python27\lib\si
te-packages\scipy\linalg\_clapack.pyd -> scipy.linalg._clapack.pyd)
creating python loader for extension 'scipy.spatial._distance_wrap' (C:\Python27
\lib\site-packages\scipy\spatial\_distance_wrap.pyd -> scipy.spatial._distance_w
rap.pyd)
creating python loader for extension 'scipy.interpolate._fitpack' (C:\Python27\l
ib\site-packages\scipy\interpolate\_fitpack.pyd -> scipy.interpolate._fitpack.py
d)
creating python loader for extension 'matplotlib._cntr' (C:\Python27\lib\site-pa
ckages\matplotlib\_cntr.pyd -> matplotlib._cntr.pyd)
creating python loader for extension 'sklearn.metrics.pairwise_fast' (C:\Python2
7\lib\site-packages\sklearn\metrics\pairwise_fast.pyd -> sklearn.metrics.pairwis
e_fast.pyd)
creating python loader for extension 'numpy.core.multiarray' (C:\Python27\lib\si
te-packages\numpy\core\multiarray.pyd -> numpy.core.multiarray.pyd)
creating python loader for extension 'scipy.sparse._csparsetools' (C:\Python27\l
ib\site-packages\scipy\sparse\_csparsetools.pyd -> scipy.sparse._csparsetools.py
d)
creating python loader for extension 'matplotlib.ft2font' (C:\Python27\lib\site-
packages\matplotlib\ft2font.pyd -> matplotlib.ft2font.pyd)
creating python loader for extension '_multiprocessing' (C:\Python27\DLLs\_multi
processing.pyd -> _multiprocessing.pyd)
creating python loader for extension 'scipy.special._ufuncs' (C:\Python27\lib\si
te-packages\scipy\special\_ufuncs.pyd -> scipy.special._ufuncs.pyd)
creating python loader for extension 'scipy.optimize._lbfgsb' (C:\Python27\lib\s
ite-packages\scipy\optimize\_lbfgsb.pyd -> scipy.optimize._lbfgsb.pyd)
creating python loader for extension 'numpy.linalg.lapack_lite' (C:\Python27\lib
\site-packages\numpy\linalg\lapack_lite.pyd -> numpy.linalg.lapack_lite.pyd)
creating python loader for extension 'scipy.linalg._cblas' (C:\Python27\lib\site
-packages\scipy\linalg\_cblas.pyd -> scipy.linalg._cblas.pyd)
creating python loader for extension '_socket' (C:\Python27\DLLs\_socket.pyd ->
_socket.pyd)
creating python loader for extension 'sklearn.utils.sparsefuncs_fast' (C:\Python
27\lib\site-packages\sklearn\utils\sparsefuncs_fast.pyd -> sklearn.utils.sparsef
uncs_fast.pyd)
creating python loader for extension 'scipy.stats.statlib' (C:\Python27\lib\site
-packages\scipy\stats\statlib.pyd -> scipy.stats.statlib.pyd)
creating python loader for extension 'sklearn.utils.seq_dataset' (C:\Python27\li
b\site-packages\sklearn\utils\seq_dataset.pyd -> sklearn.utils.seq_dataset.pyd)
creating python loader for extension 'sklearn.utils.arrayfuncs' (C:\Python27\lib
\site-packages\sklearn\utils\arrayfuncs.pyd -> sklearn.utils.arrayfuncs.pyd)
creating python loader for extension 'sklearn.utils._logistic_sigmoid' (C:\Pytho
n27\lib\site-packages\sklearn\utils\_logistic_sigmoid.pyd -> sklearn.utils._logi
stic_sigmoid.pyd)
creating python loader for extension 'sklearn.svm.libsvm' (C:\Python27\lib\site-
packages\sklearn\svm\libsvm.pyd -> sklearn.svm.libsvm.pyd)
creating python loader for extension 'numpy.core._dotblas' (C:\Python27\lib\site
-packages\numpy\core\_dotblas.pyd -> numpy.core._dotblas.pyd)
creating python loader for extension 'sklearn.utils.murmurhash' (C:\Python27\lib
\site-packages\sklearn\utils\murmurhash.pyd -> sklearn.utils.murmurhash.pyd)
creating python loader for extension 'sklearn.utils._random' (C:\Python27\lib\si
te-packages\sklearn\utils\_random.pyd -> sklearn.utils._random.pyd)
creating python loader for extension 'scipy.optimize._cobyla' (C:\Python27\lib\s
ite-packages\scipy\optimize\_cobyla.pyd -> scipy.optimize._cobyla.pyd)
*** finding dlls needed ***
error: [Errno 2] No such file or directory: 'MSVCP90.dll'

C:\Python27>

And I have changed NLTK thing to,pos_tag(wordpunct_tokenize(inp))

Regards,
Subhabrata Banerjee. 

 



From lac at openend.se  Mon Jun 15 12:29:12 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 15 Jun 2015 18:29:12 +0200
Subject: Creating .exe file in Python
In-Reply-To: Message from subhabrata.banerji@gmail.com of "Mon,
 15 Jun 2015 09:11:51 -0700."
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com><2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
Message-ID: <201506151629.t5FGTCG9021793@fido.openend.se>

I don't have a windows system, so my knowledge of such things is
minimal.  But looks like this person had the same problem you have,
and got some suggestions on how to fix it.

http://stackoverflow.com/questions/12127869/error-msvcp90-dll-no-such-file-or-directory-even-though-microsoft-visual-c

But your code is working fine now?  There is no point in packaging
something that doesn't run.

Laura



From sohcahtoa82 at gmail.com  Mon Jun 15 13:09:36 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Mon, 15 Jun 2015 10:09:36 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.483.1434370085.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <mailman.483.1434370085.13271.python-list@python.org>
Message-ID: <0c30ab1a-3b52-4279-b13c-19b8635b0dd8@googlegroups.com>

On Monday, June 15, 2015 at 5:08:58 AM UTC-7, Mark Lawrence wrote:
> On 15/06/2015 12:42, subhabrata.banerji at gmail.com wrote:
> > Dear Group,
> >
> > I am trying to learn how to create .exe file for Python. I tried to work around
> > http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program went nice.
> > But if I try to make exe for larger programs with methods and classes I am getting error.
> >
> > If any one of the esteemed members may kindly suggest how to work out.
> > I am using Python2.7+ on Windows 7 Professional.
> >
> > Regards,
> > Subhabrata Banerjee.
> >
> 
> I'm awfully sorry but both my main and backup crystal balls are being 
> repaired due to overuse.  "I am getting error" is about as much use as a 
> wet fart in a thunderstorm.  So given that I never much liked guessing 
> games anyway as I'm not much good at them, how about telling us 
> precisely what error you're getting.  In that way one or more of our 
> most esteemed colleagues might be able to help.
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence

"as much use as a wet fart in a thunderstorm"

I gotta remember that one.  :-D

But on topic...I really don't understand what makes people think we can *possibly* help them by only saying "I'm getting an error" without saying what the error is.

I mean, even before I ever became a programmer, I knew that if I needed help fixing something, I needed to give as much information as I could.  Exact error messages, detailed description of what I was doing, what I've done to try to fix it, etc.


From minscheung at googlemail.com  Mon Jun 15 17:01:31 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Mon, 15 Jun 2015 14:01:31 -0700 (PDT)
Subject: python financial data cleaning
In-Reply-To: <mailman.480.1434363576.13271.python-list@python.org>
References: <b0cbc75c-cc0b-4f27-a8d6-e4c20c356e6d@googlegroups.com>
 <mailman.480.1434363576.13271.python-list@python.org>
Message-ID: <55a0ccda-572c-4324-9169-08028d4fb619@googlegroups.com>

On Monday, June 15, 2015 at 11:19:48 AM UTC+1, Mark Lawrence wrote:
> On 15/06/2015 11:12, Sebastian M Cheung via Python-list wrote:
> > How to do financial data cleaning ? Say I assume a list of 1000 finance series data in myList = Open, High, Low and Close. For missing Close Price data, What is best practice to clean data in Python
> >
> 
> http://pandas.pydata.org/
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence


Hi Mark,

Below I read in DirtyData (financial data) from Excel and then find the number of NaN missing Closed Pricing data:

xls = pd.ExcelFile('DirtyData.xlsm')
df = xls.parse('Dirty Data', index_col=None, na_values=['NA'])
print(df.isnull().astype(int).sum()) 

So if I were to clean missing Open Price data, I could copy from previous or row's Close Price data, but how would I implement it? Thanks



From steve+comp.lang.python at pearwood.info  Mon Jun 15 19:57:43 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 15 Jun 2015 23:57:43 GMT
Subject: Set a flag on the function or a global?
Message-ID: <557f6676$0$21718$c3e8da3@news.astraweb.com>

I have a function in a module which is intended to be used by importing 
that name alone, then used interactively:

    from module import edir
    edir(args)


edir is an enhanced version of dir, and one of the enhancements is that 
you can filter out dunder methods. I have reason to believe that people 
are split on their opinion on whether dunder methods should be shown by 
default or not: some people want to see them, others do not. Since edir 
is meant to be used interactively, I want to give people a setting to 
control whether they get dunders by default or not.

I have two ideas for this, a module-level global, or a flag set on the 
function object itself. Remember that the usual way of using this will be 
"from module import edir", there are two obvious ways to set the global:

import module
module.dunders = False

# -or-

edir.__globals__['dunders'] = False


Alternatively, I can use a flag set on the function object itself:

edir.dunders = False


Naturally you can always override the default by explicitly specifying a 
keyword argument edir(obj, dunders=flag).

Thoughts and feedback? Please vote: a module global, or a flag on the 
object? Please give reasons, and remember that the function is intended 
for interactive use.


-- 
Steven D'Aprano


From rosuav at gmail.com  Mon Jun 15 20:07:52 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 16 Jun 2015 10:07:52 +1000
Subject: Set a flag on the function or a global?
In-Reply-To: <557f6676$0$21718$c3e8da3@news.astraweb.com>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
Message-ID: <CAPTjJmr9r7Ev2Dw3+fY=mzOF3dz5TaXWX07TFH9R0u7JVxrzCg@mail.gmail.com>

On Tue, Jun 16, 2015 at 9:57 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> I have two ideas for this, a module-level global, or a flag set on the
> function object itself. Remember that the usual way of using this will be
> "from module import edir", there are two obvious ways to set the global:
>
> import module
> module.dunders = False
>
> # -or-
>
> edir.__globals__['dunders'] = False
>
>
> Alternatively, I can use a flag set on the function object itself:
>
> edir.dunders = False
>

For most situations, the last one is extremely surprising - attributes
on functions aren't normally meant to be changed by outside callers,
it always feels wrong (they belong to the function itself). But since
this is interactive, I'd advise going for the absolute simplest, which
this would be. Go for the function attribute IMO.

ChrisA


From ethan at stoneleaf.us  Mon Jun 15 20:19:59 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 15 Jun 2015 17:19:59 -0700
Subject: Set a flag on the function or a global?
In-Reply-To: <557f6676$0$21718$c3e8da3@news.astraweb.com>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
Message-ID: <557F6BAF.3080702@stoneleaf.us>

On 06/15/2015 04:57 PM, Steven D'Aprano wrote:

> Thoughts and feedback? Please vote: a module global, or a flag on the
> object? Please give reasons, and remember that the function is intended
> for interactive use.

Function attribute.

Setting a global on the module (which I may not have, and probably didn't, import) for only one function is overkill.

   edir.dunders = False # or True

is simple, elegant, easier to type, and perfectly in keeping with Python's dynamic nature.

--
~Ethan~


From ben+python at benfinney.id.au  Mon Jun 15 20:20:38 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 16 Jun 2015 10:20:38 +1000
Subject: Set a flag on the function or a global?
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <CAPTjJmr9r7Ev2Dw3+fY=mzOF3dz5TaXWX07TFH9R0u7JVxrzCg@mail.gmail.com>
Message-ID: <85oakgtsfd.fsf@benfinney.id.au>

Chris Angelico <rosuav at gmail.com> writes:

> On Tue, Jun 16, 2015 at 9:57 AM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
> > I can use a flag set on the function object itself:
> >
> > edir.dunders = False
>
> For most situations, the last one is extremely surprising - attributes
> on functions aren't normally meant to be changed by outside callers,
> it always feels wrong (they belong to the function itself).

I'm surprised by your assertion. To my mind, outside callers get simple
and direct access to the attribute, whereas the code of the function
itself does not have such easy access; unlike ?self? for the current
instance of a class, there's no obvious name to use for referring to the
function object within the function object's own code.

In what sense do they ?belong to? the function itself *more than* to
outside callers?

-- 
 \       ?It's easy to play any musical instrument: all you have to do |
  `\       is touch the right key at the right time and the instrument |
_o__)                        will play itself.? ?Johann Sebastian Bach |
Ben Finney



From ethan at stoneleaf.us  Mon Jun 15 20:21:43 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 15 Jun 2015 17:21:43 -0700
Subject: Set a flag on the function or a global?
In-Reply-To: <CAPTjJmr9r7Ev2Dw3+fY=mzOF3dz5TaXWX07TFH9R0u7JVxrzCg@mail.gmail.com>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <CAPTjJmr9r7Ev2Dw3+fY=mzOF3dz5TaXWX07TFH9R0u7JVxrzCg@mail.gmail.com>
Message-ID: <557F6C17.7010101@stoneleaf.us>

On 06/15/2015 05:07 PM, Chris Angelico wrote:
> On Tue, Jun 16, 2015 at 9:57 AM, Steven D'Aprano wrote:

>> I have two ideas for this, a module-level global, or a flag set on the
>> function object itself. Remember that the usual way of using this will be
>> "from module import edir", there are two obvious ways to set the global:
>>
>> import module
>> module.dunders = False
>>
>> # -or-
>>
>> edir.__globals__['dunders'] = False
>>
>>
>> Alternatively, I can use a flag set on the function object itself:
>>
>> edir.dunders = False
>>
>
> For most situations, the last one is extremely surprising - attributes
> on functions aren't normally meant to be changed by outside callers,

I find this viewpoint surprising, since function attributes are fairly rare.

> it always feels wrong (they belong to the function itself).

This seems silly -- a function is just another instance of some class.

--
~Ethan~


From ron3200 at gmail.com  Mon Jun 15 20:24:34 2015
From: ron3200 at gmail.com (Ron Adam)
Date: Mon, 15 Jun 2015 20:24:34 -0400
Subject: Set a flag on the function or a global?
In-Reply-To: <CAPTjJmr9r7Ev2Dw3+fY=mzOF3dz5TaXWX07TFH9R0u7JVxrzCg@mail.gmail.com>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <CAPTjJmr9r7Ev2Dw3+fY=mzOF3dz5TaXWX07TFH9R0u7JVxrzCg@mail.gmail.com>
Message-ID: <mlnqc3$i04$1@ger.gmane.org>



On 06/15/2015 08:07 PM, Chris Angelico wrote:
> On Tue, Jun 16, 2015 at 9:57 AM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info>  wrote:
>> >I have two ideas for this, a module-level global, or a flag set on the
>> >function object itself. Remember that the usual way of using this will be
>> >"from module import edir", there are two obvious ways to set the global:
>> >
>> >import module
>> >module.dunders = False
>> >
>> ># -or-
>> >
>> >edir.__globals__['dunders'] = False
>> >
>> >
>> >Alternatively, I can use a flag set on the function object itself:
>> >
>> >edir.dunders = False
>> >
> For most situations, the last one is extremely surprising - attributes
> on functions aren't normally meant to be changed by outside callers,

Or inside callers either.  You can't be sure of the name and there is no self.


> it always feels wrong (they belong to the function itself). But since
> this is interactive, I'd advise going for the absolute simplest, which
> this would be. Go for the function attribute IMO.


Another way is to make it an object with a __call__ method.

The the attribute can be accessed from both outside and inside dependably.

Cheers,
    Ron



From sohcahtoa82 at gmail.com  Mon Jun 15 20:24:37 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Mon, 15 Jun 2015 17:24:37 -0700 (PDT)
Subject: Set a flag on the function or a global?
In-Reply-To: <557f6676$0$21718$c3e8da3@news.astraweb.com>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
Message-ID: <4449bc52-4dba-486c-b426-7095e228a316@googlegroups.com>

On Monday, June 15, 2015 at 4:57:53 PM UTC-7, Steven D'Aprano wrote:
> I have a function in a module which is intended to be used by importing 
> that name alone, then used interactively:
> 
>     from module import edir
>     edir(args)
> 
> 
> edir is an enhanced version of dir, and one of the enhancements is that 
> you can filter out dunder methods. I have reason to believe that people 
> are split on their opinion on whether dunder methods should be shown by 
> default or not: some people want to see them, others do not. Since edir 
> is meant to be used interactively, I want to give people a setting to 
> control whether they get dunders by default or not.
> 
> I have two ideas for this, a module-level global, or a flag set on the 
> function object itself. Remember that the usual way of using this will be 
> "from module import edir", there are two obvious ways to set the global:
> 
> import module
> module.dunders = False
> 
> # -or-
> 
> edir.__globals__['dunders'] = False
> 
> 
> Alternatively, I can use a flag set on the function object itself:
> 
> edir.dunders = False
> 
> 
> Naturally you can always override the default by explicitly specifying a 
> keyword argument edir(obj, dunders=flag).
> 
> Thoughts and feedback? Please vote: a module global, or a flag on the 
> object? Please give reasons, and remember that the function is intended 
> for interactive use.
> 
> 
> -- 
> Steven D'Aprano

Using a keyword argument for the edir function is the most intuitive and easy to read, IMO.

Also, if two people are working on the same script, it could create problems if one person wants to filter them, but the other doesn't.  That would create a state that they would both have to monitor and keep setting back and forth, rather than each one just setting an argument on their calls.


From rosuav at gmail.com  Mon Jun 15 20:32:41 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 16 Jun 2015 10:32:41 +1000
Subject: Set a flag on the function or a global?
In-Reply-To: <85oakgtsfd.fsf@benfinney.id.au>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <CAPTjJmr9r7Ev2Dw3+fY=mzOF3dz5TaXWX07TFH9R0u7JVxrzCg@mail.gmail.com>
 <85oakgtsfd.fsf@benfinney.id.au>
Message-ID: <CAPTjJmqO+ckQm=nWCy0uAXjF=hsd=q571gmWYngwNWh=RDg6EQ@mail.gmail.com>

On Tue, Jun 16, 2015 at 10:20 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Chris Angelico <rosuav at gmail.com> writes:
>
>> On Tue, Jun 16, 2015 at 9:57 AM, Steven D'Aprano
>> <steve+comp.lang.python at pearwood.info> wrote:
>> > I can use a flag set on the function object itself:
>> >
>> > edir.dunders = False
>>
>> For most situations, the last one is extremely surprising - attributes
>> on functions aren't normally meant to be changed by outside callers,
>> it always feels wrong (they belong to the function itself).
>
> I'm surprised by your assertion. To my mind, outside callers get simple
> and direct access to the attribute, whereas the code of the function
> itself does not have such easy access; unlike ?self? for the current
> instance of a class, there's no obvious name to use for referring to the
> function object within the function object's own code.
>
> In what sense do they ?belong to? the function itself *more than* to
> outside callers?

Custom function attributes (as in, those not set by the interpreter
itself) are pretty rare, but I've usually seen them only being defined
by the module that created them. Setting that kind of attribute
externally, from a different module, seems odd - and undiscoverable.
But for interactive work, that should be fine.

ChrisA


From python at mrabarnett.plus.com  Mon Jun 15 20:35:59 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 16 Jun 2015 01:35:59 +0100
Subject: Set a flag on the function or a global?
In-Reply-To: <4449bc52-4dba-486c-b426-7095e228a316@googlegroups.com>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <4449bc52-4dba-486c-b426-7095e228a316@googlegroups.com>
Message-ID: <557F6F6F.2080402@mrabarnett.plus.com>

On 2015-06-16 01:24, sohcahtoa82 at gmail.com wrote:
> On Monday, June 15, 2015 at 4:57:53 PM UTC-7, Steven D'Aprano wrote:
>> I have a function in a module which is intended to be used by
>> importing that name alone, then used interactively:
>>
>> from module import edir edir(args)
>>
>>
>> edir is an enhanced version of dir, and one of the enhancements is
>> that you can filter out dunder methods. I have reason to believe
>> that people are split on their opinion on whether dunder methods
>> should be shown by default or not: some people want to see them,
>> others do not. Since edir is meant to be used interactively, I want
>> to give people a setting to control whether they get dunders by
>> default or not.
>>
>> I have two ideas for this, a module-level global, or a flag set on
>> the function object itself. Remember that the usual way of using
>> this will be "from module import edir", there are two obvious ways
>> to set the global:
>>
>> import module module.dunders = False
>>
>> # -or-
>>
>> edir.__globals__['dunders'] = False
>>
>>
>> Alternatively, I can use a flag set on the function object itself:
>>
>> edir.dunders = False
>>
>>
>> Naturally you can always override the default by explicitly
>> specifying a keyword argument edir(obj, dunders=flag).
>>
>> Thoughts and feedback? Please vote: a module global, or a flag on
>> the object? Please give reasons, and remember that the function is
>> intended for interactive use.
>>
>>
>> -- Steven D'Aprano
>
> Using a keyword argument for the edir function is the most intuitive
> and easy to read, IMO.
>
> Also, if two people are working on the same script, it could create
> problems if one person wants to filter them, but the other doesn't.
> That would create a state that they would both have to monitor and
> keep setting back and forth, rather than each one just setting an
> argument on their calls.
>
But it's meant to be used interactively. If they're using it in a
script, they'd most likely set the argument appropriately.


From no.email at nospam.invalid  Mon Jun 15 20:37:33 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Mon, 15 Jun 2015 17:37:33 -0700
Subject: Set a flag on the function or a global?
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
Message-ID: <87381strn6.fsf@jester.gateway.sonic.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
> Thoughts and feedback? Please vote: a module global, or a flag on the 
> object? Please give reasons, and remember that the function is intended 
> for interactive use.

Both are bad.  More state to remember, ugh.  Instead have separate entry
points for filtering or not filtering the dunders.  Something like:

  edir(obj) = no dunders
  edir_(obj) = dunders.  

If you also support the keyword arg, then you could have

  edir_(obj)=functools.partial(edir,dunders=True).


From ethan at stoneleaf.us  Mon Jun 15 20:53:01 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 15 Jun 2015 17:53:01 -0700
Subject: Set a flag on the function or a global?
In-Reply-To: <87381strn6.fsf@jester.gateway.sonic.net>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <87381strn6.fsf@jester.gateway.sonic.net>
Message-ID: <557F736D.4000701@stoneleaf.us>

On 06/15/2015 05:37 PM, Paul Rubin wrote:
> Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
>> Thoughts and feedback? Please vote: a module global, or a flag on the
>> object? Please give reasons, and remember that the function is intended
>> for interactive use.
>
> Both are bad.  More state to remember, ugh.  Instead have separate entry
> points for filtering or not filtering the dunders.  Something like:
>
>    edir(obj) = no dunders
>    edir_(obj) = dunders.

`edir_` ?  What a horrible name.  I hate trailing underscores.  Too easy to miss.

--
~Ethan~


From python at mrabarnett.plus.com  Mon Jun 15 21:15:50 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 16 Jun 2015 02:15:50 +0100
Subject: Set a flag on the function or a global?
In-Reply-To: <557F736D.4000701@stoneleaf.us>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <87381strn6.fsf@jester.gateway.sonic.net> <557F736D.4000701@stoneleaf.us>
Message-ID: <557F78C6.4050700@mrabarnett.plus.com>

On 2015-06-16 01:53, Ethan Furman wrote:
> On 06/15/2015 05:37 PM, Paul Rubin wrote:
>> Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
>>> Thoughts and feedback? Please vote: a module global, or a flag on the
>>> object? Please give reasons, and remember that the function is intended
>>> for interactive use.
>>
>> Both are bad.  More state to remember, ugh.  Instead have separate entry
>> points for filtering or not filtering the dunders.  Something like:
>>
>>    edir(obj) = no dunders
>>    edir_(obj) = dunders.
>
> `edir_` ?  What a horrible name.  I hate trailing underscores.  Too easy to miss.
>
Especially when there's also `edir`! :-)



From malik.a.rumi at gmail.com  Mon Jun 15 22:00:52 2015
From: malik.a.rumi at gmail.com (Malik Rumi)
Date: Mon, 15 Jun 2015 19:00:52 -0700 (PDT)
Subject: Struggling with os.path.join and fileinput (was 'Path, strings,
 and lines'
In-Reply-To: <mailman.458.1434219928.13271.python-list@python.org>
References: <56ac4185-3d58-473a-ad40-99d9366a8bb3@googlegroups.com>
 <mailman.438.1434141083.13271.python-list@python.org>
 <58a13725-92f7-407b-8444-71080ada1538@googlegroups.com>
 <mailman.458.1434219928.13271.python-list@python.org>
Message-ID: <e5772193-0427-4b96-9e09-1e0f60f81d17@googlegroups.com>

On Saturday, June 13, 2015 at 1:25:52 PM UTC-5, MRAB wrote:
> On 2015-06-13 05:48, Malik Rumi wrote:
> > On Friday, June 12, 2015 at 3:31:36 PM UTC-5, Ian wrote:
> >> On Fri, Jun 12, 2015 at 1:39 PM, Malik Rumi wrote:
> >> >  I am trying to find a list of strings in a directory of files. Here is my code:
> >> >
> >> > # -*- coding: utf-8 -*-
> >> > import os
> >> > import fileinput
> >> >
> >> > s2 = os.listdir('/home/malikarumi/Projects/P5/shortstories')
> >>
> >> Note that the filenames that will be returned here are not fully
> >> qualified: you'll just get filename.txt, not
> >> /home/.../shortstories/filename.txt.
> >>
> >
> > Yes, that is what I want.
> >
> >> > for line in lines:
> >> >      for item in fileinput.input(s2):
> >>
> >> fileinput doesn't have the context of the directory that you listed
> >> above, so it's just going to look in the current directory.
> >
> > Can you explain a little more what you mean by fileinput lacking the context of s4?
> >
> listdir returns the names of the files that are in the folder, not
> their paths.
> 
> If you give fileinput only the names of the files, it'll assume they're
> in the current folder (directory), which they (probably) aren't. You
> need to give fileinput the complete _paths_ of the files, not just
> their names.
> 
> >>
> >> >          if line in item:
> >> >             with open(line + '_list', 'a+') as l:
> >> >                 l.append(filename(), filelineno(), line)
> >>
> >> Although it's not the problem at hand, I think you'll find that you
> >> need to qualify the filename() and filelineno() function calls with
> >> the fileinput module.
> >
> > By 'qualify', do you mean something like
> > l.append(fileinput.filename())?
> >
> >>
> >> > FileNotFoundError: [Errno 2] No such file or directory: 'THE LAND OF LOST TOYS~'
> >>
> >> And here you can see that it's failing to find the file because it's
> >> looking in the wrong directory. You can use the os.path.join function
> >> to add the proper directory path to the filenames that you pass to
> >> fileinput.
> >
> > I tried new code:
> >
> > # -*- coding: utf-8 -*-
> > import os
> > import fileinput
> >
> >
> os.join _returns its result.
> 
> > os.path.join('/Projects/Pipeline/4 Transforms', '/Projects/P5/shortstories/')
> > s2 = os.listdir('/Projects/P5/shortstories/')
> 
> At this point, s2 contains a list of _names_.
> 
> You pass those names to fileinput.input, but where are they? In which
> folder? It assumes they're in the current folder (directory), but
> they're not!
> 
> > for item in fileinput.input(s2):
> >       if 'penelope' in item:
> >          print(item)
> >
> > But still got the same errors even though the assignment of the path variable seems to have worked:
> >
> [snip]
> 
> Try this:
> 
>      filenames = os.listdir('/Projects/P5/shortstories/')
>      paths = [os.join('/Projects/P5/shortstories/', name) for name in names]
>      for item in fileinput.input(paths):

I have struggled with this for several hours and not made much progress. I was not sure if your 'names' variable was supposed to be the same as 'filenames'. Also, it should be 'os.path.join', not os.join. Anyway, I thought you had some good ideas so I worked with them but as I say I keep getting stuck at one particular point. Here is the current version of my code:

# -*- coding: utf-8 -*-
import os
import fileinput

path1 = os.path.join('Projects', 'P5', 'shortstories', '/')
path2 = os.path.join('Projects', 'P5')
targets = os.listdir(path1)
path3 = ((path1 + target) for target in targets)
path4 = os.path.join(path2,'list_stories')

with open(path4) as arrows:
    quiver = arrows.readlines()
<snip>

And here is my error message:

In [112]: %run algo_h1.py
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
/home/malikarumi/Projects/algo_h1.py in <module>()
      9 path4 = os.path.join(path2,'list_stories')
     10 
---> 11 with open(path4) as arrows:
     12     quiver = arrows.readlines()
     13     for arrow in quiver:

FileNotFoundError: [Errno 2] No such file or directory: 'Projects/P5/list_stories'

I have tried many different ways but can't get python to find list_stories, which is the list of story names I want to find in the texts contained in path3. When I got a lesser version of this to work I had all relevant files in the same directory. This is a more realistic situation, but I can't make it work. Suggestions?

On a slightly different but closely related issue:

As I continued to work on this on my own, I learned that I could use the class, fileinput.FileInput, instead of fileinput.input. The supposed advantage is that there can be many simultaneous instances with the class. http://stackoverflow.com/questions/21443601/runtimeerror-input-already-active-file-loop. I tried this with a very basic version of my code, one that had worked with fileinput.input, and FileInput worked just as well. Then I wanted to try a 'with' statement, because that would take care of closing the file objects for me. I took my formulation directly from the docs, https://docs.python.org/3.4/library/fileinput.html#fileinput.FileInput, but got a NameError:

In [81]: %run algo_g3.py
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
/home/malikarumi/Projects/P5/shortstories/algo_g3.py in <module>()
      4 ss = os.listdir()
      5 
----> 6 with FileInput(files = ss) as input:
      7     if 'Penelope' in input:
      8         print(input)

NameError: name 'FileInput' is not defined

Well, I am pretty much stumped by that. If it isn't right in the docs, what hope do I have? What am I missing here? Why did I get this error?

I decided to try tinkering with the parens

with FileInput(files = (ss)) as input:

But that got me the same result

NameError: name 'FileInput' is not defined

Then I changed how FileInput was called:

with fileinput.FileInput(ss) as input:

This time I got nothing. Zip. Zero:

In [83]: %run algo_g5.py

In [84]: 

In [84]: %run algo_g5.py

In [85]: 

Then I ran a different function

In [85]: fileinput.filename()

and got

RuntimeError: no active input()

Which means the file object is closed. But when? How? As part of the with statement that got the NameError? And since it is closed, why didn't running the last iteration of my script re-open it?



From no.email at nospam.invalid  Mon Jun 15 22:04:28 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Mon, 15 Jun 2015 19:04:28 -0700
Subject: Set a flag on the function or a global?
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <87381strn6.fsf@jester.gateway.sonic.net>
 <mailman.495.1434415991.13271.python-list@python.org>
Message-ID: <87y4jks91v.fsf@jester.gateway.sonic.net>

Ethan Furman <ethan at stoneleaf.us> writes:
>>    edir_(obj) = dunders.
> `edir_` ?  What a horrible name.  I hate trailing underscores.  Too easy to miss.

They've worked ok for me at various times.  edir_u (for unfiltered) is
another idea.


From ian.g.kelly at gmail.com  Mon Jun 15 22:44:06 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 15 Jun 2015 20:44:06 -0600
Subject: Struggling with os.path.join and fileinput (was 'Path, strings,
 and lines'
In-Reply-To: <e5772193-0427-4b96-9e09-1e0f60f81d17@googlegroups.com>
References: <56ac4185-3d58-473a-ad40-99d9366a8bb3@googlegroups.com>
 <mailman.438.1434141083.13271.python-list@python.org>
 <58a13725-92f7-407b-8444-71080ada1538@googlegroups.com>
 <mailman.458.1434219928.13271.python-list@python.org>
 <e5772193-0427-4b96-9e09-1e0f60f81d17@googlegroups.com>
Message-ID: <CALwzidn_q_CdiACTAa9UDrJx8hn8WxWacZaoC1Ah0DkBLNLaxg@mail.gmail.com>

On Mon, Jun 15, 2015 at 8:00 PM, Malik Rumi <malik.a.rumi at gmail.com> wrote:
> I have struggled with this for several hours and not made much progress. I was not sure if your 'names' variable was supposed to be the same as 'filenames'. Also, it should be 'os.path.join', not os.join. Anyway, I thought you had some good ideas so I worked with them but as I say I keep getting stuck at one particular point. Here is the current version of my code:
>
> # -*- coding: utf-8 -*-
> import os
> import fileinput
>
> path1 = os.path.join('Projects', 'P5', 'shortstories', '/')

You don't need to join the trailing slash on. Also, you don't really
need os.path.join here, unless you're trying to be compatible with
systems that don't allow / as a path separator, which is probably not
worth worrying about, so you could leave out the join call here and
just write '/Projects/P5/shortstories'.

> path2 = os.path.join('Projects', 'P5')

Ditto.

> targets = os.listdir(path1)
> path3 = ((path1 + target) for target in targets)

This however is not just a constant so you *should* use os.path.join
rather than concatenation.

> path4 = os.path.join(path2,'list_stories')
>
> with open(path4) as arrows:
>     quiver = arrows.readlines()
> <snip>
>
> And here is my error message:
>
> In [112]: %run algo_h1.py
> ---------------------------------------------------------------------------
> FileNotFoundError                         Traceback (most recent call last)
> /home/malikarumi/Projects/algo_h1.py in <module>()
>       9 path4 = os.path.join(path2,'list_stories')
>      10
> ---> 11 with open(path4) as arrows:
>      12     quiver = arrows.readlines()
>      13     for arrow in quiver:
>
> FileNotFoundError: [Errno 2] No such file or directory: 'Projects/P5/list_stories'

Before you were using '/Projects/P5/list_stories'. Now as the error
message shows you are using 'Projects/P5/list_stories', which is not
the same thing. The former is an absolute path and the latter is
relative to the current directory. You can fix that by adding the
leading / to the os.path.join call above, or just include it in the
string constant as I suggested.

> As I continued to work on this on my own, I learned that I could use the class, fileinput.FileInput, instead of fileinput.input. The supposed advantage is that there can be many simultaneous instances with the class. http://stackoverflow.com/questions/21443601/runtimeerror-input-already-active-file-loop. I tried this with a very basic version of my code, one that had worked with fileinput.input, and FileInput worked just as well. Then I wanted to try a 'with' statement, because that would take care of closing the file objects for me. I took my formulation directly from the docs, https://docs.python.org/3.4/library/fileinput.html#fileinput.FileInput, but got a NameError:
>
> In [81]: %run algo_g3.py
> ---------------------------------------------------------------------------
> NameError                                 Traceback (most recent call last)
> /home/malikarumi/Projects/P5/shortstories/algo_g3.py in <module>()
>       4 ss = os.listdir()
>       5
> ----> 6 with FileInput(files = ss) as input:
>       7     if 'Penelope' in input:
>       8         print(input)
>
> NameError: name 'FileInput' is not defined
>
> Well, I am pretty much stumped by that. If it isn't right in the docs, what hope do I have? What am I missing here? Why did I get this error?

How did you import it? If you just did 'import fileinput' then the
name that you imported is just the name of the module, fileinput, so
you would have to write 'fileinput.FileInput', not just 'FileInput'.
If OTOH you had used 'from fileinput import FileInput', then the name
you would have imported would be the name of the class, and in that
case the above should be correct.

> I decided to try tinkering with the parens
>
> with FileInput(files = (ss)) as input:
>
> But that got me the same result
>
> NameError: name 'FileInput' is not defined

As the error says, the issue is that the name 'FileInput' is not
defined, not anything to do with the arguments that you're passing to
it. As it happens, the change that you made was a no-op. The parens
only affect grouping, and the grouping is the same as before.

> Then I changed how FileInput was called:
>
> with fileinput.FileInput(ss) as input:
>
> This time I got nothing. Zip. Zero:

Because it worked this time, meaning that it successfully opened the
files, but then it failed to find the string that you were searching
for. Your "if 'Penelope' in input" conditional will iterate over each
line in input, and if it finds a line that is exactly 'Penelope'
without any trailing newline, then it will print the line. You
probably wanted this instead:

with fileinput.FileInput(ss) as input:
    for line in input:
        if 'Penelope' in line:
            print(line)

> Then I ran a different function
>
> In [85]: fileinput.filename()
>
> and got
>
> RuntimeError: no active input()
>
> Which means the file object is closed. But when? How? As part of the with statement that got the NameError? And since it is closed, why didn't running the last iteration of my script re-open it?

This is after the with statement above, so the fileinput object will
have been closed when the with statement was exited.


From python at mrabarnett.plus.com  Mon Jun 15 22:51:13 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 16 Jun 2015 03:51:13 +0100
Subject: Struggling with os.path.join and fileinput (was 'Path, strings,
 and lines'
In-Reply-To: <e5772193-0427-4b96-9e09-1e0f60f81d17@googlegroups.com>
References: <56ac4185-3d58-473a-ad40-99d9366a8bb3@googlegroups.com>
 <mailman.438.1434141083.13271.python-list@python.org>
 <58a13725-92f7-407b-8444-71080ada1538@googlegroups.com>
 <mailman.458.1434219928.13271.python-list@python.org>
 <e5772193-0427-4b96-9e09-1e0f60f81d17@googlegroups.com>
Message-ID: <557F8F21.6000502@mrabarnett.plus.com>

On 2015-06-16 03:00, Malik Rumi wrote:
> On Saturday, June 13, 2015 at 1:25:52 PM UTC-5, MRAB wrote:
>> On 2015-06-13 05:48, Malik Rumi wrote:
>> > On Friday, June 12, 2015 at 3:31:36 PM UTC-5, Ian wrote:
>> >> On Fri, Jun 12, 2015 at 1:39 PM, Malik Rumi wrote:
>> >> >  I am trying to find a list of strings in a directory of files. Here is my code:
>> >> >
>> >> > # -*- coding: utf-8 -*-
>> >> > import os
>> >> > import fileinput
>> >> >
>> >> > s2 = os.listdir('/home/malikarumi/Projects/P5/shortstories')
>> >>
>> >> Note that the filenames that will be returned here are not fully
>> >> qualified: you'll just get filename.txt, not
>> >> /home/.../shortstories/filename.txt.
>> >>
>> >
>> > Yes, that is what I want.
>> >
>> >> > for line in lines:
>> >> >      for item in fileinput.input(s2):
>> >>
>> >> fileinput doesn't have the context of the directory that you listed
>> >> above, so it's just going to look in the current directory.
>> >
>> > Can you explain a little more what you mean by fileinput lacking the context of s4?
>> >
>> listdir returns the names of the files that are in the folder, not
>> their paths.
>>
>> If you give fileinput only the names of the files, it'll assume they're
>> in the current folder (directory), which they (probably) aren't. You
>> need to give fileinput the complete _paths_ of the files, not just
>> their names.
>>
>> >>
>> >> >          if line in item:
>> >> >             with open(line + '_list', 'a+') as l:
>> >> >                 l.append(filename(), filelineno(), line)
>> >>
>> >> Although it's not the problem at hand, I think you'll find that you
>> >> need to qualify the filename() and filelineno() function calls with
>> >> the fileinput module.
>> >
>> > By 'qualify', do you mean something like
>> > l.append(fileinput.filename())?
>> >
>> >>
>> >> > FileNotFoundError: [Errno 2] No such file or directory: 'THE LAND OF LOST TOYS~'
>> >>
>> >> And here you can see that it's failing to find the file because it's
>> >> looking in the wrong directory. You can use the os.path.join function
>> >> to add the proper directory path to the filenames that you pass to
>> >> fileinput.
>> >
>> > I tried new code:
>> >
>> > # -*- coding: utf-8 -*-
>> > import os
>> > import fileinput
>> >
>> >
>> os.join _returns its result.
>>
>> > os.path.join('/Projects/Pipeline/4 Transforms', '/Projects/P5/shortstories/')
>> > s2 = os.listdir('/Projects/P5/shortstories/')
>>
>> At this point, s2 contains a list of _names_.
>>
>> You pass those names to fileinput.input, but where are they? In which
>> folder? It assumes they're in the current folder (directory), but
>> they're not!
>>
>> > for item in fileinput.input(s2):
>> >       if 'penelope' in item:
>> >          print(item)
>> >
>> > But still got the same errors even though the assignment of the path variable seems to have worked:
>> >
>> [snip]
>>
>> Try this:
>>
>>      filenames = os.listdir('/Projects/P5/shortstories/')
>>      paths = [os.join('/Projects/P5/shortstories/', name) for name in names]
>>      for item in fileinput.input(paths):
>
> I have struggled with this for several hours and not made much progress. I was not sure if your 'names' variable was supposed to be the same as 'filenames'. Also, it should be 'os.path.join', not os.join.

Yes on both points. Apologies.

Anyway, I thought you had some good ideas so I worked with them but as I 
say I keep getting stuck at one particular point. Here is the current 
version of my code:
>
> # -*- coding: utf-8 -*-
> import os
> import fileinput
>
> path1 = os.path.join('Projects', 'P5', 'shortstories', '/')
> path2 = os.path.join('Projects', 'P5')
> targets = os.listdir(path1)
> path3 = ((path1 + target) for target in targets)
> path4 = os.path.join(path2,'list_stories')
>
> with open(path4) as arrows:
>      quiver = arrows.readlines()
> <snip>
>
> And here is my error message:
>
> In [112]: %run algo_h1.py
> ---------------------------------------------------------------------------
> FileNotFoundError                         Traceback (most recent call last)
> /home/malikarumi/Projects/algo_h1.py in <module>()
>        9 path4 = os.path.join(path2,'list_stories')
>       10
> ---> 11 with open(path4) as arrows:
>       12     quiver = arrows.readlines()
>       13     for arrow in quiver:
>
> FileNotFoundError: [Errno 2] No such file or directory: 'Projects/P5/list_stories'
>
So it can't find 'Projects/P5/list_stories'. That's a relative path (it
doesn't begin at the root ('/')), so it starts look in the current
directory.

Should it be '/home/malikarumi/Projects/P5/list_stories'?

If that's the correct full path, but '/home/malikarumi' isn't the
current directory, then it won't find it.

> I have tried many different ways but can't get python to find list_stories, which is the list of story names I want to find in the texts contained in path3. When I got a lesser version of this to work I had all relevant files in the same directory. This is a more realistic situation, but I can't make it work. Suggestions?
>
> On a slightly different but closely related issue:
>
> As I continued to work on this on my own, I learned that I could use the class, fileinput.FileInput, instead of fileinput.input. The supposed advantage is that there can be many simultaneous instances with the class. http://stackoverflow.com/questions/21443601/runtimeerror-input-already-active-file-loop. I tried this with a very basic version of my code, one that had worked with fileinput.input, and FileInput worked just as well. Then I wanted to try a 'with' statement, because that would take care of closing the file objects for me. I took my formulation directly from the docs, https://docs.python.org/3.4/library/fileinput.html#fileinput.FileInput, but got a NameError:
>
> In [81]: %run algo_g3.py
> ---------------------------------------------------------------------------
> NameError                                 Traceback (most recent call last)
> /home/malikarumi/Projects/P5/shortstories/algo_g3.py in <module>()
>        4 ss = os.listdir()
>        5
> ----> 6 with FileInput(files = ss) as input:
>        7     if 'Penelope' in input:
>        8         print(input)
>
> NameError: name 'FileInput' is not defined
>
> Well, I am pretty much stumped by that. If it isn't right in the docs, what hope do I have? What am I missing here? Why did I get this error?
>
You imported the module 'fileinput':

     import fileinput

but did you tell it where 'FileInput' comes from? :-)

You should be explicit and say 'fileinput.FileInput'.

Alternatively, you could say:

     from fileinput import FileInput

and then you refer to it as just 'FileInput'.

> I decided to try tinkering with the parens
>
> with FileInput(files = (ss)) as input:
>
> But that got me the same result
>
> NameError: name 'FileInput' is not defined
>
> Then I changed how FileInput was called:
>
> with fileinput.FileInput(ss) as input:
>
> This time I got nothing. Zip. Zero:
>
> In [83]: %run algo_g5.py
>
> In [84]:
>
> In [84]: %run algo_g5.py
>
> In [85]:
>
> Then I ran a different function
>
> In [85]: fileinput.filename()
>
> and got
>
> RuntimeError: no active input()
>
> Which means the file object is closed. But when? How? As part of the with statement that got the NameError? And since it is closed, why didn't running the last iteration of my script re-open it?
>
It doesn't mean that the file is closed, it means that it's not
currently reading from a file, which will also be the case if you never
opened one!



From nestorjb at gmail.com  Mon Jun 15 23:31:51 2015
From: nestorjb at gmail.com (=?UTF-8?B?TsOpc3RvciBCb3Njw6Fu?=)
Date: Mon, 15 Jun 2015 23:01:51 -0430
Subject: Trying to configure Apache and Python 2.7 on Red Hat I get 403
 Forbidden
In-Reply-To: <CABD09x9JQrb0dUXf7JNbqJT7OsDDgM6mftU9PcBU8YFP4PXdvQ@mail.gmail.com>
References: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>
 <CABD09x9JQrb0dUXf7JNbqJT7OsDDgM6mftU9PcBU8YFP4PXdvQ@mail.gmail.com>
Message-ID: <CALyLaQKbeQxQwgxQfSN=czWDOsYx1zz2b9oMzbt7+Y7eQ7QccQ@mail.gmail.com>

Tried it and I keep having the same error. Isn't there a log file where I
can check what is causing this?

Regards,

N?stor

On Sat, Jun 13, 2015 at 8:47 AM, zhou weitao <zhouwtlord at gmail.com> wrote:

> selinux is causing this, I guess. Please try run *setenforce 1* to bypass
> it firstly. If it works then google the related configuration.
>
>
> from ex-redhatter
>
> 2015-06-12 20:51 GMT+08:00 N?stor Bosc?n <nestorjb at gmail.com>:
>
>> Hi
>>
>> I've been trying to configure Apache and Python 2.7 on Red Hat. I've
>> tried the different configurations i've seen on the web, I've given chmod
>> -R 777 tu my python code but I still get 403 Forbidden and I don't get any
>> errors.
>>
>> Regards,
>>
>> N?stor
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150615/38dc45ff/attachment.html>

From breamoreboy at yahoo.co.uk  Tue Jun 16 02:06:30 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 16 Jun 2015 07:06:30 +0100
Subject: Set a flag on the function or a global?
In-Reply-To: <557f6676$0$21718$c3e8da3@news.astraweb.com>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
Message-ID: <mloed7$ih1$1@ger.gmane.org>

On 16/06/2015 00:57, Steven D'Aprano wrote:
> I have a function in a module which is intended to be used by importing
> that name alone, then used interactively:
>
>      from module import edir
>      edir(args)
>
>
> edir is an enhanced version of dir, and one of the enhancements is that
> you can filter out dunder methods. I have reason to believe that people
> are split on their opinion on whether dunder methods should be shown by
> default or not: some people want to see them, others do not. Since edir
> is meant to be used interactively, I want to give people a setting to
> control whether they get dunders by default or not.
>
> I have two ideas for this, a module-level global, or a flag set on the
> function object itself. Remember that the usual way of using this will be
> "from module import edir", there are two obvious ways to set the global:
>
> import module
> module.dunders = False
>
> # -or-
>
> edir.__globals__['dunders'] = False
>
>
> Alternatively, I can use a flag set on the function object itself:
>
> edir.dunders = False
>
>
> Naturally you can always override the default by explicitly specifying a
> keyword argument edir(obj, dunders=flag).
>
> Thoughts and feedback? Please vote: a module global, or a flag on the
> object? Please give reasons, and remember that the function is intended
> for interactive use.
>
>

For interactive use I'd be perfectly happy with just the keyword 
argument.  Why bother toggling something when I can explicitly set it in 
the call each and every time?  If I have to choose it's a flag on the 
object, just no competition.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From steve+comp.lang.python at pearwood.info  Tue Jun 16 04:18:27 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 16 Jun 2015 18:18:27 +1000
Subject: Set a flag on the function or a global?
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <4449bc52-4dba-486c-b426-7095e228a316@googlegroups.com>
 <mailman.494.1434414969.13271.python-list@python.org>
Message-ID: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>

On Tuesday 16 June 2015 10:35, MRAB wrote:

> On 2015-06-16 01:24, sohcahtoa82 at gmail.com wrote:

>> Using a keyword argument for the edir function is the most intuitive
>> and easy to read, IMO.

edir() has a keyword argument: edir(x, dunders=False) suppresses the return 
of dunder names. But since the primary purpose of [e]dir is, in my opinion, 
to be used interactively, needing to type an extra 15 characters to hide 
dunders is too inconvenient.


>> Also, if two people are working on the same script, it could create
>> problems if one person wants to filter them, but the other doesn't.
>> That would create a state that they would both have to monitor and
>> keep setting back and forth, rather than each one just setting an
>> argument on their calls.
>>
> But it's meant to be used interactively. If they're using it in a
> script, they'd most likely set the argument appropriately.

Yes.

The primary use-case (at least *my* use-case, and hopefully others) is to 
have "from module import edir as dir" in their Python startup file. That 
means that when running interactively, they will get the enhanced version of 
dir, but when running a script or an application they'll just get the 
regular one.

(Ideally, the regular one will eventually gain the same superpowers as edir 
has, but that's a discussion for another day.)

Besides, apart from the inspect module, which probably shouldn't, who uses 
dir() programmatically?

(If you do, you'll be glad to hear that edir() behaves the same as regular 
dir() by default.)


-- 
Steve



From info at egenix.com  Tue Jun 16 04:19:21 2015
From: info at egenix.com (eGenix Team: M.-A. Lemburg)
Date: Tue, 16 Jun 2015 10:19:21 +0200
Subject: ANN: eGenix pyOpenSSL Distribution 0.13.10
Message-ID: <557FDC09.8050608@egenix.com>

________________________________________________________________________
ANNOUNCING

                   eGenix.com pyOpenSSL Distribution

                            Version 0.13.10

            An easy-to-install and easy-to-use distribution
            of the pyOpenSSL Python interface for OpenSSL -
           available for Windows, Mac OS X and Unix platforms


This announcement is also available on our web-site for online reading:
http://www.egenix.com/company/news/eGenix-pyOpenSSL-Distribution-0.13.10.html

________________________________________________________________________
INTRODUCTION

The eGenix.com pyOpenSSL Distribution includes everything you need to
get started with SSL in Python.

It comes with an easy-to-use installer that includes the most recent
OpenSSL library versions in pre-compiled form, making your application
independent of OS provided OpenSSL libraries:

    http://www.egenix.com/products/python/pyOpenSSL/

pyOpenSSL is an open-source Python add-on that allows writing SSL/TLS-
aware network applications as well as certificate management tools:

    https://launchpad.net/pyopenssl/

OpenSSL is an open-source implementation of the SSL/TLS protocol:

    http://www.openssl.org/

________________________________________________________________________
NEWS

This new release of the eGenix.com pyOpenSSL Distribution includes the
following updates:

New in OpenSSL
--------------

 * OpenSSL 1.0.1o fixes an ABI incompatibility introduced in OpenSSL
   1.0.1n, which was released on 2015-06-11.

These were the updates in eGenix pyOpenSSL 0.13.9, which we had
released on 2015-06-12:

New in eGenix pyOpenSSL
-----------------------

 * Fixed a bug in the build process which resulted in the CA bundle
   files not get installed in the OpenSSL/ package dir.

 * Added a work-around for recent pip versions not showing the
   installer output, causing an apparently hanging installation
   process. The installer will now use a timeout when entering the
   crypto confirmation and report how to fix the problem (by using an
   environment variable EGENIX_CRYPTO_CONFIRM for confirmation).

 * Updated the Mozilla CA root bundle to version 2015-04-22.

 * Various minor fixes to the web installer to make installations on
   Linux and FreeBSD more robust, having pip uninstall not remove the
   .pyc/.pyo files, intermittent error causing a source installation
   in some rare cases.

New in OpenSSL
--------------

 * Updated included OpenSSL libraries from OpenSSL 1.0.1m to
   1.0.1n. See https://www.openssl.org/news/secadv_20150611.txt ?for a
   complete list of changes. The following fixes are relevant for
   pyOpenSSL applications:

   - CVE-2015-1788: Possible infinite loop during client
     authentication, which can be used for Denial of Service (DoS)
     attacks.

   - CVE-2015-1789: X509_cmp_time does not properly check the length
     of the ASN1_TIME string and can read a few bytes out of bounds,
     which can lead to a segmentation fault.

   - CVE-2015-1790: The PKCS#7 parsing code does not handle missing
     inner EncryptedContent correctly, which can lead to a NULL
     pointer dereference on parsing.

   - CVE-2015-1792: When verifying a signedData message the CMS code
     can enter an infinite loop if presented with an unknown hash
     function OID.

   - CVE-2015-1791: If a NewSessionTicket is received by a
     multi-threaded client when attempting to reuse a previous ticket
     then a race condition can occur potentially leading to a double
     free of the ticket data.

Please see the product changelog for the full set of changes.

    http://www.egenix.com/products/python/pyOpenSSL/changelog.html


pyOpenSSL / OpenSSL Binaries Included
-------------------------------------

In addition to providing sources, we make binaries available that
include both pyOpenSSL and the necessary OpenSSL libraries for all
supported platforms: Windows, Linux, Mac OS X and FreeBSD, for x86 and
x64.

To simplify installation, we have uploaded a web installer to PyPI
which will automatically choose the right binary for your platform, so
a simple

    pip install egenix-pyopenssl

will get you the package with OpenSSL libraries installed. Please see
our installation instructions for details:

    http://www.egenix.com/products/python/pyOpenSSL/#Installation

We have also added .egg-file distribution versions of our eGenix.com
pyOpenSSL Distribution for Windows, Linux and Mac OS X to the
available download options. These make setups using e.g. zc.buildout
and other egg-file based installers a lot easier.

________________________________________________________________________
DOWNLOADS

The download archives and instructions for installing the package can
be found at:

    http://www.egenix.com/products/python/pyOpenSSL/

________________________________________________________________________
UPGRADING

Before installing this version of pyOpenSSL, please make sure that
you uninstall any previously installed pyOpenSSL version. Otherwise,
you could end up not using the included OpenSSL libs.

_______________________________________________________________________
SUPPORT

Commercial support for these packages is available from eGenix.com.
Please see

    http://www.egenix.com/services/support/

for details about our support offerings.

________________________________________________________________________
MORE INFORMATION

For more information about the eGenix pyOpenSSL Distribution, licensing
and download instructions, please visit our web-site or write to
sales at egenix.com.

About eGenix (http://www.egenix.com/):

    eGenix is a software project, consulting and product company
    focusing on expert project services and professional quality
    products for companies, Python users and developers.

Enjoy,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 16 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2015-06-10: Released mxODBC Plone/Zope DA 2.2.2   http://egenix.com/go76
2015-07-20: EuroPython 2015, Bilbao, Spain ...             34 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From steve+comp.lang.python at pearwood.info  Tue Jun 16 04:30:31 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 16 Jun 2015 18:30:31 +1000
Subject: Set a flag on the function or a global?
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <87381strn6.fsf@jester.gateway.sonic.net>
Message-ID: <557fdeab$0$1656$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 16 June 2015 10:37, Paul Rubin wrote:

> Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
>> Thoughts and feedback? Please vote: a module global, or a flag on the
>> object? Please give reasons, and remember that the function is intended
>> for interactive use.
> 
> Both are bad.  More state to remember, ugh.  Instead have separate entry
> points for filtering or not filtering the dunders.  Something like:
> 
>   edir(obj) = no dunders
>   edir_(obj) = dunders.

I certainly wouldn't call it edir_ but I'd maybe call it edir2 or ddir 
("Dunderless DIR") or just_like_builtin_dir_only_better. <wink>

Normally I would agree with you, as a matter of principle, functions which 
differ only by an argument called as a constant, particular when that 
argument is a flag, should be split into multiple functions.

That is, instead of foo(x, flag=True) and foo(x, flag=False), have foo(x) 
and foo2(x).

One disadvantage of this, though, is that it may not be self-evident which 
function does what. Who remembers the difference between all the os.exec* 
functions? Speaking of which:

py> dir(os, 'exec*')
['execl', 'execle', 'execlp', 'execlpe', 'execv', 'execve', 'execvp', 
'execvpe']


But in this case, because [e]dir is intended to be run interactively, I 
think that the convenience of a single function is more important. Most 
people will set the default setting to whatever they prefer and then 95% of 
the time just use it as given, only occasionally giving the dunders keyword 
argument when they need the opposite setting.

Or at least, that's how *I* use it.

-- 
Steve



From steve+comp.lang.python at pearwood.info  Tue Jun 16 05:07:28 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 16 Jun 2015 19:07:28 +1000
Subject: Set a flag on the function or a global?
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <CAPTjJmr9r7Ev2Dw3+fY=mzOF3dz5TaXWX07TFH9R0u7JVxrzCg@mail.gmail.com>
 <mailman.490.1434414054.13271.python-list@python.org>
Message-ID: <557fe755$0$2827$c3e8da3$76491128@news.astraweb.com>

On Tuesday 16 June 2015 10:20, Ben Finney wrote:

> Chris Angelico <rosuav at gmail.com> writes:
> 
>> On Tue, Jun 16, 2015 at 9:57 AM, Steven D'Aprano
>> <steve+comp.lang.python at pearwood.info> wrote:
>> > I can use a flag set on the function object itself:
>> >
>> > edir.dunders = False
>>
>> For most situations, the last one is extremely surprising - attributes
>> on functions aren't normally meant to be changed by outside callers,
>> it always feels wrong (they belong to the function itself).
> 
> I'm surprised by your assertion. To my mind, outside callers get simple
> and direct access to the attribute, whereas the code of the function
> itself does not have such easy access; unlike ?self? for the current
> instance of a class, there's no obvious name to use for referring to the
> function object within the function object's own code.
> 
> In what sense do they ?belong to? the function itself *more than* to
> outside callers?

I won't answer for Chris, but speaking for myself, I think that an attribute 
attached to a function is very clearly "part of" the function object in a 
way that a global is not.

A bare global gives no hint as to which function(s) it will affect:

dunders = True

We can simulate a namespace by giving the name a prefix:

edir_dunders = True
hasattr(edir, "dunders")  # False, despite the simulated namespace


but using a real namespace is much better:

edir.dunders = True
hasattr(edir, "dunders")  # True


But having said that, I think that especially in Python the implication here 
is that such an attribute is public and intended for outsiders to at least 
read, if not write.

Adding public attributes to functions is deliberately supported in Python, 
but it is greatly under-utilized.


-- 
Steve



From steve+comp.lang.python at pearwood.info  Tue Jun 16 05:15:52 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 16 Jun 2015 19:15:52 +1000
Subject: Set a flag on the function or a global?
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <CAPTjJmr9r7Ev2Dw3+fY=mzOF3dz5TaXWX07TFH9R0u7JVxrzCg@mail.gmail.com>
 <mailman.492.1434414286.13271.python-list@python.org>
Message-ID: <557fe949$0$1655$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 16 June 2015 10:24, Ron Adam wrote:

> Another way is to make it an object with a __call__ method.
> 
> The the attribute can be accessed from both outside and inside dependably.

That's what functions are, objects with a __call__ method:


py> (lambda: None).__call__
<method-wrapper '__call__' of function object at 0xb7301a04>


One slight disadvantage is that functions don't take a "self" parameter by 
default, which means they have to refer to themselves by name:

def spam():
    print spam.attr


Here's a fun hack:

py> from types import MethodType
py> def physician(func):
...     # As in, "Physician, Know Thyself" :-)
...     return MethodType(func, func)
... 
py> @physician
... def spam(this, n):
...     return this.food * n
... 
py> spam.__func__.food = "spam-and-eggs "
py> spam(3)
'spam-and-eggs spam-and-eggs spam-and-eggs '


Alas, you cannot write directly to the method object itself :-(



-- 
Steve



From steve+comp.lang.python at pearwood.info  Tue Jun 16 05:23:03 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 16 Jun 2015 19:23:03 +1000
Subject: Trying to configure Apache and Python 2.7 on Red Hat I get 403
 Forbidden
References: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>
 <CABD09x9JQrb0dUXf7JNbqJT7OsDDgM6mftU9PcBU8YFP4PXdvQ@mail.gmail.com>
 <mailman.499.1434425540.13271.python-list@python.org>
Message-ID: <557feaf7$0$1655$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 16 June 2015 13:31, N?stor Bosc?n wrote:

> Tried it and I keep having the same error. Isn't there a log file where I
> can check what is causing this?

Probably. Have you googled for "selinux log file"? 

We're not experts on SELinux, this is a Python mailing list, not a 
specialist SELinux one, so most of us here would have to google for 
information of SELinux in order to answer.

What happens if you try to just download the python file instead of run it? 
Do you still get a 404? If so, that proves it's not a Python issue, it's 
something to do with Apache, file permissions, or SELinux.


-- 
Steve



From steve+comp.lang.python at pearwood.info  Tue Jun 16 05:28:44 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 16 Jun 2015 19:28:44 +1000
Subject: Set a flag on the function or a global?
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <mailman.500.1434434802.13271.python-list@python.org>
Message-ID: <557fec4d$0$1664$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 16 June 2015 16:06, Mark Lawrence wrote:

> On 16/06/2015 00:57, Steven D'Aprano wrote:
[...]
>> Naturally you can always override the default by explicitly specifying a
>> keyword argument edir(obj, dunders=flag).
>>
>> Thoughts and feedback? Please vote: a module global, or a flag on the
>> object? Please give reasons, and remember that the function is intended
>> for interactive use.
>>
>>
> 
> For interactive use I'd be perfectly happy with just the keyword
> argument.  Why bother toggling something when I can explicitly set it in
> the call each and every time?  If I have to choose it's a flag on the
> object, just no competition.

The idea is to set the default behaviour: does edir(obj) display dunder 
methods by default or not? I've found that many people don't want to see the 
dunder methods, and find dir() less pleasant or useful because it shows 
them. Others disagree and want to see them. So you can set the default 
behaviour you want, and then only worry about giving an explicit argument 
when you want the opposite.

There is no intention for people to toggle the flag between calls!

# don't do this!
edir.dunders = True
edir(x)
edir.dunders = False
edir(y)


# do this instead
edir.dunders = True  # if that is your preferred setting
edir(x)
edir(y, dunders=False)




-- 
Steve



From jonas at wielicki.name  Tue Jun 16 06:00:43 2015
From: jonas at wielicki.name (Jonas Wielicki)
Date: Tue, 16 Jun 2015 12:00:43 +0200
Subject: Set a flag on the function or a global?
In-Reply-To: <557f6676$0$21718$c3e8da3@news.astraweb.com>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
Message-ID: <557FF3CB.2030600@wielicki.name>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512



On 16.06.2015 01:57, Steven D'Aprano wrote:
> Alternatively, I can use a flag set on the function object itself:
> 
> edir.dunders = False
> 
> 
> Naturally you can always override the default by explicitly
> specifying a keyword argument edir(obj, dunders=flag).
> 
> Thoughts and feedback? Please vote: a module global, or a flag on
> the object? Please give reasons, and remember that the function is
> intended for interactive use.

I like a flag on the object, and this would be my implementation:

```
def edir_impl():
    # implementation of edir
    pass

class Edir(object):
    def __init__(self):
        self.kwargs = {}

    def __getattr__(self, name):
        return self.kwargs[name]

    def __setattr__(self, name, value):
        self.kwargs[name] = value

    def __delattr__(self, name):
        del self.kwargs[name]

    def __call__(self, *args, **kwargs):
        kwargs_to_use = self.kwargs.copy()
        kwargs_to_use.update(kwargs)
        edir_impl(*args, **kwargs_to_use)

edir = Edir()
```
(untested, but you get the idea, hopefully)

This also allows to have several versions of edir around in a shell
with different default settings.

regards,
jwi
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJVf/PLAAoJEMBiAyWXYliKFM4P/jf/ZDZFacrRuk29tNzNFjA2
6UT4GURF6UX+C129gGgaqxpDY9oD9herfUpqbFxfNWq4Rye9MX+xkv3uQUsP5BXE
G7VjrSC0qd1GmdwFxdIQEb1fa6LgMmOnnYe/sIw5XWF2Rtbkqlisw7ZO3YDsWT9d
cadoEh0ShFbM6z4MUDEd8tu6RQ/j61v906TwsVgY+4uI8WWOzqHumM7pkWHxP2ns
3b/2ijQJNNLwH3UfbPBo3YhJWt0c+ACzuxr5MamVHUJBZpNatcszti3mkn2CtDCb
m5Kc0gO0RxHVaHV4RWrjYlcllhysQ/zeAuOz1PHEqevlitj8z1cBy3p4sfz8GOZe
XAKChaZHVAq7eTGzgtLksBQ0dujLxy9zLWiHfeLfMnTyZoYFKVCM2rlSYutnkI0f
7VdaaKStVpqkXeJHX1n1LvHBOroQCFtUFn2F0FylpR/nBA7ar9epZYiCWbrb21xO
DYs8T7dC7DI1waLeXJ2JbXiXqh6cQ9Fy86fm+VE1lmkJ6LvboRCN9eWMQ1o1+RtM
Dla6CvH2zxgW2u3HZCIoU9TusYaNLR9kSxqCMc2yLuMiUQpj0HfjZCR+ZP2/9phS
wx+qZx+5X25ybXlCZRtlvzb409BclOJ7JOynboX4MU4M9sSCPJhb5/ZcM79r7nJa
o0y06nE5eReu71TMwmDW
=r9uV
-----END PGP SIGNATURE-----


From nestorjb at gmail.com  Tue Jun 16 06:11:57 2015
From: nestorjb at gmail.com (=?UTF-8?B?TsOpc3RvciBCb3Njw6Fu?=)
Date: Tue, 16 Jun 2015 05:41:57 -0430
Subject: Trying to configure Apache and Python 2.7 on Red Hat I get 403
 Forbidden
In-Reply-To: <557feaf7$0$1655$c3e8da3$5496439d@news.astraweb.com>
References: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>
 <CABD09x9JQrb0dUXf7JNbqJT7OsDDgM6mftU9PcBU8YFP4PXdvQ@mail.gmail.com>
 <mailman.499.1434425540.13271.python-list@python.org>
 <557feaf7$0$1655$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CALyLaQK5=HQcuHkTm9QHcs5p05NXvJ7j-wpi9GffqDrcA14GVg@mail.gmail.com>

Hi

I disabled selinux completely and the page worked.

Regards,

Nestor

On Tuesday, June 16, 2015, Steven D'Aprano <
steve+comp.lang.python at pearwood.info> wrote:

> On Tuesday 16 June 2015 13:31, N?stor Bosc?n wrote:
>
> > Tried it and I keep having the same error. Isn't there a log file where I
> > can check what is causing this?
>
> Probably. Have you googled for "selinux log file"?
>
> We're not experts on SELinux, this is a Python mailing list, not a
> specialist SELinux one, so most of us here would have to google for
> information of SELinux in order to answer.
>
> What happens if you try to just download the python file instead of run it?
> Do you still get a 404? If so, that proves it's not a Python issue, it's
> something to do with Apache, file permissions, or SELinux.
>
>
> --
> Steve
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150616/e8ecef5f/attachment.html>

From mal at europython.eu  Tue Jun 16 06:21:36 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Tue, 16 Jun 2015 12:21:36 +0200
Subject: EuroPython 2015: Schedule online
Message-ID: <557FF8B0.9090409@europython.eu>

We are happy to announce the schedule for EuroPython 2015 in Bilbao.

With over 200 sessions, over 180 speakers, 5 days of talks, trainings,
keynotes, lightning talks and open spaces, followed by 2 days of
sprints, EuroPython will be one of the most exciting and vibrant
Python events this year:

                 *** EuroPython 2015 Schedule ***

         https://ep2015.europython.eu/p3/schedule/ep2015/

The schedule is available in table and list format. Please note
that we are still applying changes to the slots and will also add
some more interesting special sessions to it in the coming days.

If you want to join the fun, be sure to get your tickets as soon as
possible, since we are starting to sell out quickly:

    https://ep2015.europython.eu/en/registration/

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From subhabrata.banerji at gmail.com  Tue Jun 16 06:46:22 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Tue, 16 Jun 2015 03:46:22 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.487.1434385760.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com><2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
Message-ID: <d352894a-0088-456d-a321-ef191b6a38ae@googlegroups.com>

On Monday, June 15, 2015 at 9:59:33 PM UTC+5:30, Laura Creighton wrote:
> I don't have a windows system, so my knowledge of such things is
> minimal.  But looks like this person had the same problem you have,
> and got some suggestions on how to fix it.
> 
> http://stackoverflow.com/questions/12127869/error-msvcp90-dll-no-such-file-or-directory-even-though-microsoft-visual-c
> 
> But your code is working fine now?  There is no point in packaging
> something that doesn't run.
> 
> Laura

Hi,

Thank you for your help. Yes now the file is giving output as I am calling it in command prompt. I had downloaded Microsoft Visual Studio 2008 Redistributable.
I have MS VC 10.0 installed so my corresponding CRT file is 100 and not 90. I am checking this and working out. 
Meanwhile got another post http://stackoverflow.com/questions/323424/py2exe-fails-to-generate-an-executable
and pasted libiomp5md.dll from numpy now it is giving NLQ3.exe in dist and generating few errors in a corresponding log file.

Regards,
Subhabrata Banerjee. 


From ron3200 at gmail.com  Tue Jun 16 07:02:03 2015
From: ron3200 at gmail.com (Ron Adam)
Date: Tue, 16 Jun 2015 07:02:03 -0400
Subject: Set a flag on the function or a global?
In-Reply-To: <557fe949$0$1655$c3e8da3$5496439d@news.astraweb.com>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <CAPTjJmr9r7Ev2Dw3+fY=mzOF3dz5TaXWX07TFH9R0u7JVxrzCg@mail.gmail.com>
 <mailman.492.1434414286.13271.python-list@python.org>
 <557fe949$0$1655$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mlovnd$94p$1@ger.gmane.org>



On 06/16/2015 05:15 AM, Steven D'Aprano wrote:
> On Tuesday 16 June 2015 10:24, Ron Adam wrote:
>
>> >Another way is to make it an object with a __call__ method.
>> >
>> >The the attribute can be accessed from both outside and inside dependably.
> That's what functions are, objects with a __call__ method:
>
>
> py> (lambda: None).__call__
> <method-wrapper '__call__' of function object at 0xb7301a04>

Yes ;-)


> One slight disadvantage is that functions don't take a "self" parameter by
> default, which means they have to refer to themselves by name:
>
> def spam():
>      print spam.attr
>
>
> Here's a fun hack:
>
> py> from types import MethodType
> py> def physician(func):
> ...     # As in, "Physician, Know Thyself":-)
> ...     return MethodType(func, func)
> ...
> py> @physician
> ... def spam(this, n):
> ...     return this.food * n
> ...
> py> spam.__func__.food = "spam-and-eggs "
> py> spam(3)
> 'spam-and-eggs spam-and-eggs spam-and-eggs'


How about this?:  (paste it into your console)

#---------------------
import sys
class EDir:
     long = False
     def __call__(self, obj=None):
         if obj == None:
             d = sys._getframe(1).f_locals
         else:
             d = dir(obj)
         return [x for x in d if self.long or not x.startswith("_")]


edir = EDir()

edir()

edir(edir)

edir.long = True
edir(edir)

edir.long = False
edir(edir)
#----------------------


I didn't test how that works from other modules or in nested scopes.  Also 
replacing None with a unique sentinel object may be better so dir(None) 
will work.

Cheers,
    Ron









From oscar.j.benjamin at gmail.com  Tue Jun 16 08:45:01 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 16 Jun 2015 13:45:01 +0100
Subject: Set a flag on the function or a global?
In-Reply-To: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <4449bc52-4dba-486c-b426-7095e228a316@googlegroups.com>
 <mailman.494.1434414969.13271.python-list@python.org>
 <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
Message-ID: <CAHVvXxQF7nqfo-o3pjzipR0ki1hHnMOQvk=u3bSbFiSoB8MmQg@mail.gmail.com>

On 16 June 2015 at 09:18, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
>
> The primary use-case (at least *my* use-case, and hopefully others) is to
> have "from module import edir as dir" in their Python startup file. That
> means that when running interactively, they will get the enhanced version of
> dir, but when running a script or an application they'll just get the
> regular one.
>
> (Ideally, the regular one will eventually gain the same superpowers as edir
> has, but that's a discussion for another day.)
>
> Besides, apart from the inspect module, which probably shouldn't, who uses
> dir() programmatically?
>
> (If you do, you'll be glad to hear that edir() behaves the same as regular
> dir() by default.)

What's the point in giving edir two modes if one of them is the same
as dir? You could just do "from module import edir" and then use
dir/edir as desired.

Personally I just use ipython's tab-completion instead of dir. It
shows the dunders if you first type underscores but hides them
otherwise e.g.:

In [1]: a = []

In [2]: a.<tab>
a.append   a.count    a.extend   a.index    a.insert   a.pop
a.remove   a.reverse  a.sort

In [2]: a.__<tab>
a.__add__           a.__format__        a.__imul__          a.__new__
         a.__setslice__
a.__class__         a.__ge__            a.__init__
a.__reduce__        a.__sizeof__
a.__contains__      a.__getattribute__  a.__iter__
a.__reduce_ex__     a.__str__
a.__delattr__       a.__getitem__       a.__le__            a.__repr__
         a.__subclasshook__
a.__delitem__       a.__getslice__      a.__len__           a.__reversed__
a.__delslice__      a.__gt__            a.__lt__            a.__rmul__
a.__doc__           a.__hash__          a.__mul__           a.__setattr__
a.__eq__            a.__iadd__          a.__ne__            a.__setitem__


--
Oscar


From torriem at gmail.com  Tue Jun 16 09:44:57 2015
From: torriem at gmail.com (Michael Torrie)
Date: Tue, 16 Jun 2015 07:44:57 -0600
Subject: Set a flag on the function or a global?
In-Reply-To: <85oakgtsfd.fsf@benfinney.id.au>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <CAPTjJmr9r7Ev2Dw3+fY=mzOF3dz5TaXWX07TFH9R0u7JVxrzCg@mail.gmail.com>
 <85oakgtsfd.fsf@benfinney.id.au>
Message-ID: <55802859.30607@gmail.com>

On 06/15/2015 06:20 PM, Ben Finney wrote:
> I'm surprised by your assertion. To my mind, outside callers get simple
> and direct access to the attribute, whereas the code of the function
> itself does not have such easy access; unlike ?self? for the current
> instance of a class, there's no obvious name to use for referring to the
> function object within the function object's own code.

Of course it has access, and it's obvious and easy as well:

def foo():
    foo.flag = True

The only thing I'm not sure of is a clean way to test for the
attribute's existence.  The __init__.py of a package, or the
initialization code of the module could preset the attribute, though.



From subhabrata.banerji at gmail.com  Tue Jun 16 09:56:12 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Tue, 16 Jun 2015 06:56:12 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.487.1434385760.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com><2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
Message-ID: <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>

On Monday, June 15, 2015 at 9:59:33 PM UTC+5:30, Laura Creighton wrote:
> I don't have a windows system, so my knowledge of such things is
> minimal.  But looks like this person had the same problem you have,
> and got some suggestions on how to fix it.
> 
> http://stackoverflow.com/questions/12127869/error-msvcp90-dll-no-such-file-or-directory-even-though-microsoft-visual-c
> 
> But your code is working fine now?  There is no point in packaging
> something that doesn't run.
> 
> Laura

I have mostly solved the problem. But there are two issues making me slightly confusing:
i) In a bigger code I am calling scipy and after building .exe as I am calling 
\dist\...exe it is giving ImportError: No module named _ufuncs_cxx
I have checked the resolution http://stackoverflow.com/questions/20169728/exe-created-with-py2exe-doesnt-work-and-returns-logfile-with-errors
but it is not giving much help.
ii) In a class how may I include if __name__ == "__main__": with multiple methods? But I think this is easy question there should be lot of web help. 

If anyone may kindly suggest.

Regards,
Subhabrata Banerjee. 
 


From torriem at gmail.com  Tue Jun 16 09:56:21 2015
From: torriem at gmail.com (Michael Torrie)
Date: Tue, 16 Jun 2015 07:56:21 -0600
Subject: Set a flag on the function or a global?
In-Reply-To: <557F6BAF.3080702@stoneleaf.us>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <557F6BAF.3080702@stoneleaf.us>
Message-ID: <55802B05.1050807@gmail.com>

On 06/15/2015 06:19 PM, Ethan Furman wrote:
> Setting a global on the module (which I may not have, and probably
> didn't, import) for only one function is overkill.

What do you mean?  Even if you pull in just one function from the
module on an import, the module's initialization code still runs.






From __peter__ at web.de  Tue Jun 16 09:59:32 2015
From: __peter__ at web.de (Peter Otten)
Date: Tue, 16 Jun 2015 15:59:32 +0200
Subject: Set a flag on the function or a global?
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
Message-ID: <mlpa45$q6l$1@ger.gmane.org>

Steven D'Aprano wrote:

> I have a function in a module which is intended to be used by importing
> that name alone, then used interactively:
> 
>     from module import edir
>     edir(args)
> 
> 
> edir is an enhanced version of dir, and one of the enhancements is that
> you can filter out dunder methods. I have reason to believe that people
> are split on their opinion on whether dunder methods should be shown by
> default or not: some people want to see them, others do not. Since edir
> is meant to be used interactively, I want to give people a setting to
> control whether they get dunders by default or not.
> 
> I have two ideas for this, a module-level global, or a flag set on the
> function object itself. Remember that the usual way of using this will be
> "from module import edir", there are two obvious ways to set the global:
> 
> import module
> module.dunders = False
> 
> # -or-
> 
> edir.__globals__['dunders'] = False
> 
> 
> Alternatively, I can use a flag set on the function object itself:
> 
> edir.dunders = False
> 
> 
> Naturally you can always override the default by explicitly specifying a
> keyword argument edir(obj, dunders=flag).
> 
> Thoughts and feedback? Please vote: a module global, or a flag on the
> object? Please give reasons, and remember that the function is intended
> for interactive use.

"""
In general I'm wary of routines that take flags (such as 'swapped')
that really mean "use a different version of the function" -- these
flags are almost always passed from constants and so it would be more
efficient to have a second name for the variant function.
"""
http://legacy.python.org/search/hypermail/python-1994q2/0852.html




From ethan at stoneleaf.us  Tue Jun 16 10:57:07 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 16 Jun 2015 07:57:07 -0700
Subject: Set a flag on the function or a global?
In-Reply-To: <55802B05.1050807@gmail.com>
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <557F6BAF.3080702@stoneleaf.us> <55802B05.1050807@gmail.com>
Message-ID: <55803943.8010700@stoneleaf.us>

On 06/16/2015 06:56 AM, Michael Torrie wrote:
> On 06/15/2015 06:19 PM, Ethan Furman wrote:
>>
>> Setting a global on the module (which I may not have, and probably
>> didn't, import) for only one function is overkill.
>
> What do you mean?  Even if you pull in just one function from the
> module on an import, the module's initialization code still runs.

For me, an appropriate use of globals requires that it will be used by many other pieces of code in that module.  If only one piece of code is using it, it's not really global.

--
~Ethan~


From lac at openend.se  Tue Jun 16 11:04:57 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 16 Jun 2015 17:04:57 +0200
Subject: Creating .exe file in Python
In-Reply-To: Message from subhabrata.banerji@gmail.com of "Tue,
 16 Jun 2015 06:56:12 -0700."
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com><2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org><5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
Message-ID: <201506161504.t5GF4vEI021706@fido.openend.se>

In a message of Tue, 16 Jun 2015 06:56:12 -0700, subhabrata.banerji at gmail.com w
rites:
>ii) In a class how may I include if __name__ == "__main__": with multiple methods? But I think this is easy question there should be lot of web help. 
>
>If anyone may kindly suggest.
>
>Regards,
>Subhabrata Banerjee. 

There is something wrong in the way you are thinking about this.
You don't _want_ to stick if __name__ = "__main__":
into every class you write.

See if reading this helps.
http://effbot.org/pyfaq/tutor-what-is-if-name-main-for.htm

Laura



From davros at bellaliant.net  Tue Jun 16 11:09:52 2015
From: davros at bellaliant.net (John McKenzie)
Date: Tue, 16 Jun 2015 15:09:52 GMT
Subject: Keypress Input
References: <rBHbx.75089$MO7.40532@fx10.iad> <Tbtfx.178924$Jv.102633@fx26.iad>
Message-ID: <4%Wfx.125346$HH2.100763@fx20.iad>


 It appears that one of my posts was cut off. It contains my script but 
none of the lengthy text in front of it.

 To summarize, my set-up consists of three "massive arcade buttons" from 
Adafruit. one red, one blue, one yellow. They are connected to a Kade 
Device that is connected to a Raspberry Pi via USB cable and emulates a 
USB keyboard. The goal is to get the LED light strip attached to the Pi 
via a Blinkstick Pro and LED adapter to glow the colour of the button 
pushed. Also, I want the Pi to record how long it spent as each colour.

 For development purposes I am using an actual keyboard.

 The script brings up a blank tkinker window (expected) and the window 
seems to have focus when it is created. Pressing the keys does nothing.

 Would appreciate any insight into why it does not respond. Just look a 
few posts above (Date: Mon, 15 Jun 2015 05:15:31 GMT) and you will see my 
script.

 Tried a bunch of different approaches and in some of them, an error 
message appeared when I pressed a key. It may not have worked, but it was 
registering the key press.

 Thanks.



From subhabrata.banerji at gmail.com  Tue Jun 16 11:17:52 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Tue, 16 Jun 2015 08:17:52 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.515.1434467117.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com><2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org><5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
Message-ID: <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>

On Tuesday, June 16, 2015 at 8:35:39 PM UTC+5:30, Laura Creighton wrote:
> In a message of Tue, 16 Jun 2015 06:56:12 -0700,  writes:
> >ii) In a class how may I include if __name__ == "__main__": with multiple methods? But I think this is easy question there should be lot of web help. 
> >
> >If anyone may kindly suggest.
> >
> >Regards,
> >Subhabrata Banerjee. 
> 
> There is something wrong in the way you are thinking about this.
> You don't _want_ to stick if __name__ = "__main__":
> into every class you write.
> 
> See if reading this helps.
> http://effbot.org/pyfaq/tutor-what-is-if-name-main-for.htm
> 
> Laura

Hi,
Thanks. The scipy issue seems solved. But this silly issue is giving so much of time. I am checking. Please see a sample code, 

import sys
sys.stderr = sys.stdout
class Colors:
    def Blue(self):
        self.var="This is Blue"
        print self.var
    def Red(self):
        print self.var
        
    
       
if __name__ == "__main__":
    Colors().Blue() #THIS IS FINE
    Colors().Red() #NOT FINE

Regards,
Subhabrata Banerjee. 


From ian.g.kelly at gmail.com  Tue Jun 16 11:49:27 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 16 Jun 2015 09:49:27 -0600
Subject: Pyitect - Plugin architectural system for Python 3.0+ (feedback?)
In-Reply-To: <CAO-rxQsTP8k_nmaV80Zg2OZnbun040jpYG7_tLQpb-q22V3jSw@mail.gmail.com>
References: <CAO-rxQsTP8k_nmaV80Zg2OZnbun040jpYG7_tLQpb-q22V3jSw@mail.gmail.com>
Message-ID: <CALwzidnGexPj-2gEb6exk76T=iEo5m_gDPstjy6Y_GyLzFPsHA@mail.gmail.com>

On Mon, Jun 8, 2015 at 10:42 PM, Ben Powers <ryexander at gmail.com> wrote:
> As importlib has been added in python 3 and up I decided to use it's
> abilities to create a plugin system for truly modular development in python.
>
> Pyitect has the ability to drop in components and resolve dependencies. Even
> load different versions of a dependency if two different libraries require
> two different versions.
>
> Pyitect gives each "plugin" it won private namespace to import the
> components it needs. and nothing else.
>
> Pyitect  organizes it self around the concept of a System. Each system can
> scan and load it's own set of plugins, in each system plugins can be enabled
> individually.
>
> Pyitect even comes with a very simple event system that it uses internal to
> fire event when plugins are found/loaded/ect for the developer to use.
>
> I've written up What I think is a quite comprehensive readme to go with it.
>
> Available on PyPi https://pypi.python.org/pypi?:action=display&name=pyitect
> and Github https://github.com/Ryex/pyitect
>
> I'm looking for feedback on what flaws the system may have and where I can
> improve. I'm always open to pull requests
>
> Looking forward to what the comunity thinks of this little project of mine.
> Thanks for your input and criticism.

You need to run your readme through a spell-check, as it's riddled
with typographical errors and quite hard to read. A grammar check and
a proofreader would also be useful.

> "on_enable": "dostuff.py", // optional, runs this file when the plugin is enabled

A path to a function would feel more Pythonic.

> mode -> (OPTIONAL) defaults to import on python 3.4 and up ecec otherwise: sets the import mode

Do the plugins need to be on the sys.path in order to be imported?

> provides -> a mapping of provided component names to prefix mappings

What are the "prefix mappings" and what are they used for?

> #file.py
> from PyitectConsumes import foo
>
> class Bar(object):
>     def __init__():
>         foo("it's a good day to be a plugin")

I don't understand this example. Is "PyitectConsumes" the name of a
component or part of Pyitect? Or is "foo" the name of the component?

> plugins = [system.plugins[n][v] for n in system.plugins for v in system.plugins[n]]

What are n and v?

> Plugins are loaded on demand when a component is loaded via
>
> System.load("<component name>")

What's the difference between this and the "PyitectConsumes" import?

> ### both in both cases relative imports DO NOT WORK. the plugin folder is temporarily added to the search path so absolute imports work but relatives will not.
>
> UNLESS the name of the file is __init__.py . In this special case the plugin folder is reconsidered as a python package and relative imports work as normal.

Isn't this just how relative imports normally work? Of course you
can't use a relative import outside of a package.

> Loaded pluginss do NOT store their module object in sys.modules

What about imports of plugin-local modules that are performed inside
the plugins? Do those also get cleaned up from sys.modules?

If I have version 1 of a plugin that imports a module from its
package, and version 2 of the same plugin that imports a module from
its package, are those imports going to conflict with one another?

> System.useing
> System.enabeled_plugins

Are these misspellings actually part of the API?

Also, earlier you appeared to instantiate the System class, but now
you appear to be just accessing attributes from the class directly.
What's the distinction between the class-level state and the
instance-level state?

> Providing multiple versions of a component from the same plugin

Is there a reason to do this as opposed to simply providing multiple components?


From ian.g.kelly at gmail.com  Tue Jun 16 12:00:22 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 16 Jun 2015 10:00:22 -0600
Subject: Creating .exe file in Python
In-Reply-To: <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
Message-ID: <CALwzidkPc4_vNT6nZ_O1HAb0Pqd+_dHQBAUorC_0FRZR39mjNA@mail.gmail.com>

On Tue, Jun 16, 2015 at 9:17 AM,  <subhabrata.banerji at gmail.com> wrote:
> On Tuesday, June 16, 2015 at 8:35:39 PM UTC+5:30, Laura Creighton wrote:
>> In a message of Tue, 16 Jun 2015 06:56:12 -0700,  writes:
>> >ii) In a class how may I include if __name__ == "__main__": with multiple methods? But I think this is easy question there should be lot of web help.
>> >
>> >If anyone may kindly suggest.
>> >
>> >Regards,
>> >Subhabrata Banerjee.
>>
>> There is something wrong in the way you are thinking about this.
>> You don't _want_ to stick if __name__ = "__main__":
>> into every class you write.
>>
>> See if reading this helps.
>> http://effbot.org/pyfaq/tutor-what-is-if-name-main-for.htm
>>
>> Laura
>
> Hi,
> Thanks. The scipy issue seems solved. But this silly issue is giving so much of time. I am checking. Please see a sample code,
>
> import sys
> sys.stderr = sys.stdout
> class Colors:
>     def Blue(self):
>         self.var="This is Blue"
>         print self.var
>     def Red(self):
>         print self.var
>
>
>
> if __name__ == "__main__":
>     Colors().Blue() #THIS IS FINE
>     Colors().Red() #NOT FINE

I'm assuming that when you run this, you get the output "This is Blue"
followed by an AttributeError from Colors.Red complaining that the var
attribute is not set. If you want this to work, then maybe you should
set the attribute.

But as Laura noted, you probably don't want to be calling more than
one thing in your if __name__ == "__main__" block. Just call the entry
point of your program.


From rosuav at gmail.com  Tue Jun 16 12:03:39 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 17 Jun 2015 02:03:39 +1000
Subject: Creating .exe file in Python
In-Reply-To: <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
Message-ID: <CAPTjJmoxHKV1Ckm_2OEQ+44dbY68Fj7XqGB5FCr+4-cUE-f0gA@mail.gmail.com>

On Wed, Jun 17, 2015 at 1:17 AM,  <subhabrata.banerji at gmail.com> wrote:
> Thanks. The scipy issue seems solved. But this silly issue is giving so much of time. I am checking. Please see a sample code,
>
> import sys
> sys.stderr = sys.stdout
> class Colors:
>     def Blue(self):
>         self.var="This is Blue"
>         print self.var
>     def Red(self):
>         print self.var
>
>
>
> if __name__ == "__main__":
>     Colors().Blue() #THIS IS FINE
>     Colors().Red() #NOT FINE

You're still not saying what's going on. Did you try this code as a
simple Python script first, before trying to bundle it up into an .exe
file?

Fortunately, my primary crystal ball is active, and I believe what's
going on is that you expect Blue() to set something and then Red() to
see it. However, you're calling those methods on two different
throw-away objects, so they have separate state. What you expect to
happen, I honestly have no idea. (Also, why are you fiddling with
sys.stderr? You don't then appear to be using it, unless you have an
issue with exceptions getting printed to the other stream.)

ChrisA


From subhabrata.banerji at gmail.com  Tue Jun 16 12:23:08 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Tue, 16 Jun 2015 09:23:08 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.518.1434470627.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
Message-ID: <e87670d2-a26e-4da5-8e49-ab3c490e3f1a@googlegroups.com>

On Tuesday, June 16, 2015 at 9:33:58 PM UTC+5:30, Chris Angelico wrote:
> On Wed, Jun 17, 2015 at 1:17 AM,  wrote:
> > Thanks. The scipy issue seems solved. But this silly issue is giving so much of time. I am checking. Please see a sample code,
> >
> > import sys
> > sys.stderr = sys.stdout
> > class Colors:
> >     def Blue(self):
> >         self.var="This is Blue"
> >         print self.var
> >     def Red(self):
> >         print self.var
> >
> >
> >
> > if __name__ == "__main__":
> >     Colors().Blue() #THIS IS FINE
> >     Colors().Red() #NOT FINE
> 
> You're still not saying what's going on. Did you try this code as a
> simple Python script first, before trying to bundle it up into an .exe
> file?
> 
> Fortunately, my primary crystal ball is active, and I believe what's
> going on is that you expect Blue() to set something and then Red() to
> see it. However, you're calling those methods on two different
> throw-away objects, so they have separate state. What you expect to
> happen, I honestly have no idea. (Also, why are you fiddling with
> sys.stderr? You don't then appear to be using it, unless you have an
> issue with exceptions getting printed to the other stream.)
> 
> ChrisA

Thank you for your reply. I am trying to experimenting over small ones before going to actual code. Your error interpretations are right.
I could work out a small solution please check if it is going fine.
It is giving me result.

if __name__ == "__main__":
    c1=Colors()
    c2=c1.Blue()
    c3=c1.Red()

Regards,
Subhabrata Banerjee. 


From torriem at gmail.com  Tue Jun 16 13:15:58 2015
From: torriem at gmail.com (Michael Torrie)
Date: Tue, 16 Jun 2015 11:15:58 -0600
Subject: Keypress Input
In-Reply-To: <Ojtfx.178941$Jv.137884@fx26.iad>
References: <rBHbx.75089$MO7.40532@fx10.iad> <Ojtfx.178941$Jv.137884@fx26.iad>
Message-ID: <558059CE.6060103@gmail.com>

On 06/14/2015 11:23 PM, John McKenzie wrote:
>  Thank to the others who joined in and posted replies.
> 
>  Michael, your assumption is correct. To quote my original post, "and I 
> want this working on a Raspberry Pi." Doing a superficial look at curses 
> and getch it looks excessively complicated. I was under the impression it 
> was not multi-platform and Linux was excluded. Searching for getch and 
> raspberry pi on the web I see it is not and is available for Raspian.
> 
>  Worried about implementing it but I will consider it again. May spend 
> time reading about during my treatment tomorrow as the hospital has wifi 
> and Internet access and I am allowed to have my laptop there.

I'm not sure exactly what you mean by not available on Linux.  Curses
provides the same API on all platforms it runs on and I know it can read
keystrokes.  Curses getch is certainly available on Linux.  In fact,
here's an example:

http://stackoverflow.com/questions/10693256/how-to-accept-keypress-in-command-line-python

Again, however, you probably can't just fire up your program from a
startup script and have it run automatically, since stdin for a script
is not going to be the tty.  One idea would be to run your program from
init as a getty on tty1 (the default tty).  That would run as soon as it
boots up.

Though question for you. Why are you hooking the arcade buttons up to a
kade USB device?  Would not it be far simpler to hook the buttons
directly to a GPIO pin and read them from python?  You're already using
GPIO pins to run the LEDs.  Seems like the usb keyboard hack is
unnecessary and it's making your task more difficult.



From mal at europython.eu  Tue Jun 16 13:16:33 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Tue, 16 Jun 2015 19:16:33 +0200
Subject: EuroPython 2015: Financial Aid
Message-ID: <558059F1.1050806@europython.eu>

The financial aid work group has started to email everyone who applied
for financial aid this year.

We are fortunate to have a budget of almost 20,000 EUR available for
financial aid this year, thanks to well running ticket sales, speakers
who donated their speaker discounts towards the finaid budget, and a
generous grant from the Python Software Foundation (PSF) as our
financial aid sponsor:

               *** Python Software Foundation (PSF) ***

                      https://www.python.org/psf/

We received 52 request for financial aid from all around the world, 12
from Europe, 2 from Latin America, 2 from Asia, 1 from Africa and 1
from North America.

This year we focused on students (40%) and unemployed people (30%),
with the remaining funds going to attendees from startups, academia
and research. We also have specific diversity grants for Django Girls,
PyLadies and motivated women. 75% of the financial aid was given to
speakers, poster presenters, volunteers and Python community
contributors.

Waiting list
------------

Financial aid applicants who were not selected for the first round of
grants will be placed on a waiting list.

Looking for additional financial aid sponsors
---------------------------------------------

Companies which want to help raise the financial aid budget to also
support those people who are currently on the waiting list, please
contact the sponsor work group at sponsoring at europython.eu.

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From kwpolska at gmail.com  Tue Jun 16 14:10:26 2015
From: kwpolska at gmail.com (Chris Warrick)
Date: Tue, 16 Jun 2015 20:10:26 +0200
Subject: Trying to configure Apache and Python 2.7 on Red Hat I get 403
 Forbidden
In-Reply-To: <CALyLaQK5=HQcuHkTm9QHcs5p05NXvJ7j-wpi9GffqDrcA14GVg@mail.gmail.com>
References: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>
 <CABD09x9JQrb0dUXf7JNbqJT7OsDDgM6mftU9PcBU8YFP4PXdvQ@mail.gmail.com>
 <mailman.499.1434425540.13271.python-list@python.org>
 <557feaf7$0$1655$c3e8da3$5496439d@news.astraweb.com>
 <CALyLaQK5=HQcuHkTm9QHcs5p05NXvJ7j-wpi9GffqDrcA14GVg@mail.gmail.com>
Message-ID: <CAMw+j7Lhpt8hs8uZ3CFFNyyntg_RPEqrN82cPG2bfCvU_0rTRw@mail.gmail.com>

On Tue, Jun 16, 2015 at 12:11 PM, N?stor Bosc?n <nestorjb at gmail.com> wrote:
> I disabled selinux completely and the page worked.

So, selinux was the problem (which is typical, it?s a really dumb
piece of software)

The command to disable enforcing temporarily is actually "setenforce
0". Though you would need to issue it on every restart if you did not
change the config file.

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16


From nestorjb at gmail.com  Tue Jun 16 14:14:45 2015
From: nestorjb at gmail.com (=?UTF-8?B?TsOpc3RvciBCb3Njw6Fu?=)
Date: Tue, 16 Jun 2015 13:44:45 -0430
Subject: Trying to configure Apache and Python 2.7 on Red Hat I get 403
 Forbidden
In-Reply-To: <CAMw+j7Lhpt8hs8uZ3CFFNyyntg_RPEqrN82cPG2bfCvU_0rTRw@mail.gmail.com>
References: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>
 <CABD09x9JQrb0dUXf7JNbqJT7OsDDgM6mftU9PcBU8YFP4PXdvQ@mail.gmail.com>
 <mailman.499.1434425540.13271.python-list@python.org>
 <557feaf7$0$1655$c3e8da3$5496439d@news.astraweb.com>
 <CALyLaQK5=HQcuHkTm9QHcs5p05NXvJ7j-wpi9GffqDrcA14GVg@mail.gmail.com>
 <CAMw+j7Lhpt8hs8uZ3CFFNyyntg_RPEqrN82cPG2bfCvU_0rTRw@mail.gmail.com>
Message-ID: <CALyLaQJfL8iaxmPgj+WFj6WaV19xJwBoCWOeS2W-kw3NjRisWg@mail.gmail.com>

I tried that but it didn't work.

 I had to change /etc/selinux/config and reboot to make it work. It would
be nice if the wsgi module generated some log that explains why you get the
403. There are several posibilities.

Regards,

Nestor

On Tuesday, June 16, 2015, Chris Warrick <kwpolska at gmail.com> wrote:

> On Tue, Jun 16, 2015 at 12:11 PM, N?stor Bosc?n <nestorjb at gmail.com
> <javascript:;>> wrote:
> > I disabled selinux completely and the page worked.
>
> So, selinux was the problem (which is typical, it?s a really dumb
> piece of software)
>
> The command to disable enforcing temporarily is actually "setenforce
> 0". Though you would need to issue it on every restart if you did not
> change the config file.
>
> --
> Chris Warrick <https://chriswarrick.com/>
> PGP: 5EAAEA16
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150616/df023783/attachment.html>

From kwpolska at gmail.com  Tue Jun 16 14:19:35 2015
From: kwpolska at gmail.com (Chris Warrick)
Date: Tue, 16 Jun 2015 20:19:35 +0200
Subject: Trying to configure Apache and Python 2.7 on Red Hat I get 403
 Forbidden
In-Reply-To: <CALyLaQJfL8iaxmPgj+WFj6WaV19xJwBoCWOeS2W-kw3NjRisWg@mail.gmail.com>
References: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>
 <CABD09x9JQrb0dUXf7JNbqJT7OsDDgM6mftU9PcBU8YFP4PXdvQ@mail.gmail.com>
 <mailman.499.1434425540.13271.python-list@python.org>
 <557feaf7$0$1655$c3e8da3$5496439d@news.astraweb.com>
 <CALyLaQK5=HQcuHkTm9QHcs5p05NXvJ7j-wpi9GffqDrcA14GVg@mail.gmail.com>
 <CAMw+j7Lhpt8hs8uZ3CFFNyyntg_RPEqrN82cPG2bfCvU_0rTRw@mail.gmail.com>
 <CALyLaQJfL8iaxmPgj+WFj6WaV19xJwBoCWOeS2W-kw3NjRisWg@mail.gmail.com>
Message-ID: <CAMw+j7JXD7H29XJUyW_W18gPNZSPfYXBbTz3BZ7XSLPxPGqwnw@mail.gmail.com>

On Tue, Jun 16, 2015 at 8:14 PM, N?stor Bosc?n <nestorjb at gmail.com> wrote:
> I tried that but it didn't work.
>
>  I had to change /etc/selinux/config and reboot to make it work. It would be
> nice if the wsgi module generated some log that explains why you get the
> 403. There are several posibilities.

Well, that?s not wsgi?s fault.  There was a ?permission denied? error,
that?s everything wsgi ever knew.
As a RHEL sysadmin, you should know that this is most likely caused by
braindead syslinux and to read the audit log.

PS. please don?t top-post.

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16


From nestorjb at gmail.com  Tue Jun 16 14:22:43 2015
From: nestorjb at gmail.com (=?UTF-8?B?TsOpc3RvciBCb3Njw6Fu?=)
Date: Tue, 16 Jun 2015 13:52:43 -0430
Subject: Trying to configure Apache and Python 2.7 on Red Hat I get 403
 Forbidden
In-Reply-To: <CAMw+j7JXD7H29XJUyW_W18gPNZSPfYXBbTz3BZ7XSLPxPGqwnw@mail.gmail.com>
References: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>
 <CABD09x9JQrb0dUXf7JNbqJT7OsDDgM6mftU9PcBU8YFP4PXdvQ@mail.gmail.com>
 <mailman.499.1434425540.13271.python-list@python.org>
 <557feaf7$0$1655$c3e8da3$5496439d@news.astraweb.com>
 <CALyLaQK5=HQcuHkTm9QHcs5p05NXvJ7j-wpi9GffqDrcA14GVg@mail.gmail.com>
 <CAMw+j7Lhpt8hs8uZ3CFFNyyntg_RPEqrN82cPG2bfCvU_0rTRw@mail.gmail.com>
 <CALyLaQJfL8iaxmPgj+WFj6WaV19xJwBoCWOeS2W-kw3NjRisWg@mail.gmail.com>
 <CAMw+j7JXD7H29XJUyW_W18gPNZSPfYXBbTz3BZ7XSLPxPGqwnw@mail.gmail.com>
Message-ID: <CALyLaQKbKUFK5H70cN7WvZh2-5-MOgcg+mkRWr+98yaG6KM1PQ@mail.gmail.com>

On Tuesday, June 16, 2015, Chris Warrick <kwpolska at gmail.com> wrote:

> On Tue, Jun 16, 2015 at 8:14 PM, N?stor Bosc?n <nestorjb at gmail.com
> <javascript:;>> wrote:
> > I tried that but it didn't work.
> >
> >  I had to change /etc/selinux/config and reboot to make it work. It
> would be
> > nice if the wsgi module generated some log that explains why you get the
> > 403. There are several posibilities.
>
> Well, that?s not wsgi?s fault.  There was a ?permission denied? error,
> that?s everything wsgi ever knew.
> As a RHEL sysadmin, you should know that this is most likely caused by
> braindead syslinux and to read the audit log.
>
> PS. please don?t top-post.
>
> --
> Chris Warrick <https://chriswarrick.com/>
> PGP: 5EAAEA16
>

Yes but it could log which os file was it trying to access.

Thanks for all the replies.

Regards,

Nestor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150616/7eec935c/attachment.html>

From PointedEars at web.de  Tue Jun 16 15:18:35 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Tue, 16 Jun 2015 21:18:35 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
Message-ID: <17701105.fuEQZlnPee@PointedEars.de>

Ned Batchelder wrote:

> You aren't agreeing because you are arguing about different things.
> Thomas is talking about the relative probability of sequences of digits.

There is no such thing as ?relative probability?, except perhaps in popular-
scientific material and bad translations.  You might mean relative 
_frequency_, but I was not talking about that specifically.

> Chris is talking about the probability of a single digit never appearing
> in the output.

I do not think that what I am talking about and what you think Chris is 
talking about are different things.

> Thomas: let's say I generate streams of N digits drawn randomly from 0-9.
> I then consider the probability of a zero *never appearing once* in my
> stream.  Let's call that P(N)
 
In probability theory, it is called the probability P(E) of the event E that 
in n trials the probability variable X never assumes the value 0, which can 
be defined as

  P(E), E = {e_i | n ? ? \ {0}, i = 1, ?, n} \ {X ? 0}, ? = {1, 2, ?, 9} 

where the e_i are the singular events, or outcomes, of the probabilistic 
experiment, and ? is the sample space of the e_i.

> Do you agree that as N increases, P(N) decreases?

I do not agree that P(E), as defined above, decreases as n increases.

See also: <http://rationalwiki.org/wiki/Gambler%27s_fallacy>

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From breamoreboy at yahoo.co.uk  Tue Jun 16 15:40:13 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 16 Jun 2015 20:40:13 +0100
Subject: python financial data cleaning
In-Reply-To: <55a0ccda-572c-4324-9169-08028d4fb619@googlegroups.com>
References: <b0cbc75c-cc0b-4f27-a8d6-e4c20c356e6d@googlegroups.com>
 <mailman.480.1434363576.13271.python-list@python.org>
 <55a0ccda-572c-4324-9169-08028d4fb619@googlegroups.com>
Message-ID: <mlpu2u$h9k$1@ger.gmane.org>

On 15/06/2015 22:01, Sebastian M Cheung via Python-list wrote:
> On Monday, June 15, 2015 at 11:19:48 AM UTC+1, Mark Lawrence wrote:
>> On 15/06/2015 11:12, Sebastian M Cheung via Python-list wrote:
>>> How to do financial data cleaning ? Say I assume a list of 1000 finance series data in myList = Open, High, Low and Close. For missing Close Price data, What is best practice to clean data in Python
>>>
>>
>> http://pandas.pydata.org/
>>
>> --
>> My fellow Pythonistas, ask not what our language can do for you, ask
>> what you can do for our language.
>>
>> Mark Lawrence
>
>
> Hi Mark,
>
> Below I read in DirtyData (financial data) from Excel and then find the number of NaN missing Closed Pricing data:
>
> xls = pd.ExcelFile('DirtyData.xlsm')
> df = xls.parse('Dirty Data', index_col=None, na_values=['NA'])
> print(df.isnull().astype(int).sum())
>
> So if I were to clean missing Open Price data, I could copy from previous or row's Close Price data, but how would I implement it? Thanks
>

I'm sorry but my knowledge of pandas is limited, I just know it's pretty 
much best of breed.  Try stackoverflow or 
https://groups.google.com/forum/#!forum/pydata which is gated to 
gmane.comp.python.pydata

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From davros at bellaliant.net  Tue Jun 16 16:06:58 2015
From: davros at bellaliant.net (John McKenzie)
Date: Tue, 16 Jun 2015 20:06:58 GMT
Subject: Keypress Input
References: <rBHbx.75089$MO7.40532@fx10.iad> <Ojtfx.178941$Jv.137884@fx26.iad>
 <mailman.519.1434474965.13271.python-list@python.org>
Message-ID: <Cl%fx.2$WK.0@fx07.iad>


 That was the impression I got reading some comments people made online 
and doing research, so I focused on tkinter. As I mentioned in the 4th 
sentence of the post you quoted I discovered that was not the case, but 
by then I had already done some work on the tkinter script so I kept with 
it.

 Before I actually tried any of this and was just thinking of it 
conceptually I bought the Kade device. It was my hope that it would save 
on soldering, something I hate so very, very much, and I had not started 
learning Python at that point. It never occurred to me something so 
simple as keystrokes would not be present in Python, a language rated as 
being terrific by everyone I know who knows it.

 Out of great ignorance I thought I could hook up the Kade device and do 
something like if keypress == r then do this. Also out of ignorance I 
thought that using the GPIO would require days of soldering and 
incredibly complicated coding to talk to the pins.

 Only since I got much more deep into actually doing this did I start to 
find out otherwise. Spent the last few days looking at breakout boards to 
see if they would make the wiring easier to test out. When I saw one that 
had built in electrical protection (RaspIO Pro) I thought about getting 
it and going the route you suggest. However, I cannot find any option 
that does not require a breadboard. The lack of compactness and all the 
exposed wires does bother me a bit. More than it should, but I need a 
compact setup I can place outdoors. Breadboards and loose wires worry me 
in this case.

 Still, I must admit I spent the last 3 or 4 days thinking about exactly 
what you said and I may do it that way after all just to get it to work 
software wise.

 And BTW, I am not using any GPIO for the lights, the LED light strip is 
hooked up to a Blinkstick Pro through a USB port.



From random832 at fastmail.us  Tue Jun 16 16:23:41 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 16 Jun 2015 16:23:41 -0400
Subject: Testing random
In-Reply-To: <17701105.fuEQZlnPee@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
Message-ID: <1434486221.2586529.297335913.2DD21F25@webmail.messagingengine.com>

On Tue, Jun 16, 2015, at 15:18, Thomas 'PointedEars' Lahn wrote:
> There is no such thing as ?relative probability?, except perhaps in
> popular-
> scientific material and bad translations.  You might mean relative 
> _frequency_, but I was not talking about that specifically.

The probability of one event (getting a red on a roulette wheel) can be
larger than the probability of another event (getting a 17).

This is the plain english sense of the word "relative".


You have not answered my last question:


On Fri, Jun 12, 2015, at 18:57, random832 at fastmail.us wrote:
> On Fri, Jun 12, 2015, at 18:09, Thomas 'PointedEars' Lahn wrote:
> > Do you deny that ?123456789? *is* ?a sequence containing all nine
> > numbers?
> 
> Do you deny that "123456798" *is* "a sequence containing all nine
> numbers"?
> 
> Does this mean that "123456789" *is* "123456798" by the transitive
> property? NO.

The temperature today is hot. The temperature tomorrow is also hot. This
is true even if the temperature today is (for example) 37 and the
temperature tomorrow is 35.


From ned at nedbatchelder.com  Tue Jun 16 16:48:04 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 16 Jun 2015 13:48:04 -0700 (PDT)
Subject: Testing random
In-Reply-To: <17701105.fuEQZlnPee@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
Message-ID: <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>

On Tuesday, June 16, 2015 at 3:21:46 PM UTC-4, Thomas 'PointedEars' Lahn wrote:
> Ned Batchelder wrote:
> 
> > You aren't agreeing because you are arguing about different things.
> > Thomas is talking about the relative probability of sequences of digits.
> 
> There is no such thing as ?relative probability?, except perhaps in popular-
> scientific material and bad translations.  You might mean relative 
> _frequency_, but I was not talking about that specifically.
> 
> > Chris is talking about the probability of a single digit never appearing
> > in the output.
> 
> I do not think that what I am talking about and what you think Chris is 
> talking about are different things.
> 
> > Thomas: let's say I generate streams of N digits drawn randomly from 0-9.
> > I then consider the probability of a zero *never appearing once* in my
> > stream.  Let's call that P(N)
>  
> In probability theory, it is called the probability P(E) of the event E that 
> in n trials the probability variable X never assumes the value 0, which can 
> be defined as
> 
>   P(E), E = {e_i | n ? ? \ {0}, i = 1, ?, n} \ {X ? 0}, ? = {1, 2, ?, 9} 
> 
> where the e_i are the singular events, or outcomes, of the probabilistic 
> experiment, and ? is the sample space of the e_i.
> 
> > Do you agree that as N increases, P(N) decreases?
> 
> I do not agree that P(E), as defined above, decreases as n increases.
> 
> See also: <http://rationalwiki.org/wiki/Gambler%27s_fallacy>

I apologize, I'm sure I've been using the mathematical terms imprecisely.
We are all intelligent people, so I still believe we disagree because we
are talking about different things.

To put us all on a footing where (I hope) we have a shared understanding,
this Python program demonstrates what I was talking about:



import random

def die_roll():
    """Roll the die once, produce a number 0-9."""
    return random.randint(0, 9)

def die_rolls(n):
    """Roll the die n times, produce a list of numbers from 0-9."""
    return [die_roll() for _ in xrange(n)]

def any_zeros(seq):
    """Is there any zero in `seq`?"""
    return any(x == 0 for x in seq)

def probability_of_no_zero(nrolls, nseq):
    """Determine the chance of no zero in a sequence of rolls.

    This is done empirically, by producing `nseq` sequences of
    `nrolls` rolls of the die.  Each sequence is examined to
    see if it has a zero.  The total number of no-zero
    sequences divided `nseq` is the probability.
    """
    no_zeros = 0
    for _ in xrange(nseq):
        seq = die_rolls(nrolls)
        if not any_zeros(seq):
            no_zeros += 1
    return float(no_zeros)/nseq

for n in range(10, 101, 10):
    # Calculate the probability of getting no zeros by trying
    # it a million times.
    prob = probability_of_no_zero(n, 1000000)
    print "n = {:3d}, P(no zero) = {:.8f}".format(n, prob)



Running this gives:

$ pypy testrandom.py
n =  10, P(no zero) = 0.34867300
n =  20, P(no zero) = 0.12121900
n =  30, P(no zero) = 0.04267000
n =  40, P(no zero) = 0.01476600
n =  50, P(no zero) = 0.00519900
n =  60, P(no zero) = 0.00174100
n =  70, P(no zero) = 0.00061600
n =  80, P(no zero) = 0.00020600
n =  90, P(no zero) = 0.00006300
n = 100, P(no zero) = 0.00002400


As n increases, the probability of having no zeros goes down.

--Ned.


From invalid at invalid.invalid  Tue Jun 16 16:49:46 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Tue, 16 Jun 2015 20:49:46 +0000 (UTC)
Subject: Keypress Input
References: <rBHbx.75089$MO7.40532@fx10.iad> <Ojtfx.178941$Jv.137884@fx26.iad>
 <mailman.519.1434474965.13271.python-list@python.org>
 <Cl%fx.2$WK.0@fx07.iad>
Message-ID: <mlq25a$188$1@reader1.panix.com>

On 2015-06-16, John McKenzie <davros at bellaliant.net> wrote:

> It never occurred to me something so simple as keystrokes would not
> be present in Python, a language rated as being terrific by everyone
> I know who knows it.

Ah, but in reality "keystrokes" is not simple at all.  Keyboards and
input handling is a very messy, complicated area.

-- 
Grant Edwards               grant.b.edwards        Yow! FOOLED you!  Absorb
                                  at               EGO SHATTERING impulse
                              gmail.com            rays, polyester poltroon!!


From nimbiotics at gmail.com  Tue Jun 16 17:03:36 2015
From: nimbiotics at gmail.com (Mario R. Osorio)
Date: Tue, 16 Jun 2015 14:03:36 -0700 (PDT)
Subject: Pandas SQL Update
In-Reply-To: <d03cf991-e753-4ccf-9890-78befbe25edc@googlegroups.com>
References: <d03cf991-e753-4ccf-9890-78befbe25edc@googlegroups.com>
Message-ID: <d2b9ac2e-9e0f-4187-9976-a4461d11a497@googlegroups.com>


You are not specifying how are you doing the comparison, but here is my 2 cents:

Import the foxpro tables into the MySQL database and then you'll be able to do your update in a single SQL statement, which, even for that many records would take some only a few seconds, then delete th imported data.

Importing the foxpro tables into MySQL will take a little longer but I'm sure that, once automated, the whole process shouldn't take more than a couple of minutes.

HTH


From no.email at nospam.invalid  Tue Jun 16 17:22:38 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Tue, 16 Jun 2015 14:22:38 -0700
Subject: Keypress Input
References: <rBHbx.75089$MO7.40532@fx10.iad>
Message-ID: <87r3pbs601.fsf@jester.gateway.sonic.net>

John McKenzie <davros at bellaliant.net> writes:
>  Would like a set-up where something happens when a key is pressed. Not 
> propose a question, have the user type something, then hit return, then 
> something happens, but just the R key is pressed, something happens, 

The quick answer is that you want raw mode tty input.  Type "stty raw"
to the shell, and sys.stdin.read(1) will give you the bytes of the typed
characters immediately.  Use "stty -raw" to put it back in cooked mode,
or maybe better, "stty sane" to restore normal settings in general.

Yes there are messy details like function keys actually sending several
characters.  In practice you'll also want to control the tty modes from
your program instead of the shell, but that should get you started.  You
may also want to turn off echo, i.e. "stty raw -echo".


From PointedEars at web.de  Tue Jun 16 17:57:59 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Tue, 16 Jun 2015 23:57:59 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
Message-ID: <2089101.KkvqnN9Xah@PointedEars.de>

Ned Batchelder wrote:

> [?]
>     This is done empirically, by producing `nseq` sequences of
>     `nrolls` rolls of the die.  Each sequence is examined to
>     see if it has a zero.  The total number of no-zero
>     sequences divided `nseq` is the probability.

No, it is not.  It is the relative frequency for *this* number of trials and 
*this* run of the experiment.

>     """
>     no_zeros = 0
>     for _ in xrange(nseq):
>         seq = die_rolls(nrolls)
>         if not any_zeros(seq):
>             no_zeros += 1
>     return float(no_zeros)/nseq
> 
> for n in range(10, 101, 10):
>     # Calculate the probability of getting no zeros by trying
>     # it a million times.
>     prob = probability_of_no_zero(n, 1000000)
>     print "n = {:3d}, P(no zero) = {:.8f}".format(n, prob)
> 
> 
> 
> Running this gives:
> 
> $ pypy testrandom.py
> n =  10, P(no zero) = 0.34867300
> n =  20, P(no zero) = 0.12121900
> n =  30, P(no zero) = 0.04267000
> n =  40, P(no zero) = 0.01476600
> n =  50, P(no zero) = 0.00519900
> n =  60, P(no zero) = 0.00174100
> n =  70, P(no zero) = 0.00061600
> n =  80, P(no zero) = 0.00020600
> n =  90, P(no zero) = 0.00006300
> n = 100, P(no zero) = 0.00002400
> 
> 
> As n increases, the probability of having no zeros goes down.

Your programmatic "proof", as all the other intuitive-empirical "proofs", 
and all the other counter-arguments posted before in this thread, is flawed.  
As others have pointed out at the beginning of this thread, you *cannot* 
measure or calculate probability or determine randomness programmatically 
(at least not with this program).  I repeat: Probability is what relative 
frequency (which you can measure) *approaches* for *large* numbers.  100 is 
anything but large, to begin with.  What is ?large? depends on the 
experiment, not on the experimentator.  And with independent events, the 
probability for getting zero does not increase because you have been getting 
non-zeros before.  It simply does not work this way.

If you had read the article I referred you to, you would have known that 
this approach, this program, is bogus.  I see no logic to continue here as 
long as you do not realize and recognize the Gambler?s fallacy on which all 
arguments presented so far, including yours, are based.  I would only be 
wasting more precious free time by repeating myself in one way or another.

This should give you pause: In real mathematics, events with zero 
probability can occur.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From sohcahtoa82 at gmail.com  Tue Jun 16 18:30:29 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Tue, 16 Jun 2015 15:30:29 -0700 (PDT)
Subject: Testing random
In-Reply-To: <2089101.KkvqnN9Xah@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
Message-ID: <5e767961-9a7c-4c70-a821-c66d79675ebf@googlegroups.com>

On Tuesday, June 16, 2015 at 3:01:06 PM UTC-7, Thomas 'PointedEars' Lahn wrote:
> Ned Batchelder wrote:
> 
> > [...]
> >     This is done empirically, by producing `nseq` sequences of
> >     `nrolls` rolls of the die.  Each sequence is examined to
> >     see if it has a zero.  The total number of no-zero
> >     sequences divided `nseq` is the probability.
> 
> No, it is not.  It is the relative frequency for *this* number of trials and 
> *this* run of the experiment.
> 
> >     """
> >     no_zeros = 0
> >     for _ in xrange(nseq):
> >         seq = die_rolls(nrolls)
> >         if not any_zeros(seq):
> >             no_zeros += 1
> >     return float(no_zeros)/nseq
> > 
> > for n in range(10, 101, 10):
> >     # Calculate the probability of getting no zeros by trying
> >     # it a million times.
> >     prob = probability_of_no_zero(n, 1000000)
> >     print "n = {:3d}, P(no zero) = {:.8f}".format(n, prob)
> > 
> > 
> > 
> > Running this gives:
> > 
> > $ pypy testrandom.py
> > n =  10, P(no zero) = 0.34867300
> > n =  20, P(no zero) = 0.12121900
> > n =  30, P(no zero) = 0.04267000
> > n =  40, P(no zero) = 0.01476600
> > n =  50, P(no zero) = 0.00519900
> > n =  60, P(no zero) = 0.00174100
> > n =  70, P(no zero) = 0.00061600
> > n =  80, P(no zero) = 0.00020600
> > n =  90, P(no zero) = 0.00006300
> > n = 100, P(no zero) = 0.00002400
> > 
> > 
> > As n increases, the probability of having no zeros goes down.
> 
> Your programmatic "proof", as all the other intuitive-empirical "proofs", 
> and all the other counter-arguments posted before in this thread, is flawed.  
> As others have pointed out at the beginning of this thread, you *cannot* 
> measure or calculate probability or determine randomness programmatically 
> (at least not with this program).  I repeat: Probability is what relative 
> frequency (which you can measure) *approaches* for *large* numbers.  100 is 
> anything but large, to begin with.  What is "large" depends on the 
> experiment, not on the experimentator.  And with independent events, the 
> probability for getting zero does not increase because you have been getting 
> non-zeros before.  It simply does not work this way.

Nobody is arguing that.  You're arguing against something that nobody is suggesting.


> 
> If you had read the article I referred you to, you would have known that 
> this approach, this program, is bogus.  I see no logic to continue here as 
> long as you do not realize and recognize the Gambler's fallacy on which all 
> arguments presented so far, including yours, are based.  I would only be 
> wasting more precious free time by repeating myself in one way or another.


If the odds of getting a 0 is 1 out of 10, then the odds of NOT getting a zero is 9/10.  Which means the odds of picking 10 numbers and not getting ANY zeroes is (9/10)^10, which is approximately 0.35.  This has NOTHING to do with the Gambler's Fallacy.



> 
> This should give you pause: In real mathematics, events with zero 
> probability can occur.

Nobody will disagree with that.  The probability of me winning the lottery is zero if I don't buy a ticket.

> 
> -- 
> PointedEars
> 
> Twitter: @PointedEars2
> Please do not cc me. / Bitte keine Kopien per E-Mail.


From steve+comp.lang.python at pearwood.info  Tue Jun 16 18:46:03 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 16 Jun 2015 22:46:03 GMT
Subject: Set a flag on the function or a global?
References: <557f6676$0$21718$c3e8da3@news.astraweb.com>
 <4449bc52-4dba-486c-b426-7095e228a316@googlegroups.com>
 <mailman.494.1434414969.13271.python-list@python.org>
 <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
 <mailman.510.1434458729.13271.python-list@python.org>
Message-ID: <5580a72a$0$1667$c3e8da3$5496439d@news.astraweb.com>

On Tue, 16 Jun 2015 13:45:01 +0100, Oscar Benjamin wrote:

> On 16 June 2015 at 09:18, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>>
>> The primary use-case (at least *my* use-case, and hopefully others) is
>> to have "from module import edir as dir" in their Python startup file.
>> That means that when running interactively, they will get the enhanced
>> version of dir, but when running a script or an application they'll
>> just get the regular one.
>>
>> (Ideally, the regular one will eventually gain the same superpowers as
>> edir has, but that's a discussion for another day.)
>>
>> Besides, apart from the inspect module, which probably shouldn't, who
>> uses dir() programmatically?
>>
>> (If you do, you'll be glad to hear that edir() behaves the same as
>> regular dir() by default.)
> 
> What's the point in giving edir two modes if one of them is the same as
> dir? You could just do "from module import edir" and then use dir/edir
> as desired.

Oh ye of little faith :-)

I practically live in the Python interactive interpreter, and this has 
evolved from functionality I have wished dir has. It's a lot more than 
just whether or not dunders are displayed by default.

The most important feature of edir() is that it supports globbing and 
substring matches:


py> dir("", "low")
['islower', 'lower']
py> dir('', '*er')
['center', 'isidentifier', 'islower', 'isupper', 'lower', 'upper']


By default, globs are case-insensitive, but you can force them to be case-
sensitive with a metachar. I added this because I kept coming across 
classes with methods that used CamelCase when I expected lowercase. It 
understands the standard metacharacters ? * and [] as well as prefixes ! 
to invert the match and = to force case-sensitivity.

It also optionally includes the metaclass of the object. This was 
requested by somebody else as standard behaviour for dir, on python-ideas 
and the bug tracker, but rejected.

The reason this was requested is that dir() intentionally doesn't return 
all the attributes visible from an object:

py> "mro" in dir(str)
False
py> str.mro
<built-in method mro of type object at 0x81db360>


I thought it seemed like an interesting idea, and added it as an option 
with a keyword-only argument:

py> "mro" in dir(str, meta=True)
True


but frankly I'm not entirely sure this feature is useful. Time will tell.


> Personally I just use ipython's tab-completion instead of dir. It shows
> the dunders if you first type underscores but hides them otherwise e.g.:

Tab completion is great, but it solves a different problem. One with some 
overlap, admittedly, but still different.


-- 
Steven D'Aprano


From ian.g.kelly at gmail.com  Tue Jun 16 18:58:26 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 16 Jun 2015 16:58:26 -0600
Subject: Testing random
In-Reply-To: <5e767961-9a7c-4c70-a821-c66d79675ebf@googlegroups.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <5e767961-9a7c-4c70-a821-c66d79675ebf@googlegroups.com>
Message-ID: <CALwzidke0nQB152sgAdcRx7Yh0RNj9r7iHvBC0xzQe2hzLmr8A@mail.gmail.com>

On Tue, Jun 16, 2015 at 4:30 PM,  <sohcahtoa82 at gmail.com> wrote:
> On Tuesday, June 16, 2015 at 3:01:06 PM UTC-7, Thomas 'PointedEars' Lahn wrote:
>> This should give you pause: In real mathematics, events with zero
>> probability can occur.
>
> Nobody will disagree with that.  The probability of me winning the lottery is zero if I don't buy a ticket.

I believe he's actually referring to this:

https://en.wikipedia.org/wiki/Almost_surely

Not that this has anything to do with the probabilities under
discussion in this thread.


From ian.g.kelly at gmail.com  Tue Jun 16 19:24:11 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 16 Jun 2015 17:24:11 -0600
Subject: Testing random
In-Reply-To: <CALwzidke0nQB152sgAdcRx7Yh0RNj9r7iHvBC0xzQe2hzLmr8A@mail.gmail.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <5e767961-9a7c-4c70-a821-c66d79675ebf@googlegroups.com>
 <CALwzidke0nQB152sgAdcRx7Yh0RNj9r7iHvBC0xzQe2hzLmr8A@mail.gmail.com>
Message-ID: <CALwzid=KfKyW9epwXxv2qOdEinUd011cMQRF0nmXuqvxohhCww@mail.gmail.com>

On Jun 16, 2015 4:58 PM, "Ian Kelly" <ian.g.kelly at gmail.com> wrote:
>
> On Tue, Jun 16, 2015 at 4:30 PM,  <sohcahtoa82 at gmail.com> wrote:
> > On Tuesday, June 16, 2015 at 3:01:06 PM UTC-7, Thomas 'PointedEars'
Lahn wrote:
> >> This should give you pause: In real mathematics, events with zero
> >> probability can occur.
> >
> > Nobody will disagree with that.  The probability of me winning the
lottery is zero if I don't buy a ticket.
>
> I believe he's actually referring to this:
>
> https://en.wikipedia.org/wiki/Almost_surely
>
> Not that this has anything to do with the probabilities under
> discussion in this thread.

Actually, I take that back. The coin flip example in the Wikipedia article
is exactly what we've been discussing, taken to the ultimate extreme of an
infinitely long sequence.

Unfortunately for Thomas, the article does not agree with his position,
describing the event of never flipping a tail in such an infinite sequence
as "almost never". If, as Thomas maintains, the probability of this event
does not decrease as the length of the sequence increases, then the
probability for the infinite sequence would have to be non-zero since the
probability for any finite sequence is non-zero.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150616/a24552ab/attachment.html>

From ned at nedbatchelder.com  Tue Jun 16 19:26:16 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 16 Jun 2015 16:26:16 -0700 (PDT)
Subject: Testing random
In-Reply-To: <2089101.KkvqnN9Xah@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
Message-ID: <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>

On Tuesday, June 16, 2015 at 6:01:06 PM UTC-4, Thomas 'PointedEars' Lahn wrote:
> Ned Batchelder wrote:
> 
> > [...]
> >     This is done empirically, by producing `nseq` sequences of
> >     `nrolls` rolls of the die.  Each sequence is examined to
> >     see if it has a zero.  The total number of no-zero
> >     sequences divided `nseq` is the probability.
> 
> No, it is not.  It is the relative frequency for *this* number of trials and 
> *this* run of the experiment.
> 
> >     """
> >     no_zeros = 0
> >     for _ in xrange(nseq):
> >         seq = die_rolls(nrolls)
> >         if not any_zeros(seq):
> >             no_zeros += 1
> >     return float(no_zeros)/nseq
> > 
> > for n in range(10, 101, 10):
> >     # Calculate the probability of getting no zeros by trying
> >     # it a million times.
> >     prob = probability_of_no_zero(n, 1000000)
> >     print "n = {:3d}, P(no zero) = {:.8f}".format(n, prob)
> > 
> > 
> > 
> > Running this gives:
> > 
> > $ pypy testrandom.py
> > n =  10, P(no zero) = 0.34867300
> > n =  20, P(no zero) = 0.12121900
> > n =  30, P(no zero) = 0.04267000
> > n =  40, P(no zero) = 0.01476600
> > n =  50, P(no zero) = 0.00519900
> > n =  60, P(no zero) = 0.00174100
> > n =  70, P(no zero) = 0.00061600
> > n =  80, P(no zero) = 0.00020600
> > n =  90, P(no zero) = 0.00006300
> > n = 100, P(no zero) = 0.00002400
> > 
> > 
> > As n increases, the probability of having no zeros goes down.
> 
> Your programmatic "proof", as all the other intuitive-empirical "proofs", 
> and all the other counter-arguments posted before in this thread, is flawed.  
> As others have pointed out at the beginning of this thread, you *cannot* 
> measure or calculate probability or determine randomness programmatically 
> (at least not with this program).  

You *can* estimate probability with a program, which is what is happening
here.

> I repeat: Probability is what relative 
> frequency (which you can measure) *approaches* for *large* numbers.  100 is 
> anything but large, to begin with.  

The number of trials in this program is not 100, it is 1 million.  You seem
uninterested in trying to understand.

> What is "large" depends on the 
> experiment, not on the experimentator.  And with independent events, the 
> probability for getting zero does not increase because you have been getting 
> non-zeros before.  It simply does not work this way.

Again, if you look at the code, you'll see that we are not talking about
the probability of getting a zero on the next roll.  We are talking about the
probability of getting no zeros in an N-roll sequence.  I have no idea how you
have misunderstood this for so long.

I'll stop trying to explain it.

--Ned.


From PointedEars at web.de  Tue Jun 16 19:45:27 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Wed, 17 Jun 2015 01:45:27 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
Message-ID: <3049728.zPiIGXGd9A@PointedEars.de>

Ned Batchelder wrote:

> On Tuesday, June 16, 2015 at 6:01:06 PM UTC-4, Thomas 'PointedEars' Lahn
> wrote:
>> Your programmatic "proof", as all the other intuitive-empirical "proofs",
>> and all the other counter-arguments posted before in this thread, is
>> flawed. As others have pointed out at the beginning of this thread, you
>> *cannot* measure or calculate probability or determine randomness
>> programmatically (at least not with this program).
> 
> You *can* estimate probability with a program, which is what is happening
> here.

No.  Just no.

>> I repeat: Probability is what relative
>> frequency (which you can measure) *approaches* for *large* numbers.  100
>> is anything but large, to begin with.
> 
> The number of trials in this program is not 100, it is 1 million.  You
> seem uninterested in trying to understand.

It still would _not_ a measure or a calculation of *probability*.  So much 
for ?uninterested in trying to understand?.
 
>> What is "large" depends on the experiment, not on the experimentator.  
>> And with independent events, the probability for getting zero does not 
>> increase because you have been getting non-zeros before.  It simply does 
>> not work this way.
> 
> Again, if you look at the code, you'll see that we are not talking about
> the probability of getting a zero on the next roll.  We are talking about
> the probability of getting no zeros in an N-roll sequence.  I have no idea 
> how you have misunderstood this for so long.

You do not understand that it boils down to the same problem.  The 
probability of only having sons is _not_ greater than that of having
sons and one daughter or vice-versa.  And for that it does _not_ matter
how many children you have *because* it does _not_ matter how many
children you had before.  The probability for a boy or a girl is *always*
the same.  You are _not_ due for a boy if you have many girls, and not for a 
girls if you have many boys.  But that is precisely what your flawed logic 
is implying.

Learn probability theory, and use a dictionary in Python when you want to 
count random hits.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From cs at zip.com.au  Tue Jun 16 19:51:18 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 17 Jun 2015 09:51:18 +1000
Subject: Set a flag on the function or a global?
In-Reply-To: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
References: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
Message-ID: <20150616235118.GA71994@cskk.homeip.net>

On 16Jun2015 18:18, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
>On Tuesday 16 June 2015 10:35, MRAB wrote:
>> On 2015-06-16 01:24, sohcahtoa82 at gmail.com wrote:
>>> Using a keyword argument for the edir function is the most intuitive
>>> and easy to read, IMO.
>
>edir() has a keyword argument: edir(x, dunders=False) suppresses the return
>of dunder names. But since the primary purpose of [e]dir is, in my opinion,
>to be used interactively, needing to type an extra 15 characters to hide
>dunders is too inconvenient.

I'm just catching up here, but have now read the whole thread.

I am personally against a global (==> single variable affecting all callers) of 
any kind, be it a function attribute or whatever. Why? For that same reason 
that we discourage use of functions like os.chdir or os.umask except in rare 
circumstances: the single call inevitably affects the entire program behaviour.

Your intended use case may be interactive, where I agree the convenience looks 
nice. However, I think it inevitable that someone imports your edir function as 
an aid to writing friendly debugging messages is a larger program, and thus is 
escapes into noninteractive use. (As an example, I have a class named "O" which 
I use as a base class for many classes, especially in the debugging phase; its 
major semantic is friendlier __str__ and __repr__ for exactly the same reasons 
you wrote "edir", and with similar effect.)

On that basis (avoiding global state) my preference would be strongly to rely 
entirely on your keyword argument (dunder=False) and to offer two flavors, such 
as the "edir" and "edir_" suggested elsewhere. The user can always import 
"edir_ as edir_noisy" if they are of the mindset which dislikes single trailing 
underscores.

[...]
>> But it's meant to be used interactively. If they're using it in a
>> script, they'd most likely set the argument appropriately.
>
>Yes.

Ideally yes. "Most likely"? I have less confidence there.

>The primary use-case (at least *my* use-case, and hopefully others) is to
>have "from module import edir as dir" in their Python startup file. That
>means that when running interactively, they will get the enhanced version of
>dir, but when running a script or an application they'll just get the
>regular one.

This fits well with two functions, then they can import "edir" or "edir_" as 
dir as they see fit.

>Besides, apart from the inspect module, which probably shouldn't, who uses
>dir() programmatically?

Directly, perhaps rarely. But I use my O.__str__ method implicitly quite a lot, 
and it has a similar purpose to your edir. (It is not the same, so the parallel 
is not perfect.)

Cheers,
Cameron Simpson <cs at zip.com.au>

All who love Liberty are enemies of the State.  - Karl Hess


From rosuav at gmail.com  Tue Jun 16 19:58:13 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 17 Jun 2015 09:58:13 +1000
Subject: Trying to configure Apache and Python 2.7 on Red Hat I get 403
 Forbidden
In-Reply-To: <CALyLaQKbKUFK5H70cN7WvZh2-5-MOgcg+mkRWr+98yaG6KM1PQ@mail.gmail.com>
References: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>
 <CABD09x9JQrb0dUXf7JNbqJT7OsDDgM6mftU9PcBU8YFP4PXdvQ@mail.gmail.com>
 <mailman.499.1434425540.13271.python-list@python.org>
 <557feaf7$0$1655$c3e8da3$5496439d@news.astraweb.com>
 <CALyLaQK5=HQcuHkTm9QHcs5p05NXvJ7j-wpi9GffqDrcA14GVg@mail.gmail.com>
 <CAMw+j7Lhpt8hs8uZ3CFFNyyntg_RPEqrN82cPG2bfCvU_0rTRw@mail.gmail.com>
 <CALyLaQJfL8iaxmPgj+WFj6WaV19xJwBoCWOeS2W-kw3NjRisWg@mail.gmail.com>
 <CAMw+j7JXD7H29XJUyW_W18gPNZSPfYXBbTz3BZ7XSLPxPGqwnw@mail.gmail.com>
 <CALyLaQKbKUFK5H70cN7WvZh2-5-MOgcg+mkRWr+98yaG6KM1PQ@mail.gmail.com>
Message-ID: <CAPTjJmr3Qy6wxxp6Zu1OZarOmQUK+qEFheX-rBV-2U=tjkHwyw@mail.gmail.com>

On Wed, Jun 17, 2015 at 4:22 AM, N?stor Bosc?n <nestorjb at gmail.com> wrote:
>> On Tue, Jun 16, 2015 at 8:14 PM, N?stor Bosc?n <nestorjb at gmail.com> wrote:
>> > I tried that but it didn't work.
>> >
>> >  I had to change /etc/selinux/config and reboot to make it work. It
>> > would be
>> > nice if the wsgi module generated some log that explains why you get the
>> > 403. There are several posibilities.
>>
>> Well, that?s not wsgi?s fault.  There was a ?permission denied? error,
>> that?s everything wsgi ever knew.
>> As a RHEL sysadmin, you should know that this is most likely caused by
>> braindead syslinux and to read the audit log.
>
>
> Yes but it could log which os file was it trying to access.

If you want to know what a program was doing when it got some error,
the obvious sledgehammer to reach for is strace. Cutting the situation
down to size so you can actually find what you want in strace's
output, now, that's a whole new problem. But the information will be
there. You want a log? Be careful what you wish for, you might get one
that spews more text than python-list does :)

ChrisA


From zondervanz at gmail.com  Tue Jun 16 20:24:03 2015
From: zondervanz at gmail.com (Dr. John Q. Hacker)
Date: Tue, 16 Jun 2015 19:24:03 -0500
Subject: operations on types
Message-ID: <CADL-DvhtaBaosv=W9ODEaUgqsRKzdBt21KvFqL9U+5xy9HsULw@mail.gmail.com>

[Dr. Bigcock wrote:]
> The current syntax for adding functionality to a class using mix-in style
via inheritance list conflates two very different things.
>
> A different way would be to create (sensible) operations on types:
>
> NewType = OldObject + mixin  #object composition
>
> Thoughts?

That's an awesome idea.  It's like a final way to understand and operate
with metaclasses.

One would have to decide how to merge the two classes, but this could be as
straightforward as merging two dictionaries, presumably in the order given.

This might also decide the fate of the super() function.  True is-a
inheritance shouldn't need to call super, the language should enforce the
relationship, and ensure it gets done before other methods are called in
the same routine.

Neat, I don't think any language has such a meta-feature for working with
types.  I'm surprised no one had thought of it before.

Zipher
(Did you steal that name from me? ;^)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150616/b6eafe58/attachment.html>

From sohcahtoa82 at gmail.com  Tue Jun 16 20:36:52 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Tue, 16 Jun 2015 17:36:52 -0700 (PDT)
Subject: Testing random
In-Reply-To: <3049728.zPiIGXGd9A@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
Message-ID: <abcf102d-e523-49bb-94e9-9a6d15971021@googlegroups.com>

On Tuesday, June 16, 2015 at 4:48:36 PM UTC-7, Thomas 'PointedEars' Lahn wrote:
> Ned Batchelder wrote:
> 
> > On Tuesday, June 16, 2015 at 6:01:06 PM UTC-4, Thomas 'PointedEars' Lahn
> > wrote:
> >> Your programmatic "proof", as all the other intuitive-empirical "proofs",
> >> and all the other counter-arguments posted before in this thread, is
> >> flawed. As others have pointed out at the beginning of this thread, you
> >> *cannot* measure or calculate probability or determine randomness
> >> programmatically (at least not with this program).
> > 
> > You *can* estimate probability with a program, which is what is happening
> > here.
> 
> No.  Just no.
> 
> >> I repeat: Probability is what relative
> >> frequency (which you can measure) *approaches* for *large* numbers.  100
> >> is anything but large, to begin with.
> > 
> > The number of trials in this program is not 100, it is 1 million.  You
> > seem uninterested in trying to understand.
> 
> It still would _not_ a measure or a calculation of *probability*.  So much 
> for "uninterested in trying to understand".
>  
> >> What is "large" depends on the experiment, not on the experimentator.  
> >> And with independent events, the probability for getting zero does not 
> >> increase because you have been getting non-zeros before.  It simply does 
> >> not work this way.
> > 
> > Again, if you look at the code, you'll see that we are not talking about
> > the probability of getting a zero on the next roll.  We are talking about
> > the probability of getting no zeros in an N-roll sequence.  I have no idea 
> > how you have misunderstood this for so long.
> 
> You do not understand that it boils down to the same problem.

Actually, no, they're not.  They're entirely different problems.  "What are the odds of getting 8 zeros in a row?" is a *COMPLETELY* different question from "What are the odds of getting a zero on the next roll?"


>The 
> probability of only having sons is _not_ greater than that of having
> sons and one daughter or vice-versa.  And for that it does _not_ matter
> how many children you have *because* it does _not_ matter how many
> children you had before.  The probability for a boy or a girl is *always*
> the same.  You are _not_ due for a boy if you have many girls, and not for a 
> girls if you have many boys.  But that is precisely what your flawed logic 
> is implying.

Yes, we all know what the gambler's fallacy is, but that's not what anyone is arguing.

If you pick 8 random numbers between 0 and 9 (inclusive), then the odds of getting all zeros is (1/10)^8.  Do you agree with that?

The odds of getting NO zeros is (9/10)^8.  Do you agree with that?

Note that NEITHER of these scenarios say anything about a pre-condition.  The first question is *NOT* asking "If you picked 7 random numbers between 0 and 9 and got 0 for all 7, what are the odds of getting another 0?"  The answer to that is obviously 1/10, and anybody arguing something else would certainly be committing the Gambler's fallacy.

> 
> Learn probability theory, and use a dictionary in Python when you want to 
> count random hits.

I know enough probability theory to know that you're either wrong or you keep changing the problem to something nobody else has said in order to think you're right.

> 
> -- 
> PointedEars
> 
> Twitter: @PointedEars2
> Please do not cc me. / Bitte keine Kopien per E-Mail.



From python at mrabarnett.plus.com  Tue Jun 16 20:42:03 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 17 Jun 2015 01:42:03 +0100
Subject: Testing random
In-Reply-To: <3049728.zPiIGXGd9A@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
Message-ID: <5580C25B.2060307@mrabarnett.plus.com>

On 2015-06-17 00:45, Thomas 'PointedEars' Lahn wrote:
> Ned Batchelder wrote:
>
>> On Tuesday, June 16, 2015 at 6:01:06 PM UTC-4, Thomas 'PointedEars' Lahn
>> wrote:
>>> Your programmatic "proof", as all the other intuitive-empirical "proofs",
>>> and all the other counter-arguments posted before in this thread, is
>>> flawed. As others have pointed out at the beginning of this thread, you
>>> *cannot* measure or calculate probability or determine randomness
>>> programmatically (at least not with this program).
>>
>> You *can* estimate probability with a program, which is what is happening
>> here.
>
> No.  Just no.
>
>>> I repeat: Probability is what relative
>>> frequency (which you can measure) *approaches* for *large* numbers.  100
>>> is anything but large, to begin with.
>>
>> The number of trials in this program is not 100, it is 1 million.  You
>> seem uninterested in trying to understand.
>
> It still would _not_ a measure or a calculation of *probability*.  So much
> for ?uninterested in trying to understand?.
>
>>> What is "large" depends on the experiment, not on the experimentator.
>>> And with independent events, the probability for getting zero does not
>>> increase because you have been getting non-zeros before.  It simply does
>>> not work this way.
>>
>> Again, if you look at the code, you'll see that we are not talking about
>> the probability of getting a zero on the next roll.  We are talking about
>> the probability of getting no zeros in an N-roll sequence.  I have no idea
>> how you have misunderstood this for so long.
>
> You do not understand that it boils down to the same problem.  The
> probability of only having sons is _not_ greater than that of having
> sons and one daughter or vice-versa.  And for that it does _not_ matter
> how many children you have *because* it does _not_ matter how many
> children you had before.  The probability for a boy or a girl is *always*
> the same.  You are _not_ due for a boy if you have many girls, and not for a
> girls if you have many boys.  But that is precisely what your flawed logic
> is implying.
>
> Learn probability theory, and use a dictionary in Python when you want to
> count random hits.
>
I think that different people are talking about different things in
this thread. You're talking about the probability of each event, while
everybody else is talking about the probability of certain combinations
of events.

If you have, say, two children, the possibilities are:

	boy, boy
	boy, girl
	girl, boy
	girl, girl

The probability of each boy or girl is 1/2.

The probability of only boys is 1/4 and of a son and a daughter is 1/4
+ 1/4 = 1/2.

Therefore, the probability of having only boys is less than the
probability of having a son and a daughter.



From rosuav at gmail.com  Tue Jun 16 21:01:39 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 17 Jun 2015 11:01:39 +1000
Subject: Testing random
In-Reply-To: <abcf102d-e523-49bb-94e9-9a6d15971021@googlegroups.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
 <abcf102d-e523-49bb-94e9-9a6d15971021@googlegroups.com>
Message-ID: <CAPTjJmqLyTaMd85OzaKhdb_MpBsJbBg6UwkVbB2fVSf2pN-=5A@mail.gmail.com>

On Wed, Jun 17, 2015 at 10:36 AM,  <sohcahtoa82 at gmail.com> wrote:
> Yes, we all know what the gambler's fallacy is, but that's not what anyone is arguing.

The only instance of gambler's fallacy I'm seeing here is "PointedEars
didn't understand the last dozen emails, so he's due to understand the
next one". I've given up trying to explain.

ChrisA


From torriem at gmail.com  Tue Jun 16 21:22:14 2015
From: torriem at gmail.com (Michael Torrie)
Date: Tue, 16 Jun 2015 19:22:14 -0600
Subject: Keypress Input
In-Reply-To: <mlq25a$188$1@reader1.panix.com>
References: <rBHbx.75089$MO7.40532@fx10.iad> <Ojtfx.178941$Jv.137884@fx26.iad>
 <mailman.519.1434474965.13271.python-list@python.org> <Cl%fx.2$WK.0@fx07.iad>
 <mlq25a$188$1@reader1.panix.com>
Message-ID: <5580CBC6.3000402@gmail.com>

On 06/16/2015 02:49 PM, Grant Edwards wrote:
> On 2015-06-16, John McKenzie <davros at bellaliant.net> wrote:
> 
>> It never occurred to me something so simple as keystrokes would not
>> be present in Python, a language rated as being terrific by everyone
>> I know who knows it.
> 
> Ah, but in reality "keystrokes" is not simple at all.  Keyboards and
> input handling is a very messy, complicated area.

If you do choose to go with the GPIO route, unless your code for
accessing the GPIO lines does debouncing, you will have to debounce the
key.  There are lots of examples out there (most in C on the arduino,
but still applicable). Most of them check for a button press, then do a
timer count-down to let things settle out before recording a button
press.  So it's still complicated even if you talk directly to the
buttons!  No way around some complexity though.




From ethan at stoneleaf.us  Tue Jun 16 21:32:37 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 16 Jun 2015 18:32:37 -0700
Subject: Testing random
In-Reply-To: <CAPTjJmqLyTaMd85OzaKhdb_MpBsJbBg6UwkVbB2fVSf2pN-=5A@mail.gmail.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
 <abcf102d-e523-49bb-94e9-9a6d15971021@googlegroups.com>
 <CAPTjJmqLyTaMd85OzaKhdb_MpBsJbBg6UwkVbB2fVSf2pN-=5A@mail.gmail.com>
Message-ID: <5580CE35.1040103@stoneleaf.us>

On 06/16/2015 06:01 PM, Chris Angelico wrote:

> The only instance of gambler's fallacy I'm seeing here is "PointedEars
> didn't understand the last dozen emails, so he's due to understand the
> next one". I've given up trying to explain.

+1 QotW


From steve+comp.lang.python at pearwood.info  Tue Jun 16 21:35:41 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 17 Jun 2015 01:35:41 GMT
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
Message-ID: <5580ceed$0$1667$c3e8da3$5496439d@news.astraweb.com>

On Tue, 16 Jun 2015 13:48:04 -0700, Ned Batchelder wrote:

> I apologize, I'm sure I've been using the mathematical terms
> imprecisely. We are all intelligent people, so I still believe we
> disagree because we are talking about different things.

Neil, I believe that your actual mistake is assuming that Thomas is 
arguing in good faith. I see no evidence that he is, especially given the 
content of his latest posts.

Multiple people have repeatedly explained the difference between his 
argument and what everyone else is talking about. Others, including me, 
have demonstrated empirically that he is mistaken, using both simulated 
tests and direct calculation of the probabilities.

At this point, his insistence that we are making the gambler's fallacy is 
clearly not a mere misunderstanding due to confusion. It is wilful and 
deliberate misrepresentation of what we are saying.

Thomas is correct for a completely different question. Rather than 
acknowledge that he has misunderstood the question, at every point he 
doubles down harder and insists that he is right and we are wrong. We 
have given *absolutely no reason* to think we have fallen for the 
Gambler's Fallacy. Throughout this thread, Thomas has repeatedly picked 
on trivial and unimportant errors in terminology as an excuse for 
dismissing what others have said, while ignoring the substance of their 
argument. When people have given mathematically indisputable and correct 
arguments, he has ignored them, or misrepresented them.



-- 
Steven D'Aprano


From steve+comp.lang.python at pearwood.info  Tue Jun 16 21:36:35 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 17 Jun 2015 01:36:35 GMT
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
Message-ID: <5580cf22$0$1667$c3e8da3$5496439d@news.astraweb.com>

On Wed, 17 Jun 2015 01:45:27 +0200, Thomas 'PointedEars' Lahn wrote:

> The
> probability of only having sons is _not_ greater than that of having
> sons and one daughter or vice-versa.

Take a family of four children. We can enumerate all the possibilities, 
using S for son and D for daughter, there are exactly 2**4 = 16 of them:

SSSS
SSSD
SSDS
SSDD
SDSS
SDSD
SDDS
SDDD
DSSS
DSSD
DSDS
DSDD
DDSS
DDSD
DDDS
DDDD


There is exactly 1 outcome which is "all sons", 4 outcomes which is 
"three sons and one daughter", 14 outcomes which is "at least one son and 
one daughter", 15 outcomes which are "at least one son", and 1 outcome is 
"no sons".

If you have *two* children, those probabilities are different:

SS, SD, DS, DD

All sons: 1/4, not 1/16.
Three sons and one daughter: 0, not 4/16.
At least one son and one daughter: 2/4, not 14/16.
At least one son: 3/4, not 15/16.
No sons: 1/4, not 1/16.


With four children, it is true that the probability of these are the same:

    SSSS vs SSDS

but that's not the question. The question is to compare the probability 
of these:

    SSSS vs (SSDS or SSSD or SDSS or DSSS)




> And for that it does _not_ matter
> how many children you have

Of course it does. Assuming the births are independent and the 
probability of a boy is 1/2, the probability of having "no boys" depends 
on how many children you have:

Pr(no boys, given no children) = 1
Pr(no boys, given 1 child) = 1/2
Pr(no boys, given 2 children) = 1/4
Pr(no boys, given 3 children) = 1/8
Pr(no boys, given 4 children) = 1/16

and in general:

Pr(no boys, given n children) = 1/2**n



> *because* it does _not_ matter how many
> children you had before.  The probability for a boy or a girl is
> *always* the same.

That is completely irrelevant, as has been explained to you over and over 
again. A little learning is a dangerous thing.


>  You are _not_ due for a boy if you have many girls, and not for a girls
> if you have many boys.  But that is precisely what your flawed logic is
> implying.

You are the only one talking about being "due" for a result. You are 
ignoring what we are saying, ignoring the clear and detailed analysis we 
have repeatedly given, ignored the simulations we have given, and keep 
coming back to your arrogant *AND WRONG* accusation that we are making 
the gambler's fallacy.

Time to man up and admit your mistake.



-- 
Steven D'Aprano


From zondervanz at gmail.com  Tue Jun 16 21:55:00 2015
From: zondervanz at gmail.com (Dr. John Q. Hacker)
Date: Tue, 16 Jun 2015 20:55:00 -0500
Subject: mix-in classes
In-Reply-To: <5561b200$0$13002$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.1.1432432413.5151.python-list@python.org>
 <5561b200$0$13002$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CADL-Dvh1W-_3C2m5AcgYct1gUb1oQjX30PKGM8JjJeRYNZAa9g@mail.gmail.com>

On Sun, May 24, 2015 at 6:11 AM, Steven D'Aprano <steve at pearwood.info>
wrote:

> On Sun, 24 May 2015 11:53 am, Dr. John Q. Hacker wrote:
> But, frankly, what you describe is more likely to be a weakness of multiple
> inheritance and mixins, one which should be avoided. One attempt to avoid
> this problem is with traits, an alternative to mixins which explicitly
> deals with the problem of mixin conflicts.
>
> http://www.artima.com/weblogs/viewpost.jsp?thread=246488
>

Interesting.  This brings up an issue another poster brought up:  In my
usage of the term "parent", I use it to mean the class that is a product of
object composition:

class Parent(child1, child2):  pass

I figure that it is this "Parent" class which must manage the methods that
it is inheriting with child1 and child2 -- mixins or otherwise.  In this
usage, super() should be called "delegate" as whatever I don't accomplish
in my specialized Parent class, I will get the child classes to do.  Python
automagically delegates any methods that I don't define in Parent to
methods found in child1 and child2.

It seems the issues that everyone encounters with multiple inheritance,
mixins, and such has more to do with terminology (and proper
implementation) than in actuality.

zipher
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150616/fd8c7017/attachment.html>

From steve+comp.lang.python at pearwood.info  Tue Jun 16 22:04:56 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: 17 Jun 2015 02:04:56 GMT
Subject: operations on types
References: <mailman.531.1434500646.13271.python-list@python.org>
Message-ID: <5580d5c8$0$1667$c3e8da3$5496439d@news.astraweb.com>

On Tue, 16 Jun 2015 19:24:03 -0500, Dr. John Q. Hacker wrote:

> [Dr. Bigcock wrote:]
>> The current syntax for adding functionality to a class using mix-in
>> style via inheritance list conflates two very different things.

I'm not sure why you are taking "Dr Bigcock" seriously. You know he isn't 
actually a doctor?


>> A different way would be to create (sensible) operations on types:
>>
>> NewType = OldObject + mixin  #object composition
>>
>> Thoughts?
> 
> That's an awesome idea.  It's like a final way to understand and operate
> with metaclasses.

Unfortunately, it has nothing to do with metaclasses.

I can see why:

    NewType = OldType + Mixin

appears more attractive at first glance than:

    class NewType(OldType, Mixin):
        pass


since it saves a lot of boilerplate. But in practice, the body of the 
class is not often "pass". More often, you end up overriding or adding 
new methods. In that case, the suggested syntax has no advantage.

I think that this might be a bit more attractive in statically-typed 
languages. If you wrote:

    class NewType(OldType + Mixin): ...

the compiler could optimize away the intermediary class (OldType+Mixin) 
and just use direct inheritance, but I don't think that would work in 
Python. And a statically-typed language could avoid errors where you 
forget to instantiate a class before doing regular addition:

    result = OldType + x  # oops, meant OldType()


-- 
Steven D'Aprano


From zondervanz at gmail.com  Tue Jun 16 22:42:47 2015
From: zondervanz at gmail.com (Dr. John Q. Hacker)
Date: Tue, 16 Jun 2015 21:42:47 -0500
Subject: operations on types
In-Reply-To: <5580d5c8$0$1667$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.531.1434500646.13271.python-list@python.org>
 <5580d5c8$0$1667$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CADL-Dvh8NB2sZU7tVAxqnicy-xGaJjBOhQq2B69OT0c1V1_m=Q@mail.gmail.com>

On Tue, Jun 16, 2015 at 9:04 PM, Steven D'Aprano <
steve+comp.lang.python at pearwood.info> wrote:

> On Tue, 16 Jun 2015 19:24:03 -0500, Dr. John Q. Hacker wrote:
>
> > [Dr. Bigcock wrote:]
> >> The current syntax for adding functionality to a class using mix-in
> >> style via inheritance list conflates two very different things.
>
> I'm not sure why you are taking "Dr Bigcock" seriously. You know he isn't
> actually a doctor?
>

Actually, I asked him.  He said he was has a PhD in Theology.

> > That's an awesome idea.  It's like a final way to understand and operate
> > > with metaclasses.
> >
> > Unfortunately, it has nothing to do with metaclasses.
>

 No, it *does*:  it's operations on classes themselves (i.e. types) and not
on *instantiations* of classes.  If that isn't metaclassing, then python's
been calling it wrong.

> I can see why:
> >
> >     NewType = OldType + Mixin
> >
> > appears more attractive at first glance than:
> >
> >     class NewType(OldType, Mixin):
> >         pass
> >
> > since it saves a lot of boilerplate. But in practice, the body of the
> > class is not often "pass". More often, you end up overriding or adding
> > new methods. In that case, the suggested syntax has no advantage.
>

Adding new methods is exactly what this syntax is supposed to do.

The tricky part is overriding existing methods.  If Python's dict (or
defaultdict) type, provided an optional collision parameter, this could be
user-defined.  Probably the __doc__ attribute could be concatenated (the
simple, naive solution) while the code would have to be merged in some
fashion.

zipher
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150616/3f4e1b7b/attachment.html>

From torriem at gmail.com  Wed Jun 17 00:39:21 2015
From: torriem at gmail.com (Michael Torrie)
Date: Tue, 16 Jun 2015 22:39:21 -0600
Subject: mix-in classes
In-Reply-To: <CADL-Dvh1W-_3C2m5AcgYct1gUb1oQjX30PKGM8JjJeRYNZAa9g@mail.gmail.com>
References: <mailman.1.1432432413.5151.python-list@python.org>
 <5561b200$0$13002$c3e8da3$5496439d@news.astraweb.com>
 <CADL-Dvh1W-_3C2m5AcgYct1gUb1oQjX30PKGM8JjJeRYNZAa9g@mail.gmail.com>
Message-ID: <5580F9F9.80505@gmail.com>

On 06/16/2015 07:55 PM, Dr. John Q. Hacker wrote:
> Interesting.  This brings up an issue another poster brought up:  In my
> usage of the term "parent", I use it to mean the class that is a product of
> object composition:
> 
> class Parent(child1, child2):  pass

Hmm. This is a definition of "parent" I've never heard of before.  And
it's opposite to the use of the word in all contexts I'm aware of.



From jpiitula at ling.helsinki.fi  Wed Jun 17 00:41:20 2015
From: jpiitula at ling.helsinki.fi (Jussi Piitulainen)
Date: Wed, 17 Jun 2015 07:41:20 +0300
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
Message-ID: <lf5k2v3ymj3.fsf@ling.helsinki.fi>

Ned Batchelder writes:

> Thomas: let's say I generate streams of N digits drawn randomly from
> 0-9.  I then consider the probability of a zero *never appearing once*
> in my stream.  Let's call that P(N).  Do you agree that as N
> increases, P(N) decreases?

In probability theory, that could be phrased as the probability that N
unknown digits d_1, ..., d_N are all positive, assuming the digits are
independent (so learning one digit doesn't reveal anything about any
other digit), and for each d_k, the probability p_k of having a positive
digit is the same. Mathematicians often abbreviate these assumptions as
"i.i.d" for "independent" and "identically distributed".

Also assuming uniform distributions, p_1 = p_2 = ... = p_N = 9/10.

P(d_k > 0 for k = 1, ..., N) =
P(d_1 > 0 and d_2 > 0 and ... and d_N > 0) = (by independence)
P(d_1 > 0) * P(d_2 > 0) * ... * P(d_N > 0) =
p_1 * p_2 * ... * p_N = (by identical uniform distribution)
(9/10)^N

In mathematics, (9/10)^N decreases as N increases, so one should indeed
agree. Using more impressive notation and terminology correctly will not
change the analysis.


From ian.g.kelly at gmail.com  Wed Jun 17 00:58:55 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 16 Jun 2015 22:58:55 -0600
Subject: operations on types
In-Reply-To: <5580d5c8$0$1667$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.531.1434500646.13271.python-list@python.org>
 <5580d5c8$0$1667$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CALwzid=x5_EmzmKeSKANiAc6MNzBy2Je8XAu7ND4KoT7892zPg@mail.gmail.com>

On Tue, Jun 16, 2015 at 8:04 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Tue, 16 Jun 2015 19:24:03 -0500, Dr. John Q. Hacker wrote:
>
>> [Dr. Bigcock wrote:]
>>> The current syntax for adding functionality to a class using mix-in
>>> style via inheritance list conflates two very different things.
>
> I'm not sure why you are taking "Dr Bigcock" seriously. You know he isn't
> actually a doctor?

I'm reasonably certain that they're actually the same person, and he's
now engaging in sockpuppetry [1] by starting conversations with
himself. He's not trying very hard to hide it, signing posts with both
accounts as "zipher" and then lamp-shading that fact by commenting on
it. I can't even find any record that "Dr. Bigcock" ever actually
posted the text that "Dr. John Q. Hacker" is now responding to.

[1] https://en.wikipedia.org/wiki/Sockpuppet_(Internet)


From rosuav at gmail.com  Wed Jun 17 01:33:28 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 17 Jun 2015 15:33:28 +1000
Subject: operations on types
In-Reply-To: <CALwzid=x5_EmzmKeSKANiAc6MNzBy2Je8XAu7ND4KoT7892zPg@mail.gmail.com>
References: <mailman.531.1434500646.13271.python-list@python.org>
 <5580d5c8$0$1667$c3e8da3$5496439d@news.astraweb.com>
 <CALwzid=x5_EmzmKeSKANiAc6MNzBy2Je8XAu7ND4KoT7892zPg@mail.gmail.com>
Message-ID: <CAPTjJmq4V+tqbuxexr9EM9y-XHWtEwcvgH_TdcK_UCjPvpGe3g@mail.gmail.com>

On Wed, Jun 17, 2015 at 2:58 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> I'm reasonably certain that they're actually the same person, and he's
> now engaging in sockpuppetry [1] by starting conversations with
> himself.

I used to think there was a limit to the depths of human boredom. Then
I met the internet, and learned better.

ChrisA


From steve+comp.lang.python at pearwood.info  Wed Jun 17 01:39:03 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 17 Jun 2015 15:39:03 +1000
Subject: operations on types
References: <mailman.531.1434500646.13271.python-list@python.org>
 <5580d5c8$0$1667$c3e8da3$5496439d@news.astraweb.com>
 <mailman.537.1434508975.13271.python-list@python.org>
Message-ID: <558107fc$0$11127$c3e8da3@news.astraweb.com>

On Wednesday 17 June 2015 12:42, Dr. John Q. Hacker wrote:

> On Tue, Jun 16, 2015 at 9:04 PM, Steven D'Aprano <
> steve+comp.lang.python at pearwood.info> wrote:
> 
>> On Tue, 16 Jun 2015 19:24:03 -0500, Dr. John Q. Hacker wrote:
>>
>> > [Dr. Bigcock wrote:]
>> >> The current syntax for adding functionality to a class using mix-in
>> >> style via inheritance list conflates two very different things.
>>
>> I'm not sure why you are taking "Dr Bigcock" seriously. You know he isn't
>> actually a doctor?
>>
> 
> Actually, I asked him.  He said he was has a PhD in Theology.

You believe somebody calling himself "Dr Bigcock" (among other pseudonyms)? 
Just how much credibility does somebody calling themselves Bigcock get? Even 
in the "Carry On" movies they didn't use any names as unsubtle as that.

https://www.youtube.com/watch?v=wcBOX1JBcjQ

Besides, I have a Doctorate of Divinity, and I'm not a doctor either.


>> > That's an awesome idea.  It's like a final way to understand and
>> > operate
>> > > with metaclasses.
>> >
>> > Unfortunately, it has nothing to do with metaclasses.
>>
> 
>  No, it *does*:  it's operations on classes themselves (i.e. types) and
>  not
> on *instantiations* of classes.  If that isn't metaclassing, then python's
> been calling it wrong.

Python's definition of "metaclass" (the class of a class) is the same as 
that used by (at least) Objective C and Smalltalk. If you're going to argue 
that *Smalltalk* is wrong, well, good luck with that, Smalltalk invented the 
concept.

http://www.cocoawithlove.com/2010/01/what-is-meta-class-in-objective-c.html

http://pharo.gforge.inria.fr/PBE1/PBE1ch14.html

If you think there's another definition of "metaclass" in widespread use, 
please give some citations or links.

What I will accept is that implementing class composition using the + 
operator *could* be implemented via the metaclass, at least in languages 
with metaclasses. But that's not the only way to do it. Java, for example, 
has no metaclasses (classes are not themselves objects in Java), but the 
Java language could easily define "Type + Type" as a way of composing types. 
It just wouldn't work via the metaclass.

That's what I mean by "it has nothing to do with metaclasses" -- you can 
implement this without using metaclasses.


>> I can see why:
>> >
>> >     NewType = OldType + Mixin
>> >
>> > appears more attractive at first glance than:
>> >
>> >     class NewType(OldType, Mixin):
>> >         pass
>> >
>> > since it saves a lot of boilerplate. But in practice, the body of the
>> > class is not often "pass". More often, you end up overriding or adding
>> > new methods. In that case, the suggested syntax has no advantage.
>>
> 
> Adding new methods is exactly what this syntax is supposed to do.

No, it creates a new class which inherits *existing* methods belonging to 
the two composed classes, OldType and Mixin. It doesn't create any methods 
that don't already exist.


> The tricky part is overriding existing methods. 

Precisely. If you want NewType.method to override either parent 
(OldType.method and/or Mixin.method) you need more than just a type 
composition operator. In Python you would use a class statement:

class NewType(OldType + Mixin):
    def method(self):
        ...


You would also do the same to add a new method not provided by either 
OldType or the Mixin.


-- 
Steve



From PointedEars at web.de  Wed Jun 17 02:53:46 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Wed, 17 Jun 2015 08:53:46 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
 <mailman.532.1434501913.13271.python-list@python.org>
Message-ID: <16305487.vjehVXsXp1@PointedEars.de>

MRAB wrote:

> On 2015-06-17 00:45, Thomas 'PointedEars' Lahn wrote:
>> Learn probability theory, and use a dictionary in Python when you want to
>> count random hits.
>>
> I think that different people are talking about different things in
> this thread.

I think that you, too, do not know what you are talking about.

> You're talking about the probability of each event, while
> everybody else is talking about the probability of certain combinations
> of events.

No.

> If you have, say, two children, the possibilities are:
> 
> boy, boy
> boy, girl
> girl, boy
> girl, girl
> 
> The probability of each boy or girl is 1/2.
> 
> The probability of only boys is 1/4 and of a son and a daughter is 1/4
> + 1/4 = 1/2.
> 
> Therefore, the probability of having only boys is less than the
> probability of having a son and a daughter.

And half of the population is homosexual as with 50 % males and 50 % females 
there is an equal number of homosexual pairs (male?male, female?female) as 
heterosexual ones (male?female, female?male).  NOT.

Final posting by me in this thread as the misconceptions about probability 
are apparently too deeply embedded in the general public to be resolved 
here.

In summary:

1. Combinations matter, not permutations.

2. <https://en.wikipedia.org/wiki/Law_of_averages>

3. 
<http://www.teacherlink.org/content/math/interactive/probability/interactivequiz/home.html>
   (Whereas I predict that the ignorant will see the correct answer to
    question 3 as proof of the correctness of their misconception.)

EOD.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From DMusiige at sennheisercommunications.com  Wed Jun 17 03:09:03 2015
From: DMusiige at sennheisercommunications.com (Deogratius Musiige)
Date: Wed, 17 Jun 2015 07:09:03 +0000
Subject: Get off the list
Message-ID: <A804AEE67207BF42BAF8CEC79839C8D63404C8FA@kbnmxexc11.Demant.com>

Hi,
How can i get off this mailing list?

Best regards / Med venlig hilsen

Deogratius Musiige
Software Development Engineer

Sennheiser Communications A/S
Industriparken 27
DK-2750 Ballerup

Direct                  +45 5618 0320
Mail                     demu at senncom.com
Web                    www.sennheiser.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150617/c272bc34/attachment.html>

From auriocus at gmx.de  Wed Jun 17 03:22:27 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Wed, 17 Jun 2015 09:22:27 +0200
Subject: Testing random
In-Reply-To: <16305487.vjehVXsXp1@PointedEars.de>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
 <mailman.532.1434501913.13271.python-list@python.org>
 <16305487.vjehVXsXp1@PointedEars.de>
Message-ID: <mlr755$34o$2@dont-email.me>

Am 17.06.15 um 08:53 schrieb Thomas 'PointedEars' Lahn:
> 3.
> <http://www.teacherlink.org/content/math/interactive/probability/interactivequiz/home.html>
>     (Whereas I predict that the ignorant will see the correct answer to
>      question 3 as proof of the correctness of their misconception.)

I'v been to the UK recently for cycling holidays. A nightmare! So many 
ignorant people all driving on the wrong side of the road!

> EOD.


	:)



From rosuav at gmail.com  Wed Jun 17 03:28:35 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 17 Jun 2015 17:28:35 +1000
Subject: Testing random
In-Reply-To: <mlr755$34o$2@dont-email.me>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
 <mailman.532.1434501913.13271.python-list@python.org>
 <16305487.vjehVXsXp1@PointedEars.de> <mlr755$34o$2@dont-email.me>
Message-ID: <CAPTjJmrCHE5bwo7J7W4cEJeseS6tgQa3kfON2+AfGjX=CZVF7Q@mail.gmail.com>

On Wed, Jun 17, 2015 at 5:22 PM, Christian Gollwitzer <auriocus at gmx.de> wrote:
> I'v been to the UK recently for cycling holidays. A nightmare! So many
> ignorant people all driving on the wrong side of the road!

You might argue that you drive on the right side of the road, but if
you call what British people drive on the "wrong" side, then you don't
understand the language. It's called "English", remember, not
"American", and the word you want is "left". Get it *right*... oh
wait.

ChrisA


From mail at timgolden.me.uk  Wed Jun 17 03:30:52 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 17 Jun 2015 08:30:52 +0100
Subject: Testing random
In-Reply-To: <mlr755$34o$2@dont-email.me>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
 <mailman.532.1434501913.13271.python-list@python.org>
 <16305487.vjehVXsXp1@PointedEars.de> <mlr755$34o$2@dont-email.me>
Message-ID: <5581222C.5010301@timgolden.me.uk>

On 17/06/2015 08:22, Christian Gollwitzer wrote:
> Am 17.06.15 um 08:53 schrieb Thomas 'PointedEars' Lahn:
>> 3.
>> <http://www.teacherlink.org/content/math/interactive/probability/interactivequiz/home.html>
>>
>>     (Whereas I predict that the ignorant will see the correct answer to
>>      question 3 as proof of the correctness of their misconception.)
> 
> I'v been to the UK recently for cycling holidays. A nightmare! So many
> ignorant people all driving on the wrong side of the road!

Yes, we do get many foreign tourists at this time of the year :)

TJG



From breamoreboy at yahoo.co.uk  Wed Jun 17 04:41:28 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 17 Jun 2015 09:41:28 +0100
Subject: Testing random
In-Reply-To: <CAPTjJmqLyTaMd85OzaKhdb_MpBsJbBg6UwkVbB2fVSf2pN-=5A@mail.gmail.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
 <abcf102d-e523-49bb-94e9-9a6d15971021@googlegroups.com>
 <CAPTjJmqLyTaMd85OzaKhdb_MpBsJbBg6UwkVbB2fVSf2pN-=5A@mail.gmail.com>
Message-ID: <mlrbrp$83s$1@ger.gmane.org>

On 17/06/2015 02:01, Chris Angelico wrote:
> On Wed, Jun 17, 2015 at 10:36 AM,  <sohcahtoa82 at gmail.com> wrote:
>> Yes, we all know what the gambler's fallacy is, but that's not what anyone is arguing.
>
> The only instance of gambler's fallacy I'm seeing here is "PointedEars
> didn't understand the last dozen emails, so he's due to understand the
> next one". I've given up trying to explain.
>
> ChrisA
>

An alternative explanation is that he's just a plain, old fashioned 
troll, as pointed out by Denis McMahon some weeks ago.  Now what is the 
probability of that? :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lac at openend.se  Wed Jun 17 05:28:46 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 17 Jun 2015 11:28:46 +0200
Subject: Testing random
In-Reply-To: Message from Ian Kelly <ian.g.kelly@gmail.com> of "Tue,
 16 Jun 2015 16:58:26 -0600."
 <CALwzidke0nQB152sgAdcRx7Yh0RNj9r7iHvBC0xzQe2hzLmr8A@mail.gmail.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <5e767961-9a7c-4c70-a821-c66d79675ebf@googlegroups.com><CALwzidke0nQB152sgAdcRx7Yh0RNj9r7iHvBC0xzQe2hzLmr8A@mail.gmail.com>
Message-ID: <201506170928.t5H9Sk03015042@fido.openend.se>

In a message of Tue, 16 Jun 2015 16:58:26 -0600, Ian Kelly writes:
>On Tue, Jun 16, 2015 at 4:30 PM,  <sohcahtoa82 at gmail.com> wrote:
>> On Tuesday, June 16, 2015 at 3:01:06 PM UTC-7, Thomas 'PointedEars' Lahn wrote:
>>> This should give you pause: In real mathematics, events with zero
>>> probability can occur.
>>
>> Nobody will disagree with that.  The probability of me winning the lottery is zero if I don't buy a ticket.
>
>I believe he's actually referring to this:
>
>https://en.wikipedia.org/wiki/Almost_surely
>
>Not that this has anything to do with the probabilities under
>discussion in this thread.
>-- 
>https://mail.python.org/mailman/listinfo/python-list

I think he is talking about what mathematicians call 'discrepancy':
https://en.wikipedia.org/wiki/Low-discrepancy_sequence

Laura


From Cecil at decebal.nl  Wed Jun 17 05:57:19 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 17 Jun 2015 11:57:19 +0200
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
 <mailman.532.1434501913.13271.python-list@python.org>
 <16305487.vjehVXsXp1@PointedEars.de> <mlr755$34o$2@dont-email.me>
 <mailman.542.1434526258.13271.python-list@python.org>
Message-ID: <87a8vyhd34.fsf@Equus.decebal.nl>

On Wednesday 17 Jun 2015 09:30 CEST, Tim Golden wrote:

> On 17/06/2015 08:22, Christian Gollwitzer wrote:
>> Am 17.06.15 um 08:53 schrieb Thomas 'PointedEars' Lahn:
>>> 3.
>>> <http://www.teacherlink.org/content/math/interactive/probability/interactivequiz/home.html>
>>>
>>> (Whereas I predict that the ignorant will see the correct answer
>>> to question 3 as proof of the correctness of their misconception.)
>>
>> I'v been to the UK recently for cycling holidays. A nightmare! So
>> many ignorant people all driving on the wrong side of the road!
>
> Yes, we do get many foreign tourists at this time of the year :)

??_???

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From lac at openend.se  Wed Jun 17 06:33:54 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 17 Jun 2015 12:33:54 +0200
Subject: Testing random
In-Reply-To: Message from MRAB <python@mrabarnett.plus.com>
 of "Wed, 17 Jun 2015 01:42:03 +0100." <5580C25B.2060307@mrabarnett.plus.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de><5580C25B.2060307@mrabarnett.plus.com>
Message-ID: <201506171033.t5HAXsNK016564@fido.openend.se>

Stick to dice.  Stay away from children.  One thing we know of, for sure,
is that certain breeding pairs are more likely to produce males, and
others are more likely to produce females.  We will ignore  those born who
are of indeterminate sex, for this discussion.

In human beings, as well as a whole lot of animal species, infant-and-
chidhood mortality is not sex neutral.  More baby boys die than baby
girls.  But when you look at breeding age men and women you will find
that the ratio is a lot closer to 50/50 than the birth age ratio.
Human beings, as a population currently produce more males than
females.

Now one way that could happen is if all human beings are predisposed to
have slightly more boy babies than girl babies when they breed.  But
this isn't what we think happens.  

Fisher's Principle says:

        Suppose sexually active males are less common than females in
	a population.

	A male then has better mating prospects than a female, and
  	can expect to have more offspring.  (Because some lucky males
	can expect to mate with more than 1 female).

	Therefore parents genetically disposed to produce males tend to
	have more than average numbers of grandchildren born to them.

        Thus genes for producing males will spread.

        But this means that more males will be produced.  Which means
	that at some point, the first priciple no longer holds --
	sexually active	males are no longer less common	than females
	in the population.  You	can reach a state where	females	are
	less common.

	But the	same thing kicks in, once more.	 And this time parents
	who produce daughters tend to get more offspring.  If you produce
	a son, who doesn't get to breed at all, from an evolutionary
	standpoint you have wasted your time in a bad investment.

	Thus the sex ratio will tend to 50/50.  It is self-adjusting
	that way.

-----------

Now, human beings, despite being very interesting to us, are very bad to
study this way.  There are too many other complicating factors in human
reproduction, and human beings take too long to reach sexual maturity
anyway.  But you can study this in animals where you can artificially
bias the population any way you like, and who reach sexual maturity
quickly so you can watch and see if it plays out like Fisher says it
would.  And in lots of places that has been  studied, it does.

It is a little hard to see how you would get the proper correspondence
between infant mortality rates and sex ratio -- this species produces
precisely enough extra girl babies to offset the fact that more girls
die as infants -- unless something like this was going on.  And this
is measured to be so for some animals studied other than humans.

Bets are off in species where most of the males don't get to breed
anyway.  The logic doesn't work for them.

But what this shows is that the statement that a given birth is as likely
to be a son as a daughter (or 52% likely to be a son and 48% likely
to be a daughter) only works for large populations.  It's not going to
help you know if your next child is going to be a son or a daughter.
The parents of 4 boys who are trying, once again, thinking 'this time
it must be a girl!' may be up against something rather worse than
the gamblers fallacy. The odds that they have a girl may be substantially
lower than 50% -- it is quite possible that they are set up to only produce
boys, or to produce boys 80% of the time.  We have no way of knowing
this, because right now we don't know exactly how a bias in producing
one sex or another is expressed.  It is also quite possible that the
parents of 4 boys who want a girl have just been unlucky, because
they have no bias one way or another.  Or extremely unlucky because
their bias is for daughters, but they keep getting results against the
odds.

All we know is that notion that every human birth has a 50/50
chance of being male or female is wrong. 

Laura



From steve at pearwood.info  Wed Jun 17 07:51:52 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 17 Jun 2015 21:51:52 +1000
Subject: operations on types
References: <mailman.531.1434500646.13271.python-list@python.org>
 <5580d5c8$0$1667$c3e8da3$5496439d@news.astraweb.com>
 <mailman.539.1434517186.13271.python-list@python.org>
Message-ID: <55815f58$0$1662$c3e8da3$5496439d@news.astraweb.com>

On Wed, 17 Jun 2015 02:58 pm, Ian Kelly wrote:

> On Tue, Jun 16, 2015 at 8:04 PM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>> On Tue, 16 Jun 2015 19:24:03 -0500, Dr. John Q. Hacker wrote:
>>
>>> [Dr. Bigcock wrote:]
>>>> The current syntax for adding functionality to a class using mix-in
>>>> style via inheritance list conflates two very different things.
>>
>> I'm not sure why you are taking "Dr Bigcock" seriously. You know he isn't
>> actually a doctor?
> 
> I'm reasonably certain that they're actually the same person, and he's
> now engaging in sockpuppetry [1] by starting conversations with
> himself. He's not trying very hard to hide it, signing posts with both
> accounts as "zipher" and then lamp-shading that fact by commenting on
> it. I can't even find any record that "Dr. Bigcock" ever actually
> posted the text that "Dr. John Q. Hacker" is now responding to.


I must admit that the possibility of a sockpuppet crossed my mind too, even
to the point of checking the headers of their posts to see if they came
from the same IP address. (Inconclusive.)

But there was an earlier post from Mark Jansen, a.k.a. Mark
Rosenblitt-Janssen, Dr Bigcock, and Zipher, starting this thread a few days
earlier. I won't link to it, but the relevant headers are:


Subject: operations on types
From: <dreamingforward at gmail.com>
Date: Tue, 16 Jun 2015 09:30:01 -0700 (PDT)
Groups: comp.lang.python


posted from Google Groups. "Dr Hacker", if he really is a doctor or a
hacker, replied via the python-list mailing list, but broke threading when
doing so. (Alas, this is not an unusual thing to happen.)

At the moment, I'm still willing to give Dr Hacker the benefit of the doubt
re the sockpuppet suspicion.



-- 
Steven



From steve at pearwood.info  Wed Jun 17 08:47:37 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 17 Jun 2015 22:47:37 +1000
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de><5580C25B.2060307@mrabarnett.plus.com>
 <mailman.545.1434537246.13271.python-list@python.org>
Message-ID: <55816c68$0$1673$c3e8da3$5496439d@news.astraweb.com>

On Wed, 17 Jun 2015 08:33 pm, Laura Creighton wrote:

> Stick to dice.  Stay away from children.  One thing we know of, for sure,
> is that certain breeding pairs are more likely to produce males, and
> others are more likely to produce females.  We will ignore  those born who
> are of indeterminate sex, for this discussion.
[...]
> All we know is that notion that every human birth has a 50/50
> chance of being male or female is wrong.


This is certainly true. In human beings, certain families tend to run with
all girls or all boys, more often than you would expect from just chance.
This suggests that at least some people are predisposed to have boys, or
girls, rather than equal chance of both.

Oh, and those intersex children? Surprisingly common: Wikipedia suggests
that according to some definition, as many as 1.7% (about one in sixty)
newborns may have ambiguous genitalia, or something other than XY/XX
chromosomes. The frequency of non-XX, non-XY children alone is 3 in 5000.

While it is important to know these facts, to understand that human beings
(like other mammals) do not divide neatly into two distinct sexes, I think
that's a complication not really necessary for basic statistics classes. As
a first approximation, the idea that the proportion of boys and girls are
both roughly equal at 1/2 isn't too far off the truth.

After all, we make other simplifying assumptions about probability too.
There are magicians who are capable of forcing coins to land the required
way up, and somebody once built a machine capable of tossing a coin with
the precise equal force and velocity every single time. Dice are rarely
unbiased, and neither are roulette wheels. Nevertheless, we ignore those
factual biases for the sake of simplicity.


> In human beings, as well as a whole lot of animal species, infant-and-
> chidhood mortality is not sex neutral.  More baby boys die than baby
> girls.

Correct. And more Y sperm die than X sperm. Together, these two factors lead
to a small but real bias towards girl children, at least on countries that
don't practice wide-spread sex-specific abortion.


> But when you look at breeding age men and women you will find 
> that the ratio is a lot closer to 50/50 than the birth age ratio.
> Human beings, as a population currently produce more males than
> females.

Surely that depends on where (and when) you are?

I know that, today, both India and China both abort far more female fetuses
than male ones, leading to a large excess of men. But in countries that
don't practice selective abortions, my understanding is that there is a
small excess of women at virtually all ages, especially among the elderly.
Men tend to die earlier than women in every age bracket.



-- 
Steven



From subhabrata.banerji at gmail.com  Wed Jun 17 09:10:45 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Wed, 17 Jun 2015 06:10:45 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.518.1434470627.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
Message-ID: <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>

On Tuesday, June 16, 2015 at 9:33:58 PM UTC+5:30, Chris Angelico wrote:
> On Wed, Jun 17, 2015 at 1:17 AM, wrote:
> > Thanks. The scipy issue seems solved. But this silly issue is giving so much of time. I am checking. Please see a sample code,
> >
> > import sys
> > sys.stderr = sys.stdout
> > class Colors:
> >     def Blue(self):
> >         self.var="This is Blue"
> >         print self.var
> >     def Red(self):
> >         print self.var
> >
> >
> >
> > if __name__ == "__main__":
> >     Colors().Blue() #THIS IS FINE
> >     Colors().Red() #NOT FINE
> 
> You're still not saying what's going on. Did you try this code as a
> simple Python script first, before trying to bundle it up into an .exe
> file?
> 
> Fortunately, my primary crystal ball is active, and I believe what's
> going on is that you expect Blue() to set something and then Red() to
> see it. However, you're calling those methods on two different
> throw-away objects, so they have separate state. What you expect to
> happen, I honestly have no idea. (Also, why are you fiddling with
> sys.stderr? You don't then appear to be using it, unless you have an
> issue with exceptions getting printed to the other stream.)
> 
> ChrisA

Dear Group,

Thank you all. It seems going fine now. I have one additional question if I run the .exe files created in Non Python Windows environment. Linux has Python builtin but in Non Python environment how may I run it? Is there any set of prequisites I have to follow. I am not finding much web help, if any one may kindly suggest. 

Regards,
Subhabrata Banerjee.


From rosuav at gmail.com  Wed Jun 17 09:20:28 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 17 Jun 2015 23:20:28 +1000
Subject: Creating .exe file in Python
In-Reply-To: <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
 <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
Message-ID: <CAPTjJmrOopVdCWLFcL_4V+opj_sMfQVQ1VqZKJAJyfjSARrTvw@mail.gmail.com>

On Wed, Jun 17, 2015 at 11:10 PM,  <subhabrata.banerji at gmail.com> wrote:
> Thank you all. It seems going fine now. I have one additional question if I run the .exe files created in Non Python Windows environment. Linux has Python builtin but in Non Python environment how may I run it? Is there any set of prequisites I have to follow. I am not finding much web help, if any one may kindly suggest.
>

There have been some proposals to make an easy way to produce a single
package that has a Windows executable header, but can be easily
unpacked and run using a system-provided Linux or Mac OS Python.
However, I don't think any have been deployed yet. So the simple
answer is: You can't do what you're trying to do. Instead, take a step
back, and look at just getting your Python program to run on all
platforms... and that's easy, just distribute the .py files. Much MUCH
easier, more reliable, and simpler.

ChrisA


From hamilton at nothere.com  Wed Jun 17 09:33:19 2015
From: hamilton at nothere.com (hamilton)
Date: Wed, 17 Jun 2015 07:33:19 -0600
Subject: Creating .exe file in Python
In-Reply-To: <mailman.547.1434547236.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
 <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
 <mailman.547.1434547236.13271.python-list@python.org>
Message-ID: <mlrssm$j8p$1@dont-email.me>

On 6/17/2015 7:20 AM, Chris Angelico wrote:
> On Wed, Jun 17, 2015 at 11:10 PM,  <subhabrata.banerji at gmail.com> wrote:
>> Thank you all. It seems going fine now. I have one additional question if I run the .exe files created in Non Python Windows environment. Linux has Python builtin but in Non Python environment how may I run it? Is there any set of prequisites I have to follow. I am not finding much web help, if any one may kindly suggest.
>>
>
> There have been some proposals to make an easy way to produce a single
> package that has a Windows executable header, but can be easily
> unpacked and run using a system-provided Linux or Mac OS Python.
> However, I don't think any have been deployed yet. So the simple
> answer is: You can't do what you're trying to do. Instead, take a step
> back, and look at just getting your Python program to run on all
> platforms... and that's easy, just distribute the .py files. Much MUCH
> easier, more reliable, and simpler.
>
> ChrisA
>
Yes, this is easy.

However, the python source can be read by anyone.

As a .exe, the source can not be read.

Just because the interpreter is open source,
does not mean my application should be.



From subhabrata.banerji at gmail.com  Wed Jun 17 09:33:28 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Wed, 17 Jun 2015 06:33:28 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.547.1434547236.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
 <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
 <mailman.547.1434547236.13271.python-list@python.org>
Message-ID: <96672d23-c333-49e5-9f61-028a189bdc91@googlegroups.com>

On Wednesday, June 17, 2015 at 6:50:48 PM UTC+5:30, Chris Angelico wrote:
> On Wed, Jun 17, 2015 at 11:10 PM, wrote:
> > Thank you all. It seems going fine now. I have one additional question if I run the .exe files created in Non Python Windows environment. Linux has Python builtin but in Non Python environment how may I run it? Is there any set of prequisites I have to follow. I am not finding much web help, if any one may kindly suggest.
> >
> 
> There have been some proposals to make an easy way to produce a single
> package that has a Windows executable header, but can be easily
> unpacked and run using a system-provided Linux or Mac OS Python.
> However, I don't think any have been deployed yet. So the simple
> answer is: You can't do what you're trying to do. Instead, take a step
> back, and look at just getting your Python program to run on all
> platforms... and that's easy, just distribute the .py files. Much MUCH
> easier, more reliable, and simpler.
> 
> ChrisA

Dear Sir,

Thank you for your kind and prompt answer. I was about to explore another package. Regards, Subhabrata Banerjee.


From miki.tebeka at gmail.com  Wed Jun 17 09:49:08 2015
From: miki.tebeka at gmail.com (Miki Tebeka)
Date: Wed, 17 Jun 2015 06:49:08 -0700 (PDT)
Subject: Problem with PyPI login
Message-ID: <4127850f-e1a8-4924-837d-2a709a026e88@googlegroups.com>

Greetings,

After confirming email I can't login to PyPI with the password and when trying to rest I get: user "Cyberint" is unknwon to me. However when trying to re-register it says the user is already there.

The forums at https://sourceforge.net/p/pypi/support-requests/ (which is linked from "PyPI Support") seems pretty deserted. What is the right location for these kind of issues?

Thanks,
--
Miki


From lac at openend.se  Wed Jun 17 09:50:41 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 17 Jun 2015 15:50:41 +0200
Subject: Testing random
In-Reply-To: Message from "Steven D'Aprano" <steve@pearwood.info> of "Wed,
 17 Jun 2015 22:47:37 +1000."
 <55816c68$0$1673$c3e8da3$5496439d@news.astraweb.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de><5580C25B.2060307@mrabarnett.plus.com>
 <mailman.545.1434537246.13271.python-list@python.org><55816c68$0$1673$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <201506171350.t5HDofFQ021813@fido.openend.se>

In a message of Wed, 17 Jun 2015 22:47:37 +1000, "Steven D'Aprano" writes:
>There are magicians who are capable of forcing coins to land the required
>way up, and somebody once built a machine capable of tossing a coin with
>the precise equal force and velocity every single time. Dice are rarely
>unbiased, and neither are roulette wheels. Nevertheless, we ignore those
>factual biases for the sake of simplicity.

Simlplicity is fine.  But if you use this for real world decisions,
such as 'Let's try one more time, I really want a daughter!'
because you didn't know that the simplicity is there, or rather
that what is true in populations isn't true because it is also
true for each individual in a population .. well, that can be tragic.

>Correct. And more Y sperm die than X sperm. Together, these two factors lead
>to a small but real bias towards girl children, at least on countries that
>don't practice wide-spread sex-specific abortion.

It is not clear, last I checked, that 'more Y sperm die' matters, because
most of the time, for reasons we still don't know, the ratio of male
and female sperm produced is not 50/50 either.  So it may matter.  But
maybe not.

>> But when you look at breeding age men and women you will find 
>> that the ratio is a lot closer to 50/50 than the birth age ratio.
>> Human beings, as a population currently produce more males than
>> females.
>
>Surely that depends on where (and when) you are?
>
>I know that, today, both India and China both abort far more female fetuses
>than male ones, leading to a large excess of men. But in countries that
>don't practice selective abortions, my understanding is that there is a
>small excess of women at virtually all ages, especially among the elderly.
>Men tend to die earlier than women in every age bracket.

No.  Before breeding age there is an excess of males.

I am excluding the notion of abortion here.  (Another reason why humans
make lousy experimental subjects for this sort of study.)  What I am
saying is that a whole lot more boy babies get born than female ones.
If infant mortality was sex-neutral, we would expect that, when we
took a census of 16-year-olds we would find that there continued to
be an excess of boys at that level, too.  Instead, we have a small
excess of females, enough to believe that:

       For populations of human beings, when they hit breeding age, it
       is desirable to have a sex ratio near 50/50 but with a small
       excess of females.

'Desirable' here just means that this is the answer that pops out when
you study real populations (or model them with computer simulations).

And to get this Desirable outcome, you have to produce many more male
babies than female ones.

Laura



From rosuav at gmail.com  Wed Jun 17 09:52:02 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 17 Jun 2015 23:52:02 +1000
Subject: Creating .exe file in Python
In-Reply-To: <mlrssm$j8p$1@dont-email.me>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
 <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
 <mailman.547.1434547236.13271.python-list@python.org>
 <mlrssm$j8p$1@dont-email.me>
Message-ID: <CAPTjJmqdoOxH5XNQYZ5f29ZC9khtC6y0=QG9Z1rK1QMnUonwCQ@mail.gmail.com>

On Wed, Jun 17, 2015 at 11:33 PM, hamilton <hamilton at nothere.com> wrote:
> On 6/17/2015 7:20 AM, Chris Angelico wrote:
>>
>> On Wed, Jun 17, 2015 at 11:10 PM,  <subhabrata.banerji at gmail.com> wrote:
>>>
>>> Thank you all. It seems going fine now. I have one additional question if
>>> I run the .exe files created in Non Python Windows environment. Linux has
>>> Python builtin but in Non Python environment how may I run it? Is there any
>>> set of prequisites I have to follow. I am not finding much web help, if any
>>> one may kindly suggest.
>>>
>>
>> There have been some proposals to make an easy way to produce a single
>> package that has a Windows executable header, but can be easily
>> unpacked and run using a system-provided Linux or Mac OS Python.
>> However, I don't think any have been deployed yet. So the simple
>> answer is: You can't do what you're trying to do. Instead, take a step
>> back, and look at just getting your Python program to run on all
>> platforms... and that's easy, just distribute the .py files. Much MUCH
>> easier, more reliable, and simpler.
>>
>> ChrisA
>>
> Yes, this is easy.
>
> However, the python source can be read by anyone.
>
> As a .exe, the source can not be read.
>
> Just because the interpreter is open source,
> does not mean my application should be.

Actually, all you've done is obscure your code. This is a very common
fallacy; people think that since they cannot themselves read the code
out of what they're distributing, nor can anyone else. There's no way
to completely stop people from reading your code other than to not
give them the code.

If you truly want to protect your code from prying eyes, therefore,
there's only one way to do it: host it on a server, and let people
access the server without seeing the code. This is true of all
languages, not just Python, and it's always been the way. You can try
to put legal guards around your code ("opening this package
constitutes acceptance, blah blah, you agree that you will not reverse
engineer, disassemble, blah blah"), but that won't stop people. It's
best to either give them the code and let them see it clearly (open
source) or keep the code away from them in all forms (server-side
deployment).

ChrisA


From lac at openend.se  Wed Jun 17 09:54:56 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 17 Jun 2015 15:54:56 +0200
Subject: Creating .exe file in Python
In-Reply-To: Message from subhabrata.banerji@gmail.com of "Wed,
 17 Jun 2015 06:10:45 -0700."
 <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org><9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
Message-ID: <201506171354.t5HDsuW5021916@fido.openend.se>

In a message of Wed, 17 Jun 2015 06:10:45 -0700, subhabrata.banerji at gmail.com w
rites:

>Dear Group,
>
>Thank you all. It seems going fine now. I have one additional question if I run the .exe files created in Non Python Windows environment. Linux has Python builtin but in Non Python environment how may I run it? Is there any set of prequisites I have to follow. I am not finding much web help, if any one may kindly suggest. 
>
>Regards,
>Subhabrata Banerjee.

If you use py2exe or PyInstaller (or whatever it is you used) to make
your exe, then all of this is handled for you.  Hand the windows users
the exe and they are good to go.  But the last time you posted here,
your py2exe wasn't working well.  You were getting Import Errors.
If you make a broken exe, well, it will be broken for the windows
users as well.

Laura




From lac at openend.se  Wed Jun 17 10:00:24 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 17 Jun 2015 16:00:24 +0200
Subject: Problem with PyPI login
In-Reply-To: Message from Miki Tebeka <miki.tebeka@gmail.com> of "Wed,
 17 Jun 2015 06:49:08 -0700."
 <4127850f-e1a8-4924-837d-2a709a026e88@googlegroups.com>
References: <4127850f-e1a8-4924-837d-2a709a026e88@googlegroups.com>
Message-ID: <201506171400.t5HE0OE5022125@fido.openend.se>

In a message of Wed, 17 Jun 2015 06:49:08 -0700, Miki Tebeka writes:
>Greetings,
>
>After confirming email I can't login to PyPI with the password and when trying to rest I get: user "Cyberint" is unknwon to me. However when trying to re-register it says the user is already there.
>
>The forums at https://sourceforge.net/p/pypi/support-requests/ (which is linked from "PyPI Support") seems pretty deserted. What is the right location for these kind of issues?
>
>Thanks,
>--
>Miki

That is one place.
https://bitbucket.org/pypa/pypi/issues

is another.

Good luck,
Laura



From subhabrata.banerji at gmail.com  Wed Jun 17 10:04:22 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Wed, 17 Jun 2015 07:04:22 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.550.1434549305.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org><9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
 <mailman.550.1434549305.13271.python-list@python.org>
Message-ID: <480c4ec4-ef15-420e-84a2-d66ff8da59eb@googlegroups.com>

On Wednesday, June 17, 2015 at 7:25:39 PM UTC+5:30, Laura Creighton wrote:
> In a message of Wed, 17 Jun 2015 06:10:45 -0700, w
> rites:
> 
> >Dear Group,
> >
> >Thank you all. It seems going fine now. I have one additional question if I run the .exe files created in Non Python Windows environment. Linux has Python builtin but in Non Python environment how may I run it? Is there any set of prequisites I have to follow. I am not finding much web help, if any one may kindly suggest. 
> >
> >Regards,
> >Subhabrata Banerjee.
> 
> If you use py2exe or PyInstaller (or whatever it is you used) to make
> your exe, then all of this is handled for you.  Hand the windows users
> the exe and they are good to go.  But the last time you posted here,
> your py2exe wasn't working well.  You were getting Import Errors.
> If you make a broken exe, well, it will be broken for the windows
> users as well.
> 
> Laura

Thanks. I will check. I am calling the .exe from C:\Python27\dist... would it run in non Python Windows Environment? Regards, Subhabrata Banerjee.


From invalid at invalid.invalid  Wed Jun 17 10:04:29 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Wed, 17 Jun 2015 14:04:29 +0000 (UTC)
Subject: Testing random
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
 <abcf102d-e523-49bb-94e9-9a6d15971021@googlegroups.com>
 <CAPTjJmqLyTaMd85OzaKhdb_MpBsJbBg6UwkVbB2fVSf2pN-=5A@mail.gmail.com>
 <mailman.543.1434530510.13271.python-list@python.org>
Message-ID: <mlrupd$i93$1@reader1.panix.com>

On 2015-06-17, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:

> An alternative explanation is that he's just a plain, old fashioned 
> troll, as pointed out by Denis McMahon some weeks ago.  Now what is
> the probability of that? :)

Looks to me like it's asymptotically approaching 1.  I'm going to stop
paying attention now, but I think somebody should keep this thread
going for an infinite number of posts to see if it ever really does
get to 1.

-- 
Grant Edwards               grant.b.edwards        Yow! Four thousand
                                  at               different MAGNATES, MOGULS
                              gmail.com            & NABOBS are romping in my
                                                   gothic solarium!!


From breamoreboy at yahoo.co.uk  Wed Jun 17 10:10:07 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 17 Jun 2015 15:10:07 +0100
Subject: Problem with PyPI login
In-Reply-To: <4127850f-e1a8-4924-837d-2a709a026e88@googlegroups.com>
References: <4127850f-e1a8-4924-837d-2a709a026e88@googlegroups.com>
Message-ID: <mlrv40$vhh$1@ger.gmane.org>

On 17/06/2015 14:49, Miki Tebeka wrote:
> Greetings,
>
> After confirming email I can't login to PyPI with the password and when trying to rest I get: user "Cyberint" is unknwon to me. However when trying to re-register it says the user is already there.
>
> The forums at https://sourceforge.net/p/pypi/support-requests/ (which is linked from "PyPI Support") seems pretty deserted. What is the right location for these kind of issues?
>
> Thanks,
> --
> Miki
>

I guess https://bitbucket.org/pypa/pypi/issues

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From subhabrata.banerji at gmail.com  Wed Jun 17 10:16:33 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Wed, 17 Jun 2015 07:16:33 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
Message-ID: <67f1c9b8-fa76-45ca-a6a1-dcb29208d333@googlegroups.com>

On Monday, June 15, 2015 at 5:12:24 PM UTC+5:30, subhabrat... at gmail.com wrote:
> Dear Group,
> 
> I am trying to learn how to create .exe file for Python. I tried to work around 
> http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program went nice.
> But if I try to make exe for larger programs with methods and classes I am getting error. 
> 
> If any one of the esteemed members may kindly suggest how to work out. 
> I am using Python2.7+ on Windows 7 Professional. 
> 
> Regards,
> Subhabrata Banerjee.

But if I am moving the file from Python environment and to another folder in C: drive it is not running, it is giving me following error messages,
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\tutorial>hello.exe
LoadLibrary(pythondll) failedThe specified module could not be found.
C:\tutorial\PYTHON27.DLL
C:\tutorial>

Regards,
Subhabrata Banerjee. 


From random832 at fastmail.us  Wed Jun 17 10:17:35 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 17 Jun 2015 10:17:35 -0400
Subject: Creating .exe file in Python
In-Reply-To: <mlrssm$j8p$1@dont-email.me>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
 <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
 <mailman.547.1434547236.13271.python-list@python.org>
 <mlrssm$j8p$1@dont-email.me>
Message-ID: <1434550655.3796249.298024217.353C5B17@webmail.messagingengine.com>

On Wed, Jun 17, 2015, at 09:33, hamilton wrote:
> However, the python source can be read by anyone.
> 
> As a .exe, the source can not be read.
> 
> Just because the interpreter is open source,
> does not mean my application should be.

Being readable isn't the same thing as being open source. If someone
steals your application, you can sue them.


From miki.tebeka at gmail.com  Wed Jun 17 10:19:20 2015
From: miki.tebeka at gmail.com (Miki Tebeka)
Date: Wed, 17 Jun 2015 07:19:20 -0700 (PDT)
Subject: Problem with PyPI login
In-Reply-To: <mailman.551.1434549633.13271.python-list@python.org>
References: <4127850f-e1a8-4924-837d-2a709a026e88@googlegroups.com>
 <mailman.551.1434549633.13271.python-list@python.org>
Message-ID: <9c923d33-aff9-49fb-90bd-606877c5b725@googlegroups.com>

> >The forums at https://sourceforge.net/p/pypi/support-requests/ (which is linked from "PyPI Support") seems pretty deserted. What is the right location for these kind of issues?

> That is one place.
> https://bitbucket.org/pypa/pypi/issues

Thanks!


From ned at nedbatchelder.com  Wed Jun 17 10:26:10 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 17 Jun 2015 07:26:10 -0700 (PDT)
Subject: operations on types
In-Reply-To: <55815f58$0$1662$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.531.1434500646.13271.python-list@python.org>
 <5580d5c8$0$1667$c3e8da3$5496439d@news.astraweb.com>
 <mailman.539.1434517186.13271.python-list@python.org>
 <55815f58$0$1662$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <9312f979-209d-4778-bb9d-9d3204b543fc@googlegroups.com>

On Wednesday, June 17, 2015 at 7:52:05 AM UTC-4, Steven D'Aprano wrote:
> On Wed, 17 Jun 2015 02:58 pm, Ian Kelly wrote:
> 
> > On Tue, Jun 16, 2015 at 8:04 PM, Steven D'Aprano
> > <steve+comp.lang.python at pearwood.info> wrote:
> >> I'm not sure why you are taking "Dr Bigcock" seriously. You know he isn't
> >> actually a doctor?
> > 
> > I'm reasonably certain that they're actually the same person, and he's
> > now engaging in sockpuppetry [1] by starting conversations with
> > himself. He's not trying very hard to hide it, signing posts with both
> > accounts as "zipher" and then lamp-shading that fact by commenting on
> > it. I can't even find any record that "Dr. Bigcock" ever actually
> > posted the text that "Dr. John Q. Hacker" is now responding to.
> 
> 
> I must admit that the possibility of a sockpuppet crossed my mind too, even
> to the point of checking the headers of their posts to see if they came
> from the same IP address. (Inconclusive.)
> 
> At the moment, I'm still willing to give Dr Hacker the benefit of the doubt
> re the sockpuppet suspicion.

If you search for "zondervanz", you will find the GitHub account theProhpet
with that email address: https://github.com/theProphet

There you will find a repo with commits like this one where theProphet adds
attributions to Mark Janssen:

https://github.com/theProphet/GlassBeadGame/commit/a1f5082feb76fb0d51b069dc81de2228ff36a221

The INSTALL file lists the owner as Mark:

https://github.com/theProphet/GlassBeadGame/blob/master/INSTALL

The Sourceforge page from that INSTALL is pangaia.sf.net, which 
lists the Twitter account https://twitter.com/Xer0Dynamite, which 
includes this tweet, asking to be called Dr Bigcock:

https://twitter.com/Xer0Dynamite/status/594217063725592577

We've been through this before.  Mark is energetic, and tantalizing,
but not worth engaging.

--Ned.


From subhabrata.banerji at gmail.com  Wed Jun 17 10:30:45 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Wed, 17 Jun 2015 07:30:45 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.553.1434550667.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
 <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
 <mailman.547.1434547236.13271.python-list@python.org>
 <mlrssm$j8p$1@dont-email.me>
 <mailman.553.1434550667.13271.python-list@python.org>
Message-ID: <d3a76974-7d4a-4573-bf13-1967e6219bff@googlegroups.com>

On Wednesday, June 17, 2015 at 7:47:59 PM UTC+5:30, rand... at fastmail.us wrote:
> On Wed, Jun 17, 2015, at 09:33, hamilton wrote:
> > However, the python source can be read by anyone.
> > 
> > As a .exe, the source can not be read.
> > 
> > Just because the interpreter is open source,
> > does not mean my application should be.
> 
> Being readable isn't the same thing as being open source. If someone
> steals your application, you can sue them.

Guido can create another Python. I know my limits. So I never steal or copy unnecessarily I learn. 


From rosuav at gmail.com  Wed Jun 17 10:38:45 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 18 Jun 2015 00:38:45 +1000
Subject: Creating .exe file in Python
In-Reply-To: <1434550655.3796249.298024217.353C5B17@webmail.messagingengine.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
 <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
 <mailman.547.1434547236.13271.python-list@python.org>
 <mlrssm$j8p$1@dont-email.me>
 <1434550655.3796249.298024217.353C5B17@webmail.messagingengine.com>
Message-ID: <CAPTjJmo7b-j=rc-HySQg73CRg5DAb=70ys_Mcso3AYMiHOxKwg@mail.gmail.com>

On Thu, Jun 18, 2015 at 12:17 AM,  <random832 at fastmail.us> wrote:
> On Wed, Jun 17, 2015, at 09:33, hamilton wrote:
>> However, the python source can be read by anyone.
>>
>> As a .exe, the source can not be read.
>>
>> Just because the interpreter is open source,
>> does not mean my application should be.
>
> Being readable isn't the same thing as being open source. If someone
> steals your application, you can sue them.

Right, though most people who look at obscuring their code are more
concerned with making it impossible (as opposed to illegal) to see the
source. And that's the one that doesn't work.

ChrisA


From steve at pearwood.info  Wed Jun 17 10:59:08 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 18 Jun 2015 00:59:08 +1000
Subject: Set a flag on the function or a global?
References: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
 <mailman.530.1434500492.13271.python-list@python.org>
Message-ID: <55818b3c$0$1665$c3e8da3$5496439d@news.astraweb.com>

On Wed, 17 Jun 2015 09:51 am, Cameron Simpson wrote:

> On 16Jun2015 18:18, Steven D'Aprano <steve+comp.lang.python at pearwood.info>
> wrote:
>>On Tuesday 16 June 2015 10:35, MRAB wrote:
>>> On 2015-06-16 01:24, sohcahtoa82 at gmail.com wrote:
>>>> Using a keyword argument for the edir function is the most intuitive
>>>> and easy to read, IMO.
>>
>>edir() has a keyword argument: edir(x, dunders=False) suppresses the
>>return of dunder names. But since the primary purpose of [e]dir is, in my
>>opinion, to be used interactively, needing to type an extra 15 characters
>>to hide dunders is too inconvenient.
> 
> I'm just catching up here, but have now read the whole thread.

Thanks for doing so before commenting :-)


> I am personally against a global (==> single variable affecting all
> callers) of any kind, be it a function attribute or whatever. Why? For
> that same reason that we discourage use of functions like os.chdir or
> os.umask except in rare circumstances: the single call inevitably affects
> the entire program behaviour.

Well yes. And I would normally agree. But I think a better analogy here is
with print. By default, print outputs to stdout. You can customise that by
giving an extra argument to print:


print(spam, file=sys.stderr)  # Python 3
print >>sys.stderr, spam  # Python 2


or you can re-direct sys.stdout. Both are allowed. But doing the later is
potentially more disruptive, and should be done with care. Nevertheless, it
is supported, and Python will set up stdout to the appropriate file for you
when you start.

Although I think you're being overly cautious, I've changed the behaviour:

- the default is dunders=True, always;

- you can manually override the default with an optional, explicit dunders
keyword-only argument, always;

- you can change the default to dunders=False by setting an attribute, only
when running in interactive mode.



-- 
Steven



From ahlusar.ahluwalia at gmail.com  Wed Jun 17 11:00:11 2015
From: ahlusar.ahluwalia at gmail.com (Saran A)
Date: Wed, 17 Jun 2015 08:00:11 -0700 (PDT)
Subject: JSON Object to CSV file
Message-ID: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>

I would like to have this JSON object written out to a CSV file so that the keys are header fields (for each of the columns) and the values are values that are associated with each header field. Is there a best practice for working with this? Ideally I would like to recursively iterate through the key value pairs. Thank you in advance. I am using Python 3.4 on Windows. My editor is Sublime 2. 

{
"CF": {
"A": "5",
"FEC": "1/1/0001 12:00:00 AM",
"TE": null,
"Locator": null,
"Message": "Transfer Fee",
"AT": null,
"FT": null,
"FR": "True",
"FY": null,
"FR": null,
"FG": "0",
"Comment": null,
"FUD": null,
"cID": null,
"GEO": null,
"ISO": null,
"TRID": null,
"XTY": "931083",
"ANM": null,
"NM": null
},
"CF": "Fee",
"ID": "2"
}


From ian.g.kelly at gmail.com  Wed Jun 17 11:01:39 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 17 Jun 2015 09:01:39 -0600
Subject: Testing random
In-Reply-To: <mlrupd$i93$1@reader1.panix.com>
References: <87oaksowwg.fsf@Equus.decebal.nl>
 <1451048.pW9z17ilMA@PointedEars.de>
 <mailman.242.1433677915.13271.python-list@python.org>
 <3158703.Lr4HFMbMOd@PointedEars.de>
 <mailman.257.1433694323.13271.python-list@python.org>
 <1656356.OjxBvjpJ5d@PointedEars.de>
 <mailman.259.1433695485.13271.python-list@python.org>
 <5515034.V7dcXEWAvK@PointedEars.de>
 <88ba163e-74f4-40fa-ab3e-1feeb051b961@googlegroups.com>
 <17701105.fuEQZlnPee@PointedEars.de>
 <ff4a8ffd-50fe-4705-b33a-fcf63bcccc11@googlegroups.com>
 <2089101.KkvqnN9Xah@PointedEars.de>
 <dfc293e6-01b8-4179-92c3-2d7c2ffcbbe8@googlegroups.com>
 <3049728.zPiIGXGd9A@PointedEars.de>
 <abcf102d-e523-49bb-94e9-9a6d15971021@googlegroups.com>
 <CAPTjJmqLyTaMd85OzaKhdb_MpBsJbBg6UwkVbB2fVSf2pN-=5A@mail.gmail.com>
 <mailman.543.1434530510.13271.python-list@python.org>
 <mlrupd$i93$1@reader1.panix.com>
Message-ID: <CALwzidnBHBw=QUHEh83w48NhpJskx7OjQWxL2VcpZJr8wnGmLg@mail.gmail.com>

On Wed, Jun 17, 2015 at 8:04 AM, Grant Edwards <invalid at invalid.invalid> wrote:
> On 2015-06-17, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>
>> An alternative explanation is that he's just a plain, old fashioned
>> troll, as pointed out by Denis McMahon some weeks ago.  Now what is
>> the probability of that? :)
>
> Looks to me like it's asymptotically approaching 1.  I'm going to stop
> paying attention now, but I think somebody should keep this thread
> going for an infinite number of posts to see if it ever really does
> get to 1.

He's apparently closed out the thread already. If he is a troll, at
least he's one who eventually gets bored and leaves. I think it's more
likely though that he just has a massively overblown ego that won't
allow him to admit (maybe even consider) that he may have made a
mistake.


From lac at openend.se  Wed Jun 17 11:06:47 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 17 Jun 2015 17:06:47 +0200
Subject: Set a flag on the function or a global?
In-Reply-To: Message from "Steven D'Aprano" <steve@pearwood.info> of "Thu,
 18 Jun 2015 00:59:08 +1000."
 <55818b3c$0$1665$c3e8da3$5496439d@news.astraweb.com>
References: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
 <mailman.530.1434500492.13271.python-list@python.org><55818b3c$0$1665$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <201506171506.t5HF6lYf023821@fido.openend.se>

To figure out what I like, I would need to play with edir, and the
suite that it comes with.

I suspect there is command like:

stop_showing_me_all_this_uninteresting_stuff = True

in my future, and dunders is only a small part of that.

Laura



From lac at openend.se  Wed Jun 17 11:21:37 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 17 Jun 2015 17:21:37 +0200
Subject: Creating .exe file in Python
In-Reply-To: Message from subhabrata.banerji@gmail.com of "Wed,
 17 Jun 2015 07:16:33 -0700."
 <67f1c9b8-fa76-45ca-a6a1-dcb29208d333@googlegroups.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com><67f1c9b8-fa76-45ca-a6a1-dcb29208d333@googlegroups.com>
Message-ID: <201506171521.t5HFLbMk024216@fido.openend.se>

In a message of Wed, 17 Jun 2015 07:16:33 -0700, subhabrata.banerji at gmail.com w
rites:
>On Monday, June 15, 2015 at 5:12:24 PM UTC+5:30, subhabrat... at gmail.com wrote:
>> Dear Group,
>> 
>> I am trying to learn how to create .exe file for Python. I tried to work around 
>> http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program went nice.
>> But if I try to make exe for larger programs with methods and classes I am getting error. 
>> 
>> If any one of the esteemed members may kindly suggest how to work out. 
>> I am using Python2.7+ on Windows 7 Professional. 
>> 
>> Regards,
>> Subhabrata Banerjee.
>
>But if I am moving the file from Python environment and to another folder in C: drive it is not running, it is giving me following error messages,
>Microsoft Windows [Version 6.1.7601]
>Copyright (c) 2009 Microsoft Corporation.  All rights reserved.
>
>C:\tutorial>hello.exe
>LoadLibrary(pythondll) failedThe specified module could not be found.
>C:\tutorial\PYTHON27.DLL
>C:\tutorial>
>
>Regards,
>Subhabrata Banerjee. 

You have to distribute the DLL as well.
But at this point you need somebody who knows more about packaging
for windows than I do.  It's not something I do often, or well.

Laura


From subhabrata.banerji at gmail.com  Wed Jun 17 11:30:25 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Wed, 17 Jun 2015 08:30:25 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <mailman.557.1434554505.13271.python-list@python.org>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com><67f1c9b8-fa76-45ca-a6a1-dcb29208d333@googlegroups.com>
 <mailman.557.1434554505.13271.python-list@python.org>
Message-ID: <610d75de-ee89-4149-b513-28dc2e69c19a@googlegroups.com>

On Wednesday, June 17, 2015 at 8:52:16 PM UTC+5:30, Laura Creighton wrote:
> In a message of Wed, 17 Jun 2015 07:16:33 -0700,  w
> rites:
> >On Monday, June 15, 2015 at 5:12:24 PM UTC+5:30, wrote:
> >> Dear Group,
> >> 
> >> I am trying to learn how to create .exe file for Python. I tried to work around 
> >> http://www.py2exe.org/index.cgi/Tutorial of Py2exe. The sample program went nice.
> >> But if I try to make exe for larger programs with methods and classes I am getting error. 
> >> 
> >> If any one of the esteemed members may kindly suggest how to work out. 
> >> I am using Python2.7+ on Windows 7 Professional. 
> >> 
> >> Regards,
> >> Subhabrata Banerjee.
> >
> >But if I am moving the file from Python environment and to another folder in C: drive it is not running, it is giving me following error messages,
> >Microsoft Windows [Version 6.1.7601]
> >Copyright (c) 2009 Microsoft Corporation.  All rights reserved.
> >
> >C:\tutorial>hello.exe
> >LoadLibrary(pythondll) failedThe specified module could not be found.
> >C:\tutorial\PYTHON27.DLL
> >C:\tutorial>
> >
> >Regards,
> >Subhabrata Banerjee. 
> 
> You have to distribute the DLL as well.
> But at this point you need somebody who knows more about packaging
> for windows than I do.  It's not something I do often, or well.
> 
> Laura

Hi,
My need is done. But I can learn. I saw few .pyd files in dist folder.
Regards,
Subhabrata Banerjee. 


From steve at pearwood.info  Wed Jun 17 11:39:20 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 18 Jun 2015 01:39:20 +1000
Subject: Creating .exe file in Python
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
 <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
 <mailman.547.1434547236.13271.python-list@python.org>
 <mlrssm$j8p$1@dont-email.me>
 <mailman.549.1434549125.13271.python-list@python.org>
Message-ID: <558194a8$0$1672$c3e8da3$5496439d@news.astraweb.com>

On Wed, 17 Jun 2015 11:52 pm, Chris Angelico wrote:

> If you truly want to protect your code from prying eyes, therefore,
> there's only one way to do it: host it on a server, and let people
> access the server without seeing the code.

Stop giving people ideas.



-- 
Steven



From steve at pearwood.info  Wed Jun 17 11:42:33 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 18 Jun 2015 01:42:33 +1000
Subject: operations on types
References: <mailman.531.1434500646.13271.python-list@python.org>
 <5580d5c8$0$1667$c3e8da3$5496439d@news.astraweb.com>
 <mailman.539.1434517186.13271.python-list@python.org>
 <55815f58$0$1662$c3e8da3$5496439d@news.astraweb.com>
 <9312f979-209d-4778-bb9d-9d3204b543fc@googlegroups.com>
Message-ID: <55819568$0$1672$c3e8da3$5496439d@news.astraweb.com>

On Thu, 18 Jun 2015 12:26 am, Ned Batchelder wrote:

> On Wednesday, June 17, 2015 at 7:52:05 AM UTC-4, Steven D'Aprano wrote:

>> At the moment, I'm still willing to give Dr Hacker the benefit of the
>> doubt re the sockpuppet suspicion.
> 
> If you search for "zondervanz", you will find the GitHub account
> theProhpet with that email address: https://github.com/theProphet
[...]

Ah, well that certainly clears up any lingering doubt. Thanks for your
detective work.


-- 
Steven



From ahlusar.ahluwalia at gmail.com  Wed Jun 17 11:45:15 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Wed, 17 Jun 2015 08:45:15 -0700 (PDT)
Subject: JSON Object to CSV file
In-Reply-To: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
References: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
Message-ID: <5747bb7f-04c0-4097-8c09-1102499b49b2@googlegroups.com>

On Wednesday, June 17, 2015 at 11:00:24 AM UTC-4, Saran A wrote:
> I would like to have this JSON object written out to a CSV file so that the keys are header fields (for each of the columns) and the values are values that are associated with each header field. Is there a best practice for working with this? Ideally I would like to recursively iterate through the key value pairs. Thank you in advance. I am using Python 3.4 on Windows. My editor is Sublime 2. 
> 
> {
> "CF": {
> "A": "5",
> "FEC": "1/1/0001 12:00:00 AM",
> "TE": null,
> "Locator": null,
> "Message": "Transfer Fee",
> "AT": null,
> "FT": null,
> "FR": "True",
> "FY": null,
> "FR": null,
> "FG": "0",
> "Comment": null,
> "FUD": null,
> "cID": null,
> "GEO": null,
> "ISO": null,
> "TRID": null,
> "XTY": "931083",
> "ANM": null,
> "NM": null
> },
> "CF": "Fee",
> "ID": "2"
> }

My apologies:

I originally parsed this from an XML file.

It really should be:

{
"Fee": {
"A": "5",
"FEC": "1/1/0001 12:00:00 AM",
"TE": null,
"Locator": null,
"Message": "Transfer Fee",
"AT": null,
"FT": null,
"FR": "True",
"FY": null,
"FR": null,
"FG": "0",
"Comment": null,
"FUD": null,
"cID": null,
"GEO": null,
"ISO": null,
"TRID": null,
"XTY": "931083",
"ANM": null,
"NM": null
},
"CF": "Fee",
"ID": "2"
}

The value, "Fee" associated with the key, "CF" does should not be included as a column header.

The CSV file, when opened with an application such as MS Excel, should be as follows:

(Column Header)----> CF               A          FEC

(Field Value)---->   Fee              5        1/1/0001 12:00:00 AM

My apologies for an confusion.


From subhabrata.banerji at gmail.com  Wed Jun 17 11:46:50 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Wed, 17 Jun 2015 08:46:50 -0700 (PDT)
Subject: Creating .exe file in Python
In-Reply-To: <558194a8$0$1672$c3e8da3$5496439d@news.astraweb.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
 <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
 <mailman.547.1434547236.13271.python-list@python.org>
 <mlrssm$j8p$1@dont-email.me>
 <mailman.549.1434549125.13271.python-list@python.org>
 <558194a8$0$1672$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <ba23b248-4a9d-45c1-a2cf-bca12190bf29@googlegroups.com>

On Wednesday, June 17, 2015 at 9:09:32 PM UTC+5:30, Steven D'Aprano wrote:
> On Wed, 17 Jun 2015 11:52 pm, Chris Angelico wrote:
> 
> > If you truly want to protect your code from prying eyes, therefore,
> > there's only one way to do it: host it on a server, and let people
> > access the server without seeing the code.
> 
> Stop giving people ideas.
> 
> 
> 
> -- 
> Steven

Why sir?
Regards,
Subhabrata Banerjee. 


From steve at pearwood.info  Wed Jun 17 11:55:36 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 18 Jun 2015 01:55:36 +1000
Subject: Set a flag on the function or a global?
References: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
 <mailman.530.1434500492.13271.python-list@python.org><55818b3c$0$1665$c3e8da3$5496439d@news.astraweb.com>
 <mailman.556.1434553624.13271.python-list@python.org>
Message-ID: <55819878$0$1658$c3e8da3$5496439d@news.astraweb.com>

On Thu, 18 Jun 2015 01:06 am, Laura Creighton wrote:

> To figure out what I like, I would need to play with edir, and the
> suite that it comes with.
> 
> I suspect there is command like:
> 
> stop_showing_me_all_this_uninteresting_stuff = True
> 
> in my future, and dunders is only a small part of that.


The full signature is:

edir([object [, glob=''] [, dunders=True] [, meta=False]])

All four arguments are optional, and dunders and meta are keyword-only. It
is intended to work with Python 2.4 or better. It may not work in Jython on
IronPython, depending on whether or not sys._getframe is supported.

The glob argument is the most interesting, in my opinion. If you give a
non-empty glob, only names matching that glob are returned:

py> dir('', 'is*')
['isalnum', 'isalpha', 'isdigit', 'islower', 'isspace', 'istitle', 'isupper']


If the glob contains no metacharacters, a plain substring match is done:

py> dir('', 'up')
['isupper', 'upper']


Matches are case-insensitive by default, but can be made case-sensitive by
using a prefix "=". To invert the match (return names which don't match),
use "!" as the prefix. You can use "!=" as a combined prefix.

I haven't yet released this as an independent package as yet, but you can
find it here:

https://code.google.com/p/my-startup-file/

in the "enhanced_dir.py" module.

Consider it beta quality and free for personal use; if anyone wishes a more
formal licence, either wait until I publish it independently, or contact me
off list.




-- 
Steven



From rosuav at gmail.com  Wed Jun 17 11:58:52 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 18 Jun 2015 01:58:52 +1000
Subject: Creating .exe file in Python
In-Reply-To: <558194a8$0$1672$c3e8da3$5496439d@news.astraweb.com>
References: <349bb66f-bd3c-47ec-bd1d-35f46d23cf95@googlegroups.com>
 <2219d0b2-24a4-498b-b981-456faf0ceb44@googlegroups.com>
 <mailman.487.1434385760.13271.python-list@python.org>
 <5f6b802f-049b-48e3-b1fd-a5915b7f3dbb@googlegroups.com>
 <mailman.515.1434467117.13271.python-list@python.org>
 <2de8f2c1-5a92-4f8b-be2d-c793de69d463@googlegroups.com>
 <mailman.518.1434470627.13271.python-list@python.org>
 <9e068291-09c2-4b2a-8e1c-8803050cfba1@googlegroups.com>
 <mailman.547.1434547236.13271.python-list@python.org>
 <mlrssm$j8p$1@dont-email.me>
 <mailman.549.1434549125.13271.python-list@python.org>
 <558194a8$0$1672$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmrGAXEBvJQQMQkjH6CQD=z2pAYa38abws7q8aExwAHUSw@mail.gmail.com>

On Thu, Jun 18, 2015 at 1:39 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Wed, 17 Jun 2015 11:52 pm, Chris Angelico wrote:
>
>> If you truly want to protect your code from prying eyes, therefore,
>> there's only one way to do it: host it on a server, and let people
>> access the server without seeing the code.
>
> Stop giving people ideas.

Not sure what the problem here is. There are plenty of servers that
run closed-source software. Yes, there are people like rms who believe
that *all* software should be free, but I don't agree. There are
people who wish to keep their code to themselves, and they should be
allowed to do that. I'm just pointing out that they then have to go to
the effort of making the *service* (rather than the software)
available.

ChrisA


From torriem at gmail.com  Wed Jun 17 12:48:24 2015
From: torriem at gmail.com (Michael Torrie)
Date: Wed, 17 Jun 2015 10:48:24 -0600
Subject: operations on types
In-Reply-To: <55819568$0$1672$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.531.1434500646.13271.python-list@python.org>
 <5580d5c8$0$1667$c3e8da3$5496439d@news.astraweb.com>
 <mailman.539.1434517186.13271.python-list@python.org>
 <55815f58$0$1662$c3e8da3$5496439d@news.astraweb.com>
 <9312f979-209d-4778-bb9d-9d3204b543fc@googlegroups.com>
 <55819568$0$1672$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5581A4D8.5060000@gmail.com>

On 06/17/2015 09:42 AM, Steven D'Aprano wrote:
> On Thu, 18 Jun 2015 12:26 am, Ned Batchelder wrote:
> 
>> On Wednesday, June 17, 2015 at 7:52:05 AM UTC-4, Steven D'Aprano wrote:
> 
>>> At the moment, I'm still willing to give Dr Hacker the benefit of the
>>> doubt re the sockpuppet suspicion.
>>
>> If you search for "zondervanz", you will find the GitHub account
>> theProhpet with that email address: https://github.com/theProphet
> [...]
> 
> Ah, well that certainly clears up any lingering doubt. Thanks for your
> detective work.

Indeed thank you. I've added this new email address to my filter (though
gmail's filters are rather awkward).


From torriem at gmail.com  Wed Jun 17 13:00:57 2015
From: torriem at gmail.com (Michael Torrie)
Date: Wed, 17 Jun 2015 11:00:57 -0600
Subject: Trying to configure Apache and Python 2.7 on Red Hat I get 403
 Forbidden
In-Reply-To: <CAMw+j7JXD7H29XJUyW_W18gPNZSPfYXBbTz3BZ7XSLPxPGqwnw@mail.gmail.com>
References: <CALyLaQKsLBh7DvqaDRaLnceAE_zRBB2p0Yg+Vm8YXukQ=x98OA@mail.gmail.com>
 <CABD09x9JQrb0dUXf7JNbqJT7OsDDgM6mftU9PcBU8YFP4PXdvQ@mail.gmail.com>
 <mailman.499.1434425540.13271.python-list@python.org>
 <557feaf7$0$1655$c3e8da3$5496439d@news.astraweb.com>
 <CALyLaQK5=HQcuHkTm9QHcs5p05NXvJ7j-wpi9GffqDrcA14GVg@mail.gmail.com>
 <CAMw+j7Lhpt8hs8uZ3CFFNyyntg_RPEqrN82cPG2bfCvU_0rTRw@mail.gmail.com>
 <CALyLaQJfL8iaxmPgj+WFj6WaV19xJwBoCWOeS2W-kw3NjRisWg@mail.gmail.com>
 <CAMw+j7JXD7H29XJUyW_W18gPNZSPfYXBbTz3BZ7XSLPxPGqwnw@mail.gmail.com>
Message-ID: <5581A7C9.3080902@gmail.com>

On 06/16/2015 12:19 PM, Chris Warrick wrote:
> On Tue, Jun 16, 2015 at 8:14 PM, N?stor Bosc?n <nestorjb at gmail.com> wrote:
>> I tried that but it didn't work.
>>
>>  I had to change /etc/selinux/config and reboot to make it work. It would be
>> nice if the wsgi module generated some log that explains why you get the
>> 403. There are several posibilities.
> 
> Well, that?s not wsgi?s fault.  There was a ?permission denied? error,
> that?s everything wsgi ever knew.
> As a RHEL sysadmin, you should know that this is most likely caused by
> braindead syslinux and to read the audit log.

The officially-blessed software collections has a version of python2.7
that should work fine with selinux, and it comes with a mod_wsgi module
that will plug it into the stock apache.

https://www.softwarecollections.org/en/scls/rhscl/python27/
https://rhn.redhat.com/errata/RHBA-2014-0610.html

I think the package is called python27-mod_wsgi




From __peter__ at web.de  Wed Jun 17 14:19:46 2015
From: __peter__ at web.de (Peter Otten)
Date: Wed, 17 Jun 2015 20:19:46 +0200
Subject: JSON Object to CSV file
References: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
 <5747bb7f-04c0-4097-8c09-1102499b49b2@googlegroups.com>
Message-ID: <mlsdo3$301$1@ger.gmane.org>

Sahlusar wrote:

> On Wednesday, June 17, 2015 at 11:00:24 AM UTC-4, Saran A wrote:
>> I would like to have this JSON object written out to a CSV file so that
>> the keys are header fields (for each of the columns) and the values are
>> values that are associated with each header field. Is there a best
>> practice for working with this? Ideally I would like to recursively
>> iterate through the key value pairs. Thank you in advance. I am using
>> Python 3.4 on Windows. My editor is Sublime 2.
>> 
>> {
>> "CF": {
>> "A": "5",
>> "FEC": "1/1/0001 12:00:00 AM",
>> "TE": null,
>> "Locator": null,
>> "Message": "Transfer Fee",
>> "AT": null,
>> "FT": null,
>> "FR": "True",
>> "FY": null,
>> "FR": null,
>> "FG": "0",
>> "Comment": null,
>> "FUD": null,
>> "cID": null,
>> "GEO": null,
>> "ISO": null,
>> "TRID": null,
>> "XTY": "931083",
>> "ANM": null,
>> "NM": null
>> },
>> "CF": "Fee",
>> "ID": "2"
>> }
> 
> My apologies:
> 
> I originally parsed this from an XML file.
> 
> It really should be:
> 
> {
> "Fee": {
> "A": "5",
> "FEC": "1/1/0001 12:00:00 AM",
> "TE": null,
> "Locator": null,
> "Message": "Transfer Fee",
> "AT": null,
> "FT": null,
> "FR": "True",
> "FY": null,
> "FR": null,
> "FG": "0",
> "Comment": null,
> "FUD": null,
> "cID": null,
> "GEO": null,
> "ISO": null,
> "TRID": null,
> "XTY": "931083",
> "ANM": null,
> "NM": null
> },
> "CF": "Fee",
> "ID": "2"
> }
> 
> The value, "Fee" associated with the key, "CF" does should not be included
> as a column header.
> 
> The CSV file, when opened with an application such as MS Excel, should be
> as follows:
> 
> (Column Header)----> CF               A          FEC
> 
> (Field Value)---->   Fee              5        1/1/0001 12:00:00 AM
> 
> My apologies for an confusion.

Forget about Excel, what should the CSV look like when opened in a text 
editor?

Here's my guess:


$ cat input.json
{
"Fee": {
"A": "5",
"FEC": "1/1/0001 12:00:00 AM",
"TE": null,
"Locator": null,
"Message": "Transfer Fee",
"AT": null,
"FT": null,
"FR": "True",
"FY": null,
"FR": null,
"FG": "0",
"Comment": null,
"FUD": null,
"cID": null,
"GEO": null,
"ISO": null,
"TRID": null,
"XTY": "931083",
"ANM": null,
"NM": null
},
"CF": "Fee",
"ID": "2"
}
$ cat json2csv.py
import csv
import json
import sys

def hook(obj):
    return obj

def flatten(obj):
    for k, v in obj:
        if isinstance(v, list):
            yield from flatten(v)
        else:
            yield k, v

if __name__ == "__main__":
    with open("input.json") as f:
        data = json.load(f, object_pairs_hook=hook)

    pairs = list(flatten(data))

    writer = csv.writer(sys.stdout)
    writer.writerow([k for k, v in pairs])
    writer.writerow([v for k, v in pairs])
$ python3 json2csv.py 
A,FEC,TE,Locator,Message,AT,FT,FR,FY,FR,FG,Comment,FUD,cID,GEO,ISO,TRID,XTY,ANM,NM,CF,ID
5,1/1/0001 12:00:00 AM,,,Transfer Fee,,,True,,,0,,,,,,,931083,,,Fee,2

But do you really want duplicate column names?



From suscricions at gmail.com  Wed Jun 17 15:21:19 2015
From: suscricions at gmail.com (Jason P.)
Date: Wed, 17 Jun 2015 12:21:19 -0700 (PDT)
Subject: Classic OOP in Python
Message-ID: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>

Hello Python community.

I come from a classic background in what refers to OOP. Mostly Java and PHP (> 5.3). I'm used to abstract classes, interfaces, access modifiers and so on.

Don't get me wrong. I know that despite the differences Python is fully object oriented. My point is, do you know any book or resource that explains in deep the pythonic way of doing OOP?

For example, I'm gonna try to develop a modest application from ground up using TDD. If it had been done in Java for instance, I would made extensive use of interfaces to define the boundaries of my system. How would I do something like that in Python?


Many thanks!


From minscheung at googlemail.com  Wed Jun 17 15:22:08 2015
From: minscheung at googlemail.com (Sebastian M Cheung)
Date: Wed, 17 Jun 2015 12:22:08 -0700 (PDT)
Subject: anaconda upgrade
Message-ID: <a07c5f61-9179-4946-a0cb-ca8a5803aa48@googlegroups.com>

What is the best way to upgrade an existing virtual environment anaconda from 2.7.9 to 2.7.10


From skip.montanaro at gmail.com  Wed Jun 17 15:38:12 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Wed, 17 Jun 2015 14:38:12 -0500
Subject: Classic OOP in Python
In-Reply-To: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
Message-ID: <CANc-5UwsYqFh2MsBLNmX8S6SDQ37L8TaH+51VWh24__bRFJe1w@mail.gmail.com>

On Wed, Jun 17, 2015 at 2:21 PM, Jason P. <suscricions at gmail.com> wrote:
> I'm gonna try to develop a modest application from ground up using
> TDD. If it had been done in Java for instance, I would made
> extensive use of interfaces to define the boundaries of my
> system. How would I do something like that in Python?

Maybe start here:

http://stackoverflow.com/questions/2124190/how-do-i-implement-interfaces-in-python

and see if abstract base classes will be sufficient.  (I wouldn't
know. I've never used Java or PHP.)

More generally, you might try search the web for "interfaces in
python".  The above just seemed to be the most relevant response to
me.

Skip


From ned at nedbatchelder.com  Wed Jun 17 15:44:38 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 17 Jun 2015 12:44:38 -0700 (PDT)
Subject: Classic OOP in Python
In-Reply-To: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
Message-ID: <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>

On Wednesday, June 17, 2015 at 3:21:32 PM UTC-4, Jason P. wrote:
> Hello Python community.
> 
> I come from a classic background in what refers to OOP. Mostly Java and PHP (> 5.3). I'm used to abstract classes, interfaces, access modifiers and so on.
> 
> Don't get me wrong. I know that despite the differences Python is fully object oriented. My point is, do you know any book or resource that explains in deep the pythonic way of doing OOP?
> 
> For example, I'm gonna try to develop a modest application from ground up using TDD. If it had been done in Java for instance, I would made extensive use of interfaces to define the boundaries of my system. How would I do something like that in Python?
> 
> 
> Many thanks!

What other languages do with interfaces, Python does with duck-typing. You
can build something like interfaces in Python, but many people don't bother.

I don't know if your project will be web-based, but here is an entire book
about developing Python web sites with a TDD approach:

http://www.obeythetestinggoat.com/

(Don't mind the unusual domain name, it's a bit of an inside joke...)

TDD and interfaces are separate concepts, and I'm not sure they even
intersect.  TDD is about writing tests as a way to design the best system,
and putting testing at the center of your development workflow.  It works
great with Python even without interfaces.

--Ned.


From ahlusar.ahluwalia at gmail.com  Wed Jun 17 16:20:48 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Wed, 17 Jun 2015 13:20:48 -0700 (PDT)
Subject: JSON Object to CSV file
In-Reply-To: <mailman.561.1434565217.13271.python-list@python.org>
References: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
 <5747bb7f-04c0-4097-8c09-1102499b49b2@googlegroups.com>
 <mailman.561.1434565217.13271.python-list@python.org>
Message-ID: <c51635d4-174d-4ef0-8117-d80f4a8623ed@googlegroups.com>

On Wednesday, June 17, 2015 at 2:21:05 PM UTC-4, Peter Otten wrote:
> Sahlusar wrote:
> 
> > On Wednesday, June 17, 2015 at 11:00:24 AM UTC-4, Saran A wrote:
> >> I would like to have this JSON object written out to a CSV file so that
> >> the keys are header fields (for each of the columns) and the values are
> >> values that are associated with each header field. Is there a best
> >> practice for working with this? Ideally I would like to recursively
> >> iterate through the key value pairs. Thank you in advance. I am using
> >> Python 3.4 on Windows. My editor is Sublime 2.
> >> 
> >> {
> >> "CF": {
> >> "A": "5",
> >> "FEC": "1/1/0001 12:00:00 AM",
> >> "TE": null,
> >> "Locator": null,
> >> "Message": "Transfer Fee",
> >> "AT": null,
> >> "FT": null,
> >> "FR": "True",
> >> "FY": null,
> >> "FR": null,
> >> "FG": "0",
> >> "Comment": null,
> >> "FUD": null,
> >> "cID": null,
> >> "GEO": null,
> >> "ISO": null,
> >> "TRID": null,
> >> "XTY": "931083",
> >> "ANM": null,
> >> "NM": null
> >> },
> >> "CF": "Fee",
> >> "ID": "2"
> >> }
> > 
> > My apologies:
> > 
> > I originally parsed this from an XML file.
> > 
> > It really should be:
> > 
> > {
> > "Fee": {
> > "A": "5",
> > "FEC": "1/1/0001 12:00:00 AM",
> > "TE": null,
> > "Locator": null,
> > "Message": "Transfer Fee",
> > "AT": null,
> > "FT": null,
> > "FR": "True",
> > "FY": null,
> > "FR": null,
> > "FG": "0",
> > "Comment": null,
> > "FUD": null,
> > "cID": null,
> > "GEO": null,
> > "ISO": null,
> > "TRID": null,
> > "XTY": "931083",
> > "ANM": null,
> > "NM": null
> > },
> > "CF": "Fee",
> > "ID": "2"
> > }
> > 
> > The value, "Fee" associated with the key, "CF" does should not be included
> > as a column header.
> > 
> > The CSV file, when opened with an application such as MS Excel, should be
> > as follows:
> > 
> > (Column Header)----> CF               A          FEC
> > 
> > (Field Value)---->   Fee              5        1/1/0001 12:00:00 AM
> > 
> > My apologies for an confusion.
> 
> Forget about Excel, what should the CSV look like when opened in a text 
> editor?
> 
> Here's my guess:
> 
> 
> $ cat input.json
> {
> "Fee": {
> "A": "5",
> "FEC": "1/1/0001 12:00:00 AM",
> "TE": null,
> "Locator": null,
> "Message": "Transfer Fee",
> "AT": null,
> "FT": null,
> "FR": "True",
> "FY": null,
> "FR": null,
> "FG": "0",
> "Comment": null,
> "FUD": null,
> "cID": null,
> "GEO": null,
> "ISO": null,
> "TRID": null,
> "XTY": "931083",
> "ANM": null,
> "NM": null
> },
> "CF": "Fee",
> "ID": "2"
> }
> $ cat json2csv.py
> import csv
> import json
> import sys
> 
> def hook(obj):
>     return obj
> 
> def flatten(obj):
>     for k, v in obj:
>         if isinstance(v, list):
>             yield from flatten(v)
>         else:
>             yield k, v
> 
> if __name__ == "__main__":
>     with open("input.json") as f:
>         data = json.load(f, object_pairs_hook=hook)
> 
>     pairs = list(flatten(data))
> 
>     writer = csv.writer(sys.stdout)
>     writer.writerow([k for k, v in pairs])
>     writer.writerow([v for k, v in pairs])
> $ python3 json2csv.py 
> A,FEC,TE,Locator,Message,AT,FT,FR,FY,FR,FG,Comment,FUD,cID,GEO,ISO,TRID,XTY,ANM,NM,CF,ID
> 5,1/1/0001 12:00:00 AM,,,Transfer Fee,,,True,,,0,,,,,,,931083,,,Fee,2
> 
> But do you really want duplicate column names?

@Peter Otten: Thank you for your feedback. No, I do not want to have duplicates. Any thoughts on how to avoid this?


From marko at pacujo.net  Wed Jun 17 16:39:17 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 17 Jun 2015 23:39:17 +0300
Subject: Classic OOP in Python
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
Message-ID: <87eglaqdca.fsf@elektro.pacujo.net>

Ned Batchelder <ned at nedbatchelder.com>:

> TDD is about writing tests as a way to design the best system, and
> putting testing at the center of your development workflow. It works
> great with Python even without interfaces.

I wonder how great it really is. Testing is important, that's for sure,
but to make it a dogmatic starting point of development is not that
convincing.

The way it was explained to me was that in TDD you actually don't write
code to any requirements or design: you simply do the least to pass the
tests. Thus, say you need to write a program that inputs a string and
outputs the same string surrounded by parentheses (the requirement), the
starting point might be this test case:

   - run the program
   - give it the word "hello" as input
   - check that the program prints out "(hello)"

The right TDD thing would be to satisfy the test with this program:

   input()
   print("(hello)")

That *ought* to be the first version of the program until further test
cases are added that invalidate it.


Another interesting ism I have read about is the idea that the starting
point of any software project should be the user manual. The developers
should then go and build the product that fits the manual.


Marko


From breamoreboy at yahoo.co.uk  Wed Jun 17 17:04:37 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 17 Jun 2015 22:04:37 +0100
Subject: anaconda upgrade
In-Reply-To: <a07c5f61-9179-4946-a0cb-ca8a5803aa48@googlegroups.com>
References: <a07c5f61-9179-4946-a0cb-ca8a5803aa48@googlegroups.com>
Message-ID: <mlsnd5$2rs$1@ger.gmane.org>

On 17/06/2015 20:22, Sebastian M Cheung via Python-list wrote:
> What is the best way to upgrade an existing virtual environment anaconda from 2.7.9 to 2.7.10
>

Do some research, e.g. use a search engine, then try the upgrade.  If 
that doesn't work ask another question, giving the precise details as to 
what you tried, the error messages, your OS and Python version.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Wed Jun 17 17:07:43 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 17 Jun 2015 22:07:43 +0100
Subject: Classic OOP in Python
In-Reply-To: <87eglaqdca.fsf@elektro.pacujo.net>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net>
Message-ID: <mlsnj0$5ks$1@ger.gmane.org>

On 17/06/2015 21:39, Marko Rauhamaa wrote:
> Ned Batchelder <ned at nedbatchelder.com>:
>
>> TDD is about writing tests as a way to design the best system, and
>> putting testing at the center of your development workflow. It works
>> great with Python even without interfaces.
>
> I wonder how great it really is. Testing is important, that's for sure,
> but to make it a dogmatic starting point of development is not that
> convincing.
>
> The way it was explained to me was that in TDD you actually don't write
> code to any requirements or design: you simply do the least to pass the
> tests. Thus, say you need to write a program that inputs a string and
> outputs the same string surrounded by parentheses (the requirement), the
> starting point might be this test case:
>
>     - run the program
>     - give it the word "hello" as input
>     - check that the program prints out "(hello)"
>
> The right TDD thing would be to satisfy the test with this program:
>
>     input()
>     print("(hello)")
>
> That *ought* to be the first version of the program until further test
> cases are added that invalidate it.
>
>
> Another interesting ism I have read about is the idea that the starting
> point of any software project should be the user manual. The developers
> should then go and build the product that fits the manual.
>
>
> Marko
>

Awesome, one of the funniest things I've read in years.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From ned at nedbatchelder.com  Wed Jun 17 17:14:34 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 17 Jun 2015 14:14:34 -0700 (PDT)
Subject: Classic OOP in Python
In-Reply-To: <87eglaqdca.fsf@elektro.pacujo.net>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net>
Message-ID: <49f789cf-cb67-4812-8a91-218a4f049958@googlegroups.com>

On Wednesday, June 17, 2015 at 4:39:31 PM UTC-4, Marko Rauhamaa wrote:
> The way it was explained to me was that in TDD you actually don't write
> code to any requirements or design: you simply do the least to pass the
> tests. Thus, say you need to write a program that inputs a string and
> outputs the same string surrounded by parentheses (the requirement), the
> starting point might be this test case:
> 
>    - run the program
>    - give it the word "hello" as input
>    - check that the program prints out "(hello)"
> 
> The right TDD thing would be to satisfy the test with this program:
> 
>    input()
>    print("(hello)")
> 
> That *ought* to be the first version of the program until further test
> cases are added that invalidate it.

The true TDD acolytes advocate a very idiosyncratic workflow, it's true.
I don't do this, but I also don't consider myself a TDD person. I value
tests a great deal, and put a lot of effort into them, but I don't write
trivial functions to get my tests to pass and then go back to change them.

But people whose opinion I value do advocate that, and it's possible that
in time I will understand their methods and use them myself.  Stranger
things have happened...

--Ned.


From sohcahtoa82 at gmail.com  Wed Jun 17 17:16:18 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Wed, 17 Jun 2015 14:16:18 -0700 (PDT)
Subject: Classic OOP in Python
In-Reply-To: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
Message-ID: <f0c76343-7230-499c-ba30-33d7f6513503@googlegroups.com>

On Wednesday, June 17, 2015 at 12:21:32 PM UTC-7, Jason P. wrote:
> Hello Python community.
> 
> I come from a classic background in what refers to OOP. Mostly Java and PHP (> 5.3). I'm used to abstract classes, interfaces, access modifiers and so on.
> 
> Don't get me wrong. I know that despite the differences Python is fully object oriented. My point is, do you know any book or resource that explains in deep the pythonic way of doing OOP?
> 
> For example, I'm gonna try to develop a modest application from ground up using TDD. If it had been done in Java for instance, I would made extensive use of interfaces to define the boundaries of my system. How would I do something like that in Python?
> 
> 
> Many thanks!

You don't need interfaces with Python.  Duck typing makes that all possible.  This is perfectly valid:

class Dog(object):
    def move(self):
        print "Dog is moving"

class Car(object):
    def move(self):
        print "Car is moving"

things = [Dog(), Car()]
for thing in things:
    thing.move()

Despite Dog and Car being completely different classes with no relationship with each other, the code runs just fine.


From sohcahtoa82 at gmail.com  Wed Jun 17 17:33:43 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Wed, 17 Jun 2015 14:33:43 -0700 (PDT)
Subject: Classic OOP in Python
In-Reply-To: <87eglaqdca.fsf@elektro.pacujo.net>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net>
Message-ID: <6944ae86-86eb-4a9b-be05-21d0be7bb206@googlegroups.com>

On Wednesday, June 17, 2015 at 1:39:31 PM UTC-7, Marko Rauhamaa wrote:
> Ned Batchelder <ned at nedbatchelder.com>:
> 
> > TDD is about writing tests as a way to design the best system, and
> > putting testing at the center of your development workflow. It works
> > great with Python even without interfaces.
> 
> I wonder how great it really is. Testing is important, that's for sure,
> but to make it a dogmatic starting point of development is not that
> convincing.
> 
> The way it was explained to me was that in TDD you actually don't write
> code to any requirements or design: you simply do the least to pass the
> tests. Thus, say you need to write a program that inputs a string and
> outputs the same string surrounded by parentheses (the requirement), the
> starting point might be this test case:
> 
>    - run the program
>    - give it the word "hello" as input
>    - check that the program prints out "(hello)"
> 
> The right TDD thing would be to satisfy the test with this program:
> 
>    input()
>    print("(hello)")
> 
> That *ought* to be the first version of the program until further test
> cases are added that invalidate it.
> 
> 
> Another interesting ism I have read about is the idea that the starting
> point of any software project should be the user manual. The developers
> should then go and build the product that fits the manual.
> 
> 
> Marko

I had a Java class where we had to learn TDD, and that's the way TDD was taught to us, and I hated it.  We watched a video of this guy explaining TDD with a hat that was red on the front and green on the back.  It involved writing a simple failing unit test, then write code to fix it, then refactor the tests and/or code.

As an in-class exercise, we had to write an implementation of Conway's Game of Life.  I threw TDD out the window and just wrote the damn program in under 15 minutes, then another 10 minutes to write unit tests that tested every possible code branch and several invalid inputs.  Meanwhile, the people doing TDD the "right" way didn't even have a complete program after over an hour.

The brand of TTD we were taught would end up multiplying development time by at least a factor of 3, and by the time you were done, at least 75% of the tests you had written will have been removed due to rampant refactoring.

IMO, that kind of TTD is an utter waste of time.


From breamoreboy at yahoo.co.uk  Wed Jun 17 17:49:28 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 17 Jun 2015 22:49:28 +0100
Subject: Classic OOP in Python
In-Reply-To: <6944ae86-86eb-4a9b-be05-21d0be7bb206@googlegroups.com>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net>
 <6944ae86-86eb-4a9b-be05-21d0be7bb206@googlegroups.com>
Message-ID: <mlsq19$c9g$1@ger.gmane.org>

On 17/06/2015 22:33, sohcahtoa82 at gmail.com wrote:
> On Wednesday, June 17, 2015 at 1:39:31 PM UTC-7, Marko Rauhamaa wrote:
>> Ned Batchelder <ned at nedbatchelder.com>:
>>
>>> TDD is about writing tests as a way to design the best system, and
>>> putting testing at the center of your development workflow. It works
>>> great with Python even without interfaces.
>>
>> I wonder how great it really is. Testing is important, that's for sure,
>> but to make it a dogmatic starting point of development is not that
>> convincing.
>>
>> The way it was explained to me was that in TDD you actually don't write
>> code to any requirements or design: you simply do the least to pass the
>> tests. Thus, say you need to write a program that inputs a string and
>> outputs the same string surrounded by parentheses (the requirement), the
>> starting point might be this test case:
>>
>>     - run the program
>>     - give it the word "hello" as input
>>     - check that the program prints out "(hello)"
>>
>> The right TDD thing would be to satisfy the test with this program:
>>
>>     input()
>>     print("(hello)")
>>
>> That *ought* to be the first version of the program until further test
>> cases are added that invalidate it.
>>
>>
>> Another interesting ism I have read about is the idea that the starting
>> point of any software project should be the user manual. The developers
>> should then go and build the product that fits the manual.
>>
>>
>> Marko
>
> I had a Java class where we had to learn TDD, and that's the way TDD was taught to us, and I hated it.  We watched a video of this guy explaining TDD with a hat that was red on the front and green on the back.  It involved writing a simple failing unit test, then write code to fix it, then refactor the tests and/or code.
>
> As an in-class exercise, we had to write an implementation of Conway's Game of Life.  I threw TDD out the window and just wrote the damn program in under 15 minutes, then another 10 minutes to write unit tests that tested every possible code branch and several invalid inputs.  Meanwhile, the people doing TDD the "right" way didn't even have a complete program after over an hour.
>
> The brand of TTD we were taught would end up multiplying development time by at least a factor of 3, and by the time you were done, at least 75% of the tests you had written will have been removed due to rampant refactoring.
>
> IMO, that kind of TTD is an utter waste of time.
>

Couldn't agree with you more.  To once again quote my favourite part of 
the Zen of Python, "practicality beats purity".

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From ahlusar.ahluwalia at gmail.com  Wed Jun 17 17:49:35 2015
From: ahlusar.ahluwalia at gmail.com (Saran Ahluwalia)
Date: Wed, 17 Jun 2015 17:49:35 -0400
Subject: JSON Object to CSV Question
Message-ID: <CAC9q6M4w5mNj2Lb2MxND_46jbiSwk123mKGKBJckmYCo+_MoSw@mail.gmail.com>

Good Evening Everyone:

I would like to have this JSON object written out to a CSV file so that the
keys are header fields (for each of the columns) and the values are values
that are associated with each header field. Is there a best practice for
working with this? Ideally I would like to recursively iterate through the
key value pairs. Thank you in advance. I am using Python 3.4 on Windows. My
editor is Sublime 2.

Here is the JSON object:

{
"Fee": {
"A": "5",
"FEC": "1/1/0001 12:00:00 AM",
"TE": null,
"Locator": null,
"Message": "Transfer Fee",
"AT": null,
"FT": null,
"FR": "True",
"FY": null,
"FR": null,
"FG": "0",
"Comment": null,
"FUD": null,
"cID": null,
"GEO": null,
"ISO": null,
"TRID": null,
"XTY": "931083",
"ANM": null,
"NM": null
},
"CF": "Fee",
"ID": "2"
}

The value, "Fee" associated with the key, "CF" should not be included as a
column header (only as a value of the key "CF").

Other than the former, the keys should be headers and the corresponding
tuples - the field values.

In essence, my goal is to the following:

You get a dictionary object (the "outer" dictionary)
You get the data from the "CF" key (fixed name?) which is a string ("Fee" in
your example)
You use that value as a key to obtain another value from the same "outer"
dictionary, which should be a another dictionary (the "inner" dictionary)
You make a CSV file with:

   - a header that contains "CF" plus all keys in the "inner" dictionary
   that have an associated value
   - the value from key "CF" in the "outer" dictionary plus all non-null
   values in the "inner" dictionary.

I have done the following:

import csv
import json
import sys

def hook(obj):
    return obj

def flatten(obj):
    for k, v in obj:
        if isinstance(v, list):
            yield from flatten(v)
        else:
            yield k, v

if __name__ == "__main__":
    with open("data.json") as f:
        data = json.load(f, object_pairs_hook=hook)

    pairs = list(flatten(data))

    writer = csv.writer(sys.stdout)
    writer.writerow([k for k, v in pairs])
    writer.writerow([v for k, v in pairs])

The output is as follows:

$ python3 json2csv.py
A,FEC,TE,Locator,Message,AT,FT,FR,FY,FR,FG,Comment,FUD,
cID,GEO,ISO,TRID,XTY,ANM,NM,CF,ID
5,1/1/0001 12:00:00 AM,,,Transfer Fee,,,True,,,0,,,,,,,931083,,,Fee,2

I do not want to have duplicate column names.

Any advice on other best practices that I may utilize?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150617/133a4657/attachment.html>

From phbrt25 at gmail.com  Wed Jun 17 17:55:05 2015
From: phbrt25 at gmail.com (Paul Hubert)
Date: Wed, 17 Jun 2015 14:55:05 -0700 (PDT)
Subject: Posting gzip'd image file - server says Malformed Upload?
Message-ID: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>

Any idea why a server might be returning the message, in json format, "Malformed Upload"? The image is gzipped as the server requires... Someone on another forum said that he thinks Python requests automatically gzips?

I have the code working in vb.net... no idea why it wont work in Python.

This is my code.

def uploadPhoto(url, sid, dafile):

headers = {'User-Agent':useragent, 
                     'Session-Id':sessionid, 
                     'Content-Type':'image/jpeg', 
                     'Accept-Encoding':'gzip, deflate'}

f_in = open(dafile, 'rb')
f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
f_out.writelines(f_in)
f_out.close()
f_in.close()

files = {('Image', open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'rb'))}
r = requests.post(url, headers=headers, proxies=proxies, files=files)    

print r.status_code
print r.request.headers
return str(r.text)



The server replies with:

{"message":"Malformed Upload","code":1012}


From lac at openend.se  Wed Jun 17 18:09:40 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 18 Jun 2015 00:09:40 +0200
Subject: Classic OOP in Python
In-Reply-To: Message from Marko Rauhamaa <marko@pacujo.net>
 of "Wed, 17 Jun 2015 23:39:17 +0300." <87eglaqdca.fsf@elektro.pacujo.net>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com><87eglaqdca.fsf@elektro.pacujo.net>
Message-ID: <201506172209.t5HM9eBH001218@fido.openend.se>

In a message of Wed, 17 Jun 2015 23:39:17 +0300, Marko Rauhamaa writes:
>Ned Batchelder <ned at nedbatchelder.com>:
>
>> TDD is about writing tests as a way to design the best system, and
>> putting testing at the center of your development workflow. It works
>> great with Python even without interfaces.
>
>I wonder how great it really is. Testing is important, that's for sure,
>but to make it a dogmatic starting point of development is not that
>convincing.

Try it and see.

>The way it was explained to me was that in TDD you actually don't write
>code to any requirements or design: you simply do the least to pass the
>tests. Thus, say you need to write a program that inputs a string and
>outputs the same string surrounded by parentheses (the requirement), the
>starting point might be this test case:
>
>   - run the program
>   - give it the word "hello" as input
>   - check that the program prints out "(hello)"
>
>The right TDD thing would be to satisfy the test with this program:
>
>   input()
>   print("(hello)")
>
>That *ought* to be the first version of the program until further test
>cases are added that invalidate it.

You have been talking to people who are beginners at TDD.  They have
been taught some hard and fast rules to keep them from the problem
of over-designing.  Or you may have been taking to fools.

Both beginners and fools badly want to replace the wisdom that they
don't have in their heads with some rules from somebody else's head.
Beginners can work this way, and develop wisdom over time.  Fools,
pretty much by definition cannot.

The way I usually work is to take a list of requirements from some
user(s) and turn them into test cases.  And then the job of design
is to get from where I am 'have no code' to I can begin satisfying
some of these requirements.

Just because 
>   input()
>   print("(hello)")

would satisfy the test you wrote doesn't mean it is a correct TTD thing
to do -- because it doesn't get you any closer to solving the real
problem your design is supposed to do.

This doesn't mean that you need to write your tests in a more
complicated way to prevent smart-asses from doing stuff like the
above.  It means that your tests should make you get closer to your
goal.  You will have to make a bridge of tests to get from 'I
have nothing' to the first user supplied test you need to meet,
but you do have a destination, and you need to go there.

I teach a bunch of 11 year olds to program in python using TTD and
pytest.  A good number of them end up writing tests to test that
the python standard library works the way they expect it to.  In some
sense, this is a waste of their time, but I resist the urge to
tell them so.  They are developing wisdom and understanding of the
standard library, and confidence.  When they are wiser, they can
delete some of these superfluous tests, but as for now, these tests
are a measure of their growing understanding.  If you have
oversimplified what you have been told, then it may be that you have
missed this 'and I learned something here' bit they were telling you
about, even if you think that they ought to have known it all along.

Teaching design is hard.

Teaching design to smart-asses who don't want to learn design at all
but want some ruleset to be 'better than design' is impossible.

But teaching design using TTD as a tool for design.  Well, that is
_wonderful_.


>Another interesting ism I have read about is the idea that the starting
>point of any software project should be the user manual. The developers
>should then go and build the product that fits the manual.

Unless the manual is bloody short, in which case it is a series of
acceptance tests -- this is a rotten idea.  The marketing department
makes the user manual, the engineers implement it, and the users get
screwed as they get, not what they want, but what somebody in
marketing, or (only slightly better) engineering thinks that they
wanted, or ought to want.

If at all possible, you want the starting point of your software
development to be a conversation between the people who are going to
write the thing and the people who are going to use the thing, with
particular attention to what the users are using now, and why they
want something else instead of just more of the same.

>
>
>Marko

ps -- Marko, we have ample evidence that you are an extremely clever
person.  But the purpose of TTD is not to make clever code, but
wise code.  TTD in the hands of a fool will never produce that.
But how else do you have to check that your design, while clever
isn't _too damn clever_?  Aren't you plagued with that problem?  I am
and TDD is the best thing I ever learned to keep that tendancy I have
to write too clever code in check.

Laura again




From lac at openend.se  Wed Jun 17 18:22:35 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 18 Jun 2015 00:22:35 +0200
Subject: Classic OOP in Python
In-Reply-To: Message from sohcahtoa82@gmail.com of "Wed,
 17 Jun 2015 14:33:43 -0700."
 <6944ae86-86eb-4a9b-be05-21d0be7bb206@googlegroups.com>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net><6944ae86-86eb-4a9b-be05-21d0be7bb206@googlegroups.com>
Message-ID: <201506172222.t5HMMZ24001918@fido.openend.se>

In a message of Wed, 17 Jun 2015 14:33:43 -0700, sohcahtoa82 at gmail.com writes:

>I had a Java class where we had to learn TDD, and that's the way TDD
>was taught to us, and I hated it.  We watched a video of this guy
>explaining TDD with a hat that was red on the front and green on the
>back.  It involved writing a simple failing unit test, then write
>code to fix it, then refactor the tests and/or code.  As an in-class
>exercise, we had to write an implementation of Conway's Game of Life.
>I threw TDD out the window and just wrote the damn program in under
>15 minutes, then another 10 minutes to write unit tests that tested
>every possible code branch and several invalid inputs.  Meanwhile,
>the people doing TDD the "right" way didn't even have a complete
>program after over an hour.  The brand of TTD we were taught would
>end up multiplying development time by at least a factor of 3, and by
>the time you were done, at least 75% of the tests you had written
>will have been removed due to rampant refactoring.  IMO, that kind of
>TTD is an utter waste of time.

I'd hate that too.  But that's not the TDD I know, or teach.  Please
don't write off TDD as a result of this experience.

Laura


From breamoreboy at yahoo.co.uk  Wed Jun 17 18:27:48 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 17 Jun 2015 23:27:48 +0100
Subject: Classic OOP in Python
In-Reply-To: <201506172209.t5HM9eBH001218@fido.openend.se>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com><87eglaqdca.fsf@elektro.pacujo.net>
 <201506172209.t5HM9eBH001218@fido.openend.se>
Message-ID: <mlss94$gdd$1@ger.gmane.org>

On 17/06/2015 23:09, Laura Creighton wrote:
>
> ps -- Marko, we have ample evidence that you are an extremely clever
> person.  But the purpose of TTD is not to make clever code, but
> wise code.  TTD in the hands of a fool will never produce that.
> But how else do you have to check that your design, while clever
> isn't _too damn clever_?  Aren't you plagued with that problem?  I am
> and TDD is the best thing I ever learned to keep that tendancy I have
> to write too clever code in check.
>

Strange, I see no such evidence, just a person who lives firmly in cloud 
cuckoo land.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lac at openend.se  Wed Jun 17 18:33:11 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 18 Jun 2015 00:33:11 +0200
Subject: Classic OOP in Python
In-Reply-To: Message from Ned Batchelder <ned@nedbatchelder.com> of "Wed,
 17 Jun 2015 14:14:34 -0700."
 <49f789cf-cb67-4812-8a91-218a4f049958@googlegroups.com>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net><49f789cf-cb67-4812-8a91-218a4f049958@googlegroups.com>
Message-ID: <201506172233.t5HMXBIm002277@fido.openend.se>

In a message of Wed, 17 Jun 2015 14:14:34 -0700, Ned Batchelder writes:

>The true TDD acolytes advocate a very idiosyncratic workflow, it's true.
>I don't do this, but I also don't consider myself a TDD person. I value
>tests a great deal, and put a lot of effort into them, but I don't write
>trivial functions to get my tests to pass and then go back to change them.

Grrr.  I am a true TDD acolyte.  And I don't do that either.

>But people whose opinion I value do advocate that, and it's possible that
>in time I will understand their methods and use them myself.  Stranger
>things have happened...
>
>--Ned.

Seriously?  Oh you humble man.

I have convinced a huge number of people who used to have that sort of
dogma that the reason they advocated such baby steps is that they
didn't have a good coverage tool. ***grin***

TDD is supposed to make you brave, not cowards, and it's
Ned's most excellent tool
http://nedbatchelder.com/code/coverage/
that I recommend to TDD dogmatic cowards.

Even if you don't want to use TTD, you will enjoy Ned's tool.  It's GREAT.
Thank you, Ned.

Laura



From thebalancepro at gmail.com  Wed Jun 17 18:55:35 2015
From: thebalancepro at gmail.com (Nick Mellor)
Date: Wed, 17 Jun 2015 15:55:35 -0700 (PDT)
Subject: Generating list of unique search sub-phrases
In-Reply-To: <d7b239fc-c814-4935-8172-5c6a6c782d9c@googlegroups.com>
References: <d7b239fc-c814-4935-8172-5c6a6c782d9c@googlegroups.com>
Message-ID: <b1b6e40f-85ca-4e0b-8da3-4ef954ebddaa@googlegroups.com>

On Saturday, 30 May 2015 06:39:44 UTC+10, Nick Mellor  wrote:
> Hi all,
> 
> My own solution works but I'm sure it could be simpler or read better. How would you do it?
> 
> Say you've got a list of companies:
> 
> Aerosonde Ltd
> Amcor
> ANCA
> Austal Ships
> Australia Post
> Australian Air Express
> Australian Defence Industries
> Australian Railroad Group
> Australian Submarine Corporation
> 
> and you need to extract phrases from the company names that uniquely identify that company. The results for the above list of companies should be:
> 
> Company: 'Aerosonde Ltd'
>  Aliases: Aerosonde,Ltd,Aerosonde Ltd
> 
> Company: 'Amcor'
>  Aliases: Amcor
> 
> Company: 'ANCA'
>  Aliases: ANCA
> 
> Company: 'Austal Ships'
>  Aliases: Austal,Ships,Austal Ships
> 
> Company: 'Australia Post'
>  Aliases: Post,Australia Post
> 
> Company: 'Australian Air Express'
>  Aliases: Air,Express,Australian Air,Air Express,Australian Air Express
> 
> Company: 'Australian Defence Industries'
>  Aliases: Defence,Industries,Australian Defence,Defence Industries,Australian Defence Industries
> 
> Company: 'Australian Railroad Group'
>  Aliases: Railroad,Group,Australian Railroad,Railroad Group,Australian Railroad Group
> 
> Company: 'Australian Submarine Corporation'
>  Aliases: Submarine,Corporation,Australian Submarine,Submarine Corporation,Australian Submarine Corporation
> 
> Here's my solution:
> 
> from itertools import combinations, chain
> 
> companies = [
>     "Aerosonde Ltd",
>     "Amcor",
>     "ANCA",
>     "Austal Ships",
>     "Australia Post",
>     "Australian Air Express",
>     "Australian Defence Industries",
>     "Australian Railroad Group",
>     "Australian Submarine Corporation",
> ]
> 
> def flatten(i):
>     return list(chain.from_iterable(i))
> 
> companies_as_text_stream = ' '.join(companies)
> for company in companies:
>         word_combinations = [list(combinations(company.split(), r)) for r in range(1, len(company))]
>         phrases = [' '.join(phrase) for phrase in flatten(word_combinations)]
>         unique_phrases = [phrase for phrase in phrases if companies_as_text_stream.count(phrase) == 1]
>         aliases = ','.join(unique_phrases)
>         print("Company: '{0}'\n Aliases: {1}\n".format(company, aliases))

Great reply, Peter, thank you. Lots to think about.

Cheers,

Nick


From ned at nedbatchelder.com  Wed Jun 17 18:55:42 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 17 Jun 2015 15:55:42 -0700 (PDT)
Subject: Classic OOP in Python
In-Reply-To: <mailman.569.1434580406.13271.python-list@python.org>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net><49f789cf-cb67-4812-8a91-218a4f049958@googlegroups.com>
 <mailman.569.1434580406.13271.python-list@python.org>
Message-ID: <4af2c707-3e9a-46db-9dcb-39bc1c9fe087@googlegroups.com>

On Wednesday, June 17, 2015 at 6:34:23 PM UTC-4, Laura Creighton wrote:
> TDD is supposed to make you brave, not cowards, and it's
> Ned's most excellent tool
> http://nedbatchelder.com/code/coverage/
> that I recommend to TDD dogmatic cowards.
> 
> Even if you don't want to use TTD, you will enjoy Ned's tool.  It's GREAT.
> Thank you, Ned.
> 
> Laura

So much for humility I guess.  *blush*.  Thanks!

--Ned.


From steve at pearwood.info  Wed Jun 17 18:58:38 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 18 Jun 2015 08:58:38 +1000
Subject: Classic OOP in Python
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net>
Message-ID: <5581fb9d$0$1672$c3e8da3$5496439d@news.astraweb.com>

On Thu, 18 Jun 2015 06:39 am, Marko Rauhamaa wrote:

> Ned Batchelder <ned at nedbatchelder.com>:
> 
>> TDD is about writing tests as a way to design the best system, and
>> putting testing at the center of your development workflow. It works
>> great with Python even without interfaces.
> 
> I wonder how great it really is. Testing is important, that's for sure,
> but to make it a dogmatic starting point of development is not that
> convincing.
> 
> The way it was explained to me was that in TDD you actually don't write
> code to any requirements or design: you simply do the least to pass the
> tests.

You still need requirements and a design. Otherwise how do you know what
tests you need to write?


> Thus, say you need to write a program that inputs a string and 
> outputs the same string surrounded by parentheses (the requirement), the
> starting point might be this test case:
> 
>    - run the program
>    - give it the word "hello" as input
>    - check that the program prints out "(hello)"
>
> The right TDD thing would be to satisfy the test with this program:
> 
>    input()
>    print("(hello)")
> 
> That *ought* to be the first version of the program until further test
> cases are added that invalidate it.

That's certainly the more pedantic approach of TDD proponents. But frankly I
expect that outside of TDD tutorials, nobody *really* starts off that small
in practice. I know I don't.

I would start with this function, and a separate test suite:


# === application.py ===
def bracket(s):
    """Return string s bracketed in parentheses.

    >>> bracket("Hello")
    '(Hello)'

    """
    return "(%s)" % s


# === test_application.py ===
import application

import doctest
import unittest

def load_tests(loader, tests, ignore):
    tests.addTests(doctest.DocTestSuite())
    tests.addTests(doctest.DocTestSuite(application))
    return tests


if __name__ == '__main__':
    unittest.main()


Then call 

python test_application.py


to run the tests. Or even skip the test suite and just call 

python -m doctest application.py


And from there, I would build up extra doc tests and unit tests as needed.

The point is, I don't believe that there is any *necessity* to write
deliberately failing tests for *trivial* functions that are only one or two
lines. You can practically see that the function is correct without even a
test. Deliberately making it fail has only one advantage:

* Getting you used to failing tests and learning that they aren't in and of
themselves a bad thing or something to be ashamed of;

* Instilling the discipline to actually write tests.

Er, that's two advantages.

But really, in practice, how many trivial one liner functions do you end up
writing, as a percentage of your entire application? Unless the app is
pretty simple, chances are that percentage is very small.



-- 
Steven



From tjreedy at udel.edu  Wed Jun 17 19:46:23 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 17 Jun 2015 19:46:23 -0400
Subject: Classic OOP in Python
In-Reply-To: <87eglaqdca.fsf@elektro.pacujo.net>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net>
Message-ID: <mlt0tr$mqh$1@ger.gmane.org>

On 6/17/2015 4:39 PM, Marko Rauhamaa wrote:
> Ned Batchelder <ned at nedbatchelder.com>:
>
>> TDD is about writing tests as a way to design the best system, and
>> putting testing at the center of your development workflow. It works
>> great with Python even without interfaces.

I use what I might call 'smart TDD'.  I think it a mistake to dismiss 
TDD because 'stupid TDD' is possible, and maybe even extant.

The reason for having a test initially fail is to make sure that the 
test is called.  A unittest example:

class NoTest(unittest.TestCase):
     def text_ohmygod(self):
         assertEqual(True, False)

Why does this 'pass'?  Within the last year, it was discovered that a 
TestCase subclass in one of the Python test files was not being run.  I 
made a similar mistake in my own code.  The more I do TDD 
(intelligently), the more I like it.

> I wonder how great it really is. Testing is important, that's for sure,
> but to make it a dogmatic starting point of development is not that
> convincing.

The bugs in Python, whether already fixed, on the tracker, or yet to be 
recorded, are the result of CPython not originally using TDD of some 
version.  In my opinion, CPython maintenance and enhancement has 
improved since some semblance of TDD was instituted.  A runable test is 
now the first stage for tracker issues.

> The way it was explained to me was that in TDD you actually don't write
> code to any requirements or design:

Writing sane code to pass tests based on requirements or a design is 
intended to ensure that one is actually meeting the design.

> you simply do the least to pass the tests.

The incantation I have read is 'simplest', not 'least'.  I personally 
interpret 'simplest' to include some presumption of sanity in relation 
to the ultimate goal.

 > Thus, say you need to write a program that inputs a string and
> outputs the same string surrounded by parentheses (the requirement),

One should write a reusable and easily testable *function* that takes a 
string as input (and outputs the same string surrounded by parens). 
Call the function paren_around.

 > the starting point might be this test case:
>
>     - run the program
>     - give it the word "hello" as input
>     - check that the program prints out "(hello)"

Doubly bad.

1. I consider mixing program i/o with function not related to i/o to be 
a bad idea.  TDD should cure one of this madness, not reinforce it.

2. The first test for smart TDD should best be with a falsey input (or 
other corner or edge case).  With this guideline, there is a good chance 
the 'simplest code' will be augmented rather than replaced with 
additional tests. A smart TDD test might be

     assertEqual(paren_arount(''), '()')

> The right TDD thing would be to satisfy the test with this program:
>
>     input()
>     print("(hello)")

This does not work.  The above adds '\n' to the end of the string, which 
is not part of the specification.  A human may not notice, but an 
automated test that captures stdout and compares with "(hello)" will 
notice and fail.

> That *ought* to be the first version of the program until further test
> cases are added that invalidate it.

As a statistician, I know about the mistake of overfitting functions to 
data.  'Teaching to the test' is a related mistake  Smart TDD should not 
reinforce bad practice.  This simple code passes the initial testcase above.

def paren_around(s):
     "Return string s surrounded by '(' and ')'."
     return '()'

Add a more generic (non-null) testcase.

     assertEqual(paren_around('abc', '(abc)')

The irrelevant details should usually be ignored when augmenting the 
code*.  The simplicity of this example makes it easy to write a correct 
assert, but is not relevant to the intent of the function.

def paren_around(s):
     "Return string s surrounded by '(' and ')'."
     return '({})'.format(s)

Code added, with nothing deleted.

> Another interesting ism I have read about is the idea that the starting
> point of any software project should be the user manual. The developers
> should then go and build the product that fits the manual.

The corresponding TDD approach is to write tests from the manual, and 
then write code.  One can also write tests from docs for existing code 
- and then see if the code passes the doc-based tests.  This is one way 
people discover bugs.

-- 
Terry Jan Reedy



From ben+python at benfinney.id.au  Wed Jun 17 20:04:46 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 18 Jun 2015 10:04:46 +1000
Subject: Documenting a function signature (was: Set a flag on the function or
 a global?)
References: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
 <mailman.530.1434500492.13271.python-list@python.org>
 <55818b3c$0$1665$c3e8da3$5496439d@news.astraweb.com>
 <mailman.556.1434553624.13271.python-list@python.org>
 <55819878$0$1658$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <85egl9ubj5.fsf_-_@benfinney.id.au>

Steven D'Aprano <steve at pearwood.info> writes:

> The full signature is:
>
> edir([object [, glob=''] [, dunders=True] [, meta=False]])
>
> All four arguments are optional, and dunders and meta are
> keyword-only.

The official documentation seems to prefer this style::

    edit(object, glob='', *, dunders=True, meta=False)

(I think that's the style, anyway.)

I like that it gets rid of square brackets; with the notation showing a
default value, that already communicates that a parameter is optional.

Since the introduction of keyword-only arguments in Python functions,
the question arises of how to communicate this in documentation.

The lone asterisk showing the separation of keyword-only arguments from
the rest is confusing to me. Not least because it is (if I understand
correctly) invalid syntax to actually have that in Python code.

What are your thoughts, dear reader, on the documentation style for
showing a Python function signature, now that we have not only default
arguments but also keyword-only arguments?

-- 
 \      ?It is difficult to get a man to understand something when his |
  `\   salary depends upon his not understanding it.? ?Upton Sinclair, |
_o__)                                                             1935 |
Ben Finney



From rosuav at gmail.com  Wed Jun 17 20:07:21 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 18 Jun 2015 10:07:21 +1000
Subject: Classic OOP in Python
In-Reply-To: <87eglaqdca.fsf@elektro.pacujo.net>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmp3Gs_YUC+OCMfd=ir=5rpzfgHrMJfThz2ESaeBnAMLAQ@mail.gmail.com>

On Thu, Jun 18, 2015 at 6:39 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Another interesting ism I have read about is the idea that the starting
> point of any software project should be the user manual. The developers
> should then go and build the product that fits the manual.

I've seldom met a *user* manual that's truly the right way to start
building, but there have been times when I've built *API*
documentation prior to the code. That can work fairly well. I'll also
often start a project with a copious set of notes that aren't quite
user-facing, aren't quite programmer-friendly, but are somewhere in
between. Here's a new project I'm starting now:

https://github.com/Rosuav/ThirdSquare

Prior to actually creating that repo, I'd done some thrash testing of
the basic concepts (which is how I know that the basic idea will work
- my thrash test achieved 100tps, massively exceeding the 40tps that I
need, ergo it's worth proceeding to code), but the project itself
started with the README, then the .sql files giving a basic run-down
of the tables required. Now, and only now, I'm starting to look at
actual code. Is that starting with the user manual? Not quite, but I
think it captures the same concept that that's trying to capture.

ChrisA


From ian.g.kelly at gmail.com  Wed Jun 17 20:10:45 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 17 Jun 2015 18:10:45 -0600
Subject: Documenting a function signature (was: Set a flag on the function
 or a global?)
In-Reply-To: <85egl9ubj5.fsf_-_@benfinney.id.au>
References: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
 <mailman.530.1434500492.13271.python-list@python.org>
 <55818b3c$0$1665$c3e8da3$5496439d@news.astraweb.com>
 <mailman.556.1434553624.13271.python-list@python.org>
 <55819878$0$1658$c3e8da3$5496439d@news.astraweb.com>
 <85egl9ubj5.fsf_-_@benfinney.id.au>
Message-ID: <CALwzid=amQL9io3xgdxOV5R1Xeeu5Mj5vBRYGFL=UYX4vy-O0Q@mail.gmail.com>

On Wed, Jun 17, 2015 at 6:04 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Steven D'Aprano <steve at pearwood.info> writes:
>
>> The full signature is:
>>
>> edir([object [, glob=''] [, dunders=True] [, meta=False]])
>>
>> All four arguments are optional, and dunders and meta are
>> keyword-only.
>
> The official documentation seems to prefer this style::
>
>     edit(object, glob='', *, dunders=True, meta=False)
>
> (I think that's the style, anyway.)
>
> I like that it gets rid of square brackets; with the notation showing a
> default value, that already communicates that a parameter is optional.
>
> Since the introduction of keyword-only arguments in Python functions,
> the question arises of how to communicate this in documentation.
>
> The lone asterisk showing the separation of keyword-only arguments from
> the rest is confusing to me. Not least because it is (if I understand
> correctly) invalid syntax to actually have that in Python code.

No, this is valid syntax in Python 3. It means the same thing to the
compiler -- the remaining arguments are keyword-only -- that it means
in the documentation.


From rosuav at gmail.com  Wed Jun 17 20:14:54 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 18 Jun 2015 10:14:54 +1000
Subject: Documenting a function signature (was: Set a flag on the function
 or a global?)
In-Reply-To: <85egl9ubj5.fsf_-_@benfinney.id.au>
References: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
 <mailman.530.1434500492.13271.python-list@python.org>
 <55818b3c$0$1665$c3e8da3$5496439d@news.astraweb.com>
 <mailman.556.1434553624.13271.python-list@python.org>
 <55819878$0$1658$c3e8da3$5496439d@news.astraweb.com>
 <85egl9ubj5.fsf_-_@benfinney.id.au>
Message-ID: <CAPTjJmpPh=onWt=WE1pf2eym9jSnNRxxGx-3Aq_22ye+VQ9A7g@mail.gmail.com>

On Thu, Jun 18, 2015 at 10:04 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Steven D'Aprano <steve at pearwood.info> writes:
>
>> The full signature is:
>>
>> edir([object [, glob=''] [, dunders=True] [, meta=False]])
>>
>> All four arguments are optional, and dunders and meta are
>> keyword-only.
>
> The official documentation seems to prefer this style::
>
>     edit(object, glob='', *, dunders=True, meta=False)

Mostly. That would imply that object is a mandatory parameter, which
AIUI isn't the case for Steven's edir. The downside of this kind of
signature is that it's hard to show the parameters that have unusual
defaults (either sentinel objects and custom code to cope, or they're
pulled out of *a or **kw).

> The lone asterisk showing the separation of keyword-only arguments from
> the rest is confusing to me. Not least because it is (if I understand
> correctly) invalid syntax to actually have that in Python code.

It's perfectly valid, and it does exactly what it looks like. What's
not valid syntax is the slash that comes up in some Argument Clinic
functions:

>>> help(list.__init__)
__init__(self, /, *args, **kwargs)
    Initialize self.  See help(type(self)) for accurate signature.

> What are your thoughts, dear reader, on the documentation style for
> showing a Python function signature, now that we have not only default
> arguments but also keyword-only arguments?

I like the executable form when it's fairly simple. If you have a
boolean that defaults to True but you can set it to false, saying
"dunders=True" is awesomely clear. It's a bit harder when you want to
do something more complicated, but so far as it's possible, I'd like
to see signatures that look like they were copied and pasted straight
from the function definition.

ChrisA


From rosuav at gmail.com  Wed Jun 17 20:23:26 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 18 Jun 2015 10:23:26 +1000
Subject: Posting gzip'd image file - server says Malformed Upload?
In-Reply-To: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>
References: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>
Message-ID: <CAPTjJmoSYqRyWNj4pQ14e16qRvx+B+xo3qvHfZATxkg=iEC7KA@mail.gmail.com>

On Thu, Jun 18, 2015 at 7:55 AM, Paul Hubert <phbrt25 at gmail.com> wrote:
> f_in = open(dafile, 'rb')
> f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
> f_out.writelines(f_in)
> f_out.close()
> f_in.close()

Are you sure you want iteration and writelines() here? I would be
inclined to avoid those for any situation that isn't plain text. If
the file isn't too big, I'd just read it all in a single blob and then
write it all out at once.

ChrisA


From phbrt25 at gmail.com  Wed Jun 17 20:45:12 2015
From: phbrt25 at gmail.com (Paul Hubert)
Date: Wed, 17 Jun 2015 17:45:12 -0700 (PDT)
Subject: Posting gzip'd image file - server says Malformed Upload?
In-Reply-To: <mailman.575.1434587009.13271.python-list@python.org>
References: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>
 <mailman.575.1434587009.13271.python-list@python.org>
Message-ID: <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>

On Wednesday, June 17, 2015 at 8:24:17 PM UTC-4, Chris Angelico wrote:

> Are you sure you want iteration and writelines() here? I would be
> inclined to avoid those for any situation that isn't plain text. If
> the file isn't too big, I'd just read it all in a single blob and then
> write it all out at once.
> 
> ChrisA

Do you think that would fix my issue? Could you give me an example? 


From torriem at gmail.com  Wed Jun 17 21:10:18 2015
From: torriem at gmail.com (Michael Torrie)
Date: Wed, 17 Jun 2015 19:10:18 -0600
Subject: Posting gzip'd image file - server says Malformed Upload?
In-Reply-To: <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
References: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>
 <mailman.575.1434587009.13271.python-list@python.org>
 <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
Message-ID: <55821A7A.8040605@gmail.com>

On 06/17/2015 06:45 PM, Paul Hubert wrote:
> On Wednesday, June 17, 2015 at 8:24:17 PM UTC-4, Chris Angelico wrote:
> 
>> Are you sure you want iteration and writelines() here? I would be
>> inclined to avoid those for any situation that isn't plain text. If
>> the file isn't too big, I'd just read it all in a single blob and then
>> write it all out at once.
>>
>> ChrisA
> 
> Do you think that would fix my issue? 

I recommend you try it and see.  That's the only way to know.  We don't
know any details of your program or situation really, so we can only guess.

> Could you give me an example?

Instead of writelines(), just write() the data.  There are examples in
the official python docs I'm sure.


From rosuav at gmail.com  Wed Jun 17 21:46:03 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 18 Jun 2015 11:46:03 +1000
Subject: Posting gzip'd image file - server says Malformed Upload?
In-Reply-To: <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
References: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>
 <mailman.575.1434587009.13271.python-list@python.org>
 <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
Message-ID: <CAPTjJmopsExeDKejoMAk183BQeXs-wrVEvxo8A1zKRbun=RpGw@mail.gmail.com>

On Thu, Jun 18, 2015 at 10:45 AM, Paul Hubert <phbrt25 at gmail.com> wrote:
> On Wednesday, June 17, 2015 at 8:24:17 PM UTC-4, Chris Angelico wrote:
>
>> Are you sure you want iteration and writelines() here? I would be
>> inclined to avoid those for any situation that isn't plain text. If
>> the file isn't too big, I'd just read it all in a single blob and then
>> write it all out at once.
>>
>> ChrisA
>
> Do you think that would fix my issue? Could you give me an example?

Sorry for the abrupt and terse previous email; I had a student arrive
just as I was posting that, and hit Ctrl-Enter when I should really
have just left the email as a draft. Here's what I'm thinking:

# Was:
f_in = open(dafile, 'rb')
f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
f_out.writelines(f_in)
f_out.close()
f_in.close()

# Now:
gz = '/Users/Paul/Desktop/scripts/pic.jpg.gz'
with open(dafile, 'rb') as f_in, gzip.open(gz, 'wb') as f_out:
    f_out.write(f_in.read())

You might actually be able to write to a StringIO rather than to a
file, given that you appear to be just reading the data back again
straight away. But in case you want to keep the file around for some
other reason, this still works the exact same way you had it.

The main difference is that this version swallows the entire file in a
single gulp, then passes it all to the gzip writer. If your file is
tiny (under 1MB), this is perfect. If it's huge (over 1GB), you may
have problems. In between, it'll probably work, but might be
inefficient.

Hope that helps!

ChrisA


From phbrt25 at gmail.com  Wed Jun 17 23:48:16 2015
From: phbrt25 at gmail.com (Paul Hubert)
Date: Wed, 17 Jun 2015 20:48:16 -0700 (PDT)
Subject: Posting gzip'd image file - server says Malformed Upload?
In-Reply-To: <mailman.577.1434591971.13271.python-list@python.org>
References: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>
 <mailman.575.1434587009.13271.python-list@python.org>
 <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
 <mailman.577.1434591971.13271.python-list@python.org>
Message-ID: <db74b4ee-b88f-4e41-8844-2c80671a6bee@googlegroups.com>

On Wednesday, June 17, 2015 at 9:46:25 PM UTC-4, Chris Angelico wrote:
> On Thu, Jun 18, 2015 at 10:45 AM, Paul Hubert <phbrt25 at gmail.com> wrote:
> > On Wednesday, June 17, 2015 at 8:24:17 PM UTC-4, Chris Angelico wrote:
> >
> >> Are you sure you want iteration and writelines() here? I would be
> >> inclined to avoid those for any situation that isn't plain text. If
> >> the file isn't too big, I'd just read it all in a single blob and then
> >> write it all out at once.
> >>
> >> ChrisA
> >
> > Do you think that would fix my issue? Could you give me an example?
> 
> Sorry for the abrupt and terse previous email; I had a student arrive
> just as I was posting that, and hit Ctrl-Enter when I should really
> have just left the email as a draft. Here's what I'm thinking:
> 
> # Was:
> f_in = open(dafile, 'rb')
> f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
> f_out.writelines(f_in)
> f_out.close()
> f_in.close()
> 
> # Now:
> gz = '/Users/Paul/Desktop/scripts/pic.jpg.gz'
> with open(dafile, 'rb') as f_in, gzip.open(gz, 'wb') as f_out:
>     f_out.write(f_in.read())
> 
> You might actually be able to write to a StringIO rather than to a
> file, given that you appear to be just reading the data back again
> straight away. But in case you want to keep the file around for some
> other reason, this still works the exact same way you had it.
> 
> The main difference is that this version swallows the entire file in a
> single gulp, then passes it all to the gzip writer. If your file is
> tiny (under 1MB), this is perfect. If it's huge (over 1GB), you may
> have problems. In between, it'll probably work, but might be
> inefficient.
> 
> Hope that helps!
> 
> ChrisA

Same result - server says malformed upload. :/


From rosuav at gmail.com  Wed Jun 17 23:56:08 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 18 Jun 2015 13:56:08 +1000
Subject: Posting gzip'd image file - server says Malformed Upload?
In-Reply-To: <db74b4ee-b88f-4e41-8844-2c80671a6bee@googlegroups.com>
References: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>
 <mailman.575.1434587009.13271.python-list@python.org>
 <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
 <mailman.577.1434591971.13271.python-list@python.org>
 <db74b4ee-b88f-4e41-8844-2c80671a6bee@googlegroups.com>
Message-ID: <CAPTjJmrWOAZHX8q6dbiw21kffT1d1VyXYK-kTKHYJzxDuW3U9w@mail.gmail.com>

On Thu, Jun 18, 2015 at 1:48 PM, Paul Hubert <phbrt25 at gmail.com> wrote:
>> # Now:
>> gz = '/Users/Paul/Desktop/scripts/pic.jpg.gz'
>> with open(dafile, 'rb') as f_in, gzip.open(gz, 'wb') as f_out:
>>     f_out.write(f_in.read())
>>
>
> Same result - server says malformed upload. :/

Oh well, was worth a shot. Since the file's sitting around still, you
should be able to attempt to decompress it with the stand-alone gzip
command; does that give the right result? Also, you should be able to
manually send that .gz file up to the server, which would be worth
confirming.

Are you certain that the content should be gzipped as a file? It's not
meant to be done as a transfer-level encoding?

ChrisA


From torriem at gmail.com  Wed Jun 17 23:59:35 2015
From: torriem at gmail.com (Michael Torrie)
Date: Wed, 17 Jun 2015 21:59:35 -0600
Subject: Posting gzip'd image file - server says Malformed Upload?
In-Reply-To: <db74b4ee-b88f-4e41-8844-2c80671a6bee@googlegroups.com>
References: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>
 <mailman.575.1434587009.13271.python-list@python.org>
 <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
 <mailman.577.1434591971.13271.python-list@python.org>
 <db74b4ee-b88f-4e41-8844-2c80671a6bee@googlegroups.com>
Message-ID: <55824227.70108@gmail.com>

On 06/17/2015 09:48 PM, Paul Hubert wrote:
> Same result - server says malformed upload. :/

You may want to run a sniffer like wireshark and see what the difference
is between the packets coming from your C# program and coming from Python.




From admin.dslcomputer at gmail.com  Thu Jun 18 02:52:33 2015
From: admin.dslcomputer at gmail.com (admin.dslcomputer at gmail.com)
Date: Thu, 18 Jun 2015 06:52:33 +0000
Subject: =?utf-8?Q?Hotspot_Locater?=
Message-ID: <55826af6.e281440a.068d.35da@mx.google.com>


Hi everyone:




There is interest in our group in the development of Python code to locate WiFi hotspots.




My question: Is there a Python solution in the location of WiFi hotspots?




Hal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150618/1456e3dc/attachment.html>

From Cecil at decebal.nl  Thu Jun 18 03:05:20 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 18 Jun 2015 09:05:20 +0200
Subject: Get off the list
References: <mailman.582.1434610744.13271.python-list@python.org>
Message-ID: <87d20tijin.fsf@Equus.decebal.nl>

On Wednesday 17 Jun 2015 09:09 CEST, Deogratius Musiige wrote:

> How can i get off this mailing list? 

Looking at the headers:
    List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
     <mailto:python-list-request at python.org?subject=unsubscribe>

So I would send an email with the email account you are subscribed to
the list, with the subject ?unsubscribe?.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From ian.g.kelly at gmail.com  Thu Jun 18 03:36:07 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 18 Jun 2015 01:36:07 -0600
Subject: Get off the list
In-Reply-To: <87d20tijin.fsf@Equus.decebal.nl>
References: <mailman.582.1434610744.13271.python-list@python.org>
 <87d20tijin.fsf@Equus.decebal.nl>
Message-ID: <CALwzidm9du2va0pBJL=Pb8fC8FEOyY675fXaQ8J1DvwWjiR35A@mail.gmail.com>

On Thu, Jun 18, 2015 at 1:05 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> On Wednesday 17 Jun 2015 09:09 CEST, Deogratius Musiige wrote:
>
>> How can i get off this mailing list?
>
> Looking at the headers:
>     List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
>      <mailto:python-list-request at python.org?subject=unsubscribe>
>
> So I would send an email with the email account you are subscribed to
> the list, with the subject ?unsubscribe?.

Not to the list. python-list and python-list-request are different addresses.

Personally though, I'd just use the web interface.


From temp.sha at gmail.com  Thu Jun 18 03:41:52 2015
From: temp.sha at gmail.com (ravi)
Date: Thu, 18 Jun 2015 00:41:52 -0700 (PDT)
Subject: help in understanding the stackless code
Message-ID: <f0d40f43-36ee-4c41-9892-9b5ee524fea3@googlegroups.com>

hi,
I am new to python and need to know why the calling of switch(1) invokes the function "listen" twice in the below program?



import stackless

class EventHandler:
    def __init__(self,*outputs):
        if outputs==None:
            self.outputs=[]
        else:
            self.outputs=list(outputs)

        self.channel = stackless.channel()
        stackless.tasklet(self.listen)()

    def listen(self):
        print "in listen()..."
        while 1:
            val = self.channel.receive()
            self.processMessage(val)
            for output in self.outputs:
                self.notify(output)
                
    def processMessage(self,val):
        pass

    def notify(self,output):
        pass

    def registerOutput(self,output):
        print "in registerOutput()..."
        self.outputs.append(output)
    
    def __call__(self,val):
        print "in __call__ ..."
        self.channel.send(val)

class Switch(EventHandler):
    def __init__(self,initialState=0,*outputs):
        EventHandler.__init__(self,*outputs)
        self.state = initialState

    def processMessage(self,val):
        print "in processMessage() of Switch..."
        self.state = val

    def notify(self,output):
        print "in notify() of switch..."
        output((self,self.state))

class Reporter(EventHandler):
    def __init__(self,msg="%(sender)s send message %(value)s"):
        EventHandler.__init__(self)
        self.msg = msg
        
    def processMessage(self,msg):
        print "in processMessage() of Reporter..."
        sender,value=msg
        print self.msg % {'sender':sender,'value':value}


if __name__ == "__main__":
    reporter = Reporter()
    switch = Switch(0,reporter) 
    switch(1)




output:
=========

in __call__ ...
in listen()...
in listen()...
in processMessage() of Switch...
in notify() of switch...
in __call__ ...
in processMessage() of Reporter...
<__main__.Switch instance at 0x8d822cc> send message 1





thanks.


From __peter__ at web.de  Thu Jun 18 03:43:12 2015
From: __peter__ at web.de (Peter Otten)
Date: Thu, 18 Jun 2015 09:43:12 +0200
Subject: JSON Object to CSV Question
References: <CAC9q6M4w5mNj2Lb2MxND_46jbiSwk123mKGKBJckmYCo+_MoSw@mail.gmail.com>
Message-ID: <mltsqh$7p1$1@ger.gmane.org>

Saran Ahluwalia wrote:

> Good Evening Everyone:
> 
> I would like to have this JSON object written out to a CSV file so that
> the keys are header fields (for each of the columns) and the values are
> values that are associated with each header field. Is there a best
> practice for working with this? Ideally I would like to recursively
> iterate through the key value pairs. Thank you in advance. I am using
> Python 3.4 on Windows. My editor is Sublime 2.
> 
> Here is the JSON object:
> 
> {
> "Fee": {
> "A": "5",
> "FEC": "1/1/0001 12:00:00 AM",
> "TE": null,
> "Locator": null,
> "Message": "Transfer Fee",
> "AT": null,
> "FT": null,
> "FR": "True",
> "FY": null,
> "FR": null,
> "FG": "0",
> "Comment": null,
> "FUD": null,
> "cID": null,
> "GEO": null,
> "ISO": null,
> "TRID": null,
> "XTY": "931083",
> "ANM": null,
> "NM": null
> },
> "CF": "Fee",
> "ID": "2"
> }
> 
> The value, "Fee" associated with the key, "CF" should not be included as a
> column header (only as a value of the key "CF").
> 
> Other than the former, the keys should be headers and the corresponding
> tuples - the field values.
> 
> In essence, my goal is to the following:
> 
> You get a dictionary object (the "outer" dictionary)
> You get the data from the "CF" key (fixed name?) which is a string ("Fee"
> in your example)
> You use that value as a key to obtain another value from the same "outer"
> dictionary, which should be a another dictionary (the "inner" dictionary)
> You make a CSV file with:
> 
>    - a header that contains "CF" plus all keys in the "inner" dictionary
>    that have an associated value
>    - the value from key "CF" in the "outer" dictionary plus all non-null
>    values in the "inner" dictionary.
> 
> I have done the following:

For some value of "I" :(
> 
> import csv
> import json
> import sys
> 
> def hook(obj):
>     return obj
> 
> def flatten(obj):
>     for k, v in obj:
>         if isinstance(v, list):
>             yield from flatten(v)
>         else:
>             yield k, v
> 
> if __name__ == "__main__":
>     with open("data.json") as f:
>         data = json.load(f, object_pairs_hook=hook)
> 
>     pairs = list(flatten(data))
> 
>     writer = csv.writer(sys.stdout)
>     writer.writerow([k for k, v in pairs])
>     writer.writerow([v for k, v in pairs])
> 
> The output is as follows:
> 
> $ python3 json2csv.py
> A,FEC,TE,Locator,Message,AT,FT,FR,FY,FR,FG,Comment,FUD,
> cID,GEO,ISO,TRID,XTY,ANM,NM,CF,ID
> 5,1/1/0001 12:00:00 AM,,,Transfer Fee,,,True,,,0,,,,,,,931083,,,Fee,2
> 
> I do not want to have duplicate column names.

If you tell the exact output you want and in particular how you want to 
resolve any name clashes I might give you a hint. I'm unlikely to produce 
more code though.

> Any advice on other best practices that I may utilize?

Align your engineering efforts and capabilities to deliver on your strategy 
and, in particular, your three core, I mean your four core ambitions...



From marc at marcsnet.com  Thu Jun 18 04:32:30 2015
From: marc at marcsnet.com (Marc Lucke)
Date: Thu, 18 Jun 2015 18:32:30 +1000
Subject: Get off the list
In-Reply-To: <A804AEE67207BF42BAF8CEC79839C8D63404C8FA@kbnmxexc11.Demant.com>
References: <A804AEE67207BF42BAF8CEC79839C8D63404C8FA@kbnmxexc11.Demant.com>
Message-ID: <5582821E.1080208@marcsnet.com>

there is no way.  You are trapped forever.  Resistance is futile.

On 17/06/2015 5:09 PM, Deogratius Musiige wrote:
> Hi,
> 
> How can i get off this mailing list?
> 
>  
> 
> Best regards / Med venlig hilsen
> 
> Deogratius Musiige
> Software Development Engineer
> 
> *Sennheiser Communications A/S*
> Industriparken 27
> DK-2750 Ballerup
> 
> Direct                  +45 5618 0320
> Mail                     demu at senncom.com
> Web                    www.sennheiser.com
> 
> 
>  
> 
> 
> 


From marco.buttu at gmail.com  Thu Jun 18 05:00:57 2015
From: marco.buttu at gmail.com (Marco Buttu)
Date: Thu, 18 Jun 2015 11:00:57 +0200
Subject: Classic OOP in Python
In-Reply-To: <6944ae86-86eb-4a9b-be05-21d0be7bb206@googlegroups.com>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net>
 <6944ae86-86eb-4a9b-be05-21d0be7bb206@googlegroups.com>
Message-ID: <558288C9.3020906@oa-cagliari.inaf.it>

On 17/06/2015 23:33, sohcahtoa82 at gmail.com wrote:
> I had a Java class where we had to learn TDD, and that's the way TDD was taught to us, and I hated it. We watched a video of this guy explaining TDD with a hat that was red on the front and green on the back.  It involved writing a simple failing unit test, then write code to fix it, then refactor the tests and/or code.

I do not think it is important that the test fails before writing the 
code, but for sure it is really wise to ensure every test fails at least 
once. If I wrote a test and it has never failed before, then I always 
change something in the test itself, in order to have an expected 
failure. And sometimes the expected failures make me better understand 
the behavior of the code.

> As an in-class exercise, we had to write an implementation of Conway's Game of Life.  I threw TDD out the window and just wrote the damn program in under 15 minutes, then another 10 minutes to write unit tests that tested every possible code branch and several invalid inputs.  Meanwhile, the people doing TDD the "right" way didn't even have a complete program after over an hour.
>
> The brand of TTD we were taught would end up multiplying development time by at least a factor of 3, and by the time you were done, at least 75% of the tests you had written will have been removed due to rampant refactoring.
>
> IMO, that kind of TTD is an utter waste of time.

I think TDD helps a lot to design the code to be "testable" as much as 
possible. I believe good and experienced programmers could think in 
advance the way to write testable code, without be driven by tests. 
However, I also think that the bigger part of programmers, as me, do not 
have this ability. That's the case TDD comes in handy helping us to 
write testable code.

In case of bug fixing, I think also experienced and good programmers 
that do not believe in TDD should always write the tests in advance, for 
instance because:

* it is really the better way to understand the problem and point to it
* chances are that you will not write a test after fixing the bug, i.e. 
because you may have other priorities, so you will miss a regression test


-- 
Marco Buttu

INAF-Osservatorio Astronomico di Cagliari
Via della Scienza n. 5, 09047 Selargius (CA)
Phone: 070 711 80 217
Email: mbuttu at oa-cagliari.inaf.it



From jeanmichel at sequans.com  Thu Jun 18 05:39:55 2015
From: jeanmichel at sequans.com (Jean-Michel Pichavant)
Date: Thu, 18 Jun 2015 11:39:55 +0200 (CEST)
Subject: ctypes and byte order
In-Reply-To: <818768874.4716193.1434617881522.JavaMail.root@sequans.com>
Message-ID: <1972345947.4721181.1434620395686.JavaMail.root@sequans.com>

Hi list,

I'm currently writing python code that writes a small binary file to be used by another device which code is written in C.
The python code runs on a little endian CPU, and unfortunately, the other device is using a big endian MIPS.

My problem is the following: I cannot make an array of n int work in big endian, I can workaround the problem by creating n intergers, which is nicely done thanks to list comprehension but that seems to me like a misuse of the ctypes modules.

ctypes is expecting a 'c_uint_be_Array_5' (note the _be_ for big endian), and I don't know how to construct such object.

I hope I am clear.

Thanks,

JM

here's a code sample (python 2.7):

--------------------
import ctypes, sys

iarray = ctypes.c_uint32*5  # array of 5 unsigned 32 bits

class Foo(ctypes.Structure):
	"""Native byte order, won't work on my big endian device"""
	_fields_ = [('bar', iarray)] 

class Foo_be(ctypes.BigEndianStructure):
	"""big endian version, but python code fails"""
	_fields_ = [('bar', iarray)] 

class Foo_be_working(ctypes.BigEndianStructure):
	"""Working big endian version, looks more like a workaround."""
	_fields_ = [('bar%i'%i, ctypes.c_uint32) for i in range(5)]


print sys.version
f = Foo(iarray(0,1,2,3,4))
print "uint32 array: ", f.bar

f_be = Foo_be_working(0,1,2,3,4)
print "bar0 and bar5: ", f_be.bar0, f_be.bar5

f_be = Foo_be(iarray(0,1,2,3,4)) # will raise an exception

--------------------


The output

2.7.3 (default, Mar 13 2014, 11:03:55) 
[GCC 4.7.2]
uint32 array:  <__main__.c_uint_Array_5 object at 0x1cf4560>
bar0 and bar4:  0 4


TypeError: incompatible types, c_uint_Array_5 instance instead of c_uint_be_Array_5 instance
     24 
---> 25 f_be = Foo_be(iarray(0,1,2,3,4))
     26 



-- IMPORTANT NOTICE: 

The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

From productivi.co at gmail.com  Thu Jun 18 05:51:15 2015
From: productivi.co at gmail.com (Productivi .co)
Date: Thu, 18 Jun 2015 11:51:15 +0200
Subject: Time saving tips for Pythonists
Message-ID: <CADHijmn9P2rg2NhV3vGyc-NNarQETSNLrNw6Cyo0xc6mdvT1RA@mail.gmail.com>

Hello Pythonists!

I'm preparing an article to show up at simplilearn.com about the best time
saving tips Pyhonists use, and thus, conducting interviews with Pythonists
like you.

The interview question I would like to ask you is:

*What are your best time saving tips when programming Python?*

Looking forward to your replies to this question, and will keep you updated
about the article.

Thanks.
Abder-Rahman
productivi.co
*Let The Productivity Journey Begin.*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150618/688fa389/attachment.html>

From Cecil at decebal.nl  Thu Jun 18 06:13:21 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 18 Jun 2015 12:13:21 +0200
Subject: Get off the list
References: <mailman.582.1434610744.13271.python-list@python.org>
 <87d20tijin.fsf@Equus.decebal.nl>
 <mailman.588.1434620692.13271.python-list@python.org>
Message-ID: <874mm5iata.fsf@Equus.decebal.nl>

On Thursday 18 Jun 2015 09:36 CEST, Ian Kelly wrote:

> On Thu, Jun 18, 2015 at 1:05 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> On Wednesday 17 Jun 2015 09:09 CEST, Deogratius Musiige wrote:
>>
>>> How can i get off this mailing list?
>>
>> Looking at the headers: List-Unsubscribe:
>> <https://mail.python.org/mailman/options/python-list>,
>> <mailto:python-list-request at python.org?subject=unsubscribe>
>>
>> So I would send an email with the email account you are subscribed
>> to the list, with the subject ?unsubscribe?.
>
> Not to the list. python-list and python-list-request are different
> addresses.

Oops, I was not specific enough again. :'-( The email address was
shown, but it would be very easy to overlook it.


> Personally though, I'd just use the web interface.

That is the other option. I prefer sending an email, but for someone
asking how to unsubscribe, the web interface probably is a better
option. Next time I should think better about the receiver of the
information.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From __peter__ at web.de  Thu Jun 18 06:33:08 2015
From: __peter__ at web.de (Peter Otten)
Date: Thu, 18 Jun 2015 12:33:08 +0200
Subject: ctypes and byte order
References: <818768874.4716193.1434617881522.JavaMail.root@sequans.com>
 <1972345947.4721181.1434620395686.JavaMail.root@sequans.com>
Message-ID: <mlu6p5$dbo$1@ger.gmane.org>

Jean-Michel Pichavant wrote:

> I'm currently writing python code that writes a small binary file to be
> used by another device which code is written in C. The python code runs on
> a little endian CPU, and unfortunately, the other device is using a big
> endian MIPS.
> 
> My problem is the following: I cannot make an array of n int work in big
> endian, I can workaround the problem by creating n intergers, which is
> nicely done thanks to list comprehension but that seems to me like a
> misuse of the ctypes modules.
> 
> ctypes is expecting a 'c_uint_be_Array_5' (note the _be_ for big endian),
> and I don't know how to construct such object.
> 
> I hope I am clear.

Yes, very clear, thanks for the effort, although the nitpicker in me has to 
mention that an off-by-one bug slipped into your code ;)

> class Foo_be_working(ctypes.BigEndianStructure):
>     """Working big endian version, looks more like a workaround."""
>     _fields_ = [('bar%i'%i, ctypes.c_uint32) for i in range(5)]
[...]
> f_be = Foo_be_working(0,1,2,3,4)
> print "bar0 and bar5: ", f_be.bar0, f_be.bar5

I'm not very familiar with ctypes, but a random walk through the source 
found the _endian module and the __ctypes_be__ attribute. So
> 
> iarray = ctypes.c_uint32*5  # array of 5 unsigned 32 bits
> 
> class Foo(ctypes.Structure):
>     """Native byte order, won't work on my big endian device"""
>     _fields_ = [('bar', iarray)]
> 
> class Foo_be(ctypes.BigEndianStructure):
>     """big endian version, but python code fails"""
>     _fields_ = [('bar', iarray)]

becomes

$ cat be2.py
import ctypes, sys

iarray_be = ctypes.c_uint32.__ctype_be__*5

class Foo_be(ctypes.BigEndianStructure):
        _fields_ = [('bar', iarray_be)] 

print sys.version
f_be = Foo_be((0,1,2,3,0x11223344))
print hex(f_be.bar[4])

$ python be2.py 
2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2]
0x11223344L

which might do what you want.



From breamoreboy at yahoo.co.uk  Thu Jun 18 06:50:28 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 18 Jun 2015 11:50:28 +0100
Subject: Classic OOP in Python
In-Reply-To: <201506172233.t5HMXBIm002277@fido.openend.se>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net><49f789cf-cb67-4812-8a91-218a4f049958@googlegroups.com>
 <201506172233.t5HMXBIm002277@fido.openend.se>
Message-ID: <mlu7pl$u1t$1@ger.gmane.org>

On 17/06/2015 23:33, Laura Creighton wrote:
> In a message of Wed, 17 Jun 2015 14:14:34 -0700, Ned Batchelder writes:
>
>
> TDD is supposed to make you brave, not cowards, and it's
> Ned's most excellent tool
> http://nedbatchelder.com/code/coverage/
> that I recommend to TDD dogmatic cowards.
>
> Even if you don't want to use TTD, you will enjoy Ned's tool.  It's GREAT.
> Thank you, Ned.
>
> Laura
>

Throw in http://clonedigger.sourceforge.net/ as well and you've a really 
awesome combination.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From fabien.maussion at gmail.com  Thu Jun 18 07:03:14 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Thu, 18 Jun 2015 13:03:14 +0200
Subject: Classic OOP in Python
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <f0c76343-7230-499c-ba30-33d7f6513503@googlegroups.com>
Message-ID: <mlu8hh$ntr$1@speranza.aioe.org>

On 06/17/2015 11:16 PM, sohcahtoa82 at gmail.com wrote:
> You don't need interfaces with Python.  Duck typing makes that all possible.

Yes, but I also like interfaces (or in python: mimicked interfaces with 
NotImplementedError) for their clarity and documentation purposes.

Would you consider the following kind of program "unpythonic"?

class MovingObject(object):
     """Great doc about what a moving object is"""

     def move(self):
     	"""Great doc about move"""
         raise NotImplementedError()

class Dog(MovingObject):
     def move(self):
         print "Dog is moving"

class Car(MovingObject):
     def move(self):
         print "Car is moving"

(Disclaimer: I learned OOP with Java)

Fabien


From toddrjen at gmail.com  Thu Jun 18 07:07:53 2015
From: toddrjen at gmail.com (Todd)
Date: Thu, 18 Jun 2015 13:07:53 +0200
Subject: Classic OOP in Python
In-Reply-To: <mlu8hh$ntr$1@speranza.aioe.org>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <f0c76343-7230-499c-ba30-33d7f6513503@googlegroups.com>
 <mlu8hh$ntr$1@speranza.aioe.org>
Message-ID: <CAFpSVpKOT20yvL=uEH8r0bUac=VJhwmB5mN7h8diUCDTx0GqwQ@mail.gmail.com>

On Thu, Jun 18, 2015 at 1:03 PM, Fabien <fabien.maussion at gmail.com> wrote:

> On 06/17/2015 11:16 PM, sohcahtoa82 at gmail.com wrote:
>
>> You don't need interfaces with Python.  Duck typing makes that all
>> possible.
>>
>
> Yes, but I also like interfaces (or in python: mimicked interfaces with
> NotImplementedError) for their clarity and documentation purposes.
>
> Would you consider the following kind of program "unpythonic"?
>
> class MovingObject(object):
>     """Great doc about what a moving object is"""
>
>     def move(self):
>         """Great doc about move"""
>         raise NotImplementedError()
>
> class Dog(MovingObject):
>     def move(self):
>         print "Dog is moving"
>
> class Car(MovingObject):
>     def move(self):
>         print "Car is moving"
>
> (Disclaimer: I learned OOP with Java)
>
>
I think this is what abstract base classes are for in Python.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150618/021c025d/attachment.html>

From suscricions at gmail.com  Thu Jun 18 07:21:15 2015
From: suscricions at gmail.com (Jason P.)
Date: Thu, 18 Jun 2015 04:21:15 -0700 (PDT)
Subject: Classic OOP in Python
In-Reply-To: <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
Message-ID: <01adfef5-7edf-447c-9397-853c4fd65e53@googlegroups.com>

El mi?rcoles, 17 de junio de 2015, 21:44:51 (UTC+2), Ned Batchelder  escribi?:
> On Wednesday, June 17, 2015 at 3:21:32 PM UTC-4, Jason P. wrote:
> > Hello Python community.
> > 
> > I come from a classic background in what refers to OOP. Mostly Java and PHP (> 5.3). I'm used to abstract classes, interfaces, access modifiers and so on.
> > 
> > Don't get me wrong. I know that despite the differences Python is fully object oriented. My point is, do you know any book or resource that explains in deep the pythonic way of doing OOP?
> > 
> > For example, I'm gonna try to develop a modest application from ground up using TDD. If it had been done in Java for instance, I would made extensive use of interfaces to define the boundaries of my system. How would I do something like that in Python?
> > 
> > 
> > Many thanks!
> 
> What other languages do with interfaces, Python does with duck-typing. You
> can build something like interfaces in Python, but many people don't bother.
> 
> I don't know if your project will be web-based, but here is an entire book
> about developing Python web sites with a TDD approach:
> 
> http://www.obeythetestinggoat.com/
> 
> (Don't mind the unusual domain name, it's a bit of an inside joke...)
> 
> TDD and interfaces are separate concepts, and I'm not sure they even
> intersect.  TDD is about writing tests as a way to design the best system,
> and putting testing at the center of your development workflow.  It works
> great with Python even without interfaces.
> 
> --Ned.

I'm aware of duck typing. The point in using interfaces is to be explicit about the boundaries of a system.

Quite a red "Growing Object-Oriented Software, Guided by Tests", by the way. In fact interfaces are key components in the style of building software they propose, in good company with TDD.

Thx!


From suscricions at gmail.com  Thu Jun 18 07:22:51 2015
From: suscricions at gmail.com (Jason P.)
Date: Thu, 18 Jun 2015 04:22:51 -0700 (PDT)
Subject: Classic OOP in Python
In-Reply-To: <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
Message-ID: <48b428f0-7373-4847-b1fd-2c8532c056ee@googlegroups.com>

El mi?rcoles, 17 de junio de 2015, 21:44:51 (UTC+2), Ned Batchelder  escribi?:
> On Wednesday, June 17, 2015 at 3:21:32 PM UTC-4, Jason P. wrote:
> > Hello Python community.
> > 
> > I come from a classic background in what refers to OOP. Mostly Java and PHP (> 5.3). I'm used to abstract classes, interfaces, access modifiers and so on.
> > 
> > Don't get me wrong. I know that despite the differences Python is fully object oriented. My point is, do you know any book or resource that explains in deep the pythonic way of doing OOP?
> > 
> > For example, I'm gonna try to develop a modest application from ground up using TDD. If it had been done in Java for instance, I would made extensive use of interfaces to define the boundaries of my system. How would I do something like that in Python?
> > 
> > 
> > Many thanks!
> 
> What other languages do with interfaces, Python does with duck-typing. You
> can build something like interfaces in Python, but many people don't bother.
> 
> I don't know if your project will be web-based, but here is an entire book
> about developing Python web sites with a TDD approach:
> 
> http://www.obeythetestinggoat.com/
> 
> (Don't mind the unusual domain name, it's a bit of an inside joke...)
> 
> TDD and interfaces are separate concepts, and I'm not sure they even
> intersect.  TDD is about writing tests as a way to design the best system,
> and putting testing at the center of your development workflow.  It works
> great with Python even without interfaces.
> 
> --Ned.

I forgot to mention that the book you recommend seems to be a good starting point ;)


From suscricions at gmail.com  Thu Jun 18 07:24:58 2015
From: suscricions at gmail.com (Jason P.)
Date: Thu, 18 Jun 2015 04:24:58 -0700 (PDT)
Subject: Classic OOP in Python
In-Reply-To: <87eglaqdca.fsf@elektro.pacujo.net>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net>
Message-ID: <42563233-5269-41e7-87a9-1bde66480d09@googlegroups.com>

El mi?rcoles, 17 de junio de 2015, 22:39:31 (UTC+2), Marko Rauhamaa  escribi?:
> Ned Batchelder <ned at nedbatchelder.com>:
> 
> > TDD is about writing tests as a way to design the best system, and
> > putting testing at the center of your development workflow. It works
> > great with Python even without interfaces.
> 
> I wonder how great it really is. Testing is important, that's for sure,
> but to make it a dogmatic starting point of development is not that
> convincing.
> 
> The way it was explained to me was that in TDD you actually don't write
> code to any requirements or design: you simply do the least to pass the
> tests. Thus, say you need to write a program that inputs a string and
> outputs the same string surrounded by parentheses (the requirement), the
> starting point might be this test case:
> 
>    - run the program
>    - give it the word "hello" as input
>    - check that the program prints out "(hello)"
> 
> The right TDD thing would be to satisfy the test with this program:
> 
>    input()
>    print("(hello)")
> 
> That *ought* to be the first version of the program until further test
> cases are added that invalidate it.
> 
> 
> Another interesting ism I have read about is the idea that the starting
> point of any software project should be the user manual. The developers
> should then go and build the product that fits the manual.
> 
> 
> Marko

The refactor phase is key in TDD (red-green-refactor). Again, GOOS is an advisable source of knowledge in this matter.


From marko at pacujo.net  Thu Jun 18 07:25:26 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 18 Jun 2015 14:25:26 +0300
Subject: Classic OOP in Python
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <f0c76343-7230-499c-ba30-33d7f6513503@googlegroups.com>
 <mlu8hh$ntr$1@speranza.aioe.org>
 <mailman.593.1434625694.13271.python-list@python.org>
Message-ID: <87oakdz2ah.fsf@elektro.pacujo.net>

Todd <toddrjen at gmail.com>:

> On Thu, Jun 18, 2015 at 1:03 PM, Fabien <fabien.maussion at gmail.com> wrote:
>> Would you consider the following kind of program "unpythonic"?
>>
>> class MovingObject(object):
>>     """Great doc about what a moving object is"""
>>
>>     def move(self):
>>         """Great doc about move"""
>>         raise NotImplementedError()
>>
>> class Dog(MovingObject):
>>     def move(self):
>>         print "Dog is moving"
>>
>> class Car(MovingObject):
>>     def move(self):
>>         print "Car is moving"
>>
>> (Disclaimer: I learned OOP with Java)
>>
>>
> I think this is what abstract base classes are for in Python.

And they can be ok as long as you're not making them into a habit.


Marko


From jeanmichel at sequans.com  Thu Jun 18 07:25:35 2015
From: jeanmichel at sequans.com (Jean-Michel Pichavant)
Date: Thu, 18 Jun 2015 13:25:35 +0200 (CEST)
Subject: ctypes and byte order
In-Reply-To: <mlu6p5$dbo$1@ger.gmane.org>
Message-ID: <678134472.4779269.1434626735692.JavaMail.root@sequans.com>

----- Original Message -----
> From: "Peter Otten" <__peter__ at web.de>
> becomes
> 
> $ cat be2.py
> import ctypes, sys
> 
> iarray_be = ctypes.c_uint32.__ctype_be__*5
> 
> class Foo_be(ctypes.BigEndianStructure):
>         _fields_ = [('bar', iarray_be)]
> 
> print sys.version
> f_be = Foo_be((0,1,2,3,0x11223344))
> print hex(f_be.bar[4])
> 
> $ python be2.py
> 2.7.6 (default, Mar 22 2014, 22:59:56)
> [GCC 4.8.2]
> 0x11223344L
> 
> which might do what you want.

Brilliant !

I've tested it and it yields the exact same results (binary file content wise) than my "workaround" structure.
But that's way better since my actual structure is more complex and arrays will be required.

Though I'm slightly puzzled by the ctypes author(s) choice, this is not documented and requires to peek into the source code. Dunder attributes are rarely part of an interface.

Anyway, thanks for your help !

JM



-- IMPORTANT NOTICE: 

The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

From rosuav at gmail.com  Thu Jun 18 07:43:28 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 18 Jun 2015 21:43:28 +1000
Subject: Classic OOP in Python
In-Reply-To: <mlu8hh$ntr$1@speranza.aioe.org>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <f0c76343-7230-499c-ba30-33d7f6513503@googlegroups.com>
 <mlu8hh$ntr$1@speranza.aioe.org>
Message-ID: <CAPTjJmrHkkqn_2CmwSjFtk9L3Ks5HBQU88dcmnbeJJq7AzUJqg@mail.gmail.com>

On Thu, Jun 18, 2015 at 9:03 PM, Fabien <fabien.maussion at gmail.com> wrote:
> Would you consider the following kind of program "unpythonic"?
>
> class MovingObject(object):
>     """Great doc about what a moving object is"""
>
>     def move(self):
>         """Great doc about move"""
>         raise NotImplementedError()
>
> class Dog(MovingObject):
>     def move(self):
>         print "Dog is moving"
>
> class Car(MovingObject):
>     def move(self):
>         print "Car is moving"
>
> (Disclaimer: I learned OOP with Java)

Now try extending the concept to two, three, or four such interfaces.
Files can be moved, opened, copied, and unlinked. Should they share
any sort of interface with dogs, boxes, handwriting, and railway
carriages? It's much better to simply define the attributes of each
object separately; most non-trivial cases don't involve simple methods
with no additional arguments, and the chances of an incompatibility
(or worse, a forced compatibility) go up.

ChrisA


From ned at nedbatchelder.com  Thu Jun 18 07:54:10 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Thu, 18 Jun 2015 04:54:10 -0700 (PDT)
Subject: Classic OOP in Python
In-Reply-To: <01adfef5-7edf-447c-9397-853c4fd65e53@googlegroups.com>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <01adfef5-7edf-447c-9397-853c4fd65e53@googlegroups.com>
Message-ID: <69585e5b-a8c6-413b-849b-ce51585b72c7@googlegroups.com>

On Thursday, June 18, 2015 at 7:21:29 AM UTC-4, Jason P. wrote:
> El mi?rcoles, 17 de junio de 2015, 21:44:51 (UTC+2), Ned Batchelder  escribi?:
> > On Wednesday, June 17, 2015 at 3:21:32 PM UTC-4, Jason P. wrote:
> > > Hello Python community.
> > > 
> > > I come from a classic background in what refers to OOP. Mostly Java and PHP (> 5.3). I'm used to abstract classes, interfaces, access modifiers and so on.
> > > 
> > > Don't get me wrong. I know that despite the differences Python is fully object oriented. My point is, do you know any book or resource that explains in deep the pythonic way of doing OOP?
> > > 
> > > For example, I'm gonna try to develop a modest application from ground up using TDD. If it had been done in Java for instance, I would made extensive use of interfaces to define the boundaries of my system. How would I do something like that in Python?
> > > 
> > > 
> > > Many thanks!
> > 
> > What other languages do with interfaces, Python does with duck-typing. You
> > can build something like interfaces in Python, but many people don't bother.
> > 
> > I don't know if your project will be web-based, but here is an entire book
> > about developing Python web sites with a TDD approach:
> > 
> > http://www.obeythetestinggoat.com/
> > 
> > (Don't mind the unusual domain name, it's a bit of an inside joke...)
> > 
> > TDD and interfaces are separate concepts, and I'm not sure they even
> > intersect.  TDD is about writing tests as a way to design the best system,
> > and putting testing at the center of your development workflow.  It works
> > great with Python even without interfaces.
> > 
> > --Ned.
> 
> I'm aware of duck typing. The point in using interfaces is to be explicit about the boundaries of a system.
> 
> Quite a red "Growing Object-Oriented Software, Guided by Tests", by the way. In fact interfaces are key components in the style of building software they propose, in good company with TDD.

Yes, that book uses interfaces, because that book uses Java.  Different
languages offer different tools.  You can make interfaces in Python, but you
don't need to.  They aren't enforced by Python, so you aren't gaining much
other than documentation from them, so why not just use documentation?

Abstract classes provide another tool.  They can insist that you provide
implementations of abstract methods.  In my experience, it is easy to get
to a point where you are struggling to satisfy your simple-minded abstract
classes, rather than writing the code that you know you need to solve your
actual problem.  As Chris just mentioned elsewhere in this thread, you have
to be very careful how you define your abstract classes.  I've worked in 
Java projects where I had to provide 10 dummy implementations of methods I
knew I wasn't going to need, just because the interface insisted they had
to exist.

The Python culture is to document your expectations, and write enough tests
to verify that your code does what it claims to do.  You are already planning
on a TDD flow, so you will have plenty of tests.

Try doing without interfaces or abstract classes.  See how it goes. It's the
Python way. :)

--Ned.


From python at mrabarnett.plus.com  Thu Jun 18 08:14:46 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 18 Jun 2015 13:14:46 +0100
Subject: help in understanding the stackless code
In-Reply-To: <f0d40f43-36ee-4c41-9892-9b5ee524fea3@googlegroups.com>
References: <f0d40f43-36ee-4c41-9892-9b5ee524fea3@googlegroups.com>
Message-ID: <5582B636.9030006@mrabarnett.plus.com>

On 2015-06-18 08:41, ravi wrote:
> hi,
> I am new to python and need to know why the calling of switch(1) invokes the function "listen" twice in the below program?
>
>
>
> import stackless
>
> class EventHandler:
>      def __init__(self,*outputs):
>          if outputs==None:
>              self.outputs=[]
>          else:
>              self.outputs=list(outputs)
>
>          self.channel = stackless.channel()
>          stackless.tasklet(self.listen)()
>
>      def listen(self):
>          print "in listen()..."
>          while 1:
>              val = self.channel.receive()
>              self.processMessage(val)
>              for output in self.outputs:
>                  self.notify(output)
>
>      def processMessage(self,val):
>          pass
>
>      def notify(self,output):
>          pass
>
>      def registerOutput(self,output):
>          print "in registerOutput()..."
>          self.outputs.append(output)
>
>      def __call__(self,val):
>          print "in __call__ ..."
>          self.channel.send(val)
>
> class Switch(EventHandler):
>      def __init__(self,initialState=0,*outputs):
>          EventHandler.__init__(self,*outputs)
>          self.state = initialState
>
>      def processMessage(self,val):
>          print "in processMessage() of Switch..."
>          self.state = val
>
>      def notify(self,output):
>          print "in notify() of switch..."
>          output((self,self.state))
>
> class Reporter(EventHandler):
>      def __init__(self,msg="%(sender)s send message %(value)s"):
>          EventHandler.__init__(self)
>          self.msg = msg
>
>      def processMessage(self,msg):
>          print "in processMessage() of Reporter..."
>          sender,value=msg
>          print self.msg % {'sender':sender,'value':value}
>
>
> if __name__ == "__main__":
>      reporter = Reporter()
>      switch = Switch(0,reporter)
>      switch(1)
>
>
>
>
> output:
> =========
>
> in __call__ ...
> in listen()...
> in listen()...
> in processMessage() of Switch...
> in notify() of switch...
> in __call__ ...
> in processMessage() of Reporter...
> <__main__.Switch instance at 0x8d822cc> send message 1
>
Is it because EventHandler has 2 subclasses, namely Switch and
Reporter, and you have an instance of each?



From __peter__ at web.de  Thu Jun 18 08:24:03 2015
From: __peter__ at web.de (Peter Otten)
Date: Thu, 18 Jun 2015 14:24:03 +0200
Subject: ctypes and byte order
References: <mlu6p5$dbo$1@ger.gmane.org>
 <678134472.4779269.1434626735692.JavaMail.root@sequans.com>
Message-ID: <mlud94$qak$1@ger.gmane.org>

Jean-Michel Pichavant wrote:

> ----- Original Message -----
>> From: "Peter Otten" <__peter__ at web.de>
>> becomes
>> 
>> $ cat be2.py
>> import ctypes, sys
>> 
>> iarray_be = ctypes.c_uint32.__ctype_be__*5
>> 
>> class Foo_be(ctypes.BigEndianStructure):
>>         _fields_ = [('bar', iarray_be)]
>> 
>> print sys.version
>> f_be = Foo_be((0,1,2,3,0x11223344))
>> print hex(f_be.bar[4])
>> 
>> $ python be2.py
>> 2.7.6 (default, Mar 22 2014, 22:59:56)
>> [GCC 4.8.2]
>> 0x11223344L
>> 
>> which might do what you want.
> 
> Brilliant !
> 
> I've tested it and it yields the exact same results (binary file content
> wise) than my "workaround" structure. But that's way better since my
> actual structure is more complex and arrays will be required.
> 
> Though I'm slightly puzzled by the ctypes author(s) choice, this is not
> documented and requires to peek into the source code. Dunder attributes
> are rarely part of an interface.

On further reflection it dawned on me that what may have made this work is 
passing in the tuple instead of an array:

>>> import ctypes
>>> array = ctypes.c_uint32 * 3
>>> class B(ctypes.BigEndianStructure):
...     _fields_ = [("bar", array)]
... 
>>> b = B((1,2,3))
>>> b.bar
<ctypes._endian.c_uint_be_Array_3 object at 0x7f87700798c0>
>>> b.bar[2]
3L
>>> b = B(array(1,2,3))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: incompatible types, c_uint_Array_3 instance instead of 
c_uint_be_Array_3 instance

Oops...

So the only thing that doesn't work transparently is initialising a data 
structure (or at least array) from its "other-endian" twin.





From random832 at fastmail.us  Thu Jun 18 08:37:12 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 18 Jun 2015 08:37:12 -0400
Subject: Documenting a function signature (was: Set a flag on the function
 or a global?)
In-Reply-To: <CAPTjJmpPh=onWt=WE1pf2eym9jSnNRxxGx-3Aq_22ye+VQ9A7g@mail.gmail.com>
References: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
 <mailman.530.1434500492.13271.python-list@python.org>
 <55818b3c$0$1665$c3e8da3$5496439d@news.astraweb.com>
 <mailman.556.1434553624.13271.python-list@python.org>
 <55819878$0$1658$c3e8da3$5496439d@news.astraweb.com>
 <85egl9ubj5.fsf_-_@benfinney.id.au>
 <CAPTjJmpPh=onWt=WE1pf2eym9jSnNRxxGx-3Aq_22ye+VQ9A7g@mail.gmail.com>
Message-ID: <1434631032.4121167.298938409.7D2684C7@webmail.messagingengine.com>

On Wed, Jun 17, 2015, at 20:14, Chris Angelico wrote:
> Mostly. That would imply that object is a mandatory parameter, which
> AIUI isn't the case for Steven's edir. The downside of this kind of
> signature is that it's hard to show the parameters that have unusual
> defaults (either sentinel objects and custom code to cope, or they're
> pulled out of *a or **kw).

My instinct would be to put =... for that case and explain in the
documentation "If foo is not provided, [behavior]".

Of course, that runs the risk of people thinking you actually mean
Ellipsis, or that providing Ellipsis explicitly will give the same
behavior as the default.


From random832 at fastmail.us  Thu Jun 18 08:38:51 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 18 Jun 2015 08:38:51 -0400
Subject: Posting gzip'd image file - server says Malformed Upload?
In-Reply-To: <CAPTjJmoSYqRyWNj4pQ14e16qRvx+B+xo3qvHfZATxkg=iEC7KA@mail.gmail.com>
References: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>
 <CAPTjJmoSYqRyWNj4pQ14e16qRvx+B+xo3qvHfZATxkg=iEC7KA@mail.gmail.com>
Message-ID: <1434631131.4121351.298941169.05993FA2@webmail.messagingengine.com>

On Wed, Jun 17, 2015, at 20:23, Chris Angelico wrote:
> On Thu, Jun 18, 2015 at 7:55 AM, Paul Hubert <phbrt25 at gmail.com> wrote:
> > f_in = open(dafile, 'rb')
> > f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
> > f_out.writelines(f_in)
> > f_out.close()
> > f_in.close()
> 
> Are you sure you want iteration and writelines() here? I would be
> inclined to avoid those for any situation that isn't plain text. If
> the file isn't too big, I'd just read it all in a single blob and then
> write it all out at once.

Is there a reason not to use shutil.copyfileobj?


From rosuav at gmail.com  Thu Jun 18 08:42:28 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 18 Jun 2015 22:42:28 +1000
Subject: Posting gzip'd image file - server says Malformed Upload?
In-Reply-To: <1434631131.4121351.298941169.05993FA2@webmail.messagingengine.com>
References: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>
 <CAPTjJmoSYqRyWNj4pQ14e16qRvx+B+xo3qvHfZATxkg=iEC7KA@mail.gmail.com>
 <1434631131.4121351.298941169.05993FA2@webmail.messagingengine.com>
Message-ID: <CAPTjJmpm8mTUTBjqpx6KqMPOQCb0H2=LhHvwR3mTuKhAeYj9dA@mail.gmail.com>

On Thu, Jun 18, 2015 at 10:38 PM,  <random832 at fastmail.us> wrote:
> On Wed, Jun 17, 2015, at 20:23, Chris Angelico wrote:
>> On Thu, Jun 18, 2015 at 7:55 AM, Paul Hubert <phbrt25 at gmail.com> wrote:
>> > f_in = open(dafile, 'rb')
>> > f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
>> > f_out.writelines(f_in)
>> > f_out.close()
>> > f_in.close()
>>
>> Are you sure you want iteration and writelines() here? I would be
>> inclined to avoid those for any situation that isn't plain text. If
>> the file isn't too big, I'd just read it all in a single blob and then
>> write it all out at once.
>
> Is there a reason not to use shutil.copyfileobj?

If the file is too big (or might be too big) to want to fit into
memory, then sure. But a typical JPEG image isn't going to be
gigabytes of content; chances are it's going to be a meg or so at
most, and that doesn't justify the overhead of chunking. Just read it,
write it, job done.

ChrisA


From ndbecker2 at gmail.com  Thu Jun 18 09:10:53 2015
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 18 Jun 2015 09:10:53 -0400
Subject: thinking with laziness
Message-ID: <mlug0t$8cu$1@ger.gmane.org>

http://begriffs.com/posts/2015-06-17-thinking-with-laziness.html



From jstrickler at gmail.com  Thu Jun 18 09:53:22 2015
From: jstrickler at gmail.com (John Strick)
Date: Thu, 18 Jun 2015 06:53:22 -0700 (PDT)
Subject: Time saving tips for Pythonists
In-Reply-To: <mailman.590.1434622221.13271.python-list@python.org>
References: <mailman.590.1434622221.13271.python-list@python.org>
Message-ID: <cc027613-bb10-486b-a895-7b4a69745633@googlegroups.com>

On Thursday, June 18, 2015 at 6:11:11 AM UTC-4, Productivi .co wrote:
> What are your best time saving tips when programming Python?
PyCharm!


From steve at pearwood.info  Thu Jun 18 09:53:43 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 18 Jun 2015 23:53:43 +1000
Subject: thinking with laziness
References: <mailman.601.1434633089.13271.python-list@python.org>
Message-ID: <5582cd67$0$1671$c3e8da3$5496439d@news.astraweb.com>

On Thu, 18 Jun 2015 11:10 pm, Neal Becker wrote:

> http://begriffs.com/posts/2015-06-17-thinking-with-laziness.html

I wanted to think about that post, but I'm too lazy to read it.


My-apologies-I-couldn't-resist-it-ly y'rs,

-- 
Steven



From cousinstanley at gmail.com  Thu Jun 18 11:10:53 2015
From: cousinstanley at gmail.com (Cousin Stanley)
Date: Thu, 18 Jun 2015 08:10:53 -0700
Subject: Classic OOP in Python
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net>
 <5581fb9d$0$1672$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mlumvh$3ja$1@dont-email.me>


> ....
> python -m doctest application.py
> 
> And from there, I would build up extra doc tests
> ....

An extra doc test .... 

  that fails ....


#!/usr/bin/env python

"""
    NewsGroup .... comp.lang.python 
    Subject ...... Classic OOP in Python
    Date ......... 2015-06-17
    Post_By ...... Steven D'Aprano
    Edit_By ...... Stanley C. Kitching
"""

def bracket( s ) :

    """Return string s bracketed in parentheses.

    >>> bracket( "Hello" )
    '(Hello)'

    >>> bracket( "Yo Mama is a Perl Monkey" )
    '(Yo Mama is a Java Monkey')

    """

    return "(%s)" % s


-- 
Stanley C. Kitching
Human Being
Phoenix, Arizona


From breamoreboy at yahoo.co.uk  Thu Jun 18 12:15:38 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 18 Jun 2015 17:15:38 +0100
Subject: thinking with laziness
In-Reply-To: <5582cd67$0$1671$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.601.1434633089.13271.python-list@python.org>
 <5582cd67$0$1671$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mluqrc$b3i$1@ger.gmane.org>

On 18/06/2015 14:53, Steven D'Aprano wrote:
> On Thu, 18 Jun 2015 11:10 pm, Neal Becker wrote:
>
>> http://begriffs.com/posts/2015-06-17-thinking-with-laziness.html
>
> I wanted to think about that post, but I'm too lazy to read it.
>
>
> My-apologies-I-couldn't-resist-it-ly y'rs,
>

Reminds me of last night's AGM of the Apathy Society, which was an 
outstanding success as nobody turned up.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Thu Jun 18 12:23:48 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 19 Jun 2015 02:23:48 +1000
Subject: thinking with laziness
In-Reply-To: <mluqrc$b3i$1@ger.gmane.org>
References: <mailman.601.1434633089.13271.python-list@python.org>
 <5582cd67$0$1671$c3e8da3$5496439d@news.astraweb.com>
 <mluqrc$b3i$1@ger.gmane.org>
Message-ID: <CAPTjJmq_jXgebTDz6Rfa=_13rXPV8GJp2ODZdGVLxwGvcy8FeQ@mail.gmail.com>

On Fri, Jun 19, 2015 at 2:15 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> Reminds me of last night's AGM of the Apathy Society, which was an
> outstanding success as nobody turned up.

How do you know for sure? Nobody bothered to take minutes.

ChrisA


From toddrjen at gmail.com  Thu Jun 18 12:40:06 2015
From: toddrjen at gmail.com (Todd)
Date: Thu, 18 Jun 2015 18:40:06 +0200
Subject: thinking with laziness
In-Reply-To: <mluqrc$b3i$1@ger.gmane.org>
References: <mailman.601.1434633089.13271.python-list@python.org>
 <5582cd67$0$1671$c3e8da3$5496439d@news.astraweb.com>
 <mluqrc$b3i$1@ger.gmane.org>
Message-ID: <CAFpSVpLCcOirHr9ZjWP+DRpKAe8P_WB9kwW-QNLEqsu=ujiumA@mail.gmail.com>

On Thu, Jun 18, 2015 at 6:15 PM, Mark Lawrence <breamoreboy at yahoo.co.uk>
wrote:

> On 18/06/2015 14:53, Steven D'Aprano wrote:
>
>> On Thu, 18 Jun 2015 11:10 pm, Neal Becker wrote:
>>
>>  http://begriffs.com/posts/2015-06-17-thinking-with-laziness.html
>>>
>>
>> I wanted to think about that post, but I'm too lazy to read it.
>>
>>
>> My-apologies-I-couldn't-resist-it-ly y'rs,
>>
>>
> Reminds me of last night's AGM of the Apathy Society, which was an
> outstanding success as nobody turned up.
>

I thought about it, but couldn't be bothered to find the address.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150618/cd609d21/attachment.html>

From dmaziuk at bmrb.wisc.edu  Thu Jun 18 12:53:54 2015
From: dmaziuk at bmrb.wisc.edu (dmaziuk)
Date: Thu, 18 Jun 2015 09:53:54 -0700 (PDT)
Subject: executable egg?
Message-ID: <d0a50110-66af-45cd-bdd2-7f8ecba68981@googlegroups.com>

Hi all,

Here's my directory structure:

myegg/
    pkg1/
        __init__.py
        ...
    pkg2/
        __init__.py
        ...
    pkg3/
        __init__.py
        ...
    setup.py

I can make an egg with "python setup.py bdist_egg" and it works just fine. I'd like to make it executable with "python myegg.egg" which is supposed to be doable in 2.6+. According to everything I can find, all I need is to add

myegg/
    __main__.py

The end result, however, is "/usr/bin/python: can't find '__main__' module". It is correct: checking the egg with unzip -l shows that __main__.py is indeed not in it. So I played with options for including package data, also made a stab or two at entry_points with no luck.

So, what am I missing. Or does it only work with one package in the egg?

(I'm using 2.7 & 2.6 on centos 7 & 6 resp.)
TIA,


From tjreedy at udel.edu  Thu Jun 18 13:02:16 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 18 Jun 2015 13:02:16 -0400
Subject: ctypes and byte order
In-Reply-To: <1972345947.4721181.1434620395686.JavaMail.root@sequans.com>
References: <818768874.4716193.1434617881522.JavaMail.root@sequans.com>
 <1972345947.4721181.1434620395686.JavaMail.root@sequans.com>
Message-ID: <mlutk5$r49$1@ger.gmane.org>

On 6/18/2015 5:39 AM, Jean-Michel Pichavant wrote:

> I'm currently writing python code that writes a small binary file to
> be used by another device which code is written in C. The python code
> runs on a little endian CPU, and unfortunately, the other device is
> using a big endian MIPS.

The struct module is designed for this.  It reads and writes packed 
binary data of various types and sizes in either big or little endian 
order.  It should be easier than ctypes.

-- 
Terry Jan Reedy



From gilcanmachado at gmail.com  Thu Jun 18 13:57:01 2015
From: gilcanmachado at gmail.com (Gilcan Machado)
Date: Thu, 18 Jun 2015 14:57:01 -0300
Subject: Why this list of dictionaries doesn't work?
Message-ID: <CACUcfw3b1s5fhmFR+6+1Jk5LbeCw7eRxA6e5QyyYitgxOiaJCg@mail.gmail.com>

Hi,

I'm trying to write a list of dictionaries like:

people = (
         {'name':'john', 'age':12} ,
         {'name':'kacey', 'age':18}
    )


I've thought the code below would do the task.

But it doesn't work.

And if I "print(people)" what I get is not the organize data structure like
above.

Thanks of any help!
[]s
Gilcan

#!/usr/bin/env python
from collections import defaultdict

person = defaultdict(dict)
people = list()

person['name'] = 'jose'
person['age'] = 12

people.append(person)

person['name'] = 'kacey'
person['age'] = 18

people.append(person)

for person in people:
    print( person['nome'] )
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150618/8a148e82/attachment.html>

From gherron at digipen.edu  Thu Jun 18 14:27:12 2015
From: gherron at digipen.edu (Gary Herron)
Date: Thu, 18 Jun 2015 11:27:12 -0700
Subject: Why this list of dictionaries doesn't work?
In-Reply-To: <CACUcfw3b1s5fhmFR+6+1Jk5LbeCw7eRxA6e5QyyYitgxOiaJCg@mail.gmail.com>
References: <CACUcfw3b1s5fhmFR+6+1Jk5LbeCw7eRxA6e5QyyYitgxOiaJCg@mail.gmail.com>
Message-ID: <55830D80.3030309@digipen.edu>

On 06/18/2015 10:57 AM, Gilcan Machado wrote:
> Hi,
>
> I'm trying to write a list of dictionaries like:
>
> people = (
>          {'name':'john', 'age':12} ,
>          {'name':'kacey', 'age':18}
>     )
>
>
> I've thought the code below would do the task.
>
> But it doesn't work.

Never say "it doesn't work" on this list.  Tell us what it did, and what 
you expected.  Provide a copy of the full error message. That's much 
more helpful than making us guess.

>
> And if I "print(people)" what I get is not the organize data structure 
> like above.
>
> Thanks of any help!
> []s
> Gilcan
>
> #!/usr/bin/env python
> from collections import defaultdict
>
> person = defaultdict(dict)

If you want *two* different dictionaries, you'll have to create *two* of 
them.  You code creates only this one.

> people = list()
>
> person['name'] = 'jose'
> person['age'] = 12
>
> people.append(person)

Here's where you need to create the second one.

>
> person['name'] = 'kacey'
> person['age'] = 18
>
> people.append(person)
>
> for person in people:
>     print( person['nome'] )

Typo here:  'name', not 'nome'.

>
>
>


-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150618/8b88d538/attachment.html>

From python at mrabarnett.plus.com  Thu Jun 18 14:37:40 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 18 Jun 2015 19:37:40 +0100
Subject: Why this list of dictionaries doesn't work?
In-Reply-To: <CACUcfw3b1s5fhmFR+6+1Jk5LbeCw7eRxA6e5QyyYitgxOiaJCg@mail.gmail.com>
References: <CACUcfw3b1s5fhmFR+6+1Jk5LbeCw7eRxA6e5QyyYitgxOiaJCg@mail.gmail.com>
Message-ID: <55830FF4.7010409@mrabarnett.plus.com>

On 2015-06-18 18:57, Gilcan Machado wrote:
> Hi,
>
> I'm trying to write a list of dictionaries like:
>
> people = (
>           {'name':'john', 'age':12} ,
>           {'name':'kacey', 'age':18}
>      )
>
That's not a list; it's a tuple. If you want a list, use '[' and ']'.
>
> I've thought the code below would do the task.
>
> But it doesn't work.
>
> And if I "print(people)" what I get is not the organize data structure
> like above.
>
> Thanks of any help!
> []s
> Gilcan
>
> #!/usr/bin/env python
> from collections import defaultdict
>
You don't need a defaultdict, just a normal dict.

This creates an empty defaultdict whose default value is a dict:

> person = defaultdict(dict)
> people = list()
>
This puts some items into the dict:

> person['name'] = 'jose'
> person['age'] = 12
>
This puts the dict into the list:

> people.append(person)
>
This _reuses_ the dict and overwrites the items:

> person['name'] = 'kacey'
> person['age'] = 18
>
This puts the dict into the list again:

> people.append(person)
>
The list 'people' now contains 2 references to the _same_ dict.

> for person in people:
>      print( person['nome'] )

Initially there's no such key as 'nome' (not the spelling), so it
creates one with the default value, a dict.

If you print out the people list, you'll see:

[defaultdict(<class 'dict'>, {'name': 'kacey', 'age': 18, 'nome': {}}), 
defaultdict(<class 'dict'>, {'name': 'kacey', 'age': 18, 'nome': {}})]



From ian.g.kelly at gmail.com  Thu Jun 18 14:55:34 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 18 Jun 2015 12:55:34 -0600
Subject: CLI Arguments That Call Functions?
In-Reply-To: <7DDgx.1121732$LW6.214416@fx09.am4>
References: <7DDgx.1121732$LW6.214416@fx09.am4>
Message-ID: <CALwzid=nTDp6pHPPnMC3jOGrm4MosNczqJdDjUP+kaAAfG7WkA@mail.gmail.com>

On Thu, Jun 18, 2015 at 11:56 AM, Tony the Tiger <tony at tiger.invalid> wrote:
> I would have assumed there would be something built in to the
> ArgumentParser, but I can't detect anything that seems to do what I want,
> so I wrote the following:

[SNIP]

> So, is there something already in the Python libs? Do I continue with
> this? Or what?
>
> There ought to be something already built in to Python, but I think I've
> missed it. Sure hope so. I'd hate to continue with this mess.


You can specify custom actions for arguments. See
https://docs.python.org/3.4/library/argparse.html#action-classes

For example (untested):

class AppendCallable(argparse.Action):

    def __call__(self, parser, namespace, values, option_string):
        callables = getattr(namespace, self.dest, None) or []
        callables.append(functools.partial(self.const, *values))
        setattr(namespace, self.dest, callables)

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument(
        '-a', '--alfa',
        action=AppendCallable,
        const=option_alfa,
        dest='callables',
        nargs=1,
        type=int)

    parser.add_argument(
        '-b', '--bravo',
        action=AppendCallable,
        const=option_bravo,
        dest='callables',
        nargs=0)

    args = parser.parse_args()
    for callable in args.callables:
        callable()

You might also be interested in reading
https://docs.python.org/3.4/library/argparse.html#sub-commands in case
that's related to what you're trying to accomplish.


From torriem at gmail.com  Thu Jun 18 15:10:37 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 18 Jun 2015 13:10:37 -0600
Subject: CLI Arguments That Call Functions?
In-Reply-To: <NODgx.1121733$LW6.127114@fx09.am4>
References: <7DDgx.1121732$LW6.214416@fx09.am4>
 <NODgx.1121733$LW6.127114@fx09.am4>
Message-ID: <558317AD.4000500@gmail.com>

On 06/18/2015 12:08 PM, Tony the Tiger wrote:
> Forgot to add, I don't read or see anything posted from outside of the 
> groups. 

Posting from the mailing list here.  I assume the nntp gateway is
two-way.  Unless you're manually blocking message originating in google
groups, I don't see why you wouldn't pick them up (bad formatting and
all) on Usenet.  If not, you're not missing anything, nor are we here
outside of usenet.


From oscar.j.benjamin at gmail.com  Thu Jun 18 15:35:05 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 18 Jun 2015 19:35:05 +0000
Subject: Keypress Input
In-Reply-To: <5580CBC6.3000402@gmail.com>
References: <rBHbx.75089$MO7.40532@fx10.iad> <Ojtfx.178941$Jv.137884@fx26.iad>
 <mailman.519.1434474965.13271.python-list@python.org>
 <Cl%fx.2$WK.0@fx07.iad>
 <mlq25a$188$1@reader1.panix.com> <5580CBC6.3000402@gmail.com>
Message-ID: <CAHVvXxSH5fd2vMLYe4JXD6+jfQDfNjEnVuT_zdKtOZuDx70HLQ@mail.gmail.com>

On Wed, 17 Jun 2015 at 02:23 Michael Torrie <torriem at gmail.com> wrote:

> On 06/16/2015 02:49 PM, Grant Edwards wrote:
> > On 2015-06-16, John McKenzie <davros at bellaliant.net> wrote:
> >
> >> It never occurred to me something so simple as keystrokes would not
> >> be present in Python, a language rated as being terrific by everyone
> >> I know who knows it.
> >
> > Ah, but in reality "keystrokes" is not simple at all.  Keyboards and
> > input handling is a very messy, complicated area.
>
> If you do choose to go with the GPIO route, unless your code for
> accessing the GPIO lines does debouncing, you will have to debounce the
> key.  There are lots of examples out there (most in C on the arduino,
> but still applicable). Most of them check for a button press, then do a
> timer count-down to let things settle out before recording a button
> press.  So it's still complicated even if you talk directly to the
> buttons!  No way around some complexity though.
>

I use the following. I found in testing that when you push the button it
prints 'Button pressed' 10 times a second (in actual use it calls poweroff
so I guess bounce isn't an issue there). Is there some reason it needs to
be cleverer in this case?

 #!/usr/bin/env python

import RPi.GPIO as GPIO
import subprocess
import time

PIN_NUM = 21

GPIO.setmode(GPIO.BCM)

GPIO.setup(PIN_NUM, GPIO.IN, pull_up_down=GPIO.PUD_UP)

while True:
    time.sleep(0.1)
    if not GPIO.input(PIN_NUM):
        print('Button pressed')

--
Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150618/70f36c0e/attachment.html>

From temp.sha at gmail.com  Thu Jun 18 15:41:17 2015
From: temp.sha at gmail.com (ravi)
Date: Thu, 18 Jun 2015 12:41:17 -0700 (PDT)
Subject: help in understanding the stackless code
In-Reply-To: <mailman.596.1434629697.13271.python-list@python.org>
References: <f0d40f43-36ee-4c41-9892-9b5ee524fea3@googlegroups.com>
 <mailman.596.1434629697.13271.python-list@python.org>
Message-ID: <8e34cc5a-9577-4f63-b08a-fe70aa406621@googlegroups.com>

yes It has instance of both Reporter and Switch.
moreover I could not get why instance "reporter" is passed to class Switch 
as a parameter ?

> >      reporter = Reporter()
> >      switch = Switch(0,reporter)
> >      switch(1)


thanks



On Thursday, June 18, 2015 at 5:45:08 PM UTC+5:30, MRAB wrote:
> On 2015-06-18 08:41, ravi wrote:
> > hi,
> > I am new to python and need to know why the calling of switch(1) invokes the function "listen" twice in the below program?
> >
> >
> >
> > import stackless
> >
> > class EventHandler:
> >      def __init__(self,*outputs):
> >          if outputs==None:
> >              self.outputs=[]
> >          else:
> >              self.outputs=list(outputs)
> >
> >          self.channel = stackless.channel()
> >          stackless.tasklet(self.listen)()
> >
> >      def listen(self):
> >          print "in listen()..."
> >          while 1:
> >              val = self.channel.receive()
> >              self.processMessage(val)
> >              for output in self.outputs:
> >                  self.notify(output)
> >
> >      def processMessage(self,val):
> >          pass
> >
> >      def notify(self,output):
> >          pass
> >
> >      def registerOutput(self,output):
> >          print "in registerOutput()..."
> >          self.outputs.append(output)
> >
> >      def __call__(self,val):
> >          print "in __call__ ..."
> >          self.channel.send(val)
> >
> > class Switch(EventHandler):
> >      def __init__(self,initialState=0,*outputs):
> >          EventHandler.__init__(self,*outputs)
> >          self.state = initialState
> >
> >      def processMessage(self,val):
> >          print "in processMessage() of Switch..."
> >          self.state = val
> >
> >      def notify(self,output):
> >          print "in notify() of switch..."
> >          output((self,self.state))
> >
> > class Reporter(EventHandler):
> >      def __init__(self,msg="%(sender)s send message %(value)s"):
> >          EventHandler.__init__(self)
> >          self.msg = msg
> >
> >      def processMessage(self,msg):
> >          print "in processMessage() of Reporter..."
> >          sender,value=msg
> >          print self.msg % {'sender':sender,'value':value}
> >
> >
> > if __name__ == "__main__":
> >      reporter = Reporter()
> >      switch = Switch(0,reporter)
> >      switch(1)
> >
> >
> >
> >
> > output:
> > =========
> >
> > in __call__ ...
> > in listen()...
> > in listen()...
> > in processMessage() of Switch...
> > in notify() of switch...
> > in __call__ ...
> > in processMessage() of Reporter...
> > <__main__.Switch instance at 0x8d822cc> send message 1
> >
> Is it because EventHandler has 2 subclasses, namely Switch and
> Reporter, and you have an instance of each?


From temp.sha at gmail.com  Thu Jun 18 15:47:14 2015
From: temp.sha at gmail.com (ravi)
Date: Thu, 18 Jun 2015 12:47:14 -0700 (PDT)
Subject: python program without stackless.run()
Message-ID: <fc770e8e-a269-4325-b211-efe09327c3ea@googlegroups.com>


Hi,

I could not understand how the below program executes function "fun" without calling stackless.run() in the program?  Here "fun" runs as a tasklet and as per my knowledge for that stackless.run() is must.



-----------------------------------------------------------------
import stackless

class A:
    def __init__(self,name):
        self.name = name
        self.ch = stackless.channel()
        stackless.tasklet(self.fun)()

    def __call__(self,val):
        self.ch.send(val)

    def fun(self):
       while 1:
         v = self.ch.receive()
         print "hi" , v


if __name__ == "__main__":
    obj = A("sh")
    obj(6)
-----------------------------------------------------------------

output:
----------
hi 6





thanks,
ravi


From temp.sha at gmail.com  Thu Jun 18 15:48:36 2015
From: temp.sha at gmail.com (ravi)
Date: Thu, 18 Jun 2015 12:48:36 -0700 (PDT)
Subject: how to dump tasklets status in python
Message-ID: <631357a4-f0f0-4b05-95dd-f991d1c2bd79@googlegroups.com>

hi,

I have a complex python program running 100 tasklets simultaneously. I want to take dump of all the running tasklets including their current status and back trace at the time of exception. Can any one let me know how can this be done ?

Thanks,
ravi


From ian.g.kelly at gmail.com  Thu Jun 18 16:10:10 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 18 Jun 2015 14:10:10 -0600
Subject: python program without stackless.run()
In-Reply-To: <fc770e8e-a269-4325-b211-efe09327c3ea@googlegroups.com>
References: <fc770e8e-a269-4325-b211-efe09327c3ea@googlegroups.com>
Message-ID: <CALwzidnYveSuGwYoLo+pC8btT6M1cfGCc0h7hVQGdvuqn0uu=w@mail.gmail.com>

On Thu, Jun 18, 2015 at 1:47 PM, ravi <temp.sha at gmail.com> wrote:
> I could not understand how the below program executes function "fun" without calling stackless.run() in the program?  Here "fun" runs as a tasklet and as per my knowledge for that stackless.run() is must.

You seem to have a lot of questions about stackless. You might find
that you get a better, more focused response if you ask your questions
on the stackless mailing list:
http://www.stackless.com/mailman/listinfo/stackless


From temp.sha at gmail.com  Thu Jun 18 17:00:37 2015
From: temp.sha at gmail.com (ravi)
Date: Thu, 18 Jun 2015 14:00:37 -0700 (PDT)
Subject: python program without stackless.run()
In-Reply-To: <mailman.612.1434658252.13271.python-list@python.org>
References: <fc770e8e-a269-4325-b211-efe09327c3ea@googlegroups.com>
 <mailman.612.1434658252.13271.python-list@python.org>
Message-ID: <2ab67226-2890-4fef-ae90-21b035f5420d@googlegroups.com>

On Friday, June 19, 2015 at 1:41:36 AM UTC+5:30, Ian wrote:
> On Thu, Jun 18, 2015 at 1:47 PM, ravi <temp.sha at gmail.com> wrote:
> > I could not understand how the below program executes function "fun" without calling stackless.run() in the program?  Here "fun" runs as a tasklet and as per my knowledge for that stackless.run() is must.
> 
> You seem to have a lot of questions about stackless. You might find
> that you get a better, more focused response if you ask your questions
> on the stackless mailing list:
> http://www.stackless.com/mailman/listinfo/stackless

thanks for your pointer. I will post my queries to stackless mailing list.


From nmichalowsky at gmail.com  Thu Jun 18 17:04:50 2015
From: nmichalowsky at gmail.com (Naftali)
Date: Thu, 18 Jun 2015 14:04:50 -0700 (PDT)
Subject: Python File as the Default PDF handler for Windows
Message-ID: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>

Long time lurker. 

I'm looking to register a python script as the default pdf reader for windows. I assume I can just register the .py in the section windows section for registering default handlers, but I'm wondering how to access the file from within the program. 

The issue is this:

We have Java application that outputs user uploaded pdf files. It does this simply by instructing windows to open the downloaded pdf file and windows takes it from there. The data entry person will view the pdf and usually upload it into another part of the system. Problem is the second leg of the system modifies the pdf, and thus crashes when the pdf is protected against writing. Data entry make use of a program to unlock them as needed but it is an extra step and it only comes to their awareness after their client crashes on the locked pdf (because it doesn't make sense to check them proactively.

I cannot change the Java system. 

What I want to do is write a pdf handler to handle windows open instruction. In the script I would run a command line pdf unlocker on the file and open the unlocked file with adobe (or the like). 

I've googled and though I get tons of 'how to open pdf from a python script' I haven't found anything describing how to write and set up my python program to deal with the pdf hand-off from the OS.


From ahlusar.ahluwalia at gmail.com  Thu Jun 18 17:19:57 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Thu, 18 Jun 2015 14:19:57 -0700 (PDT)
Subject: Reassigning keys in dictionary of lists and then writing out to CSV
 file?
Message-ID: <caa771ea-9b7b-4f3d-a0b4-d29c8e52db79@googlegroups.com>

I am currently attempting to work on converting a fairly sizeable JSON object and convert it into a CSV format. However, when I attempt to do so, using a conventional approach (that seems to work with other files). I am presented with a "ValueError: too many values to unpack"

I have tried to flatten the JSON object using this function:

# def flatten(d, parent_key=''):
#     items = []
#     for k, v in d.items():
#         try:
#             items.extend(flatten(v, '%s%s_' % (parent_key, k)).items())
#         except AttributeError:
#             items.append(('%s%s' % (parent_key, k), v))
#     return dict(items)


However this concatenates the keys. I am now attempting to remove the nested key and reassigning to the outer key. 

Here are some samples. I would remove Int32, Double and DateTime. I am wondering if there is a function that would then allow me to assign the new keys as column headers in a CSV and concatenate all of the values within the list (as corresponding fields). I hope that I was clear in my description. Thank you all for your help. 

"FC": {"Int32": ["0","0","0","0","0","0"]} and
                 
"PBA": {"Double": ["0","0","0","0","0","0","0","0"]}

and such examples:

"PBDD": {
						"DateTime": ["1/1/0001 12:00:00 AM",
						"1/1/0001 12:00:00 AM",
						"1/1/0001 12:00:00 AM",
						"1/1/0001 12:00:00 AM",
						"1/1/0001 12:00:00 AM",
						"1/1/0001 12:00:00 AM",
						"1/1/0001 12:00:00 AM",
						"1/1/0001 12:00:00 AM"]
					},


From lac at openend.se  Thu Jun 18 17:38:45 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 18 Jun 2015 23:38:45 +0200
Subject: Documenting a function signature (was: Set a flag on the function
 or a global?)
In-Reply-To: Message from Ben Finney <ben+python@benfinney.id.au>
 of "Thu, 18 Jun 2015 10:04:46 +1000." <85egl9ubj5.fsf_-_@benfinney.id.au>
References: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
 <mailman.530.1434500492.13271.python-list@python.org>
 <55818b3c$0$1665$c3e8da3$5496439d@news.astraweb.com>
 <mailman.556.1434553624.13271.python-list@python.org>
 <55819878$0$1658$c3e8da3$5496439d@news.astraweb.com><85egl9ubj5.fsf_-_@benfinney.id.au>
Message-ID: <201506182138.t5ILcj8Q008705@fido.openend.se>

In a message of Thu, 18 Jun 2015 10:04:46 +1000, Ben Finney writes:
>Since the introduction of keyword-only arguments in Python functions,
>the question arises of how to communicate this in documentation.

I suppose it is way too late to scream "I hate keyword-only arguments"!

>The lone asterisk showing the separation of keyword-only arguments from
>the rest is confusing to me. Not least because it is (if I understand
>correctly) invalid syntax to actually have that in Python code.

Me too.

Laura



From lac at openend.se  Thu Jun 18 17:44:41 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 18 Jun 2015 23:44:41 +0200
Subject: Posting gzip'd image file - server says Malformed Upload?
In-Reply-To: Message from Chris Angelico <rosuav@gmail.com> of "Thu,
 18 Jun 2015 13:56:08 +1000."
 <CAPTjJmrWOAZHX8q6dbiw21kffT1d1VyXYK-kTKHYJzxDuW3U9w@mail.gmail.com>
References: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com>
 <mailman.575.1434587009.13271.python-list@python.org>
 <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
 <mailman.577.1434591971.13271.python-list@python.org>
 <db74b4ee-b88f-4e41-8844-2c80671a6bee@googlegroups.com><CAPTjJmrWOAZHX8q6dbiw21kffT1d1VyXYK-kTKHYJzxDuW3U9w@mail.gmail.com>
Message-ID: <201506182144.t5ILif5n009114@fido.openend.se>

I got to this party late.

One way to get the malformed upload message is is you gzip something
that already is gzipped, and send that up the pipe.

worth checking.

Laura



From lac at openend.se  Thu Jun 18 17:59:09 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 18 Jun 2015 23:59:09 +0200
Subject: =?utf-8?Q?Hotspot_Locater?=
In-Reply-To: Message from <admin.dslcomputer@gmail.com> of "Thu,
 18 Jun 2015 06:52:33 -0000." <55826af6.e281440a.068d.35da@mx.google.com>
References: <55826af6.e281440a.068d.35da@mx.google.com>
Message-ID: <201506182159.t5ILx9AR009888@fido.openend.se>

yes.  wifi https://wifi.readthedocs.org/en/latest/

see this answer, not raspberry pi specific
http://stackoverflow.com/questions/20470626/python-script-for-raspberrypi-to-connect-wifi-automatically

but is linux specific, what OS do you need?

Laura




From lac at openend.se  Thu Jun 18 18:14:28 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 19 Jun 2015 00:14:28 +0200
Subject: help in understanding the stackless code
In-Reply-To: Message from ravi <temp.sha@gmail.com> of "Thu,
 18 Jun 2015 00:41:52 -0700."
 <f0d40f43-36ee-4c41-9892-9b5ee524fea3@googlegroups.com>
References: <f0d40f43-36ee-4c41-9892-9b5ee524fea3@googlegroups.com>
Message-ID: <201506182214.t5IMESZ0010600@fido.openend.se>

You need to send your message over here.
http://www.stackless.com/mailman/listinfo/stackless

I think I know the answer, from my work in duplicating stackless
for greenlets in pypy.  But that's the answer in theory.  In
practice, you need real stackless users.

Laura



From torriem at gmail.com  Thu Jun 18 18:42:14 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 18 Jun 2015 16:42:14 -0600
Subject: Keypress Input
In-Reply-To: <CAHVvXxSH5fd2vMLYe4JXD6+jfQDfNjEnVuT_zdKtOZuDx70HLQ@mail.gmail.com>
References: <rBHbx.75089$MO7.40532@fx10.iad> <Ojtfx.178941$Jv.137884@fx26.iad>
 <mailman.519.1434474965.13271.python-list@python.org> <Cl%fx.2$WK.0@fx07.iad>
 <mlq25a$188$1@reader1.panix.com> <5580CBC6.3000402@gmail.com>
 <CAHVvXxSH5fd2vMLYe4JXD6+jfQDfNjEnVuT_zdKtOZuDx70HLQ@mail.gmail.com>
Message-ID: <55834946.8000202@gmail.com>

On 06/18/2015 01:35 PM, Oscar Benjamin wrote:
> I use the following. I found in testing that when you push the button it
> prints 'Button pressed' 10 times a second (in actual use it calls poweroff
> so I guess bounce isn't an issue there). Is there some reason it needs to
> be cleverer in this case?

Yes, that would be expected, given your code has a while loop that never
exits.  Just curious what you expect the code to do that it's not doing.

You are probably right that debouncing isn't important in your
application.  So just add your poweroff command after the print()
statement, and break out of the loop:

...
while True:
    time.sleep(0.1)
    if not GPIO.input(PIN_NUM):
        print('Button pressed')
        # run shutdown command here
        os.system('/usr/sbin/shutdown')
        break





From lac at openend.se  Thu Jun 18 19:01:54 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 19 Jun 2015 01:01:54 +0200
Subject: Classic OOP in Python
In-Reply-To: Message from Mark Lawrence <breamoreboy@yahoo.co.uk>
 of "Thu, 18 Jun 2015 11:50:28 +0100." <mlu7pl$u1t$1@ger.gmane.org>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net><49f789cf-cb67-4812-8a91-218a4f049958@googlegroups.com>
 <201506172233.t5HMXBIm002277@fido.openend.se><mlu7pl$u1t$1@ger.gmane.org>
Message-ID: <201506182301.t5IN1sdj012524@fido.openend.se>

In a message of Thu, 18 Jun 2015 11:50:28 +0100, Mark Lawrence writes:
>Throw in http://clonedigger.sourceforge.net/ as well and you've a really 
>awesome combination.
>
>Mark Lawrence
>

I didn't know about that one.
Hey thank you, Mark.  Looks great.

It needs its own entry in
https://wiki.python.org/moin/PythonTestingToolsTaxonomy

You add it, or me?

Laura


From rosuav at gmail.com  Thu Jun 18 19:46:10 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 19 Jun 2015 09:46:10 +1000
Subject: Python File as the Default PDF handler for Windows
In-Reply-To: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>
References: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>
Message-ID: <CAPTjJmpwOeHMftc3q672BpBDDLAaYs8xEjascLibN9RUMWiY2Q@mail.gmail.com>

On Fri, Jun 19, 2015 at 7:04 AM, Naftali <nmichalowsky at gmail.com> wrote:
> What I want to do is write a pdf handler to handle windows open instruction. In the script I would run a command line pdf unlocker on the file and open the unlocked file with adobe (or the like).
>
> I've googled and though I get tons of 'how to open pdf from a python script' I haven't found anything describing how to write and set up my python program to deal with the pdf hand-off from the OS.
>

Possibly the reason you've found nothing is that this isn't actually a
Python issue :) What you want to do is tell your OS that when you
double-click on a PDF, it should run this program. That's better
referred to as "file associations". Different versions of Windows put
that in different places, but poke around in your GUI or on the
internet and you should be able to find something on updating
associations.

If you're having trouble getting a Python program to run in that
situation, it may help to explicitly name a Python interpreter - an
executable binary.

ChrisA


From steve at pearwood.info  Thu Jun 18 20:22:35 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 19 Jun 2015 10:22:35 +1000
Subject: Why this list of dictionaries doesn't work?
References: <mailman.606.1434650688.13271.python-list@python.org>
Message-ID: <558360cb$0$1655$c3e8da3$5496439d@news.astraweb.com>

On Fri, 19 Jun 2015 03:57 am, Gilcan Machado wrote:

> Hi,
> 
> I'm trying to write a list of dictionaries like:
> 
> people = (
>          {'name':'john', 'age':12} ,
>          {'name':'kacey', 'age':18}
>     )
> 
> 
> I've thought the code below would do the task.

Why don't you just use exactly what you have above? Just change the round
brackets (parentheses) to square brackets to change people from a tuple to
a list:

people = [{'name': 'john', 'age': 12}, {'name': 'kacey', 'age': 18}]

and you now have a list of two dicts.



> But it doesn't work.

What does it do? Does the computer catch fire? Blue Screen Of Death? Does
Python crash, or start printing "All work and no play makes Jack a dull
boy?" over and over again?

*wink*

You need to explain what you expected to happen and what actually happened,
not just "it doesn't work".


> And if I "print(people)" what I get is not the organize data structure
> like above.

Remember that Python doesn't *pretty-print* lists or dicts by default. If
you print people, you'll get all the information, but it may not be
displayed in what you consider a particularly pleasing or organized manner:

py> people = [{'name': 'john', 'age': 12}, {'name': 'kacey', 'age': 18}]
py> print(people)
[{'age': 12, 'name': 'john'}, {'age': 18, 'name': 'kacey'}]


If you want it displayed as in your original sample, you will need to write
your own print function.


> #!/usr/bin/env python
> from collections import defaultdict
> 
> person = defaultdict(dict)
> people = list()
> 
> person['name'] = 'jose'
> person['age'] = 12
> 
> people.append(person)

This is a little different from what you have above. It uses a defaultdict
instead of a regular dict, I presume you have some good reason for that.


> person['name'] = 'kacey'
> person['age'] = 18

This, however, if where you go wrong. You're not creating a second dict, you
are modifying the existing one. When you modify a dict, you modify it
everywhere it appears. So it doesn't matter whether you look at the
variable person, or if you look at the list people containing that dict,
you see the same value:

    print(person)
    print(people[0])

Both print the same thing, because they are the same dict (not mere copies).

Another way to put it, Python does *not* copy the dict when you insert it
into a list. So whether you look at people or person[0], you are looking at
the same object.

> people.append(person)

And now you append the same dict to the list, so it is in the list twice.
Now you have:

    person
    people[0]
    people[1]


being three different ways to refer to the same dict, not three different
dicts.



-- 
Steven



From ben+python at benfinney.id.au  Thu Jun 18 20:41:02 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 19 Jun 2015 10:41:02 +1000
Subject: Documenting a function signature
References: <557fdbd6$0$11097$c3e8da3@news.astraweb.com>
 <mailman.530.1434500492.13271.python-list@python.org>
 <55818b3c$0$1665$c3e8da3$5496439d@news.astraweb.com>
 <mailman.556.1434553624.13271.python-list@python.org>
 <55819878$0$1658$c3e8da3$5496439d@news.astraweb.com>
 <85egl9ubj5.fsf_-_@benfinney.id.au>
 <201506182138.t5ILcj8Q008705@fido.openend.se>
Message-ID: <85a8vwttr5.fsf@benfinney.id.au>

Laura Creighton <lac at openend.se> writes:

> In a message of Thu, 18 Jun 2015 10:04:46 +1000, Ben Finney writes:
> >Since the introduction of keyword-only arguments in Python functions,
> >the question arises of how to communicate this in documentation.
>
> I suppose it is way too late to scream "I hate keyword-only
> arguments"!

It's never too late for screams, go right ahead.

If you're hoping those screams will result in the keyword-arguments not
being part of Python now and in the future; yes, I think it's too late
for that :-)

-- 
 \      ?If sharing a thing in no way diminishes it, it is not rightly |
  `\      owned if it is not shared.? ?Augustine of Hippo (354?430 CE) |
_o__)                                                                  |
Ben Finney



From nmichalowsky at gmail.com  Thu Jun 18 21:03:14 2015
From: nmichalowsky at gmail.com (Naftali)
Date: Thu, 18 Jun 2015 18:03:14 -0700 (PDT)
Subject: Python File as the Default PDF handler for Windows
In-Reply-To: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>
References: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>
Message-ID: <e97c8320-1085-46e1-b004-3b90e30d2e0b@googlegroups.com>

On Thursday, June 18, 2015 at 5:05:15 PM UTC-4, Naftali wrote:
> Long time lurker. 
> 
> I'm looking to register a python script as the default pdf reader for windows. I assume I can just register the .py in the section windows section for registering default handlers, but I'm wondering how to access the file from within the program. 
> 
> The issue is this:
> 
> We have Java application that outputs user uploaded pdf files. It does this simply by instructing windows to open the downloaded pdf file and windows takes it from there. The data entry person will view the pdf and usually upload it into another part of the system. Problem is the second leg of the system modifies the pdf, and thus crashes when the pdf is protected against writing. Data entry make use of a program to unlock them as needed but it is an extra step and it only comes to their awareness after their client crashes on the locked pdf (because it doesn't make sense to check them proactively.
> 
> I cannot change the Java system. 
> 
> What I want to do is write a pdf handler to handle windows open instruction. In the script I would run a command line pdf unlocker on the file and open the unlocked file with adobe (or the like). 
> 
> I've googled and though I get tons of 'how to open pdf from a python script' I haven't found anything describing how to write and set up my python program to deal with the pdf hand-off from the OS.

Thank you for responding, Chris:

I may be missing something in your reply, but I am *not* wondering how to associate python with the .pdf file extension. That I know how to do. What I want to know is how from within the program that I've told windows to run when the user clicks on the file --- how to access the file that the user wishes to open. Windows must somehow make that available to my program. How to I access it. Tomorrow, I'll try doing it an looping through the sys.argvs and see what's there. maybe it's a simple as passing a path into the called program.


From rosuav at gmail.com  Thu Jun 18 21:18:10 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 19 Jun 2015 11:18:10 +1000
Subject: Python File as the Default PDF handler for Windows
In-Reply-To: <e97c8320-1085-46e1-b004-3b90e30d2e0b@googlegroups.com>
References: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>
 <e97c8320-1085-46e1-b004-3b90e30d2e0b@googlegroups.com>
Message-ID: <CAPTjJmpdh+c29rAHraSfVXKmR2WvM2mijpOQ=aaWP7ZibWxQLg@mail.gmail.com>

On Fri, Jun 19, 2015 at 11:03 AM, Naftali <nmichalowsky at gmail.com> wrote:
> I may be missing something in your reply, but I am *not* wondering how to associate python with the .pdf file extension. That I know how to do. What I want to know is how from within the program that I've told windows to run when the user clicks on the file --- how to access the file that the user wishes to open. Windows must somehow make that available to my program. How to I access it. Tomorrow, I'll try doing it an looping through the sys.argvs and see what's there. maybe it's a simple as passing a path into the called program.
>

Oh! I see what you mean. You should get the full path and name of the
file in sys.argv, but if you don't, I think what you need to do is add
"%s" to the end of the association definition. My apologies for the
prior misunderstanding.

ChrisA


From nmichalowsky at gmail.com  Thu Jun 18 21:22:22 2015
From: nmichalowsky at gmail.com (Naftali)
Date: Thu, 18 Jun 2015 18:22:22 -0700 (PDT)
Subject: Python File as the Default PDF handler for Windows
In-Reply-To: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>
References: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>
Message-ID: <8696771d-2cd2-46f9-9965-088f472426ad@googlegroups.com>

On Thursday, June 18, 2015 at 5:05:15 PM UTC-4, Naftali wrote:
> Long time lurker. 
> 
> I'm looking to register a python script as the default pdf reader for windows. I assume I can just register the .py in the section windows section for registering default handlers, but I'm wondering how to access the file from within the program. 
> 
> The issue is this:
> 
> We have Java application that outputs user uploaded pdf files. It does this simply by instructing windows to open the downloaded pdf file and windows takes it from there. The data entry person will view the pdf and usually upload it into another part of the system. Problem is the second leg of the system modifies the pdf, and thus crashes when the pdf is protected against writing. Data entry make use of a program to unlock them as needed but it is an extra step and it only comes to their awareness after their client crashes on the locked pdf (because it doesn't make sense to check them proactively.
> 
> I cannot change the Java system. 
> 
> What I want to do is write a pdf handler to handle windows open instruction. In the script I would run a command line pdf unlocker on the file and open the unlocked file with adobe (or the like). 
> 
> I've googled and though I get tons of 'how to open pdf from a python script' I haven't found anything describing how to write and set up my python program to deal with the pdf hand-off from the OS.


Thank you, Chris.


From python at mrabarnett.plus.com  Thu Jun 18 21:39:58 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 19 Jun 2015 02:39:58 +0100
Subject: Python File as the Default PDF handler for Windows
In-Reply-To: <CAPTjJmpdh+c29rAHraSfVXKmR2WvM2mijpOQ=aaWP7ZibWxQLg@mail.gmail.com>
References: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>
 <e97c8320-1085-46e1-b004-3b90e30d2e0b@googlegroups.com>
 <CAPTjJmpdh+c29rAHraSfVXKmR2WvM2mijpOQ=aaWP7ZibWxQLg@mail.gmail.com>
Message-ID: <558372EE.9090408@mrabarnett.plus.com>

On 2015-06-19 02:18, Chris Angelico wrote:
> On Fri, Jun 19, 2015 at 11:03 AM, Naftali <nmichalowsky at gmail.com> wrote:
>> I may be missing something in your reply, but I am *not* wondering how to associate python with the .pdf file extension. That I know how to do. What I want to know is how from within the program that I've told windows to run when the user clicks on the file --- how to access the file that the user wishes to open. Windows must somehow make that available to my program. How to I access it. Tomorrow, I'll try doing it an looping through the sys.argvs and see what's there. maybe it's a simple as passing a path into the called program.
>>
>
> Oh! I see what you mean. You should get the full path and name of the
> file in sys.argv, but if you don't, I think what you need to do is add
> "%s" to the end of the association definition. My apologies for the
> prior misunderstanding.
>
I don't think it should be "%s", but "%*".

This has more details:

https://technet.microsoft.com/en-us/library/bb490912.aspx



From rosuav at gmail.com  Thu Jun 18 21:46:29 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 19 Jun 2015 11:46:29 +1000
Subject: Python File as the Default PDF handler for Windows
In-Reply-To: <558372EE.9090408@mrabarnett.plus.com>
References: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>
 <e97c8320-1085-46e1-b004-3b90e30d2e0b@googlegroups.com>
 <CAPTjJmpdh+c29rAHraSfVXKmR2WvM2mijpOQ=aaWP7ZibWxQLg@mail.gmail.com>
 <558372EE.9090408@mrabarnett.plus.com>
Message-ID: <CAPTjJmqMrL5SpSN1Q_3DCi1wiFHJGUDm3GjQ095+HJKUQJ3ajA@mail.gmail.com>

On Fri, Jun 19, 2015 at 11:39 AM, MRAB <python at mrabarnett.plus.com> wrote:
> I don't think it should be "%s", but "%*".

Thanks, it's been a while since I fiddled with Windows associations.

ChrisA


From ahlusar.ahluwalia at gmail.com  Thu Jun 18 21:47:30 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Thu, 18 Jun 2015 18:47:30 -0700 (PDT)
Subject: JSON Object to CSV File Troubleshooting
Message-ID: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>

Good Evening,

I have a conundrum regarding JSON objects and converting them to CSV:

Context

I am converting XML files to a JSON object (please see snippet below) and then finally producing a CSV file. Here is a an example JSON object:

			"PAC": {
				"Account": [{
					"PC": "0",
					"CMC": "0",
					"WC": "0",
					"DLA": "0",
					"CN": null,
					"FC": {
						"Int32": ["0",
						"0",
						"0",
						"0",
						"0"]
					},
					"F": {
						"Description": null,
						"Code": "0"
					}

In general, when I convert any of the files from JSON to CSV, I have been successful when using the following strategy (credit to Peter Otten):


import csv 
import json 
import sys 

def hook(obj): 
    return obj 

def flatten(obj): 
    for k, v in obj: 
        if isinstance(v, list): 
            yield from flatten(v) 
        else: 
            yield k, v 

if __name__ == "__main__": 
    with open("somefileneame.json") as f: 
        data = json.load(f, object_pairs_hook=hook) 

    pairs = list(flatten(data)) 

    writer = csv.writer(sys.stdout) 
    header = writer.writerow([k for k, v in pairs]) 
    row = writer.writerow([v for k, v in pairs]) #writer.writerows for any other iterable object


However with the example JSON object (above) i receive the following error when applying this function:

ValueError: too many values to unpack 


Here are some more samples. 

"FC": {"Int32": ["0","0","0","0","0","0"]} 
"PBA": {"Double": ["0","0","0","0","0","0","0","0"]} 

3.          "PBDD": { 
                                                "DateTime": ["1/1/0001 12:00:00 AM", 
                                                "1/1/0001 12:00:00 AM", 
                                                "1/1/0001 12:00:00 AM", 
                                                "1/1/0001 12:00:00 AM", 
                                                "1/1/0001 12:00:00 AM", 
                                                "1/1/0001 12:00:00 AM", 
                                                "1/1/0001 12:00:00 AM", 
                                                "1/1/0001 12:00:00 AM"] 
                                        }, 



In the above example, I would like to remove the keys Int32, Double and DateTime. I am wondering if there is a function or methodology that would allow me to remove such nested keys and reassign the new keys to the outer key (in this case above FC, PBA and PBDD) as column headers in a CSV and concatenate all of the values within the list (as corresponding fields). 

Also, here is how I strategized my XML to CSV conversion (if this is of any use):


import xml.etree.cElementTree as ElementTree
from xml.etree.ElementTree import XMLParser
import json 
import csv
import tokenize
import token
try:
    from collections import OrderedDict
    import json
except ImportError:
    from ordereddict import OrderedDict
    import simplejson as json
import itertools
import six
import string
from csvkit import CSVKitWriter


class XmlListConfig(list):
    def __init__(self, aList):
        for element in aList:
            if element:
                # treat like dict
                if len(element) == 1 or element[0].tag != element[1].tag:
                    self.append(XmlDictConfig(element))
                # treat like list
                elif element[0].tag == element[1].tag:
                    self.append(XmlListConfig(element))
            elif element.text:
                text = element.text.strip()
                if text:
                    self.append(text)


class XmlDictConfig(dict):
    '''
    Example usage:

    >>> tree = ElementTree.parse('your_file.xml')
    >>> root = tree.getroot()
    >>> xmldict = XmlDictConfig(root)

    Or, if you want to use an XML string:

    >>> root = ElementTree.XML(xml_string)
    >>> xmldict = XmlDictConfig(root)

    And then use xmldict for what it is..a dictionary.
    '''
    def __init__(self, parent_element):
        if parent_element.items():
            self.update(dict(parent_element.items()))
        for element in parent_element:
            if element:
                # treat like dict - we assume that if the first two tags
                # in a series are different, then they are all different.
                if len(element) == 1 or element[0].tag != element[1].tag:
                    aDict = XmlDictConfig(element)
                # treat like list - we assume that if the first two tags
                # in a series are the same, then the rest are the same.
                else:
                    # here, we put the list in dictionary; the key is the
                    # tag name the list elements all share in common, and
                    # the value is the list itself 
                    aDict = {element[0].tag: XmlListConfig(element)}
                # if the tag has attributes, add those to the dict
                if element.items():
                    aDict.update(dict(element.items()))
                self.update({element.tag: aDict})
            # this assumes that if you've got an attribute in a tag,
            # you won't be having any text. This may or may not be a 
            # good idea -- time will tell. It works for the way we are
            # currently doing XML configuration files...
            elif element.items():
                self.update({element.tag: dict(element.items())})
            # finally, if there are no child tags and no attributes, extract
            # the text
            else:
                self.update({element.tag: element.text})



def main():

    #Lines 88-89stantiate the class Elementree
    #and applies the method to recursively traverse from the root node
    #XmlDictConfig is instantiated in line 90

    with open('C:\\Users\\wynsa2\\Desktop\\Python Folder\\PCSU\\Trial2_PCSU\\2-Response.xml', 'r', encoding='utf-8') as f:
        xml_string = f.read()
    xml_string= xml_string.replace('&#x0;', '')
    root = ElementTree.XML(xml_string)
    xmldict = XmlDictConfig(root)
    json_str = json.dumps(xmldict, sort_keys=True, indent=4, separators=(',', ': '))
    newly_formatted_data = json.loads(json_str) #encode into JSON 
    with open('data2.json', 'w') as f:  #writing JSON file
        json.dump(newly_formatted_data, f) 



I hope that I was clear in my description. Thank you all for your help. 

Sincerely,
Saran


From auriocus at gmx.de  Fri Jun 19 01:20:03 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Fri, 19 Jun 2015 07:20:03 +0200
Subject: Keypress Input
In-Reply-To: <Tbtfx.178924$Jv.102633@fx26.iad>
References: <rBHbx.75089$MO7.40532@fx10.iad> <Tbtfx.178924$Jv.102633@fx26.iad>
Message-ID: <mm08nk$caf$1@dont-email.me>

Am 15.06.15 um 07:15 schrieb John McKenzie:
>
> from Tkinter import *
> from blinkstick import blinkstick
>
> led = blinkstick.find_first()
>
> timered = 0
> timeyellow = 0
> timeblue = 0
>
> colour = None
>
> root = Tk()
> root.title('eFlag 1')
>
>
>
> def red1(event):
>      colour = 1
>
> def yellow1(event):
>      colour = 2
>
> def blue1(event):
>      colour = 3
>
> root.bind_all('r', red1)
> root.bind_all('b', blue1)
> root.bind_all('y', yellow1)

The nonsense starts here:

===================
> root.mainloop()
>
> while colour == None:
>      led.pulse(red=0, green=255, blue=0, repeats=1, duration=5000,
> steps=50)
>
> while colour == 1:
>      led.pulse(red=255, green=0, blue=0, repeats=1, duration=3000,
> steps=50)
>      timered += 1
>
> while colour == 2:
>      led.pulse(red=255, green=255, blue=0, repeats=1, duration=3000,
> steps=50)
>      timeyellow += 1
>
> while colour == 3:
>      led.pulse(red=0, green=0, blue=255, repeats=1, duration=2000,
> steps=50)
>      timeblue += 1
====================

it seems you don't understand event based programming. root.mainloop() 
never exits. It waits for the user input and does the dispatching, i.e. 
when a key is pressed, then according to your bindings, the functions 
red1, yellow1, blue1 are called, which set a variable but do not do 
nything else. To see that, just insert a print statement into these 
functions:

  def red1(event):
       colour = 1
       print("Red ws called")


Now your job is to also do the functionality there, i.e. you have to 
reformulate your task (waiting for red, then blue...) as a state 
machine. Alternatively you can circumvent to redo the logic in a state 
machine by using a coroutine.

You should read a text about GUI programming, or more specifically event 
based programming, to understand your mistake.

	Christian


>
> def exit_handler():
>      print '\033[0;41;37mRed Team:\033[0m ', timered
>      print '\033[0;43;30mYellow Time:\033[0m ', timeyellow
>      print '\033[0;44;37mBlue Time:\033[0m ', timeblue
>      flog = open('flag1log.text', 'a')
>      flog.write(timestamp + '\n' + 'Red Team: ' + str(timered) + '\n' +
> 'Yellow Team: ' + str(timeyellow) + '\n' + 'Blue Team: ' + str(timeblue)
> + '\n')
>      flog.close()
>      atexit.register(exit_handler)
>
>



From hayesstw at telkomsa.net  Fri Jun 19 01:21:04 2015
From: hayesstw at telkomsa.net (Steve Hayes)
Date: Fri, 19 Jun 2015 07:21:04 +0200
Subject: JSON Object to CSV Question
References: <mailman.581.1434610734.13271.python-list@python.org>
Message-ID: <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>

On Wed, 17 Jun 2015 17:49:35 -0400, Saran Ahluwalia
<ahlusar.ahluwalia at gmail.com> wrote:

>Good Evening Everyone:
>
>I would like to have this JSON object written out to a CSV file so that the

You've already said that in another thread, and got several answers.
What are you? Some kind of troll?


-- 
Steve Hayes from Tshwane, South Africa
Web:  http://www.khanya.org.za/stevesig.htm
Blog: http://khanya.wordpress.com
E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk


From hayesstw at telkomsa.net  Fri Jun 19 01:23:07 2015
From: hayesstw at telkomsa.net (Steve Hayes)
Date: Fri, 19 Jun 2015 07:23:07 +0200
Subject: JSON Object to CSV File Troubleshooting
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
Message-ID: <gm97oa94ubm30c7euo3d42p60eae87utc6@4ax.com>

On Thu, 18 Jun 2015 18:47:30 -0700 (PDT), Sahlusar
<ahlusar.ahluwalia at gmail.com> wrote:

>Good Evening,
>
>I have a conundrum regarding JSON objects and converting them to CSV:

That's the THIRD time you've asked this, in three separate threads.

Why don't you read the answers you were given the first time?

[follow-ups set]


-- 
Steve Hayes from Tshwane, South Africa
Web:  http://www.khanya.org.za/stevesig.htm
Blog: http://khanya.wordpress.com
E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk


From alex at duboaa.net  Fri Jun 19 03:01:32 2015
From: alex at duboaa.net (Alexis Dubois)
Date: Fri, 19 Jun 2015 00:01:32 -0700 (PDT)
Subject: [Solved] Python.exe has stopped working
In-Reply-To: <mailman.216.1433590803.13271.python-list@python.org>
References: <8a6ed08d-7dd6-4a4a-a466-9d3ad98f8dff@googlegroups.com>
 <c5370bc0-00ee-49be-9ecd-2e25008a78c5@googlegroups.com><mkrp99$b6d$1@dont-email.me>
 <mailman.216.1433590803.13271.python-list@python.org>
Message-ID: <ea13e85b-bb82-4cb2-99d7-5296ebce733c@googlegroups.com>

Le samedi 6 juin 2015 13:40:13 UTC+2, Laura Creighton a ?crit?:
> In a message of Fri, 05 Jun 2015 11:15:31 +0200, Christian Gollwitzer writes:
> >Am 05.06.15 um 11:03 schrieb Alexis Dubois:
> >> Anyone else for an idea on that?
> >>
> >Well, it is a crash on exit. Looks like a memory error inside of PyQT. 
> >If you've got the time, you could run it inside of a debugger, or 
> >better, a memory checker like AppVerifier to find the culprit. These 
> >things are usually quite hard to diagnose, and unless someone has seen 
> >it here already, the fun starts now ;) Another possible reason might be 
> >mixing up DLLs from PyQT and another QT installation. You can find out 
> >by listing all loaded DLLs when the program is running, e.g. from a 
> >debugger or this tool: 
> >https://technet.microsoft.com/sysinternals/bb896653.aspx
> >
> >	Christian
> >-- 
> >https://mail.python.org/mailman/listinfo/python-list
> 
> I don't have a windows machine, so I cannot do any more looking for this,
> but this may be relevant.
> http://stackoverflow.com/questions/20102419/pyqt-5-error-on-exit
> 
> Laura

Hello Everybody,
Thank you for your support and sorry for the late answer, I'm back from holidays.
I tried each solution but unfortunately, nothing works.
Finally, I solved my issue by adding "sip.setdestroyonexit(False)" just before "sys.exit(app.exec_())"
I don't really know what it does but it works well and I don't see any beside effect.
It seems that my issue was linked with a QtGui4.dll crash.


From info at egenix.com  Fri Jun 19 04:05:02 2015
From: info at egenix.com (eGenix Team: M.-A. Lemburg)
Date: Fri, 19 Jun 2015 10:05:02 +0200
Subject: ANN: eGenix Talks & Videos: Python Keynote and Application Design Talk
Message-ID: <5583CD2E.1020309@egenix.com>

________________________________________________________________________

ANNOUNCING

                        eGenix Talks & Videos:

              Python Keynote and Application Design Talk

                          PyWaw Summit 2015

This announcement is also available on our web-site for online reading:
http://www.egenix.com/company/news/PyWaw-Summit-2015-Keynote-and-Application-Design.html

________________________________________________________________________

CONFERENCE: PyWaw Summit 2015 in Warsaw, Poland

Marc-Andr? Lemburg, Python Core Developer and Senior Software
Architect, was invited to give a keynote and a technical talk at the
first PyWaw Summit 2015 in Warsaw, Poland.

The event was a big success, thanks to great organization of the PyWaw
team, and we can only recommend anyone looking for two very intense
days of talks to consider attending next year's summit.

We have now turned the talks into a video presentations for easy
viewing and also released the presentation slides.

________________________________________________________________________

KEYNOTE: Python Community - Present, Past and Future

 This keynote was held at the first PyWaw Summit 2015 in Warsaw,
 Poland, giving a personal view of what makes the Python community so
 special.

 Talk video and slides:
 http://www.egenix.com/library/presentations/PyWaw-Summit-2015-Keynote-Python-Community-Past-Present-Future/

In this keynote, I take the audience on a tour through the development
of the Python community. Starting 20 years ago with how I entered
Python, looking at important milestones, showing current developments
and giving an outlook of where we might be heading in the next few
years.

Hope you'll enjoy the ride.

-- Marc-Andr? Lemburg

________________________________________________________________________

TALK: Designing Large Scale Applications in Python

 This talk was given at the first PyWaw Summit 2015 in Warsaw, Poland,
 demonstrating a guided and structured approach to Python application
 design.

 Talk video and slides:
 http://www.egenix.com/library/presentations/PyWaw-Summit-2015-Designing-Large-Scale-Applications-in-Python/

Python is often referred to as a scripting language. While Python is
an ideal platform for scripting, integration or plugin tasks, it does
in fact cover all the concepts needed for truly large-scale object
oriented application development. However, complex applications bring
new challenges.

The talk draws on my experience with large-scale application design
using Python as central implementation language and provides a
cookbook approach to many of the problems you face when designing and
organizing complex application frameworks.

The approach has been used in many real-life projects, ranging from
web application servers in varying fields of application to complete
finance trading systems. Python has always proven to be a truly good
choice with an outstanding time-to-market advantage over other
implementation languages such as Java or C++. It allows small software
development companies to successfully compete against multi-million
dollar ventures.

-- Marc-Andr? Lemburg

________________________________________________________________________

PYTHON COACHING AND CONSULTING

If you are interested in learning more about these advanced
techniques, eGenix now offers Python project coaching and consulting
services to give your project teams advice on how to design Python
applications, successfully run projects, or find excellent Python
programmers. Please contact our eGenix Sales Team for information.

    http://www.egenix.com/services/

More interesting eGenix presentations are available in the
presentations and talks community section of our website.

    http://www.egenix.com/

________________________________________________________________________

INFORMATION

About Python (http://www.python.org/):

    Python is an object-oriented Open Source programming language
    which runs on all modern platforms. By integrating ease-of-use,
    clarity in coding, enterprise application connectivity and rapid
    application design, Python establishes an ideal programming
    platform for today's IT challenges.

About eGenix (http://www.egenix.com/):

    eGenix is a software project, consulting and product company
    focusing on expert project services and professional quality
    products for companies, Python users and developers.

Enjoy,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 19 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2015-06-16: Released eGenix pyOpenSSL 0.13.10 ... http://egenix.com/go78
2015-06-10: Released mxODBC Plone/Zope DA 2.2.2   http://egenix.com/go76
2015-07-20: EuroPython 2015, Bilbao, Spain ...             31 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From robin at reportlab.com  Fri Jun 19 05:29:14 2015
From: robin at reportlab.com (Robin Becker)
Date: Fri, 19 Jun 2015 10:29:14 +0100
Subject: instance as module
Message-ID: <5583E0EA.2090309@chamonix.reportlab.co.uk>

I'm trying to overcome a recursive import issue in reportlab.

Module reportlab.rl_config uses various sources (eg ~/.reportlab_settings) to 
initialize various defaults eg canvas_basefontname. If a user wants to utilize 
reportlab to set up such a default, it's not possible to do so in the settings 
file because a recursive import would occur. Normal defaults are always 
primitive python objects eg float/int/str etc etc.

If I make the rl_config module into an object (using the GvR blessed approach 
indicated here http://stackoverflow.com/questions/2447353/getattr-on-a-module) 
then I can use callables in the settings module and get them evaluated lazily 
which overcomes the recursion.

Are there any gotcha's that need to be considered when using this instance as 
module approach? My trial implementation seems to work, but it's not clear 
exactly how a module differs from an instance. I have set various dunders on the 
instance eg __file__, __doc__, __all__ & __name__ and I made the object a borg, 
but it still seems a bit hacky.
-- 
Robin Becker



From ahlusar.ahluwalia at gmail.com  Fri Jun 19 05:50:45 2015
From: ahlusar.ahluwalia at gmail.com (Saran Ahluwalia)
Date: Fri, 19 Jun 2015 05:50:45 -0400
Subject: JSON Object to CSV Question
In-Reply-To: <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
Message-ID: <EB18A320-98CB-4873-A90C-E7B3B2B9A3BC@gmail.com>

 Steve, 

No I am not a troll. I am attempting to clarify an evolving problem. If you read the most recent thread that I just posted it states far more information - in my humble opinion.
Peter was of great assistance. The data becomes more convoluted; as a fairly new programmer I am looking for best practice.

Finally, although this is virtual space, it is important to maintain etiquette. I would appreciate it if you would maintain civility and neutrality in discourse.

Saran

Sent from my iPhone

> On Jun 19, 2015, at 1:21 AM, Steve Hayes <hayesstw at telkomsa.net> wrote:
> 
> On Wed, 17 Jun 2015 17:49:35 -0400, Saran Ahluwalia
> <ahlusar.ahluwalia at gmail.com> wrote:
> 
>> Good Evening Everyone:
>> 
>> I would like to have this JSON object written out to a CSV file so that the
> 
> You've already said that in another thread, and got several answers.
> What are you? Some kind of troll?
> 
> 
> -- 
> Steve Hayes from Tshwane, South Africa
> Web:  http://www.khanya.org.za/stevesig.htm
> Blog: http://khanya.wordpress.com
> E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From jeanmichel at sequans.com  Fri Jun 19 05:56:19 2015
From: jeanmichel at sequans.com (Jean-Michel Pichavant)
Date: Fri, 19 Jun 2015 11:56:19 +0200 (CEST)
Subject: ctypes and byte order
In-Reply-To: <mlutk5$r49$1@ger.gmane.org>
Message-ID: <1400510579.4904105.1434707779524.JavaMail.root@sequans.com>

----- Original Message -----
> From: "Terry Reedy" <tjreedy at udel.edu>
> To: python-list at python.org
> Sent: Thursday, 18 June, 2015 7:02:16 PM
> Subject: Re: ctypes and byte order
> 
> On 6/18/2015 5:39 AM, Jean-Michel Pichavant wrote:
> 
> > I'm currently writing python code that writes a small binary file
> > to
> > be used by another device which code is written in C. The python
> > code
> > runs on a little endian CPU, and unfortunately, the other device is
> > using a big endian MIPS.
> 
> The struct module is designed for this.  It reads and writes packed
> binary data of various types and sizes in either big or little endian
> order.  It should be easier than ctypes.
> 
> --
> Terry Jan Reedy

Yep, I knew about struct but I did go for ctypes because... I wanted to try it and thought it would be a suitable solution (with more capabilities).
Looks like I fooled myself (not in the way ctypes is badly designed, it's just not what I needed).

Now I'm left with a dilemma:

My current ctypes code is working (thanks Peter !), but the code is rather cumbersome.

1/ don't change it, and wish good luck to my next co-worker looking at the code
2/ rewrite everything with struct
3/ rewrite everything with cffi (Laura's suggestion) which has the awsomeness to suppport C code copy/paste. It's still a "C Foreign Function Interface" like ctypes and could be overkill


We all know how this ends up...

JM


-- IMPORTANT NOTICE: 

The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

From Cecil at decebal.nl  Fri Jun 19 06:07:13 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 19 Jun 2015 12:07:13 +0200
Subject: JSON Object to CSV Question
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
Message-ID: <87bngcggfi.fsf@Equus.decebal.nl>

Top posting is frowned upon. Could you put your reply under where you
reply on next time?

On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:

> No I am not a troll. I am attempting to clarify an evolving problem.
> If you read the most recent thread that I just posted it states far
> more information - in my humble opinion. Peter was of great

You should put more thought about how your postings will look for the
receiver. I also saw those mails and was wondering why you kept
posting the same question. You was not, but you seemed to. Besides
that it is annoying for the users of the list, you lower the change of
getting a response. When I think I saw the question already, I get
annoyed, do not read it, so I will not respond.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From __peter__ at web.de  Fri Jun 19 06:23:09 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 19 Jun 2015 12:23:09 +0200
Subject: instance as module
References: <5583E0EA.2090309@chamonix.reportlab.co.uk>
Message-ID: <mm0qig$a4d$1@ger.gmane.org>

Robin Becker wrote:

> I'm trying to overcome a recursive import issue in reportlab.
> 
> Module reportlab.rl_config uses various sources (eg ~/.reportlab_settings)
> to initialize various defaults eg canvas_basefontname. If a user wants to
> utilize reportlab to set up such a default, it's not possible to do so in
> the settings file because a recursive import would occur. Normal defaults
> are always primitive python objects eg float/int/str etc etc.
> 
> If I make the rl_config module into an object (using the GvR blessed
> approach indicated here
> http://stackoverflow.com/questions/2447353/getattr-on-a-module) then I can
> use callables in the settings module and get them evaluated lazily which
> overcomes the recursion.
> 
> Are there any gotcha's that need to be considered when using this instance
> as module approach? My trial implementation seems to work, but it's not
> clear exactly how a module differs from an instance. I have set various
> dunders on the instance eg __file__, __doc__, __all__ & __name__ and I
> made the object a borg, but it still seems a bit hacky.

Do I understand this correctly? You got bitten by a complex setup and now 
you are hoping to improve the situation by making it even more complex?

How about reordering initialisation in such a way that the user defaults are 
the last thing being set?



From ahlusar.ahluwalia at gmail.com  Fri Jun 19 06:24:10 2015
From: ahlusar.ahluwalia at gmail.com (Saran Ahluwalia)
Date: Fri, 19 Jun 2015 06:24:10 -0400
Subject: JSON Object to CSV Question
In-Reply-To: <87bngcggfi.fsf@Equus.decebal.nl>
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
 <87bngcggfi.fsf@Equus.decebal.nl>
Message-ID: <ACF6A352-0711-4F9E-85B6-2938C91848C2@gmail.com>

Thank you for clarifying. I will be more cognizant to follow this protocol. 

Sent from my iPhone

> On Jun 19, 2015, at 6:07 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> 
> Top posting is frowned upon. Could you put your reply under where you
> reply on next time?
> 
>> On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:
>> 
>> No I am not a troll. I am attempting to clarify an evolving problem.
>> If you read the most recent thread that I just posted it states far
>> more information - in my humble opinion. Peter was of great
> 
> You should put more thought about how your postings will look for the
> receiver. I also saw those mails and was wondering why you kept
> posting the same question. You was not, but you seemed to. Besides
> that it is annoying for the users of the list, you lower the change of
> getting a response. When I think I saw the question already, I get
> annoyed, do not read it, so I will not respond.
> 
> -- 
> Cecil Westerhof
> Senior Software Engineer
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From robin at reportlab.com  Fri Jun 19 07:07:34 2015
From: robin at reportlab.com (Robin Becker)
Date: Fri, 19 Jun 2015 12:07:34 +0100
Subject: instance as module
In-Reply-To: <mm0qig$a4d$1@ger.gmane.org>
References: <5583E0EA.2090309@chamonix.reportlab.co.uk>
 <mm0qig$a4d$1@ger.gmane.org>
Message-ID: <5583F7F6.9000500@chamonix.reportlab.co.uk>

On 19/06/2015 11:23, Peter Otten wrote:
> Robin Becker wrote:
.........
>
> Do I understand this correctly? You got bitten by a complex setup and now
> you are hoping to improve the situation by making it even more complex?
>
> How about reordering initialisation in such a way that the user defaults are
> the last thing being set?
>

I don't think re-ordering is feasible or desirable.

For the specific case of the canvas_basefontname I don't actually need to do 
anything specific since it is just a string, however, before it can be used in 
action support has to be provided ie I must register the font used. I could just 
make the few usages of this value check for callability at use time, but that 
would scatter the problem; if one default is special why not all.

Effectively the defaults setup is not complex enough to allow use of itself; 
that must be a fairly common problem.

Probably the correct thing to do was always to use an object to hold the 
defaults. For a long time a module satisfied, then people wanted long running 
processes, reset etc etc now they want to define things ahead of time (probably 
for good reason).

I could just move the rl_config module into reportlab's __init__ as an object, 
but I'm almost sure that would break someone's

    from reportlab.rl_config import *

which although deprecated is still allowed. Defining a module like object seems 
a reasonable way out if I really need it.
--
Robin Becker



From ahlusar.ahluwalia at gmail.com  Fri Jun 19 08:47:09 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Fri, 19 Jun 2015 05:47:09 -0700 (PDT)
Subject: JSON Object to CSV Question
In-Reply-To: <87bngcggfi.fsf@Equus.decebal.nl>
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
 <87bngcggfi.fsf@Equus.decebal.nl>
Message-ID: <737437b9-e0c7-401d-9bc4-29b5fdf39e84@googlegroups.com>

On Friday, June 19, 2015 at 6:16:40 AM UTC-4, Cecil Westerhof wrote:
> Top posting is frowned upon. Could you put your reply under where you
> reply on next time?
> 
> On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:
> 
> > No I am not a troll. I am attempting to clarify an evolving problem.
> > If you read the most recent thread that I just posted it states far
> > more information - in my humble opinion. Peter was of great
> 
> You should put more thought about how your postings will look for the
> receiver. I also saw those mails and was wondering why you kept
> posting the same question. You was not, but you seemed to. Besides
> that it is annoying for the users of the list, you lower the change of
> getting a response. When I think I saw the question already, I get
> annoyed, do not read it, so I will not respond.
> 
> -- 
> Cecil Westerhof
> Senior Software Engineer
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof

On another note, are there any suggestions for how to address this problem. I should elaborate that I have recently switched careers in to development (formally an educator). I am seeking guidance for best practice. Thank you for your help everyone. 


From ahlusar.ahluwalia at gmail.com  Fri Jun 19 09:44:00 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Fri, 19 Jun 2015 06:44:00 -0700 (PDT)
Subject: JSON Object to CSV Question
In-Reply-To: <737437b9-e0c7-401d-9bc4-29b5fdf39e84@googlegroups.com>
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
 <87bngcggfi.fsf@Equus.decebal.nl>
 <737437b9-e0c7-401d-9bc4-29b5fdf39e84@googlegroups.com>
Message-ID: <80a79fbd-6531-4ab9-a363-bd9dcff37a30@googlegroups.com>

On Friday, June 19, 2015 at 8:47:25 AM UTC-4, Sahlusar wrote:
> On Friday, June 19, 2015 at 6:16:40 AM UTC-4, Cecil Westerhof wrote:
> > Top posting is frowned upon. Could you put your reply under where you
> > reply on next time?
> > 
> > On Friday 19 Jun 2015 11:50 CEST, Saran Ahluwalia wrote:
> > 
> > > No I am not a troll. I am attempting to clarify an evolving problem.
> > > If you read the most recent thread that I just posted it states far
> > > more information - in my humble opinion. Peter was of great
> > 
> > You should put more thought about how your postings will look for the
> > receiver. I also saw those mails and was wondering why you kept
> > posting the same question. You was not, but you seemed to. Besides
> > that it is annoying for the users of the list, you lower the change of
> > getting a response. When I think I saw the question already, I get
> > annoyed, do not read it, so I will not respond.
> > 
> > -- 
> > Cecil Westerhof
> > Senior Software Engineer
> > LinkedIn: http://www.linkedin.com/in/cecilwesterhof
> 
> On another note, are there any suggestions for how to address this problem. I should elaborate that I have recently switched careers in to development (formally an educator). I am seeking guidance for best practice. Thank you for your help everyone.

Here is a more (I hope focused question):

http://stackoverflow.com/questions/30938462/using-python-to-convert-json-objects-in-file-to-csv-2-parts.



From fabien.maussion at gmail.com  Fri Jun 19 10:01:02 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Fri, 19 Jun 2015 16:01:02 +0200
Subject: Catching exceptions with multi-processing
Message-ID: <mm17as$i8m$1@speranza.aioe.org>

Folks,

I am developing a tool which works on individual entities (glaciers) and 
do a lot of operations on them. There are many tasks to do, one after 
each other, and each task follows the same interface:

def task_1(path_to_glacier_dir):
     open file1 in path_to_glacier_dir
     do stuff
     if dont_work:
         raise RuntimeError("didnt work")
     write file2 in path_to_glacier_dir

This way, the tasks can be run in parallel very easily:

import multiprocessing as mp
pool = mp.Pool(4)

dirs = [list_of_dirs]
pool.map(task1, dirs, chunksize=1)
pool.map(task2, dirs, chunksize=1)
pool.map(task3, dirs, chunksize=1)

... and so forth. I tested the tool for about a hundred glaciers but now 
it has to run for thousands of them. There are going to be errors, some 
of them are even expected for special outliers. What I would like the 
tool to do is that in case of error, it writes the identifier of the 
problematic glacier somewhere, the error encountered and more info if 
possible. Because of multiprocessing, I can't write in a shared file, so 
I thought that the individual processes should write a unique "error 
file" in a dedicated directory.

What I don't know how to, however, is how to do this at minimal cost and 
in a generic way for all tasks. Also, the task2 should not be run if 
task1 threw an error. Sometimes (for debugging), I'd rather keep the 
normal behavior of raising an error and stopping the program.

Do I have to wrap all tasks with a "try: exept:" block? How to switch 
between behaviors? All the solutions I could think about look quite ugly 
to me. And it seems that this is a general problem that someone cleverer 
than me had solved before ;-)

Thanks,

Fabien









From andres.riancho at gmail.com  Fri Jun 19 10:25:29 2015
From: andres.riancho at gmail.com (Andres Riancho)
Date: Fri, 19 Jun 2015 11:25:29 -0300
Subject: Catching exceptions with multi-processing
In-Reply-To: <mm17as$i8m$1@speranza.aioe.org>
References: <mm17as$i8m$1@speranza.aioe.org>
Message-ID: <CA+1Rt64eS33bn8siimxAF=KMHOmhXLMv+KGqEPWM630WTL6KZg@mail.gmail.com>

Fabien,

   My recommendation is that you should pass some extra arguments to the task:
    * A unique task id
    * A result multiprocessing.Queue

    When an exception is raised you put (unique_id, exception) to the
queue. When it succeeds you put (unique_id, None). In the main process
you consume the queue and do your error handling.

    Note that some exceptions can't be serialized, there is where
tblib [0] comes handy.

[0] https://pypi.python.org/pypi/tblib

Regards,

On Fri, Jun 19, 2015 at 11:01 AM, Fabien <fabien.maussion at gmail.com> wrote:
> Folks,
>
> I am developing a tool which works on individual entities (glaciers) and do
> a lot of operations on them. There are many tasks to do, one after each
> other, and each task follows the same interface:
>
> def task_1(path_to_glacier_dir):
>     open file1 in path_to_glacier_dir
>     do stuff
>     if dont_work:
>         raise RuntimeError("didnt work")
>     write file2 in path_to_glacier_dir
>
> This way, the tasks can be run in parallel very easily:
>
> import multiprocessing as mp
> pool = mp.Pool(4)
>
> dirs = [list_of_dirs]
> pool.map(task1, dirs, chunksize=1)
> pool.map(task2, dirs, chunksize=1)
> pool.map(task3, dirs, chunksize=1)
>
> ... and so forth. I tested the tool for about a hundred glaciers but now it
> has to run for thousands of them. There are going to be errors, some of them
> are even expected for special outliers. What I would like the tool to do is
> that in case of error, it writes the identifier of the problematic glacier
> somewhere, the error encountered and more info if possible. Because of
> multiprocessing, I can't write in a shared file, so I thought that the
> individual processes should write a unique "error file" in a dedicated
> directory.
>
> What I don't know how to, however, is how to do this at minimal cost and in
> a generic way for all tasks. Also, the task2 should not be run if task1
> threw an error. Sometimes (for debugging), I'd rather keep the normal
> behavior of raising an error and stopping the program.
>
> Do I have to wrap all tasks with a "try: exept:" block? How to switch
> between behaviors? All the solutions I could think about look quite ugly to
> me. And it seems that this is a general problem that someone cleverer than
> me had solved before ;-)
>
> Thanks,
>
> Fabien
>
>
>
>
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Andr?s Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3


From jeanmichel at sequans.com  Fri Jun 19 10:31:05 2015
From: jeanmichel at sequans.com (Jean-Michel Pichavant)
Date: Fri, 19 Jun 2015 16:31:05 +0200 (CEST)
Subject: Catching exceptions with multi-processing
In-Reply-To: <mm17as$i8m$1@speranza.aioe.org>
Message-ID: <2042920174.4933523.1434724265494.JavaMail.root@sequans.com>

----- Original Message -----
> From: "Fabien" <fabien.maussion at gmail.com>
> To: python-list at python.org
> Sent: Friday, 19 June, 2015 4:01:02 PM
> Subject: Catching exceptions with multi-processing
> 
> Folks,
> 
> I am developing a tool which works on individual entities (glaciers)
> and
> do a lot of operations on them. There are many tasks to do, one after
> each other, and each task follows the same interface:
> 
> def task_1(path_to_glacier_dir):
>      open file1 in path_to_glacier_dir
>      do stuff
>      if dont_work:
>          raise RuntimeError("didnt work")
>      write file2 in path_to_glacier_dir
> 
> This way, the tasks can be run in parallel very easily:
> 
> import multiprocessing as mp
> pool = mp.Pool(4)
> 
> dirs = [list_of_dirs]
> pool.map(task1, dirs, chunksize=1)
> pool.map(task2, dirs, chunksize=1)
> pool.map(task3, dirs, chunksize=1)
> 
> ... and so forth. I tested the tool for about a hundred glaciers but
> now
> it has to run for thousands of them. There are going to be errors,
> some
> of them are even expected for special outliers. What I would like the
> tool to do is that in case of error, it writes the identifier of the
> problematic glacier somewhere, the error encountered and more info if
> possible. Because of multiprocessing, I can't write in a shared file,
> so
> I thought that the individual processes should write a unique "error
> file" in a dedicated directory.
> 
> What I don't know how to, however, is how to do this at minimal cost
> and
> in a generic way for all tasks. Also, the task2 should not be run if
> task1 threw an error. Sometimes (for debugging), I'd rather keep the
> normal behavior of raising an error and stopping the program.
> 
> Do I have to wrap all tasks with a "try: exept:" block? How to switch
> between behaviors? All the solutions I could think about look quite
> ugly
> to me. And it seems that this is a general problem that someone
> cleverer
> than me had solved before ;-)
> 
> Thanks,
> 
> Fabien

https://azylstra.net/blog/post/exception-handling-with-python-processes

shows how subprocesses can send their exception to the main process.



-- IMPORTANT NOTICE: 

The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

From steve at pearwood.info  Fri Jun 19 10:47:41 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 20 Jun 2015 00:47:41 +1000
Subject: JSON Object to CSV Question
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
Message-ID: <55842b8d$0$1674$c3e8da3$5496439d@news.astraweb.com>

On Fri, 19 Jun 2015 07:50 pm, Saran Ahluwalia wrote:

> If you read the most recent thread that I just posted it states far more
> information

The problem is, you are posting *too much* of the *wrong* information.

The exception you are getting appears to be a simple one: you are getting
the exception 

ValueError: too many values to unpack


but since you don't show the traceback, only the error message, we have no
idea *where* that exception is happening. It could be a bug in your code,
in the JSON library, or who knows where.

Python provides lots of useful debugging information in the traceback, but
you are ignoring it. You should look at the complete traceback, everything
from the initial line:

    Traceback

to the end of the error message, not just the error message itself. That
will show you the series of function calls that lead to the error, and
usually also the actual line of code that caused the problem. That alone
might be enough to solve the problem. If not, it is one extra clue. Either
way, when asking for help, you should always post the complete traceback
(unless it is hundreds of lines long, in which case, ask first).


Unfortunately you have dumped over 100 lines of code in our lap, from two
different files. To solve this problem for you, we would have to copy your
code, install the libraries you use, somehow get a copy of your XML file:

C:\\Users\\wynsa2\\Desktop\\Python Folder\\PCSU\\Trial2_PCSU\\2-Response.xml

(don't ask me how we can possibly do that!), run your code, see what the
full traceback says, and debug the code for you. That is simply not going
to happen, not unless you pay someone.

The most critical skill a programmer can have is to learn how to focus on
what is essential when troubleshooting, and not get lost in a sea of
irrelevant code. The way to do that is to narrow the problem down to the
smallest possible code sample which demonstrates the same error, and see
whether that shows you the solution. If it does, you have your solution. If
not, you can gradually add more complexity until you have your answer.

You can read more about this here:

http://sscce.org/

Although it is written for Java, the principles apply equally to Python.

Here is the minimum code needed to give your same error:

py> x, y, z = (1, 2, 3, 4)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ValueError: too many values to unpack


Here I am trying to assign four values into three variables. The solution is
obvious:

py> a, x, y, z = (1, 2, 3, 4)

will work. Does that help you see the solution you need? If not, we will
need a SSCCE to work with.


Some other things to consider.

You can avoid all those doubled backslashes by using forward slashes.
Windows accepts both:

C:/Users/wynsa2/Desktop/Python Folder/PCSU/Trial2_PCSU/2-Response.xml

which makes for much neater and easier file names.

Also, it doesn't help to ask multiple unrelated questions in the same post.
If they are directly related, that is okay, but you are asking (at least)
two questions:

- how to fix the ValueError exception;

- something about "I am wondering if there is a function or methodology that
would allow me to remove such nested keys and reassign the new keys to the
outer key..."

The second question is very vague, and vague questions get vague answers (or
no answers at all.) Yes, there is such a methodology. It is
called "programming".

Again, simplify what you are trying to do. Show us a *minimal* example of
the data you start with, and the result you hope to end up with. I'm afraid
I don't understand your question at all, simplifying it and showing an
example will help a lot.

You should post these in separate threads. People interested in one thread
may not be interested in the other. By posting them together, you only get
responses from people interested in both.


-- 
Steven



From oscar.j.benjamin at gmail.com  Fri Jun 19 11:01:17 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Fri, 19 Jun 2015 16:01:17 +0100
Subject: Catching exceptions with multi-processing
In-Reply-To: <mm17as$i8m$1@speranza.aioe.org>
References: <mm17as$i8m$1@speranza.aioe.org>
Message-ID: <CAHVvXxRBGJKk6CfNFyWhCNJdhiqLJTA=vke6=f41pxZNiwF_ow@mail.gmail.com>

On 19 June 2015 at 15:01, Fabien <fabien.maussion at gmail.com> wrote:
> Folks,
>
> I am developing a tool which works on individual entities (glaciers) and do
> a lot of operations on them. There are many tasks to do, one after each
> other, and each task follows the same interface:
>
> def task_1(path_to_glacier_dir):
>     open file1 in path_to_glacier_dir
>     do stuff
>     if dont_work:
>         raise RuntimeError("didnt work")
>     write file2 in path_to_glacier_dir
>
> This way, the tasks can be run in parallel very easily:
>
> import multiprocessing as mp
> pool = mp.Pool(4)
>
> dirs = [list_of_dirs]
> pool.map(task1, dirs, chunksize=1)
> pool.map(task2, dirs, chunksize=1)
> pool.map(task3, dirs, chunksize=1)
>
> ... and so forth. I tested the tool for about a hundred glaciers but now it
> has to run for thousands of them. There are going to be errors, some of them
> are even expected for special outliers. What I would like the tool to do is
> that in case of error, it writes the identifier of the problematic glacier
> somewhere, the error encountered and more info if possible. Because of
> multiprocessing, I can't write in a shared file, so I thought that the
> individual processes should write a unique "error file" in a dedicated
> directory.
>
> What I don't know how to, however, is how to do this at minimal cost and in
> a generic way for all tasks. Also, the task2 should not be run if task1
> threw an error. Sometimes (for debugging), I'd rather keep the normal
> behavior of raising an error and stopping the program.
>
> Do I have to wrap all tasks with a "try: exept:" block? How to switch
> between behaviors? All the solutions I could think about look quite ugly to
> me. And it seems that this is a general problem that someone cleverer than
> me had solved before ;-)

A simple way to approach this could be something like:

#!/usr/bin/env python3

import math
import multiprocessing

def sqrt(x):
    if x < 0:
        return 'error', x
    else:
        return 'success', math.sqrt(x)

if __name__ == "__main__":
    numbers = [1, 2, 3, -1, -3]
    pool = multiprocessing.Pool()
    for ret, val in pool.imap(sqrt, numbers):
        if ret == 'error':
            raise ValueError(val)
        print(val)

Just replace the raise statement with whatever you want to do (write
to a file etc). Since all errors are handled in the master process
there are no issues with writing to a file.

--
Oscar


From jeanmichel at sequans.com  Fri Jun 19 11:17:10 2015
From: jeanmichel at sequans.com (Jean-Michel Pichavant)
Date: Fri, 19 Jun 2015 17:17:10 +0200 (CEST)
Subject: Catching exceptions with multi-processing
In-Reply-To: <CAHVvXxRBGJKk6CfNFyWhCNJdhiqLJTA=vke6=f41pxZNiwF_ow@mail.gmail.com>
Message-ID: <939757481.4939774.1434727030209.JavaMail.root@sequans.com>

----- Original Message -----
> From: "Oscar Benjamin" <oscar.j.benjamin at gmail.com>
> A simple way to approach this could be something like:
> 
> #!/usr/bin/env python3
> 
> import math
> import multiprocessing
> 
> def sqrt(x):
>     if x < 0:
>         return 'error', x
>     else:
>         return 'success', math.sqrt(x)
> 
> if __name__ == "__main__":
>     numbers = [1, 2, 3, -1, -3]
>     pool = multiprocessing.Pool()
>     for ret, val in pool.imap(sqrt, numbers):
>         if ret == 'error':
>             raise ValueError(val)
>         print(val)
> 
> Just replace the raise statement with whatever you want to do (write
> to a file etc). Since all errors are handled in the master process
> there are no issues with writing to a file.
> 
> --
> Oscar

The main problem with this approach is that it does not handle unexpected exceptions within subprocesses.

JM


-- IMPORTANT NOTICE: 

The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

From steve at pearwood.info  Fri Jun 19 11:41:54 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 20 Jun 2015 01:41:54 +1000
Subject: Catching exceptions with multi-processing
References: <mm17as$i8m$1@speranza.aioe.org>
Message-ID: <55843842$0$1644$c3e8da3$5496439d@news.astraweb.com>

On Sat, 20 Jun 2015 12:01 am, Fabien wrote:

> Folks,
> 
> I am developing a tool which works on individual entities (glaciers) and
> do a lot of operations on them. There are many tasks to do, one after
> each other, and each task follows the same interface:

I'm afraid your description is contradictory. Here you say the tasks run one
after another, but then you say:

> This way, the tasks can be run in parallel very easily:

and then later still you contradict this:

> Also, the task2 should not be run if task1 threw an error.


If task2 relies on task1, then you *cannot* run them in parallel. You have
to run them one after the other, sequentially.


You also ask:

> There are going to be errors, some 
> of them are even expected for special outliers. What I would like the 
> tool to do is that in case of error, it writes the identifier of the 
> problematic glacier somewhere, the error encountered and more info if 
> possible. Because of multiprocessing, I can't write in a shared file, so 
> I thought that the individual processes should write a unique "error 
> file" in a dedicated directory.

The documentation for the logging module has examples of using
multiprocessing write to a single log file from multiple processes. It's a
bit complicated, since *directly* writing to a single log from multiple
processes is not supported, but it is possible.

https://docs.python.org/3/howto/logging-cookbook.html#logging-to-a-single-file-from-multiple-processes


Or if you are on a Unix or Linux system, you can log to syslog and let
syslog handle it.

Since your sample code appears to have a lot of file I/O, it may be that you
can use threads rather than multiprocessing. That would allow all the
threads to communicate with a single thread that handles logging.

Or use a lock file:

http://stackoverflow.com/questions/1444790/python-module-for-creating-pid-based-lockfile



-- 
Steven



From ahlusar.ahluwalia at gmail.com  Fri Jun 19 11:43:09 2015
From: ahlusar.ahluwalia at gmail.com (Saran Ahluwalia)
Date: Fri, 19 Jun 2015 11:43:09 -0400
Subject: JSON Object to CSV Question
In-Reply-To: <55842b8d$0$1674$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
 <55842b8d$0$1674$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAC9q6M65WD70Wm0=QDqMD4mis4wSB3xkO-k8pnjn9u-+A3kPLA@mail.gmail.com>

Hi Steven,

Okay, I understand now how convoluted this sounds. Here is a more focused
issue (I hope) referencing the following sample JSON object within a file:


   1. "PAC": {
   2. "Account": [{
   3. "PC": "0",
   4. "CMC": "0",
   5. "WC": "0",
   6. "DLA": "0",
   7. "CN": null,
   8. "FC": {
   9. "Int32": ["0",
   10. "0",
   11. "0",
   12. "0",
   13. "0"]
   14. },
   15. "F": {
   16. "Description": null,
   17. "Code": "0"
   18. }
   19.

The rationale for the following: the out put will need to be a CSV. All of
the keys should be headers in the CSV. The values are all corresponding
with the headers. However, Int32 should be deleted and the key reassigned
to FC. Account should be deleted and reassigned to PAC.

Essentially, where all of the instances where you have a key with a list of
dictionary, that outer key should be deleted and reassigned to the
outermost key.

Finally (in this example) the "0"s should be all concatenated in the output
CSV so accessing the lists will need to be addressed first. I know that the
following:

def hook(obj):
    return obj
def flatten(obj):
    for k, v in obj:
        if isinstance(v, list):
            yield from flatten(v)
        else:
            yield k, v
if __name__ == "__main__":
    with open("somefileneame.json") as f:
        data = json.load(f, object_pairs_hook=hook)
    pairs = list(flatten(data))
    writer = csv.writer(sys.stdout)
    header = writer.writerow([k for k, v in pairs])
    row = writer.writerow([v for k, v in pairs]) #writer.writerows for
any other iterable object

is not the best strategy. The list objects cannot be accessed (hence, the
"ValueError: too many values to unpack") is caught at the function
json.loads(). I am wondering how to add the exception when 1) when one
encounters the list and 2) how to delete and re-asssign the keys (as
described above).

Thank you for your guidance.

On Fri, Jun 19, 2015 at 10:47 AM, Steven D'Aprano <steve at pearwood.info>
wrote:

> On Fri, 19 Jun 2015 07:50 pm, Saran Ahluwalia wrote:
>
> > If you read the most recent thread that I just posted it states far more
> > information
>
> The problem is, you are posting *too much* of the *wrong* information.
>
> The exception you are getting appears to be a simple one: you are getting
> the exception
>
> ValueError: too many values to unpack
>
>
> but since you don't show the traceback, only the error message, we have no
> idea *where* that exception is happening. It could be a bug in your code,
> in the JSON library, or who knows where.
>
> Python provides lots of useful debugging information in the traceback, but
> you are ignoring it. You should look at the complete traceback, everything
> from the initial line:
>
>     Traceback
>
> to the end of the error message, not just the error message itself. That
> will show you the series of function calls that lead to the error, and
> usually also the actual line of code that caused the problem. That alone
> might be enough to solve the problem. If not, it is one extra clue. Either
> way, when asking for help, you should always post the complete traceback
> (unless it is hundreds of lines long, in which case, ask first).
>
>
> Unfortunately you have dumped over 100 lines of code in our lap, from two
> different files. To solve this problem for you, we would have to copy your
> code, install the libraries you use, somehow get a copy of your XML file:
>
> C:\\Users\\wynsa2\\Desktop\\Python
> Folder\\PCSU\\Trial2_PCSU\\2-Response.xml
>
> (don't ask me how we can possibly do that!), run your code, see what the
> full traceback says, and debug the code for you. That is simply not going
> to happen, not unless you pay someone.
>
> The most critical skill a programmer can have is to learn how to focus on
> what is essential when troubleshooting, and not get lost in a sea of
> irrelevant code. The way to do that is to narrow the problem down to the
> smallest possible code sample which demonstrates the same error, and see
> whether that shows you the solution. If it does, you have your solution. If
> not, you can gradually add more complexity until you have your answer.
>
> You can read more about this here:
>
> http://sscce.org/
>
> Although it is written for Java, the principles apply equally to Python.
>
> Here is the minimum code needed to give your same error:
>
> py> x, y, z = (1, 2, 3, 4)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> ValueError: too many values to unpack
>
>
> Here I am trying to assign four values into three variables. The solution
> is
> obvious:
>
> py> a, x, y, z = (1, 2, 3, 4)
>
> will work. Does that help you see the solution you need? If not, we will
> need a SSCCE to work with.
>
>
> Some other things to consider.
>
> You can avoid all those doubled backslashes by using forward slashes.
> Windows accepts both:
>
> C:/Users/wynsa2/Desktop/Python Folder/PCSU/Trial2_PCSU/2-Response.xml
>
> which makes for much neater and easier file names.
>
> Also, it doesn't help to ask multiple unrelated questions in the same post.
> If they are directly related, that is okay, but you are asking (at least)
> two questions:
>
> - how to fix the ValueError exception;
>
> - something about "I am wondering if there is a function or methodology
> that
> would allow me to remove such nested keys and reassign the new keys to the
> outer key..."
>
> The second question is very vague, and vague questions get vague answers
> (or
> no answers at all.) Yes, there is such a methodology. It is
> called "programming".
>
> Again, simplify what you are trying to do. Show us a *minimal* example of
> the data you start with, and the result you hope to end up with. I'm afraid
> I don't understand your question at all, simplifying it and showing an
> example will help a lot.
>
> You should post these in separate threads. People interested in one thread
> may not be interested in the other. By posting them together, you only get
> responses from people interested in both.
>
>
> --
> Steven
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150619/90544184/attachment.html>

From fabien.maussion at gmail.com  Fri Jun 19 12:07:13 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Fri, 19 Jun 2015 18:07:13 +0200
Subject: Catching exceptions with multi-processing
References: <mm17as$i8m$1@speranza.aioe.org>
 <55843842$0$1644$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mm1enf$66l$1@speranza.aioe.org>

On 06/19/2015 05:41 PM, Steven D'Aprano wrote:
> On Sat, 20 Jun 2015 12:01 am, Fabien wrote:
>
>> >Folks,
>> >
>> >I am developing a tool which works on individual entities (glaciers) and
>> >do a lot of operations on them. There are many tasks to do, one after
>> >each other, and each task follows the same interface:
> I'm afraid your description is contradictory. Here you say the tasks run one
> after another, but then you say:
>
>> >This way, the tasks can be run in parallel very easily:
> and then later still you contradict this:
>
>> >Also, the task2 should not be run if task1 threw an error.
>
> If task2 relies on task1, then you*cannot*  run them in parallel. You have
> to run them one after the other, sequentially.

Hi Steve,

I meant: "for several glaciers in parallel" as shown by the code snippet:

import multiprocessing as mp
pool = mp.Pool(4)

dirs = [list_of_dirs]
pool.map(task1, dirs, chunksize=1)
pool.map(task2, dirs, chunksize=1)
pool.map(task3, dirs, chunksize=1)

which should be changed to something like (after some of the responses):

dirs = [list_of_dirs]
pool.map(task1, dirs, ...)
# handle exceptions
dirs_reduced = [dirs that did not fail]
pool.map(task2, dirs_reduced, ...)

this way the tasks are run sequentially for each glacier but in parallel 
over a list of glaciers...

Fabien



From fabien.maussion at gmail.com  Fri Jun 19 12:16:47 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Fri, 19 Jun 2015 18:16:47 +0200
Subject: Catching exceptions with multi-processing
References: <mm17as$i8m$1@speranza.aioe.org>
 <mailman.637.1434723958.13271.python-list@python.org>
Message-ID: <mm1f9d$751$1@speranza.aioe.org>

On 06/19/2015 04:25 PM, Andres Riancho wrote:
> Fabien,
>
>     My recommendation is that you should pass some extra arguments to the task:
>      * A unique task id
>      * A result multiprocessing.Queue
>
>      When an exception is raised you put (unique_id, exception) to the
> queue. When it succeeds you put (unique_id, None). In the main process
> you consume the queue and do your error handling.
>
>      Note that some exceptions can't be serialized, there is where
> tblib [0] comes handy.
>
> [0]https://pypi.python.org/pypi/tblib
>
> Regards,

Thanks, I wasn't aware of the multiprocessing.Queue workflow. It seems 
like its going to require some changes in the actual code of the tasks 
though. Did I get it right that I should stop raising exceptions then?

Something like:

def task_1(path, q):
     # Do stuffs
     if dont_work:
         q.put(RuntimeError("didnt work"))
	return
     # finished
     q.put(None)
     return


Fabien







From scott.howard.james at gmail.com  Fri Jun 19 12:18:15 2015
From: scott.howard.james at gmail.com (Scott)
Date: Fri, 19 Jun 2015 12:18:15 -0400
Subject: No subject
Message-ID: <6264c933-0ed4-4055-baad-01b322bb6dd4@Calculus-Fantasticus-2.local>

We've been using a simple container implementation of a mathematical relation (https://simple.wikipedia.org/wiki/Relation_(mathematics)) (i.e. an invertible M:M mapping) for some time.
We've been waiting for many years (decades actually) to have this concept incorporated as a standard container in one of the modern programming languages so we could swap our code with a language-standard container. To further these efforts, we have decided to initiate a conversation with the python community on adding a relation as a standard Python container package. 

Briefly the uses of a relation are:

- quickly finding the values/range associated with keys/domain (e.g. *inversion*)
- maintaining a unique relationship between keys and values,(e.g. *isomorphism* or *aliasing*)
- using keys to categorize (one-to-many) values (e.g. *partitioning*)
- associating two sets in an arbitrary/bipartite manner (e.g. *tagging*)

Below is a link to an implementation, including an ipython notebook with light exposition and some canonical examples. We'll also be presenting a poster at SciPy 2015. Let us know if this, or something like it, seems like a worthwhile addition to the standard Python distribution.

Thank you,

Scott James, James Larkin

scott.james at noblis.org 
james.larkin at noblis.org


https://github.com/scott-howard-james/relate 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150619/f33fffe9/attachment.html>

From scott.howard.james at gmail.com  Fri Jun 19 13:14:09 2015
From: scott.howard.james at gmail.com (Scott)
Date: Fri, 19 Jun 2015 13:14:09 -0400
Subject: Relation: a new standard (PEP) container?
Message-ID: <92c9b8b1-8f83-4b1c-9764-23226a92e391@Calculus-Fantasticus-2.local>

We've been using a simple container implementation of a mathematical relation (https://simple.wikipedia.org/wiki/Relation_(mathematics)) (i.e. an invertible M:M mapping) for some time.
We've been waiting for many years (decades actually) to have this concept incorporated as a standard container in one of the modern programming languages so we could swap our code with a language-standard container. To further these efforts, we have decided to initiate a conversation with the python community on adding a relation as a standard Python container package. 

Briefly the uses of a relation are:

- quickly finding the values/range associated with keys/domain (e.g. *inversion*)
- maintaining a unique relationship between keys and values,(e.g. *isomorphism* or *aliasing*)
- using keys to categorize (one-to-many) values (e.g. *partitioning*)
- associating two sets in an arbitrary/bipartite manner (e.g. *tagging*)

Below is a link to an implementation, including an ipython notebook with light exposition and some canonical examples. We'll also be presenting a poster at SciPy 2015. Let us know if this, or something like it, seems like a worthwhile addition to the standard Python distribution.

Thank you,

Scott James, James Larkin

scott.james at noblis.org 
james.larkin at noblis.org


https://github.com/scott-howard-james/relate 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150619/40b27dc5/attachment.html>

From nmichalowsky at gmail.com  Fri Jun 19 13:24:56 2015
From: nmichalowsky at gmail.com (Naftali)
Date: Fri, 19 Jun 2015 10:24:56 -0700 (PDT)
Subject: Opening PDF Using subprocess.Popen Failing
Message-ID: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>

It actually doesn't fail but it 'cannot open in protected mode' (see here http://blogs.adobe.com/dmcmahon/2012/07/27/adobe-reader-cannot-open-protected-mode-due-to-a-problem-with-your-system-configuration/)

I am using subprocess.Popen("AcroRe32.exe /n <file.pdf>") which is the actuall adobe reader command I'd issue on the command line to open the pdf (the /n option opens it the file in a new instance of reader).

Now, when I issue the command straight from powershell, the pdf opens no problem, but when I open in my script (whether a .py or py2exe) I get the pop up complaining that the PDF cannot be opened in 'protected mode.' One of the options is to open it anyways, which works. 

Looking into it (see the link in the first paragraph) my best guess is that it's due to something like "JS-invoked processes: Launching a process through JavaScript is not allowed with Protected Mode enabled." 

But my naive understanding was that when I give Popen instruction, the command is handed off to windows and the called program is unaware of how it got called, so my thinking is that either that is incorrect or windows somehow 'cooperates' with reader to figure things out. 

I am looking for *any* insight as to how to deal with this, and the 'turn off protected mode" option wont work for me. 

Here is my code,

outputname = " unlocked.pdf"

commandstr = "qpdf --decrypt " + sys.argv[1] + outputname
os.system(commandstr)

new_command_str = "AcroRd32.exe /n" + outputname
subprocess.Popen(new_command_str)

sys.exit(0)
 


From lac at openend.se  Fri Jun 19 14:17:13 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 19 Jun 2015 20:17:13 +0200
Subject: Opening PDF Using subprocess.Popen Failing
In-Reply-To: Message from Naftali <nmichalowsky@gmail.com> of "Fri,
 19 Jun 2015 10:24:56 -0700."
 <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
References: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
Message-ID: <201506191817.t5JIHDtB011718@fido.openend.se>

In a message of Fri, 19 Jun 2015 10:24:56 -0700, Naftali writes:
>It actually doesn't fail but it 'cannot open in protected mode' (see here http://blogs.adobe.com/dmcmahon/2012/07/27/adobe-reader-cannot-open-protected-mode-due-to-a-problem-with-your-system-configuration/)
>
>I am using subprocess.Popen("AcroRe32.exe /n <file.pdf>") which is the actuall adobe reader command I'd issue on the command line to open the pdf (the /n option opens it the file in a new instance of reader).
>
>Now, when I issue the command straight from powershell, the pdf opens no problem, but when I open in my script (whether a .py or py2exe) I get the pop up complaining that the PDF cannot be opened in 'protected mode.' One of the options is to open it anyways, which works. 
>
>Looking into it (see the link in the first paragraph) my best guess is that it's due to something like "JS-invoked processes: Launching a process through JavaScript is not allowed with Protected Mode enabled." 
>
>But my naive understanding was that when I give Popen instruction, the command is handed off to windows and the called program is unaware of how it got called, so my thinking is that either that is incorrect or windows somehow 'cooperates' with reader to figure things out. 
>
>I am looking for *any* insight as to how to deal with this, and the 'turn off protected mode" option wont work for me. 
>
>Here is my code,
>
>outputname = " unlocked.pdf"
>
>commandstr = "qpdf --decrypt " + sys.argv[1] + outputname
>os.system(commandstr)
>
>new_command_str = "AcroRd32.exe /n" + outputname
>subprocess.Popen(new_command_str)
>
>sys.exit(0)
> 
>-- 
>https://mail.python.org/mailman/listinfo/python-list

I don't have a windows system to test this on, but I read
https://www.adobe.com/devnet-docs/acrobatetk/tools/AppSec/protectedmode.html

It looks like if you put your output file one of the places that
Acrobat allows, then things might work.  You can then move the thing
later to where you want it to go.

There is also this tantalising FAQ entry:


      When custom policies fail for certain workflows, what are the
      options other than disabling Protected Mode?

      One option is to add custom policies to bypass protected mode 
      restrictions.

Now, I haven't found out _how you do that_ yet, but seems like a
promising area for research.

Laura


From nmichalowsky at gmail.com  Fri Jun 19 14:45:17 2015
From: nmichalowsky at gmail.com (Naftali)
Date: Fri, 19 Jun 2015 11:45:17 -0700 (PDT)
Subject: Opening PDF Using subprocess.Popen Failing
In-Reply-To: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
References: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
Message-ID: <84df09d0-29cb-4cf0-9faa-7e9cf694aef4@googlegroups.com>

On Friday, June 19, 2015 at 1:25:12 PM UTC-4, Naftali wrote:
> It actually doesn't fail but it 'cannot open in protected mode' (see here http://blogs.adobe.com/dmcmahon/2012/07/27/adobe-reader-cannot-open-protected-mode-due-to-a-problem-with-your-system-configuration/)
> 
> I am using subprocess.Popen("AcroRe32.exe /n <file.pdf>") which is the actuall adobe reader command I'd issue on the command line to open the pdf (the /n option opens it the file in a new instance of reader).
> 
> Now, when I issue the command straight from powershell, the pdf opens no problem, but when I open in my script (whether a .py or py2exe) I get the pop up complaining that the PDF cannot be opened in 'protected mode.' One of the options is to open it anyways, which works. 
> 
> Looking into it (see the link in the first paragraph) my best guess is that it's due to something like "JS-invoked processes: Launching a process through JavaScript is not allowed with Protected Mode enabled." 
> 
> But my naive understanding was that when I give Popen instruction, the command is handed off to windows and the called program is unaware of how it got called, so my thinking is that either that is incorrect or windows somehow 'cooperates' with reader to figure things out. 
> 
> I am looking for *any* insight as to how to deal with this, and the 'turn off protected mode" option wont work for me. 
> 
> Here is my code,
> 
> outputname = " unlocked.pdf"
> 
> commandstr = "qpdf --decrypt " + sys.argv[1] + outputname
> os.system(commandstr)
> 
> new_command_str = "AcroRd32.exe /n" + outputname
> subprocess.Popen(new_command_str)
> 
> sys.exit(0)

Yes, this is excellent. thank you so much. Will update the list when this works out.


From liik.joonas at gmail.com  Fri Jun 19 14:47:42 2015
From: liik.joonas at gmail.com (Joonas Liik)
Date: Fri, 19 Jun 2015 21:47:42 +0300
Subject: JSON Object to CSV Question
In-Reply-To: <CAC9q6M65WD70Wm0=QDqMD4mis4wSB3xkO-k8pnjn9u-+A3kPLA@mail.gmail.com>
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
 <55842b8d$0$1674$c3e8da3$5496439d@news.astraweb.com>
 <CAC9q6M65WD70Wm0=QDqMD4mis4wSB3xkO-k8pnjn9u-+A3kPLA@mail.gmail.com>
Message-ID: <CAB1GNpTPGoAkOYNHQV8HGNOQV8XjFcGft=mxgL+K=4_OD+zWkQ@mail.gmail.com>

You say you are taking this from an xml file and want to get a CSV file..

Why are you making an intermediate JSON file?
Why do you need the CSV output?
Could you perhaps be better off using another format?

Your data seems to be a quite deeply nested hierarchical structure and
doesn't
  seem to suit the simple CSV format very well.. (i see several layers of
nested arrays for one)..
  how to you plan to map these nested structures to the CSV format?


And your example json bit.. the JSON you posted in your last post seems
quite different from some of the first ones you posted.

You cant expect to give people 5% of a malformed simplified example and to
get anything useful (much less usable) back.

And finally, the original xml? mock at least?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150619/8ba8b4b1/attachment.html>

From info at wingware.com  Fri Jun 19 15:58:49 2015
From: info at wingware.com (Wingware)
Date: Fri, 19 Jun 2015 15:58:49 -0400
Subject: ANN:  Wing IDE 5.1.5 released
Message-ID: <55847479.9070205@wingware.com>

Hi,

Wingware has released version 5.1.5 of Wing IDE, our cross-platform 
integrated development environment for the Python programming language.

Wing IDE features a professional code editor with vi, emacs, visual 
studio, and other key bindings, auto-completion, call tips, 
context-sensitive auto-editing, goto-definition, find uses, refactoring, 
a powerful debugger, version control, unit testing, search, project 
management, and many other features.

This release includes the following improvements:

     Support for Python 3.5b2
     Added Step Out To Here to return to currently selected stack frame
     Add syntax highlighting for Less and HSS CSS extension languages
     Enable auto-fixups and auto-invocation in the Python Shell and 
Debug Probe
     Fix running and debugging Django 1.8 unit tests (thanks to Rafal 
Zawadzki)
     Don't truncate output as readily in the Python Shell or Debug I/O
     Fix Debug Restart when debugging a file and no main entry point is set
     Fix spurious syntax errors for Python 3 code (up to and including 
3.4 language features)
     Improved tracebacks in the Python Shell and Debug Probe with more 
useful file names
     Several auto-editing improvements
     Display source file when click on an exception line in a 
Cython-compiled module
     About 40 other improvements

For details see http://wingware.com/news/2015-06-19 and 
http://wingware.com/pub/wingide/5.1.5/CHANGELOG.txt

What's New in Wing 5.1:

Wing IDE 5.1 adds multi-process and child process debugging, syntax 
highlighting in the shells, support for pytest, Find Symbol in Project, 
persistent time-stamped unit test results, auto-conversion of indents on 
paste, an XCode keyboard personality, support for Flask, Django 1.7 and 
1.8, Python 3.5 and recent Google App Engine versions, improved 
auto-completion for PyQt, recursive snippet invocation, and many other 
minor features and improvements.

Free trial: http://wingware.com/wingide/trial
Downloads: http://wingware.com/downloads
Feature list: http://wingware.com/wingide/features
Sales: http://wingware.com/store/purchase
Upgrades: https://wingware.com/store/upgrade

Questions?  Don't hesitate to email us at support at wingware.com.

Thanks,

-- 

Stephan Deibel
Wingware | Python IDE

The Intelligent Development Environment for Python Programmers

wingware.com




From ahlusar.ahluwalia at gmail.com  Fri Jun 19 16:00:22 2015
From: ahlusar.ahluwalia at gmail.com (Saran Ahluwalia)
Date: Fri, 19 Jun 2015 16:00:22 -0400
Subject: JSON Object to CSV Question
In-Reply-To: <CAB1GNpTPGoAkOYNHQV8HGNOQV8XjFcGft=mxgL+K=4_OD+zWkQ@mail.gmail.com>
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
 <55842b8d$0$1674$c3e8da3$5496439d@news.astraweb.com>
 <CAC9q6M65WD70Wm0=QDqMD4mis4wSB3xkO-k8pnjn9u-+A3kPLA@mail.gmail.com>
 <CAB1GNpTPGoAkOYNHQV8HGNOQV8XjFcGft=mxgL+K=4_OD+zWkQ@mail.gmail.com>
Message-ID: <CAC9q6M7uX0yeWoQQCJjD_vTwyfCaevLX4Y8-HctLcqYV--N60Q@mail.gmail.com>

My answers are below in red.

On Fri, Jun 19, 2015 at 2:47 PM, Joonas Liik <liik.joonas at gmail.com> wrote:

>
> You say you are taking this from an xml file and want to get a CSV file..
>
> Why are you making an intermediate JSON file?
>

I thought that this would be the most efficient method as it emulates
Python dictionaries. In addition the json module has numerous functional
purposes.


> Why do you need the CSV output?
>

Client requirements


Could you perhaps be better off using another format?
>

Perhaps...not sure if it worth revisiting at this point.

>
> Your data seems to be a quite deeply nested hierarchical structure and
> doesn't
>   seem to suit the simple CSV format very well.. (i see several layers of
> nested arrays for one)..
>   how to you plan to map these nested structures to the CSV format?
>
>
 That's my greatest challenge. From what I understand, I need to replicate
each key and associate it with n number of items in the array. I have been
able to address by denesting by one level using this function:



   1. def flatten(d, parent_key=''):
   2. items = []
   3. for k, v in d.items():
   4. try:
   5. items.extend(flatten(v, '%s%s_' % (parent_key, k)).items())
   6. except AttributeError:
   7. items.append(('%s%s' % (parent_key, k), v))
   8. return dict(items)
   9.
   10. final = (flatten(data2, parent_key =''))



For example:


   1. data2 = {
   2.
   3. "OTF": "0",
   4. "F": "False",
   5. "F": {
   6. "Int32": ["0",
   7. "0",
   8. "0",
   9. "0",
   10. "0",
   11. "0",
   12. "0",
   13. "0",
   14. "0",
   15. "0",
   16. "0",
   17. "0",
   18. "0",
   19. "0",
   20. "0",
   21. "0",
   22. "0",
   23. "0",
   24. "0",
   25. "0",
   26. "0",
   27. "0",
   28. "0",
   29. "0",
   30. "0"]
   31. },
   32. "D": {
   33. "B": ["0",
   34. "0",
   35. "0",
   36. "0",
   37. "0",
   38. "0",
   39. "0",
   40. "0",
   41. "0",
   42. "0",
   43. "0"]
   44. },
   45.
   46. "PBDS": {
   47. "DateTime": ["1/1/0001 12:00:00 AM",
   48. "1/1/0001 12:00:00 AM",
   49. "1/1/0001 12:00:00 AM",
   50. "1/1/0001 12:00:00 AM",
   51. "1/1/0001 12:00:00 AM",
   52. "1/1/0001 12:00:00 AM",
   53. "1/1/0001 12:00:00 AM",
   54. "1/1/0001 12:00:00 AM"]
   55. },
   56.
   57. "PBDS": {
   58. "Double": ["0",
   59. "0",
   60. "0",
   61. "0",
   62. "0",
   63. "0",
   64. "0",
   65. "0"]
   66. },
   67.
   68. "SCS": {
   69. "String": ["1",
   70. "2"]
   71. }
   72.
   73. }



is turned into


   1. {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
   2. 'F_Int32': ['0',
   3. '0',
   4. '0',
   5. '0',
   6. '0',
   7. '0',
   8. '0',
   9. '0',
   10. '0',
   11. '0',
   12. '0',
   13. '0',
   14. '0',
   15. '0',
   16. '0',
   17. '0',
   18. '0',
   19. '0',
   20. '0',
   21. '0',
   22. '0',
   23. '0',
   24. '0',
   25. '0',
   26. '0'],
   27. 'OTF': '0',
   28. 'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'],
   29. 'SCS_String': ['1', '2']}


I would need an output that maps each key with each item in the array.

 DB1 : 0, DB2: 0, DB3: 0 etc. and F1: 0, F1: 0. DB1, DB2 would be the
headers and the 0s as values in the CSV file.


> And your example json bit.. the JSON you posted in your last post seems
> quite different from some of the first ones you posted.
>
> You cant expect to give people 5% of a malformed simplified example and to
> get anything useful (much less usable) back.
>
> And finally, the original xml? mock at least?
>

All of our XML from our client is malformed.

>
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150619/53fe677a/attachment.html>

From liik.joonas at gmail.com  Fri Jun 19 16:32:42 2015
From: liik.joonas at gmail.com (Joonas Liik)
Date: Fri, 19 Jun 2015 23:32:42 +0300
Subject: JSON Object to CSV Question
In-Reply-To: <CAC9q6M7uX0yeWoQQCJjD_vTwyfCaevLX4Y8-HctLcqYV--N60Q@mail.gmail.com>
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
 <55842b8d$0$1674$c3e8da3$5496439d@news.astraweb.com>
 <CAC9q6M65WD70Wm0=QDqMD4mis4wSB3xkO-k8pnjn9u-+A3kPLA@mail.gmail.com>
 <CAB1GNpTPGoAkOYNHQV8HGNOQV8XjFcGft=mxgL+K=4_OD+zWkQ@mail.gmail.com>
 <CAC9q6M7uX0yeWoQQCJjD_vTwyfCaevLX4Y8-HctLcqYV--N60Q@mail.gmail.com>
Message-ID: <CAB1GNpRjjqPa3aopSoagRrZYoff7OpNXa1YZrQ_VsFOZ0VyW0A@mail.gmail.com>

this.. might not throw an eror, but you have 2 keys with the same name
"F", and 1 of them will probably be disgarded..., you have data
corruption even before you try to process it.

{
 "F": "False",
"F": {
"Int32": ["0",
"0",
"0"]
},
 }

you mentioned Excel at one point.
perhaps you could mock up what you'd like your finished data to look
like in a spreadsheet (google docs for instance, since thats easy to
link to) and reference there.

just having a list of headers doesnt say much about the data format you want.

"client wants csv" hmm..they want "csv" or they want "csv that fists
this very particular description that fits our special decoder or the
like" ?

do you know how the client will use this data. could that info be used
to simplify the output to some degree?

and finally..
the client gives you malformed xml??
I'm very sorry to hear that. also does the client know they are
emitting invalid xml?
    is it rly xml?
    is it valid in another language? (html is much more lenient for
instance, an html parser might be able to gleam more meaning)
    by what definition is it malformed? is it outright structuralyl
broken, does it fail to meet some schema?
                      does it fail to meet some expectation you have
for some reason ("client said it has these properties")

you also mentioned you use JSON because it maps nicely to python
dicts.. this is true ofc.. but why not just read that in to a python
dict in the first place?

> DB1 : 0, DB2: 0, DB3: 0 etc. and F1: 0, F1: 0. DB1, DB2 would be the headers and the 0s as values in the CSV file.

DB1 etc seems ok at first glance however...
say there are 2 nested items and each of them have a DB property which
is an array, you will have name collisions.
you need more thought in to naming the headers at the very least.

if this is meant for a spreadsheet.. then you will end up with 2 very
very very long rows, it will NOT be readable by any stretch of the
imagination.. do you want this.

i'm afraid you'll essentially end up with a translation that looks sth like

{A:{b:"c",q:"w"}}
===============
"A.b", "A.q"
"c", "w"

if you just want key-value pairs there are better options out
there..besides csv..


From rosuav at gmail.com  Fri Jun 19 16:58:13 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 20 Jun 2015 06:58:13 +1000
Subject: Catching exceptions with multi-processing
In-Reply-To: <55843842$0$1644$c3e8da3$5496439d@news.astraweb.com>
References: <mm17as$i8m$1@speranza.aioe.org>
 <55843842$0$1644$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmpyqsLUQUUtrBFtDkBp-rWw-hiUEVWtjd3oP9jLEd-D=Q@mail.gmail.com>

On Sat, Jun 20, 2015 at 1:41 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sat, 20 Jun 2015 12:01 am, Fabien wrote:
>
>> Folks,
>>
>> I am developing a tool which works on individual entities (glaciers) and
>> do a lot of operations on them. There are many tasks to do, one after
>> each other, and each task follows the same interface:
>
> I'm afraid your description is contradictory. Here you say the tasks run one
> after another, but then you say:
>
>> This way, the tasks can be run in parallel very easily:
>
> and then later still you contradict this:
>
>> Also, the task2 should not be run if task1 threw an error.
>
>
> If task2 relies on task1, then you *cannot* run them in parallel. You have
> to run them one after the other, sequentially.

AIUI what he's doing is all the subparts of task1 in parallel, then
all the subparts of task2:

pool.map(task1, dirs, chunksize=1)
pool.map(task2, dirs, chunksize=1)
pool.map(task3, dirs, chunksize=1)

task1 can be done on all of dirs in parallel, as no instance of task1
depends on any other instance of task1; but task2 should be started
only if all task1s finish successfully. OP, is this how it is? If not,
I apologize for the noise.

ChrisA


From ahlusar.ahluwalia at gmail.com  Fri Jun 19 17:04:23 2015
From: ahlusar.ahluwalia at gmail.com (Saran Ahluwalia)
Date: Fri, 19 Jun 2015 17:04:23 -0400
Subject: JSON Object to CSV Question
In-Reply-To: <CAB1GNpRjjqPa3aopSoagRrZYoff7OpNXa1YZrQ_VsFOZ0VyW0A@mail.gmail.com>
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
 <55842b8d$0$1674$c3e8da3$5496439d@news.astraweb.com>
 <CAC9q6M65WD70Wm0=QDqMD4mis4wSB3xkO-k8pnjn9u-+A3kPLA@mail.gmail.com>
 <CAB1GNpTPGoAkOYNHQV8HGNOQV8XjFcGft=mxgL+K=4_OD+zWkQ@mail.gmail.com>
 <CAC9q6M7uX0yeWoQQCJjD_vTwyfCaevLX4Y8-HctLcqYV--N60Q@mail.gmail.com>
 <CAB1GNpRjjqPa3aopSoagRrZYoff7OpNXa1YZrQ_VsFOZ0VyW0A@mail.gmail.com>
Message-ID: <CAC9q6M5j=SN6UozvbkszYCX2ppx7vxZ5EsK0iJ_q03Lbq5nXEg@mail.gmail.com>

@Joonas:

The previous example was a typo. Please use the below example as a case
study.


   1. {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
   2. 'F_Int32': ['0',
   3. '0',
   4. '0',
   5. '0',
   6. '0',
   7. '0',
   8. '0',
   9. '0',
   10. '0',
   11. '0',
   12. '0',
   13. '0',
   14. '0',
   15. '0',
   16. '0',
   17. '0',
   18. '0',
   19. '0',
   20. '0',
   21. '0',
   22. '0',
   23. '0',
   24. '0',
   25. '0',
   26. '0'],
   27. 'OTF': '0',
   28. 'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'],
   29. 'SCS_String': ['1', '2']}
   30.


All of the questions regarding XML I have asked. I have to work within
their parameters. The CSV, for example may look like this:

DLA,FC,PC,WC,CN,Description,Code,CMC
0,00000,0,0,,,0,0

I have made some head way, as stated previously. I have now been able to
experiment with a list comprehension that will demonstrates how each header
will need  to be for headers with repeating values.


   1. >>> key = 'spam'
   2. >>> L = ['foo', 'bar', 'baz']
   3. >>> [('{}{}'.format(key, i), value) for i, value in enumerate(L, 1)]
   4. [('spam1', 'foo'), ('spam2', 'bar'), ('spam3', 'baz')]



Is there a way that you know of that will:

1) Allow me to preserve the following two functions:

port json
import sys

def hook(obj):
    return obj

def flatten(obj):
    for k, v in obj:
        if isinstance(v, list):
            yield from flatten(v)
        else:
            yield k, v

if __name__ == "__main__":
    with open("somefileneame.json") as f:
        data = json.load(f, object_pairs_hook=hook)

    pairs = list(flatten(data))

    writer = csv.writer(sys.stdout)
    header = writer.writerow([k for k, v in pairs])
    row = writer.writerow([v for k, v in pairs]) #writer.writerows for any
other iterable object




and

2)

Will conditionally allow me to recursively check if a key has a nested
array. If so, then apply the  [('{}{}'.format(key, i), value) for i, value
in enumerate(L, 1)] list comprehension?


On Fri, Jun 19, 2015 at 4:32 PM, Joonas Liik <liik.joonas at gmail.com> wrote:

> this.. might not throw an eror, but you have 2 keys with the same name
> "F", and 1 of them will probably be disgarded..., you have data
> corruption even before you try to process it.
>
> {
>  "F": "False",
> "F": {
> "Int32": ["0",
> "0",
> "0"]
> },
>  }
>
> you mentioned Excel at one point.
> perhaps you could mock up what you'd like your finished data to look
> like in a spreadsheet (google docs for instance, since thats easy to
> link to) and reference there.
>
> just having a list of headers doesnt say much about the data format you
> want.
>
> "client wants csv" hmm..they want "csv" or they want "csv that fists
> this very particular description that fits our special decoder or the
> like" ?
>
> do you know how the client will use this data. could that info be used
> to simplify the output to some degree?
>
> and finally..
> the client gives you malformed xml??
> I'm very sorry to hear that. also does the client know they are
> emitting invalid xml?
>     is it rly xml?
>     is it valid in another language? (html is much more lenient for
> instance, an html parser might be able to gleam more meaning)
>     by what definition is it malformed? is it outright structuralyl
> broken, does it fail to meet some schema?
>                       does it fail to meet some expectation you have
> for some reason ("client said it has these properties")
>
> you also mentioned you use JSON because it maps nicely to python
> dicts.. this is true ofc.. but why not just read that in to a python
> dict in the first place?
>
> > DB1 : 0, DB2: 0, DB3: 0 etc. and F1: 0, F1: 0. DB1, DB2 would be the
> headers and the 0s as values in the CSV file.
>
> DB1 etc seems ok at first glance however...
> say there are 2 nested items and each of them have a DB property which
> is an array, you will have name collisions.
> you need more thought in to naming the headers at the very least.
>
> if this is meant for a spreadsheet.. then you will end up with 2 very
> very very long rows, it will NOT be readable by any stretch of the
> imagination.. do you want this.
>
> i'm afraid you'll essentially end up with a translation that looks sth like
>
> {A:{b:"c",q:"w"}}
> ===============
> "A.b", "A.q"
> "c", "w"
>
> if you just want key-value pairs there are better options out
> there..besides csv..
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150619/5ecd620a/attachment.html>

From ben+python at benfinney.id.au  Fri Jun 19 17:13:25 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 20 Jun 2015 07:13:25 +1000
Subject: instance as module
References: <5583E0EA.2090309@chamonix.reportlab.co.uk>
 <mm0qig$a4d$1@ger.gmane.org>
 <5583F7F6.9000500@chamonix.reportlab.co.uk>
Message-ID: <85616jtn9m.fsf@benfinney.id.au>

Robin Becker <robin at reportlab.com> writes:

> For the specific case of the canvas_basefontname I don't actually need
> to do anything specific since it is just a string

The configuration values should be nothing but immutable values:
strings, integers, booleans. Possibly collections of those.

You seem to be implying that some configuration values are complex
objects; that's undesirable.

You seem to be further implying that configuration is executable code;
that is *definitely* undesirable (look at the awful mess created by
making Distutils configuration an executable program).

> however, before it can be used in action support has to be provided ie
> I must register the font used.

That's fine, it is a step that should come *after* configuration is
entirely completed and the static values are known.

> I could just make the few usages of this value check for callability
> at use time, but that would scatter the problem; if one default is
> special why not all.

Indeed. The correct solution is not to make configuration executable,
but instead make it static data, read by the program as simple immutable
values.

> Effectively the defaults setup is not complex enough to allow use of
> itself; that must be a fairly common problem.

The common solution is:

* Use a non-executable configuration file format. ?configparser? from
  the standard library is a good option, YAML is another.

* Use the conventional precedence order for applying configuration
  values: default values, overridden by config files, overridden by
  environment variables, overridden by command-line options.

* Only after configuration using the above precedence is complete, use
  the immutable values defined to change program behaviour (e.g. create
  or modify dynamic, complex objects that respond to the config
  options).

If you've got a config file that is executable, and attempts to define
program objects, then yes you're going to get the complex mess you
describe in your initial posting of this thread.

Avoid that by migrating away from such a poor configuration format:
Switch to a ?configparser? format, or some other static, non-executable
data file.

> Probably the correct thing to do was always to use an object to hold
> the defaults.

Yes, a ?configparser.ConfigParser()? instance can store defaults which
will be over-ridden by the sequence of configuration files read.

-- 
 \           ?I do not believe in forgiveness as it is preached by the |
  `\        church. We do not need the forgiveness of God, but of each |
_o__)                    other and of ourselves.? ?Robert G. Ingersoll |
Ben Finney



From cs at zip.com.au  Fri Jun 19 23:14:55 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 20 Jun 2015 13:14:55 +1000
Subject: Catching exceptions with multi-processing
In-Reply-To: <mm1f9d$751$1@speranza.aioe.org>
References: <mm1f9d$751$1@speranza.aioe.org>
Message-ID: <20150620031455.GA74178@cskk.homeip.net>

On 19Jun2015 18:16, Fabien <fabien.maussion at gmail.com> wrote:
>On 06/19/2015 04:25 PM, Andres Riancho wrote:
>>    My recommendation is that you should pass some extra arguments to the task:
>>     * A unique task id
>>     * A result multiprocessing.Queue
>>
>>     When an exception is raised you put (unique_id, exception) to the
>>queue. When it succeeds you put (unique_id, None). In the main process
>>you consume the queue and do your error handling.
>>
>>     Note that some exceptions can't be serialized, there is where
>>tblib [0] comes handy.
>>
>>[0]https://pypi.python.org/pypi/tblib
>>
>>Regards,
>
>Thanks, I wasn't aware of the multiprocessing.Queue workflow. It seems 
>like its going to require some changes in the actual code of the tasks 
>though. Did I get it right that I should stop raising exceptions then?
>
>Something like:
>
>def task_1(path, q):
>    # Do stuffs
>    if dont_work:
>        q.put(RuntimeError("didnt work"))
>	return
>    # finished
>    q.put(None)
>    return

I would keep your core logic Pythonic, raise exceptions. But I would wrap each 
task in something to catch any Exception subclass and report back to the queue.  
Untested example:

  def subwrapper(q, callable, *args, **kwargs):
    try:
      q.put( ('COMPLETED', callable(*args, **kwargs)) )
    except Exception as e:
      q.put( ('FAILED', e, callable, args, kwargs) )

then dispatch tasks like this:

  pool.map(subwrapper, q, task1, dirs, chunksize=1)

and have a thread (or main program) collect things from the queue for logging 
and other handling. Obviously you might return something more sophisticated 
that my simple tuple above, but I'm sure you get the idea.

Cheers,
Cameron Simpson <cs at zip.com.au>

He's silly and he's ignorant, but he's got guts, and guts is enough.
        - Sgt. Hartmann


From dieter at handshake.de  Sat Jun 20 02:10:34 2015
From: dieter at handshake.de (dieter)
Date: Sat, 20 Jun 2015 08:10:34 +0200
Subject: instance as module
References: <5583E0EA.2090309@chamonix.reportlab.co.uk>
Message-ID: <877fqyucyt.fsf@handshake.de>

Robin Becker <robin at reportlab.com> writes:

> I'm trying to overcome a recursive import issue in reportlab.
> ... sketched solution ...

In the "zope" project, the same problem was approached in a slightly different
way -- see the product "zope.deferredimport". It allows to defer
an actual import for an imported name until this name is accessed.

Looking a bit deeper, I recognize that behind the scenes,
"zope.deferredimport" uses a technique similar to yours:
it replaces the original module by a proxy - which almost behaves
like the original module but has the additional capacity to do special things
for accessed to names not yet in the modules dictionary.

"zope.deferredproxy" is widely used in "zope" projects and I have not
seen any related problem report. Looks as its implementation technique
is rather safe.


You might be able to use "zope.deferredimport" directly for your purposes
or look at its implementation to learn what things the implementors
have taken care of. Or you might just learn from it, that replacing
a module by something similar is quite safe.

Dieter



From steve at pearwood.info  Sat Jun 20 03:24:40 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 20 Jun 2015 17:24:40 +1000
Subject: instance as module
References: <mailman.626.1434706168.13271.python-list@python.org>
Message-ID: <55851538$0$1667$c3e8da3$5496439d@news.astraweb.com>

On Fri, 19 Jun 2015 07:29 pm, Robin Becker wrote:

> I'm trying to overcome a recursive import issue in reportlab.
> 
> Module reportlab.rl_config uses various sources (eg ~/.reportlab_settings)
> to initialize various defaults eg canvas_basefontname. If a user wants to
> utilize reportlab to set up such a default, it's not possible to do so in
> the settings file because a recursive import would occur. Normal defaults
> are always primitive python objects eg float/int/str etc etc.

I'm afraid I don't understand what you are trying to say here. Why can't the
user just set up "such a default" e.g. canvas_basefontname. Surely that is
a string, e.g. "Comic Sans".

Why will a recursive import occur? If the user doesn't want a recursive
import, they need only ensure that they don't import anything that relies
on rl_config.


> If I make the rl_config module into an object

I'd just like to point out that modules are already objects.

py> import sys
py> isinstance(sys, object)
True



> (using the GvR blessed approach indicated here
> http://stackoverflow.com/questions/2447353/getattr-on-a-module) then I can
> use callables in the settings module and get them evaluated lazily which
> overcomes the recursion.
> 
> Are there any gotcha's that need to be considered when using this instance
> as module approach? My trial implementation seems to work, but it's not
> clear exactly how a module differs from an instance. 

Um, precisely the same way an int or a str or an instance of Foo class
differs from an instance. An instance of what?

> I have set various 
> dunders on the instance eg __file__, __doc__, __all__ & __name__ and I
> made the object a borg, but it still seems a bit hacky.

Making it a borg is one concrete difference. Modules are singletons, not
borgs, so anyone who does

    if the_thing is somemodule

may be surprised. I wouldn't necessarily worry about borgifying the module
proxy, since the import mechanics will ensure[1] it remains a singleton.

But frankly, this approach sounds complex and complicated, and we know what
the Zen says about them.




[1] For some definition of "ensure".


-- 
Steven



From fabien.maussion at gmail.com  Sat Jun 20 05:01:43 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Sat, 20 Jun 2015 11:01:43 +0200
Subject: Catching exceptions with multi-processing
References: <mm17as$i8m$1@speranza.aioe.org>
 <55843842$0$1644$c3e8da3$5496439d@news.astraweb.com>
 <mailman.647.1434747496.13271.python-list@python.org>
Message-ID: <mm3a5l$eb$1@speranza.aioe.org>

On 06/19/2015 10:58 PM, Chris Angelico wrote:
> AIUI what he's doing is all the subparts of task1 in parallel, then
> all the subparts of task2:
>
> pool.map(task1, dirs, chunksize=1)
> pool.map(task2, dirs, chunksize=1)
> pool.map(task3, dirs, chunksize=1)
>
> task1 can be done on all of dirs in parallel, as no instance of task1
> depends on any other instance of task1; but task2 should be started
> only if all task1s finish successfully. OP, is this how it is? If not,
> I apologize for the noise.

That's it! Thanks for clarifying, I might have trouble explaining myself 
sometimes ;-)

Fabien


From fabien.maussion at gmail.com  Sat Jun 20 05:03:51 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Sat, 20 Jun 2015 11:03:51 +0200
Subject: Catching exceptions with multi-processing
References: <mm1f9d$751$1@speranza.aioe.org>
 <mailman.649.1434771749.13271.python-list@python.org>
Message-ID: <mm3a9l$eb$2@speranza.aioe.org>

On 06/20/2015 05:14 AM, Cameron Simpson wrote:
> I would keep your core logic Pythonic, raise exceptions. But I would
> wrap each task in something to catch any Exception subclass and report
> back to the queue. Untested example:
>
>   def subwrapper(q, callable, *args, **kwargs):
>     try:
>       q.put( ('COMPLETED', callable(*args, **kwargs)) )
>     except Exception as e:
>       q.put( ('FAILED', e, callable, args, kwargs) )
>
> then dispatch tasks like this:
>
>   pool.map(subwrapper, q, task1, dirs, chunksize=1)
>
> and have a thread (or main program) collect things from the queue for
> logging and other handling. Obviously you might return something more
> sophisticated that my simple tuple above, but I'm sure you get the idea.
>
> Cheers,
> Cameron Simpson

Perfect! Much more elegant and easier to implement on top of my existing 
workflow based on raising exceptions.

thanks to all responses,

Fabien





From Cecil at decebal.nl  Sat Jun 20 06:58:33 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 20 Jun 2015 12:58:33 +0200
Subject: Working with jython in openSUSE 13.2
Message-ID: <87lhfer6hy.fsf@Equus.decebal.nl>

I installed Jython in openSUSE 13.2. But when calling jython I get:
    /usr/bin/build-classpath: error: JAVA_LIBDIR must be set
    Error: Could not find or load main class org.python.util.jython

Does anyone have an idea what I need to do to get it working?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From lac at openend.se  Sat Jun 20 09:01:12 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 20 Jun 2015 15:01:12 +0200
Subject: Working with jython in openSUSE 13.2
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Sat, 20 Jun 2015 12:58:33 +0200." <87lhfer6hy.fsf@Equus.decebal.nl>
References: <87lhfer6hy.fsf@Equus.decebal.nl>
Message-ID: <201506201301.t5KD1CE5004000@fido.openend.se>

In a message of Sat, 20 Jun 2015 12:58:33 +0200, Cecil Westerhof writes:
>I installed Jython in openSUSE 13.2. But when calling jython I get:
>    /usr/bin/build-classpath: error: JAVA_LIBDIR must be set
>    Error: Could not find or load main class org.python.util.jython
>
>Does anyone have an idea what I need to do to get it working?

Go look for something called java.conf

Mine is in /etc/java

It looks like this:

# System-wide Java configuration file -*- sh -*-
#
# JPackage Project <http://www.jpackage.org/>
# Location of jar files on the system
JAVA_LIBDIR=/usr/share/java
# Location of arch-specific jar files on the system
JNI_LIBDIR=/usr/lib/java
# List of known java homes (used for autodetection if none is provided)
JAVA_HOME_LIST=$JAVA_LIBDIR-utils/java_home.list
# Root of all JVM installations
JVM_ROOT=/usr/lib/jvm
# Default jvm
JAVA_HOME=$JVM_ROOT/java
# Options to pass to the java interpreter
JAVACMD_OPTS=

-------

This is the version for Suse, which I think is the one you ought
to have.  But I am not sure if this is where you ought to find it.

Laura


From Cecil at decebal.nl  Sat Jun 20 10:41:26 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 20 Jun 2015 16:41:26 +0200
Subject: Working with jython in openSUSE 13.2
References: <87lhfer6hy.fsf@Equus.decebal.nl>
 <mailman.658.1434805301.13271.python-list@python.org>
Message-ID: <87egl6qw6h.fsf@Equus.decebal.nl>

On Saturday 20 Jun 2015 15:01 CEST, Laura Creighton wrote:

> In a message of Sat, 20 Jun 2015 12:58:33 +0200, Cecil Westerhof
> writes:
>> I installed Jython in openSUSE 13.2. But when calling jython I get:
>> /usr/bin/build-classpath: error: JAVA_LIBDIR must be set
>> Error: Could not find or load main class org.python.util.jython
>>
>> Does anyone have an idea what I need to do to get it working?
>
> Go look for something called java.conf
>
> Mine is in /etc/java
>
> It looks like this:
>
> # System-wide Java configuration file -*- sh -*- JPackage Project
> #<http://www.jpackage.org/> Location of jar files on the system
> #JAVA_LIBDIR=/usr/share/java Location of arch-specific jar files on
> #the system JNI_LIBDIR=/usr/lib/java List of known java homes (used
> #for autodetection if none is provided)
> #JAVA_HOME_LIST=$JAVA_LIBDIR-utils/java_home.list Root of all JVM
> #installations JVM_ROOT=/usr/lib/jvm Default jvm
> #JAVA_HOME=$JVM_ROOT/java Options to pass to the java interpreter
> #JAVACMD_OPTS=
>
> -------
>
> This is the version for Suse, which I think is the one you ought
> to have.  But I am not sure if this is where you ought to find it.

Thanks. Some things where there, but commented out. And some things
where not there. I changed it and now I can run jython, but it is
version 2.2.1. So probably it is better to download and install the
latest version.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From davros at bellaliant.net  Sat Jun 20 10:59:29 2015
From: davros at bellaliant.net (John McKenzie)
Date: Sat, 20 Jun 2015 14:59:29 GMT
Subject: Keypress Input
References: <rBHbx.75089$MO7.40532@fx10.iad>
 <Tbtfx.178924$Jv.102633@fx26.iad> <mm08nk$caf$1@dont-email.me>
Message-ID: <ldfhx.15511$C_.5118@fx15.iad>


 Christian, are you suggesting I learn to do everything perfectly before 
I ask how to do everything perfectly?

 Despite your tone and insults I honestly appreciate the response. I know 
what to focus on and less than 5 minutes from now I will be looking for e-
books on the specific subjects you point me to and will have a better 
idea of where the issues are.




From ned at nedbatchelder.com  Sat Jun 20 11:01:10 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sat, 20 Jun 2015 08:01:10 -0700 (PDT)
Subject: Relation: a new standard (PEP) container?
In-Reply-To: <mailman.654.1434780628.13271.python-list@python.org>
References: <mailman.654.1434780628.13271.python-list@python.org>
Message-ID: <97f3a5c7-57dc-4c1e-a5b0-59d6bab42817@googlegroups.com>

On Saturday, June 20, 2015 at 2:10:50 AM UTC-4, Scott wrote:
> We've been using a simple container implementation of a mathematical relation (https://simple.wikipedia.org/wiki/Relation_(mathematics)) (i.e. an invertible M:M mapping) for some time.
> 
> ?
> We've been waiting for many years (decades actually) to have this concept incorporated as a standard container in one of the modern programming languages so we could swap our code with a language-standard container. ?To further these efforts, we have decided to initiate a conversation with the python community on adding a relation as a standard Python container package.
> ?
> Briefly the uses of a relation are:
> ?
> - quickly finding the values/range associated with keys/domain (e.g. *inversion*)
> - maintaining a unique relationship between keys and values,(e.g. *isomorphism* or *aliasing*)
> - using keys to categorize (one-to-many) values (e.g. *partitioning*)
> - associating two sets in an arbitrary/bipartite manner (e.g. *tagging*)
> ?
> Below is a link to an implementation, including an ipython notebook with light exposition and some canonical examples. ?We'll also be presenting a poster at SciPy 2015. Let us know if this, or something like it, seems like a worthwhile addition to the standard Python distribution.

When people propose new additions to the standard library, the ususal
question is, Why is it important to be in the standard library, why not
simply let people use it as a third-party package?

You've already made it available on PyPI.  People who need it can find
it there, right?

The standard library is extended very conservatively.  The Python-Ideas
mailing list is where previous discussions like this have take place. You
can search there for similar suggestions.

--Ned.


From davros at bellaliant.net  Sat Jun 20 11:02:58 2015
From: davros at bellaliant.net (John McKenzie)
Date: Sat, 20 Jun 2015 15:02:58 GMT
Subject: Keypress Input
References: <rBHbx.75089$MO7.40532@fx10.iad> <Ojtfx.178941$Jv.137884@fx26.iad>
 <mailman.519.1434474965.13271.python-list@python.org>
Message-ID: <Cgfhx.15512$C_.11002@fx15.iad>


 Guys, thanks for the various code examples for GPIO and the warning 
about debouncing issues. I am still considering going the route of more 
complex wiring and doing it a more traditional GPIO way.


From ned at nedbatchelder.com  Sat Jun 20 11:04:06 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sat, 20 Jun 2015 08:04:06 -0700 (PDT)
Subject: JSON Object to CSV Question
In-Reply-To: <mailman.652.1434780605.13271.python-list@python.org>
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
 <55842b8d$0$1674$c3e8da3$5496439d@news.astraweb.com>
 <CAC9q6M65WD70Wm0=QDqMD4mis4wSB3xkO-k8pnjn9u-+A3kPLA@mail.gmail.com>
 <CAB1GNpTPGoAkOYNHQV8HGNOQV8XjFcGft=mxgL+K=4_OD+zWkQ@mail.gmail.com>
 <CAC9q6M7uX0yeWoQQCJjD_vTwyfCaevLX4Y8-HctLcqYV--N60Q@mail.gmail.com>
 <CAB1GNpRjjqPa3aopSoagRrZYoff7OpNXa1YZrQ_VsFOZ0VyW0A@mail.gmail.com>
 <mailman.652.1434780605.13271.python-list@python.org>
Message-ID: <d0c5bac4-33ea-4356-8848-32bf962386c9@googlegroups.com>

On Saturday, June 20, 2015 at 2:10:51 AM UTC-4, Sahlusar wrote:
> @Joonas:
> 
> 
> The previous example was a typo. Please use the below example as a case study.?
> 
> 
> 
> 
> {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'], 'F_Int32': ['0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0'], 'OTF': '0', 'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'], 'SCS_String': ['1', '2']}
> 
> 
> All of the questions regarding XML I have asked. I have to work within their parameters. The CSV, for example may look like this:
> 
> 
> DLA,FC,PC,WC,CN,Description,Code,CMC?
> 0,00000,0,0,,,0,0

There isn't a single word in common between the Python dictionary you've
shown and the CSV sample output you've shown.  There's no way anyone can
tell what mapping you intend here.

--Ned.


From torriem at gmail.com  Sat Jun 20 12:30:37 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sat, 20 Jun 2015 10:30:37 -0600
Subject: Keypress Input
In-Reply-To: <Cgfhx.15512$C_.11002@fx15.iad>
References: <rBHbx.75089$MO7.40532@fx10.iad> <Ojtfx.178941$Jv.137884@fx26.iad>
 <mailman.519.1434474965.13271.python-list@python.org>
 <Cgfhx.15512$C_.11002@fx15.iad>
Message-ID: <5585952D.2090701@gmail.com>

On 06/20/2015 09:02 AM, John McKenzie wrote:
> 
>  Guys, thanks for the various code examples for GPIO and the warning 
> about debouncing issues. I am still considering going the route of more 
> complex wiring and doing it a more traditional GPIO way.

You can wire up the button without a little breadboard or circuit board
using this general diagram:

https://www.cl.cam.ac.uk/projects/raspberrypi/tutorials/robot/buttons_and_switches/

Just solder the resistors to the wires themselves, and keep them close
to the pi to keep them clean.  Then you you just have to route one wire
per switch, with one common wire hitting the other side of each switch.
 Probably you will want to use the pull-up version where the switch
pulls the signal down to ground.  You may want to use a header to plug
into the rpi.  But besides that should just be a matter of routing wires.

But on the original track, you might want to ask on the kade forums if
there's any way to interact with kade and get events besides HID
emulation.  For example, maybe you could communicate with it over a
virtual serial port.  Or some other protocol so you don't have to worry
about ttys and Linux kernel HID handling for your program, especially in
a headless environment.  Using HID emulation makes sense in a MAME
arcade environment but not for a headless raspberry pi application.




From ahlusar.ahluwalia at gmail.com  Sat Jun 20 13:27:55 2015
From: ahlusar.ahluwalia at gmail.com (Saran Ahluwalia)
Date: Sat, 20 Jun 2015 13:27:55 -0400
Subject: JSON Object to CSV Question
In-Reply-To: <d0c5bac4-33ea-4356-8848-32bf962386c9@googlegroups.com>
References: <mailman.581.1434610734.13271.python-list@python.org>
 <2k97oaprps5hm4q8gu723esgeopnmedj94@4ax.com>
 <mailman.627.1434707458.13271.python-list@python.org>
 <55842b8d$0$1674$c3e8da3$5496439d@news.astraweb.com>
 <CAC9q6M65WD70Wm0=QDqMD4mis4wSB3xkO-k8pnjn9u-+A3kPLA@mail.gmail.com>
 <CAB1GNpTPGoAkOYNHQV8HGNOQV8XjFcGft=mxgL+K=4_OD+zWkQ@mail.gmail.com>
 <CAC9q6M7uX0yeWoQQCJjD_vTwyfCaevLX4Y8-HctLcqYV--N60Q@mail.gmail.com>
 <CAB1GNpRjjqPa3aopSoagRrZYoff7OpNXa1YZrQ_VsFOZ0VyW0A@mail.gmail.com>
 <mailman.652.1434780605.13271.python-list@python.org>
 <d0c5bac4-33ea-4356-8848-32bf962386c9@googlegroups.com>
Message-ID: <CAC9q6M4RPoG7RJwAr89MRUBg+f61LiDX2VZo0=Mqrpk4Jt683w@mail.gmail.com>

Here is the dictionary that corresponds with the CSV printed to the console:


   1. {
   2. "PAC": {
   3. "Account": [{
   4. "PC": "0",
   5. "CMC": "0",
   6. "WC": "0",
   7. "DLA": "0",
   8. "CN": null,
   9. "FC": {
   10. "Int32": ["0",
   11. "0",
   12. "0",
   13. "0",
   14. "0"]
   15. },
   16. "F": {
   17. "Description": null,
   18. "Code": "0"
   19. }
   20. }]
   21. }
   22. }


On Sat, Jun 20, 2015 at 11:04 AM, Ned Batchelder <ned at nedbatchelder.com>
wrote:

> On Saturday, June 20, 2015 at 2:10:51 AM UTC-4, Sahlusar wrote:
> > @Joonas:
> >
> >
> > The previous example was a typo. Please use the below example as a case
> study.
> >
> >
> >
> >
> > {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
> 'F_Int32': ['0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',
> '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',  '0',
> '0',  '0',  '0'], 'OTF': '0', 'PBDS_Double': ['0', '0', '0', '0', '0', '0',
> '0', '0'], 'SCS_String': ['1', '2']}
> >
> >
> > All of the questions regarding XML I have asked. I have to work within
> their parameters. The CSV, for example may look like this:
> >
> >
> > DLA,FC,PC,WC,CN,Description,Code,CMC
> > 0,00000,0,0,,,0,0
>
> There isn't a single word in common between the Python dictionary you've
> shown and the CSV sample output you've shown.  There's no way anyone can
> tell what mapping you intend here.
>
> --Ned.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150620/790203f8/attachment.html>

From nad at acm.org  Sat Jun 20 17:39:54 2015
From: nad at acm.org (Ned Deily)
Date: Sat, 20 Jun 2015 14:39:54 -0700
Subject: (unknown)
References: <6264c933-0ed4-4055-baad-01b322bb6dd4@Calculus-Fantasticus-2.local>
Message-ID: <nad-3F7530.14395420062015@news.gmane.org>

In article 
<6264c933-0ed4-4055-baad-01b322bb6dd4 at Calculus-Fantasticus-2.local>,
 Scott <scott.howard.james at gmail.com> wrote:
> We've been using a simple container implementation of a mathematical relation 
> (https://simple.wikipedia.org/wiki/Relation_(mathematics)) (i.e. an 
> invertible M:M mapping) for some time.
[...]
> Below is a link to an implementation, including an ipython notebook with 
> light exposition and some canonical examples. We'll also be presenting a 
> poster at SciPy 2015. Let us know if this, or something like it, seems like a 
> worthwhile addition to the standard Python distribution.

Scott, James:

The best place to have a discussion about adding new features to Python 
is on the python-ideas mailing list:

https://mail.python.org/mailman/listinfo/python-ideas

-- 
 Ned Deily,
 nad at acm.org



From breamoreboy at yahoo.co.uk  Sat Jun 20 18:52:09 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 20 Jun 2015 23:52:09 +0100
Subject: Classic OOP in Python
In-Reply-To: <201506182301.t5IN1sdj012524@fido.openend.se>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net><49f789cf-cb67-4812-8a91-218a4f049958@googlegroups.com>
 <201506172233.t5HMXBIm002277@fido.openend.se><mlu7pl$u1t$1@ger.gmane.org>
 <201506182301.t5IN1sdj012524@fido.openend.se>
Message-ID: <mm4qqt$53m$1@ger.gmane.org>

On 19/06/2015 00:01, Laura Creighton wrote:
> In a message of Thu, 18 Jun 2015 11:50:28 +0100, Mark Lawrence writes:
>> Throw in http://clonedigger.sourceforge.net/ as well and you've a really
>> awesome combination.
>>
>> Mark Lawrence
>>
>
> I didn't know about that one.
> Hey thank you, Mark.  Looks great.
>
> It needs its own entry in
> https://wiki.python.org/moin/PythonTestingToolsTaxonomy
>
> You add it, or me?
>
> Laura
>

Thanks all the same but I'll leave it to you, as I suspect I'd be my 
usual combination of headless chicken and bull in a china shop if let 
loose on a wiki :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Sat Jun 20 20:29:56 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 21 Jun 2015 01:29:56 +0100
Subject: HOPE: A Python just-in-time compiler for astrophysical computations
Message-ID: <mm50i8$hbq$1@ger.gmane.org>

Another beasty I've just stumbled across which you may find interesting 
http://www.sciencedirect.com/science/article/pii/S2213133714000687

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From nma at 12000.org  Sat Jun 20 21:57:04 2015
From: nma at 12000.org (Nasser M. Abbasi)
Date: Sat, 20 Jun 2015 20:57:04 -0500
Subject: How to construct matrix from vectors?
Message-ID: <mm55le$4sj$1@speranza.aioe.org>

I just started to learn some python today for first time,
so be easy on me.

I am having some trouble figuring how do the problem shown in this link

http://12000.org/my_notes/mma_matlab_control/KERNEL/KEse44.htm

Given 4 column vectors, v1,v2,v3,v4, each is 3 rows.

I want to use these to construct matrix mat, which is

       [[v1,v2],
        [v3,v4]]

So the resulting matrix is as shown in the link.  i.e.
it will be 6 rows and 2 columns.

This is what I tried:

import numpy as np
v1=np.array([1,2,3]);
v2=np.array([4,5,6]);
v3=np.array([7,8,9]);
v4=np.array([10,11,12]);

And now I get stuck, I tried

m=np.array([[v1,v2],[v3,v4]])  #no good

Also

m=np.array([v1,v2,v3,v4])
m.shape
    Out[153]: (4L, 3L)
m.T

array([[ 1,  4,  7, 10],
        [ 2,  5,  8, 11],
        [ 3,  6,  9, 12]])

Not what I want.

I need to get the shape as in the above link, 6 rows by 2 columns,
where each column vector is stacked as shown.  I also tried

v1=np.array([1,2,3]); v1.shape=3,1
v2=np.array([4,5,6]); v2.shape=3,1
v3=np.array([7,8,9]); v3.shape=3,1
v4=np.array([10,11,12]); v4.shape=3,1
mat=np.array([[v1,v2],[v3,v4]])

What is the correct way to do this in Python?

thanks,
--Nasser


From rustompmody at gmail.com  Sat Jun 20 21:59:47 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sat, 20 Jun 2015 18:59:47 -0700 (PDT)
Subject: Lawful != Mutable (was Can Python function return multiple data?)
In-Reply-To: <mailman.258.1433694866.13271.python-list@python.org>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
 <mailman.212.1433569420.13271.python-list@python.org>
 <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>
 <4fdc38b3-6b51-4f98-9ba3-42b265582fc0@googlegroups.com>
 <mailman.258.1433694866.13271.python-list@python.org>
Message-ID: <762816ac-c744-480b-a9f9-a0110451c516@googlegroups.com>

On Sunday, June 7, 2015 at 10:04:37 PM UTC+5:30, Chris Angelico wrote:
> On Mon, Jun 8, 2015 at 2:20 AM, Rustom Mody  wrote:
> > Ok now rewrite that para above with
> > s/tuple/numbers like 3 or 666/
> > So I put '3' on the ram and grind it to finest powder.
> > Have all trinities (of religious or secular variety) disappeared?
> > 666 gone has the devil been banished from God's (or Steven's) universe?
> 
> If you write down your phone number and give it to a girl you like,
> and she burns that piece of paper, do you no longer have a phone
> number? No, but she certainly doesn't have your phone number any more.
> There's a difference between destroying a representation and
> destroying the original entity. But Steven's point wasn't about how
> easy/hard it is to destroy something; it was about a tuple's
> immutability NOT being a prevention of its destruction, and therefore
> it's not that sense of the word in which they're immutable. Similarly,
> you could destroy a document on which God's laws are written (in the
> original dictionary sense; if you don't want to believe in immutable
> laws of a Deity, you can substitute in a law of physics - for
> instance, the law that two objects with mass exert a force of
> attraction on each other due to gravity), and it wouldn't destroy the
> law itself.
> 
> ChrisA

Recent thread on python ideas
https://mail.python.org/pipermail/python-ideas/2015-June/034177.html

Since "python's immutable" ? "really immutable", we now need a "really immutable"


From python at mrabarnett.plus.com  Sat Jun 20 22:20:24 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 21 Jun 2015 03:20:24 +0100
Subject: How to construct matrix from vectors?
In-Reply-To: <mm55le$4sj$1@speranza.aioe.org>
References: <mm55le$4sj$1@speranza.aioe.org>
Message-ID: <55861F68.7000704@mrabarnett.plus.com>

On 2015-06-21 02:57, Nasser M. Abbasi wrote:
> I just started to learn some python today for first time,
> so be easy on me.
>
> I am having some trouble figuring how do the problem shown in this link
>
> http://12000.org/my_notes/mma_matlab_control/KERNEL/KEse44.htm
>
> Given 4 column vectors, v1,v2,v3,v4, each is 3 rows.
>
> I want to use these to construct matrix mat, which is
>
>         [[v1,v2],
>          [v3,v4]]
>
> So the resulting matrix is as shown in the link.  i.e.
> it will be 6 rows and 2 columns.
>
> This is what I tried:
>
> import numpy as np
> v1=np.array([1,2,3]);
> v2=np.array([4,5,6]);
> v3=np.array([7,8,9]);
> v4=np.array([10,11,12]);
>
> And now I get stuck, I tried
>
> m=np.array([[v1,v2],[v3,v4]])  #no good
>
> Also
>
> m=np.array([v1,v2,v3,v4])
> m.shape
>      Out[153]: (4L, 3L)
> m.T
>
> array([[ 1,  4,  7, 10],
>          [ 2,  5,  8, 11],
>          [ 3,  6,  9, 12]])
>
> Not what I want.
>
> I need to get the shape as in the above link, 6 rows by 2 columns,
> where each column vector is stacked as shown.  I also tried
>
> v1=np.array([1,2,3]); v1.shape=3,1
> v2=np.array([4,5,6]); v2.shape=3,1
> v3=np.array([7,8,9]); v3.shape=3,1
> v4=np.array([10,11,12]); v4.shape=3,1
> mat=np.array([[v1,v2],[v3,v4]])
>
> What is the correct way to do this in Python?
>
Here's one way, one step at a time:

>>> r1 = np.concatenate([v1, v2])
>>> r1
array([1, 2, 3, 4, 5, 6])
>>> r2 = np.concatenate([v3, v4])
>>> r2
array([ 7,  8,  9, 10, 11, 12])
>>> m = np.array([r1, r2])
>>> m
array([[ 1,  2,  3,  4,  5,  6],
        [ 7,  8,  9, 10, 11, 12]])
>>> m.transpose()
array([[ 1,  7],
        [ 2,  8],
        [ 3,  9],
        [ 4, 10],
        [ 5, 11],
        [ 6, 12]])
>>>



From rosuav at gmail.com  Sat Jun 20 22:32:23 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 21 Jun 2015 12:32:23 +1000
Subject: Lawful != Mutable (was Can Python function return multiple data?)
In-Reply-To: <762816ac-c744-480b-a9f9-a0110451c516@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
 <mailman.212.1433569420.13271.python-list@python.org>
 <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>
 <4fdc38b3-6b51-4f98-9ba3-42b265582fc0@googlegroups.com>
 <mailman.258.1433694866.13271.python-list@python.org>
 <762816ac-c744-480b-a9f9-a0110451c516@googlegroups.com>
Message-ID: <CAPTjJmq9d0o7FLvaYY9NrG1_sd+h_eToRZbKOk8A6TaoAfuVrg@mail.gmail.com>

On Sun, Jun 21, 2015 at 11:59 AM, Rustom Mody <rustompmody at gmail.com> wrote:
> Recent thread on python ideas
> https://mail.python.org/pipermail/python-ideas/2015-June/034177.html
>
> Since "python's immutable" ? "really immutable", we now need a "really immutable"

That's because it requires mutable memory to keep track of reference
counts. In a non-refcountinig Python, strings and integers can be
truly immutable; that thread is suggesting (among other options)
simply storing the refcounts in a separate table. Conceptually, a
string or integer IS immutable, but CPython currently needs to be able
to fiddle with some bookkeeping data about them, in order to pretend
it has plenty of memory. A Python implementation could theoretically
just abandon its strings whenever it's done with them, wasting gobs of
memory in the process, and having them in read-only memory. More
practically, a pure mark-and-sweep GC (such as MicroPython uses)
trades prompt disposal for simple management, and in doing so,
achieves (I think) the same total immutability.

You have to distinguish between Python (the language) and CPython (the
implementation) here. Python's notion of mutability has nothing to do
with thread safety. You could achieve perfect thread safety for
immutables by simply replicating them into every thread; it'd be
inefficient, but perfectly legal.

ChrisA


From rustompmody at gmail.com  Sat Jun 20 22:50:21 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sat, 20 Jun 2015 19:50:21 -0700 (PDT)
Subject: Lawful != Mutable (was Can Python function return multiple data?)
In-Reply-To: <mailman.666.1434853946.13271.python-list@python.org>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
 <mailman.212.1433569420.13271.python-list@python.org>
 <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>
 <4fdc38b3-6b51-4f98-9ba3-42b265582fc0@googlegroups.com>
 <mailman.258.1433694866.13271.python-list@python.org>
 <762816ac-c744-480b-a9f9-a0110451c516@googlegroups.com>
 <mailman.666.1434853946.13271.python-list@python.org>
Message-ID: <5037b4a5-94a9-4995-828f-9624804da20a@googlegroups.com>

On Sunday, June 21, 2015 at 8:03:18 AM UTC+5:30, Chris Angelico wrote:
> On Sun, Jun 21, 2015 at 11:59 AM, Rustom Mody wrote:
> > Recent thread on python ideas
> > https://mail.python.org/pipermail/python-ideas/2015-June/034177.html
> >
> > Since "python's immutable" ? "really immutable", we now need a "really immutable"
> 
> That's because it requires mutable memory to keep track of reference
> counts. In a non-refcountinig Python, strings and integers can be
> truly immutable; that thread is suggesting (among other options)
> simply storing the refcounts in a separate table. Conceptually, a
> string or integer IS immutable, but CPython currently needs to be able
> to fiddle with some bookkeeping data about them, in order to pretend
> it has plenty of memory. A Python implementation could theoretically
> just abandon its strings whenever it's done with them, wasting gobs of
> memory in the process, and having them in read-only memory. More
> practically, a pure mark-and-sweep GC (such as MicroPython uses)
> trades prompt disposal for simple management, and in doing so,
> achieves (I think) the same total immutability.
> 
> You have to distinguish between Python (the language) and CPython (the
> implementation) here. Python's notion of mutability has nothing to do
> with thread safety. You could achieve perfect thread safety for
> immutables by simply replicating them into every thread; it'd be
> inefficient, but perfectly legal.
> 
> ChrisA

Here is Eric Snow:

| Keep in mind that by "immutability" I'm talking about *really*
| immutable, perhaps going so far as treating the full memory space
| associated with an object as frozen.  For instance, we'd have to
| ensure that "immutable" Python objects like strings, ints, and tuples
| do not change (i.e. via the C API).  The contents of involved
| tuples/containers would have to be likewise immutable.  Even changing
| refcounts could be too much, hence the idea of moving refcounts out to
| a separate table.
|  
| This level of immutability would be something new to Python.  We'll
| see if it's necessary.  If it isn't too much work it might be a good
| idea regardless of the multi-core proposal.

Does the second para look like CPython implementation or python-the-language?

Also note the 'Even' in the first para. ie Eric is talking of low-level
(ie thread-safety, refcounting etc) immutability after the even and higher
level semantic immutability before


From nmichalowsky at gmail.com  Sat Jun 20 23:14:14 2015
From: nmichalowsky at gmail.com (Naftali)
Date: Sat, 20 Jun 2015 20:14:14 -0700 (PDT)
Subject: Opening PDF Using subprocess.Popen Failing
In-Reply-To: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
References: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
Message-ID: <7710dd71-6f37-4d1b-b868-bc233d4da3b7@googlegroups.com>

On Friday, June 19, 2015 at 1:25:12 PM UTC-4, Naftali wrote:
> It actually doesn't fail but it 'cannot open in protected mode' (see here http://blogs.adobe.com/dmcmahon/2012/07/27/adobe-reader-cannot-open-protected-mode-due-to-a-problem-with-your-system-configuration/)
> 
> I am using subprocess.Popen("AcroRe32.exe /n <file.pdf>") which is the actuall adobe reader command I'd issue on the command line to open the pdf (the /n option opens it the file in a new instance of reader).
> 
> Now, when I issue the command straight from powershell, the pdf opens no problem, but when I open in my script (whether a .py or py2exe) I get the pop up complaining that the PDF cannot be opened in 'protected mode.' One of the options is to open it anyways, which works. 
> 
> Looking into it (see the link in the first paragraph) my best guess is that it's due to something like "JS-invoked processes: Launching a process through JavaScript is not allowed with Protected Mode enabled." 
> 
> But my naive understanding was that when I give Popen instruction, the command is handed off to windows and the called program is unaware of how it got called, so my thinking is that either that is incorrect or windows somehow 'cooperates' with reader to figure things out. 
> 
> I am looking for *any* insight as to how to deal with this, and the 'turn off protected mode" option wont work for me. 
> 
> Here is my code,
> 
> outputname = " unlocked.pdf"
> 
> commandstr = "qpdf --decrypt " + sys.argv[1] + outputname
> os.system(commandstr)
> 
> new_command_str = "AcroRd32.exe /n" + outputname
> subprocess.Popen(new_command_str)
> 
> sys.exit(0)


I am running the script via powershell. that sounds very promising. I'm going to read the link Laura pointed to upthread and see what happens outside powershell on Monday when I get back to the windows environment. 

But thank you for the heads up cause that makes a lot sense. 


From nma at 12000.org  Sat Jun 20 23:47:26 2015
From: nma at 12000.org (Nasser M. Abbasi)
Date: Sat, 20 Jun 2015 22:47:26 -0500
Subject: How to construct matrix from vectors?
References: <mm55le$4sj$1@speranza.aioe.org>
 <mailman.665.1434853238.13271.python-list@python.org>
Message-ID: <mm5c4c$fmm$1@speranza.aioe.org>

On 6/20/2015 9:20 PM, MRAB wrote:

> Here's one way, one step at a time:
>
>>>> r1 = np.concatenate([v1, v2])
>>>> r1
> array([1, 2, 3, 4, 5, 6])
>>>> r2 = np.concatenate([v3, v4])
>>>> r2
> array([ 7,  8,  9, 10, 11, 12])
>>>> m = np.array([r1, r2])
>>>> m
> array([[ 1,  2,  3,  4,  5,  6],
>          [ 7,  8,  9, 10, 11, 12]])
>>>> m.transpose()
> array([[ 1,  7],
>          [ 2,  8],
>          [ 3,  9],
>          [ 4, 10],
>          [ 5, 11],
>          [ 6, 12]])
>>>>
>

But your output is wrong.

I did manage to find a way:

---------------------------------
r1 =np.hstack([(v1,v2)]).T
r2 =np.hstack([(v3,v4)]).T
mat = np.vstack((r1,r2))
-----------------------------

Out[211]:
array([[ 1,  4],
        [ 2,  5],
        [ 3,  6],
        [ 7, 10],
        [ 8, 11],
        [ 9, 12]])

But it is not as intuitive as with Matlab, where one can just write

-------------------------------
   v1=[1,2,3]'; v2=[4,5,6]';
   v3=[7,8,9]'; v4=[10,11,12]';
   m=[v1 v2;v3 v4]
-------------------------------

--Nasser





From miki.tebeka at gmail.com  Sun Jun 21 00:14:14 2015
From: miki.tebeka at gmail.com (Miki Tebeka)
Date: Sat, 20 Jun 2015 21:14:14 -0700 (PDT)
Subject: Time saving tips for Pythonists
In-Reply-To: <mailman.590.1434622221.13271.python-list@python.org>
References: <mailman.590.1434622221.13271.python-list@python.org>
Message-ID: <5e57960a-4cd7-4ebe-9104-bcef462b1b61@googlegroups.com>

> What are your best time saving tips when programming Python?
* Use the REPL. Write small chunks of code and test them as you go
* Know what's available in the standard library (sets, Counter, deque ...)
* Learn how to pick good packages from PyPI (community, last commit ...)
* import this


From nma at 12000.org  Sun Jun 21 01:21:34 2015
From: nma at 12000.org (Nasser M. Abbasi)
Date: Sun, 21 Jun 2015 00:21:34 -0500
Subject: How to construct matrix from vectors?
References: <mm55le$4sj$1@speranza.aioe.org>
 <mailman.665.1434853238.13271.python-list@python.org>
 <mm5c4c$fmm$1@speranza.aioe.org>
Message-ID: <mm5hks$p52$1@speranza.aioe.org>

On 6/20/2015 10:47 PM, Nasser M. Abbasi wrote:

> I did manage to find a way:
>
> ---------------------------------
> r1 =np.hstack([(v1,v2)]).T
> r2 =np.hstack([(v3,v4)]).T
> mat = np.vstack((r1,r2))
> -----------------------------
>
> Out[211]:
> array([[ 1,  4],
>          [ 2,  5],
>          [ 3,  6],
>          [ 7, 10],
>          [ 8, 11],
>          [ 9, 12]])
>
> But it is not as intuitive as with Matlab, where one can just write
>
> -------------------------------
>     v1=[1,2,3]'; v2=[4,5,6]';
>     v3=[7,8,9]'; v4=[10,11,12]';
>     m=[v1 v2;v3 v4]
> -------------------------------

Here is a way a little closer to Matlab's method: First
make all the vectors column vectors

v1=np.array([(1,2,3)]).T
v2=np.array([(4,5,6)]).T
v3=np.array([(7,8,9)]).T
v4=np.array([(10,11,12)]).T

mat =np.hstack(( np.vstack((v1,v3)), np.vstack((v2,v4))) )

Out[236]:
array([[ 1,  4],
        [ 2,  5],
        [ 3,  6],
        [ 7, 10],
        [ 8, 11],
        [ 9, 12]])

There are way too many '(([[]]))' things  in Python :)




From denismfmcmahon at gmail.com  Sun Jun 21 02:45:59 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sun, 21 Jun 2015 06:45:59 +0000 (UTC)
Subject: JSON Object to CSV file
References: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
Message-ID: <mm5mj6$5m8$3@dont-email.me>

On Wed, 17 Jun 2015 08:00:11 -0700, Saran A wrote:

> I would like to have this JSON object written out to a CSV file so that
> the keys are header fields (for each of the columns) and the values are
> values that are associated with each header field.

> {
> "CF": {
.......
> "CF": "Fee",

Your json object seems to have the same key used for two elements at the 
same level, are you sure this is legal json?

-- 
Denis McMahon, denismfmcmahon at gmail.com


From denismfmcmahon at gmail.com  Sun Jun 21 02:50:11 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sun, 21 Jun 2015 06:50:11 +0000 (UTC)
Subject: JSON Object to CSV File Troubleshooting
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
Message-ID: <mm5mr3$5m8$4@dont-email.me>

On Thu, 18 Jun 2015 18:47:30 -0700, Sahlusar wrote:

> I have a conundrum regarding JSON objects and converting them to CSV:
> 
> Context
> 
> I am converting XML files to a JSON object (please see snippet below)
> and then finally producing a CSV file. Here is a an example JSON object:

This is where you're going wrong. If you want CSV data, take the XML and 
generate CSV data from it. Converting and writing it out to JSON and then 
reading it back and converting to CSV involves an extra conversion step 
where errors can creep in.

If you want to convert XML to CSV, go straight from XML to CSV, there is 
little added value in using some arbitrary intermediate format unless 
you're actually going to use the data in the intermediate format for 
something other than converting to the final format.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From denismfmcmahon at gmail.com  Sun Jun 21 02:53:18 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sun, 21 Jun 2015 06:53:18 +0000 (UTC)
Subject: JSON Object to CSV File Troubleshooting
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
Message-ID: <mm5n0u$5m8$5@dont-email.me>

On Thu, 18 Jun 2015 18:47:30 -0700, Sahlusar wrote:

> I have a conundrum regarding JSON objects and converting them to CSV:

I think your conundrum is that you've taken on a coding task beyond your 
abilities to comprehend, and as a result not only can you not code it, 
you can't even adequately describe it.

At least, it seems that every time you do try and describe it either the 
data format or the task description changes.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From Cecil at decebal.nl  Sun Jun 21 03:53:23 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 21 Jun 2015 09:53:23 +0200
Subject: Jython and can't write cache file
Message-ID: <874mm1qyz0.fsf@Equus.decebal.nl>

On openSUSE there is a very old version of Jython installed (2.2.1),
so I installed the latest version (2.7.0). But when starting this I
get:
    *sys-package-mgr*: can't write cache file for '/var/lib/h2/h2-1.3.176/bin/h2-1.3.176.jar'
    *sys-package-mgr*: can't write cache file for '/home/cecil/java/jars/sqlite-jdbc.jar'

Is that something to worry about?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun 21 03:56:37 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 21 Jun 2015 09:56:37 +0200
Subject: No =?utf-8?Q?=E2=80=98from?= __future__ import =?utf-8?Q?print=5F?=
 =?utf-8?Q?function=E2=80=99?= in latest jython
Message-ID: <87zj3tpk96.fsf@Equus.decebal.nl>

When I execute in the latest Jython:
    from __future__ import print_function
I get:
    Traceback (innermost last):
      (no code object) at line 0
      File "<console>", line 1
    SyntaxError: future feature print_function is not defined

This is not implemented in the latest jython? I prefer to keep all my
code as far as possible the same. The closer I get to DRY, the happier
I am. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun 21 04:09:02 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 21 Jun 2015 10:09:02 +0200
Subject: No =?utf-8?Q?=E2=80=98from?= __future__ import =?utf-8?Q?prin?=
 =?utf-8?Q?t=5Ffunction=E2=80=99?= in latest jython
References: <87zj3tpk96.fsf@Equus.decebal.nl>
Message-ID: <87vbehpjoh.fsf@Equus.decebal.nl>

On Sunday 21 Jun 2015 09:56 CEST, Cecil Westerhof wrote:

> When I execute in the latest Jython:
> from __future__ import print_function
> I get:
> Traceback (innermost last):
> (no code object) at line 0
> File "<console>", line 1
> SyntaxError: future feature print_function is not defined
>
> This is not implemented in the latest jython? I prefer to keep all
> my code as far as possible the same. The closer I get to DRY, the
> happier I am. ;-)

My bad, I started the old version instead of the latest when I tested
this. Sorry for the noise.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun 21 04:12:06 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 21 Jun 2015 10:12:06 +0200
Subject: How to check in script if Python or Jython is used
Message-ID: <87r3p5pjjd.fsf@Equus.decebal.nl>

I installed Jython and will start playing with it. There probably will
be differences between Python and Jython. Is there a way to determine
if a script is run by Python or Jython? Then different execution paths
could be taken. With sys.version(_info) you do not get this
information.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun 21 04:14:15 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 21 Jun 2015 10:14:15 +0200
Subject: Using Jython for Android development?
Message-ID: <87mvztpjfs.fsf@Equus.decebal.nl>

I have no experience yet with Jython or Android development. But I was
wondering: would it be possible to write applications for Android with
Jython? You normally use Java for it, but I think I would like Jython
more. :-D

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From lac at openend.se  Sun Jun 21 04:26:09 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 21 Jun 2015 10:26:09 +0200
Subject: Classic OOP in Python
In-Reply-To: Message from Mark Lawrence <breamoreboy@yahoo.co.uk>
 of "Sat, 20 Jun 2015 23:52:09 +0100." <mm4qqt$53m$1@ger.gmane.org>
References: <23b2fae3-91d5-486b-9898-78b34bee486f@googlegroups.com>
 <16dc8c92-50e3-4fc8-ae25-cbc5b13e8971@googlegroups.com>
 <87eglaqdca.fsf@elektro.pacujo.net><49f789cf-cb67-4812-8a91-218a4f049958@googlegroups.com>
 <201506172233.t5HMXBIm002277@fido.openend.se><mlu7pl$u1t$1@ger.gmane.org>
 <201506182301.t5IN1sdj012524@fido.openend.se><mm4qqt$53m$1@ger.gmane.org>
Message-ID: <201506210826.t5L8Q9tk029101@fido.openend.se>

Ah, turns out there was an entry.  I updated it.

Laura



From breamoreboy at yahoo.co.uk  Sun Jun 21 04:32:55 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 21 Jun 2015 09:32:55 +0100
Subject: How to construct matrix from vectors?
In-Reply-To: <mm5c4c$fmm$1@speranza.aioe.org>
References: <mm55le$4sj$1@speranza.aioe.org>
 <mailman.665.1434853238.13271.python-list@python.org>
 <mm5c4c$fmm$1@speranza.aioe.org>
Message-ID: <mm5srr$7k7$1@ger.gmane.org>

On 21/06/2015 04:47, Nasser M. Abbasi wrote:

>
> But it is not as intuitive as with Matlab
>

For those of us who don't know would you be kind enough to do a cost 
comparison of Matlab vs Python licenses?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From fabien.maussion at gmail.com  Sun Jun 21 04:49:14 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Sun, 21 Jun 2015 10:49:14 +0200
Subject: How to construct matrix from vectors?
References: <mm55le$4sj$1@speranza.aioe.org>
 <mailman.665.1434853238.13271.python-list@python.org>
 <mm5c4c$fmm$1@speranza.aioe.org> <mm5hks$p52$1@speranza.aioe.org>
Message-ID: <mm5tq9$gtp$1@speranza.aioe.org>

On 06/21/2015 07:21 AM, Nasser M. Abbasi wrote:
> v1=np.array([(1,2,3)]).T
> v2=np.array([(4,5,6)]).T
> v3=np.array([(7,8,9)]).T
> v4=np.array([(10,11,12)]).T
>
> mat =np.hstack(( np.vstack((v1,v3)), np.vstack((v2,v4))) )
>
> Out[236]:
> array([[ 1,  4],
>         [ 2,  5],
>         [ 3,  6],
>         [ 7, 10],
>         [ 8, 11],
>         [ 9, 12]])
>
> There are way too many '(([[]]))' things  in Python :)

another solution with less "(([[]]))", and less ";". There are way too 
many ";" in Matlab ;)

import numpy as np
v1 = [1, 2, 3]
v2 = [4, 5, 6]
v3 = [7, 8, 9]
v4 = [10, 11, 12]
np.hstack([[v1, v2], [v3, v4]]).T
Out[]:
array([[ 1,  4],
        [ 2,  5],
        [ 3,  6],
        [ 7, 10],
        [ 8, 11],
        [ 9, 12]])



From lac at openend.se  Sun Jun 21 04:56:43 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 21 Jun 2015 10:56:43 +0200
Subject: Jython and can't write cache file
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Sun, 21 Jun 2015 09:53:23 +0200." <874mm1qyz0.fsf@Equus.decebal.nl>
References: <874mm1qyz0.fsf@Equus.decebal.nl>
Message-ID: <201506210856.t5L8uhAg030040@fido.openend.se>

In a message of Sun, 21 Jun 2015 09:53:23 +0200, Cecil Westerhof writes:
>On openSUSE there is a very old version of Jython installed (2.2.1),
>so I installed the latest version (2.7.0). But when starting this I
>get:
>    *sys-package-mgr*: can't write cache file for '/var/lib/h2/h2-1.3.176/bin/h2-1.3.176.jar'
>    *sys-package-mgr*: can't write cache file for '/home/cecil/java/jars/sqlite-jdbc.jar'
>
>Is that something to worry about?
>
>-- 
>Cecil Westerhof
>Senior Software Engineer
>LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>-- 
>https://mail.python.org/mailman/listinfo/python-list

Yes.

Read this:
http://www.prasannatech.net/2009/02/jython-sys-package-mgr-processing-jar.html

Laura


From lac at openend.se  Sun Jun 21 05:05:18 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 21 Jun 2015 11:05:18 +0200
Subject: No =?utf-8?Q?=E2=80=98from?= __future__ import
 =?utf-8?Q?print=5F?= =?utf-8?Q?function=E2=80=99?= in latest jython
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Sun, 21 Jun 2015 09:56:37 +0200." <87zj3tpk96.fsf@Equus.decebal.nl>
References: <87zj3tpk96.fsf@Equus.decebal.nl>
Message-ID: <201506210905.t5L95I73030252@fido.openend.se>

Do you have Jython 2.7 released a few weeks ago?

Laura



From lac at openend.se  Sun Jun 21 05:14:19 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 21 Jun 2015 11:14:19 +0200
Subject: Lawful != Mutable (was Can Python function return multiple data?)
In-Reply-To: Message from Rustom Mody <rustompmody@gmail.com> of "Sat,
 20 Jun 2015 19:50:21 -0700."
 <5037b4a5-94a9-4995-828f-9624804da20a@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.85.1433289398.13271.python-list@python.org>
 <mkn139$6hb$1@r01.glglgl.de>
 <CAPTjJmqQ339eamz7dBoKP83JA8QB5Z4R67T4anzgy+ZrhiscDg@mail.gmail.com>
 <mailman.133.1433368610.13271.python-list@python.org>
 <e43a00ed-2b7d-497c-aa95-e30cbc04c04a@googlegroups.com>
 <557056f9$0$13009$c3e8da3$5496439d@news.astraweb.com>
 <mailman.168.1433438784.13271.python-list@python.org>
 <5570ce43$0$12991$c3e8da3$5496439d@news.astraweb.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com> <mai!
 lman.212.1433569420.13271.python-list@python.org>
 <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>
 <4fdc38b3-6b51-4f98-9ba3-42b265582fc0@googlegroups.com>
 <mailman.258.1433694866.13271.python-list@python.org>
 <762816ac-c744-480b-a9f9-a0110451c516@googlegroups.com>
 <mailman.666.1434853946.13271.python-list@python.org><5037b4a5-94a9-4995-828f-9624804da20a@googlegroups.com>
Message-ID: <201506210914.t5L9EJ1X030483@fido.openend.se>

In a message of Sat, 20 Jun 2015 19:50:21 -0700, Rustom Mody writes:
>Here is Eric Snow:
>
>| Keep in mind that by "immutability" I'm talking about *really*
>| immutable, perhaps going so far as treating the full memory space
>| associated with an object as frozen.  For instance, we'd have to
>| ensure that "immutable" Python objects like strings, ints, and tuples
>| do not change (i.e. via the C API).  The contents of involved
>| tuples/containers would have to be likewise immutable.  Even changing
>| refcounts could be too much, hence the idea of moving refcounts out to
>| a separate table.
>|  
>| This level of immutability would be something new to Python.  We'll
>| see if it's necessary.  If it isn't too much work it might be a good
>| idea regardless of the multi-core proposal.
>
>Does the second para look like CPython implementation or python-the-language?

Regardless of how it looks, it's only about the CPython implementation.
PyPy and Jython will not have this problem, as they don't have a C API
to worry about.  Of course the PyPy position is that STM will solve
all of this in a much cleaner way, and we're getting closer to a
perfectly working solution for as many cores as you have (as opposed
to only 4) that works on all OS's (as opposed to just 64 bit linux)
all the time.

This is very much a 'how to hack CPython to support multiple cores'
idea.  It's a 'kill the GIL' proposal.  But PyPy-STM and Jython
don't have a GIL, so they don't need to kill it.

>Also note the 'Even' in the first para. ie Eric is talking of low-level
>(ie thread-safety, refcounting etc) immutability after the even and higher
>level semantic immutability before

The ref-counting should be a very strong hint that this is a CPython
implementation proposal.  Again, Jython and PyPy don't use a ref counting
GC.

Laura



From lac at openend.se  Sun Jun 21 05:22:56 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 21 Jun 2015 11:22:56 +0200
Subject: How to check in script if Python or Jython is used
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Sun, 21 Jun 2015 10:12:06 +0200." <87r3p5pjjd.fsf@Equus.decebal.nl>
References: <87r3p5pjjd.fsf@Equus.decebal.nl>
Message-ID: <201506210922.t5L9MusK030722@fido.openend.se>

In a message of Sun, 21 Jun 2015 10:12:06 +0200, Cecil Westerhof writes:
>I installed Jython and will start playing with it. There probably will
>be differences between Python and Jython. Is there a way to determine
>if a script is run by Python or Jython? Then different execution paths
>could be taken. With sys.version(_info) you do not get this
>information.
>
>-- 
>Cecil Westerhof
>Senior Software Engineer
>LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>-- 
>https://mail.python.org/mailman/listinfo/python-list

import platform
platform.python_implementation()

If your jython is old (pre 2.6) you will not have this.

Then try
platform.system()
which will give you 'Java'

Laura




From lac at openend.se  Sun Jun 21 05:26:30 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 21 Jun 2015 11:26:30 +0200
Subject: Using Jython for Android development?
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Sun, 21 Jun 2015 10:14:15 +0200." <87mvztpjfs.fsf@Equus.decebal.nl>
References: <87mvztpjfs.fsf@Equus.decebal.nl>
Message-ID: <201506210926.t5L9QUsx030814@fido.openend.se>

In a message of Sun, 21 Jun 2015 10:14:15 +0200, Cecil Westerhof writes:
>I have no experience yet with Jython or Android development. But I was
>wondering: would it be possible to write applications for Android with
>Jython? You normally use Java for it, but I think I would like Jython
>more. :-D
>
>-- 
>Cecil Westerhof
>Senior Software Engineer
>LinkedIn: http://www.linkedin.com/in/cecilwesterhof

You cannot use Jython for this at present.  PyPy for android coming.
(Works great on ARM, but oh, the environment changes to get the
libtraries to work. )

In the meatime, use kivy.  Actually, use kivy anyway, it is just too
much fun. :)

http://kivy.org/#home

Laura


From bc at freeuk.com  Sun Jun 21 05:29:32 2015
From: bc at freeuk.com (BartC)
Date: Sun, 21 Jun 2015 10:29:32 +0100
Subject: HOPE: A Python just-in-time compiler for astrophysical
 computations
In-Reply-To: <mailman.664.1434846609.13271.python-list@python.org>
References: <mailman.664.1434846609.13271.python-list@python.org>
Message-ID: <2uvhx.910489$G53.315907@fx13.am4>

On 21/06/2015 01:29, Mark Lawrence wrote:
> Another beasty I've just stumbled across which you may find interesting
> http://www.sciencedirect.com/science/article/pii/S2213133714000687

Blimey, that's a lot of waffle in there, but I suppose that's to be 
expected from a published paper.

I think the gist of it is, that you highlight specific Python functions 
that you need to be fast (add a decorator), then it tries to translate 
those into actual C++ by inferring types. All done transparently at 
runtime (although I imagine it would be hard to hide the huge machinery 
of a C++ compiler in action).

The benchmarks seem to be individual functions which it presumably 
successfully translated fully into C++, so it is effectively comparing 
CPython to C++.

It also puts in a good dig at PyPy by including one benchmark where it 
is 6 times as slow as CPython!

It's not clear why it's particularly useful for astrophysics.

-- 
Bartc


From Cecil at decebal.nl  Sun Jun 21 05:40:37 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 21 Jun 2015 11:40:37 +0200
Subject: Jython and can't write cache file
References: <874mm1qyz0.fsf@Equus.decebal.nl>
 <mailman.670.1434877014.13271.python-list@python.org>
Message-ID: <87pp4p8kmi.fsf@Equus.decebal.nl>

On Sunday 21 Jun 2015 10:56 CEST, Laura Creighton wrote:

> In a message of Sun, 21 Jun 2015 09:53:23 +0200, Cecil Westerhof
> writes:
>> On openSUSE there is a very old version of Jython installed
>> (2.2.1), so I installed the latest version (2.7.0). But when
>> starting this I get: *sys-package-mgr*: can't write cache file for
>> '/var/lib/h2/h2-1.3.176/bin/h2-1.3.176.jar' *sys-package-mgr*:
>> can't write cache file for '/home/cecil/java/jars/sqlite-jdbc.jar'
>>
>> Is that something to worry about?
>>
>> -- 
>> Cecil Westerhof
>> Senior Software Engineer
>> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>
> Yes.
>
> Read this:
> http://www.prasannatech.net/2009/02/jython-sys-package-mgr-processing-jar.html

Thanks. Good that I asked it. :-D

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun 21 06:21:03 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 21 Jun 2015 12:21:03 +0200
Subject: No =?utf-8?Q?=E2=80=98from?= __future__ import =?utf-8?Q?prin?=
 =?utf-8?Q?t=5Ffunction=E2=80=99?= in latest jython
References: <87zj3tpk96.fsf@Equus.decebal.nl>
 <mailman.671.1434877523.13271.python-list@python.org>
Message-ID: <87h9q18ir4.fsf@Equus.decebal.nl>

On Sunday 21 Jun 2015 11:05 CEST, Laura Creighton wrote:

> Do you have Jython 2.7 released a few weeks ago?

Yes, but I was dumb enough to start the old version when I did this.
:-(

There is still one problem:
======================================================================
>>> from __future__ import print_function
>>> print()
()
======================================================================

While it should be:
======================================================================
>>> from __future__ import print_function
>>> print()

======================================================================

But I think I can live with it for the moment.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun 21 06:24:01 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 21 Jun 2015 12:24:01 +0200
Subject: How to check in script if Python or Jython is used
References: <87r3p5pjjd.fsf@Equus.decebal.nl>
 <mailman.673.1434878583.13271.python-list@python.org>
Message-ID: <87d20p8im6.fsf@Equus.decebal.nl>

On Sunday 21 Jun 2015 11:22 CEST, Laura Creighton wrote:

> In a message of Sun, 21 Jun 2015 10:12:06 +0200, Cecil Westerhof
> writes:
>> I installed Jython and will start playing with it. There probably
>> will be differences between Python and Jython. Is there a way to
>> determine if a script is run by Python or Jython? Then different
>> execution paths could be taken. With sys.version(_info) you do not
>> get this information.
>>
>> -- 
>> Cecil Westerhof
>> Senior Software Engineer
>> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>
> import platform
> platform.python_implementation()
>
> If your jython is old (pre 2.6) you will not have this.
>
> Then try
> platform.system()
> which will give you 'Java'

Works like a charm. CPython gives (on my system) 'Linux'. But as long
I only work with CPython and Jython, I only have to check for 'Java'.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun Jun 21 06:25:08 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 21 Jun 2015 12:25:08 +0200
Subject: Using Jython for Android development?
References: <87mvztpjfs.fsf@Equus.decebal.nl>
 <mailman.674.1434878796.13271.python-list@python.org>
Message-ID: <878ubd8ikb.fsf@Equus.decebal.nl>

On Sunday 21 Jun 2015 11:26 CEST, Laura Creighton wrote:

> In a message of Sun, 21 Jun 2015 10:14:15 +0200, Cecil Westerhof
> writes:
>> I have no experience yet with Jython or Android development. But I
>> was wondering: would it be possible to write applications for
>> Android with Jython? You normally use Java for it, but I think I
>> would like Jython more. :-D
>>
>> -- 
>> Cecil Westerhof
>> Senior Software Engineer
>> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>
> You cannot use Jython for this at present.  PyPy for android coming.
> (Works great on ARM, but oh, the environment changes to get the
> libtraries to work. )
>
> In the meatime, use kivy.  Actually, use kivy anyway, it is just too
> much fun. :)
>
> http://kivy.org/#home

Something else to evaluate. I do not have to be afraid I will become
bored. :-D

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From lac at openend.se  Sun Jun 21 06:54:28 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 21 Jun 2015 12:54:28 +0200
Subject: No =?utf-8?Q?=E2=80=98from?= __future__ import =?utf-8?Q?prin?=
 =?utf-8?Q?t=5Ffunction=E2=80=99?= in latest jython
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Sun, 21 Jun 2015 12:21:03 +0200." <87h9q18ir4.fsf@Equus.decebal.nl>
References: <87zj3tpk96.fsf@Equus.decebal.nl>
 <mailman.671.1434877523.13271.python-list@python.org><87h9q18ir4.fsf@Equus.decebal.nl>
Message-ID: <201506211054.t5LAsSEd001098@fido.openend.se>

In a message of Sun, 21 Jun 2015 12:21:03 +0200, Cecil Westerhof writes:
>On Sunday 21 Jun 2015 11:05 CEST, Laura Creighton wrote:
>
>> Do you have Jython 2.7 released a few weeks ago?
>
>Yes, but I was dumb enough to start the old version when I did this.
>:-(
>
>There is still one problem:
>======================================================================
>>>> from __future__ import print_function
>>>> print()
>()
>======================================================================
>
>While it should be:
>======================================================================
>>>> from __future__ import print_function
>>>> print()
>
>======================================================================
>
>But I think I can live with it for the moment.
>
>-- 
>Cecil Westerhof
>Senior Software Engineer
>LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>-- 
>https://mail.python.org/mailman/listinfo/python-list

That's a bug.  Go add it to
http://bugs.jython.org/issue2007  so when the bugs with
print_function get fixed, they will be sure to fix that one too.

Laura


From df at see.replyto.invalid  Sun Jun 21 07:49:16 2015
From: df at see.replyto.invalid (Dave Farrance)
Date: Sun, 21 Jun 2015 12:49:16 +0100
Subject: How to construct matrix from vectors?
References: <mm55le$4sj$1@speranza.aioe.org>
 <mailman.665.1434853238.13271.python-list@python.org>
 <mm5c4c$fmm$1@speranza.aioe.org> <mm5hks$p52$1@speranza.aioe.org>
 <mm5tq9$gtp$1@speranza.aioe.org>
Message-ID: <n88doa55115nuvmejapjal4cl1adpq1is7@4ax.com>

Fabien <fabien.maussion at gmail.com> wrote:

>another solution with less "(([[]]))", and less ";". There are way too 
>many ";" in Matlab ;)
>
>import numpy as np
>v1 = [1, 2, 3]
>v2 = [4, 5, 6]
>v3 = [7, 8, 9]
>v4 = [10, 11, 12]
>np.hstack([[v1, v2], [v3, v4]]).T
>Out[]:
>array([[ 1,  4],
>        [ 2,  5],
>        [ 3,  6],
>        [ 7, 10],
>        [ 8, 11],
>        [ 9, 12]])

Neat. And if the OP wants "vectors" in np array form to start with, and
to stack them together without transposing at that point, he could do it
like this:

>>> v1=np.vstack([1,2,3])
>>> v2=np.vstack([4,5,6])
>>> v3=np.vstack([7,8,9])
>>> v4=np.vstack([10,11,12])
>>> np.r_[np.c_[v1,v2],np.c_[v3,v4]]
array([[ 1,  4],
       [ 2,  5],
       [ 3,  6],
       [ 7, 10],
       [ 8, 11],
       [ 9, 12]])

And since he seems to want a Matlab-like environment, then the somewhat
depreciated pylab was intended to dump a Matlab-like set of functions
into the namespace, which is OK for an interactive environment, an not
too much of a problem for a short program in a single module. Probably
best to do that with iPython, though.

>>> from matplotlib.pylab import *
>>> v1=vstack([1,2,3])
>>> v2=vstack([4,5,6])
>>> v3=vstack([7,8,9])
>>> v4=vstack([10,11,12])
>>> r_[c_[v1,v2],c_[v3,v4]]
array([[ 1,  4],
       [ 2,  5],
       [ 3,  6],
       [ 7, 10],
       [ 8, 11],
       [ 9, 12]])



From fomcl at yahoo.com  Sun Jun 21 07:54:09 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Sun, 21 Jun 2015 04:54:09 -0700
Subject: How to check in script if Python or Jython is used
Message-ID: <1434887649.32396.BPMail_high_carrier@web163801.mail.gq1.yahoo.com>



------------------------ 
Sun, Jun 21, 2015 12:24 PM CEST Cecil Westerhof wrote:

>On Sunday 21 Jun 2015 11:22 CEST, Laura Creighton wrote:
>
>> In a message of Sun, 21 Jun 2015 10:12:06 +0200, Cecil Westerhof
>> writes:
>> I installed Jython and will start playing with it. There probably
>> will be differences between Python and Jython. Is there a way to
>> determine if a script is run by Python or Jython? Then different
>> execution paths could be taken. With sys.version(_info) you do not
>> get this information.
>>

os.path.basename(sys.executable),
if sys.implementation is not an option




From dfnsonfsduifb at gmx.de  Sun Jun 21 08:14:16 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sun, 21 Jun 2015 14:14:16 +0200
Subject: Jython and can't write cache file
In-Reply-To: <87pp4p8kmi.fsf@Equus.decebal.nl>
References: <874mm1qyz0.fsf@Equus.decebal.nl>	<mailman.670.1434877014.13271.python-list@python.org>
 <87pp4p8kmi.fsf@Equus.decebal.nl>
Message-ID: <mm69qo$nfv$1@news.albasani.net>

On 21.06.2015 11:40, Cecil Westerhof wrote:

> Thanks. Good that I asked it. :-D

Good for you that you found someone able to enter words into a Google
query. That's a skill you might want to acquire some time in the future.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From Cecil at decebal.nl  Sun Jun 21 08:38:42 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 21 Jun 2015 14:38:42 +0200
Subject: No =?utf-8?Q?=E2=80=98from?= __future__ import =?utf-8?Q?prin?=
 =?utf-8?Q?t=5Ffunction=E2=80=99?= in latest jython
References: <87zj3tpk96.fsf@Equus.decebal.nl>
 <mailman.671.1434877523.13271.python-list@python.org>
 <87h9q18ir4.fsf@Equus.decebal.nl>
 <mailman.675.1434884076.13271.python-list@python.org>
Message-ID: <87zj3t6xt9.fsf@Equus.decebal.nl>

On Sunday 21 Jun 2015 12:54 CEST, Laura Creighton wrote:

> In a message of Sun, 21 Jun 2015 12:21:03 +0200, Cecil Westerhof
> writes:
>> On Sunday 21 Jun 2015 11:05 CEST, Laura Creighton wrote:
>>
>>> Do you have Jython 2.7 released a few weeks ago?
>>
>> Yes, but I was dumb enough to start the old version when I did
>> this. :-(
>>
>> There is still one problem:
>> ======================================================================
>>>>> from __future__ import print_function
>>>>> print()
>> ()
>> ======================================================================
>>
>> While it should be:
>> ======================================================================
>>>>> from __future__ import print_function
>>>>> print()
>>
>> ======================================================================
>>
>> But I think I can live with it for the moment.
>>
>> -- 
>> Cecil Westerhof
>> Senior Software Engineer
>> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>
> That's a bug.  Go add it to
> http://bugs.jython.org/issue2007  so when the bugs with
> print_function get fixed, they will be sure to fix that one too.

Done.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From ron3200 at gmail.com  Sun Jun 21 08:55:53 2015
From: ron3200 at gmail.com (Ron Adam)
Date: Sun, 21 Jun 2015 08:55:53 -0400
Subject: Lawful != Mutable (was Can Python function return multiple data?)
In-Reply-To: <5037b4a5-94a9-4995-828f-9624804da20a@googlegroups.com>
References: <3bbe49da-e989-4a8c-a8a9-75d3a786f508@googlegroups.com>
 <mailman.181.1433458775.13271.python-list@python.org>
 <55710b69$0$12980$c3e8da3$5496439d@news.astraweb.com>
 <6c78b294-efd6-4096-a572-1841aa71a5eb@googlegroups.com>
 <557182af$0$13014$c3e8da3$5496439d@news.astraweb.com>
 <33a2f501-5587-4fd9-91fa-7094d4f6c512@googlegroups.com>
 <mailman.207.1433541572.13271.python-list@python.org>
 <81cb1232-ecf8-4cf0-b29c-ecf2ca47ade0@googlegroups.com>
 <mailman.209.1433561282.13271.python-list@python.org>
 <c58d7868-feaf-492e-9fb9-9b0108c5914d@googlegroups.com>
 <mailman.212.1433569420.13271.python-list@python.org>
 <5fa1f672-50de-4481-9c5b-2c68c4e30f12@googlegroups.com>
 <4fdc38b3-6b51-4f98-9ba3-42b265582fc0@googlegroups.com>
 <mailman.258.1433694866.13271.python-list@python.org>
 <762816ac-c744-480b-a9f9-a0110451c516@googlegroups.com>
 <mailman.666.1434853946.13271.python-list@python.org>
 <5037b4a5-94a9-4995-828f-9624804da20a@googlegroups.com>
Message-ID: <mm6c8q$vp3$1@ger.gmane.org>



On 06/20/2015 10:50 PM, Rustom Mody wrote:
> Here is Eric Snow:
>
> | Keep in mind that by "immutability" I'm talking about*really*
> | immutable, perhaps going so far as treating the full memory space
> | associated with an object as frozen.  For instance, we'd have to
> | ensure that "immutable" Python objects like strings, ints, and tuples
> | do not change (i.e. via the C API).  The contents of involved
> | tuples/containers would have to be likewise immutable.  Even changing
> | refcounts could be too much, hence the idea of moving refcounts out to
> | a separate table.
> |
> | This level of immutability would be something new to Python.  We'll
> | see if it's necessary.  If it isn't too much work it might be a good
> | idea regardless of the multi-core proposal.
>
> Does the second para look like CPython implementation or python-the-language?
>
> Also note the 'Even' in the first para. ie Eric is talking of low-level
> (ie thread-safety, refcounting etc) immutability after the even and higher
> level semantic immutability before

It seems to me, this will take a lot more changes to python overall.

Adding a way to raise an exception if an object is mutated would be a good 
initial step.  Have it turned off by default.

I think it will be needed to test how any of the above is working.

It may also allow some multiprocessing just by avoiding raising any 
MutatedObject exceptions.

Cheers,
    Ron



From lac at openend.se  Sun Jun 21 09:43:12 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 21 Jun 2015 15:43:12 +0200
Subject: HOPE: A Python just-in-time compiler for astrophysical
 computations
In-Reply-To: Message from BartC <bc@freeuk.com>
 of "Sun, 21 Jun 2015 10:29:32 +0100." <2uvhx.910489$G53.315907@fx13.am4>
References: <mailman.664.1434846609.13271.python-list@python.org><2uvhx.910489$G53.315907@fx13.am4>
Message-ID: <201506211343.t5LDhCHK005380@fido.openend.se>

In a message of Sun, 21 Jun 2015 10:29:32 +0100, BartC writes:
>It also puts in a good dig at PyPy by including one benchmark where it 
>is 6 times as slow as CPython!
>
>It's not clear why it's particularly useful for astrophysics.
>
>-- 
>Bartc

It's not that good a dig, as they say that it took less than 1 second
to run most of their benchmarks, which mostly means that the PyPy JIT
won't have started to run anyway.  One can argue that HOPE isn't
actually a method JIT at all -- or if it is one, then so is Cython.

Laura


From sahluwalia at wynyardgroup.com  Sun Jun 21 09:57:01 2015
From: sahluwalia at wynyardgroup.com (sahluwalia at wynyardgroup.com)
Date: Sun, 21 Jun 2015 06:57:01 -0700 (PDT)
Subject: JSON Object to CSV file
In-Reply-To: <mm5mj6$5m8$3@dont-email.me>
References: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
 <mm5mj6$5m8$3@dont-email.me>
Message-ID: <e1a0eb4d-9594-480f-b082-142dc566dea2@googlegroups.com>

On Sunday, 21 June 2015 02:47:31 UTC-4, Denis McMahon  wrote:
> On Wed, 17 Jun 2015 08:00:11 -0700, Saran A wrote:
> 
> > I would like to have this JSON object written out to a CSV file so that
> > the keys are header fields (for each of the columns) and the values are
> > values that are associated with each header field.
> 
> > {
> > "CF": {
> .......
> > "CF": "Fee",
> 
> Your json object seems to have the same key used for two elements at the 
> same level, are you sure this is legal json?
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com

I converted this from an XML file given to me from a third party. It is as is. I am not sure what you mean by "valid"; that is a very subjective measure for any form of quantitative or qualitative data. 


From sahluwalia at wynyardgroup.com  Sun Jun 21 09:58:29 2015
From: sahluwalia at wynyardgroup.com (Sahlusar)
Date: Sun, 21 Jun 2015 06:58:29 -0700 (PDT)
Subject: JSON Object to CSV File Troubleshooting
In-Reply-To: <mm5n0u$5m8$5@dont-email.me>
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
 <mm5n0u$5m8$5@dont-email.me>
Message-ID: <c18d65d8-d575-4711-a660-ef84bc796aad@googlegroups.com>

On Sunday, 21 June 2015 02:54:48 UTC-4, Denis McMahon  wrote:
> On Thu, 18 Jun 2015 18:47:30 -0700, Sahlusar wrote:
> 
> > I have a conundrum regarding JSON objects and converting them to CSV:
> 
> I think your conundrum is that you've taken on a coding task beyond your 
> abilities to comprehend, and as a result not only can you not code it, 
> you can't even adequately describe it.
> 
> At least, it seems that every time you do try and describe it either the 
> data format or the task description changes.
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com


It is difficult to explain this to someone asynchronously and without an in person meeting. Moreover, the strict guidelines for disclosing information make it difficult for me to explain the client's requirements and the problems that they face. 

I do agree with you Denis that this is an unconventional approach. 




From ahlusar.ahluwalia at gmail.com  Sun Jun 21 10:22:47 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Sun, 21 Jun 2015 07:22:47 -0700 (PDT)
Subject: JSON Object to CSV file
In-Reply-To: <mm5mj6$5m8$3@dont-email.me>
References: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
 <mm5mj6$5m8$3@dont-email.me>
Message-ID: <33c13578-e0c4-4fe1-b925-b7d57cf038ed@googlegroups.com>

On Sunday, June 21, 2015 at 2:47:31 AM UTC-4, Denis McMahon wrote:
> On Wed, 17 Jun 2015 08:00:11 -0700, Saran A wrote:
> 
> > I would like to have this JSON object written out to a CSV file so that
> > the keys are header fields (for each of the columns) and the values are
> > values that are associated with each header field.
> 
> > {
> > "CF": {
> .......
> > "CF": "Fee",
> 
> Your json object seems to have the same key used for two elements at the 
> same level, are you sure this is legal json?
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com

I converted this from an XML file given to me from a third party. It is as is. I am not sure what you mean by "valid"; that is a very subjective measure for any form of quantitative or qualitative data. 


From ahlusar.ahluwalia at gmail.com  Sun Jun 21 10:38:13 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Sun, 21 Jun 2015 07:38:13 -0700 (PDT)
Subject: JSON Object to CSV File Troubleshooting
In-Reply-To: <mm5n0u$5m8$5@dont-email.me>
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
 <mm5n0u$5m8$5@dont-email.me>
Message-ID: <44a2d55f-9312-4b70-8b23-637466b21b11@googlegroups.com>


It is difficult to explain this to someone asynchronously and without an in person meeting. Moreover, the strict guidelines for disclosing information make it difficult for me to explain the client's requirements and the problems that they face. 

I do agree with you Denis that this is an unconventional approach. I was wondering then that perhaps I should add additional functionality at the XML to JSON step? So far, with JSON objects without nested lists (as values) I have been successful with this (the following is rather lengthy):

import xml.etree.cElementTree as ElementTree 
from xml.etree.ElementTree import XMLParser 
import json 
import csv 
import tokenize 
import token 
try: 
    from collections import OrderedDict 
    import json 
except ImportError: 
    from ordereddict import OrderedDict 
    import simplejson as json 
import itertools 
import six 
import string 
from csvkit import CSVKitWriter 


class XmlListConfig(list): 
    def __init__(self, aList): 
        for element in aList: 
            if element: 
                # treat like dict 
                if len(element) == 1 or element[0].tag != element[1].tag: 
                    self.append(XmlDictConfig(element)) 
                # treat like list 
                elif element[0].tag == element[1].tag: 
                    self.append(XmlListConfig(element)) 
            elif element.text: 
                text = element.text.strip() 
                if text: 
                    self.append(text) 


class XmlDictConfig(dict): 
    ''' 
    Example usage: 

    >>> tree = ElementTree.parse('your_file.xml') 
    >>> root = tree.getroot() 
    >>> xmldict = XmlDictConfig(root) 

    Or, if you want to use an XML string: 

    >>> root = ElementTree.XML(xml_string) 
    >>> xmldict = XmlDictConfig(root) 

    And then use xmldict for what it is..a dictionary. 
    ''' 
    def __init__(self, parent_element): 
        if parent_element.items(): 
            self.update(dict(parent_element.items())) 
        for element in parent_element: 
            if element: 
                # treat like dict - we assume that if the first two tags 
                # in a series are different, then they are all different. 
                if len(element) == 1 or element[0].tag != element[1].tag: 
                    aDict = XmlDictConfig(element) 
                # treat like list - we assume that if the first two tags 
                # in a series are the same, then the rest are the same. 
                else: 
                    # here, we put the list in dictionary; the key is the 
                    # tag name the list elements all share in common, and 
                    # the value is the list itself 
                    aDict = {element[0].tag: XmlListConfig(element)} 
                # if the tag has attributes, add those to the dict 
                if element.items(): 
                    aDict.update(dict(element.items())) 
                self.update({element.tag: aDict}) 
            # this assumes that if you've got an attribute in a tag, 
            # you won't be having any text. This may or may not be a 
            # good idea -- time will tell. It works for the way we are 
            # currently doing XML configuration files... 
            elif element.items(): 
                self.update({element.tag: dict(element.items())}) 
            # finally, if there are no child tags and no attributes, extract 
            # the text 
            else: 
                self.update({element.tag: element.text}) 



def main(): 

    #Lines 88-89stantiate the class Elementree 
    #and applies the method to recursively traverse from the root node 
    #XmlDictConfig is instantiated in line 90 

    with open('C:\\Users\\somefile.xml', 'r', encoding='utf-8') as f: 
        xml_string = f.read() 
    xml_string= xml_string.replace('&#x0;', '') #optional to remove ampersands. 
    root = ElementTree.XML(xml_string) 
    xmldict = XmlDictConfig(root) 
    json_str = json.dumps(xmldict, sort_keys=True, indent=4, separators=(',', ': ')) 
    newly_formatted_data = json.loads(json_str) #encode into JSON 
    with open('data2.json', 'w') as f:  #writing JSON file 
        json.dump(newly_formatted_data, f) 

Peter Otten was very helpful with subsequently converting aJ SON string to a CSV file:


import csv 
import json 
import sys 

def hook(obj): 
    return obj 

def flatten(obj): 
    for k, v in obj: 
        if isinstance(v, list): 
            yield from flatten(v) 
        else: 
            yield k, v 

if __name__ == "__main__": 
    with open("somefileneame.json") as f: 
        data = json.load(f, object_pairs_hook=hook) 

    pairs = list(flatten(data)) 

    writer = csv.writer(sys.stdout) 
    header = writer.writerow([k for k, v in pairs]) 
    row = writer.writerow([v for k, v in pairs]) #writer.writerows for any other iterable object 

However, for the nested keys as dictionaries with values as dictionaries, I have been taking smaller chunks of the JSON objects and developing some functional programs that allow me to blow up lists:

For example for this example JSON snippet, to de-nest by level the nested dictionaries, I use this function:

def flatten(d, parent_key=''):
    items = []
    for k, v in d.items():
        try:
            items.extend(flatten(v, '%s%s_' % (parent_key, k)).items())
        except AttributeError:
            items.append(('%s%s' % (parent_key, k), v))
    return dict(items)
final = (flatten(data2, parent_key =''))


##JSON sample:

data2 = {
    "OTF": "0",
    "F": "False",
    "F": {
        "Int32": ["0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0"]
    },
                "D": {
                    "B": ["0",
                    "0",
                    "0",
                    "0",
                    "0",
                    "0",
                    "0",
                    "0",
                    "0",
                    "0",
                    "0"]
                },
                "PBDS": {
                    "DateTime": ["1/1/0001 12:00:00 AM",
                    "1/1/0001 12:00:00 AM",
                    "1/1/0001 12:00:00 AM",
                    "1/1/0001 12:00:00 AM",
                    "1/1/0001 12:00:00 AM",
                    "1/1/0001 12:00:00 AM",
                    "1/1/0001 12:00:00 AM",
                    "1/1/0001 12:00:00 AM"]
                },
                "PBDS": {
                    "Double": ["0",
                    "0",
                    "0",
                    "0",
                    "0",
                    "0",
                    "0",
                    "0"]
                },
                "SCS": {
                    "String": ["1",
                    "2"]
                }
            }

The result:


{'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
 'F_Int32': ['0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0',
  '0'],
 'OTF': '0',
 'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'],
 'SCS_String': ['1', '2']}

I then have used a decorator to explode the lists (thank you to Gribouillis for feedback):

from postprocess import post_process
@post_process(dict)
def explode_lists(adict):
    for key, value in adict.items():
        if isinstance(value, list):
            if all(isinstance(x, str) for x in value):
                for i, x in enumerate(value, 1):
                    yield ('{}{}'.format(key, i), x)
                continue
        yield key, value


# postprocess.py
def post_process(*filters):
    """Decorator to post process a function's return value through a
    sequence of filters (functions with a single argument).
    Example:
        @post_process(f1, f2, f3)
        def f(*args, **kwd):
            ...
            return value
        then calling f(...) will actually return f3( f2( f1( f(...)))).
        This can also be used to convert a generator to a function
        returning a sequence type:
        @post_process(dict)
        def my_generator():
            ...
            yield key, value
    """
    def decorate(func):
        from functools import wraps
        @wraps(func)
        def wrapper(*args, **kwd):
            rv = func(*args, **kwd)
            for f in filters:
                rv = f(rv)
            return rv
        return wrapper
return decorate

I know that this is alot of sequential steps. I am wondering if I could insert or conditionally pass these functions when originally parsing the XML, so that the JSON is formatted for more recursive reading of the JSON dictionary and then writing to CSV? I welcome constructive feedback for refactoring....


From ahlusar.ahluwalia at gmail.com  Sun Jun 21 10:54:32 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Sun, 21 Jun 2015 07:54:32 -0700 (PDT)
Subject: JSON Object to CSV file
In-Reply-To: <mm5mj6$5m8$3@dont-email.me>
References: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
 <mm5mj6$5m8$3@dont-email.me>
Message-ID: <c934be6b-c732-466c-849e-7891d4dd461b@googlegroups.com>

On Sunday, June 21, 2015 at 2:47:31 AM UTC-4, Denis McMahon wrote:
> On Wed, 17 Jun 2015 08:00:11 -0700, Saran A wrote:
> 
> > I would like to have this JSON object written out to a CSV file so that
> > the keys are header fields (for each of the columns) and the values are
> > values that are associated with each header field.
> 
> > {
> > "CF": {
> .......
> > "CF": "Fee",
> 
> Your json object seems to have the same key used for two elements at the 
> same level, are you sure this is legal json?
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com

This is a duplicate to the post titled: JSON to CSV Troubleshooting:

It is difficult to explain this to someone asynchronously and without an in person meeting. Moreover, the strict guidelines for disclosing information make it difficult for me to explain the client's requirements and the problems that they face. 

I do agree with you Denis that this is an unconventional approach. I was wondering then that perhaps I should add additional functionality at the XML to JSON step? So far, with JSON objects without nested lists (as values) I have been successful with this (the following is rather lengthy): 

import xml.etree.cElementTree as ElementTree 
from xml.etree.ElementTree import XMLParser 
import json 
import csv 
import tokenize 
import token 
try: 
    from collections import OrderedDict 
    import json 
except ImportError: 
    from ordereddict import OrderedDict 
    import simplejson as json 
import itertools 
import six 
import string 
from csvkit import CSVKitWriter 


class XmlListConfig(list): 
    def __init__(self, aList): 
        for element in aList: 
            if element: 
                # treat like dict 
                if len(element) == 1 or element[0].tag != element[1].tag: 
                    self.append(XmlDictConfig(element)) 
                # treat like list 
                elif element[0].tag == element[1].tag: 
                    self.append(XmlListConfig(element)) 
            elif element.text: 
                text = element.text.strip() 
                if text: 
                    self.append(text) 


class XmlDictConfig(dict): 
    ''' 
    Example usage: 

    >>> tree = ElementTree.parse('your_file.xml') 
    >>> root = tree.getroot() 
    >>> xmldict = XmlDictConfig(root) 

    Or, if you want to use an XML string: 

    >>> root = ElementTree.XML(xml_string) 
    >>> xmldict = XmlDictConfig(root) 

    And then use xmldict for what it is..a dictionary. 
    ''' 
    def __init__(self, parent_element): 
        if parent_element.items(): 
            self.update(dict(parent_element.items())) 
        for element in parent_element: 
            if element: 
                # treat like dict - we assume that if the first two tags 
                # in a series are different, then they are all different. 
                if len(element) == 1 or element[0].tag != element[1].tag: 
                    aDict = XmlDictConfig(element) 
                # treat like list - we assume that if the first two tags 
                # in a series are the same, then the rest are the same. 
                else: 
                    # here, we put the list in dictionary; the key is the 
                    # tag name the list elements all share in common, and 
                    # the value is the list itself 
                    aDict = {element[0].tag: XmlListConfig(element)} 
                # if the tag has attributes, add those to the dict 
                if element.items(): 
                    aDict.update(dict(element.items())) 
                self.update({element.tag: aDict}) 
            # this assumes that if you've got an attribute in a tag, 
            # you won't be having any text. This may or may not be a 
            # good idea -- time will tell. It works for the way we are 
            # currently doing XML configuration files... 
            elif element.items(): 
                self.update({element.tag: dict(element.items())}) 
            # finally, if there are no child tags and no attributes, extract 
            # the text 
            else: 
                self.update({element.tag: element.text}) 



def main(): 

    #Lines 88-89stantiate the class Elementree 
    #and applies the method to recursively traverse from the root node 
    #XmlDictConfig is instantiated in line 90 

    with open('C:\\Users\\somefile.xml', 'r', encoding='utf-8') as f: 
        xml_string = f.read() 
    xml_string= xml_string.replace('&#x0;', '') #optional to remove ampersands. 
    root = ElementTree.XML(xml_string) 
    xmldict = XmlDictConfig(root) 
    json_str = json.dumps(xmldict, sort_keys=True, indent=4, separators=(',', ': ')) 
    newly_formatted_data = json.loads(json_str) #encode into JSON 
    with open('data2.json', 'w') as f:  #writing JSON file 
        json.dump(newly_formatted_data, f) 

Peter Otten was very helpful with subsequently converting aJ SON string to a CSV file: 


import csv 
import json 
import sys 

def hook(obj): 
    return obj 

def flatten(obj): 
    for k, v in obj: 
        if isinstance(v, list): 
            yield from flatten(v) 
        else: 
            yield k, v 

if __name__ == "__main__": 
    with open("somefileneame.json") as f: 
        data = json.load(f, object_pairs_hook=hook) 

    pairs = list(flatten(data)) 

    writer = csv.writer(sys.stdout) 
    header = writer.writerow([k for k, v in pairs]) 
    row = writer.writerow([v for k, v in pairs]) #writer.writerows for any other iterable object 

However, for the nested keys as dictionaries with values as dictionaries, I have been taking smaller chunks of the JSON objects and developing some functional programs that allow me to blow up lists: 

For example for this example JSON snippet, to de-nest by level the nested dictionaries, I use this function: 

def flatten(d, parent_key=''): 
    items = [] 
    for k, v in d.items(): 
        try: 
            items.extend(flatten(v, '%s%s_' % (parent_key, k)).items()) 
        except AttributeError: 
            items.append(('%s%s' % (parent_key, k), v)) 
    return dict(items) 
final = (flatten(data2, parent_key ='')) 


##JSON sample: 

data2 = { 
    "OTF": "0", 
    "F": "False", 
    "F": { 
        "Int32": ["0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0", 
        "0"] 
    }, 
                "D": { 
                    "B": ["0", 
                    "0", 
                    "0", 
                    "0", 
                    "0", 
                    "0", 
                    "0", 
                    "0", 
                    "0", 
                    "0", 
                    "0"] 
                }, 
                "PBDS": { 
                    "DateTime": ["1/1/0001 12:00:00 AM", 
                    "1/1/0001 12:00:00 AM", 
                    "1/1/0001 12:00:00 AM", 
                    "1/1/0001 12:00:00 AM", 
                    "1/1/0001 12:00:00 AM", 
                    "1/1/0001 12:00:00 AM", 
                    "1/1/0001 12:00:00 AM", 
                    "1/1/0001 12:00:00 AM"] 
                }, 
                "PBDS": { 
                    "Double": ["0", 
                    "0", 
                    "0", 
                    "0", 
                    "0", 
                    "0", 
                    "0", 
                    "0"] 
                }, 
                "SCS": { 
                    "String": ["1", 
                    "2"] 
                } 
            } 

The result: 


{'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'], 
 'F_Int32': ['0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0', 
  '0'], 
 'OTF': '0', 
 'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'], 
 'SCS_String': ['1', '2']} 

I then have used a decorator to explode the lists (thank you to Gribouillis for feedback): 

from postprocess import post_process 
@post_process(dict) 
def explode_lists(adict): 
    for key, value in adict.items(): 
        if isinstance(value, list): 
            if all(isinstance(x, str) for x in value): 
                for i, x in enumerate(value, 1): 
                    yield ('{}{}'.format(key, i), x) 
                continue 
        yield key, value 


# postprocess.py 
def post_process(*filters): 
    """Decorator to post process a function's return value through a 
    sequence of filters (functions with a single argument). 
    Example: 
        @post_process(f1, f2, f3) 
        def f(*args, **kwd): 
            ... 
            return value 
        then calling f(...) will actually return f3( f2( f1( f(...)))). 
        This can also be used to convert a generator to a function 
        returning a sequence type: 
        @post_process(dict) 
        def my_generator(): 
            ... 
            yield key, value 
    """ 
    def decorate(func): 
        from functools import wraps 
        @wraps(func) 
        def wrapper(*args, **kwd): 
            rv = func(*args, **kwd) 
            for f in filters: 
                rv = f(rv) 
            return rv 
        return wrapper 
return decorate 

I know that this is alot of sequential steps. I am wondering if I could insert or conditionally pass these functions when originally parsing the XML, so that the JSON is formatted for more recursive reading of the JSON dictionary and then writing to CSV? I welcome constructive feedback for refactoring.... 


From kaviraj at launchyard.com  Sun Jun 21 11:17:38 2015
From: kaviraj at launchyard.com (Kaviraj Kanagaraj)
Date: Sun, 21 Jun 2015 20:47:38 +0530
Subject: Arguments for "type metaclass" __init__ method
Message-ID: <CAO36yJ6X5aaLbi7T-avji4rFoqyRY6nsf3nxH7LAU+sXAwk5PQ@mail.gmail.com>

Hi All,

I was reading about meta programming in "Pro Django" book.

I came across the "type" class which will be acting as default meta class
for all other class. Also "type" metaclass is where actuall class object is
created

To dynamically create a class:
DynamicClass = type("DynamicClass", (object,), {'eggs' : 'spams'})

which means type's __init__ method accepts ClassName, bases(tuple) and
attrbs(dict) as args.


But in case of creating class via class definition("type" act as default
metaclass), the type is called by following syntax,

type(cls, classname, bases, attrbs)

My doubt is how it possible to pass "cls" as argument(as class is not even
created before calling meta class). Also how "type" is accepting different
args in two different cases?

Need help in understanding

Thanks in advance.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150621/a97ae2c1/attachment.html>

From ned at nedbatchelder.com  Sun Jun 21 11:31:31 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sun, 21 Jun 2015 08:31:31 -0700 (PDT)
Subject: JSON Object to CSV file
In-Reply-To: <c934be6b-c732-466c-849e-7891d4dd461b@googlegroups.com>
References: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
 <mm5mj6$5m8$3@dont-email.me>
 <c934be6b-c732-466c-849e-7891d4dd461b@googlegroups.com>
Message-ID: <09fe59ad-5895-4190-85ca-e068659f60a1@googlegroups.com>

On Sunday, June 21, 2015 at 10:54:44 AM UTC-4, Sahlusar wrote:

> This is a duplicate to the post titled: JSON to CSV Troubleshooting:

Please don't do that.

--Ned.


From ahlusar.ahluwalia at gmail.com  Sun Jun 21 11:39:30 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Sun, 21 Jun 2015 08:39:30 -0700 (PDT)
Subject: JSON Object to CSV file
In-Reply-To: <09fe59ad-5895-4190-85ca-e068659f60a1@googlegroups.com>
References: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
 <mm5mj6$5m8$3@dont-email.me>
 <c934be6b-c732-466c-849e-7891d4dd461b@googlegroups.com>
 <09fe59ad-5895-4190-85ca-e068659f60a1@googlegroups.com>
Message-ID: <f14ccc24-723e-4a5c-a7dd-7f0967fb9f01@googlegroups.com>

On Sunday, 21 June 2015 11:31:44 UTC-4, Ned Batchelder  wrote:
> On Sunday, June 21, 2015 at 10:54:44 AM UTC-4, Sahlusar wrote:
> 
> > This is a duplicate to the post titled: JSON to CSV Troubleshooting:
> 
> Please don't do that.
> 
> --Ned.

My apologies - is it possible to delete one of my posts?


From lanuradha at gmail.com  Sun Jun 21 12:00:08 2015
From: lanuradha at gmail.com (lanuradha at gmail.com)
Date: Sun, 21 Jun 2015 09:00:08 -0700 (PDT)
Subject: How to construct matrix from vectors?
In-Reply-To: <mailman.679.1434899573.13271.python-list@python.org>
References: <mm55le$4sj$1@speranza.aioe.org>
 <mailman.665.1434853238.13271.python-list@python.org>
 <mm5c4c$fmm$1@speranza.aioe.org> <mm5srr$7k7$1@ger.gmane.org>
 <mailman.679.1434899573.13271.python-list@python.org>
Message-ID: <29d53e0b-b9ed-4f8c-854e-1a1667021211@googlegroups.com>

On Sunday, 21 June 2015 20:43:15 UTC+5:30, Dennis Lee Bieber  wrote:
> On Sun, 21 Jun 2015 09:32:55 +0100, Mark Lawrence declaimed the following:
> 
> >On 21/06/2015 04:47, Nasser M. Abbasi wrote:
> >
> >>
> >> But it is not as intuitive as with Matlab
> >>
> >
> >For those of us who don't know would you be kind enough to do a cost 
> >comparison of Matlab vs Python licenses?
> 
> 	Or even Matlab vs Octave

Well if its a parentheses minimization contest, APL will beat the pants of everything.

Heres a web-session at http://baruchel.hd.free.fr/apps/apl/

   v1 ? 1 2 3
   v2 ? 4 5 6
   v3 ? 7 8 9
   v4 ? 10 11 12
   v1
1 2 3

# We need to reshape (?)
   v1 ? 3 1? v1
   v1
1
2
3
# Likewise
   v2 ? 3 1? v2
   v3 ? 3 1? v3
   v4 ? 3 1? v4
   v1,v2
1 4
2 5
3 6

# So we need to catenate in a different dimension

   v1?v2
1
2
3
4
5
6

#Likewise
   v3?v4
 7
 8
 9
10
11
12

# Thats the only parentheses used
   (v1?v2),(v3?v4)
1  7
2  8
3  9
4 10
5 11
6 12


# And we dont even need all these
   (v1?v2),v3?v4
1  7
2  8
3  9
4 10
5 11
6 12


From PointedEars at web.de  Sun Jun 21 12:12:52 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 21 Jun 2015 18:12:52 +0200
Subject: How to check in script if Python or Jython is used
References: <87r3p5pjjd.fsf@Equus.decebal.nl>
Message-ID: <2671202.GDDaZ1Saa3@PointedEars.de>

Cecil Westerhof wrote:

> I installed Jython and will start playing with it. There probably will
> be differences between Python and Jython. Is there a way to determine
> if a script is run by Python or Jython? Then different execution paths
> could be taken. With sys.version(_info) you do not get this
> information.

?print sys.__doc__? in (C)python(2) hinted at ?sys.platform?.  And so:

$ python -V
Python 2.7.10

$ python -c 'from sys import platform; print platform'
linux2

$ python3 -V
Python 3.4.3+

$ python3 -c 'from sys import platform; print(platform)'
linux

$ jython -V
"my" variable $jythonHome masks earlier declaration in same scope at 
/usr/bin/jython line 15.
Jython 2.5.3

$ jython -c 'from sys import platform; print platform'
"my" variable $jythonHome masks earlier declaration in same scope at 
/usr/bin/jython line 15.
java1.7.0_79

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From jpcookie at gmail.com  Sun Jun 21 13:08:47 2015
From: jpcookie at gmail.com (John T. Haggerty)
Date: Sun, 21 Jun 2015 17:08:47 +0000
Subject: No subject
Message-ID: <CAJGLaBJvbJzYLTVoP0Vh4wWiCqzruUvPspo1QSA7g=V=JfSWTQ@mail.gmail.com>

I'm looking to just have a simple program that will do a SQLite query pull
a random record and then copy that record too the clipboard the system. I'm
not quite seeing how to do this perhaps this is already been done elsewhere
but I spent quite a bit of time trying to figure out how to do that and I'm
not seeing a listing anywhere. any help would be greatly appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150621/6b8debae/attachment.html>

From jonas.thornvall at gmail.com  Sun Jun 21 13:21:43 2015
From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com)
Date: Sun, 21 Jun 2015 10:21:43 -0700 (PDT)
Subject: Python functionality in Javascript
Message-ID: <4fc46f21-b31e-474c-b788-86475add432d@googlegroups.com>

Check this out using a 8 digit base with a 100 digit number no problem.
http://jt.node365.se/baseconversion3.html


From chris at cdreimer.com  Sun Jun 21 15:32:46 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sun, 21 Jun 2015 12:32:46 -0700
Subject: Do I need license to release the Python version of old BASIC games?
Message-ID: <5587115E.30703@cdreimer.com>

Greetings,

I'm in the process of converting 101 old BASIC games into Python (see 
link below).

http://www.atariarchives.org/basicgames/

The short term goal is to learn the finer aspects of the Python language 
and reliving my misbegotten past on the Commodore 64. The long term goal 
is to use the completed project as part of a programming portfolio, as 
many of my past I.T. jobs have required no programming and my current 
job requires some PowerShell scripting (meh).

Many of these BASIC games floated around the university computer labs 
and DEC for years before being published in Creative Computing magazine 
and into book form in 1978. My Python scripts are faithful to the video 
output as listed in the book, but underlying code is vastly different 
than the BASIC code.

Do I need to release my scripts under a license? If so, which one?

Thanks,

Chris R.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150621/e08074f3/attachment.html>

From lac at openend.se  Sun Jun 21 16:00:45 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 21 Jun 2015 22:00:45 +0200
Subject: Do I need license to release the Python version of old BASIC
 games?
In-Reply-To: Message from "C.D. Reimer" <chris@cdreimer.com>
 of "Sun, 21 Jun 2015 12:32:46 -0700." <5587115E.30703@cdreimer.com>
References: <5587115E.30703@cdreimer.com>
Message-ID: <201506212000.t5LK0jk3014266@fido.openend.se>

In a message of Sun, 21 Jun 2015 12:32:46 -0700, "C.D. Reimer" writes:

>Do I need to release my scripts under a license? If so, which one?

You should, because if you don't you could pop up some day and
assert copyright and sue the hell out of people who use your code,
which means that many people won't touch it until you license it.

http://blog.codinghorror.com/pick-a-license-any-license/

is a pretty good introduction to the quagmire, and talks about
some popular choices.

Laura


From no.email at nospam.invalid  Sun Jun 21 16:27:56 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Sun, 21 Jun 2015 13:27:56 -0700
Subject: Catching exceptions with multi-processing
References: <mm17as$i8m$1@speranza.aioe.org>
Message-ID: <87fv5kiz77.fsf@jester.gateway.sonic.net>

Fabien <fabien.maussion at gmail.com> writes:
> I am developing a tool which works on individual entities (glaciers)
> and do a lot of operations on them. There are many tasks to do, one
> after each other, and each task follows the same interface: ...

If most of the resources will be spent on computation and the
communications overhead is fairly low, the path of least resistance may
be to:

1) write a script that computes just one glacier (no multiprocessing)
2) write a control script that runs the glacier script through something
   like os.popen(), so normally it will collect an answer, but it can
   also notice if the glacier script crashes, or kill it from a timeout
   if it takes too long
3) Track the glacier tasks in an external queue server: I've used Redis
   (redis.io) for this, since it's simple and powerful, but there are
   other tools like 0mq that might be more precisely fitted.
4) The control script can read the queue server for tasks and update the
   queue server when results are ready

The advantages of this over multiprocessing are:

1) Redis is a TCP server which means you can spread your compute scripts
over multiple computers easily, getting more parallelism.  You can write
values into it as JSON strings if they are compound values that are not
too large.  Otherwise you probably have to use files, but can pass the
filenames through Redis.  You can connect new clients whenever you want
through the publish/subscribe interface, etc.

2) by using a simple control script you don't have to worry too much
about the many ways that the computation script might fail, you can
restart it, you can put the whole thing under your favorite supervision
daemon (cron, upstart, systemd or whatever) so it can restart
automatically even if your whole computer reboots, etc.  Redis can even
mirror itself to a failover server in real time if you think you need
that, plus it can checkpoint its state to disk.


From denismfmcmahon at gmail.com  Sun Jun 21 16:52:57 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sun, 21 Jun 2015 20:52:57 +0000 (UTC)
Subject: JSON Object to CSV file
References: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
 <mm5mj6$5m8$3@dont-email.me>
 <e1a0eb4d-9594-480f-b082-142dc566dea2@googlegroups.com>
Message-ID: <mm7879$n8l$1@dont-email.me>

On Sun, 21 Jun 2015 06:57:01 -0700, sahluwalia wrote:

> On Sunday, 21 June 2015 02:47:31 UTC-4, Denis McMahon  wrote:
>> On Wed, 17 Jun 2015 08:00:11 -0700, Saran A wrote:
>> 
>> > I would like to have this JSON object written out to a CSV file so
>> > that the keys are header fields (for each of the columns) and the
>> > values are values that are associated with each header field.
>> 
>> > {
>> > "CF": {
>> .......
>> > "CF": "Fee",
>> 
>> Your json object seems to have the same key used for two elements at
>> the same level, are you sure this is legal json?

> I converted this from an XML file given to me from a third party. It is
> as is. I am not sure what you mean by "valid"; that is a very subjective
> measure for any form of quantitative or qualitative data.

Put it this way, when I feed your json object into a jason lint, the 
output is the following:

{
    "CF": "Fee",
    "ID": "2"
}

The second occurrence in the parent object of name "CF" with a value of 
string literal "Fee" overwrites the earlier name "CF" whose value is (in 
python terms) a dictionary or (in json terms) an object.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From chris at cdreimer.com  Sun Jun 21 16:53:50 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sun, 21 Jun 2015 13:53:50 -0700
Subject: Do I need license to release the Python version of old BASIC
 games?
In-Reply-To: <201506212000.t5LK0jk3014266@fido.openend.se>
References: <5587115E.30703@cdreimer.com>
 <201506212000.t5LK0jk3014266@fido.openend.se>
Message-ID: <5587245E.9020700@cdreimer.com>

On 6/21/2015 1:00 PM, Laura Creighton wrote:
> In a message of Sun, 21 Jun 2015 12:32:46 -0700, "C.D. Reimer" writes:
>
>> Do I need to release my scripts under a license? If so, which one?
> You should, because if you don't you could pop up some day and
> assert copyright and sue the hell out of people who use your code,
> which means that many people won't touch it until you license it.

I want to strike a right balance between respecting the 1987 copyright 
of the book, which much of the code was either in the public domain or 
submitted to Creative Computing magazine, and protecting my own code 
that uses the video output from the book. I'm leaning towards the MIT 
license as many of games were developed in university computer labs and 
freely shared among computer users.

For a copyright blast from the past, consider Bill Gate's open letter to 
hobbyists stealing Microsoft Basic in 1976.

https://en.wikipedia.org/wiki/Open_Letter_to_Hobbyists

Thanks,

Chris R.


From marko at pacujo.net  Sun Jun 21 16:58:18 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sun, 21 Jun 2015 23:58:18 +0300
Subject: Do I need license to release the Python version of old BASIC
 games?
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
Message-ID: <87r3p4ok2d.fsf@elektro.pacujo.net>

Laura Creighton <lac at openend.se>:

> In a message of Sun, 21 Jun 2015 12:32:46 -0700, "C.D. Reimer" writes:
>
>>Do I need to release my scripts under a license? If so, which one?
>
> You should, because if you don't you could pop up some day and assert
> copyright and sue the hell out of people who use your code, which
> means that many people won't touch it until you license it.

Converting BASIC games to Python results in derived works, which are
under the original copyright of the BASIC games.

>From the given link:

   BASIC Computer Games is copyright ? 1978 by David H. Ahl, and is
   posted on www.atariarchives.org with permission. Do not redistribute,
   mirror, or copy this online book.

So a license from David H. Ahl is required before publishing Python
translations.

As for licensing one's own code, you can grant a license (or several),
or you could place your code in the public domain.


Marko


From denismfmcmahon at gmail.com  Sun Jun 21 17:06:05 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sun, 21 Jun 2015 21:06:05 +0000 (UTC)
Subject: JSON Object to CSV File Troubleshooting
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
 <mm5n0u$5m8$5@dont-email.me>
 <44a2d55f-9312-4b70-8b23-637466b21b11@googlegroups.com>
Message-ID: <mm78vs$n8l$2@dont-email.me>

On Sun, 21 Jun 2015 07:38:13 -0700, Sahlusar wrote:

> It is difficult to explain this to someone asynchronously and without an
> in person meeting. Moreover, the strict guidelines for disclosing
> information make it difficult for me to explain the client's
> requirements and the problems that they face.
> 
> I do agree with you Denis that this is an unconventional approach. I was
> wondering then that perhaps I should add additional functionality at the
> XML to JSON step? So far, with JSON objects without nested lists (as
> values) I have been successful with this (the following is rather
> lengthy):

No, step back and force yourself to answer these questions:

Why use JSON as an intermediate step?
What benefit does using JSON as an intermediate step bring me?

I see no evidence in any of your posts that the use of JSON as an 
intermediate format for the data brings any benefit whatsoever, however I 
have seen evidence that it may be introducing errors and potential data 
loss, and it is certainly adding coding complexity.

None of these are good reasons to do it, and all of them are good reasons 
not to do it.

If your data is in XML and your requirement is for CSV, then you should 
be converting from XML to CSV.

Also stop posting reams of code. No-one is reading it. If you have a 
specific error you need to fix, then post a shortest possible example of 
code that generates the error. This should never be more than about 10 
lines.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From torriem at gmail.com  Sun Jun 21 17:28:18 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sun, 21 Jun 2015 15:28:18 -0600
Subject: Do I need license to release the Python version of old BASIC
 games?
In-Reply-To: <87r3p4ok2d.fsf@elektro.pacujo.net>
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net>
Message-ID: <55872C72.4000906@gmail.com>

On 06/21/2015 02:58 PM, Marko Rauhamaa wrote:
> Laura Creighton <lac at openend.se>:
> 
>> In a message of Sun, 21 Jun 2015 12:32:46 -0700, "C.D. Reimer" writes:
>>
>>> Do I need to release my scripts under a license? If so, which one?
>>
>> You should, because if you don't you could pop up some day and assert
>> copyright and sue the hell out of people who use your code, which
>> means that many people won't touch it until you license it.
> 
> Converting BASIC games to Python results in derived works, which are
> under the original copyright of the BASIC games.
> 
> From the given link:
> 
>    BASIC Computer Games is copyright ? 1978 by David H. Ahl, and is
>    posted on www.atariarchives.org with permission. Do not redistribute,
>    mirror, or copy this online book.
> 
> So a license from David H. Ahl is required before publishing Python
> translations.
> 
> As for licensing one's own code, you can grant a license (or several),
> or you could place your code in the public domain.

I disagree.  Especially where the resulting python program is not a
transliteration (which it certainly won't be) of the original BASIC
programs.  As well, these programs implement well-known algorithms and
games.  There are only so many ways to implement certain algorithms, and
each implementation is going to be closely similar.  The games and
algorithms in his BASIC programs are by no means original or not
well-known and well-discussed.

Also the quoted copyright notice is for the entire work, which is to say
the book.  Chris is in no way reproducing the guy's text in whole or in
part--I note that the notice says nothing about fair use.

That's not to say the original book author couldn't pursue a copyright
claim, even if it's without merit.

As to the question of assigning a copyright license to code, in this
case I suggest just releasing the code marked as public domain.  The
programs are small and trivial.  If you do not release it explicitly as
public domain than you must release it under *some* license, because to
not do so prohibits anyone from doing *anything* with your code by
default, something a lot of githubbers seem to forget.  Also as long as
the code is entirely your work (and it is, despite Marko's assertions
about derivative works), you can change the license at any time.


From chris at cdreimer.com  Sun Jun 21 17:52:14 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sun, 21 Jun 2015 14:52:14 -0700
Subject: Do I need license to release the Python version of old BASIC
 games?
In-Reply-To: <87r3p4ok2d.fsf@elektro.pacujo.net>
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net>
Message-ID: <5587320E.3050105@cdreimer.com>



On 6/21/2015 1:58 PM, Marko Rauhamaa wrote:
> Converting BASIC games to Python results in derived works, which are
> under the original copyright of the BASIC games.
>
>  From the given link:
>
>     BASIC Computer Games is copyright ? 1978 by David H. Ahl, and is
>     posted on www.atariarchives.org with permission. Do not redistribute,
>     mirror, or copy this online book.
>
> So a license from David H. Ahl is required before publishing Python
> translations.

The copyright applies to the book ("Do not redistribute, mirror, or copy 
this *online book*.") and any derivative work is based on the book. 
Using the video output from the BASIC games in the book could fall 
underneath the fair use provision, which allows me to use a small 
portion of the book without infringing on the copyright. I'm not 
publishing a book. I just want to put my code on a website as an 
educational example of what I did to convert a spaghetti language into a 
modern scripting language.

Thanks,

Chris R.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150621/2553a81d/attachment.html>

From liik.joonas at gmail.com  Sun Jun 21 17:55:11 2015
From: liik.joonas at gmail.com (Joonas Liik)
Date: Mon, 22 Jun 2015 00:55:11 +0300
Subject: JSON Object to CSV File Troubleshooting
In-Reply-To: <44a2d55f-9312-4b70-8b23-637466b21b11@googlegroups.com>
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
 <mm5n0u$5m8$5@dont-email.me>
 <44a2d55f-9312-4b70-8b23-637466b21b11@googlegroups.com>
Message-ID: <CAB1GNpT08XLeUbGXrh6z3THM1Qb64p+iUtBFvsw6wpRAn+DPLw@mail.gmail.com>

On 21 June 2015 at 17:38, Sahlusar <ahlusar.ahluwalia at gmail.com> wrote:
>
> [snip]
> I do agree with you Denis that this is an unconventional approach. I was wondering then that perhaps I should add additional functionality at the XML to JSON step? So far, with JSON objects without nested lists (as values) I have been successful with this (the following is rather lengthy):
> [snip]


> ##JSON sample:
>
> data2 = {
>     "OTF": "0",
>     "F": "False",
>     "F": {
>         "Int32": ["0",
>         "0",
>         "0",
>         "0"]
>     },
> [snip]
>                 "PBDS": {
>                     "DateTime": ["1/1/0001 12:00:00 AM",
>                     "1/1/0001 12:00:00 AM",
>                     "1/1/0001 12:00:00 AM",
>                     "1/1/0001 12:00:00 AM"]
>                 },
>                 "PBDS": {
>                     "Double": ["0",
>                     "0",
>                     "0"]
>                 },
>                 "SCS": {
>                     "String": ["1",
>                     "2"]
>                 }
>             }
>
> The result:

and compare those closely now....

>
> {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
>  'F_Int32': ['0',
>   '0',
>   '0',
>   '0'],
>  'OTF': '0',
>  'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'],
>  'SCS_String': ['1', '2']}
>
Notice in the original text you have 2 entries under the name F and
later 2 entiries under the name PBDS. in the result you are missing
the first entry of each.
you say you have succeeded in generating json, unless you meant to
throw away huge swafts of data i would say... nope..



[snip]
>
> I know that this is alot of sequential steps. I am wondering if I could insert or conditionally pass these functions when originally parsing the XML, so that the JSON is formatted for more recursive reading of the JSON dictionary and then writing to CSV? I welcome constructive feedback for refactoring....

theres things you could do to fix up the generated json .. tho really,

stop generating json when you need to generate csv.
you are winning nothing. you are losing.. well pretty much .. a little
of everything .. by doing this

there are fundemental properties of xml and json you fail to grasp,
you are touting code claiming that it works when the output it
produces is horribly deformed :(

In xml for instance this is valid:

<a>
 <b>1</b>
</a>
.. and so is this:
<a>
 <b>1</b>
 <b>2</b>
</a>

a naive translatio n of the first might yield
{"a":
 {"b":1}
}
but this will not work with the second example, it would emit
{"a":
 {"b":1,"b":2}
}
which really means
{"a":
 {"b":2}
}

if you insist on emitting json as an intermediate step you need to
take care of these inconsistencies somehow.
you need to decide which behaviour you want and be explicit about it.
is it desireable that the last entry overrites the previous one? (you
have this now, i doubt this is what you want)
would you like some mergine behaviour? (some config file might work
well with this, or not)
would you like to have every entry be a list? (this is simple, but you
will end up with a lot of junk like {a:[{b:[1]}]}
do you wrap some things in list but not others?

and the conversion from json to CSV has similar issues ofc.


From torriem at gmail.com  Sun Jun 21 18:01:58 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sun, 21 Jun 2015 16:01:58 -0600
Subject: Do I need license to release the Python version of old BASIC
 games?
In-Reply-To: <5587320E.3050105@cdreimer.com>
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net> <5587320E.3050105@cdreimer.com>
Message-ID: <55873456.4060107@gmail.com>

On 06/21/2015 03:52 PM, C.D. Reimer wrote:
> The copyright applies to the book ("Do not redistribute, mirror, or copy 
> this *online book*.") and any derivative work is based on the book. 
> Using the video output from the BASIC games in the book could fall 
> underneath the fair use provision, which allows me to use a small 
> portion of the book without infringing on the copyright. I'm not 
> publishing a book. I just want to put my code on a website as an 
> educational example of what I did to convert a spaghetti language into a 
> modern scripting language.

Completely agree. Either post code with your own copyright and pick any
license (you can change it later), such as BSD, or just release it as
public domain.  I think your work could really be a good thing for
teaching purposes.



From marko at pacujo.net  Sun Jun 21 18:02:08 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 22 Jun 2015 01:02:08 +0300
Subject: Do I need license to release the Python version of old BASIC
 games?
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net>
 <mailman.684.1434923546.13271.python-list@python.org>
Message-ID: <87k2uwoh3z.fsf@elektro.pacujo.net>

"C.D. Reimer" <chris at cdreimer.com>:

> Using the video output from the BASIC games in the book could fall
> underneath the fair use provision, which allows me to use a small
> portion of the book without infringing on the copyright. I'm not
> publishing a book. I just want to put my code on a website as an
> educational example of what I did to convert a spaghetti language into
> a modern scripting language.

As they say, tell that to the judge.


Marko


From marko at pacujo.net  Sun Jun 21 18:13:08 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 22 Jun 2015 01:13:08 +0300
Subject: Do I need license to release the Python version of old BASIC
 games?
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net>
 <mailman.683.1434922110.13271.python-list@python.org>
Message-ID: <87fv5kogln.fsf@elektro.pacujo.net>

Michael Torrie <torriem at gmail.com>:

> On 06/21/2015 02:58 PM, Marko Rauhamaa wrote:
>> Converting BASIC games to Python results in derived works, which are
>> under the original copyright of the BASIC games.
>> 
>> [...]
>
> I disagree. Especially where the resulting python program is not a
> transliteration (which it certainly won't be) of the original BASIC
> programs. As well, these programs implement well-known algorithms and
> games. There are only so many ways to implement certain algorithms,
> and each implementation is going to be closely similar. The games and
> algorithms in his BASIC programs are by no means original or not
> well-known and well-discussed.

The court will have to decide if the Python version is a
reimplementation or a more or less direct translation of the original.

> Also the quoted copyright notice is for the entire work, which is to say
> the book.  Chris is in no way reproducing the guy's text in whole or in
> part--I note that the notice says nothing about fair use.

Fair use is not granted.

   In United States copyright law, fair use is a doctrine that permits
   limited use of copyrighted material without acquiring permission from
   the rights holders.

   <URL: https://en.wikipedia.org/wiki/Fair_use>

Again, whether a fair use defense applies in this case is for the courts
to decide. My guess is it wouldn't apply here.


Marko


From breamoreboy at yahoo.co.uk  Sun Jun 21 18:35:03 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 21 Jun 2015 23:35:03 +0100
Subject: Do I need license to release the Python version of old BASIC
 games?
In-Reply-To: <5587320E.3050105@cdreimer.com>
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net> <5587320E.3050105@cdreimer.com>
Message-ID: <mm7e6o$oc6$1@ger.gmane.org>

On 21/06/2015 22:52, C.D. Reimer wrote:
>
>
> On 6/21/2015 1:58 PM, Marko Rauhamaa wrote:
>> Converting BASIC games to Python results in derived works, which are
>> under the original copyright of the BASIC games.
>>
>>  From the given link:
>>
>>     BASIC Computer Games is copyright ? 1978 by David H. Ahl, and is
>>     posted onwww.atariarchives.org  with permission. Do not redistribute,
>>     mirror, or copy this online book.
>>
>> So a license from David H. Ahl is required before publishing Python
>> translations.
>
> The copyright applies to the book ("Do not redistribute, mirror, or copy
> this *online book*.") and any derivative work is based on the book.
> Using the video output from the BASIC games in the book could fall
> underneath the fair use provision, which allows me to use a small
> portion of the book without infringing on the copyright. I'm not
> publishing a book. I just want to put my code on a website as an
> educational example of what I did to convert a spaghetti language into a
> modern scripting language.
>
> Thanks,
>
> Chris R.
>

Regarding fair use I suppose it depends on which country you're (plural) 
in.  From 
http://www.copyrightservice.co.uk/copyright/p22_derivative_works.en.htm

<quote>
Can I claim that my copy is fair use/fair dealing, or de minimis?
Unless your activities are explicitly allowed under law, there is no 
solid legal footing for such a claim.
</quote>

Having said that I agree with Michael Torrie's earlier comments so I 
suggest you get on with it, noting at the same time that my legal skills 
are nowhere near as good as my computing skills, which is really saying 
something :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From denismfmcmahon at gmail.com  Sun Jun 21 19:33:17 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sun, 21 Jun 2015 23:33:17 +0000 (UTC)
Subject: JSON Object to CSV File Troubleshooting
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
 <mm5n0u$5m8$5@dont-email.me>
 <44a2d55f-9312-4b70-8b23-637466b21b11@googlegroups.com>
 <mailman.685.1434923721.13271.python-list@python.org>
Message-ID: <mm7hjt$n8l$4@dont-email.me>

On Mon, 22 Jun 2015 00:55:11 +0300, Joonas Liik wrote:

> In xml for instance this is valid:

> <a>
>  <b>1</b>
> </a>
> .. and so is this:
> <a>
>  <b>1</b> <b>2</b>
> </a>

What the OP needs to do is sit down with the XML and work out how it 
needs to be represented in CSV terms, and then code that transformation, 
but it appears that he's not listening.

ie, does your xml:

<a>
  <b>1</b> <b>2</b>
</a>

translate to CSV:

"a","b"   // headers
"b",1     // data row 1
"b",2     // data row 2

or to CSV:

"a", "b", "b"     // headers
"", 1, 2          // data row

or even CSV:

"b"   // headers
1     // data row 1
2     // data row 2

If he can't codify that in a consistent manner across all the XML he 
wishes to process, then he really does need to find someone competent to 
do the job instead of wallowing around in json until the client gives up 
in despair at the lack of progress and finds someone else to do the job.

This should really have been defined by whoever set the task to do the 
conversion. If the job is to convert from some XML DTD to a CSV format, 
then there should be a clear description of what extracts from the XML 
are expected to be in which positions in the CSV.

This is the sort of data conversion code I generally turn out in a day or 
so, it's hardly rocket science as long as you have a clear description of 
what is required. If you don't have a clear description of what is 
required, you have to keep asking questions until you get one.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From ahlusar.ahluwalia at gmail.com  Sun Jun 21 19:38:40 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Sun, 21 Jun 2015 16:38:40 -0700 (PDT)
Subject: JSON Object to CSV file
In-Reply-To: <mm7879$n8l$1@dont-email.me>
References: <dd791644-35b4-4400-a683-d409808c0c5b@googlegroups.com>
 <mm5mj6$5m8$3@dont-email.me>
 <e1a0eb4d-9594-480f-b082-142dc566dea2@googlegroups.com>
 <mm7879$n8l$1@dont-email.me>
Message-ID: <0eadbf86-420c-4916-b94d-612b222e92a0@googlegroups.com>

On Sunday, June 21, 2015 at 4:54:27 PM UTC-4, Denis McMahon wrote:
> On Sun, 21 Jun 2015 06:57:01 -0700, sahluwalia wrote:
> 
> > On Sunday, 21 June 2015 02:47:31 UTC-4, Denis McMahon  wrote:
> >> On Wed, 17 Jun 2015 08:00:11 -0700, Saran A wrote:
> >> 
> >> > I would like to have this JSON object written out to a CSV file so
> >> > that the keys are header fields (for each of the columns) and the
> >> > values are values that are associated with each header field.
> >> 
> >> > {
> >> > "CF": {
> >> .......
> >> > "CF": "Fee",
> >> 
> >> Your json object seems to have the same key used for two elements at
> >> the same level, are you sure this is legal json?
> 
> > I converted this from an XML file given to me from a third party. It is
> > as is. I am not sure what you mean by "valid"; that is a very subjective
> > measure for any form of quantitative or qualitative data.
> 
> Put it this way, when I feed your json object into a jason lint, the 
> output is the following:
> 
> {
>     "CF": "Fee",
>     "ID": "2"
> }
> 
> The second occurrence in the parent object of name "CF" with a value of 
> string literal "Fee" overwrites the earlier name "CF" whose value is (in 
> python terms) a dictionary or (in json terms) an object.
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com

I am stepping back and rethinking the notion of JSON/dictionaries. Maybe this was a very overcomplicated approach for this type of data. 


From ahlusar.ahluwalia at gmail.com  Sun Jun 21 19:42:21 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Sun, 21 Jun 2015 16:42:21 -0700 (PDT)
Subject: JSON Object to CSV File Troubleshooting
In-Reply-To: <mailman.685.1434923721.13271.python-list@python.org>
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
 <mm5n0u$5m8$5@dont-email.me>
 <44a2d55f-9312-4b70-8b23-637466b21b11@googlegroups.com>
 <mailman.685.1434923721.13271.python-list@python.org>
Message-ID: <ef21afd1-354b-40ae-b8a4-d5e88c5796e2@googlegroups.com>

On Sunday, June 21, 2015 at 5:56:00 PM UTC-4, Waffle wrote:
> On 21 June 2015 at 17:38, Sahlusar <ahlusar.ahluwalia at gmail.com> wrote:
> >
> > [snip]
> > I do agree with you Denis that this is an unconventional approach. I was wondering then that perhaps I should add additional functionality at the XML to JSON step? So far, with JSON objects without nested lists (as values) I have been successful with this (the following is rather lengthy):
> > [snip]
> 
> 
> > ##JSON sample:
> >
> > data2 = {
> >     "OTF": "0",
> >     "F": "False",
> >     "F": {
> >         "Int32": ["0",
> >         "0",
> >         "0",
> >         "0"]
> >     },
> > [snip]
> >                 "PBDS": {
> >                     "DateTime": ["1/1/0001 12:00:00 AM",
> >                     "1/1/0001 12:00:00 AM",
> >                     "1/1/0001 12:00:00 AM",
> >                     "1/1/0001 12:00:00 AM"]
> >                 },
> >                 "PBDS": {
> >                     "Double": ["0",
> >                     "0",
> >                     "0"]
> >                 },
> >                 "SCS": {
> >                     "String": ["1",
> >                     "2"]
> >                 }
> >             }
> >
> > The result:
> 
> and compare those closely now....
> 
> >
> > {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
> >  'F_Int32': ['0',
> >   '0',
> >   '0',
> >   '0'],
> >  'OTF': '0',
> >  'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'],
> >  'SCS_String': ['1', '2']}
> >
> Notice in the original text you have 2 entries under the name F and
> later 2 entiries under the name PBDS. in the result you are missing
> the first entry of each.
> you say you have succeeded in generating json, unless you meant to
> throw away huge swafts of data i would say... nope..
> 
> 
> 
> [snip]
> >
> > I know that this is alot of sequential steps. I am wondering if I could insert or conditionally pass these functions when originally parsing the XML, so that the JSON is formatted for more recursive reading of the JSON dictionary and then writing to CSV? I welcome constructive feedback for refactoring....
> 
> theres things you could do to fix up the generated json .. tho really,
> 
> stop generating json when you need to generate csv.
> you are winning nothing. you are losing.. well pretty much .. a little
> of everything .. by doing this
> 
> there are fundemental properties of xml and json you fail to grasp,
> you are touting code claiming that it works when the output it
> produces is horribly deformed :(
> 
> In xml for instance this is valid:
> 
> <a>
>  <b>1</b>
> </a>
> .. and so is this:
> <a>
>  <b>1</b>
>  <b>2</b>
> </a>
> 
> a naive translatio n of the first might yield
> {"a":
>  {"b":1}
> }
> but this will not work with the second example, it would emit
> {"a":
>  {"b":1,"b":2}
> }
> which really means
> {"a":
>  {"b":2}
> }
> 
> if you insist on emitting json as an intermediate step you need to
> take care of these inconsistencies somehow.
> you need to decide which behaviour you want and be explicit about it.
> is it desireable that the last entry overrites the previous one? (you
> have this now, i doubt this is what you want)
> would you like some mergine behaviour? (some config file might work
> well with this, or not)
> would you like to have every entry be a list? (this is simple, but you
> will end up with a lot of junk like {a:[{b:[1]}]}
> do you wrap some things in list but not others?
> 
> and the conversion from json to CSV has similar issues ofc.


I am rethinking my strategy and going back to the drawing board. The data is malformed. I am quite familiar with JSON and XML. I am trying to work within the parameters given to me. I will rethink this. Thank you for giving me a reality check with a splash of water. Sometimes it is hard to get one's head out of the ground....especially when one is new to enterprise and commercial development. 


From chris at cdreimer.com  Sun Jun 21 19:54:02 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sun, 21 Jun 2015 16:54:02 -0700
Subject: Do I need license to release the Python version of old BASIC
 games?
In-Reply-To: <87k2uwoh3z.fsf@elektro.pacujo.net>
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net>
 <mailman.684.1434923546.13271.python-list@python.org>
 <87k2uwoh3z.fsf@elektro.pacujo.net>
Message-ID: <55874E9A.6050303@cdreimer.com>

On 6/21/2015 3:02 PM, Marko Rauhamaa wrote:
> As they say, tell that to the judge. 

More than likely, the original copyright owner can issue an DMCA take 
down notice and that will be end of that.

Thanks,

Chris R.




From ahlusar.ahluwalia at gmail.com  Sun Jun 21 19:56:27 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Sun, 21 Jun 2015 16:56:27 -0700 (PDT)
Subject: JSON Object to CSV File Troubleshooting
In-Reply-To: <mm7hjt$n8l$4@dont-email.me>
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
 <mm5n0u$5m8$5@dont-email.me>
 <44a2d55f-9312-4b70-8b23-637466b21b11@googlegroups.com>
 <mailman.685.1434923721.13271.python-list@python.org>
 <mm7hjt$n8l$4@dont-email.me>
Message-ID: <411541bf-7da6-42c1-b332-2761494d29ba@googlegroups.com>

On Sunday, June 21, 2015 at 7:34:47 PM UTC-4, Denis McMahon wrote:
> On Mon, 22 Jun 2015 00:55:11 +0300, Joonas Liik wrote:
> 
> > In xml for instance this is valid:
> 
> > <a>
> >  <b>1</b>
> > </a>
> > .. and so is this:
> > <a>
> >  <b>1</b> <b>2</b>
> > </a>
> 
> What the OP needs to do is sit down with the XML and work out how it 
> needs to be represented in CSV terms, and then code that transformation, 
> but it appears that he's not listening.
> 
> ie, does your xml:
> 
> <a>
>   <b>1</b> <b>2</b>
> </a>
> 
> translate to CSV:
> 
> "a","b"   // headers
> "b",1     // data row 1
> "b",2     // data row 2
> 
> or to CSV:
> 
> "a", "b", "b"     // headers
> "", 1, 2          // data row
> 
> or even CSV:
> 
> "b"   // headers
> 1     // data row 1
> 2     // data row 2
> 
> If he can't codify that in a consistent manner across all the XML he 
> wishes to process, then he really does need to find someone competent to 
> do the job instead of wallowing around in json until the client gives up 
> in despair at the lack of progress and finds someone else to do the job.
> 
> This should really have been defined by whoever set the task to do the 
> conversion. If the job is to convert from some XML DTD to a CSV format, 
> then there should be a clear description of what extracts from the XML 
> are expected to be in which positions in the CSV.
> 
> This is the sort of data conversion code I generally turn out in a day or 
> so, it's hardly rocket science as long as you have a clear description of 
> what is required. If you don't have a clear description of what is 
> required, you have to keep asking questions until you get one.
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com



On Sunday, June 21, 2015 at 7:34:47 PM UTC-4, Denis McMahon wrote:
> On Mon, 22 Jun 2015 00:55:11 +0300, Joonas Liik wrote:
> 
> > In xml for instance this is valid:
> 
> > <a>
> >  <b>1</b>
> > </a>
> > .. and so is this:
> > <a>
> >  <b>1</b> <b>2</b>
> > </a>
> 
> What the OP needs to do is sit down with the XML and work out how it 
> needs to be represented in CSV terms, and then code that transformation, 
> but it appears that he's not listening.
> 
> ie, does your xml:
> 
> <a>
>   <b>1</b> <b>2</b>
> </a>
> 
> translate to CSV:
> 
> "a","b"   // headers
> "b",1     // data row 1
> "b",2     // data row 2
> 
> or to CSV:
> 
> "a", "b", "b"     // headers
> "", 1, 2          // data row
> 
> or even CSV:
> 
> "b"   // headers
> 1     // data row 1
> 2     // data row 2
> 
> If he can't codify that in a consistent manner across all the XML he 
> wishes to process, then he really does need to find someone competent to 
> do the job instead of wallowing around in json until the client gives up 
> in despair at the lack of progress and finds someone else to do the job.
> 
> This should really have been defined by whoever set the task to do the 
> conversion. If the job is to convert from some XML DTD to a CSV format, 
> then there should be a clear description of what extracts from the XML 
> are expected to be in which positions in the CSV.
> 
> This is the sort of data conversion code I generally turn out in a day or 
> so, it's hardly rocket science as long as you have a clear description of 
> what is required. If you don't have a clear description of what is 
> required, you have to keep asking questions until you get one.
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com



On Sunday, June 21, 2015 at 7:34:47 PM UTC-4, Denis McMahon wrote:
> On Mon, 22 Jun 2015 00:55:11 +0300, Joonas Liik wrote:
> 
> > In xml for instance this is valid:
> 
> > <a>
> >  <b>1</b>
> > </a>
> > .. and so is this:
> > <a>
> >  <b>1</b> <b>2</b>
> > </a>
> 
> What the OP needs to do is sit down with the XML and work out how it 
> needs to be represented in CSV terms, and then code that transformation, 
> but it appears that he's not listening.
> 
> ie, does your xml:
> 
> <a>
>   <b>1</b> <b>2</b>
> </a>
> 
> translate to CSV:
> 
> "a","b"   // headers
> "b",1     // data row 1
> "b",2     // data row 2
> 
> or to CSV:
> 
> "a", "b", "b"     // headers
> "", 1, 2          // data row
> 
> or even CSV:
> 
> "b"   // headers
> 1     // data row 1
> 2     // data row 2
> 
> If he can't codify that in a consistent manner across all the XML he 
> wishes to process, then he really does need to find someone competent to 
> do the job instead of wallowing around in json until the client gives up 
> in despair at the lack of progress and finds someone else to do the job.
> 
> This should really have been defined by whoever set the task to do the 
> conversion. If the job is to convert from some XML DTD to a CSV format, 
> then there should be a clear description of what extracts from the XML 
> are expected to be in which positions in the CSV.
> 
> This is the sort of data conversion code I generally turn out in a day or 
> so, it's hardly rocket science as long as you have a clear description of 
> what is required. If you don't have a clear description of what is 
> required, you have to keep asking questions until you get one.
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com


I have gone back to the drawing board and scrapped that idea of an intermediate JSON/dictionary. 


This is the output that I seek:

> "a","b"   // headers
> "b",1     // data row 1
> "b",2     // data row 2
> 

Here is an example XML document that I am working with:

<Response ID="24856-775" RequestType="Moverview">        
        <MonthDayCount>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
        </MonthDayCount>
        <Warnings />
        <SList />
        <LList />
        <EA>Y</EA>
        <EHA>Y</EHA>
        <EBY>Y</EBY>
        <EOTH>Y</EOTH>
        <EIL>Y</EIL>
        <EM>Y</EM>
        <ED>Y</ED>
        <EQ>Y</EQ>
        <ERS>Y</ERS>
        <ECCS>Y</ECCS>
        <EES>Y</EES>
        <UAS>Y</UAS>
        <PA>False</PA>
        <PL>False</PL>
        <PC>False</PC>
        <PCs>False</PCs>
        <PJ>False</PJ>
        <OITC>0</OITC>
        <MG />
        <R />
        <CCGoods />
    </MO>
</Response>


So far I have gotten this output:

""" my output -->
('Response.RequestType', 'Moverview')
('Response.ID', '24856-775')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.MonthDayCount.Int32.', '0')
('Response.Warnings.', None)
('Response.SList.', None)
('Response.LList.', None)
('Response.EA.', 'Y')
('Response.EHA.', 'Y')
('Response.EBY.', 'Y')
('Response.EOTH.', 'Y')
('Response.EIL.', 'Y')
('Response.EM.', 'Y')
('Response.ED.', 'Y')
('Response.EQ.', 'Y')
('Response.ERS.', 'Y')
('Response.ECCS.', 'Y')
('Response.EES.', 'Y')
('Response.UAS.', 'Y')
('Response.PA.', 'False')
('Response.PL.', 'False')
('Response.PC.', 'False')
('Response.PCs.', 'False')
('Response.PJ.', 'False')
('Response.OITC.', '0')
('Response.MG.', None)
('Response.R.', None)
('Response.CCGoods.', None)
"""

The function that produced this has been redesigned to flatten the XML using a generator:

import xml.etree.cElementTree as ElementTree 
from xml.etree.ElementTree import XMLParser 
import json 
import csv 
import tokenize 
import token 
try: 
    from collections import OrderedDict 
    import json 
except ImportError: 
    from ordereddict import OrderedDict 
    import simplejson as json 
import itertools 
import six 
import string 
#from csvkit import CSVKitWriter 
def flatten_list(aList, prefix=''): 
    for element in aList:
        if element: 
            # treat like dict 
            if len(element) == 1 or element[0].tag != element[1].tag: 
                yield from flatten_dict(element, prefix)
            # treat like list 
            elif element[0].tag == element[1].tag: 
                yield from flatten_list(element, prefix)
        elif element.text: 
            text = element.text.strip() 
            if text: 
                yield prefix, text
def flatten_dict(parent_element, prefix=''):
    prefix = prefix + parent_element.tag + '.'
    if parent_element.items():
        for k, v in parent_element.items():
            yield prefix + k, v
    for element in parent_element:
        eprefix = prefix + element.tag + '.'
        if element:
            # treat like dict - we assume that if the first two tags 
            # in a series are different, then they are all different. 
            if len(element) == 1 or element[0].tag != element[1].tag: 
                yield from flatten_dict(element, prefix=prefix)
            # treat like list - we assume that if the first two tags 
            # in a series are the same, then the rest are the same. 
            else: 
                # here, we put the list in dictionary; the key is the 
                # tag name the list elements all share in common, and 
                # the value is the list itself
                yield from flatten_list(element, prefix=eprefix+element[0].tag+'.')
            # if the tag has attributes, add those to the dict
            if element.items():
                for k, v in element.items():
                    yield eprefix+k, v 
        # this assumes that if you've got an attribute in a tag, 
        # you won't be having any text. This may or may not be a 
        # good idea -- time will tell. It works for the way we are 
        # currently doing XML configuration files... 
        elif element.items(): 
            for k, v in element.items():
                yield eprefix+k, v
        # finally, if there are no child tags and no attributes, extract 
        # the text 
        else:
            yield eprefix, element.text
def main():
    with open('source.xml', 'r', encoding='utf-8') as f: 
        xml_string = f.read() 
    xml_string= xml_string.replace('&#x0;', '') #optional to remove ampersands. 
    root = ElementTree.XML(xml_string) 
    for item in flatten_dict(root):
        print(item)
if __name__ == "__main__":
    main()


I now seek to work on the following. 

1.

(**'Response.RequestType'**, 'Moverview') 
(**'Response.ID', **'24856-775') 

The ** 'sample.text.in_tag ** corresponds with the column headers. The values would be the second item in the tuple.

All of the "Int32" should be stripped away and replaced with an enumeration number corresponding with an index for the header "MonthDayCount" (for example MonthDayCount_1, MonthDayCount_2, MonthDayCount_3, MonthDayCount_4 etc)


The Responses for the rest of the elements (save for the first two above (under item number 1) could be removed). I am still trying to figure how to do that as well...

Again The first and second item in the tuple would be the header and corresponding value, respectively - just like your initial assumption. 

You are welcome to contribute and provide me with feedback. Thank you for your continued feedback and guidance. 


From rosuav at gmail.com  Sun Jun 21 22:01:53 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 22 Jun 2015 12:01:53 +1000
Subject: Do I need license to release the Python version of old BASIC
 games?
In-Reply-To: <55874E9A.6050303@cdreimer.com>
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net>
 <mailman.684.1434923546.13271.python-list@python.org>
 <87k2uwoh3z.fsf@elektro.pacujo.net> <55874E9A.6050303@cdreimer.com>
Message-ID: <CAPTjJmqisjsAtHBqOeAxP10HodO1pZy+YQHCVj_xcPPeYTjdCA@mail.gmail.com>

On Mon, Jun 22, 2015 at 9:54 AM, C.D. Reimer <chris at cdreimer.com> wrote:
> On 6/21/2015 3:02 PM, Marko Rauhamaa wrote:
>>
>> As they say, tell that to the judge.
>
>
> More than likely, the original copyright owner can issue an DMCA take down
> notice and that will be end of that.

Or, alternatively, the original copyright owner will be tickled to see
his work given a new lease on life, and will be thanking you for doing
it. That's also possible.

As long as you acknowledge the prior work, most people are happy to
see their stuff used. Slap a simple free license on your actual code
(you suggested MIT earlier, and I support that notion strongly), and
include some sort of pointer to the BASIC code and book, and most
likely the matter will never go to court. For extra safety, though, do
your best to get in contact with the original author and ask how best
to acknowledge the book. That would cover you.

ChrisA


From steve at pearwood.info  Sun Jun 21 22:17:46 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 22 Jun 2015 12:17:46 +1000
Subject: Do I need license to release the Python version of old BASIC
 games?
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net>
 <mailman.684.1434923546.13271.python-list@python.org>
 <87k2uwoh3z.fsf@elektro.pacujo.net>
 <mailman.688.1434930854.13271.python-list@python.org>
Message-ID: <5587704a$0$1656$c3e8da3$5496439d@news.astraweb.com>

On Mon, 22 Jun 2015 09:54 am, C.D. Reimer wrote:

> On 6/21/2015 3:02 PM, Marko Rauhamaa wrote:
>> As they say, tell that to the judge.
> 
> More than likely, the original copyright owner can issue an DMCA take
> down notice and that will be end of that.

The way the DMCA is implemented in practice, anybody can issue a DMCA take
down notice and that will be the end of that.


-- 
Steven



From steve at pearwood.info  Sun Jun 21 22:27:38 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 22 Jun 2015 12:27:38 +1000
Subject: Do I need license to release the Python version of old BASIC
 games?
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net>
 <mailman.683.1434922110.13271.python-list@python.org>
Message-ID: <5587729a$0$1663$c3e8da3$5496439d@news.astraweb.com>

On Mon, 22 Jun 2015 07:28 am, Michael Torrie wrote:

> As to the question of assigning a copyright license to code, in this
> case I suggest just releasing the code marked as public domain.

Public domain is not a licence, and many places (including the US) do not
allow individuals to put works into the public domain. (US government works
are a special case.) Some places will not recognise a public domain
dedication, and will treat the work as being legally copyrighted by you
with All Rights Reserved. Some legal jurisdictions prohibit owners from
surrendering some or all rights.

You might not choose to pursue a copyright claim, but your heirs might,
which means that a "public domain" release is legally unsafe unless the
work actually is in the public domain due to age.

For that reason, you are better off publishing the work with a Creative
Commons CC0 licence:

https://creativecommons.org/about/cc0

https://wiki.creativecommons.org/wiki/Public_domain


As far as whether your work counts as a derivative work, you *may* be on
solid ground if you just re-use the algorithms from the BASIC code, and not
either the author's descriptive text or the BASIC code itself. But I am not
a lawyer (neither is anyone else here).

My advice is to make an attempt to contact the original author, or at least
the atarigames website, explain that you have converted the code to Python,
and ask permission to publish under the cc0 licence or similar. It's just
safer that way.



-- 
Steven



From torriem at gmail.com  Mon Jun 22 00:23:54 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sun, 21 Jun 2015 22:23:54 -0600
Subject: Do I need license to release the Python version of old BASIC
 games?
In-Reply-To: <5587729a$0$1663$c3e8da3$5496439d@news.astraweb.com>
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net>
 <mailman.683.1434922110.13271.python-list@python.org>
 <5587729a$0$1663$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <55878DDA.4070407@gmail.com>

On 06/21/2015 08:27 PM, Steven D'Aprano wrote:
> Public domain is not a licence, and many places (including the US) do not
> allow individuals to put works into the public domain. (US government works
> are a special case.) Some places will not recognise a public domain
> dedication, and will treat the work as being legally copyrighted by you
> with All Rights Reserved. Some legal jurisdictions prohibit owners from
> surrendering some or all rights.
> 
> You might not choose to pursue a copyright claim, but your heirs might,
> which means that a "public domain" release is legally unsafe unless the
> work actually is in the public domain due to age.

Yes of course public domain is not a source code license.  That's not
what I meant anyway.  But sure, pick some license like MIT or BSD and
call it good.

As for it being not possible to place something in the public domain,
that's news to me and certainly news to a number of projects including
Libravox. All of their recordings state that they are in the public
domain.  Of course the texts being read are in the public domain by
virtue of their copyright expiring.  But the recordings themselves could
be copyrighted, but are not.

>From some brief research, it appears there is some question about the
ability to declare something to be in the public domain, but it is by no
means a sure thing and lots of people feel it's just fine to declare
something to be in the public domain.  What his heirs try to assert is
probably not his concern nor would I worry about it.  This is wandering
in the weeds in typical python-list style!


From rosuav at gmail.com  Mon Jun 22 00:45:12 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 22 Jun 2015 14:45:12 +1000
Subject: Do I need license to release the Python version of old BASIC
 games?
In-Reply-To: <55878DDA.4070407@gmail.com>
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net>
 <mailman.683.1434922110.13271.python-list@python.org>
 <5587729a$0$1663$c3e8da3$5496439d@news.astraweb.com>
 <55878DDA.4070407@gmail.com>
Message-ID: <CAPTjJmoGWP+YbUFgx5ngp2ZRwdvBqTFmoU9LQVuO7dbO=8Vvug@mail.gmail.com>

On Mon, Jun 22, 2015 at 2:23 PM, Michael Torrie <torriem at gmail.com> wrote:
> From some brief research, it appears there is some question about the
> ability to declare something to be in the public domain, but it is by no
> means a sure thing and lots of people feel it's just fine to declare
> something to be in the public domain.  What his heirs try to assert is
> probably not his concern nor would I worry about it.  This is wandering
> in the weeds in typical python-list style!

It's a consideration for anyone who wants to make use of it. Imagine,
instead of stating that the recordings are in the public domain, it
said "Copyright, blah blah, but we pledge not to sue anyone for
copyright violation for an unspecified and unspecifiable period".
Would you use it? Because that's potentially what a "public domain"
declaration is, with the period being defined as "until the current
owner dies". That's not something to depend on, so I'd much rather
give things away under something more explicit like MIT.

ChrisA


From ryexander at gmail.com  Mon Jun 22 04:32:48 2015
From: ryexander at gmail.com (Ben Powers)
Date: Mon, 22 Jun 2015 02:32:48 -0600
Subject: Pyitect - Plugin architectural system for Python 3.0+
 (feedback?)
Message-ID: <CAO-rxQuEZ3e2MiaTh4dpWdcbQFcETrFEOH9oKiyh5AmQW=+Agw@mail.gmail.com>

on Tue, Jun 16,  2015 at 17:49 Ian Kelly <ian.g.kelly at gmail.com> wrote

>On Mon, Jun 8, 2015 at 10:42 PM, Ben Powers <ryexander at gmail.com> wrote:
>> As importlib has been added in python 3 and up I decided to use it's
>> abilities to create a plugin system for truly modular development in python.
>>
>> Pyitect has the ability to drop in components and resolve dependencies. Even
>> load different versions of a dependency if two different libraries require
>> two different versions.
>>
>> Pyitect gives each "plugin" it won private namespace to import the
>> components it needs. and nothing else.
>>
>> Pyitect  organizes it self around the concept of a System. Each system can
>> scan and load it's own set of plugins, in each system plugins can be enabled
>> individually.
>>
>> Pyitect even comes with a very simple event system that it uses internal to
>> fire event when plugins are found/loaded/ect for the developer to use.
>>
>> I've written up What I think is a quite comprehensive readme to go with it.
>>
>> Available on PyPi https://pypi.python.org/pypi?:action=display&name=pyitect
>> and Github https://github.com/Ryex/pyitect
>>
>> I'm looking for feedback on what flaws the system may have and where I can
>> improve. I'm always open to pull requests
>>
>> Looking forward to what the comunity thinks of this little project of mine.
>> Thanks for your input and criticism.>

>You need to run your readme through a spell-check, as it's riddled
>with typographical errors and quite hard to read. A grammar check and
>a proofreader would also be useful.>


Actually ya, looking through it again there are number that I didn't find

>> "on_enable": "dostuff.py", // optional, runs this file when the plugin is enabled>

>A path to a function would feel more Pythonic.>


You right that WOULD be more pythonic, Why I didn't think of that at
the time is a compleat mistery to me.


>> mode -> (OPTIONAL) defaults to import on python 3.4 and up ecec otherwise: sets the import mode>

>Do the plugins need to be on the sys.path in order to be imported?>


No. plugins need only be refered to by a path relative to their .json file.

>> provides -> a mapping of provided component names to prefix mappings>

>What are the "prefix mappings" and what are they used for?>

>On Mon, Jun 8, 2015 at 10:42 PM, Ben Powers <ryexander at gmail.com> wrote:
>> As importlib has been added in python 3 and up I decided to use it's
>> abilities to create a plugin system for truly modular development in python.
>>
>> Pyitect has the ability to drop in components and resolve dependencies. Even
>> load different versions of a dependency if two different libraries require
>> two different versions.
>>
>> Pyitect gives each "plugin" it won private namespace to import the
>> components it needs. and nothing else.
>>
>> Pyitect  organizes it self around the concept of a System. Each system can
>> scan and load it's own set of plugins, in each system plugins can be enabled
>> individually.
>>
>> Pyitect even comes with a very simple event system that it uses internal to
>> fire event when plugins are found/loaded/ect for the developer to use.
>>
>> I've written up What I think is a quite comprehensive readme to go with it.
>>
>> Available on PyPi https://pypi.python.org/pypi?:action=display&name=pyitect
>> and Github https://github.com/Ryex/pyitect
>>
>> I'm looking for feedback on what flaws the system may have and where I can
>> improve. I'm always open to pull requests
>>
>> Looking forward to what the comunity thinks of this little project of mine.
>> Thanks for your input and criticism.>

>You need to run your readme through a spell-check, as it's riddled
>with typographical errors and quite hard to read. A grammar check and
>a proofreader would also be useful.>

Actually ya, looking through it again there are number that I didn't find

>> "on_enable": "dostuff.py", // optional, runs this file when the plugin is enabled>

>A path to a function would feel more Pythonic.>

You right that WOULD be more pythonic, Why I didn't think of that at
the time is a compleat mistery to me.

>> mode -> (OPTIONAL) defaults to import on python 3.4 and up ecec otherwise: sets the import mode>

>Do the plugins need to be on the sys.path in order to be imported?>


No. plugins need only be refered to by a path relative to their .json file.

>> provides -> a mapping of provided component names to prefix mappings>

>What are the "prefix mappings" and what are they used for?>

That should actually read "postfix mappings" my bad.

normally when you list the components a plugin provides the names of
the components are assumed to exist in the plugin's module name space,
and when the components are required pyitect simply pulls the objects
by name out of the module and passes them along.

postfix mappings allow components to be named differently in the
module than they are provided and allows the developer to attach
version postfixes to the component.

an example from the README:
{
    "name": "Im-A-Plugin",
    ...
    "version": "0.0.1",
    ...
    "provides": {
        "Bar": "bar_type_1=bar1"
    }
}

This provides the "Bar" component at version 0.0.1-bar_type_1 by
pulling the name `bar1` from the imported plugin module.
the the version postfix string was "=bar1" then the postfix the the
version would be removed just 0.0.1 it would still import under `bar1`
instead of `Bar`

>> #file.py
>> from PyitectConsumes import foo
>>
>> class Bar(object):
>>     def __init__():
>>         foo("it's a good day to be a plugin")>

>I don't understand this example. Is "PyitectConsumes" the name of a
>component or part of Pyitect? Or is "foo" the name of the component?>


`PyitectComsumes` is a special module injected into sys.modules when
the plugin module is imported and removed after the import is
complete. it's name-space contains all the components declared as
required in the plugin's .json


>> plugins = [system.plugins[n][v] for n in system.plugins for v in system.plugins[n]]>

>What are n and v?>

`n` and `v` are short variable names for `name` and 'version'

system.plugins is a mapping of plugin names to a mapping of plugin
versions to plugin objects.

ie:

{
	"Plugin_Name": {
	    "0.0.1": <pyitect.Plugin Object>
	}
}

this is working python code that pulls all the plugin objects in the
system into an array, potentially to be enabled.
filtering could then be done when creating this array or after if it
needed to prevent plugins from being enabled or the list could be
constructed manually from a white list.

plugins can be searched for and or enabled at anytime in the plugin
system life cycle but the components they provide are not availably
until they are enabled.


>> Plugins are loaded on demand when a component is loaded via
>>
>> System.load("<component name>")>

>What's the difference between this and the "PyitectConsumes" import?>


The PyitectConsumes import is for use inside plugins where
dependencies are declared prior to import.

System.load is for use in the application to load components. Plugins
are not intended to know about the System they are being used in.


>> ### both in both cases relative imports DO NOT WORK. the plugin folder is temporarily added to the search path so absolute imports work but relatives will not.
>>
>> UNLESS the name of the file is __init__.py . In this special case the plugin folder is reconsidered as a python package and relative imports work as normal.>

>Isn't this just how relative imports normally work? Of course you
>can't use a relative import outside of a package.>


This is true, however some might make the mistake of assuming that
plugins are treated like packages even they do not include a
__init__.py file in the plugin's folder when really they are treated
as any python module.


>> Loaded pluginss do NOT store their module object in sys.modules>

>What about imports of plugin-local modules that are performed inside
>the plugins? Do those also get cleaned up from sys.modules?>


Actually no... thats an oversight on my part. really I should
temporally clone sys.modules inject the PyitectConsumes special module
and then set it back to the original sys.modules dict after the import
has been performed, that would secure the process more and prevent
cluttering of sys.modules.

thank you for pointing this out.

>If I have version 1 of a plugin that imports a module from its
>package, and version 2 of the same plugin that imports a module from
>its package, are those imports going to conflict with one another?>

when the plugin's are enabled their components are mapped into the
system. the first time a component is required and a compatible
version to the request is available a fresh module object with a blank
namespace is constructed and the file declared in the .json is
imported/executed (via `compile` ing a code object then `exec` acuting
the code object) with the fresh plugin obj's namespace.

as the only way to have to plugins with different version is the have
two different folders and given my understanding of how python
searches for modules to import I do believe it is impossible for there
to be conflict with imports internal to a plugin.



>Also, earlier you appeared to instantiate the System class, but now
>you appear to be just accessing attributes from the class directly.
>What's the distinction between the class-level state and the
>instance-level state?>


The class level state does not exist. that I suppose is a
miscommunication of the README. to use a plugin system you must
instantiate a System object and then populate it with plugins to use,
then enable the plugins to make their provided components available
for use.


>> System.useing
>> System.enabeled_plugins

>Are these misspellings actually part of the API?>

Yes and no, to my shame System.useing instead of System.using exists
but System.enabled_plugins is correctly spelled in the API

>Also, earlier you appeared to instantiate the System class, but now
>you appear to be just accessing attributes from the class directly.
>What's the distinction between the class-level state and the
>instance-level state?>

The class level state does not exist. that I suppose is a
miscommunication of the README. to use a plugin system you must
instantiate a System object and then populate it with plugins to use,
then enable the plugins to make their provided components available
for use.

>> Providing multiple versions of a component from the same plugin>

>Is there a reason to do this as opposed to simply providing multiple components?

Yes actually.

imagine the case where you have a set of data to which there are many
operation that could be done. you are going to operate on this data
through a interface.
lets say you dub components that implement this interface "tools". you
want plugins to provide tools to work with the data in different ways.
So, you use a "Tool" component. Different versions of the "Tool"
component in this case can be assumed to be different tools.
If someone wants to provide several tools but only wants to write one
plugin then they would need to provide different version of the same
component from the same plugin.

The application could then enumerate over the different versions of
the component available, perhaps filtering out those that appear to be
different version of the same tool if there was some convention
applied to the version string (ie. including the tool name in the
version string 0.0.1-tool-x" (postfix mapping?))
and call on each tool as necessary

There are many possible use cases, the point was to make the system
flexible. Perhaps there is a better way of achieving this goal but
thats what I came up with.


Hopefully I answer your questions to your satisfaction, thanks for the feedback.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150622/7ec2cf74/attachment.html>

From mal at europython.eu  Mon Jun 22 04:45:42 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Mon, 22 Jun 2015 10:45:42 +0200
Subject: EuroPython 2015: Partner program deadline extended
Message-ID: <5587CB36.9080804@europython.eu>

We have negotiated an extension of the deadline to register for the
partner program tours with the local tour operator.

              *** EuroPython 2015 Partner Program ***

      https://ep2015.europython.eu/en/events/partner-program/

There is plenty to see in and around Bilbao. We have worked out a
set of interesting tours, together with a local tour company to choose
from, for partners and EuroPython attendees alike:

 * Panoramic tour of Bilbao
 * Excursion to San Sebastian
 * Guggenheim Museum
 * Excursion to La Rioja
 * Boat tour of Bilbao

The tours include travel, lunch and tickets (where needed).

We?d like to encourage early sign-up, since seats are limited.

                New registration deadline is July 4th.

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From lac at openend.se  Mon Jun 22 05:20:51 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 22 Jun 2015 11:20:51 +0200
Subject: Do I need license to release the Python version of old BASIC
 games?
In-Reply-To: Message from Michael Torrie <torriem@gmail.com>
 of "Sun, 21 Jun 2015 22:23:54 -0600." <55878DDA.4070407@gmail.com>
References: <5587115E.30703@cdreimer.com>
 <mailman.681.1434916857.13271.python-list@python.org>
 <87r3p4ok2d.fsf@elektro.pacujo.net>
 <mailman.683.1434922110.13271.python-list@python.org>
 <5587729a$0$1663$c3e8da3$5496439d@news.astraweb.com><55878DDA.4070407@gmail.com>
Message-ID: <201506220920.t5M9KpLA000586@fido.openend.se>

In a message of Sun, 21 Jun 2015 22:23:54 -0600, Michael Torrie writes:
>>From some brief research, it appears there is some question about the
>ability to declare something to be in the public domain, but it is by no
>means a sure thing and lots of people feel it's just fine to declare
>something to be in the public domain.  What his heirs try to assert is
>probably not his concern nor would I worry about it.  This is wandering
>in the weeds in typical python-list style!

No the worry about his heirs is something that the users of his code
have to worry about.  i.e. a teacher who wants to use it but needs
to get permission from the school.  The school may have a blanket
'no copyright, no use' policy, just to protect them from the
possible claims of heirs.

Laura



From robin at reportlab.com  Mon Jun 22 05:24:22 2015
From: robin at reportlab.com (Robin Becker)
Date: Mon, 22 Jun 2015 10:24:22 +0100
Subject: instance as module
In-Reply-To: <55851538$0$1667$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.626.1434706168.13271.python-list@python.org>
 <55851538$0$1667$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5587D446.10808@chamonix.reportlab.co.uk>

On 20/06/2015 08:24, Steven D'Aprano wrote:
> On Fri, 19 Jun 2015 07:29 pm, Robin Becker wrote:
>
>> I'm trying to overcome a recursive import issue in reportlab.
..........
>
> I'm afraid I don't understand what you are trying to say here. Why can't the
> user just set up "such a default" e.g. canvas_basefontname. Surely that is
> a string, e.g. "Comic Sans".
>
> Why will a recursive import occur? If the user doesn't want a recursive
> import, they need only ensure that they don't import anything that relies
> on rl_config.
>
>............
I'm afraid this is a failure on the part of reportlab's font handling. For 
almost all user access the font is indeed a string; unfortunately we need also 
to load data for a typical font (eg arial) so that it can be used properly. To 
load the data properly requires other bits of reportlab which use rl_config.

Being able to specify both the font token to use and the method of 
initialization simultaneously seems useful.
-- 
Robin Becker



From robin at reportlab.com  Mon Jun 22 05:24:22 2015
From: robin at reportlab.com (Robin Becker)
Date: Mon, 22 Jun 2015 10:24:22 +0100
Subject: instance as module
In-Reply-To: <55851538$0$1667$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.626.1434706168.13271.python-list@python.org>
 <55851538$0$1667$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5587D446.10808@chamonix.reportlab.co.uk>

On 20/06/2015 08:24, Steven D'Aprano wrote:
> On Fri, 19 Jun 2015 07:29 pm, Robin Becker wrote:
>
>> I'm trying to overcome a recursive import issue in reportlab.
..........
>
> I'm afraid I don't understand what you are trying to say here. Why can't the
> user just set up "such a default" e.g. canvas_basefontname. Surely that is
> a string, e.g. "Comic Sans".
>
> Why will a recursive import occur? If the user doesn't want a recursive
> import, they need only ensure that they don't import anything that relies
> on rl_config.
>
>............
I'm afraid this is a failure on the part of reportlab's font handling. For 
almost all user access the font is indeed a string; unfortunately we need also 
to load data for a typical font (eg arial) so that it can be used properly. To 
load the data properly requires other bits of reportlab which use rl_config.

Being able to specify both the font token to use and the method of 
initialization simultaneously seems useful.
-- 
Robin Becker


From robin at reportlab.com  Mon Jun 22 06:33:47 2015
From: robin at reportlab.com (Robin Becker)
Date: Mon, 22 Jun 2015 11:33:47 +0100
Subject: Opening PDF Using subprocess.Popen Failing
In-Reply-To: <7710dd71-6f37-4d1b-b868-bc233d4da3b7@googlegroups.com>
References: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
 <7710dd71-6f37-4d1b-b868-bc233d4da3b7@googlegroups.com>
Message-ID: <5587E48B.2040700@chamonix.reportlab.co.uk>

.....

Naftali,

I ran the following from python prompt


import os
os.system('"C:\Program Files (x86)\Adobe\Reader 11.0\Reader\AcroRd32.exe" /n 
thello.pdf')


and this worked as did

import subprocess
subprocess.Popen(['C:\Program Files (x86)\Adobe\Reader 
11.0\Reader\AcroRd32.exe','/n','thello.pdf'])

however I also tried running

import subprocess
subprocess.Popen('AcroRd32.exe /nthello.pdf')

which seems closer to your

> new_command_str = "AcroRd32.exe /n" + outputname
> subprocess.Popen(new_command_str)

and that fails; the python error is


> C:\Users\rptlab\tmp>tpdf.py
> Traceback (most recent call last):
>   File "C:\Users\rptlab\tmp\tpdf.py", line 6, in <module>
>     subprocess.Popen('AcroRd32.exe /nthello.pdf')
>   File "c:\python27\lib\subprocess.py", line 710, in __init__
>     errread, errwrite)
>   File "c:\python27\lib\subprocess.py", line 958, in _execute_child
>     startupinfo)
> WindowsError: [Error 2] The system cannot find the file specified



-- 
Robin Becker



From robin at reportlab.com  Mon Jun 22 06:33:47 2015
From: robin at reportlab.com (Robin Becker)
Date: Mon, 22 Jun 2015 11:33:47 +0100
Subject: Opening PDF Using subprocess.Popen Failing
In-Reply-To: <7710dd71-6f37-4d1b-b868-bc233d4da3b7@googlegroups.com>
References: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
 <7710dd71-6f37-4d1b-b868-bc233d4da3b7@googlegroups.com>
Message-ID: <5587E48B.2040700@chamonix.reportlab.co.uk>

.....

Naftali,

I ran the following from python prompt


import os
os.system('"C:\Program Files (x86)\Adobe\Reader 11.0\Reader\AcroRd32.exe" /n 
thello.pdf')


and this worked as did

import subprocess
subprocess.Popen(['C:\Program Files (x86)\Adobe\Reader 
11.0\Reader\AcroRd32.exe','/n','thello.pdf'])

however I also tried running

import subprocess
subprocess.Popen('AcroRd32.exe /nthello.pdf')

which seems closer to your

> new_command_str = "AcroRd32.exe /n" + outputname
> subprocess.Popen(new_command_str)

and that fails; the python error is


> C:\Users\rptlab\tmp>tpdf.py
> Traceback (most recent call last):
>   File "C:\Users\rptlab\tmp\tpdf.py", line 6, in <module>
>     subprocess.Popen('AcroRd32.exe /nthello.pdf')
>   File "c:\python27\lib\subprocess.py", line 710, in __init__
>     errread, errwrite)
>   File "c:\python27\lib\subprocess.py", line 958, in _execute_child
>     startupinfo)
> WindowsError: [Error 2] The system cannot find the file specified



-- 
Robin Becker


From robin at reportlab.com  Mon Jun 22 06:55:04 2015
From: robin at reportlab.com (Robin Becker)
Date: Mon, 22 Jun 2015 11:55:04 +0100
Subject: Opening PDF Using subprocess.Popen Failing
In-Reply-To: <5587E48B.2040700@chamonix.reportlab.co.uk>
References: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
 <7710dd71-6f37-4d1b-b868-bc233d4da3b7@googlegroups.com>
 <5587E48B.2040700@chamonix.reportlab.co.uk>
Message-ID: <5587E988.3040906@chamonix.reportlab.co.uk>

On 22/06/2015 11:33, Robin Becker wrote:
> .....
>
> Naftali,
>
> I ran the following from python prompt
>
........
>
>


for what it's worth this also works on my machine

> Windows PowerShell
> Copyright (C) 2009 Microsoft Corporation. All rights reserved.
>
> PS C:\Users\rptlab> cd tmp
> PS C:\Users\rptlab\tmp> python
> Python 2.7.8 (default, Jun 30 2014, 16:08:48) [MSC v.1500 64 bit (AMD64)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import subprocess
>>>> subprocess.Popen(['C:\Program Files (x86)\Adobe\Reader 11.0\Reader\AcroRd32.exe','/n','thello.pdf'])
> <subprocess.Popen object at 0x0000000001D853C8>
>>>>

not sure if you are using 64bits or 32bit windows, but perhaps you just need to 
specify the full path to your AcroRd32.exe.

-- 
Robin Becker



From robin at reportlab.com  Mon Jun 22 06:55:04 2015
From: robin at reportlab.com (Robin Becker)
Date: Mon, 22 Jun 2015 11:55:04 +0100
Subject: Opening PDF Using subprocess.Popen Failing
In-Reply-To: <5587E48B.2040700@chamonix.reportlab.co.uk>
References: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
 <7710dd71-6f37-4d1b-b868-bc233d4da3b7@googlegroups.com>
 <5587E48B.2040700@chamonix.reportlab.co.uk>
Message-ID: <5587E988.3040906@chamonix.reportlab.co.uk>

On 22/06/2015 11:33, Robin Becker wrote:
> .....
>
> Naftali,
>
> I ran the following from python prompt
>
........
>
>


for what it's worth this also works on my machine

> Windows PowerShell
> Copyright (C) 2009 Microsoft Corporation. All rights reserved.
>
> PS C:\Users\rptlab> cd tmp
> PS C:\Users\rptlab\tmp> python
> Python 2.7.8 (default, Jun 30 2014, 16:08:48) [MSC v.1500 64 bit (AMD64)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import subprocess
>>>> subprocess.Popen(['C:\Program Files (x86)\Adobe\Reader 11.0\Reader\AcroRd32.exe','/n','thello.pdf'])
> <subprocess.Popen object at 0x0000000001D853C8>
>>>>

not sure if you are using 64bits or 32bit windows, but perhaps you just need to 
specify the full path to your AcroRd32.exe.

-- 
Robin Becker


From python.list at tim.thechases.com  Mon Jun 22 07:51:46 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Mon, 22 Jun 2015 06:51:46 -0500
Subject: Issuing a sqilte query, picking a random result, and copying to
 the system clipboard
In-Reply-To: <CAJGLaBJvbJzYLTVoP0Vh4wWiCqzruUvPspo1QSA7g=V=JfSWTQ@mail.gmail.com>
References: <CAJGLaBJvbJzYLTVoP0Vh4wWiCqzruUvPspo1QSA7g=V=JfSWTQ@mail.gmail.com>
Message-ID: <20150622065146.7654f597@bigbox.christie.dr>

On 2015-06-21 17:08, John T. Haggerty wrote:
> I'm looking to just have a simple program that will do a SQLite
> query pull a random record and then copy that record too the
> clipboard the system. I'm not quite seeing how to do this perhaps
> this is already been done elsewhere but I spent quite a bit of time
> trying to figure out how to do that and I'm not seeing a listing
> anywhere. any help would be greatly appreciated.

First, connect to your data source and obtain a cursor:

  import sqlite3
  conn = sqlite3.connect('file.sqlite')
  cur = conn.cursor()

then select a random row:

  cur.execute("SELECT * FROM table ORDER BY RANDOM() LIMIT 1")
  data = cur.fetchone()

then things get tricky because you don't specify what OS you have, as
there are plenty of occasions when there is no system clipboard.

On Win32, you'd need the Win32 add-on libraries to shove things onto
the clipboard, while under X, you'd need other facilities (either
using Tkinter or piping to something like xclip(1)), and yet another
way of doing things on MacOS.

-tkc






From robin at reportlab.com  Mon Jun 22 08:03:22 2015
From: robin at reportlab.com (Robin Becker)
Date: Mon, 22 Jun 2015 13:03:22 +0100
Subject: rl_config
Message-ID: <5587F98A.6010509@chamonix.reportlab.co.uk>

Anyone wishing to bend their minds around instance as module can see the code 
I've tested on //annapurna/tmp/rl_config.py.


-- 
Robin Becker



From robin at reportlab.com  Mon Jun 22 08:10:25 2015
From: robin at reportlab.com (Robin Becker)
Date: Mon, 22 Jun 2015 13:10:25 +0100
Subject: rl_config
In-Reply-To: <5587F98A.6010509@chamonix.reportlab.co.uk>
References: <5587F98A.6010509@chamonix.reportlab.co.uk>
Message-ID: <5587FB31.8020005@chamonix.reportlab.co.uk>

On 22/06/2015 13:03, Robin Becker wrote:
> Anyone wishing to bend their minds around instance as module can see the code
> I've tested on //annapurna/tmp/rl_config.py.
>
>
whoops misposted sorry

-- 
Robin Becker



From robin at reportlab.com  Mon Jun 22 08:10:25 2015
From: robin at reportlab.com (Robin Becker)
Date: Mon, 22 Jun 2015 13:10:25 +0100
Subject: rl_config
In-Reply-To: <5587F98A.6010509@chamonix.reportlab.co.uk>
References: <5587F98A.6010509@chamonix.reportlab.co.uk>
Message-ID: <5587FB31.8020005@chamonix.reportlab.co.uk>

On 22/06/2015 13:03, Robin Becker wrote:
> Anyone wishing to bend their minds around instance as module can see the code
> I've tested on //annapurna/tmp/rl_config.py.
>
>
whoops misposted sorry

-- 
Robin Becker


From nmichalowsky at gmail.com  Mon Jun 22 09:15:51 2015
From: nmichalowsky at gmail.com (Naftali)
Date: Mon, 22 Jun 2015 06:15:51 -0700 (PDT)
Subject: Opening PDF Using subprocess.Popen Failing
In-Reply-To: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
References: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
Message-ID: <5358b9aa-8d0a-44bf-a5bc-782c05588845@googlegroups.com>

On Friday, June 19, 2015 at 1:25:12 PM UTC-4, Naftali wrote:
> It actually doesn't fail but it 'cannot open in protected mode' (see here http://blogs.adobe.com/dmcmahon/2012/07/27/adobe-reader-cannot-open-protected-mode-due-to-a-problem-with-your-system-configuration/)
> 
> I am using subprocess.Popen("AcroRe32.exe /n <file.pdf>") which is the actuall adobe reader command I'd issue on the command line to open the pdf (the /n option opens it the file in a new instance of reader).
> 
> Now, when I issue the command straight from powershell, the pdf opens no problem, but when I open in my script (whether a .py or py2exe) I get the pop up complaining that the PDF cannot be opened in 'protected mode.' One of the options is to open it anyways, which works. 
> 
> Looking into it (see the link in the first paragraph) my best guess is that it's due to something like "JS-invoked processes: Launching a process through JavaScript is not allowed with Protected Mode enabled." 
> 
> But my naive understanding was that when I give Popen instruction, the command is handed off to windows and the called program is unaware of how it got called, so my thinking is that either that is incorrect or windows somehow 'cooperates' with reader to figure things out. 
> 
> I am looking for *any* insight as to how to deal with this, and the 'turn off protected mode" option wont work for me. 
> 
> Here is my code,
> 
> outputname = " unlocked.pdf"
> 
> commandstr = "qpdf --decrypt " + sys.argv[1] + outputname
> os.system(commandstr)
> 
> new_command_str = "AcroRd32.exe /n" + outputname
> subprocess.Popen(new_command_str)
> 
> sys.exit(0)

Thank you, Robin, just got in, happy Monday. I don't think it's the path because I added the acrobat exe to my path env variable which is why it didn't crash and burn. I tried running it via the standard prompt (as opposed to powershell) and ran into the same issue but the prompt gave a bit more information the power shell, in that it said 'open unlocked.pdf: Permission denied. based on your success I'm thinking that it has to do with the qpdf creates the file. I'm going to play around with that and will update. thank you so much for your help.


From lac at openend.se  Mon Jun 22 09:37:33 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 22 Jun 2015 15:37:33 +0200
Subject: Opening PDF Using subprocess.Popen Failing
In-Reply-To: Message from Naftali <nmichalowsky@gmail.com> of "Mon,
 22 Jun 2015 06:15:51 -0700."
 <5358b9aa-8d0a-44bf-a5bc-782c05588845@googlegroups.com>
References: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com><5358b9aa-8d0a-44bf-a5bc-782c05588845@googlegroups.com>
Message-ID: <201506221337.t5MDbXHZ006583@fido.openend.se>

I think that your problem is that you have Protected Mode enabled.
If you do, you either have to disable that, or write a policy config
file.

https://www.adobe.com/devnet-docs/acrobatetk/tools/AppSec/protectedmode.html

says.
From: Policy configuration

Protected mode prevents a number of actions which IT can bypass by
creating a white list of allowed actions. The component that reads
these policies is called a ?broker.? The broker performs actions based
on those policies, and when an admin provides a properly configured
policy file, the broker can bypass the application?s default
restrictions.

The broker first reads and applies all custom policies prior to
applying the default policies. Since custom policies take precedence,
they are useful for fixing broken workflows, supporting third party
plug-ins, and cases where unsupported machine configurations cause the
Protected Mode to impair required functionality.

Configurable policies have two requirements:

    They must reside in the Reader install directory adjacent to
        AcroRd32.exe in the install folder. for example: D:\Program
        Files (x86)\Adobe\Reader 10.0\Reader\ The name of the policy
        file must be ProtectedModeWhitelistConfig.txt.

So, well, I don't have a windows machine.  I cannot test this any more
for you, alas.

Laura


From nmichalowsky at gmail.com  Mon Jun 22 10:23:40 2015
From: nmichalowsky at gmail.com (Naftali)
Date: Mon, 22 Jun 2015 07:23:40 -0700 (PDT)
Subject: Opening PDF Using subprocess.Popen Failing
In-Reply-To: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
References: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
Message-ID: <63ba19bd-e13e-4e5e-9d1d-2fe6392cb12e@googlegroups.com>

On Friday, June 19, 2015 at 1:25:12 PM UTC-4, Naftali wrote:
> It actually doesn't fail but it 'cannot open in protected mode' (see here http://blogs.adobe.com/dmcmahon/2012/07/27/adobe-reader-cannot-open-protected-mode-due-to-a-problem-with-your-system-configuration/)
> 
> I am using subprocess.Popen("AcroRe32.exe /n <file.pdf>") which is the actuall adobe reader command I'd issue on the command line to open the pdf (the /n option opens it the file in a new instance of reader).
> 
> Now, when I issue the command straight from powershell, the pdf opens no problem, but when I open in my script (whether a .py or py2exe) I get the pop up complaining that the PDF cannot be opened in 'protected mode.' One of the options is to open it anyways, which works. 
> 
> Looking into it (see the link in the first paragraph) my best guess is that it's due to something like "JS-invoked processes: Launching a process through JavaScript is not allowed with Protected Mode enabled." 
> 
> But my naive understanding was that when I give Popen instruction, the command is handed off to windows and the called program is unaware of how it got called, so my thinking is that either that is incorrect or windows somehow 'cooperates' with reader to figure things out. 
> 
> I am looking for *any* insight as to how to deal with this, and the 'turn off protected mode" option wont work for me. 
> 
> Here is my code,
> 
> outputname = " unlocked.pdf"
> 
> commandstr = "qpdf --decrypt " + sys.argv[1] + outputname
> os.system(commandstr)
> 
> new_command_str = "AcroRd32.exe /n" + outputname
> subprocess.Popen(new_command_str)
> 
> sys.exit(0)

Sadly so far it looks like the only answer is to disable protected mode, either generally or on a my app basis, which is the same thing because my app will be registered as the default handler for pdf. So this is a deal breaker. I've even tried routing the opening through a bat file... so either I get this to open up in protected mode or I don't see anyway to make it go, with out using a different reader. But thank you all for your help, will update should I have any success in this.


From rustompmody at gmail.com  Mon Jun 22 10:41:21 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 22 Jun 2015 07:41:21 -0700 (PDT)
Subject: CRLF  problems in CPython sources
Message-ID: <af9e993f-4f34-4fd6-9bf7-92f540ad18a3@googlegroups.com>

Trying to setup CPython repo I found some CRLF messes:

1. Some file-types that are inconsistently LF or CRLF
   Lib/test/decimaltestdata

2. Some files that are plain dirty (both LF and CRLF)
   Lib/venv/scripts/nt/Activate.ps1

Is this worth a bug report?
Is python-dev the place for talking of such?

Note 1: Been bitten rather badly by this in svn days and would rather waste time 
upfront rather than later.
Note 2: Not trying to nitpick. git barfed about this
Looking around I found I can get mercurial to likewise complain


From zachary.ware+pylist at gmail.com  Mon Jun 22 10:52:22 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Mon, 22 Jun 2015 09:52:22 -0500
Subject: CRLF problems in CPython sources
In-Reply-To: <af9e993f-4f34-4fd6-9bf7-92f540ad18a3@googlegroups.com>
References: <af9e993f-4f34-4fd6-9bf7-92f540ad18a3@googlegroups.com>
Message-ID: <CAKJDb-NVDJ43PGa0p3bRvP2_8s2x5Mz7yUbf96sfj=+FZpY4vA@mail.gmail.com>

On Monday, June 22, 2015, Rustom Mody <rustompmody at gmail.com> wrote:

> Trying to setup CPython repo I found some CRLF messes:
>
> 1. Some file-types that are inconsistently LF or CRLF
>    Lib/test/decimaltestdata
>
> 2. Some files that are plain dirty (both LF and CRLF)
>    Lib/venv/scripts/nt/Activate.ps1
>
> Is this worth a bug report?
> Is python-dev the place for talking of such?
>
> Note 1: Been bitten rather badly by this in svn days and would rather
> waste time
> upfront rather than later.
> Note 2: Not trying to nitpick. git barfed about this
> Looking around I found I can get mercurial to likewise complain
>

For Mercurial, enable the eol extension and then refresh your checkout (hg
update null && hg update <rev you want>).

Some of the files in the checkout need to be CRLF for Windows' benefit, the
ones in Lib/test/decimaltestdata are as they were when pulled from their
source.  For others, a bug report would be in order. If you submit such a
bug, please make me nosy (zach.ware).

--
Zach


-- 
Sent from Gmail Mobile
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150622/339a7581/attachment.html>

From rosuav at gmail.com  Mon Jun 22 10:54:27 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 23 Jun 2015 00:54:27 +1000
Subject: Arguments for "type metaclass" __init__ method
In-Reply-To: <CAO36yJ6X5aaLbi7T-avji4rFoqyRY6nsf3nxH7LAU+sXAwk5PQ@mail.gmail.com>
References: <CAO36yJ6X5aaLbi7T-avji4rFoqyRY6nsf3nxH7LAU+sXAwk5PQ@mail.gmail.com>
Message-ID: <CAPTjJmo_Afr97b7zDten8RHKwfsG5zEz8MGjmgkyaAoC_YPppA@mail.gmail.com>

On Mon, Jun 22, 2015 at 1:17 AM, Kaviraj Kanagaraj
<kaviraj at launchyard.com> wrote:
> To dynamically create a class:
> DynamicClass = type("DynamicClass", (object,), {'eggs' : 'spams'})
>
> which means type's __init__ method accepts ClassName, bases(tuple) and
> attrbs(dict) as args.
>
>
> But in case of creating class via class definition("type" act as default
> metaclass), the type is called by following syntax,
>
> type(cls, classname, bases, attrbs)
>
> My doubt is how it possible to pass "cls" as argument(as class is not even
> created before calling meta class). Also how "type" is accepting different
> args in two different cases?

If you're looking at the parameters to __init__, it's normal for the
first parameter to be called 'self', and to be the newly-created
object. I suspect that might be what you're seeing here; when you
explicitly call the type, it goes through standard object creation
steps, calling __new__ and (usually) then __init__, resulting in a
call that has the additional argument on it.

So 'cls' is the class that's busily being defined.

ChrisA


From nmichalowsky at gmail.com  Mon Jun 22 12:13:16 2015
From: nmichalowsky at gmail.com (Naftali)
Date: Mon, 22 Jun 2015 09:13:16 -0700 (PDT)
Subject: Opening PDF Using subprocess.Popen Failing
In-Reply-To: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
References: <4857b9a4-386f-4ae4-861b-09a9564557c7@googlegroups.com>
Message-ID: <f97383ba-34fa-4226-a6ea-425c04b3bad5@googlegroups.com>

On Friday, June 19, 2015 at 1:25:12 PM UTC-4, Naftali wrote:
> It actually doesn't fail but it 'cannot open in protected mode' (see here http://blogs.adobe.com/dmcmahon/2012/07/27/adobe-reader-cannot-open-protected-mode-due-to-a-problem-with-your-system-configuration/)
> 
> I am using subprocess.Popen("AcroRe32.exe /n <file.pdf>") which is the actuall adobe reader command I'd issue on the command line to open the pdf (the /n option opens it the file in a new instance of reader).
> 
> Now, when I issue the command straight from powershell, the pdf opens no problem, but when I open in my script (whether a .py or py2exe) I get the pop up complaining that the PDF cannot be opened in 'protected mode.' One of the options is to open it anyways, which works. 
> 
> Looking into it (see the link in the first paragraph) my best guess is that it's due to something like "JS-invoked processes: Launching a process through JavaScript is not allowed with Protected Mode enabled." 
> 
> But my naive understanding was that when I give Popen instruction, the command is handed off to windows and the called program is unaware of how it got called, so my thinking is that either that is incorrect or windows somehow 'cooperates' with reader to figure things out. 
> 
> I am looking for *any* insight as to how to deal with this, and the 'turn off protected mode" option wont work for me. 
> 
> Here is my code,
> 
> outputname = " unlocked.pdf"
> 
> commandstr = "qpdf --decrypt " + sys.argv[1] + outputname
> os.system(commandstr)
> 
> new_command_str = "AcroRd32.exe /n" + outputname
> subprocess.Popen(new_command_str)
> 
> sys.exit(0)

Well a happy if not completely satisfying ending. So alternative readers worked ok, but IT was ok with turning off protective mode and also disabling javascript in adobe reader, considering the relative benign provenance of the pdf's coming into the system (though I am curious whether or not disabling java script is actually *more* secure than a sandbox, i.e. can a system be harmed by rendering a pdf with no java script?


From jvadams at usgs.gov  Mon Jun 22 12:54:05 2015
From: jvadams at usgs.gov (Adams, Jean)
Date: Mon, 22 Jun 2015 11:54:05 -0500
Subject: ASA Conference on Statistical Practice - deadline Thursday
Message-ID: <CAN5YmCHX_2yoHiPmQL3JZp=iJvFrvQUUZwNQt=3oF+L-2MGyzg@mail.gmail.com>

Python users,

Abstracts are now being accepted for the
     2016 ASA Conference on Statistical Practice,
     February 18-20,
     San Diego, CA, USA.

Conference attendees are not typically familiar with Python.  It would be
great to have someone from the Python community give a brief overview of
how Python might be useful to an applied statistician.

The deadline for submission is June 25.  Presentations will be 35 minutes
long and fall into four broad themes:
     Communication, Impact, and Career Development
     Data Modeling and Analysis
     Big Data Prediction and Analytics
     Software, Programming, and Graphics

Abstracts may be submitted at
http://www.amstat.org/meetings/csp/2016/abstracts.cfm

Thank you.

Jean V. Adams
on behalf of the ASA-CSP 2016 Steering Committee



`?.,,  ><(((?>   `?.,,  ><(((?>   `?.,,  ><(((?>

Jean V. Adams
Statistician
U.S. Geological Survey
Great Lakes Science Center
223 East Steinfest Road
Antigo, WI 54409  USA
http://www.glsc.usgs.gov
http://profile.usgs.gov/jvadams
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150622/90652522/attachment.html>

From rustompmody at gmail.com  Mon Jun 22 13:36:25 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 22 Jun 2015 10:36:25 -0700 (PDT)
Subject: CRLF problems in CPython sources
In-Reply-To: <mailman.708.1434984745.13271.python-list@python.org>
References: <af9e993f-4f34-4fd6-9bf7-92f540ad18a3@googlegroups.com>
 <mailman.708.1434984745.13271.python-list@python.org>
Message-ID: <9274f031-7303-401a-b886-41d52654616b@googlegroups.com>

On Monday, June 22, 2015 at 2:53:21 PM UTC, Zachary Ware wrote:
> On Monday, June 22, 2015, Rustom Mody  wrote:
> Trying to setup CPython repo I found some CRLF messes:
> 
> 
> 
> 1. Some file-types that are inconsistently LF or CRLF
> 
> ? ?Lib/test/decimaltestdata
> 
> 
> 
> 2. Some files that are plain dirty (both LF and CRLF)
> 
> ? ?Lib/venv/scripts/nt/Activate.ps1
> 
> 
> 
> Is this worth a bug report?
> 
> Is python-dev the place for talking of such?
> 
> 
> 
> Note 1: Been bitten rather badly by this in svn days and would rather waste time
> 
> upfront rather than later.
> 
> Note 2: Not trying to nitpick. git barfed about this
> 
> Looking around I found I can get mercurial to likewise complain
> 
> 
> 
> 
> For Mercurial, enable the eol extension and then refresh your checkout (hg update null && hg update <rev you want>).

Yeah It was a bit of a struggle to get that up.
[hg docs quite poor in this area]

> 
> 
> Some of the files in the checkout need to be CRLF for Windows' benefit, the ones in Lib/test/decimaltestdata are as they were when pulled from their source.? For others, a bug report would be in order. If you submit such a bug, please make me nosy (zach.ware).

Thanks Zachary -- will do.
[On an half-installed machine right now... Repos elsewhere]


From battibaha at gmail.com  Mon Jun 22 14:30:05 2015
From: battibaha at gmail.com (baha)
Date: Mon, 22 Jun 2015 11:30:05 -0700 (PDT)
Subject: what should every python expert know
Message-ID: <05443fd3-494d-4fe9-8353-c8cdfdc3a193@googlegroups.com>

I don't like to move to another programming language until being guru .. (python is my first programming language) what are things that I must know and discover and understand more deeply 
what are the projects that can be useful for a beginner like me and the advanced tricks 


From steve at pearwood.info  Mon Jun 22 14:59:54 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 23 Jun 2015 04:59:54 +1000
Subject: what should every python expert know
References: <05443fd3-494d-4fe9-8353-c8cdfdc3a193@googlegroups.com>
Message-ID: <55885b29$0$1655$c3e8da3$5496439d@news.astraweb.com>

On Tue, 23 Jun 2015 04:30 am, baha wrote:

> I don't like to move to another programming language until being guru ..
> (python is my first programming language) what are things that I must know
> and discover and understand more deeply what are the projects that can be
> useful for a beginner like me and the advanced tricks

What should every Python expert know? That depends what they want to do.

Your question is a bit like "what should every artist know" -- some artists
specialise, others generalise. Same goes for engineers, doctors, scientists
(actually, these days, pretty much all scientists specialise).


What could you learn?

- web frameworks like Django, CherryPy and others, HTML, Javascript;
- databases and SQL;
- floating point maths and numeric algorithms, including numpy, scipy, 
  sage and others;
- writing extensions in C, Fortran, Java, or other languages;
- or using numba, cython, hope or others;
- integration with other tools using Applescript, Swift, bash scripting 
  or Windows Powershell;
- Unicode and legacy encodings;
- natural language processing with nltk;
- text processing with regular expressions;
- text parsers;
- source control using hg, git or others;
- graphics using PIL or pillow;
- GUI applications with tkinter, PyQt or others;
- decorators, metaclasses, comprehensions, closures;
- mixins, traits, multiple inheritance;
- doc tests, unit tests, regression tests, smoke tests, integration tests,
  fuzz tests, and others;
- writing documentation;
- project planning;
- and many other things.


It is difficult for us to teach you "the advanced tricks" when we don't know
what you already know and what you don't know. For example, maybe you will
consider this an advanced trick:


py> a = 1
py> b = 2
py> print a, b
1 2
py> a, b = b, a
py> print a, b
2 1


and maybe you won't.



-- 
Steven



From ian.g.kelly at gmail.com  Mon Jun 22 15:16:36 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 22 Jun 2015 13:16:36 -0600
Subject: Pyitect - Plugin architectural system for Python 3.0+
 (feedback?)
In-Reply-To: <CAO-rxQuEZ3e2MiaTh4dpWdcbQFcETrFEOH9oKiyh5AmQW=+Agw@mail.gmail.com>
References: <CAO-rxQuEZ3e2MiaTh4dpWdcbQFcETrFEOH9oKiyh5AmQW=+Agw@mail.gmail.com>
Message-ID: <CALwzidkQbzjQDPsmWa1t2e8io25Xw0FKxzPRoSByPNWj3Ftujg@mail.gmail.com>

On Mon, Jun 22, 2015 at 2:32 AM, Ben Powers <ryexander at gmail.com> wrote:
> on Tue, Jun 16,  2015 at 17:49 Ian Kelly <ian.g.kelly at gmail.com> wrote
>
>>On Mon, Jun 8, 2015 at 10:42 PM, Ben Powers <ryexander at gmail.com> wrote:
>>> #file.py
>>> from PyitectConsumes import foo
>>>
>>> class Bar(object):
>>>     def __init__():
>>>         foo("it's a good day to be a plugin")>
>
>>I don't understand this example. Is "PyitectConsumes" the name of a
>>component or part of Pyitect? Or is "foo" the name of the component?>
>
>
> `PyitectComsumes` is a special module injected into sys.modules when the
> plugin module is imported and removed after the import is complete. it's
> name-space contains all the components declared as required in the plugin's
> .json

So this import can only be performed when the module is loaded, not
later when a component's functionality is being invoked?

>>> Plugins are loaded on demand when a component is loaded via
>>>
>>> System.load("<component name>")>
>
>>What's the difference between this and the "PyitectConsumes" import?>
>
>
> The PyitectConsumes import is for use inside plugins where dependencies are
> declared prior to import.
>
> System.load is for use in the application to load components. Plugins are
> not intended to know about the System they are being used in.
>
>
>>> Loaded pluginss do NOT store their module object in sys.modules>
>
>>What about imports of plugin-local modules that are performed inside
>>the plugins? Do those also get cleaned up from sys.modules?>
>
>
> Actually no... thats an oversight on my part. really I should temporally
> clone sys.modules inject the PyitectConsumes special module and then set it
> back to the original sys.modules dict after the import has been performed,
> that would secure the process more and prevent cluttering of sys.modules.

It seems to me that this could create new problems. For example,
suppose a plugin being loaded imports urllib, and then later the
application tries to import urllib. It's not in sys.modules, so the
import framework loads it again, and now there are two copies of
urllib floating around in the interpreter. I think you actually need
to diff sys.modules and then remove anything that's added but only if
its path falls within the path of the plugin directory.

The more that I think about this, I don't think that overloading the
import machinery like this is the right way for plugins to gain access
to components. If instead you just pass the module a context object
with a get_component method, then you won't have to muck around with
sys.modules as much, and the context object can remain available to
the plugin for later use.


From drsalists at gmail.com  Mon Jun 22 15:39:09 2015
From: drsalists at gmail.com (Dan Stromberg)
Date: Mon, 22 Jun 2015 12:39:09 -0700
Subject: How to check in script if Python or Jython is used
In-Reply-To: <87r3p5pjjd.fsf@Equus.decebal.nl>
References: <87r3p5pjjd.fsf@Equus.decebal.nl>
Message-ID: <CAGGBd_rCYWQr-yg3C+OwdOc5fJC+g4i2cZOhZOW0_3xNck1QXg@mail.gmail.com>

IMO, it's usually better to test for features and use them if they are
present, than to build a list of features available in specific
interpreters.

I see it as analogous to the difference between huge C #ifdef's on OS,
and autoconf.

On Sun, Jun 21, 2015 at 1:12 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> I installed Jython and will start playing with it. There probably will
> be differences between Python and Jython. Is there a way to determine
> if a script is run by Python or Jython? Then different execution paths
> could be taken. With sys.version(_info) you do not get this
> information.
>
> --
> Cecil Westerhof
> Senior Software Engineer
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
> --
> https://mail.python.org/mailman/listinfo/python-list


From dchokshi at cisco.com  Mon Jun 22 16:42:21 2015
From: dchokshi at cisco.com (Devaki Chokshi (dchokshi))
Date: Mon, 22 Jun 2015 20:42:21 +0000
Subject: MGCP Server implementation
Message-ID: <E0FE14A926805D41A32BEA79FD6E7AD626467438@xmb-rcd-x10.cisco.com>

Hello,

I have a use case where a SIP voice call will be passing through an MGCP gateway.

Is there a python implementation that simulates MGCP gateway/server?

Thank you
Devaki Chokshi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150622/ddf54837/attachment.html>

From Cecil at decebal.nl  Mon Jun 22 17:38:50 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 22 Jun 2015 23:38:50 +0200
Subject: Published timeDecebal.py
Message-ID: <874mlz779x.fsf@Equus.decebal.nl>

I published timeDecebal.py:
    https://github.com/CecilWesterhof/PythonLibrary/blob/master/timeDecebal.py

Not much yet: the Timer class from Steven D'Aprano and the function
time_test. With this function you can get the results of a function
AND the time it took to execute the function. (I find that useful.)

Requests for extra functionality are welcome. :-D 
No guarantees for implementation o-course. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From piet at vanoostrum.org  Mon Jun 22 17:49:31 2015
From: piet at vanoostrum.org (Piet van Oostrum)
Date: Mon, 22 Jun 2015 23:49:31 +0200
Subject: Calling a function is faster than not calling it?
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <lxoak74dn8.fsf@cochabamba.vanoostrum.org>

Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:

> Why is calling a function faster than bypassing the function object and
> evaluating the code object itself? And not by a little, but by a lot?
>
> Here I have a file, eval_test.py:
>
> # === cut ===
> from timeit import Timer
>
> def func():
>     a = 2
>     b = 3
>     c = 4
>     return (a+b)*(a-b)/(a*c + b*c)
>
>
> code = func.__code__
> assert func() == eval(code)
>
> t1 = Timer("eval; func()", setup="from __main__ import func")
> t2 = Timer("eval(code)", setup="from __main__ import code")
>
> # Best of 10 trials.
> print (min(t1.repeat(repeat=10)))
> print (min(t2.repeat(repeat=10)))
>
> # === cut ===
>
>
> Note that both tests include a name lookup for eval, so that as much as
> possible I am comparing the two pieces of code on an equal footing.

They are not on equal footing. The first one only looks up eval, but the second actually calls eval. The overhead of calling eval is what makes the difference. To get them on equal footing you will have to insert an eval call also in the first example.

dummy = compile("0", 'string', "eval")

t1 = Timer("eval(dummy); func()", setup="from __main__ import dummy, func")
t2 = Timer("0; eval(code)", setup="from __main__ import code")

And then you'll see that t1 is slightly slower than t2.
-- 
Piet van Oostrum <piet at vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]


From travisgriggs at gmail.com  Mon Jun 22 18:31:58 2015
From: travisgriggs at gmail.com (Travis Griggs)
Date: Mon, 22 Jun 2015 15:31:58 -0700
Subject: Simplest/Idiomatic way to count files in a directory (using pathlib)
Message-ID: <8B8C6022-75BE-41FC-9B18-7D89AD34A35A@gmail.com>

Subject nearly says it all.

If i?m using pathlib, what?s the simplest/idiomatic way to simply count how many files are in a given directory?

I was surprised (at first) when

    len(self.path.iterdir())

I don?t say anything on the in the .stat() object that helps me.

I could of course do the 4 liner:

    count = 0
    for _ in self.path.iterdir():
        count += 1
    return count

The following seems to obtuse/clever for its own good:

    return sum(1 for _ in self.path.iterdir())

From travisgriggs at gmail.com  Mon Jun 22 18:33:18 2015
From: travisgriggs at gmail.com (Travis Griggs)
Date: Mon, 22 Jun 2015 15:33:18 -0700
Subject: Simplest/Idiomatic way to count files in a directory (using
 pathlib)
In-Reply-To: <8B8C6022-75BE-41FC-9B18-7D89AD34A35A@gmail.com>
References: <8B8C6022-75BE-41FC-9B18-7D89AD34A35A@gmail.com>
Message-ID: <E2B820B1-2238-4914-BD20-02AAF60F3243@gmail.com>

<I should proof my posts before I send them, sorry>

Subject nearly says it all.

If i?m using pathlib, what?s the simplest/idiomatic way to simply count how many files are in a given directory?

I was surprised (at first) when

   len(self.path.iterdir())

didn?t work.

I don?t see anything in the .stat() object that helps me.

I could of course do the 4 liner:

   count = 0
   for _ in self.path.iterdir():
       count += 1
   return count

The following seems to obtuse/clever for its own good:

   return sum(1 for _ in self.path.iterdir())

From ian.g.kelly at gmail.com  Mon Jun 22 18:48:23 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 22 Jun 2015 16:48:23 -0600
Subject: Simplest/Idiomatic way to count files in a directory (using
 pathlib)
In-Reply-To: <E2B820B1-2238-4914-BD20-02AAF60F3243@gmail.com>
References: <8B8C6022-75BE-41FC-9B18-7D89AD34A35A@gmail.com>
 <E2B820B1-2238-4914-BD20-02AAF60F3243@gmail.com>
Message-ID: <CALwzidm8OF8VxGEmmunqyVu2=ue3wOB381k5ow=FC7eiUfu03A@mail.gmail.com>

On Mon, Jun 22, 2015 at 4:33 PM, Travis Griggs <travisgriggs at gmail.com> wrote:
> <I should proof my posts before I send them, sorry>
>
> Subject nearly says it all.
>
> If i?m using pathlib, what?s the simplest/idiomatic way to simply count how many files are in a given directory?
>
> I was surprised (at first) when
>
>    len(self.path.iterdir())
>
> didn?t work.

len doesn't work on iterators for a number of reasons:

* It would exhaust the iterator, preventing further use.
* The iterator is not necessarily finite.
* Even if it's finite, the length is not necessarily deterministic.
Consider this simple generator:

def gen():
    while random.randrange(2):
        yield 42

> I don?t see anything in the .stat() object that helps me.
>
> I could of course do the 4 liner:
>
>    count = 0
>    for _ in self.path.iterdir():
>        count += 1
>    return count
>
> The following seems to obtuse/clever for its own good:
>
>    return sum(1 for _ in self.path.iterdir())

This is the usual idiom for counting the number of items yielded from
an iterator. Alternatively you can use len(list(self.path.iterdir()))
if you don't mind constructing a list of the entire directory listing.


From rosuav at gmail.com  Mon Jun 22 20:43:33 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 23 Jun 2015 10:43:33 +1000
Subject: JSON Object to CSV File Troubleshooting
In-Reply-To: <mm7hjt$n8l$4@dont-email.me>
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
 <mm5n0u$5m8$5@dont-email.me>
 <44a2d55f-9312-4b70-8b23-637466b21b11@googlegroups.com>
 <mailman.685.1434923721.13271.python-list@python.org>
 <mm7hjt$n8l$4@dont-email.me>
Message-ID: <CAPTjJmr3WRkidL2OG8-R16m8k+u7V2SCThafxr787_KOosGKsA@mail.gmail.com>

On Mon, Jun 22, 2015 at 9:33 AM, Denis McMahon <denismfmcmahon at gmail.com> wrote:
> This is the sort of data conversion code I generally turn out in a day or
> so, it's hardly rocket science as long as you have a clear description of
> what is required. If you don't have a clear description of what is
> required, you have to keep asking questions until you get one.

And this all before you even write a line of code. Getting the specs
down is simply a matter of taking an input file and hand-crafting an
ideal output file. One of my brothers claims he isn't a programmer
(though he's better than he lets on), but when it comes to figuring
iout a coherent spec, he's awesome... if I could put someone like him
between the programmer and the end user, it'd save so much time.

ChrisA


From steve at pearwood.info  Mon Jun 22 21:02:19 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 23 Jun 2015 11:02:19 +1000
Subject: How do I get the currently installed tab completion function?
Message-ID: <5588b01b$0$1649$c3e8da3$5496439d@news.astraweb.com>

I have two (or more) different types of tab completion, and I want one to
apply under certain circumstances, and the other to apply at others. For
example, let's say I want one to apply inside a function which uses
raw_input (or input in Python 3), and the other to apply the rest of the
time.

So I might use this:

def getfilename():
    prev = get_the_current_completer()  # <<--- this
    readline.set_completer(filename_completer)
    try:
        return raw_input("What file do you want? ")
    finally:
        readline.set_completer(prev)


Is that the best (only) way?

How do I get the currently installed completer?

Solutions for any version of Python acceptable, but if they work all the way
back to 2.4 or older, even better.



-- 
Steven



From rosuav at gmail.com  Mon Jun 22 21:23:55 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 23 Jun 2015 11:23:55 +1000
Subject: How do I get the currently installed tab completion function?
In-Reply-To: <5588b01b$0$1649$c3e8da3$5496439d@news.astraweb.com>
References: <5588b01b$0$1649$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmo1ycRkvCBf75usZYZjqhGyYbXR1hjuspD6KygNOezjdQ@mail.gmail.com>

On Tue, Jun 23, 2015 at 11:02 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> How do I get the currently installed completer?
>
> Solutions for any version of Python acceptable, but if they work all the way
> back to 2.4 or older, even better.

Whether there's a way to avoid the whole try/finally I can't say, but
I just went looking for the obvious "readline.get_completer()", and it
does seem to be there. Is there something I'm missing here?

Found it on 3.4, 3.5, 3.6, and 2.7, but I don't have anything older to test on.

ChrisA


From no.email at nospam.invalid  Mon Jun 22 21:32:52 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Mon, 22 Jun 2015 18:32:52 -0700
Subject: Simplest/Idiomatic way to count files in a directory (using
 pathlib)
References: <mailman.712.1435012328.13271.python-list@python.org>
Message-ID: <87k2uvjjjv.fsf@nightsong.com>

Travis Griggs <travisgriggs at gmail.com> writes:
> The following seems to obtuse/clever for its own good:
>     return sum(1 for _ in self.path.iterdir())

I've generally done something like that.  I suppose it could be added to
itertools.


From steve at pearwood.info  Mon Jun 22 21:41:43 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 23 Jun 2015 11:41:43 +1000
Subject: How do I get the currently installed tab completion function?
References: <5588b01b$0$1649$c3e8da3$5496439d@news.astraweb.com>
 <mailman.716.1435022639.13271.python-list@python.org>
Message-ID: <5588b957$0$1664$c3e8da3$5496439d@news.astraweb.com>

On Tue, 23 Jun 2015 11:23 am, Chris Angelico wrote:

> On Tue, Jun 23, 2015 at 11:02 AM, Steven D'Aprano <steve at pearwood.info>
> wrote:
>> How do I get the currently installed completer?
>>
>> Solutions for any version of Python acceptable, but if they work all the
>> way back to 2.4 or older, even better.
> 
> Whether there's a way to avoid the whole try/finally I can't say, but
> I just went looking for the obvious "readline.get_completer()", and it
> does seem to be there. Is there something I'm missing here?

No, but there's obviously something *I'm* missing.

I don't know how I missed that :-(

It's especially embarrassing because it is available all the way back to
version 2.4, which is exactly what I need.


Sorry for the noise.


-- 
Steven



From rosuav at gmail.com  Mon Jun 22 22:02:33 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 23 Jun 2015 12:02:33 +1000
Subject: How do I get the currently installed tab completion function?
In-Reply-To: <5588b957$0$1664$c3e8da3$5496439d@news.astraweb.com>
References: <5588b01b$0$1649$c3e8da3$5496439d@news.astraweb.com>
 <mailman.716.1435022639.13271.python-list@python.org>
 <5588b957$0$1664$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmrZDAQTtbMHnCE-0xKhuBTin8hqugwqgbvCm_R++jUWdg@mail.gmail.com>

On Tue, Jun 23, 2015 at 11:41 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Tue, 23 Jun 2015 11:23 am, Chris Angelico wrote:
>
>> On Tue, Jun 23, 2015 at 11:02 AM, Steven D'Aprano <steve at pearwood.info>
>> wrote:
>>> How do I get the currently installed completer?
>>>
>>> Solutions for any version of Python acceptable, but if they work all the
>>> way back to 2.4 or older, even better.
>>
>> Whether there's a way to avoid the whole try/finally I can't say, but
>> I just went looking for the obvious "readline.get_completer()", and it
>> does seem to be there. Is there something I'm missing here?
>
> No, but there's obviously something *I'm* missing.
>
> I don't know how I missed that :-(
>
> It's especially embarrassing because it is available all the way back to
> version 2.4, which is exactly what I need.
>
>
> Sorry for the noise.

That's still only the lesser option, of course. Better would be a way
to say raw_input("prompt? ", completer=filename_completer) but that's
not an option, so it'd have to be an explicit readline.something()
call. I can't find any way to actually ask the readline module to read
a line, though, but given that my experience with that module is
effectively zip, someone else may well be able to offer a superior
suggestion.

ChrisA


From steve at pearwood.info  Mon Jun 22 22:32:22 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 23 Jun 2015 12:32:22 +1000
Subject: How do I get the currently installed tab completion function?
References: <5588b01b$0$1649$c3e8da3$5496439d@news.astraweb.com>
 <mailman.716.1435022639.13271.python-list@python.org>
 <5588b957$0$1664$c3e8da3$5496439d@news.astraweb.com>
 <mailman.717.1435024955.13271.python-list@python.org>
Message-ID: <5588c536$0$1668$c3e8da3$5496439d@news.astraweb.com>

On Tue, 23 Jun 2015 12:02 pm, Chris Angelico wrote:

> On Tue, Jun 23, 2015 at 11:41 AM, Steven D'Aprano <steve at pearwood.info>
> wrote:
>> On Tue, 23 Jun 2015 11:23 am, Chris Angelico wrote:
>>
>>> On Tue, Jun 23, 2015 at 11:02 AM, Steven D'Aprano <steve at pearwood.info>
>>> wrote:
>>>> How do I get the currently installed completer?
>>>>
>>>> Solutions for any version of Python acceptable, but if they work all
>>>> the way back to 2.4 or older, even better.
>>>
>>> Whether there's a way to avoid the whole try/finally I can't say, but
>>> I just went looking for the obvious "readline.get_completer()", and it
>>> does seem to be there. Is there something I'm missing here?
>>
>> No, but there's obviously something *I'm* missing.
>>
>> I don't know how I missed that :-(
>>
>> It's especially embarrassing because it is available all the way back to
>> version 2.4, which is exactly what I need.
>>
>>
>> Sorry for the noise.
> 
> That's still only the lesser option, of course. Better would be a way
> to say raw_input("prompt? ", completer=filename_completer) but that's
> not an option, so it'd have to be an explicit readline.something()
> call. I can't find any way to actually ask the readline module to read
> a line, though, but given that my experience with that module is
> effectively zip, someone else may well be able to offer a superior
> suggestion.

The point is to have the completion available while the user types at the
prompt, not to apply it afterwards. So you have to install it as a
completer function under readline.




-- 
Steven



From denismfmcmahon at gmail.com  Tue Jun 23 02:34:15 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Tue, 23 Jun 2015 06:34:15 +0000 (UTC)
Subject: JSON Object to CSV File Troubleshooting
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
 <mm5n0u$5m8$5@dont-email.me>
 <44a2d55f-9312-4b70-8b23-637466b21b11@googlegroups.com>
 <mailman.685.1434923721.13271.python-list@python.org>
 <mm7hjt$n8l$4@dont-email.me>
 <411541bf-7da6-42c1-b332-2761494d29ba@googlegroups.com>
Message-ID: <mmaul7$a7t$1@dont-email.me>

On Sun, 21 Jun 2015 16:56:27 -0700, Sahlusar wrote:

> Here is an example XML document that I am working with:

> You are welcome to contribute and provide me with feedback. Thank you
> for your continued feedback and guidance.

Your XML is invalid! You have a closing MO tag with no opening tag.

> This is the output that I seek:

> > "a","b"   // headers 
> > "b",1     // data row 1 
> > "b",2     // data row 2

I assume from this that where you have a list of identical elements at 
the deepest nesting level of the XML file, you require one CSV record for 
each element in that list, and that you want all higher level element 
values and attributes duplicated in each CSV record.

I assume that you want to use the tag name of each element as the 
identifier for the text content of the element (where there is any), and 
that where an element has no text content, a 0 value is appropriate.

I also assume that as long as the relationship between the headers and 
the data is correct, it does not matter what order the headers are in, ie 
the data and headers can both be sorted by the header.

The output of my code generated once I had corrected your broken XML by 
inserting an 'MO' opening tag between the 'Response' opening tag and the 
'MonthDayCount' opening tag can be seen at:

http://www.sined.co.uk/tmp/xml_to_csv.txt

This was generated from xml file:

http://www.sined.co.uk/tmp/xml_data.txt

If you want the code that produced it, we can discuss fees, it took a few 
hours and for consultancy like this I expect a few 10s of $ per hour.

There may be a generic method to do what you want involving parsing the 
xml to a nested dictionary / list data object, and then flattening that 
object, but I don't see that generating you one line of CSV for each 
Int32 in MonthDayCount.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From sabri.pllana at gmail.com  Tue Jun 23 05:51:23 2015
From: sabri.pllana at gmail.com (SP)
Date: Tue, 23 Jun 2015 02:51:23 -0700 (PDT)
Subject: Final CFP: 2015 Workshop on Exascale Multi/many Core Computing
 Systems (E-MuCoCoS)
Message-ID: <35cc5ba3-f0bb-4fa0-abad-5cfbe7f79ac1@googlegroups.com>


CALL FOR PAPERS

2015 Workshop on Exascale Multi/many Core Computing Systems (E-MuCoCoS)

- Paper submission deadline: June 30, 2015 (firm deadline)
- Workshop proceedings are published by the IEEE


* CONTEXT

Exascale computing will revolutionize computational science and
engineering by providing 1000x the capabilities of currently available
computing systems, while having a similar power footprint. The HPC
community is working towards the development of the first Exaflop
computer (expected around 2020) after reaching the Petaflop milestone
in 2008. There are concerns that computer designs based on existing
multi-core and many-core solutions will not scale to Exascale
considering technical challenges (such as, productivity, energy
consumption or reliability) and reasonable economic constraints.
Therefore, novel multi-core and many-core solutions are required to
reach Exascale.

E-MuCoCoS will be organized in conjunction with the 18th IEEE
Conference on Computational Science and Engineering (CSE 2015), Porto,
Portugal, October 21 - 23, 2015.

Previous editions of MuCoCoS workshop include: MuCoCoS 2014 (Porto,
PT), MuCoCoS 2013 (Edinburgh, UK), MuCoCoS 2012 (SLC, US), MuCoCoS
2011 (Seoul, KR), MuCoCoS 2010 (Krakow, PL), MuCoCoS 2009 (Fukuoka,
JP), MuCoCoS 2008 (Barcelona, ES).

* TOPICS OF INTEREST

E-MuCoCoS focuses on multi/many core languages, system software and
architectural solutions towards Exascale computing systems. The topics
of the workshop include but are not limited to:

:: Methods and tools for preparing applications for Exascale
:: Programming models, languages, libraries and compilation techniques
:: Run-time systems and hardware support
:: Patterns, algorithms and data structures
:: Performance analysis, modeling, optimization and tuning

* SUBMISSION GUIDELINES

- The papers should be prepared using the IEEE format, and no longer
than 6 pages. Submitted papers will be carefully evaluated based on
originality, significance to workshop topics, technical soundness, and
presentation quality.
- Submission of the paper implies that should the paper be accepted,
at least one of the authors will register and present the paper at the
workshop.
- Please submit your paper (as PDF) electronically using the online
submission system https://easychair.org/conferences/?conf=emucocos2015

* IMPORTANT DATES

- Submission: June 30, 2015 (firm deadline)
- Notification: July 28, 2015
- Camera ready: September 4, 2015
- Registration: September 4, 2015
- Workshop: October 20, 2015


* PROGRAM COMMITTEE

- Erika Abraham, RWTH Aachen University (DE)
- Siegfried Benkner, University of Vienna (AT)
- Eduardo Cesar, UAB (ES)
- Jiri Dokulil, University of Vienna, (AT)
- Norbert Eicker, J?lich Supercomputing Centre (DE)
- Franz Franchetti, Carnegie Mellon University (US)
- Samir Genaim, Universidad Complutense de Madrid (ES)
- Houcine Hassan, Universitat Politecnica de Valencia, (ES)
- Atsushi Hori, RIKEN AICS (JP)
- Einar Broch Johnsen, University of Oslo, (NO)
- Christos Kartsaklis, Oak Ridge National Laboratory (US)
- J?rg Keller, FernUniversit?t in Hagen (DE)
- Christoph Kessler, Link?ping University (SE)
- Joanna Kolodziej, Cracow University of Technology (PL)
- Ivan Kondov, Karlsruhe Institute of Technology (DE)
- Erwin Laure, KTH/PDC (SE)
- Renato Miceli, SENAI CIMATEC (BR)
- Lasse Natvig, Norwegian University of Science and Technology (NO)
- Dana Petcu, West University of Timisoara (RO)
- Uwe Schwiegelshohn, TU Dortmund University (DE)
- Achim Streit, Karlsruhe Institute for Technology (DE)
- Osamu Tatebe, University of Tsukuba, (JP)
- Josef Weidendorfer, TUM (DE)


* WORKSHOP CHAIR

- Sabri Pllana, Linnaeus University, Sweden
http://homepage.lnu.se/staff/saplaa/



From ilia.uk at gmail.com  Tue Jun 23 06:11:08 2015
From: ilia.uk at gmail.com (ilia.uk at gmail.com)
Date: Tue, 23 Jun 2015 03:11:08 -0700 (PDT)
Subject: Extract Stores from Store Locators
Message-ID: <01f50743-a63d-4b18-8c38-f4e54cc49a51@googlegroups.com>

Hi Everyone,

I was curious whether there was a way to get a list of all stores from a store locator? Currently I am limited to 10 so using requests in python doesn't give me much advantage over the web-browser interface:

import requests

url = 'http://promotions.coral.co.uk/iframes/shop_locator/location-data/getLocations.php'

headers = {
    'Content-Type': 'application/x-www-form-urlencoded',
    'User-Agent' : 'Mozilla Firefox',
    'Host' : 'promotions.coral.co.uk',
    'Content-Length': '8'
}

payload = {
    'q':'london'
}

ourRequest = requests.post(url, headers = headers, data = payload)
response = ourRequest.json()

for x in response['locations']:
    print(x['address'])


From breamoreboy at yahoo.co.uk  Tue Jun 23 06:13:46 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 23 Jun 2015 11:13:46 +0100
Subject: MGCP Server implementation
In-Reply-To: <E0FE14A926805D41A32BEA79FD6E7AD626467438@xmb-rcd-x10.cisco.com>
References: <E0FE14A926805D41A32BEA79FD6E7AD626467438@xmb-rcd-x10.cisco.com>
Message-ID: <mmbbgt$kk0$1@ger.gmane.org>

On 22/06/2015 21:42, Devaki Chokshi (dchokshi) wrote:
> Hello,
>
> I have a use case where a SIP voice call will be passing through an MGCP
> gateway.
>
> Is there a python implementation that simulates MGCP gateway/server?
>
> Thank you
>
> Devaki Chokshi
>

There appears to be, but I'll let you type "python MGCP gateway/server" 
into your favourite search engine and see if you can find the same thing.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lac at openend.se  Tue Jun 23 07:21:19 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 23 Jun 2015 13:21:19 +0200
Subject: Simplest/Idiomatic way to count files in a directory (using
 pathlib)
In-Reply-To: Message from Travis Griggs <travisgriggs@gmail.com> of "Mon,
 22 Jun 2015 15:31:58 -0700." <8B8C6022-75BE-41FC-9B18-7D89AD34A35A@gmail.com>
References: <8B8C6022-75BE-41FC-9B18-7D89AD34A35A@gmail.com>
Message-ID: <201506231121.t5NBLJ6J002731@fido.openend.se>

I use len(list(self.path.iterdir()))

You get an extra list created in there.  Do you care?

Laura



From ahlusar.ahluwalia at gmail.com  Tue Jun 23 07:32:01 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Tue, 23 Jun 2015 04:32:01 -0700 (PDT)
Subject: JSON Object to CSV File Troubleshooting
In-Reply-To: <mmaul7$a7t$1@dont-email.me>
References: <d5bc266f-2e87-406f-b3f8-9c784112874b@googlegroups.com>
 <mm5n0u$5m8$5@dont-email.me>
 <44a2d55f-9312-4b70-8b23-637466b21b11@googlegroups.com>
 <mailman.685.1434923721.13271.python-list@python.org>
 <mm7hjt$n8l$4@dont-email.me>
 <411541bf-7da6-42c1-b332-2761494d29ba@googlegroups.com>
 <mmaul7$a7t$1@dont-email.me>
Message-ID: <f55f25d7-6643-4163-87e2-029e8877840a@googlegroups.com>

On Tuesday, June 23, 2015 at 2:35:50 AM UTC-4, Denis McMahon wrote:
> On Sun, 21 Jun 2015 16:56:27 -0700, Sahlusar wrote:
> 
> > Here is an example XML document that I am working with:
> 
> > You are welcome to contribute and provide me with feedback. Thank you
> > for your continued feedback and guidance.
> 
> Your XML is invalid! You have a closing MO tag with no opening tag.

Thank you Denis for reiterating this; I am already well aware that my XML document has arbitrarily placed tags.


> 
> > This is the output that I seek:
> 
> > > "a","b"   // headers 
> > > "b",1     // data row 1 
> > > "b",2     // data row 2
> 
> I assume from this that where you have a list of identical elements at 
> the deepest nesting level of the XML file, you require one CSV record for 
> each element in that list, and that you want all higher level element 
> values and attributes duplicated in each CSV record.

Almost, the items in the list should have separate "headers" associated with each tag (and enumerated):

MonthDayCount1, MonthDayCount2, MonthDayCount3, MonthDayCount4 etc....

0                      0              0             0

> 
> I assume that you want to use the tag name of each element as the 
> identifier for the text content of the element (where there is any), and 
> that where an element has no text content, a 0 value is appropriate.

Yes, the tag name is the "key", so to speak. 

> 
> I also assume that as long as the relationship between the headers and 
> the data is correct, it does not matter what order the headers are in, ie 
> the data and headers can both be sorted by the header.
> 
> The output of my code generated once I had corrected your broken XML by 
> inserting an 'MO' opening tag between the 'Response' opening tag and the 
> 'MonthDayCount' opening tag can be seen at:
> 
> http://www.sined.co.uk/tmp/xml_to_csv.txt
> 
> This was generated from xml file:
> 
> http://www.sined.co.uk/tmp/xml_data.txt
> 
> If you want the code that produced it, we can discuss fees, it took a few 
> hours and for consultancy like this I expect a few 10s of $ per hour.


I already have solved this and produced a CSV that meets the clients proposed (yet amorphous requirements). As ChrisA pointed out, programmers should have an intermediate person who could potentially serve as a translator between the client and developer. Unfortunately, this is an ideal that is stifled by labor economics. 

> 
> There may be a generic method to do what you want involving parsing the 
> xml to a nested dictionary / list data object, and then flattening that 
> object, but I don't see that generating you one line of CSV for each 
> Int32 in MonthDayCount.


I have done just that. Thanks again for your help. 

> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com


From lac at openend.se  Tue Jun 23 07:46:42 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 23 Jun 2015 13:46:42 +0200
Subject: MGCP Server implementation
In-Reply-To: Message from "Devaki Chokshi (dchokshi)" <dchokshi@cisco.com>
 of "Mon, 22 Jun 2015 20:42:21 -0000."
 <E0FE14A926805D41A32BEA79FD6E7AD626467438@xmb-rcd-x10.cisco.com>
References: <E0FE14A926805D41A32BEA79FD6E7AD626467438@xmb-rcd-x10.cisco.com>
Message-ID: <201506231146.t5NBkgZD003602@fido.openend.se>

In a message of Mon, 22 Jun 2015 20:42:21 -0000, "Devaki Chokshi (dchokshi)" wr
ites:
>Hello,
>
>I have a use case where a SIP voice call will be passing through an MGCP gateway.
>
>Is there a python implementation that simulates MGCP gateway/server?
>
>Thank you
>Devaki Chokshi
>
>-- 
>https://mail.python.org/mailman/listinfo/python-list
>

I think http://www.secdev.org/projects/scapy/ will do this for you.
In particular
https://code.google.com/p/pythonxy/source/browse/src/python/_scapy/PURELIB/scapy/layers/mgcp.py?repo=xy-27&name=v2.7.2.1&r=c4a08530c8ff74fd6078507743ce9982675aea8c

But I have only used Sacpy for other protocols, so I am not certain.

Laura


From __peter__ at web.de  Tue Jun 23 08:02:09 2015
From: __peter__ at web.de (Peter Otten)
Date: Tue, 23 Jun 2015 14:02:09 +0200
Subject: Simplest/Idiomatic way to count files in a directory (using
 pathlib)
References: <8B8C6022-75BE-41FC-9B18-7D89AD34A35A@gmail.com>
Message-ID: <mmbhs3$tdj$1@ger.gmane.org>

Travis Griggs wrote:

> Subject nearly says it all.
> 
> If i?m using pathlib, what?s the simplest/idiomatic way to simply count
> how many files are in a given directory?
> 
> I was surprised (at first) when
> 
>     len(self.path.iterdir())
> 
> I don?t say anything on the in the .stat() object that helps me.
> 
> I could of course do the 4 liner:
> 
>     count = 0
>     for _ in self.path.iterdir():
>         count += 1
>     return count
> 
> The following seems to obtuse/clever for its own good:
> 
>     return sum(1 for _ in self.path.iterdir())

Put the "cleverness" into its own function

def iterlen(items):
    return sum(1 for _ items)

num_files = iterlen(path.iterdir())

With the proper unit tests in place you may even switch to more exotic 
approaches later:

def iterlen(items):
    """Exhaust an iterator an return the number of items.

    Does not check for a __len__() method.

    >>> iterlen(())
    0
    >>> iterlen([42])
    1
    >>> iterlen("abc")
    3

    >>> class A:
    ...     def __len__(self): return 42
    ...     def __iter__(self): yield from "abc"
    >>> a = A()
    >>> len(a)
    42
    >>> iterlen(a)
    3
    """
    d = collections.deque(enumerate(items, 1), maxlen=1)
    return d.pop()[0] if d else 0




From nmichalowsky at gmail.com  Tue Jun 23 09:39:44 2015
From: nmichalowsky at gmail.com (Naftali)
Date: Tue, 23 Jun 2015 06:39:44 -0700 (PDT)
Subject: Python File as the Default PDF handler for Windows
In-Reply-To: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>
References: <60c5531e-b4b4-43b2-a3b9-0b53e1490dee@googlegroups.com>
Message-ID: <2bba2e7f-ef41-4b08-8768-fa154c219042@googlegroups.com>

On Thursday, June 18, 2015 at 5:05:15 PM UTC-4, Naftali wrote:
> Long time lurker. 
> 
> I'm looking to register a python script as the default pdf reader for windows. I assume I can just register the .py in the section windows section for registering default handlers, but I'm wondering how to access the file from within the program. 
> 
> The issue is this:
> 
> We have Java application that outputs user uploaded pdf files. It does this simply by instructing windows to open the downloaded pdf file and windows takes it from there. The data entry person will view the pdf and usually upload it into another part of the system. Problem is the second leg of the system modifies the pdf, and thus crashes when the pdf is protected against writing. Data entry make use of a program to unlock them as needed but it is an extra step and it only comes to their awareness after their client crashes on the locked pdf (because it doesn't make sense to check them proactively.
> 
> I cannot change the Java system. 
> 
> What I want to do is write a pdf handler to handle windows open instruction. In the script I would run a command line pdf unlocker on the file and open the unlocked file with adobe (or the like). 
> 
> I've googled and though I get tons of 'how to open pdf from a python script' I haven't found anything describing how to write and set up my python program to deal with the pdf hand-off from the OS.

Just to update, you are correct, Chris, the file short name is passed into sys.argv. didn't need to add anything to the path. But a gotcha -- Windows didn't like my .py, clicking on the pdf causes Windows to complain about 'file x' is not a valid windows executable. I didn't research it so cant speak definitely, but converting the .py to a py2exe correlated with that complaint going away. In the end, though since, making any changes to the program required redoing the py2exe and since this is to be running on a colleagues machine, I ended up putting the handler commands into a .bat file. My hunch is that it's faster that way, too.


From felixonmars at archlinux.org  Tue Jun 23 10:08:36 2015
From: felixonmars at archlinux.org (Felix Yan)
Date: Tue, 23 Jun 2015 22:08:36 +0800
Subject: Issuing a sqilte query, picking a random result, and copying to
 the system clipboard
In-Reply-To: <20150622065146.7654f597@bigbox.christie.dr>
References: <CAJGLaBJvbJzYLTVoP0Vh4wWiCqzruUvPspo1QSA7g=V=JfSWTQ@mail.gmail.com>
 <20150622065146.7654f597@bigbox.christie.dr>
Message-ID: <55896864.2010405@archlinux.org>

On 06/22/2015 07:51 PM, Tim Chase wrote:
> On Win32, you'd need the Win32 add-on libraries to shove things onto
> the clipboard, while under X, you'd need other facilities (either
> using Tkinter or piping to something like xclip(1)), and yet another
> way of doing things on MacOS.

Or you may want an existing library for all these. For example, pyperclip:

>>> import pyperclip
>>> pyperclip.copy('The text to be copied to the clipboard.')

FYI: https://pypi.python.org/pypi/pyperclip

-- 
Regards,
Felix Yan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20150623/f27cc3f7/attachment.sig>

From jeanmichel at sequans.com  Tue Jun 23 10:26:28 2015
From: jeanmichel at sequans.com (Jean-Michel Pichavant)
Date: Tue, 23 Jun 2015 16:26:28 +0200 (CEST)
Subject: Python File as the Default PDF handler for Windows
In-Reply-To: <2bba2e7f-ef41-4b08-8768-fa154c219042@googlegroups.com>
Message-ID: <863086551.5226529.1435069587998.JavaMail.root@sequans.com>

> Just to update, you are correct, Chris, the file short name is passed
> into sys.argv. didn't need to add anything to the path. But a gotcha
> -- Windows didn't like my .py, clicking on the pdf causes Windows to
> complain about 'file x' is not a valid windows executable. 

I'm not an expert of windows but you probably need to specify the python interpreter, not the script itself.

something like

C:\Python2.7\python.exe yourscript.py %*

in the windows file association panel.

JM


-- IMPORTANT NOTICE: 

The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

From rosuav at gmail.com  Tue Jun 23 11:00:03 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 24 Jun 2015 01:00:03 +1000
Subject: Python File as the Default PDF handler for Windows
In-Reply-To: <863086551.5226529.1435069587998.JavaMail.root@sequans.com>
References: <2bba2e7f-ef41-4b08-8768-fa154c219042@googlegroups.com>
 <863086551.5226529.1435069587998.JavaMail.root@sequans.com>
Message-ID: <CAPTjJmpSCZmk4Q98_jO4XxE1gsANnq417S_PNm_TDJg-n0cgUg@mail.gmail.com>

On Wed, Jun 24, 2015 at 12:26 AM, Jean-Michel Pichavant
<jeanmichel at sequans.com> wrote:
>> Just to update, you are correct, Chris, the file short name is passed
>> into sys.argv. didn't need to add anything to the path. But a gotcha
>> -- Windows didn't like my .py, clicking on the pdf causes Windows to
>> complain about 'file x' is not a valid windows executable.
>
> I'm not an expert of windows but you probably need to specify the python interpreter, not the script itself.
>
> something like
>
> C:\Python2.7\python.exe yourscript.py %*
>
> in the windows file association panel.

Yep, that's what I'd recommend too. You shouldn't need to py2exe it.

ChrisA


From tjreedy at udel.edu  Tue Jun 23 12:17:34 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 23 Jun 2015 12:17:34 -0400
Subject: Simplest/Idiomatic way to count files in a directory (using
 pathlib)
In-Reply-To: <87k2uvjjjv.fsf@nightsong.com>
References: <mailman.712.1435012328.13271.python-list@python.org>
 <87k2uvjjjv.fsf@nightsong.com>
Message-ID: <mmc0r0$2qo$1@ger.gmane.org>

On 6/22/2015 9:32 PM, Paul Rubin wrote:
> Travis Griggs <travisgriggs at gmail.com> writes:
>> The following seems to obtuse/clever for its own good:
>>      return sum(1 for _ in self.path.iterdir())

I disagree.  For one who understands counting and Python, this is a 
direct way to define the count of a finite iterable.  A functionalist 
would prefer it to the obvious 3-line version with a for statement.

> I've generally done something like that.  I suppose it could be added to
> itertools.

itertools callables all produce iterators.  The above only consumes an 
iterable.

-- 
Terry Jan Reedy



From laurent.pointal at free.fr  Tue Jun 23 12:53:27 2015
From: laurent.pointal at free.fr (Laurent Pointal)
Date: Tue, 23 Jun 2015 18:53:27 +0200
Subject: HOPE: A Python just-in-time compiler for astrophysical
 computations
References: <mailman.664.1434846609.13271.python-list@python.org>
Message-ID: <55898f07$0$3088$426a74cc@news.free.fr>

Mark Lawrence wrote:

> Another beasty I've just stumbled across which you may find interesting
> http://www.sciencedirect.com/science/article/pii/S2213133714000687
 
Why use a JIT complation when you could use some C++ generation then 
compilation as Python module, like with Pythran ?

https://github.com/serge-sans-paille/pythran

For heavy computing, you may loose some time before running computation, to 
have (generate) the right tool, then do the job with real bonus.

?
A+
Laurent.



From ahlusar.ahluwalia at gmail.com  Tue Jun 23 13:15:56 2015
From: ahlusar.ahluwalia at gmail.com (Sahlusar)
Date: Tue, 23 Jun 2015 10:15:56 -0700 (PDT)
Subject: To write headers once with different values in separate row in CSV
Message-ID: <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c@googlegroups.com>

I have the following script for writing out to CSV two items in a list to a CSV in such a format, such that if we have:

L = [('A', 1), ('B', 2), ('C', 3), ('D', 4),('A', 5), ('B', 6), ('D', 8)]

we want

A B C D
1 2 3 4
5 6   8

And with this 

L = [('A', 1), ('B', 2), ('C', 3), ('D', 4),('D', 5), ('B', 6), ('A', 8)]

A B C D
1 2 3 4
8 6   5

I have this script: 

    def makerows(pairs):
        headers = []
        columns = {}
        for k, v in pairs:
            if k in columns:
                columns[k].extend((v,))
            else:
                headers.append(k)
                columns[k] = [k, v]
        m = max(len(c) for c in columns.values())
        for c in columns.values():
            c.extend('' for i in range(len(c), m))
        L = [columns[k] for k in headers]
        rows = list(zip(*L))
        return rows

//With this test//


    if __name__ == '__main__':
        
        lists = [
            [('A', 1), ('B', 2), ('C', 3), ('D', 4),('A', 5), ('B', 6), ('C', 7), ('D', 8)],
            [('A', 1), ('B', 2), ('C', 3), ('D', 4),('A', 5), ('B', 6), ('D', 8)],
            [('A', 1), ('B', 2), ('C', 3), ('D', 4),('D', 5), ('B', 6), ('A', 8)],
        ]
        from pprint import pprint
        for data in lists:
            print(data)
            pprint(makerows(data))


output:
    
    [('A', 1), ('B', 2), ('C', 3), ('D', 4), ('A', 5), ('B', 6), ('C', 7), ('D', 8)]
    [('A', 'B', 'C', 'D'), (1, 2, 3, 4), (5, 6, 7, 8)]
    [('A', 1), ('B', 2), ('C', 3), ('D', 4), ('A', 5), ('B', 6), ('D', 8)]
    [('A', 'B', 'C', 'D'), (1, 2, 3, 4), (5, 6, '', 8)]
    [('A', 1), ('B', 2), ('C', 3), ('D', 4), ('D', 5), ('B', 6), ('A', 8)]  

[('A', 'B', 'C', 'D'), (1, 2, 3, 4), (8, 6, '', 5)]

However, when I extrapolate this same logic with a list like:

('Response.MemberO.PMembers.PMembers.Member.CurrentEmployer.EmployerAddress.TimeAtPreviousAddress.', None), where the headers/columns are the first item (only to be written out once) with different values. I receive an output CSV with repeating headers and values all printed in one long string (when opened in an application like Excel). 

I use this script:

def makerows(pairs):

        headers = []
        columns = {}
        for k, v in pairs:
            if k in columns:
                columns[k].extend((v,))
            else:
                headers.append(k)
                columns[k] = [k, v]
        m = max(len(c) for c in columns.values())
        for c in columns.values():
            c.extend(' ' for i in range(len(c), m))
        L = [columns[k] for k in headers]
        rows = list(zip(*L))
        return rows
    
    
    def main():
        with open('sample.xml', 'r', encoding='utf-8') as f: 
            xml_string = f.read() 
        xml_string= xml_string.replace('&#x0;', '') #optional to remove ampersands. 
        root = ElementTree.XML(xml_string) 
        for item in root:
            print(root)
        writer = csv.writer(open("test_out.csv", 'wt'))
        writer.writerows(makerows(flatten_dict(root)))
        
    if __name__ == "__main__":
            main()


************FYI, flatten_dict() parses XML elements and their text in to key, value pairs and then converts them into a list. That is not the underlying issue. Any thoughts or suggestions would be very helpful. Thank you. 


From jtim.arnold at gmail.com  Tue Jun 23 14:32:40 2015
From: jtim.arnold at gmail.com (Tim)
Date: Tue, 23 Jun 2015 11:32:40 -0700 (PDT)
Subject: method returns object reference
Message-ID: <109aafce-613c-4d6c-9f78-4bbe424f5901@googlegroups.com>

I spent a while finding this problem which looks something like the old "mutable default argument" problem. http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments

I'm not crystal clear on the details, but once I found it, the fix was easy.
I'm posting here just because I thought it was interesting.

The situation is that I have a Worker() which has a Client(). 
The worker queries the client; client returns its self.response
worker manipulates that response. Now client's response has same changes.

The code below prints the following two lines, showing how the 'newkey' is now inside the Client response, even though it was set in the worker.  This must be bad practice!  In my real code, the response is no longer an instance variable, which fixed the problem. It never had any business being bound to the client anyway.

output:
[('text', 'first thing')]
[('text', 'second thing'), ('newkey', 'my new value')]
 

class Client(object):
    def __init__(self):
        self.response = dict()

    def query(self, text):
        self.response['text'] = text
        print self.response.items()
        return self.response


class Worker(object):
    def __init__(self):
        self.client = Client()

    def work(self, expression):
        data = self.client.query(expression)
        data['newkey'] = 'my new value'


if __name__ == '__main__':
    t = Worker()
    t.work('first thing')
    t.work('second thing')


From randall at tnr.cc  Tue Jun 23 15:02:45 2015
From: randall at tnr.cc (Randall Smith)
Date: Tue, 23 Jun 2015 14:02:45 -0500
Subject: Pure Python Data Mangling or Encrypting
Message-ID: <mmcagn$aa5$1@ger.gmane.org>

Chunks of data (about 2MB) are to be stored on machines using a 
peer-to-peer protocol.  The recipient of these chunks can't assume that 
the payload is benign.  While the data senders are supposed to encrypt 
data, that's not guaranteed, and I'd like to protect the recipient 
against exposure to nefarious data by mangling or encrypting the data 
before it is written to disk.

My original idea was for the recipient to encrypt using AES.  But I want 
to keep this software pure Python "batteries included" and not require 
installation of other platform-dependent software.  Pure Python AES and 
even DES are just way too slow.  I don't know that I really need 
encryption here, but some type of fast mangling algorithm where a bad 
actor sending a payload can't guess the output ahead of time.

Any ideas are appreciated.  Thanks.

-Randall



From gordon at panix.com  Tue Jun 23 15:12:28 2015
From: gordon at panix.com (John Gordon)
Date: Tue, 23 Jun 2015 19:12:28 +0000 (UTC)
Subject: To write headers once with different values in separate row in CSV
References: <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c@googlegroups.com>
Message-ID: <mmcb2s$dcm$1@reader1.panix.com>

In <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c at googlegroups.com> Sahlusar <ahlusar.ahluwalia at gmail.com> writes:

> However, when I extrapolate this same logic with a list like:

> ('Response.MemberO.PMembers.PMembers.Member.CurrentEmployer.EmployerAddress
> .TimeAtPreviousAddress.', None), where the headers/columns are the first
> item (only to be written out once) with different values. I receive an
> output CSV with repeating headers and values all printed in one long string

First, I would try to determine if the problem is in the makerows()
function, or if the problem is elsewhere.

Have you tried creating some dummy data by hand and seeing how makerows()
handles it?

(By the way, if your post had included some sample data that illustrates
the problem, it would have been much easier to figure out a solution.
Instead, we are left guessing at your XML format, and at the particular
implementation of flatten_dict().)

-- 
John Gordon                   A is for Amy, who fell down the stairs
gordon at panix.com              B is for Basil, assaulted by bears
                                -- Edward Gorey, "The Gashlycrumb Tinies"



From ian.g.kelly at gmail.com  Tue Jun 23 15:30:46 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 23 Jun 2015 13:30:46 -0600
Subject: method returns object reference
In-Reply-To: <109aafce-613c-4d6c-9f78-4bbe424f5901@googlegroups.com>
References: <109aafce-613c-4d6c-9f78-4bbe424f5901@googlegroups.com>
Message-ID: <CALwzidmQRmDAZk720XR1dD1Ln-Fp9vAKRp8skrO3VYQ6FmXkTQ@mail.gmail.com>

On Tue, Jun 23, 2015 at 12:32 PM, Tim <jtim.arnold at gmail.com> wrote:
> The code below prints the following two lines, showing how the 'newkey' is now inside the Client response, even though it was set in the worker.  This must be bad practice!  In my real code, the response is no longer an instance variable, which fixed the problem. It never had any business being bound to the client anyway.

I agree. There may be reasons why the client would want to hold onto
the response, e.g. caching, but there are none evident in the code you
posted. What it *shouldn't* do if it's going to hold on to state like
that is share it with external code without at least a contract
defining allowed mutations. If the consumer can't reasonably be
expected not to modify the shared state, then it should be copied, not
shared.


From ahlusar.ahluwalia at gmail.com  Tue Jun 23 15:57:25 2015
From: ahlusar.ahluwalia at gmail.com (Saran Ahluwalia)
Date: Tue, 23 Jun 2015 15:57:25 -0400
Subject: To write headers once with different values in separate row in CSV
In-Reply-To: <mmcb2s$dcm$1@reader1.panix.com>
References: <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c@googlegroups.com>
 <mmcb2s$dcm$1@reader1.panix.com>
Message-ID: <CAC9q6M5y9HVvCRQRDXuNZkE=fb=Lwb0ciQ9-BDqo53OaOef1jw@mail.gmail.com>

It was an XML formatting issue; apparently you have to explicitly call a
delimiter via the Command Prompt on my OS. IPython Notebook seems to make
certain assumptions regarding read and write.

On Tue, Jun 23, 2015 at 3:12 PM, John Gordon <gordon at panix.com> wrote:

> In <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c at googlegroups.com> Sahlusar <
> ahlusar.ahluwalia at gmail.com> writes:
>
> > However, when I extrapolate this same logic with a list like:
>
> >
> ('Response.MemberO.PMembers.PMembers.Member.CurrentEmployer.EmployerAddress
> > .TimeAtPreviousAddress.', None), where the headers/columns are the first
> > item (only to be written out once) with different values. I receive an
> > output CSV with repeating headers and values all printed in one long
> string
>
> First, I would try to determine if the problem is in the makerows()
> function, or if the problem is elsewhere.
>
> Have you tried creating some dummy data by hand and seeing how makerows()
> handles it?
>
> (By the way, if your post had included some sample data that illustrates
> the problem, it would have been much easier to figure out a solution.
> Instead, we are left guessing at your XML format, and at the particular
> implementation of flatten_dict().)
>
> --
> John Gordon                   A is for Amy, who fell down the stairs
> gordon at panix.com              B is for Basil, assaulted by bears
>                                 -- Edward Gorey, "The Gashlycrumb Tinies"
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150623/c3343fff/attachment.html>

From ahlusar.ahluwalia at gmail.com  Tue Jun 23 16:16:09 2015
From: ahlusar.ahluwalia at gmail.com (kbtyo)
Date: Tue, 23 Jun 2015 13:16:09 -0700 (PDT)
Subject: Organizing function calls once files have been moved to a directory
Message-ID: <66a66a08-a5f2-4571-819a-38427c46fcf4@googlegroups.com>

I am working on a workflow module that will allow one to recursively check for file extensions and if there is a match move them to a folder for processing (parsing, data wrangling etc). 

I have a simple search process, and log for the files that are present (see below). However, I am puzzled by what the most efficient method/syntax is to call functions once the selected files have been moved? I have the functions and classes written in another file. Should I import them or should I include them in the same file as the following mini-script?

Moreover, should I create another log file for processing? If so, what is an idiomatically correct method to do so? 

if __name__ == '__main__':

# The top argument for name in files
    topdir = '.'
    dest = 'C:\\Users\\wynsa2\\Desktop\\'
    extens = ['docs', 'docx', 'pdf'] # the extensions to search for     
    found = {x: [] for x in extens} # lists of found files    
 
    # Directories to ignore
    ignore = ['docs', 'doc', 'py', 'pdf']     
    logname = "file_search.log"    
    print('Beginning search for files in %s' % os.path.realpath(topdir))  
  
    # Walk the tree
    for dirpath, dirnames, files in os.walk(topdir):
        # Remove directories in ignore
        # directory names must match exactly!
        for idir in ignore:
            if idir in dirnames:
                dirnames.remove(idir)
     
        # Loop through the file names for the current step
        for name in files:
     #Calling str.rsplit on name then 
    #splits the string into a list (from the right) 
    #with the first argument "."" delimiting it, 
    #and only making as many splits as the second argument (1). 
    #The third part ([-1]) retrieves the last element of the list--we 
    #use this instead of an index of 1 because if no splits are made 
    #(if there is no "."" in name), no IndexError will be raised

            ext = name.lower().rsplit('.', 1)[-1]
     
            # Save the full name if ext matches
            #log_results, errlog and batchcopy are functions
            if ext in extens:
                found[ext].append(os.path.join(dirpath, name))
                log_results(logname, found)
                batchcopy(found, dest, errlog=None)

Thank you for your help. 
              


From breamoreboy at yahoo.co.uk  Tue Jun 23 16:20:21 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 23 Jun 2015 21:20:21 +0100
Subject: Python File as the Default PDF handler for Windows
In-Reply-To: <h5fjoali71hjsrvs6u739kn0pukjdu4iq6@4ax.com>
References: <2bba2e7f-ef41-4b08-8768-fa154c219042@googlegroups.com>
 <863086551.5226529.1435069587998.JavaMail.root@sequans.com>
 <h5fjoali71hjsrvs6u739kn0pukjdu4iq6@4ax.com>
Message-ID: <mmcf28$l2s$1@ger.gmane.org>

On 23/06/2015 21:12, Dennis Lee Bieber wrote:
> On Tue, 23 Jun 2015 16:26:28 +0200 (CEST), Jean-Michel Pichavant
> <jeanmichel at sequans.com> declaimed the following:
>
>>> Just to update, you are correct, Chris, the file short name is passed
>>> into sys.argv. didn't need to add anything to the path. But a gotcha
>>> -- Windows didn't like my .py, clicking on the pdf causes Windows to
>>> complain about 'file x' is not a valid windows executable.
>>
>> I'm not an expert of windows but you probably need to specify the python interpreter, not the script itself.
>>
>> something like
>>
>> C:\Python2.7\python.exe yourscript.py %*
>>
>> in the windows file association panel.
>>
> 	Need to check the specific install, I've seen two or three different
> assoc names used in the past...
>
> Microsoft Windows [Version 6.1.7601]
> Copyright (c) 2009 Microsoft Corporation.  All rights reserved.
>
> C:\Users\Wulfraed\Documents>assoc .py
> .py=Python.File
>
> C:\Users\Wulfraed\Documents>ftype python.file
> python.file="C:\Python_x64\Python27\python.exe" "%1" %*
>
> 	ftype links a "file type" handler to the executable path (note the "%1"
> is the placeholder for the script name itself, and arguments to the script
> are collected with %*). Assoc then links the extension to the handler (so
> you can also assoc .pyc files to the same interpreter).
>

With Python 3.4 or if you've installed the pylauncher independantly.

C:\Users\Mark\Documents\MyPython>assoc .py
.py=Python.File

C:\Users\Mark\Documents\MyPython>ftype python.file
python.file="C:\Windows\py.exe" "%1" %*

C:\Users\Mark\Documents\MyPython>assoc .pyw
.pyw=Python.NoConFile

C:\Users\Mark\Documents\MyPython>ftype python.noconfile
python.noconfile="C:\Windows\pyw.exe" "%1" %*

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From ned at nedbatchelder.com  Tue Jun 23 16:59:03 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 23 Jun 2015 13:59:03 -0700 (PDT)
Subject: method returns object reference
In-Reply-To: <109aafce-613c-4d6c-9f78-4bbe424f5901@googlegroups.com>
References: <109aafce-613c-4d6c-9f78-4bbe424f5901@googlegroups.com>
Message-ID: <99d81f4b-fbdb-4309-9bd7-f278e0119079@googlegroups.com>

On Tuesday, June 23, 2015 at 2:32:55 PM UTC-4, Tim wrote:
> I spent a while finding this problem which looks something like the old "mutable default argument" problem. http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments
> 
> I'm not crystal clear on the details, but once I found it, the fix was easy.
> I'm posting here just because I thought it was interesting.
> 
> The situation is that I have a Worker() which has a Client(). 
> The worker queries the client; client returns its self.response
> worker manipulates that response. Now client's response has same changes.

Assignment in Python never makes a copy, it only makes a new name refer to
an existing value.  This talk covers the details in depth:

    http://nedbatchelder.com/text/names1.html

--Ned.


From drsalists at gmail.com  Tue Jun 23 20:21:55 2015
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 23 Jun 2015 17:21:55 -0700
Subject: Looking up a dictionary _key_ by key?
Message-ID: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>

I know that sounds strange: usually we look up values by key, not keys.

But suppose you have a strange key type that despite being "equal", is
not identical in some fields, and you need to see those fields.

Is there a way of getting the key used by the dictionary, short of
storing a reference to it in the value, or using a second dictionary?

Thanks!


From ben+python at benfinney.id.au  Tue Jun 23 20:33:50 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 24 Jun 2015 10:33:50 +1000
Subject: Looking up a dictionary _key_ by key?
References: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
Message-ID: <851th2t05t.fsf@benfinney.id.au>

Dan Stromberg <drsalists at gmail.com> writes:

> Is there a way of getting the key used by the dictionary, short of
> storing a reference to it in the value, or using a second dictionary?

The dictionary knows its keys and can provide them on request. Call the
?dict.keys? method to get them as a collection.

-- 
 \     ?Capitalism has destroyed our belief in any effective power but |
  `\      that of self interest backed by force.? ?George Bernard Shaw |
_o__)                                                                  |
Ben Finney



From drsalists at gmail.com  Tue Jun 23 20:47:58 2015
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 23 Jun 2015 17:47:58 -0700
Subject: Looking up a dictionary _key_ by key?
In-Reply-To: <851th2t05t.fsf@benfinney.id.au>
References: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
 <851th2t05t.fsf@benfinney.id.au>
Message-ID: <CAGGBd_qaNC38Yg0u_b8+qBAckq6FRTZMxbWwuzV=MUYE_=ZzDw@mail.gmail.com>

On Tue, Jun 23, 2015 at 5:33 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Dan Stromberg <drsalists at gmail.com> writes:
>
>> Is there a way of getting the key used by the dictionary, short of
>> storing a reference to it in the value, or using a second dictionary?
>
> The dictionary knows its keys and can provide them on request. Call the
> ?dict.keys? method to get them as a collection.

So it appears:

$ ./pythons --command 'd={1:2, 2:4, 3:8}; print(d.keys())'
/usr/local/cpython-2.4/bin/python good [1, 2, 3]
/usr/local/cpython-2.5/bin/python good [1, 2, 3]
/usr/local/cpython-2.6/bin/python good [1, 2, 3]
/usr/local/cpython-2.7/bin/python good [1, 2, 3]
/usr/local/cpython-3.0/bin/python good <dict_keys object at 0x8514fa4>
/usr/local/cpython-3.1/bin/python good dict_keys([1, 2, 3])
/usr/local/cpython-3.2/bin/python good dict_keys([1, 2, 3])
/usr/local/cpython-3.3/bin/python good dict_keys([1, 2, 3])
/usr/local/cpython-3.4/bin/python good dict_keys([1, 2, 3])
/usr/local/cpython-3.5/bin/python good dict_keys([1, 2, 3])
/usr/local/jython-2.7rc1/bin/jython good [3, 2, 1]
/usr/local/pypy-2.5.0/bin/pypy good [1, 2, 3]
/usr/local/pypy3-2.4.0/bin/pypy3 good dict_keys([1, 2, 3])

But:

$ ./pythons --command 'd={1:2, 2:4, 3:8}; print(d.keys()[1])'
/usr/local/cpython-2.4/bin/python good 2
/usr/local/cpython-2.5/bin/python good 2
/usr/local/cpython-2.6/bin/python good 2
/usr/local/cpython-2.7/bin/python good 2
/usr/local/cpython-3.0/bin/python bad
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    TypeError: 'dict_keys' object does not support indexing
/usr/local/cpython-3.1/bin/python bad
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    TypeError: 'dict_keys' object does not support indexing
/usr/local/cpython-3.2/bin/python bad
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    TypeError: 'dict_keys' object does not support indexing
/usr/local/cpython-3.3/bin/python bad
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    TypeError: 'dict_keys' object does not support indexing
/usr/local/cpython-3.4/bin/python bad
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    TypeError: 'dict_keys' object does not support indexing
/usr/local/cpython-3.5/bin/python bad
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    TypeError: 'dict_keys' object does not support indexing
/usr/local/jython-2.7rc1/bin/jython good 2
/usr/local/pypy-2.5.0/bin/pypy good 2
/usr/local/pypy3-2.4.0/bin/pypy3 bad
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    TypeError: 'dict_keys' object is not subscriptable

Would I have to do an O(n) search to find my key?

Thanks!


From rosuav at gmail.com  Tue Jun 23 21:02:03 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 24 Jun 2015 11:02:03 +1000
Subject: Looking up a dictionary _key_ by key?
In-Reply-To: <CAGGBd_qaNC38Yg0u_b8+qBAckq6FRTZMxbWwuzV=MUYE_=ZzDw@mail.gmail.com>
References: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
 <851th2t05t.fsf@benfinney.id.au>
 <CAGGBd_qaNC38Yg0u_b8+qBAckq6FRTZMxbWwuzV=MUYE_=ZzDw@mail.gmail.com>
Message-ID: <CAPTjJmq_PO_4b9mS-vxHdr_kvmytOSMPD3pFAmLMzS812KpZtg@mail.gmail.com>

On Wed, Jun 24, 2015 at 10:47 AM, Dan Stromberg <drsalists at gmail.com> wrote:
> Would I have to do an O(n) search to find my key?

Iterate over it - it's an iterable view in Py3 - and compare.

ChrisA


From no.email at nospam.invalid  Tue Jun 23 21:02:48 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Tue, 23 Jun 2015 18:02:48 -0700
Subject: Looking up a dictionary _key_ by key?
References: <mailman.0.1435105323.3674.python-list@python.org>
Message-ID: <87h9pxyl3b.fsf@nightsong.com>

Dan Stromberg <drsalists at gmail.com> writes:
> suppose you have a strange key type that despite being "equal", is
> not identical in some fields

Don't do that.  You were asking for trouble and you got some.


From no.email at nospam.invalid  Tue Jun 23 21:06:45 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Tue, 23 Jun 2015 18:06:45 -0700
Subject: Looking up a dictionary _key_ by key?
References: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
 <851th2t05t.fsf@benfinney.id.au>
 <CAGGBd_qaNC38Yg0u_b8+qBAckq6FRTZMxbWwuzV=MUYE_=ZzDw@mail.gmail.com>
 <mailman.3.1435107726.3674.python-list@python.org>
Message-ID: <87a8vpykwq.fsf@nightsong.com>

Chris Angelico <rosuav at gmail.com> writes:
>> Would I have to do an O(n) search to find my key?
> Iterate over it - it's an iterable view in Py3 - and compare.

I think the question was whether the O(n) search could be avoided, not
how to do it.  I don't see a way to avoid it.  There is fundamental
brokenness in having unequal objects compare as equal, and the breakage
messes up the dictionary when those objects are used as keys.

Solution is to either fix the object equality test, or wrap them in
something (maybe a tuple containing the objects and the distinguishing
fields that are missing from the original object's equality method) that
treats unequal objects as unequal.


From ben+python at benfinney.id.au  Tue Jun 23 21:07:46 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 24 Jun 2015 11:07:46 +1000
Subject: Looking up a dictionary _key_ by key?
References: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
 <851th2t05t.fsf@benfinney.id.au>
 <CAGGBd_qaNC38Yg0u_b8+qBAckq6FRTZMxbWwuzV=MUYE_=ZzDw@mail.gmail.com>
Message-ID: <85wpytsyl9.fsf@benfinney.id.au>

Dan Stromberg <drsalists at gmail.com> writes:

> On Tue, Jun 23, 2015 at 5:33 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> > Dan Stromberg <drsalists at gmail.com> writes:
> >
> >> Is there a way of getting the key used by the dictionary, short of
> >> storing a reference to it in the value, or using a second dictionary?
> >
> > The dictionary knows its keys and can provide them on request. Call the
> > ?dict.keys? method to get them as a collection.
>
> So it appears:
>
> $ ./pythons --command 'd={1:2, 2:4, 3:8}; print(d.keys())'
[?]
> /usr/local/cpython-3.0/bin/python good <dict_keys object at 0x8514fa4>
> /usr/local/cpython-3.1/bin/python good dict_keys([1, 2, 3])
> /usr/local/cpython-3.2/bin/python good dict_keys([1, 2, 3])
> /usr/local/cpython-3.3/bin/python good dict_keys([1, 2, 3])
> /usr/local/cpython-3.4/bin/python good dict_keys([1, 2, 3])
> /usr/local/cpython-3.5/bin/python good dict_keys([1, 2, 3])
> /usr/local/pypy3-2.4.0/bin/pypy3 good dict_keys([1, 2, 3])

Note that none of these (from Python 3) are lists; I don't promise a
list, only a collection :-)

> Would I have to do an O(n) search to find my key?

If you're wanting to *locate* a particular item, you'll need to find it
using whatever comparisons the values support. You described some custom
type; the answer will depend on what that type supports for comparing
values.

What I can say at this point is: we are no longer talking about anything
special to dictionary keys. You just have a collection of values, which
you can iterate over and compare to some target.

-- 
 \           ?The long-term solution to mountains of waste is not more |
  `\      landfill sites but fewer shopping centres.? ?Clive Hamilton, |
_o__)                                                _Affluenza_, 2005 |
Ben Finney



From python at mrabarnett.plus.com  Tue Jun 23 21:32:52 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 24 Jun 2015 02:32:52 +0100
Subject: Looking up a dictionary _key_ by key?
In-Reply-To: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
References: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
Message-ID: <558A08C4.6040301@mrabarnett.plus.com>

On 2015-06-24 01:21, Dan Stromberg wrote:
> I know that sounds strange: usually we look up values by key, not keys.
>
> But suppose you have a strange key type that despite being "equal", is
> not identical in some fields, and you need to see those fields.
>
> Is there a way of getting the key used by the dictionary, short of
> storing a reference to it in the value, or using a second dictionary?
>
Yes!

Start with a dict:

 >>> d = {1.0: 'one', 2: 'two', 3+0j: 'three'}
 >>> d[1]
'one'
 >>> d[2]
'two'
 >>> d[3]
'three'

Suppose we want the key that matches 3:

 >>> k = 3

Get the keys as a set:

 >>> s = set(s)
 >>> s
{1.0, 2, (3+0j)}

Using intersection doesn't return what we want:

 >>> s & {k}
{3}
 >>> {k} & s
{3}

so we have to get creative.

Remove the keys that _don't_ match:

 >>> s - {k}
{1.0, 2}

and then remove them from the set:

 >>> s - (s - {k})
{(3+0j)}

Finally, get the key:

 >>> (s - (s - {k})).pop()
(3+0j)

Simple, really! :-)



From cjvijf at gmail.com  Tue Jun 23 21:40:10 2015
From: cjvijf at gmail.com (L E)
Date: Tue, 23 Jun 2015 19:40:10 -0600
Subject: rhythmbox plugin problem
Message-ID: <CAHiU2s3qtAXc=rNjfm6D05djeHACeJg-Ob-Y6o2oj+txuwLMyA@mail.gmail.com>

Hello,

I am trying to get some old plugins I wrote to wrote on anewer version of
rhythmbox.

When I try to load the plugin I see:

(rhythmbox:3092): libpeas-WARNING **: nowplaying-lcd:
/usr/lib/rhythmbox/plugins/nowplaying-lcd/libnowplaying-lcd.so: cannot open
shared object file: No such file or directory

(rhythmbox:3092): libpeas-WARNING **: Could not load plugin module:
'nowplaying-lcd'


any ideas about what is going on here?

thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150623/7de3758f/attachment.html>

From steve at pearwood.info  Tue Jun 23 21:49:25 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 24 Jun 2015 11:49:25 +1000
Subject: Looking up a dictionary _key_ by key?
References: <mailman.0.1435105323.3674.python-list@python.org>
Message-ID: <558a0ca5$0$1670$c3e8da3$5496439d@news.astraweb.com>

On Wed, 24 Jun 2015 10:21 am, Dan Stromberg wrote:

> I know that sounds strange: usually we look up values by key, not keys.
> 
> But suppose you have a strange key type that despite being "equal", is
> not identical in some fields, and you need to see those fields.
> 
> Is there a way of getting the key used by the dictionary, short of
> storing a reference to it in the value, or using a second dictionary?

A nice problem from an ugly situation. The short answer is, no.

Python's dictionary lookup algorithm could be something like this:

- take the hash of the key, h;
- inspect slot h of the internal array;
- if it contains a key equal to the input key, return the value
- otherwise deal with a collision somehow


You're basically asking for step 3 to return the input key instead of the
value. Short of re-writing the dict type, no, there is no way to do so.

Without knowing your application, I can't really know what the right
approach is. One might be to ensure that only a canonical version of the
key is used in the first place, by interning it.


ALIVE = {}
def intern(key):
    return ALIVE.setdefault(key, key)


If you pass two objects that are equal but not identical, the first one will
become the canonical interned value, and the second time you will get the
first one back.

If interning doesn't suit, you can do a linear search through the keys to
find the one which matches:

for k in mydict:
    if k == key:
        assert mydict[k] is mydict[key]
        print(k, "equals but is not identical to", key)


-- 
Steven



From steve at pearwood.info  Tue Jun 23 21:50:39 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 24 Jun 2015 11:50:39 +1000
Subject: To write headers once with different values in separate row in CSV
References: <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c@googlegroups.com>
Message-ID: <558a0cef$0$1670$c3e8da3$5496439d@news.astraweb.com>

On Wed, 24 Jun 2015 03:15 am, Sahlusar wrote:

> That is not the underlying issue. Any thoughts or suggestions would be
> very helpful.


Thank you for spending over 100 lines to tell us what is NOT the underlying
issue. I will therefore tell you what is NOT the solution to your problem
(whatever it is, since I can't tell). The solution is NOT to squeeze lemon
juice into your keyboard.

If someday you feel like telling us what the issue actually IS, instead of
what it IS NOT, then perhaps we will have a chance to help you find a
solution.



-- 
Steven



From steve at pearwood.info  Tue Jun 23 22:18:28 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 24 Jun 2015 12:18:28 +1000
Subject: Organizing function calls once files have been moved to a
 directory
References: <66a66a08-a5f2-4571-819a-38427c46fcf4@googlegroups.com>
Message-ID: <558a1374$0$1638$c3e8da3$5496439d@news.astraweb.com>

On Wed, 24 Jun 2015 06:16 am, kbtyo wrote:

> I am working on a workflow module that will allow one to recursively check
> for file extensions and if there is a match move them to a folder for
> processing (parsing, data wrangling etc).
> 
> I have a simple search process, and log for the files that are present
> (see below). However, I am puzzled by what the most efficient
> method/syntax is to call functions once the selected files have been
> moved? 

The most efficient syntax is the regular syntax that you always use when
calling a file:

    function(arg, another_arg)


What else would you use?


> I have the functions and classes written in another file. Should I 
> import them or should I include them in the same file as the following
> mini-script?

That's entirely up to you. Some factors you might consider:

- Are these functions and classes reusable by other code? then you might
want to keep them separate in another file, treated as a library, and
import the library into your application.

- If you merge the two files together, will it be so big that it is
difficult to work with? Then don't merge them together. My opinion is that
the decimal module from the standard library is about as big as a single
module should every be, and it is almost 6,500 lines. So if your
application is bigger than that, you might want to split it.



> Moreover, should I create another log file for processing? If so, what is
> an idiomatically correct method to do so?

I don't know. Do you want a second log file? How will it be different from
the first?

As for creating another log file, I guess the most correct way to do so
would be the same way you created the first log file.

I'm not sure I actually understand your questions so far.

Some further comments on your code:

> if __name__ == '__main__':
> 
> # The top argument for name in files
>     topdir = '.'
>     dest = 'C:\\Users\\wynsa2\\Desktop\\'

Rather than escaping backslashes, you can use regular forward slashes:

dest = 'C:/Users/wynsa2/Desktop/'


Windows will accept either.


>     extens = ['docs', 'docx', 'pdf'] # the extensions to search for
>     found = {x: [] for x in extens} # lists of found files
>  
>     # Directories to ignore
>     ignore = ['docs', 'doc', 'py', 'pdf']
>     logname = "file_search.log"
>     print('Beginning search for files in %s' % os.path.realpath(topdir))
>   
>     # Walk the tree
>     for dirpath, dirnames, files in os.walk(topdir):
>         # Remove directories in ignore
>         # directory names must match exactly!
>         for idir in ignore:
>             if idir in dirnames:
>                 dirnames.remove(idir)
>      
>         # Loop through the file names for the current step
>         for name in files:
>      #Calling str.rsplit on name then
>     #splits the string into a list (from the right)
>     #with the first argument "."" delimiting it,
>     #and only making as many splits as the second argument (1).
>     #The third part ([-1]) retrieves the last element of the list--we
>     #use this instead of an index of 1 because if no splits are made
>     #(if there is no "."" in name), no IndexError will be raised
> 
>             ext = name.lower().rsplit('.', 1)[-1]

The better way to split the extension from the file name is to use
os.path.splitext(name):


py> import os
py> os.path.splitext("this/file.txt")
('this/file', '.txt')
py> os.path.splitext("this/file")  # no extension
('this/file', '')
py> os.path.splitext("this/file.tar.gz")
('this/file.tar', '.gz')


-- 
Steven



From steve at pearwood.info  Tue Jun 23 22:26:45 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 24 Jun 2015 12:26:45 +1000
Subject: Looking up a dictionary _key_ by key?
References: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
 <mailman.5.1435109582.3674.python-list@python.org>
Message-ID: <558a1566$0$1637$c3e8da3$5496439d@news.astraweb.com>

On Wed, 24 Jun 2015 11:32 am, MRAB wrote:

> On 2015-06-24 01:21, Dan Stromberg wrote:
>> I know that sounds strange: usually we look up values by key, not keys.
>>
>> But suppose you have a strange key type that despite being "equal", is
>> not identical in some fields, and you need to see those fields.
>>
>> Is there a way of getting the key used by the dictionary, short of
>> storing a reference to it in the value, or using a second dictionary?
>>
> Yes!
> 
> Start with a dict:
> 
>  >>> d = {1.0: 'one', 2: 'two', 3+0j: 'three'}
>  >>> d[1]
> 'one'
>  >>> d[2]
> 'two'
>  >>> d[3]
> 'three'
> 
> Suppose we want the key that matches 3:
> 
>  >>> k = 3
> 
> Get the keys as a set:
> 
>  >>> s = set(s)
>  >>> s
> {1.0, 2, (3+0j)}


What's s?

I think you mean, s = set(d), which will work. You could also use a key
view:

# Python 2
s = d.viewkeys()

# Python 3
s = d.keys()


> Using intersection doesn't return what we want:
> 
>  >>> s & {k}
> {3}
>  >>> {k} & s
> {3}
> 
> so we have to get creative.
> 
> Remove the keys that _don't_ match:
> 
>  >>> s - {k}
> {1.0, 2}
> 
> and then remove them from the set:
> 
>  >>> s - (s - {k})
> {(3+0j)}
> 
> Finally, get the key:
> 
>  >>> (s - (s - {k})).pop()
> (3+0j)
> 
> Simple, really! :-)


That's certainly creative, but I'm not sure if it will be faster than a
linear search if you have a big dict with millions of keys. Still, it's a
nice trick, and it works with views:


py> d = {1.0: 'one', 2: 'two', 3+0j: 'three'}
py> (d.viewkeys() - (d.viewkeys() - {3})).pop()
(3+0j)




-- 
Steven



From __peter__ at web.de  Wed Jun 24 02:44:00 2015
From: __peter__ at web.de (Peter Otten)
Date: Wed, 24 Jun 2015 08:44 +0200
Subject: Looking up a dictionary _key_ by key?
References: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
Message-ID: <mmdjji$kc5$1@ger.gmane.org>

Dan Stromberg wrote:

> I know that sounds strange: usually we look up values by key, not keys.
> 
> But suppose you have a strange key type that despite being "equal", is
> not identical in some fields, and you need to see those fields.
> 
> Is there a way of getting the key used by the dictionary, short of
> storing a reference to it in the value, or using a second dictionary?

$ cat grab_key.py
class GrabKey:
    def __init__(self, key):
        self.key = key

    def __eq__(self, other):
        if self.key == other:
            self.dict_key = other
            return True
        return False

    def __hash__(self):
        return hash(self.key)


def grab_key(k, d):
    g = GrabKey(k)
    d[g]
    return g.dict_key


if __name__ == "__main__":
    d = {1: int, 2.0: float}
    print(grab_key(1.0, d))
    print(grab_key(2, d))
$ python3 grab_key.py 
1
2.0




From ian.g.kelly at gmail.com  Wed Jun 24 02:50:46 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 24 Jun 2015 00:50:46 -0600
Subject: Looking up a dictionary _key_ by key?
In-Reply-To: <87a8vpykwq.fsf@nightsong.com>
References: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
 <851th2t05t.fsf@benfinney.id.au>
 <CAGGBd_qaNC38Yg0u_b8+qBAckq6FRTZMxbWwuzV=MUYE_=ZzDw@mail.gmail.com>
 <mailman.3.1435107726.3674.python-list@python.org>
 <87a8vpykwq.fsf@nightsong.com>
Message-ID: <CALwzidk9XspL0itf6A3PnsN193a=UEkAufeADWq1FcYShtwM2A@mail.gmail.com>

On Tue, Jun 23, 2015 at 7:06 PM, Paul Rubin <no.email at nospam.invalid> wrote:
> Chris Angelico <rosuav at gmail.com> writes:
>>> Would I have to do an O(n) search to find my key?
>> Iterate over it - it's an iterable view in Py3 - and compare.
>
> I think the question was whether the O(n) search could be avoided, not
> how to do it.  I don't see a way to avoid it.  There is fundamental
> brokenness in having unequal objects compare as equal, and the breakage
> messes up the dictionary when those objects are used as keys.

I don't think that it's fundamentally broken. A simple example would
be the int 3, vs. the float 3, vs. the Decimal 3. All of them compare
equal to one another, but they are distinct values, and sometimes it
might be useful to be able to determine which one is actually a key in
the dict.


From marko at pacujo.net  Wed Jun 24 02:55:51 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 24 Jun 2015 09:55:51 +0300
Subject: Looking up a dictionary _key_ by key?
References: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
 <851th2t05t.fsf@benfinney.id.au>
 <CAGGBd_qaNC38Yg0u_b8+qBAckq6FRTZMxbWwuzV=MUYE_=ZzDw@mail.gmail.com>
 <mailman.3.1435107726.3674.python-list@python.org>
 <87a8vpykwq.fsf@nightsong.com>
 <mailman.7.1435128694.3674.python-list@python.org>
Message-ID: <87a8vpmw7c.fsf@elektro.pacujo.net>

Ian Kelly <ian.g.kelly at gmail.com>:

> I don't think that it's fundamentally broken. A simple example would
> be the int 3, vs. the float 3, vs. the Decimal 3. All of them compare
> equal to one another, but they are distinct values, and sometimes it
> might be useful to be able to determine which one is actually a key in
> the dict.

One possibility is to enter the key on the value side as well:

    d[key] = (key, value)

    ...

    canonical_key, value = d[key]


Marko


From jeanpierreda at gmail.com  Wed Jun 24 03:44:02 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Wed, 24 Jun 2015 00:44:02 -0700
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmcagn$aa5$1@ger.gmane.org>
References: <mmcagn$aa5$1@ger.gmane.org>
Message-ID: <CABicbJ+5YgGzkGwFpOyFMfsteUXqMnPjS615=fU6ZCNeSyJuTw@mail.gmail.com>

How about a random substitution cipher? This will be ultra-weak, but
fast (using bytes.translate/bytes.maketrans) and seems to be the kind
of thing you're asking for.

-- Devin

On Tue, Jun 23, 2015 at 12:02 PM, Randall Smith <randall at tnr.cc> wrote:
> Chunks of data (about 2MB) are to be stored on machines using a peer-to-peer
> protocol.  The recipient of these chunks can't assume that the payload is
> benign.  While the data senders are supposed to encrypt data, that's not
> guaranteed, and I'd like to protect the recipient against exposure to
> nefarious data by mangling or encrypting the data before it is written to
> disk.
>
> My original idea was for the recipient to encrypt using AES.  But I want to
> keep this software pure Python "batteries included" and not require
> installation of other platform-dependent software.  Pure Python AES and even
> DES are just way too slow.  I don't know that I really need encryption here,
> but some type of fast mangling algorithm where a bad actor sending a payload
> can't guess the output ahead of time.
>
> Any ideas are appreciated.  Thanks.
>
> -Randall
>
> --
> https://mail.python.org/mailman/listinfo/python-list


From breamoreboy at yahoo.co.uk  Wed Jun 24 04:47:46 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 24 Jun 2015 09:47:46 +0100
Subject: Looking up a dictionary _key_ by key?
In-Reply-To: <CAGGBd_qaNC38Yg0u_b8+qBAckq6FRTZMxbWwuzV=MUYE_=ZzDw@mail.gmail.com>
References: <CAGGBd_pd9SA7SczanGARFr-+Z9SxP9Mkhfo-RivpcJLrnuexng@mail.gmail.com>
 <851th2t05t.fsf@benfinney.id.au>
 <CAGGBd_qaNC38Yg0u_b8+qBAckq6FRTZMxbWwuzV=MUYE_=ZzDw@mail.gmail.com>
Message-ID: <mmdqrl$5s5$1@ger.gmane.org>

On 24/06/2015 01:47, Dan Stromberg wrote:
>
> Would I have to do an O(n) search to find my key?
>

can you use something from here 
https://pypi.python.org/pypi/sortedcontainers/0.9.6 with the bisect module?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From maria.martin at codingame.com  Wed Jun 24 05:29:10 2015
From: maria.martin at codingame.com (Maria Martin)
Date: Wed, 24 Jun 2015 02:29:10 -0700 (PDT)
Subject: Next CodinGame online programming contest on June,
 27 -  Python available
Message-ID: <db6531a3-cdd1-4884-bbbe-14e3fb3ac836@googlegroups.com>

Hi everyone! 

On June 27th, "Code of the Rings", an online coding battle will launch. It's Free & open to all. You will have 24 hours to code and optimize your solution to a puzzle. 

What will be exciting and fun is that it will be VERY EASY to start and to get something that works, but complex to produce the most efficient code... Over the 24 hours, you will be able to submit your code as much as you like, whenever you like. No restrictions, no obligation :) 

Registration is open: http://www.codingame.com/challenge/code-of-the-rings

- Duration: 24 hours ; 1 game to solve 
- Participation is 100% online and free 
- Over 23 coding languages to choose from including Python
- Prizes to win, and 25+ t-shirts 
- Apply to sponsoring companies offering jobs and internships International Leaderboard + Leaderboard by University 

Please, feel free to share the information and your feedback are more than welcome! :) 

Hope to see you there and keep coding!


From ahlusar.ahluwalia at gmail.com  Wed Jun 24 07:31:23 2015
From: ahlusar.ahluwalia at gmail.com (kbtyo)
Date: Wed, 24 Jun 2015 04:31:23 -0700 (PDT)
Subject: To write headers once with different values in separate row in CSV
In-Reply-To: <mmcb2s$dcm$1@reader1.panix.com>
References: <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c@googlegroups.com>
 <mmcb2s$dcm$1@reader1.panix.com>
Message-ID: <3e93574e-a6b5-4ca4-8bba-c37433848f1a@googlegroups.com>

On Tuesday, June 23, 2015 at 3:12:40 PM UTC-4, John Gordon wrote:
> In <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c at googlegroups.com> Sahlusar <ahlusar.ahluwalia at gmail.com> writes:
> 
> > However, when I extrapolate this same logic with a list like:
> 
> > ('Response.MemberO.PMembers.PMembers.Member.CurrentEmployer.EmployerAddress
> > .TimeAtPreviousAddress.', None), where the headers/columns are the first
> > item (only to be written out once) with different values. I receive an
> > output CSV with repeating headers and values all printed in one long string
> 
> First, I would try to determine if the problem is in the makerows()
> function, or if the problem is elsewhere.
> 
> Have you tried creating some dummy data by hand and seeing how makerows()
> handles it?
> 
> (By the way, if your post had included some sample data that illustrates
> the problem, it would have been much easier to figure out a solution.
> Instead, we are left guessing at your XML format, and at the particular
> implementation of flatten_dict().)
> 
> -- 
> John Gordon                   A is for Amy, who fell down the stairs
> gordon at panix.com              B is for Basil, assaulted by bears
>                                 -- Edward Gorey, "The Gashlycrumb Tinies"



On Tuesday, June 23, 2015 at 3:12:40 PM UTC-4, John Gordon wrote:
> In <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c at googlegroups.com> Sahlusar <ahlusar.ahluwalia at gmail.com> writes:
> 
> > However, when I extrapolate this same logic with a list like:
> 
> > ('Response.MemberO.PMembers.PMembers.Member.CurrentEmployer.EmployerAddress
> > .TimeAtPreviousAddress.', None), where the headers/columns are the first
> > item (only to be written out once) with different values. I receive an
> > output CSV with repeating headers and values all printed in one long string
> 
> First, I would try to determine if the problem is in the makerows()
> function, or if the problem is elsewhere.
> 
> Have you tried creating some dummy data by hand and seeing how makerows()
> handles it?
>


Yes I did do this.  


> (By the way, if your post had included some sample data that illustrates
> the problem, it would have been much easier to figure out a solution.
> Instead, we are left guessing at your XML format, and at the particular
> implementation of flatten_dict().)

Yes, unfortunately, due to NDA protocols I cannot share this. 
> 
> -- 
> John Gordon                   A is for Amy, who fell down the stairs
> gordon at panix.com              B is for Basil, assaulted by bears
>                                 -- Edward Gorey, "The Gashlycrumb Tinies"



From steve at pearwood.info  Wed Jun 24 07:36:41 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 24 Jun 2015 21:36:41 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mailman.10.1435130890.3674.python-list@python.org>
Message-ID: <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>

On Wed, 24 Jun 2015 05:02 am, Randall Smith wrote:

> Chunks of data (about 2MB) are to be stored on machines using a
> peer-to-peer protocol.  The recipient of these chunks can't assume that
> the payload is benign.  While the data senders are supposed to encrypt
> data, that's not guaranteed, and I'd like to protect the recipient
> against exposure to nefarious data by mangling or encrypting the data
> before it is written to disk.

I don't understand how mangling the data is supposed to protect the
recipient. Don't they have the ability unmangle the data, and thus expose
themselves to whatever nasties are in the files?

If not, you can save all that time and effort implementing the peer-to-peer
business and just dump 2MB chunks of random data on their disks.


> My original idea was for the recipient to encrypt using AES.  But I want
> to keep this software pure Python "batteries included" and not require
> installation of other platform-dependent software.  Pure Python AES and
> even DES are just way too slow.  I don't know that I really need
> encryption here, but some type of fast mangling algorithm where a bad
> actor sending a payload can't guess the output ahead of time.

Again, I don't understand your threat model here. Why does the bad actor
need to guess the mangling? Putting on my Black Hat and twirling my
moustache wickedly, I decide to send you a JPG of Goatse. (Don't google
it.) Or, a more serious threat, a zip bomb:

http://www.ghacks.net/2008/07/27/42-kilobytes-unzipped-make-45-petabytes/

or malware of some description. So I P2P you the file. How it gets encrypted
on your disk is irrelevant to me, eventually you're going to unencrypted it
and try to access it.

We need to understand what threat you are defending against before we can
advise you.



-- 
Steven



From ahlusar.ahluwalia at gmail.com  Wed Jun 24 07:37:01 2015
From: ahlusar.ahluwalia at gmail.com (kbtyo)
Date: Wed, 24 Jun 2015 04:37:01 -0700 (PDT)
Subject: To write headers once with different values in separate row in CSV
In-Reply-To: <558a0cef$0$1670$c3e8da3$5496439d@news.astraweb.com>
References: <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c@googlegroups.com>
 <558a0cef$0$1670$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <22c60270-d8e8-43eb-b58b-b6d86dc6732e@googlegroups.com>

On Tuesday, June 23, 2015 at 9:50:50 PM UTC-4, Steven D'Aprano wrote:
> On Wed, 24 Jun 2015 03:15 am, Sahlusar wrote:
> 
> > That is not the underlying issue. Any thoughts or suggestions would be
> > very helpful.
> 
> 
> Thank you for spending over 100 lines to tell us what is NOT the underlying
> issue. I will therefore tell you what is NOT the solution to your problem
> (whatever it is, since I can't tell). The solution is NOT to squeeze lemon
> juice into your keyboard.
> 
> If someday you feel like telling us what the issue actually IS, instead of
> what it IS NOT, then perhaps we will have a chance to help you find a
> solution.
> 
> 
> 
> -- 
> Steven

Curious - what should I have provided? Detailed and constructive feedback (like your reply to my post regarding importing functions) is more useful than to "squeeze lemon juice" into one's keyboard. 


From ahlusar.ahluwalia at gmail.com  Wed Jun 24 07:59:26 2015
From: ahlusar.ahluwalia at gmail.com (kbtyo)
Date: Wed, 24 Jun 2015 04:59:26 -0700 (PDT)
Subject: Organizing function calls once files have been moved to a
 directory
In-Reply-To: <558a1374$0$1638$c3e8da3$5496439d@news.astraweb.com>
References: <66a66a08-a5f2-4571-819a-38427c46fcf4@googlegroups.com>
 <558a1374$0$1638$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5918d05d-bb0e-4c50-b0c6-f59d9ff3cfbd@googlegroups.com>

On Tuesday, June 23, 2015 at 10:18:43 PM UTC-4, Steven D'Aprano wrote:
> On Wed, 24 Jun 2015 06:16 am, kbtyo wrote:
> 
> > I am working on a workflow module that will allow one to recursively check
> > for file extensions and if there is a match move them to a folder for
> > processing (parsing, data wrangling etc).
> > 
> > I have a simple search process, and log for the files that are present
> > (see below). However, I am puzzled by what the most efficient
> > method/syntax is to call functions once the selected files have been
> > moved? 
> 
> The most efficient syntax is the regular syntax that you always use when
> calling a file:
> 
>     function(arg, another_arg)
> 
> 
> What else would you use?
> 
> 
> > I have the functions and classes written in another file. Should I 
> > import them or should I include them in the same file as the following
> > mini-script?
> 
> That's entirely up to you. Some factors you might consider:
> 
> - Are these functions and classes reusable by other code? then you might
> want to keep them separate in another file, treated as a library, and
> import the library into your application.
> 
> - If you merge the two files together, will it be so big that it is
> difficult to work with? Then don't merge them together. My opinion is that
> the decimal module from the standard library is about as big as a single
> module should every be, and it is almost 6,500 lines. So if your
> application is bigger than that, you might want to split it.
> 
> 
> 
> > Moreover, should I create another log file for processing? If so, what is
> > an idiomatically correct method to do so?
> 
> I don't know. Do you want a second log file? How will it be different from
> the first?
> 
> As for creating another log file, I guess the most correct way to do so
> would be the same way you created the first log file.
> 
> I'm not sure I actually understand your questions so far.
> 
> Some further comments on your code:
> 
> > if __name__ == '__main__':
> > 
> > # The top argument for name in files
> >     topdir = '.'
> >     dest = 'C:\\Users\\wynsa2\\Desktop\\'
> 
> Rather than escaping backslashes, you can use regular forward slashes:
> 
> dest = 'C:/Users/wynsa2/Desktop/'
> 
> 
> Windows will accept either.
> 
> 
> >     extens = ['docs', 'docx', 'pdf'] # the extensions to search for
> >     found = {x: [] for x in extens} # lists of found files
> >  
> >     # Directories to ignore
> >     ignore = ['docs', 'doc', 'py', 'pdf']
> >     logname = "file_search.log"
> >     print('Beginning search for files in %s' % os.path.realpath(topdir))
> >   
> >     # Walk the tree
> >     for dirpath, dirnames, files in os.walk(topdir):
> >         # Remove directories in ignore
> >         # directory names must match exactly!
> >         for idir in ignore:
> >             if idir in dirnames:
> >                 dirnames.remove(idir)
> >      
> >         # Loop through the file names for the current step
> >         for name in files:
> >      #Calling str.rsplit on name then
> >     #splits the string into a list (from the right)
> >     #with the first argument "."" delimiting it,
> >     #and only making as many splits as the second argument (1).
> >     #The third part ([-1]) retrieves the last element of the list--we
> >     #use this instead of an index of 1 because if no splits are made
> >     #(if there is no "."" in name), no IndexError will be raised
> > 
> >             ext = name.lower().rsplit('.', 1)[-1]
> 
> The better way to split the extension from the file name is to use
> os.path.splitext(name):
> 
> 
> py> import os
> py> os.path.splitext("this/file.txt")
> ('this/file', '.txt')
> py> os.path.splitext("this/file")  # no extension
> ('this/file', '')
> py> os.path.splitext("this/file.tar.gz")
> ('this/file.tar', '.gz')
> 
> 
> -- 
> Steven



On Tuesday, June 23, 2015 at 10:18:43 PM UTC-4, Steven D'Aprano wrote:
> On Wed, 24 Jun 2015 06:16 am, kbtyo wrote:
> 
> > I am working on a workflow module that will allow one to recursively check
> > for file extensions and if there is a match move them to a folder for
> > processing (parsing, data wrangling etc).
> > 
> > I have a simple search process, and log for the files that are present
> > (see below). However, I am puzzled by what the most efficient
> > method/syntax is to call functions once the selected files have been
> > moved? 
> 
> The most efficient syntax is the regular syntax that you always use when
> calling a file:
> 
>     function(arg, another_arg)
> 
> 
> What else would you use?
> 
> 
> > I have the functions and classes written in another file. Should I 
> > import them or should I include them in the same file as the following
> > mini-script?
> 
> That's entirely up to you. Some factors you might consider:
> 
> - Are these functions and classes reusable by other code? then you might
> want to keep them separate in another file, treated as a library, and
> import the library into your application.

I think I will do just that.  
> 
> - If you merge the two files together, will it be so big that it is
> difficult to work with? Then don't merge them together. My opinion is that
> the decimal module from the standard library is about as big as a single
> module should every be, and it is almost 6,500 lines. So if your
> application is bigger than that, you might want to split it.
> 
 Yes, I think that is the best course of action. 
> 
> > Moreover, should I create another log file for processing? If so, what is
> > an idiomatically correct method to do so?
> 
> I don't know. Do you want a second log file? How will it be different from
> the first?

The second log file would provide information to the user on the files that:

1. Have been processed (specifically - parsed from XML to CSV (3 functions conduct this)

2. Files that did not pass whichever function 

3. Files that have malformed XML (this is particularly essential)


> As for creating another log file, I guess the most correct way to do so
> would be the same way you created the first log file.
> 

This is a little difficult, for me to conceptualize, as I am wondering if the above 1,2,3 should have separate functions for each type of error that ties to the specific functions that parse XML and write to CSV, respectively or should be a specific second error log. 

I am wondering how I should organize this such that I can "automate" this process.

FYI The first error log is simply this:

def log_results(logpath, results):
	# The header in our logfile
	loghead = 'Search log from filefind for files in {}\n\n'.format(
	              os.path.realpath(topdir)
	          )
	# The body of our log file
	logbody = ''
	 
	# loop thru results
	for searchResult in results:
	    # Concatenate the result from the found dict
	    logbody += "<< Results with the extension '%s' >>" % searchResult
	    logbody += '\n\n%s\n\n' % '\n'.join(found[searchResult])
	 
	# Write results to the logfile
	with open(logpath, 'w') as logfile:
	    logfile.write(loghead)
	    logfile.write(logbody)

Calling it can be instantiated via the below (which you already commented upon - thank you for the feedback). 

> I'm not sure I actually understand your questions so far.
> 
> Some further comments on your code:
> 
> > if __name__ == '__main__':
> > 
> > # The top argument for name in files
> >     topdir = '.'
> >     dest = 'C:\\Users\\wynsa2\\Desktop\\'
> 
> Rather than escaping backslashes, you can use regular forward slashes:
> 
> dest = 'C:/Users/wynsa2/Desktop/'
> 
> 
> Windows will accept either.

Ah - good to know!
> 
> 
> >     extens = ['docs', 'docx', 'pdf'] # the extensions to search for
> >     found = {x: [] for x in extens} # lists of found files
> >  
> >     # Directories to ignore
> >     ignore = ['docs', 'doc', 'py', 'pdf']
> >     logname = "file_search.log"
> >     print('Beginning search for files in %s' % os.path.realpath(topdir))
> >   
> >     # Walk the tree
> >     for dirpath, dirnames, files in os.walk(topdir):
> >         # Remove directories in ignore
> >         # directory names must match exactly!
> >         for idir in ignore:
> >             if idir in dirnames:
> >                 dirnames.remove(idir)
> >      
> >         # Loop through the file names for the current step
> >         for name in files:
> >      #Calling str.rsplit on name then
> >     #splits the string into a list (from the right)
> >     #with the first argument "."" delimiting it,
> >     #and only making as many splits as the second argument (1).
> >     #The third part ([-1]) retrieves the last element of the list--we
> >     #use this instead of an index of 1 because if no splits are made
> >     #(if there is no "."" in name), no IndexError will be raised
> > 
> >             ext = name.lower().rsplit('.', 1)[-1]
> 
> The better way to split the extension from the file name is to use
> os.path.splitext(name):
> 
> 
> py> import os
> py> os.path.splitext("this/file.txt")
> ('this/file', '.txt')
> py> os.path.splitext("this/file")  # no extension
> ('this/file', '')
> py> os.path.splitext("this/file.tar.gz")
> ('this/file.tar', '.gz')
> 
> 
> -- 
> Steven

Thank you for your feedback on utilizing the OS module more efficiently.


From steve at pearwood.info  Wed Jun 24 08:38:10 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 24 Jun 2015 22:38:10 +1000
Subject: To write headers once with different values in separate row in CSV
References: <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c@googlegroups.com>
 <558a0cef$0$1670$c3e8da3$5496439d@news.astraweb.com>
 <22c60270-d8e8-43eb-b58b-b6d86dc6732e@googlegroups.com>
Message-ID: <558aa4b1$0$1671$c3e8da3$5496439d@news.astraweb.com>

On Wed, 24 Jun 2015 09:37 pm, kbtyo wrote:

> On Tuesday, June 23, 2015 at 9:50:50 PM UTC-4, Steven D'Aprano wrote:
>> On Wed, 24 Jun 2015 03:15 am, Sahlusar wrote:
>> 
>> > That is not the underlying issue. Any thoughts or suggestions would be
>> > very helpful.
>> 
>> 
>> Thank you for spending over 100 lines to tell us what is NOT the
>> underlying issue. I will therefore tell you what is NOT the solution to
>> your problem (whatever it is, since I can't tell). The solution is NOT to
>> squeeze lemon juice into your keyboard.
>> 
>> If someday you feel like telling us what the issue actually IS, instead
>> of what it IS NOT, then perhaps we will have a chance to help you find a
>> solution.
>> 
>> 
>> 
>> --
>> Steven
> 
> Curious - what should I have provided? 

To start with, you should tell us what is the problem you are having. You
gave us some code, and then said "That is not the underlying issue". Okay,
so what is the underlying issue? What is the problem you want help solving?

In another post, you responded to John Gordon's question:

    # John
    Have you tried creating some dummy data by hand and seeing 
    how makerows() handles it?


by answering:

    Yes I did do this.


Okay. What was the result? Do you want us to guess what result you got?


John also suggested that you provide sample data, and an implementation of
flatten_dict, and your answer is:

    Yes, unfortunately, due to NDA protocols I cannot share this.


You don't have to provide your *actual* data. You can provide *sample* data,
that does not contain any of your actual confidential values. If your XML
file looks like this:

<?xml version="1.0"?>
<catalog>
   <book id="bk101">
      <author>Gambardella, Matthew</author>
      <title>XML Developer's Guide</title>
      <genre>Computer</genre>
      <price>44.95</price>
      <publish_date>2000-10-01</publish_date>
      <description>An in-depth look at creating applications 
      with XML.</description>
   </book>
</catalog>


you can replace the data:

<?xml version="1.0"?>
<catalog>
   <book id="1111">
      <author>Smith, John</author>
      <title>ABCDEF</title>
      <genre>Widgets</genre>
      <price>9999.99</price>
      <publish_date>1900-01-01</publish_date>
      <description>blah blah blah blah</description>
   </book>
</catalog>


You can even change the tags:


<?xml version="1.0"?>
<whatzit>
   <spam id="1111">
      <a>Smith, John</a>
      <b>ABCDEF</b>
      <c>Widgets</c>
      <d>9999.99</d>
      <e>1900-01-01</e>
      <f>blah blah blah blah</f>
   </spam>
</whatzit>


If you're still worried that the sample XML has the same structure as your
real data, you can remove some fields and add new ones:

<?xml version="1.0"?>
<whatzit>
   <spam id="1111">
      <b>ABCDEF</b>
      <d>9999.99</d>
      <e>1900-01-01</e>
      <z>fe fi fo fum</z>
      <f>blah blah blah blah</f>
   </spam>
</whatzit>


If you can't share the flatten_dict() function, either: 

(1) get permission to share it from your manager or project leader.
flatten_dict is not a trade secret or valuable in any way, and
half-competent Python programmer can probably come up with two or three
different ways to flatten a dict in five minutes. They're all going to look
more or less the same, because there's only so many ways to flatten a dict.

(2) Or accept that we can't help you, and deal with it on your own.



> Detailed and constructive feedback 
> (like your reply to my post regarding importing functions) is more useful
> than to "squeeze lemon juice" into one's keyboard.

Of course. That is why I said it was NOT the solution. Don't waste your time
squeezing lemon juice over your keyboard, it won't solve your problem.

But you can't expect us to guess what your problem is, or debug code we
can't see, or read your mind and understand your data.

Before you ask any more questions, please read this:

http://sscce.org/



-- 
Steven



From ahlusar.ahluwalia at gmail.com  Wed Jun 24 08:52:58 2015
From: ahlusar.ahluwalia at gmail.com (kbtyo)
Date: Wed, 24 Jun 2015 05:52:58 -0700 (PDT)
Subject: To write headers once with different values in separate row in CSV
In-Reply-To: <558aa4b1$0$1671$c3e8da3$5496439d@news.astraweb.com>
References: <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c@googlegroups.com>
 <558a0cef$0$1670$c3e8da3$5496439d@news.astraweb.com>
 <22c60270-d8e8-43eb-b58b-b6d86dc6732e@googlegroups.com>
 <558aa4b1$0$1671$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <a0bf011e-dbae-4787-9525-ebad6e42fbf8@googlegroups.com>

On Wednesday, June 24, 2015 at 8:38:24 AM UTC-4, Steven D'Aprano wrote:
> On Wed, 24 Jun 2015 09:37 pm, kbtyo wrote:
> 
> > On Tuesday, June 23, 2015 at 9:50:50 PM UTC-4, Steven D'Aprano wrote:
> >> On Wed, 24 Jun 2015 03:15 am, Sahlusar wrote:
> >> 
> >> > That is not the underlying issue. Any thoughts or suggestions would be
> >> > very helpful.
> >> 
> >> 
> >> Thank you for spending over 100 lines to tell us what is NOT the
> >> underlying issue. I will therefore tell you what is NOT the solution to
> >> your problem (whatever it is, since I can't tell). The solution is NOT to
> >> squeeze lemon juice into your keyboard.
> >> 
> >> If someday you feel like telling us what the issue actually IS, instead
> >> of what it IS NOT, then perhaps we will have a chance to help you find a
> >> solution.
> >> 
> >> 
> >> 
> >> --
> >> Steven
> > 
> > Curious - what should I have provided? 
> 
> To start with, you should tell us what is the problem you are having. You
> gave us some code, and then said "That is not the underlying issue". Okay,
> so what is the underlying issue? What is the problem you want help solving?
> 
> In another post, you responded to John Gordon's question:
> 
>     # John
>     Have you tried creating some dummy data by hand and seeing 
>     how makerows() handles it?
> 
> 
> by answering:
> 
>     Yes I did do this.
> 
> 
> Okay. What was the result? Do you want us to guess what result you got?
> 
> 
> John also suggested that you provide sample data, and an implementation of
> flatten_dict, and your answer is:
> 
>     Yes, unfortunately, due to NDA protocols I cannot share this.
> 
> 
> You don't have to provide your *actual* data. You can provide *sample* data,
> that does not contain any of your actual confidential values. If your XML
> file looks like this:
> 
> <?xml version="1.0"?>
> <catalog>
>    <book id="bk101">
>       <author>Gambardella, Matthew</author>
>       <title>XML Developer's Guide</title>
>       <genre>Computer</genre>
>       <price>44.95</price>
>       <publish_date>2000-10-01</publish_date>
>       <description>An in-depth look at creating applications 
>       with XML.</description>
>    </book>
> </catalog>
> 
> 
> you can replace the data:
> 
> <?xml version="1.0"?>
> <catalog>
>    <book id="1111">
>       <author>Smith, John</author>
>       <title>ABCDEF</title>
>       <genre>Widgets</genre>
>       <price>9999.99</price>
>       <publish_date>1900-01-01</publish_date>
>       <description>blah blah blah blah</description>
>    </book>
> </catalog>
> 
> 
> You can even change the tags:
> 
> 
> <?xml version="1.0"?>
> <whatzit>
>    <spam id="1111">
>       <a>Smith, John</a>
>       <b>ABCDEF</b>
>       <c>Widgets</c>
>       <d>9999.99</d>
>       <e>1900-01-01</e>
>       <f>blah blah blah blah</f>
>    </spam>
> </whatzit>
> 
> 
> If you're still worried that the sample XML has the same structure as your
> real data, you can remove some fields and add new ones:
> 
> <?xml version="1.0"?>
> <whatzit>
>    <spam id="1111">
>       <b>ABCDEF</b>
>       <d>9999.99</d>
>       <e>1900-01-01</e>
>       <z>fe fi fo fum</z>
>       <f>blah blah blah blah</f>
>    </spam>
> </whatzit>
> 
> 
> If you can't share the flatten_dict() function, either: 
> 
> (1) get permission to share it from your manager or project leader.
> flatten_dict is not a trade secret or valuable in any way, and
> half-competent Python programmer can probably come up with two or three
> different ways to flatten a dict in five minutes. They're all going to look
> more or less the same, because there's only so many ways to flatten a dict.
> 
> (2) Or accept that we can't help you, and deal with it on your own.
> 
> 
> 
> > Detailed and constructive feedback 
> > (like your reply to my post regarding importing functions) is more useful
> > than to "squeeze lemon juice" into one's keyboard.
> 
> Of course. That is why I said it was NOT the solution. Don't waste your time
> squeezing lemon juice over your keyboard, it won't solve your problem.
> 
> But you can't expect us to guess what your problem is, or debug code we
> can't see, or read your mind and understand your data.
> 
> Before you ask any more questions, please read this:
> 
> http://sscce.org/
> 
> 
> 
> -- 
> Steven



On Wednesday, June 24, 2015 at 8:38:24 AM UTC-4, Steven D'Aprano wrote:
> On Wed, 24 Jun 2015 09:37 pm, kbtyo wrote:
> 
> > On Tuesday, June 23, 2015 at 9:50:50 PM UTC-4, Steven D'Aprano wrote:
> >> On Wed, 24 Jun 2015 03:15 am, Sahlusar wrote:
> >> 
> >> > That is not the underlying issue. Any thoughts or suggestions would be
> >> > very helpful.
> >> 
> >> 
> >> Thank you for spending over 100 lines to tell us what is NOT the
> >> underlying issue. I will therefore tell you what is NOT the solution to
> >> your problem (whatever it is, since I can't tell). The solution is NOT to
> >> squeeze lemon juice into your keyboard.
> >> 
> >> If someday you feel like telling us what the issue actually IS, instead
> >> of what it IS NOT, then perhaps we will have a chance to help you find a
> >> solution.
> >> 
> >> 
> >> 
> >> --
> >> Steven
> > 
> > Curious - what should I have provided? 
> 
> To start with, you should tell us what is the problem you are having. You
> gave us some code, and then said "That is not the underlying issue". Okay,
> so what is the underlying issue? What is the problem you want help solving?
> 
> In another post, you responded to John Gordon's question:
> 
>     # John
>     Have you tried creating some dummy data by hand and seeing 
>     how makerows() handles it?
> 
> 
> by answering:
> 
>     Yes I did do this.
> 
> 
> Okay. What was the result? Do you want us to guess what result you got?
> 
> 
> John also suggested that you provide sample data, and an implementation of
> flatten_dict, and your answer is:
> 
>     Yes, unfortunately, due to NDA protocols I cannot share this.
> 
> 
> You don't have to provide your *actual* data. You can provide *sample* data,
> that does not contain any of your actual confidential values. If your XML
> file looks like this:
> 
> <?xml version="1.0"?>
> <catalog>
>    <book id="bk101">
>       <author>Gambardella, Matthew</author>
>       <title>XML Developer's Guide</title>
>       <genre>Computer</genre>
>       <price>44.95</price>
>       <publish_date>2000-10-01</publish_date>
>       <description>An in-depth look at creating applications 
>       with XML.</description>
>    </book>
> </catalog>
> 
> 
> you can replace the data:
> 
> <?xml version="1.0"?>
> <catalog>
>    <book id="1111">
>       <author>Smith, John</author>
>       <title>ABCDEF</title>
>       <genre>Widgets</genre>
>       <price>9999.99</price>
>       <publish_date>1900-01-01</publish_date>
>       <description>blah blah blah blah</description>
>    </book>
> </catalog>
> 
> 
> You can even change the tags:
> 
> 
> <?xml version="1.0"?>
> <whatzit>
>    <spam id="1111">
>       <a>Smith, John</a>
>       <b>ABCDEF</b>
>       <c>Widgets</c>
>       <d>9999.99</d>
>       <e>1900-01-01</e>
>       <f>blah blah blah blah</f>
>    </spam>
> </whatzit>
> 
> 
> If you're still worried that the sample XML has the same structure as your
> real data, you can remove some fields and add new ones:
> 
> <?xml version="1.0"?>
> <whatzit>
>    <spam id="1111">
>       <b>ABCDEF</b>
>       <d>9999.99</d>
>       <e>1900-01-01</e>
>       <z>fe fi fo fum</z>
>       <f>blah blah blah blah</f>
>    </spam>
> </whatzit>
> 
> 
> If you can't share the flatten_dict() function, either: 
> 
> (1) get permission to share it from your manager or project leader.
> flatten_dict is not a trade secret or valuable in any way, and
> half-competent Python programmer can probably come up with two or three
> different ways to flatten a dict in five minutes. They're all going to look
> more or less the same, because there's only so many ways to flatten a dict.
> 
> (2) Or accept that we can't help you, and deal with it on your own.
> 
> 
> 
> > Detailed and constructive feedback 
> > (like your reply to my post regarding importing functions) is more useful
> > than to "squeeze lemon juice" into one's keyboard.
> 
> Of course. That is why I said it was NOT the solution. Don't waste your time
> squeezing lemon juice over your keyboard, it won't solve your problem.
> 
> But you can't expect us to guess what your problem is, or debug code we
> can't see, or read your mind and understand your data.
> 
> Before you ask any more questions, please read this:
> 
> http://sscce.org/
> 
> 
> 
> -- 
> Steven

Thanks for the feedback. To be quite honest with you, I have just started a new position as a data engineer after switching careers from secondary education. This feedback will help me adapt to this community. Thank you again for your continued support. 


From invalid at invalid.invalid  Wed Jun 24 10:02:08 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Wed, 24 Jun 2015 14:02:08 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mmed90$nho$1@reader1.panix.com>

On 2015-06-24, Steven D'Aprano <steve at pearwood.info> wrote:
> On Wed, 24 Jun 2015 05:02 am, Randall Smith wrote:
>
>> Chunks of data (about 2MB) are to be stored on machines using a
>> peer-to-peer protocol.  The recipient of these chunks can't assume that
>> the payload is benign.  While the data senders are supposed to encrypt
>> data, that's not guaranteed, and I'd like to protect the recipient
>> against exposure to nefarious data by mangling or encrypting the data
>> before it is written to disk.
>
> I don't understand how mangling the data is supposed to protect the
> recipient. Don't they have the ability unmangle the data, and thus expose
> themselves to whatever nasties are in the files?

And how does writing unmangled data to disk expose anybody to
anything?  I've never heard of an exploit where writing an evilly
crafted bit-pattern to disk causes a any sort of problem.

-- 
Grant Edwards               grant.b.edwards        Yow! My mind is making
                                  at               ashtrays in Dayton ...
                              gmail.com            


From mal at europython.eu  Wed Jun 24 10:06:37 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Wed, 24 Jun 2015 16:06:37 +0200
Subject: =?UTF-8?Q?EuroPython_2015:_Beginner=e2=80=99s_Day?=
Message-ID: <558AB96D.8080805@europython.eu>

We?re pleased to announce a new venture at this year?s EuroPython...

                *** The EuroPython Beginner?s Day ***

         https://ep2015.europython.eu/en/events/beginners-day/

If you?re thinking of coming to the conference but you?re new to
Python, this could be the session for you. Whether you?re totally new
to programming or you already know another language, this day is to
give you a crash-course in Python, and the ecosystem around it, to
give you the context you need to get the most out of EuroPython.

Bring your laptop, as a large part of the day will be devoted to
learning Python on your own PC. This session will take place on the
first day of the conference, the Monday. It will be presented in
English (although a few of the coaches do speak basic Spanish, French
and Italian).

Sessions will include:

 * A high-level introduction to Python and programming in general.
   Where did Python come from, what is programming all about, and what
   do I need to know to understand all these in-jokes about cheese
   shops?

 * A self-directed learning session, with specific tutorials for total
   beginners and more experienced programmers, accompanied by coaches
   who will be there to answer your questions and help you when you
   get stuck.  Learn at your own pace!

 * A session on the Python ?ecosystem?  An introduction to the Python
   ecosystem: some topics and bits of jargon that are bound to come up
   this week: open source, free software, github, packages, pip, pypi,
   scientific computing, scipy, numpy, pandas, ipython notebook, web
   frameworks, django, flask, asyncio, the BDFL, the Zen of Python,
   etc etc.  What are the tools, areas of interest, in-jokes, people
   of note.

 * ?How to get the best out of the conference? - recommended talks,
   what to do at lunchtimes or in the evenings, tips on when and how
   to ask questions (hint: as often as possible!), what an ?open
   space? is, and more.

We really need to get an idea of numbers for this session, so if you
are interested in attending, please drop a quick email to Harry
Percival <obeythetestinggoat at gmail.com> from the program work group.

Also, be sure to get your tickets in time, since ticket sales have
picked up a lot since we announced the schedule.

PS: We?re also looking for volunteers to help with coaching students
during this session. If you enjoy teaching Python to beginners, and
you don?t mind sacrificing your EuroPython Monday to it, please do get
in touch with Harry Percival <obeythetestinggoat at gmail.com> !

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From rosuav at gmail.com  Wed Jun 24 10:33:09 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 25 Jun 2015 00:33:09 +1000
Subject: =?UTF-8?Q?Re=3A_EuroPython_2015=3A_Beginner=E2=80=99s_Day?=
In-Reply-To: <558AB96D.8080805@europython.eu>
References: <558AB96D.8080805@europython.eu>
Message-ID: <CAPTjJmrSWDKNkcw-sZdKh9Y=U3u8zssdj3j2gvaaxX39YKdQZA@mail.gmail.com>

On Thu, Jun 25, 2015 at 12:06 AM, M.-A. Lemburg <mal at europython.eu> wrote:
> * A high-level introduction to Python and programming in general.
>    Where did Python come from, what is programming all about, and what
>    do I need to know to understand all these in-jokes about cheese
>    shops?
>
>  * A session on the Python ?ecosystem?  An introduction to the Python
>    ecosystem: some topics and bits of jargon that are bound to come up
>    this week: open source, free software, github, packages, pip, pypi,
>    scientific computing, scipy, numpy, pandas, ipython notebook, web
>    frameworks, django, flask, asyncio, the BDFL, the Zen of Python,
>    etc etc.  What are the tools, areas of interest, in-jokes, people
>    of note.

Heh, I wish this had been available when I started digging into
Python. Took me ages to figure out some of the in-jokes....

ChrisA


From emile at fenx.com  Wed Jun 24 11:52:25 2015
From: emile at fenx.com (Emile van Sebille)
Date: Wed, 24 Jun 2015 08:52:25 -0700
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmed90$nho$1@reader1.panix.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mmed90$nho$1@reader1.panix.com>
Message-ID: <mmejnp$i0p$1@ger.gmane.org>

On 6/24/2015 7:02 AM, Grant Edwards wrote:
> And how does writing unmangled data to disk expose anybody to
> anything?  I've never heard of an exploit where writing an evilly
> crafted bit-pattern to disk causes a any sort of problem.

Unless that code is executed at boot.  Mangling would at least prevent 
it from executing.

Emile






From rosuav at gmail.com  Wed Jun 24 11:55:00 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 25 Jun 2015 01:55:00 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmejnp$i0p$1@ger.gmane.org>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mmed90$nho$1@reader1.panix.com> <mmejnp$i0p$1@ger.gmane.org>
Message-ID: <CAPTjJmqe9gX8qFyS-DRpcW+Dpn7FB0Gbb6jZ+pTSOTfXLi8Exw@mail.gmail.com>

On Thu, Jun 25, 2015 at 1:52 AM, Emile van Sebille <emile at fenx.com> wrote:
> On 6/24/2015 7:02 AM, Grant Edwards wrote:
>>
>> And how does writing unmangled data to disk expose anybody to
>> anything?  I've never heard of an exploit where writing an evilly
>> crafted bit-pattern to disk causes a any sort of problem.
>
>
> Unless that code is executed at boot.  Mangling would at least prevent it
> from executing.

Or it's on Windows. It's pretty easy to trick Windows into running
some code somewhere. But you can often disrupt that by simply renaming
the file to have no extension.

ChrisA


From teja_kaipa at yahoo.com  Wed Jun 24 11:56:24 2015
From: teja_kaipa at yahoo.com (Knss Teja)
Date: Wed, 24 Jun 2015 08:56:24 -0700
Subject: No subject
Message-ID: <1435161384.7311.YahooMailWin8@web141002.mail.bf1.yahoo.com>

I WANT TO install 4.3? version ... but the MSI file is giving a DLL error .. what should I do :/
please use REPLY ALL .. so that I get the mail to my?gmail inbox
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150624/0c76dad2/attachment.html>

From emile at fenx.com  Wed Jun 24 12:09:29 2015
From: emile at fenx.com (Emile van Sebille)
Date: Wed, 24 Jun 2015 09:09:29 -0700
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CAPTjJmqe9gX8qFyS-DRpcW+Dpn7FB0Gbb6jZ+pTSOTfXLi8Exw@mail.gmail.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mmed90$nho$1@reader1.panix.com> <mmejnp$i0p$1@ger.gmane.org>
 <CAPTjJmqe9gX8qFyS-DRpcW+Dpn7FB0Gbb6jZ+pTSOTfXLi8Exw@mail.gmail.com>
Message-ID: <mmeknq$805$1@ger.gmane.org>

On 6/24/2015 8:55 AM, Chris Angelico wrote:
> On Thu, Jun 25, 2015 at 1:52 AM, Emile van Sebille <emile at fenx.com> wrote:
>> On 6/24/2015 7:02 AM, Grant Edwards wrote:
>>>
>>> And how does writing unmangled data to disk expose anybody to
>>> anything?  I've never heard of an exploit where writing an evilly
>>> crafted bit-pattern to disk causes a any sort of problem.
>>
>>
>> Unless that code is executed at boot.  Mangling would at least prevent it
>> from executing.
>
> Or it's on Windows. It's pretty easy to trick Windows into running
> some code somewhere. But you can often disrupt that by simply renaming
> the file to have no extension.

ISTR that windows may look into the file to see if it can 'guess' the 
appropriate application, so dropping the extension may not be 
sufficient.  But maybe they've changed that as my windows experience 
doesn't run much past XP.

Emile





From invalid at invalid.invalid  Wed Jun 24 12:16:37 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Wed, 24 Jun 2015 16:16:37 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mmed90$nho$1@reader1.panix.com>
 <mailman.18.1435161161.3674.python-list@python.org>
Message-ID: <mmel55$ojb$1@reader1.panix.com>

On 2015-06-24, Emile van Sebille <emile at fenx.com> wrote:
> On 6/24/2015 7:02 AM, Grant Edwards wrote:
>> And how does writing unmangled data to disk expose anybody to
>> anything?  I've never heard of an exploit where writing an evilly
>> crafted bit-pattern to disk causes a any sort of problem.
>
> Unless that code is executed at boot.

Don't write it somewhere where that might happen.  [Of course you
don't let a remote user determine where the untrusted data gets
written -- that would be completely beyond the pale.] Or does Windows
pick files at random from the disk and execute them?

> Mangling would at least prevent it from executing.

If you don't want a file to be executed, then don't make it
executable.  Or doesn't Windows have any way to control whether a file
is executable or not?

-- 
Grant Edwards               grant.b.edwards        Yow! You were s'posed
                                  at               to laugh!
                              gmail.com            


From rosuav at gmail.com  Wed Jun 24 12:23:32 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 25 Jun 2015 02:23:32 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmel55$ojb$1@reader1.panix.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mmed90$nho$1@reader1.panix.com>
 <mailman.18.1435161161.3674.python-list@python.org>
 <mmel55$ojb$1@reader1.panix.com>
Message-ID: <CAPTjJmpZS1xS-JtG9SUsbBfnxhLyVUxMewWdnZeROAWzPS---A@mail.gmail.com>

On Thu, Jun 25, 2015 at 2:16 AM, Grant Edwards <invalid at invalid.invalid> wrote:
> On 2015-06-24, Emile van Sebille <emile at fenx.com> wrote:
>> Mangling would at least prevent it from executing.
>
> If you don't want a file to be executed, then don't make it
> executable.  Or doesn't Windows have any way to control whether a file
> is executable or not?

Windows doesn't have the Unix file system concept of execute
permission, no. If a file has the .exe extension and the first 512
bytes look like an appropriate header (MZ etc), Windows will happily
run it. With other extensions, similarly - just create a .bat file and
double-click it, it'll run the commands.

ChrisA


From rxjwg98 at gmail.com  Wed Jun 24 12:53:56 2015
From: rxjwg98 at gmail.com (fl)
Date: Wed, 24 Jun 2015 09:53:56 -0700 (PDT)
Subject: Why does the unit test fail of the pyPDF2 package?
Message-ID: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>

Hi,
I want to learn some coding on PDF. After I download and install pyPDF2,
it cannot pass unit test, which is coming from the package.

I put a screen shot link here to show the console message:

http://tinypic.com/view.php?pic=fbdpg0&s=8#.VYre8_lVhBc

[IMG]http://i57.tinypic.com/fbdpg0.png[/IMG]


This Windows 7 PC has both Python 2.7 and Enthought Canopy (3.4?) installed.

I don't know whether it has conflicts or not.


Thanks, 


From steve at pearwood.info  Wed Jun 24 13:35:12 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 25 Jun 2015 03:35:12 +1000
Subject: Why does the unit test fail of the pyPDF2 package?
References: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
Message-ID: <558aea4f$0$1654$c3e8da3$5496439d@news.astraweb.com>

On Thu, 25 Jun 2015 02:53 am, fl wrote:

> Hi,
> I want to learn some coding on PDF. After I download and install pyPDF2,
> it cannot pass unit test, which is coming from the package.
> 
> I put a screen shot link here to show the console message:

Please don't use screen shots:

(1) We cannot copy and paste text from a screen shot.

(2) Blind people and those with poor vision may not be able to see the
screen shot, and their screen readers do not work on images.

(3) People may be reading your post via email or news, and not be able to
access the website containing the screen shot.

Instead, copy and paste the text from the console and include it in the body
of your message. Do you know how to copy text from the Windows console?


> http://tinypic.com/view.php?pic=fbdpg0&s=8#.VYre8_lVhBc
> 
> [IMG]http://i57.tinypic.com/fbdpg0.png[/IMG]


Have you read the test failure? It is obvious why it failed. Your test
expects to get:

"TheCrazyOnesOctober14,1998Herestothecrazyones..."

but instead gets:

"TheCrazyOnes\nOctober14,1998\nHerestothecrazyones..."

The difference is obvious. Your expected output doesn't include newlines.




-- 
Steven



From fomcl at yahoo.com  Wed Jun 24 13:45:31 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Wed, 24 Jun 2015 17:45:31 +0000 (UTC)
Subject: windows and file names > 256 bytes
Message-ID: <mmeqbr$r92$1@ger.gmane.org>

Hi,

Consider the following calls, where very_long_path is more than 256 bytes:
[1] os.mkdir(very_long_path)
[2] os.getsize(very_long_path)
[3] shutil.rmtree(very_long_path)

I am using Python 2.7 and [1] and [2] fail under Windows XP [3] fails 
under Win7 (not sure about XP). This is even when I use the "special" 
notations \\?\c:\dir\file or \\?\UNC\server\share\file, e.g.
os.path.getsize("\\\\?\\" + "c:\\dir\\file")
(Oddly, os.path.getsize(os.path.join("\\\\?", "c:\\dir\\file")) will 
truncate the prefix)

My questions:
1. How can I get the file size of very long paths under XP?
2. Is this a bug in Python? I would prefer if Python dealt with the gory 
details of Windows' silly behavior.

Regards,
Albert-Jan

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus




From chcarr at sas.upenn.edu  Wed Jun 24 13:48:29 2015
From: chcarr at sas.upenn.edu (Charles Carr)
Date: Wed, 24 Jun 2015 10:48:29 -0700
Subject: 404 Error when using local CGI Server
Message-ID: <CAKDUBcvmfCy45r+Nx+1a7=O3apZLnMTC8V98kOmTmoJnr6=L9A@mail.gmail.com>

I am running a local cgi server from python on a windows 7 computer.
Whenever I try to serve the output of a cgi file by entering the following
into my browser: http://localhost:8080/filename.py , I get a 404 error
message that the file was not found. I'm positive that the files I am
trying to serve are in the same directory as the server script that is
running. Are there any tips as to where I should save my files in order to
avoid this error?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150624/a0caa24e/attachment.html>

From rxjwg98 at gmail.com  Wed Jun 24 13:52:27 2015
From: rxjwg98 at gmail.com (fl)
Date: Wed, 24 Jun 2015 10:52:27 -0700 (PDT)
Subject: Why does the unit test fail of the pyPDF2 package?
In-Reply-To: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
References: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
Message-ID: <252d5aad-e71e-4916-803a-f3b1762f2123@googlegroups.com>

On Wednesday, June 24, 2015 at 9:54:12 AM UTC-7, fl wrote:
> Hi,
> I want to learn some coding on PDF. After I download and install pyPDF2,
> it cannot pass unit test, which is coming from the package.
> 
> I put a screen shot link here to show the console message:
> 
> http://tinypic.com/view.php?pic=fbdpg0&s=8#.VYre8_lVhBc
> 
> [IMG]http://i57.tinypic.com/fbdpg0.png[/IMG]
> 
> 
> This Windows 7 PC has both Python 2.7 and Enthought Canopy (3.4?) installed.
> 
> I don't know whether it has conflicts or not.
> 
> 
> Thanks,

Thanks, Steven. I don't know how to copy command console window contents 
to the forum post. I even try redirection hoping to screen contents to a 
text file, but it fails.


Yes, there are extra '\n' in the extracted, but I don't know how to suppress
it. Does anyone know how to make it the same of the expected?

Thanks,


From david at drmaciver.com  Wed Jun 24 13:53:24 2015
From: david at drmaciver.com (David MacIver)
Date: Wed, 24 Jun 2015 18:53:24 +0100
Subject: Looking for people who are using Hypothesis and are willing to say so
Message-ID: <CADZYRLfsOQyvu8X4OPhG2FomU2bLizNGyXS5DdaRAcGRziMyvw@mail.gmail.com>

Hi there,

Author of Hypothesis here. (If you don't know what Hypothesis is, you're
probably not the target audience for this email but you should totally
check it out: https://hypothesis.readthedocs.org/ Unless you like spending
ages writing tests and still shipping buggy code).

I keep finding out about new people using Hypothesis who I've never heard
of. e.g. turns out that depending on how you count there are between two
and four talks about Hypothesis happening at Europython this year, and many
of them are from people I don't know.

On the one hand, it's great that people are using and excited about it! No
complaints from me there. I was bowled over when I realised about the
EuroPython talks.

On the other hand, it's really quite useful to have more visibility of
usage - both for me to have it and also for other people to see - it's a
much easier sell that people should start using it if they can see that
lots of other people are too.

SO, the point. If you are one of those people using Hypothesis, I'd really
like it if you would say so publicly.

The #1 best way for you to do this for me is to add your name and usage to
https://github.com/DRMacIver/hypothesis/blob/master/docs/endorsements.rst
so it will show up on the endorsements page at
http://hypothesis.readthedocs.org/en/latest/endorsements.html

I'm also thrilled that people are speaking at it, and would love more talks
and blog posts about it. Even tweeting enthusiastically about it is good
too.

Finally, if you are using Hypothesis but can't/don't want to speak about
doing so publicly because your company is doing super top secret stuff (or
any other reason), I'd really appreciate just a short email saying roughly
what sort of things you're using it for, maybe give me an idea of your
workflow. The other reason that I want to know who is using it is so I can
learn where it needs to improve and also help other people use it better
(I'm pretty sure some of the users of Hypothesis at this point have a
better idea how to deploy it than I do).

Regards and thanks,
David R. MacIver
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150624/52afa72c/attachment.html>

From randall at tnr.cc  Wed Jun 24 14:20:07 2015
From: randall at tnr.cc (Randall Smith)
Date: Wed, 24 Jun 2015 13:20:07 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <558AF4D7.1070205@tnr.cc>

On 06/24/2015 06:36 AM, Steven D'Aprano wrote:
> I don't understand how mangling the data is supposed to protect the
> recipient. Don't they have the ability unmangle the data, and thus expose
> themselves to whatever nasties are in the files?

They never look at the data and wouldn't care to unmangle it.  The 
purpose is primarily to prevent automated software (file indexers, virus 
scanners) from doing bad things to the data.

-Randall



From randall at tnr.cc  Wed Jun 24 14:20:07 2015
From: randall at tnr.cc (Randall Smith)
Date: Wed, 24 Jun 2015 13:20:07 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <558AF4D7.1070205@tnr.cc>

On 06/24/2015 06:36 AM, Steven D'Aprano wrote:
> I don't understand how mangling the data is supposed to protect the
> recipient. Don't they have the ability unmangle the data, and thus expose
> themselves to whatever nasties are in the files?

They never look at the data and wouldn't care to unmangle it.  The 
purpose is primarily to prevent automated software (file indexers, virus 
scanners) from doing bad things to the data.

-Randall




From invalid at invalid.invalid  Wed Jun 24 14:23:17 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Wed, 24 Jun 2015 18:23:17 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mmed90$nho$1@reader1.panix.com>
 <mailman.18.1435161161.3674.python-list@python.org>
 <mmel55$ojb$1@reader1.panix.com>
 <mailman.21.1435163021.3674.python-list@python.org>
Message-ID: <mmesil$8a8$2@reader1.panix.com>

On 2015-06-24, Chris Angelico <rosuav at gmail.com> wrote:
> On Thu, Jun 25, 2015 at 2:16 AM, Grant Edwards <invalid at invalid.invalid> wrote:
>> On 2015-06-24, Emile van Sebille <emile at fenx.com> wrote:
>>
>>> Mangling would at least prevent it from executing.
>>
>> If you don't want a file to be executed, then don't make it
>> executable.  Or doesn't Windows have any way to control whether a
>> file is executable or not?
>
> Windows doesn't have the Unix file system concept of execute
> permission, no. If a file has the .exe extension and the first 512
> bytes look like an appropriate header (MZ etc), Windows will happily
> run it. With other extensions, similarly - just create a .bat file
> and double-click it, it'll run the commands.

So can prevent execution, just by changing the filename?  Maybe 30
years using Unix has biased me, but that just seems so wrong...

-- 
Grant Edwards               grant.b.edwards        Yow! Here I am in the
                                  at               POSTERIOR OLFACTORY LOBULE
                              gmail.com            but I don't see CARL SAGAN
                                                   anywhere!!


From randall at tnr.cc  Wed Jun 24 14:24:46 2015
From: randall at tnr.cc (Randall Smith)
Date: Wed, 24 Jun 2015 13:24:46 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CABicbJ+5YgGzkGwFpOyFMfsteUXqMnPjS615=fU6ZCNeSyJuTw@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <CABicbJ+5YgGzkGwFpOyFMfsteUXqMnPjS615=fU6ZCNeSyJuTw@mail.gmail.com>
Message-ID: <558AF5EE.3040205@tnr.cc>

On 06/24/2015 02:44 AM, Devin Jeanpierre wrote:
> How about a random substitution cipher? This will be ultra-weak, but
> fast (using bytes.translate/bytes.maketrans) and seems to be the kind
> of thing you're asking for.
>
> -- Devin


I tried this out and it seems to be just what I need.  Thanks Devin!

It's pure Python, fast, and mangles the data sufficiently.

--Randall


From randall at tnr.cc  Wed Jun 24 14:24:46 2015
From: randall at tnr.cc (Randall Smith)
Date: Wed, 24 Jun 2015 13:24:46 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CABicbJ+5YgGzkGwFpOyFMfsteUXqMnPjS615=fU6ZCNeSyJuTw@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <CABicbJ+5YgGzkGwFpOyFMfsteUXqMnPjS615=fU6ZCNeSyJuTw@mail.gmail.com>
Message-ID: <558AF5EE.3040205@tnr.cc>

On 06/24/2015 02:44 AM, Devin Jeanpierre wrote:
> How about a random substitution cipher? This will be ultra-weak, but
> fast (using bytes.translate/bytes.maketrans) and seems to be the kind
> of thing you're asking for.
>
> -- Devin


I tried this out and it seems to be just what I need.  Thanks Devin!

It's pure Python, fast, and mangles the data sufficiently.

--Randall



From invalid at invalid.invalid  Wed Jun 24 14:29:58 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Wed, 24 Jun 2015 18:29:58 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.25.1435170024.3674.python-list@python.org>
Message-ID: <mmesv6$8a8$3@reader1.panix.com>

On 2015-06-24, Randall Smith <randall at tnr.cc> wrote:
> On 06/24/2015 06:36 AM, Steven D'Aprano wrote:
>
>> I don't understand how mangling the data is supposed to protect the
>> recipient. Don't they have the ability unmangle the data, and thus
>> expose themselves to whatever nasties are in the files?
>
> They never look at the data and wouldn't care to unmangle it. 

I obviously don't "get it". If the recipient is never going look at
the data or unmangle it, why not convert every received file to a
single null byte?  That way you save on disk space as well --
especially if you just create links for all files after the initial
one.  ;)

[I supposed next you're going to tell me that Windows filesystems
don't support links.]

> The purpose is primarily to prevent automated software (file
> indexers, virus scanners) from doing bad things to the data.

Life under windows must be more tiresome than I imagined (or could
imagine) if you have to jump through such hoops to keep "automated
software" from doing bad things to your data files.

-- 
Grant Edwards               grant.b.edwards        Yow! My mind is making
                                  at               ashtrays in Dayton ...
                              gmail.com            


From randall at tnr.cc  Wed Jun 24 14:36:14 2015
From: randall at tnr.cc (Randall Smith)
Date: Wed, 24 Jun 2015 13:36:14 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
Message-ID: <558AF89E.1070105@tnr.cc>

On 06/24/2015 07:19 AM, Dennis Lee Bieber wrote:


> 	Pardon, but that description has me confused. Perhaps I just don't
> understand the full use-case.
>
> 	Who exactly is supposed to be protected from what? You state "data
> senders are supposed to encrypt" which, if the recipient doesn't have the
> decryption key, implies the recipient -- isn't the real recipient but just
> a transport/storage place until the data is retrieved by the end-user.

You got it.  I didn't want to explain any more than necessary.  But yes, 
the recipient just stores the data for the end-user.

>
> 	If "you" do the encryption on the storage machine, then you need to
> also do the decryption when returning the data to the end-user -- which
> means the key is available somewhere on the storage machine, and the local
> user might obtain access to it and the stored data.

Right again.  A legitimate data owner would encrypt the data.  The 
storage machine is encrypting to protect itself against unwanted 
exposure to unencrypted malware.  Not that they would go looking at the 
files, but their virus scanner or file indexer might.

>
> 	Given the assumptions I'm making, my recommendation is likely to be
> something on the nature of: use an OS designed with security at the core of
> the file system; each sender has their own login UID, and the file system
> is configured to grant r/w access only to the login -- no execute
> permissions, no access by someone not logged in as that user, etc.

Yes.  This is done for "imaged" systems, but I don't have control over 
the storage machines.

I'm leaning towards using a random substitution cipher suggested by 
Devin Jeanpierre.  If you see any weaknesses in that solution, I'd like 
to hear them.

Thanks for your response.


--Randall


From randall at tnr.cc  Wed Jun 24 14:36:14 2015
From: randall at tnr.cc (Randall Smith)
Date: Wed, 24 Jun 2015 13:36:14 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
Message-ID: <558AF89E.1070105@tnr.cc>

On 06/24/2015 07:19 AM, Dennis Lee Bieber wrote:


> 	Pardon, but that description has me confused. Perhaps I just don't
> understand the full use-case.
>
> 	Who exactly is supposed to be protected from what? You state "data
> senders are supposed to encrypt" which, if the recipient doesn't have the
> decryption key, implies the recipient -- isn't the real recipient but just
> a transport/storage place until the data is retrieved by the end-user.

You got it.  I didn't want to explain any more than necessary.  But yes, 
the recipient just stores the data for the end-user.

>
> 	If "you" do the encryption on the storage machine, then you need to
> also do the decryption when returning the data to the end-user -- which
> means the key is available somewhere on the storage machine, and the local
> user might obtain access to it and the stored data.

Right again.  A legitimate data owner would encrypt the data.  The 
storage machine is encrypting to protect itself against unwanted 
exposure to unencrypted malware.  Not that they would go looking at the 
files, but their virus scanner or file indexer might.

>
> 	Given the assumptions I'm making, my recommendation is likely to be
> something on the nature of: use an OS designed with security at the core of
> the file system; each sender has their own login UID, and the file system
> is configured to grant r/w access only to the login -- no execute
> permissions, no access by someone not logged in as that user, etc.

Yes.  This is done for "imaged" systems, but I don't have control over 
the storage machines.

I'm leaning towards using a random substitution cipher suggested by 
Devin Jeanpierre.  If you see any weaknesses in that solution, I'd like 
to hear them.

Thanks for your response.


--Randall



From python at mrabarnett.plus.com  Wed Jun 24 14:48:37 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 24 Jun 2015 19:48:37 +0100
Subject: Why does the unit test fail of the pyPDF2 package?
In-Reply-To: <252d5aad-e71e-4916-803a-f3b1762f2123@googlegroups.com>
References: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
 <252d5aad-e71e-4916-803a-f3b1762f2123@googlegroups.com>
Message-ID: <558AFB85.8020602@mrabarnett.plus.com>

On 2015-06-24 18:52, fl wrote:
> On Wednesday, June 24, 2015 at 9:54:12 AM UTC-7, fl wrote:
>> Hi,
>> I want to learn some coding on PDF. After I download and install pyPDF2,
>> it cannot pass unit test, which is coming from the package.
>>
>> I put a screen shot link here to show the console message:
>>
>> http://tinypic.com/view.php?pic=fbdpg0&s=8#.VYre8_lVhBc
>>
>> [IMG]http://i57.tinypic.com/fbdpg0.png[/IMG]
>>
>>
>> This Windows 7 PC has both Python 2.7 and Enthought Canopy (3.4?) installed.
>>
>> I don't know whether it has conflicts or not.
>>
>>
>> Thanks,
>
> Thanks, Steven. I don't know how to copy command console window contents
> to the forum post. I even try redirection hoping to screen contents to a
> text file, but it fails.
>
You can make a rectangular selection by dragging over the console
window the mouse pointer.

>
> Yes, there are extra '\n' in the extracted, but I don't know how to suppress
> it. Does anyone know how to make it the same of the expected?
>
> Thanks,
>



From randall at tnr.cc  Wed Jun 24 15:00:38 2015
From: randall at tnr.cc (Randall Smith)
Date: Wed, 24 Jun 2015 14:00:38 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmesv6$8a8$3@reader1.panix.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.25.1435170024.3674.python-list@python.org>
 <mmesv6$8a8$3@reader1.panix.com>
Message-ID: <558AFE56.2000400@tnr.cc>

On 06/24/2015 01:29 PM, Grant Edwards wrote:
> On 2015-06-24, Randall Smith <randall at tnr.cc> wrote:
>> On 06/24/2015 06:36 AM, Steven D'Aprano wrote:
>>
>>> I don't understand how mangling the data is supposed to protect the
>>> recipient. Don't they have the ability unmangle the data, and thus
>>> expose themselves to whatever nasties are in the files?
>>
>> They never look at the data and wouldn't care to unmangle it.
>
> I obviously don't "get it". If the recipient is never going look at
> the data or unmangle it, why not convert every received file to a
> single null byte?  That way you save on disk space as well --
> especially if you just create links for all files after the initial
> one.  ;)
>
> [I supposed next you're going to tell me that Windows filesystems
> don't support links.]
>
>> The purpose is primarily to prevent automated software (file
>> indexers, virus scanners) from doing bad things to the data.
>
> Life under windows must be more tiresome than I imagined (or could
> imagine) if you have to jump through such hoops to keep "automated
> software" from doing bad things to your data files.
>

These are machines storing chunks of other people's data.  The data 
owner chunks a file, compresses and encrypts it, then sends it to 
several storage servers.  The storage server might be a Raspberry PI 
with a USB disk or a Windows XP machine - I can't know which.


I don't use Windows and don't recommend it for this software. 
Nevertheless, many people do use it.

-Randall


From randall at tnr.cc  Wed Jun 24 15:00:38 2015
From: randall at tnr.cc (Randall Smith)
Date: Wed, 24 Jun 2015 14:00:38 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmesv6$8a8$3@reader1.panix.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.25.1435170024.3674.python-list@python.org>
 <mmesv6$8a8$3@reader1.panix.com>
Message-ID: <558AFE56.2000400@tnr.cc>

On 06/24/2015 01:29 PM, Grant Edwards wrote:
> On 2015-06-24, Randall Smith <randall at tnr.cc> wrote:
>> On 06/24/2015 06:36 AM, Steven D'Aprano wrote:
>>
>>> I don't understand how mangling the data is supposed to protect the
>>> recipient. Don't they have the ability unmangle the data, and thus
>>> expose themselves to whatever nasties are in the files?
>>
>> They never look at the data and wouldn't care to unmangle it.
>
> I obviously don't "get it". If the recipient is never going look at
> the data or unmangle it, why not convert every received file to a
> single null byte?  That way you save on disk space as well --
> especially if you just create links for all files after the initial
> one.  ;)
>
> [I supposed next you're going to tell me that Windows filesystems
> don't support links.]
>
>> The purpose is primarily to prevent automated software (file
>> indexers, virus scanners) from doing bad things to the data.
>
> Life under windows must be more tiresome than I imagined (or could
> imagine) if you have to jump through such hoops to keep "automated
> software" from doing bad things to your data files.
>

These are machines storing chunks of other people's data.  The data 
owner chunks a file, compresses and encrypts it, then sends it to 
several storage servers.  The storage server might be a Raspberry PI 
with a USB disk or a Windows XP machine - I can't know which.


I don't use Windows and don't recommend it for this software. 
Nevertheless, many people do use it.

-Randall



From rxjwg98 at gmail.com  Wed Jun 24 15:02:52 2015
From: rxjwg98 at gmail.com (fl)
Date: Wed, 24 Jun 2015 12:02:52 -0700 (PDT)
Subject: Why does the unit test fail of the pyPDF2 package?
In-Reply-To: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
References: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
Message-ID: <16dfcef6-4740-45b9-b04f-0f5bc0899c01@googlegroups.com>

On Wednesday, June 24, 2015 at 9:54:12 AM UTC-7, fl wrote:
> Hi,
> I want to learn some coding on PDF. After I download and install pyPDF2,
> it cannot pass unit test, which is coming from the package.
> 
> I put a screen shot link here to show the console message:
> 
> http://tinypic.com/view.php?pic=fbdpg0&s=8#.VYre8_lVhBc
> 
> [IMG]http://i57.tinypic.com/fbdpg0.png[/IMG]
> 
> 
> This Windows 7 PC has both Python 2.7 and Enthought Canopy (3.4?) installed.
> 
> I don't know whether it has conflicts or not.
> 
> 
> Thanks,

You can make a rectangular selection by dragging over the console 
 window the mouse pointer. 

Excuse me. I don't understand your idea. On the command window, there is
no content copied through a mouse click/drag (even no screen difference).
Do you mean using Snipping Tool? That will be an image, which is not
advised as a previous poster.
Thanks,


From gordon at panix.com  Wed Jun 24 15:13:11 2015
From: gordon at panix.com (John Gordon)
Date: Wed, 24 Jun 2015 19:13:11 +0000 (UTC)
Subject: Why does the unit test fail of the pyPDF2 package?
References: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
 <16dfcef6-4740-45b9-b04f-0f5bc0899c01@googlegroups.com>
Message-ID: <mmevg7$k62$1@reader1.panix.com>

In <16dfcef6-4740-45b9-b04f-0f5bc0899c01 at googlegroups.com> fl <rxjwg98 at gmail.com> writes:

> Excuse me. I don't understand your idea. On the command window, there is
> no content copied through a mouse click/drag (even no screen difference).

Right-click the command window title bar and select Edit -> Mark.
Then use the mouse to select a rectangular area of text.
Then right-click the title bar again and select Edit -> Copy.

-- 
John Gordon                   A is for Amy, who fell down the stairs
gordon at panix.com              B is for Basil, assaulted by bears
                                -- Edward Gorey, "The Gashlycrumb Tinies"



From jon+usenet at unequivocal.co.uk  Wed Jun 24 15:13:43 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Wed, 24 Jun 2015 19:13:43 +0000 (UTC)
Subject: Why does the unit test fail of the pyPDF2 package?
References: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
 <16dfcef6-4740-45b9-b04f-0f5bc0899c01@googlegroups.com>
Message-ID: <slrnmom0do.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-24, fl <rxjwg98 at gmail.com> wrote:
> You can make a rectangular selection by dragging over the console 
>  window the mouse pointer. 
>
> Excuse me. I don't understand your idea. On the command window, there is
> no content copied through a mouse click/drag (even no screen difference).
> Do you mean using Snipping Tool? That will be an image, which is not
> advised as a previous poster.

Click on the icon at the top-left of the command window.
Choose "Edit->Mark". Drag to select the text you want.
Click the menu again, and choose "Edit->Copy".


From rxjwg98 at gmail.com  Wed Jun 24 15:50:12 2015
From: rxjwg98 at gmail.com (fl)
Date: Wed, 24 Jun 2015 12:50:12 -0700 (PDT)
Subject: Why does the unit test fail of the pyPDF2 package?
In-Reply-To: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
References: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
Message-ID: <af71ac41-0030-45c8-834f-6730185be2c1@googlegroups.com>

On Wednesday, June 24, 2015 at 9:54:12 AM UTC-7, fl wrote:
> Hi,
> I want to learn some coding on PDF. After I download and install pyPDF2,
> it cannot pass unit test, which is coming from the package.
> 
> I put a screen shot link here to show the console message:
> 
> http://tinypic.com/view.php?pic=fbdpg0&s=8#.VYre8_lVhBc
> 
> [IMG]http://i57.tinypic.com/fbdpg0.png[/IMG]
> 
> 
> This Windows 7 PC has both Python 2.7 and Enthought Canopy (3.4?) installed.
> 
> I don't know whether it has conflicts or not.
> 
> 
> Thanks,

Thanks for the trick! I know now how new I am to the Windows.

Below is the installation message, and the unittest message.

Suspecting there are differences between Linux and Windows on '\n', I
 install pyPDF2 on Ubuntu. It has the same error. What the hell of pyPDF2
 is? I don't know what use/purpose of its test script for. This process is
 also for my learning on Python. Does anyone have the same or different 
 experiences on pyPDF2?

Thanks again.



/////////////////////






ImportError: No module named Tests

C:\Python27\Tools\PyPDF2-master\Tests>cd ..

C:\Python27\Tools\PyPDF2-master>C:\python27\python.exe setup.py install
running install
running build
running build_py
creating build
creating build\lib
creating build\lib\PyPDF2
copying PyPDF2\filters.py -> build\lib\PyPDF2
copying PyPDF2\generic.py -> build\lib\PyPDF2
copying PyPDF2\merger.py -> build\lib\PyPDF2
copying PyPDF2\pagerange.py -> build\lib\PyPDF2
copying PyPDF2\pdf.py -> build\lib\PyPDF2
copying PyPDF2\utils.py -> build\lib\PyPDF2
copying PyPDF2\xmp.py -> build\lib\PyPDF2
copying PyPDF2\_version.py -> build\lib\PyPDF2
copying PyPDF2\__init__.py -> build\lib\PyPDF2
running install_lib
creating C:\python27\Lib\site-packages\PyPDF2
copying build\lib\PyPDF2\filters.py -> C:\python27\Lib\site-packages\PyPDF2
copying build\lib\PyPDF2\generic.py -> C:\python27\Lib\site-packages\PyPDF2
copying build\lib\PyPDF2\merger.py -> C:\python27\Lib\site-packages\PyPDF2
copying build\lib\PyPDF2\pagerange.py -> C:\python27\Lib\site-packages\PyPDF2
copying build\lib\PyPDF2\pdf.py -> C:\python27\Lib\site-packages\PyPDF2
copying build\lib\PyPDF2\utils.py -> C:\python27\Lib\site-packages\PyPDF2
copying build\lib\PyPDF2\xmp.py -> C:\python27\Lib\site-packages\PyPDF2
copying build\lib\PyPDF2\_version.py -> C:\python27\Lib\site-packages\PyPDF2
copying build\lib\PyPDF2\__init__.py -> C:\python27\Lib\site-packages\PyPDF2
byte-compiling C:\python27\Lib\site-packages\PyPDF2\filters.py to filters.pyc
byte-compiling C:\python27\Lib\site-packages\PyPDF2\generic.py to generic.pyc
byte-compiling C:\python27\Lib\site-packages\PyPDF2\merger.py to merger.pyc
byte-compiling C:\python27\Lib\site-packages\PyPDF2\pagerange.py to pagerange.py
c
byte-compiling C:\python27\Lib\site-packages\PyPDF2\pdf.py to pdf.pyc
byte-compiling C:\python27\Lib\site-packages\PyPDF2\utils.py to utils.pyc
byte-compiling C:\python27\Lib\site-packages\PyPDF2\xmp.py to xmp.pyc
byte-compiling C:\python27\Lib\site-packages\PyPDF2\_version.py to _version.pyc
byte-compiling C:\python27\Lib\site-packages\PyPDF2\__init__.py to __init__.pyc
running install_egg_info
Writing C:\python27\Lib\site-packages\PyPDF2-1.24-py2.7.egg-info











C:\Python27\Tools\PyPDF2-master>C:\python27\python.exe -m unittest Tests.tests >
> logt
F
======================================================================
FAIL: test_PdfReaderFileLoad (Tests.tests.PdfReaderTestCases)
Test loading and parsing of a file. Extract text of the file and compare to expe
cted
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Tests\tests.py", line 35, in test_PdfReaderFileLoad
    % (pdftext, ipdf_p1_text.encode('utf-8', errors='ignore')))
AssertionError: PDF extracted text differs from expected value.

Expected:

'TheCrazyOnesOctober14,1998Herestothecrazyones.Themis\xcb\x9dts.Therebels.Thetro
ublemakers.Theroundpegsinthesquareholes.Theoneswhoseethingsdi\xcb\x99erently.The
yrenotfondofrules.Andtheyhavenorespectforthestatusquo.Youcanquotethem,disagreewi
ththem,glorifyorvilifythem.Abouttheonlythingyoucantdoisignorethem.Becausetheycha
ngethings.Theyinvent.Theyimagine.Theyheal.Theyexplore.Theycreate.Theyinspire.The
ypushthehumanraceforward.Maybetheyhavetobecrazy.Howelsecanyoustareatanemptycanva
sandseeaworkofart?Orsitinsilenceandhearasongthatsneverbeenwritten?Orgazeataredpl
anetandseealaboratoryonwheels?Wemaketoolsforthesekindsofpeople.Whilesomeseethema
sthecrazyones,weseegenius.Becausethepeoplewhoarecrazyenoughtothinktheycanchanget
heworld,aretheoneswhodo.'

Extracted:

'TheCrazyOnes\nOctober14,1998\nHerestothecrazyones.Themis\xcb\x9dts.Therebels.Th
etroublemakers.\nTheroundpegsinthesquareholes.\nTheoneswhoseethingsdi\xcb\x99ere
ntly.Theyrenotfondofrules.And\ntheyhavenorespectforthestatusquo.Youcanquotethem,
\ndisagreewiththem,glorifyorvilifythem.\nAbouttheonlythingyoucantdoisignorethem.
Becausetheychange\nthings.Theyinvent.Theyimagine.Theyheal.Theyexplore.They\ncrea
te.Theyinspire.Theypushthehumanraceforward.\nMaybetheyhavetobecrazy.\nHowelsecan
youstareatanemptycanvasandseeaworkofart?Or\nsitinsilenceandhearasongthatsneverbe
enwritten?Orgazeat\naredplanetandseealaboratoryonwheels?\nWemaketoolsforthesekin
dsofpeople.\nWhilesomeseethemasthecrazyones,weseegenius.Becausethe\npeoplewhoare
crazyenoughtothinktheycanchangetheworld,\naretheoneswhodo.\n'



----------------------------------------------------------------------
Ran 1 test in 0.016s

FAILED (failures=1)

C:\Python27\Tools\PyPDF2-master>
C:\Python27\Tools\PyPDF2-master>


From breamoreboy at yahoo.co.uk  Wed Jun 24 16:02:45 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 24 Jun 2015 21:02:45 +0100
Subject: Why does the unit test fail of the pyPDF2 package?
In-Reply-To: <558AFB85.8020602@mrabarnett.plus.com>
References: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
 <252d5aad-e71e-4916-803a-f3b1762f2123@googlegroups.com>
 <558AFB85.8020602@mrabarnett.plus.com>
Message-ID: <mmf2d9$qe1$1@ger.gmane.org>

On 24/06/2015 19:48, MRAB wrote:
> On 2015-06-24 18:52, fl wrote:
>> On Wednesday, June 24, 2015 at 9:54:12 AM UTC-7, fl wrote:
>>> Hi,
>>> I want to learn some coding on PDF. After I download and install pyPDF2,
>>> it cannot pass unit test, which is coming from the package.
>>>
>>> I put a screen shot link here to show the console message:
>>>
>>> http://tinypic.com/view.php?pic=fbdpg0&s=8#.VYre8_lVhBc
>>>
>>> [IMG]http://i57.tinypic.com/fbdpg0.png[/IMG]
>>>
>>>
>>> This Windows 7 PC has both Python 2.7 and Enthought Canopy (3.4?)
>>> installed.
>>>
>>> I don't know whether it has conflicts or not.
>>>
>>>
>>> Thanks,
>>
>> Thanks, Steven. I don't know how to copy command console window contents
>> to the forum post. I even try redirection hoping to screen contents to a
>> text file, but it fails.
>>
> You can make a rectangular selection by dragging over the console
> window the mouse pointer.
>

An alternative is to install ConEmu and set up a startup.txt file.  My 
heavily encrypted :) version follows

cmd /F:ON /T:02 /K cd C:\Users\Mark\Documents\MyPython 
"-new_console:t:MyPython"
cmd /F:ON /T:02 /K cd c:\Users\Mark\pythonissues "-new_console:t:Python 
Issues"
cmd /F:ON /T:02 /K cd c:\cPython\PCBuild "-new_console:t:cPython"
cmd /F:ON /T:02 /K cd C:\Users\Mark\Documents\Cash\Python 
"-new_console:t:Cash Python"
C:\Python34\Scripts\ipython.exe --matplotlib "-new_console:t:iPython"

Once you've overcome the encryption life is far, far easier on Windows. 
  You can do really advanced concepts like cut and paste relatively 
easily, but please don't take my word for it, try it yourself.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From invalid at invalid.invalid  Wed Jun 24 17:24:38 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Wed, 24 Jun 2015 21:24:38 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.25.1435170024.3674.python-list@python.org>
 <mmesv6$8a8$3@reader1.panix.com>
 <mailman.32.1435172453.3674.python-list@python.org>
Message-ID: <mmf76l$qg9$1@reader1.panix.com>

On 2015-06-24, Randall Smith <randall at tnr.cc> wrote:
> On 06/24/2015 01:29 PM, Grant Edwards wrote:
>> On 2015-06-24, Randall Smith <randall at tnr.cc> wrote:
>>> On 06/24/2015 06:36 AM, Steven D'Aprano wrote:
>>>
>>>> I don't understand how mangling the data is supposed to protect the
>>>> recipient. Don't they have the ability unmangle the data, and thus
>>>> expose themselves to whatever nasties are in the files?
>>>
>>> They never look at the data and wouldn't care to unmangle it.
>>
>> I obviously don't "get it". If the recipient is never going look at
>> the data or unmangle it, why not convert every received file to a
>> single null byte?  That way you save on disk space as well --
>> especially if you just create links for all files after the initial
>> one.  ;)
>
> These are machines storing chunks of other people's data.  The data 
> owner chunks a file, compresses and encrypts it, then sends it to 
> several storage servers.  The storage server might be a Raspberry PI 
> with a USB disk or a Windows XP machine - I can't know which.

OK.  But if the recipient (the server) mangles the data and then never
unmangles or reads the data, there doesn't seem to be any point in
storing it.  I must be misunderstanding your statement that the data
is never read/unmangled.

-- 
Grant Edwards               grant.b.edwards        Yow! A can of ASPARAGUS,
                                  at               73 pigeons, some LIVE ammo,
                              gmail.com            and a FROZEN DAQUIRI!!


From no.email at nospam.invalid  Wed Jun 24 18:13:36 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Wed, 24 Jun 2015 15:13:36 -0700
Subject: Looking for people who are using Hypothesis and are willing to
 say so
References: <mailman.22.1435168426.3674.python-list@python.org>
Message-ID: <871th0yctr.fsf@nightsong.com>

David MacIver <david at drmaciver.com> writes:
> Author of Hypothesis here. (If you don't know what Hypothesis is, you're
> probably not the target audience for this email but you should totally
> check it out: https://hypothesis.readthedocs.org/

Oh very cool: a QuickCheck-like unit test library.  I heard of something
like that for Python recently, that might or might not have been
Hypothesis.  I certainly plan to try it out.  The original QuickCheck
(for Haskell) used the static type signatures on the functions under
test to know what test cases to generate, but Erlang QuickCheck has had
some good successes, including finding some subtle bugs during
development in the HAMT (Clojure-like hash array mapped trie)
implementation just released with Erlang/OTP 18.0 this week.

I see Hypothesis use decorators that look sort of like Erlang Dialyzer
so that can help with test cases.  Maybe later, it use Python 3 type
annotations, though I think those are still much less precise than
Dialyzer or Haskell types.


From billy.earney at gmail.com  Wed Jun 24 18:37:54 2015
From: billy.earney at gmail.com (Billy Earney)
Date: Wed, 24 Jun 2015 17:37:54 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmf76l$qg9$1@reader1.panix.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.25.1435170024.3674.python-list@python.org>
 <mmesv6$8a8$3@reader1.panix.com>
 <mailman.32.1435172453.3674.python-list@python.org>
 <mmf76l$qg9$1@reader1.panix.com>
Message-ID: <CAB1ii-fOvD+aSUEGAUYo7LhSWvtrnRZP9_LN4G4UK-SV-VLMgg@mail.gmail.com>

Freenet seems to come to mind.. :)

On Wed, Jun 24, 2015 at 4:24 PM, Grant Edwards <invalid at invalid.invalid>
wrote:

> On 2015-06-24, Randall Smith <randall at tnr.cc> wrote:
> > On 06/24/2015 01:29 PM, Grant Edwards wrote:
> >> On 2015-06-24, Randall Smith <randall at tnr.cc> wrote:
> >>> On 06/24/2015 06:36 AM, Steven D'Aprano wrote:
> >>>
> >>>> I don't understand how mangling the data is supposed to protect the
> >>>> recipient. Don't they have the ability unmangle the data, and thus
> >>>> expose themselves to whatever nasties are in the files?
> >>>
> >>> They never look at the data and wouldn't care to unmangle it.
> >>
> >> I obviously don't "get it". If the recipient is never going look at
> >> the data or unmangle it, why not convert every received file to a
> >> single null byte?  That way you save on disk space as well --
> >> especially if you just create links for all files after the initial
> >> one.  ;)
> >
> > These are machines storing chunks of other people's data.  The data
> > owner chunks a file, compresses and encrypts it, then sends it to
> > several storage servers.  The storage server might be a Raspberry PI
> > with a USB disk or a Windows XP machine - I can't know which.
>
> OK.  But if the recipient (the server) mangles the data and then never
> unmangles or reads the data, there doesn't seem to be any point in
> storing it.  I must be misunderstanding your statement that the data
> is never read/unmangled.
>
> --
> Grant Edwards               grant.b.edwards        Yow! A can of ASPARAGUS,
>                                   at               73 pigeons, some LIVE
> ammo,
>                               gmail.com            and a FROZEN DAQUIRI!!
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150624/8ea10be0/attachment.html>

From randall at tnr.cc  Wed Jun 24 19:13:06 2015
From: randall at tnr.cc (Randall Smith)
Date: Wed, 24 Jun 2015 18:13:06 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmf76l$qg9$1@reader1.panix.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.25.1435170024.3674.python-list@python.org>
 <mmesv6$8a8$3@reader1.panix.com>
 <mailman.32.1435172453.3674.python-list@python.org>
 <mmf76l$qg9$1@reader1.panix.com>
Message-ID: <558B3982.9060305@tnr.cc>

On 06/24/2015 04:24 PM, Grant Edwards wrote:

>
> OK.  But if the recipient (the server) mangles the data and then never
> unmangles or reads the data, there doesn't seem to be any point in
> storing it.  I must be misunderstanding your statement that the data
> is never read/unmangled.
>

When the storage server sends the data (on request), it decodes the data 
before sending.  I'm currently testing this on a Raspberry PI using a 
random substitution with bytearray.maketrans and bytearray.translate on 
Raspberry PI and it is working quite well.

Thanks.

-Randall



From randall at tnr.cc  Wed Jun 24 19:13:06 2015
From: randall at tnr.cc (Randall Smith)
Date: Wed, 24 Jun 2015 18:13:06 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmf76l$qg9$1@reader1.panix.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.25.1435170024.3674.python-list@python.org>
 <mmesv6$8a8$3@reader1.panix.com>
 <mailman.32.1435172453.3674.python-list@python.org>
 <mmf76l$qg9$1@reader1.panix.com>
Message-ID: <558B3982.9060305@tnr.cc>

On 06/24/2015 04:24 PM, Grant Edwards wrote:

>
> OK.  But if the recipient (the server) mangles the data and then never
> unmangles or reads the data, there doesn't seem to be any point in
> storing it.  I must be misunderstanding your statement that the data
> is never read/unmangled.
>

When the storage server sends the data (on request), it decodes the data 
before sending.  I'm currently testing this on a Raspberry PI using a 
random substitution with bytearray.maketrans and bytearray.translate on 
Raspberry PI and it is working quite well.

Thanks.

-Randall




From rxjwg98 at gmail.com  Wed Jun 24 19:52:10 2015
From: rxjwg98 at gmail.com (fl)
Date: Wed, 24 Jun 2015 16:52:10 -0700 (PDT)
Subject: Could you explain this rebinding (or some other action) on "nums =
 nums"?
Message-ID: <94c2e42e-1e5f-40cf-9259-26035e277bf3@googlegroups.com>

Hi,

I read a blog written by Ned and find it is very interesting, but I am still
unclear it in some parts. In the following example, I am almost lost at the
last line: 

nums = num


Could anyone explain it in a more detail to me?

Thanks,





.......................
The reason is that list implements __iadd__ like this (except in C, not Python):

class List:
    def __iadd__(self, other):
        self.extend(other)
        return self
When you execute "nums += more", you're getting the same effect as:

nums = nums.__iadd__(more)
which, because of the implementation of __iadd__, acts like this:

nums.extend(more)
nums = nums
So there is a rebinding operation here, but first, there's a mutating operation, and the rebinding operation is a no-op.


From rosuav at gmail.com  Wed Jun 24 20:16:47 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 25 Jun 2015 10:16:47 +1000
Subject: Could you explain this rebinding (or some other action) on "nums
 = nums"?
In-Reply-To: <94c2e42e-1e5f-40cf-9259-26035e277bf3@googlegroups.com>
References: <94c2e42e-1e5f-40cf-9259-26035e277bf3@googlegroups.com>
Message-ID: <CAPTjJmp8jxUGjCmurVriLsEg9QvwKKX_+TfcT7U1ZuHu9nA1bw@mail.gmail.com>

On Thu, Jun 25, 2015 at 9:52 AM, fl <rxjwg98 at gmail.com> wrote:
> The reason is that list implements __iadd__ like this (except in C, not Python):
>
> class List:
>     def __iadd__(self, other):
>         self.extend(other)
>         return self
> When you execute "nums += more", you're getting the same effect as:
>
> nums = nums.__iadd__(more)
> which, because of the implementation of __iadd__, acts like this:
>
> nums.extend(more)
> nums = nums
> So there is a rebinding operation here, but first, there's a mutating operation, and the rebinding operation is a no-op.

It's not a complete no-op, as can be demonstrated if you use something
other than a simple name:

>>> tup = ("spam", [1, 2, 3], "ham")
>>> tup[1]
[1, 2, 3]
>>> tup[1].extend([4,5])
>>> tup[1] = tup[1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> tup
('spam', [1, 2, 3, 4, 5], 'ham')
>>> tup[1] += [6,7]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> tup
('spam', [1, 2, 3, 4, 5, 6, 7], 'ham')

The reason for the rebinding is that += can do two completely
different things: with mutable objects, like lists, it changes them in
place, but with immutables, it returns a new one:

>>> msg = "Hello"
>>> msg += ", world!"
>>> msg
'Hello, world!'

This didn't change the string "Hello", because you can't do that.
Instead, it rebound msg to "Hello, world!". For consistency, the +=
operator will *always* rebind, but in situations where that's not
necessary, it rebinds to the exact same object.

Does that answer the question?

ChrisA


From steve at pearwood.info  Wed Jun 24 21:20:28 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 25 Jun 2015 11:20:28 +1000
Subject: Why does the unit test fail of the pyPDF2 package?
References: <3d9cd33f-f12d-48bc-a645-26e55c31f168@googlegroups.com>
 <252d5aad-e71e-4916-803a-f3b1762f2123@googlegroups.com>
Message-ID: <558b575c$0$1666$c3e8da3$5496439d@news.astraweb.com>

On Thu, 25 Jun 2015 03:52 am, fl wrote:

> Thanks, Steven. I don't know how to copy command console window contents
> to the forum post.

I don't know either, because I don't use Windows, but you can google for
instructions:

https://duckduckgo.com/html/?q=copy+text+windows+console

https://startpage.com/do/search?q=copy+text+windows+console

http://www.bing.com/search?q=copy+text+windows+console

http://au.search.yahoo.com/search?p=copy+text+windows+console

Even Google works:

https://www.google.com.au/search?q=copy+text+windows+console


> Yes, there are extra '\n' in the extracted, but I don't know how to
> suppress it. Does anyone know how to make it the same of the expected?

Where you enter the expected output, instead of entering:

"TheCrazyOnesOctober14,1998Herestothecrazyones..."

instead enter:

"TheCrazyOnes\nOctober14,1998\nHerestothecrazyones..."


The point is that your expected output should contain the text actually in
the PDF file. If your expected output is different from the actual
contents, then the expectations are wrong, not your code. The test itself
is buggy.



-- 
Steven



From steve at pearwood.info  Thu Jun 25 00:07:33 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 25 Jun 2015 14:07:33 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
Message-ID: <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>

On Thu, 25 Jun 2015 04:36 am, Randall Smith wrote:

> On 06/24/2015 07:19 AM, Dennis Lee Bieber wrote:
> 
> 
>> Pardon, but that description has me confused. Perhaps I just don't
>> understand the full use-case.
>>
>> Who exactly is supposed to be protected from what? You state "data
>> senders are supposed to encrypt" which, if the recipient doesn't have the
>> decryption key, implies the recipient -- isn't the real recipient but
>> just a transport/storage place until the data is retrieved by the
>> end-user.
> 
> You got it.  I didn't want to explain any more than necessary.  But yes,
> the recipient just stores the data for the end-user.

Trust me. That's not all they are doing.


>> If "you" do the encryption on the storage machine, then you need to
>> also do the decryption when returning the data to the end-user -- which
>> means the key is available somewhere on the storage machine, and the
>> local user might obtain access to it and the stored data.
> 
> Right again.  A legitimate data owner would encrypt the data.  The
> storage machine is encrypting to protect itself against unwanted
> exposure to unencrypted malware.  Not that they would go looking at the
> files, but their virus scanner or file indexer might.

Okay, you're worrying me now. If this is legitimate business, then you
shouldn't be worried about the virus scanner or file indexer *scanning* the
content of the file.

But giving you the benefit of the doubt, that there's nothing underhanded
happening, I don't think you have a good model for the potential threats in
your software. I think there are at least three different threats:

Sender of the data versus the storage machine:

- the sender of the data may deliberately send malware, intending to attack
the people storing the file;

Storage machine versus the end recipient:

- the storage machine may be infected by malware which corrupts the file;

- the owner of the storage machine may deliberately corrupt the data (this
is a special case of the previous);

- the owner of the storage machine may want to spy on the files, that is,
read the contents without changing the files (attack on privacy).


There may be others threats as well, e.g. man-in-the-middle attacks. If this
is anything like Bittorrent, you have a whole range of threats.

But just sticking to the three above, the first one is partially mitigated
by allowing virus scanners to scan the data, but that implies that the
owner of the storage machine can spy on the files. So you have a conflict
here.

Honestly, the *only* real defence against the spying issue is to encrypt the
files. Not obfuscate them with a lousy random substitution cipher. The
storage machine can keep the files as long as they like, just by making a
copy, and spend hours bruteforcing them. They *will* crack the substitution
cipher. In pure Python, that may take a few days or weeks; in C, hours or
days. If they have the resources to throw at it, minutes. Substitution
ciphers have not been effective encryption since, oh, the 1950s, unless you
use a one-time pad. Which you won't be.

That's assuming they don't just look at the Python source code, grab the
cipher key, and decrypt in seconds.

If you're serious about protecting your users privacy and their data
integrity, you need to use modern strong encryption, and you need to solve
the issue of how to get the key from the trusted source to the untrusted
storage machine. I have no idea how to do that -- you need to talk to
actual security experts, not random Python programmers.

A pure Python solution for the encryption is likely to be too slow for more
than toy files. Bite the bullet and use a library written in C. Python uses
C code for all sorts of modules: math, decimal, bisect, pickle, io, etc.
all delegate to C code when available. There's no shame in it.

Not to put too fine a point on it, using a substitution cipher because it's
easy and fast in pure Python code is like making a boat out of styrofoam
because it's light and floats and using aluminium or fibreglass is too
expensive. Sure that will work for toy applications, like paddling around
the swimming pool in your back yard, but nobody in their right mind would
trust it on the deep ocean or a white-water river.



-- 
Steven



From jeanpierreda at gmail.com  Thu Jun 25 00:27:50 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Wed, 24 Jun 2015 21:27:50 -0700
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CABicbJLZJq7pA9ws8Edfzuch2cpkobpP83MQUsmvwGXaYs4rtw@mail.gmail.com>

On Wed, Jun 24, 2015 at 9:07 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> But just sticking to the three above, the first one is partially mitigated
> by allowing virus scanners to scan the data, but that implies that the
> owner of the storage machine can spy on the files. So you have a conflict
> here.

If it's encrypted malware, and you can't decrypt it, there's no threat.

> Honestly, the *only* real defence against the spying issue is to encrypt the
> files. Not obfuscate them with a lousy random substitution cipher. The
> storage machine can keep the files as long as they like, just by making a
> copy, and spend hours bruteforcing them. They *will* crack the substitution
> cipher. In pure Python, that may take a few days or weeks; in C, hours or
> days. If they have the resources to throw at it, minutes. Substitution
> ciphers have not been effective encryption since, oh, the 1950s, unless you
> use a one-time pad. Which you won't be.

The original post said that the sender will usually send files they
encrypted, unless they are malicious. So if the sender wants them to
be encrypted, they already are.

"While the data senders are supposed to encrypt data, that's not
guaranteed, and I'd like to protect the recipient against exposure to
nefarious data by mangling or encrypting the data before it is written
to disk."

The cipher is just to keep the sender from being able to control what
is on disk.

I am usually very oppositional when it comes to rolling your own
crypto, but am I alone here in thinking the OP very clearly laid out
their case?

-- Devin


From skyjjogari at hanmail.net  Thu Jun 25 02:49:16 2015
From: skyjjogari at hanmail.net (=?utf-8?B?66y46rG07Z2s?=)
Date: Thu, 25 Jun 2015 15:49:16 +0900 (KST)
Subject: Python realted to question
Message-ID: <20150625154916.HM.a000000000EuLTj@skyjjogari.wwl1526.hanmail.net>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150625/2f9f4b28/attachment.html>

From steve+comp.lang.python at pearwood.info  Thu Jun 25 03:05:51 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 25 Jun 2015 17:05:51 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <558ba851$0$11089$c3e8da3@news.astraweb.com>

On Thursday 25 June 2015 14:07, Steven D'Aprano wrote:

>> You got it.  I didn't want to explain any more than necessary.  But yes,
>> the recipient just stores the data for the end-user.
> 
> Trust me. That's not all they are doing.

Hmm, sorry, that's a glib answer.

What I meant to say is, you can't *trust* that this is all they are doing, 
not unless all your users are within a single organisation where everyone 
trusts everyone else.

Obviously some people are more trustworthy, or less inquisitive, than 
others. But you don't know which ones are which.


-- 
Steven



From david at drmaciver.com  Thu Jun 25 03:53:23 2015
From: david at drmaciver.com (David MacIver)
Date: Thu, 25 Jun 2015 08:53:23 +0100
Subject: Looking for people who are using Hypothesis and are willing to
 say so
In-Reply-To: <871th0yctr.fsf@nightsong.com>
References: <mailman.22.1435168426.3674.python-list@python.org>
 <871th0yctr.fsf@nightsong.com>
Message-ID: <CADZYRLeeEB6kmafkX_EzWvR70Qiuv9Jug=vb5ZXXiYNHnEWytA@mail.gmail.com>

Actually one of the things that's helped the most in the course of
designing Hypothesis is the realisation that types are something of a red
herring for this sort of testing. Thinking purely in terms of custom
generators helps a lot, because it means you can do things like specify
size bounds on lists, integers, etc. as well as map and filter over the
resulting data (e.g. lists(integers().map(lambda x: x * 2), min_size=1,
max_size=10). Lists of length between 1 and 10 only containing even
integers. So in this regard the design of Hypothesis should be considered
more closely related to that of the Erlang than the Haskell quickcheck
(although I'd only ever used statically typed quickchecks before writing
Hypothesis).

In particular Hypothesis's strategies are best thought of in terms of how
to provide data as the argument rather than a type - you can't check
whether a given value is producable from a given strategy.

I was originally thinking it would be worth writing some auto derivation
functionality for using the new python 3 type annotations, but I actually
don't think it would be useful. You lose far too much flexibility.



On 24 June 2015 at 23:13, Paul Rubin <no.email at nospam.invalid> wrote:

> David MacIver <david at drmaciver.com> writes:
> > Author of Hypothesis here. (If you don't know what Hypothesis is, you're
> > probably not the target audience for this email but you should totally
> > check it out: https://hypothesis.readthedocs.org/
>
> Oh very cool: a QuickCheck-like unit test library.  I heard of something
> like that for Python recently, that might or might not have been
> Hypothesis.  I certainly plan to try it out.  The original QuickCheck
> (for Haskell) used the static type signatures on the functions under
> test to know what test cases to generate, but Erlang QuickCheck has had
> some good successes, including finding some subtle bugs during
> development in the HAMT (Clojure-like hash array mapped trie)
> implementation just released with Erlang/OTP 18.0 this week.
>
> I see Hypothesis use decorators that look sort of like Erlang Dialyzer
> so that can help with test cases.  Maybe later, it use Python 3 type
> annotations, though I think those are still much less precise than
> Dialyzer or Haskell types.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150625/17c97221/attachment.html>

From sjeik_appie at hotmail.com  Thu Jun 25 04:00:02 2015
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Thu, 25 Jun 2015 08:00:02 +0000 (UTC)
Subject: windows and file names > 256 bytes
Message-ID: <mmgce2$cd4$1@ger.gmane.org>

Hi,

Consider the following calls, where very_long_path is more than 256 bytes:
[1] os.mkdir(very_long_path)
[2] os.getsize(very_long_path)
[3] shutil.rmtree(very_long_path)

I am using Python 2.7 and [1] and [2] fail under Windows XP [3] fails 
under Win7 (not sure about XP). It throws: ?WindowsError: [Error 206] The 
filename or extension is too long? This is even when I use the "special" 
notations \\?\c:\dir\file or \\?\UNC\server\share\file, e.g.
os.path.getsize("\\\\?\\" + "c:\\dir\\file")
(Oddly, os.path.getsize(os.path.join("\\\\?", "c:\\dir\\file")) will 
truncate the prefix)

My questions:
1. How can I get the file size of very long paths under XP?
2. Is this a bug in Python? I would prefer if Python dealt with the gory 
details of Windows' silly behavior.

Regards,
Albert-Jan

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus




From info at egenix.com  Thu Jun 25 04:12:44 2015
From: info at egenix.com (eGenix Team: M.-A. Lemburg)
Date: Thu, 25 Jun 2015 10:12:44 +0200
Subject: ANN: eGenix mxODBC 3.3.3 - Python ODBC Database Interface
Message-ID: <558BB7FC.70402@egenix.com>

________________________________________________________________________

ANNOUNCING

                         eGenix.com mxODBC

                   Python ODBC Database Interface

                           Version 3.3.3


    mxODBC is our commercially supported Python extension providing
         ODBC database connectivity to Python applications
            on Windows, Mac OS X, Unix and BSD platforms
           with many advanced Python DB-API extensions and
                 full support of stored procedures


This announcement is also available on our web-site for online reading:
http://www.egenix.com/company/news/eGenix-mxODBC-3.3.3-GA.html

________________________________________________________________________

INTRODUCTION

mxODBC provides an easy-to-use, high-performance, reliable and robust
Python interface to ODBC compatible databases such as MS SQL Server,
Oracle Database, IBM DB2, Informix and Netezza, SAP Sybase ASE and
Sybase Anywhere, Teradata, MySQL, MariaDB, PostgreSQL, SAP MaxDB and
many more:

    http://www.egenix.com/products/python/mxODBC/

The "eGenix mxODBC - Python ODBC Database Interface" product is a
commercial extension to our open-source eGenix mx Base Distribution:

    http://www.egenix.com/products/python/mxBase/

________________________________________________________________________

NEWS

The 3.3.3 release of our mxODBC is a patch level release of our
popular Python ODBC Interface for Windows, Linux, Mac OS X and
FreeBSD. It includes these enhancements and fixes:

Driver Compatibility
--------------------

MS SQL Server

 MS SQL Server Native Client

 * Added a fix for the MS SQL Server Native Client error
   "[Microsoft][ODBC Driver 11 for SQL Server][SQL Server]The data
   types varchar and text are incompatible in the equal to operator."
   when trying to bind a string of more than 256 bytes to a *VARCHAR*
   column while using cursor.executedirect(). cursor.execute() was
   unaffected by this. Thanks to Paul Perez for reporting this.

 * Added a note to *avoid using "execute "* when calling stored
   procedures with MS SQL Server. This can result in '[Microsoft][SQL
   Native Client]Invalid Descriptor Index' errors. Simply dropping the
   "execute " will have the error go away.

 FreeTDS ODBC Driver

 * Added a work-around to address the FreeTDS driver error
   '[FreeTDS][SQL Server]The data types varbinary and image are
   incompatible in the equal to operator.' when trying to bind binary
   strings longer than 256 bytes to a *VARBINARY* column. This problem
   does not occur with the MS SQL Server Native Client.

 * Reenabled returning *cursor.rowcount* for FreeTDS >= 0.91. In
   previous versions, FreeTDS could return wrong data for .rowcount
   when using SELECTs.This should make *SQLAlchemy* users happy again.

 * Add work-around to have FreeTDS ODBC driver accept *binary data* in
   strings as input for VARBINARY columns. A side effect of this is
   that FreeTDS will now also accept binary data in VARCHAR columns.

SAP Sybase ASE

 * Added work-arounds and improvements for Sybase ASE ODBC drivers to
   enable working with *BINARY* and *VARBINARY* columns.

 * Added a work-around for a *cursor.rowcount* problem with Sybase ASE's
   ODBC driver on 64-bit platforms. It sometimes returns 4294967295
   instead of -1.

 * Added note about random segfault problems with the
   *Sybase ASE 15.7 ODBC driver* on Windows. Unfortunately, there's
   nothing much we can do about this, other than recommend using the
   Sybase ASE 15.5 ODBC driver version which does not have these
   stability problems.

Misc:

 * Added improved documentation on the *direct execution model*
   available in mxODBC. This can help in more complex parameter
   binding situations and also provides performance boosts for a few
   databases, including e.g. MS SQL Server.

 * Improved tests and added more data binding tests, esp. for SELECT
   queries with bound parameters.

 * Fixed some minor issues with the *package web installer* related to
   Linux2 vs. Linux3, FreeBSD installations and an intermittent error
   related to hash seeds, which sometimes caused prebuilt archives to
   not install correctly.

For the full set of changes please check the mxODBC change log:

    http://www.egenix.com/products/python/mxODBC/changelog.html

________________________________________________________________________

FEATURES

mxODBC 3.3 was released on 2014-04-08. Please see the full
announcement for highlights of the 3.3 release:

    http://www.egenix.com/company/news/eGenix-mxODBC-3.3.0-GA.html

For the full set of features mxODBC has to offer, please see:

    http://www.egenix.com/products/python/mxODBC/#Features

________________________________________________________________________

EDITIONS

mxODBC is available in these two editions:

 * The Professional Edition, which gives full access to all mxODBC features.

 * The Product Development Edition, which allows including mxODBC in
   applications you develop.

For a complete overview of the available editions, please see the
product page:

    http://www.egenix.com/products/python/mxODBC/#mxODBCEditions

________________________________________________________________________

DOWNLOADS

The download archives and instructions for installing the package can
be found at:

    http://www.egenix.com/products/python/mxODBC/

In order to use the eGenix mxODBC package you will first need to
install the eGenix mx Base package:

    http://www.egenix.com/products/python/mxBase/

You can also simply use:

    pip install egenix-mxodbc

and then get evaluation licenses from our website to try mxODBC:

    http://www.egenix.com/products/python/mxODBC/#Evaluation

________________________________________________________________________

UPGRADING

Users are encouraged to upgrade to this latest mxODBC release to
benefit from the new features and updated ODBC driver support.

We have taken special care not to introduce backwards incompatible
changes, making the upgrade experience as smooth as possible.

Customers who have purchased mxODBC 3.3 licenses can continue to use
their licenses with this patch level release.

For upgrade purchases, we will give out 20% discount coupons going
from mxODBC 2.x to 3.3 and 50% coupons for upgrades from mxODBC 3.x to
3.3. Please contact the eGenix.com Sales Team with your existing
license serials for details for an upgrade discount coupon.

If you want to try the new release before purchase, you can request
30-day evaluation licenses by visiting our web-site

    http://www.egenix.com/products/python/mxODBC/#Evaluation

or writing to sales at egenix.com, stating your name (or the name of the
company) and the number of eval licenses that you need.

_______________________________________________________________________

SUPPORT

Commercial support for this product is available from eGenix.com.
Please see

    http://www.egenix.com/services/support/

for details about our support offerings.

_______________________________________________________________________

INFORMATION

About Python (http://www.python.org/):

    Python is an object-oriented Open Source programming language
    which runs on all modern platforms. By integrating ease-of-use,
    clarity in coding, enterprise application connectivity and rapid
    application design, Python establishes an ideal programming
    platform for today's IT challenges.

About eGenix (http://www.egenix.com/):

    eGenix is a software project, consulting and product company
    focusing on expert services and professional quality products for
    companies, Python users and developers.

Enjoy,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 25 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2015-06-16: Released eGenix pyOpenSSL 0.13.10 ... http://egenix.com/go78
2015-07-20: EuroPython 2015, Bilbao, Spain ...             25 days to go
2015-07-29: Python Meeting Duesseldorf ...                 34 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From steve+comp.lang.python at pearwood.info  Thu Jun 25 05:16:31 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 25 Jun 2015 19:16:31 +1000
Subject: windows and file names > 256 bytes
References: <mailman.44.1435219506.3674.python-list@python.org>
Message-ID: <558bc6f1$0$11104$c3e8da3@news.astraweb.com>

On Thursday 25 June 2015 18:00, Albert-Jan Roskam wrote:

> Hi,
> 
> Consider the following calls, where very_long_path is more than 256 bytes:
> [1] os.mkdir(very_long_path)
> [2] os.getsize(very_long_path)
> [3] shutil.rmtree(very_long_path)
> 
> I am using Python 2.7 and [1] and [2] fail under Windows XP [3] fails
> under Win7 (not sure about XP). It throws: ?WindowsError: [Error 206] The
> filename or extension is too long? 

I don't think this is a bug. It seems to be a limitation of Windows.

https://msdn.microsoft.com/en-
us/library/windows/desktop/aa365247%28v=vs.85%29.aspx#maxpath

> This is even when I use the "special"
> notations \\?\c:\dir\file or \\?\UNC\server\share\file, e.g.
> os.path.getsize("\\\\?\\" + "c:\\dir\\file")

However, that may be a bug.

What happens if you use a Unicode string?

path = u"\\\\?\\c:a\\very\\long\\path"
os.mkdir(path)


Can you open an existing file?

open(u"\\\\?\\c:a\\very\\long\\path\\file.txt")



> (Oddly, os.path.getsize(os.path.join("\\\\?", "c:\\dir\\file")) will
> truncate the prefix)

That's worth reporting as a bug.


> My questions:
> 1. How can I get the file size of very long paths under XP?

If all else fails:

last = os.getcwd()
try:
    os.chdir('C:/a/very/long')
    os.chdir('path/with/many')
    os.chdir('nested/folders')
    os.path.getsize('/and/even/more/file.txt')
finally:
    os.chdir(last)


> 2. Is this a bug in Python? I would prefer if Python dealt with the gory
> details of Windows' silly behavior.

I would say that it is a bug that it doesn't work with extended-length paths 
(those starting with \\?\) but may or may not be a bug with regular paths.


-- 
Steve



From rosuav at gmail.com  Thu Jun 25 05:23:16 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 25 Jun 2015 19:23:16 +1000
Subject: windows and file names > 256 bytes
In-Reply-To: <558bc6f1$0$11104$c3e8da3@news.astraweb.com>
References: <mailman.44.1435219506.3674.python-list@python.org>
 <558bc6f1$0$11104$c3e8da3@news.astraweb.com>
Message-ID: <CAPTjJmqOK63ifOPJXio4s25J6buxuoFFamFgZrMzpWJDGKSV1A@mail.gmail.com>

On Thu, Jun 25, 2015 at 7:16 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
>> 2. Is this a bug in Python? I would prefer if Python dealt with the gory
>> details of Windows' silly behavior.
>
> I would say that it is a bug that it doesn't work with extended-length paths
> (those starting with \\?\) but may or may not be a bug with regular paths.

I'd go further and say that the OP is right in expecting Python to
deal with the gory details. Would it break anything for Python to
prepend \\?\ to all file names before giving them to certain APIs?
Then the current behaviour of stripping off that prefix would be fine.

Are there any times when you *don't* want Windows to use the
extended-length path?

ChrisA


From steve+comp.lang.python at pearwood.info  Thu Jun 25 05:25:37 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 25 Jun 2015 19:25:37 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
Message-ID: <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>

On Thursday 25 June 2015 14:27, Devin Jeanpierre wrote:

> On Wed, Jun 24, 2015 at 9:07 PM, Steven D'Aprano <steve at pearwood.info>
> wrote:
>> But just sticking to the three above, the first one is partially
>> mitigated by allowing virus scanners to scan the data, but that implies
>> that the owner of the storage machine can spy on the files. So you have a
>> conflict here.
> 
> If it's encrypted malware, and you can't decrypt it, there's no threat.

If the *only* threat is that the sender will send malware, you can mitigate 
around that by dropping the file in an unencrypted container. Anything good 
enough to prevent Windows from executing the code, accidentally or 
deliberately, say, a tar file with a custom extension.

But encrypting the file is also a good solution, and it prevents the storage 
machine spying on the file contents too. Provided the encryption is strong.


>> Honestly, the *only* real defence against the spying issue is to encrypt
>> the files. Not obfuscate them with a lousy random substitution cipher.
>> The storage machine can keep the files as long as they like, just by
>> making a copy, and spend hours bruteforcing them. They *will* crack the
>> substitution cipher. In pure Python, that may take a few days or weeks;
>> in C, hours or days. If they have the resources to throw at it, minutes.
>> Substitution ciphers have not been effective encryption since, oh, the
>> 1950s, unless you use a one-time pad. Which you won't be.
> 
> The original post said that the sender will usually send files they
> encrypted, unless they are malicious. So if the sender wants them to
> be encrypted, they already are.

The OP *hopes* that the sender will encrypt the files. I think that's a 
vanishingly faint hope, unless the application itself encrypts the file.

Most people don't have any encryption software beyond password-protecting 
zip files. Zip 2.0 legacy encryption is crap, and there are plenty of tools 
available to break it. Winzip has an extension for 128-bit and 256-bit AES 
encryption, both of which are probably strong enough unless you're targeted 
by the NSA, but the weak link in the chain is the idea that people will 
encrypt the software before sending it. Even if they have the tools, 
laziness being the defining characteristic of most people, they won't use 
them.

> "While the data senders are supposed to encrypt data, that's not
> guaranteed, and I'd like to protect the recipient against exposure to
> nefarious data by mangling or encrypting the data before it is written
> to disk."
> 
> The cipher is just to keep the sender from being able to control what
> is on disk.

The sender has a copy of the application? Then they can see the type of 
obfuscation used. If they know the key, or can guess it, they can take their 
malware, *decrypt* it, and send that, so that *encrypting* that file puts 
the malicious code on the disk.

E.g. suppose I want to send you an insult, but I know your program 
automatically ROT-13s the strings I send you. Then I send you:

'lbhe sngure fzryyf bs ryqreoreevrf'

and your program ROT-13s it to:

'your father smells of elderberries'

I know that the OP doesn't propose using ROT-13, but a classical 
substitution cipher isn't that much stronger.


> I am usually very oppositional when it comes to rolling your own
> crypto, but am I alone here in thinking the OP very clearly laid out
> their case?


I don't think any of us *really* understand his use-case or the potential 
threats, but to my way of thinking, you can never have too strong a cipher 
or underestimate the risk of users taking short-cuts.



-- 
Steve



From jeanpierreda at gmail.com  Thu Jun 25 05:41:51 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Thu, 25 Jun 2015 02:41:51 -0700
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
Message-ID: <CABicbJ+sgkp4WZOMUETDoU0GeGf8BzNtWiBSAtYaH7u7NaXOXA@mail.gmail.com>

On Thu, Jun 25, 2015 at 2:25 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Thursday 25 June 2015 14:27, Devin Jeanpierre wrote:
>> The original post said that the sender will usually send files they
>> encrypted, unless they are malicious. So if the sender wants them to
>> be encrypted, they already are.
>
> The OP *hopes* that the sender will encrypt the files. I think that's a
> vanishingly faint hope, unless the application itself encrypts the file.
>
> Most people don't have any encryption software beyond password-protecting
> zip files. Zip 2.0 legacy encryption is crap, and there are plenty of tools
> available to break it. Winzip has an extension for 128-bit and 256-bit AES
> encryption, both of which are probably strong enough unless you're targeted
> by the NSA, but the weak link in the chain is the idea that people will
> encrypt the software before sending it. Even if they have the tools,
> laziness being the defining characteristic of most people, they won't use
> them.

You're right, I was supposing that since they wrote the server, they
also wrote the client, and were just protecting from the protocol
itself being weak.

> I know that the OP doesn't propose using ROT-13, but a classical
> substitution cipher isn't that much stronger.

Yes, it is. It requires the attacker being able to see something about
the ciphertext, unlike ROT13. But it is reasonable to suppose that
maybe the attacker can trigger the file getting executed, at which
point maybe you can deduce from the behavior what the starting bytes
are...?

> I don't think any of us *really* understand his use-case or the potential
> threats, but to my way of thinking, you can never have too strong a cipher
> or underestimate the risk of users taking short-cuts.

This is truth. It would be nice if something like keyczar came in the stdlib.

(Otherwise, users of Python take shortcuts and use randomized
substitution ciphers instead of AES.)

-- Devin


From rosuav at gmail.com  Thu Jun 25 05:57:20 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 25 Jun 2015 19:57:20 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CABicbJ+sgkp4WZOMUETDoU0GeGf8BzNtWiBSAtYaH7u7NaXOXA@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <CABicbJ+sgkp4WZOMUETDoU0GeGf8BzNtWiBSAtYaH7u7NaXOXA@mail.gmail.com>
Message-ID: <CAPTjJmp242O_U+=JvP_qMY_zSO+FHMp3kb=Sg2G-QWgFjY_Gxw@mail.gmail.com>

On Thu, Jun 25, 2015 at 7:41 PM, Devin Jeanpierre
<jeanpierreda at gmail.com> wrote:
>> I know that the OP doesn't propose using ROT-13, but a classical
>> substitution cipher isn't that much stronger.
>
> Yes, it is. It requires the attacker being able to see something about
> the ciphertext, unlike ROT13. But it is reasonable to suppose that
> maybe the attacker can trigger the file getting executed, at which
> point maybe you can deduce from the behavior what the starting bytes
> are...?
>

If a symmetric cipher is being used and the key is known, anyone can
simply perform a decryption operation on the desired bytes, get back a
pile of meaningless encrypted junk, and submit that. When it's
encrypted with the same key, voila! The cleartext will reappear.

Asymmetric ciphers are a bit different, though. AIUI you can't perform
a decryption without the private key, whereas you can encrypt with
only the public key. So you ought to be safe on that one; the only way
someone could deliberately craft input that, when encrypted with your
public key, produces a specific set of bytes, would be to brute-force
it. (But I might be wrong on that. I'm no crypto expert.)

ChrisA


From liik.joonas at gmail.com  Thu Jun 25 06:00:33 2015
From: liik.joonas at gmail.com (Joonas Liik)
Date: Thu, 25 Jun 2015 13:00:33 +0300
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CABicbJ+sgkp4WZOMUETDoU0GeGf8BzNtWiBSAtYaH7u7NaXOXA@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <CABicbJ+sgkp4WZOMUETDoU0GeGf8BzNtWiBSAtYaH7u7NaXOXA@mail.gmail.com>
Message-ID: <CAB1GNpRyu6FgaiJhCRTCg-jva1Fb5SZa=Hgsz2ni24AABPNn1w@mail.gmail.com>

Personally, i have had AVG give at least 2 false positives (fyi one of
them was like python2.6)

as long as antivirus software can give so many false positives i would
thing preventing your AV from nuking someone elses data is a
reasonable thing.


From rosuav at gmail.com  Thu Jun 25 06:02:21 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 25 Jun 2015 20:02:21 +1000
Subject: 404 Error when using local CGI Server
In-Reply-To: <CAKDUBcvmfCy45r+Nx+1a7=O3apZLnMTC8V98kOmTmoJnr6=L9A@mail.gmail.com>
References: <CAKDUBcvmfCy45r+Nx+1a7=O3apZLnMTC8V98kOmTmoJnr6=L9A@mail.gmail.com>
Message-ID: <CAPTjJmrZA65pVVSdaU7R0-RcyaJ6n_smTVFTqmc=ZUus8odbrg@mail.gmail.com>

On Thu, Jun 25, 2015 at 3:48 AM, Charles Carr <chcarr at sas.upenn.edu> wrote:
> I am running a local cgi server from python on a windows 7 computer.
> Whenever I try to serve the output of a cgi file by entering the following
> into my browser: http://localhost:8080/filename.py , I get a 404 error
> message that the file was not found. I'm positive that the files I am trying
> to serve are in the same directory as the server script that is running. Are
> there any tips as to where I should save my files in order to avoid this
> error?

It depends entirely on how your server is set up. What I would
recommend is completely ignoring the file system, and designing a web
site using one of the frameworks that are available for Python, such
as Flask or Django. Your URLs are defined in your code; you can have a
'static' directory from which simple files (images, CSS, etc) get
served, but the file system does not define executable entry points.
This avoids the *massive* problems of PHP, where an attacker can
upgrade a file delivery exploit into remote code execution; the worst
that can happen with Python+Flask+static is that the file gets
uploaded into static/ and is then available as-is for download (it
won't be run on the server).

ChrisA


From jon+usenet at unequivocal.co.uk  Thu Jun 25 06:03:52 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Thu, 25 Jun 2015 10:03:52 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
Message-ID: <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-25, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
> On Thursday 25 June 2015 14:27, Devin Jeanpierre wrote:
>> If it's encrypted malware, and you can't decrypt it, there's no threat.
>
> If the *only* threat is that the sender will send malware, you can mitigate 
> around that by dropping the file in an unencrypted container. Anything good 
> enough to prevent Windows from executing the code, accidentally or 
> deliberately, say, a tar file with a custom extension.

That won't stop virus scanners etc potentially making their own minds
up about the file.

> But encrypting the file is also a good solution, and it prevents the storage 
> machine spying on the file contents too. Provided the encryption is strong.

How would the receiver encrypting the file after receiving it prevent
the receiver from seeing what's in the file?

>> The original post said that the sender will usually send files they
>> encrypted, unless they are malicious. So if the sender wants them to
>> be encrypted, they already are.
>
> The OP *hopes* that the sender will encrypt the files. I think that's a 
> vanishingly faint hope, unless the application itself encrypts the file.

Yes, the application itself encrypts the file. Haven't you been
reading what he's saying?

> The sender has a copy of the application? Then they can see the type of 
> obfuscation used. If they know the key, or can guess it, they can take their 
> malware, *decrypt* it, and send that, so that *encrypting* that file puts 
> the malicious code on the disk.

Not if they don't know the key they can't.

> E.g. suppose I want to send you an insult, but I know your program 
> automatically ROT-13s the strings I send you. Then I send you:
>
> 'lbhe sngure fzryyf bs ryqreoreevrf'
>
> and your program ROT-13s it to:
>
> 'your father smells of elderberries'
>
> I know that the OP doesn't propose using ROT-13, but a classical 
> substitution cipher isn't that much stronger.

Replace "ROT-13" with "ROT-n" where 'n' is a secret known only to the
receiver, and suddenly it's not such a bad method of obfuscation.
Improve it to the random-translation-map method he's actually using
and you've got really quite a reasonable system.

>> I am usually very oppositional when it comes to rolling your own
>> crypto, but am I alone here in thinking the OP very clearly laid out
>> their case?
>
> I don't think any of us *really* understand his use-case or the potential 
> threats, but to my way of thinking, you can never have too strong a cipher 
> or underestimate the risk of users taking short-cuts.

The use case is pretty obvious (a peer-to-peer dropbox type thing) but
it does appear to be being misunderstood. This isn't actually a crypto
problem at all and "users taking short-cuts" isn't an issue.


From mail at timgolden.me.uk  Thu Jun 25 06:10:14 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 25 Jun 2015 11:10:14 +0100
Subject: windows and file names > 256 bytes
In-Reply-To: <CAPTjJmqOK63ifOPJXio4s25J6buxuoFFamFgZrMzpWJDGKSV1A@mail.gmail.com>
References: <mailman.44.1435219506.3674.python-list@python.org>
 <558bc6f1$0$11104$c3e8da3@news.astraweb.com>
 <CAPTjJmqOK63ifOPJXio4s25J6buxuoFFamFgZrMzpWJDGKSV1A@mail.gmail.com>
Message-ID: <558BD386.8030207@timgolden.me.uk>

On 25/06/2015 10:23, Chris Angelico wrote:
> On Thu, Jun 25, 2015 at 7:16 PM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>>> 2. Is this a bug in Python? I would prefer if Python dealt with the gory
>>> details of Windows' silly behavior.
>>
>> I would say that it is a bug that it doesn't work with extended-length paths
>> (those starting with \\?\) but may or may not be a bug with regular paths.
> 
> I'd go further and say that the OP is right in expecting Python to
> deal with the gory details. Would it break anything for Python to
> prepend \\?\ to all file names before giving them to certain APIs?
> Then the current behaviour of stripping off that prefix would be fine.
> 
> Are there any times when you *don't* want Windows to use the
> extended-length path?

Yes: when you're passing a relative filepath. Which could pretty much be
any time. As you might imagine, this has come up before -- there's an
issue on the tracker for it somewhere. I just don't think it's simple
enough for Python to know when and when not to use the extended path
syntax without danger of breaking something.

Bear in mind that the \\?\ prefix doesn't just extend the length: it
also allows otherwise special-cased characters such as "." or "..". It's
a general-purpose mechanism for handing something straight to the file
system without parsing it first.

TJG



From jeanpierreda at gmail.com  Thu Jun 25 06:18:26 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Thu, 25 Jun 2015 03:18:26 -0700
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CAPTjJmp242O_U+=JvP_qMY_zSO+FHMp3kb=Sg2G-QWgFjY_Gxw@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <CABicbJ+sgkp4WZOMUETDoU0GeGf8BzNtWiBSAtYaH7u7NaXOXA@mail.gmail.com>
 <CAPTjJmp242O_U+=JvP_qMY_zSO+FHMp3kb=Sg2G-QWgFjY_Gxw@mail.gmail.com>
Message-ID: <CABicbJLQzKrxTcOSdS-3iKfUUJWW9V=fbm6fTNX2SHzLx3tz_Q@mail.gmail.com>

On Thu, Jun 25, 2015 at 2:57 AM, Chris Angelico <rosuav at gmail.com> wrote:
> On Thu, Jun 25, 2015 at 7:41 PM, Devin Jeanpierre
> <jeanpierreda at gmail.com> wrote:
>>> I know that the OP doesn't propose using ROT-13, but a classical
>>> substitution cipher isn't that much stronger.
>>
>> Yes, it is. It requires the attacker being able to see something about
>> the ciphertext, unlike ROT13. But it is reasonable to suppose that
>> maybe the attacker can trigger the file getting executed, at which
>> point maybe you can deduce from the behavior what the starting bytes
>> are...?
>>
>
> If a symmetric cipher is being used and the key is known, anyone can
> simply perform a decryption operation on the desired bytes, get back a
> pile of meaningless encrypted junk, and submit that. When it's
> encrypted with the same key, voila! The cleartext will reappear.
>
> Asymmetric ciphers are a bit different, though. AIUI you can't perform
> a decryption without the private key, whereas you can encrypt with
> only the public key. So you ought to be safe on that one; the only way
> someone could deliberately craft input that, when encrypted with your
> public key, produces a specific set of bytes, would be to brute-force
> it. (But I might be wrong on that. I'm no crypto expert.)

Yes, so it should be random.

-- Devin


From rosuav at gmail.com  Thu Jun 25 06:23:16 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 25 Jun 2015 20:23:16 +1000
Subject: windows and file names > 256 bytes
In-Reply-To: <558BD386.8030207@timgolden.me.uk>
References: <mailman.44.1435219506.3674.python-list@python.org>
 <558bc6f1$0$11104$c3e8da3@news.astraweb.com>
 <CAPTjJmqOK63ifOPJXio4s25J6buxuoFFamFgZrMzpWJDGKSV1A@mail.gmail.com>
 <558BD386.8030207@timgolden.me.uk>
Message-ID: <CAPTjJmpMxG7iL8jiv+Tj4f93tteU06o+A7hoa8qpd867Br3paw@mail.gmail.com>

On Thu, Jun 25, 2015 at 8:10 PM, Tim Golden <mail at timgolden.me.uk> wrote:
>> Are there any times when you *don't* want Windows to use the
>> extended-length path?
>
> Yes: when you're passing a relative filepath. Which could pretty much be
> any time. As you might imagine, this has come up before -- there's an
> issue on the tracker for it somewhere. I just don't think it's simple
> enough for Python to know when and when not to use the extended path
> syntax without danger of breaking something.

Oh blah. So I suppose that means there's fundamentally no way to use a
long (>256 byte) relative path on Windows?

> Bear in mind that the \\?\ prefix doesn't just extend the length: it
> also allows otherwise special-cased characters such as "." or "..". It's
> a general-purpose mechanism for handing something straight to the file
> system without parsing it first.

Ohh. So... hmm. So what this really means is that a path could get
\\?\ prepended when, and ONLY when, it becomes absolute. Windows can
be a real pest...

ChrisA


From breamoreboy at yahoo.co.uk  Thu Jun 25 06:58:09 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 25 Jun 2015 11:58:09 +0100
Subject: Windows install error
In-Reply-To: <1435161384.7311.YahooMailWin8@web141002.mail.bf1.yahoo.com>
References: <1435161384.7311.YahooMailWin8@web141002.mail.bf1.yahoo.com>
Message-ID: <mmgms5$mik$1@ger.gmane.org>

On 24/06/2015 16:56, Knss Teja via Python-list wrote:
> I WANT TO install 4.3 version ... but the MSI file is giving a DLL error
> .. what should I do :/
> please use REPLY ALL .. so that I get the mail to my gmail inbox
>

I'll assume that you mean 3.4.x.  Please give the x, your Windows 
version and the precise error message that you're getting, then we 
should be able to help.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Thu Jun 25 07:06:34 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 25 Jun 2015 12:06:34 +0100
Subject: windows and file names > 256 bytes
In-Reply-To: <mmgce2$cd4$1@ger.gmane.org>
References: <mmgce2$cd4$1@ger.gmane.org>
Message-ID: <mmgnbt$u0o$1@ger.gmane.org>

On 25/06/2015 09:00, Albert-Jan Roskam wrote:
> Hi,
>
> Consider the following calls, where very_long_path is more than 256 bytes:
> [1] os.mkdir(very_long_path)
> [2] os.getsize(very_long_path)
> [3] shutil.rmtree(very_long_path)
>
> I am using Python 2.7 and [1] and [2] fail under Windows XP [3] fails
> under Win7 (not sure about XP). It throws: ?WindowsError: [Error 206] The
> filename or extension is too long? This is even when I use the "special"
> notations \\?\c:\dir\file or \\?\UNC\server\share\file, e.g.
> os.path.getsize("\\\\?\\" + "c:\\dir\\file")
> (Oddly, os.path.getsize(os.path.join("\\\\?", "c:\\dir\\file")) will
> truncate the prefix)
>
> My questions:
> 1. How can I get the file size of very long paths under XP?

Please see 
https://msdn.microsoft.com/en-gb/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath

> 2. Is this a bug in Python? I would prefer if Python dealt with the gory
> details of Windows' silly behavior.

I don't see why Python should work around any particular limitation of 
any given OS.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From liik.joonas at gmail.com  Thu Jun 25 08:04:37 2015
From: liik.joonas at gmail.com (Joonas Liik)
Date: Thu, 25 Jun 2015 15:04:37 +0300
Subject: windows and file names > 256 bytes
In-Reply-To: <mmgnbt$u0o$1@ger.gmane.org>
References: <mmgce2$cd4$1@ger.gmane.org>
	<mmgnbt$u0o$1@ger.gmane.org>
Message-ID: <CAB1GNpS=7F3ebrHenUu7+MvyWuk5v60G6N6L81c0W3C-mQPi2w@mail.gmail.com>

It sounds to me more like it is possible to use long file names on windows
but it is a pain and in python, on windows it is basically impossible.

So shouldn't it be possible to maniulate these files with extended names..

I mean even if you had to use some special function to ask for long names
it would still be better than no support at all.


From mail at timgolden.me.uk  Thu Jun 25 08:34:36 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 25 Jun 2015 13:34:36 +0100
Subject: windows and file names > 256 bytes
In-Reply-To: <CAB1GNpS=7F3ebrHenUu7+MvyWuk5v60G6N6L81c0W3C-mQPi2w@mail.gmail.com>
References: <mmgce2$cd4$1@ger.gmane.org> <mmgnbt$u0o$1@ger.gmane.org>
 <CAB1GNpS=7F3ebrHenUu7+MvyWuk5v60G6N6L81c0W3C-mQPi2w@mail.gmail.com>
Message-ID: <558BF55C.2010701@timgolden.me.uk>

On 25/06/2015 13:04, Joonas Liik wrote:
> It sounds to me more like it is possible to use long file names on windows
> but it is a pain and in python, on windows it is basically impossible.

Certainly not impossible: you could write your own wrapper function:

def extended_path(p):
    return r"\\?\%s" % os.path.abspath(p)

where you knew that there was a possibility of long paths and that an
absolute path would work.

TJG


From lac at openend.se  Thu Jun 25 08:48:05 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 25 Jun 2015 14:48:05 +0200
Subject: Windows install error
In-Reply-To: Message from Mark Lawrence <breamoreboy@yahoo.co.uk>
 of "Thu, 25 Jun 2015 11:58:09 +0100." <mmgms5$mik$1@ger.gmane.org>
References: <1435161384.7311.YahooMailWin8@web141002.mail.bf1.yahoo.com><mmgms5$mik$1@ger.gmane.org>
Message-ID: <201506251248.t5PCm5bB024153@fido.openend.se>

In a message of Thu, 25 Jun 2015 11:58:09 +0100, Mark Lawrence writes:
>On 24/06/2015 16:56, Knss Teja via Python-list wrote:
>> I WANT TO install 4.3 version ... but the MSI file is giving a DLL error
>> .. what should I do :/
>> please use REPLY ALL .. so that I get the mail to my gmail inbox
>>
>
>I'll assume that you mean 3.4.x.  Please give the x, your Windows 
>version and the precise error message that you're getting, then we 
>should be able to help.
>
>-- 
>My fellow Pythonistas, ask not what our language can do for you, ask
>what you can do for our language.
>
>Mark Lawrence

Note that some people I know of, via webmaster, found that they
could install 3.4.x with the ActiveState installer but not the python.org
one.  Apparantly ActiveState bundles up some DLLs in its installer
'in case the user doesn't have them' whereas we don't -- and turns out
some users don't have them.  I have never been able to get enough out
of a user to find out exactly what magic sauce they have that we do
not -- it would be really nice to find out so that we can add it too.

No guarantees that this particular user has this particular problem,
of course.

Laura



From rosuav at gmail.com  Thu Jun 25 09:02:00 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 25 Jun 2015 23:02:00 +1000
Subject: windows and file names > 256 bytes
In-Reply-To: <mmgnbt$u0o$1@ger.gmane.org>
References: <mmgce2$cd4$1@ger.gmane.org>
	<mmgnbt$u0o$1@ger.gmane.org>
Message-ID: <CAPTjJmqJMQwJTuSm=Pivj2quNkbkW3_nYVyLwT2J5z50fK1JKA@mail.gmail.com>

On Thu, Jun 25, 2015 at 9:06 PM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>> 2. Is this a bug in Python? I would prefer if Python dealt with the gory
>> details of Windows' silly behavior.
>
>
> I don't see why Python should work around any particular limitation of any
> given OS.

Check out the multiprocessing module, and then tell me whether it's
better that Python paper over the OS differences or if you'd rather do
all that yourself. The biggest difference left between Windows and
POSIX is that on Windows, your main module has to be importable (which
doesn't hurt on POSIX). Python deals with all the mess of "can we
fork, or do we have to do it differently?".

ChrisA


From torriem at gmail.com  Thu Jun 25 09:30:35 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 25 Jun 2015 07:30:35 -0600
Subject: No subject
In-Reply-To: <1435161384.7311.YahooMailWin8@web141002.mail.bf1.yahoo.com>
References: <1435161384.7311.YahooMailWin8@web141002.mail.bf1.yahoo.com>
Message-ID: <558C027B.6050006@gmail.com>

On 06/24/2015 09:56 AM, Knss Teja via Python-list wrote:
> I WANT TO install 4.3  version ... but the MSI file is giving a DLL error .. what should I do :/
> please use REPLY ALL .. so that I get the mail to my gmail inbox

No idea what you mean about wanting to get mail to your gmail inbox...
I'd think the mailing list would do just that.

Anyway, what is your DLL error?


From torriem at gmail.com  Thu Jun 25 09:35:06 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 25 Jun 2015 07:35:06 -0600
Subject: windows and file names > 256 bytes
In-Reply-To: <558BF55C.2010701@timgolden.me.uk>
References: <mmgce2$cd4$1@ger.gmane.org> <mmgnbt$u0o$1@ger.gmane.org>
 <CAB1GNpS=7F3ebrHenUu7+MvyWuk5v60G6N6L81c0W3C-mQPi2w@mail.gmail.com>
 <558BF55C.2010701@timgolden.me.uk>
Message-ID: <558C038A.5010103@gmail.com>

On 06/25/2015 06:34 AM, Tim Golden wrote:
> On 25/06/2015 13:04, Joonas Liik wrote:
>> It sounds to me more like it is possible to use long file names on windows
>> but it is a pain and in python, on windows it is basically impossible.
> 
> Certainly not impossible: you could write your own wrapper function:
> 
> def extended_path(p):
>     return r"\\?\%s" % os.path.abspath(p)
> 
> where you knew that there was a possibility of long paths and that an
> absolute path would work.

The OP mentions that even when he manually supplies extended paths,
os.mkdir, os.getsize, and shutil.rmtree return errors for him in Python
2.7.  So there's more to this problem.


From mail at timgolden.me.uk  Thu Jun 25 09:37:55 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 25 Jun 2015 14:37:55 +0100
Subject: windows and file names > 256 bytes
In-Reply-To: <558C038A.5010103@gmail.com>
References: <mmgce2$cd4$1@ger.gmane.org> <mmgnbt$u0o$1@ger.gmane.org>
 <CAB1GNpS=7F3ebrHenUu7+MvyWuk5v60G6N6L81c0W3C-mQPi2w@mail.gmail.com>
 <558BF55C.2010701@timgolden.me.uk> <558C038A.5010103@gmail.com>
Message-ID: <558C0433.2050601@timgolden.me.uk>

On 25/06/2015 14:35, Michael Torrie wrote:
> On 06/25/2015 06:34 AM, Tim Golden wrote:
>> On 25/06/2015 13:04, Joonas Liik wrote:
>>> It sounds to me more like it is possible to use long file names on windows
>>> but it is a pain and in python, on windows it is basically impossible.
>>
>> Certainly not impossible: you could write your own wrapper function:
>>
>> def extended_path(p):
>>     return r"\\?\%s" % os.path.abspath(p)
>>
>> where you knew that there was a possibility of long paths and that an
>> absolute path would work.
> 
> The OP mentions that even when he manually supplies extended paths,
> os.mkdir, os.getsize, and shutil.rmtree return errors for him in Python
> 2.7.  So there's more to this problem.
> 

He's probably not passing unicode strings: the extended path only works
for unicode string. For 3.x that's what you do by default.

TJG


From steve at pearwood.info  Thu Jun 25 11:13:02 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 26 Jun 2015 01:13:02 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>

On Thu, 25 Jun 2015 08:03 pm, Jon Ribbens wrote:

> On 2015-06-25, Steven D'Aprano <steve+comp.lang.python at pearwood.info>
> wrote:
>> On Thursday 25 June 2015 14:27, Devin Jeanpierre wrote:
>>> If it's encrypted malware, and you can't decrypt it, there's no threat.
>>
>> If the *only* threat is that the sender will send malware, you can
>> mitigate around that by dropping the file in an unencrypted container.
>> Anything good enough to prevent Windows from executing the code,
>> accidentally or deliberately, say, a tar file with a custom extension.
> 
> That won't stop virus scanners etc potentially making their own minds
> up about the file.

*shrug* Sure, but I was specifically referring to the risk of the malware
being executed, not being detected by a virus scanner.

Encrypting the file won't even necessarily stop the virus scanner from
finding false positives. It might even increase the chances. But it will
prevent the virus scanner from finding actual viruses. You may or may not
consider that a problem.


>> But encrypting the file is also a good solution, and it prevents the
>> storage machine spying on the file contents too. Provided the encryption
>> is strong.
> 
> How would the receiver encrypting the file after receiving it prevent
> the receiver from seeing what's in the file?

I didn't say it ought to be encrypted by the receiver. Obviously the
encryption needs to be done in a way that the recipient doesn't get access
to the key. The obvious way to do that is for the application to encrypt
the data before it sends it. Then the receiver just writes the encrypted
bytes directly to a file. That would have the benefit of protecting against
man-in-the-middle attacks as well, since the file is never transmitted in
the clear.

 
>>> The original post said that the sender will usually send files they
>>> encrypted, unless they are malicious. So if the sender wants them to
>>> be encrypted, they already are.
>>
>> The OP *hopes* that the sender will encrypt the files. I think that's a
>> vanishingly faint hope, unless the application itself encrypts the file.
> 
> Yes, the application itself encrypts the file. Haven't you been
> reading what he's saying?

I have been reading what the OP has been saying. I'm not sure if you have
been. The OP doesn't want to encrypt the file, because he wants the
application to be pure Python and encryption in pure Python is too slow. So
he wants to obfuscate it with some sort of substitution cipher or
equivalent, which may be easily crackable by anyone who really wants to.

I've been arguing that the application *should* encrypt the file, and not
mess about giving the illusion of security.


>> The sender has a copy of the application? Then they can see the type of
>> obfuscation used. If they know the key, or can guess it, they can take
>> their malware, *decrypt* it, and send that, so that *encrypting* that
>> file puts the malicious code on the disk.
> 
> Not if they don't know the key they can't.

"If they know the key, or can guess it, ..."
"Not if they don't know the key they can't."

Really? Glad you're around to point that out to me.

But seriously, they have the application. If the application is using a
symmetric substitution cipher, it needs the key (because there is only
one), so the receiver will have the cipher.

With the sort of substitution cipher the OP is experimenting with, forcing a
particular result is trivially easy. The sender has access to the
application, knows the cipher, knows the key, and can easily generate a
file which will generate whatever content the sender wants after being
obfuscated.

Modern asymmetric ciphers like AES are quite resistant to that sort of
attack. There is, so far as I know, no way to generate a file which results
in a specific content after encryption.


>> E.g. suppose I want to send you an insult, but I know your program
>> automatically ROT-13s the strings I send you. Then I send you:
>>
>> 'lbhe sngure fzryyf bs ryqreoreevrf'
>>
>> and your program ROT-13s it to:
>>
>> 'your father smells of elderberries'
>>
>> I know that the OP doesn't propose using ROT-13, but a classical
>> substitution cipher isn't that much stronger.
> 
> Replace "ROT-13" with "ROT-n" where 'n' is a secret known only to the
> receiver, and suddenly it's not such a bad method of obfuscation.

There are only 256 possible values for n, one of which doesn't transform the
data at all (ROT-0). If you're thinking of attacking this by pencil and
paper, 255 transformations sounds like a lot. For a computer, that's barely
harder than a single transformation.


> Improve it to the random-translation-map method he's actually using
> and you've got really quite a reasonable system.

No, truly you haven't. The OP is experimenting with bytearray.translate,
which likely makes it a monoalphabetic substitution cipher, and the
techniques for cracking those go back to the 9th century AD. That's over a
thousand years of experience in cracking these things.

The situation is a bit harder than the sort of traditional ciphers, instead
of using an alphabet of 26 letters we have one of 256 bytes. But that's
only an order of magnitude bigger, and the cipher is still vulnerable to
frequency analysis and other attacks.

The only positive to this scheme is that the "encryption" is so weak (it's
been effectively obsolete since World War 2, if not before it) that you
might find it hard to find ready-made cracking tools for it unless you work
for the NSA, CIA or similar. You're relying on security by obscurity:
nobody uses this sort of thing any more, because it's so insecure, and that
obscurity does give you a *tiny* bit of security against a casual,
unmotivated attacker.

But once this system starts getting popular, that obscurity will not last.
It won't be difficult to build fast cracking programs that will break the
so-called "encryption", if it is based on a classical symmetric
monoalphabetic substitution cipher.

Here's an online tool which can be used for cracking "encrypted" English
text:

http://www.simonsingh.net/The_Black_Chamber/substitutioncrackingtool.html

You obviously wouldn't use that specific site on arbitrary files, but it
demonstrates that these classical ciphers are *not* secure.


>>> I am usually very oppositional when it comes to rolling your own
>>> crypto, but am I alone here in thinking the OP very clearly laid out
>>> their case?
>>
>> I don't think any of us *really* understand his use-case or the potential
>> threats, but to my way of thinking, you can never have too strong a
>> cipher or underestimate the risk of users taking short-cuts.
> 
> The use case is pretty obvious (a peer-to-peer dropbox type thing) but
> it does appear to be being misunderstood. This isn't actually a crypto
> problem at all and "users taking short-cuts" isn't an issue.

Yes it is. If users don't properly pre-encrypt their files before sending it
out to the cloud, AND THEY WON'T, receivers WILL be able to read those
files, half-arsed attempts to "encrypt" them or not.

The solution to all(?) these security problems is for the application to
handle the encryption, using a modern crypto library. But the OP doesn't
want to do that because it's too slow when written as pure Python.



-- 
Steven



From rxjwg98 at gmail.com  Thu Jun 25 11:20:39 2015
From: rxjwg98 at gmail.com (fl)
Date: Thu, 25 Jun 2015 08:20:39 -0700 (PDT)
Subject: Could you explain why this program runs?
Message-ID: <43ee9297-3182-4c03-9b78-837d697460fd@googlegroups.com>

Hi,

I download and install pyPDF2 library online. It says the test can run by:


python -m unittest Tests.tests


tests.py is under folder PyPDF2-master\Tests\


The above command line does run and give output message, but I don't 
understand why it run after I read tests.py:


///////////////////////////////
import os, sys, unittest

# Configure path environment
TESTS_ROOT = os.path.abspath(os.path.dirname(__file__))
PROJECT_ROOT = os.path.dirname(TESTS_ROOT)
RESOURCE_ROOT = os.path.join(PROJECT_ROOT, 'Resources')

sys.path.append(PROJECT_ROOT)

# Test imports
import unittest
from PyPDF2 import PdfFileReader


class PdfReaderTestCases(unittest.TestCase):

	def test_PdfReaderFileLoad(self):
		'''	Test loading and parsing of a file. Extract text of the file and compare to expected
			textual output. Expected outcome: file loads, text matches expected.
		'''
		with open(os.path.join(RESOURCE_ROOT, 'crazyones.pdf'), 'rb') as inputfile:
			
			# Load PDF file from file
			ipdf = PdfFileReader(inputfile)
			ipdf_p1 = ipdf.getPage(0)
			
			# Retrieve the text of the PDF
			pdftext_file = open(os.path.join(RESOURCE_ROOT, 'crazyones.txt'), 'r')
			pdftext = pdftext_file.read()
			ipdf_p1_text = ipdf_p1.extractText()
			
			# Compare the text of the PDF to a known source
			self.assertEqual(ipdf_p1_text.encode('utf-8', errors='ignore'), pdftext,
				msg='PDF extracted text differs from expected value.\n\nExpected:\n\n%r\n\nExtracted:\n\n%r\n\n'
					% (pdftext, ipdf_p1_text.encode('utf-8', errors='ignore')))
//////////////////////////////////////

It only gives a class PdfReaderTestCases() substantiation. I have read
usage on class, but I have not found the answer.
Can you help me on why the command line can run the test?

Thanks,


From jon+usenet at unequivocal.co.uk  Thu Jun 25 11:26:05 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Thu, 25 Jun 2015 15:26:05 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-25, Steven D'Aprano <steve at pearwood.info> wrote:
> On Thu, 25 Jun 2015 08:03 pm, Jon Ribbens wrote:
>> That won't stop virus scanners etc potentially making their own minds
>> up about the file.
>
> *shrug* Sure, but I was specifically referring to the risk of the malware
> being executed, not being detected by a virus scanner.
>
> Encrypting the file won't even necessarily stop the virus scanner from
> finding false positives. It might even increase the chances.

That seems spectacularly unlikely.

> But it will prevent the virus scanner from finding actual viruses.
> You may or may not consider that a problem.

The OP would consider it a benefit.

> I didn't say it ought to be encrypted by the receiver. Obviously the
> encryption needs to be done in a way that the recipient doesn't get access
> to the key.

No, you're still misunderstanding. The encryption needs to be done in
a way that the *sender* doesn't get access to the key. The recipient
has access to it by definition because the recipient chooses it.

> The obvious way to do that is for the application to encrypt the
> data before it sends it.

Yes, he already said the application does that. The problem is,
what if the sender is not the genuine application but is instead
a malicious attacker?

> Then the receiver just writes the encrypted bytes directly to a file.

That's precisely what he's trying to avoid.

> That would have the benefit of protecting against man-in-the-middle
> attacks as well, since the file is never transmitted in the clear.

With what he's talking about, the file after encryption is never
transmitted *at all*.

> I've been arguing that the application *should* encrypt the file, and not
> mess about giving the illusion of security.

You haven't understood the threat model.

> But seriously, they have the application. If the application is using a
> symmetric substitution cipher, it needs the key (because there is only
> one), so the receiver will have the cipher.

There is not only one key. The recipient would invent a new key for
each file after the file is received.

> With the sort of substitution cipher the OP is experimenting with, forcing a
> particular result is trivially easy. The sender has access to the
> application, knows the cipher, knows the key, and can easily generate a
> file which will generate whatever content the sender wants after being
> obfuscated.

No, because the sender does not know the key.

>> Replace "ROT-13" with "ROT-n" where 'n' is a secret known only to the
>> receiver, and suddenly it's not such a bad method of obfuscation.
>
> There are only 256 possible values for n, one of which doesn't transform the
> data at all (ROT-0). If you're thinking of attacking this by pencil and
> paper, 255 transformations sounds like a lot. For a computer, that's barely
> harder than a single transformation.

Well, it means you need to send 256 times as much data, which is a
start. If you're instead using a 256-byte translation table then
an attack becomes utterly impractical.

>> Improve it to the random-translation-map method he's actually using
>> and you've got really quite a reasonable system.
>
> No, truly you haven't. The OP is experimenting with bytearray.translate,
> which likely makes it a monoalphabetic substitution cipher, and the
> techniques for cracking those go back to the 9th century AD.

Only if you have the ciphertext, which the attacker in this scenario
does not. The attacker gets to set the plaintext, knows the algorithm,
does not know the key (unless the method of choosing the key has a
flaw), and wants to set the ciphertext to some specific string.
Frequency analysis doesn't even begin to apply to this scenario.

> You're relying on security by obscurity

No, he really isn't.

>> The use case is pretty obvious (a peer-to-peer dropbox type thing) but
>> it does appear to be being misunderstood. This isn't actually a crypto
>> problem at all and "users taking short-cuts" isn't an issue.
>
> Yes it is. If users don't properly pre-encrypt their files before sending it
> out to the cloud, AND THEY WON'T,

Yes they will. He said his application encrypts the files for them,
presumably he is indeed using "proper crypto" for that.

> receivers WILL be able to read those files,

That's a problem for the sender not the receiver.


From rxjwg98 at gmail.com  Thu Jun 25 11:51:58 2015
From: rxjwg98 at gmail.com (fl)
Date: Thu, 25 Jun 2015 08:51:58 -0700 (PDT)
Subject: Could you explain why this program runs?
In-Reply-To: <43ee9297-3182-4c03-9b78-837d697460fd@googlegroups.com>
References: <43ee9297-3182-4c03-9b78-837d697460fd@googlegroups.com>
Message-ID: <319f6dc4-9af8-4eff-9b07-8810fb8b7043@googlegroups.com>

On Thursday, June 25, 2015 at 8:20:52 AM UTC-7, fl wrote:
> Hi,
> 
> I download and install pyPDF2 library online. It says the test can run by:
> 
> 
> python -m unittest Tests.tests
> 
> 
> tests.py is under folder PyPDF2-master\Tests\
> 
> 
> The above command line does run and give output message, but I don't 
> understand why it run after I read tests.py:
> 
> 
> ///////////////////////////////
> import os, sys, unittest
> 
> # Configure path environment
> TESTS_ROOT = os.path.abspath(os.path.dirname(__file__))
> PROJECT_ROOT = os.path.dirname(TESTS_ROOT)
> RESOURCE_ROOT = os.path.join(PROJECT_ROOT, 'Resources')
> 
> sys.path.append(PROJECT_ROOT)
> 
> # Test imports
> import unittest
> from PyPDF2 import PdfFileReader
> 
> 
> class PdfReaderTestCases(unittest.TestCase):
> 
> 	def test_PdfReaderFileLoad(self):
> 		'''	Test loading and parsing of a file. Extract text of the file and compare to expected
> 			textual output. Expected outcome: file loads, text matches expected.
> 		'''
> 		with open(os.path.join(RESOURCE_ROOT, 'crazyones.pdf'), 'rb') as inputfile:
> 			
> 			# Load PDF file from file
> 			ipdf = PdfFileReader(inputfile)
> 			ipdf_p1 = ipdf.getPage(0)
> 			
> 			# Retrieve the text of the PDF
> 			pdftext_file = open(os.path.join(RESOURCE_ROOT, 'crazyones.txt'), 'r')
> 			pdftext = pdftext_file.read()
> 			ipdf_p1_text = ipdf_p1.extractText()
> 			
> 			# Compare the text of the PDF to a known source
> 			self.assertEqual(ipdf_p1_text.encode('utf-8', errors='ignore'), pdftext,
> 				msg='PDF extracted text differs from expected value.\n\nExpected:\n\n%r\n\nExtracted:\n\n%r\n\n'
> 					% (pdftext, ipdf_p1_text.encode('utf-8', errors='ignore')))
> //////////////////////////////////////
> 
> It only gives a class PdfReaderTestCases() substantiation. I have read
> usage on class, but I have not found the answer.
> Can you help me on why the command line can run the test?
> 
> Thanks,

Thanks for reading. I make it out that it is a feature of unittest module.


From ethan at stoneleaf.us  Thu Jun 25 11:59:41 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 25 Jun 2015 08:59:41 -0700
Subject: converting boolean filter function to lambda
Message-ID: <558C256D.7080903@stoneleaf.us>

I have the following function:

def phone_found(p):
   for c in contacts:
     if p in c:
       return True
   return False

with the following test data:

contacts = ['672.891.7280 x999', '291.792.9000 x111']
main = ['291.792.9001', '291.792.9000']

which works:

filter(phone_found, main)
# ['291.792.9000']

My attempt at a lambda function fails:

filter(lambda p: (p in c for c in contacts), main)
# ['291.792.9001', '291.792.9000']

Besides using a lambda ;) , what have I done wrong?

--
~Ethan~


From breamoreboy at yahoo.co.uk  Thu Jun 25 12:01:35 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 25 Jun 2015 17:01:35 +0100
Subject: Could you explain why this program runs?
In-Reply-To: <43ee9297-3182-4c03-9b78-837d697460fd@googlegroups.com>
References: <43ee9297-3182-4c03-9b78-837d697460fd@googlegroups.com>
Message-ID: <mmh8l2$8bh$1@ger.gmane.org>

On 25/06/2015 16:20, fl wrote:
> Hi,
>
> I download and install pyPDF2 library online. It says the test can run by:
>
> python -m unittest Tests.tests
>

The -m flag says run unittest as a script which then calls Tests.tests.

You can find out what all flags do by typing at the command prompt.

python --help

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From ian.g.kelly at gmail.com  Thu Jun 25 12:09:53 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 25 Jun 2015 10:09:53 -0600
Subject: converting boolean filter function to lambda
In-Reply-To: <558C256D.7080903@stoneleaf.us>
References: <558C256D.7080903@stoneleaf.us>
Message-ID: <CALwzidnZg8K_6OWoDQjW-UnFZTDuL7UBTv2tfi-dhkBD0__ryg@mail.gmail.com>

On Thu, Jun 25, 2015 at 9:59 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
> I have the following function:
>
> def phone_found(p):
>   for c in contacts:
>     if p in c:
>       return True
>   return False
>
> with the following test data:
>
> contacts = ['672.891.7280 x999', '291.792.9000 x111']
> main = ['291.792.9001', '291.792.9000']
>
> which works:
>
> filter(phone_found, main)
> # ['291.792.9000']
>
> My attempt at a lambda function fails:
>
> filter(lambda p: (p in c for c in contacts), main)
> # ['291.792.9001', '291.792.9000']
>
> Besides using a lambda ;) , what have I done wrong?

The lambda returns a generator, not a boolean. All generators are truthy.

I think you want this instead:

filter(lambda p: any(p in c for c in contacts), main)


From __peter__ at web.de  Thu Jun 25 12:11:33 2015
From: __peter__ at web.de (Peter Otten)
Date: Thu, 25 Jun 2015 18:11:33 +0200
Subject: converting boolean filter function to lambda
References: <558C256D.7080903@stoneleaf.us>
Message-ID: <mmh97n$ibf$1@ger.gmane.org>

Ethan Furman wrote:

> I have the following function:
> 
> def phone_found(p):
>    for c in contacts:
>      if p in c:
>        return True
>    return False
> 
> with the following test data:
> 
> contacts = ['672.891.7280 x999', '291.792.9000 x111']
> main = ['291.792.9001', '291.792.9000']
> 
> which works:
> 
> filter(phone_found, main)
> # ['291.792.9000']
> 
> My attempt at a lambda function fails:
> 
> filter(lambda p: (p in c for c in contacts), main)
> # ['291.792.9001', '291.792.9000']
> 
> Besides using a lambda ;) , what have I done wrong?

The lambda returns a generator expression and that expression is always true 
in a boolean context:

>>> bool(False for _ in ())
True

you're missing any() ...

>>> contacts = ['672.891.7280 x999', '291.792.9000 x111']
>>> main = ['291.792.9001', '291.792.9000']
>>> filter(lambda p: any(p in c for c in contacts), main)
<filter object at 0x7f475ce0e748>

... and list() if you were using Python 3.

>>> list(_)
['291.792.9000']




From ahlusar.ahluwalia at gmail.com  Thu Jun 25 12:37:12 2015
From: ahlusar.ahluwalia at gmail.com (kbtyo)
Date: Thu, 25 Jun 2015 09:37:12 -0700 (PDT)
Subject: To write headers once with different values in separate row in CSV
In-Reply-To: <558aa4b1$0$1671$c3e8da3$5496439d@news.astraweb.com>
References: <c0ea6bec-b6b1-48fd-9291-0fedcda7b76c@googlegroups.com>
 <558a0cef$0$1670$c3e8da3$5496439d@news.astraweb.com>
 <22c60270-d8e8-43eb-b58b-b6d86dc6732e@googlegroups.com>
 <558aa4b1$0$1671$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <c51f1026-547b-4322-9469-17f8b88e8973@googlegroups.com>

Okay, so I have gone back to the drawing board and have the following predicament (my apologies, in advance for the indentation):

Here is my sample:
    <Response ID="24856-775" RequestType="Moverview">        
            <MonthDayCount>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
                <Int32>0</Int32>
            </MonthDayCount>
    			<FeeCount>
    					<Int32>0</Int32>
    					<Int32>0</Int32>
    					<Int32>0</Int32>
    					<Int32>0</Int32>
    					<Int32>0</Int32>
    					<Int32>0</Int32>
    			</FeeCount>
    			<PaymentBucketAmount>
    						<Double>0</Double>
    						<Double>0</Double>
    						<Double>0</Double>
    						<Double>0</Double>
    						<Double>0</Double>
    						<Double>0</Double>
    						<Double>0</Double>
    						<Double>0</Double>
    					</PaymentBucketAmount>
    					<PaymentBucketDueDate>
    						<DateTime>1/1/0001 12:00:00 AM</DateTime>
    						<DateTime>1/1/0001 12:00:00 AM</DateTime>
    						<DateTime>1/1/0001 12:00:00 AM</DateTime>
    						<DateTime>1/1/0001 12:00:00 AM</DateTime>
    						<DateTime>1/1/0001 12:00:00 AM</DateTime>
    						<DateTime>1/1/0001 12:00:00 AM</DateTime>
    						<DateTime>1/1/0001 12:00:00 AM</DateTime>
    						<DateTime>1/1/0001 12:00:00 AM</DateTime>
    					</PaymentBucketDueDate>
            <Warnings />
            <SList />
            <LList />
            <PA>False</PA>
            <PL>False</PL>
            <PC>False</PC>
            <PCs>False</PCs>
            <PJ>False</PJ>
            <OITC>0</OITC>
            <MG />
            <R />
            <CCGoods />
    </Response>


Using this:


    import xml.etree.cElementTree as ElementTree 
    from xml.etree.ElementTree import XMLParser
    import csv
    
    def flatten_list(aList, prefix=''):
        for i, element in enumerate(aList, 1):
            eprefix = "{}{}".format(prefix, i)
            if element:
                # treat like dict 
                if len(element) == 1 or element[0].tag != element[1].tag: 
                    yield from flatten_dict(element, eprefix)
                # treat like list 
                elif element[0].tag == element[1].tag: 
                    yield from flatten_list(element, eprefix)
            elif element.text: 
                text = element.text.strip() 
                if text: 
                    yield eprefix[:].rstrip('.'), element.text
    
    def flatten_dict(parent_element, prefix=''):
        prefix = prefix + parent_element.tag 
        if parent_element.items():
            for k, v in parent_element.items():
                yield prefix + k, v
        for element in parent_element:
            eprefix = prefix + element.tag  
            if element:
                # treat like dict - we assume that if the first two tags 
                # in a series are different, then they are all different. 
                if len(element) == 1 or element[0].tag != element[1].tag: 
                    yield from flatten_dict(element, prefix=prefix)
                # treat like list - we assume that if the first two tags 
                # in a series are the same, then the rest are the same. 
                else: 
                    # here, we put the list in dictionary; the key is the 
                    # tag name the list elements all share in common, and 
                    # the value is the list itself
                    yield from flatten_list(element, prefix=eprefix)
                # if the tag has attributes, add those to the dict
                if element.items():
                    for k, v in element.items():
                        yield eprefix+k
            # this assumes that if you've got an attribute in a tag, 
            # you won't be having any text. This may or may not be a 
            # good idea -- time will tell. It works for the way we are 
            # currently doing XML configuration files... 
            elif element.items(): 
                for k, v in element.items():
                    yield eprefix+k
            # finally, if there are no child tags and no attributes, extract 
            # the text 
            else:
                yield eprefix, element.text                
    
    def makerows(pairs):
        headers = []
        columns = {}
        for k, v in pairs:
            if k in columns:
                columns[k].extend((v,))
            else:
                headers.append(k)
                columns[k] = [k, v]
        m = max(len(c) for c in columns.values())
        for c in columns.values():
            c.extend(' ' for i in range(len(c), m))
        L = [columns[k] for k in headers]
        rows = list(zip(*L))
        return rows                   
                        
    
    def main():
        with open('sample.xml', 'r', encoding='utf-8') as f: 
            xml_string = f.read() 
        xml_string= xml_string.replace('&#x0;', '') #optional to remove ampersands. 
        root = ElementTree.XML(xml_string) 
        for key, value in flatten_dict(root):
            key = key.rstrip('.').rsplit('.', 1)[-1]
            print(key,value)            
            
    if __name__ == "__main__":
        main()


I receive this output:

    ResponseRequestType Moverview
    ResponseID 24856-775
    ResponseMonthDayCount1 0
    ResponseMonthDayCount2 0
    ResponseMonthDayCount3 0
    ResponseMonthDayCount4 0
    ResponseMonthDayCount5 0
    ResponseMonthDayCount6 0
    ResponseMonthDayCount7 0
    ResponseMonthDayCount8 0
    ResponseMonthDayCount9 0
    ResponseMonthDayCount10 0
    ResponseMonthDayCount11 0
    ResponseMonthDayCount12 0
    ResponseMonthDayCount13 0
    ResponseMonthDayCount14 0
    ResponseMonthDayCount15 0
    ResponseMonthDayCount16 0
    ResponseMonthDayCount17 0
    ResponseMonthDayCount18 0
    ResponseMonthDayCount19 0
    ResponseMonthDayCount20 0
    ResponseMonthDayCount21 0
    ResponseMonthDayCount22 0
    ResponseMonthDayCount23 0
    ResponseMonthDayCount24 0
    ResponseMonthDayCount25 0
    ResponseFeeCount1 0
    ResponseFeeCount2 0
    ResponseFeeCount3 0
    ResponseFeeCount4 0
    ResponseFeeCount5 0
    ResponseFeeCount6 0
    ResponsePaymentBucketAmount1 0
    ResponsePaymentBucketAmount2 0
    ResponsePaymentBucketAmount3 0
    ResponsePaymentBucketAmount4 0
    ResponsePaymentBucketAmount5 0
    ResponsePaymentBucketAmount6 0
    ResponsePaymentBucketAmount7 0
    ResponsePaymentBucketAmount8 0
    ResponsePaymentBucketDueDate1 1/1/0001 12:00:00 AM
    ResponsePaymentBucketDueDate2 1/1/0001 12:00:00 AM
    ResponsePaymentBucketDueDate3 1/1/0001 12:00:00 AM
    ResponsePaymentBucketDueDate4 1/1/0001 12:00:00 AM
    ResponsePaymentBucketDueDate5 1/1/0001 12:00:00 AM
    ResponsePaymentBucketDueDate6 1/1/0001 12:00:00 AM
    ResponsePaymentBucketDueDate7 1/1/0001 12:00:00 AM
    ResponsePaymentBucketDueDate8 1/1/0001 12:00:00 AM
    ResponseWarnings None
    ResponseSList None
    ResponseLList None
    ResponsePA False
    ResponsePL False
    ResponsePC False
    ResponsePCs False
    ResponsePJ False
    ResponseOITC 0
    ResponseMG None
    ResponseR None
    ResponseCCGoods None


When I write it out to the CSV, using :        


    writer = csv.writer(open("try2.csv", 'wt')) 
    writer.writerows(makerows(flatten_dict(root)))`


I still receive the headers with the **Response** chained to the sub-elements of the root with the tags' text as the values (which is just fine). My goal is to only have the sublements as the headers (along with their values), sans the Response (unless it is the tag name). Thanks you all for your feedback! For example, ResponsePaymentBucketAmount1 0, should just be PaymentBucketAmount1 as the header.


From tjreedy at udel.edu  Thu Jun 25 12:59:13 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 25 Jun 2015 12:59:13 -0400
Subject: Python realted to question
In-Reply-To: <20150625154916.HM.a000000000EuLTj@skyjjogari.wwl1526.hanmail.net>
References: <20150625154916.HM.a000000000EuLTj@skyjjogari.wwl1526.hanmail.net>
Message-ID: <mmhc14$3gf$1@ger.gmane.org>

On 6/25/2015 2:49 AM, ??? wrote:

> Two raspberryPi2 Model B was connected to a socket to communicate with
> each other.
> I want to send the video by using the communication.
> Method of transmitting a text file is known. However, the video file is
> not sent.

What are the symptoms of 'not sent'?  What Python code are you using?

> So, Please tell me how to transfer a video file...

If you are using Python (2 or 3?), you will have to read and send the 
file as a binary file, not a text file.

You might want to send this question to an RPi-specific mailing list.

When posting to a mailing list, send plain text, not html with 'remote 
content' (which good readers will block).  This rule is true for all 
English-language mailing lists unless specified otherwise.

-- 
Terry Jan Reedy




From tjreedy at udel.edu  Thu Jun 25 13:03:53 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 25 Jun 2015 13:03:53 -0400
Subject: windows and file names > 256 bytes
In-Reply-To: <558bc6f1$0$11104$c3e8da3@news.astraweb.com>
References: <mailman.44.1435219506.3674.python-list@python.org>
 <558bc6f1$0$11104$c3e8da3@news.astraweb.com>
Message-ID: <mmhc9s$3gf$2@ger.gmane.org>

On 6/25/2015 5:16 AM, Steven D'Aprano wrote:
> On Thursday 25 June 2015 18:00, Albert-Jan Roskam wrote:
>
>> Hi,
>>
>> Consider the following calls, where very_long_path is more than 256 bytes:
>> [1] os.mkdir(very_long_path)
>> [2] os.getsize(very_long_path)
>> [3] shutil.rmtree(very_long_path)
>>
>> I am using Python 2.7 and [1] and [2] fail under Windows XP [3] fails
>> under Win7 (not sure about XP). It throws: ?WindowsError: [Error 206] The
>> filename or extension is too long?
>
> I don't think this is a bug. It seems to be a limitation of Windows.
>
> https://msdn.microsoft.com/en-
> us/library/windows/desktop/aa365247%28v=vs.85%29.aspx#maxpath
>
>> This is even when I use the "special"
>> notations \\?\c:\dir\file or \\?\UNC\server\share\file, e.g.
>> os.path.getsize("\\\\?\\" + "c:\\dir\\file")
>
> However, that may be a bug.
>
> What happens if you use a Unicode string?
>
> path = u"\\\\?\\c:a\\very\\long\\path"
> os.mkdir(path)
>
>
> Can you open an existing file?
>
> open(u"\\\\?\\c:a\\very\\long\\path\\file.txt")
>
>
>
>> (Oddly, os.path.getsize(os.path.join("\\\\?", "c:\\dir\\file")) will
>> truncate the prefix)
>
> That's worth reporting as a bug.

If possible, please try the same operations with Python 3.4 or .5 before 
making a report


-- 
Terry Jan Reedy




From random832 at fastmail.us  Thu Jun 25 13:53:44 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 25 Jun 2015 13:53:44 -0400
Subject: windows and file names > 256 bytes
In-Reply-To: <558C038A.5010103@gmail.com>
References: <mmgce2$cd4$1@ger.gmane.org> <mmgnbt$u0o$1@ger.gmane.org>
 <CAB1GNpS=7F3ebrHenUu7+MvyWuk5v60G6N6L81c0W3C-mQPi2w@mail.gmail.com>
 <558BF55C.2010701@timgolden.me.uk> <558C038A.5010103@gmail.com>
Message-ID: <1435254824.377071.307742473.1B992AF9@webmail.messagingengine.com>

On Thu, Jun 25, 2015, at 09:35, Michael Torrie wrote:
> The OP mentions that even when he manually supplies extended paths,
> os.mkdir, os.getsize, and shutil.rmtree return errors for him in Python
> 2.7.  So there's more to this problem.

The byte versions of the underlying OS APIs use a 256-character buffer
to do conversion - he needs to also be passing unicode strings.


From rxjwg98 at gmail.com  Thu Jun 25 14:14:13 2015
From: rxjwg98 at gmail.com (fl)
Date: Thu, 25 Jun 2015 11:14:13 -0700 (PDT)
Subject: Could you explain "[1, 2, 3].remove(2)" to me?
Message-ID: <e5890b67-939e-432a-b1ef-521bef393ae2@googlegroups.com>

Hi,

I see a code snippet online:

[1, 2, 3].remove(42)

after I modify it to:

[1, 2, 3].remove(2)

and

aa=[1, 2, 3].remove(2)


I don't know where the result goes. Could you help me on the question?

Thanks,


From ian.g.kelly at gmail.com  Thu Jun 25 14:37:24 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 25 Jun 2015 12:37:24 -0600
Subject: Could you explain "[1, 2, 3].remove(2)" to me?
In-Reply-To: <e5890b67-939e-432a-b1ef-521bef393ae2@googlegroups.com>
References: <e5890b67-939e-432a-b1ef-521bef393ae2@googlegroups.com>
Message-ID: <CALwzid=Cgr=fCcn7iBxr=7K7VffvTBY3Y=f29y86Re9HSKZgLQ@mail.gmail.com>

On Thu, Jun 25, 2015 at 12:14 PM, fl <rxjwg98 at gmail.com> wrote:
> Hi,
>
> I see a code snippet online:
>
> [1, 2, 3].remove(42)

I don't know where you pulled this from, but if this is from a
tutorial then it doesn't seem to be a very good one.

This constructs a list containing the elements 1, 2, and 3, and
attempts to remove the non-existent element 42, which will raise a
ValueError.

> after I modify it to:
>
> [1, 2, 3].remove(2)

This at least removes an element that is actually in the list, so it
won't throw an error, but the list is then discarded, so nothing was
actually accomplished by it.

> and
>
> aa=[1, 2, 3].remove(2)

This does the same thing, but it sets the variable aa to the result of
the *remove* operation. The remove operation, as it happens, returns
None, so the the list is still discarded, and the only thing
accomplished is that aa is now bound to None.

> I don't know where the result goes. Could you help me on the question?

You need to store the list somewhere before you start calling
operations on it. Try this:

aa = [1, 2, 3]
aa.remove(2)

Now you have the list in the variable aa, and the value 2 has been
removed from it.


From ahlusar.ahluwalia at gmail.com  Thu Jun 25 14:39:53 2015
From: ahlusar.ahluwalia at gmail.com (kbtyo)
Date: Thu, 25 Jun 2015 11:39:53 -0700 (PDT)
Subject: enumerate XML tags (keys that will become headers) along with text
 (values) and write to CSV in one row (as opposed to "stacked" values with one
 header)
Message-ID: <14aeae7a-41ab-4619-8331-7995e2420e54@googlegroups.com>

My question can be found here:


http://stackoverflow.com/questions/31058100/enumerate-column-headers-in-csv-that-belong-to-the-same-tag-key-in-python


Here is an additional sample sample of the XML that I am working with: 

<Response ID="123546 - 7831" RequestType="Moverview">        
        <MonthDayCount>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
            <Int32>0</Int32>
        </MonthDayCount>
            <FeeCount>
                    <Int32>0</Int32>
                    <Int32>0</Int32>
                    <Int32>0</Int32>
                    <Int32>0</Int32>
                    <Int32>0</Int32>
                    <Int32>0</Int32>
            </FeeCount>
            <PaymentBucketAmount>
                        <Double>0</Double>
                        <Double>0</Double>
                        <Double>0</Double>
                        <Double>0</Double>
                        <Double>0</Double>
                        <Double>0</Double>
                        <Double>0</Double>
                        <Double>0</Double>
                    </PaymentBucketAmount>
                    <PaymentBucketDueDate>
                        <DateTime>1/1/0001 12:00:00 AM</DateTime>
                        <DateTime>1/1/0001 12:00:00 AM</DateTime>
                        <DateTime>1/1/0001 12:00:00 AM</DateTime>
                        <DateTime>1/1/0001 12:00:00 AM</DateTime>
                        <DateTime>1/1/0001 12:00:00 AM</DateTime>
                        <DateTime>1/1/0001 12:00:00 AM</DateTime>
                        <DateTime>1/1/0001 12:00:00 AM</DateTime>
                        <DateTime>1/1/0001 12:00:00 AM</DateTime>
                    </PaymentBucketDueDate>
        <Warnings />
        <SList />
        <LList />
        <PA>False</PA>
        <PL>False</PL>
        <PC>False</PC>
        <PCs>False</PCs>
        <PJ>False</PJ>
        <OITC>0</OITC>
        <MG />
        <R />
        <CCGoods />
</Response>


From t-erice at microsoft.com  Thu Jun 25 14:49:00 2015
From: t-erice at microsoft.com (Eric Edmond)
Date: Thu, 25 Jun 2015 18:49:00 +0000
Subject: Reference Counting Irregularity
Message-ID: <BLUPR03MB37441732F74C57CA2BE7834E0AE0@BLUPR03MB374.namprd03.prod.outlook.com>

Hi,

I have been writing a C++ extension for Python recently, and am currently fixing the reference counting throughout the extension. As I am very new to this topic, my question may have a simple answer, but I was unable to find any mention of the behavior online.

When using the PyObject_GetItem(obj, key) function, I noticed inconsistent behavior with various types of obj (for reference, key is always a str object). When obj is a standard dictionary, the key's reference count remains unchanged, as expected. However, when obj is a pandas DataFrame object, the key's reference count increases by 2. This could very well be by design of the DataFrame object doing some internal caching of the string, but does not appear in the documentation, so I thought I would bring up the issue.

Thanks,

Eric Edmond
University of Michigan | Class of 2016

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150625/bb8de9ea/attachment.html>

From randall at tnr.cc  Thu Jun 25 14:58:11 2015
From: randall at tnr.cc (Randall Smith)
Date: Thu, 25 Jun 2015 13:58:11 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <558C4F43.9070609@tnr.cc>

Thanks Jon.  I couldn't have answered those questions better myself, and 
I wrote the software in question.

I didn't intend to describe the entire system, but rather just enough of 
it to present the issue at hand.  You seem to understand it quite well.

I'm now using a randomly generated 256 byte translation table, which 
performs very well on the lowly Raspberry PI ARM chip.  The Raspberry PI 
is to be my recommended storage node platform.

For those that care, the storage system is something like Amazon S3, 
except storage is distributed peer to peer.  Clients compress, encrypt, 
and chunk data, then send it to storage nodes. Storage nodes propagate 
the data.  Encryption and Authentication are handled through TLS.  Files 
use AES encryption for storage.  Storage Nodes are monitored for 
availability, integrity, and performance.  Data transfers are 
coordinated by a centralized service which tracks storage and transfers. 
  Redundancy is configurable by chunk. Storage nodes are compensated for 
storage x time.  Uploads and downloads can utilize several storage nodes 
simultaneously to increase throughput.

-Randall

On 06/25/2015 10:26 AM, Jon Ribbens wrote:
> On 2015-06-25, Steven D'Aprano <steve at pearwood.info> wrote:
>> On Thu, 25 Jun 2015 08:03 pm, Jon Ribbens wrote:
>>> That won't stop virus scanners etc potentially making their own minds
>>> up about the file.
>>
>> *shrug* Sure, but I was specifically referring to the risk of the malware
>> being executed, not being detected by a virus scanner.
>>
>> Encrypting the file won't even necessarily stop the virus scanner from
>> finding false positives. It might even increase the chances.
>
> That seems spectacularly unlikely.
>
>> But it will prevent the virus scanner from finding actual viruses.
>> You may or may not consider that a problem.
>
> The OP would consider it a benefit.
>
>> I didn't say it ought to be encrypted by the receiver. Obviously the
>> encryption needs to be done in a way that the recipient doesn't get access
>> to the key.
>
> No, you're still misunderstanding. The encryption needs to be done in
> a way that the *sender* doesn't get access to the key. The recipient
> has access to it by definition because the recipient chooses it.
>
>> The obvious way to do that is for the application to encrypt the
>> data before it sends it.
>
> Yes, he already said the application does that. The problem is,
> what if the sender is not the genuine application but is instead
> a malicious attacker?
>
>> Then the receiver just writes the encrypted bytes directly to a file.
>
> That's precisely what he's trying to avoid.
>
>> That would have the benefit of protecting against man-in-the-middle
>> attacks as well, since the file is never transmitted in the clear.
>
> With what he's talking about, the file after encryption is never
> transmitted *at all*.
>
>> I've been arguing that the application *should* encrypt the file, and not
>> mess about giving the illusion of security.
>
> You haven't understood the threat model.
>
>> But seriously, they have the application. If the application is using a
>> symmetric substitution cipher, it needs the key (because there is only
>> one), so the receiver will have the cipher.
>
> There is not only one key. The recipient would invent a new key for
> each file after the file is received.
>
>> With the sort of substitution cipher the OP is experimenting with, forcing a
>> particular result is trivially easy. The sender has access to the
>> application, knows the cipher, knows the key, and can easily generate a
>> file which will generate whatever content the sender wants after being
>> obfuscated.
>
> No, because the sender does not know the key.
>
>>> Replace "ROT-13" with "ROT-n" where 'n' is a secret known only to the
>>> receiver, and suddenly it's not such a bad method of obfuscation.
>>
>> There are only 256 possible values for n, one of which doesn't transform the
>> data at all (ROT-0). If you're thinking of attacking this by pencil and
>> paper, 255 transformations sounds like a lot. For a computer, that's barely
>> harder than a single transformation.
>
> Well, it means you need to send 256 times as much data, which is a
> start. If you're instead using a 256-byte translation table then
> an attack becomes utterly impractical.
>
>>> Improve it to the random-translation-map method he's actually using
>>> and you've got really quite a reasonable system.
>>
>> No, truly you haven't. The OP is experimenting with bytearray.translate,
>> which likely makes it a monoalphabetic substitution cipher, and the
>> techniques for cracking those go back to the 9th century AD.
>
> Only if you have the ciphertext, which the attacker in this scenario
> does not. The attacker gets to set the plaintext, knows the algorithm,
> does not know the key (unless the method of choosing the key has a
> flaw), and wants to set the ciphertext to some specific string.
> Frequency analysis doesn't even begin to apply to this scenario.
>
>> You're relying on security by obscurity
>
> No, he really isn't.
>
>>> The use case is pretty obvious (a peer-to-peer dropbox type thing) but
>>> it does appear to be being misunderstood. This isn't actually a crypto
>>> problem at all and "users taking short-cuts" isn't an issue.
>>
>> Yes it is. If users don't properly pre-encrypt their files before sending it
>> out to the cloud, AND THEY WON'T,
>
> Yes they will. He said his application encrypts the files for them,
> presumably he is indeed using "proper crypto" for that.
>
>> receivers WILL be able to read those files,
>
> That's a problem for the sender not the receiver.
>



From randall at tnr.cc  Thu Jun 25 14:58:11 2015
From: randall at tnr.cc (Randall Smith)
Date: Thu, 25 Jun 2015 13:58:11 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <558C4F43.9070609@tnr.cc>

Thanks Jon.  I couldn't have answered those questions better myself, and 
I wrote the software in question.

I didn't intend to describe the entire system, but rather just enough of 
it to present the issue at hand.  You seem to understand it quite well.

I'm now using a randomly generated 256 byte translation table, which 
performs very well on the lowly Raspberry PI ARM chip.  The Raspberry PI 
is to be my recommended storage node platform.

For those that care, the storage system is something like Amazon S3, 
except storage is distributed peer to peer.  Clients compress, encrypt, 
and chunk data, then send it to storage nodes. Storage nodes propagate 
the data.  Encryption and Authentication are handled through TLS.  Files 
use AES encryption for storage.  Storage Nodes are monitored for 
availability, integrity, and performance.  Data transfers are 
coordinated by a centralized service which tracks storage and transfers. 
  Redundancy is configurable by chunk. Storage nodes are compensated for 
storage x time.  Uploads and downloads can utilize several storage nodes 
simultaneously to increase throughput.

-Randall

On 06/25/2015 10:26 AM, Jon Ribbens wrote:
> On 2015-06-25, Steven D'Aprano <steve at pearwood.info> wrote:
>> On Thu, 25 Jun 2015 08:03 pm, Jon Ribbens wrote:
>>> That won't stop virus scanners etc potentially making their own minds
>>> up about the file.
>>
>> *shrug* Sure, but I was specifically referring to the risk of the malware
>> being executed, not being detected by a virus scanner.
>>
>> Encrypting the file won't even necessarily stop the virus scanner from
>> finding false positives. It might even increase the chances.
>
> That seems spectacularly unlikely.
>
>> But it will prevent the virus scanner from finding actual viruses.
>> You may or may not consider that a problem.
>
> The OP would consider it a benefit.
>
>> I didn't say it ought to be encrypted by the receiver. Obviously the
>> encryption needs to be done in a way that the recipient doesn't get access
>> to the key.
>
> No, you're still misunderstanding. The encryption needs to be done in
> a way that the *sender* doesn't get access to the key. The recipient
> has access to it by definition because the recipient chooses it.
>
>> The obvious way to do that is for the application to encrypt the
>> data before it sends it.
>
> Yes, he already said the application does that. The problem is,
> what if the sender is not the genuine application but is instead
> a malicious attacker?
>
>> Then the receiver just writes the encrypted bytes directly to a file.
>
> That's precisely what he's trying to avoid.
>
>> That would have the benefit of protecting against man-in-the-middle
>> attacks as well, since the file is never transmitted in the clear.
>
> With what he's talking about, the file after encryption is never
> transmitted *at all*.
>
>> I've been arguing that the application *should* encrypt the file, and not
>> mess about giving the illusion of security.
>
> You haven't understood the threat model.
>
>> But seriously, they have the application. If the application is using a
>> symmetric substitution cipher, it needs the key (because there is only
>> one), so the receiver will have the cipher.
>
> There is not only one key. The recipient would invent a new key for
> each file after the file is received.
>
>> With the sort of substitution cipher the OP is experimenting with, forcing a
>> particular result is trivially easy. The sender has access to the
>> application, knows the cipher, knows the key, and can easily generate a
>> file which will generate whatever content the sender wants after being
>> obfuscated.
>
> No, because the sender does not know the key.
>
>>> Replace "ROT-13" with "ROT-n" where 'n' is a secret known only to the
>>> receiver, and suddenly it's not such a bad method of obfuscation.
>>
>> There are only 256 possible values for n, one of which doesn't transform the
>> data at all (ROT-0). If you're thinking of attacking this by pencil and
>> paper, 255 transformations sounds like a lot. For a computer, that's barely
>> harder than a single transformation.
>
> Well, it means you need to send 256 times as much data, which is a
> start. If you're instead using a 256-byte translation table then
> an attack becomes utterly impractical.
>
>>> Improve it to the random-translation-map method he's actually using
>>> and you've got really quite a reasonable system.
>>
>> No, truly you haven't. The OP is experimenting with bytearray.translate,
>> which likely makes it a monoalphabetic substitution cipher, and the
>> techniques for cracking those go back to the 9th century AD.
>
> Only if you have the ciphertext, which the attacker in this scenario
> does not. The attacker gets to set the plaintext, knows the algorithm,
> does not know the key (unless the method of choosing the key has a
> flaw), and wants to set the ciphertext to some specific string.
> Frequency analysis doesn't even begin to apply to this scenario.
>
>> You're relying on security by obscurity
>
> No, he really isn't.
>
>>> The use case is pretty obvious (a peer-to-peer dropbox type thing) but
>>> it does appear to be being misunderstood. This isn't actually a crypto
>>> problem at all and "users taking short-cuts" isn't an issue.
>>
>> Yes it is. If users don't properly pre-encrypt their files before sending it
>> out to the cloud, AND THEY WON'T,
>
> Yes they will. He said his application encrypts the files for them,
> presumably he is indeed using "proper crypto" for that.
>
>> receivers WILL be able to read those files,
>
> That's a problem for the sender not the receiver.
>




From jpiitula at ling.helsinki.fi  Thu Jun 25 15:01:14 2015
From: jpiitula at ling.helsinki.fi (Jussi Piitulainen)
Date: Thu, 25 Jun 2015 22:01:14 +0300
Subject: Could you explain "[1, 2, 3].remove(2)" to me?
References: <e5890b67-939e-432a-b1ef-521bef393ae2@googlegroups.com>
Message-ID: <lf5lhf7vchx.fsf@ling.helsinki.fi>

fl writes:

> aa=[1, 2, 3].remove(2)
>
> I don't know where the result goes. Could you help me on the question?

That method modifies the list and returns None (or raises an exception).

Get a hold on the list first:

aa=[1, 2, 3]

*Then* call the method. Just call the method, do not try to store the
value (which will be None) anywhere:

aa.remove(2)

*Now* you can see that the list has changed. Try it.

By the way, it's no use to try [1, 2, 3].remove(2). That will only
modify and throw away a different list that just happens to have the
same contents initially. Try these two:

aa=[1, 2, 3]
bb=[1, 2, 3]  # a different list!

aa=[1, 2, 3]
bb=aa         # the same list!

In both cases, try removing 2 from aa and then watch what happens to aa
and what happens to bb.


From randall at tnr.cc  Thu Jun 25 15:09:00 2015
From: randall at tnr.cc (Randall Smith)
Date: Thu, 25 Jun 2015 14:09:00 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CABicbJLZJq7pA9ws8Edfzuch2cpkobpP83MQUsmvwGXaYs4rtw@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <CABicbJLZJq7pA9ws8Edfzuch2cpkobpP83MQUsmvwGXaYs4rtw@mail.gmail.com>
Message-ID: <558C51CC.3010907@tnr.cc>

On 06/24/2015 11:27 PM, Devin Jeanpierre wrote:
> On Wed, Jun 24, 2015 at 9:07 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> But just sticking to the three above, the first one is partially mitigated
>> by allowing virus scanners to scan the data, but that implies that the
>> owner of the storage machine can spy on the files. So you have a conflict
>> here.
>
> If it's encrypted malware, and you can't decrypt it, there's no threat.
>
>> Honestly, the *only* real defence against the spying issue is to encrypt the
>> files. Not obfuscate them with a lousy random substitution cipher. The
>> storage machine can keep the files as long as they like, just by making a
>> copy, and spend hours bruteforcing them. They *will* crack the substitution
>> cipher. In pure Python, that may take a few days or weeks; in C, hours or
>> days. If they have the resources to throw at it, minutes. Substitution
>> ciphers have not been effective encryption since, oh, the 1950s, unless you
>> use a one-time pad. Which you won't be.
>
> The original post said that the sender will usually send files they
> encrypted, unless they are malicious. So if the sender wants them to
> be encrypted, they already are.
>
> "While the data senders are supposed to encrypt data, that's not
> guaranteed, and I'd like to protect the recipient against exposure to
> nefarious data by mangling or encrypting the data before it is written
> to disk."
>
> The cipher is just to keep the sender from being able to control what
> is on disk.
>
> I am usually very oppositional when it comes to rolling your own
> crypto, but am I alone here in thinking the OP very clearly laid out
> their case?
>
> -- Devin
>

Thanks Devin.  You understand the issue perfectly despite my limited 
description of the system.  I've fully implemented and performance 
tested your suggested solution and am quite happy with it.

Though the issue is solved, I would be glad to listen to any remaining 
criticisms, suggestions or questions.

--Randall


From randall at tnr.cc  Thu Jun 25 15:09:00 2015
From: randall at tnr.cc (Randall Smith)
Date: Thu, 25 Jun 2015 14:09:00 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CABicbJLZJq7pA9ws8Edfzuch2cpkobpP83MQUsmvwGXaYs4rtw@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <CABicbJLZJq7pA9ws8Edfzuch2cpkobpP83MQUsmvwGXaYs4rtw@mail.gmail.com>
Message-ID: <558C51CC.3010907@tnr.cc>

On 06/24/2015 11:27 PM, Devin Jeanpierre wrote:
> On Wed, Jun 24, 2015 at 9:07 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> But just sticking to the three above, the first one is partially mitigated
>> by allowing virus scanners to scan the data, but that implies that the
>> owner of the storage machine can spy on the files. So you have a conflict
>> here.
>
> If it's encrypted malware, and you can't decrypt it, there's no threat.
>
>> Honestly, the *only* real defence against the spying issue is to encrypt the
>> files. Not obfuscate them with a lousy random substitution cipher. The
>> storage machine can keep the files as long as they like, just by making a
>> copy, and spend hours bruteforcing them. They *will* crack the substitution
>> cipher. In pure Python, that may take a few days or weeks; in C, hours or
>> days. If they have the resources to throw at it, minutes. Substitution
>> ciphers have not been effective encryption since, oh, the 1950s, unless you
>> use a one-time pad. Which you won't be.
>
> The original post said that the sender will usually send files they
> encrypted, unless they are malicious. So if the sender wants them to
> be encrypted, they already are.
>
> "While the data senders are supposed to encrypt data, that's not
> guaranteed, and I'd like to protect the recipient against exposure to
> nefarious data by mangling or encrypting the data before it is written
> to disk."
>
> The cipher is just to keep the sender from being able to control what
> is on disk.
>
> I am usually very oppositional when it comes to rolling your own
> crypto, but am I alone here in thinking the OP very clearly laid out
> their case?
>
> -- Devin
>

Thanks Devin.  You understand the issue perfectly despite my limited 
description of the system.  I've fully implemented and performance 
tested your suggested solution and am quite happy with it.

Though the issue is solved, I would be glad to listen to any remaining 
criticisms, suggestions or questions.

--Randall



From randall at tnr.cc  Thu Jun 25 15:41:43 2015
From: randall at tnr.cc (Randall Smith)
Date: Thu, 25 Jun 2015 14:41:43 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <05mmoa5b3lev74esrmsu6ce5h1s99lhdb3@4ax.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <558AF4D7.1070205@tnr.cc> <05mmoa5b3lev74esrmsu6ce5h1s99lhdb3@4ax.com>
Message-ID: <558C5977.9080309@tnr.cc>

On 06/24/2015 08:33 PM, Dennis Lee Bieber wrote:
> On Wed, 24 Jun 2015 13:20:07 -0500, Randall Smith <randall at tnr.cc>
> declaimed the following:
>
>> On 06/24/2015 06:36 AM, Steven D'Aprano wrote:
>>> I don't understand how mangling the data is supposed to protect the
>>> recipient. Don't they have the ability unmangle the data, and thus expose
>>> themselves to whatever nasties are in the files?
>>
>> They never look at the data and wouldn't care to unmangle it.  The
>> purpose is primarily to prevent automated software (file indexers, virus
>> scanners) from doing bad things to the data.
>>
>
> 	Which leads to the question: what is "doing bad things".

Storage nodes are computers running the software in discussion, that 
store chunks of data they are sent (recipient) and send it upon request. 
  Their job (as related to this software) is to accept, store and send 
chunks of data upon request.  So losing data is a bad thing.

The storage node software is cross platform and should run on anything 
from a dedicated Raspberry PI to an old Windows PC.  Data integrity is 
insured using encryption and hashes generated by the original data 
owners.  Normally, a data chunk would look like random bytes, because it 
is encrypted.  However, the storage node cannot prevent the client 
(uploader) from sending unencrypted data.  The purpose of this 
obfuscation is to protect the storage node, as many potential users have 
expressed hesitation in storing other peoples data.

Example: A storage node runs a Desktop OS with an image indexer. It 
receives an unencrypted nasty image or movie. The indexer picks it up 
and shows it in the person's image or movie "Library".

Does that clear things up?


-Randall


From randall at tnr.cc  Thu Jun 25 15:41:43 2015
From: randall at tnr.cc (Randall Smith)
Date: Thu, 25 Jun 2015 14:41:43 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <05mmoa5b3lev74esrmsu6ce5h1s99lhdb3@4ax.com>
References: <mailman.10.1435130890.3674.python-list@python.org>
 <558a9649$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <558AF4D7.1070205@tnr.cc> <05mmoa5b3lev74esrmsu6ce5h1s99lhdb3@4ax.com>
Message-ID: <558C5977.9080309@tnr.cc>

On 06/24/2015 08:33 PM, Dennis Lee Bieber wrote:
> On Wed, 24 Jun 2015 13:20:07 -0500, Randall Smith <randall at tnr.cc>
> declaimed the following:
>
>> On 06/24/2015 06:36 AM, Steven D'Aprano wrote:
>>> I don't understand how mangling the data is supposed to protect the
>>> recipient. Don't they have the ability unmangle the data, and thus expose
>>> themselves to whatever nasties are in the files?
>>
>> They never look at the data and wouldn't care to unmangle it.  The
>> purpose is primarily to prevent automated software (file indexers, virus
>> scanners) from doing bad things to the data.
>>
>
> 	Which leads to the question: what is "doing bad things".

Storage nodes are computers running the software in discussion, that 
store chunks of data they are sent (recipient) and send it upon request. 
  Their job (as related to this software) is to accept, store and send 
chunks of data upon request.  So losing data is a bad thing.

The storage node software is cross platform and should run on anything 
from a dedicated Raspberry PI to an old Windows PC.  Data integrity is 
insured using encryption and hashes generated by the original data 
owners.  Normally, a data chunk would look like random bytes, because it 
is encrypted.  However, the storage node cannot prevent the client 
(uploader) from sending unencrypted data.  The purpose of this 
obfuscation is to protect the storage node, as many potential users have 
expressed hesitation in storing other peoples data.

Example: A storage node runs a Desktop OS with an image indexer. It 
receives an unencrypted nasty image or movie. The indexer picks it up 
and shows it in the person's image or movie "Library".

Does that clear things up?


-Randall



From rxjwg98 at gmail.com  Thu Jun 25 16:53:38 2015
From: rxjwg98 at gmail.com (fl)
Date: Thu, 25 Jun 2015 13:53:38 -0700 (PDT)
Subject: Could you give me the detail process of 'make_incrementor(22)(33)'?
Message-ID: <20661fcb-7773-4f17-bbfa-8533914e637f@googlegroups.com>

Hi,

I read a tutorial on lambda on line. I don't think that I am clear about
the last line in its example code. It gives two parameters (22, 23). 
Is 22 for n, and 23 for x? Or, it creates two functions first. Then,
each function gets 22 while the other function gets 23?


Please help me on this interesting problem. Thanks,






>>> def make_incrementor (n): return lambda x: x + n
>>> 
>>> f = make_incrementor(2)
>>> g = make_incrementor(6)
>>> 
>>> print f(42), g(42)
44 48
>>> 
>>> print make_incrementor(22)(33)


From ian.g.kelly at gmail.com  Thu Jun 25 16:56:54 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 25 Jun 2015 14:56:54 -0600
Subject: Reference Counting Irregularity
In-Reply-To: <BLUPR03MB37441732F74C57CA2BE7834E0AE0@BLUPR03MB374.namprd03.prod.outlook.com>
References: <BLUPR03MB37441732F74C57CA2BE7834E0AE0@BLUPR03MB374.namprd03.prod.outlook.com>
Message-ID: <CALwzidkCA67EZDwh+6wTEwdqBJ2fwm8R6hVU+fyh-CvscRjJjA@mail.gmail.com>

On Thu, Jun 25, 2015 at 12:49 PM, Eric Edmond <t-erice at microsoft.com> wrote:
> Hi,
>
> I have been writing a C++ extension for Python recently, and am currently
> fixing the reference counting throughout the extension. As I am very new to
> this topic, my question may have a simple answer, but I was unable to find
> any mention of the behavior online.
>
> When using the PyObject_GetItem(obj, key) function, I noticed inconsistent
> behavior with various types of obj (for reference, key is always a str
> object). When obj is a standard dictionary, the key?s reference count
> remains unchanged, as expected. However, when obj is a pandas DataFrame
> object, the key?s reference count increases by 2. This could very well be by
> design of the DataFrame object doing some internal caching of the string,
> but does not appear in the documentation, so I thought I would bring up the
> issue.

What is your question? If you want to know why 2 is added to the
reference count for pandas objects, the first place to check would be
with the pandas developers, as PyObject_GetItem is just going to call
the implementation for that type. You can find the CPython code here:

https://hg.python.org/cpython/file/9aad116baee8/Objects/abstract.c#l136


From ian.g.kelly at gmail.com  Thu Jun 25 17:22:45 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 25 Jun 2015 15:22:45 -0600
Subject: Could you give me the detail process of
 'make_incrementor(22)(33)'?
In-Reply-To: <20661fcb-7773-4f17-bbfa-8533914e637f@googlegroups.com>
References: <20661fcb-7773-4f17-bbfa-8533914e637f@googlegroups.com>
Message-ID: <CALwzid=b4ZDaj=Rev9jOiorXrq8PB9yEnS11QJjSy24q73mr0w@mail.gmail.com>

On Thu, Jun 25, 2015 at 2:53 PM, fl <rxjwg98 at gmail.com> wrote:
> Hi,
>
> I read a tutorial on lambda on line. I don't think that I am clear about
> the last line in its example code. It gives two parameters (22, 23).
> Is 22 for n, and 23 for x? Or, it creates two functions first. Then,
> each function gets 22 while the other function gets 23?
>
>
>>>> def make_incrementor (n): return lambda x: x + n
>>>>
>>>> f = make_incrementor(2)
>>>> g = make_incrementor(6)
>>>>
>>>> print f(42), g(42)
> 44 48
>>>>
>>>> print make_incrementor(22)(33)

make_incrementor is a function that takes an argument n. It returns a
function that takes an argument x. So when you do make_incrementor(22)
that passes 22 to make_incrementor as the value of n, and when you do
make_incrementor(22)(33), that also passes 22 to make_incrementor as
the value of n, and then it passes 33 to the returned function as the
value of x.


From rosuav at gmail.com  Thu Jun 25 19:59:00 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 26 Jun 2015 09:59:00 +1000
Subject: converting boolean filter function to lambda
In-Reply-To: <558C256D.7080903@stoneleaf.us>
References: <558C256D.7080903@stoneleaf.us>
Message-ID: <CAPTjJmpuwkapSOWKXVJUEscNwVjB+qnr9WZ6AwJjjjA0KY54sQ@mail.gmail.com>

On Fri, Jun 26, 2015 at 1:59 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
> My attempt at a lambda function fails:
>
> filter(lambda p: (p in c for c in contacts), main)
> # ['291.792.9001', '291.792.9000']
>
> Besides using a lambda ;) , what have I done wrong?

This looks like a job for a list comprehension!

(Cue the swooping-in cape-wearing list comp, coming to save the day)

ChrisA


From breamoreboy at yahoo.co.uk  Thu Jun 25 20:09:18 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 26 Jun 2015 01:09:18 +0100
Subject: converting boolean filter function to lambda
In-Reply-To: <CAPTjJmpuwkapSOWKXVJUEscNwVjB+qnr9WZ6AwJjjjA0KY54sQ@mail.gmail.com>
References: <558C256D.7080903@stoneleaf.us>
 <CAPTjJmpuwkapSOWKXVJUEscNwVjB+qnr9WZ6AwJjjjA0KY54sQ@mail.gmail.com>
Message-ID: <mmi57h$tjv$1@ger.gmane.org>

On 26/06/2015 00:59, Chris Angelico wrote:
> On Fri, Jun 26, 2015 at 1:59 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
>> My attempt at a lambda function fails:
>>
>> filter(lambda p: (p in c for c in contacts), main)
>> # ['291.792.9001', '291.792.9000']
>>
>> Besides using a lambda ;) , what have I done wrong?
>
> This looks like a job for a list comprehension!
>
> (Cue the swooping-in cape-wearing list comp, coming to save the day)
>
> ChrisA
>

In exactly the same way that swooping-in cape-wearing Greg Ewing, from 
the home of 10 man rugby, had to point out to Ethan some years back what 
"booted into touch" meant? :)

I've also seen Ethan refer to bowlers today, I just hope he's very 
careful and doesn't get caught on the back foot.  Next question from 
Ethan is?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Thu Jun 25 20:33:15 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 26 Jun 2015 10:33:15 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>

On Fri, Jun 26, 2015 at 1:26 AM, Jon Ribbens
<jon+usenet at unequivocal.co.uk> wrote:
>> There are only 256 possible values for n, one of which doesn't transform the
>> data at all (ROT-0). If you're thinking of attacking this by pencil and
>> paper, 255 transformations sounds like a lot. For a computer, that's barely
>> harder than a single transformation.
>
> Well, it means you need to send 256 times as much data, which is a
> start. If you're instead using a 256-byte translation table then
> an attack becomes utterly impractical.
>

Utterly impractical? Maybe, if you attempt a pure brute-force approach
- there are 256! possible translation tables, which is roughly e500
attempts [1], and at roughly four a microsecond [2] that'd still take
a ridiculously long time. But there are two gigantic optimizations you
could do. Firstly, there are frequency-based attacks, and byte value
duplicates will tell you a lot - classic cryptographic work. And
secondly, you can simply take the first few bytes of a file - let's
say 16, although a lot of files can be recognized in less than that.
Even if there are no duplicate bytes, that'd be a maximum of 16!
translation tables that truly matter, or just 2e13. At the same speed,
that makes about a million seconds of computing time required. Divide
that across a bunch of separate computers (the job is embarrassingly
parallel after all), and you could get that result pretty easily. Cut
the prefix to just 8 bytes and you have a mere 40K encryption keys to
try - so quick that you wouldn't even see it happen. Nope, a simple
substitution cipher is still not secure. Even the famous Enigma
machine was a lot more than just letter-for-letter substitution - a
double letter in the cleartext wouldn't be represented by a double
letter in the result - and once the machine's secrets were figured
out, the day's key could be reassembled fairly readily.

ChrisA

[1] It's actually closer to 8.6e506, if you care.
[2] timeit result from my laptop - you could do better, but that's a
reasonable average


From adhamyos at gmail.com  Thu Jun 25 20:34:15 2015
From: adhamyos at gmail.com (adhamyos at gmail.com)
Date: Thu, 25 Jun 2015 17:34:15 -0700 (PDT)
Subject: mutual coaching
Message-ID: <703059a2-621c-4205-b0a2-5bacfa632901@googlegroups.com>

hello anyone wants to study python? we can learn together! pm me my name is adham128 iam at the #programming room


From ethan at stoneleaf.us  Thu Jun 25 20:57:23 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 25 Jun 2015 17:57:23 -0700
Subject: converting boolean filter function to lambda
In-Reply-To: <mmi57h$tjv$1@ger.gmane.org>
References: <558C256D.7080903@stoneleaf.us>
 <CAPTjJmpuwkapSOWKXVJUEscNwVjB+qnr9WZ6AwJjjjA0KY54sQ@mail.gmail.com>
 <mmi57h$tjv$1@ger.gmane.org>
Message-ID: <558CA373.2000409@stoneleaf.us>

On 06/25/2015 05:09 PM, Mark Lawrence wrote:
> On 26/06/2015 00:59, Chris Angelico wrote:
>> On Fri, Jun 26, 2015 at 1:59 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
>>> My attempt at a lambda function fails:
>>>
>>> filter(lambda p: (p in c for c in contacts), main)
>>> # ['291.792.9001', '291.792.9000']
>>>
>>> Besides using a lambda ;) , what have I done wrong?
>>
>> This looks like a job for a list comprehension!
>>
>> (Cue the swooping-in cape-wearing list comp, coming to save the day)
>>
>> ChrisA
>>
>
> In exactly the same way that swooping-in cape-wearing Greg Ewing, from the home of 10 man rugby, had to point out to Ethan some years back what "booted into touch" meant? :)
>
> I've also seen Ethan refer to bowlers today, I just hope he's very careful and doesn't get caught on the back foot.  Next question from Ethan is?

Uh, "caught on the back foot"?  I thought I was okay as long as I didn't slide across the line or gutter the ball.  ;)

--
~Ethan~


From ian.g.kelly at gmail.com  Thu Jun 25 21:01:19 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 25 Jun 2015 19:01:19 -0600
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
Message-ID: <CALwzidmzh-z7KEXHbOatHik8_ZA6m=kF1gmiM0F3kdUFd+TXeQ@mail.gmail.com>

On Thu, Jun 25, 2015 at 6:33 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Fri, Jun 26, 2015 at 1:26 AM, Jon Ribbens
> <jon+usenet at unequivocal.co.uk> wrote:
>>> There are only 256 possible values for n, one of which doesn't transform the
>>> data at all (ROT-0). If you're thinking of attacking this by pencil and
>>> paper, 255 transformations sounds like a lot. For a computer, that's barely
>>> harder than a single transformation.
>>
>> Well, it means you need to send 256 times as much data, which is a
>> start. If you're instead using a 256-byte translation table then
>> an attack becomes utterly impractical.
>>
>
> Utterly impractical? Maybe, if you attempt a pure brute-force approach
> - there are 256! possible translation tables, which is roughly e500
> attempts [1], and at roughly four a microsecond [2] that'd still take
> a ridiculously long time. But there are two gigantic optimizations you
> could do. Firstly, there are frequency-based attacks, and byte value
> duplicates will tell you a lot - classic cryptographic work. And
> secondly, you can simply take the first few bytes of a file - let's
> say 16, although a lot of files can be recognized in less than that.
> Even if there are no duplicate bytes, that'd be a maximum of 16!
> translation tables that truly matter, or just 2e13. At the same speed,
> that makes about a million seconds of computing time required. Divide
> that across a bunch of separate computers (the job is embarrassingly
> parallel after all), and you could get that result pretty easily. Cut
> the prefix to just 8 bytes and you have a mere 40K encryption keys to
> try - so quick that you wouldn't even see it happen. Nope, a simple
> substitution cipher is still not secure. Even the famous Enigma
> machine was a lot more than just letter-for-letter substitution - a
> double letter in the cleartext wouldn't be represented by a double
> letter in the result - and once the machine's secrets were figured
> out, the day's key could be reassembled fairly readily.

You're making the same mistake that Steven did in misunderstanding the
threat model. The goal isn't to prevent the attacker from working out
the key for a file that has already been obfuscated. Any real data
that might be exposed by a vulnerability in the server is presumed to
have already been strongly encrypted by the user.

The goal is to prevent the attacker from guessing a key that hasn't
even been generated yet, which could be exploited to engineer the
obfuscated content into something malicious. There are no
frequency-based attacks possible here, because you can't do frequency
analysis on the result of a key that hasn't even been generated yet.
Assuming that you have no attack on the key generation itself, the
best you can do is send a file deobfuscated with a random key and hope
that the recipient randomly chooses the same key; the odds of that
happening are 1 in 256!.

That said, I do see a potential weakness here: if the attacker can
create a malicious payload using only a subset of the 256 possible
byte values, then the odds of getting a correct key are increased,
since multiple keys will work. For an extreme example, if the attacker
can manage to craft a malicious payload that uses only the two byte
values 32 and 47, then the probability of getting a key that will
obfuscate to that is increased to 1 in 256! / 254!, or 1 in 65280. If
they distribute 65280 copies of that payload to various recipients,
then they can expect that one recipient on average will get the
payload in its malicious form.


From rxjwg98 at gmail.com  Thu Jun 25 21:07:30 2015
From: rxjwg98 at gmail.com (fl)
Date: Thu, 25 Jun 2015 18:07:30 -0700 (PDT)
Subject: Can anybody explain the '-' in a 2-D creation code?
Message-ID: <e90e10c2-6762-4636-9e05-1f96c7c0b2c2@googlegroups.com>

Hi,

I read Ned's tutorial on Python. It is very interesting. On its last
example, I cannot understand the '_' in:



board=[[0]*8 for _ in range(8)]


I know  '_' is the precious answer, but it is still unclear what it is
in the above line. Can you explain it to me?


Thanks,


From breamoreboy at yahoo.co.uk  Thu Jun 25 21:08:42 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 26 Jun 2015 02:08:42 +0100
Subject: converting boolean filter function to lambda
In-Reply-To: <558CA373.2000409@stoneleaf.us>
References: <558C256D.7080903@stoneleaf.us>
 <CAPTjJmpuwkapSOWKXVJUEscNwVjB+qnr9WZ6AwJjjjA0KY54sQ@mail.gmail.com>
 <mmi57h$tjv$1@ger.gmane.org> <558CA373.2000409@stoneleaf.us>
Message-ID: <mmi8mt$ife$1@ger.gmane.org>

On 26/06/2015 01:57, Ethan Furman wrote:
> On 06/25/2015 05:09 PM, Mark Lawrence wrote:
>> On 26/06/2015 00:59, Chris Angelico wrote:
>>> On Fri, Jun 26, 2015 at 1:59 AM, Ethan Furman <ethan at stoneleaf.us>
>>> wrote:
>>>> My attempt at a lambda function fails:
>>>>
>>>> filter(lambda p: (p in c for c in contacts), main)
>>>> # ['291.792.9001', '291.792.9000']
>>>>
>>>> Besides using a lambda ;) , what have I done wrong?
>>>
>>> This looks like a job for a list comprehension!
>>>
>>> (Cue the swooping-in cape-wearing list comp, coming to save the day)
>>>
>>> ChrisA
>>>
>>
>> In exactly the same way that swooping-in cape-wearing Greg Ewing, from
>> the home of 10 man rugby, had to point out to Ethan some years back
>> what "booted into touch" meant? :)
>>
>> I've also seen Ethan refer to bowlers today, I just hope he's very
>> careful and doesn't get caught on the back foot.  Next question from
>> Ethan is?
>
> Uh, "caught on the back foot"?  I thought I was okay as long as I didn't
> slide across the line or gutter the ball.  ;)
>
> --
> ~Ethan~

I takes it you're referring to the cissies game where everybody has to 
wear a glove just to catch the ball?  And there's no seam to cut your 
hands if you get it wrong, so I guess they don't bother rubbing white 
spirit into the hands to help toughen them up? :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Thu Jun 25 21:17:03 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 26 Jun 2015 02:17:03 +0100
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
Message-ID: <mmi96j$pa6$1@ger.gmane.org>

On 26/06/2015 01:33, Chris Angelico wrote:
> On Fri, Jun 26, 2015 at 1:26 AM, Jon Ribbens
> <jon+usenet at unequivocal.co.uk> wrote:
>>> There are only 256 possible values for n, one of which doesn't transform the
>>> data at all (ROT-0). If you're thinking of attacking this by pencil and
>>> paper, 255 transformations sounds like a lot. For a computer, that's barely
>>> harder than a single transformation.
>>
>> Well, it means you need to send 256 times as much data, which is a
>> start. If you're instead using a 256-byte translation table then
>> an attack becomes utterly impractical.
>>
>
> Utterly impractical? Maybe, if you attempt a pure brute-force approach
> - there are 256! possible translation tables, which is roughly e500
> attempts [1], and at roughly four a microsecond [2] that'd still take
> a ridiculously long time. But there are two gigantic optimizations you
> could do. Firstly, there are frequency-based attacks, and byte value
> duplicates will tell you a lot - classic cryptographic work. And
> secondly, you can simply take the first few bytes of a file - let's
> say 16, although a lot of files can be recognized in less than that.
> Even if there are no duplicate bytes, that'd be a maximum of 16!
> translation tables that truly matter, or just 2e13. At the same speed,
> that makes about a million seconds of computing time required. Divide
> that across a bunch of separate computers (the job is embarrassingly
> parallel after all), and you could get that result pretty easily. Cut
> the prefix to just 8 bytes and you have a mere 40K encryption keys to
> try - so quick that you wouldn't even see it happen. Nope, a simple
> substitution cipher is still not secure. Even the famous Enigma
> machine was a lot more than just letter-for-letter substitution - a
> double letter in the cleartext wouldn't be represented by a double
> letter in the result - and once the machine's secrets were figured
> out, the day's key could be reassembled fairly readily.
>

The day's key for a given network, with the Luftwaffe easily being the 
worst offenders.  Some networks remained unbroken at the end of WWII.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From gary.herron at islandtraining.com  Thu Jun 25 21:18:15 2015
From: gary.herron at islandtraining.com (Gary Herron)
Date: Thu, 25 Jun 2015 18:18:15 -0700
Subject: Can anybody explain the '-' in a 2-D creation code?
In-Reply-To: <e90e10c2-6762-4636-9e05-1f96c7c0b2c2@googlegroups.com>
References: <e90e10c2-6762-4636-9e05-1f96c7c0b2c2@googlegroups.com>
Message-ID: <558CA857.3000806@islandtraining.com>

On 06/25/2015 06:07 PM, fl wrote:
> Hi,
>
> I read Ned's tutorial on Python. It is very interesting. On its last
> example, I cannot understand the '_' in:
>
>
>
> board=[[0]*8 for _ in range(8)]
>
>
> I know  '_' is the precious answer, but it is still unclear what it is
> in the above line. Can you explain it to me?
>
>
> Thanks,

He uses _ to indicate a variable whose name and value he does not care 
about, but it *is* a valid variable name.

He could have used
   ... for i in range ...
or
   ... for unused_variable in range ...

This is a valid, though probably unclear, use of that same name:

 >>> _ = 123
 >>> print(_)
123
 >>>


Gary Herron





From breamoreboy at yahoo.co.uk  Thu Jun 25 21:23:20 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 26 Jun 2015 02:23:20 +0100
Subject: Can anybody explain the '-' in a 2-D creation code?
In-Reply-To: <e90e10c2-6762-4636-9e05-1f96c7c0b2c2@googlegroups.com>
References: <e90e10c2-6762-4636-9e05-1f96c7c0b2c2@googlegroups.com>
Message-ID: <mmi9ic$uht$1@ger.gmane.org>

On 26/06/2015 02:07, fl wrote:
> Hi,
>
> I read Ned's tutorial on Python. It is very interesting. On its last
> example, I cannot understand the '_' in:
>
>
>
> board=[[0]*8 for _ in range(8)]
>
>
> I know  '_' is the precious answer, but it is still unclear what it is
> in the above line. Can you explain it to me?
>
>
> Thanks,
>

Lots of people could carry on explaining things to you, but you don't 
appear to be making any attempt to do some research before posing your 
questions, so how about using a search engine?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From andre.roberge at gmail.com  Thu Jun 25 21:24:01 2015
From: andre.roberge at gmail.com (=?ISO-8859-1?Q?Andr=E9_Roberge?=)
Date: Thu, 25 Jun 2015 18:24:01 -0700 (PDT)
Subject: Can anybody explain the '-' in a 2-D creation code?
In-Reply-To: <e90e10c2-6762-4636-9e05-1f96c7c0b2c2@googlegroups.com>
References: <e90e10c2-6762-4636-9e05-1f96c7c0b2c2@googlegroups.com>
Message-ID: <7dea671b-cc91-467b-b0ea-137923032ce3@googlegroups.com>

On Thursday, 25 June 2015 22:07:42 UTC-3, fl  wrote:
> Hi,
> 
> I read Ned's tutorial on Python. It is very interesting. On its last
> example, I cannot understand the '_' in:
> 
> 
> 
> board=[[0]*8 for _ in range(8)]
> 
> 
> I know  '_' is the precious answer, but it is still unclear what it is
> in the above line. Can you explain it to me?

'_' is the previous answer ONLY when using the read-eval-print-loop interpreter.

Here, it is the "name" of a variable; since we don't care about the particular name (it is used just for looping a fixed number of times), the common practice of using '_' has been used.  As you will have noted (since it confused you), '_' doesn't seem to designate anything of interest - unlike a variable name like 'string_index' or 'character', etc.

Sometimes, people will use the name "dummy" instead of '_', with the same idea in mind.
> 
> 
> Thanks,



From atnakus.arzah at gmail.com  Thu Jun 25 21:29:28 2015
From: atnakus.arzah at gmail.com (Atnakus Arzah)
Date: Thu, 25 Jun 2015 18:29:28 -0700
Subject: Can anybody explain the '-' in a 2-D creation code?
In-Reply-To: <e90e10c2-6762-4636-9e05-1f96c7c0b2c2@googlegroups.com>
References: <e90e10c2-6762-4636-9e05-1f96c7c0b2c2@googlegroups.com>
Message-ID: <20150626012928.GA35007@sukantas-mbp>

On Thu, Jun 25, 2015 at 06:07:30PM -0700, fl wrote:
>Hi,
>
>I read Ned's tutorial on Python. It is very interesting. On its last
>example, I cannot understand the '_' in:
>
>
>
>board=[[0]*8 for _ in range(8)]
>
>
>I know  '_' is the precious answer, but it is still unclear what it is
>in the above line. Can you explain it to me?

Its just a variable whose value isn't used in this specific expression, you could use 'x', 'y', or any other valid variable
name there and it should work.


-- 
Atnakus Arzah <atnakus.arzah at gmail.com>

When in doubt, have a cookie!



From rxjwg98 at gmail.com  Thu Jun 25 21:40:57 2015
From: rxjwg98 at gmail.com (fl)
Date: Thu, 25 Jun 2015 18:40:57 -0700 (PDT)
Subject: Can anybody explain the '-' in a 2-D creation code?
In-Reply-To: <mailman.92.1435281823.3674.python-list@python.org>
References: <e90e10c2-6762-4636-9e05-1f96c7c0b2c2@googlegroups.com>
 <mailman.92.1435281823.3674.python-list@python.org>
Message-ID: <88b1f3fe-2a0f-408d-8dd6-283cd26e48f8@googlegroups.com>

On Thursday, June 25, 2015 at 6:24:07 PM UTC-7, Mark Lawrence wrote:
> On 26/06/2015 02:07, fl wrote:
> > Hi,
> >
> > I read Ned's tutorial on Python. It is very interesting. On its last
> > example, I cannot understand the '_' in:
> >
> >
> >
> > board=[[0]*8 for _ in range(8)]
> >
> >
> > I know  '_' is the precious answer, but it is still unclear what it is
> > in the above line. Can you explain it to me?
> >
> >
> > Thanks,
> >
> 
> Lots of people could carry on explaining things to you, but you don't 
> appear to be making any attempt to do some research before posing your 
> questions, so how about using a search engine?
> 
> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence

Excuse me. On one hand, I am busying on cram these Python stuff quickly for
a position. On the other hand, the search seems to me needing a little 
skill to get the goal I hope. I would really appreciate if someone can give
an example on what phrase to use in the search. I am not a lazy guy.
Thanks to all the response.


From rosuav at gmail.com  Thu Jun 25 22:05:09 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 26 Jun 2015 12:05:09 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CALwzidmzh-z7KEXHbOatHik8_ZA6m=kF1gmiM0F3kdUFd+TXeQ@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <CALwzidmzh-z7KEXHbOatHik8_ZA6m=kF1gmiM0F3kdUFd+TXeQ@mail.gmail.com>
Message-ID: <CAPTjJmqnMQWU_ETsdkHYSbOHY=Pa2W-ErQhmXf=devBWuM4PNA@mail.gmail.com>

On Fri, Jun 26, 2015 at 11:01 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Thu, Jun 25, 2015 at 6:33 PM, Chris Angelico <rosuav at gmail.com> wrote:
>> On Fri, Jun 26, 2015 at 1:26 AM, Jon Ribbens
>> <jon+usenet at unequivocal.co.uk> wrote:
>>> Well, it means you need to send 256 times as much data, which is a
>>> start. If you're instead using a 256-byte translation table then
>>> an attack becomes utterly impractical.
>>>
>>
>> Utterly impractical? <chomp analysis>
>
> You're making the same mistake that Steven did in misunderstanding the
> threat model.

To be honest, I wasn't actually answering anything about the original
threat model, but only responding to the statement that a 256-byte
"anything-to-anything" cipher is somehow incredibly secure. It isn't,
but that might not be a problem for the original purpose.

ChrisA


From rosuav at gmail.com  Thu Jun 25 22:06:38 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 26 Jun 2015 12:06:38 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmi96j$pa6$1@ger.gmane.org>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mmi96j$pa6$1@ger.gmane.org>
Message-ID: <CAPTjJmpAtFAA1Bzhf0vNgR=XAsP8rxrH074NRzk5G_c1BPWL=A@mail.gmail.com>

On Fri, Jun 26, 2015 at 11:17 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>> Even the famous Enigma
>> machine was a lot more than just letter-for-letter substitution - a
>> double letter in the cleartext wouldn't be represented by a double
>> letter in the result - and once the machine's secrets were figured
>> out, the day's key could be reassembled fairly readily.
>>
>
> The day's key for a given network, with the Luftwaffe easily being the worst
> offenders.  Some networks remained unbroken at the end of WWII.

I was massively oversimplifying, here. But there's a reason that
modern crypto doesn't use str.translate() level ciphers.

ChrisA


From breamoreboy at yahoo.co.uk  Thu Jun 25 22:24:34 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 26 Jun 2015 03:24:34 +0100
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CAPTjJmpAtFAA1Bzhf0vNgR=XAsP8rxrH074NRzk5G_c1BPWL=A@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mmi96j$pa6$1@ger.gmane.org>
 <CAPTjJmpAtFAA1Bzhf0vNgR=XAsP8rxrH074NRzk5G_c1BPWL=A@mail.gmail.com>
Message-ID: <mmid52$iq8$1@ger.gmane.org>

On 26/06/2015 03:06, Chris Angelico wrote:
> On Fri, Jun 26, 2015 at 11:17 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>>> Even the famous Enigma
>>> machine was a lot more than just letter-for-letter substitution - a
>>> double letter in the cleartext wouldn't be represented by a double
>>> letter in the result - and once the machine's secrets were figured
>>> out, the day's key could be reassembled fairly readily.
>>>
>>
>> The day's key for a given network, with the Luftwaffe easily being the worst
>> offenders.  Some networks remained unbroken at the end of WWII.
>
> I was massively oversimplifying, here. But there's a reason that
> modern crypto doesn't use str.translate() level ciphers.
>
> ChrisA
>

I should know.  Ever heard of DISCON?  Like to hazard a guess as to who 
worked on it all those years ago?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Thu Jun 25 22:27:45 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 26 Jun 2015 03:27:45 +0100
Subject: Can anybody explain the '-' in a 2-D creation code?
In-Reply-To: <88b1f3fe-2a0f-408d-8dd6-283cd26e48f8@googlegroups.com>
References: <e90e10c2-6762-4636-9e05-1f96c7c0b2c2@googlegroups.com>
 <mailman.92.1435281823.3674.python-list@python.org>
 <88b1f3fe-2a0f-408d-8dd6-283cd26e48f8@googlegroups.com>
Message-ID: <mmidb1$n37$1@ger.gmane.org>

On 26/06/2015 02:40, fl wrote:
> On Thursday, June 25, 2015 at 6:24:07 PM UTC-7, Mark Lawrence wrote:
>> On 26/06/2015 02:07, fl wrote:
>>> Hi,
>>>
>>> I read Ned's tutorial on Python. It is very interesting. On its last
>>> example, I cannot understand the '_' in:
>>>
>>>
>>>
>>> board=[[0]*8 for _ in range(8)]
>>>
>>>
>>> I know  '_' is the precious answer, but it is still unclear what it is
>>> in the above line. Can you explain it to me?
>>>
>>>
>>> Thanks,
>>>
>>
>> Lots of people could carry on explaining things to you, but you don't
>> appear to be making any attempt to do some research before posing your
>> questions, so how about using a search engine?
>>
>> --
>> My fellow Pythonistas, ask not what our language can do for you, ask
>> what you can do for our language.
>>
>> Mark Lawrence
>
> Excuse me. On one hand, I am busying on cram these Python stuff quickly for
> a position. On the other hand, the search seems to me needing a little
> skill to get the goal I hope. I would really appreciate if someone can give
> an example on what phrase to use in the search. I am not a lazy guy.
> Thanks to all the response.
>

http://stackoverflow.com/questions/5893163/what-is-the-purpose-of-the-single-underscore-variable-in-python

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Thu Jun 25 22:29:06 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 26 Jun 2015 12:29:06 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmid52$iq8$1@ger.gmane.org>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mmi96j$pa6$1@ger.gmane.org>
 <CAPTjJmpAtFAA1Bzhf0vNgR=XAsP8rxrH074NRzk5G_c1BPWL=A@mail.gmail.com>
 <mmid52$iq8$1@ger.gmane.org>
Message-ID: <CAPTjJmqB6CAkfGTDKLE=7PTxuvZcyfNcgAcC9+Li3A_To3SHDA@mail.gmail.com>

On Fri, Jun 26, 2015 at 12:24 PM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> On 26/06/2015 03:06, Chris Angelico wrote:
>>
>> On Fri, Jun 26, 2015 at 11:17 AM, Mark Lawrence <breamoreboy at yahoo.co.uk>
>> wrote:
>>>>
>>>> Even the famous Enigma
>>>> machine was a lot more than just letter-for-letter substitution - a
>>>> double letter in the cleartext wouldn't be represented by a double
>>>> letter in the result - and once the machine's secrets were figured
>>>> out, the day's key could be reassembled fairly readily.
>>>>
>>>
>>> The day's key for a given network, with the Luftwaffe easily being the
>>> worst
>>> offenders.  Some networks remained unbroken at the end of WWII.
>>
>>
>> I was massively oversimplifying, here. But there's a reason that
>> modern crypto doesn't use str.translate() level ciphers.
>>
>> ChrisA
>>
>
> I should know.  Ever heard of DISCON?  Like to hazard a guess as to who
> worked on it all those years ago?

No, not familiar with it. But I'm guessing you have the crypto
background to know all this stuff, which means you aren't the sort of
person I need to explain things to. Great! :)

ChrisA


From godson.g at gmail.com  Thu Jun 25 22:42:49 2015
From: godson.g at gmail.com (Godson Gera)
Date: Thu, 25 Jun 2015 21:42:49 -0500
Subject: [ANN] PySWITCH 0.2
Message-ID: <CAPTXd0pRN=PO+OMEcSLraiss2-KBVFNZbHdnJL3YcuzPX0XmSg@mail.gmail.com>

=========
pyswitch 0.2
=========

PySWITCH 0.2 is released

Please, note that PySWITCH 0.2 is not available on PyPI because of name
conflict

Major changes to 0.2

*Many new FreeSWITCH API and dialplan commands added

Full list of changes can be found at
http://pyswitch.sourceforge.net/pages/changelog.html

About PySWITCH
=============

PySWITCH is a Python <http://python.org/> library to communicate with
FreeSWITCH <http://freeswitch.org/> server via EventSocketLayer (ESL). It's
based on Twisted library. Unlike the default python ESL library that comes
with FreeSWITCH, this library is designed to handle high volume of
concurrent calls with an easy to use API. PySWITCH currently supports
extensive set of FreeSWITCH API and dialplan commands. You need to be
familiar with Twisted library inorder use this.

Project website: http://pyswitch.sf.net
Download Page: http://pyswitch.sourceforge.net/pages/download.html

-- 
Godson Gera
Python Consultant <http://godson.in>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150625/4f203d59/attachment.html>

From liam.oshea at o2-v2.com  Thu Jun 25 22:51:57 2015
From: liam.oshea at o2-v2.com (liam.oshea at o2-v2.com)
Date: Thu, 25 Jun 2015 19:51:57 -0700 (PDT)
Subject: Turning string into object (name)
Message-ID: <3ba959bc-f25f-4655-8c1f-ce2888210e1b@googlegroups.com>

Hi all,
I have something like this:

def dbconn():
	#Establishes connection to local db
	try:
		conn = client()
		db = conn.db_solar #dbname
		collection = db.main # dbcollection / Table
		print "Connected to database successfully!!!"
		return(db,collection)
	except errors.ConnectionFailure, e:
		print "Could not connect to MongoDB: %s" % e 
		sys.exit()

Now I want to remove the hardcoded 'db_solar' and 'main' (from db.main) and load these values from a config file.

Once I have loaded in a string 'db_solar' from a config file how do I use it such that something like db=conn.db_solar will be constructed and run as expected.
Thanks


From rosuav at gmail.com  Fri Jun 26 00:00:07 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 26 Jun 2015 14:00:07 +1000
Subject: Turning string into object (name)
In-Reply-To: <3ba959bc-f25f-4655-8c1f-ce2888210e1b@googlegroups.com>
References: <3ba959bc-f25f-4655-8c1f-ce2888210e1b@googlegroups.com>
Message-ID: <CAPTjJmotCUCYkSPgVZguMwyEvHX6ce+W8vfFX4NEJNFy6vEuzQ@mail.gmail.com>

On Fri, Jun 26, 2015 at 12:51 PM,  <liam.oshea at o2-v2.com> wrote:
> def dbconn():
>         #Establishes connection to local db
>         try:
>                 conn = client()
>                 db = conn.db_solar #dbname
>                 collection = db.main # dbcollection / Table
>                 print "Connected to database successfully!!!"
>                 return(db,collection)
>         except errors.ConnectionFailure, e:
>                 print "Could not connect to MongoDB: %s" % e
>                 sys.exit()
>
> Now I want to remove the hardcoded 'db_solar' and 'main' (from db.main) and load these values from a config file.
>
> Once I have loaded in a string 'db_solar' from a config file how do I use it such that something like db=conn.db_solar will be constructed and run as expected.
> Thanks

You can retrieve attributes using strings like this:

# Same effect:
db = conn.db_solar
db = getattr(conn, "db_solar")

So a variable attribute name can be handled the same way:

db = getattr(conn, dbname)
collection = getattr(db, dbcollection)

Incidentally, I would suggest not having the try/except at all, since
all it does is print an error and terminate (which is the same result
you'd get if that error bubbled all the way to top level). But if you
are going to use it, then I strongly recommend using the newer syntax:

except errors.ConnectionFailure as e:

unless you have some reason for supporting ancient versions of Python.
For most modern code, you can usually rely on at least 2.7, so the new
syntax works; it's unambiguous in the face of multiple-exception
handlers, and it works identically on Python 3 (the "except type,
name:" syntax isn't supported on Py3). Since it's a trivial syntactic
change, there's generally no reason to use the old form, unless you
actually need your code to run on Python 2.5.

But for what you're doing, chances are you can just let the exception
bubble up. That way, a calling function gets the choice of handling it
some other way, which currently isn't an option.

ChrisA


From tommyc168168 at gmail.com  Fri Jun 26 00:34:09 2015
From: tommyc168168 at gmail.com (Tommy C)
Date: Thu, 25 Jun 2015 21:34:09 -0700 (PDT)
Subject: Questions on Pandas
Message-ID: <663b9bb8-8252-4982-a959-4f68f7d35414@googlegroups.com>

Hi there, I have a number of questions related to the Pandas exercises found from the book, Python for Data Analysis by Wes McKinney. Particularly, these exercises are from Chapter 6 of the book. It'd be much appreciated if you could answer the following questions!

1.
[code]
Input: pd.read_csv('ch06/ex2.csv', header=None)
Output:
 X.1 X.2 X.3 X.4 X.5
0 1 2 3 4 hello
1 5 6 7 8 world
2 9 10 11 12 foo

[/code]

Does the header appear as "X.#" by default when it is set to be None?

2.
[code]
Input: chunker = pd.read_csv('ch06/ex6.csv', chunksize=1000)
Input: chunker
Output: <pandas.io.parsers.TextParser at 0x8398150>

[/code]

Please explain the idea of chunksize and the output meaning.


3.
[code]
The TextParser object returned by read_csv allows you to iterate over the parts of the
file according to the chunksize. For example, we can iterate over ex6.csv, aggregating
the value counts in the 'key' column like so:
chunker = pd.read_csv('ch06/ex6.csv', chunksize=1000)
tot = Series([])
for piece in chunker:
 tot = tot.add(piece['key'].value_counts(), fill_value=0)
tot = tot.order(ascending=False)
We have then:
In [877]: tot[:10]
Out[877]:
E 368
X 364
L 346
O 343
Q 340
M 338
J 337
F 335
K 334
H 330

[/code]

I couldn't run the Series function successfully... is there something missing in this code?

4.
[code]
Data can also be exported to delimited format. Let's consider one of the CSV files read
above:
In [878]: data = pd.read_csv('ch06/ex5.csv')
In [879]: data
Out[879]:
 something a b c d message
0 one 1 2 3 4 NaN
1 two 5 6 NaN 8 world
2 three 9 10 11 12 foo

Missing values appear as empty strings in the output. You might want to denote them
by some other sentinel value:
In [883]: data.to_csv(sys.stdout, na_rep='NULL')
,something,a,b,c,d,message
0,one,1,2,3.0,4,NULL
1,two,5,6,NULL,8,world
2,three,9,10,11.0,12,foo


[/code]

Error occured as I tried to run this code with sys.stdout.


5.
[code]
class of csv.Dialect:
class my_dialect(csv.Dialect):
 lineterminator = '\n'
 delimiter = ';'
 quotechar = '"'
reader = csv.reader(f, dialect=my_dialect)

[/code]

An error occurred when I tried to run this code: "quotechar must be an 1-character integer... please explain.


6.
[code]
with open('mydata.csv', 'w') as f:
 writer = csv.writer(f, dialect=my_dialect)
 writer.writerow(('one', 'two', 'three'))
 writer.writerow(('1', '2', '3'))
 writer.writerow(('4', '5', '6'))
 writer.writerow(('7', '8', '9'))
[/code]

An error occurred when I ran this code. Please explain the cause of the error.


7.
[code]
But these are objects representing HTML elements; to get the URL and link text you
have to use each element's get method (for the URL) and text_content method (for
the display text):
In [908]: lnk = links[28]
In [909]: lnk
Out[909]: <Element a at 0x6c48dd0>
In [910]: lnk.get('href')
Out[910]: 'http://biz.yahoo.com/special.html'
In [911]: lnk.text_content()
Out[911]: 'Special Editions'
Thus, getting a list of all URLs in the document is a matter of writing this list comprehension:
In [912]: urls = [lnk.get('href') for lnk in doc.findall('.//a')]
In [913]: urls[-10:]
Out[913]:
['http://info.yahoo.com/privacy/us/yahoo/finance/details.html',
 'http://info.yahoo.com/relevantads/',
 'http://docs.yahoo.com/info/terms/',
 'http://docs.yahoo.com/info/copyright/copyright.html',
 'http://help.yahoo.com/l/us/yahoo/finance/forms_index.html',
 'http://help.yahoo.com/l/us/yahoo/finance/quotes/fitadelay.html',
 'http://help.yahoo.com/l/us/yahoo/finance/quotes/fitadelay.html',

[/code]

An error related to line 912 occurred as I tried to run the code. Please explain.

8.
[code]
Using lxml.objectify, we parse the file and get a reference to the root node of the XML
file with getroot:
from lxml import objectify
path = 'Performance_MNR.xml'
parsed = objectify.parse(open(path))
root = parsed.getroot()

[/code]

An error occured when I tried to run the code to access the XML file. Please explain.


9.
[code]
ML data can get much more complicated than this example. Each tag can have metadata,
too. Consider an HTML link tag which is also valid XML:
from StringIO import StringIO
tag = '<a href="http://www.google.com">Google</a>'
root = objectify.parse(StringIO(tag)).getroot()
You can now access any of the fields (like href) in the tag or the link text:
In [930]: root
Out[930]: <Element a at 0x88bd4b0>
In [931]: root.get('href')
Out[931]: 'http://www.google.com'
In [932]: root.text
Out[932]: 'Google'
[/code]

The outputs for line 930 and 931 are the same as line 932 (i.e., Google). Please explain


10.

[code]
One of the easiest ways to store data efficiently in binary format is using Python's builtin
pickle serialization. Conveniently, pandas objects all have a save method which
writes the data to disk as a pickle:
In [933]: frame = pd.read_csv('ch06/ex1.csv')
In [934]: frame
Out[934]:
 a b c d message
0 1 2 3 4 hello
1 5 6 7 8 world
2 9 10 11 12 foo
In [935]: frame.save('ch06/frame_pickle')
You read the data back into Python with pandas.load, another pickle convenience
function:
In [936]: pd.load('ch06/frame_pickle')
Out[936]:
 a b c d message
0 1 2 3 4 hello
1 5 6 7 8 world
2 9 10 11 12 foo

[/code]

I couldn't run this code successfully. Please explain.


11.
[code]
HDF5 to provide multiple flexible data containers, table indexing, querying capability,
and some support for out-of-core computations.
pandas has a minimal dict-like HDFStore class, which uses PyTables to store pandas
objects:
In [937]: store = pd.HDFStore('mydata.h5')
In [938]: store['obj1'] = frame
In [939]: store['obj1_col'] = frame['a']
In [940]: store
Out[940]:
<class 'pandas.io.pytables.HDFStore'>
File path: mydata.h5
obj1 DataFrame
obj1_col Series
Objects contained in the HDF5 file can be retrieved in a dict-like fashion:
In [941]: store['obj1']
Out[941]:
 a b c d message
0 1 2 3 4 hello
1 5 6 7 8 world
2 9 10 11 12 foo

[/code]

Do I need to import Pytables in order to successfully run this code?


12.
[code]
We can then make a list of the tweet fields of interest then pass the results list to DataFrame:
In [951]: tweet_fields = ['created_at', 'from_user', 'id', 'text']
In [952]: tweets = DataFrame(data['results'], columns=tweet_fields)
In [953]: tweets
Out[953]:
<class 'pandas.core.frame.DataFrame'>
Int64Index: 15 entries, 0 to 14
Data columns:
created_at 15 non-null values
from_user 15 non-null values
id 15 non-null values
text 15 non-null values
dtypes: int64(1), object(3)
Each row in the DataFrame now has the extracted data from each tweet:
In [121]: tweets.ix[7]
Out[121]:
created_at Thu, 23 Jul 2012 09:54:00 +0000
from_user deblike
id 227419585803059201
text pandas: powerful Python data analysis toolkit
Name: 7
[/code]

An error message occured: 
KeyError: 'Results'

Please explain.

13.
[code]
Storing and Loading Data in MongoDB
NoSQL databases take many different forms. Some are simple dict-like key-value stores
like BerkeleyDB or Tokyo Cabinet, while others are document-based, with a dict-like
object being the basic unit of storage. I've chosen MongoDB (http://mongodb.org) for
my example. I started a MongoDB instance locally on my machine, and connect to it
on the default port using pymongo, the official driver for MongoDB:
import pymongo
con = pymongo.Connection('localhost', port=27017)
[/code]

I have trouble importing pymongo as it is a database package. How do I import MongoDB properly into python 2.7?

Again, your help will be greatly appreciated!


From kevin.p.dwyer at gmail.com  Fri Jun 26 01:54:04 2015
From: kevin.p.dwyer at gmail.com (Kev Dwyer)
Date: Fri, 26 Jun 2015 06:54:04 +0100
Subject: mutual coaching
References: <703059a2-621c-4205-b0a2-5bacfa632901@googlegroups.com>
Message-ID: <mmipdu$a3d$1@ger.gmane.org>

adhamyos at gmail.com wrote:

> hello anyone wants to study python? we can learn together! pm me my name
> is adham128 iam at the #programming room

Welcome to Python!

To improve your chances of finding someone who wants to learn with you, you 
might try posting your message in the Python Tutor list too: 
https://mail.python.org/mailman/listinfo/tutor

Good luck!

Kev



From steve at pearwood.info  Fri Jun 26 02:26:04 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 26 Jun 2015 16:26:04 +1000
Subject: Questions on Pandas
References: <663b9bb8-8252-4982-a959-4f68f7d35414@googlegroups.com>
Message-ID: <558cf07b$0$1655$c3e8da3$5496439d@news.astraweb.com>

On Fri, 26 Jun 2015 02:34 pm, Tommy C wrote:

> Hi there, I have a number of questions related to the Pandas exercises
> found from the book, Python for Data Analysis by Wes McKinney.
> Particularly, these exercises are from Chapter 6 of the book. It'd be much
> appreciated if you could answer the following questions!

Too many questions for one post!


> Does the header appear as "X.#" by default when it is set to be None?

Why don't you try setting header to something else and see what happens?


> 2.
> [code]
> Input: chunker = pd.read_csv('ch06/ex6.csv', chunksize=1000)
> Input: chunker
> Output: <pandas.io.parsers.TextParser at 0x8398150>
> 
> [/code]
> 
> Please explain the idea of chunksize and the output meaning.

The output shows that chunker is a TextParser object.

The chunksize let's you set how much data is read at once. Just like it says
in your next question:

> 3.
> [code]
> The TextParser object returned by read_csv allows you to iterate over the
> parts of the file according to the chunksize. 

Do you have access to the csv file? How many rows does it take to get the
results you see below?

> For example, we can iterate 
> over ex6.csv, aggregating the value counts in the 'key' column like so:
> chunker = pd.read_csv('ch06/ex6.csv', chunksize=1000)
> tot = Series([])
> for piece in chunker:
>  tot = tot.add(piece['key'].value_counts(), fill_value=0)
> tot = tot.order(ascending=False)
> We have then:
> In [877]: tot[:10]
> Out[877]:
> E 368
> X 364
> L 346
> O 343
> Q 340
> M 338
> J 337
> F 335
> K 334
> H 330
> 
> [/code]
> 
> I couldn't run the Series function successfully... is there something
> missing in this code?

I don't know. What did you do, and what error did you get? "I couldn't run
this successfully..." could mean anything:

- is the keyboard plugged in?
- did you import Pandas?
- did you make a typo?

and about a million other possible things could have gone wrong. Unless you
tell us *what you did* and *what happened*, how can we possibly guess why
you can't run the code?


> Error occured as I tried to run this code with sys.stdout.

Again, are we supposed to guess what the error was?

And now I'm bored and stopped reading. I suggest you think a bit more
carefully about the questions you ask, and the way you ask them. Imagine
that we're not watching over your shoulder to see what you did wrong when
you get an error. Don't assume that an error means the tutorial is wrong.
Please give more detail: what you did, and COPY AND PASTE the result you
got, don't summarise it, or re-type it from memory.

Ideally, you should have one question per post, or at least no more than a
few *related* questions. No, "all part of the same tutorial" doesn't make
them related. Try to give each set of questions a descriptive subject line,
so people can keep track of what you are asking and which questions they
care about and which ones they don't.



-- 
Steven



From denismfmcmahon at gmail.com  Fri Jun 26 03:44:45 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Fri, 26 Jun 2015 07:44:45 +0000 (UTC)
Subject: enumerate XML tags (keys that will become headers) along with
 text (values) and write to CSV in one row (as opposed to "stacked" values
 with one header)
References: <14aeae7a-41ab-4619-8331-7995e2420e54@googlegroups.com>
Message-ID: <mmivtd$fqa$1@dont-email.me>

On Thu, 25 Jun 2015 11:39:53 -0700, kbtyo wrote:

> My question can be found here:
> 
> http://stackoverflow.com/questions/31058100/enumerate-column-headers-in-
csv-that-belong-to-the-same-tag-key-in-python

I suggest you look on stack overflow for the answer.

You appear to have failed to comprehend (again) the fact that the xml 
data is an unordered list, and are trying to assign an order to it.

If the xml data was ordered, either each tag would be different, or each 
tag would have an attribute specifying a sequence number.

If the original data is ordered, then you need to go back to the code 
that creates the xml and make it add the ordering information to the xml 
(ideally as an attribute of the tags concerned) to specify the order.

You can not reliably and with any certainty of being accurate try and 
assign some sequence to the list of similarly named elements simply from 
their position in the list.

You don't know if the code that creates the xml is walking the array like 
this:

i = sizeof(arr);
while (i--) addXmlElement(arr[i]);

or like this:

for (i = 0; i < arr.len; i++) addXmlElement(arr[i]);

or even like this:

while (sizeof(arr)) addXmlElement(popArrElement(arr[sizeof(arr)/2]));

Granted the latter is unlikely and perverse, but still not wholly 
impossible. More perverse and less likely mechanisms are also available.

But my point remains - that if the ordering data is not passed into the 
xml, then you can not magically add it when you later use the xml.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From jon+usenet at unequivocal.co.uk  Fri Jun 26 06:49:06 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Fri, 26 Jun 2015 10:49:06 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.87.1435278799.3674.python-list@python.org>
Message-ID: <slrnmoqbjk.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-26, Chris Angelico <rosuav at gmail.com> wrote:
> On Fri, Jun 26, 2015 at 1:26 AM, Jon Ribbens
><jon+usenet at unequivocal.co.uk> wrote:
>> Well, it means you need to send 256 times as much data, which is a
>> start. If you're instead using a 256-byte translation table then
>> an attack becomes utterly impractical.
>
> Utterly impractical? Maybe, if you attempt a pure brute-force approach
> - there are 256! possible translation tables, which is roughly e500
> attempts [1], and at roughly four a microsecond [2] that'd still take
> a ridiculously long time. But there are two gigantic optimizations you
> could do. Firstly, there are frequency-based attacks,

No, there aren't. As I already said, the attacker does not have the
ciphertext. He can't do anything related to frequency analysis.


From kwpolska at gmail.com  Fri Jun 26 07:16:22 2015
From: kwpolska at gmail.com (Chris Warrick)
Date: Fri, 26 Jun 2015 13:16:22 +0200
Subject: [ANN] PySWITCH 0.2
In-Reply-To: <CAPTXd0pRN=PO+OMEcSLraiss2-KBVFNZbHdnJL3YcuzPX0XmSg@mail.gmail.com>
References: <CAPTXd0pRN=PO+OMEcSLraiss2-KBVFNZbHdnJL3YcuzPX0XmSg@mail.gmail.com>
Message-ID: <CAMw+j7K371-crgwS_mbnV-k2na4ASjzvddgdufPkhz6u6inW_Q@mail.gmail.com>

On 26 June 2015 at 04:42, Godson Gera <godson.g at gmail.com> wrote:
>
> =========
> pyswitch 0.2
> =========
>
> PySWITCH 0.2 is released
>
> Please, note that PySWITCH 0.2 is not available on PyPI because of name conflict

This is not a good idea. You should just change your name, or upload
under a slightly different name to PyPI. Everyone is on PyPI, everyone
expects to be able to type `pip install`, so please work on that.

> [snip]
> Project website: http://pyswitch.sf.net
> Download Page: http://pyswitch.sourceforge.net/pages/download.html

You are not going to get a lot of downloads with that. SourceForge
spreads malware, and lots of people avoid them nowadays.

Upload to PyPI and switch to GitHub, or you won?t be successful in this world.

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16


From rosuav at gmail.com  Fri Jun 26 08:02:34 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 26 Jun 2015 22:02:34 +1000
Subject: [ANN] PySWITCH 0.2
In-Reply-To: <CAMw+j7K371-crgwS_mbnV-k2na4ASjzvddgdufPkhz6u6inW_Q@mail.gmail.com>
References: <CAPTXd0pRN=PO+OMEcSLraiss2-KBVFNZbHdnJL3YcuzPX0XmSg@mail.gmail.com>
 <CAMw+j7K371-crgwS_mbnV-k2na4ASjzvddgdufPkhz6u6inW_Q@mail.gmail.com>
Message-ID: <CAPTjJmpVFEeJ18dDXpYgUwn34CKKeyPPYFV5=kQOVeU_Dnye3Q@mail.gmail.com>

On Fri, Jun 26, 2015 at 9:16 PM, Chris Warrick <kwpolska at gmail.com> wrote:
>> Project website: http://pyswitch.sf.net
>> Download Page: http://pyswitch.sourceforge.net/pages/download.html
>
> You are not going to get a lot of downloads with that. SourceForge
> spreads malware, and lots of people avoid them nowadays.
>
> Upload to PyPI and switch to GitHub, or you won?t be successful in this world.

Well, GitHub isn't the only option by any means, but I agree that SF
isn't going to inspire confidence.

ChrisA


From mal at europython.eu  Fri Jun 26 10:50:54 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Fri, 26 Jun 2015 16:50:54 +0200
Subject: EuroPython 2015: Standard rates deadline extended
Message-ID: <558D66CE.6000209@europython.eu>

We had originally planned switching to On-Desk Rates yesterday, but
because ticket sales are going well, we are extending the deadline for
more than a week.

               *** Buy your EuroPython 2015 tickets ***

             https://ep2015.europython.eu/en/registration/


On-Desk Rates
-------------

On-Desk Rates will be active starting on Monday, 2015-07-06, 00:00:00 CEST:

 * Student: 180 EUR
 * Personal: 440 EUR
 * Business: 690 EUR
   (all incl. 10% Spanish VAT)

This means that you can save up to 30% on ticket prices, if you buy
your tickets before:

                   Sunday, 2015-07-05, 23:59:59 CEST

and still benefit from our Standard Rates:

 * Student: 120 EUR
 * Personal: 340 EUR
 * Business: 530 EUR
   (all incl. 10% Spanish VAT)

Social Event
------------

Please also remember to book your Social Event Ticket. The price for
this, EUR 40, won?t change, but we only have a limited number of
tickets available.

       https://ep2015.europython.eu/en/events/social-event/

For more details on the social event ticket, please see our announcement.

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From steve at pearwood.info  Fri Jun 26 13:06:57 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 27 Jun 2015 03:06:57 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
Message-ID: <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>

On Fri, 26 Jun 2015 11:01 am, Ian Kelly wrote:

> You're making the same mistake that Steven did in misunderstanding the
> threat model.

I don't think I'm misunderstanding the threat, I think I'm pointing out a
threat which the OP is hoping to just ignore.

In an earlier post, I suggested that the threat model should involve at
least *three* different attacks, apart from the usual man-in-the-model
attacks of data in transit.

One is that the attacker is the person sending the data. E.g. I want to send
a nasty payload (say, malware, or an offensive image). Another is that the
attacker is the recipient of the file, who wants to read the sender's data.

As far as I can tell, the OP's plan to defend the sender's privacy is to
dump responsibility for encrypting the files in the sender's lap. As far as
I'm concerned, perhaps as many as one user in 20000 will pre-encrypt their
files. (Early adopters will be unrepresentative of the eventual user base
of this system. If this takes off, the user base will likely end up
dominated by people who think that "qwerty" is the epitome of unguessable
passwords.)

Users just don't use crypto unless their applications do it for them.

My opinion is that the application ought to do so, and not expect Aunt
Tillie to learn how to correctly use encryption software before uploading
her files. 

http://www.catb.org/jargon/html/A/Aunt-Tillie.html

It is the OP's prerogative to disagree, of course, but to me, if the OP's
app doesn't use strong crypto to encrypt users' data, that's tantamount to
saying they don't care about their users' data privacy. Using a
monoalphabetic substitution cipher to obfuscate the data is not strong
crypto.


> The goal isn't to prevent the attacker from working out 
> the key for a file that has already been obfuscated. Any real data
> that might be exposed by a vulnerability in the server is presumed to
> have already been strongly encrypted by the user.

I think that's a ridiculously unrealistic presumption, unless your user-base
is entirely taken from a very small subset of security savvy and
pedantically careful users.


> The goal is to prevent the attacker from guessing a key that hasn't
> even been generated yet, which could be exploited to engineer the
> obfuscated content into something malicious.

They don't need to predict the key exactly. If they can predict that the key
will be, lets say, one of these thousand values, then they can generate one
thousand files and upload them. One of them will match the key, and there's
your exploit. That's one attack.

A second attack is to force the key. The attacker controls the machine the
application is running on, they control /dev/urandom and can feed your app
whatever not-so-random numbers they like, so potentially they can force the
app to use the key of their choosing. Then they don't need 1000 files, they
just need one.

That's two. Does anyone think that I've thought of all the possible attacks?

(Well, hypothetical attacks. I acknowledge that I don't know the
application, and cannot be sure that it *actually is* vulnerable to these
attacks.)

The problem here is that a monoalphabetic substitution cipher is not
resistant to preimage attacks. Your only defence is that the key is
unknown. If the attacker can force the key, or predict the key, or guess a
small range of keys, they can exploit your weak cipher.

(Technically, "preimage attack" is usually used to refer to attacks on hash
functions. I'm not sure if the same name is used for attacks on ciphers.)

https://en.wikipedia.org/wiki/Preimage_attack

With a strong crypto cipher, there are no known preimage attacks. Even if
the attacker knows exactly what key you are using, they cannot predict what
preimage they need to supply in order to generate the malicious payload
they want after encryption. (As far as I know.)

That is the critical issue right there. The sort of simple monoalphabetic
substitution cipher using bytes.translate that the OP is using is
vulnerable to preimage attacks. Strong crypto is not.


> There are no 
> frequency-based attacks possible here, because you can't do frequency
> analysis on the result of a key that hasn't even been generated yet.

Frequency-based attacks apply to a different threat. I'm referring to at
least two different attacks here, with different attackers and different
victims. Don't mix them up.


> Assuming that you have no attack on the key generation itself, the

Not a safe assumption!


> best you can do is send a file deobfuscated with a random key and hope
> that the recipient randomly chooses the same key; the odds of that
> happening are 1 in 256!.

It's easy to come up with attacks which are no better than brute force. It's
the attacks which are better than brute force that you have to watch out
for.


-- 
Steven



From georgeryoung at gmail.com  Fri Jun 26 13:12:19 2015
From: georgeryoung at gmail.com (georgeryoung at gmail.com)
Date: Fri, 26 Jun 2015 10:12:19 -0700 (PDT)
Subject: how to replace some methods with instrumented ones
Message-ID: <1bd45a06-6141-429d-89bc-79b5aa7de0ff@googlegroups.com>

[python 2.7, linux]
I have a python app. I cannot modify the file.  But I can import it and mess with it.  I need to perform brief tasks before and after some of the member functions.  
I'd like to do this in as clear and maintainable way as possible (no third party imports).  Here's what I have now(which works):

from newvbr import *  #get all the stuff from the newvbr app.

def perf_time(tag):  # I do a little more than this, but you get the idea.
    print >>sys.stderr, datetime.datetime.now().isoformat() + tag

def perf(tag):
    def decorator(fn):
        def wrapper(*args):
            perf_time('start %s' % tag)
            ret = fn(*args)
            perf_time('end %s' % tag)
            return ret
        return wrapper
    return decorator


I use this on various members of various classes in the newvbr app:

old_database_snapshot = VerticaSession.database_snapshot
@perf('db_snap')
def new_database_snapshot(self, name):
    old_database_snapshot(self, name)
VerticaSession.database_snapshot = new_database_snapshot

old_object_snapshot = VerticaSession.object_snapshot
@perf('obj_snap')
def new_object_snapshot(self, name):
    old_object_snapshot(self, name)
VerticaSession.object_snapshot = new_object_snapshot

main()  #do the usual stuff of the imported app, but with annotations.


This is still pretty laborious.  How can I make something roughly like:

   super_tagger(VerticaSession.database_snapshot) 

that does all the above?  I need to tag a bunch of function members like this.  I don't insist on using decorators, it just looked like that kind of problem.


From torriem at gmail.com  Fri Jun 26 13:32:48 2015
From: torriem at gmail.com (Michael Torrie)
Date: Fri, 26 Jun 2015 11:32:48 -0600
Subject: HOPE: A Python just-in-time compiler for astrophysical
 computations
In-Reply-To: <55898f07$0$3088$426a74cc@news.free.fr>
References: <mailman.664.1434846609.13271.python-list@python.org>
 <55898f07$0$3088$426a74cc@news.free.fr>
Message-ID: <558D8CC0.6000409@gmail.com>

On 06/23/2015 10:53 AM, Laurent Pointal wrote:
> Mark Lawrence wrote:
> 
>> Another beasty I've just stumbled across which you may find interesting
>> http://www.sciencedirect.com/science/article/pii/S2213133714000687
>  
> Why use a JIT complation when you could use some C++ generation then 
> compilation as Python module, like with Pythran ?
> 
> https://github.com/serge-sans-paille/pythran
> 
> For heavy computing, you may loose some time before running computation, to 
> have (generate) the right tool, then do the job with real bonus.

Not really. I suspect Hope caches the compiled subroutine so only the
very first running of the function on the very first execution of the
pthon program will be slow.  After that, the program will run quickly
each time.  It's a pretty good idea.

I've never heard of pythran; I'll have to check it out and see how it
compares to the ever-growing crop of Python dialect compilers.  Cython
could be used in a similar way as you suggest, and in PyPy we have the
RPython dialect that compiles to C.


From __peter__ at web.de  Fri Jun 26 13:37:14 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 26 Jun 2015 19:37:14 +0200
Subject: how to replace some methods with instrumented ones
References: <1bd45a06-6141-429d-89bc-79b5aa7de0ff@googlegroups.com>
Message-ID: <mmk2kf$odm$1@ger.gmane.org>

georgeryoung at gmail.com wrote:

> [python 2.7, linux]
> I have a python app. I cannot modify the file.  But I can import it and
> mess with it.  I need to perform brief tasks before and after some of the
> member functions.
> I'd like to do this in as clear and maintainable way as possible (no third
> party imports).  Here's what I have now(which works):
> 
> from newvbr import *  #get all the stuff from the newvbr app.
> 
> def perf_time(tag):  # I do a little more than this, but you get the idea.
>     print >>sys.stderr, datetime.datetime.now().isoformat() + tag
> 
> def perf(tag):
>     def decorator(fn):
>         def wrapper(*args):
>             perf_time('start %s' % tag)
>             ret = fn(*args)
>             perf_time('end %s' % tag)
>             return ret
>         return wrapper
>     return decorator
> 
> 
> I use this on various members of various classes in the newvbr app:
> 
> old_database_snapshot = VerticaSession.database_snapshot
> @perf('db_snap')
> def new_database_snapshot(self, name):
>     old_database_snapshot(self, name)
> VerticaSession.database_snapshot = new_database_snapshot
> 
> old_object_snapshot = VerticaSession.object_snapshot
> @perf('obj_snap')
> def new_object_snapshot(self, name):
>     old_object_snapshot(self, name)
> VerticaSession.object_snapshot = new_object_snapshot
> 
> main()  #do the usual stuff of the imported app, but with annotations.
> 
> 
> This is still pretty laborious.  How can I make something roughly like:
> 
>    super_tagger(VerticaSession.database_snapshot)
> 
> that does all the above?  I need to tag a bunch of function members like
> this.  I don't insist on using decorators, it just looked like that kind
> of problem.

Remember that

@foo
def bar(...):
   ...

is just syntactic sugar for

def bar(...): ...
bar = foo(bar)

In your case the latter form is more convenient, so instead of

> old_object_snapshot = VerticaSession.object_snapshot
> @perf('obj_snap')
> def new_object_snapshot(self, name):
>     old_object_snapshot(self, name)
> VerticaSession.object_snapshot = new_object_snapshot

use

VerticalSession.object_snapshot = perf("obj_snap")(VerticalSession.object_snapshot)

If you like you can write a helper function along these lines

def wrap_attr(obj, attrname, name=None):
    if name is None: 
        name = attrname
    setattr(obj, attrname
        perf(name)(getattr(obj, attrname)))

wrap_attr(VerticalSession, "object_snapshot")



From python at mrabarnett.plus.com  Fri Jun 26 13:44:42 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 26 Jun 2015 18:44:42 +0100
Subject: how to replace some methods with instrumented ones
In-Reply-To: <1bd45a06-6141-429d-89bc-79b5aa7de0ff@googlegroups.com>
References: <1bd45a06-6141-429d-89bc-79b5aa7de0ff@googlegroups.com>
Message-ID: <558D8F8A.7070003@mrabarnett.plus.com>

On 2015-06-26 18:12, georgeryoung at gmail.com wrote:
> [python 2.7, linux]
> I have a python app. I cannot modify the file.  But I can import it and mess with it.  I need to perform brief tasks before and after some of the member functions.
> I'd like to do this in as clear and maintainable way as possible (no third party imports).  Here's what I have now(which works):
>
> from newvbr import *  #get all the stuff from the newvbr app.
>
> def perf_time(tag):  # I do a little more than this, but you get the idea.
>      print >>sys.stderr, datetime.datetime.now().isoformat() + tag
>
> def perf(tag):
>      def decorator(fn):
>        a  def wrapper(*args):
>              perf_time('start %s' % tag)
>              ret = fn(*args)
>              perf_time('end %s' % tag)
>              return ret
>          return wrapper
>      return decorator
>
>
> I use this on various members of various classes in the newvbr app:
>
> old_database_snapshot = VerticaSession.database_snapshot
> @perf('db_snap')
> def new_database_snapshot(self, name):
>      old_database_snapshot(self, name)
> VerticaSession.database_snapshot = new_database_snapshot
>
> old_object_snapshot = VerticaSession.object_snapshot
> @perf('obj_snap')
> def new_object_snapshot(self, name):
>      old_object_snapshot(self, name)
> VerticaSession.object_snapshot = new_object_snapshot
>
> main()  #do the usual stuff of the imported app, but with annotations.
>
>
> This is still pretty laborious.  How can I make something roughly like:
>
>     super_tagger(VerticaSession.database_snapshot)
>
> that does all the above?  I need to tag a bunch of function members like this.  I don't insist on using decorators, it just looked like that kind of problem.
>
You could do something like this:

def super_tagger(module, function_name, tag):
     original_function = getattr(module, function_name)

     @perf(tag)
     def wrapper(*args, **kwargs):
         return original_function(*args, **kwargs)

     setattr(module, function_name, wrapper)

and then:

super_tagger(VerticaSession, 'database_snapshot', 'db'snap')


It might be cleaner to merge the functionality of 'perf' into it:

def super_tagger(module, function_name, tag):
     original_function = getattr(module, function_name)

     def wrapper(*args, **kwargs):
         perf_time('start %s' % tag)
         result = original_function(*args, **kwargs)
         perf_time('end %s' % tag)
         return result

     setattr(module, function_name, wrapper)



From sjeik_appie at hotmail.com  Fri Jun 26 13:48:54 2015
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Fri, 26 Jun 2015 17:48:54 +0000 (UTC)
Subject: windows and file names > 256 bytes
References: <mmgce2$cd4$1@ger.gmane.org> <mmgnbt$u0o$1@ger.gmane.org>
 <CAB1GNpS=7F3ebrHenUu7+MvyWuk5v60G6N6L81c0W3C-mQPi2w@mail.gmail.com>
 <558BF55C.2010701@timgolden.me.uk> <558C038A.5010103@gmail.com>
 <558C0433.2050601@timgolden.me.uk>
Message-ID: <mmk3a6$hrb$1@ger.gmane.org>

On Thu, 25 Jun 2015 14:37:55 +0100, Tim Golden wrote:

> On 25/06/2015 14:35, Michael Torrie wrote:
>> On 06/25/2015 06:34 AM, Tim Golden wrote:
>>> On 25/06/2015 13:04, Joonas Liik wrote:
>>>> It sounds to me more like it is possible to use long file names on
>>>> windows but it is a pain and in python, on windows it is basically
>>>> impossible.
>>>
>>> Certainly not impossible: you could write your own wrapper function:
>>>
>>> def extended_path(p):
>>>     return r"\\?\%s" % os.path.abspath(p)
>>>
>>> where you knew that there was a possibility of long paths and that an
>>> absolute path would work.
>> 
>> The OP mentions that even when he manually supplies extended paths,
>> os.mkdir, os.getsize, and shutil.rmtree return errors for him in Python
>> 2.7.  So there's more to this problem.
>> 
>> 
> He's probably not passing unicode strings: the extended path only works
> for unicode string. For 3.x that's what you do by default.

Hi all,

Thanks for your replies. I've been messing with this a bit more. I 
created a little test script (see below). However, this only works with 
drive letters, not with UNC paths. I tried using os.chdir, DOS pushd, 
subst, net use but they all don't seem to work with with looooong paths. 
I finally managed to remove an absurdly long dir with shutil.rmtree, 
after changing sys.setrecursionlimit. But my main goal was to get the 
file size (and, actually, also the file owner) of a long file name on XP.

import os
import shutil
import sys
 
# create an insanely long directory tree
p = os.getenv("TEMP")
#p = ur"\\server\share\blah\temp"
tmpdir = p
os.chdir(tmpdir)
for i in xrange(1000):
    tmpdir = os.path.join(tmpdir, "sub")
    os.mkdir("\\\\?\\" + tmpdir)
    #os.mkdir(u"\\\\?\\UNC" + tmpdir[1:])
 
# write a file to it
deep = "\\\\?\\" + os.path.join(tmpdir, "deep.txt")
assert os.path.exists(deep)
with open(deep, "w") as f:
    f.write("Deep!\r\n")
 
# try if the file size can be determined (requires special \\?\ notation)
print "@@@@ %d bytes" % os.path.getsize(deep)
 
# now delete the whole directory and its contents.
path = "\\\\?\\" + os.path.join(p, "sub")
path = path.decode(sys.getfilesystemencoding())
sys.setrecursionlimit(10 ** 7)  # net use, pushd, subst will not work
shutil.rmtree(path)

Any feedback is welcome. I will post the solution somewhere so somebody 
else will be spared this nuisance. :-)

Regards,
Albert-Jan


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus




From sjeik_appie at hotmail.com  Fri Jun 26 13:58:57 2015
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Fri, 26 Jun 2015 17:58:57 +0000 (UTC)
Subject: windows and file names > 256 bytes
References: <mmgce2$cd4$1@ger.gmane.org> <mmgnbt$u0o$1@ger.gmane.org>
 <CAB1GNpS=7F3ebrHenUu7+MvyWuk5v60G6N6L81c0W3C-mQPi2w@mail.gmail.com>
 <558BF55C.2010701@timgolden.me.uk> <558C038A.5010103@gmail.com>
 <558C0433.2050601@timgolden.me.uk> <mmk3a6$hrb$1@ger.gmane.org>
Message-ID: <mmk3t1$hrb$2@ger.gmane.org>

<snip> 

> import os import shutil import sys
>  
> # create an insanely long directory tree p = os.getenv("TEMP")
> #p = ur"\\server\share\blah\temp"
> tmpdir = p os.chdir(tmpdir)
> for i in xrange(1000):
>     tmpdir = os.path.join(tmpdir, "sub") os.mkdir("\\\\?\\" + tmpdir)
>     #os.mkdir(u"\\\\?\\UNC" + tmpdir[1:])
>  
> # write a file to it deep = "\\\\?\\" + os.path.join(tmpdir, "deep.txt")
> assert os.path.exists(deep)

sorry, this "assert" should of course follow 'with open(..' 

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus




From randall at tnr.cc  Fri Jun 26 16:09:51 2015
From: randall at tnr.cc (Randall Smith)
Date: Fri, 26 Jun 2015 15:09:51 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mmkbij$dpm$1@ger.gmane.org>

On 06/26/2015 12:06 PM, Steven D'Aprano wrote:
> On Fri, 26 Jun 2015 11:01 am, Ian Kelly wrote:
>
>> You're making the same mistake that Steven did in misunderstanding the
>> threat model.
>
> I don't think I'm misunderstanding the threat, I think I'm pointing out a
> threat which the OP is hoping to just ignore.

I'm not hoping to ignore anything.  I didn't explain the entire system, 
as it was not necessary to find a solution to the problem at hand.  But 
since you want to make negative assumptions about what I didn't tell 
you, I'll gladly address your accusations of negligence.

>
> In an earlier post, I suggested that the threat model should involve at
> least *three* different attacks, apart from the usual man-in-the-model
> attacks of data in transit.

All communication is secured using TLS and authentication handled by 
X.509 certificates.  This prevents man in the middle attacks. 
Certificates are signed by CAs I control.

>
> One is that the attacker is the person sending the data. E.g. I want to send
> a nasty payload (say, malware, or an offensive image). Another is that the
> attacker is the recipient of the file, who wants to read the sender's data.

The only person who can read a file is the owner.   AES encryption is 
built into the client software.  The only way data can be uploaded 
unencrypted is if encryption is intentionally disabled.

>
> As far as I can tell, the OP's plan to defend the sender's privacy is to
> dump responsibility for encrypting the files in the sender's lap. As far as
> I'm concerned, perhaps as many as one user in 20000 will pre-encrypt their
> files. (Early adopters will be unrepresentative of the eventual user base
> of this system. If this takes off, the user base will likely end up
> dominated by people who think that "qwerty" is the epitome of unguessable
> passwords.)

Making assumptions again.  See above.  The client software encrypts by 
default.  You're also assuming there is no password strength checking.

>
> Users just don't use crypto unless their applications do it for them.

And it does.

>
> My opinion is that the application ought to do so, and not expect Aunt
> Tillie to learn how to correctly use encryption software before uploading
> her files.
>
> http://www.catb.org/jargon/html/A/Aunt-Tillie.html
>
> It is the OP's prerogative to disagree, of course, but to me, if the OP's
> app doesn't use strong crypto to encrypt users' data, that's tantamount to
> saying they don't care about their users' data privacy. Using a
> monoalphabetic substitution cipher to obfuscate the data is not strong
> crypto.

You've gone on a rampage about nothing.  My original description said 
the client was supposed to encrypt the data, but you want to assume the 
opposite for some unknown reason.

>
>
>> The goal isn't to prevent the attacker from working out
>> the key for a file that has already been obfuscated. Any real data
>> that might be exposed by a vulnerability in the server is presumed to
>> have already been strongly encrypted by the user.
>
> I think that's a ridiculously unrealistic presumption, unless your user-base
> is entirely taken from a very small subset of security savvy and
> pedantically careful users.

The difference is he's not assuming I'm a moron.  He's giving me the 
benefit of the doubt.  That plus I actually said, "data senders are 
supposed to encrypt data".

In a networked system, you can't make assumptions about what the other 
peers are doing.  You have to handle what comes across the wire.  You 
also have to consider that you may come under attack.  That's what this 
is about.

>
>
>> The goal is to prevent the attacker from guessing a key that hasn't
>> even been generated yet, which could be exploited to engineer the
>> obfuscated content into something malicious.
>
> They don't need to predict the key exactly. If they can predict that the key
> will be, lets say, one of these thousand values, then they can generate one
> thousand files and upload them. One of them will match the key, and there's
> your exploit. That's one attack.

Thousand Values ???  Isn't it 256!, which is just freaking huge!  import 
math; math.factorial(256)

>
> A second attack is to force the key. The attacker controls the machine the
> application is running on, they control /dev/urandom and can feed your app
> whatever not-so-random numbers they like, so potentially they can force the
> app to use the key of their choosing. Then they don't need 1000 files, they
> just need one.
>

If the attacker controlled the machine the app was on, why would it fool 
with /dev/urandom?  I think he'd just plant the files he wanted to plant 
and be done.  This is non-nonsensical anyway.

> That's two. Does anyone think that I've thought of all the possible attacks?
>
> (Well, hypothetical attacks. I acknowledge that I don't know the
> application, and cannot be sure that it *actually is* vulnerable to these
> attacks.)
>
> The problem here is that a monoalphabetic substitution cipher is not
> resistant to preimage attacks. Your only defence is that the key is
> unknown. If the attacker can force the key, or predict the key, or guess a
> small range of keys, they can exploit your weak cipher.
>
> (Technically, "preimage attack" is usually used to refer to attacks on hash
> functions. I'm not sure if the same name is used for attacks on ciphers.)
>
> https://en.wikipedia.org/wiki/Preimage_attack
>
> With a strong crypto cipher, there are no known preimage attacks. Even if
> the attacker knows exactly what key you are using, they cannot predict what
> preimage they need to supply in order to generate the malicious payload
> they want after encryption. (As far as I know.)
>
> That is the critical issue right there. The sort of simple monoalphabetic
> substitution cipher using bytes.translate that the OP is using is
> vulnerable to preimage attacks. Strong crypto is not.

It isn't vulnerable to preimage attacks unless you can guess the key out 
of 256! possibilities.  The key doesn't even exist until after the data 
is sent.  Give me one plausible scenario where an attacker can cause 
malware to hit the disk after bytearray.translate with a 256 byte 
translation table and I'll be thankful to you.  As it stands now, you're 
either ignoring information I've already given, assuming I've made 
moronic design decisions the pouncing on them, or completely 
misunderstanding the issue at hand.

>
>
>> There are no
>> frequency-based attacks possible here, because you can't do frequency
>> analysis on the result of a key that hasn't even been generated yet.
>
> Frequency-based attacks apply to a different threat. I'm referring to at
> least two different attacks here, with different attackers and different
> victims. Don't mix them up.
>
>
>> Assuming that you have no attack on the key generation itself, the
>
> Not a safe assumption!

For this case it is a safe assumption.  For the same reason you're 
assuming the PSU isn't defective.  An attack on /dev/urandom for 
instance would also compromise TLS, and every sort of secure key 
generation, not just the key for byte translation.  That is a separate 
problem altogether with a separate solution.

>
>
>> best you can do is send a file deobfuscated with a random key and hope
>> that the recipient randomly chooses the same key; the odds of that
>> happening are 1 in 256!.
>
> It's easy to come up with attacks which are no better than brute force. It's
> the attacks which are better than brute force that you have to watch out
> for.
>

And that's why we're having this discussion.  Do you know of an attack 
in which you can control the output (say at least 100 consecutive bytes) 
for data which goes through a 256 byte translation table, chosen 
randomly from 256! permutations after the data is sent.  If you do, I'm 
all ears!  But at this point you're just setting up straw men and 
knocking them down.



From dfnsonfsduifb at gmx.de  Fri Jun 26 17:07:48 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Fri, 26 Jun 2015 23:07:48 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mailman.111.1435349412.3674.python-list@python.org>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
Message-ID: <mmkev4$uhl$1@news.albasani.net>

On 26.06.2015 22:09, Randall Smith wrote:

> You've gone on a rampage about nothing.  My original description said
> the client was supposed to encrypt the data, but you want to assume the
> opposite for some unknown reason.

While you seem to think that Steven is rampaging about nothing, he does
have a fair point: You consistently were vague about wheter you want to
have encryption, authentication or obfuscation of data. This suggests
that you may not be so sure yourself what it is you actually want.

All Steven is doing is pointing out that people do good crypto for a
reason. It's 2015 and we're still discussion "substitution ciphers",
really? Good crypto is available, it's fast, it has awesome
cryptanalysis. All Steven is pointing out is that when ten crypto-laymen
meet in a Python newsgroup and think they have invented a soooper secure
scheme, it may still be complete and utter crap. Just not everone can
see it.

You always play around with the 256! which would be a ridiculously high
security margin (1684 bits of security, woooo!). You totally ignore that
the system can be broken in a linear fashion. I don't need to know all
256 characters to do damage, sometimes even a handful will already give
me part of what I need and the option to crack more and more. This is
something that would ultimately and instantly disqualify your
"crypto"system as utterly insecure.

Nobody assumes you're a moron. But it's safe to assume that you're a
crypto layman, because only laymen have no clue on how difficult it is
to get cryptography even remotely right. Everyone who knows the trade
uses proven constructions not because it's inconvenient, but because
it's one of the very few ways to achieve a secure system.

That said, for your solution this type of obfuscation may be fine. And
chances are that nobody will ever notice. But don't claim you weren't
warned about the abyss when you designed your solution and people break
this stuff. Because then you might *look* like a moron (even if you're
not), since the first question people will ask will be: "Why? Why on
earth?" It's a blatantly obvious bad idea(tm).

That people in 2015 actually defend inventing a substitution-cipher
"crypto"system sends literally shivers down my spine.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From dfnsonfsduifb at gmx.de  Fri Jun 26 17:11:06 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Fri, 26 Jun 2015 23:11:06 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mailman.111.1435349412.3674.python-list@python.org>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
Message-ID: <mmkf5b$uvj$1@news.albasani.net>

On 26.06.2015 22:09, Randall Smith wrote:

> And that's why we're having this discussion.  Do you know of an attack
> in which you can control the output (say at least 100 consecutive bytes)
> for data which goes through a 256 byte translation table, chosen
> randomly from 256! permutations after the data is sent.  If you do, I'm
> all ears!  But at this point you're just setting up straw men and
> knocking them down.

Oh and I wanted to comment on this as well, but sent my reply too soon.

You misunderstand. This is now how it works, this is not how any of this
works. Steven does not *at all* have to prove to you your system is
breakable or show actual attacks. YOU have to prove that your system is
secure. Either analytically or you wait until you have peer review and
cryptanalysis by actual experts.

It's *very* easy to set up a badly flawed obfuscation system that can't
be broken by laymen in a Python newsgroup and which appers to be secure.
This does not imply one bit that it is even remotely secure.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From jon+usenet at unequivocal.co.uk  Fri Jun 26 17:29:56 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Fri, 26 Jun 2015 21:29:56 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkev4$uhl$1@news.albasani.net>
Message-ID: <slrnmorh57.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-26, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
> On 26.06.2015 22:09, Randall Smith wrote:
>> You've gone on a rampage about nothing.  My original description said
>> the client was supposed to encrypt the data, but you want to assume the
>> opposite for some unknown reason.
>
> While you seem to think that Steven is rampaging about nothing, he does
> have a fair point: You consistently were vague about wheter you want to
> have encryption, authentication or obfuscation of data. This suggests
> that you may not be so sure yourself what it is you actually want.

He hasn't been vague, you and Steven just haven't been paying
attention.

> You always play around with the 256! which would be a ridiculously high
> security margin (1684 bits of security, woooo!). You totally ignore that
> the system can be broken in a linear fashion.

No, it can't, because the attacker does not have access to the
ciphertext.

> Nobody assumes you're a moron. But it's safe to assume that you're a
> crypto layman, because only laymen have no clue on how difficult it is
> to get cryptography even remotely right.

Amateur crypto is indeed a bad idea. But what you're still not getting
is that what he's doing here *isn't crypto*. He's just trying to avoid
letting third parties write completely arbitrary data to the disk. You
know what would be a perfectly good solution to his problem? Base 64
encoding. That would solve the issue pretty much completely, the only
reason it's not an ideal solution is that it of course increases the
size of the data.

> That people in 2015 actually defend inventing a substitution-cipher
> "crypto"system sends literally shivers down my spine.

Nobody is defending such a thing, you just haven't understood what
problem is being solved here.


From breamoreboy at yahoo.co.uk  Fri Jun 26 17:55:11 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 26 Jun 2015 22:55:11 +0100
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <slrnmorh57.1nu.jon+usenet@frosty.unequivocal.co.uk>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkev4$uhl$1@news.albasani.net>
 <slrnmorh57.1nu.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <mmkho0$irr$1@ger.gmane.org>

On 26/06/2015 22:29, Jon Ribbens wrote:
> On 2015-06-26, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
>> On 26.06.2015 22:09, Randall Smith wrote:
>>> You've gone on a rampage about nothing.  My original description said
>>> the client was supposed to encrypt the data, but you want to assume the
>>> opposite for some unknown reason.
>>
>> While you seem to think that Steven is rampaging about nothing, he does
>> have a fair point: You consistently were vague about wheter you want to
>> have encryption, authentication or obfuscation of data. This suggests
>> that you may not be so sure yourself what it is you actually want.
>
> He hasn't been vague, you and Steven just haven't been paying
> attention.
>
>> You always play around with the 256! which would be a ridiculously high
>> security margin (1684 bits of security, woooo!). You totally ignore that
>> the system can be broken in a linear fashion.
>
> No, it can't, because the attacker does not have access to the
> ciphertext.
>
>> Nobody assumes you're a moron. But it's safe to assume that you're a
>> crypto layman, because only laymen have no clue on how difficult it is
>> to get cryptography even remotely right.
>
> Amateur crypto is indeed a bad idea. But what you're still not getting
> is that what he's doing here *isn't crypto*. He's just trying to avoid
> letting third parties write completely arbitrary data to the disk. You
> know what would be a perfectly good solution to his problem? Base 64
> encoding. That would solve the issue pretty much completely, the only
> reason it's not an ideal solution is that it of course increases the
> size of the data.
>
>> That people in 2015 actually defend inventing a substitution-cipher
>> "crypto"system sends literally shivers down my spine.
>
> Nobody is defending such a thing, you just haven't understood what
> problem is being solved here.
>

To be perfectly blunt I gave up days ago trying to follow what was being 
said, just too many words from all angles and too few diagrams for me to 
follow.  I sincerely hope it doesn't end in tears.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From ian.g.kelly at gmail.com  Fri Jun 26 17:58:45 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 26 Jun 2015 15:58:45 -0600
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmkev4$uhl$1@news.albasani.net>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkev4$uhl$1@news.albasani.net>
Message-ID: <CALwzidk+iGpowTSSK6QDV-=ouegLu2nZnuojM14OrnskZ=t_TQ@mail.gmail.com>

On Fri, Jun 26, 2015 at 3:07 PM, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
> That people in 2015 actually defend inventing a substitution-cipher
> "crypto"system sends literally shivers down my spine.

I think that the people defending this have been reasonably consistent
about using the word "obfuscation", not "crypto".


From trentonwesley10 at gmail.com  Fri Jun 26 18:07:17 2015
From: trentonwesley10 at gmail.com (trentonwesley10 at gmail.com)
Date: Fri, 26 Jun 2015 15:07:17 -0700 (PDT)
Subject: Why pay DICE When TheGongzuo.com is !! FREE !!
Message-ID: <f5a5d7cf-f8b6-4a9b-8386-c3c3e16b791c@googlegroups.com>

Greetings!
You been Invited as a Beta User for TheGongzuo.com ( Absolutely Extended Trial).
We bring to you TheGongzuo.com, Top notch highly talented IT (Information Technology / Software Industry) skilled Professional Candidates, where you can find more than a resume.
________________________________________
For a NO COST TRIAL all you have to do is:
	*	Register & Post  only 1 Active Job.
*	Start interacting with more 100,000 IT professionals and Innovators/Ideators.
________________________________________
Before you get started , there are few things you should know,
1.       We are getting better  :- We will be updating our site with new material daily basis and we welcome your feedback on newly added material. Please send us your feedback by using Feedback option on the site or send us an email @ support at thegongzuo.com.
2.       We like Criticism   :-  As a fresh site, We are sure you can find lots to Criticize - remember , though we would prefer constructive criticism, so we can make site better. Please send us your valuable criticism by email @ support at thegongzuo.com.

________________________________________
 
Please make us a part of your distribution list,
We will start sending resumes to you.
If you do not wish to be a part of our mailing list just reply to us with unsubscribe from this list or else it will consider it as your consent to be part of our distribution list.

________________________________________
Thanks for trying TheGongzuo.com and please let us know , What you think !

Thanks,
TheGongzuo.com Team !



From dfnsonfsduifb at gmx.de  Fri Jun 26 18:42:52 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sat, 27 Jun 2015 00:42:52 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <slrnmorh57.1nu.jon+usenet@frosty.unequivocal.co.uk>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkev4$uhl$1@news.albasani.net>
 <slrnmorh57.1nu.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <mmkkhc$m5d$1@news.albasani.net>

On 26.06.2015 23:29, Jon Ribbens wrote:

>> While you seem to think that Steven is rampaging about nothing, he does
>> have a fair point: You consistently were vague about wheter you want to
>> have encryption, authentication or obfuscation of data. This suggests
>> that you may not be so sure yourself what it is you actually want.
> 
> He hasn't been vague, you and Steven just haven't been paying
> attention.

Bullshit. Even the topic indicates that he doesn't know what he wants:
"data mangling" or "encryption", which one is it?

>> You always play around with the 256! which would be a ridiculously high
>> security margin (1684 bits of security, woooo!). You totally ignore that
>> the system can be broken in a linear fashion.
> 
> No, it can't, because the attacker does not have access to the
> ciphertext.

Or so you claim.

I could go into detail about how the assumtion that the ciphertext is
secret is not a smart one in the context of cryptography. And how side
channels and other leakage may affect overall system security. But I'm
going to save my time on that. I do get paid to review cryptographic
systems and part of the job is dealing with belligerent people who have
read Schneier's blog and think they can outsmart anyone else. Since I
don't get paid to convice you, it's absolutely fine that you think your
substitution scheme is the grand prize.

>> Nobody assumes you're a moron. But it's safe to assume that you're a
>> crypto layman, because only laymen have no clue on how difficult it is
>> to get cryptography even remotely right.
> 
> Amateur crypto is indeed a bad idea. But what you're still not getting
> is that what he's doing here *isn't crypto*. 

So the topic says "Encrypting". If you look really closely at the word,
the part "crypt" might give away to you that cryptography is involved.

> He's just trying to avoid
> letting third parties write completely arbitrary data to the disk.

There's your requirement. Then there's obviously some kind of
implication when a third party *can* write arbitrary data to disk. And
your other solution to that problem...

> You
> know what would be a perfectly good solution to his problem? Base 64
> encoding. That would solve the issue pretty much completely, the only
> reason it's not an ideal solution is that it of course increases the
> size of the data.

...wow.

That's a nice interpretation of not letting a third party write
completely arbitrary data. According to your definition, this would be:
It's okay if the attacker can control 6 of 8 bits.

>> That people in 2015 actually defend inventing a substitution-cipher
>> "crypto"system sends literally shivers down my spine.
> 
> Nobody is defending such a thing, you just haven't understood what
> problem is being solved here.

Oh I understand your "solutions" plenty well. The only thing I don't
understand is why you don't own a Fields medal yet for your
groundbreaking work on bulletproof obfuscation.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From jeanpierreda at gmail.com  Fri Jun 26 19:26:33 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Fri, 26 Jun 2015 16:26:33 -0700
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmkkhc$m5d$1@news.albasani.net>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkev4$uhl$1@news.albasani.net>
 <slrnmorh57.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <mmkkhc$m5d$1@news.albasani.net>
Message-ID: <CABicbJ+d6kMSVuv7TePQY22_j62s3Z1AmC8tV5UMdWkRi1F0UA@mail.gmail.com>

Johannes, I agree with a lot of what you say, but can you please have
less of a mean attitude?

-- Devin

On Fri, Jun 26, 2015 at 3:42 PM, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
> On 26.06.2015 23:29, Jon Ribbens wrote:
>
>>> While you seem to think that Steven is rampaging about nothing, he does
>>> have a fair point: You consistently were vague about wheter you want to
>>> have encryption, authentication or obfuscation of data. This suggests
>>> that you may not be so sure yourself what it is you actually want.
>>
>> He hasn't been vague, you and Steven just haven't been paying
>> attention.
>
> Bullshit. Even the topic indicates that he doesn't know what he wants:
> "data mangling" or "encryption", which one is it?
>
>>> You always play around with the 256! which would be a ridiculously high
>>> security margin (1684 bits of security, woooo!). You totally ignore that
>>> the system can be broken in a linear fashion.
>>
>> No, it can't, because the attacker does not have access to the
>> ciphertext.
>
> Or so you claim.
>
> I could go into detail about how the assumtion that the ciphertext is
> secret is not a smart one in the context of cryptography. And how side
> channels and other leakage may affect overall system security. But I'm
> going to save my time on that. I do get paid to review cryptographic
> systems and part of the job is dealing with belligerent people who have
> read Schneier's blog and think they can outsmart anyone else. Since I
> don't get paid to convice you, it's absolutely fine that you think your
> substitution scheme is the grand prize.
>
>>> Nobody assumes you're a moron. But it's safe to assume that you're a
>>> crypto layman, because only laymen have no clue on how difficult it is
>>> to get cryptography even remotely right.
>>
>> Amateur crypto is indeed a bad idea. But what you're still not getting
>> is that what he's doing here *isn't crypto*.
>
> So the topic says "Encrypting". If you look really closely at the word,
> the part "crypt" might give away to you that cryptography is involved.
>
>> He's just trying to avoid
>> letting third parties write completely arbitrary data to the disk.
>
> There's your requirement. Then there's obviously some kind of
> implication when a third party *can* write arbitrary data to disk. And
> your other solution to that problem...
>
>> You
>> know what would be a perfectly good solution to his problem? Base 64
>> encoding. That would solve the issue pretty much completely, the only
>> reason it's not an ideal solution is that it of course increases the
>> size of the data.
>
> ...wow.
>
> That's a nice interpretation of not letting a third party write
> completely arbitrary data. According to your definition, this would be:
> It's okay if the attacker can control 6 of 8 bits.
>
>>> That people in 2015 actually defend inventing a substitution-cipher
>>> "crypto"system sends literally shivers down my spine.
>>
>> Nobody is defending such a thing, you just haven't understood what
>> problem is being solved here.
>
> Oh I understand your "solutions" plenty well. The only thing I don't
> understand is why you don't own a Fields medal yet for your
> groundbreaking work on bulletproof obfuscation.
>
> Cheers,
> Johannes
>
> --
>>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
>> Zumindest nicht ?ffentlich!
> Ah, der neueste und bis heute genialste Streich unsere gro?en
> Kosmologen: Die Geheim-Vorhersage.
>  - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>
> --
> https://mail.python.org/mailman/listinfo/python-list


From randall at tnr.cc  Fri Jun 26 20:12:08 2015
From: randall at tnr.cc (Randall Smith)
Date: Fri, 26 Jun 2015 19:12:08 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmkho0$irr$1@ger.gmane.org>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkev4$uhl$1@news.albasani.net>
 <slrnmorh57.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <mmkho0$irr$1@ger.gmane.org>
Message-ID: <mmkpos$b7d$1@ger.gmane.org>

On 06/26/2015 04:55 PM, Mark Lawrence wrote:

>
> To be perfectly blunt I gave up days ago trying to follow what was being
> said, just too many words from all angles and too few diagrams for me to
> follow.  I sincerely hope it doesn't end in tears.
>

Mark.

There's not much to follow.  The solution was simple and complete.

The original description was limited to a small part of a large, more 
complex system.  The reason you've had trouble following is because 
several people made (very bad) assumptions about what the rest of the 
system did.  Everything required for the solution was present in the 
original post.

-Randall



From jon+usenet at unequivocal.co.uk  Fri Jun 26 20:21:40 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sat, 27 Jun 2015 00:21:40 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkev4$uhl$1@news.albasani.net>
 <slrnmorh57.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <mmkkhc$m5d$1@news.albasani.net>
Message-ID: <slrnmorr76.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-26, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
> On 26.06.2015 23:29, Jon Ribbens wrote:
>>> While you seem to think that Steven is rampaging about nothing, he does
>>> have a fair point: You consistently were vague about wheter you want to
>>> have encryption, authentication or obfuscation of data. This suggests
>>> that you may not be so sure yourself what it is you actually want.
>> 
>> He hasn't been vague, you and Steven just haven't been paying
>> attention.
>
> Bullshit. Even the topic indicates that he doesn't know what he wants:
> "data mangling" or "encryption", which one is it?

He wants data mangling and he was asking whether he needed encryption
to achieve it. The answer is no, he doesn't.

> I could go into detail about how the assumtion that the ciphertext is
> secret is not a smart one in the context of cryptography.

But, and I've already pointed this out and you don't seem to have
quite got the picture yet, we're not in the context of cryptography.

> And how side channels and other leakage may affect overall system
> security. But I'm going to save my time on that. I do get paid to
> review cryptographic systems and part of the job is dealing with
> belligerent people who have read Schneier's blog and think they can
> outsmart anyone else.

You seem to be describing your own attitude to a tee.

> Since I don't get paid to convice you, it's absolutely fine that you
> think your substitution scheme is the grand prize.

"My" scheme? It wasn't my suggestion.

> So the topic says "Encrypting". If you look really closely at the word,
> the part "crypt" might give away to you that cryptography is involved.

If you were to actually read past the subject line and continue on to
read the text of the articles, you would discover that cryptography is
not involved. No wonder you're confused if you're disengaging your
brain the instant you get past the subject line.

>> He's just trying to avoid letting third parties write completely
>> arbitrary data to the disk.
>
> There's your requirement.

"My" requirement?

>> You know what would be a perfectly good solution to his problem?
>> Base 64 encoding. That would solve the issue pretty much
>> completely, the only reason it's not an ideal solution is that it
>> of course increases the size of the data.
>
> ...wow.
>
> That's a nice interpretation of not letting a third party write
> completely arbitrary data.

It's an accurate interpretation. Something that seems not to be your
forte.

> According to your definition, this would be: It's okay if the
> attacker can control 6 of 8 bits.

Yes, it probably is ok. Add a bit of random gunk at the top and tail
of the file and it's almost certainly ok. Why do you think it's not?

> Oh I understand your "solutions" plenty well.

Evidently not.

> The only thing I don't understand is why you don't own a Fields
> medal yet for your groundbreaking work on bulletproof obfuscation.

That is clearly a very long way from the only thing you don't
understand.


From randall at tnr.cc  Fri Jun 26 20:23:26 2015
From: randall at tnr.cc (Randall Smith)
Date: Fri, 26 Jun 2015 19:23:26 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmkev4$uhl$1@news.albasani.net>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkev4$uhl$1@news.albasani.net>
Message-ID: <mmkqe2$kh4$1@ger.gmane.org>

On 06/26/2015 04:07 PM, Johannes Bauer wrote:
> You consistently were vague about wheter you want to
> have encryption, authentication or obfuscation of data.

I knew (possibly extra) encryption wasn't necessary at this stage, but I 
also knew that encryption would provide good obfuscation.  Problem is, I 
didn't want an extra C library to install. See the original post.

"... I'd like to protect the recipient against exposure to nefarious 
data by mangling or encrypting the data before it is written to disk. My 
original idea was for the recipient to encrypt using AES.  But I want to 
keep this software pure Python "batteries included" and not require 
installation of other platform-dependent software ... I don't know that 
I really need encryption here, but some type of fast mangling algorithm 
where a bad actor sending a payload can't guess the output ahead of time."

-Randall





From randall at tnr.cc  Fri Jun 26 20:55:16 2015
From: randall at tnr.cc (Randall Smith)
Date: Fri, 26 Jun 2015 19:55:16 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmkkhc$m5d$1@news.albasani.net>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkev4$uhl$1@news.albasani.net>
 <slrnmorh57.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <mmkkhc$m5d$1@news.albasani.net>
Message-ID: <mmks9p$fbe$1@ger.gmane.org>

On 06/26/2015 05:42 PM, Johannes Bauer wrote:
> On 26.06.2015 23:29, Jon Ribbens wrote:
>
>>> While you seem to think that Steven is rampaging about nothing, he does
>>> have a fair point: You consistently were vague about wheter you want to
>>> have encryption, authentication or obfuscation of data. This suggests
>>> that you may not be so sure yourself what it is you actually want.
>>
>> He hasn't been vague, you and Steven just haven't been paying
>> attention.
>
> Bullshit. Even the topic indicates that he doesn't know what he wants:
> "data mangling" or "encryption", which one is it?

I knew exactly what I wanted and spelled it out.

"protect the recipient against exposure to nefarious data ... before it 
is written to disk"

You shouldn't need to make assumptions about other parts of the system. 
  Just prevent potential malware from hitting the disk as such.

Before this thread, I knew that encryption would definitely work and 
data mangling might.  Now I know that data mangling is a really nice 
solution for the given requirements.


>
>>> You always play around with the 256! which would be a ridiculously high
>>> security margin (1684 bits of security, woooo!). You totally ignore that
>>> the system can be broken in a linear fashion.
>>
>> No, it can't, because the attacker does not have access to the
>> ciphertext.
>
> Or so you claim.

No the attacker does not have access to the ciphertext.  What would lead 
you to think they did?

This statement is central to the problem:

"I'd like to protect the recipient against exposure to nefarious data by 
mangling or encrypting the data before it is written to disk"

This makes it clear I'm not trying to encrypt data to protect the data. 
  I'm trying to protect the recipient (storage server) from an attack. 
This specific attack being malware.  Yes, AES encryption would have 
worked here, but encryption is not the objective.

>
> I could go into detail about how the assumtion that the ciphertext is
> secret is not a smart one in the context of cryptography. And how side
> channels and other leakage may affect overall system security. But I'm
> going to save my time on that. I do get paid to review cryptographic
> systems and part of the job is dealing with belligerent people who have
> read Schneier's blog and think they can outsmart anyone else. Since I
> don't get paid to convice you, it's absolutely fine that you think your
> substitution scheme is the grand prize.


All of which has nothing to do with this thread.  Actual encryption is 
handled using AES and TLS.  This is not about encryption.

>
>>> Nobody assumes you're a moron. But it's safe to assume that you're a
>>> crypto layman, because only laymen have no clue on how difficult it is
>>> to get cryptography even remotely right.
>>
>> Amateur crypto is indeed a bad idea. But what you're still not getting
>> is that what he's doing here *isn't crypto*.
>
> So the topic says "Encrypting". If you look really closely at the word,
> the part "crypt" might give away to you that cryptography is involved.
>


This isn't about encrypting data to protect the data.  All the 
encryption I do uses standard AES and TLS.  Yes, I do understand that 
crypto is best left to experts.  The topic says "Encrypting" because I 
knew that encrypting the data would properly obfuscate it.


>> He's just trying to avoid
>> letting third parties write completely arbitrary data to the disk.
>
> There's your requirement. Then there's obviously some kind of
> implication when a third party *can* write arbitrary data to disk. And
> your other solution to that problem...

It's a network protocol.  Just like when writing a web app, you have to 
deal with bad actors.  That's what I'm doing here.  The entire service 
is about handling arbitrary data.  Just like Amazon S3 handles people's 
"arbitrary data".  Not sure what you mean by "third party".  It would be 
a registered peer.  But registration doesn't prevent the scenario in 
discussion.






From rosuav at gmail.com  Fri Jun 26 21:21:13 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 27 Jun 2015 11:21:13 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmkbij$dpm$1@ger.gmane.org>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
Message-ID: <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>

On Sat, Jun 27, 2015 at 6:09 AM, Randall Smith <randall at tnr.cc> wrote:
> Give me one plausible scenario where an attacker can cause malware to hit
> the disk after bytearray.translate with a 256 byte translation table and
> I'll be thankful to you.

The entire 256-byte translation table is significant ONLY if you need
all 256 possible bytes. Suppose I want to generate the following byte
sequence:

"\xCD\x19"

(Okay, this is a slightly oversimplified example, as this attack
doesn't work on a modern Windows. But back in the days of DOS, this
program would reboot your computer.)

How many truly different translation tables are there if I'm trying to
produce this? Just 256*255, or 65280. If I send random two-byte files,
there is one chance in that of my "malware" successfully landing. Once
I've sent about 45,000 of those files, I have a fifty-fifty chance of
having hit it. Send twice as many, I have a 75% chance of success,
etc.

Malware can be crafted to fit within certain restrictions. I saw a
proof-of-concept and analysis document detailing a particular remote
code execution/privilege escalation attack that involved stuffing
"text" into an entry field and then inducing the program to read that
into its stack, finally triggering it by some sort of buffer overflow,
I think. The text had to be no more than X bytes long (because that's
all the entry field was set to accept - it'd truncate after that), and
had to not contain any NUL bytes, and there might have been other
restrictions too. Sure, it makes it harder to write your malware...
but imagine if you can write something in just a handful of different
bytes, which then goes and triggers something else. You could have an
extremely plausible attack that might need only a day's uploading to
deliver.

It makes no difference that there are 256! possible encryption keys,
if most of them have the same result.

ChrisA


From steve at pearwood.info  Fri Jun 26 23:38:46 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 27 Jun 2015 13:38:46 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
Message-ID: <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>

On Sat, 27 Jun 2015 06:09 am, Randall Smith wrote:

> On 06/26/2015 12:06 PM, Steven D'Aprano wrote:
>> On Fri, 26 Jun 2015 11:01 am, Ian Kelly wrote:
>>
>>> You're making the same mistake that Steven did in misunderstanding the
>>> threat model.
>>
>> I don't think I'm misunderstanding the threat, I think I'm pointing out a
>> threat which the OP is hoping to just ignore.
> 
> I'm not hoping to ignore anything.  I didn't explain the entire system,
> as it was not necessary to find a solution to the problem at hand.  But
> since you want to make negative assumptions about what I didn't tell
> you, I'll gladly address your accusations of negligence.

"Negligence" is *your* word, not mine. I've never said that. And I'm not
*assuming* anything, everything I've stated has been based on the evidence
of what you have written. I've even gone so far as to EXPLICITLY say that I
cannot know for a fact that your application is vulnerable to these
threats, since I'm only going from a description rather than the app
itself. But your responses don't suggest that you have these threats under
control, on the contrary, they indicate that you are *far* underestimating
the seriousness of them and overestimating the difficulty of running a
secure application on a machine you cannot trust.

If your application has any saving grace, it is that there are easier ways
to get malware onto somebody else's computer. There are a hundred million
unsecured Windows boxen out there, if I were malicious I would just hire a
bot net rather than spend the time trying to hack your system. But maybe
somebody else will do it just for the lulz, or to prove it can be done.
Some black hats like a challenge, and yours appears to fall nicely into
that middle ground of hard enough to be interesting but not hard enough to
be really difficult.


>> In an earlier post, I suggested that the threat model should involve at
>> least *three* different attacks, apart from the usual man-in-the-model
>> attacks of data in transit.
> 
> All communication is secured using TLS and authentication handled by
> X.509 certificates.  This prevents man in the middle attacks.
> Certificates are signed by CAs I control.

You control the CAs? Presumably that means they're self-signed (unless you
mean you get to choose the CA). I don't know if that makes a difference or
not.


>> One is that the attacker is the person sending the data. E.g. I want to
>> send a nasty payload (say, malware, or an offensive image). Another is
>> that the attacker is the recipient of the file, who wants to read the
>> sender's data.
> 
> The only person who can read a file is the owner.   AES encryption is
> built into the client software.  The only way data can be uploaded
> unencrypted is if encryption is intentionally disabled.

With respect Randall, you contradict yourself. Is there any wonder that some
of us (well, me at least) is suspicious and confused, when your story
changes as often as the weather?

Sometimes you say that the client software uses AES encryption. Sometimes
you say that you don't want to use AES encryption because you want the
client to be pure Python, and a pure-Python implementation would be too
slow. Your very first post says:

    My original idea was for the recipient to encrypt using AES.  But
    I want to keep this software pure Python "batteries included" and
    not require installation of other platform-dependent software.  
    Pure Python AES and even DES are just way too slow.


Sometimes you say the user is supposed to encrypt the data themselves:

    While the data senders are supposed to encrypt data, that's not
    guaranteed


Now you say that the application encrypts the data, except that the user can
turn that option off.

Just make the AES encryption mandatory, not optional. Then the user cannot
upload unencrypted malicious data, and the receiver cannot read the data.
That's two problems solved.

Making AES or similarly strong encryption mandatory protects both the sender
of data and the receiver of data. I cannot imagine why you are considering
making it optional, since that only adds more work for you and reduces the
security of your users.

Oh, and DES is not good enough.


>> As far as I can tell, the OP's plan to defend the sender's privacy is to
>> dump responsibility for encrypting the files in the sender's lap. As far
>> as I'm concerned, perhaps as many as one user in 20000 will pre-encrypt
>> their files. (Early adopters will be unrepresentative of the eventual
>> user base of this system. If this takes off, the user base will likely
>> end up dominated by people who think that "qwerty" is the epitome of
>> unguessable passwords.)
> 
> Making assumptions again.  See above.  The client software encrypts by
> default.  You're also assuming there is no password strength checking.

My comment about "qwerty" as a password was a comment on the majority of
people on the internet, not an assumption about your application.


>> Users just don't use crypto unless their applications do it for them.
> 
> And it does.

Great to hear it! Just make sure your application always encrypts the
uploaded files, and you protect both the sender of the files and the
receiver.

At least from these threats. There are others.

Just to indicate how hard this is, here is a ten year old timing attack
against AES:

http://cr.yp.to/antiforgery/cachetiming-20050414.pdf

Is your application vulnerable to timing attacks? No idea. Talk to a
security expert.



>> My opinion is that the application ought to do so, and not expect Aunt
>> Tillie to learn how to correctly use encryption software before uploading
>> her files.
>>
>> http://www.catb.org/jargon/html/A/Aunt-Tillie.html
>>
>> It is the OP's prerogative to disagree, of course, but to me, if the OP's
>> app doesn't use strong crypto to encrypt users' data, that's tantamount
>> to saying they don't care about their users' data privacy. Using a
>> monoalphabetic substitution cipher to obfuscate the data is not strong
>> crypto.
> 
> You've gone on a rampage about nothing.  My original description said
> the client was supposed to encrypt the data, but you want to assume the
> opposite for some unknown reason.

Hardly a "rampage", and not "some unknown reason". End-users don't do
crypto. About one person in a hundred thousand digitally signs their
emails. Even people who know better don't use crypto. I don't use crypto.
If you are relying on people to encrypt their files, as you have suggested
in the past, *it won't happen*.

If the app does encrypt the data with AES before sending, then you don't
gain any benefit by obfuscating an encrypted file with a classical
monoalphabetic substitution cipher.


>>> The goal isn't to prevent the attacker from working out
>>> the key for a file that has already been obfuscated. Any real data
>>> that might be exposed by a vulnerability in the server is presumed to
>>> have already been strongly encrypted by the user.
>>
>> I think that's a ridiculously unrealistic presumption, unless your
>> user-base is entirely taken from a very small subset of security savvy
>> and pedantically careful users.
> 
> The difference is he's not assuming I'm a moron.  He's giving me the
> benefit of the doubt.  That plus I actually said, "data senders are
> supposed to encrypt data".

I'm not assuming you are a moron, I'm making a judgement based on your posts
that you might be out of your depth when it comes to crypto. Maybe I'm
wrong, and you know what you're doing, you just don't know how to
communicate that fact. This is an example: the sender is supposed to
encrypt the data, or the application encrypts it for them? Sometimes you
say the sender encrypts the data, sometimes you say the application
encrypts the data. It makes a big difference whether or not Aunt Tillie has
to run a separate encryption application before uploading her files.

If the application does it (a very good thing!) then there is the mystery of
why you would allow people to turn that option off. That can only make more
work for you and reduce the security of your application for both senders
and receivers.


> In a networked system, you can't make assumptions about what the other
> peers are doing.  You have to handle what comes across the wire.  You
> also have to consider that you may come under attack.  That's what this
> is about.

Right.

And you're in a difficult situation because your users (sender and receiver)
can't trust each other. This is harder than (say) Bittorrent.

With BT, the receiver can't trust the sender, but there's no threat to the
sender. (Well, legal threats, but that's a social issue, not a technical
one.) If I (let's assume legally) upload a file to a BT network, I don't
worry about downloaders reading the file. I want them to read it. That's
not the case here.


>>> The goal is to prevent the attacker from guessing a key that hasn't
>>> even been generated yet, which could be exploited to engineer the
>>> obfuscated content into something malicious.
>>
>> They don't need to predict the key exactly. If they can predict that the
>> key will be, lets say, one of these thousand values, then they can
>> generate one thousand files and upload them. One of them will match the
>> key, and there's your exploit. That's one attack.
> 
> Thousand Values ???  Isn't it 256!, which is just freaking huge!  import
> math; math.factorial(256)

No. There are 256! possible keys *in total*, but that doesn't necessarily
mean that the keys are unpredictable.

Suppose that you hire an intern to write the "choose key" function, and not
knowing any better, he simply iterates through the keys in numeric order,
one after the other. So the first upload will use key 0, the second key 1,
the third key 2, and so on, until key 256! - 1, then start again. In that
case, predicting the next key is *trivial*. If I can work out what key you
send now (I just upload a file containing "\x00\x01\x02...\xFF" to myself
and see what I get), then I know what key the app will use next.

Now obviously you're not going to let the new intern write such a critical
part of the application, not without a senior developer doing a code
review. He reviews the code, and mixes the keys up in some fashion, the
more unpredictable the better. So he uses a random number generator to
choose the key. 

Maybe you use Python's standard library and the Mersenne Twister. The period
of that is huge, possibly bigger than 256! (or not, I forget, and I'm too
lazy to look it up). So you think that's safe. But it's not: Mersenne
Twister is not a cryptographically secure pseudorandom number generator. If
I can get some small number of values from the Twister (by memory,
something of the order of 100 such values) then I can predict the rest for
ever.

Even if I can't do that, I might be able to guess the seed: I know what time
the application started up, to within a few milliseconds, and I know (or
can guess) how many random numbers you have used, so I can predict that it
will be so far into the period, give or take a few hundred or a thousand
values. Instead of having to guess the key out of 256! possible values, I
now only have to guess the key out of 1000 possible values. I can keep
sending files to myself until I work out which key it is, and from that
point on I can now predict every key with 100% certainty.

Okay, so you're smarter than that. You know that Python's Mersenne Twister
is not a CSPRNG, and you use /dev/urandom or its Windows equivalent.
Excellent.

Except... you're getting your random numbers from a system *I* control. I
can just feed you whatever "random" numbers I want.

I don't know how to solve that one. Writing secure code where you cannot
trust the machine you are running on is *immeasurably tougher* than writing
secure code on a trusted machine.


>> A second attack is to force the key. The attacker controls the machine
>> the application is running on, they control /dev/urandom and can feed
>> your app whatever not-so-random numbers they like, so potentially they
>> can force the app to use the key of their choosing. Then they don't need
>> 1000 files, they just need one.
>>
> 
> If the attacker controlled the machine the app was on, why would it fool
> with /dev/urandom?  I think he'd just plant the files he wanted to plant
> and be done.  This is non-nonsensical anyway.

No, you don't understand the nature of the attack. In this scenario, the
sender is the attacker. I want to upload malicious files to the receiver.
You are trying to stop me, that's the whole point of "mangling or
encrypting" the files. (Your words.) So I, the sender, prepare a file such
that when you mangle it, the resulting mangled content is the malicious
content I want.

If you use a substitution cipher, I can do this if I can guess or force the
key. If you use strong crypto, I can't.

However, I can hack the application. The client sits on my computer, it's
pure Python, even if it isn't I can still hack the application, I don't
need access to the source code.


>> That's two. Does anyone think that I've thought of all the possible
>> attacks?
>>
>> (Well, hypothetical attacks. I acknowledge that I don't know the
>> application, and cannot be sure that it *actually is* vulnerable to these
>> attacks.)
>>
>> The problem here is that a monoalphabetic substitution cipher is not
>> resistant to preimage attacks. Your only defence is that the key is
>> unknown. If the attacker can force the key, or predict the key, or guess
>> a small range of keys, they can exploit your weak cipher.
>>
>> (Technically, "preimage attack" is usually used to refer to attacks on
>> hash functions. I'm not sure if the same name is used for attacks on
>> ciphers.)
>>
>> https://en.wikipedia.org/wiki/Preimage_attack
>>
>> With a strong crypto cipher, there are no known preimage attacks. Even if
>> the attacker knows exactly what key you are using, they cannot predict
>> what preimage they need to supply in order to generate the malicious
>> payload they want after encryption. (As far as I know.)
>>
>> That is the critical issue right there. The sort of simple monoalphabetic
>> substitution cipher using bytes.translate that the OP is using is
>> vulnerable to preimage attacks. Strong crypto is not.
> 
> It isn't vulnerable to preimage attacks unless you can guess the key out
> of 256! possibilities.

Yes. Do you think that's hard for an attacker who has access to your
application, possibly including the source code, and controls all the
sources of entropy on the system your application is running on?

I don't have to *randomly* guess. I control what time your application
starts, I control what randomness you get from /dev/urandom, I control how
many keys you go through, I might even be able to read the source code of
the application (not that I need to, that just makes it easier).


> The key doesn't even exist until after the data 
> is sent.  Give me one plausible scenario where an attacker can cause
> malware to hit the disk after bytearray.translate with a 256 byte
> translation table and I'll be thankful to you.  As it stands now, you're
> either ignoring information I've already given, assuming I've made
> moronic design decisions the pouncing on them, or completely
> misunderstanding the issue at hand.

If you still don't understand the security threats after I've explained them
repeatedly, then your app is doomed to join the multitude of insecure and
unsafe applications on the internet.

Actually, the more I think about this, the more I come to think that the
only way this can be secure is for both the sending client application and
the receiving client appl to *both* encrypt the data. The sender can't
trust the receiver not to read the files, so the sender has to encrypt; the
receiver can't trust the sender not to send malicious files, so the
receiver has to encrypt too.


>>> There are no
>>> frequency-based attacks possible here, because you can't do frequency
>>> analysis on the result of a key that hasn't even been generated yet.
>>
>> Frequency-based attacks apply to a different threat. I'm referring to at
>> least two different attacks here, with different attackers and different
>> victims. Don't mix them up.
>>
>>
>>> Assuming that you have no attack on the key generation itself, the
>>
>> Not a safe assumption!
> 
> For this case it is a safe assumption.  For the same reason you're
> assuming the PSU isn't defective.

A detective power supply isn't a security threat. It doesn't enable me to
upload malicious files to an unsuspecting receiver.


> An attack on /dev/urandom for 
> instance would also compromise TLS, and every sort of secure key
> generation, not just the key for byte translation.  That is a separate
> problem altogether with a separate solution.

Er, yes? And how does that make it invalid?



>>> best you can do is send a file deobfuscated with a random key and hope
>>> that the recipient randomly chooses the same key; the odds of that
>>> happening are 1 in 256!.
>>
>> It's easy to come up with attacks which are no better than brute force.
>> It's the attacks which are better than brute force that you have to watch
>> out for.
>>
> 
> And that's why we're having this discussion.  Do you know of an attack
> in which you can control the output (say at least 100 consecutive bytes)
> for data which goes through a 256 byte translation table, chosen
> randomly from 256! permutations after the data is sent.  If you do, I'm
> all ears!  But at this point you're just setting up straw men and
> knocking them down.

Ah yes, the good ol' "any argument I do not have an answer for must be a
straw man" gambit.

Whatever you say dude. Obviously you've got this security thing down to a
fine art. After all, if you can't think of a way to break the system,
nobody else can either.



-- 
Steven



From jeanpierreda at gmail.com  Sat Jun 27 00:05:12 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Fri, 26 Jun 2015 21:05:12 -0700
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CABicbJL=mxjC-+_trxEgavJo0z5D9BphmayNYCPzTQmYANUDpQ@mail.gmail.com>

On Fri, Jun 26, 2015 at 8:38 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> Now you say that the application encrypts the data, except that the user can
> turn that option off.
>
> Just make the AES encryption mandatory, not optional. Then the user cannot
> upload unencrypted malicious data, and the receiver cannot read the data.
> That's two problems solved.

No, because another application could pretend to be the file-sending
application, but send unencrypted data instead of encrypted data.

-- Devin


From rustompmody at gmail.com  Sat Jun 27 00:11:42 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Fri, 26 Jun 2015 21:11:42 -0700 (PDT)
Subject: Turning string into object (name)
In-Reply-To: <mailman.99.1435291217.3674.python-list@python.org>
References: <3ba959bc-f25f-4655-8c1f-ce2888210e1b@googlegroups.com>
 <mailman.99.1435291217.3674.python-list@python.org>
Message-ID: <37e631f2-16ea-431a-aa18-fb74cdedc278@googlegroups.com>

On Friday, June 26, 2015 at 9:30:38 AM UTC+5:30, Chris Angelico wrote:
> Incidentally, I would suggest not having the try/except at all, since
> all it does is print an error and terminate (which is the same result
> you'd get if that error bubbled all the way to top level). But if you
> are going to use it, then I strongly recommend using the newer syntax:
> 
> except errors.ConnectionFailure as e:

I wonder Chris if you know anything about this [yeah a bit of a hijack]:

There is this git plugin https://github.com/felipec/git-remote-hg
that allows one to bi-direct talk to hg

I tried to add a little better error checking so that it doesnt barf with a backtrace when connecting to git (instead of hg which is the intent):

My modifs: https://github.com/rusimody/git-remote-hg/commit/51cc665eff174e0d6bbf2e1b61ca332af345e76a

I got some response (from github) to the effect that travis-continuous failed 
or some such.
Yeah I know about nothing about github pull requests but it just could be
this -- obsolete except syntax.

Any thoughts? Suggestions?


From rosuav at gmail.com  Sat Jun 27 00:30:54 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 27 Jun 2015 14:30:54 +1000
Subject: Turning string into object (name)
In-Reply-To: <37e631f2-16ea-431a-aa18-fb74cdedc278@googlegroups.com>
References: <3ba959bc-f25f-4655-8c1f-ce2888210e1b@googlegroups.com>
 <mailman.99.1435291217.3674.python-list@python.org>
 <37e631f2-16ea-431a-aa18-fb74cdedc278@googlegroups.com>
Message-ID: <CAPTjJmrhtXuVE_-Dra6nfKg-6PWWu4WQP3J-Fi__YAUhkZGPvg@mail.gmail.com>

On Sat, Jun 27, 2015 at 2:11 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> On Friday, June 26, 2015 at 9:30:38 AM UTC+5:30, Chris Angelico wrote:
>> Incidentally, I would suggest not having the try/except at all, since
>> all it does is print an error and terminate (which is the same result
>> you'd get if that error bubbled all the way to top level). But if you
>> are going to use it, then I strongly recommend using the newer syntax:
>>
>> except errors.ConnectionFailure as e:
>
> I wonder Chris if you know anything about this [yeah a bit of a hijack]:

Well.... yes, that's a complete thread hijack. Whatever.

> There is this git plugin https://github.com/felipec/git-remote-hg
> that allows one to bi-direct talk to hg
>
> I tried to add a little better error checking so that it doesnt barf with a backtrace when connecting to git (instead of hg which is the intent):
>
> My modifs: https://github.com/rusimody/git-remote-hg/commit/51cc665eff174e0d6bbf2e1b61ca332af345e76a
>
> I got some response (from github) to the effect that travis-continuous failed
> or some such.
> Yeah I know about nothing about github pull requests but it just could be
> this -- obsolete except syntax.
>
> Any thoughts? Suggestions?

Standard principle when asking for help with a Python program: Quote
the error text! You say you got a response back - quote it!

Standard debugging technique: If you have an idea, try it! Most code
is deterministic enough that you can simply change to the modern
except syntax and try again. Maybe it won't have any effect, but maybe
it'll tell you exactly what the problem is.

Beyond that, I don't think I can much help you.

ChrisA


From dfnsonfsduifb at gmx.de  Sat Jun 27 01:24:28 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sat, 27 Jun 2015 07:24:28 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mailman.117.1435366542.3674.python-list@python.org>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkev4$uhl$1@news.albasani.net>
 <slrnmorh57.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <mmkkhc$m5d$1@news.albasani.net>
 <mailman.117.1435366542.3674.python-list@python.org>
Message-ID: <mmlc2c$lkj$1@news.albasani.net>

On 27.06.2015 02:55, Randall Smith wrote:

> No the attacker does not have access to the ciphertext.  What would lead
> you to think they did?

Years of practical experience in the field of applied cryptography.
Knowledge of how side channels work and how easily they can be
constructed for bad schemes.

Rest snipped, explanation futile.
Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From ian.g.kelly at gmail.com  Sat Jun 27 01:47:22 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 26 Jun 2015 23:47:22 -0600
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CALwzidmNA3kRK9nj2jgKOcpjDA5Ge4ok2hZyeXi=ip6xuwBcNw@mail.gmail.com>

On Fri, Jun 26, 2015 at 9:38 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> With respect Randall, you contradict yourself. Is there any wonder that some
> of us (well, me at least) is suspicious and confused, when your story
> changes as often as the weather?
>
> Sometimes you say that the client software uses AES encryption. Sometimes
> you say that you don't want to use AES encryption because you want the
> client to be pure Python, and a pure-Python implementation would be too
> slow. Your very first post says:
>
>     My original idea was for the recipient to encrypt using AES.  But
>     I want to keep this software pure Python "batteries included" and
>     not require installation of other platform-dependent software.
>     Pure Python AES and even DES are just way too slow.

In the context of the initial post, this was referring to the data
mangling done by the receiver; it has no bearing on the form of the
data sent by the application.

> Sometimes you say the user is supposed to encrypt the data themselves:
>
>     While the data senders are supposed to encrypt data, that's not
>     guaranteed

Whereas this clearly describes the behavior of the application itself.

> Now you say that the application encrypts the data, except that the user can
> turn that option off.
>
> Just make the AES encryption mandatory, not optional. Then the user cannot
> upload unencrypted malicious data, and the receiver cannot read the data.
> That's two problems solved.

And what if somebody else writes a competing version of the client
software that doesn't bother with the encryption step at all? The
point was that while encryption is expected, it cannot be assumed by
the receiver, and in fact if the data is actually malicious, then it
likely is not even being sent by the client software in the first
place.

> If the app does encrypt the data with AES before sending, then you don't
> gain any benefit by obfuscating an encrypted file with a classical
> monoalphabetic substitution cipher.

Only if the recipient can *trust* the sender to have performed the
encryption, which it can't, no matter how mandatory the OP tries to
make it.

> Suppose that you hire an intern to write the "choose key" function, and not
> knowing any better, he simply iterates through the keys in numeric order,
> one after the other. So the first upload will use key 0, the second key 1,
> the third key 2, and so on, until key 256! - 1, then start again. In that
> case, predicting the next key is *trivial*. If I can work out what key you
> send now (I just upload a file containing "\x00\x01\x02...\xFF" to myself
> and see what I get), then I know what key the app will use next.

If you upload a file to yourself, the result that you get will have no
bearing on what key might be chosen when you upload a file to somebody
else.

> Even if I can't do that, I might be able to guess the seed: I know what time
> the application started up, to within a few milliseconds,

How?

> and I know (or
> can guess) how many random numbers you have used,

How?

> Except... you're getting your random numbers from a system *I* control.

No you don't. If you did already control the target system, then as
already suggested, you have no need to attack the data upload; you can
just write whatever data you want to disk. This is like suggesting
that the sudoers file is insecure because a user with root access
would be able to add themselves to it.

>> If the attacker controlled the machine the app was on, why would it fool
>> with /dev/urandom?  I think he'd just plant the files he wanted to plant
>> and be done.  This is non-nonsensical anyway.
>
> No, you don't understand the nature of the attack. In this scenario, the
> sender is the attacker. I want to upload malicious files to the receiver.
> You are trying to stop me, that's the whole point of "mangling or
> encrypting" the files. (Your words.) So I, the sender, prepare a file such
> that when you mangle it, the resulting mangled content is the malicious
> content I want.
>
> If you use a substitution cipher, I can do this if I can guess or force the
> key. If you use strong crypto, I can't.
>
> However, I can hack the application. The client sits on my computer, it's
> pure Python, even if it isn't I can still hack the application, I don't
> need access to the source code.

If the recipient system is using the system random to generate the
key, then you can hack the application all you want, and it will give
you precisely zero information about the state of the entropy pool on
the remote system.

> Yes. Do you think that's hard for an attacker who has access to your
> application, possibly including the source code, and controls all the
> sources of entropy on the system your application is running on?
>
> I don't have to *randomly* guess. I control what time your application
> starts, I control what randomness you get from /dev/urandom, I control how
> many keys you go through, I might even be able to read the source code of
> the application (not that I need to, that just makes it easier).

I think what's going on here is that you've missed the point that the
obfuscation is done by the recipient, not by the sender. The sender
has no control over any of those things, and no access to either the
key or the obfuscated data unless they can gain it through some other
attack vector (which is certainly a factor to consider before
implementing this).


From ian.g.kelly at gmail.com  Sat Jun 27 01:59:45 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 26 Jun 2015 23:59:45 -0600
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
Message-ID: <CALwzidkr_WgZktoUd0YyL4tk4oudja+iLqG0MMeB+Zuo_nNkhA@mail.gmail.com>

On Fri, Jun 26, 2015 at 7:21 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Sat, Jun 27, 2015 at 6:09 AM, Randall Smith <randall at tnr.cc> wrote:
>> Give me one plausible scenario where an attacker can cause malware to hit
>> the disk after bytearray.translate with a 256 byte translation table and
>> I'll be thankful to you.
>
> The entire 256-byte translation table is significant ONLY if you need
> all 256 possible bytes. Suppose I want to generate the following byte
> sequence:
>
> "\xCD\x19"
>
> (Okay, this is a slightly oversimplified example, as this attack
> doesn't work on a modern Windows. But back in the days of DOS, this
> program would reboot your computer.)

Nice! When I suggested the possibility of a two byte value malicious
payload, I thought it an extreme example of the hypothetical attack. I
didn't expect that somebody might actually produce one.


From steve at pearwood.info  Sat Jun 27 02:16:38 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 27 Jun 2015 16:16:38 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.119.1435377960.3674.python-list@python.org>
Message-ID: <558e3fc4$0$1658$c3e8da3$5496439d@news.astraweb.com>

On Sat, 27 Jun 2015 02:05 pm, Devin Jeanpierre wrote:

> On Fri, Jun 26, 2015 at 8:38 PM, Steven D'Aprano <steve at pearwood.info>
> wrote:
>> Now you say that the application encrypts the data, except that the user
>> can turn that option off.
>>
>> Just make the AES encryption mandatory, not optional. Then the user
>> cannot upload unencrypted malicious data, and the receiver cannot read
>> the data. That's two problems solved.
> 
> No, because another application could pretend to be the file-sending
> application, but send unencrypted data instead of encrypted data.

Did you stop reading my post when you got to that? Because I went on to say:

"Actually, the more I think about this, the more I come to think that the
only way this can be secure is for both the sending client application and
the receiving client appl to both encrypt the data. The sender can't
trust the receiver not to read the files, so the sender has to encrypt; the
receiver can't trust the sender not to send malicious files, so the
receiver has to encrypt too."




-- 
Steven



From rosuav at gmail.com  Sat Jun 27 02:52:38 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 27 Jun 2015 16:52:38 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CALwzidkr_WgZktoUd0YyL4tk4oudja+iLqG0MMeB+Zuo_nNkhA@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <CALwzidkr_WgZktoUd0YyL4tk4oudja+iLqG0MMeB+Zuo_nNkhA@mail.gmail.com>
Message-ID: <CAPTjJmqT4yfsc9z_sFeJaFA447es1JzXgbUYHgZZSYWM+2VAog@mail.gmail.com>

On Sat, Jun 27, 2015 at 3:59 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Fri, Jun 26, 2015 at 7:21 PM, Chris Angelico <rosuav at gmail.com> wrote:
>> On Sat, Jun 27, 2015 at 6:09 AM, Randall Smith <randall at tnr.cc> wrote:
>>> Give me one plausible scenario where an attacker can cause malware to hit
>>> the disk after bytearray.translate with a 256 byte translation table and
>>> I'll be thankful to you.
>>
>> The entire 256-byte translation table is significant ONLY if you need
>> all 256 possible bytes. Suppose I want to generate the following byte
>> sequence:
>>
>> "\xCD\x19"
>>
>> (Okay, this is a slightly oversimplified example, as this attack
>> doesn't work on a modern Windows. But back in the days of DOS, this
>> program would reboot your computer.)
>
> Nice! When I suggested the possibility of a two byte value malicious
> payload, I thought it an extreme example of the hypothetical attack. I
> didn't expect that somebody might actually produce one.

I'm fairly sure this won't actually work on a modern system (I tried
it and all that happened was that debug.exe terminated), but it's
entirely possible there are other attacks. Or attacks that require
only a small number of bytes - maybe create a gzip bomb that will
expand to petabytes of data, that probably wouldn't need many unique
byte values.

ChrisA


From robert.kern at gmail.com  Sat Jun 27 03:58:46 2015
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 27 Jun 2015 08:58:46 +0100
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mmll3n$ofn$1@ger.gmane.org>

On 2015-06-27 04:38, Steven D'Aprano wrote:

> Maybe you use Python's standard library and the Mersenne Twister. The period
> of that is huge, possibly bigger than 256! (or not, I forget, and I'm too
> lazy to look it up). So you think that's safe. But it's not: Mersenne
> Twister is not a cryptographically secure pseudorandom number generator. If
> I can get some small number of values from the Twister (by memory,
> something of the order of 100 such values) then I can predict the rest for
> ever.

634.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From robert.kern at gmail.com  Sat Jun 27 04:07:21 2015
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 27 Jun 2015 09:07:21 +0100
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmll3n$ofn$1@ger.gmane.org>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mmll3n$ofn$1@ger.gmane.org>
Message-ID: <mmlljq$3qn$1@ger.gmane.org>

On 2015-06-27 08:58, Robert Kern wrote:
> On 2015-06-27 04:38, Steven D'Aprano wrote:
>
>> Maybe you use Python's standard library and the Mersenne Twister. The period
>> of that is huge, possibly bigger than 256! (or not, I forget, and I'm too
>> lazy to look it up). So you think that's safe. But it's not: Mersenne
>> Twister is not a cryptographically secure pseudorandom number generator. If
>> I can get some small number of values from the Twister (by memory,
>> something of the order of 100 such values) then I can predict the rest for
>> ever.
>
> 634.

Bah! 624.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From __peter__ at web.de  Sat Jun 27 04:29:06 2015
From: __peter__ at web.de (Peter Otten)
Date: Sat, 27 Jun 2015 10:29:06 +0200
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
Message-ID: <mmlmsk$m2e$1@ger.gmane.org>

Randall Smith wrote:

> Chunks of data (about 2MB) are to be stored on machines using a
> peer-to-peer protocol.  The recipient of these chunks can't assume that
> the payload is benign.  While the data senders are supposed to encrypt
> data, that's not guaranteed, and I'd like to protect the recipient
> against exposure to nefarious data by mangling or encrypting the data
> before it is written to disk.
> 
> My original idea was for the recipient to encrypt using AES.  But I want
> to keep this software pure Python "batteries included" and not require
> installation of other platform-dependent software.  Pure Python AES and
> even DES are just way too slow.  I don't know that I really need
> encryption here, but some type of fast mangling algorithm where a bad
> actor sending a payload can't guess the output ahead of time.
> 
> Any ideas are appreciated.  Thanks.

Would it be sufficient to prepend the chunk with one block, say, of random 
data? To unmangle you'd just strip off that block.

BLOCK = os.urandom(BLOCKSIZE)

def mangle(source, dest):
    dest.write(BLOCK)
    shutil.copyfileobj(source, dest)

def unmangle(source, dest):
    source.read(BLOCKSIZE)
    shutil.copyfileobj(source, dest)

Disclaimer: I did not follow the ongoing discussion.



From steve at pearwood.info  Sat Jun 27 04:38:39 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 27 Jun 2015 18:38:39 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
Message-ID: <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>

On Sat, 27 Jun 2015 03:47 pm, Ian Kelly wrote:

[...]
>> Just make the AES encryption mandatory, not optional. Then the user
>> cannot upload unencrypted malicious data, and the receiver cannot read
>> the data. That's two problems solved.
> 
> And what if somebody else writes a competing version of the client
> software that doesn't bother with the encryption step at all? The
> point was that while encryption is expected, it cannot be assumed by
> the receiver, and in fact if the data is actually malicious, then it
> likely is not even being sent by the client software in the first
> place.

Right. As I said later in my post, you have a situation where neither party
can trust the other. I'm trying to store data on your computer, and I can't
trust you not to snoop on it, and you can't trust me not to send you
malware.


>> If the app does encrypt the data with AES before sending, then you don't
>> gain any benefit by obfuscating an encrypted file with a classical
>> monoalphabetic substitution cipher.
> 
> Only if the recipient can *trust* the sender to have performed the
> encryption, which it can't, no matter how mandatory the OP tries to
> make it.

True. But in either case, a classical (i.e. insecure) cipher doesn't do the
job.


>> Suppose that you hire an intern to write the "choose key" function, and
>> not knowing any better, he simply iterates through the keys in numeric
>> order, one after the other. So the first upload will use key 0, the
>> second key 1, the third key 2, and so on, until key 256! - 1, then start
>> again. In that case, predicting the next key is *trivial*. If I can work
>> out what key you send now (I just upload a file containing
>> "\x00\x01\x02...\xFF" to myself and see what I get), then I know what key
>> the app will use next.
> 
> If you upload a file to yourself, the result that you get will have no
> bearing on what key might be chosen when you upload a file to somebody
> else.

I admit it: I was getting a bit confused between attacks on the sender side
and attacks on the receiver side. The attacks I describe depend on the
sender's application doing encryption, but given that a malicious uploader
can just write their own client, that's redundant. There are easier attacks
on sender-side encryption. A back-and-forth argument on Usenet is no
substitute between a careful security analysis.

Can the sender attack encryption on the client side? Well, Chris has already
demonstrated one actual attack, based on a two-byte malicious payload. That
proves that the concept is at least possible, even if nobody uses DOS any
more.

As you go on to say:

> If the recipient system is using the system random to generate the
> key, then you can hack the application all you want, and it will give
> you precisely zero information about the state of the entropy pool on
> the remote system.

You're right. Are there other attacks where I, the sender, can get the
recipient to leak information about the key from the receiver?

Would you like to bet the answer is always No? I wouldn't.

Can you say "timing attack"?

http://codahale.com/a-lesson-in-timing-attacks/

Can you [generic you] believe that attackers can *reliably* attack remote
systems based on a 20?s timing differences? If you say "No", then you fail
Security 101 and should step away from the computer until a security expert
can be called in to review your code.

I'm not a security expert. I'm not even a talented amateur. *Every time* I
suggest that "X is secure", the security guy at work shoots me down in
flames. But nicely, because I pay his wages <wink>

If I say "such-and-such an attack is impossible", feel free to scoff and
laugh, because I'm probably wrong. But if I say "I don't know what it is,
but there's probably an attack you haven't thought of", unless you're a
security guy yourself, you probably ought to listen. (And if you are a
security guy, then you know how hard it is to secure against unknown
attacks.)

Tens of millions of zombie computers in botnets are proof that there are
exploitable attacks that programmers didn't think of. Or rather, *some*
programmers didn't think of them. Some other guys did.

I've said it before, and I will say it again: a classical substitution
cipher is trivially vulnerable to a preimage attack, strong crypto ciphers
are not. You're betting everything on the key being secret. If the keys
leaks, or is predictable, the attacker can successfully write malware on
the receiver's system. If the keys leak with AES, the system is still
secure against a preimage attack.


"Nobody will be able to guess the key, we don't need strong crypto."

"The Titanic is unsinkable, we don't need lifeboats for everyone."




-- 
Steven



From rosuav at gmail.com  Sat Jun 27 04:53:29 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 27 Jun 2015 18:53:29 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmrwcsmWmbaHPDoiEKzOnLtzf0cHekaEKji++QGsaYkiNQ@mail.gmail.com>

On Sat, Jun 27, 2015 at 6:38 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> I'm not a security expert. I'm not even a talented amateur. *Every time* I
> suggest that "X is secure", the security guy at work shoots me down in
> flames. But nicely, because I pay his wages <wink>

Just out of interest, is _anybody_ active in this thread an expert on
security? I certainly am not, which means that the proposal I'm
currently putting together probably has a whole bunch of
vulnerabilities that I haven't thought of. (Though there's no emphasis
on encryption anywhere, just signing. I'm *hoping* that RSA public key
verification is sufficient, but if it isn't, it would be possible for
a malicious user to make a serious mess of stuff.) But I'm under no
delusions. I don't say "this is secure" - all I'm saying is "this
works in proof-of-concept".

ChrisA


From dfnsonfsduifb at gmx.de  Sat Jun 27 05:07:14 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sat, 27 Jun 2015 11:07:14 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mailman.130.1435395217.3674.python-list@python.org>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
Message-ID: <mmlp42$p4v$1@news.albasani.net>

On 27.06.2015 10:53, Chris Angelico wrote:
> On Sat, Jun 27, 2015 at 6:38 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> I'm not a security expert. I'm not even a talented amateur. *Every time* I
>> suggest that "X is secure", the security guy at work shoots me down in
>> flames. But nicely, because I pay his wages <wink>
> 
> Just out of interest, is _anybody_ active in this thread an expert on
> security?

Yes. I've done a good 10 years of work in the field doing security
(mostly applied cryptography on embedded systems with a focus on side
channels like DPA, but also security concepts and threat/risk analysis)
and spent the last 3-4 years working on my PhD in the field of IT
security. My thesis is almost(tm) finished. I would claim to be an
expert, yes.

> I certainly am not, which means that the proposal I'm
> currently putting together probably has a whole bunch of
> vulnerabilities that I haven't thought of. (Though there's no emphasis
> on encryption anywhere, just signing. I'm *hoping* that RSA public key
> verification is sufficient, but if it isn't, it would be possible for
> a malicious user to make a serious mess of stuff.) But I'm under no
> delusions. I don't say "this is secure" - all I'm saying is "this
> works in proof-of-concept".

I must admit that I haven't seen your ideas in this thread?

Best regards,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From dfnsonfsduifb at gmx.de  Sat Jun 27 05:12:07 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sat, 27 Jun 2015 11:12:07 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mmlpd7$n64$1@news.albasani.net>

On 27.06.2015 10:38, Steven D'Aprano wrote:

> Can you say "timing attack"?
> 
> http://codahale.com/a-lesson-in-timing-attacks/
> 
> Can you [generic you] believe that attackers can *reliably* attack remote
> systems based on a 20?s timing differences? If you say "No", then you fail
> Security 101 and should step away from the computer until a security expert
> can be called in to review your code.

Yes, as people do more and more proper crypto (in contrast to crappy
stuff like LFSR-based custom keystream generators and such), side
channels become of great importance.

> I'm not a security expert. I'm not even a talented amateur. *Every time* I
> suggest that "X is secure", the security guy at work shoots me down in
> flames. But nicely, because I pay his wages <wink>

:-)

Being shot down in flames is the way to become a security expert,
probably the *only* way. I don't know anyone who is an expert who hasn't
had that horrible experience at least a dozen of times.

It is amazing how many holes you can poke in designs if you look at it
from enough angles. Having holes poked in my designs gives you a
thourough appreciation for the true crypto experts (i.e. people doing
theoretical cryptography).

Best regards,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From rosuav at gmail.com  Sat Jun 27 05:17:16 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 27 Jun 2015 19:17:16 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmlp42$p4v$1@news.albasani.net>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
Message-ID: <CAPTjJmrss=dJQzDiPE9MUdyD--KuiGMRsxmTvkbNqW1e38v+cQ@mail.gmail.com>

On Sat, Jun 27, 2015 at 7:07 PM, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
> On 27.06.2015 10:53, Chris Angelico wrote:
>> On Sat, Jun 27, 2015 at 6:38 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>>> I'm not a security expert. I'm not even a talented amateur. *Every time* I
>>> suggest that "X is secure", the security guy at work shoots me down in
>>> flames. But nicely, because I pay his wages <wink>
>>
>> Just out of interest, is _anybody_ active in this thread an expert on
>> security?
>
> Yes. I've done a good 10 years of work in the field doing security
> (mostly applied cryptography on embedded systems with a focus on side
> channels like DPA, but also security concepts and threat/risk analysis)
> and spent the last 3-4 years working on my PhD in the field of IT
> security. My thesis is almost(tm) finished. I would claim to be an
> expert, yes.

Good, so this isn't like that episode of Yes Minister when they were
trying to figure out whether to allow a chemical factory to be built.

>> I certainly am not, which means that the proposal I'm
>> currently putting together probably has a whole bunch of
>> vulnerabilities that I haven't thought of. (Though there's no emphasis
>> on encryption anywhere, just signing. I'm *hoping* that RSA public key
>> verification is sufficient, but if it isn't, it would be possible for
>> a malicious user to make a serious mess of stuff.) But I'm under no
>> delusions. I don't say "this is secure" - all I'm saying is "this
>> works in proof-of-concept".
>
> I must admit that I haven't seen your ideas in this thread?

No, the proposal I'm putting together is unrelated. You'll see the
*vast* extent of my security skills here:

https://github.com/Rosuav/ThirdSquare

My contribution to this thread has been fairly minor, just suggesting
one attack that doesn't even work any more, not much else.

ChrisA


From jon+usenet at unequivocal.co.uk  Sat Jun 27 05:26:04 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sat, 27 Jun 2015 09:26:04 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.122.1435384827.3674.python-list@python.org>
Message-ID: <slrnmosr3v.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-27, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Fri, Jun 26, 2015 at 7:21 PM, Chris Angelico <rosuav at gmail.com> wrote:
>> On Sat, Jun 27, 2015 at 6:09 AM, Randall Smith <randall at tnr.cc> wrote:
>>> Give me one plausible scenario where an attacker can cause malware to hit
>>> the disk after bytearray.translate with a 256 byte translation table and
>>> I'll be thankful to you.
>>
>> The entire 256-byte translation table is significant ONLY if you need
>> all 256 possible bytes. Suppose I want to generate the following byte
>> sequence:
>>
>> "\xCD\x19"
>>
>> (Okay, this is a slightly oversimplified example, as this attack
>> doesn't work on a modern Windows. But back in the days of DOS, this
>> program would reboot your computer.)
>
> Nice! When I suggested the possibility of a two byte value malicious
> payload, I thought it an extreme example of the hypothetical attack. I
> didn't expect that somebody might actually produce one.

It's a good example of the interesting things that people can come up
with (for example, "binary" executable files that in fact are
comprised entirely of printable ASCII characters), but it isn't in
any sense an "attack" on the system described in this thread.


From jon+usenet at unequivocal.co.uk  Sat Jun 27 05:27:43 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sat, 27 Jun 2015 09:27:43 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org>
Message-ID: <slrnmosr71.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-27, Chris Angelico <rosuav at gmail.com> wrote:
> On Sat, Jun 27, 2015 at 7:07 PM, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
>> On 27.06.2015 10:53, Chris Angelico wrote:
>>> On Sat, Jun 27, 2015 at 6:38 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>>>> I'm not a security expert. I'm not even a talented amateur. *Every time* I
>>>> suggest that "X is secure", the security guy at work shoots me down in
>>>> flames. But nicely, because I pay his wages <wink>
>>>
>>> Just out of interest, is _anybody_ active in this thread an expert on
>>> security?
>>
>> Yes. I've done a good 10 years of work in the field doing security
>> (mostly applied cryptography on embedded systems with a focus on side
>> channels like DPA, but also security concepts and threat/risk analysis)
>> and spent the last 3-4 years working on my PhD in the field of IT
>> security. My thesis is almost(tm) finished. I would claim to be an
>> expert, yes.
>
> Good, so this isn't like that episode of Yes Minister when they were
> trying to figure out whether to allow a chemical factory to be built.

Johannes might have all the education in the world, but he's
demonstrated quite comprehensively in this thread that he doesn't
have a clue what he's talking about.


From dfnsonfsduifb at gmx.de  Sat Jun 27 06:05:28 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sat, 27 Jun 2015 12:05:28 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <slrnmosr71.1nu.jon+usenet@frosty.unequivocal.co.uk>
References: <mmcagn$aa5$1@ger.gmane.org>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org>
 <slrnmosr71.1nu.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <mmlsh8$kj6$1@news.albasani.net>

On 27.06.2015 11:27, Jon Ribbens wrote:

> Johannes might have all the education in the world, but he's
> demonstrated quite comprehensively in this thread that he doesn't
> have a clue what he's talking about.

Oh, how hurtful. I might even shed a tear or two, but it's pretty clear
to me that you're just suffering under the Dunning-Kruger effect. No
worries, champ, it's just a phase that'll go away eventually.

Hugs and kisses,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From rosuav at gmail.com  Sat Jun 27 06:16:47 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 27 Jun 2015 20:16:47 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmlsh8$kj6$1@news.albasani.net>
References: <mmcagn$aa5$1@ger.gmane.org>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org>
 <slrnmosr71.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <mmlsh8$kj6$1@news.albasani.net>
Message-ID: <CAPTjJmqE_GbFCXQERD7xzxs0iBsEOsCg-3i_sRpuLCPUTavQgg@mail.gmail.com>

On Sat, Jun 27, 2015 at 8:05 PM, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
> On 27.06.2015 11:27, Jon Ribbens wrote:
>
>> Johannes might have all the education in the world, but he's
>> demonstrated quite comprehensively in this thread that he doesn't
>> have a clue what he's talking about.
>
> Oh, how hurtful. I might even shed a tear or two, but it's pretty clear
> to me that you're just suffering under the Dunning-Kruger effect. No
> worries, champ, it's just a phase that'll go away eventually.

Okay, Johannes, NOW you're proving that you don't have a clue what
you're talking about. D-K effect doesn't go away...

ChrisA


From dfnsonfsduifb at gmx.de  Sat Jun 27 06:18:29 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sat, 27 Jun 2015 12:18:29 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mailman.131.1435396645.3674.python-list@python.org>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org>
Message-ID: <mmlt9l$m7s$1@news.albasani.net>

On 27.06.2015 11:17, Chris Angelico wrote:

> Good, so this isn't like that episode of Yes Minister when they were
> trying to figure out whether to allow a chemical factory to be built.

I must admit that I have no clue about that show or that epsisode in
particular and needed to read up on it:
https://en.wikipedia.org/wiki/The_Greasy_Pole

>> I must admit that I haven't seen your ideas in this thread?
> 
> No, the proposal I'm putting together is unrelated. You'll see the
> *vast* extent of my security skills here:
> 
> https://github.com/Rosuav/ThirdSquare
> 
> My contribution to this thread has been fairly minor, just suggesting
> one attack that doesn't even work any more, not much else.

Well, if people already have a solution ready there's a good chance that
any criticism falls on deaf ears. In any case something that others have
to be responsible for, their party, their choice.

I've looked at your code even though I don't know pike. That's the
typesafe JavaScript derivative, isn't it?

The only thing that I found horrible was the ssh key format to PKCS
parsing. Man that's hacky :-) You're creating a DER structure on-the-fly
that you fill with the key and that you then have parsed back. I've only
seen ssh-keygen used to generate keys (not to initiate actual ssh
connections), why don't you use openssl to generate the keys? I think
you can generate a RSA keypair in openssl (also valid for ssh should you
need it) and I'm pretty sure that you can generate a ssh public key with
ssh-keygen from that private keypair file. That would eliminate the need
to do this kind of parsing, but it's just a PoC as I understand it.

It appears to be online-only, is that correct? Is Internet coverage so
good down under? I wish this were the case in Germany :-/

Not 100% about it, but I think that the bus concepts that are active in
Germany (locally in some cities) either user asymmetric transponders
(i.e. SmartMX), which gives a beautiful, decentralized, secure and
offline solution at the cost of being comparatively expensive. The
others use symmetric transponders which have limited off-line
functionality: i.e. monotonic counters which are reset in a
cryptographically secured way by backend systems every time a
online-connection persists and which are counted down in the offline case.

In any case, interesting. Thanks for sharing.
Best regards,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From jon+usenet at unequivocal.co.uk  Sat Jun 27 06:26:43 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sat, 27 Jun 2015 10:26:43 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org>
 <slrnmosr71.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <mmlsh8$kj6$1@news.albasani.net>
Message-ID: <slrnmosull.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-27, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
> On 27.06.2015 11:27, Jon Ribbens wrote:
>> Johannes might have all the education in the world, but he's
>> demonstrated quite comprehensively in this thread that he doesn't
>> have a clue what he's talking about.
>
> Oh, how hurtful. I might even shed a tear or two, but it's pretty clear
> to me that you're just suffering under the Dunning-Kruger effect. No
> worries, champ, it's just a phase that'll go away eventually.

I guess we need to add the Dunning-Kruger effect to that ever-growing
list of things that you don't understand then...


From dfnsonfsduifb at gmx.de  Sat Jun 27 06:55:14 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sat, 27 Jun 2015 12:55:14 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mailman.132.1435400215.3674.python-list@python.org>
References: <mmcagn$aa5$1@ger.gmane.org>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org>
 <slrnmosr71.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <mmlsh8$kj6$1@news.albasani.net>
 <mailman.132.1435400215.3674.python-list@python.org>
Message-ID: <mmlvei$i27$1@news.albasani.net>

On 27.06.2015 12:16, Chris Angelico wrote:

> Okay, Johannes, NOW you're proving that you don't have a clue what
> you're talking about. D-K effect doesn't go away...

:-D

It does in some people. I've seen it happen, with knowledge comes
humility. Not saying Jon is a lost cause just yet. He's just in
intellectual puberty right now. I'm giving him a few years to re-judge.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From lac at openend.se  Sat Jun 27 07:25:27 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 27 Jun 2015 13:25:27 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: Message from Johannes Bauer <dfnsonfsduifb@gmx.de>
 of "Sat, 27 Jun 2015 12:18:29 +0200." <mmlt9l$m7s$1@news.albasani.net>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org><mmlt9l$m7!
 s$1@news.albasani.net>
Message-ID: <201506271125.t5RBPRkr026768@fido.openend.se>

Johannes, if you don't know "Yes, Minister" then you most likely do
not know the Politician's Syllogism (which now has its own wikipedia
page :)  And I _didn't_ do it!  Honest!)

Something must be done.
This is something.
Therefore we must do it!

:)

Unfortunatetely, the Politician's Syllogism is not restricted to television
comedies.  It's alive and well in Brussels, and mixes very nicely
with the Dunning-Kruger effect.

Laura


From rosuav at gmail.com  Sat Jun 27 07:33:33 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 27 Jun 2015 21:33:33 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmlt9l$m7s$1@news.albasani.net>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org>
 <mmlt9l$m7s$1@news.albasani.net>
Message-ID: <CAPTjJmrqM4nA_WsdQmuZdBiGrsUvOt=aSGQo-C=KA0hcodtnWw@mail.gmail.com>

On Sat, Jun 27, 2015 at 8:18 PM, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
> On 27.06.2015 11:17, Chris Angelico wrote:
>
>> Good, so this isn't like that episode of Yes Minister when they were
>> trying to figure out whether to allow a chemical factory to be built.
>
> I must admit that I have no clue about that show or that epsisode in
> particular and needed to read up on it:
> https://en.wikipedia.org/wiki/The_Greasy_Pole
>
>>> I must admit that I haven't seen your ideas in this thread?
>>
>> No, the proposal I'm putting together is unrelated. You'll see the
>> *vast* extent of my security skills here:
>>
>> https://github.com/Rosuav/ThirdSquare
>>
>> My contribution to this thread has been fairly minor, just suggesting
>> one attack that doesn't even work any more, not much else.
>
> Well, if people already have a solution ready there's a good chance that
> any criticism falls on deaf ears. In any case something that others have
> to be responsible for, their party, their choice.
>
> I've looked at your code even though I don't know pike. That's the
> typesafe JavaScript derivative, isn't it?

Not really; it's more like "Python semantics meets C++ syntax". But
that's still off-topic for this list; I'd be happy to continue
discussion off-list with anyone who's interested.

The most interesting part of the project is the README, to be honest.
Even if you can't understand a single line of the code, you'll be able
to see the specs. Grokking the code is a bonus.

> The only thing that I found horrible was the ssh key format to PKCS
> parsing. Man that's hacky :-) You're creating a DER structure on-the-fly
> that you fill with the key and that you then have parsed back. I've only
> seen ssh-keygen used to generate keys (not to initiate actual ssh
> connections), why don't you use openssl to generate the keys? I think
> you can generate a RSA keypair in openssl (also valid for ssh should you
> need it) and I'm pretty sure that you can generate a ssh public key with
> ssh-keygen from that private keypair file. That would eliminate the need
> to do this kind of parsing, but it's just a PoC as I understand it.

Yeah, it's pretty disgusting. I could actually use Pike to generate
the keys, rather than using ssh-keygen at all, but I wanted to
demonstrate that this is using a well-known key generation method,
ergo I don't need to separately prove that the keys are appropriately
random. (Not that I distrust Pike, but it's one less thing to try to
prove.)

> It appears to be online-only, is that correct? Is Internet coverage so
> good down under? I wish this were the case in Germany :-/

Correct, that's one of the key changes. Our current system (Myki) is a
stored-value card - if you recharge a hundred bucks, then clone your
card, you'd have two cards with a hundred bucks each. With
ThirdSquare, if you clone your card, you have two cards that draw on
the same hundred bucks. (Though I still don't want people cloning
cards, as it would confuse the system some. Plus, it'd be really
REALLY bad if someone could clone someone *else's* card, thus
effectively stealing a copy of it. So the cards themselves need some
kind of security, but short of public key crypto performed actually on
the cards, I'm not sure how to do that.)

My plan is to stick a 3G/4G device onto each bus. So long as there's
mobile phone coverage on all routes, which should be fine in suburbia,
the system will work. It can cope with short dropouts (up to ten
minutes), queueing requests in the client.

> Not 100% about it, but I think that the bus concepts that are active in
> Germany (locally in some cities) either user asymmetric transponders
> (i.e. SmartMX), which gives a beautiful, decentralized, secure and
> offline solution at the cost of being comparatively expensive. The
> others use symmetric transponders which have limited off-line
> functionality: i.e. monotonic counters which are reset in a
> cryptographically secured way by backend systems every time a
> online-connection persists and which are counted down in the offline case.

Thanks for the name, I'll check that out. Ideally, I'd like to use
off-the-shelf hardware for everything, and open-source software. It
should be possible for anyone to pick up the specs, buy their own
hardware, and create something that interoperates with the rest of the
system - for instance, the Red Engine Group could allow their
customers to ding their tickets to buy coffee - simply by providing
the appropriate public keys to the central authorizing database. That
would be a massive improvement over Melbourne's previous ticketing
system (Metcard), which was entirely proprietary; expansion of the
fleet required additional validators, and basically the public
transport operators had to beg, cap in hand, for the company to do
them a favour - for which, of course, they then also had to pay the
earth for. But I digress.

ChrisA


From jpiitula at ling.helsinki.fi  Sat Jun 27 08:23:07 2015
From: jpiitula at ling.helsinki.fi (Jussi Piitulainen)
Date: Sat, 27 Jun 2015 15:23:07 +0300
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org>
 <mailman.133.1435404332.3674.python-list@python.org>
Message-ID: <lf5d20h9w7o.fsf@ling.helsinki.fi>

Laura Creighton writes:

> Johannes, if you don't know "Yes, Minister" then you most likely do
> not know the Politician's Syllogism (which now has its own wikipedia
> page :)  And I _didn't_ do it!  Honest!)
>
> Something must be done.
> This is something.
> Therefore we must do it!

Surely that's to be worded as follows? To have a stricter syllogistic
form.

We need to do something.
This is something.
Therefore we need to do this.

Or,

We must do something.
This is something.
Therefore we must do this.

Or, but this feels weaker to me,

Something must be done.
This is something.
Therefore this must be done.

ISWIM. In particular, I think the move from the agentless passive (must
be done) to the specific expression of agency (we must do) seems to me
to be a *different* joke.

(I was tempted to call it passive temperature but that would have been
yet another, unrelated joke.)

:)


From lac at openend.se  Sat Jun 27 08:27:38 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 27 Jun 2015 14:27:38 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: Message from Chris Angelico <rosuav@gmail.com> of "Sat,
 27 Jun 2015 20:16:47 +1000."
 <CAPTjJmqE_GbFCXQERD7xzxs0iBsEOsCg-3i_sRpuLCPUTavQgg@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org>
 <slrnmosr71.1nu.jon+usenet@frosty.unequivocal.co.uk> <!
 mmlsh8$kj6$1@news.albasani.net><CAPTjJmqE_GbFCXQERD7xzxs0iBsEOsCg-3i_sRpuLCPUTavQgg@mail.gmail.com>
Message-ID: <201506271227.t5RCRcjL028373@fido.openend.se>

In a message of Sat, 27 Jun 2015 20:16:47 +1000, Chris Angelico writes:

>Okay, Johannes, NOW you're proving that you don't have a clue what
>you're talking about. D-K effect doesn't go away...
>
>ChrisA

You need to read the paper again.  That was the whole point -- when
Kruger and Dunning went and taught the people at the bottom quadrile
some basic skill in the task being estimated, and taught people at
the top quadrile how poorly their peers were performing, their ability
to estimate how they would score relative to their peers improved
a whole lot.

But, of course, since these were academics studying students, they had
access to bottom-quadrile performers who actually wanted to learn and
improve.  In the real world, it is getting the bottom-performers to
even notice that they need improvement that may be the most difficult task.

Laura











From invalid at invalid.invalid  Sat Jun 27 08:38:33 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Sat, 27 Jun 2015 12:38:33 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
Message-ID: <mmm5g9$7au$2@reader1.panix.com>

On 2015-06-26, Randall Smith <randall at tnr.cc> wrote:

> The only person who can read a file is the owner.

That's always the plan, but many a successful exploit has been based
on breaking that assumption.  If privacy actually matters, that's not
a good assumption to rely on as a single point of failure.

--
Grant



From lac at openend.se  Sat Jun 27 08:48:09 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 27 Jun 2015 14:48:09 +0200
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: Message from Jussi Piitulainen <jpiitula@ling.helsinki.fi>
 of "Sat, 27 Jun 2015 15:23:07 +0300." <lf5d20h9w7o.fsf@ling.helsinki.fi>
References: <mmcagn$aa5$1@ger.gmane.org>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org>
 <mailman.133.1435404332.3674.python-list@python.org><l!
 f5d20h9w7o.fsf@ling.helsinki.fi>
Message-ID: <201506271248.t5RCm9rt028993@fido.openend.se>

In a message of Sat, 27 Jun 2015 15:23:07 +0300, Jussi Piitulainen writes:
>Laura Creighton writes:
>
>> Johannes, if you don't know "Yes, Minister" then you most likely do
>> not know the Politician's Syllogism (which now has its own wikipedia
>> page :)  And I _didn't_ do it!  Honest!)
>>
>> Something must be done.
>> This is something.
>> Therefore we must do it!
>
>Surely that's to be worded as follows? To have a stricter syllogistic
>form.
>
>We need to do something.
>This is something.
>Therefore we need to do this.

Somehow doesn't have the same comedic ring, though.  So the version I
posted is what was on the tv show.  (Or rather in Yes, Prime Minister.)
The Minister becomes Prime Minister for the last 2 seasons.

In the television show it is just called 'Politician's Logic'.  Not
sure who started calling it the Politicians Syllogism.

Laura



From gheskett at wdtv.com  Sat Jun 27 09:26:01 2015
From: gheskett at wdtv.com (Gene Heskett)
Date: Sat, 27 Jun 2015 09:26:01 -0400
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <201506271227.t5RCRcjL028373@fido.openend.se>
References: <mmcagn$aa5$1@ger.gmane.org>
 <CAPTjJmqE_GbFCXQERD7xzxs0iBsEOsCg-3i_sRpuLCPUTavQgg@mail.gmail.com>
 <201506271227.t5RCRcjL028373@fido.openend.se>
Message-ID: <201506270926.01865.gheskett@wdtv.com>



On Saturday 27 June 2015 08:27:38 Laura Creighton wrote:
> In a message of Sat, 27 Jun 2015 20:16:47 +1000, Chris Angelico writes:
> >Okay, Johannes, NOW you're proving that you don't have a clue what
> >you're talking about. D-K effect doesn't go away...
> >
> >ChrisA
>
> You need to read the paper again.  That was the whole point -- when
> Kruger and Dunning went and taught the people at the bottom quadrile
> some basic skill in the task being estimated, and taught people at
> the top quadrile how poorly their peers were performing, their ability
> to estimate how they would score relative to their peers improved
> a whole lot.
>
> But, of course, since these were academics studying students, they had
> access to bottom-quadrile performers who actually wanted to learn and
> improve.  In the real world, it is getting the bottom-performers to
> even notice that they need improvement that may be the most difficult
> task.
>
> Laura

The rest of the readers of this list would do well to change "may" above 
to "is", and carve the last sentence into something fairly substantial 
as it is a basic truth.

Zircon crystal would be ideal, we've found a few grains of it over 4 
Billion years old, but granite would do for this generation.  Laura 
obviously "gets it".

Sadly, it is entirely too true in the real world. Too often the bottom 
person who made a good sales pitch, once hired, is either incapable of 
learning, or loses interest after he has been hired. I've seen both. The 
basic education they received is to blame for much of that effect. So 
they wind up getting shuffled around to various sub-jobs until you find 
something they can do efficiently.  Many times they weren't ever aware 
of why they were being moved.  Telling them depresses them, so its 
usually best to just let it work itself out.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>


From ian.g.kelly at gmail.com  Sat Jun 27 10:59:37 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 27 Jun 2015 08:59:37 -0600
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CAPTjJmrqM4nA_WsdQmuZdBiGrsUvOt=aSGQo-C=KA0hcodtnWw@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.130.1435395217.3674.python-list@python.org>
 <mmlp42$p4v$1@news.albasani.net>
 <mailman.131.1435396645.3674.python-list@python.org>
 <mmlt9l$m7s$1@news.albasani.net>
 <CAPTjJmrqM4nA_WsdQmuZdBiGrsUvOt=aSGQo-C=KA0hcodtnWw@mail.gmail.com>
Message-ID: <CALwzid=+grOhfRtXYk-Dh7j=hLtxauSxS3gZr+iYBXTn=vC1kA@mail.gmail.com>

On Sat, Jun 27, 2015 at 5:33 AM, Chris Angelico <rosuav at gmail.com> wrote:
> On Sat, Jun 27, 2015 at 8:18 PM, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
>> I've looked at your code even though I don't know pike. That's the
>> typesafe JavaScript derivative, isn't it?
>
> Not really; it's more like "Python semantics meets C++ syntax". But
> that's still off-topic for this list; I'd be happy to continue
> discussion off-list with anyone who's interested.

That description could apply equally well to Javascript, though. ;-)


From ian.g.kelly at gmail.com  Sat Jun 27 11:09:28 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 27 Jun 2015 09:09:28 -0600
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CALwzid=-PqY-ObJQCkFThoxfbhtaHcHwAB7qPRz7uJfafkZq1A@mail.gmail.com>

On Sat, Jun 27, 2015 at 2:38 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Can you [generic you] believe that attackers can *reliably* attack remote
> systems based on a 20?s timing differences? If you say "No", then you fail
> Security 101 and should step away from the computer until a security expert
> can be called in to review your code.

Of course. I wouldn't bet the house on it, but with the proposed
substitution cipher system, I don't see why there would be any
measurable timing differences at all based on the choice of key. The
time to obfuscate a single byte is constant, so the total time to
obfuscate the payload should just be a function of the length of the
data.

Secondly, the 200 (or whatever) response to the client does not depend
on the outcome of the obfuscation step, so there is no reason that the
server cannot simply respond first and obfuscate after, giving the
client nothing to time.


From torriem at gmail.com  Sat Jun 27 13:02:31 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sat, 27 Jun 2015 11:02:31 -0600
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmkf5b$uvj$1@news.albasani.net>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkf5b$uvj$1@news.albasani.net>
Message-ID: <558ED727.1070804@gmail.com>

On 06/26/2015 03:11 PM, Johannes Bauer wrote:
> You misunderstand. This is now how it works, this is not how any of this
> works. Steven does not *at all* have to prove to you your system is
> breakable or show actual attacks. YOU have to prove that your system is
> secure. 

Ahh the holy grail of computer science.  Now it's been a while since I
finished my CS degree, but I recall spending a lot of time in class
talking about the proving code correctness, which is a similar problem,
and learning that that was thought to be NP complete.  Furthermore you
cannot prove a negative, which is what proving security is for anything
but the trivial case.

Are you saying this is untrue?

Obviously there are best practices, which you are an expert in.  But how
does one prove a system is secure except by enumerating attack vectors
and addressing each one, preferably in the design phase?



From randall at tnr.cc  Sat Jun 27 13:08:46 2015
From: randall at tnr.cc (Randall Smith)
Date: Sat, 27 Jun 2015 12:08:46 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
Message-ID: <mmmlb3$btl$1@ger.gmane.org>

On 06/26/2015 08:21 PM, Chris Angelico wrote:
> On Sat, Jun 27, 2015 at 6:09 AM, Randall Smith <randall at tnr.cc> wrote:
>> Give me one plausible scenario where an attacker can cause malware to hit
>> the disk after bytearray.translate with a 256 byte translation table and
>> I'll be thankful to you.
>
> The entire 256-byte translation table is significant ONLY if you need
> all 256 possible bytes. Suppose I want to generate the following byte
> sequence:
>
> "\xCD\x19"
>
> (Okay, this is a slightly oversimplified example, as this attack
> doesn't work on a modern Windows. But back in the days of DOS, this
> program would reboot your computer.)
>
> How many truly different translation tables are there if I'm trying to
> produce this? Just 256*255, or 65280. If I send random two-byte files,
> there is one chance in that of my "malware" successfully landing. Once
> I've sent about 45,000 of those files, I have a fifty-fifty chance of
> having hit it. Send twice as many, I have a 75% chance of success,
> etc.
>

Yes, that's true.  It's even an issue with AES, which uses padding to 
overcome.  That said, remember these are bytes going straight to disk. 
I'm really not concerned about 2 or 3 byte malware and the probability 
plunges after that.  And remember, normal use case is AES encrypted data.

Quite interesting.

Though I didn't mention it in the description, the storage server is 
appending a CRC32 checksum for routine integrity checks.  So by the time 
the data hits the disk, it will have added both a 256 byte translation 
table and a 4 byte checksum.  I think that would interfere with any 
extremely short malware.


> Malware can be crafted to fit within certain restrictions. I saw a
> proof-of-concept and analysis document detailing a particular remote
> code execution/privilege escalation attack that involved stuffing
> "text" into an entry field and then inducing the program to read that
> into its stack, finally triggering it by some sort of buffer overflow,
> I think. The text had to be no more than X bytes long (because that's
> all the entry field was set to accept - it'd truncate after that), and
> had to not contain any NUL bytes, and there might have been other
> restrictions too. Sure, it makes it harder to write your malware...
> but imagine if you can write something in just a handful of different
> bytes, which then goes and triggers something else. You could have an
> extremely plausible attack that might need only a day's uploading to
> deliver.
>
> It makes no difference that there are 256! possible encryption keys,
> if most of them have the same result.
>
> ChrisA
>


Thankyou.  Nice points. I do think given the risks (there are always 
risks) discussed, a successful attack of this nature is not very likely. 
  Worse case, something that looks like this would land on the disk.

crc32 checksum + translation table + malware

with a generated base64 name and no extension.


Doesn't seem like much of a threat.   Much less likely than a bug in the 
standard Crytpo libraries.

-Randall




From steve at pearwood.info  Sat Jun 27 13:35:20 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 28 Jun 2015 03:35:20 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.139.1435417817.3674.python-list@python.org>
Message-ID: <558eded8$0$1642$c3e8da3$5496439d@news.astraweb.com>

On Sun, 28 Jun 2015 01:09 am, Ian Kelly wrote:

> On Sat, Jun 27, 2015 at 2:38 AM, Steven D'Aprano <steve at pearwood.info>
> wrote:
>> Can you [generic you] believe that attackers can *reliably* attack remote
>> systems based on a 20?s timing differences? If you say "No", then you
>> fail Security 101 and should step away from the computer until a security
>> expert can be called in to review your code.
> 
> Of course. I wouldn't bet the house on it, but with the proposed
> substitution cipher system, I don't see why there would be any
> measurable timing differences at all based on the choice of key.

I wouldn't bet one wooden nickle on it. Not without a security audit of the
application. And then what happens when the implementation changes and the
audit is no longer valid?

Despite his initial claim that he doesn't want to use AES because it's too
slow implemented as pure Python, Randall has said that the application will
offer AES encryption as an option. (He says it is enabled by default,
except that the user can turn it off.) So the code is already there, all he
has to do is call it.

It might not be a timing attack. Maybe there's a vulnerability in the
application that if you upload a sufficiently large file, a buffer will
overflow and you can force the key of your choosing. Who knows? Bugs
happen. The nature of how the hypothetical key leakage happens is less
important than the consequences if there is one.

Randall can:

(1) bet the security of his application and his users on the key never
leaking; 

"Why have you situated a naked flame right next to the gas tank?"

"It's okay, I'm confident that the tank will never leak."


or


(2) use something which, *even if the key leaks*, is still resistant to
preimage attacks.


The choice ought to be a no-brainer. The fact that folks are seriously
considering using something barely one step up from a medieval substitution
cipher in 2015 for something with real security consequences if it is
broken goes to show what a lousy job the IT industry does for security.



> The time to obfuscate a single byte is constant, 

Are you sure about that? Bet your house? How about your computer?


# Python 3.3 on Linux, YMMV

py> text = 'NOBODY expects the Spanish Inquisition!'*50000
py> import string
py> s = string.digits + string.ascii_letters
py> t = (string.ascii_uppercase + string.digits[::-1] + 
... string.ascii_lowercase)
py> trans1 = str.maketrans('abcdef', 'fedcba')
py> trans2 = str.maketrans(s, t)
py> trans3 = str.maketrans('aZ', 'Za')
py> with Stopwatch():
...     x = str.translate(text, trans1)
...
time taken: 0.427513 seconds
py> with Stopwatch():
...     x = str.translate(text, trans2)
...
time taken: 0.228869 seconds
py> with Stopwatch():
...     x = str.translate(text, trans3)
...
time taken: 0.387105 seconds



> so the total time to 
> obfuscate the payload should just be a function of the length of the
> data.


Good thing you didn't bet your house.



-- 
Steven



From no.email at nospam.invalid  Sat Jun 27 13:45:30 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Sat, 27 Jun 2015 10:45:30 -0700
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkf5b$uvj$1@news.albasani.net>
 <mailman.140.1435424557.3674.python-list@python.org>
Message-ID: <876169vydh.fsf@nightsong.com>

Michael Torrie <torriem at gmail.com> writes:
> Furthermore you cannot prove a negative, which is what proving
> security is for anything but the trivial case. Are you saying this is
> untrue?

I've always thought that there are no two even numbers that when you add
them together, give you an odd number.  Are you saying that statement
can't be proven?

> But how does one prove a system is secure except by enumerating attack
> vectors

In the case of encryption, you do a reduction proof to a recognized
primitive like AES.  That is, you show that if your system is breakable,
you can transform the break into a break against AES itself.  That's the
best you can do at the moment, because the open status of the P!=NP
problem means that no one knows how to prove that any primitive (such as
AES) is secure.  The reduction proof means that the evidence for AES's
security also applies to your system.

Of course that's just for the cipher itself.  For the entire surrounding
software/hardware/process system which is mostly not mathematical,
you're right, there's no way to (mathematically) prove security or even
to define it.


From steve at pearwood.info  Sat Jun 27 13:58:02 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 28 Jun 2015 03:58:02 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.139.1435417817.3674.python-list@python.org>
 <558eded8$0$1642$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <558ee429$0$1662$c3e8da3$5496439d@news.astraweb.com>

On Sun, 28 Jun 2015 03:35 am, Steven D'Aprano wrote:

> On Sun, 28 Jun 2015 01:09 am, Ian Kelly wrote:

>> The time to obfuscate a single byte is constant,
> 
> Are you sure about that? Bet your house? How about your computer?

Correction: the example I showed uses str, not bytes.

With bytes, the timing differences are much smaller. Are they statistically
distinguishable? Don't know. On my machine, they appear to be, although
that could be just a fluke. Is there a guarantee that bytes.translate will
always be constant time per byte? No of course not. Might the application
itself some day start using str.translate? Who knows?

The point is, you cannot rely on this. Preventing leakage is *hard*.


-- 
Steven



From randall at tnr.cc  Sat Jun 27 14:17:09 2015
From: randall at tnr.cc (Randall Smith)
Date: Sat, 27 Jun 2015 13:17:09 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmlmsk$m2e$1@ger.gmane.org>
References: <mmcagn$aa5$1@ger.gmane.org> <mmlmsk$m2e$1@ger.gmane.org>
Message-ID: <mmmpba$72e$1@ger.gmane.org>

On 06/27/2015 03:29 AM, Peter Otten wrote:


> Would it be sufficient to prepend the chunk with one block, say, of random
> data? To unmangle you'd just strip off that block.
>
> BLOCK = os.urandom(BLOCKSIZE)
>
> def mangle(source, dest):
>      dest.write(BLOCK)
>      shutil.copyfileobj(source, dest)
>
> def unmangle(source, dest):
>      source.read(BLOCKSIZE)
>      shutil.copyfileobj(source, dest)
>
> Disclaimer: I did not follow the ongoing discussion.
>

That is happening as a side effect.  Though not completely random, after 
running the data through a translation table, the 256 byte table is 
prepended.  Then a 4 byte checksum is calculated and prepended.

-Randall




From randall at tnr.cc  Sat Jun 27 14:22:33 2015
From: randall at tnr.cc (Randall Smith)
Date: Sat, 27 Jun 2015 13:22:33 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <mmm5g9$7au$2@reader1.panix.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmm5g9$7au$2@reader1.panix.com>
Message-ID: <mmmple$bij$1@ger.gmane.org>

On 06/27/2015 07:38 AM, Grant Edwards wrote:
> On 2015-06-26, Randall Smith <randall at tnr.cc> wrote:
>
>> The only person who can read a file is the owner.
>
> That's always the plan, but many a successful exploit has been based
> on breaking that assumption.  If privacy actually matters, that's not
> a good assumption to rely on as a single point of failure.
>
> --
> Grant
>


The owner (client software) encrypts the data using AES.  This is the 
default behavior of the client software.  If the client chooses to 
disable encryption, that's their issue for sure.  I'm trying to make 
sure it doesn't become the storage server's issue too.

-Randall



From torriem at gmail.com  Sat Jun 27 14:34:39 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sat, 27 Jun 2015 12:34:39 -0600
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <876169vydh.fsf@nightsong.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmkf5b$uvj$1@news.albasani.net>
 <mailman.140.1435424557.3674.python-list@python.org>
 <876169vydh.fsf@nightsong.com>
Message-ID: <CAM1e3vr5NfLsO7JLk0qaJ2aN4fXyS0QOC35-+yMC5pjK4_4AKw@mail.gmail.com>

On Jun 27, 2015 11:51 AM, "Paul Rubin" <no.email at nospam.invalid> wrote:
>
> Michael Torrie <torriem at gmail.com> writes:
> > Furthermore you cannot prove a negative, which is what proving
> > security is for anything but the trivial case. Are you saying this is
> > untrue?
>
> I've always thought that there are no two even numbers that when you add
> them together, give you an odd number.  Are you saying that statement
> can't be proven?
>
> > But how does one prove a system is secure except by enumerating attack
> > vectors
>
> In the case of encryption, you do a reduction proof to a recognized
> primitive like AES.  That is, you show that if your system is breakable,
> you can transform the break into a break against AES itself.  That's the
> best you can do at the moment, because the open status of the P!=NP
> problem means that no one knows how to prove that any primitive (such as
> AES) is secure.  The reduction proof means that the evidence for AES's
> security also applies to your system.
>
> Of course that's just for the cipher itself.  For the entire surrounding
> software/hardware/process system which is mostly not mathematical,
> you're right, there's no way to (mathematically) prove security or even
> to define it.

Ahh okay. So what he's referring to must be such reductions and proofs of
these provable aspects, though he spoke very broadly.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150627/bb45e011/attachment.html>

From steve at pearwood.info  Sat Jun 27 14:50:02 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 28 Jun 2015 04:50:02 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
Message-ID: <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>

On Sun, 28 Jun 2015 03:08 am, Randall Smith wrote:

> Though I didn't mention it in the description, the storage server is
> appending a CRC32 checksum for routine integrity checks.  So by the time
> the data hits the disk, it will have added both a 256 byte translation
> table and a 4 byte checksum.


http://stackoverflow.com/questions/1515914/crc32-collision




-- 
Steven



From steve at pearwood.info  Sat Jun 27 14:51:36 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 28 Jun 2015 04:51:36 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmm5g9$7au$2@reader1.panix.com>
 <mailman.143.1435429368.3674.python-list@python.org>
Message-ID: <558ef0b6$0$1673$c3e8da3$5496439d@news.astraweb.com>

On Sun, 28 Jun 2015 04:22 am, Randall Smith wrote:

> The owner (client software) encrypts the data using AES. ?This is the
> default behavior of the client software. ?If the client chooses to
> disable encryption, that's their issue for sure.

I cannot imagine what you think you gain from allowing that to be optional.
Apart from privacy and security breaches.



-- 
Steven



From ian.g.kelly at gmail.com  Sat Jun 27 16:16:16 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 27 Jun 2015 14:16:16 -0600
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558eded8$0$1642$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.139.1435417817.3674.python-list@python.org>
 <558eded8$0$1642$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CALwzid=jJxpQmE=u1Ey5oks15bfPzxFsBUrDz-xLSXB=u6ejAg@mail.gmail.com>

On Sat, Jun 27, 2015 at 11:35 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sun, 28 Jun 2015 01:09 am, Ian Kelly wrote:
>
>> On Sat, Jun 27, 2015 at 2:38 AM, Steven D'Aprano <steve at pearwood.info>
>> wrote:
>>> Can you [generic you] believe that attackers can *reliably* attack remote
>>> systems based on a 20?s timing differences? If you say "No", then you
>>> fail Security 101 and should step away from the computer until a security
>>> expert can be called in to review your code.
>>
>> Of course. I wouldn't bet the house on it, but with the proposed
>> substitution cipher system, I don't see why there would be any
>> measurable timing differences at all based on the choice of key.
>
> I wouldn't bet one wooden nickle on it. Not without a security audit of the
> application. And then what happens when the implementation changes and the
> audit is no longer valid?

I don't disagree about the security audit, although I think you'll
find that such things will require a greater investment of resources
than a wooden nickel.

> Despite his initial claim that he doesn't want to use AES because it's too
> slow implemented as pure Python, Randall has said that the application will
> offer AES encryption as an option.

Once again you're confusing what he said about the server with what he
said about the client. Just because he considers it too slow for data
mangling on the server doesn't make it too slow for any use.

>> The time to obfuscate a single byte is constant,
>
> Are you sure about that? Bet your house? How about your computer?
>
>
> # Python 3.3 on Linux, YMMV
>
> py> text = 'NOBODY expects the Spanish Inquisition!'*50000
> py> import string
> py> s = string.digits + string.ascii_letters
> py> t = (string.ascii_uppercase + string.digits[::-1] +
> ... string.ascii_lowercase)
> py> trans1 = str.maketrans('abcdef', 'fedcba')
> py> trans2 = str.maketrans(s, t)
> py> trans3 = str.maketrans('aZ', 'Za')
> py> with Stopwatch():
> ...     x = str.translate(text, trans1)
> ...
> time taken: 0.427513 seconds
> py> with Stopwatch():
> ...     x = str.translate(text, trans2)
> ...
> time taken: 0.228869 seconds
> py> with Stopwatch():
> ...     x = str.translate(text, trans3)
> ...
> time taken: 0.387105 seconds

Your examples are using partial keys of different sizes. It's hardly
surprising that the timing varies when you pass dicts of varying sizes
as the translation tables.

py> a = list(range(256))
py> b = random.sample(a, 256)
py> c = random.sample(a, 256)
py> d = random.sample(a, 256)
py> min(timeit.repeat("str.translate(text, a)", "from __main__ import
text, a", number=10, repeat=10))
0.9780099680647254
py> min(timeit.repeat("str.translate(text, b)", "from __main__ import
text, b", number=10, repeat=10))
0.9837233647704124
py> min(timeit.repeat("str.translate(text, c)", "from __main__ import
text, c", number=10, repeat=10))
0.9627216667868197
py> min(timeit.repeat("str.translate(text, d)", "from __main__ import
text, d", number=10, repeat=10))
0.9793561780825257
py> min(timeit.repeat("str.translate(text, c)", "from __main__ import
text, c", number=10, repeat=10))
0.9840573272667825

I ran it on c a second time to see if the 0.962 timing was systemic or
a fluke. The fact that c produced both the shortest and longest
timings out of only two runs lends me confidence (for the purpose of
this discussion) that the variation seen in these timings is random
and not correlated to the keys used.


From jeanpierreda at gmail.com  Sat Jun 27 16:30:06 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sat, 27 Jun 2015 13:30:06 -0700
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558e3fc4$0$1658$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.119.1435377960.3674.python-list@python.org>
 <558e3fc4$0$1658$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CABicbJ+hCxXTvYjmbNB=qeTv6zb+3Ld5VEFf03EzNTsLcEOnHA@mail.gmail.com>

On Fri, Jun 26, 2015 at 11:16 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sat, 27 Jun 2015 02:05 pm, Devin Jeanpierre wrote:
>
>> On Fri, Jun 26, 2015 at 8:38 PM, Steven D'Aprano <steve at pearwood.info>
>> wrote:
>>> Now you say that the application encrypts the data, except that the user
>>> can turn that option off.
>>>
>>> Just make the AES encryption mandatory, not optional. Then the user
>>> cannot upload unencrypted malicious data, and the receiver cannot read
>>> the data. That's two problems solved.
>>
>> No, because another application could pretend to be the file-sending
>> application, but send unencrypted data instead of encrypted data.
>
> Did you stop reading my post when you got to that? Because I went on to say:

At that point I quit in frustration, yeah.

> "Actually, the more I think about this, the more I come to think that the
> only way this can be secure is for both the sending client application and
> the receiving client appl to both encrypt the data. The sender can't
> trust the receiver not to read the files, so the sender has to encrypt; the
> receiver can't trust the sender not to send malicious files, so the
> receiver has to encrypt too."

When you realize you've said something completely wrong, you should
edit your email.

-- Devin


From rosuav at gmail.com  Sat Jun 27 19:05:53 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 28 Jun 2015 09:05:53 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558ef0b6$0$1673$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <mmm5g9$7au$2@reader1.panix.com>
 <mailman.143.1435429368.3674.python-list@python.org>
 <558ef0b6$0$1673$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmqydUSZdYLdv0AVO4vmn_ULp9bxTba5N3ndtETEOsWGuw@mail.gmail.com>

On Sun, Jun 28, 2015 at 4:51 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sun, 28 Jun 2015 04:22 am, Randall Smith wrote:
>
>> The owner (client software) encrypts the data using AES.  This is the
>> default behavior of the client software.  If the client chooses to
>> disable encryption, that's their issue for sure.
>
> I cannot imagine what you think you gain from allowing that to be optional.
> Apart from privacy and security breaches.

I've no idea whether this is the case or not, but one thing you might
gain is independence from a third-party module. You could, for
instance, automatically AES-encrypt your data, but only if "from
Crypto.Cipher import AES" didn't raise ImportError. That effectively
makes encryption optional (the program won't barf for lack of pycrypto
installation), while still clearly being the default - and if you have
a nice loud warning, then it's clear that encryption is the normal
state, and the fallback is a lesser state.

ChrisA


From steve at pearwood.info  Sat Jun 27 21:18:17 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 28 Jun 2015 11:18:17 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.119.1435377960.3674.python-list@python.org>
 <558e3fc4$0$1658$c3e8da3$5496439d@news.astraweb.com>
 <mailman.146.1435437048.3674.python-list@python.org>
Message-ID: <558f4b59$0$1655$c3e8da3$5496439d@news.astraweb.com>

On Sun, 28 Jun 2015 06:30 am, Devin Jeanpierre wrote:

> On Fri, Jun 26, 2015 at 11:16 PM, Steven D'Aprano <steve at pearwood.info>
> wrote:
>> On Sat, 27 Jun 2015 02:05 pm, Devin Jeanpierre wrote:
>>
>>> On Fri, Jun 26, 2015 at 8:38 PM, Steven D'Aprano <steve at pearwood.info>
>>> wrote:
>>>> Now you say that the application encrypts the data, except that the
>>>> user can turn that option off.
>>>>
>>>> Just make the AES encryption mandatory, not optional. Then the user
>>>> cannot upload unencrypted malicious data, and the receiver cannot read
>>>> the data. That's two problems solved.
>>>
>>> No, because another application could pretend to be the file-sending
>>> application, but send unencrypted data instead of encrypted data.
>>
>> Did you stop reading my post when you got to that? Because I went on to
>> say:
> 
> At that point I quit in frustration, yeah.
> 
>> "Actually, the more I think about this, the more I come to think that the
>> only way this can be secure is for both the sending client application
>> and the receiving client appl to both encrypt the data. The sender can't
>> trust the receiver not to read the files, so the sender has to encrypt;
>> the receiver can't trust the sender not to send malicious files, so the
>> receiver has to encrypt too."
> 
> When you realize you've said something completely wrong, you should
> edit your email.

If both the sender and receiver encrypt the data, how is is "completely
wrong" to say that encrypting data should be mandatory?




-- 
Steven



From jeanpierreda at gmail.com  Sat Jun 27 22:11:14 2015
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sat, 27 Jun 2015 19:11:14 -0700
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558f4b59$0$1655$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.119.1435377960.3674.python-list@python.org>
 <558e3fc4$0$1658$c3e8da3$5496439d@news.astraweb.com>
 <mailman.146.1435437048.3674.python-list@python.org>
 <558f4b59$0$1655$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CABicbJJXkhvq1himE79bjFJbafCZ_P-j5p-UJyVR=idVTUpZjQ@mail.gmail.com>

On Sat, Jun 27, 2015 at 6:18 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sun, 28 Jun 2015 06:30 am, Devin Jeanpierre wrote:
>
>> On Fri, Jun 26, 2015 at 11:16 PM, Steven D'Aprano <steve at pearwood.info>
>> wrote:
>>> On Sat, 27 Jun 2015 02:05 pm, Devin Jeanpierre wrote:
>>>
>>>> On Fri, Jun 26, 2015 at 8:38 PM, Steven D'Aprano <steve at pearwood.info>
>>>> wrote:
>>>>> Now you say that the application encrypts the data, except that the
>>>>> user can turn that option off.
>>>>>
>>>>> Just make the AES encryption mandatory, not optional. Then the user
>>>>> cannot upload unencrypted malicious data, and the receiver cannot read
>>>>> the data. That's two problems solved.
>>>>
>>>> No, because another application could pretend to be the file-sending
>>>> application, but send unencrypted data instead of encrypted data.
>>>
>>> Did you stop reading my post when you got to that? Because I went on to
>>> say:
>>
>> At that point I quit in frustration, yeah.
>>
>>> "Actually, the more I think about this, the more I come to think that the
>>> only way this can be secure is for both the sending client application
>>> and the receiving client appl to both encrypt the data. The sender can't
>>> trust the receiver not to read the files, so the sender has to encrypt;
>>> the receiver can't trust the sender not to send malicious files, so the
>>> receiver has to encrypt too."
>>
>> When you realize you've said something completely wrong, you should
>> edit your email.
>
> If both the sender and receiver encrypt the data, how is is "completely
> wrong" to say that encrypting data should be mandatory?

That isn't what I was calling completely wrong. This is:

>>>>> Just make the AES encryption mandatory, not optional. Then the user
>>>>> cannot upload unencrypted malicious data, and the receiver cannot read
>>>>> the data. That's two problems solved.

The user can still upload unencrypted malicious data by writing their
own client that doesn't have mandatory AES encryption.

You realized this later in the email, apparently, which is why you
should have edited your own email to delete your original, insecure,
suggestion. :(

That said, I appreciate the work you've done here asking for a
specific threat model and pushing back on the idea that it's up to
python-list to prove something is insecure, not the other way around.
That's important. I think, for the same reasons, it's also important
to be really careful what cryptosystems we discuss, and not suggest or
appear to suggest ones that won't work.


P.S. FWIW, the base64 idea has a lot of promise and is probably
fundamentally better than a crypto algorithm. With something along the
lines of base64 -- say, encoding a file using just the letters 'a' and
'b' -- one might try to make it it literally impossible to write "bad"
things to disk, whereas with any crypto, it is always possible to
obtain the key, so one has to be careful with key management to
prevent/mitigate that.  (One might add: why not both? Beats me. I like
using extension modules.)

P.P.S.: of course, I'm not an expert.

-- Devin


From rustompmody at gmail.com  Sun Jun 28 00:51:25 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sat, 27 Jun 2015 21:51:25 -0700 (PDT)
Subject: Turning string into object (name)
In-Reply-To: <mailman.120.1435379463.3674.python-list@python.org>
References: <3ba959bc-f25f-4655-8c1f-ce2888210e1b@googlegroups.com>
 <mailman.99.1435291217.3674.python-list@python.org>
 <37e631f2-16ea-431a-aa18-fb74cdedc278@googlegroups.com>
 <mailman.120.1435379463.3674.python-list@python.org>
Message-ID: <fadf7b95-a963-4ff3-82a7-8cd8afb335b4@googlegroups.com>

On Saturday, June 27, 2015 at 10:01:15 AM UTC+5:30, Chris Angelico wrote:
> On Sat, Jun 27, 2015 at 2:11 PM, Rustom Mody  wrote:
> Beyond that, I don't think I can much help you.

Yeah its more python-OT than I first thought...
Need to go through the ropes of linking github with 
https://travis-ci.org/
etc etc

[O the sweet pleasures of cloud-computing!]


From stefan_ml at behnel.de  Sun Jun 28 03:31:31 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 28 Jun 2015 09:31:31 +0200
Subject: HOPE: A Python just-in-time compiler for astrophysical
 computations
In-Reply-To: <558D8CC0.6000409@gmail.com>
References: <mailman.664.1434846609.13271.python-list@python.org>
 <55898f07$0$3088$426a74cc@news.free.fr> <558D8CC0.6000409@gmail.com>
Message-ID: <mmo7sk$d49$1@ger.gmane.org>

Michael Torrie schrieb am 26.06.2015 um 19:32:
> I've never heard of pythran; I'll have to check it out and see how it
> compares to the ever-growing crop of Python dialect compilers.

My feeling is that Python seems such a simple language at the surface that
people who want to write a special purpose "Python subset" compiler prefer
starting from scratch, rather than contributing to the existing tools. It
takes a while until they understand the actual size of that undertaking and
that's the point where most of these projects just die.

I don't mean all of them. If you have enough time and/or money, you can
certainly get a project going that's relevant enough for a critical
(special purpose) user base to provide an actual benefit. But then, why
invest that time into something completely new that requires major
long-term maintenance efforts, when implementing the desired feature in an
existing compiler would be a one-time investment with a much smaller
overall impact on further maintenance costs?

Not Invented Here Syndrome, I guess...

Stefan




From stefan_ml at behnel.de  Sun Jun 28 03:46:36 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 28 Jun 2015 09:46:36 +0200
Subject: enumerate XML tags (keys that will become headers) along with
 text (values) and write to CSV in one row (as opposed to "stacked" values
 with one header)
In-Reply-To: <mmivtd$fqa$1@dont-email.me>
References: <14aeae7a-41ab-4619-8331-7995e2420e54@googlegroups.com>
 <mmivtd$fqa$1@dont-email.me>
Message-ID: <mmo8ot$rku$1@ger.gmane.org>

Denis McMahon schrieb am 26.06.2015 um 09:44:
> xml data is an unordered list, and are trying to assign an order to it.
> 
> If the xml data was ordered, either each tag would be different, or each 
> tag would have an attribute specifying a sequence number.

XML is not unordered. The document order is well defined and entirely
obvious from the data. Whether this order is relevant and has a meaning or
not is, however, not part of XML itself but is left to the semantics of the
specific document format at hand. Meaning, XML document formats can choose
to ignore that order and define it as irrelevant. That doesn't mean it's
not there for a given document, but it may mean that a re-transmission of
the same document would be allowed to use a different order without
changing the information.

This property applies to pretty much all structured data formats and not
just XML, by the way, also to CSV and other tabular formats.

Stefan




From arsh840 at gmail.com  Sun Jun 28 08:18:09 2015
From: arsh840 at gmail.com (Great Avenger Singh)
Date: Sun, 28 Jun 2015 05:18:09 -0700 (PDT)
Subject: imaplib Thread fetching
Message-ID: <1bd59b19-b3bd-4f9e-835d-116762a926a8@googlegroups.com>

Hello Everyone,

I reached this place by searching about Python and Imaplib. After reading few conversations I found this place quite colourful covering/discussing wide range of things.

I am using Imaplib for Gmail Analysis program.

Kind of results I need is as follows:

https://drive.google.com/file/d/0B8zS1kI3rfhUcU81cVlOOWYzUWM/view?usp=sharing

Formulation to get above results is as follows. ( I think so ;) ) 

1. Fetch Gmail Conversations/threads

2. Parse messages based on intervals(5 mins, 15 mins and so on ) [Actually right now I am not sure how it will happen but I am completely believing that "Python is amazing for parsing Horrible things"]
   
3. Parse Each thread to get number of Emails reached and sent. (after the threads will be classified as intervals)

So here my question is Can I get any working example of code which can show me How to fetch Gmail Conversations using python+imaplib?





From jon+usenet at unequivocal.co.uk  Sun Jun 28 09:41:59 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sun, 28 Jun 2015 13:41:59 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mailman.111.1435349412.3674.python-list@python.org>
 <558e1ac6$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.121.1435384091.3674.python-list@python.org>
 <558e610f$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.139.1435417817.3674.python-list@python.org>
 <558eded8$0$1642$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <slrnmovufq.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-27, Steven D'Aprano <steve at pearwood.info> wrote:
> Despite his initial claim that he doesn't want to use AES because it's too
> slow implemented as pure Python, Randall has said that the application will
> offer AES encryption as an option. (He says it is enabled by default,
> except that the user can turn it off.) So the code is already there, all he
> has to do is call it.

You're still not listening to what he's saying. Everything you have
said in the above paragraph is false. He said he is using AES
encryption in the client, but that the server does not have the
processing power to do so (nor does it need to). He has not said
that the user "can turn it off", he's just acknowledging the fact
that since the user controls their own computer, they can rewrite
the client code to do whatever they want, and there's nothing he
can do to stop them.

> The choice ought to be a no-brainer. The fact that folks are seriously
> considering using something barely one step up from a medieval substitution
> cipher in 2015 for something with real security consequences if it is
> broken goes to show what a lousy job the IT industry does for security.

The fact that you think that is happening when it isn't shows what
a lousy job you have been doing of following the thread.


From jon+usenet at unequivocal.co.uk  Sun Jun 28 10:21:49 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sun, 28 Jun 2015 14:21:49 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
Message-ID: <slrnmp00qf.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-27, Randall Smith <randall at tnr.cc> wrote:
> Thankyou.  Nice points. I do think given the risks (there are always 
> risks) discussed, a successful attack of this nature is not very likely. 
>   Worse case, something that looks like this would land on the disk.
>
> crc32 checksum + translation table + malware
>
> with a generated base64 name and no extension.

I'm not sure why you're bothering with the checksum, it doesn't seem
to me that it buys you anything. Personally I'd do something like
this (pseudocode):

  def obfuscate(data):
      encode_key = list(range(256))
      random.shuffle(encode_key)
      encode_key = bytes(encode_key)
      decode_key = bytes(encode_key.index(i) for i in range(256))
      return decode_key + data.translate(encode_key) + decode_key

  def deobfuscate(data):
      return data[256:-256].translate(data[:256])

The reason for appending the key as well as prepending it is that some
anti-virus or malware scanners may well look at the last part of the
file first, so putting something entirely locally-generated there may
add a bit of safety. You could also simply pad with nulls or something
of course, but again I can imagine some tools skipping backwards past
nulls.


From steve at pearwood.info  Sun Jun 28 14:55:37 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 29 Jun 2015 04:55:37 +1000
Subject: Readline -- cannot bind to both Ctrl-tab and tab at the same time?
Message-ID: <55904328$0$1636$c3e8da3$5496439d@news.astraweb.com>

I'm trying to use a readline binding for both TAB and Ctrl-TAB, but it's not
working for me. Whichever I install second seems to over-ride the first.

In the Python interactive interpreter under Linux, enter these two lines:


import readline
readline.parse_and_bind('Control-tab: "import"')


Then on the next line, press Ctrl-TAB and readline should insert the
word "import". You don't need to import anything, that's just to prove that
the binding works.

Now enter:


readline.parse_and_bind('tab: "raise"')


and on the next line, press TAB and readline should insert the word "raise".
Again, no need to raise anything.

Try Ctrl-TAB again, and you'll get "raise" instead of "import".

Can anyone else replicate this issue?

Is this a Python issue, a problem with the terminal I am using, or readline
in general?

I have also tried with the alternate syntax:


readline.parse_and_bind(r'"\C-\t": "import"')


but it just silently fails to install the binding at all.

Can anyone else successfully bind two different functions to TAB and
Ctrl-TAB?


-- 
Steven



From denismfmcmahon at gmail.com  Sun Jun 28 17:00:41 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sun, 28 Jun 2015 21:00:41 +0000 (UTC)
Subject: enumerate XML tags (keys that will become headers) along with
 text (values) and write to CSV in one row (as opposed to "stacked" values
 with one header)
References: <14aeae7a-41ab-4619-8331-7995e2420e54@googlegroups.com>
 <mmivtd$fqa$1@dont-email.me>
 <mailman.150.1435477605.3674.python-list@python.org>
Message-ID: <mmpn9p$667$1@dont-email.me>

On Sun, 28 Jun 2015 09:46:36 +0200, Stefan Behnel wrote:

> Denis McMahon schrieb am 26.06.2015 um 09:44:
>> xml data is an unordered list, and are trying to assign an order to it.
>> 
>> If the xml data was ordered, either each tag would be different, or
>> each tag would have an attribute specifying a sequence number.
> 
> XML is not unordered. The document order is well defined and entirely
> obvious from the data. Whether this order is relevant and has a meaning
> or not is, however, not part of XML itself but is left to the semantics
> of the specific document format at hand. Meaning, XML document formats
> can choose to ignore that order and define it as irrelevant. That
> doesn't mean it's not there for a given document, but it may mean that a
> re-transmission of the same document would be allowed to use a different
> order without changing the information.
> 
> This property applies to pretty much all structured data formats and not
> just XML, by the way, also to CSV and other tabular formats.

The point I am trying to make to OP is that the following two XML 
fragments define the same data:

<things>
  <thing>string 1</thing>
  <thing>string 2</thing>
  <thing>string 3</thing>
</things>

and:

<things>
  <thing>string 3</thing>
  <thing>string 2</thing>
  <thing>string 1</thing>
</things>

Each <thing> is just a member of the collection things, the xml does not 
contain sufficient information to state that <things> is an ordered 
collection containing a specific sequence of <thing>.

Mechanisms such as node.firstChild and node.getChild(x) are all very well 
for manipulating the xml, but any specific ordering of the original data 
should be carried out by using an appropriate attribute of the ordered 
data elements at the point where the xml representation is created.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From lac at openend.se  Sun Jun 28 17:22:19 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 28 Jun 2015 23:22:19 +0200
Subject: Readline -- cannot bind to both Ctrl-tab and tab at the same time?
In-Reply-To: Message from "Steven D'Aprano" <steve@pearwood.info> of "Mon,
 29 Jun 2015 04:55:37 +1000."
 <55904328$0$1636$c3e8da3$5496439d@news.astraweb.com>
References: <55904328$0$1636$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <201506282122.t5SLMJY6010169@fido.openend.se>

I can reproduce this with Python 2.7.9 (default, Mar  1 2015, 12:57:24)
on my debian unstable system.

On pypy both forms just silently fail to install the binding at all.
Since PyPy has its own readline, this is circumstantial evidence that
readline is the problem.  It's not just you.

Laura



From ned at nedbatchelder.com  Sun Jun 28 20:07:00 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sun, 28 Jun 2015 17:07:00 -0700 (PDT)
Subject: enumerate XML tags (keys that will become headers) along with
 text (values) and write to CSV in one row (as opposed to "stacked" values
 with one header)
In-Reply-To: <mmpn9p$667$1@dont-email.me>
References: <14aeae7a-41ab-4619-8331-7995e2420e54@googlegroups.com>
 <mmivtd$fqa$1@dont-email.me>
 <mailman.150.1435477605.3674.python-list@python.org>
 <mmpn9p$667$1@dont-email.me>
Message-ID: <8e7d3dd8-f7c1-4466-ae82-f25e4a252fc7@googlegroups.com>

On Sunday, June 28, 2015 at 5:02:19 PM UTC-4, Denis McMahon wrote:
> On Sun, 28 Jun 2015 09:46:36 +0200, Stefan Behnel wrote:
> 
> > Denis McMahon schrieb am 26.06.2015 um 09:44:
> >> xml data is an unordered list, and are trying to assign an order to it.
> >> 
> >> If the xml data was ordered, either each tag would be different, or
> >> each tag would have an attribute specifying a sequence number.
> > 
> > XML is not unordered. The document order is well defined and entirely
> > obvious from the data. Whether this order is relevant and has a meaning
> > or not is, however, not part of XML itself but is left to the semantics
> > of the specific document format at hand. Meaning, XML document formats
> > can choose to ignore that order and define it as irrelevant. That
> > doesn't mean it's not there for a given document, but it may mean that a
> > re-transmission of the same document would be allowed to use a different
> > order without changing the information.
> > 
> > This property applies to pretty much all structured data formats and not
> > just XML, by the way, also to CSV and other tabular formats.
> 
> The point I am trying to make to OP is that the following two XML 
> fragments define the same data:
> 
> <things>
>   <thing>string 1</thing>
>   <thing>string 2</thing>
>   <thing>string 3</thing>
> </things>
> 
> and:
> 
> <things>
>   <thing>string 3</thing>
>   <thing>string 2</thing>
>   <thing>string 1</thing>
> </things>
> 
> Each <thing> is just a member of the collection things, the xml does not 
> contain sufficient information to state that <things> is an ordered 
> collection containing a specific sequence of <thing>.

You are right that XML does not specify that <things> is an ordered collection.
But XML does preserve the order of the children.  There are many XML schema
that rely on XML's order-preserving nature.

> 
> Mechanisms such as node.firstChild and node.getChild(x) are all very well 
> for manipulating the xml, but any specific ordering of the original data 
> should be carried out by using an appropriate attribute of the ordered 
> data elements at the point where the xml representation is created.
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com



From random832 at fastmail.us  Sun Jun 28 20:14:16 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Sun, 28 Jun 2015 20:14:16 -0400
Subject: Readline -- cannot bind to both Ctrl-tab and tab at the same time?
Message-ID: <1435536856.310694.310028969.2624B059@webmail.messagingengine.com>

On Sun, Jun 28, 2015, at 14:55, Steven D'Aprano wrote:
> I'm trying to use a readline binding for both TAB and Ctrl-TAB, but it's
> not
> working for me. Whichever I install second seems to over-ride the first.

Can you bind it directly to whatever escape sequence ctrl-tab produces?
Some terminals may not be able to generate an escape sequence for
ctrl-tab, or may require a special option to be enabled (e.g.
modifyOtherKeys for xterm).


From breamoreboy at yahoo.co.uk  Sun Jun 28 21:14:43 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 29 Jun 2015 02:14:43 +0100
Subject: How to debug TypeError: required field "lineno" missing from expr?
Message-ID: <mmq666$1e7$1@ger.gmane.org>

Purely as an exercise I've been converting Grant Jenks' pypatt[1] from 
2.7 to 3.4.  I've managed to sort out most of the required changes by 
checking on what I can see with an AST pretty printer[2].  So it's 
rather frustrating to have the compile stage throw the error given in 
the subject line.

The code has a call to the ast fix_missing_locations function.  I'm 
aware of the Armin Ronacher tweet[3] stating

<quote>
"TypeError: required field "lineno" missing from stmt" ? no, what you 
actually mean is "tuple is not a statement'.
</quote>

but I don't think this is the problem.  Still I've clearly managed to 
screw something up somewhere along so line, so any bright ideas as to 
how I can proceed?

Further would it be worth raising an enhancement request to get some 
better diagnostics from the built-in compile function, or is such a 
thing already on the bug tracker?  Or is it simply too difficult, or what?

No panic needed, it's past 2am BST and I'm off to bed for my beauty 
sleep, so give me a couple of months :)

[1] https://github.com/grantjenks/pypatt_python_pattern_matching
[2] http://alexleone.blogspot.co.uk/2010/01/python-ast-pretty-printer.html
[3] https://twitter.com/mitsuhiko/status/91169383254200320

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Sun Jun 28 21:36:25 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 29 Jun 2015 11:36:25 +1000
Subject: Readline -- cannot bind to both Ctrl-tab and tab at the same time?
In-Reply-To: <55904328$0$1636$c3e8da3$5496439d@news.astraweb.com>
References: <55904328$0$1636$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmrHW6zyOqQYutjZNdq_7X5r+fZzSiFqJKJW9J7oas+cPQ@mail.gmail.com>

On Mon, Jun 29, 2015 at 4:55 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Try Ctrl-TAB again, and you'll get "raise" instead of "import".
>
> Can anyone else replicate this issue?
>
> Is this a Python issue, a problem with the terminal I am using, or readline
> in general?

Confirmed with Python 3.6 on Debian Jessie. Delving into Laura's
suggestion that it's a readline problem, I came across this:

http://stackoverflow.com/questions/12044574/getting-complete-and-menu-complete-to-work-together

Testing with "showkey -a" suggests that tab and ctrl-tab indeed send
the same bytes. I wonder if there's a way to change terminal type or
fiddle with terminfo to change this? GUI programs obviously don't have
this conflation.

ChrisA


From tjreedy at udel.edu  Sun Jun 28 22:20:48 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 28 Jun 2015 22:20:48 -0400
Subject: How to debug TypeError: required field "lineno" missing from expr?
In-Reply-To: <mmq666$1e7$1@ger.gmane.org>
References: <mmq666$1e7$1@ger.gmane.org>
Message-ID: <mmqa23$lch$1@ger.gmane.org>

On 6/28/2015 9:14 PM, Mark Lawrence wrote:
> Purely as an exercise I've been converting Grant Jenks' pypatt[1] from
> 2.7 to 3.4.

With 2to3 or by hand?

>  I've managed to sort out most of the required changes by
> checking on what I can see with an AST pretty printer[2].  So it's
> rather frustrating to have the compile stage throw the error given in
> the subject line.
>
> The code has a call to the ast fix_missing_locations function.

ast.fix_missing_locations does not have a raise statement

>  I'm aware of the Armin Ronacher tweet[3] stating
>
> <quote>
> "TypeError: required field "lineno" missing from stmt" ? no, what you
> actually mean is "tuple is not a statement'.
> </quote>

As near as I can tell, it is impossible for lineno to be missing from a 
call to the inner function _fix.  Grepping for "TypeError: required 
field" in 3.4 .c and .py files (with Idle) does not return any hits.

astpp.py does not raise that particular error either.

A random tweet without context or example is not terribly helpful.

> but I don't think this is the problem.  Still I've clearly managed to
> screw something up somewhere along so line, so any bright ideas as to
> how I can proceed?

How many times have you asked people asking such questions to post both 
the complete traceback and the code.  Where does the traceback say the 
exception is from?  The What is the code that compiled with 2.7 and the 
(modified?) code that does not compile.  What is a minimal example that 
exhibits the problem?

-- 
Terry Jan Reedy




From steve at pearwood.info  Sun Jun 28 22:44:13 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 29 Jun 2015 12:44:13 +1000
Subject: How to debug TypeError: required field "lineno" missing from expr?
References: <mailman.153.1435540503.3674.python-list@python.org>
Message-ID: <5590b0fd$0$1642$c3e8da3$5496439d@news.astraweb.com>

On Mon, 29 Jun 2015 11:14 am, Mark Lawrence wrote:

> Purely as an exercise I've been converting Grant Jenks' pypatt[1] from
> 2.7 to 3.4.  I've managed to sort out most of the required changes by
> checking on what I can see with an AST pretty printer[2].  So it's
> rather frustrating to have the compile stage throw the error given in
> the subject line.

Now now Mark, you know how this works. Where's your COPY and PASTED
traceback? Where's the Short, Self Contained, Correct Example?

http://sscce.org/

Our crystal ball is in no better state than yours :-)


> The code has a call to the ast fix_missing_locations function.  I'm
> aware of the Armin Ronacher tweet[3] stating
> 
> <quote>
> "TypeError: required field "lineno" missing from stmt" ? no, what you
> actually mean is "tuple is not a statement'.
> </quote>
> 
> but I don't think this is the problem.

That's what I dislike about Twitter. There's no context, and no way to tell
what Armin is talking about. I've never seen this error, and cannot imagine
how to get it, or what he means by "tuple is not a statement" for that
matter. A tuple can be a statement:

if condition:
    (1, 2)  # Construct a tuple, then throw it away.


> Still I've clearly managed to 
> screw something up somewhere along so line, so any bright ideas as to
> how I can proceed?
> 
> Further would it be worth raising an enhancement request to get some
> better diagnostics from the built-in compile function, 

If you think it is worth waiting months or years before you can actually use
those enhanced diagnostics, sure. What sort of diagnostics?

Peering into my crystal ball, I see that you're maybe generating some code
dynamically, then running something like:

    code = compile(the_text, something, something)

and that's giving you an error. What happens if you do this?

    print(the_text)
    code = compile(the_text, something, something)


Can you extract the offending line that way? What happens if you copy and
paste the_text into a Python file, and then try to import or run that file? 




-- 
Steven



From lac at openend.se  Mon Jun 29 03:08:53 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 29 Jun 2015 09:08:53 +0200
Subject: How to debug TypeError: required field "lineno" missing from expr?
In-Reply-To: Message from Mark Lawrence <breamoreboy@yahoo.co.uk>
 of "Mon, 29 Jun 2015 02:14:43 +0100." <mmq666$1e7$1@ger.gmane.org>
References: <mmq666$1e7$1@ger.gmane.org>
Message-ID: <201506290708.t5T78rhr024655@fido.openend.se>

In a message of Mon, 29 Jun 2015 02:14:43 +0100, Mark Lawrence writes:
>Purely as an exercise I've been converting Grant Jenks' pypatt[1] from 
>2.7 to 3.4.  I've managed to sort out most of the required changes by 
>checking on what I can see with an AST pretty printer[2].  So it's 
>rather frustrating to have the compile stage throw the error given in 
>the subject line.

Are you trying to use ast.copy_location recursively somewhere?
http://bugs.python.org/issue3530  Maybe you have found something else
that doesn't work recursively?

Laura



From tjreedy at udel.edu  Mon Jun 29 07:17:24 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 29 Jun 2015 07:17:24 -0400
Subject: How to debug TypeError: required field "lineno" missing from expr?
In-Reply-To: <201506290708.t5T78rhr024655@fido.openend.se>
References: <mmq666$1e7$1@ger.gmane.org>
 <201506290708.t5T78rhr024655@fido.openend.se>
Message-ID: <mmr9g5$66h$1@ger.gmane.org>

On 6/29/2015 3:08 AM, Laura Creighton wrote:
> In a message of Mon, 29 Jun 2015 02:14:43 +0100, Mark Lawrence writes:
>> Purely as an exercise I've been converting Grant Jenks' pypatt[1] from
>> 2.7 to 3.4.  I've managed to sort out most of the required changes by
>> checking on what I can see with an AST pretty printer[2].  So it's
>> rather frustrating to have the compile stage throw the error given in
>> the subject line.
>
> Are you trying to use ast.copy_location recursively somewhere?
> http://bugs.python.org/issue3530   Maybe you have found something else
> that doesn't work recursively?

I do not understand the closing of this issue given that a) the failing 
part of the code is copied from the doc (3.4 has the same in the 
docstring) and b) that there is no recursion for a single node ast, 
which the example should be.

The error message comes from
F:\Python\dev\34\Python\Python-ast.c: 3752:
PyErr_SetString(PyExc_TypeError, "required field \"lineno\" missing from 
stmt");

-- 
Terry Jan Reedy



From robert.kern at gmail.com  Mon Jun 29 09:04:45 2015
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 29 Jun 2015 14:04:45 +0100
Subject: enumerate XML tags (keys that will become headers) along with
 text (values) and write to CSV in one row (as opposed to "stacked" values
 with one header)
In-Reply-To: <mmpn9p$667$1@dont-email.me>
References: <14aeae7a-41ab-4619-8331-7995e2420e54@googlegroups.com>
 <mmivtd$fqa$1@dont-email.me>
 <mailman.150.1435477605.3674.python-list@python.org>
 <mmpn9p$667$1@dont-email.me>
Message-ID: <mmrfpd$gru$1@ger.gmane.org>

On 2015-06-28 22:00, Denis McMahon wrote:
> On Sun, 28 Jun 2015 09:46:36 +0200, Stefan Behnel wrote:
>
>> Denis McMahon schrieb am 26.06.2015 um 09:44:
>>> xml data is an unordered list, and are trying to assign an order to it.
>>>
>>> If the xml data was ordered, either each tag would be different, or
>>> each tag would have an attribute specifying a sequence number.
>>
>> XML is not unordered. The document order is well defined and entirely
>> obvious from the data. Whether this order is relevant and has a meaning
>> or not is, however, not part of XML itself but is left to the semantics
>> of the specific document format at hand. Meaning, XML document formats
>> can choose to ignore that order and define it as irrelevant. That
>> doesn't mean it's not there for a given document, but it may mean that a
>> re-transmission of the same document would be allowed to use a different
>> order without changing the information.
>>
>> This property applies to pretty much all structured data formats and not
>> just XML, by the way, also to CSV and other tabular formats.
>
> The point I am trying to make to OP is that the following two XML
> fragments define the same data:
>
> <things>
>    <thing>string 1</thing>
>    <thing>string 2</thing>
>    <thing>string 3</thing>
> </things>
>
> and:
>
> <things>
>    <thing>string 3</thing>
>    <thing>string 2</thing>
>    <thing>string 1</thing>
> </things>
>
> Each <thing> is just a member of the collection things, the xml does not
> contain sufficient information to state that <things> is an ordered
> collection containing a specific sequence of <thing>.

Without reference to a schema that explicitly defines the children of <things> 
to be unordered, this is not true. The XML Information Set defines the children 
of all elements to be an ordered set (in contrast to attributes, which are 
unordered). A particular XML schema may choose to consider the order of children 
of a particular element to be irrelevant, but XML itself keeps them ordered.

http://www.w3.org/TR/xml-infoset/#infoitem.element

If it didn't, then XHTML would have a hell of a time with ordered constructs 
like this:

   <ol>
     <li>First item</li>
     <li>Second item</li>
     <li>Third item</li>
   </ol>

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From random832 at fastmail.us  Mon Jun 29 10:03:24 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Mon, 29 Jun 2015 10:03:24 -0400
Subject: Recreate file associations?
Message-ID: <1435586604.265153.310526457.172CA608@webmail.messagingengine.com>

I installed Python 2.1 for historical interest and it associated python
files with the python 2.1 interpreter. I installed 3.4 afterwards (I had
3.3 as my latest python version before) expecting this to fix it, but it
left it alone. Even deleting all the Python associations from the
registry and then repairing 3.4, it didn't create them. How do I get it
to make the file associations the same as the first time I installed
Python 3?


From sahluwalia at wynyardgroup.com  Mon Jun 29 10:52:07 2015
From: sahluwalia at wynyardgroup.com (Sahlusar)
Date: Mon, 29 Jun 2015 07:52:07 -0700 (PDT)
Subject: enumerate XML tags (keys that will become headers) along with
 text (values) and write to CSV in one row (as opposed to "stacked" values
 with one header)
In-Reply-To: <mailman.150.1435477605.3674.python-list@python.org>
References: <14aeae7a-41ab-4619-8331-7995e2420e54@googlegroups.com>
 <mmivtd$fqa$1@dont-email.me>
 <mailman.150.1435477605.3674.python-list@python.org>
Message-ID: <a27fd45c-0579-44c0-8eaa-25214fbbfdb1@googlegroups.com>

On Sunday, 28 June 2015 03:46:56 UTC-4, Stefan Behnel  wrote:
> Denis McMahon schrieb am 26.06.2015 um 09:44:
> > xml data is an unordered list, and are trying to assign an order to it.
> > 
> > If the xml data was ordered, either each tag would be different, or each 
> > tag would have an attribute specifying a sequence number.
> 
> XML is not unordered. The document order is well defined and entirely
> obvious from the data. Whether this order is relevant and has a meaning or
> not is, however, not part of XML itself but is left to the semantics of the
> specific document format at hand. Meaning, XML document formats can choose
> to ignore that order and define it as irrelevant. That doesn't mean it's
> not there for a given document, but it may mean that a re-transmission of
> the same document would be allowed to use a different order without
> changing the information.
> 
> This property applies to pretty much all structured data formats and not
> just XML, by the way, also to CSV and other tabular formats.
> 
> Stefan

@Stefan, Ned, and Robert: You have all hit the nail on the head. I do not have an authentic and veritable XSD (or XML data structures for that matter). So far it is all deprecated and/or anonymized data from the client. Therefore, I can only hypothesize what the end output will be for the database architecture that I am working with. 

>From what I understand, therefore, based on your constructive insight, is that the 14 occurrences of the same tag (regardless of placement relative to neighbouring children and the root are all being defined as the same key. However, their individual values are also being treated as the same (from the algorithm that I wrote in my Stack Overflow post (please see above)). The constraint is that I am anticipating terabytes of data every day from the client in the coming months. The algorithm should be able to parse, and write out to CSV in the most efficient manner. That is my design constraint. I welcome your feedback on this. 

Here is the post, again, for your convenience: 

http://stackoverflow.com/questions/31058100/enumerate-column-headers-in-csv-that-belong-to-the-same-tag-key-in-python 




From tjreedy at udel.edu  Mon Jun 29 14:25:11 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 29 Jun 2015 14:25:11 -0400
Subject: Recreate file associations?
In-Reply-To: <1435586604.265153.310526457.172CA608@webmail.messagingengine.com>
References: <1435586604.265153.310526457.172CA608@webmail.messagingengine.com>
Message-ID: <mms2i8$hhs$1@ger.gmane.org>

On 6/29/2015 10:03 AM, random832 at fastmail.us wrote:
> I installed Python 2.1 for historical interest and it associated python
> files with the python 2.1 interpreter. I installed 3.4 afterwards (I had
> 3.3 as my latest python version before) expecting this to fix it, but it
> left it alone. Even deleting all the Python associations from the
> registry and then repairing 3.4, it didn't create them. How do I get it
> to make the file associations the same as the first time I installed
> Python 3?

Registry => Windows.
Did you check [X] Make this my default python?

-- 
Terry Jan Reedy



From dchokshi at cisco.com  Mon Jun 29 16:06:05 2015
From: dchokshi at cisco.com (Devaki Chokshi (dchokshi))
Date: Mon, 29 Jun 2015 20:06:05 +0000
Subject: Scapy and MGCP
Message-ID: <E0FE14A926805D41A32BEA79FD6E7AD626469C43@xmb-rcd-x10.cisco.com>

Hello,

As per reply received I have begun to use scapy for MGCP. 

I started off with  reading a .pcap file with MGCP packets. 

For example:

from scapy.all import *
from scapy.utils import *
from scapy.layers.mgcp import *

mgcp_pkts = rdpcap("my-mgcp-capture.pcap")

However, rdpcap() is returning all MGCP packets as UDP packets. The output is something like:

<my-mgcp-capture.pcap: TCP:33 UDP:262 ICMP:1 Other:0>

All 262 UDP packets are actually MGCP packets. 

I would have thought rdpcap() will understand MGCP packet format. 

Would appreciate help on how I can make scapy read MGCP packets from a .pcap file.

Thank you
Devaki

-----Original Message-----
From: Laura Creighton [mailto:lac at openend.se] 
Sent: Tuesday, June 23, 2015 7:47 AM
To: Devaki Chokshi (dchokshi)
Cc: python-list at python.org; shtoom at python.org; lac at openend.se
Subject: Re: MGCP Server implementation

In a message of Mon, 22 Jun 2015 20:42:21 -0000, "Devaki Chokshi (dchokshi)" wr
ites:
>Hello,
>
>I have a use case where a SIP voice call will be passing through an MGCP gateway.
>
>Is there a python implementation that simulates MGCP gateway/server?
>
>Thank you
>Devaki Chokshi
>
>-- 
>https://mail.python.org/mailman/listinfo/python-list
>

I think http://www.secdev.org/projects/scapy/ will do this for you.
In particular
https://code.google.com/p/pythonxy/source/browse/src/python/_scapy/PURELIB/scapy/layers/mgcp.py?repo=xy-27&name=v2.7.2.1&r=c4a08530c8ff74fd6078507743ce9982675aea8c

But I have only used Sacpy for other protocols, so I am not certain.

Laura


From lac at openend.se  Mon Jun 29 16:19:12 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 29 Jun 2015 22:19:12 +0200
Subject: Scapy and MGCP
In-Reply-To: Message from "Devaki Chokshi (dchokshi)" <dchokshi@cisco.com>
 of "Mon, 29 Jun 2015 20:06:05 -0000."
 <E0FE14A926805D41A32BEA79FD6E7AD626469C43@xmb-rcd-x10.cisco.com>
References: <E0FE14A926805D41A32BEA79FD6E7AD626469C43@xmb-rcd-x10.cisco.com>
Message-ID: <201506292019.t5TKJCk3011768@fido.openend.se>

In a message of Mon, 29 Jun 2015 20:06:05 -0000, "Devaki Chokshi (dchokshi)" wr
ites:
>Hello,
>
>As per reply received I have begun to use scapy for MGCP. 
>
>I started off with  reading a .pcap file with MGCP packets. 
>
>For example:
>
>from scapy.all import *
>from scapy.utils import *
>from scapy.layers.mgcp import *
>
>mgcp_pkts = rdpcap("my-mgcp-capture.pcap")
>
>However, rdpcap() is returning all MGCP packets as UDP packets. The output is something like:
>
><my-mgcp-capture.pcap: TCP:33 UDP:262 ICMP:1 Other:0>
>
>All 262 UDP packets are actually MGCP packets. 
>
>I would have thought rdpcap() will understand MGCP packet format. 
>
>Would appreciate help on how I can make scapy read MGCP packets from a .pcap file.
>
>Thank you
>Devaki

Ask on the secdev mailing list.
see: https://bitbucket.org/secdev/scapy/wiki/doc/index

This 'everything is UDP' is normal, but I forget how to turn it off
if you don't want that, and it's been over 3 years so there may be
better ways than what I used .... plus for me it wasn't MGCP packets
anyhow.  At any rate  they will know over there.

Laura


From florian.ruechel.pythonlist at inexplicity.de  Mon Jun 29 16:43:19 2015
From: florian.ruechel.pythonlist at inexplicity.de (=?UTF-8?Q?Florian_R=c3=bcchel?=)
Date: Mon, 29 Jun 2015 22:43:19 +0200
Subject: Segmentation Fault with multiple interpreters
Message-ID: <5591ADE7.4000606@inexplicity.de>

Hey there,

I'm not sure this is the correct list considering the level of internal
Python knowledge it likely requires. If I should take this to another
list, please let me know.

I have written an application that links against libpython and starts
multiple interpreters within one thread. Beyond that it actually links
against libpython2.7 *and* libpython3.4 (though I doubt that makes a
difference here). In addition to launching my own interpreters I created
a custom C module that can be called from Python (and will be loaded
during my own init function).

The problem is a bit weird and not easy to reproduce. There are two
types of behaviors launched, both running the same C code but diverting
during interpreter runtime. The problem happens only if I start 8
instances of type A and at least one of type B. All instances of type A
start before any instance of type B starts. On the first start of tpye B
the program segfaults. Decreasing the number to 7 yields normal execution.

I could now go along and post a lot of code here and explain the complex
setup but I think it is much more helpful if I can get hints on how to
debug this myself. An excerpt of the traceback (full can be found here:
http://pastebin.com/unAaG6qi)

#0  0x00007f922096cd6a in visit_decref (op=0x7f921d801518
<__hoisted_globals+216>, data=0x0) at Modules/gcmodule.c:360
#1  0x00007f92208aadf2 in meth_traverse (m=0x7f921d4f0d40,
visit=0x7f922096cd52 <visit_decref>, arg=0x0) at Objects/methodobject.c:166
#2  0x00007f922096ce2c in subtract_refs (containers=0x7f9220c108e0
<generations+96>) at Modules/gcmodule.c:385
#3  0x00007f922096dac9 in collect (generation=0x2) at Modules/gcmodule.c:925
#4  0x00007f922096de0a in collect_generations () at Modules/gcmodule.c:1050
#5  0x00007f922096e939 in _PyObject_GC_Malloc (basicsize=0x20) at
Modules/gcmodule.c:1511
#6  0x00007f922096e9d9 in _PyObject_GC_NewVar (tp=0x7f9220bf1540
<PyTuple_Type>, nitems=0x1) at Modules/gcmodule.c:1531
#7  0x00007f92208c7681 in PyTuple_New (size=0x1) at Objects/tupleobject.c:90
...
#58 0x00007f9220953ff7 in initsite () at Python/pythonrun.c:726
#59 0x00007f9220953d94 in Py_NewInterpreter () at Python/pythonrun.c:621
#60 0x00007f921d60024b in prepare_interpreter (argc=0x9, argv=0x387ee70,
m=0x3861e80) at
/home/javex/Thesis/src/shadow-plugin-extras/python/src/python.c:19
#61 0x00007f921d5ff925 in python_new (argc=0x9, argv=0x387ee70,
log=0x421420 <_thread_interface_log>) at
/home/javex/Thesis/src/shadow-plugin-extras/python/src/python.c:160

with the exact point of segfault being this:

355     /* A traversal callback for subtract_refs. */
356     static int
357     visit_decref(PyObject *op, void *data)
358     {
359         assert(op != NULL);
360         if (PyObject_IS_GC(op)) { <---- segfault here
361             PyGC_Head *gc = AS_GC(op);
362             /* We're only interested in gc_refs for objects in the
363              * generation being collected, which can be recognized
364              * because only they have positive gc_refs.

With "op" being:
gdb$ print *op
$4 = {ob_refcnt = 0x1, ob_type = 0x0}

Now I vaguely remember having seen this error before. It *might* have
been that I was passing back Py_None without increasing its reference
(but it later being decref'ed). That case is now solved and I don't
return Py_None at another place.

So the big question is: How do I find out what is happening here? From
what I can gather it looks like GC is cleaning up so I guess I have a
refcount wrong. But which? Where? Can I debug this?

If you think this might help in any case, my source code is also
available:
https://github.com/Javex/shadow-plugin-extras/tree/master/python/src
Here the files python-logging.c, python.c and python-plugin.c are in use
with python.c having most of the magic and python-logging.c being my own
module. Beyond that there's only python code, no more C of my own.

Thanks in advance for any help you can provide. Kind regards,
Florian


From randall at tnr.cc  Mon Jun 29 16:46:36 2015
From: randall at tnr.cc (Randall Smith)
Date: Mon, 29 Jun 2015 15:46:36 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <slrnmp00qf.1nu.jon+usenet@frosty.unequivocal.co.uk>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <slrnmp00qf.1nu.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <mmsarl$ptl$1@ger.gmane.org>

On 06/28/2015 09:21 AM, Jon Ribbens wrote:
> On 2015-06-27, Randall Smith <randall at tnr.cc> wrote:
>> Thankyou.  Nice points. I do think given the risks (there are always
>> risks) discussed, a successful attack of this nature is not very likely.
>>    Worse case, something that looks like this would land on the disk.
>>
>> crc32 checksum + translation table + malware
>>
>> with a generated base64 name and no extension.
>
> I'm not sure why you're bothering with the checksum, it doesn't seem
> to me that it buys you anything. Personally I'd do something like
> this (pseudocode):


Same reason newer filesystems like BTRFS use checkusms (BTRFS uses 
CRC32).  The storage machine runs periodic file integrity checks.  It 
has no control over the underlying filesystem.


>
>    def obfuscate(data):
>        encode_key = list(range(256))
>        random.shuffle(encode_key)
>        encode_key = bytes(encode_key)
>        decode_key = bytes(encode_key.index(i) for i in range(256))
>        return decode_key + data.translate(encode_key) + decode_key
>
>    def deobfuscate(data):
>        return data[256:-256].translate(data[:256])
>
> The reason for appending the key as well as prepending it is that some
> anti-virus or malware scanners may well look at the last part of the
> file first, so putting something entirely locally-generated there may
> add a bit of safety. You could also simply pad with nulls or something
> of course, but again I can imagine some tools skipping backwards past
> nulls.
>




From jon+usenet at unequivocal.co.uk  Mon Jun 29 16:49:04 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Mon, 29 Jun 2015 20:49:04 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <slrnmp00qf.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.170.1435610818.3674.python-list@python.org>
Message-ID: <slrnmp3bsk.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-29, Randall Smith <randall at tnr.cc> wrote:
> Same reason newer filesystems like BTRFS use checkusms (BTRFS uses 
> CRC32).  The storage machine runs periodic file integrity checks.  It 
> has no control over the underlying filesystem.

True, but presumably neither does it have anything it can do to
rectify the situation if it finds a problem, and the client will
have to keep its own secure hash of its file anyway. (Unless I suppose
the server actually can request a new copy from the client or another
server if it finds a corrupt file?)


From randall at tnr.cc  Mon Jun 29 16:52:37 2015
From: randall at tnr.cc (Randall Smith)
Date: Mon, 29 Jun 2015 15:52:37 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mmsb6u$ptl$2@ger.gmane.org>

On 06/27/2015 01:50 PM, Steven D'Aprano wrote:
> On Sun, 28 Jun 2015 03:08 am, Randall Smith wrote:
>
>> Though I didn't mention it in the description, the storage server is
>> appending a CRC32 checksum for routine integrity checks.  So by the time
>> the data hits the disk, it will have added both a 256 byte translation
>> table and a 4 byte checksum.
>
>
> http://stackoverflow.com/questions/1515914/crc32-collision
>
>
>
>


Not sure why you posted the link.  The crc32 checksum is just to check 
for possible filesystem corruption.  The system does periodic data 
corruption checks.  BTRFS uses crc32 checksums also.  Please explain.

-Randall



From tjreedy at udel.edu  Mon Jun 29 16:59:23 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 29 Jun 2015 16:59:23 -0400
Subject: How to debug TypeError: required field "lineno" missing from expr?
In-Reply-To: <mmr9g5$66h$1@ger.gmane.org>
References: <mmq666$1e7$1@ger.gmane.org>
 <201506290708.t5T78rhr024655@fido.openend.se> <mmr9g5$66h$1@ger.gmane.org>
Message-ID: <mmsbjc$6mc$1@ger.gmane.org>

On 6/29/2015 7:17 AM, Terry Reedy wrote:
> On 6/29/2015 3:08 AM, Laura Creighton wrote:
>> In a message of Mon, 29 Jun 2015 02:14:43 +0100, Mark Lawrence writes:
>>> Purely as an exercise I've been converting Grant Jenks' pypatt[1] from
>>> 2.7 to 3.4.  I've managed to sort out most of the required changes by
>>> checking on what I can see with an AST pretty printer[2].  So it's
>>> rather frustrating to have the compile stage throw the error given in
>>> the subject line.
>>
>> Are you trying to use ast.copy_location recursively somewhere?
>> http://bugs.python.org/issue3530   Maybe you have found something else
>> that doesn't work recursively?

The issue is that copy_location is not recursive and does not fix 
children of the node being fixed, whereas the example method, copied 
from the NodeTransformer doc, creates 3 new nodes, not just 1, that need 
fixing.  I reopened the issue as a doc issue to fix the example and 
recommend the use of recursive fix_missing_locations() instead of the 
non-recursive copy_location.

The only other location-related function in ast is increment_lineno, 
which also works on multiple nodes.  I would not be surprised is Mark's 
issue results from Mark or someone else copying the bad doc example.

> I do not understand the closing of this issue given that a) the failing
> part of the code is copied from the doc (3.4 has the same in the
> docstring) and b) that there is no recursion for a single node ast,
> which the example should be.

The original ast is actually an Expression with a Name.  But the issue 
is that the Name is replaced with a Subscript with Name and Str children 
and all three need locations added.

> The error message comes from
> F:\Python\dev\34\Python\Python-ast.c: 3752:
> PyErr_SetString(PyExc_TypeError, "required field \"lineno\" missing from
> stmt");

-- 
Terry Jan Reedy



From breamoreboy at yahoo.co.uk  Mon Jun 29 18:07:28 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 29 Jun 2015 23:07:28 +0100
Subject: How to debug TypeError: required field "lineno" missing from expr?
In-Reply-To: <mmsbjc$6mc$1@ger.gmane.org>
References: <mmq666$1e7$1@ger.gmane.org>
 <201506290708.t5T78rhr024655@fido.openend.se> <mmr9g5$66h$1@ger.gmane.org>
 <mmsbjc$6mc$1@ger.gmane.org>
Message-ID: <mmsfj4$5b5$1@ger.gmane.org>

On 29/06/2015 21:59, Terry Reedy wrote:
> On 6/29/2015 7:17 AM, Terry Reedy wrote:
>> On 6/29/2015 3:08 AM, Laura Creighton wrote:
>>> In a message of Mon, 29 Jun 2015 02:14:43 +0100, Mark Lawrence writes:
>>>> Purely as an exercise I've been converting Grant Jenks' pypatt[1] from
>>>> 2.7 to 3.4.  I've managed to sort out most of the required changes by
>>>> checking on what I can see with an AST pretty printer[2].  So it's
>>>> rather frustrating to have the compile stage throw the error given in
>>>> the subject line.
>>>
>>> Are you trying to use ast.copy_location recursively somewhere?
>>> http://bugs.python.org/issue3530   Maybe you have found something else
>>> that doesn't work recursively?
>
> The issue is that copy_location is not recursive and does not fix
> children of the node being fixed, whereas the example method, copied
> from the NodeTransformer doc, creates 3 new nodes, not just 1, that need
> fixing.  I reopened the issue as a doc issue to fix the example and
> recommend the use of recursive fix_missing_locations() instead of the
> non-recursive copy_location.
>
> The only other location-related function in ast is increment_lineno,
> which also works on multiple nodes.  I would not be surprised is Mark's
> issue results from Mark or someone else copying the bad doc example.
>

FTR there is no call to copy_location in Grant's existing code.  Whether 
or not one is actually needed in 3.4 when compared to 2.7 is another 
question.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From jonas.thornvall at gmail.com  Mon Jun 29 18:39:15 2015
From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com)
Date: Mon, 29 Jun 2015 15:39:15 -0700 (PDT)
Subject: Linear time baseconversion
Message-ID: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>

http://jt.node365.se/baseconversion8.html 


From ian.g.kelly at gmail.com  Mon Jun 29 18:56:39 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 29 Jun 2015 16:56:39 -0600
Subject: Linear time baseconversion
In-Reply-To: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
Message-ID: <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>

On Mon, Jun 29, 2015 at 4:39 PM,  <jonas.thornvall at gmail.com> wrote:
> http://jt.node365.se/baseconversion8.html

Back of the envelope mental calculation, that appears to be quadratic,
not linear. Doubling the length of the input results in an approximate
quadrupling of the time taken to produce the output.

That noted, this is off-topic for this list, which is for discussion
about Python. Please take this to somewhere else like
comp.lang.javascript instead.


From ian.g.kelly at gmail.com  Mon Jun 29 19:10:52 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 29 Jun 2015 17:10:52 -0600
Subject: Linear time baseconversion
In-Reply-To: <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
Message-ID: <CALwzidnHUSe3ofiiBZMRAxKckhk862KEVViehbCaAi-u7Hp6vw@mail.gmail.com>

On Mon, Jun 29, 2015 at 4:56 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Mon, Jun 29, 2015 at 4:39 PM,  <jonas.thornvall at gmail.com> wrote:
>> http://jt.node365.se/baseconversion8.html
>
> Back of the envelope mental calculation, that appears to be quadratic,
> not linear. Doubling the length of the input results in an approximate
> quadrupling of the time taken to produce the output.
>
> That noted, this is off-topic for this list, which is for discussion
> about Python. Please take this to somewhere else like
> comp.lang.javascript instead.

By the way, I think you have a bug. I did my time testing by
concatenating the default input number to itself several times. At
5184 decimal digits, which was the last case I tried, the 58th output
digit was 1111111, after which point the remaining 672 output digits
are all 12665464, which seemed rather improbable.


From jonas.thornvall at gmail.com  Mon Jun 29 19:23:00 2015
From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com)
Date: Mon, 29 Jun 2015 16:23:00 -0700 (PDT)
Subject: Linear time baseconversion
In-Reply-To: <mailman.175.1435619500.3674.python-list@python.org>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
Message-ID: <bcfd90d4-c302-4358-88ae-1fe9b4368078@googlegroups.com>

Den tisdag 30 juni 2015 kl. 01:11:51 UTC+2 skrev Ian:
> On Mon, Jun 29, 2015 at 4:56 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> > On Mon, Jun 29, 2015 at 4:39 PM,  <jonas.thornvall at gmail.com> wrote:
> >> http://jt.node365.se/baseconversion8.html
> >
> > Back of the envelope mental calculation, that appears to be quadratic,
> > not linear. Doubling the length of the input results in an approximate
> > quadrupling of the time taken to produce the output.
> >
> > That noted, this is off-topic for this list, which is for discussion
> > about Python. Please take this to somewhere else like
> > comp.lang.javascript instead.
> 
> By the way, I think you have a bug. I did my time testing by
> concatenating the default input number to itself several times. At
> 5184 decimal digits, which was the last case I tried, the 58th output
> digit was 1111111, after which point the remaining 672 output digits
> are all 12665464, which seemed rather improbable.

No there is no bug, try 12665465^77 wolfram alpha gives
7976781785088446084873306945009387424538813088491673402748035781202063831184317457635773873975198429536727051788368983080016275695338973787545408588270715883570324803120051806974411884509607864124408509087974736382639726555779113221450078243163340395784592348080359062163885323462144257596380766496966084237358222613649636595300081207683175410584139478853066960795599027150403706812116411354218686846750615496852415389467898730884554700066802269059177874281189887271741594702334892800692129299635716260471508809448693000376806594431400299072265625

Try that number you get a ONE followed by 77 ZEROS


From andrew3 at r3dsolutions.com  Mon Jun 29 19:45:42 2015
From: andrew3 at r3dsolutions.com (Andrew Robinson)
Date: Mon, 29 Jun 2015 23:45:42 +0000
Subject: Cross compiling C Python2.7.10 for arm on x86_64 linux box.
Message-ID: <5591D8A6.5010702@r3dsolutions.com>

Hi,

I'm Needing to get python 2.7.10 to cross compile correctly for an ARM 
embedded device.
I'm very close, as it does build with warnings, but the result is 
defective and I'm not sure how to fix it.
For some odd reason, the interpreter does run -- but I either get random 
segfaults -- or if I configure --without-pymalloc, I get messages when I 
import libraries saying that:

-----------------------------------------------------------------------
Python 2.7.10 (default, Jun 29 2015, 23:00:31)
[GCC 4.8.1 20130401 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> import math
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ImportError: /mnt/user/lib/python2.7/lib-dynload/math.so: undefined 
symbol: Py_InitModule4
[40857 refs]
 >>>
-----------------------------------------------------------------------

That message suggests this might be an issue with 32/64 bit machine 
architectures confilicts according to information I googled on it.

I am also seeing many warnings during build like the following which 
confirms some kind of build size mismatch:
*** WARNING: renaming "pyexpat" since importing it failed: 
build/lib.linux-x86_64-2.7/pyexpat.so: wrong ELF class: ELFCLASS32 
building '_elementtree' extension

I think Py_InitModule4 is normally found in libpython2.7.so,  as the 
symbol at least exists in the 32 and 64 bit complations of python that I 
have ; but when I check the cross compiled version, it's not there.
So I think the python ./configure script might be getting confused about 
the size of the target it's trying to build as I am building it on a 64 
bit x86 machine, and the target is a 32 bit arm processor.


The following is what I am doing from bash shell (linux) to try and 
cross compile python.
And idea of what I am doing wrong ?

-----------------------------------------------------------------------
# A par# A parser generator and build system version of python are 
supposed to be
# needed to run parts of the cross compilation;  I do see python used in the
# Makefile, but no references to a buid version of PGEN are defined,
# so I don't know if PGEN gets used or not -- but I build it anyway...
# As this is what receipies on the web say to do...

make distclean
./configure
make Parser/pgen python
mv python python_for_build
mv Parser/pgen Parser/pgen_for_build
make distclean

# fix setup.py to handle installing to the target system's fake install
# directory found on the build system at $DEVICEROOT.

if grep -q os.environ['DEVICEROOT'] ; then echo "Already patched" ; else
     sed -i setup.py 's%^[[:space:]]*math_libs = [].*$%    if 
'DEVICEROOT' in os.environ:\n        lib_dirs += 
os.environ['DEVICEROOT']+'/mnt/user/lib\n        lib_dirs += 
os.environ['DEVICEROOT']+'/mnt/user/include\n'%
fi

# We want utf-8, unicode terminal handling -- so make sure python compiles
# with ncursesw substituted for curses.

CURSESFLAGS=`pkg-config --cflags ncursesw`

# Configure python to be built
CFLAGS="${CFLAGS} ${CURSESFLAGS} -g3 -ggdb -gdwarf-4" ./configure 
--host=${CROSSTARGET} --build=i686-linux --enable-unicode 
--enable-shared --with-pydebug --prefix=/mnt/user --disable-ipv6 
--without-pymalloc ac_cv_file__dev_ptmx=yes ac_cv_file__dev_ptc=no 
ac_cv_have_long_long_format=yes PYTHON_FOR_BUILD=${PWD}/python_for_build

# Fix a bug in the Makefile
# The build version of python, ought not try to actually use the ARM 
libraries.
sed -i -e 's%\([[:space:]]\)\(PYTHONPATH=$(DESTDIR)$(LIBDEST)\)%\1-\2%' 
Makefile
echo "Fix the makefile if you can"
sleep 10

make PYTHON_FOR_BUILD=${PWD}/python_for_build CROSS_COMPILE_TARGET=yes

echo " Waiting to allow you to see error messages before installing "
sleep 10

# Optionally, binary file stripping could be carried out on the python 
binary
# Don't strip if you are doing debugging of python
# strip --strip-unneeded python

make install DESTDIR=${DEVICEROOT} 
PYTHON_FOR_BUILD=${PWD}/python_for_build 
PGEN_FOR_BUILD=${PWD}/pgen_for_build
ser generator and build system version of python are supposed to be
# needed to run parts of the cross compilation;  I do see python used in the
# Makefile, but no references to a buid version of PGEN are defined,
# so I don't know if PGEN gets used or not -- but I build it anyway...
# As this is what receipies on the web say to do...

make distclean
./configure
make Parser/pgen python
mv python python_for_build
mv Parser/pgen Parser/pgen_for_build
make distclean

# fix setup.py to handle installing to the target system's fake install
# directory found on the build system at $DEVICEROOT.

if grep -q os.environ['DEVICEROOT'] ; then echo "Already patched" ; else
     sed -i setup.py 's%^[[:space:]]*math_libs = [].*$%    if 
'DEVICEROOT' in os.environ:\n        lib_dirs += 
os.environ['DEVICEROOT']+'/mnt/user/lib\n        lib_dirs += 
os.environ['DEVICEROOT']+'/mnt/user/include\n'%
fi

# We want utf-8, unicode terminal handling -- so make sure python compiles
# with ncursesw substituted for curses.

CURSESFLAGS=`pkg-config --cflags ncursesw`

# Configure python to be built
CFLAGS="${CFLAGS} ${CURSESFLAGS} -g3 -ggdb -gdwarf-4" ./configure 
--host=${CROSSTARGET} --build=i686-linux --enable-unicode 
--enable-shared --with-pydebug --prefix=/mnt/user --disable-ipv6 
--without-pymalloc ac_cv_file__dev_ptmx=yes ac_cv_file__dev_ptc=no 
ac_cv_have_long_long_format=yes PYTHON_FOR_BUILD=${PWD}/python_for_build

# Fix a bug in the Makefile
# The build version of python, ought not try to actually use the ARM 
libraries.
sed -i -e 's%\([[:space:]]\)\(PYTHONPATH=$(DESTDIR)$(LIBDEST)\)%\1-\2%' 
Makefile
echo "Fix the makefile if you can"
sleep 10

make PYTHON_FOR_BUILD=${PWD}/python_for_build CROSS_COMPILE_TARGET=yes

echo " Waiting to allow you to see error messages before installing "
sleep 10

# Optionally, binary file stripping could be carried out on the python 
binary
# Don't strip if you are doing debugging of python
# strip --strip-unneeded python

make install DESTDIR=${DEVICEROOT} 
PYTHON_FOR_BUILD=${PWD}/python_for_build 
PGEN_FOR_BUILD=${PWD}/pgen_for_build

--------------------------------------------------------------------------






From ben.usenet at bsb.me.uk  Mon Jun 29 20:09:06 2015
From: ben.usenet at bsb.me.uk (Ben Bacarisse)
Date: Tue, 30 Jun 2015 01:09:06 +0100
Subject: Linear time baseconversion
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
Message-ID: <87r3ouawgt.fsf@bsb.me.uk>

Ian Kelly <ian.g.kelly at gmail.com> writes:

> On Mon, Jun 29, 2015 at 4:56 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> On Mon, Jun 29, 2015 at 4:39 PM,  <jonas.thornvall at gmail.com> wrote:
>>> http://jt.node365.se/baseconversion8.html
<snip>
> By the way, I think you have a bug. I did my time testing by
> concatenating the default input number to itself several times. At
> 5184 decimal digits, which was the last case I tried, the 58th output
> digit was 1111111, after which point the remaining 672 output digits
> are all 12665464, which seemed rather improbable.

Yes, it's a bug.  Because it's off-topic, I won't say more here but I
posted a simpler test case to comp.lang.javacript just now (10000000
converted to base 100).  I would not normally reply here, but I wanted
to acknowledge your post as prompting me to look for other cases.
Followup-to: set.

-- 
Ben.


From cs at zip.com.au  Mon Jun 29 20:25:05 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 30 Jun 2015 10:25:05 +1000
Subject: Readline -- cannot bind to both Ctrl-tab and tab at the same time?
In-Reply-To: <CAPTjJmrHW6zyOqQYutjZNdq_7X5r+fZzSiFqJKJW9J7oas+cPQ@mail.gmail.com>
References: <CAPTjJmrHW6zyOqQYutjZNdq_7X5r+fZzSiFqJKJW9J7oas+cPQ@mail.gmail.com>
Message-ID: <20150630002505.GA64266@cskk.homeip.net>

On 29Jun2015 11:36, Chris Angelico <rosuav at gmail.com> wrote:
>On Mon, Jun 29, 2015 at 4:55 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> Try Ctrl-TAB again, and you'll get "raise" instead of "import".
>>
>> Can anyone else replicate this issue?
>>
>> Is this a Python issue, a problem with the terminal I am using, or readline
>> in general?
>
>Confirmed with Python 3.6 on Debian Jessie. Delving into Laura's
>suggestion that it's a readline problem, I came across this:
>
>http://stackoverflow.com/questions/12044574/getting-complete-and-menu-complete-to-work-together
>
>Testing with "showkey -a" suggests that tab and ctrl-tab indeed send
>the same bytes. I wonder if there's a way to change terminal type or
>fiddle with terminfo to change this? GUI programs obviously don't have
>this conflation.

Yes, this was my surmise (and expectation, frankly - what did you expect 
ctrl-TAB to send?)  So there is no way for the program in the terminal to 
distinguish one from the other; you will always only get one of your bindings.

I would not expect terminfo to get you anything on its own - it is consulted by 
the receiving program. The sending program (the terminal emulator) needs to be 
configured to send different sequences for TAB and ctrl-TAB. Once that is done, 
_then_ you may be able to augument your terminfo to express this behaviour and 
have readline honour the change.

1: Work backward from the readline internals to find out if it consults a 
terminfo property to find out what ctrl-TAB might be.

2: Decide on what you want ctrl-TAB to send instead of the TAB code.

3: Make a new terminfo clause defining that (for readline's use) and hooking 
onto your old terminfo definition for the rest (tc=other_terminal in termcap, 
possibly the same in terminfo).

4: Configure your terminal emulator to send that sequence.

Regarding 4, with proper old-style X11 terminals you could control all this 
directly with the VT100.Translations properties. Example from my xterm 
Xdefaults file; the leading F1 definition is probably what you want:

  XTerm*.VT100.Translations:  #override \n\
    <KeyPress>F1        : string(0x1b) \n \
    <KeyPress>F10       : secure() \n \
    <KeyPress>KP_End        : scroll-forw(100,page) \n \
    <KeyPress>KP_Prior      : scroll-back(1,halfpage) \n \
    <KeyPress>KP_Next       : scroll-forw(1,halfpage) \n\
    <KeyPress>End       : scroll-forw(100,page) \n \
    Ctrl<KeyPress>Up        : scroll-back(1,line) \n \
    Ctrl<KeyPress>Down      : scroll-forw(1,line) \n\
    <KeyPress>Prior     : scroll-back(1,halfpage) \n \
    <KeyPress>Next      : scroll-forw(1,halfpage) \n\
    Meta<KeyPress>      : string(0x1b) insert() \n\
    Alt<KeyPress>       : string(0x1b) insert() \n\
    <KeyPress>BackSpace     : string(0x08) \n \
    Shift<Btn1Down>     : ignore() \n\
    Shift<Btn1Up>       : insert-selection(PRIMARY,CUT_BUFFER0)\n\
    <Btn2Down>          : ignore() \n\
    <Btn2Up>            : insert-selection(PRIMARY,CUT_BUFFER0)\n\
    Shift<Btn2Down>     : ignore() \n\
    Shift<Btn2Up>       : string("us ") insert-selection(PRIMARY,CUT_BUFFER0) 
string("\n") \n\
    ~Ctrl ~Meta<Btn3Down>   : ignore()\n\
    ~Ctrl ~Meta<Btn3Up>     : ignore()\n
  ! commented out, previous experiments
  ! XTerm*VT100.translations: #override Shift<Key>'F1':insert("hi guys")
  ! XTerm*VT100.translations: #override 
  Ctrl<Key>'U':uphalfscreen()\nCtrl<Key>'D'downhalfscreen()

The new fangled terminals like gnome-terminal or Konsole probably threw that 
stuff out as they threw out all the other normal X11 config hooks, and 
doubtless use their own arcane system. Enjoy.

Cheers,
Cameron Simpson <cs at zip.com.au>

The top three answers:  Yes I *am* going to a fire!
                        Oh! We're using *kilometers* per hour now.
                        I have to go that fast to get back to my own time.
- Peter Harper <bo165 at FreeNet.Carleton.CA>


From denismfmcmahon at gmail.com  Mon Jun 29 20:54:22 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Tue, 30 Jun 2015 00:54:22 +0000 (UTC)
Subject: enumerate XML tags (keys that will become headers) along with
 text (values) and write to CSV in one row (as opposed to "stacked" values
 with one header)
References: <14aeae7a-41ab-4619-8331-7995e2420e54@googlegroups.com>
 <mmivtd$fqa$1@dont-email.me>
 <mailman.150.1435477605.3674.python-list@python.org>
 <mmpn9p$667$1@dont-email.me>
 <8e7d3dd8-f7c1-4466-ae82-f25e4a252fc7@googlegroups.com>
Message-ID: <mmspbt$88c$1@dont-email.me>

On Sun, 28 Jun 2015 17:07:00 -0700, Ned Batchelder wrote:

> On Sunday, June 28, 2015 at 5:02:19 PM UTC-4, Denis McMahon wrote:

>> <things>
>>   <thing>string 3</thing>
>>   <thing>string 2</thing>
>>   <thing>string 1</thing>
>> </things>

>> Each <thing> is just a member of the collection things, the xml does
>> not contain sufficient information to state that <things> is an ordered
>> collection containing a specific sequence of <thing>.
> 
> You are right that XML does not specify that <things> is an ordered
> collection.
> But XML does preserve the order of the children.  There are many XML
> schema that rely on XML's order-preserving nature.

But what we *don't* know is whether the order of the umpteen identical 
tags in the XML has any significance in terms of the original data, 
although the OP seems intent on assigning some significance to that order 
without any basis for doing so.

Consider the following tuple:

t = (tuplemember_1, tuplemember_2, .... tuplemember_n)

Can we safely assume that if the tuple is ever converted to xml, either 
now or at some future date using whatever the code implementation is 
then, that the order of the items will be preserved:

<tuple>
  <item>tuplemember_1</item>
  <item>tuplemember_2</item>
....
  <item>tuplemember_n/item>
</tuple>

And if we're reading that xml structure at some point in the future, is 
it safe to assume that the tuple members are in the same order in the xml 
as they were in the original tuple?

For sanity <item> should have an attribute specifying the sequence of the 
item in it's tuple.
-- 
Denis McMahon, denismfmcmahon at gmail.com


From pkpearson at nowhere.invalid  Mon Jun 29 20:56:26 2015
From: pkpearson at nowhere.invalid (Peter Pearson)
Date: 30 Jun 2015 00:56:26 GMT
Subject: Matplotlib X-axis timezone trouble
Message-ID: <cve7ppFk1viU1@mid.individual.net>

The following code produces a plot with a line running from (9:30, 0) to
(10:30, 1), not from (8:30, 0) to (9:30, 1) as I desire.

If I use timezone None instead of pacific, the plot is as desired, but
of course that doesn't solve the general problem of which this is a
much-reduced example.

If I use timezone US/Central, I get the same (bad) plot.

import matplotlib.pyplot as plt
import datetime
import pytz
pacific = pytz.timezone("US/Pacific")
fig = plt.figure()
plt.plot([datetime.datetime(2014, 10, 7, 8, 30, tzinfo=pacific),
          datetime.datetime(2014, 10, 7, 9, 30, tzinfo=pacific)],
         [0,1], marker="o", color="green")
fig.autofmt_xdate()
plt.show()

Does anybody know why this shift is occurring?  Is Matplotlib
confused about what timezone to use in labeling the axis?  How
would I tell it what timezone to use (preferably explicitly in
the code, not in matplotlibrc)?

Thanks.

-- 
To email me, substitute nowhere->runbox, invalid->com.


From lac at openend.se  Mon Jun 29 22:03:57 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 30 Jun 2015 04:03:57 +0200
Subject: Matplotlib X-axis timezone trouble
In-Reply-To: Message from Peter Pearson <pkpearson@nowhere.invalid>
 of "30 Jun 2015 00:56:26 +0000." <cve7ppFk1viU1@mid.individual.net>
References: <cve7ppFk1viU1@mid.individual.net>
Message-ID: <201506300204.t5U23vFw020482@fido.openend.se>

In a message of 30 Jun 2015 00:56:26 +0000, Peter Pearson writes:
>The following code produces a plot with a line running from (9:30, 0) to
>(10:30, 1), not from (8:30, 0) to (9:30, 1) as I desire.
>
>If I use timezone None instead of pacific, the plot is as desired, but
>of course that doesn't solve the general problem of which this is a
>much-reduced example.
>
>If I use timezone US/Central, I get the same (bad) plot.
>
>import matplotlib.pyplot as plt
>import datetime
>import pytz
>pacific = pytz.timezone("US/Pacific")
>fig = plt.figure()
>plt.plot([datetime.datetime(2014, 10, 7, 8, 30, tzinfo=pacific),
>          datetime.datetime(2014, 10, 7, 9, 30, tzinfo=pacific)],
>         [0,1], marker="o", color="green")
>fig.autofmt_xdate()
>plt.show()
>
>Does anybody know why this shift is occurring?  Is Matplotlib
>confused about what timezone to use in labeling the axis?  How
>would I tell it what timezone to use (preferably explicitly in
>the code, not in matplotlibrc)?
>
>Thanks.
>
>-- 
>To email me, substitute nowhere->runbox, invalid->com.
>-- 
>https://mail.python.org/mailman/listinfo/python-list


I know that your problem is that all your times are in UTC, even
though you do not want this.  I had this too.  I forget what I did.

I do not know if this solution from stackoverflow.com will work for
you -- it is definitely different from what I did, but that does
not in any way make it wrong.  Try and see?  and report back?

http://stackoverflow.com/questions/4485607/matplotlib-plot-date-keeping-times-in-utc-even-with-custom-timezone


From rosuav at gmail.com  Mon Jun 29 22:11:31 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 30 Jun 2015 12:11:31 +1000
Subject: Matplotlib X-axis timezone trouble
In-Reply-To: <cve7ppFk1viU1@mid.individual.net>
References: <cve7ppFk1viU1@mid.individual.net>
Message-ID: <CAPTjJmrTwUKJ0n6CK9OMBtEDeMbf0FrKNvoOByGmB=T0RVrVLg@mail.gmail.com>

On Tue, Jun 30, 2015 at 10:56 AM, Peter Pearson
<pkpearson at nowhere.invalid> wrote:
> The following code produces a plot with a line running from (9:30, 0) to
> (10:30, 1), not from (8:30, 0) to (9:30, 1) as I desire.
>
> pacific = pytz.timezone("US/Pacific")
> plt.plot([datetime.datetime(2014, 10, 7, 8, 30, tzinfo=pacific),
>           datetime.datetime(2014, 10, 7, 9, 30, tzinfo=pacific)],
>          [0,1], marker="o", color="green")

Sounds to me like a bug in DST handling - in October 2014, Los Angeles
was on Pacific Daylight Time, which would give a one-hour offset if
matplotlib is handling things on the basis of Pacific Standard Time. I
don't know if that helps at all with figuring out a solution, though.

ChrisA


From steve at pearwood.info  Mon Jun 29 23:00:45 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 30 Jun 2015 13:00:45 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>
 <mailman.171.1435611181.3674.python-list@python.org>
Message-ID: <5592065e$0$1675$c3e8da3$5496439d@news.astraweb.com>

On Tue, 30 Jun 2015 06:52 am, Randall Smith wrote:

> Not sure why you posted the link.  The crc32 checksum is just to check
> for possible filesystem corruption.  The system does periodic data
> corruption checks.  BTRFS uses crc32 checksums also.  Please explain.

The file system can trust that anything writing to a file is allowed to
write to it, in doesn't have to defend against malicious writes. As I
understand it, your application does.

Here is the attack scenario I have in mind:

- you write a file to my computer, and neglect to encrypt it;
- and record the checksum for later;
- I insert malware into your file;
- you retrieve the file from me;
- if the checksum matches what you have on record, you accept the file;
- since you are using CRC, it is quite easy for me to ensure the 
  checksums match after inserting malware;
- and I have now successfully injected malware into your computer.

I'm making an assumption here -- I assume that the sender records a checksum
for uploaded files so that when they get something back again they can tell
whether or not it is the same content they uploaded.

* * * 

By the way, regarding the use of a substitution cipher, I spoke to the
crypto guy at work, and "preimage attack" is not quite the right
terminology, since that's normally used in the context of hash functions.
It's almost a "known ciphertext attack", but not quite, since that
terminology refers to guessing the key from the ciphertext.

I was wrong: cryptographically strong ciphers are generally NOT resistant to
what I described as a preimage attack. If the key leaks, using AES won't
save you: an attacker with access to the key can produce a ciphertext that
decrypts to the malware of his choice, regardless of whether you use
AES-256 or rot-13. There may be other encryption methods which don't suffer
from that, but he doesn't know of them off the top of his head.

His comment was, "don't leak the key".

The other threat I mentioned is that the receiver will read the content of
the file. For that, a strong cipher is much to be preferred over a weak
one, and it needs to be encrypted by the sending end, not the receiving
end. (If the receiving end does it, it has to keep the key so it can
decrypt before sending back, which means the computer's owner can just grab
the key and read the files.)



-- 
Steven



From pkpearson at nowhere.invalid  Tue Jun 30 00:49:06 2015
From: pkpearson at nowhere.invalid (Peter Pearson)
Date: 30 Jun 2015 04:49:06 GMT
Subject: Matplotlib X-axis timezone trouble
References: <cve7ppFk1viU1@mid.individual.net>
 <mailman.179.1435630294.3674.python-list@python.org>
Message-ID: <cvele1Fmra9U1@mid.individual.net>

On Tue, 30 Jun 2015 12:11:31 +1000, Chris Angelico <rosuav at gmail.com> wrote:
> On Tue, Jun 30, 2015 at 10:56 AM, Peter Pearson
><pkpearson at nowhere.invalid> wrote:
>> The following code produces a plot with a line running from (9:30, 0) to
>> (10:30, 1), not from (8:30, 0) to (9:30, 1) as I desire.
>>
>> pacific = pytz.timezone("US/Pacific")
>> plt.plot([datetime.datetime(2014, 10, 7, 8, 30, tzinfo=pacific),
>>           datetime.datetime(2014, 10, 7, 9, 30, tzinfo=pacific)],
>>          [0,1], marker="o", color="green")
>
> Sounds to me like a bug in DST handling - in October 2014, Los Angeles
> was on Pacific Daylight Time, which would give a one-hour offset if
> matplotlib is handling things on the basis of Pacific Standard Time. I
> don't know if that helps at all with figuring out a solution, though.

Thanks, I think you're onto something.  Points in a non-daylight-saving
month get plotted correctly:

plt.plot([datetime.datetime(2014, 12, 7, 8, 30, tzinfo=pacific),
          datetime.datetime(2014, 12, 7, 9, 30, tzinfo=pacific)],
         [0,1], marker="o", color="green")

Time zones teem with sneaky software problems, and so does daylight-saving
time, so this problem might strain my brain.  Maybe it's going to turn
out that my expectations are unreasonable . . . as in, "Well, smarty pants,
how do you want the axis labelled when the abscissas straddle the
beginning of daylight-saving time?"  I'll research and digest.

-- 
To email me, substitute nowhere->runbox, invalid->com.


From pkpearson at nowhere.invalid  Tue Jun 30 00:57:11 2015
From: pkpearson at nowhere.invalid (Peter Pearson)
Date: 30 Jun 2015 04:57:11 GMT
Subject: Matplotlib X-axis timezone trouble
References: <cve7ppFk1viU1@mid.individual.net>
 <mailman.178.1435629849.3674.python-list@python.org>
Message-ID: <cvelt7Fmra9U2@mid.individual.net>

On Tue, 30 Jun 2015 04:03:57 +0200, Laura Creighton <lac at openend.se> wrote:
> In a message of 30 Jun 2015 00:56:26 +0000, Peter Pearson writes:
>>The following code produces a plot with a line running from (9:30, 0) to
>>(10:30, 1), not from (8:30, 0) to (9:30, 1) as I desire.
>>
>>If I use timezone None instead of pacific, the plot is as desired, but
>>of course that doesn't solve the general problem of which this is a
>>much-reduced example.
>>
>>If I use timezone US/Central, I get the same (bad) plot.
>>
>>import matplotlib.pyplot as plt
>>import datetime
>>import pytz
>>pacific = pytz.timezone("US/Pacific")
>>fig = plt.figure()
>>plt.plot([datetime.datetime(2014, 10, 7, 8, 30, tzinfo=pacific),
>>          datetime.datetime(2014, 10, 7, 9, 30, tzinfo=pacific)],
>>         [0,1], marker="o", color="green")
>>fig.autofmt_xdate()
>>plt.show()
>>
>>Does anybody know why this shift is occurring?  Is Matplotlib
>>confused about what timezone to use in labeling the axis?  How
>>would I tell it what timezone to use (preferably explicitly in
>>the code, not in matplotlibrc)?
[snip]
>
> I know that your problem is that all your times are in UTC, even
> though you do not want this.  I had this too.  I forget what I did.

Actually, UTC is very helpful in this project, which has to
deal with timestamped data from various timezones.  After I finally
stumbled upon datetime's astimezone() method, things seemed to be
going smoothly . . .

> I do not know if this solution from stackoverflow.com will work for
> you -- it is definitely different from what I did, but that does
> not in any way make it wrong.  Try and see?  and report back?
>
> http://stackoverflow.com/questions/4485607/matplotlib-plot-date-keeping-times-in-utc-even-with-custom-timezone

Thanks for the pointer.  I'll report back.


-- 
To email me, substitute nowhere->runbox, invalid->com.


From jonas.thornvall at gmail.com  Tue Jun 30 02:49:46 2015
From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com)
Date: Mon, 29 Jun 2015 23:49:46 -0700 (PDT)
Subject: Linear time baseconversion
In-Reply-To: <mailman.175.1435619500.3674.python-list@python.org>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
Message-ID: <adc1ffef-5e45-4836-af39-6667b583cbd7@googlegroups.com>

Den tisdag 30 juni 2015 kl. 01:11:51 UTC+2 skrev Ian:
> On Mon, Jun 29, 2015 at 4:56 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> > On Mon, Jun 29, 2015 at 4:39 PM,  <jonas.thornvall at gmail.com> wrote:
> >> http://jt.node365.se/baseconversion8.html
> >
> > Back of the envelope mental calculation, that appears to be quadratic,
> > not linear. Doubling the length of the input results in an approximate
> > quadrupling of the time taken to produce the output.
> >
> > That noted, this is off-topic for this list, which is for discussion
> > about Python. Please take this to somewhere else like
> > comp.lang.javascript instead.
> 
> By the way, I think you have a bug. I did my time testing by
> concatenating the default input number to itself several times. At
> 5184 decimal digits, which was the last case I tried, the 58th output
> digit was 1111111, after which point the remaining 672 output digits
> are all 12665464, which seemed rather improbable.

It isn't i used the power to check with wolfram, but there is a plus 1 bug as ben noticed.

But not in this slower version.
http://jt.node365.se/baseconversion5.html


From rosuav at gmail.com  Tue Jun 30 03:01:15 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 30 Jun 2015 17:01:15 +1000
Subject: Matplotlib X-axis timezone trouble
In-Reply-To: <cvele1Fmra9U1@mid.individual.net>
References: <cve7ppFk1viU1@mid.individual.net>
 <mailman.179.1435630294.3674.python-list@python.org>
 <cvele1Fmra9U1@mid.individual.net>
Message-ID: <CAPTjJmoy_KSX8obGXnkVcK8LN_UkVoDkPMcZ=ozh0Zw3iEsM0w@mail.gmail.com>

On Tue, Jun 30, 2015 at 2:49 PM, Peter Pearson
<pkpearson at nowhere.invalid> wrote:
> Time zones teem with sneaky software problems, and so does daylight-saving
> time, so this problem might strain my brain.  Maybe it's going to turn
> out that my expectations are unreasonable . . . as in, "Well, smarty pants,
> how do you want the axis labelled when the abscissas straddle the
> beginning of daylight-saving time?"  I'll research and digest.

That's entirely possible. Even more so if you go across some other
civil time change - if you look at the history of timezones in tzdata,
there's no end of messes as different places adopted standard time,
redefined standard time, and unified with someone else's standard
time. And some of that happened relatively recently.

UTC is much easier for this kind of thing. Especially if the
granularity of your data lets you ignore leap seconds.

ChrisA


From jonas.thornvall at gmail.com  Tue Jun 30 04:52:08 2015
From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com)
Date: Tue, 30 Jun 2015 01:52:08 -0700 (PDT)
Subject: Linear time baseconversion
In-Reply-To: <87r3ouawgt.fsf@bsb.me.uk>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
Message-ID: <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>

Den tisdag 30 juni 2015 kl. 02:09:17 UTC+2 skrev Ben Bacarisse:
> Ian Kelly <ian.g.kelly at gmail.com> writes:
> 
> > On Mon, Jun 29, 2015 at 4:56 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> >> On Mon, Jun 29, 2015 at 4:39 PM,  <jonas.thornvall at gmail.com> wrote:
> >>> http://jt.node365.se/baseconversion8.html
> <snip>
> > By the way, I think you have a bug. I did my time testing by
> > concatenating the default input number to itself several times. At
> > 5184 decimal digits, which was the last case I tried, the 58th output
> > digit was 1111111, after which point the remaining 672 output digits
> > are all 12665464, which seemed rather improbable.
> 
> Yes, it's a bug.  Because it's off-topic, I won't say more here but I
> posted a simpler test case to comp.lang.javacript just now (10000000
> converted to base 100).  I would not normally reply here, but I wanted
> to acknowledge your post as prompting me to look for other cases.
> Followup-to: set.
> 
> -- 
> Ben.

Thank you, for finding the bug Ben, it turned out i did basicly same thing two times. One of the times was the more general case and that turned out to do the correct thing.

jt.node365.se/baseconversion9.html

It still bug out on very big numbers if base outside integer scope.
I am very keen on suggestions regarding the logic to make it faster.
I will read in base into an array holding bignumb, and then it should work for anybase and any number as far i can understand. 


From auriocus at gmx.de  Tue Jun 30 05:07:50 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Tue, 30 Jun 2015 11:07:50 +0200
Subject: Linear time baseconversion
In-Reply-To: <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
Message-ID: <mmtm6k$8b7$1@dont-email.me>

Am 30.06.15 um 10:52 schrieb jonas.thornvall at gmail.com:
> It still bug out on very big numbers if base outside integer scope.
> I am very keen on suggestions regarding the logic to make it faster.

Concerning the algorithmic complexity, it can't be faster than square 
time in the number of digits N. Baseconversion needs to do a sequence of 
division operations, where every operation gves you one digit in the new 
base. The number of digits in the new base is proportional to the number 
of digits in the old base (the ratio is log b1/log b2). Therefore it 
will be O(N^2).

	Christian




From robert.kern at gmail.com  Tue Jun 30 05:16:03 2015
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 30 Jun 2015 10:16:03 +0100
Subject: enumerate XML tags (keys that will become headers) along with
 text (values) and write to CSV in one row (as opposed to "stacked" values
 with one header)
In-Reply-To: <mmspbt$88c$1@dont-email.me>
References: <14aeae7a-41ab-4619-8331-7995e2420e54@googlegroups.com>
 <mmivtd$fqa$1@dont-email.me>
 <mailman.150.1435477605.3674.python-list@python.org>
 <mmpn9p$667$1@dont-email.me>
 <8e7d3dd8-f7c1-4466-ae82-f25e4a252fc7@googlegroups.com>
 <mmspbt$88c$1@dont-email.me>
Message-ID: <mmtmoj$rh4$1@ger.gmane.org>

On 2015-06-30 01:54, Denis McMahon wrote:
> On Sun, 28 Jun 2015 17:07:00 -0700, Ned Batchelder wrote:
>
>> On Sunday, June 28, 2015 at 5:02:19 PM UTC-4, Denis McMahon wrote:
>
>>> <things>
>>>    <thing>string 3</thing>
>>>    <thing>string 2</thing>
>>>    <thing>string 1</thing>
>>> </things>
>
>>> Each <thing> is just a member of the collection things, the xml does
>>> not contain sufficient information to state that <things> is an ordered
>>> collection containing a specific sequence of <thing>.
>>
>> You are right that XML does not specify that <things> is an ordered
>> collection.
>> But XML does preserve the order of the children.  There are many XML
>> schema that rely on XML's order-preserving nature.
>
> But what we *don't* know is whether the order of the umpteen identical
> tags in the XML has any significance in terms of the original data,
> although the OP seems intent on assigning some significance to that order
> without any basis for doing so.
>
> Consider the following tuple:
>
> t = (tuplemember_1, tuplemember_2, .... tuplemember_n)
>
> Can we safely assume that if the tuple is ever converted to xml, either
> now or at some future date using whatever the code implementation is
> then, that the order of the items will be preserved:
>
> <tuple>
>    <item>tuplemember_1</item>
>    <item>tuplemember_2</item>
> ....
>    <item>tuplemember_n/item>
> </tuple>

Barring bugs, yes!

> And if we're reading that xml structure at some point in the future, is
> it safe to assume that the tuple members are in the same order in the xml
> as they were in the original tuple?

Yes! Any conforming XML implementation will preserve the order.

> For sanity <item> should have an attribute specifying the sequence of the
> item in it's tuple.

While it may make you more comfortable, it's hardly a requirement for sanity.

I think you had a point in your first paragraph here, but you are obscuring it 
with FUD. The problem is not whether unadorned XML elements can be used to 
represent an ordered collection. They can and are, frequently, without any 
problem because XML elements are intrinsically ordered.

The real problem that you almost get around to articulating is that XML elements 
can *also* be used to represent unordered collections simply by ignoring the 
(preserved) order of the elements. And if you are completely blind as to the 
schema as the OP apparently is, and you are simply given a load of XML and told 
to do "something" with it, you don't know if any given collection is meant to be 
ordered or unordered. Of course, the only sensible thing to do is just preserve 
the order given to you as that is what the semantics of XML requires of you in 
the absence of a schema that says otherwise. You can always disregard the order 
later.

That said, if the data is regular enough to actually be restructured into a 
table (i.e. if <MonthDayCount> always has the same number of children, etc.), 
then it probably does represent an ordered collection. If it's variable, then 
putting it into a table structure probably doesn't make any sense regardless of 
ordering issues.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From redstone-cold at 163.com  Tue Jun 30 05:18:20 2015
From: redstone-cold at 163.com (iMath)
Date: Tue, 30 Jun 2015 02:18:20 -0700 (PDT)
Subject: get Coursera video download link via program
Message-ID: <32cd5caa-e0c6-4d54-81fe-b75bc036f08e@googlegroups.com>

I want extract Coursera video download link via program(mainly Python) behind those links

https://www.coursera.org/learn/human-computer-interaction/lecture/s4rFQ/the-interaction-design-specialization

https://www.coursera.org/learn/calculus1/lecture/IYGhT/why-is-calculus-going-to-be-so-much-fun

After red a lot of articles about this, still cannot find a way to extract the video download link via program, anyone can offer a step by step solution of extracting the video download link ? Thanks!

P.S. I know this project ,
https://github.com/coursera-dl/coursera
 but the code is so complex , so I dropped out.


From jonas.thornvall at gmail.com  Tue Jun 30 05:20:50 2015
From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com)
Date: Tue, 30 Jun 2015 02:20:50 -0700 (PDT)
Subject: Linear time baseconversion
In-Reply-To: <mmtm6k$8b7$1@dont-email.me>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
Message-ID: <901f608a-6e8b-4f6a-87e7-175fe0ca5cf8@googlegroups.com>

Den tisdag 30 juni 2015 kl. 11:08:01 UTC+2 skrev Christian Gollwitzer:
> Am 30.06.15 um 10:52 schrieb jonas.thornvall at gmail.com:
> > It still bug out on very big numbers if base outside integer scope.
> > I am very keen on suggestions regarding the logic to make it faster.
> 
> Concerning the algorithmic complexity, it can't be faster than square 
> time in the number of digits N. Baseconversion needs to do a sequence of 
> division operations, where every operation gves you one digit in the new 
> base. The number of digits in the new base is proportional to the number 
> of digits in the old base (the ratio is log b1/log b2). Therefore it 
> will be O(N^2).
> 
> 	Christian

There is not a single division "except for the 2 one, used in halfinterval search" 
This algorithm only use multiplication and a modified GCD search algorithm.

http://jt.node365.se/baseconversion9.html


From redstone-cold at 163.com  Tue Jun 30 05:29:40 2015
From: redstone-cold at 163.com (iMath)
Date: Tue, 30 Jun 2015 02:29:40 -0700 (PDT)
Subject: get Coursera video download link via program
In-Reply-To: <32cd5caa-e0c6-4d54-81fe-b75bc036f08e@googlegroups.com>
References: <32cd5caa-e0c6-4d54-81fe-b75bc036f08e@googlegroups.com>
Message-ID: <fa2bb1c0-5630-461a-901b-3a9fb5c1c054@googlegroups.com>

I know the two packages : BeautifulSoup and requests may help, I am also able to login in Coursera via requests, the difficulty is how to find out  the video download link from the page.


From jonas.thornvall at gmail.com  Tue Jun 30 05:34:56 2015
From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com)
Date: Tue, 30 Jun 2015 02:34:56 -0700 (PDT)
Subject: Linear time baseconversion
In-Reply-To: <mmtm6k$8b7$1@dont-email.me>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
Message-ID: <c92c3dc9-8a00-4244-bbd7-383f34983935@googlegroups.com>

Den tisdag 30 juni 2015 kl. 11:08:01 UTC+2 skrev Christian Gollwitzer:
> Am 30.06.15 um 10:52 schrieb jonas.thornvall at gmail.com:
> > It still bug out on very big numbers if base outside integer scope.
> > I am very keen on suggestions regarding the logic to make it faster.
> 
> Concerning the algorithmic complexity, it can't be faster than square 
> time in the number of digits N. Baseconversion needs to do a sequence of 
> division operations, where every operation gves you one digit in the new 
> base. The number of digits in the new base is proportional to the number 
> of digits in the old base (the ratio is log b1/log b2). Therefore it 
> will be O(N^2).
> 
> 	Christian

Any new digit will be found in SQRT(base) comparissons. 
Averaged case will be in (SQRT(base)*(SQRT(base)+1))/2


From jonas.thornvall at gmail.com  Tue Jun 30 05:43:45 2015
From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com)
Date: Tue, 30 Jun 2015 02:43:45 -0700 (PDT)
Subject: Linear time baseconversion
In-Reply-To: <c92c3dc9-8a00-4244-bbd7-383f34983935@googlegroups.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
 <c92c3dc9-8a00-4244-bbd7-383f34983935@googlegroups.com>
Message-ID: <0439e7f6-2604-4a07-802b-81ea03782f62@googlegroups.com>

Den tisdag 30 juni 2015 kl. 11:35:06 UTC+2 skrev jonas.t... at gmail.com:
> Den tisdag 30 juni 2015 kl. 11:08:01 UTC+2 skrev Christian Gollwitzer:
> > Am 30.06.15 um 10:52 schrieb jonas.thornvall at gmail.com:
> > > It still bug out on very big numbers if base outside integer scope.
> > > I am very keen on suggestions regarding the logic to make it faster.
> > 
> > Concerning the algorithmic complexity, it can't be faster than square 
> > time in the number of digits N. Baseconversion needs to do a sequence of 
> > division operations, where every operation gves you one digit in the new 
> > base. The number of digits in the new base is proportional to the number 
> > of digits in the old base (the ratio is log b1/log b2). Therefore it 
> > will be O(N^2).
> > 
> > 	Christian
> 
> Any new digit will be found in SQRT(base) comparissons. 
> Averaged case will be in (SQRT(base)*(SQRT(base)+1))/2

Well that averaging was balloney. How do i write that the digit will be found in.
Average values from 1 to SQRT(base)?


From mal at europython.eu  Tue Jun 30 07:12:09 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Tue, 30 Jun 2015 13:12:09 +0200
Subject: EuroPython 2015: Call for On-site Volunteers
Message-ID: <55927989.3090200@europython.eu>

EuroPython is organized and run by volunteers from the Python
community, but we?re only a few and we will need more help to make the
conference run smoothly.

We need your help !
-------------------

We will need help with the conference and registration desk, giving
out the swag bags and t-shirts, session chairing, entrance control,
set up and tear down, etc.

Perks for Volunteers
--------------------

In addition to endless fame and glory as official EuroPython
Volunteer, we have also added some real-life few perks for you:

 * We will grant each volunteer a compensation of EUR 22 per shift

 * Volunteers will be eligible for student house rooms we have
   available and can use their compensation to pay for these

 * Get an awesome EuroPython Volunteer T-Shirt that you can keep and
   show off to your friends :-)

Register as Volunteer
---------------------

Please see our EuroPython Volunteers page for details and the
registration form:

    https://ep2015.europython.eu/en/registration/volunteers/

If you have questions, please write to our helpdesk at europython.eu.

Hope to see you in Bilbao :-)

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/



From jon+usenet at unequivocal.co.uk  Tue Jun 30 08:19:21 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 30 Jun 2015 12:19:21 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>
 <mailman.171.1435611181.3674.python-list@python.org>
 <5592065e$0$1675$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <slrnmp52ct.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-30, Steven D'Aprano <steve at pearwood.info> wrote:
> On Tue, 30 Jun 2015 06:52 am, Randall Smith wrote:
>> Not sure why you posted the link.  The crc32 checksum is just to check
>> for possible filesystem corruption.  The system does periodic data
>> corruption checks.  BTRFS uses crc32 checksums also.  Please explain.
>
> The file system can trust that anything writing to a file is allowed to
> write to it, in doesn't have to defend against malicious writes. As I
> understand it, your application does.
>
> Here is the attack scenario I have in mind:
>
> - you write a file to my computer, and neglect to encrypt it;

Eh? The game is over right there. I don't trust you, and yet
I have just given you my private data, unencrypted. Checksums
don't even come into it, we have failed utterly at step 1.

> - since you are using CRC, it is quite easy for me to ensure the 
>   checksums match after inserting malware;

No, you have yet *again* misunderstood the difference between the
client and the server.

> I was wrong: cryptographically strong ciphers are generally NOT resistant to
> what I described as a preimage attack. If the key leaks, using AES won't
> save you: an attacker with access to the key can produce a ciphertext that
> decrypts to the malware of his choice, regardless of whether you use
> AES-256 or rot-13. There may be other encryption methods which don't suffer
> from that, but he doesn't know of them off the top of his head.

lol. I suspected as much. You and Johannes were even more wrong than
was already obvious.

> The other threat I mentioned is that the receiver will read the content of
> the file. For that, a strong cipher is much to be preferred over a weak
> one, and it needs to be encrypted by the sending end, not the receiving
> end. (If the receiving end does it, it has to keep the key so it can
> decrypt before sending back, which means the computer's owner can just grab
> the key and read the files.)

Yes, that is utterly basic.


From rosuav at gmail.com  Tue Jun 30 08:40:27 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 30 Jun 2015 22:40:27 +1000
Subject: enumerate XML tags (keys that will become headers) along with
 text (values) and write to CSV in one row (as opposed to "stacked" values
 with one header)
In-Reply-To: <dpr3padr9qb3tbghjq9970ap3lbur15mqf@4ax.com>
References: <14aeae7a-41ab-4619-8331-7995e2420e54@googlegroups.com>
 <mmivtd$fqa$1@dont-email.me>
 <mailman.150.1435477605.3674.python-list@python.org>
 <a27fd45c-0579-44c0-8eaa-25214fbbfdb1@googlegroups.com>
 <dpr3padr9qb3tbghjq9970ap3lbur15mqf@4ax.com>
Message-ID: <CAPTjJmqfOPPafMYAQNt6iCSQ6WEpif6=1VcXc1Vb0yWWqD7eOw@mail.gmail.com>

On Tue, Jun 30, 2015 at 11:26 AM, Dennis Lee Bieber
<wlfraed at ix.netcom.com> wrote:
>         My system takes something like three hours just to generate a 500GB
> backup (one partition each week -- I have a 4TB backup drive with only
> 740GB free; the other drives are only half full or I'd need an 8TB backup).
> And that's using a compiled backup program -- I'd hate to consider what
> Python would require to backup the partition.

Probably about the same, actually. In my experience, there's often
very little speed difference between a straight 'dd' from one
partition to another (say, making a disk image prior to data recovery)
and doing more complicated work (say, archiving and compressing).
Until you actually manage to saturate your CPU with the workload
(video editing or something), the time is most likely to be dominated
by the disk platters.

ChrisA


From jonas.thornvall at gmail.com  Tue Jun 30 09:22:26 2015
From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com)
Date: Tue, 30 Jun 2015 06:22:26 -0700 (PDT)
Subject: Linear time baseconversion
In-Reply-To: <0439e7f6-2604-4a07-802b-81ea03782f62@googlegroups.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
 <c92c3dc9-8a00-4244-bbd7-383f34983935@googlegroups.com>
 <0439e7f6-2604-4a07-802b-81ea03782f62@googlegroups.com>
Message-ID: <332f7cc7-4cc7-469f-a1c3-cbba40b7b641@googlegroups.com>

Den tisdag 30 juni 2015 kl. 11:43:55 UTC+2 skrev jonas.t... at gmail.com:
> Den tisdag 30 juni 2015 kl. 11:35:06 UTC+2 skrev jonas.t... at gmail.com:
> > Den tisdag 30 juni 2015 kl. 11:08:01 UTC+2 skrev Christian Gollwitzer:
> > > Am 30.06.15 um 10:52 schrieb jonas.thornvall at gmail.com:
> > > > It still bug out on very big numbers if base outside integer scope.
> > > > I am very keen on suggestions regarding the logic to make it faster.
> > > 
> > > Concerning the algorithmic complexity, it can't be faster than square 
> > > time in the number of digits N. Baseconversion needs to do a sequence of 
> > > division operations, where every operation gves you one digit in the new 
> > > base. The number of digits in the new base is proportional to the number 
> > > of digits in the old base (the ratio is log b1/log b2). Therefore it 
> > > will be O(N^2).
> > > 
> > > 	Christian
> > 
> > Any new digit will be found in SQRT(base) comparissons. 
> > Averaged case will be in (SQRT(base)*(SQRT(base)+1))/2
> 
> Well that averaging was balloney. How do i write that the digit will be found in.
> Average values from 1 to SQRT(base)?

Regarding the time it seem to double the digits quadruple the time. And that is still linear or?


From jonas.thornvall at gmail.com  Tue Jun 30 10:13:31 2015
From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com)
Date: Tue, 30 Jun 2015 07:13:31 -0700 (PDT)
Subject: Linear time baseconversion
In-Reply-To: <332f7cc7-4cc7-469f-a1c3-cbba40b7b641@googlegroups.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
 <c92c3dc9-8a00-4244-bbd7-383f34983935@googlegroups.com>
 <0439e7f6-2604-4a07-802b-81ea03782f62@googlegroups.com>
 <332f7cc7-4cc7-469f-a1c3-cbba40b7b641@googlegroups.com>
Message-ID: <84ac9885-f0f2-47fb-92bc-a856ede6d818@googlegroups.com>

Den tisdag 30 juni 2015 kl. 15:22:44 UTC+2 skrev jonas.t... at gmail.com:
> Den tisdag 30 juni 2015 kl. 11:43:55 UTC+2 skrev jonas.t... at gmail.com:
> > Den tisdag 30 juni 2015 kl. 11:35:06 UTC+2 skrev jonas.t... at gmail.com:
> > > Den tisdag 30 juni 2015 kl. 11:08:01 UTC+2 skrev Christian Gollwitzer:
> > > > Am 30.06.15 um 10:52 schrieb jonas.thornvall at gmail.com:
> > > > > It still bug out on very big numbers if base outside integer scope.
> > > > > I am very keen on suggestions regarding the logic to make it faster.
> > > > 
> > > > Concerning the algorithmic complexity, it can't be faster than square 
> > > > time in the number of digits N. Baseconversion needs to do a sequence of 
> > > > division operations, where every operation gves you one digit in the new 
> > > > base. The number of digits in the new base is proportional to the number 
> > > > of digits in the old base (the ratio is log b1/log b2). Therefore it 
> > > > will be O(N^2).
> > > > 
> > > > 	Christian
> > > 
> > > Any new digit will be found in SQRT(base) comparissons. 
> > > Averaged case will be in (SQRT(base)*(SQRT(base)+1))/2
> > 
> > Well that averaging was balloney. How do i write that the digit will be found in.
> > Average values from 1 to SQRT(base)?
> 
> Regarding the time it seem to double the digits quadruple the time. And that is still linear or?

2x seem linear to me?


From victorhooi at gmail.com  Tue Jun 30 11:24:56 2015
From: victorhooi at gmail.com (Victor Hooi)
Date: Tue, 30 Jun 2015 08:24:56 -0700 (PDT)
Subject: Parsing logfile with multi-line loglines, separated by timestamp?
Message-ID: <b8916490-20f3-4070-86dc-821adadd895b@googlegroups.com>

Hi,

I'm trying to parse iostat -xt output using Python. The quirk with iostat is that the output for each second runs over multiple lines. For example:

06/30/2015 03:09:17 PM
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.03    0.00    0.03    0.00    0.00   99.94

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
xvdap1            0.00     0.04    0.02    0.07     0.30     3.28    81.37     0.00   29.83    2.74   38.30   0.47   0.00
xvdb              0.00     0.00    0.00    0.00     0.00     0.00    11.62     0.00    0.23    0.19    2.13   0.16   0.00
xvdf              0.00     0.00    0.00    0.00     0.00     0.00    10.29     0.00    0.41    0.41    0.73   0.38   0.00
xvdg              0.00     0.00    0.00    0.00     0.00     0.00     9.12     0.00    0.36    0.35    1.20   0.34   0.00
xvdh              0.00     0.00    0.00    0.00     0.00     0.00    33.35     0.00    1.39    0.41    8.91   0.39   0.00
dm-0              0.00     0.00    0.00    0.00     0.00     0.00    11.66     0.00    0.46    0.46    0.00   0.37   0.00

06/30/2015 03:09:18 PM
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    0.50    0.00    0.00   99.50

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
xvdap1            0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
xvdb              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
xvdf              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
xvdg              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
xvdh              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
dm-0              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00

06/30/2015 03:09:19 PM
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00    0.50    0.00    0.00   99.50

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
xvdap1            0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
xvdb              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
xvdf              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
xvdg              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
xvdh              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
dm-0              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00

Essentially I need to parse the output in "chunks", where each chunk is separated by a timestamp.

I was looking at itertools.groupby(), but that doesn't seem to quite do what I want here - it seems more for grouping lines, where each is united by a common key, or something that you can use a function to check for.

Another thought was something like:

    for line in f:
        if line.count("/") == 2 and line.count(":") == 2:
            current_time = datetime.strptime(line.strip(), '%m/%d/%y %H:%M:%S')
        while line.count("/") != 2 and line.count(":") != 2:
            print(line)
            continue

But that didn't quite seem to work.

Is there a Pythonic way of parsing the above iostat output, and break it into chunks split by the timestamp?

Cheers,
Victor


From ian.g.kelly at gmail.com  Tue Jun 30 11:29:39 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 30 Jun 2015 09:29:39 -0600
Subject: Linear time baseconversion
In-Reply-To: <84ac9885-f0f2-47fb-92bc-a856ede6d818@googlegroups.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
 <c92c3dc9-8a00-4244-bbd7-383f34983935@googlegroups.com>
 <0439e7f6-2604-4a07-802b-81ea03782f62@googlegroups.com>
 <332f7cc7-4cc7-469f-a1c3-cbba40b7b641@googlegroups.com>
 <84ac9885-f0f2-47fb-92bc-a856ede6d818@googlegroups.com>
Message-ID: <CALwzidmADFOBz+jrqT9wLR4N01yV4n8FMeTKck+uK5Kz6Bamrg@mail.gmail.com>

On Tue, Jun 30, 2015 at 8:13 AM,  <jonas.thornvall at gmail.com> wrote:
>> Regarding the time it seem to double the digits quadruple the time. And that is still linear or?
>
> 2x seem linear to me?

That's not linear, nor is it "2x". If doubling the size of the input
quadruples the time, then doubling the size of the input twice (i.e.
quadrupling the size of the input) results in the time being increased
by a factor of 16. Double it three times, and the time taken is
increased by a factor of 64. That's quadratic behavior.

If the algorithm were actually linear, then a constant factor of "2x"
wouldn't matter when calculating the growth. Doubling the size of the
input would simply double the time taken.

Of course, this is just an empirical estimation of the asymptotic
complexity of the algorithm, not a proof. Maybe after 10,000 digits it
actually does become linear, although I doubt it. Proving it would
require analysis of the code, and I'm not willing to dig into 500
lines of Javascript just for the sake of it.


From zljubisic at gmail.com  Tue Jun 30 11:34:24 2015
From: zljubisic at gmail.com (zljubisic at gmail.com)
Date: Tue, 30 Jun 2015 08:34:24 -0700 (PDT)
Subject: Python 3 resuma a file download
Message-ID: <9a629cf3-e256-494a-8ff8-3f1f6fc2218c@googlegroups.com>

Hi,

I would like to download a file (http://video.hrt.hr/2906/otv296.mp4)

If the connection is OK, I can download the file with:

import urllib.request
urllib.request.urlretrieve(remote_file, local_file)

Sometimes when I am connected on week wireless (not mine) network I get WinError 10054 exception (windows 7).

When it happens, I would like to resume download instead of doing everything from very beginning.

How to do that?

I read about Range header and chunks, but this server doesn't have any headers.

What options do I have with this particular file?

Regards.


From ian.g.kelly at gmail.com  Tue Jun 30 11:40:32 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 30 Jun 2015 09:40:32 -0600
Subject: Linear time baseconversion
In-Reply-To: <mmtm6k$8b7$1@dont-email.me>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
Message-ID: <CALwzidkHuWJkqFEDHp6LT2UJxHh6sHZM=2qde0acrSvp12vyRw@mail.gmail.com>

On Tue, Jun 30, 2015 at 3:07 AM, Christian Gollwitzer <auriocus at gmx.de> wrote:
> Am 30.06.15 um 10:52 schrieb jonas.thornvall at gmail.com:
>>
>> It still bug out on very big numbers if base outside integer scope.
>> I am very keen on suggestions regarding the logic to make it faster.
>
>
> Concerning the algorithmic complexity, it can't be faster than square time
> in the number of digits N. Baseconversion needs to do a sequence of division
> operations, where every operation gves you one digit in the new base. The
> number of digits in the new base is proportional to the number of digits in
> the old base (the ratio is log b1/log b2). Therefore it will be O(N^2).

I don't think that's true. Here's a linear hexadecimal to binary function:

>>> def hextobin(value):
...     digits = {'0': '0000', '1': '0001', '2': '0010', '3': '0011',
...         '4': '0100', '5': '0101', '6': '0110', '7': '0111',
...         '8': '1000', '9': '1001', 'A': '1010', 'B': '1011',
...         'C': '1100', 'D': '1101', 'E': '1110', 'F': '1111'}
...     return ''.join(digits[d.upper()] for d in value)
...
>>> hextobin('3f')
'00111111'

I believe this approach can be extended to arbitrary bases with some
effort, although for converting arbitrary base b1 to b2, you would
need up to b2 different mappings if b1 and b2 are relatively prime.


From ian.g.kelly at gmail.com  Tue Jun 30 11:45:05 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 30 Jun 2015 09:45:05 -0600
Subject: Linear time baseconversion
In-Reply-To: <CALwzidkHuWJkqFEDHp6LT2UJxHh6sHZM=2qde0acrSvp12vyRw@mail.gmail.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
 <CALwzidkHuWJkqFEDHp6LT2UJxHh6sHZM=2qde0acrSvp12vyRw@mail.gmail.com>
Message-ID: <CALwzidnTP_2k-m0iRn5=nEHmNzQoDDw45g58gDXXa9kYhJnbiQ@mail.gmail.com>

On Tue, Jun 30, 2015 at 9:40 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Tue, Jun 30, 2015 at 3:07 AM, Christian Gollwitzer <auriocus at gmx.de> wrote:
>> Am 30.06.15 um 10:52 schrieb jonas.thornvall at gmail.com:
>>>
>>> It still bug out on very big numbers if base outside integer scope.
>>> I am very keen on suggestions regarding the logic to make it faster.
>>
>>
>> Concerning the algorithmic complexity, it can't be faster than square time
>> in the number of digits N. Baseconversion needs to do a sequence of division
>> operations, where every operation gves you one digit in the new base. The
>> number of digits in the new base is proportional to the number of digits in
>> the old base (the ratio is log b1/log b2). Therefore it will be O(N^2).
>
> I don't think that's true. Here's a linear hexadecimal to binary function:
>
>>>> def hextobin(value):
> ...     digits = {'0': '0000', '1': '0001', '2': '0010', '3': '0011',
> ...         '4': '0100', '5': '0101', '6': '0110', '7': '0111',
> ...         '8': '1000', '9': '1001', 'A': '1010', 'B': '1011',
> ...         'C': '1100', 'D': '1101', 'E': '1110', 'F': '1111'}
> ...     return ''.join(digits[d.upper()] for d in value)
> ...
>>>> hextobin('3f')
> '00111111'
>
> I believe this approach can be extended to arbitrary bases with some
> effort, although for converting arbitrary base b1 to b2, you would
> need up to b2 different mappings if b1 and b2 are relatively prime.

Actually, I think you need up to b1 * b2 mappings, as you're
effectively building a state machine with b1 * b2 states. The mappings
can be pre-computed, however, so actually running the state machine
would then just be a linear algorithm.


From skip.montanaro at gmail.com  Tue Jun 30 11:47:16 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 30 Jun 2015 10:47:16 -0500
Subject: Parsing logfile with multi-line loglines, separated by timestamp?
In-Reply-To: <b8916490-20f3-4070-86dc-821adadd895b@googlegroups.com>
References: <b8916490-20f3-4070-86dc-821adadd895b@googlegroups.com>
Message-ID: <CANc-5Uw-w8pLssy0usPSpnVEPuHJQCz6RJyNfsiHNUwP=KpqpQ@mail.gmail.com>

Maybe define a class which wraps a file-like object. Its next() method (or
is it __next__() method?) can just buffer up lines starting with one which
successfully parses as a timestamp, accumulates all the rest, until a blank
line or EOF is seen, then return that, either as a list of strings, one
massive string, or some higher level representation (presumably an instance
of another class) which represents one "paragraph" of iostat output.

Skip


On Tue, Jun 30, 2015 at 10:24 AM, Victor Hooi <victorhooi at gmail.com> wrote:

> Hi,
>
> I'm trying to parse iostat -xt output using Python. The quirk with iostat
> is that the output for each second runs over multiple lines. For example:
>
> 06/30/2015 03:09:17 PM
> avg-cpu:  %user   %nice %system %iowait  %steal   %idle
>            0.03    0.00    0.03    0.00    0.00   99.94
>
> Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz
> avgqu-sz   await r_await w_await  svctm  %util
> xvdap1            0.00     0.04    0.02    0.07     0.30     3.28
> 81.37     0.00   29.83    2.74   38.30   0.47   0.00
> xvdb              0.00     0.00    0.00    0.00     0.00     0.00
> 11.62     0.00    0.23    0.19    2.13   0.16   0.00
> xvdf              0.00     0.00    0.00    0.00     0.00     0.00
> 10.29     0.00    0.41    0.41    0.73   0.38   0.00
> xvdg              0.00     0.00    0.00    0.00     0.00     0.00
>  9.12     0.00    0.36    0.35    1.20   0.34   0.00
> xvdh              0.00     0.00    0.00    0.00     0.00     0.00
> 33.35     0.00    1.39    0.41    8.91   0.39   0.00
> dm-0              0.00     0.00    0.00    0.00     0.00     0.00
> 11.66     0.00    0.46    0.46    0.00   0.37   0.00
>
> 06/30/2015 03:09:18 PM
> avg-cpu:  %user   %nice %system %iowait  %steal   %idle
>            0.00    0.00    0.50    0.00    0.00   99.50
>
> Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz
> avgqu-sz   await r_await w_await  svctm  %util
> xvdap1            0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
> xvdb              0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
> xvdf              0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
> xvdg              0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
> xvdh              0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
> dm-0              0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
>
> 06/30/2015 03:09:19 PM
> avg-cpu:  %user   %nice %system %iowait  %steal   %idle
>            0.00    0.00    0.50    0.00    0.00   99.50
>
> Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz
> avgqu-sz   await r_await w_await  svctm  %util
> xvdap1            0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
> xvdb              0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
> xvdf              0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
> xvdg              0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
> xvdh              0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
> dm-0              0.00     0.00    0.00    0.00     0.00     0.00
>  0.00     0.00    0.00    0.00    0.00   0.00   0.00
>
> Essentially I need to parse the output in "chunks", where each chunk is
> separated by a timestamp.
>
> I was looking at itertools.groupby(), but that doesn't seem to quite do
> what I want here - it seems more for grouping lines, where each is united
> by a common key, or something that you can use a function to check for.
>
> Another thought was something like:
>
>     for line in f:
>         if line.count("/") == 2 and line.count(":") == 2:
>             current_time = datetime.strptime(line.strip(), '%m/%d/%y
> %H:%M:%S')
>         while line.count("/") != 2 and line.count(":") != 2:
>             print(line)
>             continue
>
> But that didn't quite seem to work.
>
> Is there a Pythonic way of parsing the above iostat output, and break it
> into chunks split by the timestamp?
>
> Cheers,
> Victor
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150630/35404868/attachment.html>

From rosuav at gmail.com  Tue Jun 30 12:02:41 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 1 Jul 2015 02:02:41 +1000
Subject: Parsing logfile with multi-line loglines, separated by timestamp?
In-Reply-To: <CANc-5Uw-w8pLssy0usPSpnVEPuHJQCz6RJyNfsiHNUwP=KpqpQ@mail.gmail.com>
References: <b8916490-20f3-4070-86dc-821adadd895b@googlegroups.com>
 <CANc-5Uw-w8pLssy0usPSpnVEPuHJQCz6RJyNfsiHNUwP=KpqpQ@mail.gmail.com>
Message-ID: <CAPTjJmqDA=qMs-=6s9oG-GainJ3cdwjfAdk9uos4Z_RwhRU5hw@mail.gmail.com>

On Wed, Jul 1, 2015 at 1:47 AM, Skip Montanaro <skip.montanaro at gmail.com> wrote:
> Maybe define a class which wraps a file-like object. Its next() method (or
> is it __next__() method?) can just buffer up lines starting with one which
> successfully parses as a timestamp, accumulates all the rest, until a blank
> line or EOF is seen, then return that, either as a list of strings, one
> massive string, or some higher level representation (presumably an instance
> of another class) which represents one "paragraph" of iostat output.

next() in Py2, __next__() in Py3. But I'd do it, instead, as a
generator - that takes care of all the details, and you can simply
yield useful information whenever you have it. Something like this
(untested):

def parse_iostat(lines):
    """Parse lines of iostat information, yielding ... something

    lines should be an iterable yielding separate lines of output
    """
    block = None
    for line in lines:
        line = line.strip()
        try:
            tm = datetime.datetime.strptime(line, "%m/%d/%Y %I:%M:%S %p")
            if block: yield block
            block = [tm]
        except ValueError:
            # It's not a new timestamp, so add it to the existing block
            block.append(line)
    if block: yield block

This is a fairly classic line-parsing generator. You can pass it a
file-like object, a list of strings, or anything else that it can
iterate over; it'll yield some sort of aggregate object representing
each time's block. In this case, all it does is append strings to a
list, so this will result in a series of lists of strings, each one
representing a single timestamp; you can parse the other lines in any
way you like and aggregate useful data. Usage would be something like
this:

with open("logfile") as f:
    for block in parse_iostat(f):
        # do stuff with block

This will work quite happily with an ongoing stream, too, so if you're
working with a pipe from a currently-running process, it'll pick stuff
up just fine. (However, since it uses the timestamp as its signature,
it won't yield anything till it gets the *next* timestamp. If the
blank line is sufficient to denote the end of a block, you could
change the loop to look for that instead.)

Hope that helps!

ChrisA


From rosuav at gmail.com  Tue Jun 30 12:10:15 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 1 Jul 2015 02:10:15 +1000
Subject: Linear time baseconversion
In-Reply-To: <CALwzidnTP_2k-m0iRn5=nEHmNzQoDDw45g58gDXXa9kYhJnbiQ@mail.gmail.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
 <CALwzidkHuWJkqFEDHp6LT2UJxHh6sHZM=2qde0acrSvp12vyRw@mail.gmail.com>
 <CALwzidnTP_2k-m0iRn5=nEHmNzQoDDw45g58gDXXa9kYhJnbiQ@mail.gmail.com>
Message-ID: <CAPTjJmrSp3BCyaJuO=m-k97XH2r_XORNDTPpVam=5YwEObYpOg@mail.gmail.com>

On Wed, Jul 1, 2015 at 1:45 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Tue, Jun 30, 2015 at 9:40 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> On Tue, Jun 30, 2015 at 3:07 AM, Christian Gollwitzer <auriocus at gmx.de> wrote:
>>> Am 30.06.15 um 10:52 schrieb jonas.thornvall at gmail.com:
>>>>
>>>> It still bug out on very big numbers if base outside integer scope.
>>>> I am very keen on suggestions regarding the logic to make it faster.
>>>
>>>
>>> Concerning the algorithmic complexity, it can't be faster than square time
>>> in the number of digits N. Baseconversion needs to do a sequence of division
>>> operations, where every operation gves you one digit in the new base. The
>>> number of digits in the new base is proportional to the number of digits in
>>> the old base (the ratio is log b1/log b2). Therefore it will be O(N^2).
>>
>> I don't think that's true. Here's a linear hexadecimal to binary function:
>>
>>>>> def hextobin(value):
>> ...     digits = {'0': '0000', '1': '0001', '2': '0010', '3': '0011',
>> ...         '4': '0100', '5': '0101', '6': '0110', '7': '0111',
>> ...         '8': '1000', '9': '1001', 'A': '1010', 'B': '1011',
>> ...         'C': '1100', 'D': '1101', 'E': '1110', 'F': '1111'}
>> ...     return ''.join(digits[d.upper()] for d in value)
>> ...
>>>>> hextobin('3f')
>> '00111111'
>>
>> I believe this approach can be extended to arbitrary bases with some
>> effort, although for converting arbitrary base b1 to b2, you would
>> need up to b2 different mappings if b1 and b2 are relatively prime.
>
> Actually, I think you need up to b1 * b2 mappings, as you're
> effectively building a state machine with b1 * b2 states. The mappings
> can be pre-computed, however, so actually running the state machine
> would then just be a linear algorithm.

Arbitrary base conversion has to be stateful. You can take a shortcut
like this when the bases are related (eg binary, octal, hexadecimal),
but otherwise, you need the division. Consider what happens when you
convert the binary digit "1" to decimal, and then follow it with
varying numbers of zeros:

1, 2, 4, 8, 16, 32, 64,... 32768, 65536, 131072,... 1048576,... 1073741824,...

You can certainly do some useful analyses on the last digits (they'll
never be anything other than 2, 4, 8, 6, except for the special case
of 1 itself), but there's a lot of variation in the intermediate
digits.

When there's a simple ratio between the bases, it's fairly
straight-forward to convert a few digits at a time. Converting base
256 into base 64, for instance, can be done by taking three digits and
yielding four. But within that, you would still need a complete table
of all sixteen million possibilities, if you want to do the lookup
table. And that only works when there is that kind of relationship.

ChrisA


From torriem at gmail.com  Tue Jun 30 12:12:22 2015
From: torriem at gmail.com (Michael Torrie)
Date: Tue, 30 Jun 2015 10:12:22 -0600
Subject: Linear time baseconversion
In-Reply-To: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
Message-ID: <5592BFE6.9060409@gmail.com>

Do you have some Python code to show us?



From jonas.thornvall at gmail.com  Tue Jun 30 12:24:19 2015
From: jonas.thornvall at gmail.com (jonas.thornvall at gmail.com)
Date: Tue, 30 Jun 2015 09:24:19 -0700 (PDT)
Subject: Linear time baseconversion
In-Reply-To: <mailman.195.1435680754.3674.python-list@python.org>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <mailman.195.1435680754.3674.python-list@python.org>
Message-ID: <ff9ca195-e15b-4dc0-87a7-d502251df03f@googlegroups.com>

Den tisdag 30 juni 2015 kl. 18:12:46 UTC+2 skrev Michael Torrie:
> Do you have some Python code to show us?

No i just thought you would find the digit search algorithm interesting.


From marko at pacujo.net  Tue Jun 30 12:32:06 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 30 Jun 2015 19:32:06 +0300
Subject: enumerate XML tags (keys that will become headers) along with
 text (values) and write to CSV in one row (as opposed to "stacked" values
 with one header)
References: <14aeae7a-41ab-4619-8331-7995e2420e54@googlegroups.com>
 <mmivtd$fqa$1@dont-email.me>
 <mailman.150.1435477605.3674.python-list@python.org>
 <mmpn9p$667$1@dont-email.me>
 <8e7d3dd8-f7c1-4466-ae82-f25e4a252fc7@googlegroups.com>
 <mmspbt$88c$1@dont-email.me>
 <mailman.184.1435655772.3674.python-list@python.org>
Message-ID: <87ioa5kvi1.fsf@elektro.pacujo.net>

Robert Kern <robert.kern at gmail.com>:

>> <tuple>
>>    <item>tuplemember_1</item>
>>    <item>tuplemember_2</item>
>> ....
>>    <item>tuplemember_n/item>
>> </tuple>
>
> [...]
>
> Yes! Any conforming XML implementation will preserve the order.

And not only the order: the newlines and other whitespace around the
<item>s are also preserved as children of <tuple>.

> And if you are completely blind as to the schema as the OP apparently
> is, and you are simply given a load of XML and told to do "something"
> with it, you don't know if any given collection is meant to be ordered
> or unordered. Of course, the only sensible thing to do is just
> preserve the order given to you as that is what the semantics of XML
> requires of you in the absence of a schema that says otherwise.

XML is an unfortunate creation. You cannot fully parse it without
knowing the schema (or document type). For example, these constructs
might or might not be equivalent:

    <stuff>&aring;</stuff>

    <stuff>?</stuff>

That is because &...; entities are defined in the document type.

Similarly, these two constructs might or might not be equivalent:

   <stuff greet="hello"/>

   <stuff greet=" hello "/>

By default, they would be, but the document type can declare whitespce
significant around an attribute value.

Finally, these two constructs might or might not be equivalent:

   <count> 7 </count>
   <count>7</count>

By default, they wouldn't be, but the document type can declare
whitespace *insignificant* around element contents.

Sigh, S-expressions would have been a much better universal data format.


Marko


From ian.g.kelly at gmail.com  Tue Jun 30 12:34:06 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 30 Jun 2015 10:34:06 -0600
Subject: Linear time baseconversion
In-Reply-To: <CAPTjJmrSp3BCyaJuO=m-k97XH2r_XORNDTPpVam=5YwEObYpOg@mail.gmail.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
 <CALwzidkHuWJkqFEDHp6LT2UJxHh6sHZM=2qde0acrSvp12vyRw@mail.gmail.com>
 <CALwzidnTP_2k-m0iRn5=nEHmNzQoDDw45g58gDXXa9kYhJnbiQ@mail.gmail.com>
 <CAPTjJmrSp3BCyaJuO=m-k97XH2r_XORNDTPpVam=5YwEObYpOg@mail.gmail.com>
Message-ID: <CALwzidnqR9g4vXZic2Z7h92JPSUtWOQKJLLiFOCiZtU7kPD5Gw@mail.gmail.com>

On Tue, Jun 30, 2015 at 10:10 AM, Chris Angelico <rosuav at gmail.com> wrote:
> When there's a simple ratio between the bases, it's fairly
> straight-forward to convert a few digits at a time. Converting base
> 256 into base 64, for instance, can be done by taking three digits and
> yielding four. But within that, you would still need a complete table
> of all sixteen million possibilities, if you want to do the lookup
> table. And that only works when there is that kind of relationship.

You're right. I was thinking that for base 5 to base 7, for instance,
one could read digits in groups of 7, but that doesn't work out; you
can't map any discrete number of base 5 digits to a corresponding
number of base 7 digits.


From pkpearson at nowhere.invalid  Tue Jun 30 12:42:04 2015
From: pkpearson at nowhere.invalid (Peter Pearson)
Date: 30 Jun 2015 16:42:04 GMT
Subject: Matplotlib X-axis timezone trouble
References: <cve7ppFk1viU1@mid.individual.net>
 <mailman.179.1435630294.3674.python-list@python.org>
 <cvele1Fmra9U1@mid.individual.net>
 <mailman.181.1435647684.3674.python-list@python.org>
Message-ID: <cvfv6rF2irmU1@mid.individual.net>

On Tue, 30 Jun 2015 17:01:15 +1000, Chris Angelico <rosuav at gmail.com> wrote:
> On Tue, Jun 30, 2015 at 2:49 PM, Peter Pearson
><pkpearson at nowhere.invalid> wrote:
>> Time zones teem with sneaky software problems, and so does daylight-saving
>> time, so this problem might strain my brain.  Maybe it's going to turn
>> out that my expectations are unreasonable . . . as in, "Well, smarty pants,
>> how do you want the axis labelled when the abscissas straddle the
>> beginning of daylight-saving time?"  I'll research and digest.
>
> That's entirely possible. Even more so if you go across some other
> civil time change - if you look at the history of timezones in tzdata,
> there's no end of messes as different places adopted standard time,
> redefined standard time, and unified with someone else's standard
> time. And some of that happened relatively recently.
>
> UTC is much easier for this kind of thing. Especially if the
> granularity of your data lets you ignore leap seconds.

Hear, hear!  I'm an enthusiastic fan of UTC.  But I need to interact
with nontechnical people who will say, "Hey, I logged that widget at
9AM, why do you show it as 4PM?"  Another funny thing that happens in
the pure-UTC world is my software gets confused by the way Frank
abruptly shifts his logging schedule by an hour ... twice a year ...
when DST starts or stops.

I'm just glad I don't have to worry about the distinctions among 
UTC, GMT, TAI, and UT1.

-- 
To email me, substitute nowhere->runbox, invalid->com.


From rosuav at gmail.com  Tue Jun 30 12:55:56 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 1 Jul 2015 02:55:56 +1000
Subject: Matplotlib X-axis timezone trouble
In-Reply-To: <cvfv6rF2irmU1@mid.individual.net>
References: <cve7ppFk1viU1@mid.individual.net>
 <mailman.179.1435630294.3674.python-list@python.org>
 <cvele1Fmra9U1@mid.individual.net>
 <mailman.181.1435647684.3674.python-list@python.org>
 <cvfv6rF2irmU1@mid.individual.net>
Message-ID: <CAPTjJmrB=Q1HxpUuBhG+OJHaSqUHS0UFMDboav9x31R2Mie0JA@mail.gmail.com>

On Wed, Jul 1, 2015 at 2:42 AM, Peter Pearson <pkpearson at nowhere.invalid> wrote:
> I'm just glad I don't have to worry about the distinctions among
> UTC, GMT, TAI, and UT1.

Fortunately, that's often the case. GMT can be ignored, and the other
three differ by less seconds than most humans ever care about. If
you're scheduling a meeting, for instance, nobody's going to care if
your clocks differ by 36 seconds, which is as much as TAI and UTC will
differ in a few hours (there's a leap second scheduled for midnight at
the end of June 30th this year, and as my clock is now showing July,
it'll be happening soon). The difference between UTC and UT1 is
insignificant to virtually all humans, given that it's never more than
a single second; it's only high-precision calculations that ever need
concern themselves with that. So, yep! I share your gladness.
Distinctions exist but most of us don't have to worry about them.

ChrisA


From randall at tnr.cc  Tue Jun 30 13:39:30 2015
From: randall at tnr.cc (Randall Smith)
Date: Tue, 30 Jun 2015 12:39:30 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <5592065e$0$1675$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>
 <mailman.171.1435611181.3674.python-list@python.org>
 <5592065e$0$1675$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mmuk8t$akj$1@ger.gmane.org>

On 06/29/2015 10:00 PM, Steven D'Aprano wrote:
> On Tue, 30 Jun 2015 06:52 am, Randall Smith wrote:
>
>> Not sure why you posted the link.  The crc32 checksum is just to check
>> for possible filesystem corruption.  The system does periodic data
>> corruption checks.  BTRFS uses crc32 checksums also.  Please explain.
>
> The file system can trust that anything writing to a file is allowed to
> write to it, in doesn't have to defend against malicious writes. As I
> understand it, your application does.
>
> Here is the attack scenario I have in mind:
>
> - you write a file to my computer, and neglect to encrypt it;
> - and record the checksum for later;
> - I insert malware into your file;
> - you retrieve the file from me;
> - if the checksum matches what you have on record, you accept the file;
> - since you are using CRC, it is quite easy for me to ensure the
>    checksums match after inserting malware;
> - and I have now successfully injected malware into your computer.
>
> I'm making an assumption here -- I assume that the sender records a checksum
> for uploaded files so that when they get something back again they can tell
> whether or not it is the same content they uploaded.

Yes.  The client software computes sha256 checksums.

>
> * * *
>
> By the way, regarding the use of a substitution cipher, I spoke to the
> crypto guy at work, and "preimage attack" is not quite the right
> terminology, since that's normally used in the context of hash functions.
> It's almost a "known ciphertext attack", but not quite, since that
> terminology refers to guessing the key from the ciphertext.
>
> I was wrong: cryptographically strong ciphers are generally NOT resistant to
> what I described as a preimage attack. If the key leaks, using AES won't
> save you: an attacker with access to the key can produce a ciphertext that
> decrypts to the malware of his choice, regardless of whether you use
> AES-256 or rot-13. There may be other encryption methods which don't suffer
> from that, but he doesn't know of them off the top of his head.
>
> His comment was, "don't leak the key".

I'm pretty sure all encryption hinges on guarding the key.

>
> The other threat I mentioned is that the receiver will read the content of
> the file. For that, a strong cipher is much to be preferred over a weak
> one, and it needs to be encrypted by the sending end, not the receiving
> end. (If the receiving end does it, it has to keep the key so it can
> decrypt before sending back, which means the computer's owner can just grab
> the key and read the files.)
>


And again, that's why the client (data owner) uses AES.




From randall at tnr.cc  Tue Jun 30 13:43:11 2015
From: randall at tnr.cc (Randall Smith)
Date: Tue, 30 Jun 2015 12:43:11 -0500
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <slrnmp3bsk.1nu.jon+usenet@frosty.unequivocal.co.uk>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <slrnmp00qf.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.170.1435610818.3674.python-list@python.org>
 <slrnmp3bsk.1nu.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <mmukfp$e1j$1@ger.gmane.org>

On 06/29/2015 03:49 PM, Jon Ribbens wrote:
> On 2015-06-29, Randall Smith <randall at tnr.cc> wrote:
>> Same reason newer filesystems like BTRFS use checkusms (BTRFS uses
>> CRC32).  The storage machine runs periodic file integrity checks.  It
>> has no control over the underlying filesystem.
>
> True, but presumably neither does it have anything it can do to
> rectify the situation if it finds a problem, and the client will
> have to keep its own secure hash of its file anyway. (Unless I suppose
> the server actually can request a new copy from the client or another
> server if it finds a corrupt file?)
>

Yes.  The storage servers are monitored for integrity.  They can request 
a new copy, though frequent corruption results in the server being 
marked as unreliable.

-Randall



From steve at pearwood.info  Tue Jun 30 14:17:24 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 01 Jul 2015 04:17:24 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>
 <mailman.171.1435611181.3674.python-list@python.org>
 <5592065e$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <slrnmp52ct.1nu.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <5592dd34$0$1675$c3e8da3$5496439d@news.astraweb.com>

On Tue, 30 Jun 2015 10:19 pm, Jon Ribbens wrote:

> On 2015-06-30, Steven D'Aprano <steve at pearwood.info> wrote:
>> On Tue, 30 Jun 2015 06:52 am, Randall Smith wrote:
>>> Not sure why you posted the link.  The crc32 checksum is just to check
>>> for possible filesystem corruption.  The system does periodic data
>>> corruption checks.  BTRFS uses crc32 checksums also.  Please explain.
>>
>> The file system can trust that anything writing to a file is allowed to
>> write to it, in doesn't have to defend against malicious writes. As I
>> understand it, your application does.
>>
>> Here is the attack scenario I have in mind:
>>
>> - you write a file to my computer, and neglect to encrypt it;
> 
> Eh? The game is over right there. I don't trust you, and yet
> I have just given you my private data, unencrypted.

Yes. That is exactly the problem. If the application doesn't encrypt the
data for me, *it isn't going to happen*. We are in violent agreement that
the sender needs to encrypt the data.

I think Randall has been somewhat less than clear about what the application
actually does and how it works. He probably thinks he doesn't need to
explain, that its none of our business, and wishes we'd just shut up about
it. That's his right.

It's also my right to discuss the possible security implications of some
hypothetical peer-to-peer dropbox-like application which may, or may not,
be similar to Randall's application. Whether Randall learns anything from
that discussion, or just tunes it out, is irrelevant. I've already learned
at least one thing from this discussion, so as far as I'm concerned it's a
win.

Randall has suggested that encryption is optional. It isn't clear whether he
means there is an option to turn encryption off, or whether he means I can
hack the application and disable it, or write my own application. I don't
expect him to be responsible for rogue applications that have been hacked
or written independently, which (out of malice or stupidity) don't encrypt
the uploaded data. But I think that it is foolish to support an unencrypted
mode of operation.

It's not unreasonable to raise this issue. The default state of security
among IT professionals is something worse than awful:

https://medium.com/message/everything-is-broken-81e5f33a24e1

One of Australia's largest ISPs recently was hacked, and guess how they
stored their customer's passwords? Yes, you got it: in plain text. There is
no security mistake so foolish that IT professionals won't make it.


> Checksums don't even come into it, we have failed utterly at step 1.

*shrug*

You're right. But having failed at step 1, there are multiple attacks that
can follow. The first attack is the obvious one: the ability to read the
unencrypted data.

If you can trick me into turning encryption off (say, you use a social
engineering attack on me and convince me to delete "the virus crypto.py"),
then I might inadvertently upload unencrypted data to you. Or maybe you
find an attack on the application that can fool it into dropping down to
unencrypted mode. If there's no unencrypted mode in the first place, that's
much harder.

Earlier, Chris suggested that the application might choose to import the
crypto module, and if it's not available, just keep working without
encryption. This hypothetical attack demonstrates that this would be a
mistake. It's hard for an attacker to convince a naive user to open up the
application source code and edit the code. It's easier to convince them to
delete a file.

Or, the application just has a bug in it. It accidentally flips the sense of
the "use encryption" flag. That's a failure mode that simply cannot occur
if there is no such flag in the first place.

If our attacker has managed to disable encryption in the sender's
application, then they can not only read my data, but tamper with it. These
are *separate attacks* with the same underlying cause. I can mitigate one
without mitigating the other.

We can mitigate against the second attack by using a cryptographically
strong hash function to detect tampering. These *are* resistant to preimage
attacks. If I give you a SHA512 checksum, there is no known practical
method to generate a file with that same checksum. If I give you a CRC
checksum, you can.

(Naturally the checksum has to be under the sender's control. If the
receiver has the checksum and the data, they can just replace the checksum
with one of their choosing.)

That's a separate issue from detecting non-malicious data corruption,
although of course a SHA512 checksum will detect that as well.


>> - since you are using CRC, it is quite easy for me to ensure the
>>   checksums match after inserting malware;
> 
> No, you have yet *again* misunderstood the difference between the
> client and the server.

This was described as a peer-to-peer application. You even stated that it
was a "pretty obvious" use-case, a "peer-to-peer dropbox". So is it
peer-to-peer or client-server?

In any case, since Randall has refused to go into specific details of how
his application works, as is his right, we can treat this as an
intellectual exercise for how it *might* work. If such an application keeps
a checksum of the files uploaded, it should be one which can detect
deliberate tampering and not just accidental corruption.


>> I was wrong: cryptographically strong ciphers are generally NOT resistant
>> to what I described as a preimage attack. If the key leaks, using AES
>> won't save you: an attacker with access to the key can produce a
>> ciphertext that decrypts to the malware of his choice, regardless of
>> whether you use AES-256 or rot-13. There may be other encryption methods
>> which don't suffer from that, but he doesn't know of them off the top of
>> his head.
> 
> lol. I suspected as much. You and Johannes were even more wrong than
> was already obvious.

You "suspected as much"? Such a pity you didn't speak up earlier and explain
that cryptographic ciphers aren't generally resistant to preimage attacks.

In any case, I have learned something from this discussion, which makes it
worthwhile.



-- 
Steven



From rosuav at gmail.com  Tue Jun 30 14:33:56 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 1 Jul 2015 04:33:56 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <5592dd34$0$1675$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>
 <mailman.171.1435611181.3674.python-list@python.org>
 <5592065e$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <slrnmp52ct.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <5592dd34$0$1675$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmpx_UJ6CMKOVoo+fu4TggSZ_WX4gdJ=dSWM7dwzBtthXA@mail.gmail.com>

On Wed, Jul 1, 2015 at 4:17 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> If you can trick me into turning encryption off (say, you use a social
> engineering attack on me and convince me to delete "the virus crypto.py"),
> then I might inadvertently upload unencrypted data to you. Or maybe you
> find an attack on the application that can fool it into dropping down to
> unencrypted mode. If there's no unencrypted mode in the first place, that's
> much harder.
>
> Earlier, Chris suggested that the application might choose to import the
> crypto module, and if it's not available, just keep working without
> encryption. This hypothetical attack demonstrates that this would be a
> mistake. It's hard for an attacker to convince a naive user to open up the
> application source code and edit the code. It's easier to convince them to
> delete a file.

And I'm sure Steven knows about this, but if anyone else isn't
convinced that this is a serious vulnerability, look into various
forms of downgrade attack, such as the recent POODLE. Security doesn't
exist if an attacker can convince your program to turn it off without
your knowledge.

>>> - since you are using CRC, it is quite easy for me to ensure the
>>>   checksums match after inserting malware;
>>
>> No, you have yet *again* misunderstood the difference between the
>> client and the server.
>
> This was described as a peer-to-peer application. You even stated that it
> was a "pretty obvious" use-case, a "peer-to-peer dropbox". So is it
> peer-to-peer or client-server?

I've never managed to get any sort of grasp of what this application
actually *is*, but "peer-to-peer Dropbox" is certainly something that
it *might be*. It could be simultaneously peer-to-peer from the
human's point of view, and client-server from the application's -
imagine BitTorrent protocol, but where one end connects to a socket
that the other's listening on, and the active socket always pushes
data to the passive socket. (With BitTorrent, it's truly symmetrical -
doesn't matter who listens and who connects. But imagine if it weren't
that way.) From the software's point of view, it has two distinct
modes: server, in which it listens on a socket and receives data, and
client, in which it connects to other people's sockets and sends data.
As such, the "server" mode is the only one that receives untrusted
data from another user and stores it on the hard disk.

But this is just one theory of what the program *might* be, based on
what I've gathered in this thread. Or rather, it's a vague theory of
something that's mostly plausible, without necessarily even being
useful.

ChrisA


From jon+usenet at unequivocal.co.uk  Tue Jun 30 14:37:37 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 30 Jun 2015 18:37:37 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>
 <mailman.171.1435611181.3674.python-list@python.org>
 <5592065e$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <slrnmp52ct.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <5592dd34$0$1675$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <slrnmp5oi4.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-30, Steven D'Aprano <steve at pearwood.info> wrote:
> On Tue, 30 Jun 2015 10:19 pm, Jon Ribbens wrote:
>> Eh? The game is over right there. I don't trust you, and yet
>> I have just given you my private data, unencrypted.
>
> Yes. That is exactly the problem. If the application doesn't encrypt the
> data for me, *it isn't going to happen*. We are in violent agreement that
> the sender needs to encrypt the data.

It's a good thing that he's said it will then.

> Randall has suggested that encryption is optional.

No he hasn't. You just keep creatively misreading what he says, for
some reason.

> It's not unreasonable to raise this issue.

It is unreasonable to raise it over and over again however,
especially when there's no reason at all to think it's relevant,
and nothing has changed from the last time you raised it.

> We can mitigate against the second attack by using a cryptographically
> strong hash function to detect tampering.

Not on the server you can't. If the attacker can edit the files he can
edit the hashes too.

> These *are* resistant to preimage attacks. If I give you a SHA512
> checksum, there is no known practical method to generate a file with
> that same checksum. If I give you a CRC checksum, you can.

Randall didn't suggest any usage of CRCs where preimage attacks are
relevant. You just made that bit up.

>>> - since you are using CRC, it is quite easy for me to ensure the
>>>   checksums match after inserting malware;
>> 
>> No, you have yet *again* misunderstood the difference between the
>> client and the server.
>
> This was described as a peer-to-peer application. You even stated that it
> was a "pretty obvious" use-case, a "peer-to-peer dropbox". So is it
> peer-to-peer or client-server?

Both. It sounds a bit like there are clients which upload files
to a cloud of servers which are peers of each other. But seriously,
is this the source of all your confusion? Even if all the nodes
are pure "peers" (which it doesn't sound like they are), any
particular file will still have a source node which is therefore
the "client" for that file. You're trying to draw a hard distinction
where there is none.

>> lol. I suspected as much. You and Johannes were even more wrong than
>> was already obvious.
>
> You "suspected as much"? Such a pity you didn't speak up earlier and
> explain that cryptographic ciphers aren't generally resistant to
> preimage attacks.

I think you're misusing that phrase. But taking what you meant,
I suspected it was true (would they be reistant, after all?)
but I couldn't be bothered to check because the whole "crypto" bit
was a complete red-herring in the first place. The original discussion
wasn't about crypto, all the discussion about that was only because
you and Johannes wrongly insisted it was necessary.


From steve at pearwood.info  Tue Jun 30 14:59:43 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 01 Jul 2015 04:59:43 +1000
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>
 <mailman.171.1435611181.3674.python-list@python.org>
 <5592065e$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.198.1435685990.3674.python-list@python.org>
Message-ID: <5592e71e$0$1674$c3e8da3$5496439d@news.astraweb.com>

On Wed, 1 Jul 2015 03:39 am, Randall Smith wrote:

> On 06/29/2015 10:00 PM, Steven D'Aprano wrote:

>> I'm making an assumption here -- I assume that the sender records a
>> checksum for uploaded files so that when they get something back again
>> they can tell whether or not it is the same content they uploaded.
> 
> Yes.  The client software computes sha256 checksums.

Thanks for clarifying.


[...]
>> His comment was, "don't leak the key".
> 
> I'm pretty sure all encryption hinges on guarding the key.

That would be Kerckhoffs' Principle, also known as Shannon's Maxim.

I don't think there has been much research into keeping at least *some*
security even when keys have been compromised, apart from as it relates to
two-factor authentication. (Assume that other people know the password to
your bank account. They can read your balance, but they can't steal your
money unless they first steal your phone or RSA token.)

In the past, and still today among people who don't understand Kerckhoffs'
principle, people have tried to keep the cipher secret and not have a key
at all. E.g. atbash, or caesar cipher, which once upon a time were cutting
edge ciphers, as laughably insecure as they are today. If the method was
compromised, all was lost. 

Today, if the key is compromised, all is lost. Is it possible that there are
ciphers that are resistant to discovery of the key? Obviously if you know
the key you can read encrypted messages, that's what the key is for, but
there are scenarios where you would want security to degrade gracefully
instead of in a brittle all-or-nothing manner:

- even if the attacker can read my messages, he cannot tamper with 
  them or write new ones as me.

(I'm pretty sure that, for example, the military would consider it horrible
if the enemy could listen in on their communications, but *even worse* if
the enemy could send false orders that appear to be legitimate.)

Sixty years ago, the idea of having a separate encryption key that you keep
secret and a decryption key that you can give out to everyone (public key
encryption) probably would have seemed ridiculous too.



-- 
Steven



From rosuav at gmail.com  Tue Jun 30 15:20:50 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 1 Jul 2015 05:20:50 +1000
Subject: Pure Python Data Mangling or Encrypting
In-Reply-To: <5592e71e$0$1674$c3e8da3$5496439d@news.astraweb.com>
References: <mmcagn$aa5$1@ger.gmane.org>
 <q67loadve6lsrh672u47omr04tvqttt6jd@4ax.com>
 <mailman.29.1435170987.3674.python-list@python.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>
 <mailman.171.1435611181.3674.python-list@python.org>
 <5592065e$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.198.1435685990.3674.python-list@python.org>
 <5592e71e$0$1674$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmpJesoBtGrNC5UhSSSeriHeZCr4LLm70b+gPkAdyCmkbw@mail.gmail.com>

On Wed, Jul 1, 2015 at 4:59 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Today, if the key is compromised, all is lost. Is it possible that there are
> ciphers that are resistant to discovery of the key? Obviously if you know
> the key you can read encrypted messages, that's what the key is for, but
> there are scenarios where you would want security to degrade gracefully
> instead of in a brittle all-or-nothing manner:
>
> - even if the attacker can read my messages, he cannot tamper with
>   them or write new ones as me.
>
> (I'm pretty sure that, for example, the military would consider it horrible
> if the enemy could listen in on their communications, but *even worse* if
> the enemy could send false orders that appear to be legitimate.)

That would be accomplished by a two-fold enveloping of signing and
encrypting. If I sign something using my private key, then encrypt it
using your public key, someone who's compromised your private key
could snoop and read the message, but couldn't forge a message from
me. Of course, that just means there are lots more secrets to worry
about getting compromised.

ChrisA


From torriem at gmail.com  Tue Jun 30 15:55:26 2015
From: torriem at gmail.com (Michael Torrie)
Date: Tue, 30 Jun 2015 13:55:26 -0600
Subject: Linear time baseconversion
In-Reply-To: <ff9ca195-e15b-4dc0-87a7-d502251df03f@googlegroups.com>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <mailman.195.1435680754.3674.python-list@python.org>
 <ff9ca195-e15b-4dc0-87a7-d502251df03f@googlegroups.com>
Message-ID: <5592F42E.6070006@gmail.com>

On 06/30/2015 10:24 AM, jonas.thornvall at gmail.com wrote:
> Den tisdag 30 juni 2015 kl. 18:12:46 UTC+2 skrev Michael Torrie:
>> Do you have some Python code to show us?
> 
> No i just thought you would find the digit search algorithm interesting.

Yeah it is interesting, although I didn't really see an explanation of
your algorithm in your posts.  Did I miss it?


From auriocus at gmx.de  Tue Jun 30 18:22:01 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Wed, 01 Jul 2015 00:22:01 +0200
Subject: Linear time baseconversion
In-Reply-To: <mailman.193.1435680388.3674.python-list@python.org>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
 <mailman.193.1435680388.3674.python-list@python.org>
Message-ID: <mmv4nm$mi6$1@dont-email.me>

Am 30.06.15 um 17:40 schrieb Ian Kelly:
> On Tue, Jun 30, 2015 at 3:07 AM, Christian Gollwitzer <auriocus at gmx.de> wrote:
>> Concerning the algorithmic complexity, it can't be faster than square time
>> in the number of digits N. Baseconversion needs to do a sequence of division
>> operations, where every operation gves you one digit in the new base. The
>> number of digits in the new base is proportional to the number of digits in
>> the old base (the ratio is log b1/log b2). Therefore it will be O(N^2).
>
> I don't think that's true. Here's a linear hexadecimal to binary function:
>
>>>> def hextobin(value):
> ...     digits = {'0': '0000', '1': '0001', '2': '0010', '3': '0011',
> ...         '4': '0100', '5': '0101', '6': '0110', '7': '0111',
> ...         '8': '1000', '9': '1001', 'A': '1010', 'B': '1011',
> ...         'C': '1100', 'D': '1101', 'E': '1110', 'F': '1111'}
> ...     return ''.join(digits[d.upper()] for d in value)
> ...
>>>> hextobin('3f')
> '00111111'
>
> I believe this approach can be extended to arbitrary bases with some
> effort, although for converting arbitrary base b1 to b2, you would
> need up to b2 different mappings if b1 and b2 are relatively prime.
>

OK. Show it for bases 2 and 3. It will just be a table of 6 entries, no?

Actually, you showed a very special case, for conversion from base b^4 
to base b. I'm pretty convinced it is not possible for the general case.

	Christian



From auriocus at gmx.de  Tue Jun 30 18:27:57 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Wed, 01 Jul 2015 00:27:57 +0200
Subject: Linear time baseconversion
In-Reply-To: <mailman.196.1435682093.3674.python-list@python.org>
References: <777831f0-d4b4-48f6-ae0b-c9b1ea7ffc06@googlegroups.com>
 <CALwzidms2BGRjHCuLB5_uszbx9Q-rdtOob_LGHQoO3K3ZD=Q7Q@mail.gmail.com>
 <mailman.175.1435619500.3674.python-list@python.org>
 <87r3ouawgt.fsf@bsb.me.uk>
 <7c6dac9d-5722-4179-bd7e-ceaac6698490@googlegroups.com>
 <mmtm6k$8b7$1@dont-email.me>
 <CALwzidkHuWJkqFEDHp6LT2UJxHh6sHZM=2qde0acrSvp12vyRw@mail.gmail.com>
 <CALwzidnTP_2k-m0iRn5=nEHmNzQoDDw45g58gDXXa9kYhJnbiQ@mail.gmail.com>
 <CAPTjJmrSp3BCyaJuO=m-k97XH2r_XORNDTPpVam=5YwEObYpOg@mail.gmail.com>
 <mailman.196.1435682093.3674.python-list@python.org>
Message-ID: <mmv52q$nq6$1@dont-email.me>

Am 30.06.15 um 18:34 schrieb Ian Kelly:
> On Tue, Jun 30, 2015 at 10:10 AM, Chris Angelico <rosuav at gmail.com> wrote:
>> When there's a simple ratio between the bases, it's fairly
>> straight-forward to convert a few digits at a time. Converting base
>> 256 into base 64, for instance, can be done by taking three digits and
>> yielding four. But within that, you would still need a complete table
>> of all sixteen million possibilities, if you want to do the lookup
>> table. And that only works when there is that kind of relationship.
>
> You're right. I was thinking that for base 5 to base 7, for instance,
> one could read digits in groups of 7, but that doesn't work out; you
> can't map any discrete number of base 5 digits to a corresponding
> number of base 7 digits.

Yes, because there is no n for which 5^n=7^n (except n=0). This gives 
more-or-less the proof that the algorithm must be O(N^2) - at least that 
is my feeling. Fun fact, though: you can convert pi to hexadeicmal base 
without computing the preceding digits:

https://en.wikipedia.org/wiki/Bailey%E2%80%93Borwein%E2%80%93Plouffe_formula

	Christian



From moller at mollerware.com  Tue Jun 30 18:29:54 2015
From: moller at mollerware.com (Chris Moller)
Date: Tue, 30 Jun 2015 18:29:54 -0400
Subject: Capturing PyRun_String stdout
Message-ID: <55931862.50608@mollerware.com>

I sent this to the wrong list a few hours ago.  I hope I've gotten it 
right this time...

=====================================================

I expect this has been asked before, but I can't find out much about it...

I'm trying to embed Python in a GTK+/C app  as a scripting language and 
I need to capture the output of PyRun_String(), PyEval_EvalCode(), or 
whatever as a char * (or wchar_t * or whatever) rather than have it go 
to stdout.

I'm using Python 3.3.2 under plain C, not C++

And, while I'm interrupting everyone's evening, another question: if I  
pass Py_single_input to PyRun_String() or 
Py_CompileString()/PyEval_EvalCode(), it accepts statements like "a=10" 
and can then properly do stuff like "print(a)".  If I use Py_eval_input 
instead, I get error messages.  In both cases, I'm using the same 
global_dict and local_dict, if that makes any difference.  What am I 
doing wrong?

Thanks,
Chris Moller
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150630/4103de0d/attachment.html>

From breamoreboy at yahoo.co.uk  Tue Jun 30 19:01:01 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 01 Jul 2015 00:01:01 +0100
Subject: How to debug TypeError: required field "lineno" missing from expr?
In-Reply-To: <5590b0fd$0$1642$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.153.1435540503.3674.python-list@python.org>
 <5590b0fd$0$1642$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mmv73h$qh7$1@ger.gmane.org>

On 29/06/2015 03:44, Steven D'Aprano wrote:
> On Mon, 29 Jun 2015 11:14 am, Mark Lawrence wrote:
>
>> Purely as an exercise I've been converting Grant Jenks' pypatt[1] from
>> 2.7 to 3.4.  I've managed to sort out most of the required changes by
>> checking on what I can see with an AST pretty printer[2].  So it's
>> rather frustrating to have the compile stage throw the error given in
>> the subject line.
>
> Now now Mark, you know how this works. Where's your COPY and PASTED
> traceback? Where's the Short, Self Contained, Correct Example?
>
> http://sscce.org/
>
> Our crystal ball is in no better state than yours :-)

The traceback was effectively that in the subject line.  I'd have 
provided a SSCCE if I'd had the faintest idea of how to produce one, 
which I didn't have a couple of days ago.

>
>
>> The code has a call to the ast fix_missing_locations function.  I'm
>> aware of the Armin Ronacher tweet[3] stating
>>
>> <quote>
>> "TypeError: required field "lineno" missing from stmt" ? no, what you
>> actually mean is "tuple is not a statement'.
>> </quote>
>>
>> but I don't think this is the problem.
>
> That's what I dislike about Twitter. There's no context, and no way to tell
> what Armin is talking about. I've never seen this error, and cannot imagine
> how to get it, or what he means by "tuple is not a statement" for that
> matter. A tuple can be a statement:
>
> if condition:
>      (1, 2)  # Construct a tuple, then throw it away.
>
>
>> Still I've clearly managed to
>> screw something up somewhere along so line, so any bright ideas as to
>> how I can proceed?
>>
>> Further would it be worth raising an enhancement request to get some
>> better diagnostics from the built-in compile function,
>
> If you think it is worth waiting months or years before you can actually use
> those enhanced diagnostics, sure. What sort of diagnostics?
>
> Peering into my crystal ball, I see that you're maybe generating some code
> dynamically, then running something like:
>
>      code = compile(the_text, something, something)
>
> and that's giving you an error. What happens if you do this?
>
>      print(the_text)
>      code = compile(the_text, something, something)
>
>
> Can you extract the offending line that way? What happens if you copy and
> paste the_text into a Python file, and then try to import or run that file?
>

I managed to find the problem with the aid of 
https://github.com/simonpercivall/astunparse which allowed me to compare 
the 2.7 output  with the 3.4 and pinpoint precisely where I'd gone wrong.

Thanks anyway for your reply and all the others, it certainly got my 
brain engaged in a logical manner.  I might not be an expert on Python 
ASTs but I've certainly enjoyed playing with them.  All I've got to do 
now is correct the next problem in the pipeline.  Maybe I'll bore you 
all with that tomorrow :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From cs at zip.com.au  Tue Jun 30 19:21:52 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 1 Jul 2015 09:21:52 +1000
Subject: Python 3 resuma a file download
In-Reply-To: <9a629cf3-e256-494a-8ff8-3f1f6fc2218c@googlegroups.com>
References: <9a629cf3-e256-494a-8ff8-3f1f6fc2218c@googlegroups.com>
Message-ID: <20150630232152.GA4688@cskk.homeip.net>

On 30Jun2015 08:34, zljubisic at gmail.com <zljubisic at gmail.com> wrote:
>I would like to download a file (http://video.hrt.hr/2906/otv296.mp4)
>If the connection is OK, I can download the file with:
>
>import urllib.request
>urllib.request.urlretrieve(remote_file, local_file)
>
>Sometimes when I am connected on week wireless (not mine) network I get WinError 10054 exception (windows 7).
>
>When it happens, I would like to resume download instead of doing everything from very beginning.
>
>How to do that?
>
>I read about Range header and chunks, but this server doesn't have any headers.
>
>What options do I have with this particular file?

You need to use a Range: header. I don't know what you mean when you say "this 
server doesn't have any headers". All HTTP requests and responses use headers.  
Possibly you mean you code isn't setting any headers.

What you need to do is separate your call to urlretrieve into a call to 
construct a Request object, add a Range header, then fetch the URL using the 
Request object, appending the results (if successful) to the end of your local 
file.

If you go to:

  https://docs.python.org/3/library/urllib.request.html#urllib.request.urlretrieve

and scroll up you will find example code doing that kind of thing in the 
examples above.

Cheers,
Cameron Simpson <cs at zip.com.au>

The British Interplanetary Society? How many planets are members then?
        - G. Robb


From jon+usenet at unequivocal.co.uk  Tue Jun 30 19:25:01 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 30 Jun 2015 23:25:01 +0000 (UTC)
Subject: Pure Python Data Mangling or Encrypting
References: <mmcagn$aa5$1@ger.gmane.org>
 <558b7e85$0$1648$c3e8da3$5496439d@news.astraweb.com>
 <mailman.42.1435206516.3674.python-list@python.org>
 <558bc912$0$2899$c3e8da3$76491128@news.astraweb.com>
 <slrnmonkip.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <558c1a7e$0$1668$c3e8da3$5496439d@news.astraweb.com>
 <slrnmoo7ev.1nu.jon+usenet@frosty.unequivocal.co.uk>
 <CAPTjJmqOzLTv9cFFKjV-BLtaFZhEZ4B870ALqNMmanCEN-VqXA@mail.gmail.com>
 <mailman.89.1435280528.3674.python-list@python.org>
 <558d86b0$0$1659$c3e8da3$5496439d@news.astraweb.com>
 <mmkbij$dpm$1@ger.gmane.org>
 <CAPTjJmon4abegr01-+mo5aLML+gA3wnR_+_kTTU7gGuCn=xrDA@mail.gmail.com>
 <mailman.141.1435424948.3674.python-list@python.org>
 <558ef059$0$1673$c3e8da3$5496439d@news.astraweb.com>
 <mailman.171.1435611181.3674.python-list@python.org>
 <5592065e$0$1675$c3e8da3$5496439d@news.astraweb.com>
 <mailman.198.1435685990.3674.python-list@python.org>
 <5592e71e$0$1674$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <slrnmp69d1.1nu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-06-30, Steven D'Aprano <steve at pearwood.info> wrote:
> I don't think there has been much research into keeping at least *some*
> security even when keys have been compromised, apart from as it relates to
> two-factor authentication.

That's because "the key" is all the secret part. If an attacker knows
the algorithm, and the key, and the ciphertext, then *by definition*
all is lost. If you mean keeping the algorithm secret too then that's
just considered bad crypto.

> In the past, and still today among people who don't understand Kerckhoffs'
> principle, people have tried to keep the cipher secret and not have a key
> at all. E.g. atbash, or caesar cipher, which once upon a time were cutting
> edge ciphers, as laughably insecure as they are today. If the method was
> compromised, all was lost. 

Caesar cipher has a key. It's just very small, so is easy to guess.

> Today, if the key is compromised, all is lost. Is it possible that there are
> ciphers that are resistant to discovery of the key? Obviously if you know
> the key you can read encrypted messages, that's what the key is for, but
> there are scenarios where you would want security to degrade gracefully
> instead of in a brittle all-or-nothing manner:
>
> - even if the attacker can read my messages, he cannot tamper with 
>   them or write new ones as me.

I suppose that could be achieved by having separate encryption and
signing keys, but you could do the same but better by encrypting
with multiple algorithms. It's not an unstudied area:
https://en.wikipedia.org/wiki/Multiple_encryption


From thesneakercop at gmail.com  Tue Jun 30 21:11:42 2015
From: thesneakercop at gmail.com (thesneakercop at gmail.com)
Date: Wed, 1 Jul 2015 01:11:42 +0000
Subject: =?utf-8?Q?Can't_get_it_to_download?=
Message-ID: <55936775.81d90d0a.7535.3479@mx.google.com>

I?ve made several attempts to download python to my laptop, but every time I open it, it ask to repair or uninstall. Can you assist me with downloading it?






Sent from Windows Mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150701/ab140f76/attachment.html>