[Tutor] All of Kermit's E-Mails [was: Global Variables]

[Luke Paireepinart]

Hello, Kermit.
Kermit Rose wrote:
>  
>  
> From: Bob Gailer 
> Date: 08/14/06 20:24:00 
> To: kermit at polaris.net; tutor at python.org 
> Subject: RE: [Tutor] Global variables 
>  
> A while back you attached factor34.py. Is that the program you are 
> having trouble with? 
>  
>  
> ****
>  
> Yes!
>  
> And you said it misbehaves "consistently with certain numbers to be 
> factored." 
> What are these numbers? 
>  
>  
> *****
>  
> One of the numbers for which strongfac fails to return factors which it
> correctly calculates is
>  
> Search for a factor of  100000000000037
>   
I really dislike the way you reply.
Follow the standard format! Leave all the greater-than signs before the 
previous person's reply
so that it will be structured correctly and easy to read.  All of this 
star business is bad.
There are two reasons why I think this.
First, consider this example.

Hello, how are you today?
***
I'm fine.
Did you have a good day?
***
Yes, I did.

Now while I'm reading this, I see the dialog going like this.
person: 'hello, how are you today?'
kermit: 'i'm fine.  did you have a good day?'
Oh wait! there are stars there.  That must mean that part of what I 
thought was you
talking was actually person talking.  So I have to actually try to judge 
which part was
the original post and which part is your reply.
On this example it's easy to see that 'did you have a good day' is 
person's speech and not yours
because there are only two lines of text between the stars and we know 
that the first line
immediately after any set of stars has to be  your reply.

Now consider this example:
 > Hello, how are you today?
I'm fine.
 > Did you have a good day?
Yes, I did.

Tell me that isn't orders of magnitude clearer.

Point 2:
I recently read an article on UI design.
Basically, what it said was 'the easier it is for your users to 
intuitively understand your application,
the more they'll enjoy it and consider it a good application.  This is 
largely governed by how they've
experienced similar applications working in the past, and they expect if 
they perform similar
actions in your application they'll garner similar results as in your 
predecessor.'

How this applies to you:
Right away, when I see those stars, I get really agitated, because I 
know I'm going to have to
work twice as hard trying to decipher who's talking when I read your 
e-mail, and right off the bat
you've alienated me and made me not want to address the issue at hand.
Clearly you can see this as I've written quite a long reply that is 
based on nothing
to do with your original question.

The place where I had a problem understanding who was speaking in this 
particular e-mail was:

>A while back you attached factor34.py. Is that the program you are 
>having trouble with? 
>
>
>****
>
>Yes!
>
>And you said it misbehaves "consistently with certain numbers to be 
>factored." 
>What are these numbers? 
>
>
>*****
>
>One of the numbers for which strongfac fails to return factors which it
>correctly calculates is

As I was reading this, I assumed the 'Yes!' and following paragraph were 
part of the same thought.
When I got to the second set of stars, my immediate reaction was "Why is 
kermit replying to something she
just said?" That was followed by "Oh, somewhere along the line it 
must've switched between the original poster's
and kermit's dialog."  I figured out that the 'And you said it 
misbehaves ...' part was where the OP continued to speak.


Now that I'm done with that topic, let's move on.

 From one of your other e-mails:

 
 
>From: Alan Gauld 
 
>The names have very little to do with it, the danger of global 
>variable 
>use is the reliance on side-effects and the tight coupling that you 
>introduce between the calling module and the called module. 
>Changes to the state of the calling module in an unpredictable 
>manner lead to subtle bugs which are extremely hard to see and fix. 
>*****
>Huh???
>What side effects do you have in mind?
>I certainly did not know any side effects existed.
>What do you mean by tight coupling?
>The only change I can see that would lead to a bug would be if I changed the
>name of the global variable in
>the calling routine and not in the function it called.
>I would know not to do that.


Okay.
>From this it appears that you have no academic training in programming.
The term 'side-effect' means that something outside the scope of the 
called function is modified by the called function.


Here is a very basic example.

#--- test.py
a = ['a','b','c']
def modify_list(alist):
    del(alist[0])

modify_list(a)
modify_list(a)
print a
#---

#--- output of test.py
['c']
#---


Now remember that in Python, when you pass a variable to a function,
only the reference is passed to the function.
Imagine that variables are themselves references (which they are.)

a = ['a','b','c']

creates the following picture:

a ------> ['a','b','c']

modify_list(a)

doesn't do

modify_list(['a','b','c'])

it does

modify_list(ptr) where ptr ------> ['a','b','c'] <----- a

they both refer to the same object ( a python list containing three characters.)

now when in the body of modify_list we say del(alist[0]), what takes place is
the index operator, [0], says 'what object is the variable 'alist' pointing to?
and python says 'alist ----> ['a','b','c'] <---- a'
so when we delete the element out of alist we're also deleting the element
out of the calling function's list.
This is probably NOT desired behavior.
As such it's an unwanted side-effect.

A function printing to the screen is also considered a side-effect.
Anything a function does that affects anything outside of its own scope
is considered a side-effect.


To refresh your memory, here's the next part of the e-mail I'm going to address:

>The names have very little to do with it, the danger of global 
>variable 
>use is the reliance on side-effects and the tight coupling that you 
>introduce between the calling module and the called module. 
>Changes to the state of the calling module in an unpredictable 
>manner lead to subtle bugs which are extremely hard to see and fix. 
>*****
>What do you mean by tight coupling?
>The only change I can see that would lead to a bug would be if I changed the
>name of the global variable in
>the calling routine and not in the function it called.
>I would know not to do that.


'tight coupling' is an English phrase meaning roughly
(for our purposes) 'close interaction' or 'dependence on each other.'

you say, I quote:
'The only change ... that would lead to a bug would be if I changed the
name of the global variable in the calling routine.'
I think you're misunderstanding Alan.
consider this following example
(it was hard to come up with something this convoluted)

#--- start code
def b():
    global has_been_greeted
    has_been_greeted = False
    print "function B is doing lots of stuff"
    return "whatever we did in function B that's so important"

def greet():
    global has_been_greeted
    if has_been_greeted:
        print 'we already greeted you.'
    else:
        print 'hello!'
        has_been_greeted = True

def c():
    print "function C is doing lots of stuff."
    a = b()
    print "function C is using value a here."

b()
greet()
greet()
c()
greet()

# --- end code.

Now we know we're going to call b() every time on startup,
so we set has_been_greeted to false inside of it.
Then we call greet, once b() is done calculating its stuff,
so it will greet the customer.
Then we call greet again to make sure that we already greeted them.
(say we're debugging our program and we want to make sure greet's 
working correctly.)

Now we call function C, which relies on function b to get a value
that it needs for one of its calculations.
Oops, we forgot that b() resets has_been_greeted back to false
every time it's called.
Now when we call greet after function c again, it greets them again!
But it's already greeted them so it shouldn't have greeted them.

Now imagine this is much more complicated and there are multiple global 
variables
that you're setting all over the place to store your values,
and then imagine trying to find something this obscure.

A much more straightforward way to do this would be:

#--- start code
def b():
    print "function B is doing lots of stuff"
    return "whatever we did in function B that's so important"

def greet():
    print 'hello!'

def c():
    print "function C is doing lots of stuff."
    a = b()
    print "function C is using value a here."

b()
greet()
has_been_greeted = True
if has_been_greeted == False: greet()
c()
if has_been_greeted == False: greet()
#---

I know this was a stupid example but it's all I could come up with.
does it help you see how global variables could give you trouble?
Maybe Alan can fix my example so it makes more sense.



>>>>>>>>>>> >>>>>>>>>>>
>>>>>>>>>>>                       

I see here you started using Greater-than signs to separate the Original Post from your replies.
WTF. RTFM. This is almost as bad as before.
Now I don't have the additional stress of trying to sort out who is speaking at any given time,
you or the person you're replying to, but I have these big columns of multi-colored lines to deal with.
In my e-mail client, Thunderbird, it converts every > sign to a vertical line of a different color,
so I can tell the different levels of replies in any given e-mail.
This is super-helpful, unless something like this comes along, and all it does is space
out all the messages a tremendous amount and look ugly.
Why can't you just use the convention and leave the >> things there?
They're there for a reason.
Don't fix somethin' that ain't broke, please.
 

>Confusion of names is of very little import, that really isn't the 
>issue. 
 
>**********
 
>Is it that global variables are no implemented correctly?
 
>I can't imagine what the issue would be if it isn't confusion of names.

No, it's not that global variables are not implemented correctly.
You must understand that there are thousands of people using Python every day,
and phrases like 'not implemented correctly' or 'there's a bug in python' are
converted in my ears to '*i'm complaining because i can't get my program to work*'
It's identical, to me, as when I'm playing Counter-Strike and someone starts
yelling 'HACKER!' every time they get killed.

It's ____much____ more likely that the person is just good at Counter-Strike than they're cheating,
especially since there are a lot of anti-cheat measures.

It's ____much____ more likely that your program has a bug than Python itself,
because there is a lot of quality-control when a programming language is used
frequently by many very skilled coders.
 

>>>>>>>>>> >>>>>>>>>>
>>>>>>>>>>                     

You already know my feelings about this.

 
 
>I'd be very very doubtful that its a bug in Python. 
>Python is very well tested and while occasionally bugs do surface, 
>the type of bug you are describing is extremely unl;ikely to have 
>remained hidden. It is far more likely to be an error in the code 
>or in the data. 
>*******
>I understand your skepiticism.   I would be too in if I were in your
>position.
>I've just sent the documentation to the list,  in my message to Luke.

Alan is being diplomatic here.

I would not call what you sent 'documentation.'
The tangled mess of your functions doesn't prove there's a bug in python,
it just makes me think you messed up the value somewhere along the way.


>>>>> >>>>>
>>>>>           
Yay, more of this stuff.

 
>All of which points to an error in the code not in Python. 
>The way Python is written it is virtually never going to result in 
>that kind of error dependant on data values. 
 
 
>******
 
>Which is why it surprised me.

What Alan means is that your program incorrectly handles certain data values
because your functions aren't written correctly.

Your first thought when your program doesn't do what you expect it to
should not be to blame the return statement of being buggy,
but to first blame yourself.
It's much more likely that you're at fault.
I'm not saying this directly at you.
If I wrote a function that returned incorrect values,
I would assume it's my fault too.

You have to understand that Python has been used for years
and if there was such a fundamental bug, no matter how obscure (only occuring on certain return values),
it would have been found by now.

 

>>>>>>>>>>> >>>>>>>>>>>
>>>>>>>>>>>                       
:(
 
 
>That might happen 
>is if the values are very close to zero and a type conversion 
>occurs, but otherwise I'm very dubious about a Python bug 
>of that type. 
 
 
>*****
 
>Indeed.  Errors of this type would be found by chance, like I found it.
 
>It would be impractical to search for this type of error.
 
>It is not a type conversion.
 
Because it's impractical to search for a certain type of error it must not be that error?
Just dwell on that statement a bit and tell me if you still believe it.



>The error is occuring between the return statement in the called function
>and the picking up of that value
>in the calling function. 

so show us.
This is the way I would prove this, if it were my program that uncovered this 'bug.'


#---
def func_that_causes_bug(a):

    print "function is doing a bunch of complex calculations and such."
    v = 2322**a
    c = 55/float(v)
    d = v - a + 543
    value = d / 45

    print "this is where the problem arises."
    print "Before I return the value, it is :%s" %  value
    return value

tmp = func_that_causes_bug(564)
print "And after I return the value, it is :%s" % tmp

#---
 
Do you understand why this would evidence the bug you mentioned?
You say it's a problem with the returning of the value.
So print the value immediately before the return and immediately after.
We'll see if it changes in transit as you claim.




Onto your next e-mail:


 >Can you post actual code to illustrate the problem?  Don't post your
 >entire module; just show us the functions involved, the input that
 >causes the problem, and what output you expect to get.
 >--
 >John.
 >
 >******* 
 >
 >Here is the output that illustratrates the problem.  After the output I'll
 >list the code for the called function,
 >strongfac,
 > 
 >and the calling function,  fermat.
 
[snip a ton of debug output.]

This doesn't help us at all.  What the heck was all that?
You must understand that I have no idea what your function is supposed 
to do.
Treat me like a child.  Explain everything to me.

What part of this output isn't what you wanted?


 >#    def strongfac(z,w):
 >#        x = gcd(z,w-1)
 >#        if x > 1:
 >#            if x < z:
 >[snip the rest of strongfac]
 >#    def fermat(z,maxdiff,pmap,plist,sqlist,sqresidue,torials,limitsteps):
 >#        for j in range(2,3):
 >#            print " "
 >#            print " Try to find factors by using power of ",j," mod z 
as a
 >[snip the rest of fermat]


Why the CRAP did you put # signs before every line?
I spent about 10 minutes pressing delete, down-arrow, delete, down-arrow.
Going through all that trouble just so I could run the code was a pain 
in the rear.
Remember when we were talking about UI usability earlier?
Your code usability would be greatly improved if I didn't have to remove
a comment from every line to test it.

Also, in your example call:
 
 >IDLE 1.1.2
 > >>> import factor34
 > >>> from factor34 import testrange
 > >>> testrange(10**14+37,10**14+37)

You called testrange.
Now John originally said 'show us which functions are involved.'
Well, after I had gone through the trouble of un-commenting all your code,
it turned out that I didn't even have all the pieces there.
So I went back in e-mail history and downloaded the attached 
'factor34.py' file.
Now I wish I didn't have to do this, because you could've made 
improvements in it since then.
But it's the only way I can get a functional test file.

Turns out testrange depends on x depends on  y depends on z depends on .....
basically all of the functions in the module depend on each other.

This is obviously a case for ... .dun-dun-dun!.... object-oriented 
programming.
When you have a collection of functions that all operate together to 
create something,
you make a Class to contain them.

I think just in the process of doing this you might learn what's wrong 
with your program.



Link me to all the Wikipedia articles I need in order to understand what 
you're trying to do.
I'll help you rewrite this program from scratch if you want.
We'll make it object-oriented the whole way.
But there is no hope I will understand factor34.
I will not even try.


Sorry if I was too harsh anywhere along the way.
Feel free to point any of this out and I'll apologize.
But this is my honest opinion of the e-mails I've received from you so far.

I don't think this is a good programming project for you to be learning
a new language with.  Perhaps you should do something simpler.
If you'd like some recommendations I'd be happy to give some.

Have a good day.
-Luke