carry **arguments through different scopes/functions

[Steven D'Aprano]

On Sun, 31 Jan 2016 11:19 pm, c.buhtz at posteo.jp wrote:

> I am not sure what the problem is here, so I don't really know how I
> should call the subject for that question. Please offer a better
> subject.
> 
> The code below is a extrem simplified example of the original one. But
> it reproduce the problem very nice. Please focus on the variable
> `return_code`.

The problem with return_code is that you are trying to use it as an "output
parameter" or "call by reference" parameter. You want behaviour like this:


a = 1
b = 2

def test_output_parameter(the_var):
    print(the_var)
    # Set the_var by reference.
    the_var = 999

test_output_parameter(a)  # sets a
test_output_parameter(b)  # sets b
assert (a == 999) and (b == 999)



This cannot work in Python: Python is never call by reference. If somebody
has told you that it is call by reference, they are wrong.

You can *simulate* call by reference output parameters by using a list, but
that's the wrong way to solve this problem. The right way is for your
function to return a value, which you then assign to the variable you want
to change:


a = 1
b = 2

def test_return(the_var):
    print(the_var)
    return 999

a = test_return(a)
b = test_return(b)
assert (a == 999) and (b == 999)




Looking at your example code, we can replace (most of it) with these six
lines:


# Version 1: best, shortest, fastest way, with no callbacks.

ids = [1,2,3,4,5,6,8,9]  # NO 7
print(ids)
if 7 in ids:
    print('7 is in')
else:
    print('no 7 in it')



ignoring what looks like prints trying to debug the "return_code" problem.

If you really need to use a callback structure, then try something like
this, using a generator to return each individual callback result:



# Version 2: a mess -- don't do this unless you must!

def walker(ids, handlerFunction, **handlerArgs):
    for one_id in ids:
        print('handler-call for id {}\t{}'.format(one_id, handlerArgs))
        yield handlerFunction(one_id=one_id, **handlerArgs)

def on_id_callback(one_id, return_code):
    if return_code is False:
        # Why are we skipping this case?
        return False
    return_code = (one_id == 7)
    print('one_id: {}\treturn_code: {}'.format(one_id, return_code))

def seven_in_it(ids):
    return_code = True
    for return_code in walker(ids, on_id_callback, return_code=return_code):
        # This loop runs for the side-effects!
        pass
    return return_code


ids = [1,2,3,4,5,6,8,9]  # NO 7
print(ids)

if seven_in_it(ids):
    print('7 is in')
else:
    print('no 7 in it')




Running that code gives this output:

[1, 2, 3, 4, 5, 6, 8, 9]
handler-call for id 1   {'return_code': True}
one_id: 1       return_code: False
handler-call for id 2   {'return_code': True}
one_id: 2       return_code: False
handler-call for id 3   {'return_code': True}
one_id: 3       return_code: False
handler-call for id 4   {'return_code': True}
one_id: 4       return_code: False
handler-call for id 5   {'return_code': True}
one_id: 5       return_code: False
handler-call for id 6   {'return_code': True}
one_id: 6       return_code: False
handler-call for id 8   {'return_code': True}
one_id: 8       return_code: False
handler-call for id 9   {'return_code': True}
one_id: 9       return_code: False
no 7 in it



But this is truly awful code. My sympathies if you are forced to use it, my
apologies for being so blunt if you wrote it. A better way to deal with
this would be something like this:


# Version 3: better.

def verbose_equals_seven(one_id):
    flag = (one_id == 7)
    print('one_id: {}\treturn_code: {}'.format(one_id, flag))
    return flag

ids = [1,2,3,4,5,6,8,9]  # NO 7
print(ids)

if any(verbose_equals_seven(id) for id in ids):
    print('7 is in')
else:
    print('no 7 in it')




Running this Version 3 code gives this output:


[1, 2, 3, 4, 5, 6, 8, 9]
one_id: 1       return_code: False
one_id: 2       return_code: False
one_id: 3       return_code: False
one_id: 4       return_code: False
one_id: 5       return_code: False
one_id: 6       return_code: False
one_id: 8       return_code: False
one_id: 9       return_code: False
no 7 in it




Not very exciting. But if we try again with a list containing id=7, we see
that any() stops processing as soon as it has a success:

ids = [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
print(ids)
if any(verbose_equals_seven(id) for id in ids):
    print('7 is in')
else:
    print('no 7 in it')



prints this output:


[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
one_id: 9       return_code: False
one_id: 8       return_code: False
one_id: 7       return_code: True
7 is in


> In the original code I will
> use some more complexe data structures with `**handlerArgs`.

Have you tried it? If this complex data structure is *mutable*, and you
change it, that change will be seen everywhere. Remember right at the start
of my post I mentioned that you can simulate "output parameters" with a
list? This is similar: because objects are not copied when you pass them to
a function, mutations to the object will be seen no matter where you make
the mutation.



# Version 4: mutating an object instead of returning flags.
# Don't do this unless you really must. Python is not Java
# and code like this is not considered good style

class ComplexData:
    def __init__(self):
        self.flag = False
    def __repr__(self):
        msg = "<%s instance with object id %s and flag %s>"
        return msg % (type(self).__name__, id(self), self.flag)

def walker(ids, handlerFunction, **handlerArgs):
    for one_id in ids:
        print('handler-call for id {}\t{}'.format(one_id, handlerArgs))
        handlerFunction(one_id=one_id, **handlerArgs)

def on_id_callback(one_id, something_complex):
    print('one_id: {}\tsomething_complex: {}'
          .format(one_id, something_complex))
    if something_complex.flag is False:
        something_complex.flag = (one_id == 7)

def seven_in_it(ids):
    data = ComplexData()
    walker(ids, on_id_callback, something_complex=data)
    return data.flag


ids = [9, 8, 7, 6, 5, 4, 3, 2, 1]
print(ids)

if seven_in_it(ids):
    print('7 is in')
else:
    print('no 7 in it')




Running version 4 of the code prints this output:


[9, 8, 7, 6, 5, 4, 3, 2, 1]
handler-call for id 9   {'something_complex': <ComplexData instance with
object id 3082144364 and flag False>}
one_id: 9       something_complex: <ComplexData instance with object id
3082144364 and flag False>
handler-call for id 8   {'something_complex': <ComplexData instance with
object id 3082144364 and flag False>}
one_id: 8       something_complex: <ComplexData instance with object id
3082144364 and flag False>
handler-call for id 7   {'something_complex': <ComplexData instance with
object id 3082144364 and flag False>}
one_id: 7       something_complex: <ComplexData instance with object id
3082144364 and flag False>
handler-call for id 6   {'something_complex': <ComplexData instance with
object id 3082144364 and flag True>}
one_id: 6       something_complex: <ComplexData instance with object id
3082144364 and flag True>
handler-call for id 5   {'something_complex': <ComplexData instance with
object id 3082144364 and flag True>}
one_id: 5       something_complex: <ComplexData instance with object id
3082144364 and flag True>
handler-call for id 4   {'something_complex': <ComplexData instance with
object id 3082144364 and flag True>}
one_id: 4       something_complex: <ComplexData instance with object id
3082144364 and flag True>
handler-call for id 3   {'something_complex': <ComplexData instance with
object id 3082144364 and flag True>}
one_id: 3       something_complex: <ComplexData instance with object id
3082144364 and flag True>
handler-call for id 2   {'something_complex': <ComplexData instance with
object id 3082144364 and flag True>}
one_id: 2       something_complex: <ComplexData instance with object id
3082144364 and flag True>
handler-call for id 1   {'something_complex': <ComplexData instance with
object id 3082144364 and flag True>}
one_id: 1       something_complex: <ComplexData instance with object id
3082144364 and flag True>
7 is in




-- 
Steven