[Tutor] Implicit passing of argument select functions being called

alexander-rodis alexanderrhodis at gmail.com
Sun Jul 10 02:51:19 EDT 2022


I'm working on a project, where accessibility is very important as it's 
addressed to non - specialists, who may even have no knowledge of coding 
of Python.

In a specific section, I've come up with this API to make data 
transformation pipeline:

make_pipeline(

     load_data(fpath,...),

     transform1(arg1,arg2,....),

     ....,

     transform2(arg1,arg2,....),

     transform3(arg1,arg2,....),

     transformN(arg1,arg2,....),

)

transformN are all callables (writing this in a functional style), 
decorated with a partial call in the background. make_pipeline just uses 
a forloop to call the functions

successively like so e = arg(e). Which transforms from the many 
available in the library I'm writting, are used will vary between uses. 
load_data returns a pandas

DataFrame. Some transforms may need to operate on sub-arrays. I've 
written functions (also decorated with a partial call) to select the sub 
array

and return it to its original shape.  The problem is, I currently have 
to pass the data filtering function explicitly as an argument to each 
function that needs it, but

this seems VERY error prone, even if I cautiously document it. I want 
the the filter function to be specified in one place and made 
automatically  available to all

transforms that need it. Something roughly equivalent to:

make_pipeline(

     load_data(fpath,...),

     transform1(arg1,arg2,....),

     ....,

     transform2(arg1,arg2,....),

     transform3(arg1,arg2,....),

     transformN(arg1,arg2,....),

     , data_filter = simple_filter(start=0,) )

I thought all aliases local to caller make_pipelines becomes 
automatically available to the called functions. This seems to work but 
only on some small toy examples, not in

this case. global is usually considered bad practice, so I'm trying to 
avoid it and I'm not using any classes so not OOP. I've also tried using 
inspect.signature to check if each

callable accepts a certain argument and pass it if that's the case, 
however this raises an "incorrect signature error" which I could find 
documented anywhere. I've also considered

passing it to all functions with a try/except and ignore thrown errors, 
but it seems this could also be error prone, namely it could catch other 
errors too. So my question is, is there an

elegant and Pythonic way to specify data_filter in only one place and 
implicitly pass it to all functions that need it without global or classes?


Thanks



More information about the Tutor mailing list