Setuptools, __init__ and __main__

[Rob Gaddi]

On Fri, 06 Feb 2015 16:44:26 -0500, Dave Angel wrote:

> On 02/06/2015 04:35 PM, Ben Finney wrote:
>> Rob Gaddi <rgaddi at technologyhighland.invalid> writes:
>>
>>> So I'm trying to wrap my head around packaging issues
>>
>> Congratulations on tackling this. You will likely find the Python
>> Packaging User Guide <URL:https://packaging.python.org/> helpful.
>>
>> Also know that Python packaging was in a terrible state for a long
>> stretch of years, but now the Python Packaging Authority exists
>> <URL:https://www.pypa.io/> which has tended to steadily improve the
>> ecosystem in recent years.
>>
>>> I'm developing applications of various size for internal deployment.
>>> When I'm actually getting the whole thing written, I've got my
>>> initialization stub in __main__.py, because it's an application rather
>>> than a library. But then when I migrated over to setuptools I had to
>>> switch it to __init__.py and change around all my import statements to
>>> compensate. Then I can reference it as an entry point.
>>
>> Yes, it's one of the enduring bad parts of Python that it makes the
>> command-line invocation of an application overly awkward.
>>
>> What do you mean by “initialization stub”? That could mean a lot of
>> different things, and I don't want to guess what you mean.
>>
>> As for the difference between library versus application: You should be
>> writing all your code as discretely-testable units anyway, and unit
>> tests should be importing those in order to test them. So the whole lot
>> needs to be in a package hierarchy of some form in order to do that.
>>
>> That said, I haven't made use of the “execute a directory as a program”
>> Python feature, so I might be misunderstanding what the problem is.
>>
>>> This feels silly. I'm sure I could go stumbling around until I come up
>>> with an elegant solution, but does anyone already have one?
>>
>> It seems to me the problem is avoidable by ensuring from the start that
>> all your code is importable by your unit tests; that way, it's ready
>> for a Setuptools entry point declaration without changes.
>>
>>
> To this I'd like to add the caveat that both __init__.py and __main__.py
> are reserved names, and putting arbitrary code in either one is quite
> risky.
> 

Not so much risky as just, there's a thing that it does. 

> I suspect you're putting library code in your application code file, and
> naming it __main__ because that way you can cheat around the recursive
> import restrictions.  Please don't do that.
> 
> Organize your code so that there are no loops in the import library
> hierarchy.  If that means your application file is only six lines, so be
> it.
> 
> And don't name any source code __main__.py,
> 
> Finally, the mere presence of __init__.py has a special meaning, and
> code in it should probably be limited to doing namespace munging that a
> package may require to keep its public interface simpler.

So, I'll take the app I'm working on right now as an example.  I've got 
348 lines in my __init__.py, and another 200 in another library.  Little 
baby program.

My library code isn't in __init__.py (or previously in __main__) because 
I'm trying to dodge recursive imports; it's there because the code is 
short enough and tightly-coupled enough that I didn't see upside in 
splitting any of it out to a separate file.  Under other circumstances 
when the app has become substantial, the bulk of the application code 
winds up in, e.g. mainwindow.py, and the __main__ does nothing but call 
mainwindow.main().

The advice to reduce the entry point code (be it __init__ or __main__) to 
a small stub and move the bulk of the code to another library doesn't 
change the underlying situation.  Setuptools seems to require that the 
application be a "package", i.e. a thing that has an __init__.py.  But 
the python application start mechanism seems to be steering me to using a 
__main__.

I found a recommendation at https://chriswarrick.com/blog/2014/09/15/
python-apps-the-right-way-entry_points-and-scripts/ that seems to say 
that I should get around this by simply having an empty __init__.  I 
guess I can move the program to foobar.py, have an empty __init__, a stub 
__main__ that just calls foobar.main(), and an entry-point of 
foobar.foobar:main .  It just feels like a bit of a heavy solution to 
have two unnecessary files.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.