WedWonder: Scripts and Modules

[Cameron Simpson]

On 13Sep2019 08:40, DL Neil <PythonList at DancesWithMice.info> wrote:
>On 12/09/19 10:59 AM, Cameron Simpson wrote:
>>On 12Sep2019 08:24, DL Neil <PythonList at DancesWithMice.info> wrote:
>>>In this day-and-age do you have a script in live/production-use, 
>>>which is also a module? What is the justification/use case?
>>
>>Many. Many many.
>>
>>1: Many of my modules run their unit tests if invoked as the main 
>>programme.
>
>I used to do this, but now prefer to keep tests in separate modules - 
>and separate directories.

My tests are also in a separate module, but I can invoke them from the 
primary module. Just convenience.

>>2: Several modules are their own utility programme. I've got a heap of 
>>these - anything that provides useful access to something can often be 
>>usefully used from the command line.
>
>This is an very interesting idea - you're way ahead of me on this!
>
>Would it be fair to describe these as more Python for systems 
>programming/system administration, than an application for (simple) 
>users?

Perhaps. I'm not sure this is a meaningful distinction, except in so far 
as simple users generally expecting a GUI, or programmers expecting an 
API instead of a command line.

Having a command line, particularly one doing the common pattern of 
"command op args..." with various "op"s for common simple tasks makes it 
easier to use the module in a script, such as a non-Python script such 
as the shell.

That said, half these "op"s tend to be admin tasks or simple 
integration-like tests. So "make a new app user record in the database" 
=> "command new-user ...". But also "test this application function" ==> 
"command do-thing ..." where "do-thing" is something I'm debugging or 
testing which is normally painfully buried in the module or at the end 
of some tedious user interaction. And of course "command selftest" to 
run the unit tests :-)

>Gives me pause for thought: perhaps I lean too heavily on putting 
>'stuff' in the test routines (and view the application from that 
>'direction' too often).

My weakness lies in the other direction - not enough test driven 
development. IMO, my test writing ability is weak. So I've got this 
chunk of code and now (==later) I want to test some part of it.

Things I am finding useful recently include:

- doctests (which have the advantage that I can also use them as example 
  code in the docstrings) which are nice for laying out common use and 
  also specific corner cases

- the icontract Python module, which lets me put @require and @ensure 
  decorators on functions to express preconditions and postconditions;
  really great for catching misuse, especially if you also put some 
  insinstance type checks in there for getting parameters misordered

Both of these lack the cumbersomeness of a unit test suite. OTOH, they 
do not lend themselves to complex tests.

>Multiple entry-point systems seem relatively unusual these days - 
>perhaps a natural flow-on effect of the rise of gui- and menu-based 
>systems?

Unsure. Quite possibly.

>With one client, over the years, we've developed a number of 
>(basically) statistical analyses. Each analysis was born from a 
>separate project. Each lives in its own directory (tree). There are 
>some common modules held in a separate 'utility' 
>library/package/directory. Whilst it has been suggested, the idea of 
>an "overarching" menu-type top-level controller/distributor has never 
>risen up the ranks of the "backlog" (sensible criticism: the money 
>would be better spent on other priorities) - they (or at least the 
>older ones) are quite happy to 'drop' to the command line and type the 
>required command+args. I think that's the closest I come to what you 
>have described. (out of interest) Would you characterise it as a common 
>project structure?

It certainly seems reasonable. And as you suggest, there's you've got a 
bunch of main programmes associated with distinct modules/packages.

Cheers,
Cameron Simpson <cs at cskk.id.au>