[Tutor] Module Architecture

[David]

On Tue, 16 Aug 2022 at 08:22, Alan Gauld via Tutor <tutor at python.org> wrote:
> On 15/08/2022 17:07, Gordon Stewart wrote:

> > I'd like some recommendations on how to design and layout a module.
> > There are any number of sources to help with discrete coding problems
> > (on this site for instance), but I'm interested in a higher level view
> > on how to assemble these discrete components into something easily
> > maintained and understood.

Hi Gordon,

Alan has given an excellent answer as usual.

The regular posters here usually give valuable and comprehensive advice.

I am no Python or programming guru. I am also learning on my own. That's
why I read this list.

However occasionally I have something worth sharing, and here I have three
points. I am speaking up because I can remember and relate to your uncertainty
about working with Python modules, and I want to share how I grew my skills
out of that situation.

First point:

I want to share that there have been several steps where I have learned to
use a new tool and in each case the tool gave me a profound increase in my
ability to manage code.

These tools (not in any particular order) were:
1) a version control system (VCS)
2) 'black' (I modified my copy to support tab indentation, which I prefer)
3) 'pylint'
4) 'pytest'
5) implement a Python app as a collection of small modules rather than one
   large file.

I have configured #2 #3 #4 to be invoked with an easy keystroke in my
editor, and so I use them all frequently:

  #2 ('black'): Minimises unnecessary noise in diffs.

  #3 ('pylint'): Catches syntax errors before they are committed, without
  needing the code to execute. This is important in Python, where any
  syntax errors that are not in a code path that is actually executed
  during any test will not be noticed by the test.

  #4 ('pytest'): Allows test-driven-development and more importantly
  removes any trepidation when doing significant refactoring.  I tried
  doctest and unittest, and I strongly prefer pytest.

#1 (VCS): A decent version control system VCS is valuable to eliminate
any fear of refactoring.

I use 'git' (I suggest 'gitk --all' when learning about it).
'git' isn't easy to learn because it is a complex toolkit and you have to
figure out your own personal workflow. After that, it is great.

It's fine to use whatever version control system you are comfortable with
that works, but these days I would not work on any software project without
using one.

Of course each tool has a learning curve.
The easy ones are 'pytest', 'pylint', and 'black'.
I strongly suggest you look at 'pytest'.

When I read that you are going to do some refactoring, I hope that at
minimum you will use tools such as #1 and #4.  With the full set of tools
#1 #2 #3 #4 it actually becomes enjoyable.

To finish, after reading Alan's answer, I want to add the following:

Recap: what are some benefits of a module:
- easier to read and modify code when it is all nearby
- it can be *tested* independently
- it can be *imported* as an entity, to elegantly provide facilites used
  by the primary project
- it can be *imported* as an entity, to elegantly provide facilites to
  other projects

Second point: *Imports*.

When using Alan's concept of cohesion and coupling, I suggest to think in
detail about what import statements will be necessary in each refactored
module in order for them to work together in an elegant way. This becomes
more important when the project involves several modules that are all
interacting together.

> It is easier to test a module if the data it uses is in the same module.
> But testing code is a whole topic by itself with support modules for
> doing so. Try reading up on unit testing and the python unittest module
> docs (although many prefer other test tools such as pytest)

Third point: *Testing*.

You mentioned wanting to test your my_funcs.py after refactoring.
This feels backwards to me! :)

I urge you to set up tests *before* refactoring.

In the same way I would recommend setting up the safety net before
beginning learning to walk on a tight-rope. It takes away the fear, in fact
the chance, of failure.

I strongly recommend 'pytest' for this. I would write tests for everything
before refactoring anything. Using 'pytest' it will not be hard to adapt
the tests no matter what module the functions might be later refactored
into.

I hope these thoughts are useful.
Good luck with your project.