[Python-Dev] Deprecating whrandom; and deprecating in general

[M.-A. Lemburg]

Guido van Rossum wrote:
> 
> [MAL]
> > Some existing scripts still use e.g. regex which is deprecated
> > and probably will get removed from the core distribution soon.
> > We can't update that code to use re because the script will have
> > to expose the exact same semantics as regex provides and I don't
> > know how to do this reliably with re (pcre or SRE). Note that
> > these scripts are usually tools which again are used by other
> > applications, so changing semantics is not an option either.
> 
> I'm not sure I understand.  Are these other packages generating
> regular expressions in the regex syntax that your script then applies
> somewhere?  This is indeed a problem -- unless you can keep your
> customers on Python <= 2.2 forever, eventually the regex module will
> die of old age.

No, the problem is that the tools get used in e.g. shell scripts
all over the place, so other software relies on the Python script
using the regex RE syntax.
 
> Perhaps what's necessary is a translation module that translates the
> regex syntax into re syntax?  Such a module could be provided as part
> of future releases as regex, without deprecation (at least until you
> stop using it :-).
> 
> Granted, creating such a translation module is a one-time effort, but
> it seems inevitable that the current regex module will eventually stop
> working.
> 
> If you think keeping regex alive is easier, perhaps you can take it up
> as a 3rd party module?  I really don't want to have to have three
> different regular expression implementations in the Python
> distribution forever.  (I do understand that deprecating and removing
> a very heavily used package like regex needs to be done more carefully
> than e.g. deprecating dircmp, which nobody uses.)

I can understand that, but I also think that working software
need not be removed altogether. Rather than moving it to
some lib-old directory it would be better to add it to a
separate distutils package which people who wish to
continue to use these modules can install along with the
standard Python installation.

To make people aware of this package, python.org should
link to it. The same could be done for e.g. the Asian
codecs.

I think it's worthwhile shipping Python as a core package
and a standard set of distutils based add-ons.

> > It is acceptable to tell sys-admins of our customers
> > to install the one extra package when they update Python,
> > it is not acceptable to require them to change existing code
> > and it's a maintenance nightmare to provide them with
> > updated software since it is not actively being maintained
> > anymore or was specifically written for the customer.
> 
> Is it acceptable to tell customers not to upgrade to newer Python
> versions, or to coordinate such upgrades with additional installs from
> you?

It's acceptable to tell them that they should install a
special add-on package along with Python ... they'll
have to do that anyway with the mx extensions, so
adding another package to the list wouldn't hurt.
 
> [Guido]
> > > Suppose we decide to deprecate whrandom.  The first opportunity to do
> > > so will be in 2.3.  (I think we shouldn't deprecate modules in micro
> > > releases; the PEP is silent about that but I think it would be a bad
> > > idea.)  Then it will have to continue to exist for at least a year.
> > > Because whrandom is still in active use (see above), I think the one
> > > year deprecation period is too short in this case and we should use
> > > two years.  Maybe the warning should only be introduced in the second
> > > year?  That probably means (assuming releases every 6 months):
> > >
> > > 2.3 - deprecate in docs
> > >
> > > 2.5 - issue warning
> > >
> > > 2.7 - move to Lib/lib-old
> 
> [MAL]
> > Perhaps we could make Lib/lib-old a separate package ?!
> 
> I'm not sure what you're proposing.  A separate distribution?  Or a
> Python package, which would have to be renamed to e.g. lib_old and
> from which you could then import it using "from lib_old import
> whrandom" ?

Sorry for the confusion. I meant a distutils package, not a
Python package.

The idea is to move code from lib-old into that a
separately downloadable distutils package, e.g. 
python-libold.

People would then first install Python and then add
python-libold and possibly python-asiancodecs as
needed. distutils makes this easy.

> > > Use case #1 is what we usually think of.  Some programmer is
> > > maintaining some Python code.  When he upgrades the Python version
> > > he is using, being the programmer he probably wants to find out
> > > about deprecated features in the new Python version, so that he
> > > can fix his code so that it won't break outright in the future.
> > > IOW, this programmer is glad that there are deprecation warnings
> > > (compared to the alternative, where he had no idea that he was
> > > using a deprecated feature until it was removed and his program
> > > broke outright).
> >
> > True; however, deprecation warnings are run-time messages
> > and not every module in a package is always used by the
> > programmer, so the warnings may appear after a few months
> > down the road.
> 
> Well, that's life.

It's specific to the dynamic nature of Python. E.g. in C
you'd get linker errors at link time or compiler errors
for changed interfaces.
 
> > I would much rather like a tool like PyChecker scan *all*
> > the code for deprecated modules and maybe a similar tool
> > for the C API.
> 
> PyChecker can't check everything.  E.g. a source code inspection can't
> possibly detect int divisions reliably.

No, but it certainly can check for 'import whrandom'.
 
> > > This is annoying: maybe the warning goes to a log file where
> > > unexpected things cause alarms to go off, maybe it shows up in an
> > > end user's shell window where it causes confusion, maybe there are
> > > a lot of warnings and the extra output is simply annoying.  These
> > > users probably aren't Python savvy, so they may panic and call
> > > customer service.  Understandably, nobody is happy.  Marc-Andre
> > > and Fredrik already know their program issues a warning, and
> > > they've fixed it in their own version, and they don't want to
> > > waste time telling every user how to disable or ignore the
> > > warning, or shipping upgrades.
> >
> > It not so much about wasting time, it's about bad PR: to the
> > customer it appears that you've written malfunctioning code.
> 
> Somewhat granted, though it suggests that the customer doesn't
> understand what they're talking about.  Customers always blame the
> wrong party (e.g. whenever some Windows game breaks the Python
> installer, Python gets the blame).

True :-/ (this is actually more about politics than 
understanding of the issues)
 
> > > I would like to suggest that instead, professional software
> > > distributors like eGenix and PythonWare tweak their distributions to
> > > turn of warnings by default.  This is easy enough: in the main
> > > program, insert
> > >
> > >     import warnings
> > >     warnings.filterwarnings("ignore")
> > >
> > > at the top (before other imports).  If the code needs to run on Python
> > > 2.0 or before, insert the whole thing inside a try: / except
> > > ImportError: pass.  If you want an easy way to enable warnings, you
> > > can invent something else (e.g. check an environment variable).
> > >
> > > Would this be acceptable?
> >
> > Yes.
> 
> Great!
> 
> > I would still like the option to continue using code which was
> > deprecated by PythonLabs. Just because code is unmaintained
> > doesn't mean that it cannot be used anymore.
> 
> Granted.
> 
> > So how about this:
> >
> > * deprecated modules get moved into a separate distutils
> >   based package and are made available on python.org together
> >   with the new Python version; the package should contain
> >   a copy of PEP 4 for the curious reader
> 
> If you want to maintain this, OK.

I'd need help from other developers.
 
> > * deprecated C APIs are grouped in a file pyold.h which
> >   only gets included by Python.h if Py_DEPRECATED is
> >   defined; that header files should contain a list
> >   similar to PEP 4 for the deprecated C APIs and if
> >   possible include a description of how to update existing
> >   code
> 
> I think this is fine too (though I admit I haven't been following the
> Py_DEPRECATED discussion in detail).

Good.
 
> > Both techniques aim for stability: they make the deprecation
> > process more visible and they provide downgrade paths for
> > existing installed code.
> 
> As long as you and others who will benefit from this help out, I think
> we can do this.

Great.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/