[Python-Dev] Tracker cleanup roadmap

[Daniel (ajax) Diniz]

This is the janitorial plan I mentioned earlier...
It's so humongously huge by now that I'm not sure whether I should
submit it to e.g. the Python Papers or just print it and set it on
fire and run screaming. Fortunately, a tl;dl summary is provided :)

Daniel

Summary
Let's improve the tracker UI to better fit our needs. Then, classify
them bugs and separate garbage from real development. Lastly, bug
reporters should get a better UI. That's it,  any help is welcome.

The wall of text

Developer time constraints are arguably the main bottleneck in
handling tickets, which is only a subset of Python development.
Optimizing the application of main/core developers' time to Python
issues is a no-brainer. Being able to add volunteers to the productive
time pool is also very desirable. This document outlines a tentative
plan to move towards a better workflow in the Python Tracker.

Summary of tracker issues
The Python Tracker contains more than 17000 tickets, with
approximately 2350 still open. Of the open tickets, 500 were last
updated more than a year ago, while about 1150 were created before
that.

Current problems
Core developers, volunteers and newcomers are burdened with a lot of
inefficient and/or nonproductive chores when handling issues.
Requesting and waiting for feedback/confirmation/patch testing,
performing multiple searches, missing relevant discussion in
similar/duplicated issues and other low-level tasks and gotchas get in
the way of solving real problems.

The low signal/noise ratio of open issues is a major component of this
burden. At least 850 issues have outdated/missing version information,
while about 850 don't have a type (RFE, behavior, crash) set. About
800 issues have patches. Priorities carry little information, mostly
because they are left in the default setting: 1200 issues are set to
'normal' (a SF.net artifact) and 750 have no priority (a Roundup
artifact). Less than 200 tickets have 'low' priority. Most of these
issues boil down to tracker features being underused.

Another relevant time-sink consists of inadequacies of the current
interface. Many search features are hard to use or notice, among them
date spans and entering multiple inputs as lists. Other search
features are lacking, mostly simple boolean relations, e.g. those
including more than one keyword, full search terms, type, component,
etc. Besides searching, the lack of interfaces (and backend support)
for selecting and working with multiple issues tends to waste
considerable amounts of time.

Proposed plan
The suggested approach consists of a three parts effort focusing first
on developer-side tracker UI, then on massive issue categorization and
lastly on user-side UI. This optimizes the work of a single tracker
janitorveloper, but can be better partitioned if more janitorvelopers
are available.

The developer-side UI effort will focus, in this order, on making
available tracker features easier to use, adding new search features
and mass-handling of issues. This should improve both developers' and
tracker janitors' workflow.

Issue categorization will look like this last Bug Season (adding
details to tickets), but hopefully with a saner workflow. There are a
few clear sets of similar issues, e.g. about socket, handling of
characters on network protocol and format parsing modules, Tkinter +
IDLE, etc., that could use grouping and closing of duplicates. A
suggested form of grouping would be 'umbrella' or 'grab-bag' issues,
with existing issues of a topic as dependencies.

Finally, bug-reporter UI. The idea is to make it easier for users to
provide good bug reports, make the options clearer and minimize the
need for (preliminary) issue post-processing by developers. This might
include a template or wizard for reports, adding information about
which versions receive fixes or RFE, etc.

Deliverables (WIP)

Developer-side UI
  Patches for client-side Roundup:
    - add missing fields/options to search (e.g. Stages, fix 'not closed')
    - allow multiple choices for versions, components and keywords
    - add checkboxes to multiselects
    - avoid drop-downs above multiselects (janitors misclick a lot!)
    - support for mass-selection/display (3rd party patch available)
    - add a clutter-free 'search in all issues' button
  Patches for server-side Roundup:
    - boolean searches
    - support for mass-updates (3rd party patch available)
    - maybe add regex support (3rd party patch available)

Issue categorization
    - present better statistics about the open issues
    - maybe add grab-bag issues (3rd party patch available)
    - assorted closing, updating and poking old issues

User-side interface
  Patches for client-side Roundup:
    - inline help for fields in issue creation
    - adapt current docs (and Brett's guide) for easy access during reporting
  Patches for server-side Roundup:
    - maybe add a wizard for filling bugs (3rd party patch available)

Timeline
TBD, first part should definitely be done before PyCon, how much of
second part can be done for PyCon and 2.7/3.1 is hard to predict
without more specific goals in this area.

Conclusion
TBD, should learn to write shorter, clear messages :/