Yet another Python SNMP lib

[Kouli]

Python list users, this is a rather long message primarily addressed to
Vincent, but I am sending a copy to this list for those interested in SNMP
and/or "object mapping frameworks"...

Hello Vincent,

I have been using your snimpy in few small scripts. I like it for replacing
long dot/number OID strings with human-readable identifiers. Thanks for
your work! However, I think you introduced too much abstraction to the SNMP
protocol itself. It is fine for smaller projects, but when I tried to talk
to a more sophisticated SNMP manager, it does not use the protocol in an
optimal way. You must know it, I have already discussed some minor changes
with you two years ago...

People usually use SNMP to read some status or statistical data. When
talking about performance, they usually mean to read only few variables
from hundreds to thousands of devices. But, in such a case, it isn't a
challenge to write few numerical OID strings in their scripts. The
"identifier abstraction" is useful when you have to get/set tens to
hundreds variables on a single large SNMP manager (an Ethernet switch fully
manageable by SNMP - i.e. to the same extent as by command line -, or, in
my case, a GPON OLT serving thousands of ONTs).

The Easy SNMP package you mention in snimpy docs is optimized for the first
usage scenario - few variables @ thousands managers. For the second -
hundreds variables @ single manager - (a) a full control over varbind sets
(to lower the number of round-trips needed) and (b) easy mapping of SNMP
OIDs to constructs used in your programming language is what matters.

With snimpy, you concentrated at (b), but I am not able to:

   - aggregate GET messages,
   - (similar) walk tables reading more than 1 column efficiently.

Well, why am I writing this? I would like to discuss with you, Vincent,
"yet another Python SNMP package" - a one I would like to create. I believe
you must have solved many issues while creating snimpy so I am asking you
to evaluate my thoughts and advise me to any potential problems.

The new package should:

   1. represent atom SNMP variables using some lightweight Python classes:
   instances would serve as variable values, classes would know its OID (and
   original name, comments/textual conventions from MIB etc.),
   2. (not sure) represent SNMP tables using Python classes to make some
   table operations easier, especially those with "conceptual rows" (SMIv2
   RowStatus),
   3. be able to "import" whole (or part of a) MIB file as if it was a
   Python module: this import will define the above mentioned classes; the
   MIBs would not be stored in a system-wide central directory, but rather at
   a place where application-specific packages are; even Python's package
   import machinery could be misused (read: overriden) to achieve this,
   4. (not sure) make the above import available to autocompletion in text
   editors,
   5. (far future) be able to hint/change the imported classes: for MIBs
   which are not fully standards compliant, or, to *shorten* variable names
   (get rid of long prefixes in names, which exist only because of need of
   global [at least within a MIB] uniqueness - now the names will be scoped
   [columns scoped to table class do not need table's identification in their
   names' prefixes], so can be shorter),
   6. replace snimpy's `manager.varname = varvalue` with
   `manager.set(varinst, ...)` (varinst being above described class instance),
   add `manager.get(varclass, ...)`, add more such methods for bulk walks,
   get-next, methods raising exceptions vs. methods returning varbinds etc. --
   allow for full control over content of SNMP message payload (including bulk
   repetition counts etc.)
   7. (not sure if it can lift performance for any manager, to be proven in
   C at first) support asynchronous SNMP transactions (asyncio?), i.e. send a
   next request to the same manager before we receive a response to previous
   one (some managed Ethernet switches use multi-core CPUs, so they -
   potentially - might be able to process parallel requests; example: a very
   large table with many columns - I can bulk walk columns in parallel, or,
   walk all index values [only] at first and, knowing them, issue requests for
   "complete rows" in parallel)
   8. (not sure, future) make the performance-critical part, the SNMP
   protocol library (ASN.1, v3 encryption) "pluggable" - pure PySNMP / faster
   Net-SNMP with binary dependencies (any recommendation with respect to (7.)?)
   9. (probably) support only recent Python 3, drop support for Python 2
   completely.

Thank you for any review/idea!

Milan