[IPython-dev] [nbconvert] Reining in the number of constructor arguments

[Paul Ivanov]

On Fri, Nov 30, 2012 at 3:05 PM, David Warde-Farley
<d.warde.farley at gmail.com> wrote:
> I'm wondering how to solve (and how you guys think I should solve) a
> certain problem I foresee, of a potentially rapidly growing number of
> constructor arguments for Converter classes, which need to be
> maintained across constructors in order to keep the front-end calling
> code generic (one could use *args and **kwargs catch-alls but that's
> pretty ugly).

I don't think it's ugly - I think it's the right way to do it. It allows
one to easily add keyword arguments to all of the subclasses of
Converter, for example, by writing the relevant code only in one place.
See the --stdout flag I've proposed in
https://github.com/ipython/nbconvert/pull/62


> A pull request that @Jovani started and I have finished off adds a
> highlight_source argument to the constructors for all of the classes.
> I have some features in mind that control fine-grained output
> behavior. Passing these as either positional or keyword arguments
> requires every other Converter class to know about every possible
> argument, which is a maintenance nightmare, and doesn't necessarily
> make sense: part of the problem is that not every backend even
> *supports* all of the options, e.g. the ConverterPy can't highlight
> source in code blocks because that doesn't make any sense in terms of
> the output type.

And for this reason, that flag should not be exposed globally, the same
way that all possible flags to ipython notebook aren't exposed when one
does just "ipython --help".

The solution I think makes sense is to have formatter specific flags be
exposed only when a specific formatter is selected.

So `nbconvert -f html -h` should show --highlight-source  as an option,
whereas `nbconvert -f py -h` should not.


> My thought on solving this goes something like this:
> - on the front-end, delegate all but very core functions that are
> backend-agnostic to a sub-parser
> - on the back-end, have either the "namespace" object produced by
> argparse or a dictionary constructed from it passed as a *single*
> argument to the constructor.
> - have each class examine this namespace

The nice thing about kwargs is that it's evident from the method
signature where they are being used, whereas passing around an opaque
"namespace" object as a single argument doesn't really buy you anything,
since **kwarg is a single argument, anyway ;)

But I haven't thought about this very carefully yet.

> Questions:
> - Should a backend that can't highlight source, or perform optional
> function X, raise an error, a warning, or fail silently? My gut says
> warning is the right balance of informative and yet annoying. (should
> I use the warnings module, logging.warn, ...?)

I think it should fail, the same way `ipython -X` fails with

[TerminalIPythonApp] Bad config encountered during initialization:
[TerminalIPythonApp] Unrecognized flag: '-X'


> This raises one problem, though, in that some defaults as per argparse
> will raise warnings automatically in some backends, e.g. without
> --no-highlight, the default argument of `True` will raise a warning
> when the output type is a Python script.
>
> A possible solution: make argparse's default be 'None' for options
> even when True/False is the default. Have converter classes expose a
> class-level `config_defaults` dict, where values are 2-tuples:
> (default value, legal_values), which the class uses to address its
> configuration needs for things that come up as None in the
> configuration object passed in, and raise warnings for invalid values
> where appropriate.
>
> Thoughts? Am I off my rocker?

In summary, I think it would be annoying to expose all of the flags that
don't apply to all the converters globally. If the Converter base class
accepts it as a flag, we have it show up in the docs, whereas formatter
specific flags should only show up in the help printout for specific
formatters.


best,
--
Paul Ivanov
314 address only used for lists,  off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7