root/masterdriverz/use-expand/dev-notes/commandline.rst @ marienz%2540gentoo.org-20061109120338-8e9edd06a397bb65

=======================
 Commandline framework
=======================

Overview
========

pkgcore's own commandline tools and ideally also most external tools
use a couple of utilities from pkgcore.util.commandline to enforce a
consistent interface and reduce boilerplate. There are also some
helpers for writing tests for scripts using the utilities. Finally,
pkgcore's own scripts are started through a single wrapper (just to
reduce boilerplate).

Writing a script
================

Whether your script is intended for inclusion with pkgcore itself or
not, the first things you should write are a commandline.OptionParser
subclass (unless your script takes no commandline arguments) and a
main function. The OptionParser is a lightly customized
optparse.OptionParser, so the standard `optparse documentation`_
applies. Differences include:

* A couple of standard options and defaults are added. Some of this
  uses __init__.py, so if you override that (which you will) remember
  to call the base class (with any keyword arguments you received).
* The "Values" object used is a subclass, with a "config" property
  that autoloads the user's configuration. You should access this as
  late as possible for a more responsive ui.
* check_values applies some minor cleanups, see the module for
  details. Remember to call the base method (you will usually want to
  do some things here).

The "main" function takes an optparse "values" object generated by
your option parser and two pkgcore.util.formatters.Formatter
instances, one for stdout and one for stderr. This one should do the
actual work your script does.

The return value of the main function is your script's exit status.
Returning None is the same thing as returning 0 (success).

If you have used optparse before you might wonder why main only
receives an optparse values object, not the remaining arguments. This
is handled a bit differently in pkgcore: if you handle arguments you
should sanity-check them in check_values and store them on the values
object. check_values should always return an empty tuple as second
argument, either because no arguments were passed or because they were
all accepted by check_values. We believe this makes more sense, since
it stores everything learned from the commandline on a single object.

All output *has* to go through the formatter. If you use "print"
directly the formatter will lose track of where it is in the line,
which will cause weird output if you use the "wrap" option of the
formatter. The test helpers also rely on all output going through the
formatters.

To actually run your script you call pkgcore.util.commandline.main
(do not confuse this with your own script's main function, the two are
quite different). The simplest (and most common) call is
``commandline.main({None: (yourscript.OptionParser, yourscript.main)})``.
The weird dict is used for subcommands_. The recommended place to put
this call is in a tiny script that just imports your actual script
module and calls commandline.main. Making your script an actual module
you can import means it can be tested (and it can be useful in
interactive python or for quick hacky scripts).

commandline.main takes care of a couple of things, including setting
up a reporter for the standard library's logging_ package and
swallowing exceptions from the configuration system. It does *not*
swallow any other exceptions your script might raise (although this might
become an option in the future).

check_values and main: what goes where
--------------------------------------

The idea (as you can guess from the names) is that check_values makes
sure everything passed on the commandline makes sense, but no more
than that.

* The best way to report incorrect commandline parameters is by
  calling ``error("error message goes here")`` on the option parser.
  You cannot do this from main, since it has no access to the option
  parser. Please do not try to print something similar through the
  ``err`` formatter here, shift the code to check_values.
* check_values does not have access to the out or err formatter. The
  only way it should "communicate" is through the error (or possibly
  exit) methods. If you want to produce different kinds of output, do
  it in main. (it is possible the option parser will grow a
  ``warning`` method at some point, if this would be useful let us
  know (file a trac ticket).
* Use common sense. If it is part of your script's main task it should
  be in main. If it changes the filesystem it should definitely be in
  main.

Subcommands
===========

The main function recently gained some support for subcommands (which
you probably know from most version control systems). If you find
yourself trying to reimplement this kind of interface with optparse,
or one similar to emerge with a couple of mutually exclusive switches
selecting a mode (--depclean, --sync etc.) then you should try using
this subcommand system instead.

To use it, simply define a separate OptionParser and main function for
every subcommand and use the subcommand name as the key in the dict
passed to commandline.main. The key ``None`` used for "no subcommand"
can still be used too, but this is probably not a good idea.

If there is no parser/main pair with the key ``None`` and an
unrecognized subcommand is passed (including ``--help``) an overview
of subcommands is printed. This uses the docstring of the __main__
function, so put something useful there. If there is a ``None`` parser
you should include the valid subcommands in its ``--help`` output,
since there is no way to get at commandline.main's autogenerated
subcommand help if a ``None`` parser is present.

pwrapper
========

Because having a dozen of different scripts each just calling
commandline.main would be silly pkgcore's own scripts are all symlinks
to a single wrapper which imports the right actual script based on the
``sys.argv[0]`` it is called with and runs it. The script module needs
to define either a commandline_commands dict (for a script with
subcommands) or a class called OptionParser and function called main
for this to work.

The script used in the source tree also takes care of inserting the
right pkgcore package on sys.path. Installed pkgcore uses a different
wrapper without this magic.

If you write a new script that should go into pkgcore itself, use the
wrapper. If you maintain it externally and do not have a lot of
scripts, don't bother duplicating this wrapper system. Don't bother
duplicating the path manipulation either: if you put your script in
the same directory your actual package or module is in (no separate
"bin" directory) and don't run it as root no path manipulation is
required.

Tests
=====

Because additions to the default options pkgcore uses can make your
script unrunnable it is *critical* to have at least rudimentary tests
that just instantiate your parser. Because optparse defaults to
calling sys.exit for a parse failure and the pkgcore version also
likes to load the user's configuration files, writing those tests is
slightly tricky. ``pkgcore.test.scripts.helpers`` tries to make it
easier. It contains a mangle_parser function that takes an
OptionParser instance and makes it raise exceptions instead of
exiting. It also contains a mixin with some extra assert methods that
check if your option parser and main function have the desired effect
on various arguments and configurations. See the docstrings for more
information.

.. _`optparse documentation`:
     http://docs.python.org/lib/module-optparse.html
.. _logging: http://docs.python.org/lib/module-logging.html