felixschl<neodoc>
Beautiful, handcrafted command line interfaces
About | Features | Installation | Usage | Language overview | License | Playground new
About
<neodoc> is a revised implementation of the docopt language for node. In brief, it offers a unique way to author command lines by writing the command line's help text first and then deriving a matching parser from it, which can then be applied to user input. The advantages are numerous:
- No boilerplate - just write your help-text
- Full control over beautiful, hand-crafted help texts
- Documentation comes first - hence your users come first
- Documentation is always right - your help-text is necessarily correct
- Version-controlled help-text - the help-text becomes a regular part of your codebase
This implementation features error reporting, both for users and developers, reading values from environment variables, type coercion and much more. For an (in-)comprehensive comparison to the original, click here. To take neodoc for a ride, click here.
A note to potential adopters and contributors: Neodoc is divided into two distinct parts — parsing the specification and parsing the argv, given the specificiation. Theoretically, the origin of the specification does not matter and the argv parser could be used standalone as it offers a more "correct" parse than most cli parsers out there, since it parses the input guided by the specification, rather than parsing the input and then matching it to the specification. See the "Features" section below. If neodoc finds adoption, I would not be surprised to see projects mimicking a yargs-like interface that use the neodoc parser, even though it somewhat defies the original idea of docopt.
Features
- Derive command line interface from help text
- Helpful error messages for both developers and users
- Options-first parsing to compose large programs (see git example)
- Fallback to alternate values:
Argv -> Environment -> Defaults -> Empty - Convenient, concise and widely accepted POSIX-style syntax
-f[=ARG], --foo[=ARG]options<arg>,ARGpositionalsclone,pull, etc. commands[<arg>]optional groupings(<arg>])required groupings[-f ARG]POSIX-style flags-f[=ARG]...repeating elements--end of options separator-stdin marker- 99% compatible with a typical
git <command> --helpoutput - Full overview of the language →
- Stop parsing at any option and collect successive input as the argument to
that option. Similar to
--but for named options (and their aliases). - Specification parsing (help-text parsing) is separated from argv parsing and can be used for other projects outside of neodoc. Work is underway to make the argv parser usable from JS as well.
- Count repeated flags
- Parses values into primitive JS types (bool, string, number)
- Correct and smart argument parsing. For example, neodoc has absolutely no
problem parsing this input:
tar -xvzfsome-dir/some-file, given a specification of:usage: tar [-xvzf FILE]while other parses would not know that the option stack ends at-fand falsly parse this as-x -v -z -f -s -o -m -e=-dir/some-fileat best.
Installation
npm install --save neodocUsage
neodoc.run(help | spec, opts)
Parse and apply the given docopt help text. Alternatively, pass the output of
neodoc.parse. If no options are provided, apply it to process.argv and
process.env. The result is a mapping of key -> value, where the key is the
canonical form of the option and its alias, if available.
Options:
opts.dontExit- Do not exit upon error or when parsing--helpor--version. Instead throw and error / return the value.opts.env- Overrideprocess.envopts.argv- Overrideprocess.argvopts.optionsFirst- Parse until the firstcommandor<positional>argument, then collect the rest into an array, given the help indicates another, repeatable, positional argument, e.g. :[options] <ommand> [<args>...]opts.smartOptions- Enable parsing groups that "look like" options as options. For example:[-f ARG...]means[-f=ARG...]opts.stopAt- Stop parsing at the given options, i.e.[ -n ]. It's value will be the rest of argv.opts.requireFlags- Require flags be present in the input. In neodoc, flags are optional by default and can be omitted. This option forces the user to pass flags explicitly, failing the parse otherwise.opts.laxPlacement- Relax placement rules. Positionals and commands are no longer solid anchors. The order amongs them, however, remains fixed. This implies that options can appear anywhere.opts.versionFlags- An array of flags that trigger the special version behavior: Print the program version and exit with code 0.opts.version- The version to print for the special version behavior. Defaults to finding the version of the nearest package.json file, relative to the executing main module. Note that disk IO is only performed ifopts.versionFlagsis non-empty andopts.versionis not set.opts.helpFlags- An array of flags that trigger the special help behavior: Print the full program help text and exit with code 0.opts.repeatableOptions- Allow options to be repeated even if the spec does not explicitly allow this. This "loosens" up the parser to accept more input and makes for a more intuitive command line. Please note: repeatability is still subject to chunking (useopts.laxPlacementto relax this further).opts.transforms.presolve- an array of functions to be called prior to "solving" the input. This function takes the spec as it's only parameter. At this point, the spec is mostly untouched by neodoc with the exception of smart-options which runs as a fixed transform prior to user-provided callbacks ifsmart-optionsis true. Transforms that need to be aware of option stacks and[...-options]references should run here as this information is lost during the solving transforms.opts.transforms.postsolve- an array of functions to be called after "solving" the input, just prior to passing the spec to the arg-parser. This function takes the spec as it's only parameter. At this point, the spec has been fully solved, expanded and canonicalised.opts.allowUnknown- Collect unknown options under a special key?instead of failing. Useful to send an unknown subset of options to another program.
For example:
#!/usr/bin/env node
const neodoc = require('neodoc');
const args = neodoc.run(`
usage: git [--version] [--help] [-C <path>] [-c <name=value>]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p|--paginate|--no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>...]
`, { optionsFirst: true, smartOptions: true });
if (args['<command>'] === 'remote') {
const remoteArgs = neodoc.run(`
usage:
git remote [-v | --verbose]
git remote add [-t <branch>] [-m <master>] [-f] [--tags|--no-tags]
[--mirror=<fetch|push>] <name> <url>
git remote rename <old> <new>
git remote remove <name>
git remote [-v | --verbose] show [-n] <name>...
git remote prune [-n | --dry-run] <name>...
git remote [-v | --verbose] update [-p | --prune] [(<group> | <remote>)...]
`, { argv: ['remote'].concat(args['<args>']), smartOptions: true })
// ...
} else { /* ... */ }See the examples folder for a more sophisticated version of the above example.
neodoc.parse(help, opts)
Parse the docopt text and derive the specification along with some meta information.
The specification is the canonical representation of the CLI as described by it's
help text and can be used for building parsers etc. The output is a plain JS object
and can be serialized. The output can further be passed to neodoc.run. This avoids
neodoc having to parse and solve the original help text again, since parsing JSON
is a order of magnitude faster to parse.
Language overview and terminology
This section gives an overview over the neodoc cli specification language. Keywords are highlighted.
The over-arching format could be described as follows:
Usage: <program> [<argument>...] [| <argument> [<argument>...]]
[ [or:] <program> [<argument>...] [| <argument> [<argument>...]]
]*
[options:
[<argument> [<description and meta tags>]
]*
]*Where <argument> may be any of the arguments described in the following
subsections.
A full example:
usage: git fetch [options] [<repository> [<refspec>...]]
or: git fetch [options] <group>
or: git fetch --multiple [options] [(<repository> | <group>)...]
or: git fetch --all [options]
options:
-v, --verbose be more verbose
-q, --quiet be more quiet
--all fetch from all remotes
-a, --append append to .git/FETCH_HEAD instead of overwriting
--upload-pack <path> path to upload pack on remote end
-f, --force force overwrite of local branch
-m, --multiple fetch from multiple remotes
-t, --tags fetch all tags and associated objects
[...]1. Arguments
At the heart of the language are command line arguments. There are three
fundamental types of arguments: options, positional arguments and
commands. Options are arguments that start with either a single or a double
dash ('-'), commands are literal matches of a certain string and positionals
constitute everything else. Read on below for more detail on each argument
type.
1.1. Options
Options are those arguments that start with either one or two dashes. They are referred to as "short" and "long" options respectively throughout this document and the source code of neodoc.
Options may take an potentially optional option-argument. Options that do not are referred to as flags. Options that do specify an option-argument but declare it as being optional may behave as flags if an argument could not be consumed at runtime.
The following is true for all options:
- Options may take an optional "option-argument"
- Options may be repeated using
... - Adjacent options are not fixed in position:
-a -bis equivalent to-b -a. Likewise,-abis equivalent to-ba. This also holds true for options that take option-arguments. - Options that are repeated collect values into an array
- Flags that are repeated count the number of occurrences
- Flags that are not repeated simply yield
trueif present - Flags and options with an optional option-argument can always be omitted from the input. They simply won't yield anything in the output value mapping.
- Options may alias a short (one-character) form with a long form, e.g.:
-v, --verbose - Options that take an argument can specify a
[default: <value>]in the option section as fallback. - Options that take an argument can specify a
[env: <key>]in the option section as fallback.
1.1.1. Long options
Long options are lead by two dashes and may take an potentially optional option-argument.\ For example:
--long <ARG>the option-argument is loosely bound--long <ARG>the option-argument is loosely bound and optional. (#55)--long=<ARG>the option-argument is explicitly bound--long[=<ARG>]the option-argument is explicitly bound an optional[--long <ARG>]the option-argument is explicitly bound via thesmart-optionssetting[--long [<ARG>]]the option-argument is loosely bound via thesmart-optionssetting and optional
Note that all of the above forms could be followed by a ..., indicating that
this option may appear one or more times. The repeated occurrence does not
necessarily need to be adjacent to the previous match. Repeated occurrences
are collected into an array or into a count if the option qualifies as a flag.
Note that successive dashes are allowed: --very-long-option.
1.1.2. Short options
Short options are lead by one dash and may take an potentially optional option-argument. A short option is a one character identifier, but can be "stacked".
For example:
-a <ARG>the option-argument is loosely bound to-a-a=<ARG>the option-argument is explicitly bound to-a-a<ARG>the option-argument is explicitly bound to-a-aARGthe option-argument is loosely bound to-a-a [<ARG>]the option-argument is loosely bound to-a. (#55)-a=<ARG>the option-argument is explicitly bound to-a-a[=<ARG>]the option-argument is explicitly bound to-aan optional[-a <ARG>]the option-argument is explicitly bound to-avia thesmart-optionssetting[-a [<ARG>]]the option-argument is loosely bound to-avia thesmart-optionssetting and optional
Note, however that only the last option in the "option stack" may actually bind an argument:
-abcis equivalent to-a -b -c-abc <ARG>is equivalent to-a -b -c <ARG>-abc=<ARG>is equivalent to-a -b -c=<ARG>-abc[=<ARG>]is equivalent to-a -b -c=<ARG>
...essentially nothing changes when options are stacked. Key is that only the last option in the stack may bind and consume arguments.
Again, note that all of the above forms could be followed by a ..., indicating
that this option may appear one or more times. It is important to note that the
repeatability is assigned to all options in the stack! Repeated occurrences
are collected into an array or into a count if the option qualifies as a flag
(hence for all but the last options in the stack).
1.1.3. Option-arguments
Option-arguments are arguments bound to options. If an option is said to take an option argument that is not optional, any attempt to match an option without the argument will result in an immediate parse error. Should an option-argument be declared optional and not matched during parsing, it may be treated as a flag and be substituted.
1.1.3.1. Option-argument bindings
- "loose" binding: the option-argument is in adjacent position, but needs to be confirmed in the 'options' section. Should confirmation not take place, the adjacent argument is treated as a positional.
- "explicit" binding: the option-argument is explicitly bound due to a lack of whitespace, an equal sign or through 'smart-options'.
1.1.4 The option secion
The option section gives a chance to add more information about options, such as their default value, their alias or their backing environment variable. Furthermore, options appearing in the option section may also indicate if the option is supposed to be repeatable or not. There is more information on this topic in section "1.7 - References - [options]".
- An alias is assigned via
-v, --verbose - A default value is assigned via
[default: value] - An environment variable is assigned via
[env: MY_KEY]
For example:
options:
-f, --foo BAR This is foo bar. [env: FOO_BAR] [default: 123]The text is pretty flexible and can be arranged as the author pleases. For example:
options:
-f, --foo BAR...
This is foo bar.
[env: FOO_BAR] [default: 123]1.2. Positionals
Positionals are arguments that do not lead with any dashes. The position of their occurrence matters and options are "bounded" by them in that an option declared before an positional argument may not occur after that positional. (#24) Positional arguments are distinguished from commands by being either enclosed in angled brackets or being all upper case.
For example:
<ARG>is a positional element named<ARG>ARGis a positional element namedARG[<ARG>]is an optional positional element named<ARG>[ARG]is an optional positional element namedARG[<ARG>]...is an optional positional element named<ARG>that repeats<ARG>...is a positional element named<ARG>that repeats
Positional arguments either yield a single value if not repeated or an array of values if repeated. Note that contrary to options, repetition must occur directly adjacent to the previous match. (#24)
1.3. Commands
Commands are a specialized form of positionals that require to be matched literally, including casing. All other rules that apply to positionals apply to commands. They yield a boolean indicating their presence or a count indicating the number of their occurrences if repeated.
For example:
commandmust be matched with input "command" on argvcommand...must be matched on ore more times with input "command" on argv
1.4. EOA - end-of-arguments
The EOA (end-of-arguments) is understood as the separator between known and
unknown arguments. The eoa is typically -- but any option can become one by
using the 'stop-at' setting.\o
For example:
---- ARGS-- ARGS...[-- ARGS...][-- [ARGS...]]- ...and so on — they all have the same meaning.
1.5. Stdin marker
The stdin flag is a special, unnamed short-option: -. It's presence
indicates that the program should be reading from standard input.
1.6. Groups
Groups are the only recursive argument type. Groups describe one or more mutually exclusive sets of arguments called "branches". At least one branch needs to yield a successful parse for the group to succeed.
For example:
(foo | bar qux)means either match commandfooor commandbardirectly followed by commandqux.[foo | bar qux]means either match commandfooor commandbardirectly followed by commandqux, but backtrack on failure and ignore the group.(foo | bar qux)...means either match commandfooor commandbardirectly followed by commandqux, repeatedly. During repetition another branch can be matched, so this is valid:foo bar qux bar qux foo. The output is:{ "foo": 2, "bar": 2, "qux": 2 }.
The following is true for all groups:
- Groups can be repeated:
... - Groups can be optional using brackets:
[ foo ] - Groups can be required using parenthesis:
( foo ) - Groups must not be empty
- Groups must contain 1 or more branches
- Groups succeed if at least one branch succeeds
- Multiple successful branch matches are weighted and scored
1.6.1. Matching branches
Branches describe multiple mutually exclusive ways to parse a valid program. Branches can appear at the top-level or in any group. Since branches are mutually exclusive, only one branch can ever succeed. If multiple branches succeed, the highest scoring winner is elected. Generally, the depth of the parse within the branch (that is how deep into the branch the parse succeeded) as well as the weighting of the matched arguments matters. Arguments that were substituted by values in environment variables, or by their defaults or empty values, will have a lower ranking score than those that were read from argv.
1.7. References - [options]
This is not a real argument and not part of the canonical specification. It is
used to indicate that the entire "options" section should be expanded in it's
place. Since this approach lacks information about the relation between options,
options are all expanded as optional and are exchangeable with adjacent options
(#57). One exception to this rule is where an option that
is defined in the option section also appears directly adjacent to the
[options] reference tag.
For example:
usage: prog [options] (-f | -b)
options:
-f foo
-b barThis program won't accept the input -f -b as -f and -b are declared
mutually exclusive from one another.
Likewise:
usage: prog [options] --foo ARG
options:
-f, --foo ARGHere, --foo won't be expanded again and hence remain required.
Deviations from the original
This implementation tries to be compatible where sensible, but does cut ties when it comes down to it. The universal docopt test suite has been adjusted accordingly.
- Better Error reporting. Let the user of your utility know why input was rejected and how to fix it
- Optional arguments. Neodoc understands
--foo[=BAR](or-f[=<bar>]) as an option that can be provided either with or without an argument. - Alias matches. If
--verboseyields a value, so will-v(given that's the assigned alias). - Flags are optional by default. There's arguably no reason to force the
user to explicitly pass an option that takes no argument as the absence of the
flag speaks — the key will be omitted from the output. This is also the
case for flags inside required groups. E.g.: The group
(-a -b)will match inputs-a -b,-ab,-ba,-b -a,-b,-aand the empty input. To disable this behaviour, enableoptions.requireFlags(seeneodoc.run).\ Please note that the default behaviour may change in a future version of neodoc — refer to #61. - All arguments in a group are always required. This is regardless of
whether or not the group itself is required - once you start matching into the
group, all elements that are indicated as required have to be matched, either
by value or via fallbacks.\
For example:
Usage: prog [<name> <type>]prog foo, but passprog foo bar. The rationale being that this is more general, since if the opposite behaviour (any match) was desired, it could be expressed as such:Usage: prog [[<name>] [<type>]] - No abbreviations:
--verdoes not match--verbose. (mis-feature in the original implementation) - There is no
nullin the resulting value map.nullsimply means not matched - so the key is omitted from the resulting value map. - Smart-options. Options can be inferred from groups that "look like"
options:
Usage: foo [-f FILE]would then expand toUsage: foo [-f=FILE] - Environment variables. Options can fall back to environment variables,
if they are not explicitly defined. The order of evaluation is:
- User input (per
process.argv) - Environment variables (per
[env: ...]tag) - Option defaults (per
[default: ...]tag)
- User input (per
Stricter document layout. Neodoc imposes more restrictions on the format of the help text in order to achieve certain goals, such as:
Neodoc allows associating option aliases over newlines:
options: -f, --foo this is fooNeodoc does not require 2 spaces between option and argument. Instead, only those arguments that visually "look like" arguments are considered for binding (i.e. all-caps:
ARGand in-angles:<arg>):options: -f, --foo ARGShould there be any ambiguity, the option can also be explicitly bound:
options: -f, --foo=ARGThe same is true for optional arguments:
options: -f, --foo [ARG] -b, --bar[=ARG]Neodoc is more conservative on learning about options in order to prevent subtly introducing options:
usage: prog [options] options: --foo this is foo, and it is similar to some-command's --bar in that it does qux.Here, the author is talking about
--barin another context, so it should not be considered an option to the program. Neodoc figures this out based on indentation of the previous description start.Neodoc allows interspersing spaces in between usage layouts:
usage: prog foo bar prog qux wooit is important to not that in this format (above), identation of the layout is required. or:
usage: prog foo bar or: prog qux woo- Neodoc requires an
Options:section title in order to parse any options (see #76 for discussion). note: I am interested in a proposal on how to lift this requirement.
- Neodoc requires an
License
<neodoc> is released under the MIT LICENSE.
See file LICENSE for a more detailed description of its terms.