Package detail

@iarna/toml

iarna10.2mISC2.2.5

Better TOML parsing and stringifying all in that familiar JSON interface.

toml, toml-parser, toml-stringifier, parser, stringifer, emitter, ini, tomlify, encoder, decoder

readme

@iarna/toml

Better TOML parsing and stringifying all in that familiar JSON interface.

TOML 1.0.0-rc.1

TOML Spec Support

The most recent version as of 2019-04-21: 1.0.0-rc.1

Other Versions

1.0.0-rc.1 parsers can load almost any TOML 0.4 and TOML 0.5 document, but TOML 1.0.0-rc.1 docs are not always compatible with TOML 0.4 and TOML 0.5 parsers. If you're using this to generate TOML documents and you want an older parser to be able to read them you may want to use the latest TOML 0.5 version of this module.

Example

const TOML = require('@iarna/toml')
const obj = TOML.parse(`[abc]
foo = 123
bar = [1,2,3]`)
/* obj =
{abc: {foo: 123, bar: [1,2,3]}}
*/
const str = TOML.stringify(obj)
/* str =
[abc]
foo = 123
bar = [ 1, 2, 3 ]
*/

Visit the project github for more examples!

Why @iarna/toml

Support for TOML 1.0.0-rc.1!
Highly correct! Careful adherence to spec.
See TOML-SPEC-SUPPORT for a comparison of which TOML features are supported by the various Node.js TOML parsers.
Speedy! See benchmarks at end.
BigInt support on Node 10!
100% test coverage.
Small parser bundle (if you use @iarna/toml/parse-string).
No deps.
Detailed and easy to read error messages‼

> TOML.parse(src)
Error: Unexpected character, expecting string, number, datetime, boolean, inline array or inline table at row 6, col 5, pos 87:
5: "abc\"" = { abc=123,def="abc" }
6> foo=sdkfj
       ^
7:

TOML.parse(str) → Object (example)

Also available with: require('@iarna/toml/parse-string')

Synchronously parse a TOML string and return an object.

TOML.stringify(obj) → String (example)

Also available with: require('@iarna/toml/stringify)

Serialize an object as TOML.

[your-object].toJSON

If an object TOML.stringify is serializing has a toJSON method then it will call it to transform the object before serializing it. This matches the behavior of JSON.stringify.

The one exception to this is that toJSON is not called for Date objects because JSON represents dates as strings and TOML can represent them natively.

moment objects are treated the same as native Date objects, in this respect.

TOML.stringify.value(obj) -> String

Also available with: require('@iarna/toml/stringify').value

Serialize a value as TOML would. This is a fragment and not a complete valid TOML document.

Promises and Streaming

The parser provides alternative async and streaming interfaces, for times that you're working with really absurdly big TOML files and don't want to tie-up the event loop while it parses.

TOML.parse.async(str[, opts]) → Promise(Object) (example)

Also available with: require('@iarna/toml/parse-async')

opts.blocksize is the amount text to parser per pass through the event loop. Defaults to 40kb.

Asynchronously parse a TOML string and return a promise of the resulting object.

TOML.parse.stream(readable) → Promise(Object) (example)

Also available with: require('@iarna/toml/parse-stream')

Given a readable stream, parse it as it feeds us data. Return a promise of the resulting object.

readable.pipe(TOML.parse.stream()) → Transform (example)

Also available with: require('@iarna/toml/parse-stream')

Returns a transform stream in object mode. When it completes, emit the resulting object. Only one object will ever be emitted.

Lowlevel Interface (example) (example w/ parser debugging)

You construct a parser object, per TOML file you want to process:

const TOMLParser = require('@iarna/toml/lib/toml-parser.js')
const parser = new TOMLParser()

Then you call the parse method for each chunk as you read them, or in a single call:

parser.parse(`hello = 'world'`)

And finally, you call the finish method to complete parsing and retrieve the resulting object.

const data = parser.finish()

Both the parse method and finish method will throw if they find a problem with the string they were given. Error objects thrown from the parser have pos, line and col attributes. TOML.parse adds a visual summary of where in the source string there were issues using parse-pretty-error and you can too:

const prettyError = require('./parse-pretty-error.js')
const newErr = prettyError(err, sourceString)

What's Different

Version 3 of this module supports TOML 1.0.0-rc.1. Please see the CHANGELOG for details on exactly whats changed.

TOML we can't do

-nan is a valid TOML value and is converted into NaN. There is no way to produce -nan when stringifying. Stringification will produce positive nan.
Detecting and erroring on invalid utf8 documents: This is because Node's UTF8 processing converts invalid sequences into the placeholder character and does not have facilities for reporting these as errors instead. We can detect the placeholder character, but it's valid to intentionally include them in documents, so erroring on them is not great.
On versions of Node < 10, very large Integer values will lose precision. On Node >=10, bigints are used.
Floating/local dates and times are still represented by JavaScript Date objects, which don't actually support these concepts. The objects returned have been modified so that you can determine what kind of thing they are (with isFloating, isDate, isTime properties) and that their ISO representation (via toISOString) are representative of their TOML value. They will correctly round trip if you pass them to TOML.stringify.
Binary, hexadecimal and octal values are converted to ordinary integers and will be decimal if you stringify them.

Changes

I write a by hand, honest-to-god, CHANGELOG for this project. It's a description of what went into a release that you the consumer of the module could care about, not a list of git commits, so please check it out!

Benchmarks

You can run them yourself with:

$ npm run benchmark

The results below are from my desktop using Node 13.13.0. The library versions tested were @iarna/toml@3.0.0, `toml-j0.4@1.1.1,toml@3.0.0,@sgarciac/bombadil@2.3.0,@ltd/j-toml@0.5.107, andfast-toml@0.5.4`. The speed value is megabytes-per-second that the parser can process of that document type. Bigger is better. The percentage after average results is the margin of error.

New here is fast-toml. fast-toml is very fast, for some datatypes, but it also is missing most error checking demanded by the spec. For 0.4, it is complete except for detail of multiline strings caught by the compliance tests. Its support for 0.5 is incomplete. Check out the spec compliance doc for details.

As this table is getting a little wide, with how npm and github display it, you can also view it seperately in the BENCHMARK document.

Tests

The test suite is maintained at 100% coverage:

The spec was carefully hand converted into a series of test framework independent (and mostly language independent) assertions, as pairs of TOML and YAML files. You can find those files here: spec-test.

Further tests were written to increase coverage to 100%, these may be more implementation specific, but they can be found in coverage and coverage-error.

I've also written some quality assurance style tests, which don't contribute to coverage but do cover scenarios that could easily be problematic for some implementations can be found in: test/qa.js and test/qa-error.js.

All of the official example files from the TOML spec are run through this parser and compared to the official YAML files when available. These files are from the TOML spec as of: 357a4ba6 and specifically are:

The stringifier is tested by round-tripping these same files, asserting that TOML.parse(sourcefile) deepEqual TOML.parse(TOML.stringify(TOML.parse(sourcefile)). This is done in test/roundtrip-examples.js There are also some tests written to complete coverage from stringification in: test/stringify.js

Tests for the async and streaming interfaces are in test/async.js and test/stream.js respectively.

Tests for the parser's debugging mode live in test/devel.js.

And finally, many more stringification tests were borrowed from @othiym23's toml-stream module. They were fetched as of b6f1e26b572d49742d49fa6a6d11524d003441fa and live in test/toml-stream.

Improvements to make

In stringify:
- Any way to produce comments. As a JSON stand-in I'm not too worried about this. That said, a document orientated fork is something I'd like to look at eventually…
- Stringification could use some work on its error reporting. It reports what's wrong, but not where in your data structure it was.
Further optimize the parser:
- There are some debugging assertions left in the main parser, these should be moved to a subclass.
- Make the whole debugging parser thing work as a mixin instead of as a superclass.

changelog

2.2.5

Docs: Updated benchmark results. Add fast-toml to result list. Improved benchmark layout.
Update @sgarciac/bombadil and @ltd/j-toml in benchmarks and compliance tests.
Dev: Some dev dep updates that shouldn't have any impact.

2.2.4

Bug fix: Plain date literals (not datetime) immediately followed by another statement (no whitespace or blank line) would crash. Fixes #19 and #23, thank you @arnau and @jschaf for reporting this!
Bug fix: Hex literals with lowercase Es would throw errors. (Thank you @DaeCatt for this fix!) Fixed #20
Some minor doc tweaks
Added Node 12 and 13 to Travis. (Node 6 is failing there now, mysteriously. It works on my machine™, shipping anyway. 🙃)

2.2.3

This release just updates the spec compliance tests and benchmark data to better represent @ltd/j-toml.

2.2.2

Fixes

Support parsing and stringifying objects with __proto__ properties. (@LongTengDao)

Misc

Updates for spec compliance and benchmarking:
- @sgarciac/bombadil -> 2.1.0
- toml -> 3.0.0
Added spec compliance and benchmarking for:
- @ltd/j-toml

2.2.1

Fixes

Fix bug where keys with names matching javascript Object methods would error. Thanks @LongTengDao for finding this!
Fix bug where a bundled version would fail if util.inspect wasn't provided. This was supposed to be guarded against, but there was a bug in the guard. Thanks @agriffis for finding and fixing this!

Misc

Update the version of bombadil for spec compliance and benchmarking purposes to 2.0.0

Did you know?

Node 6 and 8 are measurably slower than Node 6, 10 and 11, at least when it comes to parsing TOML!

2.2.0

Features

Typescript: Lots of improvements to our type definitions, many many to @jorgegonzalez and @momocow for working through these.

Fixes

Very large integers (>52bit) are stored as BigInts on runtimes that support them. BigInts are 128bits, but the TOML spec limits its integers to 64bits. We now limit our integers to 64bits as well.
Fix a bug in stringify where control characters were being emitted as unicode chars and not escape sequences.

Misc

Moved our spec tests out to an external repo
Improved the styling of the spec compliance comparison

2.1.1

Fixes

Oops, type defs didn't end up in the tarball, ty @jorgegonzalez‼

2.1.0

Features

Types for typescript support, thank you @momocow!

Fixes

stringify: always strip invalid dates. This fixes a bug where an invalid date in an inline array would not be removed and would instead result in an error.
stringify: if an invalid type is found make sure it's thrown as an error object. Previously the type name was, unhelpfully, being thrown.
stringify: Multiline strings ending in a quote would generate invalid TOML.
parse: Error if a signed integer has a leading zero, eg, -01 or +01.
parse: Error if _ appears at the end of the integer part of a float, eg 1_.0. _ is only valid between digits.

Fun

BurntSushi's comprehensive TOML 0.4.0 test suite is now used in addition to our existing test suite.
You can see exactly how the other JS TOML libraries stack up in testing against both BurntSushi's tests and my own in the new TOML-SPEC-SUPPORT doc.

2.0.0

With 2.0.0, @iarna/toml supports the TOML v0.5.0 specification. TOML 0.5.0 brings some changes:

Delete characters (U+007F) are not allowed in plain strings. You can include them with escaped unicode characters, eg \u007f.
Integers are specified as being 64bit unsigned values. These are supported using BigInts if you are using Node 10 or later.
Keys may be literal strings, that is, you can use single quoted strings to quote key names, so the following is now valid: 'a"b"c' = 123
The floating point values nan, inf and -inf are supported. The stringifier will no longer strip NaN, Infinity and -Infinity, instead serializing them as these new values..
Datetimes can separate the date and time with a space instead of a T, so 2017-12-01T00:00:00Z can be written as 2017-12-01 00:00:00Z.
Datetimes can be floating, that is, they can be represented without a timezone. These are represented in javascript as Date objects whose isFloating property is true and whose toISOString method will return a representation without a timezone.
Dates without times are now supported. Dates do not have timezones. Dates are represented in javascript as a Date object whose isDate property is true and whose toISOString method returns just the date.
Times without dates are now supported. Times do not have timezones. Times are represented in javascript as a Date object whose isTime property is true and whose toISOString method returns just the time.
Keys can now include dots to directly address deeper structures, so a.b = 23 is the equivalent of a = {b = 23} or [a] b = 23. These can be used both as keys to regular tables and inline tables.
Integers can now be specified in binary, octal and hexadecimal by prefixing the number with 0b, 0o and 0x respectively. It is now illegal to left pad a decimal value with zeros.

Some parser details were also fixed:

Negative zero (-0.0) and positive zero (0.0) are distinct floating point values.
Negative integer zero (-0) is not distinguished from positive zero (0).

1.7.1

Another 18% speed boost on our overall benchmarks! This time it came from switching from string comparisons to integer by converting each character to its respective code point. This also necessitated rewriting the boolean parser to actually parse character-by-character as it should. End-of-stream is now marked with a numeric value outside of the Unicode range, rather than a Symbol, meaning that the parser's char property is now monomorphic.

Bug fix, previously, 'abc''def''' was accepted (as the value: abcdef). Now it will correctly raise an error.

Spec tests now run against bombadil as well (it fails some, which is unsurprising given its incomplete state).

1.7.0

This release features an overall 15% speed boost on our benchmarks. This came from a few things:

Date parsing was rewritten to not use regexps, resulting in a huge speed increase.
Strings of all kinds and bare keywords now use tight loops to collect characters when this will help.
Regexps in general were mostly removed. This didn't result in a speed change, but it did allow refactoring the parser to be a lot easier to follow.
The internal state tracking now uses a class and is constructed with a fixed set of properties, allowing v8's optimizer to be more effective.

In the land of new features:

Errors in the syntax of your TOML will now have the fromTOML property set to true. This is in addition to the line, col and pos properties they already have.

The main use of this is to make it possible to distinguish between errors in the TOML and errors in the parser code itself. This is of particular utility when testing parse errors.

1.6.0

FIXES

TOML.stringify: Allow toJSON properties that aren't functions, to align with JSON.stringify's behavior.
TOML.stringify: Don't use ever render keys as literal strings.
TOML.stringify: Don't try to escape control characters in literal strings.

FEATURES

New Export: TOML.stringify.value, for encoding a stand alone inline value as TOML would. This produces a TOML fragment, not a complete valid document.

1.5.6

String literals are NOT supported as key names.
Accessing a shallower table after accessing it more deeply is ok and no longer crashes, eg:
```
[a.b]
[a]
```
Unicode characters in the reserved range now crash.
Empty bare keys, eg [.abc] or [] now crash.
Multiline backslash trimming supports CRs.
Multiline post quote trimming supports CRs.
Strings may not contain bare control chars (0x00-0x1f), except for \n, \r and \t.

1.5.5

Yet MORE README fixes. 🙃

1.5.4

README fix

1.5.3

Benchmarks!
More tests!
More complete LICENSE information (some dev files are from other, MIT licensed, projects, this is now more explicitly documented.)

1.5.2

parse: Arrays with mixed types now throw errors, per the spec.
parse: Fix a parser bug that would result in errors when trying to parse arrays of numbers or dates that were not separated by a space from the closing ].
parse: Fix a bug in the error pretty printer that resulted in errors on the first line not getting the pretty print treatment.
stringify: Fix long standing bug where an array of Numbers, some of which required decimals, would be emitted in a way that parsers would treat as mixed Integer and Float values. Now if any Numbers in an array must be represented with a decimal then all will be emitted such that parsers will understand them to be Float.

1.5.1

README fix

1.5.0

A brand new TOML parser, from scratch, that performs like toml-j0.4 without the crashes and with vastly better error messages.
100% test coverage for both the new parser and the existing stringifier. Some subtle bugs squashed!

v1.4.2

Revert fallback due to its having issues with the same files. (New plan will be to write my own.)

v1.4.1

Depend on both toml and toml-j0.4 with fallback from the latter to the former when the latter crashes.

v1.4.0

Ducktype dates to make them compatible with moment and other Date replacements.

v1.3.1

Update docs with new toml module.

v1.3.0

Switch from toml to toml-j0.4, which is between 20x and 200x faster. (The larger the input, the faster it is compared to toml).

v1.2.0

Return null when passed in null as the top level object.
Detect and skip invalid dates and numbers

v1.1.0

toJSON transformations are now honored (for everything except Date objects, as JSON represents them as strings).
Undefined/null values no longer result in exceptions, they now just result in the associated key being elided.

v1.0.1

Initial release

Package detail

readme

@iarna/toml

** TOML 1.0.0-rc.1 **

TOML Spec Support

Other Versions

Example

Why @iarna/toml

TOML.parse(str) → Object (example)

TOML.stringify(obj) → String (example)

[your-object].toJSON

TOML.stringify.value(obj) -> String

Promises and Streaming

TOML.parse.async(str[, opts]) → Promise(Object) (example)

TOML.parse.stream(readable) → Promise(Object) (example)

readable.pipe(TOML.parse.stream()) → Transform (example)

Lowlevel Interface (example) (example w/ parser debugging)

What's Different

TOML we can't do

Changes

Benchmarks

Tests

Improvements to make

changelog

2.2.5

2.2.4

2.2.3

2.2.2

Fixes

Misc

2.2.1

Fixes

Misc

Did you know?

2.2.0

Features

Fixes

Misc

2.1.1

Fixes

2.1.0

Features

Fixes

Fun

2.0.0

1.7.1

1.7.0

1.6.0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

v1.4.2

v1.4.1

v1.4.0

v1.3.1

v1.3.0

v1.2.0

v1.1.0

v1.0.1

TOML 1.0.0-rc.1