juanjoDiaz
@json2csv/node
Fast and highly configurable JSON to CSV converter. It fully support conversion following the RFC4180 specification as well as other similar text delimited formats as TSV.
@json2csv/node exposes two modules to integrate json2csv with the Node.js Stream API for stream processing of JSON data.
This package includes two modules:
- Node Transform: Node.js Transform Stream that ingests JSON and outputs csv. Ideal to process streams (http responses, file contents, ...) in Node.js.
- Node Async Parser: Wraps the
Node Transformto offer a friendly promise-based API.
Features
- Fast and lightweight
- Support for standard JSON as well as NDJSON
- Scalable to infinitely large datasets (using stream processing)
- Advanced data selection (automatic field discovery, underscore-like selectors, custom data getters, default values for missing fields, ...)
- Support for custom input data transformation
- Support for custom csv cell formatting.
- Highly customizable (supporting custom quotation marks, delimiters, eol values, etc.)
- Automatic escaping (preserving new lines, quotes, etc.)
- Optional headers
- Unicode encoding support
- Pretty printing in table format to stdout
Other json2csv packages
There are multiple flavours of json2csv:
- Plainjs: Includes the
ParserAPI and a newStreamParserAPI which doesn't the conversion in a streaming fashion in pure js. - Node: Includes the
Node TransformandNode Async ParserAPIs for Node users. - WHATWG: Includes the
WHATWG Transform StreamandWHATWG Async ParserAPIs for users of WHATWG streams (browser, Node or Deno). - CLI: Includes the
CLIinterface.
And a couple of libraries that enable additional configurations:
- Transforms: Includes the built-in
transformsfor json2csv (unwind and flatten) allowing the using to transform data before is parsed. - Formatters: Includes the built-in
formattersfor json2csv (one for each data type, an excel-specific one, etc.). Formatters convert JSON data types into CSV-compatible strings.
Requirements
- Node v16+
Installation
NPM
You can install json2csv as a dependency using NPM.
$ npm install --save @json2csv/nodeYarn
You can install json2csv as a dependency using Yarn.
$ yarn add --save @json2csv/nodeNode Transform
For Node.js users, the Streaming API is wrapped in a Node.js Stream Transform. This approach ensures a consistent memory footprint and avoids blocking JavaScript's event loop.
The async API takes a second options arguments that is directly passed to the underlying streams and accepts the same options as the standard Node.js streams, plus the options supported by the Stream Parser.
This Transform uses the StreamParser under the hood and support similar events.
Usage
import { createReadStream, createWriteStream } from 'fs';
import { Transform } from '@json2csv/node';
const input = createReadStream(inputPath, { encoding: 'utf8' });
const output = createWriteStream(outputPath, { encoding: 'utf8' });
const opts = {};
const transformOpts = {};
const asyncOpts = {};
const parser = new Transform(opts, asyncOpts, transformOpts);
const processor = input.pipe(parser).pipe(output);
// You can also listen for events on the conversion and see how the header or the lines are coming out.
parser
.on('header', (header) => console.log(header))
.on('line', (line) => console.log(line));Parameters
Options
ndjson<Boolean> indicates that the data is in NDJSON format. Only effective when using the streaming API and not in object mode.fields<DataSelector[]> Defaults to toplevel JSON attributes.transforms<Transform[]> Array of transforms to apply to the data. A transform is a function that receives a data recod and returns a transformed record. Transforms are executed in order.formatters<Formatters> Object where the each key is a Javascript data type and its associated value is a formatters for the given type.defaultValue<Any> value to use when missing data. Defaults to<empty>if not specified. (Overridden byfields[].default)delimiter<String> delimiter of columns. Defaults to,if not specified.eol<String> overrides the default OS line ending (i.e.\non Unix and\r\non Windows).header<Boolean> determines whether or not CSV file will contain a title column. Defaults totrueif not specified.includeEmptyRows<Boolean> includes empty rows. Defaults tofalse.withBOM<Boolean> with BOM character. Defaults tofalse.
Transform Options
See the Duplex stream options for more details.
Async Options
Options used by the underlying parsing library to process the binary or text stream.
Not relevant when running in objectMode.
Buffering is only relevant if you expect very large strings/numbers in your JSON.
See @streamparser/json for more details about buffering.
stringBufferSize<number> Size of the buffer used to parse strings. Defaults to 0 which means to don't buffer. Min valid value is 4.numberBufferSize<number> Size of the buffer used to parse numbers. Defaults to 0 to don't buffer.
Complete Documentation
See https://juanjodiaz.github.io/json2csv/#/parsers/node-transform.
Node Async Parser
To facilitate usage, NodeAsyncParser wraps NodeTransform exposing a single parse method similar to the sync API. This method accepts JSON arrays/objects, TypedArrays, strings and readable streams as input and returns a stream that produces the CSV.
NodeAsyncParser also exposes a convenience promise method which turns the stream into a promise that resolves to the whole CSV.
Usage
import { AsyncParser } from '@json2csv/node';
const opts = {};
const transformOpts = {};
const asyncOpts = {};
const parser = new AsyncParser(opts, asyncOpts, transformOpts);
const csv = await parser.parse(data).promise();
// The parse method return the transform stream.
// So data can be passed to a writable stream (a file, http request, etc.)
parser.parse(data).pipe(writableStream);Parameters
Options
ndjson<Boolean> indicates that the data is in NDJSON format. Only effective when using the streaming API and not in object mode.fields<DataSelector[]> Defaults to toplevel JSON attributes.transforms<Transform[]> Array of transforms to apply to the data. A transform is a function that receives a data recod and returns a transformed record. Transforms are executed in order.formatters<Formatters> Object where the each key is a Javascript data type and its associated value is a formatters for the given type.defaultValue<Any> value to use when missing data. Defaults to<empty>if not specified. (Overridden byfields[].default)delimiter<String> delimiter of columns. Defaults to,if not specified.eol<String> overrides the default OS line ending (i.e.\non Unix and\r\non Windows).header<Boolean> determines whether or not CSV file will contain a title column. Defaults totrueif not specified.includeEmptyRows<Boolean> includes empty rows. Defaults tofalse.withBOM<Boolean> with BOM character. Defaults tofalse.
Transform Options
See the Duplex stream options for more details.
Async Options
Options used by the underlying parsing library to process the binary or text stream.
Not relevant when running in objectMode.
Buffering is only relevant if you expect very large strings/numbers in your JSON.
See @streamparser/json for more details about buffering.
stringBufferSize<number> Size of the buffer used to parse strings. Defaults to 0 which means to don't buffer. Min valid value is 4.numberBufferSize<number> Size of the buffer used to parse numbers. Defaults to 0 to don't buffer.
Complete Documentation
See https://juanjodiaz.github.io/json2csv/#/parsers/node-async-parser.
License
See LICENSE.md.