Important: This documentation covers Yarn 1 (Classic).
For Yarn 2+ docs and migration guide, see yarnpkg.com.

Package detail

dyno

mapbox911MITdeprecated1.4.1

This module has been moved. Please install @mapbox/dyno instead

Simple DynamoDB client

dynamodb

readme

Dyno

Build Status

Dyno provides a DynamoDB client that adds additional functionality beyond what is provided by the aws-sdk-js.

Overview

Native JavaScript objects

Dyno operates as an extension to the aws-sdk's DocumentClient. This means that instead of interacting with typed JavaScript objects representing your DynamoDB records, you can use more "native" objects. For example, the following represents a typed object that could be stored in DynamoDB:

{
  id: { S: 'my-record' },
  numbers: { L: [{ N: '1' }, { N: '2' }, { N: '3' }] },
  data: { B: new Buffer('Hello World!') },
  version: { N: '5' },
}

Using Dyno, you can represent the same data in a "native" object:

{
  id: 'my-record',
  numbers: [1, 2, 3],
  data: new Buffer('Hello World!'),
  version: 5
}

Streaming query and scan results

A large query or scan operation may require multiple HTTP requests in order to retrieve all the desired data. Dyno provides functions that allow you to read those data from a native Node.js Readable Stream. Behind-the-scenes, Dyno manages making paginated requests to DynamoDB for you, and emits objects representing each of the records in the aggregated response.

Chunked batch getItem and writeItem requests

BatchGetItem and BatchWriteItem requests come with limits as to how much data you can ask for in a single HTTP request. Dyno functions allow you to present the entire set of batch requests that you wish to make. Dyno breaks your set up into an array of request objects each of which is within the limits of a single acceptable request. You can then send each of these requests and handle each of their responses individually.

Multi-table client

For situations where you may wish to write to one database and read from another. Dyno allows you to configure a client with parameters for two different tables, then routes your individual requests to the appropriate one.

De/serialization

Dyno exposes functions capable of serializing and deserializing native JavaScript objects representing DynamoDB records to and from wire-formatted strings acceptable as the body of any DynamoDB HTTP request.

Command-line interface

Dyno provides a CLI tool capable of reading and writing individual records, scanning, exporting and importing data into a DynamoDB table.

Documentation

See API.md for documentation of the JavaScript client, and use dyno --help for details on CLI usage.

changelog

v1.4.0

  • updates aws-sdk dependency to 2.x to help reduce the weight of the aws-sdk in your bundle

v1.3.0

  • updates aws-sdk dependency to v2.7+ from v2.1+

v1.2.1

  • fixes a bug where certain errors were not handled correctly by .batchWriteAll()

v1.2.0

  • adds .batchGetAll() and .batchWriteAll() methods
  • adds autopagination to .query() and .scan() methods via a Pages parameter.
  • fixes a bug in the cli which prevented export --> import roundtrips

v1.1.0

  • adds dyno.putStream(), a writable stream to batch individual records into BatchWriteItem requests
  • bug fixes in cli

v1.0.1

  • fixes a bug in converting JavaScript arrays to DynamoDB Sets if the first item in the array is falsy (e.g 0)

v1.0.0

The overarching goal is to prefer to utilize the aws-sdk commands where possible, inheriting their bug fixes, updates, and documentation. Some aspects of dyno's API are really nice (e.g. .getItem(key) instead of .getItem({ Key: key })), but having documentation that we don't have to maintain is arguably better.

generally speaking

... the big changes are:

  • "dyno-style" function arguments, conditions, and options go away. Instead all function arguments are as described for the aws-sdk's document client.
  • dyno navigates the differences between the aws-sdk's regular client and document client for you behind the scenes.
  • with the exception of explicit stream functions, none of the function calls fan out into multiple HTTP requests. It doesn't handle pagination, it doesn't split large batch requests into a series of acceptably-sized requests, and it doesn't attempt to retry errors. Configuring when and how errors are retried becomes the client's responsibility using aws-sdk's mechanisms.

breaking changes

  • client configuration requires you to provide a table name
  • dyno.getItems renamed to dyno.batchGetItem, as a passthrough to aws-sdk.
    • no longer accepts an array of keys and optional options object. See aws-sdk docs for parameters
    • does not allow you to request more than 100 items per function call
    • does not provide a readable stream of response records
  • dyno.deleteItems and dyno.putItems replaced by dyno.batchWriteItem, a passthrough to aws-sdk.
    • no longer accepts an array of items to put / keys to delete and optional options object. See aws-sdk docs for parameters
    • does not allow you to write > 25 items or 16 MB per function call
  • dyno.deleteItem becomes a passthrough to aws-sdk. No longer accepts a key and optional options object. See aws-sdk docs for parameters
  • dyno.getItem becomes a passthrough to aws-sdk. No longer accepts a key and optional options object. See aws-sdk docs for parameters
  • dyno.putItem becomes a passthrough to aws-sdk. No longer accepts a record and optional options object. See aws-sdk docs for parameters
  • dyno.updateItem becomes a passthrough to aws-sdk. No longer accepts a key, "dyno-style" update definition and optional options object. See aws-sdk docs for parameters
  • dyno.query becomes a passthrough to aws-sdk.
    • no longer accepts the "dyno-style" query conditions and optional options object. See aws-sdk docs for parameters
    • does not paginate through the reponse for you. Pagination can be accomplished manually or via aws-sdk's functions.
    • does not provide a readable stream of response records
  • dyno.scan becomes a passthrough to aws-sdk.
  • drops dyno.estimateSize
  • drops all kinesis-related functionality

new functions

  • dyno.queryStream provides a readable stream of query responses. Parameters passed to the function are identical to dyno.query, but the function returns a readable stream that will paginate through results as you read from it.
  • dyno.scanStream provides a readable stream of scan responses. Parameters passed to the function are identical to dyno.scan, but the function returns a readable stream that will paginate through results as you read from it.
  • dyno.batchWriteItemRequests provides an array of AWS.Request objects representing BatchWriteItem requests. Parameters passed to the function are identical to dyno.batchWriteItem, except that there is no size/count limit to the number or write requests you may provide.
  • dyno.batchGetItemRequests provides an array of AWS.Request objects representing BatchGetItem requests. Parameters passed to the function are identical to dyno.batchGetItem, except that there is no size/count limit to the number or get requests you may provide.

v0.17.0

  • Return native format (not wire format) for metas[*].last (LastEvaluatedKey).
  • Callers must use native format (not wire format) when passing in opts.start (ExclusiveStartKey).

v0.16.0

  • Fixed a bug that did not allow the CLI tool to function using EC2 IAM Role-based credentials
  • Adds a function to estimate the number of bytes required to store an item
  • Allows caller to set concurrency on batch requests

v0.15.1

  • Batch write requests now return any keys that went unprocessed

v0.15.0

  • Add support for document types (List and Map), Boolean, and Null
  • Native JavaScript arrays will now transform into lists instead of sets
  • Add Dyno.createSet(), which constructs sets (of number, string, or binary type) that transform to the DynamoDB wire format correctly.
  • Drop support for using a wire-formatted object as input.
  • Add Dyno.serialize() and Dyno.deserialize() to convert items to/from strings