node-docker-registry-client

A Docker Registry API client for node.js.

Limitations: Currently only support for Registry API v1 (https://docs.docker.com/v1.6/reference/api/registry_api/) pull support (i.e. excluding API endpoints for push) and Registry API v2 pull support. Support for v2 push endpoints is coming.

Note: This repository is part of the Joyent Triton project. See the contribution guidelines and general documentation at the main Triton project page.

Install

npm install docker-registry-client

Overview

Most usage of this package involves creating a Registry API client for a specific repository and calling its methods.

A Registry client requires a repository name (called a repo in the code):

[INDEX/]NAME                # a "repo name"

Examples:

mongo                       # implies default index (docker.io) and namespace (library)
docker.io/mongo             # same thing
docker.io/library/mongo     # same thing

myreg.example.com:5000/busybox   # a "busybox" repo on a private registry

quay.io/trentm/foo          # trentm's "foo" repo on the quay.io service

The parseRepo function is used to parse these. See "examples/parseRepo.js" to see how they are parsed:

$ node examples/parseRepo.js mongo
{
    "index": {
        "name": "docker.io",
        "official": true
    },
    "official": true,
    "remoteName": "library/mongo",
    "localName": "mongo",
    "canonicalName": "docker.io/mongo"
}

Commonly, a "repo name and tag" string is used for working with a Docker registry, e.g. docker pull busybox:latest. The v2 API adds support for using "repo name and digest" to stably identify images, e.g. docker pull alpine@sha256:fb9f16730ac6316afa4d97caa5130219927bfcecf0b0ce35c01dcb612f449739. This package provides a parseRepoAndRef (and the synonym parseRepoAndTag) for that, e.g.:

$ node examples/parseRepoAndRef.js myreg.example.com:5000/busybox:foo
{
    "index": {
        "name": "myreg.example.com:5000",
        "official": false
    },
    "official": false,
    "remoteName": "busybox",
    "localName": "myreg.example.com:5000/busybox",
    "canonicalName": "myreg.example.com:5000/busybox",
    "tag": "foo"
}

Slightly different than docker.git's parsing, this package allows the scheme to be given on the index:

$ node examples/parseRepoAndRef.js https://quay.io/trentm/foo
{
    "index": {
        "scheme": "https",              // <--- scheme
        "name": "quay.io",
        "official": false
    },
    "official": false,
    "remoteName": "trentm/foo",
    "localName": "quay.io/trentm/foo",
    "canonicalName": "quay.io/trentm/foo",
    "tag": "latest"                     // <--- default to 'latest' tag
}

If a scheme isn't given, then "https" is assumed.

Usage

If you know, for example, that you are only dealing with a v2 Docker Registry, then simple usage will look like this:

var drc = require('docker-registry-client');
var REPO = 'alpine';
var client = drc.createClientV2({name: REPO});

client.listTags(function (err, tags) {
    // ...
    console.log(JSON.stringify(tags, null, 4));

    /*
     * Because the client is typically using Keep-Alive, it will maintain
     * open connections. Therefore you should call `.close()` to close
     * those when finished.
     */
    client.close();
});

A more complete example (showing logging, auth, etc.):

var bunyan = require('bunyan');
var drc = require('docker-registry-client');

// This package uses https://github.com/trentm/node-bunyan for logging.
var log = bunyan.createLogger({
    name: 'regplay',
    // TRACE-level logging will show you all request/response activity
    // with the registry. This isn't suggested for production usage.
    level: 'trace'
});

var REPO = 'alpine';
var client = drc.createClientV2({
    name: REPO,
    log: log,
    // Optional basic auth to the registry
    username: <username>,
    password: <password>,
    // Optional, for a registry without a signed TLS certificate.
    insecure: <true|false>,
    // ... see the source code for other options
});

This package also supports the nominal technique for pinging the registry to see if it supports v2, otherwise falling back to v1:

var drc = require('docker-registry-client');

var REPO = 'alpine';
drc.createClient({name: REPO, /* ... */}, function (err, client) {
    console.log('Got a Docker Registry API v%d client', client.version);
    // ...
});

v2 API

A mapping of the Docker Registry API v2 endpoints to the API equivalents in this client lib. "NYI" means the endpoint is not yet implemented by this client lib.

See "examples/v2/*.js" for short code examples one can run from the CLI for each API endpoint. E.g.:

$ npm install   # install dependencies for this module
$ node examples/v2/listTags.js busybox
{
    "name": "library/busybox",
    "tags": [
        "buildroot-2013.08.1",
        "buildroot-2014.02",
        "latest",
        "ubuntu-12.04",
        "ubuntu-14.04"
    ]
}

You can also get logging on processing and HTTP requests/responses via the -v option to the example scripts. This library uses Bunyan for logging, so you'll want to pipe the log output (on stderr) through the bunyan command for pretty output:

$ node examples/v2/listTags.js -v busybox 2>&1 | ./node_modules/.bin/bunyan
[2015-06-01T22:26:44.065Z] TRACE: v2.listTags/registry/23400 on grape.local: request sent
    GET /v2/ HTTP/1.1
    Host: registry-1.docker.io
    accept: application/json
    user-agent: node-docker-registry-client/2.0.0 (x64-darwin; node/0.10.28)
    date: Mon, 01 Jun 2015 22:26:44 GMT
[2015-06-01T22:26:44.480Z] TRACE: v2.listTags/registry/23400 on grape.local: Response received (client_res={})
    HTTP/1.1 401 Unauthorized
    content-type: application/json; charset=utf-8
    docker-distribution-api-version: registry/2.0
    www-authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io"
    date: Mon, 01 Jun 2015 22:26:44 GMT
    content-length: 114
    connection: close
    strict-transport-security: max-age=3153600
[2015-06-01T22:26:44.482Z] TRACE: v2.listTags/registry/23400 on grape.local:
    body received:
    {"errors":[{"code":"UNAUTHORIZED","message":"access to the requested resource is not authorized","detail":null}]}
[2015-06-01T22:26:44.485Z] TRACE: v2.listTags/registry/23400 on grape.local: login
[2015-06-01T22:26:44.487Z] DEBUG: v2.listTags/registry/23400 on grape.local: login: get Bearer auth token
...
[2015-06-01T22:26:45.680Z] TRACE: v2.listTags/registry/23400 on grape.local:
    body received:
    {"name":"library/busybox","tags":["buildroot-2013.08.1","buildroot-2014.02","latest","ubuntu-12.04","ubuntu-14.04"]}
{
    "name": "library/busybox",
    "tags": [
        "buildroot-2013.08.1",
        "buildroot-2014.02",
        "latest",
        "ubuntu-12.04",
        "ubuntu-14.04"
    ]
}

v1 API

A mapping of the Docker Registry API v1 endpoints to the API equivalents in this client lib.

Limitation: Only the read endpoints of the v1 API are implement. I.e. putting layers, deleting repositories, setting/deleting tags are all not implemented.

See "examples/v1/*.js" for short code examples one can run from the CLI for each API endpoint. E.g.:

$ npm install   # install dependencies for this module
$ node examples/v1/listRepoTags.js busybox
{
    "buildroot-2013.08.1": "3dba22db9896eb5f020691e1f8cda46735de533ca7b6b1b3b072272752935bad",
    "buildroot-2014.02": "8c2e06607696bd4afb3d03b687e361cc43cf8ec1a4a725bc96e39f05ba97dd55",
    "latest": "8c2e06607696bd4afb3d03b687e361cc43cf8ec1a4a725bc96e39f05ba97dd55",
    "ubuntu-12.04": "faf804f0e07b2936e84c9fe4ca7c60a6246cc669cf2ff70969f14a9eab6efb48",
    "ubuntu-14.04": "32e97f5f5d6bd15b9cc293b38a5102a7ddac736852811110043992b20553177a"
}

You can also get logging on processing and HTTP requests/responses via the -v option to the example scripts. This library uses Bunyan for logging, so you'll want to pipe the log output (on stderr) through the bunyan command for pretty output:

$ node listRepoTags.js -v busybox 2>&1 | bunyan
[2015-09-02T22:14:35.790Z] TRACE: listRepoTags/registry/13388 on danger0.local: get session token/cookie
[2015-09-02T22:14:35.836Z] TRACE: listRepoTags/registry/13388 on danger0.local: request sent
    GET /v1/repositories/library/busybox/images HTTP/1.1
    Host: index.docker.io
    X-Docker-Token: true
    accept: application/json
    user-agent: node-docker-registry-client/2.0.0 (x64-darwin; node/0.10.40)
    date: Wed, 02 Sep 2015 22:14:35 GMT
[2015-09-02T22:14:36.374Z] TRACE: listRepoTags/registry/13388 on danger0.local: Response received (client_res={})
    HTTP/1.1 200 OK
    server: nginx/1.6.2
    date: Wed, 02 Sep 2015 22:14:36 GMT
...
[2015-09-02T22:14:37.888Z] TRACE: listRepoTags/registry/13388 on danger0.local:
    body received:
    {"buildroot-2013.08.1": "3dba22db9...
[2015-09-02T22:14:37.888Z] TRACE: listRepoTags/registry/13388 on danger0.local: close http client (host=index.docker.io)
[2015-09-02T22:14:37.889Z] TRACE: listRepoTags/registry/13388 on danger0.local: close http client (host=registry-1.docker.io)
{
    "buildroot-2013.08.1": "3dba22db9896eb5f020691e1f8cda46735de533ca7b6b1b3b072272752935bad",
    "buildroot-2014.02": "8c2e06607696bd4afb3d03b687e361cc43cf8ec1a4a725bc96e39f05ba97dd55",
    "latest": "8c2e06607696bd4afb3d03b687e361cc43cf8ec1a4a725bc96e39f05ba97dd55",
    "ubuntu-12.04": "faf804f0e07b2936e84c9fe4ca7c60a6246cc669cf2ff70969f14a9eab6efb48",
    "ubuntu-14.04": "32e97f5f5d6bd15b9cc293b38a5102a7ddac736852811110043992b20553177a"
}

V1 client usage:

var repo = 'alpine';
var client = drc.createClientV1({
    name: repo,
    // ...
});
client.listRepoTags(function (err, repoTags) {
    client.close();
    if (err) {
        console.log(err);
        process.exit(1);
    }
    console.log(JSON.stringify(repoTags, null, 4));
});

Development

Naming convensions

For naming this package attempts to consistently use repo for repository, img for image, etc.

Commits

Before commit, ensure that the following checks are clean:

make prepush

Releases

Changes with possible user impact should:

1. Add a note to the changelog (CHANGES.md).
2. Bump the package version appropriately.

3. Once merged to master, the new version should be tagged and published to npm via:

    make cutarelease

   To list to npm accounts that have publish access:

    npm owner ls docker-registry-client

The desire is that users of this package use published versions in their package.json dependencies, rather than depending on git shas.

node-docker-registry-client Changelog

not yet released

(nothing yet)

3.4.0

• TRITON-2403 Add support for docker OCI manifest format

As of this version, v1 repositories are still supported, but most tests have been disabled due to the lack of available public repositories.

3.3.0

• DOCKER-524 Implement docker push. Adds uploadBlob and putManifest API methods.

3.2.13

• DOCKER-1050 add redirect handling for docker registry client getManifest call

3.2.12

• joyent/node-docker-registry-client#23 Namespace validation too strict

3.2.11

• DOCKER-1095 docker pull should distinguish between auth error and not found error

3.2.10

• DOCKER-1104 support docker manifest lists

3.2.9

• DOCKER-1097 docker pull fails for registry server that has no auth setup

3.2.8

• DOCKER-1091 Cannot login into Artifactory private docker registry. This change drops authorization headers from the first ping request.

3.2.7

• joyent/node-docker-registry-client#16 Allow a repo name to have a '/', e.g. registry.gitlab.com/masakura/docker-registry-client-bug-sample/image (by Józsa Péter).

3.2.6

• DOCKER-984 Add a 10s connect timeout for endpoints involved in initial contact with a given Docker registry. This allows faster failing for unreachable hosts.

3.2.5

• DOCKER-983 Change v1 registry session setup to not do retries. This allows failing fast when, for example, the registry endpoint URL is bogus.

3.2.4

• DOCKER-959 Unable to pull from registry when registry response sends multiple Docker-Distribution-Api-Version headers.

3.2.3

• IMGAPI-596 Move from restify-clients fork to restify-clients@1.4.0 with upstreamed HTTP proxy support.

3.2.2

• DOCKER-950 Docker pull fails on registry that does not use authentication

3.2.1

• DOCKER-663 Support Amazon ECR Registry

3.2.0

• DOCKER-940: Update to support node v4 without build errors.
• DOCKER-825: Support Google Registry
• DOCKER-929: Support the docker v2.2 manifest format. The createClientV2 and getManifest functions now allow an optional maxSchemaVersion argument to allow you to specify the maximum manifest schema version you will receive for getManifest calls. Default is 1 for a schemaVersion 1 format (aka v2.1), a value of 2 means schemaVersion 2 (aka v2.2).

• 8 Fix usage against a http registry.

• 7 Fix test suite against Docker Hub.

3.1.5

• DOCKER-772: Change pingV2 to callback(err) rather than throw err if the given opts.indexName is invalid.

3.1.4

• DOCKER-764: As of Docker 1.11, docker login will no longer prompt for an email (docker/docker#20565). This updates drc.login() to no blow up attempting to do v1 login if an email is not provided.

3.1.3

• DOCKER-689 Don't overwrite err.body for an HTTP response error that doesn't have a JSON body.

3.1.2

• IMGAPI-546: 'docker pull nope.example.com/nope' throws exception in IMGAPI

3.1.1

• IMGAPI-542: Don't require the Docker-Content-Digest header on v2 GetBlob server responses: RegistryClientV2.prototype.createBlobReadStream. Also don't throw if can't parse an existing Docker-Content-Digest header.

3.1.0

• [pull #6, plus some additions] Allow one to ping a v2 registry using bearer token auth. Typically one shouldn't need to login for the ping endpoint, because the point is to check for 404 (doesn't support v2) or not (supports v2). However, there is no good reason one shouldn't be able to ping it with auth.

  This adds RegistryClientV2.prototype.login() and the top-level pingV2() accepts a opts.authInfo as returned by loginV2().

  Sample code doing this:

    var drc = require('docker-registry-client');
    var bunyan = require('bunyan');
  
    var log = bunyan.createLogger({name: 'pingplay', level: 'trace'});
    var client = drc.createClientV2({
        name: 'docker.io',
        log: log,
        username: 'MY-USERNAME',
        password: 'MY-PASSWORD'
    });
  
    client.ping(function(err, body, res) {
        if (!err) {
            console.log('Success on first try');
        } else if (err.statusCode === 401) {
            client.login({
                pingErr: err,
                pingRes: res
            }, function(err, result) {
                client.ping(function(err, body, res) {
                    if (!err) {
                        console.log('Success');
                    } else {
                        console.log('Fail:', err);
                    }
                });
            });
        } else {
            console.log('Huh?', err);
        }
    });

3.0.5

• DOCKER-643: RegistryClientV2 doesn't pass insecure flag to ping() function properly.

3.0.4

• DOCKER-640: Fix assertion hit when attempting getManifest (or other v2 API endpoints) against a v2 Docker Registry that uses Basic auth but when not passing auth info.

  Triton's IMGAPI currently does the latter as part of a docker pull to determine if the repo is private. See https://github.com/joyent/sdc-imgapi/blob/f5384db7ddc4e1ef9a54f6e6fbe15d11093e3155/lib/images.js#L3378-L3448.

• Fix my config error that made me think the v2.quayioprivate tests were failing. Re-enable that test in make test.

3.0.3

• DOCKER-627: Fix login for v2 quay.io. Two issues here:
  1. quay.io has a bug where www-authenticate header isn't always in 401 response headers
  2. passing "scope=" (empty value) query param to quay.io/v2/auth results in a 400

3.0.2

• DOCKER-625: Fix v1 pull of 'rhel7' from Docker Hub, which redirects to the registry.access.redhat.com repository. Warning: This only works if you know apriori to use v1... which typically you would NOT know because Docker Hub supports v2. IOW, it is up to the caller if this lib to do the appropriate v2 -> v1 fallback logic.

• DOCKER-625: Fix v1 API against registry.access.redhat.com. E.g. for docker pull registry.access.redhat.com/rhel7. This implementation of the Docker Registry v1 involves 3xx redirs for more endpoints. Needed to support that.

• DOCKER-622: Tests for quay.io v2: working without auth, fails with auth. make test is currently wired to skip the known failing v2 quay.io private tests. Will revisit this later. See "test/v2.quayio.test.js" for some notes on what I think are quay.io v2 bugs that I'll report.

3.0.1

• DOCKER-586 Allow opts.proxy to be a boolean.

3.0.0

• DOCKER-586 Add drc.pingV2, drc.loginV2 and drc.login. The latter knows how to login to a Docker Registry API via v1 or v2 (discovering which properly).

• (Backward incompatible.) The callback from pingV1 has this form:

    function (err, result)

  That result has changed from:

    {"Status": "(string status)"}    # old

  to:

    {"status": "(string status)"}    # new (lowercase 'status')

  because (a) let's not depart from the typical starts-with-lowercase naming in this module just to follow the form of the Docker Remote API "GET /auth" response, and (b) this will mean being compatible with pingV2.

2.0.2

• DOCKER-583: Docker pull from v1 private registry failed with Resource Not Found error

2.0.1

• DOCKER-580 Fix "too few args to sprintf" error when failing token auth.
• Fix passing through of username (for token 'account' field) and basic authorization for token auth.

2.0.0

• A start at Docker Registry API v2 support. For now just pull support (the subset of the API needed for docker pull). Support for the rest of the API is coming. See the updated README and "examples/v2/".

• Fix reliability issues with v2 download of large layer files. My understanding is that this was due to https://github.com/nodejs/node/issues/3055. This module is now avoiding agent: false, at least for the blob downloads.

1.4.1

• DOCKER-549 Fix pulling from quay.io private repos.

1.4.0

• Test suites improvements to actually check against docker.io, quay.io, and optionally a jfrog artifactory Docker v1 repo.
• DOCKER-380: Support for quay.io. Quay.io uses cookies for repository sessions. This involved refactoring the standalone/token handling to be more generically for "sessions". It simplified the code quite a lot.
• DOCKER-540 Update examples to take a '-k,--insecure' option to allow access to registries with self-signed certs.
• DOCKER-539 Allow passing through a proxy for drc.login(...).
• v1 Ping should send auth headers, if any. A private registry might very well require auth on the ping endpoint.
• v1 Search doesn't need to do a ping. Presumably this is historical for mistakenly thinking there was a need to determine this.standalone.

1.3.0

• [DOCKER-526] HTTP proxy support. This should work either (a) with the http_proxy/https_proxy environment variables, or (b) in code like this:

    var url = require('url');
    var drc = require('docker-registry-client');
    var client = drc.createClient({
        name: 'busybox',
        proxy: url.parse('http://my-proxy.example.com:8080'),
        //...
    });

  One side-effect of this change is the underlying HTTP connections no longer set agent: false to avoid Node's default HTTP agent. This means that if you are not using a proxy, you will now get keep-alive, which means a persistent connection that could hold your node script/app from exiting. All users of this client should explicitly close the client when complete:

    client.close();

  The "example/*.js" scripts all do this now.

1.2.2

• [pull #4] Fix getImgAncestry when using node 0.12 (by github.com/vincentwoo).

1.2.1

• Sanitize the non-json (text/html) err.message from listRepoImgs on a 404. See before and after: https://gist.github.com/trentm/94c11e1243fb7fd4fe90

1.2.0

• Add drc.login(...) for handling a Docker Engine would use for the Remote API side of docker login.

1.1.0

• RegistryClient.ping will not retry so that a ping failure check is quick. Without this it was retrying for ~15s.
• RegistryClient.search now does a ping check before searching (fail fast).
• createClient({userAgent: ...}) option. Defaults to 'node-docker-registry-client/$ver (...)'.
• A client to localhost will default to the 'http' scheme to (mostly) match docker-docker's behaviour here.

1.0.0

A major re-write with lots of backwards compat breakage. This release adds support for indeces/registries other than the "official" docker.io default. It changes usage to feel a bit more like docker/docker.git's "registry" package. So far this still only supports Registry API v1.

Basically the whole usage has changed. There is no longer a "Registry Session", session handling is done lazily under the hood. There is no longer a separate "Index API Client", the only client object is the "RegistryClient" setup via:

var client = drc.createClient({...});

There is a drc.pingIndex() which can be used to check that a registry host (aka an "index" from the old separation of an "Index API") is up. Usage is best learned from the complete set of examples in "examples/".

• Backward incompat change in return value from parseRepoAndTag. In addition to adding support for the optional index prefix, and a "@DIGEST" suffix, the fields on the return value have been changed to more closely match Docker's RepositoryInfo fields:

    {
        index: {
            name: INDEX_NAME
            official: BOOL
        },
        remoteName: ...,
        localName: ...,
        canonicalName: ...,
        official: BOOL
    }

  E.g.

    {
        index: {
            name: 'docker.io'
            official: true
        },
        remoteName: 'library/mongo',
        localName: 'mongo',
        canonicalName: 'docker.io/mongo',
        official: true
    }

  See https://github.com/docker/docker/blob/2e4d36ed80855375231501983f19616ba0238a84/registry/types.go#L71-L96 for an example.

  Before:

    {
        ns: NS,
        name: NAME,
        repo: NS/NAME,
        tag: TAG || 'latest'
    }

  e.g.:

    {
        ns: 'library',
        name: 'mongo',
        repo: 'library/mongo',
        tag: '1.2.3'
    }

0.3.2

Note: Any 0.x work (I don't anticipate any) will be on the "0.x" branch.

• Update deps to move fwd to 0.12-supporting versions of things.

0.3.1

• Switch to '^x.y.x' for deps to allow for node_modules dedupe in apps using this module.

0.3.0

• Add RegistrySession.getImgId(), parseRepoAndTag(), RegistrySession.getImgLayerStream().
• Export parseRepoAndTag() function.
• Add repoImgs to object returned by IndexClient.getRepoAuth. For images with "checksum" values this could possibly be useful for validation of subsequent downloads.
• URL encode params in API call paths.

0.2.0

Started changelog after this version.