Action file generator

Pass inputs (data) and set of actions
Returns set of virtual file system (VFS) operations

Each time we run the docker-gen, we need to compare the new commit with previous commit, to determine the "diff" that we should commit to our Version Control System (VCS), most likely Git. We can make the commit via a REST API for the VCS.

Dependencies

simple-vfs - Virtual file system
bazed - logging, validation, notifications etc.

Sample configurations

The project contains a samples folder with sample configurations using file-gen.

The tsconfig.json is configured to only consider and compile files in the following root folders:

    "rootDirs": [
      "src",
      "tests"
    ]

Usage

The File gen currently supports generating an output of virtual files by passing a set of nodes (JS objects) through a generator that can either:

repeat the same template for all nodes and generate a virtual file tree for each node
run one template for all nodes and output a single virtual file tree

The repeat/many (iterate and take each node as item) template generator is useful for generating a project folder for each node, such as a micro service lerna package

The single template (take all nodes as single list) template generator is useful for generating index files, such as a Docker file or f.ex a package.json file for all nodes.

TODO

We need to split out the generic generation from the micro service specific

Future considerations

We need to be able to nest generators at multiple levels. Generators should be able to generate whatever JSON (object) output that is needed for down-stream generators to work on. Generators should be able to run asynchronously.

A set of generators should be hostable as AWS lambda functions or via a REST (or GraphQL API?). The users can then compose their generators from micro services.

The file generator should expose a set of utilities that can be used by said micros services to build on.

A micro service can contain conditional or any other logic to determine how to compose the down-stream generation.

Micro services can also do side-effects such as committing to one or more remote repo providers and then return a JSON result indicating the success of these commits.

Generating a Micro Service project

Please see tests/project/micro-service/micro-service.test.js which demonstrates how to generate a micro services project given nodes data (as exported from node-red)

Generator

File-gen can generate files that make up a file tree (project)

import {
  generate,
  generateMicroServiceProject
 } from '@tecla5/file-gen'

generateMicroServiceProject

The generateMicroServiceProject function can be used to directly generate a Micro Service project:

let results = await generateMicroServiceProject({
  inputs,
  actions
}, opts)

Please see the ProjectGenerator and MicroServiceProjectGenerator classes for more details. The key is the configActions method of MicroServiceProjectGenerator which sets up the default actions for this particular project generator.

Runner

The main run function

async run(inputs) {
  return await this.generateForAll(inputs || this.inputs)
}

generateForAll iterates each of the entries in inputs, and calls the pre-configured RunGenerator with either:

singleActions - the input object is passed to each action
listActions - each object in the input list is passed to each of the actions

Each entry in inputs is matched of with an entry in actions:

inputs: {
  service: {
    nodes
  },
  serviceIndex: nodes
}

generator: {
  actions: {
    service: [{
      type: 'add',
      //...
    }, {
      //...
    }],
    serviceIndex: [{
      //...
    }]
  }
}

To make this more flexible, we also allow the following matching scheme:

inputs: {
  service: {
    type: 'item',
    data: nodes,
    action: 'service'
  },
  serviceIndex: {
    type: 'list',
    data: nodes,
    actions: ['serviceIndex', 'tests']
  }
}

Actions

The file generator uses a set of actions to be executed with the incoming data. Currently only the built in add (file) action is used:

The destination and template can both act as mustache templates. {{rootPath}} is the rootPath option (app by default).

{
    type: "add",
    destination: "{{root}}/services/{{dashCase name}}/.gitignore",
    template: "plop-templates/gitignore.tpl"
  },

The {{dashCase name}} is the name of the node, "dashed", ie where spaces become -. So for a node named addFifty it would generate a file at:

app/services/add-fifty/.gitignore with the add file action.

Micro Service project generation

The pre-configured Micro Service project generator, uses inputs (data) with the keys list and item of this form:

let data = {
  list: nodes,
  item: {
    dockerServices: nodes
  }
}

The dockerServices data entry is used by the template docker-compose.yml.tpl to generate each service entry in the generated docker-compose.yml file.

{{#each dockerServices }}

Nodes as single item

Used to generate files that each contain information on multiple nodes, such as index files. The item data entry is linked to the actions/file/item.js

module.exports = [{
  type: "add",
  destination: "{{rootPath}}/docker-compose.yml",
  template: "plop-templates/docker-compose.yml.tpl"
}]

This means that all the nodes are sent to each template as a single item, such as:

{
  dockerServices: [
    // nodes here
  ]
}

Nodes as list of items

Used to generate multiple files (or a file tree) for each node, like a sub-project. The list data entry is linked to the actions/file/list.js

module.exports = [{
    type: "add",
    destination: "{{root}}/services/{{dashCase name}}/.babelrc",
    template: "plop-templates/babelrc.tpl"
  }, {
    type: "add",
    destination: "{{root}}/services/{{dashCase name}}/.dockerignore",
    template: "plop-templates/dockerignore.tpl"
  }, {

Here a set of new file changes (one per action) is generated for each entry in the list (of nodes).

Results

The result is of the form:

{
  changes: [
    // file change
    {
        type: 'add'
        filePath: 'app/service/Readme.md'
        data: 'hello world'
    }
  , {
    // file change
  }],
  failures: [{
    // failure desc
  }]
}

The main generate function, will take the results and reduce to the following more convenient representation, where each changes key is a path to the file affected by the action.

{
  changes: {
    'app/service/Readme.md': {
      type: 'add'
      filePath: 'app/service/Redme.md',
      data: 'hello world'
    }
  },
  failures: {
    // failure desc
  }
}

Changes

A change result for add action will be:

{
  filePath: 'my/path/to/file.txt',
  data: 'hello'
}

This is the lastWritten object returned by the writeFile method of VirtualFileSystem (see @tecla5/vfs module)

  async writeFile(filePath, data) {
    let lastWritten = {
      filePath,
      data
    }
    this.config.files[filePath] = lastWritten
    this.config.lastWritten = lastWritten
  }

As can be seen, each config.files entry has the form:

{
    type: 'add'
    filePath: 'app/service/Redme.md'
    data: 'hello world'
}

Failures

Failures are of the form:

{
  error: true,
  type: 'template',
  msg: `Template render error: my/awesome/template-file.tpl`,
  cause: {
    // ... the actual thrown Error instance
  }
}

Package detail

action-file-gen

readme