Important: This documentation covers Yarn 1 (Classic).
For Yarn 2+ docs and migration guide, see yarnpkg.com.

Package detail

symbol-statistics-service-typescript-fetch-client

symbol1051.1.5TypeScript support: included

readme

Symbol Statistics Service

It periodically collects information from the network's nodes and produces useful metrics and insights, such as the total number of nodes, their health or their geographical distribution.

Architecture

The project uses Typescript, Node Express, and MongoDB as database.

                       +---------------+
                       |  Application  |
                       +----+----^-----+
                            |    |
                            |    |
                            |    |  Request / Response
                            |    |
 Symbol Statistics Service  |    |
+---------------------------v----+--+
|                                   |
| +-Express Server--------+         |
| |                       |         |
| |                       |         |
| |     Task Manager      |         |
| |                       |         |
| |                       |         |
| +-----------+-----------+         |
|             |                     |
| +-Services--v-----------------+   |
| |                             |   |
| | - Node Monitor              |   |
| |                             |   |   +------------------+
| | - Chain Height Monitor      +---+--->                  |
| |                             |   |   | Mongodb Database |
| | - Geolocation Monitor       <---+---+                  |
| |                             |   |   +------------------+
| +-----------------------------+   |
|                                   |
+-----------------------------------+

Node Monitor Task

To discover nodes on the network, a request is made to the catapult-rest endpoint /node/peer. This returns the list of peers known to that node, which the task then explores recursively.

To start the process, the task uses the nodes maintained by the community.

Once the whole list of nodes is known, additional information is gathered from each one of them such as /node/info and /chain/info.

When the search completes, all node information is updated on the Node collection.

Chain Height Monitor Task

The finalized height from each node is gathered and height statistics are produced. This can help monitoring chain forks, for example.

Geolocation Monitor Task

A 3rd party service (ip-api.com) is used to obtain geolocalization for each node. This can be used by users to select a nearby access node, for example.

Source Code Structure

  • /src/config: Statistics service configuration.
  • /src/routes: REST endpoint routes.
  • /src/models: Database collection and mongoose schema.
  • /src/service: Service logic, such as requesting geolocation.
  • /src/infrastructure: Logger and pagination component.

Reference Documentation

Configuration

The default config file can be found at src/config/config.json.

If you want to overwrite the configuration you need to create an .env file in the root path.

Installation

  1. Clone the project.

    git clone https://github.com/symbol/symbol-statistics-service.git
  2. Install the required dependencies.

    cd symbol-statistics-service
    npm install
  3. Run the application in development mode.

    npm run dev

Docker Setup

  1. Build Docker image.

    docker build -t symbol-statistics-service .
  2. Run the Docker image with the following Docker compose file.

    version: "3"
    services:
        app:
            container_name: statistics-service
            restart: always
            build: .
            environment:
            - MONGODB_ENDPOINT=mongodb+srv://mongodbURI
            - NODES=["http://symbol-node-1.io:3000",http://symbol-node-2.io:3000"]
            - PORT=4001
            ports:
            - "3000:4001"

changelog

CHANGELOG

All notable changes to this project will be documented in this file.

The changelog format is based on Keep a Changelog.

v1.2.0 - 9-Apr-2025

Package Version Link
REST Core v2.5.0 catapult-rest
SDK Core v3.2.3 symbol-sdk

Fix

  • Geo info unable to insert cause of district and zip value possibly empty string #302
  • Remove unavailable node from list #313
  • Update node version to latest #332
  • Prevent duplicate data insertion into the database. #336 #342

Upgrade

  • Dependencies and supported node version to v20 #289
  • Docker build process #317

v1.1.9 - 25-Oct-2022

Package Version Link
REST Core v2.4.2 [catapult-rest][catapult-rest@v2.4.2]
SDK Core v2.0.3 [symbol-sdk][symbol-sdk@v2.0.3]
  • Updated symbol-sdk version to 2.0.3

v1.1.8 - 07-Jun-2022

Package Version Link
REST Core v2.4.0 catapult-rest
SDK Core v2.0.0 symbol-sdk
  • Fixed WebSocket connection leak issue #158

v1.1.7 - 12-May-2022

Package Version Link
REST Core v2.4.0 catapult-rest
SDK Core v2.0.0 symbol-sdk
  • Fixed node monitoring scheduled job hanging after a while issue #145

v1.1.6 - 23-Mar-2022

Package Version Link
REST Core v2.4.0 catapult-rest
SDK Core v2.0.0 symbol-sdk
  • Migrate the CI from Travis to Jenkins #139
  • Upgrade the Symbol SDK to v2.0 #140

v1.1.5 - 24-Jan-2022

Package Version Link
REST Core v2.4.0 catapult-rest
SDK Core v1.0.3 symbol-sdk

Fix

  • Fixed duplicate nodes and nodes from the wrong networks removal issue #134

v1.1.4 - 18-Jan-2022

Package Version Link
REST Core v2.4.0 catapult-rest
SDK Core v1.0.3 symbol-sdk

Fix

  • tsconfig.build.json added to exclude test files from build but keep for tests #128
  • Config.KEEP_STALE_NODES_FOR_HOURS is added to clean up stale nodes from database #127
  • Preferred nodes filter fixed, Config.PREFERRED_NODES is now empty by default (will be set in the testnet/mainnet environment) #126
  • Node discovery extended, empty host info bug fixed, logging enhanced #121

v1.1.3 - 30-Nov-2021

Package Version Link
REST Core v2.4.0 catapult-rest
SDK Core v1.0.3 symbol-sdk

Add

  • Add endpoint that will allow get node by nodePublicKey #95
  • Add web socket filter option in nodes. #106
  • Add typescript-fetch generated client #107
  • Add node version stats #110
  • Add node list ordering[random, natural] #111
  • Add symbol-sdk 1.0.3

Fix

  • Fix node mix from different network #113

Remove

  • Remove default version filter #109

v1.1.2 - 5-Nov-2021

Package Version Link
REST Core v2.3.7 [catapult-rest][catapult-rest@v2.3.7]
SDK Core v1.0.1 symbol-sdk
  • Fix node list not being fully discovered #91

v1.1.1 - 31-Oct-2021

Milestone: [catapult-server@v1.0.2.0][catapult-server@v1.0.2.0]

Package Version Link
REST Core v2.3.7 [catapult-rest][catapult-rest@v2.3.7]
SDK Core v1.0.1 symbol-sdk
  • Fix npm vulnerabilities #87
  • Fix chain height stat #88

v1.1.0 - 30-Oct-2021

Milestone: [catapult-server@v1.0.2.0][catapult-server@v1.0.2.0]

Package Version Link
REST Core v2.3.7 [catapult-rest][catapult-rest@v2.3.7]
SDK Core v1.0.1 symbol-sdk
  • support https/http nodes. #64
  • Removed node rewards related source code. #65
  • Added new filter on nodes endpoint #71

v1.0.1 - 25-May-2021

Milestone: catapult-server@v1.0.0.0

Package Version Link
REST Core v2.3.6 catapult-rest
SDK Core v1.0.1 symbol-sdk
  • Added the network type check to the Node Monitor Service #53
  • Added version of REST to the Node List #55
  • Fixed the peer node status check issue #58
  • Improved DB collection store #58
  • Included the initial nodes from config to the list #58
  • Added missing utf8 dependency

v1.0.0 - 14-Mar-2021

Milestone: catapult-server@v1.0.0.0

Package Version Link
REST Core v2.3.4 catapult-rest
SDK Core v1.0.0 symbol-sdk
  • Release for Symbol mainnet.

v0.3.0 - 12-Mar-2021

Milestone: catapult-server@v0.10.0.8

Package Version Link
REST Core v2.3.3 catapult-rest
SDK Core v0.23.3 symbol-sdk

Added

  • TimeSeriesService
  • Node cout over time statistic

Fixed

  • Use mainPublicKey to fetch Node Rewards info

v0.2.0 - 24-Feb-2021

Milestone: [catapult-server@v0.10.0.7][catapult-server@v0.10.0.7]

Package Version Link
REST Core v2.3.0 catapult-rest
SDK Core v0.23.0 symbol-sdk

Added

  • Node Rewards Service
  • Node Chain Height Monitor Service
  • Geolocation Monitor Service
  • Nodes Stats service
  • Api Node Service
  • Improved speed of getting a node list

Fixed

  • Code refactor

v0.1.0 - 26-Oct-2020

Milestone: catapult-server@v0.10.0.3

Package Version Link
REST Core v2.1.0 catapult-rest

Added

  • Support docker images build. #2
  • Support mongodb services. #9 #3
  • Support error handling. #15
  • Support route to get specific node infomation. #14
  • Support tcp port checking on node. #7
  • Support checking node location module. #5
  • Support node monitor module. #1 #8 #6
  • Support logger. #4

Fixes

  • Cors error. #13