Arranger

Generate and manage your own genomic data portal.

Note: This branch is being preserved for v2.X support only.

All new implementation is being done in v3+ at the `main` and `develop` branches, and such features/fixes may not be backported herein.

Develop (Edge):

Master (Release):

Documentation

This file is meant as a quick introduction, but for more in-detail documentation, you should explore Arranger's "Read the Docs". If interested, see our Open Source License

Getting Started
- Development Setup
- Dockerized Setup (a.k.a. Quickstart)
Motivation
What is a "Data Portal"?
Roadmap
Development Details

Getting Started

While our team is actively working on a new Arranger (see the develop branch), the version in this branch is considered "legacy"; being maintained only to support existing projects, it has been tested using NodeJS v14. It may not work well with other versions.

- Development Setup

Setting up the project, and prepare things to make changes

# 1. clone the repository
  git clone git@github.com:overture-stack/arranger.git

# 2. enter the project's folder
  cd arranger

# 3. install the dependencies
  npm ci

# 4. install the module's own dependencies
  npm run bootstrap

Now you should be able to start the following processes from the project's root folder:

# watch all modules and rebuild them when you make changes
  npm run watch

# test all modules at once
  npm run test

# run the server (on port 5050)
  npm run server

# serve the component dashboard (on port 6060)
  npm run dashboard

# serve the component portal (on port 7070)
  npm run portal

# run storybook (on port 8080)
  npm run storybook

- Dockerized Setup

A bit more friendly "quickstart", if you just want to get things started

# Start all services at once, using some default settings.
# This runs the following services: Elasticsearch, kibana, arranger-server, and arranger-ui
  make start

# ^^^ which runs the following command behind the scenes:
# ES_USERNAME=elastic ES_PASSWORD=myelasticpassword docker-compose -f docker-compose.yml up -d -build
# Note: these ES_* values may be customized when running your own Arranger instance


---
# Afterwards, in another bash process, you may seed an example file_centric index
  make init-es

# ^^^ which runs the following command behind the scenes:
# ./docker/elasticsearch/load-es-data.sh ./docker/elasticsearch elastic myelasticpassword
# That SH script may give you ideas on how to automate uploading indexes to your instance.


---
# Bonus: ----------------------------- #
# See other preprogrammed make targets
  make help
# e.g. utilities to list the indexes, or clear the Elasticsearch; list the running docker containers, etc.

Motivation

The Ontario Institute for Cancer Research (OICR) has built a few Data Portals. e.g.:

International Cancer Genome Consortium (ICGC) Data Portal
Genomic Data Commons (GDC) Data Portal (joint effort with University of Chicago)

Although they are not identical in architecture, available data or overall purpose, there is tremendous amount of overlap in how they function and how users interact with them, despite being implemented differently. It's no coincidence. The GDC Data Portal was directly influenced by the ICGC Data Portal.

With new projects ahead of us, there is an opportunity to create a framework designed to act as a core library for any given data portal, similar to what Elastic's Kibana accomplishes; but based on the features of our existing portals, and the expectation of continuous improvement and domain specific customization.

There are many potential benefits:

Reduce duplicate code
Ability to fix bugs and add features to many projects at once
Pool developer resources
Increase cross-team communication
Encourage open source contribution

What Is A "Data Portal"?

Topology

DP Topology this is way too simplistic. needs an update

Development Details

Arranger is a lerna flavored monorepo. The modules exposed by Arranger compose all of the necessary code required to build an application such as the Genomic Data Commons.*

Migration notes

If you're coming from an older version of Arranger and you use the Docker images, bear in mind we've made a changes to the Admin-UI image:

The app's port will be 3000 inside the container by default
you need the new environment variable REACT_APP_ARRANGER_ADMIN_ROOT for the container to start

Releasing Instructions

From master branch, run npm run tag <version>
Publishing process will be run by Jenkins

* The GDC contains many features that are out of Arranger's scope

Package detail

@arranger/mapping-utils

readme