Open in app

Sign in

Write

Sign in

Private EOS network — Part 2: Hyperion

Levan Ilashvili
6 min readFeb 14, 2022

--

Hello 🤘

In Private EOS network — Part 1:Setup we talked about setting up a private EOS network with two block producers.

Today we’re gonna make one VERY important enhancement and set up Hyperion History API for our network.

So, what is Hyperion?

Hyperion is the father of the goddess EOS and means “the one who watches from above”.

Hyperion is a full history API for any EOSIO-powered blockchain, storing and indexing their historical data.

Hyperion itself relies on a few components:

  • ElasticSearch cluster
  • Kibana
  • RabbitMQ
  • Redis
  • Nodeos History Plugin

and it's very own

  • Hyperion Indexer
  • Hyperion API (which has two parts — v1 is standard API provided by Nodeos history plugin and v2 — provided by Hyperion)

Hyperion Docs describe two paths we can take for installing Hyperion — manual and using installation script. Since our goal is to set up Hyperion and not dive into depths of installing ElasticSearch/Kibana/RMQ/etc, let’s go with installation script.

Setup

Before we move to setting up Hyperion, let’s enable State History Plugin on one of our nodes. I’m assuming you have start.sh script from the previous tutorial. So, simply go to one of the nodes’ directory and edit start.sh

cd ~/eos/node2
nano start.sh

add the following parameters to nodeos start config:

--plugin eosio::state_history_plugin \
--state-history-dir $DATADIR/"state-history" \
--disable-replay-opts \
--trace-history \
--chain-state-history \
--state-history-endpoint 0.0.0.0:8080 \

and then, do the same for genesis_start.sh .

Now, clone Hyperion and checkout v3.3.5

cd ~/
git clone https://github.com/eosrio/hyperion-history-api.git
cd hyperion-history-api
git checkout v3.3.5

Now, install Node16 manually — installation script (at the moment of writing this tutorial) contains Node13 and that’s gonna mess up lots of things for us.

cd ~
curl -sL https://deb.nodesource.com/setup_16.x | sudo bash -
sudo apt -y install nodejs

verify Node version:

node -v

should output

v16.14.0

Now, let’s continue with Hyperion installation. Simply run:

cd ~/hyperion-hiirory-api
sudo ./install_env.sh

Script will ask you to enter username/password for RMQ and will do the rest of stuff on its own. If everything goes well, soon you’ll see this lovely message:

Let’s set up our Hyperion with hyp-config

First, let’s have a look into example-connections.json

{
  "amqp": {
    "host": "127.0.0.1:5672",
    "api": "127.0.0.1:15672",
    "protocol": "http",
    "user": "username",
    "pass": "password",
    "vhost": "hyperion",
    "frameMax": "0x10000"
  },
  "elasticsearch": {
    "protocol": "http",
    "host": "127.0.0.1:9200",
    "ingest_nodes": [
      "127.0.0.1:9200"
    ],
    "user": "elastic",
    "pass": "password"
  },
  "redis": {
    "host": "127.0.0.1",
    "port": "6379"
  },
  "chains": {
    "eos": {
      "name": "EOS Mainnet",
      "chain_id": "aca376f206b8fc25a6ed44dbdc66547c36c6c33e3a119ffbeaef943642f0e906",
      "http": "http://127.0.0.1:8888",
      "ship": "ws://127.0.0.1:8080",
      "WS_ROUTER_PORT": 7001,
      "WS_ROUTER_HOST": "127.0.0.1"
    }
  }
}

unfortunately, ./hyp-config init connections command is broken, so we’ll have to update ampq user/password and elasticsearch password manually. You’ll find ElasticSearch passwords in the same directory — just cat elastic_pass.txt

cp example-connections.json connections.json

and update connection params. After updateconnections.json looks like this:

{
  "amqp": {
    "host": "127.0.0.1:5672",
    "api": "127.0.0.1:15672",
    "protocol": "http",
    "user": "hyperion",
    "pass": "123456",
    "vhost": "/hyperion",
    "frameMax": "0x10000"
  },
  "elasticsearch": {
    "protocol": "http",
    "host": "127.0.0.1:9200",
    "ingest_nodes": [
      "127.0.0.1:9200"
    ],
    "user": "elastic",
    "pass": "aqCL9wTcjxGkVvUVKv5E"
  },
  "redis": {
    "host": "127.0.0.1",
    "port": "6379"
  },
  "chains": {
  }
}

make sure your nodeos is running with correct params:

cat ~/eos/node2/start.sh#!/bin/bash
DATADIR="./blockchain"
CURRENTDIR=${PWD##*/}
PRODUCER_PUB_KEY="EOS..."
PRODUCER_PRIV_KEY="5..."
HTTP_PORT=8012
P2P_PORT=9012
if [ ! -d $DATADIR ]; then
mkdir -p $DATADIR;
finodeos \
--signature-provider $PRODUCER_PUB_KEY=KEY:$PRODUCER_PRIV_KEY \
--plugin eosio::producer_plugin \
--plugin eosio::producer_api_plugin \
--plugin eosio::chain_plugin \
--plugin eosio::chain_api_plugin \
--plugin eosio::http_plugin \
--plugin eosio::history_api_plugin \
--plugin eosio::history_plugin \
--plugin eosio::state_history_plugin \
--state-history-dir $DATADIR/"state-history" \
--disable-replay-opts \
--trace-history \
--chain-state-history \
--state-history-endpoint 0.0.0.0:8080 \
--data-dir $DATADIR"/data" \
--blocks-dir $DATADIR"/blocks" \
--config-dir $DATADIR"/config" \
--producer-name $CURRENTDIR \
--http-server-address 127.0.0.1:$HTTP_PORT \
--p2p-listen-endpoint 127.0.0.1:$P2P_PORT \
--access-control-allow-origin=* \
--contracts-console \
--http-validate-host=false \
--verbose-http-errors \
--enable-stale-production \
--p2p-peer-address localhost:9010 \
--p2p-peer-address localhost:9012 \
>> $DATADIR"/nodeos.log" 2>&1 & \
echo $! > $DATADIR"/eosd.pid"

now, let’s run hyp-config with the following params:

cd ~/hyperion-history-api
./hyp-config new chain pvt --http "http://127.0.0.1:8012" --ship "ws://127.0.0.1:8080"

output should look like this:

Creating new config for pvt...Testing connection on http://127.0.0.1:8012Parser version set to 2.1Chain ID: 84a10492c6c58a01031e87ef391723624c8c7d72fc37b8535ae4a6d7b193d47dTesting connection on ws://127.0.0.1:8080Received SHIP Abi Version: eosio::abi/1.1{name: 'Pvt',
ship: 'ws://127.0.0.1:8080',
http: 'http://127.0.0.1:8012',
chain_id: '84a10492c6c58a01031e87ef391723624c8c7d72fc37b8535ae4a6d7b193d47d',
WS_ROUTER_HOST: '127.0.0.1',
WS_ROUTER_PORT: 7001}Saving connections.json...Saving chains/pvt.config.json...

Let’s start our API and Indexer:

./run pvt-api
./run pvt-indexer

You’ll see above message on pvt-indexer ‘s first run. Once you see that message (Block Range Completed) — stop the indexer:

./stop.sh pvt-indexer

Here comes the tricky part — let’s have a look into chains/pvt.config.json — you’ll notice indexer options:

"indexer": {
"enabled": true,
"node_max_old_space_size": 4096,
"start_on": 0,
"stop_on": 0,
"rewrite": false,
"purge_queues": false,
"live_reader": false,
"live_only_mode": false,
"abi_scan_mode": true,
"fetch_block": true,
"fetch_traces": true,
"disable_reading": false,
"disable_indexing": false,
"process_deltas": true,
"disable_delta_rm": true
},

On the first run, you must have abi_scan_mode set to true, on the second run — set it to false and then — set live_reader to true . Huh. Let’s do it — update pvt.config.json and run the indexer again:

./run.sh pvt-indexer

It syncs and then stops processing blocks:

2022-02-14T00:39:32: [18566 - 00_master] W:13 | R:2596 | C:776.6 | A:388.2 | D:2537.2 | I:3263.4 | 3883/12980/12979 | syncs in a few seconds (29.9% 100.0%)2022-02-14T00:39:37: [18566 - 00_master] W:13 | R:0 | C:1036.8 | A:1404.2 | D:5291 | I:7725.2 | 9067/12980/12979 | syncs in a few seconds (69.9% 100.0%)2022-02-14T00:39:42: [18566 - 00_master] W:13 | R:0 | C:782.6 | A:819.4 | D:3997 | I:6049.22022-02-14T00:39:47: [18566 - 00_master] W:13 | R:0 | C:0 | A:0 | D:0 | I:02022-02-14T00:39:47: [18566 - 00_master] No blocks are being processed, please check your state-history node!

Now set live_reader to true in chains/pvt.config.json and restart the indexer. You’ll see it’s running and indexing blocks:

2022-02-14T00:41:51: [18981 - 00_master] W:15 | R:58.2 | C:58.2 | A:57.8 | D:290 | I:407.42022-02-14T00:41:56: [18981 - 00_master] W:15 | R:2 | C:2 | A:2 | D:10 | I:142022-02-14T00:42:01: [18981 - 00_master] W:15 | R:2 | C:2 | A:2 | D:10 | I:142022-02-14T00:42:06: [18981 - 00_master] W:15 | R:2 | C:2 | A:2 | D:10 | I:142022-02-14T00:42:11: [18981 - 00_master] W:15 | R:2 | C:2 | A:2 | D:10 | I:142022-02-14T00:42:16: [18981 - 00_master] W:15 | R:2 | C:2 | A:2 | D:10 | I:142022-02-14T00:42:21: [18981 - 00_master] W:15 | R:2 | C:2 | A:2 | D:10 | I:142022-02-14T00:42:26: [18981 - 00_master] W:15 | R:2 | C:2 | A:2 | D:10 | I:142022-02-14T00:42:31: [18981 - 00_master] W:15 | R:2 | C:2 | A:2 | D:10 | I:142022-02-14T00:42:36: [18981 - 00_master] W:15 | R:2 | C:2 | A:2 | D:10 | I:142022-02-14T00:42:41: [18981 - 00_master] W:15 | R:2 | C:2 | A:2 | D:10 | I:14

Time to look at our Hyperion —Hyperion API is running on Port 7000 — make sure you have inbound connections allowed. Also, let’s tweak API config a bit — go to pvt.config.json and make sure these options look like this:

"server_addr": "0.0.0.0",
"server_port": 7000,
"server_name": "YOUR_IP:7000",

Let’s hit our Hyperion!

Navigate to http://YOUR_SERVER_IP:7000/v2 — you should see Swagger 🤘

Hit the health endpoint:

{
  "version": "3.3.5",
  "version_hash": "62ee9b2ac739aabe3bdf718f9c834f07a83cd169",
  "host": "......",
  "health": [
    {
      "service": "RabbitMq",
      "status": "OK",
      "time": 1644799728975
    },
    {
      "service": "NodeosRPC",
      "status": "OK",
      "service_data": {
        "head_block_num": 14106,
        "head_block_time": "2022-02-14T00:48:49.000",
        "time_offset": -21,
        "last_irreversible_block": 14091,
        "chain_id": "84a10492c6c58a01031e87ef391723624c8c7d72fc37b8535ae4a6d7b193d47d"
      },
      "time": 1644799728979
    },
    {
      "service": "Elasticsearch",
      "status": "OK",
      "service_data": {
        "last_indexed_block": 14105,
        "total_indexed_blocks": 14105,
        "active_shards": "100.0%"
      },
      "time": 1644799728988
    }
  ],
  "features": {
    "streaming": {
      "enable": false,
      "traces": false,
      "deltas": false
    },
    "tables": {
      "proposals": true,
      "accounts": true,
      "voters": true
    },
    "index_deltas": true,
    "index_transfer_memo": false,
    "index_all_deltas": true,
    "deferred_trx": false,
    "failed_trx": false,
    "resource_limits": false,
    "resource_usage": false
  },
  "query_time_ms": 44.824
}

And make sure last_indexed_block and total_indexed_blocks are not 1 — it means our indexer is running and doing it’s job.

That’s it — good job! We have our Hyperion API running for our private EOS network.

Eosio
Eos
Hyperion

--

--

Levan Ilashvili
Levan Ilashvili

Written by Levan Ilashvili

47 Followers
43 Following

Hooman of two doggos, software engineer

No responses yet

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams