*Note: This review and score is purely based on the information disclosed by the validator service and the scoring rubric.
Last Updated: Oct 6, 2019
Validator.Network runs validator services out of Northern Europe. The team consists of individuals with years of experience in IT and highly available systems.
Team Background (38/100)
- Full-Time/Part-Time (5/10)
- Prior Blockchain Dev/Impact (0/10)
- Systems Experience (10/10)
- Recognizability (0/10)
Current Voting Power (60/100)
- Total Staked: (8/10)
- Unique Self-Bonders: (5/10)
- Commissions: (5/10)
Historical Metrics (90/100)
- Uptime (8/10)
- Proposals (10/10)
- Legal Compliance/Insurance (0)
- Innovations (0)
Validator.Network is comprised by the team behind E-money, a project utilizing blockchain technology to provide the next generation of money. E-money / Validator.Network was co-founded by Martin Dyring-Andersen and Henrik Aasted Sorsensen. Both individuals have 15+ years of experience in IT and financial infrastructure. Prior to this, Henrik ran his own IT-consultancy within Financial Services and Software Engineering. Outside of their current work, Validator.Network and E-money, the team has not worked on major open-source blockchain projects.
E-money is being built on top of the Cosmos and IRISnet networks, hence the launch of their validator service on both networks. The incentives are nicely aligned between E-money and Validator.Network. However, with the long term goal being E-money, the team must split their time and improve Validator.Network on a part-time basis.
Their validator service on the Cosmos mainnet currently takes on a 10% commision rate. Amongst validators, Validator.Network ranks #14 amongst the 100 active validators in network. The service’s staked atoms accounts for 2.03% of the voting power. To align incentives with their delegators, the team and investors have continually staked atoms using their own service. This was reported to be about ~4% of their staked atoms.
Validator.Network also runs on the IRISnet blockchain with a 10% commision rate. Due to the smaller network size, their service holds a larger stake and voting power, currently at ~20.8M iris with ~4% voting power. The team and investors also use the service to stake tokens.
Validator.Network’s service has been registered on the Cosmos mainnet since the genesis block. Since their registration, the team has maintained a 100% uptime. Hubble shows no reports of the service missing precommits or blocks. The team has shown more activity than just running a highly available network by also publishing an improvement proposal (#4 Proposal for issuance of fungible tokens directly on the Comsus Hub). The proposal in the end was not accepted by the Cosmos Hub but still captured 45.85% of the network in YES votes.
Validator.Network’s service has also registered on the IRISnet mainnet at launch and has since maintained a 100% uptime. To date, IRISnet has not had any registered proposals. However, towards the end of May 2019, the Hubble block explorer, shows three separate incidents of missed precommits.
Validator.Network participated in Game of Stakes before the Cosmos launch. Their service was never jailed and stayed in the validator set for the entire game.
Validator.Network’s service expects delegators to understand the risks of staking atoms. The service does not provide an insurance policy to cover losses due to slashing or service downtime.
- Failover (16/30)
- Private Peering (10/10)
- Agreements with other Validators (10/10)
- Sentry Scaling (10/10)
- Backup Strategy
Validator.Network uses a hybrid of public cloud and private servers. Private servers are used when dealing with sensitive keys and running validator nodes. The private servers used by Validator.Network are located in a secure datacenter in the Interxion Copenhagen Campus. The data center is monitored 24/7 by CCTV and security patrols. The campus has leveled access, biometric readers and is known for its resilience in connectivity, power, and environment. The private rack used by Validator.Network is only accessible to key personnel. Those with this ability can reach the campus in 15 to 30 minutes.
Validator.Network operates two validator nodes co-located in the datacenter. These nodes are exclusively connected through their sentry and peering nodes. Only 1 validator node is actively signing data while the second is in a hot-standby setup. No automated failover technique has been implemented. The second node in standby must be manually activated by the appropriate person. This does not require physical access into the datacenter. The servers can be accessed through a VPN safeguarded by SSH keys and additional IP restriction. This manual intervention is a source of potential downtime in this scheme. However, this is done to prevent unintended double-signing, which has more severe consequences than being offline temporarily.
Validator.Network makes use of the common sentry based node architecture to scale their service. The public cloud is used to achieve the desired scale. Validator.Network currently only makes use of AWS to deploy their sentry and peering nodes. Their sentry nodes are layered behind load balancers to provide high availability. Unlike their validator architecture, auto scaling is setup in AWS to accommodate demand to their public sentries. Single points of failures are mitigated against by distributing the sentry nodes across several availability zones. Although it’s a small likelihood that AWS has a large scale outage or attack, the sole use of AWS for their public cloud is one single point of failure in this architecture. A multi-cloud architecture is a design the team encompasses implementing in the future. It has been noted that many teams rely on cloud credits to fund their computing costs. The team at Validator.Network does not rely on cloud credits and will expand their computing resources through cost-benefit analysis.
The service maintains it’s network connectivity through private peering nodes with trusted third-party validators. Their website currently states that their service currently peers with 9 trusted validators whom are all highly recognized within the community. When asked, these peers were constructed on an informal basis. There are no contractual agreements are in place.
At the time of this writing, Validator.Network has not written custom code on top of the existing infrastructure built by the Cosmos and Tendermint teams.
The team has published scripts and pre-compiled binaries on their github, https://github.com/validator-network, to make it easy for their delegators to interact with their services and reinvest staking rewards to receive compounding interest.
Monitoring Tools (50/100)
- Network Level (5/10)
- Hardware Level (5/10)
- Paging (5/10)
Single Point of Failure (50/100)
- Multi-Cloud (0/10)
- Multi-Region (10/10)
Key Management (75/100)
- HSM Selections (10/10)
- Smart Key Management (5/10)
Validator Access (100/100)
- Physical/Remote (10/10)
The team uses standard monitoring tools to ensure a healthy service. Prometheus is an open source tool used to monitor usage. Metrics like CPU, Traffic, and Memory load are checked against to ensure that their validator nodes are receiving and signing blocks. These tools are setup to page the team when unusual activity is suspected.
Single Points of Failure
The Validator.Network service does not have any troubling single points of failure other than the security of their data center and health of AWS as a cloud provider. Sporadic tests should be run to test how quickly the team can manually failover their validator node to the hot standby and keep an accurate measure of their predicted downtime.
In the case of an AWS outage, the team has stated that their fallback plan would be to expose their private servers to the public internet. However, in this scenario with no autoscaling in place outside of AWS, this does expose the service to DoS attack.
The validator nodes in the data centers use an HSM, YubiHSM2, to sign data. Custom code cannot be run on the HSM, eliminating hardware double signature protection. Their HSM is coupled with Tendermint’s KMS set up on a separate server that offers software based double signature protection.
The HSM is only accessible to the same individuals who can access the server rack in the data center.
As outlined earlier, the data center is secured from unauthorized entry. The validator nodes are also not exposed to the internet and are layered behind sentry nodes. Even if an individual was to find the address of the validator node, firewalls exist to prevent access.
When asked, the team provided some areas of improvements for their validator service.
- Implementation of a multi-cloud architecture to mitigate the single point of failure and cloud provider redundancy.
- Automated failover with their co-located validator nodes. The team notes the risk and difficulty of a safe implementation given the comparative consequences for double signed block versus temporary downtime. For example, with the current rules in the Cosmos network, a validator would have to miss 95% of blocks in a ~14 hour time period to receive a 0.01% penalty. This is more than sufficient time to conduct a manual failover if a validator node experiences downtime. Comparatively, there is a 5% penalty for double signing a block which can mistakenly occur due to a bug in an automated failover mechanism.