Sequencer Proposal: H2O Nodes

Sequencer Proposal: H2O Nodes

Executive Summary

H2O Nodes GmbH is a Vienna, Austria based node operator company founded in 2022 by Daniel Urstöger and Max Sherwood, currently staking >$50m in assets. Our core competency is running bare metal infrastructure, with the aim of boosting decentralization for liquid staking pools and node operator sets of new layer-1 blockchains. We would be delighted to run a decentralized sequencer for the Metis L2, contributing to its decentralization and good performance.

We currently run mainnet validators on Solana, Sui, and Ethereum, and testnet validators on Aleo, Berachain, dWallet, Supra, and NodeSet. On Ethereum we are long-time supporters of DVT, having tested the Obol and SSV implementations, and now are part of Lido’s SimpleDVT module on mainnet. We are also actively engaged running validators at scale on Ethereum’s Holesky testnet to explore remote signers and other technologies - our performance is ranked in the top 15%.

Service Level Agreements (SLAs)

We are confident in our ability to run a Metis sequencer with a 99.95% uptime. This is based on the uptime of our most challenging chain, Solana, where we have achieved a 30-day uptime of 99.98%. On Ethereum, by using technologies like Vouch/Dirk or DVT, we are able to achieve fault tolerance via threshold signing, so that an issue with a single instance or key does not affect the liveness of a validator. For more “vanilla” setups, we run a number backup beacon nodes synced with a variety of clients, so that we can connect to any in case of a problem with a primary. Most of our validators are run on our own hardware in our “home” data center outside Frankfurt, but we also have access to bare metal hardware from other providers in other locations to boost geographic decentralization, if desired.

On a hardware level, our network is built with redundancy in mind so there is no single point of failure - components like switches and routers each have a backup. As DDOS protection we use Arbor as an on site solution, our firewall solution is an OPNsense cluster with IDS and IPS from Zenarmor. Physical access to the datacenter is monitored, logged and audited to industry standards. (ISO 27001, IDW PS951) The access system uses biometric data to validate access. Management access to the nodes is secured via VPN (2FA only), access to the OS is handled with Public / Private keys. For change management we have a ticket system with audit logs and a test setup which we use to verify and validate the updates.

We have comprehensive monitoring and alerting, and our team is able to respond to issues at any time. For alerting we run Atlassian OpsGenie, and as a backup we have Pager Duty. For internal monitoring tools we use check_mk, Loki, Prometheus and Grafana. We also have alerts for hardware-level warnings, as well as for the virtual machines which run the validator clients.

Benefits for Community (optional)

Co-founder Max Sherwood is an active member of the communities that H2O Nodes is a part of, for example his research highlighting the decentralization of Solana’s validator set was featured by multiple publications and inspired the Solana Foundation to publish their first-ever “validator health report” two weeks later. During the two most recent chain outage incidents on Solana, H2O Nodes were among the first validators to signal readiness for restart and were inspired to help organize a monthly Solana validator community-led discussion which has now been ongoing for over one year.

We have found that organizing such validator community discussions on a regular basis have been invaluable for sharing key learnings, voicing concerns, and discussing important governance topics, and would be happy to help with the organization of such a call for Metis operators, if there is appetite.

On the technical side, we are always happy to share dashboards, performance metrics for our hardware setup, and key learnings in discord, especially during critical testnet phases.

Integration Roadmap

Our current work is focused on helping bring Aleo, Berachain, dWallet, and Supra through their testnet phases and eventually to mainnet. We are also bringing on more freelancers to help boost our resources and develop deeper focuses on individual blockchains. Aside from purchasing our own hardware, we are furthering our partnerships with bare metal providers, expanding our footprint on Ethereum with the set-up of infrastructure for SSV, StakeWise, and NodeSet, and gaining experience on Ethereum testnet with fault-tolerant technologies like SSV DVT, Obol DVT, and Dirk / Vouch for eventual deployment on mainnet.

Official Links: Website, Docs, Audits, etc.

Can you answer this?

How does H2O Nodes GmbH’s extensive experience with validator technologies like DVT, threshold signing, and remote signers contribute to its ability to ensure high uptime and fault tolerance for the Metis sequencer, and what specific strategies or technologies would you employ to achieve this?