Armon Dadgar on HashiCorp Research, the Evolution of Infrastructure Tooling, and Standardisation
01 Oct 2024 (10 days ago)
HashiCorp Research and its Impact on Products
- HashiCorp, a cloud infrastructure company, has a research division that strives to increase collaboration between academic computer science and the mainstream computing industry. (26s)
- HashiCorp has incorporated fundamental computer science research into its products, such as Consul and Nomad. (3m51s)
Consul and Nomad: Built on Research
- Consul and Nomad are built on foundational computer science algorithms and research, with Consul utilizing a consensus algorithm and raft at its core. (4m32s)
- Console uses a distributed gossip layer and network coordinate system to function as a GPS for networks, enabling features like nearest neighbor routing. (5m11s)
Lifeguard: A Novel Extension to the Gossip Protocol
- HashiCorp published a paper on Lifeguard, their novel extension to the gossip protocol, which improves the resilience of the system by addressing the issue of unhealthy nodes spreading misinformation. (6m56s)
- HashiCorp presented the Lifeguard paper at DSN, an academic conference with an industry track, and received positive feedback, including from the original authors of the Swim protocol. (7m37s)
HashiCorp Research: Focus on Practical Applications
- HashiCorp Research focuses on Industrial Research, prioritizing projects with practical applications or market viability within 18 to 24 months. (9m31s)
- HashiCorp invests in three key areas: making tools more user-friendly, ensuring seamless integration with major communities and technologies like Kubernetes and serverless offerings, and enhancing the stability, scalability, and enterprise-readiness of their tools. (12m9s)
Incremental Improvements and User Focus
- HashiCorp aims for incremental improvements rather than radical departures to avoid disrupting users in production environments. (13m42s)
- HashiCorp has invested in learn.hashicorp.com, a learning platform with tutorials and a team dedicated to content creation, to help people learn and adopt tools like Terraform. (14m36s)
Community Engagement and Certification
- A community portal was created to centralize questions and answers about HashiCorp products, addressing the issue of fragmented community discussions across various platforms. (14m51s)
- HashiCorp plans to introduce certifications for their tools to help individuals demonstrate their expertise and employers to gauge competency levels. (15m42s)
- Terraform is a provisioning tool that can handle bare metal, virtual machines, containers, and serverless environments. (18m12s)
- The network is a great common denominator that allows different technologies, such as mainframes, bare metal, virtual machines, and containers, to communicate with each other using TCP. (19m11s)
Importance of Standards for Interoperability
- Standards are important for interoperability, especially in areas with high switching costs, such as telemetry, where retooling applications to switch providers can be a significant undertaking. (19m28s)