Principal Java Engineer – Distributed Systems, Serverless – Elasticsearch

Referral Board; United States; On-site; USD 180,000 - 260,000; Full time

Job summary

Referral Board is seeking a Principal Software Engineer for the Elasticsearch – Distributed Systems team in the United States. You will tackle deep distributed systems challenges, focusing on scale, performance, and resilience within our Serverless offering.

You will design, build, and operate core pieces, improve concurrent indexing, and maintain cluster coordination. You’ll own projects end-to-end and mentor others while advancing AI-assisted development.

Qualifications

• Deep background in distributed systems and consensus algorithms.
• Strong skills in core Java and standard data structures and concurrency concepts.
• Experience building large-scale distributed systems with multi-machine deployments.
• Ability to own projects end-to-end and collaborate across teams.
• Proven track record of using AI to accelerate development and debugging.

Responsibilities

• Design, build, and operate core pieces of a serverless Elasticsearch offering.
• Improve components for concurrent and consistent indexing across machines.
• Maintain cluster coordination to preserve performance as nodes join/leave.
• Push limits on shards, nodes, and petabytes handled by Elasticsearch.
• Investigate performance or concurrency issues and propose solutions.
• Support engineers with complex problems and own end-to-end investigations.

Skills

Distributed systems

Consensus algorithms

Java

Data structures

Concurrency

Algorithms

Project ownership

Cross-functional collaboration

AI in development

Tools

Netty

JDK

Job description

What Is The Role:

We are on the lookout for a Principal Software Engineer to join our Elasticsearch – Distributed Systems teamand focus on how Elasticsearch provides scale, performance, and resilience, especially in the context of our Serverless offering. You will be tackling deep distributed systems challenges in how nodes in an Elasticsearch cluster communicate, and how data are indexed, allocated, and managed across nodes.

What You Will Be Doing:

• Leveraging your deep distributed systems expertise to design, build, and operate core pieces of our next-generation Serverless Elasticsearch offering, directly impacting the experience of customers worldwide.
• Improving Elasticsearch’s components that support concurrent and consistent indexing across multiple machines.
• Maintaining our cluster coordination system to keep performance high even though nodes come and go from the cluster and data moves around, while maintaining the safety and liveness properties of the system as a whole.
• Pushing the limits on the number of shards, nodes, and petabytes that Elasticsearch can handle today.
• Looking into all kinds of issues, including performance or concurrency issues, and proposing solutions.
• Supporting our support engineers with the harder problems.
• Owning operational investigations end-to-end, until performance or stability issues are successfully resolved.

What You Bring:

• You have a deep background in distributed systems and consensus algorithms.
• You have strong skills in core Java and are conversant in the standard library of data structures and concurrency constructs, as well as newer language features.
• You have a deep technical proficiency in algorithms.
• You have experience building large-scale distributed systems, rather than making use of off-the-shelf services.
• You have experience building and debugging complex features with a broad impact, running on multiple machines.
• You are able to own projects from beginning to end. This covers both technical design and working with others to develop needed components.
• You have experience managing projects involving multiple engineers.
• You have a proven track record of using AI to accelerate development, debug complex systems, and optimize code, while still owning the final outcomes.
• You possess the ability to collaborate across functions and teams and seamlessly transition between different projects, codebases, or teams based on business priorities
• You can work autonomously, drive decisions and results in a distributed team by leveraging asynchronous, direct, and transparent communication.

Bonus Points:

• Data stores
• Search, analytics, Lucene
• Strong knowledge of the JDK
• Asynchronous event-driven network frameworks such as Netty