Senior Benchmarking Engineer, Performance & Competition

Elastic; United States; Hybrid; USD 140,000 - 200,000; Full time

Benefits offered by this job

Competitive pay

Health coverage

Flexible locations

Generous vacation

Donation matching

Volunteer hours

Parental leave

Job summary

Elastic is seeking a Senior Software Engineer to lead competitive benchmarking within the Elasticsearch Performance team. You will build workloads, automate experiments, and compare Elasticsearch with alternative systems, guiding improvements and helping customers make informed technical decisions.

This hands-on role focuses on benchmarks for concurrent ingestion and hybrid search, with publication of clear technical reports and close collaboration with engineering and content partners.

Qualifications

• Solid software engineering skills with automation and analysis tooling in Python or similar.
• Experience evaluating performance in distributed systems, DBs, or search infrastructure.
• Depth in at least one area (search, vector retrieval, analytical DBs, or observability).
• Experience designing representative datasets and workloads with concurrency and ingestion behavior.
• Sound experimental judgment with repeated measurements and noise analysis.
• Experience building reproducible test environments and documenting configs.

Responsibilities

• Own prioritized competitive benchmarking initiatives across search, vector and hybrid workloads.
• Build and maintain automated benchmark harnesses, datasets, query sets, and load profiles reflecting real-world usage.
• Design fair comparisons with documented versions, configurations, and resource budgets.
• Evaluate latency, throughput, resource use and cost; measure recall at set targets.
• Extend workload tooling like Rally, ClickBench, and vector benchmark frameworks.
• Investigate performance differences using profiling and controlled experiments; collaborate with engineering teams to validate improvements.
• Evaluate self-managed and cloud deployments, including serverless, considering scaling behavior and constraints.
• Publish reproducible benchmark artifacts and clear technical reports with methodology and tradeoffs.
• Partner with engineering and product management to prioritize benchmark coverage and translate results into product improvements.

Skills

Python

Distributed systems

Benchmarking

Performance analysis

Technical writing

Collaboration

Tools

Elasticsearch Rally

ClickBench

Vector benchmarks

Job description

Elastic is seeking a Senior Software Engineer to lead competitive benchmarking within the Elasticsearch Performance team. You will build workloads, automate experiments, and compare Elasticsearch with alternative systems, guiding improvements and helping customers make informed technical decisions.

This hands-on role focuses on benchmarks for concurrent ingestion and hybrid search, with publication of clear technical reports and close collaboration with engineering and content partners.