Senior Benchmarking Engineer - Performance & Research

Elastic; Mountain View (CA); On-site; USD 133,000 - 211,000; Full time

Benefits offered by this job

Health coverage

Flexible locations & schedules

Vacation days

Donation matching up to $2000

Volunteer hours

Parental leave (16 weeks)

Job summary

Elastic is seeking a Senior Software Engineer to lead competitive benchmarking initiatives within the Elasticsearch Performance team in Mountain View, CA. You will develop workloads, automate experiments, and compare Elasticsearch against alternative systems to guide product improvements.

This hands-on role focuses on building benchmarks, profiling, and publishing technical reports, with collaboration across engineering and content teams to ensure meaningful, reproducible results.

Qualifications

• Solid software engineering skills with automation and analysis tools in Python or another relevant language.
• Hands-on experience evaluating performance in distributed systems, databases, or search infrastructure.
• Depth in at least one relevant area (search, vector retrieval, analytical databases, or observability) and ability to learn unfamiliar systems.
• Experience designing representative datasets and workloads, including query patterns, concurrency, and ingestion behavior.
• Sound experimental judgment, including repeated measurements, variability analysis, warm-up and cache effects.
• Experience building reproducible test environments and documenting configurations, procedures, and results.
• Clear technical writing and communication skills to explain findings and limitations.

Responsibilities

• Own prioritized competitive benchmarking initiatives across search, vector and hybrid retrieval, and analytical and observability workloads.
• Build and maintain automated benchmark harnesses, datasets, query sets, and load profiles reflecting real-world usage.
• Design fair comparisons with documented system versions, configurations, tuning decisions, resource budgets, and differences.
• Evaluate latency distributions, throughput, resource consumption, and cost efficiency; measure performance at recall targets.
• Extend workload-appropriate tools such as Elasticsearch Rally, ClickBench, and vector search benchmark frameworks.
• Investigate performance differences using profiling and system metrics; collaborate with engineering to identify causes and improvements.
• Evaluate self-managed and managed cloud deployments, including serverless offerings and scaling behavior.
• Publish reproducible benchmark artifacts and technical reports with methodology, findings, limitations, and tradeoffs.
• Partner with engineering and product management to translate findings into product improvements.

Skills

Python

Distributed systems

Benchmarking

Performance analysis

Technical writing

Collaboration

Tools

Elasticsearch Rally

ClickBench

Job description

Elastic is seeking a Senior Software Engineer to lead competitive benchmarking initiatives within the Elasticsearch Performance team in Mountain View, CA. You will develop workloads, automate experiments, and compare Elasticsearch against alternative systems to guide product improvements.

This hands-on role focuses on building benchmarks, profiling, and publishing technical reports, with collaboration across engineering and content teams to ensure meaningful, reproducible results.