Software Engineer, Inference - Performance Optimization

OpenAI; San Francisco (CA); On-site; USD 295,000 - 555,000; Full time

Job summary

OpenAI is looking for a performance modeler in San Francisco who will analyze inference stack performance and build cost-to-serve estimates. In this role, candidates should have expertise in performance profiling and enjoy reasoning about distributed systems.

The position offers a compensation range of $295K to $555K and requires collaboration with engineering and research teams to enhance performance and address system bottlenecks.

Qualifications

• Expertise in performance profiling, analysis, and optimization.
• Understanding of distributed systems and microbenchmarking.
• Ability to collaborate with engineering and research teams.

Responsibilities

• Build and refine performance models from microbenchmark results.
• Analyze inference workloads end to end across the application.
• Enhance tooling to identify bottlenecks for latency and throughput.

Skills

Performance profiling

Benchmarking

Systems analysis

Model inference

Optimization techniques

Job description

About The Team

Our team analyzes inference stack performance across the application, model, and fleet layers to identify bottlenecks and drive faster, cheaper inference. We combine systems profiling, benchmarking, and analysis to understand where time and cost are spent, then turn that understanding into performance optimizations and models that project performance and capacity needs for future launches.

About The Role

In this role, you will model inference performance across application, model, and fleet layers with higher fidelity. You will build cost-to-serve estimates from microbenchmarks and create tools that help cross-functional teams reason about latency, capacity, utilization, and cost tradeoffs.

In this role, you will

• Build and refine performance models that translate microbenchmark results into cost-to-serve estimates.
• Analyze inference workloads end to end across applications, models, and fleet infrastructure. Enhance tooling to identify bottlenecks across layers for latency and throughput.
• Partner with other teams to turn performance insights into concrete improvements and project how future changes affect inference.

Role

You might thrive in this role if you:

• Enjoy reasoning from first principles about distributed systems, model inference, and hardware efficiency.
• Are comfortable working across abstraction layers, from application behavior to kernels, accelerators, networking, and fleet scheduling.
• Have deep expertise with performance profiling, benchmarking, analysis, and optimization.
• Enjoy collaborating with engineering and research teams to improve real production systems.

Background Checks

Background checks for applicants will be administered in accordance with applicable law, and qualified applicants with arrest or conviction records will be considered for employment consistent with those laws, including the San Francisco Fair Chance Ordinance, the Los Angeles County Fair Chance Ordinance for Employers, and the California Fair Chance Act, for US-based candidates. For unincorporated Los Angeles County workers: we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: protect computer hardware entrusted to you from theft, loss or damage; return all computer hardware in your possession (including the data contained therein) upon termination of employment or end of assignment; and maintain the confidentiality of proprietary, confidential, and non-public information. In addition, job duties require access to secure and protected information technology systems and related data security obligations.

Equal Opportunity Statement

We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic. For additional information, please see OpenAI’s affirmative action and equal employment opportunity policy statement.

Compensation Range

$295K - $555K