Software Engineer, ML Systems

Harmonic Security; Palo Alto (CA); On-site; USD 140,000 - 210,000; Full time

Benefits offered by this job

Unlimited PTO

401(k) matching

100% employer-paid health, vision, and

dental benefits for employees and 50%

Health Savings Account (HSA) available

Job summary

Harmonic is seeking a pragmatic Software Engineer to own the productionization of our research pipelines in Palo Alto. This implementation-heavy role requires turning nascent ideas into scalable, reliable ML workflows running in our cloud infrastructure.

You will build end-to-end ML pipelines, refactor research code, establish CI/CD for models, and manage Kubernetes deployments with observability tooling to monitor training health.

Qualifications

• BS in Computer Science, a related technical field, or equivalent industry experience.
• 2+ years of relevant industry experience.
• Expert-level Python skills and a disciplined approach to software engineering (testing, versioning, and modular design).
• Experience building and managing end-to-end ML pipelines in a production or research-intensive environment.

Responsibilities

• Pipeline Engineering: Build and manage end-to-end ML pipelines (ETL and automated evaluation) that are the bedrock of our RL research.
• Bottleneck Resolution: Identify and refactor inefficient research code. You act as the primary engineer ensuring that a promising idea reaches its full potential through scalable code.
• Standardization: Establish best practices for versioning, experiment tracking, and CI/CD for ML models to ensure reliability.
• Cloud Infrastructure & Observability: Manage the deployment and scaling of workloads on Kubernetes. Implement the tooling and telemetry that allows the team to understand agent behavior and training health at a glance.

Skills

Python

End-to-end ML pipelines

Software engineering discipline

Education

BS in Computer Science or related field

Equivalent industry experience

Tools

Docker

Kubernetes

Kubeflow

Airflow

Metaflow

AWS/GCP/Azure

Job description

About Harmonic

At Harmonic, we are building a mathematical reasoning engine that operates with absolute precision. While most AI makes maximum-likelihood guesses, Harmonic's Aristotle uses Lean 4 and reinforcement learning to verify its reasoning and results.
Following our Gold Medal-level performance on the 2025 International Math Olympiad (IMO) and the successful resolution of long-standing open problems, we are proving that AI can master the most rigorous domains of human thought. Backed by some of the world’s most prominent investors, we are intentionally scaling an elite technical team.
Visit our company blog to learn more about what we are working on!

About the Role

We are looking for a pragmatic, Software Engineer to own the productionization of our research pipelines. This is an implementation-heavy role designed for an engineer who can take a nascent research idea and build the robust, scalable machinery required to prove it at scale within our cloud infrastructure.

Key Responsibilities

• Pipeline Engineering: Build and manage end-to-end ML pipelines (ETL and automated evaluation) that are the bedrock of our RL research.

• Bottleneck Resolution: Identify and refactor inefficient research code. You act as the primary engineer ensuring that a promising idea reaches its full potential through scalable code.

• Standardization: Establish best practices for versioning, experiment tracking, and CI/CD for ML models to ensure reliability.

• Cloud Infrastructure & Observability: Manage the deployment and scaling of workloads on Kubernetes. Implement the tooling and telemetry that allows the team to understand agent behavior and training health at a glance.

Minimum Qualifications

• BS in Computer Science, a related technical field, or equivalent industry experience

• 2+ years of relevant industry experience

• Expert-level Python skills and a disciplined approach to software engineering (testing, versioning, and modular design).

• Experience building and managing end-to-end ML pipelines in a production or research-intensive environment.

Preferred Qualifications

• Full-stack ML experience: Comfortable moving from data engineering to model debugging.

• Experience refactoring research-grade code into high-quality, scalable production packages.

• Proven ability to design and implement complex data-loading and evaluation systems for non-deterministic models.

• Experience with workflow orchestration tools (e.g., Kubeflow, Airflow, or Metaflow).

• Experience managing large-scale experiments on cloud providers (AWS, GCP, or Azure).

• Proven track record collaborating directly with researchers to translate algorithmic requirements into engineering roadmaps.

• Hands-on experience with containerization (Docker) and orchestration (Kubernetes).

What We Offer

• Unlimited PTO

• 401(k) matching

• 100% employer-paid health, vision, and dental benefits for employees and 50% coverage for dependents. Harmonic offers varied health coverage options to select what is best for you and your family.

• Health Savings Account (HSA) available for qualifying health plans

Equal Opportunity Statement

Harmonic is committed to diversity and inclusivity in the workplace. We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.