Senior ML Engineer - From Research to Production (Remote)

Openkrill; United States; Hybrid; USD 211,000 - 249,000; Full time

Benefits offered by this job

Inclusive team

Career growth

Hybrid & remote work

Competitive compensation

Job summary

Horizon3 is seeking a Senior Machine Learning Engineer to bring defensive models into production on the Autonomous Defense team. You will partner with AI researchers, build pipelines, serve infrastructure, and own the path from model to production across thousands of customer tenants.

Responsibilities include training/post-training pipelines, model serving, release management, monitoring, and cost optimization, with collaboration to align research with production constraints and business needs.

Qualifications

• Bachelor’s Degree in Computer Science, Computer Engineering or related field, or equivalent practical experience.
• 5+ yrs professional software engineering experience, with strong production Python.
• Demonstrated experience taking ML or LLM-backed systems from prototype to production and operating them.
• Hands-on experience with ML pipelines and tooling: training or fine-tuning workflows, experiment tracking, artifact and model registries, and reproducible data preparation.
• Experience building and operating inference or model-serving infrastructure in production, including latency and cost optimization.
• Experience building applications on cloud computing platforms such as AWS, Azure, GCP, using container technologies such as Docker and Kubernetes.
• Solid proficiency in SQL and experience with production data pipelines.

Responsibilities

• Build and own the training and post-training pipelines — data preparation, fine-tuning and preference optimization runs, experiment tracking, artifact management, and reproducibility.
• Build the inference and serving layer: model gateway with provider routing and fallback, regional pinning for data residency, batching, and caching.
• Own the release path for model-layer artifacts: version prompts, model selections, and tool definitions as deployable config; run shadow and canary deployments by tenant; make rollback fast and boring.
• Build monitoring for the model layer — behavioral drift, regression detection, output quality signals, latency, and per-tenant token and cost accounting with budget enforcement.
• Build the data and context pipelines that feed inference, including retrieval and embedding infrastructure over attack path, configuration, and remediation data, with tenant isolation enforced end to end.
• Optimize cost and latency across the inference path, and make the tradeoffs visible so product decisions are made with real numbers.
• Develop core product features in ETL and GraphQL where model outputs, run history, and evaluation results need to reach the product and internal tooling.
• Partner with researchers to move prototypes into production, and feed production constraints and failure data back into research direction.

Skills

Python

ML pipelines

SQL

CI/CD

Education

Bachelor’s Degree in Computer Science/Engineering

Tools

Docker

Kubernetes

PostgreSQL

Neo4j

GraphQL

Job description

Horizon3 is seeking a Senior Machine Learning Engineer to bring defensive models into production on the Autonomous Defense team. You will partner with AI researchers, build pipelines, serve infrastructure, and own the path from model to production across thousands of customer tenants.

Responsibilities include training/post-training pipelines, model serving, release management, monitoring, and cost optimization, with collaboration to align research with production constraints and business needs.