Senior ML Infra Engineer: Automation & CI/CD

fal-ai; United States; Remote; USD 150,000 - 210,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

Job summary

fal-ai seeks a seasoned software engineer to build automation and infrastructure that helps machine learning teams develop, test, and release generative AI models and inference services. The role covers CI/CD pipelines, model quality checks, performance benchmarking, deployment safeguards, and developer tools to reduce manual effort and improve reliability.

You will design scalable pipelines, accelerate CI with caching/parallelism, validate outputs across GPUs, benchmark latency/throughput, and

Qualifications

• 5+ years of software engineering experience including production infrastructure or developer tooling.
• Strong Python skills and experience with testing, builds, dependency management.
• Experience designing and operating CI/CD systems with caching and parallel execution.

Responsibilities

• Design and maintain automated testing, validation, and deployment pipelines for models and inference services.
• Speed up CI runs using parallel execution, caching, and efficient compute allocation.
• Validate model outputs across models, GPU architectures, and configurations, and catch quality regressions.
• Benchmark inference latency, throughput, GPU utilization, and cost to detect performance regressions.
• Add automated checks for pricing, billing settings, API schemas, and deployments before release.
• Improve release safety with verification, monitoring, rollback mechanisms, and AI-assisted tooling for diagnosing CI failures and reducing repetitive work.

Skills

Python

CI/CD pipelines

Distributed systems

Troubleshooting

Logging & monitoring

DevOps

Tools

Docker

Cloud platforms

Job description

fal-ai seeks a seasoned software engineer to build automation and infrastructure that helps machine learning teams develop, test, and release generative AI models and inference services. The role covers CI/CD pipelines, model quality checks, performance benchmarking, deployment safeguards, and developer tools to reduce manual effort and improve reliability.

You will design scalable pipelines, accelerate CI with caching/parallelism, validate outputs across GPUs, benchmark latency/throughput, and