Distributed Training and Inference Engineer - sciforium

OpenTalent; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

Sciforium is seeking a distributed training and inference engineer to build and optimize the ML software stack for large-scale AI workloads. You will work across CUDA/ROCm runtimes to high-level frameworks like JAX and PyTorch to ensure fast, scalable training and serving.

This role emphasizes deep systems engineering, debugging hardware–software interactions, and optimizing performance at every layer of the ML stack, enabling training and deployment of next‑gen LLMs and generative AI models.

Qualifications

• 5+ years of industry experience in ML systems or distributed training.
• Strong programming experience in Python and C++ with distributed systems.
• Familiarity with ML tooling and profiling tools.
• Solid academic background in CS/CE/EE.

Responsibilities

• Maintain and optimize ML libraries and frameworks across environments.
• Own end-to-end ML software stack from drivers to tooling.
• Tune distributed training for scalability and efficiency.
• Profile and optimize performance across multi-node clusters.
• Debug hardware–software interactions and ensure stability.
• Collaborate with research and kernel teams to improve throughput.

Skills

Python

C++

Distributed systems

ML tooling

Education

Bachelor's or Master’s degree in CS/CE/EE

Tools

Nsight

ROCm Profiler

XLA profiler

TPU tools

Job description

Sciforium is an AI infrastructure company developing next-generation multimodal AI models and a proprietary, high-efficiency serving platform. Backed by multi-million-dollar funding and direct sponsorship from AMD with hands-on support from AMD engineers the team is scaling rapidly to build the full stack powering frontier AI models and real-time applications.

ABOUT THE ROLE

Sciforium is seeking a highly skilled Distributed Training and Inference Engineer to build, optimize, and maintain the critical software stack that powers our large-scale AI training and serving workloads. In this role, you will work across the entire machine learning infrastructure from low-level CUDA/ROCm runtimes to high-level frameworks like JAX and PyTorch to ensure our distributed training systems are fast, scalable, stable, and efficient.

This position is ideal for someone who loves deep systems engineering, debugging complex hardware–software interactions, and optimizing performance at every layer of the ML stack. You will play a pivotal role in enabling the training and deployment of next-generation LLMs and generative AI models.

WHAT YOU'LL DO

• Software Stack Maintenance: Maintain, update, and optimize critical ML libraries and frameworks including JAX, PyTorch, CUDA, and ROCm across multiple environments and hardware configurations.
• End-to-End Stack Ownership: Build, maintain, and continuously improve the entire ML software stack from ROCm/CUDA drivers to high-level JAX/PyTorch tooling.
• Distributed System Optimization: Ensure all model implementations are efficiently sharded, partitioned, and configured for large-scale distributed training and serving.
• System Integration: Continuously integrate and validate modules for runtime correctness, memory efficiency, and scalability across multi-node GPU/accelerator clusters.
• Profiling & Performance Analysis: Conduct detailed profiling of compilation graphs, training workloads, and runtime execution to optimize performance and eliminate bottlenecks.
• Debugging & Reliability: Troubleshoot complex hardware–software interaction issues, including vLLM compilation failures on ROCm, CUDA memory leaks, distributed runtime failures, and kernel-level inconsistencies.
• Collaborate with research, infrastructure, and kernel engineering teams to improve system throughput, stability, and developer experience.

IDEAL CANDIDATE PROFILE

• 5+ years of industry experience in ML systems, distributed training, or related fields.
• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or related technical fields.
• Strong programming experience in Python, C++, and familiarity with ML tooling and distributed systems.
• Deep understanding of profiling tools (e.g., Nsight, ROCm Profiler, XLA profiler, TPU tools).