Senior ML Infra Engineer – ML Platform & GPU Ops

Prudence Holdings; New York (NY); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Equity package

Medical, dental, and vision coverage

Agentic tooling budget

Luncheons and dinners provided

Job summary

Rebar, a startup reshaping AI-driven construction workflows, is seeking a Senior ML Infrastructure Engineer to own and expand the ML infrastructure platform used by our ML engineers. You will build scalable services, manage model registries, and drive production-grade pipelines for training, evaluation, deployment, and monitoring.

You should be obsessed with developer experience, have strong Python/PyTorch skills, and bring a track record of optimizing ML workloads at scale.

Qualifications

• 6+ years of experience building production backend systems, with significant time on internal developer platforms or ML platforms.
• Strong Python and PyTorch, including profiling and debugging below the model code.
• 3+ years of experience with cloud infrastructure (AWS preferred).
• Hands-on GPU performance and inference work with production optimizations (batching, mixed precision, quantization).
• Experience managing a model zoo / model registry - versioning, promotion, governance from experiment to production.

Responsibilities

• Platform and Developer Experience: build services as the single front door to our ML platform.
• Model and Data Governance: instrument active learning pipelines, retrainings, model registry.
• Infrastructure Consolidation and Integration: smooth GPU inference and cloud stack.
• Observability and Operations: extend DataDog and ensure clear GPU util metrics.
• Collaboration and Roadmap: work with ML engineers to turn scripts into self-serve platform features and guide architecture.

Skills

Python

PyTorch

AWS

GPU performance

Model registry

Tools

ONNX Runtime

TensorRT

Torch.compile

Job description

Rebar, a startup reshaping AI-driven construction workflows, is seeking a Senior ML Infrastructure Engineer to own and expand the ML infrastructure platform used by our ML engineers. You will build scalable services, manage model registries, and drive production-grade pipelines for training, evaluation, deployment, and monitoring.

You should be obsessed with developer experience, have strong Python/PyTorch skills, and bring a track record of optimizing ML workloads at scale.