Platform Engineer - ML Data Pipelines

Humyn Labs; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Humyn Labs in Bengaluru is seeking an experienced Infrastructure/Platform engineer to own production systems for large-scale AI data pipelines. You will containerize research prototypes, design end-to-end workflows with AWS Batch and Step Functions, and manage Terraform-based infrastructure.

Your focus is cost-aware, reliable, and scalable systems that process thousands of hours of data. You will work closely with researchers and production teams, implementing observability, retries, and

Qualifications

• 5+ years of infrastructure or platform experience with real ownership of production systems.
• Strong Python coding skills for production pipelines.
• Deep AWS experience across compute, networking, IAM, and storage, with cost awareness.
• Hands-on Docker/containerization and orchestration (ECS/EKS).
• Terraform or IaC experience in a team across environments.

Responsibilities

• Productionize research prototypes into reproducible Docker images and pipelines.
• Own orchestration with Step Functions, AWS Batch, and Lambda glue.
• Maintain Terraform IaC, per-environment stacks, networking, and security.
• Drive cost efficiency and optimize GPU/instance usage.
• Build reliable pipelines with logging, metrics, and data-quality gates.
• Manage a fleet of GPU images, ensure compatibility and fast rebuilds.
• Collaborate with researchers to move from prototype to production.

Tools

Python

Docker

Terraform

AWS

ECS/EKS

Go/Rust

CI/CD

Linux

SQL/NoSQL

Monitoring

Job description

Humyn Labs builds the intelligence layer for physical-world AI — systems that perceive, reason, and act in real environments. Our work sits at the intersection of egocentric video understanding, embodied AI, robotics perception, and voice-driven interaction. We move fast, obsess over data quality, and ship at scale.

Humyn Labs converts human action - across sound, sight, movement, and touch - into high-quality multi-modal data signals for physical AI. Operating across 20+ countries in India, southeast Asia, Latin America, and the Middle East: the real-world environments where physical AI deploys, not the labs where it is built.

Our data isn't just collected; it's evaluated, defended, and production-ready. Because before AI can be trusted, its training data must be.

THE OPPORTUNITY

We are building the production data engine behind large-scale egocentric (first-person) data collection for robotics and embodied-AI research. Raw multimodal recordings - stereo video, depth, IMU, hand and camera pose - flow off hardware in the field, through GPU-heavy ML labelling stages, and out as clean, training-ready datasets consumed by model teams.

The pipeline is real and running today: a multi-stage system on AWS where every stage -sensor normalization, metric stereo depth, monocular depth, depth fusion, 3D hand-pose labelling, visual-inertial odometry, ML quality validation, LLM-based metadata passes - runs as a containerized GPU workload, orchestrated end to end by Step Functions and AWS Batch, with all infrastructure defined in Terraform.

Your job is to own this platform end to end - and, most importantly, to be the bridge between

researchers hand you a prototype that works on one machine, on one clip, with a hand-tuned environment. You turn it into a versioned, containerized, orchestrated, cost-efficient pipeline stage that reliably processes thousands of hours of data. That gap - from "it works in a notebook" to "it ran on 10,000 clips last night, unattended, at the lowest possible cost" - is the job.

WHAT YOU'LL OWN

• Productionize research prototypes. Take model code from researchers (PyTorch, CUDA, exotic dependency stacks) and deliver production pipeline stages: reproducible Docker images, pinned GPU/CUDA/cuDNN environments, clean I/O contracts, retries, idempotency, and observability. Real examples of this work from our pipeline:

  • A researcher's stereo depth-estimation model → a GPU Batch stage with a locked CUDA base image, standardized S3 input/output layout, and per-clip cost tracking.
  • A fused MediaPipe + WiLoR hand-pose prototype → a single orchestrated labelling stage with well-defined intermediate artifacts and failure-isolation per clip.
  • A monocular depth model + a stereo model → a three-stage fuse pipeline where resolution-alignment invariants are enforced by the platform, not by tribal knowledge.

• Own the orchestration layer. Design and evolve Step Functions state machines, AWS Batch compute environments and job queues, Lambda glue, and selective stage re-execution (re-run just one stage across a fleet of clips without redoing everything).

• Own the infrastructure as code. All of it lives in Terraform — modules, per-environment stacks, ECR, IAM, networking. You'll extend and harden this, not click around a console.

• Drive cost efficiency as a first-class feature. Spot capacity strategies, right-sizing GPU instance families, eliminating GPU idle time (we've measured it, we hunt it), storage lifecycle policies on multi-TB S3 datasets, batching strategies that keep expensive GPUs saturated.

• Make it reliable at scale. Structured logging, metrics, and alerting across stages; dead-letter handling and automatic retries for flaky clips; data-quality gates so bad inputs fail fast and loudly instead of silently poisoning downstream datasets.

• Manage the container fleet. A dozen-plus GPU images with heavy, conflicting ML dependencies. Keep builds fast, images slim, CUDA stacks consistent, and breakage (e.g., an upstream wheel disappearing from an index) fixed within hours, not weeks.

• Move fast with researchers. Sit close to the research loop: prototype, deploy to staging, run on real fleet data, iterate on feedback, promote to production.

WHAT WE'RE LOOKING FOR

• 5+ years in infrastructure, platform — with real ownership of production systems

• Strong Python. Not just scripting: you write clean, tested, maintainable pipeline and tooling code that other engineers build on

• Deep AWS experience — compute, networking, IAM, storage — and strong opinions about cost

• A track record of taking rough prototypes (ideally ML/research code) to production

• Hands-on Docker/containerization depth: you debug CUDA base-image conflicts and dependency hell without flinching; ECS/EKS or other orchestration experience

• Terraform (or equivalent IaC) used seriously, in a team, across environments

• Working GPU knowledge: what saturates a GPU, what leaves it idle, how instance choice and batching change the bill

• Cost-optimization instinct: spot strategies, right-sizing, storage tiering, and the discipline to measure before and after

• Systems thinking and bias to ship: you'd rather run it on real data today and iterate than perfect it in isolation

Our Stack

• Languages: Python (primary — you must be genuinely strong here), Bash; Go/Rust a plus (ECS/EKS experience welcome)

• IaC: Terraform (modules, multi-environment) families on AWS, spot vs. on-demand economics

• Data: S3 at multi-TB scale, structured artifact layouts, dataset versioning, high-throughput transfer

• Observability: CloudWatch logs/metrics/alarms, cost attribution and reporting

Nice to Have

• Experience with ML labelling/inference pipelines, video or multimodal sensor data

• Exposure to computer-vision workloads (depth estimation, pose estimation, SLAM/VIO)

• EKS/Kubernetes at scale; Ray or other distributed-compute frameworks

• CI/CD for container-heavy repos (CodeBuild, GitHub Actions)