Member of Technical Staff | Inference Platform

Avra; São Paulo; Vor Ort; BRL 300.000 - 520.000; Vollzeit

Zusammenfassung

Avra in Brazil seeks a Platform Engineer to own model serving and large-scale batch compute on Kubernetes. You will manage efficient inference, scalable batch jobs, and robust deployments across cloud and customer dataplans.

You’ll optimize training and inference pipelines, develop controllers, and focus on cost-aware performance for enterprise and mid-market customers. This role blends cloud-native ops with production-grade software ownership.

Qualifikationen

• Experience running model serving or large-scale batch compute on Kubernetes (production-grade).
• Experience building Kubernetes controllers or operators.
• Skill at profiling and optimizing data-heavy Python pipelines.
• Strong sense of cost and efficiency in compute.
• Production-quality code and reviews, with ownership of what you build.

Aufgaben

• Evolve inference runtime on Kubernetes.
• Run large batch inference on ephemer jobs with multi-dimensional admission control (CPU, memory, GPU).
• Build and extend the controller and its Kubernetes custom resources.
• Optimize each model's inference engine and feature processing.
• Serve graphs and data efficiently.
• Own execution of training, post-training, and fine-tuning jobs in cloud and on customer dataplanes/on-premise cloud.
• Drive autoscaling, GPU serving, performance, and cost optimization with telemetry.

Kenntnisse

Kubernetes serving

Kubernetes controllers

Python data pipelines

Cost awareness

Production-grade code

Tools

Kubernetes

Python

Ray Serve

Jobbeschreibung

About The Role At Avra, every technical IC is a Member of Technical Staff (MTS). The title doesn't put anyone in a silo: you own systems and outcomes, not steps in a function, and you keep building depth in your area.

About The Role At Avra, every technical IC is a Member of Technical Staff (MTS). The title doesn't put anyone in a silo: you own systems and outcomes, not steps in a function, and you keep building depth in your area.

In this role, you'll join the Platform team to own where our models execute. Customers consume our models through large batches of millions of records and through real-time APIs, and they make business decisions on every response. You'll run governed model releases reliably and efficiently — in our cloud and on customer-hosted Kubernetes — and make inference fast, predictable, and cheap enough to serve both enterprise and mid-market customers.

What you'll do

• Evolve Sophos, our online and batch inference runtime, built on Kubernetes.
• Run large batch inference on ephemer jobs, with multi-dimensional admission control (CPU, memory, GPU).
• Build and extend the controller and its Kubernetes custom resources.
• Optimize each model's inference engine and feature processing.
• Serve graphs and data efficiently.
• Own execution of training, post-training, and fine-tuning jobs, in our cloud and in customer dataplanes / on-premisse cloud.
• Drive autoscaling, GPU serving, performance, and cost optimization, with telemetry for every model we run.

How we measure success

• 99.9% serving availability.
• p95/p99 latency for online inference and throughput for batch.
• Cost per prediction and per training job.
• GPU utilization: paid capacity versus capacity actually used.
• Training and batch jobs that finish on time and succeed without manual retries.

What we're looking for

• Experience running model serving or large-scale batch compute on Kubernetes.
• Experience building Kubernetes controllers or operators.
• Skill at profiling and optimizing data-heavy Python pipelines.
• A clear sense of cost: you treat compute efficiency as a product feature.
• Production-quality code and reviews, and a willingness to operate what you build.

Nice to have

• Ray, Ray Serve, or KubeRay in production.
• Admission-control systems.
• GPU serving and performance optimization.
• Arrow, Parquet, Lance, or other columnar formats.
• Shipping software to customer-hosted Kubernetes.
• GCP/AWS and GKE/EKS, and financial services or regulated environments.