Inference Platform Engineer

Coreweave; Sunnyvale (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401(k) with generous match

Flexible PTO and remote-friendly

Equity awards

Discretionary bonus

Job summary

CoreWeave is seeking a Junior Software Engineer on the Inference team to own a production model-serving service end-to-end, including design, testing, deployment, and operations. You will enhance observability, tune latency and throughput, and participate in on-call duties with peers across runtime, scheduling, and networking.

Ideal candidates have 0–3 years in cloud/distributed services, know Python or Go, and understand Kubernetes and CI/CD.

Qualifications

• 0–3 years software engineering experience building cloud/distributed services in production.
• Proficiency in Python or Go; familiarity with gRPC/HTTP APIs and containerized deployments.
• Working knowledge of Kubernetes (or Slurm/Ray) and CI/CD pipelines.
• Experience with metrics/alerting (Prometheus/Grafana/OpenTelemetry) and structured logging.
• Basic understanding of GPU inference concepts (KV caching, mixed precision, streaming tokens).

Responsibilities

• Own one model-serving service end-to-end: design, implementation, tests, deployment, and ops.
• Add and tune observability (dashboards, alerts, traces); automate runbook steps to reduce toil.
• Improve latency/throughput via micro-batching, concurrency tuning, and caching.
• Participate in on-call; lead post-incident action items for your service.
• Review peers’ PRs and contribute design feedback in team reviews.

Job description

CoreWeave is seeking a Junior Software Engineer on the Inference team to own a production model-serving service end-to-end, including design, testing, deployment, and operations. You will enhance observability, tune latency and throughput, and participate in on-call duties with peers across runtime, scheduling, and networking.

Ideal candidates have 0–3 years in cloud/distributed services, know Python or Go, and understand Kubernetes and CI/CD.