Senior Platform Engineer – Inference API & Streaming

General Compute Inc.; New York (NY); On-site; USD 130,000 - 160,000; Full time

Job summary

General Compute Inc. is seeking a Senior IC Engineer to develop the platform layer of their inference cloud, focusing on the OpenAI-compatible API. This role requires managing complex routing and optimization tasks while ensuring high performance and reliability.

The ideal candidate will have at least 5 years of experience in building production systems, with specific expertise in high-throughput APIs and distributed systems.

Qualifications

• 5+ years writing production systems code.
• Experience operating a high-throughput API service.
• Strong skills in concurrency and distributed systems.

Responsibilities

• Own the OpenAI-compatible API surface and manage latency.
• Build the request path that fronts the ASIC fleet.
• Integrate with OpenRouter and manage billing hooks.

Skills

Production systems code

High-throughput API service

Concurrency and memory fundamentals

LLM inference knowledge

Profiler usage

Self-directed work

Tools

CUDA

Triton

Job description

General Compute Inc. is seeking a Senior IC Engineer to develop the platform layer of their inference cloud, focusing on the OpenAI-compatible API. This role requires managing complex routing and optimization tasks while ensuring high performance and reliability.

The ideal candidate will have at least 5 years of experience in building production systems, with specific expertise in high-throughput APIs and distributed systems.