Backend Engineer (LLM inference)
Aleroy
A complete application in a minute
Tailored resume and cover letter, ready to send.
Benefits offered by this job
Job summary
Aleroy is seeking a Senior Backend Engineer to own the architecture and evolution of our core backend services. This is a hands-on role with real technical ownership across Go, TypeScript/Node.js, and Python.
You will design and maintain OpenAI-compatible APIs, build distributed systems that scale, and ensure high performance with PostgreSQL, Redis, and S3-compatible storage. Come join a small, senior team in Abu Dhabi delivering AI infrastructure.
Qualifications
- 5+ years of professional backend engineering experience.
- Strong commercial experience with both Go and TypeScript/JavaScript (Node.js); Python is a plus.
- Proven experience building and operating distributed systems, with an understanding of consistency, fault tolerance, and service-to-service communication.
- Deep understanding of concurrency models and how they work under the hood: Go's goroutines, channels, and scheduler, and Node.js's event loop and async/await model, including the trade-offs between them.
- Strong database knowledge: schema design, indexing, query optimization, transactions and isolation levels, and locking, primarily in PostgreSQL.
- Experience with Redis, Valkey, or similar in-memory stores.
- Experience building backend systems that serve LLM inference, or a strong understanding of the request lifecycle, token streaming, and long-running requests.
- Experience building streaming APIs (SSE/WebSockets), including cancellation and client disconnect handling.
- Understanding of OpenAI-compatible API design and its streaming and error semantics.
- Experience with request routing, load balancing, backpressure, and admission control for workloads where request cost varies widely.
- Hands-on experience designing REST and/or gRPC APIs.
- Experience building auth systems with JWT, Auth0, or similar.
- Track record of leading the design of systems or major features end to end, including writing design docs and driving decisions across teams.
- Experience debugging and resolving complex production issues in systems you owned.
- Ability to make trade-offs between speed, reliability, and maintainability, and to explain them clearly.
Responsibilities
- Design, build, and maintain backend services in Go and TypeScript/JavaScript (Node.js), plus Python where it fits.
- Build and scale distributed systems that stay reliable as traffic and complexity grow.
- Own features end to end, from design docs and implementation through testing and release.
- Identify tech debt, bottlenecks, and reliability risks before they become problems.
- Design and maintain OpenAI-compatible APIs for chat completions, streaming, and tool calls.
- Build token streaming over SSE/WebSockets with proper cancellation, disconnect handling, and partial responses.
- Build request routing and load balancing across inference backends by model, capacity, latency, and region.
- Implement backpressure, queueing, and admission control for variable-cost LLM workloads.
- Design REST and gRPC interfaces between internal services.
- Design and optimize data models in PostgreSQL.
- Use Redis/Valkey for caching, state, and high-performance workloads.
- Integrate with S3-compatible storage and Cloudflare R2.
- Build authentication and authorization flows with Auth0 and JWT.
- Build token-based rate limiting and usage metering for multi-tenant APIs.
- Build reliable third-party integrations, including Stripe payments, with retries, idempotency, and proper failure handling.
- Profile and optimize services for latency, throughput, and resource usage.
- Implement caching, concurrency, rate limiting, and timeout strategies.
- Write automated tests using Go testing tools, Jest/Supertest, pytest, and Testcontainers.
- Mentor engineers and raise the bar through code reviews and design discussions.
Skills
Tools
Job description
Aleroy is building a unified platform for high-performance AI inference, making it easier for companies to deploy and run AI models at scale.
Through a single API, Aleroy provides access to ready-to-use and custom AI models across cloud infrastructure, dedicated capacity, and customers' own hardware. Our platform optimizes performance, cost, and deployment flexibility while giving enterprises control over where and how their AI workloads run.
We work in small teams, collaborate directly, and focus on solving complex engineering challenges to build reliable, scalable AI infrastructure.
We’re looking for a Senior Backend Engineer to own the architecture and evolution of our core backend services. This is a hands-on role with real technical ownership. You’ll turn ambiguous problems into clear technical plans, make architectural decisions, and take responsibility for what happens after your code ships.
Backend Architecture & Development
Design, build, and maintain backend services in Go and TypeScript/JavaScript (Node.js), plus Python where it fits.
Build and scale distributed systems that stay reliable as traffic and complexity grow.
Own features end to end, from design docs and implementation through testing and release.
Identify tech debt, bottlenecks, and reliability risks before they become problems.
LLM Serving & APIs
Design and maintain OpenAI-compatible APIs for chat completions, streaming, and tool calls.
Build token streaming over SSE/WebSockets with proper cancellation, disconnect handling, and partial responses.
Build request routing and load balancing across inference backends by model, capacity, latency, and region.
Implement backpressure, queueing, and admission control for variable-cost LLM workloads.
Design REST and gRPC interfaces between internal services.
Data & Storage
Design and optimize data models in PostgreSQL.
Use Redis/Valkey for caching, state, and high-performance workloads.
Integrate with S3-compatible storage and Cloudflare R2.
Auth, Billing & Integrations
Build authentication and authorization flows with Auth0 and JWT.
Build token-based rate limiting and usage metering for multi-tenant APIs.
Build reliable third-party integrations, including Stripe payments, with retries, idempotency, and proper failure handling.
Performance & Quality
Profile and optimize services for latency (TTFT, inter-token latency), throughput, and resource usage.
Implement caching, concurrency, rate limiting, and timeout strategies.
Write automated tests using Go testing tools, Jest/Supertest, pytest, and Testcontainers.
Mentor engineers and raise the bar through code reviews and design discussions.
5+ years of professional backend engineering experience.
Strong commercial experience with both Go and TypeScript/JavaScript (Node.js); Python is a plus.
Proven experience building and operating distributed systems, with an understanding of consistency, fault tolerance, and service-to-service communication.
Deep understanding of concurrency models and how they work under the hood: Go's goroutines, channels, and scheduler, and Node.js's event loop and async/await model, including the trade-offs between them.
Strong database knowledge: schema design, indexing, query optimization, transactions and isolation levels, and locking, primarily in PostgreSQL.
Experience with Redis, Valkey, or similar in-memory stores.
Experience building backend systems that serve LLM inference, or a strong understanding of the request lifecycle, token streaming, and long-running requests.
Experience building streaming APIs (SSE/WebSockets), including cancellation and client disconnect handling.
Understanding of OpenAI-compatible API design and its streaming and error semantics.
Experience with request routing, load balancing, backpressure, and admission control for workloads where request cost varies widely.
Hands-on experience designing REST and/or gRPC APIs.
Experience building auth systems with JWT, Auth0, or similar.
Track record of leading the design of systems or major features end to end, including writing design docs and driving decisions across teams.
Experience debugging and resolving complex production issues in systems you owned.
Ability to make trade-offs between speed, reliability, and maintainability, and to explain them clearly.
Familiarity with inference engines like vLLM, SGLang, or TGI, and concepts like continuous batching and KV cache.
Experience with fair scheduling and tenant isolation on shared inference capacity.
Experience with failover and retries for inference requests without duplicating billing or output.
Experience with usage metering and billing for AI or developer APIs.
Stripe or other payment infrastructure.
S3-compatible storage or Cloudflare R2.
Building developer-facing APIs or platform products.
Real ownership of the backend that powers Aleroy inference platform.
Influence over backend architecture and product direction.
Hands-on work on hard problems in distributed systems, real-time streaming, and LLM serving.
A small, senior team with fast execution and low bureaucracy.
Competitive salary with performance-based incentives.
24 days annual leave plus public holidays.
Health insurance.
Modern office in Yas Creative Hub.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
