AI Gateway Backend Engineer: Real-Time, Secure Routing

P4N Solutions; Texas; On-site; USD 150,000 - 210,000; Full time

Job summary

P4N Solutions is recruiting an AI Gateway Backend Engineer to build a high‑performance, secure intermediary layer that routes, inspects, and governs traffic between enterprise apps and LLM/AI providers.

The role emphasizes async Python, FastAPI, containerization, and cloud infra, with experience in streaming data, security, and scalable architectures. On-site opportunity with enterprise-scale challenges.

Qualifications

• Experience building high-performance gateway services with low-latency networking.
• Proficient in async Python and modern API frameworks.
• Hands-on with cloud infrastructure, containers, and Kubernetes.

Responsibilities

• Design and operate a scalable AI gateway handling real-time streaming traffic.
• Implement secure, policy-driven routing and inspection between apps and AI providers.
• Develop REST/Streaming APIs with FastAPI and maintain CI/CD pipelines.

Skills

Async Python

Network Programming

HTTP/2 & HTTP/3

gRPC

FastAPI

Kubernetes

Redis Streams

Kafka /RabbitMQ

PostgreSQL

Vault

Terraform

Envoy / Istio

Tools

Docker

AKS/EKS/GKE

Pydantic

TLS / mTLS

PgBouncer

Canary / Blue-Green Deployments

Envoy

Job description

Job Description

Job Description

We are seeking a highly skilled AI Gateway Backend Engineer to join our team and help build a next-generation AI Gateway platform. This role will focus on designing and operating a high-performance, secure, and scalable intermediary layer that manages, routes, inspects, and governs traffic between enterprise applications and leading LLM/AI service providers.

Position Summary:

We are building an AI Gateway — a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale.

Key Responsibilities:

• Async Python & Network Programming (3 + yrs) – Strong hands‐on experience with asyncio, aiohttp/anyio, low‐level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection‐pooling, keep‐alive, and zero‐copy buffering
• Streaming & Large‐Payload Handling – Ability to manage chunked transfer, bidirectional streaming, back‐pressure and high‐throughput data paths for real‐time LLM inference
• Protocol Knowledge – Working familiarity with HTTP/2 & HTTP/3 (multiplexing, HPACK), gRPC, and occasional custom protocol parsers
• FastAPI‐Based Backend (3 + yrs) – Design, develop and test production‐grade REST/Streaming APIs; pragmatic use of Pydantic, dependency injection and OpenAPI docs
• Containerization & Kubernetes – Docker, AKS/EKS/GKE deployment experience; HPA/VPA scaling, rolling updates, and service‐mesh (e.g., Istio/Envoy) basics
• Async Messaging & Event‐Driven Design – Production use of Redis Streams, RabbitMQ or Kafka for decoupled task queues, retries and back‐off logic
• PostgreSQL & Vault – Advanced query tuning, partitioning, PgBouncer pooling, plus HashiCorp Vault (dynamic secrets, transit encryption) for key management
• Cloud Networking & Security – L4/L7 load balancing, TLS termination, reverse‐proxy (NGINX/Envoy), mTLS between services, and automated cert lifecycle (ACME/Let’s Encrypt)
• IaC & CI/CD – Terraform modules for multi‐env infra, plus GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue‐green deployments

Requirements/Qualifications:

• Enterprise Security Gateways – Integration with CASB/DLP/WSS or similar edge‐security platforms; centralized logging & audit trails
• IAM & SSO – Configuring Azure AD/Entra ID, Okta, SAML 2.0 or OIDC SSO flows; managing Conditional Access & SCIM provisioning
• AI/LLM Pipelines – Building prompt‐engineering workflows, PII anonymization, and monitoring Azure OpenAI (or comparable) usage, rate limits & cost
• API‐Gateway Operations – Custom webhook/REST integrations, circuit‐breaker patterns, rate‐limiting, auto‐retry and health‐checking
• Observability & FinOps – End‐to‐end monitoring with Datadog, Prometheus/Grafana or Azure Monitor; proactive cost‐optimization and 24/7 incident response (SLA/SLO)

Company Description

P4N Solutions is a full-service recruiting and staffing agency partnering with globally leading companies to connect them with top talent. We specialize in sourcing, placing, and supporting highly qualified candidates across a range of industries, helping both clients and candidates achieve long-term success.
Beyond recruitment, we provide end-to-end workforce support, including onboarding coordination, contractor management, payroll processing, workers’ compensation administration, and ongoing employee engagement through regular check-ins and follow-ups. We also manage client contractor portals to ensure accurate updates on pay rates, billing, and employment status.
At People4Net, Inc, we are committed to delivering efficient, reliable, and people-focused staffing solutions that drive business growth and create meaningful career opportunities.

Company Description

P4N Solutions is a full-service recruiting and staffing agency partnering with globally leading companies to connect them with top talent. We specialize in sourcing, placing, and supporting highly qualified candidates across a range of industries, helping both clients and candidates achieve long-term success.
Beyond recruitment