Backend Engineer - AI Gateway Developer

P4N Solutions; Town of Texas (WI); On-site; USD 150,000 - 210,000; Full time

Job summary

P4N Solutions is recruiting an AI Gateway Backend Engineer to build a high‑performance, secure intermediary layer that routes, inspects, and governs traffic between enterprise apps and LLM/AI providers.

The role emphasizes async Python, FastAPI, containerization, and cloud infra, with experience in streaming data, security, and scalable architectures. On-site opportunity with enterprise-scale challenges.

Qualifications

• Experience building high-performance gateway services with low-latency networking.
• Proficient in async Python and modern API frameworks.
• Hands-on with cloud infrastructure, containers, and Kubernetes.

Responsibilities

• Design and operate a scalable AI gateway handling real-time streaming traffic.
• Implement secure, policy-driven routing and inspection between apps and AI providers.
• Develop REST/Streaming APIs with FastAPI and maintain CI/CD pipelines.

Skills

Async Python

Network Programming

HTTP/2 & HTTP/3

gRPC

FastAPI

Kubernetes

Redis Streams

Kafka /RabbitMQ

PostgreSQL

Vault

Terraform

Envoy / Istio

Tools

Docker

AKS/EKS/GKE

Pydantic

TLS / mTLS

PgBouncer

Canary / Blue-Green Deployments

Envoy

Job description

Job Description

We are seeking a highly skilled AI Gateway Backend Engineer to join our team and help build a next-generation AI Gateway platform. This role will focus on designing and operating a high-performance, secure, and scalable intermediary layer that manages, routes, inspects, and governs traffic between enterprise applications and leading LLM/AI service providers.

Position Summary:

We are building an AI Gateway — a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale.

Key Responsibilities:

• Async Python & Network Programming (3 + yrs) – Strong hands‑on experience with asyncio, aiohttp/anyio, low‑level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection‑pooling, keep‑alive, and zero‑copy buffering
• Streaming & Large‑Payload Handling – Ability to manage chunked transfer, bidirectional streaming, back‑pressure and high‑throughput data paths for real‑time LLM inference
• Protocol Knowledge – Working familiarity with HTTP/2 & HTTP/3 (multiplexing, HPACK), gRPC, and occasional custom protocol parsers
• FastAPI‑Based Backend (3 + yrs) – Design, develop and test production‑grade REST/Streaming APIs; pragmatic use of Pydantic, dependency injection and OpenAPI docs
• Containerization & Kubernetes – Docker, AKS/EKS/GKE deployment experience; HPA/VPA scaling, rolling updates, and service‑mesh (e.g., Istio/Envoy) basics
• Async Messaging & Event‑Driven Design – Production use of Redis Streams, RabbitMQ or Kafka for decoupled task queues, retries and back‑off logic
• PostgreSQL & Vault – Advanced query tuning, partitioning, PgBouncer pooling, plus HashiCorp Vault (dynamic secrets, transit encryption) for key management
• Cloud Networking & Security – L4/L7 load balancing, TLS termination, reverse‑proxy (NGINX/Envoy), mTLS between services, and automated cert lifecycle (ACME/Let’s Encrypt)
• IaC & CI/CD – Terraform modules for multi‑env infra, plus GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue‑green deployments

Requirements/Qualifications:

• Enterprise Security Gateways – Integration with CASB/DLP/WSS or similar edge‑security platforms; centralized logging & audit trails
• IAM & SSO – Configuring Azure AD/Entra ID, Okta, SAML 2.0 or OIDC SSO flows; managing Conditional Access & SCIM provisioning
• AI/LLM Pipelines – Building prompt‑engineering workflows, PII anonymization, and monitoring Azure OpenAI (or comparable) usage, rate limits & cost
• API‑Gateway Operations – Custom webhook/REST integrations, circuit‑breaker patterns, rate‑limiting, auto‑retry and health‑checking
• Observability & FinOps – End‑to‑end monitoring with Datadog, Prometheus/Grafana or Azure Monitor; proactive cost‑optimization and 24/7 incident response (SLA/SLO)

Company Description

P4N Solutions is a full‑service recruiting and staffing agency partnering with globally leading companies to connect them with top talent. We specialise in sourcing, placing, and supporting highly qualified candidates across a range of industries, helping both clients and candidates achieve long‑term success. Beyond recruitment, we provide end‑to‑end workforce support, including onboarding coordination, contractor management, payroll processing, workers’ compensation administration, and ongoing employee engagement through regular check‑ins and follow‑ups. We also manage client contractor portals to ensure accurate updates on pay rates, billing, and employment status. At People4Net, Inc, we are committed to delivering efficient, reliable, and people‑focused staffing solutions that drive business growth and create meaningful career opportunities.