Technical Lead, AI Platform

ZealTech; San Jose (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

ZealTech is seeking a Technical Lead to drive the platform's architectural vision and lead a high-performing team. You will contribute code, shape the platform's security, observability, and reliability, and partner across enterprise teams to align with business goals.

The role requires deep expertise in distributed systems, Python, and AI tooling, with hands-on experience deploying LLM-powered agents in production.

Qualifications

• 8+ years of experience architecting and operating production backend or distributed systems.
• 2+ years serving as a Technical Lead or equivalent.
• Bachelor's degree in CS or a related field.
• Hands-on experience deploying LLM-powered applications in production.

Responsibilities

• Drive Platform Architecture & Delivery from concept to production operation.
• Engineer reusable platform primitives with clear interfaces and versioning.
• Hands-on coding, design reviews, and tackling hard engineering roadblocks.
• Set engineering standards for security, observability, and reliability.

Skills

Technical Leadership

Python Proficiency

Distributed Systems

LLM Deployment

API Design

Education

Bachelor's degree in Computer Science or related field

Tools

FastAPI

asyncio

Pydantic

Kafka

LangGraph

Job description

We are engineering the foundational enterprise platform for AI agents, allowing teams to build, deploy, and scale secure, reliable, and highly efficient agentic workflows. From customer-facing tools to internal operations, agents built on our platform support users across diverse products and channels. As Technical Lead, you will drive the platform's architectural vision end-to-end, lead a team of high-performing engineers, and maintain strong hands-on code contributions.

You will be hands-on across the entire platform ecosystem:

Foundational Infrastructure & Security

• Hardening AI Infra & Governance: Building secure, policy-controlled access layers for models, tools, and data.
• Scaling Context & Memory: Engineering high-performance systems to deliver real-time data to agents.

Developer Tools & Quality Assurance

• Architecting the Agent Harness & SDK: Designing the foundational frameworks engineers use to build and run agents.
• Driving Evaluation & Observability: Implementing deep tracking for agent quality, cost, and live behavior.

Product Delivery & User Experience

• Deploying Applied Agents: Leading the practical application and rollout of production agent use cases.
• Crafting AI User Experiences: Defining how agents surface in applications, assistants, and partner integrations.

Key Scope of Work:

Architectural Strategy & Design

• Drive Platform Architecture & Delivery: Own the technical design, roadmap, and delivery of the platform from initial concept through production operation.
• Engineer Reusable Capabilities: Build robust, highly available platform primitives that engineering teams can depend on, enforcing clear interfaces, strict versioning, and thorough documentation.
• Maintain Hands-On Technical Leadership: Write production code, review complex system designs and pull requests, and personally tackle the hardest engineering roadblocks.
• Establish Engineering Excellence: Set and uphold rigorous engineering standards for platform quality, end-to-end security, observability, and evaluation frameworks.

Operational Excellence & Collaboration

• Optimize Production Workloads: Direct continuous improvements to minimize latency, maximize reliability, and reduce total cost for high-scale AI workloads.
• Own Operational Health (Run What You Build): Define production SLOs, participate actively in the on-call rotation, and lead comprehensive post-mortem incident reviews.
• Partner Cross-Functionally: Collaborate closely with enterprise architects, product managers, security teams, and business domains to align the platform with organizational needs.

Qualifications:

Leadership & Core Experience

• Proven Technical Leadership: 8+ years of experience architecting and operating production backend or distributed systems, including at least 2+ years acting as a Technical Lead.
• Educational Foundation: Bachelor’s degree in Computer Science, a related technical field, or equivalent practical experience.
• Production AI Tracking: Hands-on experience successfully deploying LLM-powered applications or agents to production, with a strong understanding of handling edge cases, failures, and mitigation.
• Framework & Tooling Engineering: Advanced Python proficiency combined with a track record of designing robust frameworks, developer SDKs, or platform APIs tailored for other engineering teams.

Foundational Engineering & Communication

• Distributed Systems Mastery: Deep understanding of distributed system fundamentals, including API design, asynchronous/event-driven processing, data streaming, caching strategies, rate limiting, and fault-tolerant architectures.
• Technical Communication: Excellent written communication skills, with proven ability to author comprehensive system design documents that articulate technical trade-offs clearly to both engineering teams and executive leadership.

Technical Capabilities & Domain Expertise:

We are looking for a well-rounded technical leader with broad exposure across these domains and deep expertise in a few key areas:

• Agent Development: Architecting tool/function calling, multi-step and multi-agent orchestration, task delegation, durable execution, and human-in-the-loop approval workflows.
• Agent Interoperability: Designing open protocols to expose agents and tools seamlessly to other applications and external agents.
• AI User Experiences: Engineering streaming agent outputs to web clients and building generative or dynamic user interfaces.

Context Engineering & Lifecycle Management

• Context Engineering: Implementing Retrieval-Augmented Generation (RAG), vector/hybrid search, short- and long-term memory systems, automatic summarization, and routing across extensive tool catalogs.
• LLM Evaluation & Observability: Building evaluation datasets and rubrics, leveraging LLM-as-a-judge, running regression testing in CI, and establishing distributed tracing, logging, metrics, SLOs, and online production evaluation.

Enterprise Platform, Security & Governance

• LLM Infrastructure: Managing cross-provider model routing, token consumption, cost optimization, response caching, and latency reduction.
• AI Safety & Governance: Deploying prompt-injection defenses, PII detection/redaction, guardrails, policy engines, and sandboxed code execution environments.
• Core & Multi-Tenant Security: Designing authentication/authorization, service-to-service identity, least-privilege access controls, strict tenant isolation, and detailed audit logging.
• Cloud-Native Infrastructure: Operating containers, container orchestration, robust CI/CD pipelines, and infrastructure as code (IaC).

Core Tech Stack & Toolkit

Expertise in Python is required. For the rest of the stack, we look for hands-on exposure to most of these technologies:

• Languages: Expert-level Python; solid working knowledge of SQL. Java experience is a valuable asset.
• Backend APIs & Streaming: High proficiency in FastAPI, asyncio, and Pydantic; practical experience with REST, gRPC, Server-Sent Events (SSE), WebSockets, and event streaming via Kafka or similar.

AI, Agentic Frameworks & Protocols

• Agent Frameworks & Protocols: Hands-on experience with LangGraph and LangChain, or comparable frameworks (e.g., OpenAI Agents SDK, Google ADK, Pydantic AI) along with agent protocols like MCP, A2A, and A2UI.
• Models, Providers & Gateways: Interfacing with enterprise APIs (OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Azure OpenAI), serving open-weight models via vLLM, and managing LLM gateways like LiteLLM.

Data, Evaluation & Observability

• Data & Retrieval Engines: PostgreSQL with pgvector; vector and search engines (OpenSearch, Elasticsearch, Qdrant); Redis; and embedding/reranking models.
• Evaluation & Telemetry: OpenTelemetry (including GenAI semantic conventions); LLM tracing/eval platforms (LangSmith, Ragas, DeepEval); and standard metrics stacks (Prometheus, Grafana).

Security, Infrastructure & Frontend

• Security & Governance Tools: OAuth 2.0, OIDC, JWT, and workload identities (SPIFFE/SPIRE); policy-as-code (Open Policy Agent, Cedar); secrets management (HashiCorp Vault); and guardrail frameworks (NeMo Guardrails, Llama Guard, Presidio).
• Cloud-Native & Sandboxing: Docker, Kubernetes, Helm, and Terraform; experience with at least one major cloud provider (AWS, Azure, GCP); GitHub Actions or Jenkins; and secure sandbox runtimes (gVisor, Firecracker).
• Frontend Engineering: React; streaming chat interfaces; and generative UI frameworks or specifications (Vercel AI SDK, A2UI, MCP-UI).

Preferred Qualifications:

• Platform Scale & Adoption: Proven experience building internal platforms, developer SDKs, or frameworks that were successfully adopted across multiple engineering teams.
• Enterprise AI Experience: Track record of deploying, running, and managing production AI systems tailored for external enterprise customers.
• Open Source & Ecosystem Impact: Active contributions to open-source agentic frameworks or emerging agent protocols.