AI Systems Architect: Agentic LLMs & Cloud

ApTask; New York (NY); On-site; USD 170,000 - 200,000; Full time

Job summary

ApTask is seeking an experienced AI engineer to design and build agentic systems in a NY-area role. You will lead architecture for tool-calling agents, retrieval and reasoning pipelines, and secure action execution with least-privilege access.

You will productionize LLM applications, own data pipelines and backend services, and drive reliability, governance, and cost optimization across distributed systems. 8–14 years of software engineering, strong Python, and cloud experience are expected.

Qualifications

• 8–14 years of software engineering experience with strong hands-on Python.
• Strong depth in at least one systems language — Go, Rust, Java, or C/C++ — and knowing when to reach for it.
• Strong data structures and algorithms.
• Strong understanding of APIs, microservices and system design.
• Hands-on experience building and operating data pipelines and production-grade distributed systems.
• 2+ years of hands-on LLM engineering with agentic systems
• Production experience with agent frameworks — LangGraph, Google ADK, CrewAI, Claude Agent SDK, or equivalent
• Experience building MCP servers and tool-calling interfaces
• RAG from first principles: chunking strategy, embeddings, vector and hybrid retrieval, reranking, and response validation
• Strong experience with vector databases (Milvus, Pinecone, Weaviate, FAISS) or cloud equivalents
• Design of guardrails and reliability patterns — validators, policy checks, self-correction loops, deterministic fallbacks, circuit breakers, and rollback paths
• Deep familiarity with token optimization and context-window management
• Latency and cost optimization via caching, routing, batching, streaming, and parallel tool calls
• Performance testing and tuning against defined SLOs
• Experience building evaluation frameworks for LLM systems
• Instrumentation and traceability for regulated environments using LangSmith, Langfuse, etc.
• Hands‑on AWS: ECS/EKS, Lambda, S3, DynamoDB, Redshift; Azure/GCP equivalents valued
• Familiarity with CI/CD pipelines and DevOps
• Infrastructure as code with Terraform or CloudFormation, mature CI/CD

Responsibilities

• Design and build agentic systems: architecture and implementation of tool‑calling agents with retrieval, reasoning, and secure action execution.
• Productionize LLM applications: develop retrieval pipelines, prompt synthesis, validation, and self‑correction loops with evaluation.
• Own the full stack: data pipelines, backend services, distributed compute, orchestration for agentic systems.
• Engineer for reliability and governance: validators, adversarial tests, policy checks; deterministic fallbacks and rollback strategies; instrument evaluation.
• Optimize for cost and latency: improve token efficiency, response time, and unit economics against SLOs.
• Codebase ownership: build and maintain high‑quality Python and SQL, scalable components.
• Cloud integration: deploy AI apps on AWS, Azure, or GCP with robust CI/CD.
• Cross‑functional collaboration: work with product owners, data scientists, and SMEs to define requirements and deliver AI products.
• Mentoring and technical leadership: set standards and share knowledge across the team.

Skills

Analytical problem solving

Cross-team communication

Ambiguity tolerance

Tools

LLM engineering

Agent frameworks

MCP servers

RAG pipelines

Vector databases

Guardrails design

AWS (ECS/EKS)

Terraform/CloudFormation

CI/CD

Job description

ApTask is seeking an experienced AI engineer to design and build agentic systems in a NY-area role. You will lead architecture for tool-calling agents, retrieval and reasoning pipelines, and secure action execution with least-privilege access.

You will productionize LLM applications, own data pipelines and backend services, and drive reliability, governance, and cost optimization across distributed systems. 8–14 years of software engineering, strong Python, and cloud experience are expected.