Senior AI Platform Architect

Boston Consulting Group; New York (NY); On-site; USD 210,000 - 260,000; Full time

Benefits offered by this job

Zero dollar health insurance premiums

Generous retirement contributions

Paid parental leave

Paid time off

Job summary

Boston Consulting Group is seeking a senior AI Architecture lead to guide the design of scalable AI and LLM-based solutions, across enterprise environments. You will architect memory-enabled agentic systems, RAG integrations, and cross-functional deployments to deliver measurable business value.

You will work with product managers, data scientists, and engineers to translate strategic goals into robust AI platforms, mentoring teams and shaping governance.

Qualifications

• 10+ years of technology consulting or software engineering experience.
• 5+ years in architecture discipline.
• Strong foundation in software architecture and distributed systems design.
• Proficiency in Python, JavaScript, TensorFlow, Keras, PyTorch, TypeScript, or Node.js.
• Familiarity with LangChain, LangGraph, Google ADK, Strands, Semantic Kernel, AutoGen, CrewAI, LlamaIndex.

Responsibilities

• Define system architectures for AI and LLM-based solutions, ensuring scalability, low-latency inference, and modularity.
• Design agentic AI solutions to optimize business workflow automation, including memory, agentic reflection, and task planning.
• Design AI integration patterns, such as RAG, for model orchestration, context management, prompt orchestration, and external system connectivity.
• Establish architectural blueprints, design principles, and governance frameworks for model versioning, data lineage and responsible AI solutions.
• Architect polyglot model runtimes and hybrid pipelines with attention to cost, latency, and risk.

Skills

Technology Architecture

AI Platforms

Python

LLMOps / MLOps

Cloud Architecture

RAG / Agentic AI

Kubernetes

Distributed Systems

Education

Bachelor's degree in IT / CS

Tools

LangChain

LangGraph

Google ADK

Semantic Kernel

LangSmith

Prometheus

Grafana

OpenTelemetry

Docker

Kubernetes

Job description

Boston Consulting Group is seeking a senior AI Architecture lead to guide the design of scalable AI and LLM-based solutions, across enterprise environments. You will architect memory-enabled agentic systems, RAG integrations, and cross-functional deployments to deliver measurable business value.

You will work with product managers, data scientists, and engineers to translate strategic goals into robust AI platforms, mentoring teams and shaping governance.