BCG Platinion | Principal AI Tech Architect - AI Platforms

Boston Consulting Group (BCG); Nashville (TN); On-site; USD 180,000 - 240,000; Full time

Job summary

Boston Consulting Group (BCG) seeks Principal AI Tech Architects to design and evolve next-generation AI and agentic systems. You will bridge high-level system design with hands-on AI engineering, crafting scalable architectures for AI-driven applications and integrating large language models, data pipelines, and enterprise systems.

You will lead across AI, data, product, and engineering teams to define architecture patterns, governance, and implementation strategies, guiding teams toward

Qualifications

• Extensive experience designing AI architectures for enterprise-scale systems.
• Proven ability to translate business needs into scalable AI solutions.
• Strong collaboration with product, data science, and engineering teams.

Responsibilities

• Define system architectures for AI and LLM-based solutions, ensuring scalability and low latency.
• Develop and validate agentic AI workflows and orchestration patterns.
• Lead design reviews, governance, and best practices across AI programs.

Skills

Collaborative

Systems thinking

Technical expertise

Ambiguity tolerance

Change management

Agile methods

Innovative thinking

Strategic partnering

Team mentorship

Tools

LangChain

LlamaIndex

LangGraph

Semantic Kernel

OpenAI Agentkit

n8n

Job description

Locations: Atlanta | Austin | Boston | Brooklyn | Chicago | Dallas | Denver | Detroit | Houston | Miami | Minneapolis | Nashville | Summit | New York | Philadelphia | Pittsburgh | Durham | Washington

Who We Are

Boston Consulting Group (BCG) is a global consulting firm that partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities. Our success depends on a spirit of deep collaboration and a global community of diverse individuals determined to make the world and each other better every day.

BCG's Tech and Digital Advantage (TDA) practice focuses on helping clients deliver competitive advantage and superior business performance through the use of data, technology and digital. Platinion is the fastest growing practice at BCG and at the heart of the strategic impact we have with our clients. Our consultants and experts globally work across all industries and provide deep experience and expertise across a number of topics including Digital Transformation, Data & Digital Platforms, AI at Scale, Agile, Cybersecurity and Digitizing the Tech Function.

At BCG, we bring together the right people to conquer complexity, drive material change, and initiate positive, long-term impact. Explore our BCG Culture and Values for more information.

About BCG Platinion

BCG Platinion's presence spans across the globe, with offices in Asia, Europe, and South and North America. We achieve digital excellence for clients with sustained solutions to the most complex and time-sensitive challenge. We guide clients into the future to push the status quo, overcome tech limitations, and enable our clients to go further in their digital journeys than what has ever been possible in the past. At BCG Platinion, we deliver business value through the innovative use of technology at a rapid pace. We roll up our sleeves to transform business, revolutionize approaches, satisfy customers, and change the game through Architecture, Cybersecurity, Digital Transformation, Enterprise Application and Risk functions. We balance vision with a pragmatic path to change transforming strategies into leading-edge tech platforms, at scale.

What You'll Do

Principal AI Tech Architects At BCG Platinion Are

• Collaborative. They are interdisciplinary team players who seek alignment and establish relationships ranging from cross-functional stakeholder groups to existing teams.
• Systems thinkers. They are well-versed in designing sophisticated and robust AI solutions to common business problems.
• Technical experts. They are critical thinkers and have extensive Technology & AI expertise that drives novel solutions.
• Comfortable with ambiguity. They know the path forward isn't always well-defined. They are comfortable and confident working through the unknown.
• Change agents. They know how to make change happen across an organization. They can align and onboard teams to implement new processes and toolsets. They embrace complex challenges and guide an organization to optimize technology best practices.
• Agile advocates. They are well-versed in agile methodologies and use their expertise to strategically tackle difficult challenges and implement change.
• Innovative. They are creative thinkers who apply their deep technology architecture and AI Platform expertise to envision novel design patterns for the next generation of technology solutions.
• Knowledgeable. They have a sophisticated understanding of Technology Architecture and AI Platforms, acting as a viable resource on teams as both a hands‑on practitioner and subject matter expert.
• Strategic partners. They work closely with senior client stakeholders, including C-level executives, to build hypothesis-driven solutions at the intersection of technology and business. They bridge the gap between IT and business functions, ensuring technology strategies are aligned with organizational goals to solve critical business challenges.

We are seeking Principal AI Tech Architects to design, guide, and evolve the architecture of next‑generation intelligent agentic systems. This role bridges high‑level system design with hands‑on AI engineering - ideal for someone who can envision complex AI‑enabled ecosystems while still being comfortable prototyping or coding to validate ideas. You’ll work across AI, data, product, and engineering teams to define scalable architectures for AI‑driven applications, enabling the seamless integration of large language models (LLMs), data pipelines, and enterprise systems.

AI Architecture & Solution Design

• Define system architectures for AI and LLM‑based solutions, ensuring scalability, low‑latency inference, and modularity.
• Design Agentic AI solutions to optimize business workflow automation, including the use of memory, agentic reflection, and task planning.
• Design AI integration patterns, such as RAG, for model orchestration, context management, prompt orchestration, and external system connectivity.
• Establish architectural blueprints, design principles, and governance frameworks for model versioning, data lineage and responsible AI solutions across the organization.
• Architect polyglot model runtimes and hybrid pipelines (batch training + streaming inference) with attention to cost, latency, and risk.

Prototyping & Validation

• Develop minimal‑viable prototypes or proof‑of‑concepts to test model architectures, RAG flows, and agent orchestration.
• Evaluate emerging AI frameworks and tools (e.g., LangChain, LlamaIndex, LangGraph, Strands, Google ADK, Semantic Kernel) for suitability and integration.
• Develop solutions using the latest agentic orchestration engines, such as n8n, Retool, OpenAI Agentkit, Microsoft Copilot Studio, and AWS Flows.

Implementation Management & Support

• Support restructuring IT processes and organization to adopt new systems.
• Oversee system performance, observability, and governance as solutions transition from prototype to production using tools such as LangSmith, Prometheus, Grafana, Evidently, WhyLogs, and OpenTelemetry.
• Optimize the application development processes – define standards for versioning, testing, and monitoring of models and AI services (LLMOps).

Communication & Collaboration

• Craft compelling narratives for Agentic reimagination of businesses, translating complex ideas for both technical and non‑technical audiences.
• Deliver impactful presentations, leading modules, presenting findings, and building consensus with clients.
• Build collaboration, mentor teams, and share knowledge to drive collective success – partner with product managers, data scientists, and engineers to translate business goals into robust AI system designs.
• Facilitate technical and strategic working sessions and workshops with both client and internal teams – provide technical direction and mentorship to engineering teams on architectural patterns, scalability, and best practices.
• Provide directions on key work items and feedback to other team members.

Team Management