Senior AI Architect - Enterprise Integrations

IBM Computing; Jersey City (NJ); Remote; USD 160,000 - 230,000; Full time

Job summary

IBM Consulting is seeking a Senior AI Architect - Enterprise Integrations to lead AI integration engagements for large enterprise clients across the US. You will shape architecture, mentor teams, and deliver production-grade AI-powered solutions.

You will design MCP servers, implement agent orchestration, and work with multiple LLM providers to optimize cost and performance. This is a remote role with nationwide reach.

Qualifications

• 8+ years in application or solution architecture with AI/ML focus.
• Hands-on MCP server design and implementation.
• Proven experience integrating AI agents with enterprise systems.
• Deep knowledge of agentic AI frameworks (LangChain, LangGraph, AutoGen, CrewAI).
• Strong API design and enterprise integration patterns.
• Experience with major LLM providers and their APIs (OpenAI, Azure, IBM, AWS, Google).

Responsibilities

• Serve as an internal champion for AI integration architecture strategy and reference assets.
• Mentor architects and consultants on AI integration patterns.
• Lead discovery and design sessions to define AI integration strategies.
• Translate business requirements into scalable, secure AI-powered solutions.
• Architect and implement integrations between AI agents and enterprise systems.
• Establish a semantic data layer to contextualize enterprise data.
• Define patterns for agent orchestration, memory management, and human-in-the-loop workflows.
• Design MCP gateway/servers exposing data, tools, and APIs.
• Ensure security, observability, and reliability; deliver production-grade solutions.
• Role can be performed from anywhere in the US.

Skills

AI/ML architecture

MCP (Model Context Protocol)

AI integration patterns

API design (REST/GraphQL)

Python/TypeScript

Tools

LangChain

LangGraph

AutoGen

CrewAI

Job description

Introduction

A career in IBM Consulting is built on long-term client relationships and close collaboration worldwide. You’ll work with leading companies across industries, helping them shape their hybrid cloud and AI journeys. With support from our strategic partners, robust IBM technology, and Red Hat, you’ll have the tools to drive meaningful change and accelerate client impact. At IBM Consulting, curiosity fuels success. You’ll be encouraged to challenge the norm, explore new ideas, and create innovative solutions that deliver real results. Our culture of growth and empathy focuses on your long-term career development while valuing your unique skills and experiences.

Your role and responsibilities

We are seeking a Senior AI Architect - Enterprise Integrations to join our growing AI practice. As demand for intelligent, connected enterprise solutions accelerates, we are building out a specialized capability in AI-native architecture and agentic system design. This is a senior, client-facing role where you will provide thought leadership, shape practice offerings, and lead delivery on high-impact AI integration engagements.

You will be the subject matter expert at the intersection of AI agents and enterprise systems - designing and building the connective tissue that makes AI solutions work in the real world.

What You’ll Do

Thought Leadership & Practice Development

• Serve as an internal champion for AI integration architecture strategy, helping to define and grow this capability within the firm * Develop points of view, reference architectures, and reusable assets that accelerate client delivery

• Mentor and upskill other architects and consultants on AI integration patterns and emerging standards

Client Engagement & Solution Architecture

• Lead discovery, design, and architecture sessions with clients to define AI integration strategies

• Translate complex business requirements into scalable, secure, and maintainable AI-powered solutions

• Provide cloud and LLM consumption cost estimates, optimizing for cost-effective architecture through fit-for-purpose model selection, efficient prompting & caching, and scalable infrastructure design

• Present architectural recommendations to executive and technical stakeholders with clarity and confidence

• Lead cross-functional development teams to deliver enterprise-grade, production-ready AI solutions, ensuring alignment to architecture standards, scalability, and operational excellence

Technical Delivery

• Architect and implement integrations between AI agents and enterprise systems including CRMs, ERPs, data platforms, and third-party APIs

• Establish a semantic data layer that standardizes and contextualizes enterprise data, enabling AI agents to reliably discover, interpret, and interact with systems & data

• Define patterns for agent orchestration, tool calling, memory management, and human-in-the-loop workflows

• Design and build Model Context Protocol (MCP) gateway & servers that expose enterprise data, tools, and APIs as structured context for AI agents

• Ensure solutions meet enterprise standards for security, observability, and reliability

• This role can be performed from anywhere in the US.

Required technical and professional expertise

• 8+ years of experience in application or solution architecture, with at least 2+ years focused on AI/ML systems

• Hands-on experience designing and building MCP (Model Context Protocol) servers

• Proven experience integrating AI agents with enterprise systems and workflows

• Deep understanding of agentic AI frameworks (e.g., LangChain, LangGraph, AutoGen, CrewAI, or similar)

• Strong knowledge of API design (REST, GraphQL), event-driven architecture, and enterprise integration patterns

• Experience working with LLM providers (OpenAI, Anthropic, Azure, IBM, AWS, Google, etc.) and their APIs * Proficiency in Python and/or TypeScript/JavaScript for building AI integration solutions

• Consulting or professional services experience - comfortable owning client relationships and leading engagements

Preferred technical and professional experience

• Familiarity with cloud platforms (Azure, AWS, or GCP) and cloud-native deployment patterns

• Experience with RAG (Retrieval-Augmented Generation) architectures and vector databases

• Background in enterprise middleware or iPaaS platforms (MuleSoft, Boomi, Azure Integration Services)

• Knowledge of AI security considerations including prompt injection, data governance, and access controls

IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.