Senior Architect, Databricks

West Monroe (Campus); Los Angeles (CA); Hybrid; USD 205,000 - 234,000; Full time

Job summary

West Monroe in Los Angeles is seeking a Senior Architect to join our Technology and Experience Practice. You will design data pipelines and data architectures, with emphasis on Databricks, Spark, and AI readiness, collaborating with data science and engineering teams to productionize AI workloads.

The role demands 10+ years of professional experience and 8+ years in consulting, deep Databricks expertise, and leadership to coach consultants.

Qualifications

• 10+ years of professional experience, with 3+ years in Databricks.
• 8+ years of prior consulting experience.
• Deep Databricks architecture, performance optimization, Delta Lake, Unity Catalog, and security.
• Experience deploying AI/ML or Generative AI in enterprise environments.
• Hands-on with LLM ecosystems and frameworks such as OpenAI, Azure OpenAI, LangChain, Semantic Kernel, MLflow, or Databricks Mosaic AI.
• Knowledge of Retrieval-Augmented Generation (RAG), vector databases, embeddings, prompt engineering, AI orchestration.
• Familiarity with AI governance, model lifecycle management, responsible AI principles, and enterprise security considerations.
• Experience integrating AI into data platforms, analytics solutions, or customer-facing applications.
• Knowledge of standard methodologies, concepts, and best practices in Big Data.
• Travel up to 30-50% and hybrid in-office 1-2 days.

Responsibilities

• Design data flow and processing pipelines for analysis using Spark on Scala, Storm, Flume, Sqoop
• Design data structures for ingestion and reporting, specific to use case and technology
• Provide data management expertise to evaluate requirements and develop data architecture and platform selection
• Collaborate with data science and engineering teams to productionize AI workloads within cloud ecosystems
• Establish best practices for AI data readiness, feature engineering, model monitoring, observability, and governance
• Lead technical discussions around AI-enabled modernization strategies including semantic layers, vector databases, AI agents, and knowledge retrieval systems
• Evaluate emerging AI technologies and recommend fit-for-purpose tooling aligned to client objectives, scalability, security, and compliance requirements
• Guide and coach consultants, share best practices, perform code reviews
• Coordinate activities with data source application owners to ensure integration with analytics platform and source systems
• Ensure that data integrity issues are addressed
• Liaise and attend meetings necessary to perform duties and aid business and technical development

Skills

Databricks expertise

Big Data

AI/ML integration

LLM ecosystems

Data governance

Cloud data engineering

Data architecture

Leadership coaching

Tools

Spark/Scala

Storm

Flume

Sqoop

Delta Lake

Unity Catalog

MLflow

Databricks Mosaic AI

Vector databases

Job description

Are you ready to make an impact?

West Monroe is seeking a Senior Architect to join our thriving Technology and Experience Practice. This is an excellent opportunity to work with clients developing Big Data strategy and roadmaps with emphasis on Databricks, while gaining exposure to a variety of industries including Healthcare, Financial Services, Insurance, and Utilities.

Responsibilities:

• Design data flow and processing pipelines for analysis using modern toolsets such as Spark on Scala, Storm, Flume, Sqoop
• Design data structures for ingestion and reporting, specific to use case and technology
• Provide data management expertise to our clients in evaluating requirements and developing data architecture and performing platform selection and design. Data management includes appropriate structuring, stewardship of data, semantics/syntax of data attributes, coding structures, and mapping schemes
• Collaborate with data science and engineering teams to productionize machine learning and Generative AI workloads within modern cloud ecosystems
• Establish best practices for AI data readiness, feature engineering, model monitoring, observability, and governance
• Lead technical discussions around AI-enabled modernization strategies including semantic layers, vector databases, AI agents, and knowledge retrieval systems
• Evaluate emerging AI technologies and recommend fit-for-purpose tooling aligned to client objectives, scalability, security, and compliance requirements
• Guide and coach consultants, share best practices, perform code reviews
• Coordinate activities with data source application owners to ensure integration with analytics platform and source systems
• Ensure that data integrity issues are addressed
• Liaise and attend meetings necessary to perform duties and aid business and technical development

Qualifications :

• 10+ years of professional experience and 3+ years of expeirence with Databricks
• 8+ years of prior consulting experience
• Very strong technical expertise in Databricks, including architecture, performance optimization, Delta Lake, Unity Catalog, workflow orchestration, security, governance, and cloud-native data engineering best practices
• Experience architecting and deploying AI/ML or Generative AI solutions within enterprise environments
• Hands-on experience with LLM ecosystems and frameworks such as OpenAI, Azure OpenAI, LangChain, Semantic Kernel, MLflow, or Databricks Mosaic AI
• Knowledge of Retrieval-Augmented Generation (RAG), vector databases, embeddings, prompt engineering, and AI orchestration patterns
• Familiarity with AI governance, model lifecycle management, responsible AI principles, and enterprise security considerations
• Experience integrating AI capabilities into data platforms, analytics solutions, or customer-facing applications
• Knowledge of standard methodologies, concepts, best practices, and procedures within a Big Data environment
• A commitment to inclusion and diversity, and openness to new ideas and perspectives
• Travel requirements up to 30 to 50%
• Hybrid role: 1 to 2 days a week in office

Based on pay transparency guidelines, the salaryrange for this role canvary based on your proximity to one of our West Monroe offices (seetable below). Information on our competitive total rewards package,including our bonus structure and benefits is here. Individual salaries are determined by evaluating a variety of factors including geography,experience, skills, education, and internal equity.

Employees (and their families) are covered by medical, dental, vision, and basic life insurance. Employeesare able toenroll in our company’s 401k plan,purchase shares from our employee stock ownership program and be eligible toreceive annual bonuses. Employees will also receive unlimited flexible time offand ten paid holidays throughout the calendar year. Eligibility for ten weeks of paid parental leave will also be available upon hire date.

Seattle or Washington, D.C.

$199,100—$223,400 USD

Los Angeles

$205,400—$234,100 USD

New York City or San Francisco

$215,200—$244,700 USD

A location not listed above

$183,800—$212,800 USD

Other consultancies talk at you.

At West Monroe, we work with you.

We’re a global business and technology consulting firm passionate about creating measurable value for our clients, delivering real-world solutions.

The combination of business and technology is not new, but how we bring them together is unique. We’re fluent in both. We know that technology alone is not the answer, but how we apply it is. We rely on data to constantly adapt and solve new challenges. Actions that work today with outcomes that generate value for years to come.

At West Monroe, we zero in on the heart of the opportunity, getting to results faster and preparing people for what’s next.

You’ll feel the difference in how we work. We show up personally. We’re right there in the room with you, co-creating through the challenges. With West Monroe, collaboration isn’t a lofty promise, but a daily action. We work together with you to turn vision into clear action with lasting impact.

West Monroe** ****is an Equal Employment Opportunity Employer**
We believe in treating each employee and applicant for employment fairly and with dignity. We base our employment decisions on merit, experience, and potential, without regard to race, color, national origin, sex, sexual orientation, gender identity, marital status, age, religion, disability, veteran status, or any other characteristic prohibited by federal, state or local law. To learn more about diversity, equity and inclusion at West Monroe, visit www.westmonroe.com/inclusion. If you require a reasonable accommodation to participate in our recruiting process, please inquire by sending an email to [e-mail].

Please review our current policy regarding use of generative artificial intelligence during the application process.

If you are based in California, we encourage you to read West Monroe’s Notice at Collection for California residents, provided pursuant to the California Consumer Privacy Act (CCPA) and linked here.