EY - GDS Consulting - AI And DATA - Databricks Architect - Senior Manager

EY

Kochi

On-site

INR 3,500,000 - 6,500,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Career development
Flexible work options
Supportive culture

Job summary

EY is seeking a Senior Data & AI Architect (Databricks Architect) to lead enterprise Data & AI solutions built on the Databricks Lakehouse. You will own end-to-end data pipelines, AGI and RAG solutions, and cloud architecture on AWS/Azure, guiding client workshops and strategic proposals.

You will mentor teams, drive scalable architecture, and engage with clients to translate business needs into robust technical designs. EY offers development and a collaborative, high-impact environment.

Qualifications

  • 13+ years of experience in Data Engineering / Data Architecture.
  • Hands-on experience with Databricks, Hadoop, Spark/PySpark, and cloud data platforms.

Responsibilities

  • Architect, design, and deliver scalable Data & AI solutions on the Databricks Lakehouse platform.
  • Design and optimize large-scale distributed data processing pipelines using Hadoop, Spark/PySpark.
  • Build reusable Python frameworks and pipelines for batch and streaming data engineering workloads.
  • Define and implement cloud data architecture on AWS and/or Azure with Databricks integration.
  • Own end-to-end ETL/ELT design, data ingestion, transformation, orchestration, and data quality.

Skills

Databricks
Hadoop
Spark/PySpark
Python
AWS
Azure
SQL
Agentic AI
RAG
Git & CI/CD

Tools

Databricks Lakehouse
Unity Catalog
Terraform
Databricks Asset Bundles

Job description

EY - GDS Consulting - AI And DATA - Databricks Architect - Senior Manager

Other locations: Primary Location Only

Requisition ID: 1745792

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all.

EY-Consulting - Data and Analytics – Senior Manager - Databricks Architect

EY's Consulting Services is a unique, industry-focused business unit that provides a broad range of integrated services that leverage deep industry experience with strong functional and technical capabilities and product knowledge. EY’s financial services practice provides integrated Consulting services to financial institutions and other capital markets participants, including commercial banks, retail banks, investment banks, broker-dealers & asset management firms, and insurance firms from leading Fortune 500 Companies. Within EY’s Consulting Practice, Data and Analytics team solves big, complex issues and capitalize on opportunities to deliver better working outcomes that help expand and safeguard the businesses, now and in the future. This way we help create a compelling business case for embedding the right analytical practice at the heart of client’s decision-making.

The opportunity

Senior Data & AI Architect with 13+ years of experience in data engineering and data architecture, strong hands‑on expertise in Databricks, Hadoop, Spark/PySpark, and cloud data platforms.

This role combines deep technical ownership of enterprise Lakehouse and data platform architecture with hands‑on experience building Agentic AI and Retrieval‑Augmented Generation (RAG) solutions on Databricks. The ideal candidate is equally comfortable designing large‑scale, production‑grade data pipelines, architecting modern AI‑driven solutions, and engaging directly with clients on strategic proposals and solution design.

Your key responsibilities
  • Architect, design, and deliver scalable Data & AI solutions on the Databricks Lakehouse platform, covering ingestion, transformation, storage, governance, and consumption layers.
  • Design and optimize large‑scale distributed data processing pipelines using Hadoop, Apache Spark, and PySpark.
  • Build reusable, production‑grade Python frameworks and pipelines for batch and streaming data engineering workloads.
  • Define and implement cloud data architecture on AWS and/or Azure, covering storage, IAM, networking, security, and monitoring, with native Databricks integration.
  • Own end-to-end ETL/ELT design, data ingestion, transformation, orchestration, and data quality frameworks across the platform.
  • Design dimensional data models (star/snowflake schemas, fact/dimension tables, SCD) to support analytical and reporting use cases.
  • Design and build Agentic AI solutions on Databricks, including LLM integration, tool/function calling, agent workflows, guardrails, and enterprise data integration.
  • Architect and implement RAG (Retrieval‑Augmented Generation) solutions using Databricks — document ingestion, chunking, embeddings, vector search, retrieval, grounding, and evaluation.
  • Establish and enforce engineering best practices for Git‑based version control, code reviews, and CI/CD across development, test, and production environments.
  • Lead performance optimization and production deployment of Databricks jobs, workflows, and pipelines at scale.
  • Participate in strategic RFP/RFI responses, technical proposals, solution shaping, effort estimation, and architecture presentations.
  • Act as a trusted technical advisor in customer workshops, translating business requirements into robust technical architectures.
  • Mentor engineering teams on Databricks, Spark, cloud architecture, and modern AI/RAG engineering practices.
Skills and attributes for success

13+ years of experience in Data Engineering / Data Architecture, with strong hands‑on experience across Databricks, Hadoop, Spark/PySpark, and cloud data platforms.

  • Databricks
    • Strong expertise in Databricks Lakehouse, Delta Lake, Databricks SQL, and Unity Catalog.
    • Hands‑on experience with Jobs/Workflows, performance optimization, and production deployments.
  • Hadoop & Spark
    • Strong knowledge of the Hadoop ecosystem, HDFS, and Hive.
    • Strong hands‑on experience with Apache Spark and PySpark for large‑scale distributed data processi
  • Python
    • Strong proficiency in Python for building scalable, reusable, production‑grade data pipelines and frameworks.
    • Cloud
    • Strong experience with AWS and/or Azure, including cloud storage, IAM, networking, security, and monitoring.
    • Experience integrating Databricks with cloud‑native services.
  • Expertise in ETL/ELT, batch and streaming processing, data ingestion, transformation, orchestration, and data quality.
  • Proven ability to design scalable, production‑ready pipeline architectures.
  • Strong SQL skills, with experience in dimensional modelling, star/snowflake schemas, fact/dimension models, SCD, and analytical data models.
  • Agentic AI
    • Hands‑on experience designing and building Agentic AI solutions with Databricks, including LLMs, tool/function calling, agent workflows, guardrails, and enterprise data integration.
  • RAG (Retrieval‑Augmented Generation)
    • Strong experience building RAG solutions using Databricks, including document ingestion, chunking, embeddings, vector search, retrieval, grounding, and evaluation.
  • Git & CI/CD
    • Strong knowledge of Git, branching strategies, pull requests, and code reviews.
    • Experience with CI/CD pipelines and deployment across multiple environments.
    • Experience with Databricks Asset Bundles and/or Terraform is a plus.
  • Strategic RFPs & Solutioning
    • Ability to participate in strategic RFP/RFI responses, solution shaping, technical proposals, and effort estimations.
    • Experience delivering architecture presentations and leading customer workshops.
  • Client Communication
  • Strong client‑facing and communication skills, with the ability to understand business requirements, translate them into scalable technical solutions, and clearly communicate architectural decisions to both technical and non‑technical stakeholders.
Ideally, you’ll also have
  • Prior experience in a consulting, systems integrator, or client‑facing delivery role.
  • Relevant cloud or Databricks certifications (e.g., Databricks Certified Data Engineer / Solutions Architect, AWS/Azure certifications).
  • Exposure to enterprise data governance, MLOps, or LLMOps practices.
What working at EY offers

At EY, we’re dedicated to helping our clients, from start‑ups to Fortune 500 companies — and the work we do with them is as varied as they are.

You get to work with inspiring and meaningful projects. Our focus is education and coaching alongside practical experience to ensure your personal development. We value our employees and you will be able to control your own development with an individual progression plan. You will quickly grow into a responsible role with challenging and stimulating assignments. Moreover, you will be part of an interdisciplinary environment that emphasizes high quality and knowledge exchange. Plus, we offer:

  • Support, coaching and feedback from some of the most engaging colleagues around
  • Opportunities to develop new skills and progress your career
  • The freedom and flexibility to handle your role in a way that’s right for you
EY | Building a better working world

EY exists to build a better working world, helping to create long‑term value for clients, people and society and build trust in the capital markets.

Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.

Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.

Select how often (in days) to receive an alert:

EY refers to the global organization, and may refer to one or more, of the member firms of Ernst & Young Global Limited, each of which is a separate legal entity. Ernst & Young Global Limited, a UK company limited by guarantee, does not provide services to clients.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

EY - GDS Consulting - AI And DATA - Databricks Architect - Senior Manager
EY - GDS Consulting - AI And DATA - Databricks Architect - Senior Manager

EY • Hyderabad

On-site
INR 4,000,000 - 7,000,000
Support and coaching
Career development opportunities
Flexible work style
Agile-DE - Cloud - Senior Data Platform Engineer - GDSN02
Agile-DE - Cloud - Senior Data Platform Engineer - GDSN02

EY • Chennai

Hybrid
INR 2,500,000 - 4,200,000
EY - GDS Consulting - AI And DATA -AWS Data Engineer - Manager
EY - GDS Consulting - AI And DATA -AWS Data Engineer - Manager

EY • Coimbatore District

On-site
INR 3,500,000 - 6,000,000
TechOps-CloudOps-Data Platform And Engineering Support
TechOps-CloudOps-Data Platform And Engineering Support

EY • Kanayannur

On-site
INR 2,400,000 - 4,200,000
Agile-DE-Application Engineer Technical Lead- II-VG-CS0048-B1
Agile-DE-Application Engineer Technical Lead- II-VG-CS0048-B1

EY • Hyderabad

On-site
INR 3,500,000 - 6,000,000
Agile-EY - GDS Consulting - AI And DATA -Databricks DE - senior
Agile-EY - GDS Consulting - AI And DATA -Databricks DE - senior

EY • Hyderabad

On-site
INR 1,200,000 - 2,000,000
DE-Application Engineer Technical Lead- II-VG-CS0048-B1
DE-Application Engineer Technical Lead- II-VG-CS0048-B1

EY • Hyderabad

On-site
INR 4,200,000 - 6,400,000
Agile-EY GDS - Senior Manager-Appsec-AI-Engineering-Architect
Agile-EY GDS - Senior Manager-Appsec-AI-Engineering-Architect

EY • Thiruvananthapuram

On-site
INR 4,000,000 - 6,500,000
Premium benefits
Flexible environment
Diversity & Inclusion
+1
EY - GDS Consulting - AI and DATA - Databricks - Senior
EY - GDS Consulting - AI and DATA - Databricks - Senior

EY • Hyderabad

On-site
INR 1,800,000 - 3,000,000
EY - GDS Consulting - AIA - Pre-Sales Support - Manager
EY - GDS Consulting - AIA - Pre-Sales Support - Manager

EY • Kolkata

On-site
INR 4,000,000 - 7,000,000