Lead Data Architect

Tata Digital; Bengaluru; On-site; INR 2,800,000 - 5,600,000; Full time

Job summary

Tata Digital in Bengaluru seeks an experienced Data and AI Architect to define and own the enterprise data and AI architecture vision. You will design end-to-end data platforms on Azure and Databricks, building scalable lakehouse-based solutions.

The role requires deep expertise in batch, streaming, and real-time pipelines, governance, and collaboration with data engineers, scientists, and business stakeholders to deliver robust, scalable data products.

Qualifications

• Bachelor's or master's in CS/IT/Engineering or related field.
• 10+ years in data engineering/architecture/platform engineering or related disciplines.
• Strong hands-on with Azure, Azure Databricks, Spark, PySpark, SQL, and Delta Lake.
• Proven experience designing enterprise-grade data architectures and modern lakehouse platforms.
• Experience with batch, streaming, and near real-time pipelines.
• Data modeling, domain-oriented data design, and reusable data products.
• Exposure to AI/ML data pipelines and model data prep.

Responsibilities

• Define and own enterprise data and AI architecture vision and standards.
• Architect end-to-end data platforms on Azure and Databricks using lakehouse principles.
• Design scalable frameworks for batch, streaming, real-time, and incremental data processing.
• Establish architecture patterns for ingestion, transformation, storage, serving, and data consumption.
• Lead AI-ready data foundations including curated datasets, feature pipelines, and model-serving interfaces.
• Ensure data architecture supports analytics, ML, GenAI, and intelligent automation use cases.
• Partner with engineers, scientists, and stakeholders to translate business needs into robust solutions.
• Provide leadership on Spark optimization, Delta Lake design, and performance engineering.
• Define and enforce governance, metadata, lineage, privacy, security, and compliance standards.
• Drive data quality, observability, and reliability across the platform.
• Guide orchestration, CI/CD, testing, and release management for data products.
• Evaluate emerging technologies and architectural patterns for scalability and cost efficiency.
• Mentor engineering teams and act as a design authority for data and AI initiatives.

Skills

Architectural thinking

Cross-functional leadership

Analytical mindset

Collaboration

Problem solving

Education

Bachelor's or Master's degree in CS/IT/Engineering

Tools

Azure

Azure Databricks

Apache Spark

PySpark

SQL

Delta Lake

Airflow

Databricks Workflows

Azure Data Factory

Job description

Responsibilities:

• Define and own the enterprise data and AI architecture vision, reference architecture, and design standards.
• Architect end-to-end data platforms on Azure and Databricks using modern lakehouse and cloud-native principles.
• Design scalable frameworks for batch, streaming, real-time, and incremental data processing.
• Establish architecture patterns for ingestion, transformation, storage, serving, and consumption of data across domains.
• Lead the design of AI-ready data foundations, including curated datasets, feature pipelines, semantic layers, and model-serving data interfaces.
• Ensure data architecture supports advanced analytics, machine learning, GenAI, and intelligent automation use cases.
• Partner with data engineers, data scientists, platform teams, and business stakeholders to translate business objectives into robust technical solutions.
• Provide technical leadership on Spark optimization, Databricks workload tuning, Delta Lake design, and performance engineering.
• Define and enforce standards for data governance, metadata, lineage, privacy, security, and compliance.
• Drive adoption of data quality, observability, and reliability practices across the platform.
• Guide implementation of orchestration, CI/CD, testing, and release management for data products and pipelines.
• Evaluate emerging technologies and recommend architectural patterns that improve scalability, resilience, and cost efficiency.
• Mentor engineering teams and act as a design authority for critical data and AI initiatives.

Requirements:

• Bachelor's or master's degree in computer science, information technology, engineering, or a related field.
• 10+ years of experience in data engineering, data architecture, data platform engineering, or related disciplines.
• Strong hands-on experience with Azure, Azure Databricks, Apache Spark, PySpark, SQL, and Delta Lake.
• Proven experience designing enterprise-grade data architectures and modern lakehouse platforms.
• Strong expertise in complex data processing, including batch, streaming, and near real-time pipelines.
• Solid understanding of data modeling, domain-oriented data design, and reusable data products.
• Working knowledge of AI/ML data pipelines, feature engineering, model data preparation, and data enablement for AI solutions.
• Experience with data governance, security, access control, privacy, lineage, and quality management.
• Exposure to orchestration tools such as Azure Data Factory, Databricks Workflows, Airflow, or equivalent platforms.
• Strong ability to balance strategic architecture thinking with practical implementation guidance.
• Excellent communication, stakeholder management, and problem-solving skills.

Competencies:

• Strong architectural thinking and solution ownership.
• Ability to lead complex, cross-functional technology initiatives.
• Strong analytical and problem-solving mindset.
• Excellent collaboration and influencing skills.
• Ability to work in a fast-paced, evolving technology environment.
• High attention to detail and commitment to quality.

Role Expectations:

• Establish a future-ready data and AI architecture that enables business growth and innovation.
• Create scalable, governed, and reusable data platform capabilities.
• Improve data engineering maturity across ingestion, processing, governance, and consumption layers.
• Enable AI and advanced analytics through reliable, high-quality, and well-architected data foundations.
• Serve as a trusted advisor to technology and business leadership on data and AI decisions.