Data Architect

Experion Technologies; Ernakulam; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Experion Technologies seeks a senior Data Architect to lead the architecture, design and delivery of enterprise-scale Databricks data platforms across Azure, AWS, or GCP. You will guide data engineers, establish governance standards and drive modernization across cloud-native services.

The role requires deep expertise in Databricks, Lakehouse, Delta Lake and Medallion Architecture, with strong Spark, PySpark, Python and SQL skills. Leadership and stakeholder collaboration are essential.

Qualifications

• 10+ years of experience in Data Engineering / Data Architecture.
• Strong hands-on experience with Databricks and Lakehouse architecture.
• Strong knowledge of Apache Spark, PySpark, Python and SQL.
• Experience with Delta Lake and Medallion Architecture.
• Strong experience in data modelling, ETL/ELT and data integration.
• Experience designing scalable batch and real-time data solutions.
• Strong knowledge of at least one major cloud platform: Azure, AWS or GCP.
• Understanding of cloud data services and integration patterns.
• Experience with Databricks Workflows/Jobs and Unity Catalog.
• Strong understanding of data security, governance, quality and lineage.
• Experience with CI/CD, Git and DevOps practices.
• Strong solution architecture and technical leadership skills.

Responsibilities

• Design and implement enterprise-scale Databricks Lakehouse architectures.
• Define architecture across Azure, AWS and GCP based on business and technical requirements.
• Design data ingestion, transformation and processing using Databricks, Spark, PySpark, SQL and Python.
• Establish Lakehouse architecture, including Delta Lake, Medallion Architecture and data modelling.
• Design batch and real-time data processing and integration solutions.
• Architect and implement Databricks Workflows, Jobs, Unity Catalog and governance capabilities.
• Define data security, access control, lineage, quality, monitoring and governance standards.
• Design scalable and optimized solutions for performance and cloud cost.
• Lead migration and modernization of legacy data platforms to Databricks.
• Integrate Databricks with cloud-native services across Azure / AWS / GCP.
• Establish CI/CD, Git, DevOps and deployment standards for data engineering.
• Provide technical leadership through design reviews, code reviews and engineering standards.
• Guide and mentor Data Engineers and technical teams.
• Conduct POCs and evaluate new Databricks/cloud capabilities.
• Work with business stakeholders, project managers and enterprise architects to translate requirements into technical solutions.
• Support estimation, solution proposals, technical presentations and client discussions.
• Adhere to the Information Security Management policies and procedures.

Skills

Databricks

Lakehouse architecture

Apache Spark

PySpark

Python

SQL

Delta Lake

Medallion Architecture

ETL/ELT

Data modelling

Real-time data processing

Cloud platforms (Azure, AWS, GCP)

Databricks Workflows/Jobs

Unity Catalog

CI/CD

Git

DevOps practices

Technical leadership

Tools

Databricks Workflows/Jobs

Unity Catalog

Job description

Lead the architecture, design and delivery of enterprise-scale Databricks data platforms across Azure, AWS or GCP. The role will provide technical leadership for modern data engineering, Lakehouse architecture, data integration, data migration, analytics and AI/ML workloads while ensuring scalable, secure, performant and cost-effective solutions.

Responsibilities

• Design and implement enterprise-scale Databricks Lakehouse architectures.
• Define architecture across Azure, AWS and GCP based on business and technical requirements.
• Design data ingestion, transformation and processing using Databricks, Spark, PySpark, SQL and Python.
• Establish Lakehouse architecture, including Delta Lake, Medallion Architecture and data modelling.
• Design batch and real-time data processing and integration solutions.
• Architect and implement Databricks Workflows, Jobs, Unity Catalog and governance capabilities.
• Define data security, access control, lineage, quality, monitoring and governance standards.
• Design scalable and optimized solutions for performance and cloud cost.
• Lead migration and modernization of legacy data platforms to Databricks.
• Integrate Databricks with cloud-native services across Azure / AWS / GCP.
• Establish CI/CD, Git, DevOps and deployment standards for data engineering.
• Provide technical leadership through design reviews, code reviews and engineering standards.
• Guide and mentor Data Engineers and technical teams.
• Conduct POCs and evaluate new Databricks/cloud capabilities.
• Work with business stakeholders, project managers and enterprise architects to translate requirements into technical solutions.
• Support estimation, solution proposals, technical presentations and client discussions.
• Adhere to the Information Security Management policies and procedures.

Required Skills

• 10+ years of experience in Data Engineering / Data Architecture.
• Strong hands-on experience with Databricks and Lakehouse architecture.
• Strong knowledge of Apache Spark, PySpark, Python and SQL.
• Experience with Delta Lake and Medallion Architecture.
• Strong experience in data modelling, ETL/ELT and data integration.
• Experience designing scalable batch and real-time data solutions.
• Strong knowledge of at least one major cloud platform: Azure, AWS or GCP.
• Understanding of cloud data services and integration patterns.
• Experience with Databricks Workflows/Jobs and Unity Catalog.
• Strong understanding of data security, governance, quality and lineage.
• Experience with CI/CD, Git and DevOps practices.
• Strong solution architecture and technical leadership skills.

Preferred Skills /Good to Have

• Snowflake