Senior Data Engineer

Anblicks

Hyderabad

On-site

INR 3,500,000 - 7,000,000

Full time

14 days+

Application generator

Stand out for this role — generate a tailored resume and cover letter in about a minute.

Get past ATS filters

Job summary

Anblicks is seeking a Lead / Senior Data Engineer in Hyderabad to lead design, development, and governance of enterprise-scale data platforms on Azure and AWS. The role involves hands-on leadership, building data pipelines with Databricks, Snowflake, Delta Lake, and Delta Live Tables, and mentoring teams for analytics, reporting, and AI/ML initiatives.

Qualifications include 8+ years in data engineering, strong Python, PySpark and SQL skills, and experience with ADF, Synapse, and Unity Catalog.

Qualifications

Bachelor's degree in CS/IT/Engineering or related discipline.
8+ years in Data Engineering, Data Warehousing, and Cloud Data Platforms.
Proven leadership in enterprise data transformation and modernization.
Strong track record delivering scalable data solutions in Azure cloud environments.

Responsibilities

Design, develop, and maintain scalable ETL/ELT pipelines using ADF, Databricks, Python, PySpark, and SQL.
Architect modern Lakehouse and Data Warehouse solutions across Azure and AWS.
Develop analytics-ready datasets for BI, reporting, and analytics use cases.
Implement data quality, validation, and governance controls.
Lead technical design discussions and mentor data engineering teams.

Skills

Data engineering

Leadership

Azure Databricks

Snowflake

Delta Lake

Unity Catalog

Delta Live Tables

Azure Data Factory

Python

PySpark

SQL

Spark SQL

AWS

Azure Synapse

ADLS Gen2

Azure Data Lake

Microsoft Fabric

Education

Bachelor's Degree in Computer Science/Information Technology/Engineering

Tools

Azure Databricks

Snowflake

Delta Lake

Delta Live Tables

Azure Data Factory

Python

PySpark

SQL

Azure Synapse

ADLS Gen2

Unity Catalog

Job description

Job Description (JD)

Lead / Senior Data Engineer

Location: Hyderabad, India

Experience: 8+ Years

Employment Type: Full-Time

Notice Period : Immediate / 15 Days.

About the Role

We are looking for a highly skilled and results-driven Lead / Senior Data Engineer with extensive experience in designing, developing, and managing enterprise-scale data platforms across Azure and AWS cloud ecosystems. The ideal candidate will bring strong expertise in Azure Databricks, Snowflake, Delta Lake, Unity Catalog, Delta Live Tables (DLT), Azure Data Factory, Python, PySpark, and SQL, combined with proven leadership experience in delivering modern data warehouse and lakehouse solutions.

This role requires a hands-on technical leader who can drive architecture decisions, build scalable and governed data platforms, mentor engineering teams, and collaborate closely with business stakeholders to deliver data-driven solutions that enable analytics, reporting, and AI/ML initiatives.

Key Responsibilities

Data Architecture & Engineering

Design, develop, and maintain scalable and high-performance ETL/ELT pipelines using Azure Data Factory, Azure Databricks, Python, PySpark, and SQL.
Build metadata-driven and parameterized data ingestion frameworks supporting both batch and real-time processing.
Architect modern Lakehouse and Data Warehouse solutions leveraging Azure and AWS cloud technologies.
Develop analytics-ready datasets and enterprise data products for business intelligence, reporting, and advanced analytics use cases.
Implement robust data quality frameworks, reconciliation processes, and validation controls.

Azure Databricks & Lakehouse Development

Design and implement Medallion Architecture (Bronze, Silver, Gold) using Delta Lake.
Build and manage Delta Live Tables (DLT) pipelines for reliable and quality-enforced data processing.
Develop PySpark and Spark SQL transformations for large-scale data workloads.
Implement Unity Catalog governance, including data lineage, auditing, metadata management, and fine-grained access controls.
Optimize Databricks workloads through partitioning strategies, caching, broadcast joins, Adaptive Query Execution (AQE), and data-skew handling.

Snowflake Data Warehousing

Design and implement Snowflake-based data warehouse solutions.
Develop and optimize Snowflake schemas, data models, RBAC frameworks, and data ingestion processes using Snowpipe and COPY operations.
Perform SQL query tuning, warehouse sizing, performance optimization, and cost management.
Support reporting, analytics, and enterprise BI workloads through scalable and efficient data models.

Cloud & Streaming Solutions

Develop real-time and event-driven data processing solutions using Azure Event Hubs, AWS Kinesis, Azure Functions, and AWS Lambda.
Build scalable ingestion frameworks for streaming and batch data sources.
Support hybrid and multi-cloud data ecosystems across Azure and AWS platforms.

Data Modeling & Governance

Design dimensional data models using Star Schema and Snowflake Schema methodologies.
Implement Slowly Changing Dimensions (SCD Type 1, 2, and 3).
Support Data Mesh and domain-oriented data product initiatives.
Establish enterprise data governance practices, security controls, and data stewardship standards.

DevOps & Automation

Develop and maintain CI/CD pipelines using Azure DevOps and GitHub Actions.
Implement automated deployment, monitoring, testing, and release management processes.
Promote engineering best practices through code reviews, version control standards, and development frameworks.

Leadership & Stakeholder Management

Lead technical design discussions, architectural reviews, and project delivery activities.
Mentor and guide data engineering teams through technical coaching and knowledge sharing.
Collaborate with business stakeholders, architects, data scientists, and analytics teams to define and implement data solutions.
Drive technical strategy and ensure successful project execution from design through production support.

Required Qualifications

Education

Bachelor's Degree in Computer Science, Information Technology, Engineering, or a related discipline.

Experience

8+ years of experience in Data Engineering, Data Warehousing, and Cloud Data Platforms.
Proven experience leading enterprise data transformation and modernization initiatives.
Strong track record of delivering scalable data solutions in Azure cloud environments.

Technical Skills

Cloud Platforms