ADG Analyst/Senior Analyst (Data Engineer-Databricks & Pyspark)

Essex Lake Group; Gurugram District; On-site; INR 1,200,000 - 2,000,000; Full time

Job summary

Essex Lake Group is seeking a Senior Analyst (Data Engineer-Databricks & PySpark) in Gurgaon, India to design, build and maintain scalable data pipelines and curated datasets for analytics in the banking domain. You will partner with analytics, business and technology teams to deliver reliable, governed data assets.

Responsibilities include building ETL/ELT pipelines with Databricks and PySpark, preparing datasets for reporting and ML, implementing data-quality checks and optimizing Spark

Qualifications

• Strong hands-on experience with Databricks and PySpark.
• Proficiency in Python and SQL.
• Experience building ETL/ELT pipelines and working with Data Lake.
• Understanding of distributed processing, workflow orchestration and performance tuning.
• Working knowledge of the banking domain, including banking data and metrics.
• Understanding of data-quality requirements, including validation, completeness, accuracy and consistency.

Responsibilities

• Develop data ingestion and transformation pipelines using Databricks, PySpark, Python and SQL.
• Integrate data from multiple sources and prepare datasets for reporting, analytics and machine-learning use cases.
• Build reusable transformation components and analytical data models.
• Implement data-quality checks, reconciliation, exception handling and pipeline monitoring.
• Optimize Spark workloads for performance, reliability and cost.
• Apply data security, access controls, lineage and documentation standards.
• Support automated deployment, production troubleshooting and ongoing pipeline maintenance.
• Translate banking business requirements into sustainable data solutions.

Skills

Databricks

PySpark

Python

SQL

Data Lake

ETL/ELT

Banking domain

Data quality

Tools

Azure

Power BI

Job description

Position: ADG Senior Analyst (Data Engineer-Databricks & Pyspark)

Experience- 3-5 Years

Location: Gurgaon, India

Work Mode: Work from Office (Monday- Friday)

Time: 2:00 PM 11:00 PM IST

Role Summary

Design, build and maintain scalable data pipelines and curated datasets to support analytics, new capabilities and technology experimentation in the banking domain. Partner with analytics, business and technology teams to deliver reliable, reusable and governed data assets.

Key Responsibilities

• Develop data ingestion and transformation pipelines using Databricks, PySpark, Python and SQL.
• Integrate data from multiple sources and prepare datasets for reporting, analytics and machine-learning use cases.
• Build reusable transformation components and analytical data models.
• Implement data-quality checks, reconciliation, exception handling and pipeline monitoring.
• Optimize Spark workloads for performance, reliability and cost.
• Apply data security, access controls, lineage and documentation standards.
• Support automated deployment, production troubleshooting and ongoing pipeline maintenance.
• Translate banking business requirements into sustainable data solutions.

Required Skills & Experience

• Strong hands‑on experience with Databricks and PySpark
• Proficiency in Python and SQL
• Experience building ETL/ELT pipelines and working with Data Lake
• Understanding of distributed processing, workflow orchestration and performance tuning
• Working knowledge of the banking domain, including banking data and metrics
• Understanding of data-quality requirements, including validation, completeness, accuracy and consistency

Preferred Skills

• Experience with Azure-based data platforms
• Familiarity with enterprise data governance and sensitive-data handling
• Experience preparing data for Power BI and machine-learning applications