Data Engineer-Azure data factory ADF,Python,PySpark,SQL

RiDiK; Gurugram District; Hybrid; INR 4,000,000 - 6,000,000; Full time

Job summary

MNC in Gurugram is seeking a Senior Data Engineer to design and implement scalable data pipelines and back-end services using Python, PySpark, and Databricks. The role involves building ETL/ELT workflows with Spark SQL, Delta Lake architectures and Azure cloud deployments.

You will optimize SQL queries for large datasets, collaborate with stakeholders, and contribute to CI/CD pipelines and DevOps practices. Hybrid work, 3 days in-office, immediate joining.

Qualifications

• Strong Python development experience with data engineering focus.
• Hands-on PySpark, Spark SQL and Databricks proficiency.
• Experience with Delta Lake and Azure cloud services.
• Advanced SQL and query optimization skills.
• Knowledge of CI/CD and DevOps practices in data pipelines.
• Experience with data warehousing/lakehouse architectures and REST APIs.

Responsibilities

• Design, develop and maintain scalable data pipelines and backend services using Python.
• Build and optimize ETL/ELT workflows with PySpark, Spark SQL and ADF.
• Manage data ingestion, transformation and processing on Databricks.
• Implement Delta Lake architectures for reliable, high-performance data platforms.
• Write optimized SQL for large datasets; collaborate with stakeholders and cross-functional teams.
• Deploy and monitor data apps on Azure; implement CI/CD pipelines with Azure DevOps.
• Troubleshoot, tune performance and ensure coding standards and security guidelines.
• Participate in design discussions, code reviews and technical documentation.

Skills

Python

PySpark

Spark SQL

Databricks

Delta Lake

Advanced SQL

Azure

Azure Data Factory

CI/CD

DevOps

Data modelling

ETL/ELT

REST APIs

Agile/Scrum

Monitoring

Education

Bachelor's or Master's in CS/IT/Engineering

Tools

Databricks

Azure Data Factory

Delta Lake

Azure DevOps

REST APIs

Job description

Dear Candidates,

Currently we are hiring for Senior Data Engineer for leading MNC for Gurugram Location on a permanent/Full Time basis.

JD:

Position: Data Engineer

Exp:6+ yrs

Company Name: MNC

Job Location: Gurugram

Job Mode: Full Time

Working days: 3 Days work from office(Hybrid)

Notice Period: Immediate Joiner

Responsibilities

Design, develop, and maintain scalable data pipelines and backend services using Python.

Build and optimize ETLELT workflows using PySpark, Spark SQL, and Azure Data Factory ADF.

Develop and manage data ingestion, transformation, and processing solutions on Databricks.

Implement and maintain Delta Lake architectures to support reliable and high-performance data platforms.

Write efficient and optimized SQL queries for large-scale datasets.

Collaborate with business stakeholders, architects, data analysts, and cross-functional teams to understand requirements and deliver robust solutions.

Work with Azure cloud services to deploy, monitor, and maintain data applications.

Implement CICD pipelines and deployment automation using Azure DevOps.

Perform troubleshooting, performance tuning, and optimization of data workloads.

Ensure adherence to coding standards, security guidelines, and best engineering practices.

Participate in solution design discussions, code reviews, and technical documentation.

Required Skills

Must-Have Technical Skills Strong programming experience in Python

Hands-on experience with PySpark and Spark SQL

Strong experience working with Databricks

Good understanding of Delta Lake Advanced SQL development and query optimization skills

Experience with Microsoft Azure services Hands-on experience with Azure Data Factory ADF

Experience in CICD implementation using Azure

DevOps Strong understanding of data modeling, ETL ELT concepts, and distributed data processing

Experience working with large-scale data platforms and cloud-based solutions

Preferred Skills Experience with data warehousing and lakehouse architectures

Understanding of data governance and data quality frameworks

Experience with REST APIs and backend service development

Exposure to AgileScrum methodologies

Knowledge of monitoring and observability tools

Soft Skills

Excellent verbal and written communication skills

Ability to interact confidently with business stakeholders and technical teams

Strong analytical and problem-solving abilities Self-driven with a proactive approach to ownership and delivery

Strong collaboration and teamwork mindset

Ability to clearly articulate technical concepts to both technical and non-technical audiences

Qualifications

Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

4-6 years of relevant industry experience in Data Engineering, Data Platform Engineering, or Backend Python Development.