Sr. Data Engineer

iLink Digital; Pune District; On-site; INR 1,800,000 - 3,200,000; Full time

Job summary

iLink Digital is seeking a Senior Data Engineer in Pune to design, develop, and optimize data pipelines for large-scale analytics. The role focuses on PySpark, SQL, and Python on Azure Databricks, with emphasis on data quality and performance.

The ideal candidate has 7–8 years in data engineering, strong problem-solving skills, and experience shaping ETL architectures within an agile team. Familiarity with Azure Data services is a plus.

Qualifications

• 7–8 years of experience in Data Engineering.
• Strong programming skills in Python and SQL.
• Hands-on experience with PySpark for large-scale data processing.
• Good working knowledge of Azure Databricks.
• Strong problem-solving and debugging skills.
• Experience in designing and optimizing ETL pipelines.
• Knowledge of Agile methodologies (Scrum/Kanban) is a plus.
• Nice to have familiarity with other Azure services such as Data Lake, Synapse, or Data Factory.
• Exposure to CI/CD practices and version control systems (e.g., Git).

Responsibilities

• Design, build, and maintain scalable data pipelines and ETL processes.
• Work with large datasets to ensure high data quality, accuracy, and performance.
• Develop data solutions leveraging Azure Databricks, PySpark, and SQL.
• Collaborate with cross-functional teams including Data Scientists, Analysts, and Product teams to support data-driven initiatives.
• Troubleshoot and optimize data workflows to improve reliability and efficiency.
• Ensure compliance with best practices in data governance, security, and performance.
• Contribute to agile ceremonies and work in a collaborative, fast-paced environment.

Skills

Python

SQL

PySpark

Agile/Scrum/Kanban

Tools

Azure Databricks

Job description

We are seeking a highly skilled Senior Data Engineer with 7–8 years of experience to design, develop, and optimize data pipelines and solutions. The ideal candidate will have expertise in PySpark, SQL, and Python, with solid knowledge of Azure Databricks. A strong understanding of agile methodologies will be an added advantage.

Key Responsibilities:

Design, build, and maintain scalable data pipelines and ETL processes.

Work with large datasets to ensure high data quality, accuracy, and performance.

Develop data solutions leveraging Azure Databricks, PySpark, and SQL.

Collaborate with cross-functional teams including Data Scientists, Analysts, and Product teams to support data-driven initiatives.

Troubleshoot and optimize data workflows to improve reliability and efficiency.

Ensure compliance with best practices in data governance, security, and performance.

Contribute to agile ceremonies and work in a collaborative, fast-paced environment.

Required Skills & Qualifications:

7–8 years of experience in Data Engineering.

Strong programming skills in Python and SQL.

Hands‑on experience with PySpark for large-scale data processing.

Good working knowledge of Azure Databricks.

Strong problem‑solving and debugging skills.

Experience in designing and optimizing ETL pipelines.

Knowledge of Agile methodologies (Scrum/Kanban) is a plus.

Nice to Have:

Familiarity with other Azure services such as Data Lake, Synapse, or Data Factory.

Exposure to CI/CD practices and version control systems (e.g., Git).

Experience working in a cloud-first data ecosystem.