Data Engineer

Intact; Gurugram District; On-site; INR 1,500,000 - 1,900,000; Full time

Job summary

Intact is seeking a skilled Data Engineer to design, develop, and maintain scalable data solutions in a distributed environment. The role emphasizes modern data platforms, testing, and CI/CD practices, with strong collaboration across architects and business stakeholders.

The ideal candidate will build robust data ingestion and transformation pipelines, ensure data quality, and optimize platform reliability and performance while aligning with data governance standards.

Qualifications

• Hands-on experience with distributed data processing and modern data platforms.
• Ability to design end-to-end data pipelines and data models.
• Strong testing and CI/CD mindset.

Responsibilities

• Design, develop, and maintain scalable data ingestion, transformation, and serving pipelines.
• Build and optimize data solutions using Azure Databricks, PySpark, Delta Lake, and Azure data services.
• Develop data integration workflows using Azure Data Factory and other Azure-native services.
• Design and implement data models that support analytics, reporting, and downstream applications.
• Implement robust data quality, validation, monitoring, and observability solutions.
• Develop automated testing strategies and maintain high code quality standards.
• Implement CI/CD pipelines and deployment automation using Azure DevOps.
• Collaborate with architects, product owners, analysts, and business stakeholders to deliver data products.
• Troubleshoot production issues and continuously improve platform reliability, scalability, and performance.
• Ensure adherence to data governance, security, and engineering best practices.

Skills

Python/Java/Scala

Advanced SQL

PySpark

Databricks

Data pipelines

Data warehousing

Git

Testing frameworks

Production-grade data solutions

Software engineering best practices

Tools

Azure Databricks

Delta Lake

Azure Data Factory

Azure Synapse

Job description

Job Description:

We are seeking a highly skilled Data Engineer to design, develop, and maintain scalable, secure, and high-performance data solutions. The ideal candidate will be a strong analytical problem solver with the ability to independently design, develop, test, and deliver end-to-end data engineering solutions. The role requires hands-on experience with distributed data processing technologies, modern data platforms, and software engineering best practices, including testing and CI/CD.

Key Responsibilities

• Design, develop, and maintain scalable data ingestion, transformation, and serving pipelines.
• Build and optimize data solutions using Azure Databricks, PySpark, Delta Lake, and Azure data services.
• Develop data integration workflows using Azure Data Factory and other Azure-native services.
• Design and implement data models that support analytics, reporting, and downstream applications.
• Implement robust data quality, validation, monitoring, and observability solutions.
• Develop automated testing strategies and maintain high code quality standards.
• Implement CI/CD pipelines and deployment automation using Azure DevOps.
• Collaborate with architects, product owners, analysts, and business stakeholders to deliver data products.
• Troubleshoot production issues and continuously improve platform reliability, scalability, and performance.
• Ensure adherence to data governance, security, and engineering best practices.

Mandatory Skills

• Programming skills in Python, Java, Scala, or another modern programming language.
• Advanced SQL skills.
• Hands-on experience with Apache Spark / PySpark.
• Experience with Databricks or similar distributed data processing platforms.
• Experience building scalable data pipelines and data integration solutions.
• Good understanding of data warehousing and dimensional modelling concepts.
• Proficiency with Git or any source control practices.
• Experience with automated testing frameworks such as PyTest, JUnit, or equivalent.
• Strong analytical, troubleshooting, and problem-solving skills.
• Knowledge of software engineering best practices, including code reviews, testing, and reusable design patterns.
• Experience developing and supporting production-grade data solutions.

Preferred / Good-to-Have

• Experience with Azure could.
• Experience with Databricks, Delta Lake, Delta Live Tables (DLT), or equivalent technologies.
• Experience with Azure Data Factory (ADF), Azure Synapse, AWS Glue, EMR, or comparable cloud-native data services.
• Experience with data quality, observability, and monitoring frameworks.
• Cloud or Databricks certifications.
• Experience working in Agile delivery environments.