Data Engineer

Intact Insurance; India; On-site; INR 1,500,000 - 2,300,000; Full time

Job summary

Intact Insurance is seeking a skilled Data Engineer to design, develop, and maintain scalable, secure data solutions in a distributed data ecosystem. You will work with Azure Databricks, Delta Lake, and Azure Data Factory to deliver end-to-end data products.

You will design data models for analytics and reporting, implement robust data quality and observability, and collaborate with cross-functional teams to ensure reliable, scalable solutions. CI/CD and modern software practices are expected.

Qualifications

• Must have hands-on experience with distributed data processing and modern data platforms.
• Proficient in Python/Scala/Java and advanced SQL.
• Experience with PyTest/JUnit and CI/CD workflows.
• Experience with Databricks or similar distributed data processing platforms.
• Strong ability to design scalable data pipelines and data integration solutions.

Responsibilities

• Design, develop, and maintain scalable data ingestion, transformation, and serving pipelines.
• Build and optimize data solutions using Azure Databricks, PySpark, Delta Lake, and Azure data services.
• Develop data integration workflows using Azure Data Factory and other Azure-native services.
• Design data models supporting analytics, reporting, and downstream applications.
• Implement data quality, validation, monitoring, and observability solutions.
• Develop automated testing strategies and maintain high code quality standards.
• Implement CI/CD pipelines and deployment automation using Azure DevOps.
• Collaborate with architects, product owners, analysts, and business stakeholders to deliver data products.
• Troubleshoot production issues and improve platform reliability, scalability, and performance.
• Ensure adherence to data governance, security, and engineering best practices.

Skills

Python/Java/Scala

Advanced SQL

PySpark

Apache Spark

Data pipelines

Data warehousing

Git

Automated testing (PyTest/JUnit)

Troubleshooting

CI/CD

Software engineering best practices

End-to-end data solutions

Tools

Databricks

Delta Lake

Azure Data Factory

Azure Databricks

Azure Synapse

AWS Glue

Delta Live Tables

Job description

Position type

Permanent

Role Summary

We are seeking a highly skilled Data Engineer to design, develop, and maintain scalable, secure, and high-performance data solutions. The ideal candidate will be a strong analytical problem solver with the ability to independently design, develop, test, and deliver end-to-end data engineering solutions. The role requires hands-on experience with distributed data processing technologies, modern data platforms, and software engineering best practices, including testing and CI/CD.

Key Responsibilities

• Design, develop, and maintain scalable data ingestion, transformation, and serving pipelines.
• Build and optimize data solutions using Azure Databricks, PySpark, Delta Lake, and Azure data services.
• Develop data integration workflows using Azure Data Factory and other Azure-native services.
• Design and implement data models that support analytics, reporting, and downstream applications.
• Implement robust data quality, validation, monitoring, and observability solutions.
• Develop automated testing strategies and maintain high code quality standards.
• Implement CI/CD pipelines and deployment automation using Azure DevOps.
• Collaborate with architects, product owners, analysts, and business stakeholders to deliver data products.
• Troubleshoot production issues and continuously improve platform reliability, scalability, and performance.
• Ensure adherence to data governance, security, and engineering best practices.

Mandatory Skills

• Programming skills in Python, Java, Scala, or another modern programming language.
• Advanced SQL skills.
• Hands-on experience with Apache Spark / PySpark.
• Experience with Databricks or similar distributed data processing platforms.
• Experience building scalable data pipelines and data integration solutions.
• Good understanding of data warehousing and dimensional modelling concepts.
• Proficiency with Git or any source control practices.
• Experience with automated testing frameworks such as PyTest, JUnit, or equivalent.
• Strong analytical, troubleshooting, and problem-solving skills.
• Knowledge of software engineering best practices, including code reviews, testing, and reusable design patterns.
• Experience developing and supporting production-grade data solutions.

Preferred / Good-to-Have

• Experience with Azure could.
• Experience with Databricks, Delta Lake, Delta Live Tables (DLT), or equivalent technologies.
• Experience with Azure Data Factory (ADF), Azure Synapse, AWS Glue, EMR, or comparable cloud-native data services.
• Experience with data quality, observability, and monitoring frameworks.
• Cloud or Databricks certifications.
• Experience working in Agile delivery environments.