Data Quality Engineer

Insight Global; Bengaluru; Hybrid; INR 2,500,000 - 4,000,000; Full time

Job summary

Insight Global in Bengaluru, India, is seeking a highly technical Data Quality Engineer to support data validation across Azure/Databricks platforms. As a senior developer, you will drive data quality testing, automation, and governance in large-scale enterprise environments.

The role focuses on end-to-end validation for pipelines on Azure Data Factory, Databricks, and ADLS, with Python and Spark-based validation, SLAs, KPIs, and best practices to ensure data health and trust across the

Qualifications

• 6-8+ years experience in data quality engineer, SDET, data engineering testing, etc.
• Strong hands-on experience with Azure Data Factory, Databricks (PySpark), and SQL for data quality validation
• Proven experience building automated data quality frameworks from scratch
• Strong understanding of data governance, lineage, and metadata
• Experience integrating with enterprise systems (ERP, CRM, MDM, etc.)

Responsibilities

• Design and implement end-to-end data validation strategies for pipelines built on Azure Data Factory, Databricks (Delta Lake), and ADLS
• Build and perform large-scale data validation using Python and Spark-based validation frameworks
• Validate data movement, orchestration workflows, and failure handling
• Define and implement data quality SLAs and KPIs along with tracking data/pipeline health
• Drive best practices for data quality and testing standards

Skills

Data quality engineering

SDET

Data testing

Python

Spark

SQL

Data governance

Data lineage

Metadata

Tools

Azure Data Factory

Databricks PySpark

dbt

Azure DevOps

GitHub Actions

Job description

Job Description

:

This role is a highly technical Data Quality Engineer supporting enterprise data validation and quality engineer efforts across the Azure/Databricks platform. As a senior level developer, this role requires expertise in data testing, automation, distributed systems, and modern data architectures. Other key responsibilities include:

• Design and implement end-to-end data validation strategies for pipelines built on Azure Data Factory, Databricks (Delta Lake), and ADLS


• Build and perform large-scale data validation using Python and Spark-based validation frameworks


• Validate data movement, orchestration workflows, and failure handling


• Define and implement data quality SLAs and KPIs along with tracking data/pipeline health


• Drive best practices for data quality and testing standards

Must Haves

• 6-8+ years' experience in data quality engineer, SDET, data engineering testing, etc.


• Strong hands-on experience with Azure Data Factory, Databricks (PySpark), and SQL for data quality validation


• Proven experience building automated data quality frameworks from scratch


• Strong understanding of data governance, lineage, and metadata


• Experience integrating with enterprise systems (ERP, CRM, MDM, etc.)

Plusses

• Data observability platforms (Monte Carlo, Soda, Datadog)


• Experience building CI/CD data testing pipelines (Azure DevOps / GitHub Actions)


• Streaming Technologies (Kafka, Event Hub)


• Hands-on experience with dbt (Data Built Tool)


• Azure Data Engineer (DP-203) Certification or Databricks Certified Professional


• Experience within supply chain or large enterprise data environments