Data Quality Engineer

i4DM; Millersville (MD); On-site; USD 90,000 - 140,000; Full time

Benefits offered by this job

Healthcare coverage

HSA Plan

Life and Disability Insurance

Retirement Savings Plan

Job summary

i4DM seeks an experienced Data Quality Engineer to support the VA ATLAS platform. You will design automated validation, testing, profiling, and monitoring to ensure multi-source data accuracy and reliability across cloud environments.

You will work with Azure Synapse, Databricks, Collibra Data Quality, Profisee, SQL, and Python, building scalable validation pipelines and governances that enable trusted data for Veterans and their families.

Qualifications

• 3+ years in a cloud-based data environment.
• Proficiency in SQL and Python for data querying and processing.
• Experience building automated data quality tests.
• Hands-on experience with Azure Synapse and Databricks.
• Ability to obtain and maintain Public Trust or Suitability/Fitness.

Responsibilities

• Design data validation and QA processes for cloud environments.
• Automate data quality checks and monitoring for anomalies.
• Collaborate on governance and compliance with data engineers and analysts.
• Profile structured and unstructured data for accuracy.
• Build automated testing frameworks for data quality assurance.
• Utilize Azure Synapse and Databricks in ELT workflows.
• Create test plans using JIRA/X-RAY.
• Implement tests with Collibra Data Quality or Profisee.

Skills

SQL

Python

Data profiling

Automated testing

Cloud environments

Test planning

Education

Bachelor’s degree or 4+ years of relevant experience

Tools

Azure Synapse

Databricks

Collibra Data Quality

Profisee

JIRA/X-RAY

Job description

About the Role

We are seeking an experienced Data Quality Engineer to support ATLAS, the VA’s open-architected, multi-cloud enterprise data management and analytics platform. In this role, you will design and automate validation, testing, profiling, and monitoring frameworks that ensure the accuracy, consistency, completeness, and reliability of multi-source data as it is ingested, conditioned, curated, and delivered as consumable enterprise data products. Working with Azure Synapse, Databricks, Collibra Data Quality, Profisee, SQL, Python, and related tools, you will detect anomalies, enforce governance standards, and improve trust in data across interoperating ATLAS platforms. You will collaborate with data engineers, analysts, data scientists, and product teams to provide consistent, secure access to high-quality data that enables evidence-based decisions and improves outcomes for Veterans and their families.

Key Responsibilities

• Design and implement data validation and quality assurance processes for cloud-based environments.
• Automate data quality checks and develop monitoring solutions to detect data anomalies.
• Collaborate with data engineers and analysts to ensure data governance and compliance.
• Analyze and profile structured and unstructured datasets for accuracy and consistency.
• Build and maintain automated testing frameworks to proactively catch data issues.
• Utilize Azure data engineering and ELT tools, including Azure Synapse and Databricks.
• Create and execute comprehensive test plans using tools such as JIRA’s X-RAY.
• Implement and execute test automation using CoTS tooling like Collibra's Data Quality, Microsoft’s Profisee, or custom scripts in Python or Scala.

Minimum Qualifications

• 3+ years of experience working in a cloud-based environment.
• Experience in analyzing and profiling data.
• Proficiency in querying and working with data using SQL and Python.
• Experience in building automated testing frameworks for data quality assurance.
• Hands‑on experience with Azure tools for data engineering and ELT, including Azure Synapse and Databricks.
• Strong knowledge of best testing practices and patterns in cloud environments.
• Ability to work independently within a mid‑sized team and rapidly develop test plans using JIRA's X-RAY.
• Proficiency in implementing automated test plans using Collibra's Data Quality, Microsoft Profisee, or script-based testing with Python or Scala.
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
• Bachelor’s degree or 4+ years of relevant experience in lieu of a degree.
• Experience with Synapse Pipelines, Azure Data Factory, Starburst, Azure Data Lake Service, Azure ML/AI services, and Pyramid Analytics/Power BI for visualization.

Preferred Qualifications

• Familiarity with JIRA and Confluence for project tracking and documentation.
• Ability to integrate testing protocols with monitoring applications such as Dynatrace, Splunk, or Log Analytics.
• Bachelor’s degree in Computer Science or a related field.
• Azure Data Engineer Certification.

Benefits

• Comprehensive healthcare coverage, HSA Plan, Life and Disability Insurance, and Retirement Savings Plan.