Senior Data Quality Engineer: AI‑Driven Data Validation & Automation

Optum; Raleigh (NC); On-site; USD 91,700 - 163,700; Full time

Benefits offered by this job

Telecommuting from anywhere in the U.S

Comprehensive benefits

401k contribution

Job summary

Optum seeks a data quality engineer to design and implement scalable quality frameworks across enterprise data domains. You will automate checks with SQL, Python, and PySpark in a Databricks environment, validating Power BI datasets and dashboards for accuracy.

This role emphasizes AI-assisted quality practices and integration with ETL/ELT pipelines while collaborating with analytics teams to ensure trusted data for decision making. Remote across the U.S. is supported.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or IT-related field
• 5+ years in data quality engineering, quality engineering, or data engineering with automation focus
• 5+ years SQL experience for profiling, reconciliation, validation
• 5+ years automating data quality in ETL/ELT workflows
• 3+ years Python and PySpark, building reusable notebooks
• 3+ years Databricks experience supporting large data pipelines
• 2+ years validating Power BI datasets, dashboards, KPIs against sources

Responsibilities

• Design, build, and maintain scalable data quality frameworks across data domains
• Develop and automate data quality checks using SQL, Python, PySpark in Databricks
• Implement validation and anomaly detection across batch and streaming pipelines
• Embed automated quality checks into ETL/ELT to prevent defective data
• Create reusable data quality components and libraries for reuse across teams
• Validate Power BI datasets, semantic models, and dashboards for accuracy
• Partner with analytics teams to validate Power BI measures against sources
• Monitor data quality metrics and report trends and issues
• Perform root cause analysis with upstream/downstream teams to fix issues
• Leverage GenAI for rule generation, test cases, anomaly detection, and summaries
• Enable AI-driven data observability to identify drift and schema changes
• Support governance by documenting quality controls and dashboard certifications
• Continuously improve data quality practices with automation and AI-driven enhancements
• Design, develop, and deploy AI-powered solutions addressing complex business challenges

Skills

Data quality engineering

SQL

Python

PySpark

Databricks

Power BI

ETL/ELT automation

AI/GenAI awareness

Education

Bachelor's degree in Computer Science/Engineering/IT

Tools

Databricks

Power BI

Job description

Optum seeks a data quality engineer to design and implement scalable quality frameworks across enterprise data domains. You will automate checks with SQL, Python, and PySpark in a Databricks environment, validating Power BI datasets and dashboards for accuracy.

This role emphasizes AI-assisted quality practices and integration with ETL/ELT pipelines while collaborating with analytics teams to ensure trusted data for decision making. Remote across the U.S. is supported.