Data Assurance Lead

Deeplight; Dubai; Hybrid; AED 120,000 - 200,000; Full time

Job summary

A leading AI consultancy in Dubai is seeking a Data Assurance Lead to enhance data trust across client environments. This pivotal role requires designing and implementing a robust Data Assurance Framework, ensuring quality standards are upheld. With a minimum of 8 years of experience, particularly in financial services, the candidate will lead cross-functional teams and automate processes using tools like Soda Core and AWS services. The position offers competitive salary, flexible working arrangements, and opportunities for career advancement.

Benefits

Competitive salary and performance bonuses

Comprehensive health insurance

Professional development support

Flexible working arrangements

Career advancement opportunities

Qualifications

• 8+ years of experience in data engineering or data quality roles, with at least 3 years in data assurance leadership.
• Experience in Lakehouse architectures and Medallion design.
• Proficiency in AWS services (S3, Glue, Athena) and data validation techniques.

Responsibilities

• Implement Soda Core data quality checks across Medallion layers.
• Create automated validation and reconciliation processes.
• Collaborate with Data Factory squads to embed data quality best practices.
• Provide QA on Glue jobs to ensure compliance with the assurance framework.

Skills

Data quality frameworks

AWS services

Analytical skills

Automation

Stakeholder management

Tools

Soda Core

OpenMetadata

Jira

Job description

DeepLight AI is a specialist AI and data consultancy with extensive experience implementing intelligent enterprise systems across multiple industries, with particular depth in financial services and banking. Our team combines deep expertise in data science, statistical modeling, AI/ML technologies, workflow automation, and systems integration with a practical understanding of complex business operations.

We are looking for a Data Assurance Lead to serve as the ultimate guardian of data trust across our clients\' complex Lakehouse environments. In this high-impact consultancy role, you will design, implement, and operationalize a robust Data Assurance Framework from the ground up. Utilizing Soda Core across the Medallion architecture (Bronze, Silver, Gold), you will define rigorous quality rules, automate data contracts, and integrate scorecards into OpenMetadata.

As a consultant, you are a strategic advocate for data reliability. You must possess the communication mastery to explain the value of data assurance to stakeholders, lead cross-functional squads through migration targets, and embed a culture of quality within AWS Glue and Data Factory workflows. If you have 8+ years of experience—ideally within the Financial Services or Banking sectors—and thrive in fast-paced, Agile environments where you can automate the complex and influence senior stakeholders, this is your next challenge.

Responsibilities

• Data Quality Framework Implementation
• Automation & Governance
• Monitoring & Incident Management
• Collaboration & Training

• Data Quality Framework Implementation: Implement Soda Core data quality checks across all Medallion layers (Bronze, Silver, Gold). Define and maintain a data quality rules library, covering: Schema validation (completeness, uniqueness, data types); Business rule validation (referential integrity, value ranges); Reconciliation checks (source-to-target, row counts, checksums); Freshness and timeliness SLAs as per data contracts.
• Automation & Governance: Implement automated data contracts using Soda and secure sign-off with producers and consumers. Create automated validation and reconciliation processes. Establish data quality scorecards in OpenMetadata, integrated with Soda. Retrofit assurance checks across all live data flows. Develop automated regression test suites for critical flows.
• Monitoring & Incident Management: Configure data quality alerts and thresholds. Define escalation procedures for data incidents. Collaborate with operational teams for alerting and monitoring. Analyze data issues, implement countermeasures, and optimize check performance.
• Collaboration & Training: Work closely with Data Factory squads to embed data quality best practices. Provide QA on Glue jobs to ensure compliance with the assurance framework. Maintain comprehensive documentation and runbooks. Train teams on data quality standards and tools.

Qualifications

• Strong experience with data quality frameworks and tools (Soda Core or similar).
• Hands-on experience with Lakehouse architectures and Medallion design.
• Proficiency in AWS services (S3, Glue, Athena) and data validation techniques.
• Strong analytical and problem-solving skills.
• 8+ years experience in data engineering or data quality roles, with at least 3 years in data assurance leadership.

• You will also have experience in: identifying ways to automate work and repetitive tasks; dealing with data contracts, metadata management, and lineage tools (OpenMetadata preferred); building automated validation and reconciliation pipelines; communicating and stakeholder management; leading cross-functional initiatives and drive adoption; working in a fast-paced environment and delivering aggressive migration targets; working with Jira and agile way of working.

Benefits

• Competitive salary and performance bonuses
• Comprehensive health insurance
• Professional development and certification support
• Opportunity to work on cutting-edge AI projects
• Flexible working arrangements
• Career advancement opportunities in a rapidly growing AI company

This position offers a unique opportunity to shape the future of AI implementation while working with a talented team of professionals at the forefront of technological innovation. The successful candidate will play a crucial role in driving our company\'s success in delivering transformative AI solutions to our clients.

At DeepLight AI, we recognise that diversity drives innovation. We are committed to fostering an inclusive environment where individuals with different thinking styles can thrive and contribute their unique strengths to our specialised AI and data solutions.

Our goal is to ensure our application and interview process is accessible, predictable, and fair for all candidates. If you require any specific adjustments to the application process, or if you require any reasonable adjustments should you be successful in being processed to the interview stage, please do let us know. This information will be kept strictly confidential and will not impact hiring decisions.