Lead Data and BI Engineer

Visionary Tech Services LLC; Abu Dhabi; On-site; AED 120,000 - 200,000; Full time

Job summary

A technology innovation firm in Abu Dhabi is seeking a Lead Data and BI Engineer to oversee the full data and BI stack. This role involves managing processes from data ingestion to dashboard delivery, while leading a team of engineers and ensuring high standards of data governance. Ideal candidates will have significant experience in data engineering and BI tools, alongside strong leadership skills.

Qualifications

• 8+ years across data engineering and BI engineering.
• Proven experience with semantic models and metrics layers.
• Strong data governance, quality practices and API design skills.

Responsibilities

• Own end-to-end data and BI stack from ingestion to dashboards.
• Lead design and delivery of scalable ELT/ETL pipelines.
• Line-manage Associate and Senior Data Engineers and BI Engineers.

Skills

Data governance and quality practices

API design

Leadership

Real-time analytics

Scalable ELT/ETL pipelines

Education

Bachelor’s or Master’s in CS, Software Engineering, Data Engineering, or related

Tools

Azure Data Factory

Databricks

Airflow

Power BI

Tableau

Job description

The Lead Data and BI Engineer makes trusted, timely data and insight delivery for AI. Reporting to the Head of AI and Data Engineering, this role owns the end‑to‑end data and BI stack from ingestion and transformation to semantic layers, metrics, and dashboards and line‑manages Associate and Senior Data Engineers and BI Engineers to ship secure, governed, and reusable data products at scale.

Requirements

• Bachelor’s or Master’s in CS, Software Engineering, Data Engineering, or related.
• 8+ years across data engineering and BI engineering with ownership of production data platforms, pipelines, and analytics delivery.
• Deep hands‑on with batch and streaming patterns(ELT / ETL, CDC, event streams), orchestration (such as Azure Data Factory, Databricks, and Airflow), and Lakehouse stacks (such as Delta / Snowflake / BigQuery) on Azure / AWS / GCP.
• Proven experience with semantic models and metrics layers (dbt + semantic layer, MetricFlow, or equivalent),dimensional modeling, and KPI frameworks powering BI tools (such as Power BI / Tableau).
• Strong data governance and quality practices : lineage, cataloging, PII handling, access controls, testing and observability (such as Great Expectations / Open Lineage / Data Hub), cost and performance optimization.
• API design and data product mindset : well‑defined interfaces, contracts, SLAs, and reusable components; experience enabling self‑serve analytics and domain data ownership.
• Leadership : line management, hiring, coaching, code and model review, operating standards, and cross‑functional delivery with AI, Product, Security, and Platform teams.
• Nice to have : real‑time analytics, metric stores, headless BI, dbt Semantic Layer, Microsoft Fabric; familiarity with RAI / privacy controls for AI and analytics.
• Strategy and ownership
• Own the data and BI architecture and roadmap across ingestion, transformation, semantic layers, and presentation.
• Define standards for modeling, testing, observability, CI / CD, and access controls; ensure reuse and documentation.
• Data engineering
• Lead design and delivery of scalable ELT / ETL, CDC, and streaming pipelines; implement data contracts and SLAs.
• Optimize reliability, performance, and cost on cloud data platforms; enforce data quality gates and lineage.
• BI engineering and semantic modeling
• Establish conformed dimensions and governed metrics; implement a semantic layer and metric definitions used across BI tools.
• Lead build of analytics datasets and dashboards for exec and operational use; standardize KPI frameworks and reporting rhythms.
• Governance and security
• Embed governance‑by‑design : cataloging, lineage, RBAC / ABAC, PII handling, and audit trails; partner with Security and RAI.
• Set data validation, testing, and monitoring policies; act on data incidents with clear runbooks.
• Collaboration and enablement
• Partner with AI / ML Engineering to supply reliable features and serve model outputs back into the warehouse / lake house.
• Enable self‑serve analytics with curated data products, documentation, and training for analysts and business teams.
• Leadership and people
• Line‑manage Associate and Senior Data Engineers and BI Engineers : coach, set goals, review designs and code, and grow capabilities.
• Drive continuous improvement in developer experience, pipeline reliability, and time‑to‑insight via templates and reference implementations.
• Builds reliable data, analytics and BI data products that business teams trust for decisions. Standardizes models, metrics, and pipelines to improve performance, governance, and time‑to‑insight.