AI Data Platforms and Integration Lead

VerityX Talent; Dubai; In loco; AED 550.000 - 900.000; Tempo pieno

Descrizione del lavoro

VerityX Talent is seeking a senior data infrastructure leader to own the end-to-end delivery of enterprise-grade vector databases, data pipelines, and LLMOps governance for production AI in banking. You will architect, build, test, and harden data platforms connected to core banking systems with strong security and reliability requirements.

You will drive architecture decisions, provisioning, and live production operations, ensuring SLAs for freshness, latency, and uptime while maintaining

Competenze

• 15+ years in data engineering, backend architecture, or MLOps.
• 10+ years building prod AI data infrastructure at scale.
• Experience in regulated or high-stakes environments.
• Deep expertise in vector databases, SQL/NoSQL, and distributed pipelines.
• Strong knowledge of UAE banking regulations and IAM requirements.

Mansioni

• Architecture & Build: create secure, scalable data platforms and integration layers for AI initiatives across channels.
• Pipeline Delivery: develop ingestion/transformation pipelines with zero data leakage and minimal downtime.
• Vector Search & Retrieval: scale vector databases and knowledge graphs with drift-safe embedding updates.
• LLMOps & Compliance: implement monitoring, auditing, drift tracking, and RBAC guardrails in production.
• Adversarial Hardening: design to resist prompt injection and data exfiltration from untrusted sources.

Conoscenze

Vector databases

SQL/NoSQL

Distributed data pipelines

Cloud security

MLOps

Azure

Azure Databricks

Azure Synapse

Azure Data Factory

Terraform/Bicep

RBAC/ IAM

Data governance

Strumenti

Pinecone

Milvus

pgvector

Azure Data Factory

Kafka

Azure Databricks

Power BI

Terraform

Bicep

Azure OpenAI

Descrizione del lavoro

This role owns the data backbone for every AI product in the bank: the pipelines that move and clean data, the vector infrastructure that makes it retrievable at low latency, and the integration layer that connects large language models (LLMs) and agents to core banking systems without compromising security or availability.

This is a build-and-operate mandate. The person in this role is expected to write architecture decisions into working systems — provisioning infrastructure, standing up retrieval pipelines, wiring real-time connections into core banking, and hardening all of it against the failure modes specific to production AI: stale embeddings, retrieval drift, prompt injection through untrusted data, and silent degradation of model outputs.

As AI moves from pilot to embedded infrastructure across FAB's digital channels, this role ensures a system regulators, risk, and customers can actually trust.

Key Mandate:

Own the end-to-end delivery of enterprise-grade vector databases, data pipelines, and LLMOps governance frameworks — architected, built, tested, hardened against adversarial and operational failure modes, and running in production at bank scale.

Key Responsibilities:

• Architecture & Build: Design and build secure, horizontally scalable data platforms and integration layers supporting every AI initiative across Digital Channels, from schema design through deployment and capacity planning.
• Pipeline Delivery: Deliver data ingestion and transformation pipelines and core banking API connections that run without data leakage, duplication, or downtime — validated through automated testing, replay tooling, and continuous monitoring before go-live.
• Vector Search & Retrieval Architecture: Build and scale high-performance vector databases (Pinecone, Milvus, pgvector) and knowledge graphs supporting real-time agent retrieval, with embedding refresh strategies that prevent retrieval drift as source data changes.
• LLMOps & Compliance Engineering: Implement automated monitoring, evaluation harnesses, auditing, model and embedding drift tracking, and security/RBAC guardrails as live production systems — with alerting, not just dashboards.
• Adversarial Hardening: Design data ingestion and retrieval pipelines to resist prompt injection, data poisoning, and unauthorized data exfiltration through agent tool calls, treating every external data source as untrusted by default.
• Risk & Compliance Integration: Work directly with enterprise risk, infosec, and compliance from design stage, producing evidence and controls that pass regulatory sign-off on the first pass rather than after remediation cycles.
• Platform Reliability: Own SLAs for data freshness, retrieval latency, and pipeline uptime, with incident response ownership when production AI systems degrade or fail.

Technical Environment:

• Vector databases: Pinecone, Milvus, pgvector, and evaluation of emerging alternatives
• Data pipelines: streaming and batch ETL/ELT, CDC from core banking systems
• LLMOps tooling: evaluation harnesses, drift detection, prompt/version management, observability stacks
• Cloud-native, containerized deployment with strict IAM, encryption-at-rest and in-transit, and RBAC enforcement
• Integration with core banking APIs under strict change-control and regulatory review

What Success Looks Like:

• Production data pipelines run with measurable SLAs for freshness, latency, and uptime — not best-effort
• Vector infrastructure scales with usage without manual intervention or embedding staleness
• Regulatory and infosec sign-offs achieved on first submission due to controls designed in from the start
• Zero unremediated data leakage or unauthorized access incidents across AI systems under this role's ownership

Ideal Candidate Profile:

• Experience: 15+ years in data engineering, backend architecture, or MLOps, with 10+ years specifically building generative AI data infrastructure at production scale.
• Delivery Record: Has personally taken data infrastructure projects from design through production deployment in a regulated or high-stakes environment — able to speak to specific systems built, scaled, and operated.
• Technical Mastery: Deep, hands-on expertise in vector databases, SQL/NoSQL systems, distributed data pipelines, streaming architectures, and cloud security protocols.
• AI Systems Fluency: Working knowledge of embedding models, retrieval-augmented generation architectures, and the specific failure modes of production LLM systems — drift, hallucination amplification via bad retrieval, and adversarial inputs.
• Banking Acumen: Strong grasp of financial data privacy, regulatory compliance (UAE Central Bank and international standards), and enterprise IAM requirements.

Technical Skills:

• Microsoft Azure, Azure Data Lake Gen2, Azure Synapse Analytics, Microsoft Fabric
• Azure Databricks, Unity Catalog, Delta Lake/Delta Tables, Delta Live Tables (DLT)
• Azure Data Factory (ADF), Event Hub, Service Bus, Kafka
• Azure SQL, Cosmos DB, Azure AI Search, Azure OpenAI
• Apache Spark, PySpark, SQL, Python
• Machine Learning: Azure Machine Learning, MLflow, Feature Store, MLOps, Model Deployment & Monitoring
• Power BI, Semantic Models, Microsoft Purview (Governance, Lineage)
• Lakehouse, Medallion Architecture, Data Mesh, Event-Driven Architecture
• CI/CD, Azure DevOps, Terraform/Bicep
• Security: Entra ID, Key Vault, RBAC, Private Endpoints