Head of Data Platform & Enterprise AI Engineering

S&P Global; Washington; On-site; USD 185,000 - 265,000; Full time

Benefits offered by this job

Health & Wellness

Flexible Downtime

Continuous Learning

Invest in Your Future

Job summary

S&P Dow Jones Indices is seeking a senior engineering leader to scale a 50+ person data/AI engineering organization and drive enterprise data governance and AI initiatives. You will guide architecture across Delta Lake, Iceberg, and AWS services, ensure data quality and security, and partner with Product, Security, and leadership to map roadmaps.

The role emphasizes governance, multi-cloud data platforms, and turning business requirements into technical execution at scale.

Qualifications

• 15+ years of engineering experience with 5+ years directly managing and scaling large engineering teams at Director or Head of Engineering level, including experience managing managers and Staff/Principal-level individual contributors
• Deep technical expertise in data platform architecture including Delta Lake and Apache Iceberg internals, with proven ability to guide senior engineers on complex architectural trade-offs and design decisions
• Demonstrated experience leading enterprise data management initiatives including data lineage automation, schema management, data governance, and security implementations at petabyte scale across distributed platforms
• Hands-on experience with cloud data platforms such as AWS services (S3, Glue, Lake Formation, EMR), Databricks Data Intelligence Platform, or similar enterprise-scale data infrastructure technologies
• Proven track record integrating operational data catalogs with enterprise metadata platforms, implementing complex access controls (RBAC/CBAC/ABAC), and managing data security across multi-cloud environments
• Strong leadership and communication skills with experience driving organizational transformation, building high-performance engineering cultures, and partnering effectively with cross-functional teams including Product, Security, and executive leadership

Responsibilities

• Lead and scale a large, multi-disciplinary engineering organization of 50+ data engineers, ML engineers, and platform developers, including managing managers and staff-level engineers across cross-functional teams
• Drive the strategic architecture and implementation of enterprise data management and governance, integrating Databricks Unity Catalog with enterprise data catalog systems and AWS-native governance services to ensure automated data lineage, security, and accuracy at petabyte scale
• Oversee the development of production-grade autonomous AI systems using Databricks Mosaic AI, ensuring AI agents are grounded in highly governed, accurate data through robust semantic modeling and RAG architectures
• Guide the technical strategy for zero-copy, multi-platform lakehouse architecture leveraging Delta Lake and Apache Iceberg, with deep integration across AWS cloud data and analytics services including S3, Glue, and Lake Formation
• Partner with Product, Security, and executive leadership to translate business and compliance requirements into actionable engineering roadmaps while fostering a high-performance culture of innovation and engineering excellence
• Champion organizational transformation initiatives that position the company at the forefront of enterprise AI and data platform capabilities, directly impacting business growth and operational excellence

Skills

Large team leadership

Data platform architecture

Delta Lake / Iceberg

AWS cloud platforms

RBAC/ABAC security

Tools

Databricks Unity Catalog

Databricks Mosaic AI

Apache Iceberg

Delta Lake

AWS (S3, Glue, Lake Formation)

Job description

S&P Dow Jones Indices is seeking a senior engineering leader to scale a 50+ person data/AI engineering organization and drive enterprise data governance and AI initiatives. You will guide architecture across Delta Lake, Iceberg, and AWS services, ensure data quality and security, and partner with Product, Security, and leadership to map roadmaps.

The role emphasizes governance, multi-cloud data platforms, and turning business requirements into technical execution at scale.