Senior Data Engineer, AI-Driven Data Products

World Bank Group; Washington (District of Columbia); Hybrid; USD 140,000 - 190,000; Full time

Job summary

The World Bank Group's IFC unit is seeking an experienced Associate IT Officer, Data and Information Management to lead data engineering and product ownership across analytics and AI use cases. You will design scalable pipelines, modernize legacy workloads to a Databricks lakehouse, and define contracts, quality rules, and adoption metrics for data products.

This senior role blends data engineering with product ownership, requiring hands-on Python/SQL, PySpark, and AI-enabled practices, with

Qualifications

• Master's degree in Computer Science, Engineering, Data Science, or related field.
• 5+ years of relevant data engineering experience in enterprise environments.
• Strong programming in Python and SQL, with hands-on Apache Spark experience.
• Proven experience with the Databricks ecosystem (notebooks, jobs, Delta Lake, Unity Catalog).
• Experience with cloud platforms (Azure, AWS, or GCP) and lakehouse architecture.
• Hands-on use of AI-assisted engineering tools and responsible AI practices.

Responsibilities

• Design, build, and maintain scalable data pipelines (batch/streaming) using Python, SQL, PySpark.
• Lead ETL/ELT, metadata enrichment, and AI-ready prep for analytics and GenAI use cases.
• Develop data models, lakehouse layers, and governance contracts for data products.
• Modernize legacy data workloads onto Databricks lakehouse with AI-assisted methods.
• Define acceptance criteria, product owners, and metrics for data products.
• Collaborate with analytics teams to encode metric meanings and semantic definitions.

Skills

Python

SQL

PySpark

Data modeling

AI tools

Education

Master's degree in Computer Science/Engineering/Data Science

Tools

Databricks

Unity Catalog

CI/CD

Job description

The World Bank Group's IFC unit is seeking an experienced Associate IT Officer, Data and Information Management to lead data engineering and product ownership across analytics and AI use cases. You will design scalable pipelines, modernize legacy workloads to a Databricks lakehouse, and define contracts, quality rules, and adoption metrics for data products.

This senior role blends data engineering with product ownership, requiring hands-on Python/SQL, PySpark, and AI-enabled practices, with