AI Data Platform Engineer — Build Lakehouse Pipelines

Enfint; Greater London; On-site; GBP 85,000 - 140,000; Full time

Benefits offered by this job

Training and development opportunities

Firmwide networks

Benefits and wellness programs

Mindfulness programs

Job summary

Goldman Sachs in London seeks a data engineering professional to build and maintain batch and streaming data pipelines on the Lakehouse and AI Data Platform. You will design data models, ensure data quality and collaborate with engineers and data consumers on usable data products.

Candidates should have strong programming in Python/Java, SQL proficiency, and experience with Spark. The role offers training opportunities and a path to technical leadership for senior engineers.

Qualifications

• Bachelor’s or master’s degree in a relevant discipline, or equivalent practical experience
• Strong quantitative skills or data engineering expertise
• Strong hands‑on programming experience in Python or Java
• Good working knowledge of SQL, including troubleshooting, optimisation and data analysis
• Ability to learn new tools, internal platforms and delivery workflows quickly
• Familiarity with version control, testing, release discipline and CI/CD practices
• Understanding of temporal data modelling, schema design, schema evolution and data compatibility
• Understanding of partitioning, clustering and other techniques for improving data performance at scale
• Ability to choose between normalised and denormalised models and between natural and surrogate keys
• Practical approach to data quality, reconciliation and root‑cause analysis
• Experience building or supporting production data pipelines in a collaborative engineering environment
• Experience with distributed data processing frameworks such as Apache Spark
• Working knowledge of JSON, Avro and Parquet
• Nice to have: technical design ownership across multiple datasets or pipeline domains, experience guiding implementation standards and engineering practices, ability to lead delivery for a workstream and support less experienced engineers

Responsibilities

• Build, enhance and support batch and streaming data pipelines on the Lakehouse and AI data platform
• Refactor and modernise existing data flows to improve reliability, performance and maintainability
• Build reusable tooling to improve delivery, consistency and operational support
• Ensure data pipelines are production-ready, well tested and operationally supportable
• Develop raw, refined and curated datasets for analytics, reporting and AI use cases
• Apply data modelling principles to represent business entities, relationships and historical change
• Work with consumers to shape usable, documented data products aligned with business needs
• Implement controls to validate data completeness, accuracy and consistency
• Use reconciliation approaches to validate production outputs and investigate data breaks
• Contribute to standards for testing, monitoring and issue resolution
• Improve testing, monitoring and reconciliation tooling to strengthen platform reliability and delivery
• Work with engineers, platform teams and data consumers to deliver agreed outcomes on time and to quality expectations
• Communicate progress, risks, dependencies and design choices
• For more senior candidates, contribute to technical leadership, task breakdown and support for junior engineers

Skills

Strong quantitative skills

Data engineering expertise

Ability to learn new tools

Education

Bachelor’s or master’s degree in a relevant discipline

Tools

Python

Java

SQL

Apache Spark

JSON

Avro

Parquet

Job description

Goldman Sachs in London seeks a data engineering professional to build and maintain batch and streaming data pipelines on the Lakehouse and AI Data Platform. You will design data models, ensure data quality and collaborate with engineers and data consumers on usable data products.

Candidates should have strong programming in Python/Java, SQL proficiency, and experience with Spark. The role offers training opportunities and a path to technical leadership for senior engineers.