Senior Data Platform Engineer – Pipelines & Performance

YLD; United States; Remote; USD 110,000 - 160,000; Full time

Job summary

YLD is a software engineering and design consultancy that helps clients build lasting digital capabilities. They have offices in London, Lisbon, and Porto and focus on attracting, inspiring, developing, and retaining extraordinary people.

We are seeking a data platform engineer to contribute to core infrastructure, develop robust Python code, and design scalable data pipelines using dbt, Kafka, Spark, and AWS services.

Qualifications

• Proven experience writing maintainable, performant Python code.
• Experience building modern data pipelines using dbt, Kafka, Spark, AWS Kinesis, AWS Lambda, and Apache Airflow.
• Deep knowledge of complex SQL with emphasis on CTEs, window functions, and performance.
• Strong problem-solving balancing innovation with pragmatic tech choices.
• Experience in dynamic production environments and managing client expectations.

Responsibilities

• Build core infrastructure software as part of data platform team.
• Instrument systems for performance and enhancement.
• Ensure data offerings are secure and accessible to stakeholders.
• Collaborate with engineering leadership and architects to deliver high-quality solutions.
• Work with clients to manage expectations in production environment.

Skills

Python

SQL

Problem solving

Client management

Production environment

Tools

dbt

Kafka

Spark

AWS Kinesis

AWS Lambda

Apache Airflow

Job description

YLD is a software engineering and design consultancy that helps clients build lasting digital capabilities. They have offices in London, Lisbon, and Porto and focus on attracting, inspiring, developing, and retaining extraordinary people.

We are seeking a data platform engineer to contribute to core infrastructure, develop robust Python code, and design scalable data pipelines using dbt, Kafka, Spark, and AWS services.