Senior AWS Data Lakehouse Engineer - Scalable Pipelines

TESTQ Technologies; Basildon; On-site; GBP 90,000 - 130,000; Full time

Job summary

TESTQ Technologies is seeking a seasoned Data Lakehouse Engineer to design and optimize scalable lakehouse and data platform solutions. You will work across ingestion, processing, and storage layers to enable data-driven decision making while ensuring reliability and data quality across the ecosystem.

You will collaborate with cross-functional teams to implement governance, security, and performance enhancements, while leveraging modern data tooling and CI/CD practices.

Qualifications

• 7+ years in data engineering in a fast-paced, enterprise-scale environment.
• Bachelor's or Master's degree in Computer Science, Data Engineering or related field.
• Strong SQL and data integration platform skills (IBM Streamsets, Informatica).
• Extensive AWS experience (IAM, Lambda, EKS, S3, EMR, PySpark, MWAA, Lakehouse).
• Experience with Apache Iceberg and Snowflake performance tuning.
• Experience with batch pipelines and streaming technologies (StreamSets, Kafka).

Responsibilities

• Design and implement lakehouse integration architectures with ELT/ETL data flows.
• Build, optimize, and maintain scalable data pipelines for ingestion and storage.
• Implement data governance, security, lineage, and compliance practices.
• Monitor, troubleshoot and fine-tune data workflows and storage for performance.
• Provide production support with observability and proactive issue resolution.
• Collaborate with platform, analytics, and engineering teams on data modeling and storage optimization.
• Leverage Iceberg/Delta patterns for robust lakehouse design.

Skills

AWS Lambda

Data lakehouse

ETL concepts

SQL proficiency

Snowflake

StreamSets

Kafka

PySpark

MWAA

Iceberg

Education

Bachelor's or Master's in CS/Data Engineering

Tools

IBM Streamsets

Informatica

Kafka

Apache Iceberg

Job description

TESTQ Technologies is seeking a seasoned Data Lakehouse Engineer to design and optimize scalable lakehouse and data platform solutions. You will work across ingestion, processing, and storage layers to enable data-driven decision making while ensuring reliability and data quality across the ecosystem.

You will collaborate with cross-functional teams to implement governance, security, and performance enhancements, while leveraging modern data tooling and CI/CD practices.