Data Lakehouse Engineer

N Consulting Limited; Warszawa; On-site; PLN 212,404 - 297,366; Full time

Job summary

N Consulting Limited in Warsaw seeks a skilled AWS Data Lake Engineer to design, build, and maintain scalable data lake solutions on AWS. The ideal candidate will have strong experience in cloud-based data engineering and modern data architecture.

Key responsibilities include developing data lake solutions with AWS services, optimizing data pipelines using Python and PySpark, and ensuring data quality. Applicants should be proficient in CI/CD tools and Infrastructure as Code using Terraform.

Qualifications

• Strong experience with AWS data services and cloud architecture.
• Proficiency in Python and PySpark.
• Hands-on experience with Apache Iceberg.

Responsibilities

• Design and develop data lake solutions using AWS services.
• Build and optimize data pipelines using Python and PySpark.
• Implement and manage Apache Iceberg tables.

Skills

AWS data services

Python

PySpark

Apache Iceberg

CI/CD tools

Terraform

Job description

Data Lakehouse Engineer at N Consulting Ltd

Location: Warsaw (Poland)

Job Summary:

We are seeking a skilled AWS Data Lake Engineer to design, build, and maintain scalable data lake solutions on AWS. The ideal candidate will have strong experience in cloud-based data engineering, big data processing, and modern data architecture.

Key Responsibilities:

• Design and develop data lake solutions using AWS services such as IAM, Lambda, EKS, S3, EMR, and MWAA
• Build and optimize data pipelines using Python and PySpark
• Implement and manage Apache Iceberg tables for efficient data storage and querying
• Develop and maintain CI/CD pipelines and infrastructure using Terraform
• Ensure data quality, security, and governance across the platform
• Orchestrate workflows and automate data processing tasks

Required Skills:

• Strong experience with AWS data services and cloud architecture
• Proficiency in Python and PySpark
• Hands-on experience with Apache Iceberg
• Experience with CI/CD tools and Infrastructure as Code (Terraform)
• Knowledge of data quality, data security, and workflow orchestration