Data Engineer

KWAN; Lisboa; On-site; EUR 45,000 - 75,000; Full time

Benefits offered by this job

Competitive salary

Awesome benefits

Career Growth Support

Empowering work culture

Job summary

A leading tech company in Lisbon seeks a passionate Data Engineer to build and refine a core lakehouse infrastructure. The ideal candidate will have over 3 years of experience and skills in Python, PySpark, and data pipelines. This position offers a competitive salary, benefits, and a dedicated People Experience Partner to support your career growth. Be part of a respectful and empowering work culture that values your aspirations and well-being.

Qualifications

• 3+ years of experience in Data Engineering.
• Strong coding skills in Python and PySpark.
• Experience with modern data lake formats like Iceberg, Hudi, or Delta Lake.
• Understanding of Apache Kafka and event-driven architecture.
• Experience constructing and managing large-scale data pipelines.
• Advanced SQL capabilities.
• Experience with modern workflow schedulers like Airflow.

Responsibilities

• Lead the transition of data workloads to Apache Iceberg.
• Engineer and deploy reliable batch and real-time data pipelines.
• Work with Backend Engineering to integrate APIs.
• Design and implement a data lineage and governance strategy.

Skills

Python

PySpark

Apache Kafka

SQL

Data Modeling

Airflow

Flink

Tools

Apache Iceberg

DataHub

Delta Lake

Apache Kafka

Airflow

Dagster

Job description

We are looking for a Data Engineer who's eager to embark on an exciting new opportunity! 💪

We connect IT professionals with projects that match their skills, professional experience, and goals, then we offer a career progression program, delivered by in-house specialists - our People Experience Partners - to guide them through our journey with us.

🛠 The Challenge:

• Lead the transition of our current data workloads to Apache Iceberg, building and refining our core lakehouse infrastructure from the ground up.
• Engineer and deploy highly reliable batch and real-time streaming data pipelines using Spark and Flink.
• Work hand-in-hand with Backend Engineering to seamlessly integrate APIs and establish reliable, formal data contracts.
• Help design and implement a comprehensive data lineage and governance strategy using DataHub.

🔎What you bring to the table:

• 3+ years of experience in Data Engineering;
• Strong, proven coding skills in Python and PySpark.
• Practical, hands-on experience with modern data lake formats like Iceberg, Hudi, or Delta Lake.
• A deep understanding of Apache Kafka and event-driven architecture.
• A proven track record of constructing and managing large-scale data pipelines.
• Advanced SQL capabilities paired with a solid foundation in data modeling techniques.
• Experience utilizing modern workflow schedulers such as Airflow, Dagster, or equivalent tools.

👉 Do you meet most of the position requirements? Then grab this opportunity and apply now! 🤩💪

✨ What we offer

• A project that matches your skills and ambitions, as well as your preferences for working policies and culture.
• A competitive salary with awesome benefits and opportunities to leverage your knowledge and network to earn additional income.
• An empowering and respectful work culture enriched with social and learning events.
• A People Experience Partner specially assigned to you - your go-to career guide, responsible for supporting your growth, facilitating training, and ensuring your work-life balance at KWAN.

“At KWAN, they make sure that I feel comfortable with the client I’m working for and that what I’m doing aligns with my career aspirations.” - Luis Caldeira, DevOps @ KWAN