Data Engineer - Automation & Innovation Department

Magentateam; Town of Poland (NY); On-site; USD 48,000 - 64,000; Full time

Benefits offered by this job

Sports packages

Medical care

Life insurance

Development plans

Birthday/anniversary leave

Team-building events

Well-being programs

Job summary

Magentateam is seeking a Data Engineer to build and optimize data pipelines, ingest diverse data sources into a Data Lake, and ensure governance and security across a hybrid work setup.

You will collaborate with analysts and scientists to deliver high-quality data for analytics, while exploring new technologies to improve performance and reliability across multi-country data projects.

Qualifications

• Minimum 3 years of experience in a Data Engineering role.
• Practical experience with Cloud (preferably GCP) services, including BigQuery, Dataflow, Google Cloud Storage (GCS), Data Catalog, and dbt.
• Advanced SQL proficiency, including CTEs and window functions.
• Python coding skills (Pandas/NumPy a plus).
• Experience with Apache Airflow for orchestration and monitoring.
• Experience with Spark, Hadoop, Hive for large-scale data processing.
• ETL/ELT pipelines for batch and streaming data, including Kafka and API ingestion.
• Data integration, data warehouse architectures, data models, and data layers.
• CI/CD pipelines and GitLab for version control and documentation.
• Experience in Agile/Scrum and strong English communication.
• Nice to have: degree in data-related field, ML/MLOps basics, governance, international projects, strong business storytelling, continuous learning.

Responsibilities

• Build and maintain data ingestion processes from various sources into the Data Lake.
• Design, develop, and optimize complex data pipelines for reliable data flow.
• Build, develop, and maintain frameworks for data pipelines.
• Implement end-to-end testing frameworks for data pipelines.
• Collaborate with data analysts and scientists to ensure quality data delivery.
• Ensure robust data governance, security, and compliance practices.
• Explore and implement emerging technologies to improve data pipeline performance.
• Utilize and integrate data from Kafka, MQ, SFTP, databases, APIs, and file shares.

Skills

Cloud (GCP)

BigQuery

Dataflow

GCS (Google Cloud Storage)

Data Catalog

dbt

Advanced SQL

CTEs

Window functions

Python

Pandas

NumPy

Apache Airflow

Apache Spark

Hadoop

Hive

ETL/ELT

Kafka

API ingestion

Data integration

Data warehouse

CI/CD

GitLab

Agile/Scrum

English

Education

Bachelor’s or Master’s degree in Data Science, Statistics, Computer Science, Economics, or a related field

Tools

Apache Airflow

Apache Spark

Hadoop

Hive

Kafka

GitLab

Job description

• Contract: Employment contract
• Working mode: Hybrid (2-3 days a weekn in the office)
• Build and maintain data ingestion processes from various sources into the Data Lake.
• Design, develop, and optimize complex data pipelines for reliable data flow.
• Build, develop, and maintain frameworks that facilitate the construction of data pipelines.
• Implement end-to-end testing frameworks for data pipelines.
• Collaborate with data analysts and scientists to ensure the delivery of quality data.
• Ensure robust data governance, security, and compliance practices.
• Explore and implement emerging technologies to improve data pipeline performance.
• Utilize and integrate data from various source system types, including Kafka, MQ, SFTP, databases, APIs, and file shares

What skills will be appreciated?

• Minimum 3 years of experience in a Data Engineering role.
• Practical experience with Cloud (preferably GCP) services, including BigQuery, Dataflow, Google Cloud Storage (GCS), Data Catalog, and dbt.
• Advanced SQL proficiency, including writing complex queries, query optimization, Common Table Expressions (CTEs), window functions, and related advanced SQL techniques.
• Python coding skills (experience with Pandas and NumPy is a plus)
• Experience working with Apache Airflow for pipeline orchestration, workflow automation, monitoring, data quality validation, alerting, and troubleshooting.
• Experience with Apache Spark, Hadoop, and Hive for large-scale data processing.
• Experience designing, developing, testing, maintaining, and optimizing ETL/ELT pipelines for batch and streaming data processing, including Apache Kafka, API-based data ingestion.
• Experience designing and implementing data integration solutions, data warehouse architectures, data models, and data layers to support analytics and reporting.
• Experience implementing and maintaining CI/CD pipelines and using GitLab for version control and technical documentation.
• Experience working in Agile/Scrum
• Good command of English.

Nice to have:

• Bachelor’s or Master’s degree in Data Science, Statistics, Computer Science, Economics, or a related field.
• Basic understanding of Machine Learning and MLOps principles
• Strong understanding of data governance principles, including metadata management and data quality frameworks.
• Experience in international or multi-country data projectsis preferred.
• Strong communication skills to translate technical findings into business-oriented insights.
• Self-starter with a continuous learning mindsetand strong attention to detail.

Our offer for you:

You will join a diverse, international team of data professionals working across multiple countries. In this role, you will collaborate with both local experts and international colleagues to build a unified, high-performance data infrastructure. We are looking for a Data Engineer who enjoys solving complex integration challenges and wants to have a real impact on how our organization uses data globally.

Benefits for you and your loved ones: sports packages, medical care, life insurance

Limitless growth: WeGrow - individual development plans, webinars and external trainings, Festiwal Rozwoju, access to learning platforms

Celebrate in magenta style: an additional day off for your birthday/name day, anniversary leave, team-building events, internal contests

Benefits tailored to your needs: access to the benefit platform, ZFŚŚ, unlimited phone offer (also for private use), special offer for T-Mobile products, as well as discounts from our partners

Support, comfort, security: free parking spot and gym at the office (Warsaw, Marynarska), modem with Internet for home office, well-being and volunteering programs, Znam Talent - referral program, sports initiatives, DE&I employee resource groups

Our recruitment process

In our recruitment process, you can expect honesty and transparency. We consider each candidacy carefully and fairly, without any exceptions. We don’t look at your race, skin color, sexual orientation, gender identity, nationality, disability status, beliefs, appearance or religion. It’s your competencies that count.

We also guarantee that you will always receive feedback when participating in recruitment for #MagentaTeam