Data Engineer ID94415

AgileEngine; Województwo pomorskie; Hybride; PLN 355 000 - 512 000; Plein temps

Avantages offerts par ce poste

Professional growth

Competitive compensation

Exciting projects

Flextime

Résumé du poste

AgileEngine is seeking a Data Engineer to execute large-scale data platform migrations to Snowflake and support reliable Airflow workflows.

You will work within the data engineering function, migrate ~1,000 datasets, adapt pipelines to Snowflake, and ensure data quality. Experience with Airflow, Spark, Kafka, and AWS S3 is required, with a focus on large-scale datasets and English fluency. Flexible remote/office options available.

Qualifications

• 3+ years hands-on Python development.
• Advanced SQL with complex joins, CTEs, window functions, performance tuning.
• Snowflake data modeling.
• Apache Airflow: developing and testing pipelines in Airflow and Python.
• Apache Kafka.
• Apache Spark, including PySpark.
• Iceberg and Parquet data formats.
• AWS experience, especially S3 (data lakes, partitioning, lifecycle).
• Testing strategy experience with hands-on debugging and testing code.
• At least 2 years of industry experience with large-scale datasets, including Airflow and Spark.
• Overlap until 12 PM Pacific Time.
• Upper-intermediate English level.

Responsabilités

• Embed within the data engineering function to support a large-scale data platform modernization.
• Execute migration of ~1,000 datasets to Snowflake.
• Adapt pipelines to Snowflake following defined architecture.
• Ensure data quality and reliability during migration.
• Support workflow scheduling and monitoring in Airflow 3.
• Execute time-sensitive migrations including Airflow 2 to 3, Confluent region migration, and Iceberg move.
• Aim to complete migrations by end of March 2027.

Connaissances

Python development

Advanced SQL

Airflow

Snowflake modeling

Kafka

Spark / PySpark

Iceberg & Parquet

AWS S3

Testing strategy

Large-scale datasets

English level

Overlap until PT

Outils

Snowflake

Airflow

Spark

Parquet

Description du poste

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

WHY JOIN US

If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!

ABOUT THE ROLE

We are looking for a Data Engineer to execute large-scale data platform migrations to Snowflake and support reliable Airflow workflows.

MUST HAVES

• Strong software engineering fundamentals.
• 3+ years of hands-on Python development experience.
• Advanced SQL, including complex joins, CTEs, window functions, and performance tuning.
• Snowflake data modeling.
• Apache Airflow — developing and testing pipelines in Airflow and Python.
• Apache Kafka.
• Apache Spark, including hands-on PySpark experience.
• Iceberg and Parquet data formats.
• AWS experience, particularly S3 (data lakes, partitioning, and lifecycle policies).
• Testing strategy experience, with the ability to hands-on debug and test code.
• At least 2 years of industry experience working with large-scale datasets, including Airflow and Spark.
• Overlap until 12 PM Pacific Time.
• Upper-intermediate English level.

NICE TO HAVES

• Hands-on experience with Snowflake migrations (data warehouse modernization, performance and cost optimization).
• Experience with Apache Airflow migrations (version upgrades, executor changes, or platform migrations).
• Experience using AI tools for development.

WHAT YOU WILL DO

• Work embedded within the data engineering function, partnering closely with the data platform team to support a large-scale data platform modernization effort.
• Execute the migration of approximately 1,000 datasets to Snowflake.
• Adapt existing pipelines to Snowflake, following the core architecture and technical approach defined by the internal team.
• Ensure data quality and reliability throughout the migration process.
• Support workflow scheduling and monitoring in Apache Airflow 3.
• Execute time-sensitive migrations, including Airflow 2 to 3, Confluent region migration, and the move to Iceberg.
• Help accelerate the migration initiatives targeted for completion by the end of March 2027.

PERKS AND BENEFITS

• Professional growth: Mentorship, TechTalks, and personalized growth roadmaps.
• Competitive compensation: USD-based pay with education, fitness, and team activity budgets.
• Exciting projects: Modern solutions with Fortune 500 and top product companies.
• Flextime: Flexible schedule with remote and office options.

Meet Our Recruitment Process

Application → Coding Challenge → Video Interview → Technical Interview or Hiring Manager Interview. Each step helps us understand your skills and overall fit. If it’s a match, you’ll receive an offer.