Data Engineer - Snowflake Migrations & Airflow Expert

AgileEngine, LLC.; Aveiro; Hybrid; EUR 80,000 - 134,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation

Exciting projects

Flextime

Job summary

AgileEngine is seeking a Data Engineer to drive large-scale data platform migrations to Snowflake and ensure reliable Airflow workflows. You will work within the data engineering function to migrate ~1,000 datasets, adapt pipelines for Snowflake, and guarantee data quality during the modernization effort.

The role requires strong Python development, advanced SQL, and hands-on experience with Airflow, Spark, Iceberg, Parquet, and AWS S3.

Qualifications

• 3+ years of hands-on Python development experience.
• Advanced SQL, including complex joins, CTEs, window functions, and performance tuning.
• Snowflake data modeling.
• Apache Airflow — developing and testing pipelines in Airflow and Python.
• Iceberg and Parquet data formats.
• AWS experience, particularly S3 (data lakes, partitioning, and lifecycle policies).
• Testing strategy experience, with the ability to hands-on debug and test code.
• At least 2 years of industry experience working with large-scale datasets, including Airflow and Spark.
• Overlap until 12 PM Pacific Time.

Responsibilities

• Work embedded within the data engineering function, partnering closely with the data platform team to support a large-scale data platform modernization effort.
• Execute the migration of approximately 1,000 datasets to Snowflake.
• Adapt existing pipelines to Snowflake, following the core architecture and technical approach defined by the internal team.
• Ensure data quality and reliability throughout the migration process.
• Support workflow scheduling and monitoring in Apache Airflow 3.
• Execute time-sensitive migrations, including Airflow 2 to 3, Confluent region migration, and the move to Iceberg.
• Help accelerate the migration initiatives targeted for completion by the end of March 2027.

Skills

Python development

Advanced SQL

Large-scale data processing

Airflow development

Spark

Data pipeline design

Testing/debugging

Data migration

AWS data lakes

Tools

Snowflake

Apache Airflow

Iceberg

Parquet

Spark

AWS S3

Job description

AgileEngine is seeking a Data Engineer to drive large-scale data platform migrations to Snowflake and ensure reliable Airflow workflows. You will work within the data engineering function to migrate ~1,000 datasets, adapt pipelines for Snowflake, and guarantee data quality during the modernization effort.

The role requires strong Python development, advanced SQL, and hands-on experience with Airflow, Spark, Iceberg, Parquet, and AWS S3.