Data Engineer: Snowflake Migrations & Airflow Modernization

AgileEngine; Portugal; Hybrid; EUR 45,000 - 70,000; Full time

Benefits offered by this job

Professional growth

Competitive USD-based compensation

Flextime

Hybrid/remote work options

Projects with Fortune 500 clients

Job summary

AgileEngine is seeking a Data Engineer to lead large-scale migrations to Snowflake and optimize Airflow workflows within a dynamic data platform team. You will migrate about 1,000 datasets, adapt pipelines to Snowflake, and ensure data quality while coordinating across time zones, including Pacific Time constraints, to accelerate modernization by March 2027.

The role requires Python, SQL, Snowflake, Airflow, Kafka, Spark, Iceberg, Parquet, and AWS S3 experience, with a focus on reliability and

Qualifications

• 3+ years of Python development experience.
• Advanced SQL skills with complex joins and window functions.
• Snowflake data modeling expertise.
• Apache Airflow experience for development and testing.
• Iceberg and Parquet data formats proficiency.
• AWS S3 data lakes and lifecycle policies.
• Testing and debugging code effectively.
• 2+ years with large-scale Airflow/Spark data.
• Overlap until 12 PM Pacific Time.

Responsibilities

• Support a large-scale data platform modernization effort.
• Migrate approximately 1,000 datasets to Snowflake.
• Adapt pipelines to Snowflake following established architecture.
• Ensure data quality and reliability during migration.
• Support workflow scheduling and monitoring in Apache Airflow.
• Execute migrations including Airflow 2 to 3 and Iceberg move.
• Coordinate across time zones to accelerate modernization by March 2027.

Skills

Python

SQL

Data engineering fundamentals

Problem solving

Communication

Tools

Snowflake

Airflow

Kafka

Spark/PySpark

Iceberg

Parquet

AWS S3

Testing/debugging

Job description

AgileEngine is seeking a Data Engineer to lead large-scale migrations to Snowflake and optimize Airflow workflows within a dynamic data platform team. You will migrate about 1,000 datasets, adapt pipelines to Snowflake, and ensure data quality while coordinating across time zones, including Pacific Time constraints, to accelerate modernization by March 2027.

The role requires Python, SQL, Snowflake, Airflow, Kafka, Spark, Iceberg, Parquet, and AWS S3 experience, with a focus on reliability and