Data Engineer ID94415

AgileEngine, LLC.; Madrid; Hybrid; EUR 60,000 - 120,000; Full time

Benefits offered by this job

Professional growth

Competitive compensation

Exciting projects

Flextime

Job summary

AgileEngine is seeking a Data Engineer to execute large-scale data platform migrations to Snowflake and support reliable Airflow workflows, based in Madrid. You will collaborate with the data platform team on a modernization effort and ensure robust data pipelines.

The role requires 3+ years Python development, advanced SQL, Snowflake data modeling, and AWS S3 experience. You will migrate datasets, adapt pipelines, test thoroughly, and help accelerate the move to Iceberg, with a focus on data

Qualifications

• 3+ years of hands-on Python development
• Advanced SQL with complex joins, CTEs, window functions and tuning
• Snowflake data modeling and Snowflake migrations experience
• Apache Airflow development and testing pipelines in Python
• Experience with Iceberg and Parquet data formats
• AWS experience, particularly S3 (data lakes, partitioning and lifecycle policies)
• Testing strategy experience with hands-on debugging/test code
• At least 2 years industry experience with large-scale datasets, including Airflow and Spark
• Overlap until 12 PM Pacific Time

Responsibilities

• Thoughtfully migrate and adapt pipelines to Snowflake
• Support large-scale data platform modernization efforts
• Ensure data quality and reliability during migrations
• Assist with workflow scheduling and monitoring in Apache Airflow
• Execute time-sensitive migrations (Airflow 2 to 3, Confluent region migration, Iceberg move)
• Contribute to accelerating migration initiatives through end of March 2027

Skills

Python development

Advanced SQL

Testing strategy

Large-scale data experience

Tools

Snowflake

Apache Airflow

Spark

Iceberg

Parquet

AWS S3

Job description

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

WHY JOIN US

If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!

ABOUT THE ROLE

We are looking for a Data Engineer to execute large-scale data platform migrations to Snowflake and support reliable Airflow workflows.

MUST HAVES

• - Strong software engineering fundamentals.
• - 3+ years of hands-on Python development experience.
• - Advanced SQL, including complex joins, CTEs, window functions, and performance tuning.
• - Snowflake data modeling.
• - Apache Airflow — developing and testing pipelines in Airflow and Python.
• - Iceberg and Parquet data formats.
• - AWS experience, particularly S3 (data lakes, partitioning, and lifecycle policies).
• - Testing strategy experience, with the ability to hands-on debug and test code.
• - At least 2 years of industry experience working with large-scale datasets, including Airflow and Spark.
• - Overlap until 12 PM Pacific Time.

NICE TO HAVES

• - Hands-on experience with Snowflake migrations (data warehouse modernization, performance and cost optimization).
• - Experience with Apache Airflow migrations (version upgrades, executor changes, or platform migrations).

WHAT YOU WILL DO

• - Work embedded within the data engineering function, partnering closely with the data platform team to support a large-scale data platform modernization effort.
• - Execute the migration of approximately 1,000 datasets to Snowflake.
• - Adapt existing pipelines to Snowflake, following the core architecture and technical approach defined by the internal team.
• - Ensure data quality and reliability throughout the migration process.
• - Support workflow scheduling and monitoring in Apache Airflow 3.
• - Execute time-sensitive migrations, including Airflow 2 to 3, Confluent region migration, and the move to Iceberg.
• - Help accelerate the migration initiatives targeted for completion by the end of March 2027.

PERKS AND BENEFITS

• - Professional growth: Mentorship, TechTalks, and personalized growth roadmaps.
• - Competitive compensation: USD-based pay with education, fitness, and team activity budgets.
• - Exciting projects: Modern solutions with Fortune 500 and top product companies.
• - Flextime: Flexible schedule with remote and office options.

Meet Our Recruitment Process

Each step helps us understand your skills and overall fit.

If it’s a match, you’ll receive an offer.