Hybrid Data Engineer: Airflow, Python & SQL in Cloud

Agile Resources, Inc.; St. Louis (MO); Hybrid; USD 69,000 - 94,000; Full time

Job summary

Agile Resources, Inc. seeks an experienced Data Engineer for a 6-month contract in St. Louis, MO.

Hybrid arrangement: four days onsite, one day remote. You will build, optimize, and support production data pipelines using Python, advanced SQL, and Apache Airflow. Responsibilities include designing ETL/ELT processes, managing SQL databases, monitoring DAGs, troubleshooting issues, and collaborating with cloud data platforms.

Qualifications

• 4-7 years of professional Data Engineering experience.
• Experience with Python and advanced SQL.
• Hands-on experience with Apache Airflow, including development, scheduling, monitoring, and troubleshooting of DAGs.
• Experience with SQL-based database management systems, including schema design and query optimization.
• Experience with cloud-based data platforms and data lake/data warehouse environments.

Responsibilities

• Author, debug, and optimize data engineering scripts using Python and advanced SQL.
• Design, develop, schedule, and monitor Apache Airflow DAGs and data pipelines.
• Build and maintain ETL/ELT processes, data transformations, and data integration workflows.
• Design and manage SQL-based databases, including schema development, query performance tuning, and data warehousing.
• Improve existing pipelines by resolving recurring failures, bottlenecks, and optimizing resource utilization.
• Install, configure, and maintain data pipeline utilities and connectors.
• Collaborate with engineering and technical teams using Agile development methodologies.
• Support CI/CD deployment pipelines for data engineering code and workflows.

Skills

Python programming

SQL optimization

Agile development

Cloud data platforms

Tools

Apache Airflow

Docker

Kubernetes

CI/CD tooling

Git

Job description

Agile Resources, Inc. seeks an experienced Data Engineer for a 6-month contract in St. Louis, MO.

Hybrid arrangement: four days onsite, one day remote. You will build, optimize, and support production data pipelines using Python, advanced SQL, and Apache Airflow. Responsibilities include designing ETL/ELT processes, managing SQL databases, monitoring DAGs, troubleshooting issues, and collaborating with cloud data platforms.