Data Engineer

Agile; Maryland Heights (MO); Hybrid; USD 69,000 - 94,000; Full time

Job summary

Agile Resources, Inc. is seeking an experienced Data Engineer for a 6-month contract in St. Louis, MO. The role is hybrid—4 days onsite and 1 day remote—focusing on building and maintaining production data pipelines using Python, SQL, and Apache Airflow.

You will design ETL/ELT processes, optimize data delivery, troubleshoot production issues, and support cloud-based data platforms. Strong Docker/Kubernetes, CI/CD, and Agile collaboration are required to succeed.

Qualifications

• 4–7 years of professional Data Engineering experience.
• Strong proficiency with Python and advanced SQL.
• Hands-on experience with Apache Airflow, including development, scheduling, monitoring, and troubleshooting of DAGs.

Responsibilities

• Author, debug, and optimize complex data engineering scripts using Python and advanced SQL.
• Design, develop, schedule, and monitor complex Apache Airflow DAGs and data pipelines.
• Build and maintain ETL/ELT processes, data transformations, and data integration workflows.
• Design and manage SQL-based databases, including schema development, query performance tuning, and data warehousing.
• Improve existing pipelines by resolving recurring failures, eliminating bottlenecks, and optimizing resource utilization.
• Install, configure, and maintain data pipeline utilities, custom Airflow providers, and database connectors.
• Support cloud-based data platform infrastructure and data lake/data warehouse integration efforts.
• Work with containerization technologies such as Docker and Kubernetes.
• Identify opportunities to automate, innovate, and scale data delivery across the organization.
• Perform root cause analysis and incident management for data quality issues, pipeline/DAG failures, and database outages.
• Create and maintain technical documentation covering data architecture, DAG workflows, and operational troubleshooting procedures.
• Collaborate with engineering and technical teams using Agile development methodologies.
• Support CI/CD deployment pipelines for data engineering code and workflows.

Skills

Python

Advanced SQL

Airflow

ETL/ELT

Data Warehousing

Cloud Data Platforms

Docker/Kubernetes

CI/CD

Agile

Tools

Docker

Kubernetes

Job description

Data Engineer Location: St. Louis, MO Work Arrangement: Hybrid – 4 days onsite / 1 day remote Contract: 6 months potential extension Pay Rate: $58.82/hour NO C2C or third parties. Only those authorized to work in the US without sponsorship on a W2 basis will be considered

We are seeking an experienced Data Engineer to join our client in St. Louis, MO on a 6-month contract. This role is ideal for a hands‑on Data Engineer with strong Python, advanced SQL, and Apache Airflow experience who enjoys building, optimizing, and supporting production data pipelines. The successful candidate will play an important role in developing and maintaining reliable data pipelines, improving data delivery, troubleshooting production issues, and supporting modern cloud‑based data infrastructure.

What You’ll Do

• Author, debug, and optimize complex data engineering scripts using Python and advanced SQL.
• Design, develop, schedule, and monitor complex Apache Airflow DAGs and data pipelines.
• Build and maintain ETL/ELT processes, data transformations, and data integration workflows.
• Design and manage SQL‑based databases, including schema development, query performance tuning, and data warehousing.
• Improve existing pipelines by resolving recurring failures, eliminating bottlenecks, and optimizing resource utilization.
• Install, configure, and maintain data pipeline utilities, custom Airflow providers, and database connectors.
• Support cloud‑based data platform infrastructure and data lake/data warehouse integration efforts.
• Work with containerization technologies such as Docker and Kubernetes.
• Identify opportunities to automate, innovate, and scale data delivery across the organization.
• Perform root cause analysis and incident management for data quality issues, pipeline/DAG failures, and database outages.
• Create and maintain technical documentation covering data architecture, DAG workflows, and operational troubleshooting procedures.
• Collaborate with engineering and technical teams using Agile development methodologies.
• Support CI/CD deployment pipelines for data engineering code and workflows.

What We’re Looking For

• 4–7 years of professional Data Engineering experience.
• Strong proficiency with Python and advanced SQL.
• Hands‑on experience with Apache Airflow, including development, scheduling, monitoring, and troubleshooting of DAGs.
• Experience with SQL‑based database management systems, including schema design and query optimization.
• Strong understanding of ETL/ELT, data transformation, data integration, and data pipeline development.
• Experience working with cloud‑based data platforms and data lake/data warehouse environments.
• Experience with Docker and/or Kubernetes.
• Experience with CI/CD and version‑controlled development workflows.
• Understanding of Agile software development methodologies.
• Strong production troubleshooting, root cause analysis, and incident‑management skills.
• Ability to work effectively in a fast‑paced, sense‑of‑urgency environment.

Ideal Background

The strongest candidates will have experience owning production data pipelines from development through deployment and ongoing support, with particular strength in Apache Airflow, Python, SQL, and cloud data platforms. This is a hands‑on engineering role for someone who enjoys solving complex pipeline and data infrastructure problems—not a primarily reporting, analytics, or dashboard‑focused position.