Data Engineer

Agile Resources, Inc.; St. Louis (MO); Hybrid; USD 69,000 - 94,000; Full time

Job summary

Agile Resources, Inc. seeks an experienced Data Engineer for a 6-month contract in St. Louis, MO.

Hybrid arrangement: four days onsite, one day remote. You will build, optimize, and support production data pipelines using Python, advanced SQL, and Apache Airflow. Responsibilities include designing ETL/ELT processes, managing SQL databases, monitoring DAGs, troubleshooting issues, and collaborating with cloud data platforms.

Qualifications

• 4-7 years of professional Data Engineering experience.
• Experience with Python and advanced SQL.
• Hands-on experience with Apache Airflow, including development, scheduling, monitoring, and troubleshooting of DAGs.
• Experience with SQL-based database management systems, including schema design and query optimization.
• Experience with cloud-based data platforms and data lake/data warehouse environments.

Responsibilities

• Author, debug, and optimize data engineering scripts using Python and advanced SQL.
• Design, develop, schedule, and monitor Apache Airflow DAGs and data pipelines.
• Build and maintain ETL/ELT processes, data transformations, and data integration workflows.
• Design and manage SQL-based databases, including schema development, query performance tuning, and data warehousing.
• Improve existing pipelines by resolving recurring failures, bottlenecks, and optimizing resource utilization.
• Install, configure, and maintain data pipeline utilities and connectors.
• Collaborate with engineering and technical teams using Agile development methodologies.
• Support CI/CD deployment pipelines for data engineering code and workflows.

Skills

Python programming

SQL optimization

Agile development

Cloud data platforms

Tools

Apache Airflow

Docker

Kubernetes

CI/CD tooling

Git

Job description

Location: St. Louis, MO

Work Arrangement: Hybrid - 4 days onsite / 1 day remote

Contract: 6 months potential extension

Pay Rate: $58.82/hour

No C2C or third parties. Only those authorized to work in the US without sponsorship will be considered. W2 only.

We are seeking an experienced Data Engineer to join our client in St. Louis, MO on a 6-month contract. This role is ideal for a hands-on Data Engineer with strong Python, advanced SQL, and Apache Airflow experience who enjoys building, optimizing, and supporting production data pipelines.

The successful candidate will play an important role in developing and maintaining reliable data pipelines, improving data delivery, troubleshooting production issues, and supporting modern cloud-based data infrastructure.

What You'll Do

• Author, debug, and optimize complex data engineering scripts using Python and advanced SQL.


• Design, develop, schedule, and monitor complex Apache Airflow DAGs and data pipelines.


• Build and maintain ETL/ELT processes, data transformations, and data integration workflows.


• Design and manage SQL-based databases, including schema development, query performance tuning, and data warehousing.


• Improve existing pipelines by resolving recurring failures, eliminating bottlenecks, and optimizing resource utilization.


• Install, configure, and maintain data pipeline utilities, custom Airflow providers, and database connectors.


• Support cloud-based data platform infrastructure and data lake/data warehouse integration efforts.


• Work with containerization technologies such as Docker and Kubernetes.


• Identify opportunities to automate, innovate, and scale data delivery across the organization.


• Perform root cause analysis and incident management for data quality issues, pipeline/DAG failures, and database outages.


• Create and maintain technical documentation covering data architecture, DAG workflows, and operational troubleshooting procedures.


• Collaborate with engineering and technical teams using Agile development methodologies.


• Support CI/CD deployment pipelines for data engineering code and workflows.

What We're Looking For

• 4-7 years of professional Data Engineering experience.


• Strong proficiency with Python and advanced SQL.


• Hands-on experience with Apache Airflow, including development, scheduling, monitoring, and troubleshooting of DAGs.


• Experience with SQL-based database management systems, including schema design and query optimization.


• Strong understanding of ETL/ELT, data transformation, data integration, and data pipeline development.


• Experience working with cloud-based data platforms and data lake/data warehouse environments.


• Experience with Docker and/or Kubernetes.


• Experience with CI/CD and version-controlled development workflows.


• Understanding of Agile software development methodologies.


• Strong production troubleshooting, root cause analysis, and incident-management skills.


• Ability to work effectively in a fast-paced, sense-of-urgency environment.

Ideal Background

The strongest candidates will have experience owning production data pipelines from development through deployment and ongoing support, with particular strength in Apache Airflow, Python, SQL, and cloud data platforms.

This is a hands-on engineering role for someone who enjoys solving complex pipeline and data infrastructure problems- not a primarily reporting, analytics, or dashboard-focused position.