Python Data Pipeline Engineer (ETL/ELT)

Insight Global; Atlanta (GA); On-site; USD 5,510 - 9,643; Full time

Job summary

Insight Global is seeking a backend data engineer to build and deploy automated batch data pipelines using Apache Airflow and Python. You will engineer scalable ETL/ELT processes moving data between SFTP, GCP, and Snowflake, and implement SQL queries for high-volume workloads.

You will also help design API layers (REST/GraphQL) and collaborate on backend microservices (Spring Boot where applicable) to ensure data accessibility, reliability, and SLA adherence.

Qualifications

• 1-2 years in backend development building ETL/ELT pipelines.
• Experience with orchestrating data workflows using Apache Airflow.
• Proficiency in Python for automation and data transformation.
• Experience integrating with cloud data platforms (GCP, Snowflake) and relational databases.

Responsibilities

• Build and deploy automated batch data pipelines with Airflow and Python.
• Develop ETL/ELT scripts to move data between SFTP, GCP, and Snowflake.
• Design and optimize API layers (REST/GraphQL) and backend microservices (Spring Boot applicable).
• Monitor pipelines, troubleshoot failures, and ensure data quality and SLA delivery.

Skills

Python

Apache Airflow

SQL

REST APIs

GraphQL

CI/CD

Docker

SFTP

Bitbucket

Spring Boot

Tools

Docker

Bitbucket

Kubernetes

Spring Boot

PostgreSQL

Snowflake

GCP

Job description

Insight Global is seeking a backend data engineer to build and deploy automated batch data pipelines using Apache Airflow and Python. You will engineer scalable ETL/ELT processes moving data between SFTP, GCP, and Snowflake, and implement SQL queries for high-volume workloads.

You will also help design API layers (REST/GraphQL) and collaborate on backend microservices (Spring Boot where applicable) to ensure data accessibility, reliability, and SLA adherence.