Data Engineer: Python & AWS Data Pipelines

Quantum Technologies. LLC; Plano (TX); On-site; USD 120,000 - 160,000; Full time

Job summary

Quantum Technologies LLC is seeking a hands-on Data Engineer to design and operate data pipelines using Python and AWS services, focusing on lakehouse and data lake platforms.

You will build ingestion and connectivity across enterprise data platforms, work with S3, Glue, Lake Formation, and CI/CD pipelines, and collaborate with platform and engineering teams to improve data quality and reliability.

Qualifications

• Proficiency in Python for building data pipelines.
• Hands-on AWS experience with S3, Glue, Lake Formation.
• Experience with data lakes and data lakehouse architectures.
• Familiarity with CI/CD tools like Jenkins and Bitbucket.
• Ability to test, troubleshoot, and maintain production pipelines.

Responsibilities

• Build and maintain data pipelines using Python and AWS services.
• Develop data ingestion and connectivity solutions for enterprise data platforms.
• Work with AWS data lake services, including S3, Glue, and Lake Formation.
• Use AWS CLI and EC2 to support data engineering workloads.
• Implement and maintain CI/CD pipelines using Jenkins and Bitbucket.
• Troubleshoot data pipeline issues and improve data quality and reliability.
• Collaborate with platform and engineering teams to support shared data services.

Skills

Python

AWS services

Data pipelines

CI/CD

Production pipelines

Tools

Snowflake

Databricks

Apache Iceberg

Terraform

PySpark

Apache Spark

Job description

Quantum Technologies LLC is seeking a hands-on Data Engineer to design and operate data pipelines using Python and AWS services, focusing on lakehouse and data lake platforms.

You will build ingestion and connectivity across enterprise data platforms, work with S3, Glue, Lake Formation, and CI/CD pipelines, and collaborate with platform and engineering teams to improve data quality and reliability.