GCP Data Engineer: Build & Optimize Scalable Pipelines

Quantum Technologies. LLC; Chicago (IL); Presencial; USD 110 000 - 150 000; Tempo integral

Resumo da oferta

Quantum Technologies. LLC is seeking experienced Data Engineers with hands-on Google Cloud Platform expertise to design and optimize large-scale data pipelines.

The role focuses on migrating and modernizing our enterprise data environment from Azure to GCP, building production-grade pipelines, and delivering reliable data services. You will work with Python and SQL, craft ETL/ELT workflows, and ensure data quality and performance across batch and streaming processes, using Airflow/Cloud Composer

Qualificações

• 6+ years of experience in data engineering, data integration, and data pipeline development.
• Strong hands-on GCP data engineering experience is mandatory.
• Experience with BigQuery, Dataflow, Cloud Storage, and relevant GCP data services.
• Strong expertise in Python, SQL, ETL/ELT development, and data transformations.
• Experience building scalable batch and/or streaming data pipelines.
• Experience with PySpark/Spark and data processing frameworks.
• Strong understanding of data warehousing, data modeling, data quality, and performance optimization.
• Experience with data pipeline orchestration and scheduling tools such as Airflow/Cloud Composer.

Conhecimentos

Data engineering experience

GCP data engineering

Python

SQL

ETL/ELT development

Batch processing

Streaming data pipelines

PySpark/Spark

Data warehousing

Data modeling

Airflow/Cloud Composer

Troubleshooting

Communication skills

Ferramentas

BigQuery

Dataflow

Cloud Storage

Airflow

Terraform

CI/CD

Pub/Sub

Descrição da oferta de emprego

Quantum Technologies. LLC is seeking experienced Data Engineers with hands-on Google Cloud Platform expertise to design and optimize large-scale data pipelines.

The role focuses on migrating and modernizing our enterprise data environment from Azure to GCP, building production-grade pipelines, and delivering reliable data services. You will work with Python and SQL, craft ETL/ELT workflows, and ensure data quality and performance across batch and streaming processes, using Airflow/Cloud Composer