Senior GCP Data Engineer – AI-Driven Pipelines

Quantum Technologies. LLC; Hartford (CT); On-site; USD 115,000 - 155,000; Full time

Job summary

Quantum Technologies. LLC is seeking a senior data engineer to design and implement robust data pipelines on Google Cloud Platform. You will build real-time processing with Kafka, manage BigQuery queries, and develop Airflow workflows with Cloud Composer.

Strong Python, SQL, and NoSQL design skills are required, along with hands-on GCP experience. You will work in an agile SAFe environment, collaborate across onshore/offshore teams, and document architecture and data flows for stakeholders.

Qualifications

• 7+ years of hands-on experience with GCP data services.
• Strong understanding of distributed systems and data engineering.
• Excellent communication and documentation skills.
• Experience in Agile SAFe environments.
• Experience with onshore/offshore collaboration.
• Experience with GitHub, RTC, automation tools preferred.

Responsibilities

• Design and implement robust data pipelines using GCP services (BigQuery, Cloud Storage, Pub/Sub).
• Develop and manage workflows with Cloud Composer for scheduling and orchestration.
• Write clean, scalable Python code and develop complex SQL queries in BigQuery.
• Build real-time data processing with Apache Kafka.
• Model NoSQL databases, especially MongoDB, with scalable schemas.
• Develop API integration scripts and maintain data validation techniques.
• Create data flow diagrams and communicate architecture to stakeholders.
• Collaborate in an agile SAFe environment, coordinating with onshore and offshore teams.

Skills

AI/ML in practice

GCP data services

Distributed systems

Python & SQL

Tools

BigQuery

Cloud Storage

Pub/Sub

Cloud Composer / Airflow

Kafka

MongoDB

Shell scripting

Linux

Job description

Quantum Technologies. LLC is seeking a senior data engineer to design and implement robust data pipelines on Google Cloud Platform. You will build real-time processing with Kafka, manage BigQuery queries, and develop Airflow workflows with Cloud Composer.

Strong Python, SQL, and NoSQL design skills are required, along with hands-on GCP experience. You will work in an agile SAFe environment, collaborate across onshore/offshore teams, and document architecture and data flows for stakeholders.