Senior GCP Data Engineer — AI & Real-Time Pipelines

Robotics Technologies LLC; Hartford (CT); On-site; USD 120,000 - 160,000; Full time

Job summary

Robotics Technologies LLC is seeking a data engineer to design and implement robust data pipelines on GCP. You will develop workflows with Cloud Composer, write Python code, and craft complex SQL in BigQuery, while building real-time data processing with Kafka and managing MongoDB schemas.

The role requires strong communication, collaboration in Agile/SAFe, and the ability to work independently in a fast-paced environment in Hartford, CT.

Qualifications

• Experience designing and implementing robust data pipelines on GCP.
• Strong SQL and Python coding for data processing and transformations.
• Proven ability to build real-time data processing with Kafka and NoSQL design.
• Experience with Airflow/Cloud Composer and scheduling workflows.
• Excellent communication and documentation skills.
• Experience in Agile/SAFe environments is preferred.

Responsibilities

• Design and implement robust data pipelines using GCP services (BigQuery, Cloud Storage, Pub/Sub).
• Develop and manage workflows with Cloud Composer (Airflow) for scheduling and orchestration.
• Write clean, scalable Python code for data processing.
• Craft complex SQL queries in BigQuery with advanced techniques.
• Build real-time data processing using Apache Kafka.
• Model MongoDB schemas for scalable data access.
• Use Shell scripting and Linux administration to support data infra.
• Profile data and implement validation to ensure quality and integrity.
• Develop API integration scripts for automation and data exchange.
• Troubleshoot data issues with strong analytical skills.
• Create data flow diagrams to communicate architecture clearly.
• Develop and manage Cloud Composer 3 workflows.

Skills

GCP data services

BigQuery

Python

SQL

MongoDB

Shell scripting

Linux administration

Data profiling

API integrations

Agile / SAFe

Education

GCP Professional Data Engineer Certification

Tools

Cloud Composer

Apache Kafka

GitHub

RTC

Automation tools

Job description

Robotics Technologies LLC is seeking a data engineer to design and implement robust data pipelines on GCP. You will develop workflows with Cloud Composer, write Python code, and craft complex SQL in BigQuery, while building real-time data processing with Kafka and managing MongoDB schemas.

The role requires strong communication, collaboration in Agile/SAFe, and the ability to work independently in a fast-paced environment in Hartford, CT.