Senior Data Engineer: GCP Pipelines & ML Workflows

Cloud Hybrid Technologies LLC; Mountain View (CA); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Cloud Hybrid Technologies LLC is hiring a data engineer to develop and enhance Python frameworks for data processing and ML operations. You will design scalable data pipelines on Google Cloud Platform, implement robust monitoring, and contribute to data tooling standards across the team.

The role emphasizes collaboration with data scientists and engineers to optimize data models, warehousing concepts, and ML feature stores using BigQuery, Dataflow, and Vertex.

Qualifications

• Proficiency in Python for building data processing frameworks and libraries.
• Experience with data pipelines, governance, quality, and lineage.
• Strong knowledge of data warehousing concepts and ETL/ELT patterns.

Responsibilities

• Develop and enhance Python frameworks and libraries for data processing and ML operations.
• Design, build, and maintain scalable data pipelines on GCP.
• Implement monitoring, logging, and alerting for data infrastructure reliability.
• Build batch pipelines using BigQuery, Dataflow, and Airflow/Composer on GCP.
• Design data models for storage and ML modeling using Bigtable and Vertex Feature Store.
• Contribute to shared data tooling and standards to improve team productivity.

Skills

Python

Data processing

GCP

BigQuery

Data modeling

ETL/ELT

Airflow

CI/CD

Communication

Problem solving

Education

Bachelor’s or Master’s degree in CS/Engineering/Data Science or related field

Tools

Git/GitHub

TensorFlow

Apache Spark

scikit-learn

Jupyter Notebook

Pandas/NumPy

Cloud platforms (GCP)

BigQuery/Cloud Dataflow

Job description

Cloud Hybrid Technologies LLC is hiring a data engineer to develop and enhance Python frameworks for data processing and ML operations. You will design scalable data pipelines on Google Cloud Platform, implement robust monitoring, and contribute to data tooling standards across the team.

The role emphasizes collaboration with data scientists and engineers to optimize data models, warehousing concepts, and ML feature stores using BigQuery, Dataflow, and Vertex.