Data Engineer

TidyHire; Hyderabad; On-site; INR 3,500,000 - 5,500,000; Full time

Job summary

TidyHire seeks a Data Engineer – Senior II with GCP and AI/ML focus to lead design and ownership of scalable data platforms. You will drive architectural decisions, enable AI/ML use cases, mentor engineers, and ensure platform reliability while aligning with strategic business goals.

You'll design and implement batch and streaming pipelines, advance data models, and collaborate with data scientists and product teams to deliver end-to-end data and ML solutions on Google Cloud.

Qualifications

• 10–12 years of data engineering experience on cloud platforms.
• Hands-on leader for designing scalable data platforms on GCP.
• Experience with AI/ML data pipelines and Vertex AI workflows.

Responsibilities

• Design, build, and own large-scale batch and streaming pipelines on GCP.
• Lead data architecture decisions aligned with enterprise strategy.
• Develop ETL/ELT pipelines using BigQuery, Dataflow (Apache Beam), Cloud Composer (Airflow), and Cloud Storage.
• Create feature data models to support reporting and analytics.
• Collaborate with data scientists, analysts, architects, and product teams to deliver end-to-end data/ML solutions.
• Provide production support, performance tuning, and root-cause analysis for data issues.

Skills

Python

SQL

Java/Scala

PySpark

Pub/Sub

Kafka

Tools

BigQuery

Snowflake

Airflow

Dataflow

Cloud Storage

Vertex AI

Job description

Be yourself - Charlie makes interviews comfortable and conversational

Interview Process

Quick video chat with Charlie, our AI interviewer

1

Verify your identity

Email and mobile verification

2

Mic and camera check

3

Chat with Charlie

Brief conversation about the role

4

We'll be in touch soon

Step -1 of 4

This is an AI-powered experience; despite best efforts, responses may at times be unexpected or inaccurate.

Skills & Keywords

Required Skills

Good to have

Job Description

Role & responsibilities

Job Description – Data Engineer (Senior II) | GCP with AI/ML Focus

Experience: 10-12 Years

About the Role

The Data Engineer – Senior II is a hands-on technical leader responsible for designing,

building, and owning scalable, cloud-native data platforms on Google Cloud Platform

(GCP). This role goes beyond implementation to drive architectural decisions, enable

AI/ML use cases, mentor engineers, and ensure platform reliability while supporting

strategic business initiatives.

Key Responsibilities

• Design, develop, and own large-scale batch and streaming data pipelines on GCP.
• Lead data architecture and platform design decisions aligned with enterprise strategy.
• Build and optimize ETL/ELT pipelines using BigQuery, Dataflow (Apache Beam), Cloud
• Composer (Airflow), and Cloud Storage.
• Develop complex transformations using Python, advanced SQL, and PySpark for
• Design and own analytical and feature data models to support reporting, analytics, and
• Enable and support AI/ML use cases by building feature pipelines, preparing training
• datasets, and integrating with Vertex AI workflows.
• Ensure data quality, governance, security, and compliance across the data and ML
• ecosystem.
• Provide production support, performance tuning, and deep root-cause analysis for critical
• data issues.
• Collaborate with data scientists, analysts, architects, and product teams to deliver
• end-to-end data and ML solutions.
• Drive DevOps best practices including CI/CD, automation, monitoring, and operational
• Mentor Senior I and junior engineers; set engineering standards and promote best
• practices.
• Evaluate emerging GCP capabilities and AI/ML patterns to drive innovation and platform

Key Skills

• Programming: Python (advanced), SQL (advanced), Java/Scala (preferred), PySpark
• Streaming: Pub/Sub (Kafka experience acceptable)
• Data Warehousing: BigQuery (Snowflake experience transferable)

Skills & Qualities

• Strong system design and end-to-end data architecture skills
• Ability to translate business and ML requirements into scalable data solutions
• Deep expertise in root-cause analysis and production issue resolution
• Proven mentoring and technical leadership capability
• Ability to work independently and own complex initiatives