Senior GCP Data Engineer

Jobtailor; Hyderabad; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Jobtailor is seeking a senior Data Engineer to design, build, and optimize scalable data pipelines and ETL processes. You will shape foundational data architecture for identity resolution and ID graphs, handling ingestion, normalization, matching, and deduplication across multiple sources.

You will work with Python and SQL on large datasets in cloud environments (GCP/AWS), contribute to system design, and ensure data quality through monitoring, validation, and production-readiness.

Qualifications

• 5–8+ years of hands-on data engineering experience.
• Production-grade pipelines and distributed systems experience.
• Experience with large-scale data platforms.
• Strong SQL and Python for data processing.
• Experience with cloud data services on GCP or AWS.
• CI/CD, containers, and orchestration familiarity.

Responsibilities

• Design, build, and optimize scalable data pipelines.
• Develop foundational data architecture for identity resolution.
• Process and unify multi-source datasets.
• Write efficient Python/SQL code for large-scale processing.
• Collaborate with data, platform, DevOps, and product teams.
• Document pipelines and workflows.
• Drive data quality via monitoring and alerts.

Skills

Python Programming

SQL & Relational Databases

Data Engineering

Distributed Systems

Data Modeling

Cloud Platforms

Tools

Postgres

BigQuery

Redshift

Google Cloud Dataflow

AWS S3

Docker

Kubernetes

Git

Jira

Confluence

Job description

• Design, build, and optimize scalable data pipelines and ETL/ELT workflows for large, complex datasets.
• Design and implement foundational data architecture supporting identity resolution and ID graph systems.
• Develop and enhance systems supporting identity resolution and ID graph construction (data ingestion, normalization, matching, and deduplication).
• Process and unify multi-source datasets (cookies, device IDs, behavioral data, third-party and proprietary data).
• Write efficient, testable, and maintainable code using Python and SQL for large-scale data processing.
• Optimize data models, queries, and storage strategies for performance, scalability, and cost efficiency.
• Build and maintain data validation, monitoring, and alerting systems to ensure data quality and reliability.
• Troubleshoot, debug, and improve existing data pipelines and infrastructure.
• Take ownership of complex data problems end-to-end, from initial design through production deployment.
• Contribute to key technical decisions related to data architecture, scalability, and system design.
• Collaborate with data, platform, DevOps, and product teams to deliver scalable, production-ready solutions.
• Translate business and product requirements into practical, performant data solutions.
• Document data pipelines, systems, and workflows clearly.
• Continuously improve system performance, data quality, and pipeline resilience.
• Contribute to building new capabilities that improve how customers understand and leverage data insights.

Requirements

• 5–8+ years of hands-on experience in data engineering or large-scale data processing.
• Proven experience building and maintaining production-grade data pipelines and distributed systems.
• Experience contributing to the architecture and delivery of large-scale data platforms or mission-critical data systems.
• Strong expertise in:
• SQL and relational databases (Postgres, BigQuery, Redshift, etc.)
• Python for data processing and analysis
• Experience with Google Cloud Platform (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Cloud Functions) and/or AWS (S3, Redshift, EMR, RDS).
• Experience working with large-scale datasets (hundreds of millions to billions of records).
• Strong understanding of data modeling, partitioning, indexing, and query optimization.
• Experience with distributed data processing and parallelization techniques.
• Experience moving large volumes of data across systems and architectures.
• Familiarity with CI/CD, containerization, and orchestration tools (Docker, Kubernetes, GitHub Actions, etc.).
• Strong debugging and troubleshooting skills in complex data environments.
• Experience with version control (Git) and Agile tools (Jira, Confluence, etc.).
• Highly analytical with strong attention to detail and a data-driven mindset.
• Ability to hit the ground running, quickly understand systems, and deliver independently.
• Comfortable working in a remote, fast-paced, and collaborative environment.
• Strong ability to contribute to system design and deliver implementations independently.

Core Competencies

Demonstrates expertise in designing and optimizing scalable data pipelines and ETL/ELT workflows, with strong proficiency in Python and SQL for large-scale data processing. Capable of contributing to data architecture and delivering production-grade solutions in cloud environments like Google Cloud Platform and AWS.

Highest-signal resume keywords

• Data Pipeline Development
• Python Programming
• SQL and Relational Databases
• Google Cloud Platform
• Data Modeling and Optimization

ATS Optimization Keywords

Hard Skills

• Data Engineering
• ETL/ELT Workflows
• Data Architecture
• Data Processing
• Data Validation
• Debugging and Troubleshooting
• Distributed Systems
• Data Ingestion
• Data Normalization
• Data Deduplication

Soft Skills

• Analytical Skills
• Attention to Detail
• Independent Delivery
• Collaboration
• Adaptability

Industry Keywords

• Large-Scale Data Processing
• Data Quality
• Data Insights
• Cloud Storage
• CI/CD

Tools & Technologies

• Postgres
• BigQuery
• Redshift
• Google Cloud Dataflow
• AWS S3
• Docker
• Kubernetes
• Git
• Jira
• Confluence