Data Engineer - GCP (Mid/Senior level)

Cloudesign Technology Solutions LLP.; Pune District; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Cloudesign Technology Solutions LLP is seeking a hands-on Data Engineer to design, build, and maintain robust streaming ETL pipelines using Python, Confluent, and GCP. The role emphasizes event-driven architecture, scalable data solutions, and reliable production-grade systems.

You'll collaborate with cross-functional teams to ensure data reliability and performance, contribute to solution design, and promote engineering best practices.

Qualifications

• Strong programming skills, particularly in Python.
• Significant experience with relational or NoSQL databases, such as SQL Server, CockroachDB, or PostgreSQL.
• Proven experience designing and building efficient, scalable data pipelines.
• Experience scheduling and maintaining automated workflows, particularly with Apache Airflow.
• Ability to write clean, understandable, maintainable code and documentation.
• Experience building streaming pipelines using Kafka and/or the Confluent platform.
• Experience developing streaming pipelines on GCP, using services such as Pub/Sub, BigQuery, Google Cloud Storage (GCS), and Dataflow.
• Strong understanding of ETL pipelines and the ability to build pipelines from scratch.
• Strong communication, collaboration, and teamwork skills.

Responsibilities

• Independently plan, design, and deliver high-quality, production-ready data solutions that are scalable, globally distributed, and highly available.
• Develop systems using event-driven architecture, with a focus on performance, resilience, and fault tolerance.
• Write clean, reliable, and maintainable code, favouring simplicity and selecting the right tools for the task.
• Collaborate with cross-functional teams to build and optimise business-critical data pipelines.
• Communicate effectively with technical and non-technical stakeholders to align on objectives and share progress.
• Participate in code reviews, pair programming, and knowledge-sharing activities to strengthen team capabilities and maintain engineering standards.
• Support project delivery through practical estimation, progress tracking, and technical support for system and user issues.
• Adapt to new technologies and challenges while promoting continuous improvement and shared ownership.

Skills

Python

SQL Server

CockroachDB

PostgreSQL

Data pipelines

Apache Airflow

Kafka

Confluent

GCP

Pub/Sub

BigQuery

GCS

Dataflow

ETL

Documentation

Communication

Teamwork

Job description

We are looking for a hands-on Data Engineer to design, build, and maintain robust streaming ETL pipelines. Using Python, Confluent, and Google Cloud Platform (GCP), you will work with event-driven architecture to develop production-quality data systems. You will help ensure the reliability and performance of existing data systems, components, and platforms. Working closely with technology teams, you will support the seamless collection and transfer of critical data. This role requires strong software engineering fundamentals, clean and efficient coding practices, and the ability to deliver practical solutions to business challenges. You will also contribute to technical solution design, team support, and engineering best practices.

• Strong programming skills, particularly in Python.
• Significant experience with relational or NoSQL databases, such as SQL Server, CockroachDB, or PostgreSQL.
• Proven experience designing and building efficient, scalable data pipelines.
• Experience scheduling and maintaining automated workflows, particularly with Apache Airflow.
• Ability to write clean, understandable, maintainable code and documentation.
• Experience building streaming pipelines using Kafka and/or the Confluent platform.
• Experience developing streaming pipelines on GCP, using services such as Pub/Sub, BigQuery, Google Cloud Storage (GCS), and Dataflow.
• Strong understanding of ETL pipelines and the ability to build pipelines from scratch.
• Strong communication, collaboration, and teamwork skills.

Key Responsibilities

• Independently plan, design, and deliver high-quality, production-ready data solutions that are scalable, globally distributed, and highly available.
• Develop systems using event-driven architecture, with a focus on performance, resilience, and fault tolerance.
• Write clean, reliable, and maintainable code, favouring simplicity and selecting the right tools for the task.
• Collaborate with cross-functional teams to build and optimise business-critical data pipelines.
• Communicate effectively with technical and non-technical stakeholders to align on objectives and share progress.
• Participate in code reviews, pair programming, and knowledge-sharing activities to strengthen team capabilities and maintain engineering standards.
• Support project delivery through practical estimation, progress tracking, and technical support for system and user issues.
• Adapt to new technologies and challenges while promoting continuous improvement and shared ownership.

Equal Opportunity :

We are committed to creating an inclusive, respectful, and merit-based workplace. Employment decisions are made without discrimination on any basis protected by applicable law. Applications from candidates of all backgrounds are encouraged, and reasonable accommodation can be provided throughout the recruitment process where required.

Required Skills