Data Engineer (GCP)

Innovature Software Labs (P) Ltd; Ernakulam; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Infopark is seeking a Data Engineer (GCP) to design and maintain scalable data pipelines using BigQuery, Dataflow, and Cloud Composer. You will write clean SQL and Python, implement data quality checks, and optimise data models.

You will collaborate with analysts and business stakeholders to deliver ready-to-use datasets and ensure data lineage and documentation are comprehensive. 4–6 years of experience preferred.

Qualifications

• 4–6 years of experience in data engineering with production data pipelines.
• Excellent communication and presentation skills.
• Strong SQL and Python skills.
• Experience with BigQuery, Dataflow, Cloud Composer (Airflow), and Pub/Sub.
• Good understanding of modelling concepts.
• Good understanding of Git, CI/CD.
• Experience with AI-assisted development.

Responsibilities

• Design, develop, and maintain scalable batch and streaming data pipelines.
• Develop clean, maintainable, and well-tested SQL and Python code.
• Implement data quality and validation checks and document datasets.
• Optimise BigQuery queries, data models, and storage.
• Collaborate with data analysts and stakeholders to provide datasets.

Skills

SQL

Python

BigQuery

Dataflow

Cloud Composer

Airflow

Pub/Sub

CI/CD

AI-assisted development

Data pipelines

Communication

Tools

Git

Airflow

BigQuery

Pub/Sub

Job description

Data Engineer (GCP)

4–6 years of experience in data engineering, with hands-on experience managing production data pipelines.

Key Responsibilities

• Design, develop, and maintain scalable batch and streaming data pipelines using BigQuery, Dataflow, Cloud Composer.
• Develop clean, maintainable, and well-tested SQL and Python code.
• Implement data quality and validation checks, and maintain comprehensive documentation for datasets, schemas, data flows, and lineage.
• Optimise BigQuery queries, data models, and storage to improve performance.
• Collaborate with data analysts and business stakeholders to provide business-ready datasets.

Skill set

• 4–6 years of experience in data engineering, with hands-on experience managing production data pipelines.
• Excellent communication and presentation skills are essential.
• Strong SQL and Python skills.
• Hands-on experience with BigQuery, Dataflow, Cloud Composer (Airflow), and Pub/Sub.
• Good understanding of modelling concepts
• Good understanding of Git, CI/CD.
• Experience with AI-assisted development