GCP Data Engineer

Artech L.L.C.; Telangana; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Artech L.L.C. is seeking an experienced GCP Data Engineer in Hyderabad to design, build, and optimize scalable data pipelines. The role demands strong Python, Apache Beam, and SQL skills with solid expertise in GCP services such as BigQuery, Dataflow, and Pub/Sub.

You will work with cross-functional teams to deliver data-driven solutions, ensure data governance, and improve pipeline reliability and performance in a fast-paced environment.

Qualifications

• 6+ years of data engineering experience.
• Strong Python and SQL proficiency.
• Expertise with GCP services including BigQuery, Dataflow, Pub/Sub, and Cloud Storage.

Responsibilities

• Design, develop, and optimize end-to-end data pipelines using Python and Apache Beam.
• Architect scalable data workflows on GCP for reliability and performance.
• Write complex SQL queries for data extraction and analysis.
• Collaborate with data scientists, analysts, and DevOps to meet data requirements.
• Contribute to coding standards, testing, and deployment practices within the squad.
• Develop and maintain data models, ETL/ELT processes, and data integration solutions.
• Perform data analysis to identify trends, anomalies, and insights for decision making.
• Ensure data security, governance, and compliance across environments.
• Monitor pipelines and troubleshoot performance bottlenecks and failures.
• Stay updated on data engineering trends in cloud and distributed systems.

Skills

Python

Apache Beam

GCP

BigQuery

Dataflow

Pub/Sub

SQL

Data modeling

Airflow

Docker

Kubernetes

NoSQL

CI/CD

Tools

Spark

Flink

Job description

Role: GCP Data Engineer

Exp: 6+ Years

Loc: Hyderabad

Job Description

About this role:

Experienced with a strong background in Data Engineering to join our team. The ideal candidate will have expertise in Python, Apache Beam, and experience working with Google Cloud Platform (GCP) environments. Additionally, strong SQL and analytical skills are mandatory for this role. You will play a critical role in designing, developing, and optimizing scalable data pipelines and solutions to support our business needs.

Job Expectations:

• Contribute to design, development, and implementation of end-to-end data pipelines using Python and Apache Beam.
• Architect and optimize data workflows for GCP environments, ensuring scalability, reliability, and performance.
• Write complex SQL queries for data extraction, transformation, and analysis, ensuring data accuracy and consistency.
• Collaborate with cross-functional teams, including data scientists, analysts, and DevOps, to understand data requirements and deliver solutions.
• Provide technical contributions for a squad of engineers, ensuring best practices in coding, testing, and deployment.
• Develop and maintain data models, ETL processes, and data integration solutions.
• Perform data analysis to identify trends, anomalies, and insights to support business decision-making.
• Ensure data security, compliance, and governance across all environments.
• Monitor and troubleshoot data pipelines, addressing performance bottlenecks and failures.
• Stay up to date with the latest trends and technologies in data engineering, cloud computing, and distributed systems.

Desired Qualifications:

• Expertise in Python with Apache Beam for building data pipelines.
• Strong expertise in GCP environments, including services like BigQuery, Apache IceBerg, Dataflow, Pub/Sub, and Cloud Storage.
• Advanced SQL skills, with the ability to write and optimize complex queries for large datasets.
• Strong analytical skills, with experience in interpreting data and providing actionable insights.
• Solid understanding of distributed systems, data processing frameworks, and ETL/ELT workflows.
• Experience with CI/CD pipelines, containerization (Docker), and orchestration tools (Kubernetes, Airflow).
• Familiarity with relational and NoSQL databases, as well as data modeling concepts.
• Excellent problem-solving skills and the ability to work in a fast-paced, dynamic environment.

Additional Skills (Good to have):

• Experience with other data processing frameworks like Spark or Flink.
• Knowledge of data governance, security, and compliance best practices.
• Certification in GCP (e.g., Professional Data Engineer) is a plus.
• Experience with hybrid cloud solutions and migrating workloads from On-Premises to GCP.