Data Engineer

Randstad Global Capability Center; Hyderabad; On-site; INR 1,200,000 - 2,000,000; Full time

Job summary

Randstad Global Capability Center is seeking a Data Engineer to design and implement data solutions on Google Cloud Platform. You will collaborate with business users, BI analysts, and fellow data engineers to translate data needs into scalable technical solutions.

The role involves building data pipelines with GCP services (Dataflow, Dataproc, BigQuery), modeling data in BigQuery, and optimizing storage on Cloud Storage while ensuring data quality and security.

Qualifications

• 2-10 years of experience as a Data Engineer or similar role.
• Proven track record of designing and implementing data solutions on Google Cloud Platform.

Responsibilities

• Collaborate with stakeholders: Engage in discussions with business users, BI analysts, data engineers to understand data needs and translate them into technical solutions.
• Design and implement data pipelines using GCP services like Cloud Dataflow, Cloud Dataproc, and BigQuery to ensure efficient data ingestion, transformation, and delivery.
• Develop data models in BigQuery, focusing on performance, scalability, and security requirements.
• Manage and optimize data storage on Google Cloud Storage, balancing cost and performance.
• Ensure data quality with validation and quality checks throughout the pipeline.
• Troubleshoot and resolve data-related issues to maintain smooth data flow and performance.
• Stay updated on GCP offerings and best practices to continually enhance data solutions.

Skills

GCP data services

SQL

Python

Data modeling

Data warehousing

Data pipelines

BigQuery

Apache Beam

Apache Spark

Data governance

Data security

Job description

• 2-10 years of experience as a Data Engineer or similar role:

Proven track record of designing and implementing data solutions on Google Cloud

Platform.

What a typical day at work would look like?

• Collaborate with stakeholders: Engage in discussions with business users, BI analysts, data engineers to understand their data needs and translate them into technical solutions.
• Design and implement data pipelines: Architect and build data pipelines using GCP services like Cloud Dataflow, Cloud Dataproc, and BigQuery to ensure efficient data ingestion, transformation, and delivery.
• Develop data models: Design and implement data models in BigQuery, considering performance, scalability, and security requirements.
• Manage and optimize data storage: Manage data storage on Google Cloud Storage, optimizing for cost-efficiency and performance.
• Ensure data quality: Implement data validation and quality checks throughout the data pipeline to ensure data accuracy and integrity.
• Troubleshoot and resolve issues: Diagnose and solve complex data-related issues, ensuring smooth data flow and optimal performance.
• Stay updated on GCP advancements: Keep abreast of the latest Google Cloud Platform offerings and best practices to continually enhance our data solutions.

Your Key Knowledge Areas shall be:

• Deep understanding of GCP data services: Expertise in using GCP services such as Cloud Dataflow, Cloud Dataproc, BigQuery, Cloud Storage, Data Catalog, and Data Fusion.
• Data modeling and architecture: Proven experience designing and implementing data models, including dimensional modeling and data warehousing principles.
• Data warehousing and data lakes: Strong understanding of data warehousing concepts, data lake architectures, and best practices for data management.
• Data pipeline development: Proficiency in building data pipelines using tools like Apache Beam, Apache Spark, or other relevant technologies.
• SQL and Python programming: Strong command of SQL for data querying and manipulation, and proficiency in Python for data analysis and automation.
• Data governance and security: Understanding of data governance principles, data security best practices, and compliance requirements.