GCP Data Engineer

CustomerLabs; Chennai; On-site; INR 800,000 - 1,200,000; Full time

Job summary

A leading company in Chennai is seeking a Data Engineer to develop and maintain scalable data pipelines. The ideal candidate will have experience with big data technologies and cloud platforms, ensuring data quality and integration across systems. This role involves collaboration with software engineers and staying updated with industry trends. Strong problem-solving skills and proficiency in programming languages like Python and Golang are essential.

Qualifications

• 2 to 3 years experience as a Data Engineer or similar role.
• Proficiency in Python, Golang, Node.js.
• Hands-on experience with big data technologies.

Responsibilities

• Develop and maintain scalable data pipelines and ETL processes.
• Implement data transformation and cleaning processes.
• Collaborate with software engineers to integrate data solutions.

Skills

Data Engineering Principles

Problem Solving

Communication

Tools

Apache Spark

Apache Kafka

Docker

Kubernetes

Google Cloud

Jenkins

Travis CI

GitHub Actions

Job description

Responsibilities:

Develop, and maintain scalable data pipelines and ETL processes to ingest, process, and store data from various sources.

Implement data transformation and cleaning processes to ensure data quality and consistency.

Integrate data from diverse sources including APIs, databases, and third-party services.

Ensure seamless data flow between systems and platforms.

Troubleshoot and resolve issues related to data processing and data quality.

Collaborate with software engineers to integrate data solutions into existing applications and systems.

Maintain comprehensive documentation of data pipelines, architectures, and processes.

Provide regular status updates and reports on data engineering projects and performance metrics.

Implement data security measures and ensure compliance with relevant data protection regulations and policies.

Stay current with industry trends, technologies, and best practices in data engineering.

Recommend and implement new tools and technologies to improve data engineering processes and capabilities.

Qualifications:

Proven experience of 2 to 3yrs as a Data Engineer or similar role(Software Developer, Backend developer) with a strong understanding of data engineering principles and practices.

Familiarity with cloud platforms -Google Cloud.

Proficiency in programming languages such as Python, Golang, Nodejs.

Experience with data modeling, ETL processes, and data warehousing solutions.

Hands-on experience with big data technologies such as Apache Spark, Apache Kafka.

Knowledge in AI Code Generation

Hands on experience in tools likeCopilot, Cursor, Claude, etc.,

Strong knowledge of SQL and experience with relational and NoSQL databases.

Strong problem-solving skills and attention to detail.

Excellent communication and collaboration skills.

Preferred Skills:

Experience with microservices

Experience with containerization and orchestration tools (e.g., Docker, Kubernetes).

Experience with Bigquery, Pubsub, Dataflow, Apache Airflow

Experience with CI/CD Framework tools such asJenkins, Travis CI , Github Actions etc.