Cloud Data Engineer

enGen Global; Chennai District; On-site; INR 1,800,000 - 3,200,000; Full time

Job summary

enGen Global is seeking an experienced Data Engineer to design and implement robust data pipelines using Databricks and PySpark, loading data from multiple sources into Google Cloud. You will collaborate across India and USA teams to plan, execute workflows, and ensure load accuracy.

The role requires strong SQL skills, 6+ years in data, and hands-on experience with GCP services like Dataproc and Airflow. Excellent communication and ownership are essential for success in this position.

Qualifications

• Bachelor’s degree from a reputed university.
• 7 to 12 years overall experience.
• Excellent SQL/data knowledge (6+ years) in creating complex SQL queries and stored procedures.
• Experience in developing enterprise-grade ETL applications (any tool).
• Experience in Google BigQuery or any cloud database.
• Experience in Google Cloud services like Dataproc (Spark) and Airflow.
• Working knowledge in PySpark / Python.
• Good communication and positive attitude.
• Ownership mindset to drive tasks without follow-ups.

Responsibilities

• Design, develop, and implement high-performance data pipelines using Databricks/PySpark for diverse data integration requirements.
• Leverage SQL expertise for complex data manipulation, analysis, and query optimization.
• Create custom scripts in Python / PySpark.
• Assist with development and enhancement for new or custom requirements.
• Validate data between source and target and rerun jobs as needed.
• Lead other developers, review code, and ensure quality and optimal delivery.
• Provide regular updates on tasks, status, and risks to the project manager.

Skills

SQL

PySpark

Python

BigQuery

US Healthcare

ETL

GCP

Education

Bachelor's degree

Job description

Skills Required : SQL, Pyspark/Python, Big Query, US Healthcare, ETL, GCP

enGen Global is an emerging global healthcare partner that delivers strategic innovation, expertise, and flexibility to its healthcare partners. Being a US healthcare conglomerate captive, we have direct access to deeper insights that help us accelerate our learning process and keeps us ahead of the curve. enGen Global delivers next‑generation solutions that enable our healthcare partners to provide positive experiences to their consumers.

Our global collaborative of healthcare, operations, and IT experts creates innovative and sustainable processes for our clients, which keeps the ever‑evolving consumers engaged and assists them in managing the future of their healthcare better. We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. enGen Global is an equal opportunity employer and places a high value on integrity, diversity, and inclusion in the organization. We do not discriminate based on any protected attribute. For more information about the organization, please visit enGen

Role Summary:

This job involves creating data pipelines using a configurable framework (developed using Databricks / pyspark), which are used to load data from multiple sources to Google Cloud. It also involves extensive collaboration with other team members (India & USA) to follow a plan, execute workflows, ensure data load accuracy and debug in case of issues.

• Design, develop, and implement robust, high-performance data pipelines using Databricks/PySpark to meet diverse data integration requirements. Understand data and its relation / significance from business perspective
• Leverage extensive SQL expertise for complex data manipulation, analysis, and query optimization.
• Create custom scripts in python / pyspark
• Minor development and enhancement activities as needed to handle new / custom requirements
• Validate data between source and target & highlight issues and rerun jobs as needed
• Lead other developers, review their code and ensure quality & optimal code is being delivered
• Provide regular updates on the tasks, status and risks to project manager

The experience we are looking to add to our team

Required

• Bachelor’s degree from a reputed university
• 7 to 12 years overall experience
• Excellent SQL / data knowledge (6+ years) in creating complex SQL queries, Stored Procedures etc.
• Experience in developing enterprise grade ETL applications (any tool)
• Experience in Google BigQuery or any cloud database
• Experience in Google Cloud services like Managed service for Apache Spark(Dataproc), Airflow etc.
• Working knowledge in Pyspark / Python
• Good communication and positive attitude
• Attitude to take ownership and drive tasks without follow-ups

Good to have

• Experience in Big data technologies like Hadoop, Hive, etc.