Data Engineer_Databricks (AWS)

Planet Solutions; Kolkata District; On-site; INR 1,500,000 - 2,400,000; Full time

Job summary

Planet Solutions is seeking a senior Databricks Data Engineer to lead data engineering efforts in a healthcare-focused environment. You will design and optimize PySpark/SQL pipelines, implement Delta Lake on AWS S3, and drive ETL improvements.

You will use Airflow, Glue and EMR to orchestrate data workflows, ensuring high data quality and reliability while collaborating with architects and data teams on healthcare data solutions.

Qualifications

• 7-14 years of hands-on experience with PySpark, Spark, Python and SQL in enterprise data environments.
• Strong AWS data platform experience including S3 and Delta Lake.
• Healthcare data domain exposure and data quality/ETL expertise needed.

Responsibilities

• Lead Databricks data engineering efforts and build scalable PySpark/SQL pipelines.
• Implement Delta Lake on AWS S3 and optimize end-to-end ETL workflows.
• Ensure data quality, monitor pipelines, and collaborate with architects on healthcare solutions.

Skills

PySpark

Spark

Python

SQL

AWS

S3

Delta Lake

ETL

Airflow

Tools

Glue

EMR

Job description

Lead Databricks data engineering, build scalable PySpark/SQL pipelines, implement Delta Lake on AWS S3, optimize ETL, ensure data quality, use Airflow/Glue/EMR, and collaborate with architects on healthcare data solutions.

Required Candidate profile

Databricks Data Engineer with 7-14 years' experience in PySpark, Spark, Python, SQL, AWS, S3, Delta Lake, ETL, Airflow and healthcare data, with strong data pipeline expertise.