Data Engineer (AWS + Databricks)

Alike Thoughts; India; Hybrid; INR 1,200,000 - 2,400,000; Full time

Job summary

Alike Thoughts in Pan-India is seeking a Lead Data Engineer with strong AWS and Databricks expertise to design and develop large-scale data processing solutions, build robust pipelines, and collaborate with cross-functional teams to deliver secure, scalable data platforms.

You will work hands-on with Databricks, Spark, SQL modeling, and Delta Lake, while aligning with governance, security, and Agile practices. Experience with ML workloads on Databricks and familiarity with MLflow will be valued.

Qualifications

• Experience building scalable data pipelines with AWS and Databricks.
• Strong knowledge of Spark (Spark SQL, PySpark) and SQL data modeling.
• Experience with Delta Lake, schema evolution, and ACID transactions.
• Familiarity with ML/AI workloads on Databricks and model tracking (MLflow).

Responsibilities

• Hands-on Databricks development for large-scale data processing.
• Build and optimize ETL/ELT pipelines for batch and near realtime data.
• Collaborate with architects, BI, and data science teams to deliver end-to-end data solutions.
• Design analytical and dimensional data models (Star/Snowflake).
• Ensure data governance, security, and compliance.

Skills

AWS

Databricks

Apache Spark

PySpark

SQL

Data Modeling

Python

Scala

Power BI

Delta Lake

Tools

Power BI

MLflow

Job description

Data Engineer (AWS + Databricks)

Experience: 812 years Location: Pan India(All LTM locations) Grade: P4

Role Summary

We are seeking a highly skilled Lead Data Engineer with strong expertise in AWS and Databricks. The role involves designing and developing largescale data processing solutions, building robust pipelines, and collaborating with crossfunctional teams to deliver secure, scalable, and highperformance data platforms.

Key Responsibilities

• Handson Databricks development for largescale data processing and analytics.
• Build and maintain ETL/ELT pipelines for batch and near realtime ingestion.
• Work with Apache Spark, PySpark, Spark SQL, Python, Scala, Power BI.
• Implement and optimize Delta Lake features (ACID transactions, schema evolution, performance tuning).
• Design and implement analytical and dimensional data models (Star/Snowflake).
• Leverage AWS cloud storage systems for scalable data solutions.
• Ensure adherence to coding standards, documentation, and data engineering best practices.
• Collaborate with architects, BI teams, and data science teams for endtoend delivery.
• Exposure to ML/AI workloads on Databricks, including feature engineering.
• Familiarity with MLflow or similar model tracking tools.
• Experience with Databricks CI/CD and Gitbased version control.
• Strong understanding of data governance, security, and compliance.
• Work in Agile/Scrum delivery environments.
• Domain experience in Insurance preferred.

Mandatory Skills

• AWS (cloud storage, services).
• Databricks (development, pipelines, Delta Lake).
• Apache Spark / PySpark / Spark SQL.
• SQL and Data Modelling.

Good to Have Skills

• Exposure to ML/AI workloads on Databricks.
• Familiarity with MLflow.
• Experience with Power BI.
• Domain knowledge in Insurance.