Sr Data Engineer

The Versatile Club; India; Remote; INR 1,500,000 - 2,500,000; Full time

Job summary

A leading consulting firm is looking for a Senior Data Engineer to design and implement scalable data architectures. The perfect candidate will have over 10 years of experience in data engineering, strong expertise in the Databricks Lakehouse Platform, and proficiency in AWS services. The role involves building data pipelines, managing the end-to-end data lifecycle, and integrating data from SAP and non-SAP systems. This full-time position emphasizes collaboration with cross-functional teams across the IT sector.

Qualifications

• 10+ years in data engineering/data architecture.
• Strong expertise in Databricks (Delta Lake, Medallion Architecture, DLT, PySpark, SQL Warehouse, Unity Catalog).
• Proficiency in Python, SQL, PySpark.

Responsibilities

• Architect and build scalable data pipelines, models, and products (Databricks, AWS).
• Manage end-to-end data lifecycle and develop enterprise data models.
• Integrate data from SAP ECC/S/4HANA and non-SAP systems.
• Develop batch, real-time, and streaming data solutions.

Skills

Data engineering

Data architecture

Databricks

AWS (S3, Lambda, EMR, Redshift, Bedrock)

Python

SQL

PySpark

CI/CD

Data modeling

Metadata management

Job description

5 days ago Be among the first 25 applicants

Role Summary

We are looking for an experienced Senior Data Engineer to design and implement scalable data architectures and AI‑ready data products. The ideal candidate has deep expertise in the Databricks Lakehouse Platform, strong skills in AWS cloud services, and exposure to SAP data processing.

Key Responsibilities

• Architect and build scalable data pipelines, models, and products (Databricks, AWS).
• Manage end‑to‑end data lifecycle and develop enterprise data models.
• Integrate data from SAP ECC/S/4HANA and non‑SAP systems.
• Develop batch, real‑time, and streaming data solutions.
• Implement data governance, security, quality, and observability.
• Optimize performance and cost across platforms.
• Collaborate with cross‑functional teams to deliver enterprise‑ready data solutions.

Required Skills

• 10+ years in data engineering/data architecture.
• Strong expertise in Databricks (Delta Lake, Medallion Architecture, DLT, PySpark, SQL Warehouse, Unity Catalog).
• Proficiency in Python, SQL, PySpark.
• AWS experience: S3, Lambda, EMR, Redshift, Bedrock.
• Data modeling (ER & Dimensional) and metadata management.
• CI/CD and DevOps awareness.

Preferred Skills

• SAP S/4HANA data extraction (DataSphere, SLT, BDC).
• ABAP, CDS views, manufacturing domain experience.

Certifications Databricks Data Engineer/Architect certification (preferred).

Skills: data engineering, data architecture, delta lake, datasphere, slt, bdc, sap s/4hana data extraction, python, sql warehouse, medallion architecture, databricks lakehouse platform, pyspark workbooks, dlt pipelines, databricks.

Seniority Level

Mid‑Senior level

Employment Type

Full‑time

Job Function

Information Technology

Industries

Outsourcing and Offshoring Consulting