Data Engineer

eClerx LLC; Navi Mumbai, Pune District; Hybrid; INR 1,500,000 - 2,100,000; Full time

Job summary

eClerx LLC is seeking a Data Engineer with Databricks experience for hybrid roles in Pune and Navi Mumbai. You will lead scalable data solutions, develop real-time and batch pipelines, and collaborate with architects, analysts and data scientists to deliver reusable datasets for analytics.

The role emphasizes governance, CI/CD, and best practices in a fast-paced environment. The ideal candidate will demonstrate strong Python/SQL skills, PySpark expertise, and hands-on Databricks/Delta Lake

Qualifications

• Proficient in Python and SQL with strong PySpark experience.
• Hands-on Databricks, Delta Lake, and data lakehouse design.
• Experience with Airflow or similar orchestration tools.
• Mentor engineers and collaborate with cross-functional teams.

Responsibilities

• Lead design and implementation of scalable Databricks Lakehouse data solutions.
• Build high-volume batch and near-real-time data pipelines using PySpark, SQL, Python, Delta Lake.
• Develop end-to-end frameworks (CDC, incremental processing, deduplication, SCD, schema evolution).
• Design data models and processing frameworks for analytics and downstream apps.
• Contribute to data governance, security, and lineage using Databricks Unity Catalog.
• Promote engineering best practices: Git, reviews, tests, CI/CD, documentation, production support.
• Collaborate with data architects, analysts, BI, data scientists and stakeholders to translate requirements.
• Experience with Airflow or other modern orchestration tools.
• Mentor intermediate engineers and guide data-engineering practices.

Skills

Python

SQL

PySpark

Databricks

Delta Lake

Airflow

Git

Code debugging

Data modeling

Tools

Terraform

Docker/Kubernetes

Databricks Jobs/Workflows

Job description

eClerx is Hiring Data Engineer (Databricks)

Dear Candidates,

eClerx is hiring Data Engineers with Databricks experience for its Pune and Navi Mumbai locations.

Locations: Pune & Navi Mumbai
Work Mode: Hybrid
Notice Period: Immediate Joiners to candidates with a Last Working Day on or before 20th November 2026

Good luck!

Role & responsibilities

• 1. Lead the design and implementation of scalable Databricks Lakehouse data solutions supporting analytics and business applications.
• 2. Design and optimize high-volume batch and near-real-time data pipelines using PySpark, SQL, Python, Delta Lake, and Databricks workflows.
• 3. Design and develop end to end frameworks using patterns such as CDC, incremental processing, deduplication, SCD, and schema evolution.
• 4. Design data models and processing frameworks that provide reliable, performant, and reusable datasets for analytics and downstream applications.
• 5. Contribute to data governance, security, access control, and lineage using Databricks and Unity Catalog.
• 6. Drive engineering best practices including Git, code reviews, automated testing, CI/CD, documentation, and production support.
• 7. Collaborate with data architects, analysts, BI teams, data scientists, and business stakeholders to translate requirements into scalable data solutions.
• 8. Experience with Airflow or other modern orchestration tools.
• 9. Mentor intermediate engineers and provide technical guidance on data-engineering practices and platform capabilities.

Programming & Data Engineering

• Strong Python and advanced SQL skills.
• Strong hands-on experience with Apache Spark/PySpark.
• Strong hands-on experience with Databricks and Delta Lake.
• Strong hands on experience in creating data models and data governance framework
• Strong understanding of batch data processing and streaming concepts.
• Strong knowledge of CDC, incremental processing, deduplication, SCD, and schema evolution.
• Experience designing scalable and maintainable data pipelines and data models.
• Good analytical and debugging skill

Databricks

• Experience designing and implementing Databricks-based data solutions.
• Experience with Databricks Jobs/workflows and production pipeline management.

AI / GenAI

• Experience with Databricks Lakeflow Pipelines, Lakeflow Connect, Auto Loader, or Structured Streaming.
• Exposure to Terraform or infrastructure-as-code practices.
• Experience with data observability, pipeline monitoring, and cloud cost/performance optimization.
• Experience modernizing or migrating legacy ETL/data warehouse workloads to Databricks.
• Exposure to Data Mesh, Data Products, or domain-oriented data architecture.
• Exposure to Docker/Kubernetes or other cloud-native technologies.
• Experience designing data platforms or pipelines supporting ML/AI/GenAI workloads.
• Experience using AI-powered development tools or GenAI coding assistants.
• Exposure to Databricks AI/ML capabilities or equivalent cloud AI platforms

Regards,

Rucha Labhe