Data Bricks Engineer

Navlakha Management Services; Mumbai; On-site; INR 1,800,000 - 4,000,000; Full time

Job summary

Navlakha Management Services in Mumbai seeks a senior Data Engineer to design, develop, and maintain scalable data solutions using Databricks, PySpark, SQL, and Azure analytics.

You will build and optimize notebooks, clusters, jobs, and pipelines, work with Delta Lake and Lakehouse patterns, and implement CI/CD with Git. The role requires strong communication and collaboration in an office-based environment.

Qualifications

• Minimum 5+ years hands-on data engineering experience.
• Databricks implementation and delivery experience.
• Good understanding of modern data modeling and governance practices.

Responsibilities

• Design, develop, and maintain scalable data engineering solutions.
• Build and optimize Databricks notebooks, clusters, Jobs, Workflows, and SQL Warehouses.
• Develop efficient data pipelines and transformation frameworks for large-scale data processing.
• Work with Delta Lake, Lakehouse and Medallion architectures.
• Optimize Spark jobs, queries, clusters, and pipelines for performance and cost.
• Implement CI/CD and Git-based version control for data workflows.
• Collaborate with business, analytics, technology, and cross-functional teams.
• Troubleshoot data pipeline issues and drive improvements.

Skills

Databricks & Spark

Python & PySpark

SQL & data modelling

Azure cloud data services

Analytical & problem solving

Stakeholder management

Office-based work

Education

BE / B.Tech / MCA in CS/IT

Tools

Azure Data Factory

Airflow

dbt

Git & CI/CD

Unity Catalog

Delta Lake

Spark tuning

Job description

Key Responsibilities

• Design, develop, and maintain scalable and reliable data engineering solutions using Databricks, Python, PySpark, SQL, and Apache Spark.
• Build and optimize Databricks notebooks, clusters, Jobs, Workflows, and SQL Warehouses.
• Develop efficient data pipelines and transformation frameworks for large-scale data processing.
• Work with Delta Lake, Lakehouse architecture, Medallion architecture (Bronze,Silver, Gold), and modern data-modeling practices.
• Optimize Spark jobs, queries, clusters, and data pipelines for performance,scalability, and cost efficiency.
• Develop and maintain data pipelines using Azure Data Factory, Airflow, and dbt.
• Work with Azure storage, identity, networking, and security services to build secure and scalable data solutions.
• Implement and manage Git-based version control and CI/CD pipelines for data engineering workflows.
• Work with Unity Catalog for data governance, access management, lineage, and discoverability.
• Ensure data quality, reliability, security, and compliance across data pipelines and platforms.
• Collaborate with business, analytics, technology, and cross-functional teams to understand requirements and deliver effective data solutions.
• Troubleshoot data pipeline issues and proactively identify opportunities for process and performance improvements.
• Contribute to data engineering standards, best practices, documentation, and continuous improvement initiatives.

Required Skills & Expertise

• Strong hands-on experience with Databricks and Apache Spark.
• Advanced proficiency in Python, PySpark, and SQL.
• Strong understanding of Delta Lake, Lakehouse architecture, Medallion architecture, and data modelling.
• Experience working with Databricks Notebooks, Clusters, Jobs, Workflows, and SQL Warehouses.
• Good understanding of Azure cloud data services, including storage, identity, networking, and security.
• Experience with Azure Data Factory, Airflow, dbt, Git, and CI/CD.
• Knowledge of Unity Catalog, data governance, lineage, data quality, and performance tuning.
• Strong analytical and problem-solving skills.
• Good communication and stakeholder-management skills.
• Ability to work effectively in a full-time work-from-office environment.

Experience & Qualification

Years of Experience: 57 Years

Relevant Experience: Minimum 35 years of hands-on data engineering experience,

including Databricks implementation and delivery.

Educational Qualification

• BE / B. Tech. / MCA in Computer Science, Information Technology, Engineering, or a related discipline.

Additional Qualification

• Databricks certification is preferred.
• Experience or knowledge of pharmaceutical processes and business functions will be an added advantage.