Databricks engineer

Ajanta Pharma Ltd; Mumbai; On-site; INR 1,800,000 - 3,000,000; Full time

Benefits offered by this job

Free Gym Facility

Subsidized Food

Meaningful Data Initiatives

Collaborative Work Environment

Job summary

Ajanta Pharma Ltd in Mumbai is seeking an experienced Databricks Engineer to join our Data Engineering team, designing and optimizing scalable data solutions for pharma processes. The ideal candidate has hands-on experience with Databricks, Python, PySpark, SQL, Apache Spark, and Azure cloud data platforms, plus knowledge of Delta Lake and data governance.

Collaborate with analytics and technology teams to deliver reliable data pipelines, ensure data quality and security, and support enterprise

Qualifications

• 5–7 years of hands-on data engineering experience.
• Experience with Databricks and Spark is required.
• Databricks certification preferred.

Responsibilities

• Design, develop, and maintain scalable data engineering solutions using Databricks, Python, PySpark, SQL, and Apache Spark.
• Build and optimize Databricks notebooks, clusters, Jobs, Workflows, and SQL Warehouses.
• Develop efficient data pipelines and transformation frameworks for large-scale data processing.
• Work with Delta Lake, Lakehouse architecture, Medallion architecture (Bronze, Silver, Gold), and modern data-modeling practices.
• Optimize Spark jobs, queries, clusters, and data pipelines for performance, scalability, and cost efficiency.
• Develop and maintain data pipelines using Azure Data Factory, Airflow, and dbt.
• Work with Azure storage, identity, networking, and security services to build secure and scalable data solutions.
• Implement and manage Git-based version control and CI/CD pipelines for data engineering workflows.
• Work with Unity Catalog for data governance, access management, lineage, and discoverability.
• Ensure data quality, reliability, security, and compliance across data pipelines and platforms.
• Collaborate with business, analytics, technology, and cross-functional teams to understand requirements and deliver effective data solutions.
• Troubleshoot data pipeline issues and proactively identify opportunities for process and performance improvements.
• Contribute to data engineering standards, best practices, documentation, and continuous improvement initiatives.

Skills

Databricks

Python

PySpark

SQL

Apache Spark

Azure cloud data platforms

Delta Lake

Lakehouse architecture

Medallion architecture

Data modelling

Git

CI/CD

Education

BE / B. Tech. / MCA in CS/IT/Engineering

Tools

Databricks Notebooks

Databricks Clusters

Databricks Jobs

Databricks Workflows

SQL Warehouses

Job description

Employment Type: Full-Time | Work from Office

Experience: 5–7 Years

Function: IT

About the Role

We are looking for an experienced Databricks Engineer to join our Data Engineering team and play a key role in designing, developing, and optimizing scalable data solutions.

The ideal candidate will have strong hands‑on experience with Databricks, Python, PySpark, SQL, Apache Spark, and Azure cloud data platforms, along with a solid understanding of modern data architecture, governance, and data engineering best practices.

Experience in the pharmaceutical industry or pharma business processes will be an added advantage.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data engineering solutions using Databricks, Python, PySpark, SQL, and Apache Spark.
• Build and optimize Databricks notebooks, clusters, Jobs, Workflows, and SQL Warehouses.
• Develop efficient data pipelines and transformation frameworks for large‑scale data processing.
• Work with Delta Lake, Lakehouse architecture, Medallion architecture (Bronze, Silver, Gold), and modern data‑modeling practices.
• Optimize Spark jobs, queries, clusters, and data pipelines for performance, scalability, and cost efficiency.
• Develop and maintain data pipelines using Azure Data Factory, Airflow, and dbt.
• Work with Azure storage, identity, networking, and security services to build secure and scalable data solutions.
• Implement and manage Git‑based version control and CI/CD pipelines for data engineering workflows.
• Work with Unity Catalog for data governance, access management, lineage, and discoverability.
• Ensure data quality, reliability, security, and compliance across data pipelines and platforms.
• Collaborate with business, analytics, technology, and cross‑functional teams to understand requirements and deliver effective data solutions.
• Troubleshoot data pipeline issues and proactively identify opportunities for process and performance improvements.
• Contribute to data engineering standards, best practices, documentation, and continuous improvement initiatives.

Required Skills & Expertise

• Strong hands‑on experience with Databricks and Apache Spark.
• Advanced proficiency in Python, PySpark, and SQL.
• Strong understanding of Delta Lake, Lakehouse architecture, Medallion architecture, and data modelling.
• Experience working with Databricks Notebooks, Clusters, Jobs, Workflows, and SQL Warehouses.
• Good understanding of Azure cloud data services, including storage, identity, networking, and security.
• Experience with Azure Data Factory, Airflow, dbt, Git, and CI/CD.
• Knowledge of Unity Catalog, data governance, lineage, data quality, and performance tuning.
• Strong analytical and problem‑solving skills.
• Good communication and stakeholder‑management skills.
• Ability to work effectively in a full‑time work‑from‑office environment.

Experience & Qualification

Years of Experience: 5–7 Years

Relevant Experience: Minimum 3–5 years of hands‑on data engineering experience, including Databricks implementation and delivery.

Educational Qualification:

• BE / B. Tech. / MCA in Computer Science, Information Technology, Engineering, or a related discipline.

Additional Qualification:

• Databricks certification is preferred.
• Experience or knowledge of pharmaceutical processes and business functions will be an added advantage.

What We Offer

• Free Gym Facility – Supporting employee health and wellness.
• Subsidized Food – Convenient and affordable meals at the workplace.
• Opportunity to Work on Meaningful Data Initiatives – Exposure to enterprise‑scale data engineering, analytics, and digital transformation projects.
• Collaborative Work Environment – Work alongside experienced technology and business teams on impactful projects.