Solution Platform Engineer (Azure Databricks)

Prescience Decision Solutions; Bengaluru; On-site; INR 3,000,000 - 6,000,000; Full time

Job summary

Prescience Decision Solutions in Bengaluru, India, seeks a senior Data Engineer to lead Databricks-based platform modernization engagements for Fortune 500 and mid-sized clients. You will design and implement secure, scalable data pipelines, mentor a technical team, and help advance our Data Engineering practice across AI, Gen AI, ML, and AI Ops initiatives.

Join a fast-growing, equal-opportunity company with a strong project pipeline and collaborative culture.

Qualifications

• 12-15 years of experience in Data Engineering, architecting scalable solutions.
• 3+ years hands-on Azure Databricks (Unity Catalog, DLT, Workflows, Delta Lake).
• Databricks Certified Data Engineer Professional or Platform Administrator certification preferred.
• Strong technical experience in SQL, Python, PySpark, SparkSQL.
• Experience with legacy to modern data platform migration.
• Hands on experience with Open-source stack for Lakehouse or Warehouse implementation.
• Working experience with big data (gigabytes to petabytes) in prior projects.
• Tech leadership roles with exposure to pre-sales, solutioning and client management.
• Data Modeling, Data Governance and Data Security tools and solutions.
• Strong verbal and written communication skills.
• Competitive salary and performance-based bonuses.

Responsibilities

• Lead hands-on implementation of Databricks environments including workspace configuration, cluster size and policies, networking, security perimeters, Infrastructure-as-Code, CI/CD and Azure cloud integration.
• Architect and implement Unity Catalog for unified data governance, including metastore setup, catalog/schema design, data lineage, access control policies, and cross-workspace sharing.
• Define and enforce platform-level security standards covering identity management, row/column-level security, credential management, private endpoints, and audit logging.
• Design and review scalable ETL/ELT pipelines using Delta Live Tables, Databricks Workflows, and open-source frameworks.
• Team management responsibilities – technical mentoring of the team, interviewing and identifying right candidates for the team and training to develop the skills
• Technical review, mentoring and oversight of technical deliverables.
• Thought leadership – Develop a robust Data Engineering practice in conjunction with AI / Gen AI/ ML / AI Ops.
• Collaborate on client pitches, RFP responses, and solution proposals specific to Databricks-based platform modernization engagements.

Skills

Azure Databricks

Unity Catalog

Delta Lake

Python

SQL

SparkSQL

Delta Live Tables

Data Modeling

Data Governance

Data Security

Leadership

Pre-sales

Job description

Prescience Decision Solutions | Full time

Prescience Decision Solutions, established in 2017, partners with Fortune 500 and mid-sized companies globally to unlock enterprise data and drive improved decision-making. We specialize in Data Science, Advanced Analytics, and Data Engineering, integrating AI and machine learning to deliver measurable business value and ROI.

Headquartered in Bangalore, our ‘Business Backward’ approach ensures a deep understanding of your challenges, enabling us to provide tailored, impactful solutions.

Founded by industry leaders from IIT and IIM, Prescience is a bootstrapped and profitable company with a strong project pipeline.

Our commitment to excellence has earned us recognition as one of the top three startups in the People Excellence category by xto10x, with a 97 percentile employee NPS, 100% mutual respect among employees, and a 98% employee pride rate.

As an equal opportunity employer, we are dedicated to fostering a diverse and inclusive workplace, free from discrimination based on caste, religion, gender, sexual orientation, nationality, or other protected characteristics.

Role Overview

This roles involves developing solutions for specific requirements. Coming up with ideas/solutions/proposals for pitching and help in conversion conversation and developing Reference implementations on new technologies – the technology new in the market or maybe new to the client context.

Job Description

• Solutioning – developing solutions for specific client requirements. Coming up with ideas/solutions/proposals for pitching and help in conversion conversations
  
• Developing Reference implementations on new technologies – the technology maybe new in the market or maybe new to the client context
  
• Lead hands-on implementation of Databricks environments including workspace configuration, cluster size and policies, networking, security perimeters, Infrastructure-as-Code, CI/CD and Azure cloud integration.
  
• Architect and implement Unity Catalog for unified data governance, including metastore setup, catalog/schema design, data lineage, access control policies, and cross-workspace sharing.
  
• Define and enforce platform-level security standards covering identity management, row/column-level security, credential management, private endpoints, and audit logging.
  
• Design and review scalable ETL/ELT pipelines using Delta Live Tables, Databricks Workflows, and open-source frameworks.
  
• Team management responsibilities – technical mentoring of the team, interviewing and identifying right candidates for the team and training to develop the skills
  
• Technical review, mentoring and oversight of technical deliverables.
  
• Thought leadership – Develop a robust Data Engineering practice in conjunction with areas such as AI / Gen AI/ ML / AI Ops etc.
  
• Collaborate on client pitches, RFP responses, and solution proposals specific to Databricks-based platform modernization engagements.
  

Requirements

• 12-15 years of experience in Data Engineering, architecting scalable solutions.
  
• 3+ years hands-on Azure Databricks (Unity Catalog, DLT, Workflows, Delta Lake)
  
• Databricks Certified Data Engineer Professional or Platform Administrator certification preferred
  
• Strong technical experience in SQL, Python, PySpark, SparkSQL
  
• Experience with legacy to modern data platform migration
  
• Hands on experience with Open-source stack for Lakehouse or Warehouse implementation.
  
• Working experience with big data (gigabytes to petabytes) in prior projects
  
• Demonstrated Tech leadership roles in recent past with significant exposure to pre-sales, solutioning and managing clients.
  
• Hands-on experience and good understanding of Data Modelling, Data Governance and Data Security tools and solutions.
  
• Strong verbal and written communication skills.
  
• Competitive salary and performance-based bonuses.
  
• Collaborative and supportive work environment.
  
• Chance to learn and grow with a talented team.