Databricks- Resident Solution Architect

Unison Group; Hyderabad; On-site; INR 1,500,000 - 2,000,000; Full time

Job summary

Unison Group is seeking a seasoned Data Engineer in Hyderabad with over 10 years of experience. The candidate will design scalable data architectures with Databricks lakehouse, ensuring alignment with business objectives. Responsibilities include building data pipelines using PySpark, implementing data governance frameworks, and collaborating with teams to translate requirements into solutions. Strong leadership and mentoring experience are essential, as well as expertise in cloud platforms and data processing.

Qualifications

• 10+ years of experience in data engineering, data architecture, or big data platforms.
• At least 4–5 years of hands-on Databricks and Spark experience.
• Databricks certifications preferred with strong leadership and mentoring experience.

Responsibilities

• Design and implement scalable data architectures using Databricks lakehouse.
• Act as Databricks subject matter expert, driving best practices.
• Build and optimize end-to-end data pipelines using PySpark and Spark.
• Implement data governance, security, and access control frameworks.
• Collaborate with stakeholders and mentor junior team members.

Skills

Python

PySpark

Spark SQL

SQL

Databricks

Cloud platforms

Tools

Microsoft Azure

Data Factory

ADLS

Job description

Responsibilities

• Design and implement scalable data architectures using Databricks lakehouse, ensuring alignment with enterprise data strategy and business objectives
• Act as the resident Databricks subject matter expert, driving best practices across Delta Lake, Spark optimization, and workload performance tuning
• Build and optimize end‑to‑end data pipelines using PySpark and Spark, including modernization and migration of legacy ETL and PL/SQL workloads to cloud‑based platforms
• Implement data governance, security, and access control frameworks such as Unity Catalog to ensure compliance and data integrity across platforms
• Collaborate with business stakeholders, data engineers, and analytics teams to translate requirements into scalable solutions while mentoring junior team members and enforcing engineering standards

Qualifications

• 10+ years of experience in data engineering, data architecture, or big data platforms with at least 4–5 years of hands‑on Databricks and Spark experience
• Strong proficiency in Python, PySpark, Spark SQL, and SQL with experience handling large‑scale distributed data processing
• Hands‑on experience with cloud platforms, preferably Microsoft Azure including Azure Databricks, Data Factory, and ADLS
• Proven expertise in lakehouse architecture, data modeling, performance tuning, and ETL framework design
• Databricks certifications preferred along with strong leadership, stakeholder management, and mentoring experience