Azure Databricks Engineer

Synergech; Chennai District; On-site; INR 3,500,000 - 7,000,000; Full time

Job summary

Synergech in Chennai, India seeks a Senior Data Architect to design and implement enterprise data architecture using Databricks Lakehouse, Delta Lake, and Azure services.

You will build scalable data pipelines, collaborate with actuarial, underwriting and BI teams, and enforce data governance while mentoring teams on cloud data strategies and best practices.

Qualifications

• 7+ years of experience in data architecture or data engineering, with at least 4 years in Databricks.
• Strong experience with insurance data models including policy, claims, premium, fees, agency, underwriting, accounting, and insured domains.
• Experience with cloud platforms (AWS, Azure, or GCP) and modern data lakehouse architecture.
• Proficiency in SQL, PySpark, Delta Lake, and Databricks SQL.
• Experience integrating with BI tools and data governance tools.
• Excellent communication and stakeholder management skills.

Responsibilities

• Design and implement enterprise data architecture leveraging Databricks Lakehouse, Delta Lake, and Azure cloud-native services.
• Create scalable and secure data pipelines handling structured and unstructured data from internal and external sources.
• Author and maintain technological roadmap mapped to business capabilities.
• Experiment and establish adoption blueprint for emerging capabilities within Databricks and data technology.
• Secure data with appropriate NAIC/GDPR/HIPAA-aligned controls.
• Lead design of ETL/ELT pipelines using PySpark, SQL, Delta Live Tables, Lakeflow, DBT, and Databricks Workflows.
• Partner with actuarial, underwriting, and BI teams to design semantic layers and analytics-ready datasets.
• Optimize performance and cost of Databricks clusters and workflows.
• Enable machine learning and predictive analytics by providing clean, governed datasets.
• Review vendor and divisional solutions for scalability, resilience, and cost-effectiveness.
• Establish data quality, lineage, and metadata management frameworks.

Skills

Databricks

PySpark

SQL

Delta Lake

Cloud platforms

Data modeling

Stakeholder mgmt

Communication

Power BI

Education

Bachelor's or Master's in CS/Data Eng/IS

Tools

Databricks

Unity Catalog

Collibra

Alation

DBT

Delta Live Tables

Power BI/Tableau/Looker

Databricks Workflows

Job description

Key Responsibilities:

• Design and implement enterprise data architecture leveraging Databricks Lakehouse, Delta Lake, and Azure cloud-native services
• Create scalable and secure data pipelines that handle structured and unstructured data from internal and external sources.

• Author and maintain technological roadmap with mappings to business capabilities
• Experiment and establish adoption blueprint for emerging capabilities within Databrick and in data technology
• Secure the data with appropriate NIST cyber controls

Data Engineering & Analytics Enablement

• Lead the design of ETL/ELT pipelines using PySpark, SQL, Delta Live Tables, Lakeflow connect, Autoloader, Lake flow declarative pipelines, DBT, and Databricks Workflows.
• Partner with actuarial, underwriting, and BI teams to design semantic layers and analytics-ready datasets.
• Optimize performance and cost of Databricks clusters and workflows.
• Enable machine learning and predictive analytics by providing clean, governed, and feature-rich data sets.

Governance & Quality

• Review solution developed by the divisional teams and vendors to ensure the solution uses modern technology, built to scale, has resilience, and is cost effective to operate.
• Implement data quality, lineage, and metadata management frameworks using tools like Unity Catalog, Collibra, or Alation.
• Establish and enforce data security and compliance policies aligned with insurance regulations (e.g., NAIC, Privacy, HIPAA, GDPR, CFIUS).

Collaboration & Leadership

• Collaborate with business leaders, data engineers, divisional data teams, data visualization experts, and platform head to align architecture with organizational goals.
• Mentor teams on best practices for data modeling, Databricks optimization, and cloud data architecture.
• Evaluate new technologies to continuously improve data strategy and capabilities.

Required Qualifications:

• Bachelors or masters degree in computer science, Data Engineering, Information Systems, or related field.
• 7+ years of experience in data architecture or data engineering, with at least 4 years in Databricks.
• Strong experience with insurance data models, including policy, claims, premium, fees, agency, underwriting, accounting, and insured domains.
• Expertise in cloud platforms (AWS, Azure, or GCP) and modern data lakehouse architecture.
• Proficiency in SQL, PySpark, Delta Lake, and Databricks SQL.
• Experience integrating with BI tools (Power BI, Tableau, Looker) and data governance tools.
• Excellent communication and stakeholder management skills.

Preferred Skills:

• Experience with streaming data frameworks (Kafka, Delta Live Tables).
• Familiarity with AI/ML pipelines in Databricks.

Certification(s):

• - Databricks Certified Data Engineer Professional
• - Azure/AWS Certified Data Architect
• - Insurance Data Management Association (IDMA) certifications