Senior Data Platform Engineer

Avisoft; Hyderabad, Pune District; On-site; INR 3,000,000 - 5,400,000; Full time

Job summary

Avisoft in Hyderabad is seeking a Senior Data Platform Engineer to design, build, and scale enterprise data platforms on Azure and Databricks. You will own end-to-end data pipelines and metadata-driven ingestion frameworks.

The role requires 8+ years in data engineering with strong PySpark, SQL, Python, and cloud-native architectures. You will collaborate with cross-functional teams to deliver production-ready solutions while upholding quality, security, and performance.

Qualifications

• 8+ years of experience in Data Engineering.
• Strong PySpark, Azure, and Databricks expertise.
• Experience with enterprise-scale data platforms and cloud-native architectures.

Responsibilities

• Design, develop, and optimize scalable data pipelines using Azure and Databricks.
• Build and enhance metadata-driven ingestion frameworks.
• Develop batch and streaming data solutions using PySpark.
• Ensure data quality, performance, reliability, and security.
• Collaborate with cross-functional teams to deliver production-ready data solutions.
• Drive engineering best practices, code reviews, and platform optimization.

Skills

PySpark

SQL

Python

Data Engineering

Tools

Azure Data Services

Databricks

Databricks DLT

Databricks Autoloader

Databricks Workflows

Job description

About the Role
We are looking for a Senior Data Platform Engineer with strong hands-on experience in building and scaling enterprise data platforms on Azure and Databricks.

Mandatory Skills

• PySpark
• Azure Data Services
• Databricks (DLT, Autoloader, Workflows)
• Data Engineering and ETL/ELT development
• SQL and Python

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using Azure and Databricks.
• Build and enhance metadata-driven ingestion frameworks.
• Develop batch and streaming data solutions using PySpark.
• Ensure data quality, performance, reliability, and security.
• Collaborate with cross-functional teams to deliver production-ready data solutions.
• Drive engineering best practices, code reviews, and platform optimization.

Experience

• 8+ years of experience in Data Engineering.
• Strong expertise in PySpark, Azure, and Databricks is mandatory.
• Experience with enterprise-scale data platforms and cloud-native architectures.