Sr. Software Engineer - Azure Databricks Job

YASH Technologies; Hyderabad; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

YASH Technologies is seeking a Senior Software Engineer – Azure Data Engineer / Databricks to design and deliver enterprise-grade data solutions on Azure.

You will build scalable data pipelines, optimize Spark workloads, and collaborate with stakeholders across analytics and AI teams to drive digital transformation.

Qualifications

• Strong experience in Azure Databricks, Spark, and PySpark.
• Hands-on with Azure Data Factory and SQL/Python.
• Experience with Delta Lake and Lakehouse architecture.
• Knowledge of ADLS Gen2, Synapse, and Key Vault.
• CI/CD with Azure DevOps, Git, and automation practices.
• Ability to optimize Spark jobs and data pipelines.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Databricks, PySpark, and Azure Data Factory (ADF).
• Build and optimize ETL/ELT workflows for processing large-scale datasets.
• Develop data solutions using Delta Lake and Medallion Architecture (Bronze, Silver, Gold).
• Integrate data from databases, APIs, cloud storage, and enterprise apps.
• Implement data transformation, cleansing, validation, and quality frameworks.
• Monitor production workloads, troubleshoot data issues, and perform root cause analysis.
• Collaborate with stakeholders, solution architects, data scientists, and dev teams.
• Participate in Agile ceremonies, code reviews, design discussions, and mentoring.

Skills

Azure Databricks

Apache Spark

PySpark

SQL

Python

Delta Lake

Lakehouse Architecture

Azure Data Factory

ADLS Gen2

Azure Synapse Analytics

Azure SQL Database

Azure Key Vault

Azure Monitor

Unity Catalog

Azure

Tools

Azure DevOps

Git

CI/CD

Job description

YASH Technologies is a leading technology integrator specializing in helping clients reimagine operating models, enhance competitiveness, optimize costs, foster exceptional stakeholder experiences, and drive business transformation.

At YASH, we’re a cluster of the brightest stars working with cutting-edge technologies. Our purpose is anchored in a single truth – bringing real positive changes in an increasingly virtual world and it drives us beyond generational gaps and disruptions of the future.

We are looking forward to hire Azure Databricks Professionals in the following areas :

Job Description: Senior Software Engineer - Azure Data Engineer / Databrick s

Position: Senior Software Engineer

Role: Azure Data Engineer / Databricks Engineer

Experience: 5+ Years

• Key Responsibilities Design, develop, and maintain scalable data pipelines using Azure Databricks, PySpark, and Azure Data Factory (ADF). Build and optimize ETL/ELT workflows for processing large-scale structured and unstructured datasets. Develop data solutions using Delta Lake and Medallion Architecture (Bronze, Silver, Gold layers). Integrate data from multiple sources including databases, APIs, cloud storage, and enterprise applications. Implement data transformation, cleansing, validation, and quality frameworks. Optimize Spark jobs, cluster configurations, and SQL queries to improve performance and reduce costs. Develop and maintain data models supporting business intelligence, reporting, and analytics requirements. Utilize Azure services such as ADLS Gen2, Azure Synapse Analytics, Azure SQL Database, Key Vault, and Azure Monitor. Implement CI/CD pipelines using Azure DevOps and Git-based version control practices. Ensure data security, governance, compliance, and access management using Unity Catalog and Azure security best practices. Monitor production workloads, troubleshoot data issues, and perform root cause analysis. Collaborate with business stakeholders, solution architects, data scientists, and development teams to deliver end-to-end data solutions. Participate in Agile ceremonies, code reviews, design discussions, and technical mentoring of junior team members. Support cloud migration and modernization initiatives from on-premises platforms to Azure.
• Required Skills Strong experience in Azure Databricks, Apache Spark, and PySpark. Hands-on experience with Azure Data Factory (ADF). Proficiency in SQL and Python. Experience with Delta Lake and Lakehouse Architecture. Strong understanding of Azure Data Lake Storage (ADLS Gen2). Experience with Azure DevOps, Git, and CI/CD implementations. Knowledge of Azure Synapse Analytics, Azure SQL Database, and Azure Key Vault. Expertise in data modeling, performance tuning, and data warehousing concepts. Strong analytical, problem-solving, and communication skills.

Role Summary

The Senior Software Engineer will be responsible for designing and delivering enterprise-grade data engineering solutions on Azure, leveraging Databricks, Spark, and Azure-native services to support business intelligence, analytics, and digital transformation initiatives.

At YASH, you are empowered to create a career that will take you to where you want to go while working in an inclusive team environment. We leverage career-oriented skilling models and optimize our collective intelligence aided with technology for continuous learning, unlearning, and relearning at a rapid pace and scale.

Our Hyperlearning workplace is grounded upon four principles

• Flexible work arrangements, Free spirit, and emotional positivity
• Agile self-determination, trust, transparency, and open collaboration
• All Support needed for the realization of business goals,
• Stable employment with a great atmosphere and ethical corporate culture