Sr. Software Engineer - Python

YASH Technologies, Inc.; Pune District; On-site; INR 1,200,000 - 2,200,000; Full time

Job summary

YASH Technologies, Inc. is seeking a Python Data Engineer with 4+ years of experience to design and maintain scalable data pipelines using PySpark and SQL within Microsoft Fabric.

You will migrate data from Azure Data Lake and on-prem sources, build ETL/ELT frameworks, and collaborate with analysts to define data models for reporting and analytics. The role requires strong knowledge of Fabric components, Azure/Synapse, and data warehousing concepts, with emphasis on data quality, monitoring, and

Qualifications

• 4+ years of experience as a Data Engineer or similar role.
• Hands-on experience with Python Notebook, PySpark and SQL for large-scale data processing.
• Strong knowledge of Microsoft Fabric components (Lakehouse, Data Factory, Warehouse)
• Strong knowledge of Azure and Synapse ecosystem.
• Experience in data migration projects and building ETL/ELT frameworks.
• Solid understanding of data warehousing concepts, big data architecture, and CI/CD practices.
• Familiarity with monitoring tools (e.g., Azure Monitor, Log Analytics) and automation frameworks.
• Exposure to AI/ML concepts and ability to integrate them into data workflows.
• Experience with version control systems like Git.
• Strong communication skills to work with cross-functional teams.

Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark and SQL within Microsoft Fabric.
• Migrate data from legacy systems (Azure Data Lake, on-prem sources) into Fabric Lakehouse.
• Build ingestion and transformation frameworks for structured and unstructured datasets.
• Collaborate with data analysts and business stakeholders to define data models supporting reporting and analytics.
• Ensure data quality, integrity, and consistency across datasets.
• Optimize workflows for performance, scalability, and cost efficiency in Fabric and Azure environments.
• Establish monitoring and alerting frameworks to ensure reliability of data pipelines in production.
• Drive automation in ETL/ELT processes, testing, and operational tasks.
• Explore and integrate AI capabilities within Fabric for intelligent data processing and advanced analytics.
• Monitor and troubleshoot production data issues, performing root cause analysis.
• Document technical processes, system designs, and operational procedures.

Skills

Python

PySpark

SQL

Data Modeling

Tools

Microsoft Fabric (Lakehouse)

Azure

Data Factory

Azure Synapse

Job description

Select how often (in days) to receive an alert: Create Alert

YASH Technologies is a leading technology integrator specializing in helping clients reimagine operating models, enhance competitiveness, optimize costs, foster exceptional stakeholder experiences, and drive business transformation.

At YASH, we’re a cluster of the brightest stars working with cutting-edge technologies. Our purpose is anchored in a single truth – bringing real positive changes in an increasingly virtual world and it drives us beyond generational gaps and disruptions of the future.

We are looking forward to hirePython Professionals in the following areas :

Experience required: 4-7 years

Primary Skill : Python, PySpark, SQL, Data Modeling

Primary Platform : Python and Microsoft Fabric (Lakehouse, Data Factory, Warehouse), Azure

Roles and Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark and SQL within Microsoft Fabric.
• Migrate data from legacy systems (Azure Data Lake, on-prem sources) into Fabric Lakehouse.
• Build ingestion and transformation frameworks for structured and unstructured datasets.
• Collaborate with data analysts and business stakeholders to define data models supporting reporting and analytics.
• Ensure data quality, integrity, and consistency across datasets.
• Optimize workflows for performance, scalability, and cost efficiency in Fabric and Azure environments.
• Establish monitoring and alerting frameworks to ensure reliability of data pipelines in production.
• Drive automation in ETL/ELT processes, testing, and operational tasks.
• Explore and integrate AI capabilities within Fabric for intelligent data processing and advanced analytics.
• Monitor and troubleshoot production data issues, performing root cause analysis.
• Document technical processes, system designs, and operational procedures.

Must-Have Skills

• 4+ years of experience as a Data Engineer or similar role.
• Hands-on experience with Python Notebook, PySpark and SQL for large-scale data processing.
• Strong knowledge of Microsoft Fabric components (Lakehouse, Data Factory, Warehouse)
• Strong knowledge of Azure and Synapse ecosystem.
• Experience in data migration projects and building ETL/ELT frameworks.
• Solid understanding of data warehousing concepts, big data architecture, and CI/CD practices.
• Familiarity with monitoring tools (e.g., Azure Monitor, Log Analytics) and automation frameworks.
• Exposure to AI/ML concepts and ability to integrate them into data workflows.
• Experience with version control systems like Git.
• Strong communication skills to work with cross-functional teams.

At YASH, you are empowered to create a career that will take you to where you want to go while working in an inclusive team environment.We leverage career-oriented skilling models and optimize our collective intelligence aided with technology for continuous learning, unlearning, and relearning at a rapid pace and scale.

Our Hyperlearning workplace is grounded upon four principles

• Flexible work arrangements, Free spirit, and emotional positivity
• Agile self-determination, trust, transparency, and open collaboration
• All Support needed for the realization of business goals,
• Stable employment with a great atmosphere and ethical corporate culture