Sr. Data Analyst - PySpark Job

YASH Technologies, Inc.; Bengaluru; On-site; INR 1,800,000 - 2,400,000; Full time

Job summary

YASH Technologies, Inc. seeks PySpark professionals to design and develop scalable data processing solutions using PySpark in Microsoft Fabric.

The role involves building ETL/ELT pipelines and optimizing Spark DataFrames and Spark SQL for large-scale datasets. You will develop data ingestion, transformation, and aggregation workflows using Fabric Notebooks, Lakehouse, and Data Factory Pipelines within a collaborative, inclusive workplace at YASH.

Qualifications

• Hands-on experience designing and developing scalable data processing solutions using PySpark in Microsoft Fabric.
• Experience building and optimizing ETL/ELT pipelines for processing large-scale structured and semi-structured datasets using Spark DataFrames and Spark SQL.
• Experience developing data ingestion, transformation, and aggregation workflows using Fabric Notebooks, Lakehouse, and Data Factory Pipelines.

Responsibilities

• Design and implement scalable PySpark data processing solutions.
• Develop and optimize ETL/ELT pipelines for large datasets with Spark DataFrames and Spark SQL.
• Build data ingestion, transformation, and aggregation workflows using Fabric Notebooks, Lakehouse, and Data Factory Pipelines.

Skills

PySpark

Spark SQL

ETL/ELT

Data processing

Tools

Microsoft Fabric

Azure Data Factory

Fabric Notebooks

Job description

Select how often (in days) to receive an alert: Create Alert

YASH Technologies is a leading technology integrator specializing in helping clients reimagine operating models, enhance competitiveness, optimize costs, foster exceptional stakeholder experiences, and drive business transformation.

At YASH, we're a cluster of the brightest stars working with cutting-edge technologies. Our purpose is anchored in a single truth - bringing real positive changes in an increasingly virtual world and it drives us beyond generational gaps and disruptions of the future.

We are looking forward to hirePySpark Professionals in the following areas :

Job Description:

Experience required: 5-7 years.

Experience required:

• Hands-on experience in designing and developing scalable data processing solutions using PySpark in Microsoft Fabric.
• Experience building and optimizing ETL/ELT pipelines for processing large-scale structured and semi-structured datasets using Spark DataFrames and Spark SQL.
• Experience developing data ingestion, transformation, and aggregation workflows using Fabric Notebooks, Lakehouse, and Data Factory Pipelines.

At YASH, you are empowered to create a career that will take you to where you want to go while working in an inclusive team environment.We leverage career-oriented skilling models and optimize our collective intelligence aided with technology for continuous learning, unlearning, and relearning at a rapid pace and scale.

Our Hyperlearning workplace is grounded upon four principles

• Flexible work arrangements, Free spirit, and emotional positivity
• Agile self-determination, trust, transparency, and open collaboration
• All Support needed for the realization of business goals,
• Stable employment with a great atmosphere and ethical corporate culture