Data Engineer - PySpark Job

Yash Technologies; Bengaluru; On-site; INR 1,800,000 - 3,600,000; Full time

Job summary

YASH Technologies is seeking Senior Data Engineer with 5-12 years of experience to design and build data pipelines using Palantir Foundry, AWS, and Snowflake. You will partner with data scientists and business stakeholders to deliver scalable data platforms for analytics.

The role emphasizes data governance, ontology modeling, and cost-efficient pipeline optimization, with opportunities to mentor teammates in a collaborative, innovation-driven environment.

Qualifications

• 5-12 years of experience in data engineering, analytics, or software roles.
• Strong hands-on experience with PySpark, Palantir Foundry, AWS, including ontology modelling, pipelines, data fusion, and transformation logic.
• Proficient in Python, SQL, Snowflake and data processing frameworks.
• Solid understanding of data integration, ETL workflows, and data modelling principles.
• Experience with relational databases, data lakes, cloud data warehouses (AWS, Azure, GCP, Snowflake).
• Strong problem-solving skills and ability to debug complex data workflows.
• Familiarity with CI/CD pipelines and version control tools like Git.
• Excellent communication and ability to work with cross-functional teams.

Responsibilities

• Design, implement, and maintain scalable data pipelines using Palantir Foundry and AWS.
• Solid experience in web front-end development and web services.
• Collaborate with data scientists, analysts, and stakeholders to translate data requirements into technical solutions.
• Build reusable data assets, integrate data from multiple sources, and ensure data quality and security.
• Develop transformation workflows, ontology models, and data governance within Foundry.
• Optimize data pipelines for performance, scalability, and cost efficiency.
• Troubleshoot, monitor, and resolve data issues and pipeline failures in real-time.
• Mentor junior engineers and contribute to team knowledge sharing and documentation.

Skills

PySpark

Palantir Foundry

AWS

Python

SQL

Data modelling

CI/CD

Git

Communication

Tools

Snowflake

Job description

Select how often (in days) to receive an alert: Create Alert

YASH Technologies is a leading technology integrator specializing in helping clients reimagine operating models, enhance competitiveness, optimize costs, foster exceptional stakeholder experiences, and drive business transformation.

At YASH, we’re a cluster of the brightest stars working with cutting‑edge technologies. Our purpose is anchored in a single truth – bringing real positive changes in an increasingly virtual world and it drives us beyond generational gaps and disruptions of the future.

We are looking forward to hirePySpark Professionals in the following areas :

Experience required: 5-12 years

We are seeking a highly motivated and technically skilled Senior Data Engineer with strong expertise in data pipeline development and advanced data modelling to join our data team. The ideal candidate will have extensive experience on designing, building, and refactoring scalable data pipelines and data models within a modern data engineering ecosystem using Palantir Foundry, AWS & Snowflake, along with broader expertise in data architecture, governance, and processing. This role offers the opportunity to work on cutting‑edge data platforms and deliver impactful solutions for enterprise data integration, transformation, and advanced analytics.

Key Responsibilities:

• Design, implement, and maintain scalable and reliable data pipelines using Palantir Foundry, AWS
• Solid experience in web front‑end development, web‑services.
• Some exposure to business analytics, financial forecasting, or a related domain would be preferable.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
• Build reusable data assets, integrate data from multiple sources, and ensure data quality, integrity, and security.
• Develop and maintain transformation workflows, ontology models, and data governance frameworks within Foundry.
• Optimize data pipelines for performance, scalability, and cost efficiency.
• Troubleshoot, monitor, and resolve data issues and pipeline failures in real‑time.
• Implement best practices in data versioning, lineage, and observability.
• Assist in defining data architecture strategies and tooling roadmaps.
• Mentor junior engineers and contribute to team knowledge sharing and documentation.

Required Qualifications:

• 5-12 years of experience in data engineering, analytics, or software engineering roles.
• Strong hands‑on experience with Pyspark, Palantir Foundry, AWS, including ontology modelling, pipelines, data fusion, and transformation logic.
• Proficient in Python, SQL, Snowflake and data processing frameworks.
• Solid understanding of data integration, ETL workflows, and data modelling principles.
• Experience working with relational databases, data lakes, cloud data warehouses (AWS, Azure, GCP, Snowflake, Web‑services).
• Strong problem‑solving skills and ability to debug complex data workflows.
• Familiarity with CI/CD pipelines and version control tools like Git.
• Excellent communication skills and ability to work collaboratively with cross‑functional teams.

At YASH, you are empowered to create a career that will take you to where you want to go while working in an inclusive team environment.We leverage career‑oriented skilling models and optimize our collective intelligence aided with technology for continuous learning, unlearning, and relearning at a rapid pace and scale.

Our Hyperlearning workplace is grounded upon four principles

• Flexible work arrangements, Free spirit, and emotional positivity
• Agile self‑determination, trust, transparency, and open collaboration
• All Support needed for the realization of business goals,
• Stable employment with a great atmosphere and ethical corporate culture