Sr. Software Engineer - Databricks Job

YASH Technologies; Maharashtra; Hybrid; INR 900,000 - 1,200,000; Full time

Job summary

YASH Technologies is seeking a Data Engineer with 4-6 years of experience to design and implement scalable data solutions using Databricks, PySpark, and AWS data services.

You will build and maintain ETL/ELT pipelines (SQL, Python) and migrate DB2/assets to AWS PostgreSQL, ensuring data quality and governance across the modernization journey.

Qualifications

• Strong expertise in SQL, including stored procedures, functions, indexes, query optimization, and performance tuning.
• Hands-on experience with PySpark, Python, and Databricks for designing and developing scalable data solutions.
• Strong experience in designing, developing, and supporting ETL/ELT pipelines for data ingestion, transformation, and integration.
• Experience working with AWS cloud data services such as RDS, S3, and Glue, and building end-to-end data pipelines.
• Strong experience with PostgreSQL, including database design, query development, performance tuning, data modeling, and database administration concepts.
• Experience in data migration, data integration, organization-wide reporting, and building data products.
• Good understanding of Databricks workflow orchestration, pipeline execution, monitoring, and optimization.
• Experience in data cleansing, data validation, reconciliation, and data quality management.
• Strong analytical and problem-solving skills with the ability to organize, analyze, and interpret large datasets.
• Good understanding of data warehousing concepts and best practices.

Responsibilities

• Analyze key mainframe jobs and identify DB2 database components requiring migration.
• Identify and assess DB2 schemas, tables, and dependent objects for migration.
• Lead the decommissioning of DB2 schemas as part of application and database modernization initiatives.
• Plan and execute migration of identified DB2 schemas and data to AWS PostgreSQL.
• Perform schema conversion, data validation, reconciliation, and post-migration testing.
• Migrate identified schemas/data to AMS while ensuring data integrity and business continuity.
• Collaborate with cross-functional teams to support migration planning, cutover activities, and production deployment.
• Ensure compliance with data quality, governance, and security standards throughout the migration lifecycle.

Skills

SQL

Python

PySpark

Databricks

ETL/ELT

PostgreSQL

AWS

Data Migration

Data Warehousing

Data Quality

Tools

Git

CI/CD

Job description

YASH Technologies is a leading technology integrator specializing in helping clients reimagine operating models, enhance competitiveness, optimize costs, foster exceptional stakeholder experiences, and drive business transformation.

At YASH, we’re a cluster of the brightest stars working with cutting‑edge technologies. Our purpose is anchored in a single truth – bringing real positive changes in an increasingly virtual world and it drives us beyond generational gaps and disruptions of the future.

We are looking forward to hire Databricks Professionals in the following areas :

Data Engineer -- Databricks, Python, ETL, Postgres

Job Description

Experience Level 4 -6 Years

• Strong expertise in SQL, including stored procedures, functions, indexes, query optimization, and performance tuning.
• Hands‑on experience with PySpark, Python, and Databricks for designing and developing scalable data solutions.
• Strong experience in designing, developing, and supporting ETL/ELT pipelines for data ingestion, transformation, and integration.
• Experience working with AWS cloud data services such as RDS, S3, and Glue, and building end‑to‑end data pipelines.
• Strong experience with PostgreSQL, including database design, query development, performance tuning, data modeling, and database administration concepts.
• Experience in data migration, data integration, organization‑wide reporting, and building data products.
• Good understanding of Databricks workflow orchestration, pipeline execution, monitoring, and optimization.
• Experience in data cleansing, data validation, reconciliation, and data quality management.
• Strong analytical and problem‑solving skills with the ability to organize, analyze, and interpret large datasets.
• Good understanding of data warehousing concepts and best practices.
• Knowledge of Scala, Git, and CI/CD practices for code versioning and deployment automation.
• Strong verbal, written, and interpersonal communication skills.
• Organized and structured approach with the ability to work independently and collaboratively in a team environment.

Migration & Modernization Responsibilities

• Analyze key mainframe jobs and identify DB2 database components requiring migration.
• Identify and assess DB2 schemas, tables, and dependent objects for migration.
• Lead the decommissioning of DB2 schemas as part of application and database modernization initiatives.
• Plan and execute migration of identified DB2 schemas and data to AWS PostgreSQL.
• Perform schema conversion, data validation, reconciliation, and post‑migration testing.
• Migrate identified schemas/data to AMS while ensuring data integrity and business continuity.
• Collaborate with cross‑functional teams to support migration planning, cutover activities, and production deployment.
• Ensure compliance with data quality, governance, and security standards throughout the migration lifecycle.

Primary Skills

Databricks, PySpark, Python, SQL, PostgreSQL, ETL/ELT, AWS (S3, RDS, Glue), Data Migration

Secondary Skills

Scala, Git, CI/CD, DB2 Migration, Mainframe Modernization, Data Quality, Data Validation & Reconciliation

At YASH, you are empowered to create a career that will take you to where you want to go while working in an inclusive team environment. We leverage career‑oriented skilling models and optimize our collective intelligence aided with technology for continuous learning, unlearning, and relearning at a rapid pace and scale.

Our Hyperlearning workplace is grounded upon four principles

• Flexible work arrangements, Free spirit, and emotional positivity
• Agile self‑determination, trust, transparency, and open collaboration
• All Support needed for the realization of business goals,
• Stable employment with a great atmosphere and ethical corporate culture