SAP BW

Infosys; Hyderabad; On-site; INR 1,800,000 - 3,600,000; Full time

Job summary

Infosys in Hyderabad seeks an experienced Data Engineer to lead end-to-end data engineering solutions using Databricks and PySpark for scalable processing.

You will collaborate with SAP BW stakeholders to design modern data platform solutions, optimize batch and near-real-time pipelines, ensure data quality, and mentor engineers in best practices and secure code. The role emphasizes delivery governance in an agile environment.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• 7-9 years of experience with Databricks and PySpark for large-scale data processing.
• Strong experience with SAP BW concepts and data structures.

Responsibilities

• Lead end-to-end delivery of data engineering solutions leveraging Databricks and PySpark.
• Design, build, and optimize batch/near-real-time pipelines for performance, cost efficiency, and reliability.
• Collaborate with SAP BW stakeholders to translate requirements into modern data platform designs.
• Mentor engineers, conduct code reviews, and enforce secure, maintainable implementations.

Skills

Databricks

PySpark

SAP BW

Data modeling

ETL/ELT

Performance tuning

Sprint planning

Leadership

Code reviews

Education

B.Tech / M.Tech / MCA / MSC

Tools

Delta Lake

Databricks notebooks

CI/CD tooling

Job description

Technology->Business Intelligence - Reporting->SAP BW

Key Responsibilities:

• Lead end-to-end delivery of data engineering solutions leveraging Databricks and PySpark for scalable processing and transformation.
• Design, build, and optimize batch/near-real-time pipelines with strong focus on performance, cost efficiency, and reliability.
• Collaborate with SAP BW stakeholders to understand source models, data semantics, and reporting needs; translate them into modern data platform designs.
• Develop reusable frameworks, coding standards, and best practices for Spark-based development and notebook/job orchestration.
• Ensure data quality through validations, reconciliation checks, and robust error handling/monitoring across pipelines.
• Drive technical discussions, estimate work, plan sprints, and manage delivery risks/dependencies for the team.
• Mentor engineers, conduct code reviews, and enforce secure, maintainable, and well-documented implementations.
• Partner with platform/DevOps teams to support CI/CD, environment management, and production readiness for Databricks workloads.

Minimum Qualifications:

• Bachelor's or Master's degree (BTECH, MTECH, MCA, MSC) in Computer Science, Engineering, or a related field.
• 7-9 years of overall experience with strong hands-on expertise in Databricks and PySpark for large-scale data processing.
• Strong experience working with SAP BW concepts and data structures to support analytics/reporting requirements.
• Solid understanding of data modeling, ETL/ELT patterns, and performance tuning in distributed data processing environments.
• Ability to lead technical delivery, collaborate with stakeholders, and communicate clearly across teams.

Preferred Qualifications:

• Experience modernizing or integrating SAP BW data with cloud-based lakehouse architectures using Databricks.
• Strong knowledge of Spark optimization techniques (partitioning, caching, joins, file sizing) and production-grade pipeline design.
• Exposure to Delta Lake patterns (ACID tables, MERGE, time travel) and building curated layers for analytics consumption.
• Experience with workflow orchestration and operational monitoring for data pipelines, including alerting and SLA management.
• Proven ability to guide teams through design reviews, code quality improvements, and delivery governance in agile environments.