Engineering Expert - Business Data Cloud - Data Product Runtime Team

SAP; São Leopoldo; Híbrido; BRL 120.000 - 160.000; Tempo integral

Resumo da oferta

A leading software company in São Leopoldo is seeking an Engineering Expert to optimize data product runtime architecture. This role involves analyzing Spark workloads, implementing monitoring solutions, and mentoring engineering teams. The ideal candidate should possess deep knowledge of distributed computing and expertise in performance tuning. Join a collaborative team focused on innovation and excellence.

Qualificações

• Deep familiarity with distributed computing, concurrency, and fault tolerance.
• Comprehensive knowledge of Apache Spark architecture.
• Expertise in performance profiling and tuning Spark applications.
• Skilled in writing efficient SQL queries and transformations.
• Strong debugging skills to resolve runtime issues.
• Experience with monitoring tools for diagnosing performance.

Responsabilidades

• Analyze Spark workloads for performance optimization.
• Implement monitoring solutions for diagnosing issues.
• Collaborate to build scalable frameworks.
• Document best practices for Spark optimization.
• Lead AI-automation initiatives to improve scalability.
• Mentor engineering teams across the globe.

Conhecimentos

Distributed computing principles

Apache Spark architecture

Performance profiling and tuning

SQL queries and transformations

Debugging skills

Monitoring tools (Spark UI, Grafana)

AI and ML trends

Leadership and mentorship

Strategic vision

Descrição da oferta de emprego

Overview

We help the world run better. At SAP, we enable you to bring out your best. Our company culture is focused on collaboration and a shared passion to help the world run better. We focus every day on building the foundation for tomorrow and creating a workplace that embraces differences, values flexibility, and is aligned to our purpose-driven and future-focused work. We offer a highly collaborative, caring team environment with a strong focus on learning and development, recognition for your individual contributions, and a variety of benefit options for you to choose from.

Join our team as an Engineering Expert to lead cutting-edge projects focused on optimizing our data product runtime architecture within SAP Business Data Cloud. Your expertise will drive innovation and strategic decision-making.

Responsibilities

• Analyze and refine Spark workloads to enhance performance and resource efficiency. This includes tuning parameters, re-engineering data processing pipelines, and ensuring optimal execution strategies.
• Implement monitoring solutions to detect and diagnose performance bottlenecks or failures in Spark applications. Solve complex runtime issues related to resource allocation, execution errors, or data inconsistencies.
• Collaborate with various parts of the organization to design and build a framework that allows Business Data Cloud workloads to scale effortlessly and elastically across distributed data environments.
• Produce documentation on best practices for Spark optimization and conduct training sessions or workshops to enhance the broader team’s expertise in performance tuning and support strategies.
• Lead AI-automation initiatives to improve workload scalability and efficiency.
• Mentor engineering teams and lead cross-functional global collaborations

Qualifications

• Deep familiarity with principles of distributed computing, including concurrency, fault tolerance, and network latency, which are essential for optimizing distributed data processing.
• Comprehensive knowledge of Apache Spark architecture, including its core components like the driver, executor, and cluster manager, as well as Spark's execution model.
• Expertise in performance profiling and tuning Spark applications, including optimizing resource allocation, parallelism, and shuffling processes to reduce execution time and improve efficiency.
• Skilled in writing efficient SQL queries and transformations using SparkSQL and DataFrames, optimizing operations to reduce computation overhead.
• Strong debugging skills to identify and resolve runtime issues, optimize code paths, and rectify configuration or environment-related problems.
• Experience with tools such as Spark's web UI, Ganglia, Grafana, or Prometheus for monitoring application status and diagnosing performance bottlenecks.
• Comprehensive understanding of AI, ML trends and their applications in the distributed Data query execution space.
• Excellent leadership, mentorship, and communication skills.
• Strategic vision and deep analysis skills to foresee industry advancement.

Inclusion and Legal Notice

SAP’s culture of inclusion, focus on health and well-being, and flexible working models help ensure that everyone – regardless of background – feels included and can run at their best. SAP is proud to be an equal opportunity workplace and is an affirmative action employer. We are committed to the values of Equal Employment Opportunity and provide accessibility accommodations to applicants with physical and/or mental disabilities. If you are interested in applying for employment with SAP and are in need of accommodation or special assistance to navigate our website or to complete your application, please send an e-mail with your request to Recruiting Operations Team: [e-mail]. For SAP employees: Only permanent roles are eligible for the SAP Employee Referral Program, according to the eligibility rules set in the SAP Referral Policy. Specific conditions may apply for roles in Vocational Training.

EOE AA M/F/Vet/Disability: Qualified applicants will receive consideration for employment without regard to their age, race, religion, national origin, ethnicity, age, gender (including pregnancy, childbirth, et al), sexual orientation, gender identity or expression, protected veteran status, or disability. Successful candidates might be required to undergo a background verification with an external vendor.

Requisition ID: 430179 | Work Area: Software-Design and Development | Expected Travel: 0 - 10% | Career Status: Professional | Employment Type: Regular Full Time | Additional Locations: #LI-Hybrid.