Senior Data Engineer - PySpark and Python

SAP SE; Vancouver; On-site; CAD 90,000 - 120,000; Full time

Job summary

A leading technology company is seeking a Senior Data Engineer in Vancouver to design and maintain data infrastructure. The ideal candidate will have over 5 years of experience in data engineering, proficiency in PySpark, SQL, and AWS tools, along with strong analytical and stakeholder communication skills. You will work closely with various teams to generate insights that drive business decisions and support product performance improvement.

Qualifications

• 5+ years of data engineering experience.
• Highly experienced with data-driven decision making.
• Proficient in analytics and data visualization.
• Experience developing dashboards and data models.

Responsibilities

• Provide insights into customer behavior and operational efficiency.
• Define key success metrics for data-driven decision-making.
• Partner with teams to identify business pain points.
• Build a large-scale, production-grade data pipeline.

Skills

Data engineering experience

Data-driven decision making

Analytics and data visualization

Stakeholder communication

PySpark

SQL

Python

AWS tooling

Databricks or Delta Lake

Data privacy implementation

Tools

AWS Glue

AWS RedShift

AWS DynamoDB

AWS Athena

Job description

Senior Data Engineer - PySpark and Python

We help the world run better. At SAP, we keep it simple: you bring your best to us, and we'll bring out the best in you. We're builders touching over 20 industries and 80% of global commerce, and we need your unique talents to help shape what's next.

We are looking for a Senior Data Engineer to bridge the gap between data engineering and business analytics. This role requires building robust data pipelines, integrating disjointed data sources, and generating meaningful insights to drive key business and product decisions.

You will design and maintain data infrastructure, ensuring that clean, reliable, and structured data is readily available for decision-making. Additionally, you will build reports, dashboards, and assess. You will be working closely with the science, engineering, and product teams to measure product success and improve performance.

This is a high-impact individual contributor role for someone who thrives in a fast-paced, data-driven environment, with a passion for both engineering and analytics.

Required Skills and Experience:

• 5+ years of data engineering experience
• Highly experienced with data-driven decision making
• Proficient in analytics and data visualization, experienced with stakeholder communication
• Experience developing dashboards, reports, and data models to help teams track product performance and business health.
• Proficient in PySpark, SQL, Python
• Proficient in AWS tooling, including Glue, RedShift, Dynamo, Athena (SQL). Alternatively, experience with serverless data management in other cloud platforms, e.g. Azure, Google Cloud
• Experienced with implementation, maintenance, and conducting queries of distributed production-grade python applications, preferably on top of big data

Preferred Skills and Experience:

• Experience with Databricks or Delta Lake
• Experience with vector database
• Experience with data privacy and its implementation in data pipelines

Tasks:

• Provide insights into customer behavior, feature adoption, and operational efficiency.
• Define key success metrics and establish best practices for data-driven decision-making.
• Partner with product managers, engineers, and leadership to identify business pain points, actionable insights, define and track KPIs.
• Use data to drive feature prioritization, roadmap planning, and go-to-market strategies.
• Build a distributed large-scale, production-grade data pipeline on big data in AWS
• Maintain a robust and scalable data pipeline to ensure it meets needs from product, data privacy, and compliance perspective

SAP is committed to the values of Equal Employment Opportunity and provides accessibility accommodations to applicants with physical and/or mental disabilities. Qualified applicants will receive consideration for employment without regard to their age, race, religion, national origin, ethnicity, gender (including pregnancy, childbirth, et al), sexual orientation, gender identity or expression, protected veteran status, or disability, in compliance with applicable federal, state, and local legal requirements.