Senior Big Data Platform Engineer: Scale Petabyte Data & AI

Pinterest; San Francisco (CA); On-site; USD 250,000 - 300,000; Full time

Job summary

Pinterest is seeking a senior leader to direct the data infrastructure strategy and the technical direction for big data and AI applications. You will oversee scalable frameworks, compute engines, and scheduling for petabyte-scale datasets, while collaborating with internal customers on critical business use cases.

In this role, you’ll drive thought leadership on fast and reliable data processing and storage at scale, contribute to the long-term roadmap, and mentor cross-functional teams to

Qualifications

• 12+ years of industry experience with a proven track record of technical excellence.
• 8+ years building and supporting large scalable Kubernetes or big data platforms.
• Deep knowledge of big data / ML technologies (Flink, Spark, Presto, Kubernetes, Ray, PyTorch/TensorFlow).

Responsibilities

• Lead the strategy and technical direction of Pinterest’s data infrastructure for big data and AI applications.
• Build and scale data infrastructure frameworks for petabyte-scale datasets, including compute engines, job management, resource management, scheduling, and remote shuffling.
• Work with internal customers on critical business use cases relying on big data.
• Provide company-wide thought leadership on processing and storing data reliably, quickly, and efficiently at scale.
• Contribute to the team’s technical vision and long-term roadmap.

Skills

Big Data Technologies

Kubernetes Platform Development

Data Infrastructure Frameworks

Programming in Java, Go, Scala, or Py3

AWS Technologies

Education

Bachelor’s degree in Computer Science or related field

Tools

AWS

Job description

Pinterest is seeking a senior leader to direct the data infrastructure strategy and the technical direction for big data and AI applications. You will oversee scalable frameworks, compute engines, and scheduling for petabyte-scale datasets, while collaborating with internal customers on critical business use cases.

In this role, you’ll drive thought leadership on fast and reliable data processing and storage at scale, contribute to the long-term roadmap, and mentor cross-functional teams to