Senior Data Platform Engineer - Distributed ML & Lakehouse

Jobtailor; California (MO); On-site; USD 140,000 - 200,000; Full time

Job summary

Jobtailor in the United States seeks an experienced engineer to design and develop the next generation distributed ML data platform (Ingestion, Processing, Serving) using GCP and open-source frameworks.

You will collaborate with ML & Data Engineers, translate needs into requirements, build self-serve capabilities and drive adoption, deliver technical projects, and mentor junior engineers, shaping high-throughput, low-latency systems with cost efficiency.

Qualifications

• BA or BS in Computer Science, Electrical Engineering, Mathematics, Physics, or another relevant field; or equivalent real-world experience
• 3+ years in building petabyte-scale distributed data platforms, specifically evolving Data Lakes into Lakehouses using major cloud providers and open-source frameworks
• Expert-level proficiency in Java, C++, or Python, with a proven track record of designing and implementing robust, distributed systems
• Expertise in implementing Data Processing Frameworks (Beam, Spark) and serving layers optimized for high-throughput, low-latency delivery
• Experience optimizing services for cost efficiency, performance & reliability
• Experience with Micro services architecture and proven ability to manage the full operational lifecycle of systems

Responsibilities

• Design and develop the next generation distributed ML data platform (Ingestion, Processing, Serving) using GCP and open-source frameworks
• Collaborate with stakeholders (ML & Data Engineers), translate needs & pain points into requirements, build self-serve capabilities and drive adoption
• Deliver technical projects owning major technical decisions and tradeoffs & contribute to the team’s strategic roadmap
• Champion engineering & operational excellence by continuously improving systems and processes
• Actively participate in team’s planning, code reviews and design discussions
• Conduct technical interviews, onboard new and mentor junior engineers

Skills

Distributed Data Platform Development

Java, C++, Python proficiency

Data Processing Frameworks Beam Spark

Microservices Architecture

Technical Interviewing

Mentoring Junior Engineers

Education

Bachelor's degree in CS / EE / Math / Physics or equivalent

Tools

GCP

Spark

Beam

Open-Source Frameworks

Job description

Jobtailor in the United States seeks an experienced engineer to design and develop the next generation distributed ML data platform (Ingestion, Processing, Serving) using GCP and open-source frameworks.

You will collaborate with ML & Data Engineers, translate needs into requirements, build self-serve capabilities and drive adoption, deliver technical projects, and mentor junior engineers, shaping high-throughput, low-latency systems with cost efficiency.