Cloud Data Platform Engineer — AI & Real-Time Analytics

FHLB Des Moines; San Mateo (CA); Hybrid; USD 184,000 - 312,000; Full time

Benefits offered by this job

Hybrid in San Mateo

Health and wellness benefits

Paid time off

Market-competitive pay and incentive

Job summary

Guidewire is seeking a Staff Software Engineer for the Data Platform to architect and build cloud-native analytics services. You’ll lead design for real-time, large-scale systems using Kafka, Hadoop, Spark, Flink, and Hive, and mentor multiple teams in a fast-paced, AI-focused environment.

The role emphasizes collaboration across engineering, cloud, and data platform adoption, with a hybrid work option in San Mateo and a focus on secure, scalable solutions.

Qualifications

• 8+ years’ experience in software engineering focused on data platforms or data engineering.
• Deep expertise in Java and designing large-scale distributed systems.
• Experience with Kubernetes, AWS, Docker, and infrastructure-as-code tools.
• Design and development of Machine Learning Platform.
• Strong understanding of distributed systems principles (consistency, availability, durability, reliability, fault-tolerance, consensus algorithms).
• Advanced knowledge of algorithms, data structures, and performance optimization.

Responsibilities

• Architect, design, and implement high-performance, cloud-native data platform and analytics SaaS services that process billions of real-time data points.
• Lead the development of large-scale, distributed systems using modern big data technologies (Kafka, Hadoop, Spark, Flink, Hive) and Java/Scala.
• Drive the design and build of real-time stream processing systems and microservices, ensuring scalability, reliability, and data security.
• Provide technical leadership and mentorship to multiple teams, fostering a collaborative and innovative engineering culture.
• Own and resolve complex code defects, and champion best practices in software development and system design.

Skills

Java

Distributed systems

Cloud-native

Big data

Kubernetes

AWS

Docker

APIs/microservices

Mentoring

Communication

Education

BS/MS in CS or related

Tools

Kafka

Hadoop

Spark

Flink

Hive

Terraform

CloudFormation

TeamCity

Job description

Guidewire is seeking a Staff Software Engineer for the Data Platform to architect and build cloud-native analytics services. You’ll lead design for real-time, large-scale systems using Kafka, Hadoop, Spark, Flink, and Hive, and mentor multiple teams in a fast-paced, AI-focused environment.

The role emphasizes collaboration across engineering, cloud, and data platform adoption, with a hybrid work option in San Mateo and a focus on secure, scalable solutions.