Staff Engineer, Spark Structured Streaming Architect

United States Digital Space LLC

Seattle (WA)

On-site

USD 182,400 - 247,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

United States Digital Space LLC is seeking a Staff Engineer to steer Spark Structured Streaming's technical direction across OSS and our Data Intelligence Platform. You will architect enhancements for state management, new operators, and latency/throughput gains, while reimagining the engine to reduce cost.

You will mentor engineers, set long-term architecture, and partner with product teams to align strategy with customer needs.

Qualifications

BS or higher in Computer Science or related field, or equivalent practical experience.
8+ years building big-data, Spark, or database internals.
Interest in database/storage/distributed systems, language design, or performance optimization.
Able to work toward a multi-year vision with incremental deliverables.
Proven track record delivering features with high operating quality and reliability.
Comfortable working cross-functionally with product mgmt and customers.

Responsibilities

Set and drive the technical vision for Spark Structured Streaming across OSS and the company Data Intelligence Platform.
Design and build core engine capabilities — state management, new operators, and architectural improvements to latency and throughput.
Raise the bar for engineering quality and operability, building software that is easy to run in production.
Make company-wide impact by driving stream processing adoption across the product portfolio.
Guide long-term architecture and technical debt decisions, balancing them against the product roadmap.
Mentor and technically lead engineers on the team, and partner with the Engineering Manager to attract and grow top-tier talent.