Petabyte-Scale Data Ingest & Storage Architect

Eon Systems PBC; San Francisco (CA); On-site; USD 137,760 - 413,280; Full time

Job summary

A tech firm specializing in data collection is seeking an experienced individual to design and implement high-throughput data pipelines. The role requires expertise in storage solutions with a focus on reliability and performance. Candidates must have over 5 years of experience in deploying systems that can handle multi-petabyte data. This is a contract position based in San Francisco with competitive hourly compensation between $100-$300.

Qualifications

• 5+ years designing high-throughput storage or HPC pipelines.
• Experience with NVMe RAID, ZFS, and Linux performance tuning.
• Proven delivery of multi-GB/s systems in production.

Responsibilities

• Architect and implement ingest and storage solutions.
• Set up ingest paths with adequate burst headroom.
• Optimize footprint with on-the-fly compression and cost control.

Skills

Designing and deploying high-throughput storage pipelines

Hands-on with NVMe RAID, Linux performance tuning

Building multi-GB/s ingest systems

Experience with tiered storage systems

Practical S3/object-storage knowledge

Data integrity and reliability techniques

Networking familiarity with 25/40/100 GbE

Specifying and racking hardware

Job description

A tech firm specializing in data collection is seeking an experienced individual to design and implement high-throughput data pipelines. The role requires expertise in storage solutions with a focus on reliability and performance. Candidates must have over 5 years of experience in deploying systems that can handle multi-petabyte data. This is a contract position based in San Francisco with competitive hourly compensation between $100-$300.