Member of Technical Staff, Software Engineer (Data Infrastructure)

Orbifold AI, Inc.; Palo Alto (CA); On-site; USD 140,000 - 200,000; Full time

Job summary

engineeringjobs.net, Inc. seeks a data infrastructure lead to design and own the platform for multimodal capture, covering ingest, storage, indexing, and delivery.

You will ensure durability, queryability, and performance across massive datasets for large-scale distributed training runs. The role requires 3+ years of production-grade data infra experience, strong Python and systems programming skills, and deep expertise in storage layout and distributed processing frameworks.

Qualifications

• At least 3 years of experience building production-grade data infrastructure with strong proficiency in Python and systems programming.
• Deep expertise in storage layout, distributed processing frameworks, and data reproducibility at scale.

Responsibilities

• Design and own the data infrastructure platform for multimodal capture, including ingest, storage, indexing, and delivery.
• Ensure massive datasets are durable, queryable, and performant enough to support large-scale distributed training runs.

Skills

Python

Systems programming

Data infrastructure

Storage architecture

Distributed processing

Ray

Spark

Data indexing

Data versioning

Video processing

Sensor data

Schema evolution

Performance optimization

Data lineage

Distributed training

Job description

You will design and own the data infrastructure platform for multimodal capture, including ingest, storage, indexing, and delivery. Your work ensures that massive datasets are durable, queryable, and performant enough to support large-scale distributed training runs.

Requirements: Candidates must have at least 3 years of experience building production-grade data infrastructure with strong proficiency in Python and systems programming. You should possess deep expertise in storage layout, distributed processing frameworks, and data reproducibility at scale.

Key Skills: Data infrastructure, Python, Systems programming, Storage architecture, Distributed processing, Ray, Spark, Data indexing, Data versioning, Video processing, Sensor data, Schema evolution, Performance optimization, Data lineage, Distributed training