Senior Data Infra Engineer — Scale Global Open-Source Data

Applied Intuition; Sunnyvale (CA); On-site; USD 153,000 - 222,000; Full time

Job summary

Applied Intuition in Sunnyvale, CA seeks infrastructure engineers to scale open-source data infrastructure across cloud, hybrid, and on-prem deployments. You will handle data collection, ingestion, storage, and querying, collaborating with business units to design and build data products for the platform.

You will own end-to-end data pipelines, tune processing and indexing, and provide performant query interfaces with easy integration hooks, using modern tooling and open-source technologies.

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, or equivalent.
• 3+ years of professional experience.
• Experience with large-scale open source data technologies (Spark, Kafka, Hudi, Flyte).
• Experience with containerization and modern software development workflows.

Responsibilities

• Scale infrastructure to support cloud, hybrid, and on-prem deployments.
• Handle end-to-end data lifecycle from ingestion to retrieval across the data platform.
• Build features to tune data processing pipelines for fast ingestion and indexing.
• Enable product workflows with performant query interfaces and easy integration hooks.
• Develop and deploy high-quality software using modern OSS tooling.

Skills

Data infrastructures

Open source tech

Containerization

Cloud & on-prem

Data quality

Education

Bachelor's degree in Computer Science or Software Engineering or equivalent

Tools

Spark

Kafka

Hudi

Flyte

Kubernetes

Job description

Applied Intuition in Sunnyvale, CA seeks infrastructure engineers to scale open-source data infrastructure across cloud, hybrid, and on-prem deployments. You will handle data collection, ingestion, storage, and querying, collaborating with business units to design and build data products for the platform.

You will own end-to-end data pipelines, tune processing and indexing, and provide performant query interfaces with easy integration hooks, using modern tooling and open-source technologies.