Remote Infra Reliability Engineer – AI Data Pipelines

Bright Vision Technologies; Chesterfield (MO); Remote; USD 125,000 - 170,000; Full time

Job summary

Bright Vision Technologies is seeking an Infrastructure Reliability Engineer to design and operate large-scale data systems powering AI training and evaluation pipelines. You will own ingestion, transformation, quality assurance, lineage, and high-throughput data delivery across modalities.

The role requires 6+ years of data engineering experience, strong Python and JVM language skills, and hands-on work with Spark, Ray, or Beam in petabyte-scale environments. Remote U.S.

Qualifications

• Bachelor's or Master's degree in Computer Science or related field.
• 6+ years of data engineering experience supporting ML/AI workloads.
• Strong Python and at least one JVM or systems language.
• Experience with Spark, Ray, Beam or equivalent data processing frameworks.
• Hands-on with petabyte-scale storage and pipelines.
• Knowledge of distributed systems, data modeling, and storage formats.
• Experience with dataset versioning, lineage, and reproducibility.

Responsibilities

• Design and operate large-scale data pipelines for AI training and evaluation.
• Build ingestion systems for text, image, audio, video, and structured data.
• Implement data cleaning, deduplication, filtering, and QA at petabyte scale.
• Develop dataset versioning, lineage, and provenance tracking for reproducible training.
• Create high-throughput data loading to maximize GPU utilization.
• Build labeling workflows and active learning pipelines with human-in-the-loop tooling.
• Design cost-efficient, scalable storage architectures across data tiers.
• Ensure data privacy, redaction, and consent enforcement throughout pipelines.
• Collaborate with ML researchers to align data systems with model needs.
• Monitor data quality, drift, and pipeline health across the data estate.
• Optimize performance via compression, formats, and caching strategies.
• Document data systems and workflows for internal use.
• Stay current with AI data infrastructure research and open-source tools.

Skills

Python

Java

Spark

Ray

Beam

CI/CD

Distributed systems

Data pipelines

ML workloads

Education

Bachelor's or Master's in CS

Tools

Spark

Ray

Beam

Job description

Bright Vision Technologies is seeking an Infrastructure Reliability Engineer to design and operate large-scale data systems powering AI training and evaluation pipelines. You will own ingestion, transformation, quality assurance, lineage, and high-throughput data delivery across modalities.

The role requires 6+ years of data engineering experience, strong Python and JVM language skills, and hands-on work with Spark, Ray, or Beam in petabyte-scale environments. Remote U.S.