Remote Infrastructure Reliability Engineer – AI Data Pipelines

Bright Vision Technologies; United States; Remote; USD 125,000 - 170,000; Full time

Benefits offered by this job

Remote work (US)

Job summary

Bright Vision Technologies seeks an Infrastructure Reliability Engineer to design and operate petabyte-scale data systems powering AI training and evaluation.

You will build ingestion, cleaning, and provenance tracking pipelines, optimize GPU data loading, and ensure data privacy and quality across modalities. Strong Python and JVM skills, distributed systems expertise, and 6+ years in ML/AI data pipelines are required.

Qualifications

• Bachelor’s or Master’s degree in Computer Science or a related field.
• 6+ years of data engineering experience with ML/AI workloads.
• Proficiency in Python and at least one JVM language.
• Experience with Spark, Ray, or Beam.
• Experience operating petabyte-scale storage and pipelines.
• Strong understanding of distributed systems and data modeling.
• Experience with dataset versioning and reproducibility for ML workflows.
• Experience with high-throughput data loading for accelerator-based training.
• Strong software engineering practices including testing, CI/CD, and code review.
• Excellent communication and cross-functional collaboration.

Responsibilities

• Design and operate large-scale data pipelines supporting AI training, evaluation, and continual improvement workflows.
• Build ingestion systems for diverse modalities including text, image, audio, video, and structured signals.
• Implement data cleaning, deduplication, filtering, and quality assurance at petabyte scale.
• Develop dataset versioning, lineage, and provenance tracking systems for reproducible training.
• Build high-throughput data loading systems that maximize GPU utilization during training.
• Implement labeling workflows, active learning pipelines, and human-in-the-loop data improvement systems.
• Design storage architectures balancing cost, throughput, and latency across data tiers.
• Build evaluation dataset construction pipelines with strict integrity and contamination controls.
• Implement data privacy, redaction, and consent enforcement throughout the pipeline.
• Collaborate with ML researchers and engineers to align data systems with model development needs.
• Drive observability of data quality, drift, and pipeline health across the AI data estate.
• Optimize cost and performance through compression, format selection, and caching strategies.
• Document data systems, schemas, and operational procedures for broad internal use.
• Stay current with AI data infrastructure research and emerging open-source tools.

Skills

Python

Java

Spark

Ray

Beam

Distributed systems

Petabyte-scale storage

Dataset versioning

CI/CD

Data modeling

Communication

Education

Bachelor's or Master's degree in CS or related field

Tools

CI/CD tooling

Job description

Bright Vision Technologies seeks an Infrastructure Reliability Engineer to design and operate petabyte-scale data systems powering AI training and evaluation.

You will build ingestion, cleaning, and provenance tracking pipelines, optimize GPU data loading, and ensure data privacy and quality across modalities. Strong Python and JVM skills, distributed systems expertise, and 6+ years in ML/AI data pipelines are required.