Remote Infrastructure Reliability Engineer AI Data Pipelines

Bright Vision Technologies; Round Rock (TX); On-site; USD 125,000 - 170,000; Full time

Job summary

Bright Vision Technologies is seeking an Infrastructure Reliability Engineer to design and operate large-scale data pipelines powering AI training and evaluation. You will handle ingestion, cleaning, lineage, and high-throughput data delivery across multimodal datasets, ensuring reproducibility and model quality.

The role requires 6+ years in data engineering for ML workloads, strong Python and JVM skills, and experience with Spark/Ray/Beam.

Qualifications

• Bachelor’s or Master’s degree in CS or related field.
• 6+ years of data engineering experience, esp. ML/AI workloads.
• Proficient in Python and at least one JVM language.
• Strong experience with Spark, Ray, or Beam.
• Experience operating petabyte-scale storage and pipelines.
• Knowledge of distributed systems, data modeling, and storage formats.
• Experience with dataset versioning, lineage, reproducibility for ML workflows.
• Familiar with high-throughput data loading for accelerator training.
• Strong software engineering practices with testing and CI/CD.
• Excellent communication and collaboration.

Responsibilities

• Design and operate large-scale data pipelines powering AI training and evaluation.
• Build ingestion systems for multimodal data (text, image, audio, video, structured signals).
• Implement data cleaning, deduplication, filtering, and QA at petabyte scale.
• Develop dataset versioning, lineage, and provenance tracking for reproducible training.
• Build high-throughput data loading to maximize GPU utilization during training.
• Implement labeling workflows, active learning, and human-in-the-loop improvements.
• Design storage architectures balancing cost, throughput, and latency.
• Create evaluation data pipelines with strong integrity and controls.
• Enforce data privacy, redaction, and consent throughout the pipeline.
• Collaborate with ML researchers to align data systems with model needs.
• Monitor data quality, drift, and pipeline health across the estate.
• Optimize cost and performance via compression, formats, caching.
• Document data systems and procedures for internal use.
• Stay current with AI data infrastructure research and OSS tooling.

Skills

Python

JVM language

Distributed systems

CI/CD

Communication

Education

Bachelor's or Master's in Computer Science

Tools

Spark

Ray

Beam

Job description

Bright Vision Technologies is seeking an Infrastructure Reliability Engineer to design and operate large-scale data pipelines powering AI training and evaluation. You will handle ingestion, cleaning, lineage, and high-throughput data delivery across multimodal datasets, ensuring reproducibility and model quality.

The role requires 6+ years in data engineering for ML workloads, strong Python and JVM skills, and experience with Spark/Ray/Beam.