Remote AI Data Pipeline Engineer (Petabyte-Scale)

Bright Vision Technologies; Centennial (CO); Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking an AI Pipeline Engineer to design and operate large-scale data systems powering AI training and evaluation pipelines. This role focuses on ingestion, transformation, quality assurance, lineage, and high-throughput data delivery across modalities.

The ideal candidate has 6+ years of data engineering experience, strong Python and JVM skills, and hands-on work with Spark, Ray, or Beam, plus experience with petabyte-scale storage and reproducibility for ML

Qualifications

• Bachelor’s or Master’s degree in Computer Science or related field.
• Six or more years of data engineering experience supporting ML/AI workloads.
• Strong Python and at least one JVM or systems language.
• Experience with modern data processing frameworks (Spark, Ray, Beam).
• Hands-on experience with petabyte-scale storage and pipelines.
• Understanding of distributed systems, data modeling, and storage formats.
• Experience with dataset versioning, lineage, reproducibility for ML workflows.
• Familiarity with high-throughput data loading for accelerator-based training.
• Strong software engineering practices (testing, CI/CD, code reviews).
• Excellent communication and cross-functional collaboration skills.

Responsibilities

• Design and operate large-scale data pipelines supporting AI training and evaluation workflows.
• Build ingestion systems for text, image, audio, video, and structured signals.
• Implement data cleaning, deduplication, filtering, and quality assurance at petabyte scale.
• Develop dataset versioning, lineage, and provenance tracking for reproducible training.
• Build high-throughput data loading systems to maximize GPU utilization.
• Implement labeling workflows and active learning pipelines.
• Design storage architectures balancing cost, throughput, and latency.
• Create pipelines for evaluation dataset construction with integrity controls.
• Enforce data privacy, redaction, and consent throughout pipelines.
• Collaborate with ML researchers and engineers to align data systems with model needs.
• Drive observability of data quality, drift, and pipeline health.
• Optimize cost and performance via compression, formats, and caching.
• Document data systems and procedures for broad internal use.
• Stay current with AI data infrastructure research and open-source tools.

Skills

Python

Java/Scala

Spark

Ray

Beam

Petabyte storage

Data pipelines

CI/CD

Data modeling

Data lineage

Reproducibility

Data privacy

Education

Bachelor's in Computer Science

Master's in Computer Science

Tools

Spark

Ray

Beam

Job description

Bright Vision Technologies is seeking an AI Pipeline Engineer to design and operate large-scale data systems powering AI training and evaluation pipelines. This role focuses on ingestion, transformation, quality assurance, lineage, and high-throughput data delivery across modalities.

The ideal candidate has 6+ years of data engineering experience, strong Python and JVM skills, and hands-on work with Spark, Ray, or Beam, plus experience with petabyte-scale storage and reproducibility for ML