Remote AI Data Pipeline Engineer (Petabyte-Scale Systems)

Bright Vision Technologies; Kirkland (WA); Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking an AI Pipeline Engineer to design and operate large-scale data pipelines powering AI training and evaluation. You will build ingestion, transformation, and QA for multimodal data, ensuring lineage and reproducibility across petabyte-scale systems.

The role emphasizes high-throughput data loading and privacy controls. The ideal candidate has 6+ years of data engineering experience, strong Python and JVM language skills, and experience with Spark, Ray, or

Qualifications

• Six+ years of data engineering experience, with ML/AI workload support.

Responsibilities

• Design and operate large-scale data pipelines for AI training, evaluation, and improvement workflows.
• Build ingestion systems for text, image, audio, video, and structured data.
• Implement data cleaning, deduplication, filtering, and QA at petabyte scale.
• Develop dataset versioning, lineage, and provenance tracking for reproducible training.
• Create high-throughput data loading to maximize GPU utilization.
• Implement labeling workflows, active learning pipelines, and human-in-the-loop data improvement.
• Design storage architectures balancing cost, throughput, and latency.
• Build evaluation dataset pipelines with data integrity and contamination controls.
• Ensure data privacy, redaction, and consent enforcement throughout pipelines.
• Collaborate with ML researchers to align data systems with model needs.
• Monitor data quality, drift, and pipeline health across the data estate.
• Optimize cost and performance via compression, formats, and caching.
• Document data systems and procedures for internal use.
• Stay current with AI data infrastructure research and open-source tools.

Skills

Python

Java

Spark

Ray

Beam

Petabyte-scale storage

CI/CD

Data modeling

Education

Bachelor’s or Master’s in Computer Science

Tools

Spark

Ray

Beam

Job description

Bright Vision Technologies is seeking an AI Pipeline Engineer to design and operate large-scale data pipelines powering AI training and evaluation. You will build ingestion, transformation, and QA for multimodal data, ensuring lineage and reproducibility across petabyte-scale systems.

The role emphasizes high-throughput data loading and privacy controls. The ideal candidate has 6+ years of data engineering experience, strong Python and JVM language skills, and experience with Spark, Ray, or