Senior AI Data Pipeline Engineer - Remote

Bright Vision Technologies; Woodbury (MN); Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is hiring an AI Pipeline Engineer to design and operate large-scale data pipelines powering AI training across diverse modalities. The role focuses on ingestion, transformation, quality assurance, lineage, and high-throughput data delivery to training jobs in a remote US setting.

Candidates should have 6+ years in data engineering, strong Python and JVM skills, and experience with Spark, Ray, or Beam. This full-time position offers growth and competitive compensation.

Qualifications

• Bachelor’s or Master’s degree in Computer Science or related field.
• Six+ years of data engineering experience supporting ML/AI workloads.
• Strong Python and at least one JVM or systems language.
• Experience with Spark, Ray, or Beam.
• Hands-on work with petabyte-scale storage and pipelines.
• Knowledge of distributed systems and data modeling.
• Experience with dataset versioning and reproducibility for ML workflows.
• High-throughput data loading for accelerator-based training.
• Solid software engineering with testing, CI/CD, and code reviews.
• Excellent communication and cross-functional collaboration skills.

Responsibilities

• Design and operate large-scale data pipelines for AI training and evaluation.
• Build ingestion systems for text, image, audio, video, and structured data.
• Implement data cleaning, deduplication, filtering at petabyte scale.
• Develop dataset versioning, lineage, and provenance tracking.
• Create high-throughput data loading to maximize GPU utilization.
• Build labeling workflows and active learning pipelines.
• Design cost-efficient storage architectures for throughput and latency.
• Develop clean dataset construction pipelines with integrity controls.
• Ensure data privacy, redaction, and consent enforcement.
• Collaborate with ML researchers to align data systems with model needs.
• Monitor data quality, drift, and pipeline health across data estate.
• Optimize cost and performance via compression and caching.

Skills

Python

JVM languages

Distributed systems

CI/CD

Communication

Cross-functional collaboration

Education

Bachelor’s or Master’s degree in Computer Science

Tools

Spark

Ray

Beam

Job description

Bright Vision Technologies is hiring an AI Pipeline Engineer to design and operate large-scale data pipelines powering AI training across diverse modalities. The role focuses on ingestion, transformation, quality assurance, lineage, and high-throughput data delivery to training jobs in a remote US setting.

Candidates should have 6+ years in data engineering, strong Python and JVM skills, and experience with Spark, Ray, or Beam. This full-time position offers growth and competitive compensation.