Remote AI Data Engineer: Build Petabyte-Scale Pipelines

Bright Vision Technologies; Framingham (MA); On-site; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking an AI Data Engineer to build and operate large-scale data systems powering AI training and evaluation pipelines. The role emphasizes ingestion, transformation, quality assurance, lineage, and high-throughput data delivery across modalities.

The ideal candidate has 10+ years of data engineering experience, strong Python and JVM capabilities, and deep experience with Spark, Ray, or Beam, plus petabyte-scale storage expertise and solid CI/CD practices.

Qualifications

• BS or MS in Computer Science or related field.
• 10+ years of data engineering experience supporting ML/AI workloads.
• Strong Python and at least one JVM or systems language.
• Experience with modern data processing frameworks (Spark, Ray, Beam).
• Experience operating petabyte-scale storage and pipelines.
• Understanding of distributed systems, data modeling, and storage formats.
• Experience with dataset versioning, lineage, and reproducibility for ML workflows.
• Experience with high-throughput data loading for accelerator-based training.
• Strong software engineering practices (testing, CI/CD, code review).
• Excellent communication and cross-functional collaboration skills.

Responsibilities

• Design and operate large-scale data pipelines for AI training and evaluation.
• Build ingestion systems for text, image, audio, video, and structured data.
• Implement data cleaning, deduplication, and quality assurance at scale.
• Develop dataset versioning, lineage, and provenance tracking.
• Build high-throughput data loading to maximize GPU utilization.
• Implement labeling workflows and human-in-the-loop data improvement.
• Design storage architectures balancing cost, throughput, and latency.
• Create evaluation dataset pipelines with integrity and contamination controls.
• Ensure data privacy, redaction, and consent enforcement.
• Collaborate with ML researchers and engineers to align data systems with model needs.
• Drive observability of data quality, drift, and pipeline health.
• Document data systems, schemas, and procedures.

Skills

Python

Java

Spark

Ray

Beam

CI/CD

Communication

Education

Bachelor’s or Master’s in CS

Tools

Spark

Ray

Beam

CI/CD tooling

Job description

Bright Vision Technologies is seeking an AI Data Engineer to build and operate large-scale data systems powering AI training and evaluation pipelines. The role emphasizes ingestion, transformation, quality assurance, lineage, and high-throughput data delivery across modalities.

The ideal candidate has 10+ years of data engineering experience, strong Python and JVM capabilities, and deep experience with Spark, Ray, or Beam, plus petabyte-scale storage expertise and solid CI/CD practices.