Remote AI Data Engineer: Large-Scale Pipelines

Bright Vision Technologies; South Windsor (CT); On-site; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies seeks an AI Data Engineer to design and operate large-scale data systems powering AI training and evaluation pipelines. The role combines data engineering excellence with a deep understanding of AI workloads to handle ingestion, transformation, quality assurance, lineage, and high-throughput data delivery across modalities.

The ideal candidate has experience with petabyte-scale data systems, strong software engineering, and awareness of how data infrastructure

Qualifications

• Bachelor’s or Master’s degree in Computer Science or a related field.
• 10+ years of data engineering experience with ML/AI workloads.
• Strong Python and at least one JVM or systems language.
• Experience with Spark, Ray, or Beam.
• Petabyte-scale storage and pipeline systems experience.
• Knowledge of distributed systems, data modeling, and storage formats.
• Experience with dataset versioning and reproducibility for ML workflows.
• High-throughput data loading for accelerator-based training.
• Solid software engineering practices, including testing and CI/CD.
• Excellent communication and cross-functional collaboration skills.

Responsibilities

• Design and operate large-scale data pipelines for AI training and evaluation.
• Build ingestion systems for text, image, audio, video, and structured data.
• Implement data cleaning, deduplication, and quality assurance at petabyte scale.
• Develop dataset versioning, lineage, and provenance tracking for reproducible training.
• Create high-throughput data loading to maximize GPU utilization.
• Design storage architectures balancing cost, throughput, and latency.
• Implement data privacy, redaction, and consent enforcement.
• Collaborate with ML researchers to align data systems with model needs.
• Drive observability of data quality, drift, and pipeline health.

Skills

Python

Spark

Ray

Beam

CI/CD

Data pipelines

Petabyte storage

Data modeling

Unit testing

Education

Bachelor's or Master's in CS

Tools

HDFS

Kafka

Airflow

Job description

Bright Vision Technologies seeks an AI Data Engineer to design and operate large-scale data systems powering AI training and evaluation pipelines. The role combines data engineering excellence with a deep understanding of AI workloads to handle ingestion, transformation, quality assurance, lineage, and high-throughput data delivery across modalities.

The ideal candidate has experience with petabyte-scale data systems, strong software engineering, and awareness of how data infrastructure