Remote Infra Reliability Engineer – AI Data Pipelines

Bright Vision Technologies; Wildwood (MO); Remote; USD 125,000 - 170,000; Full time

Job summary

Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States. This role focuses on building and operating large-scale data systems powering AI training and evaluation pipelines, with emphasis on ingestion, transformation, quality assurance, lineage, and high-throughput delivery across modalities.

The candidate should have 6+ years in data engineering for ML/AI workloads, strong Python, and

Qualifications

• Bachelor's or Master's degree in Computer Science or related field.
• 6+ years of data engineering experience, with ML/AI workloads.
• Strong Python proficiency and at least one JVM or systems language.
• Deep experience with data processing frameworks such as Spark, Ray, or Beam.
• Hands-on experience with petabyte-scale storage and pipelines.
• Understanding of distributed systems, data modeling, and storage formats.
• Experience with dataset versioning, lineage, and reproducibility for ML workflows.
• Experience with high-throughput data loading for accelerator-based training.
• Strong software engineering practices including testing, CI/CD, and code review.
• Excellent communication and cross-functional collaboration skills.

Responsibilities

• Design and operate large-scale data pipelines for AI training and evaluation.
• Build ingestion systems for multiple modalities including text, image, audio, video, and structured signals.
• Implement data cleaning, deduplication, filtering, and QA at petabyte scale.
• Develop dataset versioning, lineage, and provenance tracking for reproducible training.
• Build high-throughput data loading systems to maximize GPU utilization.
• Implement labeling workflows and active learning pipelines.
• Design storage architectures balancing cost, throughput, and latency.
• Create pipelines for evaluation dataset construction with integrity controls.
• Ensure data privacy, redaction, and consent enforcement across pipelines.
• Collaborate with ML researchers and engineers to align data systems with model needs.
• Drive observability of data quality, drift, and pipeline health.
• Optimize cost and performance via compression, format selection, and caching.
• Document data systems and procedures for internal use.
• Stay current with AI data infrastructure research and open-source tools.

Skills

Python

JVM

Systems programming

Spark

Ray

Beam

Data engineering

CI/CD

Communication

Education

Bachelor's or Master's degree in Computer Science

Tools

Spark

Ray

Beam

CI/CD tooling

Job description

Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States. This role focuses on building and operating large-scale data systems powering AI training and evaluation pipelines, with emphasis on ingestion, transformation, quality assurance, lineage, and high-throughput delivery across modalities.

The candidate should have 6+ years in data engineering for ML/AI workloads, strong Python, and