AI Pipeline Engineer

Bright Vision Technologies; Woodbury (MN); Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is hiring an AI Pipeline Engineer to design and operate large-scale data pipelines powering AI training across diverse modalities. The role focuses on ingestion, transformation, quality assurance, lineage, and high-throughput data delivery to training jobs in a remote US setting.

Candidates should have 6+ years in data engineering, strong Python and JVM skills, and experience with Spark, Ray, or Beam. This full-time position offers growth and competitive compensation.

Qualifications

• Bachelor’s or Master’s degree in Computer Science or related field.
• Six+ years of data engineering experience supporting ML/AI workloads.
• Strong Python and at least one JVM or systems language.
• Experience with Spark, Ray, or Beam.
• Hands-on work with petabyte-scale storage and pipelines.
• Knowledge of distributed systems and data modeling.
• Experience with dataset versioning and reproducibility for ML workflows.
• High-throughput data loading for accelerator-based training.
• Solid software engineering with testing, CI/CD, and code reviews.
• Excellent communication and cross-functional collaboration skills.

Responsibilities

• Design and operate large-scale data pipelines for AI training and evaluation.
• Build ingestion systems for text, image, audio, video, and structured data.
• Implement data cleaning, deduplication, filtering at petabyte scale.
• Develop dataset versioning, lineage, and provenance tracking.
• Create high-throughput data loading to maximize GPU utilization.
• Build labeling workflows and active learning pipelines.
• Design cost-efficient storage architectures for throughput and latency.
• Develop clean dataset construction pipelines with integrity controls.
• Ensure data privacy, redaction, and consent enforcement.
• Collaborate with ML researchers to align data systems with model needs.
• Monitor data quality, drift, and pipeline health across data estate.
• Optimize cost and performance via compression and caching.

Skills

Python

JVM languages

Distributed systems

CI/CD

Communication

Cross-functional collaboration

Education

Bachelor’s or Master’s degree in Computer Science

Tools

Spark

Ray

Beam

Job description

AI Pipeline Engineer – Remote

Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States.

This is a fantastic opportunity to join an established and well-respected organization offering tremendous career growth potential.

Job Title: AI Pipeline Engineer

Location: 100% Remote (U.S.)

Position Type: Full-time, Direct W2

Salary Range: $100,000–$150,000 Annually

Experience Required: 6+ years

Sponsorship: U.S. Citizens, Green CardHolders, EADHolders, and H-1B transfer candidates are encouraged to apply. We are unable to sponsor new H-1B visa petitions for this position.

Job Summary

We are seeking an AI Pipeline Engineer to build and operate the large-scale data systems that power modern AI training and evaluation pipelines. The role combines deep data engineering expertise with a strong understanding of AI workloads, focusing on ingestion, transformation, quality assurance, lineage, and high-throughput delivery of data to training jobs across diverse modalities. The ideal candidate has experience operating petabyte-scale data systems, strong software engineering fundamentals, and clear understanding of how data infrastructure choices propagate into model quality and training efficiency.

Key Responsibilities

• Design and operate large-scale data pipelines supporting AI training, evaluation, and continual improvement workflows.


• Build ingestion systems for diverse modalities including text, image, audio, video, and structured signals.


• Implement data cleaning, deduplication, filtering, and quality assurance at petabyte scale.


• Develop dataset versioning, lineage, and provenance tracking systems suitable for reproducible training.


• Build high-throughput data loading systems that maximize GPU utilization during training.


• Implement labeling workflows, active learning pipelines, and human-in-the-loop data improvement systems.


• Design storage architectures balancing cost, throughput, and latency across data tiers.


• Build evaluation dataset construction pipelines with strict integrity and contamination controls.


• Implement data privacy, redaction, and consent enforcement throughout the pipeline.


• Collaborate with ML researchers and engineers to align data systems with model development needs.


• Drive observability of data quality, drift, and pipeline health across the AI data estate.


• Optimize cost and performance through compression, format selection, and caching strategies.


• Document data systems, schemas, and operational procedures for broad internal use.


• Stay current with AI data infrastructure research and emerging open-source tools.

Required Qualifications

• Bachelor’s or Master’s degree in Computer Science or a related field.


• Six or more years of data engineering experience, with significant work supporting ML or AI workloads.


• Strong proficiency in Python and at least one JVM or systems language.


• Deep experience with modern data processing frameworks such as Spark, Ray, or Beam.


• Hands-on experience operating petabyte-scale storage and pipeline systems.


• Strong understanding of distributed systems, data modeling, and storage formats.


• Experience with dataset versioning, lineage, and reproducibility for ML workflows.


• Familiarity with high-throughput data loading for accelerator-based training.


• Strong software engineering practices including testing, CI/CD, and code review.


• Excellent communication and cross-functional collaboration skills.

Preferred Qualifications

• Experience with multimodal datasets at large scale.


• Familiarity with data quality tooling and dataset evaluation methodology.


• Exposure to privacy-preserving data systems and regulated data handling.


• Open-source contributions to data infrastructure projects.


• Experience supporting frontier model training pipelines.

Bright Vision Technologies is an Equal Opportunity Employer.

Equal Employment Opportunity (EEO) Statement

Bright Vision Technologies (BV Teck) is committed to equal employment opportunity (EEO) for all employees and applicants without regard to race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, veteran status, or any other protected status as defined by applicable federal, state, or local laws. This commitment extends to all aspects of employment, including recruitment, hiring, training, compensation, promotion, transfer, leaves of absence, termination, layoffs, and recall.

BV Teck expressly prohibits any form of workplace harassment or discrimination. Any improper interference with employees' ability to perform their job duties may result in disciplinary action up to and including termination of employment.