AI Data Engineer: Vector Pipelines, RAG & MLOps

VBeyond Corporation; Scottsdale (AZ); On-site; USD 120,000 - 180,000; Full time

Job summary

VBeyond Corporation is seeking an experienced AI Data Engineer to design and maintain data infrastructure for AI/ML initiatives. The role focuses on high-quality, governed data for traditional analytics and AI applications.

You will build pipelines for ingestion, cleaning, and transformation of structured and unstructured data, including documents, PDFs, transcripts, and logs. Collaboration with ML/DS teams is key.

Qualifications

• Strong hands-on experience with SQL and a programming language (Python or Scala).
• Experience with cloud-based data platforms.
• Hands-on experience with Airflow and dbt.
• Understanding of embedding models, vector search, and RAG architectures.
• Experience processing structured and unstructured data.
• Familiarity with ML lifecycle concepts, including model training and inference.

Responsibilities

• Design and build data pipelines to ingest, clean, and transform data from documents, PDFs, transcripts, and logs.
• Build and maintain vector embedding pipelines and integrate with vector databases.
• Develop feature stores and data pipelines for ML model training and inference.
• Implement data pipelines for RAG architectures with chunking, embedding refresh cycles, and metadata tagging.
• Collaborate with ML Engineers and Data Scientists to understand data requirements for model training and evaluation.
• Ensure data lineage, quality, and governance across AI pipelines.
• Optimize data pipelines for cost and performance, including Dynamic Tables and streaming ingestion.
• Support MLOps/LLMOps workflows, including dataset versioning and tracking for reproducibility.
• Implement data entitlement and access control mechanisms to restrict data access.

Skills

SQL

Python

Scala

Cloud platforms

Airflow

dbt

Embedding models

Vector search

RAG architectures

Data processing

ML lifecycle

Tools

Airflow

dbt

Snowflake Cortex

Cortex Search

LangChain

LlamaIndex

Job description

VBeyond Corporation is seeking an experienced AI Data Engineer to design and maintain data infrastructure for AI/ML initiatives. The role focuses on high-quality, governed data for traditional analytics and AI applications.

You will build pipelines for ingestion, cleaning, and transformation of structured and unstructured data, including documents, PDFs, transcripts, and logs. Collaboration with ML/DS teams is key.