AI Data Engineer

VBeyond Corporation; Scottsdale (AZ); On-site; USD 120,000 - 180,000; Full time

Job summary

VBeyond Corporation is seeking an experienced AI Data Engineer to design and maintain data infrastructure for AI/ML initiatives. The role focuses on high-quality, governed data for traditional analytics and AI applications.

You will build pipelines for ingestion, cleaning, and transformation of structured and unstructured data, including documents, PDFs, transcripts, and logs. Collaboration with ML/DS teams is key.

Qualifications

• Strong hands-on experience with SQL and a programming language (Python or Scala).
• Experience with cloud-based data platforms.
• Hands-on experience with Airflow and dbt.
• Understanding of embedding models, vector search, and RAG architectures.
• Experience processing structured and unstructured data.
• Familiarity with ML lifecycle concepts, including model training and inference.

Responsibilities

• Design and build data pipelines to ingest, clean, and transform data from documents, PDFs, transcripts, and logs.
• Build and maintain vector embedding pipelines and integrate with vector databases.
• Develop feature stores and data pipelines for ML model training and inference.
• Implement data pipelines for RAG architectures with chunking, embedding refresh cycles, and metadata tagging.
• Collaborate with ML Engineers and Data Scientists to understand data requirements for model training and evaluation.
• Ensure data lineage, quality, and governance across AI pipelines.
• Optimize data pipelines for cost and performance, including Dynamic Tables and streaming ingestion.
• Support MLOps/LLMOps workflows, including dataset versioning and tracking for reproducibility.
• Implement data entitlement and access control mechanisms to restrict data access.

Skills

SQL

Python

Scala

Cloud platforms

Airflow

dbt

Embedding models

Vector search

RAG architectures

Data processing

ML lifecycle

Tools

Airflow

dbt

Snowflake Cortex

Cortex Search

LangChain

LlamaIndex

Job description

Locations:

Scottsdale, AZ | La Vista, NE (Omaha) | Atlanta, GA

Work Location:

Candidate must be based in one of the listed locations.

Job Description:

We are looking for an experienced AI Data Engineer to build and maintain data infrastructure supporting AI/ML initiatives, including data pipelines for model training, Retrieval-Augmented Generation (RAG), and production ML systems.

The role focuses on ensuring high-quality, governed, and accessible data for traditional analytics as well as AI-driven applications.

Key Responsibilities:

• Design and develop data pipelines to ingest, clean, and transform structured and unstructured data, including documents, PDFs, transcripts, and logs.


• Build and maintain vector embedding pipelines and integrate with vector databases.


• Develop feature stores and data pipelines for ML model training and inference.


• Implement data pipelines supporting RAG architectures, including chunking strategies, embedding refresh cycles, and metadata tagging.


• Collaborate with ML Engineers and Data Scientists to understand data requirements for model training and evaluation.


• Ensure data lineage, quality, and governance across pipelines supporting AI systems.


• Optimize data pipelines for cost and performance, including Dynamic Tables and streaming ingestion.


• Support MLOps/LLMOps workflows, including dataset versioning and tracking data used for model training to ensure reproducibility and compliance.


• Implement data entitlement and access control mechanisms to ensure AI systems access only authorized data.

Required Skills & Experience:

• Strong hands-on experience with SQL and at least one programming language: Python or Scala.


• Experience working with cloud-based data platforms.


• Hands-on experience with data orchestration tools such as Airflow and dbt.


• Understanding of embedding models, vector search, and RAG architectures.


• Experience processing structured and unstructured data.


• Familiarity with ML lifecycle concepts, including model training and inference.

Nice to Have:

• Experience with Snowflake Cortex, Cortex Search, or similar in-warehouse AI tools.


• Experience working in regulated industries, preferably financial services.


• Understanding of data entitlement, governance, and compliance requirements.


• Exposure to LLM application frameworks such as LangChain and LlamaIndex.