AI Data Engineer — Vector Data Pipelines & MLOps Expert

Valiant Technologies LLC; Atlanta (GA); On-site; USD 110,000 - 170,000; Full time

Job summary

Valiant Technologies LLC seeks an experienced AI Data Engineer to design, develop, and maintain scalable data pipelines supporting AI/ML initiatives. Strong SQL and Python/Scala skills, cloud data platforms, and expertise with vector databases and embeddings are required.

The role involves building embedding pipelines, feature stores for ML, RAG pipelines, and collaboration with ML engineers to ensure data quality, governance, and scalable performance.

Qualifications

• Strong SQL and proficiency in Python or Scala.
• Experience with cloud data platforms.
• Hands-on experience with Apache Airflow, dbt, or similar orchestration tools.
• Understanding of embedding models, vector search, and RAG architectures.
• Experience processing structured and unstructured data.
• Knowledge of ML lifecycle concepts and data engineering best practices.

Responsibilities

• Design and develop data pipelines to ingest, clean, and transform data.
• Build and maintain vector embedding pipelines and integrate vector databases.
• Develop data pipelines and feature stores for ML training and inference.
• Implement RAG pipelines, including document chunking, embedding refresh, and metadata tagging.
• Collaborate with ML Engineers and Data Scientists to support model training and evaluation.
• Ensure data quality, lineage, governance, and security across AI data pipelines.
• Optimize data pipelines for performance, scalability, and cost efficiency.
• Support MLOps/LLMOps workflows, including dataset versioning and reproducibility.
• Implement data access controls and entitlement logic to ensure secure AI data usage.

Skills

SQL

Python/Scala

Cloud platforms

Airflow

dbt

Embedding models

Vector search

RAG architectures

Structured data

Unstructured data

ML lifecycle

Tools

Apache Airflow

dbt

Job description

Valiant Technologies LLC seeks an experienced AI Data Engineer to design, develop, and maintain scalable data pipelines supporting AI/ML initiatives. Strong SQL and Python/Scala skills, cloud data platforms, and expertise with vector databases and embeddings are required.

The role involves building embedding pipelines, feature stores for ML, RAG pipelines, and collaboration with ML engineers to ensure data quality, governance, and scalable performance.