AI Data Engineer: Scalable Pipelines & LLM Ops

Valiant Technologies LLC; Georgia; On-site; USD 120,000 - 180,000; Full time

Job summary

Valiant Technologies LLC seeks an experienced AI Data Engineer to build scalable data pipelines and infrastructure for AI/ML, RAG, and LLM applications. The role requires strong SQL, Python/Scala, and cloud data platforms with experience processing structured and unstructured data.

Key responsibilities include designing ingestion pipelines, building embedding pipelines, integrating vector databases, and enabling RAG, chunking, and vector search while supporting ML lifecycle workflows and

Qualifications

• Strong SQL and Python/Scala.
• Apache Airflow and dbt.
• RAG, vector databases, embeddings, and vector search.
• Unstructured data processing.
• ML lifecycle knowledge.

Responsibilities

• Design and develop data ingestion, transformation, and processing pipelines.
• Build embedding pipelines and integrate vector databases.
• Develop data pipelines supporting RAG, chunking, and vector search.
• Process structured and unstructured data (PDFs, documents, logs, transcripts).
• Implement data quality, governance, lineage, and access controls.
• Support ML model training, inference, MLOps, and LLMOps workflows.
• Optimize data pipelines for performance, scalability, and cost.

Skills

SQL

Python/Scala

RAG & vector databases

Unstructured data processing

ML lifecycle knowledge

Tools

Apache Airflow

dbt

Job description

Valiant Technologies LLC seeks an experienced AI Data Engineer to build scalable data pipelines and infrastructure for AI/ML, RAG, and LLM applications. The role requires strong SQL, Python/Scala, and cloud data platforms with experience processing structured and unstructured data.

Key responsibilities include designing ingestion pipelines, building embedding pipelines, integrating vector databases, and enabling RAG, chunking, and vector search while supporting ML lifecycle workflows and