AI Data Engineer: Scalable Pipelines & Vector AI

Machine Learning Technologies, LLC; Atlanta (GA); On-site; USD 120,000 - 180,000; Full time

Job summary

Machine Learning Technologies, LLC in Atlanta seeks an experienced AI Data Engineer to design, develop, and maintain scalable data pipelines supporting AI initiatives.

You will leverage SQL and Python/Scala, cloud data platforms, and orchestration tools like Airflow and dbt, with embedding models, vector databases, and RAG architectures to enable model training, evaluation, and MLOps.

Qualifications

• Experience designing data pipelines for structured and unstructured data.
• Strong SQL and Python/Scala proficiency.
• Cloud data platform experience required.
• Experience with Airflow or dbt.
• Familiarity with embeddings, vector databases and RAG architectures.
• Knowledge of ML lifecycle data engineering best practices.

Responsibilities

• Design and develop scalable data pipelines for AI/ML workloads.
• Build vector embedding pipelines and integrate vector databases.
• Develop data pipelines and feature stores for model training and inference.
• Implement RAG pipelines with document chunking and embedding refresh.
• Collaborate with ML engineers and data scientists to support training and evaluation.
• Ensure data quality, lineage, governance, and security across pipelines.
• Optimize pipelines for performance, scalability, and cost.
• Support MLOps/LLMOps workflows including dataset versioning and reproducibility.
• Implement access controls and entitlement logic for secure data usage.

Skills

SQL

Python

Scala

Cloud data platforms

Airflow

dbt

Embeddings

Vector databases

RAG architectures

Tools

Apache Airflow

dbt

Job description

Machine Learning Technologies, LLC in Atlanta seeks an experienced AI Data Engineer to design, develop, and maintain scalable data pipelines supporting AI initiatives.

You will leverage SQL and Python/Scala, cloud data platforms, and orchestration tools like Airflow and dbt, with embedding models, vector databases, and RAG architectures to enable model training, evaluation, and MLOps.