AI Data Engineer: Scalable Pipelines & LLM Ops

MACHINE LEARNING TECHNOLOGIES LLC; Scottsdale (AZ); On-site; USD 91,000 - 129,000; Full time

Job summary

MACHINE LEARNING TECHNOLOGIES LLC is seeking an experienced AI Data Engineer to build and maintain scalable data pipelines and infrastructure that support AI/ML, RAG, and LLM applications. The role requires strong SQL and Python/Scala, plus expertise with cloud data platforms and processing of both structured and unstructured data.

Key responsibilities include designing ingestion pipelines, building embedding pipelines, enabling vector databases, and ensuring data quality, governance, and access

Qualifications

• Strong SQL and Python/Scala experience.
• Cloud data platforms expertise.
• Experience with Apache Airflow and dbt.
• Knowledge of RAG, vector databases, embeddings and vector search.
• Processing structured and unstructured data.
• ML lifecycle and MLOps familiarity.

Responsibilities

• Design and develop data ingestion, transformation, and processing pipelines.
• Build embedding pipelines and integrate vector databases.
• Develop data pipelines supporting RAG, chunking, and vector search.
• Process PDFs, documents, logs, and transcripts.
• Implement data quality, governance, lineage, and access controls.
• Support ML model training, inference, MLOps, and LLMOps workflows.
• Optimize data pipelines for performance, scalability, and cost.

Skills

SQL

Python/Scala

Cloud Data Platforms

AI/ML Data Engineering

RAG/Vector Search

Unstructured Data Processing

ML Lifecycle

ETL/Data Pipelines

Tools

Apache Airflow

dbt

Snowflake Cortex

LangChain/LlamaIndex

Job description

MACHINE LEARNING TECHNOLOGIES LLC is seeking an experienced AI Data Engineer to build and maintain scalable data pipelines and infrastructure that support AI/ML, RAG, and LLM applications. The role requires strong SQL and Python/Scala, plus expertise with cloud data platforms and processing of both structured and unstructured data.

Key responsibilities include designing ingestion pipelines, building embedding pipelines, enabling vector databases, and ensuring data quality, governance, and access