AI Data Engineer: LLM Pipelines, VectorDB & MLOps

Cloud Analytics Technologies, LLC; Scottsdale (AZ); On-site; USD 120,000 - 170,000; Full time

Job summary

Cloud Analytics Technologies, LLC in Scottsdale, AZ is seeking an experienced AI Data Engineer to design and maintain scalable data pipelines and infrastructure that support AI/ML, RAG, and LLM applications. You will work with SQL, Python/Scala, cloud data platforms, and both structured and unstructured data.

Responsibilities include building ingestion and embedding pipelines, enabling vector search, ensuring data quality and governance, and supporting MLOps workflows to optimize performance and

Qualifications

• Strong SQL and Python/Scala skills.
• Experience with Apache Airflow and dbt.
• Expertise in RAG, vector databases, embeddings and vector search.
• Experience handling unstructured data processing.
• Knowledge of ML lifecycle concepts.

Responsibilities

• Design and develop data ingestion, transformation, and processing pipelines.
• Build embedding pipelines and integrate vector databases.
• Develop pipelines for RAG, chunking, and vector search.
• Process structured and unstructured data (PDFs, documents, logs, transcripts).
• Implement data quality, governance, lineage, and access controls.
• Support ML model training, inference, MLOps, and LLMOps workflows.
• Optimize data pipelines for performance, scalability, and cost.

Skills

SQL

Python/Scala

Apache Airflow

dbt

RAG

Vector Databases

Embeddings

Vector Search

Unstructured Data

ML Lifecycle

Tools

Snowflake Cortex

Cortex Search

Job description

Cloud Analytics Technologies, LLC in Scottsdale, AZ is seeking an experienced AI Data Engineer to design and maintain scalable data pipelines and infrastructure that support AI/ML, RAG, and LLM applications. You will work with SQL, Python/Scala, cloud data platforms, and both structured and unstructured data.

Responsibilities include building ingestion and embedding pipelines, enabling vector search, ensuring data quality and governance, and supporting MLOps workflows to optimize performance and