Generative AI Data Engineer — GPT/LLM, RAG Pipelines

Artifintech; Chicago (IL); On-site; USD 120,000 - 160,000; Full time

Job summary

Artifintech is seeking an experienced Generative AI Data Engineer to build scalable data pipelines, prepare enterprise data, and integrate data platforms with GPT and other Generative AI models in a Chicago-area, on-site role.

You will design ETL/ELT workflows, handle structured and unstructured data, and collaborate with AI/ML teams to implement RAG pipelines, embedding workflows, and vector DB integrations using Python, SQL, Spark, and cloud services.

Qualifications

• Experience designing scalable data pipelines for structured and unstructured data.
• Hands-on experience with Python and SQL.
• Experience with Apache Spark / PySpark.
• Knowledge of GPT, LLMs, and Generative AI concepts.
• Hands-on experience with RAG architecture.
• Experience with vector databases such as Pinecone, Weaviate, FAISS, or Azure AI Search.
• Knowledge of embeddings, document chunking, and semantic search.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Experience with data warehouses and data lakes.
• Understanding of REST APIs and data integration.

Responsibilities

• Design, develop, and maintain scalable data pipelines for structured and unstructured data.
• Build and optimize ETL/ELT workflows for data ingestion, transformation, and processing.
• Develop data pipelines to support GPT, LLM, and Generative AI applications.
• Implement Retrieval-Augmented Generation (RAG) pipelines.
• Prepare, clean, chunk, and transform documents for embedding generation and indexing.
• Integrate vector databases and embedding models with enterprise data platforms.
• Work with OpenAI GPT models and Azure OpenAI services.
• Develop data workflows using Python, SQL, PySpark, and Spark.
• Implement data quality, validation, governance, and monitoring processes.
• Collaborate with AI/ML engineers, data scientists, and application development teams.

Skills

Data Engineering

ETL/ELT pipelines

Python

SQL

GPT/LLMs

RAG architecture

Embeddings

Vector databases

Cloud platforms

Data warehouses/lakes

REST APIs

Tools

Apache Spark / PySpark

Pinecone

Weaviate

FAISS

Azure AI Search

Job description

Artifintech is seeking an experienced Generative AI Data Engineer to build scalable data pipelines, prepare enterprise data, and integrate data platforms with GPT and other Generative AI models in a Chicago-area, on-site role.

You will design ETL/ELT workflows, handle structured and unstructured data, and collaborate with AI/ML teams to implement RAG pipelines, embedding workflows, and vector DB integrations using Python, SQL, Spark, and cloud services.