Generative AI Data Engineer: GPT, RAG & Vector Pipelines

Ethereum Technologies; Chicago (IL); On-site; USD 120,000 - 200,000; Full time

Job summary

Ethereum Technologies in Chicago seeks an experienced Generative AI Data Engineer to build scalable data pipelines, prepare enterprise data, and integrate platforms with GPT and Generative AI models.

You will design ETL/ELT workflows, implement RAG pipelines, process documents for embeddings, and work with vector databases and OpenAI/Azure OpenAI. Strong Python, SQL, Spark, and cloud experience are required.

Qualifications

• Experience building scalable ETL/ELT pipelines for structured and unstructured data.
• Strong Python and SQL skills with hands-on Spark/PySpark experience.
• Hands-on experience with GPT, LLMs, and Generative AI concepts.
• Experience with RAG architecture and vector databases for embedding.
• Proficiency with cloud data platforms (AWS/Azure/GCP) and data warehouses/lakes.

Responsibilities

• Design, develop, and maintain scalable data pipelines for structured and unstructured data.
• Build and optimize ETL/ELT workflows for data ingestion, transformation, and processing.
• Develop data pipelines to support GPT, LLM, and Generative AI applications.
• Implement Retrieval-Augmented Generation (RAG) pipelines.
• Prepare, clean, chunk, and transform documents for embedding generation and indexing.
• Integrate vector databases and embedding models with enterprise data platforms.
• Work with OpenAI GPT models and Azure OpenAI services.
• Develop data workflows using Python, SQL, PySpark, and Spark.
• Implement data quality, validation, governance, and monitoring processes.
• Collaborate with AI/ML engineers, data scientists, and application development teams.

Skills

Data engineering

Python

SQL

Spark / PySpark

GPT / LLMs / Generative AI

RAG architecture

Vector databases

Cloud platforms

Data lakes / warehouses

REST APIs

Tools

Pinecone

Weaviate

FAISS

Azure AI Search

Job description

Ethereum Technologies in Chicago seeks an experienced Generative AI Data Engineer to build scalable data pipelines, prepare enterprise data, and integrate platforms with GPT and Generative AI models.

You will design ETL/ELT workflows, implement RAG pipelines, process documents for embeddings, and work with vector databases and OpenAI/Azure OpenAI. Strong Python, SQL, Spark, and cloud experience are required.