Generative AI (GPT) Data Engineer

MACHINE LEARNING TECHNOLOGIES LLC; Chicago (IL); On-site; USD 120,000 - 180,000; Full time

Job summary

MACHINE LEARNING TECHNOLOGIES LLC in Chicago, IL is seeking a Generative AI Data Engineer to design, build, and maintain scalable data pipelines for structured and unstructured data, enabling GPT and LLM workflows.

You will implement RAG pipelines, prepare embeddings, integrate vector stores, and collaborate with AI/ML teams using Python, SQL, PySpark, and Spark on cloud platforms.

Qualifications

• Experience designing scalable data pipelines for structured and unstructured data.

Responsibilities

• Design, develop, and maintain scalable data pipelines for structured and unstructured data.
• Build and optimize ETL/ELT workflows for data ingestion, transformation, and processing.
• Develop data pipelines to support GPT, LLM, and Generative AI applications.
• Implement Retrieval-Augmented Generation (RAG) pipelines.
• Prepare, clean, chunk, and transform documents for embedding generation and indexing.
• Integrate vector databases and embedding models with enterprise data platforms.
• Work with OpenAI GPT models and Azure OpenAI services.
• Develop data workflows using Python, SQL, PySpark, and Spark.
• Implement data quality, validation, governance, and monitoring processes.
• Collaborate with AI/ML engineers, data scientists, and application development teams.

Skills

Data Engineering

ETL/ELT pipelines

Python

SQL

Apache Spark / PySpark

GPT / LLM concepts

RAG architecture

Vector databases

Cloud platforms

Tools

Pinecone

Weaviate

FAISS

Azure AI Search

Job description

Job Overview

Job ID: J53736

Job Title: Generative AI (GPT) Data Engineer

Location: Chicago, IL

Duration: 18 Months + Extension

Domain Exposure: Pharmaceuticals

Work Authorization: US Citizen,Green Card,OPT-EAD,CPT,H-1B,H4-EAD,L2-EAD,GC-EAD

Client: To Be Discussed Later

Employment Type: W-2, 1099, C2C

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for structured and unstructured data.
• Build and optimize ETL/ELT workflows for data ingestion, transformation, and processing.
• Develop data pipelines to support GPT, LLM, and Generative AI applications.
• Implement Retrieval-Augmented Generation (RAG) pipelines.
• Prepare, clean, chunk, and transform documents for embedding generation and indexing.
• Integrate vector databases and embedding models with enterprise data platforms.
• Work with OpenAI GPT models and Azure OpenAI services.
• Develop data workflows using Python, SQL, PySpark, and Spark.
• Implement data quality, validation, governance, and monitoring processes.
• Collaborate with AI/ML engineers, data scientists, and application development teams.

Required Skills

• Strong experience in Data Engineering and ETL/ELT pipelines.
• Hands-on experience with Python and SQL.
• Experience with Apache Spark / PySpark.
• Knowledge of GPT, LLMs, and Generative AI concepts.
• Hands-on experience with RAG architecture.
• Experience with vector databases such as Pinecone, Weaviate, FAISS, or Azure AI Search.
• Knowledge of embeddings, document chunking, and semantic search.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Experience with data warehouses and data lakes.
• Understanding of REST APIs and data integration.

Equal Opportunity Employer

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.