AI-Driven Data Engineer: Pipelines & Knowledge Graphs

GEP Worldwide; Clark (NJ); Hybrid; USD 90,000 - 130,000; Full time

Benefits offered by this job

Medical, dental, and vision coverage

401(k) with 3% company match

Paid time off and company holidays

Professional development opportunities

Job summary

GEP Worldwide in Clark, NJ is seeking a Data Engineer to help build and maintain data infrastructure that keeps data flowing from source to insight. You will contribute to AI-driven workflows and conversational reporting to enable explainable answers from procurement and supply chain data.

This hybrid role requires strong Python, Spark, SQL, and cloud experience, with collaboration across Data Scientists and Engineering teams to deliver production-grade pipelines for scalable business insights.

Qualifications

• Hands-on Data Engineering experience building and owning production pipelines.
• Strong Python skills used for real pipeline and data engineering work.
• PySpark or Apache Spark experience for processing data at scale.
• Solid SQL skills — schema design, query optimization, working with relational databases.
• Experience with at least one major cloud platform (AWS, Azure, or GCP) using data services, not just compute.
• AI embedded in data engineering work — RAG pipelines, LLM-driven workflows, ML model infrastructure, or similar.

Responsibilities

• Design, build, and maintain data pipelines that move and transform large volumes of data reliably, at scale, every day.
• Embed AI into your engineering work — RAG pipelines, LLM-driven workflows, or model scoring built directly into the data systems you own.
• Build the data infrastructure that Data Scientists need to train, retrain, and run ML models in production.
• Work with cloud platforms (AWS, Azure, or GCP) to design and operate cloud-native data solutions.
• Use orchestration tools like Apache Airflow / Azure ADF to schedule, monitor, and manage pipeline workflows.
• Set up data quality checks and automated testing so problems are caught before they reach downstream systems.
• Collaborate closely with Data Scientists, Engineers, and Product teams to turn business requirements into data infrastructure.
• Contribute to a codebase that is production-grade — reviewed, tested, documented, and built to last.

Skills

Data engineering

Python

PySpark/Spark

SQL

Cloud platforms

AI in data eng

Tools

Databricks

Microsoft Fabric

Apache Airflow

Azure ADF

LangChain

Vector databases

Neo4j/GraphRAG

Job description

GEP Worldwide in Clark, NJ is seeking a Data Engineer to help build and maintain data infrastructure that keeps data flowing from source to insight. You will contribute to AI-driven workflows and conversational reporting to enable explainable answers from procurement and supply chain data.

This hybrid role requires strong Python, Spark, SQL, and cloud experience, with collaboration across Data Scientists and Engineering teams to deliver production-grade pipelines for scalable business insights.