Data Engineer: AI-Driven Data Pipelines & Cloud Solutions

Guidehouse; Fairview Heights (IL); On-site; USD 89,000 - 148,000; Full time

Benefits offered by this job

Medical Insurance

401(k) Plan

Tuition Reimbursement

Employee Assistance Program

Paid Holidays

Job summary

Guidehouse seeks a Data Engineer with AI experience to design, build, test, and maintain data pipelines and analytics platforms. The role requires hands-on experience with modern data engineering tools, CI/CD, and cloud-based solutions, collaborating with cross-functional teams to deliver scalable data solutions.

The position emphasizes building and optimizing data architectures, implementing CI/CD, and leveraging Azure Functions, Docker, and Kubernetes to deploy robust data products.

Qualifications

• U.S. Citizenship/Green Card required and must obtain and maintain a Federal or DHS Public Trust.
• Bachelor’s degree required in a related field.
• 5+ years of experience in data engineering, software development, or related roles.
• Experience in programming languages and technologies including Java, Python, SQL, and PySpark, plus API development (RESTful and SOAP).
• Experience with CI/CD and version control (Git), Databricks, and relational/NoSQL databases.

Responsibilities

• Design, develop, test and maintain data pipelines and ETL processes for diverse data sources (SQL, R, Python).
• AI experience with prompt engineering, machine learning, agentic AI and RAG concepts.
• Build and optimize data architectures and models to support analytics and operations.
• Implement and manage CI/CD pipelines for data engineering workflows using Git.
• Develop and deploy cloud-based solutions leveraging Azure Functions, and containerization (Docker, Kubernetes).
• Collaborate with DBAs and application developers to design and integrate data models, stored procedures, and APIs.
• Ensure data quality and integrity through validation, monitoring, and logging.
• Triage and resolve production issues ensuring high availability and reliability.
• Document data flows, processes, and technical solutions for knowledge sharing and compliance.
• Stay current with emerging technologies and best practices in data engineering, cloud, and DevOps.

Skills

Java

Python

SQL

PySpark

RESTful APIs

SOAP

CI/CD

Git

Azure Functions

Docker

Kubernetes

Databricks

Agile

Co-Pilot

CodeX

Education

Bachelor's degree

Tools

Git

GitHub

Databricks

Azure Functions

Docker

Kubernetes

Jira

Confluence

CI/CD tooling

Job description

Guidehouse seeks a Data Engineer with AI experience to design, build, test, and maintain data pipelines and analytics platforms. The role requires hands-on experience with modern data engineering tools, CI/CD, and cloud-based solutions, collaborating with cross-functional teams to deliver scalable data solutions.

The position emphasizes building and optimizing data architectures, implementing CI/CD, and leveraging Azure Functions, Docker, and Kubernetes to deploy robust data products.