AI Data Engineer: Scalable Pipelines & AI Solutions

Delan Associates, Inc.; Atlanta (GA); On-site; USD 110,000 - 160,000; Full time

Job summary

Delan Associates, Inc. seeks a Data Engineer to design, build, and maintain data pipelines, AI solutions, and analytics platforms across diverse data sources. The role emphasizes collaboration with cross-functional teams to deliver scalable data solutions.

The position requires five years of data engineering or related experience, proficiency in Java, Python, SQL, PySpark, and API development, and experience with CI/CD and cloud technologies such as Azure Functions, Docker, and Kubernetes.

Qualifications

• Bachelor's degree required.
• Five years of experience in data engineering or related roles.
• Ability to obtain and maintain a Public Trust clearance.
• Experience with AI methods and prompt engineering.
• Experience building and maintaining data pipelines and ETL processes.
• Experience with Databricks and cloud-based data architectures.

Responsibilities

• Design, develop, test and maintain data pipelines and ETL processes for ingesting, transforming, and loading data from diverse sources.
• Build and optimize data architectures and models to support analytics, reporting, and operational needs.
• Implement and manage CI/CD pipelines for data engineering workflows using GitHub.
• Develop and deploy cloud-based solutions leveraging Azure Functions, Docker, and Kubernetes.
• Collaborate with DBAs and developers to design data models, stored procedures, and APIs.
• Ensure data quality and integrity through validation, monitoring, and logging.
• Document data flows and technical solutions for knowledge sharing and compliance.

Skills

Java

Python

SQL

PySpark

RESTful API design

SOAP API

Data pipelines

CI/CD pipelines

Git

Education

Bachelor's degree

Tools

Databricks

Azure Functions

Docker

Kubernetes

GitHub

Azure DevOps

Job description

Delan Associates, Inc. seeks a Data Engineer to design, build, and maintain data pipelines, AI solutions, and analytics platforms across diverse data sources. The role emphasizes collaboration with cross-functional teams to deliver scalable data solutions.

The position requires five years of data engineering or related experience, proficiency in Java, Python, SQL, PySpark, and API development, and experience with CI/CD and cloud technologies such as Azure Functions, Docker, and Kubernetes.