AI Data Engineer

Delan Associates, Inc.; Atlanta (GA); On-site; USD 110,000 - 160,000; Full time

Job summary

Delan Associates, Inc. seeks a Data Engineer to design, build, and maintain data pipelines, AI solutions, and analytics platforms across diverse data sources. The role emphasizes collaboration with cross-functional teams to deliver scalable data solutions.

The position requires five years of data engineering or related experience, proficiency in Java, Python, SQL, PySpark, and API development, and experience with CI/CD and cloud technologies such as Azure Functions, Docker, and Kubernetes.

Qualifications

• Bachelor's degree required.
• Five years of experience in data engineering or related roles.
• Ability to obtain and maintain a Public Trust clearance.
• Experience with AI methods and prompt engineering.
• Experience building and maintaining data pipelines and ETL processes.
• Experience with Databricks and cloud-based data architectures.

Responsibilities

• Design, develop, test and maintain data pipelines and ETL processes for ingesting, transforming, and loading data from diverse sources.
• Build and optimize data architectures and models to support analytics, reporting, and operational needs.
• Implement and manage CI/CD pipelines for data engineering workflows using GitHub.
• Develop and deploy cloud-based solutions leveraging Azure Functions, Docker, and Kubernetes.
• Collaborate with DBAs and developers to design data models, stored procedures, and APIs.
• Ensure data quality and integrity through validation, monitoring, and logging.
• Document data flows and technical solutions for knowledge sharing and compliance.

Skills

Java

Python

SQL

PySpark

RESTful API design

SOAP API

Data pipelines

CI/CD pipelines

Git

Education

Bachelor's degree

Tools

Databricks

Azure Functions

Docker

Kubernetes

GitHub

Azure DevOps

Job description

Client seeks a Data Engineer with some AI experience to support in building, optimizing, and maintaining data pipelines, AI solutions, and analytics platforms. The ideal candidate will have hands-on experience with modern data engineering tools, AI solutions, CI/CD, and version control, and will collaborate with cross-functional teams to deliver high-quality, scalable data solutions.

• AI experience with prompt engineering, machine learning, agentic AI and RAG concepts
• Design, develop, test and maintain data pipelines and ETL processes for ingesting, transforming, and loading data from diverse sources (SQL, R, Python).
• Build and optimize data architectures and models to support analytics, reporting, and operational needs.
• Implement and manage CI/CD pipelines for data engineering workflows using tools such Git.
• Develop and deploy cloud-based solutions leveraging Azure Functions, and containerization (Docker, Kubernetes).
• Collaborate with DBAs and application developers to design and integrate data models, stored procedures, and APIs.
• Ensure data quality and integrity through validation, monitoring, and logging.
• Manage multiple tasks effectively across competing priorities.
• Troubleshoot and resolve issues in production environments, ensuring high availability and reliability.
• Document data flows, processes, and technical solutions for knowledge sharing and compliance.
• Stay current with emerging technologies and best practices in data engineering, cloud, and DevOps.

What You Will Need

• Must be able to OBTAIN and MAINTAIN a "PUBLIC TRUST".
• A bachelor’s degree is required.
• FIVE (5) years of experience in data engineering, software development, or related roles.
• Experience with Co-Pilot and CodeX., and AI methods.
• Experience presenting status and work products.
• Experience in programming languages and technologies, including Java, Python, SQL, and PySpark, as well as API development (RESTful and SOAP).
• Experience designing and developing data pipelines and ETL processes.
• Extensive experience with relational and NoSQL databases, such as MySQL, PostgreSQL, SQL Server, and MongoDB, including stored procedure development.
• Experience working with Databricks.
• Experience with CI/CD tools and pipelines, such as GitHub.
• Experience operating in Agile development environments.

What Would Be Nice To Have

• Experience supporting federal programs and/or large-scale data modernization projects.
• Familiarity with version control and collaboration tools, including Git, and Azure Dev Ops
• Ability to work independently in a fast-paced environment.
• Strong analytical and troubleshooting skills.
• Excellent verbal and written communication skills, with the ability to clearly communicate complex technical concepts to diverse audiences.
• Familiarity with containerization and orchestration technologies
• Familiarity with security best practices and compliance in federal environments.
• Experience with microservices, Spring Boot, and integration of AI/ML components.
• Previous consulting experience