AI Data Engineer

Delan Associates, Inc.; United States; On-site; USD 120,000 - 160,000; Full time

Job summary

Delan Associates, Inc. is seeking a Data Engineer with AI experience to support data pipelines, AI solutions, and analytics platforms.

You will design and maintain ETL processes, build cloud-based data solutions, and implement CI/CD for data workflows. Ideal candidates have 5 years of data engineering experience, strong programming skills (Java, Python, SQL, PySpark), and familiarity with Databricks, GitHub, and Azure DevOps.

Qualifications

• Bachelor's degree required; 5 years in data engineering or related roles.
• Must obtain and maintain a Public Trust.
• Experience with Java, Python, SQL, PySpark and API development (REST/SOAP).
• Experience with Databricks, CI/CD and Git.

Responsibilities

• Design, develop, test and maintain data pipelines and ETL processes for ingesting, transforming, and loading data from diverse sources (SQL, R, Python).
• Build and optimize data architectures and models to support analytics, reporting, and operational needs.
• Implement and manage CI/CD pipelines for data engineering workflows using Git.
• Develop and deploy cloud-based solutions leveraging Azure Functions, and containerization (Docker, Kubernetes).
• Collaborate with DBAs and application developers to design and integrate data models, stored procedures, and APIs.
• Ensure data quality and integrity through validation, monitoring, and logging.
• Document data flows, processes, and technical solutions for knowledge sharing and compliance.

Skills

Java

Python

SQL

PySpark

AI methods

Co-Pilot and CodeX

REST APIs

Education

Bachelor's degree

Tools

Databricks

GitHub

Azure DevOps

CI/CD

Azure Functions

Docker

Kubernetes

MySQL

PostgreSQL

SQL Server

MongoDB

APIs (REST/SOAP)

Job description

Client seeks a Data Engineer with some AI experience to support in building, optimizing, and maintaining data pipelines, AI solutions, and analytics platforms. The ideal candidate will have hands-on experience with modern data engineering tools, AI solutions, CI/CD, and version control, and will collaborate with cross-functional teams to deliver high-quality, scalable data solutions.

• AI experience with prompt engineering, machine learning, agentic AI and RAG concepts
• Design, develop, test and maintain data pipelines and ETL processes for ingesting, transforming, and loading data from diverse sources (SQL, R, Python).
• Build and optimize data architectures and models to support analytics, reporting, and operational needs.
• Implement and manage CI/CD pipelines for data engineering workflows using tools such Git.
• Develop and deploy cloud-based solutions leveraging Azure Functions, and containerization (Docker, Kubernetes).
• Collaborate with DBAs and application developers to design and integrate data models, stored procedures, and APIs.
• Ensure data quality and integrity through validation, monitoring, and logging.
• Manage multiple tasks effectively across competing priorities.
• Troubleshoot and resolve issues in production environments, ensuring high availability and reliability.
• Document data flows, processes, and technical solutions for knowledge sharing and compliance.
• Stay current with emerging technologies and best practices in data engineering, cloud, and DevOps.

What You Will Need:

• Must be able to OBTAIN and MAINTAIN a "PUBLIC TRUST".
• A bachelor’s degree is required.
• FIVE (5) years of experience in data engineering, software development, or related roles.
• Experience with Co-Pilot and CodeX., and AI methods.
• Experience presenting status and work products.
• Experience in programming languages and technologies, including Java, Python, SQL, and PySpark, as well as API development (RESTful and SOAP).
• Experience designing and developing data pipelines and ETL processes.
• Extensive experience with relational and NoSQL databases, such as MySQL, PostgreSQL, SQL Server, and MongoDB, including stored procedure development.
• Experience working with Databricks.
• Experience with CI/CD tools and pipelines, such as GitHub.
• Experience operating in Agile development environments.

What Would Be Nice To Have:

• Experience supporting federal programs and/or large-scale data modernization projects.
• Familiarity with version control and collaboration tools, including Git, and Azure Dev Ops
• Ability to work independently in a fast-paced environment.
• Strong analytical and troubleshooting skills.
• Excellent verbal and written communication skills, with the ability to clearly communicate complex technical concepts to diverse audiences.
• Familiarity with containerization and orchestration technologies
• Familiarity with security best practices and compliance in federal environments.
• Experience with microservices, Spring Boot, and integration of AI/ML components.
• Previous consulting experience

Originally posted on Himalayas