AI-Powered Data Engineer for Scalable Pipelines

Delan Associates, Inc.; United States; Presencial; USD 120.000 - 160.000; Jornada completa

Descripción de la vacante

Delan Associates, Inc. is seeking a Data Engineer with AI experience to support data pipelines, AI solutions, and analytics platforms.

You will design and maintain ETL processes, build cloud-based data solutions, and implement CI/CD for data workflows. Ideal candidates have 5 years of data engineering experience, strong programming skills (Java, Python, SQL, PySpark), and familiarity with Databricks, GitHub, and Azure DevOps.

Formación

• Bachelor's degree required; 5 years in data engineering or related roles.
• Must obtain and maintain a Public Trust.
• Experience with Java, Python, SQL, PySpark and API development (REST/SOAP).
• Experience with Databricks, CI/CD and Git.

Responsabilidades

• Design, develop, test and maintain data pipelines and ETL processes for ingesting, transforming, and loading data from diverse sources (SQL, R, Python).
• Build and optimize data architectures and models to support analytics, reporting, and operational needs.
• Implement and manage CI/CD pipelines for data engineering workflows using Git.
• Develop and deploy cloud-based solutions leveraging Azure Functions, and containerization (Docker, Kubernetes).
• Collaborate with DBAs and application developers to design and integrate data models, stored procedures, and APIs.
• Ensure data quality and integrity through validation, monitoring, and logging.
• Document data flows, processes, and technical solutions for knowledge sharing and compliance.

Conocimientos

Java

Python

SQL

PySpark

AI methods

Co-Pilot and CodeX

REST APIs

Educación

Bachelor's degree

Herramientas

Databricks

GitHub

Azure DevOps

CI/CD

Azure Functions

Docker

Kubernetes

MySQL

PostgreSQL

SQL Server

MongoDB

APIs (REST/SOAP)

Descripción del empleo

Delan Associates, Inc. is seeking a Data Engineer with AI experience to support data pipelines, AI solutions, and analytics platforms.

You will design and maintain ETL processes, build cloud-based data solutions, and implement CI/CD for data workflows. Ideal candidates have 5 years of data engineering experience, strong programming skills (Java, Python, SQL, PySpark), and familiarity with Databricks, GitHub, and Azure DevOps.