Azure Databricks Data Engineer | PySpark & API Focus

TALENT Software Services; Minnetonka Mills (MN); On-site; USD 110,000 - 140,000; Full time

Job summary

TALENT Software Services is seeking a Data Engineer / API Role with AI Builder in Minnesota to design, build, and optimize cloud-based data pipelines on Azure. You will work with Databricks, ADF, PySpark, and Delta Lake to deliver scalable solutions and integrated APIs.

The role emphasizes end-to-end delivery, data governance, and security, with an on-site setup in Minnesota for an experienced professional.

Qualifications

• Experience designing scalable data pipelines on Azure.
• Experience building REST/GraphQL APIs and deploying on Kubernetes/AKS.
• Exposure to AI/LLM technologies and Databricks AI/ML?
• Knowledge of Unity Catalog and data governance.

Responsibilities

• Own end-to-end solution delivery for data and API workloads.
• Analyze and optimize data pipelines for performance and reliability.
• Collaborate with cross-functional teams and stakeholders.
• Build scalable data pipelines and cloud-based solutions on Azure.
• Implement enterprise governance, security, and deployment standards.
• Optimize API performance and governance for enterprise use.

Skills

Databricks

Azure Data Factory

PySpark

Python

SQL

GitHub

CI/CD

Azure Cloud

Delta Lake

DDD/Data Modeling

Tools

Azure Databricks

PySpark

Scala

Azure Data Factory (ADF)

ADLS Gen2

Delta Lake

Unity Catalog

Python

SQL

GitHub

CI/CD

Kubernetes

AKS

Job description

TALENT Software Services is seeking a Data Engineer / API Role with AI Builder in Minnesota to design, build, and optimize cloud-based data pipelines on Azure. You will work with Databricks, ADF, PySpark, and Delta Lake to deliver scalable solutions and integrated APIs.

The role emphasizes end-to-end delivery, data governance, and security, with an on-site setup in Minnesota for an experienced professional.