Azure Databricks + AI Engineer

Signature IT World Inc; Illinois; On-site; USD 140,000 - 190,000; Full time

Job summary

Signature IT World Inc. is seeking anexperienced Azure Databricks + AI Engineer to build scalable data, AI/ML, and Generative AI solutions on the Azure platform.

The ideal candidate will have hands-on expertise in Azure Databricks, PySpark, Python, SQL, Delta Lake, and Lakehouse architecture, and will design end-to-end data pipelines with integration to ADLS Gen2, Azure Data Factory, and Azure OpenAI. Responsibilities include developing AI/ML and Generative AI capabilities, deploying models with

Qualifications

• 6+ years of experience in Data Engineering / AI / ML.
• Strong hands-on Azure Databricks, PySpark, Python, and SQL.
• Strong knowledge of Delta Lake and Lakehouse architecture.
• Experience with Azure Data Factory and ADLS Gen2.
• Hands-on experience with GenAI, LLMs, RAG, embeddings, and vector databases.
• Experience with Azure OpenAI, MLflow, or Databricks ML.
• Understanding of MLOps, CI/CD, cloud security, and data governance.

Responsibilities

• Design and develop data and AI solutions using Azure Databricks and Azure services.
• Build scalable pipelines using PySpark, Python, SQL, Delta Lake, and Lakehouse architecture.
• Develop AI/ML and Generative AI solutions using LLMs, RAG, embeddings, and vector search.
• Work with Databricks ML, MLflow, Mosaic AI, and model deployment workflows.
• Integrate Databricks with ADLS Gen2, Azure Data Factory, Azure Synapse, Azure OpenAI, and other Azure services.
• Implement MLOps, CI/CD, monitoring, security, and governance.
• Optimize Spark jobs, data pipelines, and AI workloads for performance and cost.
• Collaborate with data scientists, engineers, architects, and business teams.

Skills

Azure Databricks

PySpark

Python

SQL

Delta Lake

Lakehouse

GenAI

LLMs

RAG

embeddings

vector databases

Azure Data Factory

ADLS Gen2

Azure OpenAI

MLflow

Databricks ML

MLOps

CI/CD

cloud security

data governance

Tools

Databricks ML

MLflow

Mosaic AI

Job description

We are looking for an experienced Azure Databricks + AI Engineer to build scalable data, AI/ML, and Generative AI solutions on the Azure platform. The ideal candidate should have strong hands-on expertise in Azure Databricks, PySpark, Python, SQL, and GenAI.

Key Responsibilities

• Design and develop data and AI solutions using Azure Databricks and Azure services.
• Build scalable pipelines using PySpark, Python, SQL, Delta Lake, and Lakehouse architecture.
• Develop AI/ML and Generative AI solutions using LLMs, RAG, embeddings, and vector search.
• Work with Databricks ML, MLflow, Mosaic AI, and model deployment workflows.
• Integrate Databricks with ADLS Gen2, Azure Data Factory, Azure Synapse, Azure OpenAI, and other Azure services.
• Implement MLOps, CI/CD, monitoring, security, and governance.
• Optimize Spark jobs, data pipelines, and AI workloads for performance and cost.
• Collaborate with data scientists, engineers, architects, and business teams.

Required Skills

• 6+ years of experience in Data Engineering / AI / ML.
• Strong hands-on Azure Databricks, PySpark, Python, and SQL.
• Strong knowledge of Delta Lake and Lakehouse architecture.
• Experience with Azure Data Factory and ADLS Gen2.
• Hands-on experience with GenAI, LLMs, RAG, embeddings, and vector databases.
• Experience with Azure OpenAI, MLflow, or Databricks ML.
• Understanding of MLOps, CI/CD, cloud security, and data governance.