Azure Data Engineer with Gen AI

9nexus; Bengaluru; On-site; INR 1,200,000 - 2,500,000; Full time

Job summary

9nexus in Bengaluru seeks a data engineer to develop scalable data pipelines using Azure Databricks, PySpark, Python and SQL. You will build ETL/ELT workflows with Azure Data Factory, ADLS Gen2, Delta Lake and Synapse, while optimizing Spark jobs for cost and performance.

You will also implement CI/CD with Git and Azure DevOps and design Generative AI solutions on Azure, including Azure OpenAI and vector search, integrating data with Gen AI apps across teams.

Qualifications

• Experience building scalable data pipelines on Azure cloud platforms.
• Strong knowledge of Spark, Databricks, PySpark, Python and SQL.
• Hands-on experience with CI/CD using Git and Azure DevOps.
• Proficiency with Azure Data Factory, ADLS Gen2, Delta Lake and Synapse.
• Familiarity with Gen AI on Azure, OpenAI services and vector search.

Responsibilities

• Develop and maintain scalable data pipelines using Azure Databricks, PySpark, Python and SQL.
• Build ETL/ELT pipelines with Azure Data Factory, ADLS Gen2, Delta Lake and Synapse.
• Optimize Spark jobs and SQL queries for performance and cost.
• Implement CI/CD using Git and Azure DevOps.
• Design and develop Generative AI solutions on Azure, including Azure OpenAI, AI Search, RAG, embeddings and vector search.
• Integrate enterprise data with Gen AI applications and ensure secure, scalable solutions.
• Collaborate with customer technical and business teams.

Skills

Azure Databricks

PySpark

Python

SQL

Azure Data Factory

ADLS Gen2

Delta Lake

Azure Synapse

Git

Azure DevOps

Generative AI

Azure OpenAI

Vector search

Tools

Spark

Job description

• Develop and maintain scalable data pipelines using Azure Databricks, PySpark, Python, and SQL.
• Build ETL/ELT pipelines using Azure Data Factory, ADLS Gen2, Delta Lake, and Azure Synapse.
• Optimize Spark jobs, Databricks workloads, and SQL queries for performance and cost.
• Implement CI/CD using Git and Azure DevOps.
• Design and develop Generative AI solutions on Azure, including Azure OpenAI, Azure AI Search, RAG, embeddings, and vector search.
• Integrate enterprise data with Gen AI applications and implement secure, scalable solutions.
• Collaborate directly with customer technical and business teams.