Data Engineer with AI/GenAI

3shool Technology Consultants; Dadri, Hyderabad, Bengaluru; On-site; INR 900,000 - 1,800,000; Full time

Job summary

3shool Technology Consultants in India invites experienced data engineering and AI/ML professionals to join the team. The role focuses on building scalable data pipelines using Databricks, Spark, Snowflake, and real-time streaming, while integrating Agentic AI frameworks and Cortex-based KPI dashboards.

You will work with Azure Functions, ADF/Airflow/dbt, and DevOps practices to deliver robust analytics solutions.

Qualifications

• Databricks (PySpark, Spark SQL, Structured Streaming) proficiency.
• Python programming experience.
• Snowflake (SQL, Snowpark, Snowpipe Streaming, Data Aggregations, ELT) expertise.
• Kafka and real-time streaming pipelines experience.
• Agentic AI frameworks knowledge (Semantic Kernel, LangChain, AutoGen, CrewAI).
• Azure Functions / serverless deployments knowledge.
• KPI dashboard automation using Cortex Analyst/Cortex Agents preferred.
• ADF / Airflow / dbt exposure.
• DevOps practices with Git, CI/CD, Azure DevOps or GitHub Actions.
• Telecom domain experience with KPI/Telemetry data preferred.

Skills

Databricks

Python

Snowflake

Kafka

Agentic AI Frameworks

Azure Functions

KPI Dashboard Automation

ADF / Airflow / dbt

DevOps

Telecom Domain experience

MLOps

Power BI

SQL

Vector Databases

Generative AI

Tools

Databricks

Snowflake

Agentic AI Frameworks

Azure Functions

ADF

Airflow

dbt

Git

CI/CD

Job description

Skills

Databricks, Snowflake, Agentic AI, RAG, MLOps, Machine Learning, Generative AI, Power BI, SQL, Python, Vector Databases

Detailed Mandatory Skills:

• Databricks (PySpark, Spark SQL, Structured Streaming)
• Python
• Snowflake (SQL, Snowpark, Snowpipe Streaming, Data Aggregations, ELT)
• Kafka & Real-time Streaming Pipelines
• Agentic AI Frameworks (Semantic Kernel / LangChain / AutoGen / CrewAI)
• Azure Functions / Serverless Deployments
• KPI Dashboard Automation (Cortex Analyst/Cortex Agents preferred)
• ADF / Airflow / dbt
• DevOps (Git, CI/CD, Azure DevOps/GitHub Actions)
• Telecom Domain experience with Network KPI/Telemetry data preferred