Senior Azure AI Data Engineer

Jobtailor; Kolkata District; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Jobtailor is seeking a data engineer to design, build, and operate scalable data pipelines and AI-enabled data solutions in an enterprise environment.

You will work with Azure OpenAI, LangChain, and PySpark to deliver end-to-end data workflows, collaborating with engineers and stakeholders to translate requirements into robust architectures.

Qualifications

• Bachelor's degree in Computer Science or related field required.
• Hands-on experience with Azure data services and Databricks (PySpark) preferred.
• Experience building ETL/ELT pipelines and lakehouse architectures on Azure.

Responsibilities

• Actively contribute within a team of Engineers and Architects to design, prototype, and deliver scalable data and AI solutions.
• Design and build scalable data pipelines and processing frameworks using Azure services, Databricks, and PySpark.
• Collaborate with stakeholders to translate requirements into robust data architectures and AI-enabled designs.
• Develop and operationalize AI-enabled data use cases including RAG patterns and LLM integrations.
• Implement end-to-end data solutions with performance, scalability, and reliability.
• Work with Azure OpenAI, LangChain, and related tools to enable intelligent data workflows.
• Contribute to CI/CD pipelines, automation, and deployment processes for data and AI workflows.

Education

Bachelor's degree in Computer Science or related field

Tools

Azure Databricks (PySpark)

Azure Synapse

Azure data services

Databricks

LangChain

APIs integration

Job description

Responsibilities

• Actively contribute within a team of Engineers and Architects to design, prototype, and deliver scalable data and AI solutions, including proof of concepts (POCs) and production deployments.
• Design and build scalable data pipelines and processing frameworks using Azure services, Databricks, and PySpark across large‑scale enterprise environments.
• Collaborate with business and technical stakeholders to translate requirements into robust data architectures and AI‑enabled solution designs.
• Develop and operationalize AI‑enabled data use cases, including integration of enterprise data with LLM‑based applications, implementation of Retrieval Augmented Generation (RAG) solutions, and enabling intelligent data discovery and consumption using AI tools.
• Implement end‑to‑end data solutions adhering to best practices in data engineering, including performance optimization, scalability, and reliability.
• Work with modern AI and data ecosystems (e.g., Azure OpenAI, LangChain) to enable intelligent data workflows integrated with enterprise platforms.
• Contribute to CI/CD pipelines, automation, and deployment processes for both data pipelines and AI workflows.

Requirements

• 4–7 years of hands‑on experience in data engineering
• 3+ years of experience with Azure data services and Databricks (PySpark)
• Strong experience in building ETL/ELT pipelines and lakehouse architectures on Azure
• Solid understanding of data modelling, data integration patterns, and metadata management
• Experience with enterprise data warehousing solutions (Azure Synapse, Databricks, or Microsoft Fabric)
• Working knowledge of LLM‑based applications, embeddings, and RAG patterns
• Hands‑on experience with AI frameworks such as Azure OpenAI, LangChain, or similar
• Experience in integrating systems using APIs, ETL tools, and modern data platforms
• Strong experience with CI/CD practices, version control, testing, and deployment
• Ability to solve complex data challenges at scale
• Strong communication skills with ability to articulate technical and AI concepts clearly
• Bachelor’s degree in Computer Science or related field
• Experience with NoSQL databases and advanced data modelling techniques.