Consultant-Data Engineer

Allianz Services; Thiruvananthapuram; On-site; INR 1,500,000 - 2,300,000; Full time

Job summary

Allianz Services in India seeks a Data Engineer + AI with 5-7 years of experience to design, build and maintain data pipelines using Azure Databricks and Synapse, enabling AI-powered data solutions for business insights.

You will collaborate with cross-functional teams, ensure data governance and security, and drive scalable ETL/ELT workflows including real-time and batch data processing. The role emphasizes continuous improvement, AI adoption, and delivering high-value data products.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Systems or related field.
• Proven 5-7 years as a Data Engineer with Azure Synapse or Databricks.
• Strong SQL and Python skills for data manipulation and automation.
• Familiarity with Databricks / Synapse Workflows and other orchestration tools.
• Knowledge of data modelling methodologies and data lakehouse concepts.
• Experience with version control and CI/CD pipelines.
• Understanding of data governance, security and compliance frameworks.
• Exposure to AI/ML workloads and embedding models.

Responsibilities

• Design, develop and maintain scalable ETL/ELT pipelines from diverse data sources.
• Collaborate with internal customers to deliver fit-for-purpose data solutions.
• Implement data security measures to ensure privacy compliance.
• Peer review code to improve robustness and understanding of the code base.
• Work with stakeholders to understand data requirements and deliver value.
• Stay updated on data engineering trends and AI tech to drive improvements.
• Support data platforms enabling Generative AI, RAG, vector search, and semantic retrieval.
• Develop embeddings, document indexing and AI-ready data products.
• Leverage AI tools to improve code quality and delivery velocity.
• Adopt modern automation and orchestration to optimize operations.

Skills

SQL

Python

Azure Databricks

Azure Synapse

Data modeling

GitHub

CI/CD

Data governance

AI/ML workloads

Copilot

Education

Bachelor’s or Master’s in Computer Science / Information Systems

Tools

Databricks

Synapse

GitHub

CI/CD

Job description

Job Description:

Data Engineer + AI

Experience Range: 5-7 years

We are seeking a talented and experienced Azure Databricks / Synapse Engineer to join our dynamic team at Allianz where you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will collaborate with cross-functional teams to ensure the efficient collection, storage and retrieval of data for analysis and reporting purposes enabling better insight and enabling decision making.

As a trusted expert, you will design and implement reliable ETL/ELT workflows, integrate both real-time and batch data sources, and ensure secure, well-governed data pipelines and products. You will excel in a collaborative, client-focused environment, with a passion for solving complex data challenges, driving innovation, and delivering high-quality data solutions.

In addition, you will support and enable AI-driven data solutions by designing and maintaining data architectures that power Generative AI use cases, including Retrieval-Augmented Generation (RAG), vector databases, embedding models, semantic search, and knowledge retrieval platforms.

A continuous improvement mindset is essential, with the ability to identify opportunities to automate repetitive processes, streamline data engineering workflows, and adopt emerging AI technologies to enhance efficiency, reliability, and business value.

The key responsibilities of the role include:

• Working with peers and senior colleagues on solutions design activity for items allocated to you in the backlog.
• Developing, implementing, testing and documenting scalable ETL (Extract, Transform, Load) data pipelines and workflows to efficiently ingest and transform data from various sources.
• Working collaboratively with internal customers to deliver high-quality fit for purpose solutions.
• Implementing data security measures to ensure compliance with data privacy regulations.
• Peer reviewing work of colleagues to ensure delivery of robust solutions and improve your understanding of the code base.
• Collaborating with stakeholders and analysts to understand their data requirements.
• Staying up-to-date with industry trends and best practices in data engineering to drive continuous improvement and innovation.
• Design and support data platforms that enable Generative AI, RAG, vector search, and semantic retrieval solutions.
• Develop and optimize data pipelines for embedding generation, document indexing, and AI-ready data products.
• Leverage Microsoft Copilot and other AI-assisted engineering tools to improve code quality, automate development tasks, accelerate troubleshooting, and increase delivery velocity.
• Continuously evaluate and adopt modern automation, orchestration, and AI technologies to optimize platform operations and engineering efficiency.

About You

Experience & Skills:

• Bachelor’s or Master’s degree, ideally in Computer Science, Information Systems or a related field
• Proven 5-7 years of work experience as a Data Engineer with either Azure Synapse or Databricks (Spark/PySpark, Delta Lake, Unity Catalog etc)
• Strong SQL and Python (or equivalent language) skills for data manipulation and automation.
• Familiarity with Databricks / Synapse Workflows and other orchestration tools.
• Knowledge of data modelling methodologies (star schemas, Data Vault).
• Familiarity with medallion architecture, data lakehouse principles and distributed data processing.
• Experience with version control tools (GitHub) and CI/CD pipelines.
• Understanding of data governance, security, and compliance frameworks.
• Working Exposure to AI/ML workloads like embedding model, RAG.
• Utilize advanced AI, automation, and productivity tools (e.g., Microsoft Copilot, Claude) to enhance efficiency, automate routine activities, and accelerate the delivery of business solutions.
• Ideally some knowledge of the Motor Insurance Market

105635 | Data & AI | Professional | Non-Executive | Allianz Services | Full-Time | Permanent

Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges, is what makes us a unique employer. Together we can build an environment where everyone feels empowered and has the confidence to explore, to grow and to shape a better future for our customers and the world around us.

At Allianz, we stand for unity: we believe that a united world is a more prosperous world, and we are dedicated to consistently advocating for equal opportunities for all. And the foundation for this is our inclusive workplace, where people and performance both matter, and nurtures a culture grounded in integrity, fairness, inclusion and trust.

We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, social class, disability or sexual orientation, or any other characteristics protected under applicable local laws and regulations.

Join us. Lets care for tomorrow.