Data Engineer - AI Labs

IDFC FIRST Bank; Bengaluru; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

IDFC FIRST Bank in Bengaluru is seeking a Data Engineer GenAI within the Data & Analytics function. You will work with Data Scientists to support GenAI solutions across text, audio, image, and tabular data, building robust data pipelines and scalable data architecture.

The role focuses on data ingestion, governance, APIs, and collaboration with cross-functional teams to deliver reliable GenAI-powered data products. Requires 2+ years of experience with big data technologies and cloud platforms.

Responsibilities

• Build and maintain data engineering pipelines, with a focus on unstructured data.
• Design, build, and optimize data architecture and ETL pipelines for accessibility by Data Scientists and GenAI products.
• Manage the full data lifecycle: ingestion, transformation, and consumption.
• Ensure high standards of data reliability, integrity, and governance.
• Work with APIs to enable seamless data integration and usability.
• Create technical design documentation for data pipelines and projects.
• Debug technical issues and manage code versioning using Git.
• Demonstrate experience with big data infrastructure such as MapReduce, Hive, HDFS, YARN, HBase, MongoDB, DynamoDB, etc.
• Apply machine learning and predictive analytics techniques where relevant.
• Leverage domain knowledge in banking or financial services to enhance data solutions.
• Present data insights using effective storytelling and visualization techniques.

Job description

Job Requirements

About the Role

The Data Engineer GenAI within the Data & Analytics function will work closely with Data Scientists to support the development of generative AI solutions across text, audio, image, and tabular data domains. This role is responsible for managing large volumes of structured and unstructured data, ensuring its efficient storage, retrieval, and augmentation to power GenAI models and applications. The ideal candidate will be skilled in building robust data pipelines, optimizing data architecture, and ensuring high standards of data reliability and governance.

Key Responsibilities

Primary Responsibilities

• Build and maintain data engineering pipelines, with a focus on unstructured data.
• Conduct requirements gathering and scoping sessions with business users and stakeholders to define GenAI-related data needs.
• Design, build, and optimize data architecture and ETL pipelines for accessibility by Data Scientists and GenAI products.
• Manage the full data lifecycle: ingestion, transformation, and consumption.
• Ensure high standards of data reliability, integrity, and governance.
• Work with APIs to enable seamless data integration and usability.
• Create technical design documentation for data pipelines and projects.
• Debug technical issues and manage code versioning using Git.
• Demonstrate experience with big data infrastructure such as MapReduce, Hive, HDFS, YARN, HBase, MongoDB, DynamoDB, etc.

Secondary Responsibilities

• Apply machine learning and predictive analytics techniques where relevant.
• Leverage domain knowledge in banking or financial services to enhance data solutions.
• Present data insights using effective storytelling and visualization techniques.

What We Are Looking For

Education

• Bachelor’s or master’s degree in computer science or Data Engineering or Information Systems or a related field.

Experience

• 2+ years of Proven experience in building and managing data pipelines and architectures.
• Hands-on experience with big data technologies and cloud platforms (AWS, GCP, or Azure).
• Exposure to GenAI applications and working with unstructured data formats.

Skills and Attributes

• Strong programming and debugging skills.
• Proficiency in data architecture, ETL design, and pipeline optimization.
• Familiarity with API integration and cloud services.
• Ability to work collaboratively with cross-functional teams.
• Excellent documentation and communication skills.
• Strong problem-solving mindset and attention to detail.
• Ability to deliver high-quality outputs under tight timelines.