Senior LLM/RAG Engineer

Data Meaning; São Paulo; Teletrabalho; USD 80.000 - 120.000; Tempo integral

Resumo da oferta

A prominent consulting firm in Business Intelligence is seeking a Senior LLM/RAG Engineer to enhance their data management capabilities. This role focuses on API integrations and developing Retrieval-Augmented Generation systems using Azure OpenAI. The ideal candidate will possess robust Python skills and a solid understanding of data governance. Join a collaborative team dedicated to innovation and excellence in data solutions.

Qualificações

• Strong proficiency in Python and experience with API integrations.
• Knowledgeable in implementing and managing RAG or similar AI-driven systems.
• Understanding of data governance and metadata management best practices.

Responsabilidades

• Design and implement API connections to fetch metadata.
• Develop and maintain a RAG system to augment metadata.
• Build and deploy scalable applications integrating RAG and AI technologies.

Conhecimentos

Python

API integrations

Data governance

Metadata management

Formação académica

Bachelor’s degree in Computer Science, Information Technology, or related field

Ferramentas

Azure

Collibra

Descrição da oferta de emprego

Senior LLM/RAG Engineer

Location: BR/LATAM

Contract Type: TBD

Data Meaning is a front-runner in Business Intelligence and Data Analytics consulting, renowned for our high-quality consulting services throughout the US and LATAM. Our expertise lies in delivering tailored solutions in Business Intelligence, Data Warehousing, and Project Management. We are dedicated to bringing premier services to our diverse clientele. We have a global team of 95+ consultants, all working remotely, embodying a collaborative, inclusive, and innovation-driven work culture.

Overview
We are looking for a skilled RAG Developer to join our data management team. The role focuses on developing and implementing Retrieval-Augmented Generation (RAG) techniques to enhance metadata quality and provide insightful table and column descriptions using Azure OpenAI. This position requires robust Python programming skills, experience with API integrations, and a keen interest in AI-driven data enhancement.

Responsibilities:

API Integration

• Design and implement API connections to fetch metadata from various sources, including Collibra.
• Maintain and optimize API performance to ensure efficient data retrieval.

RAG Implementation

• Develop and maintain a RAG system to augment metadata with accurate and contextually relevant information.
• Utilize Azure OpenAI to generate and enhance metadata descriptions.

Application Development

• Build and deploy scalable applications that integrate RAG and AI technologies.
• Ensure application compliance with security, privacy, and data management standards.

Collaboration and Support

• Collaborate with data governance and IT teams to align RAG outputs with organizational needs.

• Support project management with updates, documentation, and insights from the RAG system.

Documentation and Compliance

• Document all processes and maintain records for internal standards and regulatory compliance.

• Develop reports to monitor the effectiveness and accuracy of the RAG system.

Requirements:

• Bachelor’s degree in Computer Science, Information Technology, or related field.

• Strong proficiency in Python and experience with API integrations.

• Knowledgeable in implementing and managing RAG or similar AI-driven systems.

• Understanding of data governance, metadata management, and related best practices.

• Experience with cloud platforms, preferably Microsoft Azure.

Preferred Qualifications:

• Experience with Collibra APIs is highly advantageous.

• Master’s degree or advanced certifications in data science, AI, or cloud technologies.

• Proven ability to document processes and create detailed reports.