Junior Applied AI Engineer (LLM Systems)

encora10; United States; Remote; USD 60,000 - 90,000; Full time

Job summary

Coforge seeks a Junior Applied AI Engineer (LLM Systems) to design and evaluate AI agents, build guardrails, and optimize prompts in a remote LATAM setup. You will develop MCP servers, support fine-tuning, and monitor performance using Langfuse while collaborating with engineering, data science, and compliance teams.

The role emphasizes Python, FastAPI development, and experience with LLMs in regulated domains, such as financial services, ensuring secure, compliant AI solutions.

Qualifications

• 1-3+ years of experience in ML, Applied AI, DS, or software engineering.
• Experience with Large Language Models (LLMs) and AI-powered applications.
• Knowledge of LLM evaluation techniques, benchmarking, and LLM-as-a-Judge approaches.
• Understanding of AI agent behavior across multi-turn conversations and tool-calling workflows.
• Experience with prompt engineering, structured outputs, and prompt optimization.
• Familiarity with AI safety concepts including moderation, prompt injection prevention, jailbreak mitigation, and guardrails.
• Hands‑on experience with AWS cloud services, preferably Amazon Bedrock.
• Experience with Langfuse or similar observability and evaluation platforms for LLM applications.
• Strong programming skills in Python.
• Experience developing REST APIs using FastAPI or similar frameworks.
• Knowledge of software engineering best practices including unit testing, code reviews, and Git‑based development workflows.
• Advanced English communication skills (written and verbal).

Responsibilities

• Design and execute evaluation frameworks for AI agents, including test datasets, LLM-as-a-Judge methodologies, human reviews, and regression testing.
• Build, configure, and optimize AI guardrails and moderation systems for regulated domains such as financial services and credit products.
• Develop, test, and continuously improve prompts based on evaluation results and production trace analysis.
• Create and maintain MCP (Model Context Protocol) servers and tool integrations that enable AI agents to interact with internal data and services.
• Support fine-tuning and evaluation of language models using domain-specific datasets.
• Prototype and validate AI workflows using System One models such as Jev and Laya for classification, routing, and verification tasks.
• Monitor AI system quality, latency, performance, and costs through Langfuse and implement improvement initiatives.
• Collaborate with Engineering, Data Science, and Compliance teams to ensure AI solutions are reliable, secure, and compliant.

Skills

LLMs

Generative AI

AWS Bedrock

Prompt Engineering

AI Agent Evaluation

Langfuse

Python

FastAPI

MCP

Guardrails & Moderation

REST APIs

Git

Tools

FastAPI

MCP

Langfuse

Job description

Job Title

Junior Applied AI Engineer (LLM Systems)

Key Skills

• LLMs
• Generative AI
• AWS Bedrock
• Prompt Engineering
• AI Agent Evaluation
• Langfuse
• Python
• FastAPI
• MCP
• Guardrails & Moderation
• REST APIs
• Git

Experience

1-3+ years of experience in AI/ML Engineering, Applied AI, Data Science, or related fields.

Location

LATAM (Remote)

Mode

Remote

We at Coforge are hiring Junior Applied AI Engineer (LLM Systems) (20848-10-1) with the following skill set.

Key Responsibilities

• Design and execute evaluation frameworks for AI agents, including test datasets, LLM-as-a-Judge methodologies, human reviews, and regression testing.
• Build, configure, and optimize AI guardrails and moderation systems for regulated domains such as financial services and credit products.
• Develop, test, and continuously improve prompts based on evaluation results and production trace analysis.
• Create and maintain MCP (Model Context Protocol) servers and tool integrations that enable AI agents to interact with internal data and services.
• Support fine-tuning and evaluation of language models using domain-specific datasets.
• Prototype and validate AI workflows using System One models such as Jev and Laya for classification, routing, and verification tasks.
• Monitor AI system quality, latency, performance, and costs through Langfuse and implement improvement initiatives.
• Collaborate with Engineering, Data Science, and Compliance teams to ensure AI solutions are reliable, secure, and compliant.

Required Skills & Qualifications

• 1-3+ years of experience in Machine Learning, Applied AI, Data Science, or Software Engineering.
• Experience working with Large Language Models (LLMs) and AI-powered applications.
• Knowledge of LLM evaluation techniques, including benchmark datasets, scoring frameworks, and LLM-as-a-Judge approaches.
• Understanding of AI agent behavior across multi-turn conversations and tool-calling workflows.
• Experience with prompt engineering, structured outputs, and prompt optimization.
• Familiarity with AI safety concepts including moderation, prompt injection prevention, jailbreak mitigation, and guardrails.
• Hands‑on experience with AWS cloud services, preferably Amazon Bedrock.
• Experience with Langfuse or similar observability and evaluation platforms for LLM applications.
• Strong programming skills in Python.
• Experience developing REST APIs using FastAPI or similar frameworks.
• Knowledge of software engineering best practices including unit testing, code reviews, and Git‑based development workflows.
• Advanced English communication skills (written and verbal).

Preferred Skills

• Experience developing MCP (Model Context Protocol) servers.
• Hands‑on experience with fine‑tuning LLMs using techniques such as LoRA or full fine‑tuning.
• Familiarity with Small Language Models (SLMs), model distillation, and deployment strategies.
• Exposure to System One models such as Jev and Laya.
• Experience in financial services, fintech, banking, or other regulated industries.
• Knowledge of AI evaluation, observability, and production monitoring best practices.

Posted On 27-10-2026

At Coforge, we hire professionals based solely on their skills and qualifications and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.