LLM Solutions Developer

Open Systems Technologies; Mississauga; On-site; CAD 110,000 - 170,000; Full time

Job summary

Open Systems Technologies seeks an LLM Solutions Developer to design, build, and deploy production-grade LLM-powered solutions in Mississauga. The role focuses on end-to-end architecture, multi-agent orchestration, and safe, scalable AI systems. Contract-to-hire with collaboration across product, data science, and platform teams.

Mandatory if Karat test is cleared, you will work on API development, tool integrations, RAG pipelines, and observability to ensure reliable AI services.

Qualifications

• Hands-on experience building and deploying LLM-based applications in production environments.
• Deep understanding of prompt engineering, few-shot learning, chain-of-thought, and instruction tuning.
• Experience with agentic AI frameworks such as LangChain, LangGraph, AutoGen, CrewAI, or similar.
• Familiarity with MCP and context window management strategies.
• Experience implementing guardrails for LLM outputs (content filtering, hallucination mitigation, policy enforcement).
• Knowledge of RAG architectures, vector search, and embedding pipelines.

Responsibilities

• Design & develop end-to-end LLM-based solutions for production use.
• Build autonomous and semi-autonomous AI agents with multi-step reasoning and tool use.
• Architect LLM orchestration pipelines and manage memory/context handling.
• Implement MCP-compliant architectures for structured model communication.
• Integrate guardrails to ensure safety, accuracy, and policy compliance.
• Develop RESTful or gRPC APIs for LLM-powered services; integrate with external systems.
• Create tools, plugins, and function-calling integrations to extend capabilities.
• Develop RAG pipelines using vector databases and embeddings.
• Establish observability with tracing and monitoring dashboards.

Skills

LLM Development

Prompt Engineering

Agentic AI frameworks

MCP Context Protocol

Guardrails & Safety

RAG & Embeddings

Job description

We are looking for LLM Solutions Developer for a contract to Hire role in Mississauga.

Its mandatory if you clear Karat test.

We are looking for a talented and forward-thinking LLM Solutions Developer to join our AI Engineering team. In this role, you will design, build, and deploy production-grade solutions powered by Large Language Models (LLMs). You will work at the intersection of cutting-edge AI research and real-world software engineering, delivering intelligent, reliable, and scalable agentic systems.

Key Responsibilities

• Design & Develop LLM-Based Solutions: Architect and implement end-to-end applications leveraging LLMs (e.g., GPT-4, Claude, Gemini, Llama) for tasks such as reasoning, summarization, code generation, and decision support.


• Agentic AI Systems: Build autonomous and semi-autonomous AI agents capable of multi-step reasoning, tool use, and goal-directed behavior using frameworks such as LangGraph, AutoGen, CrewAI, or custom implementations.


• Orchestration: Design and manage complex LLM orchestration pipelines, including multi-agent workflows, task routing, memory management, and context handling.


• Model Context Protocol (MCP): Implement and integrate MCP-compliant architectures to enable structured, context-aware communication between models, tools, and external systems.


• Guardrails & Safety: Integrate guardrail frameworks (e.g., NeMo Guardrails, Guardrails AI, custom rule engines) to enforce output safety, factual accuracy, policy compliance, and ethical AI standards.


• API Development & Integration: Design and expose RESTful or gRPC APIs for LLM-powered services; integrate with third-party APIs, enterprise systems, and data sources.


• Tool & Plugin Development: Build custom tools, plugins, and function-calling integrations that extend LLM capabilities (e.g., web search, database queries, code execution, document retrieval).


• RAG Pipelines: Develop Retrieval-Augmented Generation (RAG) systems using vector databases (e.g., Pinecone, Weaviate, pgvector) and embedding models.


• Evaluation & Observability: Implement LLM evaluation frameworks, tracing (e.g., LangSmith, OpenTelemetry), and monitoring dashboards to ensure quality, performance, and reliability.


• Collaboration: Work closely with product managers, data scientists, and platform engineers to translate business requirements into robust AI solutions.


• Documentation: Produce clear technical documentation, architecture diagrams, and runbooks for all developed systems.

Required Skills & Experience

Core LLM & AI

• Hands-on experience building and deploying LLM-based applications in production environments


• Deep understanding of prompt engineering, few-shot learning, chain-of-thought, and instruction tuning


• Experience with agentic AI frameworks (LangChain, LangGraph, AutoGen, CrewAI, Semantic Kernel, or similar)


• Familiarity with Model Context Protocol (MCP) and context window management strategies


• Experience implementing guardrails for LLM outputs (content filtering, hallucination mitigation, policy enforcement)


• Knowledge of RAG architectures, vector search, and embedding pipelines

Orchestration & Infrastructure

• Experience designing multi-agent orchestration workflows and task delegation patterns


• Proficiency with API design and development (REST, GraphQL, or gRPC)


• Familiarity with tool/function calling patterns in LLM APIs (OpenAI function calling, Anthropic tool use, etc.)


• Experience with cloud platforms (AWS, GCP, or Azure) and containerization (Docker, Kubernetes)

Software Engineering

• Experience with asynchronous programming, microservices, and event-driven architectures


• Solid understanding of software design patterns, clean code principles, and test-driven development


• Version control with Git and CI/CD pipeline experience