AI Engineer – Applied AI, Data & LLM Systems

247 Labs Inc.; Toronto; On-site; CAD 120,000 - 180,000; Full time

Job summary

247 Labs Inc. in Toronto is seeking a hands-on AI Engineer to design, develop, deploy, and maintain production AI solutions for clients. You will work across the full AI development lifecycle, from problem analysis to model selection and deployment.

The role emphasizes data engineering, retrieval systems, and scalable backend services using Python, FastAPI, and cloud platforms. You will collaborate with cross-functional teams to deliver reliable, high-quality AI applications.

Qualifications

• 4+ years of professional experience in software, data, ML, or AI engineering.
• Strong Python programming skills, including API development and debugging.
• Strong SQL knowledge and experience with relational databases.
• Experience with Git, Docker, CI/CD, and at least one major cloud platform.

Responsibilities

• AI solution architecture and development: design end-to-end AI applications with LLMs, ML models, and agentic workflows.
• Data engineering and integration: build ingestion/ETL pipelines, work with relational databases and APIs.
• Retrieval and knowledge systems: develop RAG pipelines with embeddings and indexing.
• AI evaluation and optimization: create automated evaluation frameworks and baselines.
• Backend engineering and production deployment: develop Python services, REST APIs, CI/CD, and cloud deployments.
• Client collaboration and technical ownership: translate client goals into architecture and plans, communicate trade-offs.

Skills

Python

API development

Async programming

ML / AI engineering

Data pipelines

Problem solving

Tools

Docker

Git

CI/CD

FastAPI

AWS/Azure/GCP

LangChain

Vector databases

Job description

Company Overview:

247labs is a leading software development company headquartered in Toronto, Canada. We specialize in providing custom software solutions, mobile app development, web development, and digital transformation services to clients across various industries. Our team of experienced professionals is dedicated to delivering innovative, high-quality solutions that drive business growth and exceed client expectations.

• Toronto, ON

Position Overview:

About the role
247 Labs is looking for a hands-on AI Engineer to design, develop, deploy, and maintain production AI solutions for our clients.

We are looking for a software engineer with strong AI and data engineering expertise, not someone whose experience is limited to prompt engineering, chatbot configuration, or AI-assisted development.
The successful candidate will understand how AI models work, how to prepare and integrate data, and how to build reliable, scalable applications around machine learning and large language models.

You will work across the complete AI development lifecycle, from analyzing a client’s business problem and data sources to selecting models, designing architectures, implementing services, evaluating results, and deploying solutions to production.

Key responsibilities

• 1. AI solution architecture and development
  Design and implement end-to-end AI applications using LLMs, machine learning models, retrieval systems, and agentic workflows.
  Evaluate business requirements and determine when to use deterministic logic, traditional ML, RAG, fine-tuning, or agent-based architectures.
  Integrate commercial and open-source AI models through APIs or model-serving infrastructure.
  Implement tool calling, structured outputs, state management, and multi-step AI workflows.
  Select appropriate AI frameworks and technologies based on scalability, reliability, maintainability, and cost.
• 2. Data engineering and integration
  Design and develop ingestion and transformation pipelines for structured, semi-structured, and unstructured data.
  Work with relational databases, APIs, documents, object storage, and external enterprise systems.
  Use Python and SQL to extract, clean, validate, transform, and prepare data for AI applications.
  Implement reliable ETL/ELT workflows, incremental synchronization, data validation, and error recovery.
  Design data storage, metadata management, indexing, and access-control strategies.
• 3. Retrieval and knowledge systems
  Build and optimize RAG pipelines, including document processing, chunking, embeddings, indexing, retrieval, reranking, and generation.
  Implement semantic, keyword, and hybrid search.
  Evaluate retrieval performance and generated-answer quality using measurable metrics.
  Maintain accurate, current, permission-aware knowledge bases with document citations and source traceability.
• 4. AI evaluation and optimization
  Develop automated evaluation frameworks and representative test datasets.
  Evaluate model outputs, retrieval relevance, grounding, tool execution, and end-to-end task completion.
  Establish baselines and regression tests to measure improvements and detect failures.
  Optimize AI solutions for quality, latency, reliability, and operational cost.
  Investigate production issues using traces, metrics, logs, and systematic error analysis.
• 5. Backend engineering and production deployment
  Develop Python services and REST APIs using FastAPI or equivalent frameworks.
  Implement robust integrations with databases, client applications, and third-party systems.
  Build automated tests, CI/CD pipelines, and containerized services.
  Deploy, monitor, and troubleshoot AI applications on AWS, Azure, or GCP.
  Implement appropriate caching, retries, timeouts, concurrency controls, and background processing.
  Apply secure software engineering practices, including authentication, authorization, secrets management, and data protection.
• 6. Client collaboration and technical ownership
  Participate in technical discovery and assess client data readiness and integration requirements.
  Translate business goals into AI architecture, implementation plans, and measurable acceptance criteria.
  Explain architectural decisions, technical limitations, risks, and costs to clients and project managers.
  Develop technical proofs of concept, demonstrate solutions, and evolve successful prototypes into production systems.
  Collaborate with backend, frontend, QA, and DevOps engineers.

Must-have qualifications

Software and backend engineering

• 4+ years of professional experience in software, data, ML, or AI engineering, including meaningful hands-on production LLM development.
• Strong Python programming skills, including API development, asynchronous programming, automated testing, and debugging.
• Strong SQL knowledge and experience with relational databases.
• Experience with Git, Docker, CI/CD, and at least one major cloud platform.

Data engineering

• Practical experience designing and maintaining data ingestion and transformation pipelines.
• Experience integrating databases, REST APIs, and structured and unstructured data sources.
• Understanding of data quality, schema design, storage, indexing, and access control.

AI and machine learning

• Solid understanding of modern ML and LLM fundamentals, including embeddings, transformers, tokens, context windows, inference, and model limitations.
• Experience integrating AI models into real applications using model APIs or inference services.
• Ability to explain and evaluate architectural trade-offs between prompting, RAG, fine-tuning, agents, and conventional software.

LLM and retrieval engineering

• Hands-on experience building and deploying at least one substantive AI system used in a real production environment.
• Experience developing RAG pipelines using vector databases or search engines.
• Experience with tool calling, workflow orchestration, structured outputs, or agent frameworks.
• Understanding of retrieval strategies and techniques to improve answer accuracy and grounding.

AI evaluation and operations

• Experience developing evaluation datasets, automated tests, and measurable AI quality metrics.
• Experience troubleshooting LLM applications using logs, traces, and performance metrics.
• Understanding of AI security risks, including prompt injection, sensitive data exposure, and excessive tool permissions.

Communication

• Ability to explain technical decisions to clients and non-technical stakeholders.
• Ability to independently investigate complex engineering problems and take ownership of solutions from design to production.

Nice-to-have qualifications

• Experience with LangGraph, LangChain, Semantic Kernel, or equivalent AI orchestration technologies.
• Experience with model evaluation frameworks such as LangSmith, Ragas, DeepEval, or Langfuse.
• Experience with data orchestration tools such as Airflow, Dagster, or Prefect.
• Knowledge of Hugging Face, PyTorch, and model fine-tuning techniques such as LoRA/PEFT.
• Experience deploying open-source models using vLLM or comparable serving technologies.
• Experience with streaming data, distributed processing, or scalable data infrastructure.
• Experience implementing multi-tenant enterprise AI applications.
• Experience developing secure AI solutions for healthcare, finance, insurance, or other regulated industries.
• Experience working with multiple clients in a consultancy, agency, or software delivery environment.

What success looks like

• An effective AI Engineer at 247 Labs can independently take a client use case from discovery to production, including:
  Understanding the business problem and defining measurable success criteria.
  Identifying, accessing, preparing, and validating the required data.
  Selecting an appropriate AI architecture and technology stack.
  Implementing a secure, maintainable, and testable solution.
  Evaluating solution quality against an agreed baseline.
  Deploying and monitoring the solution in a real production environment.
  Explaining the solution, its limitations, and its operational costs to the client.

We value practical engineering experience, strong technical fundamentals, and demonstrated problem-solving ability over familiarity with a long list of AI frameworks.