Lead Software Engineer

Caseware; Colombia; On-site; COP 120,000,000 - 180,000,000; Full time

Job summary

Caseware is seeking an experienced AI Platform Engineer to own the end-to-end AI production stack, including architecture, deployment, operations, cost, and quality. You will design and operate LLM services, retrieval pipelines, multi-agent orchestration, and agent execution runtimes with on-call accountability.

You will build memory infrastructure, LLMOps, and automated evaluation to monitor drift and performance.

Qualifications

• 2+ years building and operating production AI systems.
• Experience with agent frameworks, agent memory systems, or orchestration of tool-using AI systems.
• Production AWS experience, including Infrastructure as Code (CDK, CloudFormation, or Terraform).
• Degree in Computer Science, Software Engineering, or equivalent practical experience; 8+ years of professional software development experience with demonstrated impact beyond a single team.
• Proven experience designing and operating distributed, cloud-native, SaaS/multi-tenant platforms at scale, ideally on AWS.

Responsibilities

• Create architecture, prototypes, and design proposals that deliver near-term business value while aligning with the future-state product and platform vision.
• Influence technical direction through architecture reviews, RFCs, design discussions, and hands-on collaboration.
• Partner with senior developers, architects, and tech leads to define and execute the product/platform architecture roadmap.
• Balance delivery speed, quality, maintainability, and long-term platform health when making technical decisions.
• Troubleshoot and eliminate root causes of persistent production issues; drive reliability, observability, monitoring, and incident response.
• Mentor other developers through design reviews, joint agentic-AI sessions, and feedback on skills, prompts, and other agent inputs; contribute to raising overall engineering maturity.

Job description

• You build it, you run it. End-to-end accountability for the AI platform in production: architecture, delivery, operations, cost, and quality, with on-call for what you ship. No hand-off to a separate ops or QA function.
• Design, build, and operate agentic AI systems: LLM services, retrieval pipelines, multi-agent orchestration, agent execution runtimes, and human-in-the-loop capabilities.
• Build and operate memory infrastructure for agent systems (storage, retrieval, promotion/demotion mechanics). Applied Science owns promotion criteria and scientific soundness.
• Own LLMOps: the harnesses, pipelines, and infrastructure that keep non-deterministic systems reliable and correct in production, including dynamic model selection by task, cost, and latency.
• Build automated eval infrastructure that continuously compares offline and online metrics and alerts on drift or regression. Applied Science owns the methodology and acceptance thresholds it checks against.
• Evaluate, adopt, and integrate third-party LLMOps and eval platforms where they accelerate delivery over building in-house.
• Build the technical controls, telemetry, and guardrails that enforce compliance frameworks (e.g., ISO 42001, AIUC-1) in the architecture itself.
• Build and maintain the agent definition management APIs and the tool registry definitions reference, keeping definitions stable as tools and models version independently.
• Lead the engineering build of proof-of-concepts. Applied Science defines the architecture bet and validation criteria being tested.

Core Responsibilities

• Create architecture, prototypes, and design proposals that deliver near-term business value while aligning with the future-state product and platform vision.
• Influence technical direction through architecture reviews, RFCs, design discussions, and hands‑on collaboration.
• Partner with senior developers, architects, and tech leads to define and execute the product/platform architecture roadmap.
• Balance delivery speed, quality, maintainability, and long-term platform health when making technical decisions.
• Break down large initiatives into parallelizable chunks of work that deliver incremental business value.
• Champion proposals through all phases of the SDLC, from design through production implementation.
• Contribute to improving the core product build, CI/CD pipelines, and overall SDLC.
• Troubleshoot and eliminate root causes of persistent production issues; drive reliability, observability, monitoring, and incident response.
• Keep technical documentation current and create new artifacts as needed.
• Facilitate design discussions within and across teams.
• Represent the team in technical discussions with Tech Leads, Operations, Product/UX, Security, Domain SMEs, and external stakeholders.
• Act as a trusted technical leader, helping teams succeed through collaboration, influence, and shared ownership.
• Mentor other developers through design reviews, joint agentic-AI sessions, and feedback on skills, prompts, and other agent inputs; contribute to raising overall engineering maturity.
• Share knowledge of new technologies and industry best practices.
• Evaluate emerging technologies, frameworks, and cloud capabilities that could benefit the platform.

AI-Augmented Development Expectations:

• Use AI‑assisted development tools — inline completion, conversational assistants, and agentic workflows — as a standard part of your daily workflow.
• Critically evaluate and take ownership of all AI‑generated output before it ships — verify correctness, security, and alignment with architecture and team standards.
• Identify opportunities for agentic automation within your team, and lead the rollout of new AI‑assisted workflows and tooling.
• Mentor other developers on effective, responsible use of AI‑assisted development tools.
• Help establish team‑level practices and guardrails for using AI tools safely and effectively.
• Build and refine feedback loops that improve the team's collective use of AI tooling over time.

What You Will Bring:

• 2+ years building and operating production AI systems, with a working grasp of the latency, cost, accuracy, and reliability trade‑offs involved.
• Experience with agent frameworks, agent memory systems, or orchestration of tool‑using AI systems.
• Production AWS experience, including Infrastructure as Code (CDK, CloudFormation, or Terraform).
• Degree in Computer Science, Software Engineering, or equivalent practical experience; 8+ years of professional software development experience with demonstrated impact beyond a single team.
• Proven experience designing and operating distributed, cloud‑native, SaaS/multi‑tenant platforms at scale, ideally on AWS.
• Strong software fundamentals: OOP, design patterns, SOLID principles, and a track record of advocating for code quality.
• Experience with agentic‑AI software development practices.
• Experience designing and consuming well‑designed HTTP APIs (REST, GraphQL, or similar).
• Experience mentoring developers and influencing architectural decisions across teams.
• Strong written and verbal communication skills; comfortable operating in fast‑moving environments with ambiguity and evolving requirements.
• Strong initiative to improve processes, tools, methodologies, and product quality.

Nice to Have

• Experience implementing AI guardrails, governance controls, and safety mechanisms, and translating compliance frameworks (e.g., ISO 42001, AIUC-1, NIST AI RMF) into technical controls.
• Experience evaluating and integrating third-­party ML/LLMOps platforms, with sound buy vs. build judgment.
• Experience with retrieval systems (RAG), embedding pipelines, or hybrid search (vector + keyword).
• LLMOps experience: evaluation harnesses, automated offline/online metric comparison, drift and regression alerting.
• Experience operating systems in regulated or compliance-heavy domains.
• Familiarity with accounting, auditing, or financial workflows.
• Reliability & Observability: New Relic, CloudWatch, Prometheus, OpenTelemetry or equivalent
• Automation & Scripting: Python, Bash, TypeScript or equivalent
• Incident & Operations: runbooks, alerting workflows, incident management tools, post‑incident review practices
• Tooling: GitHub, GitHub Actions, Jira, Confluence, Microsoft Teams, Slack
• ¨Contrato a termino Indefinido¨ with all the legal benefits
• Life insurance and funeral assistance
• Home office stipend
• Competitive compensation — above the market average
• 100% remote work environment and an excellent work‑life balance
• 5 Personal Time Off days per year
• Sick Leave Top up to total 100% of salary paid by the employer from Day 3 to 90.
• Recognition Award, additional paid time off in recognition of the corresponding year of service
• Upgrade vacation starting at 5 years of service
• Opportunity to work for a growing global SaaS leader company
• A culture that promotes independence, innovation, trust, and accountability
• Open space to be creative, innovative and strategize for the future
• Mentorship by highly experienced professional
• Budget for training, we want you to grow
• AI‑first environment: Be part of an AI‑first engineering organization that embraces modern tools, automation, and AI‑driven ways of working.