Lead – AI Optimization

Franklin Templeton; San Ramon (CA); Hybrid; USD 156,000 - 183,000; Full time

Benefits offered by this job

Annual discretionary bonus

401(k) match

Health insurance

Stock investment program

LEAP education assistance

Generous PTO and leave options

Job summary

Franklin Templeton is seeking a Lead - AI Optimization to drive technical delivery of AI solutions across business groups. You will translate needs into practical designs, guide testing, deployment, and production readiness while mentoring engineers and sharing reusable components.

The role focuses on building evaluation capabilities for LLMs and AI apps, establishing benchmarks, and guiding model selection with emphasis on quality, reliability, latency, and cost.

Qualifications

• Bachelor's degree or equivalent practical experience in a technical field and 6+ years in applied ML/AI engineering.
• Experience leading through ambiguity with cross-functional partners and mentoring engineers.
• Strong hands-on Python development, design, testing, and production-minded coding.
• Experience evaluating LLMs and AI applications using datasets and automated tests.
• Proven ability to design experiments, analyze results, and drive decisions.

Responsibilities

• Translate business needs into practical AI system designs and lead delivery.
• Develop reusable evaluation tools and benchmarks for LLMs and AI apps.
• Improve quality, latency, and cost through prompting and retrieval optimization.
• Run structured experiments and A/B tests with clear hypotheses and guardrails.
• Communicate trade-offs, decisions, and risks to stakeholders.

Skills

Python development

LLM evaluation

prompt optimization

retrieval optimization

communication

technical leadership

Education

Bachelor's degree in CS/Engineering or quantitative field

Advanced degree preferred

Tools

Python

LLM frameworks

Vector tooling

Model serving systems

Job description

At Franklin Templeton, we believe success is built through powerful partnerships. As a forward thinking asset manager, we build dynamic relationships with clients, understand their goals, and navigate complex markets together. We leverage cutting edge strategies and deep insights to unlock opportunities for long term wealth creation. Our talented, global teams bring expertise that is both broad and unique.

From our welcoming, inclusive, and supportive culture to our globally diverse business, we offer opportunities not only to help you reach your potential, but also to contribute to our clients’ success.

About The Department

This role sits within Franklin Templeton's AI Platform team, which develops and supports shared AI tools and capabilities across investment, distribution, and corporate functions. The team partners with business stakeholders, product owners, engineers, and data and governance specialists to deliver practical AI solutions responsibly.

The team is hands-on and cross-functional, with a collaborative approach to solving business and technical problems. Team members are expected to challenge assumptions constructively, make sound engineering decisions, share knowledge, and take ownership of delivering solutions that are useful, reliable, and maintainable.

Job Overview

The Lead - AI Optimization drives the technical delivery and continuous improvement of AI solutions across business groups. Working with engineers, data scientists, product owners, and business stakeholders, the role translates business needs into practical technical approaches and guides initiatives from discovery and solution design through implementation, testing, and production readiness.

Evaluation is a core responsibility: the role leads the development and improvement of reusable evaluation capabilities for large language models (LLMs) and AI applications. It establishes meaningful benchmarks, helps teams understand performance and limitations, and uses the findings to guide model selection and improvements in quality, reliability, latency, and cost.

This is a hands-on individual‑contributor role with no people‑management responsibilities. Leadership is exercised through technical ownership, direct contribution, mentoring, and influence. Clear communication is essential to aligning teams, explaining technical trade‑offs, and keeping stakeholders informed of decisions, progress, and risks.

How You Will Add Value

• Lead technical delivery: Translate business needs into practical designs, assessing feasibility, dependencies, and AI‑assisted versus traditional approaches. Build critical components, review code, and guide initiatives through testing, deployment, and production readiness.
• Own the evaluation framework: Develop reusable tools for LLM and application evaluation, including benchmarks, representative datasets, scoring methods, and regression tests. Combine automated checks, LLM‑as‑judge methods, and human review to assess accuracy, grounding, safety, and reliability.
• Optimize quality and performance: Apply advanced prompting and improve retrieval, context construction, model selection, routing, and serving approaches. Use appropriate tuning, caching, and batching to improve quality and throughput while reducing latency and unit cost.
• Validate improvements with evidence: Run structured experiments and A/B tests with clear hypotheses, guardrails, and rollback criteria. Compare models, techniques, and vendors against internal benchmarks, document limitations, and use findings to prioritize changes.
• Align teams and communicate clearly: Partner across business and technical teams to resolve dependencies and meet enterprise standards. Explain trade‑offs and evaluation results, document decisions, and communicate progress, risks, and next steps.
• Scale effective practices: Mentor engineers and share reusable components, evaluation tooling, and practical guidance. Apply AI where it improves testing, code review, documentation, and monitoring, and measure the benefits.

Technical Environment

• Core stack: Python, LLM and machine‑learning frameworks, evaluation and experimentation libraries, and retrieval and vector tooling for AI applications.
• Platforms: Cloud or hybrid model platforms, approved LLM endpoints, model gateways and registries, and GPU or accelerated compute where required by the solution.
• Engineering workflow: Version‑controlled code and experiments, automated tests, reproducible evaluations, continuous integration, and staged deployment.
• Quality and reliability: Application and model monitoring, quality benchmarks, latency and cost metrics, regression detection, safety guardrails, and feedback from production use.

What Will Help You Be Successful In This Role

• Bachelor's degree in Computer Science, Engineering, or a quantitative discipline, or equivalent practical experience, and six or more years in applied machine learning or AI engineering, including production LLM or model‑serving systems. An advanced degree is preferred.
• Demonstrated technical leadership from ambiguous requirements through delivery, with the ability to align cross‑functional partners, mentor engineers, and balance new initiatives with ongoing optimization without formal management authority.
• Strong hands‑on Python development and software‑engineering judgement, including solution design, code review, testing, troubleshooting, and building maintainable production components.
• Hands‑on experience evaluating LLMs and AI applications using representative datasets, scoring methods, automated tests, and failure analysis. Familiarity with LLM‑as‑judge methods and human evaluation, including validating results against expert judgment and using findings to compare models and identify regressions.
• Strong experience in prompt and retrieval optimization and model performance tuning, with practical evidence of improving quality, latency, reliability, and cost in production systems.
• Rigorous experimental discipline and statistical judgement, including designing meaningful comparisons, accounting for variability in model outputs, interpreting results, and recognizing limitations.
• Strong written and verbal communication: explain technical concepts and trade‑offs to nonspecialists, present clear recommendations, document decisions, and communicate risks and dependencies early.

Nice To Have Skills

• Experience in asset management, wealth management, or other financial‑services AI applications.
• Experience with fine‑tuning, preference optimization, quantization, or distillation of production models.
• Experience with GPU performance engineering or inference‑server optimization.
• Experience with LLM observability tooling, production drift detection, or evaluation practices in regulated environments.

Work Schedule & Locations

• This position will work a hybrid schedule, in the office 3 days/week in either our New York City or San Ramon, CA office.

Franklin Templeton offers employees a competitive and valuable range of total rewards – monetary and non‑monetary – designed to support their well‑being and recognize their time, talents, and results. Along with base compensation, employees are eligible for an annual discretionary bonus, a 401(k) plan with a generous match, and recognition rewards. We also offer a comprehensive benefits package, which includes a range of competitive healthcare options, insurance, and disability benefits, employee stock investment program, learning resources, career development programs, reimbursement for certain education expenses, paid time off (vacation / holidays / sick / leave / parental & caregiving leave / bereavement / volunteering / floating holidays) and a motivational wellbeing program. We expect the annual salary for this position to range between $155,700 – $183,200, depending on location and level of relevant experience, plus discretionary bonus.

#MID_SENIOR_LEVEL

At Franklin Templeton, we believe your benefits should support your life, your goals, and your future. That’s why we offer a comprehensive Total Rewards package designed to help you thrive both personally and professionally.

Highlights Of Our Benefits Include

• Paid Time Off: Three weeks of PTO in your first year
• Health Coverage: Competitive medical, dental, and vision insurance to support your well‑being
• Retirement Savings: 401(k) plan with an 85% company match on pre‑tax and/or Roth contributions, up to IRS limits
• Equity & Investing: Employee Stock Investment Plan (ESIP) with discounted share purchase opportunities
• Learning Education Assistance Program (LEAP): To support your ongoing growth and career advancement
• Employee Investment Benefits: Opportunity to purchase company funds with no sales charge

Franklin Templeton is an Equal Opportunity Employer. We are committed to providing equal employment opportunities to all applicants and employees, and we evaluate qualified applicants without regard to ancestry, age, color, disability, genetic information, gender, gender identity, or gender expression, marital status, medical condition, military or veteran status, national origin, race, religion, sex, sexual orientation, and any other basis protected by federal, state, or local law, ordinance, or regulation.

Artificial intelligence or automated tools may be used to assist in the screening, assessment, or selection of applicants for this position.