Principal Machine Learning Engineer

Jobtailor; North Carolina; Hybrid; USD 150,000 - 210,000; Full time

Job summary

Jobtailor is seeking an experienced AI Solution Architect to design and deliver secure, scalable AI-powered applications for government-related initiatives. The role focuses on LLMs, RAG, embeddings, and AI agents, with heavy emphasis on architectural decisions, governance, and collaboration across product, security, and data teams.

The candidate will prototype and implement end-to-end AI services, mentor engineers, and drive production-grade solutions aligned with policy and security standards.

Qualifications

• Bachelor’s degree in computer science, engineering, data science, information systems, or equivalent practical experience.
• Significant professional software engineering experience delivering production applications or platforms.
• Hands-on experience developing AI, machine-learning, or data-intensive solutions.
• Proficiency in Python.
• Experience with APIs, cloud services, data pipelines, and source control.
• Experience with modern generative-AI patterns including LLMs, RAG, embeddings, and AI agents.
• Understanding of cloud-native architecture, IAM, observability, and secure development practices.
• Ability to evaluate alternatives and explain architecture decisions and tradeoffs.
• Experience across product, engineering, data, security, and business teams.
• Strong written and verbal communication skills.

Responsibilities

• Design, prototype, and implement AI-powered applications, services, agents, and workflows.
• Translate requirements into practical technical solutions.
• Develop solutions using large language models, retrieval-augmented generation, ML, NLP, CV, and other AI tech.
• Build reusable AI components, services, APIs, evaluation frameworks, and reference implementations.
• Integrate AI with enterprise platforms, data sources, and existing apps.
• Balance rapid experimentation with secure, reliable production engineering.
• Evaluate models and vendors based on performance, cost, security, scalability, and mission fit.
• Provide technical guidance, code reviews, and mentoring to teams.
• Grow into ownership of end-to-end AI solution architecture and broader strategy.

Skills

Python

APIs

Cloud services

Data pipelines

Software development

MLOps

Vector databases

Model evaluation

Prompt engineering

Security practices

Education

Bachelor’s degree in computer science or related field

Tools

AWS

Azure

Google Cloud

Git

IDAM basics

Job description

• Design, prototype, and implement AI-powered applications, services, agents, and workflows
• Translate business, user, and government mission requirements into practical technical solutions
• Develop solutions using large language models, retrieval-augmented generation, machine learning, natural language processing, computer vision, and other AI technologies
• Build reusable AI components, services, APIs, evaluation frameworks, and reference implementations
• Integrate AI capabilities with enterprise platforms, data sources, workflows, and existing applications
• Balance rapid experimentation with secure, reliable production engineering
• Evaluate models, platforms, frameworks, and vendors based on performance, cost, security, scalability, and mission fit
• Contribute to solution architectures spanning applications, models, data, integrations, infrastructure, security, and operational monitoring
• Establish AI architecture patterns, guardrails, standards, and reference architectures with senior architects
• Identify shared AI services and reusable capabilities across products, contracts, and government agencies
• Participate in architecture reviews and document technical decisions, tradeoffs, assumptions, and risks
• Provide technical guidance, code reviews, mentoring, and hands-on support to engineering and data science teams
• Grow into ownership of end-to-end AI solution architecture and broader technical strategy
• Support technical discovery sessions, demonstrations, proofs of concept, proposals, RFIs, and RFP responses
• Communicate AI concepts, limitations, risks, and tradeoffs to technical and non-technical audiences
• Move successful prototypes into secure, supportable, scalable production capabilities
• Stay informed about government AI policies, standards, acquisition practices, and responsible-AI expectations
• Partner with cybersecurity, legal, privacy, compliance, and governance teams throughout the solution lifecycle

Requirements

• Must be a U.S. citizen
• Must successfully pass a background investigation and achieve Public Trust security clearance
• Must be located near Horsham, PA for a hybrid onsite schedule
• Bachelor’s degree in computer science, engineering, data science, information systems, or a related field, or equivalent practical experience
• Significant professional software engineering experience, including experience delivering production applications or platforms
• Hands-on experience developing AI, machine-learning, or data-intensive solutions
• Proficiency in Python
• Experience with APIs, cloud services, data pipelines, software development practices, and source control
• Experience with modern generative-AI patterns, including large language models, retrieval-augmented generation, embeddings, vector search, tool use, structured outputs, or AI agents
• Understanding of cloud-native architecture, system integration, identity and access management, observability, and secure development practices
• Ability to evaluate technical alternatives and explain architecture decisions and tradeoffs
• Experience working across product, engineering, data, security, and business teams
• Strong written and verbal communication skills
• Ability to satisfy applicable government background, suitability, or contractual requirements
• Ability to incorporate security, privacy, accessibility, explainability, auditability, and human oversight into solution designs
• Ability to establish evaluation methods for accuracy, relevance, groundedness, bias, safety, and reliability
• Ability to design controls for model and prompt versioning, data lineage, access management, logging, monitoring, and traceability
• Ability to partner with cybersecurity, legal, privacy, compliance, and governance teams
• Ability to ensure solutions align with organizational policies, contractual obligations, and government requirements
• Ability to design human-in-the-loop controls
• Preferred: Experience delivering technology solutions for U.S. federal, state, or local government customers
• Preferred: Experience in government contracting, regulated environments, or programs involving sensitive data
• Preferred: Familiarity with NIST, FedRAMP, FISMA, or agency-specific controls
• Preferred: Experience with AWS, Microsoft Azure, or Google Cloud
• Preferred: Experience with AI orchestration frameworks, model gateways, vector databases, evaluation platforms, MLOps, or LLMOps
• Preferred: Understanding of model evaluation, prompt engineering, fine-tuning, data governance, and responsible-AI practices
• Preferred: Experience moving AI proofs of concept into production
• Preferred: Experience supporting technical proposals, RFIs, RFPs, solution demonstrations, or customer workshops
• Preferred: Prior technical leadership, mentoring, or architecture-review experience
• Preferred: Relevant cloud, architecture, security, data, or AI certifications

Core Competencies

Demonstrates expertise in designing and implementing AI-powered applications and services, with a strong focus on machine learning, natural language processing, and cloud-native architecture. Proficient in developing secure, scalable solutions while collaborating across technical and non-technical teams.

Highest-signal resume keywords

• AI Application Development
• Machine Learning
• Python Programming
• Cloud Services Integration
• Technical Communication

ATS Optimization Keywords

Hard Skills

• AI Technologies
• Natural Language Processing
• Large Language Models
• Data Pipelines
• APIs
• Software Development Practices
• Model Evaluation
• Prompt Engineering
• Data Governance
• Generative AI Patterns

Soft Skills

• Strong Written Communication
• Strong Verbal Communication
• Mentoring
• Collaboration
• Problem-Solving

Certifications & Qualifications

• Relevant Cloud Certifications
• Architecture Certifications
• Security Certifications
• Data Certifications
• AI Certifications

Industry Keywords

• Government Contracting
• NIST
• FedRAMP
• FISMA
• Sensitive Data Compliance
• Public Trust Security Clearance
• Human-in-the-Loop Controls

Tools & Technologies

• AWS
• Microsoft Azure
• Google Cloud
• MLOps
• AI Orchestration Frameworks
• Vector Databases
• Evaluation Platforms