Senior AI Software Engineer & Developer

Powerhouse Institute Inc; Washington (District of Columbia); Remote; USD 155,000 - 189,000; Full time

Job summary

Paylocity is seeking a Senior AI Software Engineer & Developer to lead hands-on development of large-scale AI systems for federal programs. This fully remote role requires strong Python expertise, cloud-native architecture, and strategic technical leadership across multiple programs.

The successful candidate will drive architecture, ML lifecycle, and MLOps practices while coordinating with senior stakeholders.

Qualifications

• 12+ years of software engineering experience with senior leadership and enterprise architecture.
• 8+ years of applied AI/ML experience including production systems.
• Expert-level Python development with ML pipelines and microservices.
• Experience with regulated federal environments and security compliance required.

Responsibilities

• Lead the technical strategy, design, and delivery of large-scale AI systems for federal programs.
• Define AI and application architecture across multiple programs.
• Establish modernization roadmaps aligned to mission outcomes and compliance.
• Architect scalable AI systems, cloud-native and hybrid deployments.
• Drive cross-program technical decisions for interoperability and security.
• Mentor and guide senior engineers and technical leaders.

Skills

Python

AI/ML systems

Cloud-native deployment

Microservices

DevSecOps practices

Distributed training

RESTful APIs

GPU-based infrastructure

Executive communication

Education

BS or MS degree in engineering/data science/computer science/statistics

Tools

LangChain

LlamaIndex

Semantic Kernel

AWS Bedrock

Azure OpenAI Service

Google Vertex AI

Job description

All Jobs > Senior AI Software Engineer & Developer

Fully Remote • Remote - Washington DC Metro Area (DMV), DC

Full-time

Description

NOTE: This opportunity is full-time employment position only (no 1099 or C2C engagements, or third parties or staffing agencies, please). The candidate MUST be a U.S. Citizen and Permanent Resident (Green Card holder). This is a remote opportunity; candidate must be based in the U.S.; have resided in the U.S. for at least 3 years in the past 5 years; ET time zone work schedule.

Daily Responsibilities

• Serve as a senior, hands-on full-stack AI engineer and technical authority, leading the technical strategy, design, and delivery of large-scale mission-critical AI systems supporting federal programs.


• Serve as the primary technical authority, defining AI and application architecture across multiple programs.


• Establish enterprise modernization roadmaps aligned to mission outcomes, compliance, and scalability.


• Lead architecture for distributed, cloud-native, and hybrid AI systems.


• Define and enforce reference architectures, standards, and reusable frameworks.


• Drive cross-program technical decision-making to ensure interoperability, security, and long-term sustainability.


• Advise senior federal stakeholders (SES-level and above) on AI adoption, modernization, and risk management.


• Lead design, development, and deployment of advanced AI solutions using Python as the primary development language, including large language models (LLMs) and foundation models, Retrieval-Augmented Generation (RAG) systems, agentic workflows, and orchestration frameworks.


• Architect and implement scalable ML systems and services built on Python-based frameworks and APIs.


• Build full-stack AI applications end to end, from user-facing interfaces to back-end services, APIs, and data layers.


• Integrate AI and LLM capabilities into existing enterprise applications and legacy platforms (e.g., content management, case management, and records systems) via APIs, middleware, and event-driven patterns.


• Define and implement distributed training strategies (GPU/TPU clusters, parallelization, optimization).


• Oversee full ML lifecycle in partnership with the senior data scientist: data pipelines, feature engineering, training, evaluation, deployment, and monitoring.


• Drive model optimization techniques (quantization, distillation, caching) to improve performance and cost.


• Establish robust MLOps practices leveraging Python-driven automation, pipelines, and tooling.


• Stand up the enterprise CI/CD-to-AI/MLOps pipeline, beginning with time-boxed proofs of concept and MVP implementations that mature into production systems.


• Serve as SME in federal AI policy (e.g., NIST AI RMF, OMB M-25-21 and M-25-22, Executive Order 14179).


• Define and operationalize Responsible AI frameworks, including model validation and evaluation, bias mitigation and fairness, and explainability, auditability, and safety.


• Ensure compliance with FISMA, FedRAMP, NIST 800-53, privacy, and Section 508 requirements.


• Lead large-scale modernization initiatives (e.g., legacy-to-cloud, microservices transformation, including Python-based refactoring and re-platforming efforts).


• Define repeatable modernization frameworks and accelerators.


• Oversee DevSecOps pipelines, CI/CD automation, zero-trust architectures, and secure software supply chain practices.


• Ensure delivery of resilient, high-availability systems in regulated federal environments.


• Lead multiple concurrent engineering efforts across integrated teams.


• Provide technical leadership to architects, engineers, and DevSecOps specialists, including establishing Python coding standards and engineering best practices.


• Mentor senior engineers and technical leaders; elevate engineering excellence and code quality.


• Support technical strategy in proposals, captures, and client engagements.


• Contribute to thought leadership (whitepapers, architecture patterns, platform strategy).


• Executive communication skills with experience influencing senior leaders.

Requirements

• Must of a U.S. Citizen or Permanent Resident (Green Card holder), as mandated by our government client.


• Must be able to complete/pass/hold at a minimum a Public Trust Investigation / background check. An active clearance (e.g. Public Trust, Secret, or higher) is preferred.


• Must be based / reside in the U.S.


• 12+ years of software engineering experience combining senior technical leadership, hands-on expertise in Python-based AI/ML systems (including large language models), and ownership of enterprise architecture, governance, and innovation.


• 8+ years of applied AI/ML experience, including building and deploying production systems (LLMs, generative AI, and large-scale or distributed model systems).


• Expert-level Python development experience, including designing production-grade ML systems, data pipelines, and microservices-based architectures.


• Hands-on experience with ML frameworks (PyTorch, TensorFlow, JAX) and distributed training (DeepSpeed, FSDP, Horovod).


• Full-stack engineering skills, including front-end frameworks, back-end services, RESTful APIs, microservices, and cloud-native deployment (e.g., containers, Kubernetes).


• Deep experience with cloud platforms (Azure, AWS, GCP), including FedRAMP environments and designing AI systems in cloud-native, distributed environments.


• Deep expertise in machine learning and deep learning, particularly transformer-based models and LLMs.


• Hands-on experience with LLM application stacks, including orchestration frameworks (e.g., LangChain, LlamaIndex, Semantic Kernel), embeddings, vector databases, and prompt engineering.


• Experience with AI platforms and architectures (e.g., AWS Bedrock, Azure OpenAI Service, Google Vertex AI, RAG, agents).


• Proven success delivering enterprise-scale systems and modernization programs.


• Strong background in microservices, APIs, distributed systems, and DevSecOps practices.


• Experience managing GPU-based infrastructure or high-performance ML environments.


• Demonstrated ability to translate AI research into production system.


• Proven ability to integrate AI capabilities into existing and legacy enterprise systems (e.g., legacy CMS or COTS platforms) using APIs, middleware, connectors, and event-driven architectures.


• Strong understanding of large-scale data systems and ML evaluation methodologies.


• Experience working with sensitive data, including PII safeguards such as anonymization, masking, and data loss prevention.


• Experience with enterprise integration technologies, including REST/SOAP services, message queues, ETL pipelines, and SQL/NoSQL databases.


• Proficiency with managed generative AI services (e.g., AWS Bedrock, Azure OpenAI Service, Google Vertex AI) and integrating frontier models such as GPT, Claude, and Gemini.


• Demonstrated ability to own solutions end to end — from discovery and prototyping through production deployment, integration, and ongoing support.


• Ability to balance strategic vision with deep hands-on technical execution.


• Excellent analytical skills, attention to detail, and strong problem-solving abilities.


• Excellent communication and collaboration skills to communicate complex analytical insights to executive and non-technical stakeholders. Ability to translate ambiguous business questions into analytical solutions.


• BS or MS degree (preferred) in engineering, data science, computer science, statistics or related field.

The following experience is PREFERRED

• Experience with federal civilian agencies preferred.

Compensation decisions depend on a wide range of factors, including but not limited to skill sets, experience and training, security clearances, licensure and certifications, and other business and organizational needs. $155k -$189k.