Senior AI Inference Engineer — TS/SCI Clearance

Bitwise; Laurel (MD); On-site; USD 282,000 - 332,000; Full time

Benefits offered by this job

Top salaries

PTO 3-5 weeks

Federal holidays

Snow days

401(k) match

Healthcare coverage

Education assistance

Spot bonuses

Job summary

Bitwise in Laurel, MD is seeking a Senior Software Engineer – Inference to lead evaluation, configuration, and deployment of inference models, ensuring access to high-quality LLMs within our AI infrastructure. You will mentor engineers, shape best practices, and collaborate across teams to deliver reliable inference services and integrate LLM-powered tools for customer needs.

This role requires 12+ years of software experience or a BS + 8+ years, strong Python, CI/CD, Kubernetes, AWS, and a

Qualifications

• 12+ years of software engineering experience, or a BS + 8+ years in technical field.
• Proficiency with Python and/or other modern programming languages.
• Experience with Argo CD and/or other CI/CD frameworks.
• Experience with Kubernetes and Helm.
• Experience with AWS or other cloud service providers.
• Strong communication skills and mentoring ability.
• Leadership and stakeholder management skills.
• Ability to balance hands-on engineering with leadership responsibilities.
• Experience with vLLM, LiteLLM, or similar inference-serving frameworks.
• Experience with other LLM hosting frameworks and practices.
• Experience with SRE best practices.

Responsibilities

• Lead evaluation, configuration, and deployment of inference models for users.
• Mentor engineers and promote development best practices.
• Develop in-house services to guarantee high-quality inference service.
• Collaborate across teams to integrate LLM-powered tools for user needs.
• Balance hands-on engineering with leadership and coordination.

Skills

Python

Communication

Leadership

Mentoring

SRE

Observability

Docker

Containerization

AI hosting

Cloud AWS

CI/CD

Linguistic models

Education

B.S. in a technical discipline

Tools

Argo CD

Kubernetes

Helm

AWS

Docker

Grafana/Prometheus

Job description

Bitwise in Laurel, MD is seeking a Senior Software Engineer – Inference to lead evaluation, configuration, and deployment of inference models, ensuring access to high-quality LLMs within our AI infrastructure. You will mentor engineers, shape best practices, and collaborate across teams to deliver reliable inference services and integrate LLM-powered tools for customer needs.

This role requires 12+ years of software experience or a BS + 8+ years, strong Python, CI/CD, Kubernetes, AWS, and a