Senior LLM Inference Engineer — AI Infrastructure Lead

Bitwise; Laurel (MD); On-site; USD 282,000 - 332,000; Full time

Benefits offered by this job

Top salaries

PTO flexibility 3-5 weeks

11 federal holidays

2 snow days

401(k) match up to 24%

Employer-paid health/dental/vision

Education assistance $5,250

Spot bonuses

Job summary

Bitwise is hiring a Senior Software Engineer – Inference in Laurel, MD to lead evaluation, configuration, and deployment of AI inference models. You will mentor engineers and help grow best practices while building in-house services for reliable high-quality inference.

You will collaborate across teams to establish robust AI infrastructure, ensuring access to top large language models within our inference stack and contributing to the broader AI-enabled ecosystem.

Qualifications

• 12+ years of software engineering experience or a B.S. plus 8+ years
• Proficiency with Python and/or other modern programming languages
• Experience with Argo CD and/or other CI/CD frameworks
• Experience with Kubernetes and Helm
• Experience with AWS or other cloud providers
• Strong communication skills and demonstrated ability to mentor engineers
• Excellent leadership and stakeholder management skills
• Balance hands-on engineering with leadership and coordination responsibilities
• Experience with vLLM, LiteLLM, or similar inference-serving frameworks
• Experience with other LLM hosting frameworks and practices
• Experience with SRE best practices and observability tooling

Responsibilities

• Lead evaluation, configuration, and deployment of inference models
• Mentor engineers and improve development practices
• Develop in-house services to guarantee high-quality inference
• Collaborate across teams to build scalable AI infrastructure
• Balance surge efforts with ongoing delivery and mentorship

Skills

Python

Communication

Leadership

Mentoring

Education

B.S. in Computer Science or related field

Tools

Kubernetes

Docker

Argo CD

CI/CD tooling

AWS

Job description

Bitwise is hiring a Senior Software Engineer – Inference in Laurel, MD to lead evaluation, configuration, and deployment of AI inference models. You will mentor engineers and help grow best practices while building in-house services for reliable high-quality inference.

You will collaborate across teams to establish robust AI infrastructure, ensuring access to top large language models within our inference stack and contributing to the broader AI-enabled ecosystem.