Software Engineer, AI Inference - TS/SCI Eligible

Bitwise; Laurel (MD); On-site; USD 198,000 - 233,000; Full time

Benefits offered by this job

Top salaries

PTO 3-5 weeks

Federal holidays paid

Snow days paid

401(k) with company match 24%

Employer-paid health, dental, vision

Education assistance $5,250

Spot bonuses

Job summary

Bitwise is seeking a software engineer to build and maintain AI infrastructure supporting customer AI capabilities. You will ensure access to high-quality LLMs across the inference software stack and collaborate with teams to develop reliable pipelines for closed-source models.

The role emphasizes contributing to mission-focused intelligence solutions, learning new technologies quickly, and working in a culture of technical excellence and growth.

Qualifications

• 3 years of experience in software engineering, OR a B.S. in a technical discipline plus 4 additional years of related experience.
• Experience with Python and/or other modern programming languages.
• Familiarity with Argo CD and/or other CI/CD frameworks.
• Experience with Kubernetes/Helm.
• Familiarity with AWS or other cloud service providers.
• Ability to learn new technologies quickly.
• Strong communication skills and willingness to ask questions.
• Experience with vLLM, LiteLLM, or similar inference-serving frameworks.
• Experience with other LLM hosting frameworks and practices.
• Experience supporting production software using Site Reliability Engineering (SRE) best practices.
• Experience with Elastic, Grafana/Prometheus, or other observability frameworks and practices.
• Experience with Docker and containerization.
• Experience in traffic shaping and quality-of-service engineering.
• Knowledge of and interest in hosting AI capabilities.

Responsibilities

• Procure, configure, and test new inference models, preparing them for release to our user base
• Develop in-house services and techniques to guarantee continual high-quality inference service for our customer
• Work with model vendor teams and representatives to create reliable pipelines for closed-source model usage
• Collaborate with teammates on surge efforts to support short-term, high-priority inference needs from our customer
• Engage with other teams in our organization to establish solid infrastructure for our services and integrate LLM-powered tools for user needs

Skills

Python

Kubernetes

Docker

CI/CD

SRE

AWS

LLM

Observability

Education

B.S. in Computer Science or related

Tools

Argo CD

Job description

Bitwise is seeking a software engineer to build and maintain AI infrastructure supporting customer AI capabilities. You will ensure access to high-quality LLMs across the inference software stack and collaborate with teams to develop reliable pipelines for closed-source models.

The role emphasizes contributing to mission-focused intelligence solutions, learning new technologies quickly, and working in a culture of technical excellence and growth.