Junior Software Engineer – Inference

Neuralsolutions; Columbia, Northern (MD, KY); Hybrid; USD 138,000 - 163,000; Full time

Job summary

Neuralsolutions in Columbia, MD, is seeking a Software Engineer focused on AI infrastructure to build and maintain the foundation for customer AI capabilities.

You will work on configuring inference models, developing in-house services, and creating reliable pipelines with vendor teams, while collaborating across teams to enable LLM-powered tools for users. A TS/SCI with Polygraph clearance is required.

Qualifications

• Bachelor's degree in a technical discipline with 3+ years experience.
• 7+ years of experience without a degree is also acceptable.

Responsibilities

• Procure, configure, and test new inference models, preparing them for release to our user base.
• Develop in-house services and techniques to guarantee continual high-quality inference service for our customer.
• Work with model vendor teams to create reliable pipelines for closed-source model usage.
• Collaborate with teammates on surge efforts to support short-term, high-priority inference needs from our customer.
• Engage with other teams to establish solid infrastructure for services and integrate LLM-powered tools for user needs.

Skills

Python

CI/CD

Kubernetes

AWS

Adaptability

Communication

Education

Bachelor's degree in a technical discipline

Tools

Docker

Grafana/Prometheus

vLLM

LiteLLM

Job description

Software Engineer - AI Infrastructure

We’re seeking a software engineer to support our AI infrastructure team in Columbia, MD. In this role, you’ll help build and maintain the foundation for customer AI capabilities while supporting a broader ecosystem of AI-enabled applications.

Responsibilities:

• Procure, configure, and test new inference models, preparing them for release to our user base.
• Develop in-house services and techniques to guarantee continual high-quality inference service for our customer.
• Work with model vendor teams and representatives to create reliable pipelines for closed-source model usage.
• Collaborate with teammates on surge efforts to support short-term, high-priority inference needs from our customer.
• Engage with other teams in our organization to establish solid infrastructure for our services and integrate LLM-powered tools for user needs.

Required Skills:

• Experience with Python and/or other modern programming languages.
• Familiarity with Argo CD and/or other CI/CD frameworks.
• Experience with Kubernetes/Helm.
• Familiarity with AWS or other cloud service providers.
• Ability to learn new technologies quickly.
• Strong communication skills and willingness to ask questions.

Nice to Have:

• Experience with vLLM, LiteLLM, or similar inference-serving frameworks.
• Experience with other LLM hosting frameworks and practices.
• Experience supporting production software using Site Reliability Engineering (SRE) best practices.
• Experience with Elastic, Grafana/Prometheus, or other observability frameworks and practices.
• Experience with Docker and containerization.
• Experience in traffic shaping and quality-of-service engineering.
• Knowledge of and interest in hosting AI capabilities.

Experience Required: 3 years with Bachelor's degree in a technical discipline or 7 additional years without degree

Location: Columbia, MD

Clearance: TS/SCI with Polygraph required

Salary Range: $138,000 - $163,000