Senior Consultant, AI/ML Engineer

Hollstadt Consulting; Minnesota; On-site; USD 150,000 - 210,000; Full time

Job summary

Hollstadt Consulting seeks a Senior ML/AI Engineer in Minnesota to bridge data science and platform engineering within our AI Center of Excellence. You will train and deploy production ML models, ship GenAI features, and help build the shared AI infrastructure.

You will own model evaluation, RAG pipelines, embeddings, and production-grade Python services on AWS, collaborating across teams to set patterns and mentor others.

Qualifications

• 5+ years building and shipping ML/AI systems in production
• Strong data science fundamentals: feature engineering, metrics, validation
• Experience building ranking, scoring, or survival models
• Hands-on GenAI/LLM engineering: embeddings, vector search, prompts
• Excellent Python for production-grade APIs/services
• AWS experience: Bedrock/SageMaker, Lambda, S3; containers (Docker)

Responsibilities

• Train and evaluate ML models with robust metrics and validation
• Integrate models into decision systems with business rules and LLM reasoning
• Design and deploy LLM-powered features (RAG, agents, extraction, summarization)
• Build and extend AI platform infrastructure, gateways and SDKs
• Own embeddings, vector stores, retrieval quality, grounding strategies
• Develop offline/online evaluation, guardrails, and MLOps tooling
• Productionize pipelines as observable services (Python, containers, AWS)
• Lead technically: set patterns, review designs, mentor engineers

Skills

ML fundamentals

Python

AWS Bedrock/SageMaker

LLM/RAG engineering

Model evaluation

Tools

Docker

Open source ML libs

Job description

The AI CoE builds AI products and the shared platform that powers them. We're looking for a Senior ML / AI Engineer who is equally comfortable building models and building the platform around them: someone

who can train and evaluate an ML model, ship a production LLM/GenAI application, and extend shared AI infrastructure and tooling that the wider team depends on.

This is a senior builder role at the intersection of applied data science, GenAI application engineering, and AI platform engineering. Our work spans predictive modeling, LLM-based systems, and the platform underneath them; you'll take problems from data and prototype through to governed, monitored production services, and set the technical patterns other engineers build on. Projects vary over time — we value engineers who can move across the stack rather than stay in one lane

Performance expectations:

• Build ML models — frame problems, engineer features, train and evaluate models (ranking, scoring, survival/time-to-event, classification, forecasting), and reason rigorously about metrics (AUC, C-index, calibration), validation strategy, subgroup performance, and failure modes.
• Integrate models into decision systems — combine model output with business/domain rules and LLM reasoning to produce explainable, trustworthy recommendations.
• Ship GenAI applications — design and deploy LLM-powered features: RAG pipelines, agents, structured extraction, summarization, decision-reasoning trails, and evaluation harnesses using Claude/Bedrock and other models.
• Engineer the AI platform — extend the shared AI gateway (unified multi-model access, API keys, per-team budgets, failover, observability) and reusable libraries/SDKs that other teams build on.
• Own the RAG/data layer — embeddings, vector stores, retrieval quality, chunking, and grounding strategies; measure and improve retrieval and answer quality.
• Build evaluation & quality tooling — offline/online eval, LLM-as-judge, regression suites, statistical validation, and guardrails so model and prompt changes ship safely.
• Productionize — wrap models and pipelines as tested, observable services (Python, containers, AWS Lambda/SageMaker/EKS), with monitoring for quality, cost, latency, and drift.
• Lead technically — set patterns and standards, review designs and code, mentor engineers, and partner with data scientists, MLOps, clinical/domain experts, and product owners to move prototypes to production.

Required Qualifications

• 5+ years building and shipping ML / AI systems in production (not just notebooks/POCs), including technical leadership of non-trivial projects.
• Strong data science / ML fundamentals — feature engineering, model training and evaluation, metrics (AUC, C-index, calibration, precision/recall), gradient-boosted trees (XGBoost), and sound experimental methodology (validation strategy, subgroup analysis).
• Experience building ranking, scoring, or survival/time-to-event models, and integrating model output into a larger decision system.
• Hands-on GenAI / LLM engineering — RAG, prompt engineering, function/tool calling, embeddings and vector search, and LLM evaluation.
• Excellent Python — production-grade, tested, well-structured code; comfortable building APIs/services and shared libraries.
• AWS experience — Bedrock and/or SageMaker, Lambda, S3, plus containers (Docker) and Git-based workflows.
• Solid software engineering practice: version control, testing, code review, CI/CD; ability to reason about cost, latency, and reliability of AI systems in production.

Preferred Qualifications

• AI platform engineering — building shared gateways/proxies, model routing, multi-tenancy, quota/budget enforcement, or internal AI SDKs.
• Experience with agent frameworks, real-time/voice AI, or streaming inference.
• Vector databases (Qdrant, OpenSearch, pgvector) and retrieval-quality tuning at scale.
• IaC (Terraform), observability (OpenTelemetry/CloudWatch), and FinOps for AI workloads.
• Healthcare / clinical ML — survival analysis, outcome prediction, or working with clinical/scientific datasets alongside domain experts.
• Serving models as endpoints (SageMaker), cross-account inference, and train/serve parity.
• Experience in a regulated / PHI-handling environment (HIPAA) — data governance, PII handling, auditability.