Data Scientist

Datavault AI Inc.; Atlanta (GA); Hybrid; USD 110,000 - 170,000; Full time

Benefits offered by this job

Competitive salary & benefits

Fast-paced environment

Direct exposure to leadership

Cutting-edge technologies

Culture of innovation

Job summary

Datavault AI Inc. in Atlanta, GA, is seeking a Data Scientist to lead evaluation strategies for multi-agent AI systems in a hybrid, full-time role.

You’ll own ground-truth corpus design, statistical rigor, and the metrics that make AI outputs meaningful and measurable across industry verticals.

Qualifications

• Bachelor’s degree in Computer Science, Statistics, Data Science, Machine Learning, or related quantitative discipline, or equivalent professional experience. Master’s or PhD preferred.
• 3+ years of professional data science, ML engineering, or AI evaluation experience shipping models or AI systems to production.
• Strong Python skills (Pandas, NumPy, scikit-learn, PyTorch or TensorFlow), with comfort writing production-quality code that engineers will run in CI.
• Proven experience evaluating LLM-based systems: prompt experimentation, structured-output validation, hallucination detection, retrieval evaluation, judge-LLM patterns.
• Solid grasp of classical statistics: hypothesis testing, confidence intervals, sample-size calculation, power analysis, calibration.
• SQL proficiency for ad-hoc analysis on PostgreSQL; comfortable with embedded analytical databases for offline evaluation.
• Experience with vector databases and embedding models.
• Ability to translate business goals into measurable evaluation criteria, and willingness to push back when a “metric” doesn’t measure what stakeholders think.

Responsibilities

• Design, build, and maintain evaluation frameworks for our multi-agent LLM pipelines covering classification, PII detection, valuation, retrieval, segmentation, and synthesis — with regression-detection rigor.
• Curate, expand, and version synthetic and real-world test corpora that exercise our AI pipelines end-to-end across 20+ industry verticals.
• Quantify model performance: precision, recall, calibration, inter-rater agreement against human-verified ground truth, drift detection across releases.
• Partner with engineering to design prompt experiments, agent variants, and structured-output schema iterations; report results with statistical confidence intervals — not anecdotes.
• Improve vector-search comparable retrieval: embedding model selection, retrieval evaluation (recall@k, MRR), taxonomy refinement, classification accuracy uplift.
• Evaluate prompt strategies, tool-use patterns, and routing logic; recommend model-tier choices backed by cost/accuracy data.
• Profile production traces to identify failure modes (hallucinated outputs, mis-classifications, missed PII), then design experiments to fix them.
• Work cross-functionally with engineering, product, and domain experts to translate fuzzy product goals (“the analysis should feel insightful”) into quantitative success metrics.
• Communicate findings through written reports, dashboards, and decision memos that executive leadership can act on.

Education

Bachelor’s degree in Computer Science, Statistics, Data Science, Machine Learning, or related quantitative discipline, or equivalent professional experience

Master’s or PhD preferred

Tools

Python (Pandas, NumPy, scikit-learn)

PyTorch or TensorFlow

SQL (PostgreSQL)

Vector databases & embeddings

CI/CD for production code

Job description

Data Scientist

Location: Atlanta, GA Hybrid

Employment Type: Full-Time

About Us: Datavault AI, along with its event-technology subsidiary Event Citadel (formerly CompuSystems), operates across a diverse portfolio of technology and service divisions.

Datavault AI Inc. delivers high-performance computing software, Web 3.0 data-management solutions, and advanced audio technologies to a broad range of industries.

Event Citadel (formerly CompuSystems), founded in 1976, is a trusted provider of end-to-end event technology solutions, offering registration, ticketing, lead retrieval, and attendee-engagement services for events of all sizes across trade, association, corporate, and government markets.

Job Description:

We’re looking for a Data Scientist to drive measurable improvement of our AI systems — including a multi-agent LLM pipeline that profiles, classifies, and values customer data assets, and a classification service that builds our reference dataset from public sources. You’ll own the evaluation strategy, ground-truth corpus design, and statistical rigor that turns “the agent feels better” into “the agent is measurably 18% more accurate at industry classification on our latest corpus.” This is a hands-on, high-ownership role where you’ll be the technical authority on what “good” looks like for our AI outputs.

Key Responsibilities:

• Design, build, and maintain evaluation frameworks for our multi-agent LLM pipelines covering classification, PII detection, valuation, retrieval, segmentation, and synthesis — with regression-detection rigor.
• Curate, expand, and version synthetic and real-world test corpora that exercise our AI pipelines end-to-end across 20+ industry verticals.
• Quantify model performance: precision, recall, calibration, inter-rater agreement against human-verified ground truth, drift detection across releases.
• Partner with engineering to design prompt experiments, agent variants, and structured-output schema iterations; report results with statistical confidence intervals — not anecdotes.
• Improve vector-search comparable retrieval: embedding model selection, retrieval evaluation (recall@k, MRR), taxonomy refinement, classification accuracy uplift.
• Evaluate prompt strategies, tool-use patterns, and routing logic; recommend model-tier choices backed by cost/accuracy data.
• Profile production traces to identify failure modes (hallucinated outputs, mis-classifications, missed PII), then design experiments to fix them.
• Work cross-functionally with engineering, product, and domain experts to translate fuzzy product goals (“the analysis should feel insightful”) into quantitative success metrics.
• Communicate findings through written reports, dashboards, and decision memos that executive leadership can act on.

Qualifications:

• Bachelor’s degree in Computer Science, Statistics, Data Science, Machine Learning, or related quantitative discipline, or equivalent professional experience. Master’s or PhD preferred.
• 3+ years of professional data science, ML engineering, or AI evaluation experience shipping models or AI systems to production.
• Strong Python skills (Pandas, NumPy, scikit-learn, PyTorch or TensorFlow), with comfort writing production-quality code that engineers will run in CI.
• Proven experience evaluating LLM-based systems: prompt experimentation, structured-output validation, hallucination detection, retrieval evaluation, judge-LLM patterns.
• Solid grasp of classical statistics: hypothesis testing, confidence intervals, sample-size calculation, power analysis, calibration.
• SQL proficiency for ad-hoc analysis on PostgreSQL; comfortable with embedded analytical databases for offline evaluation.
• Experience with vector databases and embedding models.
• Ability to translate business goals into measurable evaluation criteria, and willingness to push back when a “metric” doesn’t measure what stakeholders think.

What We Offer:

• Competitive salary and benefits package.
• A fast-paced, high-impact work environment.
• Opportunity to work closely with executive leadership.
• The chance to work with cutting-edge technologies and make a significant impact.
• A culture of innovation, ownership, and growth.