Data Scientist

Brooksource; Saint Paul (MN); On-site; USD 96,432 - 110,208; Full time

Job summary

An established industry player is seeking a skilled Data Scientist to enhance generative AI solutions. In this pivotal role, you will scope, develop, and maintain scalable AI systems while collaborating with experts to optimize prompt engineering for various applications. Your expertise in data science, particularly in managing both structured and unstructured data, will be crucial. This position offers an exciting opportunity to work at the forefront of AI technology, ensuring continuous improvement of models and delivering real-time insights. Join a dynamic team that values innovation and excellence in the healthcare sector.

Qualifications

• Proven experience in data science, managing structured and unstructured data.
• Expertise in statistical techniques and predictive analytics.

Responsibilities

• Develop and maintain scalable generative AI solutions.
• Design experiments to optimize Large Language Models (LLMs).
• Monitor LLM applications for drift and performance issues.

Skills

Data Science

Natural Language Processing (NLP)

Machine Learning (ML)

Deep Learning (DL)

Statistical Techniques

Predictive Analytics

Problem-Solving Skills

Education

Bachelor's Degree in Data Science or related field

Master's Degree in Data Science or related field

Tools

PyTorch

Statistical Analysis Tools

Data Visualization Tools

Vector Databases

Graph Databases

Job description

This range is provided by Brooksource. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

$70.00/hr - $80.00/hr

IT/ Engineering Recruiter | Connecting IT Professionals with Industry-Leading Opportunities

Data Scientist

Our Fortune 50 Healthcare Insurance client is seeking a highly skilled Data Scientist resource to play a pivotal role in the development, expansion, operation, and maintenance of generative AI solutions. The primary responsibilities include running an experimental framework to determine the optimal prompt engineering approaches, tuning prompts, and collaborating with subject matter experts (SMEs) for evaluations and results. This role requires a deep understanding of evaluating models output in production, particularly when ground metrics are absent, monitoring for issues such as model drift and hallucinations, and optimizing for offline and online metrics.

Key Responsibilities:

1. Scope, develop, expand, operate, and maintain scalable, reliable and safe generative AI solutions.
2. Design and execute prompt engineering experiments to optimize Large Language Models (LLMs) for various use cases.
3. Collaborate with SMEs to evaluate prompt effectiveness and align AI solutions with business needs.
4. Understand and apply offline and online evaluation metrics for LLMs, ensuring continuous model improvements.
5. Evaluate production models using live data in the absence of ground metrics, implementing robust monitoring systems.
6. Monitor LLM applications for model drift, hallucinations, and performance degradation.
7. Ensure smooth integration of LLMs into existing workflows, providing real-time insights and predictive analytics.

Qualifications:

1. Proven experience in data science, with expertise in managing structured and unstructured data.
2. Proficiency in statistical techniques, predictive analytics, and reporting results.
3. Experience in applied science in fields like Natural Language Processing (NLP), Machine Learning (ML), Deep Learning (DL), or Multimodal Analysis.
4. Strong background in software development, data modeling, or data engineering.
5. Deep understanding of building and scaling ML models, specifically LLMs.
6. Familiarity with open-source tools such as PyTorch, statistical analysis, and data visualization tools.
7. Experience with vector databases and graph databases is a plus.

Preferred Skills:

1. Experience in prompt engineering and prompt optimization.
2. Expertise in running experiments to evaluate generative AI performance.
3. Knowledge of production-level monitoring tools for ML models, including drift detection and mitigation strategies.
4. Excellent problem-solving skills and ability to work cross-functionally with data scientists, engineers, and SMEs.
5. Experience with safety, security and responsible use of AI.
6. Experience with red-teaming (adversarial testing) of generative AI.
7. Experience with developing AI applications with sensitive data such as PHI, PII and highly confidential data.

Seniority level

Mid-Senior level

Employment type

Full-time

Job function

Information Technology

Industries

IT Services and IT Consulting and Hospitals and Health Care