Business Intelligence Analyst Senior (AI Evaluation & Monitoring)

Geisinger; Danville (PA); In loco; USD 85.000 - 110.000; Tempo pieno

Descrizione del lavoro

Geisinger is looking for a Senior BI Analyst based in Pennsylvania to leverage analytic expertise in supporting AI Evaluation & Monitoring. This position requires strong skills in data analysis, SQL, and Python, along with a minimum of 8 years of relevant experience. The analyst will work closely with stakeholders, develop reporting solutions, and act as a mentor to junior analysts. A High School Diploma or equivalent is required. The role allows for a hybrid or office work environment.

Competenze

• 8+ years of relevant experience in data analysis or related field.
• Experience in healthcare or regulated industry data preferred.
• Strong proficiency with SQL and Python for data analysis.

Mansioni

• Consult with AI stakeholders to gather data requirements.
• Run validation studies and analyze results.
• Develop and execute complex queries for data analysis.

Conoscenze

Data analysis

Statistical methods

SQL

Python (pandas, numpy, scikit-learn)

Mentoring

Communication

Formazione

High School Diploma or Equivalent (GED)

Strumenti

Business intelligence software

Clinical data warehouses

Descrizione del lavoro

Job Summary

Senior BI Analyst serves as a partner to the business or clinical area supported (AI Evaluation & Monitoring) and uses analytic expertise—including knowledge of data structures, programming, and statistics—to help the organization make data‑driven decisions. The role involves collecting, cleaning, and interpreting data sets to answer questions or solve problems, identifying improvement opportunities, and supporting the evaluation of existing AI/ML projects. Senior BI Analysts act as the analytics expert in meetings and can operate independently. The position requires strong oral and written communication skills, including the ability to translate analyses to non‑technical stakeholders, and excellent initiative, organizational, and customer‑relationship skills. Mentorship of junior BI developers, code reviews, and training are also part of the role.

Work is typically performed in an office or remote environment and the analyst is accountable for satisfying all job‑specific obligations and complying with all organization policies and procedures.

Job Duties

• Independently consults with AI stakeholders to gather evaluation and data requirements and define scope on projects of all sizes.
• Executes evaluation work that keeps the AI portfolio safe, performant, and accountable—running validation studies, instrumenting production monitoring, analyzing results, and turning findings into clear written work products for governance and program teams.
• Collaborates with internal partners to design and develop reporting solutions and analytic studies to support business needs.
• Plans projects and requests completion tasks.
• Provides time and effort estimates based on requirements gathering; manages multiple projects independently.
• Demonstrates working knowledge of relevant source systems and proficiency with databases/data marts within the first year in the role. Stays abreast of industry trends and becomes familiar with business intelligence software systems used at Geisinger.
• Recommends analysis approaches and additional analyses or reporting needed based on business questions, requirements, and initial findings.
• Develops, validates, and executes complex queries; creates complex reports and performs complex data analysis that answers business questions for internal customers; interprets results and makes recommendations.
• Presents analysis and reports to internal customers and stakeholders.
• Uses statistical methods during analyses and selects visualization techniques that accurately reflect the intended message and limitations.
• Reports and troubleshoots data quality issues during profiling, testing, or validation.
• Collaborates and shares knowledge with team members.
• Participates in strategic departmental initiatives.
• Acts as a mentor to other analysts.
• May participate in industry advisory and/or user groups.
• Maintains strong attention to detail and clarity when describing results of data analyses.
• Proactively offers suggestions, provides resources, volunteers for assignments, and removes barriers to help the team accomplish its goals.
• Follows all department policies and procedures, including request intake/fulfillment, change management, and data governance practices.

Preferred Skills

• Healthcare, clinical, or regulated‑industry data experience strongly preferred.
• Applied statistics: hypothesis testing, confidence intervals, power and sample size, regression, and non‑parametric methods.
• SQL experience with large, messy clinical data warehouses, including window functions, CTEs, debugging query plans, and performance tuning.
• Python for analysis (pandas, numpy, scikit‑learn, statsmodels, common evaluation libraries).
• Experience designing and executing production monitoring: drift detection, performance decay tracking, adoption and outcome metrics.
• Deployment and evaluation of LLM and generative AI: golden sets, judge‑based scoring, hallucination and grounding checks.

Education

High School Diploma or Equivalent (GED) – Required

Experience

Minimum of 8 years relevant experience – Required

Certifications and Licenses

None specified.

EEO Statement

We are an affirmative action, equal opportunity employer and all qualified applicants will receive consideration for employment regardless of race, color, religion, sex, sexual orientation, gender identity, national origin, disability or status as a protected veteran.