AI Data Engineer — Research Pipelines & Insights

Insurance Institute for Business & Home Safety; Richburg (SC); On-site; USD 110,000 - 190,000; Full time

Benefits offered by this job

Health Plan

401k/IRA

Life Insurance

Paid Time Off

Family Leave

Disability Insurance

Training & Development

Job summary

The Insurance Institute for Business & Home Safety (IBHS) is seeking a Data Engineer, AI Support to translate enterprise needs into scalable data solutions for AI/ML workloads. You will collaborate with AI researchers and data engineers to prepare datasets, build pipelines, and ensure data quality and reproducibility across research projects.

Join a team focused on responsible AI adoption, data governance, and experimental workflows while improving decision-making and operational capabilities

Qualifications

• Bachelor’s degree in data science, statistics, computer science, mathematics, engineering, or a related quantitative field.
• Experience building ETL/data pipelines to clean, transform, integrate, and prepare structured, semi-structured, and unstructured data for AI/ML workflows.
• Strong Python data manipulation skills, including efficient use of vectorized libraries for large-scale data processing, exploration, and visualization.
• Working knowledge of SQL, NoSQL, data modeling, columnar formats, and modern data storage technologies.
• Familiarity with preparing and versioning LLM/VLM training and evaluation datasets, including QA, preference/RL, multimodal, and human-annotated data.

Responsibilities

• Work closely with AI researchers and Data Engineering to prepare, organize, and improve the data used across research and machine learning projects.
• Build reliable data workflows that move research data from raw sources into usable datasets for analysis, experimentation, training, and evaluation.
• Explore and understand new datasets, identify quality issues or gaps, and help determine the best way to structure and use the data.
• Support the preparation of datasets for a range of AI applications, including language, vision, multimodal, and retrieval-based systems.
• Help ensure research datasets are consistent, traceable, reproducible, and well documented as they evolve over time.
• Automate recurring data preparation and processing tasks to make research workflows more efficient and repeatable.
• Develop clear summaries and visualizations that help the team understand datasets, patterns, and potential issues.
• Support the integration of data across research tools, internal systems, and AI platforms.
• Help protect sensitive information and follow appropriate data handling practices throughout the data lifecycle.
• Contribute to an experimental research environment where datasets, methods, and requirements may change as projects develop.
• Stay current on relevant developments in artificial intelligence, machine learning, data science, and emerging analytical technologies.

Skills

Python data manipulation

SQL

NoSQL

Data modeling

Linux Bash

CI/CD

Airflow

Experiment tracking

Langfuse/Weights & Biases

Education

Bachelor’s degree in data science, statistics, CS, math, engineering

Master’s degree (preferred) in data science or AI fields

Tools

Airflow

Dagster

Spark

Terraform

GitHub Actions

Job description

The Insurance Institute for Business & Home Safety (IBHS) is seeking a Data Engineer, AI Support to translate enterprise needs into scalable data solutions for AI/ML workloads. You will collaborate with AI researchers and data engineers to prepare datasets, build pipelines, and ensure data quality and reproducibility across research projects.

Join a team focused on responsible AI adoption, data governance, and experimental workflows while improving decision-making and operational capabilities