Data Engineer, AI Support

Insurance Institute for Business & Home Safety - IBHS

Richburg (SC)

On-site

USD 120,000 - 170,000

Full time

6 days ago

Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health plan

Retirement plan

Life insurance

PTO

Family leave

Disability insurance

Training and development

Job summary

IBHS, based in the United States, seeks a Data Engineer, AI Support to translate enterprise needs into scalable data solutions enriching decision-making and operations. You will collaborate with AI researchers and data engineering to prepare data, build pipelines, and evaluate AI/ML workflows.

The role emphasizes responsible AI practices, data quality, reproducibility, and documentation, with opportunities to influence research datasets and tooling across IBHS AI initiatives.

Qualifications

Bachelor's degree or higher in a quantitative field and hands-on experience building data pipelines for AI/ML workflows.
Strong Python and SQL skills with experience in data manipulation, exploration, and visualization.
Familiarity with LLM/VLM data preparation, embedding pipelines, vector databases, semantic search, and reproducible data practices.

Responsibilities

Prepare, organize, and improve data used across AI research and ML projects.
Build reliable data workflows moving data from raw sources to usable datasets for analysis and training.
Ensure datasets are consistent, traceable, and well documented as they evolve over time.
Automate recurring data preparation tasks to improve research efficiency.
Develop clear summaries and visualizations to surface data insights and potential issues.

Skills

Python data manipulation

SQL

NoSQL

Data modeling

LLM/VLM datasets

Embedding pipelines

Vector databases

Semantic search

RAG

Metadata retrieval

Graph databases

Data quality

Schema validation

Dataset versioning

Experiment tracking

LLM observability

CI/CD

Terraform

GitHub Actions

Bash scripting

Education

Bachelor's degree in data science, statistics, computer science, mathematics, engineering

Master's degree in data science

Tools

Airflow

Prefect

Dagster

Spark

Azure

AWS

Google Cloud

Label Studio

Prodigy

Job description

About The Role

The Data Engineer, AI Support is a key technical partner in IBHS's responsible adoption and application of artificial intelligence, machine learning, and advanced analytics. This position translates enterprise-wide needs across IBHS into practical, scalable data solutions that improve decision-making, operational effectiveness, and employee capabilities.

About The Role

Why This Role Matters

This role strengthens IBHS's enterprise-wide ability to use data and artificial intelligence thoughtfully, responsibly, and effectively. By connecting organizational priorities with technical capabilities, the Data Engineer, AI Support helps IBHS identify valuable use cases, improve access to actionable information, and build confidence in AI-enabled solutions. The position also helps establish consistent practices for solution quality, documentation, data handling, human review, and responsible AI use.

What You'll Do

Work closely with AI researchers and Data Engineering to prepare, organize, and improve the data used across research and machine learning projects
Build reliable data workflows that move research data from raw sources into usable datasets for analysis, experimentation, training, and evaluation
Explore and understand new datasets, identify quality issues or gaps, and help determine the best way to structure and use the data
Support the preparation of datasets for a range of AI applications, including language, vision, multimodal, and retrieval-based systems
Help ensure research datasets are consistent, traceable, reproducible, and well documented as they evolve over time
Automate recurring data preparation and processing tasks to make research workflows more efficient and repeatable
Develop clear summaries and visualizations that help the team understand datasets, patterns, and potential issues
Support the integration of data across research tools, internal systems, and AI platforms
Help protect sensitive information and follow appropriate data handling practices throughout the data lifecycle
Contribute to an experimental research environment where datasets, methods, and requirements may change as projects develop
Stay current on relevant developments in artificial intelligence, machine learning, data science, and emerging analytical technologies

What We're Looking For

Bachelor's degree in data science, statistics, computer science, mathematics, engineering, or a related quantitative field
Experience building ETL/data pipelines to clean, transform, integrate, and prepare structured, semi-structured, and unstructured data for AI/ML workflows
Strong Python data manipulation skills, including efficient use of vectorized libraries for large-scale data processing, exploration, and visualization
Working knowledge of SQL, NoSQL, data modeling, columnar formats, and modern data storage technologies
Familiarity with preparing and versioning LLM/VLM training and evaluation datasets, including QA, preference/RL, multimodal, and human-annotated data
Familiarity with embedding pipelines, vector databases, semantic search, RAG, and metadata-aware retrieval workflows
Exposure to graph databases, knowledge graphs, and graph-based data modeling for AI applications
Understanding of data quality, schema validation, dataset versioning, metadata, lineage, and reproducible train/validation/test splits with leakage prevention
Familiarity with distributed data processing and workflow orchestration concepts such as DAGs, task dependencies, scheduling, and pipeline monitoring
Comfortable working in Linux environments with Bash/shell scripting and basic automation
Familiarity with experiment tracking and LLM observability tools such as Weights & Biases and Langfuse
Basic understanding of PII handling, masking, hashing, tokenization, and de-identification within data pipelines
Familiarity with CI/CD and infrastructure automation tools such as GitHub Actions, GitLab CI, and Terraform
Comfortable working with research datasets that may be incomplete, inconsistent, or evolving, and able to investigate the data before implementing a solution
Strong written communication, presentation, and technical-documentation skills
Ability to build effective working relationships across business and technical functions
Ability to exercise sound judgment, manage multiple priorities, and work independently while contributing to cross-functional initiatives

Preferred Qualifications

Master's degree in data science, statistics, computer science, artificial intelligence, machine learning, or a related field
Experience supporting AI/ML research, scientific computing, or other data-intensive research environments
Hands‑on experience with cloud data platforms or services in Azure, AWS, or Google Cloud
Experience with data orchestration and distributed processing tools such as Airflow, Prefect, Dagster, Spark, or similar technologies
Familiarity with data annotation and human-in-the-loop platforms such as Label Studio or Prodigy, particularly for machine learning or multimodal datasets

Note: IBHS welcomes applications from all qualified candidates who are legally authorized to work in the United States. Employment visa sponsorship is not available for this position. Candidates who require IBHS sponsorship for an employment visa, now or in the future, will not be considered for this role.

Benefits