Remote AI Data Engineer for Healthcare LLM Pipelines

C the Signs Limited; Rhode Island; Hybrid; USD 120,000 - 170,000; Full time

Benefits offered by this job

Remote or Hybrid options available

Competitive salary and benefits

Life-changing AI technology impact

Professional development

Mission to save lives and improve care

Job summary

C the Signs Limited is seeking a Data Engineer to develop and optimize data pipelines for large language models and ML workflows in healthcare. You will own the data lifecycle from ingestion to quality assurance, working with data scientists to meet data requirements and compliance standards.

The role requires strong cloud, big data, and Spark expertise, with a focus on enabling scalable access to high-volume healthcare datasets in a collaborative team environment.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related field.
• Proven experience as a Data Engineer with big data tech.
• Strong Python, Scala, or Java programming skills.
• Extensive experience with data warehousing, ETL, and modeling.
• Experience with AWS, GCP, or Azure and their data storage and processing services.
• Hands-on experience with Apache Spark for distributed processing.
• Excellent problem-solving and independent work ability.
• Strong communication and collaboration skills.
• Experience with healthcare data standards (FHIR/HL7).
• Familiarity with data orchestration tools (Airflow).

Responsibilities

• Collaborate with data scientists and ML engineers to understand data requirements for LLM and model fine-tuning.
• Design, build, and maintain scalable data pipelines to ingest, process, and store healthcare datasets.
• Implement data validation and monitoring to ensure data integrity.
• Implement data cleaning, validation, and transformation to ensure data quality.
• Develop and optimize data structures and schemas for efficient access by LLMs.
• Identify and acquire new data sources while ensuring HIPAA compliance.
• Monitor pipeline performance, troubleshoot issues, and optimize for reliability.
• Document data engineering processes, models, and dictionaries.
• Stay up-to-date with data engineering and ML advancements.

Skills

Python

Scala

Java

Big data technologies

Spark

Airflow

AWS

GCP

Azure

Data warehousing

ETL

Data modeling

FHIR/HL7

Healthcare data

Communication

Problem solving

Education

Bachelor’s degree in Computer Science, Engineering, or related field

Master’s degree in a related field

Tools

Apache Airflow

Job description

C the Signs Limited is seeking a Data Engineer to develop and optimize data pipelines for large language models and ML workflows in healthcare. You will own the data lifecycle from ingestion to quality assurance, working with data scientists to meet data requirements and compliance standards.

The role requires strong cloud, big data, and Spark expertise, with a focus on enabling scalable access to high-volume healthcare datasets in a collaborative team environment.