Senior Data & AI Engineer - RAG-Ready Pipelines

Carlyle; New York (NY); On-site; USD 160,000 - 180,000; Full time

Job summary

Carlyle is seeking a Data & AI Engineer to design and operate AI-ready data pipelines, semantic models, and RAG components at enterprise scale. You will work with data science and AI teams to deliver trusted data products and actionable insights for multiple business units.

You will implement retrieval and grounding patterns, maintain production-grade ELT and streaming pipelines, and ensure governance, observability, and security across the data platform.

Qualifications

• Bachelor’s degree in computer science, data engineering, information systems, or related field.
• 6+ years of data engineering/analytics engineering experience.
• 1–2 years of hands-on generative AI/ML production experience.
• Proven experience implementing RAG, vector stores, embeddings, and grounded AI components.
• Strong Python and SQL skills; knowledge of distributed processing like Spark.
• Experience with dbt, Fivetran, Snowflake in AWS environments.

Responsibilities

• Build and operate AI-ready data pipelines: embedding generation, chunking, indexing, refreshing.
• Implement RAG components with vector stores, hybrid search, grounding logic.
• Develop interfaces for agents to query and act on data with guardrails and logging.
• Collaborate with data science and AI engineering to operationalize feature stores and datasets.
• Define semantic models and data contracts for trusted self-service analytics.

Skills

Python

SQL

Spark

Data pipelines

Education

Bachelor’s degree in CS/DI or related

Tools

dbt

Fivetran

Snowflake

Job description

Carlyle is seeking a Data & AI Engineer to design and operate AI-ready data pipelines, semantic models, and RAG components at enterprise scale. You will work with data science and AI teams to deliver trusted data products and actionable insights for multiple business units.

You will implement retrieval and grounding patterns, maintain production-grade ELT and streaming pipelines, and ensure governance, observability, and security across the data platform.