Senior Data Engineer - AI Platform (6021)

NextPath Workforce Solutions; New York (NY); Remote; USD 160,000 - 200,000; Full time

Job summary

NextPath Workforce Solutions is seeking a Senior Data Engineer - AI Platform for a fully remote direct-hire role. The position targets a leader who will design and scale an enterprise Data & AI platform to support analytics and AI initiatives.

The ideal candidate has 8+ years of experience building data platforms, strong SQL/Python, Databricks, and hands-on experience with RAG pipelines and vector search, enabling AI-powered applications across diverse data workloads.

Qualifications

• 8+ years building and operating enterprise data platforms with hands-on AI/data work.
• Advanced SQL and Python with production-grade software practices.
• Hands-on Databricks experience and modern data modeling.
• Experience building Retrieval-Augmented Generation (RAG) pipelines.
• Experience with vector databases and AI search (embeddings, indexing).

Responsibilities

• Design, build, and scale an enterprise Data & AI platform.
• Develop scalable data pipelines across batch, streaming, structured, and unstructured data.
• Build AI-ready data products and retrieval capabilities for AI-powered apps.
• Improve vector search, hybrid search, and RAG capabilities.
• Develop event-driven/streaming solutions for near real-time insights.
• Establish engineering best practices for data modeling, testing, observability, governance, security and reliability.
• Create self-service and automation to improve developer productivity.
• Partner with cross-functional teams to deliver trusted data solutions for critical initiatives.
• Mentor engineers and contribute to architecture decisions and platform direction.

Skills

SQL

Python

RAG pipelines

Tools

Databricks

Vector databases

AI search

Job description

NextPath Workforce Solutions is currently seeking a Senior Data Engineer - AI Platform to join our client's team. This is a Remote Direct Hire position.

SALARY: $160,000 - $200,000 (depending on experience)

Unfortunately, at this time our client is unable to sponsor or transfer visas. Only candidates authorized to work in the US without sponsorship should apply.

REQUIREMENTS

• 8+ years building and operating enterprise data platforms, including recent hands-on AI/data work
• Advanced SQL and Python skills with strong production-grade software engineering practices
• Hands-on Databricks experience, plus modern data modeling and transformation practices
• Experience building and operating Retrieval-Augmented Generation (RAG) pipelines in production
• Experience with vector databases and AI search (embeddings, indexing/search patterns, retrieval strategies)

RESPONSIBILITIES

• Design, build, and scale an enterprise Data & AI platform supporting analytics and AI initiatives
• Develop and optimize scalable data pipelines across batch, streaming, structured, and unstructured workloads
• Build AI-ready data products and retrieval capabilities that enable AI-powered applications
• Implement and improve vector search, hybrid search, and RAG capabilities to enhance AI effectiveness
• Develop event-driven/streaming solutions to support near real-time insights and operational intelligence
• Establish engineering best practices for data modeling, testing, observability, governance, security, and reliability
• Create self-service and automation capabilities that improve developer productivity and time-to-insight
• Partner with cross-functional teams to deliver trusted, high-quality data solutions for critical initiatives
• Mentor engineers and contribute to architecture decisions and platform direction

NEXTPATH WORKFORCE SOLUTIONS

NextPath is a strategic talent partner connecting skilled professionals with organizations driving innovation and growth. Our recruiting team brings more than 80 years of combined experience delivering workforce solutions that align the right talent with the right opportunities. We work with leading companies across the country, offering flexible engagement models including contract, project-based, contract-to-hire, and direct hire roles.

We are an Equal Opportunity Employer