AI Platform & Data Engineer – Databricks / MLOps

JCW Group; Charlotte (NC); Hybrid; USD 120,000 - 170,000; Full time

Job summary

JCW Group is hiring an AI Platform & Data Engineer to build the data foundation behind its enterprise GenAI platform in Charlotte, NC. You will develop Databricks pipelines, vector embeddings and data governance to support AI/GenAI use cases and production deployment.

You will collaborate with Architecture, Technology, Data and business stakeholders, leveraging Azure Databricks and Unity Catalog to deliver scalable, auditable data products for AI models and RAG applications.

Qualifications

• Bachelor’s degree in Computer Science, Data Engineering, Data Science or a related field.
• 5+ years of hands-on data engineering experience, with recent work supporting AI/ML or GenAI use cases in production.
• Strong Python, PySpark and SQL skills.
• Deep hands-on Databricks experience, including Delta Lake, Unity Catalog, Workflows and Asset Bundles, along with Azure data services.
• Experience preparing data for GenAI, such as document processing, chunking, embeddings and vector databases (e.g., Databricks Vector Search, Azure AI Search).
• Hands-on experience with MLflow and MLOps/LLMOps practices.

Responsibilities

• Build and maintain scalable data pipelines on Databricks and Azure that ingest, clean, transform and deliver data for AI and GenAI use cases.
• Prepare data for RAG and AI applications, including document ingestion, chunking, embeddings and managing vector indexes with Databricks Vector Search.
• Own data quality across the AI platform, defining quality checks and monitoring for data drift, freshness and completeness.
• Implement end-to-end data lineage, versioning and governance across data, prompts, models and responses using Unity Catalog and MLflow, delivering audit-ready artifacts for regulatory review.
• Build CI/CD and infrastructure-as-code using Databricks Asset Bundles to promote data pipelines, models and AI workflows across DEV/QA/REL/PROD.
• Support the MLOps/LLMOps lifecycle, including model deployment, monitoring for performance and grounding, and cost visibility.
• Apply security and privacy controls to sensitive data used in AI, including access management, secrets management and protection of data and embeddings.
• Support day-2 operations through runbooks, incident response and SLAs, and help educate wider teams on data and AI platform best practices

Skills

Python

PySpark

SQL

Databricks

Delta Lake

Unity Catalog

MLflow

MLOps

Azure Data Services

CI/CD

Infrastructure as Code

Data Governance

Education

Bachelor's degree in Computer Science / Data Engineering / Data Science

Tools

Databricks Asset Bundles

Terraform

Kubernetes

GitHub Actions

Azure

Job description

Candidates must be authorized to work in the US without current or future sponsorship (including H-1B transfer or OPT/STEM OPT). Work authorization will be verified as part of the hiring process.

About the Role

JCW is partnering with a global financial institution to hire an AI Platform & Data Engineer to build and run the data foundation behind its enterprise GenAI platform. AI is only as good as the data feeding it, and this role owns that layer. You will build the Databricks pipelines that prepare, govern and deliver trusted data to AI models, RAG applications and agents. You will also make sure those AI systems run reliably in production. You will work closely with Architecture, Technology, Data and business stakeholders, and partner with Azure, Databricks and other infrastructure providers. This is a hands-on engineering seat for someone who is strong in data engineering on Databricks and has worked on getting AI into production.

What You’ll Do

• Build and maintain scalable data pipelines on Databricks and Azure that ingest, clean, transform and deliver data for AI and GenAI use cases
• Prepare data for RAG and AI applications, including document ingestion, chunking, embeddings and managing vector indexes with Databricks Vector Search
• Own data quality across the AI platform, defining quality checks and monitoring for data drift, freshness and completeness
• Implement end-to-end data lineage, versioning and governance across data, prompts, models and responses using Unity Catalog and MLflow, delivering audit-ready artifacts for regulatory review
• Build CI/CD and infrastructure-as-code using Databricks Asset Bundles to promote data pipelines, models and AI workflows across DEV/QA/REL/PROD
• Support the MLOps/LLMOps lifecycle, including model deployment, monitoring for performance and grounding, and cost visibility
• Apply security and privacy controls to sensitive data used in AI, including access management, secrets management and protection of data and embeddings
• Support day-2 operations through runbooks, incident response and SLAs, and help educate wider teams on data and AI platform best practices

What We’re Looking For

• Bachelor’s degree in Computer Science, Data Engineering, Data Science or a related field
• 5+ years of hands-on data engineering experience, with recent work supporting AI/ML or GenAI use cases in production
• Strong Python, PySpark and SQL skills
• Deep hands-on Databricks experience, including Delta Lake, Unity Catalog, Workflows and Asset Bundles, along with Azure data services
• Experience preparing data for GenAI, such as document processing, chunking, embeddings and vector databases (e.g., Databricks Vector Search, Azure AI Search)
• Hands-on experience with MLflow and MLOps/LLMOps practices
• Experience building data quality checks, monitoring and lineage in a production environment
• Experience with CI/CD, infrastructure-as-code and containerization
• Familiarity with data governance and security controls in a regulated environment, ideally financial services
• Excellent communication skills, with the ability to partner with technical and non-technical stakeholders

The Details

This role is based in Charlotte, NC, with 2 days per week on site. Candidates looking to relocate to Charlotte are welcome to apply, though relocation assistance is not provided. Applicants must be US Citizens or Green Card holders; visa sponsorship is not available.

JCW is proud to be an equal opportunity recruiter. We are committed to fostering an inclusive recruitment process and ensuring equal consideration for all applicants regardless of gender, age, disability, religion, sexual orientation, marital status, or race.