Data Engineer for AI-Driven Threat & Risk Platform (Remote)

Cornerstone Global Partners (CGP Group)

United States

Remote

USD 120,000 - 170,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Competitive salary
Medical insurance
401(k) with company match
Generous PTO
Company-paid holidays
Remote work

Job summary

Cornerstone Global Partners (CGP Group) is seeking a Data Engineer to build AI-enabled data ingestion pipelines for a new threat and risk management platform. You will implement batch and streaming ingestion on Databricks, apply medallion architecture, and collaborate with Data Scientists to productionize extraction pipelines.

US-based team, remote work, and overlapping hours with India team are required. This role emphasizes CI/CD, governance, and scalable data pipelines across GIS, SCADA, and

Qualifications

  • Hands-on experience building production pipelines on Databricks including Spark/PySpark.
  • Hands-on experience with Delta Lake, medallion architecture, and lakehouse engineering.
  • Strong SQL skills.
  • 3-5 years of experience in data engineering, ETL development, or cloud data platform engineering.
  • Hands-on CI/CD experience for data pipelines.
  • Experience using AI-assisted coding tools in development work.
  • Git-based development in a code-reviewed team.

Responsibilities

  • Build and maintain ingestion pipelines for structured and semi-structured sources (GIS, SCADA, maintenance systems).
  • Implement batch and streaming ingestion using Databricks Workflows, Spark, PySpark, SQL.
  • Apply medallion architecture for transformation, standardization, and enrichment.
  • Implement CDC, SCD, schema evolution, and data-validation rules.
  • Normalize third-party data feeds into the shared data model.
  • Automate normalization of units, schemas, and semantics across data.
  • Work with Data Scientists to productionize document-extraction pipelines and review low-confidence extractions.
  • Configure and manage Delta Lake tables, partitioning, and Unity Catalog."
  • Build connectors to customer systems of record and support geospatial processing."
  • Implement data-quality tests, profiling, drift monitoring, and lineage for regulatory traceability.
  • Build, schedule, and monitor data workflows; own alerting and failure handling.
  • Implement CI/CD for pipeline code with version control and tests.
  • Troubleshoot production incidents and optimize performance and cost.
  • Participate in architecture reviews and document pipelines and runbooks.

Skills

Databricks
Spark/PySpark
Delta Lake
SQL
Python
CI/CD
Data modeling
Data governance
Git

Tools

Databricks
SQL

Job description

Cornerstone Global Partners (CGP Group) is seeking a Data Engineer to build AI-enabled data ingestion pipelines for a new threat and risk management platform. You will implement batch and streaming ingestion on Databricks, apply medallion architecture, and collaborate with Data Scientists to productionize extraction pipelines.

US-based team, remote work, and overlapping hours with India team are required. This role emphasizes CI/CD, governance, and scalable data pipelines across GIS, SCADA, and

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer (Mid-Level) US
Data Engineer (Mid-Level) US

Cornerstone Global Partners (CGP Group) • United States

Remote
USD 120,000 - 170,000
Competitive salary
Medical insurance
401(k) with company match
+3
Data Engineer
Data Engineer

Virtusa Corporation • United States

Remote
USD 120,000 - 160,000
Data Engineer - Databricks ELT & GenAI Enablement (Remote)
Data Engineer - Databricks ELT & GenAI Enablement (Remote)

Primoris Services Corporation • Dallas (TX)

Remote
USD 115,000 - 120,000
401k with employer match
Health/Dental/Vision insurance
Paid time off
+2
Data Engineer – Databricks Pipelines & GenAI (Remote)
Data Engineer – Databricks Pipelines & GenAI (Remote)

100 Primoris Services Corporation • Dallas (TX)

Remote
USD 115,000 - 120,000
401k with employer match
Health/Dental/Vision insurance
Paid time off
+2
Remote Databricks Data Engineer - AI & Cloud Platform
Remote Databricks Data Engineer - AI & Cloud Platform

International Business Machines Corporation • Jefferson City (MO)

Remote
USD 120,000 - 160,000
Databricks Data Engineer - GenAI Pipelines (Remote)
Databricks Data Engineer - GenAI Pipelines (Remote)

Primoris Services Corporation • United States

Remote
USD 115,000 - 120,000
401k w/employer match
Health/Dental/Vision insurance plans
Paid time off
+2
Senior Data Engineering Tech Lead - AI & Databricks (Remote)
Senior Data Engineering Tech Lead - AI & Databricks (Remote)

Jobgether • United States

Remote
USD 55,000 - 90,000
Meal allowance
Medical and dental insurance
Home office allowance
+3
Senior Data Engineer (Databricks) (Fully Remote)
Senior Data Engineer (Databricks) (Fully Remote)

Object Edge • United States

Remote
USD 110,000 - 170,000
Data Engineer for AI Data Platform (Databricks)
Data Engineer for AI Data Platform (Databricks)

Focused • Denver (CO)

Hybrid
USD 140,000 - 170,000
Senior Data & AI Engineer — Remote (AWS, Databricks)
Senior Data & AI Engineer — Remote (AWS, Databricks)

Encora • United States

Remote
USD 120,000 - 180,000