Data Engineer for Enterprise AI & Databricks

Focused; Chicago (IL); On-site; USD 145,000 - 175,000; Full time

Job summary

Focused, a Chicago-based software partner, is seeking a Data Engineer to build the data foundation behind an enterprise AI platform. You will own data pipelines, governance, and data readiness for retrieval across analytics and AI workloads, mostly in Databricks.

In this role, you will design batch and streaming pipelines, model data in lakehouse architectures, implement Unity Catalog governance, and collaborate with AI engineers.

Qualifications

• 5+ years in data engineering.
• Experience with Databricks, Spark, Delta Lake.
• Proficiency in Python and SQL.
• Experience building production data pipelines in cloud environments (Azure or GCP preferred).
• Experience with data modeling and data quality practices.

Responsibilities

• Design and build batch and streaming pipelines in Databricks (Spark, Delta Lake, Workflows).
• Model data in a lakehouse architecture for analytics and AI workloads.
• Implement governance and access controls with Unity Catalog.
• Monitor data quality and pipeline reliability.
• Build ingestion and chunking pipelines for unstructured data feeding RAG systems.
• Generate, store, and refresh embeddings; operate vector search.
• Build datasets for LLM evaluation and measure retrieval quality.
• Partner with AI and platform engineers to ensure data with correct permissions.
• Deploy data infrastructure on Azure and/or GCP using IaC.
• Bring software engineering habits to data work: version control, testing, CI/CD.

Skills

Python

SQL

Databricks

Spark

Delta Lake

Data modeling

Data quality

Azure or GCP

Job description

Focused, a Chicago-based software partner, is seeking a Data Engineer to build the data foundation behind an enterprise AI platform. You will own data pipelines, governance, and data readiness for retrieval across analytics and AI workloads, mostly in Databricks.

In this role, you will design batch and streaming pipelines, model data in lakehouse architectures, implement Unity Catalog governance, and collaborate with AI engineers.