Senior Data Engineer: Databricks & AI Platform

Brillio; United States; On-site; USD 140,000 - 180,000; Full time

Job summary

Brillio is seeking a Senior Data Engineer to design, build, and scale enterprise-grade data pipelines and AI-ready data foundations. You will work at the intersection of Databricks, Azure AI, and knowledge management to deliver trusted data products for analytics and AI initiatives.

The role requires strong Python, PySpark, and SQL skills, plus experience with Delta Lake and Unity Catalog within cloud data platforms.

Qualifications

• 7+ years in data engineering or big data environments.
• Hands-on with Databricks including Delta Lake and Unity Catalog.
• Experience in building large-scale ETL/ELT pipelines.
• Proficient in Python, PySpark, and SQL.
• Experience with Azure Data Services and cloud data platforms.

Responsibilities

• Design, develop, and maintain scalable data pipelines on Databricks.
• Create AI-ready data foundations including ontology and knowledge graphs.
• Support AI/ML and Generative AI use cases with data prep and pipelines.
• Implement data governance, lineage, metadata, and security controls.
• Optimize performance, cost, and reusable engineering templates.

Skills

Databricks

Python

SQL

Cloud platforms

ETL/ELT

Data Modeling

Governance

Education

Bachelor's or Master's in CS or related

Tools

Delta Lake

Unity Catalog

Spark

Workflows

Databricks

Job description

Brillio is seeking a Senior Data Engineer to design, build, and scale enterprise-grade data pipelines and AI-ready data foundations. You will work at the intersection of Databricks, Azure AI, and knowledge management to deliver trusted data products for analytics and AI initiatives.

The role requires strong Python, PySpark, and SQL skills, plus experience with Delta Lake and Unity Catalog within cloud data platforms.