Senior Lakehouse Data Engineer for AI Pipelines

Baker Tilly Advisory Group, LP; Santa Clara (CA); On-site; USD 185,000 - 230,000; Full time

Job summary

Baker Tilly Advisory Group, LP in Santa Clara, CA, seeks a Sr Data Engineer to join the AI Product Group and build scalable, secure data infrastructure for AI applications and agents. You will own production pipelines, lakehouse architecture, governance, and retrieval systems, focusing on data quality and reliability while collaborating with global teams across platforms and domains.

This role emphasizes automated testing, versioning, monitoring, and practical delivery to enable enterprise AI

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• 3 years of hands-on data engineering or software engineering experience.
• Experience with modern lakehouse platforms and production pipelines.
• Strong Python and SQL; production-grade data engineering.
• Experience with data governance, lineage, and automated testing.
• Familiarity with data modeling principles and scalable structures.
• Experience with cloud-based data platforms, preferably Azure.
• Knowledge graphs, vector/search retrieval, and entity resolution.

Responsibilities

• Design, develop, implement, and maintain scalable data pipelines in a lakehouse environment.
• Build lakehouse architecture with medallion data layers and data modeling.
• Implement validation, lineage tracking, metadata management, and governance.
• Develop data models, contracts, and entity resolution across sources.
• Collaborate with AI/ Platform teams and external partners; ensure reliability.
• Maintain data freshness, monitoring, and production-readiness of pipelines.

Tools

Databricks

Apache Spark

Delta Lake

Medallion architecture

Microsoft Azure

Unity Catalog

Knowledge graphs

Graph databases

Vector search

Hybrid search

Retrieval-based applications

Document extraction

Entity resolution

Master data management

Change data capture

Job description

Baker Tilly Advisory Group, LP in Santa Clara, CA, seeks a Sr Data Engineer to join the AI Product Group and build scalable, secure data infrastructure for AI applications and agents. You will own production pipelines, lakehouse architecture, governance, and retrieval systems, focusing on data quality and reliability while collaborating with global teams across platforms and domains.

This role emphasizes automated testing, versioning, monitoring, and practical delivery to enable enterprise AI