Senior Data Engineer — AI Data Platform & Lakehouse

Baker Tilly (US); San Jose (CA); On-site; USD 185,000 - 230,000; Full time

Job summary

Baker Tilly is seeking a Sr Data Engineer to join the founding team of the AI Product Group (APG). This hands-on role builds scalable data infrastructure powering AI applications and intelligent agents.

Focus areas include a governed data foundation, knowledge layer, and robust data pipelines within a modern lakehouse environment. The ideal candidate has strong Python/SQL skills, experience with lakehouse tech, and a zeal for engineering excellence in a fast-paced setting.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field.
• 3 years of hands-on data engineering or software engineering experience.
• Experience building and operating production pipelines on a modern lakehouse platform.
• Experience implementing data quality, lineage, or data contracts within production pipelines.
• Strong Python and SQL skills for production data engineering.
• Strong understanding of data modeling principles for evolving sources.
• Experience designing batch and incremental data pipelines for structured and unstructured data.
• Familiarity with lakehouse technologies (Databricks, Spark, Delta Lake) and Azure.
• Knowledge of data governance capabilities (catalogs, access controls, lineage).
• Experience with knowledge graphs, graph data modeling, or retrieval-based apps is preferred.

Responsibilities

• Design and build scalable batch and incremental data pipelines transforming information into clean datasets in a lakehouse.
• Develop and maintain lakehouse architecture and data models with proper governance.
• Ensure data quality, lineage, and provenance across data products.
• Build data models and identity resolution across enterprise sources.
• Implement data governance and access controls for authorized use by AI agents.
• Develop knowledge graphs, search indexes, and retrieval pipelines for AI applications.
• Support AI data readiness with pipelines capturing activity and feedback.
• Establish benchmarks and testing for data extraction, retrieval, and grounding accuracy.
• Maintain data freshness and reliability with CDC and versioning.
• Own production delivery, monitoring, and documentation of data pipelines.
• Collaborate with global teams to align with enterprise standards.
• Participate in strategic initiatives to advance AI and data platforms.

Skills

Python

SQL

Data modeling

Data governance

Version control

Automated testing

Education

Bachelor's degree in CS/Engineering

Tools

Databricks

Apache Spark

Delta Lake

Medallion architecture

Azure

Job description

Baker Tilly is seeking a Sr Data Engineer to join the founding team of the AI Product Group (APG). This hands-on role builds scalable data infrastructure powering AI applications and intelligent agents.

Focus areas include a governed data foundation, knowledge layer, and robust data pipelines within a modern lakehouse environment. The ideal candidate has strong Python/SQL skills, experience with lakehouse tech, and a zeal for engineering excellence in a fast-paced setting.