Senior Manager, Data Engineering (AI enablement, context engineering)

Diligente Technologies; San Jose (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Diligente Technologies in San Jose, CA, seeks a Senior Manager, Data Engineering to lead a high-performing team and drive scalable cloud data platforms and AI-enabled analytics.

You will shape lakehouse architectures, DataOps, data governance, and robust data pipelines using Databricks, Snowflake, AWS Redshift and BigQuery, enabling data-driven decisions. This hands-on leadership role requires deep SQL and cloud data platform expertise plus strong cross-functional collaboration.

Qualifications

• 10+ years of experience in Data Engineering, Data Architecture, or related fields.
• 5+ years of experience leading and developing technical teams.
• Strong hands-on experience with SQL and modern cloud data platforms.

Responsibilities

• Architect and implement scalable cloud-based Lakehouse/Warehouse solutions.
• Design, optimize, and govern enterprise data models and semantic layers.
• Build and maintain high-performance batch and real-time data pipelines.
• Establish data quality frameworks, monitoring, and validation processes.
• Implement DataOps and CI/CD pipelines for automated deployments.
• Establish data governance including cataloging, lineage, and metadata management.
• Design strong data security and access controls.

Skills

Data Engineering

Data Architecture

SQL

Cloud Data Platforms

Databricks

Snowflake

AWS Redshift

BigQuery

ETL/ELT

DataOps

Data Modeling

Data Governance

Data Security

CI/CD

Leadership

Mentoring

Cross-functional Collaboration

Tools

Alteryx

Informatica

Job description

We’re looking for a Senior Manager, Data Engineering to lead a high-performing data engineering team and drive the design and implementation of scalable, secure, and modern enterprise data platforms. This is an excellent opportunity for a hands-on technical leader passionate about cloud data engineering, lakehouse architecture, DataOps, and AI/advanced analytics enablement.

Note: Experience in AI enablement for context engineering and ML models including causal and colinear modeling

• Architect and implement scalable cloud-based Lakehouse/Warehouse solutions using Databricks and AWS
• Design, optimize, and govern enterprise data models, including dimensional modeling and semantic layers
• Build and maintain high-performance batch and real-time data pipelines using ETL/ELT and streaming technologies
• Establish robust data quality frameworks, monitoring, and validation processes
• Implement CI/CD pipelines and DataOps best practices for automated, secure, and reliable deployments
• Establish data governance, including data cataloging, lineage, metadata management, and standards
• Design and implement strong data security, access controls, and governance frameworks
• Build scalable data foundations supporting AI, automation, machine learning, and advanced analytics
• Lead, mentor, and develop a team of Data Engineers, establishing technical standards and engineering best practices
• Partner cross-functionally with business and technology stakeholders to promote data excellence, accountability, and continuous innovation
• 10+ years of experience in Data Engineering, Data Architecture, or related fields
• 5+ years of experience leading and developing technical teams
• Deep expertise in SQL, cloud data platforms, and distributed data processing
• Strong hands-on experience with modern cloud data platforms such as:
• Databricks
• Snowflake
• AWS Redshift
• Google BigQuery
• Strong experience with ETL/ELT tools, particularly Alteryx and Informatica
• Strong understanding of data modeling, including dimensional modeling and semantic layers
• Experience with data orchestration, workflow management, and automated data pipelines
• Proven ability to architect and deliver scalable, secure, highly available, and reliable enterprise data solutions
• Strong technical leadership, communication, mentoring, and cross-functional collaboration skill