Data Product Engineer Databricks

Compunnel, Inc.; Plano (TX); On-site; USD 150,000 - 190,000; Full time

Job summary

Compunnel, Inc. seeks a seasoned Data Product Engineer to design and implement enterprise Data Product standards within the Databricks ecosystem.

You will lead architecture discussions, collaborate with business stakeholders, and deliver MVP data products spanning ingestion, transformation, quality, and governance. The role emphasizes hands-on Databricks, Delta Lake, Unity Catalog, Spark, Python, SQL, and cloud-native platforms, with opportunities to drive modernization and scalable data

Qualifications

• 10+ years in data engineering/architecture or enterprise data management.

Responsibilities

• Lead the design and implementation of enterprise Data Product standards, architecture patterns, and engineering practices within the Databricks ecosystem.
• Partner with business stakeholders, Risk domain teams, Data Product Owners, and technology leadership to identify high-value data products supporting critical business use cases and analytics outcomes.
• Define and implement reusable data product frameworks, reference architectures, governance controls, metadata standards, and quality requirements.
• Architect and build end-to-end MVP Data Products in Databricks, including data ingestion, transformation, quality controls, metadata management, security, lineage, and consumption layers.
• Develop scalable data pipelines and Lakehouse solutions using Databricks, Delta Lake, Unity Catalog, Spark, and cloud-native services.
• Collaborate with Data Engineers, Data Architects, Governance teams, and platform teams to establish delivery standards supporting product-based data management and domain-oriented ownership.
• Design and implement data product templates, CI/CD patterns, operational monitoring, observability controls, and automated testing frameworks.
• Evaluate existing data assets, business processes, and technology capabilities to identify opportunities for modernization and improved data product adoption.
• Create proof-of-value demonstrations and working MVP solutions to validate architecture decisions, demonstrate business value, and establish implementation patterns.
• Lead technical workshops, architecture reviews, and stakeholder discussions focused on data product design, data contracts, interoperability, discoverability, and consumption patterns.
• Provide technical leadership and mentorship to engineering teams while aligning strategic business objectives with practical delivery execution.
• Support proposal development, solution architecture discussions, effort estimation, and client presentations related to enterprise Data Product and Databricks transformation initiatives.

Skills

Databricks

Delta Lake

Unity Catalog

Spark

Python

SQL

Cloud Platforms

Data Governance

Data Mesh

CI/CD

Data Pipelines

Stakeholder Engagement

Tools

MLflow

Workflows

Databricks Platform

Lakehouse Architecture

Job description

Seeking a Data Product Engineer with 10+ years of experience in Data Engineering, Data Architecture, Analytics Engineering, or Enterprise Data Management. The role focuses on designing and implementing enterprise Data Product standards, architecture patterns, and engineering practices within the Databricks ecosystem. The ideal candidate will have strong hands-on experience with Databricks, Delta Lake, Unity Catalog, Spark, Python, SQL, cloud-native data platforms, and Data Product operating models, along with the ability to work directly with business stakeholders and provide technical leadership.

Key Responsibilities:

• Lead the design and implementation of enterprise Data Product standards, architecture patterns, and engineering practices within the Databricks ecosystem.
• Partner with business stakeholders, Risk domain teams, Data Product Owners, and technology leadership to identify high-value data products supporting critical business use cases and analytics outcomes.
• Define and implement reusable data product frameworks, reference architectures, governance controls, metadata standards, and quality requirements.
• Architect and build end-to-end MVP Data Products in Databricks, including data ingestion, transformation, quality controls, metadata management, security, lineage, and consumption layers.
• Develop scalable data pipelines and Lakehouse solutions using Databricks, Delta Lake, Unity Catalog, Spark, and cloud-native services.
• Collaborate with Data Engineers, Data Architects, Governance teams, and platform teams to establish delivery standards supporting product-based data management and domain-oriented ownership.
• Design and implement data product templates, CI/CD patterns, operational monitoring, observability controls, and automated testing frameworks.
• Evaluate existing data assets, business processes, and technology capabilities to identify opportunities for modernization and improved data product adoption.
• Create proof-of-value demonstrations and working MVP solutions to validate architecture decisions, demonstrate business value, and establish implementation patterns.
• Lead technical workshops, architecture reviews, and stakeholder discussions focused on data product design, data contracts, interoperability, discoverability, and consumption patterns.
• Provide technical leadership and mentorship to engineering teams while aligning strategic business objectives with practical delivery execution.
• Support proposal development, solution architecture discussions, effort estimation, and client presentations related to enterprise Data Product and Databricks transformation initiatives.

Required Qualifications:

• 10+ years of experience in Data Engineering, Data Architecture, Analytics Engineering, or Enterprise Data Management.
• 5+ years of experience designing and implementing modern cloud-based data platforms and data products.
• 4+ years of hands‑on Databricks experience, including Delta Lake, Unity Catalog, Workflows, MLflow, and Lakehouse architecture patterns.
• 5+ years of experience designing scalable data architectures and distributed data processing solutions.
• 4+ years of experience building enterprise data pipelines using Spark, Python, SQL, and cloud-native technologies.
• 3+ years of experience implementing Data Product operating models, Data Mesh concepts, domain‑driven data ownership, or product‑oriented data delivery approaches.
• 3+ years of experience delivering cloud‑based solutions on Azure, AWS, or Google Cloud Platform.
• Experience implementing data governance, metadata management, data quality frameworks, lineage, and security controls within modern data platforms.
• Experience designing reusable engineering standards, architecture patterns, and platform accelerators that support large‑scale enterprise adoption.
• Experience engaging directly with business stakeholders to translate business requirements into scalable data product solutions.
• Strong communication and consulting skills with the ability to lead architecture workshops, executive discussions, and technical solution reviews.