Lead Data Mesh Architect - Cloud Data Lake Champion

Compunnel Inc.; San Francisco (CA); Remote; USD 120,000 - 160,000; Full time

Job summary

A leading tech solutions company is looking for a Senior Solution Architect to design, implement, and manage scalable data products within a cloud-based ecosystem. This role will involve collaborating with diverse teams to optimize data pipelines and ensure high-quality data management practices. The ideal candidate will possess extensive experience in data engineering, familiar with tools such as Databricks and AWS, and will play a pivotal role in establishing governance and data quality standards.

Qualifications

• 7+ years of experience across the full project lifecycle including design, development, testing, implementation, and production support.
• Advanced expertise with Databricks, Immuta, Starburst, Collibra, and AWS data services.
• Strong background in establishing data quality standards and governance frameworks.

Responsibilities

• Design and implement data mesh and lake solutions in AWS environments.
• Provide technical leadership to data engineering teams on best practices.
• Optimize data pipelines and infrastructure to improve system performance.

Skills

Data engineering

Architectural leadership

Collaboration with cross-functional teams

Analytical problem-solving

Strategic thinking

Education

Bachelor’s degree in computer science, information systems, engineering, or related field

Tools

Databricks

AWS services

Collibra

Immuta

Starburst

Job description

The Senior Solution Architect will be responsible for designing, implementing, and managing data products and scalable data solutions within a modern cloud-based data ecosystem.

This role supports complex, integrated systems across a data lakehouse environment and requires strong expertise in data engineering, analytics platforms, technical strategy, and architectural leadership.

The architect will collaborate with cross-functional teams to deliver scalable, secure, and efficient data solutions using technologies such as Databricks, Starburst, Collibra, Immuta, and AWS services.

Key Responsibilities

• Design and implement data mesh and data lake solutions focused on ingestion, validation, and orchestration using Databricks, Starburst, Collibra, and Immuta within AWS environments.
• Create reusable data management and integration patterns and participate in architectural review processes.
• Provide technical leadership and guidance to data engineering teams on best practices for implementing solutions using the core technology stack.
• Collaborate with cross-functional teams to design scalable data and analytics solutions that deliver measurable business value.
• Establish and enforce data quality standards, governance practices, and data management best practices.
• Optimize data pipelines, queries, and infrastructure to improve system performance.
• Identify, assess, and mitigate risks related to data engineering, ensuring security, contingency, and compliance.
• Lead technical and business discussions regarding future architectural direction.
• Identify organizational needs related to resources, structures, and change management.
• Lead multiple architectural engagements across business units.
• Participate in IT strategy development, opportunity identification, and innovation planning.
• Develop business cases for new architectural initiatives.
• Develop and communicate domain-wide policies, standards, guidelines, and procedures.
• Lead architecture governance processes.
• Conduct research and evaluations for emerging technologies relevant to data platforms.

Required Qualifications

• Bachelor’s degree in computer science, information systems, engineering, or related field, or equivalent experience.
• 7+ years of experience across the full project lifecycle, including design, development, testing, implementation, and production support.
• Advanced expertise with Databricks, Immuta, Starburst, Collibra, and AWS data services.
• Experience designing and implementing data mesh and data lake solutions within cloud environments.
• Strong background in establishing data quality standards, governance frameworks, and data management practices.
• Subject matter expertise across multiple architectural and solution patterns.
• Advanced knowledge of emerging technologies relevant to modern data ecosystems.
• Strong strategic thinking, enterprise consulting skills, and cross-functional collaboration abilities.
• Strong analytical problem-solving skills and ability to communicate complex technical concepts effectively.
• Ability to build strong partnerships and relationships across technical and business teams.

Preferred Qualifications

• Experience with technologies such as Python, R, Alteryx, Tableau, or sparklyr.
• Familiarity with data lakehouse architectures, Delta Lake, MLflow, or Databricks SQL.
• Experience in performance tuning, workload optimization, and cost-efficient architecture design.
• Strong understanding of security architecture including access controls, encryption, and compliance considerations.
• Experience influencing technical direction and guiding teams in complex environments.
• Ability to align architectural solutions with business strategy and outcomes.