Staff Data Platform Engineer - Data Discovery & Governance

TrendMicro; Austin (TX); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401(k) with company match

Paid Time Off

Parental leave

Job summary

TrendAI is seeking a Staff Data Engineer to design and build the Vision One Data Platform, enabling data discovery, governance, and self-service data transformation with dbt across OLTP sources. You will shape data movement, lineage, and semantic discovery while collaborating with security and platform teams.

The role requires deep expertise in distributed SQL, CDC, and large-scale data processing using Databricks, Spark, and modern analytics engines.

Qualifications

• 8+ years of experience in data engineering, distributed systems, backend engineering, or a related field.
• Strong experience designing and building large-scale data platforms and data pipelines.
• Deep understanding of OLTP and OLAP architectures and tradeoffs.
• Hands-on experience with CDC, data replication, and streaming architectures.
• Strong SQL expertise and experience with relational databases at scale.

Responsibilities

• Design and build the Vision One Data Platform, including data ingestion, replication, transformation, governance, discovery, and serving capabilities.
• Build scalable data pipelines and CDC architectures using technologies such as Datastream and similar streaming/replication technologies.
• Enable self-service dbt across Vision One OLTP data sources, providing governed capabilities to transform, model, test, document, and expose data.
• Establish patterns for safely extracting and replicating data from OLTP and distributed SQL databases into analytical and downstream data systems.
• Design architectures around distributed SQL technologies such as YugabyteDB, CockroachDB, or similar platforms.
• Work with Databricks and modern analytical processing engines to support large-scale data processing and analytics.
• Build a comprehensive data catalog and metadata foundation for data understanding and reuse.
• Develop semantic data discovery capabilities enabling data discovery by business meaning.
• Establish data governance standards covering ownership, lineage, schemas, quality, access control, retention, and lifecycle management.
• Develop reusable data platform primitives and APIs for self-service consumption.
• Build automated mechanisms for data quality, schema evolution, lineage, observability, and freshness monitoring.
• Design for evolving product schemas and manage changes without duplication.
• Partner with security and platform teams for fine-grained data access controls.
• Help establish source-of-truth architecture and minimize duplication.
• Provide technical leadership and architectural guidance to other data engineers and product teams.
• Drive standards around reliability, scalability, performance, cost efficiency, and operational excellence.

Skills

SQL expertise

CDC experience

Data pipelines

Distributed systems

API design

CI/CD

Observability

Leadership

Tools

Datastream

dbt

Databricks

Spark

Trino

Snowflake

BigQuery

YugabyteDB

CockroachDB

Job description

TrendAI is seeking a Staff Data Engineer to design and build the Vision One Data Platform, enabling data discovery, governance, and self-service data transformation with dbt across OLTP sources. You will shape data movement, lineage, and semantic discovery while collaborating with security and platform teams.

The role requires deep expertise in distributed SQL, CDC, and large-scale data processing using Databricks, Spark, and modern analytics engines.