Data Platform Engineer: Databricks, Governance & Real-time ML Ops

AgenticBricks.com; Seattle (WA); On-site; USD 160,000 - 210,000; Full time

Job summary

AgenticBricks.com is seeking an experienced Senior Data Platform Engineer to design and operate enterprise data architectures on Databricks with Unity Catalog, Privacera, and Collibra. You will build governed ETL pipelines, maintain data models, and operationalize ML in batch and real-time while ensuring compliance.

Responsibilities include designing architectures, developing data pipelines, maintaining models, enforcing governance, and collaborating with data scientists, engineers, and analysts

Qualifications

• 10+ years data platform or data engineering experience.
• Hands-on expertise with Databricks ETL pipelines, Delta Lake, and performance tuning.
• Production experience with Unity Catalog for access control and data governance.
• Privacera experience for data security, privacy, and regulatory compliance.
• Collibra experience for metadata management, lineage, and data catalog operations.
• Data modeling and schema design across batch and streaming workloads.
• MLOps: operationalizing ML models in batch and real-time pipelines with governance controls.
• Strong collaboration with data scientists, engineers, and analysts.

Responsibilities

• Design and implement data architecture using Databricks, Unity Catalog, Privacera, and Collibra
• Develop, optimize, and manage data pipelines for ETL processes on Databricks, ensuring data integrity and quality
• Design and maintain data models and schemas, incorporating Unity Catalog and Collibra governance practices
• Ensure data security and regulatory compliance using Databricks and Privacera features
• Establish and enforce data governance standards including metadata management, lineage, and data quality
• Operationalize Machine Learning models in batch and real-time data pipelines with governance controls
• Collaborate with cross-functional teams including data scientists, engineers, and analysts to deliver scalable data solutions

Skills

Databricks

Delta Lake

Data governance

Data modeling

MLOps

Cross-functional collaboration

Tools

Unity Catalog

Privacera

Collibra

Job description

AgenticBricks.com is seeking an experienced Senior Data Platform Engineer to design and operate enterprise data architectures on Databricks with Unity Catalog, Privacera, and Collibra. You will build governed ETL pipelines, maintain data models, and operationalize ML in batch and real-time while ensuring compliance.

Responsibilities include designing architectures, developing data pipelines, maintaining models, enforcing governance, and collaborating with data scientists, engineers, and analysts