Data Architect + AI

Turtle Trax S.A.; United States; À distance; USD 150 000 - 210 000; Plein temps

Résumé du poste

Turtle Trax S.A. is seeking a Data Architect + AI to design end-to-end Lakehouse architectures on Databricks, including Bronze-Silver-Gold pipelines, governance, and scalable data models.

The role requires deep expertise in Spark internals, Delta Lake ACID, Unity Catalog, and cloud infrastructure (AWS/Azure/GCP). Strong SQL and Python/Scala coding skills are essential; experience with GenAI and MLflow a plus.

Qualifications

• Expertise in end-to-end data architectures with Bronze-Silver-Gold layers.
• Deep understanding of distributed computing in Spark and its tuning.
• ACID transactions, schema enforcement, time travel with Delta Lake.

Connaissances

Lakehouse Architecture

Medallion Architecture

Spark Internals

Delta Lake

Unity Catalog

Cloud Infrastructure

SQL

Python or Scala

Cost Optimization

Performance Tuning

Outils

Databricks

Apache Spark

Delta Lake

Unity Catalog

AWS

Azure

GCP

Description du poste

Data Architect + AI

Overall Stack: Databricks Lakehouse Platform, Apache Spark, Delta Lake, Unity Catalog, and modern cloud data architecture

Must Have

• Lakehouse & Medallion Architecture: Expertise in designing end-to-end data architectures (Bronze, Silver, Gold layers) for reliable, production-ready pipelines
• Databricks & Spark Internals: Deep understanding of distributed computing. They must know how to troubleshoot and tune large-scale Spark jobs using caching, partitioning, and broadcast joins
• Delta Lake: Must understand ACID transactions, schema enforcement, time travel, and optimization operations like Z-ordering
• Unity Catalog & Data Governance: Proven ability to design unified governance models for data and AI assets, including role-based access control (RBAC), row/column-level security, and data lineage
• Cloud Infrastructure (AWS, Azure, or GCP): Strong grasp of the native cloud ecosystem they work in (e.g., ADLS/Entra for Azure, S3/IAM for AWS), including Virtual Network (VNet) setups and IAM roles
• Coding Proficiency: Advanced SQL skills and fluency in Python or Scala
• Cost Optimization & Performance Tuning: Ability to monitor DBUs (Databricks Units), right-size serverless and multi-node clusters, and implement best practices for avoiding cloud bill shock

Nice to Have

• Databricks Certifications: Candidates holding valid Databricks Certified Data Architect or Databricks Certified Data Engineer Professional badges generally have a proven, up-to-date baseline of the platform's features
• Generative AI & MLflow Integration: Experience building, deploying, and monitoring GenAI applications and ML models using Databricks Model Serving, Vector Search, and the Mosaic AI suite
• CI/CD & DevOps Practices: Experience automating Databricks workflows using Git (Databricks Repos) and orchestration tools like dbt, Azure Data Factory, or Apache Airflow
• Streaming Data: Familiarity with Databricks Structured Streaming and Auto Loader for real-time data ingestion and processing
• Data Warehousing & BI: Understanding of Databricks SQL, Serverless Warehouses, and integration with downstream BI tools like Power BI

Remote

Adavenced english