Data Warehouse Architect

AVM Consulting Inc; Teletrabalho; BRL 160.000 - 200.000; Tempo integral

Resumo da oferta

A prominent gaming consulting firm is looking for a hands-on Data Warehouse Architect with deep expertise in Databricks and AWS data services. This role involves designing a new data warehouse instance and developing scalable data pipelines. The ideal candidate will have strong skills in data lakehouse architecture and AWS-native integration. This opportunity is fully remote, located in Brazil, with a contract lasting over one year.

Qualificações

• Expertise in setting up and managing Databricks workspaces and Unity Catalog.
• Advanced skills in PySpark/SQL for ETL and transformations.
• Strong grasp of modern dimensional modeling (star/snowflake schemas).

Responsabilidades

• Design and deploy new Databricks Lakehouse instance for client needs.
• Architect and implement data ingestion pipelines using Spark.
• Integrate AWS services with Databricks for performance and scalability.

Conhecimentos

Databricks Lakehouse Architecture

AWS Glue

PySpark/SQL

Data Warehousing

CI/CD

Ferramentas

Databricks

AWS

Terraform

CloudWatch

Descrição da oferta de emprego

Databricks Data Warehouse Architect

About the role

One of the largest companies in the world in the Gaming industry is seeking a hands‑on Data Architect with Data Warehouse Engineer expertise in Databricks (DBX) and AWS‑native data services to spearhead the design and implementation of a new data warehouse instance for a major product line. This role will involve building from the ground up—architecting scalable pipelines, optimizing lakehouse performance, and integrating seamlessly with diverse real‑time and batch data sources across AWS.

The ideal candidate is passionate about data architecture, thrives in fast‑moving environments, and has a proven track record of setting up high‑performance lakehouse platforms on Databricks with a strong foundation in data warehousing principles.

Key Responsibilities

• Design and deploy a new Databricks Lakehouse instance tailored to the client’s product‑level data needs.
• Architect and implement robust data ingestion pipelines using Spark (PySpark/Scala) and Delta Lake.
• Integrate AWS‑native services (S3, Glue, Athena, Redshift, Lambda) with Databricks for optimized performance and scalability.
• Define data models, optimize query performance, and establish warehouse governance best practices.
• Collaborate cross‑functionally with product teams, data scientists, and DevOps to streamline data workflows.
• Maintain CI/CD, preferably DBX for data pipelines using GitOps and Infrastructure‑as‑Code.
• Monitor data jobs and resolve performance bottlenecks or failures across environments.

Required Skills & Experience

Databricks / Lakehouse Architecture

• End‑to‑end setup of Databricks workspaces and Unity Catalog
• Expertise in Delta Lake internals, file compaction, and schema enforcement
• Advanced PySpark/SQL skills for ETL and transformations

AWS Native Integration

• Deep experience with AWS Glue, S3, Redshift Spectrum, Lambda, and Athena
• IAM and VPC configuration knowledge for secure cloud integrations

Data Warehousing & Modeling

• Strong grasp of modern dimensional modeling (star/snowflake schemas)
• Experience setting up lakehouse design patterns for mixed workloads

Automation & DevOps

• Familiarity with CI/CD for data engineering using tools like DBX, Terraform, GitHub Actions, or Azure DevOps
• Proficient in monitoring tools like CloudWatch, Datadog, or New Relic for data pipelines

Bonus/Nice to Have

• Experience supporting gaming or real‑time analytics workloads
• Familiarity with Airflow, Kafka, or EventBridge
• Exposure to data privacy and compliance practices (GDPR, CCPA)

Other Details

• Location: Latin America (LATAM) region - Remote, USA - Remote
• Length: 1+ Year

Client: Gaming giant