Data Architect - Databricks & AWS

KTek Resourcing; EE. UU.; A distancia; USD 150.000 - 190.000; Jornada completa

Descripción de la vacante

KTek Resourcing is seeking an experienced Data Architect to design, build and govern modern data platforms on AWS and Databricks. This remote, full-time role requires hands-on expertise with Delta Lake, Unity Catalog, Spark/PySpark and cloud data services, plus leadership of data engineers and collaboration with business stakeholders.

You will define scalable architectures for ingestion, storage, processing, and consumption, and establish standards for data quality, security, and lineage while

Formación

• 8+ years in data engineering or data architecture roles.
• Hands-on design and implementation of enterprise-scale data platforms.
• Extensive AWS production experience.
• Proven leadership of data engineers and cross-functional collaboration.

Responsabilidades

• Design end-to-end data platforms using AWS and Databricks.
• Define ingestion, storage, processing, and consumption architectures.
• Build scalable ETL/ELT pipelines for high-volume data.
• Lead Databricks environments, clusters, jobs, and workflows.
• Govern data access, security, and lineage via Unity Catalog.
• Collaborate with architects and stakeholders on standards.

Conocimientos

AWS

Databricks

Delta Lake

Unity Catalog

Apache Spark / PySpark

Data Governance

ETL/ELT pipelines

Lakehouse architecture

Herramientas

AWS

Apache Spark

Python

Descripción del empleo

Job Title: Data Architect - AWS & Databricks

Location: Remote

Full Time Position

Role Overview

We are seeking an experienced Data Architect to design, build, and govern modern enterprise data platforms leveraging AWS and Databricks. This role requires a strong combination of data architecture, hands-on data engineering, cloud expertise, and technical leadership.

The ideal candidate will have deep hands-on experience with the AWS and Databricks ecosystems, including Delta Lake, Unity Catalog, Apache Spark/PySpark, data pipelines, orchestration, and cloud data services. This person will also lead and mentor Data Engineers, establish technical standards, participate in architecture decisions, and work closely with business stakeholders and other technical architects.

Key Responsibilities

• Design and architect end-to-end modern data platforms using AWS and Databricks.
• Define architecture for data ingestion, storage, processing, transformation, warehousing, and consumption.
• Design scalable and resilient pipelines for structured, semi-structured, and unstructured data.
• Develop and implement data architectures using Databricks Lakehouse and Delta Lake.
• Define appropriate AWS services for compute, storage, networking, orchestration, security, and data processing.
• Design and implement scalable ETL/ELT data pipelines capable of handling high-volume enterprise data.
• Establish data lake and lakehouse architecture patterns using technologies such as Amazon S3, Databricks, Delta Lake, and AWS Glue.
• Leverage Apache Spark/PySpark for large-scale data processing and transformation.
• Design Databricks environments including clusters, jobs, workflows, notebooks, and production workloads.
• Implement and govern data access, security, cataloging, and lineage using Unity Catalog and related cloud security capabilities.
• Define data architecture standards covering data quality, governance, metadata, lineage, security, and lifecycle management.
• Design data models that support operational reporting, analytics, business intelligence, and downstream data applications.
• Apply appropriate dimensional, canonical, normalized, and denormalized modeling techniques based on business requirements.
• Evaluate and optimize data pipelines and workloads for performance, reliability, scalability, and cost efficiency.
• Establish best practices for data partitioning, indexing, file optimization, caching, workload management, and compute utilization where applicable.
• Collaborate with enterprise, solution, and cloud architects to ensure alignment with broader technology architecture.

Required Qualifications

• 8+ years of experience in data engineering, data architecture, cloud data platforms, or related technical roles.
• Strong hands-on experience designing and implementing enterprise-scale data platforms.
• Extensive experience with AWS cloud services in production environments.
• Strong hands-on experience with Databricks and the Databricks Lakehouse ecosystem.
• Strong understanding of Delta Lake and modern lakehouse architecture.
• Hands-on experience with Unity Catalog, data governance, access control, and data lineage.
• Strong experience with Apache Spark and PySpark for large-scale data processing.
• Experience designing and implementing ETL/ELT pipelines.
• Strong understanding of data ingestion patterns and integration from multiple source systems.
• Experience working with structured, semi-structured, and unstructured data.
• Strong understanding of data warehousing, data lakes, and Lakehouse architecture.