Senior Data Architect - AWS & Databricks

KTek Resourcing LLC; United States; Remote; USD 150,000 - 190,000; Full time

Job summary

KTek Resourcing LLC is seeking an experienced Data Architect to design, build, and govern modern enterprise data platforms leveraging AWS and Databricks. The role requires hands-on data engineering, cloud expertise, and technical leadership, including Delta Lake, Unity Catalog, and Spark/PySpark.

You will lead Data Engineers, set technical standards, and collaborate with stakeholders on architecture decisions. Remote, full-time, with significant scope across AWS services and Databricks Lakehouse.

Qualifications

• 8+ years in data engineering or data architecture.
• Extensive hands-on experience with AWS production environments.
• Strong Databricks Lakehouse and Delta Lake expertise.
• Experience with Unity Catalog, data governance, and data lineage.
• Proficiency with Spark/PySpark for large-scale processing.
• Proven ability to design end-to-end data platforms and pipelines.

Responsibilities

• Design end-to-end data platforms using AWS and Databricks.
• Define data ingestion, storage, processing, warehousing, and consumption architecture.
• Build scalable ETL/ELT pipelines for diverse data types.
• Lead data engineers and establish technical standards.
• Govern data access, security, cataloging, and lineage with Unity Catalog.
• Collaborate with architects to ensure alignment with enterprise roadmaps.

Skills

AWS

Databricks

Delta Lake

Unity Catalog

Apache Spark

PySpark

Data Pipelines

ETL/ELT

Data Governance

Data Security

Tools

Databricks Lakehouse

AWS Glue

Amazon S3

Spark UI

Job description

Job Title: Data Architect - AWS & Databricks

Location: Remote

Full Time

Role Overview

We are seeking an experienced Data Architect to design, build, and govern modern enterprise data platforms leveraging AWS and Databricks. This role requires a strong combination of data architecture, hands-on data engineering, cloud expertise, and technical leadership.

The ideal candidate will have deep hands-on experience with the AWS and Databricks ecosystems, including Delta Lake, Unity Catalog, Apache Spark/PySpark, data pipelines, orchestration, and cloud data services. This person will also lead and mentor Data Engineers, establish technical standards, participate in architecture decisions, and work closely with business stakeholders and other technical architects.

Key Responsibilities

• Design and architect end-to-end modern data platforms using AWS and Databricks.
• Define architecture for data ingestion, storage, processing, transformation, warehousing, and consumption.
• Design scalable and resilient pipelines for structured, semi-structured, and unstructured data.
• Develop and implement data architectures using Databricks Lakehouse and Delta Lake.
• Define appropriate AWS services for compute, storage, networking, orchestration, security, and data processing.
• Design and implement scalable ETL/ELT data pipelines capable of handling high-volume enterprise data.
• Establish data lake and lakehouse architecture patterns using technologies such as Amazon S3, Databricks, Delta Lake, and AWS Glue.
• Leverage Apache Spark/PySpark for large-scale data processing and transformation.
• Design Databricks environments including clusters, jobs, workflows, notebooks, and production workloads.
• Implement and govern data access, security, cataloging, and lineage using Unity Catalog and related cloud security capabilities.
• Define data architecture standards covering data quality, governance, metadata, lineage, security, and lifecycle management.
• Design data models that support operational reporting, analytics, business intelligence, and downstream data applications.
• Apply appropriate dimensional, canonical, normalized, and denormalized modeling techniques based on business requirements.
• Evaluate and optimize data pipelines and workloads for performance, reliability, scalability, and cost efficiency.
• Establish best practices for data partitioning, indexing, file optimization, caching, workload management, and compute utilization where applicable.
• Collaborate with enterprise, solution, and cloud architects to ensure alignment with broader technology architecture.

Required Qualifications

• 8+ years of experience in data engineering, data architecture, cloud data platforms, or related technical roles.
• Strong hands-on experience designing and implementing enterprise-scale data platforms.
• Extensive experience with AWS cloud services in production environments.
• Strong hands-on experience with Databricks and the Databricks Lakehouse ecosystem.
• Strong understanding of Delta Lake and modern lakehouse architecture.
• Hands-on experience with Unity Catalog, data governance, access control, and data lineage.
• Strong experience with Apache Spark and PySpark for large-scale data processing.
• Experience designing and implementing ETL/ELT pipelines.
• Strong understanding of data ingestion patterns and integration from multiple source systems.
• Experience working with structured, semi-structured, and unstructured data.
• Strong understanding of data warehousing, data lakes, and Lakehouse architecture.