Data Platform Architect

TaskUs; India; On-site; INR 3,000,000 - 6,000,000; Full time

Job summary

TaskUs is seeking a senior data architect to lead platform design for scalable lakehouse environments, enabling high-concurrency access across engines from Databricks to Redshift/Snowflake. The role emphasizes governance, ADR documentation, and cost-aware optimization.

You will mentor engineers, shape CI/CD for pipelines, and communicate complex trade-offs to executives. Ideal candidates bring hands-on experience with Iceberg/Delta Lake, dbt Core, PySpark, and Kubernetes, and possess strong

Qualifications

• Proven track record delivering production-grade Lakehouse environments for high-concurrency organizations.
• Experience with Databricks, Unity Catalog, and high-performance warehouses (Redshift/Snowflake).
• Hands-on with Apache Iceberg or Delta Lake, including partitioning and schema evolution.
• Deep SQL-based modeling (dbt Core) and PySpark/Python data processing.
• Familiarity with vectorized compute options and Open-Table formats.
• Strong orchestration skills (Airflow) and cloud services (AWS/Azure/GCP).
• Ability to translate technical debt and architectural roadmaps for executives.

Responsibilities

• Lead architectural decisions for lakehouse design and multi-engine interoperability.
• Define standards, governance patterns, and CI/CD for data pipelines.
• Mentor engineers and conduct deep-dive reviews of data models and workflows.
• Monitor performance and FinOps to optimize spend while ensuring sub-second queries.
• Document ADRs and translate roadmaps for C-level audiences.

Skills

Data engineering

Data architecture

DataOps

SQL

Python

PySpark

Executive communication

Infrastructure strategy

Tools

Databricks

Amazon Redshift

Snowflake

Delta Lake

Iceberg

dbt Core

Apache Airflow

Kubernetes

Polaris

DuckDB

Unity Catalog

Job description

• Platform Design & Research: Lead architectural decisions regarding compute engine selection, open-table format implementation, and tiered storage design.
• Agnostic Infrastructure: Architect a decoupled data environment that ensures interoperability across multiple engines and prevents proprietary vendor lock-in.
• Governance & Compliance: Design and oversee the implementation of automated data governance, including PII discovery, row/column-level security, and auditability.
• Standards & Frameworks: Define the "Definition of Done" for data pipelines, establishing coding standards, CI/CD patterns, and technical documentation requirements.
• Architectural Decision Records (ADR): Maintain a version-controlled repository of all consequential technical decisions, documenting the rationale, trade-offs, and long-term implications.
• Performance & FinOps: Monitor and optimize platform performance and spend, ensuring sub-second query speeds for massive user bases while maintaining a lean cloud footprint.
• Technical Stewardship: Conduct deep-dive code and design reviews for all data models and orchestration workflows; mentor and unblock senior engineering staff.

Technical Skills & Experience

• :8+ Years in Data Engineering / Architecture: Proven experience delivering production-grade Lakehouse environments for high-concurrency (1,000+ user) organizations
• .Modern Data Stack Fluency: Extensive experience with Databricks (Lakehouse/Unity Catalog) and high-performance warehouses like Amazon Redshift or Snowflake
• .Open-Table Formats: Deep hands-on expertise with Apache Iceberg or Delta Lake, including optimization strategies for partitioning and schema evolution
• .Transformation & Modeling: Mastery of dbt (Core) for complex SQL-based modeling and PySpark or Python for sophisticated data processing
• .High-Efficiency Compute: Familiarity with vectorized/embedded engines like DuckDB for specialized or cost-sensitive processing tasks
• .Orchestration Mastery: Advanced experience with Apache Airflow, specifically in designing resilient, dependency-aware DAGs in resource-constrained environments
• .Cloud Ecosystems: Expert-level knowledge of AWS (S3, EC2, IAM) or equivalent services in Azure/GCP, with a focus on storage-compute separation
• .Experience with open-source catalog implementations like Apache Polaris
• .Knowledge of Data Ops principles and automated data quality testing frameworks
• .Experience translating technical debt and architectural roadmaps for C-level executives
• .Background in managing fixed-resource infrastructure (e.g., EC2/VM-based processing) vs. elastic serverless models
• .Kubernetes (K8s). Deep understanding of Pods, Deployments, Services, ConfigMaps, and Secrets management