BI / Data Architect

Tata Consultancy Services; Alpharetta (GA); On-site; USD 150,000 - 160,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Medical Coverage

Parental Leaves

Auto & Home Insurance

Commuter Benefits

Certification Reimbursement

Vacation & Holidays

401K Plan

Job summary

Tata Consultancy Services is seeking a data platform engineer to shape our enterprise lakehouse strategy in Alpharetta. You will design Bronze-Silver-Gold layers, build ingestion frameworks, and drive scalable data pipelines using Spark, Kafka, and dbt.

You will implement metadata governance, RBAC, encryption, and performance optimizations, enabling AI-ready semantic layers and GenAI workloads. Expect cloud-based, CI/CD, and observability practices.

Qualifications

• Experience with data lakehouse architectures and governance.
• Proficiency in Spark, Kafka, dbt, and cloud data services.
• Ability to design scalable data pipelines and metadata management.

Responsibilities

• Define enterprise lakehouse architecture, standards, governance, and implementation roadmap.
• Design and build Apache Iceberg-based Bronze, Silver, and Gold data layers.
• Develop ingestion frameworks for batch, streaming, CDC, APIs, and event-driven integrations.
• Build scalable Spark, Flink, Kafka, dbt, SQL, and Python pipelines.
• Implement metadata management, cataloging, lineage, RBAC, encryption, and data quality.
• Optimize partitioning, compaction, snapshots, metadata, and query performance.
• Enable AI-ready semantic layers, feature engineering, RAG, and GenAI workloads.
• Implement Infrastructure as Code, CI/CD, monitoring, and observability.

Skills

Cloud platforms

Python/Scala/SQL/Java

Airflow/Dagster/Prefect

Data lakehouse concepts

Tools

Apache Iceberg

Spark

Flink

Kafka

dbt

Trino

Snowflake

Databricks

Kubernetes

Docker

Terraform

Job description

Job Description

• AWS, Azure, or Google Cloud
• Python, Scala, SQL, Java
• Airflow, Dagster, or Prefect
• Polaris, Unity Catalog, Glue Catalog, Apache Nessie
• Define enterprise lakehouse architecture, standards, governance, and implementation roadmap.
• Design and build Apache Iceberg-based Bronze, Silver, and Gold data layers.
• Develop ingestion frameworks for batch, streaming, CDC, APIs, and event-driven integrations.
• Build scalable Spark, Flink, Kafka, dbt, SQL, and Python pipelines.
• Implement metadata management, cataloging, lineage, RBAC, encryption, and data quality.
• Optimize partitioning, compaction, snapshots, metadata, and query performance.
• Enable AI-ready semantic layers, feature engineering, RAG, and GenAI workloads.
• Implement Infrastructure as Code, CI/CD, monitoring, and observability.

Must Have Technical/Functional Skills

• Apache Iceberg, Spark, Flink, Kafka, dbt, Trino, Snowflake, Databricks
• AWS, Azure, or Google Cloud
• Python, Scala, SQL, Java
• Kubernetes, Docker, Terraform
• Airflow, Dagster, or Prefect
• Polaris, Unity Catalog, Glue Catalog, Apache Nessie

Roles & Responsibilities

• Define enterprise lakehouse architecture, standards, governance, and implementation roadmap.
• Design and build Apache Iceberg-based Bronze, Silver, and Gold data layers.
• Develop ingestion frameworks for batch, streaming, CDC, APIs, and event-driven integrations.
• Build scalable Spark, Flink, Kafka, dbt, SQL, and Python pipelines.
• Implement metadata management, cataloging, lineage, RBAC, encryption, and data quality.
• Optimize partitioning, compaction, snapshots, metadata, and query performance.
• Enable AI-ready semantic layers, feature engineering, RAG, and GenAI workloads.
• Implement Infrastructure as Code, CI/CD, monitoring, and observability.

TCS Employee Benefits Summary

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Aut& Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Salary Range-$150,000-$160,000 a year