BI / Data Architect

Tata Consultancy Services; Marlborough (MA); On-site; USD 150,000 - 160,000; Full time

Benefits offered by this job

Discretionary annual incentive

Comprehensive medical coverage

Parental leave

Insurance options

Training reimbursement

Vacation and holidays

401K plan

Job summary

Tata Consultancy Services in Marlborough, MA is hiring a data platform engineer to define enterprise lakehouse standards, governance, and an implementation roadmap.

You will design and build Iceberg-based Bronze, Silver, and Gold layers, develop ingestion frameworks, and create scalable pipelines with Spark, Flink, Kafka, dbt, SQL, and Python while enabling AI-ready semantic layers and robust metadata management.

Qualifications

• Experience with Iceberg-based architectures and data lakehouse governance.
• Proficiency with Spark, Flink, Kafka, and dbt tooling in cloud environments.
• Strong programming skills in Python/Scala/SQL/Java.
• Hands-on with orchestration tools (Airflow, Dagster, Prefect).
• Familiarity with data catalogs (Polaris, Unity Catalog, Glue Catalog, Nessie).
• Experience with cloud platforms (AWS/Azure/Google Cloud).

Responsibilities

• Define enterprise lakehouse architecture, standards, governance, and implementation roadmap.
• Design and build Iceberg-based Bronze, Silver, and Gold data layers.
• Develop ingestion frameworks for batch, streaming, CDC, APIs, and event-driven integrations.
• Build scalable Spark, Flink, Kafka, dbt, SQL, and Python pipelines.
• Implement metadata management, cataloging, lineage, RBAC, encryption, and data quality.
• Optimize partitioning, compaction, snapshots, metadata, and query performance.
• Enable AI-ready semantic layers, feature engineering, RAG, and GenAI workloads.
• Implement Infrastructure as Code, CI/CD, monitoring, and observability.

Skills

Apache Iceberg

Spark

Flink

Kafka

dbt

Trino

Snowflake

Databricks

AWS/Azure/Google Cloud

Python/Scala/SQL/Java

Kubernetes

Docker

Terraform

Airflow/Dagster/Prefect

Polaris/Unity Catalog/Glue Catalog/Ap.

Tools

Kubernetes

Docker

Terraform

Job description

Job Description

• AWS, Azure, or Google Cloud
• Python, Scala, SQL, Java
• Airflow, Dagster, or Prefect
• Polaris, Unity Catalog, Glue Catalog, Apache Nessie
• Define enterprise lakehouse architecture, standards, governance, and implementation roadmap.
• Design and build Apache Iceberg-based Bronze, Silver, and Gold data layers.
• Develop ingestion frameworks for batch, streaming, CDC, APIs, and event-driven integrations.
• Build scalable Spark, Flink, Kafka, dbt, SQL, and Python pipelines.
• Implement metadata management, cataloging, lineage, RBAC, encryption, and data quality.
• Optimize partitioning, compaction, snapshots, metadata, and query performance.
• Enable AI-ready semantic layers, feature engineering, RAG, and GenAI workloads.
• Implement Infrastructure as Code, CI/CD, monitoring, and observability.

Must Have Technical/Functional Skills

• Apache Iceberg, Spark, Flink, Kafka, dbt, Trino, Snowflake, Databricks
• AWS, Azure, or Google Cloud
• Python, Scala, SQL, Java
• Kubernetes, Docker, Terraform
• Airflow, Dagster, or Prefect
• Polaris, Unity Catalog, Glue Catalog, Apache Nessie

Roles & Responsibilities

• Define enterprise lakehouse architecture, standards, governance, and implementation roadmap.
• Design and build Apache Iceberg-based Bronze, Silver, and Gold data layers.
• Develop ingestion frameworks for batch, streaming, CDC, APIs, and event-driven integrations.
• Build scalable Spark, Flink, Kafka, dbt, SQL, and Python pipelines.
• Implement metadata management, cataloging, lineage, RBAC, encryption, and data quality.
• Optimize partitioning, compaction, snapshots, metadata, and query performance.
• Enable AI-ready semantic layers, feature engineering, RAG, and GenAI workloads.
• Implement Infrastructure as Code, CI/CD, monitoring, and observability.

TCS Employee Benefits Summary

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Aut& Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Salary Range-$150,000-$160,000 a year