Data Warehouse Specialist

Tata Consultancy Services; Chicago (IL); On-site; USD 100,000 - 130,000; Full time

Benefits offered by this job

Annual incentive

Medical coverage

Family support

Insurance options

Training reimbursement

Vacation & holidays

401K plan

Job summary

Tata Consultancy Services Limited in Chicago, IL is seeking a Senior Data Engineer to architect, build, and optimize enterprise-grade data pipelines across cloud, on-prem, and hybrid environments. The role emphasizes Snowflake, Qlik Replicate, DBT Cloud, Astronomer Airflow, and Python-based ingestion frameworks.

You will implement schema governance, CI/CD, monitoring, and performance tuning, with a focus on automation and secure, metadata-driven ingestion.

Qualifications

• 8+ years hands-on data engineering experience.
• Deep expertise with Snowflake, including masking, RBAC, performance tuning.
• Strong experience with Qlik Replicate for CDC and replication.
• Excellent proficiency in Python and PySpark for ingestion frameworks.
• Hands on experience with DBT Cloud and Astronomer Airflow.
• Experience with schema drift detection and evolution patterns.
• Experience with GitLab and CI/CD pipelines.
• Familiarity with Protegrity or similar data protection platforms.

Responsibilities

• Design and implement scalable, resilient data pipelines using Snowflake features.
• Build and maintain DBT models with testing, documentation, and lineage.
• Develop Python ingestion frameworks for files and APIs, incl. schema validation.
• Ingest CSV, fixed-width, JSON, XML, Excel and semi-structured formats.
• Design Mainframe VSAM ingestion for complex EBCDIC data formats.
• Detect, analyze, and manage schema drift; implement metadata-driven evolution.
• Configure Qlik Replicate tasks for CDC and full load replication.
• Implement Snowflake Data Masking and Protegrity tokenization; enforce RBAC.
• Build and schedule workflows with Astronomer Airflow; integrate CI/CD.

Skills

Snowflake

Qlik Replicate

Python

PySpark

DBT Cloud

Astronomer Airflow

GitLab CI/CD

Protegrity

Schema drift

CI/CD pipelines

Tools

Snowflake

Qlik Replicate

DBT Cloud

Astronomer Airflow

Python

PySpark

GitLab

Azure DevOps

Protegrity

Snowpipe/Streams

Job description

Job Description

Role - Data Warehouse Specialist

Experience Required -5+ Years

Must Have Technical/Functional Skills

We are seeking a highly skilled Senior Data Engineer to architect, build, and optimize enterprise grade data pipelines across cloud, on prem, and hybrid environments. This role requires deep expertise in Qlik Replicate, Snowflake, DBT Cloud, Astronomer Airflow, and Python based ingestion frameworks, with strong engineering discipline around schema governance, DevOps CI/CD, monitoring, and performance optimization. The ideal candidate thrives in complex data ecosystems and brings a strong mindset around automation, metadata driven design, and secure, governed ingestion.

Responsibilities:

• Data Pipeline Architecture & Development
  • Design and implement scalable, resilient data pipelines using Snowflake features including Snowpipe, Tasks, Streams, Dynamic Tables, and advanced SQL.
  • Build and maintain DBT models with strong testing, documentation, and lineage.
  • Develop Python ingestion frameworks for files and APIs, including schema validation, retries, and metadata capture.
  • Engineer ingestion for CSV, fixed width multi record layouts, JSON, XML, Excel, and semi structured formats.
  • Design Mainframe VSAM data ingestion pattern for complex EBCDIC data formats.
• Schema Drift & Schema Evolution
  • Detect, analyze, and manage schema drift across file, API, and replicated database sources.
  • Implement metadata driven schema evolution strategies to ensure downstream stability.
  • Coordinate schema changes through controlled CI/CD workflows.
• Database Replication & CDC
  • Configure and manage Qlik Replicate tasks for CDC and full load replication from Oracle, SQL Server, and DB2.
  • Ensure idempotent, auditable, and recoverable replication pipelines with strong monitoring and reconciliation.
• Data Governance, Security & Tokenization
  • Implement and maintain Snowflake Data Masking policies, including dynamic masking, conditional masking, and role based masking rules.
  • Apply Protegrity tokenization for sensitive data fields across ingestion and transformation layers.
  • Enforce RBAC, data access controls, and governance standards across Snowflake and supporting systems.
• Orchestration & Automation
  • Build and schedule workflows using Astronomer Airflow, ensuring dependency management, retries, SLAs, and observability.
  • Integrate pipelines with enterprise DevOps processes using GitLab and Azure DevOps for CI/CD automation.
• Version Control & Code Quality
  • Manage code repositories using GitLab, including branching strategies, merge requests, code reviews, and approvals.
• Monitoring, Alerting & Performance Optimization
  • Implement monitoring and alerting for ingestion pipelines, schema drift, replication, and transformation workloads.
  • Optimize Snowflake compute, storage, and query performance; scale ingestion pipelines to meet evolving data volume and latency requirements.

Required Skills & Experience:

• 8+ years of hands on data engineering experience.
• Deep expertise with Snowflake, including data masking policies, RBAC, performance tuning, and advanced SQL.
• Strong experience with Qlik Replicate for CDC and database replication.
• Excellent proficiency in Python and Pyspark for ingestion frameworks and automation.
• Hands on experience with DBT Cloud and Astronomer Airflow.
• Experience with schema drift detection and schema evolution patterns.
• Experience with GitLab and CI/CD pipelines.
• Familiarity with Protegrity or similar data protection platforms.

Base Salary Range : $100,000 to $130,000 Per Annum

TCS Employee Benefits Summary:

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Auto & Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Location

Chicago, IL

Function

TECHNOLOGY

Role

Specialist

ID

433108

Skills

SNOW

Salary Range

$100,000 - $130,000