Data Warehouse Specialist

Tata Consultancy Services; Chicago (IL); On-site; USD 100,000 - 130,000; Full time

Benefits offered by this job

Discretionary incentive

Medical coverage

Parental leave

401K plan

Training reimbursement

Job summary

Tata Consultancy Services is seeking a Senior Data Engineer to architect and optimize enterprise data pipelines across cloud, on‑prem, and hybrid environments.

Ideal candidates have 8+ years in data engineering, deep Snowflake and Qlik Replicate experience, and strong Python/PySpark skills for ingestion and automation.

Qualifications

• 8+ years of hands-on data engineering experience.
• Deep expertise with Snowflake features, security, and performance tuning.
• Strong Python and PySpark for ingestion frameworks and automation.
• Proven experience with Qlik Replicate for CDC and data replication.
• Experience with DBT Cloud and Astronomer Airflow.

Responsibilities

• Design and implement scalable, resilient data pipelines across cloud and on‑prem environments.
• Architect data pipelines leveraging Snowflake features (Snowpipe, Tasks, Streams).
• Develop Python ingestion frameworks with schema validation and metadata capture.
• Coordinate CI/CD workflows and governance for data pipelines.

Skills

Python

Pyspark

CI/CD

Data governance

RBAC

Tools

Snowflake

Qlik Replicate

DBT Cloud

Astronomer Airflow

Job description

• Job description: We are seeking a highly skilled Senior Data Engineer to architect,
• build, and optimize enterprise grade data pipelines across cloud, on prem, and hybrid
• environments. This role requires deep expertise in Qlik Replicate, Snowflake, DBT Cloud,
• Astronomer Airflow, and Python based ingestion frameworks, with strong engineering
• discipline around schema governance, DevOps CI/CD, monitoring, and performance
• optimization. The ideal candidate thrives in complex data ecosystems and brings a
• strong mindset around automation, metadata driven design, and secure, governed ingestion.
• Responsibilities:
• Design and implement scalable, resilient data pipelines using Snowflake features

Job Description

Role - Data Warehouse Specialist

Experience Required -5+ Years

• Job description: We are seeking a highly skilled Senior Data Engineer to architect,
• build, and optimize enterprise grade data pipelines across cloud, on prem, and hybrid
• environments. This role requires deep expertise in Qlik Replicate, Snowflake, DBT Cloud,
• Astronomer Airflow, and Python based ingestion frameworks, with strong engineering
• discipline around schema governance, DevOps CI/CD, monitoring, and performance
• optimization. The ideal candidate thrives in complex data ecosystems and brings a
• strong mindset around automation, metadata driven design, and secure, governed ingestion.
• Responsibilities:
• Data Pipeline Architecture & Development
• Design and implement scalable, resilient data pipelines using Snowflake features

including Snowpipe, Tasks, Streams, Dynamic Tables, and advanced SQL.

• Build and maintain DBT models with strong testing, documentation, and lineage.
• Develop Python ingestion frameworks for files and APIs, including schema validation, retries, and metadata capture.

• Engineer ingestion for CSV, fixed width multi record layouts, JSON, XML, Excel, and semi structured formats.

• Design Mainframe VSAM data ingestion pattern for complex EBCDIC data formats.
• Schema Drift & Schema Evolution
• Detect, analyze, and manage schema drift across file, API, and replicated database sources.
• Implement metadata driven schema evolution strategies to ensure downstream stability.
• Coordinate schema changes through controlled CI/CD workflows.
• Database Replication & CDC
• Configure and manage Qlik Replicate tasks for CDC and full load replication from Oracle,
• SQL Server, and DB2.
• Ensure idempotent, auditable, and recoverable replication pipelines with strong monitoring and reconciliation.
• Data Governance, Security & Tokenization
• Implement and maintain Snowflake Data Masking policies, including dynamic masking, conditional masking, and role based masking rules.
• Apply Protegrity tokenization for sensitive data fields across ingestion and transformation layers.

• Enforce RBAC, data access controls, and governance standards across Snowflake and supporting systems.
• Orchestration & Automation
• Build and schedule workflows using Astronomer Airflow, ensuring dependency management, retries, SLAs, and observability.
• Integrate pipelines with enterprise DevOps processes using GitLab and Azure DevOps for CI/CD automation.
• Version Control & Code Quality
• Manage code repositories using GitLab, including branching strategies, merge requests, code reviews, and approvals.
• Monitoring, Alerting & Performance Optimization
• Implement monitoring and alerting for ingestion pipelines, schema drift, replication, and transformation workloads.
• Optimize Snowflake compute, storage, and query performance; scale ingestion pipelines to meet evolving data volume and latency requirements.

• Required Skills & Experience:
• 8+ years of hands on data engineering experience.
• Deep expertise with Snowflake, including data masking policies, RBAC, performance tuning, and advanced SQL.
• Strong experience with Qlik Replicate for CDC and database replication.
• Excellent proficiency in Python and Pyspark for ingestion frameworks and automation.
• Hands on experience with DBT Cloud and Astronomer Airflow.
• Experience with schema drift detection and schema evolution patterns.
• Experience with GitLab and CI/CD pipelines.
• Familiarity with Protegrity or similar data protection platforms.

Base Salary Range : $100,000 to $130,000 Per Annum

TCS Employee Benefits Summary

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Auto & Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.