Chicago-based Senior Data Engineer: Snowflake & Pipelines

Tata Consultancy Services Limited; Chicago (IL); On-site; USD 100,000 - 130,000; Full time

Benefits offered by this job

Annual incentive

Medical coverage

Family support

Insurance options

Training reimbursement

Vacation & holidays

401K plan

Job summary

Tata Consultancy Services Limited in Chicago, IL is seeking a Senior Data Engineer to architect, build, and optimize enterprise-grade data pipelines across cloud, on-prem, and hybrid environments. The role emphasizes Snowflake, Qlik Replicate, DBT Cloud, Astronomer Airflow, and Python-based ingestion frameworks.

You will implement schema governance, CI/CD, monitoring, and performance tuning, with a focus on automation and secure, metadata-driven ingestion.

Qualifications

• 8+ years hands-on data engineering experience.
• Deep expertise with Snowflake, including masking, RBAC, performance tuning.
• Strong experience with Qlik Replicate for CDC and replication.
• Excellent proficiency in Python and PySpark for ingestion frameworks.
• Hands on experience with DBT Cloud and Astronomer Airflow.
• Experience with schema drift detection and evolution patterns.
• Experience with GitLab and CI/CD pipelines.
• Familiarity with Protegrity or similar data protection platforms.

Responsibilities

• Design and implement scalable, resilient data pipelines using Snowflake features.
• Build and maintain DBT models with testing, documentation, and lineage.
• Develop Python ingestion frameworks for files and APIs, incl. schema validation.
• Ingest CSV, fixed-width, JSON, XML, Excel and semi-structured formats.
• Design Mainframe VSAM ingestion for complex EBCDIC data formats.
• Detect, analyze, and manage schema drift; implement metadata-driven evolution.
• Configure Qlik Replicate tasks for CDC and full load replication.
• Implement Snowflake Data Masking and Protegrity tokenization; enforce RBAC.
• Build and schedule workflows with Astronomer Airflow; integrate CI/CD.

Skills

Snowflake

Qlik Replicate

Python

PySpark

DBT Cloud

Astronomer Airflow

GitLab CI/CD

Protegrity

Schema drift

CI/CD pipelines

Tools

Snowflake

Qlik Replicate

DBT Cloud

Astronomer Airflow

Python

PySpark

GitLab

Azure DevOps

Protegrity

Snowpipe/Streams

Job description

Tata Consultancy Services Limited in Chicago, IL is seeking a Senior Data Engineer to architect, build, and optimize enterprise-grade data pipelines across cloud, on-prem, and hybrid environments. The role emphasizes Snowflake, Qlik Replicate, DBT Cloud, Astronomer Airflow, and Python-based ingestion frameworks.

You will implement schema governance, CI/CD, monitoring, and performance tuning, with a focus on automation and secure, metadata-driven ingestion.