Senior Data Engineer: Azure Data Pipelines & PySpark Expert

Tata Consultancy Services; Chicago (IL); On-site; USD 100,000 - 120,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Comprehensive Medical Coverage

Maternal & Parental Leaves

Auto & Home Insurance

401K Plan

Job summary

Tata Consultancy Services is seeking a skilled developer in Chicago, Illinois to design and build end-to-end data pipelines. Candidates should have a Bachelor's degree in Computer Science and expertise in SQL, PySpark, and Azure Data Factory.

The role involves optimizing SQL and Python code, working in Azure environments, and ensuring production- grade quality. This full-time position offers a competitive salary ranging from $100,000 to $120,000 per year, along with comprehensive medical coverage and various employee benefits.

Responsibilities

• Design and build end-to-end data pipelines.
• Develop and optimize SQL and PySpark transformations.
• Build production-grade Python components.
• Create and maintain ADF pipelines.
• Work within Azure environments.
• Provision Azure resources using Pulumi.
• Participate in code reviews, documentation, and operational support.

Skills

ETL/ELT concepts

Advanced SQL

Python

PySpark

Azure Data Factory

Databricks

Azure

Pulumi

Azure API Services

Azure Function Apps

Education

Bachelor of Computer Science

Job description

Tata Consultancy Services is seeking a skilled developer in Chicago, Illinois to design and build end-to-end data pipelines. Candidates should have a Bachelor's degree in Computer Science and expertise in SQL, PySpark, and Azure Data Factory.

The role involves optimizing SQL and Python code, working in Azure environments, and ensuring production- grade quality. This full-time position offers a competitive salary ranging from $100,000 to $120,000 per year, along with comprehensive medical coverage and various employee benefits.