Senior Engineer

Tata Consultancy Services; West Chester (Chester County); On-site; USD 90,000 - 120,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Medical Coverage

Family Leave

Commuter Benefits

Training Reimbursement

Paid Time Off

401K & Financial Benefits

Job summary

Tata Consultancy Services in the United States is seeking a Data Engineer to design and build robust ETL/ELT pipelines using PySpark and SQL. You will ingest data from APIs, flat files, and relational sources, and work with Snowflake data models.

Responsibilities include data modeling with star/snowflake schemas, performance tuning of Snowflake queries and Spark jobs, and implementing automated data quality checks. The role requires hands-on experience with Teradata and version control in Git.

Qualifications

• Snowflake expertise required.
• Strong PySpark with Python for distributed processing.
• Advanced proficiency in Teradata SQL.
• Experience designing ETL/ELT pipelines and data modeling.

Responsibilities

• Pipeline Development: Design and implement robust ETL/ELT pipelines using PySpark and SQL.
• Data Modeling: Develop and manage complex data models (e.g., Star/Snowflake schemas) and maintain Fact and Dimension tables within Snowflake.
• Performance Optimization: Monitor and tune Snowflake queries and Spark jobs to optimize performance and costs.
• Data Quality & Integrity: Implement automated data validation frameworks and testing procedures to ensure a single source of truth.
• Operations: Troubleshoot production pipeline issues, manage version control via Git.

Skills

Data Modeling

Data Quality

Git

SQL

Education

Bachelor of Computer Science

Tools

Snowflake

PySpark

Teradata

Job description

• PySpark: Strong hands-on experience using Apache Spark with Python for distributed data processing and transformation.


• Database & SQL Knowledge: Advanced proficiency in Teradata


• Pipeline Development: Design and implement robust ETL/ELT pipelines using PySpark and SQL to ingest data from diverse sources including APIs, flat files, and relational databases


• Data Modeling: Develop and manage complex data models (e.g., Star/Snowflake schemas) and maintain Fact and Dimension tables within Snowflake


• Performance Optimization: Monitor and tune Snowflake queries and Spark jobs to optimize performance, reduce latency, and manage computational costs


• Data Quality & Integrity: Implement automated data validation frameworks and testing procedures to ensure a \"single source of truth\" and high data reliability


• Operations: Troubleshoot production pipeline issues, manage version control via Git

Job Description

Data Engineer

Must Have Technical/Functional Skills

• Snowflake: Deep expertise in Snowflake


• PySpark: Strong hands-on experience using Apache Spark with Python for distributed data processing and transformation.


• Database & SQL Knowledge: Advanced proficiency in Teradata

Roles & Responsibilities

• Pipeline Development: Design and implement robust ETL/ELT pipelines using PySpark and SQL to ingest data from diverse sources including APIs, flat files, and relational databases


• Data Modeling: Develop and manage complex data models (e.g., Star/Snowflake schemas) and maintain Fact and Dimension tables within Snowflake


• Performance Optimization: Monitor and tune Snowflake queries and Spark jobs to optimize performance, reduce latency, and manage computational costs


• Data Quality & Integrity: Implement automated data validation frameworks and testing procedures to ensure a \"single source of truth\" and high data reliability


• Operations: Troubleshoot production pipeline issues, manage version control via Git

Education

Engineering

TCS Employee Benefits Summary

• Discretionary Annual Incentive.


• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.


• Family Support: Maternal & Parental Leaves.


• Insurance Options: Auto & Home Insurance, Identity Theft Protection.


• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.


• Time Off: Vacation, Time Off, Sick Leave & Holidays.


• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Salary Range: $90000 - 120000 a year

Qualifications

• BACHELOR OF COMPUTER SCIENCE