Data Pipeline Engineer - Big Data & Analytics

TransUnion; Northern, New York (KY, NY); Hybrid; USD 72,000 - 105,000; Full time

Benefits offered by this job

Medical, dental, vision coverage

401(k) with employer match

Paid time off & holidays

Job summary

TransUnion is seeking a Data Development Consultant to support Marketing Services data pipelines and onboarding across MTA and MMM. You will analyze data issues, fix code in the big data pipeline, and collaborate with Product, Architecture and Engineering teams.

The role is hybrid, requiring both remote work and in-person presence at a TU office location for at least two days per week.

Qualifications

• Excellent SQL skills and in-depth understanding of relational databases.
• Experience with distributed systems frameworks like Hadoop and hands-on with HDFS/MapReduce.
• Bachelor’s Degree with 5+ years IT experience in Big Data.
• Good knowledge of UNIX environments and shell scripting.
• Working knowledge of large multi-node clusters with HDFS/BigQuery/Hive.

Responsibilities

• Perform end-to-end data onboarding and build data pipelines for applications.
• Analyze source data to map data and set business rules.
• Develop, code and implement data pipelines and fix data issues.
• Collaborate with Product, Solution Architecture, Data Modeler and Consultancy Services to meet team goals.
• Learn in-house tools and suggest improvements while supporting Advisory Services.

Skills

SQL

UNIX shell

Python

Java

Education

Bachelor’s Degree

Tools

Hadoop

HDFS

MapReduce

Google Cloud Platform

BigQuery

Hive

Job description

TransUnion is seeking a Data Development Consultant to support Marketing Services data pipelines and onboarding across MTA and MMM. You will analyze data issues, fix code in the big data pipeline, and collaborate with Product, Architecture and Engineering teams.

The role is hybrid, requiring both remote work and in-person presence at a TU office location for at least two days per week.