Senior Data Engineer, Big Data & Cloud Pipelines

Vistar Media; New York (NY); On-site; USD 105,100 - 189,600; Full time

Job summary

Vistar Media is seeking a Senior Data Engineer to design and build data pipelines, analytics tools, and visualizations. You will work with Python, Spark and Scala to deliver scalable ETL solutions in a cloud-first environment.

Applicants should have a Bachelor's degree with 5+ years or an advanced degree with 3+ years in Computer Engineering or CS, plus hands-on experience with streaming data, databases, and distributed processing. A background in GIS is a plus.

Qualifications

• Bachelor's degree plus 5 years of related work experience or advanced degree with 3 years of related experience.
• Hands-on experience with streaming data, relational and non-relational databases, and distributed processing.
• Experience building data pipelines and cloud-based services (GCP) with Airflow and Kafka.
• Proficiency in Python, SQL, Spark and related libraries (pandas, scikit-learn, scipy, numpy).

Responsibilities

• Design and develop data engineering solutions for data pipelines, visualization, and analytics.
• Build scalable big data pipelines and ETL processes.
• Write robust Python, Spark and Scala code for production use.
• Share knowledge and drive new data engineering capabilities.
• Support project estimation, planning, and scoping with stakeholders.

Skills

Python

Spark

Scala

SQL

Airflow

Kafka

BigQuery

GIS

Education

Bachelor's Degree + 5 years

Advanced degree + 3 years

Computer Engineering / CS

Streaming data & distributed processing

Job description

Vistar Media is seeking a Senior Data Engineer to design and build data pipelines, analytics tools, and visualizations. You will work with Python, Spark and Scala to deliver scalable ETL solutions in a cloud-first environment.

Applicants should have a Bachelor's degree with 5+ years or an advanced degree with 3+ years in Computer Engineering or CS, plus hands-on experience with streaming data, databases, and distributed processing. A background in GIS is a plus.