Senior Data Engineer: Real-Time Pipelines & Architecture

Human Power BG; New York (NY); On-site; USD 120,000 - 150,000; Full time

Benefits offered by this job

Equal opportunity workplace

Exceptional company culture

Job summary

A leading performance marketing engine in New York is seeking a Senior Data Engineer to contribute to the development of Micro Batch/Real Time streaming pipelines. This role involves working closely with the engineering team and collaborating with key stakeholders to drive architecture strategies. The ideal candidate should have a degree in Computer Science and experience with big data tools like Bigtable and Apache Spark. This position offers opportunities for innovation and professional growth in a dynamic environment.

Qualifications

• Understanding of distributed data systems.
• Previous experience working with big data tools such as Bigtable and Apache Spark.
• Experience running production Golang systems.

Responsibilities

• Contribute to the development of Micro Batch/Real Time streaming pipelines.
• Work closely with the engineering team on data preprocessing.
• Collaborate with key stakeholders to drive architecture strategies.

Skills

Distributed data systems

Bigtable

Apache Spark

Low Latency / High Throughput Architectural System Designs

Data science knowledge

Education

BA/BS or MS degree in Computer Science

Tools

BigQuery

Apache Spark Streaming

Apache Heron

Job description

A leading performance marketing engine in New York is seeking a Senior Data Engineer to contribute to the development of Micro Batch/Real Time streaming pipelines. This role involves working closely with the engineering team and collaborating with key stakeholders to drive architecture strategies. The ideal candidate should have a degree in Computer Science and experience with big data tools like Bigtable and Apache Spark. This position offers opportunities for innovation and professional growth in a dynamic environment.