Senior Data Engineer: Build Scalable Data Pipelines & ML

TripleLift; New York (NY); On-site; USD 130,000 - 170,000; Full time

Benefits offered by this job

Medical, Dental & Vision Plans

Flexible PTO

401k w/ employer match

Full benefits after onboarding

Job summary

TripleLift is seeking a Senior Data Engineer to join a small, influential Data Engineering team. This hire will be responsible for expanding and optimizing our data pipeline architecture, as well as optimizing data flow and collection for cross functional teams.

The Senior Data Engineer will support our software engineers, product managers, business intelligence analysts and data scientists on data initiatives, ensuring optimal data delivery across projects.

Qualifications

• Bachelor’s degree in Computer Science or related field.
• 5+ years of professional software engineering experience in production at scale.
• Experience with optimizing big data and ML pipelines like Spark and Kafka at scale.
• Fundamental knowledge of ML and AI operations; MLFlow, Feast, Ray are a plus.
• Understanding of databases such as OpenSearch, Druid, Clickhouse, InfluxDB.
• Proven cross-functional data modeling experience; strong SQL and programming in Python/Scala/Java.
• Experience with containerization, Kubernetes and ArgoCD is a plus.

Responsibilities

• Create and maintain optimal data pipeline architecture.
• Assemble large, complex data sets that meet functional/non-functional requirements.
• Identify and implement internal process improvements to automate and optimize data delivery.
• Build infrastructure for ETL using Spark, EMR, Kafka and other big data technologies.
• Collaborate with product managers, engineers and analysts to support data infrastructure needs.

Skills

Data pipeline design

Big data frameworks

Spark

Kafka

SQL

Python

Scala

Java

Kubernetes

Cross-functional collaboration

Education

Bachelor's degree in Computer Science or related field

Tools

Spark

Kafka

OpenSearch

Druid

ClickHouse

Job description

TripleLift is seeking a Senior Data Engineer to join a small, influential Data Engineering team. This hire will be responsible for expanding and optimizing our data pipeline architecture, as well as optimizing data flow and collection for cross functional teams.

The Senior Data Engineer will support our software engineers, product managers, business intelligence analysts and data scientists on data initiatives, ensuring optimal data delivery across projects.