Data Engineer

LanceSoft, Inc.

India

Remote

INR 123,000 - 134,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

A leading technology firm is seeking a motivated Sr. Data Engineer to build a state-of-the-art marketing data platform. This remote position focuses on developing data pipelines for consumer products. Ideal candidates will have over 8 years in software engineering and strong data processing expertise. Join a dynamic team and help innovate solutions in a fast-paced environment.

Qualifications

8+ years of experience in software engineering.
5+ years of experience in data engineering.
Expertise in building and managing large-volume data processing platform.

Responsibilities

Design and develop the data platform for business needs.
Build software for event-driven and batch data processing.
Ingest and process massive campaign datasets.

Skills

Java

Scala

Python

Kafka

Kinesis

Apache Spark

Snowflake

No-SQL

Education

Bachelor’s degree in computer science or similar discipline

Tools

Apache Airflow

Tableau

Druid

AWS

Kubernetes

Job description

Pay Rate: $11.00/hr to $12.00/hr

Location: 100% Remote working from India

Duration of contract: 4 to 4.5 months contract job

Working Hours: US working hours (Night Shift from India)

This team is looking to hire a motivated Sr. Data Engineer who will be working closely with a team of highly motivated data engineers to build a state‑of‑the‑art marketing data platform to solve various data‑driven use cases across the organization. This marketing platform will host various data products such as, but not limited to, Subscription, Content, and Product Analytics, Personalization and Recommendation, Marketing & Ad‑Sales enablement. You will be working on flagship direct to consumer products such as MAX and Discovery Plus. You will be charged with building and maintaining various marketing focused data pipelines—handling both streaming and batch data processing. You will help data engineers, analysts, and scientists perform their functions by building highly scalable capabilities across the platform. This individual will bring in his/her expertise in a wide variety of big data processing frameworks (both open source and proprietary), large‑scale database systems (OLAP and OLTP), stream data processing, API development, machine learning operationalization, and cloud automation to build and support all the data needs.

Responsibilities

Design and develop the data platform to efficiently and cost effectively address various data needs across the business.
Build software across our entire cutting‑edge data platform, including event‑driven and batch data processing, storage, and serving through various data integration techniques such as APIs, Lambda, File Feeds etc.
Develop and maintain data integrations including but not limited to direct digital marketing, social channels etc.
Ingest, process massive campaign datasets to analyze campaign performance.
Change how we think, act, and utilize our data by performing exploratory and quantitative analytics, data mining, and discovery.
Think of new ways to make our data platform more scalable, resilient, and reliable and then work across our team to put your ideas into action.
Help us stay ahead of the curve by working closely with data architects, stream processing specialists, API developers, our DevOps team, and analysts to design systems which can scale elastically in ways that make other groups jealous.
Work closely with data analysts and business stakeholders to make data easily accessible and understandable to them.
Ensure data quality by implementing reusable data quality frameworks.
Work closely with various other data engineering teams to roll out new capabilities.
Work closely with AI/ML team to operationalize and utilize models for business needs.
Develop and enforce data engineering, security, data quality standards through automation.
Participate in supporting the platform 24X7.
Be passionate about growing the team—hire and mentor engineers and analysts.
Be responsible for cloud cost and improving efficiency.

The Essentials

Bachelor’s degree in computer science or similar discipline.
8+ years of experience in software engineering.
5+ years of experience in data engineering.
Prior experience with Marketing is a plus.
Go above and beyond attitude.
Ability to work in fast‑paced, high‑pressure, agile environment.
Ability and willingness to learn any new technologies and apply them at work to stay ahead of the curve.
Expertise in at least a few programming languages – Java, Scala, Python or similar.
Expertise in building and managing large‑volume data processing (both streaming and batch) platform is a must.
Expertise in OLAP databases such as Snowflake or Redshift.
Expertise in stream processing systems such as Kafka, Kinesis, Pulsar or Similar.
Expertise in building micro services and managing containerized deployments, preferably using Kubernetes.
Expertise in distributed data processing frameworks such as Apache Spark, Flink or Similar.
No‑SQL (Apache Cassandra, DynamoDB or similar) is a huge plus.
Experience in operationalizing and scaling machine models is a huge plus.
Experience with variety of data tools & frameworks (example: Apache Airflow, Druid) will be a huge plus.
Experience with analytics tools such as Looker, Tableau is preferred.
Cloud (AWS) experience is preferred.
Ability to learn and teach new languages and frameworks.
Excellent data analytical skills.
Direct to consumer digital business experience preferred.
Digital Advertising Tech experience will be a huge plus.
Strong interpersonal, communication and presentation skills.
Strong team focus with outstanding organizational and resource management skills.