Data Engineer - Billing Platform

AppsFlyer; United States; On-site; USD 120,000 - 160,000; Full time

Job summary

AppsFlyer is looking for a Data Engineer to join the usage metering & billing platform team, building end-to-end ETLs and data infrastructure with Spark, Hadoop, and Airflow to support scalable data processing.

You will work with a polyglot stack (Scala, Python, Go, Node) to ensure data quality and enable data-driven business decisions at scale, while mentoring teammates and adopting new technologies.

Qualifications

• 3-5 years of experience in data engineering in Big Data environments.
• 2+ years of experience with Spark and its ecosystem.
• Proficiency with Python and at least one JVM-based language (Scala, Java, or Kotlin).
• Experience with SQL and ETL and data modeling/development.
• Experience with public cloud environments (AWS considered a bonus).
• Ability to see the big picture and the business context.

Responsibilities

• Develop end-to-end ETLs and data infrastructure features from data processing logic to database choice and implementation through to API development.
• Improve the scalability and performance of existing data pipelines.
• Delve into existing pipelines and business logic, target weaknesses and add functionality.
• Mentor, advise, and work collaboratively within the Biz Data & Dev group, with other development teams, and with all other business teams of the company.
• Continuously learn and follow new technologies both for personal growth and to ensure informed technology choices.

Skills

Data engineering

Spark ecosystem

Python

SQL

NoSQL databases

ETL development

AWS

Big Data

Tools

Kafka

BigQuery

Airflow

S3

PostgreSQL

Redis

Job description

Are you ready to take your career to the next level?

AppsFlyer is rapidly growing! More customers are using more of our products while generating more data that we process and analyze in order to deliver insights that ensure our customers make the best choices. AppsFlyer is a data driven company in the most extreme sense. Dozens of researchers and analysts from all departments build dashboards and run analyses that move business and product decisions in AppsFlyer.

We are looking for a Data Engineer to join our usage metering & billing platform team! with a strong background in writing ETLs and complicated data processing pipelines. You’re deeply familiar with modern data processing stack (especially Spark, Hadoop and Airflow), have a strong problem-solving attitude and, most importantly, you’re passionate about understanding the business side of the platform. Our platform engineers have an end-to-end understanding of the data our products generate and the commercial models and data structures in which they fit.

Working with this enormous volume of data begins with massive ETL pipelines built using cutting-edge technologies such as Kafka, BigQuery, Airflow, S3, PostgreSQL, Athena, SQS, Redis, and especially Spark. We are big believers in working with the right tool for the right job, including programming languages. We run a truly polyglot shot with Scala, Python, Go and Node being especially common here at AppsFlyer. The extreme growth rate of our data and client base means that solutions need to be designed to handle 10X growth or more.

What you’ll do

• Develop end-to-end ETLs and data infrastructure features from data processing logic to database choice and implementation through to API development
• Improve the scalability and performance of existing data pipelines
• Delve into existing pipelines and business logic, target weaknesses and add functionality
• Mentor, advise, and work collaboratively within the Biz Data & Dev group, with other development teams, and with all other business teams of the company
• Continuously learn and follow new technologies both for personal growth and to ensure we are making the most informed technology choices possible

What you have

• 3-5 years of experience in data engineering in Big Data environments
• 2+ years of experience with Spark and its ecosystem.
• Experience using JVM stack, build tools, and environment (We mostly use Scala)
• Experience with public cloud environments (AWS will be a bonus), Hadoop ecosystem, SQL and NoSQL databases
• High proficiency with Python and at least one JVM-based language (Scala, Java, or Kotlin)
• Proficiency with SQL and experience with ETL and data modeling and development
• Positive attitude and willingness to take ownership
• A mindset of continuous learning and the desire to try new things
• Ability to see the "big picture" and to understand the business side of things

As a global company operating from 20 offices worldwide, we reflect the human mosaic of the diverse and multicultural world in which we live. We ensure equal opportunities for all of our employees and promote the recruitment of diverse talents to our global teams without consideration of race, gender, culture, or sexual orientation. We value and encourage curiosity, diversity, and innovation from all our employees, customers, and partners.