Sr. Software Engineer, Data Engineering

SupportFinity™; Redwood City (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

Poshmark is seeking a data engineer to scale its Datalake and ensure real-time access to quality data for stakeholders. You will design and maintain large-scale batch and streaming pipelines using Spark, Flink, and AWS, collaborating with Data Science and Platform Engineering teams.

Ideal candidates have 3+ years in software engineering with 2+ years in data engineering, strong problem solving, and hands-on experience with Hadoop, Spark, Kafka, and Redshift. On-site in Redwood City, CA.

Qualifications

• 3+ years of software engineering, with 2+ years in data engineering.
• Experience building scalable batch and real-time data pipelines.
• Strong problem solving and code quality focus.
• Experience with Hadoop, Spark, Kafka, Flink, and Redshift.

Responsibilities

• Build scalable data processing pipelines for large-scale data.
• Collaborate with Data Science and Analytics teams.
• Own critical data pipelines and improve reliability.

Skills

Data engineering

Scala/Java/Python

SQL

Distributed systems

AWS

Education

Bachelor's degree in Computer Science or related field

Tools

Hadoop

Spark

Kafka

Flink

Hive

Redshift

AWS S3

Airflow

Jenkins

EMR

Job description

About Poshmark

Poshmark is the leading fashion marketplace where style comes alive through discovery, self-expression, and human connection. Powered by a vibrant community of 165 million members, Poshmark brings real people and taste to shopping through a social experience shaped by shared discovery. Buying and selling fashion feels simple, joyful, and personal, while every item tells its own story. Poshmark empowers sellers to grow meaningful businesses, keeps fashion in circulation longer, and gives shoppers access to unique and trusted finds, from everyday pieces to one-of-a-kind vintage and luxury.

The Big Data team is a central player in the Poshmark organization. Our mission is to build a world-class big data platform to bring value out of data for us and for our customers. Our goal is to democratize data, support exploding business, provide reporting and analytics self-service tools, and fuel existing and new business critical initiatives.

The Data Engineering team at Poshmark is looking for an experienced software engineer to scale Datalake, ensuring real-time access to quality data for all the stakeholders. The role provides opportunity to showcase in-depth software development skills to build and maintain real-time and batch data pipelines with a focus on scalability and optimizations while collaborating with Data Science, Analytics, and Platform Engineering teams to build tools to access petabyte scale data.

Responsibilities

• Build highly scalable, available, fault-tolerant data processing systems using AWS technologies, MapReduce, Hive, Kafka, Spark, Flink and other big data technologies to handle batch and real-time data processing over 10s of terabytes of data ingested every day and a petabyte-sized data warehouse.
• Responsible for taking care of some of the most critical data pipelines at Poshmark.
• Participate in architecture discussions, influence product roadmap, build best practices, take ownership and responsibility over new projects.
• Create WBS (Work breakdown structure) for projects and execute them, taking full ownership, with minimal guidance.

Desired Skills & Experience

• 3+ years of relevant software engineering experience.
• 2+ year of Data Engineering experience.
• Excellent technical problem solving using data structures and algorithms, with emphasis on optimization and code quality.
• Expertise in architecting and building large-scale data processing systems using Big Data technologies like Hadoop, Hbase, Spark, Kafka, Druid, Flink, DataLake, Redshift etc.
• Eagerness to try out newer technologies and adopt them as and when needed.
• Object Oriented Patterns, Scala / Java / Python / C++ / SQL / Apache Spark, Flink, Hadoop, AWS S3, Redshift / Postgresql, Kinesis, EMR, Apache Airflow, Jenkins.

About the company

Poshmark