Senior Data Engineer - Databricks & PySpark Expert

Socket.dev; Cincinnati (OH); On-site; USD 98,000 - 169,000; Full time

Benefits offered by this job

Health benefits

401(k) with match

Paid time off

Job summary

84.51° is a retail data science, insights and media company. As a Senior Data Engineer, you will build solutions that ingest, transform, store, and distribute big data for data scientists and products.

You will work with PySpark/Python, Databricks, and related data engineering technologies to deliver data capabilities across teams. This role also involves Python microservice development with FastAPI, Helm, AKS.

Qualifications

• 3+ years proven ability of professional Data Development experience.
• 3+ years proven ability of developing with Databricks.
• 3+ years of experience with PySpark/Spark.
• 3+ years of experience developing with either Python or Java.
• Full understanding of ETL concepts and Data Warehousing concepts.
• Experience with Pipeline Orchestration
• Experience with CI/CD
• Experience with version control software
• Strong understanding of Agile Principles (Scrum)
• Demonstrates an understanding of AI concepts and patterns.

Responsibilities

• Take ownership of features and drive them to completion through all phases of the entire 84.51° SDLC.
• Participate in the design and development of Databricks and Cloud-based solutions
• Implement automated unit and integration testing
• Collaborate with architecture and lead engineers to ensure consistent development practices
• Provide mentoring to junior engineers
• Participate in retrospective reviews
• Participate in the estimation process for new work and releases
• Collaborate with other engineers to solve and bring new perspectives to complex problems
• Drive improvements in data engineering practices, procedures, and ways of working
• Embrace new technologies and an ever-changing environment

Skills

Databricks

PySpark

Python

Java

ETL concepts

Data Warehousing

CI/CD

Version control

Agile

Education

Bachelor’s Degree (CS/MIS/Math/Analytics/STEM)

Tools

Databricks Delta Tables

Delta Lake

Delta Live Tables

AKS

Kubernetes

Helm

Job description

84.51° is a retail data science, insights and media company. As a Senior Data Engineer, you will build solutions that ingest, transform, store, and distribute big data for data scientists and products.

You will work with PySpark/Python, Databricks, and related data engineering technologies to deliver data capabilities across teams. This role also involves Python microservice development with FastAPI, Helm, AKS.