Data Engineer (DATABRICKS / PYSPARK)

Career Sigma Software; United States; Remote; USD 120,000 - 150,000; Full time

Job summary

Career Sigma Software is seeking a data engineer to build and support data lakehouse solutions that process large volumes of marketing and advertising data. The role covers batch and near real-time pipelines, distributed data processing, and cloud-based analytics systems.

Responsibilities include developing data pipelines with PySpark, Python, and SQL, supporting Databricks and Delta Lake, and working with medallion architecture and ETL workflows to handle high data volumes.

Qualifications

• 2+ years in big data development and database design.
• Strong SQL skills.
• Strong Python development skills.
• Hands-on PySpark experience.
• Experience developing and maintaining data solutions.

Responsibilities

• Develop and maintain data pipelines using PySpark, Python, and SQL.
• Support Databricks and Delta Lake based lakehouse solutions.
• Work with medallion architecture and ETL workflows.
• Process high volumes of data in batch and near real-time environments.
• Contribute to scalable data platforms and database design.

Skills

Big data development

SQL

Python

PySpark

Tools

Databricks

Delta Lake

Medallion architecture

ETL pipelines

AWS S3

Job description

Role overview

Build and support data lakehouse solutions that process large volumes of marketing and advertising data. The work covers batch and near real-time pipelines, distributed data processing, and cloud-based analytics systems.

Responsibilities

• Develop and maintain data pipelines using PySpark, Python, and SQL.
• Support data lakehouse solutions built with Databricks and Delta Lake.
• Work with medallion architecture and ETL workflows.
• Help process high volumes of data in batch and near real-time environments.
• Contribute to scalable data platforms and database design.

Requirements

• At least two years of experience in big data development and database design.
• Strong hands-on SQL skills.
• Strong Python development skills.
• Practical experience with PySpark.
• Experience developing and maintaining data solutions.

Nice to have

• Familiarity with Databricks, Delta Lake, medallion architecture, ETL pipelines, or AWS S3.

Benefits and work setup

• Middle-level role, remote in Europe.