Data Engineer (DATABRICKS / PYSPARK)

Sigma Software; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Sigma Software is seeking a Middle Data Engineer to build scalable data solutions for a leading AdTech and analytics company. You will work with Data Lakehouse architectures, Databricks, PySpark, Delta Lake, and cloud-native tech to process large marketing and advertising data in batch and near real-time environments.

Join a team of experienced data engineers to deepen expertise in Databricks, PySpark, and scalable data platforms while delivering enterprise-scale data pipelines and analytics

Qualifications

• 2+ years of experience in Big Data development and database design
• Strong hands-on experience with SQL
• Strong Python development skills
• Hands-on experience with PySpark
• Experience developing and maintaining data pipelines in Databricks
• Experience with Delta Lake
• Experience with Medallion Architecture (Bronze, Silver, Gold layers)
• Strong analytical and problem-solving skills
• Intermediate or higher English level
• Experience with AWS services, including S3

Responsibilities

• Develop and maintain data pipelines for processing large volumes of data in batch and near real-time environments
• Work with Databricks, PySpark, and SQL to transform, validate, and optimize data
• Support implementation of Data Lakehouse solutions following engineering standards and best practices
• Build, test, and automate ETL processes and data workflows
• Improve data quality, organization, and reliability across platforms
• Collaborate with engineers, analysts, and stakeholders to deliver scalable solutions
• Troubleshoot, optimize, and monitor data pipelines
• Stay up to date with modern data engineering technologies
• Ensure deliverables meet quality standards and project requirements

Skills

SQL

Python

PySpark

Big Data

English

Tools

Databricks

Delta Lake

Medallion Architecture

AWS S3

Job description

Описание:

We are looking for a Middle Data Engineer to join our team and contribute to building scalable data solutions for a leading AdTech and analytics company. In this role, you will work with modern Data Lakehouse architecture, high-load distributed systems, and cloud-native technologies while helping process large volumes of marketing and advertising data in batch and near real-time environments. This is an excellent opportunity for engineers who want to deepen their expertise in Databricks, PySpark, Delta Lake, and scalable data platforms while collaborating with experienced data engineering professionals on enterprise-scale solutions.

About Sigma Software:

Sigma Software is a global IT consulting company delivering software development and IT consulting services to clients worldwide across multiple industries.

CUSTOMER

Our customer is a leading AdTech and data analytics company specializing in high-load platforms for large-scale data processing and analytics. The company develops cloud-based solutions that help businesses manage data, build analytical models, and generate actionable insights for marketing and advertising operations. The customer operates globally and focuses on scalable microservices architecture, modern engineering practices, and enterprise-grade analytics platforms.

PROJECT

The project focuses on developing and supporting scalable Data Lakehouse solutions for processing large volumes of marketing and advertising data. The team works with batch and near real-time data pipelines, cloud-native infrastructure, and modern big data technologies.

Key Technologies:

• Databricks
• PySpark
• Python
• SQL
• Delta Lake
• Medallion Architecture
• ETL Pipelines
• AWS S3

Задачи:

• Develop and maintain data pipelines for processing large volumes of data in batch and near real-time environments
• Work with Databricks, PySpark, and SQL to transform, validate, and optimize data
• Support implementation of Data Lakehouse solutions following engineering standards and best practices
• Build, test, and automate ETL processes and data workflows
• Improve data quality, organization, and reliability across platforms
• Collaborate with engineers, analysts, and stakeholders to deliver scalable solutions
• Troubleshoot, optimize, and monitor data pipelines
• Stay up to date with modern data engineering technologies
• Ensure deliverables meet quality standards and project requirements

Требования:

• 2+ years of experience in Big Data development and database design
• Strong hands-on experience with SQL
• Strong Python development skills
• Hands-on experience with PySpark
• Experience developing and maintaining data pipelines in Databricks
• Experience with Delta Lake
• Experience with Medallion Architecture (Bronze, Silver, Gold layers)
• Strong analytical and problem-solving skills
• Intermediate or higher English level
• Experience with AWS services, including S3