Senior Data Engineer — Cloud Analytics & Spark Pipelines

Sigma Software; United States; Remote; USD 120,000 - 170,000; Full time

Job summary

Sigma Software is seeking an experienced Data Engineer to design scalable data platforms powering analytics and reporting. You will build robust pipelines using Spark/pySpark, Databricks, and Azure-based services, while modeling data for insights and ensuring data quality across sources.

Collaboration with analysts and engineers is essential. The role emphasizes production-grade pipelines, cloud-first architecture, and ongoing evaluation of new tools to improve reliability and performance.

Qualifications

• 4+ years of hands-on data engineering experience.
• Strong SQL and Python for pipeline development.
• Experience with distributed processing frameworks (Spark/pySpark, Databricks).
• Cloud data platform design experience (Azure preferred).
• Production-scale analytics systems and data modeling experience.
• ETL/ELT design and data warehousing knowledge.
• Familiarity with data lake architectures and Parquet/JSON formats.
• Airflow or similar workflow orchestration experience.
• Database development and optimization experience.
• Ability to design scalable solutions with minimal supervision.
• Strong collaboration and English proficiency.

Responsibilities

• Design and develop scalable data platforms and pipelines for analytics and operations.
• Implement data processing workflows using Spark/pySpark and Databricks.
• Evolve the cloud data platform leveraging Azure Data Factory, Synapse, and Data Lake Storage.
• Model data to support analytics, reporting, and downstream consumption.
• Integrate data from multiple sources ensuring quality and consistency.
• Monitor and optimize pipeline performance for reliability and scalability.
• Collaborate with analysts, scientists, and engineers to translate requirements into solutions.
• Improve data engineering standards, testing, and operational reliability.
• Evaluate and introduce new tools and technologies in the data engineering ecosystem.

Skills

SQL

Python

Data modeling

ETL/ELT

Spark

Databricks

Airflow

Azure

Cloud platforms

Data warehousing

Communication

English proficiency

Tools

Apache Spark

pySpark

Databricks

Airflow

Azure Data Factory

Azure Synapse

Event Hub

Azure Data Lake Storage

Job description

Sigma Software is seeking an experienced Data Engineer to design scalable data platforms powering analytics and reporting. You will build robust pipelines using Spark/pySpark, Databricks, and Azure-based services, while modeling data for insights and ensuring data quality across sources.

Collaboration with analysts and engineers is essential. The role emphasizes production-grade pipelines, cloud-first architecture, and ongoing evaluation of new tools to improve reliability and performance.