Strong Data Engineer

Sigma Software; United States; À distance; USD 120 000 - 170 000; Plein temps

Résumé du poste

Sigma Software is seeking an experienced Data Engineer to design scalable data platforms powering analytics and reporting. You will build robust pipelines using Spark/pySpark, Databricks, and Azure-based services, while modeling data for insights and ensuring data quality across sources.

Collaboration with analysts and engineers is essential. The role emphasizes production-grade pipelines, cloud-first architecture, and ongoing evaluation of new tools to improve reliability and performance.

Qualifications

• 4+ years of hands-on data engineering experience.
• Strong SQL and Python for pipeline development.
• Experience with distributed processing frameworks (Spark/pySpark, Databricks).
• Cloud data platform design experience (Azure preferred).
• Production-scale analytics systems and data modeling experience.
• ETL/ELT design and data warehousing knowledge.
• Familiarity with data lake architectures and Parquet/JSON formats.
• Airflow or similar workflow orchestration experience.
• Database development and optimization experience.
• Ability to design scalable solutions with minimal supervision.
• Strong collaboration and English proficiency.

Responsabilités

• Design and develop scalable data platforms and pipelines for analytics and operations.
• Implement data processing workflows using Spark/pySpark and Databricks.
• Evolve the cloud data platform leveraging Azure Data Factory, Synapse, and Data Lake Storage.
• Model data to support analytics, reporting, and downstream consumption.
• Integrate data from multiple sources ensuring quality and consistency.
• Monitor and optimize pipeline performance for reliability and scalability.
• Collaborate with analysts, scientists, and engineers to translate requirements into solutions.
• Improve data engineering standards, testing, and operational reliability.
• Evaluate and introduce new tools and technologies in the data engineering ecosystem.

Connaissances

SQL

Python

Data modeling

ETL/ELT

Spark

Databricks

Airflow

Azure

Cloud platforms

Data warehousing

Communication

English proficiency

Outils

Apache Spark

pySpark

Databricks

Airflow

Azure Data Factory

Azure Synapse

Event Hub

Azure Data Lake Storage

Description du poste

Описание:

Are you passionate about building scalable, high-performance data solutions? We are looking for a Data Engineer to join our team and help design and develop robust data platforms that power analytics, reporting, and data-driven decision-making. Why join us? You will work on challenging projects, collaborate with talented experts, and have access to professional growth opportunities, a supportive environment, and competitive benefits.

Задачи:

• Design and develop scalable, reliable data platforms and pipelines for analytics and operations
• Implement data processing workflows using distributed frameworks (Apache Spark/pySpark, Databricks)
• Evolve the organization’s cloud data platform leveraging Azure technologies (Data Factory, Synapse, Event Hub, Azure Data Lake Storage)
• Model data to support analytics, reporting, and downstream consumption
• Integrate and process data from multiple sources ensuring quality and consistency
• Monitor and optimize pipeline performance, ensuring reliability and scalability
• Collaborate with analysts, scientists, and engineers to translate requirements into solutions
• Improve data engineering standards, testing, and operational reliability
• Evaluate and introduce new tools and technologies in the data engineering ecosystem

Требования:

• 4+ years of hands‑on experience in data engineering or building data processing platforms
• Strong SQL skills and solid Python experience for pipeline development
• Experience with distributed processing frameworks (Apache Spark/pySpark, Databricks preferred)
• Practical experience designing and implementing pipelines in cloud environments (Azure preferred)
• Experience with production‑scale analytical systems and data modeling
• Understanding of ETL/ELT design, dimensional modeling, and data warehousing principles
• Experience with modern data lake architectures and formats (Parquet, JSON)
• Familiarity with workflow orchestration tools (Airflow)
• Experience with database development and optimization
• Ability to design scalable solutions with minimal supervision
• Strong collaboration and communication skills
• Professional proficiency in English