Data Engineer

Sigma Software; United States; À distance; USD 120 000 - 180 000; Plein temps

Résumé du poste

Sigma Software is seeking a Senior Data Engineer to join the Data Engineering Team. You will own high-impact, greenfield data initiatives, build cloud-native platforms, and migrate on-premises systems to the cloud. Collaborate with ML/DS/Product teams and drive R&D around agentic AI architectures and LLM-ready pipelines.

You will design scalable pipelines, enforce data contracts, optimize Spark/PySpark, and lead technical initiatives across the stack while ensuring quality and reliability.

Qualifications

• 5+ years of professional experience in Data Engineering.
• Strong Python and SQL development skills.
• Hands-on experience with Apache Spark and PySpark including query optimization and performance tuning.
• Experience working with Databricks or Snowflake.
• Practical experience with at least one major cloud provider such as Azure, AWS, or GCP.
• Experience with stream processing technologies including Kafka or Spark Structured Streaming.
• Strong understanding of ETL/ELT patterns, data modelling, and data warehousing concepts.
• Experience with orchestration tools such as Apache Airflow or Azure Data Factory.
• Knowledge of Infrastructure as Code tools including Terraform.
• Understanding of production-grade systems including observability, scalability, reliability, and performance.
• Ability to independently lead technical initiatives from concept to delivery.
• Strong communication and collaboration skills.
• Upper-Intermediate or higher English level.
• Familiarity with RAG pipeline design and LLM integration patterns.
• Knowledge of data governance frameworks and tools such as Unity Catalog or Apache Atlas.
• Experience with dbt for data transformation and modelling.
• Familiarity with MLflow, Feature Stores, or ML platform integrations.

Responsabilités

• Design and build scalable cloud-native data platforms from greenfield to production.
• Design and implement near-real-time ingestion pipelines using event-driven patterns.
• Define and enforce data platform standards including Data Lake and Lakehouse principles, medallion architecture, and data contracts.
• Refactor, optimize, and modernize Spark and PySpark scripts for performance and maintainability.
• Introduce best practices for code quality, testing, and CI/CD across data pipelines.
• Drive adoption of AI tooling and agentic workflows within the Data Engineering team.
• Ensure data quality, observability, scalability, and reliability across all platforms and pipelines.
• Design and deliver self-service tooling and microservices that simplify platform usage.
• Collaborate with cross-functional stakeholders including Product, Machine Learning, and Data Science teams.
• Contribute to architecture decisions and technical R&D initiatives.

Connaissances

Python

SQL

Apache Spark

PySpark

Databricks

Snowflake

Cloud platforms

Kafka

Airflow

Terraform

LLM integration

Description du poste

Описание: We are looking for a Senior Data Engineer to join our Data Engineering Team and take ownership of high-impact, greenfield data initiatives. You will work on building modern cloud-native data platforms, migrating on-premises legacy systems to the cloud, and laying the architectural foundation for AI-ready data infrastructure. In this role, you will collaborate closely with Machine Learning, Data Science, and Product teams, serving as a key technical contributor and thought leader. You will also drive R&D efforts around agentic AI architectures, event-driven systems, and LLM-ready data pipelines – turning architectural concepts into production-grade solutions.

Задачи

• Design and build scalable cloud-native data platforms from greenfield to production
• Design and implement near-real-time ingestion pipelines using event-driven patterns
• Define and enforce data platform standards including Data Lake and Lakehouse principles, medallion architecture, and data contracts
• Refactor, optimize, and modernize Spark and PySpark scripts for performance and maintainability
• Introduce best practices for code quality, testing, and CI/CD across data pipelines
• Drive adoption of AI tooling and agentic workflows within the Data Engineering team
• Ensure data quality, observability, scalability, and reliability across all platforms and pipelines
• Design and deliver self-service tooling and microservices that simplify platform usage
• Collaborate with cross-functional stakeholders including Product, Machine Learning, and Data Science teams
• Contribute to architecture decisions and technical R&D initiatives

Требования

• 5+ years of professional experience in Data Engineering
• Strong Python and SQL development skills
• Hands‑on experience with Apache Spark and PySpark including query optimization and performance tuning
• Experience working with Databricks or Snowflake
• Practical experience with at least one major cloud provider such as Azure, AWS, or GCP
• Experience with stream processing technologies including Kafka or Spark Structured Streaming
• Strong understanding of ETL/ELT patterns, data modelling, and data warehousing concepts
• Experience with orchestration tools such as Apache Airflow or Azure Data Factory
• Knowledge of Infrastructure as Code tools including Terraform
• Understanding of production-grade systems including observability, scalability, reliability, and performance
• Ability to independently lead technical initiatives from concept to delivery
• Strong communication and collaboration skills
• Upper-Intermediate or higher English level
• Familiarity with RAG pipeline design and LLM integration patterns
• Knowledge of data governance frameworks and tools such as Unity Catalog or Apache Atlas
• Experience with dbt for data transformation and modelling
• Familiarity with MLflow, Feature Stores, or ML platform integrations

Additional Information

PERSONAL PROFILE

• Proactive and self-driven mindset
• Strong analytical and architectural thinking
• Ability to work independently and take ownership of technical decisions
• Passion for innovation and modern engineering practices
• Knowledge-sharing and team-oriented approach
• Strong problem-solving skills