Senior Data Engineer (Databricks & Cloud Analytics)

System One; Magna (UT); Hybrid; USD 120,000 - 180,000; Full time

Job summary

System One is seeking a Senior Data Engineer to design, build, and optimize enterprise data platforms and cloud analytics solutions from Salt Lake City. The role emphasizes hands-on work with Databricks, Spark, Python, SQL, and Scala to deliver scalable data pipelines.

You will manage Databricks ecosystem components, implement ETL/ELT with Azure Data Factory, and collaborate in Agile teams while mentoring junior engineers. Hybrid onsite/remote work offered.

Qualifications

• Bachelor's degree in CS/IS/Engineering or equivalent experience.
• 6+ years designing and supporting enterprise data platforms or cloud data solutions.
• Hands-on Databricks, Spark, Python, SQL, and Scala experience.
• Strong SQL development and query tuning.
• Experience with Azure Data Factory.
• Knowledge of Kafka or similar event streaming systems.
• Familiarity with Git, Linux, and scripting.
• Terraform and CI/CD (Azure DevOps, GitHub Actions).
• Experience in Agile methodologies.

Responsibilities

• Design and support enterprise data platforms and cloud analytics.
• Build scalable data pipelines with Databricks, PySpark, Python, SQL, and Scala.
• Manage Databricks components (Workspaces, Delta Lake, Unity Catalog, Delta Live Tables, Databricks SQL, MLflow, Databricks Jobs).
• Optimize SQL queries and performance through tuning.
• Design ETL/ELT using cloud platforms like Azure Data Factory.
• Work with Kafka and REST API integrations.
• Use ADLS Gen2 and other cloud storage.
• Collaborate in Linux environments with shell scripting and Git.
• Participate in Agile teams (Scrum/Kanban/SAFe).
• Provide technical mentoring and leadership within data engineering teams.

Skills

Databricks

Apache Spark

PySpark

Python

SQL

Scala

Git

Linux

Agile

Terraform

CI/CD

Education

Bachelor's degree in Computer Science, Information Systems, Engineering

Tools

Azure Data Factory

Kafka

Azure DevOps

GitHub Actions

Terraform

Azure Data Lake Storage

Job description

Job Title: Senior Data Engineer (Databricks & Cloud Analytics)

Location: Salt Lake City, Utah

Work Model: Hybrid – onsite and remote

Responsibilities

• Design, develop, and support enterprise data platforms, cloud analytics solutions, or large-scale data engineering initiatives.
• Work hands-on with Databricks, Apache Spark (PySpark), Python, SQL, and Scala to build scalable data pipelines and analytics solutions.
• Manage and implement Databricks ecosystem components, including Workspaces, Delta Lake, Unity Catalog, Delta Live Tables, Databricks SQL, MLflow, and Databricks Jobs.
• Optimize SQL queries and improve query performance through tuning and best practices.
• Design and implement ETL/ELT processes utilizing cloud platforms such as Azure Data Factory.
• Work with event streaming technologies like Apache Kafka and integrate enterprise applications via REST APIs and web services.
• Utilize Azure Data Lake Storage (ADLS Gen2) and other cloud storage solutions.
• Collaborate within Linux environments, including shell scripting and command-line utilities.
• Use source control tools like Git for collaborative development.
• Participate in Agile teams practicing Scrum, Kanban, or SAFe methodologies.
• Provide technical mentoring and leadership within data engineering teams.

Requirements

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field or equivalent professional experience.
• At least 6 years of experience designing and supporting enterprise data platforms or cloud data solutions.
• Extensive hands‑on experience with Databricks, Apache Spark, Python, SQL, and Scala.
• Proven SQL development skills, including query optimization and performance tuning.
• Experience designing ETL/ELT solutions with cloud platforms such as Azure Data Factory.
• Knowledge of Kafka or similar event streaming systems.
• Familiarity with cloud data storage (Azure Data Lake Storage or equivalents).
• Proficiency using Git for version control.
• Experience working in Linux environments and scripting.
• Preferred: Infrastructure as Code (Terraform), CI/CD pipelines (Azure DevOps, GitHub Actions).
• Strong analytical and troubleshooting skills.
• Prior experience working in Agile methodologies.
• Ability to prioritize multiple tasks independently and mentor team members.