16194 - Senior Data Engineer (Databricks & Cloud Analytics)

Smith Johnson Group; Reno (NV); On-site; USD 120,000 - 180,000; Full time

Job summary

Smith Johnson Group is seeking a Senior Data Engineer to design, build and support secure, scalable enterprise data platforms using cloud and big data technologies. You’ll leverage Databricks, Apache Spark, Delta Lake, Azure Data Factory and Kafka to deliver high-performance data pipelines, analytics, and AI-ready data models within an Agile, collaborative environment.

The role emphasizes mentorship, continuous improvement, and partnership with architects, product owners, and clients to

Qualifications

• Bachelor's degree in CS, IS, Engineering, or related field
• 6+ years designing, developing, and supporting enterprise data platforms or cloud analytics solutions
• Hands-on with Databricks, Spark, Python, SQL, and Scala
• Experience with ETL/ELT design on Azure cloud or comparable platforms
• Experience with Kafka, ADLS Gen2, and REST APIs or data integration tools

Responsibilities

• Design, develop, test, deploy, and maintain enterprise data engineering solutions using modern cloud and big data technologies
• Design and implement scalable ETL/ELT pipelines utilizing Databricks, Apache Spark (PySpark), Delta Lake, and Azure Data Factory
• Build high-performance data ingestion, transformation, and integration frameworks supporting enterprise analytics and reporting
• Develop, optimize, and maintain complex SQL queries, stored procedures, and data transformation processes
• Design and implement scalable data models supporting BI, analytics, and AI initiatives
• Build and integrate RESTful APIs, event-driven architectures, and legacy SOAP services to facilitate data exchange
• Develop and maintain streaming and messaging solutions using Apache Kafka
• Monitor, troubleshoot, and optimize production data pipelines; perform root cause analysis and long-term improvements
• Implement data quality, governance, security, and performance best practices across platforms
• Collaborate with Solution Architects, Product Owners, Business Analysts, and cross-functional teams to translate requirements into technical solutions
• Participate in Agile ceremonies and produce technical docs, deployment artifacts, runbooks
• Mentor junior engineers and contribute to engineering standards and best practices

Skills

Databricks

Apache Spark (PySpark)

Python

SQL

Scala

Education

Bachelor's degree in Computer Science/Information Systems/Engineering or related field

Tools

Azure Data Factory

Azure Data Lake Storage (ADLS Gen2)

Kafka

Delta Lake

Databricks Unity Catalog

MLflow

Git

Linux

Job description

Are you passionate about building modern data platforms that enable organizations to make smarter, faster decisions? Do you thrive on solving complex engineering challenges using cloud technologies, big data frameworks, and modern analytics platforms? As a Senior Data Engineer, you will play a critical role in designing, implementing, and supporting secure, scalable, and high-performing enterprise data platforms.

As a trusted technical consultant, you'll leverage technologies including **Databricks, Apache Spark (PySpark), Azure Data Factory, Azure Data Lake, Kafka, Python, SQL, Scala, and cloud-native services** to build high-performance data pipelines, modern integrations, and scalable analytics platforms.

This is an excellent opportunity for an engineer who enjoys combining technical expertise with business problem-solving while working in a collaborative Agile environment focused on innovation, quality, and continuous improvement.

This position is based onsite at a client location in the **Salt Lake City, UT** area.

• Working on challenging enterprise and public sector projects that create meaningful impact
• Opportunities to leverage cutting-edge cloud, analytics, AI, and big data technologies
• A collaborative culture focused on mentorship, innovation, and continuous improvement
• Access to ongoing professional development, technical training, and certification opportunities
• Career growth supported by one of the world's most respected IT consulting organizations
• A team environment where your expertise helps shape the future of digital transformation for our clients

**Ready to build the next generation of enterprise data platforms?**

Responsibilities:

• Design, develop, test, deploy, and maintain enterprise data engineering solutions using modern cloud and big data technologies
• Design and implement scalable ETL/ELT pipelines utilizing **Databricks, Apache Spark (PySpark), Delta Lake, and Azure Data Factory**
• Build high-performance data ingestion, transformation, and integration frameworks supporting enterprise analytics and reporting
• Develop, optimize, and maintain complex SQL queries, stored procedures, and data transformation processes
• Design and implement scalable data models supporting business intelligence, analytics, and AI initiatives
• Build and integrate RESTful APIs, event-driven architectures, and legacy SOAP services to facilitate seamless enterprise data exchange
• Develop and maintain streaming and messaging solutions using **Apache Kafka**
• Monitor, troubleshoot, and optimize production data pipelines while performing root cause analysis and implementing long-term solutions
• Implement data quality, governance, security, and performance best practices across enterprise platforms
• Manage source code using Git while following CI/CD and enterprise DevOps best practices
• Collaborate with Solution Architects, Product Owners, Business Analysts, and cross-functional engineering teams to translate business requirements into technical solutions
• Participate in Agile ceremonies including sprint planning, backlog refinement, architecture discussions, code reviews, and retrospectives
• Create technical documentation, deployment artifacts, testing documentation, and operational runbooks
• Mentor junior engineers and contribute to engineering standards, best practices, and continuous improvement initiatives

Qualifications:

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related technical discipline (or equivalent professional experience)
• 6+ years** of experience designing, developing, and supporting enterprise data platforms, cloud analytics solutions, or large-scale data engineering initiatives
• Strong hands-on experience with:
• Databricks
• Apache Spark (PySpark)
• Python
• SQL
• Scala
• Experience working with the Databricks ecosystem, including:
• Databricks Workspaces
• Unity Catalog
• Databricks SQL
• MLflow
• Databricks Jobs
• Strong SQL development experience, including query optimization and performance tuning
• Experience designing and implementing ETL/ELT solutions using Azure Data Factory or comparable cloud integration platforms
• Experience with Apache Kafka or other event streaming technologies
• Experience integrating enterprise applications using REST APIs, JSON, XML, and SOAP web services
• Experience with Azure Data Lake Storage (ADLS Gen2) or comparable cloud storage platforms
• Experience using Git for source code management and collaborative software development
• Experience working within Linux environments, including shell scripting and command-line utilities
• Familiarity with Infrastructure as Code (IaC) tools such as Terraform is preferred
• Experience implementing CI/CD pipelines using Azure DevOps, GitHub Actions, or similar DevOps platforms is preferred
• Strong analytical, troubleshooting, and problem-solving skills
• Experience working in Agile environments utilizing Scrum, Kanban, or SAFe methodologies
• Demonstrated ability to manage multiple priorities while delivering high-quality solutions
• Proven ability to work independently while mentoring teammates and contributing to technical leadership

Desired Skills:

• Microsoft Fabric
• Power BI
• Data governance and metadata management
• DataOps and MLOps practices
• Enterprise data warehousing
• Master Data Management (MDM)

Successful candidates will demonstrate the ability to:

• Clearly communicate complex technical concepts to both technical and non-technical audiences
• Collaborate effectively with architects, developers, business analysts, product owners, and client stakeholders
• Build trusted relationships across cross-functional teams while delivering innovative, high-quality solutions
• Foster a culture of knowledge sharing, continuous learning, and engineering excellence