Senior Cloud Data Engineer — Spark & Azure

TECHNOLOGY PARTNERS INC; St. Louis (MO); On-site; USD 68,000 - 96,000; Full time

Job summary

Technology Partners is seeking a Senior Software Engineer (Cloud Data Engineering) in St. Louis to design and build scalable data pipelines using Apache Spark and Azure.

You will contribute to the Cloud Data Engineering Product Services team, enabling data-driven decision making for mobility solutions. Responsibilities include creating workflows to ingest data into the Enterprise Data Lake, supporting Databricks and Azure-based workflows, and collaborating with cross-functional teams to improve

Qualifications

• 3+ years of full-lifecycle development experience with testing and data validation.
• Hands-on expertise with Apache Spark and Azure-based data pipelines.
• Experience with relational databases (Oracle, SQL Server, Postgres, or Teradata).
• Proficiency in developing scalable pipelines using Python, Scala, or Java.
• CI/CD familiarity (GitHub Actions, Azure DevOps).
• Must reside in St. Louis, MO or surrounding areas; authorized to work in the U.S. without sponsorship.
• Work hours aligned to U.S. Central Time.

Responsibilities

• Create workflows to ingest data into the Enterprise Data Lake.
• Provide Tier 1 support for Databricks workflows, Spark streaming, and Azure Functions apps.
• Triage production products and support domain teams' development projects.
• Identify and implement hardening and resiliency improvements across the environment.
• Collaborate with engineers to develop new data pipelines and optimize data solutions.

Skills

Spark fundamentals

Troubleshooting

Communication

Team collaboration

Data validation

Time zone alignment

Education

Bachelor’s degree in CS / CIS / MIS

Tools

Azure Databricks

Azure Functions

Azure SQL

Databricks

GitHub Actions

Azure DevOps

Python

Scala

Java

Job description

Technology Partners is seeking a Senior Software Engineer (Cloud Data Engineering) in St. Louis to design and build scalable data pipelines using Apache Spark and Azure.

You will contribute to the Cloud Data Engineering Product Services team, enabling data-driven decision making for mobility solutions. Responsibilities include creating workflows to ingest data into the Enterprise Data Lake, supporting Databricks and Azure-based workflows, and collaborating with cross-functional teams to improve