Senior Software Engineer (Cloud Data Engineering)

TECHNOLOGY PARTNERS INC; St. Louis (MO); On-site; USD 68,000 - 96,000; Full time

Job summary

Technology Partners is seeking a Senior Software Engineer (Cloud Data Engineering) in St. Louis to design and build scalable data pipelines using Apache Spark and Azure.

You will contribute to the Cloud Data Engineering Product Services team, enabling data-driven decision making for mobility solutions. Responsibilities include creating workflows to ingest data into the Enterprise Data Lake, supporting Databricks and Azure-based workflows, and collaborating with cross-functional teams to improve

Qualifications

• 3+ years of full-lifecycle development experience with testing and data validation.
• Hands-on expertise with Apache Spark and Azure-based data pipelines.
• Experience with relational databases (Oracle, SQL Server, Postgres, or Teradata).
• Proficiency in developing scalable pipelines using Python, Scala, or Java.
• CI/CD familiarity (GitHub Actions, Azure DevOps).
• Must reside in St. Louis, MO or surrounding areas; authorized to work in the U.S. without sponsorship.
• Work hours aligned to U.S. Central Time.

Responsibilities

• Create workflows to ingest data into the Enterprise Data Lake.
• Provide Tier 1 support for Databricks workflows, Spark streaming, and Azure Functions apps.
• Triage production products and support domain teams' development projects.
• Identify and implement hardening and resiliency improvements across the environment.
• Collaborate with engineers to develop new data pipelines and optimize data solutions.

Skills

Spark fundamentals

Troubleshooting

Communication

Team collaboration

Data validation

Time zone alignment

Education

Bachelor’s degree in CS / CIS / MIS

Tools

Azure Databricks

Azure Functions

Azure SQL

Databricks

GitHub Actions

Azure DevOps

Python

Scala

Java

Job description

Technology Partners is currently seeking a talented Senior Software Engineer (Cloud Data Engineering).

Do you have experiencebuilding scalable data pipelines using Spark and Azure, and supporting cloud-based data engineering workloads?

Let us help you make your next big career move a reality!

What You Will Be Doing:

A leading provider of mobility solutions is seeking aSenior Software Engineerto join their Cloud Data Engineering Product Services team. You'll be central to innovating, designing, and developing powerful technology solutions that are redefining how customers rent, buy, and share vehicles globally.

Key Responsibilities:

• Create workflows toingest datainto the Enterprise Data Lake.
• Provide initial Tier 1 support forDatabricks workflows, Spark streaming applications, and Azure Function applicationswithin the Cloud Data Engineering ecosystem.
• Be part of a shared services team responsible for initial triage of production products.
• Support domain teams' development projects.
• Identify and implement hardening and resiliency opportunities across the environment.
• Collaborate with other engineers to develop newdata pipelines.
• UtilizeMicrosoft Azure Cloud PaaS technologiesto engineer data solutions, optimizing information for better decision-making and customer needs.

Required Skills & Experience:

• 3+ years of full-lifecycle development experience including testing and data validation.
• Strong understanding ofApache Spark fundamentals, including DataFrames, Spark SQL, and distributed computing.
• Experience with relational databases (Oracle, SQL Server, Postgres, or Teradata).
• Hands-on experience withAzure servicessuch as Azure Databricks, Azure Functions, and Azure SQL.
• Proficiency in developing scalable pipelines usingPython, Scala, or Java.
• Strong troubleshooting skills, including debugging multi-threaded applications.
• Familiarity with Databricks workspaces, clusters, notebooks, and job orchestration.
• Familiarity with CI/CD pipelines (GitHub Actions, Azure DevOps).
• Excellent written and verbal communication skills.
• Ability to collaborate in a team environment as well as work independently.
• Must reside in St. Louis, MO or surrounding areas.
• Must be authorized to work in the U.S. without sponsorship.
• Must be able to work within U.S. Central Time business hours.
• Most critical skill/experience:Hands-on expertise with Apache Spark and Azure-based data engineering pipelines(core to the role’s daily responsibilities and required for both development and production support).

Desired Skills & Experience:

• Bachelor’s degree in Computer Science, CIS, MIS, or related field.
• Experience with big data solutions and NoSQL databases.
• Experience working withADLS Gen2, Blob Storage, Delta Lake, or Cosmos DB.
• Knowledge of change data capture (CDC), SCD1, or SCD2 incremental processing patterns.
• Experience developing and supporting Azure Functions and APIs.
• Background in Azure, AWS, or other public cloud platforms.
• Experience with Event Hubs, Data Factory, DevOps tooling, or large-scale ingestion pipelines.
• Previous experience in the vehicle rental or fleet management domain.
• Prior experience leading large-scale or enterprise-level engineering projects.

Pay:

$49.00 - $70.00 /hr.

Keywords:

Senior Software Engineer, Cloud Data Engineering, Azure, Spark, Databricks, Data Lake, Data Pipeline, Data Ingestion, Data Validation, ETL

We are interested in every qualified candidate who is eligible to work in the United States. However, we are not able to provide sponsorship at this time or accept candidates who would require a corp-to-corp agreement.

All offers of employment at Technology Partners are contingent upon clear results of a thorough background check and drug screening that meet corresponding laws and regulations at the city, state and federal level.

Pay ranges are influenced by candidate qualifications, experience, and role specifics, with the actual rate determined considering skills, market conditions, and are subject to change by the employer; pay negotiations follow all state and federal legal guidelines.