Senior Consultant, Data Developer

United States Digital Space LLC; United States; Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Competitive salary

Incentive pay

Employee share plan

Pension plan

Vacation allowance

Wellbeing support

Recognition program

Job summary

United States Digital Space LLC is seeking a Senior Consultant, Databricks Developer to lead design, development, and optimization of data engineering solutions using Azure Databricks, Spark, PySpark, Python and SQL.

You will collaborate with architects, analysts, and stakeholders, mentor junior engineers, and help deliver complex data initiatives in a hybrid work setting with on-site 1-3 days per week in the United States.

Qualifications

• Deep Databricks and Spark expertise in cloud environments.
• Translate business needs into scalable data solutions with strong design decisions.
• 4+ years in engineering and optimization of data pipelines and lakehouse architectures.
• Experience with data quality, governance, and secure development practices.

Responsibilities

• Design and develop scalable ETL/ELT pipelines using Databricks, Spark, PySpark, Python, and SQL.
• Translate business requirements into practical data solutions and contribute to architecture decisions across lakehouse and enterprise environments.
• Tune Spark jobs and Databricks clusters for performance, reliability, and cost efficiency.
• Implement data quality controls, governance, and security practices across pipelines.

Skills

Databricks

Apache Spark

PySpark

Python

SQL

Azure

Data pipelines

Tools

Azure Data Factory

Airflow

GitHub Actions

Autosys

Databricks Unity Catalog

Artifactory

Vaults

Job description

We’re building a relationship-oriented bank for the modern world. We need talented, passionate professionals who are dedicated to doing what’s right for our clients.

At the company, we embrace your strengths and your ambitions, so you are empowered at work. Our team members have what they need to make a meaningful impact and are truly valued for who they are and what they contribute.

To learn more about the company, please visit the company.com

What You’ll Be Doing

You’ll join a team focused on building modern, scalable data solutions that power analytics, reporting, and enterprise data transformation. As a Senior Consultant, Databricks Developer, you’ll lead the design, development, and optimization of data engineering solutions using Azure Databricks, Apache Spark, and cloud-based data services. You’ll translate business and analytics needs into reliable technical solutions, supporting batch and near real-time data processing across complex environments. In this role, you’ll drive engineering excellence by improving performance, strengthening data quality and security controls, and contributing to architecture decisions across lakehouse and enterprise data platforms. You’ll also collaborate closely with architects, analysts, DevOps partners, and business stakeholders while mentoring junior team members and supporting successful delivery across the full development lifecycle.

*At the company we enable the work environment most optimal for you to thrive in your role. You’ll have the flexibility to manage your work activities within a hybrid work arrangement where you’ll spend 1-3 days per week on-site, while other days will be remote.*

How You’ll Succeed

• Design and development - Lead the design, build, and maintenance of scalable ETL and ELT pipelines using Databricks, Apache Spark, PySpark, Python, and SQL. Develop reusable frameworks, utilities, and components that improve consistency, efficiency, and maintainability.
• Solution architecture - Translate business requirements into practical data solutions and contribute to architecture decisions across lakehouse, data warehouse, and enterprise integration environments. Support the implementation of bronze, silver, and gold data layers and promote sound data modeling and storage practices.
• Performance optimization - Tune Spark jobs, Databricks clusters, notebooks, and workflows to improve scalability, reliability, and cost efficiency. Strengthen monitoring, logging, alerting, and error-handling practices to support stable production operations.
• Data quality and governance - Implement validation, reconciliation, and data quality controls across pipelines and datasets. Ensure solutions align with enterprise expectations for governance, lineage, privacy, access control, and secure development practices.
• Delivery leadership - Partner with architects, analysts, data scientists, application teams, and business stakeholders to deliver complex data initiatives. Review code, uphold engineering standards, provide technical guidance, and contribute to agile planning, estimation, releases, and production support.
• Continuous improvement - Investigate incidents, resolve defects, and perform root cause analysis to improve long-term platform health. Recommend enhancements to deployment pipelines, testing practices, and operational readiness that increase development productivity and solution resilience.

Who You Are

• You have deep Databricks and Spark expertise. You bring strong hands-on experience building and optimizing data engineering solutions with Databricks, Apache Spark, PySpark, Python, and SQL in cloud-based environments.
• You’re a solution-focused technical leader. You can turn complex business and analytics requirements into scalable, secure, and practical data solutions. You make thoughtful design decisions and understand the trade-offs between speed, quality, cost, and risk.
• You have strong engineering and optimization skills, at least 4+ years. You understand distributed data processing, Data Lake, data lakes, lakehouse architecture, performance tuning, and pipeline optimization. You’re comfortable improving jobs, workflows, and platform performance across development and production environments.
• You know how to build with control and care. You have experience with data quality practices, governance requirements, metadata management, and secure development approaches, including access controls and auditability. Experience working in regulated environments such as banking or financial services is an asset.
• You work well with others and elevate the team. You communicate clearly with technical and non-technical partners, collaborate effectively across cross-functional teams, and enjoy mentoring developers to support strong delivery outcomes and shared engineering standards.
• You have modern delivery experience. You’re familiar with orchestration and delivery tools such as Azure Data Factory, Feed Hub, Autosys Airflow, GitHub, Powershell Scripts and DevSecOps or CI/CD practices using GIT Hub Actions, Artifactory and Vaults experience with Structured Streaming, Unity Catalog, or machine learning pipelines on Databricks is considered an asset, Knowledge of Giene and Claude on Data Bricks is an Asset.
• Values matter to you. You bring your real self to work and live our values of trust, teamwork, and accountability.

What the company Offers

At the company, your goals are a priority. We start with your strengths and ambitions as an employee and strive to create opportunities to tap into your potential. We aspire to give you a career, rather than just a paycheck.

• We work to recognize you in meaningful, personalized ways including a competitive salary, incentive pay, banking benefits, a benefits program*, defined benefit pension plan*, an employee share purchase plan, a vacation offering, wellbeing support, and MomentMakers, our social, points-based recognition program.
• Our spaces and technological toolkit will make it simple to bring together great minds to create innovative solutions that make a difference for our clients.
• We cultivate a culture where you can express your ambition through initiatives like Purpose Day; a paid day off dedicated for you to use to invest in your growth and development.

• Subject to plan and program terms and conditions

What you need to know

• the company is committed to creating an inclusive environment where all team members and clients feel like they belong. We seek applicants with a wide range of abilities and we provide an accessible candidate experience. If you need accommodation, please contact Mailbox.careers\[e-mail]
• the company is committed to clarity in our hiring process. All roles posted are opportunities we’re actively recruiting for, unless stated otherwise.
• You need to be legally eligible to work at the location(s) specified above and, where applicable, must have a valid work or study permit.
• We may ask you to complete an attribute-based assessment and other skills test (such as simulation, coding, French proficiency).
• We use artificial intelligence tools during the recruitment process. Our goal for the application process is to get to know more about you, all that you have to offer, and give you the opportunity to learn more about us.