Senior Data Platform Engineer

Archer; EE. UU.; Presencial; USD 140.000 - 210.000; Jornada completa

Descripción de la vacante

Archer is seeking a Databricks Platform SME to join Version 1’s Data & AI Practice. You will own Databricks administration, governance, entitlements, performance optimisation, and operational management, guiding scalable, cloud-native data solutions enabling analytics and AI at scale.

This delivery-focused role requires solving complex data challenges, taking ownership of platform outcomes, and applying engineering discipline in real-world environments.

Formación

• 8+ years in hands-on data platform engineering.
• Strong Databricks expertise including platform administration, governance, and performance optimisation.
• Hands-on experience managing Databricks workspaces, clusters, jobs, notebooks, and production Spark workloads.
• At least 2 Databricks platform implementations from design to production in the last 4 years.
• Strong Python/Java and SQL capabilities.
• Proven experience building and operating batch and streaming pipelines in production.
• Hands-on experience with CI/CD, monitoring/observability tooling, and data quality.

Responsabilidades

• Partner with stakeholders to build scalable data platforms.
• Take primary ownership of the Databricks platform, including admin, governance, and cost efficiency.
• Manage workspaces, access controls, configuration, and performance.
• Build cloud-native data solutions on AWS.
• Troubleshoot production systems and implement continuous improvements.
• Ensure solutions are production-ready: secure, tested, observable, and cost-efficient.
• Design, build, and operate batch and streaming pipelines.
• Develop high-throughput, low-latency data processing.
• Orchestrate data workflows using Airflow or similar tools.
• Write clean, well-tested code following best practices.

Conocimientos

Databricks Admin

Governance

Access Management

Performance Optimisation

Data Pipelines

Python

SQL

Airflow

dbt

Spark

Herramientas

Databricks

dbt

Airflow

Postgres

SQL Server

Descripción del empleo

Archer is collaborating with Version 1 to connect them with exceptional professionals for this role.

The Role

We are looking for a Databricks Platform SME to join Version 1’s Data & AI Practice, with strong Databricks expertise and the ability to take primary ownership of our Databricks platform. You will be responsible for Databricks administration, governance, entitlements, performance optimisation, and overall operational management, while contributing to scalable, cloud-native data solutions that enable analytics and AI at scale.

This is a delivery-focused role for an experienced engineer who enjoys solving complex data challenges, takes ownership of platform outcomes, and applies strong engineering discipline in real-world environments.

What You’ll Be Doing

• Partnering with business and technology stakeholders to build and enhance reliable, scalable data platforms
• Taking primary ownership of the Databricks platform, including administration, governance, entitlements, optimisation, and operational management
• Managing Databricks workspaces, users, groups, access controls, platform configuration, performance, reliability, and cost efficiency
• Building and maintaining cloud‑native data solutions on AWS
• Supporting production systems by troubleshooting issues and implementing continuous improvements
• Ensuring solutions are production‑ready: secure, tested, observable, and cost‑efficient
• Optimising data performance, reliability, and data quality in live environments
• Designing, building, and operating batch and streaming data pipelines
• Developing high‑throughput, low‑latency data processing systems
• Orchestrating data workflows using Airflow or similar orchestration tools
• Writing clean, efficient, and well‑tested code following software engineering best practices
• Collaborating closely with product, analytics, and platform teams

What You’ll Bring

• 8+ years’ experience in hands‑on data platform engineering roles
• Strong Databricks expertise, including platform administration, entitlements/access management, governance, performance optimisation, and operational management
• Hands‑on experience managing Databricks workspaces, clusters, jobs, notebooks, and production Spark workloads
• Demonstrated ability to manage and optimize Databricks platform operations at scale
• At least 2 Databricks platform implementations from design to production in the last 4 years.
• Strong hands‑on coding skills in Python, Java, or similar languages, with strong SQL capability
• Proven experience building and operating batch and streaming pipelines in production
• Hands‑on experience using dbt for transformations and Airflow (or equivalent) for orchestration
• Solid understanding of data modelling, data quality, and production data systems
• Experience working with relational databases such as Postgres or SQL Server
• Familiarity with CI/CD pipelines, testing frameworks, and monitoring / observability tooling

Nice to Have

• Databricks Professional Certification
• Experience enabling ML, AI, or GenAI workloads through strong data foundations
• Exposure to modern BI and analytics tooling
• Experience working in regulated or security‑conscious environments
• Consulting or client‑facing delivery experience