Data Platform Engineer

Buyers Edge Platform; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Medical benefits

Dental benefits

Vision benefits

FSA

401(k) with company match

Life insurance

Job summary

Buyers Edge Platform is seeking a Databricks Platform Administrator to own administration, governance, security and performance of our Databricks environment across AWS. You will partner with BI, Data Science, AI, DevOps and Security to ensure a secure, scalable data platform built for growth.

You will combine platform administration expertise with data engineering work, balance performance and cost, and drive migration of workloads to Databricks.

Qualifications

• Bachelor's degree in Computer Science, Information Systems, or a related field, or equivalent professional experience.
• 3+ years of data engineering experience, including at least 2 years of hands-on Databricks experience.
• Experience administering a production Databricks environment with Unity Catalog, user permissions, governance, and cost management.
• Experience building and supporting AWS-based data solutions in a production environment.
• Experience migrating data workloads to a new platform.
• Databricks certification, including Data Engineer or Platform Administrator certifications, is a plus.

Responsibilities

• Administer Databricks workspaces, compute resources, user permissions, security policies, governance controls, and monitoring.
• Support and improve data infrastructure across Databricks and AWS (S3, Glue, EMR, Redshift, Aurora, DynamoDB).
• Lead migration of data workloads onto Databricks to support analytics, AI, and business operations.
• Define best practices, standards, and training for teams to improve performance and efficiency.
• Monitor utilization and cost, and identify optimization opportunities.
• Collaborate with BI, Data Science, AI, DevOps, and Security teams to enable scalable datasets and platform reliability.
• Provide platform support and timely issue resolution; after-hours support as needed.
• Evaluate emerging technologies and recommend new tools to drive business value.

Skills

Databricks platform administration

Platform governance

Cost management

Cross-functional collaboration

Security

Data engineering

Python/Scala

AWS data solutions

Education

Bachelor's degree in Computer Science/Information Systems or related field

Tools

Databricks

Unity Catalog

AWS (S3, Glue, EMR, Redshift, DynamoDB)

Terraform/CloudFormation

Git

Datadog

Job description

Buyers Edge Platform is the technology and procurement platform behind more than $100 billion of foodservice purchasing for roughly 300,000 restaurant, hospitality, healthcare and institutional locations. We are founder-led, private-equity backed, profitable, and growing through both organic expansion and acquisition. Our family of brands spans purchasing programs, supply chain software, fresh produce, business dining and restaurant back-office tools, and the technology underneath them is becoming one platform.

Why this role, why now

Data is at the center of how Buyers Edge Platform operates, innovates, and scales. As our data ecosystem continues to grow, we are investing in a modern lakehouse architecture and expanding our use of Databricks as a core platform for analytics, AI, and operational data workloads.

The Databricks Platform Administrator will serve as our in-house Databricks expert, owning platform administration, governance, security, and performance while helping drive the migration and modernization of data workloads across the organization. This role partners closely with Business Intelligence, Data Science, AI, DevOps, and Security teams to ensure our data platform is secure, reliable, cost-effective, and built for growth.

What you will own

• The Databricks platform. Administration of Databricks workspaces, compute resources, user permissions, security policies, governance controls, and platform monitoring. You will oversee platform performance, adoption, and cost management.
• Data infrastructure. Support and improvement of existing data infrastructure across Databricks and AWS, including services such as S3, Glue, EMR, Redshift, Aurora, and DynamoDB. You will help design, implement, and evolve scalable data solutions.
• Platform modernization. Lead the migration of data workloads onto Databricks, helping establish a modern data platform that supports analytics, AI, and business operations.
• Standards and best practices. Serve as the organization's Databricks subject matter expert by defining best practices, establishing standards, training teams, and helping developers and analysts improve performance and efficiency.
• Cost optimization. Monitor platform utilization and spending, identify opportunities for improvement, and recommend solutions that balance performance, scalability, and cost.
• Data enablement. Partner with Business Intelligence, Data Science, and AI teams to create high-performance datasets and support advanced analytics initiatives.
• Security and operations. Collaborate with DevOps and Security teams to create secure, compliant, and redeployable cloud-based infrastructure and data resources.
• Platform support. Respond to service issues and requests in a timely manner and provide operational support for platform users. Occasional after-hours support may be required.
• Innovation. Evaluate emerging technologies and recommend new tools and capabilities that help solve business challenges and improve platform effectiveness.

What success looks like in year one

• Databricks administration standards, governance controls, and operational processes are established and consistently followed.
• Existing data workloads are successfully migrated to Databricks according to business priorities.
• Usage, performance, and cost monitoring processes are implemented, with measurable optimization opportunities identified and delivered.
• Data platform security and governance standards are strengthened through collaboration with Security and DevOps teams.
• Business Intelligence, Data Science, and AI teams are enabled with scalable, high-performing datasets and improved platform reliability.
• Documentation, best practices, and team training programs are established to support platform adoption and long-term scale.
• New technologies and process improvements are evaluated and implemented where they provide measurable business value.

How you work

• You combine platform administration expertise with hands-on data engineering skills.
• You balance performance, governance, scalability, and cost when making technical decisions.
• You collaborate effectively across technical and business teams to deliver practical solutions.
• You are proactive in identifying issues, driving improvements, and recommending modern approaches.
• You stay current with advancements in cloud data platforms, analytics, AI, and distributed computing technologies.

What you bring

• Experience administering, developing, and maintaining solutions in Databricks with a focus on performance, cost management, data integration and federation, Unity Catalog design, and data governance and security.
• Deep experience building data integration and ETL solutions on at least one distributed computing platform; PySpark experience is a plus.
• Strong understanding of data modeling and performance tuning across data warehouses, relational databases, and NoSQL platforms, including Redshift, MySQL, Aurora, PostgreSQL, Microsoft SQL Server, and DynamoDB.
• Thorough understanding of Data Lakehouse Architecture, open table formats including Apache Iceberg, Glue Catalog, and S3-based storage.
• Experience with at least one programming language such as Python, Scala, Golang, NodeJS, or TypeScript, along with a high-level understanding of messaging, queueing, and streaming technologies used for near real-time integrations.
• Familiarity with infrastructure-as-code tools such as Terraform or CloudFormation, Git version control practices, and monitoring, logging, and application performance management platforms; Datadog experience is a plus.
• Effective written and verbal communication skills.
• Experience and education
• Bachelor's degree in Computer Science, Information Systems, or a related field, or equivalent professional experience.
• 3+ years of data engineering experience, including at least 2 years of hands-on Databricks experience.
• Experience administering a production Databricks environment, including Unity Catalog, user permissions, governance, and cost management.
• Experience building and supporting AWS-based data solutions in a production environment.
• Experience migrating data workloads to a new platform.
• Experience supporting Business Intelligence, analytics, or Data Science teams.
• Databricks certification, including Data Engineer or Platform Administrator certifications, is a plus.

Nice to have

• Experience supporting enterprise-scale analytics or AI environments.
• Experience with modern lakehouse architectures and large-scale cloud data migrations.
• Experience operating in a private-equity-backed, acquisition-driven environment.
• Experience with foodservice, supply chain, procurement, or SaaS businesses.

What's in this for you:

• Great benefits from day one. We offer medical, dental, vision, FSA, company-paid life insurance, and more—plus a 401(k) with company match.
• Grow with us. Enjoy strong training, development, and competitive pay.
• Work-life balance. Our flexible PTO policy lets you take time when you need it—no accrual required.