Senior Data Engineer - Databricks AI & Platform Architect

PharmaLink, Inc.; Largo (FL); On-site; USD 120,000 - 170,000; Full time

Job summary

PharmaLink, Inc. is seeking a Senior Data Engineer to lead and evolve its enterprise data, analytics, and AI platforms using the Databricks ecosystem.

This hands-on role spans data engineering, lakehouse design, dashboards, and AI apps, requiring strong engineering foundations, problem-solving, and ownership from design to production. You will design scalable data products, implement Unity Catalog and data governance, optimize pipelines, and collaborate with governance, BI, and business teams to

Qualifications

• 8+ years in data engineering or related roles.
• 4+ years hands-on Databricks in production.
• Advanced SQL, Python, PySpark, and Spark skills.
• Strong Delta Lake and Unity Catalog experience.
• ETL/ELT, data modeling, and enterprise data architecture.
• Databricks SQL and AI/BI dashboards experience.
• Experience with AI/ML, Generative AI, and Lakebase.
• Git, CI/CD, automated testing, monitoring, and production support.
• Experience with cloud-based enterprise data platforms.
• Positive attitude in a high-performance team; background check/drug screening.

Responsibilities

• Design, build, and operate enterprise data solutions on Databricks.
• Develop scalable ETL/ELT pipelines using Python, PySpark, Spark SQL, and Delta Lake.
• Design curated data models and reusable data products for analytics, applications, and AI.
• Implement Unity Catalog, security, lineage, access controls, and data-governance.
• Build and optimize Databricks SQL Warehouses and AI/BI Dashboards.
• Develop data and AI applications using Databricks Apps.
• Design and implement Databricks Lakebase solutions.
• Enable AI/ML and Generative AI use cases using Databricks capabilities.
• Build automated data-quality, reconciliation, anomaly-detection, and monitoring.
• Integrate data from databases, SaaS platforms, APIs, files, and enterprise systems.
• Optimize Spark workloads, SQL queries, pipelines, performance, and costs.
• Establish engineering standards for Git, CI/CD, testing, monitoring, and production deployment.
• Troubleshoot complex production and data issues and perform root-cause analysis.
• Partner with Data Governance, BI, application, and business teams to deliver trusted data products.

Skills

Databricks expertise

Advanced SQL

Python

PySpark

Apache Spark

Data modeling

ETL/ELT design

Data architecture

AI/ML familiarity

Git CI/CD

Cloud data platforms

Tools

Delta Lake

Unity Catalog

Databricks SQL

Power BI

Azure Databricks

Job description

PharmaLink, Inc. is seeking a Senior Data Engineer to lead and evolve its enterprise data, analytics, and AI platforms using the Databricks ecosystem.

This hands-on role spans data engineering, lakehouse design, dashboards, and AI apps, requiring strong engineering foundations, problem-solving, and ownership from design to production. You will design scalable data products, implement Unity Catalog and data governance, optimize pipelines, and collaborate with governance, BI, and business teams to