Lead Databricks Integration Analyst

Altria Group, Inc.; Richmond (VA); Hybrid; USD 105,000 - 152,000; Full time

Benefits offered by this job

Hybrid work schedule

Relocation benefits

Paid time off and holidays

401(k) matching

Job summary

Altria Group, Inc. is seeking a Lead Databricks Integration Analyst in Richmond, VA on a hybrid schedule. You will design, build, and operate the data integration layer connecting enterprise sales apps to the Databricks Lakehouse, partnering with architects, engineers and business partners to deliver reliable pipelines.

You will lead technical direction, reviews, and mentoring while translating business requirements into scalable technical solutions. Relocation benefits are available.

Qualifications

• Bachelor's degree in CS/IS/Engineering or related technical field with 6+ years of enterprise data integration experience.
• 3+ years hands-on Databricks experience designing, building, and operating data ingestion and transformation pipelines.
• Experience with Lakeflow Connect: CDC, incremental ingestion, troubleshooting.
• Experience with Lakeflow Jobs orchestration: scheduling, dependencies, retries, monitoring.
• Experience building Spark Declarative Pipelines with streaming or incremental processing and data quality controls.
• Strong SQL and Python development; tuning Spark workloads; preferred Azure, CI/CD, Agile delivery.

Responsibilities

• Design and implement bi-directional integrations between enterprise sales apps and Databricks Lakehouse.
• Build and optimize ingestion pipelines for databases, SaaS, files, and streaming data.
• Develop and maintain transformation pipelines using Databricks tooling with QoS and scalable processing.
• Enforce data quality, monitoring, logging, and incident resolution for production pipelines.
• Collaborate with owners and platform teams to establish connectivity, authentication, and data contracts.
• Provide technical direction, conduct design/code reviews, and mentor engineers on Databricks integration practices.

Skills

Databricks

SQL

Python

Spark Pipelines

Lakeflow Connect

Lakeflow Jobs

CI/CD

Azure Cloud

Agile Delivery

Unity Catalog

Education

Bachelor's degree in Computer Science/Information Systems/Engineering

Tools

Lakeflow Connect

Lakeflow Jobs

Spark Declarative Pipelines

Azure Cloud Services

CI/CD tooling

Unity Catalog

Job description

Job Description:

Shape the future of enterprise data integration!

Are you passionate about building scalable data integrations that power business decisions? Do you enjoy solving complex data challenges while influencing technical direction? If so, we want to hear from you! We are seeking a Lead Databricks Integration Analyst to help design, build, and operate the data integration layer connecting enterprise sales applications to our Databricks Lakehouse platform. You'll partner with architects, engineers, application teams, and business partners to deliver reliable, governed, and scalable data pipelines while serving as a technical leader across our sales integration portfolio. This position is in Richmond, VA with a Hybrid work schedule. Relocation benefits are available.

What you will be doing:

• Designing and implementing bi-directional integrations between enterprise sales applications and the Databricks Lakehouse using Databricks-native and Azure integration technologies.
• Building, configuring, and optimizing ingestion pipelines for databases, SaaS platforms, file feeds, and streaming data, including change data capture, incremental processing, and schema evolution.
• Developing and maintaining transformation pipelines using native Databricks integration tooling (i.e. Spark Declarative Pipelines, Lakeflow Jobs, and Lakehouse Connect), leveraging built-in quality controls, dependency management, and scalable processing patterns.
• Establishing and enforcing data quality standards, monitoring, logging, operational support processes, and incident resolution practices for production pipelines.
• Partnering with application owners, vendors, and platform teams to establish connectivity, authentication, data contracts, and integration standards.
• Leading integration initiatives by providing technical direction, conducting design and code reviews, and mentoring team members on Databricks integration practices.
• Translating business requirements into scalable technical solutions while effectively communicating priorities, risks, tradeoffs, and progress to technical and non-technical partners.

We want you to have:

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related technical field, and 6+ years of directly related professional experience designing, developing, and supporting enterprise data integrations and data pipelines in production environments.
• 3+ years of hands-on experience with Databricks, including designing, building, and operating data ingestion and transformation pipelines.
• Experience with Lakeflow Connect, including connector configuration, change data capture (CDC), incremental ingestion, and troubleshooting production integration issues.
• Experience developing and operating Lakeflow Jobs and multi-step workflow orchestration, including scheduling, dependency management, retries, monitoring, and alerting.
• Experience building Spark Declarative Pipelines, including streaming or incremental processing and embedded data quality controls.
• Strong SQL and Python development skills, including experience tuning and troubleshooting Spark workloads.
• Preferred: Experience with medallion architecture, Unity Catalog, CI/CD and infrastructure-as-code practices, automated pipeline testing, Azure cloud services, Agile Project Delivery and leading technical projects or mentoring other engineers.

The starting salary is based on but not limited to experience, knowledge, and qualifications in determining compensation decisions. The Salary Range for this position is: $104,900.00 - $152,150.00.

Why You’ll Love Building Your Career at Altria

• Work where connection and flexibility meet - enjoy the benefits of in-person collaboration three days a week (Tuesday - Thursday), while maintaining the flexibility to work remotely on Mondays and Fridays. Our hybrid approach empowers you to stay connected, collaborate with colleagues, and manage your schedule in a way that works best for you.
• Own your time - start with 15 days of paid time off, 13 paid holidays, 2 floating holiday days, and a 37.5-hour workweek so you can recharge and live fully.
• A place where you belong - where your ideas are welcomed, your growth is encouraged, and your impact is real.
• Get recognized for your work - annual merit increases and performance bonus.
• A future you can count on - 401(k) matching from day one; plus Deferred Profit Sharing, an annual company contribution in an amount equal to 13%-17% of your base salary.
• Help with your goals - get help with student loan repayment assistance, attend a conference, or gain a new certification with professional development stipends.
• Support for what matters most - comprehensive medical, dental, and vision coverage for you and your family.
• Celebrating your milestones - paid parental and bonding leave for life's biggest moments.
• Wellness that goes beyond work -