Senior Data Architect: Lakehouse & Databricks Lead

Bloomberg Industry Group; Virginia (MN); On-site; USD 180,000 - 220,000; Full time

Job summary

Bloomberg Industry Group seeks a data engineer to design, build, and operate scalable data pipelines in a modern cloud environment. You will own end-to-end data workflows, shape the Product Analytics Lakehouse, and drive governance, quality, and reusable patterns across datasets and workloads.

You will mentor engineers, collaborate with analytics and product teams, and lead migration and modernization efforts within Databricks-based architectures to support analytics, experimentation, and

Qualifications

• Strong experience building and operating data pipelines using SQL and Python in a modern cloud environment.
• Deep expertise in SQL, including complex transformations, data modeling, query optimization, and performance tuning at scale.
• 2+ years of recent, hands-on production experience with Databricks, including designing, building, optimizing, and operating production data workloads.
• Strong hands-on experience with Spark/PySpark and distributed data processing in a production environment.
• Strong understanding of modern data architecture patterns, including Lakehouse architecture, ELT, and layered data models (bronze/silver/gold).
• Proven experience designing data models for analytics, including dimensional or domain-oriented approaches.
• Experience driving database and data engineering best practices, including schema design, migrations, and performance optimization.
• Demonstrated ability to own consequential architecture and engineering decisions and drive them from design through production in environments with limited structure.

Responsibilities

• Partner with product analytics stakeholders to translate business-defined KPIs and data requirements into scalable, production-grade datasets.
• Own the design, build, and operation of scalable data pipelines end-to-end (ingestion → transformation → serving).
• Define and evolve the architecture of the Product Analytics Lakehouse, making technical decisions that improve scalability, performance, reliability, governance, and consistency across datasets and workloads.
• Build and maintain production-grade, well-modeled datasets (Gold layer) that power analytics and AI use cases.
• Define, implement, and drive adoption of reusable data engineering patterns, frameworks, standards, and guardrails that reduce duplication, improve engineering leverage, and make the right development patterns easier to adopt.
• Own data quality and reliability for production datasets, including validation, monitoring, SLAs, and incident resolution.
• Productionize and scale prototype datasets and logic developed by analytics partners into reliable, maintainable data pipelines.
• Build governed, purpose-built datasets to support AI/ML use cases while enforcing controlled and secure data access patterns.
• Lead the technical evolution of workloads into Databricks, evaluating existing architecture and determining appropriate migration, modernization, and coexistence strategies.
• Make and communicate architectural tradeoffs across performance, cost, reliability, governance, maintainability, and developer experience.
• Provide technical leadership and architectural guidance across Product Analytics, helping engineers and analytics partners make sound data architecture, modeling, and platform decisions.
• Mentor and provide technical guidance to engineers and other technical contributors, raising engineering standards through hands-on leadership rather than formal authority.

Skills

SQL

Python

Databricks

Spark/PySpark

Data modeling

Data governance

CI/CD

Cloud

Education

Bachelor’s degree in CS

Master’s degree preferred

Tools

CI/CD pipelines

GCP/AWS

Job description

Bloomberg Industry Group seeks a data engineer to design, build, and operate scalable data pipelines in a modern cloud environment. You will own end-to-end data workflows, shape the Product Analytics Lakehouse, and drive governance, quality, and reusable patterns across datasets and workloads.

You will mentor engineers, collaborate with analytics and product teams, and lead migration and modernization efforts within Databricks-based architectures to support analytics, experimentation, and