Data Engineering Architect, Senior

Bloomberg; Virginia (MN); On-site; USD 120,000 - 160,000; Full time

Job summary

Bloomberg is seeking a data engineer who thrives in a collaborative environment to build scalable data systems on a modern cloud platform. You will shape data modeling, governance, and consumption, leading Databricks-based Lakehouse modernization and migrations.

The role drives production-grade datasets used for analytics, experimentation, and decision-making. You will translate product and user behavior into reliable datasets, balancing technical implementation with strategic architecture and

Qualifications

• Strong experience building and operating data pipelines using SQL and Python in a modern cloud environment.
• Deep expertise in SQL with complex transformations, data modeling, and performance tuning.

Responsibilities

• Partner with stakeholders to translate KPIs and data requirements into scalable datasets.
• Own end-to-end data pipelines from ingestion to serving.
• Define and evolve the Product Analytics Lakehouse architecture for scalability and governance.
• Build and maintain production-grade datasets (Gold layer) for analytics and AI use cases.
• Define reusable data engineering patterns, standards, and guardrails.
• Own data quality and reliability for production datasets, including monitoring and SLAs.
• Lead modernization efforts of workloads to Databricks and migrate legacy workloads.

Skills

SQL

Python

Databricks

Spark/PySpark

Data modeling

Data pipelines

Cloud environments

Lakehouse architecture

Job description

You are a data engineer who thrives in a highly collaborative environment, partnering with product, analytics, and engineering teams to deliver high-quality, trusted data. You're motivated by building scalable data systems and shaping how data is modeled, governed, and consumed across a modern cloud platform. You bring deep, hands-on data engineering experience and are equally comfortable designing future-state architecture, building production solutions, and establishing the patterns and standards that allow others to build effectively. You bring recent, hands-on production experience with Databricks and will play a leading role in evolving our Databricks-based Lakehouse architecture, including the modernization and migration of existing data workloads. You enjoy translating complex product and user behavior into well-structured, reliable datasets that power analytics, experimentation, and decision-making. You can move comfortably between technical implementation and strategic architecture, communicating complex decisions clearly and influencing technical direction across teams.

You bring recent, hands-on production experience with Databricks and will play a leading role in evolving our Databricks-based Lakehouse architecture, including the modernization and migration of existing data workloads. You enjoy translating complex product and user behavior into well-structured, reliable datasets that power analytics, experimentation, and decision-making. You can move comfortably between technical implementation and strategic architecture, communicating complex decisions clearly and influencing technical direction across teams.

Primary Responsibilities

• Partner with product analytics stakeholders to translate business-defined KPIs and data requirements into scalable, production-grade datasets.
• Own the design, build, and operation of scalable data pipelines end-to-end (ingestion → transformation → serving).
• Define and evolve the architecture of the Product Analytics Lakehouse, making technical decisions that improve scalability, performance, reliability, governance, and consistency across datasets and workloads.
• Build and maintain production-grade, well-modeled datasets (Gold layer) that power analytics and AI use cases.
• Define, implement, and drive adoption of reusable data engineering patterns, frameworks, standards, and guardrails that reduce duplication, improve engineering leverage, and make the right development patterns easier to adopt.
• Own data quality and reliability for production datasets, including validation, monitoring, SLAs, and incident resolution.
• Productionize and scale prototype datasets and logic developed by analytics partners into reliable, maintainable data pipelines.
• Build governed, purpose-built datasets to support AI/ML use cases while enforcing controlled and secure data access patterns.
• Lead the technical evolution of workloads into Databricks, evaluating existing architecture and determining appropriate migration, modernization, and coexistence strategies.
• Make and communicate architectural tradeoffs across performance, cost, reliability, governance, maintainability, and developer experience.
• Provide technical leadership and architectural guidance across Product Analytics, helping engineers and analytics partners make sound data architecture, modeling, and platform decisions.
• Mentor and provide technical guidance to engineers and other technical contributors, raising engineering standards through hands-on leadership rather than formal authority.

Job Requirements

• Strong experience building and operating data pipelines using SQL and Python in a modern cloud environment.
• Deep expertise in SQL, including complex transformations, data modeling, query optimization, and performance tuning at scale.
• 2+ years of recent, hands-on production experience with Databricks, including designing, building, optimizing, and operating production data workloads.
• Strong hands-on experience with Spark/PySpark and distributed data processing in a production environment.
• Strong understanding of modern data architecture patterns, including Lakehouse architecture, ELT, and layered data models (bronze/silver/gold).
• Proven experience designing data models for analytics, including dimensional or domain-oriented approaches.
• Experience driving database and data engineering best practices, including schema design, migrations, and performance optimization.
• Demonstrated ability to own consequential architecture and engineering decisions and drive them from design through production in environments with limited structure or support.
• Demonstrated experience establishing reusable data frameworks, standards, and guardrails that have been successfully adopted beyond an individual project or pipeline.
• Experience owning production data systems, including monitoring, debugging, and resolving data pipeline failures.
• Experience working closely with business stakeholders or analysts to translate ambiguous