Azure Databricks Data Architect — Lakehouse & Governance

Pillsbury Winthrop Shaw Pittman LLP; Nashville (TN); On-site; USD 200,000 - 230,000; Full time

Job summary

Pillsbury Winthrop Shaw Pittman LLP is seeking a Data Architect to design and evolve our enterprise data foundation, focusing on lakehouse architecture, data integration, and governance to support AI, analytics, and business applications.

You will partner with legal and business teams to define data products and establish standards for data quality, lineage, and access control while implementing scalable pipelines using Python, SQL, Spark, and Delta Lake.

Qualifications

• Significant experience designing and operating enterprise data architectures.
• Advanced hands-on experience with Azure Databricks, Apache Spark, Delta Lake, and Unity Catalog in production environments.
• Strong experience designing conceptual, logical, and physical data models, including dimensional modeling and historical data management.
• Advanced proficiency with SQL and Python, including PySpark, and production data pipelines.
• Demonstrated knowledge of Azure lakehouse architecture, governance, and data security practices.

Responsibilities

• Lead the design, implementation, and evolution of the Firm’s Azure Databricks data architecture and lakehouse layers.
• Partner with attorneys, practice groups, and technology stakeholders to define data requirements and data products with measurable quality metrics.
• Design and maintain conceptual, logical, and physical data models for enterprise information across clients, matters, people, and documents.
• Build scalable data ingestion and transformation pipelines using Python, SQL, Spark, and Delta Lake with appropriate batch or streaming approaches.
• Develop source-to-target mappings, data contracts, and controls for schema changes and data quality validation.

Skills

Azure Databricks

Python

SQL

Unity Catalog

Delta Lake

Apache Spark

Data modeling

Data governance

PostgreSQL

Education

Bachelor’s degree in Computer Science / Data Engineering / Information Systems / Engineering

Master’s degree in Computer Science / Data Engineering / Information Systems or related discipline

Tools

Databricks

PostgreSQL tooling

Job description

Pillsbury Winthrop Shaw Pittman LLP is seeking a Data Architect to design and evolve our enterprise data foundation, focusing on lakehouse architecture, data integration, and governance to support AI, analytics, and business applications.

You will partner with legal and business teams to define data products and establish standards for data quality, lineage, and access control while implementing scalable pipelines using Python, SQL, Spark, and Delta Lake.