Principal Data Engineer

Jobtailor

Arizona

On-site

USD 140,000 - 200,000

Full time

3 days ago

Be an early applicant

Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Job summary

PEMCO is seeking a senior Data Engineer to design and build scalable, secure data pipelines and governed data products for analytics and AI.

You will partner with stakeholders across policy, billing, customer, and finance to modernize legacy platforms on Azure, Databricks, and lakehouse patterns, while mentoring engineers and upholding standards. Occasional travel to Seattle HQ.

Qualifications

Bachelor's degree required
10+ years of experience in Data Engineering or related fields
5+ years designing cloud-based data solutions on Azure or equivalent
Proven experience building and operating enterprise-scale data platforms
Extensive experience with Databricks, Spark, Delta Lake, PySpark, lakehouse architectures
Expert SQL, data modeling, and large-scale data processing
Experience with Data Vault or dimensional models a plus
Strong cloud-native architecture and analytics engineering knowledge
Knowledge of data quality, lineage, metadata, observability, and security controls
Ability to lead and mentor engineering teams and set standards

Responsibilities

Design and implement scalable data pipelines and data products for analytics and AI
Build cloud-native data solutions using Databricks, Spark, and Delta Lake
Collaborate with cross-functional teams to translate requirements into data architectures
Modernize legacy data platforms and migrate to cloud-native architectures
Ensure data governance, quality, and security across platforms
Mentor data engineers and participate in code and architecture reviews
Optimize performance, scalability, and cost in enterprise data platforms
Support metadata management, lineage, and observability
Provide guidance on data modeling and ETL/ELT design
Travel to Seattle HQ occasionally as needed

Skills

Data Modeling

Performance Tuning

Collaboration

Mentorship

Problem Solving

Technical Leadership

Education

Bachelor's degree in Computer Science, Information Systems, Engineering, Mathematics, Data Science, or related field

Tools

Databricks

Apache Spark

Delta Lake

PySpark

SQL Server

Azure Data Platform

T-SQL

Job description

Design and develop scalable, secure, and resilient data processing patterns, frameworks, pipelines, and reusable data products
Build and optimize cloud-native data solutions using Databricks, Spark, Delta Lake, Azure Data Platform services, SQL Server, and lakehouse architectures
Design and implement batch, streaming, and near real-time data processing frameworks for large-scale enterprise workloads
Develop trusted, governed, and reusable atomic-grain datasets for analytics, operational reporting, and AI use cases
Partner with business stakeholders, analysts, data scientists, architects, and engineers to translate business requirements into scalable data solutions
Establish and promote engineering standards, architectural patterns, and best practices for data modeling, performance, testing, observability, and reliability
Design and implement data integration solutions across policy, claims, billing, customer, finance, actuarial, and third-party insurance systems
Modernize legacy data platforms and migrate toward cloud-native data architectures and engineering practices
Optimize data pipelines, storage, compute utilization, and query performance for scalability, reliability, and cost efficiency
Collaborate with Data Governance teams on metadata management, lineage, data quality, stewardship, and security standards
Contribute to architecture reviews, code reviews, and technical design discussions
Evaluate emerging technologies, tools, and architectural patterns
Mentor data engineers through knowledge sharing and hands-on collaboration
Troubleshoot, perform root-cause analysis, and resolve complex data platform and production issues
Enable high-quality, accessible, and governed data foundations for AI, machine learning, and advanced analytics
Demonstrate PEMCO policies, values, code of ethics, and business conduct
Support the PEMCO Brand and help attract top talent
Perform other duties as assigned

Requirements

Bachelor's degree in Computer Science, Information Systems, Engineering, Mathematics, Data Science, or a related technical field is required
Equivalent combination of education and professional experience may be considered
10+ years of experience in Data Engineering, Data Warehousing, Big Data Engineering, or related disciplines is required
5+ years of experience designing and implementing cloud-based data solutions on Azure or equivalent cloud platforms is required
Proven experience building and operating enterprise-scale data platforms supporting high-volume analytics and operational workloads
Extensive experience with Databricks, Apache Spark, Delta Lake, PySpark, and lakehouse architectures
Strong experience with SQL Server, T-SQL, database engineering, performance tuning, and large-scale data processing
Experience designing dimensional models, Data Vault models, or other modern data product architectures
Expert-level proficiency in SQL, data modeling, database design, and large-scale data engineering
Advanced proficiency with Databricks, Spark, Delta Lake, and distributed data processing frameworks
Demonstrated expertise designing scalable ELT/ETL solutions and modern data architectures
Strong understanding of cloud-native architecture patterns, data platform design, and analytics engineering
Experience designing highly reusable and governed data products supporting reporting, analytics, and AI workloads
Knowledge of data quality, lineage, metadata management, observability, and operational monitoring
Understanding of data governance, security, privacy, and regulatory compliance controls
Experience optimizing performance, scalability, reliability, and cost of enterprise data platforms
Ability to solve highly complex technical and business problems in a collaborative environment
Ability to influence technical direction and engineering standards through expertise, mentorship, and technical leadership
State residency and work authorization in WA, OR, ID, AZ, TX, OH, IL, MN, NC, SC, or FL
Occasional travel to headquarters in Seattle, WA

Core Competencies

Demonstrates expertise in designing and implementing scalable, cloud-native data solutions using Databricks, Spark, and Azure Data Platform services. Proficient in data modeling, performance tuning, and building enterprise-scale data platforms for analytics and operational workloads.

Highest-signal resume keywords