Data Engineer

DataJobs; Spring (TX); On-site; USD 85,000 - 110,000; Full time

Benefits offered by this job

Medical Insurance

401K matching

Paid Time Off

In-person work

Job summary

TWFG Insurance in Spring, TX is seeking a Data Engineer to design and maintain scalable data pipelines and lakehouse architectures powering analytics and reporting for insurance operations.

You will build and optimize ELT/ETL pipelines, create semantic models, and integrate data from multiple systems, including APIs. Strong SQL, Python/PySpark, and cloud experience are essential, with a focus on governance and CI/CD.

Qualifications

• Bachelor's degree or equivalent professional experience in a related field.
• 3+ years of experience in Data Engineering, Analytics Engineering, or Data Warehousing.
• Experience designing and supporting enterprise-scale data pipelines.
• Strong SQL development skills.
• Experience with Python and/or PySpark.
• Experience with cloud data platforms: Azure, AWS or GCP.

Responsibilities

• Design and maintain scalable ELT/ETL pipelines using Microsoft Fabric and PySpark notebooks.
• Develop lakehouse layers within a Medallion architecture.
• Create dimensional models and semantic datasets for reporting and analytics.
• Integrate data from policy, carrier, vendor platforms, and APIs.
• Implement data quality controls, validation, and monitoring processes.
• Develop and maintain Fabric Lakehouses, Mirrored Databases, Dataflows, Notebooks, and Pipelines.
• Support Power BI Embedded analytics and enterprise reporting.

Skills

SQL

Python

PySpark

Microsoft Fabric

Power BI

Azure/AWS/GCP

Data Modeling

Education

Bachelor's degree

Tools

Microsoft Purview

REST APIs

Azure Data Platform

Job description

The Data Engineer role at TWFG Insurance in Spring, TX (The Woodlands) focuses on designing and maintaining scalable data pipelines and lakehouse-based architectures that power enterprise analytics for insurance operations and reporting.

Key Responsibilities

• Design, build, and maintain scalable ELT/ETL pipelines using Microsoft Fabric and PySpark notebooks.
• Develop and optimize lakehouse layers within a Medallion architecture.
• Create curated dimensional models, fact tables, and semantic-ready datasets for reporting and analytics.
• Integrate data from internal and external platforms, including policy administration systems, carrier systems, vendor platforms, and APIs.
• Implement data quality controls, validation frameworks, and monitoring processes.
• Develop and maintain Fabric Lakehouses, Mirrored Databases, Dataflows, Notebooks, and Pipelines.
• Optimize Direct Lake architectures and semantic model performance.
• Support Power BI Embedded analytics and enterprise reporting solutions.
• Manage large-scale datasets and help ensure optimal query performance across analytical workloads.
• Participate in data governance initiatives, including metadata management, lineage tracking, and cataloging.
• Contribute to CI/CD, testing, release management, and operational monitoring for data assets.
• Maintain documentation, including data dictionaries and architecture diagrams.
• Promote engineering best practices and reusable development patterns.
• Work toward certification on the organization’s cloud provider data platform if the certification is not already obtained.

Required Qualifications

• Bachelor’s degree in Computer Science, Information Systems, Engineering, Mathematics, or a related discipline.
• Equivalent professional experience will be considered.
• 3+ years of experience in Data Engineering, Analytics Engineering, or Data Warehousing.
• Experience designing and supporting enterprise-scale data pipelines.
• Strong skills in SQL development.
• Experience with Python and/or PySpark.
• Experience working with cloud-based data platforms: Azure, AWS, or GCP.

Preferred Qualifications

• Hands-on experience with Microsoft Fabric.
• Experience with Power BI semantic models and analytics workloads.
• Experience with Azure Data Engineering technologies.
• Knowledge of Delta Lake, OneLake, and Direct Lake architectures.
• Experience implementing Row-Level Security and enterprise data governance practices.
• Insurance, financial services, or highly regulated industry experience.
• Familiarity with data cataloging and lineage tools such as Microsoft Purview.

Technologies

• Microsoft Fabric
• PySpark
• ELT/ETL
• Lakehouse
• Medallion architecture
• Power BI Embedded
• Direct Lake
• Azure, AWS, GCP
• SQL
• Python
• CI/CD
• REST APIs
• Delta Lake
• OneLake
• Row-Level Security
• Microsoft Purview

Compensation and Location

Location: Spring, TX (onsite).
Salary: USD 85,000 - 110,000 per year.

Benefits

• Medical/Dental/Vision Insurance
• Life Insurance
• 401K with a matching component
• LTD (employer paid)
• Education reimbursement based on tenure
• Paid Time Off
• 9 Paid Holidays a year
• Work Location: In person