Data Engineer

Guardian Life; Town of Bethlehem (NY); On-site; USD 99,000 - 163,000; Full time

Job summary

Guardian Life is seeking a senior Data Engineer to build and maintain scalable data pipelines, transform raw data into trusted, business-ready assets, and support analytics, ML, and generative AI initiatives. The role collaborates with data architects, scientists, and IT leaders across Guardian and its subsidiaries.

You will design batch and real-time ingestion, leverage Databricks, enforce data quality, and ensure governance and observability across cloud-native data stores.

Qualifications

• 10+ Years of IT experience with 5+ years as Data Architect/engineer or equivalent data professional.
• Expert knowledge of relational and NoSQL databases and cloud-native data stores.
• Expert SQL skills including complex queries, performance tuning, data transformations, and scalable structures.
• Extensive hands-on data engineering with distributed processing and lakehouse tech (Python, PySpark, Databricks).
• Experience designing scalable data models, schemas, and data contracts; enterprise modeling tools preferred.

Responsibilities

• Collaborate with cross-functional teams to translate data requirements into pipelines.
• Design and maintain batch and real-time ingestion pipelines for databases, files, APIs, and streams.
• Develop and operationalize scalable data pipelines using Python, PySpark, and SQL.
• Leverage Databricks to build, orchestrate, optimize, and govern production-grade data pipelines.
• Create high-quality data assets for analytics, ML, and generative AI use cases.
• Implement observability, data quality checks, and alerts to ensure reliability.
• Participate in code reviews, testing, and documentation.
• Stay updated on data engineering best practices and emerging solutions.

Skills

collaboration

communication

problem-solving

data modeling

attention to detail

team orientation

Tools

Databricks

PostgreSQL

Oracle

AWS

S3

NoSQL

Python

PySpark

SQL

Job description

Job Description

Guardian is seeking a Data Engineer to join the team responsible for establishing and maintaining an optimized, scalable data pipelines to provide access to high quality, consistent data in an easy and convenient manner to all authorized end users and applications of Guardian and its subsidiaries. As a Data Engineer, you will play a vital role in our data infrastructure and analytics ecosystem. You will work closely with data architects and data scientists to build and maintain data pipelines, transform raw data into actionable insights, and contribute to the development of cutting-edge data solutions. This is an excellent hands‑on opportunity to experience the latest technologies and make a significant impact on our data-driven initiatives. The position will integrate new data sources from enterprise sources and external vendors using a variety of ingestion patterns including Structured Query Language (SQL) Ingestion, File, and Application Programming Interfaces (APIs).

You will

• Collaborate with cross-functional teams to understand data requirements and translate them into effective data pipelines.
• Design and maintain reliable batch and real-time ingestion pipelines for databases, files, APIs, event streams, and third-party data sources, including schema evolution, error handling, and data quality controls.
• Develop and operationalize scalable data pipelines using Python, PySpark, and SQL to transform ingested data into curated, trusted, and business-ready data assets.
• Leverage Databricks to build, orchestrate, optimize, and govern production‑grade data pipelines and data assets.
• Build governed, high-quality data assets that support analytics, machine learning, and generative AI use cases.
• Design scalable dimensional and domain-oriented data models, define schemas and data contracts, and optimize data structures for performance and usability.
• Implement observability, automated data quality checks, and proactive alerting to ensure pipeline reliability, performance, and compliance with service‑level objectives.
• Participate in code reviews, testing, and documentation efforts to maintain high-quality code and data documentation.
• Stay updated on industry trends and best practices in data engineering.
• Continue to look for innovative and next generation solutions for solving data challenges.

You are

A self‑starter, self‑motivated team player and have excellent communication skills, enabling you to engage with all levels of stakeholders who is fueled by collaboration, able to transform conceptual designs into reliable, scalable processes that meet or exceed customer needs. An individual who works closely with IT and business leaders across the enterprise to drive and embed the Enterprise Data Strategy into transformational roadmaps ensuring key principles of Data Governance, Data Management, Data Analytics and Architecture are implemented.

You have

• 10+ Years of IT experience, with 5+ years of experience as a Data Architect, data engineer or other data professional
• Expert knowledge of relational and NoSQL database concepts, with hands‑on experience across PostgreSQL, Oracle, Databricks, and cloud‑native data stores.
• Expert proficiency in logical, physical, dimensional, and domain-oriented data modeling, including schema design and data contracts; experience with enterprise modeling tools is preferred.
• Extensive hands‑on data engineering experience with distributed data processing and lakehouse technologies, including Python and PySpark on the Databricks platform.
• Hands‑on experience designing and optimizing large‑scale data pipelines and data‑warehouse architecture using batch and incremental processing, change data capture, orchestration, and schema evolution.
• Experience with AWS data services and modern cloud data platforms, including Amazon S3, PostgreSQL, and Databricks.
• Demonstrated expertise with data sourcing, profiling, data mapping and data analysis.
• Expert SQL skills, including complex query development, performance optimization, data transformations, and scalable data‑structure design.
• Ability to design secure, scalable cloud‑native data solutions across object storage, lakehouse platforms, SaaS integrations, relational databases, and NoSQL data stores.
• Excellent written and verbal communication skills and creative problem‑solving skills.
• Ability to adapt to changing business priorities and experience effectively communicating changes across different levels in the organization.
• Communication Skills: Articulate the vision and goals, both to the team members and other stakeholders.
• Emotional Intelligence & Empathy: Motivate, inspire, and resolve conflicts with others while building a positive rapport with team members and partners.
• Collaboration: Collaborate effectively with the Product Manager and your leadership and keep appraised with key updates and planning. Encourage partnership between team members to create a more dynamic and effective work environment.
• Strong listeners and able to provide constructive feedback.
• SAFE experience and the ability to work on a cross functional, self‑organizing development teams is a plus.
• Significant experience interfacing with both customers and management across business and IT
• Reltio proficiency and Agentic AI development with Bedrock

Location

• New York NY, Holmdel NJ or Bethlehem PA.
• Up to 10% travel within the NY/NJ/PA locations

Salary Range

$99,150.00 - $162,885.00

The salary range reflected above is a good faith estimate of base pay for the primary location of the position. The salary for this position ultimately will be determined based on the education, experience, knowledge, and abilities of the successful candidate. In addition to salary, this role may also be eligible for annual, sales, or other incentive compensation.

Our Promise

At Guardian, you’ll have the support and flexibility to achieve your professional and personal goals. Through skill‑building, leadership development and philanthropic opportunities, we provide opportunities to build communities and grow your career, surrounded by diverse colleagues with high ethical standards.

Inspire Well-Being

As part of Guardian’s Purpose – to inspire well‑being – we are committed to offering contemporary, supportive, flexible, and inclusive benefits and resources to our colleagues. Explore our company benefits at www.guardianlife.com/careers/corporate/benefits. Benefits apply to full‑time eligible employees. Interns are not eligible for most Company benefits.

Equal Employment Opportunity

Guardian is an equal opportunity employer. All qualified applicants will be considered for employment without regard to age, race, color, creed, religion, sex, affectional or sexual orientation, national origin, ancestry, marital status, disability, military or veteran status, or any other classification protected by applicable law.

Accommodations

Guardian is committed to providing access, equal opportunity and reasonable accommodation for individuals with disabilities in employment, its services, programs, and activities. Guardian also provides reasonable accommodations to qualified job applicants (and employees) to accommodate the individual's known limitations related to pregnancy, childbirth, or related medical conditions, unless doing so would create an undue hardship. If reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please contact [e-mail]. Please note: this resource is for accommodation requests only. For all other inquires related to your application and careers at Guardian, refer to the Guardian Careers site.

Visa Sponsorship

Guardian Life is not currently or in the foreseeable future sponsoring employment‑based visas (e.g., such as an H‑1B). In order to be a successful applicant, you must be legally authorized to work in the United States, without the need for employer sponsorship/support now or at any time in the future.

Notice Regarding Guardian’s Use of Artificial Intelligence in Recruitment

As part of Guardian’s job application process, Guardian may use artificial intelligence tools (“AI Tools