Senior Data Engineer – Streaming, Analytics & Visualization

Highmark Health; Northern (KY); Hybrid; USD 103,000 - 165,000; Full time

Job summary

Highmark Health is seeking a Senior Data Engineer – Streaming, Analytics & Visualization to design and operate real-time data pipelines powering enterprise reporting and decision-making. You will work with ADX, Kafka, Cribl, and Azure Data Factory to transform data into business-ready insights.

The role emphasizes data quality, governance, and collaboration with security, risk, and leadership teams, delivering interactive dashboards using Power BI and Tableau.

Qualifications

• Experience designing real-time and analytical data pipelines.
• Strong SQL, Python, and data modeling skills.
• Experience with cloud data services (Azure/AWS/GCP).
• Knowledge of data governance, security, and privacy practices.

Responsibilities

• Design, develop, and maintain real-time and analytical data pipelines.
• Build data models, warehouses, and BI-ready datasets.
• Collaborate with IT, product, and analytics teams to deliver data solutions.
• Ensure data quality, governance, and security across data lifecycle.

Skills

KQL

SQL

Python

Data pipelines

Azure Data Explorer

Kafka

Cribl

Azure Data Factory

Power BI

Tableau

Education

Bachelor’s degree in Computer Science/Information Systems/Data Science/related field

Master’s degree (preferred)

Tools

Azure Data Factory

Azure Data Explorer

Kafka

Cribl

Power BI

Tableau

Snowflake/Redshift/BigQuery

Job description

Senior Data Engineer – Streaming, Analytics & VisualizationSkip to main contentThank you for your interest in employment at a Highmark Health company. Highmark Health uses an online application process. If you participate in the online application process through this Workday site, your personal information will be collected, including but not limited to data such as your resume and resume content, education, contact information, address, city, postal code, country, phone number, email address, IP address, as well as any other personal information you choose to provide. As part of the online application process, we will provide details such as how we will use the data that we collect and where such information is processed. We will also ask for your consent to use the data for purposes contained in the Highmark Health Data Protection Statement and the GDPR Data Protection Consent for Job Applicants, and for all other permissible purposes.# CareersSenior Data Engineer – Streaming, Analytics & Visualization page is loaded## Senior Data Engineer – Streaming, Analytics & VisualizationApplyremote type: Remotelocations: PA, Working at Home - Pennsylvania: MD, Working at Home - Maryland: WA, Working at Home - Washington: NC, Working at Home - N Carolina: LA, Working at Home - Louisianatime type: Full timeposted on: Posted Todaytime left to apply: End Date: May 25, 2026 (3 days left to apply)job requisition id: J278531## **Company :**enGen## **Job Description :****JOB SUMMARY****\\*\\*\\*CANDIDATE MUST BE US Citizen (due to contractual/access requirements)\\*\\*\\***As a Senior Data Engineer, you will design and operate real‐time and analytical data pipelines that power enterprise reporting and decision‐making, including security and operational telemetry data. You will be responsible for ingesting, modeling, and optimizing high‐volume streaming and batch data in Azure Data Explorer (ADX), enabling fast, reliable analytics at scale.This role blends deep data engineering with analytics enablement. You will structure and transform data using modern cloud and streaming technologies ( Kafka, Cribl, Azure Data Factory), applying medallion architecture principles to ensure data quality, performance, and governance across the full data lifecycle.A core responsibility is translating engineered data into business‐ready insights by developing curated datasets, analytics, and Power BI (and Tableau) dashboards used by internal teams and leadership. Success in this role requires strong proficiency in KQL, SQL, and Python, experience optimizing ADX for analytical workloads, and the ability to partner with stakeholders—including security and risk teams—to turn business questions into scalable, high‐impact data products.**ESSENTIAL RESPONSIBILITIES*** Design, develop, and maintain robust data processes and solutions to ensure the efficient movement and transformation of data across multiple systems* Develop and maintain data models, databases, and data warehouses to support business intelligence and analytics needs* Collaborate with stakeholders across IT, product, analytics, and business teams to gather requirements and provide data solutions that meet organizational needs* Monitor work against production schedule, provide progress updates, and report any issues or technical difficulties to lead developers regularly* Implement and manage data governance practices, ensuring data quality, integrity, and compliance with relevant regulations.* Collaborate on the design and implementation of data security measures, including access controls, encryption, and data masking* Mentor other associate and intermediate data engineers as needed* Perform data analysis and provide insights to support decision-making across various departments* Stay current with industry trends and emerging technologies in data engineering, recommending new tools and best practices as needed* Other duties as assigned or requested.**EXPERIENCE****Required*** 5 years of experience in design and analysis of algorithms, data structures, and design patterns in the building and deploying of scalable, highly available systems* 5 years of experience in a data engineering, ETL development, or data management role.* 5 years of experience in SQL and experience with database technologies (e.g., MySQL, PostgreSQL, MongoDB).* 5 years of experience in data warehousing concepts and experience with data warehouse solutions (e.g., Snowflake, Redshift, BigQuery)**Preferred*** 7 years of experience with data pipeline and workflow management tools (e.g., Apache Airflow, AWS Glue, Azure Data Factory).* 7 years of experience defining system architectures and exploring technical feasibility trade-offs for optimizing short term execution while planning for long term technical capabilities* 7 years of experience working with a variety of technology systems, designing solutions or developing data solutions in healthcare* 7 years of experience with cloud platforms (AWS, Azure, GCP) and their respective data services* 7 years of experience in data governance, data quality, and data security best practices* 7 years of experience translating requirements, design mockups, prototypes or user stories into technical designs* 7 years of experience in producing data-related code that is fault-tolerant, efficient, and maintainable* 7 years of experience structuring and transforming data from multiple source systems.* 7 years of experience creating data visualizations and dashboards using Power BI or Tableau for internal and external audiences.* 7 years of experience using Cribl as a steaming and transformation solution.* 7 years of experience using Azure Data Factory as a transformation solution.* 5 years of experience using Azure Data Explorer**SKILLS*** Demonstrated ability to achieve stretch goals in a highly innovative and fast-paced environment* Adaptability: Strong ability to take on diverse tasks and projects, adapting to the evolving needs of the organization* Analytical Thinking: Strong analytical skills with a focus on detail and accuracy* Interest and ability to learn other data development technologies/languages as needed* Technical Proficiency: Comfortable with a range of data tools and technologies, with a willingness to learn new skills as needed* Strong track record in designing and implementing large-scale data sources* Strong sense of ownership, urgency, and drive* Demonstrated passion for user experience and improving usability* Team Collaboration: A team player who can work effectively in cross-functional environments* Experience and willingness to mentor junior data engineers and help develop their skills and leadership**EDUCATION****Required*** Bachelor’s degree in Computer Science, Information Systems, Data Science, Computer Engineering or related field**Preferred*** Master's degree in Computer Science, Information Systems, Data Science, Computer Engineering or related field**LICENSES or CERTIFICATIONS****Required*** None**Preferred*** None**Language (Other than English):**None**Travel Requirement:**0% - 25%**PHYSICAL, MENTAL DEMANDS and WORKING CONDITIONS****Position Type**Office- or Remote-basedTeaches / trains othersOccasionallyTravel from the office to various work sites or from site-to-siteRarelyWorks primarily out-of-the office selling products/services (sales employees)NeverPhysical work site requiredNoLifting: up to 10 poundsConstantlyLifting: 10 to 25 poundsOccasionallyLifting: 25 to 50 poundsRarely***Disclaimer:*** *The job description has been designed to indicate the general nature and essential duties and responsibilities of work performed by employees within this job title. It may not contain a comprehensive inventory of all duties, responsibilities, and qualifications required of employees to do this job. **Compliance Requirement**: This job adheres to the ethical and legal standards and behavioral expectations as set forth in the code of business conduct and company policies.* *As a component of job responsibilities, employees may have access to covered information, cardholder data, or other confidential customer information that must be protected at all times. In connection with this, all employees must comply with both the Health Insurance Portability Accountability Act of 1996 (HIPAA) as described in the Notice of Privacy Practices and Privacy Policies and Procedures as well as all data security guidelines established within the Company’s Handbook of Privacy Policies and Practices and Information Security Policy. Furthermore, it is every employee’s responsibility to comply with the company’s Code of Business Conduct. This includes but is not limited to adherence to applicable federal and state laws, rules, and regulations as well as company policies and training requirements.***Pay Range Minimum:**$102,700.00**Pay Range Maximum:**$164,600.00*Base pay is determined by a variety of factors including a candidate’s qualifications, experience, and expected contributions, as well as internal peer equity, market, and business considerations. The displayed salary range does not reflect any geographic differential Highmark may apply for certain locations based upon comparative markets.*