Senior Engineer, Data Engineering

MultiPlan; McLean (VA); On-site; USD 145,000 - 155,000; Full time

Job summary

Claritev in McLean, VA is seeking a Senior Data Engineer to collaborate with Data Architects and ML teams to implement data solutions using Python, PySpark, SQL, and Kafka. You will design data structures, build scalable pipelines, and ensure data quality for reports and analytics.

You will mentor engineers, promote DataOps practices, and help automate data analysis across departments while aligning with HIPAA requirements.

Qualifications

• Minimum high school diploma and 5 years of data engineering experience.
• Bachelor's degree in computer science, information technology or a similarly relevant field highly preferable.
• Minimum of 3 years of experience with OOP, SQL, schema designing, data modeling, designing, building and maintaining data processing systems.
• Strong experience with languages for analytics such as Python, Java, and R.
• Database development experience using SQL, Spark, or BigQuery and experience with relational, NoSQL databases.
• Big Data development experience with Hive, Impala, Spark and familiarity with Kafka (preferred).
• Experience in triaging data issues and end-to-end data pipelines with business users.
• Experience with data governance, data quality, and data security teams and production pipelines.
• Experience building infrastructure for ETL/ELT of data from diverse resources.

Responsibilities

• Understand business processes and how they are modeled in systems.
• Collaborate with users, technology teams, and executives to understand data needs.
• Design, organize, and implement data structures, workflows, and integrations between platforms.
• Develop and maintain scalable data pipelines and build API integrations.
• Guide decisions and best practices on data integration and future data infrastructure.
• Improve performance of databases, queries, tools, and solutions.
• Create and maintain data warehouses, SQL queries, and ingestion pipelines for reports and dashboards.
• Write complex queries to transform raw data into accessible models.

Skills

Python

PySpark

SQL

Data pipelines

Cloud platforms

Kafka

Data modeling

DevOps / DataOps

Mentoring

Docker

Education

Bachelor's degree in CS/IT or related field

5 years data engineering experience

Tools

Spark

Hadoop

BigQuery

MongoDB

Docker

Kubernetes

Job description

JOB SUMMARY:

This Senior Data Engineer will be part of the team responsible for developing and deploying Engineering and Integration solutions. Primary responsibility will be to work closely with the Data Architects and Machine Learning Team to implement data solutions for the organization using Python, Java, Kafka, and other big data solutions, creating technical specification documents and test plans. Also provide support for the data solutions across the enterprise.

JOB ROLES AND RESPONSIBILITIES:

1.) Understand business processes and how they are modeled in various systems

2.) Work with business users, technology teams, and executives to understand their data needs to create innovative solutions to fulfil them

3.) Design, organize, and implement data structures, workflows, and integrations between enterprise platforms to ensure the accurate and timely execution of business processes.

4.) Develop and maintain scalable data pipelines and build out new API integrations to support continuing increases in data volume and complexity.

5.) Guiding decisions and establishing best practices on data integration/engineering, as well as the future of our data infrastructure

6.) Managing and improving the performance of our database, queries, tools, and solutions

7.) Creating and maintaining data warehouse, databases, tables, SQL queries, and ingestion pipelines to power reports(Tableau), dashboards, predictive models, and downstream analysis

8.) Writing complex and efficient queries to transform raw data sources into easily accessible models for our teams and reporting platforms

9.) Prepare data for predictive and prescriptive modeling

10.) Identify and analyze data patterns

11.) Identify ways to improve data reliability, efficiency, and quality

12.) Work with analytics, data science, and wider engineering teams to help with automating data analysis and visualization needs, advise on transformation processes to populate data models, and explore ways to design and develop data infrastructure

13.) Document high- and low-level technical design

14.) Draw performance reports and strategic proposals from analyses results for senior data science leadership

15.) Collaborate, coordinate, and communicate across disciplines and departments.

16.) Ensure compliance with HIPAA regulations and requirements.

17.) Demonstrate Company's Core Competencies and values held within.

18.) Please note due to the exposure of PHI sensitive data -- this role is considered to be a High Risk and privileged role.

19.) The position responsibilities outlined above are in no way to be construed as all encompassing. Other duties, responsibilities, and qualifications may be required and/or assigned as necessary.

JOB SCOPE:

The incumbent works independently and relies on judgment and experience to complete job responsibilities. The incumbent possesses an extensive range of knowledge of practices and procedures within the field and acts a resource for less experienced team members. Work is varied and complex.

JOB REQUIREMENTS (Education, Experience, and Training):

• Minimum high school diploma and five years of relevant experience within data engineering. Bachelors' degree in computer science, information technology or a similarly relevant field is highly preferable.
• Required licensures, professional certifications, and/or Board certifications as applicable
• Minimum of 3 years of experience with OOP, SQL, schema designing, data modeling, designing, building, and maintaining data processing systems
• Strong experience with advanced analytics tools for Object-Oriented/object function scripting using languages such as R, Python, Java, others.
• Database development experience using SQL, SPARK, or BigQuery and experience with a variety of relational, NoSQL oriented databases like Hadoop, MongoDB, Cassandra
• Big Data Development experience using Hive, Impala, Spark, and familiarity with Kafka (preferred)
• Extensive experience in triaging data issues, analyzing end-to-end data pipelines and working with business users in resolving issues.
• Experience in working with data governance/data quality and data security teams and specifically data stewards and security officers in moving data pipelines into production with appropriate data quality, governance and security standards and certification.
• Experience building infrastructure required for optimal extraction, transformation, and loading of data from diverse data resources
• Adept in agile methodologies and capable of applying DevOps and increasingly DataOps principles to data pipelines.
• Exposure to containerization using Docker, Kubernetes etc.
• Exposure to machine learning, data science, computer vision, artificial intelligence, statistics, and/or applied mathematics
• Excellent communication skills (verbal, listening and written)
• Strong ability to design, build and manage data pipelines for data structures encompassing data transformation, data models, schemas, metadata and workload management.
• Strong attention to detail when identifying data relationships, trends, and anomalies.
• Thinking through long-term impacts of key design decisions and handling failure scenarios.
• Ability to work with both IT and business in integrating analytics and data science output into business processes and workflows.
• An agile learner who brings strong problem-solving skills and enjoys working as part of a technical, cross functional team to solve complex data problems
• A thought leader when approaching technical challenges
• Ability to provide mentoring to a team of data engineers
• Ability to prioritize and manage multiple projects and requests at any one time
• Ability to effectively share technical information, communicate technical issues and solutions to all levels of business
• Ability to meet strict deadlines, work on multiple tasks and work well under pressure
• Individual in this position must be able to work in a standard office environment which requires sitting and viewing monitor(s) for extended periods of time, operating standard office equipment such as, but not limited to, a keyboard, copier and telephone

Technical Requirements (Must-Have)

• Python
• PySpark
• SQL / T-SQL
• Data pipeline development and optimization
• Cloud-based data engineering experience (Azure, AWS, or GCP)
• Experience supporting AI/ML data initiatives-Clause Code, OpenAI Codex, AI Agent

COMPENSATION:

The salary range for this position is $145-$155k. Specific offers take into account a candidate's education, experience and skills, as well as the candidate's work location and internal equity. This position is also eligible for health insurance, 401k and bonus opportunity.

#LI-PS1

Why Claritev?

Healthcare is complex. We help make it clearer.

At Claritev, you'll do work that matters. Together, we're helping make healthcare more transparent and affordable for all through the power of data, technology, and expertise. We offer meaningful opportunities to grow your career, collaborate with talented colleagues, and make an impact on the clients and communities we serve. If you're looking for purpose, growth, and a team that succeeds together, you'll find it here.

What Guides Us

At Claritev, innovation, agility, and a focus on results drive our success. We embrace bold thinking, work as one team, take ownership, and strive for excellence in everything we do - creating meaningful impact for our clients, communities, and each other.