Senior Software Data Engineer

Complexcare Solutions, Inc.; United States; Remote; USD 129,000 - 135,000; Full time

Job summary

ComplexCare Solutions is seeking a Senior Data Engineer to contribute to data analytics and data platform development. This remote role focuses on building cloud-native data solutions using Azure Databricks, Delta Lake, and ADF.

You will work with data pipelines, data lake/warehouse, and reporting platforms to deliver insights for business stakeholders. The ideal candidate has 7+ years in data engineering, strong SQL, and experience with Power BI or similar tools.

Qualifications

• 7+ years in Data Engineering, Data Warehousing, ETL/ELT
• 5+ years with Azure Databricks and Spark SQL
• Experience building enterprise-scale data pipelines and cloud-native platforms
• Strong SQL and data modeling skills
• Healthcare data familiarity and HIPAA awareness preferred

Responsibilities

• Participate in sprint planning, backlog grooming, and daily standups
• Design, develop, and support scalable cloud-native data solutions on Azure
• Develop and optimize PySpark and SQL pipelines for batch and near real-time processing
• Design Medallion Architecture and Delta Lake solutions for enterprise analytics
• Collaborate with analytics teams to optimize semantic models and reporting
• Maintain data quality, monitoring, lineage, and operational support streams
• Create data models supporting analytics and reporting across teams
• Develop CI/CD and IaC practices for the data platform

Skills

Azure Databricks

PySpark

Delta Lake

Azure Data Factory

Power BI

SQL

Data Modeling

Cloud Architecture

Education

BS in Computer Science or equivalent

Tools

Azure Data Factory

Azure SQL

Databricks Notebook

Job description

• Pay or shift range: $129,400 USD to $135,000 USD
  The actual base pay offered may vary depending on multiple factors including, but not limited to, job-related knowledge/skills, experience, business needs, geographical location, and internal equity. At ComplexCare Solutions, it is not typical for an individual to be hired at or near the top end of the range for their role, and compensation decisions are dependent upon the facts and circumstances of each position and candidate.

Description

Senior Software Data Engineer, CCS

This position is remote

We are more than just a healthcare company; we are a community of professionals driven by the knowledge that our success leads to a positive impact on the communities that we support.

At ComplexCare Solutions, you'll play a crucial part in innovating solutions that empower health plan members and healthcare providers alike. Our collaborative environment fosters creativity and initiative, ensuring that every idea and effort contributes to meaningful change in the industry. Join us and become a part of a team that turns vision into action, making a tangible impact on the future of healthcare.

Overview: The Senior Data Engineer is responsible for contributing to data analytics, data operations, and software product development across the organization. They will support existing data operations such as data pipeline, ETL, data lake / warehouse, data structure development, and troubleshooting of data issues. They will be part of the team which is building next generation data & reporting platform. This platform will cater internal business stakeholders and external customers to provide insights & forecasting to understand current state of business, improve decision-making for their tactical and strategic goals & KPIs. They will work across a variety of products and projects as needed and engage with a variety of data types. This position may require independent work, sharing information and assisting others with work request.

Duties and Responsibilities:

• Work with agile teams and actively participate in sprint planning, backlog grooming, daily standups, retrospectives, and demonstrations.
• Contribute to requirements analysis, technical design discussions, effort estimation, task management, and status reporting.
• Design, develop, and support scalable cloud-native data solutions utilizing Azure Databricks, Delta Lake, Azure Data Factory (ADF), Azure Storage, and Azure SQL technologies.
• Develop and optimize PySpark, Spark SQL, SQL, and Databricks Notebook-based data pipelines for batch and near real-time processing.
• Design and implement Medallion Architecture (Bronze, Silver, Gold layers) and Delta Lake solutions to support enterprise data platforms.
• Build and maintain scalable ETL/ELT frameworks for ingesting, validating, transforming, cleansing, and loading large volumes of structured and unstructured data.
• Design and develop reusable data ingestion and processing frameworks that support configurable business rules, validations, exception handling, and auditability.
• Partner with data architects, business analysts, and application teams to design logical and physical data models supporting enterprise analytics and reporting.
• Design and implement STAR and SNOWFLAKE schemas, dimensional models, and curated data marts optimized for reporting and analytics workloads.
• Develop and maintain data quality, reconciliation, monitoring, lineage, and operational support frameworks to ensure data integrity and reliability.
• Design and develop scalable data export solutions for downstream applications, reporting platforms, and external customer integrations.
• Optimize Databricks workloads, Spark jobs, Delta tables, and SQL queries to improve performance, scalability, and cost efficiency.
• Implement and support CI/CD processes, Infrastructure as Code (IaC), deployment automation, and DataOps best practices across the data platform.
• Collaborate with Power BI and Analytics teams to optimize semantic models, reportingdatasets, and query performance.
• Participate in troubleshooting production issues, root cause analysis, incident resolution, and operational support activities.
• Produce technical design documentation, architecture diagrams, data flow diagrams, operational runbooks, and support documentation.
• Maintain compliance with ComplexCare Solutions (CCS) policies, procedures, security standards, and mission statement.
• Adhere to all confidentiality and HIPAA requirements as outlined within CCS policies and procedures with respect to PHI and sensitive healthcare data.
• Fulfill additional responsibilities as assigned to support operational excellence and organizational objectives.

Job Requirements:

• Minimum 7+ years of experience in Data Engineering, Data Warehousing, ETL/ELT development, and enterprise data platforms.
• 5+ years of hands-on experience with Azure Databricks, PySpark, Spark SQL, Delta Lake, and distributed data processing frameworks.
• 5+ years of experience building and supporting enterprise-scale data pipelines and cloud-native data platforms.
• Strong experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure SQL Database, and related Azure services.
• Strong SQL development skills including query optimization, performance tuning, indexing strategies, and large-scale data processing.
• Experience implementing Medallion Architecture, Delta Lake, Data Lakehouse, and modern analytics platforms.
• Experience with CI/CD, Git, Azure DevOps, Infrastructure as Code, and DataOps practices.
• Experience designing dimensional models, STAR schemas, SNOWFLAKE schemas, and enterprise data warehouses.
• Experience building data quality frameworks, reconciliation processes, monitoring, alerting, and operational support solutions.
• Experience working with reporting and analytics platforms such as Power BI, Tableau, or similar tools.
• Strong understanding of cloud architecture, data governance, security, and role-based access controls.
• Healthcare industry experience and familiarity with HIPAA, PHI, claims, clinical, or member data is highly preferred.
• Excellent problem-solving, analytical, and troubleshooting skills.
• Strong verbal and written communication skills with the ability to work effectively across technical and business teams.
• Ability to work independently, manage competing priorities, and deliver solutions in a fast-paced environment.

Education:

• BS degree in Computer Science or Computer Engineering, Business, or equivalent experience.
• Some travel (less than 10%) may be required for this position, primarily for training and collaboration purposes.

Compensation: $129,400 - $135,000 annually

The Company is proud to be an equal opportunity workplace and is an affirmative action employer. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, or any other protected characteristic under federal, state, or other applicable laws.

This company utilizes E-Verify.

Equal Opportunity Employer
This employer is required to notify all applicants of their rights pursuant to federal employment laws.For further information, please review the Know Your Rights notice from the Department of Labor.