Senior Data Engineer & Architect, Databricks Lakehouse

Innova software Services Inc; Charlotte (NC); On-site; USD 120,000 - 180,000; Full time

Job summary

Innova software Services Inc in Charlotte, NC is seeking a Senior Data Engineer / Data Architect with deep Databricks hands-on expertise and Insurance domain experience to design, build, and implement scalable enterprise data solutions that support analytics at scale.

You will lead the development of data lakes and lakehouse platforms, optimize Spark jobs, govern data quality and security, and translate business requirements into robust architectures across policy, claims, underwriting, and

Qualifications

• 12+ years in Data Engineering or Data Architecture.
• Expertise with Databricks, Spark, Python, SQL.
• Experience building data lakes/lakehouses and data warehouses.
• Cloud platform experience (Azure preferred; AWS/GCP valued).
• Knowledge of data governance, security, and lineage.
• Strong collaboration with stakeholders and engineers.

Responsibilities

• Design and build enterprise-scale data architecture and pipelines using Databricks, Spark, Python, and SQL.
• Architect data lakes, lakehouses, and data warehouses with Delta Lake, DLT, Unity Catalog.
• Develop high-performance batch/streaming ETL/ELT pipelines and data ingestion.
• Optimize Spark jobs and queries for performance and cost efficiency.
• Implement governance, data quality, security, lineage, and access controls.
• Translate business requirements into scalable technical architectures and mentor peers.

Skills

Databricks

Apache Spark

Python

SQL

Data Architecture

Azure

AWS

GCP

Delta Lake

Delta Live Tables

Unity Catalog

Data Modeling

ETL/ELT

Lakehouse

CI/CD

Azure Data Factory

Airflow

Snowflake

Kafka

Education

Bachelor's degree in Computer Science, IT, Engineering, or related field

Tools

Azure Data Factory

ADLS

AWS S3

Glue

Snowflake

Kafka

Airflow

Git

CI/CD tooling

Job description

Innova software Services Inc in Charlotte, NC is seeking a Senior Data Engineer / Data Architect with deep Databricks hands-on expertise and Insurance domain experience to design, build, and implement scalable enterprise data solutions that support analytics at scale.

You will lead the development of data lakes and lakehouse platforms, optimize Spark jobs, govern data quality and security, and translate business requirements into robust architectures across policy, claims, underwriting, and