Senior Data Engineer

Paradigm Technology; South Carolina; On-site; USD 110,000 - 160,000; Full time

Job summary

Paradigm Technology is seeking a Senior Data Engineer to lead enterprise-scale data initiatives and architect cloud-native warehousing solutions.

You will design end-to-end ETL/ELT and analytics platforms using Databricks, Informatica, Python, SQL, and cloud services; set and enforce engineering standards; and collaborate with stakeholders to translate business requirements into robust data products that power analytics, reporting, and ML workflows.

Qualifications

• 5+ years of data engineering experience with ETL/ELT and cloud data warehousing.
• Strong expertise in Databricks, Python, and SQL.
• Experience with data governance and security controls.

Responsibilities

• Architect and deliver cloud-native data platforms and pipelines.
• Migrate legacy data systems to cloud warehouses.
• Define ETL/ELT standards and security controls.
• Mentor engineers and document standards.
• Engage stakeholders to translate requirements into roadmaps.

Skills

Python

SQL

Databricks

Informatica

CI/CD for data pipelines

Data governance

Leadership

Education

Bachelor’s degree in Computer Science or related + 5 years experience

Master’s degree in Computer Science or related + 3 years experience

Tools

Airflow

Prefect

Terraform

Bicep

Delta Live Tables

Job description

Paradigm Technology is seeking a Senior Data Engineer. This position is a 6-month contract to hire at our utility client in South Carolina. Must be able to drive to the office a few times a month.

Summary

Leads complex, enterprise-scale data engineering initiatives, architect cloud-native warehousing solutions. Design end-to-end ETL/ELT and analytics platforms using Databricks, Informatica, Python, SQL, and cloud services; set and enforce engineering standards; and engage stakeholders to translate business requirements into robust, secure, and performant data products that power analytics, reporting, and ML workflows.

Responsibilities

Responsible for architecting and delivering cloud-native data platforms and pipelines that provide reliable, governed, and scalable data to the enterprise. Leads migrations to modern cloud warehouses, defines and enforces best practices (ETL design, coding standards, security), and builds modular, reusable components for analytics/reporting/ML.

The role optimizes throughput, latency, and reliability, establishes SLAs and observability, and mentors engineers to uplift team capability.

Operates with broad autonomy, collaborating with executive and technical stakeholders to translate requirements into engineering specifications, cost-aware architectures, and operational runbooks. Ensures data governance (lineage, catalog, privacy), compliance, and resilience, while driving continuous improvement across performance tuning, incident response, and reliability engineering.

• Architects and develop enterprise-scale cloud data pipelines and warehousing solutions.
• Leads migration initiatives from legacy platforms to cloud-native data warehouses.
• Defines and enforces ETL/ELT design standards, coding practices, and security controls.
• Builds modular, reusable data components for analytics, reporting, and ML feature pipelines.
• Optimizes pipeline performance (throughput, latency, reliability) and establish SLAs/monitoring.
• Provides technical leadership and mentor junior engineers; document frameworks and patterns.
• Engage stakeholders; translate business requirements into engineering specs and roadmaps.
• Governs data quality, lineage, and cataloging; ensure compliance with privacy/security policies.

Must have experience with

• Languages and platforms: Advanced Python and SQL; Databricks (Spark, jobs, Delta), Informatica (ETL/ELT).
• Data warehousing: Dimensional/star modeling, Delta/Parquet, medallion architecture, performance tuning, partitioning/z-ordering.
• Engineering excellence: CI/CD for data pipelines, environment management (conda/poetry), testing (unit/integration/data tests), observability (logging/metrics/alerts).
• Data governance and quality: Unity Catalog (or equivalent), lineage, data contracts, DQ checks, privacy/security controls.
• Leadership and collaboration: Stakeholder management, specification writing, mentoring, documentation of standards and runbooks.

Preferred experience with

• Orchestration: Airflow/Prefect, event-driven and batch scheduling; SCD patterns, CDC/streaming (Kafka/EventHub).
• Platform components: Delta Live Tables, Feature Stores, Databricks SQL/Unity governance; Terraform/Bicep for infra-as-code.
• Integration: APIs, semi-structured/unstructured data (JSON/Avro), file ingestion at scale; federated/virtualized query layers.
• Domain: Utility/energy datasets (AMI, SCADA), compliance/governance in regulated environments.

Requirements

• Bachelor’s degree in computer science, Engineering, or related discipline + 5 years’ experience or master’s degree in computer science, Engineering, or related discipline + 3 years’ experience in data engineering (ETL/ELT, cloud warehousing, analytics) with advanced expertise in Databricks, Informatica, Python, SQL, and cloud data platforms.
• Must have strong background in cloud data architecture, governance, performance optimization, and technical leadership/communication.

Why Join Paradigm

At Paradigm, integrity drives innovation. You'll collaborate with curious, dedicated teammates, solving complex problems and unlocking immense data value for leading organizations. If you seek a place where your voice is heard, growth is supported, and your work creates lasting business value, you belong at Paradigm.

Learn more at https://pt-corp.com/

Policy Disclosure:

Paradigm maintains a strict drug-free workplace policy. All offers of employment are contingent upon successfully passing a standard 5-panel drug screen. Please note that a positive test result for any prohibited substance, including marijuana, will result in disqualification from employment, regardless of state laws permitting its use. This policy applies consistently across all positions and locations.

Data and AI Consulting for Digital Transformation

We implement digital transformation consulting solutions that help enterprises meet real-world challenges and drive measurable value.