Data Engineer

SUMTER ELECTRIC COOPERATIVE, INC; Sumterville (FL); On-site; USD 110,000 - 150,000; Full time

Job summary

SECO Energy is seeking a Senior Data Engineer to design, build, test, deploy, and support governed data pipelines and analytics-ready data products on modern cloud platforms.

You will implement ingestion, transformation, and publishing patterns across lakehouse and warehouse environments, using PySpark, Spark SQL, Python, and SQL to deliver scalable, battle-tested data solutions for enterprise reporting and analytics.

Qualifications

• Bachelor's degree from an accredited institution is required.
• 4+ years of progressively responsible data engineering/ analytics experience.
• Experience with cloud data platforms and data governance.
• Knowledge of lakehouse/warehouse platforms and data pipelines.
• Strong collaboration and problem-solving skills.

Responsibilities

• Design, build, test, deploy, and support governed enterprise data pipelines.
• Implement ingestion, staging, transformation, conformance, and publishing patterns.
• Develop PySpark, Spark SQL, Python, and SQL transformations that are scalable and restartable.
• Implement full-refresh, incremental, CDC, merge, and partition-aware loads.
• Design metadata, lineage, and governance for data products.
• Build conformed dimensions, facts, and curated marts for analytics.
• Partner with SMEs to define keys, grain, and measures for analytics.
• Apply governance, metrics, and quality controls across pipelines.

Skills

PySpark

Spark SQL

Python

SQL

ETL/ELT

CI/CD

Data modeling

MDM

Education

Bachelor's degree in CS/IS/Data Engineering or related field

Tools

Microsoft Fabric

Azure Data Lake Storage

OneLake

Databricks

Snowflake

Synapse

Power BI

Git-based DevOps

Job description

Design, build, test, deploy, and support governed enterprise data pipelines and analytics-ready data products on modern cloud platforms.

Responsibilities

• Design, build, test, deploy, and support modern data platform pipelines and lakehouse or warehouse data products.
• Implement ingestion, staging, transformation, conformance, and publishing patterns across raw/bronze, stage, silver, gold, and certified consumption layers.
• Build PySpark, Spark SQL, Python, and SQL transformations that are distributed, restartable, idempotent, and operationally supportable.
• Implement full-refresh, incremental, CDC, merge, and partition-aware load patterns where appropriate.
• Design source registration, audit stamping, row hashing, metadata capture, and lineage-friendly table handling.
• Build conformed dimensions, facts, bridge tables, reference tables, and curated marts for analytics and semantic model consumption.
• Partner with business SMEs and analysts to define business keys, surrogate keys, grain, relationships, slowly changing dimension needs, and metric-ready structures.
• Implement metadata-driven configuration patterns for sources, tables, columns, jobs, schemas, quality rules, and load audits.
• Support MDM-ready outputs, reference-data management, and stewardship workflows for core entities including customer, account, product, location, device, meter, vendor, or chart of accounts.
• Maintain clear documentation of table contracts, lineage, refresh behavior, data quality rules, and operational dependencies.
• Apply governance and security practices across data products, including access control, sensitivity awareness, ownership, cataloging, endorsement, certification, and audit evidence.
• Support Microsoft Purview, OneLake catalog, or equivalent governance capabilities for lineage, data cataloging, domains, glossary terms, quality status, and data product discoverability.
• Design solutions that support domain-oriented ownership, data mesh principles, reusable platform patterns, and federated governance.
• Ensure production reports and semantic models consume approved data products rather than unfinished engineering layers unless explicitly approved for validation.

Requirements

• Minimum: Four (4) year bachelor’s degree from an accredited institution in computer science, information systems, data engineering, software engineering, mathematics, statistics, engineering, or a related technical field.
• Minimum: Four (4) years of progressively responsible experience in data engineering, ETL/ELT development, analytics engineering, database development, BI engineering, software development, cloud data platform engineering, or related technical work.
• Preferred: Certifications or formal training in Microsoft Fabric, Azure Data Engineer, Databricks, Spark, Snowflake, data governance, DevOps, data modeling, or cloud architecture.
• Preferred: Experience designing and operating lakehouse or warehouse platforms, medallion data flows, Spark/PySpark workloads, Data Factory or pipeline orchestration, metadata-driven frameworks, dimensional models, master data, and governed semantic-ready marts.
• Preferred: Experience with Microsoft Fabric, Azure Data Lake Storage, OneLake, Delta Lake, Synapse, Databricks, Snowflake, Power BI semantic models, Git-based DevOps, and enterprise operational data.
• Ability to operate a variety of office equipment, including a personal computer, printers, copy machines, telephone.
• Ability to work irregular hours for assignment completion and flexibility to change scheduling and report to work on short notice during emergency situations.
• Irregular hours in evenings and on weekends are possible during peak business times and during storm season.
• Normal work hours: eight (8) hours between 7:00 am and 5:00 pm, Monday through Friday.
• Successful completion of pre-employment background check, physical and drug screen.
• Follows safety procedures diligently, identifies potential hazards, and takes appropriate action to maintain a safe working environment.
• Provides exceptional service to members, actively listening to their needs and ensuring their satisfaction.
• Acts with honesty and integrity in all tasks, maintaining transparency and ethical standards.
• Demonstrates a strong work ethic by consistently meeting deadlines and achieving high performance in all tasks.
• Contributes to an inclusive culture by respecting and valuing diverse perspectives and collaborating effectively with all team members.
• Takes responsibility for actions and decisions to meet commitments and deliver high-quality work.
• Works collaboratively with team members, sharing information and supporting collective goals.
• Applies engineering judgment to design reliable, scalable, secure, and maintainable data solutions aligned with organizational architecture and tradeoffs across performance, governance, and usability.
• Applies PySpark, Spark SQL, Python, and SQL to build scalable data pipelines for ingestion, transformation, cleansing, enrichment, and validation while optimizing performance and avoiding distributed processing anti-patterns.
• Leverages expertise in modern lakehouse and warehouse platforms, including Fabric, ADLS, OneLake, Databricks, Snowflake, and Synapse, to architect and optimize scalable, secure, and governed data environments using industry-standard storage and data management practices.
• Leverages expertise in Data Factory, Dataflows, notebooks, APIs, and orchestration tools to develop and maintain reliable data pipelines supporting scalable processing, operational resilience, monitoring, and production stability.
• Demonstrates expertise in dimensional modeling and analytical design using facts, dimensions, keys, SCDs, bridge tables, conformed dimensions, and aggregate structures for semantic models and business intelligence solutions.
• Establishes governance frameworks covering metadata, lineage, cataloging, auditability, and data quality; implements validation controls including reconciliation, integrity checks, and duplication detection with security compliance.
• Applies MDM principles (entity resolution, survivorship, stewardship) and domain-driven design with data-as-a-product thinking, reusable contracts, and governed consumer-ready data products.
• Applies Git, CI/CD, code reviews, automated deployments, environment management, and versioning to ensure controlled, repeatable, reliable data delivery.
• Exhibits understanding of Power BI, Direct Lake, semantic models, gold-layer design, and performance-optimized analytics consumption.
• Applies AI tools (Fabric Copilot, code generation assistants) responsibly, ensuring validation, privacy compliance, and safe production usage.

Technologies

• Microsoft Fabric
• Azure data services
• Azure Data Lake Storage
• OneLake
• Delta Lake
• Databricks
• Snowflake
• Synapse
• PySpark
• Spark SQL
• Python
• SQL
• ETL/ELT
• CI/CD
• Change data capture (CDC)
• Data Factory
• Dataflows
• Notebooks
• APIs
• Git-based DevOps
• Git
• Power BI
• Direct Lake
• Microsoft Purview
• Delta / open table formats
• Dimensional modeling
• Data mesh
• Master data management (MDM)
• DevOps
• DataOps
• Fabric Copilot

Verification

The above qualifications and competencies may be verified through a combination of education, experience, interview questions and technical skills exercise(s).

Work Environment and Physical Demands

• Regularly required to sit and talk or hear.
• Occasionally required to walk; stand; use hands to finger, handle, or feel; reach with hands and arms; climb or balance; stoop, kneel, crouch, or crawl.
• Must regularly lift and/or move up to 10 pounds.
• Specific vision abilities required: close vision, distance vision, color vision, ability to adjust focus.
• General sedentary office environment.
• Noise level usually moderate.

Workplace and Compliance Statements

• SECO Energy participates in E-Verify.
• SECO Energy does not accept unsolicited resumes from any source other than directly from applicants.
• This posting may be closed and no longer available to receive applications upon SECO Energy's discretion.