Data Engineer
Job Summary
We are seeking a highly motivated Data Engineer to design, develop, support, and optimize enterprise data pipelines and reporting solutions. The successful candidate will play a key role in maintaining cloud-based data platforms, ensuring data quality, supporting operational reporting, and enabling reliable analytics across the organization. This role will work closely with business stakeholders, data teams, and vendors to deliver scalable, secure, and well-governed data solutions.
Key Responsibilities
Data Pipeline Development & Support
- Design, build, and maintain ETL/ELT pipelines that ingest data from source systems into cloud-based storage, transformation, and analytics layers.
- Implement staging, intermediate, and reporting data structures based on enterprise data architecture standards.
- Develop and maintain extraction, transformation, loading, and automation scripts.
- Support workflow orchestration and scheduling using approved tools and frameworks (e.g., Airflow, Step Functions).
- Register and maintain pipeline metadata, schemas, and data assets within the enterprise data catalog.
- Ensure data pipelines are scalable, efficient, and aligned with established engineering standards.
Reporting Operations & Execution
- Execute, monitor, and support scheduled reporting processes across Finance, Operations, Marketing, Network, and Collections functions.
- Validate successful completion of reporting jobs and perform reruns when necessary.
- Investigate, document, and elevate unresolved issues with appropriate root-cause analysis.
- Perform data completeness and accuracy checks, including:
- Row Count Validation
- Source-to-Target Reconciliation
- Threshold and Variance Analysis
- Maintain execution logs, job schedules, runbooks, and process documentation.
- Distribute reports and data outputs through authorized and secure channels.
Data Quality & Governance
- Execute and monitor data quality validations focused on:
- Completeness
- Accuracy
- Consistency
- Referential Integrity
- Timeliness
- Uniqueness
- Support technical validation activities for project deployments and production releases.
- Assist in root-cause analysis and remediation of data quality issues.
- Monitor pipeline health, job performance, and system alerts using cloud-native monitoring tools.
- Maintain audit-ready logs, reports, and supporting evidence for critical data processes.
Cloud Data Platform Operations
- Support cloud-based data environments, including storage management, lifecycle policies, and access controls.
- Monitor and execute workloads on approved orchestration and scheduling platforms.
- Support analytics and data warehouse platforms through query optimization and performance monitoring.
- Manage deployment activities across development, testing, staging, and production environments.
- Ensure compliance with established operational and security standards.
Documentation & Standards Compliance
- Adhere to enterprise naming conventions, development standards, and change management processes.
- Create and maintain technical documentation, including:
- Data Flow Diagrams
- Source-to-Target Mapping Documents
- Transformation Logic Documentation
- Pipeline Specifications
- Contribute to data governance documentation, requirements checklists, and project evidence repositories.
- Identify and elevate potential data privacy and security concerns involving Personally Identifiable Information (PII).
- Recommend improvements that enhance automation, operational efficiency, monitoring, and reliability.
Stakeholder & Vendor Coordination
- Collaborate with business teams and report owners to understand reporting requirements and support data delivery needs.
- Maintain stable, reliable, and well-documented reporting datasets and data views for analytics platforms such as Tableau.
- Investigate and resolve data extraction, refresh, access, and reporting issues within established service levels.
- Coordinate with implementation partners and vendors to obtain schema definitions, field mappings, and technical documentation as required.
- Support cross-functional initiatives that improve data accessibility, quality, and business insights.
Qualifications
- Bachelor's degree in Computer Science, Information Technology, Data Engineering, Information Systems, Mathematics, Statistics, or a related field.
- Minimum of 3 years of experience in Data Engineering, ETL Development, Data Integration, or a similar role.
- Strong experience building and supporting ETL/ELT pipelines.
- Experience working with cloud-based data platforms and data warehouse technologies.
- Proficiency in SQL and data transformation techniques.
- Experience with workflow orchestration and scheduling tools such as Airflow, Step Functions, or equivalent.
- Knowledge of data quality, data governance, and data management best practices.
- Familiarity with reporting and visualization platforms such as Tableau.
- Strong analytical, troubleshooting, and problem-solving skills.
- Excellent documentation and stakeholder communication abilities.
Preferred Qualifications
- Experience with AWS, Azure, GCP, or other cloud platforms.
- Knowledge of data cataloging, metadata management, and data governance frameworks.
- Familiarity with CI/CD practices for data engineering and analytics solutions.
- Experience supporting enterprise-scale data platforms and reporting environments.