The ETL Developer is responsible for designing, developing, and maintaining scalable and efficient data pipelines that move and transform data from various sources into target data warehouses or data lakes, ensuring data quality and accessibility for analytics.
Key Responsibilities
- Pipeline Development: Design, build, and deploy robust ETL/ELT pipelines using tools like Informatica, Talend, Apache NiFi, or cloud-native services (e.g., AWS Glue, Azure Data Factory).
- Data Transformation: Write complex SQL queries, stored procedures, and scripts to clean, transform, and aggregate data according to business rules.
- Monitoring and Optimization: Monitor pipeline performance, troubleshoot data flow issues, and optimize ETL jobs for efficiency and latency.
- Documentation: Create and maintain technical documentation for data mappings, ETL processes, and database schemas.
- Data Quality: Implement data validation and error handling logic to ensure the accuracy and consistency of data within the warehouse.
Required Qualifications
- Bachelor's degree in a technical field or equivalent work experience.
- 3+ years of hands-on experience as an ETL Developer or Data Engineer.
- Expert proficiency in SQL for writing complex queries and procedures.
- Proven experience with at least one major ETL tool (e.g., Informatica, Talend, or a cloud equivalent).
- Familiarity with data warehousing concepts and data modeling (e.g., Star Schema, Snowflake).