The Data Engineer II is responsible for processing structured and unstructured data, validating data quality, and developing and supporting data products. The Data Engineer also plays a role in Agile planning, providing advice and guidance, and monitoring emerging technologies.
WHAT YOU'LL DO
Key Responsibilities
Process Structured and Unstructured Data
- Gathers and processes raw, structured, semi-structured, and unstructured data using batch and real-time data processing frameworks.
- Understands and enforces appropriate data master management techniques.
Data Validation and Quality
- Ensures data quality and implements tools and frameworks for automating the identification of data quality issues.
- Work with internal and external data providers on data validation providing feedback and making customized changes to data feeds and data mappings.
Agile Planning
- Understands the challenges that the analytics organization faces in their day-to-day work and partner with them to design viable data solutions.
- Recommends improvements to processes, technology, and interfaces that improve the effectiveness of the team and reduce technical debt.
Product Development
- Implements and optimizes data solutions in enterprise data warehouses and big data repositories.
- Installs, maintains, monitors, and supports business intelligence, distributed computation, and big data analytics tools.
Application Support
- Provides ongoing support, monitoring, and maintenance of deployed products.
Emerging Technology Monitoring
- Actively participates in the engineering community, staying up to date on new data technologies and best practices and shares insights with others in the organization.
WHO YOU ARE
Minimum Qualifications
- Applicants must currently be authorized to work in the United States for any employer without current or future sponsorship. No OPT, CPT, STEM/OPT or visa sponsorship now or in future
- Must live within a commutable distance of Atlanta GA
- Bachelor’s degree in Computer Science or related field and minimum 2+ years of related Data Engineering experience or Masters degree and at minimum 1 year of professional Data Engineering experience
- Must have professional working experience as a Data Engineer beyond internships
- Working experience with batch and real-time data processing frameworks
- REQUIRED: Working experience with data modelling, data access, schemas, and data storage techniques within Snowflake
- Working experience in design, development, and implementation of highly scalable, high-volume software systems and components, client-facing web applications, and major Internet-oriented applications and systems
- Working experience working with relational databases such as SQL, MySQL, Postgres/PostgreSQL
- Working experience with business intelligence tools and platforms
- Working experience with data quality tools
- Working experience with application lifecycle methodologies (e.g. waterfall, agile, iterative)
- Experience with ETL processes and tools
- Experience working with Git
Preferred Qualifications
- Scrum Developer Certification or equivalent
- Working experience with Sisense, Tableau or Microsoft PowerBI platforms
- Working experience with data modeling
- Working experience with the Snowflake ecosystem
- Working experience with AWS data tools (Database Migration Service)
- Working experience with Python
- Experience working with Jira, Rally or similar tools