Data Engineer II

PDS Inc, LLC; Seattle (WA); On-site; USD 85,000 - 121,000; Full time

Job summary

PDS Inc. is seeking a Senior Data Engineer to join the Infrastructure Automation team in Seattle.

You will design, develop, and operate a data service platform using Python, SQL, and AWS technologies to build ETL, analytics, and data quality components for scalable infrastructure reporting and ML workflows. You will collaborate with business owners, product teams, and tech leaders to gather requirements and deliver robust data pipelines while leveraging AWS CodeDeploy, CodePipeline, CDK and

Qualifications

• 5+ years of related experience.
• Expertise in data modeling and warehousing with ETL pipelines.
• Experience with Python and SQL; working with big data tech.

Responsibilities

• Interface with teams to extract, transform, load data from diverse sources using Python, SQL and AWS data tech.
• Explore new AWS technologies to improve capabilities and efficiency.
• Design and implement complex data pipelines and related solutions.
• Collaborate with business owners and BI engineers to deliver data for analysis.
• Create extensible, maintainable data designs with long-term vision.
• Improve tooling and scale existing solutions per stakeholder needs.
• Understand business requirements and implement data for analytics and reporting.
• Partner with stakeholders to provide data revealing patterns and anomalies.

Skills

Data modeling

Warehousing

ETL pipelines

Python

SQL

Big data technologies

Reporting

Analytical skills

AWS big data

Tools

AWS CodeDeploy

CodePipeline

CDK

CloudFormation

Redshift

Glue

S3

IAM

CloudWatch

Job description

We are looking for a Data Engineer to join the Infrastructure Automation team. Our success depends on our world-class network and hardware infrastructure; we’re handling massive scale and rapid integration of emergent technologies. Our goal is to become “The Infrastructure Platform” for the world. The Infrastructure Automation team is responsible for delivering the software that powers our infrastructure. As a Senior DE you will create solutions to integrate with multi heterogeneous data sources, aggregate and retrieve data in a fast and safe mode, curate data that can be used in reporting, analysis, machine learning models and ad-hoc data requests. You should have excellent business and communication skills to be able to work with business owners, Product teams and Tech leaders to gather infrastructure requirements, design data infrastructure, build up data pipelines and data-sets to meet business needs. You will be responsible for designing, developing, and operating a data service platform using Python, Airflow, and SQL to build the various ETL, analytics, and data quality components. You’ll automate deployments using AWS CodeDeploy, AWS CodePipeline, AWS Cloud Development Kit (CDK), and AWS Cloud Formation. You will work with AWS services like Redshift, Glue, S3, IAM, CloudWatch, and more.

Responsibilities

• Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using Python, SQL and AWS big data technologies.
• Explore and learn the latest AWS technologies to provide new capabilities and increase efficiencies of the team.
• Designing and implementing complex pipelines and other Data Engineering solutions.
• Work closely with business owners, developers, Business Intelligence Engineer to explore new data sources and deliver the data.
• Create extensible designs and easy to maintain solutions with the long term vision in mind
• Improve tools, processes, scale existing solutions, create new solutions as required based on team and stakeholder needs
• You will work with business customers in understanding the business requirements and implementing solutions to support analytical and reporting needs.
• A good candidate can partner with business owners directly to understand their requirements and provide data which can help them observe patterns and spot anomalies.

Required Skills & Experience

• 5+ years of related experience.
• Expertise with data modeling, warehousing and building ETL pipelines
• Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
• Should have experience developing complex and a variety of reports.
• A good candidate has strong analytical skills and enjoys working with large complex data sets.
• Expert knowledge of SQL Preferred - Experience with big data technologies such as: Hadoop, Hive, Spark, EMR - Experience operating large data warehouses.

Compensation: $75/hour

PDSINC, LLC is an Equal Opportunity Employer.