Data Scientist: Spark, ETL & Analytics Dashboards

Avalore, LLC; Chantilly (VA); On-site; USD 120,000 - 160,000; Full time

Job summary

Avalore, LLC is seeking a Data Scientist to support new system capabilities and extend bulk data pipelines across customer cloud environments. The role requires security clearance, experience with Spark, PySpark, Python, and ETL processes, plus building analytic reports in CloudWatch and Kibana.

Responsibilities include data mapping, loading from varied formats, data quality analysis, and collaborating with customers and partners on objectives within an Agile framework.

Qualifications

• Active Security Clearance with appropriate Polygraph.
• Experience with Spark, PySpark and Python.
• Experience data mapping, extraction, transformation and loading.
• Experience building analytic reports in CloudWatch and Kibana.
• Experience converting OS and Data Logs into reports and dashboards.
• Experience RegEx.
• Experience SQL, MySQL, PostgreSQL.
• Experience XML, JSON data formats.
• Experience IDEs and Data Modeling via Notebooks and Visual Studio.
• Experience ETLing data from diverse formats into structured data.
• Experience data review and data quality analysis.
• Experience ETL design documentation with mappings and data dictionaries.
• Experience interfacing with customers and integration partners.
• Experience supporting Agile development.

Responsibilities

• Support requirements for new system capabilities and maintenance of bulk data pipelines.
• Enhance applications in customers' cloud computing environments.
• Collaborate with stakeholders to clarify objectives and deliver data solutions.

Skills

Big data processing

SQL

ETL

Data modeling

Data quality

Python

Spark

Pyspark

Notebooks

Visual Studio

Kibana

Databricks

ELK stack

Pandas

Cloud services

DevOps tools

Tools

Spark

PySpark

Python

SQL

MySQL

PostgreSQL

XML

JSON

Notebooks

Visual Studio

Databricks

Elastic MapReduce

ELK stack

CloudWatch

Lambda

SNS/SQS

EC2

Job description

Avalore, LLC is seeking a Data Scientist to support new system capabilities and extend bulk data pipelines across customer cloud environments. The role requires security clearance, experience with Spark, PySpark, Python, and ETL processes, plus building analytic reports in CloudWatch and Kibana.

Responsibilities include data mapping, loading from varied formats, data quality analysis, and collaborating with customers and partners on objectives within an Agile framework.