Software Engineer- Data Engineering

Noctua Technology; Washington, Virginia, California (District of Columbia, MN, MO); Remote; USD 103,000 - 170,000; Full time

Job summary

Noctua Technology is seeking a Data Engineer to design and optimize data pipelines, storage solutions, and the infrastructure supporting data-driven initiatives. Remote-friendly with proximity preference for CA or DC Metro Area teams.

You will collaborate with engineers, data scientists, and analysts to deliver scalable data infrastructure, perform exploratory analyses, and create impactful visualizations. Continuous learning and customer-focused delivery are encouraged.

Qualifications

• Strong SQL and relational database concepts.
• Experience with data warehouses, ETL tools (Beam, Airflow, Kafka).
• Experience with data analysis tools (Jupyter, Colab, Pandas).
• Experience with data visualization tools (Tableau, Looker, PowerBI, Qlik, Superset).
• Experience shaping data strategies and migrating data to production.

Responsibilities

• Acquire, clean, and preprocess diverse datasets.
• Build infrastructure for ETL and data pipelines to ensure data quality.
• Migrate data from on-prem to cloud technologies and optimize warehouses.

Skills

SQL

Relational databases

Workflow orchestration

Python

Java

C++

Git

Problem solving

Communication

Data visualization

Cloud platforms

Jupyter/Colab

Pandas

Spark

Hadoop

Education

Bachelor's or advanced degree in Computer Science, Data Science, ML, or related field

Tools

MySQL

PostgreSQL

Beam

Airflow

Kafka

Spark

Hadoop

Docker

Kubernetes

Job description

The Data Engineering & AI/ML capability serves as the backbone of our organization's data-driven operations. Comprising skilled professionals with expertise in data engineering, database management, and infrastructure architecture, our team is dedicated to designing and maintaining robust systems that ensure the seamless flow and accessibility of data. We specialize in constructing and optimizing data pipelines, implementing efficient storage solutions, and orchestrating the infrastructure necessary to support the company's data-driven initiatives. Our focus extends beyond data management, encompassing the development of scalable and reliable systems that empower organizations to extract valuable insights, make informed decisions, and drive innovation.

We are seeking a talented and motivated Data Engineer to join our dynamic Data Engineering & Analytics team. As a key member of our engineering team, you will play a crucial role in constructing and optimizing data pipelines, implementing efficient storage solutions, and orchestrating the infrastructure necessary to support our customer’s data-driven initiatives.

• Location: Primarily Remote. Candidates must be based in CA or DC Metro Area for proximity to project and client teams.
• Security Clearance Requirement: Applicants must be US citizens and eligible to obtain and maintain an active Secret security clearance or above.

Key Responsibilities

Data Collection and Processing

• Acquire, clean, and preprocess diverse datasets from various sources.
• Build required infrastructure for optimal extraction, transformation and loading of data from various data sources using CSP managed services and SQL technologies
• Develop and maintain data pipelines to ensure a continuous flow of high-quality data

Data Migrations & Optimization

• Develop data migration strategies and schemas to lead customer migrations from on-prem to cloud technologies
• Perform data migration activities
• Optimize databases and data warehouses for efficient querying and data storage

Data Analysis and Visualization

• Perform exploratory data analysis to uncover patterns, trends, and insights.
• Create visualizations and reports to communicate findings effectively to stakeholders both internally and externally

Collaboration and Documentation

• Collaborate with cross-functional teams, including software engineers, domain experts, and business analysts, to understand requirements and deliver integrated solutions.
• Create and maintain comprehensive documentation for code, algorithms, and models. Ensure that the knowledge is shared and accessible within the team.
• Act on client feedback constructively to improve services and outcomes.
• Continuously seek ways to enhance the overall customer experience.

Continuous Learning and Innovation

• Stay updated on the latest developments in machine learning, data science, and analytics.
• Drive innovation by proposing and implementing new techniques and technologies.

Qualifications

• Solid understanding and experience with SQL and relational database concepts
• Solid understanding of database technologies, data warehouses, and ETL tools (e.g., MySQL, PostgreSQL, Beam, Airflow, and Kafka).
• Experience with data analysis tools (eg., Jupyter, Colab, Pandas)
• Experience with data visualization tools (eg., Tableau, Looker, PowerBI, Qlik, and SuperSet)
• Previous experience developing data strategies and facilitating data migrations into production systems.
• Experience with cloud platforms (e.g., AWS, Azure, GCP).
• Proficiency in programming languages such as Python, Java, or C++.
• Strong software engineering skills with an emphasis on writing clean, modular, and maintainable code.
• Familiarity with version control systems (e.g., Git) and collaborative development workflows.
• Excellent problem-solving and critical-thinking skills.
• Effective communication skills and ability to work in a collaborative team environment.

Preferred qualifications:

• Bachelor's or advanced degree in Computer Science, Data Science, Machine Learning, or a related field.
• Experience with other database technologies (eg., NoSQL, Graph)
• Any of the below cloud certifications:
  • Google Cloud Professional Cloud Architect
  • Google Cloud Professional Database Engineer certification
  • Google Cloud Professional Data Engineer
• Experience with additional data processing tools and technologies (e.g., Spark, Hadoop).
• Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes).

Salary Range

$102,850 - $170,000