Senior Data Engineer - Remote, ETL & Big Data

West 4th Strategy; United States; Remote; USD 125,000 - 135,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

401k

Job summary

West 4th Strategy is seeking an experienced Data Engineer to support the National Guard Bureau's Minuteman project team. You will design end-to-end data pipelines and maintain the enterprise data environment within the DoW's analytics platform.

Work with cross-functional teams to model data and implement ETL using Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL, while ensuring governance and security compliance. This is a full-time role with comprehensive benefits.

Qualifications

• 3–10 years of directly related experience.
• 3+ years of experience as a Data Engineer.
• Proficiency in ETL processes and firsthand experience with Databricks, Python, Spark, Scala, JavaScript/JSON, SQL, and Jupyter Notebooks.
• Experience with large datasets and leveraging big data technologies to process and analyze data efficiently.
• Understanding of data modeling/visualization, database design principles, and data governance practices.

Responsibilities

• Designs, develops, and implements end to end data pipelines utilizing ETL processes and technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL.
• Works closely with cross functional teams to understand data requirements and design optimal data models and architectures.
• Creates and optimizes data pipelines from scratch, ensuring scalability, reliability, and high performance processing.
• Performs data cleansing, data integration, and data quality assurance activities to maintain the accuracy and integrity of large datasets.
• Leverages big data technologies to efficiently process and analyze large datasets.
• Collaborates with data scientists, analysts, and stakeholders to provide timely and accurate data insights and support decision making processes.
• Troubleshoots data related problems and provides innovative solutions to complex data challenges.
• Implements and enforces data governance policies and procedures, ensuring compliance with regulatory requirements and industry best practices.

Skills

Databricks

Python

Spark

Scala

JavaScript/JSON

SQL

Jupyter Notebooks

Data governance

Data modeling

Big data tech

Education

Bachelor/Master in related field

Job description

West 4th Strategy is seeking an experienced Data Engineer to support the National Guard Bureau's Minuteman project team. You will design end-to-end data pipelines and maintain the enterprise data environment within the DoW's analytics platform.

Work with cross-functional teams to model data and implement ETL using Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL, while ensuring governance and security compliance. This is a full-time role with comprehensive benefits.