Senior Data Engineer (Remote - US)

Energy Vault; United States; Remote; USD 140,000 - 165,000; Full time

Job summary

A leading company in the energy sector is seeking a Senior Data Engineer to join their Information Systems team. In this role, you will design and maintain data platforms, leveraging analytics tools and cloud technologies like AWS. This position plays a crucial role in enhancing energy efficiency and reducing greenhouse gas emissions for major clients across North America. Your expertise in SQL, NoSQL, and Python will be pivotal in delivering high-quality data solutions.

Qualifications

• Minimum 8 years of job-related experience.
• High proficiency in SQL and Python for ETL development.
• Expertise in AWS services for building data solutions.

Responsibilities

• Build and manage scalable data ingestion pipelines.
• Lead data governance and ensure data quality.
• Design ETL processes for data transformation and validation.

Skills

SQL

Python

AWS

NoSQL

Education

Bachelor’s degree in computer science or information technology

Tools

Amazon S3

Amazon Redshift

Postgres

MySQL

Job description

We are currently seeking a Senior Data Engineer to join our Information Systems team to design, develop, and maintain data platforms that support the data needs across Energy Solutions. In this role, the ideal candidate is a passionate and highly skilled professional with expertise in analytics tools and cloud technologies like AWS, Azure or similar technologies. They should be proficient in programming languages such as SQL, NoSQL, and Python, capable of processing large data sets to deliver high-quality, customer-facing data solutions and insights. This unique position is perfect for individuals with technical prowess in data field who want to have an impact on energy efficiency markets and greenhouse gas reductions through our work for major North American utilities and other clients around the country.

Energy Solutions has a remote-friendly work environment for staff located throughout the United States. We also have offices in Oakland and Orange, California as well as Portland, Boston, New York and Chicago for those that wish to work from one of our offices.

Responsibilities include but are not limited to:

• Build, automate, and manage near-real-time scalable data ingestion pipelines for master data management, deep-learning, and predictive analytics.
• Build and maintain cloud native big data environments on AWS, that are highly secure, scalable, flexible, and highly performant using appropriate SQL, NoSQL and NewSQL technologies.
• Lead data governance and data profiling efforts to ensure data quality and proper metadata documentation for data lineage.
• Provide technical input into build/buy/partner decisions for all components of the data infrastructure.
• Partner closely with Data Scientists, BI developers, and Product Managers to design and implement data models, database schemas, data structures, and processing logic to support various data science, analytics, machine learning, and BI initiatives.
• Design and develop ETL (extract-transform-load) processes to validate and transform data, calculate metrics, and model features, populate data models etc., using Spark, Python, SQL, and other technologies in the AWS.
• Lead data governance and data profiling efforts to ensure data quality and proper metadata documentation for data lineage.
• Lead by example, demonstrating best practices for code development and optimization, unit testing, CI/CD, performance testing, capacity planning, documentation, monitoring, alerting, and incident response to ensure data availability, data quality, and usability.
• Define SLAs for data availability and correctness. Automate data availability and quality monitoring and respond to alerts when data delivery SLAs are not being met.
• Communicate progress across organizations and levels from individual contributor to executive. Identify and clarify the critical few issues that need action and drive appropriate decisions and actions. Communicate results clearly.

Minimum Qualifications:

• Education: A bachelor’s degree in computer science or information technology
• A minimum of 8 years' of job related experience
• Programming Proficiency: High proficiency in programming languages commonly used in ETL development, such as PLSQL, SQL, Python. Ability to write efficient SQL queries, SQL store procedures, develop scripts for data transformations, and utilize programming frameworks and libraries to create/enhance ETL mappings and workflows.
• AWS Services: Expertise in utilizing AWS services, including but not limited to Amazon s3, glue, data catalog, Amazon redshift, redshift spectrum and Amazon Athena. Ability to leverage these services to build scalable, reliable, and performant data pipelines and analytics solutions.
• Proficiency in working with relational databases such as Postgres, Oracle, MySQL, or SQL Server. Knowledge of database design, optimization techniques, and advanced querying capabilities.
• Experience in performance tuning and optimizing database operations.
• Familiarity with data governance frameworks and data security best practices.
• Passion for learning new technologies, staying up to date with industry trends, and exploring innovative approaches to ETL development.

Compensation is commensurate with experience with the pay band of $140,000 - $165,000/Annually