Cloud ETL Developer,Richmond, VA,United States

Intellibee; Richmond (VA); On-site; USD 90,000 - 120,000; Full time

Job summary

The Virginia Department of Transportation is looking for a senior ETL Developer to manage data assets and implement a cloud-based data management platform. The role requires extensive experience in data integration, warehousing, and analytics technologies. Candidates should be proficient in Azure Data Factory, Databricks, and SQL, with a strong understanding of database architectures and ETL processes. This position involves collaboration with various teams to optimize data storage and reporting capabilities.

Qualifications

• 10 years experience in ETL processes and business intelligence.
• 10 years experience with Azure Cloud engineering.
• 5 years experience with Snowflake.

Responsibilities

• Designs and develops integrations for the Enterprise Data Asset program.
• Develop data engineering processes leveraging cloud architecture.
• Work closely with data analysts and data scientists.

Skills

Data Integration

Analytical Skills

Problem Solving

Communication

Tools

Azure Data Factory

Databricks

PowerBI

Tableau

SQL

Job description

SR ETL Developer

The Virginia Department of Transportation (VDOT) Information Technology Division is seeking a senior ETL developer to ingest/transform and load Data Assets and implement a cloud-based data management platform that will support the agency.

ETL developer to extract business data and load it into a data warehousing environment. Design, program and test the performance of the system. Consult with various teams to understand the agency’s data storage needs and develop data warehousing options. Deep knowledge of coding languages, such as Azure Data Factory, Databricks, python, XML, and SQL. Well-versed in warehousing architecture techniques such as MOLAP, ROLAP, ODS, DM, and EDW.

Responsibilities:

• Designs and develops integrations for the Enterprise Data Asset program, ETL processes, and business intelligence.
• Develop data engineering processes that leverage a cloud architecture and will extend or migrate our existing data pipelines to this architecture as needed.
• Design and supports the DW database and table schemas for new and existent data sources for the data hub and warehouse. Design and development of Data Marts.
• Work closely with data analysts, data scientists, and other data consumers within the business in an attempt to gather and populate data hub and data warehouse table structure, which is optimized for reporting.
• The Data developers partners with Data modeler and Data architect in an attempt to refine the business’s data requirements, which must be met for building and maintaining Data Assets.
• Understanding of Agile methodologies and processes

Preferred Skills:

• Advanced understanding of data integrations.
• Strong knowledge of database architectures
• Strong analytical and problem-solving skills
• Ability to build strong relationships both internally and externally
• Ability to negotiate and resolve conflicts
• Ability to effectively prioritize and handle multiple tasks and projects
• Strong written and verbal communication skills
• Desire to learn, innovate and evolve technology

Computer Skills/Ms Office/Software:

• Excellent computer skills and be highly proficient in the use of Ms Word, PowerPoint, Ms Excel, MS Project, MS Visio, and MS Team Foundation Server, which will all be necessary in the creation of visually and verbally engaging ETL, data designs and tables as well as the communication of documentation and reporting.
• Deep passion for data analytics technologies as well as analytical and dimensional modeling. The candidate must be extensively familiar with ETL (Extraction, Transformation & Load), data warehousing, and business intelligence tools such as business objects,PowerBI and Tableau.
• The candidate must also have vast knowledge of database design and modeling in the context of data warehousing.
• Experience with key data warehousing architectures including Kimball and Inmon, and has a broad experience designing solutions using a broad set of data stores (e.g., HDFS, Azure Data Lake Store, Azure Blob Storage, Azure SQL Data Warehouse, Databricks

Technologies Required:

• Scripting experience with Windows and/or Python, Linux Shell scripting

Skill Matrix:

• Required 10 Years Designs and develops systems for the maintenance of the Data Asset Program, ETL processes, and business intelligence.
• Required 10 Years Design and supports the DW database and table schemas for new and existent data sources for the data hub and warehouse. Design and development of Data
• Required 10 Years Work closely with data analysts, data scientists, and other data consumers within the business in an attempt to gather and populate data hub and data
• Required 10 Years Advanced understanding of data integrations.· Strong knowledge of database architectures, strong understanding of ingesting spatial data
• Required 10 Years Ability to negotiate and resolve conflicts,· Ability to effectively prioritize and handle multiple tasks and projects
• Required 10 Years Excellent computer skills and be highly proficient in the use of Ms Word, PowerPoint, Ms Excel, MS Project, MS Visio, and MS Team Foundation Server
• Required 10 Years Experience with key data warehousing architectures including Kimball and Inmon, and has a broad experience designing solutions using a broad set of data
• Required 10 Years expertise in Data Factory v2,Data Lake Store, Data Lake Analytics, Azure Analysis Services, Azure Synapse
• Required 10 Years Operating System Environments (Windows, Unix, etc.).· Scripting experience with Windows and/or Python, Linux Shell scripting
• Required 10 Years Experience in AZURE Cloud engineering
• Desired 5 Years Experience with Snowflake