Data Engineer

dadavidson; United States; On-site; USD 120,000 - 180,000; Full time

Job summary

D.A. Davidson Companies is seeking a Senior Data Engineer to expand and optimize data pipeline architecture. The role collaborates with software teams, DBAs and data analysts to ensure scalable data delivery that aligns with best practices.

Candidates should have 5+ years in data engineering, strong SQL and Python skills, and experience with Microsoft cloud tech in hybrid architectures. Commitment to data security and client confidentiality is required.

Qualifications

• 5+ years of Data Engineering experience in an oversight capacity.
• Experience in Financial Services industry.
• Experience with Microsoft cloud-based technologies in a hybrid architecture.
• Advanced TSQL knowledge; end-to-end data pipeline development.
• Experience with data modeling and metadata management.

Responsibilities

• Design, implement, and maintain a robust data pipeline architecture.
• Identify and implement internal process improvements for data delivery.
• Build service-based architecture for publication of data objects.
• Collaborate with data architects and analysts to evolve the data environment.
• Support data-related technical issues for business and IT stakeholders.
• Develop frameworks and best practices to ensure data integrity.

Skills

Data engineering

SQL

Python

PowerShell

Data governance

Tools

SSIS

Azure Integration Services

MS SQL Server

Microsoft Fabric

ActiveBatch

Job description

D.A. Davidson Companies is an independent, employee-owned company with a rich history spanning 90 years. We are dedicated to conducting our business in accordance with the highest standards of integrity and ethics, and delivering outstanding service to our clients and each other. We support a friendly, open and supportive culture, and encourage candid communication and productive engagement that make our companies and each of us better. Just as we work to improve our clients' financial well-being, we also work to strengthen local communities-and giving back is one of our core values. You can learn more about our company culture and impact in our latest annual report.

Summary:

Data Engineer is responsible for expanding and optimizing our data and data pipeline architecture, as well as monitoring, troubleshooting and performance tuning data flows used in the collection, integration, and provisioning of data. This role requires experience building data pipelines and optimizing entire data systems as well as greenfield opportunities. This role works alongside other software development teams, database architects, data analysts and end users on complex data initiatives, and ensures our data delivery architecture is consistent with best practices.

Company Overview:

D.A. Davidson Companies is an independent, employee-owned company with a rich history spanning 90 years. We are dedicated to conducting our business in accordance with the highest standards of integrity and ethics, while delivering outstanding service to our clients and coworkers. We support a friendly, open and adaptive culture, and encourage candid communication and productive collaboration across throughout the firm. Just as we work to improve our clients and employees' well-being, we also work to strengthen local communities-and giving back is one of our core values.

Qualifications:

• 5 or more years of Data Engineering experience in an oversight capacity
• Extensive domain-specific experience in the Financial Services industry
• Experience with Microsoft cloud-based technologies in a hybrid architecture
• Advanced working TSQL knowledge and experience working with relational databases, query authoring, stored procedure development, end-to-end data pipeline development
• Experience performing operational analysis on internal and external data and processes to uncover failure points and identify opportunities for improvement
• Strong analytic skills related to working with complex, multi-layered data pipelines and dependent datasets
• Experience facilitating calls to capture end-user requirements with technical and non-technical users
• Holistic awareness of processes supporting data preparation, metadata, dependencies, lineages, compliance and workload management
• Understanding of data modeling concepts like:
  • Conceptual data modeling
  • Logical data modeling
  • Relational data modeling
  • Dimensional and multi-dimensional data modeling
• Understanding of meta-model and metadata management requirements
• Specific skills include:
  • Extensive experience with SQL syntax
  • Experience designing and implementing data structures and interfaces for applications and business processes
  • Experience with Microsoft Fabric: pipelines, notebooks, Lakehouse/Warehouse concepts, and Medalion architecture
  • Experience with relational database platforms: preferably MS SQL Server
  • Experience with data pipeline tools such as: SSIS and Azure Integration Services
  • Experience using orchestration tools such as: ActiveBatch
  • Experience with web services and API integrations
  • Experience with scripting languages such as: Python and Powershell
  • Experience with Agile, DevOps and DataOps methods and technologies: Source Code Control, Versioning, Test Automation, Continuous Integration, Automated Deployment, Recoverability, Telemetry, etc.
• Commitment to maintain client confidentiality and data security
• Communicate in a clear and service-oriented manner; use appropriate, professional language and grammar to effectively exchange ideas and information
• Ability to proactively work with both external and internal clients; relate with others in a professional manner to accomplish work responsibilities and objectives
• Ability to maintain regular, predictable attendance

Job Responsibilities:

• Design, implement and maintain a robust, efficient and compliant data pipeline architecture
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build and maintain a service-based architecture for the publication of consumable data and information objects
• Work with data architects and data analysts to enhance and evolve the data environment
• Work with Business and IT stakeholders to assist with data-related technical issues and support their data and information requirements
• Define and develop frameworks, guidelines, best practices and examples to ensure dat