Principal Splunk Engineer - Large-Scale SOC & Observability

ALPFA Baltimore Chapter; Charlotte (NC); On-site; USD 122,000 - 200,000; Full time

Benefits offered by this job

Discretionary incentive plan

Benefits eligible

Job summary

Bank of America seeks a Principal Splunk Engineer to lead the design, operation, and evolution of a large-scale Splunk Enterprise / Splunk Cloud deployment. The platform ingests multi-terabyte daily data across security, infrastructure, and applications and is a core component of our SOC and threat-detection capabilities.

The ideal candidate will drive capacity planning, upgrades, and platform hardening, while collaborating with SOC, IT, and App teams to enable high-quality security logs,

Qualifications

• 5+ years experience administering large Splunk Enterprise or Splunk Cloud environments

Responsibilities

• Architect, operate, and optimize a distributed, large-scale Splunk environment (indexer clusters, search head clusters, cluster masters, deployment servers, IDM, ADFS/SAML integrations)
• Lead capacity planning, index design, data retention strategies, and SmartStore lifecycle management
• Maintain high availability, scaling, and resilience across multi-site deployments (including DR strategy)
• Drive Splunk version upgrades, app updates, cluster maintenance, and platform hardening
• Collaborate with SOC, Incident Response, and Threat Hunting teams to ensure high-quality security log ingestion
• Onboard and normalize logs from firewalls, EDR, identity platforms, cloud providers, network telemetry, and custom applications
• Develop and optimize detection content: correlation searches, risk-based alerting, data models, macros, lookups, summaries
• Ensure compliance with logging standards (MITRE ATT&CK mapping, CIS/SOC2/ISO27001 logging requirements)
• Build and manage ingestion pipelines, parsing, field extractions, CIM compliance, HEC configurations, and forwarder architecture
• Implement data lifecycle tiers, filtering strategies, routing, and ingestion controls to reduce cost and improve efficiency
• Optimize search performance, knowledge objects, summary indexing, and acceleration strategies
• Establish Splunk development standards, dashboards, and naming conventions
• Mentor junior engineers and act as a technical escalation point for the team
• Maintain documentation, operational runbooks, and logging onboarding guidelines
• Partner with Engineering, Cloud, SecOps, and App teams to drive company-wide observability maturity

Skills

Automation

Influence

Result Orientation

Stakeholder Management

Technical Strategy Development

Application Development

Architecture

Business Acumen

Risk Management

Solution Design

Agile Practices

Analytical Thinking

Collaboration

Data Management

Solution Delivery Process

Tools

Indexer clustering

SmartStore / S3-compatible object store

Universal / heavy forwarder

Ingest actions / parsing

KVStore / RBAC / SAML / encryption

Job description

Bank of America seeks a Principal Splunk Engineer to lead the design, operation, and evolution of a large-scale Splunk Enterprise / Splunk Cloud deployment. The platform ingests multi-terabyte daily data across security, infrastructure, and applications and is a core component of our SOC and threat-detection capabilities.

The ideal candidate will drive capacity planning, upgrades, and platform hardening, while collaborating with SOC, IT, and App teams to enable high-quality security logs,