Senior Databricks Data Engineer

capitalfarmcredit; College Station (TX); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Incentive program

Accrued time off

401(k) with employer match

Health coverage

Parental leave

Life & disability insurance

Tuition reimbursement

Wellness stipend

Job summary

Capital Farm Credit seeks a Senior Databricks Data Engineer to design, develop, and support scalable cloud-based data solutions enabling analytics, BI, regulatory reporting, and AI initiatives. You will lead a team of data engineers, drive architecture choices, and ensure secure, governed data products across the enterprise.

You will guide design, review code, mentor teammates, and work closely with data, analytics, security, and AI teams to implement Lakehouse patterns using Azure, Databricks,

Qualifications

• Bachelor's degree or equivalent in a related field is required.
• 5+ years hands-on data engineering with Azure/Databricks or similar cloud platforms.
• Experience leading data engineering teams or mentoring engineers.

Responsibilities

• Provide day-to-day technical leadership to a team of data engineers.
• Lead design sessions and establish standards for ingestion, transformation, governance, deployment, observability, and data consumption.
• Own quality through design/code reviews and release practices.
• Partner with cross-functional teams to architect enterprise data solutions on Azure/Databricks.
• Design and maintain batch and real-time pipelines using Databricks.

Skills

Databricks knowledge

Cloud data governance

Mentoring/team leadership

Data modeling

Python scripting

Education

Bachelor's degree in CS/IS/Engineering or related field

Databricks certification (Associate)

Tools

Databricks

Apache Spark

PySpark

Delta Lake

Unity Catalog

Azure Data Lake Storage

Azure SQL

Key Vault

ETL/ELT tools

SQL tools

Job description

About Us

Capital Farm Credit is the largest rural lending cooperative in Texas, serving 192 counties through nearly 70 credit offices. With over $12 billion in assets and more than 600 team members, we provide essential financial services to farmers, ranchers, rural homeowners, and agribusinesses. As part of the nationwide Farm Credit System, we are dedicated to supporting rural communities and agriculture.

Why Join Us?

We seek motivated individuals who share our core values: commitment, trust, value, and family-like respect. As a customer-owned cooperative, we align employee success with member success, offering competitive pay, growth opportunities, and a supportive environment.

Our Benefits:

• Incentive Program: Company-wide, goals-based rewards.
• Accrued Time Off: Earn 13 days of annual leave and 15 days of sick leave per year, plus enjoy 10-12 paid holidays annually.
• Retirement: 401(k) with up to 9% employer contribution/match.
• Health Coverage: Affordable medical, dental, and vision plans.
• Parental Leave: 8 weeks of paid parental leave.
• Life & Disability Insurance: Employer-paid coverage.
• Education & Wellness: Tuition reimbursement and up to $400 for wellness expenses.

At Capital Farm Credit, you'll find more than a job-you'll find purpose.

EDUCATION AND EXPERIENCE

Bachelor's degree in computer science, Information Systems, Engineering, Data Analytics, or a related field, or an equivalent combination of education and experience.

Five (5) or more years of hands-on data engineering experience using Azure, Databricks, or comparable cloud data platforms, including two (2) or more years serving as a senior engineer, technical lead, or mentor on data engineering teams or projects.

Demonstrated experience in Databricks, Spark, PySpark, Delta Lake, Unity Catalog, SQL, Python, notebooks, jobs, clusters, and Lakehouse architecture is required.

Experience with Azure Data Lake Storage (ADLS), Azure SQL, Key Vault, managed identities, ETL/ELT processes, dimensional modeling, and cloud security fundamentals is required.

Experience designing data solutions that support analytics, machine learning, artificial intelligence, governed enterprise reporting, and secure data consumption use cases is also required, including knowledge of AI/ML data preparation practices such as feature engineering, model-ready data design, data quality, lineage, and secure access to sensitive data.

Databricks Certified Data Engineer Associate certification or an equivalent current Databricks data engineering certification is required;

Databricks Certified Data Engineer Professional certification is preferred.

JOB SUMMARY

The Senior Databricks Data Engineer designs, develops, and supports scalable, secure, and governed cloud-based data solutions that enable enterprise analytics, business intelligence, regulatory reporting, artificial intelligence initiatives, and data products. This role serves as a senior technical resource for the enterprise data platform and provides day-to-day technical leadership to a team of data engineers, including work planning and assignment, design and code review, mentoring, and escalation support for complex production issues.

In addition to building, maintaining, and optimizing production data pipelines, this position leads the design and development of the organization's enterprise data platform using Azure and Databricks. The position applies and establishes modern Lakehouse engineering practices and technologies, including Databricks, Apache Spark, PySpark, Delta Lake, Unity Catalog, Azure Data Lake Storage, SQL, and Python, with an emphasis on data governance, performance, automation, security, and production reliability. The Senior Databricks Data Engineer also drives data solution architecture by establishing and applying Lakehouse and Medallion architecture patterns, developing governed and AI-ready data assets, creating reusable engineering frameworks, and supporting secure and consistent data consumption across the organization.

ESSENTIAL FUNCTIONS

• Provide day-to-day technical leadership and direction to a team of data engineers, including assigning and prioritizing work, estimating effort, tracking delivery, and removing technical obstacles in partnership with the manager.
• Mentor and develop engineers through design guidance, pair programming, knowledge sharing, and structured onboarding of new team members, contractors, and partners.
• Lead technical design sessions and establish engineering standards, patterns, and best practices for ingestion, transformation, governance, deployment, observability, and data consumption.
• Own technical quality across the team by conducting design and code reviews and enforcing testing, documentation, and release standards.
• Partner with data, analytics, application, infrastructure, security, and AI teams to architect the overall enterprise data solution using Azure and Databricks.
• Design and maintain batch and near-real-time pipelines using Databr