Solution Architect · Data Science / Data Engineering
Databricks Solutions Architect USA
work Full-time distance Remote payments $170,000 – $175,000 / yr
The Role
Cogniify ishiring a Databricks Solutions Architect (Databricks Architect) to plan and leadour Databricks lakehouse platforms. You will decide how our data is stored,cleaned, protected and used, and guide the team that builds it.
You will workwith data engineers, product managers and business teams, from the first designto a live system. You will also lead the move from older data warehouses toDatabricks.
Much of ourdata work is in energy and utilities, such as smart meter, grid and outagedata. Experience with this kind of data helps.
This is aremote role based in the United States, with some travel for workshops and keymilestones.
Core skills forthis role include Databricks, Apache Spark, PySpark, Delta Lake, Unity Catalog,Lakeflow, lakehouse architecture, AWS, Azure, Google Cloud (GCP), datamigration, CI/CD, data governance and FinOps.
What You Will Do
- Design lakehouse architectures onAWS, Azure or Google Cloud, using medallion layers, Delta Lake and UnityCatalog.
- Set how data pipelines are builtwith Lakeflow, Apache Spark and Databricks SQL, so every engineer on the teambuilds in the same way.
- Define data governance, securityand access control, along with cloud cost controls (FinOps), so data stays safeand spending stays in check.
- Lead the move from older datawarehouses and ETL tools to Databricks, and make sure the new numbers match theold ones.
- Set CI/CD and DataOps standards,including Git, Databricks Asset Bundles and automated testing, so releases aresafe and repeatable.
- Advise on machine learning andgenerative AI use cases, using tools such as MLflow and Agent Bricks where theymake sense.
- Mentor engineers, runarchitecture reviews, and explain designs and trade-offs to business leadersand other stakeholders in simple words.
- Travel for workshops, planningsessions and key milestones when needed.
What We Are Looking For
- A BS, MS or higher in ComputerScience, Engineering, Information Systems or a related field, or equivalentpractical experience.
- At least 12 years of experiencein data engineering or data architecture, with at least 4 years of hands-onwork on Databricks and Apache Spark using PySpark and SQL.
- At least 3 production Databricksimplementations that you took from design to go-live.
- Strong experience on at least onemajor cloud, AWS, Azure or Google Cloud (GCP).
- Deep knowledge of Delta Lake,Unity Catalog, performance tuning and cluster sizing.
- Strong leadership andcommunication skills, with a record of leading technical teams and working withbusiness stakeholders in large, complex organizations.
- Experience setting CI/CD andDataOps standards, including Git and automated testing.
- Comfort working remotely andtraveling when the work needs it.
- Databricks certifications such asData Engineer Professional, Machine Learning or Generative AI are a plus.
- Databricks architect level orChampion level standing, or equivalent proven experience, is a plus.
- Hands-on work with Lakebridgemigrations, Lakebase or Genie is a plus.
- Experience with streaming andreal-time pipelines, and with data modeling, is a plus.
- Experience with energy andutility data, such as smart meter, grid or outage data, is a plus.
- Experience on large enterprisedata programs is a plus.
Perks and Benefits of WorkingWith Us
- Unlimited PTO
- Generous parental leave, wellabove industry standards
- Medical, dental, and visioncoverage for employees
- Disability and life insurance
- Mental health and wellbeingsupport
- Employee Stock Purchase Plan(ESPP)
- Yearly team-building experiences
- Mentorship and sponsorshipopportunities
- Manager resources and support
- Small teams where your work makesa real difference
- Open communication withmanagement and company leadership
- An entrepreneurial culture wheretaking initiative is part of everyday work
Cogniify is an equal opportunity employer. We celebratediversity and are committed to creating an inclusive environment for allemployees. All qualified applicants will receive consideration for employmentwithout regard to race, color, religion, sex, sexual orientation, genderidentity, national origin, disability, veteran status, or any other protectedcharacteristic.