Resident Solutions Architect (Databricks)

Job Title: Resident Solutions Architect (Databricks)

Location: Fully Remote (in the US) *Some travel depending on end client needs/projects

Number of openings: Many

Experience: 10+ years

Salary: $80-90/hr W2 or $95-105 C2C

Start Date: Open roles, 2-4 week process (faster for strong candidates!

Job Description:

Core Responsibilities
Some of the major responsibilities for an RSA are:

• Engage with customers on short- to mid-term professional services projects using the Databricks platform (data engineering, data science, cloud).

• Design and build reference architectures, help with production-level use cases and deployments of big data / AI applications.

• Collaborate with engagement/project managers, customer teams, and internal engineering/support teams to ensure technical delivery meets customer needs

• Provide escalated technical support or operational assistance for customer engagements, helping to resolve issues or unblock customers

• Work hands-on: write code (e.g., Python, Scala), work with distributed compute (Apache Spark), integrate across cloud ecosystems.

• Possibly scope new engagements, support sales or pre-sales (depending on region/team) by estimating effort, defining deliverables.

• Provide feedback to product/engineering from customer engagements (help improve platform, features

Required Skills & Qualifications
an RSA typically needs:

• Several years (often 8+ years) in data engineering, analytics, platform roles.

• Strong coding ability in Python or Scala, and comfortable with distributed processing (Spark).

• Experience working across at least one cloud provider (AWS, Azure, GCP) and preferably familiarity with multiple.

• Knowledge of production deployments: CI/CD, MLOps, data architectures.

• Good client-facing / consulting / customer engagement skills (communicating technical ideas, scoping projects, working with customer teams).

• What You Should Expect from the Role

• It's billable professional services: you'll often be working as a consultant/embedded engineer with customers, delivering real implementations rather than purely strategy.

• The role is technically deep: you'll not only design solutions but often build them, debug issues, optimize performance.

Because it is delivery-oriented, you may deal with customer escalations and operational issues, not just green-field new builds.

• You'll need adaptability: customers across industries, various workloads (ETL, streaming, AI/ML), different cloud platforms.

• The role also helps the customer adopt Databricks, so you're enabling change in how they think about data/AI, not just delivering code

• Role of an RSA

  • Be a trusted advisor with authority and clarity.

  • Set expectations and simplify complex ideas.

  • Earn trust early, even before delivering technical work.

  • Align with the customer, listen actively, and empathize with their concerns.

  • Always under-promise and over-deliver.

• Strategic Nature of PS Consulting

  • Focus on solving problems to build trust and unlock future work.

  • Use clear, jargon-free communication.

  • Identify stakeholders, understand drivers of change, and mitigate risks.

  • Offer quick-win alternatives to free up customer bandwidth for strategic tasks.

• Build vs. Buy Mitigation

  • Assess if building aligns with the company's core competencies or adds technical debt.

  • Ask: Does owning this make you money, or is it a cost center?

  • Highlight Databricks' value proposition in scalability and maintainability.

  • Empathize—custom solutions are often someone's baby.

  • Offer bake-offs to demonstrate Databricks' strengths.

  • Position Databricks as a career-enabler over maintaining fragile systems.

• Spark Performance & Business Disruption Concerns

  • Educate the customer—understand the specifics: data size, current framework, future scaling needs.

  • Clarify if Spark is truly the bottleneck.

  • Offer bake-offs on speed, performance, and cost.

  • Build a clear, phased migration plan with deliverables.

  • Coordinate with the account team to align messaging and reinforce trust.

• UC Feature Gaps & Overpromising

  • Determine if missing features are true blockers.

  • Educate the account team on PUPR (Public Preview) vs. GA (General Availability).

  • If necessary, build lightweight custom solutions to bridge gaps.

  • Re-align customer expectations with reality and roadmap.

7+ years experience in data eng, data platforms & analytics,

10+ years of consulting experience

Completed Data Engineering Professional certification & required classes

Minimum 6-8+ projects delivered with hands-on experience in development on databricks

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with deep expertise in at least one

Deep experience with distributed computing with Spark with knowledge of Spark runtime internals

Familiarity with CI/CD for production deployments

Working knowledge of MLOps

Current knowledge across the breadth of Databricks product and platform features

Familiarity with optimizations for performance and scalability