Principal Backend / Performance Engineer

Principal Backend / Performance Engineer

Remote (Pakistan preferred) · Full-time · Reports to CTO/Product owner

🧭 About the role

iClosed.io is a SaaS platform for sales and marketing teams, scaling to millions of users. This role sits in our Performance Team and owns the performance and scalability of our backend and data layer.

We run a polyglot data layer (relational, document, key-value and in-memory) and a backend across multiple modern languages. We believe the biggest gains come from design, not just tuning: matching each store to its access pattern, moving work off the critical path, and drawing service boundaries that scale cleanly. You'll solve the problem in front of you, then shape the architecture so whole classes of problems never come up.

This is a senior individual contributor role with no direct reports, high autonomy and significant technical influence.

🔀 How the work splits

You'll own performance across three connected areas, with genuine depth expected in both system design and data-layer internals:

• Design: how a request flows, what runs synchronously, where state lives, and what happens when a dependency is slow.
• Diagnosis: reading traces, execution plans and metrics to find the real cause rather than the obvious one.
• Prevention: instrumentation, review standards and regression gates, so issues are caught long before they reach customers.

🎯 What you'll do

• Design critical request paths: what's synchronous, where state lives, and how the system degrades when a dependency is slow.
• Diagnose latency across the full request path using traces, execution plans and metrics.
• Match each workload to the right data store, and document the reasoning.
• Scale reads and writes through caching, replicas, partitioning, sharding and queue-buffered writes.
• Build reliable background processing with idempotency, retries, dead letters and backpressure.
• Eliminate N+1 and chatty access, including ORM-generated traffic.
• Own data-layer capacity planning and cost per request alongside latency.
• Set engineering standards: design and query review for hot paths, migration standards, CI regression gates, and load testing at production-realistic volumes.
• Review architecture proposals from feature teams, and write the design docs others build from.

🧑‍💻 What you bring

• 5+ years building and operating production backend systems, with 3+ where performance and scale were a named part of your job.
• Performance ownership of a multi-tenant SaaS product under real load.
• Systems you designed yourself, including the alternatives you rejected and the tradeoffs you accepted.
• A concrete track record: a problem you diagnosed, its root cause, your fix, and the before-and-after numbers.
• Strong system design judgement across caching, sharding, CQRS, queues, real-time delivery (WebSockets/SSE), autoscaling, high availability and resilience patterns. Knowing when not to use one matters more to us than listing its benefits.
• Real depth in at least two of relational, document, key-value and in-memory stores, with working competence in the rest.
• Production ownership in multiple backend languages, and a solid understanding of how ORMs and ODMs generate queries.
• Hands-on APM and tracing experience, plus production experience on a major public cloud, including the cost behaviour of managed data services.
• Clear writing: design docs a team can build from and an executive can follow.

🗄️ Data layer depth

Nobody is deep in all four engines, and we'd rather you tell us where you are. Across your areas of depth, we expect you to reason about:

• Relational: execution plans as routine, composite and partial indexes and their write cost, locking, isolation levels, connection pooling and replica lag.
• Document: embedding vs referencing, shard key selection, aggregation performance and working set sizing.
• Key-value: access-pattern-first modelling, partition keys, hot partition avoidance, secondary indexes and capacity cost.
• In-memory cache: caching patterns, key design, eviction and TTL, and system behaviour when the cache is cold or unavailable.
• Across engines: consistency boundaries, dual-write problems, and recognising when a query problem is really an application design problem.

🌟 What success looks like

• Diagnosis: latency is attributed to real causes with evidence, and per-tenant investigations end with clear answers.
• Design: critical paths have documented designs others can reason about, store selection is deliberate and written down, and queues behave predictably under load.
• Measurable improvement: meaningful latency reduction on the highest-cost paths, with before-and-after numbers, and cost per request trending down.
• Prevention: design and query review is standard practice for hot paths, regressions are caught before merge, and migration standards are followed across teams.
• Partnership: feature teams bring you proposals early because you make them better, not slower.

➕ Nice to have

APM, error tracking and database performance tooling · event streaming or messaging at scale · dedicated search infrastructure · real-time delivery and fan-out at scale · third-party integrations where latency is outside your control · noisy-neighbour and tenant isolation · load testing tools and building that capability · SLOs and error budgets · infrastructure as code · experience as a first dedicated performance hire

🚫 What this role is not

• Not a DBA role. Backups, patching and routine operations sit with DevOps. You set the performance requirements.
• Not feature work. Your output is latency removed and regressions prevented.
• Not a management role today. Leading the function as it grows is an open conversation.

💎 Why join

• Shape the performance function and its priorities from day one.
• Have direct influence on architecture across every engineering team.
• Work on a polyglot stack at real scale.

📋 Practicalities

• Location: Remote (Pakistan preferred)
• Type: Full-time, permanent
• Level: Senior individual contributor, no direct reports
• Compensation: Market competitive
• On-call: Part of the escalation rota