REMOTE Java software

SENIOR / STAFF BACKEND ENGINEER – JAVA / SPRING BOOT

About the Role

We are looking for a Senior/Staff Backend Engineer to design, build, and operate highly scalable and resilient backend systems that power distributed job orchestration, data ingestion, and workload scheduling.

You will work extensively with Java 17+, Spring Boot 3, Kafka, MongoDB, OpenSearch/Elasticsearch, and Kubernetes, solving complex problems around distributed systems, asynchronous processing, concurrency, reliability, and large-scale data pipelines.

This role is ideal for an engineer who enjoys taking ownership of challenging backend systems, making sound architectural decisions, and building reliable software that can operate at scale.

Key Responsibilities

1. Architecture & Backend Development

• Design, develop, and operate resilient Spring Boot microservices for distributed job orchestration and data ingestion.
• Build highly available and scalable backend services using Java 17+ and Spring Boot 3.
• Participate in architecture and design reviews, contributing practical solutions to complex distributed-system problems.
• Identify technical risks early and drive improvements in system reliability, maintainability, and scalability.
• Write clear architecture specifications, technical proposals, and engineering documentation.

2. Event Streaming & Distributed Messaging

• Design and operate event-driven architectures using Kafka and other messaging systems.
• Build reliable message-processing pipelines with:
• Idempotency
• Strict ordering
• Retry and recovery mechanisms
• Dead-letter handling
• Backpressure management
• Failure isolation
• Analyze and resolve production issues involving message delivery, consumer lag, concurrency, and distributed processing.
• Ensure messaging systems remain reliable and observable under high workloads.

3. Data Pipelines & Storage

• Design and optimize data ingestion and processing pipelines.
• Work with MongoDB, OpenSearch/Elasticsearch, and other NoSQL/search technologies.
• Optimize indexing throughput, query performance, storage efficiency, and data access patterns.
• Analyze bottlenecks across application, database, and search layers.
• Design data models and processing strategies suitable for high-volume distributed workloads.

4. Distributed Scheduling & Kubernetes

• Design, implement, and scale distributed workload/job schedulers.
• Develop scheduling mechanisms that handle concurrency, retries, workload distribution, and failure recovery.
• Deploy and operate services and workloads on Kubernetes.
• Troubleshoot pod lifecycle issues, resource constraints, networking problems, and workload failures.
• Where appropriate, leverage Kubernetes APIs/client libraries to integrate application-level scheduling with cluster-level orchestration.

5. Concurrency & Asynchronous Processing

• Design concurrent and asynchronous systems using Java concurrency primitives and modern asynchronous execution models.
• Work with thread pools, executors, non-blocking I/O, reactive/asynchronous processing, and task scheduling.
• Identify and mitigate race conditions, deadlocks, thread starvation, resource exhaustion, and other concurrency-related failure modes.
• Build systems that remain predictable and resilient under high levels of parallelism.

6. Engineering Quality, Testing & Observability

• Establish and maintain high engineering standards across design, implementation, testing, and deployment.
• Develop comprehensive unit and integration tests using JUnit 5, Testcontainers, and related tooling.
• Build reliable test strategies for distributed systems and asynchronous workflows.
• Use logs, metrics, and distributed traces to perform root-cause analysis of production incidents.
• Improve system observability and operational readiness.

7. Security & Reliability

• Apply robust security practices throughout application design and implementation.
• Implement appropriate authentication, authorization, input validation, secrets management, and secure service-to-service communication.
• Apply resilience patterns such as circuit breakers, retries, rate limiting, bulkheads, and timeouts where appropriate.
• Continuously identify and address reliability and security risks.

Required Qualifications

Technical Requirements

• 7+ years of professional backend software engineering experience.
• Strong hands-on expertise with Java 17+.
• Extensive experience with Spring Boot 3 and building production-grade microservices.
• Strong understanding of distributed systems, asynchronous processing, and concurrent programming.
• Production experience with message brokers, preferably Apache Kafka. Experience with RabbitMQ, Pulsar, or AWS SQS is also valuable.
• Strong understanding of:
• Thread management
• Thread pools and executors
• Non-blocking I/O
• Asynchronous execution
• Concurrency control
• Failure handling and recovery
• Practical experience with NoSQL/document/search databases such as:
• MongoDB
• OpenSearch
• Elasticsearch
• DynamoDB
• Strong experience with Docker and containerized applications.
• Proficiency with Maven or Gradle, including multi-module project structures.
• Strong automated testing experience with JUnit 5, Testcontainers, and integration testing.
• Experience diagnosing production issues using logs, metrics, and distributed tracing.
• Ability to write clear technical documentation, architecture proposals, and code review feedback.

Nice to Have

• Hands-on operational experience with Kubernetes, including pod lifecycle management, workload debugging, and Kubernetes APIs/client libraries.
• Experience implementing resilience patterns using Resilience4j or similar frameworks.
• Experience building and distributing standalone JVM CLI applications, workers, or background processing binaries.
• Experience with schema-driven development, including:
• OpenAPI
• JSON Schema
• Protobuf
• Avro
• Experience with cloud platforms such as AWS or Azure.
• Experience with CI/CD automation and modern DevOps practices.
• Familiarity with other programming languages such as Go, TypeScript, or Rust.

Professional Attributes

Proactive Ownership

• Take end-to-end ownership of assigned systems and technical initiatives.
• Scope work honestly and communicate assumptions, risks, and dependencies early.
• Proactively escalate technical or delivery risks before they become production problems.

Architectural Clarity

• Build systems that are understandable, maintainable, and extensible.
• Strive to leave codebases, architecture, and documentation in a better state than before.
• Make architectural decisions based on clear trade-offs and long-term maintainability.

Technical Rigor

• Actively clarify ambiguous requirements before implementation.
• Challenge assumptions constructively and contribute meaningfully to design discussions.
• Maintain high standards for code quality, testing, security, reliability, and operational readiness.
• Provide thoughtful and actionable code review feedback.

Technology Stack

Backend: Java 17+, Spring Boot 3, Spring Framework

Messaging: Apache Kafka, RabbitMQ, Pulsar, AWS SQS

Databases & Search: MongoDB, OpenSearch, Elasticsearch, DynamoDB

Infrastructure: Kubernetes, Docker

Build: Maven, Gradle

Testing: JUnit 5, Testcontainers

Resilience: Resilience4j, Circuit Breakers, Rate Limiting

API & Schemas: REST, OpenAPI, JSON Schema, Protobuf, Avro

Cloud & DevOps: AWS/Azure, CI/CD

Observability: Distributed Logs, Metrics, Tracing

What Success Looks Like

In this role, you will be expected to:

• Build and operate reliable distributed backend services at scale.
• Improve the reliability and throughput of event-driven data pipelines.
• Design scheduling and orchestration systems capable of handling concurrent workloads.
• Diagnose and resolve complex production issues efficiently.
• Raise engineering standards through better architecture, testing, observability, and documentation.
• Take ownership of technical problems from design through production operation.