Home/Knowledge Hub/Clustering & high availability
Clustering & high availability
Liferay DXP 7.4 Clustering
High availability, load balancing, shared resources, search, sessions, caching and failover
| 45 Backup is Not High Availability 23 |
|---|
| 46 High Availability vs Disaster Recovery 23 |
| 47 RPO and RTO 23 |
| 47.1 Recovery Point Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 |
| 47.2 Recovery Time Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 |
| 48 Monitoring 24 |
| 49 Important JVM Metrics 24 |
| 50 Database Monitoring 25 |
| 51 Troubleshooting: Node Not Joining Cluster 25 |
| 52 Troubleshooting: Random User Logout 25 |
| 53 Troubleshooting: File Exists on One Node Only 25 |
| 54 Troubleshooting: Different Behavior Per Request 26 |
| 55 Troubleshooting: Stale Data 26 |
| 56 Common Anti-Patterns 26 |
| 57 Recommended Checklist 27 |
| 58 Interview Questions 27 |
| 59 Multiple Choice Questions 28 |
| 60 Quick Revision Table 32 |
| 61 Architecture Summary 32 |
| 62 Key Takeaway 33 |
| 63 Version Considerations 33 |
| 64 Official Reference 33 |
Introduction
A basic Liferay installation commonly contains one application node:
User
|
v
Liferay
|
+--> Database
|
+--> Document Store
|
+--> Search EngineThis architecture is straightforward, but the Liferay application server itself can become a single point of failure. If the server fails:
For business-critical applications, this may be unacceptable. A cluster introduces multiple Liferay application nodes.
Users
|
v
+---------------+
| Load Balancer |
+-------+-------+
|
+-------+-------+
| |
v v
Liferay 1 Liferay 2If one node becomes unavailable, traffic can be directed to another.
What is High Availability?
High Availability (HA) means designing a system to remain available despite component failures. Availability can be expressed as:
For example:
still permits approximately:
of downtime. Higher availability targets require progressively stronger redundancy and operational practices.
Why Cluster Liferay?
Common objectives include:
- High availability
- Fault tolerance
- Horizontal application scaling
- Maintenance with reduced downtime
- Distribution of user traffic
- Better resilience to node failures
A cluster can therefore support:
Basic Two-Node Architecture
A simplified two-node topology is:
Internet
|
v
+---------------+
| Load Balancer |
+-------+-------+
|
+----------+----------+
| |
v v
+--------------+ +--------------+
| Liferay | | Liferay |
| Node 1 | | Node 2 |
+------+-------+ +------+-------+
| |
+----------+----------+
|
+-------------+-------------+
| | |
v v v
Database Document Store SearchThe nodes should see the same logical application data.
Main Components
A Liferay cluster generally involves several layers:
- Load Balancer
- Multiple Liferay Application Nodes
- Shared/Highly Available Database
- Shared Document Storage
- Search Infrastructure
- Cluster Communication
- Session Strategy
- Cache Strategy
- Consistent Deployments
- Monitoring and Health Checks
A useful formula is:
where:
Load Balancer
The load balancer is the entry point for incoming requests.
Users
|
v
Load Balancer
/ \
/ \
v v
Node 1 Node 2Its responsibilities can include:
- Traffic distribution
- Health checks
- Removing failed nodes
- Reintroducing recovered nodes
- TLS termination
- Sticky-session handling
Common technologies include:
- NGINX
- HAProxy
- Apache HTTP Server
- Cloud load balancers
- Hardware load balancers
Load Balancing Algorithms
Common algorithms include:
Round Robin
Least Connections
Send new traffic to the node with the fewest active connections.
N ∗ = arg min Connections(N ) N
Weighted Distribution
If Node 1 is more powerful:
then Node 1 can receive approximately twice as much traffic.
Health Checks
A load balancer should not blindly send traffic to every configured node. Instead:
Load Balancer
|
+--> Check Node 1 -> Healthy
|
+--> Check Node 2 -> Failed
|
v
Remove from PoolWhen Node 2 recovers:
Node 2 Healthy
|
v
Return to PoolLiferay Application Nodes
All application nodes should normally run compatible application versions and deployments. For example:
| Node | Liferay Version | Application Version |
|---|---|---|
| Node 1 | 7.4.x | Release 25 |
| Node 2 | 7.4.x | Release 25 |
| Node 3 | 7.4.x | Release 25 |
Running inconsistent application artifacts can lead to unpredictable behavior.
Shared Database
All Liferay nodes normally access the same logical database.
Liferay Node 1 -----+
|
+----> Liferay Database
|
Liferay Node 2 -----+This means:
From the application’s perspective, portal entities such as:
- Users
- Sites
- Roles
- Objects
- Web Content
- Workflow data
- Permissions
must represent the same shared portal state.
Database High Availability
A common mistake is:
2 Liferay Nodes
|
v
1 Database ServerThe Liferay tier is redundant, but the database remains a single point of failure. A stronger architecture is:
Liferay Cluster
|
v
Database HA Layer
|
+---+---+
| |
v v
DB 1 DB 2The exact implementation depends on the database platform.
Connection Pool Considerations
Suppose each Liferay node has a maximum database connection pool of:
For:
nodes, the potential application connection demand is:
Therefore, scaling application nodes without considering database capacity can overload the database.
Document Library Storage
Documents must also be available consistently to all Liferay nodes. Bad design:
Node 1
|
+--> Local Documents A
Node 2
|
+--> Local Documents BSuppose a user uploads a file through Node 1. Later:
If Node 2 cannot access the same document repository, the file may appear unavailable. Therefore:
Shared Document Store
Conceptually:
Node 1 -----+
|
+----> Shared Document Store
|
Node 2 -----+The actual implementation depends on the supported document library store and deployment architecture. The critical requirement is consistent access.
Search Infrastructure
Liferay relies heavily on search infrastructure. Typical search responsibilities include:
- Content indexing
- Search
- Asset discovery
- User search
- Object search
- Faceting
A production architecture should avoid treating an embedded or single-node search engine as a complete HA strategy. Conceptually:
Liferay Cluster
|
v
Search Infrastructure
/ \
v v
Search Node 1 Search Node 2Search as an Independent Layer
It is useful to think of the system as:
The Liferay application tier can scale independently from the search tier. For example:
does not imply:
Search topology should be designed according to search workload, resilience requirements, and the supported search platform.
Cluster Communication
Liferay nodes may need to communicate information associated with clustered operation. Conceptually:
Node 1 <-------> Node 2
^ ^
| |
+-------> Node 3Cluster communication can be relevant for areas such as distributed coordination and cacherelated notifications.
Cluster Link
Liferay provides cluster-link capabilities for communication among cluster nodes where required by the platform. Conceptually:
The exact configuration depends on:
- Liferay version
- Network topology
- Deployment environment
- Multicast/unicast availability
- Firewall rules
Caching in a Cluster
Caching improves performance but introduces consistency concerns. Suppose:
while:
updates User 101. Node 1 must not indefinitely continue serving stale data. Therefore, clustered applications require appropriate cache invalidation/coordination behavior. Conceptually:
Node 2
|
| Update Entity
v
Database
|
+--> Cache Invalidation / Coordination
|
v
Node 1Do Not Use JVM Memory as Shared State
Consider:
public static Map<String, Object> data =
new HashMap<>();In a cluster:
Each JVM has its own memory. Therefore, JVM-local static variables should not be treated as shared cluster state. This is especially important for:
- Counters
- User state
- Workflow assignment state
- Locks
- Business caches
- Temporary application state
Sessions
HTTP sessions require careful consideration in a cluster. Suppose:
Request 1 -> Node 1
Request 2 -> Node 2If the application depends on Node 1’s local session state, Request 2 may not see that state. There are several strategies.
Sticky Sessions
Sticky sessions attempt to keep a user’s requests on the same node.
User A
|
+--> Node 1
+--> Node 1
+--> Node 1
User B
|
+--> Node 2
+--> Node 2
+--> Node 2This is also called:
Advantages:
- Simple
- Reduces cross-node session complexity
- Can improve performance
Limitation: If Node 1 fails, its JVM-local session state may be lost unless another session persistence/replication strategy exists.
Session Replication
Application servers can support session replication. Conceptually:
Node 1
|
| Session Update
v
Node 2This can improve failover behavior, but replication has a cost:
Therefore, applications should avoid storing unnecessarily large objects in HTTP sessions.
Stateless Design
Where practical, stateless application design simplifies horizontal scaling. Instead of:
prefer appropriate external/shared state mechanisms. For example:
Statelessness reduces dependency on a particular application node.
Sticky Sessions vs Stateless Design
| Sticky Session | Stateless | |
|---|---|---|
| Node affinity | Yes | No |
| Failover complexity | Higher | Lower |
| Application design | Simpler initially | More scalable |
| Horizontal scaling | Good | Excellent |
| Node independence | Lower | Higher |
Consistent Configuration
All nodes should operate with compatible Liferay configuration. For example:
Node 1
portal-ext.properties
OSGi configuration
Client Extensions
Node 2
portal-ext.properties
OSGi configuration
Client ExtensionsConfiguration drift can cause difficult production failures. A preferred model is:
with version-controlled deployment.
Consistent OSGi Deployments
Suppose:
Node 1 -> qap-service 1.4
Node 2 -> qap-service 1.3The load balancer may send requests to either node. The user may therefore observe different behavior depending on which node receives the request. This should be avoided. A cluster should normally satisfy:
Client Extensions and Clustering
Client Extensions can reduce the amount of custom code deployed inside each Liferay node. For example:
Load Balancer
|
+---------+---------+
| |
v v
Liferay 1 Liferay 2
| |
+---------+---------+
|
v
External FastAPI
ServiceInstead of deploying specialized Python/AI/quantum logic to each Liferay node, it can run as an independently scalable service.
Microservices and HA
External microservices themselves must also be highly available if they are critical. Bad architecture:
2 Liferay Nodes
|
v
1 Critical MicroserviceThe microservice becomes the new single point of failure. Better:
Liferay Cluster
|
v
Service Load Balancer
|
+--+--+
| |
v v
MS1 MS2HA must be considered end-to-end.
Failure Scenario: Liferay Node Failure
Initial state:
LB
|
+--> Node 1
|
+--> Node 2Node 1 fails:
LB
|
+--> Node 1 [DOWN]
|
+--> Node 2 [UP]The load balancer detects the failure and removes Node 1:
LB
|
+--> Node 2Users continue through Node 2.
Failure Scenario: Database Failure
Suppose:
Node 1 ---+
|
+--> DB [DOWN]
|
Node 2 ---+Both nodes may become effectively unavailable. This demonstrates:
The database must have an appropriate HA strategy as well.
Failure Scenario: Search Failure
If the search service becomes unavailable, search-dependent functionality can fail or degrade even if Liferay application nodes remain alive. Therefore, monitor:
- Search cluster health
- Index status
- Search latency
- Disk usage
- JVM memory
Failure Scenario: Document Store Failure
If the document repository is unavailable:
but:
High availability must therefore include storage services.
Single Points of Failure
For every architecture component, ask:
Potential single points include:
- Load balancer
- Database
- Search engine
- Document storage
- DNS
- Network gateway
- Authentication provider
- Critical microservice
- Shared filesystem
Redundant Load Balancer
Even the load balancer can become a single point of failure. Instead of:
Internet
|
v
LB 1
|
Liferay Clustera stronger design is:
Internet
|
+------+------+
| |
v v
LB1 LB2
| |
+------+------+
|
+---------+---------+
| |
v v
Liferay 1 Liferay 2The exact mechanism can involve cloud-native or network-level high-availability services.
Production Architecture
A more complete architecture is:
Users
|
v
DNS / CDN
|
v
+----------------+
| Load Balancer |
+--------+-------+
|
+-----------+-----------+
| |
v v
+-------------+ +-------------+
| Liferay N1 | | Liferay N2 |
+------+------+ +------+------+
| |
+-----------+-----------+
|
+-----------------+-----------------+
| | |
v v v
Database HA Search Cluster Document Store
|
v
Backup / Replica
External services:
Liferay Cluster
|
v
API Gateway / LB
|
+---+---+
| |
v v
Service1 Service2Network Considerations
Cluster nodes need reliable network communication. Check:
- Firewall rules
- Port availability
- DNS resolution
- Network latency
- Packet loss
- Load balancer timeouts
- Proxy headers
- TLS configuration
A cluster can fail even when every server is individually healthy if network communication is unreliable.
Reverse Proxy Configuration
When a load balancer terminates HTTPS:
Browser
|
| HTTPS
v
Load Balancer
|
| HTTP / HTTPS
v
LiferayLiferay must correctly understand the original external protocol, host, and related proxy information. Otherwise issues can occur with:
- Redirect URLs
- OAuth callbacks
- OpenID Connect
- Generated links
- Secure cookies
Clustering and OpenID Connect
Suppose the portal uses an external OpenID Provider. Architecture:
OpenID Provider
^
|
| OIDC
|
User --> Load Balancer --> Liferay ClusterThe identity provider should normally interact with the stable public portal URL:
https://portal.example.comrather than individual node URLs. Thus:
Clustering and Workflow
Workflow processing may execute on any eligible application node. Therefore, custom workflow logic should not rely on JVM-local state. Bad example:
static int roundRobinCounter = 0;because:
For round-robin workflow assignment, use shared persistent state and appropriate concurrency control.
Clustering and Liferay Objects
Liferay Object data is stored in the shared application data layer. Therefore:
Request
|
+--> Node 1 --> Object Entry
|
+--> Node 2 --> Same Logical Object DataExternal consumers should normally call the public load-balanced Liferay endpoint:
https://portal.example.com/o/c/orders/rather than:
http://node1:8080/o/c/orders/This preserves failover and load balancing.
Clustering and Headless APIs
Headless API architecture:
React / FastAPI / Mobile
|
| HTTPS
v
Load Balancer
/ \
v v
Liferay 1 Liferay 2
\ /
\ /
Shared DataClients should remain unaware of which application node handles each request.
Rolling Deployments
A cluster can support reduced-downtime deployment patterns. Conceptually:
- Remove Node 1 from the load balancer.
- Deploy/test Node 1.
- Return Node 1 to the pool.
- Remove Node 2.
- Deploy/test Node 2.
- Return Node 2.
Architecture:
Step 1:
LB -> Node 2
Node 1 upgrading
Step 2:
LB -> Node 1
Node 2 upgrading
Step 3:
LB -> Node 1 + Node 2Blue-Green Deployment
Another strategy is blue-green deployment.
Load Balancer
|
+-------+-------+
| |
v v
BLUE GREEN
Current New
Version VersionAfter validating Green:
This can simplify rollback for compatible deployment scenarios.
Backup is Not High Availability
A backup provides recovery after failure. High availability aims to continue service during failure. Therefore:
but both are necessary. A good architecture needs:
High Availability vs Disaster Recovery
High availability usually protects against local component failures. Disaster Recovery (DR) addresses larger events such as:
- Data center failure
- Region outage
- Major corruption
- Catastrophic infrastructure loss
Conceptually:
Primary Region
|
| Replication / Backup
v
DR RegionRPO and RTO
Two important DR metrics are:
Recovery Point Objective
RPO asks:
Example:
Recovery Time Objective
RTO asks:
Example:
Monitoring
Monitor the complete system rather than only Liferay. Important metrics include:
- Node availability
- CPU
- JVM heap
- Garbage collection
- Thread pools
- Database connections
- Request latency
- HTTP error rate
- Search health
- Search latency
- Document storage availability
- Disk space
- Network latency
Important JVM Metrics
Monitor:
For example:
HeapUsed HeapUsage = × 100 HeapMax
× HeapMax
Sustained high heap utilization may indicate:
- Insufficient heap
- Memory leak
- Excessive caching
- Large sessions
- Application defects
Database Monitoring
Important database metrics include:
- Active connections
- Connection wait time
- Slow queries
- CPU
- Memory
- Disk I/O
- Lock contention
- Replication health
If the database is overloaded, adding more Liferay nodes can actually make the problem worse.
Troubleshooting: Node Not Joining Cluster
Check:
- Cluster configuration
- Network connectivity
- Firewall
- Node addressing
- Logs
- Liferay version consistency
Troubleshooting: Random User Logout
Possible causes include:
- Sticky-session misconfiguration
- Session replication issues
- Cookie configuration
- Proxy configuration
- Node restart
Troubleshooting: File Exists on One Node Only
Likely cause:
instead of a shared storage architecture. Check the Document Library storage configuration.
Troubleshooting: Different Behavior Per Request
If behavior appears random:
Request 1 -> Correct
Request 2 -> Old behavior
Request 3 -> Correct
Request 4 -> Old behaviorsuspect node inconsistency. Check:
- Deployed modules
- Configuration
- Client Extensions
- Liferay versions
- Environment variables
Troubleshooting: Stale Data
Possible areas include:
- Cache invalidation
- Search indexing
- Custom JVM-local caches
- External caches
- Replication delay
Do not immediately assume the database itself contains stale data.
Common Anti-Patterns
Avoid:
- Different application versions on different nodes.
- Local-only document storage.
- JVM static variables as shared state.
- Hard-coding individual node URLs.
- Single-node search for critical HA systems.
- Single database without considering DB availability.
- Unmonitored load balancers.
- Large HTTP sessions.
- Manual configuration drift.
- Assuming two Liferay nodes automatically equal HA.
Recommended Checklist
Before production:
□ Multiple Liferay nodes
□ Load balancer health checks
□ Stable public portal URL
□ Shared/HA database
□ Shared document storage
□ Production search infrastructure
□ Cluster communication verified
□ Session strategy defined
□ Identical application deployments
□ Consistent configuration
□ Monitoring configured
□ Backup tested
□ DR strategy documented
□ Node failure tested
□ Database failure considered
□ Search failure considered
Interview Questions
Multiple Choice Questions
Question 1
What is the main purpose of Liferay clustering?
A. CSS compilation
B. High availability and scalability
C. Creating JSP files
D. Replacing the database
Answer: B
Question 2
What typically distributes traffic among Liferay nodes?
A. Elasticsearch
B. Load Balancer
C. Service Builder
D. Picklist
Answer: B
Question 3
What should cluster nodes normally share logically?
A. JVM heap
B. Liferay database
C. Java static variables
D. Browser cache
Answer: B
Question 4
Why should Document Library storage be shared?
A. To compile themes
B. So every node can access the same documents
C. To replace OAuth
D. To create users
Answer: B
Question 5
What is another name for a sticky session?
A. Session affinity
B. Object relationship
C. Search replica
D. Workflow transition
Answer: A
Question 6
What happens to a Java static variable in a two-node cluster?
A. It is automatically shared
B. Each JVM has its own copy
C. It moves into Elasticsearch
D. It becomes a database column
Answer: B
Question 7
If the only database fails, what is likely to happen?
A. Both Liferay nodes can continue normally
B. The application tier may become unavailable
C. CSS stops compiling only
D. Nothing happens
Answer: B
Question 8
Which component should detect failed Liferay nodes?
A. Load balancer health checking
B. JSP
C. Picklist
D. Object Layout
Answer: A
Question 9
What is configuration drift?
A. Nodes having inconsistent configuration
B. Database backup
C. OAuth token generation
D. Creating Objects
Answer: A
Question 10
Which design is generally preferable for horizontal scaling?
A. Heavy JVM-local state
B. Stateless application design where practical
C. Hard-coded node URLs
D. Local-only documents
Answer: B
Question 11
Which statement is correct?
A. Backup and HA mean exactly the same thing
B. HA eliminates the need for backup
C. Backup and HA solve different problems
D. Clustering eliminates DR requirements
Answer: C
Question 12
What does RTO describe?
A. Acceptable recovery duration
B. Number of Liferay Objects
C. OAuth token size
D. Number of search results
Answer: A
Question 13
What does RPO describe?
A. Acceptable data-loss window
B. JVM heap size
C. Number of load balancers
D. HTTP session size
Answer: A
Question 14
What should external Headless API consumers normally call?
A. An individual internal node URL
B. The stable load-balanced portal endpoint
C. The database directly
D. A random JVM port
Answer: B
Question 15
What is a major danger of adding many application nodes without capacity planning?
A. The database or other shared infrastructure may become overloaded
B. Java stops supporting HTTP
C. Objects cannot have fields
D. Liferay stops supporting users
Answer: A
Quick Revision Table
| Concept | Meaning |
|---|---|
| Cluster | Multiple coordinated Liferay nodes |
| High Availability | Design for continued service during failures |
| Load Balancer | Distributes requests across healthy nodes |
| Health Check | Determines whether a node should receive traffic |
| Sticky Session | Keeps a user/session associated with a node |
| Shared Database | Common logical Liferay data source |
| Shared Document Store | Documents accessible to all nodes |
| Search Cluster | External scalable/resilient search infrastructure |
| Cluster Communication Configuration Drift | Communication/coordination between Liferay nodes Differences between supposedly equivalent nodes |
| Stateless Design | Minimizes dependence on node-local state |
| Rolling Deployment | Upgrade nodes sequentially where compatible |
| Blue-Green | Switch traffic between old and new environments |
| RPO | Maximum acceptable data-loss interval |
| RTO | Maximum acceptable recovery duration |
Architecture Summary
The simplest way to remember Liferay clustering is:
USER
|
v
LOAD BALANCER
/ \
/ \
v v
LIFERAY 1 LIFERAY 2
\ /
\ /
+----+----+
|
+-----------------+----------------+
| | |
v v v
DATABASE DOCUMENTS SEARCH
|
v
BACKUPBut each infrastructure component must itself be evaluated for availability. Therefore:
Instead:
Liferay HA = Redundant App Tier + Reliable Shared State + Failure Detection + Redundant Dependencies
Key Takeaway
A robust Liferay clustering strategy should answer five questions:
- Where does traffic go if a Liferay node fails?
- Where is shared application state stored?
- Can every node access the same documents and data?
- What happens when the database, search engine, or another dependency fails?
- Can a node be replaced without changing the application’s externally visible identity?
The ideal application-node principle is:
in terms of version, configuration, and deployed application artifacts. From the user’s perspective:
The user should not need to know which individual Liferay node is serving the request.
Version Considerations
Liferay 7.4 spans multiple GA and quarterly releases, and clustering configuration has evolved over time. Before applying any cluster configuration, verify:
- Exact Liferay DXP quarterly release or Portal GA
- Supported search engine/version
- Application server behavior
- Document Library store
- Cluster communication requirements
- Session strategy
- Deployment architecture
- Database platform
Avoid copying clustering properties from older Liferay versions without validating them against the documentation for the actual target release.
Official Reference
For current configuration details, consult Liferay’s official documentation on clustering for high availability. The production topology should always be adapted to the exact Liferay release, application server, database, search platform, storage architecture, and infrastructure environment being used.
Prefer to read offline? Download the PDF edition (33 pages).
