If you’re managing Oracle Cloud Infrastructure (OCI), you might have encountered a frustrating issue: your OCI Load Balancer backend health check fails even though your security lists, route tables, and firewall settings are correct.
In this blog, we’ll break down why this happens and provide a step-by-step solution.
Understanding the OCI Load Balancer Health Check
OCI Load Balancers periodically check the health of backend servers by sending requests to a specific port and protocol (TCP or HTTP). If the response is not as expected, the backend is marked unhealthy, and traffic is not routed to it.
Common symptoms of failing health checks:
-
Status:
Connection failed -
Status:
Status code mismatch -
Backend remains
Criticalin the OCI console
Even if all your network rules are correct, the LB may still mark the backend as unhealthy due to application-level issues.
Case Study: Health Check Failing Despite Correct Security Settings
Here’s an example scenario:
-
Backend VM private IP: **.**.**.**
-
OCI Load Balancer health check node IP: **.**.**.**
-
Security lists and NSGs are correctly configured to allow traffic from the LB subnet
-
Firewall on the VM is disabled
Yet, the LB health check reports:
Critical – Connection failed
Step 1: Check if the backend application is listening
Run:
ss -lntp | grep :80
-
If nothing is listening on the configured port, the health check will fail
-
In our case, starting Apache (
httpd) fixed the “Connection failed” issue:
sudo systemctl start httpd
sudo systemctl enable httpd
Step 2: Check the HTTP response code
After starting the web server, the health check may still fail with:
Status code mismatch
Run a test from the backend VM:
curl -i http://10.24.139.43/
-
In our example, the response was:
HTTP/1.1 403 Forbidden
-
OCI expects HTTP 200 OK by default. A 403 indicates Apache cannot serve the requested page, even if the server is running.
Step 3: Fix Apache configuration and permissions
-
Ensure Apache allows access to the DocumentRoot:
<Directory "/var/www/html"> Options Indexes FollowSymLinks AllowOverride None Require all granted </Directory>
-
Create a simple index file to return HTTP 200:
sudo bash -c 'echo "<html><body><h1>OK</h1></body></html>" > /var/www/html/index.html'
sudo chmod 644 /var/www/html/index.html
sudo chown apache:apache /var/www/html/index.html
sudo systemctl restart httpd
-
Test again:
curl -i http://10.24.139.43/
Output should be:
HTTP/1.1 200 OK
<html><body><h1>OK</h1></body></html>
Step 4: Verify LB health check
-
OCI Load Balancer will now mark the backend Healthy within 30–60 seconds
-
Traffic through the LB will work correctly for end users
Key Takeaways
-
Security rules alone do not guarantee a healthy backend. Always check the application layer.
-
403 Forbidden or 404 Not Found responses will cause health check failures.
-
Ensure the backend serves HTTP 200 OK on the health check path.
-
Always test using
curlorncto simulate LB requests.
Conclusion
If your OCI Load Balancer health check is failing despite correct network settings, don’t panic. Most likely, the issue is at the application level — either the server is not listening, or the HTTP response is not 200 OK.
By ensuring your backend web server is running, the correct permissions are set, and a valid index page is served, your LB will pass the health checks, and traffic will flow smoothly.
