6-2 - Client Timeout

Timeout occurs when the caller fails to obtain the corresponding response within the specified time after the request is sent.

Possible Reason

  1. The server is really slow in processing and cannot return the result within the specified time. The caller will automatically return a timeout exception response to end the call.
  2. If the server responds quickly, but when the client load is high and the load pressure is high, it will time out due to problems such as the client request not being sent out or the response stuck in the TCP Buffer. Because the client receives the data sent by the server or requests the data from the server, it will be queued at the system level. If the system load is relatively high, the time spent in the kernel state will increase, causing the client to time out when it gets the value. .
  3. Usually the business processing is too slow, and it can be executed on the service provider machine: jstack [PID] > jstack.log to analyze which method calls the threads are stuck on, which is the reason for the slowness. If performance tuning is not an option, increase the timeout threshold.

Troubleshooting and resolution steps

  1. There may be GC on both sides. Check the GC logs of the server and client. A GC that takes a long time will cause a timeout. The occurrence of a timeout probably means that the resource (CPU, memory, or network) of the caller or server has a bottleneck. It is necessary to check whether the problem is on the server or the caller to rule out suspicions such as GC jitter.
  2. Check the network quality of the server, such as the retransmission rate to rule out network suspicions.
  3. With the help of link tracking analysis services (such as Ali’s ARMS, the open source [OpenTracing](https://github.com/opentracing/ opentracing-java) System implementation Zipkin, SkyWalking etc.) to analyze the time-consuming situation of each point.

Acknowledgments

The FAQ page of this error code refers to “Dubbo Common Errors and Solutions”.

Articles cited are compiled under license CC-BY-4.0. Thanks to the original author here.