Records of several problem solving

一、A large number of “Connect Reset” when tring get access to the Gateway

1.Determine if there is an error in the gateway or in the service that initiated the call

By initiating several calls to cause error logging, while logging the incoming and outgoing records at the caller and the gateway, it is found that there is a probability that the gateway does not receive messages from the caller, while other services call the gateway without missing messages, identifying the error at the service caller

2.Determine the relevant parameters in the resttemplate that control the connection and check

As the service is called via resttemplate, the priority is to consider whether the resttemplate is set incorrectly, causing the connect to be reset frequently. After searching for information, I found that a connection pool parameter was set too large, causing

3.Debug verification and modification

Turned on Debug logging level for resttemplate separately, reproduced the problem, looked up the logs and found that a large number of closed connections were indeed being fetched from the connection pool, so changed the connection pool parameters and the problem was solved

二、Thread leakage troubleshooting

1.Service memory found to be linearly increasing

In this case, the first thing I thought of was a memory leak, but found that the in-heap memory was normal and suspected an out-of-heap memory leak, only to find out that the number of threads had increased abnormally, resulting in a large amount of out-of-heap memory being used

2.Exception thread location

The threads added to the exception were all named “pool-2-thread-1”, which is obviously a thread pool leak. The reason for this is that there is a method that creates a pool of threads without calling it once, but does not release the call, causing the threads to persist