更新时间:2023-11-29 16:43:52
经过更多的调查,并在 google 工程师的帮助下,我找到了两个解释我的内存消耗的原因.
After more investigations, and with the help of a google engineer, I've found two explanation to my memory consumption.
上下文和线程
ndb.Context
是一个线程本地"对象,只有在线程中有新请求时才会清除.所以线程在请求之间保持它.一个 GAE 实例中可能存在许多线程,并且在第二次使用线程并清除上下文之前可能需要数百个请求.
这不是内存泄漏,但内存中的上下文大小可能会超过小型 GAE 实例中的可用内存.
ndb.Context
is a "thread local" object and is only cleared when a new request come in the thread. So the thread hold on it between requests. Many threads may exist in a GAE instance and it may take hundreds of requests before a thread is used a second time and it's context cleared.
This is not a memory leak, but contexts size in memory may exceed the available memory in a small GAE instance.
解决方法:
您无法配置 GAE 实例中使用的线程数.因此,***使每个上下文尽可能小.避免上下文缓存,并在每次请求后清除它.
Workaround:
You can not configure the number of threads used in a GAE instance. So it is best to keep each context smallest possible. Avoid in-context cache, and clear it after each request.
事件队列
似乎 NDB 并不能保证在请求后清空事件队列.同样,这不是内存泄漏.但是它将 Futures
留在您的线程上下文中,您又回到了第一个问题.
It seems that NDB does not guarantee that event queue is emptied after a request. Again this is not a memory leak. But it leave Futures
in your thread context, and you're back to the first problem.
解决方法:
用 @ndb.toplevel
包装所有使用 NDB 的代码.
Workaround:
Wrap all your code that use NDB with @ndb.toplevel
.