且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

将Python对象传递给另一个Python进程

更新时间:2023-11-09 23:30:34

是否可以腌制套接字对象并将其通过IPC工作?

Would pickling the socket object and pushing it through IPC work?

不.在该对象内部是内核套接字的文件描述符或句柄.这只是进程在进行系统调用时用来标识套接字的数字.

No. Inside that object is a file descriptor or handle to the kernel socket. It's just a number that the process uses to identify the socket when making system calls.

如果您腌制该Python套接字对象并将其发送给另一个进程,则该进程将使用未打开的套接字的句柄.或更糟糕的是,该句柄可能引用了另一个打开的文件.

If you pickle that Python socket object and send it to another process, that process will be using a handle for a socket it didn't open. Or worse, that handle may refer to a different open file.

(在Linux上)最有效的处理方式如下:

The most efficient way to handle this (on Linux) is like this:

  • 主进程打开侦听套接字(例如TCP端口80)
  • 主进程派生N个孩子,他们全部继承打开套接字
  • 他们都呼叫accept()并阻止,等待新的连接
  • 当新的客户端连接时,内核将选择具有该套接字句柄的进程之一来接受连接;其他人将继续等待
  • Master process opens listening socket (e.g. TCP port 80)
  • Master process forks N children who all inherit that open socket
  • They all call accept() and block, waiting for a new connection
  • When a new client connects, the kernel will select one of the processes with a handle to that socket to accept the connection; the others will continue to wait

这样,您可以让内核处理负载平衡.

This way, you let the kernel handle the load balancing.

如果您不希望出现这种情况,则可以使用 (在UNIX中)将打开的套接字传递给另一个进程.再次,这不仅仅是句柄;内核有效地将打开的套接字复制到进程的打开文件列表中.这种机制称为SCM_RIGHTS,您可以在此处查看示例(在C语言中): http://man7.org/tlpi/code/online/dist/sockets/scm_rights_send.c.html

If you don't want this behavior, there is a way (in UNIX) to pass an open socket to another process. Again, this is more than just the handle; the kernel effectively copies the open socket to your processs's open file list. This mechanism is known as SCM_RIGHTS, and you can see an example (in C) here: http://man7.org/tlpi/code/online/dist/sockets/scm_rights_send.c.html

否则,您的主进程将需要有效地 proxy 与子进程的连接,从而降低系统的效率.

Otherwise, your master process will need to effectively proxy the connection to the child processes, reducing thr efficiency of the system.