且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

Subprocess.Popen 在解释器、可执行脚本中的行为不同

更新时间:2023-12-04 22:50:58

我写了一个小测试脚本来测试 subprocess 模块.

#!/bin/bashecho echo to stderr 1>&2回显到标准输出

然后我写了一个小的 Python 脚本来调用它:

#!/usr/bin/python导入子流程命令 = ('./joe.sh',)任务 = subprocess.Popen(command, stdout=subprocess.PIPE,stderr=subprocess.PIPE)标准输出,标准错误 = task.communicate()打印 'stdout == %r\nstderr == %r' % (stdout, stderr)

运行结果如下:

$ python joe.pystdout == 'echo to stdout\n'stderr == 'echo to stderr\n'

ipython 中运行相同序列的输出是相同的.

所以 subprocess 模块的行为方式符合您的预期,而不是您的问题中的行为方式.我认为除了 subprocess 模块之外的其他东西一定有问题,因为你所做的对我有用.

我正在运行 Python 2.7,所以另一种可能是 subprocess 模块的旧版本中可能存在某种奇怪的错误.

Let's say you have the following:

command = shlex.split("mcf -o -q -e -w %s %s" % (SOLFILE, NETFILE))
task = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = task.communicate()
print "stdout: %s" % stdout #debugging
print "stderr: %s" % stderr #debugging
if stderr:
    sys.exit("MCF crashed on %s" % NETFILE)

It's not necessary to know what mcf is, except that it's a C program which will overflow if it's not given a satisfiable netfile. (Why can't I just ensure that all the netfiles are satisfiable? Well, because the easiest way to check that is to feed it to mcf and see if it overflows...)

Anyway, when I run this in an executable script, task.communicate() doesn't seem to store anything in stdout and stderr. (To be precise, I get stdout == stderr == ''.) Instead, the stderr stream from mcf seems to be "leaking" to the terminal rather than getting captured by the subprocess pipe. Here's some sample output to illustrate:

Netfile: facility3cat_nat5000_wholesaler_capacitation_test_.net
Solfile: facility3cat_nat5000_wholesaler_capacitation_test_.sol
*** buffer overflow detected ***: mcf terminated
======= Backtrace: =========
...
...[fifty lines of Linda Blair-esque output]...
...
stdout: None
stderr: 
...[program continues, since stderr did not evaluate to True]...

This only fails when running the script from the command line. When I step through it line by line in the interpreter, stdout and stderr are correctly assigned:

>>> task = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> stdout, stderr = task.communicate()
>>> stderr
'*** buffer overflow detected ***: mcf terminated\n======= Backtrace: =========\n'
...[more headspinning and vomit]...

Could anyone help me to understand why this works in the interpreter, but not when executed? Thanks in advance!

I wrote a little test script to test the subprocess module with.

#!/bin/bash

echo echo to stderr 1>&2
echo echo to stdout

Then I wrote a small Python script that calls it:

#!/usr/bin/python

import subprocess

command = ('./joe.sh',)
task = subprocess.Popen(command, stdout=subprocess.PIPE,
                                 stderr=subprocess.PIPE)
stdout, stderr = task.communicate()
print 'stdout == %r\nstderr == %r' % (stdout, stderr)

The output of running it looks just like this:

$ python joe.py 
stdout == 'echo to stdout\n'
stderr == 'echo to stderr\n'

The output of running that same sequence in ipython is the same.

So the subprocess module is behaving in the manner you expect, and not how it's behaving for you in your question. I think something other than the subprocess module must be at fault here because what you're doing works for me.

I'm running Python 2.7, so another possibility is that maybe there is some kind of weird bug in older versions of the subprocess module.