且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

使用python从配置单元查询输出中删除空行

更新时间:2023-12-04 20:58:22

通常你会打开输入文件并写入非空行到第二个文件:

  with open('file.tsv')as infile,open('filtered_file ()):
outfile.write(line)
$ c $ outfile:
for line in infile:
如果line.strip c>

如果要过滤inplace文件,可以使用 FileInput 如果line.strip():
打印行

然而,这使用中间文件,并且可能无法在磁盘空间不足的情况下使用。



要过滤未放置的文件你可以尝试这样的额外磁盘空间:

  with open('file.tsv','r +')as infile: 
read_pos = write_pos = 0
line = infile.readline()
while line:
read_pos + = len(line)
如果line.strip():
infile.seek(write_pos)
infile.write(line)
write_pos + = len(line)
infile.seek(read_pos)
line = infile.readline( )
#将文件大小更新为可能缩小的新大小
infile.truncate(write_pos)


i am performing a hive query and storing the output in a tsv file in the local FS. I am running a for loop for the hive query and passing different parameters. If the hive query returns no output once in the for loop it prints an empty line in the tsv file. This causes NULL values to be pushed to my DB in the backend. Hence, after the for loop runs and the file is created - i have the below code to remove all the empty lines printed, but it doesn't work.

How do i remove the empty line from this file?

` 395.9   429.61  PT  
                       `

code:

with open('file.tsv','r+w') as file:
        for line in file:
          if line.strip():
            file.write(line)

thanks

Usually you would open the input file and write the non-empty lines to a second file:

with open('file.tsv') as infile, open('filtered_file.tsv', 'w') as outfile:
    for line in infile:
        if line.strip():
            outfile.write(line)

If you want to filter the file inplace you can use FileInput with the inplace option:

import fileinput
for line in fileinput.FileInput("infile", inplace=1):
    if line.strip():
        print line

however, this uses an intermediate file and may not work in low disk space situations.

To filter the file inplace without allocating any additional disk space you could try something like this:

with open('file.tsv', 'r+') as infile:
    read_pos = write_pos = 0
    line = infile.readline()
    while line:
        read_pos += len(line)
        if line.strip():
            infile.seek(write_pos)
            infile.write(line)
            write_pos += len(line)
        infile.seek(read_pos)
        line = infile.readline()
    # update file size to the new, possibly reduced, size
    infile.truncate(write_pos)