且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

在 Python 3.3.2 中计算短语频率

更新时间:2021-08-28 10:05:13

首先,这就是我将如何生成您所做的 cnt(以减少内存开销)

First of all, this is how I would generate the cnt that you do (to reduce memory overhead)

def findWords(filepath):
  with open(filepath) as infile:
    for line in infile:
      words = re.findall('w+', line.lower())
      yield from words

cnt = collections.Counter(findWords('02.2003.BenBernanke.txt'))

现在,关于短语的问题:

Now, on to your question about phrases:

from itertools import tee
phrases = {'central bank', 'high inflation'}
fw1, fw2 = tee(findWords('02.2003.BenBernanke.txt'))   
next(fw2)
for w1,w2 in zip(fw1, fw2)):
  phrase = ' '.join([w1, w2])
  if phrase in phrases:
    cnt[phrase] += 1

希望能帮到你