且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

Sklearn 字符串的余弦相似度,Python

更新时间:2022-02-04 23:12:58

对于短字符串,Levenshtein distance 可能会产生比基于单词的余弦相似度更好的结果.下面的算法改编自 维基教科书.由于这是一个距离度量,因此分数越小越好.

For short strings, Levenshtein distance will probably yield better results than cosine similarity based on words. The algorithm below is adapted from Wikibooks. Since this is a distance metric, smaller score is better.

def levenshtein(s1, s2):
    if len(s1) < len(s2):
        s1, s2 = s2, s1

    if len(s2) == 0:
        return len(s1)

    previous_row = range(len(s2) + 1)
    for i, c1 in enumerate(s1):
        current_row = [i + 1]
        for j, c2 in enumerate(s2):
            insertions = previous_row[j + 1] + 1
            deletions = current_row[j] + 1
            substitutions = previous_row[j] + (c1 != c2)
            current_row.append(min(insertions, deletions, substitutions))
        previous_row = current_row

    return previous_row[-1]/float(len(s1))

example_1 = ("I am okey", "I am okeu")
example_2 = ("I am okey", "I am crazy")

print(levenshtein(*example_1))
print(levenshtein(*example_2))