且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

在两个文档之间找到相似的句子,并计算整个文档中每个部分的相似度

更新时间:2022-10-18 21:13:13

您可以使用 difflib 模块.

此模块提供用于比较序列的类和函数.例如,它可以用于比较文件,并可以产生各种格式的差异信息,包括HTML和上下文以及统一的diff.要比较目录和文件,另请参见 filecmp 模块.

在您的情况下,您需要 difflib.SequenceMatcher 类,用于比较任何类型的序列对,只要序列元素是可哈希的.

示例:

from difflib import SequenceMatcher
text_1 = "private Thread currentThread;"
text_2 = "private volatile Thread currentThread;"
s = SequenceMatcher(lambda x: x == " ",
                    text_1,
                    text_2)

现在要测量序列的相似性,请使用ratio(),它会在[0, 1]中返回一个float.根据经验, ratio()值超过0.6表示序列是紧密匹配.

>>> s.ratio()
0.8656716417910447

I took this example from web. My document one contains:

Document 1 :

Purpose of visit : For physical check up.

History of patient : This is the first admission for this 56 year old woman, who states she was in her usual state of good health until one week prior to admission. At that time she noticed the abrupt onset (over a few seconds to a minute) of chest pain which she describes as dull and aching in character. The pain began in the left para-sternal area and radiated up to her neck.

Medications : 1. Critizin. 2. p.n.b.s

Review of Systems :

HEENT:

1 or 2 beers each weekend; 1 glass of wine once a week with dinner.

Cadiovascular:

See HPI

Document 2 contains :

Purpose of visit : For physical check up.

History of patient : This is the first admission for this 56 year old woman, who states she was in her usual state of good health until one week prior to admission. At that time she noticed the abrupt onset (over a few seconds to a minute) of chest pain which she describes as dull and aching in character. The pain began in the left para-sternal area and radiated up to her neck. She does not smoke nor does she have diabetes. She was diagnosed with hypertension 3 years ago and had a TAH with BSO 6 years ago. She is not on hormone replacement therapy. There is a family history of premature CAD. She does not know her cholesterol level.

Medications : 1. Critizin. 2. Flexon

Review of Systems :

HEENT:

1 or 2 beers each weekend; 1 glass of wine once a week with dinner.

Cadiovascular: See HPI

Genitourinary: No complaints of dysuria, nocturia, polyuria, hematuria, or vaginal bleeding.

I was thinking split each line in file on the basis of (.) and split section on the basis of (:). But sometimes in file I also have 3.5 or in medicine section all medicine are seprated by (.) like medicine 1 hello. 2 hi.

How I can calculate similarity score between these sections of two files.

You can use difflib module.

This module provides classes and functions for comparing sequences. It can be used for example, for comparing files, and can produce difference information in various formats, including HTML and context and unified diffs. For comparing directories and files, see also, the filecmp module.

In your case, you need difflib.SequenceMatcher, class for comparing pairs of sequences of any type, so long as the sequence elements are hashable.

Sample example:

from difflib import SequenceMatcher
text_1 = "private Thread currentThread;"
text_2 = "private volatile Thread currentThread;"
s = SequenceMatcher(lambda x: x == " ",
                    text_1,
                    text_2)

Now for measuring the similarity of the sequences, use ratio() which returns a float in [0, 1]. As a rule of thumb, a ratio() value over 0.6 means the sequences are close matches.

>>> s.ratio()
0.8656716417910447