且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

Sklearn Pipeline:如何为kmeans构建文本文本?

更新时间:2023-11-15 13:53:52

您需要做的是使用list1然后使用相同的 vectorizer transform训练vectorizer list1list2.这样可以解决问题.演示:

What you need to do is to train a vectorizer using list1, then with the same vectorizer, transform list1 and list2. This will solve the issue. Demo:

>>> list1 = ["My name is xyz", "My name is pqr", "I work in abc"]
>>> list2 = ["My name is xyz", "I work in abc"]
>>> vectorizer = TfidfVectorizer(min_df = 0, max_df=0.5, stop_words = "english", charset_error = "ignore", ngram_range = (1,3))
>>> vec=vectorizer.fit(list1)   # train vec using list1
>>> vectorized = vec.transform(list1)   # transform list1 using vec
>>> km=KMeans(n_clusters=2, init='k-means++', n_init=10, max_iter=1000, tol=0.0001, precompute_distances=True, verbose=0, random_state=None, cpy_x=True, n_jobs=1)
>>> km.fit(vectorized)
>>> list2Vec=vec.transform(list2)  # transform list2 using vec
>>> km.predict(list2Vec)
array([0, 0], dtype=int32)