更新时间:2023-12-02 12:14:52
可以使用 sklearn 的 StratifiedKFold
,来自在线文档:
分层 K 折交叉验证迭代器
提供训练/测试在训练测试集中拆分数据的索引.
这个交叉验证对象是 KFold 的一种变体,它返回分层折叠.褶皱是通过保留每个类的样本百分比来制作.
>>>从 sklearn 导入 cross_validation>>>X = np.array([[1, 2], [3, 4], [1, 2], [3, 4]])>>>y = np.array([0, 0, 1, 1])>>>skf = cross_validation.StratifiedKFold(y, n_folds=2)>>>长度(skf)2>>>打印(skf)sklearn.cross_validation.StratifiedKFold(labels=[0 0 1 1], n_folds=2,shuffle=False,random_state=None)>>>对于 skf 中的 train_index、test_index:...打印(火车:",train_index,测试:",test_index)... X_train, X_test = X[train_index], X[test_index]... y_train, y_test = y[train_index], y[test_index]训练:[1 3] 测试:[0 2]训练:[0 2] 测试:[1 3]
这将保留您的班级比率,以便拆分保留班级比率,这将适用于 Pandas dfs.
根据@Ali_m 的建议,您可以使用 StratifiedShuffledSplit
接受分流比参数:
sss = StratifiedShuffleSplit(y, 3, test_size=0.7, random_state=0)
会产生 70% 的分割.
I have a multi class classification problem and my dataset is skewed, I have 100 instances of a particular class and say 10 of some different class, so I want to split my dataset keeping ratio between classes, if I have 100 instances of a particular class and I want 30% of records to go in the training set I want to have there 30 instances of my 100 record represented class and 3 instances of my 10 record represented class and so on.
You can use sklearn's StratifiedKFold
, from the online docs:
Stratified K-Folds cross validation iterator
Provides train/test indices to split data in train test sets.
This cross-validation object is a variation of KFold that returns stratified folds. The folds are made by preserving the percentage of samples for each class.
>>> from sklearn import cross_validation
>>> X = np.array([[1, 2], [3, 4], [1, 2], [3, 4]])
>>> y = np.array([0, 0, 1, 1])
>>> skf = cross_validation.StratifiedKFold(y, n_folds=2)
>>> len(skf)
2
>>> print(skf)
sklearn.cross_validation.StratifiedKFold(labels=[0 0 1 1], n_folds=2,
shuffle=False, random_state=None)
>>> for train_index, test_index in skf:
... print("TRAIN:", train_index, "TEST:", test_index)
... X_train, X_test = X[train_index], X[test_index]
... y_train, y_test = y[train_index], y[test_index]
TRAIN: [1 3] TEST: [0 2]
TRAIN: [0 2] TEST: [1 3]
This will preserve your class ratios so that the splits retain the class ratios, this will work fine with pandas dfs.
As suggested by @Ali_m you could use StratifiedShuffledSplit
which accepts a split ratio param:
sss = StratifiedShuffleSplit(y, 3, test_size=0.7, random_state=0)
would produce a 70% split.