且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

如何在Dask中进行行处理和项目分配

更新时间:2023-08-23 14:56:52

Dask数据框不支持有效的迭代或行分配.通常,这些工作流很难很好地扩展.它们在熊猫本身中也相当慢.

Dask dataframe does not support efficient iteration or row assignment. In general these workflows rarely scale well. They are also quite slow in Pandas itself.

相反,您可以考虑使用 Series.where 方法.这是一个最小的示例:

Instead, you might consider using the Series.where method. Here is a minimal example:

In [1]: import pandas as pd

In [2]: df = pd.DataFrame({'x': [1, 2, 3], 'y': [3, 2, 1]})

In [3]: import dask.dataframe as dd

In [4]: ddf = dd.from_pandas(df, npartitions=2)

In [5]: ddf['z'] = ddf.x.where(ddf.x > ddf.y, ddf.y)

In [6]: ddf.compute()
Out[6]:
   x  y  z
0  1  3  3
1  2  2  2
2  3  1  3