且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

检查包中是否存在元素?

更新时间:2023-11-25 21:52:04

在 Apache Pig 中,您可以使用嵌套在 FOREACH 参见 Pig 基础知识.这是文档中的示例:AB 中的一个包.

In Apache Pig you can use statements nested in FOREACH see Pig Basics. Here is example from the documentation: A is a bag in B.

X = FOREACH B {
        S = FILTER A BY 'xyz';
        GENERATE COUNT (S.$0);
}

您可以使用 IsEmpty 和 ?: 运算符代替 COUNT

Instead of COUNT you can use IsEmpty and ?: operator

X = FOREACH B {
        S = FILTER A BY 'xyz';
        GENERATE (IsEmpty(S.$0)) ? 'xyz NOT PRESENT' : 'xyz PRESENT') as present, B;
}

或者只留下包含数据的包:

Or only to leave the bags that contain the data:

X = FOREACH B {
        S = FILTER A BY 'xyz';
        GENERATE B, S;
}
F = FILTER X BY not IsEmpty(S);
R = FOREACH F GENERATE B;

这将避免对自身进行昂贵的连接,因为额外的连接是额外的 Map Reduce 作业.

This will avoid costly join to itself, as extra joins are extra Map Reduce jobs.