更新时间:2023-11-25 21:52:04
在 Apache Pig 中,您可以使用嵌套在 FOREACH 参见 Pig 基础知识.这是文档中的示例:A
是 B
中的一个包.
In Apache Pig you can use statements nested in FOREACH see Pig Basics. Here is example from the documentation:
A
is a bag in B
.
X = FOREACH B {
S = FILTER A BY 'xyz';
GENERATE COUNT (S.$0);
}
您可以使用 IsEmpty 和 ?: 运算符代替 COUNT
Instead of COUNT you can use IsEmpty and ?: operator
X = FOREACH B {
S = FILTER A BY 'xyz';
GENERATE (IsEmpty(S.$0)) ? 'xyz NOT PRESENT' : 'xyz PRESENT') as present, B;
}
或者只留下包含数据的包:
Or only to leave the bags that contain the data:
X = FOREACH B {
S = FILTER A BY 'xyz';
GENERATE B, S;
}
F = FILTER X BY not IsEmpty(S);
R = FOREACH F GENERATE B;
这将避免对自身进行昂贵的连接,因为额外的连接是额外的 Map Reduce 作业.
This will avoid costly join to itself, as extra joins are extra Map Reduce jobs.