且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

How can R and Hadoop be used together?

更新时间:2022-09-22 11:59:37

Referer: http://www.quora.com/How-can-R-and-Hadoop-be-used-together/answer/Jay-Kreps?srid=OVd9&share=1

 

Another way to answer this question is that they don't really integrate very well.

The advantage of R is not its syntax but rather the incredible library of primitives for visualization and statistics. These libraries are fundamentally non-distributed, and almost always operate on data resident in memory. So, for example, if you are finding R's glm method slow (or completely infeasible) on a particular dataset, there is really no way to make it run faster with Hadoop.

The reason it is important to point this out is that
1. Transparently distributed R is every data geeks wet dream.
2. I have sat through numerous presentations from distributed database vendors claiming to provide this.

What hadoop and database vendors can provide is the ability to run R in parallel on lots of little data sets. Virtually none of the libraries will work on a data set larger than memory.

 

Referer: http://www.quora.com/How-can-R-and-Hadoop-be-used-together/answer/Jay-Kreps?srid=OVd9&share=1

声明:如有转载本博文章,请注明出处。您的支持是我的动力!文章部分内容来自互联网,本人不负任何法律责任。
本文转自bourneli博客园博客,原文链接:http://www.cnblogs.com/bourneli/p/3233398.html,如需转载请自行联系原作者