在YARN中，您可以为每个群集工作人员提供不同的硬件资源配置。然后，YARN将决定每个工作人员可以分配多少个任务（容器）来平衡工作负载。换句话说，您可以设置不同数量的内核和内存 yarn-site.xml 分别为每个工人。

这两个配置选项与您相关：

• yarn.nodemanager.resource.cpu-vcores 设置每个当前worker的CPU核心数节点


• yarn.nodemanager.resource.memory-mb 来设置可以使用的物理内存量（以MB为单位）当前工作节点

I currently have high power (cpu/ram) hosts in the cluster and we are considering to add some good storage but low power hosts.

My concern is that it will reduce the jobs performance. Map/Reducers from the new (less powerful) hosts will run slower and the more powerful ones will just have to wait for the result.

Is there a way to configure this in Yarn ? Maybe to set a priority for the hosts or to assign mapper/reducers according to the number of cores on each machines.

Thanks, Horatiu

In YARN you can provide a different hardware resources configuration to each of the cluster workers. The YARN will then decide how many tasks (containers) may be allocated at each worker to balance the workload.

In other words you can set different number of cores and memory in yarn-site.xml for each worker separately.

These two configuraton options are relevant for you:

• yarn.nodemanager.resource.cpu-vcores to set number of CPU cores per current worker node
• yarn.nodemanager.resource.memory-mb to set the amount of physical memory (in MB) that can be used at the current worker node