您需要变换框:填充框；

circle {
  animation: grow 2s infinite;
  transform-origin: center;
  transform-box: fill-box;
}
@keyframes grow {
  0% {
    transform: scale(1);
  }
  50% {
    transform: scale(0.5);
  }
  100% {
    transform: scale(1);
  }
}

<svg
  xmlns="http://www.w3.org/2000/svg"
  width="80"
  height="110"
  version="1.1"
>
  <rect
    width="70"
    height="100"
    x="5"
    y="5"
    fill="white"
    stroke="red"
    stroke-width="10"
    rx="5"
  />
  <circle cx="40" cy="105" r="3" fill="blue" />
</svg>