Why the contents are vertically centered?

There's no specific reason. This is the way UAs handle the position of value/content of buttons (including <button>, <input type="button">)¹.

How to remove vertical centering?

Well, there's a way to achieve that. First, a little background is needed.

But before that, you should note that <div> elements are supposed to be used where flow contents are expected. This means that they are NOT allowed to be placed inside <button> elements.

As per HTML5 spec (Which is at PR state right now):

Content model for element button:
Phrasing content, but there must be no interactive content descendant.

Therefore, a valid HTML could be like this:

<button>
    why? <br>
    are these centered vertically? <br>
    And how to remove it?
</button>

Background

In an inline flow, inline-level elements (inline, inline-block) can be aligned vertically inside the parent by vertical-align property. Using that with a value other than baseline makes inline-level elements position somewhere other than the baseline of the parent (which is the default place).

The key point is that taller elements would affect the line box / baseline.

The Solution

First, in order to handle the position of the lines, we need to wrap them by a wrapper element like <span> as follows:

<button>
    <span> <!-- Added wrapper -->
        why? <br>
        are these centered vertically? <br>
        And how to remove it?
    </span>
</button>

In cases that the parent - the <button> in this case - has an explicit height, by any chance if we could have a child element having the exact same height of the parent, we would be able to expand the height of the line box and surprisingly make our desired in-flow child - the nested <span> in this case - be aligned to the top vertically by vertical-align: top; declaration.

10.8 Line height calculations: 'vertical-align' property

This property affects the vertical positioning inside a line box of the boxes generated by an inline-level element.

top
Align the top of the aligned subtree with the top of the line box.

EXAMPLE HERE

button { width: 100%; height: 200px; }

button > span {
    display: inline-block;
    vertical-align: top;
}

button:after {
    content: "";
    display: inline-block;
    vertical-align: top;
    height: 100%;
}

Last bot not least, if you'd like to use min-height rather than height for the button, you should use min-height: inherit; for the pseudo-element as well.

EXAMPLE HERE.

---

¹_{Chrome and Firefox also display the value of text inputs vertically at the middle while IE8 doesn't for instance.}