Use Associate to link the up-down control to a companion control (sometimes called a buddy window). The up-down control automatically positions itself on the outer edge of the companion control and adjusts its height to match the companion control. 

At design-time the companion control does not move with the up-down control. To keep the up-down control and its companion control together at design time, either multi-select both controls before moving them, or first move the companion control and then reset the Associate property for the up-down control. 

AlignButton 

ArrowKeys 

Position