Trim

Handle, knob, or latch. Trims are often used on the opposite side of a door that has a panic exit bar.