the defensive coordinator signals to the middle linebacker. or in a 3-4 defense (3 lineman, 4 linebackers meaning 2 middle linebackers) one of the linebackers is designated the play caller. the defense will often huddle the same as the offense. when the offense goes no huddle, alot of the players will look to the sideline to get the signals or wait for the linebacker to hand signal or yell the formation.

middle linebacker

For most teams it's the Middle Linebacker. He's in the middle of all of the other players, and he can communicate audibles and shifts to his teammates.
However, if the linebacker is inexperienced, sometimes a different player will do it, such as an Outside Linebacker or a Safety.

the defensive cordnator signals to the middle line backer while the team is in the huddle