Because it is not the color of anyones jersey and it makes it easier to tell if someone is holding. I play highschool football in wisconsin and we have that rule too for our linemen.

It also helps referees to see if receivers made the catch. gray and brown dont mix.......

They don't have to

I was playing college football when they enacted this rule and we were told by officials in our scrimmage that the reason they changed it was easier to spot an illegal substance on the glove to help with tackiness.

since when you are concern about college players' gloves color?
Nobody really cares.