Why does the man from the new neighborhood ask the Younger family to move to a different neighborhood?

The neighborhood the Youngers were going to move into was classed as a 'white people only' neighborhood and that no one of colour could live in the neighborhood.