personally,
i would get the biggest plot that i am willing to afford.
the house plans come secondary.

i can always adjust my house plans, but there is a good chance, once i get neighbors, i can not make my land expand.