There's little need for all your colonies to be manufacturing centers. If anything, it's a bad idea, given how colony governors work.
Those that do have factories ought to be largely mineral self-sufficient, though it is possible to futz it.
The approach I like to take is to have mining colonies feeding into a major colony, which then feeds into the homeworld. Freighters automatically ship most of the major colonies output back home, picking up the >reserve amounts. Meanwhlie, the colony builds items (usually scanners, CFs, and mines) out of the reserve minerals. Any minerals the colony doesn't have in its inputs are shipped out from the homeworld; I don't usually automate that.
It's also nice to establish missile production plants on major extrasolar colonies. One game, I based my old fleet out of a major colony using/manufacturing size 4 missiles while the new construction based out of the homeworld with size 6 missiles.