As previously pouted out, it is quite possible to build small 1-2 crew missile fighters. The key is to keep a close eye on your component crew requirements and to generally keep components as small as possible. Armor and engine tech are also essential, I usually go for at least high density duranium armor before I start any practical fighter designs.
I've started using the class designations below in recent games as they jive better with my expectations of what a fighter craft should be:
Fighter: Max 3 crew, max 50 tons
Boat: Max 10 crew, max 250 tons
Craft: No crew limit but typically less than 20, displacement between 250-500 tons
Corvette: No crew limit, displacement between 500-1000 tons
With a few levels of engine tech and armor tech there are quite a lot of interesting design options for these ship sizes, fighters included