Ok, step by step
1) Research Brigade HQ
2) Build Brigade HQ
3) Open the F2 screen, go to the Ground Units Tab
4) There is a drop-down menue named "Assign Parent Headquarters".
5) Select the battalion, you want to assign to your newly build Brigade HQ and open that drop-down menue.
6) On this list, your newly build Brigade HQ should be listed - select it.
7) Hit the "Transfer HQ" button
Repeat until you have assigned 4 units to the HQ (4 units is the limit any HQ can take.
For Divisional HQs, the steps are exactely the same except that you want to assign up to 4 Brigade HQs to it.