I don't agree with your methods.
First, you should divide by 2 only AFTER you subtract the unfav combinations!
Second, stolyar, this are not combinations you're counting! I.e. GGGGYY makes 4C4*4C2 = 6 combinations, not one!
I used a different approach and i got another answer. Perhaps i just can't get your logic. Can you please check your solutions and mine so that we reconcile our answers? I'm trying to adapt this problem to quiz engine.
My solution:
Dividing in two groups is same as "taking" some balls while "leaving" others.
We can take 6 balls in 12C6 ways.
12C6 = 12! / 6! * 6! = 7 * 8 * 9 * 10 * 11 * 12 / 6 * 5 * 4 * 3 * 2 = 7 * 3 * 2 * 11 * 2 = 924.
The combinations that don't match:
No g (no v, no y) taken: 8C6. Here we count such takes as VVVYYY, VVVVYY, etc.
Total: no color taken - 3 * 8C6;
3 * 8C6 = 3 * 8! / 6! * 2! = 3 * 7 * 8 / 2 = 3 * 7 * 4 = 84
No g (no v, no y) left, but all colors taken: 4C1*4C1 = 16. Here we count such takes as GGGGYW, which would take all balls of a given color and leave none of them to the second group. Total: 3 * 16 = 48.
Answer: (924 - 48 - 84) / 2 = 792 / 2 = 396.
Why divide by 2? Because we "taken" and "left" groups are symmetrical, and we therefore have counted each combination twice.