There are 7 executives, including the CEO and CFO, that are asked to

The line "may not both be assigned to the team" is very ambiguous. What is the source?

To answer your question, its a combination problem. The formula used in OE is that of combination.

total 7C5 =7!/(5!*2!) =21
now we need to subtract from Total CEO+CFO

CEO CFO ___ ____ ____
Lets assume that the 1st and 2nd places are for CEO and CFO. then we need to arrange the rest places - 7-2 (ceo+cfo) = 5 ; 5!/3!2!=10

21-10=11

C is the answ

Find out the notation they use.
They use P(7, 5) to mean 7C5 = 7!/5!*2!
P(n, r) = nCr = n!/(r!(n-r)!) (or the way you write - C(n, r). Just that nCr is more natural to me.)

By the way, in your solution, what was your thought when you wrote C(5, 2)?
5C2 = 5C3 so the answer was correct but I would write 5C3 because I would say, "Let me select both the CEO and the CFO and 3 more people out of the remaining 5 in 5C3 ways."
That is what they do in the solution when they write P(5, 3) since they are using P(n, r) in place of nCr.

Of course, you could say that out of the remaining 5 people, I reject any 2 and all others go in the team. If that is how you chose to write 5C2, then there is nothing wrong.

Correct me if my approach is wrong.
Let's take that there are only 6 people to choose from, only the CEO is there, and the CFO is out of picture, then there are 6 ways of selection.
Again, let's take only CFO, then 6 ways.
total ways= 6+6=12.
but there is one way in which neither of them get in the team (which is also repeated in both of the situation)
so 12-1=11.