gettinit
Karishma are not to worry about the signs of M and N at the beginning of solving this problem? or do you assume if m is negative it still is m for the sake of solving for example?
Also you subtract -2(MN) from (m+n)^2 because this would actually equal m^2+n^2?
toughie.
When you assume values to be m or n, you don't have to worry about the signs. e.g. m could take the value 5 or -5.
And yes, since (m+n)^2 = m^2+n^2 + 2mn, I substituted for m^2+n^2.
Nevertheless, as Bunuel pointed it, this question involved complex roots and you will not see anything such as this in GMAT. Since the square of roots add up to give you 0, if you consider only real values, the only possible option is that both roots are 0.
If I change the question a little and make it:
Given the quadratic equation x^2-(A+2)x+(A+3) = 0, for what value of A will the sum of the squares of the roots be 13?
and use the same method as above,
m^2 + n^2 = (m + n)^2 - 2mn =13
(A + 2)^2 - 2(A +3) = 13
When you solve this, you get A = -5 or 3
Equations can be:
x^2 + 3x - 2 = 0 or x^2 -5x + 6 = 0
The sum of the squares of their roots will definitely add up to give 13.