I got 5 as well

It's true that it's not a usual GMAT question
I worked it out using an XY plan.
The figure is symetrical. So, we can have the same kind of rectangle at the bottom left of the square (1 vertex on C and the other on the circle).
By defining the origin 0(0,0) = C(0,0), the equation of the circle is:
(1) (x-r)^2 + (y-r)^2 = r^2 with r the radius of the circle.
As the vertex of the rectangle on the circle has the coordinates (2,1), we can plug the values of x & y in (1).
(2-r)^2 + (1-r)^2 = r^2
<=> 4 - 4*r + r^2 + 1 - 2*r + r^2 = r^2
<=> r^2 - 6*r + 5 = 0
We can use the "reduiced" Delta = 3^2 - 5*1 = 4 = 2^2
Thus, we have 2 possibilies :
o r= (3+2)/1 = 5
o r= (3-2)/1 = 1
For the latter radius, the vertex lying on the circle is represented by a point on the "other side" of the circle. In the case of this picture, the radius is 5.