A and B decide to play a game based on probabilities. They

The probability that A wins the game after the first round is \(\frac{5}{6}*\frac{1}{6}\) (B must get anything but 5 and A must get 5);

The probability that A wins the game after the second round is \((\frac{5}{6}*\frac{5}{6})*(\frac{5}{6}*\frac{1}{6})\) (both must get anything but 5 in the first round and A must win in the second round);

The probability that A wins the game after the third round is \((\frac{5}{6}*\frac{5}{6})*(\frac{5}{6}*\frac{5}{6})*(\frac{5}{6}*\frac{1}{6})\);

and so on...

The overall probability that A wins, will be the sum of the above probabilities: \(\frac{5}{6}*\frac{1}{6}+(\frac{5}{6}*\frac{5}{6})*(\frac{5}{6}*\frac{1}{6})+(\frac{5}{6}*\frac{5}{6})*(\frac{5}{6}*\frac{5}{6})*(\frac{5}{6}*\frac{1}{6})+...\)

Notice that we have the sum of the terms of geometric progression with the first term equal to \(\frac{5}{6}*\frac{1}{6}=\frac{5}{36}\) and the common ratio equal to \(\frac{5}{6}*\frac{5}{6}=\frac{25}{36}\).

Now, the sum of infinite geometric progression with common ratio \(|r|<1\), is \(sum=\frac{b}{1-r}\), where \(b\) is the first term.

So, the above sum equals to \(\frac{\frac{5}{36}}{1-\frac{25}{36}}=\frac{5}{11}\).

Answer: C.

P.S. Not a GMAT type of question.