Hmm. I have never see that meaning of the word "referee" used in that sense. In fact, I'm pretty sure that that is not an appropriate application of referee as a judge.
Here's definition #1 "#
One to whom something is referred, especially for settlement, decision,
or an opinion as to the thing's quality."
In the context of "reference writing", it doesn't make sense to apply a different definition of the word, especially when it's not a common (heck it's totally unknown as far as I know) use of the word.
Anyhow, the "referrer" in this case is not being asked to be a judge, they are being asked to be an advocate for the "referee". A "referee" in the sense that you are tying to apply it would, I guess, be the reader of the application, but even that would be a silly use of the term.
I'm still interested to find out which application this wording appeared in.