'looked' is not a modifier in the sentence. It is the verb. If something is a modifier, you can remove it and still have a grammatically correct sentence. Here, if you delete the word 'looked', you no longer have a verb in the main clause ('damaged' is in the subordinate clause). If you strip away the modifiers, you get the sentence "The houses looked abandoned".
I don't really like the sentence, because I find the modifier beginning with "of which" to be ambiguous, so if that sentence appeared in an SC question, I'd be looking for a better answer.