Ancient hunter-gatherers developed instincts that stigmatized selfishness, and voluntary cooperation was encouraged as well, whether it be within the group or also with outsiders.
(A) selfishness, and voluntary cooperation was encouraged as well, whether it be within the group or Use of "as well" for "encouraged" does not show similarity with the earlier use of "stigmatized", "or also" does not convey that cooperation was encouraged with insiders and outsiders. Eliminate.
(B) selfishness, encouraging voluntary cooperation as well, both within the group but Same error as in (A) with the use of "as well". "Both" shows similarity between two things whereas "but" shows contrast - a contradiction. Eliminate.
(C) selfishness and also encouraging voluntary cooperation, within the group and Use of "also" produces the same error as use of "as well" in (A) and (B). Eliminate.
(D) selfishness and they encouraged voluntary cooperation, whether it be within the group or Antecedent of "they" is ambiguous. Eliminate.
(E) selfishness and encouraged voluntary cooperation, not only within the group but Correct answer.
Hope this helps.