This is an example of a "Symbolism" question (you'll likely see one of them on Test Day). The basic concept is usually pretty simple - the prompt will "make up' a math symbol, explain to you what it "means" mathematically, then ask you to use it to perform a calculation.
Since the * symbol makes people think about multiplication, I'm going to change it to a # symbol. Here, we're told....
S#T = (S-1)(T+1)
You are meant to take the numbers that S and T represent and place them into the calculation. So, as an example, if we had...
2#3 then that would equal (2-1)(3+1) = (1)(4) = 4
Knowing that, we're given....
(-2)#X = -12 and we're asked to solve for X
Let's 'substitute in' the (-2) and the X....
(-2 - 1)(X + 1) = -12
Now, you can do the algebra....
(-3)(X + 1) = -12
(X + 1) = 4
X = 3
GMAT assassins aren't born, they're made,
Official Guide 2016 Question Breakdown:
GMAT Club Verified Reviews for EMPOWERgmat & Special Discount
EMPOWERgmat Podcast - A Wild Secret About The GMAT Algorithm