Although when a hagfish is threatened, it will secrete slime that is small in quantity, it expands several hundred times as it absorbs seawater, forming a slime ball that can coat the gills of predatory fish and either suffocate them or distress them enough to make them flee.
How do you decide whether the "it" after the underlined portion refers to the hagfish or slime? Logically, it should be slime because a fish may puff up when threatened (Puffer fish) but cannot expand several hundred times. On the other hand, slime can.
Grammatically, "it" should refer to slime. Here is how.
", forming" is a participle that refers back to the "it" that we are talking about.
If "it" refers to the hagfish, then the sentence would say
The hagfish expands ...., forming a slime ball..OKAY
If "it" refers to the slime, then the sentence would say
The slime expands ...., forming a slime ball..Makes more sense.
So, "it" should refer to Slime and not hagfish.