Even though it was not illegal for the bank to share its customers' personal and financial information with an outside marketing company in return for a commission on sales, the state's attorney general accused the bank of engaging in deceptive business practices
by failing to honor its promise to its customers to keep records private.
Option Elimination -
Usage of "by"
1. Agent of an action. For example, The book was written by Rabindranath Tagore. It means that "Rabindranath Tagore" is the book's agent or author here.
2. Method or means - I traveled to work by car. Here, "by car" acts adverbially to modify the verb "traveled." I traveled. How? By car.
3. Time or deadline - the report must be submitted by Friday.
4. Passive voice construction. The building was constructed by a team of architects.
5. Standard or unit of measurement - The land was measured by square feet.
(A) by failing to honor its promise to its customers to keep - Here, the prepositional phrase "by failing to honor its promise to its customers to keep" is modifying "the deceptive business practices" and answers "how did the bank engage in the deceptive business practices." Moreover, "failure to" is the right idiom. E.g., Failure to see the forest for the trees. Also, "promise to +base form" is the right idiom. E.g., I promise to help you with your homework.
(B) by its failure of honoring its promise to its customers to keep - we need "failure to honor."
(C) in its failing to honor its promise to its customers of keeping - "promise to keep" is correct.
(D) because of its failure in honoring its promise to its customers in keeping - "because of" usually is adverbial. So it'll modify the verb "accused," but the idea here is to have something to modify the "deceptive business practices" as to "how did the bank engage in the deceptive business practices." It would be nonsensical to say banks engaged in deceptive business practices because of failure to honor their promise to keep records private." So, "failure to honor their promise to keep records private" is "how" they indulged in the deceptive business practices and not the "cause" as shown by "because of." Moreover, "failure to honor" and "promise to keep" are the preferred idioms.
(E) because of its failure to honor its promise to its customers of keeping - the same issue of "because of." "promise to keep" is preferred.