In an interview with Accra-based Angel FM, Prof Attafuah said he was focused on his mandate and would not be swayed by his detractors including party supporters.
However, his [Prof Attafuah] comment has elicited varied responses with many NPP members expressing anger about the comment.
Gabby Asare Otchere-Darko in a Facebook post said the NIA boss could "have phrased his words intelligently and with basic sensitivity".
"He had no business stamping his feet on the expectations and hopes of unemployed party loyalists with such reckless callosity.
"Very unwise, and some may even say selfish, considering he lost this same job when NPP lost in 2008 only to regain it after NPP regained office! At least, his memory and self-benefit ought to have even guided his tongue and stopped him from speaking out of turn," he said.
He added: "My good friend, Ken, should check his excitement levels when he has an audience. It helps! The bad way he said what he actually meant to say is what has caused this whole controversy. I do not think he meant it as it came out. Surely, he couldn’t have and I know so to be the case. His loyalty to the party and party members has never been in doubt. But, he is also a true patriot. A Ghanaian is always eager to serve his nation.
"In this case, he jammed a foot in his mouth and shot from the hip when the nozzle was aimed at his own family jewels, so to speak. He must apologise and let's move on".