Axed deputy prime minister Muhyiddin Yassin has stressed that he is not a 'yes man' and reiterated that he will not relinquish his Umno deputy president post.