Poverty of the ummah (community) will become worse if the leaders are more concerned with building wealth for themselves and their own families and cliques, said the Sultan of Perak, Sultan Azlan Shah.