The rule of law in Malaysia took a turn for the worse last year, says a study that ranks 16 Asian countries on their levels of freedom and liberty.