Yes, this is an excellent tactic.
Sometimes, players prepare 13 Paladins (6 from Monks and 7 from Men-at-Arms) at the end of the week, and then another 13 at the beginning of the next week, resulting in +26 Paladins in two days.
Players quickly become mega-favorites because the number of cavalry units they have almost doubles.
It's also good to strike with them quickly, and if the cavalry is surrounded and cornered, teleport and strike again (for example, with Axel).
It's very difficult to do anything about this.