Attacking a Castle never helps. The Towers constantly fire at the Elves, and if you don't have expert Ballistae, by the end of the battle, only "horns and legs" will remain.
You can simply place a few lone Elves at the bottom of the battlefield and the main group at the top, and your losses in the first round of shooting will be minimal:smile02: