Well, I did it that way on purpose... naturally, I wanted to always have Visas and the Handmaiden's faces nearby... But there's a definite advantage: if Atton retains his levels and starts leveling up the "Jedi" from level 6-7...
I personally noticed the Handmaiden's progress:
1st playthrough - The Handmaiden is poorly leveled (doesn't become a Jedi) and has not always useful skills - Atton defeats her on Telos easily...
2nd playthrough - The Handmaiden is well-leveled (doesn't become a Jedi) and has all the necessary combat skills - Atton defeats her in most cases - there's a chance if you constantly use medpacs...
3rd playthrough - The Handmaiden is leveled with a mix of useful combat and Jedi skills (becomes a Jedi around level 16-18) - Atton defeats her with moderate results - without medpacs, the chances are questionable, but they exist...
4th playthrough - The Handmaiden is leveled (becomes a Jedi at level 6 - that is, I didn't use her until she became a Jedi) according to my personal preferences. Usually, I level her up for consular Force powers, like lightning and telekinetic blast - Atton loses to the Handmaiden in a 70 to 30 ratio (without using medpacs)...
5th playthrough - The Handmaiden is leveled (becomes a Jedi at level 6) entirely in the style of a Jedi Guardian, using only combat auras - Atton loses without a chance, there's no need to even try...