I was doing a skirmish and I had one hireling left in the fight vs one enemy. For some reason she stopped fighting and just stood there (bottom left) until I was done.
Hmm, looks like it could be a pathfinding issue. Thread title is somewhat misleading. She clearly did not exit combat mode, as her stance shows, weapon raised.