Just finished the final cave of the Caves of Chaos, but didn't empty all the caves. Advice?

I just ran the Caves of Chaos and the party skipped a few caves (I blogged about it here if anyone is interested).  Namely the Ork Lair, Minotaur Cave, and the Shunned Cavern.  The bugbear leader also escaped.  The party gained enough XP to hit level 4 just before the final fight in Temple of Evil Chaos.

When the bugbear leader escaped, he allied with the minotaur, and I repopulated that lair with monsters from the Minotaur Cave so if the party came back looking for the leader, they would have to fight with tons of stirges and fire beetles.  That seems tedious at this point.  Although swarms of stirges and firebeetles could get interesting.

The ex-prisoners also allied and took over a part of the cave.  

Should I upgrade the monsters and tell the party that the threat is still there? There was some decent loot in those caverns as well (+1 banded armor, +1 handaxe, and some other stuff).

I wanted to run the Isle of Dread next, but finishing off the final three caves might get them to level 5, and that would mean a complete re-work of the Isle.