If the creature is armed with a melee weapon and you arn't, you have disadvantage on your check.
I don't know, but when swordfighting irl, I find it actually easier to disarm someone with a free hand, especially a sword, all you do is grab the pommel and turn the right way - it doesn't even require strength, you just have to know what direction to twist (the pinkie points the way!).
I think the only thing that should cause disadvantage is if the opponent has a two-handed weapon... that's always difficult.
Also, I think combat skill is important, in 3E, you had to make an opposed attack roll, to me that makes more sense. Anyway, its one of those things that's really is to House Rule.