Since 4th edition came out, I've run many skill challenges as both a player and a DM, and I've read a number of articles from both Wizards and third parties that discuss how to create and run skill challenges. These are my thoughts on the subject.
In my eyes, the main purpose of a skill challenge is to reward players with XP outside of combat. For several editions, in both D&D and other gaming systems, experience points were generally only rewarded for enemies defeated in combat. PCs still went through the same processes of role-playing scenes, fact-finding, bribing, climbing, exploring, etc, but, at least in my games, we were rarely rewarded for it through XP. As a player, that was often annoying, to spend half the night running around playing the game but without reward. Now, that isn't to say that I only play D&D to earn experience points. But, leveling up, character advancement in general, is both fun and rewarding, so why wouldn't I want to do that often? And for previous editions and other rpgs, where different classes are not equally combat effective, players of the "skill monkey" should be equally rewarded for playing their characters at least as much as the fighter is for swinging his sword. Skill challenges fix this. They provide a simple, mechanical, predefined method of awarding experience points for skill use.
Unfortunately, this is rarely how they are used in play. Most skill challenges get bogged down with the mechanics of the system. Most players can easily give you a list of problems that exist within the skill challenge system so I won't bother repeating it. Instead, I'll explain how I think skill challenges should be run and how I try to run them at my tables.
First, never tell your players they are in a skill challenge. When you do, it often breaks down to a dice rolling frenzy. Any story aspect is quickly lost.
Second, don't use the suggested skill list. Instead, if a player can come up with a good reason to use a skill, let them try it. Ask them how they are using said skill to help the situation. Sometimes it's as simple as letting the player come up with flavor text for the roll. Instead of the barbarian rolling an Intimidate check against the crooked tax collector, he threatens to rips the agent's arms off and beat him to death with them if he doesn't provide the party with information. This is my general rule anyway, asking players how and why they want to use a skill rather than simply making a check. One danger here is accruing failures with inappropriate skill checks, using skills that, despite good flavor, have nothing to do with a given scenario. Think of it this way. If a certain skill use wouldn't garner a success, it shouldn't garner a failure either.
Third, try not to use high complexity skill challenges. At I believe it was Mike Mearl's suggestion, I tend to string lower complexity skill challenges together instead.
Fourth, and this has been said by many people over the years, skill challenges should further the story of the game. Both success and failure should be interesting, and never let your campaign hinge on the success or failure of any particular skill challenge. If the party gets to the same point regardless of whether or not they succeed, don't use a skill challenge. Moreover, an extra encounter or losing healing surges are not interesting failures. The former only delays the story of the game and the latter feels like nothing more than a big middle finger from the DM. As an example, think of the Fellowship of the Ring, when the party tries to cross the mountains (whose name escapes me at the moment) rather than going through Moria. Normally, the group could make a couple Athletics or Endurance checks and be done with it. It became a skill challenge because of Sauruman's interference. The party knew the consequences and when they failed, they were forced to go through Moria. (In this case, failure was actually more interesting than success would have been.)
Fifth, on that note, not every skill check made in your game needs to be part of a skill challenge. Only use skill challenges when another force is acting against the party (look at Sauruman in the previous example). Let's use another example. Say the party wants to gather information about something or someone. If they are simply in a new town and trying to get the lay of the land, just have them roll some skill checks (but remember to keep up the flavor). On the other hand, if they are trying to gather reconnaisance on a nefarious figure who works hard to keep a low profile and may even have the townsfolk intimidated, that might require a skill challenge.
Finally, take a lesson from your combat encounters and don't be afraid to change things up halfway through a skill challenge. Have triggered events planned for when the party reaches a certain number of successes or failures. Before 4th edition came out, Mike Mearls talked on podcast (probably the Wizards of the Coast D&D podcast) about how skills in 4th ed might work. He gave the following scenario (paraphrasing from memory): The party has an audience with the King and is trying to convince him to do thing X. Everything is going well and the King seems receptive but then his royal advisor enters the chamber, and the advisor is subtlely countering everything you say. From any number of avenues (Insight, History, Streetwise), a PC might know that the advisor is a coward. You might surmise that this could be a good time to have the barbarian scare him away. I think that would be an awesome skill challenge!
In the end, I said all that to say this. Play D&D and use skills the same way you've always used them, to accomplish (or not) the same goals in the same number of attempts. As a DM, have a rough idea of how many successful checks it takes to get through a given scenario, and reward XP accordingly. The skill challenge can simply be a rubrik. It can certainly be more than that, but it doesn't have to be.