The first thing we need to do is define "balance" in D&D, particularly the editions published by Wizards of the Coast. Not all RPGs necessarily choose "balance" as a game design goal. Balance is only important where the game is to be decided primarily on the mechanics. Other games serve as a battle of wits between the players and GM. In such games, balance is less important, or even detrimental, because the game expects the GM to "rig" encounters to force players to act in unexpected and unanticipated ways.
However, for the foreseeable future, balance is an will be important in D&D. So, with that caveat, I think there are several types of balance:Intra-Party Parity
This subset of balance attempts to ensure that all players can contribute equally in the default adventures assumed by the game. In D&D, this usually means combat, and 4e has made herculean efforts to ensure intraparty parity.... in combat.
First, intra-party parity provides predictability to let the DM know how powerful to make encounters. In 3e, which did not really have encounter-scoped intra-party parity, the fans created a tier system for classes, which ranked classes of similar level by potency. DMs have to adjust encounters based on how many tier 1 or tier 4 characters are in their party. Actual parity creates uniformity in result, which aids DMs with encounter design.
Second, intra-party parity avoids issues of power envy and accusations of power gaming amongst players.
Third, parity is integral to the concept of teamwork, on which D&D (and particularly 4e) is based. In a team, each member is expected to contribute. If one member contributes less, you don't feel like a team member - you feel like a sidekick. And while, some people may choose, for roleplaying purposes to play a sidekick, nobody should be forced into a sidekick role because the concept they want to play turns out to be unbalanced.
Intra-party parity has been defined in different ways over the decades, and can be distinguished primarily on the scope of the adventure in which the game attempts to achieve balance:
Campaign. In early editions, intra-party power was balanced over a campaign. It was considered all right for a fighter to overpower the wizard at low levels, because the wizard would overpower the fighter at high levels. This is phenomenon is known as "linear fighters, quadratic wizards". There are several problems with this. The first is that many campaigns did not begin at 1st and end at 20th. You might begin at 7th. You might run a one-shot at a specific level. If the party is not going to be adventuring together for the entire campaign, then you are not achieving balance. The second issue is that a player need not keep the same PC for an entire campaign. PCs die, or retire. Players may simply complain that the fighter he played from 1st to 6th is now boring and so he wants to play a wizard. Heck, in the 1990's I had a fighter-cleric who died in 2e at 8th level. Rather than have him be resurrected, I generated a new illusionist PC. I wasn't consciously trying to game the system. I was just getting bored with the character, and I thought the illusionist had a lot of great options. It didn't occur to me that I was killing the low levels when the illusionist would have been hiding behind my fighter-cleric.
Adventure. Another scope of intra-party parity that could be sought is across an adventure. One PC may be expected to be strong in early encounters, but weaken over time, while others can slowly build power over the adventure. While an intriguing concept, I doubt it would be fun for one reason: adventures usually end on a climactic battle, and nobody wants to be unable to shine when fighting the Lord of the Dungeon because he peaked while taking out the guards at the front gate.
Encounter. This is the balance that 4e sought to achieve, and I approve. Each player feels that in any given encounter, they should have an opportunity to shine and contribute equally.
Noncombat. D&D has always sought combat parity, but has almost never sought noncombat parity. It is still considered acceptable for one player to specialize and outshine in certain types of noncombat. That is why we have terms like "skill monkey" and "party face". Personally, I think these are outdated anachronistic concepts. Adventurers are a team who fights together, but also work together in other arenas. They should exhibit teamwork there, and contribute equally to any encounter in which they find themselves. I can only hope future editions will address noncombat balance.'Tagonist Scale
This type of balance occurs between PCs (the protagonists) and the NPCs (the antagonists). This does not mean that PCs and NPCs are built identically. That would result in too many TPKs because each battle would result in a 50% chance of defeat for the players. What 'tagonist scale represents is that the PCs and NPCs are imposing predictable and proportionate levels of damage and effects on one another at any given level. 3e accomplished this (or tried) with the CR system and by ensuring the PC wealth was about quadruple NPC wealth at the same level. 4e uses charts to ensure that NPCs have appropriate hp, and impose appropirate amounts of damage. The treasure distribution also contributes to this balance by defining how many magic items a PC will be able to bring to bear against his opponents.
Tagonist scale is relatively new. In TSR-editions, things were measured by hit dice, with a sort of vague notion of monsters divided into varying levels. But some monsters were always too powerful for their level (dragons, mostly) and others were insufficiently powerful. Also, in TSR-editions, the number of creatures who might appear was based on a random die roll. You could get 3 orcs or 300 orcs. One was a nonevent, the other, a desperate retreat from an overwhelming army. The 3e CR system was the first system, and it was notably clunky and imprecise. 4e has definitely improved things, but I think there's room for even more improvement.What Balance is Not
Many discussions of balance confuse a lot of things balance is not.
Balance is not Homogeniety. Being balanced does not mean everybody acts identically. Balance means everybody is equally effective, but how they achieve that effectiveness can vary from character to charater. Some PCs inflict individual damage, some group damage, some impose conditios. Some PCs absorb damage, some avoid damage, some ameliorate damage.
Balance is Not a Function of Class Roles. A common myth is that the class roels aid balance. They don't. They serve to differentiate character form one another and to encourage teamwork by ensuring that no character can do all things. They don't aid balance. In fact,they hamper evaluations of intraparty balance, because strikers are expected to do more damage than defenders, making it hard to determine if any given striker is balanced as to any given defender. They also hamper 'tagonist balance because a striker-heavy party will inflict more damage but can absorb less, while a defender heavy party can absorb more and inflict less. This makes it more difficult to build encounters, as the DM once again must carefully consider any given party composition's strengths and weaknesses.
Balance is Not Placing Undue Pressure on Players to Succeed. Even in a hypothetically perfectly balanced game, PCs can die, and not simply due to player error. The game still runs on dice and probabilities. Balance can only tell you the probability of victory, not the certainty of victory. And human error is not for the players alone. DMs can, intentionally, or unintentionally, design encounters where the PCs are outmatched. 'Tagonist balance makes this more unlikely, but not impossible.
Balance is Not Futile Even Though It May Be Unachievable. Balance will always be tempered by differentiation. As long as players are expected to have separate roles in a party, perfect balance is unachievable. The only perfectly balanced games are competitive ones, like chess, not cooperative ones like D&D. However, balance is not a binary proposition. There are degrees of balance. And in the end, a game designer must always try to maximize intraparty balance, 'tagonist balance, and PC differentiation in order to maximize the fun.
Balance is not the only factor for designing a game like D&D. Nor is it a necessary one. I think 4e has appropriately chosen to try to design a balanced game. If future designers also decide that balance is important, I hope they consider the above.