Since last September or so I has been on a quest to discover what I think defines D&D. Like it or not Paizo has done reasonably well for themselves probably based on using a popular rules set and fantastic adventures. WoTC has focused on rules bloat type books primarily directed towards players in both 3rd and 4th ed. This business strategy did make sense based on the knowledge they acquired from TSR in the late 90's. TSR in the 90's focused on settings and fluff type bloat and in effect they split their own market. For example a Planescape adventure was really only of use to Planescape players and this was compounded by Planescape being sold as a loss leader in theory being able to sell later supplements at a profit. That is the reason why boxed sets became extinct or limited to expensive 3rd party specials from 2000 on wards.
However there is another business theory for D&D that Lisa Stevens at Paizo has subscribed to. Jason Goodman researched a hypothetical D&D book based on the history of the game and he came the following conclusion. D&D has had two massive peaks in its popularity and several erm dips. The height of D&Ds commercial popularity was the early 80's with the second peak being in 2000 and 2001. Back in the 80's splat books as such were kind of rare in AD&D and D&D (BECMI). Adjusted for inflation this is the only time D&D has made it to around $50 million dollars with 1983 income being between 20-23 million dollars (accounts vary). TSR was not run very well however and by 1985 they were several million in debt and Gygax was forced out. Gygax himself said all that money was coming from adventures. The Keep on the Borderlands apparently sold 1 million copies. So good adventure do actually sell and can make millions of dollars- hence the success of Pathfinder despite the PF rules not being the best and my personal theory of mechanics do not matter (much) the TSR WoTC inherited produced relatively few adventures focusing instead on "story telling" type settings. Also keep in mind at the time White Wolfs World of Darkness was becoming popular and for a long time RPGs more or less meant D&D+ indie type games.
This leads me back to WoTC and the other peak of D&D's popularity which was 2000/2001. This explains why WoTC keeps making new sets of core rule books such as 3.5, 4th ed and Essentials. They seem to want to replicate those glory years all over again. They are like a drug addict trying to recreate that 1st high all over again. I would argue that 2000/2001 was one of those once in a generation things fueled by the relative quality of the d20 system, the OGL ad the fact that D&D had been semi static for decades. People were excited to play 3rd ed because the rules made a lot more sense and I suppose after 23 years or so of very similar somewhat ass backwards rules new is good. New is not so good if you change it every 3-4 years. So for 20 odd years (1989 onwards) D&D has been starved of good quality adventures at least from TSR/WoTC with the few good ones being the exception. The "good" D&D adventures are usually the ones from the early 80s or even the late 70's which are the classics. The Vaut of the Drow, Against the Slavelords, the B series for BECMI (B1-B12), Isle of Dread, Temple of Elemental Evil etc. Dungeon Magazine was also a good source for good adventures and not just Paizo era Dungeon as I really like 1st ed Dungoen adventures despite 1st ed being my least favourite out of the pre 4th ed D&Ds. I still like 1st ed a lot I just like BECMI and 2nd ed better.
Now what makes a good adventure? I would argue that a good adventure has 3 elements supported which is combat, exploration and interaction although early D&D adventures are often heavy on the combat they do tend to be a bit more than dungeon hacks. The Isle of Dread for example being exploration based and even a dungeon hack like the Temple of Elemental Evil still has Hommlet, Nulb and the surrounding lands to explore and NPCs to interact with. The exploration element back then was also the World of Greyhawk and Mystara as in effect you were exploring and building the world at the same time. Paizo has copied this approach ad their adventure Paths have been set on Golarion. A great adventure path also has a decent story/plot and ties the adventure to the world. The Serpent Skull AP for example reveals the history of Golarion and the Atzlanti, Skull and Shackles let you be a pirate and is set in The Shackles while Kingmaker lets you explore and settle the River Kingdoms. These are all locations of Golarion the Pathfinder campaign world. The Isle of Dread is officially located around 100 miles south of Karameikos on Mystara which is where a large % of the B series modules is set. With the destruction of FR I switched to Golarion and have used it in AD&D and not just Pathfinder or mined it for information and ideas. The Red Mantis Assassins are now more or less the default assassin on my worlds. A good adventure like The Night Below can be turned into a great adventure like B4 The Lost City or B5 Horror on the Hill or the Kingmaker AP just by how it interacts with the world at large.
The last aspect of a great adventure is the interaction pillar IMHO. This is the roleplaying part and also offers a way to unfold a story. Video games have also been using this idea and I will use Knights of the Old Republic (KoToR) and the Mass Effect series. KoToR is generally regarded as better than KoToR2. Mass Effect 3 was panned because of the way it ended. ME3 did not use Drew Karpyshyn who is an author in the Star Wars expanded universe and he is one of the better ones IMHO. Mass Effect 1 and 2 did. Paizo has done a very good job with tying the interaction pillar into the combat and exploration pillars even in the 3.5 APs such as The Savage Tide and the interaction with the various demon lord in order to defeat Demogorgon. In the Kingmaker AP for example there are NPCs who offer you side quests such as steal a Rocs egg to make an omlette and you are rewarded with a magic item or gold. This ties one of the encounters on the map to the larger story while finding an Elven statue for another NPCs achieves something similar but it also ties the adventure to the history of Golarion so you interact with the NPCs, explore to find the object of your desire and then have to fight a combat (maybe) or sneak to get the required statue.
D&D used to have great adventures, Pathfinder currently has them. To a large extent mechanics do no t matter for me as I will happily play AD&D or Pathfinder (3.75 really) and it is not because of their mechanics those rule systems appeal to me although PF did appeal because it was not 4th ed.. It is because of the adventures, history and feel of them. I prefer the TSR era playstyle and I prefer the Paizo fluff/AP and in an ideal world one should be able to buy a D&D that has quality APs and d20 mechanics but using TSR era play styles and game theory. Some retro clones come close
but they often have created new problems. D&DN has to be better than AD&D mechanically while avoiding the pitfalls of 3rd ed and 4th ed (to complex, feels wrong) and it also has to have quality adventures otherwise I'm going to keep buying TSR era adventure PDFs and Paizo APs.