You can’t judge a book by its cover. Unless you’re judging a book’s aesthetics, in which case you can totally judge a book by its cover. It’s actually recommended. It's a vital part of the packaging.
The Three Designs
I’ll start by quickly reiterating the three types of book design.
There’s Consume, Engage, and Cherish. Consume books are essentially gaming textbooks, they hold the information you need and you wants access as easily and quickly as possible. Engage books grab a hold of your mind and enflame your imagination. Cherish books are collectibles or works of beauty that have to be experienced.
They're not mutually exclusive but they're pretty darn close, as the readability and usefulness of Consume books makes it hard to have the expansive flavour of Engage or the physical details of Cherish books.
I personally buy all three. I Consume my copies of the Rules Compendium (3.5 and 4e) and books of rules like Unearthed Arcana. I Engage with my campaign settings and flavour-heavy books like Heroes of the Feywild. I Cherish my copies of Ptolus and Tome of Horrors: Complete. And while I used to Engage or Consume my 2e and 3e books, they've become books I Cherish: as usability drops, nostalgia brings Cherish. While the soon to be released 1e books were very much books to Consume and Engage the re-release is 100% Cherish.
The actual logo-title of the brand is pretty darn important. It needs to be readable from a distance yet distinct, but flexible enough to be featured on a range of products in addition to the TRPG books, such as the comic and board games.
It’s easy to make a bad logo, as shown by the 3e nameplate version, which was designed to match the stylistic tomes of the Core books. Because of the brass framing, it was awkward to add to the spine of books and looked different when divorced from the nameplate, which made it less distinct and diminished the iconic impact. It also didn’t have a universal colour or look, sometimes being shaded and sometimes being a single colour.
The 4th Edition design is pretty solid, having set colours and being flexible enough to be written with “Dungeons” stacked atop “Dragons” or written side-by-side or even abbreviated D&D.
I think it suffers a little for the ampersand’s simplicity (not immediately apparent as either a dragon or an ampersand) and the lack of a distinct "font" for the rest of the lettering: there's nothing unique or eye-catching about the "D"s. Meanwhile, the “logo” of the edition – as seen on the title page of the books – is just the head part of the ampersand. This works but isn’t distinct or tied to the brand. When you look at the logo you don’t see “D&D” you just see something vaguely resembling a dragon head.
D&D could probably use a better logo: more stylized Ds flanking a simple yet draconic ampersand. Something they can slap onto the side of books and boxes as well as placing on websites, apps, and MB icons. Something noticeable and eye-catching. Dungeons & Dragons is THE D&D and it should embrace this, and use the name recognition. "D&D" simple and small; most companies or brands would kill to have such a simple abbreviation. The font and design of the “–ungeons” and “–ragons” is much less important.
One thing to consider is also the flexibility of the logo. I’ll direct you to the new logo of DC Comics. The base logo is “okay” at best. There’s a semi-obscured “D” and the half-visible “C” and the suggestion of a flipping page. Passable. But the strength is the mutability of the “C”, which can change to reflect the content or focus of the book. There's a yellow "C" spattered with blood for "Watchmen", a glowing green "C" for Green Lantern, a darker "C" for the horror books.
D&D does something similar already, with how the framing of the logo changes for the various campaign settings (along with the design of the page number art). It’d be interesting to make the "D"s and/or ampersand mutable depending on the Campaign setting or theme of the book.
Returning now to the three types of book.
Personally, I’m most fond of Engage, as demonstrated by the 3e Core books. I love the distinct covers, the in-world design. (I would have loved a textured version. I thought the otherwise lovely leather-bound gilded collector’s editions missed the mark for that reason.) In comparison the 4e books seem stark and plain; on a page without art they always seem like a low-cost 3PP book than a high quality WotC product.
I don’t want just a rule book, I want something that fuels my imagination like a can of gas on a mental fire. I play board games for simple rules and quick play, I play role-playing games for stories and creativity. I need more!
However, full-page colouring was likely significantly more expensive to print, making the much more simple 4e books much more cost effective. Especially for the books designed to be the principle money makers.
The 4e books are much more effective as reference books, books to be consulted and containing rules to be found. They're very sleek: the colour coding of the books was a solid idea (blue for players and red for DMs – a reversal of the 3e cover colours – with green for monsters), with the colour banding on the cover matching that of the headers inside. They’re not bad books, they just don’t grab me the same way.
However, there’s a bit of a false dilemma (or false dichotomy) in asking for a choice between the three. As has been repeatedly shown by White Wolf’s products, it’s quite possible to have a mix page formatting, books that both Engage and Consume.
It’d be interesting to design the 5th Edition books with in-character or in-world sections; sidebars and pages with a much more stylistic design. Fluff pages or sections could have the 3e parchment background while harder rule text could have more simple and clean 4e design.
For example, imagine a races chapter, where one page is clean and white with a half-page illustration of what the race looks like and the other half game rules, with the opposite page having a parchment design and flavour on the race with their worldview and personality.
The two different types of page might have different fonts and headers, making them easy to distinguish; you know not to look for a rule on one type of page or the other. It would even help Consume, as a quick glance will identify the contents of a page as rules or flavour.
It’d be interesting to maintain the colour coding of 4e, but perhaps taking it a step farther and applying it to different sections of a book. PC sections would have certain chapter and header colouration while DM sections of the same would look entirely different. This might become more pertinent as WotC releases fewer books that have to satisfy both players and DMs.
One thing I’d like to see more of in D&D are referential sidebars. I don’t mean sidebars as the space-filling asides, but scholarly sidebars at the edges of the page that refer the reader to other places in the text. For an example of how this is done I refer you to the new Marvel Heroic Roleplaying Game by MWP (there’s an example of play here that uses them), or the excellent Ptolus product, where sidebars are used continually to cross-reference NPCs and locations.
Imagine looking at a grapple-heavy build of the monk and having a sidebar directing you to the appropriate page of the Combat section of the PHB. Or the section talking about a paladin being able to summon a warhorse directing you to the mounted combat section. When a class reccomends certain races, it could refer to the exact page number. It's like a full-book index.
Even if not universal across the products, sidebars are a must for adventures, especially ones that are more than simple dungeon crawls. Running published adventures can be problematic, as you can never find the NPC or plot or note when you need it. Gabmore Abby would be ten-times easier to use if the books and sections were all cross referenced.
On a similar note, D&D and other gaming books live or die by the quality of their index. The Essentials books made up for this with solid tables of contents, but WotC has failed to wow with the indexes for a long time. You cannot skimp on the index of a gaming product, because every second you spend lost and looking for a rule or fact is a second a table of six people are not doing anything fun and five people are bored.
Look the Part
D&D is the tabletop role-playing game. It's important for it to look like the best. This new edition should try and be timeless and not reflect topical preferences. It's too easy to just make it look like "not 4e" or embrace something currently popular, say miming the design and look of HBO's Game of Thrones.
When they discussed the look if 4e, one of the goals was showing more of the world, focusing on more than characters and heroes. I think that 5e really needs to focus on the worlds, showing realistic historical armour and dungeon punk and heroic fantasy (which might let them reuse earlier bits of art when necessary).
It'd also be tempting to go for the 1e homage, with the DMG having an updated picture of heroes fighting a gargantuan efreet by the City of Brass. While cool, I'm not sure I want the core books to ride on nostalgia. I want something my kids can look at as the retro covers. I love the cover of the 4e DMG, it is an awesome piece that is just as cool as the 1e stuff and even more iconic and memorable. Who doesn't love the little image of the heroes from the PHB in the orb?
They have their work cut out for them.