Time to rethink Kindle content generation
The announcement of the next generation of Amazon’s Kindle has set the eBook world abuzz once again. Not only are the new models more attractive than their predecessors, but they also expand the market in new, untapped territories. For authors, this is great news, of course, but often, where there’s light there’s also darkness.
In this case, the cloud on the horizon lies in the technical specs of these new devices. With a bit of worry I have observed over the past year or two that the eBook market is becoming more and more fragmented. In a very bad way, it reminds me of the mobile game space I have also been working in, where, at times, it was necessary for us to build up to 200 different versions of the same app to make sure it properly supports all the handsets in the market.
While the eBook market is not nearly as bad, of course, there is an increasing trend of changes – or call them features and improvements – that can work like sand in a ball bearing.
Fortunately we have to contend with only two generic eBook formats at this time – MOBI/KF8 and EPUB – and it is easy enough to build eBooks for both formats from the same sources.
However, since the inception of the iPad, problems have cropped up that force eBook publishers and formatters to think very hard about what it is they want to do and how to achieve the desired effect. Fixed-layout books and their particular quirks, and the lack of a general standard to create them, is just one of the issues publishers have to tackle these days, and it is exacerbated by the fact that even within the Kindle line of products, it is not possible to really create specialized builds for each platform. A fixed-format Kindle Fire eBook will inevitably make its way onto a regular Kindle – where it doesn’t belong – because Amazon does not give publishers the possibility to create specialized builds. As a result Kindle owners will look at a book that is horribly mangled and probably unreadable, while it looks mesmerizing on a Kindle Fire. I am not sure in whose best interest that is, but that’s the way Amazon does it.
The reason I am writing about this is because according to Amazon, the new Kindle Paperwhite line of models offers 65% more pixels. In plain English, it means it has a higher resolution than previous Kindles. That is really great news in regards to sharpness of the text, of course, but from a formatting standpoint it causes certain problems. An image that was perfectly sized for the Kindle’s 600-pixel resolution to date, will suddenly appear much, much smaller on the page. In many instances, this will not be overly dramatic, but if you use images deliberately as a design element, it will force you to rethink how you approach images in eBooks. Just image how tiny the image will look like when it’s being displayed on the new Kindle Fire HD with a resolution that is three times as wide as that of the original Kindle.
In the past I have sized images to suit the 600 pixel screen. It helped keep the file size in bay – why bulk up a book’s footprint for no apparent reason, especially since the publisher is being charged for the delivery of the book based on the size of the file. This approach may no longer work, however, if you want high quality images across the board.
I’ve been therefore rethinking my strategy and going forward I am sizing images to a higher resolution and then determine their on-screen size, using scaling through my CSS style sheet. This allows me to make sure the image will always appear the same on the display, without degrading it on higher resolution screens. If anything, it may degrade the quality scaling images down to the older Kindle models.
If Amazon offered platform specific builds for their line of Kindles, this would not be a problem, but things being what they are, a one-size-fits-all approach is necessary, and hopefully, this will do the job.
In many ways, I wish that Amazon would make me part of their Kindle design team or at least would allow me to work with them. After all, I’ve had over 35 years of experience as a software engineer in arenas that were a whole lot more complex than an eBook reader.
Many of you may remember my post 10 Things Amazon should correct in the Kindle from a year ago, and it is rather disheartening to see that virtually none of these issues have been addressed. In fact, if you look closely, not a single one of the issues has been addressed to date. While I have not seen a Kindle Paperwhite at this time, I doubt there will be many changes in the firmware that would address these issues. It seems to be more of a change in terms of the form factor and a hardware upgrade than a rework of the actual reader implementation – but I could be wrong, of course.
To me as a software engineer, author, publisher and professional eBook formatter, the omissions are truly painful to behold. amazon has done great things for books, by truly establishing eBooks as a reading medium, making it the new mainstream standard, all the while opening the doors for authors to publish their own work. All great achievements and I honestly doff my hat to Amazon for their incredible foresight and the vision they had during the past three years.
That, however, makes the technical shortsightedness all the more prevalent. All of the issues I raised before have been around since day one, and clearly someone within Amazon should have championed their correction. It did not happen. Not even when people like myself and others have called them out.
Amazon has never been a software or hardware developer before the Kindle and as such it was to be expected that there would be hiccups in the product and the delivery. No big deal. However, the market has reached such a maturity, that glitches like inconsistent text justification, the lack of transparency in PNG images and other omissions become glaring issues that should have been resolved two years ago.
The Kindle has to mature and it has to mature with foresight or we are gong down the road of mobile games, where you need 200 individual builds of an app. There are great developers out there who would have been happy to assist Amazon in their objective, but instead of embracing them, Amazon has often shunted them.
A command-line MOBIGEN program is just not the same as the luxury you get out of a program like Calibre. Amazon should have long looked into creating high quality content creation tools that help authors to increase the quality of their output. Too many self-published books are still created with an MS Word export or an InDesign plug-in that cause more problems than they solve.
Amazon should also have long started to put in place platform-specific delivery of eBooks, along with was for authors to properly set up books for each of these platforms.
Amazon should also have expanded their eBook format in ways that are truly practical without having to jump through hoops. The introduction of KF8 was a horrid debacle to say the least. Confusing authors and readers alike, the implementation is not what it should be – many things could have been implemented much more efficiently, making it easier for formatters to prepare the eBooks while also giving them a certain level of control over the appearance of their content. If you’ve ever tried to take a look at a black and white line-art image in the “Night” setting of your Kindle, you know what I mean, and the whole image sizing issue puts the dot on the i, I think.
I don’t want to harp on this unnecessarily excessively, but it also appears as if Amazon has long forgotten its pledge to bring KF8 support to the Kindle 3 generation of devices. As far as I can tell, that has never happened either, and yet, the train of model innovation moves on…
With all the new glitz and glamour that accompanies every new Kindle model, for publishers, each new generation brings with it a new set of challenges. It’s not necessarily a bad thing, but as I said, I wish Amazon would allow me to work with them to help them make these transitions as easy as possible, at least from a content creation standpoint. If anyone from Amazon is reading this, you know where to find me…
Here’s to Amazon calling you sooner rather than later!!! Don’t think it will happen, though (sorry) – not because they shouldn’t, but because they couldn’t. Why? Because they have nothing to win by getting their formatting act together. And a lot to lose.
They have nothing to win, IMHO, because their content creators seem to have accepted having to format and re-format endlessly. It’s not as if they have to please anyone to keep the content coming in..! They are offering a free pubbing service, and if it’s a pain in the *** to use, well, nobody is FORCING you to self-pub on Amazon.
As to losing – well, like all other e-reader producers, they like the “walled garden” that keeps the consumers with them. Any cross-gadget format would make the users think. “Why don’t all the ebook pubbers agree on a common format that will just work on ANY gadget?” And, sooner rather than later, someone would have an epiphany: “Uh, waitaminnit, guys – we ALREADY have that. Had it for coupla decades, really. It’s called html.”
And then millions of users would look at their needlessly dedicated hardware, with its pointlessly dedicated software, and … well, I’m not sure they’d smile.
Amazon has built a killer webshop, and their Cloud reader is both a step in the right direction and one in the wrong direction. Frees the user from the dedicated gizmo, and tries to keep the garden wall up via software. In my nerdish dreams, Bezos decides to strike a blow for all the consumers who’ve made him rich, and announces: “No more Kindle, hard or soft. Our Cloud reader is now any browser you like, your book collection is a stack of html files we’ll be happy to cloud-store for you, along with bookmarks and notations, etc. – and if you want local-store your stuff, just do so. It’s all just html. And anyone who wants to create books for promo and sale through Amazon can have their pick of the ton of html editors, WYSIWYG or otherwise, that you’ll find on the web.”
I think he’d make a sh*tload of money that way – the entire emerging world of authors and readers are entering the web via smartphone – they have neither the time nor the hardware-/software-purchasing money for Calibre, Sigil or a new Kindle every year.
Oh, and credit where it’s due: your html formatting guide is the coolest thing ever to be written about Kindle formatting – and that’s speaking as an IT professional. Take a bow 😉
The Kindle is not at all a walled garden, and neither are any of the other eBook readers. The Nook Tablet, perhaps because Barnes&Noble doesn’t tell anyone how to create fixed-layout books, but that’s as far as it goes. The platforms are open for anyone to put content on, even outside of their official channels.
Also, you may not be aware of this, but all these eBook formats are HTML based, so what you are asking for is already here. HTML has most of the features needed to do proper eBooks, but it also needs extensions to handle things the standard HTML requirements do not cover. It is there, where foresight is needed in planning for the future as hardware matures and the market grows. and it here, where Amazon and other eBook manufacturers have completely failed so far. They are making decisions for the short term, without looking forward and without understanding the implications these short-sighted decisions will have in the long run.
Hmm … proprietary formats that force us into needless special formatting for specialized readers (hw AND sw) sounds like walled gardens to me – especially when a cross-all-platforms format already exists, and that format (html) is the basis for all those specialized formats, anyway 😉 Other than that, I completely agree.
Neither MOBI nor EPUB are proprietary formats and you seem to misunderstand the meaning of the term “walled garden.” It typically refers to a situation where you can not see in or get in, where there is a controlling instance that dictates in what manner business is being conducted and limits it to a small elite group. Barnes&Noble is running a walled garden operation with their fixed layout books on tables, as I mentioned before. But that’s going to end eventually, too.
This is absolutely not the case in eBooks on the Kindle. Anyone can put books on any of the ebook readers and devices. It is just a certain feature-support and platform idiosyncrasies that requires extra work, which is something completely different. Whether you want to do that extra amount of work or not is up to you.
In general there’s nothing wrong with it and I would hardly criticize it, if it weren’t for the fact, that some of it could have been prevented with more foresight, just as problems that are already looming down the road, could have been prevented with some foresight and, perhaps, some practical expertise in the field.
Hmm … again 😉 According to good ol’ Wikipedia, “More generally, a “walled garden” refers to a closed or exclusive set of information services provided for users.” Which, IMHO, could easily describe a specialized format without which your stuff cannot be read/consumed w/out a specialized device, sold by a specific company. AFAIK, you can’t import self-made (or -converted) Kindle books into the free Kindle Cloud reader. I’ll be more than happy to be wrong on this point, though!
Either way, my chief gripe is the horrid waste of productive time in doing the special formatting at all, when you could create and consume the content just as satisfyingly using the existing cross-all-gizmos format. I’m not an html expert, but I know you are, so I am sure you’re right when you say a few supplementary bells and whistles on html will be necessary for complete ebook formatting (apart from forced page break, which already exists, kinda-sorta, I can’t think of any, though..?).
But that only emphasizes my point – or gripe, if you will: designing and agreeing on those few bells and whistles as additions to html would have been a damn sight more rational – except in the short-term business-interest sense. In the long term, as mentioned above, I still think that the company that returns to good ol’ html will make even more money 😉
The Kindle Keyboard got KF8 support recently through Software Update 3.4. But I haven’t tested it yet.
I didn’t even know that, and I do not recall Amazon ever sending out any information about that. What good are firmware updates when no one ever gets to see them?
Hi Guido, can you not size an image as a percentage of the page width, rather than by pixels? If so, that would allow consistency across different resolution screens.
You can, but it may not always be what you want. If you say, for example you want the image to be 50% of the screen. On a Kindle it would fine, half the screen and nicely proportioned. If, however, you look at the same page on a desktop Kindle reader on the Mac with 1920 pixels, for example, the image would be huge and in absolutely no relationship to the text. Therefore, I feel it is best to scale the image in relation to the text size, which is why I currently use ems to scale the image.
Guido you say you use ems to scale the image. Can you explain that more thoroughly. And can you give us an example of the code your use to do that?
Thanks. I now format my books using html after learning about it through your series. I’m very happy with the results but it wasn’t easy getting there.
Guido, how can I learn more about fixed-format epubs and how to produce them. Although it sees a waste to ruin an epub by taking away its flexibility (what makes the format so beautiful!) I can understand why people may want it for textbooks, kids books, or coffee table books. (If you can have coffee table ebooks :))
As far as the walled garden goes, it seems that Amazon has been open in many ways but still refuses to open some aspects of their Kindle ecosystem. The desktop app won’t sync any personal documents, and the cloud reader can’t even load them, but this seems to me not to be because Amazon can’t do it but rather that they don’t see enough benefit in adding the feature. Desktop and browser readers aren’t their target market.
Great article Guido! Thanks again for all you teach us!