The horrors of Kindle Format X
While you are reading this, Amazon is silently rolling out Kindle Format X (KFX) to supplant the previous KF8 formatting. You may have noticed the little “Enhanced Typesetting” entry on their Kindle book pages, which, when set to Enabled, indicates that the new format is in place for that particular book. Naturally, the new features will be visible only on devices that actually support the new KFX format, and as in the past, Amazon will deliver a downgraded format to all other devices to match their respective capabilities.
Right now, it’s still all hush-hush with no official information or documentation available, so everything I am pointing out here may still change going forward, but given Amazon’s track record in that respect, this is highly unlikely.
The actual implementation of KFX is being handled on Amazon’s server backend, which means no extra formatting is required in actual eBooks and not tools are currently required, though that is most definitely likely to change because it makes the creation and evaluation of content a tedium at best and a nightmare at worst. See, currently, only a small portion of books actually supports the feature, each of them picked at Amazon’s sole discretion. See, KFX is not something you actually author, as it is rather some conversion that Amazon is doing on their end to tweak the eBooks on their servers before they are delivered. This means that in order to evaluate KFX currently as a developer, not only do you need to have access to a book that Amazon has deemed worthy and that is offered with Enhanced Typogaphy, but every time you make a change you ill also have to upload the new test-version and wait about 24 hours until the new version goes live and you can see how the formatting behaves under the new KFX engine. It is, quite frankly, a nightmare and a tremendous time sink.
The Kindle eBook format could use some improvements, there can be no question although, with KF8, Amazon has brought the format so close to the ePub specs that one really has to wonder why they did not go it all the way and eliminate the majority of problems that plague the platform once and for all.
Four years ago I wrote a blog post called “10 Things Amazon should correct in the Kindle” and in retrospect it is frightening just how little the problems mentioned there have ever been addressed. As Amazon moved forward with the KF8 format, there were plenty of improvements but in many ways the new format was also a step that made things worse, as outlined in my blog post “Amazon introduces new Kindle eBook format and makes a major misstep“. Once again, nothing was done to resolve the serious problems underlying the format. With that in mind, it is clear that Amazon does not care about existing flaws and all the issues we see emerging right now will plague the format for years and device generations to come.
Naturally, I was curious to find out what would await us with the migration to KFX. Boy, do I wish I had never opened that can of worms.
The problem with Amazon’s Kindle platform has always been that it has been designed and maintained by people who, quite evidently, do not understand books. The device and its firmware has been designed by Lab126, a company that specializes in handheld devices, and the name, Lab126, could not be more fitting, because everything about the Kindle has felt like something put together by lab rats, from day one; people with no sense of the real-world application of their devices. I would not be surprised to hear that the engineers at Lab123 have never really read a book on a Kindle or tried to format one, for that matter.
So, what’s the problem with KFX, you wonder? Let’s start with what is, perhaps, the biggest whopper. The Kindle no longer supports non-breaking spaces. This means all those instances where you made sure your text units will remain together and won’t be broken apart, will all be ignored. Line wraps will wreak havoc with your finely crafted “Mr. Smith,” “5 minutes,” “November 5,” “3 p.m.” or the “1964 Mustang,” as they will all potentially be butchered by rampant line breaks, splitting the units into non-sensical individual words. To find this in a device that is designed specifically to display books is like selling a car without a steering wheel.
But the fun doesn’t end there just yet. Amazon has also fiddled with the hyphenation rules, but instead of making them smarter and more real-world-like, they have gone backwards and implemented an algorithm that conjures up memories of 8-bit computers. Imagine, an algorithm that treats every period as a point where lines can wrap. Suddenly you will find that “FM 99.8” becomes a garbled mess where the number 8 ends up on its own line. Forget about scientific texts with decimal notation or website names with a dot-something extension. They all can now be brutally butchered beyond comprehension. What an awesome improvement… what an “enhanced typography.”
Another such issue can be found when it comes to em- and en-dashes. The new KFX implementation is suddenly teaching the Kindle layout engine to do line wraps before the dash, as well as after. If I didn’t know any better, I would almost believe that Amazon wants to rewrite typography rules with botched features such as this. These problems clearly show a lack of understanding of the subject matter and a wholesome disregard for everything relating to book design.
The list goes on, and as we will be exposed to an increasing number of KFX-formatted and rendered books, I have no doubt that more problems will emerge and we are only but scratching the surface. It makes you wonder, where exactly are the “enhanced typography” features that Amazon is touting?
Well, nowhere, really. Text flow has not been improved from what I can tell, and word spacing issues have still not been addressed, while tables still remain unusable. While it can finally hyphenate, in a kind-of sorta way, when justifying text, the Kindle will still only expand word spacing and it will never ever reduce spacing the way you would do in a print book. As I said, the layout engine operates on the level of a Commodore 64 8-bit computer. This is incomprehensible because handheld devices are so powerful these days that highly sophisticated algorithms could be put to use that would properly evaluate line widths and adjustments to them to create visually pleasing layouts. Instead, the Kindle is still simply dumping text on the screen without any regard for typography.
Simply introducing a new typeface and a few ligatures does not make for enhanced typography. The Kindle still suffers endlessly from widowed words, orphaned lines, inaccurate spacing, ignorant kerning and other problems galore.
And then there are the images… For the longest time, Amazon has ignored image transparency in PNG files and in recent years they have given us a completely borked 8-bit PNG implementation that resulted in nothing but black blocks. With KFX we can now celebrate the arrival of their new JPG format, JPG XR. In and of itself, JPG XR is a wonderful development, a useful enhancement to the standard JPG format that results in smaller images with less loss in definition. With this new image support in place, Amazon is now converting all images in an eBook into JPG XR format, including all GIF and PNG images, as well as SVGs. Like they did with the KF8 format, Amazon is also creating dedicated versions of a book for devices of varying capabilities. As a result, they will now convert images to grayscale for delivery to monochrome Kindle devices, once again a step to curb in bandwidth – and thus, reducing the cost of delivery fees for you.
But there is an inherent danger in doing this and given their shoddy track record, I am not entirely sure Amazon or Lab126 have gotten the memo how to properly utilize JPG XR. Normally you would not want to convert a line art image, like a PNG, to a JPG format, but since JPG XR features a mode with lossless compression, this may no longer be true. At this time I am not entirely convinced, however, if the KFX backend is taking this into consideration, or if PNG images are simply converted to regularly compressed JPGs. As more samples emerging over the coming months, I am sure we’ll find out soon enough. Regardless of compression issues, however, at this time it is clear that Amazon is once again completely forgoing all image transparency, completely ignoring the alpha-channel support that the JPG XR format offers. So even in 2015, the use of images in eBooks appears like a clunky, broken relic of – yeah, you guessed it, the 8-bit era.
When you want to create a format that lauds itself with “enhanced typography,” the first thing you need is to have people on board who understand typography, and to Amazon’s shame this has evidently never been the case in the Kindle department, ever. The company and Kindle-developer Lab126 are stumbling from one mistake to the next, making things worse with each incarnation. Rather than finally investing some time and actual experience in the platform to hammer out a “real” eBook format, really useful eBook features, and real eBook-ready firmware, the companies still seem to leave the job to code jockeys.
With each iteration, it gets harder to explain away the ineptitude displayed by the people creating these implementations. The shortcomings of the first-generation Kindle could still be attributed to weak processing power, limited display technology, and the journey into completely uncharted waters, but in today’s market, Kindle users should expect a device that has been honed to perfection, not a tinker toy gadget that continues to miss its marks and its core audience by a mile or two.
If you thought Smashwords’ Meat Grinder was a monstrosity, welcome to Monster X and the abominable new world of Kindle!
Not exactly following the point of this article. Nothing we’ve seen of Amazon suggests they’re listening to consumers in a meaningful way. Also, you’ve suggested there’s no official information available, when in fact hovering over the area in your screenshot reveals a ‘Learn More’ link. (Amazon’s page on Enhanced Typesetting is at http://www.amazon.com/b?ie=UTF8&node=11516960011.)
Finally, you’ve said no official information / documentation is available – where are you getting these notes about non-breaking spaces or decimals? Screenshots / pictures / proof?
This is a personal blog and as such my posts do not need to have a “point.” I can – and do – write about whatever interests me.
The page you are linking to is nothing but marketing blabber. A sales pitch. It is not actual information. What I was referring to is actual developer information that makes it possible to properly develop and format books with specific KFX features in mind.
I’d love to see screenshots, especially of the non-breaking spaces issue. Curious, though, are you using named, numerical or hex entities for special chars?
As readers of my tutorial and book on eBook formatting will know, I recommend the use of nothing but named entities wherever possible.
Meant to add that I’m not so concerned with widows/orphans—a typesetting issue, rather than a typographic one—as it remains an issue with Web technologies. Yes, widows/orphans can be controlled (kinda, sorta) with CSS, but I find it more inelegant than doing nothing at all.
Kerning, though, is another matter. I do think that device makers are on the right track in commissioning fonts that meet the unique requirements of small screen reading. As long as kerning is well handled in the font tables, then it should get better on the devices. I was on the fence about Bookerly. But I now think it’s quite nice for body text. It’s terrible when used as a display font.
Guido is quite right about almost all these horrors, as I’ve already described on my own Publishing Blog (www.newselfpublishing.com/blog). The good news is that I’ve found the current Kindle team is surprisingly willing to listen to such criticism if you can reach them, and it is working hard to correct some of the mistakes pointed out to them. So, the situation will hopefully improve before long. Though, as Guido implies, what they really need is a typographer or two on staff.
As for kerning, it’s ridiculous to think that letter spacing matters when word spacing is so primitive. The only reason we’ve been given the kerning is that it’s built into Webkit, which the Kindle is now built on, while decent justification is not.
[Quido]
> reducing the cost of delivery fees for you
You really believe that? This is about increasing there margins, not mine.
[Aaron]
> the current Kindle team is surprisingly willing to listen
> to such criticism if you can reach them
Yea, so willing, they hide from you. And “listening” is NOT fixing.
> while decent justification is not.
There is no such thing as “decent” justification. While fully justified looks pretty, it’s a nightmare to read; especially on a digital device.
Kindle can’t handle hyphenation, anyway. Not sure why you would want justification?
I don’t follow your logic, Mark. Since Amazon is rolling delivery costs over to the author and takes it our of their part of the royalty, reducing the size of the delivery package does absolutely nothing for their margins.
As for justified text, I think the better question should be, why would you not want it, as it is the only way to properly lay out novels.
Guido, I’m wanting to include a few, say 10, line art images in my next book. They are currently SVGs but after reading the above, am I correct in thinking that it doesn’t matter what format I upload them they will automatically be converted to JPG XRs?
Thanks
That is only true of the book is converted to KFX format, which at this point still only a fraction of books are. I would simply embed them as SVGs if that’s the format you have them in.
The root of the problem is that both EPUB and MOBI/KF8 uses HTML5 as the underlying markup language. The “developers” of HTML5 are pushing it towards social media and games, instead of defining much-needed tags for text-based documents.
So, in defence of Amazon, their use of a flawed markup language is understandable, because HTML(5) is understood by basically any parser of XML-style markup languages. Much better would have been to design a whole new language, but then nobody would be able to read it.
I completely disagree. HTML5 may not be perfect for eBook needs but it contains countless features that would be very useful. Those, however, Amazon did not implement or supports only in some broken fashion that works only on a third of the devices, etc. With KFX they even went another step further by demolishing everything that had been established in terms of typography without any recourse.
So, no, the problem is not HTML5, the problem lies very clearly with Amazon and the fact that Apple has an implementation with iBooks that does not have Amazon’s issues (though, admittedly it introduces its own shortcomings) is further proof of that.