Thursday, November 13, 2014

BISG Field Guide to Fixed Layout Update Errors

The Book Industry Study Group (BISG) this week released a revised edition of their Field Guide to Fixed Layout for Ebooks (Version 1.2), but it is still fraught with errors not fixed from the last edition, as well as adding new confusion via poor editing and worse research. You can download it for free from the link above (once you register for an account), which is worth doing, as it does contain some useful information in among the rest; you just have to know which is which.

A new section has been added, for example, called "Authoring Tools & Limitations" which, since Tools is plural, would presumably purport to outline a number of the primary applications currently in use for the production of fixed layout ebooks, such as iBooks Author, Jutoh, CircularFlo, MadeFire, or at the very least the two free programs offered by Amazon, Kindle Comic Creator and the Kindle Kids Book Creator. But in fact, the only tool actually discussed is InDesign CC's ePub export feature. No other program is discussed, or even mentioned. That said, the extensive list of InDesign's limitations is well worth a read, since it details exactly the reasons why I do not use InDesign for ebook production, nor recommend it, at least for the creation of Kindle fixed layout source files.

And on the topic of KF8, the following are just the primary errors to be found in the section on the Kindle fixed layout format...

1.) In the information regarding the Orientation Lock options (page 26, item #3), it states that none is a valid option, along with portrait and landscape, but then later contradicts this with a paragraph stating (incorrectly) that "you cannot have a book that can be viewed in both portrait and landscape modes," which of course, is exactly what the none value is for. Moreover, the statement that "Orientation is a required value" is only partly correct, since it is optional in comics, and only required for children's ebooks, but in either case can be set to none (which is, in fact, the default value). Granted, auto-orientation does not function on many Kindle devices, but it does on some, and it is the none value that allows for this.

2.) Item #4 states incorrectly that the values of the "Book Type" entity "can only be set to children or comic." Since this entity is optional, it can also be left out (or set to none), in either case of which the effects of the fixed layout functions vary greatly from those of either of the other two book-types. See my Fixed Layout Functionality Chart for specific details on this, if you have not already. There is good reason not to include either book-type.

3.) The last sentence on the bottom of page 26 cuts off mid-steam at the top of the following page, but seems to suggest that two-page spreads are possible only with the comic book-type. This is incorrect, since it works on some devices and not on others for both children's and comics, as well as those containing no book-type value (or a value of none), as can be seen in item #2 in the Fixed Layout Functionality Chart referenced above. However, the comic book-type does support two-page spreads more widely than either of the other two options, though not consistently. While I am sure it is beyond the scope of the Field Guide to provide these specifics, such universal statements as those given tend to be misleading, and are therefore less than helpful.

4.) The Region Magnification entity is irrelevant, since the value is set automatically by Kindlegen based on the presence or absence of region mag code in the converted file itself. The value will be set to the correct value regardless of what the content creator enters in the OPF - even, in fact, if no value or entity is added to the metadata at all. Therefore, there is little point in adding it manually, one way or the other.

5.) The statement that "KF8 fixed-layout format lacks pan and zoom" is incorrect. No, you cannot pinch and zoom pop-up regions, but you can pinch and zoom background images inserted using the <img src= > embedding method after double tapping the image (while those referenced via CSS div id's are locked). This is shown by Amazon's own Comics Sample file, which uses the unlocked insertion method almost exclusively. Moreover, pinch and zoom is a fundamental feature of the Virtual Panels function for KF8 comics that don't include any custom magnification regions: the pinch and zoom is automatically applied by Kindlegen without any active effort on the content creator's part. Refer to Item #4, Notes 6 & 7, on the Fixed Layout Functionality Chart for specific details on this.

This is just a sample of the errors and misleading information found in BISG's Field Guide. For this publication to be truly useful each of the sections and sub-sections provided needs to be greatly expanded, with detailed discussions of each specific topic, and the particular details pertaining to each platform and format, particularly if they actually expect readers to donate up to $50, as prompted by the "Pay What You Want" radio buttons on the download page. I'm not sure who in the publishing world would feel satisfied paying that much for so little rudimentary information when much more detailed data is available on blogs such as this for free. But it's your money, you can spend it as you like. I recommend the free option.

Wednesday, October 29, 2014

Kindle Publishing Guidelines 2014.2

Amazon has recently issued a revised edition of the Kindle Publishing Guidelines, bringing it to Version 2014.2. Although the changes are few, some are fairly significant, so I thought I'd post a quick rundown here.

According to the Revision Notes, there are five updated sections, plus one new addition:

I'll point out the changes to each section one by one.

• Updated 3.1.1 Text Guideline #1: Body Text Must Use Defaults

The first bullet point given heretofore in this this section has been removed:
• Body text must not have a forced alignment (such as left aligned or justified).
This section applies to reflowable ebooks, and the removal of this line presumably implies that forced alignments are now allowed, otherwise the prohibition would not have been removed. Forced alignments have actually been accepted previously, regardless of the restriction, but the rule has only now been officially expunged.

(Update: Aaron Shepherd points out that on page 69, in Section 9 on Kindle Best Practices, item #3 still states that text alignment should not be forced. But this is more than likely yet another instance of Amazon's inconsistent editorial revisions. Note that Section 3.1.6 gives specific instructions on how to force a left alignment for text!)

• Updated 3.2.2 Cover Image Guideline #2: Internal Content Cover Image Is Mandatory

Two sections of the previously existing code have been underlined, with the addition of the parenthetic statement that the
 (underlined elements are mandatory)
These two elements are contained in the OPF cover references, the first being the more recently added EPUB3 "properties" attribute for the cover image entry in the manifest (the new underlined elements have been highlighted below for the sake of clarity, although of course they're not in the Guidelines):
<item id="cimage" media-type="image/jpeg" href="other_cover.jpg" properties="cover-image"/>
The second is the standard Kindle metadata entry that's been used as long as mobi files have been in existence (so far as I recall, at any rate):
<meta name="cover" content="my-cover-image" />
This latter method has always been required in Kindle ebooks, while the former has been optional until now, since its additional a year or so ago. Note, however, that it is now a required element of the manifest entry for the cover image, if the EPUB3 cover reference method is employed. You only need enter one or the other, not both.

• Updated 3.2.3 Cover Image Guideline #3: Internal Cover Must Not Appear Twice

In the immediately following section a huge chunk of text has been removed, all of which provided for the one previously allowed exception in which an HTML cover might be included in a Kindle ebook along with the required cover image itself, as referenced above.

This removed section of text provided for a spine entry for the cover "page" using the mandatory linear="no" attribute (which has never worked correctly), and the epub:type="cover" Landmarks element or type="cover" Guide item attribute. All of this has now been rescinded, and the only remaining content in this section is explicit statement not to add a cover image in any other way than by simply referencing the cover image itself directly in the OPF.

For those of you who have read this blog awhile, or have studied my tutorials or formatting guidebook on the subject, you will know that I have long advocated against adding an HTML cover image, for the very reason that Amazon themselves have stated in this section of the Guidelines, which is that it causes the cover image to appear twice in the reader (and not might as the Guidelines state, but always, in every case on every Kindle device and app).

• Updated 3.6.1 Image Guideline #1: Use Supported Input Formats

A single line has been added to this section, which is clear enough:
Use RGB as the color profile when saving your files. Kindle does not support CMYK or sRGB.
Obviously the KDP ingestion system has been having issues lately with images employing incompatible color modes. This provides the necessary clarification to correct the issue, so take heed. Do not attempt to use a CMYK print source PDF for your ebook upload!

• Updated 3.6.2 Image Guideline #2: KindleGen Performs Automatic Image Conversions

In this section a single word has been changed, which makes a world of difference:
The maximum size of an epub is 650 MB.
has been changed to:
The maximum size of a mobi is 650 MB.
Uh, yeah, that would make a big difference. This means that Kindlegen (and by extension KDP) can input epub source files larger than 650 MB (as I have already pointed out as functionally possible, both here and in the latest edition of my book), but that the resulting mobi file cannot be larger than that size, giving an upper file size limit to publishable Kindle files. Whether or not Kindlegen will, in fact, actually produce a mobi file bigger than that is unknown, as I've never attempted to produce one. But my guess is this is more a KDP restriction than an actual production limit.

• Added 3.10.9 HTML Guideline #9: File References Must Match Case and Spelling of Source

This is an entirely new section, which again apparently addresses some errant file production practices:
Per WC3 HTML standards, all file references (fonts, images, etc.) must match the case and spelling of the name of the source file exactly.
Again, the statement is clear enough that it requires no elaboration, except to say that it's unfortunate a statement such as this even needs to be made. But I suppose that's bound to happen when non-professionals attempt to produce professional level work, without first taking the time to learn the craft.

All in all this revision to the Guidelines is mostly for the sake of such technical clarifications as this, but the few alterations made to the standard rules are each important in their own way, and can in fact be quite significant in many cases.

Saturday, October 11, 2014

Why I Don't Use eBook Creation Apps

Why I don't use ebook creation apps
(but you might want to anyway) 

 Many years ago I built my first website, using Dreamweaver. It was Version 8, I believe, back when it was still by Macromedia. The website was very basic, the program very complex. It suited my needs at the time, and had plenty of potential for expansion...if I wanted to learn how to code a complex website, that is.
I didn't.
So instead, when it came time to add a store with shopping options for buying print and ebooks to my site I switched to Yahoo SiteBuilder, since Yahoo were the top web hosting service at the time, and offered a simple add-on shopping cart package (for a fee, of course). But since Yahoo used their own proprietary code for web page layout and functionality, I had to rebuild the site from scratch, being unable to import the current site and build on that. However, after much tedious effort (not to mention learning an entirely new software interface) it resulted in a nice website that filled my current needs...for a while.
But web technology moved on and Yahoo did not. Eventually I wanted to upgrade my site to include some of the nifty new features made available by HTML5 and CSS3. But that could not be done in Yahoo.
So I moved my site to another 3rd party web platform...and built it all again from scratch, since Yahoo's super secret code matrix could not be transferred to another platform.
That new 3rd party service went out of business within a year (Yahoo has fared only slightly better treading water).
So once again I started over and rebuilt the site from scratch (that's four for those keeping track). This time I decided to use the popular "open" platform Wordpress, with its highly customizable theme-based structure, thinking it would be more "universal," since there were seemingly endless theme and expansion packages available for it.
But Wordpress proved to be slow and balky, crashing frequently and losing data with nearly every update. And because all but the basic layout functions are handled by adding third party plug-ins for each new feature, these tended to conflict with one other more often than not (not being tested for compatibility with anything but the base platform), causing features or entire sections of the site to malfunction (or not function at all), and crashing the site when any one of them were updated, even locking it up in a perpetual feedback loop on more than one occasion - a cycle which could only be broken by deleting one or more of the expansions, which naturally took all of my data with it.
Moreover, due to the very nature of this "plug-in" methodology, the site was once again not exportable to any other platform. So yet again I was faced with the seemingly inevitable task of rebuilding my site from scratch.
So after building the same site five times, I am now back to where I started, using Dreamweaver to build all my site content with universally recognized web code based on standard HTML and CSS. It is infinitely expandable, limited only by my time and willingness to learn, and can be transferred from one hosting service to another as I see fit. And it's the best site I've built yet. More importantly, the underlying code can be read and edited using any standard text editor. I can fix it if it breaks, and add new features as I learn to implement new code. If I see something I like on another website I can view the source code in my browser and see how it was done. I'm not limited by what the software can do, but what I can do.
You might be asking yourself at this point what this has to do with ebooks.
An ebook is, in essence, simply a portable website, designed as fixed or responsive page layouts, and based on a subset of the very same HTML and CSS that websites use. In general, if you know how to make a basic web page you can make an ebook too. You just need to learn the specific bits of code that makes the ebook work, and the rest is left to your imagination.
And best of all, the only thing you really need is a text editor.
The very same issues that plagued my web building experience for so many years also apply to building ebooks, so take heed. Programs that build your ebook for you do so by using their own proprietary code that often can't be understood by mere mortals such as we (unless you have the patience of a saint, or the wisdom of a god, which I do not). For example, they change the names of all your input source files, so that rather than having page23overlay2.jpg, you might find img000172.jpg instead, in your new HTML page called split0000159.xhtml. Good luck finding the correct file for page 23. Or the images it contains, if you happen to want to change one.
This is made all the worse by the fact that all of your carefully labelled CSS entries will have been changed in just the same manner, so that your styling instructions for #page23panel1 is now called data-app-amzn-ke-created-style0126, or some such nonsense. And more than likely all of your neatly organized HTML will be run together in an endless stream of code. Needless to say, this is not helpful in the least.
Unless you're a machine. And a very specific machine at that.
This applies not just to Amazon's proprietary Comic and Kids ebook creator tools, but also in varying degrees to iBooks Author, InDesign's ebook export function, and ebook editors such as Calibre and Sigil. The code they create can only be effectively read by them, thus locking you into using that same platform for all future updates to that file, until such time as their usefulness runs out, they become incompatible with your now-outdated file (or operating system) after an update, another software offers better features, or the company goes out of business.
Then you'll face the same dilemma I did building websites.
You'll have to build your ebooks all over.
From scratch.

For more in-depth reviews of Amazon's Kindle Comic Creator and the newer Kindle Kids Book Creator (for those still intent on using them), follow those links to my posts on the subject. But don't say I didn't warn you.

Wednesday, September 3, 2014

Kindle Kids' Book Creator - An Analysis

Amazon today released something of a companion to their Kindle Comic Creator application that, like its predecessor, allows for relatively easy graphic layout of Kindle illustrated ebooks, this time geared toward producing children's content rather than comics. Like Kindle Comic Creator (KCC), the new Kindle Kids' Book Creator (or KBC for short) presents a very stripped down interface with a rather limited array of tools. However, this is somewhat deceptive, since users also have access (in both programs) to direct HTML/CSS editing within the application, which makes it useful both for those with some coding skills and those who only want a basic drag-and-drop style interface. For those without, however, the options for styling are somewhat slim, though they should suffice for most cases.

A companion User's Guide is available from the KKBC home page, which details in the usual sketchy technical outlines how to use the program, so I won't go through that here, but suffice it to say that the user guide isn't really necessary, since there's very little that you'll need to learn to use this thing, and most of it is fairly self-explanatory from a cursory glance at the menu bars and buttons; a half dozen random clicks will teach you all you need to know in fifteen minutes.

I ran a quick and dirty test to see how the program fared, and for the most part I was pleasantly surprised, although not without a healthy handful of caveats. Foremost of these right from the get-go is the lack of support for epub as an input source: KBC only accepts PDFs and image files, which means that if you have a nice layout to export from InDesign your only option is to make a PDF and start over from there. You can import pages with text embedded and create text pop-ups to cover them up, or import image-only pages and create both the base text and magnified text from scratch, and then move it around and size it all to fit. If you cut and paste text from another document any styles applied in the original will not transfer, since the underlying CSS will not be copied as well, but any tags you've added to the code still will be in the HTML, so you can manually add the CSS rules to the KBC file easily enough. This, of course, implies the need to delve into the underlying code, which somewhat defeats the point of using a program such as this. But that's the last we'll see of that necessity if you actually want to add some style to your work.

Unfortunately, even though you can add text as a separate layer in KBC, the "childrens" book-type that is automatically added to the output OPF renders the text inactive in the Kindle readers, so that dictionaries, highlights and word search will not work, rendering it essentially the same as text embedded in the image. This is why I discourage the use of a book-type value in fixed layout files as a general rule, but that's a choice each content creator must make for themselves. You can, however, delete the book-type entry from the OPF manually and all the text will then be active. Why Amazon has crippled this ability in children's books and comics is beyond me.

If you create the base text from scratch KBC will automatically add it to the pop-up at 150% magnification (if you add a pop-up, that is, since you don't have to), but you can "unlink" the pop-up from the base text and add any other content to the magnified region that you want. It will still be magnified at 150% of the base text, however, so if you simply want to have the base text change you'll have to learn to do some custom coding in the HTML/CSS editors to make it work (for example, if you want to produce a bi-lingual ebook, or use the pop-up like a question/answer flashcard). Unfortunately, KBC's behind-the-scenes code is a little balky, so unless you already know your way around a KF8 fixed layout file archive pretty well you'll be looking at a lot of gibberish, and even for those of us who do it's pretty messy in there. Also, to create clever tricks like using mag regions to replace one image with another isn't something that can easily by done in KBC. If you're wanting to add that kind of interactivity into your project, KBC is not the tool for you.

Other detracting factors against using KBC for anything but very simple projects are its lack of an Undo button (!), so you better be sure you want to make a change before you do, because there's no way to get it back once it is gone. You can "re-link" a base text box to a pop-up, but this deletes any other custom changes that you've made to the pop-up, replacing them with the default text, and that change cannot be undone, so whatever you had created will be lost. This is a horrendous oversight on the programmers' part, and truly unforgivable in this day and age.

Even though you import images of a given size (that you have no doubt chosen for a reason), KBC will automatically re-size them to a seemingly random dimension of its own choosing, over which you have no control. In my sample test file, for example, all my images were 1118 by 1788 in size. This was chosen as the largest size for which I could achieve the highest quality setting in Photoshop and still come in at just under the "standard resolution" image file size limits the Kindle has in place for older devices (which KBC apparently does not support - only the Fire devices are listed as supported, and available for previewing). At any rate, KBC arbitrarily assigned a landscape layout page size of 1920 wide by 1535 high for my test project, making each half page image only 959 by 1535 rather than 1118 by 1788. Curiously, however, the auto-generated "scaled-images" folder contained both the original, unchanged images, as well as "thumbnail" versions scaled to 625x1000 pixels and compressed to down around 100 kb in size, rendering them utterly atrocious in quality. Which ones the reader will see is anyone's guess, but since KBC files are apparently not compatible with the eInk devices (even though those devices do support fixed layout KF8 files) my guess is that the larger ones will be delivered to the Fire devices and the smaller ones to Android and iOS apps for phones. But I'm only guessing there, so don't quote me.

You can add single pages in portrait or landscape orientation, or two-page spreads in landscape using either two image stitched together or one image spanned across both pages. This is very nice feature made very easy during the image import stage. However, auto-orientation is unfortunately not an option here; you must choose one or the other. I should also mention that you can add or create additional pages at any point in the process, and insert them anywhere in the page sequence, and even re-sequence pages already added simply by dragging them to a new location. You can also delete pages, but once you do they are gone for good along with any content you included in them, and again this cannot be undone.

You cannot open mobi files that were not created using KBC, even if they're KF8 format children's book-type files. So KBC can import a PDF but not it's own native KF8 format. Go figure.

Above is a snapshot of the formatting toolbar available in KBC, and as you can see the options are somewhat limited. The "Add Pop-Up" button creates a mag region with no underlying text, while the "Add Text option creates both at one go. Just add your text to the base layer text box and it is automagically magnified at 150% in the pop-up box, both of which can be moved around and re-sized to your liking. You can embed custom fonts easily enough using an Add Font menu option, with any added fonts appearing in the drop down menu here. You can also change their size and color very easily here, which is a plus. There is also the slightly unexpected line height and character spacing options to help you make your text fill up the page just right, but be aware that these apply to all text in a single given text box and no others, so if you want all the text throughout the book to look the same you'll once again need to dive into the code and create a custom CSS rule for your base text. It's also not possible, of course, to only change the character spacing for just one line without embedding that line in a separate text box, thereby breaking any pop-up windows into pieces also.

Other than that you have the basic bold, italic, underline, and four standard text alignment options to choose from, and nothing else. There is no way to wrap text around images as in iBooks Author, or even to automatically align text boxes to a standard margin without resorting to advanced CSS editing. Text boxes can be moved and re-sized easily enough by dragging on their corners or edges, but there is no ruler or grid system by which to align them. Also, unless I'm blind I'm finding no color picker to alter the background color in the magnified text boxes, so again you'll need to learn some CSS to use this tool that Amazon is hyping in their promos as being designed for those who don't know any HTML or CSS, and theoretically don't need to learn. In other words, KBC is fairly dismal as a graphic design tool, and if you're used to using Adobe products or iBA for your layout work you'll feel as if you've been drugged and had your arms chopped off while trying to produce something genuinely inventive here.

There is a Console panel available that shows the log file which Kindlegen produces during conversion, but if you're advanced enough to understand that you don't need to be using this kid's toy. Honestly, I was hoping for an adult app at some point from Amazon that can create advanced layouts and complex motion graphics, but all they've given us so far are simple widgets only good for making very basic layouts. And that's probably good enough for beginners, but it's not something professional ebook formatters will ever use.

So to summarize:

  • Easy user interface for a quick and simple learning curve
  • Easily create magnified text pop-ups from base text
  • Text layers isolated from background art (for potential live text upgrades)
  • Editable CSS/HTML code from within the program
  • Two-page spreads with facing pages made super simple
  • PDF / image input for really simple import most users can manage
  • Generates the necessary file structure, including OPF & NCX, which can be edited externally

  • No epub or mobi file input
  • Cannot edit mobi files not originally created with KBC
  • Very limited layout and formatting tools
  • Only basic text interactivity (pop-ups) without advanced HTML/CSS coding skills
  • No support for eInk devices (exports only to "Fire-compatible" KF8 according to the guide)
  • Auto-Orientation not available (locked to landscape or portrait)
  • Auto-resizes images without your input (cannot select your own "original-resolution" value)
  • No Undo!
  • Previews only available for Kindle Fire devices currently
  • Did I say No Undo?

A few notes for the more advanced users:

There are a couple of new "ke-" prefixed entries in the metadata section of the OPF that are interesting:
<meta content="doublepagespread" name="ke-layout-type"/>
<meta content="start-right" name="ke-start-side"/>
<meta content="true" name="ke-hd-images"/>
The first is what allows two pages to be knit together into one page spread or a single image to be spanned across two pages, but I haven't had the chance to dive into the generated CSS to see what's being done just yet to know what all the options are, or why this is necessary when the KF8 code already has the ability to create facing pages from two images. In this case it's a two-page spread consisting of two individual images, so "doublepagespread" is the value. The value is for a two-page spread made from a single image is, not surprisingly, "singlepage".

Second up we have the "start-side" which for the first time allows a Kindle fixed layout file to have a single standing first page on the right side (or presumably left in rl-horizontal writing mode ebooks). Previously this was attempted with the page-id "layout-blank" value, which never worked correctly, and single pages always appeared centered in the viewport rather than to one side or the other.

Finally, we have an "hd-images" entity that I presume is part of the new "standard resolution" versus "high resolution" image distribution for old model versus HD devices. Here the value "true" tells me that only the HD images are going to be transferred to whatever device the file is destined for, regardless of the model. Or perhaps it means that only HD devices are supported for this file, as implied by the absence of any mention of eInk screens in the documentation. But if "false" is also a legitimate value for this entity it begs the question of just what the heck is going on here. Of course, each of these is generated automatically by KBC, so you really aren't intended to alter them outside of the settings found in the program menus, and there isn't one for this third entry. But here they are for what it's worth.

Incidentally, KBC only produces a toc.ncx rather than a nav doc with landmarks, and the OPF contains none of the EPUB3 additions that had previously been implemented in KF8.

Well, that's all I've got for now after a quick first look. I'll ponder on it more and maybe play with it a bit tomorrow. But I doubt I'll ever use it much, except for previewing my HTML/CSS code on the fly. And honestly, that's the best use I can see right now for this thing.