[LH4909] Formatting for .mobi

ke
keithyd
Posts: 38
Joined: Sat Sep 08, 2018 4:09 pm
Platform: Windows

Wed Feb 24, 2021 5:40 pm Post

You're probably sick to death of this topic, but it's really frustrating. The Compile for .mobi used to work flawlessly but it hasn't worked properly at all for a long time (I don't know exactly how long because I only use it when I have a book to compile - every 4-5 months or so.)

I attach some jpgs to demonstrate.

On Compile, I choose 'Compile for Kindle Ebook (mobi)' and select Heading (for the Chapter heading) and Section (for the bulk of the text). I've made a copy of the standard Scrivener Ebook format so I can check the formatting, as shown in the attachment, 'Section text formatted for mobi' (in the next post - limit reached). This shows that the text when compiled should have the first 3 words capitalised, paragraphs indented and single spaced. The attachment 'Result of mobi compile' shows how it actually turns out - Chapter heading ranged left, paragraphs all ranged left and separated by a line break.

The attachment 'Formatted for ePub' is what it says on the tin, and compiles correctly, apart from the ranged left opening line of dialogue. Currently I run this resulting ePub through Calibre to convert it to .mobi so I can send the file to my own Kindle to read in its 'native' format. It's part of my proofing process so I need this .mobi file. This obviously is a wasted step.

The final attachment, 'Original Scrivener formatting' shows how the text appears in my Scrivener Editor.

I have to say the use of Section Layouts in the compile function is confusing (too many) and, it appears, doesn't work. Is there something I'm missing? I'm a huge Scrivener fan and tell everyone about it, but this apparent bug is driving me nuts.

Thanks
Attachments
Result of mobi compile.jpg
Result of mobi compile.jpg (72.05 KiB) Viewed 632 times
Original Scrivener formatting.jpg
Original Scrivener formatting.jpg (145.41 KiB) Viewed 632 times
Formatted for ePub.jpg
Formatted for ePub.jpg (44.91 KiB) Viewed 632 times

ke
keithyd
Posts: 38
Joined: Sat Sep 08, 2018 4:09 pm
Platform: Windows

Wed Feb 24, 2021 5:42 pm Post

This is the additional screen grab related to my previous post.
Attachments
Section text formatted for mobi.jpg
Section text formatted for mobi.jpg (108.61 KiB) Viewed 630 times

ch
chrisdr2
Posts: 84
Joined: Wed Oct 16, 2019 9:09 am
Platform: Windows

Fri Feb 26, 2021 10:28 am Post

This issue has been around for ages. I first reported it for RC5 in May 2020. Since then, I've done the same as you - use Calibre to convert to Mobi format. It would be nice to get it fixed for the release version.

That said, when I last published a book via Amazon KDP, the web page suggested uploading EPUB format rather than MOBI. If Amazon is favouring EPUB, I wonder what the long-term future is for MOBI.

ke
keithyd
Posts: 38
Joined: Sat Sep 08, 2018 4:09 pm
Platform: Windows

Fri Feb 26, 2021 2:12 pm Post

The thing is, there are millions of Kindles out there that will read mobi (and pdf) but not ePub. As I said, this is how I proof my daily work - export to Kindle so I'm reading it in a different format to that in which I wrote it. And it USED to work but seems to be broken now. It won't even centre text - I have a section separator that I've tried to centre using the paragraph formatter (the little square icon with centred lines on it) and the Style command for Centred Text - no dice. It works when compiling to Word, but not to epub or mobi.

User avatar
AmberV
Posts: 24887
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:

Fri Feb 26, 2021 3:54 pm Post

To clarify, it sounds like the ePub version works largely fine, but the Kindle version does not?

There are some troubleshooting steps you should take to pin-point the source of the problem:

  1. In Compile overview, General Options tab on the right, set the Save source files in a folder with exported Kindle file.
  2. Compile as you normally would. Upon completion, visit the folder that will be created, using File Explorer.
  3. Load one of the numbered “body.xhtml” files in your browser.

Does the formatting look good at this stage? If so, we probably aren’t looking at a flaw in Scrivener, or the configuration of the compile settings. If the formatting isn’t correct here either, then skip the rest of the troubleshooting checklist, down to the “Doesn’t look good in browser” heading.

  1. Next test: drag the .opf file that you will find in this source file folder into Kindle Previewer.

This is essentially what Scrivener does for you automatically, by the way, when you select the Mobi option. It creates this folder, sends the .opf file to Kindle Previewer, gets a “.mobi” file back, and saves that for you. Does the output still look good in this context?

If so, then we have now eliminated whether Scrivener is producing the correct formatting, but producing it using formatting codes that Amazon doesn’t recognise. That would be unlikely given I cannot reproduce the problem, but in this case skip to the end for tips on where to find the CSS and HTML, and paste them into a response so I can take a look at it, if you need help.

  1. Next test: use Export from Kindle Previewer to save a .mobi file and side-load that onto your device.

Does this copy look different than the one “Scrivener created” (at this point it should be clear it does no such thing, it just asks KP to do this stuff). If it does, it’s a mystery! Amazon is making two different .mobi files from the same program somehow. I’d say report it to them, but they probably will just tell you that using KP to create .mobi files is no longer supported. (All of this is deprecated technology they would rather nobody use, it’s worth noting.)

Now if you’re using an iPad or iPhone to proof the book, you will probably find the .mobi file doesn’t look right from KP either. Do note you should be selecting a different option from the Export dialogue. If that looks good, then that was your problem this entire time. Your iOS device doesn’t read Mobi files correctly. If for some reason the ones from Calibre work better (it does by default tune them for side-loading rather than publication, after all), then you’re just looking at issues between Amazon’s generator and Calibre’s. I’ve never tried that though, as I’ve just always assumed all Mobi files look awful on iOS.

The thing is, there are millions of Kindles out there that will read mobi (and pdf) but not ePub.


Unfortunately, as you can probably see at this point in the troubleshooting process, it’s a little more complicated than that. What you upload to KDP is not what readers will be downloading. Amazon’s server takes the file and converts it to some other format, probably one that none of us will ever see or use. Then, when a reader buys a copy and downloads it to a device, the server exports the best version possible for that specific device and sends it. If the same reader sends a copy to their iPad, they’ll be getting a completely different file from the server.

So while you are correct in saying nobody is reading ePub files on their Kindles, or with Kindle software, it’s also accurate to say that hardly anyone is reading a Mobi file, either. With the exception of some extremely old devices (at this point), no devices are going to be getting a MobiPocket format ebook from the server. In fact even when you load a Mobi file onto your device it probably isn’t reading the MobiPocket part of it, either (the .mobi file KP creates has two parts, the ancient fall-back that old devices use, and a modern format that newer devices use).

And yes, to top it all off, as noted above uploading these double-book .mobi files to KDP is now deprecated, and you are encouraged to upload ePub3 instead. It will still work for now, but there is very little reason to use .mobi for publication at this point. The only reason to use it is for personal proofing—and that is the only reason we have not completely removed this setting from the compiler at this point.

Doesn’t look good in browser

And it USED to work but seems to be broken now. It won’t even centre text - I have a section separator that I’ve tried to centre using the paragraph formatter (the little square icon with centred lines on it) and the Style command for Centred Text - no dice. It works when compiling to Word, but not to epub or mobi.


Who knows, it’s impossible to say without more detailed information since from my testing it works fine. Here is what the HTML looks like when I use the “Heading” layout like you describe:

Code: Select all

<h2 id="doc1" class="title"><span style="font-family:'P052'; font-size: 133%; text-transform:uppercase;">RED BOOK</span></h2>


So, there are some minor issues with that to be fair; what I should be seeing is something cleaner and easier to work with:

Code: Select all

<h2 class="title" id="doc1">RED BOOK<h2>


Either way however, you can see that from this alone there is nothing about centre-alignment, nor anything that would conflict with it (just some unnecessary formatting like uppercasing letters that are already capitals).

Alignment isn’t (and shouldn’t) defined here, but rather in the stylesheet.css file, which you will find in the ‘css’ subfolder of the source file output. Since this heading is assigned to the “title” class, we would look for that in the CSS file:

Code: Select all

.title { margin: 0rem 0rem 0rem 0rem; text-indent: 0rem; text-align: center; }


And there is the magic code to adjust the text alignment (along with some formatting ensuring there are no indents on the left or right that would push the relative definition of middle off-centre).

So like I say, it should be a bit cleaner, but these minor bugs aside, neither of those issues should cause your text to be left aligned when the CSS for .title clearly indicates otherwise.

Those are the places to check in your source if it doesn’t even look right in a browser.

Lastly, if you’re getting different HTML than I am it might be best to wait for the next build. They are still fine-tuning ebook formatting in a large-scale way. I have spotted numerous issues with the current output, though nothing yet that would cause what you’re describing to happen.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

ke
keithyd
Posts: 38
Joined: Sat Sep 08, 2018 4:09 pm
Platform: Windows

Sun Feb 28, 2021 1:00 pm Post

Thanks for the comprehensive response, much appreciated.

I take your point about .mobi no longer being a recommended format for upload, so I'll bear that in mind the next time I go that route. I've downloaded KindleCreate again (changed PC since the last book upload and forgot about KC, in fact ...) so I'll try using that to create files I can read on my Kindle reader (no, I don't use iPad or iPhone - can't bear Apple!)

As far as your instructions went, I followed them and had some interesting results. Looking at the xhtml files in Edge browser, the formatting was generally fine. I'd taken to typing the first 3 words of a new section in all caps, however, so I need to change them back to standard formatting to test whether Scrivener's Compile function will actually convert the first 3 words to caps as the Section formatting says it should.

However, my 'paragraph' break icon still won't behave as Centred text. (it's actually a duplicated 'Section sign' from the character map: §§§, so not an image). Compiling the file to ePub 3 and reading it in Adobe Digital Editions, the centering is fine. Using Calibre to then convert to .mobi it's also fine - so it seems Scrivener is doing its job for that conversion, but not for conversion to .mobi. Looking at the CSS text it does identify Centred text as Centred, but that's not how it's being produced in the final mobi file:

.centered-text { margin: 0rem 0rem 0rem 0rem; text-indent: 2.08rem; text-align: center; }

In the CSS I found the entry for Chapter Title. No alignment seems to be registered:

.chapter-title { margin: 0rem 0rem 0rem 0rem; text-indent: 0rem; }

However, the visual image for this in the Compile Section Layout list shows the Chapter title as centred.

As you say, if .mobi is being deprecated by Amazon then there's no need for me to worry. I'll try using Kindle Create to make a file I can read on my Kindle ereader instead.

Thanks again for your help.

ch
chrisdr2
Posts: 84
Joined: Wed Oct 16, 2019 9:09 am
Platform: Windows

Mon Mar 01, 2021 12:21 pm Post

I unpacked a Scrivener produced .mobi file with the KindleUnpack Calibre plugin and found a potential issue with the enclosed epub file.

The chapter xtml files within the EPUB are missing the link to the stylesheet css file.

Code: Select all

<link type="text/css" rel="stylesheet" href="../Styles/style0001.css"/>


Adding the above line with Sigil fixed the style/formatting, etc.
Hope this will help the devs to track down the issue.

User avatar
AmberV
Posts: 24887
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:

Mon Mar 01, 2021 1:56 pm Post

However, my ‘paragraph’ break icon still won’t behave as Centred text. (it’s actually a duplicated ‘Section sign’ from the character map: §§§, so not an image). Compiling the file to ePub 3 and reading it in Adobe Digital Editions, the centering is fine. Using Calibre to then convert to .mobi it’s also fine - so it seems Scrivener is doing its job for that conversion, but not for conversion to .mobi.


So to make sure we’re on the same page, this special paragraph type isn’t aligned properly in the loose file .xhtml file output for Mobi, but it looks fine in ePub? If so, would it be possible to create a small sample project that demonstrates this—either that or explain in complete detail how to configure the section style you describe above. Earlier I was just testing with stock “Ebook” settings using the Heading and Text Section layouts, but it sounds like you’re doing some customisation.

Code: Select all

.chapter-title { margin: 0rem 0rem 0rem 0rem; text-indent: 0rem; }


That class is used by the “Chapter Title” section layout, as a container div around the whole title. The individual lines of text will generally have their own styles, and that is where alignment will be stipulated. This just makes sure the heading box itself isn’t padded and that all interior padding will be reset to zero unless stated otherwise.

However, the visual image for this in the Compile Section Layout list shows the Chapter title as centred.


Looking at the output generated by this Layout, it is very obviously not set up right yet. As I may have mentioned before, the settings themselves are a WIP and most of the compile formats are not in a completed state at this point in time, as they’ve only just finished the underlying text formatting code for ebook output. We’re close to having them finished though. In this case, the HTML is absurd (stripping out the clutter):

Code: Select all

<div class="chapter-title">
    <h2 class="title">Chapter Number Line</h2>
    <h1>Name of Folder</h1>
</div>


So this is a bit like saying the letter is outside of the envelope and the envelope was never addressed to begin with. H1 is the highest heading level, and should be used for the whole chapter heading, H2 would be a second level heading, like a scene title or a section name. And as for the “address”, as you can see the H1 has no class assigned to it, and nothing in the CSS is designed to exert too many formatting assumptions on the H* elements themselves.

Honestly I would just fix the format. Unfortunately there is a bug in how line breaks are handled in the compiler right now, which means the best way to handle this problem (a true multi-line paragraph rather than a sequence of paragraphs being loosely thought of as a singular heading) can’t be done.

  1. Duplicate and edit, and in Section Layouts, select “Chapter Title”.
  2. Click on the second line, where “Section Title” is printed, and set the paragraph style to “Subtitle”.
  3. Honestly I would change them both to “Heading 1” in the Styles pane. It still doesn’t make sense to have two adjacent headings, but it makes more sense than Heading 2 out of nowhere followed immediately by a Heading 3, just to change the font.

And that should do it. Now the HTML should look like (to strip out the cruft, again):

Code: Select all

<h1 id="doc1" class="title">CHAPTER ONE</h1>
<h1 class="subtitle">Red Book</h1>


And the CSS for these two classes:

Code: Select all

.title { margin: 0rem 0rem 0rem 0rem; text-indent: 0rem; text-align: center; }
.subtitle { margin: 0rem 0rem 0rem 0rem; text-indent: 0rem; text-align: center; font-style: italic; }


Ideally I would like to see HTML like this:

Code: Select all

<h1 id="doc1" class="title">CHAPTER ONE<br/>
  <span class="subtitle">Red Book</span></h1>


It should be possible to get that, but like I say the ability to insert a line break is, at the moment, broken.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

User avatar
AmberV
Posts: 24887
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:

Mon Mar 01, 2021 2:01 pm Post

chrisdr2 wrote:I unpacked a Scrivener produced .mobi file with the KindleUnpack Calibre plugin and found a potential issue with the enclosed epub file.


I would think that at that point you are looking at the conversion engine’s HTML as much (or maybe even more so) than anything Scrivener created, and potentially even the unpacking tool could be a dusty lens here too—you’re three steps away from the source here.

As noted above, when troubleshooting Scrivener-related issues, I recommend enabling the source file option so that you can examine what it is creating before it is converted to anything. You will then see all of our CSS is placed into one “stylesheet.css” file, located in a “css” subfolder.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

ch
chrisdr2
Posts: 84
Joined: Wed Oct 16, 2019 9:09 am
Platform: Windows

Mon Mar 01, 2021 6:31 pm Post

I tested the compile for epub3 and mobi, saving the output, as suggested. According to the generated body xtml files, the epub complile includes the css link, but the mobi compile does not.

epub3 compile:

Code: Select all

<?xml version='1.0' encoding='utf-8'?>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" xml:lang="en" lang="en">
  <head>
    <title>Chapter Two</title>
    <meta name="Generator" content="Scrivener for Windows HTML Writer"/>
    <meta name="ScrivenerVersion" content="2.9.9.19"/>
    <link type="text/css" rel="stylesheet" href="stylesheet.css"/>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
  </head>


mobi compile:

Code: Select all

<?xml version='1.0' encoding='utf-8'?>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" xml:lang="en" lang="en">
  <head>
    <title>Chapter Two</title>
    <meta name="CreationTime" content="2021-03-01T18:04:27Z"/>
    <meta name="ModificationTime" content="2021-03-01T18:04:27Z"/>
    <meta name="Generator" content="Scrivener for Windows HTML Writer"/>
    <meta name="ScrivenerVersion" content="2.9.9.19"/>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
  </head>


I think that's why styles won't show for mobi. Hope this helps.

User avatar
AmberV
Posts: 24887
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:

Mon Mar 01, 2021 8:28 pm Post

Thanks! That seems easy enough to reproduce, I see it as well now that I have Kindle Previewer installed again and can compile to Mobi. I've filed it has high priority, as that obviously needs to be fixed before launch.

Curious that @keithyd seemed to be getting reasonable output when viewed in Edge, there may be some additional conditions that affect things---but my test was done with pretty vanilla settings.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

ke
keithyd
Posts: 38
Joined: Sat Sep 08, 2018 4:09 pm
Platform: Windows

Mon Mar 01, 2021 11:51 pm Post

Hi again

I've made a test post just concentrating on the centering issue, really. There are screenshots of the original Scrivener page, plus shots of the Kindle and ePub output and the pair of them opened in Edge. The CSS for both versions show Centred text and Chapter title as Centered, though output results varied. I also include the .scriv file here if that's any help. It'll take a couple of posts because of the 3-attachment limit.
Attachments
Resulting mobi file, 3 words capitalised but centering not working.jpg
Resulting mobi file, 3 words capitalised but centering not working.jpg (55.56 KiB) Viewed 337 times
Original Scrivener layout for first paragraph and centred text.jpg
Original Scrivener layout for first paragraph and centred text.jpg (94.55 KiB) Viewed 337 times
Compile setup for mobi, copied format, correct layout in bottom screen..jpg
Compile setup for mobi, copied format, correct layout in bottom screen..jpg (173.4 KiB) Viewed 337 times

ke
keithyd
Posts: 38
Joined: Sat Sep 08, 2018 4:09 pm
Platform: Windows

Mon Mar 01, 2021 11:53 pm Post

Next files
Attachments
xhtml file for .mobi output, text not centred where required.jpg
xhtml file for .mobi output, text not centred where required.jpg (62.06 KiB) Viewed 337 times
Section text formatted for mobi.jpg
Section text formatted for mobi.jpg (108.61 KiB) Viewed 337 times
Compile screen for mobi with section layouts.jpg
Compile screen for mobi with section layouts.jpg (123.06 KiB) Viewed 337 times

ke
keithyd
Posts: 38
Joined: Sat Sep 08, 2018 4:09 pm
Platform: Windows

Mon Mar 01, 2021 11:57 pm Post

Epub files and scriv file, zipped.
Attachments
Test project 2.zip
(8.17 KiB) Downloaded 11 times
xhtml for epub version opened in Edge.jpg
xhtml for epub version opened in Edge.jpg (60.09 KiB) Viewed 337 times
Test compiled for epub, difference in centered section breaks.jpg
Test compiled for epub, difference in centered section breaks.jpg (53.11 KiB) Viewed 337 times

User avatar
AmberV
Posts: 24887
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:

Tue Mar 02, 2021 1:34 pm Post

Thanks for the additional information. It looks to me as though all of the Mobi screenshots are failing under the same general conditions that chrisdr2 described in a previous response. Basically without the CSS file wired up, it doesn’t really matter what’s in the CSS file. :) That, incidentally, I'm told has been fixed for the next update. So give it another go once you see that land.

The ePub issue with one of the styled text ranges losing its style look like the remaining mystery to me. Unfortunately your attachment contains only the .scrivx file, so I can’t really see what is happening directly. Hopefully you still have the project folder around to try again.

If not, what does the HTML itself look like for that first §§§ break?
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles