The L&L Blog  /  Scrivener

Scrivener 3: Redesigning Compile

Note: This blog post pertains to upcoming features in Scrivener 3, which will be released on macOS later this year and will follow on Windows some time in 2018.

Scrivener’s Compile feature is powerful and adaptable, but it also seems to be a stumbling block for many users. Whenever someone describes Scrivener as “complicated” and we ask which areas they mean in particular, Compile is usually the first thing mentioned.

There’s a certain degree of necessity to Compile’s complexity (bear with me on this). When I set out to create Scrivener—well over a decade ago now—one of the most important goals for me was that it had to be flexible. I had tried all manner of apps aimed at writers. Back then, most specialised writing apps outside of word processors were built around a particular paradigm. Some wanted you to follow a set story structure, such as Vogler’s 12-step hero’s journey. Those that weren’t built around a popular narrative theory forced a particular structure on you: three acts, each containing several chapters, for instance. Copywrite and Ulysses came the closest to doing anything like what I wanted to do (in that they allowed you to break a larger manuscript down into smaller components without an imposed structure), but they didn’t provide any way of organising the sections of your manuscript into an indented, outline-like arrangement.

What I wanted was to be able to define my own structure, which would be different depending on the project, and to be able to reshape that structure as and when needed. Maybe I’d work in parts, chapters and scenes; maybe I’d only need chapters and scenes, or only chapters. For nonfiction I might organise things differently again. To this end, I built Scrivener around the core concept of a flexible outline—the binder. In the binder you can create folders and text sections and organise and nest them however you want.

Another of my aims was to allow the user to write using any formatting he or she wanted, and then to allow all of that formatting to be changed (optionally) at the export or print stage. So you could write in your favourite font but still output to standard manuscript format.

These two core aims come together in the Compile feature, which takes all of those separate, structured sections and sews them together into a single manuscript, overriding the formatting depending on the output requirements.

The problem we’ve always faced is that these two aims—complete flexibility of organisation and the ability to stitch everything together into formatted documents with specific requirements—are, to a certain extent, at loggerheads. If an app knows that you are writing in parts and chapters, it can provide an export that formats the text accordingly. But if you can structure your manuscript any way you like, when it comes to creating the final document, how can the app know which sections should be formatted one way and which another? How does it know that, say, in Project A, this folder is supposed to be the title of a part, this folder is supposed to be the title of a chapter, and this text document is a scene? Or, in Project B, this text document is a preface but these text documents are chapters, and should be formatted as such?

This has always been the huge challenge underlying Compile, and it’s the reason it has always had a degree of complexity to it: it needs to provide options that allow for any number of possible project structures, along with the ability to format those projects for many different requirements.

For Scrivener 3, we wanted to rethink how we approached this problem. I’m not going to pretend that the solutions we have come up with will mean that you never scratch your head over a Compile option, because there’s no perfect solution (barring machines becoming conscious, in which case I hope Skynet sends back a T-900/715 rather than a T-800/101 to help me, but we’re getting sidetracked). I do hope that Compile in Scrivener 3 makes a lot more sense to people, though.

Compile in Scrivener 2

There are two aspects of Compile that, I think, cause confusion for a lot of people. The first is simply the way we throw all of the Compile options right in your face.

We tried mitigating this a little by adding a “Summary” mode, but generally you need to switch to “All Options” as soon as you want to tweak anything, so most users end up faced with the full gamut of Compile options anyway. These options are powerful and give you complete control over your manuscript, but they are also a little intimidating if all you want to do is change which documents should be treated as chapter headings.

The other source of confusion is the way Compile formatting is applied to the sections of projects. This is all set up in the “Formatting” pane—where you tell Scrivener to show a title for this chapter, insert a “Chapter [Number]” heading for each of these folders, and change the text to Papyrus 16pt for these sections (this is a joke: please under no circumstances use a Papyrus font, unless that circumstance happens to be that the T-900 sent back in time for you accidentally zapped you back to 1997 and now wants you to design the front of a vegan friendship bracelets shop).

The confusion arises over how, in the “Formatting” pane, you have to think in terms of “levels” in the current project outline, like this:

In this example, formatting set up for “Level 1” folders in Compile will be applied to the “Part” chapters in the binder, formatting for “Level 2+” folders (the “+” indicates that this formatting will be applied to level 2 folders and deeper) will be applied to the “Chapter” folders, and formatting set up for “Level 3+” text will be applied to the scene documents.

This is actually a pretty good solution, because it allows you to define how your manuscript should look based on its structure. However, over the years of use and user feedback, we’ve thought a lot of about its limitations:

  • Setting up the levels in Compile like this ties each Compile format to a particular project structure.
  • It’s difficult to have different formatting for the same types of documents on the same structural level.
  • Not everyone finds it natural to think in terms of outline levels. Users think of their documents in terms of what they are as opposed to where they are placed in the outline: as chapters and sections rather than as level 1 folders and level 2 text files.

We spent a long, long time thinking about these problems for Scrivener 3, trying and throwing out a good number of ideas—and implementations—along the way. All those options are needed to offer the flexibility central to Scrivener, and the level-based approach is at heart a good way of applying formatting to projects with an outline structure. So how could we take all that was good about Scrivener 2’s Compile but make it simpler and less intimidating, while also providing more separation between Compile formats and individual projects so that a single Compile format could be used with all sorts of project structures?

Compile in Scrivener 3

This is how Compile looks in Scrivener 3:

Some things you might notice:

  • All of the Compile formats are now neatly listed on the left, rather than being in a pop-up menu. (You can create your own formats using the buttons at the bottom.) This list shows only formats applicable to the currently-selected file type.
  • The middle column shows a preview of roughly how the text will look when compiled. It shows template text rather than text from the current project (because to show the latter would involve compiling the project in the first place). In actual fact, what it’s showing are Section Layouts, a key new concept in Scrivener 3 Compile which I’ll talk about below.
  • On the right are project-specific options. In the screenshot you can see the contents (the list of documents in the project to be compiled), but there also tabs for document meta-data (author name, title) and options (include comments? Include footnotes as endnotes or end-of-page notes?). For ebooks you can also set up the book cover and table of contents here.
  • Next to each document listed in the contents, you’ll notice there’s a Section Type—in the screenshot each document is set either as a “Chapter” or “Section”. This is another key concept which I’ll cover below.

This UI essentially replaces the old “Summary” pane, but it is much more useful. It allows you to set up how the current project should use the selected Compile format, and for everyday use will obviate the need to dig deeper into the more complex options. It can do this because, internally, all of the various Compile options have been separated into project-level options and format-level options. So, for instance, each Compile format contains formatting settings that can be applied to any project, but whether or not you want to include comments for this particular project (for example) is something that you choose here using a quick checkbox.

At the heart of Compile in Scrivener 3 are two new powerful concepts: Section Types and Section Layouts:

  • Section Types are part of a project and tell Scrivener what each document in the project is (for example, a scene or a chapter group).
  • Section Layouts are part of a Compile format and provide different ways for documents to be formatted. So, a single Compile format may contain one Section Layout for formatting chapter titles and another for formatting scenes, for instance.
  • To set up Compile, you simply tell Scrivener which Section Layouts in the selected Compile format should be applied to the different Section Types in your project (“format chapter groups using this layout and scenes using that layout”).

Section Types thus mean that you no longer have to think in terms of outline levels when working with Compile. Instead, you think in terms of what each document is. For example, here are the Section Types that have been set up for the built-in Novel project template:

Each project template comes with some Section Types set up for you, but you can change them to whatever you want. You then assign Section Types to documents in the same way you assign labels or status, via the inspector or outliner (you can also set them in the Compile panel). In this way you can set a folder as a “Chapter Group” or a “Part Group”, for example, and a text document as an “Introduction”, a “Title Page”, an “Appendix”, or whatever the project requires.

Meanwhile, each Compile format is now project-agnostic, and, instead of containing a map of how to format specific outline levels (as in Scrivener 2), they now simply contain a list of different formatting presets (Section Layouts) that can be applied to any documents in your project (based on its Section Type). For instance, the “Standard Manuscript Format” has one Section Layout that simply prints out “Chapter [Number]” in Times New Roman for any document to which it is applied; another prints out the chapter number and title; another prints out the title for a new Part; another formats scene text; yet another formats a whole chapter (a chapter title and text). When you Compile, you simply choose which Section Layouts in the Compile format should be applied to each Section Type in your project, ignoring any that you don't need:

In the screenshot above, any document that has been set as “Chapter Text” (the Section Type) in the project will be formatted as a “Numbered Synopsis” (the Section Layout) during Compile.

What this means is that Compile formats are no longer limited to working with a particular project structure, and you no longer need to learn how to edit Compile formats just to get them working with your project. Instead, each Compile format comes with a bunch of Section Layouts, and you choose which ones to use with your project. You no longer have to tweak your project structure just to get Compile to format things how you want, and you no longer have to dive into the full swathe of options just to change which documents are treated as chapter headings.

We haven’t completely thrown out the old level-based design, however—after all, it was useful for applying formatting to many similar documents at once. Instead, we’ve moved the level-based setup from Compile to the Project Settings, and it now just provides a way of setting up the default Section Types. So, you can tell Scrivener that, say, any text document at level 3 should be a “Scene” unless you change its Section Type in the inspector or outliner. Where in Scrivener 2 you had to think in terms of outline levels to format your manuscript, then, now this is just something that will save a little time if you learn how to do it. You are free to ignore it and set the Section Types manually.

In practice, if you’re using the built-in project templates, you often won’t need to do anything other than choose the format you want to use from the list on the left and hit “Compile”. But if you’re creating your own project from scratch, it’s now much easier to format that project during Compile. And it’s also easier to create Compile formats that can be used with a wide range of projects.

Under the hood, all of the Compile options that were available in Scrivener 2 are still available in Scrivener 3—and more. But now you access these options by creating your own Compile format.

Many users will never, ever have to use these options, because the pre-provided Compile formats are now so much more flexible. But for those who like to dive into the more technical stuff, or who have specific formatting requirements that aren’t provided out of the box, everything you need is still in there.

We have prepared four videos introducing users to how Compile works for when Scrivener 3 is released. These progress from the basics right through to creating your own formats. Here’s the first one so that you can see the new Compile in action:

14 Comments

Alexander Gieg

Alexander Gieg  /  14 SEPTEMBER 2017

<i>"The middle column shows (...) template text rather than text from the current project (because to show the latter would involve compiling the project in the first place)."</i>

I'd like to suggest doing this with the current text by means of partially compiling it on the fly, even if for a future version 3.5 or 4.0.

The reason is that, with the new compile version, the old routine of changing settings, compiling to see how the resulting file looks like, deciding it isn't quite how you want it to look, changing settings, recompiling, rinse, repeat, will still be needed. Writers will no doubt do it way less often than before, but there will still be need for it.

With a live preview, in contrast, manipulating the compile settings so as to make it actually do what writers want would be much more intuitive, thus almost eliminating the "change settings, recompile" routine.

About five pages around the current position in the text, two above and two below, should be enough. And it doesn't need to be perfectly accurate in aspects such as page boundaries and pages numbering, as even rough approximations on those would be extremely useful given one's looking to fix formatting details only.

The resulting workflow would look like this for many, myself included:

Enter the compile setting screen; see how the beginning of my text would look like; adjust settings; it looks nice. Move the scroll bar to the middle; wait for a quick local compile to happen if it even takes enough time to be noticeable; see how the text looks; it looks nice. Move to the end; look around; adjust something. Move to a chapter title; ditto. Move a little more around; see something odd; adjust the setting. Move again; look. Move; look. It's perfect now. Save settings. Compile. Look at the finished document. Still perfect. Done.

Please consider this, will you? It might be technically challenging, but well worth the effort for every Scrivener user out there!

KB

KB  /  14 SEPTEMBER 2017

We did consider that, Alexander, and much more besides, but it's just not feasible. When you change a setting, you are never just changing a part of the text - most settings affect the whole thing. And even if it were possible to generate only a "quick" preview of a slice of the text, you'd still be finding that either the UI felt slow every time you clicked something or that the preview lagged behind your actions. But remember that it's not possible to know where text will fall on a particular page without first compiling all the text <em>before</em> that section and laying it all out in pages. So we might be able to generate a fast preview of the first five pages (which wouldn't be that useful if they were all front matter), but not of the final five pages. And even if the preview didn't show pages, what about auto-numbering and replacements and other elements that require knowledge of all the text that has come before?

3.0's solution might not provide a live preview of your compiled text, but we think it does the next best thing given how a live preview isn't possible, and that it is going to be very useful for our users.

Alexander Gieg

Alexander Gieg  /  14 SEPTEMBER 2017

<i>"it’s not possible to know where text will fall on a particular page without first compiling all the text</i> before <i>that section and laying it all out in pages. (...) And even if the preview didn’t show pages, what about auto-numbering and replacements and other elements that require knowledge of all the text that has come before?"</i>

A quick preview, clearly labeled as a draft preview with inaccurate page boundaries, could take the current section you're pointing at, count back a few sections, let's say, about 10 kB worth of raw unformatted text (enough for one to three printed pages), set that section as the beginning of the new pseudo-full-document new page, and do a compile from that point towards 10 kB worth of sections after the current point. Within this quick compile, any counter would start from 1. Replacements would be ignored. And that would be enough to check how things look.

What wouldn't be 100% accurate, sure. But it'd 90% accurate, what's enough to cut several time consuming cycles of compile the full final document, open it, check it, go back to Scrivener, change settings, compile the full final document, open it, check it, go back to Scrivener, change settings, compile the full final document, open it...

KB

KB  /  14 SEPTEMBER 2017

It's not quite that simple (we considered all of this) - anyway, it might be worth checking out the new system before suggesting ways of improving it! :) Thanks for the suggestion, though.

Jo

Joseph  /  14 SEPTEMBER 2017

In Scrivener 2, compiling is rather daunting (folder level 2+, document level 3+). In Scrivener 3, it looks like compiling has been improved. However, I'm a little confused about structure levels. In the screenshot of how compile looks in Scrivener 3, in the right column, it shows only one level (or type) of folders and documents within drop-down boxes. My question is - are the choices in the drop-down boxes either folder or document, or are there choices for different levels of documents and folders? Or, are all of the formatting choices available in the drop-down list, since one has to choose a type for every item in the project? If that is the case, is there an "as-is" option, for cases where two types of formatting g are used in a single document (say, like when most of the document is one type of text, but you have a quotation you want highlighted)? Thanks, in advance, for your response. I look forward to Scrivener 3's release.

KB

KB  /  14 SEPTEMBER 2017

Hi Joseph, in Scrivener 3, you don't have to worry about the structure levels if you don't want to. The choices in the dropdown are whatever you want them to be for your project. So, you could have "Introduction", "Chapter", "Appendix", for instance. Or "Title Page", "Chapter Group" and "Scene". Then you just tell Compile each of these should look.

With regard to structure levels, in Scrivener 3 use can use them to say to a project, "I want all folders at level 1 to be Part Folders by default, all folders at level 2 to be Chapter Folders by default, and all text documents to be Scenes by default." Those will then be the default Section Types for documents in your project. But if you have a text document that is set to "Scene" as default and you need it to be an "Introduction", then you simply change the setting via the dropdown. And if structure levels give you a headache, then you ignore them and set all of the document types using the dropdown instead of worrying about the defaults.

All of this will become much clearer once you try it out. :) Oh, and yes, "As-Is" is now one of the section type options.

razyr

razyr  /  15 SEPTEMBER 2017

I'm looking forward to using the new compiler interface. I always thought that creating Scrivener output was somewhat akin to struggling with something like LaTeX. If you twiddle the knobs enough, you generally get something similar to what you wanted.

Some Scrivener users have resorted to using third party tools such as V<em><strong><em>m for e-book publishing. In part, this has been ease of use, but also, tools like V</em></strong></em>m, give very polished output.

Can we expect the new version of Scrivener to compete with these tools, especially with regard to the visual appeal of e-book output? I'm sure most of us would rather write and publish with a single tool (ie Scriver 3).

KB

KB  /  15 SEPTEMBER 2017

@rayzr: I'll be putting up a separate post about ebook support in 3.0 next week, but basically you'll have as much control over ebooks as you want. Scrivener 3 supports the epub 3 format, and because, for various reasons, this involved a complete rewrite of how Scrivener creates ebooks (Kindle books too), this has resulted in much cleaner ebook files. And part of this is that you can, for those who want and know how to do it, completely override the CSS that Scrivener uses in ebooks. In conjunction with the new styles system, this means that you can make ebooks that look however you want. Even the standard output from Scrivener for those who don't know how to toy with the CSS themselves is very nice, though. The ebook output from Scrivener 3 should be up there with the best ebook generators when it comes to text-based books (I hope).

razyr

razyr  /  15 SEPTEMBER 2017

Watch me while I do my happy dance!

Thx KB

Eric Beaty

Eric Beaty  /  16 SEPTEMBER 2017

These updates look amazing, and I can't wait to try out the new interface! Especially when it comes to compiling with regards to Section Types. This will be so much easier when thinking in Parts, Chapters, Scenes, etc. I myself prefer to work with folders as Chapters and Text Docs as Scenes.

I do have a question that's sort of related to compiling. Will there be a way to export entire projects from Scrivener 3 as Scrivener 2? The reason I'm asking is that if my editor doesn't want to upgrade, I'll want to be able to save the project as a previous version. Then, I could open the newly compiled/saved Scrivener 2 project back in Scrivener 3 as the final/edited version of my project.

To be honest, I'd probably pay to upgrade my editor's version myself, but there's a chance he would still rather work in version 2. You never know. I just don't want to have to switch to Google Docs to collaborate on editing when Scrivener has been working perfectly while avoiding all the copying/pasting and thus removing any formatting I've already set in Scrivener.

KB

KB  /  22 SEPTEMBER 2017

@Eric: Thanks for the kind words. I'm afraid there's no way of exporting to Scrivener 2 from Scrivener 3. I did implement that in early versions, but as Scrivener 3 evolved, it became harder to support the older format without losing a lot of data from the Scrivener 3 project, because of the new features.

Eric Beaty

Eric Beaty  /  22 SEPTEMBER 2017

@KB: Bummer. Back to the drawing board then, lol.

Ioa

Ioa  /  13 OCTOBER 2017

@Eric: to provide a point of good news for you, visit this forum thread for an update on project formats.

bwine

bwine  /  07 DECEMBER 2017

@KB My old version of Scrivener used to automatically compile my APA research papers, and it was beautiful. I can't figure out how to recreate that.

Add your comment here...

Keep up to date