Raw LaTeX, Scrivener 3 and compile process

Ch
Chris_M2018
Posts: 9
Joined: Mon Jan 22, 2018 12:59 pm
Platform: Mac + Windows

Tue Feb 06, 2018 11:09 am Post

Hello community!
I am trying to create a Scrivener Project that will fit fundamental properties needed in a scientific paper, such as:
  1. sections, subsections, subsubsections...
  2. Title page, Table of Contents, Abstract, Keywords, List of Figures and Tables..
  3. footnotes
  4. figures, tables, equations and their labels (preferably automatically numbered)
  5. bibliography


My background in LaTeX writing is intermediate, while for MMD is null! So, my intuition guide me to make Scrivener writing more LaTeX-oriented!

Is there any detailed (and even better illustrated) guide on how to prepare a Scrivener 3 project, adjust replacements and LaTeX Options. and finally compile it for MultiMarkdown -> LaTeX (.tex) format?

I have posted some of the issues I have faced in the "Wishlist" folder of the Forum:
viewtopic.php?f=4&t=50494
Any tip/trick/thought/material is greatly appreciated!!

Thanks!

User avatar
nontroppo
Posts: 1034
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Tue Feb 06, 2018 1:27 pm Post

Sorry in advance for my answers not being LaTeX specific. Many of the tricks that work with the Pandoc/MMD workflows in Scrivener 3 (particularly using styles to wrap markup around a text section) will also work with a LaTeX workflow. To be honest you shouldn't need to do too much fiddling RAW LaTeX should pass through to Pandoc/MMD quite simply (i.e. http://pandoc.org/MANUAL.html#extension-raw_tex). You can mix and match markdown and LaTeX, which is what many people do. MMD/Pandoc is really not difficult to learn (if you can install LaTeX and manage it, pandoc and mmd are a piece of cake!)

This is an interesting project, that creates a more custom Scrivener LaTeX workflow, but is not yet updated for Scrivener 3: https://github.com/mnott/texdown

I'm a scientist who uses all the features you specify except for Tables of Figures / equations, and for which my Pandoc workflow gets me a fast robust solution for output to Word and PDF via LaTeX or HTML (PrinceXML):

1. just use Binder levels
2. Title page, Table of Contents, Abstract, Keywords — can be built from the project metadata.
3. use scrivener footnotes.
4. figures/tables/equations — I use Scrivener <$placeholders> but if you need to create tables of figures, you may want to defer numbering to Pandoc (pandoc-crossref) or LaTeX itself.
5. I use Pandoc Citeproc as it is works for any output format...

Also for the metadata pedantic one could look at Pandoc Scholar which provides a nice way to build scientific papers with full JATS metadata to both LaTeX and Word outputs: https://github.com/pandoc-scholar/pandoc-scholar

User avatar
AmberV
Posts: 22061
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Tue Feb 06, 2018 9:16 pm Post

Something I’ve been working on periodically for a few weeks now is a starter template for those wishing to use Scrivener as a LaTeX “front end” without MultiMarkdown. The intended goal for this template is to alleviate some of the busy work that goes into forming a .tex file, without sacrificing the purist approach of being able to write directly in LaTeX syntax. As such it contains a bunch of helper stuff that you could use piecemeal as desired. At its most basic, you could use it like not much more than a glorified text editor, but if you employed all of its features it could handle images, footnotes, lists, basic environment formatting, and of course structural outline-based construction with automatic cross-referencing capability (this can be extended to tables, figures and equations, with automatic or semi-automatic labelling).

A second goal is to be pretty vanilla: demonstrate the basic concepts of how one can establish a document class in the compile format, and thus how to make your own and input your preamble; otherwise make very few assumptions (much like stock Pandoc/MMD LaTeX output).

The main thing that I’m still puzzling out is a way to generate bibliography entries—that might be something not worth investing any time in ultimately, as surely most people have their own system set up for that. It’s also something I have very little personal familiarity with doing.

Another area I have little personal experience with is equations. At the moment there is no framework for them. Generating something basic using the framework established for figures wouldn't be difficult, but are there any other aspects of that task that would be useful to include?

Other than that one detail (and you should remove the “Bibliography” folder from the Back Matter area), the rest of it should be pretty close to done and documented. I just used it to typeset a sample PDF (in the Research folder), so the basics all seem to be functioning.

Let me know if you have suggestions!

With Scrivener 3.0.3 available, this template is now included in the base install, under the Non-Fiction category. Thanks to everyone that helped test it and provide feedback!
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

Ch
Chris_M2018
Posts: 9
Joined: Mon Jan 22, 2018 12:59 pm
Platform: Mac + Windows

Wed Feb 07, 2018 7:21 am Post

Oh AmberV!! You've done an amazing job :shock: !!
Thank you very much for preparing this template! I'm sure that it will be greatly appreciated by those of us that want to bypass the MMD step in the Scrivener->MMD->LaTeX route!
I have not compile a full document yet, but I have some additions to your template so as to make the template more compatible with the standard way that scientific papers are produced:
  1. Several packages may be added in the "Document prefix" part for example:

    Code: Select all

    \documentclass[authoryear,review,3p,12pt]{elsarticle}
    \usepackage{geometry}
    \geometry{a4paper}
    \usepackage{graphicx}
    \usepackage{subfloat}

    %% Math packages
    \usepackage{amssymb}
    \usepackage{amsmath}
    \usepackage{amsthm}

    \usepackage{booktabs}
    \usepackage{multirow}
    \usepackage{tabularx}
    \usepackage{adjustbox}
    \usepackage{natbib}
    \journal{journal_title}
     
  2. Maths: If you import the Math packages shown above then I suppose that you pass equations easily in the text (I have not tried it yet).
  3. Bibliography: you can also add the

    Code: Select all

    \bibliographystyle{spiebib}
    % \bibliography{name_of_bib_file}

    in the "Document suffix" part, just before the \end{document} command and as long as you have a .bib file on the same folder as the .tex file, you just replace the "name_of_bib_file" and hopefully the references will be loaded (I have not tried it yet).

I have some questions concerning the Figures and Tables:
  • is the "Insert"->"Image Linked to File..." option active in this template?
  • how do we add captions and labels for Figures and Tables? I have read the documentation of the template but I didn't get which steps you follow to insert the LaTeX logo.

Finally, do you think that it is possible to produce two extra pages, one for the "List of Figures" and the other for the "List of Tables" in the beginning/end of the document that will serve as a Table of Contents style for Figures/Tables?

That's all for now, I will come back once I will compile my first document!
Thanks again for your effort!

User avatar
auxbuss
Posts: 127
Joined: Mon Nov 30, 2015 9:50 pm
Platform: Mac
Contact:

Wed Feb 07, 2018 11:59 am Post

AmberV wrote:Something I’ve been working on periodically for a few weeks now is a starter template for those wishing to use Scrivener as a LaTeX “front end” without MultiMarkdown.


This is superb. I don't get along with any of the dedicated LaTeX editors, so usually end up in Sublime. Obviously, I'd rather be in Scriv.

Thanks a lot.
Image

User avatar
AmberV
Posts: 22061
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Thu Feb 08, 2018 2:32 pm Post

Thanks for the suggestions, Chris!

So what I’ve done with some of this is made the Front Matter folder a bit modular. There is a section for equation stuff—if you need that you can click the Include in Compile button, and so on.

I’ve also had a go at adding simple equation support. There are now two document templates, one for numbered equations and other for simple displaymath style printing. There is also a character style that can be used for inline equations (very simple; just inserts $…$ around the text).

@Chris_M2018: is the “Insert”->“Image Linked to File…” option active in this template?


Yes, any method of inserting images is supported. Scrivener does not distinguish between types once it gets to that phase of the compile process. By then, internally speaking, all images have been fully embedded into the RTF file that is used for conversion to .txt, and then by this format, to .tex.

how do we add captions and labels for Figures and Tables? I have read the documentation of the template but I didn’t get which steps you follow to insert the LaTeX logo.


I’ve hopefully improved the explanation a bit in the help file. It is purely a process of using styles, and hopefully just clicking on the bits of text in the example is enough to demonstrate how to copy the technique.

Finally, do you think that it is possible to produce two extra pages, one for the “List of Figures” and the other for the “List of Tables” in the beginning/end of the document that will serve as a Table of Contents style for Figures/Tables?


That is certainly something you could add if you needed. The “LaTeX Control File” document template is set up for doing this kind of stuff (printing what you type in as-is and giving it a distinct icon). So just add one of those to your back matter folder and type in the codes.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

Ch
Chris_M2018
Posts: 9
Joined: Mon Jan 22, 2018 12:59 pm
Platform: Mac + Windows

Thu Feb 08, 2018 9:53 pm Post

Everything worked smoothly AmberV with the previous template!
Let me ask some more questions:
  • is it possible to add as a replacement for a new paragraph the "\par" command? I see that you have in the replacement tab "\\" but doesn't seem to work when compiling in LaTeX.
  • could you please provide more details on how we can cross-reference the items of the draft (i.e. chapters, sections, tables, equations, figures...)?
  • is it possible to link the font specifications (e.g. bold, underline, italics, colour, etc) of the Scrivener with the fonts actually produced in LaTeX?
  • how about having 2 or more pictures with the same caption and label? is there any short route for this feature?
Thanks again AmberV for your amazing work!

User avatar
AmberV
Posts: 22061
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Fri Feb 09, 2018 11:33 am Post

De nada!

  • I mapped line breaks (⌥⌘↩ or Insert ▸ Break ▸ Line Break) to the “\\” code. That seemed a very safe assumption to make as a basic default to me. This type of line break is not a concept in raw text, it’s really only a word-processing convention as far as I’m aware of. So if one wants to break lines of verse or whatever without typing in the break points themselves, that’s a way of going about it. Perhaps I should remove it though, if it is confusing—the whole thing is quite unintuitive without a visible character code.

    As for my mapping something to \par as a base default, I’m not sure what would work best there. What is shorter than the code itself that many might think to try without looking up? Perhaps I do not understand what you have in mind. I typically only ever use par inside my own commands.
  • Do you mean to more fully describe the compile settings that work together to create this effect? I could see that being useful, in case one doesn’t like autoref, or my “scrivauto” bookmark prefix. I’m not sure what else to say for front-end usage though. You make a document link pointing to the equation from your text—it’s no more complicated than that.
  • Only raw italics are handled specially, in that Markup compile format pane. Everything else must be style based. I’ve included a few examples, strong/emph/texttt—and pointed out how to make new environments in the help file.
  • That should be possible with both approaches as designed. How were you trying to do this? I gave it a quick test, both as two adjacent figures in a contiguous Figure style range, and as two figures within a “Figure” document type, the Layout’s prefix and suffix handles the overall environment and captioning. All the compiler does for the instance of the image itself is inject \includegraphics… Thus any place where it is valid to insert that code you can stick an image in Scrivener.

Code: Select all

\begin{figure}[htbp]
\centering
\includegraphics[width=175pt,height=73pt]{800px-LaTeX_logo.png}
\includegraphics[width=176pt,height=217pt]{templates.png}
\caption{This is the caption for two images.}
\label{scrivauto:6}
\end{figure}
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

Ch
Chris_M2018
Posts: 9
Joined: Mon Jan 22, 2018 12:59 pm
Platform: Mac + Windows

Sat Feb 10, 2018 11:24 am Post

Thank you very much AmberV for your prompt and helpful replies!!
  • Labels:
    Do you mean to more fully describe the compile settings that work together to create this effect? I could see that being useful, in case one doesn’t like autoref, or my “scrivauto” bookmark prefix. I’m not sure what else to say for front-end usage though. You make a document link pointing to the equation from your text—it’s no more complicated than that.

    My issues with labels are:
    1. I insert my figures in the main text and when I compile my draft in a LaTeX editor, the label numbers they get are the same with label of the section they live in. For example, I have two or more figure in a section labeled as "\label{id9}" and all the figures in this section are labeled as "\label{id9}". Is that normal?
    2. Following your template, I tried to create a similar to "Figure" paragraph style for Equations and Tables. Again when I am compiling the document the labels have the same id number with the section that they belong.
    3. Finally, I get that you made label numbers produced automatically by Scrivener for every object. My questions is, let's say that while I am writing some text in the editor, I want to refer to one figure/equation/table that either is included in the current section or in another section/subsection/chapter in my Draft. Since the label id numbers are not known to me, which are the steps I should follow to refer to them while I am writing in Scrivener?
    The steps that I followed to create "Equation" and "Table" environments are: File -> Compile -> Edit Format -> Styles -> add new style -> set Prefix/suffix as shown below.

    • for Equations:

      Code: Select all

      \begin{equation} \label{id<$linkID>}

      \end{equation}
    • for Tables:

      Code: Select all

      \begin{table}[h!]
        \begin{center}
          \caption{<$custom:Caption>}
          \label{id<$linkID>}


        \end{center}
      \end{table}
    and then I go back to the main editor, press the styles button in the format bar and add the styles having the same name as the those created in the previous process. I have also added a highlight box so as to see where exactly is the in-text position of those objects in.
  • Fonts:
    Only raw italics are handled specially, in that Markup compile format pane. Everything else must be style based. I’ve included a few examples, strong/emph/texttt—and pointed out how to make new environments in the help file.

    Some of the font styles that are regularly used are: bold (\textbf{...} in LaTeX), italics (\textit{...} in LaTeX), underline (\underline{...} in LaTeX). Could you please explain which are the steps to create those font formats and include them in the Styles Panel in Scrivener's Format Bar for ease of use?
  • Comments: Although I have "soul" package loaded, the comments are not included in the LaTeX document. Furthermore, another useful feature that may be included to add to-do notes/comments in the final output is loading the LaTeX commands shown in the second answer in the following forum and work for the article document class: https://tex.stackexchange.com/questions/9796/how-to-add-todo-notes,
    Do you think that it is possible to combine the two techniques? For example, make a comment in Scrivener (Insert -> Comment) and then include the commands shown in the link above (i.e. \info{...}) in the comment text of Scrivener?
  • Preamble and Packages: After spending some time with your template, I think that loading all necessary packages in separate binder documents is much more convenient. So I moved all Packages and LaTeX Commands from the "Text Layout" (except for Document prefix: \documentclass[11pt, a4paper]{article} and Document suffix: \end{document}) pane and put them in my Binder (see attached image).
    Screen Shot 2018-02-10 at 13.14.21.png
    Screen Shot 2018-02-10 at 13.14.21.png (19.44 KiB) Viewed 5549 times
  • References: For bibliography, I use the "References" document in the "Back Matter" and just type the style that I want and the name of my .bib file.

    Code: Select all

    \bibliographystyle{apalike}
    \bibliography{name_of_bib_file}

User avatar
AmberV
Posts: 22061
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Mon Feb 12, 2018 5:39 pm Post

Thanks for the follow-up clarifications.

For example, I have two or more figure in a section labeled as “\label{id9}” and all the figures in this section are labeled as “\label{id9}”. Is that normal?


It doesn’t seem to blow anything up if I try it in LaTeX, but as for whether or not that is best practice, or if it is better to use subfloats—that isn’t something I’ve ever had to do any research into. The WikiBook page on figures seems to go over a good many options.

That aside, where it comes to the usage of this template and the “Figure” section type: if you want a separate caption and label for each figure then each figure should be in its own document. And if you need additional structure to create subfloats or whatever, I think it would work fine to put that structure into the Figure document type. Since all it does is wrap the entire thing in a basic figure environment, you can then add whatever level of detail within it you require.

And if you’re using the style-based approach, just make sure your styles terminate between figures, that the empty line between them is set to “No Style” and not “Figure”. Use of a style background highlights is good for stuff like this, as you note. You can see right where a semantic block begins and terminates.

I tried to create a similar to “Figure” paragraph style for Equations and Tables. Again when I am compiling the document the labels have the same id number with the section that they belong.


The style-based approach is going to be useful if you want to declare your own labels, and you could do that with the provided label style (which itself only inserts the label code, so that can be entered into the editor wherever it is valid to insert \label{…}; in a table, after a heading, etc.).

The automatic labelling, on the other hand, the sort you find with the Figure document type is purely section based. There is no way to generate additional <$linkID>’s for a single chunk of text. That is why your style-based uses of that placeholder do not work properly. I don’t know what your goal is with this style, but in case you are not aware, the use of <$custom:Caption> inside the style prefix may also not work to your benefit. That refers to the current document’s field value in the inspector—thus you could only have one such usage of this style per document. Or to put it another way, all tables in this document would use the same caption with the way you have the style set up.

In short, it would be better to copy from my example of style-based figures for a style-based equation, than to copy from the Section Type approach and try to apply that technique to styles. They use different mechanisms for a reason.

Since the label id numbers are not known to me, which are the steps I should follow to refer to them while I am writing in Scrivener?


That’s what I referred to earlier as making a document link pointing to the section you wish to refer to. If you look in the Markup compile format pane, you see that internal document links have a suffix appended to them to generate an autoref code using the establish <$linkID> for the target item as the bookmark. Thus you don’t have to know the number. You just select some text and hyperlink it to the right section.

If you want to assign your own labels and refer to them by hand, then this approach probably isn’t the best. I would probably swap out all of the <$linkID> stuff from the Layouts and replace them with a custom metadata field for holding the section’s label. Then I’d remove the suffix from the Markup compile format pane. Document links themselves would cease having any functional impact on the output, leaving the particulars of cross-referencing entirely up to you.

I tried to find a way to make this a little more flexible—perhaps assigning an serial number label for internal usage as I described, with Document Links, but also allowing for one to create a custom label they could easily use with raw syntax or other methods elsewhere. I couldn’t find anything for doing that with LaTeX though—essentially having aliased labels, so that one cross-reference can point to “scrivauto:331” and another can point to “mysectionname” and both links resolve to “Section 23.8” and link to the same object. If anyone knows how to do that, or what package would make it possible to do so, I’d appreciate pointers.

Some of the font styles that are regularly used are: bold (\textbf{…} in LaTeX), italics (\textit{…} in LaTeX), underline (\underline{…} in LaTeX). Could you please explain which are the steps to create those font formats and include them in the Styles Panel in Scrivener’s Format Bar for ease of use?


  1. I would use the rich text features to make the text look how I want it represented in the editor—that part is entirely down to your aesthetic choice (though keep in mind that by default italics will be converted, so avoid that formatting unless you mean it, or if you don’t want to mean it, remove the syntax from the Markup compile format pane).
  2. Create a new style from the example text, set to character attributes only and any other features you prefer.
  3. In the Styles compile format pane, select the Emphasis example I provided and click +.
  4. Modify the command name in the prefix/suffix as necessary.

Although I have “soul” package loaded, the comments are not included in the LaTeX document.


Comments and inline annotations are by default stripped out of the compiled document as a global default, no matter the format you are using—the format only defines how comments are treated if they are included. Make sure they are, in the compile overview area, under general options.

After spending some time with your template, I think that loading all necessary packages in separate binder documents is much more convenient.


Yeah that’s probably going to be a choice everyone makes for themselves. What I’m trying to promote, in the way it is set up by default, is the concept that the Format is fully responsible for those aspects of the preamble that determine the overall look and basic function of the document. Those things that are particular to the document regardless of its presentation are better handled in the project itself. This way the format can be globally applied to many projects, and projects can easily switch between publication standards without breaking.

Here are two examples for how I think of it:

  • If I want to change how bullets look, I would put that in the Text Layout pane. It’s not going to break my project if I switch do a different Format, it will merely change how the itemisation looks when I compile.
  • If I write a custom environment like \callout and use packages and configuration to make that call-out box, then all of that is something that the document itself needs to remain functional and not throw errors if I switch up the document class or paper size. So for that I’d make an entry in the front matter folder and call it “Call-Out Boxes”. It’s a nice little compartmentalised package I can drag and drop into future projects as well, if I want.

I suppose another way of looking at it would be that if one were inclined to use \inputs in their preamble to make cross-publication easier then that’s the sort of scenario where delegating things to the Format might be a better approach. If one need only format for one publication, then the convenience of having the whole document essentially in the binder is undeniable.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

pe
pekak
Posts: 4
Joined: Wed Dec 26, 2007 11:08 pm
Platform: Mac

Mon Apr 23, 2018 8:38 pm Post

This is usable solution, thanks.

I wish that I could add post-processing script to it, seems arbitrary that only mmd/pandoc route has that option. Any hope for post-processing with text too?

Thanks,
Pekka

ia
ianjgrant
Posts: 7
Joined: Mon Nov 05, 2012 11:04 am
Platform: Mac

Tue Jun 05, 2018 10:35 pm Post

Hi, this is fab.

I've got it working nicely for a paper and pre-existing header, preamble, and footer, etc.

A quick question: where/how do the hyperlinks get transformed to

Code: Select all

\href{<link>}{<link>}
?

I'd like to edit the rule to produce

Code: Select all

 \url{<link>}


I'm using the xurl package

Code: Select all

\usepackage{xurl}
which breaks up long URLs across lines and it seems to work nicely.

Thanks,

ps. I've also converted an MMD / Scrivener 2 SIGGRAPH template to this workflow that seems to be working nicely. Happy to share (in a new thread)...

User avatar
nontroppo
Posts: 1034
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Wed Jun 06, 2018 12:57 am Post

Compile Format ▶︎ Markup ▶︎ External Link
Screen Shot 2018-06-06 at 08.48.58_SMALL.png
Screen Shot 2018-06-06 at 08.48.58_SMALL.png (22.2 KiB) Viewed 2910 times

ia
ianjgrant
Posts: 7
Joined: Mon Nov 05, 2012 11:04 am
Platform: Mac

Wed Jun 06, 2018 1:41 pm Post

Thanks Nontroppo, I missed that (thinking I wasn't using any markup or markdown)!
I also solved it another way by removing the automatic hyperlink, creating and applying a URL character style and adding a prefix

Code: Select all

\url {


and suffix

Code: Select all

}


Neat. I like the new styles, section types and the rest. And the modular way you can build LaTex Controls.
Thanks again, Ian

User avatar
AmberV
Posts: 22061
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Wed Jun 06, 2018 2:34 pm Post

We’re using the word “markup” in its broader definition here, for this pane, where even LaTeX itself falls under the scope of being markup. :)

pekak wrote:I wish that I could add post-processing script to it, seems arbitrary that only mmd/pandoc route has that option. Any hope for post-processing with text too?


The plain-text mode will be getting access to the Processing pane in the next update. So you would be able to, among other things, graft a mklatex or pdflatex workflow into the compile settings and end up with a PDF instead of a .tex file, if you want.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles