Tutoring/job post: Learning How To Scrivener to LaTeX to PDF

la
laleleona
Posts: 29
Joined: Mon Apr 15, 2019 8:43 am
Platform: Mac

Fri Feb 19, 2021 11:33 am Post

I am searching for someone to help me learn how to compile my Scrivener project file (a PhD thesis) via MMD/Pandoc to LaTeX to PDF - either as a kind of commissioned work or, preferably, through tutoring me (which of course I will pay for).

The project is only text and a few images, no equations involved. I think it should be fairly easy for someone who knows how to do it – but I'm having trouble figuring it out on my own. I've previously worked with LaTeX (very basic templates though), but never with MMD or Pandoc. And there are also quite a few things about Scrivener /compiling with Scrivener that I don't really get.

If you can help in any way and have even just 1 hour to spare I would very much appreciate your help. I've also posted a job post on UpWork about this but it seems like Scrivener/LaTeX integration is a rare skill. So i would very much appreciate if you could help me out

User avatar
bernardo vasconcelos
Posts: 18
Joined: Fri Aug 24, 2018 10:21 am
Platform: Mac

Fri Feb 19, 2021 7:09 pm Post

Hi, welcome.

I am also finishing up my thesis.

Have you read the manual? It is really detailed and I never failed to find what I was looking for there.

sc
scrive
Posts: 52
Joined: Mon Jul 06, 2020 8:05 am
Platform: Mac

Sat Feb 20, 2021 8:29 am Post

laleleona wrote:I am searching for someone to help me learn how to compile my Scrivener project file (a PhD thesis) via MMD/Pandoc to LaTeX to PDF - either as a kind of commissioned work or, preferably, through tutoring me (which of course I will pay for).

The project is only text and a few images, no equations involved. I think it should be fairly easy for someone who knows how to do it – but I'm having trouble figuring it out on my own. I've previously worked with LaTeX (very basic templates though), but never with MMD or Pandoc. And there are also quite a few things about Scrivener /compiling with Scrivener that I don't really get.

If you can help in any way and have even just 1 hour to spare I would very much appreciate your help. I've also posted a job post on UpWork about this but it seems like Scrivener/LaTeX integration is a rare skill. So i would very much appreciate if you could help me out

There are a number of amazing resources for TeX and LaTeX including:

    1) TUG (TeX Users Group) has an excellent tutorial "Getting started with TeX, LaTeX, and friends" on how to get started with LaTeX at https://www.tug.org/begin.html. To get serious with LaTeX, you might want to follow TUG's instructions on how to install TeX Live for Unix/GNU/Linux, MacTeX for MacOSX, and proTeXt for Windows.
    2) Overleaf, the Online LaTeX Editor at https://www.overleaf.com has a wonderful section entitled "Learn LaTeX in 30 minutes" at https://www.overleaf.com/learn/latex/Learn_LaTeX_in_30_minutes.
    3) Once you've learned the basics, Stackexchange for Tex at https://tex.stackexchange.com can be invaluable when searching for answers. If you sign up you can post questions there for some guidance that may help.
    4) On the Apple platform, I use Texpad : LaTeX edito‪r for MWE's (Minimum Working Examples) with built-in PDF image rendering, but check the reviews, your mileage may vary.

TUG is free, and offers membership for a small fee along with printed documentation available on their home page at https://www.tug.org/index.html. Both Overleaf and Stackexchange are free for basic services (Overleaf and Stackexchange both have advanced plan/features by paid subscription).

After over a year working with both software packages, I am still on a VERY steep learning curve with Scrivener+LaTeX (and will likely be for some time), but the possibilities are mind boggling. What I have found, particularly with Stackexchange, is akin to when Will Smith as Del Spooner in the movie "I, Robot" is told by Dr. Alfred Lanning (an AI)*:
I'm sorry. My responses are limited. You must ask the right questions.

Apologies if my response is a bit too prophetic, but if you have the bandwidth, the power of the Scrivener+LaTeX software combination for documents such as a thesis is 'off the charts'.

Hope this is of some help,
scrive
:D

* https://www.imdb.com/title/tt0343818/characters/nm0000226

User avatar
xiamenese
Posts: 4710
Joined: Mon Jan 29, 2007 1:32 am
Platform: Mac
Location: London or Exeter, UK.

Sat Feb 20, 2021 9:30 am Post

I can’t help myself, but search this forum for "scrivomatic", a Scrivener —> Pandoc ~~> output system set up by @nontroppo. Could that help?

:)

Mark

Edit: "scrivomatic" must be all lower-case, without the quote marks of course. :)
The Scrivenato sometimes known as Mr X.
M1 MacBook Air (2021), 11.2.1, 16GB RAM, !TB SSID
iMac 27" (late 2015) 10.15.7, 24GB RAM, 512GB SSID
2017 iPad, iPadOS 14.3, 128GB, Apple Pencil
Scrivener, Scapple, Nisus Writer Pro, Bookends …

la
laleleona
Posts: 29
Joined: Mon Apr 15, 2019 8:43 am
Platform: Mac

Sun Feb 28, 2021 9:23 am Post

Thanks a lot for these helpful comments. I am familiar with basic LaTeX and have been working with Scrivener for a while (though haven't gotten familiar enough with things like compile format). But I've now started trying to work my way through the relevant sections of the user guide and iandol's scrivomatic...

I've never worked with MMD and Pandoc, so that's a little overwhelming. But I am getting the feeling that that's the only method that makes sense for scientific work / if you need to handle a large number of references – am I right? Or is there any way the built-in Scrivener format MMD to LaTeX memoir class can do that?

Finally I didn't quite get what you meant by:
Apologies if my response is a bit too prophetic, but if you have the bandwidth, the power of the Scrivener+LaTeX software combination for documents such as a thesis is 'off the charts'.

Could you elaborate? Thanks a lot

All best

mb
mbbntu
Posts: 1306
Joined: Wed Aug 01, 2007 9:44 am
Platform: Mac + iOS
Location: Cambridge, UK.

Sun Feb 28, 2021 11:20 am Post

I find myself wondering if it is strictly necessary to use LaTeX. My thesis never went within a million miles of LaTex, I just compiled to RTF and went from there. It was a long time ago, it is true, and LaTeX was probably less fashionable then, but I wonder if the process needs to be that complicated. To my mind, the words are rather more important than the layout.
You should judge people not by how close they get to the top, but by how far they have come from the bottom. Some people have a mountain to climb just to get to the place where others start out. (Me, 2010)

User avatar
kewms
Posts: 7188
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Sun Feb 28, 2021 4:45 pm Post

mbbntu wrote:I find myself wondering if it is strictly necessary to use LaTeX. My thesis never went within a million miles of LaTex, I just compiled to RTF and went from there. It was a long time ago, it is true, and LaTeX was probably less fashionable then, but I wonder if the process needs to be that complicated. To my mind, the words are rather more important than the layout.


For the OP's project -- "only text and a few images," -- probably not. Depending of course on such details as what they're using for a bibliography manager and what their university's requirements are.

There are certainly formatting tasks that the base Scrivener engine can't handle, or that would be more easily done with other tools. And there are people who are already comfortable with LaTeX, Markdown, and similar. But I do think that sometimes people overestimate the complexity of their projects and underestimate the capabilities of Scrivener, leading to unnecessarily complex workflows.

Katherine
Scrivener Support Team

sc
scrive
Posts: 52
Joined: Mon Jul 06, 2020 8:05 am
Platform: Mac

Mon Mar 01, 2021 12:23 am Post

kewms wrote:
mbbntu wrote:I find myself wondering if it is strictly necessary to use LaTeX. My thesis never went within a million miles of LaTex, I just compiled to RTF and went from there. It was a long time ago, it is true, and LaTeX was probably less fashionable then, but I wonder if the process needs to be that complicated. To my mind, the words are rather more important than the layout.

For the OP's project -- "only text and a few images," -- probably not. Depending of course on such details as what they're using for a bibliography manager and what their university's requirements are.

There are certainly formatting tasks that the base Scrivener engine can't handle, or that would be more easily done with other tools. And there are people who are already comfortable with LaTeX, Markdown, and similar. But I do think that sometimes people overestimate the complexity of their projects and underestimate the capabilities of Scrivener, leading to unnecessarily complex workflows.

Katherine

@laleleona,

I would agree with Katherine, her assessment said it best ... if your project is one with "only text and a few images," the added flexibility that Scrivener+LaTeX could offer may not justify the upfront and ongoing time investment that LaTeX likely will require.

In response to your query re:
Finally I didn't quite get what you meant by:
Apologies if my response is a bit too prophetic, but if you have the bandwidth, the power of the Scrivener+LaTeX software combination for documents such as a thesis is 'off the charts'.

Could you elaborate? Thanks a lot

Re my use of the term 'bandwidth': my use of the term was meant, perhaps too loosely, to indicate the time requirement for an undertaking such as learning LaTeX. A thesis can often be a challenging undertaking in it's own right, and taking on the added task to learn LaTeX can simply overcomplicate the mission that is a thesis.

To be specific, IMHO, Scrivener+LaTeX is NOT the only method that makes sense for scientific work.

Please accept my apologies if in my enthusiasm for Scrivener+LaTeX, I have over-hyped the experiences I have had using the two together.

All the best,
scrive
:D

la
laleleona
Posts: 29
Joined: Mon Apr 15, 2019 8:43 am
Platform: Mac

Mon Mar 01, 2021 12:58 pm Post

Thanks to both of your for your replies. Agreed, the content is more important, but it's intertwined with the form ! In my case I don't have a lot of formulas (so I don't necessarily need to do it in LaTeX) but my university requires me to do the typesetting and layout of the final book, so in any case I will have to do some type of typesetting and post-processing work.

The alternative would be MS Word (is that what you would suggest?) – and I'm miserable with Word. Plus I hate the look of Word documents, their typesetting is so much more unpleasant to read... From my very preliminary shots at just "intuitive" compiling (having read nothing about compiling yet in the Scrivener Manual), I also get the sense that compiling to Word requires quite a bit of work (e.g. designing and assigning the section layouts, my first attempts literally look like s***) while compiling via MMD-> LaTeX using the memoir class format already works pretty well, with lots of things being automated, and yields an okay-looking book (with some errors and issues to be solved yet). So I'm not really sure that an alternative would be less work?

After these first few tries, I think I'm becoming more confident that by reading the user manual and the online resources e.g. on pandoc and scrivomatic I can manage to resolve those issues and compile the book via MMD/pandox to LaTeX and from there typeset it into a beautifully looking book. I think I forgot to mention that I've worked with LaTeX before/typeset my MA thesis via LaTeX. I also wrote a LaTeX template for how I want my thesis to look, but I don't want to write in TexShop but instead want to make use of the great writing tool that is Scrivener. So the issue for me is mainly learning how the two interlink via MMD. I guess I'll be back here posting my questions over the next weeks as I complete the project :-)

User avatar
AmberV
Posts: 24669
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:

Mon Mar 01, 2021 2:33 pm Post

Yes, I would say that if LaTeX is your comfort zone, there is nothing about how Scrivener is set up that would get in the way of you using it. And personally I would always prefer to use that to any kind of Word-based workflow, even for very simple (from a formatting and content standpoint) documents. As you say, the typesetting is nowhere near as good, and really if one is going that route they should be using desktop publishing to handle the final formatting rather than Word. Compiling is also, as you say, a bit more complicated if you’re used to a simpler markup based environment. You also benefit from better cross-referencing than the stock Scrivener setup provides, being able to refer to specific equations and figures by label.

At the most basic level of usage, imagine Scrivener is a plain .tex file editor and compile to plain-text. That’s all there is to it, though that very basic level of usage does mean composing in raw LaTeX for all things.

You can then build up from that basic concept by delegating routine syntax output to the compiler. The built in “General Non-Fiction (LaTeX)” starter template is a good place to go if you want to see an example of that pressed to the limits (for example having headings turned into \section{…} code with automatically generated labels and functional internal cross-referencing). This template is designed for pure LaTeX-based writing, rather than using Markdown as an intermediate generator. However one could get away with very little raw LaTeX in the editor (you do still have to be mindful of the fact that what you type into the editor will be dumped straight into the .tex file—which wouldn’t be seen as a downside by some).

Of course, because we are converting Scrivener’s outline to heading structure, it does need a little more Section Type / Layout wiring (though you’ll find it is set up with suitable defaults). Not having to worry about that at all is one of the chief advantages to using Markdown, I’d say.

For a Markdown-based workflow, I tend to start extremely simple, but if you select “MultiMarkdown → LaTeX” as your compile output, you’ll note a number of starting point Formats you can choose from in the left sidebar. Those are all pretty vanilla and there is a bit of a learning curve in customising them. Generally I’d say take a look at how the “Modern” example format is set up (double-click to duplicate and edit it), where you can put your preamble and such right into the compile settings. For most things, that will be the easiest way to go.

Since it sounds like you have a preamble already developed, that’s what I’d take a look at! I do still use a second tool to typeset, but do note from the LaTeX template’s “Processing” compile format pane, you could automate straight to PDF if you wanted to. I provide a little sample script for that.

laleleona wrote:I guess I’ll be back here posting my questions over the next weeks as I complete the project…


By all means!
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

User avatar
kewms
Posts: 7188
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Mon Mar 01, 2021 6:10 pm Post

laleleona wrote:The alternative would be MS Word (is that what you would suggest?) – and I'm miserable with Word. Plus I hate the look of Word documents, their typesetting is so much more unpleasant to read... From my very preliminary shots at just "intuitive" compiling (having read nothing about compiling yet in the Scrivener Manual), I also get the sense that compiling to Word requires quite a bit of work (e.g. designing and assigning the section layouts, my first attempts literally look like s***) while compiling via MMD-> LaTeX using the memoir class format already works pretty well, with lots of things being automated, and yields an okay-looking book (with some errors and issues to be solved yet). So I'm not really sure that an alternative would be less work?


Scrivener supplies "reasonable" starting Compile Formats for both Word and MMD -> LaTeX output. You're certainly not starting from scratch if you use Word.

Katherine
Scrivener Support Team

la
laleleona
Posts: 29
Joined: Mon Apr 15, 2019 8:43 am
Platform: Mac

Fri Mar 05, 2021 10:36 am Post

Thanks a lot for breaking it down analytically, that makes sense. I had understood how you could use Scrivener as a plain text editor and then compile to PDF via LaTeX and I had tried out the LaTeX template. The first seemed like a waste of Scrivener (I would only be able to use the beautiful interface that Scrivener has, but not the powerful word processing tool that it is) , the second seemed too limited for long documents, with citations, etc. Plus I also want to produce Word files since my collaborators mainly use Word. So MMD/Pandoc is where it got both confusing and attractive.

I've now done some more reading and got a basic understanding of how in such a workflow Scrivener treats my scrivenings and binder titles/headers as plain text and generates the relevant markdown surrounding that text using styles (or the compile replacements for the things that aren't done in styles) and how I could then use Pandoc to convert the generated markdown into Word, PDF via LaTeX etc. (or pandocomatic and scrivomatic if I want to automate that step and simultaneously want to get various files using my preferred format and template). I've also tried to work through Ian's scrivomatic documentation but cannot yet manage to compile successfully. In general there are still quite a few things I don't understand (e.g. i've never worked with any of the other scripts required for the metadata, etc. although they seem learnable), and it also seems like I would have to learn some more things about the post-processing, configuring a yaml file, setting up customized LaTeX templates etc. (whether as part of a compile format preamble/footer - or as part of a pandocomatic template - I am guessing these are basically interchangeable)

AmberV wrote:Since it sounds like you have a preamble already developed, that’s what I’d take a look at! I do still use a second tool to typeset, but do note from the LaTeX template’s “Processing” compile format pane, you could automate straight to PDF if you wanted to. I provide a little sample script for that.


I've also had a go at your suggestion and pasted the LaTeX preamble and footer of my previous setup into a copy of the "Modern" compile format. That works and generates a decent-looking book- but I'm realizing that my template may be toolimited and doesn't make use (does it have to?). E.g. it doesn't have any crossreferencing yet. So I am not sure my previous setup is a better place to start from than someone else's . I also noticed that the preamble and footer in the original "Modern" compile format are vastly different from my template, making large use of conditional syntax. There are big chunks I don't yet understand

So I'm wondering what you would recommend as a next step to keep learning (while making it feasible to produce a good book in 2-3 months time...)? I could try and first understand the syntax of the Modern compile format (I don't love the look of it though but of course that can be changed). Or I could try and further understand scrivomatic pandocomatic and work through Ian's yaml file, check out the templates and adapt them. Or I could take a more bottom-up approach and learn from a more basic structure to get to the complex kind of document i want to reach... any advice would be much appreciated!

Oh and also...
AmberV wrote:Yes, I would say that if LaTeX is your comfort zone!

does it ever become that? If so, I'm looking forward to reaching that divine state

User avatar
AmberV
Posts: 24669
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:

Fri Mar 05, 2021 4:59 pm Post

The first seemed like a waste of Scrivener (I would only be able to use the beautiful interface that Scrivener has, but not the powerful word processing tool that it is)…


From my perspective, Scrivener has very little to do with any of that stuff and is all about the incredibly rich writing platform it provides. How you choose to use the text editor is almost beside the point.

Pandoc probably is the perfect conjunction you’re looking for though, I agree.

That works and generates a decent-looking book- but I’m realizing that my template may be toolimited and doesn’t make use (does it have to?). E.g. it doesn’t have any crossreferencing yet. So I am not sure my previous setup is a better place to start from than someone else’s .


If you’re referring specifically to being able to put down an internal hyperlink in Scrivener’s editor and have that come out as Markdown, then you don’t need a template for that. Just go into your compile settings, and in the general options tab on the right, enable the setting to convert links. You might also want to add table and list conversion too, if you have need of those constructs.

So that’s a benefit to using Markdown—Scrivener does quite a lot of conversion out of the box (even a rather comprehensive full formatting conversion approach as an option).

I also noticed that the preamble and footer in the original “Modern” compile format are vastly different from my template, making large use of conditional syntax. There are big chunks I don’t yet understand


As I recall the main conditional aspect is to switch the font settings depending on whether one is typesetting with XeLaTeX. In that case it will use the same exact Mac system fonts that the regular Modern compile format does, when writing RTF-style. Other than that, I don’t think I do too many crazy things with it, just tweak some spacing and such to make it resemble Scrivener’s “Modern” format.

That aside, if you’re going for academic output and have need of citations and such, I think the learning curve on Scrivomatic is going to be a lot more rewarding down the line. The designer of it is very active on the forums; in fact here is an ongoing thread.

“Modern” was mainly thrown together to provide a simple example of how you could put together such a custom look yourself, and to maybe serve as a nice out of the box casual design for those that want it, too.

does it ever become that? If so, I’m looking forward to reaching that divine state


:lol: I don’t think that is something that happens, except perhaps to Donald Knuth and Leslie Lamport! Relatively speaking though, I’ll take it over a word processor any day.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

la
laleleona
Posts: 29
Joined: Mon Apr 15, 2019 8:43 am
Platform: Mac

Sat Mar 06, 2021 4:26 pm Post

Thanks a lot for clarifications and advice. Again very helpful!

AmberV wrote:That aside, if you’re going for academic output and have need of citations and such, I think the learning curve on Scrivomatic is going to be a lot more rewarding down the line.


Agreed ! And I even managed to replicate Ian's workflow, so that's good news and I can start figuring out how to adapt it for my purposes. I learned a lot from his documentation and the user guide, thanks a lot !

Now I was wondering: What difference is there between setting up a custom latex template (like in Ian's workflow) and having a customized footer, header, etc. (like in your workflow for the user guide)? Or do they amount basically to the same thing (for most purposes)?

I am asking because I'm thinking perhaps I should start simple and customize my own LaTeX compile format, design the header and footer the way i like and get that to work perfectly when compiling to MMD->PDF, before I move on t to implementing that with scrivomatic. I guess I'd then copy the footer, header and document begin from my compile format into a template that Pandoc can call on when running scrivomatic. Would that be correct ?