## Latex output "Could not convert to required format"

Mi
MikeA29
Posts: 14
Joined: Fri Mar 01, 2019 7:55 pm
Platform: Mac + Windows
Compile for: MultiMarkdown Latex. It compiles, followed by Select encoding pop-up. Document converter pop-up window then reports Unknown error.

Version: 2.9.0.20 Beta (640426) 64-bit - downloaded 03 Aug 2019. Used for producing technical book with several chapters and sections. After removing problem text, everything else compiles okay.

This sentence with reference links does not work:

It must pass [HTML5][lnk-htmlvalid01] and [CSS][lnk-cssvalid01] validation.

[lnk-htmlvalid01]: https://validator.w3.org/
[lnk-cssvalid]: https://jigsaw.w3.org/css-validator/

This works:
It must pass [HTML5][lnk-htmlvalid01] and [CSS][lnk-cssvalid] validation.

[lnk-htmlvalid01]: https://validator.w3.org/
[lnk-cssvalid]: https://jigsaw.w3.org/css-validator/

Notice numbers removed in second link. I removed text to Windows Notepad to clean out possible unwanted characters, then repasted it into Scrivener. Same problem.

HTH...

Mike A.

AmberV
Posts: 24216
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:
Notepad may not clear out all unwanted characters A surer way of stripping out invisible control codes is Edit ▸ Text Tidying ▸ Zap Gremlins on the problematic text.

I’m not sure what the issue is—obviously the referenced link is broken in the first sample, but that shouldn’t throw any errors—I’m not seeing any over here at any rate. I get a .tex file with raw Markdown reference link visible in the output, as expected.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

Mi
MikeA29
Posts: 14
Joined: Fri Mar 01, 2019 7:55 pm
Platform: Mac + Windows
I've occasionally observed the original issue across different versions of Scrivener and think I've identified one part of what's going awry. I'm currently on "Version: 2.9.0.30 Beta (757109) 64-bit - 28 Nov 2019", same issue.

Background: I use Scrivener to write a lot of technical information then compile out for Latex. Most work involves MMD. I also use comment marks to encapsulate code for compiling as-is to Latex, for example...

Code: Select all

<!-- \begin{minted}[tabsize=2,breaklines]{php} <!DOCTYPE html><html lang="{{ str_replace('_', '-', app()->getLocale()) }}"><head>...</head><body>...</body></html>\end{minted}-->

The html lang code line contains double curly braces. That's the problem!

If I escape at the second brace, like {\{ or }\} , it compiles.

I wonder if this can be overcome. It's a nuisance. However, I recollect that Scrivener employs double curly braces for containing footnotes.

Hope this helps someone!

PS: When Scrivener does it's curly brace baulk and refuses to compile, it also stops TexStudio working. To recompile from Scrivener and catch output in TexStudio I must first restart TexStudio.

AmberV
Posts: 24216
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:
One thing that might be throwing your result off is that the newer versions of MMD no longer use HTML comments to pass-thru raw LaTeX code (granted, if you installed your own copy of MMD to a standard location in the past, it may be using this older version instead of the built-in copy). They now merely operate as HTML comments and nothing more. The way to do this now is:

Code: Select all

{=latex}\stuff{ ... }

Scrivener only uses double-braces when syncing with plain-text using the folder sync feature. This setting should have no impact on compiled output. But you can always double-check to make sure Scrivener isn’t doing anything odd with your output by compiling to a plain .md file rather than having Scrivener also run MMD to convert its output to .tex. (It’s also worth noting that as of the last build, I detected that it isn’t actually passing the same .md file through to MMD that it does when compiling plain; there are still issues to be resolved with the automated conversion choices.)
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

Mi
MikeA29
Posts: 14
Joined: Fri Mar 01, 2019 7:55 pm
Platform: Mac + Windows
...the newer versions of MMD no longer use HTML comments to pass-thru raw LaTeX code.

I can't say how many times I swore when I saw that. Thank you Amber! It explains why using HTML comments within code passed through to Latex was cutting off at -->.

Continuing the research for those who may have an interest and Scrivener developers ...

How to pass this through to LaTex?...

Code: Select all

<!DOCTYPE html><html lang="{{ str_replace('_', '-', app()->getLocale()) }}"><head>    <meta charset="utf-8">    <meta name="viewport" content="width=device-width, initial-scale=1">    <title>Laravel</title>    <!-- Styles -->    <link rel="stylesheet" id="css-main" href="test.css"></head><body>        <div class="links">            <a href="https://theme01a">Back to home page</a>        </div></body></html>

Line 2 has curly braces. It throws a Scrivener compilation error when compiling MMD to LaTex, whether in Latex pass-through style using <!-- {code} --> or code block back-ticks html {code}. On Windows 7, that is a Scrivener compilation problem to be fixed, unless I've missed a setting somewhere. Try it. It's an important issue for me because it precludes using Scrivener to write publications with code for layout frameworks like Blade, Twig and anything else that wraps blocks in {{ }}.

Line 8 has a comment <!-- Styles -->, which breaks LaTex output at -->,

Turning to the current Scrivener compile to MMD LaTex...

Code: Select all

{=latex}<code>

...will not work. It produces...

Code: Select all

\begin{lstlisting}[language={=html}]

...which LaTex compilers baulk at (I use TexStudio BTW). Or have I missed a Scrivener setting somewhere? Of course, it works when using LaTex Listings package after correcting syntax, but using that package without style formats is worse than useless because it outputs bland text instead of colourful well behaved code listing (like the LaTex Minted package). Turning to an example, using the correct format in Scrivener...

Code: Select all

html,style=htmlcssjs

...htmlcssjs being a style formatted by me, produces...

Code: Select all

\begin{lstlisting}[language=html,style=htmlcssjs]

However, this part of Scrivener's approach seems wrong on so many levels. Forcing users to employ a specific command (lstlisting) in another software package is the developer version of devil worship (apologies for disturbing if you were out in the woods on a sacrifice!). FYI, locking LaTex users into the Listings package comes with a barrow load of issues including code blocks mot properly breaking across pages and the need for authors to undertake style formatting across a range of languages. Also, it kills Scrivener's potential role in automated processing of technical material (via CI/CD), for which I've almost developed a product that requires users to purchase Scrivener (nudge, nudge, hint).

May I suggest either dumping the...

Code: Select all

{lstlisting}[language=

...automatically compiled into output after code block ticks, and replacing it with a plain  code block suffix that can be followed by any code to be passed through, ergo...

Code: Select all

{minted}[tabsize=2,breaklines]{php}<code>

...to result in a correctly compiled output for LaTex...

Code: Select all

 \begin{minted}[tabsize=2,breaklines]{php} <code>\end{minted}

...or whatever MMD to LaTex compilation pass-through is required.

Or have I missed something after looking through every Scrivener setting I can find? If someone has a way to compile that first code block, I'd really appreciate knowing about it!

To the devs: if it is a Scrivener R&D issue, it would be wonderful to have this matter resolved quickly! I hope I explained it well enough to assist!

Good wishes!

AmberV
Posts: 24216
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:
Unfortunately I don’t have Windows 7 to test with, only Windows 10—and I can insert thousands of curly braces without the compiler throwing an error. This seems like a very specific thing to be OS dependent upon though—are you sure that MMD execution works in general, you can use the other output choices, and if you just type in one word into a blank project it compiles to .tex—only fails when braces are involved?

If so, it’s probably not a Scrivener error but a bug in MMD (I’m not sure if they have implemented more informative command-line failure reporting yet, or if it is still just a generic dialogue). One way you’d test for that is by compiling an .md file from Scrivener and then converting it to .tex with multimarkdown.exe on the shell.

But you can also get more detailed logging output by enabling Show internal error alerts, in Settings: General: Warnings.

And further along that vein, many of the things you are reporting here are specific interactions of MultiMarkdown syntax and how it generates .tex. Scrivener knows absolutely nothing about LaTeX, minted, and even very very little about Markdown itself. It’s by and large leaving all of that up to you and passing your content through the conversion engine.

If you aren’t pleased with how MMD generates minted code, you will have a much better chance of response to your feedback on the MMD Support board.

Also, it kills Scrivener’s potential role in automated processing of technical material (via CI/CD), for which I’ve almost developed a product that requires users to purchase Scrivener (nudge, nudge, hint).

Well on that matter specifically, since you do have precise output control of LaTeX via the pass-thru syntax, you are not limited one bit by MMD’s decisions of which package to use, etc. And in fact, Scrivener is the antidote to this problem. You’re just at the moment conflating Scrivener with MMD, which is totally wrong. The actual fact of the matter is that Scrivener is a capable generator for LaTeX syntax, as well as MMD, etc., while abstracting that generation from the writing mechanism, into the compiler. It just doesn’t do most of that by default—you have to tell it what you want.

There are a number of tools for doing this, but the two you’ll want to use the most are:

• Styles: in the Styles compile format pane, note the paragraph and overall prefix/suffix fields. Here is where you can insert custom output based on simple semantic markings made in the editor. Text can be marked “Code Block (HTML)” in the editor, and then you can implement a precise, custom minted environment you’ve created, via the compile settings. Or you can create a different compile format that exports these to Github Markdown.
• Section Layouts: if styles are the semantic tool of choice at the text level, layouts are how you encode specific implementations into the outline itself. You could create an “HTML Code Block” section type in your project, and put your snippets into the binder as discrete outline items. Then in the compiler you would design how these types of items should be printed. Again, the choice is entirely yours, and with the pass-thru syntax you can implement detailed custom code to any of the output formats. If you are creating an ODT document, you can implement custom ODT XML into the document output, for example. In fact I use this technique in some of our provided compile format examples—pop open the “MMD OpenOffice Document” compile format and examine the styles I’ve defined for indexing. These inject raw ODT XML to help a user build an index at the end of their book.

So the key things to take away here are: Scrivener can generate specific document implementations through compiler setups, which means the content can remain safely abstracted from these details. If MMD doesn’t do something at all (like indexing) then you can add that “feature” to your syntax or your stylesheet. If you don’t like how it handles code blocks, you can roll your own.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

Mi
MikeA29
Posts: 14
Joined: Fri Mar 01, 2019 7:55 pm
Platform: Mac + Windows
Thank you again Amber. Very helpful. There's a lot for me to mull over and maybe come back upon if on reflection it would seem helpful to others.

Did you try the code example I provided?

Code: Select all

<!DOCTYPE html><html lang="{{ str_replace('_', '-', app()->getLocale()) }}"><head>    <meta charset="utf-8">    <meta name="viewport" content="width=device-width, initial-scale=1">    <title>Laravel</title>    <!-- Styles -->    <link rel="stylesheet" id="css-main" href="test.css"></head><body>        <div class="links">            <a href="https://theme01a">Back to home page</a>        </div></body></html>

I ask because I'm interested in what comes out on other systems, if you or someone here doesn't mind trying to compile it using MMD->LaTex.

AmberV
Posts: 24216
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:
Yes, as noted I ran into no difficulties with either that or more extreme tests with brackets.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

Mi
MikeA29
Posts: 14
Joined: Fri Mar 01, 2019 7:55 pm
Platform: Mac + Windows
Yes, as noted I ran into no difficulties with either that or more extreme tests with brackets.

I put this out to my developers around the world. Two responded. They are getting the same as me. Would you mind providing feedback on what your compile config is, and what you surrounded my example code block with?

One thing is absolutely certain with a clean install of Scrivener Beta 30 and a simple block of code (already provided) using a MMD->LaTex compile: it's not a LaTex or MMD issue when compared to same text in Beta 19-22. This worked when <!-- --> HTML comments were used. Now it doesn't.

Hope you don't mind me asking: are you one of the technical team withing L&L?

Separately (off topic), using Debug, which you so helpfully suggested, I see...

Remote host is configured as localhost despite server host is probably not local

Am I missing a setting somewhere? I can't imagine this would affect the primary issue, but just in case...

AmberV
Posts: 24216
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:
Hmm, I had no problems compiling to LaTeX in an earlier build (which may have been more recent than b30), but now I am—so maybe it was a temporary condition. At any rate, the problem appears to have nothing to do with content, on my end.

Here’s a way of putting it: can you compile at all to LaTeX? I couldn’t get a single word “test” out of it—nor did .odt or .html work for that matter. Here is what I got from the debug log:

Code: Select all

Debug: Executed: "C:\\Program Files\\Scrivener\\docformats\\multimarkdown.exe"Debug: Arguments: "-t latex -o C:\\Users\\Test Account\\Desktop\\test.md C:\\Users\\Test Account\\Desktop\\test.md"Debug: Working Dir: "C:\\Program Files\\Scrivener\\docformats"Debug: StdOut: ""Debug: StdErr: "Error reading file 'C:\\Users\\Test Account\\Desktop\\test.md'\r\n"`

There are potentially two issues going on here:

• The working folder looks a bit odd to me. I’m not sure if it is typical practice to be using the application’s own installation folder as a working area. Seems risky to me, but it might also lead to permission issues.
• The actual file that is passed to the shell appears to include a Windows style carriage return on the end of it. Maybe that is just a logging error, but if it isn’t that’s surely not right.

I’ll get both of these issues reported.

Remote host is configured as localhost despite server host is probably not local

I have no clue, that sentence doesn’t even make sense to me. This is what you got when compiling? Are you using an NAS for the output folder or anything of that nature?

Hope you don’t mind me asking: are you one of the technical team withing L&L?

Yes, sorry if that wasn’t made clear.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

devinganger
Posts: 2001
Joined: Sat Nov 06, 2010 1:55 pm
Platform: Mac, Win + iOS
Location: Monroe, WA 98272 (CN97au)
Contact:
AmberV wrote:The working folder looks a bit odd to me. I’m not sure if it is typical practice to be using the application’s own installation folder as a working area. Seems risky to me, but it might also lead to permission issues.

Yeah, folders under the shared installation folders should not be used for temporary data -- that should be a per-user folder preferably under the TEMP folder (if one is set in the environment for the user). The installation folders, as you mention, should not be writable by ordinary users without a flip to admin privs so the Windows Installer subsystem can do the actual writing.
--
Devin L. Ganger, WA7DLG
Not a L&L employee; opinions are those of my cat
Winner "Best in Class", 2018 My First Supervillain Photo Shoot

Mi
MikeA29
Posts: 14
Joined: Fri Mar 01, 2019 7:55 pm
Platform: Mac + Windows
Amber...
Hmm, I had no problems compiling to LaTeX in an earlier build (which may have been more recent than b30), but now I am

Progress!

Amber...
Here’s a way of putting it: can you compile at all to LaTeX?

Yes. AFAICT so far, double curly braces throws Scrivener into a hairy compile fit,[1] and backticks for a code block force listings package output, both as already reported. Everything else seems okay, <drone+hint> and HTML comment marks works better than backticks.</drone+hint>

Amber...
I have no clue, that sentence doesn’t even make sense to me. This is what you got when compiling? Are you using an NAS for the output folder or anything of that nature?

Me too + me too + yes. Not using a NAS. A hangover from earlier development perhaps?

+ what @devinganger wrote!

[1] Think crazed Hell's Angel!

AmberV
Posts: 24216
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:
Okay, the all inclusive failure of every conversion method may be a symptom of the current internal build I'm using, that would explain why nobody else has reported it. I'll have to give it another go once we're past that specific misconfiguration. What might help for me is a simple test project that reproduces the braces issue---since I haven't been having luck thus far. There have been a few cases where we've found settings impacting things that shouldn't be impacted, and it may be something of that nature. I tend to test with dirt basic factory settings, only changing what is necessary to fit the bug report.

The puzzling thing is that, in theory, Scrivener should not be parsing a single byte! The only things it should be doing are:

1. Generating some baked in syntax where it makes sense to do so (embedded images, footnotes, etc.). This is blind output, and presumes you follow the rules around the spot it generates into.
2. Verbatim pass-thru on everything you type in, including all content you "program" the compiler to generate via its features.

So it really should not matter if your editor has this or that punctuation within it, or if you use fence quotes or HTML comments.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles