[RC20] Performance worse than ever

User avatar
jje
Posts: 372
Joined: Sun Jul 06, 2014 5:57 pm
Platform: Windows
Location: Sussex, UK

Wed Mar 10, 2021 12:01 pm Post

A couple of RCs ago, I thought I detected a marked improvement in loading text, especially when switching to Scrivenings view for a long chapter. However, RC19 seemed as bad as ever, and RC20 is worse still - there's now a very noticeable delay before you even see the "loading" message. Hope this can be improved in the final version!

pc
pcgeekesq
Posts: 60
Joined: Tue May 23, 2017 7:44 pm
Platform: Windows

Wed Mar 10, 2021 1:10 pm Post

For me, a 35,000 word section loaded as a scrivening quickly: about 2 seconds.
Displaying a whole book (151,0000 words) as a Scrivening took about 10 seconds.

RC20, Windows 10, Core i7-6700 (4 Ghz), 16MB DRAM, Samsung 950 Pro NVMe SSD, two Nvidia GTX1060s driving 1 - 4K monitor and 4 - 1440p monitors.

Your mileage may vary.

User avatar
tiho_d
Posts: 1548
Joined: Tue Sep 13, 2011 1:14 pm
Platform: Linux + Windows

Wed Mar 10, 2021 1:34 pm Post

@jje: RC19 and RC20 do not have any text layout changes. Around RC17/RC18 we found a bug which was indeed slowing down text layout so RC19 should be much faster then RC18 when using long documents. Try File > Save and rebuild indexes which might also improve startup times. The only thing I can imagine slowing you down between versions is adding lots of Inspector Notes and big images down the track in your project. Compiling and printing to PDF should be also faster in RC20.

Also check for any third party application on your computer stealing your CPU cycles. Whitelisting your project folder with your antivirus application might also speed things up significantly.

Gf
GfK
Posts: 43
Joined: Fri Oct 30, 2020 1:19 pm
Platform: Windows

Wed Mar 10, 2021 2:07 pm Post

Can't say I've noticed any difference (better or worse) in any recent version. But then, I built my PC when I was programming games so it's a tad overkill for writing.

Intel I9-9900K @ 3.6GHz | GeForce RTX2070 8GB graphics | 32GB RAM | 500GB NVMe M.2 SSD | 1TB HDD | Windows 10 x64.

Also, loading from a network-attached storage drive over 802.11ac Wifi.

User avatar
tiho_d
Posts: 1548
Joined: Tue Sep 13, 2011 1:14 pm
Platform: Linux + Windows

Wed Mar 10, 2021 2:19 pm Post

@jje: You might also start Scrivener via ScrivenerLog.bat to check for any unexpected warning messages upon loading Scrivenings mode. You might also see document loading times, so you will know which document is causing the biggest delays.

User avatar
jje
Posts: 372
Joined: Sun Jul 06, 2014 5:57 pm
Platform: Windows
Location: Sussex, UK

Wed Mar 10, 2021 4:48 pm Post

pcgeekesq wrote:For me, a 35,000 word section loaded as a scrivening quickly: about 2 seconds.
Displaying a whole book (151,0000 words) as a Scrivening took about 10 seconds.

RC20, Windows 10, Core i7-6700 (4 Ghz), 16MB DRAM, Samsung 950 Pro NVMe SSD, two Nvidia GTX1060s driving 1 - 4K monitor and 4 - 1440p monitors.

Your mileage may vary.


Interesting. My specs are almost identical to yours (a 3.6Ghz i7-6700, 16GB of RAM, my SSD is the SATA version of the Samsung 950 Pro, not NVMe, and the graphics card is a single RTX2070). Yet my 30,000 word chapter took six seconds to load -- five of them before the "loading" message even appeared. (And this was after rebuilding the search indexes, as recommended.) I will have to try some of the other suggestions now.

User avatar
jje
Posts: 372
Joined: Sun Jul 06, 2014 5:57 pm
Platform: Windows
Location: Sussex, UK

Wed Mar 10, 2021 4:57 pm Post

tiho_d wrote:@jje: You might also start Scrivener via ScrivenerLog.bat to check for any unexpected warning messages upon loading Scrivenings mode. You might also see document loading times, so you will know which document is causing the biggest delays.


Thanks. Just tried that. The only error was "Warning: libpng warning: iCCP: known incorrect sRGB profile" (lots of times), plus some fonts with "missing PostScript names" (?)

Document load times all seem super fast (0.012sec was the slowest).

I don't have any images in the project, nor PDF files, but I do use a lot of notes -- hundreds per chapter (I'm an academic writer), so perhaps that's what's causing the problem?

pc
pcgeekesq
Posts: 60
Joined: Tue May 23, 2017 7:44 pm
Platform: Windows

Wed Mar 10, 2021 5:05 pm Post

I should mention that I run Scrivener using the following command:

Code: Select all

C:\Windows\System32\cmd.exe /Q /D /c start "Scrivener" /affinity C /B "C:\Program Files\Scrivener\Scrivener.exe"


The start parameter "/affinity" limits which threads Scrivener can run on to two threads on a single core ("C" = 00001100" : the two SMT threads on the second core). In the past I noticed that when the OS migrated Scrivener from one core to another, performance suffered because of the need to move data from one core's cache to the others. This trick prevents that, and since Scrivener seems to run as a single instance, there's essentially no downside.

The /B start parameter runs Scrivener without opening a new Command prompt.

The cmd.exe parameters are routine: /Q cancels echo of the command, /D turns off autorun

You can watch the CPU graphs on the "CPU" tab of the Windows "Resource Monitor" system utility while you are converting to a Scrivening to see whether Scrivener is hopping between cores.
Last edited by pcgeekesq on Wed Mar 10, 2021 5:10 pm, edited 1 time in total.

sh
shaun
Posts: 169
Joined: Mon Mar 18, 2013 11:45 pm
Platform: Mac + Windows

Wed Mar 10, 2021 5:08 pm Post

jje wrote:
tiho_d wrote:@jje: You might also start Scrivener via ScrivenerLog.bat to check for any unexpected warning messages upon loading Scrivenings mode. You might also see document loading times, so you will know which document is causing the biggest delays.


Thanks. Just tried that. The only error was "Warning: libpng warning: iCCP: known incorrect sRGB profile" (lots of times), plus some fonts with "missing PostScript names" (?)

Document load times all seem super fast (0.012sec was the slowest).

I don't have any images in the project, nor PDF files, but I do use a lot of notes -- hundreds per chapter (I'm an academic writer), so perhaps that's what's causing the problem?


I've noticed this too. I also use a decent amount of notes (a few hundred per book usually) and the Windows version is substantially slower at loading than the Mac version, despite my Mac being a much older model with a slower CPU.

User avatar
narrsd
Posts: 809
Joined: Wed Sep 22, 2010 8:34 pm
Platform: Win + iOS

Wed Mar 10, 2021 9:59 pm Post

Jje, hmm on this one too -- maybe.

What caught my eye was '30,000 word chapter'. And then I did a double-take also.

32768 _characters_ has often been a boundary for the primitive level internally of a text editor - limitations of binary word size, etc. historically.

Even if that may (or not) be out of the picture, 30,000 _words_ is ten times that. You are speaking of length of about 1/3 of some novels...!

It's easy to wonder if such may be the root of the slowness, particularly according to all kinds of details possible of Scrivener's (or its internal QT's) rendering.

What I'm thinking is that such a huge chapter may not be accounted for in the strategy of optimization, at one of the posts along the line.

You could try breaking up that chapter into segments -- maybe 4 of them as a first try, and measure how your opening times react?

And/or, Tiho may be interested in that project and chapter, if it's something you can feel to use the private sending means to get to him.

pc
pcgeekesq
Posts: 60
Joined: Tue May 23, 2017 7:44 pm
Platform: Windows

Thu Mar 11, 2021 5:56 am Post

narrsd wrote:Jje, hmm on this one too -- maybe.

What caught my eye was '30,000 word chapter'. And then I did a double-take also.

32768 _characters_ has often been a boundary for the primitive level internally of a text editor - limitations of binary word size, etc. historically.

Even if that may (or not) be out of the picture, 30,000 _words_ is ten times that. You are speaking of length of about 1/3 of some novels...!

It's easy to wonder if such may be the root of the slowness, particularly according to all kinds of details possible of Scrivener's (or its internal QT's) rendering.

What I'm thinking is that such a huge chapter may not be accounted for in the strategy of optimization, at one of the posts along the line.

You could try breaking up that chapter into segments -- maybe 4 of them as a first try, and measure how your opening times react?

And/or, Tiho may be interested in that project and chapter, if it's something you can feel to use the private sending means to get to him.


I did an experiment to test this, merging 35 separate files (in 11 groups) into a single 39,000 word (222,000 character) file. It's slow to open (about 7 seconds). Viewing it as a Scrivining took about 13 seconds.
In contrast, viewing the original 35 separate files as a Scrivining took 5 seconds.

Looks like Scrivener is a little slow on large (200KB) single files.
Good thing I don't use them normally. :)

User avatar
jje
Posts: 372
Joined: Sun Jul 06, 2014 5:57 pm
Platform: Windows
Location: Sussex, UK

Thu Mar 11, 2021 8:51 am Post

narrsd wrote:Jje, hmm on this one too -- maybe.

What caught my eye was '30,000 word chapter'. And then I did a double-take also.


You and my editor both! I think this is something I need to tackle, rather than the developers. I was planning to wait for the final version of the software to create a brand new project. I thought I would move all the current chapters over and take the opportunity to reorganise the whole thing (including restructuring the chapters), but maybe I'll make a start on that now.

Thanks to everyone for their helpful suggestions.

User avatar
narrsd
Posts: 809
Joined: Wed Sep 22, 2010 8:34 pm
Platform: Win + iOS

Thu Mar 11, 2021 7:09 pm Post

jje wrote:You and my editor both! I think this is something I need to tackle, rather than the developers....


ha - I bet :)

if I understand completely how such a thing could grow, out of an initial structure, and then 'completing'.

What about using Scrivener's Split ability, to break down the individual sizes, get your snappiness back, while keeping your structure intact, at least for now?

I would think to make a 'holder' folder, put your giant chapter in it, then Split at appropriate points, then clean up the titles.

Afterwards depending on your convenience (or actually initially I think), you could change the holder to not be a folder, if it's helpful to write a little text or titling in it.

There might be some interplay with Compile settings, depending on how you're treating document names, getting titles, etc...

You can tell I've been getting clearer myself with what the document model of Scrivener actually is -- feels can be useful as well as interesting...

rw
rwfranz
Posts: 1026
Joined: Thu May 28, 2015 9:41 pm
Platform: Windows

Fri Mar 12, 2021 3:22 am Post

pcgeekesq wrote:
narrsd wrote:Jje, hmm on this one too -- maybe.

What caught my eye was '30,000 word chapter'. And then I did a double-take also.

32768 _characters_ has often been a boundary for the primitive level internally of a text editor - limitations of binary word size, etc. historically.

Even if that may (or not) be out of the picture, 30,000 _words_ is ten times that. You are speaking of length of about 1/3 of some novels...!

It's easy to wonder if such may be the root of the slowness, particularly according to all kinds of details possible of Scrivener's (or its internal QT's) rendering.

What I'm thinking is that such a huge chapter may not be accounted for in the strategy of optimization, at one of the posts along the line.

You could try breaking up that chapter into segments -- maybe 4 of them as a first try, and measure how your opening times react?

And/or, Tiho may be interested in that project and chapter, if it's something you can feel to use the private sending means to get to him.


I did an experiment to test this, merging 35 separate files (in 11 groups) into a single 39,000 word (222,000 character) file. It's slow to open (about 7 seconds). Viewing it as a Scrivining took about 13 seconds.
In contrast, viewing the original 35 separate files as a Scrivining took 5 seconds.

Looks like Scrivener is a little slow on large (200KB) single files.
Good thing I don't use them normally. :)

We've actually known this for a while. I did some testing way back early in the Beta, and Scrivener likes having smaller chapters. It's a little thing, but it makes a huge difference. I still have the test project sitting around, too. I started to see loading delays somewhere around 5k words per document, as I recall.
Just another user.

Ma
Mark Lamb
Posts: 7
Joined: Tue Feb 23, 2021 8:10 pm
Platform: Windows

Sat Mar 13, 2021 1:05 pm Post

Hi. I've just changed from RC19 to RC20 this morning, in the belief that we surely can't be far from the end. How wrong! I agree, RC20 is slower, much slower. Arguments of who's got what hardware are irrelevant given that you presumably work on the same machine between releases. I worked for a couple of hours this morning on RC19, then noticed RC20 was available. I 'upgraded' and noticed straight away that even simple tasks were taking much longer. I actually don't care about the speed though, since the difference between five and ten seconds isn't going to make me lose any sleep. What I do care about is that the development strategy must be a bit hit and miss if this kind of thing can happen at RC20 (FFS!) Do the developers have unit tests for this kind of thing? Do they have a QA suite? If they do then how come all these regressive failures make it through the tests? Do they have any concept of what it means to test prior to release (even if it's only a candidate)?